FastCompany
How Misty Plans To Build The Most Personable, Programmable Robot Ever
By: Daniel Terdiman
Last fall at a hackathon inside a featureless, low-slung warehouse in north-central Boulder, Colorado, one of the first outside software developers to get his hands on a brand-new robot had an idea: If he ordered a pizza, he could teach the robot to get more and more excited as the pie got closer to arriving.
Using GrubHub, which charts the progress of a delivery driver on a map, the developer wrote the necessary code to let the bot keep tabs on the pizza’s location. “The robot started getting really excited and moving around quickly, and when the food was right there, it just [got] really excited and [spun] around,” remembers Ian Bernstein, the founder of Misty Robotics, the startup that created the robotics platform that made this all possible. “It’s quite simple, but the robot really did feel excited, and that kind of made you excited.”
Programmable personal robots are nothing new–think of Lego’s Mindstorms line and TurtleBots. But Misty’s bots stand out for their power and ambition. They run on two powerful smartphone processors; offer autonomous navigation abilities, include a built-in microphone and speaker for use with things like Amazon’s Alexa; sport a 4K camera for eyes; and can be programmed in JavaScript.
Most importantly, the beta Misty I and forthcoming production Misty II have personality that is belied by their antiseptic-sounding names. Though they stand only 14.5 inches tall, they have heads that tilt up and down so that they seem to look right at you. A 4.3-inch LCD screen lets them communicate a range of expressions, whether they’re happy, interested, or horrified.
It shouldn’t be too surprising that these bots have such emotional range. Misty was spun out from Sphero, the company behind the famous Star Wars BB-8 robot, a $100 app-controlled rolling droid that perfectly captured the whimsical, frenetic nature of its cinematic counterpart in Star Wars: The Force Awakens. Bernstein, a Sphero cofounder, left the company early last year to start Misty after incubating the platform in what amounted to a Sphero skunkworks. Sphero took an equity stake in the new startup.
For Bernstein, getting out of Sphero in 2017 may well have been a smart move. The company, despite selling a total of 3 million robots and being valued at $190 million according to Pitchbook, recently laid off 45 workers and announced it was shifting its focus from toys to education. Misty insists that it’s been unaffected by Sphero’s change of course.
But along with Bernstein and six other former Sphero employees, Misty was also founded with a significant part of Sphero’s DNA–the idea that giving robots personality is a terrific way to create bonds with their owners. Which, by extension, might be a terrific way to sell a lot of robots.
Bernstein recalls that when he brought Sphero’s first robots to the Toy Fair show in New York, he ended up giving his personal Sphero to a kid who had spent hours hanging out at the then fledgling company’s booth. On his way home to Colorado, he felt pangs of regret, even though he knew he could just grab another one when he was back at the office.
“On that flight back, I thought, ‘How do we really create a connection between Sphero and the user?,'” he says. “And we came up with this whole experience where you can make Sphero better . . . we launched that with Sphero 2.0, and we just saw our engagement from the product go way up.”
By the time Sphero released BB-8 in 2015, that sense of connecting with a robot was in full effect. As Bernstein recounts, “People have come up to me and said, ‘I went home the other night, and my wife was cooking in the kitchen, and BB-8 was just on patrol mode, and I was like, you’re not even paying attention to it, why is it even on? And she was like,’We’re just hanging out.'”
Sphero’s robots are $100 toys. Mistys are something else altogether: programmable, multipurpose, extensible devices that, at $1,500, cost about the same as a good PC. That’s why Misty is busy creating an “Infinitely Expressive Personality Engine” that will give each of its robots a unique persona that adapts to its owner. Those personalities will manifest in many ways–sounds the robot might make, or shyness around loud noises, or facial expressions to match its mood. The software will combine a number of static digital expressions so that the robots can convey almost countless emotions and reflect what’s going on around them. No two Misty robots will behave exactly the same.
“If you accidentally take the wrong Sphero home, it’s no big deal,” says Dan Grollman, a former Sphero robot brain architect who now leads brain engineering for Misty. “But if you take the wrong dog home, you’re going to want your dog back.” The company hopes you’d feel the same way about your Misty.
THE ROAD TO ROSIE
Although it’s only a 30-person startup, Misty’s pedigree as a spinoff of Sphero has motivated the company to develop and release a 10-year plan for a new robotics ecosystem. By year eight, it hopes, it will have engineered something like Rosie from The Jetsons–an affordable robot with hands and skills covering most human needs.
“We believe there’s a sequence in how scary technologies like this are adopted,” Misty CEO Tim Enwall says. “The approach is to be clear and methodical about creating that invention.”
After developing its first bots in secrecy, the company finally showed the first beta Misty I units to a select group of journalists at CES in Las Vegas last month. Those who got demos weren’t blown away, and when I got my own preview in Boulder in December, the bots showed off only a modicum of skills. But the point isn’t what Misty could do after its in-house engineers had been building those skills for a few months. It’s what could be possible in the coming months and years.
The company isn’t catering to serious roboticists, at least not yet. That, its founders say, would be overkill, like asking an IBM mainframe expert to apply her skills to an Apple II computer in the late 1970s. Instead, they want to inspire the type of general-purpose programmer who might build the robotic equivalent of VisiCalc, the spreadsheet that originally did more than anything else to make the Apple II computer obviously useful. In short, Misty wants to build the Apple II of robotics platforms, and then see where people take it.
But first, it has to get some robots in people’s hands. At CES, the company announced that its hand-made beta Misty I robots will be available starting this month. Devoted programmers who plunk down $1,500 will get the opportunity to start hammering away at Misty’s platform. What they learn will help the company get ready for the launch later this year of the Misty II, the company’s first full-blown, mass-produced, consumer-ready robot.
HOOKED ON ROBOTICS
Ian Bernstein is a robotics wunderkind. When he was 12, his classical guitarist father agreed to give music lessons to a guy who, in trade, taught Bernstein electronics. Soon he was spending 17 hours a day tinkering with machines and fixing neighbors’ broken VCRs. After some time at New Mexico Tech trying to learn how to start a company, he dropped out, figuring that the best way to learn about business was to start one.
He got into the TechStars accelerator, and in 2010, along with Adam Wilson, launched Sphero. Originally known as Orbotix, it was one of the first companies to build products you could control with a smartphone. The startup was already on a roll when it was chosen to be part of the Disney Accelerator in 2014. There, Bernstein met Disney CEO Bob Iger, who pulled out his phone and showed Bernstein some photos from the set of Star Wars: Episode VII, including a robot character called BB-8. Could Sphero make a toy robot version?
Sphero went into hyperdrive at that point, as Disney’s movie became a phenomenon and the in-home BB-8 came along for the ride. But even as the company racked up sales and market share, Bernstein found himself consumed with idea of advancing the craft of robotics. He set up a stealth lab inside the company, and gathered a small team of six to begin developing an autonomous robotics platform.
Soon, Bernstein was ready to spin off a new company, and needed a CEO. He approarched Enwall, a serial entrepreneur and investor who, after selling his previous company, Revolv, to Google-owned Nest, was evaluating “all kinds of ideas to see what grabs hold of my heart.” After talking with Bernstein, he signed on quickly.
“We’d always start our meetings with investors with a question,” says the 34-year-old Bernstein, who seems to always be wearing a black Burton snowboarding cap and a black hoodie. “Will robots be in every home and office?” adds Enwall, finishing Bernstein’s sentence.
Venture capitalists quickly bought into Misty’s vision. “It was the fastest fundraising I’ve ever gone through,” says Enwall. By June, the company scored $11.5 million in Series A funding from Venrock and Foundry Group.
BOTS IN TRAINING
Personality is just one trait of Misty robots. At the outset, they can do things such as upload and display pictures, upload and play audio, map rooms, respond to light, and recognize individuals. They can also be used as a remote control for a TV, to manage smart home devices, to tell you if your oven’s on, and more.
The robots are also designed so that owners can easily create scripts for small actions–like, say, making the robot freak out and play a sound if there’s too much bright light–using Google’s Blockly library. Blockly currently offers about 20 functions, with many more available using Misty’s API.
Over time, the whole idea behind Misty robots is that their feature set will be dependent on the outside programmers who create skills. The company even hopes that owners will eventually be able to sell the skills they create via some sort of app store.
And someday soon, robots like the Misty II will be able to be fitted with external accessories such as 3D printed arms, a backpack Arduino system, or even a trailer that can carry things. As other people build on top of the platform that Misty has created, the sky’s the limit, much as it was with Apple’s early computers.
The very first outsiders ever to get to play with Misty’s robots were the 25 programmers whom the company invited to a “Robothon” at its Boulder headquarters late last year. Over the course of a Saturday, the visitors played with Misty’s APIs and attempted to get their robots to perform various tasks. Within 15 minutes, someone had programmed a cat face onto a robot’s screen. One team got its robot to dance. And then there was the excited-by-pizza-proximity project.
In a video of the event, one of of the guest robot programmers says, “It’s cool that finally there’s a [robotic] connection to the real world.” That was music to Enwall’s ears. “That right there,” he says of the moment in the video, “that’s the audience we’re going after. And the sentiment, the reaction.”
Misty’s leaders walked away from the event with a sense that they couldn’t get their robots into customers’ hands fast enough. But the company faces hurdles that have already disrupted its best intentions. Thanks to the limitations of physics and electronics, its original plans to get robots to developers and announce its consumer version by CES became untenable.
“Hardware’s hard,” admits Bernstein, alluding to some of the realities that set in as the company pursued its original timeline.
One of the primary problems has been the tank-like treads that Misty robots drive around on. From the beginning, the company knew that general-purpose robots for the home had to be able to work their way around—and sometimes over—all manner of objects, including electrical cords, pets, and socks. Furthermore, they would have to make their way across various terrain: hardwood floors, carpeting, linoleum, and more.
In dealing with all those scenarios, “It turns out turning and going in straight lines are different physics problems,” Enwall says. Sure enough, even two weeks before CES, Misty I prototypes were having problems working their way from hard surfaces onto carpeting without their tread coming loose.
To Bernstein, the problems are reminiscent of an issue that once vexed Sphero for weeks. Nearly two years after launching—after the company had already shipped hundreds of thousands of units—Sphero robots in its Chinese factory began failing after about an hour of testing. No one knew why. It turned out that the person who had originally created the recipe for the robots’ rubber tires had died, and someone had tweaked the formula in a way that caused problems. “Like, he was the only one who knew the recipe for Coke,” Enwall says, “and he passed away.”
After much experimentation, Sphero figured out that if it boiled the tires for eight hours—which it did using two crockpots—the robots would work just fine. “We made another couple hundred thousand units with boiled tires,” Bernstein says.
The lesson here is that Misty will have to come up with a creative solution to the tread problem. When Misty I ships to the initial developers, the company is going to have to tell them they can only operate the robot on hard surfaces. But the problem must be solved by the time Misty II ships later this year.
“We’re trying lots of materials,” Bernstein says, “trying different tread designs. We’re definitely making progress (and) it’s not going to be one solution. It will be 10 different things.”
Another unanticipated problem is that the company chose a screen that uses RGB color for its robots, but also selected a processor that doesn’t output RGB. So a translation is required.
Similarly, the robot needs to be able to map its environment and recognize objects and faces. But its processor can’t do both mapping and recognition simultaneously. Enwall says that the company is still in the process of overcoming that limitation. “But we’ve gone from never the twain shall meet to some of both . . . Neither is at [the required] quality. But we know we’ll solve it.”
These problems forced the company to delay the shipping of the Misty I from November to this month, and to hold off on formally announcing the Misty II altogether.
“We don’t want to show Misty II until we’ve solved all these technical risks,” says Enwall. “This is why [we weren’t] publicly on a show floor [at CES] with Misty I, let alone Misty II.”
PROGRAMMERS AT WORK
Already, Misty has given free beta versions of the Misty I to four initial developers who match the company’s idea of a perfect customer. Cameron Henneke, a Boulder software engineer and the founder of GQueues, an online collaborative task-manager system, got unit number one just before Christmas. “I have a ton of experience using APIs to build on top of other platforms,” Henneke says. But “I have no experience with hardware, so that’s what I found enticing about Misty. I could write code that causes something to happen in the world, not just on computer screens or phones.”
Henneke adds that he appreciates that he can get his new robot to do things without having to understand “the hairy details of robotics.” In the first two weeks he had his beta Misty I, he tried to get the robot to roam around the house, and, when it saw him, to say “Hi, Cameron,” and flash its eyes. When I spoke to him, he’d gotten the robot to perform each task separately, and he was hopeful that he’d soon successfully combine the two actions.
One might think that struggling to get the robot to do even basic things like that would be frustrating, but to Henneke, it’s exactly the opposite.
“I’m completely happy with it,” he says. “It’s so exciting. I can write 10 lines of JavaScript and make my robot do something–go around the room, change its lights, flash its eyes, play an audio clip. I think it’s a ton of fun, and I’m so excited to get early access. I feel like we’re on the cusp of personal robots hitting the mainstream. To be involved at this early stage is super exciting.”
Henneke is so committed to the idea of programming robots that he’s committed to paying full price for a Misty I when it ships, and doing so again for a Misty II later in the year.
For Misty, the goal now is to solve its physics and electronics issues, and incorporate as much feedback as possible from Misty I users in the coming months as it prepares to launch the consumer version in the months to come. And to Enwall, a ship date in 2018 is a line in the sand. He says that both he and Bernstein are confident they can make it. “But we reserve the right to find things we have to boil.”
