I Witnessed the Next Generation of AI in a Robot Capable of Instant Learning

Recently, I took a brief 15-minute trip from my home to witness some astonishing robotic feats.

I visited a startup named Generalist AI in Cambridge, Massachusetts, where I observed robotic arms tackling simple tasks such as stacking cups and placing blocks into bowls. I was amazed by their speed in problem-solving—it felt much like watching a human.

The robots learned a variety of tasks after watching a brief instructional video, impressively doing so without targeted training for each task. One particularly notable instance involved a robot tasked with sweeping a block into a bowl using a dustpan and brush. When the brush was taken away, the robot improvised by using the dustpan as a brush and flicked the block into the bowl.

Courtesy of Generalist AI

In another instance, a two-armed robot observed a video of someone unzipping a purse and taking out banknotes. I watched in awe as the robot unzipped a different purse and meticulously retrieved the notes. Remarkably, when it struggled to grab the cash, it switched from its right gripper to the left to find a better angle. “Ha,” remarked an engineer nearby. “It has never done that before.”

“This embodies the excitement people felt about GPT-3,” said Pete Florence, cofounder and CEO of Generalist, referring to OpenAI’s revolutionary language model launched in 2020. “You could prompt it for a new task, and it would genuinely attempt to accomplish it.”

Generalist seems focused on imparting a grasp of the world’s physics to its robots, taking inspiration from the natural understanding of physics that humans develop early on. This could significantly aid the models in applying what they learn from one situation to another. Indeed, some demonstrations reminded me of how children explore and improvise when presented with a task. Researchers have often been taken aback by the robots’ choices—one decided to sweep items with a banana placed in front of it, for instance. While it might seem trivial, developing physical intelligence is an area where machines still have much to learn, and the way infants absorb knowledge about their surroundings could provide valuable insights for AI developers.