COBALT lets – Georgia Tech researchers have built COBALT, a smartphone-based system that lets people with little or no computing experience remotely control robot arms using only a phone and an internet connection. The goal is not just convenience—it’s to gather the real-wo
On a phone screen, the controls look almost too simple for anything that could move a robot arm. But move your device, and the arm inside Georgia Tech’s lab follows in real time.
That’s the promise behind COBALT. a new smartphone-based system developed by researchers at Georgia Tech’s People. AI & Robotics (PAIR) Lab. It’s built to let users with little to no computing experience remotely control robot arms from virtually anywhere in the world using just a phone and an internet connection. The team’s design shifts robot operation from specialized equipment and technical setup to something that feels closer to a mobile game than industrial machinery.
Users guide the robot by moving their phones in different directions. The robot mirrors those movements on the spot. while basic tasks—grabbing. moving. and releasing objects—are handled through simple on-screen controls. In testing. participants from countries including India. Indonesia. and Pakistan remotely controlled robot arms located inside Georgia Tech’s lab even though they had no prior robotics experience.
Ayush Agarwal, a Ph.D. student in Georgia Tech’s School of Interactive Computing who leads the COBALT research team. said the system was intentionally designed to make robotics accessible to beginners rather than experts. The work is also rooted in a bigger problem robotics teams keep running into: collecting enough real-world training data to improve AI-powered robotic systems.
Modern robots, the researchers say, need enormous amounts of policy training data to perform physical tasks reliably. Simulation alone, directed by Assistant Professor Animesh Garg, who directs the PAIR Lab, is not enough for large-scale deployment. Instead. the team envisions a crowdsourced network where millions of smartphone users passively contribute operational data by remotely interacting with robots.
Garg compared the idea to tapping into the nearly five billion smartphone users worldwide. If that barrier to entry drops, researchers believe the platform could become a scalable global system that accelerates robotic learning and automation.
The education angle is already tangible. Georgia Tech researchers recently demonstrated COBALT to students from Midtown High School in Atlanta. letting them remotely operate robot arms using smartphones. The team says the interface’s simplicity could make robotics education more accessible in classrooms without expensive equipment or specialized hardware.
There’s also a potential labor rethinking in the background. Garg described the possibility of a robot-powered gig economy. where people remotely operate assistive robots in homes. warehouses. or factories from anywhere in the world. In that model. a factory robot could handle most tasks autonomously and only request human help when it hits a difficult situation—allowing a remote operator to briefly take control through a phone before handing the operation back to the AI system.
Agarwal said user studies showed smartphones were preferred over VR headsets. keyboards. or traditional controllers because they felt more intuitive while still providing high-quality control data. The team also says the system minimizes latency by using WebRTC technology—similar to platforms like Zoom and Google Meet—so robot movements and live video streams remain responsive even across long distances.
COBALT is now being prepared for a major stage: the research paper on COBALT is being presented this week at the IEEE International Conference on Robotics and Automation in Vienna. Along with showcasing the technology itself, the team is also presenting a large-scale remote operation network built around the platform.
COBALT Georgia Tech PAIR Lab robotics smartphone control robot arms remote operation WebRTC crowdsourcing AI training data IEEE International Conference on Robotics and Automation Vienna education gig economy for robots