Carnegie Mellon Develops SPOT for Human-Like Robot Planning

Researchers at Carnegie Mellon University’s Robotics Institute have introduced a groundbreaking system designed to enhance robotic capabilities in complex environments. Named Search over Point cloud Object Transformations (SPOT), this innovative approach allows robots to interpret their surroundings and execute tasks typically performed by humans, such as organizing objects and managing clutter.

SPOT addresses one of the most significant challenges in robotics: effective operation in unpredictable settings like kitchens, classrooms, and offices. This capability is crucial for advancing the usability of robots in everyday life. The research team, guided by Associate Professor David Held and Professor Maxim Likhachev, focused on enabling robots to coordinate multi-object movements. This includes tasks like putting away dishes or arranging items on a shelf.

The system employs 3D camera data to help robots understand spatial relationships and object shapes, allowing them to engage in intuitive, goal-oriented planning. Instead of relying on predefined symbolic descriptions of the environment, SPOT allows robots to analyze raw sensory input, providing them with a detailed view of the scene. As Amber Li, a Ph.D. student and co-lead researcher, noted, “SPOT operates directly in the point cloud space with raw sensory input from one camera and needs no additional information about the scene or the objects.”

This ability to perceive the world in three dimensions enables robots to make informed decisions about which objects to move and in what sequence. For instance, before stacking a plate, SPOT can identify and remove any bowls or cups that may obstruct the process. Such decision-making is akin to how humans intuitively organize their spaces without following strict rules.

In practical tests, the team utilized a Franka robotic arm and a set of plastic dishes to evaluate SPOT’s effectiveness. The robotic arm successfully rearranged the dishes into various configurations and determined the optimal order for moving objects. Notably, SPOT outperformed traditional planning methods, demonstrating its potential for practical applications.

Co-lead researcher Kallol Saha, a master’s student in robotics, emphasized the importance of intuitive decision-making in the planning process. “When humans organize our homes, we don’t have a set of rules in our minds that we follow before rearranging objects. We just look, plan, then act. SPOT brings that kind of intuitive decision-making to robots, allowing them to plan complex movements directly from what they see.”

The research has gained recognition and was accepted for presentation at the 2025 Conference on Robotic Learning in Seoul, South Korea. The project received funding from the Toyota Research Institute and the Office of Naval Research, underscoring its significance in the field of robotics. To learn more about SPOT, interested parties can visit the project website.

The development of SPOT marks a significant step in making robots more capable of assisting with everyday tasks, potentially transforming how humans interact with technology in their daily lives.