Encord, a company based in San Leandro, California, is pioneering new ways to generate high-quality training data for physical AI, particularly in robotics. One notable experiment involves using a brain wave-reading headset developed by the German startup Zander Labs to capture neural signals from operators as they perform tasks like disassembling a Jenga tower. This brain activity data aims to enhance robotic learning by providing real-time insight into mental states such as intent and error detection, potentially improving model performance. Encord is currently running trials to assess whether integrating brain wave data with traditional training datasets can yield measurable advancements before expanding this approach.

Physical AI development faces a bottleneck in data availability, as realistic, detailed training sets are scarce and expensive to produce compared to the abundant text data used for generative large language models. Encord’s head of robot learning, Vineeth Velmurugan, emphasizes that successful robotic manipulation models require datasets several times larger than popular video platforms like YouTube, making data generation a critical business challenge. To address this, Encord not only collects egocentric video data from workers wearing cameras but also experiments with multiple modalities, including brain wave readings and muscle signal sensors, to capture more precise information about human movements and intentions during complex tasks.

Within Encord’s facility, pilots like Sofia Infante and Andrew Ceja help create annotated datasets by physically operating robots to mimic human actions on various objects, such as pouring coffee or connecting cables. These datasets come with detailed annotations describing specific movements, which Velmurugan notes are significantly more valuable for training than raw egocentric footage, despite being costlier to produce. Encord’s unique position, working across numerous robotics firms, enables the company to identify emerging data collection techniques and trends that could shape the future of physical AI development, offering clients insights and solutions that leverage these advanced datasets.

The collaboration with Zander Labs marks a cutting-edge effort to integrate neuroscience into robotic training, addressing limitations in current data collection methods and aiming to enhance the precision and adaptability of AI physical behaviors. While this approach is still in early trials, it exemplifies the evolving strategy among robotics startups to move beyond simple video data and toward richer, multimodal datasets that combine visual, physical, and neural signals. This shift reflects the growing realization that building effective physical AI models requires manufacturing data with greater complexity and fidelity, setting the stage for potentially transformative advancements in robotic capabilities.