AI & Robotics ยท San Francisco
I started in physics, participating in IPhO, and working through the classical theoretical curriculum. Later I got excited about biology and the nature of intelligence, publishing in computational neuroscience, and joining DeepMind. My day-to-day work focuses on robotics, the final link between intelligence and the physical world, and the ultimate full-stack engineering challenge. Currently I'm in SF building a startup at the intersection of robotics and science.
The video shows a quadruped robot commanded to "push the trolley towards the giraffe". The controller is Gemini fine-tuned on procedurally generated simulation data. However, there were no giraffes in the training simulations! This out-of-distribution generalization comes purely from Gemini pretraining.
This 3x slo-mo video shows an OP3 mini-humanoid positioning itself before kicking a ball. It demonstrates that using modern reinforcement learning and sim2real techniques, we can design very agile controllers even for low-cost hardware.
Accurate visuals have been a bottleneck for humanoid whole-body control. Here we combined learned visuals, such as the one shown in the video, with classic simulators to train whole-body visuomotor controllers for tiny OP3 humanoids using reinforcement learning.