Condensed
Knowledge the robot carries: compact, structured 3D object models. A whole library of everyday objects fits in megabytes, not gigabytes. No data center on a tether.
Physical AI · Sensorimotor learning
Intelligence that
learns by moving.
Saccadero builds sensorimotor AI for robots. It explores an object the way you do (a glance, a touch, a test) and builds a compact 3D model it can recognize, locate and act on, in real time, on the robot.
The idea
Most robot models only learn by watching. Ours also learns by doing, and needs a fraction of the experience.
Approach
Every movement is an experiment.
Saccadero doesn't just classify pixels. It acts to find out. Each glance or touch is chosen to settle a question, and the answer updates a structured 3D model of the object in its own frame of reference.
- 01
Move
Jump the sensor to the most informative surface: the part it hasn't seen yet.
- 02
Predict
Every live hypothesis says what the sensor should find when it lands.
- 03
Sense
Look and touch. Even empty space is evidence: it rules out every guess that put a surface there.
- 04
Update
Wrong guesses are refuted, the object model sharpens, and the next move is chosen.
Architecture
Big models teach. Small models act.
Large language and vision models do their work offline, writing compact knowledge about each kind of object: what it is, which way is up, where to hold it. Every claim is checked in simulation before a robot may use it. On the robot, a fast loop runs on what it carries. No giant model in the control loop.
- 01
Glance
1 frameOne camera frame proposes a short list of object-and-pose guesses.
- 02
Check
~5 msEach guess is scored against the stored 3D model, across hundreds of surface patches at once.
- 03
Decide
settled?Act when every plausible guess calls for the same move. Otherwise, look again.
- 04
Act & learn
self-checkingExecute, compare the outcome with the prediction, and keep what worked.
Stop sensing when the action is settled, not when the label is.
Deep look
When a glance can't settle it, the robot falls back to an active sensorimotor scan. That slow path is also how it learns new objects and teaches the fast one.
Orchestrator
A language model turns an instruction into goals, once. It is asked again only when the robot is genuinely stuck, and never sits inside the control loop.
~130 msfrom camera frame to decision, fast even before GPU acceleration
97%right object picked from a single camera frame
On-robotbuilt to run on NVIDIA Jetson-class hardware
Early development measurements in simulation, not yet confirmed on held-out tests.
Principles
Small, fast, and grounded.
Action-native
Skills live in each object's own frame of reference, so where to look, push or grasp follows directly from what the robot knows.
Fast at the edge
Built for real-time inference on embedded boards like NVIDIA Jetson. Big models stay offline; the robot keeps the reflexes.
Evidence first
Headline results are pre-registered, frozen and run on held-out data. Negative results stay in the log.
Early results
Less experience. More understanding.
4.8×
pose accuracy over the baseline sensorimotor system on held-out tests
2.5×
accuracy when learning each object from a single viewing, on a standard benchmark
<20MB
memory for a whole library of everyday objects, from mugs to power drills. Megabytes, not gigabytes.
Simulated experiments (MuJoCo) with pre-registered protocols and held-out test sets. Baseline: an open-source sensorimotor learning system.
Roadmap
From knowing to doing.
- Shown in simulation
Recognize
What is it?
- Shown in simulation
Locate
Exactly how is it posed?
- Shown in simulation
Push
Move it on purpose.
- In development
Grasp
Pick it up, first try.
- Next
Deploy
On real robots, on Jetson-class hardware.
Give robots a sense
of things.
We're looking for partners who want to push the boundaries of frontier robotics with us.