Ken Goldberg

I'll be at Long Horizon in SF on Oct 20! Looking forward to talking shop

Almost all bimanual robots include wrist cameras. They add bulk, cost, and complexity, but are critical for performance. Are they really needed?

Excited to finally share EyeRobot 2.0, where we use active gaze from an ego stereo camera instead 🧵 (with @KushtimusPrime)

🎟️ CoRL 2026 registration fees have been updated.

Standard rate now runs Oct 13–31 (aligned with camera-ready). Register by Oct 12 for the best rate. See the full breakdown below.

🗓️ Nov 9–12 | Austin, TX 🤠

👉 corl.org/attending/regi…
#CoRL2026 #RobotLearning

🤖 Should robots be generalists or specialists?

At #IROS2026, @Ken_Goldberg & @andrea_bajcsy moderated one of the best debates in robotics.

Matt Mason opened with a better question:

“Should animals be humans?”

10 takeaways 🧵👇

1️⃣ @HarryXu12, for generalists:

“Don’t build Excel. Build GPT.”

We don’t know what we’ll need tomorrow, so buy generality just in case — and amortize it.

The twist: GPT didn’t replace Excel. It learned to drive it.

2️⃣ @GeorgiaChal split “general” into 4 axes:

Body. Scene. Contact. Task.

“Breadth and reliability are measured per axis. One does not imply the other.”

Her prediction: “The next task is the test.”

3️⃣ The most honest number of the day, from Georgia’s slides:

A general model calling robot tools.

Coarse block placement: 19/20 ✅
Precision insertion: 2/20 ❌

“Coarse pick and place works. Physical composition and precise contact not yet.”

Tool use ≠ dexterity.

4️⃣ @KaterinaFragiadaki’s architecture:

The generalist writes the curriculum.
The specialist does the reps.

Video → sim → RL on subgoals → specialist policies.

No teleop.

At deployment:

“Fast specialists for the common case. Flexible generalists for the long tail.”

5️⃣ Specialist bench:

“Fixture and conveyor are part of the intelligence.”

A fixture is a prior.
A conveyor is a scheduler.

Strip them to look “general” and you’ve moved work onto a pricier learned controller.

Optionality that hurts reliability isn’t intelligence. It’s cost.

6️⃣ A surgical robot doing one procedure 10,000 times learns exactly what can go wrong.

“Generality is the enemy of certification.”

From the floor:

Are human drivers generalists or specialists?

We certify generalists one task at a time.

That’s what a license is.

7️⃣ Toshio Fukuda changed the register:

“NON-AI matters.”

Kizuki: help offered before it’s asked for.

“Awareness comes from the body, not from the model.”

Japan: ~29% aged 65+, ~570k care workers short by 2040.

“If people live to 120, must our machines last that long?”

8️⃣ The floor fight:

“A human is a generalist.”

“We’re all human specialists.”

The room accidentally re-derived pretrain-then-finetune.

Veterans argued from what has shipped.
Newcomers argued from what has scaled.

Both are data.

9️⃣ The title asked for a side.

The slides drew a stack:

General planning + recovery on top.
Fast, certifiable specialists below.

A verifier is a fixture made of code.

🔟 The real question:

How thick can the middle get before the body must specialize anyway?

Generalist or specialist?

Maybe the winning architecture is neither.

It’s a stack.

#IROS2026 #Robotics #PhysicalAI #RobotLearning #WorldModels

@EricKaiyuan and @Ken_Goldberg 's Robo Sim (Studio) presentation of their recent Sim2Real2Sim results with Astra+Newton Physics were next-level, Thanks!
youtube.com/shorts/volIzqA…

Was thrilled to run into Berkeley Professor @Ken_Goldberg at the Cal Football game (😭), thanks to an invite from @jfiance and @thehousefund . Ken brilliantly launched the Jester project in 1999 which was a pioneer in the concept of collaborative filtering, essentially it was Funny or Not. We obviously took great inspiration from it, the original goal of hot or not was to create a collaborative filtering dataset for dating. It's just a reminder to me of how important the influence of one's environment can be, to learn from and to build on. Berkeley was always a place where the kooky fun ideas could blossom, I hope it still is! Thank you Ken!!

goldberg.berkeley.edu/jester-data/

Can robots help in data centers? It’s tricky when all the cables are the same color.

Introducing TRACE: bi-directional tracing + interactive perception for monochrome cables, @ #IROS2026 from @AUTOLab_Cal @UCBerkeley (1/8)

Carnegie Mellon's robotics expertise will be on global display next week as thousands of researchers and industry leaders convene in Pittsburgh for #IROS2026!

See you there! 🤖 🤝

bit.ly/3V9mSSW

Jev this, jev that, jeva tried it with graph as policy?

Spent the weekend playing with GaP. In YAM moving-cube sim, here's the median pickup time
- 5.42s with GaP + custom Rust executor + Jev
- 8.79s with native GaP + Astra

GaP gives you explicit decision nodes, dependencies and parallel execution. Jev makes decisions inside that graph much faster

graph-robots.github.io/graph-as-polic…

3D/4D reconstruction is one of the great achievements of computer vision. With roots in photogrammetry, multi-view geometry was much studied (e.g. Hartley & Zisserman). In the last decade, it was successfully combined with deep learning. Shape priors which come from recognition got incorporated (e.g. for humans, in HMR) and recent systems like VGGT and SAM 3D can do amazing stuff. This progress is directly relevant for robotics, which (duh) operates in the 3D world. Many robotics papers in the learning era weren't exploiting 3D structure, which IMHO is just wasting valuable signal. But there has been a strong real2sim2real tradition for years e.g. here are two projects from my group (and there are many others, I am not claiming exclusivity). I am glad that people have rediscovered this capability with Astra. But please do give credit to human researchers, not just AI models. zhec.github.io/rhoi/ ,
videomimic.net

Love my friends at figure, but failing half the time is not “doing real useful work.” See the 237/420 success rate below taken from the blog.

Doing useful work = generalization + reliability.