Learning Robot Behavior Priors from Non-Robotic Synthetic Data

The paper tests whether a Transformer can learn useful robot-action priors from procedurally generated data based on mathematical abstractions of control, without seeing real robot behavior during pretraining. The model achieves non-trivial zero-shot action prediction on unseen physical robot tasks through in-context learning; specialized control priors outperform generic causal priors. A compute-matched model trained on real robot data performs better only on the datasets it saw during training.