Learning Source Structure: Bayesian Prediction with Transformers
The paper uses an “informed Bayes” predictor as a falsifiable yardstick for what transformers learn from shared source structure.
Feder and Urbanke test transformers on Markov sources where evidence from one context can change predictions in another. They report that informed Bayes predicts several trained-transformer behaviors, including out-of-family loss, doubled extra loss when two components change, first-token context benefits, and when copying becomes useful. Small transformers match the benchmark when structure is visible; hidden structure takes more layers and training. ArXiv · AI/CL/LG's note
Feder and Urbanke test transformers on Markov sources where evidence from one context can change predictions in another. They report that informed Bayes predicts several trained-transformer behaviors, including out-of-family loss, doubled extra loss when two components change, first-token context benefits, and when copying becomes useful. Small transformers match the benchmark when structure is visible; hidden structure takes more layers and training. ArXiv · AI/CL/LG's note
score 4