Efficient training of language models to fill in the middle
OpenAI introduced Fill-in-the-Middle (FIM) training, enabling language models to predict arbitrary spans of text rather than only prefixes, improving code completion and in-context reasoning.
Read at source: https://openai.com/index/efficient-training-of-language-models-to-fill-in-the-middle