Better language models and their implications
OpenAI releases GPT-2, a 1.5B parameter unsupervised language model achieving SOTA on multiple benchmarks including reading comprehension, translation, and summarization.
Excerpt
We’ve trained a large-scale unsupervised language model which generates coherent paragraphs of text, achieves state-of-the-art performance on many language modeling benchmarks, and performs rudimentary reading comprehension, machine translation, question answering, and summarization—all without task-specific training.
Read at source: https://openai.com/index/better-language-models