Karl Cobbe and colleagues, 2021
To increase performance, we propose training verifiers to judge the correctness of model completions.
Don't Just Read it – Explore It
To increase performance, we propose training verifiers to judge the correctness of model completions.
We propose a new decoding strategy, self-consistency, to replace the naive greedy decoding used in chain-of-thought prompting.
These generative models have an unusual combination of predictable loss on a broad training distribution, and unpredictable specific capabilities, inputs, and outputs.
We introduce Self-Instruct, a framework for improving the instruction-following capabilities of pretrained language models by bootstrapping off their own generations.
We investigate the optimal model size and number of tokens for training a transformer language model under a given compute budget.
To address this gap, we introduce AlphaCode, a system for code generation that can create novel solutions to these problems that require deeper reasoning.
Taken together, these results strongly suggest that almost all knowledge in large language models is learned during pretraining.
We introduce Codex, a GPT language model fine-tuned on publicly available code from GitHub, and study its Python code-writing capabilities.
This paper presents a new set of foundation models, called Llama 3.
We release Code Llama, a family of large language models for code based on Llama 2.
This work introduces Gemma, a family of lightweight, state-of-the-art open models built from the research and technology used to create Gemini models.
We present DeepSeek-V2, a strong Mixture-of-Experts language model characterized by economical training and efficient inference.