Foundation Models Explainer
A foundation model is trained on broad data, usually at large scale, and can be adapted to a wide range of downstream tasks. Stanford researchers popularized the term to emphasize that one general model can become the base for many applications through prompting, retrieval, fine-tuning, or added tools.
Foundation models often use self-supervised learning, where the data itself supplies a training signal such as predicting a missing or next element. Scale and transfer let the model learn reusable representations before a developer has chosen every eventual task.
Language models, vision models, and multimodal models can all serve as foundations. Sharing a base model can reduce the data and cost needed for a new application and make advanced capabilities widely available. It also concentrates technical dependencies in a smaller number of upstream systems.
A defect in the foundation can be inherited by many downstream products. Bias, security weaknesses, opaque data practices, energy use, and unexpected capabilities may spread with the model. Evaluating an application therefore requires both the base model’s evidence and tests of the specific data, tools, users, and decisions layered on top.
A foundation model is trained on broad data at scale and can be adapted to many downstream tasks and applications.
Key terms include foundation model, pretraining, broad data, self-supervised learning, transfer learning, adaptation, downstream task, fine-tuning and prompting.
<a href="https://crfm.stanford.edu/report.html" target="_blank" rel="noopener noreferrer">Stanford CRFM: On the Opportunities and Risks of Foundation Models</a> and <a href="https://crfm.stanford.edu/2021/10/18/reflections.html" target="_blank" rel="noopener noreferrer">Stanford CRFM: Reflections on Foundation Models</a> provide the primary references for this explainer.
Broad pretraining produces reusable capabilities. Developers adapt the model through prompts, retrieval, fine-tuning, tools, and task-specific interfaces.
A foundation model can support language, vision, audio, robotics, or multiple modalities depending on its training and architecture.
Scale creates leverage: one improvement can benefit many products, while one defect can propagate through an ecosystem.
Assess training transparency, downstream context, security, bias, environmental cost, and who can challenge or correct failures.
Explore more "Explainers"
Discover additional explainers across politics, science, business, technology, and other fields. Each explainer breaks down a complex idea into clear, everyday language—helping you better understand how major concepts, systems, and debates shape the world around us.