Last updated 08-11-2026
Category:
Reviews:
Join thousands of AI enthusiasts in the World of AI!
Gopher
Gopher is a 280-billion-parameter transformer language model Google DeepMind announced in December 2021. DeepMind trained a family of models from 44 million to 280 billion parameters to study how scale affects text prediction, reading comprehension, fact-checking, and toxic-language detection.
Compared with general-purpose chatbots, Gopher was a research release, not a public app. DeepMind paired the model paper with an ethics taxonomy covering 21 risks across six themes and a separate Retrieval-Enhanced Transformer (RETRO) architecture that pulls passages from an internet-scale index to cut training cost and trace outputs back to sources.
The blog post targets AI researchers studying scaling laws, safety taxonomies, and retrieval-augmented language models. Gopher beat prior models on several Massive Multitask Language Understanding (MMLU) categories but still struggled with logical reasoning, common-sense questions, repetition, stereotypical bias, and confidently wrong answers in dialogue tests.
280-billion-parameter transformer model, the largest in a series scaling from 44 million parameters
Stronger reading comprehension, fact-checking, and toxic-language detection as model size grows
MMLU benchmark gains across humanities, science, medicine, and general knowledge categories
Dialogue tests where Gopher cited Wikipedia correctly on cell biology without dialogue fine-tuning
Companion ethics paper mapping 21 large language model risks across six thematic areas
RETRO retrieval architecture matches transformer quality with an order of magnitude fewer parameters
Documents scaling behavior from 44M to 280B parameters across concrete NLP tasks.
Pairs model results with a 21-risk ethics taxonomy for large language models.
Shows MMLU gains over GPT-3 and UnifiedQA on multiple subject categories.
RETRO adds source tracing and training efficiency on top of the Gopher work.
Not offered as a public API or chat product for general users.
Still trails human expert scores on several MMLU categories in DeepMind's charts.
Dialogue tests expose repetition, bias, and confident factual errors.
Research post is from December 2021, so newer models have since surpassed these results.
What is Gopher from Google DeepMind?
Gopher is a 280-billion-parameter transformer language model DeepMind described in a December 2021 research blog post. It was trained alongside smaller models to study how scale changes language understanding and generation.
How large is the Gopher language model?
Gopher has 280 billion parameters. DeepMind also trained related transformer models ranging from 44 million parameters up to that largest Gopher checkpoint.
What benchmarks did Gopher improve on?
Gopher exceeded prior language models on several Massive Multitask Language Understanding (MMLU) categories, including reading comprehension and fact-checking tasks highlighted in DeepMind's December 2021 post.
What are Gopher's known failure modes?
DeepMind reported that Gopher can repeat itself, reflect stereotypical biases, and state incorrect answers confidently during dialogue, especially on logical reasoning and common-sense prompts.
Is Gopher a public chatbot product?
No. Gopher was introduced as research alongside three papers on model scaling, ethical risks, and the RETRO retrieval architecture rather than as a standalone consumer application.
What is RETRO in the Gopher research release?
RETRO, or Retrieval-Enhanced Transformer, is a related DeepMind architecture released alongside Gopher that queries an internet-scale text index during generation so outputs can be traced to source passages while using fewer parameters.