Last updated 07-30-2026
Category:
Reviews:
Join thousands of AI enthusiasts in the World of AI!
A Generalist Agent
Gato is the generalist agent DeepMind introduced in its May 2022 research post "A Generalist Agent." One transformer policy with shared weights handles chat, image captioning, Atari games, and real robot arm block stacking from the same model. The post describes it as a multi-modal, multi-task, multi-embodiment policy that picks whether to emit text, joint torques, button presses, or other tokens based on context.
Most tools in this space train a separate model per task domain. Gato instead serializes every modality and environment into a flat token sequence and runs it through a transformer shaped like a large language model. That means one pretrained checkpoint moves between dialogue, vision, and control without swapping architectures.
During training, data from simulated and real environments mixes with natural language and image datasets, with loss masked so Gato predicts only action and text targets. At deployment, a prompt tokenizes first, each new observation appends to the sequence, and the model samples an action vector one token at a time inside a 1024-token context window. The blog includes domain-grouped charts of tasks where the pretrained model clears expert score thresholds.
The writeup targets AI and robotics researchers more than everyday software buyers. You read and cite the research on deepmind.google; there is no public signup flow, API tier, or download button on the pages reviewed here.
One transformer with the same weights plays Atari, captions images, chats, and stacks blocks with a real robot arm
Deployment keeps a 1024-token context window over prior observations and actions
Announced in the May 2022 A Generalist Agent post on deepmind.google
Outputs text, joint torques, button presses, or other tokens depending on the active environment
Training serializes multi-modal data into flat token batches processed like a large language model
Loss masking limits predictions to action and text targets during training
One shared transformer handles text, vision, and control without separate per-task models.
The DeepMind post documents training, deployment, and cross-domain performance charts in detail.
Demonstrates how tokenized multi-modal sequences can drive both language and robotics actions.
Not available as a public app, API, or downloadable product for end users.
Research announcement from May 2022 with no consumer deployment path on deepmind.google.
Cross-domain breadth means performance varies by task rather than matching specialist tools everywhere.
What is A Generalist Agent?
A Generalist Agent is Google DeepMind's May 2022 research post about Gato, a multi-modal agent that uses one shared transformer policy across chat, image captioning, Atari games, and robot arm control. The page documents how Gato tokenizes prompts, observations, and actions into a single sequence.
Who built Gato?
A Generalist Agent was built by Google DeepMind researchers and announced on deepmind.google in May 2022. The blog post frames Gato as a multi-modal, multi-task, multi-embodiment generalist policy trained on agent experience plus language and image datasets.
Can I use Gato as a public app?
No. A Generalist Agent is a research announcement on deepmind.google, not a hosted product with signup or an API. The page explains Gato's training and deployment flow, but it does not offer a consumer download or interactive demo.
What tasks does Gato handle?
A Generalist Agent covers chat, image captioning, Atari gameplay, and real robot arm block stacking from one model. The DeepMind post shows Gato using the same pretrained weights to switch between text, vision, and control outputs based on context.
How does Gato choose its outputs?
A Generalist Agent tokenizes the prompt and each new observation into one sequence, then samples an action vector autoregressively one token at a time. The DeepMind post says Gato can emit text, joint torques, button presses, or other tokens depending on the environment specification.
Is A Generalist Agent free to read?
Yes. The A Generalist Agent research post on deepmind.google is free to read with no payment or account required. It is open research documentation rather than a paid software plan.