Last updated 07-30-2026
Category:
Reviews:
Join thousands of AI enthusiasts in the World of AI!
Galactica
Five open-weight checkpoints from 125M to 120B parameters give researchers a science-trained language model built from 106 billion curated tokens across 48 million papers, textbooks, encyclopedias, and knowledge bases. Galactica stores, combines, and reasons across modalities including LaTeX, Python code, SMILES formulas, and amino acid sequences inside one decoder-only Transformer architecture. Meta's Papers with Code team open sourced the weights for researchers who want to study how language models organize scientific knowledge.
General-purpose LLMs train on broad web crawls where social chatter can dominate the token budget. Galactica used only curated open-access science sources, which the paper argues lets it train for multiple epochs without overfitting. Meta removed the public web demo three days after launch in November 2022 when critics showed confident but fabricated citations and equations, so today you download checkpoints from Hugging Face rather than chat through a hosted interface.
Machine learning researchers can reproduce benchmark numbers from the arXiv paper, including 77.6% on PubMedQA and 52.9% on MedMCQA dev. Computational biologists and chemists can test protein annotation and molecule tasks through Galactica's specialized tokens for amino sequences and SMILES strings. Graduate students exploring scientific QA, citation prediction, or math word problems can experiment with sizes up to 120 billion parameters without training from scratch.
Five checkpoints span 125M, 1.3B, 6.7B, 30B, and 120B parameters
Training corpus totals 106 billion tokens across 48 million scientific papers
Scores 68.2% on LaTeX equation probes versus GPT-3 at 49.0%
Hits 77.6% on PubMedQA and 52.9% on MedMCQA dev benchmarks
30B model reaches 20.4% on MATH versus PaLM 540B at 8.8%
Special tokens cover citations, SMILES formulas, and amino acid sequences
Five open-weight sizes from 125M to 120B parameters cover a wide hardware range.
Curated 106-billion-token science corpus avoids the noise of general web crawls.
Strong published numbers on PubMedQA, MedMCQA, and math reasoning benchmarks.
Special tokens let you work with citations, SMILES, and protein sequences in one model.
Public web demo was removed; galactica.org no longer resolves.
CC BY-NC 4.0 license blocks commercial deployment without a separate agreement.
Model card warns Galactica can hallucinate confident but wrong answers on niche science topics.
What is Galactica?
Galactica is a large language model from Meta AI trained on curated scientific text and data. It targets tasks like citation prediction, math reasoning, protein annotation, and scientific summarization across five open-weight sizes.
Is Galactica free to use?
Yes, Galactica model weights are free to download for non-commercial research under the CC BY-NC 4.0 license on Hugging Face. There is no paid hosted product; you run the checkpoints locally or on your own GPU infrastructure.
What sizes does Galactica offer?
Galactica ships five checkpoints labeled mini (125M), base (1.3B), standard (6.7B), large (30B), and huge (120B) parameters. Each size is published on Hugging Face under the facebook/galactica model family.
Why was the Galactica demo shut down?
Meta paused the public galactica.org demo on November 17, 2022 after users reported confident but incorrect scientific outputs, including fabricated citations. The paper and open weights remain available; only the hosted chat demo was removed.
What data trained Galactica?
Galactica trained on 106 billion tokens from open-access papers, textbooks, encyclopedias, code, knowledge bases, and scientific websites. The corpus includes roughly 48 million papers plus reference material, SMILES strings, and protein sequences.
What benchmarks does Galactica report?
The Galactica paper reports 77.6% on PubMedQA, 52.9% on MedMCQA dev, 41.3% on mathematical MMLU versus Chinchilla at 35.7%, and 68.2% on LaTeX equation probes versus GPT-3 at 49.0%.
What license covers Galactica weights?
Galactica Hugging Face checkpoints are released under CC BY-NC 4.0, which allows non-commercial research use with attribution. Commercial deployment requires separate permission because the license blocks commercial applications.