Last updated 07-30-2026
Category:
Reviews:
Join thousands of AI enthusiasts in the World of AI!
GLM-130B
GLM-130B puts a 130-billion-parameter bilingual language model in the open research stack THUDM built around the General Language Model (GLM) pre-training recipe. The weights target English and Chinese text, and the GitHub repo ships inference code, evaluation tasks, and checkpoints accepted at ICLR 2023. You can run left-to-right generation or blank infilling with [MASK] and [gMASK] tokens on hardware that fits a single multi-GPU server rather than a proprietary API.
Where most 100B+ models stay behind closed doors, GLM-130B publishes model weights, training notes, and YAML configs for 30+ benchmarks. Its INT4 quantization path is tuned so four RTX 3090 (24GB) cards can host inference with almost no accuracy drop, a much lower bar than the eight A100 (40GB) setup used for full FP16 runs. The training objective mixes autoregressive blank infilling on 95% of tokens with multi-task instruction data from T0++ and DeepStruct, which is a different bet than standard causal GPT-style pre-training.
Researchers studying bilingual zero-shot transfer, large-model quantization, or reproducible LLM benchmarks will get the most from GLM-130B. The repo focuses on evaluation and inference tooling rather than a hosted chat product, though THUDM later spun dialogue work into ChatGLM. Expect to bring your own GPUs, storage for a 260GB checkpoint, and patience for the weight download form.
130 billion parameters trained on 400+ billion tokens split evenly between English and Chinese
Full FP16 inference on one server with 8 A100 (40GB) or 8 V100 (32GB) GPUs; INT4 quantization drops requirements to 4 RTX 3090 (24GB) cards
NVIDIA FasterTransformer integration reaches up to 2.5x faster decode than Megatron on A100 hardware
Repository ships YAML evaluation configs for 30+ NLP tasks with reproducible benchmark scripts
Two mask tokens support workflows: [MASK] for short blank filling and [gMASK] for left-to-right long generation
Model checkpoint ships as a 260GB archive split across 60 downloadable chunks after form-based access approval
Open weights, code, and evaluation scripts let researchers reproduce 30+ published benchmarks.
INT4 quantization runs the 130B model on four RTX 3090 GPUs without reported performance loss.
English and Chinese pre-training with reported gains over GPT-3 175B on LAMBADA and ERNIE Titan 3.0 260B on CLUE.
FasterTransformer path cuts decode time roughly in half on A100 hardware versus Megatron baselines.
Full FP16 weights need about 260GB storage and a multi-GPU server to load.
Checkpoint access requires a Google Form request rather than a direct Hugging Face download.
The repository centers on evaluation and inference, not a hosted chat API like ChatGLM.
What is GLM-130B?
GLM-130B is an open bilingual dense language model with 130 billion parameters from THUDM, pre-trained with the General Language Model blank-infilling algorithm and accepted at ICLR 2023. The GitHub repo provides inference scripts, evaluation tasks, and access to model weights.
What hardware does GLM-130B need?
GLM-130B recommends 8 A100 (40GB) or 8 V100 (32GB) GPUs for full FP16 inference. With INT4 quantization, GLM-130B can run on a single server with 4 RTX 3090 (24GB) cards while keeping activations in FP16.
Is GLM-130B free to use?
Yes. GLM-130B is open source under Apache-2.0 for the repository code, and model weights are publicly accessible after completing THUDM's download form. You pay only for your own compute and storage.
What languages does GLM-130B support?
GLM-130B supports English and Chinese. Training used about 200 billion tokens in each language, and the model reports strong zero-shot scores on CLUE, FewCLUE, LAMBADA, and MMLU benchmarks.
How does GLM-130B compare to GPT-3?
GLM-130B reports better LAMBADA accuracy than GPT-3 175B and slightly higher MMLU scores in THUDM's published benchmarks, while also publishing weights and evaluation code GPT-3 does not offer. On Chinese CLUE and FewCLUE tasks, GLM-130B beats ERNIE Titan 3.0 260B by double-digit margins.
What license covers GLM-130B?
GLM-130B repository code is licensed under Apache-2.0. Using the GLM-130B model weights is subject to THUDM's separate Model License linked from the GitHub repository.