Chinchilla vs Terracotta

Chinchilla is an advanced artificial intelligence model with 70 billion parameters, developed to optimize both model size and the volume of training data for efficient learning. It was trained using an extraordinary 1.4 trillion tokens, with an emphasis on scaling the model and data proportionately. This method of training is based on research that suggests optimal training occurs when model size and training tokens are increased in tandem. Chinchilla shares its compute budget with another model named Gopher, but it distinguishes itself by leveraging four times more training data. Despite this difference, both models are designed to operate under the same number of FLOPs, ensuring efficient compute resource utilization. Chinchilla leverages MassiveText, a vast dataset, and employs an adaptation of the SentencePiece tokenizer to interpret and process data. For a detailed understanding of its architecture and training, one can refer to the paper that elaborates on these aspects.

Terracotta is a cutting-edge platform designed to enhance the workflow for developers and researchers working with large language models (LLMs). This intuitive and user-friendly platform allows you to manage, iterate, and evaluate your fine-tuned models with ease. With Terracotta, you can securely upload data, fine-tune models for various tasks like classification and text generation, and create comprehensive evaluations to compare model performance using both qualitative and quantitative metrics. Our tool supports connections to major providers like OpenAI and Cohere, ensuring you have access to a broad range of LLM capabilities. Terracotta is the creation of Beri Kohen and Lucas Pauker, AI enthusiasts and Stanford graduates, who are dedicated to advancing LLM development. Join our email list to stay informed on the latest updates and features that Terracotta has to offer.