Chinchilla vs ChatGPT Plugins

Chinchilla is an advanced artificial intelligence model with 70 billion parameters, developed to optimize both model size and the volume of training data for efficient learning. It was trained using an extraordinary 1.4 trillion tokens, with an emphasis on scaling the model and data proportionately. This method of training is based on research that suggests optimal training occurs when model size and training tokens are increased in tandem. Chinchilla shares its compute budget with another model named Gopher, but it distinguishes itself by leveraging four times more training data. Despite this difference, both models are designed to operate under the same number of FLOPs, ensuring efficient compute resource utilization. Chinchilla leverages MassiveText, a vast dataset, and employs an adaptation of the SentencePiece tokenizer to interpret and process data. For a detailed understanding of its architecture and training, one can refer to the paper that elaborates on these aspects.

OpenAI follows an iterative deployment philosophy, and as part of this approach, it is gradually releasing plugins for ChatGPT. The purpose of this gradual release is to study the real-world use of the plugins, assess their impact, and identify any safety and alignment challenges that may arise. Getting these aspects right is essential for OpenAI to achieve its mission.

Since the launch of ChatGPT, users have expressed interest in having access to plugins, and many developers are experimenting with similar ideas. OpenAI is initially rolling out plugins to a small set of users and plans to gradually increase access as it learns more. After an alpha period, API users who want to integrate plugins into their products will also have access. OpenAI is excited to build a community that will help shape the future of the human-AI interaction paradigm.

Developers who have been invited off the waitlist can use OpenAI's documentation to build a plugin for ChatGPT. Once the plugin is created, it is listed in the prompt shown to the language model, along with documentation that instructs the model on how to use each one. The first set of plugins has been created by Expedia, FiscalNote, Instacart, KAYAK, Klarna, Milo, OpenTable, Shopify, Slack, Speak, Wolfram, and Zapier.