Pixtral 12B - Mistral vs ChatGPT Plugins

French AI startup Mistral has recently launched its first multimodal model, Pixtral 12B, which processes both images and text. Built on Mistral's Nemo 12B text model, Pixtral 12B boasts 12 billion parameters (24GB in size), enabling it to perform tasks like captioning images and counting objects within photos.

The model is available for download on GitHub and Hugging Face, under the Apache 2.0 license, allowing for unrestricted use, including fine-tuning and commercial applications.

Pixtral 12B is comparable to other multimodal models such as Anthropic’s Claude family and OpenAI’s GPT-4, capable of answering questions based on images using URLs or base64-encoded formats. While no web demos were available at launch, Mistral plans to roll out testing through its chatbot and API platforms soon.

This release follows the company's significant $645 million funding round, highlighting Mistral’s ambition to compete with major AI players by focusing on open models and corporate consulting services.

OpenAI follows an iterative deployment philosophy, and as part of this approach, it is gradually releasing plugins for ChatGPT. The purpose of this gradual release is to study the real-world use of the plugins, assess their impact, and identify any safety and alignment challenges that may arise. Getting these aspects right is essential for OpenAI to achieve its mission.

Since the launch of ChatGPT, users have expressed interest in having access to plugins, and many developers are experimenting with similar ideas. OpenAI is initially rolling out plugins to a small set of users and plans to gradually increase access as it learns more. After an alpha period, API users who want to integrate plugins into their products will also have access. OpenAI is excited to build a community that will help shape the future of the human-AI interaction paradigm.

Developers who have been invited off the waitlist can use OpenAI's documentation to build a plugin for ChatGPT. Once the plugin is created, it is listed in the prompt shown to the language model, along with documentation that instructs the model on how to use each one. The first set of plugins has been created by Expedia, FiscalNote, Instacart, KAYAK, Klarna, Milo, OpenTable, Shopify, Slack, Speak, Wolfram, and Zapier.