AI21Labs vs GPT-4

AI21Labs presents lm-evaluation, a comprehensive evaluation suite designed for assessing the performance of large-scale language models. This robust toolkit is an important resource for developers and researchers aiming to analyze and improve language model capabilities. The suite allows for the execution of a battery of tests and supports integration with both AI21 Studio API and OpenAI's GPT3 API.

Users can easily contribute to the development of this suite by participating in the open-source project and interacting with its community on GitHub. Setting up lm-evaluation is straightforward, and its flexibility enables users to test models against multiple-choice and document probability tasks, amongst others mentioned in the Jurassic-1 Technical Paper. With detailed instructions for installation, usage, and the ability to run the suite through different providers, the lm-evaluation project is prepped to accelerate the evolution of language models.

GPT-4 is the latest milestone in OpenAI’s effort in scaling up deep learning.

GPT-4 is a large multimodal model (accepting image and text inputs, emitting text outputs) that, while less capable than humans in many real-world scenarios, exhibits human-level performance on various professional and academic benchmarks. For example, it passes a simulated bar exam with a score around the top 10% of test takers; in contrast, GPT-3.5’s score was around the bottom 10%. We’ve spent 6 months iteratively aligning GPT-4 using lessons from our adversarial testing program as well as ChatGPT, resulting in our best-ever results (though far from perfect) on factuality, steerability, and refusing to go outside of guardrails.

GPT-4 is more creative and collaborative than ever before. It can generate, edit, and iterate with users on creative and technical writing tasks, such as composing songs, writing screenplays, or learning a user’s writing style.