RWKV-LM vs GPT-4

RWKV is an innovative RNN-based Language Model that delivers the exceptional performance of transformer-level Large Language Models (LLMs). This remarkable fusion of RNN simplicity with transformer efficiency creates a model that is highly parallelizable, akin to the GPT models. RWKV is not only swift in inference but also demonstrates expedient training speeds while being memory-efficient, thereby conserving valuable VRAM.

It supports an "infinite" context length, allowing it to handle very long sequences of data seamlessly. Furthermore, users benefit from free sentence embedding capabilities, enhancing its utility for a wide array of natural language processing applications. As an Apache-2.0 licensed project, it stands as a public repository on GitHub, inviting collaboration and continued development.

GPT-4 is the latest milestone in OpenAI’s effort in scaling up deep learning.

GPT-4 is a large multimodal model (accepting image and text inputs, emitting text outputs) that, while less capable than humans in many real-world scenarios, exhibits human-level performance on various professional and academic benchmarks. For example, it passes a simulated bar exam with a score around the top 10% of test takers; in contrast, GPT-3.5’s score was around the bottom 10%. We’ve spent 6 months iteratively aligning GPT-4 using lessons from our adversarial testing program as well as ChatGPT, resulting in our best-ever results (though far from perfect) on factuality, steerability, and refusing to go outside of guardrails.

GPT-4 is more creative and collaborative than ever before. It can generate, edit, and iterate with users on creative and technical writing tasks, such as composing songs, writing screenplays, or learning a user’s writing style.