UniLM vs GPT-4

This paper introduces UniLM, a Unified pre-trained Language Model, that serves as a new benchmark for Natural Language Understanding (NLU) and Natural Language Generation (NLG) tasks. It is unique in its use of a shared Transformer network that is pre-trained on unidirectional, bidirectional, and sequence-to-sequence tasks, employing special self-attention masks for contextual prediction control. UniLM outperforms BERT in the GLUE benchmark and excels in SQuAD 2.0 and CoQA question answering, setting new records in five NLG datasets, including notable improvements in CNN/DailyMail and Gigaword summarization tasks. The models and code shared by the authors aid the research community in further advancements.

GPT-4 is the latest milestone in OpenAI’s effort in scaling up deep learning.

GPT-4 is a large multimodal model (accepting image and text inputs, emitting text outputs) that, while less capable than humans in many real-world scenarios, exhibits human-level performance on various professional and academic benchmarks. For example, it passes a simulated bar exam with a score around the top 10% of test takers; in contrast, GPT-3.5’s score was around the bottom 10%. We’ve spent 6 months iteratively aligning GPT-4 using lessons from our adversarial testing program as well as ChatGPT, resulting in our best-ever results (though far from perfect) on factuality, steerability, and refusing to go outside of guardrails.

GPT-4 is more creative and collaborative than ever before. It can generate, edit, and iterate with users on creative and technical writing tasks, such as composing songs, writing screenplays, or learning a user’s writing style.