In the past year, artificial intelligence chatbots like ChatGPT, Claude, and Gemini have made significant strides, improving their ability to maintain context in long and complex conversations as well as process increasingly large text inputs. However, these models face a major limitation: they struggle to generate responses that exceed 1,000 words. This obstacle has sparked debate among users and experts, who question why current models cannot surpass this barrier.

Tsinghua's Innovation: LLMs Capable of Generating Novels

Researchers at Tsinghua University in Beijing have tackled this challenge with an innovative approach. Led by Yushi Bai, the team has introduced two large-scale language models (LLMs) capable of generating coherent responses exceeding 10,000 words. This achievement is comparable to the length of Franz Kafka's novel "The Metamorphosis," which is around 11,000 words. The researchers developed a system called AgentWrite, which breaks down generation tasks, enabling the LLMs to produce lengthy texts without losing coherence.

Mastering Copilot for business efficiency

Maximize your productivity with Copilot in Microsoft 365.
View course

The work is based on the open-source GLM-4 9B model, developed by the same university and trained with 10 trillion tokens in 26 languages. This model was used to create LongWriter-glm4-9b, capable of generating responses up to 7,872 words in a single output, far surpassing the 1,896 words generated by a standard chatbot in the same task. Additionally, the researchers utilized the 8B version of the Llama 3.1 model, creating LongWriter-llama3.1-8b, which offers similar results.

 

The Impact of Open-Source AI

One of the most notable aspects of this advancement is its open-source nature. Unlike closed models that keep their developments under the control of companies, open-source models like LongWriter allow anyone in the world to analyze, implement, and improve these innovations. This collaborative development philosophy not only drives innovation but also democratizes access to the most advanced artificial intelligence tools.

This achievement by Tsinghua University represents not only a technical breakthrough but also highlights the power of international collaboration in AI development. Models like LongWriter open up new possibilities for creating extensive and coherent content, overcoming the limitations that have previously hindered LLMs. With these advancements, we may soon see artificial intelligence not just as a tool for short responses, but as an author capable of creating entire novels and other substantial texts.