OpenAI, the "capped" for-profit company founded by Elon Musk, is continuing to improve its GPT-3 language model and is developing a tool for summarizing books or texts. It's a development that could benefit businesses in particular.
It has now become possible to create summaries for entire books with artificial intelligence. While the field already counts some specialized companies such as Primer plus some trial work in the area from the likes of Facebook and Google, OpenAI is now positioning itself in the sector. The association has developed an AI model capable of summarizing books, or more simply texts, documentation and even studies. In fact, the model summarizes small sections of a text and then adds them together before creating a more refined and precise summary. It's a potentially effective formula that could prove a boon to productivity.
The capability of having long and tedious documents summarized may well be useful for several kinds of businesses. McKinsey reports that employees spend about 1.8 hours per day, or 9.3 hours per week, on average searching for and gathering work-related information.
In order to judge the quality of these summaries, OpenAI asked humans to evaluate them. And the computer has been getting a score of 6/7 5% of the time from people who have read the book and 5/7 about 15% of the time. While this is far from extraordinary due to lack of consistency at this point, the language is only in an experimental phase. The model sometimes generates inaccurate statements due to lack of context, at some times important details are difficult to identify.
Reinforcement learning
While the task of summarizing books may seem easy, in fact that's one of the segments that is problematic for large, pre-trained models. This new model builds on research from OpenAI, which has found in its multiple AI experiments that human involvement can help an artificial intelligence to learn in a better manner. The training aligns with human feedback to change user preferences, more commonly known as reinforcement learning.
So to create the model, OpenAI combined reinforcement learning with another idea, that of writing small, simple individual summaries and making more refined and complete summaries as they go along. This turns a difficult task (summarizing a book or a long complex text) into a much simpler assignment (summarizing several short sections of a book or text before making it understandable). It was then trained on fiction books containing over 100,000 words.
"This work is part of our ongoing research into aligning advanced AI systems, which is key to our mission," wrote the OpenAI researchers in a blog entry. "Our progress on book summarization is the first large-scale empirical work on scaling alignment techniques. Going forward, we are researching better ways to assist humans in evaluating model behavior, with the goal of finding techniques that scale to aligning artificial general intelligence."
For the moment, the firm has not yet communicated on whether or not it intends to make its service public and open source.
