This AI System Continuously Improves Its Knowledge

Contemporary large language models (LLMs) can produce stunning poetry and sophisticated code, yet they fundamentally lack the capability to learn from their experiences.

Researchers at the Massachusetts Institute of Technology (MIT) have created a method that enables LLMs to enhance themselves by adjusting their own parameters in response to valuable new data.

This advancement is a move towards developing artificial intelligence models that can learn continuously—a long-sought objective in the field—essential for machines to more accurately reflect human intelligence. In the interim, it could lead to chatbots and other AI tools that are better suited to integrate new information, including a user’s interests and preferences.

The MIT framework, known as Self Adapting Language Models (SEAL), enables an LLM to generate its own synthetic training data and adjust its procedures based on the input it gathers.

“The original concept was to investigate if tokens [units of text input to and produced by LLMs] could initiate a significant update to a model,” states Jyothish Pari, a PhD student at MIT who is part of the SEAL development team. Pari emphasizes the potential of a model’s output being utilized for its own training.

Adam Zweiger, an undergraduate researcher at MIT contributing to SEAL, points out that while newer models can make better decisions through advanced reasoning, they do not retain the benefits of this reasoning over time.

In contrast, SEAL produces new insights and incorporates them into its own weights or parameters. For example, when presented with a statement regarding the difficulties encountered during the Apollo space program, the model created additional text attempting to explain the significance of that statement. This process is likened to how a human student takes and reviews notes to facilitate learning.

The system then updated the model using the generated data and evaluated how effectively the new model could answer a series of questions. This ultimately provides a reinforcement learning signal that directs the model towards updates that enhance its overall capabilities and enable continued learning.

The researchers evaluated their method on small and medium versions of two open-source models: Meta’s Llama and Alibaba’s Qwen. They assert that this technique should also be effective for significantly larger frontier models.

The researchers examined the SEAL approach on text as well as a benchmark known as ARC, which measures an AI model’s ability to tackle abstract reasoning tasks. In both scenarios, they observed that SEAL facilitated ongoing learning well beyond the initial training period.

Pulkit Agrawal, a professor at MIT overseeing the project, mentions that the SEAL initiative addresses crucial themes in AI, including how AI can autonomously determine what it should learn. He believes it could help create more personalized AI models. “LLMs are powerful, but we don’t want their knowledge to stagnate,” he remarks.

SEAL is not yet a method for AI to improve endlessly. Notably, as Agrawal indicates, the LLMs tested suffer from “catastrophic forgetting,” a concerning phenomenon where acquiring new information can lead to the loss of previously held knowledge. This suggests a fundamental distinction between artificial and biological neural networks. Pari and Zweigler also highlight that SEAL demands considerable computational resources, and it remains unclear how to best schedule future learning phases. One intriguing idea, mentioned by Zweigler, is that, similar to humans, perhaps LLMs could undergo periods of “sleep” during which new information gets consolidated.

Nevertheless, despite its limitations, SEAL represents an exciting avenue for future AI research—and it may eventually be integrated into upcoming advanced AI models.

What are your thoughts on AI that can keep learning? Send an email to hello@wired.com to share.