Recently, Mark Zuckerberg stated that Meta aims to achieve an artificial intelligence that is smarter than humans. He seems to have a recipe for achieving this goal, and the first ingredient is human talent: Zuckerberg, it seems, tried to attract leading researchers to Meta Superintelligence Labs with nine-figure offers. The second ingredient is AI itself. Recently, the entrepreneur stated in a profit call that Meta Superintelligence Labs will focus on creating self-improving artificial intelligence, or systems that can improve themselves to achieve ever higher levels of performance.
The possibility of self-magment distinguishes artificial intelligence from other revolutionary technologies. CRISPR, a genetic editing technology, is unable to improve its accuracy in the direction of DNA sequences and fusion reactors cannot figure out how to make the technology commercially viable. However, LLMs (Large Language Models, or LLMs) can optimize the computer chips on which they run. Train other LLMs in a cheap and efficient way, and maybe even create original ideas for AI research. And they have already made some progress in all these areas.
Secondo Zuckerberg, l’auto-miglioramento dell’intelligenza artificiale potrebbe portare un mondo in cui gli esseri umani sarebbero liberati dalla routine lavorativa quotidiana e potrebbero perseguire i loro obiettivi più grandi con il supporto di compagni artificiali brillanti e iper-efficaci. Ma il miglioramento personale crea anche un rischio fondamentale, secondo Chris Painter, responsabile delle politiche dell’organizzazione di ricerca sull’intelligenza artificiale METR. Se l’IA accelerasse lo sviluppo delle proprie capacità, potrebbe migliorare rapidamente nell’hacking, nel progetto di armi e nella manipolazione delle persone. Alcuni ricercatori ipotizzano addirittura che questo ciclo di feedback positivo possa portare a un'”esplosione di intelligenza” in cui l’intelligenza artificiale si lancerebbe rapidamente ben oltre il livello delle capacità umane.
Ma non è necessario essere pessimisti per prendere sul serio le implicazioni della capacità di auto-migliorarsi. OpenAI, Anthropic e Google includono tutti i riferimenti alla ricerca automatizzata nell’IA nei loro framework di sicurezza AI (insieme strutturato di standard, politiche, procedure e best practice per rafforzare la sicurezza delle informazioni), insieme a categorie di rischio più familiari come le armi chimiche e la sicurezza informatica. “Penso che questa sia la strada più veloce per un’intelligenza artificiale potente”, afferma Jeff Clune, professore di informatica presso l’Università della Columbia Britannica e consulente di ricerca senior presso Google DeepMind. “Probabilmente è la cosa più importante a cui dovremmo pensare”.
Allo stesso modo, Clune afferma che l’automazione della ricerca e dello sviluppo dell’IA potrebbe avere enormi benefici. Da soli, noi umani potremmo non essere in grado di immaginare le innovazioni e i miglioramenti che un giorno consentiranno all’IA di affrontare problemi prodigiosi come il cancro e il cambiamento climatico.
Per ora, l’ingegno umano è ancora il motore principale del progresso dell’IA. Altrimenti, è improbabile che Meta abbia fatto offerte così esorbitanti per attirare i ricercatori nel suo laboratorio di superintelligence. Ma l’IA sta già contribuendo al proprio sviluppo e sta per assumere un ruolo ancora più importante nei prossimi anni. Ecco cinque modi in cui l’IA sta migliorando.
1. Increasing productivity
Today, the most important contribution that LLMs make to the development of AI may also be the most banal. “The greatest contribution is coding assistance,” says Tom Davidson, a senior researcher at Forethought, a non-profit AI research organization. Tools that help engineers write software faster, such as Claude Code and Cursor, seem popular throughout the AI industry: Google CEO Sundar Pichai said in October 2024 that a quarter of the company’s new code was generated by AI, and Anthropic recently documented a wide variety of ways its employees use Claude Code. If engineers become more productive due to this coding assistance, they will be able to design, test, and implement new AI systems more quickly.
But the productivity advantage that these tools confer remains uncertain: if engineers are spending a lot of time correcting errors made by AI systems, they may not be doing more work, even if they are spending less time writing code manually. A recent study by METR found that developers take about 20% more to complete tasks when using AI coding assistants, although Nate Rush, a member of the METR technical team and co-leader of the study, notes that he only examined extremely experienced developers working on large code bases. Your conclusions may not apply to AI researchers who write quick scripts to run experiments.
Conducting a similar study in cutting-edge laboratories could help provide a much clearer picture of whether coding assistants are making leading AI researchers more productive, says Rush, but this work has not yet been done. Meanwhile, simply accepting the word of software engineers is not enough: the developers studied by METR thought that AI coding tools had made them work more efficiently, although the tools, in reality, had slowed them down substantially.
2. Optimizing the infrastructure
Writing code quickly is not a big advantage if you have to wait hours, days or weeks for it to run. LLM training, in particular, is an agonizingly slow process, and more sophisticated reasoning models can take many minutes to generate a single response. These delays are important bottlenecks for the development of AI, says Azalia Mirhoseini, assistant professor of computer science at Stanford University, United States, and senior scientist at Google DeepMind. “If we can run AI faster, we can innovate more,” she says.
That’s why Mirhoseini has been using AI to optimize AI chips. In 2021, she and her collaborators at Google created a non-LLM-based AI system that could decide where to position multiple components on a computer chip to optimize efficiency. Although the work has attracted skepticism from the chip design community, Mirhoseini states that Nature magazine investigated the article and validated the validity of the work, and she notes that Google used the system designs for several generations of its custom AI chips.
More recently, Mirhoseini applied LLMs to the problem of writing kernels, low-level functions that control how various operations, such as matrix multiplication, are performed on chips. She found that even general-purpose LLMs can, in some cases, write kernels that run faster than human-designed versions.
In another part of Google, scientists built a system they used to optimize various parts of the company’s LLM infrastructure. The system, called AlphaEvolve, asks Google’s LLM Gemini to write algorithms to solve a problem, evaluates these algorithms and asks Gemini to improve the most successful ones, repeating this process several times. AlphaEvolve designed a new approach to the operation of datacenters that saved 0.7% of Google’s computing resources, made additional improvements in the design of Google’s custom chips, and designed a new kernel that accelerated Gemini training by 1%.
This may seem like a small improvement, but in a huge company like Google, it amounts to huge savings of time, money and energy. And Matej Balog, a Google DeepMind research scientist who led the AlphaEvolve project, says that he and his team tested the system only on a small component of Gemini’s overall training pipeline. Applying it more widely, he says, could result in more savings.
3. Automating training
LLMs are notoriously eager for data, and training them is expensive at each stage. In some specific domains, such as unusual programming languages, for example, real-world data is too scarce to train them effectively. Learning by reinforcement with human feedback, a technique in which humans score LLMs’ responses to prompts and models are then trained based on these scores, has been fundamental to creating models that behave according to human standards and preferences, but getting human feedback is time-consuming and expensive.
Increasingly, LLMs are being used to fill these gaps. If received with many examples, they can generate plausible synthetic data in domains in which they have not been trained, and this synthetic data can then be used for training. They can also be used effectively in reinforcement learning: in an approach called “LLM as a judge”, LLMs, instead of humans, are used to score the results of models that are being trained. This approach is fundamental to the influential “Constitutional AI” framework proposed by Anthropic researchers in 2022, in which an LLM is trained to be less harmful based on feedback from another LLM.
Data scarcity is a particularly acute problem for AI agents. The most effective need to be able to perform multi-step plans to fulfill specific tasks, but examples of successful completion of step-by-step tasks are scarce online, and using humans to generate new examples would be expensive. To overcome this limitation, Mirhoseini from Stanford and his colleagues recently tested a technique in which an agent generates a possible step-by-step approach to a given problem, a judge evaluates whether each step is valid, and then a new agent is trained with these steps. “You are no longer limited by data, because the model can simply arbitrarily generate more and more experiences,” says Mirhoseini.
Perfecting the design of agents
One area in which LLMs have not yet made significant contributions is in their own design. Today’s LLMs are all based on a neural network structure called transformer, proposed by human researchers in 2017, and the notable improvements made since then in architecture were also designed by humans.
But the emergence of LLM agents created a totally new design universe to be explored. They need tools to interact with the outside world and instructions on how to use them, and optimizing these tools and instructions is essential to produce effective agents. “Humans didn’t spend so much time mapping all these ideas, so there are many more fruits within reach,” says Clune. “It’s easier to create an AI system to pick them up.”
Together with researchers from the startup Sakana AI, Clune created a system called “Darwin Gödel Machine”: an LLM agent that can iteratively modify your prompts, tools and other aspects of your code to improve your own performance in tasks. Not only did Darwin Gödel Machine get higher scores on tasks by modifying itself, but as it evolved, it also managed to find new modifications that its original version would not be able to discover. She had entered a real cycle of self-improvement.
5. Moving forward in the research
Although LLMs are accelerating several parts of the LLM development pipeline, humans may still remain essential to AI research for a long time. Many experts point out the “taste for research”, or the ability that the best scientists have to identify new issues and promising directions for research, as a particular challenge for AI and a key ingredient in the development of AI.
But Clune says that the taste for research may not be as much of a challenge for AI as some researchers think. He and Sakana AI researchers are working on an end-to-end system for AI research that they call the “AI Scientist”. He searches the scientific literature to determine his own research question, conducts experiments to answer this question, and then writes the results.
An article he wrote earlier this year, in which he developed and tested a new training strategy with the aim of improving the combination of examples of training data by neural networks, was sent anonymously to a workshop at the (International Conference on Machine Learning, or ICML) one of the most prestigious conferences in the area, with the consent of the workshop organizers. The training strategy did not end up working, but the article was scored high enough by the reviewers to qualify it for acceptance (it is worth noting that ICML workshops have lower acceptance standards than the main conference). On another occasion, Clune says that the AI Scientist came up with a research idea that was later proposed independently by a human researcher in X, where it attracted great interest from other scientists.
“We are now looking at the moment of the AI Scientist’s GPT-1,” says Clune. “In a few years, he will be writing articles that will be accepted in the best peer-reviewed conferences and journals in the world. He will be making innovative scientific discoveries.”
Is superintelligence on the way?
With all this enthusiasm for self-improvement of AI, it seems likely that, in the coming months and years, the contributions of AI to its own development will only multiply. From what Mark Zuckerberg says, this could mean that super-intelligent models, which surpass human capabilities in many domains, are just around the corner. In reality, however, the impact of self-impressible AI is far from certain.
It is remarkable that AlphaEvolve has accelerated the training of its own core LLM system, Gemini, but this 1% increase in speed may not noticeably change the pace of Google’s AI advances. “We are still in a very slow feedback cycle,” says Balog, the researcher at AlphaEvolve. “Gemini training takes a significant amount of time. So, maybe you’ll see the exciting beginnings of this virtuous cycle, but it’s still a very slow process.”
If each subsequent version of Gemini accelerates its own training by an additional 1%, these accelerations will accumulate. And since each successive generation will be more capable than the previous one, it should be able to achieve even greater accelerations in training, not to mention all the other ways AI can invent to improve itself. Under such circumstances, the defenders of superintelligence argue, an eventual explosion of intelligence seems inevitable.
However, this conclusion ignores a key observation: innovation becomes more difficult over time. In the early days of any scientific field, discoveries arise quickly and easily. There are many obvious experiments to be carried out and ideas to be investigated, and none of them have been attempted before. But as the science of deep learning matures, finding each additional improvement may require a substantial effort from both humans and their AI collaborators. It is possible that when AI systems reach research capacity at the human level, humans or less intelligent AI systems have already reaped all the fruits within reach.
To make matters worse, the AI systems that matter most for AI development, those used within cutting-edge AI companies, are probably more advanced than those that have been released to the general public, so measuring o3 capabilities may not be a good way to infer what is happening within OpenAI.
But external researchers are doing their best. For example, monitoring the overall pace of AI development to determine whether this pace is accelerating or not. METR is following advances in AI skills, measuring the time it takes humans to perform tasks that more advanced systems can complete on their own. They found that the time required to complete tasks that AI systems can perform independently has doubled every seven months since the launch of GPT-2 in 2019.
Since 2024, this doubling time has been reduced to four months, which suggests that AI progress is, in fact, accelerating. There may be unglamorous reasons for this: cutting-edge AI labs are full of money from investors, who can spend hiring new researchers and buying new hardware. But it is totally plausible that the self-improvement of AI is also playing a role.
This is just an indirect hint. But Davidson, the Forethought researcher, says there are good reasons to expect AI to accelerate its own advance, at least for a while. METR’s work suggests that the effect of “fruit at reach” is not slowing down human researchers today, or, at least, that the increase in investments is effectively counterbalanging any slowdown. If AI significantly increases the productivity of these researchers, or even takes over a portion of the research work on its own, this balance will change in favor of accelerating research.
“You, I think, would strongly hope that there would be a period when AI progress accelerates,” says Davidson. “The big question is how long this will last.”
fontes: MIT Technology Review
