The idea of an AI system improving itself tends to bring a fairly dramatic picture to mind. A model becomes intelligent enough to redesign itself. The improved version builds an even better successor. That successor does the same thing again, creating a cycle of increasingly capable AI with humans gradually pushed out of the development process.
This is usually what people mean when they talk about recursive self-improvement. It's also not something AI systems can currently do autonomously. OpenAI explicitly said in September 2026 that fully autonomous recursive self-improvement isn't happening today, while Anthropic says it isn't there yet either.
Both companies do, however, report AI taking on more of the work involved in developing future systems. But there's another form of AI self-improvement emerging beneath that much bigger idea. An AI system doesn't necessarily need to retrain its underlying model to change how well it performs.
It can also change the prompts, memory, tools, workflows and control logic wrapped around that model. Researchers are increasingly exploring systems that can make some of those changes themselves, test the result and retain changes that appear to improve performance. The model may remain exactly the same, while the system around it doesn't.
And that creates a much more immediate question for enterprises experimenting with increasingly autonomous AI agents. What happens when the AI system you're governing is no longer necessarily the same system you originally approved?
What Does AI Self-Improvement Actually Mean?
Part of the confusion around self-improving AI comes from treating the term as though it describes one capability. It doesn't. A July 2026 survey of modern self-improving agents offers a useful distinction. Researchers describe an agent as a combination of a foundation model and an operational scaffold around it.
That scaffold can include its prompts, memory, tools and control logic. Self-improvement can therefore happen by changing the foundation model itself, changing parts of that surrounding scaffold, or eventually changing both. Changing the model is probably closest to what most people imagine.
The system's underlying model weights, essentially the internal parameters shaped during training, are updated to improve its capabilities. But scaffolding improvement works differently. Imagine giving the same employee better instructions, better tools, better access to information and a more effective way of organising their work.
The person hasn't suddenly acquired a different brain, but the system around them makes it possible to perform the job differently. Something similar can happen with AI agents. A model that struggles with a task might perform better if its prompt changes, if it remembers useful information from previous attempts, if it gains a new tool or if the workflow controlling how it approaches the task is reorganised.
Humans already make these kinds of changes when developing AI systems. The more interesting step is allowing the system to participate in deciding what should change. That still isn't the same as an AI autonomously redesigning its own intelligence. Nor is it the same as using AI to help researchers build the next generation of models.
These capabilities sit at different points on a much broader spectrum of self-improvement. What we're beginning to see now is movement along that spectrum.
AI Can Already Improve the System Around the Model
An AI system doesn't need access to its underlying model to change how it behaves. There are already plenty of things around the model that influence what an agent can do, including its instructions, tools, memory and the workflow it follows when completing a task. Together, these components are sometimes called an agent harness.
Normally, humans build and improve that harness. If an agent keeps failing at a particular step, for example, a developer might change its instructions, give it access to a different tool or alter the workflow so it checks its work before moving on.
The underlying model hasn't improved, but the overall system might perform considerably better because humans have changed how that model is being used. The next step is allowing AI to participate in that process itself.
Researchers are now experimenting with agents that can look at what happened during previous attempts, identify possible weaknesses in the way they operated and propose changes to the system around them. Those changes can then be tested, with successful ones retained for future tasks.
One example is Regularized Recursive Self-Improvement of Agent Harnesses (RRSI), published in September 2026. The researchers deliberately keep the foundation model frozen while allowing the wider agent system to evolve. Instead of changing the model's weights, the process can modify individual parts of its harness, including prompts, tools and control logic.
Think about what that could eventually mean inside an enterprise. A coding agent might repeatedly struggle with a particular type of task and discover that adding another verification step improves its results. A research agent could learn that information from one tool needs to be checked against another before it can reliably answer certain questions.
An operational agent might find that changing the order in which it completes parts of a workflow reduces failures. Those examples don't mean today's agents can simply redesign themselves however they choose. The important change is much narrower.
Some of the decisions that developers currently make when optimising an AI system could increasingly become part of the system's own improvement loop. And RRSI isn't the only research moving in this direction. Living-Harness explores how an agent can use feedback from completed tasks to update procedural knowledge that persists into future interactions.
Other work is experimenting with agents that generate changes to their own harnesses and then test whether those changes should survive. This creates an important difference between adaptation and persistent self-improvement. An AI agent can already adapt while it's completing a task. It might encounter an error, try another approach and eventually succeed.
But that doesn't necessarily mean anything has changed when the next task begins. It may start again with the same instructions, tools and workflow that caused the problem in the first place. A self-improving system can potentially carry something forward. If a change proves useful, it can become part of how the agent approaches future work.
Over time, the system around the model can therefore evolve even though the foundation model underneath it remains exactly the same. For enterprise teams, that's a much more useful way to think about self-improving AI than imagining a model suddenly rewriting its own intelligence.
The immediate possibility is an AI system that becomes less fixed between deployments because experience can influence how future versions of that system operate. But there's an obvious problem hidden inside that capability. If an AI system changes itself and then performs better, how do you know the change produced a genuine improvement rather than simply making the agent better at the tasks it used to judge itself?
Changing Is Easier Than Getting Better
This is where the idea of self-improvement becomes more complicated. An AI system can find a change that improves its performance without becoming more capable overall. RRSI provides a useful example.
Across eight benchmarks covering coding, agentic workspace and engineering design tasks, the researchers reported gains of up to 14.1 points on the tasks used during the improvement process. When the resulting harnesses were tested across five unfamiliar benchmarks, the gains reached up to 4.7 points.
The system had improved, but the size of that improvement depended heavily on where it was measured. This is the problem of AI generalisation. If an agent becomes very good at solving the problems it repeatedly encounters, that doesn't automatically mean it has developed a better way of handling problems more broadly.
For an enterprise, the distinction is important because production environments rarely stay as predictable as a test environment. Customers behave differently. Data changes. Unusual cases appear. Tools and connected systems are updated. An agent that has optimised itself around yesterday's conditions could perform extremely well until it encounters something its improvement process didn't prepare it for.
There is another risk too. Improving one part of an AI system can make another part worse. HarnessEvolve, another 2026 self-improving agent project, identifies catastrophic forgetting as one of the challenges these systems face. A new version may gain a useful capability while weakening something an earlier version could already do.
That makes the idea of a “better” agent surprisingly difficult to define. Better at what? Under which conditions? Compared with which previous version? And what, if anything, was lost in exchange? For enterprise teams, a higher performance score after an AI changes itself can't be enough to answer those questions.
A proposed improvement needs to survive unfamiliar conditions, preserve capabilities the organisation still depends on and produce enough evidence to justify replacing the version that was already trusted. Once changes can persist beyond the task that produced them, organisations are no longer just evaluating what an AI agent can do.
They're deciding when a modified version of that agent is allowed to become the new production system. That decision belongs much more naturally in change control, which is where the following section can now pick up.
Self-Improving AI Changes Enterprise Change Control
Most enterprise technology operates with a fairly simple assumption. The system does what it was designed to do until someone changes it. The process around that change may be slow or highly automated, but there is still a distinction between operating the system and modifying the system.
Self-improving agents begin to blur that line. If an AI agent can propose a new prompt, retain a useful memory, add or alter a skill, change part of its workflow or rewrite elements of its control logic, some of the architecture affecting future behaviour becomes mutable. That doesn't mean organisations need to prevent every form of adaptation.
A system that can learn from repeated failures could be considerably more useful than one that makes the same mistake forever. But enterprises do need to decide where adaptation ends and controlled system change begins.
What should an AI system be allowed to change?
Not every component needs the same level of protection. A temporary adjustment to working context is different from changing a persistent system prompt. Adding information to memory is different from changing which tools an agent can access. Optimising the order of low-risk workflow steps isn't equivalent to rewriting the logic controlling a consequential business process.
The architecture therefore needs boundaries around the parts of the system that can change autonomously, those that can change only after validation and those that remain outside the agent's control. Reversibility becomes equally important. If an agent adopts a change that later causes unexpected behaviour, teams need to know exactly what changed and be able to restore the last trusted version.
This turns familiar ideas such as versioning, permissions and rollback into part of AI change control. The important question isn't simply whether an agent is capable of modifying a component. It's whether the organisation has deliberately given it the authority to make that modification persistent.
What proves the new version is actually better?
Approval also needs evidence. If an agent changes its own operating system and then performs better on the same tasks that drove the change, the organisation has learned something useful. It hasn't yet established that the new version should replace the old one. That requires testing under conditions the improvement process wasn't optimising against.
It also means checking for regression across capabilities that weren't supposed to change. The distinction is subtle but important. The goal isn't to prove that the agent found a better answer once. It's to establish that the organisation now has a better system.
That suggests a change process where candidate improvements can be tested without immediately becoming permanent, compared against a known version and rejected or rolled back when the evidence doesn't hold up. As agents become more adaptive, the trusted object may no longer be one fixed configuration.
Trust will increasingly depend on the process that decides which changes are allowed to become part of the next version.
Self-Improvement Is Arriving Before Full Autonomy
None of this means fully autonomous recursive self-improvement has arrived. What is changing is how much of the improvement process AI can carry out before a human needs to step in. Anthropic offers a useful example. As of May 2026, the company says Claude authored more than 80 per cent of the code merged into its codebase.
Its researchers have also tested Claude on parts of the AI research process itself, where the system can run experiments and iterate effectively once humans have defined the problem and how success will be measured. Where it still struggles is further upstream, with the judgement required to decide which problems are worth pursuing in the first place.
OpenAI describes a similar shift. In September 2026, it said it had reached its goal of an automated “research intern” capable of carrying out well-defined research tasks under human direction that would otherwise take a skilled researcher several days. Its longer-term target is an automated AI researcher that still operates under human supervision.
The boundary, then, isn't simply between AI that can improve itself and AI that can't. There are stages in between. Humans can choose the objective while AI works out how to achieve it. Humans can define the test while AI experiments with ways to perform better against it.
And, as the harness research shows, AI can increasingly participate in changing parts of the system through which it operates. Independent testing suggests we're still some distance from handing over the whole process.
METR's September assessment of Claude Opus 5.5 found evidence of meaningful AI research acceleration and limited automation, but concluded that the model was unlikely to fully automate AI research and development. For enterprise leaders, the useful thing to watch isn't whether humans suddenly disappear from the loop.
It's where they remain in it. As AI takes over more of the execution, experimentation and optimisation involved in improving AI systems, human responsibility moves towards setting objectives, defining boundaries, validating results and deciding which changes are allowed to persist.
That progression can happen long before an AI system is capable of independently designing and building its own successor.
Final Thoughts: Self-Improving AI Makes Change Itself a Governance Question
The most important development in self-improving AI may not begin with a model rewriting its own intelligence. It may begin much more quietly. An agent learns from previous work. It changes a prompt, retains a useful skill or adjusts part of its workflow. The underlying model remains untouched, but the system that returns for the next task isn't quite the same as the one that handled the last one.
For enterprises, this changes what needs to be governed. It won't be enough to decide what an AI agent can access and what actions it's allowed to take. As self-improving systems develop, organisations will also need clear boundaries around what those systems can change about themselves, how those changes are tested and who has the authority to make them permanent.
Full recursive self-improvement could still be a long way from that point. It doesn't need to arrive before these questions become practical. The closer enterprise AI gets to systems that can learn from their own operation and retain the results, the more important the improvement process itself becomes. A higher score isn't enough.
Organisations need to know what changed, whether the improvement survives outside the environment that produced it and whether they can return to the version they trusted before. EM360Tech will continue following how that shift changes AI architecture, governance and the role humans play as increasingly autonomous systems begin participating in their own development.
Comments ( 0 )