Robot Training Has a Real-World Reset Problem

Imagine teaching a robot to pick up a bottle and put it in a container. It takes a few tries, but eventually the robot gets it right. You put the bottle back where it started, move the robot back into position, and let it try again. After enough practice, the robot can complete the task correctly almost every time. But what happens when nobody puts the bottle back? 

But in a real working environment, things don't always stay where they're supposed to. Objects get moved, equipment changes position, and the robot's next task might start somewhere completely different from where the last one did. 

This creates a problem for robot reinforcement learning, especially when the training process relies on putting everything back in its original position between attempts. New research from National Taiwan University is looking at what happens when those resets aren't available, and what it could mean for training robots to handle the changing conditions they'll eventually have to work in.

Why Robot Training Depends on Resets

There are several ways to teach a robot something new. Some learn by watching demonstrations, while others practise tasks through trial and error. One method researchers use is reinforcement learning, which allows a robot to improve its behaviour based on feedback from its actions. 

Rather than programming every movement, developers give the robot a task and a way to measure how well it's doing. The robot then learns which actions are most likely to help it complete that task successfully. But there's another part of the learning process that's easy to overlook. The conditions the robot practises in can have a big influence on what it actually learns.

How episodic reinforcement learning works

In traditional reinforcement learning, training is often broken down into individual attempts called episodes. Each episode starts with the robot in a particular position, continues as it tries to complete a task, and ends when it succeeds, fails or reaches a set limit. The robot gets feedback based on what happened, which helps it work out what to do differently the next time. 

For example, a robot may be learning to put a bottle on a shelf. It might receive a reward for getting the bottle into the right position. If it drops the bottle or misses the shelf, it can use that feedback to improve its next attempt. Before trying again, the environment can be reset. 

The bottle goes back to its starting position, the robot returns to where it started, and another episode starts. This is called episodic reinforcement learning. It gives researchers a way to repeat experiments under similar conditions, compare the results and see whether the robot is getting better at the task.

Why resetting a robot's environment matters

In a simulated environment, resetting everything is relatively easy. The computer can put all the virtual objects back where they belong and return the robot to its starting position in moments. Doing the same thing with a physical robot isn't always so simple. Someone might need to pick up dropped objects, rearrange equipment or move the robot's arm back into place. 

Depending on what the robot is learning, this can take time and require someone to step in between attempts. But there's another problem with always starting from the same position. If a robot practises a task over and over under similar conditions, it can become very good at handling those particular conditions. 

What it might not learn is how to deal with the same task when something has changed. For example, a robot might know exactly how to reach for a bottle when its arm starts in one position, but struggle when the arm is somewhere else. It can still recognise the bottle and understand what it needs to do. 

The difficulty comes from working out how to get there from an unfamiliar position. So while resets make training easier to manage, they can also limit the situations a robot gets to practise handling.

What Happens When Robots Can't Start Over?

A robot working through several tasks doesn't always get to start fresh. Every action can change something in its environment, and those changes can affect what happens when it tries to do the next job. 

Researchers have been studying an approach called non-episodic reinforcement learning, where robots continue training without having their environment reset after every attempt. Instead of returning everything to its original position, the robot has to keep working with whatever changes its previous actions have made. And that introduces a few complications.

Why changing starting conditions affect learning

Some modern robotic AI systems use vision-language-action models, or VLAs. These models help robots connect what they see and what they're being asked to do with the physical movements needed to complete a task. They're often trained using demonstrations of robots performing different actions. 

Developers can then build on that training to teach them new tasks, including through reinforcement learning. The problem is that a model's earlier training can influence the positions and movements it's used to dealing with. If most of its training examples involve a robot arm starting in roughly the same place, the model might struggle when the arm ends up somewhere unfamiliar. 

The same can happen when objects are scattered around a workspace instead of being arranged as expected. Researchers call these differences distribution shifts. Essentially, the robot is dealing with conditions that don't quite match what it learnt from. It doesn't necessarily need to learn a completely new task. 

It might just need to work out how to do something familiar when the starting conditions are different.

Why longer task sequences make the problem harder

Now imagine that the robot needs to complete several tasks one after another. Say it's sorting objects into different containers. Moving the first object changes the workspace. The second task starts with that change already in place, and the third starts with whatever happened during the first two. 

The robot might be perfectly capable of completing each task on its own, but struggle when it has to do them in sequence. The October 2026 LADDER robot manipulation benchmark, developed by researchers from INSAIT and NVIDIA, gives us an example of how difficult longer tasks can be. 

The researchers tested robot manipulation across 145 tasks involving up to ten smaller objectives. Their results showed that robots using a learnt policy successfully completed 85.4% of single-step tasks. But that dropped to 26.9% for two-step tasks and 0% for five-step tasks. LADDER isn't specifically about learning without resets. 

It looks at how robots handle longer sequences of actions. However, it helps illustrate why being good at individual tasks doesn't automatically mean a robot can handle several of them in a row.

What CRONOS Reveals About Robot Learning

Researchers at National Taiwan University have been looking more closely at what happens during training when robots can't rely on frequent resets. Their CRONOS research paper, published on 3 November 2026, introduces a simulation benchmark designed to test how pretrained robotic AI models learn when their environment isn't constantly being returned to its original state. 

CRONOS stands for Continual Robotic Operations in Non-episodic Settings. The researchers wanted to understand what happens when robots have to keep learning in an environment where objects move, starting positions change and one task can affect the next.

What happens when resets become less frequent?

The researchers used simulated robot manipulation tasks to see what happened when they reduced how often the environment was reset. Instead of arranging everything neatly before each attempt, they allowed objects to stay wherever the robot's previous actions had left them. The robot then had to continue training under those new conditions. 

They tested pretrained vision-language-action models, including OpenVLA and SpatialVLA, to see how well they handled the changes. And the results weren't particularly encouraging when the researchers simply reduced the number of resets. The models struggled to learn effectively as the environment became less familiar. 

One of the problems was that their previous training had left them more comfortable with certain object arrangements and robot arm positions. As those conditions changed, the models had a harder time working out what actions to take. 

So a robot could struggle with something it already knew how to do, simply because it was starting from a different position or dealing with objects that weren't where it expected them to be. The research shows that giving a robot fewer resets doesn't automatically teach it to become more adaptable. In fact, without changes to the training process, it can make learning much harder.

How researchers are making training less dependent on resets

The CRONOS researchers also looked at ways to help robots learn under these changing conditions, rather than simply expecting them to manage without resets. One approach was to use targeted resets. 

Instead of putting everything back where it started, the researchers could reset the robot's arm while leaving the objects alone. This gave the robot a more familiar position to work from without removing all the changes that had happened in the environment. They also tested curriculum learning, which means gradually making training more difficult as the robot improves. 

For example, a robot could start by practising with regular resets. Once it had learnt the basic task, researchers could gradually increase the time between resets, giving it more opportunities to practise under changing conditions. Another approach involved being more selective about when to reset the scene, rather than automatically doing it after every attempt. 

These changes helped improve learning in the researchers' experiments. They also found that training with fewer resets could help robot policies perform better when they encountered unfamiliar object positions and sequences of tasks. So the answer might not be getting rid of resets altogether. 

It could be learning when they're useful, what actually needs to be reset and how to gradually prepare robots for less predictable conditions. There's an important limitation to keep in mind, though. CRONOS was tested in simulation, not in a working factory. 

The findings don't prove that these methods will produce the same improvements in physical robots, where things like sensor errors, mechanical limitations and safety requirements can introduce additional problems.

What This Means for Real-World Robot Training

CRONOS gives researchers a better understanding of where current robot training methods fall short, along with some possible ways to improve them. But there's still a gap between what works in simulation and what can be trusted in a real working environment. 

For organisations developing or evaluating robotic systems, the next step is understanding how to test those improvements and what kind of evidence to look for before relying on them. 

Why robot training needs more realistic conditions

One of the challenges for developers is deciding how to introduce more difficult conditions without making it harder for the robot to learn in the first place. CRONOS suggests that simply reducing resets isn't enough. Developers also need to think about when to introduce changes, how quickly to increase the difficulty and when a reset might actually help the learning process.

For example, they could start by teaching the robot a basic task, then gradually introduce more varied object arrangements and longer sequences of actions. They could also test different reset strategies to see which ones help the robot improve without becoming too dependent on familiar conditions. 

This gives developers a way to build on what the robot has already learnt while gradually making training more demanding. The next challenge is measuring whether those changes actually improve the robot's performance. 

Rather than relying on a single overall success rate, developers can compare how well it handles different starting positions, how its performance changes across longer tasks and how often it needs help to recover when something goes wrong. That gives them a clearer picture of what the robot has learnt and where it still struggles.

What enterprises should look for when evaluating robot performance

For organisations considering more adaptable robotic systems, the challenge is knowing what to ask when reviewing a vendor's performance claims. Test results don't always tell you how the robot was trained, what conditions it was tested under or how much human help was needed along the way. 

These details can make a big difference when deciding whether the results are relevant to your organisation. There are a few questions worth asking when looking at training results or vendor demonstrations:

• How often was the environment reset? Did every attempt start with the same arrangement, or was the robot tested under different conditions?
• What happened between tasks? Were objects put back where they started, or did the robot have to work with the changes left by its previous actions?
• How well did it handle longer sequences? Could it keep completing tasks successfully without getting a fresh start each time?
• How much human help was needed? Did someone have to move objects, reposition the robot or reset the environment during training or testing?
• Was it tested on a physical robot? If the results came from simulation, is there evidence that the model can perform just as well on real hardware?

It's also important to remember that needing fewer resets during training isn't the same as being able to work without human help after deployment. A robot might learn more effectively under changing conditions and still need someone to step in when something goes wrong during normal operation. 

These questions won't tell you everything about how well a robot will perform in your organisation. But they can help you understand what its reported success rate actually reveals, and what still needs to be tested.

Final Thoughts: Robots Need to Learn Beyond the Reset

There's nothing wrong with resetting a robot's environment during training. It gives researchers a way to repeat experiments, measure progress and help robots learn new skills. The problem comes when a robot gets so used to those familiar starting conditions that it struggles to handle the same task when something changes. 

CRONOS shows that learning with fewer resets isn't necessarily easy, but it also suggests there are ways to make the process work better. The challenge now is finding out how well those methods translate from simulated environments into the places where robots will actually be working. 

As robotic AI becomes more capable, we need to look beyond whether a robot can complete a task. We also need to know what happens when it has to do that task again, under different conditions, without someone putting everything back where it started. 

And as the industry works towards answering those questions, EM360Tech will continue looking at the research and real-world results behind emerging technologies, helping businesses understand what's ready to use and what still has something to prove.