Why Physical AI Needs a Different Compute Strategy

AI is increasingly moving beyond screens, servers and digital workflows into machines that interact directly with the physical world. From industrial robots to autonomous systems, these applications are creating new demands on the infrastructure responsible for processing information and making decisions.

But while much of the AI conversation has focused on building more powerful data centres and faster processors, physical AI introduces a different challenge: ensuring computing resources are available where and when they're needed.

AI Infrastructure Has a Location Problem

For much of the enterprise AI boom, infrastructure discussions have centred on a familiar set of questions: how much computing power do organisations need, how quickly can models process information, and how can data centres accommodate increasingly demanding workloads?

With physical AI, however, these questions need another dimension: where should computing resources be located to support the demands of each application?

When AI operates entirely within digital environments, organisations can often rely on centralised infrastructure to provide the resources needed for model training, inference and data processing.

Physical AI changes that equation. Systems interacting with the real world must process information from sensors, interpret changing surroundings and generate responses that can have immediate physical consequences.

A robot operating on a manufacturing floor, for example, may need to recognise an obstacle and adjust its movements without waiting for a remote system to process the information. An autonomous machine operating in an area with inconsistent connectivity faces a similar challenge.

This doesn't mean centralised computing becomes less important. Training increasingly sophisticated models and running complex simulations can still require substantial computing resources that would be impractical to accommodate on individual machines.

Instead, physical AI creates a more distributed infrastructure challenge. Enterprises must determine which workloads need centralised computing, which can be processed closer to the point of operation and which require dedicated onboard resources.

The distinction matters because the infrastructure supporting an AI model during development may look very different from the hardware required to operate it reliably in the physical world.

How Does Physical AI Create Different Demands Across the Computing Lifecycle

One of the challenges of planning physical AI infrastructure is that the technology doesn't rely on a single computing workload.

Model training, simulation, inference and physical operation each have different requirements. Treating them as interchangeable risks creating infrastructure that performs well in one environment but struggles in another.

NVIDIA's Isaac ecosystem illustrates this distinction through its Three Computer Solution. The architecture separates computing resources into three environments: a training computer, a simulation and synthetic data generation computer, and a runtime computer.

Although this is a reference architecture within NVIDIA's ecosystem rather than a universal industry standard, it demonstrates why physical AI development requires more than one type of computing environment.

Model training requires centralised computing capacity

Training physical AI models involves processing substantial quantities of data to develop the capabilities required for perception, reasoning and action. This can include visual information, sensor inputs, environmental data and demonstrations of how machines should respond to particular situations.

Training increasingly sophisticated models can place considerable demands on processors, memory and data infrastructure. Centralised computing environments provide access to the resources needed to handle these workloads without imposing equivalent hardware requirements on every physical device.

For enterprises, this means existing cloud and data centre infrastructure can remain an important part of their physical AI strategy.

However, the computing requirements of training shouldn't automatically determine the hardware deployed in the machines that eventually use those models.

Simulation requires resources for development and testing

Before an AI model operates physical equipment, organisations need ways to evaluate its behaviour under different conditions.

Simulation provides an environment in which developers can test systems, introduce different scenarios and assess how models respond without relying exclusively on physical hardware.

NVIDIA Isaac Sim and Isaac Lab support workflows involving robot simulation, sensor environments, physics-based interactions and robot learning. These environments can also support synthetic data generation, helping developers create additional training and testing scenarios.

The infrastructure supporting these activities may require powerful graphics and computing resources, particularly when organisations run multiple simulations or evaluate complex environments.

However, simulated performance isn't the same as verified real-world performance. A model that behaves correctly in a virtual environment still needs to be tested against the physical conditions, hardware limitations and unexpected situations it will encounter during deployment.

Simulation is therefore an important part of the development infrastructure, but it isn't a replacement for physical validation.

Inference introduces different hardware constraints

Once a model has been developed and deployed, it needs to process new information and generate outputs. This is called inference, and its infrastructure requirements depend heavily on the application.

A physical AI system might need to interpret camera feeds, process sensor data, recognise objects or generate instructions for a machine.

Unlike training, inference doesn't always require the same scale of computing capacity. However, it can impose much stricter requirements around response times, memory availability and power consumption.

The hardware must be capable of executing the relevant models within the physical constraints of the device. This is where edge AI computing and onboard processing become particularly important.

Why Physical AI Needs Computing Closer to the Action

Centralised infrastructure provides a lot of computing power, but physical machines cannot always rely on remote systems for operational decisions. The speed at which an AI system responds can be influenced by network latency, connectivity disruptions, and the time it takes to transfer data between devices and remote infrastructure.

For applications that involve movement, object manipulation or interaction with people, delays can mean more than a slower software response.

For example, a robot may need to process sensor information locally to recognise an obstacle, adjust its position or execute a time-sensitive movement. These decisions may need to happen even when connectivity to a wider network is temporarily unavailable.

However, not every inference task requires immediate local processing. Some workloads involve higher-level planning, complex reasoning or processing that can tolerate additional communication time. Depending on the needs of the application, these may be better suited to nearby edge infrastructure or to centralised computing environments.

The difference lies in workloads that demand a real-time response versus those that can tolerate a wider processing window.

This is an important distinction. An interesting example is this study from Microsoft Research, Offloaded Inference for Real-World Physical AI Robotics (September 2026).

Researchers assessed mobile robotic manipulation workloads for onboard, edge, and cloud computing setups. They found that performing inference solely on the onboard GPUs could constrain performance, power consumption and battery life.

In their tests, offloading inference to more powerful external computing resources improved performance on several workloads . But the researchers also found trade-offs with network latency, bandwidth and available computing power

This challenges the assumption that all processing must be performed locally by physical AI. Instead, it suggests that the best-performing architecture may distribute inference across different environments, provided the communication requirements and operational conditions support it.

The Hardware Problem Extends Beyond Processing Power

Even when organisations determine which workloads should run locally, selecting suitable hardware introduces another set of challenges. Physical AI devices operate under constraints that are largely absent from traditional data centre environments.

A server can occupy dedicated rack space, receive substantial electrical power and rely on specialised cooling infrastructure. A mobile robot or embedded industrial device has much less flexibility.

Its computing hardware must fit within restrictions involving size, weight, available power and heat dissipation. These limitations affect the amount of processing power available and which models can realistically run on the device.

Memory can be as important as compute performance

Memory capacity and bandwidth can be just as important as processing power when determining the performance of physical AI systems.

AI models require access to data and intermediate calculations while processing information. If available memory cannot supply data quickly enough, additional processing capacity may not deliver the expected performance improvements.

Research into Vision-Language-Action (VLA) models provides a useful example. A March 2026 paper titled Characterizing VLA Models: Identifying the Action Generation Bottleneck for Edge AI Architectures examined the performance of the MolmoAct-7B model on NVIDIA Jetson Orin and Thor platforms.

In their tests, the researchers found that the action-generation stage was memory-bandwidth-bound and accounted for approximately 75 per cent of the model's overall processing latency.

Despite Jetson Thor providing five times the computing throughput of Orin in the reported comparison, end-to-end latency improved by only 1.4 times.

The findings illustrate why increasing theoretical processing capacity doesn't necessarily translate into equivalent improvements in real-world performance.

For enterprises, hardware selection therefore needs to consider the complete workload, including memory requirements, data movement and the frequency at which decisions must be generated.

Power efficiency becomes an operational requirement

Power consumption introduces another constraint. For hyperscale AI infrastructure, the challenge involves supplying sufficient electricity to support large computing environments while improving the efficiency of servers, networking and cooling systems.

Physical AI faces a different set of considerations. A battery-powered machine has a finite energy supply. The computing resources carried by the machine also compete for this limited energy, potentially affecting operating time and the amount of work it can complete between charging periods.

Organisations should therefore evaluate an AI model's power consumption alongside its performance.

The Jetson product family from NVIDIA illustrates how these constraints influence the design of embedded AI hardware. Its modules offer different combinations of processing capacity, memory and configurable power profiles.

For example, the Jetson Orin Nano family offers power configurations ranging from 7W to 25W, while the Jetson AGX Thor series offers configurable power options between 40W and 130W.

These specifications demonstrate the range of computing environments available for edge AI deployment. They also highlight why a hardware platform suitable for one physical AI application may be unsuitable for another.

A fixed industrial machine connected to a reliable power supply has different limitations from a mobile robot operating on battery power.

The relevant question isn't simply how much computing power a device can accommodate. It's how much useful work it can deliver within its operational constraints.

Why Centralised and Edge Infrastructure Need to Work Together

The growing requirements of physical AI don't make centralised computing obsolete. Nor do they mean every enterprise needs to move its AI workloads to the edge. Instead, organisations need to understand how different computing environments can support different stages of the same application.

A centralised environment might support model development, training and large-scale simulation. An edge server located within a facility could handle workloads shared across several machines. The devices themselves could be enabled to process time-critical information locally and execute immediate operational commands. This creates opportunities for hybrid AI infrastructure, but the exact architecture depends on the application.

Imagine an industrial setting with a number of stand-alone machines. Centralised infrastructure could allow model development, software updates and analysis across the broader operation. Some workloads may be coordinated between machines by local edge infrastructure, while onboard systems will handle immediate sensor processing and control.

Such an arrangement can reduce the processing burden on individual devices without requiring them to be entirely dependent on remote infrastructure.

A similar division of responsibilities between AWS and SoftServe was shown in a simulation-first manufacturing demo at Hannover Messe 2026.

The system combined connected robotic equipment and a local edge gateway with a cloud-based AI orchestration layer. The gateway interfaced with the cloud services and the physical devices, permitting local processing of time-sensitive commands and providing continuity of service during temporary connectivity variations.

The demo shows how cloud and edge infrastructure can support different parts of an operational flow. It shouldn't, however, be treated as a universal architecture for physical AI. Other applications may have different latency, connectivity, safety and processing requirements.

An important consideration is that infrastructure decisions should follow the workload rather than a predetermined preference for centralised or distributed computing.

What Enterprises Should Consider Before Deploying Physical AI

For organisations exploring physical AI, infrastructure planning needs to begin before hardware procurement. The first step is understanding what the application actually needs to do and the conditions under which it must operate.

Several considerations can help determine where computing resources should be located.

1. Identify the workload requirements

Separate model training, simulation, inference and operational control. Determine which activities require substantial computing capacity and which need to operate within tighter resource constraints.

2. Specify your latency and connectivity requirements

Which decisions can be processed on the spot and which can permit intercommunication with external infrastructure. Consider what the system will do in case of flaky or absent network connectivity.

3. Assess hardware limitations

Evaluate available memory, processing power, power consumption, heat dissipation and physical space. Think about how the processing requirements affect the battery life and the operating time, especially for the mobile and battery-powered devices.

4. Test performance under real-world conditions

Test models in the same hardware and environmental conditions as they will run in production. You can simulate and find potential problems. But you still have to physically test it to prove it works in operation.

5. Plan for ongoing infrastructure management

Physical AI systems require more than an initial deployment. Organisations need to consider model updates, monitoring, software compatibility, connectivity, security and the management of distributed computing resources.

6. Evaluate existing infrastructure before committing to replacements

Assess whether existing cloud, data centre and edge environments can support different aspects of the physical AI lifecycle. Physical AI may require additional hardware, but organisations shouldn't assume that deployment means replacing their entire existing infrastructure.

These considerations also help organisations avoid selecting hardware based on theoretical performance without understanding whether it can support the actual requirements of the application.

Physical AI Changes What Enterprise Compute Needs to Deliver

As physical AI develops, organisations will need to support workloads operating across centralised computing environments, local edge infrastructure and resource-constrained devices.

The challenge isn't simply supplying more computing power. It's understanding where that power is needed, how quickly it must be available and what operational limitations affect its delivery.

In a recent conversation about the future of AI infrastructure, I was struck by the growing tension between centralised hyperscale AI and the computing requirements of physical AI. While hyperscalers are focused on maximising computing capacity within available power constraints, intelligent machines introduce a different set of considerations. Power remains important, but so do safety, latency and the ability to make decisions reliably in environments where connectivity and available resources cannot always be guaranteed.

This highlights a broader shift in how we need to think about AI infrastructure. The question is no longer simply how much computing power we can provide, but how effectively we can distribute it to support the different demands of AI operating in the physical world.

For enterprise technology leaders, this means looking beyond theoretical computing performance when planning physical AI deployments. Infrastructure decisions need to reflect the actual workloads, operating environments and constraints of the systems being introduced.

Physical AI won't necessarily require organisations to choose between cloud and edge computing. It will require them to understand how different computing resources can work together, where local processing is essential and where centralised infrastructure remains the more practical option.

Ultimately, the next stage of AI infrastructure development won't be defined solely by how much computing power organisations can access. It will also depend on how effectively they can distribute that computing power to support intelligent systems operating in the real world. Stay ahead of the curve with EM360Tech’s AI reporting and ensure you’re part of the future.