Kubernetes Orchestration: How It Works And What Enterprises Still Need To Manage

Kubernetes is usually introduced as an open-source platform that automates the deployment, scaling, and management of containerised applications. That definition is accurate. It’s also doing a heroic amount of work. It tells us what Kubernetes is designed to achieve, but very little about what it’s coordinating behind the scenes. 

There are workloads to place, machines to assess, networks to connect, storage to attach, failures to correct, policies to enforce, and changing demand to accommodate. All of this can happen across thousands of containers without someone manually deciding where each one should go.

That’s the real work of Kubernetes orchestration. Kubernetes has also moved well beyond the experimental stage. The Cloud Native Computing Foundation’s 2025 Annual Cloud Native Survey found that 82 per cent of container users were running Kubernetes in production. 

At that level of adoption, understanding orchestration is no longer only useful for platform engineers. It affects infrastructure strategy, application resilience, security, cost control, and the organisation’s ability to operate services consistently at scale.

But automation can create the wrong impression. Kubernetes can coordinate an extraordinary number of infrastructure decisions. It can’t decide whether those decisions reflect the organisation’s priorities, risk tolerance, or budget. That responsibility still belongs to people.

What Kubernetes Orchestration Actually Means

Kubernetes orchestration is the continuous coordination of containerised workloads and the infrastructure they depend on. It decides where applications should run, keeps them available, adjusts capacity, connects services, and works to correct failures when the live environment no longer matches what teams intended.

The word “container” can make this sound smaller than it is. A container packages an application with the software and configuration it needs to run. That gives teams a consistent unit they can move between environments. But once an organisation has hundreds or thousands of those units, someone or something still needs to coordinate them.

An engineer could manually choose a server, start the container, configure its network connection, attach storage, monitor its health, replace it after a failure, and add more copies when demand rises. Doing that for one application is possible. Doing it continuously across a large estate would be a fairly efficient way to make everyone involved reconsider their career choices.

Kubernetes takes on much of that coordination. Teams describe how they want a workload to behave, and the platform works to maintain that condition. It considers available compute capacity, workload requirements, placement rules, service dependencies, scaling policies, security controls, and the current health of the cluster.

This is the difference between deployment automation and orchestration. Deployment automation gets an application running. Workload orchestration keeps coordinating what happens after it starts. That coordination becomes more valuable as the environment grows. A small Kubernetes cluster might support a limited group of applications owned by one team. 

An enterprise Kubernetes environment may span departments, regions, cloud providers, regulatory boundaries, and workloads with very different performance needs. At that point, orchestration isn’t only about containers. It’s about keeping a distributed system aligned with a set of declared intentions.

How Kubernetes Turns Desired State Into Running Infrastructure

Kubernetes works around a simple idea known as desired state. The desired state describes what the environment should look like. A team might specify that an application needs three running copies, a particular amount of memory, access to persistent storage, and a service that makes it available to other systems.

Kubernetes then compares that description with the actual state of the cluster. If all three copies are running, the two states match. If one fails, they don’t. Kubernetes detects the difference and attempts to create a replacement.

This process is called reconciliation. It doesn’t happen once during deployment and then stop. Kubernetes uses control loops that keep checking the environment and correcting differences between what exists and what was requested. It’s a subtle shift from the way infrastructure was traditionally managed.

A conventional script might tell a machine to perform a series of steps. If something changes afterwards, the script won’t necessarily notice unless it’s run again. Kubernetes works from the outcome instead. Teams declare what they want, and the platform keeps trying to preserve it.

That model supports resilience because the system doesn’t rely on every individual component continuing to behave perfectly. Containers can fail. Machines can disappear. Demand can change. Kubernetes keeps reassessing the environment rather than assuming the original deployment will remain untouched.

How the control plane coordinates decisions

The Kubernetes control plane is the part of the platform that receives instructions, stores the cluster’s state, and decides how the environment should respond. It isn’t one giant control process making every decision by itself. The work is divided into several parts:

• The API server is the main point of entry for instructions and requests.
• etcd is where the current state of the cluster and configuration data is stored.
• The scheduler finds suitable nodes for workloads that need somewhere to run.
• The controller manager runs control processes that watch different resources and reconcile them with their desired state.

This separation is useful because scheduling a workload is a different problem from maintaining a replica count or watching the health of a node. Each control process can concentrate on a narrower responsibility while working from the same overall picture. 

Suppose a team requests five replicas of an application. The API server accepts that request, and the desired state is stored. The controller sees that there are only three replicas and requests two more pods. The scheduler then decides which available nodes can host them.

No single component has done all the work. The control plane has coordinated several specialised decisions to produce the requested result.

How nodes and pods execute workloads

The control plane makes decisions, but the applications run on worker machines called nodes. Kubernetes usually runs workloads inside pods, which are the platform’s smallest deployable units. A pod can contain one container or a small group of closely connected containers that need to share networking and storage.

Each worker node runs a component called the kubelet. It looks for workloads assigned to that node, and works with the container runtime to start them and keep them running. This is where the declared plan becomes a running application.

If a pod stops responding, Kubernetes can create another. If a node fails, workloads can be scheduled elsewhere, provided the cluster has suitable capacity and the application has been designed to tolerate the move. That last condition is important.

Kubernetes can respond to failure, but it can’t make a poorly designed application resilient by force of personality. If an application stores essential information inside a temporary container, depends on one irreplaceable service, or can’t handle being restarted, orchestration won’t remove those weaknesses.

It’ll simply operate within the design it has been given.

What Kubernetes Is Actually Orchestrating Every Day

Once a Kubernetes environment is running, several forms of coordination happen at the same time. A workload needs a suitable machine. Its containers need enough resources. Other services need a stable way to reach it. Data may need to remain available after a pod disappears. Capacity has to change without leaving the application unavailable.

These concerns overlap, which is why Kubernetes orchestration is better understood as an operating system for distributed workloads than as a deployment tool.

Workload placement and scheduling

The Kubernetes scheduler decides which node should run each newly created pod. It considers the resources a workload requests and compares them with the capacity available across the cluster. It can also account for rules that limit or guide placement.

For example, an organisation might require a workload to run only on nodes with a particular processor. It may want replicas spread across different machines so one hardware failure doesn’t take down the whole service. Or it may need certain workloads kept apart because they compete for the same resources.

Kubernetes supports these choices through tools such as node selectors, taints, tolerations, affinity, and anti-affinity rules. The names are technical, but the purpose is straightforward. They tell the scheduler where a workload can run, where it should preferably run, or which other workloads it should be placed near or away from.

Resource requests also influence scheduling. These describe the CPU and memory a container expects to need. Resource limits set the maximum it can consume. If those values are unrealistic, scheduling becomes less reliable. Requests set too high can leave expensive capacity unused. 

Requests set too low can pack too many workloads onto the same nodes and create performance problems. So while Kubernetes makes the placement decision, teams still need to give it an honest description of the workload.

Scaling and capacity management

Kubernetes can adjust applications and infrastructure as demand changes, but scaling happens at several different levels. Horizontal scaling increases or reduces the number of pods. If traffic rises, the platform can create more copies of an application. When traffic falls, it can remove the excess.

Vertical scaling changes the resources assigned to a workload. Rather than adding another copy, the system can give an existing workload more CPU or memory. Kubernetes 1.35 moved in-place pod resource updates to general availability, allowing CPU and memory adjustments without recreating the pod. 

This is particularly useful for stateful and batch workloads where disruption carries a higher cost. Cluster scaling changes the number of nodes available underneath the workloads. If there isn’t enough room for new pods, an autoscaler may add more infrastructure. When capacity is no longer needed, nodes can be removed.

These mechanisms can help maintain availability without keeping every environment permanently sized for peak demand. But they don’t remove the need for capacity planning. Scaling policies still need sensible thresholds. Applications need time to start. New infrastructure may not become available immediately. 

Some workloads experience sudden spikes that monitoring systems can’t predict quickly enough. Autoscaling is responsive. It isn’t clairvoyant.

Networking and service connectivity

Pods are temporary by design. They can be created, replaced, and moved between nodes. Their individual network addresses may change as this happens. Applications still need a reliable way to find each other.

Kubernetes services provide a stable network identity for groups of pods. Instead of connecting to one temporary pod address, another application connects to the service. Kubernetes then routes the request to an available pod behind it. This supports service discovery and internal load balancing across distributed applications.

Traffic entering the cluster creates another layer of coordination. Traditionally, many organisations used Ingress resources and an Ingress controller to route external requests. The newer Gateway API provides a broader and more expressive model for managing traffic, roles, and routing policies.

The distinction has become more urgent since the Kubernetes project retired the community-maintained Ingress NGINX controller in March 2026. Existing deployments didn’t immediately stop working, but the project no longer receives new releases, bug fixes, or security updates.

That retirement is a useful reminder that Kubernetes networking doesn’t consist only of built-in components. Production clusters often depend on controllers, plugins, and open-source projects with their own support models and lifecycles.

The cluster may be healthy while one of its essential dependencies is heading towards the end of maintenance.

Storage and stateful workloads

Containers are easy to replace partly because they’re designed to be temporary. Data usually isn’t. A stateless web service can often be restarted somewhere else without much concern. A database, message queue, or transaction system needs its information to survive the loss of a pod.

Kubernetes supports persistent storage through volumes, persistent volume claims, storage classes, and storage drivers. Together, these tools allow workloads to request storage without needing to know every detail of the underlying hardware or cloud service.

StatefulSets provide additional support for applications that need stable identities, ordered deployment, or persistent storage attached to specific replicas. But running stateful applications on Kubernetes introduces harder operational questions.

Teams need to consider backup and recovery, replication, consistency, storage performance, failure domains, and what happens when data and compute move at different speeds. The platform can attach a volume to a replacement pod. It can’t decide whether the data on that volume is complete, current, or recoverable.

State changes the consequences of orchestration. Replacing a failed front-end container may be routine. Replacing part of a database cluster deserves more care.

Why Orchestration Gets More Complicated As Kubernetes Scales

The first Kubernetes cluster is often a technical project. The tenth is usually an organisational one. As adoption grows, different teams begin making different choices about deployment, access, networking, monitoring, and upgrades. 

Business units may need separate environments. Regulations may restrict where data and workloads can run. Resilience requirements can push services across regions or providers. The challenge shifts from operating a cluster to creating a consistent way of operating many environments.

From single clusters to multi-cluster operations

Organisations adopt multi-cluster Kubernetes for several reasons. They may want to separate production from development. Different business units may require stronger isolation. Global applications may need clusters closer to their users. 

Some workloads must remain within a specific country or regulatory region. Others need multiple failure domains so one cluster outage doesn’t interrupt the service. Each reason can be valid. Together, they create a coordination problem above the cluster level.

Policies need to remain consistent. Teams need a common way to deploy applications. Identity and access decisions can’t vary accidentally between environments. Monitoring must show what’s happening across the estate rather than presenting each cluster as an unrelated island.

There also needs to be a clear explanation for why each cluster exists. Without that discipline, organisations can reproduce the same sprawl they were trying to escape. Only now it arrives with more YAML.

Cluster count isn’t a useful measure of maturity on its own. A smaller estate with clear ownership and repeatable operations may be far more capable than a large one held together by local knowledge and good intentions.

Hybrid and multi-cloud orchestration

Hybrid and multi-cloud strategies add another layer. Kubernetes can provide a common way to define and run workloads across different environments. That portability is useful, but it can be overstated. 

A Kubernetes workload may be portable at the orchestration layer while still depending on a cloud provider’s identity services, storage, databases, networking, or security controls. Moving the container doesn’t automatically move the surrounding architecture. Operational consistency is therefore more useful than theoretical portability.

Teams need to decide which policies should remain common across environments and where local differences are necessary. They need consistent standards for deployment, security, observability, upgrades, and cost allocation. They also need to know when abstraction is helping and when it’s hiding a provider-specific dependency someone will have to deal with later.

Kubernetes can create a shared operational language. It can’t guarantee that every infrastructure environment behaves the same way.

Why AI Workloads Are Changing Kubernetes Orchestration

Most traditional applications ask the scheduler for CPU and memory. AI workloads can arrive with a shopping list. They may need a particular graphics processing unit, several connected accelerators, large amounts of high-speed memory, specialised networking, access to model data, and placement close to other parts of the workload.

The hardware is also expensive. Assigning an entire accelerator to a task that uses only part of it can waste significant capacity. Putting the wrong workload on the wrong device can hurt performance while increasing cost. This is pushing Kubernetes towards more workload-aware orchestration.

Beyond CPU and memory scheduling

Dynamic Resource Allocation, usually shortened to DRA, gives Kubernetes a more flexible way to request and assign specialised devices. The core DRA capability reached general availability in Kubernetes 1.34. Kubernetes 1.36 expanded it further with improved support for accelerators and partitionable devices, where one physical resource can be divided between several workloads.

Instead of treating every GPU as a simple countable resource, DRA can describe more detailed requirements and device attributes. This helps the scheduler match workloads with hardware that can actually support them.

For enterprises, the value isn’t limited to AI. The same model can support other specialised resources, including high-performance network interfaces and hardware accelerators. But AI makes the need easier to see.

A scheduler that only knows a workload needs “one GPU” doesn’t know whether that GPU has the right memory, model, configuration, or connection to nearby devices. Better resource descriptions create better placement decisions.

They also create more responsibility for platform teams. Someone still has to define resource classes, select drivers, manage capacity, and decide which workloads get priority when demand exceeds supply.

The rise of inference-aware traffic management

AI also changes how traffic needs to be routed. A normal web request can often be sent to any healthy instance of the same application. An inference request may need a server that has a particular model loaded, enough accelerator capacity, a manageable queue, and the ability to meet a latency target.

The Kubernetes Gateway API Inference Extension was created to support those decisions. It builds on Gateway API and adds routing capabilities designed for self-hosted generative AI and large language model inference.

This moves orchestration closer to the behaviour of the workload itself. The platform isn’t only asking whether an endpoint is available. It can consider whether that endpoint is suitable for a particular model request.

CNCF’s annual survey found that 66 per cent of organisations hosting generative AI models use Kubernetes to manage at least some of their inference workloads. As that use grows, GPU scheduling, model serving, traffic routing, and cost allocation are likely to become normal infrastructure concerns rather than specialist AI problems.

What Kubernetes Does Not Manage For You

Kubernetes is good at following declared instructions. It doesn’t know whether those instructions are sensible. If a deployment requests far more capacity than it needs, the platform can reserve it. If an access policy is too permissive, Kubernetes can enforce that policy with admirable consistency. 

If an autoscaler is poorly configured, it can turn a small traffic increase into a surprisingly educational cloud bill. Kubernetes automates coordination, not accountability. The organisation still needs to decide what should be automated, who owns the rules, how exceptions are handled, and how outcomes are reviewed.

Security still depends on policy and governance

Kubernetes includes useful security controls. Role-based access control can limit what users and services are allowed to do. Network policies can restrict communication between workloads. Admission policies can stop unapproved configurations from entering the cluster. Workload identity and secrets management can reduce reliance on embedded credentials.

None of these controls configure themselves.

Red Hat’s 2026 State of Cloud-Native Security report found that 97 per cent of surveyed organisations had experienced at least one cloud-native security incident during the previous year. Misconfigured infrastructure or services were reported by 78 per cent, making configuration failure the most common incident type in the study.

The broader problem isn’t that Kubernetes lacks security features. It’s that cloud-native environments contain many decisions, and those decisions are often spread across development, infrastructure, security, and application teams.

A secure operating model needs clear ownership of identity, permissions, network access, image sources, admission rules, secrets, and runtime monitoring. It also needs controls across the software supply chain before workloads reach the cluster.

Managed Kubernetes changes where some responsibilities sit, but it doesn’t remove them. A cloud provider may secure and operate parts of the control plane. The enterprise still owns its workloads, access decisions, configuration, data, and application behaviour.

Cost management still requires business context

Kubernetes can scale infrastructure efficiently. It can also scale waste. Resource requests, pod replicas, node pools, storage choices, and autoscaling rules all shape the final cost. The platform sees technical demand. 

It doesn’t understand whether the workload supports a critical customer service, an internal experiment, or a forgotten test environment that’s still enjoying a comfortable amount of memory. FinOps brings financial accountability into these decisions by connecting engineering activity with cost, ownership, and business value.

The FinOps Foundation warns that the elasticity of Kubernetes AI and machine learning environments can create runaway costs without appropriate controls. Its guidance recommends practices such as fractional GPU sharing, specialised node pools, stronger scale-down policies, and clearer allocation of accelerator costs.

For platform and infrastructure leaders, the useful questions go beyond whether a cluster is fully utilised. They need to know which team or product created the cost, whether the resources are producing value, and whether the workload could run more efficiently elsewhere. 

They also need cost data at a level that maps shared infrastructure back to the services consuming it. Otherwise, Kubernetes may optimise placement while the organisation remains unable to explain the bill.

Lifecycle management remains an operational responsibility

Kubernetes itself changes quickly. New versions introduce capabilities, deprecate older interfaces, and eventually remove features teams may still depend on. That means clusters need regular upgrades. Applications, manifests, operators, plugins, and deployment tools must remain compatible. 

Deprecated APIs have to be found before they disappear. Teams also need a tested way to move between versions without turning every upgrade into a bespoke event. Then there’s the wider ecosystem.

Most enterprise Kubernetes environments rely on much more than Kubernetes itself. They use networking plugins, storage drivers, ingress controllers, operators, admission webhooks, Helm charts, and other open-source or commercial components.

Each dependency has its own maintainers, vulnerabilities, release cycle, and support horizon. The retirement of Ingress NGINX shows how easily a widely adopted component can become a lifecycle problem. The project had been trying to address limited maintenance capacity for years before development stopped. 

Organisations using it now have to identify affected clusters, select a replacement, test routing behaviour, and complete a migration without interrupting production traffic. Kubernetes can orchestrate applications. It won’t maintain an accurate inventory of every ecosystem dependency unless the organisation builds a process to do so.

How To Assess Kubernetes Orchestration Maturity

It’s tempting to measure Kubernetes maturity through visible numbers. 

• How many clusters are running? 
• How many workloads have moved? 
• How much infrastructure is now automated?

Those figures may show adoption. They don’t necessarily show control.

Kubernetes maturity is better measured through consistency, visibility, ownership, and repeatability.

A mature environment doesn’t depend on one engineer remembering why a network rule exists. Teams can explain how workloads are scheduled, where policies are enforced, who owns cost decisions, and what happens when a component reaches the end of support.

Deployment and recovery processes work in more than one cluster. Upgrades follow a repeatable path. Developers can use approved workflows without needing unrestricted access to the underlying platform. Security teams can verify that controls are applied rather than assuming they probably are.

The questions leaders ask should reflect that operating reality:

• Are workload and security policies applied consistently across environments?
• Can teams explain why applications are placed on particular infrastructure?
• Are resource requests based on observed usage or copied from old templates?
• Is ownership clear when a workload creates a security, availability, or cost problem?
• Can shared cluster costs be connected to products, services, and teams?
• Are upgrades and dependency migrations tested before production?
• Does monitoring show the health of the service as well as the health of the cluster?
• Can teams recover stateful workloads within an acceptable period?
• Does the platform reduce work for developers, or move complexity somewhere less visible?
• Are exceptions documented, reviewed, and eventually removed?

The last question may be the most revealing. Kubernetes often centralises complexity inside a platform team. That can be a sensible choice because it prevents every developer from solving the same infrastructure problems alone. But centralised complexity still needs enough people, documentation, tooling, and organisational support to manage it properly.

Otherwise, the company hasn’t removed the problem. It has changed who gets paged when something goes wrong.

Final Thoughts: Orchestration Works Best When Responsibility Remains Visible

Kubernetes orchestration begins with a practical promise. Describe how an application should run, and the platform will keep working to maintain that state. It can place workloads, replace failed pods, adjust capacity, route traffic, attach storage, and coordinate resources across distributed infrastructure. 

As AI, hybrid cloud, and multi-cluster environments become more common, those decisions are becoming more detailed and more closely connected to the behaviour of each workload. But Kubernetes can only act on the information, rules, and resources it receives. 

It doesn’t know whether a policy is too broad. It doesn’t know whether an idle workload is worth its cost. It doesn’t know whether an ecosystem component is losing its maintainers or whether a recovery target still reflects what the business needs. Those are organisational decisions.

The enterprises getting the most from Kubernetes won’t necessarily be the ones with the most clusters or the highest level of automation. They’ll be the ones that can still explain what their orchestration platform is doing, why it’s doing it, and who remains accountable for the outcome.

As infrastructure becomes more capable of coordinating itself, responsibility can’t be allowed to fade into the automation around it. It needs to become easier to see. EM360Tech continues to follow the technologies, operating models, and leadership decisions shaping modern infrastructure, helping technology leaders understand not only what platforms can automate, but what their organisations still need to govern.