Why Cloud SLAs Can Miss the Slowest Serverless Requests

Your cloud provider promises high availability. Your monitoring dashboard reports healthy response times. You’ve checked with the infrastructure team and everything appears to be working as expected. So why are some customers still seeing unusually slow requests? It’s a problem that can be tough to describe, especially with serverless applications. 

Serverless computing enables businesses to run code without managing the underlying servers. This is because cloud platforms automatically provide the resources required for the code to run. It’s convenient, adaptable and well suited to many modern applications. But not every request is going to have the same level of performance. 

In fact, some of the slowest requests can even occur when the application is already up and running and ready to respond. This poses a challenge for organisations relying on cloud service-level agreements (SLAs) and traditional performance measures. 

A service can still meet its availability commitments even if individual requests take far longer than an application can reasonably tolerate. To understand why, you need to look beyond the availability of the service and at how it actually performs.

Why Serverless Requests Can Be Slow Even When Functions Are Warm

The first thing developers commonly discuss when talking about serverless latency is cold starts. A cold start occurs when a cloud platform must prepare a new execution environment before executing a function. This preparation takes time, so the request that starts it may have to wait a little longer. 

Once it’s running, the platform can reuse the environment for later requests. These are called warm starts, and they tend to be speedier, as the initial setup has already been taken care of. So if performance is suffering from cold starts, then keeping execution environments warm seems like a no-brainer. Unfortunately, this doesn't always resolve the problem.

Tremulant: Latency in Serverless Platforms beyond the 99th Percentile, a paper from October 2026, investigated several million requests on seven different Function-as-a-Service (FaaS) platforms. The researchers observed that some warm invocations had severe tail latencies, with the slowest taking hundreds of times longer than the median latency in the settings they evaluated. 

They also compared their results to four different public datasets spanning 2020 to 2025. The results indicate that excessive warm-start latency isn't unique to a single provider or a recent change in serverless infrastructure. This behaviour is called tail latency. It describes the unusually slow requests at the far end of an application's response-time distribution. 

This is a critical distinction, because a warm environment eliminates the need for a full startup, but it doesn't eliminate all possible sources of delay. Resource contention, platform scheduling, network conditions and application dependencies can affect response times, yet the Tremulant findings don't identify a specific explanation for every severe event. 

The researchers also identified significant differences between systems and configurations. Some performed consistently within commonly measured percentiles before exhibiting substantially bigger delays at the extreme end. The takeaway for enterprises isn't that serverless applications are intrinsically unreliable. 

The thing is, reducing cold starts doesn't guarantee predictable performance, especially when even rare delays might disrupt an application.

Why Average Latency and p99 Don't Tell the Whole Story

Infrastructure teams usually track application latency using averages and percentiles. These are useful metrics, but they can tell quite different stories about the same application. Average latency gives you the overall time it takes for requests to complete. The difficulty is that a good average can disguise a few really slow requests, especially when most requests are fast. 

Percentiles give further detail, explaining how long it takes a given proportion of requests to finish. For example, if an application has a p99 latency of 200 milliseconds, it means that 99 per cent of measured requests finished in 200 milliseconds or less. The one per cent that was left took longer. 

Now it may not seem like a lot, but imagine an application that processes a million requests. If 99 per cent of the requests are answered within the specified period, the slowest one per cent is still 10,000 requests. And p99 doesn’t tell you if those requests took a few more milliseconds or a few seconds. 

That's why researchers are increasingly looking at more extreme measures such as p99.9 and p99.99. These show the response times within which 99.9 and 99.99 per cent of requests finish respectively, allowing teams to analyse behaviour that less detailed measures could miss. 

Tremulant found that latency could increase sharply beyond p99, showing performance variation that might not be visible from traditional reporting. But there’s a catch. The further into the tail you measure, the more requests you need to make the data reliable. 

If an application has relatively little traffic, it might not generate enough observations to provide a trustworthy p99.99 measurement in a short time frame. So the answer isn't simply to track the highest possible percentile. Teams need to understand their request volumes, be able to identify which delays have an operational impact, and select measures that represent those needs. 

That's even more essential when cloud availability numbers say everything's running as it should.

What Cloud SLAs Actually Tell You About Application Performance

Cloud service-level agreements specify what the provider pledges to deliver and the terms under which customers may be eligible for compensation if those commitments aren't met. But an availability SLA isn't necessarily a performance guarantee. For example, each AWS region has a monthly uptime commitment of 99.95 per cent as specified in the AWS Lambda Service Level Agreement. 

It determines availability by measuring qualifying invocation errors across specified measurement periods. The Google Cloud Run Service Level Agreement also offers monthly uptime commitments depending on the service type and region. For Cloud Run services that don’t use GPUs outside of Stockholm, the published commitment is 99.95 per cent. 

It defines downtime based on qualifying infrastructure-related server errors. Neither agreement ensures that every successful application request will complete within a certain number of milliseconds. That distinction becomes relevant when an application takes too long to be useful while returning a successful result. 

A delayed transaction may still be counted as a success in the provider’s availability calculation, even if the customer has abandoned the process. So, enterprises need their own service-level objectives (SLOs). These specify how an application is expected to perform, rather than depending solely on what the cloud provider promises. 

An SLO for an application could state that 99 per cent of checkout requests complete within a specific response-time threshold. Another application that’s running in the background may be able to handle longer delays. The right target depends on the workload, its users and the implications of waiting. 

And once those expectations are set, infrastructure teams need a way to monitor whether they're meeting them.

How to Measure the Serverless Latency That Matters

The first problem with serverless performance monitoring is determining what your measurements actually include. Cloud providers give helpful indicators of performance, but those numbers don’t necessarily account for the entire time an application user spends waiting. 

For example, AWS Lambda’s CloudWatch metrics documentation states that its Duration metric captures the time spent processing an event and doesn't include cold-start initialisation time. That’s valuable for understanding function execution, but it’s not the same as tracking the whole path from an incoming request to the response received by a user. 

A request may pass through an API gateway, authentication services, a serverless function, databases and other dependencies before it's complete. Any delays along that path can affect the final response time. This means teams need to be able to separate the time taken to execute the function, platform-related delays and end-to-end application latency. 

A practical approach to serverless monitoring should take into account several areas:

• Latency distributions: Measure the appropriate percentiles alongside averages to find requests that are exceptionally slow.
• Warm and cold invocations: Differentiate startup-related delays from requests handled by existing execution environments.
• End-to-end response times: Follow a request from start to finish through gateways and downstream services.
• Concurrency and scaling: Look at how performance varies as traffic grows or as additional execution environments are established.
• Request-level tracing: Link sluggish requests with the services and operations involved, where tracing data is available.

These metrics help teams identify where delays are happening. But being able to spot an unusual request is only useful if the monitoring system can tell the difference between real performance problems and normal variation. In October 2026, researchers published Benchmarking Deep Learning Models for Anomaly Detection in Real-World Cloud Telemetry, which looked at the performance of several detection models against real-world IBM Cloud telemetry. 

The researchers found differences between models in the speed at which they detected anomalies and their rate of false alarms. Some found problems earlier but had more false positives. 

Other approaches offered a better balance. The study didn’t specifically address serverless tail latency, but it does highlight an operational challenge: identifying unusual behaviour promptly without triggering so many alerts that teams struggle to find the real issues.

For serverless apps, this involves defining alert thresholds against stated performance objectives and examining repeated or consequential delays, rather than treating every unexpected measurement as an incident. When teams can determine where performance is falling short, they can begin to decide whether adjustments are worth implementing.

What Enterprises Can Do About Extreme Serverless Latency

There isn't one single configuration change that removes all the sources of serverless latency. Depending on the cause of the delay and how much variation the application can tolerate, different approaches may be needed. Begin by establishing an acceptable response time for each critical workload. 

A customer-facing payment process may have quite different needs from an overnight reporting operation, even if they’re both running on the same serverless platform. These requirements should guide performance testing, especially under realistic traffic conditions. Testing a few requests in a quiet environment doesn't necessarily tell you what happens during traffic surges or sustained demand. 

From there, teams can study the causes of sluggish requests and decide on an optimisation approach. Providers have mechanisms for keeping execution environments warm when cold starts are a real problem. AWS Lambda offers provisioned concurrency, which means execution environments are ready to serve incoming requests without waiting for the typical on-demand startup.

Google Cloud Run has minimum instances so businesses can keep capacity ready rather than always scaling from zero. Both options can alleviate startup-related delays, although keeping capacity available adds expense. They also don't promise that all warm requests will complete within a given response time. 

If delays occur during warm execution, teams should look into other possible causes. These can include inefficient application code, slow database queries, network dependencies, resource limits or platform behaviour that’s not obvious to customers. The latter is particularly tough with managed serverless offerings.

Enterprises don't control the underlying infrastructure, so they may need to compare findings across configurations or providers to establish whether the platform is contributing to the delays. This is where the Tremulant findings are important for infrastructure planning. 

The research shows that extreme latency behaviour can differ between serverless platforms, so businesses shouldn't assume equal performance from services with similar functionality. For applications with strict response-time requirements, teams should test representative workloads across the deployment choices they're considering. 

That can include comparing different serverless configurations, seeing if reserved capacity provides enough of an improvement, or looking at containers and virtual machines if more control over execution resources is necessary. Of course, moving away from serverless comes with its own responsibilities and costs. 

A new architecture might improve predictability for a given workload, but it could also increase operational complexity. The choice therefore boils down to whether the application can satisfy its performance needs reliably and affordably. 

If the slowest requests are still beyond those limits after reasonable optimisation, that should influence the architecture decision before the application becomes harder to change.

Final Thoughts: Serverless Performance Needs Its Own Measures

A cloud service can be available, its average response time can look good, and most requests can go through without difficulties. But the occasional very slow request might still be a serious issue for apps that rely on consistent responsiveness. 

The recent research on warm-start tail latency explains why organisations shouldn’t rely on cold-start optimisation or provider availability commitments alone to understand serverless performance. Instead, infrastructure teams need to define what acceptable performance looks like for their apps, assess the entire request experience and examine delays against those expectations. 

That's not to say serverless is the wrong choice. It just means the decision must also take performance variation into account, not just scalability, convenience and cost. For companies thinking about their cloud infrastructure, the question is whether their current performance indicators reflect what customers are actually experiencing. 

It's a consideration to bear in mind as serverless becomes part of wider infrastructure decisions, and one EM360Tech continues to investigate through its coverage of cloud architecture, application performance and enterprise IT.