AI Is Turning Query Optimisation Into An Economics Problem

For most of its history, query optimisation has been a fairly straightforward idea to explain. A database receives a question, works out the different ways it could answer it, and chooses an efficient route. The goal is to get to the same answer without wasting unnecessary time or computing resources. AI makes that calculation considerably more complicated. 

Once a query can ask a model to interpret text, classify information or make a judgement about meaning, different execution plans may no longer vary only in speed. One might use a larger model. Another might make fewer model calls. A third could cost less but return a slightly different quality of answer. 

Which raises a strange question for something as established as database query optimisation. What happens when the cheapest way to answer a query may also produce a different quality of answer? Database researchers are already working on that problem. So are Google, Snowflake and other data-platform providers bringing AI functions directly into SQL. 

What they're building suggests that query optimisation is starting to become as much an economic decision as a computational one.

AI Changes What A Query Optimiser Has To Decide

Traditional database optimisers already make plenty of choices. A SQL query describes what information the user wants, but there can be several ways to get it. The optimiser might change the order of joins, choose an index or decide how data should move between parts of the system. Crucially, those alternatives are expected to lead to the same logical result. 

The optimiser isn't deciding whether a slightly worse answer would be cheaper. It's deciding which route can produce the required answer most efficiently. Semantic queries change the options available to the query planner. Instead of only asking the database to compare known values, they can ask an AI model to interpret meaning. Google gives examples such as identifying product reviews that complain about durability or support tickets resolved through a workaround. 

That brings model inference into query execution itself. An operation might be carried out using one large language model (LLM), a smaller model, a specialised proxy or some combination of them. The same semantic task could therefore have different costs, speeds and levels of accuracy depending on how it's implemented. 

Research such as Abacus is trying to make those decisions explicit. Developed by researchers including Tim Kraska, Samuel Madden and Michael Cafarella, Abacus searches for implementations of semantic operations according to objectives involving quality, cost and latency, while allowing constraints to be placed on the others. 

The database optimiser is still trying to find an efficient query plan. It's just starting to have many more definitions of “efficient” to choose from.

Every Model Call Changes The Economics Of The Query

The reason this becomes an economics problem is simple. Calling an AI model isn't the same kind of operation as comparing two values in a database table. Google says LLM invocations in the AI-powered SQL workloads it examined can add roughly 10 to 100 times the latency and around 1,000 times the cost of conventional query processing. 

At millions of rows, applying an LLM independently to every record can quickly become impractical. So where the semantic operation appears in a query plan suddenly becomes important. Imagine a query that needs to find customer records matching a normal database condition, then use an AI model to judge whether their written feedback expresses frustration. 

If ordinary SQL can reduce one million records to 10,000 before the AI runs, that's potentially 990,000 model calls the system never needs to make. The trade-off isn't always that neat. Moving an AI operation later can reduce inference, but it may force the database to process much more data along the way. 

Moving it earlier might shrink the rest of the workload, while triggering more expensive model calls. Horrila, a 2026 research system for hybrid queries combining normal relational operations with semantic ones, focuses on exactly this problem. Its optimiser decides where AI-powered operators should sit in the plan by weighing conventional database processing against the cost of LLM inference. 

Across its evaluation, it reported reductions of up to 4.29 times in cost. Even estimating which plan will be cheaper gets harder. Traditional optimisers rely heavily on selectivity, an estimate of how much data will remain after a filter runs. But how do you cheaply estimate how many images match “looks like a crowded station” before asking a model to interpret them? 

Semantic Histograms tackles that problem using embedding spaces, allowing the database to estimate the selectivity of semantic image filters without repeatedly profiling them through expensive model inference. Researchers reported up to an 86 per cent reduction in combined optimisation and execution overhead in their evaluation. 

There is an odd consequence here. The system may now need to spend resources working out how to avoid spending even more resources. Which is why simply tracking tokens after the query has run tells only part of the story. The structure of the query itself can determine how many tokens need to be spent at all.

The Cheapest Query May Not Produce The Best Answer

Cost still isn't the hardest part. Suppose two traditional execution plans perform the same join in different ways. One takes longer, but both should return the same result. Now suppose one semantic execution plan asks a large reasoning model to evaluate every record, while another routes simpler cases through a much smaller model. 

The second option could be dramatically cheaper. It may also behave differently. Google is already exploring this with proxy models in BigQuery and AlloyDB. These are lightweight models trained for a particular query and dataset, which can replace many calls to a full LLM when the task is suitable. 

For a typical one-million-row query in Google's evaluation, proxy models used around 400 times fewer tokens and cut latency by between 30 and 100 times. But Google is equally clear about the limitation. Proxy models are approximations, and there are prompts where the full reasoning capability of an LLM is still needed. 

Snowflake is approaching the problem from several directions. Its research into semantic ranking, model cascades and AI filters asks which technique can achieve an acceptable result within a given budget. The company found there isn't one ranking algorithm that performs best across every workload. That changes what “optimal” can mean. 

A query planner may be trying to minimise cost while meeting a quality requirement. Another workload may prioritise latency while holding accuracy above an agreed level. Somewhere else, quality may take priority and the challenge becomes finding the cheapest execution route that can still achieve it. 

The best query plan is increasingly dependent on what the workload actually needs. And that decision can't come from the database alone.

Query Optimisation Is Becoming A Workload Decision

There is no universal level of AI accuracy that counts as good enough. A team exploring customer sentiment across a large body of reviews may accept some approximation if it cuts the cost of the analysis significantly. A query contributing to a financial investigation, regulatory decision or other high-consequence process may have far less room for error. 

That means semantic query optimisation starts with something data teams already know well, even if the context is new: understanding the workloadccccccc. Before a system can meaningfully trade cost against quality, somebody needs to define what acceptable quality looks like. 

Without that boundary, “cheaper” doesn't tell you whether the optimisation was sensible. It only tells you less money was spent. This is where database architecture begins meeting business requirements in a more direct way. Workloads may need to carry expectations around accuracy, latency and acceptable cost, rather than leaving every semantic operation to pursue the same generic target. 

That doesn't mean a business user should be choosing models or designing query plans. It means the technical system needs enough policy and workload context to know when approximation is reasonable and when the answer needs a much higher standard of confidence. 

The economic question therefore isn't simply, “How little can we spend?” It's closer to, “What is the least expensive way to produce an answer that is still useful for what we're trying to do?”

AI-Powered SQL Is Moving The Problem Into Production

None of this would be especially urgent if semantic query optimisation existed only in database research papers. It doesn't. BigQuery now supports managed AI functions including AI.IF, AI.CLASSIFY and AI.SCORE directly inside SQL, alongside broader generative AI functions. Google says its query processing can optimise these operations to reduce unnecessary model calls and move more work back towards normal database processing where appropriate. 

Snowflake's Cortex AI Functions follow the same broader direction. Its engineers say customers are already running production pipelines that use AI functions across hundreds of millions of rows on daily schedules. In some workloads Snowflake studied, inference accounted for 80 to 90 per cent of total query token spend. 

That has pushed optimisation well beyond choosing a cheaper model. Snowflake has researched filter ordering, model cascades and budget-aware semantic ranking, all aimed at reducing unnecessary inference or choosing the right execution method for the task. 

Databricks likewise exposes AI functions inside SQL and data workflows, allowing model inference to become part of ordinary data processing rather than a separate application layer. Taken together, the direction is fairly clear. 

AI reasoning is becoming something databases can execute, not simply something applications do after retrieving data. Once that happens, its cost belongs inside the data architecture too.

Machine-Generated Queries Raise The Cost Of Getting It Wrong

There is another reason query efficiency may become harder to ignore. For decades, many analytical workloads were naturally limited by the fact that people generated them. An analyst might write several queries while investigating a problem. A dashboard might refresh on a schedule. There was only so much demand humans could create. 

AI applications and agents don't have the same limitation. They can generate queries programmatically, repeat them, change them and run new ones as they work through a task. That doesn't automatically mean every enterprise is heading towards an avalanche of machine-generated SQL. But it does change the potential scale. 

A slightly inefficient semantic query is one problem when somebody runs it a few times a day. The same inefficiency becomes something very different when an automated system runs thousands of variations without anyone manually deciding whether each model call is worth the cost. 

At that point, query optimisation is doing more than improving performance. It's helping determine how much AI reasoning the data platform consumes by default.

Data Teams Need To Define What An Efficient AI Query Means

For database and data-platform leaders, the practical change isn't to start micromanaging every LLM call. Modern optimisers exist precisely because humans shouldn't have to hand-build the best execution plan every time they ask a database a question. But teams do need to give those systems something useful to optimise towards.

What quality does the workload actually require?

Start with the outcome rather than the model. Some semantic workloads can tolerate approximation. Others can't. If a cheaper execution strategy changes the answer slightly, the important question is whether that difference affects how the result will be used. 

This is also why one organisation may need several optimisation policies rather than one universal standard. Exploratory analysis, routine operational processing and high-consequence decisions don't necessarily belong under the same quality threshold.

Where is expensive reasoning actually necessary?

Not every part of an AI-powered query needs AI. Normal relational filtering, embeddings, caching, proxy models and smaller models may be able to reduce the workload before a more capable model is called. The goal isn't to remove reasoning. It's to reserve expensive reasoning for the records and operations that genuinely require it. 

That principle is already visible in current research and commercial systems. The interesting shift is that databases are beginning to make those choices as part of query execution rather than leaving cost reduction entirely to application developers.

Can cost, latency and quality be observed together?

A query taking 30 seconds instead of three is easy to notice. A query costing £10 instead of £1 is also fairly easy to measure. What becomes harder is understanding whether the extra nine pounds bought anything useful. 

Data teams therefore need visibility across the execution decision itself: which semantic operations ran, how much inference they consumed, what models or alternatives were used, how long the query took and what level of quality resulted. Otherwise, organisations risk optimising whichever metric is easiest to see. 

And with semantic queries, the cheapest answer, fastest answer and best answer may increasingly be three different things.

Final Thoughts: The Best Query Plan May Be The Cheapest Answer That Is Good Enough

Database optimisation has always been about finding a better route from question to answer. AI doesn't change that basic goal. It changes what “better” can mean. 

Once semantic query optimisation includes model inference, efficiency can depend on how many times a model runs, which model does the work, where reasoning sits in the query plan and how much answer quality the workload can afford to trade for lower cost or latency. That makes the economic problem unusually close to the technical one. 

Data teams won't always want the cheapest possible query. They won't always want the most accurate option either, regardless of what it costs. What they need is an execution strategy that fits the value, consequence and requirements of the workload. 

As AI-powered SQL becomes more common and machine-generated data workloads grow, those decisions are unlikely to remain something organisations make after the bill arrives. They may increasingly become part of what the database optimiser is expected to decide before the query runs. 

The future of query optimisation may therefore be less about finding the fastest route at any cost, and more about finding the least expensive route to an answer the organisation can actually use. As semantic query processing moves further from research into production data platforms, EM360Tech will continue following how those changes reshape the databases, architectures and operating decisions enterprise Data teams rely on.