Data lakehouses let businesses store, manage and analyse large amounts of data without having to keep everything in one place. They've become a useful way to bring different types of data together. But as businesses start using them for more complicated tasks, they're running into a problem that traditional databases have had to deal with for years. 

What happens when several related tables need to be updated at the same time? Open table formats like Apache Iceberg and Delta Lake support ACID transactions, which are designed to keep data consistent and reliable when changes are made. But just because a system can safely update one table doesn't mean it can do the same when several tables are involved. 

em360tech image

One update might succeed while another fails, leaving different parts of the business working with information that doesn't match. Some lakehouses can now handle multi-table transactions, but not all of them work the same way. Businesses need to know what their chosen platform can actually guarantee, especially when something goes wrong.

Why Single-Table Transactions Aren't Always Enough

Imagine a business has a data pipeline that updates customer records, transaction summaries and reporting tables. All three contain information about the same activity, so they need to tell the same story. Now imagine the customer records update successfully, but something goes wrong before the transaction summaries can be updated. 

Neither table is necessarily broken. They're just showing different versions of what happened, which could cause problems for any application or team using that information. This is where single-table transactions have their limits. A single-table transaction makes sure that a change to one table is completed properly. 

If something goes wrong during the update, the system can stop an incomplete change from becoming part of the table's official data. This is called atomicity, and it's one of the four ACID properties that help databases keep information reliable. The problem is that when several tables need updating, each one might be treated as a separate transaction. 

So if the second or third update fails, the earlier changes may already have gone through. Multi-table atomicity takes that same all-or-nothing approach and applies it to several tables at once. Either every change goes through, or none of them does. But there's something else to consider. 

What happens if two applications try to update the same information at the same time? That's where transaction isolation comes in. It controls what one transaction can see while other transactions are running. For example, snapshot isolation lets a transaction work with a consistent view of the data, even while other changes are being made. 

It doesn't prevent every possible conflict, though. So it's not enough to know that several updates can be completed together. Businesses also need to understand what happens when those updates overlap with other work.

How Do Multi-Table Transactions Work in a Lakehouse?

A lakehouse uses several different parts to store, find and update data. The table format sets out how the data and information about it are organised. A processing engine does the actual work of reading and changing the data, while a catalogue keeps track of where tables are and which versions are current.

 When several tables need to change together, these parts need a way to agree on which updates belong to the same transaction and when they're complete.

Coordinating commits across tables

When a system finishes making changes, it needs to commit them. That simply means confirming that the changes have been accepted and are now part of the table's current data. With a single-table transaction, this only needs to happen for one table. 

But a multi-table commit needs something that can coordinate several updates and make sure they're all accepted together. For example, the Apache Iceberg REST Catalog Protocol includes an optional way to commit changes to multiple tables at once. However, catalogue servers don't have to support it, and the documentation notes that engines currently commit tables individually. 

So even if a lakehouse uses Apache Iceberg, that doesn't automatically mean it can handle multi-table transactions. Other platforms have found different ways to do this. Databricks, for example, supports transactions involving multiple SQL statements and tables. This allows several related changes to go through together, or be cancelled together if something fails. 

There are some conditions, though. The feature requires supported compute environments, Unity Catalog managed tables and catalogue-managed commits. The system also needs to deal with situations where two transactions try to change the same data. If their updates conflict, it must decide whether both can continue or whether one needs to stop and try again.

Why the catalogue matters

A data catalogue is a bit like an index that helps applications find the information they need. But depending on how a lakehouse is built, it can also help control which versions of that information applications are allowed to use. This means catalogue consistency can play a big part in keeping related tables in agreement.

 A November 2026 research paper called Towards Unified File Management for Data Lakehouse Systems looks at this problem through a proposed system called Raven. The researchers wanted to find a better way to manage all the information a lakehouse needs to keep track of. 

Their approach separates the information needed to manage transactions from the much larger amount used for analytics. Raven keeps the transaction-related information in a database, while still using existing Iceberg and Delta Lake structures for the rest. This allows the research system to support multi-table transactions with snapshot isolation, without having to move all its metadata into a traditional database. 

It's a promising approach, although Raven is still a research project rather than a system that's been proven in everyday business use. DuckLake takes a different approach. Its transaction documentation describes a system that uses a transactional SQL catalogue to manage changes, including all-or-nothing transactions and snapshot isolation. 

Both examples show why businesses need to look beyond whether a platform has a catalogue. What's important is what that catalogue actually does and how it helps keep changes consistent.

What Happens When a Multi-Table Transaction Fails?

It's one thing for a transaction to work when everything goes according to plan. But what happens when a connection drops, another application changes the same data or the system stops responding halfway through? These situations aren't all the same, and they can leave businesses with different problems to solve.

Partial failures and transaction recovery

Imagine a pipeline sends updates to three tables, but loses its connection before the system confirms whether the transaction was successful. The team now has a problem. They don't know whether the changes went through. If they simply run the pipeline again, they could end up with duplicate records if the first attempt actually succeeded. 

But if they assume it worked when it didn't, some of the data could be missing or out of date. A reliable system needs a way to check what happened and recover without making the problem worse. It also needs to deal with situations where different applications are trying to update the same information. 

Depending on the transaction rules, one update might be rejected because another application has already changed something it was using. When that happens, the application needs to know whether it can safely try again or whether someone needs to resolve the conflict first. 

These are the kinds of situations that transaction recovery should account for. If the platform doesn't handle them properly, the engineering team may have to build its own checks and recovery processes to keep everything working.

Safe file deletion and snapshot retention

There's another part of keeping transactions reliable that isn't quite as obvious. What happens to the old data files after an update? Lakehouses don't always replace existing files when information changes. Instead, open table formats often create new files and keep records of which ones belong to the latest version of a table. 

Older files may still be needed by applications that are running, or by someone who wants to look at an earlier version of the data. But keeping every old file forever would waste storage space. So lakehouses have ways to clean up files that are no longer needed. This process is often called garbage collection. 

The tricky part is knowing when it's actually safe to delete something. Apache Iceberg uses snapshot retention rules to decide which older versions of a table should be kept. These rules also protect snapshots that are still referenced by branches and tags. But sometimes a file is being used somewhere else. 

For example, the Delta Lake documentation on table maintenance and cloning explains how file cleanup and retention affect access to older data. Depending on how a shallow clone is set up, it may still rely on files belonging to the original table. If those files are deleted too soon, the clone could stop working properly.

Raven's researchers are looking at ways to prevent this by keeping track of which files are still being used by other parts of the system. For businesses, the important thing is making sure that cleaning up old data doesn't accidentally break something that's still relying on it.

Which Lakehouse Workloads Need Multi-Table Transactions?

Not every task needs the same level of protection. It depends on what the data is being used for and how serious the consequences would be if different tables didn't match for a while. Take a reporting pipeline that updates several dashboards overnight. If one dashboard gets its new data a few minutes after another, that might not cause any real problems. 

The business could also wait until every update is finished before making the new reports available. But imagine an automated system making decisions based on a customer's account activity, eligibility and risk information. If those details come from different points in time, the system could make a decision using information that doesn't fit together. 

In that case, consistent data publication becomes much more important. Some businesses deal with this by managing the updates themselves. They might keep track of which changes have finished, check for differences afterwards or build processes that can safely repeat failed updates. That can work, but it also gives the engineering team more to manage. 

And not every application belongs in a lakehouse in the first place. Systems that need to process large numbers of transactions with very little delay may still work better with a traditional operational database built for that kind of job. The question is really how much inconsistency a workload can tolerate. 

Are you enjoying the content so far?

If a temporary mismatch won't cause serious problems and can be corrected, single-table transactions may be enough. But if several changes must always happen together, the system needs a way to guarantee that.

How to Evaluate Multi-Table Transaction Support

It's easy to find platforms advertising ACID compliance, Apache Iceberg compatibility or support for transactional workloads. But those claims don't necessarily tell businesses what will happen when they try to update several tables at once. 

Before choosing a platform, it's worth finding out exactly what it can do and whether those capabilities work with the applications and systems the business already uses.

Verify the transaction guarantees

Start by finding out which parts of the lakehouse are involved in managing transactions. A feature might work with one processing engine or catalogue setup but not another. Then ask the provider to explain what happens when several tables are updated together and how the system handles other transactions running at the same time. Some useful questions include:

  • Which tables can be updated together? Can one transaction include tables from different catalogues, namespaces or formats?
  • Which processing engines support it? Will the same protections apply when different applications access the data?
  • When do changes become visible? Could another application see some of the updates before the others are finished?
  • How are overlapping transactions handled? What happens when two applications try to change the same information?
  • Are there any restrictions? Does the feature need a particular setup, or are certain operations and environments unsupported?

It's also worth checking whether the feature is ready for general use or still being tested as a preview. Something can technically be supported without being suitable for every production environment.

Test failures, concurrency and recovery

A provider's documentation can tell you how a feature is supposed to work. But testing it with realistic workloads is the best way to see whether it can handle what your business needs. Start with a process that updates several related tables, then deliberately interrupt it at different points. 

For example, disconnect the application while it's committing changes and see whether the system can recover without losing or duplicating information. You can also run several updates against the same data at once to see how the platform handles conflicts. Check whether other applications ever see incomplete results while those updates are happening. 

Performance is another consideration.

  • As more transactions run at the same time, how long do commits take?
  • How often are updates rejected?
  • And can the system recover when something needs to be retried?

The Raven research gives us an example of how much difference the underlying design can make. In one experiment focused on catalogue performance, Raven reached 146.4 transactions per second with 90 concurrent threads. DuckLake reached its highest result of 25.4 transactions per second with 40 threads. 

These results came from a controlled research experiment, not a real business workload. The two systems also reached their highest results with different numbers of threads, so the figures aren't a direct like-for-like comparison. But they do show how the way transactions are managed can affect performance. 

Finally, test what happens when old snapshots expire and files are cleaned up. The platform still needs to protect information that other applications or running processes haven't finished using. These tests should give businesses a much better understanding of transaction reliability than a list of supported features. 

They'll also help show where the platform might need extra work, or where its limitations could become a problem later.

Final Thoughts: Transaction Guarantees Depend on the Whole Architecture

Open lakehouses can handle multi-table transactions, but there's more to it than choosing a table format that supports ACID. The processing engine, catalogue and the way the system manages transactions and files all have a part to play in keeping data consistent. 

Businesses need to understand how those parts work together and whether they can handle the jobs they're expected to do. A lakehouse might be perfectly suited to updating analytical data, but need additional support before it can safely manage several related changes at once. 

And as businesses give lakehouses more responsibility for automated processes and everyday operations, being able to prove that those transactions work properly will become increasingly important. Making those decisions means understanding what the technology can actually deliver, rather than relying on what it's advertised to do. 

That's where EM360Tech helps enterprise technology leaders make sense of the capabilities and trade-offs behind the platforms they're considering.