Do Open Data Formats Actually Prevent Vendor Lock-In?

Vendor lock-in has always been a concern for enterprises investing in data platforms. Nobody wants to spend years building their data infrastructure around one provider, only to discover that switching to another would mean rebuilding half of it. Unfortunately, by the time that becomes obvious, leaving can be an expensive decision. 

Open data formats promise a way around this problem. Technologies like Apache Iceberg and Delta Lake allow organisations to store data in formats that different tools can understand. In theory, that gives businesses more freedom to choose their technology without being tied to a single vendor. And there's a good reason for the interest. 

Open data formats can reduce vendor lock-in by making data accessible to multiple compatible systems. However, they don't automatically make the entire data platform portable. An enterprise might be able to access its data from another processing engine while still depending on its original provider for security, governance, performance and day-to-day operations.

So, how much freedom do open formats actually provide? And how can enterprises tell whether their architecture gives them a realistic option to switch providers when they need to?

What Open Data Formats Actually Make Portable

To understand what open data formats can offer, it helps to know what they're designed to do. Traditionally, some data platforms have relied on proprietary storage formats or management systems that work best within their own technology environments. This can make it difficult to move data between platforms or use different tools without converting, copying or reorganising it. 

Open file formats help reduce that dependence by giving different systems a common way to store and interpret information. Apache Parquet, for example, is an open column-oriented file format designed for storing data efficiently for analytical workloads. Open table formats build on that foundation. 

Technologies like Apache Iceberg, Delta Lake and Apache Hudi manage collections of data files as tables, with information about their structure and changes over time. The Apache Iceberg project documentation describes how its table format allows compatible processing engines to work with the same underlying tables. 

Instead of requiring every application to maintain its own separate copy of the data, organisations can use different engines against shared storage, provided those engines support the relevant operations and features. This gives enterprises more flexibility when choosing tools for analytics, data engineering and other workloads. 

They may be able to introduce a new processing engine without first moving their entire dataset into another provider's proprietary format. It can also reduce the amount of data duplication needed to support different technologies, although the actual savings depend on how the architecture is implemented. 

But there's an important distinction here. Data portability describes an organisation's ability to access, interpret and use its data outside the system where it was originally managed. It doesn't necessarily mean every application, policy or service connected to that data can move with it. 

A table might be readable by several engines while still relying on one provider to manage permissions, coordinate operations or deliver acceptable performance. That's where the limits of an open data architecture start to become clearer.

Where Vendor Lock-In Can Still Exist

Enterprise data platforms do considerably more than store information. They help organisations find data, control who can use it, run processing workloads and keep everything working reliably. Many also provide automated management features that would otherwise require additional tools or engineering work. 

These capabilities can become closely connected to the platform providing them. And while an open format might make the underlying data accessible elsewhere, it doesn't necessarily remove those connections.

Governance and metadata may not move with the data

Consider what happens when an organisation wants to use an existing dataset through another platform. The new system needs to know where the data lives, how its tables are organised and what information those tables contain. Some of this is stored in the table format's metadata, which helps compatible engines understand the data correctly. 

But enterprises also rely on a much wider set of information to manage their data. A data catalogue, for example, helps users and systems discover datasets and understand how to access them. Depending on the platform, it may also be connected to permissions, business definitions, ownership information and other governance capabilities. 

These aren't always managed in a way that another provider can reproduce automatically. An organisation might have years of access policies, user roles and approval processes configured within its existing platform. Even if another engine can read the same tables, those controls may need to be translated or rebuilt before the data can be used safely. 

Identity management can create similar problems. Permissions linked to one provider's users, groups or security services won't necessarily carry across to another environment. There's also the question of metadata portability. 

Technical table metadata may be available through an open standard, while business definitions, audit records or data lineage information remain tied to separate management tools. The relationship between governance, catalogues and wider enterprise architecture is also an important consideration when comparing data fabric and data mesh. 

When organisations consider replacing a platform, they need to understand which of those management capabilities can continue working independently. Otherwise, they could find themselves with perfectly accessible data that their teams can no longer manage in quite the same way.

Processing and operations can create their own dependencies

The challenges don't stop with governance. A data platform might use an open table format while providing its own processing features, query optimisation, scheduling and automated maintenance. Over time, teams can build important workloads around those capabilities. 

They may use platform-specific SQL functions, integrated services or management features that aren't available elsewhere. Moving the underlying tables won't automatically make those workloads compatible with another system. Performance can be particularly difficult to judge. 

Two engines might both support the same open table format but handle queries, caching and resource allocation differently. A workload that runs efficiently on one platform could need substantial tuning on another. There are operational considerations too. Data pipelines may depend on the provider's scheduling services, monitoring tools or integrations with other applications. 

Replacing those connections can involve more engineering work than changing the storage format itself. None of this means enterprises should avoid integrated platform features. They can make systems easier to operate and reduce the work required from internal teams. The concern is understanding what an organisation is accepting in return for that convenience. 

If the platform handles a critical function, the business needs to know whether that function could be replaced, how much work it would involve and what would happen to its existing operations during the transition. Operational portability is therefore a separate consideration from data portability. 

It describes whether the processes and services an organisation depends on can continue functioning when the underlying technology changes. An open format may give enterprises more choice over where they process their data. But the practical value of that choice depends on what else needs to change before another provider becomes a workable alternative.

Why Open Standards Don't Guarantee Interoperability

There's another complication worth considering. Even when two platforms support the same open standard, they might not support it in exactly the same way. Data platform interoperability means different systems can work together using shared data and compatible operations. 

It's more demanding than simply being able to recognise the same file format. For example, one engine might be able to read a table but not write changes to it. Another might support both operations, provided the table doesn't use features that the engine hasn't implemented. 

The Delta Lake documentation on feature compatibility explains this distinction through its read and write protocols. These specify which features an application must understand before it can safely interact with a table. Enabling certain features can therefore affect whether other tools remain compatible. 

An organisation could adopt a newer capability on its existing platform and later discover that a different engine can't work with the updated table. Open standards are also evolving to address interoperability beyond storage. The Apache Iceberg REST Catalog specification defines a common interface for interacting with catalogues. 

Rather than requiring each processing engine to integrate separately with every catalogue implementation, compatible systems can communicate through a shared protocol. That's useful progress, particularly for organisations trying to avoid tightly coupled data environments. 

However, supporting a common interface doesn't guarantee that every provider offers the same catalogue features, security arrangements or operational behaviour. And interoperability isn't necessarily an all-or-nothing decision. A platform could provide excellent compatibility for the workloads an organisation currently needs, even if some less common features aren't supported. 

The important thing is to understand exactly what a vendor means when it says its platform supports an open standard. A compatibility claim can be a useful starting point. It isn't enough, on its own, to establish how easily the business could move elsewhere.

How to Test Whether a Data Platform Is Portable

It's relatively easy for a vendor to demonstrate that another engine can read data stored in an open format. But that doesn't tell an enterprise much about what would happen if it actually needed to replace the platform. A more useful approach is to test vendor independence against the systems and workloads the organisation already relies on. 

The goal isn't necessarily to eliminate every provider-specific feature. It's to understand which dependencies exist, whether they're acceptable and what options the business would have if its requirements changed.

Establish what can operate independently

Start by identifying the parts of the data environment that would need to keep working if the existing provider were no longer available. This includes more than the storage layer. Organisations need to consider how users would discover data, how access would be controlled and whether their most important workloads could run through another system. A practical data portability test should answer a few questions:

• Can an independent engine read the existing tables without copying or converting them?
• Can that engine safely update the tables and support the features already in use?
• Can another compatible catalogue or service provide the information needed to discover and manage the tables?
• Can the organisation reproduce its required access controls and policies without relying on the original provider?
• Can representative production workloads run elsewhere with acceptable performance, reliability and operating costs?

These tests should use the organisation's actual requirements, rather than a simplified demonstration built around a small dataset. It's also worth separating what works immediately from what needs additional configuration or development. For example, another engine might successfully read the data but require changes to existing queries. 

That could still represent a useful level of portability, particularly if the work is manageable. The organisation simply needs to understand the effort involved. A test should also check whether access remains possible without depending on services controlled by the provider being replaced. 

If another engine still needs the original platform's catalogue or security service to operate, the enterprise hasn't established complete independence from that provider. The results give decision-makers something more useful than a general promise of openness. They show which capabilities can operate independently and which remain connected to the existing platform.

Understand what it would take to leave

Once an organisation knows what can move, the next question is how much the move would cost. Data platform switching costs aren't limited to transferring information or purchasing replacement software. They can include rebuilding integrations, changing applications, recreating governance policies and retraining teams. 

There may also be a period when both platforms need to operate together. During that time, the business could face additional infrastructure costs, duplicated processes and the risk of inconsistent results between systems. 

For organisations with critical analytical or operational workloads, migration planning also needs to account for testing, service continuity and the possibility of unexpected performance problems. These are reasonable considerations for any major technology change. 

But they become especially important when vendor independence is one of the reasons for choosing an open architecture. Rather than asking whether a provider supports Apache Iceberg or another open format, procurement and architecture teams can ask for evidence of what would happen if they needed to leave. 

That evidence might include the supported read and write operations, documentation for exporting configurations, details of catalogue compatibility and the practical steps required to replace platform-managed services. Enterprises can also make portability part of their procurement and architecture reviews. 

Before committing to a platform, they should understand which features are standardised, which depend on proprietary services and what migration support the provider offers. The same questions can be revisited when new features are introduced or major architectural changes are proposed. 

This helps prevent an organisation from gradually building dependencies it never intended to accept. Of course, complete portability won't always be the best business decision. A proprietary feature might offer enough performance, reliability or operational value to justify the additional switching costs. 

Equally, maintaining several independent tools purely to avoid relying on one provider could introduce complexity of its own. The choice depends on the organisation's requirements and its tolerance for future migration costs. 

When comparing enterprise data warehouse tools, portability adds another dimension to the decision, particularly when the technology is expected to support the business for years. What enterprises need is a clear understanding of the trade-offs before those choices become difficult to reverse.

Final Thoughts: Open Data Is Only Part of Vendor Independence

Open data formats have given enterprises more flexibility over how they store and use information. Being able to work with the same data through different compatible technologies is a genuine advantage, particularly for organisations that want to avoid building everything around one provider. 

But open data architecture doesn't automatically mean an enterprise can replace its existing platform without disruption. Governance systems, processing capabilities and operational services can all create dependencies that remain long after the underlying data becomes portable. 

As interoperability standards continue to develop, organisations will have more opportunities to reduce some of those dependencies. They'll still need to assess what each platform supports and how those capabilities fit into their wider architecture. Ultimately, the question isn't whether an open format guarantees freedom from vendor lock-in. 

It's whether the organisation understands the dependencies it's taking on and has a realistic choice about what to do with them. For enterprises reviewing their data strategy, EM360Tech's enterprise data architecture insights offer further perspectives on building environments that can adapt as business and technology requirements change.