Zero-Copy and Zero-ETL: The Twin Illusions of Data

Executive Summary

• The Twin Myths: Enterprise architecture is captivated by two flawed promises: “zero-copy” queries across platforms and a “zero-ETL” world without data replication. Both are technically and physically unrealistic in complex environments.
• A Format is Not a Network: Shared table formats like Apache Iceberg solve for storage and schema evolution, but they do not eliminate network latency, egress costs, or the physical need to co-locate data for computation (especially for JOINs).
• Replication is a Physical Reality: Data movement is not a failure; it’s a consequence of physics. Any query that joins large, high-cardinality tables from separate platforms (e.g., Databricks and SAP) must move data over a network. BI tools already create countless hidden copies.
• The Real Goal is Reusable Information: The strategic advantage lies not in eliminating copies (a cost play) but in ensuring every copy is trustworthy, consistent, and explainable (a value play). This requires a shared understanding of meaning.
• The Semantic Solution: Instead of chasing physical impossibilities, the correct architecture accepts the need for selective data movement governed by a semantic knowledge layer. This layer unifies the meaning (semantics, identities, relationships) of your data, providing a stable foundation for analytics and AI that survives the chaos of a multi-platform reality.

Introduction: The Architectural Fallacies Sweeping the Enterprise

There’s a seductive story being told in enterprise architecture circles, a tale of two promises. The first is that because all major data platforms can now speak a common language like Apache Iceberg, we can achieve a true “zero-copy” data landscape. The second is that this enables a “zero-ETL” future where data replication is a relic of the past.

This vision of a unified, frictionless data space is powerful. It is also a dangerous misinterpretation of what technology can do. The hard truth is that a shared table format does not magically erase physical boundaries or the fundamental need for computation. Your multi-platform strategy, built on these twin illusions, will not deliver a data utopia. For the most important enterprise queries, those that cross domains, it will fail. This isn’t a matter of opinion; it’s a matter of physics.

Illusion 1: The Myth of Cross-Platform “Zero-Copy”

Let’s be precise. The legitimate goal of a “zero-copy” strategy is to eliminate unmanaged, divergent copies of data that lead to inconsistency and break governance. This is a valid and crucial objective. The illusion is not the goal itself, but the belief that a shared table format like Iceberg allows you to extend this principle to perform complex queries across physically separate platforms without moving data. The reality is that zero-copy only works under one non-negotiable condition: proximity. The compute engine and the storage layer must live within the same logical and physical boundary (e.g., inside a single Databricks environment in one cloud region).

The illusion shatters the moment a query needs to cross a platform boundary. A JOIN operation, the workhorse of analytics, requires that rows from all tables be brought into the same memory space. When your customer data lives in Databricks and your finance data lives in Azure Fabric, the engine has no choice but to pull massive amounts of data over the network.

Think of it like trying to find every person listed in two massive phone books, one for Berlin and one for Hamburg. To find every match, you can’t just shout names back and forth over the phone. The only sane way is to physically ship one book to the other city (replication) and compare them side-by-side. Enterprise JOINs, matching millions of customer IDs to billions of transactions, are the same problem. At that moment, zero-copy ends.

Illusion 2 & The Inevitable Return of ETL

The second illusion, the “zero-ETL” fallacy, is the direct consequence of the first. Because cross-platform zero-copy is a myth for complex queries, the need for data movement doesn’t disappear, it simply gets rebranded. The uncomfortable truth is that replication is everywhere and unavoidable. Even in a perfect Data Mesh, BI tools like Tableau and Power BI routinely extract, cache, and re-load data for performance. Your business is already creating more copies than your architecture admits.

Chasing fewer copies is a cost-reduction tactic; chasing reusable, trustworthy information is a value-creation strategy. The problem isn’t that copies exist; it’s that they are often inconsistent, ungoverned, and opaque. The goal should not be to eliminate data movement but to make it intelligent, minimal, and meaningful. This is, in essence, a modern, more sophisticated form of ETL (Extract, Transform, Load).

The Real Architectural Answer: From Managing Copies to Managing Meaning

If we accept that physics prevents a true cross-platform zero-copy world, the solution cannot be another physical-layer abstraction. The answer is to move up a level: stop trying to manage data copies and start managing data meaning.

This is the role of an Enterprise Knowledge Layer. It does not magically enable zero-copy analytics. Instead, it accepts the physical reality of data movement and provides the semantic governance to make that movement intelligent, consistent, and minimal. It replaces a chaotic landscape of ungoverned, divergent copies with a system where there is one clear source of truth for meaning.

Here’s how it works in practice:

• It Unifies Semantics, Not Raw Data: The Knowledge Layer, built on a formal ontology, materializes the critical identities, relationships, and business rules that define your enterprise (e.g., what a “customer” is, how it relates to a “contract”). This semantic model is a tiny fraction of your total data volume but represents 100% of its business context.

• It Enables Intelligent Query Planning: When a user asks a question, the system queries the Knowledge Graph first to understand the semantic landscape. It’s not guessing at table joins; it’s navigating a map of established business reality.
• It Orchestrates Governed Data Movement: Armed with this semantic context, the Knowledge Layer generates an optimal execution plan. For small queries, this might be simple federation. But for the complex, high-cardinality joins that drive real business insights, it orchestrates the necessary data movement, creating a temporary, fit-for-purpose, and fully governed materialization in the most efficient compute location.

The result is a system that doesn’t fight the physics of data movement but masters it. It ensures that when data is copied, it is done for a clear purpose, based on a shared understanding of meaning, and is fully traceable.

Conclusion: If You Remember One Thing…

Stop chasing the twin illusions of zero-copy and zero-ETL. A shared format does not create a shared architecture. The winning strategy is to embrace physical reality and build a stable, semantic layer of knowledge to govern it. Don’t fight physics, manage meaning. That is how you turn a collection of data islands into a truly connected and intelligent enterprise.