Why Data Mesh Failed the First Time and How We Fixed It

The PowerPoint version

About two years ago I stood in front of the architecture review board at a global logistics company and presented our data mesh strategy. Beautiful slides. Zhamak Dehghani's principles quoted correctly. Domain ownership diagrams. Federated governance model. The board approved it. Six months later the thing was dead on arrival.

Not officially dead. Nobody sends a Slack message saying "the data mesh has failed." What happens is quieter. Domains stop attending governance meetings. The self-service portal has 11 registered users, 3 of whom are on your team. Business stakeholders keep emailing the central data team for extracts. The architecture is correct on paper and useless in practice.

I'm writing this because the second attempt worked. 60% improvement in data team productivity. Time to insight from weeks to days. 70% improvement in data quality scores. 15 business domains publishing and consuming data products. The difference wasn't the theory. The theory was fine both times. The difference was treating data products like actual products.

What went wrong the first time

We led with org design instead of product design. The first attempt spent twelve weeks defining domain boundaries, RACI matrices, and governance committees. Lovely diagrams of who owned what. What we didn't do was build a single data product that anyone wanted to use. The domains nodded along in workshops then went back to their day jobs. Why wouldn't they? We'd given them a reporting structure, not a tool.

We also confused publishing data with creating value. Several domains did produce datasets. They registered them in a catalogue. The catalogue grew. Usage didn't. A CSV in a catalogue with a README is not a product. It's a dump with documentation.

And we underestimated the infrastructure gap. We told domain teams they owned their data, then gave them nothing to own it with. No self-service provisioning. No quality monitoring. No automated lineage. We expected 12 domains to independently build the same plumbing the central team had spent years assembling. I still wince thinking about this one. It was either naive or lazy, and looking back it was probably both.

What changed

The second attempt started from a different question. Instead of "how do we decentralise data ownership?" we asked "what would make a domain team actually want to publish a data product?"

The answer was not exciting.

Data products got SLOs

Every data product got a service-level objective. Freshness, completeness, schema stability, query latency. Not aspirational targets written in a wiki -- measured SLOs with dashboards the domain team and their consumers could both see. When the supply chain domain published a shipment-status product, consuming teams could check whether it was meeting its freshness SLO before they built anything on top of it. Trust went up. Adoption followed. Simple as that.

Domain ownership came with a platform

We built self-service infrastructure that let domain teams provision, test, deploy, and monitor data products without filing tickets. Automated data lineage so they could see downstream impact before making changes. Quality monitoring that caught schema drift and null-rate spikes before consumers noticed.

The central platform team's job shifted from building pipelines to building the tools that let domains build their own. This was the expensive bit, and the bit that made everything else work. You cannot hand someone ownership and then make the act of owning things a bureaucratic ordeal. Well, you can. We did, the first time.

Federated governance got teeth

First time around, governance was a quarterly committee that people stopped attending. Second time, it was code. Global standards for naming, versioning, PII classification, and retention were enforced in the CI pipeline. You couldn't publish a data product that violated the standards. The build just failed. No committee required.

The governance group still met, but to review and update the standards rather than manually audit compliance. Governance as policy review scales. Governance as compliance policing does not, and I've now proved that to myself twice.

What actually happened

Over about eighteen months we got 15 business domains publishing data products through the platform. Some were enthusiastic early. Others needed convincing. A few needed their VP to get frustrated enough with the old process to force the issue. Normal adoption curve for any internal platform, which is to say: messy and uneven.

The productivity number -- 60% improvement in data team output -- came mostly from killing duplicate work. Before the mesh, 4 or 5 teams would independently build similar extracts from the same source systems, often without knowing the others existed. With domain-owned products and cross-domain discovery, that duplication collapsed. Teams spent their time on analysis instead of plumbing.

Time to insight dropped from weeks to days. Sounds dramatic, but the baseline was genuinely bad. The old process: request ticket, prioritisation meeting, central team builds the pipeline, UAT, deployment. The self-service path: find the product in the catalogue, check the SLOs, connect. Most of the old calendar time was queue time, not work time. We didn't speed up the work. We removed the waiting.

The 70% data quality improvement is the number I'm proudest of, and the one my team deserves the most credit for. They built automated quality checks into the publishing pipeline -- freshness, completeness, referential integrity, distribution drift. Problems got caught at publish time rather than discovered six weeks later when a finance report didn't reconcile. That lag used to cause a specific kind of organisational misery: nobody could agree whose data was wrong, so everybody assumed everybody else's was.

What I'd tell you if you're starting this now

Start with 2 domains, not 12. Pick one that's enthusiastic and one that's sceptical. If you can make the sceptic productive, the architecture works. If you can only make the enthusiast happy, you've built a demo.

Invest most of your effort in the platform. Everyone does this backwards. Drawing boxes on an org chart is free. Building self-service infrastructure that a domain team trusts enough to depend on is the actual work, and it takes longer than you'd like.

Make governance automated. If your governance model requires humans to review every data product before publication, it will become a bottleneck within three months and quietly ignored within six.

Measure adoption, not architecture. The number of data products in the catalogue is vanity. The number of cross-domain consumers per product is signal.

And be honest when the first version fails. Ours did. The second worked because we stopped trying to implement a theory and started trying to solve a problem. The theory was right. We'd just been building it upside down.