Data Mesh is neither socio nor technical, it’s sociotechnical
I recently had a wide-ranging conversation with Carsten Bang on the data culture podcast. Carsten has been a keen observer of modern approaches to data, and I very much enjoyed the discussion.
One topic we tackled was the misconception that data mesh is primarily or solely an organizational concept, a recipe for data culture change. I often hear similar assumptions in the voice of many customers we talk to at Nextdata, “we are working on organizational change first, and we will think about technology later”. What happens next is that I get on a call a year or two later, and the leaders are complaining about why nothing materially changed and the promise of data mesh was not achieved.
The goal of data mesh has always been “rapid data innovation at scale” while embracing “diversity (people and technology), and complexity of organizations” at its heart; speeding up data exchange and facilitating many modes of innovation with data even as the company grows, technology platforms proliferate and the data sprawls. Such leaders find out that their vision of agility at scale doesn’t budge if all they do is try to change culture.
Maybe one reason for this misunderstanding is the way I constructed the data mesh book. It actually has three sections - the first focuses on vision and first principles from the point of view of people and organization, the second discusses the technical architecture that enables the first principles, and the third maps the transformation roadmap required to realize both the organizational and the technological shift. I suspect that most people read just the first section of the book and get tired (it’s 350 pages after all!) and so never make it to the technical architecture.
In fact, in the Part V of the book I offer a model that demonstrates all the necessary components of a transformation, and how to create change through movement - movement on technology platform uplift, business use cases supported by data products, and cultural and organizational behavior and structure.
So, allow me to correct this misunderstanding now: data mesh has always been a socio-technical paradigm. As we discuss in the podcast, I believe that technology and culture must evolve hand-in-hand.
When the technology and the cultural intention are in sync, technology drives culture change, and culture change reinforces the utility of the technology.
We can understand culture through language, behavior, values and beliefs. When data is managed in centralized monoliths, what’s valuable centers on simplistic, least common denominator beliefs and values- like measuring the amount of data collected, the number of pipelines and models built, and the sisyphean pursuit of an all-knowing, black or white “single source of truth.”
Data mesh changes data culture so that organizations value other attributes and behaviors. When an organization adopts data mesh, these values become shared by all the teams and are expected from each other, and consequently drive and govern behavior.
For example, analytical data being everyone’s responsibility across all business domains normalizes the expectation that data will be of high quality, and will be useful, accessible and in fact used across the organization. Data mesh enabling cross functional connections and insights creates a cultural value that elevates diversity and sharing of insights.
Data mesh technology exists to create a data-driven organizational culture. Technology and culture go hand-in-hand. They are constantly co-creating and re-creating each other.
This is an obvious constant throughout our human history. Everything from the discovery of fire, to agriculture, to electricity, and now all the way through to AI and beyond- technical innovation always intertwines with cultural evolution, and now ever-more rapidly as our channels for knowledge diffusion across the planet have become near-instantaneous.
It’s never either-or, which is why I’ve always described data mesh as a socio-technical transformation.
{{gear-animation="/components"}}
You can imagine the data mesh as a compound dynamic gear system, where both organization and technology gears are needed to move the whole system. One of the questions that Carsten asked me during the interview was why I decided to start Nextdata, a data mesh technology platform company, instead of continuing to consult and help organizations shift toward data mesh. The reason was that I experienced first-hand the slowness and friction of change using only one gear. The transformation at the scale of data mesh, and the complexity of global enterprises who need such change most, demands a transformational technology, a larger compound gear to accelerate the behavior change. All in service of humanity of course. Though I remain a technologist at heart, but I serve on team human!
Here’s the full Data Culture podcast if you’d like to hear more on this and many other topics.