We’re aiming high. We aspire to make data decentralization a reality for large, complex organizations. At the same time, we remain focused on simplifying the experience of a complex and fragmented data ecosystem and improving the speed of data products, from creation to value, at scale. To do this, we must think broadly about the ecosystem we intend to impact and be clear about what it will take to enact meaningful change.

The design of our products, Nextdata OS and data product containerization, has been inspired by evergreen technologies that exemplify the five principles guiding our thought process, and they all happen to start with “S”.

Standardized, Simple, Seamless, Small, and Significant.

Standardized 

All the technology that has profoundly changed our lives was built on top of standards. While we have standards for storing or transporting data, we lack standards for building and running data products, and using them in a variety of applications. 

Data product containerization is one of Nextdata's bedrock technologies. We intend to do for the data ecosystem what Docker did for software by providing a standardized framework that abstracts the complexities in the lower layers of the data stack and enables simplicity and interoperability in the upper layers — a data product container.

Data product containers are the standard for building, sharing, and managing mesh-native data products, which are autonomous units of data processing workloads, data, metadata, computational policies and data APIs together connecting data products across a large mesh.

These standards ensure interoperability, consistency, and ease of integration across diverse platforms, making it easier for organizations to adopt data mesh without running into costly interoperability challenges.

Throughout the history of computing, the introduction of standards has been instrumental in unlocking leaps forward. For example, we often take for granted the profound impact of TCP/IP, the foundational networking suite of protocols underpinning the internet. The layered architecture of TCP/IP allows interoperability, vendor neutrality, and seamless communication across large-scale heterogeneous devices. Hiding the complexity of the physical layer frees us to innovate in the digital application layer today.

Simple

The complexity of the data ecosystem has left us with two bad choices: deal with the fragmentation yourself, build your own costly-to-operate data platform, or surrender to a brittle integrated single stack and vendor lock-in.

Nextdata is putting the data ecosystem in a box — using containerization to abstract away the fragmentation and complexity of data infrastructure and unlock a simple, straightforward experience for everyone who touches an organization’s data. 

Data lives in many locations and is managed by different systems, with compute varying depending on the use case. We understand this fact and embrace this complexity, but we are committed to providing a simple experience. We have developed a simple and complete data product lifecycle management. From the moment of creation to discovery by end users, we’re simplifying every step.

The Unix filesystem is one of the most elegant examples of design with simplicity. In Unix, nearly everything is a file, including devices, processes, and inter-process communication. This greatly simplifies the interface and interactions with the system.

Seamless

The data space is populated with various roles, from data engineers and analysts to data scientists and BI professionals. All of them have a set of processes and tools they rely on. 

Nextdata allows stakeholders to continue business as usual, without disruption, by seamlessly integrating with the workflows data professionals rely on. This will be key to driving widespread data mesh adoption and maximizing the benefit and cost savings of Nextdata OS.

Moving from traditional data work to data mesh requires change, there is no doubt about that. We need to revolutionize how data products are built, shipped and shared. But we strive to do that with least intrusive and self-imposing technology shift.

Docker is a great example of such technology. It introduced containerization and portability, changing the way applications were developed, shipped, and deployed. Docker unlocked microservices architecture by simply augmenting how developers built applications. Developers continued to choose the programming language of their choice and developer tools, with an enhancement of adding a plain text docker file to package and ship their applications as containers.

Small

If you know my work, you know that I struggle with monolithic systems. We will not build yet another one. 

Our team is taking extraordinary measures to avoid the bloat which bogs down so many tools. For example, instead of traditional centralized orchestrators or control planes, we manage data product dependencies and runtime behavior with a small kernel loaded into every data product at runtime, wherever it’s hosted.

Nextdata OS kernel handles features like runtime contract matching, orchestrating transformations or provisioning the underlying infrastructure. It is designed to have a minimal footprint. This lightweight approach makes our technology easier to deploy and capable of scaling out without becoming a bottleneck.

Git is a great example of a small and decentralized technology. Unlike its predecessors, centralized version control systems, Git enables every developer with a working copy of the code that is also a complete repository with the full history of all changes. Developers can commit, branch, merge, and view history locally without needing to be connected to a central server. This immensely enhanced productivity and flexibility. Git began its life with about 10,000 lines of code.

“Small is beautiful and beauty is the truth”. Don’t you agree?

Significant

We are not building Nextdata to make a small, incremental change to the status quo. There are already too many narrow technologies in the data ecosystem doing too little to justify their existence. Our product was conceived to significantly improve the speed and reliability of the entire data value stream. Processes that took days or weeks will take minutes. Obscure and undervalued data will become discoverable and useful. Complex, global organizations will finally be able to fully use their data.

Our technology is the distillation of multiple complex data workflows, performed by multiple teams, each using their own technology stack often disconnected from others, into a single concept, a single workflow, performed by a single developer — a data product developer.

CI/CD technologies are a great example of significant change agents. Today, they are one of the most basic technologies used by development teams, and we often forget their significant impact on streamlining software development. CI/CD helped teams shift from big-bang software releases with developers manually handing over build and deployment artifacts to operations teams, to a streamlined process with continuous integration and deployment of smaller, faster change iterations.

These are the principles we rely on daily to decide what to build and how to build it. We take them seriously. If they resonate with you, we would love to hear from you. We are expanding our team and bringing on a wave of customers interested in applying our principles to their data problems. If you’re interested, please get in touch.

If you’re curious to dig further into our principles, see this interview I did with Peter Hanssens, Founder and Principal Consultant of Cloud Shuttle. Back then, there were only four principles, but it’s still worth a look.