Over the last few decades, many of us have observed both centralized IT and distributed data teams in action, with the latter often embedded in non-technical business units. These business units often see centralized IT teams as glacial, bureaucratic headaches to work with. And, IT sees data workers embedded in business units as cowhands, wrangling data in unsustainable, non-repeatable ways, on fragile data infrastructure.
How do we leverage the strengths of these two models? How do we get that de-bottlenecked, business-oriented speed of decentralization without throwing security, governance, and sustainment considerations to the wind?
Enter the data mesh. This approach casts aside centralized data warehouses and lakes for a decentralized “mesh” of smaller, interoperable data products that realize specific data needs.
Data products are built by data workers and teams embedded in business units. These people collaborate on a peer-to-peer basis and are each responsible for delivering data on specific subject areas they can grow expertise in. Each data product is built to function as an independent unit and conforms to agreed company standards on governance and security.
The data mesh represents a major cultural and organizational shift that promises to unlock a new level of productivity in data management.
Beyond the cultural and organizational shift required to make the decentralization of a data mesh viable, technology must also shift. For this to happen data platforms need to be more flexible and easier to get started with to enable more types of data workers to effectively build the data mesh.
Today, we introduce Cloud-Native Data Infrastructure, or CNDI (pronounced “Candy”) your custom modern data stack - on demand.
CNDI is the self-serve platform that takes away complexity around infrastructure provisioning and data application configuration, enabling rapid setup of full data product environments. Local data teams can now focus on the business problems in front of them. Start from zero - without necessarily even machines deployed - and have your data product stack ready to go in minutes.
CNDI provides choice and portability in deploying to cloud, multi-cloud, on-premise, or hybrid environments. Open-source licensing for both CNDI and its supported data tools avoids vendor lock-in and enables your data teams to collaborate effectively and thrive in a data-driven environment.
Embrace data mesh with a single platform you can leverage organization-wide. Foster a data-driven culture for the benefit of your employees, customers, shareholders and leadership.
Our mission at Polyseam is to give companies the data foundations they need to scale seamlessly into the future with the newest software and most efficient systems possible.
We are committed to listening to our users and providing the community with the ability to help drive our projects. In the near future, we plan on releasing any community-requested updates and adding as many open-source contributors as we can. CNDI is a community-driven project, so let’s bring new life to data management together.
Polyseam is also busy with the Data Product Registry project, which has the goal of making your data products discoverable, interoperable, and addressable. CNDI will integrate seamlessly with the registry when it launches.
We hope you are as excited about the future as we are!