Soda, the provider of open-source data reliability tools and cloud data observability platform, has released Cloud Metrics Store, providing advanced testing-as-code capabilities to enable data teams to get ahead of data issues in a more sophisticated way than ever before.
Available to all users of Soda’s Open Source (OSS) tools, Cloud Metrics Store captures historical information about the health of data to support the intelligent testing of data across every workload.
Without a clear strategy to monitor data for quality issues, many organisations fail to catch the problems that can leave their systems exposed and can result in serious downstream issues. Inspired by modern software engineering principles, Soda is giving data teams the tools to create a culture and community of good data practice through a combination of the Soda Cloud Data Observability Platform and its OSS data reliability tools, built by and for data engineers.
Also Read: Are You Keeping A Tab On The Cloud?
The Soda global data community already counts Disney, HelloFresh, and Udemy as major contributors to have deployed Soda’s data reliability tools.
With this latest OSS release, Cloud Metrics Store gives data and analytics engineers the ability to test and validate the health of data based on previous values. These historical metrics allow data tests to use a baseline understanding of what good data looks like, with any bad data efficiently quarantined for inspection before it impacts data products or downstream consumers.
Alerts are sent via popular on-call tools or Slack, so that data teams are the first to know when data issues arise, and can swiftly resolve the problem.
Soda’s data reliability tools work across the data product lifecycle. This means that it is straightforward for data engineers to test data at ingestion using Soda, and for data product managers to validate data before it is consumed in tools such as Snowflake.
All checks can be written “as-code” in an easy-to-learn configuration language. Configuration files are version controlled, and used to determine which tests to run each time new data arrives into a data platform. Soda supports every data workload, including data infrastructure, science, analysis, and streaming workloads, both on-premise and in the cloud.
“It’s advantageous for data teams to unify around a common language that allows them to specify what good data looks like across the data value chain from ingestion to consumption, irrespective of roles, skills, or subject matter expertise. Most data teams today are organized by domain, and when creating data products, they often depend on each other to provide timely, accurate, and complete data,” said Maarten Masschelein, CEO, Soda.
“For this reason, we are delighted to release Soda Cloud Metrics Store to all users of our OSS tools, as it represents another important milestone in our mission to bring everyone closer to trusted data. Cloud Metrics Store helps data teams to be explicit about what good data looks like, enabling agreements to be made between domain teams that can be easily tracked and monitored, giving data product teams the freedom to work on the next big thing.”
