Datadog, the monitoring and security platform for cloud applications, today announced the launch of Datadog Observability Pipelines — a new product that enables organisations to take greater control of their data so they can reliably scale their observability practices. Datadog Observability Pipelines is powered by Vector, an open source and high-performance framework for building telemetry pipelines.
Organisations today are tasked with managing many applications, which are powered by hundreds of services and result in the creation of trillions of logs, metrics and traces per month. The vast amount of data involved and the resulting complexity at both the application and organisation levels can lead to reliability issues, technology lock-in, security risks and runaway costs.
Datadog Observability Pipelines provides customers with a unified view to control and monitor the flow of all their infrastructure and application metrics, logs and traces. Users can now seamlessly collect, enrich, transform, sanitise and route observability data from any source to any destination — before data leaves their environment. This unified view gives enterprises enhanced visibility into how much they are spending and where, what tools they are leveraging and who has access to what data. This enables them to more precisely manage costs, reduce technology lock-in, improve compliance and standardisation of data quality and ultimately scale their observability practices.
“We want to empower our teams to use all of our systems and tools while staying within budget. Observability Pipelines give us flexibility over how our data is processed and where it ends up, so our teams are taking action on the right information as we continue to scale,” said Andreas Kasprzok, Observability Tech Lead at Blockfi.
“As the amount of telemetry continues to grow at an organisation, teams are often completely overwhelmed—if they’re not blind to—the runaway costs, reliability and compliance risks that come from a lack of visibility into infrastructure and application data,” said Zach Sherman, Senior Product Manager at Datadog. “We built Datadog Observability Pipelines to give organisations a powerful way to take back control of their data, without compromising visibility for engineering, security and SRE teams.”
Datadog Observability Pipelines helps IT and security teams in their goals to affordably manage and scale observability with complete flexibility and control over how their logs, metrics and traces are collected, transformed and routed.
This helps organisations:
- Control Costs: Aggregate, filter and route all observability data based on use case without compromising visibility
- Simplify Migrations and Reduce Lock-In: Orchestrate and monitor data processing from any source to any destination in one unified view
- Protect Sensitive Data: Filter, redact and monitor sensitive data before it leaves your network in order to better meet compliance requirements
- Enforce Data Quality: Standardise the format of logs, metrics and traces to improve observability across teams
- Scale with Confidence: Scale seamlessly with a product powered by Vector, a vendor-agnostic, open source project and engaged community with millions of monthly downloads that is deployed in production by enterprises processing petabytes of data every month
- Easily Collect and Route Data: Observability Pipelines comes with more than 80 out-of-the-box integrations so organisations can quickly and easily collect and route data to any of the tools their teams already use, without disrupting existing workflows
