Announcing the Donation of the OpenTelemetry eBPF Collector

At Splunk, we’re strongly committed to making the OpenTelemetry project a powerful and complete framework for collecting, generating and exporting observability data. This is why we’re excited to announce the donation of an eBPF-based network telemetry data source, formerly called the Flowmill eBPF Collector, to the OpenTelemetry project. This donation gives OpenTelemetry an out-of-the-box solution for generating detailed network telemetry using eBPF; a low latency approach that exposes a wide range of application and kernel telemetry that extends on the type and scope of data collected by OpenTelemetry today with negligible overheads. It will also provide a launching point for other eBPF efforts within OpenTelemetry.

Thanks to the efforts of a workgroup led by Splunk Architect, Jonathan Perry, together with representatives from AWS, Datadog, New Relic and others to validate, modify and provide a roadmap, this donation is being integrated into OpenTelemetry.

Impact

The core part of this donation is an extension to the OpenTelemetry Collector that enables network observability for microservice environments. When analyzed in a system like Splunk Observability Cloud or others, this data forms an accurate, complete model of network infrastructure and service dependencies and health without requiring any changes to application code or container images  — allowing users to measure the impact of infrastructure and network problems on distributed services in seconds to significantly reduce MTTD.

This is an important extension to OpenTelemetry for two reasons:

• As-is, the donation gives OpenTelemetry an out-of-the-box solution for generating high fidelity network telemetry that recognizes all service-to-service interactions — with no sampling and no service changes. The donation and the data that it generates conforms to OpenTelemetry’s data model and semantic conventions and can be adopted quickly by end users and contributors.
• Longer-term, this framework provides the potential to access a wide range of application and kernel telemetry with extremely low impact to user workloads via eBPF. The newly-formed eBPF group within OpenTelemetry will explore and chart the next steps for this, which may involve extending Splunk’s donation and working with and integrating other eBPF technologies and projects.
   

Splunk’s Commitment to OpenTelemetry

We believe that open-source projects like OpenTelemetry are absolutely critical to making observability an effective practice by allowing teams to go beyond outages and focus on the end-to-end user experience. Allowing data collection from anywhere enables users to unshackle their data collection from their observability strategy — speeding up digital transformation. 

Industry analyst firm Gartner® predicts that by 2025, 70% of new cloud-native application monitoring will use open-source instrumentation, rather than vendor-specific agents for improved interoperability. According to Gartner’s report, Assessing OpenTelemetry’s Impact on Application Performance Monitoring:

“OpenTelemetry promises to do for monitoring what interchangeable parts did for the Industrial Revolution: make replacement components better, faster, cheaper and readily accessible.”*

Splunk has gone all-in on OpenTelemetry. With 64K+ contributions to date and 33 contributing engineers, we are dedicated to the advancement of this project. Not only have we built our entire best-in-class Observability portfolio with open standards in order to help democratize data collection, but we’re actively working to expand the types of data users can collect. 

The donation of the OpenTelemetry eBPF Collector offers a platform upon which the community can extend the type and scope of data collected through eBPF. eBPF is an incredibly exciting technology that provides secure, high-performance kernel programmability and can expose a wide range of application and kernel telemetry that is currently unavailable. To date, this code repository has already received more than eleven thousand commits from 16+ contributors over 5+ years.

Splunk At KubeCon NA 

Donations to OpenTelemetry like the eBPF Collector and SQLcommenter (from Google) are just the beginning, and we are thrilled as a sponsor of KubeCon & CloudNativeCon North America, to show attendees how Splunk customers can utilize these new capabilities with OpenTelemetry in Splunk’s full-stack, OpenTelemetry-native Observability portfolio. Stop by our booth (P20)  where we’ll be demonstrating a collection of tools like real-user monitoring, application performance monitoring and infrastructure monitoring for better visibility as well as synthetics and incident response for better end-users experiences. Splunk’s Observability portfolio provides unified metrics, traces and logs, enabling users to monitor, explore and troubleshoot their entire stack in one platform — providing better quality services to customers.

Find us at KubeCon — either virtually or in-person at booth P20 — and don’t forget to mark your calendars! On Wednesday, October 14th at 3:25pm PT, I will be speaking with Jaana Dogan, Principle Engineer at AWS, chatting about ‘Correlating Signals in OpenTelemetry: Benefit Stories and the Road Ahead’. Add it to your Kubecon agenda here.

More Ways to Learn About eBPF

Want to learn more about how you can start using eBPF to efficiently instrument fine grain network telemetry for a real-time view of all your service-to-service dependencies?

Join us at .conf21 where we'll show you how you can get the most out of your network traffic — without ever having to change your application code or container images. Tune into our session DVO1137A, ‘Channel Your Network Observability Superpowers With eBPF Technology’ and get your free virtual pass to .conf21 today! 

^{*Gartner, Inc., Assessing OpenTelemetry’s Impact on Application Performance Monitoring, Dustin Hassemer, 23 September 2021}

^{GARTNER is a registered trademark and service mark of Gartner, Inc. and/or its affiliates in the U.S. and internationally and is used herein with permission. All rights reserved.}