The Next Frontier for Observability: Data Ownership with OpenTelemetry

Observability is a mindset that lets you use data to answer questions about business processes. In short, collecting as much data as possible from the components of your business — including applications and key business metrics — then using an AI-powered tool to help consolidate and make sense of this huge volume of data gives you observability into your business.

Having observability for your business and applications lets you make smarter decisions, faster. You save time troubleshooting and can proactively solve problems before they impact customers.

Why is Data Key?

Data underlies the entire observability process. Observability is, after all, an evolution of monitoring, and to monitor something, you must collect data about it. As applications move to the cloud and as more modern development practices are adopted, the volume of data expands non-linearly. Adding one microservice could add dozens of cloud compute instances, storage block pools, load balancers, and other miscellaneous components. Data from every one of these services must be collected, retained, and analyzed in order to fully understand the system and what’s happening inside it.

Without this data, or with incomplete data, the benefits that observability can offer will never be fully realized. The process of getting data out of your components and in to the observability system needs to be easy, repeatable, and flexible.

In observability, we call the process of setting up an application and system to emit data “instrumentation.” This instrumentation process is work, no matter what vendor you choose, or how advanced they claim to be—there will always be some manual work required to inform your observability system when a new user transaction has begun, or to tag trace data with information about your users, or for much of the other advanced Observability functionality. 

Why Would I Care About Ownership of This Data?

To try to make this process easy, many vendors suggest you deploy a proprietary agent that will just magic away all the complexity and “give you observability” after you install it. This is tempting, because it seems like less work to start with, and often in the demo it seems like you can get the same kind of results as you might with a different system. However, you’ll quickly determine that proprietary agents only give you the most basic of data out of the box, and that true observability still requires manual instrumentation work. Of course, now, when you do that manual work, you’re locking yourself into their ecosystem and environment.

It’s a bad idea to be dependent on one vendor for anything—but especially not for something that’s critical to your business like an observability system. What if you want to leave that vendor for another vendor? What if you grow beyond their ability to scale and need to? If you’re stuck emitting telemetry data in a format that other vendors can’t consume, you’re stuck with that vendor and reliant on their roadmap and operational ability. 

Data ownership of your telemetry data gives you true independence. You can choose the technology that’s right for your business, no matter where it is in the observability journey, and you can also extend this data for whatever comes after observability as well. With an open data pipeline, you can also process data before sending it to your observability system—what if you don’t want to analyze certain data points, or if there are data residency requirements that apply for some customers, for example? With a proprietary agent, you might not have any control over what’s sent and analyzed.

How Can You Own Your Data?

The easy answer to this question is to embrace OpenTelemetry. OpenTelemetry is the future of Observability, but also provides the most flexible solution for getting observability data from an application into an observability system. OpenTelemetry is an open standard for collection of metrics, traces, logs (and more to come) that can be ingested by almost every vendor (either through the native OpenTelemetry protocol or through an emitter for many other common formats used in observability.)

Adopting OpenTelemetry means you control what data is ingested, what processing is done to it, and where it is emitted. This diagram from the OpenTelemetry project website illustrates the pipeline:

^{(Above image from the OpenTelemetry Documentation, © 2022 The OpenTelemetry Authors, used under CC-BY-4.0 license)}

Why OpenTelemetry, Specifically?

As you can see in the image above, the OpenTelemetry system is endlessly flexible. You can customize nearly everything about how data is collected, processed and emitted. You can create duplicate streams and send parts of the data to multiple vendors, or to a cold storage and a vendor, or practically anything else you can imagine. You can automatically redact sensitive data in the telemetry stream before it gets to the vendor. If you have a vendor that charges for ingest, you can sample on the head-end and save money (but we recommend you don’t sample!).

OpenTelemetry is also extremely popular. It is the 2nd busiest CNCF project (2nd only to Kubernetes). Support for emitting OpenTelemetry is increasingly available in open-source products and projects, and we all know the best instrumentation is instrumentation that you don’t have to do at all. Nearly every major observability vendor contributes to OpenTelemetry, so you can truly feel confident that this technology is the future of the observability movement.

Why Splunk?

Splunk is a huge contributor to OpenTelemetry and is also fully committed to the project — there isn’t a “Splunk format” for Observability data — our products ingest OpenTelemetry natively. Moreover, one of the project’s co-founders is employed by Splunk and has making OpenTelemetry better his primary job responsibility at Splunk. We recognize that a better observability ecosystem is better for everyone — us included — so we treat OpenTelemetry as the way to get data in. We also provide support for setting up OpenTelemetry to our customers, and make sure that as you set up Splunk’s observability system that you’re actually just setting yourself up for long-term observability success.

Why not get started today with a free trial of Splunk Observability Cloud? You can see results in minutes (and yes, even during the demo you’re instrumenting and getting started with OpenTelemetry.) Reduce MTTR, solve problems faster, and keep your engineers and customers happier—the future is in your hands.