Generate Dashboards for OpenTelemetry Receivers in Splunk Observability

Dashboards as Code. Dashboards From Code!

Imagine you’re working with a new technology and lucky enough to use an OpenTelemetry receiver to bring that data in. What do you do with those  new metrics? Copy and paste all of the metric names into a chart UI? Or maybe if you’re lucky, a Terraform template? But now if you’re using an OpenTelemetry collector receiver you can easily generate a dashboard for Splunk Observability with all of those metrics! Simply point this handy script from the Observability Content Contrib repo at a receiver’s metadata.yaml file and generate a Terraform configuration for the receiver’s dashboard! 

Generated dashboards are a quick jumping off point for customizing a receiver’s metrics and attributes to your own visualization needs. Instead of scouring metric names and copy/pasting them into charts you can start with everything in one place with easily readable line charts to quickly shape into your desired dashboard. Keep in mind, these dashboards are generated and cannot replace the well thought out and curated dashboards already available for various technologies in Splunk Observability Cloud! But, if you’re looking for a quick start for an OpenTelemetry receiver, this utility will get you up and running!

Manage OpenTelemetry Dashboards Like a Pro

Why Terraform? What value do I get from managing my dashboards as code?: 

• Version management of dashboard configurations with source control
• Source control provides a single source of truth for dashboards (not to mention alerts!)
• Centralized control and deployment of dashboards and other assets
• Easily find out when a change snuck into an important asset
• Quickly edit and make changes to many charts, dashboards, and other assets at once

OpenTelemetry already encourages managing configurations as code for the collector. Why not take those same benefits from configuration as code into monitoring as code? Your dashboards, detectors, and other Splunk Observability assets are easily managed with Terraform. Deploying changes from a centralized repository and deployment pipeline can even help tighten up access controls and auditing for API token usage.

Once you’ve generated your OpenTelemetry receiver dashboard configuration and deployed it with Terraform, making changes is easy! Instead of clicking around the interface to remove extra charts, they can be easily removed from the configuration and re-deployed. Changes to many charts, such as adding specific dimensions and math functions, can also be quickly added to many chart configurations in an IDE and deployed with minimal fiddling.

^{Figure 1-1. Over 90 OpenTelemetry receivers are available. How are you making your dashboards?}

Visualize the Future

Your time is precious! Don’t spend it copy and pasting! When you start using a new OpenTelemetry receiver, generate your Splunk Observability dashboard as a Terraform configuration, and get right to work. Decrease your MTTV (Mean Time To Visualization) on metrics!

Excited about observability or OpenTelemetry but haven’t tried Splunk Observability yet? You can sign up to start a free trial of Splunk Observability today! Become an Observability wizard. Transmute metrics with OpenTelemetry. Understand the Golden Signals. You’ll even learn how to communicate context better. All with Splunk Observability!

If you’ve done something cool with Splunk Observability Cloud please checkout the open source repo for Splunk Observability Cloud on Github. We would love to have your contributions!

This blog post was authored by Jeremy Hicks, Observability Field Solutions Engineer at Splunk with special thanks to: Sam Halpern, Doug Erkkila, David Connett, and Adam Schalock