Snowflake DB: Observing a Snowflake From Cloud to Chart

You’ve probably heard something like this before: “It’s a managed service! We don’t need to worry about anything!”  But when it comes to your production workloads, database monitoring is imperative. With the new Snowflake Dashboards and Detectors in the Splunk Observability Content Contributors repository you can start seeing the details of individual Snowflakes.

Snowflake is a popular managed database which can be used to quickly access private data as well as an extensive collection of publicly available data sources. Due to being a fully managed solution, some users will have little interest in the operations of their Snowflake usage. But for those of us in software, Information Technology or service providing industries, the ability to monitor and troubleshoot data sources and their performance is essential!

Hey, What’s Goin' On?

Frequently service owners, developers and IT Ops Analysts will need to answer questions like:

• “Are our queries getting blocked or queued?”
• “Should we upgrade the size of this Snowflake Warehouse?”
• “Have there been an unusual number of failed logins?”
• “Why isn’t our service returning data? Has a query changed and started erroring out?”

With Splunk Observability you can quickly divine answers to these sorts of questions, alert on them and chart trends of other important usage or performance metrics!

To better understand what is happening and where with regards to Snowflake we’ve provided:

• 1x Dashboard Group
• 7x Dashboards
• 11x Detectors for common Golden Signals related and Billing concerns
• Detailed instructions / examples for getting data from Snowflake into Splunk Observability through the OSS OpenTelemetry Collector

Together these easily imported dashboards and detectors provide you with everything from high-level views of Snowflake usage, detailed troubleshooting metrics and trends and even a head start on detecting important Snowflake failure modes.

Aggregates and Specifics

When you want the high-level view, it’s the numbers that matter. The Snowflake Home dashboard provides you these sorts of aggregate overviews of important Snowflake usage metrics like number of Warehouses, Databases, Users, etc., along with quick summaries of daily cost and total storage usage. At a glance, you can quickly get insights on your entire Snowflake footprint.

^{Figure 1-1. Snowflake Home dashboard of aggregates}

For more detailed information other Dashboards focused on Warehouses, Databases and Schemas are available. These help break down the specific concerns associated with Snowflake. Options to slice and dice your data by these same sorts of concerns are available to really zero in on what interests you most! Charts in these dashboards focus more on trends over time to help you identify changes in queries, errors, spilling, queuing and data ingest. For even greater specificity a dashboard focused on Snowflake Queries even provides a table erroring queries and associated details.

^{Figure 1-2. Snowflake Schema dashboard for specifics}

Additionally, dashboards for Cost and Security/Login related concerns are provided to bubble up any divergent trends that may need immediate attention!

^{Figure 1-3. Snowflake Login data}

Detecting Gold Without Digging

It’s no secret that the Golden Signals are helpful metrics to track for various software services. The Detectors for Snowflake provided in the Observability Content Contributor repository also follow this pattern of looking at Latency, Errors, Traffic, and Saturation (L.E.T.S.) along with a couple of billing related detectors. These detectors are a helpful headstart and can have their thresholds easily adjusted to match expectations in your own environment.

Latency

• Long Queries in Small / X-Small Warehouses (usually a sign you may want to upgrade your Warehouse size)
• Queries taking longer than 15 minutes 

Errors

• DB Errors Total
• DB Error Rate
• Login Failure Rate by User

Traffic

• Blocked Queries by Warehouse
• No Queries seen in last X hours

Saturation

• Overloaded Queries
• Queries Queued for over X Seconds

Billing

• Credits used by Warehouse Anomaly Detection
• Warehouses with high Cloud Services Cost

Using these detectors as examples will get you more detail into what issues may be occurring during an incident and if it is related to Snowflake or other elements of your stack!

Snowflake from Cloud to Chart

Now it’s easier than ever to understand Snowflake. Don’t get lost in the blizzard of information and start seeing the details today!

Already an Observability user? Grab these Snowflake Dashboards and Detectors to better understand your Snowflake usage!

Not already an Observability user? Sign up to start a free trial of the Splunk Observability Cloud suite of products today!

This blog post was authored by Jeremy Hicks, Observability Field Solutions Engineer at Splunk with special thanks to Sam Halpern, Henanksha Sainana, and Bill Grant!