Mapping with Splunk | Splunk

I am a map geek. Maps are something I have been fascinated with ever since I was a kid. Maps were fascinating to me because they helped you discover interesting things. I mean who knew that France was just off the coast of Mexico or that the Åland Islands were an archipelago in the Baltic Sea with an interesting history? Don’t worry this is not about far-flung uninhabited territories or history but about maps and Splunk. What does Splunk have to do with maps you might ask? Well pilgrim, Splunk has a lot to offer in the mapping department and in this blog I will show you a few tricks to spice up your reports and dashboards. Maps in Splunk are more than just eye candy. They help you see patterns, summarize data and drill down into interesting events in a whole new way.

In this short entry I will show you how to:

• Add geographic component to events with an IP address
• Use those geographic components to summarize data
• Plot information on a map that you can add to a dashboard
• Lastly summarize the data by country to create a choropleth map 

To demonstrate I will use data from our Buttercup games demo data and in particular the Access Combined web log data. Each event has an IP address, the product category as well as much more data. The data looks like this:

sourcetype=access_combined_wcookie

While there are a number of different ways to add location to data based on addresses, postal codes or other information, a surprisingly accurate way to get to a visitor’s location is using IP addresses. This has two advantages. First, much of the IT data collected will have an IP address and second, Splunk comes with a handy dandy command that will assign latitude, longitude and other geographic data based on the IP address. The command is called iplocation and more info about this command can be found here.

To add location, simply run this search command:

sourcetype=access_combined_wcookie | iplocation clientip

I know, the results are underwhelming since it seems to do nothing. When we look more closely, we can see the real magic in play. iplocation extended the event data with some interesting fields. If I create a table we will see some of what was added:

sourcetype=access_combined_wcookie | iplocation clientip | table lat, lon, Country

Pretty neat! Now we have some data we can use. For example I can now get counts of downloads by country.

sourcetype=access_combined_wcookie | iplocation clientip | stats count by Country

And I can use this to make a nice column chart:

So now I can summarize my data but what if I want to see where these downloads are coming from inside a country? Showing this on a map would be even cooler! No surprise that Splunk has some commands that will help you do this. The first command I will use is geostats.

sourcetype=access_combined_wcookie | iplocation clientip | geostats latfield=lat longfield=lon count

Splunk will now show me a top-level view of where the downloaders are coming from:

Splunk does that in a pretty smart way. It takes the thousands of individual locations and clusters them in smart positioned locations for better analysis. The size of the dot represents the count of downloads. As you zoom in, these clusters will break up and reveal smaller more local clusters. Note: remember to make sure you are selecting the correct visualization (Cluster Map) in the Visualization tab otherwise you won’t see the map. Things get a little more interesting when you summarize the data on a particular field (I mean the blue dots are nice but don’t tell much of a story).

sourcetype=access_combined_wcookie | iplocation clientip | geostats latfield=lat longfield=lon count by categoryId

Instead of just the blue clusters, we can now see pie charts that reveal a breakdown of product category for each cluster. To wrap up, let’s do one last thing and use Splunk to show purchases by country on a map. There is another command, geom, that will do just what we need. geom uses cartographic data that ships with Splunk to show country boundaries on a map as well as summarized data.

sourcetype=access_combined_wcookie | iplocation clientip | stats count by Country | geom geo_countries featureIdField="Country"

Well, clearly the US dominates on the download front and in light of our earlier bar chart probably not the most earth shattering insight but you get the idea. So the next time you are wondering where all those hits to your website are coming from throw a little mapping action into your Splunk and spice up your dashboards.

Happy splunking!

----------------------------------------------------
Thanks!
Robb Bittner