Splunk as a SOA Consumer

When you think about Service Oriented Architectures (SOA), Splunk doesn’t come to mind first. However, it is important to realize that any entity that is able to consume or produce services is by definition a participant in a SOA. With that said, let me state that Splunk can easily capture and index the output of a web service later used for search.

The next question is what are the use cases. Information that can be captured in a time series manner is ideal for Splunk. For example, suppose a warehouse is using a RFID reader to capture the movement of goods in and out of its facilities. This information usually drives a software business practice, which in turn may have web services to query the current state of what is happening. With Splunk, you could use a scripted input to capture the output of a web service. The script would call a web services client written in any of the usual web service friendly languages and information such as the inventory of purchased goods would be captured in a time series manner every N seconds. After these snapshots in time of purchased goods are indexed, you can then run time delimited searches and trend reports from Splunk Web to provide instance analysis.

This example is just a limited introduction for what can occur with Splunk and SOA. As more and more people deploy SOA, services are available to capture metrics within a corporation. Splunk could be used as a quick and powerful mechanism to capture time series metrics to provide search capable insight into information flow derived via a service. To show that this is a real possibility I’ve created on Splunkbase a weather example and a stock quotes example using public web services advertised from Xmethods. Weather output looks like this:

<?xml version=”1.0″ encoding=”utf-16″?>
<CurrentWeather>
<Location>Nice, France (LFMN) 43-39N 007-12E 10M</Location>
<Time>Dec 09, 2008 – 12:00 PM EST / 2008.12.09 1700 UTC</Time>
<Wind> from the NNW (330 degrees) at 10 MPH (9 KT):0</Wind>
<Visibility> greater than 7 mile(s):0</Visibility>
<SkyConditions> mostly cloudy</SkyConditions>
<Temperature> 46 F (8 C)</Temperature>
<DewPoint> 24 F (-4 C)</DewPoint>
<RelativeHumidity> 42%</RelativeHumidity>
<Pressure> 30.00 in. Hg (1016 hPa)</Pressure>
<Status>Success</Status>
</CurrentWeather>

The Stock quote output is similar in style. Because the output is in XML format and a timestamp is already in the data, this was very easy to capture in Splunk. Feel free to try either example with your own Splunk installation using your own cities for weather and your own stock symbols. Hopefully, it will inspire you to create your own applications using SOA output for indexing into Splunk.

One last note is once XML is indexed within Splunk, any search can be piped to the Splunk xmlkv command. This will automatically create field extractions for you for all the elements in the XML stanza. These field extractions can next be used for your Splunk Reports.