Self-Service Analytics for the Shop Floor [Part I] - Splunk Core Concepts

Despite the hype around predictive maintenance, basic data collection and analysis are still high priorities for manufacturing companies and key criteria for the success of Industrial Internet of Things (IIoT) projects. It is crucial that people who are most familiar with industrial assets, like process or control engineers, have direct access to industrial data. That way inadequate situations such as breakdowns can be resolved quickly. 

It is these people who benefit from having online availability of device or process information and self-service access as they can easily spot trends, deviations or identify important correlations. By obtaining  this information, they are able to react immediately by adjusting settings or be more reactive the next time a threshold has been exceeded by creating an alarm. But there are also situations for which the actual machine behavior is not known and the same data can be used to gain a better machine understanding through the engineers. The results are used to optimize machine operations or to improve the next generation. 

Historically, Splunk has been successful at searching for data and providing predefined reports and dashboards. With the latest Splunk Enterprise release, enhanced metrics support and self-service capabilities has been added, all without the need to write in Splunk’s proprietary SPL language. Both are powerful additions for IIoT Analytics enabling engineers to perform monitoring, diagnosis and troubleshooting tasks. 

In the first part of this blog article, I am going to introduce general Splunk concepts that built the foundation for the desired features and functionalities of a control or process engineer, including data integration from industrial assets, the power of metrics and its flexible ingestion, as well as the Analytics Workspace which is the most important piece for fully supported self-service analytics.

Ingesting Industrial Data

Each industrial IoT deployment has unique characteristics in terms of connectivity. The types of data it uses, and collection methods highly depend on complex and customized machinery. Splunk is used successfully in a variety of deployments connecting to SCADA systems, Historians or ingesting data using protocols like OPC, AMQP or MQTT, etc. 

One of the ways data ingestion into Splunk is made easier is through a growing number of technology add-ons and an increasing partner ecosystem for the Industrial Internet of Things. 

Metrics are Everywhere

Splunk has its roots and development focus in IT and Security. And demand for IT infrastructure monitoring and troubleshooting has driven product investments to combat downtime by monitoring and measuring uptime, performance, and response time of mission-critical applications and the infrastructure they run on. Luckily, the same concepts can be applied within Industrial IoT scenarios in order to reduce downtime of industrial assets.

Most of the data in Industrial IoT deployments is sensor or process data which is usually numeric in nature and might be best stored within Splunk as metric data points. Each metric data point contains a timestamp and one or more measurement fields. Metric data points can also have one or more dimension fields (e.g. asset, location, vendor, measurement type).

Sometimes IIoT metrics can be buried in unstructured or semi-structured log data. The Splunk platform automatically converts log data to metrics data points and then inserts that data into a metrics index. Manual configurations for ingest-time conversions can be applied using transforms.conf and props.conf. For more details, check out “Convert event logs to metric data points“. 

Self-Service Analytics with Metrics

Splunk Analytics Workspace (previously called Splunk Metrics Workspace) provides a user interface within the Search & Reporting app for monitoring and analyzing metrics data without editing SPL. It can be used to create interactive visualizations in the workspace or to perform a variety of analytic functions to gain insight into your manufacturing asset’s health and performance. 

Key features of the Splunk Analytics Workspace include the ability to quickly create multiple time series charts, filter data to focus on certain results, split data into separate time series based on dimensions, overlay historical time periods on charts and set up alerts for external notifications.

The Splunk Analytics Workspace helps to quickly identify any aspects of an equipment that require further investigation, correction or tuning. For example, a tool change not done correctly will result in bad quality or decreased yield and has a direct impact on certain process parameters. But any of these problems can be identified by an engineer and mitigated by creating an alarm using Analytics Workspace without the involvement of IT.

Prior to Splunk 8.0, Analytics Workspace was called Metrics Workspace but has been part of Splunk Enterprise since version 7.3. For versions prior to that there is a seperate download available.

Turn Data Into Doing with a single platform for IT and IoT

Thousands of manufacturing companies around the globe are using Splunk for IT and security. Yet they are not necessarily aware of the powerful capabilities it offers their production and operational technology environments. Organizations aiming to be more data driven and proactive rather than reactive will now benefit from Splunk’s monitoring and analytics capabilities for the industrial Internet of Things delivering real-time insights and a unified view across critical industrial systems.

In the second part of this blog article I will give a technical and practical example on how to add these capabilities using the MQTT protocol.

Happy Splunking, 

Ron