How To Visualize Business Service Performance with Splunk ITSI

The complex nature of modern digital landscapes means the ability to effectively monitor and understand its impact on your business is not just desirable — it's a necessity. This is where Splunk IT Service Intelligence (ITSI) comes into play. ITSI offers a sophisticated platform for service insights and detailed analytics that can be used by digital operations teams as the first step of a troubleshooting workflow.

In recent years, several solutions have entered the IT Operations and Management (ITOM) market with the promise of complete service discovery and mapping. While the technical component discovery capabilities of such solutions are useful in pre-filling a CMDB, they offer little value in mapping out the business services running on top of the technical assets making it difficult for operations teams to troubleshoot degradations in their environments.

Splunk ITSI stands out in its versatility, capable of adapting to various business models from enterprises with distributed digital assets to cloud-native e-commerce organisations. This adaptability is crucial in an era where customers interact through both digital and in-person channels and expect seamless experiences. By providing service-level visibility across both legacy and modern digital ecosystems, ITSI enables businesses to ensure optimal performance across all fronts.

The above dashboard is an example of an organisation using ITSI to visualize their critical business metrics across e-commerce, shop fronts, and back-end infrastructure that underpins the business services. The health of each component of this organisation’s services is calculated in ITSI by constantly analysing business metrics (top left) in conjunction with average call centre wait times, number of issues raised by customers and performance of applications. However, achieving this state where business operations are correlated with technical performance is a two-step process.

First: Collaborative Workshops for Service Decomposition

Service decomposition is the process of breaking down complex business processes into manageable, logical groups termed 'services'. These services encompass various components of a single business transaction, each represented by specific Key Performance Indicators (KPIs) that signal the health of that aspect of the business. This decomposition is pivotal in understanding and monitoring the intricate web of dependencies and interactions within modern business services.

At the heart of effective service decompositions are collaborative workshops. These sessions need to bring together a diverse group of stakeholders, including business users and technical experts, to discuss and define the key components of their business services and their interdependencies. This interdisciplinary gathering is essential for identifying relevant business KPIs, such as revenue trends and user engagement metrics, and identifying the technical metrics that could impact them such as CPU usage, network bandwidth, or even temperature within data centres.

The workshops typically require at least two 90-minute sessions, allowing for a comprehensive exploration of both the business and technical aspects of the services. During these workshops, stakeholders often confront differing perspectives on how they think the business services operate versus the reality. These discrepancies can lead to robust discussions, out of which clarity and insight tend to emerge. It's not uncommon for these debates to unveil new understanding and alignment on the services' actual functionality and impact on the business.

The uniqueness of each business, from its operational model to its customer engagement strategies, necessitates a tailored approach to these sessions, and, in our experience, operations teams such as Centre of Excellence or Network Operations Centre (NOC) are best placed to facilitate them.

The value of these workshops lies not just in the technical outcome but in the process itself. The dialogue and debate foster a deeper understanding of the business, leading to more accurate and effective service monitoring in ITSI. This is where the true essence of service decomposition comes to life — the intersection of business knowledge and technical capability.

Navigating The Service Decomposition Process 

Here's a step-by-step guide on navigating this process:

1. Identify Business Transactions: Start by pinpointing the critical business transactions that need monitoring. This involves understanding the various components and how they interact to form a complete business service. Start by asking open-ended questions such as, what does the customer expect from our service, what does success look like? How do they interact with our business? How will the business be impacted if this service is not performing how it should?

2. Group Components into Services: Break down these business transactions into logical groups or services. Each service should represent a distinct part of the transaction.

3. Define KPIs for Each Service: Establish Key Performance Indicators (KPIs) for each service. These KPIs are metrics that will indicate the health and performance of the service. Select KPIs that are most relevant to the service's functionality and the overall business objectives.

4. Set Thresholds for KPIs: Determine the threshold values for each KPI. These thresholds help to identify when a service is performing within acceptable limits and when it's not. 

By the end of this exercise, you may have a diagram with logical components and their dependencies such as the one below.

Repeat the process with technical stakeholders and the infrastructure components that support the business services and connect them being mindful that the resultant service tree doesn’t replicate an architecture diagram. The idea here is to create a dependency map that helps understand error propagation in the environment. 

The open-ended queries during service decomposition will help you identify gaps in traditional monitoring approaches. For example, a healthcare provider’s service decomposition, could realize the importance of monitoring printer’s toner levels at the reception since a patient’s signature was a critical step in the out-patient journey. A mining company could recognize the importance of monitoring automated boom-gates at the mine sites since manually opening the gate slowed down the supply chain significantly.

Below is an example of logical components and physical components together.

Final: Creating the Service Tree in ITSI

Once the team is happy with the resultant service tree, it's time to replicate it in Splunk ITSI and populate it with real-world data. There are three ways to create these components in ITSI: 

• bulk upload using csv
• import services from a search
• or create them individually as we’ll explore below.

1. Creating Services in ITSI:

Go to 'Configuration' in ITSI and select 'Create New Service'. Provide the title, description, and the team that should be able to access this content.

2. Defining KPIs for Each Service:

Key Performance Indicators (KPIs) are vital for monitoring the health of each service. To make it easier to get started, ITSI comes pre-built with the most commonly used KPI’s, identifying and selecting the relevant ones for the new service. If you create your own KPI, you will need to define the source search on which to build your KPI. You can choose from four source search types: data model, ad hoc search, metrics search, or base search.

While data models are useful for testing purposes, base searches perform better in large-scale environments.

3. Setting Thresholds for KPIs

Before enabling your services, setting accurate thresholds for each KPI is essential for monitoring service performance. For each KPI, define what constitutes normal, warning, and critical performance levels. Use the Thresholds tab in the service configuration to set these values.

4. Add Service Dependencies

Once you’ve created a single service, it is time to map other services and their dependencies. To do this, select ‘service dependencies’ in your newly created service, select ‘add dependencies’, and select the dependent services and the KPI’s within those services that your service depends on.

Once the data starts to flow into ITSI, you should see a visual representation of the ‘service tree’ that you’ve whiteboarded along with error propagation across both business and technical services.

6. Applying Machine Learning for Predictive Insights

Machine learning in ITSI can be used for predictive insights and proactive service management. While the Splunk Machine Learning Toolkit (MLTK) allows advanced users to create complex use cases across their data set, ITSI offers a simple out-of-the-box GUI experience to help beginners apply ML algorithms to their data.

Once your services start getting populated with data, select ‘Predictive Analytics’ tab within a ‘Service’ configuration. Then select a time range for ITSI to provide you with an algorithm that best fits your data.

Once you are happy with the recommended algorithm, train the model and ITSI will provide you with a prediction of what the selected service performance is likely to be within the next 30 minutes.

7. Creating a Dashboard for Visualisation

Now that we have visibility across the key KPI’s of the business and technical components along with its performance data, it is time to create a visualisation that fits the needs of your teams. ITSI Glass Tables is based on WYSIWYG (what you see is what you get) design philosophy. This approach allows users to create and customize complex dashboards in an intuitive and visual manner.

To create a new dashboard, select ‘Glass Tables’ within ITSI and ‘Create Glass Table’. Use widgets like charts, graphs, and maps, linking them to your services and KPIs for a comprehensive view of your service health in real-time along with a background image of your choice.

This guide is your starting point in leveraging Splunk ITSI for effective service decomposition and performance visualisation. Each business is unique, and so should your approach to using ITSI. Whether it's a healthcare provider or an e-commerce platform, the ability to customize and adapt these practices to fit your specific needs is what makes ITSI a powerful tool in modern service management. Explore these capabilities and more in a personalized demo.