State of Observability: Overcoming Cloud Complexity, Talent Shortages and More

Although the concept is not new, observability as a practice has gained a lot more traction  recently — that is, when the pandemic accelerated the digital presence of many businesses and thus cloud adoption, which exacerbated visibility and complexity challenges. We’re seeing that observability is an essential solution to these issues, but also brings measurable payoffs.

Today Splunk launched The State of Observability 2022, research conducted with the Enterprise Strategy Group. The research — based on a survey of more than 1,250 observability practitioners, managers and experts worldwide — defines observability leadership and details its concrete outcomes, finding that leaders experience better MTTR (mean time to resolution), suffer lower downtime costs, launch more products, and more.

To define observability leadership, we examined four key factors: experience, ability to correlate data across all observability tools, progress in vendor rationalization, and adoption of AI/ML technology within their observability toolset. Leaders possessed all four markers of maturity, whereas intermediate was defined by having any three factors, and beginners two or fewer.

Observability maturity leads, of course, to things like better visibility, shorter MTTD (mean time to detect) and other well-understood benefits within IT. But we’re now seeing that the competitive edge from having a robust practice extends further: launching more products, succeeding in digital transformation, attracting top talent and more.    

Complexity Challenges

The number one driver of observability adoption is cloud complexity. Organizations moving to the cloud is old news. But add hybrid architectures, multicloud operations and microservice-based applications to the mix, and you have escalating complexity, low visibility and rampant burnout among Ops teams. Ninety-two percent of observability leaders and 68% of beginners commonly use cloud-native applications that run in multiple cloud and on-premises environments. Meanwhile, the observability talent gap is all too real, as a whopping 95% have trouble finding IT operations staff — whether it’s finding enough people, the right skills or both.

The good news is that the research suggests that observability is an effective antidote to some of these challenges and creates additional positive outcomes: accelerated development and deployment, less downtime, improved hiring efforts and more. It certainly helps with visibility challenges, because two-thirds of leaders have excellent visibility into application performance, versus just 44% of beginners.

Observability Strategy

The research found that organizations with mature observability practices tend to use more tools, but fewer vendors. Having found which vendors and tooling best fit their needs, organizations that are consolidating their vendors benefit from lower training costs, better interoperability and simpler purchasing and onboarding processes. 

Other widely adopted practices among observability leaders include using CI/CD to automate the delivery of new code (96%) and AIOps to facilitate event correlation and analysis (71%). Organizations have turned to AIOps in particular to respond to incidents with greater intelligence and automation and detect anomalies faster. And they’re getting results: faster mean time to detect and repair, faster root cause diagnosis, and an improved ability to gather data to gain a complete picture of the infrastructure. 

Topping the main challenges and concerns associated with observability is the astronomical volume of data. That points to the appeal — the necessity — of AI/ML solutions. AI/ML can address the skills gap as well, thus alleviating project delays, burnout, resignations and more. 

As the benefits of observability are better understood, it’s also becoming clear that an observability practice is not a nice-to-have, but a necessity. We anticipate that observability teams and tools will expand through the IT and developer organizations and converge with other IT functions, as cloud proliferation continues and as organizations figure out how to collaborate, innovate and excel in a multicloud reality.     

To learn more about observability strategy in today’s hybrid, multicloud world, read The State of Observability 2022 in full.

----------------------------------------------------
Thanks!
Spiros Xanthos