How Efficiency Depends on Observability

If you hadn't heard the term “this is the new normal” yet today, then you haven't been listening. While right now is not normal, current events have us all wondering how the work environment is going to change once we get there. There are a few things that we can expect:

• We are going to lean even more on technology for better communication and productivity
• We are going to think more about avoiding distractions and context switching
• We are going to be more active on digital communication channels
   

Having pipelines and applications that are observable is key to all of this.

In a recent blog post, "Adapting to the New Normal in IT Operations," from team member Alex Afflerbach, he talked about how IT functions are fundamentally going to change to a more Distributed NOC (D-NOC) mode of operation. This means that the team is going to have to rely on actionable information, visibility, and digital communication without the ability to walk to someone's desk, tapping them on the shoulder and asking for help.

“The rapid shift towards a distributed model exacerbates inefficiencies in old approaches and exposes gaps.” - Alex Afflerbach

The human element of this transition is honestly the most difficult. People are creatures of habit, routine changes are hard, and there is an amazing amount of efficiency gleaned from casual interactions. The human element is a battle both teams and individuals will have to fight to be as effective (if not more) as they embrace it.

The technical aspects of this transition are less difficult because the technologies and practices are largely there. We have no doubt that advancements in this area are going to come as a result of increased demand for better virtual work environments. But remote work is not new, and the ideas of supporting technical environments in a distributed fashion also are not new.

The “New Normal” for DevOps

“Today’s forced acceleration of change has an impact on many facets of the IT Operations Center” - Alex Afflerbach

No one can predict exactly how things will change. DevOps and application development, just like IT, are going to go through a huge transformation. In many respects development teams have a head start over operations teams, having adopted significant automation practices and use of ChatOps but in turn, will face increasing challenges along the human front. The human interaction of interaction and exchanges of information is a critical piece of modern development teams. As development teams moved into a microservices model, the teams themselves have become more intimate and members more connected. 

Expanding on Alex’s points and relating them to application development, there are 3 key strategies areas that DevOps teams need to embrace in their new mode of operation, and they ultimately break down into getting the right visibility at the right time, surfacing issues versus hunting for them, and quick dissemination of tech details. As it relates to DevOps delivery chains and application development, this is observability.

Visibility

Visibility has been a staple of high-performing IT and development environments. Monitoring tools that are very good at collecting telemetry data across systems and giving the ability to build dashboards and search across that data. Visibility in environments where application infrastructure and stacks are becoming increasingly complex, especially with microservices and Kubernetes, along with teams that are now spread everywhere means that organizations have to be deliberate about the data they collect, how they structure it, how they display it, and who they give access to. Data relevant to teams needs to be available, and dashboards have the right metrics displayed in a way that is digestible at a glance.

Surface vs. Hunt

“You have to have data in context to take action and gain value. Data by itself is just a null value. “ - Dave McAllister

It is absolutely critical as engineers do their best to focus on their on-going tasks to make sure that any and all interruptions are meaningful and warranted. Teams should not be hunting down red herrings. So monitoring systems need to surface relevant data that truly requires the teams’ attention, and give them meaningful insights to address it. They should not be digging needlessly into events only to discover they are on the wrong path, or the perceived issue was not the right one. This requires:

• Correlation: Monitoring needs to understand down to bits and bytes what is going on when something breaks in the application. It also needs to correlate relevant and insightful data to understand the blast radius of an issue and see how it might affect other,  related services. 
• Depth and breadth: Monitoring systems need to be instrumented across your entire stack in two ways:  bottoms-up from your infrastructure to the front-end, and left-to-right from your planning to production. Having point solutions that give visibility into one slice of your operation is not great for making long term decisions.
• Incident response: If and when something breaks the pipeline needs to have a tightly integrated incident response tool that is connected throughout the entire stack and can mobilize the right virtual team member at the right time.
   

Observability and the New World

Observability departs from traditional monitoring in that it recognizes that information can be a distraction, unless both timely and relevant. In modern distributed and virtual work environments, those distractions take away the team’s ability to efficiently perform needed tasks while maintaining team culture and connections. Observability, especially when paired to automation, incident response and AI/ML, enables organizations to understand the customer experience in production, help to determine what to measure and to be at the forefront of stewarding the practices of DevOps. The technologies and methodologies that drive observable environments already exist and organizations that embrace observability will deliver improved efficiencies within both DevOps and application teams.

Learn more about what observability can do for you!

----------------------------------------------------
Thanks!
Chris Riley