How to Stop Playing the Blame Game in Your IT Department

It’s a familiar scenario: a problem is discovered, and a Service Desk Team gets a help ticket. The Service Desk Team tells Operations that there’s an outage. The Operations Team suggests that the problem could be the result of bad code and passes the issue to Dev. The Dev Team responds that it doesn’t have the tools to solve the problem and asks for logs from production systems.

Suddenly the situation is escalated.

A war room’s assembled. Here you’ll often find a DBA, Docker specialist, network specialist, release manager, site reliability engineer and a developer, sometimes calling in remotely from separate locations. The pressure’s on for everyone to prove their innocence and confirm individual components of the infrastructure are ok. If everyone survives this step, people start blaming each other while the clock continues to tick. And you haven’t even begun solving the problem yet.

This is the Blame Game in a nutshell. And unlike most games, it’s one that you don’t want to play. The process of problem solving and getting teams together is time-consuming, stressful, and inefficient. And in the meantime, your reputation and revenue are at stake—you begin losing users quickly if they can’t access the service and there’s no indication as to when the issue will be resolved.

The Blame Game is similar to the game Sorry!, except there’s no clear winner and no one apologizes.

Good news: It’s possible to stop playing the Blame Game and start recovering your sanity. The key is for everyone to quickly align around your organization’s data. Additionally, you must:

• Be able to trust the data that you and other teams are using
• Have easy, real-time access to data across silos
• Empower your teams by giving them access to the same data

Chances are you’re using a bunch of tools to do this already. That’s a good first step. But the Blame Game only ends when you get full visibility from all of the tools and data sources within your organization. This is achieved with a data platform.

Gaining visibility into all of your tools and data from a single platform is key because it helps you quantify issues. It also doesn’t leave any data “unattended”—in a world where all data is relevant, your bases are covered. Perhaps mostly importantly, it enables everyone to be on the same page instantly—saving you time and stress over whose data is right. When everyone can see the same error that’s from a particular process, you can all agree that’s where you need to focus your investigation. You’ve successfully replaced stress with data and can work in parallel to find and solve the issue—making your users (and bottom line) happy.

Nothing ends the Blame Game faster than trusted data.

Learn more about using a platform approach to monitor and troubleshoot applications and the infrastructure that supports them.

Bill Emmett
Director, Solutions Marketing, Application and Mobile Intelligence
Splunk LLC

Follow @billemmett000

Follow @splunk

----------------------------------------------------
Thanks!
Bill Emmett