Context is everything — especially when it comes to a business-impacting incident. When SecOps, ITOps, and engineering teams have sufficient context around an incident or threat — piecing those breadcrumbs together creates a fuller story. This enables teams to make faster decisions and resolve problems before they become catastrophic.
But it’s not all about the data. It’s about people. Sharing and reusing data across security and observability teams should be developed over time, like muscle memory. Departments that don’t often collaborate will need support from leadership so they share the most relevant data, and they do it without friction.
Our research backs that up — according to Splunk’s State of Observability 2024, 73% of leading observability practices improved MTTR when they brought security and observability tools and workflows together. A Perspectives editor sat down with Cory Minton, Field CTO to discuss how technology leaders can reap the massive benefits of data reuse.
Perspectives: What are the main drivers of data reuse initiatives today?
Cory Minton: Responsibility and reporting have created the need for collaboration amongst these teams, because the materiality of what each of their bad days means matters to the board of directors and matters to shareholders. For example, the SEC mandates that you must report a material incident. That incident could be cyber, or it could be downtime related to a critical service or app. And so they're getting held accountable to make sure that they're sharing that information, that they're quantifying it. When you talk about materiality, the thing that you're really working towards is ‘How do I measure the actual business impact of these events?’ Because that's ultimately the question. ‘When you say material, what do you mean by material — lost revenue or lost productivity? And how do I measure that?’.
That's where security teams are leaning more on some of the observability teams that have just done a little bit more maturing in terms of connecting business outcomes on the observability side, because we're often observing those revenue-generating applications. Business context is starting to matter a lot more to cybersecurity practitioners.
The data reuse conversation has also become real because of pressure to consolidate and drive down costs associated with building security practices. It’s expensive to maintain multiple platforms — not only the cost per month in terms of subscription or licensing — but also for people to maintain them. It's hard to build a talent pipeline if you're leveraging a massive variety of tools.
Perspectives: Data from State of Observability 2024 shows that advanced observability teams are bringing security and observability tools and workflows together to improve MTTR. How do you think they achieved that?
Cory Minton: When security teams are combating an unknown threat in their tool and engineering teams are wondering why there’s high performance on servers, neither fully understands and connects those dots because they are in two disparate systems. Having the intelligence in the middle is a big benefit, so you can achieve that centralized monitoring across the environments. That reduces the amount of time it takes to identify or combat a threat and meantime to repair (MTTR).
Let’s say you've got a terminated employee. Suddenly you start seeing these interesting behaviors with their account across their environment end to end. It could be logging into a VPN. It could be downloading a file. It could be running a script on a box. Being able to have those breadcrumbs end to end — from the edge and compute, all the way to the back end or from the VPN — gives you that confidence that it is without a doubt a problem. You’re not chasing down a rabbit hole of something that may not be a problem or not. That's huge. If you put the picture together, you’ll get a much higher confidence level because you see them doing weird things across different servers, hosts, and other technology.
Perspectives: How can organizations use shared data to communicate with their teams and make data-driven decisions more effectively?
Cory Minton: If I'm a CISO, what do I want on my 60-inch monitor in my office that would give me the information I need to manage up and down? I'm probably going from meetings with my leadership team to talking with my team about the activities that we need to drive towards. In a dashboard for that persona, I would want posture metrics — the critical business-impacting metrics that I can then rapidly communicate up to a CIO or CFO. I can say, ‘Hey, you know, I think we're doing pretty good. We have zero material incidents to be concerned about.’
To understand what my individual teams are working on, give me those red, yellow, and green indications of what I need to be concerned about. That way, when I get on a call with my direct reports I go, ‘Hey, I noticed that we're having some problems with threat hunting. Talk to me about that. What kind of air cover do you need?’
When you talk about sharing data, that concept should be pervasive across the organization. If it's one platform that you're using to surface those business-impacting metrics that matter to manage up and manage down, that's our goal. Good dashboards should be like my daily paper, a dashboard that should help me manage up and down very rapidly and should guide the activities and conversations that I'm gonna have throughout that day.
Perspectives: What advice would you give to a CTO that’s starting the journey of bringing security and observability data together?
Minton: The goal of unifying security and observability data is to provide more data access to more users, while also minimizing the number of locations they access it from. However, for governance purposes, you also need to put the data in the right place with the right visibility and the right privileges. That’s why the first step is to understand what the data is and classify it. Is it security data or business data? Does it contain PII or secrets, or is it bronze classified where it's just public domain? Some security data is precious, and it falls under regulatory compliance requirements. Then, rationalize where the data is going and who is accessing what. Begin the process of saying, ‘Hey, do we need 15 different monitoring tools? Or should we have a centralized logging platform and a centralized metrics engine that serves a number of use cases or parties within the organization?’
Perspectives: What about some of the cultural barriers that security and observability teams face? How do you start the process of breaking down those silos?
Minton: Security practitioners oftentimes have their elbows out because they're like, ‘No, this is my data.’ And what they're realizing is that they all signed the same employment covenants and that it's not necessarily their data. It's just that data is useful for them and for the outcomes that they're responsible for. They take a lot of pride in making sure that they get those.
The other factor is that everybody gets busy. If security and engineering teams are left alone and they don't have to talk to anybody, they will put their elbows up and be territorial. But when they realize that, ‘Hey, you're trying to solve something that's the same thing or similar that I'm trying to solve. I might be able to help you, and you just happen to be on a different team.’ If there are benefits in it for everybody, then you have to be intentional about nurturing and fostering those relationships. That comes from leadership. If leadership's not making it a priority, it will not be one. And if leaders aren't holding their teams accountable to it, don't expect anything to change.
At a high level, you need to build people bridges. Collaborate at the top, and then make that part of the organizational cadence so that cross-functional team engagements occur on a very regular basis. That's the easiest way to break down the walls: put them in the same room.
Splunk’s State of Observability 2024: Charting a Course to Success uncovers that advanced observability teams more often collaborate and share data for better troubleshooting and incident response outcomes. For more insights on the habits of successful observability practices, download the full report.
