Calling CISOs Everywhere: Minimize Breaches by Thinking Like a Hacker 

Note: This is an auto-generated transcript, which may contain errors.

Cory Minton: In this episode of the Perspectives podcast by Splunk Craig Hyde Field, CTO Splunk sits down with Todd Bebe, the chief information Security Officer at Freeport LNG. 

Todd Beebe: Very early on in my career, I was immediately put into kind of the security role even though there wasn't security at the time, it was like, hey, we're connecting to the internet, we need firewalls, go figure it out.

And I just kind of stayed with it as just the, you know, one of the team members, I've heard this a lot with other people and that and security. They just raised their hand, right? And next thing, you know, that was their responsibility to go learn it and, and try to get it deployed. Over time, I became a very skilled in being able to validate or, or test security and was brought in to do build a practice for a major at that time, big six for attack and P so that really kind of colored. and, and kind of focused my attention on that kind of mindset even though most of my roles after that were really on the defensive side. 

I never, you know, that kind of, that always was forefront of my mind even when I would be sitting in meetings and people would be talking about something like, oh, what did you just say? And then it was like, I could almost see the path of what an attacker would take with if they deployed it that way. So, so, that's, that's probably where I think I, I differ a little bit from most information security officers or cio s is that extremely technical and I never stopped doing that.

So to this day, I'm still testing the systems, I'm still validating, doing validating patches work if there's vulnerabilities in the organization. So, so that's that that's never left my blood. I've never, I've never grown, grown up or grown out of it. So that influences a lot of my decisions and my priorities.

Craig Hyde: Yeah, with the, the technical and kind of the, the attack side, like, how do you, how do you think, how does that manifest itself in terms of differences with other C OS or other security professionals that either you've worked with or worked alongside of? And in terms of like, you know, how does it, do you bring different viewpoints? And so how do you prioritize different things? If so what those kinds of things from you know, a day to day perspective.

Todd: So, so, so I, I prioritize what we get from threat intel to being kind of the Bible versus a compliance and governance mindset of, hey, if we meet this or if we get a, you know, a maturity rating of a five, then we're good. Those are, those are and there's a lot of standards out there where they talk about hardening a device and there's, you know, 1000 different settings that are in there from, you know, when XP or, you know, windows 3.1 were around, but those never left the standard and nobody looks at it and says, what are the ones that will really prevent a breach of this system?

And so I think with my mindset, I look at it and I prioritize different, you know, it isn't that I think we shouldn't have standards and we shouldn't secure all the systems in the environment. But the difference I've seen in compliance and my mindset is prioritization, you know, and, and compliance is usually has just that your organization is at this level. It doesn't say the most critical devices you have are missing the most critical patches.

It doesn't, it doesn't know it, it, it isn't designed to prioritize because it's trying to be something that everybody can get some value out of. But it doesn't help prevent a breach in that organization specifically based upon what are my threats.

So I, I'm very threat oriented and, and paying attention to what threat actors are doing. And then actively determining if, if that activity would go undetected our environment and fixing that. It's very much a, a constant validation. It's kind of, you know, the infamous cat and mouse game of what, what, you know, what, what's the mice are doing with them in order for me to catch it. This is what I gotta do differently, right?

So, so it's, it's a lot more mentally intensive because I'm not just going off of the script of what the government says. I mean, I've, I've helped build cybersecurity frameworks, but that's just, that's what they are, they're kind of frameworks to help people get to a mature level. But then when you're at that level, it's like, what are you going to do to prevent the threat actors that are really targeting you?

Craig: That's interesting because it sounds like the way that you're thinking about it. It doesn't necessarily say this framework does not apply, right? It's not, it's not saying that the compliance frameworks that exist, call it nist or whatever the sec comes out with.

It's not, it's not saying that it's not valid, but almost that your approach to what is the most important is driven solely based upon what are the real risks that we have in the environment versus what does this 175 page document tell us we have to do?

Do you feel like sometimes people get lost in the weeds of working through the compliance and don't see the forest through the trees?

Todd:  I think part of it is if they didn't have that technical background because a lot of the, the individual they see at the meetings or they came up through the business. So they understand, I mean, I literally was at a conference where I was a speaker and I was speechless when they were talking because they were all talking, you know, risk and, and all these equations and stuff like this. And I literally looked at the next person next to me like I got nothing.

And then when I got a chance to talk, I talked about these are the security controls I would use I CS environment and then people kind of realize, OK, that's where, that's where I fit, you know what I mean? I was brought in for a specific role.

and, and to me, it's kind of the difference between a building inspector going in and making sure that it meets all the guidelines and then the building ages a little bit and if you bring in and, and they, they actually are looking at various security controls, we need these, you know, windows like this.

You bring in a former burglar and they'll notice things that maybe the building inspector didn't like. Hey, do you guys realize you have absolutely no sensors on your second floor,, windows and, you know, but that wasn't in the building code. So, so they were trying to meet the building code, which they did, but they didn't see it through the eyes of a threat who's saying I'm just trying to find the path in. And the path to whatever I need to get to. And is that path still is that, can I find a way to get to that without getting caught?

That little, that little mindset I think is the big difference that I've seen is,, I just constantly think,, my, one of my team members, she says you got a hacker brain. So, I mean, that's, that's, that's what she calls me. Oh, your hacker brain just kicked in. So, so it's, it's definitely, it's helped. But,, I think that, you know, it, it,, it's hard to find that in other CISOs that I've interacted with.

Craig: Yeah, that's interesting because a lot of, you know, you go to, you take, go to a millennium conference and a lot of the thread comes to aligning with the business goals, right? And sometimes they are just, we have to go get compliant because that is another threat in itself. If you're not complying, that opens you up to lawsuits, it opens you up to, you know, different, organizational issues.
And,, but you, your focus, it seems like the, the key focus is like, what is cybersecurity’s number one job?It's to not get hacked.

Todd: How I look at it is there's a difference between a risk to me. Risks are a hypothetical, like a hurricane will hit your organization or flood will hit you. It's, you know, probabilities.

And I've actually seen organizations where they tried to, you know, implement that probability to cyber. And then when I look, when I look at cyber here, here's the analogy or the, the story, I like to tell people that story, but little snippet is, it's one thing to think. Ok, I've, I've met the requirements. I got alarm systems and all my windows, all my doors and everything else.

I feel like I'm good. Right. Because I know there might be a, a, you know, a robbery, you know, but I, I think I got enough controls. It's a whole different story when that risk turns to an active threat, tornadoes and hurricanes, they don't change their path just to go after something that's more vulnerable. But if a police officer came to your door, knocked on and said, hey, guess what? We just found your name on a list that, that the burglary,, organization in this neighborhood is coming after your home?

Yeah, you're gonna look a lot differently at, do I really have enough security controls? And that's what's changed is the visibility and the threats that are out there that are actively trying to get into something versus it's just a matter of chance.

And that's the big difference to me is the probabilities of the hurricane hitting this every five years. Is this in this area of getting, you know, have fire and damage that goes beyond a single room. Is this ok? We got that covered. But if something's actively trying to do it, it will modify its behavior to continue it.
You know, it's like the tornado says, I'm taking your, it, it doesn't have a mind that says I'm gonna take your house out. If you got lucky, you got lucky. But,, I think that's the difference is they're applying that probability of compliance of like, you know, of these unlikely events, you know, potentially occurring to, well, if you look at your email, gateways, how many times have you blocked malicious emails trying to get in? Did they stop?

Because they, because you stopped them? No, they continue today. It's an absolute, yes. I mean, they, they're gonna, yeah, they're, until their boss tells them to move on to an easier target they're gonna keep attempting because that they, you know, that's, that's human nature, you know, I'm gonna finish my job.

Once they start thinking in that threat mindset that there's an active actor that's gonna change their behavior. Then it becomes a different risk. It's a threat. It's, it's, it's what we do because it's, this is a long term game that they're going to continue,, to try to find our weakness right in your organization.

Craig: I, I'm just curious, what type of, like, what are those big risks that are always coming in the door?

Todd: I don't wanna get too deep into,, specifics but,, the, and I don't want to talk about a specific organization but I don't know if I've been a account of this lucky, but I've been in multiple organizations where it was nation state level threat actors that was interacting with three letter agencies who are giving us play by play of how they were monitoring, actually seeing the behavior on the dark web of these threats.

So, and when, you know, once you hear that you can't un hear it like that, you know, that, that takes it to a different level of, yeah, the tornado, it's got your address and it's to do, right?It's like, ok, do I really need shutters or do I need storm windows?

It's like,, and the budget changes and, and the other thing I like to talk about is, is,, II, I, I've been lucky enough that in my, in my life, most of the organizations don't look at me and say, ok,, you know, justify this as a return on investment. It's, it's more like ok, we're gonna drive fast or we're gonna have airbags, we're gonna have seatbelts because there's the likelihood of this happening. So what do you need to, to allow us to drive 100 miles an hour safely? And, and, and, and so, so thinking of it more as a cy cyber safety thing. It's like if you want to have this business in the middle of the, you know, the craziest part of Brazil, you need to have these guards, you need to have these gates, you need to have these kind of, you know, devices out front that will stop a ramming car.

It's like you guys can do business there. It's, it's kind of, it's still the same thing, it's business and enablement, but it's looking at it from this is what you need to have in place.

So you guys don't have a lot of losses in that room because really what we're trying to do cyber is loss minimization just like firefighters just like the we're, we're trying to keep it to a single room right to, to make the house still sal salvageable versus it's a total loss and, and nobody expects fire departments, police departments, doctors to have 100% success rate like nope, there can be no robberies in your neighborhood. Otherwise we're gonna fire the, the police chief, right?

That, that's what I think that's a separate conversation about the sec where they're like having this expectation of 100% prevention. It's not so much prevent an incident, it's prevented from being catastrophic, just like, just like seatbelts and airbags. They, they're not going to prevent all injury but make it so the person can survive the incident. And just make sure the business can survive is kind of the mindset.

Craig: What do you think the key components are that are, constant or the most overlooked?

Todd: The key, the key components is assume the, assuming the controllers you put in place will continue to work as advertised because, you know, with products, you know, upgrade in a product that was a security control no longer blocks, a certain thing that we thought it was blocking that blocked when we tested it, you know, before deployment and then, you know, some something in the code base got reintroduced.

And now it no longer blocks that threat that we thought we had cover. So it's, it's, it triggers in me and, and the team that I have this, this need to routinely revalidated that the security controls we have in place.

Both detective and preventive are, are working. And I think that's what's missing is this, this assumption of we're good, you know what I mean? You know, we got airbags and everything, the anti like brakes. It's like I, I used to do this even when I was young in Nebraska with snow. When I'd be driving out, I'd hit the brakes hard to see how slippery the road was because I really, I wanted to gauge what's the risk level, right. The real risk currently right now. And it's kind of that same thing is ok.

Let's, let's essentially pin test the organization on a regular basis, and throw at it the same things that a threat actor would and see. Do we block and detect what we thought we were because then we can build from that. But if we find out hold on the gaps we thought we had covered are now uncovered again.
If we continue, just assuming we checked the box 66 years ago and we got that security control and we never test it again. We might, we'll have a false sense of security. 

Craig: with that like ongoing testing and you're all essentially there's an element of shifting left, but it's also organizationally like the, there's an element of, you know, test early test often and make an ongoing process so that you can like it's not security is not just paint, you can't paint it on the door at the end, you have to make it into the process, but there's also the organi I'm hearing the organizational testing of, ok, we have a threat. What happens with incident response? How do they react go through those, those use cases? Do you when, when you're going through this process, what's the breakdown of a manual versus automation that you have in either either the testing, the active testing or the, the responding from you know, an incident response team

Todd: The goal for the validation And we've tried to look for third party solutions to do it is to get it to the place where we can automate it and then get a report on a regular basis of ok, what, what, what detections that we're working when we set it up are no longer detecting right now. And once we see,, you know, a deviation that goes beyond just a couple, then we kick into gear and jump back into the system and figure out, ok, what happened?

Because we, we've seen it where we had,, EDR solutions where, you know, we say we test 1000 things a day, right. We, we run 1000 different simulations. It was doing fine, doing fine, doing fine all of a sudden. Now it's only detecting 600. We're like, what just happened, right. , but if we wouldn't have had that ongoing validation because it's still detected a lot that, you know, you know, a good portion of the majority,, the team would still be getting alerts, but would they know they're getting alerts for everything that we thought we were alerting on?

, so it's, it's that kind of it, you know, being able to detect that we'll call it partial break,, where something, you know, this, this leak, that when you look at it, the sprinkler systems are all going but there's one area that's not getting anything and, and, but it looks like everything's good to go.

It's that false sense of security and that, that's triggered us to be very vigilant about,, validate, validate, validate, valid it, because,, we've had, we've had PIN test where they've come in and it was almost guaranteed that one of the security controls we had in place wouldn't detect some of their activity or wouldn't block some of their activity because of some one off.

You know, and it just seems so was that always kind of just,, reenergize the team of this is why we do what we do, you know, it's grinding to continually test something and, and not get that. Hey, let's move on to something new, but firefighters, we do it too, you know, we have to, we have to trust that gear before we get to the fire and realize the hose was,, defective.

Craig: So, yeah, and then, and then what's the, what's the dynamic look like when? So you're going in doing the pen test, find vulnerabilities?, how do you, or if you find new vulnerabilities, getting new threats that are coming in, how do you orchestrate that into the overall security operation to make sure that we're looking for the, the latest threats and make sure they have a complete view of all the things that are going on in the organization. So we can, you know, do the firefighting when, when fires occur. 

Todd: So, so, so, so we, we, we, we're connected with a good number of people in three letter agencies as well as some of the more active threat. Both researchers and incident responders like Twitter, et cetera.And, and we just, we just leverage kind of the noise in the overall security community about certain things to know. OK, I mean, usually, usually it's real easy with vulnerabilities if it gets a name like Blue Keep or anything like that, that's usually a bad one. You should, you know, any organization should be like if it's got a name set up a website for it.

That's probably not a good thing. That probably means all they have to do is exploit that thing. They don't even need credentials, they'll just be on your system. So, so, again, I think it's just being in security for so long. I get, I get, you know, II I know enough people and I get a sense of where to look that that latest Intel on either, you know, some zero day or some setting or something like that that's being taken advantage of.

And then that becomes a pretty quick priority. The, the, the great thing is, is the organization that I've,, well, multiple organizations that I've been working with,, I guess they brought me on because they, they were core believers in cyber.

So I was in, and, and again, I wasn't giving them this laundry list of, here's 1000 vulnerabilities, go fix them on a, you know, 10,000 machines. It's like these two vulnerabilities, these machines that we really care about then fix the rest.
But that really, I help kind of you know, look at it through their point of view is if you're gonna make us not prioritize anything and everything is a high priority, how will we know we got it covered if something's trying to target us right now?

And so I think it's that being that because of the experience being very selective on what I asked them to deploy, whether it be a setting or a patch has as, you know, they, they've realized, OK, I'm, I'm looking at it through their eyes too.

I'm not just, hey, compliance says we have all patches done in the 1st 30 days. It's like, no, it's this one, you know what I mean? So by being able to be very surgical, I think has helped build the relationships between the teams and then helps that that urgency kick in. because they don't OK.

The only reason they're bringing it up is because this is a big deal, right. So that, that makes a lot of sense.

Craig: So when the the high priority vulnerabilities come out, you've got to, you have to basically prioritize those to get out immediately and deploy. And what about like active hacks, social engineering insider threat type things? Are you like, how do you, how do you monitor that?How do you monitor for, you know, the threat actors coming in and, you know, being an active threat where or where somebody's coming in and currently hacking, like from a SIM perspective or security operations. And how do you handle that?

Todd: Well, well, I, I guess I can say because I'm on your guys' podcast, we use Splunk, right? It's like, and, and, and, and again, I'm not just selling Splunk but,, I, you know, I was in cybersecurity before there was any kind of, and it was that, that it's same, same slogan is probably used today is if there's no logs, there was no crime, you know what I mean?

And, and the, the biggest pain in the early days of doing forensics is you literally had to log into each system and, you know, and, and, and still try to do it forensically. So it was slow and then they usually, you know, was the setting in most of the devices was like 500 K kilobyte, 512 kilobytes or something like that, like less than a meg or something.

And so So they had already rolled, right. So, so,, logging is, is core,, because you, again, just like a fire, nobody is gonna call you before a fire hits. So you can get there and prevent damage. That, that, that's, that just doesn't occur. That's, that's make believe, same thing with, I mean, it's instant response for a reason.
You know, I, I, I've said this before to other people. It's not incident prevention right? That it's incident response. There's something that started, there's smoke somewhere and we want to get there before it becomes big., but without logs, then there can be no alerts, right?It's in and I got this, saying from a guy that,, he was eventually a CEO of one of my companies, he had sold the company to Cisco.

, it was the first I DS, he had come out of the Air Force and his, his saying this is very early in my career. It was visibility and control, right? And if you don't have visibility, you certainly don't have control. And so, so, you know, but day one when I come into an organization is like, do they have a SIM?

And then do they have an EDR, you know, kind of, I need visibility because then I can get the lay of the land of what's really going on and get a sense of,, where are my gaps and where are my priorities? But lo logging is core. You have to have that before you even put in security controls.

Craig: That's interesting. I mean, it's almost like you can't, the old saying, if you can't manage what you can't measure, I've, I've never heard of like, you know, you've got to get the, the logging in before you put the controls in. But, I mean, it's true regardless of the controls. If you can't see anything, what's the kind of, what's the point?

Todd: Well, because then how do you know you're not blocking something legitimate? You don't know what normal is. So how will, you know, it's abnormal unless it's triggering some security control, like a V or a firewall or something where it has specific rules.

But then if you don't have that going somewhere, then you got to make sure that's all going to your email. So, so, so it's like you, you, you gotta have some place to get the visibility because then you can make the business cases too of, hey, we really want to isolate these 27 very critical systems. Well, you know, they're very important to us. Well, we've looked at the logs for the last 90 days and it's only these three workstations that ever talk to them.

So, so we're not going to, we, we now have evidence,, not for a crime but evidence to show you why we can put in a security control and not impact business. So, so, so eventually it does come back where I, I do interact with the business even though I'm technical. 

Craig: But, but I usually like to bring them the, you, you wouldn't be in your chair, you wouldn't be in your seat if you didn't know how to communicate and interact with the business and get what you wanted.

Todd: Well, well, that, that, that it's because I do communicate but I communicate with reality, not the, the fear stuff. It's like, like I'll, I'll literally come into a company when, when I start in a couple of weeks, first couple of weeks and I'll show them why we need a certain patch. I'll exploit the system and say, look, look how quickly. Now I'm, I'm, I'm system on the system. You know what I mean? I, I can do anything. , and they're like, ok, I, now I see the value to fix that.

It's just that it's like I'm not talking about hypotheticals. I like to use real threats to make the business case. But I also like to use real evidence of this is what, what's going on. So I'm not, I'm being very surgical of what I asked, right? These patches, these systems, these rules to block this kind of access. So, so the business can continue. You can, you can move into whatever the most dangerous neighborhood in Brazil is, but guess what if we put these in place, you'll be able to do it and survive, right?

You'll, you'll, the business can grow there. versus, yeah, we, we wanna, you know, back in the day we want to connect to the, to the internet, but we don't want a firewall. Like guys, that's not good because we have rout IP addresses inside our entire network so they can get to everything. It's like, let me show you. So, so was it, it's definitely using the information I get from the SIM and everything else to, to make the business case with reality, right?

This is hypotheticals. This isn't. So some list of to do si got from some government, government agency, this is based on experience. So I think that helps a lot and you know, allow me to influence the business with.
Craig: On this topic of like, you know, turning on the lights and starting the conversation or giving something to have the conversation with. There's we kind of think of it as, you know, everybody's got a maturity model but from the way that we've been thinking about it, looking at it, it's almost like everybody's got to have something you've got to turn on that foundational visibility, right?
Once you have that, then you can kind of like, you can start to prioritize and figure out your actions that your responders have you from that you can start to do automation and you know, kind of like get some of the manual labor off of your smart teams and, and sort of like the utopia is just like one single pane of glass that everybody's working from.

But how, how would you describe a, you know, that's how you see it. But how would you describe the a typical trajectory of maturity in a security organization? 

Todd: It's, it's really that it's, it's, it's getting visibility. So, you know, the lay of the land. So, so you're all, you're all talking about the same playing field, right? Like this, this is, this is what's in scope like because without scope, you can't prioritize, right? Because then it, then it's really prioritize prioritization based on what I last heard, right?

I heard this on the way in on the radio, I saw this on TV, one of the friends that I know we're talking about this, we don't because we have no visibility, we don't know that's not happening. So now it becomes a priority.
It's almost like you can't tell the business no about certain things of they really want to deploy this or they really need this because you can't make the case either way of why that isn't a good idea, right? Even though it's not based on, if it's not based on what's going on in the organization, it's like we only have one server.

That's that and only one workstation talks to it. I don't think, you know, you know, we don't, we, we don't need to spend a million dollars to try to secure that one server when we can just put windows firewall in and, and only allow that one IP, that's, you know what I mean?

It's like, but being able to then answer them with,, a, a solution that meets their needs. But isn't a knee jerk reaction to whatever's happening outside the organization, but all of that needs all, all of any of that needs no matter how much, how much knowledge I have in tech you know, technical experience. If I don't have visibility, I can't make any cases to say yes or no to anything, then whatever comes in hot is, is the priority, right? Or whatever we see on the news becomes a priority because I don't have the visibility to say we only have two systems that are even running that application.

And they both, you can't even get to them from the network, whatever, you know what I mean? They're, they're, you know, MZ. So, so, but if I have the visibility, I can quickly kind of educate with reality, the business on why that security control. It's ironic security team is saying I don't, we don't really need that security control.

Craig: As you're getting the visibility and starting to have those conversations, what, you know, a lot of times you'll, we'll hear that the, you know, the noise level starts to come up as you see all the logs and you're starting to open up and, and, and see kind of like the, the alerts that are coming through. What processes do you put in place to prioritize the, the the noise coming in? Are you using like any kind of, you know, ML or A I to prioritize the way that you're, you're looking and hunting for threats?

Todd: That, that's where the having me on the team is valuable. It's, it's 30 years of experience looking at it and just like I said, just when I, 

Craig: I mean, I, you can't skill yourself like you can't look at all these alerts.

Todd: Well, I, yeah, I agreed. But what we do is tune very well. So, so, so we, we figure out what the pattern is of what's the normal behavior, get them to a filter. So then it only pops the, the, the things outside of the norm, right? But it, it's, it's, we haven't got to the point where we're using a, a any of those kind of tools to replace the, the human jet.

But we're, we're leveraging,, you know, the, what, what, what we can with our SIEM (Splnk) to help bubble up what are the recurring themes of this system or this user account always is running these, you know, what look would be suspicious commands. That's, that's whitelisted. We can whitelist that out because it's, again, we get very surgical. But we can only be surgical if we have again, the, the right data that lets us see the path, right?

And you know, that's, I guess a skill that I have and a skill that are on some of my team members that, you know, seeing the patterns. So, so, so then we can filter the noise. But, you know, we, we have a which, which does sound like, I mean, it's an ongoing bt the rule sets and it's grind, I mean, and that, that, that, you know, it just, it takes the right team members that are willing to do the grind, right?

It, it, this isn't, this isn't, you know, like firefighting, it's like, ok, some days you're just, you're gonna respond to a lot of events that aren't a real fire or they're very minimal and it'll be very grinding because then you have to roll up all the hose, you're gonna do all this stuff and, you know, every so often you'll get one of those ones that,, make you feel like, ok, this is why I'm here. This is my purpose. I'm going to help save a life for, you know, you know, what I mean? But it isn't happening.

Craig: You don't want those firefighters doing all the boring report writing, spending half their time writing reports and those kinds of things. So you want to get everything out of the way that you can from automation and standards and stuff like that.

Todd: See, see, and that is a challenge. I mean, there, there's a, what do they call the, where it looks like the mind map, there's a mind map out there of the CISO’s responsibilities. Like there's one, yeah, it's like security operations then it's all these other bubbles and it was like one of those, what it feels like is, hey, Cyber, you do that.

And I was like, just, yeah, I'll take it on and then eventually I'm like, what did I just accept? So, so eventually it has to get to the point where, I mean, I don't think the physical security team is off doing, you know, audits of buildings and stuff like that. They're usually former, like FBI or, you know, some three agency and their focus is, you know, prevent a physical breach. And, and, and that's, that's all they do. Their, their security guards are not off doing other things. They're, they're, they're watching the screen watching the gates. They're, they're just looking for breaches. 

Craig: It's interesting. So, I was at a,, a meeting with a CISO of a large public company and somebody asked him the question, what keeps you up at night and he said, not getting hacked and not going to jail.

And,, you know, so like, step one, let's not get hacked or, you know,, do what you can to mitigate the, the disruption. But, but now you have this other thing of the, of the security and the earth, of the compliance perspective and it seems like every week sec or FDIC or all these, these agencies are coming out with new rules and laws that you kind of, that you just have to keep up with.

Todd: And II I don't think, is that, do they think that's gonna help the matter is, is by? And do they understand that CISO S typically work hand in hand depending on the size of the organization with the CIO and the CIO S team., or IT really deploys most of the security preventive controls.

Craig: What, what, what do you think like from, there's, there's no lack of work to go around depending on how low you go on the priority list. It's infinite work. But what do you think from your point of view are the, the, the talent bottlenecks in the security organization?

Todd: The, well, the, the, so, so the funny you mentioned that I'm, I'm, I'm trying to do something on side that'll help educate more people. I've actually given away free training. But, but, but I the, one of the biggest issues that I think is happening in Cyber, well, happened in Cyber again just through kind of, we'll call it organic growth was the, the decision of and, and I'm part of it because I, you know, I've been in Cyber forever where we decided to put the, the most rookie individuals as the people who do the diagnosis.

And I mean, you wouldn't do, you wouldn't want to do that. That, that doesn't even happen in a fire and a fire. The chief, not the chief, the, the captain who's in the front seat usually will do the 360 to determine. Ok, here's our plan of attack while everybody else is getting the equipment ready. I, I can't imagine going to a hospital and,, you know, the first year, you know, med student is doing the diagnosis of, hey, you either got cancer or you don't.

It's usually the more senior people do the diagnosis and then they hand off and say, ok, go, go take it from here. That's not what we do in cyber where cyber is, Hey, I've been doing this for five years. It's now your turn, rookie. I'm just throw it over the fence when it's something good. That, that's a very flawed assumption because then you assume the rookie will know what malicious looks like, even though they probably never seen it, they've never been part of a forensic investigation.

So, so for them it's either, you know, after a while it's like, this just kind of looks like the other,, cert U till Command. It's like, no, no, see what it did. It downloaded something from the internet, none of the other ones do that. It's like, ho how will they get that experience? And even if an incident goes down, it's usually, then they bring in, you know, most people bring in a third party, right?

So now they don't even get that experience of, ok, what should I have looked for? What did you guys, what were you guys were doing? The forensics? What did you look for to know these 15 other machines got hit, right? It's like now that experience comes in and goes out with a small engagement, right? And, and then I think that somehow we got to be able to train them better and get them more repetitions on what malicious looks like where it hits just like, you know what I can imagine?

Well, I see it on TV. I almost, I almost became a doctor. That's fun. That's a different story. But where, where the doctors standing there, the senior and they have all the first year kind of,, residents standing around a patient and like, ok, what do you see? What do you see? What do you see? What do you see and kind of checks for the first, what, two years they have to go through the residency of, well, you missed this good, good eye on you.

You got, you saw this symptom, but you guys missed this symptom that you would have gave the wrong medicine and you might have caused some kind of heart issue. You know what I mean? That they, they get multiple, you know, two years worth of diagnosis training and we don't do that in cyber.

So, so I think that's if, if we want and maybe AI and stuff like that will help where they can go ask, hey, is this a suspicious use of cert till?

And it says, yeah, it is, it's like, OK, then I better escalate, you know, because they don't like, they don't have anybody to ask if they keep escalating things and they're saying, oh, that was a false positive. It's like, OK, I, now I feel pressured to escalate the right thing.

Craig: It's almost like the, the way that the organizations are evolving end up creating more silos and drive the need for more automation because the, you know, the front line analysts don't have the experience.

Todd: I mean, I, I would not want to go to a first year doctor, if I thought I had something serious, you know what I mean? No, I'm gonna go to somebody that's diagnosed this correctly for years and we pinpoint really what the issue is. So I don't have to come back six months later when the symptoms get so worse. And they're like, oh, we should have caught it six months ago, sorry. I mean, that's exactly what happens with cyber. There was probably alerts early on.

Craig: Well, often it, you know, some of the reasons why there are hacks is because people even on, you know, on help desk who are very green, they forget a process or procedure and reset a password and, oh, sure, I’ll reset your MFA while I'm at it. No problem. And then next thing, you know, somebody's, you know, your systems. And so it's a lot of it has to do with just experience gap and there's such a dearth of experience in cyber, you know, plus the demand is going way up that, I mean, that really the only way is you have better training and have better automation on some of this, more complex stuff.

Todd: Oh, yeah, they're going in the right direction where companies are putting that kind of what, whatever they want to call it. Basically. Sidekick A I or you might nudge it, hey, you know, it's almost like they got a team next to them. That's not somebody else that's saying, hey, I got other jobs to do. I, I only do forensics, you know what I mean?

Craig: Yeah, that's the biggest thing. And, you know, the data is always everywhere. And you have to do, you know a lot of jumping around and if you can pull it together, you can do some more. We have we're obviously working on that as well as a sidekick and there's just so much demand for asking, asking the interface English questions. Hey, is this a problem? Where does the problem lie, you know, from operations, what's going on with this with this transaction from security, you know, and ask different questions here and there.

Todd: And so that's like,, definitely something I think is gonna help. I, I can see, yeah, I can see it long term. It's gonna be, it's gonna be that experienced trainer that can interact with them and, and kind of guide them. So they, they, they catch it the first time because again, a lot of what, what I've seen is, there, there's a very, very well known incident responder who does,, that I won't say his Twitter handle, but they said like 80% of the cases that he's investigated in like over a five year period. There was logs early on,, like a V logs that said, hey, I tried to block this but couldn't or whatever. So, so it wasn't like, you know, these, these attacks go unnoticed, they just go, diagnosed incorrectly.

And then once they get, like we talked to, well, you kind of mentioned about getting credentials and stuff like that, once you get credentials, whether it's MFA codes and stuff like that or whatever, then it's much harder to find. It's, it's almost like now the fire got in the walls. I mean, I've seen houses where the fire was burning hot in the walls and you could not tell, but it was going up to the next floor. 

Craig: It's like, once it gets to be more like UBA type stuff. Which is some complex, you know, more complex and, or more advanced alerting. But there's a lot of ways to do it to,, you know, there's a lot, basically you need, there's many ways to, to chase the intruder around but they can get in any, you get, if they get credential, then, then it's a whole different story.

Well, I know, you know, this is really interesting. I love the, I love the, the fire analogy, I think it, it applies so well and the building codes and the risks, I think it's really interesting the risk versus threat idea. The experienced analyst is the, the, the person that has stretched the most thin in an organization or there's the, the, the biggest need for them is that fair to say?

Todd: Yeah, in a way they become kind of a crutch where the other ones don't get it because they, they can get the answer so quickly from the person. That, that becomes the shortest path, right? Because, hey, there they're still, you know, everybody kind of sometimes treats those incidents as tickets. So people are tracking how quickly they get closed. So, so there's this, you know, in the back of people's minds is, well, the quickest path to get this resolved is go ask the expert., and then the expert will say, yeah, no, this is why and who knows if they'll remember that 15, you know, 50 alerts later because they, they, they met their mission. They, they were able to properly answer the question, not diagnosed but come up with an answer to where they could close the ticket or escalate it.

Craig: So that's awesome. Yeah, makes a lot of sense. And it's a kind of goes, you know, the thing that I was curious with is how do people do it? Because we, we can't,, we don't have the option of not doing it at all. You still have to do the regulations.

You still have to do the compliance reports and you can't get hacked,, in a, you know, or you have to stay on top of it. Something will come, but you got to know that you can respond quickly so you can absorb those shocks. And, it's good to hear some of your feedback around like moving up, that maturity model, get some visibility, and start to automate what you can so you can get some time back for those really experienced folks who are out there hunting threats.

Todd: Well, well, and hopefully that that organization can like there, hopefully there's a risk and compliance department, you know, internal audit can kind of take some of some of that weight of the guidelines and standards off the shoulders of the firefighters. You know what I mean? It's, it's, you know, that the cities do it.

Well, they, you know, they have a building inspection team and firefighters and police officers and they all kind of,, have their, their priorities and their focus

Craig: But it's different personality types as well.

Todd: It's, you know, the firefighter, different skill sets definitely de definitely different mentalities of they, they, they're comfortable and things just escalating really quickly and then eventually dying down and then going silent for a while and escalating and then, you know what I mean? Where the other ones are, were even keeled as far as here's the 27 compliance or,, inspections. I have to do these homes this week. I'll get them all done, you know, five a day, whatever, and have a slow day on Friday. So, so, so it's a different mentality because firefighters, police officers, doctors, we really can't plan our day. Same with incident responders. It's really, it's almost,, outside driven of what accidents are occurring.

Craig: So it makes all the sense based on,, personality types in the, in the different roles and just skill sets and, and so, well, I really appreciate it. We've, we're coming up on time here, but before, before we,, we wrap, do you wanna share some takeaways on,, on our discussion today and what cybersecurity professionals can, implement from, from some of your experience?

Todd: Well, it's, it's get visibility as soon as possible. Before you really try to put in controls, validate that visibility is what you're looking for where, where you might need to even bring out and bring in an outside resource that can do some red teaming or some some validation, some activity. So, so you can really get a sense of, do we see what we need to be seeing? So, so, you know, getting that handle on what are the most important alerts for our environment? And are they being triggered? And then working with the business? Where can you put in some tighter security controls?

Because because to me, when, when I look at prevention, again, I don't think we, you know, in or the security team can prevent this breach. The goal is if we put in the right controls, detective controls or preventive controls, excuse me, if we put in the right, preventive controls, then the, the alerts,, spun out from preventive controls won't, won't be,, such a high number that she'll be overwhelm the security team. It's really about preventing the security team from missing the important alerts.

So, you know, kind of knocking out kind of the commodity malware and all this commodity like a risk-based alerting. So, so, so, so then you can focus on the alerts that really will indicate that something has breached the organization and you need to contain it quickly. So, so it's really about it's really about focusing on priorities. But with visibility, then you can make business cases on where you can put in security controls to again help minimize the attack surface that the right actors will take advantage of.

Cory: Thanks for listening to this perspectives podcast by Splunk. Be sure to subscribe to this show on whatever platform you're currently using. Speaking of podcasts, you should also check out the security detail podcast by Audra Streetman and Kirsty Payne. They explore cyber threats across a variety of industries with some of the most trusted names in cybersecurity.
And don't forget to check out splunk.com/perspectives for blogs featuring the latest executive takes on today's security and technology topics by leaders and for leaders. Thanks for listening.