How to Use Splunk Security Solutions to Improve Incident Response: Lessons Learned from the GE Digital Predix SOC Team

As the responsibilities of the Security Operation Center (SOC) continue to increase, SOC teams are experiencing increased demand on their time and resources. Scaling a security team with little resources and funds can prove extremely difficult, especially when the incident response team spends most of their time chasing alerts. This alert fatigue is compounded by a global cybersecurity talent shortage which leaves many teams understaffed, making them more reactive than proactive, reducing their ability to collaborate and lack collaboration and therefore leaving their organizations at risk. 

But even in the face of increased demand on reduced teams, Incident response doesn’t need to be reactive. Strong security operations teams have learned to use the right technology to enable them to scale, while keeping their focus on people, process and outcomes and ensuring their ability to investigate, monitor, analyze and act when faced with  security threats. 

The Challenges

Before the GE Digital Predix SOC team began to use Splunk Enterprise for incident response, individual members on the team worked in silos. One team member had the job of reviewing and analyzing all the forensic evidence collected by the entire team, including memory images, hard drive snapshots and multiple operating system logs. The snapshot timeline from a single instance could contain over three million lines or events. No matter how good they might be, it was too big a job for one person. The team needed a better way for all five of them to collaborate. 

Team member Alina Dejeu took on the challenge and soon found herself investigating Splunk solutions. First she looked at available incident response apps on Splunkbase. GE Digital Predix’s cloud infrastructure consists of 90% Linux instances. Most of the apps in Splunkbase are designed to work with Windows forensic data, but Alina was able to create her own app using Splunk Enterprise knowledge objects such as lookup tables to perform categorization and risk scoring of data.

Working with her colleague David Rutstein under the guidance of her manager Chris Silveira, Alina’s began parsing sample data they normally collected for incident response one-by-one. Then they developed rules and logic to use with Splunk solutions. When Splunk found an artifact that matched one of the rules, that data would be highlighted. For example, in the screenshot above, the team was alerted to suspicious IP address activity on a vulnerable Jenkins server that had been exposed to the internet.

The Impact

This significantly improved the team’s investigation process in two ways. The team is now able to correlate multiple forensic data sources, pulling information together and highlighting red flags while adhering to the organization’s security best practices. The team can also quickly identify malicious activity with multiple team members able to view this information in one place, substantially reducing the possibility of human error. 

Watch the full session for a demo of the SOC team’s app and how leveraging Splunk Enterprise allowed them to correlate disparate data and solve other security use cases.

Want more? Check out the .conf19 session, "Using Splunk in Automating Forensic Investigations in AWS."

----------------------------------------------------
Thanks!
Jade Catalano