CI/CD Detection Engineering: Splunk's Attack Range, Part 2

At a high level, the attack_range architecture looks like this:

We will be leveraging the Terraform mode and constructing our range on an AWS VPC. We recommend to use this mode by default for a few reasons, but primarily because you can run a more complex complete environment on AWS EC2 versus your local machine. Currently, Vagrant local execution limits the size of the environment to the number of resources your local machine has. In a typical MacBook Pro with 16GB of ram, you are limited to successfully running a range with a domain controller or windows client and a Splunk server. Adding any more instances like a kali Linux attacking server, or multiple windows domain controllers joined by a domain will cause you to run out of memory. 

To build an attack_range using terraform  the first step is to install dependencies and configure your local machine. There is a great wiki page that walks you through the configuration for Terraform in a MacOS environment in detail. Although to quickly get started, I will share a simple bash script that works on any Ubuntu host that will install all tools and dependencies necessary:

#!/bin/bash

sudo apt-get update

sudo apt-get install -y python3-dev python-dev unzip python-pip awscli python-virtualenv

wget https://releases.hashicorp.com/terraform/0.12.28/terraform_0.12.28_linux_amd64.zip

unzip terraform_0.12.28_linux_amd64.zip

sudo mv terraform /usr/local/bin/

git clone https://github.com/splunk/attack_range && cd attack_range

cd terraform

terraform init

cd ..

virtualenv -p python3 venv

source venv/bin/activate

pip install -r requirements.txt

Simply save this script in your Ubuntu machine and make it executable and then run:

chmod +x install_attack_range.sh

./install_attack_range.sh 

After all the dependencies are installed we need to generate an AWS Key Pair in your desired region.  To do this run the following command:

ssh-keygen

This will create two files: a private key under ~/.ssh/id_rsa and the public key ~/.ssh/id_rsa.pub next import the id_rsa.pub publike key into AWS, you can simply follow the instructions provided by AWS here to accomplish this. ⚠️ Make note of the key name you will need it in a second, in the attack_range documentation we default to attack-range-key-pair as the name. While in AWS make sure you create a user with programmatic credentials.

Provide this new user EC2 administrative access and save the access key ID and secret access key.

Next back in the Ubuntu server to configure our newly created AWS credentials run:

aws configure

You will be prompted to provide the access key ID and secret access key as well as your default region, make sure all these match your previous choices. You are doing awesome! We are almost done 😁. Finally, edit attack_range.conf and set your key name, region, and verify your private key path. It is worth reviewing some of the major configurable parts 🧰 of the attack range. The attack range allows you to:

• Configure the machines in your environment
• Change the Splunk, Windows, and Caldera passwords, ⚠️ highly recommended!
• Install your own Splunk Enterprise, Universal Forwarder, and Apps/TA versions. 
• Apps are stored in a public s3 bucket hosted by the Splunk research team by default, but this can be configured by changing the s3_bucket_url parameter. 
• Install different versions of Splunk BOTS data sets out of the box 
• Install Splunk Enterprise Security, note the App needs to be saved under attack_range/apps/splunk-enterprise-security_<version>.spl. 
  
  

These are just highlights, it has many more capabilities and I encourage you to scan the configuration file. Otherwise please do open an issue in the project. It makes us extremely happy ❤️ to get feedback.

Once you are happy with the configuration run the build command:

python attack_range.py --mode terraform --action build

The typical build takes roughly 11-15 minutes (about 30 mins if installing Splunk Enterprise Security), this might be a good time to grab ☕️. If all went well at the end you should see a list of your machines like below:

If you have made it this far you have now built your first attack_range 🎉. Explore each machine by connecting directly to them. For Splunk use a browser and open http://<splunk_server_ip>:8000. For the windows server use an RDP client to connect to port 5389 and authenticate with the user name and password configured in attack_range.conf. Now that you have a fully running attack_range and have made yourself familiar with the environment lets launch our first attack!

Running an Attack Simulation ⚔️

By default, the attack_range supports two Adversarial Simulation Tools, MITRE’s Caldera and Red Canary’s  Atomic Red Team. Both are open-source and free to use. Caldera is commonly used by our team for its ability to execute attack “operations” instead of individual sets of attack techniques. Today we are going to focus on Atomic Red Team, and more specifically emulating Credential Dumping techniques T1003.002. To simulate this technique run:

python attack_range.py -m terraform -a simulate -st T1003.002 -t attack-range-windows-domain-controller

A few things happen when simulating a technique under the hood. The attack_range installs the Atomic Red Team execution framework to the target machine. It grabs all the available attacks AKA “atomics” that correspond to a specific MITRE Technique from the Splunk fork. By default, the attack_range is configured to download the atomics from our Splunk fork 🍴. There are three atomics under T1003.002 Security Account Manager:

• Registry dump of SAM, creds, and secrets
• Registry parse with pypykatz
• esentutl.exe SAM copy
  
  

Each of these three different ways of obtaining the “credential material from the Security Account Manager” is executed on the target host. Once the simulation is completed you should see the following output:

You have executed an attack simulation against a windows domain controller 😈 ! Now let's review what kind of data we gathered from this attack and specifically how to detect it. First, we need to go back to the Splunk server. If you ever need to see what machines are running just execute:

python attack_range.py -m terraform -lm

There are a handful data sources that are collected from the attack_range and they all land in their respective indexes, here is the current list:

• Windows Event Logs (index = win)
• Sysmon Logs (index = win)
• Powershell Logs (index = win)
• Network Logs with Splunk Stream (index = main)
• Attack Simulation Logs from Atomic Red Team and Caldera (index = attack)
  
  

Let's try to detect atomic number one, Splunk’s Security Content has detection for this one 😁.

Detecting Credential Dumping via Reg.exe 🛡

The Splunk Threat research team develops security detections built when possibly using data from an attack simulation. In this case, the detection we mentioned earlier Attempted Credential Dump From Registry via Reg exe is a great example of this, to test it manually simply open the ESCU App on the Splunk server. Navigate to the Analytics Story details tab and look up the "Credential Dumping" story. When running the detection search you should see various attempts to run reg.exe to dump various registry hives as described in the atomic. Below is an example:

Feel free to experiment with running various techniques and their associated detection. Although we haven’t tested all of them there are about 144 detections to try in security-content that are tagged with corresponding MITRE techniques.

How Much Does It Cost Me?💰

The attack_range software is an open-source project and free, but there is a cost to running an environment on AWS. It is important to understand the cost of running the attack_range for detection testing. We have gathered some cost estimates based on the Threat Research team usage of the range on a day to day basis. By default, the attack_range builds a Splunk Server and Windows Domain Controller. There is an AWS price calculator that allows us to extrapolate our cost for this kind of deployment is ~ $260 monthly or $8.00 daily. This estimate assumes a daily sporadic workload of 3 hours at a time which is the average time a Threat researcher works on a specific range at a time. This is an estimation and may vary completely on your specific usage.

How to Automatically Test All Detections 🔬👩‍🔬👨‍🔬

By now we have learned how to ship our security-content detections in part 1 of this series, and how the attack_range can be used to manually test detections with Atomic Red Team. In part 3 we will focus on how we can use a CI/CD pipeline to automate our detection testing and also many of the tests required to prepare a security-content package. Finally, I want to applaud 👏 the CALDERA team, you have built and shared with the community a really special tool, thank you. I am also very thankful 🙏 for the amazing Carrie Roberts and the Atomic Red Team community for continuing to grow and care for the project!