Windows Event Logs in Splunk 6

Quite a while ago I wrote a blog post entitled The Splunk App for Active Directory and How I tamed the Security Log. It detailed how to limit the amount of data that was going into the Splunk index through filtering. I included two techniques – firstly, filtering by event code so that you didn’t include the events you didn’t want; and secondly, filtering the explanatory text on the end of each event. Splunk 6 makes this so much easier that the prior blog post is not even relevant any more.

Let’s say you don’t want firewall events. From the previous blog post, event ID 5156 and 5157 detail the firewall connection accept and deny messages. Let’s say those are not relevant to us. Previously, we had to add a props.conf stanza to initiate a filtering action that was done in transforms.conf – it was complicated. In Splunk 6, everything is done in inputs.conf. Here is a new inputs.conf stanza for you:

[WinEventLog:Security]
disabled = false
blacklist = 5156-5157

There are two new parameters you can specify – the first, shown here, is a black list of all the event IDs you don’t want to monitor. You can use ranges (as I did here), or comma-separate the event IDs or event comma-separate ranges of event IDs. The second parameter is a whitelist – if you have more that you don’t want to keep than you want to keep. It follows the same format.

The second facility I wrote about was suppressing the explanatory text. Splunk 6 makes this easier as well. Let’s take a look at a typical windows event prior to the text suppression:

10/14/2013 08:29:33 AM
LogName=Security
SourceName=Microsoft Windows security auditing.
EventCode=4634
EventType=0
Type=Information
ComputerName=SP-SQL.bd.splunk.com
TaskCategory=Logoff
OpCode=Info
RecordNumber=3544
Keywords=Audit Success
Message=An account was logged off.

Subject:
	Security ID:		BD\a-ahall
	Account Name:		a-ahall
	Account Domain:		BD
	Logon ID:		0x5886A

Logon Type:			3

This event is generated when a logon session is destroyed. It may be positively correlated with a logon event using the Logon ID value. Logon IDs are only unique between reboots on the same computer.

You see that “This event is generated…” text – that’s the explanatory text. It’s the same for every single event. Since these events get generated every 10-15 minutes for every single user on your domain controllers and they are 100+ bytes, you can see how they can add up. And that’s just one example. Every single security event has similar explanatory text. In Splunk 6, you can add a new parameter to your inputs.conf stanza to supress the Message field:

[WinEventLog:Security]
disabled = 0
suppress_text = 1

Now when you get those events, this is what they look like:

10/14/2013 08:43:07 AM
LogName=Security
SourceName=Microsoft Windows security auditing.
EventCode=4672
EventType=0
Type=Information
ComputerName=SP-SQL.bd.splunk.com
TaskCategory=Special Logon
OpCode=Info
RecordNumber=3546
Keywords=Audit Success
Message=

You will note that is NO message text at all. This is fine for some logs (usually custom service logs) where the message is not important. However, you will still need to use the same transform as before if you want some of the message but not all – for example, with the Security log. Since all the stanzas of the same name are munged together, you should be careful about setting the suppress_text parameter. In particular, do not set the suppress_text parameter on WinEventLog:Security as it will not log any of the important contextual information within the security log.

These two changes can make your windows event log gathering more efficient, but as always – be careful of what you throw away.