Windows, Perfmon and Internationalization

When we write apps within Splunk, we are generally working with a US English focus. People don’t write logs in multiple languages, after all, so we generally don’t have to worry about multiple languages in the core applications that we write. Except, that is, for Windows. Specifically, perfmon data is delivered localized for the various languages that Windows runs under. (Windows Event Logs are also delivered localized, but this post is specifically about Perfmon data). If you have a US English version of Windows and you want to do a time chart of the percentage of the processor used over the last 24 hours, you might do a search like this:

index=perfmon object=Processor counter=”% Processor Time” 
| timechart span=10m avg(Value) by host

However, when you are using a French version of Windows, you need to do this:

index=perfmon object=Processeur counter=”% Temps Processeur” 
| timechart span=10m avg(Value) by host

Same thing – different language. How are we meant to deal with the same thing in multiple languages? The best method I have come up with involves a two-step process:

1. Convert the inputs.conf so that it is retrieving the localized version of the perfmon counters
2. Adjust the searches to do a lookup based on what I want

I’m going to focus on the second in this article. My method is to use a lookup on the object and counter. I first of all set up a lookup table. This is a CSV file that I write that looks like this:

object,counter,l_object,l_counter
Processeur,% Temps Processeur,Processor,% Processor Time

Add a line for each combination of object and counter that you want to handle. Note that the object and counter that we are receiving are on the left and the non-localized versions are on the right. We set up the lookup in transforms.conf:

[TranslatePerfmon]
filename = TranslatePerfmon.csv
max_matches = 1

Now we can apply the lookup automatically to all Perfmon data with a props.conf entry:

[Perfmon:*]
LOOKUP-perf = TranslatePerfmon object counter OUTPUT l_object l_counter

Now, instead of using the object and counter fields, we can use the l_object and l_counter fields, so our search becomes:

index=perfmon l_object=Processor l_counter=”% Processor Time”
|timechart span=10m avg(Value) by host

Note that this only works if the specific combination of object and counter are available in our lookup file. What about the ones that aren’t? In this case, we need to correct with an eval statement. In version 5.0 of Splunk, we can create evaluated fields to create a copy of the object and counter into l_object and l_counter. Since this is done prior to the lookup, the lookup will overwrite our evaluated fields later on. Our new props.conf entry looks like this:

[Perfmon:*]
EVAL-l_object = object
EVAL-l_counter = counter
LOOKUP-perf = TranslatePerfmon object counter OUTPUT l_object l_counter

Now every single perfmon event will have an l_object and l_counter. Of course, you still have to do the localization file – TranslatePerfmon.csv must be produced for every language you want to support, but you can produce a common file that translates all the languages at once. For instance, you could do the following as the contents of the CSV file

language,object,counter,l_object,l_counter
en_US,Processor,% Processor Time,Processor,% Processor Time
fr_FR,Processeur,% Temps Processeur,Processor,% Processor Time

Here you can see I am supporting both English and French together. I could easily add German, Italian, Spanish and Portuguese to this list. I could also add other objects like Memory, Network Utilization, Logical Disk and Physical Disk. You just need to add appropriate entries to the CSV file.

If you use this technique on one of the Splunk apps – Exchange, Active Directory or Windows – note that you will need to go through several files, including macros.conf, eventtypes.conf, savedsearches.conf and each view in order to change all the references.

Fortunately, most Windows Server applications that introduce new perfmon counters do not localize the counters, so you really only need to support the base Windows counters. Unfortunately, there are a lot of them!

Care to assist? We won’t be able to produce every single language ourselves. If you want to help, then send us your counters. You can obtain a counters.txt file by executing the following PowerShell command on a suitable Windows Server:

(Get-Counter –ListSet *).Counter | Out-File counters.txt

Then send the counters.txt file to microsoft@splunk.com – don’t forget to tell us what language the counters are in! I will compile all the responses we get and publish in a Splunkbase app in the future.