I did the unthinkable yesterday. I combed through my posts for non-spam comments. I apologize to everyone whom I didn’t answer – we get a lot of comment spam that I have to wade through when I do this. However, there were a couple of requests in there for future topics and I’ll try and cover those requests in the next few weeks.
The first request was for monitoring scheduled tasks. I’m going to read this as “given a Windows host, how do you determine what scheduled tasks are enabled and whether they are failing or succeeding?”. That’s a tall order, so I looked to my favorite tool – PowerShell – for the answer.
PowerShell v3 has a bunch of cmdlets that manage scheduled tasks. The first – Get-ScheduledTask – gets a list of scheduled tasks along with some information about them. Looking at the Get-Member results, we see the following:
PS> Get-ScheduledTask | Get-Member TypeName: Microsoft.Management.Infrastructure.CimInstance#Root/Microsoft/Windows/TaskScheduler/MSFT_ScheduledTask Name MemberType Definition ---- ---------- ---------- Clone Method System.Object ICloneable.Clone() Dispose Method void Dispose(), void IDisposable.Dispose() Equals Method bool Equals(System.Object obj) GetCimSessionComputerName Method string GetCimSessionComputerName() GetCimSessionInstanceId Method guid GetCimSessionInstanceId() GetHashCode Method int GetHashCode() GetObjectData Method void GetObjectData(System.Runtime.Serialization.SerializationInfo info, Sys... GetType Method type GetType() ToString Method string ToString() Actions Property CimInstance#InstanceArray Actions {get;set;} Author Property string Author {get;set;} Date Property string Date {get;set;} Description Property string Description {get;set;} Documentation Property string Documentation {get;set;} Principal Property CimInstance#Instance Principal {get;set;} PSComputerName Property string PSComputerName {get;} SecurityDescriptor Property string SecurityDescriptor {get;set;} Settings Property CimInstance#Instance Settings {get;set;} Source Property string Source {get;set;} TaskName Property string TaskName {get;} TaskPath Property string TaskPath {get;} Triggers Property CimInstance#InstanceArray Triggers {get;set;} URI Property string URI {get;} Version Property string Version {get;set;} State ScriptProperty System.Object State {get=[Microsoft.PowerShell.Cmdletization.GeneratedTypes...
You can see from this that it’s just getting the information from WMI (CIM is the new WMI in PowerShell v3 and above). Thus, we can easily get a list of the scheduled tasks using the following script:
Get-ScheduledTask | Where State -ne "Disabled" | Select TaskName,TaskPath,Source,Description,Author,State,URI,Version
That gets us the first part of the problem. Now we need the second part – how do we know when they ran and the status of the last run. There is another cmdlet for this: Get-ScheduledTaskInfo. We can run this by using the following script:
Get-ScheduledTask | Where State -ne "Disabled" | Get-ScheduledTaskInfo | Select TaskName,TaskPath,LastRunTime, LastTaskResult,NextRunTime,NumberofMissedRuns
To actually implement a monitor for scheduled tasks, I would schedule these differently. My inputs.conf (using the handy SA-ModularInput-PowerShell) would look like this:
script = Get-ScheduledTask | Where State -ne "Disabled" | Select TaskName,TaskPath,Source,Description,Author,State,URI,Version schedule = 0 30 2 ? * * source = PowerShell sourcetype = Windows:ScheduledTask script = Get-ScheduledTask | Where State -ne "Disabled" | Get-ScheduledTaskInfo | Select TaskName,TaskPath,LastRunTime, LastTaskResult,NextRunTime,NumberofMissedRuns schedule = 0 45 * ? * * source = PowerShell sourcetype = Windows:ScheduledTaskInfo
The first input stanza runs at 2:30am local time and the second input stanza runs every 60 minutes. Our list of scheduled tasks won’t change very much, so let’s create a lookup to enhance our work. This will turn a host, TaskName and TaskPath into the associated information. The search to run is this:
sourcetype=Windows:ScheduledTask | stats latest(Source) as Source, latest(Description) as Description, latest(Author) as Author, latest(State) as State, latest(URI) as URI, latest(Version) as Version by TaskName,TaskPath,host | outputlookup WindowsScheduledTask.csv
As normal, enter this all on one line. Turn this into a lookup (either through the manager or via the configuration files) and you are ready to go.
There are three things we can with the scheduled task information. Each will require its own search.
The two interesting ones are the failed tasks and missed tasks. Failed tasks can be found by looking at the LastTaskResult. The LastTaskResult is 0 on success and an error code otherwise. Run this search over the last 60 minutes:
sourcetype=Windows:ScheduledTaskInfo LastTaskResult!=0 | lookup WindowsScheduledTask host,TaskName,TaskPath OUTPUT Source,Description,Author,URI,Version | table host,TaskName,TaskPath,Description,Author,URI,LastRunTime,NextRunTime
The missed tasks search uses the NumberOfMissedRuns instead:
sourcetype=Windows:ScheduledTaskInfo NumberOfMissedRuns!=0 | lookup WindowsScheduledTask host,TaskName,TaskPath OUTPUT Source,Description,Author,URI,Version | table host,TaskName,TaskPath,Description,Author,URI,NumberOfMissedRuns,LastRunTime,NextRunTime
I mentioned earlier that the Get-ScheduledTask series of cmdlets use the CIM/WMI underneath. However, they apparently only work on NT 6.2 and above; also known as Windows Server 2012 or Windows 8. Unfortunately, this is one area of Microsoft land that changes frequently. For earlier versions, there is a WMI interface (Win32_ScheduledJob) that can be used, but it provides different information. Also, there is a log file that is maintained by the scheduler (C:\Windows\Tasks\SchedLgU.txt). However, the log file has an issue – it is exactly 32Kb in size and the system locks it and overwrites the contents constantly. Once it gets to the end, it starts at the beginning of the file again. This is good for diagnosis, but not good for monitoring purposes. Hopefully Microsoft will maintain the PowerShell cmdlets “as is” for future versions of Windows!
The Splunk platform removes the barriers between data and action, empowering observability, IT and security teams to ensure their organizations are secure, resilient and innovative.
Founded in 2003, Splunk is a global company — with over 7,500 employees, Splunkers have received over 1,020 patents to date and availability in 21 regions around the world — and offers an open, extensible data platform that supports shared data across any environment so that all teams in an organization can get end-to-end visibility, with context, for every interaction and business process. Build a strong data foundation with Splunk.