Digital Resilience Pays Off
Download this e-book to learn about the role of Digital Resilience across enterprises.
New Splunk Tools for .NET Developers
By
Splunk
Today we’re releasing a new suite of tools for .NET developers so you can supercharge your .NET development with Splunkl!!
CC image Supercharger by Eaday on Flickr
This release is a continuation of our commitment to provide developers a rich platform for developing Splunk solutions.
- C# SDK 2.0 – A new, modern, C# SDK for building cross-platform solutions that consume Splunk’s API and/or which extend Splunk.
- Logging libraries – These libraries allow you to easily wire logging in your existing .NET applications to send log data to Splunk via TCP or UDP. It provides .NET Trace Listeners as well as sinks for the Semantic Logging Application Block (SLAB).
- Visual Studio Extension – This extension makes it really easy to get started with the C# SDK and to build Modular Inputs which extend Splunk.
C# SDK
The new SDK is designed from scratch to take advantage of the latest advances in the .NET Platform by including rich support for async/await, and the dynamic keyword. Introducing support for async/await means your custom apps can be more scalable and responsive. The SDK includes several nuget packages, Splunk.Client and Splunk.ModularInputs.
Splunk.Client
The Client package is used for connecting to Splunk instances and accessing the Splunk API. Below you can see I am using the client’s Service class to connect to a Splunk instance.
var service = new Service(Scheme.Https, “localhost”, 8089);
await service.LogOnAsync(“admin”, “changeme”);
Once I have logged on, I can then issue queries, for example below is a real time query searching for errors from within the last hour and going forward.
var results = await service.ExportSearchResultsAsync(
“search error”,
new SearchExportArgs {EarliestTime = “rt-1h”, LatestTime = “rt”});
foreach (dynamic result in results)
{
Console.WriteLine(result._raw);
}
In the code I am using the
ExportSearchResultsAsync method that will push results from the server continually as they are are available. I am then looping through the results and outputting each raw event. The result object is a
Dynamic object allowing any fields that Splunk has extracted to be accessed as properties.
The results variable above is a
SearchResultStream which also implements
IObservable<T>. This means you can use it with the
Reactive Extensions (Rx). Rx offers a push based programming model which fits well with Splunk’s real time manner. Additionally Rx provides a set of operators that you can use to act on the data as it is received in a declarative fashion for applying filtering logic, group by, ordering and more.
Here is a query that goes against Splunk’s _internal index and returns a large number of results
var results = await service.ExportSearchResultsAsync(
“search index=_internal”,
new SearchExportArgs {EarliestTime = “rt-1h”, LatestTime = “rt”});
//requires the Rx-Main NuGet package
results
.ToObservable()
.Sample(new TimeSpan(0, 0, 5))
.Subscribe(
Observer.Create<dynamic>(r => Console.WriteLine(r._raw))
);
Above I use Rx’s ToObservable method to convert the results into an Observable and then use the Sample method to extract a sampling of the data every 5 seconds.
In addition to doing searches, you can also use the client to send events directly to Splunk over HTTP. Below you can see how simple it is to send events.
var transmitter = service.Transmitter;
await transmitter.SendAsync(“Hello World.”, “main”);
await transmitter.SendAsync(“Goodbye world.”, “main”);
Finally, you can also use the client to perform common management tasks. In the snippet below, I am creating a new index on the fly, as it will be used to store the results of a query that my app issued by using the Collect SPL command.
var name = “temp_” + new Guid(); //temp index name
var index = await service.Indexes.CreateAsync(indexName);
Once I am done with the index I then easily delete it.
await index.RemoveAsync();
Cross Platform and Mobile / Device development
Something really exciting about the new Splunk.Client package is it is a Portable Class Library (PCL) allowing it to be used for cross-platform development. You can use it for .NET 4.5, Windows Store, and Windows Phone 8 development. Additionally thanks to the awesome power of Xamarin, you can use it for Linux, iOS and even Android development!
Below is a screenshot showing an iOS search app being developed in Xamarin Studio. This app lets users issue ad-hoc searches against a Splunk instance which are sent using the new Splunk Client. You can see on the left that the Splunk.Client PCL package has been referenced. The app is also using Rx to provide a responsive user interface. As events stream in the UI instantly updates without blocking.
Below you can see the Application running and doing a search for shopping cart related errors.
We also have created a Windows Store app using the SDK which we’re releasing on Github. Below you can see a screenshot:
Similarly in the code you can see the calls to the Splunk Client.
Yes, you can now build Splunk Mobile / Device apps with the Splunk C# SDK 2.0!
Splunk.ModularInputs
Using the Splunk.ModularInputs package you can create Modular Inputs that extend Splunk to pull events from custom sources. You could use this to talk to devices, interact with the operating system, or connect with external APIs like Twitter or Facebook.
As an example, in our samples for the SDK, you’ll find a sample input which monitors environment variables and triggers an event whenever they change. You can find the full sample code for this sample input here. To create a Modular Input, you create a .NET 4.5 console app and then include the Splunk.ModularInputs package. Next you change your Program class to derive from the ModularInput class and implement methods for returning the scheme, performing validation of the input parameters, and actually streaming the events.
The streaming logic is done in the new StreamEventsAsync method. Below is the implementation of that method for the environment variable monitor mentioned earlier. As you can see, similar to the Splunk Client, Modular Inputs in the new SDK are completely async.
public override async Task StreamEventsAsync(InputDefinition inputDefinition, EventWriter eventWriter)
{
int interval = 1000;
try
{
// if user didn’t give value for polling_interval, type conversion to
SingleValueParameter will throw
interval = int.Parse(((SingleValueParameter)inputDefinition
.Parameters[“polling_interval”]).ToString());
}
catch (Exception)
{
}
const string Seperator = @”://”;
int index = inputDefinition.Name.IndexOf(Seperator) + Seperator.Length;
string varName = inputDefinition.Name.Substring(index);
string lastVarValue = null;
while (true)
{
await Task.Delay(interval);
string varValue = Environment.GetEnvironmentVariable(varName,
EnvironmentVariableTarget.Machine);
// Event data can’t be null for real events.
varValue = varValue ?? “(not exist)”;
// Splunk does not record lines with only white spaces.
varValue = string.IsNullOrWhiteSpace(varValue) ? “(white space)” : varValue;
if (varValue != lastVarValue)
{
await eventWriter.QueueEventForWriting(new Event
{
Stanza = varName,
Data = string.Format(
“{0}, interval={1}, inputDefinition.Name={2} , varName={3}”,
varValue, interval, inputDefinition.Name, varName)
});
lastVarValue = varValue;
}
}
}
This method first retrieves a polling interval and environment variable from the input configuration. These were configured when the input was added in Splunk. Next the code loops continually monitoring the specified variable at the interval. If the variable changes then the QueueEventForWriting async method is invoke to write back an event to Splunk.
This is just one example of what you can do with Modular Inputs, the possibilities are only limited by your imagination!
Logging Libraries
A common desire we hear from developers is to make it easy to just send log data from their applications to Splunk directly without having to first write to a file. This is very useful both for development and production environments. Enter our new logging libraries! They provide everything you need to wire up your existing apps to log directly over UDP or TCP to a Splunk instance or forwarder.
The libraries include standard .NET Trace Listeners which popular OSS logging frameworks like log4net, NLog and Enterprise Library support.
Below you can see I am configuring .NET Tracing to use the new Splunk UdpTraceListener.
var listener = new TcpTraceListener(“localhost”, 9000);
Trace.Listeners.Add(listener);
Next I am modifying the log4net configuration to configure it to write to .NET Tracing.
<appender name=“TraceAppender“ type=“log4net.Appender.TraceAppender“>
…
</appender>
On my Splunk instance, I have created a TCP input listening on port 9000.
Now when I start my application, I see my log entries instantly pouring in!
Additionally the libraries also include support for the Semantic Logging Application Block (SLAB), which leverages the highly performant ETW infrastructure that ships in Windows. Here is a snippet of wiring up SLAB to send log data to Splunk over UDP.
var _listener = newObservableEventListener();
var sink = new SplunkSink(“127.0.0.1”, 9000);
_listener.EnableEvents(TestEventSource.Log, EventLevel.LogAlways, Keywords.All);
_subscription = _listener.Subscribe(sink);
The new logging libraries are not invasive making them easy to wire up in any of your existing .NET 4.5 apps.
Visual Studio Extension
In the first part of this post you’ve learned about new .NET libraries we’ve introduced for enhancing your Splunk development. We’re also introducing a new Visual Studio extension which will further streamline your development in Visual Studio 2013 for using them.
The VS extension includes the following:
- A template for creating a new .NET Project using the Splunk Client, and optionally using the new logging libraries to send log data over TCP or UDP.
- Snippets for performing common tasks using the Splunk Client, which you can use in any .NET application that references the SDK.
- A template for creating a custom C# Modular Input.
To install the extension, go download it from the Visual Studio Gallery here.
As I have previously installed it, I can see the available templates from the “New Project” dialog and select the SDK project.
Once I hit “OK” a wizard is displayed for logging configuration. I’’ve selected to use the SLAB event sink.
Next it displays a list of transports to choose from. I’ve selected UDP.
Finally, it generates for me a starter project with a sample implementation. The project pulls in the required NuGet packages as well. You can see below that it has configured SLAB, generated a sample log entry, and also configured the Splunk Client.
I am all ready to start developing with Splunk!
Where to go next?
Head over to dev.splunk.com and checkout our docs on the SDK, Logging Libraries, and the Visual Studio Extension. You’ll find overview docs, snippets and walkthroughs as well as links for downloading the new components.
Also check out our source on github here, here and here. All our new deliverables release under the standard Apache 2 OSS license. We’d welcome your pull requests!
It’s a great time to develop with Splunk!
----------------------------------------------------
Thanks!
Glenn Block
Related Articles
About Splunk
The Splunk platform removes the barriers between data and action, empowering observability, IT and security teams to ensure their organizations are secure, resilient and innovative.
Founded in 2003, Splunk is a global company — with over 7,500 employees, Splunkers have received over 1,020 patents to date and availability in 21 regions around the world — and offers an open, extensible data platform that supports shared data across any environment so that all teams in an organization can get end-to-end visibility, with context, for every interaction and business process. Build a strong data foundation with Splunk.
Subscribe to our blog
Get the latest articles from Splunk straight to your inbox.
Connect with Splunk on X
Follow @Splunk
