Ingest Data with One Tool from Twitter to OPC, Node-RED | Splunk

“Node-RED is a programming tool for wiring together hardware devices, APIs and online services in new and interesting ways.” – NodeRED.org

Let’s decrypt this. Node-RED is a GUI-based IoT integration platform. Its foundation is built on node.js and allows users to pass json messages between Node-RED “nodes” with virtual wires. But what’s a node, you ask? A node is a configurable block that does one of the following: integrates into systems or infrastructures, transforms or merges data, or passes a message to either a dashboard or external system.

Node-RED is a quick and easy tool to spin up and move data from one source to another. It comes preinstalled on Raspbian—Raspberry’s Pi Linux based operating system—so if you’re running it on a Raspberry Pi, it's as easy as a few clicks to get up and running. 

(Side note: If you haven’t heard of Raspberry Pi, please go check them out; they run about $35 for the base systems. I use them for everything, and even have a BBQ Grill temperature controller so that I can make tasty BBQ while I sleep. Maybe I should Splunk that?)

Node-RED is a tool that I have used in the past for various projects, from edge devices collecting machine data via Bluetooth to collecting Twitter data for the fun of it. It has about every connection imaginable and now it has a Splunk node—the http-event-collector node takes a message passed from previous nodes and forwards the information to Splunk HTTP Event Collector.

To get started, you need to have Node-RED installed and running on your system of choice—Linux, Apple, or Windows. For setting that up, follow the guides over at nodered.org.

The easiest way to install the HTTP Event Collector node is through Node-RED’s built in marketplace. To do so, navigate to “Manage palette” from the menu in the top right corner of Node-RED’s interface.

Search for and install the “node-red-contrib-http-event-collector” palette.

From there, two nodes should pop up under a new Splunk category.

Before we go any further, we need to configure Splunk’s HTTP Event Collector to allow data to be sent from Node-RED. Follow the HTTP Event Collector guide at Splunk Docs. Once Splunk is configured, drag and drop the “splunk-http-event-collector” node onto the flow and double-click the node to open the configurator. Fill out the information supplied by Splunk’s HTTP Event Collector, ensuring the URL includes "https://" and the index matches what is configured in Splunk.

Once complete, link your new node up to your data source of choice and click deploy. An example Node-RED flow could look something like this:

Node-RED has a bunch of nodes that can access many applications and data sources. It may just let you Splunk that data you once thought was just out of reach or too complicated to get.

----------------------------------------------------
Thanks!
Grey Dziuba