How Microsoft Used Splunk’s Ethlogger to Turn Blockchain Data Into Supply Chain Insight

The way we ‘data’ is about to change, and Splunk’s Connect for Ethereum (aka EthLogger) is helping organizations to adapt. Splunk Connect for Ethereum enables organizations of all sizes to investigate, monitor, analyze and act upon their rapidly growing blockchain data sets across multiple chains.

Most recently, Microsoft envisioned a supply-chain network on the blockchain that ensured synchronized logistics and process data across all participants. To do so they leveraged ConsenSys Quorum’s enterprise Ethereum, hosted on Azure, and Splunk’s Connect for Ethereum to extract and transform data from the ledger to gain operational insights. The platform was established around a foundational network consortium of Microsoft’s partners, which ensures benefits, synchronized processes, and technical upgrades across all parties.

Consortiums bring a plethora of business and technical challenges. For example, a consortium or network operator needs visibility into the IT and Security aspects of all entities no matter the infrastructure whether it is hosted on-premise, in the cloud, managed or unmanaged. The operator and participants themselves will also require access to the different parts of the ledger for analytics around auditing, fraud and business use cases.  These challenges align with our blockchain mission at Splunk which is to accelerate the adoption of Distributed Ledger Technologies (DLT) by helping organizations to avoid downtime, secure their infrastructure and make the data generated by blockchains accessible to anyone.

An example architecture and live dashboard of routing and transforming IT and ledger data from Azure Blockchain Service to a consortium operator and participants

Splunk Connect for Ethereum served as a key component in Microsoft’s consortium.  It is an open-source tool that makes it easy to ingest data from Ethereum ledgers, collect node information, and relay node metrics into another system for analysis. It can also deserialize transactions and events if given a contract ABI. This means that, in addition to the binary data of transactions, information such as function or event names, parameter values, data types, parameter names, and contract names, can all be ingested with EthLogger.

Automatic ABI decoding makes extracting insights from ledger data easy in Splunk

Microsoft needed to extract both public and private transaction data from Quorum and store it in secure Azure tenants for each party’s analysis and audits. To build a system that would cater to each participant's individual requirements, Microsoft, in collaboration with Accenture, decided to use EthLogger to pipe all the data of each transaction occurring in the Quorum private state to each of the partners’ individual environments. This allowed them to have a synchronized object database for reporting. Ethlogger was selected because it’s open source, fast, can gather all transaction data, and can reliably provide data to all parties. Furthermore, EthLogger maintains state so that if indexing stops it will always restart correctly — never losing data in the case of services stopping or dropping connections. 

Deep Dive on EthLogger

You can learn more about our open source project EthLogger on Github via the detailed instructions within the readmes. Alternatively, Doge can help you discover why companies like Microsoft are saying ‘Wow’. 

Much Agnostic
Support for any Ethereum 1.0 JSON-RPC compatible blockchain or sidechain, including Quorum 🦄.

Wow Dependable
Reliably saves state between restarts so you never have missing or duplicated events 🧘‍.

So Performance
Highly performant batching and compression algorithm to optimize data in transit and at rest 🚄.

Much Power over 9,000
Extracts a rich set of node information and metrics in order to gain deep insight into your network’s nodes 🙉.

Many Intuitive
Introspects your node platform (i.e. geth, parity, besu or quorum) in order to ensure maximum data extraction 🎂.

How Insight full
Contract fingerprinting in order to match ABIs with function signatures (i.e. we can decode the parameter names too!) 🔎.

Wow Customizations
Enables awesome dashboards with Splunk’s dashboard builder (or build/manage with the code!).

In Conclusion

EthLogger is the best open source solution on the market to ingest data from Ethereum ledgers, collect node information, and expose node metrics into another system for analysis. When combined with Splunk App for Quorum or ethereum basics app, buidl-ers like you are enabled with the first true blockchain monitoring system on the market. 

We have a ton of exciting products and features planned for blockchain observability. Read more on the specifics of EthLogger by visiting this blog, Introducing Splunk Connect for Ethereum, written by our own Stephen Luedtke. If you need assistance with Ethereum analytics connect with us at blockchain@splunk.com.

Happy Buidl-ing!

----------------------------------------------------
Thanks!
Tucker Logan