Introducing the Splunk Operator for Kubernetes

In my previous blog post, "An Insider’s Guide to Splunk on Containers and Kubernetes," I provided a sneak peak into a POC we built internally that deploys Splunk Enterprise in Kubernetes. Last month at .conf19, we published the Splunk Operator for Kubernetes as an open source project on GitHub. Most Kubernetes administrators can easily deploy this today by running a single command:

kubectl apply -f https://github.com/splunk/splunk-operator/releases/download/1.0.0/splunk-operator-install.yaml

We’ve tested this across Amazon Elastic Kubernetes Service (EKS), Google Kubernetes Engine (GKE), Red Hat OpenShift (4.1), Docker Enterprise Edition (3.1), and Open Source Kubernetes (1.15.1). Please see the getting started guide for more details and examples.

Why Use A Kubernetes Operator?

Kubernetes has evolved as a standard operating system for cloud infrastructure, and the use of operators has evolved as a standard pattern for managing applications in Kubernetes. While simple, stateless microservices may be well served by the basic resources that the platform provides, extensive work is required to deploy and manage more distributed and stateful applications like Splunk Enterprise.

Operators use the control loop principle to bring life to Custom Resources, turning them into native extensions that extend the vocabulary of basic resources Kubernetes provides out of the box. A single Custom Resource can be used to represent higher level concepts, simplifying the complexity of managing many more basic, underlying components.

Operators are a lot like installers for the cloud (similar to RPM for Red Hat Linux or MSI for Microsoft Windows). They enable developers to package all the components of their application in a way that makes it easy to deploy, even across thousands of servers.

Operators also enable developers to fully automate the operations of their application. Applications often require specific sequences of actions to be performed across components. By "owning" all the underlying components and having a greater "understanding" of how they are meant to work together, operators have the potential to offload a great amount of manual work from their human counterparts.

Helm and similar tools make it easier to deploy applications, but after that you are basically on your own. Operators enable software vendors to simplify complexity by extending native Kubernetes concepts, package and deploy all the containers and other resources that power their applications, and extend their monitoring and automation capabilities to enable better day 2+ operational experiences for customers.

In addition to introducing a new Kubernetes operator to deploy and orchestrate our containers for Splunk Enterprise, we have started to use Red Hat’s Universal Base Images (UBI) to tighten the security of our container images.

Tightening Security with Red Hat’s Universal Base Images

Matthew Modestino (a fellow Splunker) hosted a great breakout session at .conf19 with Mattia Mascia (Red Hat) titled “Red Hat OpenShift and Splunk Better Together.” If you were unable to attend this in person, I recommend you check out the slides or video recording.

With the release of Splunk Enterprise 8.0, we have completed our migration to using Red Hat’s Universal Base Images. Our official splunk/splunk and splunk/universalforwarder container images are now built on ubi8-minimal. I’m pleased to report that this has enabled us to eliminate all known security vulnerabilities.

The new container images we’ve published for the Splunk Operator for Kubernetes (including splunk/splunk-operator and splunk/spark) are also built on ubi8-minimal. All four images have been certified by Red Hat and are now available in the Red Hat Container Catalog.

While joint customers of Red Hat and Splunk can now benefit from improved support coverage, you don’t have to be a Red Hat customer to use our new UBI container images. They should work equally well on any modern Linux host operating system.

Splunk’s Support for Containers and Kubernetes

Splunk provides support for our enterprise container images when used in single-instance deployments. Support for multi-instance deployments or the Splunk Operator for Kubernetes is currently only available from the open source commmunity.

Splunk recognizes that many of our customers are reaping the benefits of using Kubernetes to operate their cloud infrastructure, and the many benefits available from our more cloud native products. Today, we see great potential in leveraging Kubernetes to make products like Splunk Enterprise easier for customers to deploy and operate. 

The Splunk Operator for Kubernetes is undergoing active development and considered to be an "alpha" quality release. While we are unable to provide guidance on when it will be generally available for production use, you can expect to see many new developments in the future. Please help us evolve our plans by trying it out and sending us feedback on your requirements.

----------------------------------------------------
Thanks!
Mike Dickey