As we start to see big moves from monolith deployments to microservices, the adoption of Kubernetes has become top of mind for many SREs. Organizations can leverage the open-source system to automate deployments, scale, and manage containers, making Kubernetes one of the primary solutions for delivering workloads. However, maintaining the system can be difficult and, in some cases, overwhelming. For that reason, it is easy to find cloud platforms offering Kubernetes as a managed service like Amazon EKS. Amazon EKS helps deliver highly available and secure Kubernetes clusters in a way that is both fast and simple.
The short answer is no. Splunk APM makes it easy to leverage the OpenTelemetry collector to collect application metrics with no wasted efforts with future-proof, vendor agnostic instrumentation designed to scale.
Instrumenting applications running in EKS is very similar to instrumenting applications that aren’t running in EKS, so we’ll demonstrate how to instrument a non-EKS app first, then explain the changes needed if your app runs in EKS.
In this example, we will use a basic Java application that performs HTTP GET requests to a website of your choice. To begin, navigate to the Data Setup wizard from the hamburger menu, from the APM Instrumentation menu select Java, and click Add Connection.
Provide the following details about the environment where the application is deployed in the Configure Integration section.
After the configuration steps are complete, the next step is to install the Splunk OpenTelemetry Connector if you have not already done so. In this example, we have installed the collector on an Ubuntu server running the Java application. To verify the Splunk OpenTelemtery Connector was installed correctly, type the following to ensure the service is running:
service splunk-otel-collector status
Note the splunk-otel-collector is active (running).
The last step is to install the integration by downloading the latest version of the Splunk Distribution of OpenTelemetry for Java with the provided curl command to add to your runtime command. Set the environment variables that provide details about your deployment, automatically created based on the information provided in the previous step.
Once set, you can then update your runtime command to include the Splunk Distribution of OpenTelemetry Java. Here is my example below of how I updated my runtime command in a non-EKS (Kubernetes) deployment.
java -javaagent:./splunk-otel-javaagent.jar -jar curlapp-1.1-SNAPSHOT.jar
You can also add the environment variables to your runtime command manually. Here is an example.
java -Dexec.executable="java" -Dotel.resource.attributes=service.name=httpGoogleChecker-bm,deployment.environment=CheckingGoogleBM -Dsplunk.metrics.enabled=false -javaagent:splunk-otel-javaagent.jar -jar curlapp-1.1-SNAPSHOT.jar
Now that you have started your Java application, the data setup wizard will direct you to interact with your data by clicking Explore Service Spans.
Note the environment name (CheckingGoogleBM) highlighted in the screenshot above.
When instrumenting your Java application with Amazon EKS, the process is similar to the one previously described and only changes slightly due to how applications are deployed with Kubernetes. To add, this process will also work the same when deploying your application to a Kubernetes cluster outside of Amazon EKS. To begin instrumenting your containerized Java applications with Amazon EKS, follow the same steps to navigate to the data setup wizard. On the Configure Integration section, select Yes for Kubernetes.
Click next to proceed to the Install Integration section. The first step is installing the Splunk OpenTelemetry Connector on your Kubernetes cluster if not already done so. To verify if the Splunk OpenTelemetry Connector is installed on your Amazon EKS cluster, update the kubeconfig file for your Amazon EKS cluster and run the following command.
kubectl get pods
The output should show both the agent and cluster receiver pods.
Once installed, use the curl command to download the splunk-otel-javaagent.jar to include with your runtime command within your containerized application.
Here is an example of my Dockerfile, which copies the splunk-otel-javaagent.jar of the Java application container and executes the application within the run.sh bash script.
FROM openjdk:8-jre-alpine3.9
COPY target/curlapp-1.1-SNAPSHOT.jar /curlapp-1.1-SNAPSHOT.jar
COPY splunk-otel-javaagent.jar /splunk-otel-javaagent.jar
COPY run.sh /run.sh
ENTRYPOINT ["sh", "/run.sh"]
Contents of run.sh:
java -javaagent:/splunk-otel-javaagent.jar -jar /curlapp-1.1-SNAPSHOT.jar
Next, use the provided example to update your deployment file to inject the environment variables necessary to auto instrument your application for your Java application.
Here is an example of my deployment file updated with the required environment variables in italics.
apiVersion: v1
kind: Pod
metadata:
name: httpgooglechecker
spec:
containers:
- name: httpgooglechecker
image: docker.io/astro7982/curlappstatic:lastest
env:
- name: SPLUNK_OTEL_AGENT
valueFrom:
fieldRef:
fieldPath: status.hostIP
- name: OTEL_RESOURCE_ATTRIBUTES
value: service.name=httpGoogleChecker-k8s,deployment.environment=CheckingGoogleK8S
- name: SPLUNK_METRICS_ENDPOINT
value: http://$(SPLUNK_OTEL_AGENT):9943
- name: OTEL_EXPORTER_OTLP_ENDPOINT
value: http://$(SPLUNK_OTEL_AGENT):4317
command: ["/bin/sh"]
args: ["run.sh"]
Now that all installation requirements are completed create your Java application pod using kubectl create, and within 90 seconds, Splunk APM will begin capturing all traces from your application.
Note both the service name and environment are reflected in the ones entered in your Kubernetes configuration file.
Now that the service is instrumented, you can see RED metrics created automatically, and you can also view every single transaction’s full trace through your infrastructure. Check out our demo video to see more of what can be done once your application is instrumented.
Want to try this yourself? Feel free to – you can sign up to start a free trial of the suite of products – from Infrastructure Monitoring and APM to Real User Monitoring and Log Observer. Get a real-time view of your infrastructure and start solving problems with your microservices faster today.
----------------------------------------------------
Thanks!
Johnathan Campos
The Splunk platform removes the barriers between data and action, empowering observability, IT and security teams to ensure their organizations are secure, resilient and innovative.
Founded in 2003, Splunk is a global company — with over 7,500 employees, Splunkers have received over 1,020 patents to date and availability in 21 regions around the world — and offers an open, extensible data platform that supports shared data across any environment so that all teams in an organization can get end-to-end visibility, with context, for every interaction and business process. Build a strong data foundation with Splunk.