Monitor Containerized Deployments on AWS Bottlerocket with Splunk

We are excited to partner with AWS in launching AWS Bottlerocket, a container optimized operating system. Bottlerocket gives DevOps teams speed, efficiency and security in containerized environments. Splunk solutions have been validated and certified by AWS to run on Bottlerocket so our customers can innovate rapidly and scale efficiently by getting observability into every layer of containerized workloads deployed on Bottlerocket operating system as well as other AWS services from a single solution.

Digital initiatives have propelled the use of containers in enterprises of every size. The agility, elasticity and automation requirements have fundamentally changed how applications are built, deployed and managed. According to Gartner, by 2022, more than 75% of global organizations will be running containerized applications in production, which is a significant increase from less than 30% today.

Containers make packaging, distribution and deployment of applications a lot easier. A container image becomes a reliable and repeatable artifact with a set of dependencies and application code. The underlying Linux kernel provides the necessary isolation to run applications separately by leveraging cgroups and namespaces. Since containers rely upon the Linux kernel, they don't include the full copy of the operating system resulting in faster boot time. Further, by deploying multiple, isolated applications on the same infrastructure, DevOps teams achieve higher resource utilization and, in turn, lower cost.

Deploying containers on a general-purpose operating system poses some challenges:

1. Security: Higher number of deployed packages results in more vulnerabilities due to a bigger surface area. Bottlerocket contains less software – for example – it does not have SSH or even a shell, making it difficult for intruders to gain a foothold in the system.
   
   
2. Immutable infrastructure: Most operating systems have package managers to install software components, making it challenging to achieve image consistency across every deployed instance. Bottlerocket does not include any package manager. It uses a pre-built image for the operating system. Updating OS includes installing a new image resulting in a consistent operating system across the entire fleet.
   
   
3. Performance: Bottlerocket improves bootup and runtime performance by eliminating unnecessary packages
   
   

Although support for Amazon ECS is on the roadmap, Bottlerocket is currently available for Amazon EKS only. While Kubernetes clusters on Bottlerocket have a smaller attack surface, better startup performance, and a more consistent update mechanism, challenges remain in monitoring Kubernetes clusters' performance. According to the latest CNCF survey, complexity and monitoring are the top challenges in Kubernetes adoption.

As Kubernetes environments scale and applications get more distributed, monitoring becomes more challenging. As Kubernetes environments scale and applications get more distributed, monitoring becomes more challenging. DevOps teams struggle with:

• Gaining multi-dimensional visibility into the health of the entire Kubernetes environment
• Detecting performance issues in real-time
• Understanding the why behind performance anomalies
• Switching context while searching through multiple metrics, logs, and events platforms results in higher mean time to resolution.
  
  

With Splunk Infrastructure Investigation and Monitoring solution, your teams can detect, triage, and resolve performance issues faster than ever before. DevOps and SRE teams can successfully navigate the complexity associated with operating Kubernetes at scale by taking advantage of these features:

Dynamic Cluster Map

Starting with the bird’s eye view, quickly understand the performance with intuitive and hierarchical navigation. Select, filter, or search for any Kubernetes entity, e.g., node, pod, and container level within seconds. SignalFx automatically discovers Kubernetes components and containerized services to monitor your entire stack instantly. Understand relationships between dynamic Kubernetes components and quickly fix interdependent performance issues.

Drill-Downs

A global, at-a-glance view into the entire Kubernetes environment helps teams understand how the overall system is performing. It is equally important to have a granular, detailed view into individual components as teams narrow down to the source of the problem — drilling down from nodes to pods to containers to workloads. Our streaming architecture enables in-depth analysis with search and filters within seconds at a massive scale

In-context investigation

Seamlessly pivot to logs and get granular visibility into application, Kubernetes, and container logs to correlate performance across the entire stack without any context switching. Leverage open source Splunk Connect for Kubernnetes to ingest logs, events and metadata in your Splunk Cloud or Splunk Enterprise environments. A seamless workflow using deep-linking between SignalFx Infrastructure Monitoring and Splunk carries the context to expedite the investigation in Splunk.

Kubernetes Analyzer

To understand the “why” behind performance anomalies, Kubernetes Navigator leverages AI-driven analytics, which automatically surfaces insights and recommendations to precisely answer, in real-time, what is causing anomalies across the entire Kubernetes cluster – nodes, pods, containers, and workloads. Following suggested filters, SRE teams can narrow down to the underlying issue within minutes. Sophisticated algorithms, including Historical Performance Baselines and Sudden Change, detect system-level issues such as a sudden increase in container restarts and alert within seconds.

Get Started with Monitoring Workloads on AWS Bottlerocket

With AWS validated and certified integration of SignalFx Infrastructure Monitoring and Splunk Connect for Kubernetes on AWS Bottlerocket, confidently migrate your workloads to the new container optimized operating system to achieve improved security, updates and performance. Sign up for a free trial of SignalFx and get started quickly with helm based installation, and zero-touch configuration on Amazon EKS.

----------------------------------------------------
Thanks!
Amit Sharma