Up Close Monitoring with AWS Fargate

On our cloud-native journey, we live in a containerized world. Our environments are containers, managed by orchestrators, and living on some level of computing clusters. Of course, that means you are also responsible for managing all those bits, right?

Well, not totally. Thanks to new technologies like AWS Fargate, you’re not on the hook for much of the underlying structure. You don’t need to provision, configure or scale the VMs to run your containers. You simply tell Fargate what you want and it automatically sets it up for you. No more worrying about capacity planning or other operational needs. Similar to working with serverless (AWS Lambda), you can run in a hybrid model, where part of your work is on traditional EC2 and part moves to Fargate. 

But how do you make sure that what is going on under the surface maps to the performance and issues in your app?

Monitoring Fargate lets you understand how your containerized apps are performing. And since in the Fargate model you are paying for your usage, keeping track of your Fargate deploys makes solid financial sense. 

Fargate aligns with AWS ECS, the AWS service that lets you manage the configuration and lifecycle of your containers. Both run in the cloud and are tied into many other services. Linux containers can launch on Fargate resources or your own EC2 instances or both, but Windows containers are limited to EC2. ECS groups containers into tasks. That task has a definition that sets the details, including size, the CPU and memory required, among other options. Keep in mind that you must specify CPU and memory in your task definition file and that they have to align to one of the Fargate combinations or else you’ll get an error returned.

You can also choose to define other objects, which allow you control over the sharing of resources, such as memory. For instance, memoryReservation specifies the minimum memory for the container, and memory is the maximum. Containers can grow from their minimum if additional memory is available, but should a container try to use more memory than is set in the maximum, ECS will terminate the container. Similar rules exist for CPU with minimum and the ability to use additional CPU resources if available. When setting allocations, it’s worth keeping in mind is that ECS allocates CPUs as CPU units (or shares of a virtual CPU). Each vCPU equals 1024 CPU units.

Similarly, Fargate works with Kubernetes (EKS in this discussion). Since EKS is Kubernetes-conformant, moving from any Kubernetes orchestration to EKS should be painless. However, the terminology used by EKS is different from ECS. A cluster is the group of instances that use the same Kubernetes API server. A pod contains 1 or more containers. A deployment defines the collection of pods. There are similar definition items (memory, CPU, IP address, etc.) that can enforce limits on the resources for the pods and containers, thus impact on the applications.

However, if you are used to using DaemonSets, you are out of luck. Fargate does not support DaemonSets.

So we have lots of options to get things running and they can cross from EC2 to Fargate. But what can we measure to make sure Fargate is delivering the goods for us?

Monitoring Fargate

Monitoring Fargate is relatively straightforward. We need to monitor the resources we’ve defined to make sure we aren’t overprovisioned or being starved for resources. After all, nothing is quite as painful as having a great app become a slow consumer due to memory pressure, be it by underprovisioning or due to noisy neighbors. This focuses us on memory and CPU for each task or pod. However, we also need to understand our costs, which is based on the number of tasks (or pods) run, the amount of time running and the resources allocated to each. AWS has some great pricing information to help you understand Fargate pricing. Our monitoring will help you determine if you can optimize your spending.

Let’s start by considering modern monitoring practices, USE and RED. RED, Rate-Errors-Duration, is focused on applications, particularly on applications that are made up of multiple services. USE, Utilization-Saturation-Errors, is focused more on the infrastructure side. While it is strongly recommended to have both methods in hand, USE is our focus point for Fargate (ECS, Kubernetes) monitoring.

In general, our monitoring will focus on metrics that come from 3 major categories (and one catch-all).

• CPU
• Memory
• Cluster state
• Everything else
  
  

CPU Metrics

As you recall, we defined a number of CPU resources for our tasks or pods. Since the initial instantiation is based on our reserved minimums and hopefully has growth capacity, it’s important to monitor to make sure we are getting what we expect.

Of these metrics, our focus should be firmly on CPU utilization, to help ensure that spikes don’t breach your hard CPU limits. Depending on your specific risk factors, set your high alert to inform you of static breaches of 80% (or higher). Missing a 100% breach can have consequences that last long beyond the immediate breach, so a full-fidelity monitoring system is your friend here. You might also want to set a sudden change alert, making use of AI/ML capabilities to help with those factors that aren’t known (yet) but can cause your container’s CPU use to deviate in unexpected ways.

Likewise, it is suggested to set some sort of monitoring and or alerting on unusually low conditions. If your workloads are under 50% consistently or show a seasonal behavior of under 50%, consider ways you can reset and re-establish your definitions to match your workload better. After all, you really don’t want to pay for unused resources, but be sure to take into account all the potential activity you might face.

Memory Metrics

Remember that we had to specify the memory requirements for our containers/pods. It’s a fine line to walk, between making sure we have enough memory and available pool to run our workloads efficiently and that we aren’t overprovisioning and thereby upping our costs. Keeping an eye on memory usage (with a bit of alerting assistance) will help us understand usage and/or potential impact. 

Our memory metrics of interest are:

Similar to CPU, we are most focused on memory utilization. This is especially important because exceeding 100% memory utilization will lead to Fargate killing our container. By aggregating and analyzing our memory usage, we can determine if our memory requests are within an acceptable range. It can also help us determine if we have any cyclic behaviors which might impact our requested mins and maxes.

If you are regularly using less than your minimum request, reduce it. That should reduce costs but keep in mind that Fargate may move your workload to a smaller compute resource, so you also will need to observe for any potential impact.

Setting alerts on memory utilization should be approached similarly to the CPU metrics we set above. Set one for a static high value in alignment with your risk profile, set one for a sudden change or seasonal value, and consider setting an alert on low memory utilization. When choosing your high alerts, keep in mind that a container (or pod) that asks for more than its maximum allowed memory will be killed.

A small side note: Memory can have an impact on application performance. Too little memory may cause a resource starvation impact, unnecessarily slowing or stopping applications and resulting in unhappy users. Applications, in turn, can suffer from memory leaks, which will potentially eat memory until failure. Having APM (application Performance Monitoring) aligned and in step with your Fargate monitoring can quickly reveal the true impact memory choices have on your environment.

Cluster State Metrics

Even though you are tracking the individual elements, you also need to keep an eye on the Fargate cluster usage. Currently, you cannot exceed a hard limit per region:

100 > #ECS tasks + #EKS per region

----------------------------------------------------
Thanks!
Dave McAllister