Monitoring Container Workloads Using Amazon Managed Service for Prometheus with Wiz

Wiz, an AWS Partner founded in Israel in March 2020 and now headquartered in New York City, provides a next-generation, cloud-native application protection platform with an innovative approach to help customers identify, prioritize, and remediate security issues.

Operating such a complex and unique service requires Wiz to run hundreds of microservices on multiple Kubernetes clusters. Each microservice plays a critical role in the system. Wiz monitors every aspect of the infrastructure, including Kubernetes resources—such as nodes, pods, and databases—and aggregates metrics to a centralized remote place. “Our infrastructure and services are monitored and collected using Prometheus, an open-source project,” says Tal Marcovich, a DevOps engineer at Wiz. But with millions of workloads to monitor, Wiz needs a robust, secure solution to gather the metrics that Prometheus produces, support visibility into those metrics, and store historical data at scale.

Wiz evaluated various options and chose Amazon Managed Service for Prometheus, a Prometheus-compatible service that monitors and creates alerts on containerized applications and infrastructure at scale, for its monitoring solution. “Our first step toward using Amazon Managed Service for Prometheus was to provision the service’s workspace using Terraform, then configure all our Kubernetes clusters to send Prometheus metrics to it,” says Marcovich. “Since part of our infrastructure is also on AWS, the setup process was native and simple to use.” Wiz appreciated the fact that Amazon Managed Service for Prometheus is a managed service, which eliminates the need for the company to provision and monitor more monitoring clusters or systems on its end.
 

To build a secure, high-visibility monitoring solution, Wiz uses Amazon Managed Service for Prometheus to apply a monitoring layer to its applications running on clusters from Amazon Elastic Kubernetes Service (Amazon EKS), a managed Kubernetes service. “Every running workload on our Amazon EKS generates Prometheus metrics, which are sent to a centralized remote Amazon Managed Service for Prometheus,” says Marcovich. “We use those metrics to provide our engineering teams with a complete picture of the health of our applications and infrastructure.” This step is part of a process that gives Wiz the capacity to handle metrics reliably and securely. And that process is happening at scale.

Wiz operates cost-effectively, paying only for what it uses as a function of the AWS pay-per-use model that provisions resources and scale according to workload. “We send millions of active metrics series to Amazon Managed Service for Prometheus from dozens of Amazon EKS clusters worldwide,” says Roy Reznik, cofounder and vice president of research and development at Wiz. “It’s simple to use, helps us handle our tremendous volume of metrics, provides a long-term historical metric view, is highly available and reliable, and maintains strict security standards. By using Amazon Managed Service for Prometheus, we can deliver our code faster to our valuable customers and eliminate the need to provision and monitor more dedicated monitoring clusters and systems.”

After the engineering teams have this visibility across multiple clusters, they are able to create alerts and communicate issues and trends to the corresponding teams. Besides the out-of-the-box Prometheus metrics, the teams also use custom application metrics. The developers, for example, create their own metrics to help them understand the application’s state, identify bugs, and get a better sense of the application’s behavior.

To monitor the resources and application performance of its AWS-powered solution, Wiz uses Amazon CloudWatch, which collects and visualizes near-real-time logs, metrics, and event data in automated dashboards to streamline infrastructure and application maintenance. On Amazon CloudWatch, Wiz achieves a holistic view of its system. The service works seamlessly with other AWS services that Wiz deploys, like Amazon Relational Database Service (Amazon RDS)—a collection of managed services that makes it simple to set up, operate, and scale databases in the cloud—to compile all relevant metrics and send them to Amazon Managed Service for Prometheus for the DevOps team to monitor.