Cloud-Native

Prometheus is the second-oldest project in the CNCF and the default answer for metrics in Kubernetes. Which means most people meet it already running, configured by someone who has since left, and never get a chance to ask how the thing works. This is the maintainer track session for people in that position. We start from the beginning for anyone who has only ever touched Prometheus through a Grafana panel: what it scrapes, what it keeps, and what the query engine is actually doing while your dashboard spins. Then we go under the covers into the storage layer, the write path, and the places the memory tends to go. ...

Observability is crucial for understanding how your application operates in real-time. Among various observability signals—such as logs, traces, and continuous profiling—metrics play a significant role. They provide sampled measurements throughout the system, essential for ensuring service quality, improving performance, scalability, debuggability, security, and enabling real-time, actionable alerting. Building observable applications begins with proper instrumentation. While Prometheus tooling simplifies this process, there are still numerous opportunities for mistakes or misuse. In this talk, Jéssica Lins and Kemal Akkoyun present several useful patterns, best practices, and idiomatic methods for instrumenting critical services. They discuss common pitfalls, failure cases, and instrumentation strategies, sharing valuable insights and methods to avoid these mistakes. Additionally, they provide tips for writing simple, maintainable, and robust instrumentation facilities using real-life examples. The talk also demonstrates how to enrich metrics by correlating them with other observability signals and discusses how to best use recent changes in client_golang, the Go client library for Prometheus. ...

A sponsored keynote, which is a genre that usually means a product pitch. This one is about how Red Hat ran Prometheus across a large fleet, and why the patches we needed went upstream rather than into a fork we would have to carry forever. Past a certain scale, Prometheus stops being one binary you put on a box. It becomes a set of components you assemble, with Thanos or something like it behind it, and the interesting problems move from “how do I scrape this” to how you keep the whole assembly cheap, queryable, and boring to operate. Same ideas, more moving parts. ...

In modern days, we run our applications as loosely coupled micro-services on distributed, elastic infrastructure as (mostly) stateless workloads. Under these circumstances, observability has become a key attribute to understand how our applications run and behave in action, in order to provide highly available and resilient service. There exist several observability signals, such as “log”, “metric”, “tracing” and “profiling” that can be collected from a running service, which we can also call pillars of observability. Using these signals, we can create real-time, actionable alerts, create panels where we can monitor applications closely, and perform in-depth analysis to find the root of the systems’ failures. Within the Go and CNCF community, there are a variety of tools that can collect and make these observable signals useful. ...