Observability

The CNCF Incubated Thanos project with the large open-source community continues to push boundaries regarding observability and monitoring using Prometheus-based metrics. Together with the Prometheus community, it improves the metric story for Kubernetes clusters and beyond. Things like improved performance, better scalability, debuggability, security, metrics backfilling and query QoS is only the tip of the iceberg. As we know, observability nowadays comes in many flavours. Bunching them together is not a trivial side, given many shapes and collection points. Aside from metrics, we have logs, traces or even continuous profiling. In this talk, Kemal and Bartek, Thanos maintainers, after a quick overview of Thanos, will explain how Thanos can be integrated with those non-metric observability signals. The audience will learn an example, end-to-end ways to correlate multiple observability backends with Thanos for enhanced observability and monitoring experience. ...

Metrics are powerful tools in the cloud-native space, enabled by Prometheus. However, using facilities to enable monitoring requires instrumenting the code. Everyone wants observability, but nobody wants to go the extra mile to instrument their clusters or applications. This is where eBPF comes in. eBPF, a promising technology for observability tooling, is not news. To observe the infrastructure and applications, eBPF-based system-wide agents can help us to capture events without requiring recompilation or redeployment of applications. In this talk, attendees will discover alternative ways to collect metrics from applications and infrastructure using system-wide eBPF agents. The presenters will demonstrate what level of observability could be accomplished without instrumentation. ...

For years Google has consistently been able to cut down multiple percentage points in their fleet-wide resource usage every quarter, using techniques described in their “Google-Wide Profiling” paper. Ad-hoc profiling has long been part of the developer’s toolbox to analyze CPU and memory usage of a running process, however, through continuous profiling, the systematic collection of profiles, entirely new workflows suddenly become possible. Matthias and Kemal will start this talk with an introduction to profiling with Go and demonstrate via Conprof - an open-source continuous profiling project - how continuous profiling allows for an unprecedented fleet-wide understanding of code at runtime. Attendees will learn how to continuously profile Go code to help guide building robust, reliable, and performant software and reduce cloud spend systematically. ...

For years Google has consistently been able to cut down multiple percentage points in their fleet-wide resource usage every quarter, using techniques described in their “Google-Wide Profiling” paper. Ad-hoc profiling has long been part of the developer’s toolbox to analyze the CPU and memory usage of a running process. However, through continuous profiling, and the systematic collection of profiles, entirely new workflows suddenly become possible. The presenter will start this talk with an introduction to profiling applications, and demonstrate how one can practice it using open-source continuous profiling tools, and how continuous profiling allows for an unprecedented fleet-wide understanding of code at production runtime. ...

In modern days, we run our applications as loosely coupled micro-services on distributed, elastic infrastructure as (mostly) stateless workloads. Under these circumstances, observability has become a key attribute to understand how our applications run and behave in action, in order to provide highly available and resilient service. There exist several observability signals, such as “log”, “metric”, “tracing” and “profiling” that can be collected from a running service, which we can also call pillars of observability. Using these signals, we can create real-time, actionable alerts, create panels where we can monitor applications closely, and perform in-depth analysis to find the root of the systems’ failures. Within the Go and CNCF community, there are a variety of tools that can collect and make these observable signals useful. ...