Auto-Instrumentation

The debugging loop is slow. You cannot reproduce the bug locally, so you add a log line and redeploy. Wrong place. You do it again. An agent alone does not fix this: it still has to pick a mechanism, and it has to know what that mechanism costs. This talk is about three approaches that take the source edit out of that loop. Each acts at a different point in the software lifecycle, and each inherits the strengths and constraints of its intervention point. ...

You can’t reproduce the bug locally, so you add a log line and redeploy. Wrong place. You add another one and redeploy again. That is the debugging loop from my GopherCon UK talk, and in this last part we’re going to shorten it: a decision rule for two zero-touch tools, then a runbook an AI coding agent (not a Java-style one) can execute. The runbook starts with one boring question: is this Go service already running, or can I rebuild it? If the service has to stay untouched, OBI (OpenTelemetry eBPF Instrumentation) is the default. If I can rebuild it and need function-level spans, otelc (OpenTelemetry Go compile-time instrumentation) is the default. The rest of the decision tree is checking privileges, Go version, collector endpoint, and whether RED metrics are enough. ...

Every route in this series goes around the Go runtime: rewrite the source at build time, inject something at process start, or watch from the kernel, because Go gives us no agent API to plug into. This time we ask the runtime what it could hand us. The honest summary for 2026: one long-awaited feature shipped, two open proposals document gaps that have caused real production problems for years, and two experiments of mine are aspirations, not proposals. ...

We’re going to follow one request, GET /orders/42, and ask what “propagation” means at each hop. Three different jobs hide behind that word, the tools in this series do different ones, and the request’s own data falls into the gaps between them. The cheapest way to see a gap is a short Go program. This is a companion to the How to Instrument Go Without Changing a Single Line of Code series, between Part 4 and Part 5. The series accompanies my GopherCon UK talk. Part 1 showed tools hacking a field into g. Here we look at what that field holds, what it can’t, and who does the rest. ...

Many tools that profile Go binaries resolve function names from the ELF symbol table or DWARF debug info, and production builds strip both. Point one of those profilers at a stripped binary and you get addresses, not names. The opentelemetry-ebpf-profiler doesn’t have this problem, and we’re going to find out why. Let’s start with a binary rather than a profiler. Here is the whole program, with //go:noinline so the compiler can’t fold checkout into main: ...