Jonathan Hall and Shay Nehmad had me on Cup o’ Go for episode 167, straight after their GopherCon US recap. We spent about forty minutes on instrumentation: what an APM tool is actually measuring, and the three ways you can get telemetry out of a Go program without asking every team to rewrite their handlers.

The part I enjoyed arguing about most was blast radius. Manual instrumentation is precise and tedious, and it never finishes. eBPF buys you one agent covering every language on the box, at the cost of running code in the kernel, where a bad program can take the whole host down with it. Compile-time instrumentation rewrites the syntax tree behind go build through the -toolexec hook, which keeps failures scoped to one service and works anywhere Go compiles. People love this dark magic, which I find either reassuring or alarming depending on the day.

We also got into why a trace and a pprof profile answer different questions, how Orchestrion ended up under OpenTelemetry governance, and the least intimidating route into a SIG if you want to start contributing. Shay’s bicycle makes an appearance. I was speaking for myself rather than reading from a Datadog brochure, which I hope comes through.

Listen

Episode

Topics

  • GopherCon US recap, and Austin’s talk on fixing the proposal process
  • What APM measures, and how request tracing differs from pprof sampling
  • Manual, eBPF, and compile-time instrumentation, and what each one costs you
  • Orchestrion moving to OpenTelemetry, plus how to find your way into a SIG
  • Checklocks, the gVisor static lock checker I keep recommending to people

Related