Engineering · · 7 min

OpenTelemetry Go Compile-time Instrumentation v1

Note This post was originally published on the OpenTelemetry blog on 2026-07-16. I wrote it for the OpenTelemetry community; this is a mirror on my personal blog with some added commentary. Why this milestone matters to me I have been involved with the Go Compile-Time Instrumentation SIG since it kicked off at the start of 2025, when Alibaba and Datadog decided to pool their efforts into a single vendor-neutral project. Watching it reach v1 last week felt genuinely satisfying — not because “v1” is a magic number, but because it marks the point where we stopped calling it experimental and started calling it something people should actually run in their builds. ...

July 22, 2026 · 7 min · 1434 words · Kemal Akkoyun
The Zen of Prometheus, now part of the official Prometheus documentation
Reflection · · 4 min

From talk to docs: The Zen of Prometheus

Every now and then a project surprises you by remembering something you said years ago. This week was one of those weeks. A talk I gave at PromCon Online 2020 — The Zen of Prometheus — has quietly become part of the official Prometheus documentation. I am still sitting with it. Where it started The talk was born in the strangest year of my career. PromCon 2020 was online, like everything else. I was a few years deep into running Prometheus in anger, collecting scars from instrumenting services that didn’t want to be instrumented and writing alerts that kept me up at night for the wrong reasons. I wanted a way to package those lessons that wasn’t another forty-slide deck of bullet points. ...

May 15, 2026 · 4 min · 700 words · Kemal Akkoyun
Measuring Software Performance
Engineering · · 13 min

Measuring Software Performance: Why Your Benchmarks Are Probably Lying

A Loose Cable That Broke Physics In 2006, a team of physicists began building the OPERA experiment — a 730-kilometer underground tunnel from CERN in Switzerland to Gran Sasso in Italy, designed to measure the speed of neutrinos. Five years of construction. Roughly 100 million euros. The most rigorous experimental physics on the planet. In September 2011, the results came back. Neutrinos were traveling faster than the speed of light. The team had just broken the laws of physics. ...

March 6, 2026 · 13 min · 2680 words · Kemal Akkoyun
Auto-Instrumenting Go
Engineering · · 12 min

Auto-Instrumenting Go: From eBPF to USDT Probes

This post expands on the FOSDEM 2026 Go Devroom talk I co-presented with Hannah S. Kim. The talk, demo code, and all benchmark scenarios are available in the fosdem-2026 repository. The Problem Go is one of the best languages for building production backend services. It compiles to native binaries, has excellent concurrency primitives, and produces predictable performance characteristics. But when it comes to auto-instrumentation — adding observability without modifying source code — Go is uniquely difficult. ...

February 27, 2026 · 12 min · 2386 words · Kemal Akkoyun
OTel Unplugged EU 2026 — crowd voting on sessions
Journal · · 10 min

OTel Unplugged EU 2026: Field Notes from the Instrumentation Frontier

Brussels Again, But Make It Unplugged The day after FOSDEM, about a hundred of us gathered at Sparks Meeting on Rue Ravenstein in Brussels for OTel Unplugged EU 2026 — an unconference dedicated entirely to OpenTelemetry. Purple stage lights, a mid-century auditorium with wood paneling, and the familiar buzz of people who spend their days thinking about telemetry pipelines. If you know, you know. The format is simple: no prepared talks, no slides. Morning session brainstorming, dot-voting on topics, then self-organizing into nine rooms across four breakout slots. You vote with your feet. If a conversation isn’t working, you move. It’s chaotic, it’s honest, and it produces the kind of discussions that polished conference talks rarely achieve. ...

February 20, 2026 · 10 min · 2114 words · Kemal Akkoyun