# OBI and the eBPF profiler without Kubernetes: Docker Compose demo > [!summary] TL;DR > Run OBI (OpenTelemetry eBPF Instrumentation) and the OpenTelemetry eBPF profiler against one untouched Go server with Docker Compose, no Kubernetes. The server hashes the order ID 200,000 times per request, so the profiler has CPU to sample. OBI attaches from outside with no rebuild and produces a trace in Jaeger; the profiler reads function names out of a stripped binary via `.gopclntab`, and 100 of 106 samples land in `main.lookupOrder`. One request, `GET /orders/42`, seen twice: first as a trace, then as CPU samples. ## Key points - Parts 2 and 4 of the series describe what OBI and the profiler can do; this note makes them do it with Docker Compose and no Kubernetes: one Go server nobody touches, OBI watching its requests, the profiler watching its CPU. - It all ran on one machine: a Colima VM (Ubuntu 24.04, kernel 6.8.0, linux/arm64, 2 CPUs, 2 GB) on an M4 Max laptop, not Docker Desktop. It's one run, a demo and not a benchmark. - The server: a handler that hashes the order ID 200,000 times so the profiler has CPU to sample, with `//go:noinline` keeping `lookupOrder` on the stack, as `checkout` does in part 4. The `GET /orders/{id}` pattern is Go 1.22 routing. No OpenTelemetry, no pprof imports. - The Dockerfile builds with Go 1.27.1, strips the binary, and ships it in an empty (scratch) image. - The stripping worked: `go tool nm` reports "no symbol section" and "no symbols", yet `strings` still shows `main.lookupOrder`, `main.main`, `main.main.func1`. The names live in `.gopclntab`, which the profiler reads later. - The compose file runs five services: the `orders` server, [Jaeger 2.21.0](https://www.jaegertracing.io/docs/2.21/getting-started/) for traces, a `curl` loop (`curlimages/curl:8.11.1`) sending `GET /orders/42` a few times a second, [OBI v0.10.0](https://github.com/open-telemetry/opentelemetry-ebpf-instrumentation/releases/tag/v0.10.0) (`otel/ebpf-instrument:v0.10.0`), and the profiler (`otel/opentelemetry-collector-ebpf-profiler:0.160.0`). - OBI gets `pid: "service:orders"`, putting it in the server's PID namespace (the server is PID 1 to it), and `OTEL_EBPF_OPEN_PORT: "8080"` tells it to instrument whichever process has port 8080 open. The OBI Docker docs ask for the host PID namespace; sharing only the server's was enough here. - The profiler gets `pid: host` to sample the whole node, and `--feature-gates=+service.profilesSupport` as its README uses. Collector `0.160.0` is the newest that bundles profiler v0.0.202633, the tag part 4 cites. The profiler config is inline because bind-mounting a file from `/tmp` into the VM gave an empty directory, and `/sys/kernel/debug` is mounted read-only because tracepoints are looked up there. - OBI's log shows `cmd=/orders pid=1 ino=1071739 type=go service="" logenricher=false`: it found `/orders`, recognised a Go binary and attached with no rebuild. The server never noticed; OBI was restarted 90 seconds later, attached again, and the server's start time stayed at `07:49:59`. - The trace: one server span `GET /orders/{id}` (kind 2, 11.63 ms) with `http.request.method = GET`, `http.response.status_code = 200`, `http.route = /orders/{id}`, `url.path = /orders/42`, plus two internal spans, `in queue` (0.02 ms) and `processing` (11.61 ms), both children of `88d93c2068b37f04`. `http.route` is the registered pattern; `url.path` is the raw path. - Give traces 15 to 20 seconds to show up. On the first attempt nothing appeared in 45 seconds though OBI's log looked fine; recreating the container (with debug logging on) fixed it, and the cause was never found. - A caller's trace is continued too: sending a `traceparent` header made the server span carry the chosen trace ID, with the sent span ID as parent. Passing context on to outgoing calls needs the extra capabilities part 2 lists; that wasn't turned on, so it's untested here. - The profiler's first start failed with "neither debugfs nor tracefs are mounted" (the `sched_process_free` tracepoint): the VM has debugfs, but a privileged container can't see it, hence the `/sys/kernel/debug` mount. With it, the profiler reports "Everything is ready" and prints a profile every five seconds. - 100 of 106 samples from `orders` have `main.lookupOrder` on the stack. With no symbol table, the stack still reads `main.lookupOrder`, `net/http.(*conn).serve` and `runtime.goexit`, because `.gopclntab` is still in the file. Read top-down, it is a text flame graph: the hashing loop, called from the handler, called from `net/http`. - Limits: the signal is Alpha, and this was one request shape on an idle VM; it shows the plumbing works, and the fleet's verdict is a separate experiment. A Compose file is a Kubernetes DaemonSet in costume, and part 6 has the dry-run route for a cluster. Drawing profiles needs an OTLP profiles backend; the README offers [devfiler](https://github.com/elastic/devfiler) (a development tool, it says) and Pyroscope. Versions, links and commands checked on 4 October 2026. ## A Go server with nothing in it Parts 2 and 4 of the [series](https://kakkoyun.me/series/how-to-instrument-go-without-changing-a-single-line-of-code/) describe what OBI and the profiler can do; here we make them do it with Docker Compose and no Kubernetes. We'll follow one request, `GET /orders/42`, and see it twice, first as a trace and then as CPU samples. This is the hands-on side of the [GopherCon UK talk](https://kakkoyun.me/talks/instrument-go-without-changing-a-single-line/). Here is the whole service. Its handler hashes the order ID 200,000 times so the profiler has CPU to sample, and `//go:noinline` keeps `lookupOrder` on the stack, as `checkout` does in [[resources/documentation/Go/Instrumentation/Go Instrumentation - Continuous Profiling Without Code Changes|part 4]]: ```go package main import ( "crypto/sha256" "fmt" "log" "net/http" ) //go:noinline func lookupOrder(id string) string { sum := sha256.Sum256([]byte(id)) for range 200_000 { sum = sha256.Sum256(sum[:]) } return fmt.Sprintf("order %s, checksum %x", id, sum[:4]) } func main() { http.HandleFunc("GET /orders/{id}", func(w http.ResponseWriter, r *http.Request) { fmt.Fprintln(w, lookupOrder(r.PathValue("id"))) }) log.Fatal(http.ListenAndServe(":8080", nil)) } ``` No OpenTelemetry, no pprof. The `GET /orders/{id}` pattern is [Go 1.22 routing](https://go.dev/blog/routing-enhancements). The Dockerfile builds with Go 1.27.1 and strips the binary, as in part 4, into an empty image: ```dockerfile FROM golang:1.27.1 AS build WORKDIR /src COPY go.mod main.go ./ RUN CGO_ENABLED=0 go build -ldflags='-s -w' -o /orders . FROM scratch COPY --from=build /orders /orders ENTRYPOINT ["/orders"] ``` Did the stripping work? The binary was copied out with `docker cp`: ```console $ go tool nm orders-linux-arm64 reading orders-linux-arm64: no symbol section reading orders-linux-arm64: no symbols $ strings orders-linux-arm64 | grep -E '^main\.' main.lookupOrder main.main main.main.func1 ``` No symbols, yet the function names are still in the file. That is `.gopclntab` again, and the profiler will read it later. First, the cast. ## The compose file: server, Jaeger, load, OBI, profiler Four more services join the server: [Jaeger 2.21.0](https://www.jaegertracing.io/docs/2.21/getting-started/) for traces (as in [[resources/documentation/Go/Instrumentation/Go Instrumentation - Agent-Actionable Observability|part 6]]), a `curl` loop sending `GET /orders/42` a few times a second, [OBI v0.10.0](https://github.com/open-telemetry/opentelemetry-ebpf-instrumentation/releases/tag/v0.10.0) and the profiler. Here is the whole file: ```yaml name: see-it-run services: orders: build: ./app jaeger: image: jaegertracing/jaeger:2.21.0 ports: - "16686:16686" load: image: curlimages/curl:8.11.1 entrypoint: ["/bin/sh", "-c"] command: - 'while true; do curl -s http://orders:8080/orders/42 > /dev/null; sleep 0.2; done' depends_on: - orders obi: image: otel/ebpf-instrument:v0.10.0 privileged: true pid: "service:orders" environment: OTEL_EBPF_OPEN_PORT: "8080" OTEL_EXPORTER_OTLP_ENDPOINT: http://jaeger:4318 depends_on: - orders - jaeger profiler: image: otel/opentelemetry-collector-ebpf-profiler:0.160.0 privileged: true pid: host command: - --config=/etc/profiler.yaml - --feature-gates=+service.profilesSupport volumes: - /sys/kernel/debug:/sys/kernel/debug:ro # tracepoints are looked up here - profiles:/out configs: - source: profiler target: /etc/profiler.yaml depends_on: - orders configs: profiler: content: | receivers: profiling: exporters: debug: verbosity: detailed file: path: /out/profiles.json service: pipelines: profiles: receivers: [profiling] exporters: [debug, file] volumes: profiles: ``` OBI's `pid: "service:orders"` puts it in the server's PID namespace, so the server is PID 1 to it. [`OTEL_EBPF_OPEN_PORT`](https://github.com/open-telemetry/opentelemetry-ebpf-instrumentation/blob/v0.10.0/pkg/obi/config.go#L381) tells it to instrument whichever process has port 8080 open. The [OBI Docker docs](https://opentelemetry.io/docs/zero-code/obi/setup/docker/) ask for the host PID namespace; sharing only the server's was enough here. The profiler gets `pid: host` to sample the whole node, and the feature-gate flag its [README](https://github.com/open-telemetry/opentelemetry-ebpf-profiler/blob/v0.0.202633/README.md) uses. Collector `0.160.0` is the newest that [bundles profiler v0.0.202633](https://github.com/open-telemetry/opentelemetry-collector-releases/blob/v0.160.0/distributions/otelcol-ebpf-profiler/manifest.yaml), the tag part 4 cites. Its config is inline because bind-mounting a file from `/tmp` into the VM gave an empty directory. First, what OBI makes of the server. ## OBI finds the server and continues the trace Compose builds and starts everything. Then we read OBI's log: ```console $ docker-compose up -d --build $ docker-compose logs obi | grep "instrumenting process" obi-1 | time=2026-10-04T07:50:01.412Z level=INFO msg="instrumenting process" component=discover.traceAttacher cmd=/orders pid=1 ino=1071739 type=go service="" logenricher=false ``` OBI found `/orders`, recognised a Go binary (`type=go`) and attached, with no rebuild. The server never noticed: OBI was restarted 90 seconds later, it attached again, and the server's start time stayed at `07:49:59`. Now Jaeger's query API, condensed to the fields that matter: ```console $ curl -s 'localhost:16686/api/v3/traces?query.service_name=orders&query.num_traces=1&query.start_time_min=...&query.start_time_max=...' span 'in queue' kind=1 parent='88d93c2068b37f04' duration=0.02ms span 'processing' kind=1 parent='88d93c2068b37f04' duration=11.61ms span 'GET /orders/{id}' kind=2 parent='-' duration=11.63ms http.request.method = GET http.response.status_code = 200 http.route = /orders/{id} url.path = /orders/42 ``` One server span (kind 2) with method, status and path, plus two internal spans, `in queue` and `processing`. `http.route` is the registered pattern; `url.path` is the raw path. Give traces 15 to 20 seconds to show up. On the first attempt nothing appeared in 45 seconds, though OBI's log looked fine. Recreating the container (with debug logging on) fixed it, and the cause was never found. A caller's trace is continued too. A `traceparent` header was sent and Jaeger was asked for the chosen trace ID: the server span carried it, and its parent was the span ID sent. Passing context on to outgoing calls needs the extra capabilities [[resources/documentation/Go/Instrumentation/Go Instrumentation - OBI eBPF Auto-Instrumentation|part 2]] lists; that wasn't turned on, so it's untested here. That's the trace. The same request also burned CPU, and a trace can't tell us where. ## The profiler reads a stripped binary Why not `net/http/pprof`? It needs an import and a redeploy. The eBPF profiler already watches every process on the node. Its first start on the VM failed: ```console Error: cannot start pipelines: failed to start "profiling" receiver: failed to attach scheduler monitor: failed to configure tracepoint on tracer.hookPoint{group:"sched", name:"sched_process_free"}: neither debugfs nor tracefs are mounted ``` The VM has debugfs, but a privileged container can't see it, hence the `/sys/kernel/debug` mount in the compose file. With it the profiler reports "Everything is ready" and prints a profile every five seconds. Here are two lines of the output, the server and the profiler's tag: ```text -> process.executable.path: Str(/orders) InstrumentationScope go.opentelemetry.io/ebpf-profiler v0.0.202633 ``` The server is in there, next to `dockerd`, `containerd`, OBI and the profiler itself. The debug output refers to frames by index, so we read the `file` exporter's JSON. This script (`lookup-share.py`) prints the first `orders` stack containing `main.lookupOrder`, then counts how many samples do: ```python import json, sys hit = total = 0 first = None for line in open(sys.argv[1]): doc = json.loads(line) d = doc["dictionary"] s = d["stringTable"] def frames(stack): locs = d["stackTable"][stack]["locationIndices"] return [s[d["functionTable"][ln["functionIndex"]]["nameStrindex"]] for i in locs for ln in d["locationTable"][i].get("lines", [])] for rp in doc["resourceProfiles"]: res = {s[a["keyStrindex"]]: a["value"] for a in rp["resource"]["attributes"]} exe = res.get("process.executable.name", {}).get("stringValueStrindex") if exe is None or s[exe] != "orders": continue for sp in rp["scopeProfiles"]: for prof in sp["profiles"]: for sample in prof.get("samples", []): total += 1 names = frames(sample["stackIndex"]) if "main.lookupOrder" in names: hit += 1 first = first or names print("\n".join(first)) print(f"{hit} of {total} samples from orders have main.lookupOrder on the stack") ``` ```console $ python3 lookup-share.py profiles.json crypto/internal/fips140/sha256.blockSHA2 crypto/internal/fips140/sha256.(*Digest).Write crypto/internal/fips140/sha256.(*Digest).checkSum crypto/internal/fips140/sha256.(*Digest).Sum crypto/sha256.Sum256 main.lookupOrder main.main.func1 net/http.HandlerFunc.ServeHTTP net/http.(*ServeMux).ServeHTTP net/http.serverHandler.ServeHTTP net/http.(*conn).serve net/http.(*Server).Serve.gowrap3 runtime.goexit 100 of 106 samples from orders have main.lookupOrder on the stack ``` The binary has no symbol table, yet the stack reads `main.lookupOrder`, `net/http.(*conn).serve` and `runtime.goexit`, because `.gopclntab` is still in the file. Read from the top, that's a text flame graph: 100 of 106 samples were in the hashing loop, called from the handler, called from `net/http`. Drawing it needs an OTLP profiles backend; the README offers [devfiler](https://github.com/elastic/devfiler) (a development tool, it says) and Pyroscope. ## What this doesn't show, and running it A Compose file is a Kubernetes DaemonSet in costume, and [[resources/documentation/Go/Instrumentation/Go Instrumentation - Agent-Actionable Observability|part 6]] has the dry-run route for a cluster. The signal is Alpha, and this was one request shape on an idle VM. It shows the plumbing works; the fleet's verdict is a separate experiment. The demo was run with the standalone `docker-compose` 5.6.0 binary because the `docker compose` plugin link was broken; either spelling works. Put `compose.yaml` next to an `app/` directory with `main.go`, the Dockerfile, and a `go.mod` (`module example.com/orders`, `go 1.27`). Then: ```bash docker compose up -d --build docker compose logs obi | grep "instrumenting process" docker compose logs profiler | grep "Everything is ready" docker compose down -v ``` Jaeger's UI is on `localhost:16686`. Try one of your own stripped binaries and see what names come back. One request, two signals, zero edits to the server; it's still the same 24 lines, and it doesn't know anyone is watching. Versions, links and commands checked on 4 October 2026. ## FAQ ### Can OBI instrument a Go binary without rebuilding it? Yes. OBI attached to `/orders` (pid 1) with no rebuild, recognised it as a Go binary (`type=go`), and produced traces in Jaeger. The server's start time never changed, even when OBI itself was restarted 90 seconds later. ### How does OBI know which process to instrument in Docker Compose? `OTEL_EBPF_OPEN_PORT: "8080"` tells it to instrument whichever process has port 8080 open, and `pid: "service:orders"` shares the server's PID namespace, so the server is PID 1 to OBI. ### How long do OBI traces take to appear in Jaeger? Give them 15 to 20 seconds. On one attempt nothing appeared in 45 seconds though OBI's log looked fine; recreating the container with debug logging on fixed it, and the cause was never found. ### Can the eBPF profiler profile a stripped Go binary? Yes. `go tool nm` reports no symbols, but `.gopclntab` stays in the file, so stacks still read `main.lookupOrder`, `net/http.(*conn).serve` and `runtime.goexit`. Here 100 of 106 samples from the server had `main.lookupOrder` on the stack. ### Why does the eBPF profiler fail with "neither debugfs nor tracefs are mounted"? A privileged container cannot see the host's debugfs even when the VM has it, so the `sched_process_free` tracepoint can't be configured. Mount `/sys/kernel/debug` read-only into the profiler container. ### Does OBI continue an incoming trace? Yes. Sending a `traceparent` header made the server span carry the chosen trace ID, with the sent span ID as its parent. Passing context on to outgoing calls needs the extra capabilities [[resources/documentation/Go/Instrumentation/Go Instrumentation - OBI eBPF Auto-Instrumentation|part 2]] lists, and that path is untested here. ### What is the difference between http.route and url.path in an OBI span? `http.route` is the registered pattern (`/orders/{id}`); `url.path` is the raw path of the request (`/orders/42`). ### How do I run OBI and the eBPF profiler without Kubernetes? Use the Docker Compose file: the server, Jaeger, a curl load loop, OBI and the profiler. `docker compose up -d --build`, check the logs for "instrumenting process" and "Everything is ready", and open Jaeger's UI on `localhost:16686`. ## Related - [[resources/documentation/Go/Instrumentation/Go Instrumentation|Go Instrumentation]] (series hub) - [[resources/documentation/Go/Instrumentation/Go Instrumentation - OBI eBPF Auto-Instrumentation|OBI eBPF Auto-Instrumentation]] (part 2: OBI's capabilities and requirements) - [[resources/documentation/Go/Instrumentation/Go Instrumentation - Continuous Profiling Without Code Changes|Continuous Profiling Without Code Changes]] (part 4: the eBPF profiler) - [[resources/documentation/Go/Instrumentation/Go Instrumentation - Flight Recording and USDT|Flight Recording and USDT]] (part 5: what Go itself might add) - [[resources/documentation/Go/Instrumentation/Go Instrumentation - Agent-Actionable Observability|Agent-Actionable Observability]] (part 6: the Kubernetes dry-run route) - [[resources/documentation/Go/Go - uretprobes|Go - uretprobes]] (related) - [[resources/documentation/Go/Go - Internals|Go - Internals]] (related) ## Sources - [How to Instrument Go Without Changing a Single Line of Code (talk, GopherCon UK 2026, GoLab 2026)](https://kakkoyun.me/talks/instrument-go-without-changing-a-single-line/) - [How to Instrument Go Without Changing a Single Line of Code (series)](https://kakkoyun.me/series/how-to-instrument-go-without-changing-a-single-line-of-code/) - [Go 1.22 routing enhancements](https://go.dev/blog/routing-enhancements) - [Jaeger 2.21 getting started](https://www.jaegertracing.io/docs/2.21/getting-started/) - [OBI v0.10.0 release](https://github.com/open-telemetry/opentelemetry-ebpf-instrumentation/releases/tag/v0.10.0) - [OBI config: OTEL_EBPF_OPEN_PORT at v0.10.0](https://github.com/open-telemetry/opentelemetry-ebpf-instrumentation/blob/v0.10.0/pkg/obi/config.go#L381) - [OBI Docker setup docs](https://opentelemetry.io/docs/zero-code/obi/setup/docker/) - [eBPF profiler README at v0.0.202633](https://github.com/open-telemetry/opentelemetry-ebpf-profiler/blob/v0.0.202633/README.md) - [otelcol-ebpf-profiler manifest at v0.160.0](https://github.com/open-telemetry/opentelemetry-collector-releases/blob/v0.160.0/distributions/otelcol-ebpf-profiler/manifest.yaml) - [devfiler](https://github.com/elastic/devfiler)