# Trust a Go benchmark on your laptop: pinning, taskset, perflock > [!summary] TL;DR > `BenchmarkMakeBuffer_Correct` measured 11.46 ns/op at 4.75% CV on an idle Apple M4 Max, 34.97 ns/op at 18.88% CV with 16 background spinners, and 16.28 ns/op at 5.25% CV inside a container pinned with `--cpuset-cpus=0 --cpus=1 --memory=512m`. A pinned container on a Mac fixes variance, not absolute numbers. On Linux go further with `taskset`, `isolcpus`, `cset shield`, `nice`, `chrt` and `perflock`. ## Key points - One benchmark, three conditions, one Apple M4 Max (darwin/arm64, 16 logical CPUs), `-count=20 -benchtime=1s` per condition: idle host 11.46 mean ns/op, 0.54 stddev, 4.75% CV; host with 16 background spinners 34.97, 6.60, 18.88%; container pinned to vCPU 0 with the same spinners 16.28, 0.85, 5.25%. - The "noisy" condition starts 16 background spinners: pure busy-loops with no I/O and no allocation that saturate every logical core. The third reruns the benchmark on that same noisy host inside a `golang:1.26` container started with `--cpuset-cpus=0 --cpus=1 --memory=512m`. One run of 20 samples per condition on one machine; the host's Go version was not recorded. - Idle to noisy: the benchmark slowed by 3x and CV jumped from 4.75% to 18.88%. The spinners and the benchmark compete for the same 16 cores, so the scheduler preempts the benchmark and moves it between cores, including the slower efficiency cores. The experiment does not separate those effects. - Noisy to pinned: CV drops to 5.25%, nearly back to the idle noise floor, but the absolute runtime stays higher (16.28 ns/op against 11.46 idle). The pinned run's output has no `-16` suffix, so `GOMAXPROCS=1` inside the container. At one run per condition, the gap between 4.75% and 5.25% is within what sampling error alone could produce. - Docker Desktop on macOS runs containers inside a Linux VM (here, Apple's Virtualization.framework). `--cpuset-cpus=0` pins to vCPU 0 inside that VM, not to a physical core on the host; the host scheduler can migrate that vCPU, and nothing inside the container can pin the host CPU clock frequency. - Pinning buys a stable CPU for the benchmark's threads (they stop migrating across the VM's virtual CPUs) and a VM with nothing else running in it. Frequency scaling, thermal variation and the host's choice of physical core for the VM's threads all remain in play. - On a bare-metal Linux runner with SMT disabled and CPU frequency pinned, the FOSDEM 2026 experiments measured around 0.05% CV on a CPU-bound task: roughly a hundred times tighter than Docker on a Mac, across different workloads. - Mac numbers are good for catching obvious regressions during development, not for publication. Results that need to hold up to scrutiny belong on bare-metal Linux with SMT and frequency scaling disabled, with a note in the writeup saying so. - Linux tools: `taskset -c 0` (the Go runtime reads the affinity mask, so it also sets `GOMAXPROCS=1`, changing GC scheduling and parallel benchmarks); `isolcpus=2,3 nohz_full=2,3 rcu_nocbs=2,3` on the kernel command line (needs reboot and root, suits a dedicated CI runner); `cset shield` (cgroup v1 hosts only; `--kthread=on` is critical or kernel threads still run on the shielded cores); `nice -n -5`; `chrt -f 50` (SCHED_FIFO can starve normal processes, the display server included; the kernel's default real-time throttling leaves them about 5%). - `perflock`, written by Austin Clements of the Go team, locks the CPU to a stable frequency for the run by writing `scaling_min_freq` and `scaling_max_freq` through the cpufreq sysfs interface, via a small daemon: one of the higher-value local tools after `benchstat`. It builds on macOS, but there the governor step fails inside the daemon and the error is discarded, leaving only the mutual-exclusion lock. - Thermal steady state: a benchmark that starts on a cold chip and ends on a warm one shows a decrease across the run that has nothing to do with the code. A throwaway run before capturing helps, and waiting 30 to 60 seconds between major runs is a rule of thumb, not a measured number. - The toolchain is a condition too: since Go 1.21 the `go` directive in `go.mod` is a minimum, not a pin, and different Go versions produce different numbers. Pin with `GOTOOLCHAIN=go1.26.5` and mention the version in any result you share. - Cheap wins: close the browser, IDE and Slack; on macOS turn off Spotlight indexing with `sudo mdutil -a -i off` (undo with `-i on`); switch off Wi-Fi and Bluetooth to cut NIC interrupts. On a noisy machine, closing the browser alone can noticeably lower CV. - Versions, links and commands checked on 2 October 2026. ## One benchmark, three conditions `BenchmarkMakeBuffer_Correct` costs 11.46 ns/op on an idle Apple M4 Max. Start sixteen busy loops next to it and the same code costs 34.97 ns/op. Pin it to a single CPU inside a container and it settles at 16.28 ns/op, with its variance back near the idle level. The benchmark ran with `-count=20 -benchtime=1s` on an Apple M4 Max (darwin/arm64, 16 logical CPUs) under three conditions. The "noisy" one starts 16 background spinners: pure busy-loops with no I/O and no allocation that saturate every logical core. The third reruns the benchmark on that same noisy host inside a `golang:1.26` container started with [`--cpuset-cpus=0 --cpus=1 --memory=512m`](https://docs.docker.com/engine/containers/resource_constraints/). The spinners never stop. | Condition | Mean ns/op | Stddev | CV% | |---|---|---|---| | Idle host | 11.46 | 0.54 | 4.75 | | Host with 16 background spinners | 34.97 | 6.60 | 18.88 | | Container pinned to vCPU 0, same 16 spinners | 16.28 | 0.85 | 5.25 | That's one run of 20 samples per condition on one machine. The host's Go version wasn't recorded, which is a little embarrassing for a post about reproducibility. Idle to noisy: the benchmark slowed by 3x and CV jumped from 4.75% to 18.88%. Both effects most likely share a cause: the spinners and the benchmark compete for the same 16 cores, so the scheduler preempts the benchmark and moves it between cores, including the slower efficiency cores on this chip. This experiment does not separate those effects. Noisy to pinned is the interesting one. CV drops to 5.25%, nearly back to the idle noise floor. The absolute runtime stays higher than idle (16.28 ns/op against 11.46), which is unsurprising for a single pinned vCPU (`GOMAXPROCS=1`, judging by the missing `-16` suffix in the pinned run's output) inside a VM running a different OS, though this experiment does not isolate which of those accounts for it. The variance is the point, and it collapses, though at one run per condition the gap between 4.75% and 5.25% is within what sampling error alone could produce. To reproduce the three conditions, clone the demo at the pinned commit. This needs Docker, and the container step mounts the current directory, so run it from a path your Docker VM can see: ```bash git clone https://github.com/kakkoyun/gopherconuk-26 cd gopherconuk-26 git checkout fdce88cc0ce129b7d2edfb1a20d02fe647c17eeb cd talks/go-benchmarks-lying/demo make bench-docker make cv ``` `make bench-docker` re-runs all three conditions (expect a minute or two) and overwrites `results/`, so your numbers will differ. `make cv` recomputes the summary from whatever is in `results/`. ## What a pinned container on a Mac buys On a bare-metal Linux runner with SMT disabled and CPU frequency pinned, the [FOSDEM 2026 experiments](https://github.com/igoragoli/fosdem-2026-software-performance) measured around 0.05% CV on a CPU-bound task. That is roughly a hundred times tighter, across different workloads, than Docker on a Mac gave us. Docker Desktop on macOS runs containers inside a Linux VM (here, Apple's Virtualization.framework). When we pass `--cpuset-cpus=0`, we pin to vCPU 0 inside that VM, not to a physical core on the host. The host scheduler can migrate that vCPU between physical cores at will, and nothing inside the container can pin the host CPU clock frequency. What the pinning does buy is a stable CPU for the benchmark's threads, which stop migrating across the VM's virtual CPUs, and a VM with nothing else running in it. The table above shows that matters. Frequency scaling, thermal variation and the host's choice of physical core for the VM's threads all remain in play. The Mac numbers are good for catching obvious regressions during development, not for publication. Results that need to hold up to scrutiny belong on bare-metal Linux with SMT and frequency scaling disabled, with a note in the writeup saying so. ## Linux toolbox: taskset, isolcpus, cset shield, nice, chrt On Linux we can reach the knobs the Mac's VM hides. ### CPU affinity The OS scheduler migrates processes between cores to balance load, and each migration evicts warm cache lines. `taskset` stops that at zero setup cost: ```bash taskset -c 0 go test -bench=. -count=10 -benchtime=2s ./... ``` The `-c 0` pins the whole run to core 0. It's worth it on Linux whenever we run `-count` of 10 or more. One side effect: the Go runtime reads the affinity mask, so this also sets `GOMAXPROCS=1` for the benchmark, which changes GC scheduling and any parallel benchmarks. Free isn't the same as harmless. ### Core isolation With `isolcpus` on the kernel command line, specific cores leave the scheduler's load balancing, so only tasks we pin to them run there: ```text # Add to kernel command line isolcpus=2,3 nohz_full=2,3 rcu_nocbs=2,3 ``` After a reboot we pin benchmarks to the isolated cores with `taskset -c 2`. That suits a dedicated CI runner, since it needs a reboot and root. (`nohz_full` stops the scheduler tick on those cores while only one task runs there, and `rcu_nocbs` moves RCU callbacks off them; see the [kernel NO_HZ docs](https://docs.kernel.org/timers/no_hz.html).) For a softer version that needs no reboot, `cset shield` achieves similar isolation at runtime, but only on hosts that still use cgroup v1: ```bash sudo cset shield --cpu=2,3 --kthread=on sudo cset shield --exec -- go test -bench=. -count=10 ./... sudo cset shield --reset ``` Create the shield, run inside it, tear it down. The `--kthread=on` flag is critical: without it, kernel threads still run on the shielded cores. ### Process priority `nice` raises the scheduling priority of the benchmark relative to other processes: ```bash sudo nice -n -5 go test -bench=. -count=10 ./... ``` The effect is modest on a quiet machine but meaningful under moderate load. On dedicated machines, `chrt -f` (SCHED_FIFO) goes real-time: ```bash sudo chrt -f 50 go test -bench=. -count=10 ./... ``` SCHED_FIFO can starve normal processes, the display server included, if the benchmark misbehaves (the kernel's default real-time throttling leaves them about 5%). That's a fine way to lose your terminal, so keep it to dedicated machines or a resource-capped container. ## CPU frequency control: perflock, sysfs and thermals Dynamic frequency scaling (DFS) is a large source of hidden variance on Linux: the CPU boosts when thermals allow and throttles when they do not. [`perflock`](https://github.com/aclements/perflock), written by Austin Clements of the Go team, locks the CPU to a stable frequency for the duration of the run. It talks to a small daemon that holds the lock, so the setup has two steps: ```bash GOBIN=$PWD go install github.com/aclements/perflock/cmd/perflock@b67f3f23152f sudo install ./perflock /usr/bin/perflock sudo -b perflock -daemon perflock go test -bench=. -count=10 -benchtime=2s ./... ``` The first three lines build it at a pinned commit, install it and start the daemon. The last runs the benchmark under the lock. On Linux it pins the frequency by writing `scaling_min_freq` and `scaling_max_freq` through the cpufreq sysfs interface, which makes it one of the higher-value local tools after `benchstat`, daemon and all. - On macOS: `perflock` builds, but its frequency pinning relies on the Linux cpufreq sysfs interface. On a Mac the governor step fails inside the daemon and the error is discarded, leaving only the mutual-exclusion lock: no racing benchmark runs, and no help with frequency noise. To apply the sysfs controls by hand: ```bash # Performance governor echo performance | sudo tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor # Disable Intel Turbo Boost (intel_pstate driver) echo 1 | sudo tee /sys/devices/system/cpu/intel_pstate/no_turbo # Disable frequency boost (acpi-cpufreq or amd-pstate drivers) echo 0 | sudo tee /sys/devices/system/cpu/cpufreq/boost # Disable SMT (caution: halves logical CPU count) echo off | sudo tee /sys/devices/system/cpu/smt/control # Re-enable when done echo on | sudo tee /sys/devices/system/cpu/smt/control ``` The bare-metal SMT and DFS numbers behind these controls are in [[resources/documentation/Go/Benchmarks/Go Benchmarks - Benchmark CI Noise|part 4 of the series]]. Thermal steady state: CPUs boost when cool and throttle when hot. A benchmark that starts on a cold chip and ends on a warm one shows a performance decrease across the run, which has nothing to do with the code. In raw `go test` output, the first few runs of a `-count=20` sequence are typically faster than the last few (the expected pattern, not one we measured here). A throwaway run before capturing helps, as does waiting 30 to 60 seconds between major runs (a rule of thumb, not a measured number). Holding the frequency steady with `perflock` or by disabling DFS helps regardless of thermals. Laptops are worse than desktops here: ambient temperature, prior CPU workload and fan behavior all affect how long steady state takes. Skip the warmup and the early samples are biased warm. ## Go-native A/B comparison, toolchain pinning and cheap wins Controls reduce noise, but a structured way to compare two versions is also needed. The manual approach: capture a baseline, apply the change, capture again and let `benchstat` compare: ```bash # Capture baseline go test -bench=BenchmarkMyFunc -benchmem -count=10 -benchtime=2s . > old.txt # Apply change, then capture go test -bench=BenchmarkMyFunc -benchmem -count=10 -benchtime=2s . > new.txt # Compare benchstat old.txt new.txt ``` The flags and machine conditions have to match on both sides, or we end up benchmarking the room instead of the code. Captured minutes apart, the two files can also see different machines. The benchstat docs recommend interleaving the before and after runs, and `go test -c` makes that cheap: build both test binaries once, then alternate them: ```bash GOOS=linux go test -c -o old.test # on the base commit GOOS=linux go test -c -o new.test # with the change for i in $(seq 10); do ./old.test -test.run='^ -test.bench=. -test.count=1 >> old.txt ./new.test -test.run='^ -test.bench=. -test.count=1 >> new.txt done benchstat old.txt new.txt ``` On a Mac, the `GOOS=linux` builds run inside the pinned container (`docker run --cpuset-cpus=0 -v "$PWD":/w -w /w ...`, with the loop as the command) so both sides share its conditions. For the inner development loop, [`benchdiff`](https://github.com/willabides/benchdiff) automates the cycle, with the base ref running in a temporary git worktree: ```bash go install github.com/willabides/benchdiff/cmd/[email protected] benchdiff --base-ref=main --benchmem --count=10 --benchtime=2s ``` It runs both sides under the same conditions and compares them with a bundled, older copy of `benchstat`. The toolchain is a condition too, and the one we forgot to record in our own table. Since Go 1.21 the `go` directive in `go.mod` is a minimum, not a pin: a newer local toolchain runs the module anyway. Different Go versions produce different numbers, so pin with `GOTOOLCHAIN=go1.26.5` and mention the version in any result you share. Cheap wins that take five minutes: close the browser, IDE and Slack, the most expensive background processes on most laptops. On macOS, turn off Spotlight indexing with `sudo mdutil -a -i off` so the indexer doesn't spike mid-run, and undo it afterwards with `sudo mdutil -a -i on`. Switch off Wi-Fi and Bluetooth (airplane mode, if your OS has one) to cut NIC interrupts and background traffic. On a noisy machine, closing the browser alone can noticeably lower CV. Back to the three rows: 4.75% CV on an idle Mac, 18.88% with the spinners running, 5.25% in the pinned container. On a Mac, we run `-count=20 -benchtime=2s`, watch the `±` column in benchstat (a 95% confidence interval, not CV), compute CV as in [[resources/documentation/Go/Benchmarks/Go Benchmarks - Statistics and benchstat|part 2]], and treat a CV above 5% as a reason to investigate the environment and above 10% as a reason not to trust the comparison at all. Everywhere, that means `-count` of 10 or more, benchstat, `benchdiff` and closed apps. On Linux we add `taskset`, `perflock` and `nice`. On a dedicated machine or runner we add SMT and frequency control, `isolcpus` and `cset shield`. On a Mac, a pinned container helps with variance, not absolute numbers. Versions, links and commands checked on 2 October 2026. ## FAQ ### Can I trust a Go benchmark run on my laptop? Sometimes. On an idle Mac the demo measured 4.75% CV; with 16 background spinners the same code was 3x slower and four times noisier (18.88% CV). Treat a CV above 5% as a reason to investigate and above 10% as a reason not to trust the comparison. ### Does a pinned Docker container make Mac benchmarks reliable? It fixes variance, not absolute numbers: CV dropped from 18.88% to 5.25% while the mean stayed at 16.28 ns/op. `--cpuset-cpus=0` pins a vCPU inside the Docker VM, not a physical core, and nothing inside the container can pin the host CPU clock frequency. ### How do I pin a Go benchmark to one CPU on Linux? Run it under `taskset -c 0 go test -bench=. -count=10 -benchtime=2s ./...`. The Go runtime reads the affinity mask and sets `GOMAXPROCS=1`, which changes GC scheduling and any parallel benchmarks. ### What is perflock? A tool by Austin Clements that locks the CPU to a stable frequency for the run by writing `scaling_min_freq` and `scaling_max_freq` through the cpufreq sysfs interface, via a small daemon. On macOS the governor step fails silently and only the mutual-exclusion lock remains. ### How do I compare two Go commits locally? Build both test binaries once with `go test -c` and alternate them, then run `benchstat old.txt new.txt`; `benchdiff --base-ref=main` automates the cycle with the base ref in a git worktree. Keep flags and machine conditions identical on both sides. ### How do I quiet my machine before benchmarking? Close the browser, IDE and Slack. On macOS disable Spotlight indexing with `sudo mdutil -a -i off` and switch off Wi-Fi and Bluetooth to cut NIC interrupts. Closing the browser alone can noticeably lower CV on a noisy machine. ### Do I need a warmup run before benchmarking? Yes. CPUs boost when cool and throttle when hot, so a cold start biases the early samples. Run a throwaway capture first and wait 30 to 60 seconds between major runs. ## Related - [[resources/documentation/Go/Benchmarks/Go Benchmarks|Go Benchmarks]] (series hub) - [[resources/documentation/Go/Benchmarks/Go Benchmarks - Statistics and benchstat|Statistics and benchstat]] (part 2: CV and the run-count rules) - [[resources/documentation/Go/Benchmarks/Go Benchmarks - Benchmark CI Noise|Benchmark CI Noise]] (part 4: bare-metal SMT and DFS numbers) - [[resources/documentation/Go/Benchmarks/Go Benchmarks - Laptop Said 230 Percent|Laptop Said 230 Percent]] (a loaded laptop and CI disagreeing about one change) ## Field notes from agent sessions (2026-11-10) Follow-up benchmark sessions on the same laptop filled two gaps in the macOS picture: which knobs the operating system actually offers, and what perflock does when it cannot pin. - Docker Desktop's Linux VM cannot touch host SMT or Turbo. The remaining gap to bare metal is a property of the VM. Measure the gap. Do not assert it. - Coefficient of variation is the right statistic for environment noise. benchstat deliberately does not report it. A small awk script of about 20 lines fills that gap. - `perflock -governor=none` on macOS bypasses CPU-frequency pinning, because the Linux `/sys` cpufreq interface does not exist there. The mutual-exclusion locks still work. Benchmarks stay contention-free without a frequency governor. - macOS has no CPU-frequency API. `taskpolicy -c` clamps QoS only downward. It offers `utility`, `background` and `maintenance`, and no option for performance cores. `pmset powermode` (values 0, 1, 2) is the only knob with a governor-shaped save/set/restore cycle. High Power mode silently does nothing on unsupported hardware. - perflock silently discarded `SetGovernor` errors, so an unpinned CPU looked pinned. A pinning tool must fail loudly when pinning fails. Otherwise every macOS run is silently uncontrolled. ### Observations - [bench] Coefficient of variation captures environment noise. benchstat omits it by design, and a ~20-line awk script fills the gap. #benchmarks (source: agent sessions, 2026-08) - [bench] The remaining CV gap between a pinned container and bare metal is a property of the Linux VM. Measure it. Do not assert it. #docker (source: agent sessions, 2026-08) - [macos] `perflock -governor=none` keeps the contention locks and skips the frequency governor on macOS. #perflock (source: agent sessions, 2026-08) - [macos] `pmset powermode` is the only save/set/restore knob for CPU power on macOS. `taskpolicy -c` clamps QoS downward only. #macos (source: agent sessions, 2026-08) - [tooling] perflock discarded `SetGovernor` errors, so an unpinned CPU looked pinned. A pinning tool must fail loudly. #reliability (source: agent sessions, 2026-08) ## Sources - [Why Your Go Benchmarks Are Lying (talk, GopherCon UK 2026, GoLab 2026)](https://kakkoyun.me/talks/why-your-go-benchmarks-are-lying/) - [Why Your Go Benchmarks Are Lying (series)](https://kakkoyun.me/series/why-your-go-benchmarks-are-lying/) - [Measuring Software Performance: Why Your Benchmarks Are Probably Lying (FOSDEM 2026 post)](https://kakkoyun.me/posts/fosdem-2026-measuring-software-performance/) - [Docker resource constraints](https://docs.docker.com/engine/containers/resource_constraints/) - [Kernel NO_HZ docs](https://docs.kernel.org/timers/no_hz.html) - [perflock (Austin Clements)](https://github.com/aclements/perflock) - [FOSDEM 2026 software performance experiments](https://github.com/igoragoli/fosdem-2026-software-performance) - [benchdiff](https://github.com/willabides/benchdiff) - [benchstat documentation](https://pkg.go.dev/golang.org/x/[email protected]/cmd/benchstat) - [gopherconuk-26 repository](https://github.com/kakkoyun/gopherconuk-26) - [Demo at commit fdce88cc](https://github.com/kakkoyun/gopherconuk-26/tree/fdce88cc0ce129b7d2edfb1a20d02fe647c17eeb/talks/go-benchmarks-lying/demo)