Profiling

Profiling Python with eBPF: A New Frontier in Performance Analysis Profiling Python applications can be challenging, especially in scenarios involving high-performance requirements or complex workloads. Existing tools often require code instrumentation, making them impractical for certain use cases. Enter eBPF (Extended Berkeley Packet Filter)—a versatile Linux technology—and the open-source project Parca, which together are reshaping how Python profiling works. I’ll cover how eBPF enables continuous profiling, the challenges of stack unwinding in Python, and what modern profiling tools can do with all of this. ...

Most Python profilers ask you to decide in advance. You import something, or you wrap the process, or you restart it with a flag. That works on your laptop and helps little at 3 a.m., when the profile you want is of a process that has already been running for six hours. eBPF changes the bargain. An agent samples from the kernel on a timer, and the application never knows it is there. For compiled code that is close to free: read the registers, walk the frames, resolve the addresses against DWARF. Python is where it stops being easy. ...

https://www.polarsignals.com/blog/posts/2023/10/04/profiling-python-and-ruby-with-ebpf/

I am too lazy now a days to re-post the blog post with all its assets and animations here. So until I get to it, I have put a link to it here. Enjoy :) https://www.polarsignals.com/blog/posts/2023/03/28/how-to-read-icicle-and-flame-graphs

Profiling has long been part of the Go developer’s toolbox to analyze the resource usage of a running process. But do you ever wonder how profilers built? In this talk, I will bring eBPF (a promising Kernel technology) and Go together to build a profiler for understanding Go code at runtime. Profiling has long been part of the developer’s toolbox to analyze the resource usage of a running process. Go users are very familiar with the concept thanks to state-of-art Go tooling. For years Google has consistently been able to cut down multiple percentage points in their fleet-wide resource usage every quarter, using techniques described in their “Google-Wide Profiling” paper, which is called continuous profiling. Through continuous profiling, the systematic collection of profiles, entirely new workflows suddenly become possible. ...