Skip to main content

eBPF-based Profiling

The CodeXray Node Agent includes a built-in eBPF-based CPU profiler. It continuously profiles every process on the node — with no application-side changes, no SDKs, and no configuration beyond installing the agent.

The simplest possible profiler

You don't "enable" profiling for a service. You install the Node Agent on a host, and within a minute every service running there starts showing up in its Profiling tab with a FlameGraph. That's the entire setup.

How It Works​

The profiler uses eBPF to sample CPU stack traces at the kernel level, at a fixed sample rate, for every process it sees. For each sample:

  1. eBPF reads the current kernel + user-space stack trace
  2. The agent resolves the raw addresses into function names using the binary's symbols (or language-specific unwinders)
  3. Samples are aggregated per-process over a collection window
  4. The aggregated profile is shipped to the CodeXray backend as pprof-format data

The backend stores every profile and the UI aggregates them into FlameGraphs — wider frames mean more CPU time spent in that function.

Key parameters​

  • Sample rate: 100 Hz (samples per second per process)
  • Collection interval: 1 minute (shipped to backend every minute)
  • Stack depth: full user-space stack, kernel frames are collected but not included by default

Supported Languages​

The eBPF profiler resolves symbols for most common runtimes:

Language / RuntimeSupportNotes
Go✓Native DWARF / frame-pointer unwinding. Works out of the box.
Python✓Specialised Python unwinder reads the interpreter frames for CPython.
Java / JVM✓Requires -XX:+PreserveFramePointer on the JVM — see Java Profiling.
Native binaries✓C, C++, Rust, and anything else compiled with symbols. DWARF or frame pointers.
Node.js / JavaScriptPartialV8 frames aren't resolved natively — you'll see the V8 interpreter but not JS function names.
.NET / CLRLimitedSimilar limitation — recommended to use OpenTelemetry profiling or pprof-rs integration if deeper depth is needed.

For languages not in the list, the profile still runs — you'll see CPU time attributed to the process, but function-level resolution may be limited. Typically the fix is compiler flags (enable frame pointers, preserve symbols).

Profile Types​

eBPF-based profiling covers CPU profiles only. It captures where your code is spending CPU time, but it cannot capture:

  • Memory allocations — requires runtime cooperation (e.g., Go's runtime.MemProfile)
  • Goroutine blocking — Go-specific, exposed only via pprof
  • Mutex contention — same as above
  • Lock contention / JVM GC pauses — runtime-specific

eBPF-based CPU profiling is the default across languages; those runtime-specific signals aren't captured through eBPF.

Requirements​

Kernel​

  • Linux kernel 4.16 or newer — eBPF features the profiler depends on. Kernel 5.4+ is recommended for the best unwinder support.

Architecture​

  • amd64 or arm64 — both fully supported.

Privileges​

  • The agent needs elevated capabilities (CAP_BPF, CAP_SYS_ADMIN, CAP_PERFMON). On Kubernetes, the DaemonSet manifest handles this automatically.

Binaries (for best symbol resolution)​

  • Compile with frame pointers enabled (most languages do this by default; some strip them for marginal performance gains).
  • Keep debug symbols (DWARF) in the binary or install -dbgsym / debug packages for installed binaries.
  • Don't strip binaries aggressively — strip --strip-unneeded is fine, strip --strip-all can hurt resolution.

How to Enable​

eBPF profiling is on by default when the Node Agent is installed. No flags, no configuration.

To install the Node Agent (if not already deployed):

  • Kubernetes: one manifest creates the DaemonSet and permissions. See the Node Agent installation guide.
  • Linux VMs: one install.sh registers the systemd service.
  • Docker: one docker run with host networking, PID namespace, and cgroup mounts.

Within a minute or two, open any application → Profiling tab → the FlameGraph starts populating.

Verifying It's Working​

  1. Node Agent health: kubectl -n codexray-agents get pods or systemctl status codexray-node-agent. Agent should be running.
  2. Profile data: open any application → Profiling tab. A FlameGraph should render within a few minutes.
  3. Node Agent logs: look for messages mentioning the ProfilesEndpoint and periodic upload lines. Any missing-symbols or unwinder errors are logged explicitly.
Troubleshooting "unresolved frames"

If a FlameGraph shows a lot of <unknown> or hex-address frames, the binary in question is either stripped, missing debug symbols, or uses a runtime the profiler can't unwind natively. The fix is usually at build time — keep DWARF or add frame pointers. For Java, set -XX:+PreserveFramePointer.

Next Steps​