Repository navigation
fix(cel): tolerate exited processes during environment lookup - #1022
Conversation
Signed-off-by: Matthias Bertschy <matthias.bertschy@gmail.com>
|
Warning Review limit reachedYou've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Next included review available in 8 minutes. View limit detailsLimit details: You鈥檝e used the included review currently available. Review configuration: 鈿欙笍 Run configuration
馃搾 Files selected for processing (3)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Signed-off-by: Matthias Bertschy <matthias.bertschy@gmail.com>
Performance Benchmark ResultsNode-Agent Resource Usage
Dedup EffectivenessNo data available. |
There was a problem hiding this comment.
馃煝 Approval recommended
The implementation matches the described behavior and includes focused regression coverage.
0 open findings
What changed in this PR
Handles expected /proc/<pid> races when processes exit before CEL environment lookup.
Changes:
- Converts exited-process errors to empty environments after caching.
- Preserves permission and unrelated failures.
- Adds regression and cache-behavior tests.
| File | Description |
|---|---|
processlib.go |
Applies the fallback after cache lookup. |
process.go |
Classifies ENOENT and converts exited-process results. |
process_env_exited_test.go |
Tests exited processes, error preservation, and caching. |
馃 Review effort: Balanced
馃挕 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Performance Benchmark ResultsNode-Agent Resource Usage
Dedup EffectivenessNo data available. |
When a process exits before an exec rule reads
/proc/<pid>/environ,process.get_process_envcurrently returns a CEL error. R1180 attempts this lookup for every evaluated exec event, turning normal process churn into error-level rule-evaluation logs.In a read-only 10-minute sample from an affected EKS cluster, 463 environment lookup errors appeared across 3 of 27 node-agent pods. All were R1180 failures caused by
stat /proc/<pid>: no such file or directory; none were permission failures. Increased CPU was reported alongside the errors, but the CPU cost attributable to logging versus live reads has not been measured.Treat ENOENT from the target PID lookup or environment read as an exited process. After the function cache, convert that specific error to an empty environment so rules with variable-presence guards evaluate to false. Keeping the conversion after caching prevents caching a missing process as an empty map for a subsequently reused PID. Preserve permission, procfs-initialization, and other failures as errors.
This reduces expected error-log noise; it does not eliminate per-exec environment I/O or the detection gap for processes that have already exited.
Validation:
failed to get process environmenterror before the fix and passes afterward.go test ./pkg/rulemanager/cel/...passes.go test -race ./pkg/rulemanager/cel/libraries/process ./pkg/rulemanager/cel/libraries/cache -count=1passes.make binarypasses.--new-from-rev=HEAD; the unrestricted targeted run reports two existing QF1003 findings in unchangedprocess_test.go.git diff --checkpasses.Follow-up path/permission experiment:
hostPID: trueand run as root withSYS_PTRACE./proc/1and/host/proc/1are systemd in the same host PID namespace./proc/<kubelet-pid>/environand/host/proc/<kubelet-pid>/environwere readable; contents were discarded.These checks rule out a general proc-path or permission mismatch in the tested pod and strongly support the exit-race explanation. They do not correlate every original failing event with an observed process exit.