Repository navigation
perf(c): profile and optimize hot-path kernels #74
Description
Motivation
Profiling the current package shows that nearly all representative runtime is spent in the compiled C entry points, not R dispatch. This issue tracks performance work separately from the shared-memory safety fixes in #55.
Candidate work
- Remove the redundant
lvinitialization inpsortbefore the value is unconditionally assigned. - Replace full-size hash tables in
funiqueandpsortwith growable tables sized to the observed distinct count, especially for low-cardinality data. - Fuse default initialization and matching passes in
vswitch. - Add a bounded fallback or heap strategy to
topnfor adversarial ordered inputs while preserving the small-nfast path. - Evaluate fused initialization and mixed-type conversion in
psumwithout changing NA/NaN or integer-overflow semantics. - Evaluate direct serialization in
shareData; zero-copy/protocol redesign remains separate in shareData() without deep copy? #43.
Acceptance
- Record before/after benchmarks for representative large-vector workloads.
- Preserve ordering, attributes, factors, encoding behavior,
fromLast, and NA/NaN semantics. - Keep public API and shared-memory protocol changes out of this issue unless separately agreed.
- Use separate, focused pull requests for each optimization family.
Activity
perf(psum,c): fuse initialization pass (#74)
Merge pull request #76 from fastverse/perf/74-psum-fusion-master
davidbudzynski commentedon Sep 28, 2026
Status update on the candidate list, re-verified against the current
master.Done
psum— landed in perf(psum,c): fuse initialization pass (#74) #76 (perf(psum,c): fuse initialization pass). NA/NaN and integer-overflow semantics preserved.Dropping — no longer applicable
lvinitialization inpsort. There is no initializer to remove.src/psort.c:474readsint *restrict lv = (int*)R_alloc(xlen, sizeof(int));and every element is assigned before use atsrc/psort.c:478. The adjacentposarray is genuinelymemset-initialised because it is a counting array, so that one stays. No code change needed.Still open
funiqueandpsort. Confirmed still full-size:src/dup.c:84,src/dupLen.c:78,src/dupLen.c:121,src/dupLen.c:324, andsrc/psort.c:227/src/psort.c:264all size their tableMup front viaR_Calloc(M, int). Best remaining candidate for low-cardinality data.vswitch. Still two passes: the init loop atsrc/vswitch.c:137-139writes the default (pans[j]= nonna ? pna[j & namask] : NA_LOGICAL;) before the separate match loop.topnon adversarial ordered input. Unchanged; thelen0 > 2000/len0 > 1500radix shortcut atsrc/topn.c:44is still the only large-npath.shareData. Unchanged, stillserialize()+rawConnectionValue()inR/call.R:79-90.Leaving this issue open for the remaining four. The zero-copy protocol redesign stays in #43 per the original split.
Note that the long-vector counters noted in #61 still intersect this work in
src/dup.c:813-817,861andsrc/psort.c:389-408,569-616; worth sequencing the hash-table change after that audit so the sizing arithmetic is done onR_xlen_tfrom the start.