Repository navigation
perf(lcao): accelerate RI preparation and Libxc evaluation - #8193
Draft
Stardust0831 wants to merge 26 commits into
Draft
Stardust0831 wants to merge 26 commits into
Stardust0831 wants to merge 26 commits into
Conversation
Evaluate displacement-dependent harmonics and radial interpolation once for each Orb11 magnetic block, and reuse the vector path in Orb21 assembly. Extend scalar-reference and finite-difference coverage without changing numerical thresholds.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Depends on #8169. The GPU path uses its LibRI/DDLA integration. The current diff includes prerequisite commits; the optimization changes can be reviewed separately.
What's changed?
Fresh LCAO HSE calculations can spend most of their initialization time rebuilding orbital products and evaluating radial integrals. This change batches GPU radial quadrature, reuses common integral terms, and corrects Libxc OpenMP scheduling that left available threads idle.
Explicit radial GPU buffers and host staging are each bounded to 160 MiB per MPI rank and released before SCF. CUDA context and library allocations are outside this bound. Orbital-pair and radial dimensions are tiled; the reciprocal quadrature mesh is limited to 2,097,152 points. Product sharing reduces duplicate host storage; common species tables remain replicated across ranks.
The optimization adds no further INPUT option or dependency. GPU batching follows the RI backend selection from #8169. Shared integral reuse also benefits CPU execution, and the Libxc scheduling fix applies wherever these routines are used.
Linked Issue
Measurements and numerical checks: Stardust0831/abacus-develop#30. Extending full-table CPU preprocessing is outside this PR and tracked in Stardust0831/abacus-develop#31.
Unit Tests and/or Case Tests for my changes
Validation used V100 GPUs on SAI. At
1aaeb8ff4, a clean GPU executable build and the CPU/CUDA integral tests pass. Integral tests cover high angular momenta, full/partial batches, radial tile boundaries, origin/cutoff behavior, shared-product lifetime, magnetic components and finite-difference gradients. The existing Libxc test passes with one and four threads.The main comparison (
63ca5ef073to5156a5d4e) includes 77 fresh calculations and 16 matched comparisons: distorted Si8 at two k points, CsPbI3 at Gamma, CPU/GPU RI, displaced geometries, and one/four OpenMP threads. A further 12 calculations compare5156a5d4ewith810d4909c, covering Si64, Cs/Pb/I and complex two-k-point Si8.The dependency update (
810d4909cto1aaeb8ff4, based on #8169 at3fa1f972b) is covered by six fresh calculations: distorted two-k-point Si8 with one OpenMP thread and Gamma-point CsPbI3 with four. Four paired checks compare the two GPU revisions and CPU/GPU RI within the current revision. All calculations converge; input and asset checks pass. All 18 upstream PR checks pass at1aaeb8ff4.All matched numerical checks pass. Maximum absolute differences:
Absolute energy/force finite differences still exceed 1e-4 eV/Angstrom for some inputs; the original implementation reproduces these residuals. The tests establish implementation consistency, not fully converged absolute force accuracy. Independent strain derivatives, long MD and multi-node execution remain untested.
Performance measurements cover the main optimizations (
63ca5ef073to6283aea20). Both versions use GPU RI and cuSOLVER, one MPI rank and four OpenMP threads per GPU, and single-thread BLAS. Numerical settings and iteration counts match. Single-point and SCF-plus-force times include fresh initialization and process startup; RI initialization uses the rank-zero stage timer.The eight-GPU comparisons reverse execution order within the same allocation: 64 logical CPUs corresponding to 32 physical cores, and 120 GiB host memory. Historical per-thread affinity was not recorded and the node hosted another job. These ratios measure software changes within the GPU implementation; the DDLA force contraction is unchanged.
Verification commands and additional coverage
1aaeb8ff4. The CPU-only Si32 force comparison passed at26d01a67dand has not been repeated after the angular change.Governance Notes
<memory>is required by the owningstd::shared_ptrmember. No governance exception is requested.git diff --check 3fa1f972b..1aaeb8ff4passes.python3 tools/03_code_analysis/agent_governance_check.py --base 3fa1f972b --head 1aaeb8ff4 --event-path "$PR_EVENT" --format textreports no errors and only the header warning explained above.Contribution checklist
AGENTS.mdanddocs/developers_guide/agent_governance.md.