Repository navigation
Keep CSR column ids int32 when the edge index is int32 - #788
Conversation
GiGL Automation@ 18:48:26UTC : Starting to build base images for CUDA and CPU. |
GiGL Automation@ 18:58:18UTC : Built and pushed new images:
Updated |
zfan3-sc
left a comment
There was a problem hiding this comment.
approve to unblock. lgtm once Kyle's comment is addressed
GiGL Automation@ 05:04:05UTC : Starting to build base images for CUDA and CPU. |
GiGL Automation@ 05:12:21UTC : Built and pushed new images:
Updated |
Summary
Since #786 the partitioners hand
DistDatasetan int32 edge index whenever node ids fit, but the compact CSR build (#768) still widened the column ids to int64. This PR keeps them int32, which GiGL's patched GLT samplers (#761) read directly. Peak memory for the CSR conversion drops from ~2x one int64 edge array to ~1.5x, and the stored column ids are half the size. int64 inputs are unchanged.Approach
build_csr_from_cooallocatesindiceswith the width ofcol;indptrstays int64.User-facing changes
Topology.indiceson the compact path is int32 for int32 edge indices. Graphs with edge ids, weights or features still go through GLT'sinit_graphand stay int64.build-base-docker-imagesworkflow.Verification
csr_test,compact_topology_test(adds int32 CSR/CSC sampling parity against GLT's ownTopology),glt_int32_indices_test: 64 tests pass.distributed_neighborloader_test(36, one pre-existing skip),distributed_dataset_test(15),distributed_partitioner_test(34): pass, with real spawned loaders.config_populatortests that need CI bucket access the dev machine lacks.