Repository navigation
Conversation
Up to standards ✅🟢 Issues
|
| Metric | Results |
|---|---|
| Complexity | 227 |
| Duplication | 24 |
NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.
| error("CudaGibbsPenalty: set_up has not been called"); | ||
|
|
||
| d_scalar[0] = 0.0; | ||
| array_to_device(d_image_data, current_image_estimate); |
There was a problem hiding this comment.
are you sure we can delete this? What happens if it's not on the device?
There was a problem hiding this comment.
Good point. I checked this part, and I think the original reason for removing that transfer was related to the changes to DiscretisedDensity constructors and how they allocate memory. The idea was that the memory would now be accessible by both host and device. However, if a DiscretisedDensity object is constructed via a different route that doesn't use CuAlloc (or cudaMallocManaged) this is no longer valid.
There was a problem hiding this comment.
Possibly, array_to_device should just copy the pointer (not the data) or so when it's already on the device. not sure...
|
BTW, please commit with |
KrisThielemans
left a comment
There was a problem hiding this comment.
great stuff. some more "moving things" around comments such that we'll have a great starting base for other structures.
BTW, we have in various places std::allocate_shared<elemT[]>(CuAlloc<elemT>(), range_v.size_all()) which ATM we need to guard with STIR_WITH_CUDA. @casperdcl what about something like this in cuda_utilities.h
#ifdef STIR_WITH_CUDA
using device_allocator = CuAlloc;
#else
using device_allocator = std::allocator;
#endifsee also the discussion in #1687 (comment)
| error("CudaGibbsPenalty: set_up has not been called"); | ||
|
|
||
| d_scalar[0] = 0.0; | ||
| array_to_device(d_image_data, current_image_estimate); |
There was a problem hiding this comment.
Possibly, array_to_device should just copy the pointer (not the data) or so when it's already on the device. not sure...
|
Also, looks like you don't run You'll have to do |
|
#1772 has been merged. Please merge |
|
BTW, I think the merge commit can be pushed with |
|
feel like having a go at merging |
|
After merging, I had an issue with I also added two helper functions in |
|
See #1687 (comment). Let's keep the discussion on |
KrisThielemans
left a comment
There was a problem hiding this comment.
This is looking good except for some house-keeping. What's the current status?
|
|
||
| if (initialise_with_0) | ||
| { | ||
| std::fill(this->buffer.begin_all(), this->buffer.end_all(), 0.F); |
There was a problem hiding this comment.
ideally replace later with some CUDA-aware function
There was a problem hiding this comment.
I added a function, fill_with_zero, in cuda_utilities that follows the same idea as the copy function. If the memory is on the GPU or is CUDA managed memory it uses cudaMemset, otherwise it falls back to the original CPU implementation via std::fill.
| @@ -0,0 +1,19 @@ | |||
| #pragma once | |||
There was a problem hiding this comment.
amazingly this isn't portable so need to use our standard #ifdef ... #define ...
|
A quick update. Since the hackathon, I haven't made much progress beside some cleanup here and there. My plan is to address the housekeeping comments and to take a look at the parts we said that should be more CUDA-aware. I'm hoping to be able to work on it next week. |
|
Hi Valerio, |
|
debugging a bit with @MatteoNeelColombo 1 (
Footnotes |
|
Here are the timings on WSL2 (RTX 4090, Ryzen 9 7900, CUDA 12.9), same command and NeuroLF Esser dataset. I counted the
i would say that the timings in stir timings are not really designed for this PR, I think. #1629 copies the image to the GPU (and the result back) at every call, and master does the same with So on WSL my best bet is that the GPU reads the managed memory over PCIe instead of moving it to the GPU, but I can't check that on WSL. On native Linux the pages stay on the GPU, which is why you see this PR much faster. To properly understand what's going on under the hood i think we would need to profile them, e.g. with |
|
Using https://gist.github.com/MatteoNeelColombo/f069e64fe8ac67b4ed4d9b2e67a1641d:
P.S.: might be worth testing in CI (#1764) |
|
@casperdcl Yes please, go ahead with rebase. Thanks! I will try to go through the comments from you and @MatteoNeelColombo this weekend or beginning of next week. |
|
if we all agree to |
|
@casperdcl About the prefetch: on WSL/Windows cudaMemPrefetchAsync to the GPU returns cudaErrorInvalidDevice (no concurrentManagedAccess), I just checked. The Gibbs kernels call checkCudaError() right after the launch, so that error would show up as a kernel failure. Could you only prefetch when cudaDevAttrConcurrentManagedAccess is set? |
|
following up on the WSL point, with a correction to my earlier comment: I wrote that on WSL/Windows managed memory is migrated to the GPU at every kernel launch (that's what the CUDA programming guide says), but my tests show it isn't. I ran some tests on WSL2 comparing plain GPU memory and managed memory. What I see is that on WSL managed memory is never moved to the GPU: it stays in host RAM, So managed memory should work well on native Linux, but on Windows/WSL it makes the GPU unusable for reconstruction. In my opinion i see 2 possible solutions: 1.If the long-term plan is to move everything to managed memory, we might need a separate treatment for WSL/Windows; otherwise, on those systems it would be much faster not to use the GPU at all. It would also be interesting to know how many users run STIR on Windows (native or WSL), to see how much that separate treatment is worth. |
775795d to
0926665
Compare
Up to standards ✅🟢 Issues
|
| Metric | Results |
|---|---|
| Complexity | 88 |
| Duplication | 6 |
NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.
|
Unsure I follow @MatteoNeelColombo's comments. Personally when running CuVec:tests/test_perf.py I don't see any problems on WSL2. Anyway, just rebased over #1792 (won't rebase again unless really needed), added a explicit prefetch helper, and here are the timings:
|
|
After more digging:
So if we're migrating to
|
…use CUDA unified memory - Modified memory allocation in DiscretisedDensity and ProjDataInMemory to use cudaMallocManaged via the custom allocator CuAlloc accessed through CuVec library. - Updated ProjDataInMemory to manage CUDA pointers and use them efficiently. - Added copy constructor in DiscretisedDensity and forced managed memory allocation in VoxelsOnCartesianGrid.
…jection and image data relevant classes. - Added ProjDataInMemoryCUDA (.cu/.h) implementing GPU-accelerated element-wise operations (+=, -=, *=, /=) and linear combinations (xapyb) - Removed manual transfers between host and device in CudaGibbsPenalty and CudaRelativeDifferencePrior in favor of managed pointers - Updated CMake build scripts to include new CUDA compilation targets - Added timing benchmarks for copy_only and xapyb_operator in stir_timings
- Moved onGPU helper functions to cuda_utilities.h with stir::Array overload
- Renamed ProjDataInMemoryCUDA files to algebraic_kernels (.cu / .h)
- Included original code fallback in ProjDataInMemory allocation
- Removed temporary comments ("Modified version")
- CMakeLists.txt: Updated CUDA target dependencies. - algebraic_kernels.cu: Updated CUDA kernel implementations for compatibility with managed pointers. - ProjDataInMemory.cxx: Fixed memory allocations and pointer operations for CUDA managed buffers. - VoxelsOnCartesianGrid.cxx: Reintroduced original CPU allocation and implemented STIR_WITH_CUDA for the GPU version. - algebraic_kernels.h: Updated kernel declarations to be compatible with the new version of cuda_utilities. - cuda_utilities.h: Updated CUDA memory allocation and utility wrappers.
- cuda_utilities.h: modified the include section so that CUVEC_DISABLE_CUDA is only defined when STIR has no CUDA. Added helper functions device_readable_ptr and device_writable_ptr. They pass the array's own pointer to the kernel if it is on the GPU, otherwise they fall back to array_to_device/array_to_host with a temporary CuVec. - CudaGibbsPenalty, CudaRelativeDifferencePrior: used these helpers in the compute_* functions. Previously the kernels received raw host pointers for images not allocated with CuAlloc, causing 'illegal memory access'. This should fix this.
…ded the fill_with_zero function [ci skip]
0926665 to
df1cf5b
Compare
|
@casperdcl I tried df1cf5b on WSL2 and with the new concurrentManagedAccess check the Cuda Gibbs prior gradient and Hessian×input come out as all zeros. I think the rebase over #1792 dropped the lines in CudaGibbsPenalty.cuh that copy the result from tmp back to the image (if (!onGPU(x)) array_to_host(x, tmp_x). |
df1cf5b to
7a58506
Compare
Assisted-by: Qwen3.8-27B <noreply@pi.dev>
7a58506 to
5dddfc1
Compare
|
Same command as before, ms per call,(no touch /
I think it's because on WSL2 the image is now in host memory, but the temporary copy made before each kernel is a CuVec, so it's managed memory again, and the kernel still reads it from host RAM |
Changes in this pull request
This PR introduces an approach for using CUDA-Managed pointers in selected STIR components by modifying the underlying memory allocation mechanism. As a preliminary step, the work focused on image and projection data, with particular attention to the DiscretisedDensity and ProjDataInMemory classes and how memory allocation is handled within them.
The main changes are:
Implementations:
ProjDataInMemory.cxx: Updated buffer allocation to use CUDA unified memory via CuVec's custom allocator (CuAlloc). Added a workaround usingstd::swapinsidecreate_buffer()to ensure memory addresses remain on the GPU during buffer initialization. Optimized the copy constructor and arithmetic operators to utilizecudaMemcpyand GPU-accelerated kernels now that projection data resides in managed memory.VoxelsOnCartesianGrid.cxx: Added a temporary workaround to enforce CUDA unified memory allocation when instantiating grid data for projection operations.DiscretisedDensity.inl: Updated image data allocation to useCuAllocunified memory and implemented the corresponding copy constructor.CudaGibbsPenalty.cuh: Updated kernel to accept pointers directly from unified memory allocations. Removed redundant host-to-device and device-to-host memory transfers.algebraic_kernels.cu/h: Added definition and implementation of CUDA kernels and host wrappers for element-wise binary operations (+=, -=, *=, /=) and linear combinations (dst=a⋅x+b⋅y, via xapyb) on unified memory projection arrays, adapted from CuVec operations.Headers:
DiscretisedDensity.h: Declared the CUDA-managed copy constructor added in theDiscretisedDensityclass interface.Build and testing
CMakeLists.txt: Addedalgebraic_kernels.cuto the target build sources for CUDA compilation.stir_timings.cxx: Added timing coverage forcopy_onlyandxapyb_operatorbenchmark groups.Testing performed
The changes were tested using STIR utility
stir_timings.Related issues
Checklist before requesting a review
Contribution Notes
Please tick the following: