Repository navigation
fix(🎥): wait on the decoder's acquire fence before sampling Android video frames - #524
Merged
Merged
Conversation
…ideo frames ImageReader hands a buffer out as soon as the producer queues it and only waits on the producer's fence for CPU access (Image.getPlanes), not for Image.getHardwareBuffer. The GPU could therefore sample a frame the video decoder (or the camera) was still writing. Since API 33 the fence is exposed as Image.getFence(). SyncFences reads its file descriptor (SyncFence has no accessor; a Parcel round-trip yields a duplicate), the frame handle carries it, and GPUExternalTexture::Create imports it as a shared-fence-sync-fd fence that BeginAccess waits on, on the GPU. A device without that feature waits on the CPU instead; before API 33 nothing changes. The fallback compileSdkVersion of the library goes to 34 so the SyncFence API compiles when the app does not set one (every current React Native app does).
This was referenced Oct 14, 2026
Merged
Owner
Author
|
Device round on a Pixel 8 (Android 17, SDK 37), stacked build of #524 + #526 + #527 + #528:
|
|
🎉 This PR is included in version 0.15.0 🎉 The release is available on:
Your semantic-release bot 📦🚀 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
ImageReaderhands a buffer out as soon as the producer queues it, and only waits on the producer's fence for CPU access (Image.getPlanes()), not forImage.getHardwareBuffer().WebGPUVideoPlayer.copyLatestFrame()passed that buffer straight to Dawn, so the GPU could sample a frame the decoder (or, through the same path, a camera) was still writing. Dawn's own Android layer (tools/android/webgpu,GPUHardwareBufferWrapper.beginAccess(fences)) feeds exactly this fence intoBeginAccess.What
SyncFences.java: reads the acquire fence of anImage(Image.getFence(), API 33) as a sync file descriptor.SyncFencehas no accessor for it; writing the fence to aParceland reading it back yields a duplicate, which is detached and handed to native.VideoFrameHandle.acquireFenceFd: the frame carries the descriptor and its deleter closes it. Dawn duplicates the descriptor on import, so a frame can be imported more than once.GPUExternalTexture::Create(Android): imports the descriptor as ashared-fence-sync-fdGPUSharedFenceand passes it toBeginAccess, so the wait happens on the GPU, in front of the work that samples the texture. A device without the feature (imported from another library that did not request it) waits on the CPU with a boundedpoll()instead.copyExternalImageToTexturegoes through the same function, so it is covered too.-1, no fence).Wgpu_compileSdkVersiongoes from 31 to 34 so theSyncFenceAPI compiles when the app does not setcompileSdkVersionitself (every current React Native app does).Frames handed to JS as
NativeVideoFrame.handleand imported withimportSharedTextureMemory+beginAccessdo not get the fence yet: the JS descriptor has no way to carry it. That can be a follow-up (beginAccesscould accept the frame).Verification
:react-native-webgpu:assembleDebugfor arm64-v8a: Java and CMake both compile.writeTestVideoFileis Apple-only), so the fence path is exercised by the VisionCamera and video examples rather than by the suite.