Skip to content

fix(🎥): wait on the decoder's acquire fence before sampling Android video frames - #524

Merged
wcandillon merged 2 commits into
mainfrom
fix/android-video-acquire-fence
Oct 15, 2026
Merged

wcandillon merged 2 commits into
mainfrom
fix/android-video-acquire-fence

Conversation

@wcandillon

Copy link
Copy Markdown
Owner

Why

ImageReader hands a buffer out as soon as the producer queues it, and only waits on the producer's fence for CPU access (Image.getPlanes()), not for Image.getHardwareBuffer(). WebGPUVideoPlayer.copyLatestFrame() passed that buffer straight to Dawn, so the GPU could sample a frame the decoder (or, through the same path, a camera) was still writing. Dawn's own Android layer (tools/android/webgpu, GPUHardwareBufferWrapper.beginAccess(fences)) feeds exactly this fence into BeginAccess.

What

  • SyncFences.java: reads the acquire fence of an Image (Image.getFence(), API 33) as a sync file descriptor. SyncFence has no accessor for it; writing the fence to a Parcel and reading it back yields a duplicate, which is detached and handed to native.
  • VideoFrameHandle.acquireFenceFd: the frame carries the descriptor and its deleter closes it. Dawn duplicates the descriptor on import, so a frame can be imported more than once.
  • GPUExternalTexture::Create (Android): imports the descriptor as a shared-fence-sync-fd GPUSharedFence and passes it to BeginAccess, so the wait happens on the GPU, in front of the work that samples the texture. A device without the feature (imported from another library that did not request it) waits on the CPU with a bounded poll() instead. copyExternalImageToTexture goes through the same function, so it is covered too.
  • Before API 33 nothing changes (-1, no fence).
  • The library's fallback Wgpu_compileSdkVersion goes from 31 to 34 so the SyncFence API compiles when the app does not set compileSdkVersion itself (every current React Native app does).

Frames handed to JS as NativeVideoFrame.handle and imported with importSharedTextureMemory + beginAccess do not get the fence yet: the JS descriptor has no way to carry it. That can be a follow-up (beginAccess could accept the frame).

Verification

  • :react-native-webgpu:assembleDebug for arm64-v8a: Java and CMake both compile.
  • Not run on a device (none attached here). The Android video player spec skips in CI (writeTestVideoFile is Apple-only), so the fence path is exercised by the VisionCamera and video examples rather than by the suite.

…ideo frames

ImageReader hands a buffer out as soon as the producer queues it and only
waits on the producer's fence for CPU access (Image.getPlanes), not for
Image.getHardwareBuffer. The GPU could therefore sample a frame the video
decoder (or the camera) was still writing.

Since API 33 the fence is exposed as Image.getFence(). SyncFences reads
its file descriptor (SyncFence has no accessor; a Parcel round-trip yields
a duplicate), the frame handle carries it, and GPUExternalTexture::Create
imports it as a shared-fence-sync-fd fence that BeginAccess waits on, on
the GPU. A device without that feature waits on the CPU instead; before
API 33 nothing changes.

The fallback compileSdkVersion of the library goes to 34 so the SyncFence
API compiles when the app does not set one (every current React Native
app does).
@wcandillon

Copy link
Copy Markdown
Owner Author

Device round on a Pixel 8 (Android 17, SDK 37), stacked build of #524 + #526 + #527 + #528:

  • Full e2e suite: 43 suites, 214 tests pass on the device (the video specs still skip, writeTestVideoFile is Apple-only).
  • The fence import path works: with temporary logging, a snapshot frame from HardwareRenderer (feat(📸): draw view snapshots into the texture on the GPU on Android 13+ #527) arrives with a live fence fd and ImportSharedFence accepts it as shared-fence-sync-fd (the device exposes the feature).
  • On this device MediaCodec frames carry no fence: Image.getFence().isValid() is false for every frame of an H.264 stream (Big Buck Bunny 1080p), so the decoder path gets -1 and BeginAccess runs without a wait, as before. The PR is a no-op there; it matters on devices whose decoder hands out unsignaled buffers. Playback is correct.

@wcandillon
wcandillon merged commit 61ab4fd into main Oct 15, 2026
3 of 4 checks passed
@github-actions

Copy link
Copy Markdown

🎉 This PR is included in version 0.15.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant