Skip to content

feat: render video effects the app writes natively, release 2.26.0 - #232

Merged
hm21 merged 2 commits into
stablefrom
feat/custom-video-effects
Oct 5, 2026
Merged

hm21 merged 2 commits into
stablefrom
feat/custom-video-effects

Conversation

@hm21

@hm21 hm21 commented Oct 5, 2026 •

Copy link
Copy Markdown
Owner

Description

An app that wants an effect the plugin does not ship, such as an echo trail where moving subjects leave fading copies of where they were, had no way to put it into the export: the built-in VideoRenderData.effects are a fixed list, and the export runs natively frame by frame, where no Dart code can take part.

VideoRenderData.customEffects closes that gap. The app writes the effect in native code, registers it under an id with CustomVideoEffects.register, and names it in a render with CustomVideoEffect(id:, params:, startTime:, endTime:).

  • Native, on the export's own GPU path. On Android an effect is a CustomVideoEffectRenderer that draws with OpenGL ES; CustomVideoEffectShader turns a fragment shader into one without boilerplate. On iOS and macOS it returns a Core Image image from the compositor. The same renderer can also drive a live preview in the app's own player, so preview and export share one implementation.
  • Earlier frames. An effect lists how far back it needs to see in historyOffsetsUs, counted on the rendered video so a speed change keeps its look, and receives those frames of the same clip with every frame. That makes the result independent of how the clip was played or seeked, which an effect that kept its own state between calls could not guarantee. Android keeps the frames as GPU copies at historyScale; iOS and macOS add delayed copies of the clip to the composition, so the compositor gets them even when it renders frames out of order or concurrently, and resample them to historyScale so both platforms look alike.
  • Placement. Custom effects run right after the frame is decoded and oriented, before the built-in effects, color filters and blur, so they see the clip as recorded.
  • Failing loudly. A render that names an id nothing is registered under fails instead of exporting without the effect.
  • Scope. On iOS and macOS a render with a VideoRenderData.composition skips custom effects for now and logs that it did. Web, Windows, Linux and VideoEffectPreview ignore them.

The example app registers two effects, example.invert and example.delay, which double as the reference for writing one (example/android/.../ExampleVideoEffects.kt, example/ios/Runner/AppDelegate.swift, example/macos/Runner/MainFlutterWindow.swift), and the README gains a section on it.

Related Issue: none

Verification

  • Verified on Galaxy S26 and iPhone 12 Pro through an app that exports an echo trail effect written this way: the trail follows moving subjects as in the app's preview.
  • The new example/integration_test/custom_video_effect_test.dart (6) passes on macOS, on an iOS simulator (iPhone 17 Pro Max) and on an Android emulator (Pixel 6): the effect draws the frame, receives its params, stays inside its time range, gets the frame from exactly as far back as it asks, counted on the rendered video at 2x speed, and a render naming an unregistered id fails.
  • macOS: all 127 XCTests in RunnerTests pass, including the new CustomVideoEffectTests (7) for history track placement and argument parsing.
  • Android: all 306 JVM tests pass, including the new CustomVideoEffectHistoryTest (7) and CustomVideoEffectConfigTest (5).
  • flutter test (520) and flutter analyze pass, with the new custom_video_effect_model_test.dart and the custom effect cases of video_render_data_model_test.dart.

Version bumped to 2.26.0 with a changelog entry.

Type of Change

  • ✨ New feature (non-breaking change which adds functionality)
  • 🛠️ Bug fix (non-breaking change which fixes an issue)
  • ❌ Breaking change (fix or feature that would cause existing functionality to change)
  • 🧹 Code refactor
  • ✅ Build configuration change
  • 📝 Documentation
  • 🗑️ Chore

An app that wants an effect the plugin does not ship, such as an echo
trail, had no way to put it into the export. `VideoRenderData.customEffects`
names effects the app registers natively under an id: a GLSL shader on
Android, Core Image on iOS and macOS. An effect can ask for earlier frames
of the same clip, so a trail exports the same however the clip was played.

The example app registers two effects, which the new integration test
renders on Android, iOS and macOS.
@hm21 hm21 self-assigned this Oct 5, 2026
…ered custom effect id before the pre-render stages
@hm21
hm21 marked this pull request as ready for review October 5, 2026 14:38
@hm21
hm21 merged commit 33d4a76 into stable Oct 5, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant