Repository navigation
feat: render video effects the app writes natively, release 2.26.0 - #232
Merged
Merged
Conversation
An app that wants an effect the plugin does not ship, such as an echo trail, had no way to put it into the export. `VideoRenderData.customEffects` names effects the app registers natively under an id: a GLSL shader on Android, Core Image on iOS and macOS. An effect can ask for earlier frames of the same clip, so a trail exports the same however the clip was played. The example app registers two effects, which the new integration test renders on Android, iOS and macOS.
…ered custom effect id before the pre-render stages
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
An app that wants an effect the plugin does not ship, such as an echo trail where moving subjects leave fading copies of where they were, had no way to put it into the export: the built-in
VideoRenderData.effectsare a fixed list, and the export runs natively frame by frame, where no Dart code can take part.VideoRenderData.customEffectscloses that gap. The app writes the effect in native code, registers it under an id withCustomVideoEffects.register, and names it in a render withCustomVideoEffect(id:, params:, startTime:, endTime:).CustomVideoEffectRendererthat draws with OpenGL ES;CustomVideoEffectShaderturns a fragment shader into one without boilerplate. On iOS and macOS it returns a Core Image image from the compositor. The same renderer can also drive a live preview in the app's own player, so preview and export share one implementation.historyOffsetsUs, counted on the rendered video so a speed change keeps its look, and receives those frames of the same clip with every frame. That makes the result independent of how the clip was played or seeked, which an effect that kept its own state between calls could not guarantee. Android keeps the frames as GPU copies athistoryScale; iOS and macOS add delayed copies of the clip to the composition, so the compositor gets them even when it renders frames out of order or concurrently, and resample them tohistoryScaleso both platforms look alike.VideoRenderData.compositionskips custom effects for now and logs that it did. Web, Windows, Linux andVideoEffectPreviewignore them.The example app registers two effects,
example.invertandexample.delay, which double as the reference for writing one (example/android/.../ExampleVideoEffects.kt,example/ios/Runner/AppDelegate.swift,example/macos/Runner/MainFlutterWindow.swift), and the README gains a section on it.Related Issue: none
Verification
example/integration_test/custom_video_effect_test.dart(6) passes on macOS, on an iOS simulator (iPhone 17 Pro Max) and on an Android emulator (Pixel 6): the effect draws the frame, receives its params, stays inside its time range, gets the frame from exactly as far back as it asks, counted on the rendered video at 2x speed, and a render naming an unregistered id fails.RunnerTestspass, including the newCustomVideoEffectTests(7) for history track placement and argument parsing.CustomVideoEffectHistoryTest(7) andCustomVideoEffectConfigTest(5).flutter test(520) andflutter analyzepass, with the newcustom_video_effect_model_test.dartand the custom effect cases ofvideo_render_data_model_test.dart.Version bumped to 2.26.0 with a changelog entry.
Type of Change