Repository navigation
Configure CameraFrame effects when the camera frame is updated - #9686
Conversation
…re their uniforms CameraFrame effects read their state, defines and uniform values every frame, unlike the rest of the camera frame, where settings apply when CameraFrame#update is called. Effects now follow that contract: update() is called from CameraFrame#update for the active effects and applies their parameters, frameUpdate(frame) only receives the values of the frame, and uniform values are stored with setUniform() and bound when the effect's camera renders, so each camera renders with its own values.
Public API reportThis PR changes the public API surface (+18 / −7), per the docs' rules (@ignore / @Private / undocumented are excluded). Show API diff-CameraFrameEffect.frameUpdate(): void
+CameraFrameEffect.frameUpdate(frame: CameraFrameEffectContext): void
+CameraFrameEffect.setUniform(name: string, value: number | number[] | Texture | ArrayBufferView<ArrayBufferLike>): void
+CameraFrameEffectContext.sceneHeight: number
+CameraFrameEffectContext.sceneTexture: Texture
+CameraFrameEffectContext.sceneWidth: number
-CasEffect.frameUpdate(): void
+CasEffect.frameUpdate(frame: CameraFrameEffectContext): void
+CasEffect.setUniform(name: string, value: number | number[] | Texture | ArrayBufferView<ArrayBufferLike>): void
-ColorEnhanceEffect.frameUpdate(): void
+ColorEnhanceEffect.frameUpdate(frame: CameraFrameEffectContext): void
+ColorEnhanceEffect.setUniform(name: string, value: number | number[] | Texture | ArrayBufferView<ArrayBufferLike>): void
-ColorLutEffect.frameUpdate(): void
+ColorLutEffect.frameUpdate(frame: CameraFrameEffectContext): void
+ColorLutEffect.setUniform(name: string, value: number | number[] | Texture | ArrayBufferView<ArrayBufferLike>): void
-FringingEffect.frameUpdate(): void
+FringingEffect.frameUpdate(frame: CameraFrameEffectContext): void
+FringingEffect.setUniform(name: string, value: number | number[] | Texture | ArrayBufferView<ArrayBufferLike>): void
-GradingEffect.frameUpdate(): void
+GradingEffect.frameUpdate(frame: CameraFrameEffectContext): void
+GradingEffect.setUniform(name: string, value: number | number[] | Texture | ArrayBufferView<ArrayBufferLike>): void
-VignetteEffect.frameUpdate(): void
+VignetteEffect.frameUpdate(frame: CameraFrameEffectContext): void
+VignetteEffect.setUniform(name: string, value: number | number[] | Texture | ArrayBufferView<ArrayBufferLike>): void
+interface CameraFrameEffectContextInformational only — this never fails the build. |
Build size reportThis PR changes the size of the minified bundles.
|
mvaligursky
left a comment
There was a problem hiding this comment.
Automated PR review by Codex.
Reviewed head 67eeff8d5c7c7389ea388916773258a76cd845f0 against base ca46e3935ae842a27f62094884fcc803c0d716db, including the applied-effect snapshot, shader invalidation, built-in uniform migration, two-camera binding, TAA, lifecycle and resize behavior.
One P2 finding is attached inline: the new per-frame context is delivered before its scene texture has this frame's dimensions. A size-dependent effect migrated from the old draw-time update() to frameUpdate(frame) therefore binds incorrect resolution uniforms during startup and resizing (two frames with TAA).
Validation:
- Unit suite: 3,968 passing, 14 pending.
- Declaration build/type tests and debug/release ESM builds passed; changed engine/test files pass ESLint and the diff passes whitespace checks. CI, including WebGPU tests and the examples build, is green.
- Deterministic rendering: 20 configurations × two poses × two cameras on each backend; repeated baselines were exact, and the PR produced 0 differing pixels out of 1,966,080 checked. This covers the unchanged built-in effect math, LUT transitions, HDR/LDR, target scaling and TAA.
- Separate real-WebGL/WebGPU probes verified pending settings remain unapplied until update, configuration is not polled per frame, same-named uniforms remain camera-local, current TAA texture identity is correct, and removed effects stop receiving callbacks. The size-dependent probe is correct on the base and reproduces the inline issue on the PR.
The intended return to explicit cameraFrame.update() is not being flagged as a released-API compatibility issue.
| if (effect.active) { | ||
| effect.frameUpdate(); | ||
| } | ||
| effects[i].frameUpdate(context); |
There was a problem hiding this comment.
Automated PR review by Codex.
[P2] Provide current texture dimensions before invoking the per-frame hook
The new context is handed to effects before FrameGraph.addRenderPass() updates the scene/TAA child passes, which is when their render targets resize. Consequently an effect following the new API and computing setUniform('invSceneSize', [1 / frame.sceneTexture.width, 1 / frame.sceneTexture.height]) in frameUpdate(frame) binds stale resolution values for the draw. I reproduced this on both WebGL and WebGPU: a 128×96 camera reports 4×4 on startup; after resizing 128×96 → 160×112, the callback reports 128×96 although the composition samples 160×112. With TAA both alternating history textures lag, so the wrong dimensions persist for two rendered frames. Changing renderTargetScale has the same problem. The equivalent effect's old draw-time update() observes the correct size on the base. This causes incorrect texel offsets/kernel or pixelation sizes during startup and resize, precisely when these values now need the per-frame hook. Please ensure the source/history textures have their current-frame dimensions before invoking it, or expose the computed current dimensions in the context and document that texture sizing is deferred; cover first-frame/resize behavior with and without TAA.
The scene texture in the per-frame context is resized later in the frame, once the frame's passes update, so on the first frame and when the canvas or the render target scale changes it still had the previous size. The context now carries this frame's scene width and height, and the composition provides sceneTextureSize - width, height and their inverses - to every chunk, replacing sceneTextureInvRes.
mvaligursky
left a comment
There was a problem hiding this comment.
Automated PR re-review by Codex (OpenAI AI agent).
Reviewed the follow-up at ebc5a2440be381655cc1a7911647bd3c4dd3fc6d and its interaction with the effect update contract, texture resizing, shader uniforms and legacy compose overrides.
The earlier P2 stale-dimension finding is addressed: effects now have explicitly documented current-frame dimensions, while the composition supplies the final dimensions at draw time. One new P2 is attached inline: removing the previously released sceneTextureInvRes uniform breaks existing custom compose chunks, including the pattern shipped in the v2.23.0 example, on both backends.
Validation:
- 3,970 unit tests passing, 14 pending; declaration build/consumer type tests, debug/release ESM builds, updated engine/test lint and whitespace checks passed. Current CI is green.
- Real WebGL/WebGPU size-dependent probes: 40 observations per backend, with zero stale context dimensions. Startup, TAA alternation, immediate target resize, fractional scaling, different per-camera scales, re-enabling and removal were exercised. The new shared shader size and camera-local uniforms matched throughout.
- Built-in rendering remained byte-exact versus the prior reviewed commit across 20 configurations, two poses and two cameras on each backend: 0 of 1,966,080 pixels differed; repeated baselines were also exact.
- A legacy
composeMainEndPSchunk usingsceneTextureInvResrendered correctly before this follow-up, but failed shader compilation/pipeline creation and produced blank output on the updated commit, on both WebGL and WebGPU.
| uniform sampler2D sceneTexture; | ||
| uniform vec2 sceneTextureInvRes; | ||
| // the size of the scene texture: width, height, 1 / width, 1 / height | ||
| uniform vec4 sceneTextureSize; |
There was a problem hiding this comment.
Automated PR review by Codex (OpenAI AI agent).
[P2] Preserve sceneTextureInvRes for existing compose chunks
Replacing the old declaration and stopping its binding breaks the existing compose customization interface. sceneTextureInvRes already shipped in released versions: the v2.23.0 custom-compose example passes it to its helper from composeMainEndPS (and uses uniform.sceneTextureInvRes in WGSL). Those unchanged chunks now fail to compile because the identifier is no longer supplied. I reproduced this on both backends with a minimal legacy end chunk: it renders correctly at 67eeff8d5, but this commit reports an undeclared identifier on WebGL and an invalid shader/pipeline on WebGPU, leaving the camera output blank. Unlike the unreleased effect-hook behavior, this affects code copied from a released example. Please retain a compatible alias/declaration and current value for the old name alongside sceneTextureSize, on both shader languages, and add a legacy-chunk compatibility test.
CameraFrame effects now follow the camera frame's original contract: change the settings, call
cameraFrame.update(), and every frame rendered after it uses exactly that setup. Effect configuration runs only when the camera frame is updated rather than every frame, and the composition rebuilds its shader only when the applied effects or their defines change.Changes:
update()is called byCameraFrame#update()for the active effects, and when the camera frame is enabled or an effect is added or removed. It applies the effect's parameters: its uniform values and the defines that depend on them.frameUpdate(frame)runs every frame for the effects taking part, for values that change from frame to frame only.frameis aCameraFrameEffectContextholding this frame's scene texture, which alternates between two textures when TAA is enabled.CameraFrameEffect#setUniform(name, value). Values are stored by reference, like material parameters, and are bound when the effect's camera renders, so cameras whose effects share uniform names each render with their own values.API Changes:
CameraFrameEffect#update()is called fromCameraFrame#update()instead of every frame, and sets uniform values withsetUniform()instead of writing them to the device scope:CameraFrameEffect#setUniform(), and theCameraFrameEffectContextpassed toCameraFrameEffect#frameUpdate(frame).cameraFrame.update(), as all CameraFrame settings do. The per-frame application added in Introduce CameraFrameEffect and convert the vignette and grading effects #9666 and Convert sharpening, fringing, color enhance and color LUT to effects #9679 has not been in a release.Examples:
camera-frame/custom-compose-shadersets its uniforms withsetUniform()and callscameraFrame.update()when its controls change.Performance:
cameraFrame.update()is called. Per frame, only the stored uniform values are bound.