Skip to content

Configure CameraFrame effects when the camera frame is updated - #9686

Merged
mvaligursky merged 2 commits into
mainfrom
mv-camera-frame-effect-update
Oct 7, 2026
Merged

mvaligursky merged 2 commits into
mainfrom
mv-camera-frame-effect-update

Conversation

@mvaligursky

Copy link
Copy Markdown
Contributor

CameraFrame effects now follow the camera frame's original contract: change the settings, call cameraFrame.update(), and every frame rendered after it uses exactly that setup. Effect configuration runs only when the camera frame is updated rather than every frame, and the composition rebuilds its shader only when the applied effects or their defines change.

Changes:

  • An effect's update() is called by CameraFrame#update() for the active effects, and when the camera frame is enabled or an effect is added or removed. It applies the effect's parameters: its uniform values and the defines that depend on them.
  • frameUpdate(frame) runs every frame for the effects taking part, for values that change from frame to frame only. frame is a CameraFrameEffectContext holding this frame's scene texture, which alternates between two textures when TAA is enabled.
  • New CameraFrameEffect#setUniform(name, value). Values are stored by reference, like material parameters, and are bound when the effect's camera renders, so cameras whose effects share uniform names each render with their own values.
  • The built-in effects use the new hooks, and the composition no longer checks the effects' state every frame.

API Changes:

  • CameraFrameEffect#update() is called from CameraFrame#update() instead of every frame, and sets uniform values with setUniform() instead of writing them to the device scope:
    // before: called every frame
    update() {
        this.tintColorId.setValue([this.color.r, this.color.g, this.color.b]);
    }
    
    // after: called by cameraFrame.update()
    update() {
        this.tintColor[0] = this.color.r;
        this.tintColor[1] = this.color.g;
        this.tintColor[2] = this.color.b;
        this.setUniform('tintColor', this.tintColor);
    }
  • New: CameraFrameEffect#setUniform(), and the CameraFrameEffectContext passed to CameraFrameEffect#frameUpdate(frame).
  • Changes to the parameters of built-in and custom effects take effect on cameraFrame.update(), as all CameraFrame settings do. The per-frame application added in Introduce CameraFrameEffect and convert the vignette and grading effects #9666 and Convert sharpening, fringing, color enhance and color LUT to effects #9679 has not been in a release.

Examples:

  • camera-frame/custom-compose-shader sets its uniforms with setUniform() and calls cameraFrame.update() when its controls change.

Performance:

  • Effect configuration and the composition's change detection run only when cameraFrame.update() is called. Per frame, only the stored uniform values are bound.

…re their uniforms

CameraFrame effects read their state, defines and uniform values every
frame, unlike the rest of the camera frame, where settings apply when
CameraFrame#update is called. Effects now follow that contract: update() is
called from CameraFrame#update for the active effects and applies their
parameters, frameUpdate(frame) only receives the values of the frame, and
uniform values are stored with setUniform() and bound when the effect's
camera renders, so each camera renders with its own values.
@github-actions

github-actions Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

Public API report

This PR changes the public API surface (+18 / −7), per the docs' rules (@ignore / @Private / undocumented are excluded).

Show API diff
-CameraFrameEffect.frameUpdate(): void
+CameraFrameEffect.frameUpdate(frame: CameraFrameEffectContext): void
+CameraFrameEffect.setUniform(name: string, value: number | number[] | Texture | ArrayBufferView<ArrayBufferLike>): void
+CameraFrameEffectContext.sceneHeight: number
+CameraFrameEffectContext.sceneTexture: Texture
+CameraFrameEffectContext.sceneWidth: number
-CasEffect.frameUpdate(): void
+CasEffect.frameUpdate(frame: CameraFrameEffectContext): void
+CasEffect.setUniform(name: string, value: number | number[] | Texture | ArrayBufferView<ArrayBufferLike>): void
-ColorEnhanceEffect.frameUpdate(): void
+ColorEnhanceEffect.frameUpdate(frame: CameraFrameEffectContext): void
+ColorEnhanceEffect.setUniform(name: string, value: number | number[] | Texture | ArrayBufferView<ArrayBufferLike>): void
-ColorLutEffect.frameUpdate(): void
+ColorLutEffect.frameUpdate(frame: CameraFrameEffectContext): void
+ColorLutEffect.setUniform(name: string, value: number | number[] | Texture | ArrayBufferView<ArrayBufferLike>): void
-FringingEffect.frameUpdate(): void
+FringingEffect.frameUpdate(frame: CameraFrameEffectContext): void
+FringingEffect.setUniform(name: string, value: number | number[] | Texture | ArrayBufferView<ArrayBufferLike>): void
-GradingEffect.frameUpdate(): void
+GradingEffect.frameUpdate(frame: CameraFrameEffectContext): void
+GradingEffect.setUniform(name: string, value: number | number[] | Texture | ArrayBufferView<ArrayBufferLike>): void
-VignetteEffect.frameUpdate(): void
+VignetteEffect.frameUpdate(frame: CameraFrameEffectContext): void
+VignetteEffect.setUniform(name: string, value: number | number[] | Texture | ArrayBufferView<ArrayBufferLike>): void
+interface CameraFrameEffectContext

Informational only — this never fails the build.

@github-actions

github-actions Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

Build size report

This PR changes the size of the minified bundles.

Bundle Minified Gzip Brotli
playcanvas.min.js 2508.1 KB (−144 B, −0.01%) 648.2 KB (+53 B, +0.01%) 503.0 KB (−32 B, −0.01%)
playcanvas.min.mjs 2504.9 KB (−144 B, −0.01%) 646.8 KB (+54 B, +0.01%) 502.9 KB (+183 B, +0.04%)

@mvaligursky mvaligursky left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated PR review by Codex.

Reviewed head 67eeff8d5c7c7389ea388916773258a76cd845f0 against base ca46e3935ae842a27f62094884fcc803c0d716db, including the applied-effect snapshot, shader invalidation, built-in uniform migration, two-camera binding, TAA, lifecycle and resize behavior.

One P2 finding is attached inline: the new per-frame context is delivered before its scene texture has this frame's dimensions. A size-dependent effect migrated from the old draw-time update() to frameUpdate(frame) therefore binds incorrect resolution uniforms during startup and resizing (two frames with TAA).

Validation:

  • Unit suite: 3,968 passing, 14 pending.
  • Declaration build/type tests and debug/release ESM builds passed; changed engine/test files pass ESLint and the diff passes whitespace checks. CI, including WebGPU tests and the examples build, is green.
  • Deterministic rendering: 20 configurations × two poses × two cameras on each backend; repeated baselines were exact, and the PR produced 0 differing pixels out of 1,966,080 checked. This covers the unchanged built-in effect math, LUT transitions, HDR/LDR, target scaling and TAA.
  • Separate real-WebGL/WebGPU probes verified pending settings remain unapplied until update, configuration is not polled per frame, same-named uniforms remain camera-local, current TAA texture identity is correct, and removed effects stop receiving callbacks. The size-dependent probe is correct on the base and reproduces the inline issue on the PR.

The intended return to explicit cameraFrame.update() is not being flagged as a released-API compatibility issue.

if (effect.active) {
effect.frameUpdate();
}
effects[i].frameUpdate(context);

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated PR review by Codex.

[P2] Provide current texture dimensions before invoking the per-frame hook

The new context is handed to effects before FrameGraph.addRenderPass() updates the scene/TAA child passes, which is when their render targets resize. Consequently an effect following the new API and computing setUniform('invSceneSize', [1 / frame.sceneTexture.width, 1 / frame.sceneTexture.height]) in frameUpdate(frame) binds stale resolution values for the draw. I reproduced this on both WebGL and WebGPU: a 128×96 camera reports 4×4 on startup; after resizing 128×96 → 160×112, the callback reports 128×96 although the composition samples 160×112. With TAA both alternating history textures lag, so the wrong dimensions persist for two rendered frames. Changing renderTargetScale has the same problem. The equivalent effect's old draw-time update() observes the correct size on the base. This causes incorrect texel offsets/kernel or pixelation sizes during startup and resize, precisely when these values now need the per-frame hook. Please ensure the source/history textures have their current-frame dimensions before invoking it, or expose the computed current dimensions in the context and document that texture sizing is deferred; cover first-frame/resize behavior with and without TAA.

The scene texture in the per-frame context is resized later in the frame,
once the frame's passes update, so on the first frame and when the canvas or
the render target scale changes it still had the previous size. The context
now carries this frame's scene width and height, and the composition provides
sceneTextureSize - width, height and their inverses - to every chunk,
replacing sceneTextureInvRes.

@mvaligursky mvaligursky left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated PR re-review by Codex (OpenAI AI agent).

Reviewed the follow-up at ebc5a2440be381655cc1a7911647bd3c4dd3fc6d and its interaction with the effect update contract, texture resizing, shader uniforms and legacy compose overrides.

The earlier P2 stale-dimension finding is addressed: effects now have explicitly documented current-frame dimensions, while the composition supplies the final dimensions at draw time. One new P2 is attached inline: removing the previously released sceneTextureInvRes uniform breaks existing custom compose chunks, including the pattern shipped in the v2.23.0 example, on both backends.

Validation:

  • 3,970 unit tests passing, 14 pending; declaration build/consumer type tests, debug/release ESM builds, updated engine/test lint and whitespace checks passed. Current CI is green.
  • Real WebGL/WebGPU size-dependent probes: 40 observations per backend, with zero stale context dimensions. Startup, TAA alternation, immediate target resize, fractional scaling, different per-camera scales, re-enabling and removal were exercised. The new shared shader size and camera-local uniforms matched throughout.
  • Built-in rendering remained byte-exact versus the prior reviewed commit across 20 configurations, two poses and two cameras on each backend: 0 of 1,966,080 pixels differed; repeated baselines were also exact.
  • A legacy composeMainEndPS chunk using sceneTextureInvRes rendered correctly before this follow-up, but failed shader compilation/pipeline creation and produced blank output on the updated commit, on both WebGL and WebGPU.

uniform sampler2D sceneTexture;
uniform vec2 sceneTextureInvRes;
// the size of the scene texture: width, height, 1 / width, 1 / height
uniform vec4 sceneTextureSize;

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated PR review by Codex (OpenAI AI agent).

[P2] Preserve sceneTextureInvRes for existing compose chunks

Replacing the old declaration and stopping its binding breaks the existing compose customization interface. sceneTextureInvRes already shipped in released versions: the v2.23.0 custom-compose example passes it to its helper from composeMainEndPS (and uses uniform.sceneTextureInvRes in WGSL). Those unchanged chunks now fail to compile because the identifier is no longer supplied. I reproduced this on both backends with a minimal legacy end chunk: it renders correctly at 67eeff8d5, but this commit reports an undeclared identifier on WebGL and an invalid shader/pipeline on WebGPU, leaving the camera output blank. Unlike the unreleased effect-hook behavior, this affects code copied from a released example. Please retain a compatible alias/declaration and current value for the old name alongside sceneTextureSize, on both shader languages, and add a legacy-chunk compatibility test.

@mvaligursky
mvaligursky merged commit fda2664 into main Oct 7, 2026
11 checks passed
@mvaligursky
mvaligursky deleted the mv-camera-frame-effect-update branch October 7, 2026 08:48

This branch was successfully deployed

2 active deployments
Preview – engine — ebc5a244 Deployed Oct 7, 2026 by vercel[bot]
Preview – engine-api-docs — ebc5a244 Deployed Oct 7, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant