Repository navigation
Add SOG v3 loading and over-unity opacity rendering - #9873
Conversation
SOG v3 loading (v2 unchanged): - 24-bit exponential-map rotations read from the alpha channels of means_l, means_u and scales; quats.webp is no longer fetched for v3. - Opacity read through meta.opacity.codebook (via the lookup texture's alpha), allowing values above 1. - 4 SH bands (1536-pixel palette). - Optional blind.webp exposed as a resource stream, with getBlind() in work-buffer modifiers; the CPU iterator and decompress() emit opacity_excess and the blind column. - meta.json / lod-meta.json extras exposed as resource.extras. - Parsers report unknown SOG versions as an error. Opacity above 1 (decimated gaussians that carry more mass than a standard gaussian) is drawn with a flat-topped plateau profile whose mass matches the stored opacity, with a 0.4 plateau shrink. An R8 work-buffer stream (splatPlateau) is added only once a resource with over-unity opacity is in the world; the copy pass solves the plateau shape once per splat. Both the quad renderer (GLSL and WGSL) and the GPU-sort projector path grow the footprint and draw the profile. app.scene.gsplat.compensation = false forces opacity <= 1 for debugging. Also fixes GSplatWorkBuffer.syncWithFormat() not rebuilding its render targets when a stream is added after creation and getTexture() has already synced the streams (affects any stream added at runtime on the GPU-sort path). Not covered: shadow and pick passes draw plateau splats at the standard extent; the PLY path ignores opacity_excess. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PLY files written by decimation can carry an opacity_excess column holding the opacity above 1. GSplatResource now adds it to the sigmoid opacity in the half-float color texture and reports hasOverUnity, so the unified renderer enables the plateau stream as it does for SOG v3. The uncompressed format clamps opacity to 1 unless GSPLAT_OVERUNITY is defined, so compensation off and the non-unified renderer are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Attach the plateau stream before the frame's bake when a resource that needs it arrives, so the first bake writes it. - Cull by the plateau's grown footprint in the quad shaders, the GPU-sort projector and the shadow cull; the projector's contribution test also counts the plateau's mass. - Trim quads where the plateau profile falls to alphaClip, rather than scaling a plain gaussian's trim, so shadow, pick and prepass coverage matches the drawn shape. - Enable GSPLAT_OVERUNITY on the GPU-sort pick material and the shadow materials, which also refresh when streams are added to the format in place. - Keep meta.json extras when SOG data is decompressed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SOG v3 palettes may exceed 65536 entries: the label's third byte (shN_labels.b) holds bits 16-23. The GLSL and WGSL SH chunks and the CPU iterator decode it for v3 files; v2 and v1 still read 16-bit labels. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
A concern about where the plateau is stored. Decimated data will be the norm for large assets (most scenes over ~1M splats), and almost all of them will have some splats with opacity above 1. In our tests the merged opacity reaches about 20. So the plateau path will be on nearly all the time, and these become permanent costs:
The plateau fits in the existing storage instead, with no detection at all. 1. Work buffer: store the plateau in the alpha slotWhen the plateau is above 1, the copy pass clamps the stored opacity to exactly 1 ( Large format (alpha is a half): the value is written as is, with no loss. float getOpacity() {
// ... loads and caches cachedColor as now
return min(cachedColor.a, 1.0);
}
// call after getOpacity(), reads the same cached data
float getPlateau() {
return max(cachedColor.a, 1.0);
}Compact format (alpha is the top byte of
// writeSplat
float a = color.a;
uint alphaBits = a > 1.0 ?
191u + uint(clamp(log2(a) * 32.0 + 0.5, 1.0, 64.0)) :
uint(clamp(a, 0.0, 1.0) * 191.0 + 0.5);
// read
float getOpacity() {
uint code = cachedTransformA.a >> 24u;
return code > 191u ? 1.0 : float(code) / 191.0;
}
float getPlateau() {
uint code = cachedTransformA.a >> 24u;
return code > 191u ? exp2(float(code - 191u) / 32.0) : 1.0;
}Precision compared with the separate R8 (maximum error, true opacity 1–30):
Both are well under the error compact already has from its 8-bit log scales, up to ±8.6% in area. If low-opacity precision matters, a sqrt mapping over the 192 opacity codes would make faint splats more precise than they are today. Non-decimated data never produces a plateau above 1, so the decode can always be compiled in. The
One detail: in the large format, alpha lives in 2. Projection cache: stay at 8 wordsThe plateau can't share the cached alpha slot. That alpha includes the AA compensation and v1 = l1 * dir;
v2 = l2 * vec2f(dir.y, -dir.x);So v2 is always v1 rotated by 90° and scaled by l2 / l1. Store that ratio and the plateau in v2's word: // projector
projCache[base + 5u] = pack2x16float(vec2f(l2 / max(l1, 1e-6), plateau));
// hybrid VS
let ratioPlateau = unpack2x16float(projCache[base + 5u]);
let v2 = ratioPlateau.x * vec2f(v1.y, -v1.x);
let plateau = ratioPlateau.y;The cost is one multiply, and v2 stays exactly perpendicular to v1. The same change applies to XR (word 6) and pick (word 5). Color, alpha and the user varyings stay as they are, and the stride stays at 32 B.
|
…at runtime (#9876) GSplatWorkBuffer.syncWithFormat() compared the streams' format version before and after its own sync. GSplatStreams.getTexture() also syncs, so when a texture was fetched by name after a stream was added, the streams were already up to date and the render targets were never rebuilt. The new stream then had a texture but no attachment, and the copy pass never wrote it. Track the version the render targets were built for and rebuild whenever it is behind. Extracted from #9873. Co-authored-by: Martin Valigursky <mvaligursky@snapchat.com> Co-authored-by: Donovan Hutchence <slimbuck7@gmail.com>
The fix landed separately in playcanvas#9876 with its own test.
Resolve the over-unity plateau against the anti-aliasing filter rework: the early size cull now tests the grown footprint (minPixelSize / grow), and the contribution cull keeps the plateau mass on the cull-lowpass determinant. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Matches splat-transform's SOG v3 change: opacity is stored as a fixed byte (b <= 224 holds A = (b/224)^2, b > 224 holds (1, 1.6] linearly) instead of an opacity codebook, and over-unity gaussians are drawn with the plateau that carries exactly min(A, 1.6), without the 0.4 shrink. - SOG: decodeSogOpacity for the CPU iterator, decompress() and the codebook LUT; hasOverUnity reads meta.opacity.max. - Copy pass (GLSL + WGSL): the plateau shape comes from a closed-form fit of the inverse mass (max error 1.3e-3) instead of a per-splat bisection, stored in the R8 stream as (D - 1) / 0.8 rather than log2(D) / 6; every decode site updated. - PLY: opacity + opacity_excess is clamped to 1.6 at upload. Engine vs splat-transform on a deep LOD level under the same policy: 40.4 dB WebGPU, 40.3 dB WebGL2, 41.5 dB with compensation off. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Matches splat-transform: the SOG v3 opacity byte is now A = (b/204)^2 = (1.25 * b/255)^2 (exact 0 and 1, top 1.5625), replacing the two-segment encoding. The vertex format decodes it inline, so the codebook LUT's .a is unused again. - Copy pass (GLSL + WGSL): exact-mass plateau of min(A, 1.5625), closed-form fit refitted (max error 1.1e-3), stored in R8 as (D - 1) / 0.75; decode sites updated. - PLY upload clamps opacity + opacity_excess at 1.5625. Engine vs splat-transform on a deep LOD level: 40.6 dB WebGPU / 40.4 dB WebGL2 with compensation, 41.4 dB without. v2 SOG renders byte-identical to main on WebGL2. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…mits The over-unity plateau stream (R8) is added to the work buffer only where it fits the device's color attachment count and, on WebGPU, maxColorAttachmentBytesPerSample. GSPLATDATA_LARGE alone uses 32 bytes per sample, the WebGPU spec minimum, so on such devices the stream pushed the work-buffer pass to 33 bytes and failed validation; it is now skipped with a warning and over-unity opacity draws clamped to 1, as with compensation off. Switching data format re-checks the fit. The hybrid renderer's projection-cache size check now counts the plateau word, so a cache over the storage binding limit falls back to CPU sorting instead of failing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Opacity above 1 (up to 1.5625) now travels in the work buffer's alpha instead of a separate plateau stream. The large format already stores it as a half float; the compact format and the WebXR projection cache store it as the SOG v3 opacity byte, (b / 204)^2. The plateau profile is squeezed radially so it ends at the plain gaussian's 2 sqrt(2) sigma, with its shape D solved so the mass stays exact. Quads, culls and extents are therefore the plain gaussian's: the vertex shaders derive D from alpha only for the trim and the fragment curve, and the compute projector, shadow cull and corner code are back to main. This removes the plateau work buffer stream, its runtime attachment and device-limit fallback, and the hybrid cache word. GSplatParams#compensation is a plain property, and the manager turns GSPLAT_OVERUNITY on through setOverUnity when it is enabled and a resource carries opacity above 1. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The plateau shape fit preserved the mass of an untruncated profile, but the fragment shader subtracts the exp(-4) floor as normExp does, so opacity 1.5625 drew 1.625 times a plain gaussian's mass. The fit now solves against the drawn profile (D up to 1.76, within 0.011% of the target mass), and the test integrates the shader's profile. With compensation off, opacity above 1 is now clamped before the anti-aliasing factor scales it, in the vertex shaders and in the GPU-sort projector, which caps it with a new opacityMax uniform. The projector takes the over-unity setting from the per-view parameters, resolved before dispatch, so it matches the draw from the first frame. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Over-unity opacity no longer needs a shader variant or per-resource detection. GSplatParams#compensation sets an opacityExcessMax uniform on the gsplat material (0.5625 when on, 0 when off), and the vertex shaders cap opacity at 1 + opacityExcessMax before anti-aliasing scales it. The GPU-sort projector takes the same cap from the per-view parameters. A plateau splat always draws with peak alpha 1, so its alpha slot carries the plateau shape D to the fragment shader instead of a separate varying: an alpha above 1 is D. Plain splats take the same path as before. This removes the GSPLAT_OVERUNITY define, hasOverUnity on the resources, the manager's resource scan and setOverUnity on the renderers. v2 scenes render byte-identical to main in the large format, and over-unity renders match the previous build. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
I didn't look at this "Drive over-unity gsplat opacity with a uniform" commit in details, but I think the shader that copies data to work buffer should convert whatever alpha the resource has to over-unity, and then the renderer would be always unified and need to flags. |
|
additionally, we have two formats for work buffer:
|
yep it works like this now. |
The work buffer copy already converts every format's opacity to its true value (above 1 for merged gaussians, up to 1.5625), so the renderers need no over-unity setting at all. GSplatParams#compensation, its opacityExcessMax uniform and the projector's opacityMax cap are removed, and the manager, renderers, projector and non-unified instance are back to main. Opacity above 1 still selects the plateau profile, with its shape carried in the alpha passed to the fragment shader. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The plateau shape D for opacity A is now 1 + 1.5t - 0.25t^2 with t = A - 1. It matches the drawn mass within 0.4%, the precision of the SOG v3 opacity byte, and replaces a fitted polynomial in sqrt(t) whose extra precision the stored data cannot use. Renders match the previous fit: 64-68 dB on a decimated anti-aliased scene and 92.7 dB on its streamed LOD. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…gaussian edge The squeezed plateau is the plain gaussian with its radius remapped past an opaque core of radius u0 (a fraction of the quad), and its drawn mass is quadratic in u0, so u0 has a closed form in the opacity. This replaces the shape D, its quadratic fit (0.4% mass error), the squeeze and the plateau trim with one exact helper, gsplatPlateauCore. The fragment shader drops its inversesqrt and squeeze multiplies; the vertex shader passes 1 + u0 in alpha. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Overdraw mode passes opacity x colorRampIntensity / 32 in the fragment alpha, not 1 + a core radius, so an opaque splat at intensity 48 was drawn with a false core and at 64 divided by zero. Skip the core decoding under GSPLAT_OVERDRAW in GLSL and WGSL. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
V3 set both SOG_V2 and SOG_V3, so SOG_V2 had come to mean "uses codebooks". Set SOG_VERSION to the file's version (1 when absent) and test it with SOG_VERSION < 2 / >= 2 / >= 3 in the GLSL and WGSL SOG chunks. The preprocessed shaders are unchanged for v1, v2 and v3. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Adds support for SOG v3, proposed in playcanvas/splat-transform#362, and draws gaussians whose opacity is above 1. SOG v2 and v1 files load as before.
SOG v3 loading
means_l,means_uandscales.quats.webpis not fetched for v3.A = (b / 204)^2, decoded inline: 0 to 1.5625, with 1 exactly at byte 204.meta.opacity.maxis written only when some gaussian's opacity exceeds 1.shN_labels.bholds bits 16–23, so palettes can exceed 65536 entries. This applies to the GLSL and WGSL SH chunks and the CPU iterator.blind.webpis exposed as a resource stream, readable withgetBlind()in work-buffer modifiers. The CPU iterator anddecompress()outputopacity_excessand the blind column.extrasofmeta.json/lod-meta.jsonare exposed asresource.extras, and kept bydecompress().Over-unity opacity
Decimation merges several gaussians into one, and the merged gaussian can carry more mass than a gaussian with opacity 1 can hold. SOG v3 and PLY (
opacity_excesscolumn) store that true opacity, up to 1.5625. The unified renderer draws such a gaussian with peak alpha 1 as an opaque core with the plain gaussian's falloff stretched over the rest of its footprint. The drawn mass is quadratic in the core's radius, so the radius that matches the opacity exactly has a closed form, and the footprint is the plain gaussian's.gsplatPlateauCore), stretch the plainalphaCliptrim past the core, and pass 1 + the radius to the fragment shader as alpha. The fragment shader remaps the distance from the centre past the core and evaluates the plain gaussian. Culls, extents, the projector and the shadow cull are unchanged.GSplatResourceaddsopacity_excessto the sigmoid opacity, capped at 1.5625.Compact work buffer opacity
The compact format (the default) stored opacity as a linear byte clamped to 1, so it now uses the SOG v3 byte for every scene. Existing scenes render slightly differently in compact mode. Measured against the large format, opacity is more accurate: 57.7 vs 55.1 dB on a low-opacity scene, and unchanged on a typical one. Steps are coarser near 1 and finer at low opacity. Large-format rendering is unchanged.
Testing
test/scene/gsplat-sog-v3.test.mjscovers v3 decoding.test/scene/gsplat-unified/gsplat-overunity.test.mjschecks that the GLSL and WGSL core-radius constants match, that the radius is 0 at opacity 1, and that each opacity in (1, 1.5625] draws its mass to within 1e-6.