Give host pages a real API over the web viewer, not just "embed it and listen
for picks": read/set the camera, read/set multi-selection, enumerate every
object with its IFC identity, drive per-object visibility, and override
object colours.
The wasm boundary keeps to object_ids (u32 arrays marshalled through the heap,
with an "ask twice" convention on the getters); web/ifcviewer.js layers IFC
GlobalId resolution on top, from the element table getObjects() fetches. Every
id-taking call accepts an objectId, a GlobalId, or an element object.
Colour override needed no new mechanism: color_override_rgba8 was already
plumbed through the sidecar, the instance SSBO, the WGSL shader and the
opaque/transparent cull classifier, but nothing ever wrote a non-zero value
into it. setObjectsColor is the missing writer, which is why an alpha below 255
correctly reclassifies the instance into the transparent pass.
Two bugs surfaced while wiring this up:
- wgpu_initialized_ was only ever set by the Qt desktop host, so on web every
upload guarded on it was a silent no-op — including the pre-existing
recomposeAndUploadModel that federation transforms depend on. The core now
latches it in its own web init.
- The demo pages were copied into the build dir by a POST_BUILD command on the
wasm target, so they only refreshed when the wasm itself relinked; editing a
page left a stale copy that the dev server (and the Playwright suite) kept
serving. Each page now has its own copy rule with a real dependency, and
sample.ifcview is a LINK_DEPENDS so regenerating it forces a relink.
applyCachedModel also now keeps the element metadata it already parses on the
path-based load (it was being dropped), so the embedded sample has GUIDs and
the demo works with no file to pick.
The sample model was three coincident cubes, which made per-object hide and
colour look like no-ops — whatever you hid was still drawn by the box behind
it. make_sample.py regenerates it as a slab, a wall and a beam in distinct
places, so the fixture is reproducible rather than an opaque blob.
Demoed by web/scripting.html (linked from the index; viewer is on
window.viewer) and covered by tests/scripting.spec.mjs — 6 cases against a real
GPU, asserting visibility and colour at the pixels, not just at the API.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Rename the streamer/sidecar transfer and record types to describe what
they are rather than how they move:
MeshChunk -> StreamedMesh
InstanceChunk -> StreamedInstance
InstanceCpu -> InstanceInfo
PackedElementInfo -> ElementTableRecord
uploadMeshChunk -> uploadStreamedMesh
uploadInstanceChunk -> uploadStreamedInstance
buildMeshChunk -> buildStreamedMesh
and the two post-index sidecar metadata blocks:
"critical" metadata -> "geometry" metadata (meshes/instances/georef/TOC)
"deferred" metadata -> "element" metadata (elements + string table)
parseSidecarCritical -> parseSidecarGeometryMetadata
parseSidecarDeferred -> parseSidecarElementMetadata
The one behavioural change: the element hierarchy (parent_id) was
carried through ElementInfo, ElementTableRecord, and the sidecar element
table but never consumed, so drop it and bump SIDECAR_VERSION 16 -> 17.
No back-compat: regenerate sidecars. sample.ifcview is regenerated at v17.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The .ifcview data is hugely redundant (repeated double instance matrices,
patterned indices) — measured 12x zstd whole-file. Server Content-Encoding
can't be used (it breaks HTTP Range), so compress PER-CHUNK into the format.
Format (v16): geometry becomes per-chunk zstd(vertices)+zstd(indices) frames —
each independently Range-fetchable, so streaming is intact — and the critical +
deferred metadata blocks are single zstd frames. SidecarChunk carries the
compressed blob offsets/sizes; applyStreamedChunk (render/upload) is UNCHANGED —
decompression slots into the fetch. Full readSidecar (test/tooling) reconstructs
by decompress+scatter. zstd: desktop links libzstd (also compresses at bake);
the web build (Emscripten has no zstd port) FetchContent's the pinned zstd
source and compiles its decompress-only subset for wasm — no vendored blob,
same version as desktop. New SidecarCompress wraps it (compress guarded off
under Emscripten). Both stream paths — desktop StreamingThread worker + sync
fallback (readChunkGeometryCompressed) and web beginWebChunkLoad — decompress;
readSidecarMetadataOnly / the web bootstrap / loadDeferredMetadataWeb decompress
the metadata blocks. streamingByteProgress reports COMPRESSED bytes. MEASURED: a
752 MB v15 federation → 75 MB v16 (10x; per-file 6.7-15.3x); PP-PLP 118→15 MB,
loads 13/13 chunks on web, 0 errors.
Three fixes found while testing big federations on a real server:
- Web-streamed race: streaming_from_web was set in the deferred-header callback
(a round-trip after the model+chunks exist), so driveStreamingLoads could take
the sync fopen path meanwhile → "failed to read/decompress chunk 0". Now set
immediately after applyCachedModel.
- OOM abort on 18 models: the pool grew unbounded until an alloc failed, but on
web that's an uncatchable bad_alloc abort. Cap total pool capacity
(setMaxTotalCapacity, 3 GB) so it stops before the heap ceiling, and raise
MAXIMUM_MEMORY 2→4 GB (wasm32 max) for headroom.
- Web never evicted (grow-or-block only). At the hard budget, fall through to the
LRU/priority evictor so a big federation stays navigable (highest-contribution
chunks win) instead of freezing with holes.
113/113 desktop + 6/6 web smoke pass. No back-compat: regenerate sidecars
(desktop bakes v16; scratch conv tool migrates v15→v16).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
First-paint over a network is metadata-bound: the whole post-index metadata
(~10 MB on a 118 MB model) had to download before any geometry. But ~25% of
it — elements + string_table, the IFC element tree (names/GUIDs/hierarchy) —
is used only for UI/picking, never for rendering (ViewportCore never touches
it).
v15 splits the post-index metadata into a render-CRITICAL block (meshes,
instances, georef, chunk TOC) preceded by its byte length, then a DEFERRED
block (elements + string_table). The web loader reads only the critical block
before painting; the deferred block sits at a known, self-describing offset
([critical end, EOF)) and is fetched on demand. Desktop reads both (local).
Web on-demand path is wired and complete (not yet called — no UI consumer):
loadDeferredMetadataWeb(model_id) range-fetches + parses the deferred block
into ModelGpuData.elements/string_table, at most once; the first consumer
will be "show the selected object's name" on pick. No background prefetch —
view-only sessions never download the property data (saves 2.64 MB on this
model).
parseSidecarTail split into parseSidecarCritical + parseSidecarDeferred (pure,
unit-tested); StreamingSidecar gains the critical-block locator. Measured
(118 MB model): critical metadata 10.35 -> 7.71 MB, deferred 2.64 MB off the
path; first paint 10.5 -> 9.6 s @ 24 Mbps. (Instances still dominate the
critical block — the next metadata lever.) Format -> v15, no back-compat;
regenerate sidecars. 111/111 unit + 6/6 web smoke pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Makes large-model streaming over a network actually good — fixing read
amplification, then first-paint latency — building on the byte-range work.
v14 layout + TOC (SidecarLayout, pure + unit-tested)
The loader chunks meshes by spatial Morton order, but the sidecar stored
geometry in mesh-id order, so a chunk's meshes were scattered through the
file: streaming one chunk meant either hundreds of tiny range requests or
reading (and discarding) everything between them — a 113 MB model fetched
~340 MB, a 531 MB model 2.25 GB (4.2x). Fix: at bake, reorder meshes into
the loader's chunk order and rebuild vertex/index(LOD0+LOD1)/instance
sections so each chunk is one CONTIGUOUS byte range, and bake a chunk TOC
({first_mesh, mesh_count}). The loader builds chunks straight from the TOC
rather than re-deriving the plan — the float Morton quantisation isn't
bit-identical across toolchains (x86 baker vs wasm loader), so a re-derived
plan scatters the chunks. Format bumped to v14 (regenerate sidecars). The
reorder buckets instances by per-instance mesh_id (the baker never sets
MeshInfo.first_instance — trusting it scrambled every transform → geometry
at the origin). Multiset-verified on a 28,900-instance model: every
instance's placement + geometry preserved. Result: 531 MB fetches 531 MB
(1.0x) in 72 requests (was 2036).
Progressive streaming (concurrency cap + small chunks)
Even at 1x, geometry appeared only after ~the whole model arrived: the
browser multiplexes every in-flight Range request over one HTTP/2 conn, so
unbounded concurrency (9 in flight) split the bandwidth and nothing finished
until the end (measured: first paint after 113 of 118 MB / 35 s @ 24 Mbps).
Cap concurrent chunk loads (kMaxWebInflightChunks=2): the priority-sorted
top chunks finish and paint first, then the next → first paint 9 s. Chunk
size dropped 16->4 MB (cheap now that each chunk is one read; matches Cesium
3D Tiles / xeokit / SVF2) for smoother progression. First-paint is now
metadata-bound (~10 MB tail) — the next lever.
111/111 unit (new test_sidecar_layout: geometry preserved, contiguous layout,
Morton-identity) + 6/6 web smoke pass; desktop bake (SceneLoader) reorders
before writeSidecar; embedded web sample regenerated to v14.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Wire up wgpu init + scene load + RAF render so the embedded sample
sidecar paints on the canvas in both browsers.
Root-cause fix: BufferPool::addSubBuffer spin-waited on
PopErrorScope, which resolves via JS microtask on Dawn-web. The
spin blocked the JS event loop, so the microtask never fired and
the first allocation hung the page indefinitely. Skip the
error-scope dance on Emscripten; trust the buffer pointer.
ViewportCore: add initWgpuAsyncWeb (nested-callback adapter→device
chain with AllowSpontaneous mode, no spin) and loadSidecarFromPath
(Qt-free entry point). waitTickInstance becomes a no-op shim on
web; cull-threads / streaming_thread_ / wgpuSurfacePresent gated
off; chunk I/O runs inline.
main_web.cpp: AppState + initWgpuAsyncWeb → buildPipelines (+
HiZ/edge/pick) → loadSidecarFromPath → ready flag → Module._app_ptr
handoff. The RAF loop lives in shell.html (NOT here) because any
RAF helper called from inside Dawn-web's wgpu Promise.then chain
stalls the device callback.
CMakeLists.txt: EXIT_RUNTIME=0 + Module.noExitRuntime=true (shell)
keeps wasm alive past main() so the device promise lands; no
Asyncify; export _raf_tick_c so shell.html's RAF can call it.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Dion Moult