Makes large-model streaming over a network actually good — fixing read
amplification, then first-paint latency — building on the byte-range work.
v14 layout + TOC (SidecarLayout, pure + unit-tested)
The loader chunks meshes by spatial Morton order, but the sidecar stored
geometry in mesh-id order, so a chunk's meshes were scattered through the
file: streaming one chunk meant either hundreds of tiny range requests or
reading (and discarding) everything between them — a 113 MB model fetched
~340 MB, a 531 MB model 2.25 GB (4.2x). Fix: at bake, reorder meshes into
the loader's chunk order and rebuild vertex/index(LOD0+LOD1)/instance
sections so each chunk is one CONTIGUOUS byte range, and bake a chunk TOC
({first_mesh, mesh_count}). The loader builds chunks straight from the TOC
rather than re-deriving the plan — the float Morton quantisation isn't
bit-identical across toolchains (x86 baker vs wasm loader), so a re-derived
plan scatters the chunks. Format bumped to v14 (regenerate sidecars). The
reorder buckets instances by per-instance mesh_id (the baker never sets
MeshInfo.first_instance — trusting it scrambled every transform → geometry
at the origin). Multiset-verified on a 28,900-instance model: every
instance's placement + geometry preserved. Result: 531 MB fetches 531 MB
(1.0x) in 72 requests (was 2036).
Progressive streaming (concurrency cap + small chunks)
Even at 1x, geometry appeared only after ~the whole model arrived: the
browser multiplexes every in-flight Range request over one HTTP/2 conn, so
unbounded concurrency (9 in flight) split the bandwidth and nothing finished
until the end (measured: first paint after 113 of 118 MB / 35 s @ 24 Mbps).
Cap concurrent chunk loads (kMaxWebInflightChunks=2): the priority-sorted
top chunks finish and paint first, then the next → first paint 9 s. Chunk
size dropped 16->4 MB (cheap now that each chunk is one read; matches Cesium
3D Tiles / xeokit / SVF2) for smoother progression. First-paint is now
metadata-bound (~10 MB tail) — the next lever.
111/111 unit (new test_sidecar_layout: geometry preserved, contiguous layout,
Morton-identity) + 6/6 web smoke pass; desktop bake (SceneLoader) reorders
before writeSidecar; embedded web sample regenerated to v14.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Lets tooling read the wasm heap size (e.g. to verify a large sidecar
streams by byte range instead of loading whole, and to watch memory while
battle-testing big models over the network). Standard emscripten runtime
method, zero size cost.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds a network byte-source alongside the local Blob one. The async-chunk
infra is source-agnostic — only the two JS primitives knew it was a Blob —
so this generalises them and reuses everything else:
- ifcvReadRangeInto: local → Blob.slice; remote → fetch() with a Range
header (206). If a server ignores Range and returns 200, the requested
window is sliced out so it still works (without the bandwidth saving).
- ifcvFileSize: Blob size, or the URL's total length resolved up front.
- ifcvBeginUrlSource: resolves total size (HEAD Content-Length, else a
0-0 ranged GET's Content-Range) then fires _ifcv_source_ready.
- The metadata bootstrap is extracted into a source-agnostic
loadSidecarMetadataWeb(label); loadSidecarFromBlobWeb / FromUrlWeb are
thin entries. streaming_from_blob → streaming_from_web (now covers both).
main_web exports load_sidecar_from_url_c(url); shell.html reads a
?model=URL query param and ccalls it once the app is live (same-origin
needs no CORS; cross-origin hosts must send CORS + Accept-Ranges).
Test: serve.mjs now answers HEAD + Range (206) and falls back to the
ifcviewer-web source dir for sample.ifcview (embedded in the wasm, not in
build-web). New smoke case loads ?model=/sample.ifcview and asserts it
renders via the Range path. 6/6 web smoke + 107/107 unit pass; desktop
unaffected (web-guarded; only the shared field rename touches it).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The fragment shader pre-decodes sRGB→linear to cancel the surface's
automatic linear→sRGB write encoding, so the final bytes match the GL
backend. That only holds when the render target is an sRGB format. On
desktop the surface's preferred format already is (e.g. BGRA8UnormSrgb),
but the browser canvas only offers plain BGRA8Unorm — so nothing
re-encoded and the whole image (background + models) rendered ~3× too
dark (authored bg 0.125,0.137,0.161 → ~32,35,41 collapsed to ~3,4,6).
Fix: when the surface format isn't sRGB, render through an sRGB *view* of
it — the standard WebGPU canvas pattern. surface_view_format_ is the sRGB
sibling of surface_format_ (unchanged when already sRGB, so desktop is a
no-op); configureSurface advertises it via viewFormats, the colour
pipelines (main, MSAA target, edge) target it, and render() creates the
surface view with it. The screenshot path still reads the base texture, so
its BGRA byte-order check stays on surface_format_.
Regression test: sample a 1x1 background pixel and assert it isn't crushed
dark (R,B > 20). Verified visually too — bg is now the correct dark
blue-gray and the cube is properly lit. 5/5 web smoke + 107/107 unit pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
On first web load the sample stayed blank until a click/drag, then popped
in. Root cause: the main draw + cull run before driveStreamingLoads in
render(), so a chunk that becomes resident there is only painted a frame
later. On desktop the streaming thread keeps inFlightApprox() > 0 during a
load, so the render loop keeps ticking and the next frame paints it. On web
the sync MEMFS / Blob load finishes instantly (inFlightApprox stays 0), so
the single post-load requestFrame fired once and the on-demand loop went
idle before the geometry was ever drawn — until some input re-armed it.
Fix: arm a bounded settle burst (kStreamingSettleFrames) whenever there's
streaming activity — a load this frame, work still queued, or a visible
chunk not yet resident — and bleed it down over the next few frames, each
requesting one more. Covers the cull→display latency under an on-demand
loop and still quiesces at idle (no busy-rendering). General, not web-only.
Regression test: the sample must render with NO pointer input — a centred
patch (the framed cube) differs from a corner patch (background); a blank
stall leaves both as background. Verified empirically with a no-interaction
probe (canvas went from a static blank hash to a stable rendered one).
107/107 unit tests pass; 4/4 web smoke tests pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Third Playwright case: click dead-centre on the framed sample, assert the
canvas changes (selection highlight rendered) with zero WebGPU errors. A
broken async pick would hang init or leave the canvas unchanged. All three
cases pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Desktop pick reads the pick staging buffer back with a blocking
`while(!done) waitTickInstance()` spin. On web that spin is a no-op
(Asyncify is off) and hangs the JS event loop, so click-to-select was
dead on web. This adds an async sibling that uses the spontaneous
map-callback pattern (AllowSpontaneous + the browser microtask loop)
already proven by the HiZ readback — no blocking.
- encodePickReadbackToStaging(x,y,want_normal): the pick-pass render +
copy-texel-to-staging, extracted from pickObjectAt verbatim and shared
by both readbacks (desktop sync path unchanged).
- pickObjectAtAsync(x,y,cb) [web]: encode, then map the staging buffer
with a spontaneous callback that delivers object_id to cb. One pick in
flight at a time (a pick issued mid-map is dropped → cb(0)).
- applyPickToSelection(id, add, remove): routes a pick result through the
selection state machine (replace / Shift-add / Ctrl-remove / empty-click
clear), mirroring the desktop ViewportWindow click semantics. selection_
marks dirty so the next render's uploadSelectionFlagsIfDirty flushes the
highlight.
main_web wires it: a left release under a 4px drag threshold (no orbit) is
a pick at down-position * devicePixelRatio, with Shift/Ctrl modifiers;
the result callback applies selection and requests a frame. Web + desktop
build clean; 107/107 unit tests pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds a second Playwright case that picks the sample sidecar through the
file input, waits for the C side to confirm the blob load (console),
then asserts an orbit drag changes the canvas with zero WebGPU errors.
This exercises the #88 path distinctly from the embedded MEMFS sample —
a broken metadata-head/tail or chunk range read renders blank and fails
the orbit-changed-canvas check. Both cases pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Picked files are no longer copied whole into the wasm heap. The browser
File object stays in JS (Module.__ifcvFile) and is read lazily through
Blob.slice byte ranges, so a 200-500 MB sidecar never enters wasm linear
memory — only chunk-sized slices do.
Mechanism (web-only, #if __EMSCRIPTEN__):
- JS glue (EM_JS): ifcvFileSize + ifcvReadRangeInto — slice [off,off+n)
of the File and copy it into a caller-provided heap pointer, then call
back _ifcv_on_range_done. No malloc across the boundary; C pre-sizes
the destination from the read plan.
- webReadRangesAsync: reuses planSidecarReadRanges to coalesce a range
set into Blob.slice reads (1 MB gap — each slice is an async hop),
scatters them into a destination laid out in input order, and fires a
continuation when the whole set lands. An in-flight map keyed by id
survives unordered_map rehash (scratch buffers are heap-owned).
- loadSidecarFromBlobWeb: async metadata load — head (16 B) -> index
count -> tail-to-EOF -> parseSidecarHead/Tail -> applyCachedModel, then
tags the model streaming_from_blob and frames it.
- driveStreamingLoads: blob-sourced models route to beginWebChunkLoad
(async vertex+index range reads -> applyStreamedChunk in the callback),
holding is_loading until the bytes arrive. The embedded MEMFS sample
keeps the synchronous fopen path.
shell.html stashes the File and calls _load_sidecar_from_blob_c instead of
FS.writeFile'ing the whole thing; EXPORTED_RUNTIME_METHODS=['FS'] dropped.
Desktop is untouched (the new members + driveStreamingLoads branch are all
emscripten-guarded). Web links clean; desktop rebuilds; 107/107 unit tests
pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Drives the built web page in a real Chrome (channel:'chrome', so no
`playwright install`): waits for wgpu init, then asserts an orbit drag
changes the composited canvas — one check that simultaneously proves the
scene rendered, mouse input is wired, and the log overlay isn't eating
events — and that zero uncaptured WebGPU errors were logged. Every web
bring-up bug so far (blank render, error-buffer cascade, overlay
swallowing input) is this shape; this would have caught them.
serve.mjs statically serves build-web; the config launches headed
against the real GPU (--use-angle=vulkan + --ignore-gpu-blocklist are
load-bearing for a non-null adapter on Linux Chrome). node_modules and
results are gitignored. See README.md to run.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The status overlay was position:fixed top/left/right with max-height
80vh and pointer-events:auto — a near-fullscreen div that both hid the
model and swallowed mouse events, so orbit drags over most of the canvas
did nothing. Move it to a small bottom-left box, set pointer-events:none
so it never intercepts navigation, and collapse it to a few dimmed lines
once the app goes live (errors re-expand it). It still auto-scrolls to
the newest line.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds an "Open .ifcview…" button that opens the browser's native file
chooser. The picked file's bytes are written into MEMFS and a new
exported entry point, load_uploaded_model_c, reads them back via the
existing loadSidecarFromPath: it resetScene()s the current model
(replace, not append), loads the sidecar, and viewAll()s it. Geometry
becomes resident over subsequent frames through render()'s inline
driveStreamingLoads, same as the embedded sample.
Deliberately a file picker, not drag-drop: browser file drag-drop needs
an X11 drag source (a file manager) to drag *from*, which a minimal WM
(ratpoison) doesn't provide. The native chooser is WM-independent.
Exports _load_uploaded_model_c and the FS runtime method; URL/byte-range
fetch of remote models stays for #88.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Register emscripten HTML5 pointer/wheel handlers that translate raw
events into ViewportCore::orbitBy / panBy / dollyBy: left-drag orbits,
middle/right-drag pans, wheel zooms. mousedown binds to the canvas;
mousemove/up bind to the window so a drag keeps tracking off-canvas. A
contextmenu suppressor lets right-drag pan without the browser menu.
Pure callbacks — no Asyncify, no sync spin — so none of the web init
gotchas apply. Pick stays deferred until the async buffer-readback
rewrite. The embedded sample is now navigable on Chrome + Firefox.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Wire up wgpu init + scene load + RAF render so the embedded sample
sidecar paints on the canvas in both browsers.
Root-cause fix: BufferPool::addSubBuffer spin-waited on
PopErrorScope, which resolves via JS microtask on Dawn-web. The
spin blocked the JS event loop, so the microtask never fired and
the first allocation hung the page indefinitely. Skip the
error-scope dance on Emscripten; trust the buffer pointer.
ViewportCore: add initWgpuAsyncWeb (nested-callback adapter→device
chain with AllowSpontaneous mode, no spin) and loadSidecarFromPath
(Qt-free entry point). waitTickInstance becomes a no-op shim on
web; cull-threads / streaming_thread_ / wgpuSurfacePresent gated
off; chunk I/O runs inline.
main_web.cpp: AppState + initWgpuAsyncWeb → buildPipelines (+
HiZ/edge/pick) → loadSidecarFromPath → ready flag → Module._app_ptr
handoff. The RAF loop lives in shell.html (NOT here) because any
RAF helper called from inside Dawn-web's wgpu Promise.then chain
stalls the device callback.
CMakeLists.txt: EXIT_RUNTIME=0 + Module.noExitRuntime=true (shell)
keeps wasm alive past main() so the device promise lands; no
Asyncify; export _raf_tick_c so shell.html's RAF can call it.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
1) Async-wait spin loops in initWgpu / probeAndCreatePool / pick /
screenshot finalize all called wgpuInstanceProcessEvents in a tight
while. wgpu-native drives queued callbacks from there; Dawn-web
queues the callback for an event-loop tick that never happens
because wasm doesn't yield back to JS. Page hung at the first
await (RequestAdapter) and Firefox flagged the tab as slow.
New `waitTickInstance` helper calls emscripten_sleep(0) on
Emscripten (Asyncify unwinds wasm, JS resolves WebGPU promises,
resume) before wgpuInstanceProcessEvents drains completions into
our callback. Desktop path is unchanged. All five
`while (!done) ProcessEvents` sites switch to the helper.
2) streaming_thread_.start() inside initWgpu spawns a std::thread.
On Emscripten that needs -pthread + COOP/COEP headers from the
hosting page. None of that is wired yet, so the start is gated
`#if !defined(__EMSCRIPTEN__)`. The sync chunk-load fallback
already inside driveStreamingLoads carries the load until #88
replaces it with emscripten_fetch.
3) wgpuSurfacePresent at the end of render() aborted with
"wgpuSurfacePresent is unsupported (use requestAnimationFrame via
html5.h instead)" — Dawn-web composites the canvas at the end of
the RAF tick automatically. Call skipped on Emscripten;
WebViewportHost::requestFrame is the RAF-tick driver.
Page now loads, brings up wgpu, configures the surface, and renders
one frame with the configured background. The frame log in the
status overlay shows the first-frame startup spike (~1000 fps from
a 1 ms tick); subsequent frames are paint-on-demand, which on the
empty scene means no frames at all — consistent with the desktop
event-driven model.
Replaces the standalone wgpu-only clear-color spike in main_web.cpp
with a real WebViewportHost implementation: surface creation via the
emdawnwebgpu canvas-selector source, framebufferSize through
emscripten_get_element_css_size + dpr, requestFrame as a deferred
flag the RAF main_loop consumes, quit through emscripten_force_exit.
main_web.cpp now does the same lifecycle the desktop initWgpu shell
does: core_.initWgpu(web_limits=true) → buildPipelines → buildHiz/
Edge/Pick. The render loop runs core_.render() once per RAF tick when
the host has flagged a frame pending, with a surface reconfigure on
size changes.
Builds clean under emcc 6.0 + emdawnwebgpu (1.4 MB wasm, 278 KB JS
glue). Renders an empty scene with the configured background — the
plumbing is end-to-end through the same ViewportCore code path the
desktop build uses. No sidecar load yet: that lands with the
emscripten_fetch streaming backend (#88).
First Emscripten target. main_web.cpp brings up a wgpu instance against
a <canvas id="viewer-canvas">, requests adapter+device asynchronously
via the standard webgpu.h callback chain, configures the surface, and
clears to the BonsaiViewer slate background on each RAF tick. No
sidecar load, no pipelines, no scene state yet — the goal is to end-
to-end verify the build + canvas + wgpu plumbing.
src/ifcviewer-web/ is a separate CMake root (not a subdir under the
desktop cmake/CMakeLists.txt) so the web build doesn't have to opt out
of Qt / OpenCASCADE / IfcGeom find_packages it can't satisfy. It adds
src/ifcviewer EXCLUDE_FROM_ALL and consumes only IfcViewerCore.
src/ifcviewer/CMakeLists.txt now gates the wgpu-native fetch + the
Qt-using IfcViewer target + install commands behind NOT EMSCRIPTEN.
The wgpu_native link target still resolves under Emscripten as an
INTERFACE library that activates --use-port=emdawnwebgpu (Dawn's
webgpu.h, replaces the legacy -sUSE_WEBGPU=1).
Build:
source path/to/emsdk_env.sh
emcmake cmake -S src/ifcviewer-web -B build-web -G Ninja
ninja -C build-web
python3 -m http.server --directory build-web 8080
# open http://localhost:8080/IfcViewerWeb.html in a WebGPU-capable
# browser (Chrome 113+, Edge 113+).
Phase B step 3 of #45.
Dion Moult