Rewrite the BonsaiViewer viewport architecture page to match the current
renderer (updated type names, streaming/sidecar flow). Add a dedicated
.ifcview sidecar format reference page and link it from the ifcopenshell
formats toctree, and polish the Bonsai intro copy.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Rename the streamer/sidecar transfer and record types to describe what
they are rather than how they move:
MeshChunk -> StreamedMesh
InstanceChunk -> StreamedInstance
InstanceCpu -> InstanceInfo
PackedElementInfo -> ElementTableRecord
uploadMeshChunk -> uploadStreamedMesh
uploadInstanceChunk -> uploadStreamedInstance
buildMeshChunk -> buildStreamedMesh
and the two post-index sidecar metadata blocks:
"critical" metadata -> "geometry" metadata (meshes/instances/georef/TOC)
"deferred" metadata -> "element" metadata (elements + string table)
parseSidecarCritical -> parseSidecarGeometryMetadata
parseSidecarDeferred -> parseSidecarElementMetadata
The one behavioural change: the element hierarchy (parent_id) was
carried through ElementInfo, ElementTableRecord, and the sidecar element
table but never consumed, so drop it and bump SIDECAR_VERSION 16 -> 17.
No back-compat: regenerate sidecars. sample.ifcview is regenerated at v17.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Full section tool for the web viewport, with the gizmo + interaction shared with
desktop from one codebase.
- True-face surface pick. pickSurfaceAt had always ray-cast the instance AABB (to
skip a depth readback), so cuts sat in front of the real surface. The pick
fragment already computes the exact world_pos (it clips sections with it); now
it OUTPUTS it to a 3rd pick MRT (RGBA32F) that every pick path renders, and
pickSurfaceAt / pickSurfaceAtAsync read it back (decodeMappedPickPosition;
ray-AABB kept only as a fallback). The web async pick chains id -> normal ->
position spontaneous staging maps.
- Web tool: LMB drops a cut at the picked surface (LMB drag still orbits), K
toggles, Shift+K clears; oriented to the real MRT surface normal. Exports + a
Section / Clear cuts toolbar pair.
- Shared gizmo: lifted the section-gizmo renderer (SECTION_WGSL + thick-line AA +
quad+arrow VBO + pack + screen-space hit-test) out of the Qt-coupled
OverlayRenderer into a Qt-free SectionGizmoRenderer that ViewportCore::render
draws for BOTH desktop and web (both already render via render()). One identical
gizmo; OverlayRenderer's now-dead section code removed. Fixed 1 m size (matches
the desktop constant).
- Interaction (shared): hitTestSectionGizmo (SectionGizmoRenderer::hitTest) +
beginSectionDrag / updateSectionDrag / endSectionDrag live in ViewportCore.
Drag a gizmo arrow to slide the plane along its normal; Del/Backspace removes
the most recent cut. Desktop's ViewportWindow dropped its duplicate hit-test /
drag math + state and delegates to the core; web wires the same calls.
Tests: sectionPlaneCount add/clear/cap (Catch2, 125); web smoke "click a surface
cuts geometry, clear restores" exercises the shared gizmo + 3-MRT pick (11/11).
Desktop object-pick / marquee unaffected; BonsaiViewer builds.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add IfcOpenShell-Python and IfcViewer test-running documentation, including desktop CTest targets and web Playwright smoke tests. Move the web test README content into the Sphinx docs.\n\nGenerated with the assistance of an AI coding tool.
Update Bonsai viewer headers to identify Bonsai and GPL licensing, add Bonsai Viewer documentation, and document debug output capture.
Generated with the assistance of an AI coding tool.
Bring rubber-band box-select to the web on the Web preset's select button (RMB).
- Core: factor the pick-pass encode + rect copy out of picksInRect into
encodeBoxPickToStaging (mirroring how single-pick shares
encodePickReadbackToStaging), shared by the sync picksInRect (desktop) and a
new async picksInRectAsync (web) — the latter maps the staging buffer via a
spontaneous callback because the sync spin-map hangs the JS loop. New
applyMarqueeToSelection (plain replace / Shift add / Ctrl remove).
- Web main_web: a select-button drag past the click threshold draws a marquee
rubber-band (a plain DOM <div> positioned in CSS px — no GPU overlay pass,
which the web lib lacks) and on release box-picks the rect (device px) and
applies it to the selection. A click (no drag) still single-picks.
- Web shell.html: the #marquee div + styling, and — the reported bug — a
contextmenu preventDefault on the canvas so RMB (now the select button) doesn't
pop the browser menu. (Firefox still forces its native menu on Shift+RightClick;
that's a browser escape hatch pages can't override.)
Tests: applyMarqueeToSelection replace/add/remove + id-0 (Catch2, 124 total);
web smoke marquee drag → rubber-band shown → selection changes → hidden (10/10).
Desktop picksInRect unchanged in behaviour; BonsaiViewer builds.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Drop the obsolete diagnostic macOS WGPU workflow that referenced removed standalone viewer paths and targets.\n\nGenerated with the assistance of an AI coding tool.
Rename short local variables and parameters in the viewer loading, sidecar, and BonsaiViewer command paths to make their responsibilities clearer.\n\nGenerated with the assistance of an AI coding tool.
Make orbit/pan/select mouse bindings pure data owned by ViewportCore so both
hosts and every preset share one source of truth, and add a "Web" preset. This
rounds out the matrix: the desktop gains a web-style scheme and the web inherits
all presets, with no per-platform hardcoding.
- Core: NavBindings { orbit, pan, select button + modifier } + setNavPreset
("blender" default | "rhino" | "revit" | "web") + navBindings(). Select is
preset-driven too (was hardcoded LMB) so "web" moves it to RMB. web = orbit
LMB, pan MMB, select RMB (LMB drag orbits with no click/drag ambiguity; RMB
click-selects / drag-marquees). NavMod uses "Plain" not "None" (X11 #defines
None to 0L).
- Desktop ViewportWindow: applyNavPreset sources the core table (mapped to Qt);
marquee-arm / single-pick dispatch keys off select_button_. Default stays
blender → no behaviour change.
- Desktop config: AppSettings::NavPreset gains Web + navPresetName(); the
Settings dialog lists it. This also FIXES a pre-existing gap — the preset combo
was persisted but never applied (only WGPU_NAV_PRESET env worked). MainWindow
now applies the persisted preset at startup (env override still wins) and live
on navPresetChanged, so all four presets actually work from the dialog.
- Web main_web: classifyPress routes the pressed button through navBindings()
(orbit/pan/select), defaulting to the "web" preset; context menu already
suppressed so RMB is free.
Tests: setNavPreset table (Catch2, 123 total); web smoke select tests use RMB.
BonsaiViewer builds; 9/9 web smoke.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Coverage had lagged the recent feature work. Add both layers:
- test_viewport_camera (Catch2, headless): constructs ViewportCore with a mock
ViewportHost — construction/teardown touch no GPU (the wgpu teardown lives in
releaseWgpuModelGpuData, only reached with models loaded), and the camera ops
are pure — so it unit-tests the SHARED fly math fast and deterministically:
flyMove (forward step, 5x boost, opposing-key cancel, dt clamp, QE along +Z,
degenerate-pitch stays finite), flyLook (turn-in-place pins the eye, pitch
clamp), flyAdjustSpeed (x1.25/notch, [0.05,1000] clamp), toggleXray, and
hideSelected/showAll. Links the built IfcViewerCore (ViewportCore.cpp is the
monster TU that can't compile standalone). +9 cases → 122 desktop.
- smoke.spec.mjs (Playwright): fly (enter → W moves the camera → Esc exits),
x-ray (toggle translucency on/off), and hide-after-pick, driving the exported
C hooks end-to-end. +3 cases → 9 web.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Lift the visibility + X-ray ops into ViewportCore so desktop and web share them.
The cull already reads visibility_ (hidden objects skipped) and the frame uniform
reads xray_alpha_cap_, both per frame, so each op just mutates state + schedules
a frame — no GPU buffers to rebuild, no rendering work:
- hideSelected (hide the selection, then deselect), isolateSelected (hide every
non-selected object in a visible model), showAll (clear the hidden set),
toggleXray (flip the alpha cap 1.0<->0.3; cull routes all to the transparent
pass), xrayActive().
- Desktop ViewportWindow: the H / Shift+H / Alt+H / Alt+X keys and the menu-action
wrappers now call the core; the four inline/duplicated implementations are gone.
- Web main_web: same keys (H hide · Shift+H isolate · Alt+H show all · Alt+X
x-ray) + toolbar buttons (Hide / Isolate / Show all / X-ray, the last reflecting
active state).
Verified on web: x-ray toggles and visibly translucent-izes the scene, hide after
a pick removes geometry, 0 GPU errors. 113/113 desktop + 6/6 web smoke; desktop
app builds.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
applyCachedModel already does a one-time viewAll gated by initial_view_applied_
(desktop + web) — it frames the FIRST model loaded and never re-frames as more
arrive. Two web-only divergences from the desktop viewer, fixed here:
- loadSidecarMetadataWeb called viewAll() AGAIN, unconditionally, per model, so
every federated model that streamed in yanked the camera back to fit the whole
scene. Drop the redundant call; the shared gate handles first-model framing.
- Nothing ever cleared initial_view_applied_. On web the embedded sample sets it
at startup, so after clear_scene_c (a ?model= federation load) the gate was
already tripped and the loaded models never got framed — the camera stayed on
the prior view. (The redundant per-model viewAll above masked this until it was
removed.) Clear the flag in resetScene: a fresh scene should auto-frame its
first model — correct for desktop fresh-opens too.
113/113 desktop + 6/6 web smoke pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
DRY the fly camera into ViewportCore so desktop and web share identical math:
- flyMove(fwd,back,right,left,up,down,boost,dt): WASD in the view plane, QE
along world +Z, forward = eye→target (no snap after orbiting), Shift = 5x,
dt clamped to 0.1s. flyLook(dx,dy): turn in place (yaw/pitch, eye pinned,
0.2 deg/px, pitch ±89.9). flyAdjustSpeed(notches): wheel scales speed x1.25.
fly_move_speed_ + orbitEye now live in the core (orbitEye's duplicate removed
from ViewportWindow).
- Desktop ViewportWindow: fpsIntegrate / mouse-look / wheel-speed call the core
methods; behaviour unchanged, BonsaiViewer builds clean.
- Web main_web: Shift+F (or the Fly toolbar button) enters and pointer-locks the
canvas; W/A/S/D/Q/E + Shift are held-tracked (keydown/keyup) and integrated
each RAF frame with wall-clock dt; pointer-lock mouse deltas drive flyLook;
wheel tunes speed. Exit on Esc OR a canvas click (matches desktop) — a
pointerlockchange handler catches the browser eating the first Esc to release
the lock (so a single Esc exits), guarded by fly_locked so a denied lock on
entry doesn't insta-exit. Fly button reflects/ syncs active state.
Verified: web enter→WASD moves→click/Esc exits, 0 GPU errors; 113/113 desktop
core + 6/6 web smoke; desktop app builds.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The .ifcview data is hugely redundant (repeated double instance matrices,
patterned indices) — measured 12x zstd whole-file. Server Content-Encoding
can't be used (it breaks HTTP Range), so compress PER-CHUNK into the format.
Format (v16): geometry becomes per-chunk zstd(vertices)+zstd(indices) frames —
each independently Range-fetchable, so streaming is intact — and the critical +
deferred metadata blocks are single zstd frames. SidecarChunk carries the
compressed blob offsets/sizes; applyStreamedChunk (render/upload) is UNCHANGED —
decompression slots into the fetch. Full readSidecar (test/tooling) reconstructs
by decompress+scatter. zstd: desktop links libzstd (also compresses at bake);
the web build (Emscripten has no zstd port) FetchContent's the pinned zstd
source and compiles its decompress-only subset for wasm — no vendored blob,
same version as desktop. New SidecarCompress wraps it (compress guarded off
under Emscripten). Both stream paths — desktop StreamingThread worker + sync
fallback (readChunkGeometryCompressed) and web beginWebChunkLoad — decompress;
readSidecarMetadataOnly / the web bootstrap / loadDeferredMetadataWeb decompress
the metadata blocks. streamingByteProgress reports COMPRESSED bytes. MEASURED: a
752 MB v15 federation → 75 MB v16 (10x; per-file 6.7-15.3x); PP-PLP 118→15 MB,
loads 13/13 chunks on web, 0 errors.
Three fixes found while testing big federations on a real server:
- Web-streamed race: streaming_from_web was set in the deferred-header callback
(a round-trip after the model+chunks exist), so driveStreamingLoads could take
the sync fopen path meanwhile → "failed to read/decompress chunk 0". Now set
immediately after applyCachedModel.
- OOM abort on 18 models: the pool grew unbounded until an alloc failed, but on
web that's an uncatchable bad_alloc abort. Cap total pool capacity
(setMaxTotalCapacity, 3 GB) so it stops before the heap ceiling, and raise
MAXIMUM_MEMORY 2→4 GB (wasm32 max) for headroom.
- Web never evicted (grow-or-block only). At the hard budget, fall through to the
LRU/priority evictor so a big federation stays navigable (highest-contribution
chunks win) instead of freezing with holes.
113/113 desktop + 6/6 web smoke pass. No back-compat: regenerate sidecars
(desktop bakes v16; scratch conv tool migrates v15→v16).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Streaming picked candidates on raw frustum visibility, so viewAll over a big
federation (all models in-frustum) fetched every chunk — even fine chunks that
project to sub-pixel and the renderer never draws. Gate streaming on the SAME
contribution decision the render path already makes.
- New per-chunk contribution_visible_count: instances that passed frustum AND
the contribution cull (projected radius >= min_radius_px), counted BEFORE HiZ
— so it's stable while the camera is still (unlike the HiZ-post counters,
which flip frame-to-frame and would thrash the loader) and shifts only on
navigation, when the working set should. The candidate gate skips chunks with
count 0: the network pulls only what's resolvable now; the rest stream in as
you approach. Verified: fit view needs the geometry, zoomed-out needs 0.
- Loading UI reworked from per-model segments to a combined bar over the whole
federation: dark track = not needed for this view, dim = needed-but-unloaded,
bright = loaded. "loaded / needed" = how done THIS view is; "needed / total" =
how much of the model the view requires — "Loading 45 / 90 MB for this view ·
12% of 718 MB total". Driven by ViewportCore::streamingByteProgress + ifcv_bytes_*.
Two fixes from testing: (1) show a distinct overhead phase while total==0
("Loading model data — X MB · Y/N models ready") so the metadata download
isn't a dead-looking bar; (2) re-assert display:block every active frame so the
bar REAPPEARS when navigation reveals new chunks (was only set in
beginLoadProgress → stayed hidden after the first catch-up). Per-model exports
kept for a future detailed view.
111/111 unit + 6/6 web smoke pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The camera math already lived in the shared ViewportCore; the desktop just
bound keys to it. The web build had no keyboard handler and no camera UI, so
none of it was reachable. Wire it up (Tier 1 + 2 of nav parity; fly mode is a
separate follow-up).
Shared core:
- setStandardView(StandardView) — named Front/Back/Left/Right/Top/Bottom wrapper
over setStandardView(yaw,pitch), so the axis→angle mapping lives in one place.
- frameSelection() — lifts the desktop's "union selected AABBs → frameAabb(1.30)"
focus logic out of ViewportWindow into the core. Desktop's focusOnSelectedObject
and the X/Y/Z hotkeys now call these (DRY, behaviour unchanged).
Web:
- main_web gains a keydown handler matching the desktop bindings — Home=view all,
F=zoom to selected, P=ortho toggle, X/Y/Z (+Shift=negative)=standard views —
plus exported entry points (view_all_c / frame_selection_c / toggle_projection_c
/ projection_is_ortho_c / standard_view_c) for the toolbar.
- shell.html adds a bottom nav toolbar (Fit / Focus / Persp-Ortho / the six views)
with the hotkeys in tooltips; the ortho button reflects state.
Verified: Z key and Front button both move the camera, ortho toggles render +
label, zero GPU errors; 111/111 unit + 6/6 web smoke pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Query string: auto-load a whole federation, not one model. Accepts repeated
params (?model=a&model=b&…) and/or a comma list (?models=a,b,c); each becomes
its own streamed byte-source, clearing the embedded sample once then streaming
concurrently into one scene. A failed URL is reported without aborting the rest.
Loading UI: the bare aggregate chunk count didn't reveal the federation state,
so surface per-model progress. ViewportCore::streamingModelCount() +
streamingModelProgress(idx,…) (ordered by model_id = load order) feed
main_web's ifcv_model_count_c / ifcv_model_{resident,total}_c. shell.html draws
a panel: "Loading N models — X done · Y streaming · Z waiting · N MB" plus one
segment per model (blue fill while streaming, green when resident, grey while
its metadata is still pending) — so parallel loading and how many remain are
visible at a glance.
Verified: 10 models from a 10x ?model= query stream in parallel; the panel
steps 10 waiting → streaming → all done. 111/111 unit + 6/6 web smoke pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The scene core is already multi-model — models_gpu_ is a map, applyCachedModel
APPENDS, and per-model model_id / object_id rebasing / georef+transformation
are how the desktop federates today. The only web-specific gap was the byte
source: web had ONE global source (__ifcvFile/__ifcvUrl) and reset the scene on
every load, so it could show one file at a time. Desktop meanwhile carries a
per-model source (streaming_file_path).
Mirror that on web: give each model its own web_source_id into a JS source
registry (Module.__ifcvSources[id] = a picked File or a sized remote URL).
beginWebChunkLoad, the metadata bootstrap, and the on-demand deferred fetch all
read from the owning model's source, so several files stream concurrently into
one federated scene — reusing all the shared machinery (viewAll, picking, the
GUID fetch) untouched.
- webReadRangesAsync / ifcvReadRangeInto / ifcvSourceSize take a source id.
- loadSidecarMetadataWeb(source_id, …) appends (no resetScene); main_web
exposes load_sidecar_from_source_c(id) + clear_scene_c().
- URL size resolution moved to JS (shell.html registers + sizes sources via
HEAD/Range), retiring the C-side ifcvBeginUrlSource / ifcv_source_ready dance.
- shell.html: source registry + "Open" (replace) / "Add" (append) buttons,
multi-file selection; ?model= registers a URL source then loads.
Verified: two sidecars from two sources stream into one scene, both fully
resident, zero GPU errors. 111/111 unit + 6/6 web smoke pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
First real consumer of the v15 deferred property block, and an end-to-end
demonstration that on-demand property loading works. On a left-click pick,
logSelectedObjectGuidWeb ensures the owning model's deferred block is loaded
(loadDeferredMetadataWeb — a network fetch the FIRST time, cached after) and
logs the picked object's GUID to the console.
Fix uncovered while wiring it: applyCachedModel rebases instance object_ids to
a per-model global base (object_id_base) to keep them unique across models,
but the deferred elements carry the sidecar's original local ids — so a lookup
by the picked (global) id missed. Store object_id_base on the model and rebase
the elements by it when the deferred block loads.
Verified: with a streamed model, the deferred block is fetched ONLY after the
first pick (not at load), and the pick logs a valid 22-char IFC GUID. 111/111
unit + 6/6 web smoke pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Users had no feedback during the seconds before/while a model streams. Add a
top progress strip + caption driven by the streaming state:
- "Loading model… N MB" while the critical metadata downloads (no chunks yet),
- "Loading geometry — R / T chunks · N MB" as chunks go resident,
- "Loaded — T chunks · N MB", then it hides.
ViewportCore::streamingProgress(resident, total) sums chunk residency across
models; main_web exports ifcv_chunks_resident_c / ifcv_chunks_total_c for
shell.html to poll each RAF. The EM_JS range reader accumulates
Module.__ifcvBytesLoaded so the caption can show MB downloaded. Shown only for
streamed loads (?model= URL / picked file), not the embedded sample.
6/6 web smoke pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
First-paint over a network is metadata-bound: the whole post-index metadata
(~10 MB on a 118 MB model) had to download before any geometry. But ~25% of
it — elements + string_table, the IFC element tree (names/GUIDs/hierarchy) —
is used only for UI/picking, never for rendering (ViewportCore never touches
it).
v15 splits the post-index metadata into a render-CRITICAL block (meshes,
instances, georef, chunk TOC) preceded by its byte length, then a DEFERRED
block (elements + string_table). The web loader reads only the critical block
before painting; the deferred block sits at a known, self-describing offset
([critical end, EOF)) and is fetched on demand. Desktop reads both (local).
Web on-demand path is wired and complete (not yet called — no UI consumer):
loadDeferredMetadataWeb(model_id) range-fetches + parses the deferred block
into ModelGpuData.elements/string_table, at most once; the first consumer
will be "show the selected object's name" on pick. No background prefetch —
view-only sessions never download the property data (saves 2.64 MB on this
model).
parseSidecarTail split into parseSidecarCritical + parseSidecarDeferred (pure,
unit-tested); StreamingSidecar gains the critical-block locator. Measured
(118 MB model): critical metadata 10.35 -> 7.71 MB, deferred 2.64 MB off the
path; first paint 10.5 -> 9.6 s @ 24 Mbps. (Instances still dominate the
critical block — the next metadata lever.) Format -> v15, no back-compat;
regenerate sidecars. 111/111 unit + 6/6 web smoke pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Makes large-model streaming over a network actually good — fixing read
amplification, then first-paint latency — building on the byte-range work.
v14 layout + TOC (SidecarLayout, pure + unit-tested)
The loader chunks meshes by spatial Morton order, but the sidecar stored
geometry in mesh-id order, so a chunk's meshes were scattered through the
file: streaming one chunk meant either hundreds of tiny range requests or
reading (and discarding) everything between them — a 113 MB model fetched
~340 MB, a 531 MB model 2.25 GB (4.2x). Fix: at bake, reorder meshes into
the loader's chunk order and rebuild vertex/index(LOD0+LOD1)/instance
sections so each chunk is one CONTIGUOUS byte range, and bake a chunk TOC
({first_mesh, mesh_count}). The loader builds chunks straight from the TOC
rather than re-deriving the plan — the float Morton quantisation isn't
bit-identical across toolchains (x86 baker vs wasm loader), so a re-derived
plan scatters the chunks. Format bumped to v14 (regenerate sidecars). The
reorder buckets instances by per-instance mesh_id (the baker never sets
MeshInfo.first_instance — trusting it scrambled every transform → geometry
at the origin). Multiset-verified on a 28,900-instance model: every
instance's placement + geometry preserved. Result: 531 MB fetches 531 MB
(1.0x) in 72 requests (was 2036).
Progressive streaming (concurrency cap + small chunks)
Even at 1x, geometry appeared only after ~the whole model arrived: the
browser multiplexes every in-flight Range request over one HTTP/2 conn, so
unbounded concurrency (9 in flight) split the bandwidth and nothing finished
until the end (measured: first paint after 113 of 118 MB / 35 s @ 24 Mbps).
Cap concurrent chunk loads (kMaxWebInflightChunks=2): the priority-sorted
top chunks finish and paint first, then the next → first paint 9 s. Chunk
size dropped 16->4 MB (cheap now that each chunk is one read; matches Cesium
3D Tiles / xeokit / SVF2) for smoother progression. First-paint is now
metadata-bound (~10 MB tail) — the next lever.
111/111 unit (new test_sidecar_layout: geometry preserved, contiguous layout,
Morton-identity) + 6/6 web smoke pass; desktop bake (SceneLoader) reorders
before writeSidecar; embedded web sample regenerated to v14.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Lets tooling read the wasm heap size (e.g. to verify a large sidecar
streams by byte range instead of loading whole, and to watch memory while
battle-testing big models over the network). Standard emscripten runtime
method, zero size cost.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Battle-testing real sidecars over HTTP Range exposed severe thrash: a
531 MB model re-fetched 2.25 GB (4×) and never converged — viewAll puts
the whole model in frustum, so every chunk wants to be resident, and the
web async path made it worse two ways:
- A web load only consumes pool space when it COMPLETES (async), so the
per-frame issuance over-committed the pool; completions then failed
applyStreamedChunk on a full pool, the chunk re-candidated with no
cooldown, and re-fetched every frame.
- Pool growth is itself async on web (provisional sub-buffers validated
off the JS event loop), so even fetched chunks failed to alloc until
the pool caught up, and re-fetched.
Fix: gate web chunk issuance on VALIDATED free space + in-flight
reservation, and grow the pool BEFORE fetching:
- streaming_web_inflight_bytes_ reserves each in-flight load's footprint
so we never have more bytes in flight than the pool can place.
- When a visible chunk doesn't fit validated free, don't fetch — call
pool_.requestGrowth() (BufferPool: drives the async provisional grow
without allocating) and short-back-off; the chunk is fetched once,
after space exists. When the pool is saturated (model > GPU memory),
long-cooldown so a never-fitting chunk isn't re-fetched. Gating before
the evictor also kills phase-2 visible↔visible swap thrash.
- On async load failure, cool down (short if the pool can still grow,
long if saturated) instead of re-candidating next frame.
Result (manual battle tool, host.mjs + real files): 531 MB now loads
23/23 chunks, 322 MB loads 14/14 — resident climbs monotonically with
ZERO thrash warnings and a stable resident set, vs the old re-fetch loop.
The whole model resides on the GPU and stays. (Remaining ~3× ramp
over-fetch — per-chunk re-loads during the async-growth ramp + read
amplification from chunk byte-locality — is a separate efficiency
follow-up, not thrash.) 6/6 web smoke + 107/107 unit pass; desktop
unaffected (the gate is web-only; requestGrowth is a no-op wrapper there).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
beginWebChunkLoad read the vertex ranges, then in the completion callback
read the index ranges, then applied — two serial round trips per chunk.
On a network that's the dominant per-chunk latency. Now both reads fire
at once and a small shared join (payloads + per-read done/ok flags) runs
the apply when the second lands, halving per-chunk RTT. Model re-lookup
still happens at apply time, so a resetScene mid-flight is dropped safely.
Per-chunk concurrency stacks with the existing across-chunk concurrency
(driveStreamingLoads issues several loads per frame); the browser caps
simultaneous connections per origin, so no explicit in-flight cap is
needed. 6/6 web smoke + 107/107 unit pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds a network byte-source alongside the local Blob one. The async-chunk
infra is source-agnostic — only the two JS primitives knew it was a Blob —
so this generalises them and reuses everything else:
- ifcvReadRangeInto: local → Blob.slice; remote → fetch() with a Range
header (206). If a server ignores Range and returns 200, the requested
window is sliced out so it still works (without the bandwidth saving).
- ifcvFileSize: Blob size, or the URL's total length resolved up front.
- ifcvBeginUrlSource: resolves total size (HEAD Content-Length, else a
0-0 ranged GET's Content-Range) then fires _ifcv_source_ready.
- The metadata bootstrap is extracted into a source-agnostic
loadSidecarMetadataWeb(label); loadSidecarFromBlobWeb / FromUrlWeb are
thin entries. streaming_from_blob → streaming_from_web (now covers both).
main_web exports load_sidecar_from_url_c(url); shell.html reads a
?model=URL query param and ccalls it once the app is live (same-origin
needs no CORS; cross-origin hosts must send CORS + Accept-Ranges).
Test: serve.mjs now answers HEAD + Range (206) and falls back to the
ifcviewer-web source dir for sample.ifcview (embedded in the wasm, not in
build-web). New smoke case loads ?model=/sample.ifcview and asserts it
renders via the Range path. 6/6 web smoke + 107/107 unit pass; desktop
unaffected (web-guarded; only the shared field rename touches it).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reported: open a model in the web viewer, then launch the desktop
BonsaiViewer and open another model — the browser tab freezes, and a
fresh web tab then fails with "RequestDevice failed: Not enough memory
left". Root cause is GPU-memory contention: two heavy GPU clients on one
GPU, and the desktop app's allocations starve the browser's WebGPU
process, which reclaims our device.
We can't conjure GPU memory, but we were amplifying the symptom: with no
device-lost handler, render() kept driving a dead device —
wgpuSurfaceGetCurrentTexture returns Lost every frame and the
reconfigure + requestFrame retry becomes a tight per-RAF loop that hangs
the tab. Now the web device descriptor wires a device-lost callback that
latches device_lost_ (ignoring the intentional Destroyed reason from our
own shutdown); render() bails while set, so the loop goes idle instead of
spinning, and the console logs guidance to reload. The fresh-tab
RequestDevice OOM is genuine GPU exhaustion — surfaced as before, now with
a clearer message.
Verified the lost callback doesn't disturb Dawn-web's RequestDevice (all
5 web smoke tests still init + pass); desktop unaffected (device_lost_
stays false). The real contention path can't be reproduced in the headless
harness, so the loss handler itself is covered by review, not a test.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The fragment shader pre-decodes sRGB→linear to cancel the surface's
automatic linear→sRGB write encoding, so the final bytes match the GL
backend. That only holds when the render target is an sRGB format. On
desktop the surface's preferred format already is (e.g. BGRA8UnormSrgb),
but the browser canvas only offers plain BGRA8Unorm — so nothing
re-encoded and the whole image (background + models) rendered ~3× too
dark (authored bg 0.125,0.137,0.161 → ~32,35,41 collapsed to ~3,4,6).
Fix: when the surface format isn't sRGB, render through an sRGB *view* of
it — the standard WebGPU canvas pattern. surface_view_format_ is the sRGB
sibling of surface_format_ (unchanged when already sRGB, so desktop is a
no-op); configureSurface advertises it via viewFormats, the colour
pipelines (main, MSAA target, edge) target it, and render() creates the
surface view with it. The screenshot path still reads the base texture, so
its BGRA byte-order check stays on surface_format_.
Regression test: sample a 1x1 background pixel and assert it isn't crushed
dark (R,B > 20). Verified visually too — bg is now the correct dark
blue-gray and the cube is properly lit. 5/5 web smoke + 107/107 unit pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
On first web load the sample stayed blank until a click/drag, then popped
in. Root cause: the main draw + cull run before driveStreamingLoads in
render(), so a chunk that becomes resident there is only painted a frame
later. On desktop the streaming thread keeps inFlightApprox() > 0 during a
load, so the render loop keeps ticking and the next frame paints it. On web
the sync MEMFS / Blob load finishes instantly (inFlightApprox stays 0), so
the single post-load requestFrame fired once and the on-demand loop went
idle before the geometry was ever drawn — until some input re-armed it.
Fix: arm a bounded settle burst (kStreamingSettleFrames) whenever there's
streaming activity — a load this frame, work still queued, or a visible
chunk not yet resident — and bleed it down over the next few frames, each
requesting one more. Covers the cull→display latency under an on-demand
loop and still quiesces at idle (no busy-rendering). General, not web-only.
Regression test: the sample must render with NO pointer input — a centred
patch (the framed cube) differs from a corner patch (background); a blank
stall leaves both as background. Verified empirically with a no-interaction
probe (canvas went from a static blank hash to a stable rendered one).
107/107 unit tests pass; 4/4 web smoke tests pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Third Playwright case: click dead-centre on the framed sample, assert the
canvas changes (selection highlight rendered) with zero WebGPU errors. A
broken async pick would hang init or leave the canvas unchanged. All three
cases pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Desktop pick reads the pick staging buffer back with a blocking
`while(!done) waitTickInstance()` spin. On web that spin is a no-op
(Asyncify is off) and hangs the JS event loop, so click-to-select was
dead on web. This adds an async sibling that uses the spontaneous
map-callback pattern (AllowSpontaneous + the browser microtask loop)
already proven by the HiZ readback — no blocking.
- encodePickReadbackToStaging(x,y,want_normal): the pick-pass render +
copy-texel-to-staging, extracted from pickObjectAt verbatim and shared
by both readbacks (desktop sync path unchanged).
- pickObjectAtAsync(x,y,cb) [web]: encode, then map the staging buffer
with a spontaneous callback that delivers object_id to cb. One pick in
flight at a time (a pick issued mid-map is dropped → cb(0)).
- applyPickToSelection(id, add, remove): routes a pick result through the
selection state machine (replace / Shift-add / Ctrl-remove / empty-click
clear), mirroring the desktop ViewportWindow click semantics. selection_
marks dirty so the next render's uploadSelectionFlagsIfDirty flushes the
highlight.
main_web wires it: a left release under a 4px drag threshold (no orbit) is
a pick at down-position * devicePixelRatio, with Shift/Ctrl modifiers;
the result callback applies selection and requests a frame. Web + desktop
build clean; 107/107 unit tests pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds a second Playwright case that picks the sample sidecar through the
file input, waits for the C side to confirm the blob load (console),
then asserts an orbit drag changes the canvas with zero WebGPU errors.
This exercises the #88 path distinctly from the embedded MEMFS sample —
a broken metadata-head/tail or chunk range read renders blank and fails
the orbit-changed-canvas check. Both cases pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Picked files are no longer copied whole into the wasm heap. The browser
File object stays in JS (Module.__ifcvFile) and is read lazily through
Blob.slice byte ranges, so a 200-500 MB sidecar never enters wasm linear
memory — only chunk-sized slices do.
Mechanism (web-only, #if __EMSCRIPTEN__):
- JS glue (EM_JS): ifcvFileSize + ifcvReadRangeInto — slice [off,off+n)
of the File and copy it into a caller-provided heap pointer, then call
back _ifcv_on_range_done. No malloc across the boundary; C pre-sizes
the destination from the read plan.
- webReadRangesAsync: reuses planSidecarReadRanges to coalesce a range
set into Blob.slice reads (1 MB gap — each slice is an async hop),
scatters them into a destination laid out in input order, and fires a
continuation when the whole set lands. An in-flight map keyed by id
survives unordered_map rehash (scratch buffers are heap-owned).
- loadSidecarFromBlobWeb: async metadata load — head (16 B) -> index
count -> tail-to-EOF -> parseSidecarHead/Tail -> applyCachedModel, then
tags the model streaming_from_blob and frames it.
- driveStreamingLoads: blob-sourced models route to beginWebChunkLoad
(async vertex+index range reads -> applyStreamedChunk in the callback),
holding is_loading until the bytes arrive. The embedded MEMFS sample
keeps the synchronous fopen path.
shell.html stashes the File and calls _load_sidecar_from_blob_c instead of
FS.writeFile'ing the whole thing; EXPORTED_RUNTIME_METHODS=['FS'] dropped.
Desktop is untouched (the new members + driveStreamingLoads branch are all
emscripten-guarded). Web links clean; desktop rebuilds; 107/107 unit tests
pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Splits the v13 metadata wire-format knowledge out of the FILE*-bound
streaming reader into pure, buffer-based functions so the web byte-range
path (#88) can reuse it without loading the whole sidecar into the wasm
heap:
- parseSidecarHead — validates the 16-byte head, yields num_vertex_bytes
- parseSidecarTail — parses meshes/instances/georef/elements/strings
from an in-memory tail buffer, bounds-checked
- planSidecarReadRanges + SidecarReadPlan — the range-coalescing /
scatter planner, promoted out of the anonymous namespace
readSidecarMetadataOnly and the range readers now call these; desktop
behaviour is unchanged (head + tail are small, the bulk is still skipped
via seek). The metadata tail is split from the head around the bulk
sections, so a blob-backed loader just slices those two regions and
hands the bytes to the same parsers.
Closes a coverage gap: StreamingLoader had no unit tests. Adds
test_streaming_loader.cpp (7 cases: metadata round-trip, corrupt/truncated
rejection, vertex+index range scatter, head validation, tail truncation,
read-plan coalescing). 107/107 unit tests pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Drives the built web page in a real Chrome (channel:'chrome', so no
`playwright install`): waits for wgpu init, then asserts an orbit drag
changes the composited canvas — one check that simultaneously proves the
scene rendered, mouse input is wired, and the log overlay isn't eating
events — and that zero uncaptured WebGPU errors were logged. Every web
bring-up bug so far (blank render, error-buffer cascade, overlay
swallowing input) is this shape; this would have caught them.
serve.mjs statically serves build-web; the config launches headed
against the real GPU (--use-angle=vulkan + --ignore-gpu-blocklist are
load-bearing for a non-null adapter on Linux Chrome). node_modules and
results are gitignored. See README.md to run.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
On web we skip the desktop error-scope spin-wait (it blocks the JS event
loop and hangs the page). The old web addSubBuffer then judged success by
`buf != nullptr` — but Dawn-web returns a NON-NULL error buffer on OOM,
so the pool committed an invalid sub-buffer, alloc handed out slices in
it, and every chunk_bind_group built against it failed ("BindGroup is
invalid" spam + a wgpuQueueSubmit panic). Loading a model larger than the
browser's WebGPU budget triggered exactly this.
Add the grown sub-buffer as *provisional* (alloc and the capacity/free
tallies skip it) and validate it through a non-blocking AllowSpontaneous
PopErrorScope. resolveProvisionalGrowth() clears the flag when it's good,
or drops the sub-buffer and latches growth_disabled_ on a real OOM — at
which point the streaming evictor bounds the working set to what fits
instead of cascading. Only one provisional grow is in flight at a time
(growth_pending_). Desktop keeps its synchronous halve-retry path
unchanged. All 100 unit tests still pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The status overlay was position:fixed top/left/right with max-height
80vh and pointer-events:auto — a near-fullscreen div that both hid the
model and swallowed mouse events, so orbit drags over most of the canvas
did nothing. Move it to a small bottom-left box, set pointer-events:none
so it never intercepts navigation, and collapse it to a few dimmed lines
once the app goes live (errors re-expand it). It still auto-scrolls to
the newest line.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Thomas Krijnen