Glassvision/IfcOpenShell
1
0
Fork 0
mirror of https://github.com/IfcOpenShell/IfcOpenShell.git synced 2026-08-10 17:58:20 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs: rewrite viewport architecture page + add .ifcview format reference

Browse Source 
Rewrite the BonsaiViewer viewport architecture page to match the current
renderer (updated type names, streaming/sidecar flow). Add a dedicated
.ifcview sidecar format reference page and link it from the ifcopenshell
formats toctree, and polish the Bonsai intro copy.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Dion Moult
2026-07-03 19:26:45 +10:00
parent 66d558ec2d
commit a7f6aaa725
4 changed files with 777 additions and 243 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/bonsaiviewer/docs/viewport_architecture.rst
+402 -241
View File
@@ -1,277 +1,438 @@
Viewport architecture
=====================
This page documents the IFC viewer renderer that lives at
``src/ifcviewer/`` and underpins both BonsaiViewer and the standalone
``IfcViewerMinimal`` testbed.
This page documents the WebGPU IFC viewer renderer in ``src/ifcviewer``. The
same renderer is used by BonsaiViewer, IfcViewerMinimal, and the web build. It
explains the path from an IFC model or ``.ifcview`` cache to rendered triangles,
including the optimisation stages that decide what is loaded, what is resident
in GPU memory, and what is drawn.
Consumers
---------
Overview
--------
``src/ifcviewer/`` is a static library (``IfcViewer``) — it does not
ship as an executable on its own. Two consumers link against it:
* **BonsaiViewer** (``src/bonsaiviewer/``) — full project shell with
side panels, federation tree, settings dialog, connector picker,
ribbon. The end-user product.
* **IfcViewerMinimal** (``src/ifcviewer-minimal/``) — single-file
testbed that opens an IFC, a ``.ifcview`` sidecar, or a federation
``.ifcfed`` and shows just the viewport. Used for screenshot
regression tests, benchmarks, and isolating renderer-only issues
from the Bonsai Viewer shell.
Stack
-----
* **wgpu-native v29** for rendering, fetched as a pre-built binary in
``src/ifcviewer/CMakeLists.txt``. WebGPU API on top of Vulkan
(Linux/Windows) or Metal (macOS, via the ``MetalSurface_mac``
Cocoa bridge).
* **Qt6** for windowing (a raw ``QWindow``, not ``QOpenGLWidget``)
and the surface integration with the OS. Qt is also the source of
``QSettings`` for persisted user preferences (``AppSettings``),
the event loop driving render scheduling, and the timer/elapsed
primitives used in instrumentation.
* **IfcOpenShell C++ libs** (IfcParse / IfcGeom) for IFC parsing and
geometry generation, plus **IfcUtil** for the schema-agnostic
helpers (``Unit``, ``Geolocation``, ``Placement``).
* **Eigen3** for 4×4 matrices and small linear algebra.
* **meshoptimizer** at sidecar-build time only — decimates each
unique mesh into an LOD1 slice. Not pulled at runtime.
Five core ideas
---------------
The renderer is built around five decisions worth knowing about
before reading the code:
**1. Unique-mesh GPU instancing.** IFC scenes are dominated by
repeated geometry (identical doors, windows, studs, pipes, …). The
``IfcGeom::Iterator`` surfaces representation identity, so each
unique mesh is uploaded once into a per-model vertex buffer slice
and every placement becomes a tiny ``InstanceCpu`` record
(transform + ``object_id`` + optional colour override). On a 1 M-
placement BIM scene this collapses tens of millions of duplicate
vertices into a few hundred MB of unique meshes.
**2. Quantized 12-byte vertex format.** Each vertex is exactly 12
bytes:
The renderer is built around a small number of data transformations:
.. code-block:: text
offset 0 pos 3 × uint16 normalised → mix(mesh.aabb_min, mesh.aabb_max, t)
offset 6 normal 2 × int8 normalised → [-1,1]; octahedral-decoded
offset 8 color 4 × uint8 normalised → [0,1]; alpha is real (see "Transparency")
IFC / .ifcview input
-> SceneLoader
-> GeometryStreamer or sidecar metadata reader
-> SidecarData-shaped model metadata
-> ViewportCore::applyCachedModel()
-> per-model chunks, mesh table, instance table
-> cullModelCpuCompute()
-> visible_draws + prefix_sums
-> driveStreamingLoads()
-> applyStreamedChunk()
-> WebGPU draw() calls
-> WGSL expands visible draws into indexed triangles
The dequantisation basis is per-mesh, uploaded once in a
``MeshGpu`` SSBO at binding 2; the vertex shader reads it via the
mesh index that comes through the visible-draws table. ``int8``
normals carry ~1.4° worst-case angular error, invisible for the
overwhelmingly axis-aligned faces (walls, floors, slabs) BIM models
produce.
The important split is between **metadata** and **geometry bytes**. Metadata
describes meshes, instances, bounds, transforms, element ids, and chunk layout.
Geometry bytes are the quantised vertices and indices. A sidecar load reads
metadata first and leaves geometry chunks non-resident until the camera needs
them. A direct IFC load builds the same structures from the geometry iterator,
then uploads all chunks at finalisation.
**3. Streaming chunked storage backed by a probed VRAM pool.** Per-
model state isn't loaded all at once. The sidecar's vertex/index
data is split into spatially-coherent chunks (Morton-sorted at bake
time, packed greedily); ``ViewportWindow`` keeps each chunk's bytes
resident in a single GPU buffer pool whose size is determined at
startup by ``wgpuDevicePushErrorScope(OutOfMemory)`` probing. As
the camera moves, chunks page in and out via the
``StreamingThread`` background worker; the render thread never
issues a blocking disk read.
Main components
---------------
**4. Sidecar cache as the fast path.** Loading an IFC the first
time runs the ``IfcGeom::Iterator`` (expensive); the result is
serialised to a ``.ifcview`` file (``SIDECAR_VERSION = 13``) next
to the source. Subsequent opens go straight from disk to GPU
buffers — no iteration, no geometry-engine cost. The sidecar
format embeds chunked vertex sections with a byte-range table of
contents so the streaming thread can fetch any chunk by ``pread``
without scanning the file.
``SceneLoader``
Orchestrates model loading. It first tries to read ``.ifcview`` metadata
when sidecar reads are enabled. If that fails, it starts the geometry
streamer. It also keeps the ``ifcopenshell::file`` used by property and UI
code.
**5. Event-driven rendering.** No render timer. Frames are
scheduled only via ``QWindow::requestUpdate()`` when something
actually changes — camera motion, streaming chunk arrival, hover,
selection, settings edits. When the camera is idle, the cull pass
short-circuits and the main thread blocks in the Qt event loop;
the viewer costs zero CPU/GPU on a static scene.
``GeometryStreamer``
Runs ``IfcGeom::Iterator`` on a worker thread for raw IFC loads. It emits a
``StreamedMesh`` once for each unique representation mesh and a
``StreamedInstance`` for each placed occurrence.
Per-frame pipeline
------------------
``SidecarBuilder``
Optionally mirrors the streamer's output while a raw IFC is loading. It
writes a new ``.ifcview`` cache after the stream finishes, so the next open
can use the sidecar fast path.
Each call to ``ViewportWindow::render()`` runs the same six phases:
``ViewportWindow``
The Qt-facing window wrapper. It owns UI/window integration and forwards
renderer work to ``ViewportCore``.
``ViewportCore``
The renderer core. It owns the WebGPU device-facing model state, pipelines,
chunk residency, culling, streaming, picking, screenshot capture, and render
loop.
``ModelGpuData``
Per-model runtime state: mesh metadata, instances, chunks, GPU buffers,
per-chunk visibility scratch, object lookup maps, and CPU-side caches used
by tools.
``BufferPool``
Sub-allocates chunk vertex and index bytes from one or more WebGPU buffers.
Chunks enter and leave this pool as the streaming system pages geometry in
and out.
``StreamingThread`` and web range loading
Desktop sidecar streaming uses a background worker to read and decompress
chunk frames. The web build uses asynchronous byte-range reads from a
registered file or URL source.
Load path 1: sidecar hit
------------------------
The fast path starts when ``SceneLoader`` finds a readable ``.ifcview`` cache.
The reader validates the sidecar header, skips the compressed geometry section,
and reads the metadata blocks.
For desktop loading, ``readSidecarMetadataOnly()`` returns a
``StreamingSidecar`` containing:
- the sidecar file path
- the geometry section offset
- mesh records
- instance records
- georeferencing state
- the baked chunk table of contents
- element metadata
For web loading, ``ViewportCore::loadSidecarMetadataWeb()`` reads only the
header and critical metadata before creating the model. It records the location
of the deferred element metadata block and fetches that later only when UI code
needs it.
``ViewportCore::applyCachedModel()`` then converts sidecar metadata into
runtime model state:
1. It creates the chunk list. If the sidecar contains a baked
``SidecarChunk`` table, that table is authoritative. Each chunk is a
consecutive mesh range with compressed vertex and index frame locations. If
the model came from a direct in-memory load and has no table, the chunk plan
is derived from mesh centroids with Morton sorting and greedy packing.
2. It creates per-chunk scratch buffers: ``visible_draws``, ``prefix_sums``,
and a small uniform with draw counts.
3. It uploads the per-mesh quantisation table, ``MeshGpu``.
4. It uploads the per-instance GPU table, ``InstanceGpu``.
5. It builds CPU lookup tables: object id to instance, instance to chunk,
mesh-local offsets inside each chunk, chunk AABBs, and chunk instance lists.
6. It leaves the actual chunk vertex and index bytes non-resident. They are
fetched later by ``driveStreamingLoads()``.
At this point the viewer knows the model's structure and bounds, but a sidecar
model has not necessarily uploaded any triangles yet.
Load path 2: direct IFC stream
------------------------------
When no usable sidecar exists, ``SceneLoader`` starts ``GeometryStreamer``.
Streamer signals are queued from the worker thread into the UI/render thread:
``StreamedMesh``
A unique mesh in transfer form. Vertices are seven floats per vertex:
position, normal, and packed colour.
``StreamedInstance``
One placed occurrence of a mesh, with object id, colour override,
placement transform, and world AABB.
During the stream, ``ViewportCore::uploadStreamedMesh()`` and
``ViewportCore::uploadStreamedInstance()`` stage the data in a ``SidecarData``
shape. Mesh vertices are converted to the renderer's packed 12-byte format,
indices are stored, and instance records are accumulated. The viewport does not
incrementally draw these raw streamed mesh records one by one.
When the streamer finishes, ``SceneLoader::onStreamerFinished()`` calls
``ViewportCore::finalizeModel()``. Finalisation wraps the staged data in a
``StreamingSidecar`` object and calls the same ``applyCachedModel()`` path used
by sidecar loads. It then gathers the already-staged vertex and index bytes per
chunk and calls ``applyStreamedChunk()`` for each chunk. Direct IFC loads
therefore become resident after finalisation rather than streaming from disk.
If sidecar writes are enabled, ``SidecarBuilder`` finalises its mirrored copy,
builds LOD1 where available, reorders geometry into streaming chunk order, and
writes the ``.ifcview`` file for the next open.
Mesh and vertex representation
------------------------------
IFC geometry is represented as unique meshes plus instances. Repeated IFC
representations are uploaded once, and every placement becomes an instance that
references a mesh.
The GPU vertex format is 12 bytes per vertex:
.. code-block:: text
offset 0 position uint16[3] quantised against the mesh local AABB
offset 6 normal int8[2] octahedral encoded
offset 8 colour uint8[4] RGBA
The mesh's ``MeshGpu`` record stores ``aabb_min`` and ``aabb_max``. The vertex
shader reconstructs local positions by mapping the three 16-bit coordinates
back into that local AABB. The instance transform then moves the vertex to
world space.
Indices are ``uint32``. Each mesh has an LOD0 index range and may have an LOD1
index range. LOD1 reuses the same vertices with a smaller index list generated
at sidecar-build time.
Chunk layout and residency
--------------------------
Chunks are the unit of streaming, residency, culling priority, and rendering
bind groups. A chunk owns:
- a list of mesh ids
- a list of instance ids
- a world AABB
- compressed sidecar frame locations, for sidecar-backed models
- local vertex and index offsets for meshes inside the chunk
- GPU pool slices when resident
- cull output buffers
For sidecar-backed models, chunks start non-resident. They have metadata and
small cull buffers, but no vertex/index pool slices. A chunk becomes resident
when ``applyStreamedChunk()`` receives its decompressed vertex and index bytes.
``applyStreamedChunk()``:
1. Allocates vertex and index slices from ``BufferPool``.
2. Uploads chunk bytes with ``wgpuQueueWriteBuffer``.
3. Builds the chunk bind group.
4. Marks the chunk resident.
5. Scans alpha bytes so culling can route transparent meshes to the transparent
pass.
6. Computes mesh-local volumes and triangle CPU shadows used by measurement
tools.
``unloadChunk()`` releases the chunk's bind group and returns its pool slices
to ``BufferPool``. Metadata and cull scratch remain allocated so the chunk can
be loaded again later.
Per-frame render pipeline
-------------------------
Each frame in ``ViewportCore::render()`` follows this order:
.. code-block:: text
render():
1. fpsIntegrate() ← advance fly-mode camera by wall-clock dt
2. drainHizReadbacks() ← absorb HiZ readback completions
3. uploadSelectionFlagsIfDirty()
4. cullModelCpuCompute() ← parallel per-model frustum+contrib+HiZ+LOD
cullModelCpuUpload() ← writeBuffer visible-draws + prefix-sums
5. driveStreamingLoads() ← enqueue chunk fetches, apply completions
6. encode passes:
opaque main (depthWriteEnabled=True, no blend)
transparent main (depthWriteEnabled=True, SrcAlpha/InvSrcAlpha blend)
edge silhouette (samples depth buffer)
overlay (HUD, labels, gizmos, measurements, marquee)
wgpuSurfacePresent()
drain completed HiZ readbacks
upload selection flags if dirty
acquire surface texture
update frame uniforms
cull visible instances and upload visible draw tables
drive chunk streaming and eviction
encode opaque main pass
encode transparent main pass
encode in-pass overlays
encode edge silhouette pass
encode HiZ resolve if enabled
encode post-main overlays
submit command buffer
present surface
The two-pass alpha split (#6) routes each visible instance into
either the opaque half or the transparent half of the per-chunk
``visible_draws_scratch`` based on a per-mesh ``has_alpha`` flag +
the instance's ``color_override_rgba8`` alpha byte. The transparent
pass uses the same buffers, just with ``firstVertex`` offset to the
opaque half's end. ``Alt+X`` forces every instance through the
transparent pass via the ``xray_alpha_cap`` frame uniform, giving a
free X-ray mode.
Streaming runs after culling. That means a chunk that becomes resident during
``driveStreamingLoads()`` is usually first visible on a later frame. The
renderer requests a short burst of follow-up frames while streaming work is
settling so an event-driven render loop does not stop before newly resident
chunks are drawn.
Cull
~~~~
Culling
-------
CPU-side, runs parallel per model via ``std::async``. The cascade
per instance is:
Culling is CPU-side and model-parallel on desktop. The web build currently uses
the serial path because the WebAssembly build is not wired for pthread-backed
``std::async``.
1. **Chunk-level frustum cull** — each chunk's AABB is tested first;
off-frustum chunks skip every instance inside them in one shot.
2. **Per-instance frustum cull** — for chunks that pass.
3. **Contribution cull** — instances whose sphere projects below
``viewport/min_pixel_radius`` (stationary) or
``viewport/motion_min_pixel_radius`` (during motion) get dropped.
4. **HiZ occlusion** — projected AABB tested against the previous
frame's depth pyramid, when the VP matches. The pyramid is
captured at end-of-frame, downsampled, read back to CPU, and
queried with conservative all-fine-texels-agree semantics. Off
by default; ``WGPU_HIZ=1`` re-enables.
5. **LOD selection** — instances with sub-``viewport/lod1_pixel_threshold``
projected size and an available LOD1 slice route through the
LOD1 index range. Same vertex buffer, different ``firstIndex``.
``cullModelCpuCompute()`` is chunk-driven:
Survivors are appended to the chunk's ``visible_draws_scratch`` and
cumulative ``prefix_sums_scratch`` — the vertex shader uses a
binary search on prefix_sums to translate ``vertex_index`` into a
``(draw_index, vertex_in_draw)`` pair.
1. Reset each chunk's per-frame scratch and counters.
2. Frustum-test the chunk AABB. If the chunk is outside the frustum, every
instance inside it is skipped.
3. For frustum-passing chunks, process each instance in the chunk.
4. Skip hidden object ids.
5. Frustum-test the instance AABB.
6. Estimate projected pixel radius and projected AABB area.
7. Accumulate projected AABB area into the chunk's streaming priority.
8. Apply contribution culling. Instances below the current pixel-radius
threshold are dropped.
9. Mark the chunk contribution-visible. This is the signal that
``driveStreamingLoads()`` uses to decide whether a non-resident chunk is
worth fetching.
10. Optionally apply HiZ occlusion using the previous frame's depth pyramid.
11. Choose LOD0 or LOD1 based on projected size and LOD1 availability.
12. Classify the draw as opaque or transparent.
13. Append a ``VisibleDrawGpu`` record and extend the chunk prefix sum.
Streaming
~~~~~~~~~
The contribution threshold is higher while the camera is moving when
``viewport/motion_min_pixel_radius`` exceeds the still threshold. This drops
more sub-pixel work during navigation and reduces cull and streaming pressure.
The buffer pool (``BufferPool``) is a free-list sub-allocator over
one or more wgpu buffers, sized at startup by probing for
``WGPUErrorType_OutOfMemory`` on a series of growing allocations.
``StreamingThread`` is one worker thread that owns disk I/O:
``driveStreamingLoads()`` posts requests for the highest-priority
non-resident chunks and drains completions. The pool's eviction
policy is "evict the lowest-priority resident chunk whose priority
is below ``candidate_priority / EVICT_PRIORITY_RATIO``" — strict
enough to prevent thrash, lax enough that a single panning frame
doesn't refuse the move.
HiZ occlusion is optional. When enabled, the renderer uses a depth pyramid from
a previous compatible view-projection matrix. Contribution culling runs before
HiZ because it is much cheaper and reduces the number of HiZ tests.
Priorities come from a per-chunk screen-space score computed during
cull (chunk AABB projected to a screen-rect area). Spatial sort at
bake time means screen-adjacent chunks are byte-adjacent on disk,
so the typical fetch is a single ``pread`` range, not scatter-
gather.
Visible draw tables
-------------------
Overlay / picking / section cuts
--------------------------------
Culling does not issue draw calls directly. It writes compact per-chunk tables:
* ``OverlayRenderer`` runs after the main passes. Bundles the HUD
text, world-anchored labels (measurement readouts), the
marquee-select rect, the corner axis gizmo, the orbit pivot
indicator, and section-plane outline rendering. Has its own
pipelines with the usual ``SrcAlpha/OneMinusSrcAlpha`` blend.
* **Picking** is a second render pass with a dedicated pipeline
writing object IDs into an ``R32UInt`` framebuffer. Clicking
triggers a ``wgpuQueueOnSubmittedWorkDone`` + ``mapAsync``
readback of one pixel. No CPU-side raycasting.
* **Section cuts** (the ``K`` tool) push up to six clipping planes
into the frame uniform; ``is_section_clipped()`` in WGSL discards
fragments on the positive side. The plane gizmo and visible
cross-section outline come from OverlayRenderer.
``visible_draws``
One record per visible instance draw. It stores mesh id, instance index,
first index, and base vertex.
Federation
----------
``prefix_sums``
A monotonic array that maps a flat vertex id inside one chunk draw call to
a visible draw record.
``Federation`` (in ``IfcViewer``, not Bonsai) is the multi-model
data model. Each model in a federation has a fed_id, optional
group membership, an explicit ``ModelTransformation``, a
``CoordinateOperation`` (from the IFC's IfcMapConversion), and a
visibility flag. The viewport composes a single
``transform_meters`` per instance as:
``per_chunk_uniform``
Four counters: total visible draws, total visible vertices, opaque visible
vertices, and opaque visible draws.
After culling, ``cullModelCpuUpload()`` uploads those arrays to the chunk's GPU
buffers. Chunks with zero visible draws have their uniform zeroed and are
skipped by the render pass.
Triangle rendering
------------------
The main render uses one WebGPU ``draw()`` per resident visible chunk, not one
draw per IFC element. The CPU has already compacted the visible instances into
the chunk's ``visible_draws`` table.
The vertex shader receives a flat ``vertex_index`` from the draw call. It:
1. Binary-searches ``prefix_sums`` to find the visible draw record.
2. Computes the local vertex within that draw.
3. Reads the mesh-local index from the chunk index buffer.
4. Adds the draw's base vertex to get the packed vertex record.
5. Decodes quantised position, octahedral normal, and colour.
6. Reads the instance transform and object id.
7. Transforms the local vertex to world and clip space.
The fragment shader applies lighting, selection/active-object highlighting,
x-ray alpha cap, section clipping, and transparency output. Section clipping is
also used by picking so selected section planes match what the user sees.
Opaque and transparent passes
-----------------------------
Culling partitions visible draws into opaque and transparent halves. Opaque
draws are written first in each chunk's arrays; transparent draws are appended
after them.
The main pass then renders:
1. Opaque pipeline with depth writes and no blending, drawing
``opaque_visible_vertices``.
2. Transparent pipeline with blending, drawing the remaining vertices starting
at ``opaque_visible_vertices``.
Transparency is chosen from the baked mesh alpha flag, an instance colour
override alpha, or forced x-ray mode. The transparent pass uses the same chunk
bind group and buffers as the opaque pass.
Streaming and eviction
----------------------
``driveStreamingLoads()`` decides which non-resident sidecar chunks should be
loaded and which resident chunks may be evicted.
Each frame it:
1. Drains completed desktop worker reads and applies successful chunks.
2. Updates resident chunk visibility history.
3. Builds candidates from non-resident chunks that passed contribution
visibility in the current cull.
4. Sorts candidates by current projected-area priority.
5. Loads up to a fixed number of chunks per frame.
6. Ensures the buffer pool can fit the chunk before issuing the load.
7. Evicts lower-value resident chunks when the pool is full.
The first eviction pass drops the least-recently-visible resident chunk that
was not visible this frame. If every resident chunk is currently visible, a
second pass may evict the lowest-priority resident chunk, but only when the
candidate has meaningfully higher priority. This hysteresis prevents simple
swap loops while still allowing a saturated pool to follow the camera.
Desktop sidecar chunks are read by ``StreamingThread``. Each request contains
the sidecar path, geometry section offset, compressed vertex frame location,
and compressed index frame location. The worker reads and decompresses those
frames, then ``driveStreamingLoads()`` applies the result on the render thread.
Web sidecar chunks are fetched asynchronously by byte range. Vertex and index
frames are requested separately and joined before decompression and upload.
The web path caps concurrent chunk downloads so the highest-priority chunks can
arrive and render progressively instead of sharing bandwidth across the entire
visible set.
Sidecar format relationship
---------------------------
The ``.ifcview`` file exists to feed this renderer. Its critical metadata block
contains enough data to build ``ModelGpuData`` without reading geometry:
- mesh table
- instance table
- georeferencing cache
- chunk table of contents
The geometry section contains one compressed vertex frame and one compressed
index frame per chunk. The chunk table lets the streaming system jump directly
to the frames for a visible chunk without scanning the file or reading
unrelated geometry.
The sidecar version documented by the current code is version 16.
Picking, overlays, and tools
----------------------------
Picking is a GPU pass, not CPU raycasting. The pick pipeline uses the same
``visible_draws`` and chunk buffers as the main pass and writes object ids to
an integer texture. A click reads back the selected pixel. The section tool also
uses a normal and world-position pick target so it can create planes from the
actual rendered surface.
Overlay rendering is host-provided. ``ViewportCore`` encodes the shared section
gizmo and calls host hooks for in-pass and post-main overlays. The Qt host
forwards those hooks to the desktop overlay renderer; the web host can no-op or
provide its own implementation.
Edges are drawn after the main pass using the depth buffer. HiZ resolve, when
enabled, also happens after the main pass so a later frame can use the captured
depth pyramid for occlusion culling.
Federation and transforms
-------------------------
Federated models compose several transforms before upload:
.. code-block:: text
federated_false_origin · model_transformation · coord_op · placement
federated_false_origin
* model_transformation
* coordinate_operation
* placement_transformation
``federated_false_origin`` is shared across all models in the
session and cancels the bulk of the big surveyor coordinates BIM
files carry, so float32 vertex math stays well within precision.
The first-added model's first geometry point auto-seeds the false
origin via ``ViewportView::guessFederatedFalseOriginFromFirstModel``
when the add-model command arms the guess; see that function for
why the arm/consume mechanism exists (it replaced a stack-
overflowing slot-emit-in-slot recursion).
The sidecar stores double-precision placement transforms so large IFC
coordinates can be combined with coordinate operation and false-origin matrices
before being narrowed to the float transform used by the GPU. This protects
rendering precision for survey-coordinate models.
Files map
---------
What to remember
----------------
Render core (the wgpu side):
* ``ViewportWindow.{h,cpp}`` — the main render window. Owns the
wgpu surface, device, queue, all pipelines, the per-model
``ModelGpuData``, the cull pass, the frame uniforms. ~7500 lines;
the bulk of the renderer.
* ``ModelGpuData.h`` — per-model GPU state (chunks, instances,
meshes, pool slices, alpha flags, scratch buffers used by cull).
* ``InstancedGeometry.h`` — wire structs shared by the streamer
and the viewport (``MeshInfo``, ``InstanceCpu``, ``InstanceGpu``,
``MeshChunk``, ``InstanceChunk``, vertex layout constants).
* ``VertexQuantization.h`` — pack/unpack helpers for the 12-byte
vertex format.
* ``BufferPool.{h,cpp}`` — VRAM sub-allocator with multi-sub-buffer
growth.
* ``StreamingThread.{h,cpp}`` — background ``pread`` worker.
* ``StreamingLoader.{h,cpp}`` — chunk-priority planner driving the
thread; owns the eviction policy and the residency map.
* ``OverlayRenderer.{h,cpp}`` — HUD / labels / gizmos / section-cut
outline / marquee. Separate pipelines + WGSL shader strings.
* ``MetalSurface_mac.{h,mm}`` — Cocoa bridge for the CAMetalLayer
surface attach on macOS. Compiled only on Apple via the
``OBJCXX`` language CMake enables.
* ``SelectionState.h``, ``VisibilityState.h`` — pure CPU state
machines for the selection set and the hidden-objects set;
header-only; covered by ``ifcviewer/tests/test_selection.cpp``
and ``test_visibility.cpp``.
* ``AreaMeasurement.{h,cpp}``, ``LengthMeasurement.{h,cpp}`` — the
measurement tools (triangle-area accumulation; world-space
polyline distance).
IFC ingestion and on-disk format:
* ``GeometryStreamer.{h,cpp}`` — wraps ``IfcGeom::Iterator`` on a
background thread; emits ``MeshChunk`` (unique geometry) and
``InstanceChunk`` (one per placement) to ``SceneLoader``.
* ``SceneLoader.{h,cpp}`` — orchestrates load. Owns the per-model
``ifcopenshell::file`` for property lookup; detects sidecar
presence; serialises queue of pending loads.
* ``SidecarCache.{h,cpp}`` — binary read/write of ``.ifcview``
files. Magic ``IFVW``, version 13 (see the per-version comment
block at the top of the header for the schema evolution).
* ``SidecarBuilder.{h,cpp}`` — live-load sidecar writer. Accumulates
the data as the streamer emits it, writes the sidecar after
``finalizeModel``.
* ``LodBuilder.{h,cpp}`` — meshoptimizer wrapper that builds each
mesh's LOD1 index slice at sidecar-bake time. Gated behind
``WITH_MESH_OPTIMIZER``; otherwise the LOD1 slot is left empty
and the renderer always uses LOD0.
Federation + settings:
* ``Federation.{h,cpp}`` — multi-model session: groups, model
transforms, coordinate operations, the federated false origin.
Persisted as ``.ifcfed``. Covered by
``ifcviewer/tests/test_federation.cpp``.
* ``AppSettings.{h,cpp}`` — ``QSettings``-backed preferences:
``viewport/min_pixel_radius``, ``viewport/lod1_pixel_threshold``,
``viewport/hiz_enabled``, ``viewport/nav_preset``, etc. See
:doc:`env-vars` for the full key list.
- Raw IFC streaming produces mesh and instance chunks, but the WebGPU viewport
does not draw each emitted mesh immediately. It stages them, finalises a model,
then uses the same chunk path as sidecar loads.
- Sidecar loads create a renderable model from metadata first. Geometry becomes
visible only as chunks become resident.
- Culling produces compact visible draw tables per chunk.
- The renderer draws one flat vertex stream per visible chunk. The shader
expands that stream back into indexed, instanced triangles using
``visible_draws`` and ``prefix_sums``.
- Streaming priority comes from the current view. Chunks are fetched only when
their instances are large enough on screen to be worth drawing.
- The buffer pool is finite, so residency is dynamic. The renderer keeps the
chunks that matter most for the current view and evicts lower-value chunks
under memory pressure.
src/ifcopenshell-python/docs/bonsai.rst
+4 -2
View File
@@ -1,5 +1,7 @@
Bonsai
======
Bonsai lets you analyse, create, and modify OpenBIM with Blender. For more
information, visit the `Bonsai website <https://bonsaibim.org>`_.
Bonsai is a native IFC authoring platform to analyse, create, and modify
OpenBIM with Blender.
For more information, visit the `Bonsai website <https://bonsaibim.org>`_.
src/ifcopenshell-python/docs/ifcopenshell/formats.rst
+5
View File
@@ -101,6 +101,11 @@ without repeating tessellation, packing, quantisation, and instance rebuilds.
It is not a general interchange format or a full semantic storage backend. For
model data, pair it with the original IFC file or with RocksDB.
.. toctree::
:maxdepth: 1
ifcview_format
RDBVIEW
-------
src/ifcopenshell-python/docs/ifcopenshell/ifcview_format.rst
+366
View File
@@ -0,0 +1,366 @@
IfcView format
==============
``.ifcview`` is a lossy binary geometry cache used by the Bonsai Viewer and
IfcViewer. It is a geometry cache, not an IFC exchange format. It stores
quantised, compressed, spatially sorted streamable chunks of tessellated
geometry, instancing, LODs, georeferencing, and basic element identifiers.
The format is optimised for three requirements:
- fast load and reuse of model geometry for analysis or visualisation
- direct upload to a viewer's GPU-facing mesh and instance layout
- compressed chunked streaming, especially for web loading by byte range
It intentionally does not store non-geometric IFC data or original IFC
parametric geometry. It is recommended to be combined with the original
``.ifc`` file, a ``.ifcdb``/``.rdb`` model store, or an ``.rdbview`` package
when additional data is needed.
The format writes native C++ structs directly for several tables. This makes it
compact and cheap to load, but it also means that ``.ifcview`` should not be
treated as a stable, language-neutral interchange specification.
The source of truth for the format is the C++ sidecar implementation in
``src/ifcviewer``. The current on-disk version is version 17.
Top-level layout
----------------
.. csv-table::
:header: "Section", "Description"
"SidecarHeader", "Versioning and format metadata"
"uint64 geometry_section_size", "Number of bytes from start of geometry section to byte immediately before the geometry metadata block. May be used by streaming readers to skip bulk geometry and first fetch metadata."
"Geometry section", "Streamable compressed chunks of geometry"
"Geometry metadata block", "Mesh records, instance records, georeferencing data, and chunk frame offsets"
"Element metadata block", "Element IDs, GUIDs, names, and IFC classes"
Both metadata blocks use the same compressed block wrapper:
.. code-block:: text
uint64 compressed_size
uint64 raw_size
byte[compressed_size] zstd_frame
The geometry section is also compressed with zstd, but it is not one large
frame. Each streaming chunk has its own vertex frame and index frame so a
loader can fetch and decompress only the chunks it needs.
For web streaming, the loader would read:
1. the 20-byte head, consisting of the 12-byte header and 8-byte geometry size
2. the geometry metadata block header and compressed payload
3. the element metadata block header, so it can remember where the UI metadata
lives
After that, visible chunks are loaded asynchronously by byte range. The
element metadata payload is fetched only when UI code asks for element
metadata.
Sidecar header
--------------
``SidecarHeader`` is 12 bytes:
.. code-block:: c
struct SidecarHeader {
uint32 magic;
uint32 version;
uint32 endian;
};
The current values are:
.. list-table::
:header-rows: 1
* - Field
- Value
- Purpose
* - Magic
- ``0x49465657``
- Identifies an ``IFVW`` sidecar.
* - Version
- ``17``
- Selects the current layout version. Versions may not be compatible.
* - Endian marker
- ``0x01020304``
- Rejects files written with a different byte order.
If any of these values do not match, the ifcview cache must be rejected.
Geometry section
----------------
The geometry section is the streamable part of the file. It contains, for each
chunk, two zstd frames:
.. code-block:: text
chunk 0 vertex frame
chunk 0 index frame
chunk 1 vertex frame
chunk 1 index frame
...
The metadata does not discover these frames by scanning. Instead, the geometry
metadata block stores the ``SidecarChunk`` table that locates each chunk's
compressed vertex and index frames.
Inside a decompressed chunk, data is chunk-local:
.. code-block:: text
vertices for mesh first_mesh
vertices for mesh first_mesh + 1
...
LOD0 indices for mesh first_mesh
LOD0 indices for mesh first_mesh + 1
...
LOD1 indices for mesh first_mesh
LOD1 indices for mesh first_mesh + 1
...
This order matches the renderer's streamed chunk upload path. Vertex offsets
and index offsets in ``MeshInfo`` describe the whole-model logical layout, but
the chunk upload path computes chunk-local offsets when it builds runtime chunk
state.
Vertex format
-------------
Each stored vertex is 12 bytes:
.. list-table::
:header-rows: 1
* - Byte offset
- Type
- Meaning
* - ``0``
- ``uint16[3]``
- Quantised local position.
* - ``6``
- ``int8[2]``
- Octahedral-encoded normal.
* - ``8``
- ``uint8[4]``
- RGBA colour.
The position is quantised against the mesh's local axis-aligned bounding box,
stored in ``MeshInfo.local_aabb_min`` and ``MeshInfo.local_aabb_max``. This
bounding box is the quantisation basis. In other words, the format stores each
coordinate as a normalised integer within the mesh-local min/max range, rather
than storing the original float coordinate.
Conceptually, encoding does this per axis:
.. code-block:: text
t = (position - local_aabb_min) / (local_aabb_max - local_aabb_min)
stored = round(clamp(t, 0, 1) * 65535)
Decoding does the inverse:
.. code-block:: text
t = stored / 65535
position = mix(local_aabb_min, local_aabb_max, t)
The implementation stores ``extent_recip`` while encoding, which is simply
``1 / (local_aabb_max - local_aabb_min)`` for each axis. Degenerate axes use
``0`` so all coordinates on that axis quantise to the same value.
Normals use two signed bytes with octahedral encoding. This is less precise
than storing three floats, but is small and adequate for typical BIM geometry,
which is dominated by planar and axis-aligned surfaces. Colour is stored as the
four bytes used by the viewer's packed RGBA path.
Geometry metadata block
-----------------------
The geometry metadata block is the minimum metadata required to create the
runtime model and begin painting geometry. Its raw, decompressed order is:
.. code-block:: text
vector<MeshInfo> meshes
vector<InstanceInfo> instances
uint32 has_coordinate_operation
double[16] coordinate_operation_meters
double project_length_to_meters
double map_unit_to_meters
vector<SidecarChunk> chunks
``MeshInfo`` is 56 bytes and describes one reusable mesh:
.. list-table::
:header-rows: 1
* - Field
- Meaning
* - ``vbo_byte_offset``
- Byte offset of the mesh's vertices in the logical whole-model vertex buffer.
* - ``vertex_count``
- Number of 12-byte vertices.
* - ``ebo_byte_offset``
- Byte offset of the mesh's LOD0 indices in the logical index buffer.
* - ``index_count``
- Number of LOD0 ``uint32`` indices.
* - ``local_aabb_min`` / ``local_aabb_max``
- Mesh-local bounds and the quantisation basis for vertex positions.
* - ``first_instance`` / ``instance_count``
- Range of instances that reference this mesh after sidecar layout.
* - ``lod1_ebo_byte_offset`` / ``lod1_index_count``
- Optional decimated index range for LOD1. A count of ``0`` means LOD1 is unavailable.
``InstanceInfo`` is 232 bytes and records one placed occurrence of a mesh. The
important fields are:
.. list-table::
:header-rows: 1
* - Field
- Meaning
* - ``mesh_id``
- Index into the mesh table.
* - ``object_id``
- Viewer object identifier used for selection and lookup.
* - ``color_override_rgba8``
- Optional per-instance colour override. ``0`` means use the baked vertex colour.
* - ``model_id``
- Source model identifier within the viewer.
* - ``placement_transformation``
- Double-precision placement emitted by the geometry streamer, before final federation and false-origin composition.
* - ``transform``
- Float render transform for the default stage state. Loaders may recompute it from ``placement_transformation`` and current stage matrices.
* - ``world_aabb_min`` / ``world_aabb_max``
- World-space instance bounds used for chunk bounds, culling, and view fitting.
Both transform forms are stored for precision and reuse. The double placement
keeps large IFC coordinates intact until the viewer has applied coordinate
operation, model transformation, and false-origin matrices. The float transform
is the GPU-facing result for the default composition.
The georeferencing fields cache enough of the model's coordinate operation and
unit scale to load a sidecar without reparsing the IFC source solely to recover
map conversion state. If the source map conversion changes, the sidecar must be
deleted and rebuilt.
``SidecarChunk`` is 56 bytes and records one streamable range of meshes. Each
chunk maps to two compressed geometry frames in the geometry section:
.. list-table::
:header-rows: 1
* - Field
- Meaning
* - ``first_mesh``
- First mesh index covered by the chunk.
* - ``mesh_count``
- Number of consecutive meshes covered by the chunk.
* - ``v_comp_off`` / ``v_comp_size``
- Byte offset and compressed byte size of the chunk's vertex frame, relative to the start of the geometry section.
* - ``v_raw_size``
- Decompressed byte size of the chunk's vertex data.
* - ``i_comp_off`` / ``i_comp_size``
- Byte offset and compressed byte size of the chunk's index frame, relative to the start of the geometry section.
* - ``i_raw_size``
- Decompressed byte size of the chunk's index data.
The chunk table is part of the geometry metadata because the viewer needs it
before it can request geometry. On the web path, the loader reads the header
and geometry metadata block, creates a model with non-resident chunks, and then
starts fetching visible chunk frames by byte range.
Element metadata block
-----------------------
The element metadata block stores non-geometric element lookup data used by UI
features such as picking, tree display, object labels, and search.
Its raw, decompressed order is:
.. code-block:: text
vector<ElementTableRecord> elements
uint32 string_table_bytes
char[string_table_bytes] string_table
Each ``ElementTableRecord`` is a fixed-size 36-byte record:
.. code-block:: c
struct ElementTableRecord {
uint32 object_id;
uint32 model_id;
int32 ifc_id;
uint32 guid_offset;
uint32 guid_length;
uint32 name_offset;
uint32 name_length;
uint32 type_offset;
uint32 type_length;
};
Strings are stored once in the string table and referenced by offset and
length.
Splitting this block from geometry metadata is important for first paint. A
large model can have substantial names, GlobalIds, and type strings. The web
viewer can show geometry after the geometry metadata block is available, then
fetch this metadata later when the UI needs it.
Chunk planning
--------------
Chunks are spatial groups of meshes. During sidecar creation, the builder:
1. Computes a centroid for each mesh from the average of its instance
world-AABB centres.
2. Sorts mesh ids by a 3D Morton code, also known as Z-order.
3. Greedily packs the sorted meshes into 4 MiB vertex-byte chunks. A single
mesh larger than that limit is kept whole in an oversized chunk. Meshes are
not split.
4. Reorders meshes, vertices, indices, and instances so each chunk is a
consecutive mesh range.
5. Writes these offsets into the ``SidecarChunk`` table so that loaders can
directly jump to the compressed chunk as needed.
How the file is produced
------------------------
When no usable sidecar is available, the viewer falls back to the geometry
streamer. The streamer emits:
- ``StreamedMesh`` once for each unique representation mesh
- ``StreamedInstance`` for every placed occurrence of a mesh
- ``ElementInfo`` records for viewer metadata
The sidecar builder consumes those streams. Mesh chunks are converted from the
streamer transfer layout, which is seven floats per vertex
(``position.xyz``, ``normal.xyz``, packed colour), into the 12-byte quantised
vertex layout. Instance chunks become ``InstanceInfo`` records. Element info
records become ``ElementTableRecord`` plus string table entries.
At finalisation, the builder adds LOD1 index buffers where useful, caches
georeferencing state, lays out meshes in streaming chunk order, builds the chunk
table, and writes the file.
Binary conventions
------------------
The current writer uses these conventions:
- Multi-byte scalar values are written in native byte order and validated by
the endian marker.
- Counted vectors are written as ``uint32 count`` followed by
``count * sizeof(T)`` bytes of raw table entries.
- Metadata blocks are zstd-compressed as complete raw metadata buffers.
- Geometry chunks store vertex bytes and index bytes as separate zstd frames.
- Offsets stored in the chunk table are relative to the start of the geometry
section, not relative to the start of the file.