Glassvision/IfcOpenShell
1
0
Fork 0
mirror of https://github.com/IfcOpenShell/IfcOpenShell.git synced 2026-08-10 09:48:32 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs: split Autodesk connector docs into user + developer pages

Browse Source 
`autodesk.rst` had grown to mix end-user concerns (where do my tokens
live, how do I install the bundle, why isn't sign-in working) with
developer concerns (cargo build, fmt/clippy/test, packaging script
flow, per-OS toolchain notes, CI). Reorganise into:

* **`autodesk.rst`** — Autodesk Connector. User-facing. Bonsai-Viewer-
  level intro (Forma/APS/Docs, "Add from cloud"); install-from-zip
  per OS; first-run setup (client ID, OAuth port, browser redirect);
  where settings / cache / OAuth tokens live; proxy / TLS guidance
  for corporate installs.

* **`autodesk_development.rst`** — Autodesk Connector Development.
  Developer-facing. Tech stack (FLTK, ureq, keyring, dirs, serde,
  chrono, webbrowser); `cargo build --release`; `cargo test
  --all-features` / clippy / fmt-check; protocol probing via stdio
  pipe; packaging via `packaging/build.py`; per-OS build / keychain
  / codesign notes; CI workflow overview. Absorbs the entirety of
  the old `autodesk_packaging.rst`, which is removed.

`connectors/index.rst` toctree updated: `autodesk_packaging` →
`autodesk_development`. `cloud_sync_protocol.rst` untouched —
language-agnostic protocol spec.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
Dion Moult
2026-06-04 15:57:40 +10:00
parent 9d9f4054f6
commit ce7d2fa329
6 changed files with 279 additions and 1257 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/bonsaiviewer/docs/connectors/autodesk.rst
+92 -67
View File
@@ -1,107 +1,132 @@
Autodesk Connector
==================
The Autodesk connector integrates Bonsai Viewer with Autodesk Forma, APS, and
Docs. It implements the cloud connector protocol and runs as a separate process
that Bonsai Viewer launches and communicates with over standard input and
standard output.
The connector UI is built with CustomTkinter. The Python runtime used for
development or packaging must include ``tkinter``. On Gentoo, make sure
``dev-lang/python`` is built with ``USE="tk"``.
The Autodesk connector lets Bonsai Viewer open and save projects and models
hosted on Autodesk Forma, APS, and Docs.
Install
-------
From the repository root:
The connector ships pre-built next to the Bonsai Viewer executable. There
should be nothing for users to install.
If you do want to manually install a specific version, extract the zip into the
``connectors`` directory beside the Bonsai Viewer executable:
Linux:
.. code-block:: bash
cd src/bonsaiviewer-autodesk
python -m venv venv
source venv/bin/activate
pip install -e .
unzip autodesk-linux-x86_64.zip -d <application-dir>/connectors/
Run
---
macOS:
.. code-block:: bash
bonsaiviewer-autodesk
unzip autodesk-macos-arm64.zip -d "<application-dir>/connectors/"
The connector launches without configuration. On first run, open the settings
dialog to configure the Autodesk client ID and OAuth callback port.
Windows PowerShell:
For direct protocol testing, send newline-delimited JSON-RPC 2.0 requests on
standard input:
.. code-block:: powershell
.. code-block:: json
Expand-Archive autodesk-windows-x86_64.zip -DestinationPath "<application-dir>\connectors\"
{"jsonrpc":"2.0","id":"0","method":"open_settings"}
{"jsonrpc":"2.0","id":"1","method":"pull_ifcfed_interactive"}
{"jsonrpc":"2.0","id":"2","method":"pull_models","params":[{"display_name":"foo.ifc","id":"abc","source":{"connector":"autodesk","hub_id":"b.hub","project_id":"b.proj","item_id":"urn:adsk...","version_id":"latest"}}]}
{"jsonrpc":"2.0","id":"3","method":"push_ifcfed_interactive","params":{"path":"/tmp/project.ifcfed"}}
{"jsonrpc":"2.0","id":"4","method":"push_ifcfed","params":{"path":"/tmp/project.ifcfed","manifest":{"connector":"autodesk","hub_id":"b.hub","project_id":"b.proj","item_id":"urn:adsk..."}}}
Bonsai Viewer discovers the connector on next launch.
Bonsai Viewer is expected to launch this binary once per session and keep it
alive until shutdown. Closing the connector's standard input triggers a clean
exit.
First-run setup
---------------
Configuration
-------------
The first time you sign in, the connector needs an Autodesk APS client ID.
You provide one through the connector's settings dialog.
The connector reads the Autodesk client ID from ``settings.json`` in the
connector config directory, written by the settings dialog.
- **Client ID** — the APS application client ID you created in the Autodesk
developer portal. Used as the OAuth audience.
- **OAuth callback port** — defaults to ``8080``. The OAuth callback host is
always ``localhost``. Change this if ``8080`` is already in use on your
machine.
The OAuth callback host is always ``localhost``. The callback port defaults to
``8080`` and can be changed in the settings dialog.
When you sign in, the connector opens your default browser at the Autodesk
authorization page. After you approve, the redirect lands back at
``http://localhost:<port>/`` and the connector picks up the auth code, then
exchanges it for tokens via PKCE.
The config directory is platform-specific:
Where things are stored
-----------------------
The connector writes three things to disk: a small settings file, a cache
of resolved files, and OAuth tokens.
**Settings** (``settings.json``) — the client ID and OAuth port you set
above:
- Linux: ``~/.config/bonsaiviewer-autodesk/``
- macOS: ``~/Library/Application Support/bonsaiviewer-autodesk/``
- Windows: ``%APPDATA%\bonsaiviewer-autodesk\``
OAuth tokens are stored in the OS keychain, keyed by the client ID. Changing
the client ID starts a fresh sign-in session. The keychain backend is Secret
Service on Linux, Keychain on macOS, and Credential Manager on Windows.
Cache
-----
The connector owns its own cache. On Linux, resolved files live under:
**Cache** — every file you pull from Autodesk lands here under a content-
hashed directory:
.. code-block:: text
~/.cache/bonsaiviewer-autodesk/
~/.cache/bonsaiviewer-autodesk/ (Linux)
~/Library/Caches/bonsaiviewer-autodesk/ (macOS)
%LOCALAPPDATA%\bonsaiviewer-autodesk\Cache\ (Windows)
ifcfeds/<hash>/<name>.ifcfed[.manifest]
models/<hash>/<filename>
Each resolved file is the sole child in its directory so Bonsai Viewer can
write sidecar files, such as ``.ifcview``, next to it without colliding. A new
resolved model version lands in a fresh ``models/<hash>/`` directory. Old cache
directories may be removed manually to clear space.
write sidecar files (``.ifcview``, etc.) next to it without colliding. A
new version of a cloud model lands in a fresh ``models/<hash>/``
directory. Old cache directories can be deleted manually to reclaim disk
space; the connector will re-resolve them on next request.
Status
------
**OAuth tokens** — stored in your OS keychain, keyed by client ID:
Implemented:
- Linux: Secret Service (gnome-keyring, KWallet, …)
- macOS: Keychain
- Windows: Credential Manager
- strict JSON-RPC 2.0 host over stdio
- APS PKCE sign-in with keyring-backed token storage
- hub, project, and folder browsing
- ``pull_ifcfed_interactive``, ``pull_ifcfed``, ``pull_models``, and
``pull_models_interactive``
- ``push_ifcfed_interactive``, ``push_ifcfed``,
``push_model_interactive``, and ``push_model``
- ``open_settings`` for client ID, callback port, and sign-out
- connector-managed cache with one resolved file per directory
- adjacent ``.ifcfed.manifest`` files written and read alongside
``.ifcfed`` files
To fully sign out, use the connector's settings dialog "Sign out" button.
Token removal can also be done from the keychain UI directly if needed.
Not implemented:
Connecting through a proxy
--------------------------
- JSON-RPC notifications for progress streaming; the connector shows its own
progress dialog
- cancellation of in-flight downloads
- subdirectory upload layouts inside push destinations
The connector uses the host platform's native TLS trust store, so corporate
TLS-intercepting proxies that ship a custom CA work as long as that CA is
installed at the OS level. There is no ``cacerts.pem`` bundled inside the
connector that you would otherwise need to override.
HTTP/SOCKS proxy support is not exposed today. If you need to route through
a proxy, set ``HTTP_PROXY``/``HTTPS_PROXY`` in Bonsai Viewer's launch
environment and check ``ureq``'s upstream documentation for the supported
schemes.
Troubleshooting
---------------
**"Connector not found" on viewer launch.** Confirm the layout under
``<application-dir>/connectors/autodesk/``: it must contain
``connector.json`` and ``bonsaiviewer-autodesk[.exe]`` at the top level.
If you unzipped one level too deep you may have
``connectors/autodesk/autodesk/connector.json`` — flatten it.
**Sign-in browser tab loads, but the redirect never closes the loop.**
Check the OAuth callback port hasn't been changed under you (e.g. by
another local service binding ``8080``). Change the port in the settings
dialog and re-sign-in. Verify ``localhost`` resolves correctly — corporate
DNS occasionally rewrites it.
**Keychain prompt loop on Linux.** A Secret Service provider must be
running; gnome-keyring-daemon or KWallet. Headless servers without one of
those installed will fail token reads.
**Some pulls fail with HTTP 403.** Your APS client ID needs Autodesk Docs
and/or Forma scopes enabled on the developer portal. The connector requests
the union of scopes it knows about, but you must approve them on the
application side.
**Logs.** The connector writes diagnostic lines to its standard error
stream. Bonsai Viewer captures and surfaces these in the cloud-connector
status panel. The level of detail is fixed for now; structured logging is
a future improvement.
src/bonsaiviewer/docs/connectors/autodesk_development.rst
+186
View File
@@ -0,0 +1,186 @@
Autodesk Connector Development
==============================
This page describes building, testing, packaging, and protocol-level debugging
of the Autodesk connector for developers. For end-user install and
configuration see :doc:`autodesk`.
Tech stack
----------
The connector is a single Rust binary at ``src/bonsaiviewer-autodesk/`` in
the IfcOpenShell repository. Key crates:
- **fltk** (``fltk-bundled`` feature) — UI toolkit, statically linked at
build time, so the produced executable has no system Tcl/Tk, Qt, or GTK
dependency.
- **ureq** (``tls`` + ``native-certs``) — blocking HTTP client. The
``native-certs`` feature pulls the system trust store; no bundled CA file
or OpenSSL runtime.
- **keyring** (``apple-native`` + ``windows-native`` +
``sync-secret-service``) — OS keychain abstraction for OAuth token
storage.
- **dirs** — platform-specific config / cache / data directories.
- **serde** / ``serde_json`` — JSON-RPC framing and APS response parsing.
- **chrono** — token-expiry math.
- **webbrowser** — opens the system default browser for the OAuth redirect.
Build from source
-----------------
From the repository root:
.. code-block:: bash
cd src/bonsaiviewer-autodesk
cargo build --release
The binary lands at ``target/release/bonsaiviewer-autodesk``.
A debug build (``cargo build``) at ``target/debug/bonsaiviewer-autodesk``
is fine for iterating on UI or RPC logic; release-mode strip + LTO is what
the packaging script ships.
Run
---
.. code-block:: bash
./target/release/bonsaiviewer-autodesk
The connector launches without configuration. The first interaction should
be to open the settings dialog (either from Bonsai Viewer or via the
``open_settings`` JSON-RPC method below) and configure the APS client ID +
OAuth callback port.
Bonsai Viewer is expected to launch this binary once per session and keep
it alive until shutdown. Closing the connector's standard input triggers a
clean exit.
Test and lint
-------------
The Rust source tree ships unit tests, integration tests under ``tests/``,
and clippy-clean lints. From the connector directory:
.. code-block:: bash
cargo test --all-features
cargo clippy --all-targets --all-features -- -D warnings
cargo fmt --all -- --check
CI runs the same three commands in
``.github/workflows/build-bonsaiviewer-autodesk.yml``.
Protocol probing
----------------
The connector speaks newline-delimited JSON-RPC 2.0 over stdio. For
debugging without Bonsai Viewer in the loop, pipe requests in directly:
.. code-block:: bash
./target/release/bonsaiviewer-autodesk <<'EOF'
{"jsonrpc":"2.0","id":"0","method":"open_settings"}
{"jsonrpc":"2.0","id":"1","method":"pull_ifcfed_interactive"}
EOF
Useful methods at a glance:
.. code-block:: json
{"jsonrpc":"2.0","id":"0","method":"open_settings"}
{"jsonrpc":"2.0","id":"1","method":"pull_ifcfed_interactive"}
{"jsonrpc":"2.0","id":"2","method":"pull_models","params":[{"display_name":"foo.ifc","id":"abc","source":{"connector":"autodesk","hub_id":"b.hub","project_id":"b.proj","item_id":"urn:adsk...","version_id":"latest"}}]}
{"jsonrpc":"2.0","id":"3","method":"push_ifcfed_interactive","params":{"path":"/tmp/project.ifcfed"}}
{"jsonrpc":"2.0","id":"4","method":"push_ifcfed","params":{"path":"/tmp/project.ifcfed","manifest":{"connector":"autodesk","hub_id":"b.hub","project_id":"b.proj","item_id":"urn:adsk..."}}}
The cross-connector wire format is defined in :doc:`cloud_sync_protocol`.
Packaging
---------
The connector ships as a folder dropped into the Bonsai Viewer connectors
directory. The folder contains a single statically linked Rust executable
and a JSON manifest — no runtime interpreter, no vendored libraries
directory.
Cargo does not cross-compile out of the box (FLTK in particular wants the
host toolchain), so each operating system and architecture builds its own
package, typically through a CI matrix.
**Output:**
.. code-block:: text
dist/
autodesk/
connector.json
bonsaiviewer-autodesk[.exe]
autodesk-<os>-<arch>.zip
The ``autodesk/`` folder is what Bonsai Viewer expects under the
``connectors`` directory bundled next to the viewer executable
(``<application-dir>/connectors/``).
**Build:**
.. code-block:: bash
cd src/bonsaiviewer-autodesk
python packaging/build.py
``packaging/build.py`` uses only the Python standard library — no
``pip install`` step — and is a thin wrapper around ``cargo``. It:
1. cleans ``dist/``
2. runs ``cargo build --release``
3. copies ``connector.json`` and the produced binary into ``dist/autodesk/``
4. zips the folder as ``dist/autodesk-<os>-<arch>.zip``
Platform notes
~~~~~~~~~~~~~~
**Linux.** Build on the oldest glibc you intend to support — binaries
built on a newer glibc will not run on older distributions. Ubuntu 22.04
LTS (glibc 2.35) is a reasonable lowest common denominator in 2026. The
runtime keyring backend is Secret Service via the ``keyring`` crate's
``sync-secret-service`` feature (gnome-keyring, KWallet, …). End users
need a Secret Service provider running.
Expected output: ``autodesk-linux-x86_64.zip`` and/or an ARM64 archive.
**macOS.** Each architecture builds separately. To support Apple Silicon
and Intel, build on each architecture and ship two zips, or post-process
with ``lipo`` to produce universal binaries. The runtime keyring backend
is the system Keychain (``keyring`` crate's ``apple-native`` feature). For
distribution outside the developer's machine, codesign the executable and
notarize the bundle — unsigned binaries trigger Gatekeeper warnings.
Codesigning is left to the caller.
Expected output: ``autodesk-macos-arm64.zip`` and/or
``autodesk-macos-x86_64.zip``.
**Windows.** Build with the MSVC toolchain (Rustup default
``-x86_64-pc-windows-msvc`` target). The runtime keyring backend is
Credential Manager (``keyring`` crate's ``windows-native`` feature).
The Rust executable defaults to the Windows console subsystem, which is
appropriate because the connector speaks JSON-RPC over stdio. Bonsai
Viewer must launch it without showing a console window, for example with
Qt's ``QProcess::setCreateProcessArgumentsModifier`` and
``CREATE_NO_WINDOW``.
For distribution, sign the executable with an Authenticode certificate to
avoid SmartScreen warnings. Signing is left to the caller.
Expected output: ``autodesk-windows-x86_64.zip``.
CI
--
``.github/workflows/build-bonsaiviewer-autodesk.yml`` runs ``cargo fmt
--check``, ``cargo clippy -- -D warnings``, and
``cargo test --all-features``, then matrix-builds the four shipping
packages (Linux x86_64, macOS arm64, macOS x86_64, Windows x86_64) and
uploads each ``autodesk-<os>-<arch>.zip`` as an artefact.
src/bonsaiviewer/docs/connectors/autodesk_packaging.rst
-130
View File
@@ -1,130 +0,0 @@
Packaging the Autodesk Connector
================================
The Autodesk connector is shipped as a self-contained folder that can be
dropped into the Bonsai Viewer connectors directory. PyInstaller bundles the
Python interpreter, Qt, and dependencies so end users do not need Python
installed.
PyInstaller does not cross-compile. Each operating system must build its own
package, typically through a CI matrix.
Output
------
.. code-block:: text
dist/
autodesk/
connector.json
bonsaiviewer-autodesk[.exe]
_internal/...
autodesk-<os>-<arch>.zip
The ``autodesk/`` folder is what Bonsai Viewer expects under the
``connectors`` directory bundled next to the viewer executable
(``<application-dir>/connectors/``).
Build
-----
.. code-block:: bash
cd src/bonsaiviewer-autodesk
python -m venv venv
source venv/bin/activate
pip install -e ".[build]"
python packaging/build.py
On Windows, activate the virtual environment with
``venv\Scripts\activate``.
The build script:
1. cleans ``dist/`` and ``build/``
2. runs PyInstaller against ``packaging/bonsaiviewer-autodesk.spec``
3. renames the produced folder to ``autodesk/``
4. copies ``connector.json`` into it
5. zips the folder as ``autodesk-<os>-<arch>.zip``
The UI uses Tcl/Tk through CustomTkinter, which keeps the bundle small. Expect
roughly 50 MB unpacked or 21 MB zipped per OS. The Python used to build must
include ``tkinter``.
Linux
-----
Build on the oldest glibc you intend to support. Binaries built on a newer
glibc will not run on older distributions. Ubuntu 22.04 LTS, with glibc 2.35,
is a reasonable lowest common denominator in 2026.
The runtime keyring backend is Secret Service, such as gnome-keyring or
KWallet. End users need a Secret Service provider running.
Expected output: ``autodesk-linux-x86_64.zip`` and/or an ARM64 archive.
macOS
-----
Each architecture builds separately. To support Apple Silicon and Intel, build
on each architecture and ship two zips, or post-process with ``lipo`` to
produce universal binaries.
The runtime keyring backend is the system Keychain. For distribution outside
the developer's machine, codesign the executable and Tcl/Tk dylibs and notarize
the bundle. Unsigned binaries trigger Gatekeeper warnings. Codesigning is left
to the caller; the spec's ``codesign_identity`` field can be wired up.
Expected output: ``autodesk-macos-arm64.zip`` and/or
``autodesk-macos-x86_64.zip``.
Windows
-------
Build with the Microsoft Visual C++ runtime available, which is usually true
for modern Python distributions. The runtime keyring backend is Credential
Manager.
The executable is built with ``console=True`` because the connector speaks
JSON-RPC over stdio. Bonsai Viewer must launch it without showing a console
window, for example with Qt's ``QProcess::setCreateProcessArgumentsModifier``
and ``CREATE_NO_WINDOW``.
For distribution, sign the executable with an Authenticode certificate to avoid
SmartScreen warnings. Signing is left to the caller.
Expected output: ``autodesk-windows-x86_64.zip``.
Install a Built Connector
-------------------------
Extract the zip into the ``connectors`` directory next to the Bonsai Viewer
executable.
Linux:
.. code-block:: bash
unzip dist/autodesk-linux-x86_64.zip -d <application-dir>/connectors/
macOS:
.. code-block:: bash
unzip dist/autodesk-macos-arm64.zip -d "<application-dir>/connectors/"
Windows PowerShell:
.. code-block:: powershell
Expand-Archive dist\autodesk-windows-x86_64.zip -DestinationPath "<application-dir>\connectors\"
Bonsai Viewer discovers the connector on next launch.
Out of Scope
------------
- signing and notarization
- CI matrix setup
- auto-update
- universal macOS binaries via ``lipo``
src/bonsaiviewer/docs/connectors/index.rst
+1 -1
View File
@@ -11,4 +11,4 @@ returns the metadata needed to keep them connected to the cloud source.
cloud_sync_protocol
autodesk
autodesk_packaging
autodesk_development
src/ifcviewer/README.md
-992
View File
@@ -1,992 +0,0 @@
# IfcViewer
A high-performance native IFC viewer built on IfcOpenShell's C++ geometry
engine with a Qt6 interface and OpenGL 4.5 rendering.
## Architecture
```
+---------------------------------------------------+
| Qt6 Application (MainWindow) |
| +----------+ +----------------------------------+|
| | Element | | 3D Viewport ||
| | Tree | | (QWindow + OpenGL 4.5 Core) ||
| | (per- | | ||
| | model) | | Per-model: VAO/VBO/EBO ||
| +----------+ | instance SSBO ||
| | Property | | visible SSBO ||
| | Table | | indirect buffer ||
| +----------+ | glMultiDrawElementsIndirect ||
| | Status / Progress / Stats |
+---------------------------------------------------+
^ ^
| |
element metadata MeshChunk / InstanceChunk / Sidecar
| |
+---------------------------------------------------+
| GeometryStreamer (one per loaded model) |
| IfcGeom::Iterator with N threads |
| Dedups representations -> MeshChunk |
| Emits one InstanceChunk per placement |
+---------------------------------------------------+
```
### Key design decisions
- **QWindow viewport** embedded via `QWidget::createWindowContainer()`. Gives
us a raw native surface for OpenGL, bypassing `QOpenGLWidget`'s compositor
overhead.
- **GPU instancing as the central pillar.** IFC models are dominated by
repeated geometry — identical doors, windows, studs, pipes placed at
different transforms. IfcOpenShell's iterator surfaces representation
identity, so we upload each unique mesh exactly once and keep per-placement
data (transform, object id, optional colour override) in a separate SSBO.
For real projects this collapses tens of millions of triangles of duplicate
vertex data into a few hundred MB of unique meshes.
- **Per-model GPU buffers**: each loaded model gets its own
VAO/VBO/EBO/instance-SSBO/visible-SSBO/indirect-buffer. No cross-model
growth copies. Removing a model frees its GPU memory immediately.
- **Quantized local-coordinate vertex format (12 B):** position as
`u16x3` normalised against each mesh's local AABB, octahedral-encoded
normal as `i8x2`, packed RGBA8 colour. The normal fills what was
previously 2 bytes of padding, and shrinks from `i16x2` to `i8x2` —
int8 gives ~1.4° worst-case angular error, invisible for BIM geometry
which is overwhelmingly axis-aligned (walls, floors, slabs encode
exactly). Dequantisation basis is per mesh, uploaded once in a
`MeshGpu` SSBO at binding 2. The per-instance transform is applied in
the vertex shader. No world-baked vertex data. 12/28 = 57 % smaller
VBO than the original 28 B float layout (sidecar files shrink ~15 %
overall since indices/instances/metadata are unchanged).
- **Multi-draw indirect:** every frame the CPU builds a flat list of visible
instance indices and one `DrawElementsIndirectCommand` per non-empty mesh,
then issues a single `glMultiDrawElementsIndirect` per model. 50k visible
instances across 8k unique meshes collapse to one driver-side command
submission per model.
- **BVH frustum culling over instances**: per-model BVH trees cull whole
subtrees of placements with one frustum test. Falls back to a linear scan
during progressive upload and for very small models (< 32 instances).
- **Parallel per-model cull:** each model's CPU cull (frustum + contribution
+ HiZ + bucketing + indirect-command emit) is independent, so `render()`
fans them out via `std::async` and joins before the serial GL-upload
pass. On an 18-model scene this took wall-clock cull from ~25 ms to
~5 ms. The cull scratch buffers live on `ModelGpuData` so each worker
owns its output storage; phase-timer counters are atomic for the same
reason. `IFC_CULL_THREADS=0` forces single-threaded fallback.
- **Reflection-aware two-pass draw:** IFC placements can have negative-
determinant transforms (mirrored families). These flip the screen-space
winding of their triangles, which would make them vanish under
`GL_CULL_FACE`. The cull pass buckets visible instances into forward
(det ≥ 0) and reverse (det < 0) slices and the renderer issues two MDI
calls per model with `glFrontFace` toggled between them.
- **`reorient-shells` enabled in the iterator:** makes face winding
consistent within a shell at geometry-gen time — the only place this can
actually be fixed. Without it, files with inside-out faces produce dark
patches and swiss-cheese under backface culling. Costs iterator time but
is cached in the sidecar.
- **Progressive rendering during streaming:** the viewport is drawable
before `finalizeModel()`. Instances are pushed to the SSBO one at a time
via `glNamedBufferSubData` as they arrive, and the linear-scan cull path
handles them until the BVH is built. Orbit and pan remain interactive
through load.
- **Non-blocking sidecar loading**: sidecars are read on a background
thread; only the final GPU upload touches the main thread.
- **Event-driven rendering:** no continuous render timer. Frames are
scheduled via `QWindow::requestUpdate()` only when something changes
(camera move, streaming chunk, hover, settings). When the camera and
scene are idle the cull pass and HiZ readback are skipped entirely
and the main thread blocks in the Qt event loop — the viewer costs
zero CPU/GPU on a static scene. FPS is still reported accurately
because frame cost is measured *inside* `render()`, not as wall-clock
between frames.
- **GPU object picking**: a second render pass writes object IDs into an
R32UI framebuffer. Click reads back one pixel. No CPU-side raycasting.
- **Multi-model support**: multiple IFCs can be loaded simultaneously.
Each gets its own `GeometryStreamer` (which owns the `ifcopenshell::file`
for property lookup). Models load sequentially. Per-model
hide/show/remove.
### Files
| File | Purpose |
|------|---------|
| `main.cpp` | Application entry, GL 4.5 surface format, CLI argument parsing |
| `MainWindow.h/cpp` | Qt main window: multi-model project, element tree, properties, status |
| `ViewportWindow.h/cpp` | OpenGL 4.5 Core renderer: shaders, buffers, camera, culling, MDI draw, picking |
| `GeometryStreamer.h/cpp` | Background iterator runner; emits `MeshChunk` + `InstanceChunk` |
| `InstancedGeometry.h` | Shared structs: `MeshInfo`, `InstanceCpu`, `InstanceGpu`, chunk records |
| `BvhAccel.h/cpp` | Median-split BVH builder; operates on instance world-AABBs |
| `LodBuilder.h/cpp` | Post-stream decimation of unique meshes via meshoptimizer (`simplifySloppy`) |
| `SidecarCache.h/cpp` | Raw binary `.ifcview` (v9) sidecar read/write |
| `AppSettings.h/cpp` | Persisted preferences (geometry library, stats overlay, backface culling) |
| `SettingsWindow.h/cpp` | Settings dialog |
| `CMakeLists.txt` | Build configuration |
## Dependencies
- **Qt6** (Core, Gui, Widgets, OpenGL)
- **OpenGL 4.5** with `GL_ARB_direct_state_access` and
`GL_ARB_shader_draw_parameters` — available on Windows and Linux. macOS
will need a Vulkan/MoltenVK backend (not yet implemented; macOS caps out
at GL 4.1).
- **IfcOpenShell C++ libraries** (IfcParse, IfcGeom, and their
dependencies: Open CASCADE, Boost, Eigen3, optionally CGAL).
- **[meshoptimizer](https://github.com/zeux/meshoptimizer)** — linked via
`find_package(meshoptimizer REQUIRED)`. Used at sidecar-build time for LOD
decimation; not needed at runtime once a sidecar exists.
## Building
IfcViewer is part of the IfcOpenShell CMake project. From the repo root:
```sh
mkdir build && cd build
cmake ../cmake \
-DCMAKE_BUILD_TYPE=Release \
-DBUILD_BONSAIVIEWER=ON \
-DBUILD_CONVERT=OFF \
-DBUILD_IFCPYTHON=OFF \
-DBUILD_GEOMSERVER=OFF \
-DBUILD_DOCUMENTATION=OFF \
-DBUILD_EXAMPLES=OFF \
-DCOLLADA_SUPPORT=OFF \
-DGLTF_SUPPORT=OFF
make -j$(nproc) IfcViewer
```
If Qt6 is not in a standard location, pass `-DQT_DIR=/path/to/qt6`.
## Usage
```sh
./IfcViewer arch.ifc struct.ifc mep.ifc
./IfcViewer # then File -> Add Files
```
### Controls
| Input | Action |
|-------|--------|
| Middle mouse drag | Orbit camera |
| Shift + middle mouse drag | Pan camera |
| Scroll wheel | Zoom |
| Left click | Select object |
### Keyboard
| Key | Action |
|-----|--------|
| Ctrl+O | Add files |
| Ctrl+Q | Quit |
### Settings
- **Geometry Library** — kernel string passed to IfcOpenShell (default
`hybrid-cgal-simple-opencascade`).
- **Show Performance Stats** — overlay FPS / object / triangle / draw
counts in the status bar.
- **Backface Culling** — `GL_CULL_FACE` on closed solids. Default on.
Disable if a model uses open shells and you see missing faces.
## Performance Strategy
The viewer targets smooth orbiting at 60 fps on real-world multi-discipline
BIM projects (a "real job" being ~50 models, several million placements,
hundreds of millions of rasterised triangles when everything is in view).
Rendering performance has evolved in phases. Each builds on the previous,
and smaller models never pay for optimisations they don't need.
### Phase 1 — Per-object Frustum Culling
**Status:** implemented (and still the fallback for small models / during
streaming).
Six view-frustum planes are extracted from the view-projection matrix each
frame. Each instance's world AABB is tested with the p-vertex / n-vertex
method (one dot product + one compare per plane, 6 planes).
Surviving instance indices are written into a per-mesh bucket, then
flattened into a single `uint[]` (the "visible SSBO", binding = 1) and
accompanied by one `DrawElementsIndirectCommand` per non-empty mesh.
One `glMultiDrawElementsIndirect` call per model draws everything.
Cost: ~6 dot products per instance per frame. Fine up to ~100 k instances
per frame; above that the linear scan shows up in profiles, motivating
Phase 2.
### Phase 2 — BVH Acceleration + Sidecar Cache
**Status:** implemented.
For models exceeding ~32 instances, a bounding volume hierarchy groups
nearby placements into a binary tree and culls entire subtrees with a
single frustum test. This reduces per-frame work from O(N) to O(log N) in
the best case (camera zoomed to a corner) and remains well under 1 ms for
100 k instances in the worst case (everything on screen).
A BVH was chosen over an octree because BIM data is spatially non-uniform
— dense MEP risers in one zone, sparse open atria in another. An octree
subdivides space uniformly, wasting nodes on empty regions and creating
deep chains in dense ones. A BVH adapts its splits to the actual
placement distribution.
#### Activation
The BVH is optional and non-disruptive. Until it is built, the Phase 1
linear scan handles culling. The renderer checks for a BVH per model and
falls back to the scan for any model that doesn't have one.
It activates in one of two ways:
1. **Sidecar hit** — the `.ifcview` file next to the `.ifc` is found and
valid; its instance data is uploaded and the BVH rebuilt on the fly
from the restored AABBs (cheap — `< 100 ms` for 100 k placements).
2. **After streaming** — `finalizeModel()` builds the BVH synchronously
once all chunks are in (instances already live on the GPU, so there's
no EBO re-sort to do). The sidecar is written afterwards.
Models under 32 instances skip the BVH.
#### BVH node layout (32 B, two per cache line)
```cpp
struct BvhNode {
float aabb_min[3]; // 12 B
float aabb_max[3]; // 12 B
uint32_t right_or_first; // interior: right child index; leaf: first item index
uint16_t count; // 0 = interior, >0 = leaf
uint16_t axis; // 0/1/2 for interior; unused for leaf
};
```
Left child is always the next node (pre-order DFS). Leaf items are
indices into the per-model `instances` array; the parallel `bvh_items[]`
array carries the world AABBs.
#### Build: object-median split
1. Compute centroid of each item's AABB.
2. Pick the longest axis of the node's AABB.
3. `std::nth_element` partitions at the median on that axis — O(n).
4. Recurse until a leaf holds ≤ 8 items.
O(n log n) total. No SAH — for frustum culling (6-plane tests, early
subtree reject) the quality difference vs median is negligible.
#### Traversal: stack-based, no recursion
```
stack[64] = { 0 } // root
while stack not empty:
node = nodes[stack.pop()]
if node.aabb outside frustum: continue
if leaf:
for each item in node:
if item.aabb in frustum: emit to visible list
else:
push right child, push left child // left processed first (DFS)
```
Depth 64 is enough for billions of items on any balanced tree. The stack
is on the C++ stack, zero per-frame allocation.
#### Sidecar format (`.ifcview`, v9)
Raw memory dump, Blender-`.blend`-style — no serialisation, no parsing.
Stores everything needed to skip the `IfcGeom::Iterator` pass:
```
SidecarHeader (magic "IFVW", version, endian)
uint32_t + uint8_t[] vertex data (12 B/vert quantized; per-mesh basis in MeshInfo)
uint32_t + uint32_t[] index data (mesh-local)
uint32_t + MeshInfo[] per-unique-mesh metadata (56 B each, incl. LOD1 slice)
uint32_t + InstanceCpu[] per-placement records (transform + AABB + ids)
uint32_t + PackedElementInfo[] element tree records
uint32_t + char[] string table
```
Sidecar path is the source stem + `.ifcview` — `foo.ifc` and `foo.ifcdb/`
both map to `foo.ifcview`, so the same cache serves either source format.
Staleness is user-managed: delete the sidecar to force a rebuild.
Endianness marker rejects cross-arch caches.
Sidecars store the raw `object_id` / `model_id` values from the session
that wrote them. On load they are rebased onto the current session's ID
space (`object_id += next_object_id_ - min_id_in_sidecar`, `model_id`
overwritten with the freshly-assigned handle) before the elements hit
`element_map_` or the viewport. Without this, two cached models loaded
back-to-back collide — both start at `object_id=1` and the second model's
property lookups return the first model's data.
### GPU Instancing pipeline (the central pillar)
Everything above plugs into a single data-flow, worth documenting on its
own because it's what makes the whole thing fast.
Per-model state on the GPU:
| Buffer | Contents | Lifetime |
|--------|----------|----------|
| `VBO` | Quantized local-coord vertex data (12 B/vert: u16x3 pos, oct i8x2 normal, RGBA8). One range per unique representation. | Grow-on-demand during streaming; static after finalize. |
| `MeshGpu SSBO` (binding 2) | Per-mesh dequant basis (`vec4 aabb_min`, `vec4 aabb_max`). | Grow-on-demand; static after finalize. |
| `EBO` | Mesh-local uint32 indices. One range per unique representation. | Same. |
| `SSBO` (binding 0) | `InstanceGpu[]` (80 B each: mat4 transform, object_id, color_override, pad). | Appended during streaming, static after finalize. |
| `visible SSBO` (binding 1) | `uint32[]` — flat list of visible instance indices, ordered by mesh, uploaded each frame. | Rewritten every frame. |
| Draw-indirect buffer | `DrawElementsIndirectCommand[]` — one per non-empty mesh, uploaded each frame. | Rewritten every frame. |
Draw command:
```c
struct DrawElementsIndirectCommand {
uint32_t count; // mesh.index_count
uint32_t instanceCount; // visible-list length for this mesh
uint32_t firstIndex; // mesh.ebo_byte_offset / 4
uint32_t baseVertex; // mesh.vbo_byte_offset / 12
uint32_t baseInstance; // offset into the flat visible-index array
};
```
The vertex shader reads `visible[gl_BaseInstanceARB + gl_InstanceID]` to
get the real instance id, then indexes into the instance SSBO:
```glsl
uint slot = uint(gl_BaseInstanceARB) + uint(gl_InstanceID);
uint iid = visible[slot];
InstanceRecord inst = instances[iid];
gl_Position = u_view_projection * inst.transform * vec4(a_position, 1.0);
```
`gl_BaseInstanceARB` requires `GL_ARB_shader_draw_parameters`, which is
available on all GL-4.6-capable drivers.
Reflection handling: at upload time we store a parallel
`instance_reflected[]` byte array (1 if the transform's upper-3×3 has
det < 0). The cull pass produces two flat visible-list slices — fwd
(non-reflected) first, rev (reflected) after — concatenated into one
buffer. The renderer issues MDI twice: fwd with `glFrontFace(GL_CCW)`,
rev with `glFrontFace(GL_CW)`. `GL_CULL_FACE` stays on and does the
right thing in both passes.
### Current bottleneck — draw-bound, not upload-bound
The original README's Phase 3 ("GPU-driven indirect draw") described
moving draw submission to the GPU via compute. In the meantime, GPU
instancing and MDI made the CPU-side draw cost essentially free (10
`glMultiDrawElementsIndirect` calls per frame for 10 models). **That
goal is met.** The real ceiling lies elsewhere, and it took a couple of
bad hypotheses to pin down.
#### Profiled scene
10 models / 379 k instances / 128 M triangles, everything in view, no
camera motion, GTX 1650 (PCIe dGPU, 4 GB VRAM):
| Metric | Value |
|--------|-------|
| FPS | 6.7 |
| Frame time | 149 ms |
| gl_draws | 10 |
| Sub-draws packed in indirect buffers | 67 037 |
`nvidia-smi` reports 95 % GPU utilisation during render — the GPU is
the thing that's pinned.
#### False lead: "the per-frame uploads are the bottleneck"
The first round of probes pointed at the two `glNamedBufferSubData`
calls per model per frame (visible list ~1.5 MB + indirect buffer
~1.3 MB):
| Probe | Result | Initial interpretation |
|-------|--------|------------------------|
| Camera off-screen (nothing visible) | 60 fps | GPU idle → CPU path cheap |
| Comment out the two `glNamedBufferSubData` | 60 fps, blank screen | Uploads are the bottleneck |
This led to an aborted Phase 3A implementation of persistent-mapped
triple-buffered rings (and then staging + VRAM-resident with
`glCopyNamedBufferSubData`). Neither moved the FPS needle — both still
sat at 6.7 fps.
The probe was wrong: **commenting out the uploads emptied the indirect
buffer, so MDI drew zero triangles. "No upload" and "no draw" were
indistinguishable in the test.**
#### What actually isolates the draw cost
Two diagnostic env vars now live in `render()`:
- `IFC_SKIP_MDI=1` — keep everything (cull, upload, binds) but skip the
actual `glMultiDrawElementsIndirect` calls.
- `IFC_MAX_SUBDRAWS=N` — truncate each MDI's drawcount to N while still
running the rest of the frame.
Results on the profiled scene:
| Probe | FPS | Frame time |
|-------|-----|-----------|
| baseline | 6.7 | 149 ms |
| `IFC_SKIP_MDI=1` | 62.5 | 16 ms |
| `IFC_MAX_SUBDRAWS=30000` | 6.7 | 149 ms |
| `IFC_MAX_SUBDRAWS=10000` | 7.5 | 133 ms |
| `IFC_MAX_SUBDRAWS=1000` | 20.2 | 49 ms |
Readings:
1. `SKIP_MDI` gives 62 fps with all upload/bind machinery still running
— the non-draw path fits in ~16 ms easily. **Not upload-bound.**
2. Halving the sub-draw count (67 k → 30 k) saves 0 ms. If per-sub-draw
command-processor overhead were material, dropping 37 k sub-draws
would save measurable time no matter which sub-draws were dropped.
It doesn't. **67 k sub-draws is not the bottleneck** — the long tail
carries almost no triangles, and the heavyweights dominate.
3. Time only starts coming down once the cap is low enough to shed bulk
triangle work (1000 sub-draws → 49 ms). The curve is consistent with
a long-tailed distribution: a handful of very big meshes × instance
counts do most of the rasterisation.
**Conclusion: the GTX 1650 is rasterising 128 M triangles at ~850 M
tri/s, and that eats ~133 ms of the 149 ms frame.** No CPU-side or
upload-side work will recover it. The only way forward is to draw
fewer triangles.
### Phase 3 (revised) — Shed triangles, not bytes
In order of effort/payoff for BIM workloads:
#### 3A. Screen-space contribution culling — ✅ done
Reject frustum-visible objects whose bounding-sphere projects below a
pixel-radius threshold. Applied both at BVH-node level (whole subtrees
pruned, so distant parts of the model never touch per-instance tests)
and per-instance level. Short-circuits when the camera is inside the
AABB so nothing-you're-standing-next-to is ever lost. Pick pass uses
threshold 0 so sub-pixel objects remain clickable.
Because the pick pass re-runs the cull with its own parameters (no
contribution cull, no HiZ) and writes into each model's shared
`visible_ssbo` / indirect buffer, `pickObjectAt()` must invalidate
`have_cached_cull_` on exit. Otherwise the next `render()` sees an
unchanged camera, skips the cull, and draws the pick-pass buffers —
the user sees obviously-wrong shading until they nudge the camera.
Sphere-based (centre = AABB midpoint, radius = half-diagonal,
r_px = focal_px · radius / distance). Loses a little precision on
very elongated bounds vs. 8-corner projection, but costs ~5× less per
test, and because BVH-node pre-cull handles the long tail in one shot
it doesn't matter.
Threshold defaults to 2 px radius, overridable via `IFC_MIN_PX` env
var. Measured on the 10-model / 128 M-tri test scene (GTX 1650):
| Threshold | FPS | Triangles drawn | Objects drawn |
|-----------|-----|-----------------|---------------|
| 0 px (off) | 6.7 | 128 M | 379 k |
| 2 px | 20.2 | 40 M (31 %) | 89 k (24 %) |
| 4 px | 30.3 | 15 M (12 %) | 29 k (8 %) |
At 4 px, frame time breakdown matches: ~16 ms non-draw baseline (from
`IFC_SKIP_MDI=1`) + ~18 ms of raster (15 M tris / 850 M tri/s) ≈ 34 ms
= observed 33 ms. The ceiling is now genuinely vertex/raster
throughput on the post-cull geometry — next steps (LOD, HiZ) attack
that directly.
#### 3B. Distance / contribution LOD — ✅ done
Decimate each unique representation once (at sidecar-build time), store
the reduced index slice in the same EBO, and switch to it per-instance
per-frame whenever the projected sphere radius is small enough that the
reduced silhouette is indistinguishable from the original.
##### Pipeline
1. **After streaming finishes**, `MainWindow` calls `buildLods(sd)` on
the snapshotted `SidecarData`. Each eligible mesh's decimated index
list is appended to `sd.indices`; the per-mesh `MeshInfo` gains two
new fields:
```cpp
uint32_t lod1_ebo_byte_offset; // appended slice, same VBO
uint32_t lod1_index_count; // 0 = no LOD1 was built
```
`MeshInfo` grew from 48 to 56 bytes, which also bumps the sidecar
format to v5.
2. `viewport_->applyLodExtension(model_id, sd)` pushes the new index
suffix onto the live EBO via `glNamedBufferSubData` and replaces the
CPU-side `m.meshes` vector. The VBO and instance SSBO are untouched
— LOD1 reuses the same vertices, only the indices differ.
3. The sidecar is then written with both LOD0 and LOD1 indices baked in,
so subsequent loads of the same file pick up LOD1 for free.
##### Selection
The contribution-cull pass already computes each instance's projected
pixel radius. LOD1 is selected when that radius falls below
`IFC_LOD1_PX` (default 30 px) and the mesh has a non-empty LOD1 slice.
Camera-inside-AABB short-circuits select LOD0 (treated as "infinite
radius") so you never accidentally see the reduced mesh up close.
The visible-instance pipeline gains two more buckets (`fwd_lod1_`,
`rev_lod1_`), so the four-way split is now `{fwd, rev} × {LOD0, LOD1}`.
LOD0/LOD1 within a winding slice are contiguous — only winding requires
`glFrontFace` to flip between MDI calls, LOD does not. `firstIndex` /
`count` in the `DrawElementsIndirectCommand` pick which slice of the EBO
to walk; everything else (base vertex, base instance, SSBO bindings,
shader) is unchanged.
##### Decimator choice: `meshopt_simplifySloppy`
`meshopt_simplifySloppy` is a **voxel-clustering decimator** — it
quantises positions into cells and merges everything in a cell to a
single point. This is the only meshoptimizer decimator that works on
BIM brep output, which has per-triangle vertex duplication (hard-edge
normals) and non-manifold topology (T-junctions, coplanar slivers,
separate solids meeting at a plane). The edge-collapse decimator
(`meshopt_simplify`) needs 2-manifold edge pairs to score collapses;
on BIM geometry it returns the input unchanged.
`simplifySloppy` rounds off sharp corners and can produce slightly
degenerate triangles, so it doesn't look great at mid-screen size.
For a LOD1 that only activates below 30 px projected radius that's
invisible in practice.
##### Tuning knobs (env vars)
| Var | Default | Effect |
|-----|---------|--------|
| `IFC_LOD1_PX` | `30` | Projected sphere radius (px) below which LOD1 kicks in. `0` disables LOD1 entirely. |
| `IFC_LOD_ERROR` | `0.2` | Target relative error passed to meshopt. |
| `IFC_LOD_RATIO` | `0.25` | Target triangle-count ratio (LOD1 aims for 25 % of LOD0 tris). |
| `IFC_LOD_MIN_SAVINGS` | `0.25` | Reject the LOD1 result if it doesn't shave at least this fraction of triangles. |
| `IFC_LOD_DEBUG` | `0` | `1` prints per-mesh `tris / target / got / err` for the first 8 candidate meshes plus an accept/reject summary per model. |
##### Measured results
Same 10-model / 128 M-tri scene as Phase 3A (GTX 1650), 2 px contribution
threshold, overview camera, all models finalised with LOD1 built:
| Build | FPS | Frame time | Visible tris | Visible objs |
|-------|-----|-----------|--------------|--------------|
| Phase 3A alone (2 px) | 20.2 | 49 ms | 40 M | 89 k |
| Phase 3A + 3B (LOD1 ≤ 30 px) | **43.2** | **23 ms** | 14 M | 81 k |
Roughly half the remaining frame time, same object count (LOD is
lossless w.r.t. visibility — swapping index slice doesn't hide
anything). The triangle reduction on meshes that qualified for LOD1 is
~80 %: e.g. 4.17 M → 0.82 M tris for the 3618 eligible meshes of Model
1, 3.25 M → 0.65 M for Model 2, etc. Only about 20 % of unique meshes
qualify (the threshold is 500 tris — below that the indirect-command
overhead dominates), but those are the fat tail carrying most of the
rasterisation cost.
LOD build itself runs on the main thread inside `onStreamingFinished`;
typical cost is 100–600 ms per model, folded into the already-visible
"finalizing" step. Cached into the sidecar afterwards, so subsequent
opens skip it entirely.
#### 3C. Hierarchical-Z occlusion culling — ✅ done (v1, CPU-side)
Reject frustum-visible instances whose AABB is fully behind something
already drawn. The last drawn frame's depth buffer is the oracle — if a
region's deepest rasterised fragment is closer than an AABB's nearest
point, nothing in that AABB can win the depth test.
In dense BIM this matters most on interior views: standing inside a
building, 80–95 % of the model sits behind the walls of the current
room and contributes nothing to the frame. Phase 3A drops the
*distant-and-small* geometry, 3B drops its triangle count when kept,
and 3C drops the *close-and-big-but-hidden* bulk that neither of those
can touch. On an outdoor overview shot (nothing is occluded) 3C does
almost nothing — which is fine, 3A+3B already cover that case.
##### Pipeline (v1: CPU-side, 1-frame stale)
```
render():
draw main scene into MSAA default fb
axis gizmo
buildHizPyramid(): <-- new
glBlitFramebuffer MSAA depth → single-sample depth tex (256×128)
glReadPixels depth tex → CPU
max-reduce mip chain on CPU (8–9 levels)
store the VP that produced this frame
swapBuffers
cullAndUploadVisible():
per BVH node: frustum ∧ contribution ∧ hiz (subtree early-out)
per instance: frustum ∧ contribution ∧ hiz
```
The pyramid is always the *previous* frame's depth. On a newly loaded
scene or after a camera jump the cull is conservatively too permissive
for a frame or two (draws the occluded stuff by accident) and then
settles. No flicker because we never *wrongly reject* a visible
instance — the comparison is `aabb_near_depth > hiz_max`, so the
worst case is a kept instance that was actually occluded.
##### Why CPU-side?
Because the readback is cheap at this resolution (~128 KB / frame,
single glReadPixels ≈ 0.5 ms on PCIe) and the test itself is trivial
— ~100 k AABBs × 8 corners × a small mip lookup is well under a
millisecond on one thread. Phase 3D will port the cull to a compute
shader reading the pyramid as a texture, eliminating the readback; but
Phase 3C's CPU implementation was small enough to do first and
measure.
No MSAA complication on the write side: we just blit the default
framebuffer's multi-sample depth into a single-sample texture (GL
handles the resolve). No separate occluder pass either — we use the
previous completed frame's depth buffer directly, which is what a
temporal-reprojection HiZ reduces to when the "occluder set" is
"everything visible last frame".
##### The test
```cpp
project 8 AABB corners through hiz_vp → NDC rect + min z
if any corner has w ≤ 0: return false // crosses near plane
if rect is outside [-1, 1]²: return false
pick mip level where rect ≤ 2×2 texels
hiz_max = max(pyramid[mip][covered texels])
return aabb_near_depth > hiz_max
```
Comparing the AABB's *closest* point against the pyramid's *deepest*
value is the conservative direction — it only rejects when the AABB
is strictly beyond everything we already drew in that region. We pick
the mip at which the rect covers ≲ 2 texels on each axis so the lookup
is O(1) regardless of AABB size.
##### BVH integration
The same test runs on interior BVH node AABBs before leaf expansion,
so an occluded subtree skips all its instances in one shot. This is
where most of the per-frame cost savings show up on interior shots —
rejecting a 500-instance BVH subtree costs one 8-corner projection.
##### Tuning knobs
| Var | Default | Effect |
|-----|---------|--------|
| `IFC_NO_HIZ` | unset | `1` disables HiZ entirely (forces the Phase-3B-only path). |
| `IFC_HIZ_SIZE` | `256` | Base pyramid width in texels; height tracks viewport aspect. Raise for more accurate near-silhouette occlusion, lower to shrink readback. |
The stats overlay gains one counter, `hiz_rej`, showing how many
instances per frame the HiZ test rejected. On outdoor overview shots
it hovers near zero; on indoor shots it climbs into the hundreds of
thousands and the frame time drops accordingly.
##### Known caveats
- **Optional during camera motion (`IFC_HIZ_MOTION=1`).** The pyramid
is aligned to the previous frame's VP. On a moving camera the stale
depth can falsely occlude objects, particularly thin geometry (pipes,
railings) at oblique angles. By default HiZ is disabled during motion
(`hiz_vp_ == current_vp` check). Setting `IFC_HIZ_MOTION=1` forces
HiZ on during motion — benchmarks show this is the single biggest
perf lever (2.9× speedup), and the artifacts are transient and minor
during active orbiting. When the camera stops, a settle recull fires
with `hiz_vp_valid_ = false`, disabling HiZ for that one frame and
re-culling the full scene. This guarantees the stationary view is
artifact-free. See Phase 3G for benchmark data.
- **Conservative occlusion test.** The original "max over coarse mip"
test was too aggressive for BIM scenes where the entire depth range
compresses into 0.99–1.00. Replaced with "all fine-mip texels must
agree" — sample at mip 1, reject only if every texel has depth less
than the AABB's nearest point, early-out on the first non-occluding
texel. Queries covering >64 texels skip HiZ entirely. Eliminates
most false occlusions at the cost of fewer true rejections.
- **Depth blit replaced with shader downsample.** The original
`glBlitFramebuffer` for scaling the resolved depth to HiZ size
produced `GL_INVALID_VALUE` on some drivers. Replaced with a
fullscreen-triangle shader writing `gl_FragDepth`. The resolve
texture uses `GL_DEPTH24_STENCIL8` to match Qt's default FBO format
(which uses D24S8 even when only depth is requested).
- **Readback syncs the GPU.** `glGetTextureImage` is blocking.
Measured cost is well under a millisecond at 256×128; not a
bottleneck on the machines tested.
- **Transparent geometry would need special handling**, but the
current renderer doesn't have any, so no-op for now.
#### 3D. Parallel per-model cull (CPU, done)
A cheaper intermediate step before going full-GPU: each model's cull is
independent (no shared mutable state beyond atomic timing counters), so
`render()` fans the per-model culls out to a `std::async` pool and joins
before the serial GL-upload pass. On the 18-model / 569 k-instance test
scene this took the cull from ~25 ms wall-clock to ~5 ms — roughly a 4×
speedup on an 8-core machine, tracking `std::thread::hardware_concurrency()`
up to the model count. Load balancing is static (one job per model); a
single massive model still bottlenecks to single-threaded speed and would
need intra-model partitioning, but in practice BIM projects are
multi-discipline so the coarse partition lands well.
The stats line now reports `cull[wall X | work: clr Y trv Z emt W upl U]`:
`wall` is frame-time impact, the `work` numbers are per-thread sums showing
where CPU cycles went. `IFC_CULL_THREADS=0` forces single-threaded mode
for comparison.
#### 3E. GPU compute culling — experiments and removal
##### What we tried
Five iterations of GPU compute culling were explored on a 1.06 M-instance
/ 111-model scene (GTX 1650):
1. **Full GPU-driven rendering** — compute shader doing frustum +
contribution + HiZ + LOD + winding + indirect command building via
`glMultiDrawElementsIndirectCount`. Worked but introduced code smells
(extension loaders, ad-hoc readbacks). Reverted.
2. **GPU frustum-only validation** — minimal compute shader (64
threads/workgroup), 0.82 ms for 1.06 M instances vs 10–15 ms CPU.
Proved GPU brute-force beats CPU BVH for raw AABB-vs-frustum.
3. **Hybrid with synchronous readback** — added contribution culling,
read survivors back with `glGetNamedBufferSubData`. The 4–7 ms
pipeline stall negated all GPU savings.
4. **Async one-frame-late readback** — persistent-mapped buffer +
fence. Zero stalls, ~5.5 ms total vs ~5.5 ms CPU-only. Matched
but didn't beat.
5. **Dirty-mesh tracking** — reduced emit from O(total meshes) to
O(dirty meshes). Helped the consume phase but didn't change the
bottom line.
##### Why it was removed
Benchmark with motion-adaptive culling + HiZ active (Phase 3G):
| Path | FPS |
|------|-----|
| CPU BVH (parallel) | 51.0 |
| GPU cull + async readback | 52.0 |
The GPU cull added ~585 lines of code (compute shader, persistent-mapped
readback buffer, fence management, per-model AABB SSBOs, 8 profiling
counters, cleanup at 4 sites) for a 2% improvement that was within
measurement noise. With HiZ + motion culling reducing the visible set
to ~11 k objects, the CPU BVH path handles the work in ~2 ms — there's
nothing left for the GPU to win.
**Removed** in favour of keeping the codebase simple. The CPU BVH path
is now the only cull path.
##### Lessons learned
1. **GPU brute-force beats CPU BVH for raw frustum tests** (0.82 ms vs
10–15 ms for 1 M instances) but the advantage disappears once
higher-level culling (HiZ, contribution) reduces the working set.
2. **Synchronous readback kills GPU cull.** Persistent-mapped async
readback works but adds complexity for negligible gain.
3. **Hybrid GPU/CPU paths are maintenance-heavy** for diminishing
returns when the CPU path is already fast enough.
#### 3F. Sub-draw fragmentation analysis
##### The problem
With the culling pipeline mature (BVH + contribution + HiZ + motion
culling), the dominant cost shifts to the *drawing* side. On the 1.06 M-instance / 111-model
scene, frame times are 48–63 ms despite only 24–47 M visible
triangles — well within the GTX 1650's throughput. The culprit is
the number of indirect sub-draws (individual `DrawElementsIndirectCommand`
entries inside each `glMultiDrawElementsIndirect` call).
##### Measurement
Diagnostic instrumentation (`IFC_SUBDRAW_DIAG=1`) revealed:
**Mixed scene (111 models, 1.06 M instances):**
| instanceCount | sub_draws | % of total | instances | triangles |
|---------------|-----------|------------|-----------|-----------|
| 1 | 114,624 | **95.7%** | 114,624 | 16.9 M |
| 2 | 2,269 | 1.9% | 4,538 | 1.3 M |
| 3–4 | 1,127 | 0.9% | 3,873 | 1.6 M |
| 5–8 | 1,106 | 0.9% | 6,407 | 1.9 M |
| 9–16 | 376 | 0.3% | 4,315 | 0.8 M |
| 17–64 | 264 | 0.2% | 7,766 | 8.0 M |
| 65–256 | 29 | <0.1% | 3,331 | 2.0 M |
| 257+ | 8 | <0.1% | 9,732 | 0.4 M |
**Steel-only scene (18 models, 570 k instances):**
| instanceCount | sub_draws | % of total | instances | triangles |
|---------------|-----------|------------|-----------|-----------|
| 1 | 68,616 | **85.9%** | 68,616 | 12.5 M |
| 2 | 5,385 | 6.7% | 10,770 | 2.7 M |
| 3–4 | 2,581 | 3.2% | 9,100 | 1.3 M |
| 5+ | 3,324 | 4.2% | 66,407 | 7.0 M |
##### Consolidation potential
The mesh-level consolidation analysis found:
- **119,803 unique visible mesh IDs = 119,803 sub_draws** (perfect 1:1)
- **0 meshes split by winding or LOD buckets** — no mesh_id appears in
more than one (fwd/rev × lod0/lod1) bucket
- **0% reduction** available from merging across winding/LOD
- **114,624 meshes (95.7%)** are genuinely unique geometry placed
exactly once — instancing provides zero benefit for these
This is a fundamental property of the IFC data, not a pipeline
inefficiency. BIM models contain thousands of unique parametric
shapes (custom brackets, unique beam profiles, one-off fittings) each
placed at a single location. Only a minority of elements (standard
doors, windows, pipe fittings) share geometry across placements.
##### Conclusions
1. **Instancing is maxed out.** The pipeline already groups all
instances of each mesh into a single sub_draw. With 96% of meshes
having exactly one visible instance, there is nothing more to
group.
2. **Per-draw overhead dominates frame time.** 95–120 k sub_draws at
~20 fps = 48–50 ms/frame, but only 24–33 M triangles. A GTX 1650
can shade 1+ billion triangles/sec; the GPU is starving on
per-command overhead (command fetch, baseInstance lookup, draw
setup), not vertex/fragment throughput.
3. **The path forward is static batching.** Merge the vertex and
index data of multiple distinct single-instance meshes into
combined VBO/EBO ranges, each issued as one sub_draw. Batches of
256–1024 spatially-coherent meshes would collapse 91–115 k
sub_draws into 100–450, a 200–1000× reduction.
4. **Trade-offs of static batching:**
- Culling granularity degrades from per-mesh to per-batch. Batches
must be spatially coherent (e.g., BVH subtree leaves) or invisible
geometry gets drawn.
- Per-instance attributes (object_id, colour_override) must move
into the vertex stream or a per-vertex SSBO lookup, since
instancing no longer applies to merged meshes.
- The VBO/EBO layout changes at finalize time; existing instancing
stays for multi-instance meshes (the 4% that benefit from it).
- The sidecar format needs a version bump to cache batch membership.
5. **The steel scene validates the hypothesis.** It has better
instancing reuse (86% single-instance vs 96%) and correspondingly
better fps (49 vs 20). The ~2.5× fps ratio tracks the sub_draw
ratio (~80 k vs ~120 k), confirming per-draw overhead as the
dominant cost.
#### 3G. Motion-adaptive culling + HiZ during motion — ✅ done
The bottleneck during camera orbit is the sheer number of visible
objects and sub_draws. Two complementary strategies address this:
##### Motion-adaptive contribution culling (`IFC_MIN_PX_MOTION`)
During camera motion, use a larger pixel-radius threshold to hide
small objects that contribute little at interactive rates. When the
camera stops, a settle recull restores the base threshold and full
detail within one frame. No visual artifacts — objects below the
motion threshold are genuinely tiny on screen.
##### HiZ during motion (`IFC_HIZ_MOTION=1`)
Force the one-frame-stale HiZ pyramid to remain active during camera
motion. The stale depth causes minor false occlusions on thin
geometry at oblique angles, but these are transient during active
orbit. When the camera stops, the settle recull invalidates the HiZ
pyramid (`hiz_vp_valid_ = false`) and re-culls without HiZ,
guaranteeing the stationary view is artifact-free.
##### Benchmark results
Benchmarked on 1.06 M-instance / 111-model scene, 200-frame orbit
(103° arc, 0.5°/frame), GTX 1650:
| Configuration | avg ms | fps | speedup | obj | sub_draws | hiz_rej |
|----------------------------------|--------|------|---------|-------|-----------|---------|
| Baseline (no opts) | 61.25 | 16.3 | 1.0× | 254k | 155k | 0 |
| MIN_PX_MOTION=10 | 37.67 | 26.5 | 1.6× | 70k | 56k | 0 |
| HIZ_MOTION=1 | 21.44 | 46.6 | 2.9× | 33k | 17.5k | 28k |
| HIZ_MOTION=1 + MIN_PX_MOTION=10 | 19.62 | 51.0 | 3.1× | 11.4k | 8.7k | 11.5k |
##### Conclusions
1. **HiZ during motion is the biggest single lever** — 2.9× alone.
Artifacts are minor and transient during orbit; the stationary view
is guaranteed correct by the settle recull.
2. **Motion pixel culling is clean and effective** — 1.6× with zero
artifacts.
3. **Combining both gives diminishing returns** — 3.1× vs 2.9× (HiZ
alone) or 1.6× (MIN_PX alone). They compete over the same objects.
4. **The ~19 ms floor is GPU rendering**, not culling. At 8.6k
sub_draws the bottleneck shifts to draw dispatch + triangle
rasterization. Further improvement requires reducing sub_draws
(static batching) or moving to a more efficient draw model.
##### Benchmark CLI
Press **C** during interactive use to print the current camera as a
`--camera` argument. Then benchmark reproducibly:
```bash
./IfcViewer --camera tx,ty,tz,dist,yaw,pitch --benchmark 200 files...
```
The benchmark orbits the camera (0.5°/frame yaw), measures N frames
after a 5-frame warmup, prints avg/median/p1/p99 frame times, then
exits. Env vars control the test configuration.
### Planned follow-ups (post-Phase-3)
- **Mesh shaders / meshlets.** Ceiling-raising, but overkill until the
above are exhausted and we've hit silicon limits on vertex/raster
throughput.
## Summary table
```
Scene size Bottleneck Fix
----------- ---------- ---
< 100k instances CPU cull scan Phase 1 only
100k–500k CPU cull scan BVH (Phase 2) — done
500k+ tris / overview shot GPU vertex + raster Phase 3A contribution cull
+ Phase 3B LOD (done)
multi-million + occluders redundant rasterisation Phase 3C HiZ (done, CPU readback)
many models, serial cull single-thread BVH trv Phase 3D parallel cull (done)
orbit fps on 1M+ scenes too many vis objects Phase 3G motion culling + HiZ (done, 3.1×)
90k+ unique visible meshes per-draw GPU overhead Phase 3F static batching (next)
```
## Roadmap
- [x] Material colour support (per-vertex RGBA8)
- [x] Per-model GPU buffers (VAO/VBO/EBO per model, no cross-model copies)
- [x] Per-object frustum culling (Phase 1)
- [x] BVH acceleration with per-model trees (Phase 2)
- [x] Raw binary `.ifcview` sidecar cache
- [x] Non-blocking sidecar loading (background thread I/O)
- [x] Progressive GPU upload (VBO/EBO growth + streaming-time instance appends)
- [x] GPU instancing (unique meshes + per-placement SSBO)
- [x] `glMultiDrawElementsIndirect` draw path
- [x] Reflection-aware two-pass draw for mirrored placements
- [x] Backface culling (user-toggleable, default on)
- [x] `reorient-shells` enabled in iterator
- [x] Perf diagnostic env vars (`IFC_SKIP_MDI`, `IFC_MAX_SUBDRAWS`, `IFC_MIN_PX`, `IFC_LOD1_PX`, `IFC_NO_HIZ`, `IFC_HIZ_SIZE`, `IFC_CULL_THREADS`, `IFC_MIN_PX_MOTION`, `IFC_HIZ_MOTION`, `IFC_SUBDRAW_DIAG`)
- [x] Phase 3A — screen-space contribution culling
- [x] Phase 3B — distance / contribution LOD (meshoptimizer `simplifySloppy`)
- [x] Phase 3C — Hierarchical-Z occlusion culling (v1, CPU-side readback)
- [x] Phase 3D — Parallel per-model CPU cull (`std::async` fan-out)
- [x] Quantized VBO (12 B/vert: u16x3 pos + oct i8x2 normal + RGBA8, sidecar v7)
- [x] Event-driven rendering (zero idle CPU/GPU, cull skipped on still frames)
- [x] Phase 3E — GPU compute-shader culling (explored, removed — CPU BVH matches at ~585 fewer lines)
- [x] Phase 3G — Motion-adaptive culling + HiZ during motion (3.1× orbit speedup on 1M-instance scene)
- [x] Benchmark CLI (`--camera`, `--benchmark`, press C to capture camera)
- [ ] **Phase 3F — Static batching of single-instance meshes** (next; reduces 90k+ sub_draws to hundreds)
- [ ] Vulkan/MoltenVK backend for macOS
- [ ] Embedded Python scripting console
src/ifcviewer/settings.rst
-67
View File
@@ -1,67 +0,0 @@
IfcViewer settings and environment variables
=============================================
User-facing performance and quality settings (min pixel radius, LOD1
threshold, HiZ resolution, etc.) live in the **Settings** dialog and are
persisted via ``QSettings``. This page documents the remaining
environment variables — instrumentation knobs, regression-hunting
toggles, and LOD-build tuning — that are intentionally *not* surfaced
in the GUI because they target developers and benchmark runs.
All variables are read once at first use (most are static-cached
inside the function that consumes them), so set them in the shell
before launching ``BonsaiViewer`` rather than expecting hot-toggle
behaviour.
Diagnostic and benchmark instrumentation
----------------------------------------
These variables expose hooks that are useful when triaging performance
regressions or attributing frame cost. They have no effect on
correctness and ship disabled.
.. csv-table::
:header: "Variable", "Default", "Description"
:widths: 22, 14, 64
"``IFC_HIZ_MOTION``", "on (1)", "Trust the previous frame's HiZ pyramid even when the view-projection has drifted (the default). Set to ``0`` to revert to the strict gate, which discards the pyramid the moment the camera moves — useful when chasing the *missing-geometry* class of HiZ correctness bug, since strict gating reproduces a known-good baseline."
"``IFC_CULL_THREADS``", "on (1)", "Set to ``0`` to disable the multi-threaded culling path. Forces a single-threaded sweep through every model's BVH, which is useful when bisecting a regression suspected to live in the parallel cull."
"``IFC_SKIP_MDI``", "off", "Set to ``1`` to skip ``glMultiDrawElementsIndirect`` calls without changing any other state. Cull, upload, and bind still run; only the actual draw is elided. A large FPS jump means the workload is draw-bound (GPU front-end) rather than upload- or cull-bound."
"``IFC_MAX_SUBDRAWS``", "unlimited", "Truncate the drawcount passed to each MDI to ``N``, preserving the forward/reflected ratio. Lets you isolate per-subdraw command-processor overhead from raw triangle work — sweep ``N`` and watch where the FPS curve flattens."
"``IFC_FPS_HITCH_MS``", "0 (off)", "When non-zero, log a ``[fps-hitch]`` line for any frame that costs more than ``N`` ms. Only active in FPS (first-person) navigation mode. Captures visible objects, sub-draws, and HiZ-rejection counts per hitch so the slowdown can be attributed."
"``IFC_SUBDRAW_DIAG``", "off", "When set (any non-empty value), prints a one-shot histogram of sub-draw composition after the next ``finalizeModel`` — bucket counts of MDIs by sub-draw size, plus instances and triangles in each bucket. Useful for tuning the visible-list packing strategy."
LOD build tuning
----------------
These affect how the LOD1 representation is generated when a sidecar
is *baked*; loading an existing ``.ifcfed`` does not re-read them.
Override only when you're regenerating sidecars and want to inspect or
adjust the trade-off between LOD0 fidelity and LOD1 triangle savings.
.. csv-table::
:header: "Variable", "Default", "Description"
:widths: 22, 14, 64
"``IFC_LOD_ERROR``", "0.05 (clamped to ≥ 0.2)", "``meshopt_simplify`` ``target_error`` parameter — maximum positional error allowed when collapsing edges, normalised to the mesh AABB diagonal. BIM meshes are typically non-manifold and a 0.2 floor still looks fine at sub-4 pixel sizes; smaller values often produce zero collapses on these inputs."
"``IFC_LOD_RATIO``", "meshopt default", "``meshopt_simplify`` ``target_ratio`` parameter — desired fraction of the original index count to retain. Combined with ``target_error`` it forms the simplification budget."
"``IFC_LOD_MIN_SAVINGS``", "0.25", "Minimum fraction of triangles that must be eliminated for the LOD1 result to be accepted. Below this, the LOD1 slot is left empty and LOD0 is always drawn for that mesh — avoids paying upload cost for trivial reductions."
"``IFC_LOD_DEBUG``", "off", "Set to ``1`` to print per-mesh LOD build diagnostics for the first few meshes of each ``buildLodsForSidecar`` call: input/output triangle counts, target error, and the accept/reject decision. Caps printing automatically so it can be left on for full builds without flooding the log."
GUI-promoted settings (no longer env-var driven)
------------------------------------------------
For reference, the following knobs were previously read from
environment variables and are now driven by ``AppSettings`` and the
**Settings** dialog. Their old env-var spellings no longer have any
effect.
.. csv-table::
:header: "Setting", "QSettings key", "Old env var", "Default"
:widths: 28, 32, 22, 18
"Min Pixel Radius", "``viewport/min_pixel_radius``", "``IFC_MIN_PX``", "2.0"
"Motion Min Pixel Radius", "``viewport/motion_min_pixel_radius``", "``IFC_MIN_PX_MOTION``", "10.0"
"LOD1 Pixel Threshold", "``viewport/lod1_pixel_threshold``", "``IFC_LOD1_PX``", "30.0"
"HiZ Occlusion", "``viewport/hiz_enabled``", "``IFC_NO_HIZ`` (inverted)", "on"
"HiZ Resolution", "``viewport/hiz_resolution``", "``IFC_HIZ_SIZE``", "256"