Compare commits

..

1 Commits

Author SHA1 Message Date
Dion Moult 7845f8e8cd ci: build OCCT shared on Linux so plug-ins share one OCCT instance
`geom.tree().select()` silently returns zero results (or raises SWIG's
"An unknown error occurred") in the Linux release packages, while the same
commit built from source returns correct answers. `select_box()` agrees
between both, and geometry conversion is bit-identical -- only operations
that touch a stored TopoDS_Shape diverge.

Cause is linkage, not code. 8bdaa8c7c narrowed the Rocky builds from
`--shared` to `--ifcopenshell-shared`, which shares IfcOpenShell's own
libraries but leaves every dependency static. OCCT is then compiled
privately into each plug-in that uses it -- 15 of them, verified by their
own copies of the BRepClass3d/BRepExtrema/Standard_Failure strings.

That contradicts what the binaries already declare. tree.h:1748 casts a
`conversion_result_shape*` to `open_cascade_shape*`, moves the
TopoDS_Shape out of it and frees it; `open_cascade_shape` is defined once
in ifcopenshell_geometry_kernel_opencascade.so and left undefined in
ifcopenshell_geometry_tree_opencascade_brep.so for the loader to resolve.
So the two plug-ins are designed to share one OCCT-based type system, yet
static linking gives each its own Standard_Type registry and allocator.
Shapes get read and released by a different OCCT instance than made them.

Add `--opencascade-shared`, mirroring the existing `--ifcopenshell-shared`
precedent, and use it on both Rocky workflows. It cannot be spelled
`--occt-shared`: build-all.py parses any `occt-*` flag as a version
override.

BUILD_STATIC drives three things at once -- dependency link type,
-fvisibility=hidden, and BUILD_SHARED_LIBS -- so making one dependency
shared means overriding all three for it. Visibility is the subtle one:
OCCT's Standard_EXPORT expands to nothing on Unix, so it relies on default
visibility to export its API. Built shared under -fvisibility=hidden it
exports almost nothing and its own libraries cannot resolve against each
other (libTKMath.so fails to find
NCollection_BaseAllocator::CommonBaseAllocator in libTKernel.so). Static
archives are immune, which is why this surfaces only once OCCT goes
shared. Compile OCCT with the pre-visibility flag set instead.

Link the OCCT set with --as-needed. FindOpenCASCADE.cmake's config branch
uses OCCT's *complete* module list, Visualization included, which against a
static OCCT costs nothing -- an unreferenced module contributes no objects.
Against a shared OCCT all 47 become hard DT_NEEDED entries, and libTKV3d
pulls libGL.so.1 + libEGL.so.1, so `import ifcopenshell` fails on any
headless machine with "libEGL.so.1: cannot open shared object file" even
though nothing ever opens a window. Measured through the real find_package
path with the LINK_GROUP workaround composed: 67 DT_NEEDED without the
flag, 3 with it, TKV3d and TKOpenGl gone.

The flag is deliberately left open rather than closed with
-Wl,--no-as-needed. CMake emits the imported targets'
INTERFACE_LINK_LIBRARIES -- where OCCT lists libGL/libEGL -- after that
item, so closing the bracket switches the flag off immediately before the
libraries it exists to exclude. Verified against the shipped artifact:
closed, the kernel plug-in fell from 47 DT_NEEDED libTK entries to 14 and
lost TKV3d, yet still carried a direct libEGL.so.1 and still failed to
import on a headless server; open, the same 14 remain and libGL/libEGL are
gone. None of the 14 retained modules depends on GL.

Put the shared OCCT on LD_LIBRARY_PATH for the build itself. Nothing else
points at it -- IfcOpenShell's libraries get INSTALL_RPATH=$ORIGIN and OCCT
sits in its own dependency prefix -- so the post-build `import ifcopenshell`
check fails the same way. This is build-time only; the shipped packages get
libTK*.so* staged beside the payload with an $ORIGIN RUNPATH instead.

Suffix OCCT's install directory with `-shared` when it applies. Static and
shared installs are not interchangeable, but `build_dependency` skips any
dependency whose install dir already exists and cache_dependencies.py keys
its tarballs purely on that directory name -- so the static
`cache-occt-7.8.1.tar.gz` restored from the build-outputs repo silently
satisfied the build and BUILD_LIBRARY_TYPE was never applied. This is the
same cache stickiness 8bdaa8c7c described, pointing the other way. The
suffix makes the key configuration-aware, so it self-invalidates and the
static tarball stays valid for builds that still want static.

Packaging is the other half, and is why 8bdaa8c7c backed the flag out --
`stage_runtime_payload` only copies from install/ifcopenshell, so OCCT in
install/occt-* was never staged and `--shared` "worked by accident" off
cached static outputs. Stage libTK*.so* alongside, then give every staged
library an $ORIGIN RUNPATH: the core libs currently carry dead
build-machine RPATHs and the plug-ins carry none, resolving only because
the Python wrapper pulls them in by SONAME first. Shared OCCT has no such
first loader, since it is reached through the dlopen'd plug-ins.

The packages shrink: the python zip goes from 109.4 MB to 85.6 MB, because
the duplicated OCCT was 113 MB of the 287 MB unpacked payload (the eight
geometry_writer_ifc* plug-ins alone were 4.6 MB each) against ~67 MB for
one shared copy. Same argument as a91b1da28 ("Reduce Rocky package size")
and 402591e71.

macOS and Windows are affected too but are not fixed here. Their
packaging resolves via @loader_path install names and would need
install_name_tool rewriting, which cannot be verified from Linux; adding
the flag without that would ship a package that fails to load.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 08:19:27 +10:00
98 changed files with 1046 additions and 4349 deletions
+1 -1
View File
@@ -7,7 +7,7 @@ body:
label: Bug Description
placeholder: |
Describe what problem occurred and what you expected to happen instead.
1. To reproduce this, open file '...'
2. Click on '....'
3. See error
+1 -7
View File
@@ -284,18 +284,12 @@ PATTERNS = (
"*.cpp",
"*.h",
"*.i",
"*.cmake",
"*/CMakeLists.txt",
"*.yml",
)
REPO_ROOT = Path(subprocess.check_output(["git", "rev-parse", "--show-toplevel"], text=True).strip())
# Generated files; formatted by the express codegen, not by this script.
IGNORED_DIRS = (
REPO_ROOT / "src/ifcparse/schemas",
REPO_ROOT / "win/patches",
)
IGNORED_DIRS = (REPO_ROOT / "src/ifcparse/schemas",)
def get_tracked_files(root: Path | None = None) -> list[Path]:
@@ -26,7 +26,7 @@ jobs:
working-directory: src/bonsaiviewer-autodesk
steps:
- uses: actions/checkout@v7
- uses: actions/checkout@v6
- uses: dtolnay/rust-toolchain@stable
+73 -1
View File
@@ -135,7 +135,79 @@ jobs:
- name: Package .zip archives
run: |
uv run nix/package-zip-archives.py "macos${{ matrix.oldarch }}64"
VERSION=v`cat VERSION`
# packaging/build.py stages the connector binary + connector.json
# into dist/autodesk/; the .app loop below copies that folder into
# the bundle. Same on-disk shape as the Linux and Windows builds.
uv run src/bonsaiviewer-autodesk/packaging/build.py
autodesk_connector_dir="$PWD/src/bonsaiviewer-autodesk/dist/autodesk"
test -d "$autodesk_connector_dir"
cd ./build/`uname`/*/10.15/install/ifcopenshell
mkdir -p ~/output
install_root="$PWD"
stage_runtime_payload() {
dest="$1"
while IFS= read -r runtime_file; do
cp -L "$runtime_file" "$dest/"
done < <(
for runtime_dir in "$install_root/bin" "$install_root/lib" "$install_root/lib64"; do
[ -d "$runtime_dir" ] || continue
find "$runtime_dir" -type f \( -name "*.so" -o -name "*.so.*" -o -name "*.dylib" -o -name "*.dll" \)
done
)
}
ls -d python-* | while read py_version; do
postfix=`echo ${py_version: -1} | sed s/[0-9]//`
numbers=`echo $py_version | grep -oE '[0-9]+\.[0-9]+' | tr -d '.'`
py_version_major=python-${numbers}$postfix
pushd . > /dev/null
cd $py_version
if [ ! -d ifcopenshell ]; then
mkdir ../ifcopenshell_
mv * ../ifcopenshell_
mv ../ifcopenshell_ ifcopenshell
fi
[ -d ifcopenshell/__pycache__ ] && rm -rf ifcopenshell/__pycache__
find ifcopenshell -name "*.pyc" -delete
stage_runtime_payload ifcopenshell
zip -r -qq ifcopenshell-${py_version_major}-${VERSION}-${GITHUB_SHA:0:7}-macos${{ matrix.oldarch }}64.zip ifcopenshell
mv *.zip ~/output
popd > /dev/null
done
find "$install_root/bin" -maxdepth 1 -type f -perm /111 ! -name "*.zip" ! -name "*.so" ! -name "*.so.*" ! -name "*.dylib" ! -name "*.dll" | while read exe_path; do
exe=`basename "$exe_path"`
package_dir="$install_root/.package-${exe}"
rm -rf "$package_dir"
mkdir -p "$package_dir"
cp "$exe_path" "$package_dir/"
stage_runtime_payload "$package_dir"
pushd "$package_dir" > /dev/null
zip -qq -r "$HOME/output/${exe}-${VERSION}-${GITHUB_SHA:0:7}-macos${{ matrix.oldarch }}64.zip" .
popd > /dev/null
rm -rf "$package_dir"
done
# .app bundles (e.g. BonsaiViewer.app) live at the install-prefix
# root because their install rule uses `BUNDLE DESTINATION "."` —
# that's the layout Qt's macdeployqt expects. macdeployqt has
# already embedded the Qt frameworks inside each bundle during
# install/strip, so the only thing left to stage is the connector.
find "$install_root" -maxdepth 1 -type d -name "*.app" | while read app_path; do
app=`basename "$app_path" .app`
if [ "$app" = "BonsaiViewer" ]; then
# ConnectorDiscovery looks in applicationDirPath()/connectors,
# which for a bundle is Contents/MacOS.
mkdir -p "$app_path/Contents/MacOS/connectors"
cp -a "$autodesk_connector_dir" "$app_path/Contents/MacOS/connectors/"
fi
pushd "$install_root" > /dev/null
zip -qq -r "$HOME/output/${app}-${VERSION}-${GITHUB_SHA:0:7}-macos${{ matrix.oldarch }}64.zip" "$(basename "$app_path")"
popd > /dev/null
done
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v6
+188 -2
View File
@@ -80,7 +80,7 @@ jobs:
set -o pipefail
CXXFLAGS="-O3" CFLAGS="-O3" ADD_COMMIT_SHA=1 BUILD_CFG=Release BUILD_BONSAIVIEWER=ON \
uv run --with aqtinstall ./nix/build-all.py \
-v --diskcleanup --ifcopenshell-shared --occt-shared 2>&1 \
-v --diskcleanup --ifcopenshell-shared --opencascade-shared 2>&1 \
| tee build.log
- name: Upload Build Logs
@@ -110,7 +110,193 @@ jobs:
- name: Package .zip archives
shell: bash
run: |
uv run nix/package-zip-archives.py linux64 --occt-shared
VERSION=v`cat VERSION`
# bonsaiviewer-autodesk is now a Rust connector. packaging/build.py
# invokes `cargo build --release` and stages the binary +
# connector.json into dist/autodesk/. Same on-disk shape as the
# old PyInstaller flow so the symlink + zip steps below
# continue to work unchanged.
uv run src/bonsaiviewer-autodesk/packaging/build.py
autodesk_connector_dir="$PWD/src/bonsaiviewer-autodesk/dist/autodesk"
test -d "$autodesk_connector_dir"
cd ./build/`uname`/*/install/ifcopenshell
mkdir -p ~/output
install_root="$PWD"
QT6_VERSION="${QT6_VERSION:-6.8.3}"
if [ -z "${QT_DIR:-}" ]; then
for qt_candidate in "$(dirname "$install_root")"/qt6-${QT6_VERSION}-*/${QT6_VERSION}/*; do
if [ -d "$qt_candidate/lib" ]; then
QT_DIR="$qt_candidate"
break
fi
done
fi
# Ensure that all shared libraries in provided dest `$1`
# are present using their SONAMEs (at least as symlinks).
ensure_soname_links() {
dest="$1"
find "$dest" -maxdepth 1 -type f -name "*.so*" | while IFS= read -r shared_object; do
# TODO: actual pattern is "Library soname" instead of "Shared library"?
soname=$(readelf -d "$shared_object" 2>/dev/null | sed -n 's/.*(SONAME).*Shared library: \[\(.*\)\].*/\1/p' | head -n 1)
[ -n "$soname" ] || continue
[ -e "$dest/$soname" ] && continue
ln -s "$(basename "$shared_object")" "$dest/$soname"
done
}
# Copy the shared OCCT from the dependency prefix to the provided `$1`.
# OCCT is built shared (`--opencascade-shared`) so that the opencascade kernel
# and tree plug-ins share a single OCCT instance: `open_cascade_shape` objects
# are created by the kernel plug-in and then have their `TopoDS_Shape` moved out
# and freed by a tree plug-in. A private static OCCT per plug-in gives each its
# own Standard_Type registry and allocator, which silently corrupts those shapes.
# These libs live under `install/occt-*` rather than `install/ifcopenshell`,
# so `stage_runtime_payload` does not pick them up on its own.
stage_occt_runtime_payload() {
dest="$1"
for occt_lib_dir in "$(dirname "$install_root")"/occt-*/lib "$(dirname "$install_root")"/occt-*/lib64; do
[ -d "$occt_lib_dir" ] || continue
find "$occt_lib_dir" -maxdepth 1 \( -type f -o -type l \) -name "libTK*.so*" -exec cp -P {} "$dest/" \;
done
}
# Copy all libs from `install/ifcopenshell` to the provided `$1`.
# Set `$2` to `0` to skip including geometry writers.
stage_runtime_payload() {
dest="$1"
include_geometry_writers="${2:-1}"
while IFS= read -r runtime_file; do
if [ "$include_geometry_writers" != "1" ] && [[ "$(basename "$runtime_file")" == ifcopenshell.geometry.writer.* ]]; then
continue
fi
cp -P "$runtime_file" "$dest/"
done < <(
for runtime_dir in "$install_root/bin" "$install_root/lib" "$install_root/lib64"; do
[ -d "$runtime_dir" ] || continue
find "$runtime_dir" \( -type f -o -type l \) \( -name "*.so" -o -name "*.so.*" -o -name "*.dylib" -o -name "*.dll" \)
done
)
stage_occt_runtime_payload "$dest"
ensure_soname_links "$dest"
# The core libs ship with dead build-machine RPATHs and the plug-ins have
# none; today they resolve only because the Python wrapper ($ORIGIN) pulls
# them in by SONAME before any plug-in is dlopen'd. Shared OCCT has no such
# first loader -- it is reached through the plug-ins -- so give every staged
# library an $ORIGIN of its own.
find "$dest" -maxdepth 1 -type f -name "*.so*" -exec patchelf --set-rpath '$ORIGIN' {} \;
}
# Copy all libs from `QT_DIR` to the provided `$2`.
stage_qt_runtime_payload() {
exe_path="$1"
dest="$2"
[ -n "${QT_DIR:-}" ] && [ -d "$QT_DIR/lib" ] || return 0
# Skip executables that don't depend on QT (don't have `libQt6` referenced).
if ! LD_LIBRARY_PATH="$QT_DIR/lib:${LD_LIBRARY_PATH:-}" ldd "$exe_path" 2>/dev/null | grep -q "libQt6"; then
return 0
fi
# Copy all QT libs to `dest`.
find "$QT_DIR/lib" -maxdepth 1 \( -type f -o -type l \) -name "*.so*" -exec cp -P {} "$dest/" \;
ensure_soname_links "$dest"
# Copy QT plugins.
if [ -d "$QT_DIR/plugins" ]; then
pushd "$QT_DIR/plugins" > /dev/null
find . \( -type f -o -type l \) -name "*.so*" | while IFS= read -r plugin_file; do
mkdir -p "$dest/plugins/$(dirname "$plugin_file")"
cp -P "$plugin_file" "$dest/plugins/$plugin_file"
done
popd > /dev/null
# Point plugins rpath to `$dest`.
if [ -d "$dest/plugins" ]; then
find "$dest/plugins" -type f -name "*.so*" -exec patchelf --set-rpath '$ORIGIN/../..:$ORIGIN' {} \;
fi
fi
find "$dest" -maxdepth 1 -type f -name "*.so*" -exec patchelf --set-rpath '$ORIGIN' {} \;
printf "[Paths]\nPrefix = .\n" > "$dest/qt.conf"
}
# Check all binaries in the dest `$1`
# and report if they're still missing dependencies or are static.
check_runtime_dependencies() {
package_dir="$1"
missing=0
# Iterate over all .so files.
while IFS= read -r binary_file; do
# Skip non-binaries.
readelf -h "$binary_file" >/dev/null 2>&1 || continue
# Report non-dynamic binaries.
if ! env -u LD_LIBRARY_PATH ldd "$binary_file" > "$package_dir/.ldd.out" 2>&1; then
echo "ldd failed for $binary_file"
cat "$package_dir/.ldd.out"
missing=1
continue
fi
# Report missing dependencies.
if grep -q "not found" "$package_dir/.ldd.out"; then
echo "Missing runtime dependencies for $binary_file"
grep "not found" "$package_dir/.ldd.out"
missing=1
fi
done < <(find "$package_dir" -type f \( -perm /111 -o -name "*.so" -o -name "*.so.*" \))
rm -f "$package_dir/.ldd.out"
# TODO: should error?
if [ "$missing" -ne 0 ]; then
echo "Runtime dependency check found issues; continuing packaging."
fi
return 0
}
# Iterate over all built Python wrappers in `install/ifcopenshell/python-x.y.z`.
# and zip them, bundling all dynamic libs from `lib`.
ls -d python-* | while read py_version; do
postfix=`echo ${py_version: -1} | sed s/[0-9]//`
numbers=`echo $py_version | grep -oE '[0-9]+\.[0-9]+' | tr -d '.'`
py_version_major=python-${numbers}$postfix
pushd . > /dev/null
cd $py_version
if [ ! -d ifcopenshell ]; then
mkdir ../ifcopenshell_
mv * ../ifcopenshell_
mv ../ifcopenshell_ ifcopenshell
fi
[ -d ifcopenshell/__pycache__ ] && rm -rf ifcopenshell/__pycache__
find ifcopenshell -name "*.pyc" -delete
# TODO: packs qt libs also?
stage_runtime_payload ifcopenshell
zip -y -r -qq ifcopenshell-${py_version_major}-${VERSION}-${GITHUB_SHA:0:7}-linux64.zip ifcopenshell
mv *.zip ~/output
popd > /dev/null
done
# Iterate over all executables in `install/ifcopenshell/bin` and zip them.
# Each zip bundles dynamic libs from `lib` and also qt libs.
find "$install_root/bin" -maxdepth 1 -type f -perm /111 ! -name "*.zip" ! -name "*.so" ! -name "*.so.*" ! -name "*.dylib" ! -name "*.dll" | while read exe_path; do
exe=`basename "$exe_path"`
package_dir="$install_root/.package-${exe}"
rm -rf "$package_dir"
mkdir -p "$package_dir"
cp "$exe_path" "$package_dir/"
patchelf --set-rpath '$ORIGIN' "$package_dir/$exe"
stage_runtime_payload "$package_dir" 0
stage_qt_runtime_payload "$exe_path" "$package_dir"
if [ "$exe" = "BonsaiViewer" ]; then
mkdir -p "$package_dir/connectors"
cp -a "$autodesk_connector_dir" "$package_dir/connectors/"
fi
check_runtime_dependencies "$package_dir"
pushd "$package_dir" > /dev/null
zip -y -qq -r "$HOME/output/${exe}-${VERSION}-${GITHUB_SHA:0:7}-linux64.zip" .
popd > /dev/null
rm -rf "$package_dir"
done
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v6
+166 -2
View File
@@ -92,7 +92,7 @@ jobs:
set -o pipefail
CXXFLAGS="-O3" CFLAGS="-O3" ADD_COMMIT_SHA=1 BUILD_CFG=Release BUILD_BONSAIVIEWER=ON \
uv run --with aqtinstall ./nix/build-all.py \
-v --diskcleanup --ifcopenshell-shared 2>&1 \
-v --diskcleanup --ifcopenshell-shared --opencascade-shared 2>&1 \
| tee build.log
- name: Upload Build Logs
@@ -122,7 +122,171 @@ jobs:
- name: Package .zip archives
shell: bash
run: |
uv run nix/package-zip-archives.py linuxarm64
VERSION=v`cat VERSION`
# bonsaiviewer-autodesk is now a Rust connector. packaging/build.py
# invokes `cargo build --release` and stages the binary +
# connector.json into dist/autodesk/. Same on-disk shape as the
# old PyInstaller flow so the symlink + zip steps below
# continue to work unchanged.
uv run src/bonsaiviewer-autodesk/packaging/build.py
autodesk_connector_dir="$PWD/src/bonsaiviewer-autodesk/dist/autodesk"
test -d "$autodesk_connector_dir"
cd ./build/`uname`/*/install/ifcopenshell
mkdir -p ~/output
install_root="$PWD"
QT6_VERSION="${QT6_VERSION:-6.8.3}"
if [ -z "${QT_DIR:-}" ]; then
for qt_candidate in "$(dirname "$install_root")"/qt6-${QT6_VERSION}-*/${QT6_VERSION}/*; do
if [ -d "$qt_candidate/lib" ]; then
QT_DIR="$qt_candidate"
break
fi
done
fi
ensure_soname_links() {
dest="$1"
find "$dest" -maxdepth 1 -type f -name "*.so*" | while IFS= read -r shared_object; do
soname=$(readelf -d "$shared_object" 2>/dev/null | sed -n 's/.*(SONAME).*Shared library: \[\(.*\)\].*/\1/p' | head -n 1)
[ -n "$soname" ] || continue
[ -e "$dest/$soname" ] && continue
ln -s "$(basename "$shared_object")" "$dest/$soname"
done
}
# Copy the shared OCCT from the dependency prefix to the provided `$1`.
# OCCT is built shared (`--opencascade-shared`) so that the opencascade kernel
# and tree plug-ins share a single OCCT instance: `open_cascade_shape` objects
# are created by the kernel plug-in and then have their `TopoDS_Shape` moved out
# and freed by a tree plug-in. A private static OCCT per plug-in gives each its
# own Standard_Type registry and allocator, which silently corrupts those shapes.
# These libs live under `install/occt-*` rather than `install/ifcopenshell`,
# so `stage_runtime_payload` does not pick them up on its own.
stage_occt_runtime_payload() {
dest="$1"
for occt_lib_dir in "$(dirname "$install_root")"/occt-*/lib "$(dirname "$install_root")"/occt-*/lib64; do
[ -d "$occt_lib_dir" ] || continue
find "$occt_lib_dir" -maxdepth 1 \( -type f -o -type l \) -name "libTK*.so*" -exec cp -P {} "$dest/" \;
done
}
stage_runtime_payload() {
dest="$1"
include_geometry_writers="${2:-1}"
while IFS= read -r runtime_file; do
if [ "$include_geometry_writers" != "1" ] && [[ "$(basename "$runtime_file")" == ifcopenshell.geometry.writer.* ]]; then
continue
fi
cp -P "$runtime_file" "$dest/"
done < <(
for runtime_dir in "$install_root/bin" "$install_root/lib" "$install_root/lib64"; do
[ -d "$runtime_dir" ] || continue
find "$runtime_dir" \( -type f -o -type l \) \( -name "*.so" -o -name "*.so.*" -o -name "*.dylib" -o -name "*.dll" \)
done
)
stage_occt_runtime_payload "$dest"
ensure_soname_links "$dest"
# The core libs ship with dead build-machine RPATHs and the plug-ins have
# none; today they resolve only because the Python wrapper ($ORIGIN) pulls
# them in by SONAME before any plug-in is dlopen'd. Shared OCCT has no such
# first loader -- it is reached through the plug-ins -- so give every staged
# library an $ORIGIN of its own.
find "$dest" -maxdepth 1 -type f -name "*.so*" -exec patchelf --set-rpath '$ORIGIN' {} \;
}
stage_qt_runtime_payload() {
exe_path="$1"
dest="$2"
[ -n "${QT_DIR:-}" ] && [ -d "$QT_DIR/lib" ] || return 0
if ! LD_LIBRARY_PATH="$QT_DIR/lib:${LD_LIBRARY_PATH:-}" ldd "$exe_path" 2>/dev/null | grep -q "libQt6"; then
return 0
fi
find "$QT_DIR/lib" -maxdepth 1 \( -type f -o -type l \) -name "*.so*" -exec cp -P {} "$dest/" \;
ensure_soname_links "$dest"
if [ -d "$QT_DIR/plugins" ]; then
pushd "$QT_DIR/plugins" > /dev/null
find . \( -type f -o -type l \) -name "*.so*" | while IFS= read -r plugin_file; do
mkdir -p "$dest/plugins/$(dirname "$plugin_file")"
cp -P "$plugin_file" "$dest/plugins/$plugin_file"
done
popd > /dev/null
if [ -d "$dest/plugins" ]; then
find "$dest/plugins" -type f -name "*.so*" -exec patchelf --set-rpath '$ORIGIN/../..:$ORIGIN' {} \;
fi
fi
find "$dest" -maxdepth 1 -type f -name "*.so*" -exec patchelf --set-rpath '$ORIGIN' {} \;
printf "[Paths]\nPrefix = .\n" > "$dest/qt.conf"
}
check_runtime_dependencies() {
package_dir="$1"
missing=0
while IFS= read -r binary_file; do
readelf -h "$binary_file" >/dev/null 2>&1 || continue
if ! env -u LD_LIBRARY_PATH ldd "$binary_file" > "$package_dir/.ldd.out" 2>&1; then
echo "ldd failed for $binary_file"
cat "$package_dir/.ldd.out"
missing=1
continue
fi
if grep -q "not found" "$package_dir/.ldd.out"; then
echo "Missing runtime dependencies for $binary_file"
grep "not found" "$package_dir/.ldd.out"
missing=1
fi
done < <(find "$package_dir" -type f \( -perm /111 -o -name "*.so" -o -name "*.so.*" \))
rm -f "$package_dir/.ldd.out"
if [ "$missing" -ne 0 ]; then
echo "Runtime dependency check found issues; continuing packaging."
fi
return 0
}
ls -d python-* | while read py_version; do
postfix=`echo ${py_version: -1} | sed s/[0-9]//`
numbers=`echo $py_version | grep -oE '[0-9]+\.[0-9]+' | tr -d '.'`
py_version_major=python-${numbers}$postfix
pushd . > /dev/null
cd $py_version
if [ ! -d ifcopenshell ]; then
mkdir ../ifcopenshell_
mv * ../ifcopenshell_
mv ../ifcopenshell_ ifcopenshell
fi
[ -d ifcopenshell/__pycache__ ] && rm -rf ifcopenshell/__pycache__
find ifcopenshell -name "*.pyc" -delete
stage_runtime_payload ifcopenshell
zip -y -r -qq ifcopenshell-${py_version_major}-${VERSION}-${GITHUB_SHA:0:7}-linuxarm64.zip ifcopenshell
mv *.zip ~/output
popd > /dev/null
done
find "$install_root/bin" -maxdepth 1 -type f -perm /111 ! -name "*.zip" ! -name "*.so" ! -name "*.so.*" ! -name "*.dylib" ! -name "*.dll" | while read exe_path; do
exe=`basename "$exe_path"`
package_dir="$install_root/.package-${exe}"
rm -rf "$package_dir"
mkdir -p "$package_dir"
cp "$exe_path" "$package_dir/"
patchelf --set-rpath '$ORIGIN' "$package_dir/$exe"
stage_runtime_payload "$package_dir" 0
stage_qt_runtime_payload "$exe_path" "$package_dir"
if [ "$exe" = "BonsaiViewer" ]; then
mkdir -p "$package_dir/connectors"
cp -a "$autodesk_connector_dir" "$package_dir/connectors/"
fi
check_runtime_dependencies "$package_dir"
pushd "$package_dir" > /dev/null
zip -y -qq -r "$HOME/output/${exe}-${VERSION}-${GITHUB_SHA:0:7}-linuxarm64.zip" .
popd > /dev/null
rm -rf "$package_dir"
done
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v6
+1 -1
View File
@@ -64,7 +64,7 @@ jobs:
max-size: 5000MB
- name: Set up Python for connector build
uses: actions/setup-python@v7
uses: actions/setup-python@v6
with:
python-version: '3.12'
+11 -2
View File
@@ -16,7 +16,7 @@ on:
- 'src/ifc5d/ifc5d/**'
- 'src/ifccityjson/**'
branches:
- v0.9.0
- v0.8.0
workflow_dispatch:
jobs:
@@ -51,10 +51,19 @@ jobs:
name: "Linux Build",
short_name: linux,
}
- {
name: "MacOS Build",
short_name: macos,
}
- {
name: "MacOS ARM Build",
short_name: macosm1,
}
exclude:
# Python 3.13 is needed for Blender 5.1+ and Blender dropped Intel Mac support in 5.0.
- pyver: py313
config:
short_name: macos
steps:
- uses: actions/checkout@v7
- uses: actions/setup-python@v7 # https://github.com/actions/setup-python
@@ -119,7 +128,7 @@ jobs:
blender --command extension install-file -r user_default -e $bonsai_zip
blender --command extension list
git clone --branch ${{ github.ref_name }} --single-branch https://github.com/IfcOpenShell/IfcOpenShell.git IfcOpenShell
git clone https://github.com/IfcOpenShell/IfcOpenShell.git IfcOpenShell
# Reregister Bonsai.
# Note that running it in background might miss some errors
+9
View File
@@ -34,10 +34,19 @@ jobs:
name: "Linux Build",
short_name: linux,
}
- {
name: "MacOS Build",
short_name: macos,
}
- {
name: "MacOS ARM Build",
short_name: macosm1,
}
exclude:
# Python 3.13 is needed for Blender 5.1+ and Blender dropped Intel Mac support in 5.0.
- pyver: py313
config:
short_name: macos
steps:
- uses: actions/checkout@v7
- uses: actions/setup-python@v7 # https://github.com/actions/setup-python
@@ -34,21 +34,21 @@ jobs:
- name: Run conda cleaner
run: |
python - << EOF
import os
from datetime import datetime, timedelta
from binstar_client.utils import get_server_api
from binstar_client.errors import BinstarError
# Configuration
api_token = os.environ.get('ANACONDA_TOKEN')
pkg_name = 'ifcopenshell'
channel_name = 'ifcopenshell'
# Authenticate with Anaconda
aserver_api = get_server_api(token=api_token)
# Get the list of packages in the channel
def get_package(filter_package_name: str = None):
try:
@@ -59,12 +59,12 @@ jobs:
print(f"No packages found for {filter_package_name}.")
if len(user_packages) > 1:
raise ValueError(f"Found {len(user_packages)} package for {filter_package_name}. Will only support 1 package.")
return user_packages[0]
except BinstarError as err:
raise ValueError(f"Failed to fetch packages: {err}")
# Delete a package version
def delete_package(package_name, version):
try:
@@ -72,33 +72,33 @@ jobs:
print(f"Deleted {package_name} version {version}")
except BinstarError as err:
print(f"Failed to delete {package_name} version {version}: {err}")
# Main logic
def main():
package = get_package(pkg_name)
if not package:
print("No packages found.")
return
number_of_supported_versions = ${{ env.NUM_SUPPORTED_VERSIONS }}
package_name = package['name']
versions = package["versions"]
if len(versions) <= number_of_supported_versions:
print(f"Number of versions {len(versions)} is less than or equal to {number_of_supported_versions}.")
return
# sort the versions in descending order
print(f"Before reversal: {versions=}")
versions.reverse()
print(f"After reversal: {versions=}")
releases = versions[number_of_supported_versions:]
for release in releases:
delete_package(package_name, release)
main()
EOF
@@ -24,13 +24,13 @@ jobs:
- uses: actions/checkout@v7
- name: Set env
run: echo ok go
- name: Get current version
id: version
# Strip any trailing prerelease label and number; the dated alpha
# suffix is added below.
run: echo "version=$(sed -E 's/[[:alpha:]]+[0-9]+$//' VERSION)" >> $GITHUB_OUTPUT
- name: Get current date
id: date
run: echo "date=$(date +'%y%m%d')" >> $GITHUB_OUTPUT
@@ -39,7 +39,7 @@ jobs:
id: verdate
run: echo "verdate=${{ steps.version.outputs.version }}alpha${{ steps.date.outputs.date }}" >> $GITHUB_OUTPUT
test:
name: ${{ matrix.platform.distver }}-${{ matrix.pyver.name }}
needs: activate
@@ -64,7 +64,7 @@ jobs:
uses: pierotofy/set-swap-space@master
with:
swap-size-gb: 10
- name: set ARTIFACTS ENV vars
shell: bash
run: |
@@ -76,7 +76,7 @@ jobs:
elif [[ "$RUNNER_OS" == "Linux" ]]; then
echo "ARTIFACTS_DIR=/home/runner/work/artifacts" >> $GITHUB_ENV
fi
- uses: actions/checkout@v7
with:
submodules: recursive
+11 -12
View File
@@ -1,6 +1,6 @@
name: ci-ifcopenshell-docker
on:
on:
workflow_dispatch:
push:
tags:
@@ -37,14 +37,13 @@ jobs:
name: ccache
uses: hendrikmuhs/ccache-action@v1.2.23
-
-
name: Build ifcopenshell
run: |
mkdir build && cd build
cmake \
-DCMAKE_INSTALL_PREFIX=$PWD/install/ \
-DCMAKE_BUILD_TYPE=Release \
-DUSE_CCACHE=ON \
-DCMAKE_PREFIX_PATH=/usr \
-DCMAKE_SYSTEM_PREFIX_PATH=/usr \
-DBUILD_PACKAGE=On \
@@ -67,12 +66,12 @@ jobs:
../cmake
make -j $(nproc)
make install
-
-
name: Package
run: |
make package
working-directory: build
- name: Upload
- name: Upload
uses: actions/upload-artifact@v7
with:
# Artifact name
@@ -89,8 +88,8 @@ jobs:
- uses: actions/checkout@v7
with:
lfs: true
- name: Download
- name: Download
uses: actions/download-artifact@v8.0.1
with:
# Artifact name
@@ -101,17 +100,17 @@ jobs:
uses: docker/setup-qemu-action@v4
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v4
-
uses: docker/setup-buildx-action@v4
-
name: Login to Dockerhub
uses: docker/login-action@v4
uses: docker/login-action@v4
with:
username: aecgeeks
password: ${{ secrets.DOCKER_HUB_TOKEN }}
-
-
name: Build container image
uses: docker/build-push-action@v7
with:
with:
context: artifacts
repository: aecgeeks/ifcopenshell
# Since the dispatch is set to `tag`, `github.ref_name` should evaluate to the pushed tag
+1 -1
View File
@@ -7,7 +7,7 @@ on:
- '.github/workflows/ci-ifcsverchok-build.yml'
- 'src/ifcsverchok/*'
branches:
- v0.9.0
- v0.8.0
jobs:
activate:
+1 -3
View File
@@ -3,13 +3,11 @@ name: ci-ifctester-org
on:
workflow_dispatch:
push:
branches:
- v0.9.0
paths:
- src/ifctester/**
jobs:
publish_ifctester_org:
publish_website:
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v7
+3 -4
View File
@@ -84,7 +84,7 @@ jobs:
libocct-foundation-dev libocct-modeling-algorithms-dev libocct-modeling-data-dev libocct-ocaf-dev libocct-visualization-dev libocct-data-exchange-dev \
${OCCT_CMAKE_DEPS} \
libcgal-dev libeigen3-dev
- name: ccache
uses: hendrikmuhs/ccache-action@v1.2.23
with:
@@ -176,7 +176,7 @@ jobs:
run: |
echo $Python3_ROOT_DIR
echo ${{ env.pythonLocation }}
mkdir build && cd build
cmake \
-DCMAKE_BUILD_TYPE=Release \
@@ -185,7 +185,6 @@ jobs:
-DPYTHON_EXECUTABLE:FILEPATH=${{ env.pythonLocation }}/bin/python \
-DPYTHON_INCLUDE_DIR:PATH=${{ env.pythonLocation }}/include/python3.11 \
-DUSE_MMAP=On \
-DUSE_CCACHE=ON \
-DBUILD_SHARED_LIBS=${{ matrix.build_shared_libs }} \
"-DSCHEMA_VERSIONS=2x3;4;4x3_add2" \
-DGLTF_SUPPORT=On \
@@ -224,7 +223,7 @@ jobs:
cmake --build .
./IfcOpenHouse && test -f IfcOpenHouse.ifc
./IfcParseExamples IfcOpenHouse.ifc
./IfcAdvancedHouse && test -f IfcAdvancedHouse.ifc
./IfcAdvancedHouse && test -f IfcAdvancedHouse.ifc
./IfcAlignment && test -f FHWA_Bridge_Geometry_Alignment_Example.ifc
./IfcSimplifiedAlignment && test -f FHWA_Bridge_Geometry_Alignment_Example_Simplified.ifc
@@ -35,4 +35,4 @@ jobs:
external_repository: IfcOpenShell/bonsaibim_org_docs_unstable # Target repository
publish_branch: main # Branch to deploy to
cname: docs-unstable.bonsaibim.org # Custom domain for unstable docs
publish_dir: src/bonsai/docs/_build/html # Directory containing built docs
publish_dir: src/bonsai/docs/_build/html # Directory containing built docs
@@ -21,7 +21,7 @@ jobs:
steps:
- name: Set env
run: echo ok go
build:
needs: activate
runs-on: ubuntu-latest
+4 -4
View File
@@ -37,7 +37,7 @@ jobs:
libtbb-dev nlohmann-json3-dev \
libocct-foundation-dev libocct-modeling-algorithms-dev libocct-modeling-data-dev libocct-ocaf-dev libocct-visualization-dev libocct-data-exchange-dev \
libcgal-dev opencollada-dev
- name: Build
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
@@ -77,7 +77,7 @@ jobs:
echo ::set-output name=deb::$( ls assets/*.deb | head -n 1 | xargs basename )
working-directory: build
env:
CHANGELOG_GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
CHANGELOG_GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Release
id: release
uses: actions/create-release@v1
@@ -101,7 +101,7 @@ jobs:
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
upload_url: ${{ steps.release.outputs.upload_url }} # This pulls from the CREATE RELEASE step above, referencing it's ID to get its outputs object, which include a `upload_url`. See this blog post for more info: https://jasonet.co/posts/new-features-of-github-actions/#passing-data-to-future-steps
upload_url: ${{ steps.release.outputs.upload_url }} # This pulls from the CREATE RELEASE step above, referencing it's ID to get its outputs object, which include a `upload_url`. See this blog post for more info: https://jasonet.co/posts/new-features-of-github-actions/#passing-data-to-future-steps
asset_path: build/assets/${{ steps.package.outputs.tgz }}
asset_name: ${{ steps.package.outputs.tgz }}
asset_content_type: application/x-gzip
@@ -111,7 +111,7 @@ jobs:
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
upload_url: ${{ steps.release.outputs.upload_url }} # This pulls from the CREATE RELEASE step above, referencing it's ID to get its outputs object, which include a `upload_url`. See this blog post for more info: https://jasonet.co/posts/new-features-of-github-actions/#passing-data-to-future-steps
upload_url: ${{ steps.release.outputs.upload_url }} # This pulls from the CREATE RELEASE step above, referencing it's ID to get its outputs object, which include a `upload_url`. See this blog post for more info: https://jasonet.co/posts/new-features-of-github-actions/#passing-data-to-future-steps
asset_path: build/assets/${{ steps.package.outputs.deb }}
asset_name: ${{ steps.package.outputs.deb }}
asset_content_type: application/vnd.debian.binary-package
+20 -38
View File
@@ -134,7 +134,6 @@ option(USERSPACE_PYTHON_PREFIX "Installs IfcPython for the current user only ins
option(USE_DEBUG_PYTHON "Use debug binaries when building Debug IfcPython on Windows." OFF)
option(ADD_COMMIT_SHA "Add commit sha and branch in version number, requires git" OFF)
option(VERSION_OVERRIDE "Use VERSION as the branch label when commit information is embedded" OFF)
option(USE_CCACHE "Use ccache as a compiler launcher if it is found" OFF)
set(
PYTHON_MODULE_INSTALL_DIR
@@ -179,30 +178,26 @@ if((BUILD_CONVERT OR BUILD_GEOMSERVER OR BUILD_IFCPYTHON) AND(NOT BUILD_IFCGEOM)
set(BUILD_IFCGEOM ON)
endif()
if(USE_CCACHE)
find_program(CCACHE_FOUND ccache)
if(CCACHE_FOUND)
message(STATUS "`USE_CCACHE` is enabled and `ccache` is found, using it as a compiler launcher.")
set_property(GLOBAL PROPERTY RULE_LAUNCH_COMPILE "${CCACHE_FOUND}")
if(MSVC)
# By default Visual Studio generators will use /Zi which is not compatible
# with ccache, so tell Visual Studio to use /Z7 instead.
set(CMAKE_MSVC_DEBUG_INFORMATION_FORMAT "$<$<CONFIG:Debug,RelWithDebInfo>:Embedded>")
# Not needed for Ninja.
if(CMAKE_GENERATOR MATCHES "Visual Studio")
file(COPY_FILE
${CCACHE_FOUND} ${CMAKE_BINARY_DIR}/cl.exe
ONLY_IF_DIFFERENT)
set(CMAKE_VS_GLOBALS
"CLToolExe=cl.exe"
"CLToolPath=${CMAKE_BINARY_DIR}"
"UseMultiToolTask=true"
)
endif()
find_program(CCACHE_FOUND ccache)
if(CCACHE_FOUND)
message(STATUS "`ccache` is found, using it as a compiler launcher.")
set_property(GLOBAL PROPERTY RULE_LAUNCH_COMPILE "${CCACHE_FOUND}")
if(MSVC)
# By default Visual Studio generators will use /Zi which is not compatible
# with ccache, so tell Visual Studio to use /Z7 instead.
set(CMAKE_MSVC_DEBUG_INFORMATION_FORMAT "$<$<CONFIG:Debug,RelWithDebInfo>:Embedded>")
# Not needed for Ninja.
if(CMAKE_GENERATOR MATCHES "Visual Studio")
file(COPY_FILE
${CCACHE_FOUND} ${CMAKE_BINARY_DIR}/cl.exe
ONLY_IF_DIFFERENT)
set(CMAKE_VS_GLOBALS
"CLToolExe=cl.exe"
"CLToolPath=${CMAKE_BINARY_DIR}"
"UseMultiToolTask=true"
)
endif()
endif()
else()
message(STATUS "ccache usage is disabled, set `USE_CCACHE=ON` to enable it.")
endif()
if(MSVC AND MSVC_PARALLEL_BUILD)
@@ -316,25 +311,12 @@ if (WITH_ROCKSDB)
set(SWIG_DEFINES ${SWIG_DEFINES} -DIFOPSH_WITH_ROCKSDB)
# See https://github.com/facebook/rocksdb/issues/981.
if(TARGET RocksDB::rocksdb)
set(IFCOPENSHELL_ROCKSDB_IMPORTED_TARGET RocksDB::rocksdb)
target_link_libraries(IFCOPENSHELL_RocksDB INTERFACE RocksDB::rocksdb)
elseif(TARGET RocksDB::rocksdb-shared)
set(IFCOPENSHELL_ROCKSDB_IMPORTED_TARGET RocksDB::rocksdb-shared)
target_link_libraries(IFCOPENSHELL_RocksDB INTERFACE RocksDB::rocksdb-shared)
else()
message(FATAL_ERROR "RocksDB found but neither RocksDB::rocksdb nor RocksDB::rocksdb-shared target exists")
endif()
# Our win/build-deps.cmd builds RocksDB separately per Debug/Release config into the
# same install prefix, so the imported target only ever has DEBUG and RELEASE listed in
# IMPORTED_CONFIGURATIONS. On a multi-config generator (Visual Studio), CMake maps any
# unmatched build config to the *first* entry of that list, which happens to be DEBUG
# (RocksDBTargets-debug.cmake sorts before RocksDBTargets-release.cmake). Without an
# explicit mapping, RelWithDebInfo and MinSizeRel builds would end up linking the
# /MDd-flavored rocksdb_d.lib into an /MD (NDEBUG) binary, causing a CRT/runtime-library
# mismatch that depends on nothing but that alphabetical ordering.
set_target_properties(${IFCOPENSHELL_ROCKSDB_IMPORTED_TARGET} PROPERTIES
MAP_IMPORTED_CONFIG_RELWITHDEBINFO "RELWITHDEBINFO;RELEASE"
MAP_IMPORTED_CONFIG_MINSIZEREL "MINSIZEREL;RELEASE"
)
target_link_libraries(IFCOPENSHELL_RocksDB INTERFACE ${IFCOPENSHELL_ROCKSDB_IMPORTED_TARGET})
if (WITH_ZSTD)
# @todo do we actually need the zstd include dir or rather just pass
+35
View File
@@ -54,6 +54,41 @@ if(NOT OCC_INCLUDE_DIR AND NOT OCC_LIBRARY_DIR)
set(OpenCASCADE_LIBRARIES "$<LINK_GROUP:RESCAN,${OpenCASCADE_LIBRARIES}>")
endif()
if(UNIX AND NOT APPLE)
# Record only the OCCT modules whose symbols are actually referenced.
#
# OpenCASCADE_LIBRARIES is OCCT's *complete* module list, Visualization
# included -- TKV3d, TKOpenGl, TKService. Against a static OCCT that
# costs nothing, because an unreferenced module contributes no objects.
# Against a shared OCCT every entry becomes a hard DT_NEEDED, and
# libTKV3d pulls libGL.so.1 + libEGL.so.1, so `import ifcopenshell`
# dies on any headless machine with
# ImportError: libEGL.so.1: cannot open shared object file
# even though IfcOpenShell never opens a window. Measured on a full
# 67-module link: 67 DT_NEEDED entries without the flag, 3 with it,
# and TKV3d/TKOpenGl gone in the second case.
#
# Goes before the LINK_GROUP above rather than inside it, and is a
# no-op for a static OCCT (--as-needed only governs shared libraries),
# so this is safe for both link types.
#
# Deliberately NOT closed with -Wl,--no-as-needed. CMake emits the
# imported targets' INTERFACE_LINK_LIBRARIES -- which is where OCCT
# lists libGL/libEGL -- *after* this item, so a closing bracket
# switches the flag back off just before the libraries it was added to
# exclude. Measured: with the bracket closed the plug-ins dropped from
# 47 DT_NEEDED libTK entries to 14 and lost TKV3d, yet still carried a
# direct libEGL.so.1 and failed to import on a headless server; with it
# left open the same 14 remain and libGL/libEGL are gone.
#
# The cost is that --as-needed stays in effect for whatever follows on
# the link line. That is the default on Debian/Ubuntu so it is
# well-trodden, but it does mean a library needed only for
# static-initialiser side effects could be dropped -- the thing to
# suspect first if a serialiser stops registering itself.
set(OpenCASCADE_LIBRARIES "-Wl,--as-needed" "${OpenCASCADE_LIBRARIES}")
endif()
if(OpenCASCADE_VERSION VERSION_LESS "7.9.0" AND WIN32)
# Bug in OCCT cmake configs < 7.9.0 - missing linked library.
list(APPEND OpenCASCADE_LIBRARIES WSOCK32.lib)
+119 -242
View File
@@ -32,7 +32,27 @@ Example usage:
python build-all.py IfcParse IfcOpenShell-Python
Run with --help to see available arguments.
Available arguments:
``-py-313`` - build for specific Python version
(building for all supported Python version by default).
``-occt-xxx`` - use a specific OCCT version (e.g. ``-occt-7.8.1``) instead of the default
``-wasm`` - compile for wasm
``-without-xxx`` - do not build dependency ``xxx`` (e.g. ``--without-swig``)
``-mac-cross-compile-intel`` - cross compile for Intel Mac on Apple Silicon host
``-shared`` - build shared libraries. By default will build static.
``-ifcopenshell-shared`` - build only IfcOpenShell's own libraries as shared
(dependencies stay static). Redundant if ``-shared`` is also passed.
``-opencascade-shared`` - build OCCT as shared libraries (other dependencies stay
static). Redundant if ``-shared`` is also passed. Required whenever more than one
plug-in uses OCCT: `open_cascade_shape` instances are created by the opencascade
kernel plug-in and consumed by the opencascade tree plug-ins, which move a
`TopoDS_Shape` out of them and free them. A private static OCCT per plug-in gives
each one its own `Standard_Type` registry and allocator, so those shapes are read
and released by a different OCCT instance than the one that made them.
``-diskcleanup`` - clean up build directories after finishing building dependencies
``-build-examples`` - build IfcOpenShell examples
``-lto`` - enable link-time optimization (adds ``-flto`` to compiler flags)
``-v`` - enable verbose logs
Used environment variables:
@@ -61,8 +81,6 @@ Used environment variables:
`ADD_COMMIT_SHA` and `VERSION_OVERRIDE` will be set to `ON` while configuring IfcOpenShell
- ``BUILD_BONSAIVIEWER`` - enable building BonsaiViewer, `off` by default.
- ``IFCOS_BUILD_PYTHON_WRAPPER`` - enable building the Python wrapper, `on` by default.
- ``PYTHON_USER_SITE`` - install the Python wrapper into the user's site-packages directory
instead of the interpreter's prefix, `off` by default.
# This script builds IfcOpenShell and its dependencies #
# #
@@ -107,9 +125,6 @@ Used environment variables:
"""
from __future__ import annotations
import argparse
import glob
import logging
import multiprocessing
@@ -121,13 +136,12 @@ import subprocess as sp
import sys
import sysconfig
import tarfile
import textwrap
import threading
import time
from collections.abc import Generator, Sequence
from datetime import datetime
from pathlib import Path
from typing import Literal, NamedTuple
from typing import Literal
from urllib.request import urlretrieve
from typing_extensions import assert_never
@@ -155,9 +169,8 @@ ADD_COMMIT_SHA = is_on_off(os.getenv("ADD_COMMIT_SHA"), default=False)
IFCOS_BUILD_PYTHON_WRAPPER = is_on_off(os.getenv("IFCOS_BUILD_PYTHON_WRAPPER"), default=True)
BUILD_BONSAIVIEWER = is_on_off(os.getenv("BUILD_BONSAIVIEWER"), default=False)
USE_OCCT = is_on_off(os.getenv("USE_OCCT"), default=True)
PYTHON_USER_SITE = is_on_off(os.getenv("PYTHON_USER_SITE"), default=False)
PYTHON_VERSIONS = ["3.10.3", "3.11.8", "3.12.1", "3.13.6", "3.14.0", "3.15.0"]
PYTHON_VERSIONS = ["3.10.3", "3.11.8", "3.12.1", "3.13.6", "3.14.0"]
JSON_VERSION = "3.11.3"
OCE_VERSION = "0.18.3"
OCCT_VERSION = "7.8.1"
@@ -196,142 +209,10 @@ strip = "strip"
xz = "xz" # Used implicitly for `tar -xf *.tar.xz`.
brew = "brew"
class Args(NamedTuple):
explicit_targets: list[str]
build_examples: bool
diskcleanup: bool
lto: bool
verbose: bool
shared: bool
ifcopenshell_shared: bool
occt_shared: bool
mac_cross_compile_intel: bool
wasm: bool
class DynamicArgs(NamedTuple):
without: set[str]
py_versions: set[str]
occt_version: str | None
@classmethod
def from_unknown_flags(cls, unknown_flags: list[str], arg_parser: argparse.ArgumentParser) -> DynamicArgs:
flags = set(s.lstrip("-") for s in unknown_flags if s.startswith("-"))
without: set[str] = set()
py_versions: set[str] = set()
occt_versions: set[str] = set()
leftover: set[str] = set()
for f in flags:
if f.startswith("without-"):
without.add(f.removeprefix("without-").lower())
elif f.startswith("py-"):
py_versions.add(f.removeprefix("py-"))
elif f.startswith("occt-"):
occt_versions.add(f.removeprefix("occt-"))
else:
leftover.add(f)
if leftover:
arg_parser.error(f"unrecognized arguments: {', '.join('-' + f for f in sorted(leftover))}")
if len(occt_versions) > 1:
arg_parser.error(f"more than one OCCT version provided: {', '.join(sorted(occt_versions))}")
occt_version = next(iter(occt_versions), None)
return cls(without=without, py_versions=py_versions, occt_version=occt_version)
def parse_args() -> tuple[Args, DynamicArgs]:
arg_parser = argparse.ArgumentParser(
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog=textwrap.dedent("""\
Additional dynamic -flags (not declared above):
-py-313 build for specific Python version
(building for all supported Python versions by default)
-occt-xxx use a specific OCCT version (e.g. -occt-7.8.1) instead of the default
-without-xxx do not build dependency `xxx` (e.g. --without-swig)"""),
)
arg_parser.add_argument("explicit_targets", nargs="*", help="Targets provided by CLI.")
arg_parser.add_argument(
"--build-examples",
action="store_true",
default=False,
help="Build IfcOpenShell examples.",
)
arg_parser.add_argument(
"-diskcleanup",
"--diskcleanup",
action="store_true",
default=False,
help="Clean up build directories after finishing building dependencies.",
)
arg_parser.add_argument(
"-lto",
"--lto",
action="store_true",
default=False,
help="Enable link-time optimization (adds -flto to compiler flags).",
)
arg_parser.add_argument(
"-v",
"--verbose",
action="store_true",
default=False,
help="Enable verbose logs.",
)
arg_parser.add_argument(
"-shared",
"--shared",
action="store_true",
default=False,
help="Build shared libraries. By default will build static.",
)
arg_parser.add_argument(
"-ifcopenshell-shared",
"--ifcopenshell-shared",
action="store_true",
default=False,
help="Build only IfcOpenShell's own libraries as shared (dependencies stay static). "
"Redundant if -shared is also passed.",
)
arg_parser.add_argument(
"--occt-shared",
action="store_true",
default=False,
help="Build OCCT as shared. Redundant if -shared is also passed.",
)
arg_parser.add_argument(
"-mac-cross-compile-intel",
"--mac-cross-compile-intel",
action="store_true",
default=False,
help="Cross compile for Intel Mac on Apple Silicon host.",
)
arg_parser.add_argument("-wasm", "--wasm", action="store_true", default=False, help="Compile for wasm.")
namespace, unknown_flags = arg_parser.parse_known_args()
args = Args(
explicit_targets=namespace.explicit_targets,
build_examples=namespace.build_examples,
diskcleanup=namespace.diskcleanup,
lto=namespace.lto,
verbose=namespace.verbose,
shared=namespace.shared,
ifcopenshell_shared=namespace.ifcopenshell_shared or namespace.shared,
occt_shared=namespace.occt_shared or namespace.shared,
mac_cross_compile_intel=namespace.mac_cross_compile_intel,
wasm=namespace.wasm,
)
dynamic_args = DynamicArgs.from_unknown_flags(unknown_flags, arg_parser)
return args, dynamic_args
ARGS, DYNAMIC_ARGS = parse_args()
explicit_targets: set[str] = set(ARGS.explicit_targets)
explicit_targets = [s for s in sys.argv[1:] if not s.startswith("-")]
"""Targets provided by CLI."""
flags = set(s.lstrip("-") for s in sys.argv[1:] if s.startswith("-"))
"""CLI flags."""
# Helper function for coloured printing
@@ -350,11 +231,17 @@ def cecho(message, color=NO_COLOR):
logger.info(f"{color}{message}\033[0m")
# Flags.
BUILD_EXAMPLES = "build-examples" in flags
DISK_CLEANUP = "diskcleanup" in flags
LTO = "lto" in flags
VERBOSE = "v" in flags
APPLE = platform.system() == "Darwin"
MAC_CROSS_COMPILE_INTEL = ARGS.mac_cross_compile_intel
MAC_CROSS_COMPILE_INTEL = "mac-cross-compile-intel" in flags
assert platform.system() == "Darwin" or not MAC_CROSS_COMPILE_INTEL
WASM = ARGS.wasm
WASM = "wasm" in flags
"""Build WASM outside pyodide build environment."""
WASM_CMAKE_IS_USING_INIT_VARS = False
if WASM:
@@ -500,12 +387,12 @@ dependency_tree: dict[str, tuple[str, ...]] = {
def gather_dependencies(dep: str) -> Generator[str]:
yield dep
for d in dependency_tree[dep]:
if d.lower() not in DYNAMIC_ARGS.without:
if f"without-{d.lower()}" not in flags:
for x in gather_dependencies(d):
yield x
if ARGS.verbose:
if VERBOSE:
logger.setLevel(logging.DEBUG)
formatter = logging.Formatter("%(asctime)s - %(levelname)s - %(message)s")
ch.setFormatter(formatter)
@@ -526,26 +413,57 @@ else:
MAC_CROSS_COMPILE_INTEL_AUTOCONF_HOST_ARGS = []
OFF_ON = ["OFF", "ON"]
BUILD_STATIC = not ARGS.shared
BUILD_STATIC = "shared" not in flags
"""Whether dependencies are built static."""
IFCOPENSHELL_STATIC = BUILD_STATIC and "ifcopenshell-shared" not in flags
"""Whether IfcOpenShell's own libraries are built static."""
OCCT_STATIC = BUILD_STATIC and "opencascade-shared" not in flags
"""Whether OCCT is built static. See ``-opencascade-shared``."""
ENABLE_FLAG = "--enable-static" if BUILD_STATIC else "--enable-shared"
DISABLE_FLAG = "--disable-shared" if BUILD_STATIC else "--disable-static"
LINK_TYPE = "static" if BUILD_STATIC else "shared"
LINK_TYPE_UCFIRST = LINK_TYPE.capitalize()
LIBRARY_EXT = "a" if BUILD_STATIC else "so"
PIC = "-fPIC" if BUILD_STATIC else ""
if DYNAMIC_ARGS.py_versions:
PYTHON_VERSIONS = [pyv for pyv in PYTHON_VERSIONS if "".join(pyv.split(".")[:2]) in DYNAMIC_ARGS.py_versions]
if any(f.startswith("py-") for f in flags):
PYTHON_VERSIONS = [pyv for pyv in PYTHON_VERSIONS if f"py-{''.join(pyv.split('.')[:2])}" in flags]
if DYNAMIC_ARGS.occt_version is not None:
OCCT_VERSION = DYNAMIC_ARGS.occt_version
if any(f.startswith("occt-") for f in flags):
OCCT_VERSION = next(f.split("-", 1)[1] for f in flags if f.startswith("occt-"))
# Static and shared OCCT installs are not interchangeable, so they must not share
# a directory: `build_dependency` skips a dependency whose install dir already
# exists, and cache_dependencies.py keys its tarballs purely on that directory
# name. Without the suffix a cached static OCCT silently satisfies a shared build
# (and vice versa) and the requested link type is never applied.
OCCT_DIR_NAME = f"occt-{OCCT_VERSION}" + ("" if OCCT_STATIC else "-shared")
if not OCCT_STATIC:
# A shared OCCT has to be resolvable at run time by everything this script
# executes out of the install tree -- most visibly the post-build
# `import ifcopenshell` sanity check, which otherwise dies with
# "libTKernel.so.7.8: cannot open shared object file". Nothing points there:
# IfcOpenShell's libraries get INSTALL_RPATH=$ORIGIN (see SET_INSTALL_SELF_RPATH
# in cmake/utilities.cmake) and OCCT lives in its own dependency prefix.
#
# This is a build-time concern only. The shipped packages do not rely on it:
# the workflows stage libTK*.so* next to the payload and patchelf an $ORIGIN
# RUNPATH onto every staged library.
_occt_lib_dirs = [
os.path.join(DEPS_DIR, "install", OCCT_DIR_NAME, libdir)
for libdir in ("lib", "lib64")
]
os.environ["LD_LIBRARY_PATH"] = os.pathsep.join(
[*_occt_lib_dirs, os.environ.get("LD_LIBRARY_PATH", "")]
).rstrip(os.pathsep)
if explicit_targets:
targets = {dep for target in explicit_targets for dep in gather_dependencies(target)}
else:
targets = set(dependency_tree.keys())
targets = set(t for t in targets if t.lower() not in DYNAMIC_ARGS.without)
targets = set(t for t in targets if "without-%s" % t.lower() not in flags)
if not explicit_targets and not BUILD_BONSAIVIEWER:
targets.difference_update({"BonsaiViewer", "qt6"})
if BUILD_BONSAIVIEWER:
@@ -628,7 +546,7 @@ def restore_env(var_name: str, old_value: str | None) -> None:
os.environ[var_name] = old_value
def run(cmds: Sequence[str], cwd: str | None = None, can_fail: bool = False, env: dict[str, str] | None = None) -> str:
def run(cmds: Sequence[str], cwd: str | None = None, can_fail: bool = False) -> str:
"""
Wraps `subprocess.Popen.communicate()` and logs the command being executed,
sets up logging `stderr` to `LOG_FILE` (in append mode) and returns stdout
@@ -652,7 +570,7 @@ def run(cmds: Sequence[str], cwd: str | None = None, can_fail: bool = False, env
# Ensure both live logs available in the log file
# and the putput.
with open(LOG_FILE, "a", encoding="utf-8") as log_file_handle:
proc = sp.Popen(cmds, cwd=cwd, stdout=sp.PIPE, stderr=sp.PIPE, encoding="utf-8", env=env)
proc = sp.Popen(cmds, cwd=cwd, stdout=sp.PIPE, stderr=sp.PIPE, encoding="utf-8")
assert proc.stdout and proc.stderr
t_out = threading.Thread(target=stream_reader, args=(proc.stdout, stdout, log_file_handle))
@@ -938,7 +856,7 @@ def build_dependency(
)
logger.info(f"\rInstalled {name} \n")
if ARGS.diskcleanup:
if DISK_CLEANUP:
shutil.rmtree(build_dir, ignore_errors=True)
@@ -1044,8 +962,6 @@ ADDITIONAL_ARGS_STR = " ".join(ADDITIONAL_ARGS)
CXXFLAGS_MINIMAL = f"{CXXFLAGS} {PIC} {ADDITIONAL_ARGS_STR}"
CFLAGS_MINIMAL = f"{CFLAGS} {PIC} {ADDITIONAL_ARGS_STR}"
CXXFLAGS_SHARED = CXXFLAGS_MINIMAL
CFLAGS_SHARED = CFLAGS_MINIMAL
if WASM:
# WASM `SIDE_MODULE_` are absorbed by `emcmake` automatically.
CXXFLAGS = CXXFLAGS_MINIMAL
@@ -1055,19 +971,19 @@ elif sp.call([bash, "-c", "ld --gc-sections 2>&1 | grep -- --gc-sections &> /dev
CXXFLAGS = f"{CXXFLAGS} {PIC} -fdata-sections -ffunction-sections -fvisibility=hidden -fvisibility-inlines-hidden {ADDITIONAL_ARGS_STR}"
CFLAGS = f"{CFLAGS} {PIC} -fdata-sections -ffunction-sections -fvisibility=hidden {ADDITIONAL_ARGS_STR}"
else:
CXXFLAGS = CXXFLAGS_SHARED
CFLAGS = CFLAGS_SHARED
CXXFLAGS = CXXFLAGS_MINIMAL
CFLAGS = CFLAGS_MINIMAL
LDFLAGS = f"{LDFLAGS} -Wl,--gc-sections {ADDITIONAL_ARGS_STR}"
else:
if BUILD_STATIC:
CXXFLAGS = f"{CXXFLAGS} {PIC} -fvisibility=hidden -fvisibility-inlines-hidden {ADDITIONAL_ARGS_STR}"
CFLAGS = f"{CFLAGS} {PIC} -fvisibility=hidden -fvisibility-inlines-hidden {ADDITIONAL_ARGS_STR}"
else:
CXXFLAGS = CXXFLAGS_SHARED
CFLAGS = CFLAGS_SHARED
CXXFLAGS = CXXFLAGS_MINIMAL
CFLAGS = CFLAGS_MINIMAL
LDFLAGS = f"{LDFLAGS} {ADDITIONAL_ARGS_STR}"
if ARGS.lto:
if LTO:
for f in compiler_flags:
locals()[f] += f" -flto={IFCOS_NUM_BUILD_PROCS}"
@@ -1150,9 +1066,6 @@ if "swig" in targets:
if USE_OCCT and "occ" in targets:
occt_args: list[str] = []
patches: list[str] = []
occt_link_type = "Shared" if ARGS.occt_shared else "Static"
occt_name = f"occt-shared-{OCCT_VERSION}" if ARGS.occt_shared else f"occt-{OCCT_VERSION}"
OCCT_INSTALL_PATH = f"{DEPS_DIR}/install/{occt_name}"
if OCCT_VERSION < "7.4":
patches.append("./patches/occt/enable-exception-handling.patch")
@@ -1167,23 +1080,34 @@ if USE_OCCT and "occ" in targets:
if WASM:
patches.append("./patches/occt/no_em_js.patch")
if ARGS.occt_shared:
# Using static flags for shared builds break it
# (e.g. `-fvisibility=hidden` hides many symbols).
# So we temporarily override flags.
OLD_CPP_FLAGS = os.environ["CPPFLAGS"]
OLD_CXX_FLAGS = os.environ["CXXFLAGS"]
OLD_C_FLAGS = os.environ["CFLAGS"]
os.environ["CXXFLAGS"] = CXXFLAGS_SHARED
os.environ["CPPFLAGS"] = CXXFLAGS_SHARED
os.environ["CFLAGS"] = CFLAGS_SHARED
if not OCCT_STATIC:
# BUILD_STATIC drives three things at once: the dependency link type,
# -fvisibility=hidden, and BUILD_SHARED_LIBS. Building only OCCT shared
# means overriding all three for it, not just the link type.
#
# Visibility matters most. OCCT's Standard_EXPORT expands to nothing on
# Unix (Standard_Macro.hxx), so it relies on default visibility to export
# its API. Built shared under -fvisibility=hidden it exports almost
# nothing and its libraries fail to resolve against each other -- e.g.
# libTKMath.so cannot find NCollection_BaseAllocator::CommonBaseAllocator
# in libTKernel.so. Static archives are immune, which is why this only
# appears once OCCT goes shared. CXXFLAGS_MINIMAL is the pre-visibility
# flag set, so this restores default visibility without dropping -O3/-fPIC.
#
# These come after the generic flags in the cmake command line, and the
# last -D for a given variable wins.
occt_args.append(f"-DCMAKE_CXX_FLAGS={CXXFLAGS_MINIMAL}")
occt_args.append(f"-DCMAKE_C_FLAGS={CFLAGS_MINIMAL}")
# Suppresses the generic -DBUILD_SHARED_LIBS=OFF that BUILD_STATIC would
# otherwise add, which contradicts BUILD_LIBRARY_TYPE=Shared.
occt_args.append("-DBUILD_SHARED_LIBS=ON")
build_dependency(
name=occt_name,
name=OCCT_DIR_NAME,
mode="cmake",
build_tool_args=[
f"-DINSTALL_DIR={OCCT_INSTALL_PATH}",
f"-DBUILD_LIBRARY_TYPE={occt_link_type}",
f"-DINSTALL_DIR={DEPS_DIR}/install/{OCCT_DIR_NAME}",
f"-DBUILD_LIBRARY_TYPE={'Static' if OCCT_STATIC else 'Shared'}",
f"-DBUILD_MODULE_Draw=0",
f"-DBUILD_RELEASE_DISABLE_EXCEPTIONS=Off",
# Disable xlib explicitly, as it tries to use it on Desktop Ubuntu, adding unnecessary dependency.
@@ -1202,11 +1126,6 @@ if USE_OCCT and "occ" in targets:
patch=patches,
revision="V" + OCCT_VERSION.replace(".", "_"),
)
if ARGS.occt_shared:
restore_env("CPPFLAGS", OLD_CPP_FLAGS)
restore_env("CXXFLAGS", OLD_CXX_FLAGS)
restore_env("CFLAGS", OLD_C_FLAGS)
elif "occ" in targets:
build_dependency(
name=f"oce-{OCE_VERSION}",
@@ -1285,7 +1204,13 @@ if "OpenCOLLADA" in targets:
# OpenCOLLADAConfig.cmake.in hardcodes shared-lib targets on Unix regardless of
# whether shared libs were actually built. We make it follow `USE_SHARED` instead.
patches.append("./patches/opencollada/config_select_libs_by_use_shared.patch")
patches.append("./patches/opencollada/remove_tr1.patch")
if WASM:
# This is necessary for the WASM build, because recent versions of
# clang don't have the tr1:: namespace anymore. However, it breaks
# some versions of gcc (9.4.0 at least) due to specializing std::hash
# outside of the std:: namespace.
patches.append("./patches/opencollada/remove_tr1.patch")
build_dependency(
"OpenCOLLADA",
@@ -1310,14 +1235,6 @@ if "OpenCOLLADA" in targets:
revision=OPENCOLLADA_VERSION,
)
def python_consider_rc(python_version: str) -> str:
# TODO: remove after Python 3.15 release.
if python_version == "3.15.0":
python_version += "rc1"
return python_version
if "python" in targets and not USE_CURRENT_PYTHON_VERSION and not WASM:
# Python should not be built with -fvisibility=hidden, from experience that introduces segfaults
OLD_CPP_FLAGS = os.environ["CPPFLAGS"]
@@ -1346,16 +1263,13 @@ if "python" in targets and not USE_CURRENT_PYTHON_VERSION and not WASM:
PYTHON_CONFIGURE_ARGS.extend(["--with-universal-archs=intel-64", "--enable-universalsdk"])
for PYTHON_VERSION in PYTHON_VERSIONS:
python_version_url = PYTHON_VERSION
PYTHON_VERSION = python_consider_rc(PYTHON_VERSION)
# Don't fail silently on missing Python dependencies (e.g. openssl or zlib),
# because later ifcopenshell-python build will fail too but in a more confusing way.
build_dependency(
f"python-{PYTHON_VERSION}",
"autoconf",
PYTHON_CONFIGURE_ARGS,
f"http://www.python.org/ftp/python/{python_version_url}/",
f"http://www.python.org/ftp/python/{PYTHON_VERSION}/",
f"Python-{PYTHON_VERSION}.tgz",
)
python_install = INSTALL_DIR / f"python-{PYTHON_VERSION}"
@@ -1574,7 +1488,7 @@ if "qt6" in targets:
cecho("Building IfcOpenShell:", GREEN)
IFCOS_DIR = os.path.join(DEPS_DIR, "build", "ifcopenshell")
if not is_on_off(os.getenv("NO_CLEAN"), default=False):
if os.environ.get("NO_CLEAN", "").lower() not in {"1", "on", "true"}:
if os.path.exists(IFCOS_DIR):
shutil.rmtree(IFCOS_DIR)
os.makedirs(IFCOS_DIR, exist_ok=True)
@@ -1584,8 +1498,8 @@ os.makedirs(ifcos_build_dir, exist_ok=True)
cmake_args = [
"-DUSE_MMAP=OFF",
f"-DBUILD_EXAMPLES={OFF_ON[ARGS.build_examples]}",
"-DBUILD_SHARED_LIBS=" + OFF_ON[ARGS.ifcopenshell_shared],
f"-DBUILD_EXAMPLES={OFF_ON[BUILD_EXAMPLES]}",
"-DBUILD_SHARED_LIBS=" + OFF_ON[not IFCOPENSHELL_STATIC],
"-DGLTF_SUPPORT=ON",
"-DBoost_NO_BOOST_CMAKE=On",
"-DCREATE_BUNDLE=On",
@@ -1630,7 +1544,7 @@ if "cgal" in targets:
cmake_args.append(f"-DCGAL_WITH_GMPXX=Off")
if "occ" in targets and USE_OCCT:
cmake_args_prefix_path.append(OCCT_INSTALL_PATH)
cmake_args_prefix_path.append(f"{DEPS_DIR}/install/{OCCT_DIR_NAME}")
elif "occ" in targets:
# We don't support find_package for OCE.
@@ -1695,7 +1609,6 @@ ifcos_build_args = [
f"-DBUILD_CONVERT={OFF_ON['IfcConvert' in targets]}",
f"-DBUILD_BONSAIVIEWER={OFF_ON['BonsaiViewer' in targets]}",
f"-DCMAKE_INSTALL_PREFIX={DEPS_DIR}/install/ifcopenshell",
"-DUSE_CCACHE=ON",
]
if not WASM and (
@@ -1717,42 +1630,6 @@ if not WASM and (
run([make, f"-j{IFCOS_NUM_BUILD_PROCS}", "VERBOSE=1"], cwd=ifcos_build_dir)
run([make, "install/strip" if BUILD_CFG == "Release" else "install"], cwd=ifcos_build_dir)
def test_examples() -> None:
cecho("Running examples...", GREEN)
examples_bin_dir = Path(DEPS_DIR) / "install" / "ifcopenshell" / "bin"
examples_env = os.environ.copy()
ld_library_paths = ["../lib"]
if ARGS.occt_shared:
ld_library_paths.append(f"{OCCT_INSTALL_PATH}/lib")
examples_env["LD_LIBRARY_PATH"] = os.pathsep.join(ld_library_paths)
examples: dict[tuple[str, ...], str | None] = {
("./IfcOpenHouse",): "IfcOpenHouse.ifc",
("./IfcParseExamples", "IfcOpenHouse.ifc"): None,
("./IfcAdvancedHouse",): "IfcAdvancedHouse.ifc",
}
# Only for ifc4x3 schema.
if (examples_bin_dir / "IfcAlignment").is_file():
examples[("./IfcAlignment",)] = "FHWA_Bridge_Geometry_Alignment_Example.ifc"
examples[("./IfcSimplifiedAlignment",)] = "FHWA_Bridge_Geometry_Alignment_Example_Simplified.ifc"
produced_files: set[str] = set()
try:
for cmd, expected_file in examples.items():
run(cmd, cwd=str(examples_bin_dir), env=examples_env)
if expected_file is None:
continue
if not (examples_bin_dir / expected_file).is_file():
raise RuntimeError(f"Example `{' '.join(cmd)}` did not produce expected file '{expected_file}'.")
produced_files.add(expected_file)
finally:
for produced_file in produced_files:
(examples_bin_dir / produced_file).unlink(missing_ok=True)
if ARGS.build_examples:
test_examples()
if "IfcOpenShell-Python" in targets:
wrapper_ldflags = ""
if platform.system() == "Darwin":
@@ -1804,7 +1681,8 @@ if "IfcOpenShell-Python" in targets:
*([f"-DPYTHON_MODULE_INSTALL_DIR={REPO_PATH}"] * WASM),
f"-DPYTHON_INCLUDE_DIR={python_include}",
f"-DCMAKE_INSTALL_PREFIX={DEPS_DIR}/install/ifcopenshell/tmp",
"-DUSERSPACE_PYTHON_PREFIX=" + OFF_ON[PYTHON_USER_SITE],
"-DUSERSPACE_PYTHON_PREFIX="
+ ["Off", "On"][os.environ.get("PYTHON_USER_SITE", "").lower() in {"1", "on", "true"}],
],
cmake_dir=CMAKE_DIR,
cwd=ifcos_build_dir,
@@ -1857,7 +1735,6 @@ if "IfcOpenShell-Python" in targets:
compile_python_wrapper(platform.python_version(), python_info["include"], sys.executable)
else:
for python_version in PYTHON_VERSIONS:
python_version = python_consider_rc(python_version)
python_path = INSTALL_DIR / f"python-{python_version}"
module_dir = compile_python_wrapper(python_version, python_path=python_path)
assert module_dir
-469
View File
@@ -1,469 +0,0 @@
#!/usr/bin/env -S uv run --script
# /// script
# ///
import argparse
import logging
import os
import platform
import re
import shlex
import shutil
import subprocess
from pathlib import Path
from typing import Literal, NamedTuple
class C:
GREY = "\033[90m"
YELLOW = "\033[33m"
RED = "\033[31m"
RESET = "\033[0m"
class ColorFormatter(logging.Formatter):
COLORS = {
logging.DEBUG: C.GREY,
logging.WARNING: C.YELLOW,
logging.ERROR: C.RED,
}
def format(self, record: logging.LogRecord) -> str:
color = self.COLORS.get(record.levelno, C.RESET)
return f"{color}{super().format(record)}{C.RESET}"
handler = logging.StreamHandler()
handler.setFormatter(ColorFormatter("%(message)s"))
logging.basicConfig(level=logging.INFO, handlers=[handler])
logger = logging.getLogger(__name__)
def run(
*cmd: str,
cwd: Path | None = None,
env: dict[str, str] | None = None,
stderr: int | None = None,
) -> str:
logger.debug(f"$ {shlex.join(cmd)}")
return subprocess.check_output(cmd, cwd=cwd, env=env, stderr=stderr, text=True)
REPO_ROOT = Path(run("git", "-C", str(Path(__file__).parent), "rev-parse", "--show-toplevel").strip())
VERSION = "v" + (REPO_ROOT / "VERSION").read_text().strip()
def get_git_sha() -> str:
sha = os.getenv("GITHUB_SHA") or run("git", "rev-parse", "HEAD", cwd=REPO_ROOT).strip()
return sha[:7]
def is_platform(name: Literal["MAC", "LINUX"]) -> bool:
current = "MAC" if platform.system() == "Darwin" else "LINUX"
return current == name
def get_install_dir(arch_suffix: str) -> Path:
if is_platform("MAC"):
pattern = "Darwin/*/*/install"
else:
if "arm64" in arch_suffix:
pattern = "Linux/aarch64/install"
else:
pattern = "Linux/x86_64/install"
for data in (REPO_ROOT / "build").glob(pattern):
return data
raise Exception("No install dir found")
def find_qt_dir(install_root: Path, qt6_version: str) -> Path | None:
for qt_candidate in install_root.glob(f"qt6-{qt6_version}-*/{qt6_version}/*"):
if (qt_candidate / "lib").is_dir():
return qt_candidate
return None
def find_occt_dir(install_root: Path) -> Path:
candidates = [candidate for candidate in install_root.glob("occt-shared-*") if candidate.is_dir()]
if len(candidates) != 1:
raise Exception(f"Expected exactly one OCCT shared candidate, found: {candidates}")
return candidates[0]
def ensure_soname_links(paths: list[Path]) -> None:
"""Ensure that all shared libraries in `paths` are present using their SONAMEs (at least as symlinks)."""
for shared_object in paths:
if not shared_object.is_file():
continue
try:
readelf_output = run("readelf", "-d", str(shared_object))
except subprocess.CalledProcessError:
continue
match = re.search(r"\(SONAME\).*Library soname: \[(.*)\]", readelf_output)
if not match:
continue
soname = match.group(1)
soname_path = shared_object.parent / soname
if soname_path.exists():
continue
soname_path.symlink_to(shared_object.name)
def is_shared_library(path: Path) -> bool:
name = path.name.lower()
return name.endswith((".so", ".dylib", ".dll")) or ".so." in name
def stage_runtime_payload(install_dir: Path, dest: Path, *, include_geometry_writers: bool = True) -> None:
"""Copy all libs from `install_dir/{bin,lib,lib64}` into `dest`."""
runtime_files = []
for runtime_dir_name in ("bin", "lib", "lib64"):
runtime_dir = install_dir / runtime_dir_name
if not runtime_dir.is_dir():
continue
for runtime_file in runtime_dir.rglob("*"):
if not (runtime_file.is_symlink() or runtime_file.is_file()):
continue
if not is_shared_library(runtime_file):
continue
if not include_geometry_writers and runtime_file.name.startswith("ifcopenshell.geometry.writer."):
continue
dest_file = dest / runtime_file.name
shutil.copy(runtime_file, dest_file, follow_symlinks=False)
runtime_files.append(dest_file)
if not is_platform("MAC"):
ensure_soname_links(runtime_files)
for lib_so in runtime_files:
if lib_so.is_file():
run("patchelf", "--set-rpath", "$ORIGIN", str(lib_so))
def stage_qt_runtime_payload(exe_path: Path, dest: Path, qt_dir: Path | None) -> None:
"""Copy QT libs/plugins from `qt_dir` next to `exe_path`, if it depends on QT."""
def is_so_file(path: Path) -> bool:
return (path.is_file() or path.is_symlink()) and ".so" in path.name
if not qt_dir or not (qt_dir / "lib").is_dir():
return
# Skip executables that don't depend on QT (don't have `libQt6` referenced).
env = os.environ.copy()
env["LD_LIBRARY_PATH"] = f"{qt_dir / 'lib'}:{env.get('LD_LIBRARY_PATH', '')}"
try:
ldd_output = run("ldd", str(exe_path), env=env)
except subprocess.CalledProcessError:
return
if "libQt6" not in ldd_output:
return
# Copy all QT libs to `dest`.
qt_lib_files = []
for lib_file in (qt_dir / "lib").iterdir():
if is_so_file(lib_file):
dest_file = dest / lib_file.name
qt_lib_files.append(dest_file)
# Currently we install some qt libs to `install/ifcopenshell/lib` too,
# so there's a bit of overlap beteen stage_runtime and stage_qt_runtime,
# hence the skip.
if dest_file.exists():
continue
shutil.copy(lib_file, dest_file, follow_symlinks=False)
ensure_soname_links(qt_lib_files)
# Copy QT plugins.
plugins_dir = qt_dir / "plugins"
if plugins_dir.is_dir():
for plugin_file in plugins_dir.rglob("*"):
if not is_so_file(plugin_file):
continue
dest_plugin_file = dest / "plugins" / plugin_file.relative_to(plugins_dir)
dest_plugin_file.parent.mkdir(parents=True, exist_ok=True)
shutil.copy(plugin_file, dest_plugin_file, follow_symlinks=False)
# Point plugins rpath to `dest`.
dest_plugins_dir = dest / "plugins"
if dest_plugins_dir.is_dir():
for plugin_so in dest_plugins_dir.rglob("*.so*"):
if plugin_so.is_file():
run("patchelf", "--set-rpath", "$ORIGIN/../..:$ORIGIN", str(plugin_so))
# Non-recursive, set rpath only for top-level libs.
for lib_so in qt_lib_files:
if lib_so.is_file():
run("patchelf", "--set-rpath", "$ORIGIN", str(lib_so))
qt_conf_path = dest / "qt.conf"
qt_conf_path.write_text("[Paths]\nPrefix = .\n")
KNOWN_EXCEPTIONS = frozenset(
(
# Optional Qt SQL driver plugins we don't ship the client libs for.
"libqsqlpsql.so",
"libqsqlmysql.so",
"libqsqlmimer.so",
"libqsqlodbc.so",
)
)
def check_runtime_dependencies(package_dir: Path) -> None:
"""Check all binaries in `package_dir` and report if they're still missing dependencies or are static."""
def is_executable_or_so(path: Path) -> bool:
name = path.name
return os.access(path, os.X_OK) or name.endswith(".so") or ".so." in name
missing = False
env = os.environ.copy()
env.pop("LD_LIBRARY_PATH", None)
for binary_file in package_dir.rglob("*"):
if not binary_file.is_file() or not is_executable_or_so(binary_file):
continue
# Skip non-binaries.
try:
run("readelf", "-h", str(binary_file), stderr=subprocess.DEVNULL)
except subprocess.CalledProcessError:
continue
try:
ldd_output = run("ldd", str(binary_file), env=env, stderr=subprocess.STDOUT)
except subprocess.CalledProcessError as e:
logger.error(f"ldd failed for {binary_file}")
logger.error(e.output)
missing = True
continue
if "not found" in ldd_output:
is_known = binary_file.name in KNOWN_EXCEPTIONS
log = logger.debug if is_known else logger.warning
log(f"Missing runtime dependencies for {binary_file}")
for line in ldd_output.splitlines():
if "not found" in line:
log(line)
if not is_known:
missing = True
# TODO: should error?
if missing:
logger.warning("Runtime dependency check found issues; continuing packaging.")
def package_python_wrapper(
py_dir: Path,
ifcopenshell_install_dir: Path,
github_sha: str,
output_dir: Path,
arch_suffix: str,
occt_dir: Path | None,
) -> None:
logger.info(f"Packaging python wrapper '{py_dir.name}'")
py_version = py_dir.name
postfix = "" if py_version[-1].isdigit() else py_version[-1]
# Match and convert `x.y` -> `xy`.
version_match = re.search(r"[0-9]+\.[0-9]+", py_version)
assert version_match
numbers = "".join(version_match.group().split("."))
py_version_major = f"python-{numbers}{postfix}"
package_dir = ifcopenshell_install_dir / f".package-{py_version_major}"
if package_dir.exists():
# Clean up previous local runs.
shutil.rmtree(package_dir)
package_dir.mkdir(parents=True)
ifcopenshell_dir = package_dir / "ifcopenshell"
ifcopenshell_dir.mkdir()
for item in py_dir.iterdir():
dest = ifcopenshell_dir / item.name
if item.is_dir():
shutil.copytree(item, dest, symlinks=True)
else:
shutil.copy(item, dest, follow_symlinks=False)
if not is_platform("MAC"):
for lib_so in ifcopenshell_dir.glob("*.so*"):
if lib_so.is_file():
run("patchelf", "--set-rpath", "$ORIGIN", str(lib_so))
# Cache from test run during build.
pycache_dir = ifcopenshell_dir / "__pycache__"
if pycache_dir.is_dir():
shutil.rmtree(pycache_dir)
for pyc_file in ifcopenshell_dir.rglob("*.pyc"):
pyc_file.unlink()
# TODO: packs qt libs also?
stage_runtime_payload(ifcopenshell_install_dir, ifcopenshell_dir)
if occt_dir:
stage_runtime_payload(occt_dir, ifcopenshell_dir)
if not is_platform("MAC"):
check_runtime_dependencies(ifcopenshell_dir)
zip_path = output_dir / f"ifcopenshell-{py_version_major}-{VERSION}-{github_sha}-{arch_suffix}.zip"
run("zip", "-y", "-r", "-qq", "-1", str(zip_path), "ifcopenshell", cwd=package_dir)
shutil.rmtree(package_dir)
def is_packageable_executable(path: Path) -> bool:
if not path.is_file() or not os.access(path, os.X_OK):
return False
return not (path.name.lower().endswith(".zip") or is_shared_library(path))
def package_executable(
exe_path: Path,
ifcopenshell_install_dir: Path,
github_sha: str,
output_dir: Path,
autodesk_connector_dir: Path,
qt_dir: Path | None,
occt_dir: Path | None,
arch_suffix: str,
) -> None:
exe = exe_path.name
logger.info(f"Packaging executable '{exe}'")
package_dir = ifcopenshell_install_dir / f".package-{exe}"
if package_dir.exists():
# Clean up previous local runs.
shutil.rmtree(package_dir)
package_dir.mkdir(parents=True)
shutil.copy(exe_path, package_dir / exe)
# TODO: kept `is_platform(MAC)` to retain original bash script behaviour,
# but is this guard needed or it should be always False?
stage_runtime_payload(ifcopenshell_install_dir, package_dir, include_geometry_writers=is_platform("MAC"))
if occt_dir:
stage_runtime_payload(occt_dir, package_dir)
# On macOS, rpath is already set at build time via CMake's INSTALL_RPATH, and
# QT apps are packaged as .app bundles (`package_app_bundle`) instead.
if not is_platform("MAC"):
run("patchelf", "--set-rpath", "$ORIGIN", str(package_dir / exe))
stage_qt_runtime_payload(exe_path, package_dir, qt_dir)
if exe == "BonsaiViewer":
connectors_dir = package_dir / "connectors"
connectors_dir.mkdir()
shutil.copytree(autodesk_connector_dir, connectors_dir / autodesk_connector_dir.name, symlinks=True)
check_runtime_dependencies(package_dir)
zip_path = output_dir / f"{exe}-{VERSION}-{github_sha}-{arch_suffix}.zip"
run("zip", "-y", "-qq", "-r", str(zip_path), ".", cwd=package_dir)
shutil.rmtree(package_dir)
def package_app_bundle(
app_path: Path,
install_root: Path,
github_sha: str,
output_dir: Path,
autodesk_connector_dir: Path,
arch_suffix: str,
) -> None:
"""Zip a `.app` bundle (e.g. BonsaiViewer.app) living at the install-prefix root.
Their install rule uses `BUNDLE DESTINATION "."` - that's the layout Qt's
macdeployqt expects. macdeployqt has already embedded the Qt frameworks
inside each bundle during install/strip, so the only thing left to stage
is the connector.
"""
app = app_path.stem
logger.info(f"Packaging app bundle '{app}'")
if app == "BonsaiViewer":
# ConnectorDiscovery looks in applicationDirPath()/connectors,
# which for a bundle is Contents/MacOS.
connectors_dir = app_path / "Contents" / "MacOS" / "connectors"
connectors_dir.mkdir(parents=True)
shutil.copytree(autodesk_connector_dir, connectors_dir / autodesk_connector_dir.name, symlinks=True)
zip_path = output_dir / f"{app}-{VERSION}-{github_sha}-{arch_suffix}.zip"
run("zip", "-qq", "-r", str(zip_path), app_path.name, cwd=install_root)
ARCH_SUFFIXES = ("linux64", "linuxarm64", "macosm164")
LOG_LEVELS = ("DEBUG", "INFO", "WARNING", "ERROR")
class Args(NamedTuple):
arch_suffix: str
log_level: str
occt_shared: bool
ARGS: Args
def main() -> None:
parser = argparse.ArgumentParser()
parser.add_argument("arch_suffix", choices=ARCH_SUFFIXES, help="Zip filename suffix.")
# TODO: relax default to INFO once things get more stable.
parser.add_argument("--log-level", default="DEBUG", choices=LOG_LEVELS, help="Logging verbosity.")
# TODO: add `--shared`.
parser.add_argument("--occt-shared", action="store_true", help="OCCT was built as shared libraries.")
args = parser.parse_args()
global ARGS
ARGS = Args(arch_suffix=args.arch_suffix, log_level=args.log_level, occt_shared=args.occt_shared)
logger.setLevel(ARGS.log_level)
# bonsaiviewer-autodesk is now a Rust connector. packaging/build.py
# invokes `cargo build --release` and stages the binary +
# connector.json into dist/autodesk/. Same on-disk shape as the
# old PyInstaller flow so the symlink + zip steps below
# continue to work unchanged.
run("uv", "run", str(REPO_ROOT / "src/bonsaiviewer-autodesk/packaging/build.py"))
autodesk_connector_dir = REPO_ROOT / "src/bonsaiviewer-autodesk/dist/autodesk"
assert autodesk_connector_dir.is_dir()
# Locate the ifcopenshell install dir and stage QT6 alongside the zip output.
install_root = get_install_dir(ARGS.arch_suffix)
ifcopenshell_install_dir = install_root / "ifcopenshell"
output_dir = Path.home() / "output"
output_dir.mkdir(parents=True, exist_ok=True)
qt6_version = os.getenv("QT6_VERSION", "6.8.3")
qt_dir_env = os.getenv("QT_DIR")
qt_dir = Path(qt_dir_env) if qt_dir_env else find_qt_dir(install_root, qt6_version)
occt_dir = find_occt_dir(install_root) if ARGS.occt_shared else None
# Iterate over all built Python wrappers in `install/ifcopenshell/python-x.y.z`
# and zip them, bundling all dynamic libs from `lib`.
github_sha = get_git_sha()
for py_dir in sorted(ifcopenshell_install_dir.glob("python-*")):
package_python_wrapper(py_dir, ifcopenshell_install_dir, github_sha, output_dir, ARGS.arch_suffix, occt_dir)
# Iterate over all executables in `install/ifcopenshell/bin` and zip them.
# Each zip bundles dynamic libs from `lib` and also qt libs.
bin_dir = ifcopenshell_install_dir / "bin"
for exe_path in sorted(bin_dir.iterdir()):
if is_packageable_executable(exe_path):
package_executable(
exe_path,
ifcopenshell_install_dir,
github_sha,
output_dir,
autodesk_connector_dir,
qt_dir,
occt_dir,
ARGS.arch_suffix,
)
if is_platform("MAC"):
for app_path in sorted(install_root.glob("*.app")):
package_app_bundle(app_path, install_root, github_sha, output_dir, autodesk_connector_dir, ARGS.arch_suffix)
if __name__ == "__main__":
main()
+4 -18
View File
@@ -1,13 +1,3 @@
# Removing use of `tr1` namespace that might not be available on some systems.
#
# Current status on different systems:
# - msvc - removed `tr1` namespace in 14.51 (`_MSC_VER == 1951`)
# - gcc (with libstdc++) - currently neither deprecated nor removed, though there are plans to
# - clang (with libc++) - never had it
#
# One of the hunks in the patch is patching `_MSC_VER == 1500`, so it's not stricly needed,
# but kept it just so it will be easy to check the absense of any `tr1` use.
diff --git a/COLLADABaseUtils/include/COLLADABUhash_map.h b/COLLADABaseUtils/include/COLLADABUhash_map.h
index 8ab0fb9b..12503bfb 100644
--- a/COLLADABaseUtils/include/COLLADABUhash_map.h
@@ -37,13 +27,11 @@ index 8ab0fb9b..12503bfb 100644
- #define COLLADABU_HASH_MAP std::tr1::unordered_map
- #define COLLADABU_HASH_MULTIMAP std::tr1::unordered_multimap
- #define COLLADABU_HASH_SET std::tr1::unordered_set
- #define COLLADABU_HASH_NAMESPACE_OPEN std { namespace tr1
- #define COLLADABU_HASH_NAMESPACE_CLOSE }
+ #define COLLADABU_HASH_MAP std::unordered_map
+ #define COLLADABU_HASH_MULTIMAP std::unordered_multimap
+ #define COLLADABU_HASH_SET std::unordered_set
+ #define COLLADABU_HASH_NAMESPACE_OPEN std
+ #define COLLADABU_HASH_NAMESPACE_CLOSE
#define COLLADABU_HASH_NAMESPACE_OPEN std { namespace tr1
#define COLLADABU_HASH_NAMESPACE_CLOSE }
#define COLLADABU_HASH_FUN hash
@@ -107,12 +107,12 @@
#define COLLADABU_HASH_NAMESPACE_CLOSE
@@ -57,13 +45,11 @@ index 8ab0fb9b..12503bfb 100644
- #define COLLADABU_HASH_MAP std::tr1::unordered_map
- #define COLLADABU_HASH_MULTIMAP std::tr1::unordered_multimap
- #define COLLADABU_HASH_SET std::tr1::unordered_set
- #define COLLADABU_HASH_NAMESPACE_OPEN std { namespace tr1
- #define COLLADABU_HASH_NAMESPACE_CLOSE }
+ #define COLLADABU_HASH_MAP std::unordered_map
+ #define COLLADABU_HASH_MULTIMAP std::unordered_multimap
+ #define COLLADABU_HASH_SET std::unordered_set
+ #define COLLADABU_HASH_NAMESPACE_OPEN std
+ #define COLLADABU_HASH_NAMESPACE_CLOSE
#define COLLADABU_HASH_NAMESPACE_OPEN std { namespace tr1
#define COLLADABU_HASH_NAMESPACE_CLOSE }
#define COLLADABU_HASH_FUN hash
diff --git a/common/libBuffer/include/CommonFWriteBufferFlusher.h b/common/libBuffer/include/CommonFWriteBufferFlusher.h
index c7af45b2..fac4f133 100644
+1 -1
View File
@@ -9,7 +9,7 @@ source:
build:
script: |
BUILD_CFG=Release python nix/build-all.py -v --wasm
BUILD_CFG=Release python nix/build-all.py -v --wasm --py313
about:
home: http://ifcopenshell.org
-3
View File
@@ -113,7 +113,6 @@ unresolved-attribute = "ignore"
invalid-argument-type = "ignore"
invalid-method-override = "ignore"
invalid-assignment = "ignore"
unsound-assignment = "ignore"
invalid-parameter-default = "ignore"
missing-override-decorator = "ignore"
invalid-yield = "ignore"
@@ -127,8 +126,6 @@ no-matching-overload = "ignore"
not-subscriptable = "ignore"
unsupported-dynamic-base = "ignore"
unsupported-operator = "ignore"
# `@persistent` is incorrectly annotated as `Any` in fake-bpy, needs to be resolved upstream.
dynamic-function-decorator-return = "ignore"
[tool.ty.environment]
extra-paths = [
+3 -3
View File
@@ -1,5 +1,5 @@
black==26.5.1
ruff==0.16.4
black==26.3.1
ruff==0.16.0
poethepoet
ty==0.0.74
ty==0.0.72
gersemi==0.28.0
+23 -4
View File
@@ -69,17 +69,32 @@ endif # def PYVERSION
IFCMERGE_VERSION:=2026-04-07
ifdef PLATFORM
SUPPORTED_PLATFORMS := linux macosm1 win
SUPPORTED_PLATFORMS := linux macos macosm1 win
ifeq ($(filter $(PLATFORM),$(SUPPORTED_PLATFORMS)),)
$(error Unsupported PLATFORM=$(PLATFORM). Must be one of $(SUPPORTED_PLATFORMS))
endif
ifeq ($(PLATFORM),macos)
ifeq ($(PYVERSION),py313)
$(error Blender 5.1 with Python 3.13 doesn't support intel macOS.)
endif
endif
ifeq ($(PLATFORM), linux)
PYPI_PLATFORM:=--platform manylinux_2_17_x86_64
BLENDER_PLATFORM:=linux-x64
endif
ifeq ($(PLATFORM), macos)
ifeq ($(PYVERSION), py311)
PYPI_PLATFORM:=--platform macosx_10_10_x86_64
else
PYPI_PLATFORM:=--platform macosx_10_13_x86_64
endif
BLENDER_PLATFORM:=macos-x64
endif
ifeq ($(PLATFORM), macosm1)
PYPI_PLATFORM:=--platform macosx_11_0_arm64
BLENDER_PLATFORM:=macos-arm64
@@ -93,7 +108,7 @@ endif
endif # def PLATFORM
# Current build commit hash.
OLD:=ad113e1
OLD:=3e7b739
.PHONY: bump
bump:
ifndef NEW
@@ -179,8 +194,10 @@ endif
# Provides networkx graph analysis for project dependency calculations
cd build && . env/$(VENV_ACTIVATE) && $(PIP) download networkx --dest=./wheels
# Required by IFCDiff
# Pinned <9.1: deepdiff 9.1.0 adds the compiled dependency cachebox<6,>=5.2,
# which this platformless download cannot provide for every target platform.
# Pinned <9.1: deepdiff 9.1.0 adds cachebox<6,>=5.2 which only ships macOS x86_64
# wheels for macosx_10_12+ and is incompatible with our macos py311 --platform
# macosx_10_10_x86_64 target. Revisit once the macos py311 platform tag is bumped
# to 10_13 (matching py312/py313).
cd build && . env/$(VENV_ACTIVATE) && $(PIP) download "deepdiff<9.1" --dest=./wheels
# Required by IFCCSV and ifcopenshell.util.selector
cd build && . env/$(VENV_ACTIVATE) && $(PIP) download lark --dest=./wheels
@@ -198,6 +215,8 @@ endif
# pyradiance is using different platform versions than defaults in our makefile.
ifeq ($(PLATFORM), linux)
cd build && . env/$(VENV_ACTIVATE) && $(PIP) download pyradiance --platform manylinux_2_28_x86_64 --python-version $(PYPI_VERSION) --implementation $(PYPI_IMP) --only-binary=:all: --dest=./wheels
else ifeq ($(PLATFORM), macos)
cd build && . env/$(VENV_ACTIVATE) && $(PIP) download pyradiance --platform macosx_10_13_x86_64 --python-version $(PYPI_VERSION) --implementation $(PYPI_IMP) --only-binary=:all: --dest=./wheels
else
cd build && . env/$(VENV_ACTIVATE) && $(PIP) download pyradiance $(PYPI_PLATFORM) --python-version $(PYPI_VERSION) --implementation $(PYPI_IMP) --only-binary=:all: --dest=./wheels
endif
+12 -12
View File
@@ -50,13 +50,11 @@ from bonsai.bim.module.drawing.data import refresh as refresh_drawing_data
from bonsai.bim.prop import Attribute, BIMFilterGroup
diagram_scales_enum = []
diagram_scales_enum_system = None
def purge():
global diagram_scales_enum, diagram_scales_enum_system
global diagram_scales_enum
diagram_scales_enum = []
diagram_scales_enum_system = None
def update_target_view_doc(self: "DocProperties", context: bpy.types.Context) -> None:
@@ -125,12 +123,14 @@ def update_is_nts(self: "BIMCameraProperties", context: bpy.types.Context) -> No
def get_diagram_scales(self: "BIMCameraProperties", context: bpy.types.Context) -> list[tuple[str, str, str]]:
global diagram_scales_enum, diagram_scales_enum_system
global diagram_scales_enum
assert context.scene
system = context.scene.unit_settings.system
if len(diagram_scales_enum) < 1 or diagram_scales_enum_system != system:
diagram_scales_enum_system = system
if system == "IMPERIAL":
if (
len(diagram_scales_enum) < 1
or (context.scene.unit_settings.system == "IMPERIAL" and len(diagram_scales_enum) == 13)
or (context.scene.unit_settings.system == "METRIC" and len(diagram_scales_enum) == 31)
):
if context.scene.unit_settings.system == "IMPERIAL":
diagram_scales_enum = [
("CUSTOM", "Custom", ""),
("1'=1'-0\"|1/1", "1'=1'-0\"", ""),
@@ -144,21 +144,21 @@ def get_diagram_scales(self: "BIMCameraProperties", context: bpy.types.Context)
('1/4"=1\'-0"|1/48', '1/4"=1\'-0"', ""),
('3/16"=1\'-0"|1/64', '3/16"=1\'-0"', ""),
('1/8"=1\'-0"|1/96', '1/8"=1\'-0"', ""),
("1\"=10'|1/120", "1\"=10'", ""),
('3/32"=1\'-0"|1/128', '3/32"=1\'-0"', ""),
('1/16"=1\'-0"|1/192', '1/16"=1\'-0"', ""),
('1/32"=1\'-0"|1/384', '1/32"=1\'-0"', ""),
('1/64"=1\'-0"|1/768', '1/64"=1\'-0"', ""),
('1/128"=1\'-0"|1/1536', '1/128"=1\'-0"', ""),
("1\"=10'|1/120", "1\"=10'", ""),
("1\"=20'|1/240", "1\"=20'", ""),
("1\"=30'|1/360", "1\"=30'", ""),
('1/32"=1\'-0"|1/384', '1/32"=1\'-0"', ""),
("1\"=40'|1/480", "1\"=40'", ""),
("1\"=50'|1/600", "1\"=50'", ""),
("1\"=60'|1/720", "1\"=60'", ""),
('1/64"=1\'-0"|1/768', '1/64"=1\'-0"', ""),
("1\"=70'|1/840", "1\"=70'", ""),
("1\"=80'|1/960", "1\"=80'", ""),
("1\"=90'|1/1080", "1\"=90'", ""),
("1\"=100'|1/1200", "1\"=100'", ""),
('1/128"=1\'-0"|1/1536', '1/128"=1\'-0"', ""),
("1\"=150'|1/1800", "1\"=150'", ""),
("1\"=200'|1/2400", "1\"=200'", ""),
("1\"=300'|1/3600", "1\"=300'", ""),
-4
View File
@@ -331,10 +331,6 @@ class Material(bonsai.core.tool.Material):
@classmethod
def get_style(cls, material: ifcopenshell.entity_instance) -> Union[ifcopenshell.entity_instance, None]:
if not material.is_a("IfcMaterial"):
# material may also be an IfcMaterialConstituentSet / IfcMaterialLayerSet /
# IfcMaterialProfileSet / IfcMaterialList, none of which have HasRepresentation.
return None
for material_representation in material.HasRepresentation:
for representation in material_representation.Representations:
for item in representation.Items:
-1
View File
@@ -1,4 +1,3 @@
pytest
pytest-blender
pytest-bdd
fake-bpy-module-latest
+4 -56
View File
@@ -428,8 +428,7 @@ void MainWindow::setupPanels() {
spatial_panel_ = new modules::spatial_hierarchy::SpatialHierarchyPanel(this);
properties_panel_ = new modules::properties::PropertiesPanel(this);
models_view_ = new modules::models::ModelsPanelView(
models_panel_, session_state_, viewport_widget_->viewport(), this);
models_view_ = new modules::models::ModelsPanelView(models_panel_, session_state_, this);
spatial_view_ = new modules::spatial_hierarchy::SpatialHierarchyPanelView(spatial_panel_, session_state_, this);
properties_view_ = new modules::properties::PropertiesPanelView(properties_panel_, session_state_, this);
@@ -482,13 +481,6 @@ void MainWindow::setupStatus() {
status_mode_label_ = new QLabel("Ready", this);
status_selection_label_ = new QLabel("No selection", this);
status_perf_label_ = new QLabel(this);
status_memory_label_ = new QLabel(this);
status_memory_label_->setVisible(false);
status_memory_label_->setToolTip(
"The geometry in view needs more GPU memory than is available, so the "
"viewer keeps the largest on-screen parts resident and streams the rest "
"as you move. Right-click a model in the Models panel and choose "
"\"Unload Model\" to free its GPU memory for the others.");
status_progress_bar_ = new QProgressBar(this);
status_perf_label_->setVisible(AppSettings::instance().showStats());
status_progress_bar_->setMaximumWidth(200);
@@ -497,7 +489,6 @@ void MainWindow::setupStatus() {
statusBar()->setSizeGripEnabled(false);
statusBar()->addWidget(status_mode_label_);
statusBar()->addWidget(status_selection_label_, 1);
statusBar()->addPermanentWidget(status_memory_label_);
statusBar()->addPermanentWidget(status_perf_label_);
statusBar()->addPermanentWidget(status_progress_bar_);
@@ -563,59 +554,16 @@ void MainWindow::setupLoader() {
connect(viewport_widget_->viewport(), &ViewportWindow::frameStatsUpdated, this,
[this](const ViewportWindow::FrameStats& stats) {
const double mb = 1.0 / (1024.0 * 1024.0);
// Missing chunks are normal for a moment after every camera move
// while streaming catches up; only a shortfall that persists means
// the view does not fit, and only that is worth telling the user.
constexpr qint64 kShortfallNoticeMs = 3000;
if (stats.chunks_wanted_missing == 0) {
memory_shortfall_since_.invalidate();
status_memory_label_->setVisible(false);
} else {
if (!memory_shortfall_since_.isValid()) memory_shortfall_since_.start();
if (memory_shortfall_since_.elapsed() >= kShortfallNoticeMs) {
status_memory_label_->setText(
QString("GPU memory full: %1 of %2 visible chunks (%3 MB) not loaded")
.arg(stats.chunks_wanted_missing)
.arg(stats.chunks_wanted)
.arg(double(stats.wanted_missing_bytes) * mb, 0, 'f', 0));
status_memory_label_->setVisible(true);
}
}
if (!status_perf_label_->isVisible()) return;
QString text =
QString("%1 fps | %2 ms | %3/%4 obj | %5/%6 tri | %7 draws | VRAM %8/%9 MB")
status_perf_label_->setText(
QString("%1 fps | %2 ms | %3/%4 obj | %5/%6 tri | %7 draws")
.arg(stats.fps, 0, 'f', 1)
.arg(stats.frame_time_ms, 0, 'f', 1)
.arg(stats.visible_objects)
.arg(stats.total_objects)
.arg(stats.visible_triangles)
.arg(stats.total_triangles)
.arg(stats.gl_draw_calls)
.arg(double(stats.vram_used_bytes) * mb, 0, 'f', 0)
.arg(double(stats.vram_capacity_bytes) * mb, 0, 'f', 0);
// The budget is where the pool may grow to; the pool can also sit
// a sub-buffer above it (a release would undershoot). Show it
// only when it tells the user something capacity does not.
if (stats.vram_budget_bytes > 0
&& stats.vram_budget_bytes != stats.vram_capacity_bytes) {
text += QString(" (budget %1)")
.arg(double(stats.vram_budget_bytes) * mb, 0, 'f', 0);
}
// Device total is only known when a driver backend answered.
if (stats.device_vram_total_bytes > 0) {
text += QString(" | Device %1/%2 MB")
.arg(double(stats.device_vram_used_bytes) * mb, 0, 'f', 0)
.arg(double(stats.device_vram_total_bytes) * mb, 0, 'f', 0);
}
if (stats.chunks_wanted_missing > 0) {
text += QString(" | %1/%2 chunks waiting")
.arg(stats.chunks_wanted_missing)
.arg(stats.chunks_wanted);
}
status_perf_label_->setText(text);
.arg(stats.gl_draw_calls));
});
connect(viewport_widget_->viewport(), &ViewportWindow::objectPicked,
this, [this](uint32_t object_id) {
-5
View File
@@ -26,7 +26,6 @@
#include <QStringList>
class QLabel;
#include <QElapsedTimer>
class QDockWidget;
class QMenu;
class QProgressBar;
@@ -71,10 +70,6 @@ private:
QLabel* status_mode_label_ = nullptr;
QLabel* status_selection_label_ = nullptr;
QLabel* status_perf_label_ = nullptr;
// Shown while the visible geometry persistently exceeds what fits in
// GPU memory (see onFrameStats): the user's cue to unload models.
QLabel* status_memory_label_ = nullptr;
QElapsedTimer memory_shortfall_since_;
QProgressBar* status_progress_bar_ = nullptr;
bonsaiviewer::components::TabBar* ribbon_tabs_ = nullptr;
QStackedWidget* ribbon_pages_ = nullptr;
-4
View File
@@ -199,10 +199,6 @@ void SessionState::notifyModelGeometryReady(uint32_t session_model_id) {
emit modelGeometryReady(session_model_id);
}
void SessionState::notifyModelLoadStateChanged(const QString& model_id) {
emit modelLoadStateChanged(model_id);
}
void SessionState::notifyProjectOpened(const QString& path) {
emit projectOpened(path);
}
-5
View File
@@ -89,7 +89,6 @@ public:
void notifyFederationChanged();
void notifyVisibilityChanged();
void notifyModelGeometryReady(uint32_t session_model_id);
void notifyModelLoadStateChanged(const QString& model_id);
void notifyProjectOpened(const QString& path);
void notifyProjectSaved(const QString& path);
void notifyProjectReset();
@@ -108,10 +107,6 @@ signals:
// for both sidecar-cache and stream loads; subscribers that just need to
// re-derive view state (e.g. ViewportView::refresh) listen to this.
void modelGeometryReady(uint32_t session_model_id);
// Fires when a model was unloaded from, or loaded back onto, the GPU
// (commands::unloadModel / loadModel). The viewport is the authority
// for the state itself — ViewportWindow::isModelUnloaded.
void modelLoadStateChanged(const QString& model_id);
// Fires when a model's live IFC data source (the .ifc/.rdb, opened in the
// background after a sidecar-cache hit) becomes available for queries —
// e.g. so the spatial hierarchy can be built once the file is loaded.
-3
View File
@@ -22,7 +22,6 @@
#include "ViewerSettings.h"
#include "components/Style.h"
#include "modules/models/Commands.h"
#include "../ifcparse/parse.h"
#include <QApplication>
#include <QCommandLineParser>
@@ -56,7 +55,6 @@ int main(int argc, char* argv[]) {
QApplication app(argc, argv);
app.setApplicationName("Bonsai Viewer");
app.setOrganizationName("IfcOpenShell");
app.setApplicationVersion(QString::fromUtf8(IFCOPENSHELL_VERSION));
// Clear any .rdbview extractions left in temp by a previous session.
bonsaiviewer::modules::models::commands::cleanupRdbviewCache();
@@ -72,7 +70,6 @@ int main(int argc, char* argv[]) {
QCommandLineParser parser;
parser.setApplicationDescription("Bonsai Viewer — IfcOpenShell IFC viewer");
parser.addHelpOption();
parser.addVersionOption();
parser.process(app);
installUiFont();
@@ -262,28 +262,6 @@ void removeModel(SessionState& session, ViewportWindow& viewport, QWidget& host,
session.setStatusMessage("Models", "Model removed");
}
void unloadModel(SessionState& session, ViewportWindow& viewport, const QString& model_id) {
const uint32_t session_model_id = session.sessionModelIdForModelId(model_id);
if (session_model_id == 0) return;
if (session.loader()->isLoadingModel(session_model_id)) return;
const double freed_mb = double(viewport.modelVramBytes(session_model_id)) / (1024.0 * 1024.0);
viewport.unloadModel(session_model_id);
session.notifyModelLoadStateChanged(model_id);
session.setStatusMessage("Models", QString("Model unloaded (freed %1 MB of GPU memory)")
.arg(freed_mb, 0, 'f', 0));
}
void loadModel(SessionState& session, ViewportWindow& viewport, const QString& model_id) {
const uint32_t session_model_id = session.sessionModelIdForModelId(model_id);
if (session_model_id == 0) return;
if (!viewport.loadModel(session_model_id)) {
session.setStatusMessage("Models", "Not enough GPU memory to load this model");
return;
}
session.notifyModelLoadStateChanged(model_id);
session.setStatusMessage("Models", "Model loaded");
}
void viewModels(SessionState& session, ViewportWindow& viewport, const QStringList& model_ids) {
// Federation ids are the panel's currency; the viewport speaks session
// model ids. sessionModelIdForModelId returns 0 for a model the viewport
@@ -60,12 +60,6 @@ void moveGroup(SessionState& session, const QString& id, const QString& parent_g
void moveModels(SessionState& session, const QStringList& ids, const QString& parent_group_id);
void removeGroup(SessionState& session, QWidget& host, const QString& group_id);
void removeModel(SessionState& session, ViewportWindow& viewport, QWidget& host, const QString& model_id);
// GPU residency, distinct from visibility (hide) and from membership
// (remove): unloadModel frees everything the model holds on the device
// while it stays in the federation; loadModel brings it back. Both emit
// modelLoadStateChanged.
void unloadModel(SessionState& session, ViewportWindow& viewport, const QString& model_id);
void loadModel(SessionState& session, ViewportWindow& viewport, const QString& model_id);
// "View Selected Model" — frame the camera on just these models' geometry, the
// way View All frames the whole federation. Models that carry no loaded
// geometry (never loaded, or still streaming their metadata) contribute
@@ -25,25 +25,16 @@
#include "../../../ifcviewer/Federation.h"
#include <QBrush>
#include <QFont>
#include <QColor>
namespace bonsaiviewer::modules::models {
namespace {
QStandardItem* siblingItem(QStandardItem* name_item, Column column) {
QStandardItem* siblingVisibilityItem(QStandardItem* name_item) {
QStandardItem* parent = name_item->parent();
if (!parent) parent = name_item->model()->invisibleRootItem();
return parent->child(name_item->row(), int(column));
}
QStandardItem* siblingVisibilityItem(QStandardItem* name_item) {
return siblingItem(name_item, VisibilityColumn);
}
QString formatMegabytes(quint64 bytes) {
return QString("%1 MB").arg(double(bytes) / (1024.0 * 1024.0), 0, 'f', 0);
return parent->child(name_item->row(), 1);
}
template <typename F>
@@ -60,7 +51,7 @@ FederationItemModel::FederationItemModel(Federation* federation, QObject* parent
: QStandardItemModel(parent)
, federation_(federation)
{
setColumnCount(ColumnCount);
setColumnCount(2);
rebuildAll();
connect(federation_, &Federation::groupAdded, this, &FederationItemModel::onGroupAdded);
@@ -76,7 +67,7 @@ FederationItemModel::FederationItemModel(Federation* federation, QObject* parent
void FederationItemModel::rebuildAll() {
clear();
setColumnCount(ColumnCount);
setColumnCount(2);
id_to_name_item_.clear();
for (const auto& root_group : federation_->rootGroups()) {
@@ -130,14 +121,6 @@ QStandardItem* FederationItemModel::makeVisibilityItem(ItemKind kind, bool visib
return item;
}
QStandardItem* FederationItemModel::makeMemoryItem() const {
auto* item = new QStandardItem(QString());
item->setEditable(false);
item->setTextAlignment(Qt::AlignRight | Qt::AlignVCenter);
item->setForeground(QBrush(QColor(bonsaiviewer::ViewerSettings::instance().color("disabled_text"))));
return item;
}
void FederationItemModel::styleRowVisibility(QStandardItem* name_item, bool visible) const {
QStandardItem* vis_item = siblingVisibilityItem(name_item);
if (visible) {
@@ -150,22 +133,6 @@ void FederationItemModel::styleRowVisibility(QStandardItem* name_item, bool visi
}
}
void FederationItemModel::setModelResidency(const QString& model_id, bool unloaded, quint64 vram_bytes) {
QStandardItem* name_item = findItem(model_id);
if (!name_item) return;
QStandardItem* memory_item = siblingItem(name_item, MemoryColumn);
if (!memory_item) return;
const QString text = unloaded ? QStringLiteral("unloaded")
: vram_bytes > 0 ? formatMegabytes(vram_bytes)
: QString();
if (memory_item->text() != text) memory_item->setText(text);
QFont font = name_item->font();
if (font.italic() != unloaded) {
font.setItalic(unloaded);
name_item->setFont(font);
}
}
QStandardItem* FederationItemModel::findItem(const QString& id) const {
return id_to_name_item_.value(id, nullptr);
}
@@ -181,7 +148,7 @@ void FederationItemModel::appendModelTo(QStandardItem* parent_item, const QStrin
if (!model) return;
auto* name_item = makeModelNameItem(model_id, model->display_name);
auto* vis_item = makeVisibilityItem(ItemKind::Model, federation_->isModelEffectivelyVisible(model_id));
parent_item->appendRow({name_item, makeMemoryItem(), vis_item});
parent_item->appendRow({name_item, vis_item});
id_to_name_item_.insert(model_id, name_item);
styleRowVisibility(name_item, federation_->isModelEffectivelyVisible(model_id));
}
@@ -191,7 +158,7 @@ void FederationItemModel::appendGroupSubtreeTo(QStandardItem* parent_item, const
if (!group) return;
auto* name_item = makeGroupNameItem(group_id, group->display_name);
auto* vis_item = makeVisibilityItem(ItemKind::Group, group->visible);
parent_item->appendRow({name_item, makeMemoryItem(), vis_item});
parent_item->appendRow({name_item, vis_item});
id_to_name_item_.insert(group_id, name_item);
styleRowVisibility(name_item, group->visible);
@@ -31,7 +31,7 @@ class Federation;
namespace bonsaiviewer::modules::models {
// QStandardItemModel that mirrors the Federation tree (groups + models in
// three columns: name, GPU memory, visibility icon). Subscribes directly to Federation's
// two columns: name + visibility icon). Subscribes directly to Federation's
// granular signals so each mutation only touches the affected rows — view
// state (expansion, selection, scroll) is preserved automatically.
//
@@ -57,12 +57,6 @@ public:
// previously- and newly-active model rows.
void setActiveModelId(const QString& model_id);
// GPU residency is viewport state, not Federation state, so it is pushed
// in by the owning View: the memory column shows `vram_bytes` for a
// loaded model and "unloaded" for one the user unloaded (which is also
// drawn in italics). Models the viewport knows nothing about show blank.
void setModelResidency(const QString& model_id, bool unloaded, quint64 vram_bytes);
private slots:
void onGroupAdded(const QString& group_id);
void onGroupRemoved(const QString& group_id);
@@ -78,7 +72,6 @@ private:
QStandardItem* makeGroupNameItem(const QString& group_id, const QString& display_name) const;
QStandardItem* makeModelNameItem(const QString& model_id, const QString& display_name) const;
QStandardItem* makeVisibilityItem(ItemKind kind, bool visible) const;
QStandardItem* makeMemoryItem() const;
void styleRowVisibility(QStandardItem* name_item, bool visible) const;
QStandardItem* findItem(const QString& id) const;
+6 -28
View File
@@ -28,7 +28,6 @@
#include "../../components/Section.h"
#include "../../components/SvgIcon.h"
#include "../../../ifcviewer/Federation.h"
#include "../../../ifcviewer/ViewportWindow.h"
#include <QDataStream>
#include <QDrag>
@@ -80,7 +79,6 @@ QStringList selectedModelIdsAt(QTreeView* tree, const QModelIndex& clicked_index
}
constexpr int kVisibilityColumnWidth = 28;
constexpr int kMemoryColumnWidth = 72; // "1234 MB" / "unloaded"
// QTreeView subclass that handles drag-and-drop. Drop logic dispatches
// through commands (not directly into the model) so notifications + status
@@ -252,7 +250,7 @@ ModelsPanel::ModelsPanel(bonsaiviewer::SessionState* session_state,
connect(tree_, &QTreeView::clicked, this, [this](const QModelIndex& index) {
if (!index.isValid()) return;
if (index.column() == VisibilityColumn) {
if (index.column() == 1) {
commands::toggleVisibility(*session_state_, kindOf(index), idOf(index));
return;
}
@@ -382,24 +380,6 @@ ModelsPanel::ModelsPanel(bonsaiviewer::SessionState* session_state,
commands::saveModelAsToCloud(*session_state_, *this, id);
});
// GPU residency. Unload keeps the model in the federation (and
// its visibility) but frees everything it holds on the GPU — the
// lever when the scene does not fit in VRAM. Load brings it back.
menu.addSeparator();
const uint32_t session_model_id = session_state_->sessionModelIdForModelId(id);
const bool unloaded = session_model_id != 0 && viewport_->isModelUnloaded(session_model_id);
QAction* residency = menu.addAction(
components::icons::makeSvgIcon(":/icons/cube.svg"),
unloaded ? "Load Model" : "Unload Model");
residency->setEnabled(session_model_id != 0);
residency->setToolTip(unloaded
? "Allocate GPU memory for this model again and stream its geometry back in."
: "Free this model's GPU memory while keeping it in the federation.");
connect(residency, &QAction::triggered, this, [this, id, unloaded]() {
if (unloaded) commands::loadModel(*session_state_, *viewport_, id);
else commands::unloadModel(*session_state_, *viewport_, id);
});
menu.addSeparator();
QAction* remove = menu.addAction(
components::icons::makeSvgIcon(":/icons/minus-square.svg"), "Remove Model");
@@ -429,16 +409,14 @@ void ModelsPanel::setModel(FederationItemModel* model) {
}
void ModelsPanel::applyColumnLayout() {
// The name stretches to fill; memory and visibility are fixed.
// Column 0 (name) stretches to fill; column 1 (visibility icon) is fixed.
QHeaderView* header = tree_->header();
if (header->count() < ColumnCount) return;
if (header->count() < 2) return;
header->setStretchLastSection(false);
header->setMinimumSectionSize(kVisibilityColumnWidth);
header->setSectionResizeMode(NameColumn, QHeaderView::Stretch);
header->setSectionResizeMode(MemoryColumn, QHeaderView::Fixed);
header->resizeSection(MemoryColumn, kMemoryColumnWidth);
header->setSectionResizeMode(VisibilityColumn, QHeaderView::Fixed);
header->resizeSection(VisibilityColumn, kVisibilityColumnWidth);
header->setSectionResizeMode(0, QHeaderView::Stretch);
header->setSectionResizeMode(1, QHeaderView::Fixed);
header->resizeSection(1, kVisibilityColumnWidth);
}
} // namespace bonsaiviewer::modules::models
-8
View File
@@ -31,14 +31,6 @@ enum class ItemKind {
Model,
};
// Columns of the models tree: name | GPU memory | visibility eye.
enum Column : int {
NameColumn = 0,
MemoryColumn = 1,
VisibilityColumn = 2,
ColumnCount = 3,
};
struct TreeNode {
QString id;
QString name;
+1 -30
View File
@@ -26,9 +26,6 @@
#include "../../ViewerSettings.h"
#include "../../SessionState.h"
#include "../../../ifcviewer/Federation.h"
#include "../../../ifcviewer/ViewportWindow.h"
#include <QTimer>
namespace bonsaiviewer::modules::models {
@@ -57,19 +54,17 @@ QList<GroupOption> validMoveTargets(const Federation& federation,
ModelsPanelView::ModelsPanelView(ModelsPanel* widget,
bonsaiviewer::SessionState* session_state,
ViewportWindow* viewport,
QObject* parent)
: QObject(parent)
, widget_(widget)
, session_state_(session_state)
, viewport_(viewport)
, model_(new FederationItemModel(session_state->federation(), this))
{
widget_->setModel(model_);
// Coarse signals: full rebuild + re-style. The granular Federation
// signals are handled inside FederationItemModel and don't reach here.
auto rebuild = [this]() { model_->rebuildAll(); refreshResidency(); };
auto rebuild = [this]() { model_->rebuildAll(); };
connect(session_state_, &SessionState::projectReset, this, rebuild);
connect(session_state_, &SessionState::projectOpened, this, rebuild);
connect(&bonsaiviewer::ViewerSettings::instance(),
@@ -78,30 +73,6 @@ ModelsPanelView::ModelsPanelView(ModelsPanel* widget,
connect(session_state_, &SessionState::activeModelChanged, this, [this](const QString& model_id) {
model_->setActiveModelId(model_id);
});
// Residency: immediately on the events that change it, and on a slow
// tick for the memory figures, which move as chunks stream.
auto refresh = [this]() { refreshResidency(); };
connect(session_state_, &SessionState::modelLoadStateChanged, this, refresh);
connect(session_state_, &SessionState::modelGeometryReady, this, refresh);
connect(session_state_, &SessionState::modelsChanged, this, refresh);
auto* tick = new QTimer(this);
tick->setInterval(1000);
connect(tick, &QTimer::timeout, this, refresh);
tick->start();
}
void ModelsPanelView::refreshResidency() {
for (const auto& model : session_state_->federation()->models()) {
const uint32_t session_model_id = session_state_->sessionModelIdForModelId(model.id);
if (session_model_id == 0) {
model_->setModelResidency(model.id, false, 0);
continue;
}
model_->setModelResidency(model.id,
viewport_->isModelUnloaded(session_model_id),
viewport_->modelVramBytes(session_model_id));
}
}
} // namespace bonsaiviewer::modules::models
-10
View File
@@ -26,7 +26,6 @@
#include <QObject>
class Federation;
class ViewportWindow;
namespace bonsaiviewer { class SessionState; }
namespace bonsaiviewer::modules::models {
@@ -46,25 +45,16 @@ QList<GroupOption> validMoveTargets(const Federation& federation,
// coarse session signals (project open/reset, theme change) — those are the
// "rebuild from scratch" cases the model itself doesn't subscribe to.
// Granular Federation events are handled inside the model.
//
// Also the bridge for the one thing the tree shows that is not Federation
// state: each model's GPU residency (memory column, unloaded styling). The
// viewport owns that state, so this view polls it once a second — the
// numbers move continuously while geometry streams — and pushes it in.
class ModelsPanelView : public QObject {
Q_OBJECT
public:
explicit ModelsPanelView(ModelsPanel* widget,
bonsaiviewer::SessionState* session_state,
ViewportWindow* viewport,
QObject* parent = nullptr);
private:
void refreshResidency();
ModelsPanel* widget_ = nullptr;
bonsaiviewer::SessionState* session_state_ = nullptr;
ViewportWindow* viewport_ = nullptr;
FederationItemModel* model_ = nullptr;
};
+1 -6
View File
@@ -101,12 +101,7 @@ int main() {
// IfcFacetedBRep. If it would not be a polyhedron, serialise() can only be successful when linked
// to the IFC4 model and with `advanced` set to `true` which introduces IfcAdvancedFace. It would
// return `0` otherwise.
auto building_shape_result = ifcopenshell::geom::serialise(file, building_shell, false);
if (!building_shape_result) {
std::cerr << "Failed to serialize building shell." << std::endl;
return 1;
}
auto building_shape = building_shape_result.as<IfcSchema::IfcProductDefinitionShape>();
auto building_shape = ifcopenshell::geom::serialise(file, building_shell, false).as<IfcSchema::IfcProductDefinitionShape>();
file.add_entity(building_shape);
auto building_representations = building_shape.Representations();
-2
View File
@@ -28,9 +28,7 @@
// alignment explicitly
// Disable warnings coming from IfcOpenShell
#if defined(_MSC_VER)
#pragma warning(disable : 4018 4267 4250 4984 4985)
#endif
#include "../ifcparse/schemas/Ifc4x3_add2.h"
#include "../ifcparse/hierarchy_helper.h"
-2
View File
@@ -99,8 +99,6 @@ std::string format_string(const ifcopenshell::attribute_value& argument) {
stream << v;
return stream.str();
break; }
default:
break;
}
return "?";
}
-2
View File
@@ -28,9 +28,7 @@
// to simplify alignment construction
// Disable warnings coming from IfcOpenShell
#if defined(_MSC_VER)
#pragma warning(disable : 4018 4267 4250 4984 4985)
#endif
#include "../ifcparse/schemas/Ifc4x3_add2.h"
#include "../ifcparse/alignment_helper.h"
@@ -187,11 +187,12 @@ void ifcopenshell::geom::open_cascade_shape::triangulate(ifcopenshell::geom::set
}
}
for (int i = 1; i <= tri->NbTriangles(); ++i) {
const NCollection_Array1<Poly_Triangle>& triangles = tri->Triangles();
for (int i = 1; i <= triangles.Length(); ++i) {
int n1, n2, n3;
if (face.Orientation() == TopAbs_REVERSED)
tri->Triangle(i).Get(n3, n2, n1);
else tri->Triangle(i).Get(n1, n2, n3);
triangles(i).Get(n3, n2, n1);
else triangles(i).Get(n1, n2, n3);
if (dict[n1] == dict[n2] || dict[n2] == dict[n3] || dict[n3] == dict[n1]) {
logger.warning("GEO", 185, "Mesher generated a degenerate triangle, ignoring");
@@ -655,13 +656,14 @@ namespace {
coords.push_back(tri->Node(i).Transformed(loc).XYZ());
}
for (int i = 1; i <= tri->NbTriangles(); ++i) {
const NCollection_Array1<Poly_Triangle>& triangles = tri->Triangles();
for (int i = 1; i <= triangles.Length(); ++i) {
int n1, n2, n3;
if (face.Orientation() == TopAbs_REVERSED) {
tri->Triangle(i).Get(n3, n2, n1);
triangles(i).Get(n3, n2, n1);
} else {
tri->Triangle(i).Get(n1, n2, n3);
triangles(i).Get(n1, n2, n3);
}
const gp_XYZ& pt1 = coords[n1 - 1];
-3
View File
@@ -16,9 +16,6 @@ set_target_properties(geometry_serializer PROPERTIES
RUNTIME_OUTPUT_DIRECTORY "${geometry_serialization_plugin_runtime_dir}"
LIBRARY_OUTPUT_DIRECTORY "${geometry_serialization_plugin_runtime_dir}"
)
if (NOT CREATE_BUNDLE)
set_target_properties(geometry_serializer PROPERTIES VERSION "${PROJECT_VERSION}" SOVERSION "${PROJECT_VERSION_MAJOR}.${PROJECT_VERSION_MINOR}")
endif()
target_link_libraries(geometry_serializer plugin IfcGeom IfcParse ${OpenCASCADE_LIBRARIES})
if(NOT WASM_BUILD)
target_link_libraries(geometry_serializer geometry_kernel_opencascade)
@@ -812,9 +812,10 @@ express::base POSTFIX_SCHEMA(tesselate)(ifcopenshell::file& f, const TopoDS_Shap
cpnt.setCoordinates(xyz);
vertices.push_back(cpnt);
}
for (int i = 1; i <= tri->NbTriangles(); ++i) {
const NCollection_Array1<Poly_Triangle>& triangles = tri->Triangles();
for (int i = 1; i <= triangles.Length(); ++i) {
int n1, n2, n3;
tri->Triangle(i).Get(n1, n2, n3);
triangles(i).Get(n1, n2, n3);
std::vector<IfcSchema::IfcCartesianPoint> points {
vertices[n1 - 1], vertices[n2 - 1], vertices[n3 - 1]
};
+2 -2
View File
@@ -57,8 +57,8 @@ ifeq ($(PLATFORM), win64)
PLATFORMTAG:=win_amd64
endif
BINARY_VERSION:=0.9.0alpha0
BUILD_COMMIT:=ad113e1
BINARY_VERSION:=0.8.6
BUILD_COMMIT:=e333c1c
IOS_URL:=https://s3.amazonaws.com/ifcopenshell-builds/ifcopenshell-python-$(PYNUMBER)-v$(BINARY_VERSION)-$(BUILD_COMMIT)-$(PLATFORM).zip
IFCCONVERT_URL:=https://s3.amazonaws.com/ifcopenshell-builds/IfcConvert-v$(BINARY_VERSION)-$(BUILD_COMMIT)-$(PLATFORM).zip
@@ -95,8 +95,6 @@ from .sql import sqlite, sqlite_entity
rocksdb_lazy_instance = file_module.rocksdb_lazy_instance
decode_spf_string = ifcopenshell_wrapper.decode_spf_string
encode_spf_string = ifcopenshell_wrapper.encode_spf_string
get_log = ifcopenshell_wrapper.get_log
logger = ifcopenshell_wrapper.logger if hasattr(ifcopenshell_wrapper, "logger") else None
if hasattr(ifcopenshell_wrapper, "logger_or_root"):
@@ -116,8 +114,6 @@ def optional_logger_args(logger: ifcopenshell_wrapper.logger | None) -> tuple[lo
# (it's a requirement for a typed library)
__all__ = [
"clear_plugin_search_paths",
"decode_spf_string",
"encode_spf_string",
"entity_instance",
"file",
"get_plugin_search_paths",
@@ -1649,8 +1649,6 @@ def construct_iterator_with_include_exclude_id(
geometry_library, settings, file, elems, include, num_threads, logger=None
): ...
def convert_loop_to_function_item(loop): ...
def decode_spf_string(value: str) -> str: ...
def encode_spf_string(value: str) -> str: ...
class attribute_value_derived: ...
@@ -1,14 +1,6 @@
import ifcopenshell
def test_spf_strings_can_be_encoded_and_decoded():
decoded = "Café's \\"
encoded = r"'Caf\X2\00E9\X0\''s \\'"
assert ifcopenshell.encode_spf_string(decoded) == encoded
assert ifcopenshell.decode_spf_string(encoded) == decoded
def test_skip_over_non_entity_instance():
data = """
ISO-10303-21;
+12 -12
View File
@@ -536,7 +536,7 @@ std::pair<Ifc4x3_add2::IfcCurveSegment, Ifc4x3_add2::IfcCurveSegment> mapAlignme
auto curve_segment = file.create<Ifc4x3_add2::IfcCurveSegment>();
curve_segment.setTransition(Ifc4x3_add2::IfcTransitionCode::IfcTransitionCode_CONTSAMEGRADIENT);
curve_segment.setPlacement(file.addPlacement2d(start_point.Coordinates()[0], start_point.Coordinates()[1], cos(start_direction), sin(start_direction)));
curve_segment.setSegmentStart(create_length(file, 0.0));
curve_segment.setSegmentLength(create_length(file, 0.0));
curve_segment.setSegmentLength(create_length(file, length));
curve_segment.setParentCurve(parent_curve);
@@ -549,7 +549,7 @@ std::pair<Ifc4x3_add2::IfcCurveSegment, Ifc4x3_add2::IfcCurveSegment> mapAlignme
auto curve_segment = file.create<Ifc4x3_add2::IfcCurveSegment>();
curve_segment.setTransition(Ifc4x3_add2::IfcTransitionCode::IfcTransitionCode_CONTSAMEGRADIENT);
curve_segment.setPlacement(file.addPlacement2d(start_point.Coordinates()[0], start_point.Coordinates()[1], cos(start_direction), sin(start_direction)));
curve_segment.setSegmentStart(create_length(file, 0.0));
curve_segment.setSegmentLength(create_length(file, 0.0));
curve_segment.setSegmentLength(create_length(file, length * start_radius / std::fabs(start_radius)));
curve_segment.setParentCurve(parent_curve);
@@ -570,7 +570,7 @@ std::pair<Ifc4x3_add2::IfcCurveSegment, Ifc4x3_add2::IfcCurveSegment> mapAlignme
auto curve_segment = file.create<Ifc4x3_add2::IfcCurveSegment>();
curve_segment.setTransition(Ifc4x3_add2::IfcTransitionCode::IfcTransitionCode_CONTSAMEGRADIENT);
curve_segment.setPlacement(file.addPlacement2d(start_point.Coordinates()[0], start_point.Coordinates()[1], cos(start_direction), sin(start_direction)));
curve_segment.setSegmentStart(create_length(file, offset));
curve_segment.setSegmentLength(create_length(file, offset));
curve_segment.setSegmentLength(create_length(file, length));
curve_segment.setParentCurve(parent_curve);
@@ -606,7 +606,7 @@ std::pair<Ifc4x3_add2::IfcCurveSegment, Ifc4x3_add2::IfcCurveSegment> mapAlignme
Ifc4x3_add2::IfcCurveSegment curve_segment = file.create<Ifc4x3_add2::IfcCurveSegment>();
curve_segment.setTransition(Ifc4x3_add2::IfcTransitionCode::IfcTransitionCode_CONTSAMEGRADIENT);
curve_segment.setPlacement(file.addPlacement2d(start_point.Coordinates()[0], start_point.Coordinates()[1], cos(start_direction), sin(start_direction)));
curve_segment.setSegmentStart(create_length(file, 0.0));
curve_segment.setSegmentLength(create_length(file, 0.0));
curve_segment.setSegmentLength(create_length(file, length));
curve_segment.setParentCurve(parent_curve);
@@ -628,7 +628,7 @@ std::pair<Ifc4x3_add2::IfcCurveSegment, Ifc4x3_add2::IfcCurveSegment> mapAlignme
Ifc4x3_add2::IfcCurveSegment curve_segment = file.create<Ifc4x3_add2::IfcCurveSegment>();
curve_segment.setTransition(Ifc4x3_add2::IfcTransitionCode::IfcTransitionCode_CONTSAMEGRADIENT);
curve_segment.setPlacement(file.addPlacement2d(start_point.Coordinates()[0], start_point.Coordinates()[1], cos(start_direction), sin(start_direction)));
curve_segment.setSegmentStart(create_length(file, 0.0));
curve_segment.setSegmentLength(create_length(file, 0.0));
curve_segment.setSegmentLength(create_length(file, length));
curve_segment.setParentCurve(parent_curve);
@@ -659,7 +659,7 @@ std::pair<Ifc4x3_add2::IfcCurveSegment, Ifc4x3_add2::IfcCurveSegment> mapAlignme
Ifc4x3_add2::IfcCurveSegment curve_segment = file.create<Ifc4x3_add2::IfcCurveSegment>();
curve_segment.setTransition(Ifc4x3_add2::IfcTransitionCode::IfcTransitionCode_CONTSAMEGRADIENT);
curve_segment.setPlacement(file.addPlacement2d(start_point.Coordinates()[0], start_point.Coordinates()[1], cos(start_direction), sin(start_direction)));
curve_segment.setSegmentStart(create_length(file, offset));
curve_segment.setSegmentLength(create_length(file, offset));
curve_segment.setSegmentLength(create_length(file, length));
curve_segment.setParentCurve(parent_curve);
@@ -692,7 +692,7 @@ std::pair<Ifc4x3_add2::IfcCurveSegment, Ifc4x3_add2::IfcCurveSegment> mapAlignme
Ifc4x3_add2::IfcCurveSegment curve_segment1 = file.create<Ifc4x3_add2::IfcCurveSegment>();
curve_segment1.setTransition(Ifc4x3_add2::IfcTransitionCode::IfcTransitionCode_CONTSAMEGRADIENT);
curve_segment1.setPlacement(file.addPlacement2d(start_point.Coordinates()[0], start_point.Coordinates()[1], cos(start_direction), sin(start_direction)));
curve_segment1.setSegmentStart(create_length(file, 0.0));
curve_segment1.setSegmentLength(create_length(file, 0.0));
curve_segment1.setSegmentLength(create_length(file, length / 2));
curve_segment1.setParentCurve(parent_curve1);
@@ -724,7 +724,7 @@ std::pair<Ifc4x3_add2::IfcCurveSegment, Ifc4x3_add2::IfcCurveSegment> mapAlignme
Ifc4x3_add2::IfcCurveSegment curve_segment2 = file.create<Ifc4x3_add2::IfcCurveSegment>();
curve_segment2.setTransition(Ifc4x3_add2::IfcTransitionCode::IfcTransitionCode_CONTSAMEGRADIENT);
curve_segment2.setPlacement(file.addPlacement2d(start_point.Coordinates()[0], start_point.Coordinates()[1], cos(start_direction), sin(start_direction)));
curve_segment2.setSegmentStart(create_length(file, length / 2));
curve_segment2.setSegmentLength(create_length(file, length / 2));
curve_segment2.setSegmentLength(create_length(file, length / 2));
curve_segment2.setParentCurve(parent_curve2);
@@ -756,7 +756,7 @@ std::pair<Ifc4x3_add2::IfcCurveSegment, Ifc4x3_add2::IfcCurveSegment> mapAlignme
Ifc4x3_add2::IfcCurveSegment curve_segment = file.create<Ifc4x3_add2::IfcCurveSegment>();
curve_segment.setTransition(Ifc4x3_add2::IfcTransitionCode::IfcTransitionCode_CONTSAMEGRADIENT);
curve_segment.setPlacement(file.addPlacement2d(start_point.Coordinates()[0], start_point.Coordinates()[1], cos(start_direction), sin(start_direction)));
curve_segment.setSegmentStart(create_length(file, 0.0));
curve_segment.setSegmentLength(create_length(file, 0.0));
curve_segment.setSegmentLength(create_length(file, length));
curve_segment.setParentCurve(parent_curve);
@@ -795,7 +795,7 @@ std::pair<Ifc4x3_add2::IfcCurveSegment, Ifc4x3_add2::IfcCurveSegment> mapAlignme
auto curve_segment = file.create<Ifc4x3_add2::IfcCurveSegment>();
curve_segment.setTransition(Ifc4x3_add2::IfcTransitionCode::IfcTransitionCode_CONTSAMEGRADIENT);
curve_segment.setPlacement(file.addPlacement2d(start_distance_along, start_height, dx, dy));
curve_segment.setSegmentStart(create_length(file, 0.0));
curve_segment.setSegmentLength(create_length(file, 0.0));
curve_segment.setSegmentLength(create_length(file, segment_curve_length));
curve_segment.setParentCurve(parent_curve);
@@ -822,7 +822,7 @@ std::pair<Ifc4x3_add2::IfcCurveSegment, Ifc4x3_add2::IfcCurveSegment> mapAlignme
auto curve_segment = file.create<Ifc4x3_add2::IfcCurveSegment>();
curve_segment.setTransition(Ifc4x3_add2::IfcTransitionCode::IfcTransitionCode_CONTSAMEGRADIENT);
curve_segment.setPlacement(file.addPlacement2d(start_distance_along, start_height, dx, dy));
curve_segment.setSegmentStart(create_length(file, 0.0));
curve_segment.setSegmentLength(create_length(file, 0.0));
curve_segment.setSegmentLength(create_length(file, segment_curve_length));
curve_segment.setParentCurve(parent_curve);
@@ -848,7 +848,7 @@ std::pair<Ifc4x3_add2::IfcCurveSegment, Ifc4x3_add2::IfcCurveSegment> mapAlignme
Ifc4x3_add2::IfcCurveSegment curve_segment = file.create<Ifc4x3_add2::IfcCurveSegment>();
curve_segment.setTransition(Ifc4x3_add2::IfcTransitionCode::IfcTransitionCode_CONTSAMEGRADIENT);
curve_segment.setPlacement(file.addPlacement2d(start_distance_along, start_height, 1.0, 0.));
curve_segment.setSegmentStart(create_length(file, 0.0));
curve_segment.setSegmentLength(create_length(file, 0.0));
curve_segment.setSegmentLength(create_length(file, segment_curve_length));
curve_segment.setParentCurve(parent_curve);
+5 -1
View File
@@ -67,6 +67,10 @@ class IFC_PARSE_API character_decoder {
std::string get(size_t& offset);
};
} // namespace ifcopenshell
namespace ifcopenshell {
class IFC_PARSE_API character_encoder {
private:
std::u32string str_;
@@ -76,6 +80,6 @@ class IFC_PARSE_API character_encoder {
operator std::string();
};
} // namespace ifcopenshell
} // namespace IfcWrite
#endif
+17 -52
View File
@@ -550,26 +550,6 @@ std::string token::to_string() {
return result;
}
std::string ifcopenshell::encode_spf_string(const std::string& value) {
return character_encoder(value);
}
std::string ifcopenshell::decode_spf_string(const std::string& value) {
std::string wrapped;
auto value_p = &value;
if (!value.empty() && value.front() != '\'') {
wrapped = "'" + value + "'";
value_p = &wrapped;
}
file_reader<full_buffer_impl> reader(*value_p, caller_fed_tag{});
spf_lexer<file_reader<full_buffer_impl>> lexer(&reader);
token decoded = lexer.next();
if (!decoded.is_string()) {
throw exception("Expected an SPF string");
}
return decoded.as_string();
}
namespace {
template<typename Variant, typename T>
@@ -1651,42 +1631,27 @@ express::base::set_attribute_value(size_t i, const T& t) {
apply_individual_instance_visitor(current_attribute, (int)i).apply(visitor);
}
// A null/empty single instance attribute (e.g. an omitted optional like
// OwnerHistory, ObjectPlacement or Representation) must not be persisted
// as a "set" attribute: doing so leaves isNull() false for it afterwards,
// so generated getters proceed to as<T>() and dereference a null instance.
bool should_set = true;
if constexpr (std::is_same_v<T, express::base>) {
should_set = static_cast<bool>(t);
data()->set_attribute_value(i, t);
auto new_attribute = get_attribute_value(i);
// Register inverse indices in file
if constexpr (std::is_same_v<T, express::base> || std::is_same_v<T, std::vector<express::base>> || std::is_same_v<T, std::vector<std::vector<express::base>>>) {
register_inverse_visitor visitor(*file(), *this);
apply_individual_instance_visitor(new_attribute, (int)i).apply(visitor);
}
if (should_set) {
data()->set_attribute_value(i, t);
auto new_attribute = get_attribute_value(i);
// Register inverse indices in file
if constexpr (std::is_same_v<T, express::base> || std::is_same_v<T, std::vector<express::base>> || std::is_same_v<T, std::vector<std::vector<express::base>>>) {
register_inverse_visitor visitor(*file(), *this);
apply_individual_instance_visitor(new_attribute, (int)i).apply(visitor);
}
// Register new attribute guid in guid map
if (i == 0 && (file()->ifcroot_type() != nullptr) && this->declaration().is(*file()->ifcroot_type())) {
try {
auto guid = (std::string) new_attribute;
auto it = file()->internal_guid_map().find(guid);
if (it != file()->internal_guid_map().end()) {
file()->logger().warning("Duplicate guid " + guid);
}
file()->internal_guid_map().insert({guid, *this});
} catch (ifcopenshell::exception& e) {
file()->logger().error(e);
// Register new attribute guid in guid map
if (i == 0 && (file()->ifcroot_type() != nullptr) && this->declaration().is(*file()->ifcroot_type())) {
try {
auto guid = (std::string) new_attribute;
auto it = file()->internal_guid_map().find(guid);
if (it != file()->internal_guid_map().end()) {
file()->logger().warning("Duplicate guid " + guid);
}
file()->internal_guid_map().insert({guid, *this});
} catch (ifcopenshell::exception& e) {
file()->logger().error(e);
}
} else if (!current_attribute.isNull()) {
// The attribute previously held a value, so record the clearing
// explicitly as blank instead of silently leaving the old value in place.
data()->set_attribute_value(i, blank{});
}
}
-4
View File
@@ -47,10 +47,6 @@ extern IFC_PARSE_API const char *IFCOPENSHELL_VERSION;
namespace ifcopenshell {
IFC_PARSE_API std::string encode_spf_string(const std::string& value);
IFC_PARSE_API std::string decode_spf_string(const std::string& value);
/// A stream of tokens to be read from a file_reader.
template <typename Reader>
class IFC_PARSE_API spf_lexer {
@@ -2,19 +2,9 @@
#include <catch2/catch_test_macros.hpp>
#include <ifcparse/file.h>
#include <ifcparse/parse.h>
#include <string>
#include <vector>
TEST_CASE("SPF strings can be encoded and decoded", "[ifcparse]") {
const std::string decoded = "Caf\xC3\xA9" "'s \\";
const std::string encoded = R"('Caf\X2\00E9\X0\''s \\')";
CHECK(ifcopenshell::encode_spf_string(decoded) == encoded);
CHECK(ifcopenshell::decode_spf_string(encoded) == decoded);
CHECK(ifcopenshell::decode_spf_string(encoded.substr(1, encoded.size() - 2)) == decoded);
}
TEST_CASE("IfcPropertySetDefinitionSet references are resolved without replacing their owner", "[ifcparse]") {
const std::string fixture = std::string(IFCOPENSHELL_TEST_FIXTURES) + "/ColumnPSetsOfSets.ifc";
ifcopenshell::file file(fixture);
+1 -1
View File
@@ -115,7 +115,7 @@ target_link_options(IfcViewerWeb PRIVATE
# EMSCRIPTEN_KEEPALIVE alone keeps the symbols in the binary but doesn't
# add them to Module. ccall lets the host page (web/ifcviewer.js) pass a JS string (the ?model
# URL) to load_sidecar_from_url_c without manual heap marshalling.
"-sEXPORTED_FUNCTIONS=['_main','_malloc','_free','_raf_tick_c','_load_sidecar_from_source_c','_clear_scene_c','_ifcv_on_range_done','_ifcv_chunks_resident_c','_ifcv_chunks_total_c','_ifcv_model_count_c','_ifcv_model_resident_c','_ifcv_model_total_c','_ifcv_bytes_total_c','_ifcv_bytes_needed_c','_ifcv_bytes_loaded_c','_view_all_c','_frame_selection_c','_toggle_projection_c','_projection_is_ortho_c','_standard_view_c','_toggle_fly_c','_fly_is_active_c','_hide_selected_c','_isolate_selected_c','_show_all_c','_hide_all_c','_toggle_xray_c','_xray_is_active_c','_toggle_section_c','_clear_section_c','_section_is_active_c','_ifcv_get_camera_c','_ifcv_set_camera_c','_ifcv_set_ortho_c','_ifcv_set_nav_preset_c','_ifcv_set_background_c','_ifcv_get_selection_c','_ifcv_get_active_object_c','_ifcv_apply_selection_c','_ifcv_set_visible_c','_ifcv_get_hidden_c','_ifcv_set_color_c','_ifcv_clear_colors_c','_ifcv_request_objects_c','_ifcv_set_selection_outline_c','_ifcv_selection_outline_is_on_c','_ifcv_set_federation_unit_c','_ifcv_set_false_origin_c','_ifcv_get_false_origin_c','_ifcv_set_model_transform_c','_ifcv_clear_model_transform_c','_ifcv_set_model_name_c','_ifcv_get_model_georef_c','_ifcv_get_frame_stats_c','_ifcv_unload_model_c','_ifcv_load_model_c','_ifcv_model_unloaded_c','_ifcv_model_vram_bytes_c']"
"-sEXPORTED_FUNCTIONS=['_main','_malloc','_free','_raf_tick_c','_load_sidecar_from_source_c','_clear_scene_c','_ifcv_on_range_done','_ifcv_chunks_resident_c','_ifcv_chunks_total_c','_ifcv_model_count_c','_ifcv_model_resident_c','_ifcv_model_total_c','_ifcv_bytes_total_c','_ifcv_bytes_needed_c','_ifcv_bytes_loaded_c','_view_all_c','_frame_selection_c','_toggle_projection_c','_projection_is_ortho_c','_standard_view_c','_toggle_fly_c','_fly_is_active_c','_hide_selected_c','_isolate_selected_c','_show_all_c','_hide_all_c','_toggle_xray_c','_xray_is_active_c','_toggle_section_c','_clear_section_c','_section_is_active_c','_ifcv_get_camera_c','_ifcv_set_camera_c','_ifcv_set_ortho_c','_ifcv_set_nav_preset_c','_ifcv_set_background_c','_ifcv_get_selection_c','_ifcv_get_active_object_c','_ifcv_apply_selection_c','_ifcv_set_visible_c','_ifcv_get_hidden_c','_ifcv_set_color_c','_ifcv_clear_colors_c','_ifcv_request_objects_c','_ifcv_set_selection_outline_c','_ifcv_selection_outline_is_on_c','_ifcv_set_federation_unit_c','_ifcv_set_false_origin_c','_ifcv_get_false_origin_c','_ifcv_set_model_transform_c','_ifcv_clear_model_transform_c','_ifcv_set_model_name_c','_ifcv_get_model_georef_c']"
# ccall: the host page (web/ifcviewer.js) passes the ?model URL string to load_sidecar_from_url_c,
# and the nav-preset name to ifcv_set_nav_preset_c.
# HEAPU8: lets tooling/tests read the wasm heap size (e.g. to verify a large
-12
View File
@@ -54,21 +54,9 @@ public:
// core.render().
bool consumeFrameRequest();
// The most recent per-frame stats (fps, VRAM, working set). Latched
// here so the page can read them whenever it likes (ifcv_get_frame_stats_c)
// instead of being called back every frame across the wasm boundary.
void onFrameStats(const FrameStats& stats) override { last_stats_ = stats; }
// No measurement tools on web yet, so nothing reads the CPU triangle
// shadow — and at 12 B/vertex it is a large slice of a 4 GB-capped
// wasm heap. Flip when the tools are ported.
bool wantsCpuMeshTriangles() const override { return false; }
const FrameStats& lastFrameStats() const { return last_stats_; }
private:
std::string canvas_selector_;
bool request_frame_pending_ = true; // arm an initial frame
FrameStats last_stats_ = {};
};
#endif // WEBVIEWPORTHOST_H
+19 -99
View File
@@ -195,10 +195,9 @@ int fillIdsAscending(const std::unordered_set<std::uint32_t>& ids,
// Quote `s` as a JSON string literal. IFC names come straight from the model
// and can hold quotes, backslashes and control characters; UTF-8 continuation
// bytes are already legal JSON and pass through untouched.
void appendJsonString(std::string& out, const char* data, std::uint32_t length) {
out += '"';
for (std::uint32_t i = 0; i < length; ++i) {
const unsigned char c = (unsigned char)data[i];
std::string jsonString(const std::string& s) {
std::string out = "\"";
for (unsigned char c : s) {
switch (c) {
case '"': out += "\\\""; break;
case '\\': out += "\\\\"; break;
@@ -217,10 +216,9 @@ void appendJsonString(std::string& out, const char* data, std::uint32_t length)
}
}
}
out += '"';
return out + '"';
}
NavKind classifyPress(const ViewportCore::NavBindings& b, int em_button,
bool shift, bool ctrl, bool alt) {
using MB = ViewportCore::MouseBtn; using M = ViewportCore::NavMod;
@@ -824,49 +822,27 @@ extern "C" EMSCRIPTEN_KEEPALIVE int ifcv_get_model_georef_c(int source_id, doubl
// Promise the JS layer is holding.
extern "C" EMSCRIPTEN_KEEPALIVE void ifcv_request_objects_c(int token) {
if (!g_app || !g_app->ready) {
EM_ASM({ if (Module.__ifcvOnObjectsDone) Module.__ifcvOnObjectsDone($0); }, token);
EM_ASM({ if (Module.__ifcvOnObjects) Module.__ifcvOnObjects($0, '[]'); }, token);
return;
}
g_app->core.loadAllElementMetadataWeb([token](bool) {
// Partial failures are not fatal: a model whose element block failed to
// fetch simply contributes no rows, and the rest still resolve.
//
// Serialised one model per batch, straight from string-table slices.
// The whole-scene single-string version materialised three string
// copies per element plus a scene-sized JSON blob simultaneously —
// a 400+ MB transient at ~600k elements, and the wasm heap never
// returns pages, so that peak became the session's floor. Peak is
// now one model's JSON; the string keeps its capacity across models
// so it reallocates only up to the largest one.
std::string json;
const int model_count = g_app->core.streamingModelCount();
for (int model_index = 0; model_index < model_count; ++model_index) {
json.clear();
json += '[';
bool first = true;
g_app->core.visitModelElements(model_index,
[&](const ViewportCore::ElementSlices& e) {
if (!first) json += ',';
first = false;
json += "{\"objectId\":";
json += std::to_string(e.object_id);
json += ",\"model\":";
json += std::to_string(model_index);
json += ",\"sourceId\":";
json += std::to_string(e.source_id);
json += ",\"guid\":";
appendJsonString(json, e.guid, e.guid_len);
json += ",\"name\":";
appendJsonString(json, e.name, e.name_len);
json += ",\"type\":";
appendJsonString(json, e.type, e.type_len);
json += '}';
});
json += ']';
EM_ASM({ if (Module.__ifcvOnObjectsBatch) Module.__ifcvOnObjectsBatch($0, UTF8ToString($1)); },
token, json.c_str());
std::string json = "[";
bool first = true;
for (const ViewportCore::ElementRef& e : g_app->core.elements()) {
if (!first) json += ',';
first = false;
json += "{\"objectId\":" + std::to_string(e.object_id)
+ ",\"model\":" + std::to_string(e.model_index)
+ ",\"sourceId\":" + std::to_string(e.source_id)
+ ",\"guid\":" + jsonString(e.guid)
+ ",\"name\":" + jsonString(e.name)
+ ",\"type\":" + jsonString(e.type) + '}';
}
EM_ASM({ if (Module.__ifcvOnObjectsDone) Module.__ifcvOnObjectsDone($0); }, token);
json += ']';
EM_ASM({ if (Module.__ifcvOnObjects) Module.__ifcvOnObjects($0, UTF8ToString($1)); },
token, json.c_str());
});
}
@@ -950,62 +926,6 @@ extern "C" EMSCRIPTEN_KEEPALIVE double ifcv_bytes_loaded_c() {
return double(loaded_bytes);
}
// ---- Frame stats + GPU residency ------------------------------------------
// The latest FrameStats as doubles, in this order (see FrameStats.h):
// 0 fps, 1 frame_time_ms, 2 total_objects, 3 visible_objects,
// 4 total_triangles, 5 visible_triangles, 6 draw_calls,
// 7 vram_used_bytes, 8 vram_capacity_bytes, 9 vram_budget_bytes,
// 10 chunks_wanted, 11 chunks_wanted_missing, 12 wanted_missing_bytes.
// Device-wide VRAM is not included: there is no query for it on web.
// Returns the number of values written (0 before the first frame).
constexpr int kFrameStatsValues = 13;
extern "C" EMSCRIPTEN_KEEPALIVE int ifcv_get_frame_stats_c(double* out, int capacity) {
if (!g_app || !out || capacity < kFrameStatsValues) return 0;
const FrameStats& s = g_app->host.lastFrameStats();
out[0] = s.fps;
out[1] = s.frame_time_ms;
out[2] = s.total_objects;
out[3] = s.visible_objects;
out[4] = s.total_triangles;
out[5] = s.visible_triangles;
out[6] = s.gl_draw_calls;
out[7] = double(s.vram_used_bytes);
out[8] = double(s.vram_capacity_bytes);
out[9] = double(s.vram_budget_bytes);
out[10] = s.chunks_wanted;
out[11] = s.chunks_wanted_missing;
out[12] = double(s.wanted_missing_bytes);
return kFrameStatsValues;
}
// Per-model GPU residency, keyed by source id like the other per-model
// exports. Unload frees everything the model holds on the GPU while it stays
// in the scene; load brings it back (0 if the device cannot fit its buffers).
// Neither touches visibility.
extern "C" EMSCRIPTEN_KEEPALIVE void ifcv_unload_model_c(int source_id) {
if (!g_app) return;
const std::uint32_t session_model_id = g_app->federation.sessionModelId(source_id);
if (session_model_id == 0) return;
g_app->core.unloadModel(session_model_id);
}
extern "C" EMSCRIPTEN_KEEPALIVE int ifcv_load_model_c(int source_id) {
if (!g_app) return 0;
const std::uint32_t session_model_id = g_app->federation.sessionModelId(source_id);
if (session_model_id == 0) return 0;
return g_app->core.loadModel(session_model_id) ? 1 : 0;
}
extern "C" EMSCRIPTEN_KEEPALIVE int ifcv_model_unloaded_c(int source_id) {
if (!g_app) return 0;
const std::uint32_t session_model_id = g_app->federation.sessionModelId(source_id);
return session_model_id != 0 && g_app->core.isModelUnloaded(session_model_id) ? 1 : 0;
}
extern "C" EMSCRIPTEN_KEEPALIVE double ifcv_model_vram_bytes_c(int source_id) {
if (!g_app) return 0.0;
const std::uint32_t session_model_id = g_app->federation.sessionModelId(source_id);
return session_model_id == 0 ? 0.0 : double(g_app->core.modelVramBytes(session_model_id));
}
int main(int /*argc*/, char** /*argv*/) {
Log::info() << "ifcviewer-web: starting";
g_app = new AppState();
@@ -1,91 +0,0 @@
// Regression guard for "GPURenderPassEncoder.setBindGroup: Argument 3 can't be
// an ArrayBuffer or an ArrayBufferView larger than 2 GB".
//
// Emscripten's generated WebGPU shim implements the dynamic-offset path of
// wgpuRenderPassEncoderSetBindGroup as
//
// pass.setBindGroup(index, group, HEAPU32, ptr >>> 2, count);
//
// handing WebGPU the persistent view over the *entire* wasm linear memory.
// Browsers validate the byte length of that whole backing buffer rather than
// the (start, length) slice actually read, and reject anything past 2 GB. This
// build allows the heap to grow to 4 GB (ALLOW_MEMORY_GROWTH +
// MAXIMUM_MEMORY=4294967296, because large federations need the room), so on a
// big enough session every dynamic-offset draw throws on every frame for the
// life of the page. The axis gizmo, section gizmo and overlay lines all draw
// with dynamic offsets every frame, so the viewport dies as soon as the heap
// crosses the line. ifcviewer::setBindGroupDynamic (WgpuDynamicOffsets.h)
// copies the handful of offsets into a small Uint32Array instead.
//
// Rather than allocate 2 GB to reproduce, this asserts the invariant that
// actually matters and holds at any heap size: nothing we hand to
// setBindGroup may alias the wasm heap. Run against a build without the fix
// and it fails on the first frame — the observed buffer is the whole heap.
import { test, expect } from '@playwright/test';
// Comfortably above the 4 bytes a single dynamic offset needs, and ~5 orders
// of magnitude below INITIAL_MEMORY (256 MB), so this cannot pass by accident.
const SANE_MAX_BYTES = 4096;
test('setBindGroup is never handed the wasm heap as dynamic offsets', async ({ page }) => {
// Must be installed before the module boots so no frame is missed.
await page.addInitScript(() => {
const probe = { dynamicCalls: 0, maxBufferBytes: 0, samples: [] };
window.__bindGroupProbe = probe;
const proto = GPURenderPassEncoder.prototype;
const original = proto.setBindGroup;
proto.setBindGroup = function (index, group, data, ...rest) {
if (ArrayBuffer.isView(data)) {
probe.dynamicCalls++;
const bytes = data.buffer.byteLength;
if (bytes > probe.maxBufferBytes) probe.maxBufferBytes = bytes;
if (probe.samples.length < 5) {
probe.samples.push({ bytes, elements: data.length, ctor: data.constructor.name });
}
}
return original.call(this, index, group, data, ...rest);
};
});
const errors = [];
page.on('pageerror', (e) => errors.push(e.message));
await page.goto('/IfcViewerWeb.html');
await page.waitForFunction(
() => !!(window.Module && window.Module._app_ptr), null, { timeout: 30_000 });
await page.waitForTimeout(1200);
// The corner gizmo draws every frame on its own; an orbit drag additionally
// brings up the pivot triad, which is the other pair of axis call sites.
const box = await page.locator('#viewer-canvas').boundingBox();
await page.mouse.move(box.x + box.width / 2, box.y + box.height / 2);
await page.mouse.down();
await page.mouse.move(box.x + box.width / 2 + 90, box.y + box.height / 2 + 30, { steps: 8 });
await page.waitForTimeout(300);
await page.mouse.up();
await page.waitForTimeout(300);
const result = await page.evaluate(() => ({
...window.__bindGroupProbe,
heapBytes: window.Module.HEAPU32.buffer.byteLength,
}));
console.log('BINDGROUP ' + JSON.stringify(result));
// Without this the assertion below would pass vacuously on a build where
// nothing draws with dynamic offsets at all.
expect(
result.dynamicCalls,
'no dynamic-offset setBindGroup calls were observed — the gizmos did not draw, ' +
'so this test proved nothing',
).toBeGreaterThan(0);
expect(
result.maxBufferBytes,
`setBindGroup received a ${result.maxBufferBytes}-byte backing buffer; the wasm heap ` +
`is ${result.heapBytes} bytes. A match means the whole-heap HEAPU32 view is being ` +
`passed straight through, which throws once the heap passes 2 GB. ` +
`Samples: ${JSON.stringify(result.samples)}`,
).toBeLessThanOrEqual(SANE_MAX_BYTES);
expect(errors, `page errors during the run: ${errors.join(' | ')}`).toHaveLength(0);
});
-115
View File
@@ -1,115 +0,0 @@
import { test, expect } from '@playwright/test';
// GPU memory as the host page sees it: the per-frame stats (cache occupancy,
// working set) and the per-model unload/load lever. Mirrors what
// BonsaiViewer's status bar and Models panel show on desktop.
async function open(page) {
const errors = [];
page.on('console', (msg) => {
const t = msg.text();
if (/Uncaptured WebGPU error|is invalid|Not enough memory left/i.test(t)) errors.push(t);
});
page.on('pageerror', (e) => errors.push('pageerror: ' + e.message));
await page.goto('/scripting.html');
await page.waitForFunction(() => !!(window.viewer && window.viewer.isLive()), null,
{ timeout: 30_000 });
return errors;
}
// Add the sample as a real source (the embedded one has no source id) and
// wait until every chunk is resident.
async function addSampleAndSettle(page) {
const sid = await page.evaluate(async () => {
const v = window.viewer;
const loaded = new Promise((resolve) => {
const off = v.onModelLoaded((d) => { off(); resolve(d); });
});
const sid = await v.addUrl('/sample.ifcview', { replace: true, name: 'mem' });
await loaded;
return sid;
});
await settled(page);
return sid;
}
async function settled(page) {
await page.waitForFunction(() => {
const v = window.viewer;
if (!v.modelCount()) return false;
for (let i = 0; i < v.modelCount(); ++i) {
const p = v.modelProgress(i);
if (!(p.total > 0 && p.resident === p.total)) return false;
}
return true;
}, null, { timeout: 30_000 });
}
test('stats() reports the frame, the cache and the working set', async ({ page }) => {
const errors = await open(page);
await addSampleAndSettle(page);
await page.waitForTimeout(300);
const s = await page.evaluate(() => window.viewer.stats());
expect(s).not.toBeNull();
expect(s.fps).toBeGreaterThan(0);
expect(s.frameTimeMs).toBeGreaterThan(0);
expect(s.objects.total).toBeGreaterThan(0);
expect(s.triangles.total).toBeGreaterThan(0);
// Resident geometry occupies the cache, within its capacity, and on web
// the cache is bounded from the start (the wasm heap cap).
expect(s.vram.usedBytes).toBeGreaterThan(0);
expect(s.vram.usedBytes).toBeLessThanOrEqual(s.vram.capacityBytes);
expect(s.vram.budgetBytes).toBeGreaterThan(0);
// Everything the camera wants is resident once settled.
expect(s.workingSet.chunks).toBeGreaterThan(0);
expect(s.workingSet.chunksMissing).toBe(0);
expect(s.workingSet.missingBytes).toBe(0);
expect(errors).toEqual([]);
});
test('unloadModel frees the model\'s GPU memory and loadModel streams it back', async ({ page }) => {
const errors = await open(page);
const sid = await addSampleAndSettle(page);
const before = await page.evaluate((sid) => ({
unloaded: window.viewer.modelUnloaded(sid),
bytes: window.viewer.modelVramBytes(sid),
used: window.viewer.stats().vram.usedBytes,
}), sid);
expect(before.unloaded).toBe(false);
expect(before.bytes).toBeGreaterThan(0);
// Unload: the model's bytes go to zero immediately, and it stays listed.
const after = await page.evaluate((sid) => {
const v = window.viewer;
v.unloadModel(sid);
return {
unloaded: v.modelUnloaded(sid),
bytes: v.modelVramBytes(sid),
modelCount: v.modelCount(),
};
}, sid);
expect(after.unloaded).toBe(true);
expect(after.bytes).toBe(0);
expect(after.modelCount).toBe(1);
// The cache reflects the release on the next frame.
await page.waitForFunction((used) => {
const s = window.viewer.stats();
return s && s.vram.usedBytes < used;
}, before.used, { timeout: 10_000 });
// Load: the buffers come back and the chunks stream in again.
const reloaded = await page.evaluate((sid) => window.viewer.loadModel(sid), sid);
expect(reloaded).toBe(true);
expect(await page.evaluate((sid) => window.viewer.modelUnloaded(sid), sid)).toBe(false);
await settled(page);
const restored = await page.evaluate((sid) => window.viewer.modelVramBytes(sid), sid);
expect(restored).toBe(before.bytes);
expect(errors).toEqual([]);
});
-107
View File
@@ -1,107 +0,0 @@
import { test, expect } from '@playwright/test';
// The OPFS model cache behind addUrl(url, {cache: true}): the first load
// streams over HTTP Range and fills a local copy from those same reads; a
// reload of the page then loads the model with zero geometry traffic. One
// browser context spans both loads — OPFS is origin storage, so it survives
// page reloads within the context.
function watchRequests(page, counters) {
page.on('request', (req) => {
if (!req.url().includes('sample.ifcview')) return;
if (req.method() === 'HEAD') counters.head++;
else if (req.headers()['range']) counters.range++;
else counters.other++;
});
}
async function openScripting(page, errors) {
page.on('console', (msg) => {
const t = msg.text();
if (/Uncaptured WebGPU error|is invalid|Not enough memory left/i.test(t)) errors.push(t);
});
page.on('pageerror', (e) => errors.push('pageerror: ' + e.message));
await page.goto('/scripting.html');
await page.waitForFunction(() => !!(window.viewer && window.viewer.isLive()), null,
{ timeout: 30_000 });
}
async function addCachedSampleAndSettle(page) {
await page.evaluate(async () => {
const v = window.viewer;
const loaded = new Promise((resolve) => {
const off = v.onModelLoaded((d) => { off(); resolve(d); });
});
await v.addUrl('/sample.ifcview', { replace: true, cache: true, name: 'cached' });
await loaded;
});
await page.waitForFunction(() => {
const v = window.viewer;
if (!v.modelCount()) return false;
const p = v.modelProgress(0);
return p.total > 0 && p.resident === p.total;
}, null, { timeout: 30_000 });
// The element table is read through the same source — pull it so its
// ranges land in the cache too, then let the write chain drain.
await page.evaluate(() => window.viewer.getObjects());
await page.waitForFunction(async () => {
const info = await window.viewer.cacheInfo();
const e = info.entries.find((x) => x.url.endsWith('/sample.ifcview'));
return !!(e && e.complete);
}, null, { timeout: 30_000 });
}
test('first load fills the cache from its own reads; a reload streams nothing', async ({ page }) => {
const errors = [];
const first = { head: 0, range: 0, other: 0 };
watchRequests(page, first);
await openScripting(page, errors);
await page.evaluate(() => window.viewer.clearCache());
await addCachedSampleAndSettle(page);
expect(first.range, 'first visit must stream over HTTP Range').toBeGreaterThan(0);
const info = await page.evaluate(() => window.viewer.cacheInfo());
const entry = info.entries.find((x) => x.url.endsWith('/sample.ifcview'));
expect(entry.complete).toBe(true);
expect(entry.cachedBytes).toBe(entry.size);
// Second visit: same context, fresh page. Only the HEAD validation may
// touch the network — every byte of geometry and metadata comes from OPFS.
await page.reload();
const second = { head: 0, range: 0, other: 0 };
watchRequests(page, second);
await page.waitForFunction(() => !!(window.viewer && window.viewer.isLive()), null,
{ timeout: 30_000 });
await addCachedSampleAndSettle(page);
expect(second.range, 'a complete validated copy must stream zero ranges').toBe(0);
expect(second.other, 'and never download the file whole').toBe(0);
expect(second.head).toBeGreaterThan(0);
// The cached model is actually usable: objects enumerate with GUIDs.
const objects = await page.evaluate(() => window.viewer.getObjects());
expect(objects.length).toBeGreaterThan(0);
expect(objects.some((o) => o.guid)).toBe(true);
expect(errors).toEqual([]);
});
test('clearCache drops the entry and the next load streams again', async ({ page }) => {
const errors = [];
await openScripting(page, errors);
await addCachedSampleAndSettle(page);
const cleared = await page.evaluate(() => window.viewer.clearCache('/sample.ifcview'));
expect(cleared).toBe(1);
const info = await page.evaluate(() => window.viewer.cacheInfo());
expect(info.entries.find((x) => x.url.endsWith('/sample.ifcview'))).toBeUndefined();
await page.reload();
const counters = { head: 0, range: 0, other: 0 };
watchRequests(page, counters);
await page.waitForFunction(() => !!(window.viewer && window.viewer.isLive()), null,
{ timeout: 30_000 });
await addCachedSampleAndSettle(page);
expect(counters.range).toBeGreaterThan(0);
expect(errors).toEqual([]);
});
+1 -7
View File
@@ -6,7 +6,6 @@
// Serve dir resolution: $WEB_BUILD_DIR if set, else the repo's build-web.
import http from 'node:http';
import { readFile } from 'node:fs/promises';
import { createHash } from 'node:crypto';
import { fileURLToPath } from 'node:url';
import path from 'node:path';
@@ -47,9 +46,6 @@ http.createServer(async (req, res) => {
try { body = await readFile(inRoot); }
catch { body = await readFile(inSrc); } // fall back to the source dir
const ctype = MIME[path.extname(p)] || 'application/octet-stream';
// A strong ETag from the content, so the OPFS cache spec can exercise
// validation exactly the way a real Accept-Ranges host would offer it.
const etag = '"' + createHash('sha1').update(body).digest('hex').slice(0, 16) + '"';
// HEAD: headers only — lets the remote backend resolve total size.
if (req.method === 'HEAD') {
@@ -57,7 +53,6 @@ http.createServer(async (req, res) => {
'Content-Type': ctype,
'Content-Length': body.length,
'Accept-Ranges': 'bytes',
'ETag': etag,
});
res.end();
return;
@@ -79,13 +74,12 @@ http.createServer(async (req, res) => {
'Content-Range': `bytes ${start}-${end}/${body.length}`,
'Accept-Ranges': 'bytes',
'Content-Length': slice.length,
'ETag': etag,
});
res.end(slice);
return;
}
res.writeHead(200, { 'Content-Type': ctype, 'Accept-Ranges': 'bytes', 'ETag': etag });
res.writeHead(200, { 'Content-Type': ctype, 'Accept-Ranges': 'bytes' });
res.end(body);
} catch {
res.writeHead(404).end('not found');
+8 -65
View File
@@ -42,10 +42,6 @@
ul#model-list .name b { font-weight: 600; overflow: hidden; text-overflow: ellipsis;
white-space: nowrap; }
ul#model-list .pct { color: #8a93a6; flex: 0 0 auto; }
ul#model-list .mem { color: #8a93a6; font-size: 11px; margin-left: 8px; flex: 0 0 auto; }
ul#model-list .mem button { font-size: 11px; padding: 1px 6px; margin-left: 6px; }
ul#model-list li.unloaded b { font-style: italic; color: #6f7988; }
#gpu-memory.full { color: #e0a040; }
.bar { height: 4px; margin-top: 5px; border-radius: 2px; background: #232833; overflow: hidden; }
.bar > i { display: block; height: 100%; width: 0%; background: #3182ce; }
.empty { color: #6f7988; font-size: 12px; padding: 4px 0; }
@@ -96,7 +92,6 @@
<div class="card">
<h2>Models in scene</h2>
<ul id="model-list"><li class="empty">No models loaded.</li></ul>
<div class="hint" id="gpu-memory">GPU memory: —</div>
</div>
<div class="card">
@@ -129,61 +124,16 @@
listEl.innerHTML = '';
models.forEach(function (m, i) {
var li = document.createElement('li');
if (m.unloaded) li.className = 'unloaded';
var pct = m.total > 0 ? Math.round(100 * m.resident / m.total) : 0;
var label = m.unloaded ? 'unloaded' : m.total > 0 ? pct + '%' : '…';
var mem = m.unloaded ? '' : Math.round(m.vram / (1024 * 1024)) + ' MB';
var label = m.total > 0 ? pct + '%' : '…';
li.innerHTML =
'<div class="name"><b title="' + m.name + '">' + m.name + '</b>' +
'<span class="mem">' + mem + '<button data-i="' + i + '">' +
(m.unloaded ? 'Load' : 'Unload') + '</button></span>' +
'<span class="pct">' + label + '</span></div>' +
'<div class="bar"><i style="width:' + (m.unloaded ? 0 : pct) + '%"></i></div>';
'<div class="bar"><i style="width:' + pct + '%"></i></div>';
listEl.appendChild(li);
});
}
// Unload frees a model's GPU memory while it stays in the scene — the lever
// when the GPU memory line reports chunks that cannot be loaded.
listEl.addEventListener('click', function (ev) {
var btn = ev.target.closest('button[data-i]');
if (!btn || !activeViewer) return;
var m = models[+btn.dataset.i];
if (!m || m.sid === undefined) return;
if (m.unloaded) {
if (!activeViewer.loadModel(m.sid)) { hintEl.textContent = 'Not enough GPU memory to load ' + m.name; return; }
} else {
activeViewer.unloadModel(m.sid);
}
m.unloaded = activeViewer.modelUnloaded(m.sid);
renderList();
});
var gpuMemEl = document.getElementById('gpu-memory');
var shortfallSince = 0;
var activeViewer = null; // set once IfcViewer.create resolves
function renderGpuMemory(viewer) {
var s = viewer.stats();
if (!s) return;
var mb = function (b) { return Math.round(b / (1024 * 1024)); };
var text = 'GPU memory: ' + mb(s.vram.usedBytes) + ' / ' + mb(s.vram.capacityBytes) + ' MB';
if (s.vram.budgetBytes && s.vram.budgetBytes !== s.vram.capacityBytes) {
text += ' (budget ' + mb(s.vram.budgetBytes) + ')';
}
// A few missing chunks right after a camera move are normal; a shortfall
// that persists means the view does not fit — say so.
var now = performance.now();
if (!s.workingSet.chunksMissing) shortfallSince = 0;
else if (!shortfallSince) shortfallSince = now;
var full = shortfallSince && now - shortfallSince > 3000;
if (full) {
text += ' — full: ' + s.workingSet.chunksMissing + ' of ' + s.workingSet.chunks +
' visible chunks (' + mb(s.workingSet.missingBytes) + ' MB) not loaded. Unload a model to make room.';
}
if (gpuMemEl.textContent !== text) gpuMemEl.textContent = text;
gpuMemEl.classList.toggle('full', !!full);
}
function setSelection(guid, modelName) {
selModelEl.textContent = modelName || '—';
if (guid) { selGuidEl.textContent = guid; selGuidEl.classList.remove('none'); }
@@ -204,20 +154,16 @@
// Keep clearing until it's gone; stop once the user adds their own model.
if (!userAddedAny && viewer.modelCount() > 0) viewer.clearScene();
if (!models.length) return;
renderGpuMemory(viewer);
var changed = false;
for (var i = 0; i < models.length; i++) {
var p = viewer.modelProgress(i);
var vram = models[i].sid !== undefined ? viewer.modelVramBytes(models[i].sid) : 0;
if (p.resident !== models[i].resident || p.total !== models[i].total
|| Math.round(vram / (1024 * 1024)) !== Math.round(models[i].vram / (1024 * 1024))) {
models[i].resident = p.resident; models[i].total = p.total; models[i].vram = vram; changed = true;
if (p.resident !== models[i].resident || p.total !== models[i].total) {
models[i].resident = p.resident; models[i].total = p.total; changed = true;
}
}
if (changed) renderList();
},
}).then(function (viewer) {
activeViewer = viewer;
// Report the picked object's model + IFC GlobalId in our own DOM (empty on
// deselect). sel.modelIndex indexes our JS model list (load order).
viewer.onSelect(function (sel) {
@@ -231,10 +177,7 @@
var urlInput = document.getElementById('url-input');
var urlBtn = document.getElementById('url-btn');
function addModelEntry(name, sid) {
models.push({ name: name, sid: sid, resident: 0, total: 0, vram: 0, unloaded: false });
renderList();
}
function addModelEntry(name) { models.push({ name: name, resident: 0, total: 0 }); renderList(); }
viewer.ready.then(function () {
hintEl.textContent = 'Ready — add a .ifcview model.';
@@ -245,7 +188,7 @@
fileInput.addEventListener('change', function (ev) {
if (ev.target.files.length) userAddedAny = true;
Array.prototype.forEach.call(ev.target.files, function (file) {
viewer.addFile(file).then(function (sid) { addModelEntry(file.name, sid); });
viewer.addFile(file).then(function () { addModelEntry(file.name); });
});
fileInput.value = '';
});
@@ -255,8 +198,8 @@
if (!url) return;
userAddedAny = true;
urlBtn.disabled = true;
viewer.addUrl(url).then(function (sid) {
addModelEntry(url.split('/').pop() || url, sid);
viewer.addUrl(url).then(function () {
addModelEntry(url.split('/').pop() || url);
urlInput.value = '';
}).catch(function (e) {
hintEl.textContent = 'URL load failed: ' + e.message;
+6 -462
View File
@@ -51,349 +51,6 @@
return cr ? parseInt(cr.split('/')[1] || '0', 10) : 0;
}
// ---- OPFS model cache ------------------------------------------------------
//
// addUrl(url, {cache: true}) keeps a local copy of the sidecar in the
// Origin Private File System, filled FROM THE VIEWER'S OWN RANGED READS —
// no second download, and only the bytes the camera actually needed.
// (Browsers do not populate their HTTP cache from ranged fetches: measured
// 0 of 78 range requests served from cache even with a strong ETag.)
//
// Entries are keyed by a hash of the URL and validated by ETag (falling
// back to Last-Modified + size); a byte-span ledger records which ranges
// are really on disk, so a partial copy is never mistaken for a whole one —
// a read is served locally only when its span is fully covered. On the next
// visit a complete validated copy loads with zero geometry traffic; if the
// server is unreachable, the newest complete copy is used as-is (offline).
//
// Writes go through a dedicated worker holding a FileSystemSyncAccessHandle:
// positional writes with no copy-on-open (createWritable({keepExistingData})
// copies the whole existing file into a swap file per open — quadratic as
// the cache fills), and the handle's exclusive lock makes a second tab fall
// back to plain network instead of corrupting the entry. Browsers without
// sync access handles just never cache — behaviour is exactly as without
// the flag.
const CACHE_DIR = 'ifcviewer-cache';
const cacheWorkerSource = `
const handles = new Map();
onmessage = async (e) => {
const { id, op, name, pos, data, len } = e.data;
const reply = (msg, transfer) => postMessage(Object.assign({ id }, msg), transfer || []);
try {
if (op === 'open') {
const root = await navigator.storage.getDirectory();
const dir = await root.getDirectoryHandle('${CACHE_DIR}', { create: true });
const fh = await dir.getFileHandle(name, { create: true });
handles.set(name, await fh.createSyncAccessHandle());
reply({ ok: true, size: handles.get(name).getSize() });
} else if (op === 'write') {
handles.get(name).write(new Uint8Array(data), { at: pos });
reply({ ok: true });
} else if (op === 'read') {
const buf = new Uint8Array(len);
const n = handles.get(name).read(buf, { at: pos });
reply({ ok: n === len, data: buf.buffer }, [buf.buffer]);
} else if (op === 'close') {
const h = handles.get(name);
if (h) { h.flush(); h.close(); handles.delete(name); }
reply({ ok: true });
} else {
reply({ ok: false, error: 'unknown op ' + op });
}
} catch (err) {
reply({ ok: false, error: String((err && err.message) || err) });
}
};
`;
let cacheWorker = null; // lazily created; false once known unusable
let cacheMsgId = 0;
const cachePending = new Map();
function cacheCall(op, name, extra, transfer) {
if (cacheWorker === false) return Promise.reject(new Error('no cache worker'));
if (!cacheWorker) {
try {
cacheWorker = new Worker(URL.createObjectURL(
new Blob([cacheWorkerSource], { type: 'text/javascript' })));
cacheWorker.onmessage = (e) => {
const pending = cachePending.get(e.data.id);
if (!pending) return;
cachePending.delete(e.data.id);
if (e.data.ok) pending.resolve(e.data);
else pending.reject(new Error(e.data.error || (op + ' failed')));
};
} catch (err) {
cacheWorker = false;
return Promise.reject(err);
}
}
const id = ++cacheMsgId;
return new Promise((resolve, reject) => {
cachePending.set(id, { resolve: resolve, reject: reject });
cacheWorker.postMessage(Object.assign({ id: id, op: op, name: name }, extra || {}),
transfer || []);
});
}
async function cacheDirHandle(create) {
const root = await navigator.storage.getDirectory();
return root.getDirectoryHandle(CACHE_DIR, { create: !!create });
}
let persistAsked = false;
async function openCacheDir() {
try {
const dir = await cacheDirHandle(true);
// Without this a large cache is "best effort" and the browser may drop
// it under disk pressure — invisibly, looking like the site being slow
// again on the next visit.
if (!persistAsked && navigator.storage.persist) {
persistAsked = true;
navigator.storage.persist().catch(() => {});
}
return dir;
} catch (err) {
return null;
}
}
async function cacheEntryName(url) {
const bytes = new TextEncoder().encode(url);
const digest = await crypto.subtle.digest('SHA-256', bytes);
return Array.from(new Uint8Array(digest).slice(0, 16))
.map((b) => b.toString(16).padStart(2, '0')).join('');
}
// Sorted, merged, half-open [start, end) byte spans.
function spansAdd(spans, start, end) {
const out = [];
let s0 = start, e0 = end;
for (const [a, b] of spans) {
if (b < s0 || a > e0) out.push([a, b]);
else { s0 = Math.min(s0, a); e0 = Math.max(e0, b); }
}
out.push([s0, e0]);
out.sort((x, y) => x[0] - y[0]);
return out;
}
const spansCover = (spans, start, end) =>
spans.some(([a, b]) => a <= start && b >= end);
const spansBytes = (spans) => spans.reduce((sum, [a, b]) => sum + (b - a), 0);
async function readCacheMeta(dir, name) {
try {
const fh = await dir.getFileHandle(name + '.meta');
return JSON.parse(await (await fh.getFile()).text());
} catch (err) {
return null;
}
}
// The meta file is tiny, so main-thread createWritable is fine here; only
// the tab holding the data file's exclusive lock ever writes it.
async function writeCacheMeta(dir, name, meta) {
const fh = await dir.getFileHandle(name + '.meta', { create: true });
const w = await fh.createWritable();
await w.write(JSON.stringify(meta));
await w.close();
}
async function removeCacheEntry(dir, name) {
await dir.removeEntry(name).catch(() => {});
await dir.removeEntry(name + '.meta').catch(() => {});
}
async function headValidators(url) {
try {
const res = await fetch(url, { method: 'HEAD' });
if (!res.ok) return null;
return {
etag: res.headers.get('ETag') || null,
lastModified: res.headers.get('Last-Modified') || null,
size: parseInt(res.headers.get('Content-Length') || '0', 10) || 0,
};
} catch (err) {
return null; // offline, or CORS refused HEAD
}
}
function validatorsMatch(meta, head) {
if (meta.etag && head.etag) return meta.etag === head.etag;
if (meta.lastModified && head.lastModified) {
return meta.lastModified === head.lastModified && meta.size === head.size;
}
return false;
}
// A Blob-shaped source (`size` + `slice(a, b).arrayBuffer()`) backed by the
// OPFS entry, filling from every ranged read that goes through it. The
// wasm's read shim only ever calls those two members, so this passes for
// the File it would get from a picked file.
function fillingCacheSource(dir, name, url, meta) {
let spans = meta.spans.slice();
let dirtySince = 0; // bytes written since the ledger was persisted
let broken = false; // a write failed (quota?): serve network, stop filling
let complete = spansCover(spans, 0, meta.size);
let writeChain = Promise.resolve();
let lastForegroundRead = 0; // performance.now() of the viewer's last read
const persistLedger = () => {
dirtySince = 0;
return writeCacheMeta(dir, name, Object.assign({}, meta, { spans: spans }))
.catch(() => {});
};
const finishIfComplete = () => {
if (complete || !spansCover(spans, 0, meta.size)) return;
complete = true;
// Flush + release the lock; from here reads come off the closed file.
writeChain = writeChain
.then(() => cacheCall('close', name))
.then(persistLedger)
.catch(() => {});
};
const storeBytes = (start, stop, buf) => {
if (broken || complete || buf.byteLength !== stop - start) return;
const copy = buf.slice(0);
writeChain = writeChain
.then(() => cacheCall('write', name, { pos: start, data: copy }, [copy]))
.then(() => {
spans = spansAdd(spans, start, stop);
dirtySince += stop - start;
// Persist the ledger periodically — bytes on disk that the
// ledger does not record are merely re-fetched next visit.
if (dirtySince >= (4 << 20)) return persistLedger();
})
.then(finishIfComplete)
.catch(() => { broken = true; });
};
// Background completion: streaming only reads what the camera needs, so
// left alone the cache converges on the *viewed* bytes, not the file —
// and a user cannot be expected to orbit every model into view to
// finish it. Once the viewer has been quiet for a moment, fetch the
// uncovered spans in order, one modest range at a time, yielding
// whenever real reads resume so interactive streaming always wins.
const IDLE_MS = 1500, STEP_BYTES = 8 << 20;
let backgroundDone = false;
async function backgroundFill() {
while (!complete && !broken && !backgroundDone) {
if (performance.now() - lastForegroundRead < IDLE_MS) {
await new Promise((r) => setTimeout(r, IDLE_MS));
continue;
}
// First gap not yet covered.
let at = 0;
for (const [a, b] of spans) { if (a > at) break; at = Math.max(at, b); }
if (at >= meta.size) { finishIfComplete(); return; }
let stop = Math.min(at + STEP_BYTES, meta.size);
for (const [a] of spans) { if (a > at) { stop = Math.min(stop, a); break; } }
try {
const res = await fetch(url, {
headers: { Range: 'bytes=' + at + '-' + (stop - 1) },
});
if (res.status !== 206 && res.status !== 200) return; // server changed its mind
let buf = await res.arrayBuffer();
if (res.status === 200 && buf.byteLength > stop - at) buf = buf.slice(at, stop);
storeBytes(at, stop, buf);
await writeChain;
} catch (err) {
return; // offline etc: the foreground path is affected too, stop quietly
}
}
}
setTimeout(backgroundFill, IDLE_MS);
return {
size: meta.size,
stopBackgroundFill() { backgroundDone = true; },
slice(start, end) {
const stop = Math.min(end, meta.size);
return {
arrayBuffer: async () => {
lastForegroundRead = performance.now();
if (spansCover(spans, start, stop)) {
if (complete) {
const fh = await dir.getFileHandle(name);
return (await fh.getFile()).slice(start, stop).arrayBuffer();
}
// Serialise behind the writes so a just-written range is
// readable (sync-handle writes are visible to the same handle
// immediately; ordering through the chain keeps it simple).
return (writeChain = writeChain.then(() =>
cacheCall('read', name, { pos: start, len: stop - start })
)).then((r) => r.data);
}
const res = await fetch(url, {
headers: { Range: 'bytes=' + start + '-' + (stop - 1) },
});
if (res.status !== 206 && res.status !== 200) {
throw new Error('range fetch failed: ' + res.status);
}
let buf = await res.arrayBuffer();
if (res.status === 200 && buf.byteLength > stop - start) {
buf = buf.slice(start, stop);
}
storeBytes(start, stop, buf);
return buf;
},
};
},
};
}
// Decide what to hand the loader for a cached URL:
// {file} — a complete validated local copy (zero network)
// {source} — a Blob-shaped self-filling source
// null — cache unusable here (no OPFS / no validators / second tab):
// caller falls back to the plain URL path.
async function cachedUrlSource(url) {
if (!(crypto && crypto.subtle) || !(navigator.storage && navigator.storage.getDirectory)) {
return null;
}
const head = await headValidators(url);
const dir = await openCacheDir();
if (!dir) return null;
const name = await cacheEntryName(url);
const meta = await readCacheMeta(dir, name);
const completeLocal = async (m) => {
const fh = await dir.getFileHandle(name);
const file = await fh.getFile();
return file.size === m.size ? { file: file } : null;
};
if (!head) {
// Offline (or HEAD refused): a complete copy is better than nothing —
// this is the offline story. Anything less falls back to the URL path,
// which will fail the same way it always did.
if (meta && spansCover(meta.spans, 0, meta.size)) return completeLocal(meta);
return null;
}
if (!head.etag && !head.lastModified) return null; // nothing to validate by
if (!head.size) return null;
if (meta && validatorsMatch(meta, head)) {
if (spansCover(meta.spans, 0, meta.size)) {
const local = await completeLocal(meta);
if (local) return local;
}
// Partial copy of the still-current file: resume filling it.
} else if (meta) {
await removeCacheEntry(dir, name); // server has a different file now
}
const fresh = (!meta || !validatorsMatch(meta, head))
? { url: url, etag: head.etag, lastModified: head.lastModified,
size: head.size, spans: [] }
: meta;
try {
await cacheCall('open', name); // exclusive: a second tab lands in catch
} catch (err) {
return null;
}
await writeCacheMeta(dir, name, fresh).catch(() => {});
return { source: fillingCacheSource(dir, name, url, fresh) };
}
// Mouse navigation schemes the wasm's classifyPress understands. Named so a
// typo is an error here rather than a silent fall-back to blender in the core.
const NAV_PRESETS = ['blender', 'rhino', 'revit', 'web'];
@@ -600,22 +257,13 @@
// Completion side of ifcv_request_objects_c: the element tables have all
// landed and the scene's objects are ready as JSON. `token` matches the
// request to its pending Promise.
// token -> {rows, resolve}. The wasm side streams the objects array one
// model per batch (so the whole-scene JSON never exists in one string on
// its heap); Done resolves with the accumulated rows.
const pendingObjects = new Map();
let objectsToken = 0;
Module.__ifcvOnObjectsBatch = function (token, json) {
const pending = pendingObjects.get(token);
if (!pending) return;
const rows = JSON.parse(json);
for (let i = 0; i < rows.length; ++i) pending.rows.push(rows[i]);
};
Module.__ifcvOnObjectsDone = function (token) {
const pending = pendingObjects.get(token);
if (!pending) return;
Module.__ifcvOnObjects = function (token, json) {
const resolve = pendingObjects.get(token);
if (!resolve) return;
pendingObjects.delete(token);
pending.resolve(pending.rows);
resolve(JSON.parse(json));
};
// Some test harnesses / the fullscreen page want the raw module on window.
@@ -794,7 +442,7 @@
getObjects: function () {
const token = ++objectsToken;
return new Promise(function (resolve) {
pendingObjects.set(token, { rows: [], resolve: resolve });
pendingObjects.set(token, resolve);
Module._ifcv_request_objects_c(token);
}).then(function (objects) {
objectIndex = new Map();
@@ -882,45 +530,6 @@
};
},
// The latest frame's statistics — what BonsaiViewer's status bar shows.
// `vram` is the streamed-geometry cache: bytes held by resident chunks,
// the pool's current capacity, and the budget it may grow to (0 while
// unbounded). `workingSet` is what the camera wants resident: chunks in
// view and large enough to draw, how many of those are not resident,
// and their size — transiently non-zero after a camera move, and
// persistently non-zero when the scene does not fit in GPU memory
// (unload a model to make room). Null before the first frame.
stats: function () {
const n = 13;
const ptr = Module._malloc(n * 8);
try {
if (!Module._ifcv_get_frame_stats_c(ptr, n)) return null;
const d = Module.HEAPF64.subarray(ptr >>> 3, (ptr >>> 3) + n);
return {
fps: d[0],
frameTimeMs: d[1],
objects: { visible: d[3], total: d[2] },
triangles: { visible: d[5], total: d[4] },
drawCalls: d[6],
vram: { usedBytes: d[7], capacityBytes: d[8], budgetBytes: d[9] },
workingSet: { chunks: d[10], chunksMissing: d[11], missingBytes: d[12] },
};
} finally {
Module._free(ptr);
}
},
// GPU residency per model, by source id. Unloading frees everything
// the model holds on the GPU while it stays in the scene (its
// visibility untouched); loading brings it back, streaming the
// geometry in again on demand. loadModel resolves false when the
// device cannot fit the model's buffers. This is the lever when
// stats().workingSet.chunksMissing stays above zero.
unloadModel: function (sourceId) { Module._ifcv_unload_model_c(sourceId | 0); },
loadModel: function (sourceId) { return Module._ifcv_load_model_c(sourceId | 0) !== 0; },
modelUnloaded: function (sourceId) { return Module._ifcv_model_unloaded_c(sourceId | 0) !== 0; },
modelVramBytes: function (sourceId) { return Module._ifcv_model_vram_bytes_c(sourceId | 0); },
registerFileSource: registerFile,
registerUrlSource: registerUrl,
@@ -936,79 +545,14 @@
Module._load_sidecar_from_source_c(sid);
return sid;
},
// `cache: true` keeps a local OPFS copy filled from the viewer's own
// ranged reads (see the OPFS model cache section above): the next visit
// loads it with zero geometry traffic, and a complete copy still opens
// when the server is unreachable. Falls back to plain URL streaming
// wherever the cache cannot help (no OPFS, no validators from the
// server, another tab already filling this entry).
addUrl: async function (url, o) {
if (o && o.replace) this.clearScene();
let sid = null;
if (o && o.cache) {
const cached = await cachedUrlSource(url).catch(() => null);
if (cached) {
const src = cached.file || cached.source;
sid = Module.__ifcvSources.length;
Module.__ifcvSources.push({ file: src, url: null, size: src.size });
}
}
if (sid === null) sid = await registerUrl(url);
const sid = await registerUrl(url);
if (o && o.name) this.setModelName(sid, o.name);
Module._load_sidecar_from_source_c(sid);
return sid;
},
// What the OPFS cache holds: [{url, size, cachedBytes, complete}] plus
// the browser's storage estimate. Entries whose ledger has not caught
// up with the last few reads under-report slightly; nothing over-reports.
cacheInfo: async function () {
const out = { entries: [], estimate: null };
try {
const dir = await cacheDirHandle(false);
for await (const key of dir.keys()) {
if (!key.endsWith('.meta')) continue;
try {
const meta = JSON.parse(await (await
(await dir.getFileHandle(key)).getFile()).text());
out.entries.push({
url: meta.url,
size: meta.size,
cachedBytes: spansBytes(meta.spans),
complete: spansCover(meta.spans, 0, meta.size),
});
} catch (err) { /* torn meta: skip */ }
}
} catch (err) { /* no cache dir yet */ }
if (navigator.storage && navigator.storage.estimate) {
out.estimate = await navigator.storage.estimate().catch(() => null);
}
return out;
},
// Drop cached models — one URL, or everything. Sources already handed
// to the viewer keep working (open handles and Files stay readable);
// the next visit simply streams from the network again.
clearCache: async function (url) {
try {
const dir = await cacheDirHandle(false);
const only = url ? await cacheEntryName(url) : null;
const names = [];
for await (const key of dir.keys()) {
const base = key.endsWith('.meta') ? key.slice(0, -5) : key;
if (!only || base === only) names.push(key);
}
let n = 0;
for (const key of names) {
await dir.removeEntry(key).catch(() => {});
if (!key.endsWith('.meta')) n++;
}
return n;
} catch (err) {
return 0;
}
},
// ---- Federation ------------------------------------------------------
//
// The concepts an .ifcfed file carries, without the file format: a
+3 -4
View File
@@ -18,7 +18,6 @@
********************************************************************************/
#include "AxisIndicatorRenderer.h"
#include "WgpuDynamicOffsets.h"
#include <algorithm>
#include <cmath>
@@ -342,10 +341,10 @@ void AxisIndicatorRenderer::encodePivot(WGPURenderPassEncoder pass,
wgpuRenderPassEncoderSetVertexBuffer(pass, 0, vertex_buffer_, 0, WGPU_WHOLE_SIZE);
wgpuRenderPassEncoderSetPipeline(pass, pivot_xray_pipeline_);
ifcviewer::setBindGroupDynamic(pass, 0, bind_group_, 1, &xray_off);
wgpuRenderPassEncoderSetBindGroup(pass, 0, bind_group_, 1, &xray_off);
wgpuRenderPassEncoderDraw(pass, kAxisVertexCount, 1, 0, 0);
wgpuRenderPassEncoderSetPipeline(pass, pivot_pipeline_);
ifcviewer::setBindGroupDynamic(pass, 0, bind_group_, 1, &visible_off);
wgpuRenderPassEncoderSetBindGroup(pass, 0, bind_group_, 1, &visible_off);
wgpuRenderPassEncoderDraw(pass, kAxisVertexCount, 1, 0, 0);
}
@@ -405,7 +404,7 @@ void AxisIndicatorRenderer::encodeCornerAxis(WGPUCommandEncoder enc,
0.0f, 1.0f);
wgpuRenderPassEncoderSetPipeline(pass, corner_pipeline_);
wgpuRenderPassEncoderSetVertexBuffer(pass, 0, vertex_buffer_, 0, WGPU_WHOLE_SIZE);
ifcviewer::setBindGroupDynamic(pass, 0, bind_group_, 1, &slot_offset);
wgpuRenderPassEncoderSetBindGroup(pass, 0, bind_group_, 1, &slot_offset);
wgpuRenderPassEncoderDraw(pass, kAxisVertexCount, 1, 0, 0);
wgpuRenderPassEncoderEnd(pass);
wgpuRenderPassEncoderRelease(pass);
+43 -84
View File
@@ -18,9 +18,7 @@
********************************************************************************/
#include "BufferPool.h"
#include "GpuAllocScope.h"
#include <algorithm>
#include <cassert>
#include <cstdio>
#include <cstring>
@@ -61,22 +59,17 @@ bool BufferPool::addSubBuffer() {
if (!device_ || per_sub_buffer_capacity_ == 0) return false;
if (growth_disabled_) return false;
// 64 MB floor: smaller sub-buffers aren't worth the per-allocation
// bookkeeping cost (one bind group per chunk, free-list overhead).
// If the driver won't grant even 64 MB the pool is genuinely at
// its ceiling; growth_disabled_ latches and future grow attempts
// skip the doomed retry.
constexpr uint64_t MIN_SUB_BUFFER_BYTES = 64ull * 1024 * 1024;
uint64_t try_size = last_growth_size_ > 0
? last_growth_size_
: per_sub_buffer_capacity_;
if (try_size < MIN_SUB_BUFFER_BYTES) try_size = MIN_SUB_BUFFER_BYTES;
// Never overshoot the budget: the cache's whole job is to stop short
// of what the required tier needs, and a sub-buffer that straddles
// the line would take exactly the bytes it was told to leave. A
// budget refusal is not a driver refusal, so growth_disabled_ is not
// latched — the budget is the (already lower) ceiling.
if (max_total_capacity_bytes_ > 0) {
const uint64_t total = total_capacity_bytes();
if (total + MIN_SUB_BUFFER_BYTES > max_total_capacity_bytes_) return false;
try_size = std::min(try_size, max_total_capacity_bytes_ - total);
}
#if defined(__EMSCRIPTEN__)
// Web can't synchronously learn whether createBuffer OOM'd: the
// desktop spin-wait that drains PopErrorScope would block the JS
@@ -101,7 +94,7 @@ bool BufferPool::addSubBuffer() {
desc.label.data = label;
desc.label.length = std::strlen(label);
GpuAllocScope scope(instance_, device_);
wgpuDevicePushErrorScope(device_, WGPUErrorFilter_OutOfMemory);
WGPUBuffer buf = wgpuDeviceCreateBuffer(device_, &desc);
SubPool sp;
@@ -114,7 +107,15 @@ bool BufferPool::addSubBuffer() {
last_growth_size_ = try_size;
growth_pending_ = true;
scope.end([this](bool ok) { resolveProvisionalGrowth(!ok); });
WGPUPopErrorScopeCallbackInfo pcb = {};
pcb.mode = WGPUCallbackMode_AllowSpontaneous;
pcb.callback = [](WGPUPopErrorScopeStatus, WGPUErrorType type,
WGPUStringView, void* ud1, void* /*ud2*/) {
static_cast<BufferPool*>(ud1)->resolveProvisionalGrowth(
type != WGPUErrorType_NoError);
};
pcb.userdata1 = this;
wgpuDevicePopErrorScope(device_, pcb);
// No usable space yet: the provisional sub-buffer isn't handed out
// until validated. alloc fails this frame and retries on a later one.
@@ -131,10 +132,31 @@ bool BufferPool::addSubBuffer() {
desc.label.data = label;
desc.label.length = std::strlen(label);
GpuAllocScope scope(instance_, device_);
// wgpu-native classifies "Not enough memory left" as Validation,
// not OutOfMemory. Nested scopes: OOM inner, Validation outer.
wgpuDevicePushErrorScope(device_, WGPUErrorFilter_Validation);
wgpuDevicePushErrorScope(device_, WGPUErrorFilter_OutOfMemory);
WGPUBuffer buf = wgpuDeviceCreateBuffer(device_, &desc);
bool ok = false;
scope.end([&](bool result) { ok = result && buf; });
struct PopResult { bool done = false; bool error = false; };
auto pop = [&](PopResult& pop_result) {
WGPUPopErrorScopeCallbackInfo pcb = {};
pcb.mode = WGPUCallbackMode_AllowProcessEvents;
pcb.callback = [](WGPUPopErrorScopeStatus, WGPUErrorType type,
WGPUStringView, void* ud1, void* /*ud2*/) {
auto* p = static_cast<PopResult*>(ud1);
p->done = true;
p->error = (type != WGPUErrorType_NoError);
};
pcb.userdata1 = &pop_result;
wgpuDevicePopErrorScope(device_, pcb);
while (!pop_result.done) wgpuInstanceProcessEvents(instance_);
};
PopResult oom_pop, validation_pop;
pop(oom_pop);
pop(validation_pop);
const bool ok = buf && !oom_pop.error && !validation_pop.error;
if (ok) {
SubPool sp;
@@ -166,68 +188,6 @@ bool BufferPool::addSubBuffer() {
#endif // __EMSCRIPTEN__
}
uint64_t BufferPool::releaseNewestSubBuffer(
const std::function<void(int sub_idx)>& evict_sub_buffer) {
if (sub_pools_.empty()) return 0;
const int idx = int(sub_pools_.size()) - 1;
if (sub_pools_[size_t(idx)].provisional) return 0;
evict_sub_buffer(idx);
SubPool& sub_pool = sub_pools_[size_t(idx)];
assert(sub_pool.used == 0 && "owner must free every slice before a sub-buffer is released");
if (sub_pool.buffer && sub_pool.owns_handle) {
// Destroy, not just release: the handle may still be
// referenced by in-flight work, and destroy tells the
// backend to reclaim the memory as soon as that completes
// instead of when the last reference goes away.
wgpuBufferDestroy(sub_pool.buffer);
wgpuBufferRelease(sub_pool.buffer);
}
const uint64_t released = sub_pool.capacity;
sub_pools_.pop_back();
return released;
}
namespace {
void logRelease(uint64_t released, uint64_t total, size_t count, uint64_t budget) {
if (released == 0) return;
std::fprintf(stderr,
"[wgpu pool] released %llu MB under memory pressure; pool now %llu MB "
"across %zu sub-buffer(s), budget %llu MB\n",
(unsigned long long)(released / (1024 * 1024)),
(unsigned long long)(total / (1024 * 1024)),
count,
(unsigned long long)(budget / (1024 * 1024)));
}
} // namespace
uint64_t BufferPool::shrinkToCapacity(
uint64_t target_bytes,
const std::function<void(int sub_idx)>& evict_sub_buffer) {
uint64_t released = 0;
while (!sub_pools_.empty()) {
const SubPool& newest = sub_pools_.back();
if (newest.provisional) break;
const uint64_t capacity = total_capacity_bytes();
if (capacity < target_bytes + newest.capacity) break; // would undershoot
released += releaseNewestSubBuffer(evict_sub_buffer);
}
logRelease(released, total_capacity_bytes(), sub_pools_.size(), max_total_capacity_bytes_);
return released;
}
uint64_t BufferPool::releaseAtLeast(
uint64_t bytes,
const std::function<void(int sub_idx)>& evict_sub_buffer) {
uint64_t released = 0;
while (released < bytes) {
const uint64_t got = releaseNewestSubBuffer(evict_sub_buffer);
if (got == 0) break;
released += got;
}
logRelease(released, total_capacity_bytes(), sub_pools_.size(), max_total_capacity_bytes_);
return released;
}
#if defined(__EMSCRIPTEN__)
void BufferPool::resolveProvisionalGrowth(bool failed) {
growth_pending_ = false;
@@ -365,10 +325,9 @@ uint64_t BufferPool::largest_free_run_bytes() const {
void BufferPool::addSubBufferForTesting(WGPUBuffer fake_buffer, uint64_t capacity) {
SubPool sp;
sp.buffer = fake_buffer;
sp.capacity = capacity;
sp.used = 0;
sp.owns_handle = false;
sp.buffer = fake_buffer;
sp.capacity = capacity;
sp.used = 0;
sp.free_ranges.push_back({0, capacity});
sub_pools_.push_back(std::move(sp));
}
+1 -45
View File
@@ -23,7 +23,6 @@
#include <webgpu/webgpu.h>
#include <cstdint>
#include <functional>
#include <string>
#include <vector>
@@ -106,12 +105,6 @@ public:
uint64_t next_growth_size_bytes() const {
return last_growth_size_ > 0 ? last_growth_size_ : per_sub_buffer_capacity_;
}
// Smallest sub-buffer worth adding: below this the per-allocation
// bookkeeping (one bind group per chunk, free-list overhead) outweighs
// the space. Growth that cannot reach the floor — the driver refusing,
// or the budget leaving less than this — is not attempted.
static constexpr uint64_t MIN_SUB_BUFFER_BYTES = 64ull * 1024 * 1024;
// Whether the pool can still attempt to add a sub-buffer. Flips to
// false the first time addSubBuffer is refused even at the floor
// size — eviction callers need this to know whether a future alloc
@@ -119,17 +112,9 @@ public:
bool can_grow() const {
return !growth_disabled_ && per_sub_buffer_capacity_ > 0
&& (max_total_capacity_bytes_ == 0
|| total_capacity_bytes() + MIN_SUB_BUFFER_BYTES
<= max_total_capacity_bytes_);
|| total_capacity_bytes() < max_total_capacity_bytes_);
}
// True once the driver (not the budget) has refused growth even at the
// floor size. On platforms with no memory query this is the only device
// report there is: the owner treats the first refusal as a pressure
// event and carves the required-tier margin out of the cache before a
// required allocation has to fail for it (see ViewportCore::render).
bool growth_was_refused() const { return growth_disabled_; }
// Whether a growth is in flight. On web that window is real time — a
// provisional sub-buffer validates asynchronously a frame or two later — so
// the streaming driver has to know that free space is still on its way and
@@ -142,28 +127,6 @@ public:
// is a bad_alloc that -fno-exceptions turns into an uncatchable abort, so
// the async grow-OOM detection can't save us — we must stop first.
void setMaxTotalCapacity(uint64_t max_bytes) { max_total_capacity_bytes_ = max_bytes; }
uint64_t max_total_capacity_bytes() const { return max_total_capacity_bytes_; }
// Release whole sub-buffers, newest first. Before each is dropped,
// `evict_sub_buffer(sub_idx)` is invoked so the owner can free every
// slice that lives in it — the pool does not know what a slice holds,
// and a sub-buffer is only released once it is empty. Releasing from
// the back keeps every surviving Slice::sub_idx valid. Both return the
// bytes released. This is how the cache yields memory to the required
// tier (see GpuBudget); on web a provisional sub-buffer that is still
// validating is left alone and the caller retries once it resolves.
//
// shrinkToCapacity never goes *below* target_bytes: a sub-buffer is
// released only while doing so keeps capacity ≥ target, so an excess
// smaller than the newest sub-buffer releases nothing (the budget's
// margin absorbs it) instead of dropping 256 MB for the last 36.
uint64_t shrinkToCapacity(uint64_t target_bytes,
const std::function<void(int sub_idx)>& evict_sub_buffer);
// releaseAtLeast frees sub-buffers until at least `bytes` have gone
// (or nothing is left) — for a failed required allocation that needs
// that much back no matter the granularity.
uint64_t releaseAtLeast(uint64_t bytes,
const std::function<void(int sub_idx)>& evict_sub_buffer);
// Proactively add a sub-buffer (no allocation). On web this kicks off the
// async provisional-validation cycle so validated free space appears a
@@ -198,9 +161,6 @@ private:
// capacity/free tallies skip provisional sub-pools so an
// unvalidated (possibly invalid) buffer is never handed out.
bool provisional = false;
// False only for addSubBufferForTesting's fake handles: release
// paths (shrinkToCapacity, destroy) then skip the wgpu calls.
bool owns_handle = true;
};
// Append a new sub-buffer to the pool. Starts at last_growth_size_
@@ -216,10 +176,6 @@ private:
// ≥ MIN_SUB_BUFFER_BYTES; false only when even the minimum size is
// refused, at which point growth_disabled_ latches.
bool addSubBuffer();
// Drop the newest sub-buffer after `evict_sub_buffer` empties it.
// Returns its capacity; 0 when the pool is empty or the newest
// sub-buffer is still provisional (web).
uint64_t releaseNewestSubBuffer(const std::function<void(int sub_idx)>& evict_sub_buffer);
#if defined(__EMSCRIPTEN__)
// Web-only async-growth resolver. Called from the AllowSpontaneous
-4
View File
@@ -145,16 +145,12 @@ set(IFCVIEWER_CORE_SOURCES
InstanceCompose.cpp
LodBuilder.cpp
FederationMath.cpp
GpuAllocScope.cpp
GpuBudget.cpp
GpuMemory.cpp
SidecarCache.cpp
SidecarCompress.cpp
StreamingLoader.cpp
StreamingThread.cpp
SectionGizmoRenderer.cpp
ViewportCore.cpp
WgpuDynamicOffsets.cpp
)
# Web needs a zstd DECODER (Emscripten has no zstd port; the desktop links the
# full libzstd below). Rather than vendor a generated blob, fetch the pinned
-20
View File
@@ -38,26 +38,6 @@ struct FrameStats {
std::uint32_t unique_meshes;
std::uint32_t gl_draw_calls; // wgpu draw-call count; name kept for bonsai parity
std::uint32_t indirect_sub_draws; // sub-draws packed into the chunk-indirect lists
// Chunk geometry pool occupancy (see BufferPool): bytes held by
// resident chunks, the pool's current capacity, and the budget the
// pool may grow to (GpuBudget; 0 when still unbounded).
std::uint64_t vram_used_bytes;
std::uint64_t vram_capacity_bytes;
std::uint64_t vram_budget_bytes;
// The camera's working set: chunks the streaming driver wants resident
// (in frustum and large enough on screen) and how many of those are
// not — i.e. geometry the user should be seeing but is not yet, or
// cannot be because it does not fit the cache. Transiently non-zero
// after any camera move; persistently non-zero means the scene does
// not fit in VRAM.
std::uint32_t chunks_wanted;
std::uint32_t chunks_wanted_missing;
std::uint64_t wanted_missing_bytes; // raw vertex + index bytes of the missing chunks
// Whole-device VRAM from the driver (NVML / sysfs, see GpuMemory.h).
// Desktop only; zero on web or when no backend could answer, so
// consumers must treat 0 as "unknown" rather than as empty.
std::uint64_t device_vram_used_bytes;
std::uint64_t device_vram_total_bytes;
};
#endif // IFCVIEWER_FRAMESTATS_H
-92
View File
@@ -1,92 +0,0 @@
/********************************************************************************
* *
* This file is part of IfcOpenShell. *
* *
* IfcOpenShell is free software: you can redistribute it and/or modify *
* it under the terms of the Lesser GNU General Public License as published by *
* the Free Software Foundation, either version 3.0 of the License, or *
* (at your option) any later version. *
* *
* IfcOpenShell is distributed in the hope that it will be useful, *
* but WITHOUT ANY WARRANTY; without even the implied warranty of *
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the *
* Lesser GNU General Public License for more details. *
* *
* You should have received a copy of the Lesser GNU General Public License *
* along with this program. If not, see <http://www.gnu.org/licenses/>. *
* *
********************************************************************************/
#include "GpuAllocScope.h"
#include <cassert>
namespace {
// Shared between the two pop callbacks; freed by whichever fires last.
struct PendingPop {
GpuAllocScope::Callback on_result;
int remaining = 2;
bool error = false;
// Desktop: points at a flag on end()'s stack frame so the spin-wait
// can observe completion without touching this (freed) object.
bool* done = nullptr;
};
void onPopped(WGPUPopErrorScopeStatus, WGPUErrorType type, WGPUStringView,
void* userdata1, void* /*userdata2*/) {
auto* pending = static_cast<PendingPop*>(userdata1);
if (type != WGPUErrorType_NoError) pending->error = true;
if (--pending->remaining > 0) return;
const bool ok = !pending->error;
bool* done = pending->done;
GpuAllocScope::Callback on_result = std::move(pending->on_result);
delete pending;
on_result(ok);
if (done) *done = true;
}
} // namespace
GpuAllocScope::GpuAllocScope(WGPUInstance instance, WGPUDevice device)
: instance_(instance), device_(device) {
// Validation outer, OutOfMemory inner: each pop sees the errors of
// its own filter, and an OOM reported under either classification
// reaches one of the two.
wgpuDevicePushErrorScope(device_, WGPUErrorFilter_Validation);
wgpuDevicePushErrorScope(device_, WGPUErrorFilter_OutOfMemory);
}
GpuAllocScope::~GpuAllocScope() {
assert(ended_ && "GpuAllocScope::end() must be called exactly once");
}
void GpuAllocScope::end(Callback on_result) {
assert(!ended_);
ended_ = true;
bool done = false;
auto* pending = new PendingPop{std::move(on_result)};
WGPUPopErrorScopeCallbackInfo cb = {};
#if defined(__EMSCRIPTEN__)
// Dawn-web resolves pops from the JS event loop; the caller proceeds
// provisionally and hears back in on_result.
cb.mode = WGPUCallbackMode_AllowSpontaneous;
#else
// wgpu-native fires these from wgpuInstanceProcessEvents, which we
// spin below so on_result has run by the time end() returns.
cb.mode = WGPUCallbackMode_AllowProcessEvents;
pending->done = &done;
#endif
cb.callback = onPopped;
cb.userdata1 = pending;
wgpuDevicePopErrorScope(device_, cb); // OutOfMemory (inner)
wgpuDevicePopErrorScope(device_, cb); // Validation (outer)
#if !defined(__EMSCRIPTEN__)
while (!done) wgpuInstanceProcessEvents(instance_);
#else
(void)done;
#endif
}
-67
View File
@@ -1,67 +0,0 @@
/********************************************************************************
* *
* This file is part of IfcOpenShell. *
* *
* IfcOpenShell is free software: you can redistribute it and/or modify *
* it under the terms of the Lesser GNU General Public License as published by *
* the Free Software Foundation, either version 3.0 of the License, or *
* (at your option) any later version. *
* *
* IfcOpenShell is distributed in the hope that it will be useful, *
* but WITHOUT ANY WARRANTY; without even the implied warranty of *
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the *
* Lesser GNU General Public License for more details. *
* *
* You should have received a copy of the Lesser GNU General Public License *
* along with this program. If not, see <http://www.gnu.org/licenses/>. *
* *
********************************************************************************/
#ifndef IFCVIEWER_GPUALLOCSCOPE_H
#define IFCVIEWER_GPUALLOCSCOPE_H
#include <webgpu/webgpu.h>
#include <functional>
// Brackets one or more wgpu resource creations so an out-of-memory is
// observed instead of silently producing an invalid resource.
//
// WebGPU never returns null from createBuffer / createTexture: a failed
// allocation yields an *error* resource, and the failure is only reported
// through an error scope. Left unobserved it surfaces later as a validation
// error on the first use -- and on wgpu-native an invalid attachment in
// wgpuQueueSubmit is a Rust panic across the FFI boundary, i.e. an abort
// with no recovery path. So every allocation the renderer cannot do
// without goes through one of these.
//
// Two filters are pushed, not one: wgpu-native classifies "Not enough
// memory left" as a Validation error, Dawn as OutOfMemory.
//
// Desktop and web differ only in *when* the answer arrives. On wgpu-native
// the scope pops synchronously (the instance is spun until the callback
// fires) and `end` invokes the callback before returning. On Dawn-web the
// pop is a promise and spinning would deadlock the JS event loop, so the
// callback fires later from the event loop; callers use the resource
// provisionally and correct course in the callback if it turns out bad.
class GpuAllocScope {
public:
using Callback = std::function<void(bool ok)>;
GpuAllocScope(WGPUInstance instance, WGPUDevice device);
~GpuAllocScope();
GpuAllocScope(const GpuAllocScope&) = delete;
GpuAllocScope& operator=(const GpuAllocScope&) = delete;
// Pop the scopes and deliver the verdict: `ok` is true when no error
// fired between construction and here. Must be called exactly once.
void end(Callback on_result);
private:
WGPUInstance instance_ = nullptr;
WGPUDevice device_ = nullptr;
bool ended_ = false;
};
#endif // IFCVIEWER_GPUALLOCSCOPE_H
-86
View File
@@ -1,86 +0,0 @@
/********************************************************************************
* *
* This file is part of IfcOpenShell. *
* *
* IfcOpenShell is free software: you can redistribute it and/or modify *
* it under the terms of the Lesser GNU General Public License as published by *
* the Free Software Foundation, either version 3.0 of the License, or *
* (at your option) any later version. *
* *
* IfcOpenShell is distributed in the hope that it will be useful, *
* but WITHOUT ANY WARRANTY; without even the implied warranty of *
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the *
* Lesser GNU General Public License for more details. *
* *
* You should have received a copy of the Lesser GNU General Public License *
* along with this program. If not, see <http://www.gnu.org/licenses/>. *
* *
********************************************************************************/
#include "GpuBudget.h"
#include <algorithm>
void GpuBudget::bound(std::uint64_t budget) {
if (hard_cap_ > 0) budget = std::min(budget, hard_cap_);
bounded_ = true;
budget_ = std::max(budget, kMinCacheBudgetBytes);
}
void GpuBudget::setHardCap(std::uint64_t hard_cap_bytes) {
hard_cap_ = hard_cap_bytes;
if (hard_cap_ > 0) bound(bounded_ ? budget_ : hard_cap_);
}
void GpuBudget::update(std::uint64_t device_free_bytes,
std::uint64_t cache_capacity_bytes) {
if (device_free_bytes == 0) return;
const std::uint64_t available = cache_capacity_bytes + device_free_bytes;
const std::uint64_t margin = margin_bytes();
const std::uint64_t reading = available > margin ? available - margin : 0;
if (!had_device_report_) {
had_device_report_ = true;
bound(reading);
return;
}
const bool tight = device_free_bytes < margin / 2;
const bool roomy = device_free_bytes > margin + margin / 2 && reading > budget_;
low_reports_ = tight ? low_reports_ + 1 : 0;
high_reports_ = roomy ? high_reports_ + 1 : 0;
if (low_reports_ >= kConfirmReports) {
bound(std::min(budget_, reading));
low_reports_ = 0;
} else if (high_reports_ >= kConfirmReports) {
bound(reading);
high_reports_ = 0;
}
}
bool GpuBudget::onPressure(std::uint64_t cache_capacity_bytes,
std::uint64_t bytes_needed,
std::uint64_t device_free_bytes) {
++pressure_events_;
// The driver refused bytes_needed while reporting device_free_bytes
// free, so at least (free - needed) of what it reports is not really
// available. Remember that so update() stops short of it next time.
if (device_free_bytes > bytes_needed) {
learned_margin_ = std::max(learned_margin_,
device_free_bytes - bytes_needed + kPressureSlackBytes);
}
// What the cache may keep once the failed allocation and its slack
// have been carved out of what it holds right now. The pool's actual
// capacity, not the previous budget, is the honest baseline: the
// budget may never have been reached (unbounded, or growth refused
// earlier by the driver), and lowering a number the pool never hit
// would free nothing.
const std::uint64_t carve = bytes_needed + kPressureSlackBytes;
const std::uint64_t target = cache_capacity_bytes > carve
? cache_capacity_bytes - carve
: 0;
const std::uint64_t lowered = std::max(target, kMinCacheBudgetBytes);
if (bounded_ && lowered >= budget_) return false;
bound(lowered);
low_reports_ = high_reports_ = 0;
return true;
}
-149
View File
@@ -1,149 +0,0 @@
/********************************************************************************
* *
* This file is part of IfcOpenShell. *
* *
* IfcOpenShell is free software: you can redistribute it and/or modify *
* it under the terms of the Lesser GNU General Public License as published by *
* the Free Software Foundation, either version 3.0 of the License, or *
* (at your option) any later version. *
* *
* IfcOpenShell is distributed in the hope that it will be useful, *
* but WITHOUT ANY WARRANTY; without even the implied warranty of *
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the *
* Lesser GNU General Public License for more details. *
* *
* You should have received a copy of the Lesser GNU General Public License *
* along with this program. If not, see <http://www.gnu.org/licenses/>. *
* *
********************************************************************************/
#ifndef IFCVIEWER_GPUBUDGET_H
#define IFCVIEWER_GPUBUDGET_H
#include <cstdint>
// How much device memory the elastic geometry cache (BufferPool) may hold.
//
// GPU memory in the viewer falls in two tiers. *Required* allocations --
// render attachments, per-model metadata, readback staging -- are allocated
// eagerly at deterministic moments (surface configure, model load) and the
// frame cannot be drawn without them. The *cache* -- streamed chunk
// geometry -- is elastic: a chunk that does not fit is simply not resident
// this frame. The rule that keeps the two from colliding is that the cache
// never takes the last byte: it grows only up to this budget, and yields
// whenever a required allocation fails.
//
// The budget is *live*, the way D3D12's QueryVideoMemoryInfo and Vulkan's
// memory_budget are meant to be used: on desktop the driver's free-memory
// report (GpuMemory.h) is polled and
//
// budget = cache capacity + device free - margin
//
// is recomputed each time, so the cache tracks what the device can give as
// other processes come and go. The attachments are eager, so at any poll
// they are already inside "used" at the *actual* surface size; nothing is
// idled for a hypothetical bigger window -- a resize that no longer fits is
// answered by the pressure path instead.
//
// The margin has a fixed part for the required-tier allocations that come
// later (the next model's metadata, staging) and a *learned* part: drivers
// refuse allocations while still reporting memory free (measured here: a
// refusal with 221 MB "free"), and a budget that trusts the report would
// grow straight back into the same refusal after every shrink. A pressure
// event therefore records how much reported-free memory turned out to be
// unusable, and the margin keeps that from then on.
//
// Web has no memory query, so it keeps a fixed ceiling (the wasm heap) and
// pressure feedback alone. The budget's source differs per platform, the
// mechanism does not.
//
// Pure policy, no wgpu: the pool applies the number via
// BufferPool::setMaxTotalCapacity / shrinkToCapacity.
class GpuBudget {
public:
// Below this the viewer cannot keep even a handful of 4 MB chunks
// resident, so there is no point lowering further: a required
// allocation that still fails at the floor is a genuinely exhausted
// device, and the caller degrades instead.
static constexpr std::uint64_t kMinCacheBudgetBytes = 64ull * 1024 * 1024;
// Held back for required allocations made after the cache has grown
// (a later model's metadata buffers, readback staging, driver
// bookkeeping).
static constexpr std::uint64_t kFixedMarginBytes = 256ull * 1024 * 1024;
// Headroom added on top of a failed allocation when lowering the
// budget, so the very next small required allocation does not fail
// again and trigger another shrink cycle.
static constexpr std::uint64_t kPressureSlackBytes = 32ull * 1024 * 1024;
// The live budget moves with every driver report, and reports jitter
// (upload staging, other processes). The pool's ceiling follows the
// budget exactly, but geometry already resident is only evicted once
// the pool is over budget by this much — i.e. once the device's free
// memory has dropped below half the margin — so a transient dip does
// not cost a shrink-and-reload.
static constexpr std::uint64_t kShrinkHysteresisBytes = kFixedMarginBytes / 2;
// Absolute ceiling regardless of device memory (the wasm heap on web).
// 0 = none.
void setHardCap(std::uint64_t hard_cap_bytes);
// Desktop: a fresh driver report. `device_free_bytes` 0 = the query
// could not answer -- ignored, the budget keeps its last value.
//
// The first report bounds the cache outright. After that the budget
// moves only on *sustained* readings, because the report includes
// transients the viewer itself creates -- the upload staging behind a
// burst of chunk loads, a released sub-buffer the driver has not yet
// reclaimed -- and a budget that followed every reading oscillated:
// grow, read a momentary low, shrink, read the rebound, grow again,
// reloading the same chunks every few seconds. So: lower when free
// memory is below half the margin on kConfirmReports consecutive
// reports; raise when it is above 1.5× the margin on as many; between
// those nothing changes. A refused allocation (onPressure) is never
// deferred.
void update(std::uint64_t device_free_bytes,
std::uint64_t cache_capacity_bytes);
static constexpr int kConfirmReports = 2;
// A required allocation of `bytes_needed` failed while the cache held
// `cache_capacity_bytes` and the driver reported `device_free_bytes`
// free (0 = unknown). Lowers the budget so that shrinking the cache to
// it frees bytes_needed + slack, and learns the unusable headroom for
// future update() calls. Returns false when the budget could not be
// lowered any further (already at the floor): the device is exhausted
// and the caller must degrade rather than retry.
bool onPressure(std::uint64_t cache_capacity_bytes,
std::uint64_t bytes_needed,
std::uint64_t device_free_bytes);
// Capacity the pool should shrink to right now, or 0 when it is within
// the hysteresis band (or the budget is unbounded).
std::uint64_t shrinkTarget(std::uint64_t cache_capacity_bytes) const {
if (!bounded_ || cache_capacity_bytes < budget_ + kShrinkHysteresisBytes) return 0;
return budget_;
}
// False until something bounds the cache (a device report, a cap, or
// a pressure event). The pool then grows until the driver refuses,
// exactly as before; the first of those bounds it.
bool bounded() const { return bounded_; }
// Meaningful only when bounded().
std::uint64_t cache_budget_bytes() const { return budget_; }
// Fixed + learned margin applied by update().
std::uint64_t margin_bytes() const { return kFixedMarginBytes + learned_margin_; }
std::uint32_t pressure_events() const { return pressure_events_; }
private:
void bound(std::uint64_t budget);
bool bounded_ = false;
std::uint64_t budget_ = 0;
bool had_device_report_ = false;
int low_reports_ = 0; // consecutive reports below the lower band
int high_reports_ = 0; // consecutive reports above the upper band
std::uint64_t hard_cap_ = 0;
// Reported-free memory that a refusal proved unusable, plus slack.
std::uint64_t learned_margin_ = 0;
std::uint32_t pressure_events_ = 0;
};
#endif // IFCVIEWER_GPUBUDGET_H
-201
View File
@@ -1,201 +0,0 @@
/********************************************************************************
* *
* This file is part of IfcOpenShell. *
* *
* IfcOpenShell is free software: you can redistribute it and/or modify *
* it under the terms of the Lesser GNU General Public License as published by *
* the Free Software Foundation, either version 3.0 of the License, or *
* (at your option) any later version. *
* *
* IfcOpenShell is distributed in the hope that it will be useful, *
* but WITHOUT ANY WARRANTY; without even the implied warranty of *
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the *
* Lesser GNU General Public License for more details. *
* *
* You should have received a copy of the Lesser GNU General Public License *
* along with this program. If not, see <http://www.gnu.org/licenses/>. *
* *
********************************************************************************/
#include "GpuMemory.h"
#include <cstdio>
#include <cstring>
#include <string>
#if defined(__linux__) && !defined(__EMSCRIPTEN__)
#include <dirent.h>
#include <dlfcn.h>
#endif
namespace ifcviewer {
namespace {
#if defined(__linux__) && !defined(__EMSCRIPTEN__)
// NVML, loaded at run time rather than linked: the viewer must run on machines
// with no NVIDIA driver at all, so a link-time dependency is not an option.
// Only the four entry points needed here are resolved.
struct Nvml {
void* handle = nullptr;
int (*init)() = nullptr;
int (*shutdown)() = nullptr;
int (*device_count)(unsigned*) = nullptr;
int (*handle_by_index)(unsigned, void**) = nullptr;
int (*pci_info)(void*, void*) = nullptr;
int (*memory_info)(void*, unsigned long long*) = nullptr;
bool load() {
// .so.1 first: the unversioned name is part of the -dev package and is
// frequently absent on user machines.
for (const char* name : {"libnvidia-ml.so.1", "libnvidia-ml.so"}) {
handle = dlopen(name, RTLD_LAZY | RTLD_LOCAL);
if (handle) break;
}
if (!handle) return false;
auto sym = [&](const char* n) { return dlsym(handle, n); };
init = (int (*)())sym("nvmlInit_v2");
shutdown = (int (*)())sym("nvmlShutdown");
device_count = (int (*)(unsigned*))sym("nvmlDeviceGetCount_v2");
handle_by_index = (int (*)(unsigned, void**))sym("nvmlDeviceGetHandleByIndex_v2");
pci_info = (int (*)(void*, void*))sym("nvmlDeviceGetPciInfo_v3");
memory_info = (int (*)(void*, unsigned long long*))sym("nvmlDeviceGetMemoryInfo");
return init && shutdown && device_count && handle_by_index && memory_info;
}
~Nvml() { if (handle) dlclose(handle); }
};
// nvmlPciInfo_t. Only pciDeviceId is read; the leading char arrays are sized
// from the NVML headers so the offset is right.
struct NvmlPciInfo {
char busIdLegacy[16];
unsigned domain;
unsigned bus;
unsigned device;
unsigned pciDeviceId; // (device_id << 16) | vendor_id
unsigned pciSubSystemId;
char busId[32];
};
bool queryNvml(std::uint32_t vendor_id, std::uint32_t device_id, GpuMemoryInfo& out) {
Nvml nvml;
if (!nvml.load()) return false;
if (nvml.init() != 0) return false;
unsigned count = 0;
bool found = false;
if (nvml.device_count(&count) == 0) {
for (unsigned i = 0; i < count && !found; ++i) {
void* dev = nullptr;
if (nvml.handle_by_index(i, &dev) != 0 || !dev) continue;
// Match the card wgpu picked. With a single NVIDIA device and no
// way to read its ids, fall through to it rather than reporting
// nothing -- a slightly uncertain number beats none.
if (nvml.pci_info && (vendor_id || device_id)) {
NvmlPciInfo pci{};
if (nvml.pci_info(dev, &pci) == 0) {
const unsigned dev_id = (pci.pciDeviceId >> 16) & 0xFFFF;
const unsigned ven_id = pci.pciDeviceId & 0xFFFF;
if (device_id && dev_id != device_id) continue;
if (vendor_id && ven_id != vendor_id) continue;
}
} else if (count != 1) {
continue;
}
// nvmlMemory_t: { total, free, used }, all unsigned long long.
//
// This reports more `used` than nvidia-smi does -- measured here,
// 1427 MiB against 1046 MiB, consistently -- because the v1 call
// folds driver-reserved memory into `used` where nvidia-smi
// accounts for it separately. The larger figure is the one worth
// having: what actually allocated on this card topped out around
// 2431 MB, against 2669 MB free by this measure and 3050 MB by
// nvidia-smi's. Budgeting against the optimistic number would
// promise memory that is not there.
unsigned long long mem[3] = {0, 0, 0};
if (nvml.memory_info(dev, mem) == 0 && mem[0] > 0) {
out.total_bytes = mem[0];
out.used_bytes = mem[2];
out.valid = true;
found = true;
}
}
}
nvml.shutdown();
return found;
}
// amdgpu and i915 expose VRAM through sysfs. Reads every card and keeps the
// one whose vendor/device ids match, because the first card is often the
// integrated GPU rather than the one in use.
bool readUint64(const std::string& path, std::uint64_t& out) {
FILE* f = std::fopen(path.c_str(), "r");
if (!f) return false;
unsigned long long v = 0;
const bool ok = std::fscanf(f, "%llu", &v) == 1;
std::fclose(f);
if (ok) out = v;
return ok;
}
bool readHexId(const std::string& path, std::uint32_t& out) {
FILE* f = std::fopen(path.c_str(), "r");
if (!f) return false;
unsigned v = 0;
const bool ok = std::fscanf(f, "0x%x", &v) == 1;
std::fclose(f);
if (ok) out = v;
return ok;
}
bool querySysfs(std::uint32_t vendor_id, std::uint32_t device_id, GpuMemoryInfo& out) {
DIR* dir = opendir("/sys/class/drm");
if (!dir) return false;
bool found = false;
while (dirent* entry = readdir(dir)) {
const std::string name = entry->d_name;
// "card0", not "card0-DP-1".
if (name.rfind("card", 0) != 0 || name.find('-') != std::string::npos) continue;
const std::string base = "/sys/class/drm/" + name + "/device/";
std::uint32_t ven = 0, dev = 0;
if (vendor_id && readHexId(base + "vendor", ven) && ven != vendor_id) continue;
if (device_id && readHexId(base + "device", dev) && dev != device_id) continue;
std::uint64_t total = 0, used = 0;
if (readUint64(base + "mem_info_vram_total", total) && total > 0) {
readUint64(base + "mem_info_vram_used", used);
out.total_bytes = total;
out.used_bytes = used;
out.valid = true;
found = true;
break;
}
}
closedir(dir);
return found;
}
#endif // __linux__ && !__EMSCRIPTEN__
} // namespace
GpuMemoryInfo queryGpuMemory(std::uint32_t vendor_id, std::uint32_t device_id) {
GpuMemoryInfo info;
#if defined(__linux__) && !defined(__EMSCRIPTEN__)
if (queryNvml(vendor_id, device_id, info)) return info;
if (querySysfs(vendor_id, device_id, info)) return info;
#else
// Windows (DXGI QueryVideoMemoryInfo) and macOS
// (recommendedMaxWorkingSetSize) both expose this; not implemented here
// because neither can be verified from this machine. `valid` stays false,
// and callers fall back to behaving as they did before.
(void)vendor_id; (void)device_id;
#endif
return info;
}
} // namespace ifcviewer
-62
View File
@@ -1,62 +0,0 @@
/********************************************************************************
* *
* This file is part of IfcOpenShell. *
* *
* IfcOpenShell is free software: you can redistribute it and/or modify *
* it under the terms of the Lesser GNU General Public License as published by *
* the Free Software Foundation, either version 3.0 of the License, or *
* (at your option) any later version. *
* *
* IfcOpenShell is distributed in the hope that it will be useful, *
* but WITHOUT ANY WARRANTY; without even the implied warranty of *
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the *
* Lesser GNU General Public License for more details. *
* *
* You should have received a copy of the Lesser GNU General Public License *
* along with this program. If not, see <http://www.gnu.org/licenses/>. *
* *
********************************************************************************/
#ifndef GPUMEMORY_H
#define GPUMEMORY_H
#include <cstdint>
// How much video memory the card has, and how much of it is in use.
//
// WebGPU deliberately exposes neither -- `maxBufferSize` reports 1 TB on this
// stack and is useless as a proxy -- so this goes outside the graphics API.
// That is legitimate on desktop, where the viewer is a native app, and it is
// the number every other part of the residency story needs: a gauge to show
// the user, a budget to keep the pool under, and a pre-flight check that can
// refuse a model before it wedges the session.
//
// Matching the right GPU matters. On a laptop with switchable graphics the
// obvious sysfs entry is often the *integrated* chip rather than the one wgpu
// selected -- measured here: /sys/class/drm/card1 reports a 512 MB AMD iGPU
// while wgpu is running on a 4 GB GeForce. So the query takes the vendor and
// device ids from WGPUAdapterInfo and matches on them.
namespace ifcviewer {
struct GpuMemoryInfo {
std::uint64_t total_bytes = 0;
std::uint64_t used_bytes = 0;
// False when no backend could answer -- an unknown driver, a platform
// without a query, or a device the probe could not match. Callers must
// treat that as "unknown" and not as "zero": refusing to load because an
// unavailable query returned 0 would be worse than not asking.
bool valid = false;
std::uint64_t free_bytes() const {
return total_bytes > used_bytes ? total_bytes - used_bytes : 0;
}
};
// Query the GPU wgpu selected. `vendor_id` / `device_id` come from
// wgpuAdapterGetInfo. Cheap enough to call once a second; not per frame.
GpuMemoryInfo queryGpuMemory(std::uint32_t vendor_id, std::uint32_t device_id);
} // namespace ifcviewer
#endif
-47
View File
@@ -395,25 +395,6 @@ struct ModelGpuData {
std::vector<MeshInfo> meshes;
std::vector<InstanceInfo> instances;
// The cull-hot per-instance fields packed contiguously. InstanceInfo is
// 232 bytes with the AABB 200 bytes from the ids, so the per-frame cull
// paid two or three cache lines per instance — at half a million
// instances that is the whole frame budget on the single-threaded web
// build. 40 bytes per entry here makes the walk sequential. Rebuilt by
// rebuildCullInstances wherever instances change (applyCachedModel,
// uploadInstanceRecords — which every recompose and colour change
// already funnels through).
struct CullInstance {
float aabb_min[3];
float aabb_max[3];
std::uint32_t mesh_id;
std::uint32_t object_id;
std::uint32_t color_override_rgba8;
std::uint32_t chunk_idx;
};
static_assert(sizeof(CullInstance) == 40, "keep the cull walk dense");
std::vector<CullInstance> cull_instances;
// Per-mesh "any vertex has alpha < 255?" flag, indexed by mesh_id.
// Populated at uploadStreamedMesh / applyStreamedChunk as vertex bytes
// become CPU-resident. Used at cull time to classify each instance
@@ -453,19 +434,6 @@ struct ModelGpuData {
std::vector<uint32_t> indices; // 3 * triangle_count, LOD0
};
std::vector<MeshTriangles> mesh_triangles_cache;
// How many RESIDENT chunks currently contain each mesh (the spatial
// planner may duplicate a mesh into several chunks). Maintained by
// applyStreamedChunk / unloadChunk; when it drops to zero the mesh's
// mesh_triangles_cache entry is released — the shadow follows GPU
// residency instead of accumulating every mesh ever loaded, which on
// a large federation grew monotonically toward the whole scene's
// geometry on the CPU heap. mesh_local_volumes is NOT released: the
// Volume tool needs it for evicted meshes too, and it is 8 B/mesh.
std::vector<std::uint16_t> mesh_resident_chunk_refs;
// Bytes currently held by mesh_triangles_cache, maintained at the fill
// (applyStreamedChunk) and release (unloadChunk) sites so the heartbeat
// log can report the shadow without walking every mesh per frame.
std::uint64_t cpu_shadow_bytes = 0;
// object_id (globally rebased) → instance index in `instances`.
// Populated alongside the instance vector so the Volume tool can do
@@ -479,15 +447,6 @@ struct ModelGpuData {
// is gone; cull iterates m.chunks instead.
bool hidden = false;
// Unloaded by the user: every chunk evicted and the model's own GPU
// buffers released, while the CPU mirrors (meshes, instances, chunk
// plan, element metadata) stay so the entry remains in the scene and
// loadModel can bring it back without touching the disk. Distinct
// from hidden (a viewing state; the geometry may stay resident) and
// from removal (the model leaves the scene).
bool unloaded = false;
// Whether cull / draw / pick / streaming should consider this model.
bool drawable() const { return !hidden && !unloaded; }
// Per-model federation matrices in metres. Default identity → no
// per-model contribution to the composed transform. See bonsai's
@@ -516,11 +475,5 @@ struct ModelGpuData {
// ranges via `pool.free()`) and clear its size mirrors. Safe to call
// repeatedly; idempotent on already-released entries.
void releaseWgpuModelGpuData(ModelGpuData& m, BufferPool& pool);
// Just the model's own (non-pool) wgpu buffers: mesh + instance storage
// and the per-chunk cull buffers. Chunk bookkeeping is left intact so the
// buffers can be re-created — the undo step of a failed model load.
void releaseModelBuffers(ModelGpuData& m);
// Refresh ModelGpuData::cull_instances from instances + instance_chunk_idx.
void rebuildCullInstances(ModelGpuData& m);
#endif // WGPUMODELGPUDATA_H
+1 -2
View File
@@ -18,7 +18,6 @@
********************************************************************************/
#include "OverlayRenderer.h"
#include "WgpuDynamicOffsets.h"
#include <QFont>
#include <QFontMetrics>
@@ -887,7 +886,7 @@ void OverlayRenderer::encodeOverlayLines(WGPURenderPassEncoder pass,
wgpuQueueWriteBuffer(queue_, overlay_line_uniform_buffer_,
slot_off + 96, viewport, sizeof(viewport));
const uint32_t dynamic_offsets[1] = { uint32_t(slot_off) };
ifcviewer::setBindGroupDynamic(pass, 0, overlay_line_bind_group_,
wgpuRenderPassEncoderSetBindGroup(pass, 0, overlay_line_bind_group_,
1, dynamic_offsets);
wgpuRenderPassEncoderDraw(pass, d.vertex_count, 1, d.first_vertex, 0);
}
+1 -2
View File
@@ -18,7 +18,6 @@
********************************************************************************/
#include "SectionGizmoRenderer.h"
#include "WgpuDynamicOffsets.h"
#include <algorithm>
#include <array>
@@ -336,7 +335,7 @@ void SectionGizmoRenderer::encode(WGPURenderPassEncoder pass, const Eigen::Matri
bitangent, nn, tr, tg, tb, 1.0f, vw, vh);
const uint32_t slot_offset = uint32_t(i) * kSectionUniformSlot;
wgpuQueueWriteBuffer(queue_, uniform_buffer_, slot_offset, slot, sizeof(slot));
ifcviewer::setBindGroupDynamic(pass, 0, bind_group_, 1, &slot_offset);
wgpuRenderPassEncoderSetBindGroup(pass, 0, bind_group_, 1, &slot_offset);
wgpuRenderPassEncoderDraw(pass, uint32_t(vertex_count_), 1, 0, 0);
}
}
File diff suppressed because it is too large Load Diff
+2 -158
View File
@@ -49,8 +49,6 @@
#include "AxisIndicatorRenderer.h"
#include "BufferPool.h"
#include "GpuBudget.h"
#include "GpuMemory.h"
#include "InstanceCompose.h"
#include "InstancedGeometry.h"
#include "ModelGpuData.h"
@@ -157,17 +155,6 @@ public:
void resetScene();
void hideModel(uint32_t session_model_id);
void showModel(uint32_t session_model_id);
// Release a model's GPU memory (every chunk + its own buffers) while
// keeping it in the scene; loadModel recreates the buffers from the
// CPU mirrors and lets chunks stream back. Neither touches hidden.
// loadModel returns false when the device cannot fit the model's
// buffers even after the cache yielded (it stays unloaded).
void unloadModel(uint32_t session_model_id);
bool loadModel(uint32_t session_model_id);
bool isModelUnloaded(uint32_t session_model_id) const;
// Bytes this model currently holds on the GPU: resident chunk
// geometry plus its mesh/instance/cull buffers. 0 when unloaded.
std::uint64_t modelVramBytes(uint32_t session_model_id) const;
// Federation matrix setters. Each writes to model state and posts
// a recompose so per-instance world matrices stay consistent with
@@ -494,12 +481,6 @@ public:
// brings them in. Triggers an auto-viewAll on the first model (so a
// freshly-loaded scene frames itself).
void applyCachedModel(std::uint32_t session_model_id, StreamingSidecar metadata);
// The model's required-tier buffers (mesh + instance storage, per-chunk
// cull buffers) as one allocation unit — see allocateRequired. False
// when the device cannot fit them even after the cache yielded.
bool createModelBuffers(std::uint32_t session_model_id, ModelGpuData& m,
const std::vector<MeshGpu>& mesh_gpu,
const std::vector<InstanceGpu>& inst_gpu);
// Qt-free sidecar load: readSidecarMetadata + applyCachedModel.
// Used by the web build (and any other non-Qt embedder) so the
@@ -597,20 +578,6 @@ public:
// resident. On web that means calling loadAllElementMetadataWeb first —
// models still lazily un-fetched simply contribute nothing.
std::vector<ElementRef> elements() const;
// One model's elements (by load-order index, same as modelProgress),
// handed out as slices into the model's string table — valid only for
// the duration of the visit, no per-element string copies. The web
// objects export serialises hundreds of thousands of elements straight
// from these; materialising ElementRefs there tripled the peak heap.
struct ElementSlices {
std::uint32_t object_id = 0;
int source_id = -1;
const char* guid = nullptr; std::uint32_t guid_len = 0;
const char* name = nullptr; std::uint32_t name_len = 0;
const char* type = nullptr; std::uint32_t type_len = 0;
};
void visitModelElements(int model_index,
const std::function<void(const ElementSlices&)>& visit) const;
// The single element behind one object_id — the pick path's lookup, which
// must not pay for materialising the whole table. Scans only the model that
@@ -738,9 +705,6 @@ public:
// dimensions match. Resets ping-pong state so any in-flight map is
// dropped (caller already ensured the surface resize blocked).
void ensureHizTextures(int viewport_w, int viewport_h);
// Drop just the resolve texture + staging buffers (pipeline stays),
// resetting the ping-pong state. ensureHizTextures recreates them.
void releaseHizTextures();
// Tear down every HiZ-owned wgpu resource (pipeline + textures +
// staging buffers + pyramid). Called from shutdown() before
@@ -836,13 +800,8 @@ public:
bool buildPickPipeline();
// (Re)allocate the pick MRT attachments + readback staging buffers
// to the supplied size. Idempotent when dimensions match. Created
// eagerly with the other attachments in configureSurface; the pick
// entry points call it again only as the retry after a pressure
// shrink, and bail when it returns false.
bool ensurePickAttachments(int w, int h);
// The raw (unscoped) creation ensurePickAttachments wraps.
void createPickAttachments(int w, int h);
// to the supplied size. Idempotent when dimensions match.
void ensurePickAttachments(int w, int h);
// Encode the one-shot pick pass + copy the (x, y) texel into the pick
// staging buffer(s) and submit. Shared by the sync (pickObjectAt) and
@@ -1093,79 +1052,6 @@ public:
private:
bool createPool();
// ---- Memory tiers (see GpuBudget.h) ------------------------------------
//
// Every allocation the frame cannot do without — the per-pixel
// attachments, a model's metadata buffers, readback staging — is
// "required" and goes through one of these so an out-of-memory is
// observed and answered by shrinking the geometry cache, instead of
// surfacing as an invalid resource that aborts in wgpuQueueSubmit.
// Bytes every per-pixel attachment set costs (MSAA colour + depth,
// selection mask trio, pick MRT + depth) — sizes the pressure
// carve-out when an attachment set fails.
static std::uint64_t attachmentBytesPerPixel();
// Desktop: the driver's view of the adapter wgpu picked (GpuMemory.h);
// `valid` false on web or an unsupported driver.
ifcviewer::GpuMemoryInfo queryDeviceMemory() const;
// Desktop, at most once a second from render(): refresh the device
// figures for FrameStats and re-derive the live cache budget from
// them, shrinking the pool when the device has less to give than the
// pool holds (another process took memory).
void pollDeviceMemory();
// Push budget_ to the pool: the growth ceiling, and a shrink when the
// pool is over it by at least a sub-buffer.
void applyBudgetToPool();
// A required allocation of `bytes` (`what` names it for the log)
// failed. Lowers the budget, evicts and releases cache sub-buffers
// down to it, and on desktop waits for the device to actually reclaim
// them so an immediate retry can succeed. Returns false when the
// cache had nothing left to give: the device is exhausted and the
// caller degrades (skips the operation) rather than retrying.
bool onRequiredAllocationFailed(const char* what, std::uint64_t bytes);
// Unload every resident chunk whose slices live in pool sub-buffer
// `sub_idx`; the evictor BufferPool::shrinkToCapacity calls before it
// releases that sub-buffer.
void evictChunksInSubBuffer(int sub_idx);
// Run `create` (one or more wgpu allocations totalling ~`bytes`) under
// an allocation scope. Desktop: verified synchronously; on failure
// `release` undoes the attempt, the cache yields, and `create` runs
// again, until it succeeds or the cache has nothing left to give
// (false). Web: the resources are used
// provisionally and true is returned; if the scope later reports a
// failure the cache yields and `on_web_failure` (if any) corrects
// course, since the caller has long since moved on.
bool allocateRequired(const char* what, std::uint64_t bytes,
const std::function<void()>& create,
const std::function<void()>& release,
std::function<void()> on_web_failure = {});
// allocateRequired for a single buffer: the buffer, or null when the
// device could not fit it even after the cache yielded.
WGPUBuffer createRequiredBuffer(const WGPUBufferDescriptor& desc,
const char* what);
// (Re)create every per-pixel attachment for a width_px × height_px
// surface as one required allocation. False when they could not be
// allocated even after the cache yielded; render() then skips the
// frame rather than submitting with invalid views.
bool ensureRenderAttachments(int width_px, int height_px);
void releaseRenderAttachments();
GpuBudget budget_;
// Latch: the pool's first driver-refused growth has been answered by
// carving the margin out of the cache (see render()).
bool pool_growth_refusal_handled_ = false;
// Adapter ids, read once at init, for matching the driver's memory
// report to the card wgpu is actually using.
std::uint32_t adapter_vendor_id_ = 0;
std::uint32_t adapter_device_id_ = 0;
// Latched false by ensureRenderAttachments when the device could not
// fit the attachments; re-evaluated on the next configureSurface.
bool render_attachments_ok_ = true;
// The scene's models in load order (ascending session_model_id, minted at
// request time — see loadSidecarMetadataWeb). Every per-model API indexes
// against this, so a model keeps a stable UI slot instead of hopping with
@@ -1594,57 +1480,15 @@ private:
// for parallel-vs-serial benchmarking. Default ON.
bool cull_threads_enabled_ = true;
// ---- Cull-input tracking -------------------------------------------
//
// The CPU cull is the single largest per-frame cost (the whole frame on
// the single-threaded web build), and most requested frames do not
// change its inputs — overlay redraws, pick feedback, streaming frames
// where no chunk actually landed. scene_epoch_ is bumped by everything
// that can alter a cull's outcome besides the camera (residency,
// visibility, colours, transforms, model set, HiZ pyramid updates);
// render() re-culls only when the epoch, the camera, or a cull-relevant
// setting changed, and otherwise draws from the buffers the last cull
// uploaded.
std::uint64_t scene_epoch_ = 0;
void markCullInputsChanged() { ++scene_epoch_; }
bool has_last_cull_ = false;
Eigen::Matrix4f last_cull_vp_ = Eigen::Matrix4f::Zero();
std::uint64_t last_cull_epoch_ = 0;
float last_cull_min_px_ = -1.0f;
float last_cull_lod_px_ = -1.0f;
float last_cull_xray_ = -1.0f;
bool last_cull_hiz_ = false;
// Per-frame stats latched by render() for FrameStats emission +
// the interactive heartbeat / bench per-frame line.
std::uint32_t last_visible_objects_ = 0;
std::uint32_t last_visible_triangles_ = 0;
std::uint32_t last_sub_draws_ = 0;
// Device-wide VRAM readout for FrameStats and the live cache budget
// (pollDeviceMemory). The driver query is too slow for per-frame use,
// so it is re-polled at most once a second and the last answer is
// repeated in between.
std::uint64_t device_vram_used_bytes_ = 0;
std::uint64_t device_vram_total_bytes_ = 0;
Stopwatch device_vram_poll_timer_;
std::size_t polled_sub_buffer_count_ = 0;
double last_cull_ms_ = 0.0;
double last_cull_compute_ms_ = 0.0;
double last_cull_upload_ms_ = 0.0;
double last_stream_ms_ = 0.0;
// Motion-cull latch. The coarse motion threshold used to follow the
// per-frame "did the camera move" test directly, which flip-flops
// during a slow low-fps drag: coalesced mouse events leave frames
// where the camera happens not to change, so the cull alternated
// between the 3 px and 15 px thresholds — most of the scene vanishing
// and reappearing every few frames, with a full visible-set re-upload
// at each flip. The latch holds the coarse threshold until the camera
// has been still for kMotionHoldMs, so a drag degrades once at its
// start and restores once, shortly after it ends.
static constexpr int kMotionHoldMs = 250;
bool motion_cull_latched_ = false;
Stopwatch motion_hold_timer_;
// True when the cull just used motion_min_pixel_radius_ — render()
// schedules one more frame so the camera-now-stopped state recomputes
// the cull at the still threshold and previously dropped sub-pixel
-7
View File
@@ -99,13 +99,6 @@ public:
// is encoded; QtViewportHost forwards to `emit frameStatsUpdated(...)`.
virtual void onFrameStats(const FrameStats& /*stats*/) {}
// Whether this host's tools need the CPU-side triangle shadow
// (ModelGpuData::mesh_triangles_cache) that surface raycasts and the
// measurement tools read. It costs 12 B/vertex + 4 B/index of heap for
// every resident mesh, so hosts without those tools (the web viewer,
// for now) skip populating it entirely.
virtual bool wantsCpuMeshTriangles() const { return true; }
// Overlay encode hooks. ViewportCore::render() calls these mid-
// frame so the Qt-bound OverlayRenderer (which carries QString
// labels for the HUD) can encode its passes without core having
-4
View File
@@ -594,10 +594,6 @@ void ViewportWindow::removeModel(uint32_t session_model_id) { core_.removeMode
void ViewportWindow::resetScene() { core_.resetScene(); }
void ViewportWindow::hideModel(uint32_t session_model_id) { core_.hideModel(session_model_id); }
void ViewportWindow::showModel(uint32_t session_model_id) { core_.showModel(session_model_id); }
void ViewportWindow::unloadModel(uint32_t session_model_id) { core_.unloadModel(session_model_id); }
bool ViewportWindow::loadModel(uint32_t session_model_id) { return core_.loadModel(session_model_id); }
bool ViewportWindow::isModelUnloaded(uint32_t session_model_id) const { return core_.isModelUnloaded(session_model_id); }
std::uint64_t ViewportWindow::modelVramBytes(uint32_t session_model_id) const { return core_.modelVramBytes(session_model_id); }
void ViewportWindow::setFederatedFalseOrigin(const Eigen::Matrix4d& m) {
core_.setFederatedFalseOrigin(m);
-7
View File
@@ -142,13 +142,6 @@ public:
// consults. requestUpdate() so the change is visible immediately.
void hideModel(uint32_t session_model_id);
void showModel(uint32_t session_model_id);
// GPU residency of a model, independent of visibility: unloadModel
// frees everything it holds on the device while it stays in the
// scene; loadModel brings it back (false if the device cannot fit it).
void unloadModel(uint32_t session_model_id);
bool loadModel(uint32_t session_model_id);
bool isModelUnloaded(uint32_t session_model_id) const;
std::uint64_t modelVramBytes(uint32_t session_model_id) const;
// Federation pipeline: composed instance transform =
// FederatedFalseOrigin · ModelTransformation · CoordinateOperation
-65
View File
@@ -1,65 +0,0 @@
/********************************************************************************
* *
* This file is part of IfcOpenShell. *
* *
* IfcOpenShell is free software: you can redistribute it and/or modify *
* it under the terms of the Lesser GNU General Public License as published by *
* the Free Software Foundation, either version 3.0 of the License, or *
* (at your option) any later version. *
* *
* IfcOpenShell is distributed in the hope that it will be useful, *
* but WITHOUT ANY WARRANTY; without even the implied warranty of *
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the *
* Lesser GNU General Public License for more details. *
* *
* You should have received a copy of the Lesser GNU General Public License *
* along with this program. If not, see <http://www.gnu.org/licenses/>. *
* *
********************************************************************************/
#include "WgpuDynamicOffsets.h"
#ifdef __EMSCRIPTEN__
#include <emscripten/em_js.h>
#endif
namespace ifcviewer {
#ifdef __EMSCRIPTEN__
namespace {
EM_JS(void, ifcv_set_bind_group_dynamic_js,
(WGPURenderPassEncoder pass, uint32_t group_index, WGPUBindGroup group,
uint32_t offsets_ptr, uint32_t count), {
// HEAPU32.slice() copies into a freshly allocated buffer of exactly
// `count` elements; .subarray() would alias the whole heap again and
// reintroduce the bug this function exists to avoid.
var start = offsets_ptr >>> 2;
var small = HEAPU32.slice(start, start + count);
WebGPU.getJsObject(pass).setBindGroup(
group_index, WebGPU.getJsObject(group), small, 0, count);
});
} // namespace
void setBindGroupDynamic(WGPURenderPassEncoder pass, uint32_t group_index,
WGPUBindGroup group, uint32_t count,
const uint32_t* offsets) {
if (count == 0) {
wgpuRenderPassEncoderSetBindGroup(pass, group_index, group, 0, nullptr);
return;
}
ifcv_set_bind_group_dynamic_js(pass, group_index, group,
uint32_t(reinterpret_cast<uintptr_t>(offsets)), count);
}
#else
void setBindGroupDynamic(WGPURenderPassEncoder pass, uint32_t group_index,
WGPUBindGroup group, uint32_t count,
const uint32_t* offsets) {
wgpuRenderPassEncoderSetBindGroup(pass, group_index, group, count, offsets);
}
#endif
} // namespace ifcviewer
-61
View File
@@ -1,61 +0,0 @@
/********************************************************************************
* *
* This file is part of IfcOpenShell. *
* *
* IfcOpenShell is free software: you can redistribute it and/or modify *
* it under the terms of the Lesser GNU General Public License as published by *
* the Free Software Foundation, either version 3.0 of the License, or *
* (at your option) any later version. *
* *
* IfcOpenShell is distributed in the hope that it will be useful, *
* but WITHOUT ANY WARRANTY; without even the implied warranty of *
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the *
* Lesser GNU General Public License for more details. *
* *
* You should have received a copy of the Lesser GNU General Public License *
* along with this program. If not, see <http://www.gnu.org/licenses/>. *
* *
********************************************************************************/
#ifndef WGPUDYNAMICOFFSETS_H
#define WGPUDYNAMICOFFSETS_H
#include <cstdint>
#include <webgpu/webgpu.h>
namespace ifcviewer {
// setBindGroup with dynamic offsets. Always use this instead of calling
// wgpuRenderPassEncoderSetBindGroup with a nonzero offset count.
//
// Emscripten's generated WebGPU shim implements the dynamic-offset path as
//
// pass.setBindGroup(index, group, HEAPU32, ptr >>> 2, count);
//
// where HEAPU32 is the persistent view over the *entire* wasm linear memory.
// Browsers validate the byte length of the whole backing buffer handed to
// setBindGroup, not the (start, length) slice actually read, and reject
// anything over 2 GB:
//
// TypeError: GPURenderPassEncoder.setBindGroup: Argument 3 can't be an
// ArrayBuffer or an ArrayBufferView larger than 2 GB
//
// So once the heap grows past 2^31 bytes every dynamic-offset draw throws, on
// every frame, for the life of the page -- and this build deliberately allows
// that (ALLOW_MEMORY_GROWTH with MAXIMUM_MEMORY=4 GB, because large
// federations need the headroom). The offsets are only a handful of uint32_t,
// so the web implementation copies them into a small short-lived Uint32Array.
// Native builds forward straight through; wgpu-native reads the pointer
// directly and has no such limit.
//
// Defined out-of-line in WgpuDynamicOffsets.cpp: the web path is an EM_JS
// function, and EM_JS emits real per-translation-unit symbols that collide at
// link time if instantiated in more than one TU.
void setBindGroupDynamic(WGPURenderPassEncoder pass, uint32_t group_index,
WGPUBindGroup group, uint32_t count,
const uint32_t* offsets);
} // namespace ifcviewer
#endif
+1 -7
View File
@@ -109,12 +109,6 @@ endif()
add_ifcviewer_unit_test(test_selection)
add_ifcviewer_unit_test(test_visibility)
# GpuBudget: the pure policy deciding how much device memory the geometry
# cache may hold and how it yields under pressure. No wgpu at all.
add_ifcviewer_unit_test(test_gpu_budget
SOURCES ${IFCVIEWER_SRC}/GpuBudget.cpp
)
# BufferPool sub-allocator invariants. The pool's wgpu calls live inside
# addSubBuffer() (the growth path); tests use the addSubBufferForTesting
# seam to preseed sub-pools with fake handles, so the only wgpu touchpoint
@@ -123,7 +117,7 @@ add_ifcviewer_unit_test(test_gpu_budget
# the pool go out of scope holding any. Linking wgpu_native satisfies the
# symbol regardless.
add_ifcviewer_unit_test(test_buffer_pool
SOURCES ${IFCVIEWER_SRC}/BufferPool.cpp ${IFCVIEWER_SRC}/GpuAllocScope.cpp
SOURCES ${IFCVIEWER_SRC}/BufferPool.cpp
LIBS wgpu_native
)
if(UNIX AND NOT APPLE AND WGPU_NATIVE_LIB_DIR)
-98
View File
@@ -34,7 +34,6 @@
#include <catch2/catch_all.hpp>
#include <cstdint>
#include <vector>
namespace {
@@ -253,100 +252,3 @@ TEST_CASE("free with invalid slice is a no-op", "[buffer_pool]") {
pool.free(a);
REQUIRE(pool.total_used_bytes() == 0);
}
// ---- Budget ceiling + shrink (the cache yielding to the required tier) ----
TEST_CASE("can_grow respects the total-capacity budget with sub-buffer granularity", "[buffer_pool]") {
BufferPool pool;
FakePoolGuard guard{pool};
// No device configured, so can_grow is false regardless; the budget
// arithmetic is what we check via max_total_capacity_bytes.
pool.setMaxTotalCapacity(256ull * 1024 * 1024);
REQUIRE(pool.max_total_capacity_bytes() == 256ull * 1024 * 1024);
pool.addSubBufferForTesting(fake_handle(1), 200ull * 1024 * 1024);
// 200 MB held + 64 MB floor > 256 MB budget: a grow could not fit.
REQUIRE_FALSE(pool.can_grow());
}
TEST_CASE("shrinkToCapacity releases sub-buffers newest-first after the owner empties them", "[buffer_pool]") {
BufferPool pool;
FakePoolGuard guard{pool};
pool.addSubBufferForTesting(fake_handle(1), 1024);
pool.addSubBufferForTesting(fake_handle(2), 1024);
pool.addSubBufferForTesting(fake_handle(3), 1024);
auto a = pool.alloc(256, 16); // sub 0
auto b = pool.alloc(1024, 16); // sub 1 (sub 0 has only 768 left)
auto c = pool.alloc(512, 16); // sub 0 again (first fit)
auto d = pool.alloc(512, 16); // sub 2
REQUIRE(a.sub_idx == 0);
REQUIRE(b.sub_idx == 1);
REQUIRE(c.sub_idx == 0);
REQUIRE(d.sub_idx == 2);
std::vector<int> evicted;
auto evict = [&](int sub_idx) {
evicted.push_back(sub_idx);
if (sub_idx == 2) pool.free(d);
if (sub_idx == 1) pool.free(b);
if (sub_idx == 0) { pool.free(a); pool.free(c); }
};
// Shrink to 1024: drops sub 2 then sub 1; sub 0 and its slices survive
// with their sub_idx still valid.
const uint64_t released = pool.shrinkToCapacity(1024, evict);
REQUIRE(released == 2048);
REQUIRE(evicted == std::vector<int>{2, 1});
REQUIRE(pool.sub_buffer_count() == 1);
REQUIRE(pool.total_capacity_bytes() == 1024);
REQUIRE(pool.total_used_bytes() == 256 + 512);
REQUIRE(pool.largest_free_run_bytes() == 256);
// Already at or below target: nothing happens, evictor not consulted.
evicted.clear();
REQUIRE(pool.shrinkToCapacity(1024, evict) == 0);
REQUIRE(evicted.empty());
// Shrinking to zero empties the pool entirely.
REQUIRE(pool.shrinkToCapacity(0, evict) == 1024);
REQUIRE(pool.sub_buffer_count() == 0);
REQUIRE(evicted == std::vector<int>{0});
}
TEST_CASE("shrinkToCapacity never undershoots the target", "[buffer_pool]") {
BufferPool pool;
FakePoolGuard guard{pool};
pool.addSubBufferForTesting(fake_handle(1), 256);
pool.addSubBufferForTesting(fake_handle(2), 256);
pool.addSubBufferForTesting(fake_handle(3), 73);
pool.addSubBufferForTesting(fake_handle(4), 73);
auto evict = [](int) {};
// 658 held, target 476: 182 over. The two 73s go (36 still over);
// the 256 would undershoot, so it stays — the margin absorbs 36.
REQUIRE(pool.shrinkToCapacity(476, evict) == 146);
REQUIRE(pool.total_capacity_bytes() == 512);
// An excess smaller than the newest sub-buffer releases nothing.
REQUIRE(pool.shrinkToCapacity(500, evict) == 0);
REQUIRE(pool.total_capacity_bytes() == 512);
}
TEST_CASE("releaseAtLeast frees whole sub-buffers until the requested bytes are gone", "[buffer_pool]") {
BufferPool pool;
FakePoolGuard guard{pool};
pool.addSubBufferForTesting(fake_handle(1), 256);
pool.addSubBufferForTesting(fake_handle(2), 73);
pool.addSubBufferForTesting(fake_handle(3), 73);
std::vector<int> evicted;
auto evict = [&](int sub_idx) { evicted.push_back(sub_idx); };
// Needs 100: 73 is not enough, 73+73 is. Overshoot by a sub-buffer is
// the point — the allocation must fit.
REQUIRE(pool.releaseAtLeast(100, evict) == 146);
REQUIRE(evicted == std::vector<int>{2, 1});
REQUIRE(pool.total_capacity_bytes() == 256);
// More than the pool holds: everything goes, no crash.
REQUIRE(pool.releaseAtLeast(1000, evict) == 256);
REQUIRE(pool.sub_buffer_count() == 0);
REQUIRE(pool.releaseAtLeast(1, evict) == 0);
}
-208
View File
@@ -1,208 +0,0 @@
/********************************************************************************
* *
* This file is part of IfcOpenShell. *
* *
* IfcOpenShell is free software: you can redistribute it and/or modify *
* it under the terms of the Lesser GNU General Public License as published by *
* the Free Software Foundation, either version 3.0 of the License, or *
* (at your option) any later version. *
* *
* IfcOpenShell is distributed in the hope that it will be useful, *
* but WITHOUT ANY WARRANTY; without even the implied warranty of *
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the *
* Lesser GNU General Public License for more details. *
* *
* You should have received a copy of the Lesser GNU General Public License *
* along with this program. If not, see <http://www.gnu.org/licenses/>. *
* *
********************************************************************************/
// GpuBudget decides how much device memory the streamed-geometry cache may
// hold. It is pure policy: a live number derived from what the platform
// can tell us (desktop: the driver's free-memory report; web: nothing but
// a heap ceiling), lowered by pressure events when a required allocation
// fails anyway, and learning from those how much reported-free memory the
// driver will not actually grant. These pin down the arithmetic, the floor
// and the learning.
#include "GpuBudget.h"
#include <catch2/catch_all.hpp>
namespace {
constexpr std::uint64_t MB = 1024ull * 1024;
}
TEST_CASE("nothing known leaves the cache unbounded", "[gpu_budget]") {
GpuBudget b;
REQUIRE_FALSE(b.bounded());
b.update(0, 512 * MB); // query could not answer: still unbounded
REQUIRE_FALSE(b.bounded());
}
TEST_CASE("the first device report bounds the cache at held + free - margin", "[gpu_budget]") {
GpuBudget b;
b.update(2800 * MB, 0);
REQUIRE(b.bounded());
REQUIRE(b.cache_budget_bytes() == 2800 * MB - GpuBudget::kFixedMarginBytes);
// The pool now holds 1000 MB and the driver reports 1800 MB free: the
// cache's own bytes count as available to it, so nothing moves.
b.update(1800 * MB, 1000 * MB);
b.update(1800 * MB, 1000 * MB);
REQUIRE(b.cache_budget_bytes() == 2800 * MB - GpuBudget::kFixedMarginBytes);
}
TEST_CASE("a momentary reading never moves the budget; a sustained one does", "[gpu_budget]") {
const std::uint64_t margin = GpuBudget::kFixedMarginBytes;
GpuBudget b;
b.update(2800 * MB, 0);
const std::uint64_t initial = b.cache_budget_bytes();
// Pool at budget; one poll reads 80 MB free (upload staging in flight).
b.update(80 * MB, initial);
REQUIRE(b.cache_budget_bytes() == initial);
// The staging drained: back in the dead band, streak reset.
b.update(margin, initial);
b.update(80 * MB, initial);
REQUIRE(b.cache_budget_bytes() == initial);
// Tight on two consecutive reports: another process really took it.
b.update(80 * MB, initial);
REQUIRE(b.cache_budget_bytes() == initial + 80 * MB - margin);
const std::uint64_t lowered = b.cache_budget_bytes();
// One roomy report is not enough to raise it...
b.update(1500 * MB, lowered);
REQUIRE(b.cache_budget_bytes() == lowered);
// ...two are.
b.update(1500 * MB, lowered);
REQUIRE(b.cache_budget_bytes() == lowered + 1500 * MB - margin);
}
TEST_CASE("free memory inside the dead band changes nothing however long it lasts", "[gpu_budget]") {
const std::uint64_t margin = GpuBudget::kFixedMarginBytes;
GpuBudget b;
b.update(2800 * MB, 0);
const std::uint64_t initial = b.cache_budget_bytes();
for (int i = 0; i < 10; ++i) b.update(margin, initial); // exactly the margin
for (int i = 0; i < 10; ++i) b.update(margin + margin / 2, initial); // top of the band
for (int i = 0; i < 10; ++i) b.update(margin / 2, initial); // bottom of the band
REQUIRE(b.cache_budget_bytes() == initial);
}
TEST_CASE("a refusal lowers the budget immediately and resets the streaks", "[gpu_budget]") {
GpuBudget b;
b.update(2800 * MB, 0);
const std::uint64_t initial = b.cache_budget_bytes();
b.update(80 * MB, initial); // one tight report
REQUIRE(b.onPressure(initial, 100 * MB, 80 * MB));
REQUIRE(b.cache_budget_bytes() < initial);
const std::uint64_t after = b.cache_budget_bytes();
// The streak did not carry over: one more tight report is not two.
b.update(80 * MB, after);
REQUIRE(b.cache_budget_bytes() == after);
}
TEST_CASE("less than the margin available floors the budget, not zero", "[gpu_budget]") {
GpuBudget b;
b.update(100 * MB, 0);
REQUIRE(b.bounded());
REQUIRE(b.cache_budget_bytes() == GpuBudget::kMinCacheBudgetBytes);
}
TEST_CASE("a hard cap bounds the cache on its own (web) and clamps a device-derived budget", "[gpu_budget]") {
GpuBudget web;
web.setHardCap(3072 * MB);
REQUIRE(web.bounded());
REQUIRE(web.cache_budget_bytes() == 3072 * MB);
GpuBudget both;
both.setHardCap(3072 * MB);
both.update(8000 * MB, 0);
REQUIRE(both.cache_budget_bytes() == 3072 * MB);
GpuBudget small_device;
small_device.setHardCap(3072 * MB);
small_device.update(2800 * MB, 0);
REQUIRE(small_device.cache_budget_bytes() == 2800 * MB - GpuBudget::kFixedMarginBytes);
}
TEST_CASE("pressure lowers the budget below what the cache currently holds", "[gpu_budget]") {
GpuBudget b;
REQUIRE_FALSE(b.bounded());
// The pool grew to 2048 MB unbounded; a 120 MB attachment set then failed.
REQUIRE(b.onPressure(2048 * MB, 120 * MB, 0));
REQUIRE(b.bounded());
REQUIRE(b.cache_budget_bytes()
== 2048 * MB - 120 * MB - GpuBudget::kPressureSlackBytes);
REQUIRE(b.pressure_events() == 1);
}
TEST_CASE("pressure is measured against actual capacity, not the previous budget", "[gpu_budget]") {
// Budget said ~2500 MB but the driver only ever granted 1024 MB; a
// failure must carve out of the 1024, else nothing would be released.
GpuBudget b;
b.update(2800 * MB, 0);
REQUIRE(b.onPressure(1024 * MB, 100 * MB, 0));
REQUIRE(b.cache_budget_bytes()
== 1024 * MB - 100 * MB - GpuBudget::kPressureSlackBytes);
}
TEST_CASE("pressure never raises the budget", "[gpu_budget]") {
GpuBudget b;
b.setHardCap(500 * MB);
REQUIRE(b.onPressure(256 * MB, 0, 0));
REQUIRE(b.cache_budget_bytes() == 256 * MB - GpuBudget::kPressureSlackBytes);
// A later event whose arithmetic lands above the current budget is a no-op.
REQUIRE_FALSE(b.onPressure(4096 * MB, 0, 0));
REQUIRE(b.cache_budget_bytes() == 256 * MB - GpuBudget::kPressureSlackBytes);
}
TEST_CASE("pressure bottoms out at the floor and then reports exhaustion", "[gpu_budget]") {
GpuBudget b;
REQUIRE(b.onPressure(100 * MB, 90 * MB, 0));
REQUIRE(b.cache_budget_bytes() == GpuBudget::kMinCacheBudgetBytes);
// Already at the floor: nothing more to give.
REQUIRE_FALSE(b.onPressure(64 * MB, 90 * MB, 0));
REQUIRE(b.pressure_events() == 2);
}
TEST_CASE("a refusal with memory still reported free teaches the margin", "[gpu_budget]") {
GpuBudget b;
b.update(2800 * MB, 0);
REQUIRE(b.margin_bytes() == GpuBudget::kFixedMarginBytes);
// 59 MB refused with 221 MB "free" (the measured crash): at least
// 162 MB of what the driver reports is not usable.
REQUIRE(b.onPressure(2048 * MB, 59 * MB, 221 * MB));
REQUIRE(b.margin_bytes()
== GpuBudget::kFixedMarginBytes + 162 * MB + GpuBudget::kPressureSlackBytes);
// The next live reports stop short by the learned amount, so the pool
// does not grow straight back into the same refusal.
b.update(221 * MB, 2048 * MB);
b.update(221 * MB, 2048 * MB);
REQUIRE(b.cache_budget_bytes() == 2048 * MB + 221 * MB - b.margin_bytes());
// Learning only ever grows; a later refusal with less phantom free
// memory does not shrink it.
b.onPressure(1500 * MB, 59 * MB, 100 * MB);
REQUIRE(b.margin_bytes()
== GpuBudget::kFixedMarginBytes + 162 * MB + GpuBudget::kPressureSlackBytes);
// A refusal that needed more than was reported free teaches nothing.
b.onPressure(1500 * MB, 500 * MB, 100 * MB);
REQUIRE(b.margin_bytes()
== GpuBudget::kFixedMarginBytes + 162 * MB + GpuBudget::kPressureSlackBytes);
}
TEST_CASE("resident geometry is only shrunk once over budget by the hysteresis", "[gpu_budget]") {
GpuBudget b;
REQUIRE(b.shrinkTarget(4096 * MB) == 0); // unbounded: never
b.update(2800 * MB, 0);
const std::uint64_t budget = b.cache_budget_bytes();
REQUIRE(b.shrinkTarget(budget) == 0);
REQUIRE(b.shrinkTarget(budget + GpuBudget::kShrinkHysteresisBytes - 1) == 0);
REQUIRE(b.shrinkTarget(budget + GpuBudget::kShrinkHysteresisBytes) == budget);
}
-6
View File
@@ -962,12 +962,6 @@ private:
};
%include "../ifcparse/ifc_parse_api.h"
namespace ifcopenshell {
std::string encode_spf_string(const std::string& value);
std::string decode_spf_string(const std::string& value);
}
%include "../ifcparse/spf_header.h"
%pythoncode %{
+1 -1
View File
@@ -5,6 +5,6 @@ jsonpickle==3.0.1
passlib==1.7.4
pydantic==1.10.13
python_dateutil==2.8.2
python_jose==3.4.0
python_jose==3.3.0
py2neo==2021.2.4
+1 -1
View File
@@ -11,7 +11,7 @@ target_include_directories(plugin PUBLIC
)
if (NOT CREATE_BUNDLE)
set_target_properties(plugin PROPERTIES
set_target_properties(plugin PROPERTIES
VERSION "${PROJECT_VERSION}"
SOVERSION "${PROJECT_VERSION_MAJOR}.${PROJECT_VERSION_MINOR}"
)
+1 -1
View File
@@ -96,7 +96,7 @@ public:
std::vector<std::filesystem::path> discover(const std::string& basename_prefix) const;
std::vector<std::filesystem::path> discover_exact(const std::string& basename) const;
ifcopenshell::plugin::module load(const std::filesystem::path& path) const;
module load(const std::filesystem::path& path) const;
private:
std::vector<std::filesystem::path> search_paths_;
-1
View File
@@ -210,7 +210,6 @@ def build() -> None:
"-DGLTF_SUPPORT=ON",
"-DBUILD_EXAMPLES=OFF",
"-DBUILD_BONSAIVIEWER=ON",
"-DUSE_CCACHE=ON",
]
)
restore_env(*OLD_ADD_COMMIT_SHA)
+2 -13
View File
@@ -42,7 +42,6 @@ echo.
setlocal EnableDelayedExpansion
set SCRIPT_DIR=%~dp0
for %%I in ("%SCRIPT_DIR%..") do set "REPO_ROOT=%%~fI"
:: Make sure vcvarsall.bat is called and dev env set is up.
IF "%VSINSTALLDIR%"=="" (
@@ -124,7 +123,7 @@ if "%CMAKE_VERSION%" LSS "cmake version 3.11.4" (
)
:: NOTE Boost < 1.64 doesn't work without tricks if the user has only VS 2017 installed and no earlier versions.
set BOOST_VERSION=1.92.0
set BOOST_VERSION=1.86.0
:: Version string with underscores instead of dots.
set BOOST_VER=%BOOST_VERSION:.=_%
@@ -454,7 +453,6 @@ set DEPENDENCY_INSTALL_NAME=OpenCOLLADA
set NEXT_DEPENDENCY_LABEL=OCCT
:: Always clone it, even if it's installed, because it contains xml headers we need.
:: Use a fixed revision in order to prevent introducing breaking changes
:: TODO: commit is almost 3 years behind the latest version used in nix/build-all.py, need to test and bump.
call :GitCloneAndCheckoutRevision https://github.com/KhronosGroup/OpenCOLLADA.git "%DEPENDENCY_DIR%" 064a60b65c2c31b94f013820856bc84fb1937cc6
call :CheckInstallation
@@ -466,10 +464,6 @@ cd "%DEPENDENCY_DIR%"
:: so disable it from the build altogether as we have no use for it
findstr #add_subdirectory(COLLADAValidator) CMakeLists.txt>NUL
IF NOT %ERRORLEVEL%==0 git apply --reject --whitespace=fix "%~dp0patches\OpenCOLLADA_CMakeLists.txt.patch" --ignore-whitespace
:: std::tr1::unordered_map was a legacy MSVC compatibility shim kept around through VS2022's STL, but newer
:: toolsets (e.g. VS2026/v145) no longer provide it, breaking the build with error C2039: 'tr1' is not a member of 'std'.
findstr /C:"typedef std::unordered_map<MarkId, FilePosType > MarkIdToFilePos;" common\libBuffer\include\CommonFWriteBufferFlusher.h>NUL
IF NOT %ERRORLEVEL%==0 git apply --reject --whitespace=fix "%REPO_ROOT%\nix\patches\opencollada\remove_tr1.patch" --ignore-whitespace
:: NOTE OpenCOLLADA has been observed to have problems with switching between debug and release builds so
:: uncomment to following line in order to delete the CMakeCache.txt always if experiencing problems.
REM IF EXIST "%DEPENDENCY_DIR%\%BUILD_DIR%\CMakeCache.txt". del "%DEPENDENCY_DIR%\%BUILD_DIR%\CMakeCache.txt"
@@ -753,12 +747,7 @@ set QT6_MSVC_YEAR=%VS_VER%
IF /I "%VS_TOOLSET%"=="v141" set QT6_MSVC_YEAR=2017
IF /I "%VS_TOOLSET%"=="v142" set QT6_MSVC_YEAR=2019
IF /I "%VS_TOOLSET%"=="v143" set QT6_MSVC_YEAR=2022
:: Qt has not published prebuilt msvc2026 binaries yet (aqt only lists win64_msvc2022_64 as of
:: Qt 6.7-6.10). The v14x MSVC toolsets share a stable ABI/CRT, so fall back to the msvc2022
:: binaries until Qt ships msvc2026 ones. Revisit once `aqt list-qt windows desktop --arch <ver>`
:: shows a msvc2026 entry.
IF /I "%VS_TOOLSET%"=="v145" set QT6_MSVC_YEAR=2022
IF "%VS_VER%"=="2026" set QT6_MSVC_YEAR=2022
IF /I "%VS_TOOLSET%"=="v145" set QT6_MSVC_YEAR=2026
set QT6_ARCH=
set QT6_INSTALL_SUFFIX=