From 35f261b3a6d3b914a15afaad559fa09004e53927 Mon Sep 17 00:00:00 2001 From: Dion Moult Date: Fri, 21 Aug 2026 15:50:21 +1000 Subject: [PATCH] ci: build OCCT shared on Linux so plug-ins share one OCCT instance `geom.tree().select()` silently returns zero results (or raises SWIG's "An unknown error occurred") in the Linux release packages, while the same commit built from source returns correct answers. `select_box()` agrees between both, and geometry conversion is bit-identical -- only operations that touch a stored TopoDS_Shape diverge. Cause is linkage, not code. 8bdaa8c7c narrowed the Rocky builds from `--shared` to `--ifcopenshell-shared`, which shares IfcOpenShell's own libraries but leaves every dependency static. OCCT is then compiled privately into each plug-in that uses it -- 15 of them, verified by their own copies of the BRepClass3d/BRepExtrema/Standard_Failure strings. That contradicts what the binaries already declare. tree.h:1748 casts a `conversion_result_shape*` to `open_cascade_shape*`, moves the TopoDS_Shape out of it and frees it; `open_cascade_shape` is defined once in ifcopenshell_geometry_kernel_opencascade.so and left undefined in ifcopenshell_geometry_tree_opencascade_brep.so for the loader to resolve. So the two plug-ins are designed to share one OCCT-based type system, yet static linking gives each its own Standard_Type registry and allocator. Shapes get read and released by a different OCCT instance than made them. Add `--opencascade-shared`, mirroring the existing `--ifcopenshell-shared` precedent, and use it on both Rocky workflows. It cannot be spelled `--occt-shared`: build-all.py parses any `occt-*` flag as a version override. BUILD_STATIC drives three things at once -- dependency link type, -fvisibility=hidden, and BUILD_SHARED_LIBS -- so making one dependency shared means overriding all three for it. Visibility is the subtle one: OCCT's Standard_EXPORT expands to nothing on Unix, so it relies on default visibility to export its API. Built shared under -fvisibility=hidden it exports almost nothing and its own libraries cannot resolve against each other (libTKMath.so fails to find NCollection_BaseAllocator::CommonBaseAllocator in libTKernel.so). Static archives are immune, which is why this surfaces only once OCCT goes shared. Compile OCCT with the pre-visibility flag set instead. Put the shared OCCT on LD_LIBRARY_PATH for the build itself. Nothing else points at it -- IfcOpenShell's libraries get INSTALL_RPATH=$ORIGIN and OCCT sits in its own dependency prefix -- so the post-build `import ifcopenshell` check fails with "libTKernel.so.7.8: cannot open shared object file". This is build-time only; the shipped packages get libTK*.so* staged beside the payload with an $ORIGIN RUNPATH instead. Suffix OCCT's install directory with `-shared` when it applies. Static and shared installs are not interchangeable, but `build_dependency` skips any dependency whose install dir already exists and cache_dependencies.py keys its tarballs purely on that directory name -- so the static `cache-occt-7.8.1.tar.gz` restored from the build-outputs repo silently satisfied the build and BUILD_LIBRARY_TYPE was never applied. This is the same cache stickiness 8bdaa8c7c described, pointing the other way. The suffix makes the key configuration-aware, so it self-invalidates and the static tarball stays valid for builds that still want static. Packaging is the other half, and is why 8bdaa8c7c backed the flag out -- `stage_runtime_payload` only copies from install/ifcopenshell, so OCCT in install/occt-* was never staged and `--shared` "worked by accident" off cached static outputs. Stage libTK*.so* alongside, then give every staged library an $ORIGIN RUNPATH: the core libs currently carry dead build-machine RPATHs and the plug-ins carry none, resolving only because the Python wrapper pulls them in by SONAME first. Shared OCCT has no such first loader, since it is reached through the dlopen'd plug-ins. This shrinks the packages. Duplicated OCCT is 113 MB of the 287 MB unpacked python payload (the eight geometry_writer_ifc* plug-ins alone are 4.6 MB each); one shared copy of the 29 linked libTK is around 67 MB, and the plug-ins collapse to source-build sizes -- kernel_opencascade 15.4 MB -> 1.2 MB, tree_opencascade_brep 9.6 MB -> 248 KB. Same argument as a91b1da28 ("Reduce Rocky package size") and 402591e71. macOS and Windows are affected too but are not fixed here. Their packaging resolves via @loader_path install names and would need install_name_tool rewriting, which cannot be verified from Linux; adding the flag without that would ship a package that fails to load. Co-Authored-By: Claude Opus 5 (1M context) --- .github/workflows/build_rocky.yml | 25 ++++++++++- .github/workflows/build_rocky_arm.yml | 25 ++++++++++- nix/build-all.py | 65 +++++++++++++++++++++++++-- 3 files changed, 109 insertions(+), 6 deletions(-) diff --git a/.github/workflows/build_rocky.yml b/.github/workflows/build_rocky.yml index d6c3f50c6d..062d65405c 100644 --- a/.github/workflows/build_rocky.yml +++ b/.github/workflows/build_rocky.yml @@ -80,7 +80,7 @@ jobs: set -o pipefail CXXFLAGS="-O3" CFLAGS="-O3" ADD_COMMIT_SHA=1 BUILD_CFG=Release BUILD_BONSAIVIEWER=ON \ uv run --with aqtinstall ./nix/build-all.py \ - -v --diskcleanup --ifcopenshell-shared 2>&1 \ + -v --diskcleanup --ifcopenshell-shared --opencascade-shared 2>&1 \ | tee build.log - name: Upload Build Logs @@ -147,6 +147,22 @@ jobs: done } + # Copy the shared OCCT from the dependency prefix to the provided `$1`. + # OCCT is built shared (`--opencascade-shared`) so that the opencascade kernel + # and tree plug-ins share a single OCCT instance: `open_cascade_shape` objects + # are created by the kernel plug-in and then have their `TopoDS_Shape` moved out + # and freed by a tree plug-in. A private static OCCT per plug-in gives each its + # own Standard_Type registry and allocator, which silently corrupts those shapes. + # These libs live under `install/occt-*` rather than `install/ifcopenshell`, + # so `stage_runtime_payload` does not pick them up on its own. + stage_occt_runtime_payload() { + dest="$1" + for occt_lib_dir in "$(dirname "$install_root")"/occt-*/lib "$(dirname "$install_root")"/occt-*/lib64; do + [ -d "$occt_lib_dir" ] || continue + find "$occt_lib_dir" -maxdepth 1 \( -type f -o -type l \) -name "libTK*.so*" -exec cp -P {} "$dest/" \; + done + } + # Copy all libs from `install/ifcopenshell` to the provided `$1`. # Set `$2` to `0` to skip including geometry writers. stage_runtime_payload() { @@ -163,7 +179,14 @@ jobs: find "$runtime_dir" \( -type f -o -type l \) \( -name "*.so" -o -name "*.so.*" -o -name "*.dylib" -o -name "*.dll" \) done ) + stage_occt_runtime_payload "$dest" ensure_soname_links "$dest" + # The core libs ship with dead build-machine RPATHs and the plug-ins have + # none; today they resolve only because the Python wrapper ($ORIGIN) pulls + # them in by SONAME before any plug-in is dlopen'd. Shared OCCT has no such + # first loader -- it is reached through the plug-ins -- so give every staged + # library an $ORIGIN of its own. + find "$dest" -maxdepth 1 -type f -name "*.so*" -exec patchelf --set-rpath '$ORIGIN' {} \; } # Copy all libs from `QT_DIR` to the provided `$2`. diff --git a/.github/workflows/build_rocky_arm.yml b/.github/workflows/build_rocky_arm.yml index b805d81436..c3880b6c61 100644 --- a/.github/workflows/build_rocky_arm.yml +++ b/.github/workflows/build_rocky_arm.yml @@ -92,7 +92,7 @@ jobs: set -o pipefail CXXFLAGS="-O3" CFLAGS="-O3" ADD_COMMIT_SHA=1 BUILD_CFG=Release BUILD_BONSAIVIEWER=ON \ uv run --with aqtinstall ./nix/build-all.py \ - -v --diskcleanup --ifcopenshell-shared 2>&1 \ + -v --diskcleanup --ifcopenshell-shared --opencascade-shared 2>&1 \ | tee build.log - name: Upload Build Logs @@ -156,6 +156,22 @@ jobs: done } + # Copy the shared OCCT from the dependency prefix to the provided `$1`. + # OCCT is built shared (`--opencascade-shared`) so that the opencascade kernel + # and tree plug-ins share a single OCCT instance: `open_cascade_shape` objects + # are created by the kernel plug-in and then have their `TopoDS_Shape` moved out + # and freed by a tree plug-in. A private static OCCT per plug-in gives each its + # own Standard_Type registry and allocator, which silently corrupts those shapes. + # These libs live under `install/occt-*` rather than `install/ifcopenshell`, + # so `stage_runtime_payload` does not pick them up on its own. + stage_occt_runtime_payload() { + dest="$1" + for occt_lib_dir in "$(dirname "$install_root")"/occt-*/lib "$(dirname "$install_root")"/occt-*/lib64; do + [ -d "$occt_lib_dir" ] || continue + find "$occt_lib_dir" -maxdepth 1 \( -type f -o -type l \) -name "libTK*.so*" -exec cp -P {} "$dest/" \; + done + } + stage_runtime_payload() { dest="$1" include_geometry_writers="${2:-1}" @@ -170,7 +186,14 @@ jobs: find "$runtime_dir" \( -type f -o -type l \) \( -name "*.so" -o -name "*.so.*" -o -name "*.dylib" -o -name "*.dll" \) done ) + stage_occt_runtime_payload "$dest" ensure_soname_links "$dest" + # The core libs ship with dead build-machine RPATHs and the plug-ins have + # none; today they resolve only because the Python wrapper ($ORIGIN) pulls + # them in by SONAME before any plug-in is dlopen'd. Shared OCCT has no such + # first loader -- it is reached through the plug-ins -- so give every staged + # library an $ORIGIN of its own. + find "$dest" -maxdepth 1 -type f -name "*.so*" -exec patchelf --set-rpath '$ORIGIN' {} \; } stage_qt_runtime_payload() { diff --git a/nix/build-all.py b/nix/build-all.py index 141006d74d..e000f7e308 100644 --- a/nix/build-all.py +++ b/nix/build-all.py @@ -42,6 +42,13 @@ Available arguments: ``-shared`` - build shared libraries. By default will build static. ``-ifcopenshell-shared`` - build only IfcOpenShell's own libraries as shared (dependencies stay static). Redundant if ``-shared`` is also passed. + ``-opencascade-shared`` - build OCCT as shared libraries (other dependencies stay + static). Redundant if ``-shared`` is also passed. Required whenever more than one + plug-in uses OCCT: `open_cascade_shape` instances are created by the opencascade + kernel plug-in and consumed by the opencascade tree plug-ins, which move a + `TopoDS_Shape` out of them and free them. A private static OCCT per plug-in gives + each one its own `Standard_Type` registry and allocator, so those shapes are read + and released by a different OCCT instance than the one that made them. ``-diskcleanup`` - clean up build directories after finishing building dependencies ``-build-examples`` - build IfcOpenShell examples ``-lto`` - enable link-time optimization (adds ``-flto`` to compiler flags) @@ -410,6 +417,8 @@ BUILD_STATIC = "shared" not in flags """Whether dependencies are built static.""" IFCOPENSHELL_STATIC = BUILD_STATIC and "ifcopenshell-shared" not in flags """Whether IfcOpenShell's own libraries are built static.""" +OCCT_STATIC = BUILD_STATIC and "opencascade-shared" not in flags +"""Whether OCCT is built static. See ``-opencascade-shared``.""" ENABLE_FLAG = "--enable-static" if BUILD_STATIC else "--enable-shared" DISABLE_FLAG = "--disable-shared" if BUILD_STATIC else "--disable-static" LINK_TYPE = "static" if BUILD_STATIC else "shared" @@ -423,6 +432,32 @@ if any(f.startswith("py-") for f in flags): if any(f.startswith("occt-") for f in flags): OCCT_VERSION = next(f.split("-", 1)[1] for f in flags if f.startswith("occt-")) +# Static and shared OCCT installs are not interchangeable, so they must not share +# a directory: `build_dependency` skips a dependency whose install dir already +# exists, and cache_dependencies.py keys its tarballs purely on that directory +# name. Without the suffix a cached static OCCT silently satisfies a shared build +# (and vice versa) and the requested link type is never applied. +OCCT_DIR_NAME = f"occt-{OCCT_VERSION}" + ("" if OCCT_STATIC else "-shared") + +if not OCCT_STATIC: + # A shared OCCT has to be resolvable at run time by everything this script + # executes out of the install tree -- most visibly the post-build + # `import ifcopenshell` sanity check, which otherwise dies with + # "libTKernel.so.7.8: cannot open shared object file". Nothing points there: + # IfcOpenShell's libraries get INSTALL_RPATH=$ORIGIN (see SET_INSTALL_SELF_RPATH + # in cmake/utilities.cmake) and OCCT lives in its own dependency prefix. + # + # This is a build-time concern only. The shipped packages do not rely on it: + # the workflows stage libTK*.so* next to the payload and patchelf an $ORIGIN + # RUNPATH onto every staged library. + _occt_lib_dirs = [ + os.path.join(DEPS_DIR, "install", OCCT_DIR_NAME, libdir) + for libdir in ("lib", "lib64") + ] + os.environ["LD_LIBRARY_PATH"] = os.pathsep.join( + [*_occt_lib_dirs, os.environ.get("LD_LIBRARY_PATH", "")] + ).rstrip(os.pathsep) + if explicit_targets: targets = {dep for target in explicit_targets for dep in gather_dependencies(target)} else: @@ -1045,12 +1080,34 @@ if USE_OCCT and "occ" in targets: if WASM: patches.append("./patches/occt/no_em_js.patch") + if not OCCT_STATIC: + # BUILD_STATIC drives three things at once: the dependency link type, + # -fvisibility=hidden, and BUILD_SHARED_LIBS. Building only OCCT shared + # means overriding all three for it, not just the link type. + # + # Visibility matters most. OCCT's Standard_EXPORT expands to nothing on + # Unix (Standard_Macro.hxx), so it relies on default visibility to export + # its API. Built shared under -fvisibility=hidden it exports almost + # nothing and its libraries fail to resolve against each other -- e.g. + # libTKMath.so cannot find NCollection_BaseAllocator::CommonBaseAllocator + # in libTKernel.so. Static archives are immune, which is why this only + # appears once OCCT goes shared. CXXFLAGS_MINIMAL is the pre-visibility + # flag set, so this restores default visibility without dropping -O3/-fPIC. + # + # These come after the generic flags in the cmake command line, and the + # last -D for a given variable wins. + occt_args.append(f"-DCMAKE_CXX_FLAGS={CXXFLAGS_MINIMAL}") + occt_args.append(f"-DCMAKE_C_FLAGS={CFLAGS_MINIMAL}") + # Suppresses the generic -DBUILD_SHARED_LIBS=OFF that BUILD_STATIC would + # otherwise add, which contradicts BUILD_LIBRARY_TYPE=Shared. + occt_args.append("-DBUILD_SHARED_LIBS=ON") + build_dependency( - name=f"occt-{OCCT_VERSION}", + name=OCCT_DIR_NAME, mode="cmake", build_tool_args=[ - f"-DINSTALL_DIR={DEPS_DIR}/install/occt-{OCCT_VERSION}", - f"-DBUILD_LIBRARY_TYPE={LINK_TYPE_UCFIRST}", + f"-DINSTALL_DIR={DEPS_DIR}/install/{OCCT_DIR_NAME}", + f"-DBUILD_LIBRARY_TYPE={'Static' if OCCT_STATIC else 'Shared'}", f"-DBUILD_MODULE_Draw=0", f"-DBUILD_RELEASE_DISABLE_EXCEPTIONS=Off", # Disable xlib explicitly, as it tries to use it on Desktop Ubuntu, adding unnecessary dependency. @@ -1487,7 +1544,7 @@ if "cgal" in targets: cmake_args.append(f"-DCGAL_WITH_GMPXX=Off") if "occ" in targets and USE_OCCT: - cmake_args_prefix_path.append(f"{DEPS_DIR}/install/occt-{OCCT_VERSION}") + cmake_args_prefix_path.append(f"{DEPS_DIR}/install/{OCCT_DIR_NAME}") elif "occ" in targets: # We don't support find_package for OCE.