mirror of
https://github.com/IfcOpenShell/IfcOpenShell.git
synced 2026-08-28 07:49:59 +00:00
ci: build OCCT shared on Linux so plug-ins share one OCCT instance
`geom.tree().select()` silently returns zero results (or raises SWIG's "An unknown error occurred") in the Linux release packages, while the same commit built from source returns correct answers. `select_box()` agrees between both, and geometry conversion is bit-identical -- only operations that touch a stored TopoDS_Shape diverge. Cause is linkage, not code.8bdaa8c7cnarrowed the Rocky builds from `--shared` to `--ifcopenshell-shared`, which shares IfcOpenShell's own libraries but leaves every dependency static. OCCT is then compiled privately into each plug-in that uses it -- 15 of them, verified by their own copies of the BRepClass3d/BRepExtrema/Standard_Failure strings. That contradicts what the binaries already declare. tree.h:1748 casts a `conversion_result_shape*` to `open_cascade_shape*`, moves the TopoDS_Shape out of it and frees it; `open_cascade_shape` is defined once in ifcopenshell_geometry_kernel_opencascade.so and left undefined in ifcopenshell_geometry_tree_opencascade_brep.so for the loader to resolve. So the two plug-ins are designed to share one OCCT-based type system, yet static linking gives each its own Standard_Type registry and allocator. Shapes get read and released by a different OCCT instance than made them. Add `--opencascade-shared`, mirroring the existing `--ifcopenshell-shared` precedent, and use it on both Rocky workflows. It cannot be spelled `--occt-shared`: build-all.py parses any `occt-*` flag as a version override. BUILD_STATIC drives three things at once -- dependency link type, -fvisibility=hidden, and BUILD_SHARED_LIBS -- so making one dependency shared means overriding all three for it. Visibility is the subtle one: OCCT's Standard_EXPORT expands to nothing on Unix, so it relies on default visibility to export its API. Built shared under -fvisibility=hidden it exports almost nothing and its own libraries cannot resolve against each other (libTKMath.so fails to find NCollection_BaseAllocator::CommonBaseAllocator in libTKernel.so). Static archives are immune, which is why this surfaces only once OCCT goes shared. Compile OCCT with the pre-visibility flag set instead. Link the OCCT set with --as-needed. FindOpenCASCADE.cmake's config branch uses OCCT's *complete* module list, Visualization included, which against a static OCCT costs nothing -- an unreferenced module contributes no objects. Against a shared OCCT all 47 become hard DT_NEEDED entries, and libTKV3d pulls libGL.so.1 + libEGL.so.1, so `import ifcopenshell` fails on any headless machine with "libEGL.so.1: cannot open shared object file" even though nothing ever opens a window. Measured through the real find_package path with the LINK_GROUP workaround composed: 67 DT_NEEDED without the flag, 3 with it, TKV3d and TKOpenGl gone. The flag is deliberately left open rather than closed with -Wl,--no-as-needed. CMake emits the imported targets' INTERFACE_LINK_LIBRARIES -- where OCCT lists libGL/libEGL -- after that item, so closing the bracket switches the flag off immediately before the libraries it exists to exclude. Verified against the shipped artifact: closed, the kernel plug-in fell from 47 DT_NEEDED libTK entries to 14 and lost TKV3d, yet still carried a direct libEGL.so.1 and still failed to import on a headless server; open, the same 14 remain and libGL/libEGL are gone. None of the 14 retained modules depends on GL. Put the shared OCCT on LD_LIBRARY_PATH for the build itself. Nothing else points at it -- IfcOpenShell's libraries get INSTALL_RPATH=$ORIGIN and OCCT sits in its own dependency prefix -- so the post-build `import ifcopenshell` check fails the same way. This is build-time only; the shipped packages get libTK*.so* staged beside the payload with an $ORIGIN RUNPATH instead. Suffix OCCT's install directory with `-shared` when it applies. Static and shared installs are not interchangeable, but `build_dependency` skips any dependency whose install dir already exists and cache_dependencies.py keys its tarballs purely on that directory name -- so the static `cache-occt-7.8.1.tar.gz` restored from the build-outputs repo silently satisfied the build and BUILD_LIBRARY_TYPE was never applied. This is the same cache stickiness8bdaa8c7cdescribed, pointing the other way. The suffix makes the key configuration-aware, so it self-invalidates and the static tarball stays valid for builds that still want static. Packaging is the other half, and is why8bdaa8c7cbacked the flag out -- `stage_runtime_payload` only copies from install/ifcopenshell, so OCCT in install/occt-* was never staged and `--shared` "worked by accident" off cached static outputs. Stage libTK*.so* alongside, then give every staged library an $ORIGIN RUNPATH: the core libs currently carry dead build-machine RPATHs and the plug-ins carry none, resolving only because the Python wrapper pulls them in by SONAME first. Shared OCCT has no such first loader, since it is reached through the dlopen'd plug-ins. The packages shrink: the python zip goes from 109.4 MB to 85.6 MB, because the duplicated OCCT was 113 MB of the 287 MB unpacked payload (the eight geometry_writer_ifc* plug-ins alone were 4.6 MB each) against ~67 MB for one shared copy. Same argument asa91b1da28("Reduce Rocky package size") and402591e71. macOS and Windows are affected too but are not fixed here. Their packaging resolves via @loader_path install names and would need install_name_tool rewriting, which cannot be verified from Linux; adding the flag without that would ship a package that fails to load. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -80,7 +80,7 @@ jobs:
|
||||
set -o pipefail
|
||||
CXXFLAGS="-O3" CFLAGS="-O3" ADD_COMMIT_SHA=1 BUILD_CFG=Release BUILD_BONSAIVIEWER=ON \
|
||||
uv run --with aqtinstall ./nix/build-all.py \
|
||||
-v --diskcleanup --ifcopenshell-shared 2>&1 \
|
||||
-v --diskcleanup --ifcopenshell-shared --opencascade-shared 2>&1 \
|
||||
| tee build.log
|
||||
|
||||
- name: Upload Build Logs
|
||||
@@ -147,6 +147,22 @@ jobs:
|
||||
done
|
||||
}
|
||||
|
||||
# Copy the shared OCCT from the dependency prefix to the provided `$1`.
|
||||
# OCCT is built shared (`--opencascade-shared`) so that the opencascade kernel
|
||||
# and tree plug-ins share a single OCCT instance: `open_cascade_shape` objects
|
||||
# are created by the kernel plug-in and then have their `TopoDS_Shape` moved out
|
||||
# and freed by a tree plug-in. A private static OCCT per plug-in gives each its
|
||||
# own Standard_Type registry and allocator, which silently corrupts those shapes.
|
||||
# These libs live under `install/occt-*` rather than `install/ifcopenshell`,
|
||||
# so `stage_runtime_payload` does not pick them up on its own.
|
||||
stage_occt_runtime_payload() {
|
||||
dest="$1"
|
||||
for occt_lib_dir in "$(dirname "$install_root")"/occt-*/lib "$(dirname "$install_root")"/occt-*/lib64; do
|
||||
[ -d "$occt_lib_dir" ] || continue
|
||||
find "$occt_lib_dir" -maxdepth 1 \( -type f -o -type l \) -name "libTK*.so*" -exec cp -P {} "$dest/" \;
|
||||
done
|
||||
}
|
||||
|
||||
# Copy all libs from `install/ifcopenshell` to the provided `$1`.
|
||||
# Set `$2` to `0` to skip including geometry writers.
|
||||
stage_runtime_payload() {
|
||||
@@ -163,7 +179,14 @@ jobs:
|
||||
find "$runtime_dir" \( -type f -o -type l \) \( -name "*.so" -o -name "*.so.*" -o -name "*.dylib" -o -name "*.dll" \)
|
||||
done
|
||||
)
|
||||
stage_occt_runtime_payload "$dest"
|
||||
ensure_soname_links "$dest"
|
||||
# The core libs ship with dead build-machine RPATHs and the plug-ins have
|
||||
# none; today they resolve only because the Python wrapper ($ORIGIN) pulls
|
||||
# them in by SONAME before any plug-in is dlopen'd. Shared OCCT has no such
|
||||
# first loader -- it is reached through the plug-ins -- so give every staged
|
||||
# library an $ORIGIN of its own.
|
||||
find "$dest" -maxdepth 1 -type f -name "*.so*" -exec patchelf --set-rpath '$ORIGIN' {} \;
|
||||
}
|
||||
|
||||
# Copy all libs from `QT_DIR` to the provided `$2`.
|
||||
|
||||
@@ -92,7 +92,7 @@ jobs:
|
||||
set -o pipefail
|
||||
CXXFLAGS="-O3" CFLAGS="-O3" ADD_COMMIT_SHA=1 BUILD_CFG=Release BUILD_BONSAIVIEWER=ON \
|
||||
uv run --with aqtinstall ./nix/build-all.py \
|
||||
-v --diskcleanup --ifcopenshell-shared 2>&1 \
|
||||
-v --diskcleanup --ifcopenshell-shared --opencascade-shared 2>&1 \
|
||||
| tee build.log
|
||||
|
||||
- name: Upload Build Logs
|
||||
@@ -156,6 +156,22 @@ jobs:
|
||||
done
|
||||
}
|
||||
|
||||
# Copy the shared OCCT from the dependency prefix to the provided `$1`.
|
||||
# OCCT is built shared (`--opencascade-shared`) so that the opencascade kernel
|
||||
# and tree plug-ins share a single OCCT instance: `open_cascade_shape` objects
|
||||
# are created by the kernel plug-in and then have their `TopoDS_Shape` moved out
|
||||
# and freed by a tree plug-in. A private static OCCT per plug-in gives each its
|
||||
# own Standard_Type registry and allocator, which silently corrupts those shapes.
|
||||
# These libs live under `install/occt-*` rather than `install/ifcopenshell`,
|
||||
# so `stage_runtime_payload` does not pick them up on its own.
|
||||
stage_occt_runtime_payload() {
|
||||
dest="$1"
|
||||
for occt_lib_dir in "$(dirname "$install_root")"/occt-*/lib "$(dirname "$install_root")"/occt-*/lib64; do
|
||||
[ -d "$occt_lib_dir" ] || continue
|
||||
find "$occt_lib_dir" -maxdepth 1 \( -type f -o -type l \) -name "libTK*.so*" -exec cp -P {} "$dest/" \;
|
||||
done
|
||||
}
|
||||
|
||||
stage_runtime_payload() {
|
||||
dest="$1"
|
||||
include_geometry_writers="${2:-1}"
|
||||
@@ -170,7 +186,14 @@ jobs:
|
||||
find "$runtime_dir" \( -type f -o -type l \) \( -name "*.so" -o -name "*.so.*" -o -name "*.dylib" -o -name "*.dll" \)
|
||||
done
|
||||
)
|
||||
stage_occt_runtime_payload "$dest"
|
||||
ensure_soname_links "$dest"
|
||||
# The core libs ship with dead build-machine RPATHs and the plug-ins have
|
||||
# none; today they resolve only because the Python wrapper ($ORIGIN) pulls
|
||||
# them in by SONAME before any plug-in is dlopen'd. Shared OCCT has no such
|
||||
# first loader -- it is reached through the plug-ins -- so give every staged
|
||||
# library an $ORIGIN of its own.
|
||||
find "$dest" -maxdepth 1 -type f -name "*.so*" -exec patchelf --set-rpath '$ORIGIN' {} \;
|
||||
}
|
||||
|
||||
stage_qt_runtime_payload() {
|
||||
|
||||
@@ -54,6 +54,41 @@ if(NOT OCC_INCLUDE_DIR AND NOT OCC_LIBRARY_DIR)
|
||||
set(OpenCASCADE_LIBRARIES "$<LINK_GROUP:RESCAN,${OpenCASCADE_LIBRARIES}>")
|
||||
endif()
|
||||
|
||||
if(UNIX AND NOT APPLE)
|
||||
# Record only the OCCT modules whose symbols are actually referenced.
|
||||
#
|
||||
# OpenCASCADE_LIBRARIES is OCCT's *complete* module list, Visualization
|
||||
# included -- TKV3d, TKOpenGl, TKService. Against a static OCCT that
|
||||
# costs nothing, because an unreferenced module contributes no objects.
|
||||
# Against a shared OCCT every entry becomes a hard DT_NEEDED, and
|
||||
# libTKV3d pulls libGL.so.1 + libEGL.so.1, so `import ifcopenshell`
|
||||
# dies on any headless machine with
|
||||
# ImportError: libEGL.so.1: cannot open shared object file
|
||||
# even though IfcOpenShell never opens a window. Measured on a full
|
||||
# 67-module link: 67 DT_NEEDED entries without the flag, 3 with it,
|
||||
# and TKV3d/TKOpenGl gone in the second case.
|
||||
#
|
||||
# Goes before the LINK_GROUP above rather than inside it, and is a
|
||||
# no-op for a static OCCT (--as-needed only governs shared libraries),
|
||||
# so this is safe for both link types.
|
||||
#
|
||||
# Deliberately NOT closed with -Wl,--no-as-needed. CMake emits the
|
||||
# imported targets' INTERFACE_LINK_LIBRARIES -- which is where OCCT
|
||||
# lists libGL/libEGL -- *after* this item, so a closing bracket
|
||||
# switches the flag back off just before the libraries it was added to
|
||||
# exclude. Measured: with the bracket closed the plug-ins dropped from
|
||||
# 47 DT_NEEDED libTK entries to 14 and lost TKV3d, yet still carried a
|
||||
# direct libEGL.so.1 and failed to import on a headless server; with it
|
||||
# left open the same 14 remain and libGL/libEGL are gone.
|
||||
#
|
||||
# The cost is that --as-needed stays in effect for whatever follows on
|
||||
# the link line. That is the default on Debian/Ubuntu so it is
|
||||
# well-trodden, but it does mean a library needed only for
|
||||
# static-initialiser side effects could be dropped -- the thing to
|
||||
# suspect first if a serialiser stops registering itself.
|
||||
set(OpenCASCADE_LIBRARIES "-Wl,--as-needed" "${OpenCASCADE_LIBRARIES}")
|
||||
endif()
|
||||
|
||||
if(OpenCASCADE_VERSION VERSION_LESS "7.9.0" AND WIN32)
|
||||
# Bug in OCCT cmake configs < 7.9.0 - missing linked library.
|
||||
list(APPEND OpenCASCADE_LIBRARIES WSOCK32.lib)
|
||||
|
||||
+61
-4
@@ -42,6 +42,13 @@ Available arguments:
|
||||
``-shared`` - build shared libraries. By default will build static.
|
||||
``-ifcopenshell-shared`` - build only IfcOpenShell's own libraries as shared
|
||||
(dependencies stay static). Redundant if ``-shared`` is also passed.
|
||||
``-opencascade-shared`` - build OCCT as shared libraries (other dependencies stay
|
||||
static). Redundant if ``-shared`` is also passed. Required whenever more than one
|
||||
plug-in uses OCCT: `open_cascade_shape` instances are created by the opencascade
|
||||
kernel plug-in and consumed by the opencascade tree plug-ins, which move a
|
||||
`TopoDS_Shape` out of them and free them. A private static OCCT per plug-in gives
|
||||
each one its own `Standard_Type` registry and allocator, so those shapes are read
|
||||
and released by a different OCCT instance than the one that made them.
|
||||
``-diskcleanup`` - clean up build directories after finishing building dependencies
|
||||
``-build-examples`` - build IfcOpenShell examples
|
||||
``-lto`` - enable link-time optimization (adds ``-flto`` to compiler flags)
|
||||
@@ -410,6 +417,8 @@ BUILD_STATIC = "shared" not in flags
|
||||
"""Whether dependencies are built static."""
|
||||
IFCOPENSHELL_STATIC = BUILD_STATIC and "ifcopenshell-shared" not in flags
|
||||
"""Whether IfcOpenShell's own libraries are built static."""
|
||||
OCCT_STATIC = BUILD_STATIC and "opencascade-shared" not in flags
|
||||
"""Whether OCCT is built static. See ``-opencascade-shared``."""
|
||||
ENABLE_FLAG = "--enable-static" if BUILD_STATIC else "--enable-shared"
|
||||
DISABLE_FLAG = "--disable-shared" if BUILD_STATIC else "--disable-static"
|
||||
LINK_TYPE = "static" if BUILD_STATIC else "shared"
|
||||
@@ -423,6 +432,32 @@ if any(f.startswith("py-") for f in flags):
|
||||
if any(f.startswith("occt-") for f in flags):
|
||||
OCCT_VERSION = next(f.split("-", 1)[1] for f in flags if f.startswith("occt-"))
|
||||
|
||||
# Static and shared OCCT installs are not interchangeable, so they must not share
|
||||
# a directory: `build_dependency` skips a dependency whose install dir already
|
||||
# exists, and cache_dependencies.py keys its tarballs purely on that directory
|
||||
# name. Without the suffix a cached static OCCT silently satisfies a shared build
|
||||
# (and vice versa) and the requested link type is never applied.
|
||||
OCCT_DIR_NAME = f"occt-{OCCT_VERSION}" + ("" if OCCT_STATIC else "-shared")
|
||||
|
||||
if not OCCT_STATIC:
|
||||
# A shared OCCT has to be resolvable at run time by everything this script
|
||||
# executes out of the install tree -- most visibly the post-build
|
||||
# `import ifcopenshell` sanity check, which otherwise dies with
|
||||
# "libTKernel.so.7.8: cannot open shared object file". Nothing points there:
|
||||
# IfcOpenShell's libraries get INSTALL_RPATH=$ORIGIN (see SET_INSTALL_SELF_RPATH
|
||||
# in cmake/utilities.cmake) and OCCT lives in its own dependency prefix.
|
||||
#
|
||||
# This is a build-time concern only. The shipped packages do not rely on it:
|
||||
# the workflows stage libTK*.so* next to the payload and patchelf an $ORIGIN
|
||||
# RUNPATH onto every staged library.
|
||||
_occt_lib_dirs = [
|
||||
os.path.join(DEPS_DIR, "install", OCCT_DIR_NAME, libdir)
|
||||
for libdir in ("lib", "lib64")
|
||||
]
|
||||
os.environ["LD_LIBRARY_PATH"] = os.pathsep.join(
|
||||
[*_occt_lib_dirs, os.environ.get("LD_LIBRARY_PATH", "")]
|
||||
).rstrip(os.pathsep)
|
||||
|
||||
if explicit_targets:
|
||||
targets = {dep for target in explicit_targets for dep in gather_dependencies(target)}
|
||||
else:
|
||||
@@ -1045,12 +1080,34 @@ if USE_OCCT and "occ" in targets:
|
||||
if WASM:
|
||||
patches.append("./patches/occt/no_em_js.patch")
|
||||
|
||||
if not OCCT_STATIC:
|
||||
# BUILD_STATIC drives three things at once: the dependency link type,
|
||||
# -fvisibility=hidden, and BUILD_SHARED_LIBS. Building only OCCT shared
|
||||
# means overriding all three for it, not just the link type.
|
||||
#
|
||||
# Visibility matters most. OCCT's Standard_EXPORT expands to nothing on
|
||||
# Unix (Standard_Macro.hxx), so it relies on default visibility to export
|
||||
# its API. Built shared under -fvisibility=hidden it exports almost
|
||||
# nothing and its libraries fail to resolve against each other -- e.g.
|
||||
# libTKMath.so cannot find NCollection_BaseAllocator::CommonBaseAllocator
|
||||
# in libTKernel.so. Static archives are immune, which is why this only
|
||||
# appears once OCCT goes shared. CXXFLAGS_MINIMAL is the pre-visibility
|
||||
# flag set, so this restores default visibility without dropping -O3/-fPIC.
|
||||
#
|
||||
# These come after the generic flags in the cmake command line, and the
|
||||
# last -D for a given variable wins.
|
||||
occt_args.append(f"-DCMAKE_CXX_FLAGS={CXXFLAGS_MINIMAL}")
|
||||
occt_args.append(f"-DCMAKE_C_FLAGS={CFLAGS_MINIMAL}")
|
||||
# Suppresses the generic -DBUILD_SHARED_LIBS=OFF that BUILD_STATIC would
|
||||
# otherwise add, which contradicts BUILD_LIBRARY_TYPE=Shared.
|
||||
occt_args.append("-DBUILD_SHARED_LIBS=ON")
|
||||
|
||||
build_dependency(
|
||||
name=f"occt-{OCCT_VERSION}",
|
||||
name=OCCT_DIR_NAME,
|
||||
mode="cmake",
|
||||
build_tool_args=[
|
||||
f"-DINSTALL_DIR={DEPS_DIR}/install/occt-{OCCT_VERSION}",
|
||||
f"-DBUILD_LIBRARY_TYPE={LINK_TYPE_UCFIRST}",
|
||||
f"-DINSTALL_DIR={DEPS_DIR}/install/{OCCT_DIR_NAME}",
|
||||
f"-DBUILD_LIBRARY_TYPE={'Static' if OCCT_STATIC else 'Shared'}",
|
||||
f"-DBUILD_MODULE_Draw=0",
|
||||
f"-DBUILD_RELEASE_DISABLE_EXCEPTIONS=Off",
|
||||
# Disable xlib explicitly, as it tries to use it on Desktop Ubuntu, adding unnecessary dependency.
|
||||
@@ -1487,7 +1544,7 @@ if "cgal" in targets:
|
||||
cmake_args.append(f"-DCGAL_WITH_GMPXX=Off")
|
||||
|
||||
if "occ" in targets and USE_OCCT:
|
||||
cmake_args_prefix_path.append(f"{DEPS_DIR}/install/occt-{OCCT_VERSION}")
|
||||
cmake_args_prefix_path.append(f"{DEPS_DIR}/install/{OCCT_DIR_NAME}")
|
||||
|
||||
elif "occ" in targets:
|
||||
# We don't support find_package for OCE.
|
||||
|
||||
Reference in New Issue
Block a user