Glassvision/IfcOpenShell
1
0
Fork 0
mirror of https://github.com/IfcOpenShell/IfcOpenShell.git synced 2026-08-28 07:49:59 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
35f261b3a6d3b914a15afaad559fa09004e53927
IfcOpenShell/nix/build-all.py
T
Dion Moult 35f261b3a6 ci: build OCCT shared on Linux so plug-ins share one OCCT instance 
`geom.tree().select()` silently returns zero results (or raises SWIG's
"An unknown error occurred") in the Linux release packages, while the same
commit built from source returns correct answers. `select_box()` agrees
between both, and geometry conversion is bit-identical -- only operations
that touch a stored TopoDS_Shape diverge.

Cause is linkage, not code. 8bdaa8c7c narrowed the Rocky builds from
`--shared` to `--ifcopenshell-shared`, which shares IfcOpenShell's own
libraries but leaves every dependency static. OCCT is then compiled
privately into each plug-in that uses it -- 15 of them, verified by their
own copies of the BRepClass3d/BRepExtrema/Standard_Failure strings.

That contradicts what the binaries already declare. tree.h:1748 casts a
`conversion_result_shape*` to `open_cascade_shape*`, moves the
TopoDS_Shape out of it and frees it; `open_cascade_shape` is defined once
in ifcopenshell_geometry_kernel_opencascade.so and left undefined in
ifcopenshell_geometry_tree_opencascade_brep.so for the loader to resolve.
So the two plug-ins are designed to share one OCCT-based type system, yet
static linking gives each its own Standard_Type registry and allocator.
Shapes get read and released by a different OCCT instance than made them.

Add `--opencascade-shared`, mirroring the existing `--ifcopenshell-shared`
precedent, and use it on both Rocky workflows. It cannot be spelled
`--occt-shared`: build-all.py parses any `occt-*` flag as a version
override.

BUILD_STATIC drives three things at once -- dependency link type,
-fvisibility=hidden, and BUILD_SHARED_LIBS -- so making one dependency
shared means overriding all three for it. Visibility is the subtle one:
OCCT's Standard_EXPORT expands to nothing on Unix, so it relies on default
visibility to export its API. Built shared under -fvisibility=hidden it
exports almost nothing and its own libraries cannot resolve against each
other (libTKMath.so fails to find
NCollection_BaseAllocator::CommonBaseAllocator in libTKernel.so). Static
archives are immune, which is why this surfaces only once OCCT goes
shared. Compile OCCT with the pre-visibility flag set instead.

Put the shared OCCT on LD_LIBRARY_PATH for the build itself. Nothing else
points at it -- IfcOpenShell's libraries get INSTALL_RPATH=$ORIGIN and OCCT
sits in its own dependency prefix -- so the post-build `import ifcopenshell`
check fails with "libTKernel.so.7.8: cannot open shared object file". This
is build-time only; the shipped packages get libTK*.so* staged beside the
payload with an $ORIGIN RUNPATH instead.

Suffix OCCT's install directory with `-shared` when it applies. Static and
shared installs are not interchangeable, but `build_dependency` skips any
dependency whose install dir already exists and cache_dependencies.py keys
its tarballs purely on that directory name -- so the static
`cache-occt-7.8.1.tar.gz` restored from the build-outputs repo silently
satisfied the build and BUILD_LIBRARY_TYPE was never applied. This is the
same cache stickiness 8bdaa8c7c described, pointing the other way. The
suffix makes the key configuration-aware, so it self-invalidates and the
static tarball stays valid for builds that still want static.

Packaging is the other half, and is why 8bdaa8c7c backed the flag out --
`stage_runtime_payload` only copies from install/ifcopenshell, so OCCT in
install/occt-* was never staged and `--shared` "worked by accident" off
cached static outputs. Stage libTK*.so* alongside, then give every staged
library an $ORIGIN RUNPATH: the core libs currently carry dead
build-machine RPATHs and the plug-ins carry none, resolving only because
the Python wrapper pulls them in by SONAME first. Shared OCCT has no such
first loader, since it is reached through the dlopen'd plug-ins.

This shrinks the packages. Duplicated OCCT is 113 MB of the 287 MB
unpacked python payload (the eight geometry_writer_ifc* plug-ins alone are
4.6 MB each); one shared copy of the 29 linked libTK is around 67 MB, and
the plug-ins collapse to source-build sizes -- kernel_opencascade 15.4 MB
-> 1.2 MB, tree_opencascade_brep 9.6 MB -> 248 KB. Same argument as
a91b1da28 ("Reduce Rocky package size") and 402591e71.

macOS and Windows are affected too but are not fixed here. Their
packaging resolves via @loader_path install names and would need
install_name_tool rewriting, which cannot be verified from Linux; adding
the flag without that would ship a package that fails to load.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-21 22:19:08 +10:00

1751 lines
70 KiB
Python
Raw Blame History

#!/usr/bin/python
# /// script
# dependencies = [
# "typing_extensions",
# ]
# ///
###############################################################################
# #
# This file is part of IfcOpenShell. #
# #
# IfcOpenShell is free software: you can redistribute it and/or modify #
# it under the terms of the Lesser GNU General Public License as published by #
# the Free Software Foundation, either version 3.0 of the License, or #
# (at your option) any later version. #
# #
# IfcOpenShell is distributed in the hope that it will be useful, #
# but WITHOUT ANY WARRANTY; without even the implied warranty of #
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the #
# Lesser GNU General Public License for more details. #
# #
# You should have received a copy of the Lesser GNU General Public License #
# along with this program. If not, see <http://www.gnu.org/licenses/>. #
# #
###############################################################################
"""
Example usage:
# Build all targets by default, except BonsaiViewer (it's set explicitly).
python build-all.py
# Build just the provided targets.
python build-all.py IfcParse IfcOpenShell-Python
Available arguments:
``-py-313`` - build for specific Python version
(building for all supported Python version by default).
``-occt-xxx`` - use a specific OCCT version (e.g. ``-occt-7.8.1``) instead of the default
``-wasm`` - compile for wasm
``-without-xxx`` - do not build dependency ``xxx`` (e.g. ``--without-swig``)
``-mac-cross-compile-intel`` - cross compile for Intel Mac on Apple Silicon host
``-shared`` - build shared libraries. By default will build static.
``-ifcopenshell-shared`` - build only IfcOpenShell's own libraries as shared
(dependencies stay static). Redundant if ``-shared`` is also passed.
``-opencascade-shared`` - build OCCT as shared libraries (other dependencies stay
static). Redundant if ``-shared`` is also passed. Required whenever more than one
plug-in uses OCCT: `open_cascade_shape` instances are created by the opencascade
kernel plug-in and consumed by the opencascade tree plug-ins, which move a
`TopoDS_Shape` out of them and free them. A private static OCCT per plug-in gives
each one its own `Standard_Type` registry and allocator, so those shapes are read
and released by a different OCCT instance than the one that made them.
``-diskcleanup`` - clean up build directories after finishing building dependencies
``-build-examples`` - build IfcOpenShell examples
``-lto`` - enable link-time optimization (adds ``-flto`` to compiler flags)
``-v`` - enable verbose logs
Used environment variables:
Boolean-like env variables accept the following values (case-insensitive):
`opt-in` - `1`, `on`, `true`, `yes`; ``opt-out`` - `0`, `off`, `false`, `no`.
- ``CXXFLAGS``, ``CPPFLAGS``, ``CFLAGS``, ``LDFLAGS``
- ``BUILD_DIR`` - build directory. By default will use "build" folder in IfcOpenShell repository.
- ``DEPS_DIR`` - dependencies directory. By default will create automatic folder in build directory.
- ``BUILD_CFG`` - build configuration, 'RelWithDebInfo' by default.
- ``USE_CURRENT_PYTHON_VERSION`` - use current python config instead of compile from source
`off` by default.
- ``IFCOS_NUM_BUILD_PROCS`` - number of concurrent processes defaults to available cores + 1
- ``NO_CLEAN`` - do not clean `ifcopenshell` build directories but continue working on current build
(installed dependencies are never cleared).
By default option is disabled, to enable pass any value from `1`, `on`, `true`.
- ``IFCOS_SCHEMAS`` - schemas to be built; defaults to cmake default (8 schemas), to be supplied as `2x3;4;4x3_add2`
- ``USE_OCCT`` - whether to use official Open CASCADE instead of Community Edition
`on` by default
- ``WASM_PYTHON_PATH`` - path to WASM Python installation,
used to deduce `PYVERSION` (e.g. '3.13.2'), `PYTHONINCLUDE`,
`SIDE_MODULE_CFLAGS`, `SIDE_MODULE_LDFLAGS`.
Allows to build wasm without pyodide build environment, which can be useful for debugging build issues.
Example value: 'pyodide/cpython/installs/python-3.13.2'
- ``ADD_COMMIT_SHA`` - `off` by default. If enabled
`ADD_COMMIT_SHA` and `VERSION_OVERRIDE` will be set to `ON` while configuring IfcOpenShell
- ``BUILD_BONSAIVIEWER`` - enable building BonsaiViewer, `off` by default.
- ``IFCOS_BUILD_PYTHON_WRAPPER`` - enable building the Python wrapper, `on` by default.
# This script builds IfcOpenShell and its dependencies #
# #
# Prerequisites for this script to function correctly: #
# * cmake * git * bzip2 * tar * c(++) compilers * autoconf #
# #
# if building with USE_OCCT additionally: #
# * glx.h #
# #
# if building with OCCT 7.4.0 additionally: #
# * libfontconfig1-dev #
# #
# if building with -shared #
# * libgl1-mesa-dev libxext-dev libxmu-dev libxmu-headers libxi-dev #
# #
# for python37 to install correctly additionally: #
# * libffi(-dev[el]) #
# for Python build we also needs ssl and zlib #
# (since we do `pip install numpy` at the end) #
# * libssl-dev #
# #
# on debian 7.8 these can be obtained with: #
# $ apt-get install git gcc g++ autoconf bison bzip2 cmake #
# mesa-common-dev libffi-dev libfontconfig1-dev #
# libssl-dev xz zlib1g-dev #
# #
# on ubuntu 14.04: #
# $ apt-get install git gcc g++ autoconf bison make cmake #
# mesa-common-dev libffi-dev libfontconfig1-dev #
# libssl-dev xz-utils zlib1g-dev #
# #
# on OS X El Capitan with homebrew: #
# $ brew install git bison autoconf automake libffi cmake #
# $ # `bison` shipped with Mac is too old for swig build, #
# $ # so we use `brew`. #
# $ export PATH=$(brew --prefix bison)/bin:$PATH #
# #
# on RHEL-related distros: #
# $ dnf install git gcc gcc-c++ autoconf bison make cmake #
# mesa-libGL-devel libffi-devel fontconfig-devel bzip2 #
# automake patch byacc xz zlib-devel openssl-devel #
"""
import glob
import logging
import multiprocessing
import os
import platform
import shutil
import ssl
import subprocess as sp
import sys
import sysconfig
import tarfile
import threading
import time
from collections.abc import Generator, Sequence
from datetime import datetime
from pathlib import Path
from typing import Literal
from urllib.request import urlretrieve
from typing_extensions import assert_never
logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO)
ch = logging.StreamHandler()
logger.addHandler(ch)
def is_on_off(value: str | None, *, default: bool) -> bool:
if value is None:
return default
lowered = value.lower()
if lowered in {"1", "on", "true", "yes"}:
return True
if lowered in {"0", "off", "false", "no"}:
return False
return default
PROJECT_NAME = "IfcOpenShell"
USE_CURRENT_PYTHON_VERSION = is_on_off(os.getenv("USE_CURRENT_PYTHON_VERSION"), default=False)
ADD_COMMIT_SHA = is_on_off(os.getenv("ADD_COMMIT_SHA"), default=False)
IFCOS_BUILD_PYTHON_WRAPPER = is_on_off(os.getenv("IFCOS_BUILD_PYTHON_WRAPPER"), default=True)
BUILD_BONSAIVIEWER = is_on_off(os.getenv("BUILD_BONSAIVIEWER"), default=False)
USE_OCCT = is_on_off(os.getenv("USE_OCCT"), default=True)
PYTHON_VERSIONS = ["3.10.3", "3.11.8", "3.12.1", "3.13.6", "3.14.0"]
JSON_VERSION = "3.11.3"
OCE_VERSION = "0.18.3"
OCCT_VERSION = "7.8.1"
BOOST_VERSION = "1.86.0"
EIGEN_VERSION = "3.4.0"
PCRE_VERSION = "8.41"
LIBXML2_VERSION = "2.13.8"
SWIG_VERSION = "4.2.1"
OPENCOLLADA_VERSION = "v1.6.68"
GMP_VERSION = "6.3.0"
MPFR_VERSION = "3.1.6" # latest is 4.1.0
CGAL_VERSION = "v5.6.3"
USD_VERSION = "23.05"
TBB_VERSION = "2021.9.0"
ROCKSDB_VERSION = "10.4.2"
ZSTD_VERSION = "1.5.7"
MANIFOLD_VERSION = "3.2.1"
QT6_VERSION = os.getenv("QT6_VERSION", "6.8.3")
# binaries
cp = "cp"
bash = "bash"
git = "git"
bunzip2 = "bunzip2"
tar = "tar"
cc = "cc"
cplusplus = "c++"
autoconf = "autoconf"
automake = "automake"
make = "make"
date = "date"
curl = "curl"
wget = "wget"
strip = "strip"
xz = "xz" # Used implicitly for `tar -xf *.tar.xz`.
brew = "brew"
explicit_targets = [s for s in sys.argv[1:] if not s.startswith("-")]
"""Targets provided by CLI."""
flags = set(s.lstrip("-") for s in sys.argv[1:] if s.startswith("-"))
"""CLI flags."""
# Helper function for coloured printing
NO_COLOR = (
"\033[0m" # <ref>http://stackoverflow.com/questions/5947742/how-to-change-the-output-color-of-echo-in-linux</ref>
)
BLACK_ON_WHITE = "\033[0;30;107m"
RED = "\033[31m"
GREEN = "\033[32m"
YELLOW = "\033[33m"
MAGENTA = "\033[35m"
def cecho(message, color=NO_COLOR):
"""Logs message `message` in color `color`."""
logger.info(f"{color}{message}\033[0m")
# Flags.
BUILD_EXAMPLES = "build-examples" in flags
DISK_CLEANUP = "diskcleanup" in flags
LTO = "lto" in flags
VERBOSE = "v" in flags
APPLE = platform.system() == "Darwin"
MAC_CROSS_COMPILE_INTEL = "mac-cross-compile-intel" in flags
assert platform.system() == "Darwin" or not MAC_CROSS_COMPILE_INTEL
WASM = "wasm" in flags
"""Build WASM outside pyodide build environment."""
WASM_CMAKE_IS_USING_INIT_VARS = False
if WASM:
def get_pyodide_config_var(var_name: str) -> str:
output = sp.check_output(["pyodide", "config", "get", var_name], encoding="utf-8").strip()
return output
if "PYODIDE_ROOT" not in os.environ:
cecho("WARNING. Couldn't find 'PYODIDE_ROOT' in environment variables.", YELLOW)
cecho("Assuming building wasm outside pyodide build environment and resetting necessary variables.", YELLOW)
os.environ["SIDE_MODULE_CFLAGS"] = get_pyodide_config_var("cflags")
os.environ["SIDE_MODULE_CXXFLAGS"] = get_pyodide_config_var("cxxflags")
os.environ["SIDE_MODULE_LDFLAGS"] = get_pyodide_config_var("ldflags")
# Override cmake toolchain for all `emcmake` calls,
# needed for shared libraries (resulting .so wrapper)
# and to ensure compilation is pyodide compatible (e.g. `-fwasm-exceptions` is used in compilation flags).
os.environ["CMAKE_TOOLCHAIN_FILE"] = get_pyodide_config_var("cmake_toolchain_file")
required_vars = (
"SIDE_MODULE_CFLAGS",
"SIDE_MODULE_CXXFLAGS",
"SIDE_MODULE_LDFLAGS",
"CMAKE_TOOLCHAIN_FILE",
)
missing_vars = [v for v in required_vars if v not in os.environ]
assert not missing_vars, f"Some variables required for WASM compilation are missing: {', '.join(missing_vars)}"
def get_pyodide_build_version() -> tuple[int, ...]:
pyodide_build_suffix = "pyodide-build version:"
output = sp.check_output(["pyodide", "--version"], encoding="utf-8").strip()
assert pyodide_build_suffix in output, output
version_line = next(l for l in output.splitlines() if l.startswith(pyodide_build_suffix))
version = version_line.partition(":")[2].strip()
return tuple(map(int, version.split(".")))
# Pyodide still in transition from `FLAGS` to `FLAGS_INIT`.
# `FLAGS_INIT` allow us to provide flags using environment variables
# and providing `FLAGS` directly would break pyodide toolchain.
# NOTE: changes in pyodide-build currently got postponed:
# https://github.com/pyodide/pyodide-build/pull/249
WASM_CMAKE_IS_USING_INIT_VARS = get_pyodide_build_version() >= (99, 0, 0)
# 0.31 is required for SIDE_MODULE_CXXFLAGS to be provided.
assert get_pyodide_build_version() >= (0, 31)
TOOLSET = None
if platform.system() == "Darwin":
# C++11 features used in OCCT 7+ need a more recent stdlib
# TOOLSET = "10.9" if USE_OCCT else "10.6"
# /Users/runner/work/IfcOpenShell/IfcOpenShell/build/Darwin/arm64/10.9/build/rocksdb/cache/clock_cache.cc:732:14: error: aligned allocation function of type 'void *(std::size_t, std::align_val_t)' is only available on macOS 10.13 or newer
# /Users/runner/work/IfcOpenShell/IfcOpenShell/src/ifcparse/IfcFile.cpp:539:14: error: 'exists' is unavailable: introduced in macOS 10.15
TOOLSET = "10.15"
IFCOS_NUM_BUILD_PROCS = os.getenv("IFCOS_NUM_BUILD_PROCS", multiprocessing.cpu_count() + 1)
SCRIPT_PATH = Path(__file__).parent
REPO_PATH = SCRIPT_PATH.parent
CMAKE_DIR = (REPO_PATH / "cmake").resolve().__str__()
BUILD_DIR = os.environ.get("BUILD_DIR", (REPO_PATH / "build").__str__())
if WASM:
arch = "wasm"
elif MAC_CROSS_COMPILE_INTEL:
arch = "x86_64"
else:
arch = platform.machine()
DEFAULT_DEPS_DIR = Path(BUILD_DIR) / platform.system() / arch
if TOOLSET:
DEFAULT_DEPS_DIR = DEFAULT_DEPS_DIR / TOOLSET
DEFAULT_DEPS_DIR = os.path.realpath(DEFAULT_DEPS_DIR)
DEPS_DIR = os.getenv("DEPS_DIR", DEFAULT_DEPS_DIR)
if not os.path.exists(DEPS_DIR):
os.makedirs(DEPS_DIR)
INSTALL_DIR = Path(DEPS_DIR) / "install"
BUILD_CFG = os.getenv("BUILD_CFG", "RelWithDebInfo")
# Print build configuration information
cecho(
f"""This script fetches and builds {PROJECT_NAME} and its dependencies
""",
BLACK_ON_WHITE,
)
cecho(
"""Script configuration:
""",
GREEN,
)
cecho(f"""* USE_OCCT = {USE_OCCT}""", MAGENTA)
if USE_OCCT:
cecho(" - Compiling against official Open Cascade")
else:
cecho(" - Compiling against Open Cascade Community Edition")
cecho(f"* Build Directory = {BUILD_DIR}", MAGENTA)
cecho(f"* Dependency Directory = {DEPS_DIR}", MAGENTA)
cecho(f" - The directory where {PROJECT_NAME} dependencies are installed.")
cecho(f"* Build Config Type = {BUILD_CFG}", MAGENTA)
cecho(""" - The used build configuration type for the dependencies.
Defaults to RelWithDebInfo if not specified.""")
if BUILD_CFG == "MinSizeRel":
cecho(" WARNING: MinSizeRel build can suffer from a significant performance loss.", RED)
cecho(f"* IFCOS_NUM_BUILD_PROCS = {IFCOS_NUM_BUILD_PROCS}", MAGENTA)
cecho(""" - How many compiler processes may be run in parallel.""")
cecho(f"* IFCOS_SCHEMAS = '{os.environ.get('IFCOS_SCHEMAS')}'", MAGENTA)
cecho(""" - IFC Schemas to compile. If not provided, fallback to default provided in cmake.
""")
dependency_tree: dict[str, tuple[str, ...]] = {
"IfcParse": ("boost", "libxml2", "rocksdb"),
"IfcGeom": ("IfcParse", "occ", "manifold", "json", "cgal", "eigen", "OpenCOLLADA"),
"IfcConvert": ("IfcGeom",),
"OpenCOLLADA": ("libxml2", "pcre"),
"IfcGeomServer": ("IfcGeom",),
"IfcOpenShell-Python": ("python", "swig", "IfcGeom"),
"BonsaiViewer": ("IfcGeom", "qt6"),
"swig": (),
"boost": (),
"libxml2": (),
"python": (),
"occ": (),
"pcre": (),
"json": (),
"cgal": (),
"eigen": (),
"rocksdb": ("zstd",),
"zstd": (),
"manifold": (),
"qt6": (),
# 'usd': ('boost', 'oneTBB')
}
def gather_dependencies(dep: str) -> Generator[str]:
yield dep
for d in dependency_tree[dep]:
if f"without-{d.lower()}" not in flags:
for x in gather_dependencies(d):
yield x
if VERBOSE:
logger.setLevel(logging.DEBUG)
formatter = logging.Formatter("%(asctime)s - %(levelname)s - %(message)s")
ch.setFormatter(formatter)
else:
logger.setLevel(logging.INFO)
if MAC_CROSS_COMPILE_INTEL:
MAC_CROSS_COMPILE_INTEL_ARGS = ["-DCMAKE_OSX_ARCHITECTURES=x86_64"]
MAC_CROSS_COMPILE_INTEL_BJAM_ARGS = ["architecture=x86"]
MAC_CROSS_COMPILE_INTEL_CXX = "clang++ -arch x86_64"
MAC_CROSS_COMPILE_INTEL_CC = "clang -arch x86_64"
MAC_CROSS_COMPILE_INTEL_AUTOCONF_HOST_ARGS = ["--host=x86_64-apple-darwin"]
else:
MAC_CROSS_COMPILE_INTEL_ARGS = []
MAC_CROSS_COMPILE_INTEL_BJAM_ARGS = []
MAC_CROSS_COMPILE_INTEL_CXX = ""
MAC_CROSS_COMPILE_INTEL_CC = ""
MAC_CROSS_COMPILE_INTEL_AUTOCONF_HOST_ARGS = []
OFF_ON = ["OFF", "ON"]
BUILD_STATIC = "shared" not in flags
"""Whether dependencies are built static."""
IFCOPENSHELL_STATIC = BUILD_STATIC and "ifcopenshell-shared" not in flags
"""Whether IfcOpenShell's own libraries are built static."""
OCCT_STATIC = BUILD_STATIC and "opencascade-shared" not in flags
"""Whether OCCT is built static. See ``-opencascade-shared``."""
ENABLE_FLAG = "--enable-static" if BUILD_STATIC else "--enable-shared"
DISABLE_FLAG = "--disable-shared" if BUILD_STATIC else "--disable-static"
LINK_TYPE = "static" if BUILD_STATIC else "shared"
LINK_TYPE_UCFIRST = LINK_TYPE.capitalize()
LIBRARY_EXT = "a" if BUILD_STATIC else "so"
PIC = "-fPIC" if BUILD_STATIC else ""
if any(f.startswith("py-") for f in flags):
PYTHON_VERSIONS = [pyv for pyv in PYTHON_VERSIONS if f"py-{''.join(pyv.split('.')[:2])}" in flags]
if any(f.startswith("occt-") for f in flags):
OCCT_VERSION = next(f.split("-", 1)[1] for f in flags if f.startswith("occt-"))
# Static and shared OCCT installs are not interchangeable, so they must not share
# a directory: `build_dependency` skips a dependency whose install dir already
# exists, and cache_dependencies.py keys its tarballs purely on that directory
# name. Without the suffix a cached static OCCT silently satisfies a shared build
# (and vice versa) and the requested link type is never applied.
OCCT_DIR_NAME = f"occt-{OCCT_VERSION}" + ("" if OCCT_STATIC else "-shared")
if not OCCT_STATIC:
# A shared OCCT has to be resolvable at run time by everything this script
# executes out of the install tree -- most visibly the post-build
# `import ifcopenshell` sanity check, which otherwise dies with
# "libTKernel.so.7.8: cannot open shared object file". Nothing points there:
# IfcOpenShell's libraries get INSTALL_RPATH=$ORIGIN (see SET_INSTALL_SELF_RPATH
# in cmake/utilities.cmake) and OCCT lives in its own dependency prefix.
#
# This is a build-time concern only. The shipped packages do not rely on it:
# the workflows stage libTK*.so* next to the payload and patchelf an $ORIGIN
# RUNPATH onto every staged library.
_occt_lib_dirs = [
os.path.join(DEPS_DIR, "install", OCCT_DIR_NAME, libdir)
for libdir in ("lib", "lib64")
]
os.environ["LD_LIBRARY_PATH"] = os.pathsep.join(
[*_occt_lib_dirs, os.environ.get("LD_LIBRARY_PATH", "")]
).rstrip(os.pathsep)
if explicit_targets:
targets = {dep for target in explicit_targets for dep in gather_dependencies(target)}
else:
targets = set(dependency_tree.keys())
targets = set(t for t in targets if "without-%s" % t.lower() not in flags)
if not explicit_targets and not BUILD_BONSAIVIEWER:
targets.difference_update({"BonsaiViewer", "qt6"})
if BUILD_BONSAIVIEWER:
targets.update(gather_dependencies("BonsaiViewer"))
# Opt-out for the Python wrapper. Currently used by the bonsai CI workflow
# on macOS, where the post-plug-in-refactor wrapper hard-links
# ifcopenshell.document.rdb.dylib but the CREATE_BUNDLE install rule does
# not actually drop the dylib next to the wrapper in site-packages — see
# the commit message that introduced this gate.
if not IFCOS_BUILD_PYTHON_WRAPPER:
targets.discard("IfcOpenShell-Python")
if WASM:
SKIP_TARGETS_FOR_WASM = {
"rocksdb",
"opencollada",
"pcre",
"IfcGeom",
"IfcConvert",
"IfcGeomServer",
"BonsaiViewer",
"qt6",
}
SKIP_TARGETS_FOR_WASM = {t.lower() for t in SKIP_TARGETS_FOR_WASM}
skip_targets = {t for t in targets if t.lower() in SKIP_TARGETS_FOR_WASM}
if skip_targets:
cecho(f"Skipping targets for wasm build: {', '.join(sorted(skip_targets))}", YELLOW)
targets.difference_update(skip_targets)
print("Building:", *sorted(targets, key=lambda t: len(list(gather_dependencies(t)))))
# Check that required tools are in PATH
yacc = "yacc" # Used during swig building process, installed with `bison` on Debian / `byacc` on Red Hat.
bison = "bison"
missing_commands: list[str] = []
required_commands = [git, bunzip2, tar, cc, cplusplus, autoconf, automake, make, "patch", "cmake", yacc, xz, bison]
if WASM:
required_commands.append("pyodide")
if platform.system() == "Linux" and "BonsaiViewer" in targets:
required_commands.append("patchelf")
for cmd in required_commands:
if shutil.which(cmd) is None:
missing_commands.append(cmd)
if missing_commands:
raise ValueError(f"Required tools not installed or not added to PATH: {', '.join(missing_commands)}")
MAC_INTEL_BIN_PATH = "/usr/local/bin"
if MAC_CROSS_COMPILE_INTEL:
brew = f"{MAC_INTEL_BIN_PATH}/brew"
assert os.path.exists(brew), f"For intel cross compilation the brew path is expected to be '{brew}'."
# identifiers for the download tool (could be less memory consuming as ints, but are more verbose as strings)
download_tool_default = download_tool_py = "py"
download_tool_git = "git"
# Create log directory and file
log_dir = os.path.join(DEPS_DIR, "logs")
if not os.path.exists(log_dir):
os.makedirs(log_dir)
LOG_FILE = os.path.join(log_dir, sp.check_output([date, "+%Y%m%d"], encoding="utf-8").strip()) + ".log"
if not os.path.exists(LOG_FILE):
open(LOG_FILE, "w").close()
logger.info(f"using command log file '{LOG_FILE}'")
# Causing havoc in python 3.11 build
try:
del os.environ["__PYVENV_LAUNCHER__"]
except:
pass
def restore_env(var_name: str, old_value: str | None) -> None:
if old_value is None:
del os.environ[var_name]
else:
os.environ[var_name] = old_value
def run(cmds: Sequence[str], cwd: str | None = None, can_fail: bool = False) -> str:
"""
Wraps `subprocess.Popen.communicate()` and logs the command being executed,
sets up logging `stderr` to `LOG_FILE` (in append mode) and returns stdout
with leading and trailing whitespace removed.
"""
def timestamp() -> str:
return datetime.now().strftime("%Y-%m-%d %H:%M:%S,%f")[:-3] # same format as logging
def stream_reader(pipe, collector: list[str], log_file) -> None:
for line in iter(pipe.readline, ""):
log_file.write(f"{timestamp()} {line}")
log_file.flush()
collector.append(line)
pipe.close()
logger.debug(f"running command `{' '.join(cmds)}` in directory '{cwd}'")
stdout: list[str] = []
stderr: list[str] = []
# Ensure both live logs available in the log file
# and the putput.
with open(LOG_FILE, "a", encoding="utf-8") as log_file_handle:
proc = sp.Popen(cmds, cwd=cwd, stdout=sp.PIPE, stderr=sp.PIPE, encoding="utf-8")
assert proc.stdout and proc.stderr
t_out = threading.Thread(target=stream_reader, args=(proc.stdout, stdout, log_file_handle))
t_err = threading.Thread(target=stream_reader, args=(proc.stderr, stderr, log_file_handle))
t_out.start()
t_err.start()
t_out.join()
t_err.join()
proc.wait()
logger.debug(f"command returned {proc.returncode}")
if proc.returncode != 0 and not can_fail:
print("-" * 70)
print("".join(stderr))
print("-" * 70)
raise RuntimeError(f"Command `{' '.join(cmds)}` returned exit code {proc.returncode}")
return "".join(stdout).strip()
if platform.system() == "Darwin":
if run(["sw_vers", "-productVersion"]) >= "11.":
# Apparently not supported
PYTHON_VERSIONS = [pv for pv in PYTHON_VERSIONS if tuple(map(int, pv.split("."))) >= (3, 7)]
if run(["sw_vers", "-productVersion"]) < "10.16":
# This is now solved with the '__PYVENV_LAUNCHER__' hack
# PYTHON_VERSIONS = [pv for pv in PYTHON_VERSIONS if tuple(map(int, pv.split("."))) < (3, 11)]
pass
BOOST_VERSION_UNDERSCORE = BOOST_VERSION.replace(".", "_")
OCE_LOCATION = f"https://github.com/tpaviot/oce/archive/OCE-{OCE_VERSION}.tar.gz"
BOOST_LOCATION = f"https://github.com/boostorg/boost/releases/download/boost-{BOOST_VERSION}/"
# Helper functions
def run_autoconf(dependency_name: str, configure_args: list[str], cwd: str) -> None:
configure_path = os.path.realpath(os.path.join(cwd, "..", "configure"))
if not os.path.exists(configure_path):
run(
[bash, "./autogen.sh"], cwd=os.path.realpath(os.path.join(cwd, ".."))
) # only run autogen.sh in the directory it is located and use cwd to achieve that in order to not mess up things
# Using `sh` over `bash` fixes issues with building swig
prefix = os.path.realpath(f"{DEPS_DIR}/install/{dependency_name}")
wasm = []
if WASM:
wasm.append("emconfigure")
run(
[
*wasm,
"/bin/sh",
"../configure",
*(["--host=wasm32"] if WASM and not any(s.startswith("--host") for s in configure_args) else []),
*configure_args,
f"--prefix={prefix}",
],
cwd=cwd,
)
def run_cmake(
name, cmake_args: list[str], cmake_dir: str | None = None, cwd: str | None = None, native: bool = False
) -> None:
if cmake_dir is None:
P = ".."
else:
P = cmake_dir
wasm = []
if WASM and not native:
wasm.append("emcmake")
cmake_flags: list[str] = []
if not native and (not WASM or not WASM_CMAKE_IS_USING_INIT_VARS):
# For WASM we provide flags using just environment variables.
# If we provide them using cmake vars, it will override emscripten toolchain flags.
# Unsure if we need this in general even for non-WASM builds.
cmake_flags.extend(
[
f"-DCMAKE_CXX_FLAGS='{os.environ['CXXFLAGS']}'",
f"-DCMAKE_C_FLAGS='{os.environ['CFLAGS']}'",
]
)
if not any("BUILD_SHARED_LIBS" in f for f in cmake_args):
cmake_flags.append(
f"-DBUILD_SHARED_LIBS={OFF_ON[not BUILD_STATIC]}",
)
if WASM and native:
# Override emscripten cmake toolchain coming from environment variable.
cmake_flags.append("-DCMAKE_TOOLCHAIN_FILE=")
run(
[
*wasm,
"cmake",
P,
*cmake_flags,
*cmake_args,
f"-DCMAKE_BUILD_TYPE={BUILD_CFG}",
*([] if native else [f"-DCMAKE_SHARED_LINKER_FLAGS={os.environ['LDFLAGS']}"]),
],
cwd=cwd,
)
def git_clone_or_pull_repository(clone_url: str, target_dir: str, revision: str | None = None) -> None:
"""Lazily clones the `git` repository denoted by `clone_url` into
the `target_dir` or pulls latest changes if the `target_dir` exists (naively assumes
that a working clone exists there) and optionally checks out a revision
`revision` after cloning or in the existing clone if `revision` is not
`None`."""
if not os.path.exists(target_dir):
logger.info(f"cloning '{clone_url}' into '{target_dir}'")
run([git, "clone", "--recursive", clone_url, target_dir])
else:
logger.info(f"directory '{target_dir}' already cloned. Pulling latest changes.")
run([git, "-C", target_dir, "fetch", "--all", "--tags", "--force"])
# detect whether we are on a branch and pull
if run([git, "rev-parse", "--abbrev-ref", "HEAD"], cwd=target_dir) != "HEAD":
run([git, "pull", clone_url], cwd=target_dir)
if revision != None:
run([git, "reset", "--hard"], cwd=target_dir)
run([git, "fetch", "--all"], cwd=target_dir)
run([git, "checkout", revision], cwd=target_dir)
def build_dependency(
name: str,
mode: Literal[
"cmake",
"autoconf",
"bjam",
],
build_tool_args: list[str],
download_url: str,
download_name: str,
*,
download_tool: Literal["py", "git"] = download_tool_default,
revision: str | None = None,
patch: list[str] | None = None,
shell=None,
pre_compile_subs: Sequence[tuple[str, str, str]] = (),
additional_files: dict[str, str] | None = None,
no_append_name=False,
cmake_dir=None,
cmake_native: bool = False,
) -> None:
"""Handles building of dependencies with different tools (which are
distinguished with the `mode` argument. `build_tool_args` is expected to be
a list which is necessary in order to not mess up quoting of compiler and
linker flags.
:param pre_compile_subs: A sequence of ``(fn, before, after)``
:param additional_files: Mapping path->url.
:param cmake_native: For ``mode="cmake"``, force a native (host) build
even when building for WASM. Needed for build-time tools like swig.
"""
check_dir = os.path.join(DEPS_DIR, "install", name)
if os.path.exists(check_dir):
logger.info(f"Found existing {name}, skipping")
return
build_dir = os.path.join(DEPS_DIR, "build")
if not os.path.exists(build_dir):
os.makedirs(build_dir)
logger.info(f"\rFetching {name}... ")
if download_tool == download_tool_py:
if no_append_name:
url = download_url
else:
url = os.path.join(download_url, download_name)
download_path = os.path.join(build_dir, download_name)
if not os.path.exists(download_path):
for _ in range(3):
try:
urlretrieve(url, os.path.join(build_dir, download_path))
break
except ConnectionError as e:
print(e, "... retrying...")
time.sleep(30.0)
continue
else:
logger.info(
f"Download '{download_path}' already exists, assuming it's an undamaged download and that it has been extracted if possible, skipping"
)
elif download_tool == download_tool_git:
logger.info(f"\rChecking {name}... ")
git_clone_or_pull_repository(download_url, target_dir=os.path.join(build_dir, download_name), revision=revision)
else:
assert_never(download_tool)
download_dir = os.path.join(build_dir, download_name)
if os.path.isdir(download_dir):
extract_dir_name = download_name
extract_dir = os.path.join(build_dir, extract_dir_name)
else:
download_tarfile_path = os.path.join(build_dir, download_name)
if download_name.endswith(".tar.gz") or download_name.endswith(".tgz"):
compr = "gz"
elif download_name.endswith(".tar.bz2"):
compr = "bz2"
elif download_name.endswith(".tar.xz"):
compr = "xz"
else:
raise RuntimeError("fix source for new download type")
# ty: false positive bug upstream.
download_tarfile = tarfile.open(
name=download_tarfile_path, mode=f"r:{compr}"
) # ty:ignore[no-matching-overload]
# tarfile seriously doesn't have a function to retrieve the root directory more easily
extract_dir_name = os.path.commonprefix([x for x in download_tarfile.getnames() if x != "."])
# run([tar, "--exclude=\"*/*\"", "-tf", download_name], cwd=build_dir).strip() no longer works
if extract_dir_name is None:
extract_dir_name = run(
[bash, "-c", f"tar -tf {download_name} 2> /dev/null | head -n 1 | cut -f1 -d /"], cwd=build_dir
)
extract_dir = os.path.join(build_dir, extract_dir_name)
if not os.path.exists(extract_dir):
run([tar, "-xf", download_name], cwd=build_dir)
if additional_files:
for path, url in additional_files.items():
if not os.path.exists(path):
urlretrieve(url, os.path.join(extract_dir, path))
if patch is not None:
for p in patch:
patch_abs = (SCRIPT_PATH / p).absolute().__str__()
if os.path.exists(patch_abs):
try:
run(["patch", "-p1", "--batch", "--forward", "-i", patch_abs], cwd=extract_dir)
except Exception as e:
# Assert that the patch has already been applied
run(["patch", "-p1", "--batch", "--reverse", "--dry-run", "-i", patch_abs], cwd=extract_dir)
else:
raise FileNotFoundError(patch_abs)
if shell is not None:
sp.run(shell, shell=True, check=True, cwd=extract_dir)
if mode != "bjam":
extract_build_dir = os.path.join(extract_dir, *([cmake_dir] if cmake_dir else []), "build")
if os.path.exists(extract_build_dir):
shutil.rmtree(extract_build_dir)
os.makedirs(extract_build_dir)
logger.info(f"\rConfiguring {name}...")
if mode == "autoconf":
run_autoconf(name, build_tool_args, cwd=extract_build_dir)
elif mode == "cmake":
run_cmake(name, build_tool_args, cwd=extract_build_dir, native=cmake_native)
else:
assert_never(mode)
for fn, before, after in pre_compile_subs:
with open(os.path.join(extract_dir, fn), "r") as f:
s = f.read()
s = s.replace(before, after)
with open(os.path.join(extract_dir, fn), "w") as f:
f.write(s)
logger.info(f"\rBuilding {name}... ")
run([make, f"-j{IFCOS_NUM_BUILD_PROCS}", "VERBOSE=1"], cwd=extract_build_dir)
logger.info(f"\rInstalling {name}... ")
run([make, "install"], cwd=extract_build_dir)
logger.info(f"\rInstalled {name} \n")
else: # bjam
logger.info(f"\rConfiguring {name}...")
run([bash, "./bootstrap.sh"], cwd=extract_dir)
logger.info(f"\rBuilding {name}... ")
run(["./b2", f"-j{IFCOS_NUM_BUILD_PROCS}"] + build_tool_args, cwd=extract_dir, can_fail=WASM)
logger.info(f"\rInstalling {name}... ")
shutil.copytree(
os.path.join(extract_dir, "boost"), os.path.join(DEPS_DIR, "install", f"boost-{BOOST_VERSION}", "boost")
)
logger.info(f"\rInstalled {name} \n")
if DISK_CLEANUP:
shutil.rmtree(build_dir, ignore_errors=True)
def get_qt6_aqt_config() -> tuple[str, str, str]:
if platform.system() != "Linux":
raise ValueError("Automatic Qt6 installation with aqtinstall is only configured for Linux builds.")
machine = platform.machine().lower()
if machine in {"x86_64", "amd64"}:
return "linux", "linux_gcc_64", "gcc_64"
if machine in {"aarch64", "arm64"}:
return "linux_arm64", "linux_gcc_arm64", "gcc_arm64"
raise ValueError(f"Automatic Qt6 installation is not configured for architecture '{platform.machine()}'.")
def install_qt6() -> str:
# If the caller pre-set QT_DIR (e.g. macOS CI using Homebrew-installed
# Qt6), validate it points at a real Qt6 install and skip aqtinstall
# entirely. The aqt download is only wired for Linux; on macOS/Windows
# the supported flow is a pre-installed Qt6 advertised via QT_DIR.
preset_qt_dir = os.environ.get("QT_DIR", "").strip()
if preset_qt_dir:
preset_qt_config = Path(preset_qt_dir) / "lib" / "cmake" / "Qt6" / "Qt6Config.cmake"
if preset_qt_config.exists():
logger.info(f"Using pre-set QT_DIR={preset_qt_dir}, skipping aqt install")
return preset_qt_dir
logger.warning(
f"QT_DIR={preset_qt_dir} is set but {preset_qt_config} not found; " f"falling through to aqtinstall"
)
host, qt_arch, install_suffix = get_qt6_aqt_config()
qt_install_root = INSTALL_DIR / f"qt6-{QT6_VERSION}-{install_suffix}"
qt_dir = qt_install_root / QT6_VERSION / install_suffix
os.environ["QT_DIR"] = str(qt_dir)
qt_config = qt_dir / "lib" / "cmake" / "Qt6" / "Qt6Config.cmake"
qt_core = qt_dir / "lib" / "libQt6Core.so.6"
qt_svg = qt_dir / "lib" / "cmake" / "Qt6Svg" / "Qt6SvgConfig.cmake"
if qt_config.exists() and qt_core.exists() and qt_svg.exists():
logger.info(f"Found existing Qt6 at {qt_dir}, skipping")
return str(qt_dir)
os.makedirs(qt_install_root, exist_ok=True)
try:
import aqt # ty:ignore[unresolved-import]
except ModuleNotFoundError:
logger.error(
"Could not find an existing Qt6 install, so aqtinstall is needed to fetch it automatically. "
"Install the `aqtinstall` PyPI package or set QT_DIR."
)
exit(1)
run(
[
sys.executable,
"-m",
"aqt",
"install-qt",
host,
"desktop",
QT6_VERSION,
qt_arch,
"-O",
str(qt_install_root),
# Keep the install lean by filtering archives: qtbase provides
# Core/Gui/Widgets (and the Qt6::CorePrivate target), qtsvg provides
# Qt6::Svg. Both are base-Qt archives, not add-on modules.
"--archives",
"icu",
"qtbase",
"qtsvg",
]
)
if not (qt_config.exists() and qt_core.exists() and qt_svg.exists()):
raise RuntimeError(f"Qt6 installation did not produce a usable Qt at {qt_dir}.")
return str(qt_dir)
cecho("Collecting dependencies:", GREEN)
# Set compiler flags for 32bit builds on 64bit system
# TODO: This is untested
ADDITIONAL_ARGS = []
if platform.system() == "Darwin":
ADDITIONAL_ARGS = [f"-mmacosx-version-min={TOOLSET}"] + ADDITIONAL_ARGS
# If the linker supports GC sections, set it up to reduce binary file size
# -fPIC is required for the shared libraries to work
compiler_flags = "CFLAGS", "CXXFLAGS", "LDFLAGS"
CXXFLAGS = os.environ.get("CXXFLAGS", "")
CFLAGS = os.environ.get("CFLAGS", "")
LDFLAGS = os.environ.get("LDFLAGS", "")
ADDITIONAL_ARGS_STR = " ".join(ADDITIONAL_ARGS)
CXXFLAGS_MINIMAL = f"{CXXFLAGS} {PIC} {ADDITIONAL_ARGS_STR}"
CFLAGS_MINIMAL = f"{CFLAGS} {PIC} {ADDITIONAL_ARGS_STR}"
if WASM:
# WASM `SIDE_MODULE_` are absorbed by `emcmake` automatically.
CXXFLAGS = CXXFLAGS_MINIMAL
CFLAGS = CFLAGS_MINIMAL
elif sp.call([bash, "-c", "ld --gc-sections 2>&1 | grep -- --gc-sections &> /dev/null"]) != 0:
if BUILD_STATIC:
CXXFLAGS = f"{CXXFLAGS} {PIC} -fdata-sections -ffunction-sections -fvisibility=hidden -fvisibility-inlines-hidden {ADDITIONAL_ARGS_STR}"
CFLAGS = f"{CFLAGS} {PIC} -fdata-sections -ffunction-sections -fvisibility=hidden {ADDITIONAL_ARGS_STR}"
else:
CXXFLAGS = CXXFLAGS_MINIMAL
CFLAGS = CFLAGS_MINIMAL
LDFLAGS = f"{LDFLAGS} -Wl,--gc-sections {ADDITIONAL_ARGS_STR}"
else:
if BUILD_STATIC:
CXXFLAGS = f"{CXXFLAGS} {PIC} -fvisibility=hidden -fvisibility-inlines-hidden {ADDITIONAL_ARGS_STR}"
CFLAGS = f"{CFLAGS} {PIC} -fvisibility=hidden -fvisibility-inlines-hidden {ADDITIONAL_ARGS_STR}"
else:
CXXFLAGS = CXXFLAGS_MINIMAL
CFLAGS = CFLAGS_MINIMAL
LDFLAGS = f"{LDFLAGS} {ADDITIONAL_ARGS_STR}"
if LTO:
for f in compiler_flags:
locals()[f] += f" -flto={IFCOS_NUM_BUILD_PROCS}"
os.environ["CXXFLAGS"] = CXXFLAGS
os.environ["CPPFLAGS"] = CXXFLAGS
os.environ["CFLAGS"] = CFLAGS
os.environ["LDFLAGS"] = LDFLAGS
# Use colors in logs to make them more readable locally.
# Output from compiler used by cmake.
os.environ["CMAKE_COLOR_DIAGNOSTICS"] = "ON"
# Output from cmake itself.
os.environ["CLICOLOR_FORCE"] = "1"
# Some dependencies need a more recent CMake version than most distros provide
# @tfk: this is no longer needed
# build_dependency(name="cmake-%s" % (CMAKE_VERSION,), mode="autoconf", build_tool_args=[], download_url="https://cmake.org/files/v%s" % (CMAKE_VERSION_2,), download_name="cmake-%s.tar.gz" % (CMAKE_VERSION,))
if "json" in targets:
dependency_name = f"json-{JSON_VERSION}"
build_dependency(
name=dependency_name,
mode="cmake",
build_tool_args=[
f"-DCMAKE_INSTALL_PREFIX={DEPS_DIR}/install/{dependency_name}",
"-DJSON_BuildTests=OFF",
],
download_url=f"https://github.com/nlohmann/json/releases/download/v{JSON_VERSION}",
download_name="json.tar.xz",
)
if "eigen" in targets:
dependency_name = f"eigen-install-{EIGEN_VERSION}"
build_dependency(
name=f"{dependency_name}",
mode="cmake",
# We add '-install-' in the middle, so it won't be confused with git repo we used previously.
build_tool_args=[
f"-DCMAKE_INSTALL_PREFIX={DEPS_DIR}/install/{dependency_name}",
],
download_url=f"https://gitlab.com/libeigen/eigen/-/archive/{EIGEN_VERSION}/",
download_name=f"eigen-{EIGEN_VERSION}.tar.gz",
)
if "pcre" in targets:
OLD_CC, OLD_CXX = None, None
if MAC_CROSS_COMPILE_INTEL:
OLD_CC, OLD_CXX = os.environ.get("CC"), os.environ.get("CXX")
os.environ["CC"] = MAC_CROSS_COMPILE_INTEL_CC
os.environ["CXX"] = MAC_CROSS_COMPILE_INTEL_CXX
# Keep it autoconf as OpenCOLLADA is pretty old and might break
# if we update it's dependencies for mmore modern cmake.
build_dependency(
name=f"pcre-{PCRE_VERSION}",
mode="autoconf",
build_tool_args=[DISABLE_FLAG],
download_url=f"https://downloads.sourceforge.net/project/pcre/pcre/{PCRE_VERSION}/",
download_name=f"pcre-{PCRE_VERSION}.tar.bz2",
)
if MAC_CROSS_COMPILE_INTEL:
restore_env("CC", OLD_CC)
restore_env("CXX", OLD_CXX)
if "swig" in targets:
dependency_name = f"swig-{SWIG_VERSION}"
build_dependency(
name=dependency_name,
mode="cmake",
build_tool_args=[
"-DWITH_PCRE=OFF",
f"-DCMAKE_INSTALL_PREFIX={DEPS_DIR}/install/{dependency_name}",
],
download_url="https://github.com/swig/swig.git",
download_name="swig",
download_tool=download_tool_git,
revision=f"v{SWIG_VERSION}",
cmake_native=WASM,
)
if USE_OCCT and "occ" in targets:
occt_args: list[str] = []
patches: list[str] = []
if OCCT_VERSION < "7.4":
patches.append("./patches/occt/enable-exception-handling.patch")
# Skip ExpToCasExe as we don't need it and it requires additional dependencies.
# Before 7.7.2 ExpToCasExe is part of DataExchange, DETools doesn't exist yet.
# Since we do need DataExchange (used for iges_serializer), we use a patch to skip only ExpToCasExe.
if "7.7.2" > OCCT_VERSION >= "7.7":
patches.append("./patches/occt/no_ExpToCasExe.patch")
elif OCCT_VERSION >= "7.7.2":
occt_args.append("-DBUILD_MODULE_DETools=OFF")
if WASM:
patches.append("./patches/occt/no_em_js.patch")
if not OCCT_STATIC:
# BUILD_STATIC drives three things at once: the dependency link type,
# -fvisibility=hidden, and BUILD_SHARED_LIBS. Building only OCCT shared
# means overriding all three for it, not just the link type.
#
# Visibility matters most. OCCT's Standard_EXPORT expands to nothing on
# Unix (Standard_Macro.hxx), so it relies on default visibility to export
# its API. Built shared under -fvisibility=hidden it exports almost
# nothing and its libraries fail to resolve against each other -- e.g.
# libTKMath.so cannot find NCollection_BaseAllocator::CommonBaseAllocator
# in libTKernel.so. Static archives are immune, which is why this only
# appears once OCCT goes shared. CXXFLAGS_MINIMAL is the pre-visibility
# flag set, so this restores default visibility without dropping -O3/-fPIC.
#
# These come after the generic flags in the cmake command line, and the
# last -D for a given variable wins.
occt_args.append(f"-DCMAKE_CXX_FLAGS={CXXFLAGS_MINIMAL}")
occt_args.append(f"-DCMAKE_C_FLAGS={CFLAGS_MINIMAL}")
# Suppresses the generic -DBUILD_SHARED_LIBS=OFF that BUILD_STATIC would
# otherwise add, which contradicts BUILD_LIBRARY_TYPE=Shared.
occt_args.append("-DBUILD_SHARED_LIBS=ON")
build_dependency(
name=OCCT_DIR_NAME,
mode="cmake",
build_tool_args=[
f"-DINSTALL_DIR={DEPS_DIR}/install/{OCCT_DIR_NAME}",
f"-DBUILD_LIBRARY_TYPE={'Static' if OCCT_STATIC else 'Shared'}",
f"-DBUILD_MODULE_Draw=0",
f"-DBUILD_RELEASE_DISABLE_EXCEPTIONS=Off",
# Disable xlib explicitly, as it tries to use it on Desktop Ubuntu, adding unnecessary dependency.
f"-DUSE_XLIB=OFF",
# Avoid building 3D Viewer.
f"-DUSE_FREETYPE=OFF",
f"-DUSE_OPENGL=OFF",
f"-DUSE_GLES2=OFF",
f"-DCMAKE_POLICY_VERSION_MINIMUM=3.5",
*MAC_CROSS_COMPILE_INTEL_ARGS,
*occt_args,
],
download_url="https://github.com/Open-Cascade-SAS/OCCT",
download_name="occt",
download_tool=download_tool_git,
patch=patches,
revision="V" + OCCT_VERSION.replace(".", "_"),
)
elif "occ" in targets:
build_dependency(
name=f"oce-{OCE_VERSION}",
mode="cmake",
build_tool_args=[
f"-DOCE_DISABLE_TKSERVICE_FONT=ON",
f"-DOCE_TESTING=OFF",
f"-DOCE_BUILD_SHARED_LIB=OFF",
f"-DOCE_DISABLE_X11=ON",
f"-DOCE_VISUALISATION=OFF",
f"-DOCE_OCAF=OFF",
f"-DOCE_INSTALL_PREFIX={DEPS_DIR}/install/oce-{OCE_VERSION}",
],
download_url="https://github.com/tpaviot/oce/archive/",
download_name=f"OCE-{OCE_VERSION}.tar.gz",
)
if "manifold" in targets:
dependency_name = f"manifold-{MANIFOLD_VERSION}"
patches = []
if WASM:
patches.append("./patches/manifold/install-metadata-for-emscripten.patch")
build_dependency(
name=dependency_name,
mode="cmake",
build_tool_args=[
f"-DCMAKE_INSTALL_PREFIX={DEPS_DIR}/install/{dependency_name}",
"-DMANIFOLD_PAR=OFF",
"-DMANIFOLD_CROSS_SECTION=OFF",
"-DMANIFOLD_PYBIND=OFF",
"-DMANIFOLD_JSBIND=OFF",
"-DMANIFOLD_CBIND=OFF",
"-DMANIFOLD_TEST=OFF",
"-DMANIFOLD_EXPORT=OFF",
"-DMANIFOLD_DOWNLOADS=OFF",
*MAC_CROSS_COMPILE_INTEL_ARGS,
],
download_url="https://github.com/elalish/manifold.git",
download_name="manifold",
download_tool=download_tool_git,
revision=f"v{MANIFOLD_VERSION}",
patch=patches,
)
if "libxml2" in targets:
OLD_CC = ""
if MAC_CROSS_COMPILE_INTEL:
OLD_CC = os.environ.get("CC")
os.environ["CC"] = MAC_CROSS_COMPILE_INTEL_CC
build_tool_args = [
"--without-python",
ENABLE_FLAG,
DISABLE_FLAG,
"--without-zlib",
"--without-iconv",
"--without-lzma",
]
if WASM:
build_tool_args.append("--without-threads")
build_dependency(
f"libxml2-{LIBXML2_VERSION}",
"autoconf",
build_tool_args=build_tool_args,
download_url=f"https://download.gnome.org/sources/libxml2/{'.'.join(LIBXML2_VERSION.split('.')[0:2])}/",
download_name=f"libxml2-{LIBXML2_VERSION}.tar.xz",
)
if MAC_CROSS_COMPILE_INTEL:
if OLD_CC is None:
del os.environ["CC"]
else:
os.environ["CC"] = OLD_CC
if "OpenCOLLADA" in targets:
patches = ["./patches/opencollada/pr622_and_disable_subdirs.patch"]
# OpenCOLLADAConfig.cmake.in hardcodes shared-lib targets on Unix regardless of
# whether shared libs were actually built. We make it follow `USE_SHARED` instead.
patches.append("./patches/opencollada/config_select_libs_by_use_shared.patch")
if WASM:
# This is necessary for the WASM build, because recent versions of
# clang don't have the tr1:: namespace anymore. However, it breaks
# some versions of gcc (9.4.0 at least) due to specializing std::hash
# outside of the std:: namespace.
patches.append("./patches/opencollada/remove_tr1.patch")
build_dependency(
"OpenCOLLADA",
"cmake",
build_tool_args=[
f"-DLIBXML2_INCLUDE_DIR={DEPS_DIR}/install/libxml2-{LIBXML2_VERSION}/include/libxml2",
f"-DLIBXML2_LIBRARIES={DEPS_DIR}/install/libxml2-{LIBXML2_VERSION}/lib/libxml2.{LIBRARY_EXT}",
f"-DPCRE_INCLUDE_DIR={DEPS_DIR}/install/pcre-{PCRE_VERSION}/include",
f"-DPCRE_PCREPOSIX_LIBRARY={DEPS_DIR}/install/pcre-{PCRE_VERSION}/lib/libpcreposix.{LIBRARY_EXT}",
f"-DPCRE_PCRE_LIBRARY={DEPS_DIR}/install/pcre-{PCRE_VERSION}/lib/libpcre.{LIBRARY_EXT}",
f"-DCMAKE_INSTALL_PREFIX={DEPS_DIR}/install/OpenCOLLADA/",
# OpenCOLLADA is ancient at this point and allows cmake 2.6+, which results in error in cmake 4.
f"-DCMAKE_POLICY_VERSION_MINIMUM=3.5",
f"-DUSE_STATIC={OFF_ON[BUILD_STATIC]}",
f"-DUSE_SHARED={OFF_ON[not BUILD_STATIC]}",
*MAC_CROSS_COMPILE_INTEL_ARGS,
],
download_url="https://github.com/KhronosGroup/OpenCOLLADA.git",
download_name="OpenCOLLADA",
download_tool=download_tool_git,
patch=patches,
revision=OPENCOLLADA_VERSION,
)
if "python" in targets and not USE_CURRENT_PYTHON_VERSION and not WASM:
# Python should not be built with -fvisibility=hidden, from experience that introduces segfaults
OLD_CPP_FLAGS = os.environ["CPPFLAGS"]
OLD_CXX_FLAGS = os.environ["CXXFLAGS"]
OLD_C_FLAGS = os.environ["CFLAGS"]
os.environ["CXXFLAGS"] = CXXFLAGS_MINIMAL
os.environ["CPPFLAGS"] = CXXFLAGS_MINIMAL
os.environ["CFLAGS"] = CFLAGS_MINIMAL
# On OSX a dynamic python library is built or it would not be compatible
# with the system python because of some threading initialization
PYTHON_CONFIGURE_ARGS: list[str] = []
original_path = ""
if platform.system() == "Darwin":
PYTHON_CONFIGURE_ARGS = ["--enable-shared"]
open_ssl_prefix = run([brew, "--prefix", "openssl@3"]).strip()
# I'm not sure why, but if I do `"{open_ssl_prefix}"` (keep the quotes),
# autconf fails to find ssl.
PYTHON_CONFIGURE_ARGS.append(f"--with-openssl={open_ssl_prefix}")
if MAC_CROSS_COMPILE_INTEL:
original_path = os.environ["PATH"]
# Need to ensure python will pick up intel's `pkg-config`,
# otherwise it might attempt to use ARM libraries (e.g. `zstd`) and fail.
os.environ["PATH"] = f"{MAC_INTEL_BIN_PATH}{os.pathsep}{original_path}"
PYTHON_CONFIGURE_ARGS.extend(["--with-universal-archs=intel-64", "--enable-universalsdk"])
for PYTHON_VERSION in PYTHON_VERSIONS:
# Don't fail silently on missing Python dependencies (e.g. openssl or zlib),
# because later ifcopenshell-python build will fail too but in a more confusing way.
build_dependency(
f"python-{PYTHON_VERSION}",
"autoconf",
PYTHON_CONFIGURE_ARGS,
f"http://www.python.org/ftp/python/{PYTHON_VERSION}/",
f"Python-{PYTHON_VERSION}.tgz",
)
python_install = INSTALL_DIR / f"python-{PYTHON_VERSION}"
python_bin = python_install / "bin" / "python3"
# `_ssl` module is present -> we will be able to install `numpy` later
# to verify IfcOpenShell installation
try:
run([str(python_bin), "-c", "import _ssl"])
except RuntimeError:
print(
"ERROR: Python was built without SSL support (_ssl module is missing). "
f"To fix this: remove the installed Python at {python_install}; "
"install OpenSSL development libraries and re-run."
)
raise
if MAC_CROSS_COMPILE_INTEL:
assert original_path
os.environ["PATH"] = original_path
os.environ["CPPFLAGS"] = OLD_CPP_FLAGS
os.environ["CXXFLAGS"] = OLD_CXX_FLAGS
os.environ["CFLAGS"] = OLD_C_FLAGS
if "boost" in targets:
str_concat = lambda prefix: lambda postfix: "" if postfix.strip() == "" else "=".join((prefix, postfix.strip()))
toolset = []
if WASM:
toolset.append("toolset=emscripten")
build_dependency(
f"boost-{BOOST_VERSION}",
mode="bjam",
build_tool_args=[
f"--stagedir={DEPS_DIR}/install/boost-{BOOST_VERSION}",
"--with-system",
"--with-program_options",
"--with-regex",
"--with-thread",
"--with-date_time",
"--with-iostreams",
"--with-filesystem",
f"link={LINK_TYPE}",
*toolset,
*map(str_concat("cxxflags"), CXXFLAGS.strip().split(" ")),
*map(str_concat("linkflags"), LDFLAGS.strip().split(" ")),
"stage",
"-s",
"NO_BZIP2=1",
*MAC_CROSS_COMPILE_INTEL_BJAM_ARGS,
],
download_url=BOOST_LOCATION,
# don't remember what this is, but fail on 1.86
# patch="./patches/boost/boostorg_regex_62.patch",
download_name=f"boost-{BOOST_VERSION}-b2-nodocs.tar.gz",
)
if WASM:
# only supported on nix for now
run(
("find", ".", "-name", "*.bc", "-exec", "bash", "-c", "emar q ${1%.bc}.a $1", "bash", "{}", ";"),
cwd=f"{DEPS_DIR}/install/boost-{BOOST_VERSION}/lib",
)
if "cgal" in targets:
gmp_args: list[str] = []
mpfr_args: list[str] = []
OLD_HOST_CC = None
if WASM:
if APPLE:
# Override `HOST_CC`, otherwise `emcc` will try to use it's own `clang` which can only build
# wasm executables and build will fail.
os.environ["HOST_CC"] = "clang"
# Disable assembly, otherwise `emcc -c conftest.s` will crash due to assembly mismatch.
gmp_args.extend(("--disable-assembly", "--enable-cxx"))
mpfr_args.extend(("--host", "none"))
elif "x86" in arch:
gmp_args.append("--enable-fat") # See issues #7458 #7556
OLD_CC = None
if MAC_CROSS_COMPILE_INTEL:
OLD_CC = os.environ.get("CC")
# Otherwise it's using arm64 `gcc` and fails to build gmp.
os.environ["CC"] = MAC_CROSS_COMPILE_INTEL_CC
gmp_args.extend(MAC_CROSS_COMPILE_INTEL_AUTOCONF_HOST_ARGS)
# Fixes configure failing to find a working compiler under GCC 15's default -std=gnu23.
# Issue presumably will be resolved in any next gmp version, but currently the last one is 6.3.0.
# Patch is just applying fix from upstream meantion below:
# https://gmplib.org/list-archives/gmp-bugs/2025-February/005561.html
gmp_patches = ["./patches/gmp/001-fix-std23.patch"]
if GMP_VERSION != "6.3.0":
raise Exception(f"GMP_VERSION changed to {GMP_VERSION}, check whether {gmp_patches} is still needed.")
build_dependency(
name=f"gmp-{GMP_VERSION}",
mode="autoconf",
build_tool_args=[ENABLE_FLAG, DISABLE_FLAG, "--with-pic", *gmp_args],
pre_compile_subs=([("build/config.h", "HAVE_OBSTACK_VPRINTF 1", "HAVE_OBSTACK_VPRINTF 0")] if WASM else []),
patch=gmp_patches,
# Sometimes ftp.gnu.org is very slow, use ftpmirror.gnu.org as a workaround.
download_url="https://ftpmirror.gnu.org/gnu/gmp/",
download_name=f"gmp-{GMP_VERSION}.tar.bz2",
)
if WASM and APPLE:
restore_env("HOST_CC", OLD_HOST_CC)
build_dependency(
name=f"mpfr-{MPFR_VERSION}",
mode="autoconf",
build_tool_args=[ENABLE_FLAG, DISABLE_FLAG, *mpfr_args, f"--with-gmp={DEPS_DIR}/install/gmp-{GMP_VERSION}"],
download_url=f"http://www.mpfr.org/mpfr-{MPFR_VERSION}/",
download_name=f"mpfr-{MPFR_VERSION}.tar.bz2",
)
if MAC_CROSS_COMPILE_INTEL:
restore_env("CC", OLD_CC)
build_dependency(
name=f"cgal-{CGAL_VERSION}",
mode="cmake",
build_tool_args=[
f"-DGMP_LIBRARIES={DEPS_DIR}/install/gmp-{GMP_VERSION}/lib/libgmp.{LIBRARY_EXT}",
f"-DGMP_INCLUDE_DIR={DEPS_DIR}/install/gmp-{GMP_VERSION}/include",
f"-DMPFR_LIBRARIES={DEPS_DIR}/install/mpfr-{MPFR_VERSION}/lib/libmpfr.{LIBRARY_EXT}",
f"-DMPFR_INCLUDE_DIR={DEPS_DIR}/install/mpfr-{MPFR_VERSION}/include",
f"-DBoost_INCLUDE_DIR={DEPS_DIR}/install/boost-{BOOST_VERSION}",
f"-DCMAKE_INSTALL_PREFIX={DEPS_DIR}/install/cgal-{CGAL_VERSION}/",
f"-DCGAL_HEADER_ONLY=On",
f"-DBUILD_SHARED_LIBS=Off",
],
download_url="https://github.com/CGAL/cgal.git",
download_name="cgal",
download_tool=download_tool_git,
revision=CGAL_VERSION,
)
if "usd" in targets:
build_dependency(
name=f"oneTBB-{TBB_VERSION}",
mode="cmake",
build_tool_args=[f"-DCMAKE_INSTALL_PREFIX={DEPS_DIR}/install/tbb-{TBB_VERSION}", f"-DTBB_TEST=OFF"],
download_url="https://github.com/oneapi-src/oneTBB",
download_name="oneTBB",
download_tool=download_tool_git,
revision=f"v{TBB_VERSION}",
)
build_dependency(
name=f"usd-{USD_VERSION}",
mode="cmake",
build_tool_args=[
f"-DCMAKE_INSTALL_PREFIX={DEPS_DIR}/install/usd-{USD_VERSION}",
f"-DBOOST_ROOT={DEPS_DIR}/install/boost-{BOOST_VERSION}",
f"-DTBB_ROOT_DIR={DEPS_DIR}/install/tbb-{TBB_VERSION}",
f"-DPXR_ENABLE_PYTHON_SUPPORT=FALSE",
f"-DPXR_ENABLE_GL_SUPPORT=FALSE",
f"-DPXR_BUILD_IMAGING=FALSE",
f"-DPXR_BUILD_TUTORIALS=FALSE",
f"-DPXR_BUILD_EXAMPLES=FALSE",
f"-DPXR_BUILD_USD_TOOLS=FALSE",
f"-DPXR_BUILD_TESTS=FALSE",
],
download_url="https://github.com/PixarAnimationStudios/USD",
download_name="USD",
download_tool=download_tool_git,
revision=f"v{USD_VERSION}",
)
if "zstd" in targets:
build_dependency(
name=f"zstd-{ZSTD_VERSION}",
mode="cmake",
build_tool_args=[
f"-DCMAKE_INSTALL_PREFIX={DEPS_DIR}/install/zstd-{ZSTD_VERSION}",
f"-DZSTD_BUILD_STATIC=ON",
f"-DZSTD_BUILD_SHARED=OFF",
f"-DCMAKE_INSTALL_LIBDIR=lib",
*MAC_CROSS_COMPILE_INTEL_ARGS,
],
cmake_dir="build/cmake/",
download_url="https://github.com/facebook/zstd",
download_name="zstd",
download_tool=download_tool_git,
revision=f"v{ZSTD_VERSION}",
)
if "rocksdb" in targets:
build_dependency(
name=f"rocksdb-{ROCKSDB_VERSION}",
mode="cmake",
build_tool_args=[
f"-DCMAKE_INSTALL_PREFIX={DEPS_DIR}/install/rocksdb-{ROCKSDB_VERSION}",
f"-DFAIL_ON_WARNINGS=Off",
f"-DWITH_TESTS=OFF",
f"-DWITH_TOOLS=OFF",
f"-DWITH_GFLAGS=OFF",
f"-DWITH_BENCHMARK_TOOLS=OFF",
f"-DWITH_CORE_TOOLS=OFF",
f"-DROCKSDB_BUILD_SHARED=Off",
f"-DCMAKE_POSITION_INDEPENDENT_CODE=On",
f"-DUSE_RTTI=On",
f"-DWITH_ZSTD=On",
f"-DPORTABLE=1",
f"-DCMAKE_PREFIX_PATH={DEPS_DIR}/install/zstd-{ZSTD_VERSION}",
*MAC_CROSS_COMPILE_INTEL_ARGS,
],
download_url="https://github.com/facebook/rocksdb",
download_name="rocksdb",
download_tool=download_tool_git,
revision=f"v{ROCKSDB_VERSION}",
)
if "qt6" in targets:
install_qt6()
cecho("Building IfcOpenShell:", GREEN)
IFCOS_DIR = os.path.join(DEPS_DIR, "build", "ifcopenshell")
if os.environ.get("NO_CLEAN", "").lower() not in {"1", "on", "true"}:
if os.path.exists(IFCOS_DIR):
shutil.rmtree(IFCOS_DIR)
os.makedirs(IFCOS_DIR, exist_ok=True)
ifcos_build_dir = os.path.join(IFCOS_DIR, "build")
os.makedirs(ifcos_build_dir, exist_ok=True)
cmake_args = [
"-DUSE_MMAP=OFF",
f"-DBUILD_EXAMPLES={OFF_ON[BUILD_EXAMPLES]}",
"-DBUILD_SHARED_LIBS=" + OFF_ON[not IFCOPENSHELL_STATIC],
"-DGLTF_SUPPORT=ON",
"-DBoost_NO_BOOST_CMAKE=On",
"-DCREATE_BUNDLE=On",
"-DADD_COMMIT_SHA=" + OFF_ON[ADD_COMMIT_SHA],
"-DVERSION_OVERRIDE=" + OFF_ON[ADD_COMMIT_SHA],
*MAC_CROSS_COMPILE_INTEL_ARGS,
]
"""Default CMake args to use for all CMake configs."""
cmake_args_prefix_path: list[str] = [
f"{DEPS_DIR}/install/boost-{BOOST_VERSION}",
f"{DEPS_DIR}/install/eigen-install-{EIGEN_VERSION}",
f"{DEPS_DIR}/install/json-{JSON_VERSION}",
]
def get_cmake_args_prefix_path(additional_paths: Sequence[str] = ()) -> list[str]:
args_prefix_path = cmake_args_prefix_path.copy()
args_prefix_path.extend(additional_paths)
prefix_path = ";".join(args_prefix_path)
if WASM:
# `emcmake` is disabling search in PATH, so we provide root paths instead.
# Provide '/' to PATH, so it will be combined with provided root paths,
# otherwise, depending on environment, it might not search the root path itself.
return [f"-DCMAKE_FIND_ROOT_PATH={prefix_path}", "-DCMAKE_PREFIX_PATH=//"]
else:
return [f"-DCMAKE_PREFIX_PATH={prefix_path}"]
if WASM:
# Boost is built by the build script so should not be found
# inside of the sysroot set by the emscriptem toolchain
cmake_args.append("-DWASM_BUILD=On")
schemas = os.environ.get("IFCOS_SCHEMAS")
if schemas:
cmake_args.append(f"-DSCHEMA_VERSIONS={schemas}")
if "cgal" in targets:
cmake_args_prefix_path.append(f"{DEPS_DIR}/install/cgal-{CGAL_VERSION}")
cmake_args_prefix_path.append(f"{DEPS_DIR}/install/gmp-{GMP_VERSION}")
cmake_args_prefix_path.append(f"{DEPS_DIR}/install/mpfr-{MPFR_VERSION}")
cmake_args.append(f"-DCGAL_WITH_GMPXX=Off")
if "occ" in targets and USE_OCCT:
cmake_args_prefix_path.append(f"{DEPS_DIR}/install/{OCCT_DIR_NAME}")
elif "occ" in targets:
# We don't support find_package for OCE.
occ_include_dir = f"{DEPS_DIR}/install/oce-{OCE_VERSION}/include/oce"
occ_library_dir = f"{DEPS_DIR}/install/oce-{OCE_VERSION}/lib"
cmake_args.extend(["-DOCC_INCLUDE_DIR=" + occ_include_dir, "-DOCC_LIBRARY_DIR=" + occ_library_dir])
if "manifold" in targets:
cmake_args_prefix_path.append(f"{DEPS_DIR}/install/manifold-{MANIFOLD_VERSION}")
cmake_args.append("-DWITH_MANIFOLD=On")
if "OpenCOLLADA" in targets:
# pcre is a dependency of OpenCOLLADA, but since we `find_package`,
# we don't need to add it explicitly here as cmake will find it from the config.
cmake_args_prefix_path.append(f"{DEPS_DIR}/install/OpenCOLLADA")
else:
cmake_args.extend(
[
f"-DCOLLADA_SUPPORT=Off",
]
)
if "libxml2" in targets:
cmake_args_prefix_path.append(f"{DEPS_DIR}/install/libxml2-{LIBXML2_VERSION}")
if "usd" in targets:
cmake_args.append("-DUSD_SUPPORT=ON")
cmake_args_prefix_path.extend(
[
f"{DEPS_DIR}/install/tbb-{TBB_VERSION}",
f"{DEPS_DIR}/install/usd-{USD_VERSION}",
]
)
if "rocksdb" in targets:
cmake_args.extend(
[
f"-DWITH_ROCKSDB=On",
f"-DWITH_ZSTD=On",
]
)
cmake_args_prefix_path.extend(
[
f"{DEPS_DIR}/install/rocksdb-{ROCKSDB_VERSION}",
f"{DEPS_DIR}/install/zstd-{ZSTD_VERSION}",
]
)
if "swig" in targets:
# `cmake_args_prefix_path` won't work on wasm
# because `find_program` in emscripten toolchain don't use `find_root_path`.
# As a workaround we provide executable path directly on all platforms.
cmake_args.append(f"-DSWIG_EXECUTABLE={DEPS_DIR}/install/swig-{SWIG_VERSION}/bin/swig")
if os.environ.get("QT_DIR"):
cmake_args_prefix_path.append(os.environ["QT_DIR"])
cmake_args.append(f"-DQT_DIR={os.environ['QT_DIR']}")
ifcos_build_args = [
f"-DBUILD_IFCGEOM={OFF_ON['IfcGeom' in targets]}",
f"-DBUILD_GEOMSERVER={OFF_ON['IfcGeomServer' in targets]}",
f"-DBUILD_CONVERT={OFF_ON['IfcConvert' in targets]}",
f"-DBUILD_BONSAIVIEWER={OFF_ON['BonsaiViewer' in targets]}",
f"-DCMAKE_INSTALL_PREFIX={DEPS_DIR}/install/ifcopenshell",
]
if not WASM and (
"BonsaiViewer" in targets
or not explicit_targets
or {"IfcGeom", "IfcConvert", "IfcGeomServer", "BonsaiViewer"} & set(explicit_targets)
):
logger.info("\rConfiguring executables...")
exec_args = [
*ifcos_build_args,
f"-DBUILD_IFCPYTHON=OFF",
]
run_cmake("", exec_args + cmake_args + get_cmake_args_prefix_path(), cmake_dir=CMAKE_DIR, cwd=ifcos_build_dir)
logger.info("\rBuilding executables... ")
run([make, f"-j{IFCOS_NUM_BUILD_PROCS}", "VERBOSE=1"], cwd=ifcos_build_dir)
run([make, "install/strip" if BUILD_CFG == "Release" else "install"], cwd=ifcos_build_dir)
if "IfcOpenShell-Python" in targets:
wrapper_ldflags = ""
if platform.system() == "Darwin":
# On OSX the actual Python library is not linked against.
wrapper_ldflags = "-Wl,-undefined,dynamic_lookup"
def compile_python_wrapper(
python_version: str,
python_include: str | None = None,
python_executable: str | None = None,
python_path: Path | None = None,
) -> str | None:
"""
:return: Path to module dir if ``python_executable`` was provided, otherwise ``None``.
"""
assert bool(python_path) ^ bool(python_include)
logger.info(f"\rConfiguring python {python_version} wrapper...")
if python_path:
# We couldn't just prefix PATH and have to provide all variables explicitly,
# see ifcwrap/cmake for the details.
python_executable = (Path(python_path) / "bin" / "python3").__str__()
python_include = run(
[
python_executable,
"-c",
"import sysconfig; print(sysconfig.get_config_var('INCLUDEPY'))",
]
)
assert python_include
old_ldflags = os.environ["LDFLAGS"]
if wrapper_ldflags:
os.environ["LDFLAGS"] = f"{old_ldflags} {wrapper_ldflags}"
try:
run_cmake(
"",
ifcos_build_args
+ [
"-DBUILD_IFCPYTHON=ON",
]
+ cmake_args
+ get_cmake_args_prefix_path()
+ [
*([f"-DPYTHON_EXECUTABLE={python_executable}"] if python_executable else []),
# Needed because pyodide is expecting setup.py to be in the root.
*([f"-DPYTHON_MODULE_INSTALL_DIR={REPO_PATH}"] * WASM),
f"-DPYTHON_INCLUDE_DIR={python_include}",
f"-DCMAKE_INSTALL_PREFIX={DEPS_DIR}/install/ifcopenshell/tmp",
"-DUSERSPACE_PYTHON_PREFIX="
+ ["Off", "On"][os.environ.get("PYTHON_USER_SITE", "").lower() in {"1", "on", "true"}],
],
cmake_dir=CMAKE_DIR,
cwd=ifcos_build_dir,
)
finally:
os.environ["LDFLAGS"] = old_ldflags
logger.info(f"\rBuilding python {python_version} wrapper... ")
run([make, f"-j{IFCOS_NUM_BUILD_PROCS}", "ifcopenshell_wrapper", "VERBOSE=1"], cwd=ifcos_build_dir)
run([make, "install/local"], cwd=os.path.join(ifcos_build_dir, "ifcwrap"))
if python_executable:
run([python_executable, "-m", "ensurepip"])
run([python_executable, "-m", "pip", "install", "--user", "numpy", "typing_extensions"])
module_dir = run(
[python_executable, "-c", "import inspect, ifcopenshell; print(inspect.getfile(ifcopenshell))"]
)
# Use just the last line is used,
# because output might contain warning like `No stream support: No module named 'lark'`.
module_dir = module_dir.strip().splitlines()[-1]
module_dir = os.path.dirname(module_dir)
if platform.system() != "Darwin":
if BUILD_CFG == "Release":
for so in glob.glob(os.path.join(module_dir, "*.so")):
if WASM:
run(["wasm-strip", so, "-k", "dylink.0"])
elif os.path.basename(so).startswith("_ifcopenshell_wrapper"):
# TODO: This symbol name depends on the Python version?
run([strip, "-s", "-K", "PyInit__ifcopenshell_wrapper", so], cwd=module_dir)
else:
run([strip, "--strip-unneeded", so], cwd=module_dir)
return module_dir
if WASM:
compile_python_wrapper(
run(["pyodide", "config", "get", "python_version"]),
run(["pyodide", "config", "get", "python_include_dir"]),
)
# Copy setup.py where pyodide build system expects it.
shutil.copy(REPO_PATH / "pyodide" / "setup.py", REPO_PATH)
# Empty pyproject so it's contents won't affect the resulting wheel
# otherwise the wheel will use version and dependencies from toml, not setup.py.
(REPO_PATH / "pyproject.toml").write_text("")
elif USE_CURRENT_PYTHON_VERSION:
python_info = sysconfig.get_paths()
compile_python_wrapper(platform.python_version(), python_info["include"], sys.executable)
else:
for python_version in PYTHON_VERSIONS:
python_path = INSTALL_DIR / f"python-{python_version}"
module_dir = compile_python_wrapper(python_version, python_path=python_path)
assert module_dir
# Not sure why, but added after reading this in the logs
# cp: /Users/runner/work/IfcOpenShell/IfcOpenShell/build/Darwin/x86_64/10.15/install/ifcopenshell/python-3.9.11: No such file or directory
# D'oh this was just due to a missing f-string f but doesn't hurt to keep it in.
run(["mkdir", "-p", os.path.join(DEPS_DIR, "install", "ifcopenshell")])
dest = os.path.join(DEPS_DIR, "install", "ifcopenshell", f"python-{python_version}")
if os.path.exists(dest):
shutil.rmtree(dest)
run([cp, "-R", module_dir, dest])
logger.info("\rBuilt IfcOpenShell...\n\n")
Reference in New Issue View Git Blame Copy Permalink