Compare commits

...

71 Commits

Author SHA1 Message Date
Andrej730 e9d003c22f clear cgal cache 2025-10-30 19:20:37 +05:00
Andrej730 2e71e002fc try out just intel 2025-10-30 18:54:31 +05:00
Andrej730 40ed2fe3ab build-all.py - remove unused wasm flags
`-Wl,-undefined,suppress` have no meaning in emscripten and they are just ignored. And emscripten is ignoring undefined symbols by default without any flags.
2025-10-30 18:53:56 +05:00
Andrej730 379fe632a3 build-all.py - switch to dynamic_lookup
1) `-undefined,suppress` is deprecated and linking is sending warnings about it and it seems to be superseded by `dynamic_lookup`
2) `-flat_namespace` is not compatible with GMP when building for intel, for some reason and results in horrible errors during linking:

Example error:
```
ld: branch8 out of range 366651 in ___gmpn_add_nc
final section layout:
    __TEXT               addr=0x00000000, size=0x02d09000, fileOffset=0x00000000
        __text           addr=0x00001400, size=0x0264a5e4, fileOffset=0x00001400
        __stubs          addr=0x0264b9e4, size=0x0000423c, fileOffset=0x0264b9e4
        __stub_helper    addr=0x0264fc20, size=0x00006c58, fileOffset=0x0264fc20
        __gcc_except_tab addr=0x02656878, size=0x002efc04, fileOffset=0x02656878
        __const          addr=0x02946480, size=0x001cdc18, fileOffset=0x02946480
        __cstring        addr=0x02b140a0, size=0x0014324b, fileOffset=0x02b140a0
        __unwind_info    addr=0x02c572ec, size=0x0008d2d0, fileOffset=0x02c572ec
        __eh_frame       addr=0x02ce45c0, size=0x00024a30, fileOffset=0x02ce45c0
    __DATA_CONST         addr=0x02d09000, size=0x0022b000, fileOffset=0x02d09000
        __got            addr=0x02d09000, size=0x00001dc8, fileOffset=0x02d09000
        __mod_init_func  addr=0x02d0adc8, size=0x00000690, fileOffset=0x02d0adc8
        __const          addr=0x02d0b460, size=0x00227fd0, fileOffset=0x02d0b460
    __DATA               addr=0x02f34000, size=0x000c3000, fileOffset=0x02f34000
        __la_symbol_ptr  addr=0x02f34000, size=0x000056a0, fileOffset=0x02f34000
        __data           addr=0x02f396a0, size=0x0005f7b8, fileOffset=0x02f396a0
        __thread_vars    addr=0x02f98e58, size=0x000006d8, fileOffset=0x02f98e58
        __thread_ptrs    addr=0x02f99530, size=0x00000178, fileOffset=0x02f99530
        __thread_data    addr=0x02f996b0, size=0x000000d8, fileOffset=0x02f996b0
        __thread_bss     addr=0x02f99790, size=0x00018850, fileOffset=0x02f99790
        __bss            addr=0x02fb1fe0, size=0x0003b0c0, fileOffset=0x00000000
        __common         addr=0x02fed0a0, size=0x00009098, fileOffset=0x00000000
    __LINKEDIT           addr=0x02ff7000, size=0x0148e000, fileOffset=0x02fb2000
clang: error: linker command failed with exit code 1 (use -v to see invocation)
```
2025-10-30 18:53:55 +05:00
Andrej730 2ecaee9024 build-all.py - add VERBOSE=1 to the wrapper make command 2025-10-30 18:53:55 +05:00
Andrej730 f9a3c9a994 build-all.py - fix missing mac cross compilation host args in gmp
I guess something has changed in gmp 6.3.0 and this issue become more apparent.

Example error:
```
tmp-cnd_sub_n.s:133:14: error: tmp-cnd_add_n.sbrackets expression not supported on this target:
130 stp x8, x9, [x1,#16]
             ^
:2tmp-cnd_sub_n.s: :error: 134invalid instruction mnemonic 'bic':
6 bic x7, x13, x0:
error:  invalid instruction mnemonic 'cset'^
~Lrt: cset x0, cc~

     ^~~tmp-cnd_add_n.s~:
131:2: error: invalid instruction mnemonic 'adcs'
 adcs x8, x10, x6
 ^~~~
tmp-cnd_add_n.s:132:2: error: invalid instruction mnemonic 'adcs'
 adcs x9, x11, x7
 ^~~~
tmp-cnd_add_n.s:133:14: error: brackets expression not supported on this target
 stp x8, x9, [x1,#16]
             ^
tmp-cnd_add_n.s:134:6: error: invalid instruction mnemonic 'cset'
Lrt: cset x0, cs
     ^~~~
```
2025-10-29 14:38:24 +05:00
Andrej730 77acbff50b black . 2025-10-29 13:25:26 +05:00
Ryan Schultz 3f2cad0490 Add suppress_zero_inches parameter to imperial_length formatting function 2025-10-28 22:24:40 -05:00
Andrej730 5ec974ff94 Exclude build-all.py from main ruff check 2025-10-28 18:20:43 +05:00
Andrej730 2c585b649a Fix RUFF037 (redundant quotes) 2025-10-28 18:20:14 +05:00
Andrej730 36c28f3953 Mention in docs wasm-wheels as the main wheels source for pyodide builds 2025-10-28 18:20:13 +05:00
Andrej730 d1763237d2 ifcopenshell docs make - add server target 2025-10-28 18:20:13 +05:00
Andrej730 e8544d78ee ci.yml - note on rtti in rocksdb 2025-10-28 17:17:20 +05:00
Andrej730 e715a9467d ci.yml - exclude PYTHON_LIBRARY as redundant 2025-10-28 17:17:20 +05:00
Andrej730 1f8c12a4c8 build_pyodide - test wasm wheel working 2025-10-28 17:17:20 +05:00
Andrej730 89a6eac535 build-all.py - access pyodide for necessary variables, simplify debug build env 2025-10-28 17:17:20 +05:00
Andrej730 d1ffb329c8 pyodide/setup.py - fix missing ifcopenshell subpackages 2025-10-28 17:17:20 +05:00
Andrej730 4396126a43 build-all.py - fix for upcoming pyodide changes (cmake INIT flags) 2025-10-28 17:17:20 +05:00
Andrej730 7448feed44 build-all.py - reinitialize wasm cxx flags from cflags to accomodate issue upstream 2025-10-28 17:17:19 +05:00
Andrej730 6bc4046b17 ifcwrap cmake - fix issue with debug build after migration to FindPython
In FindPython variable was renamed to Python_LIBRARY_DEBUG
2025-10-28 17:17:19 +05:00
Andrej730 1a83e52935 ifcwrap cmake - avoid using deprecated swig_link_libraries 2025-10-28 17:17:19 +05:00
Andrej730 f16d42ba14 ifcwrap cmake - use usual name for wrapper target 2025-10-28 17:17:19 +05:00
Andrej730 75135fe45d cache_dependencies.py - fix tarfile deprecation warnings in 3.12-3.13 2025-10-28 17:17:19 +05:00
Andrej730 de89ab31cf ifcwrap cmake - move DEBUG_POSTFIX logic to one place 2025-10-28 17:17:19 +05:00
Andrej730 c8a4f1bbd2 cmake - build without Python libraries on Unix 2025-10-28 17:17:19 +05:00
Andrej730 5be1858767 ifcwrap cmake - accomodate bug in cmake makefiles generator 2025-10-28 17:17:19 +05:00
Andrej730 ace1db739f build-all - use shutil.which instead of custom function
To make it more idiomatic and custom function also was misinterpreting folders with the same name as existing executables.
2025-10-28 17:17:19 +05:00
Andrej730 80504310b6 build_pyodide - use ccache 2025-10-28 17:17:19 +05:00
Andrej730 a4dae9650f build_rocky - switch to main branch ccache 2025-10-28 17:17:18 +05:00
Andrej730 c27a995480 build_pyodide - use prebuilt pyodide build env 2025-10-28 17:17:18 +05:00
Andrej730 5b4c54abd2 build-all.py - switch to gmp ftpmirror
previous url is inaccessible
2025-10-28 17:17:18 +05:00
Andrej730 57d0beedee build_pyodide - display logs in github action output 2025-10-28 17:17:18 +05:00
Andrej730 d0ba675b1b build_pyodide - upload build logs for debugging 2025-10-28 17:17:18 +05:00
Andrej730 f4f2fa9535 build_pyodide - pack cached dependencies 2025-10-28 17:17:18 +05:00
Andrej730 67707914b9 build_osx - remove setup-python
as it's already preinstalled on the image
2025-10-28 17:17:18 +05:00
Andrej730 12766d61d4 build-all.py - print BUILD_DIR 2025-10-28 17:17:18 +05:00
Andrej730 34374cba41 build_pyodide - no need to adjust version
since it's now inferred automatically
2025-10-28 17:17:18 +05:00
Andrej730 f8bb04eae3 build-all.py - accomodate pyodide build system automatically
as it's the main way to build the wasm package
2025-10-28 17:17:18 +05:00
Andrej730 aee8cbe479 build_pyodide.yaml - move build_pyodide to a separate script 2025-10-28 17:17:18 +05:00
Andrej730 cb36e10279 Add notes for pyodide building process 2025-10-28 17:17:18 +05:00
Andrej730 7ca44b5677 build-all.py wasm - move SIDE_MODULE=2 to cmake
Next to `EXPORTED_FUNCTIONS` and avoid possible issues with pyodide in the future (pyodide is using `SIDE_MODULE=1`, but currently not providing it libraries of type `MODULE`, only for shared ones).
2025-10-28 17:17:17 +05:00
Andrej730 a6921b83a1 ifcwrap cmake - provide correct suffix for wasm python extension 2025-10-28 17:17:17 +05:00
Andrej730 c3c94b9c59 pyodide setup.py - add dependencies
so micropip would be able to gather them automatically
2025-10-28 17:17:17 +05:00
Andrej730 576ef336fc pyodide setup.py - add a note on not switching to pyproject.toml 2025-10-28 17:17:17 +05:00
Andrej730 b1fb541912 pyodide setup.py - ensure we have correct pyodide_2025_0_wasm32 suffix
Adding extension, but avoiding adding sources as we're not actually building.
As a result build will have a correct suffix `pyodide_2025_0_wasm32.whl` that's pyodide is currently using and this is pyodide is plan to use when it's going to release to PyPI, see https://peps.python.org/pep-0783/
2025-10-28 17:17:17 +05:00
Andrej730 6345b052da build-all.py - add a note about alternative gmp mirror 2025-10-28 17:17:17 +05:00
Andrej730 061554d921 cmake - deduplicate ccache set_property code 2025-10-28 17:17:17 +05:00
Andrej730 bcaa412f4b build-all.py - don't pass side module flags as usual flags
As they're parsed by emcmake automatically https://github.com/pyodide/pyodide-build/blob/bdd0b1422064de6d92264a08c599e85001a28567/pyodide_build/tools/cmake/Modules/Platform/Emscripten.cmake#L105-L107
2025-10-28 17:17:17 +05:00
Andrej730 9f6536b263 build-all.py - option to build wasm outside pyodide for debugging 2025-10-28 17:17:17 +05:00
Andrej730 226581f76e build-all.py - provide exported functions to target explicitly
Otherwise it broke compilation in some cases when cmake would try to test that linker is working.
2025-10-28 17:17:17 +05:00
Andrej730 e833d0cd13 build-all - wasm to use packages instead of providing includes/libs paths explicitly
More complicated than I anticipated😅

Details
https://discourse.cmake.org/t/provide-absolute-paths-to-dependencies-with-cmake-find-root-path-search-during-cross-compilation-wasm/15241
2025-10-28 17:17:16 +05:00
Andrej730 ed3c95d9d7 build-all.py - move unsupported dependencies to script 2025-10-28 17:17:16 +05:00
Andrej730 7a4823a1bc build-deps - use nuget to download ccache 2025-10-28 17:17:16 +05:00
Andrej730 80c0932174 build-all.py - print IFCOS_SCHEMAS 2025-10-28 17:17:16 +05:00
Andrej730 73a238e7b0 build-all.py - document required wasm variables 2025-10-28 17:17:16 +05:00
Andrej730 b95900f154 build-all.py - fix issue building wasm on apple device 2025-10-28 17:17:16 +05:00
Andrej730 da66e3c908 build-all.py - remove --host none from wasm args
As it's now obsolete and producing warnings `configure: WARNING: the "none" host is obsolete, use --disable-assembly`.
2025-10-28 17:17:16 +05:00
Andrej730 e04cc47022 build-all.py - bump freetype to avoid issues on cmake 4+ 2025-10-28 17:17:16 +05:00
Andrej730 ce427698d9 build-all.py - add note on --without-xxx option 2025-10-28 17:17:16 +05:00
Andrej730 d3beae1374 Add Python 3.14 builds 2025-10-28 17:17:16 +05:00
Andrej730 389d481364 build-deps - set NUGET_EXE from PATH if it's available 2025-10-28 17:17:15 +05:00
Andrej730 3efd88b0d6 build-all - deduce python components from python path 2025-10-28 17:17:15 +05:00
Andrej730 0a5388f386 ifcwrap cmake - migrate to FindPython
Also simplify run-cmake to just prefix the path, instead of providing 3 paths explicitly.
2025-10-28 17:17:15 +05:00
Andrej730 71fa002707 build-deps - support overriding PYTHON_VERSION 2025-10-28 17:17:15 +05:00
Andrej730 dd80115aa2 build-deps - download Python using nuget 2025-10-28 17:17:15 +05:00
Andrej730 bc66710fec build-deps - download nuget
to later download python using it
2025-10-28 17:17:15 +05:00
Andrej730 921241fbdd build-deps - include ccache for convenience 2025-10-28 17:17:15 +05:00
Andrej730 44b9dd64f6 build_win - extend cache size 2025-10-28 17:17:15 +05:00
Andrej730 bfa77255a5 build_win - use fork to resolve cache misses and duplicated cache 2025-10-28 17:17:14 +05:00
Andrej730 b125c651a8 build_win - test ccache 2025-10-28 17:17:14 +05:00
Thomas Krijnen eb1c81514b Check for attribute nil #7276 #7291 2025-10-28 10:51:31 +01:00
30 changed files with 843 additions and 336 deletions
+7 -11
View File
@@ -14,10 +14,6 @@ jobs:
runner: macos-14
arch: x64
oldarch:
- os: macos
runner: macos-14
arch: arm64
oldarch: m1
steps:
- name: Checkout Repository
@@ -34,11 +30,6 @@ jobs:
lfs: true
token: ${{ secrets.BUILD_REPO_TOKEN }}
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.x'
- name: Install Dependencies
run: |
brew update
@@ -52,6 +43,7 @@ jobs:
- name: Unpack Dependencies
run: |
rm -rf ./build/*/*/*/install/*gmp* ./build/*/*/*/install/*cgal*
install_root=$(find ./build -maxdepth 4 -type d -name install 2>/dev/null | head -n 1 || true)
[ -n "$install_root" ] && find "$install_root" -type f -name 'cache-*.tar.gz' -maxdepth 1 -exec tar -xzf {} -C "$install_root" \; || true
@@ -72,12 +64,16 @@ jobs:
# We don't use gmpcxx, but it comes preinstalled on macos runner as arm64 bottle
# and CGAL detects it and breaks cross compilation.
brew uninstall --ignore-dependencies gmp
# Otherwise Python will fallback to use arm64 `pkg-config`,
# will pick up arm64 libraries ('zstd' in particular),
# and break the build.
/usr/local/bin/brew install pkg-config
# Required by Python.
/usr/local/bin/brew install gettext openssl
fi
set -o pipefail
CXXFLAGS="-O3" CFLAGS="-O3 ${DARWIN_C_SOURCE}" ADD_COMMIT_SHA=1 BUILD_CFG=Release \
python3 ./nix/build-all.py -v --diskcleanup ${MAC_INTEL} \
IFCOS_SCHEMAS=4 CXXFLAGS="-O3" CFLAGS="-O3 ${DARWIN_C_SOURCE}" ADD_COMMIT_SHA=1 BUILD_CFG=Release \
python3 ./nix/build-all.py -v -py-313 --diskcleanup ${MAC_INTEL} \
| tee build.log
- name: Upload Build Logs
+55 -21
View File
@@ -14,33 +14,67 @@ jobs:
submodules: recursive
path: IfcOpenShell
- name: Checkout Pyodide
- name: Checkout Build Repository
uses: actions/checkout@v3
with:
submodules: recursive
repository: pyodide/pyodide
ref: '0.28.0a3'
repository: IfcOpenShell/build-outputs
path: ifcopenshell_build
ref: wasm
lfs: true
token: ${{ secrets.BUILD_REPO_TOKEN }}
path: pyodide
- name: Unpack Dependencies
run: |
cd ifcopenshell_build
python ../IfcOpenShell/pyodide/cache_dependencies.py unpack
- name: ccache
uses: hendrikmuhs/ccache-action@v1.2
with:
key: ubuntu-22.04-${{ runner.arch }}
- name: Build
run: |
VERSION=`cat IfcOpenShell/VERSION`
sed -i s/0.8.0/$VERSION/g IfcOpenShell/pyodide/meta.yaml
sed -i s/0.8.0/$VERSION/g IfcOpenShell/pyodide/setup.py
echo '#!/usr/bin/bash' > script.sh
echo 'cd pyodide' >> script.sh
echo 'make' >> script.sh
echo 'cd ..' >> script.sh
echo 'mkdir -p packages/ifcopenshell' >> script.sh
echo 'cp IfcOpenShell/pyodide/meta.yaml packages/ifcopenshell' >> script.sh
echo 'PYODIDE_ROOT=/src/pyodide \' >> script.sh
echo 'PATH=/src/pyodide/emsdk/emsdk:/src/pyodide/emsdk/emsdk/node/22.16.0_64bit/bin:/src/pyodide/emsdk/emsdk/upstream/emscripten:$PATH \' >> script.sh
echo 'pyodide build-recipes ifcopenshell --install' >> script.sh
chmod +x script.sh
sed -i s/--tty// pyodide/run_docker
pyodide/run_docker ./script.sh
mv dist/ifcopenshell-$VERSION-py3-none-any.whl dist/ifcopenshell-$VERSION+${GITHUB_SHA:0:7}-cp313-cp313-emscripten_4_0_9_wasm32.whl
./IfcOpenShell/pyodide/build_pyodide.sh
FILE=`echo dist/ifcopenshell-*.whl`
NEW_FILE=`echo $FILE | sed "s/-/+${GITHUB_SHA:0:7}-/2"`
mv $FILE $NEW_FILE
- name: Upload Build Logs
if: always()
uses: actions/upload-artifact@v4
with:
name: build-logs-pyodide
path: |
ifcopenshell_build/*/*/logs/*.log
retention-days: 30
- name: Run wheel tests
run: |
cp -r IfcOpenShell/pyodide/test test
# venv set up in build_pyodide.sh.
source .venv/bin/activate
uv pip install pytest-pyodide
PYODIDE_ROOT_DIST=`pyodide config get pyodide_root`/dist
# `pytest-pyodide` requires pyodide in 'pyodide' directory in cwd, when running `pytest`.
cp -r $PYODIDE_ROOT_DIST test/pyodide
cp dist/ifcopenshell-*.whl test/pyodide
cd test
pytest --capture=no
- name: Pack Dependencies
run: |
cd ifcopenshell_build
python ../IfcOpenShell/pyodide/cache_dependencies.py pack
- name: Commit and Push Changes to Build Repository
run: |
cd ifcopenshell_build
git config user.name "IfcOpenBot"
git config user.email "ifcopenbot@ifcopenshell.org"
git add */*/install/cache-*.tar.gz
git commit -m "Update build artifacts [skip ci]" || echo "No changes to commit"
git push || echo "Push failed"
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v4
+2 -2
View File
@@ -48,8 +48,8 @@ jobs:
[ -n "$install_root" ] && find "$install_root" -type f -name 'cache-*.tar.gz' -maxdepth 1 -exec tar -xzf {} -C "$install_root" \; || true
- name: ccache
# Using fork to support Rocky.
uses: Andrej730/ccache-action@main
# Latest release (1.2.19) doesn't support Rocky, so using a specific commit.
uses: hendrikmuhs/ccache-action@eca11c308176d48942455b9e5b1b70ff6950a778
with:
key: ubuntu-22.04-${{ runner.arch }}-rockylinux8
+2 -2
View File
@@ -48,8 +48,8 @@ jobs:
[ -n "$install_root" ] && find "$install_root" -type f -name 'cache-*.tar.gz' -maxdepth 1 -exec tar -xzf {} -C "$install_root" \; || true
- name: ccache
# Using fork to support Rocky.
uses: Andrej730/ccache-action@main
# Latest release (1.2.19) doesn't support Rocky, so using a specific commit.
uses: hendrikmuhs/ccache-action@eca11c308176d48942455b9e5b1b70ff6950a778
with:
key: ubuntu-22.04-${{ runner.arch }}-rockylinux8
+11 -49
View File
@@ -9,7 +9,6 @@ jobs:
strategy:
fail-fast: false
matrix:
python: ['3.9.11', '3.10.3', '3.11.8', '3.12.1', '3.13.0']
arch: ['x64']
steps:
- name: Checkout Repository
@@ -30,14 +29,6 @@ jobs:
run: |
choco install -y sed 7zip.install awscli
- name: Install Python
run: |
$installer = "python-${{ matrix.python }}-amd64.exe"
$url = "https://www.python.org/ftp/python/${{ matrix.python }}/$installer"
Invoke-WebRequest -Uri $url -OutFile $installer
Start-Process -Wait -FilePath .\$installer -ArgumentList '/quiet InstallAllUsers=0 PrependPath=0 Include_test=0 TargetDir=C:\Python\${{ matrix.python }}'
Remove-Item .\$installer
- name: Unpack Dependencies
run: |
cd _deps-vs2022-x64-installed
@@ -45,21 +36,21 @@ jobs:
7z x $_.FullName
}
- name: Run Build Script
- name: ccache
# Use fork to resolve cache misses / duplicated cache entries on Windows.
uses: Andrej730/ccache-action@main
with:
key: win-${{ matrix.arch }}
# Windows ccache needs ~1GB
# and with default 500MB some cache gets deleted, leading to misses.
max-size: 5000MB
- name: Run Build Script And Pack .zip Archives
shell: cmd
run: |
setlocal EnableDelayedExpansion
SET PYTHON_VERSION=${{ matrix.python }}
for /f "tokens=1,2,3 delims=." %%a in ("%PYTHON_VERSION%") do (
set PY_VER_MAJOR_MINOR=%%a%%b
)
call "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\Build\vcvars64.bat"
SET IFCOS_INSTALL_PYTHON=FALSE
cd win
echo y | call build-deps.cmd vs2022-x64 Release
SET PYTHONHOME=C:\Python\${{ matrix.python }}
call run-cmake.bat vs2022-x64 -DENABLE_BUILD_OPTIMIZATIONS=On -DGLTF_SUPPORT=ON -DADD_COMMIT_SHA=ON -DVERSION_OVERRIDE=ON
call install-ifcopenshell.bat vs2022-x64 Release
python build-all-win.py
- name: Pack Dependencies
run: |
@@ -81,35 +72,6 @@ jobs:
git commit -m "Update build artifacts [skip ci]" || echo "No changes to commit"
git push || echo "Push failed"
- name: Package .zip Archives
run: |
$VERSION = 'v' + ((Get-Content VERSION).Trim())
$SHA = ${env:GITHUB_SHA}.Substring(0, 7)
$OUTPUT_DIR = "$env:USERPROFILE\output"
New-Item -ItemType Directory -Force -Path $OUTPUT_DIR
if ("${{ matrix.python }}" -eq "3.9.11") {
# only for the first python version the executables are assembled for upload
cd _installed-vs2022-x64/bin
Get-ChildItem -Path . | ForEach-Object {
echo $_
$exe = $_.Name
$baseName = $exe.Substring(0, $exe.Length - 4)
$zipName = "${baseName}-$VERSION-$SHA-win64.zip"
7z a $zipName $exe
}
mv *.zip $OUTPUT_DIR
}
$pyVersion = "${{ matrix.python }}"
$pyVersionMajor = ($pyVersion -split '\.')[0..1] -join ''
cd C:\Python\${{ matrix.python }}\Lib\site-packages
Remove-Item -Recurse -Force ifcopenshell\__pycache__ -ErrorAction SilentlyContinue
Get-ChildItem -Path ifcopenshell -Filter "*.pyc" -Recurse | Remove-Item -Force
$zipName = "ifcopenshell-python-$pyVersionMajor-$VERSION-$SHA-win64.zip"
7z a $zipName ifcopenshell
mv $zipName $OUTPUT_DIR
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v4
with:
+6 -4
View File
@@ -26,6 +26,7 @@ jobs:
curl -LsSf https://astral.sh/uv/install.sh | sh
uv tool install ruff
uv tool install black
uv tool install poethepoet
# black doesn't catch all syntax errors, so we check them explicitly.
- name: Check syntax errors
@@ -40,15 +41,16 @@ jobs:
- name: Black formatter
id: black
run: |
uvx black --diff --check .
black --diff --check .
continue-on-error: true
- name: Ruff check
id: ruff
run: |
uvx ruff check
# It's actually Python 3.6, but ruff only supports 3.7+, but it should do.
uvx ruff check nix/build-all.py --target-version py37
ERROR=0
poe ruff-main || ERROR=1
poe ruff-old || ERROR=1
exit $ERROR
continue-on-error: true
- name: Final check
@@ -24,7 +24,7 @@ jobs:
strategy:
fail-fast: false
matrix:
pyver: [py39, py310, py311, py312, py313]
pyver: [py39, py310, py311, py312, py313, py314]
config:
- {
name: "Windows 64bit",
+1 -1
View File
@@ -19,7 +19,7 @@ jobs:
strategy:
fail-fast: false
matrix:
pyver: [py39, py310, py311, py312, py313]
pyver: [py39, py310, py311, py312, py313, py314]
config:
- {
name: "Windows 64bit",
+3 -2
View File
@@ -74,7 +74,9 @@ jobs:
with:
key: ubuntu-22.04-${{ runner.arch }}
# rocksdb on debian distros misses RTTI?
# RTTI is only enabled by default in Debug builds of rocksdb.
# Distros are using Release builds, so we're compiling it ourselves with RTTI forced on.
# https://github.com/facebook/rocksdb/blob/a3aa44a7167b8336f9bc15c8aba063260268ff68/CMakeLists.txt#L433
- name: build rocksdb
run: |
git clone https://github.com/facebook/rocksdb --branch v9.11.2
@@ -112,7 +114,6 @@ jobs:
-DOCC_LIBRARY_DIR=/usr/lib/x86_64-linux-gnu \
-DPYTHON_EXECUTABLE:FILEPATH=${{ env.pythonLocation }}/bin/python \
-DPYTHON_INCLUDE_DIR:PATH=${{ env.pythonLocation }}/include/python3.11 \
-DPYTHON_LIBRARY:FILEPATH=${{ env.pythonLocation }}/lib/libpython3.11.so \
-DCOLLADA_SUPPORT=Off \
-DUSE_MMAP=On \
"-DSCHEMA_VERSIONS=2x3;4;4x3_add2" \
+1 -1
View File
@@ -52,7 +52,7 @@ Contents
| [ifcdiff](https://docs.ifcopenshell.org/ifcdiff.html) | Compare changes between IFC models | LGPL-3.0-or-later | [![PyPI](https://img.shields.io/pypi/v/ifcdiff?label=PyPI&color=006dad)](https://pypi.org/project/ifcdiff/) |
| [ifcfm](https://docs.ifcopenshell.org/ifcfm.html) | Extract IFC data for FM handover requirements | LGPL-3.0-or-later | [![PyPI](https://img.shields.io/pypi/v/ifcfm?label=PyPI&color=006dad)](https://pypi.org/project/ifcfm/) |
| [ifcmax](https://docs.ifcopenshell.org/ifcmax.html) | Historic extension for IFC support in 3DS Max | LGPL-3.0-or-later\* | [![Official](https://img.shields.io/badge/IfcOpenShell.org-Download-70ba35)](https://docs.ifcopenshell.org/ifcmax.html)
| [ifcopenshell-python](https://docs.ifcopenshell.org/ifcopenshell-python.html) | Python library for IFC manipulation | LGPL-3.0-or-later\* | [![Official](https://img.shields.io/badge/IfcOpenShell.org-Download-70ba35)](https://docs.ifcopenshell.org/ifcopenshell-python/installation.html) [![GitHub](https://img.shields.io/github/v/release/ifcopenshell/ifcopenshell?filter=ifcopenshell-python-*&label=GitHub&color=f6f8fa)](https://github.com/IfcOpenShell/IfcOpenShell/releases?q=ifcopenshell-python&expanded=true) [![PyPI](https://img.shields.io/pypi/v/ifcopenshell?label=PyPI&color=006dad)](https://pypi.org/project/ifcopenshell/) [![Anaconda](https://img.shields.io/conda/vn/conda-forge/ifcopenshell?label=Anaconda&color=43b02a)](https://anaconda.org/conda-forge/ifcopenshell) [![Anaconda](https://img.shields.io/conda/vn/ifcopenshell/ifcopenshell?label=Anaconda-Unstable&color=43b02a)](https://anaconda.org/ifcopenshell/ifcopenshell) [![Docker](https://img.shields.io/docker/pulls/aecgeeks/ifcopenshell?label=Docker&color=1D63ED)](https://hub.docker.com/r/aecgeeks/ifcopenshell) [![AUR](https://img.shields.io/aur/version/ifcopenshell?label=AUR&color=1793d1)](https://aur.archlinux.org/packages/ifcopenshell) [![AUR Unstable](https://img.shields.io/aur/version/ifcopenshell-git?label=AUR-Unstable&color=1793d1)](https://aur.archlinux.org/packages/ifcopenshell-git) |
| [ifcopenshell-python](https://docs.ifcopenshell.org/ifcopenshell-python.html) | Python library for IFC manipulation | LGPL-3.0-or-later\* | [![Official](https://img.shields.io/badge/IfcOpenShell.org-Download-70ba35)](https://docs.ifcopenshell.org/ifcopenshell-python/installation.html) [![GitHub](https://img.shields.io/github/v/release/ifcopenshell/ifcopenshell?filter=ifcopenshell-python-*&label=GitHub&color=f6f8fa)](https://github.com/IfcOpenShell/IfcOpenShell/releases?q=ifcopenshell-python&expanded=true) [![PyPI](https://img.shields.io/pypi/v/ifcopenshell?label=PyPI&color=006dad)](https://pypi.org/project/ifcopenshell/) [![Anaconda](https://img.shields.io/conda/vn/conda-forge/ifcopenshell?label=Anaconda&color=43b02a)](https://anaconda.org/conda-forge/ifcopenshell) [![Anaconda](https://img.shields.io/conda/vn/ifcopenshell/ifcopenshell?label=Anaconda-Unstable&color=43b02a)](https://anaconda.org/ifcopenshell/ifcopenshell) [![Docker](https://img.shields.io/docker/pulls/aecgeeks/ifcopenshell?label=Docker&color=1D63ED)](https://hub.docker.com/r/aecgeeks/ifcopenshell) [![AUR](https://img.shields.io/aur/version/ifcopenshell?label=AUR&color=1793d1)](https://aur.archlinux.org/packages/ifcopenshell) [![AUR Unstable](https://img.shields.io/aur/version/ifcopenshell-git?label=AUR-Unstable&color=1793d1)](https://aur.archlinux.org/packages/ifcopenshell-git) [Pyodide WASM Wheels](https://github.com/IfcOpenShell/wasm-wheels#pyodide-test-wheels) |
| [ifcpatch](https://docs.ifcopenshell.org/ifcpatch.html) | Utility to run pre-packaged scripts to manipulate IFCs | LGPL-3.0-or-later | [![PyPI](https://img.shields.io/pypi/v/ifcpatch?label=PyPI&color=006dad)](https://pypi.org/project/ifcpatch/) |
| [ifcsverchok](https://docs.ifcopenshell.org/ifcsverchok.html) | Blender Add-on for visual node programming with IFC | GPL-3.0-or-later | [![GitHub Unstable](https://img.shields.io/github/v/release/ifcopenshell/ifcopenshell?filter=ifcsverchok-*.*.*.*&label=GitHub-Unstable&color=f6f8fa)](https://github.com/IfcOpenShell/IfcOpenShell/releases?q=ifcsverchok&expanded=true)
| [ifctester](https://docs.ifcopenshell.org/ifctester.html) | Library, CLI and webapp for IDS model auditing | LGPL-3.0-or-later | [![PyPI](https://img.shields.io/pypi/v/ifctester?label=PyPI&color=006dad)](https://pypi.org/project/ifctester/) |
+9 -8
View File
@@ -27,12 +27,6 @@ endif()
set(CMAKE_CXX_STANDARD_REQUIRED ON) # not necessary, but encouraged
set(CMAKE_EXPORT_COMPILE_COMMANDS ON)
find_program(CCACHE_FOUND ccache)
if(CCACHE_FOUND)
set_property(GLOBAL PROPERTY RULE_LAUNCH_COMPILE ccache)
message(STATUS "`ccache` is found, using it as a compiler launcher.")
endif()
add_definitions(-D_DISABLE_CONSTEXPR_MUTEX_CONSTRUCTOR)
if(POLICY CMP0141) # 3.25+
@@ -127,8 +121,10 @@ if((BUILD_CONVERT OR BUILD_GEOMSERVER OR BUILD_IFCPYTHON) AND(NOT BUILD_IFCGEOM)
set(BUILD_IFCGEOM ON)
endif()
find_program(CCACHE_FOUND ccache)
if(CCACHE_FOUND)
set_property(GLOBAL PROPERTY RULE_LAUNCH_COMPILE ccache)
message(STATUS "`ccache` is found, using it as a compiler launcher.")
set_property(GLOBAL PROPERTY RULE_LAUNCH_COMPILE "${CCACHE_FOUND}")
if(MSVC)
# By default Visual Studio generators will use /Zi which is not compatible
# with ccache, so tell Visual Studio to use /Z7 instead.
@@ -212,6 +208,8 @@ if(NOT MINIMAL_BUILD)
UNIFY_ENVVARS_AND_CACHE(LIBXML2_INCLUDE_DIR)
UNIFY_ENVVARS_AND_CACHE(LIBXML2_LIBRARIES)
UNIFY_ENVVARS_AND_CACHE(PCRE_LIBRARY_DIR)
UNIFY_ENVVARS_AND_CACHE(PYTHON_INCLUDE_DIR)
UNIFY_ENVVARS_AND_CACHE(PYTHON_LIBRARY)
UNIFY_ENVVARS_AND_CACHE(PYTHON_EXECUTABLE)
UNIFY_ENVVARS_AND_CACHE(HDF5_INCLUDE_DIR)
UNIFY_ENVVARS_AND_CACHE(HDF5_LIBRARY_DIR)
@@ -453,8 +451,11 @@ if(NOT MINIMAL_BUILD)
clear_wasm_sysroot()
if(IFCXML_SUPPORT)
if((NOT LIBXML2_INCLUDE_DIR AND NOT LIBXML2_LIBRARIES))
# First try config mode (probably works with vcpkg, Conan, macOS brew installs, but not on ubuntu 22.04?)
# First try config mode (probably works with vcpkg, Conan, macOS brew installs, but not on ubuntu 22.04)
# CONFIG is provided using root path, so no need to clear sysroot here.
restore_wasm_sysroot()
find_package(LibXml2 QUIET CONFIG)
clear_wasm_sysroot()
if(NOT LibXml2_FOUND)
# Fallback to CMake's builtin FindLibXml2 module (works on Ubuntu)
+193 -115
View File
@@ -18,8 +18,6 @@
# #
###############################################################################
# ruff: noqa: UP035
"""
Example usage:
# Build all targets by default.
@@ -33,6 +31,7 @@ Available arguments:
``-py-313`` - build for specific Python version
(building for all supported Python version by default).
``-wasm`` - compile for wasm
``-without-xxx`` - do not build dependency ``xxx`` (e.g. ``--without-swig``)
``-mac-cross-compile-intel`` - cross compile for Intel Mac on Apple Silicon host
``-shared`` - build shared libraries. By default will build static.
``-diskcleanup`` - clean up build directories after finishing building dependencies
@@ -52,6 +51,13 @@ Used environment variables:
- ``IFCOS_SCHEMAS`` - schemas to be built; defaults to cmake default (IFC2X3; IFC4; IFC4X3_ADD2) - to be supplied as `2x3;4`
- ``USE_OCCT`` - whether to use official Open CASCADE instead of Community Edition
(`true` by default, any other value is considered `false`)
- ``WASM_PYTHON_PATH`` - path to WASM Python installation,
used to deduce `PYVERSION` (e.g. '3.13.2'), `PYTHONINCLUDE`,
`SIDE_MODULE_CFLAGS`, `SIDE_MODULE_LDFLAGS`.
Allows to build wasm without pyodide build environment, which can be useful for debugging build issues.
Example value: 'pyodide/cpython/installs/python-3.13.2'
- ``WASM_TOOLCHAIN_FILE`` - path to emscripten toolchain file from pyodide ('Emscripten.cmake')
needed only if ``WASM_PYTHON_PATH`` is provided.
- ``ADD_COMMIT_SHA`` - if defined with any non-empty value then
`ADD_COMMIT_SHA` and `VERSION_OVERRIDE` will be set to `ON` while configuring IfcOpenShell
@@ -135,7 +141,7 @@ PROJECT_NAME = "IfcOpenShell"
USE_CURRENT_PYTHON_VERSION = os.getenv("USE_CURRENT_PYTHON_VERSION")
ADD_COMMIT_SHA = os.getenv("ADD_COMMIT_SHA")
PYTHON_VERSIONS = ["3.9.11", "3.10.3", "3.11.8", "3.12.1", "3.13.6"]
PYTHON_VERSIONS = ["3.9.11", "3.10.3", "3.11.8", "3.12.1", "3.13.6", "3.14.0"]
JSON_VERSION = "3.11.3"
OCE_VERSION = "0.18.3"
OCCT_VERSION = "7.8.1"
@@ -195,19 +201,59 @@ def cecho(message, color=NO_COLOR):
logger.info(f"{color}{message}\033[0m")
def which(cmd: str) -> Union[str, None]:
PATH = os.getenv("PATH")
assert PATH
for path in PATH.split(":"):
if os.path.exists(path) and cmd in os.listdir(path):
return cmd
return None
# Flags.
APPLE = platform.system() == "Darwin"
MAC_CROSS_COMPILE_INTEL = "mac-cross-compile-intel" in flags
assert platform.system() == "Darwin" or not MAC_CROSS_COMPILE_INTEL
WASM = "wasm" in flags
"""Build WASM outside pyodide build environment."""
WASM_CMAKE_IS_USING_INIT_VARS = False
if WASM:
def get_pyodide_config_var(var_name: str) -> str:
output = sp.check_output(["pyodide", "config", "get", var_name], encoding="utf-8").strip()
return output
if "PYODIDE_ROOT" not in os.environ:
cecho("WARNING. Couldn't find 'PYODIDE_ROOT' in environment variables.", YELLOW)
cecho("Assuming building wasm outside pyodide build environment and resetting necessary variables.", YELLOW)
os.environ["SIDE_MODULE_CFLAGS"] = get_pyodide_config_var("cflags")
os.environ["SIDE_MODULE_LDFLAGS"] = get_pyodide_config_var("ldflags")
# Override cmake toolchain for all `emcmake` calls,
# needed for shared libraries (resulting .so wrapper)
# and to ensure compilation is pyodide compatible (e.g. `-fwasm-exceptions` is used in compilation flags).
os.environ["CMAKE_TOOLCHAIN_FILE"] = get_pyodide_config_var("cmake_toolchain_file")
required_vars = (
"SIDE_MODULE_CFLAGS",
"SIDE_MODULE_LDFLAGS",
"CMAKE_TOOLCHAIN_FILE",
)
missing_vars = [v for v in required_vars if v not in os.environ]
assert not missing_vars, f"Some variables required for WASM compilation are missing: {', '.join(missing_vars)}"
def get_pyodide_build_version() -> "tuple[int, ...]":
pyodide_build_suffix = "pyodide-build version:"
output = sp.check_output(["pyodide", "--version"], encoding="utf-8").strip()
assert pyodide_build_suffix in output, output
version_line = next(l for l in output.splitlines() if l.startswith(pyodide_build_suffix))
version = version_line.partition(":")[2].strip()
return tuple(map(int, version.split(".")))
# Pyodide still in transition from `FLAGS` to `FLAGS_INIT`.
# `FLAGS_INIT` allow us to provide flags using environment variables
# and providing `FLAGS` directly would break pyodide toolchain.
WASM_CMAKE_IS_USING_INIT_VARS = get_pyodide_build_version() >= (0, 30, 8)
# pyodide provide empty `CXXFLAGS`, leading to issues using C++ files compiled with `-fexceptions`
# which is used by OCCT.
# https://github.com/pyodide/pyodide-build/issues/251
side_module_cxx_flags = os.environ.get("SIDE_MODULE_CXXFLAGS", "")
if side_module_cxx_flags.strip():
print("SIDE_MODULE_CXXFLAGS are already passed from pyodide build ('{side_module_cxx_flags}').")
print("Maybe it's time to stop overriding them in the script?")
os.environ["SIDE_MODULE_CXXFLAGS"] = os.environ["SIDE_MODULE_CFLAGS"]
# Set defaults for missing empty environment variables
@@ -223,9 +269,11 @@ if platform.system() == "Darwin":
IFCOS_NUM_BUILD_PROCS = os.getenv("IFCOS_NUM_BUILD_PROCS", multiprocessing.cpu_count() + 1)
CMAKE_DIR = os.path.realpath(os.path.join(os.path.dirname(__file__), "..", "cmake"))
SCRIPT_PATH = Path(__file__).parent
REPO_PATH = SCRIPT_PATH.parent
CMAKE_DIR = (REPO_PATH / "cmake").resolve().__str__()
build_dir = os.environ.get("BUILD_DIR", os.path.join(os.path.dirname(__file__), "..", "build"))
BUILD_DIR = os.environ.get("BUILD_DIR", (REPO_PATH / "build").__str__())
if WASM:
@@ -234,7 +282,7 @@ elif MAC_CROSS_COMPILE_INTEL:
arch = "x86_64"
else:
arch = platform.machine()
DEFAULT_DEPS_DIR = Path(build_dir) / platform.system() / arch
DEFAULT_DEPS_DIR = Path(BUILD_DIR) / platform.system() / arch
if TOOLSET:
DEFAULT_DEPS_DIR = DEFAULT_DEPS_DIR / TOOLSET
@@ -266,6 +314,7 @@ if USE_OCCT:
cecho(" - Compiling against official Open Cascade")
else:
cecho(" - Compiling against Open Cascade Community Edition")
cecho(f"* Build Directory = {BUILD_DIR}", MAGENTA)
cecho(f"* Dependency Directory = {DEPS_DIR}", MAGENTA)
cecho(f" - The directory where {PROJECT_NAME} dependencies are installed.")
cecho(f"* Build Config Type = {BUILD_CFG}", MAGENTA)
@@ -282,6 +331,11 @@ cecho(
""" - How many compiler processes may be run in parallel.
"""
)
cecho(f" * IFCOS_SCHEMAS = '{os.environ.get('IFCOS_SCHEMAS')}'", MAGENTA)
cecho(
""" - IFC Schemas to compile. If not provided, fallback to default provided in cmake.
"""
)
dependency_tree: "dict[str, tuple[str, ...]]" = {
"IfcParse": ("boost", "libxml2", "hdf5", "rocksdb"),
@@ -328,11 +382,13 @@ if MAC_CROSS_COMPILE_INTEL:
MAC_CROSS_COMPILE_INTEL_BJAM_ARGS = ["architecture=x86"]
MAC_CROSS_COMPILE_INTEL_CXX = "clang++ -arch x86_64"
MAC_CROSS_COMPILE_INTEL_CC = "clang -arch x86_64"
MAC_CROSS_COMPILE_INTEL_AUTOCONF_HOST_ARGS = ["--host=x86_64-apple-darwin"]
else:
MAC_CROSS_COMPILE_INTEL_ARGS = []
MAC_CROSS_COMPILE_INTEL_BJAM_ARGS = []
MAC_CROSS_COMPILE_INTEL_CXX = ""
MAC_CROSS_COMPILE_INTEL_CC = ""
MAC_CROSS_COMPILE_INTEL_AUTOCONF_HOST_ARGS = []
OFF_ON = ["OFF", "ON"]
BUILD_STATIC = "shared" not in flags
@@ -357,6 +413,23 @@ else:
targets = set(dependency_tree.keys())
targets = set(t for t in targets if "without-%s" % t.lower() not in flags)
if WASM:
SKIP_TARGETS_FOR_WASM = {
"hdf5",
"rocksdb",
"opencollada",
"swig",
"pcre",
"pcre2",
"IfcGeom",
"IfcConvert",
"IfcGeomServer",
}
SKIP_TARGETS_FOR_WASM = {t.lower() for t in SKIP_TARGETS_FOR_WASM}
skip_targets = {t for t in targets if t.lower() in SKIP_TARGETS_FOR_WASM}
if skip_targets:
cecho(f"Skipping targets for wasm build: {', '.join(sorted(skip_targets))}", YELLOW)
targets.difference_update(skip_targets)
print("Building:", *sorted(targets, key=lambda t: len(list(gather_dependencies(t)))))
@@ -365,16 +438,21 @@ yacc = "yacc" # Used during swig building process, installed with `bison` on De
missing_commands: "list[str]" = []
required_commands = [git, bunzip2, tar, cc, cplusplus, autoconf, automake, make, "patch", "cmake", yacc, xz]
if "wasm" in flags:
required_commands.remove(yacc) # yacc not needed for wasm builds
# Skip swig build for WASM.
required_commands.append("swig")
required_commands.append("pyodide")
required_commands.remove(yacc)
for cmd in required_commands:
if which(cmd) is None:
if shutil.which(cmd) is None:
missing_commands.append(cmd)
if missing_commands:
raise ValueError(f"Required tools not installed or not added to PATH: {', '.join(missing_commands)}")
MAC_INTEL_BIN_PATH = "/usr/local/bin"
if MAC_CROSS_COMPILE_INTEL:
brew = "/usr/local/bin/brew"
brew = f"{MAC_INTEL_BIN_PATH}/brew"
assert os.path.exists(brew), f"For intel cross compilation the brew path is expected to be '{brew}'."
# identifiers for the download tool (could be less memory consuming as ints, but are more verbose as strings)
@@ -398,6 +476,13 @@ except:
pass
def restore_env(var_name: str, old_value: Union[str, None]) -> None:
if old_value is None:
del os.environ[var_name]
else:
os.environ[var_name] = old_value
def run(cmds: "Sequence[str]", cwd: "Union[str, None]" = None, can_fail: bool = False) -> str:
"""
Wraps `subprocess.Popen.communicate()` and logs the command being executed,
@@ -497,16 +582,27 @@ def run_cmake(arg1, cmake_args: "list[str]", cmake_dir: Union[str, None] = None,
if "wasm" in flags:
wasm.append("emcmake")
cmake_flags: list[str] = []
if not WASM or not WASM_CMAKE_IS_USING_INIT_VARS:
# For WASM we provide flags using just environment variables.
# If we provide them using cmake vars, it will override emscripten toolchain flags.
# Unsure if we need this in general even for non-WASM builds.
cmake_flags.extend(
[
f"-DCMAKE_CXX_FLAGS='{os.environ['CXXFLAGS']}'",
f"-DCMAKE_C_FLAGS='{os.environ['CFLAGS']}'",
]
)
run(
[
*wasm,
"cmake",
P,
*cmake_flags,
*cmake_args,
f"-DCMAKE_BUILD_TYPE={BUILD_CFG}",
f"-DBUILD_SHARED_LIBS={OFF_ON[not BUILD_STATIC]}",
f"-DCMAKE_CXX_FLAGS='{os.environ['CXXFLAGS']}'",
f"-DCMAKE_C_FLAGS='{os.environ['CFLAGS']}'",
f"-DCMAKE_SHARED_LINKER_FLAGS={os.environ['LDFLAGS']}",
],
cwd=cwd,
@@ -637,7 +733,7 @@ def build_dependency(
if isinstance(patch, str):
patch = [patch]
for p in patch:
patch_abs = os.path.abspath(os.path.join(os.path.dirname(__file__), p))
patch_abs = (SCRIPT_PATH / p).absolute().__str__()
if os.path.exists(patch_abs):
try:
run(["patch", "-p1", "--batch", "--forward", "-i", patch_abs], cwd=extract_dir)
@@ -724,12 +820,13 @@ LDFLAGS = os.environ.get("LDFLAGS", "")
ADDITIONAL_ARGS_STR = " ".join(ADDITIONAL_ARGS)
CXXFLAGS_MINIMAL = f"{CXXFLAGS} {PIC} {ADDITIONAL_ARGS_STR}"
CFLAGS_MINIMAL = f"{CFLAGS} {PIC} {ADDITIONAL_ARGS_STR}"
if "wasm" in flags:
CFLAGS_MINIMAL = CXXFLAGS_MINIMAL = CFLAGS = CXXFLAGS = os.environ["SIDE_MODULE_CFLAGS"]
LDFLAGS = os.environ["SIDE_MODULE_LDFLAGS"]
# WASM `SIDE_MODULE_` are absorbed by `emcmake` automatically.
CXXFLAGS = CXXFLAGS_MINIMAL
CFLAGS = CFLAGS_MINIMAL
elif sp.call([bash, "-c", "ld --gc-sections 2>&1 | grep -- --gc-sections &> /dev/null"]) != 0:
CXXFLAGS_MINIMAL = f"{CXXFLAGS} {PIC} {ADDITIONAL_ARGS_STR}"
CFLAGS_MINIMAL = f"{CFLAGS} {PIC} {ADDITIONAL_ARGS_STR}"
if BUILD_STATIC:
CXXFLAGS = f"{CXXFLAGS} {PIC} -fdata-sections -ffunction-sections -fvisibility=hidden -fvisibility-inlines-hidden {ADDITIONAL_ARGS_STR}"
CFLAGS = f"{CFLAGS} {PIC} -fdata-sections -ffunction-sections -fvisibility=hidden {ADDITIONAL_ARGS_STR}"
@@ -738,8 +835,6 @@ elif sp.call([bash, "-c", "ld --gc-sections 2>&1 | grep -- --gc-sections &> /dev
CFLAGS = CFLAGS_MINIMAL
LDFLAGS = f"{LDFLAGS} -Wl,--gc-sections {ADDITIONAL_ARGS_STR}"
else:
CXXFLAGS_MINIMAL = f"{CXXFLAGS} {PIC} {ADDITIONAL_ARGS_STR}"
CFLAGS_MINIMAL = f"{CFLAGS} {PIC} {ADDITIONAL_ARGS_STR}"
if BUILD_STATIC:
CXXFLAGS = f"{CXXFLAGS} {PIC} -fvisibility=hidden -fvisibility-inlines-hidden {ADDITIONAL_ARGS_STR}"
CFLAGS = f"{CFLAGS} {PIC} -fvisibility=hidden -fvisibility-inlines-hidden {ADDITIONAL_ARGS_STR}"
@@ -819,9 +914,9 @@ if "eigen" in targets:
)
if "pcre" in targets:
OLD_CC, OLD_CCXX = None, None
OLD_CC, OLD_CXX = None, None
if MAC_CROSS_COMPILE_INTEL:
OLD_CC, OLD_CCXX = os.environ.get("CC"), os.environ.get("CXX")
OLD_CC, OLD_CXX = os.environ.get("CC"), os.environ.get("CXX")
os.environ["CC"] = MAC_CROSS_COMPILE_INTEL_CC
os.environ["CXX"] = MAC_CROSS_COMPILE_INTEL_CXX
# Keep it autoconf as OpenCOLLADA is pretty old and might break
@@ -834,14 +929,8 @@ if "pcre" in targets:
download_name=f"pcre-{PCRE_VERSION}.tar.bz2",
)
if MAC_CROSS_COMPILE_INTEL:
if OLD_CC is None:
del os.environ["CC"]
else:
os.environ["CC"] = OLD_CC
if OLD_CCXX is None:
del os.environ["CXX"]
else:
os.environ["CXX"] = OLD_CCXX
restore_env("CC", OLD_CC)
restore_env("CXX", OLD_CXX)
if "pcre2" in targets:
build_dependency(
@@ -873,7 +962,7 @@ if "freetype" in targets:
download_url="https://github.com/freetype/freetype",
download_name="freetype2",
download_tool=download_tool_git,
revision="VER-2-11-1",
revision="VER-2-14-0",
)
if USE_OCCT and "occ" in targets:
@@ -1012,6 +1101,7 @@ if "python" in targets and not USE_CURRENT_PYTHON_VERSION and "wasm" not in flag
# On OSX a dynamic python library is built or it would not be compatible
# with the system python because of some threading initialization
PYTHON_CONFIGURE_ARGS: "list[str]" = []
original_path = ""
if platform.system() == "Darwin":
PYTHON_CONFIGURE_ARGS = ["--enable-shared"]
open_ssl_prefix = run([brew, "--prefix", "openssl@3"]).strip()
@@ -1020,6 +1110,10 @@ if "python" in targets and not USE_CURRENT_PYTHON_VERSION and "wasm" not in flag
PYTHON_CONFIGURE_ARGS.append(f"--with-openssl={open_ssl_prefix}")
if MAC_CROSS_COMPILE_INTEL:
original_path = os.environ["PATH"]
# Need to ensure python will pick up intel's `pkg-config`,
# otherwise it might attempt to use ARM libraries (e.g. `zstd`) and fail.
os.environ["PATH"] = f"{MAC_INTEL_BIN_PATH}{os.pathsep}{original_path}"
PYTHON_CONFIGURE_ARGS.extend(["--with-universal-archs=intel-64", "--enable-universalsdk"])
for PYTHON_VERSION in PYTHON_VERSIONS:
@@ -1041,6 +1135,9 @@ if "python" in targets and not USE_CURRENT_PYTHON_VERSION and "wasm" not in flag
if not os.path.exists(os.path.join(DEPS_DIR, "install", f"python-{PYTHON_VERSION}")):
raise e
if MAC_CROSS_COMPILE_INTEL:
assert original_path
os.environ["PATH"] = original_path
os.environ["CPPFLAGS"] = OLD_CPP_FLAGS
os.environ["CXXFLAGS"] = OLD_CXX_FLAGS
os.environ["CFLAGS"] = OLD_C_FLAGS
@@ -1086,14 +1183,23 @@ if "boost" in targets:
if "cgal" in targets:
gmp_args: "list[str]" = []
mpfr_args: "list[str]" = []
if "wasm" in flags:
gmp_args.extend(("--disable-assembly", "--host", "none", "--enable-cxx"))
OLD_HOST_CC = None
if WASM:
if APPLE:
# Override `HOST_CC`, otherwise `emcc` will try to use it's own `clang` which can only build
# wasm executables and build will fail.
os.environ["HOST_CC"] = "clang"
# Disable assembly, otherwise `emcc -c conftest.s` will crash due to assembly mismatch.
gmp_args.extend(("--disable-assembly", "--enable-cxx"))
mpfr_args.extend(("--host", "none"))
OLD_CC = None
if MAC_CROSS_COMPILE_INTEL:
OLD_CC = os.environ.get("CC")
# Otherwise it's using arm64 `gcc` and fails to build gmp.
os.environ["CC"] = MAC_CROSS_COMPILE_INTEL_CC
gmp_args.extend(MAC_CROSS_COMPILE_INTEL_AUTOCONF_HOST_ARGS)
build_dependency(
name=f"gmp-{GMP_VERSION}",
@@ -1102,10 +1208,14 @@ if "cgal" in targets:
pre_compile_subs=(
[("build/config.h", "HAVE_OBSTACK_VPRINTF 1", "HAVE_OBSTACK_VPRINTF 0")] if "wasm" in flags else []
),
download_url="https://ftp.gnu.org/gnu/gmp/",
# Sometimes ftp.gnu.org is very slow, use ftpmirror.gnu.org as a workaround.
download_url="https://ftpmirror.gnu.org/gnu/gmp/",
download_name=f"gmp-{GMP_VERSION}.tar.bz2",
)
if WASM and APPLE:
restore_env("HOST_CC", OLD_HOST_CC)
build_dependency(
name=f"mpfr-{MPFR_VERSION}",
mode="autoconf",
@@ -1115,10 +1225,7 @@ if "cgal" in targets:
)
if MAC_CROSS_COMPILE_INTEL:
if OLD_CC is None:
del os.environ["CC"]
else:
os.environ["CC"] = OLD_CC
restore_env("CC", OLD_CC)
build_dependency(
name=f"cgal-{CGAL_VERSION}",
@@ -1249,15 +1356,19 @@ def get_cmake_args_prefix_path(additional_paths: "Sequence[str]" = ()) -> "list[
args_prefix_path = cmake_args_prefix_path.copy()
args_prefix_path.extend(additional_paths)
prefix_path = ";".join(args_prefix_path)
return [f"-DCMAKE_PREFIX_PATH={prefix_path}"]
if WASM:
# `emcmake` is disabling search in PATH, so we provide root paths instead.
# Provide '/' to PATH, so it will be combined with provided root paths,
# otherwise, depending on environment, it might not search the root path itself.
return [f"-DCMAKE_FIND_ROOT_PATH={prefix_path}", "-DCMAKE_PREFIX_PATH=//"]
else:
return [f"-DCMAKE_PREFIX_PATH={prefix_path}"]
if "wasm" in flags:
# Boost is built by the build script so should not be found
# inside of the sysroot set by the emscriptem toolchain
cmake_args.append("-DWASM_BUILD=On")
# set Eigen3 path for WASM to avoid find_package issues
cmake_args.append(f"-DEIGEN_DIR={DEPS_DIR}/install/eigen-install-{EIGEN_VERSION}/include/eigen3")
schemas = os.environ.get("IFCOS_SCHEMAS")
if schemas:
@@ -1267,26 +1378,9 @@ if "cgal" in targets:
cmake_args_prefix_path.append(f"{DEPS_DIR}/install/cgal-{CGAL_VERSION}")
cmake_args_prefix_path.append(f"{DEPS_DIR}/install/gmp-{GMP_VERSION}")
cmake_args_prefix_path.append(f"{DEPS_DIR}/install/mpfr-{MPFR_VERSION}")
if "wasm" in flags:
cmake_args.extend(
[
f"-DCGAL_INCLUDE_DIR={DEPS_DIR}/install/cgal-{CGAL_VERSION}/include",
f"-DGMP_INCLUDE_DIR={DEPS_DIR}/install/gmp-{GMP_VERSION}/include",
f"-DGMP_LIBRARY_DIR={DEPS_DIR}/install/gmp-{GMP_VERSION}/lib",
f"-DMPFR_INCLUDE_DIR={DEPS_DIR}/install/mpfr-{MPFR_VERSION}/include",
f"-DMPFR_LIBRARY_DIR={DEPS_DIR}/install/mpfr-{MPFR_VERSION}/lib",
]
)
if "occ" in targets and USE_OCCT:
cmake_args_prefix_path.append(f"{DEPS_DIR}/install/occt-{OCCT_VERSION}")
if "wasm" in flags:
cmake_args.extend(
[
f"-DOCC_INCLUDE_DIR={DEPS_DIR}/install/occt-{OCCT_VERSION}/include/opencascade",
f"-DOCC_LIBRARY_DIR={DEPS_DIR}/install/occt-{OCCT_VERSION}/lib",
]
)
elif "occ" in targets:
# We don't support find_package for OCE.
@@ -1307,13 +1401,6 @@ else:
if "libxml2" in targets:
cmake_args_prefix_path.append(f"{DEPS_DIR}/install/libxml2-{LIBXML2_VERSION}")
if "wasm" in flags:
cmake_args.extend(
[
f"-DLIBXML2_INCLUDE_DIR={DEPS_DIR}/install/libxml2-{LIBXML2_VERSION}/include/libxml2",
f"-DLIBXML2_LIBRARIES={DEPS_DIR}/install/libxml2-{LIBXML2_VERSION}/lib/libxml2.{LIBRARY_EXT}",
]
)
if "hdf5" in targets:
cmake_args_prefix_path.append(f"{DEPS_DIR}/install/hdf5-{HDF5_VERSION}")
@@ -1343,7 +1430,7 @@ if "rocksdb" in targets:
]
)
if not explicit_targets or {"IfcGeom", "IfcConvert", "IfcGeomServer"} & set(explicit_targets):
if not WASM and (not explicit_targets or {"IfcGeom", "IfcConvert", "IfcGeomServer"} & set(explicit_targets)):
logger.info("\rConfiguring executables...")
exec_args = [
@@ -1365,10 +1452,7 @@ if "IfcOpenShell-Python" in targets:
# On OSX the actual Python library is not linked against.
ADDITIONAL_ARGS = ""
if platform.system() == "Darwin":
ADDITIONAL_ARGS = "-Wl,-flat_namespace,-undefined,suppress"
if "wasm" in flags:
ADDITIONAL_ARGS = f"-Wl,-undefined,suppress -sSIDE_MODULE=2 -sEXPORTED_FUNCTIONS=_PyInit__ifcopenshell_wrapper"
ADDITIONAL_ARGS = "-Wl,-undefined,dynamic_lookup"
# NOTE: We don't use `CXXFLAGS` for wrappers, so wrapper is compiled with different flags
# (e.g. ` -fdata-sections` is missing, which is set by default for executables)
@@ -1382,40 +1466,47 @@ if "IfcOpenShell-Python" in targets:
os.makedirs(python_dir, exist_ok=True)
def compile_python_wrapper(
python_version: str, python_library: str, python_include: str, python_executable: Union[str, None]
python_version: str,
python_include: Union[str, None] = None,
python_executable: Union[str, None] = None,
python_path: Union[Path, None] = None,
) -> Union[str, None]:
"""
:return: Path to module dir if ``python_executable`` was provided, otherwise ``None``.
"""
assert bool(python_path) ^ bool(python_include)
logger.info(f"\rConfiguring python {python_version} wrapper...")
cache_path = os.path.join(python_dir, "CMakeCache.txt")
if os.path.exists(cache_path):
os.remove(cache_path)
os.environ["PYTHON_LIBRARY_BASENAME"] = os.path.basename(python_library)
swig_prefix_paths: list[str] = []
prefix_paths: list[str] = []
if "swig" in targets:
swig_prefix_paths.append(f"{DEPS_DIR}/install/swig")
prefix_paths.append(f"{DEPS_DIR}/install/swig")
if python_path:
# We couldn't just prefix PATH and have to provide all variables explicitly,
# see ifcwrap/cmake for the details.
python_executable = (Path(python_path) / "bin" / "python3").__str__()
python_include = run(
[
python_executable,
"-c",
"import sysconfig; print(sysconfig.get_config_var('INCLUDEPY'))",
]
)
assert python_include
run_cmake(
"",
cmake_args
+ get_cmake_args_prefix_path(swig_prefix_paths)
+ get_cmake_args_prefix_path(prefix_paths)
+ [
"-DPYTHON_LIBRARY=" + python_library,
*([f"-DPYTHON_EXECUTABLE={python_executable}"] if python_executable else []),
# *([f"-DPYTHON_MODULE_INSTALL_DIR={os.environ['PYTHONPATH']}/ifcopenshell"] if "wasm" in flags else []),
*(
[
"-DPYTHON_MODULE_INSTALL_DIR="
+ os.path.abspath(os.path.join(os.path.dirname(__file__), "..", "package"))
]
if "wasm" in flags
else []
),
"-DPYTHON_INCLUDE_DIR=" + python_include,
# Needed because pyodide is expecting setup.py to be in the root.
*([f"-DPYTHON_MODULE_INSTALL_DIR={REPO_PATH}"] * WASM),
f"-DPYTHON_INCLUDE_DIR={python_include}",
f"-DCMAKE_INSTALL_PREFIX={DEPS_DIR}/install/ifcopenshell/tmp",
"-DUSERSPACE_PYTHON_PREFIX="
+ ["Off", "On"][os.environ.get("PYTHON_USER_SITE", "").lower() in {"1", "on", "true"}],
@@ -1426,7 +1517,7 @@ if "IfcOpenShell-Python" in targets:
logger.info(f"\rBuilding python {python_version} wrapper... ")
run([make, f"-j{IFCOS_NUM_BUILD_PROCS}", "ifcopenshell_wrapper"], cwd=python_dir)
run([make, f"-j{IFCOS_NUM_BUILD_PROCS}", "ifcopenshell_wrapper", "VERBOSE=1"], cwd=python_dir)
run([make, "install/local"], cwd=os.path.join(python_dir, "ifcwrap"))
if python_executable:
@@ -1453,32 +1544,19 @@ if "IfcOpenShell-Python" in targets:
if "wasm" in flags:
compile_python_wrapper(
f"{os.environ['PYMAJOR']}.{os.environ['PYMINOR']}.{os.environ['PYMICRO']}",
f"{os.environ['TARGETINSTALLDIR']}/lib/libpython{os.environ['PYMAJOR']}.{os.environ['PYMINOR']}.a",
os.environ["PYTHONINCLUDE"],
None,
run(["pyodide", "config", "get", "python_version"]),
run(["pyodide", "config", "get", "python_include_dir"]),
)
# Copy setup.py where pyodide build system expects it.
shutil.copy(REPO_PATH / "pyodide" / "setup.py", REPO_PATH)
elif USE_CURRENT_PYTHON_VERSION:
python_info = sysconfig.get_paths()
py_path_components = [sysconfig.get_config_var("LIBDIR"), sysconfig.get_config_var("INSTSONAME")]
if sysconfig.get_config_var("multiarchsubdir"):
py_path_components.insert(1, sysconfig.get_config_var("multiarchsubdir").replace("/", ""))
python_lib = os.path.join(*py_path_components)
compile_python_wrapper(platform.python_version(), python_lib, python_info["include"], sys.executable)
compile_python_wrapper(platform.python_version(), python_info["include"], sys.executable)
else:
for python_version in PYTHON_VERSIONS:
python_library = run([bash, "-c", f"ls {DEPS_DIR}/install/python-{python_version}/lib/libpython*.*"])
python_include = run([bash, "-c", f"ls -d {DEPS_DIR}/install/python-{python_version}/include/python*"])
python_executable = os.path.join(
DEPS_DIR, "install", f"python-{python_version}", "bin", f"python{python_version[0]}"
)
module_dir = compile_python_wrapper(python_version, python_library, python_include, python_executable)
python_path = Path(DEPS_DIR) / "install" / f"python-{python_version}"
module_dir = compile_python_wrapper(python_version, python_path=python_path)
assert module_dir
# Not sure why, but added after reading this in the logs
# cp: /Users/runner/work/IfcOpenShell/IfcOpenShell/build/Darwin/x86_64/10.15/install/ifcopenshell/python-3.9.11: No such file or directory
+32
View File
@@ -0,0 +1,32 @@
There are two ways to build pyodide ifcopenshell Python wrapper wheel.
1. Using pyodide build system (`build_pyodide.yml` does it):
- install prebuilt pyodide build and emscripten environment (see `build_pyodide.sh`)
- clone IfcOpenShell to `IfcOpenShell` folder
- create `packages/ifcopenshell` folder that will be used by pyodide build system
- from `IfcOpenShell` move building recipe `pyodide/meta.yaml` to `packages/ifcopenshell`
- run `pyodide build-recipes ifcopenshell --install`, it will
- execute `meta.yaml` recipe - it will:
- copy IfcOpenShell source to build folder `packages/ifcopenhell/build/ifcopenshell-0.8.0`
- build ifcopenshell and its dependencies
- note that rerunning `pyodide build-recipes` will remove previous build folder and rebuild all dependencies.
The way to avoid it, if build fails, is to use `pyodide build-recipes-no-deps ifcopenshell --continue` instead.
- run `setup.py` in `IfcOpenShell` root, producing a wheel in `IfcOpenShell/dist`
- copy that wheel to `packages/ifcopenshell/dist`
- `--install` it to current build envrionment
- copy the wheel next to `dist` folder (in root directory, next to `packages`)
- add wheel to `dist/pyodide-lock.json`
2. Build it outside of pyodide build system.
Building inside pyodide build system should be preferred, option to build it outside is useful for debugging purposes,
since it's pure cmake without any additional moving parts.
- setup pyodide environment, see above
- clone IfcOpenShell repo next to it to `IfcOpenShell` folder
- run `python nix/build-all.py -wasm -py-313` in `IfcOpenShell`
- it will produce Python package in `IfcOpenShell/ifcopenshell`
- run `pyodide build`
- it will produce a wheel in `IfcOpenShell/dist`
+36
View File
@@ -0,0 +1,36 @@
#!/usr/bin/bash
set -ex
# Install uv.
curl -LsSf https://astral.sh/uv/install.sh | sh
uv venv --python 3.13
source .venv/bin/activate
# Install pyodide cross build environment.
# Instructions: https://pyodide.org/en/stable/development/building-packages.html
uv pip install pyodide-build
# `uv run` is required, so xbuildenv would skip using `pip`.
uv run pyodide xbuildenv install
# Emscripten doesn't come with xbuildenv.
git clone https://github.com/emscripten-core/emsdk
pushd emsdk
PYODIDE_EMSCRIPTEN_VERSION=$(pyodide config get emscripten_version)
./emsdk install ${PYODIDE_EMSCRIPTEN_VERSION}
./emsdk activate ${PYODIDE_EMSCRIPTEN_VERSION}
source emsdk_env.sh
which emcc
popd
mkdir -p packages/ifcopenshell
VERSION=`cat IfcOpenShell/VERSION`
cp IfcOpenShell/pyodide/meta.yaml packages/ifcopenshell
sed -i s/0.8.0/$VERSION/g packages/ifcopenshell/meta.yaml
# Use custom build ifcopenshell directory in build-all to make caching simpler
# Otherwise pyodide build path typically includes package version, so cached cmake configs might break.
export BUILD_DIR=`readlink -f ifcopenshell_build`
# Use build-recipes-no-deps first, so logs would be printed to stdout.
pyodide build-recipes-no-deps ifcopenshell
pyodide build-recipes ifcopenshell --install
+64
View File
@@ -0,0 +1,64 @@
"""
Cache built dependencies for builds.
This script is finding common install directory and either
packs each folder into a tar.gz archive, if it wasn't packed before,
or unpacks existing archives.
Usage: python cache_dependencies.py [pack|unpack]
"""
import tarfile
import sys
from pathlib import Path
from typing import Literal
CACHE_PREFIX = "cache-"
def get_install_dir() -> Path:
for data in Path.cwd().glob("*/*/install"):
return data
raise Exception("No install dir found")
def pack_dependencies(install_dir: Path) -> None:
# Process each install_dir
for dependency_path in install_dir.iterdir():
if not dependency_path.is_dir():
continue
dependency_name = dependency_path.name
tar_path = install_dir / f"{CACHE_PREFIX}{dependency_name}.tar.gz"
if tar_path.exists():
print(f"Skipping existing cache: '{tar_path}'")
else:
with tarfile.open(tar_path, "w:gz") as tar:
tar.add(dependency_path, arcname=dependency_path.name)
print(f"Created cache: '{tar_path}'")
def unpack_dependencies(install_dir: Path) -> None:
# `filter` argument was fully introduced in 3.12
# and results in deprecation warnings in 3.12-3.13, if not provided.
tar_filter: dict[Literal["filter"], Literal["data"]] = (
{"filter": "data"} if bool(sys.version_info >= (3, 12)) else {}
)
for tar_path in install_dir.glob(f"{CACHE_PREFIX}*.tar.gz"):
with tarfile.open(tar_path, "r:gz") as tar:
tar.extractall(path=install_dir, **tar_filter)
print(f"Extracted cache: '{tar_path.name}'.")
if __name__ == "__main__":
if len(sys.argv) != 2 or (action := sys.argv[1].lower()) not in ("pack", "unpack"):
print(__doc__)
sys.exit(1)
install_dir = get_install_dir()
print(f"Found install dir: '{install_dir}'")
if action == "pack":
pack_dependencies(install_dir)
else:
unpack_dependencies(install_dir)
+2 -3
View File
@@ -3,13 +3,12 @@ package:
version: 0.8.0
source:
# meta.yaml is placed as `packages/ifcopenshell/meta.yaml`.
path: ../../IfcOpenShell
build:
script: |
BUILD_CFG=Release python nix/build-all.py --without-rocksdb --without-hdf5 --without-opencollada --without-swig --without-pcre -v --wasm --py313 IfcOpenShell-Python
mv package/ifcopenshell .
cp pyodide/setup.py .
BUILD_CFG=Release python nix/build-all.py -v --wasm --py313
about:
home: http://ifcopenshell.org
+45 -9
View File
@@ -1,11 +1,47 @@
from setuptools import setup, find_packages
# setup.py is getting deprecated, but we still use it,
# because `tool.setuptools.ext-modules` is still experimental in pyproject.toml
# and we need it to get the wheel suffix right.
import os
from pathlib import Path
setup(name='ifcopenshell',
version='0.8.0',
description='IfcOpenShell is an open source (LGPL) software library for working with the Industry Foundation Classes (IFC) file format.',
author='Thomas Krijnen',
author_email='thomas@aecgeeks.com',
url='http://ifcopenshell.org',
packages=find_packages(),
package_data={'ifcopenshell': ['util/schema/*.json', 'util/schema/*.ifc'], '': ['*.so', '*.json', '*.ifc']},
import tomllib
from setuptools import Extension, find_packages, setup
REPO_FOLDER = Path(__file__).parent
def get_version() -> str:
if "PKG_VERSION" in os.environ:
# Inside pyodide build environment.
return os.environ["PKG_VERSION"]
return (REPO_FOLDER / "VERSION").read_text().strip()
# Read dependencies from pyproject.toml
def get_dependencies() -> list[str]:
pyproject_toml = REPO_FOLDER / "src" / "ifcopenshell-python" / "pyproject.toml"
pyproject_data = tomllib.loads(pyproject_toml.read_text())
dependencies = pyproject_data["project"]["dependencies"]
return dependencies
setup(
name="ifcopenshell",
version=get_version(),
description=(
"IfcOpenShell is an open source (LGPL) software library "
"for working with the Industry Foundation Classes (IFC) file format."
),
author="Thomas Krijnen",
author_email="thomas@aecgeeks.com",
url="https://ifcopenshell.org",
install_requires=get_dependencies(),
packages=find_packages(include=["ifcopenshell", "ifcopenshell.*"]),
package_data={
# "*.so" is needed to include prebuilt binary extension. Otherwise it would try to build it and fail.
"ifcopenshell": ["util/schema/*.json", "util/schema/*.ifc", "*.so"],
"": ["*.json", "*.ifc"],
},
# Has to provide extension to get the correct wheel suffix.
ext_modules=[Extension("ifcopenshell._ifcopenshell_wrapper", sources=[])],
)
+27
View File
@@ -0,0 +1,27 @@
from pathlib import Path
WHEEL_FILENAME = next(
p.name for p in (Path.cwd() / "pyodide").iterdir() if p.name.startswith("ifcopenshell-") and p.suffix == ".whl"
)
def test_ifcopenshell_import(selenium):
selenium.load_package("micropip")
# Important to test it with `micropip.install`
# without any dependencies loaded to ensure micropip will load them automatically.
selenium.run_async(
f"""
import micropip
await micropip.install(f"./{WHEEL_FILENAME}")
import ifcopenshell
ifc_file = ifcopenshell.file()
wall = ifc_file.create_entity("IfcWall")
wall1 = ifc_file.by_type("IfcWall")[0]
print(wall, wall1)
assert wall == wall1, "Wall entity doesn't match"
wall.Name = "Test"
assert wall.Name == "Test", f"Entity name wasn't changed: {{wall}}"
print(wall)
"""
)
+11
View File
@@ -62,3 +62,14 @@ ignore = [
"UP031", # Replace % with .format
"UP032", # Replace .format with f-string
]
[tool.poe.tasks]
ruff-main = "ruff check --extend-exclude nix/build-all.py"
# It's actually Python 3.6, but ruff only supports 3.7+, but it should do.
ruff-old = "ruff check nix/build-all.py --target-version py37"
ruff.sequence = ["ruff-main", "ruff-old"]
black = "black ."
format.sequence = ["black", "ruff-main", "ruff-old"]
+1 -1
View File
@@ -374,7 +374,7 @@ const IfcUtil::IfcBaseEntity* mapping::get_single_material_association(const Ifc
#endif
#ifdef SCHEMA_HAS_IfcMaterialConstituentSet
if (associated_material->as<IfcSchema::IfcMaterialConstituentSet>()) {
if (associated_material->as<IfcSchema::IfcMaterialConstituentSet>() && associated_material->as<IfcSchema::IfcMaterialConstituentSet>()->MaterialConstituents()) {
IfcSchema::IfcMaterialConstituentSet* constituentset = associated_material->as<IfcSchema::IfcMaterialConstituentSet>();
if (settings_.get<settings::LayersetFirst>().value ? constituentset->MaterialConstituents()->get()->size() >= 1 : constituentset->MaterialConstituents()->get()->size() == 1) {
IfcSchema::IfcMaterialConstituent* constituent = (*constituentset->MaterialConstituents()->get()->begin());
+3
View File
@@ -43,6 +43,9 @@ endif
ifeq ($(PYVERSION), py313)
PYNUMBER:=313
endif
ifeq ($(PYVERSION), py314)
PYNUMBER:=314
endif
ifndef PYNUMBER
$(error Unsupported PYVERSION '$(PYVERSION)')
endif
+6
View File
@@ -11,9 +11,15 @@ BUILDDIR = _build
# Put it first so that "make" without argument is like "make help".
help:
@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
@echo ""
@echo "You can also do 'make server' to launch html server for previously built html docs."
.PHONY: help Makefile
server:
python -m http.server 8080 --directory $(BUILDDIR)/html
.PHONY: server Makefile
# Catch-all target: route all unknown targets to Sphinx using the new
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
%: Makefile
@@ -44,6 +44,8 @@ ZIP packages
+-------------+---------------------------------+-------------------------------+---------------------------------+-----------------------------------+
| Python 3.13 | :ios_python_url:`py313-linux64` | :ios_python_url:`py313-win64` | :ios_python_url:`py313-macos64` | :ios_python_url:`py313-macosm164` |
+-------------+---------------------------------+-------------------------------+---------------------------------+-----------------------------------+
| Python 3.14 | :ios_python_url:`py314-linux64` | :ios_python_url:`py314-win64` | :ios_python_url:`py314-macos64` | :ios_python_url:`py314-macosm164` |
+-------------+---------------------------------+-------------------------------+---------------------------------+-----------------------------------+
2. Unzip the downloaded file and copy the ``ifcopenshell`` directory into your
Python path. If you're not sure where your Python path is, run the following
@@ -176,13 +178,19 @@ to launch a simple notebook.
Web Assembly
------------
IfcOpenShell is available as technology preview to be run using WASM. This
allows you to run IfcOpenShell in a browser using pyodide. This implementation
is incredibly heavy and will incur a long load time, but once loaded, will give
you full access to the entire IfcOpenShell API.
To run IfcOpenShell in a browser using pyodide, we have available pyodide WASM
packages at `wasm-wheels
<https://github.com/IfcOpenShell/wasm-wheels#pyodide-test-wheels>`__ repository.
`Click here <https://github.com/IfcOpenShell/wasm-preview>`__ to learn how to
use WASM.
There is also a technology preview to be run using WASM. This implementation
is incredibly heavy and will incur a long load time, but once loaded, will give
you full access to the entire IfcOpenShell API:
- the latest preview - `here
<https://github.com/IfcOpenShell/IfcOpenShell/blob/v0.8.0/src/pyodide/demo-app/README.md>`__
- an older preview, that may have some additional information - `here
<https://github.com/IfcOpenShell/wasm-preview>`__
Using Bonsai
------------
@@ -251,7 +251,7 @@ nest formulas, for example ``concat(title("foo"), lower("Bar"))`` will produce
"``int({{value}})``", "``int(3.123)``", "``3``", "Truncates the decimal part of the ``{{value}}``."
"``number({{value}}[, {{decimal_separator}}[, {{thousands_separator}}]])``", "``number(1234.56, "","", ""."")``", "``1.234,56``", "Formats {{value}} with an optional custom {{decimal_separator}} and {{thousands_separator}}. The default separators are ``.`` and ``,``."
"``metric_length({{value}}, {{precision}}, {{decimals}})``", "``metric_length(3.123, 0.1, 2)``", "``3.10``", "Rounds ``{{value}}`` to the nearest ``{{precision}}`` then displays using a certain amount of decimal places."
"``imperial_length({{value}}, {{precision}}, {{input_unit}}, {{output_unit}})``", "``imperial_length(3.22, 4, ""foot"")``", "``3' - 3 3/4""``", "``The {{value}}`` may be specified either as ``foot`` or ``inch`` depending on ``{{input_unit}}``. The ``{{value}}`` is then rounded to the nearest ``1/{{precision}}`` inch then formatted using fractional feet and inches if ``{{output_unit}}`` is set to ``foot`` or just inches if ``{{output_unit}}`` is set to ``inch``."
"``imperial_length({{value}}, {{precision}}, {{input_unit}}, {{output_unit}}, {{suppress_zero_inches}})``", "``imperial_length(3.0, 4, ""foot"", ""foot"", true)`` OR ``imperial_length(3.0, 4, ""foot"", ""foot"", false)``", "``3'`` OR ``3' - 0""``", "The ``{{value}}`` may be specified either as ``foot`` or ``inch`` depending on ``{{input_unit}}``. The ``{{value}}`` is then rounded to the nearest ``1/{{precision}}`` inch, then formatted using fractional feet and inches if ``{{output_unit}}`` is set to ``foot``, or just inches if ``{{output_unit}}`` is set to ``inch``. When ``{{suppress_zero_inches}}`` is ``true`` (default), measurements with zero inches will omit the inch portion (e.g., ``3'`` instead of ``3' - 0""``)."
When using queries in an IfcAnnotation tag surround with backticks.
Examples:
@@ -148,13 +148,16 @@ format_grammar = lark.Lark(
int: "int(" function ")"
format_length: metric_length | imperial_length
metric_length: "metric_length(" function "," NUMBER "," NUMBER ")"
imperial_length: "imperial_length(" function "," NUMBER ["," ESCAPED_STRING "," ESCAPED_STRING] ")"
imperial_length: "imperial_length(" function "," NUMBER ["," ESCAPED_STRING "," ESCAPED_STRING ["," boolean]] ")"
lower: "lower(" function ")"
upper: "upper(" function ")"
title: "title(" function ")"
concat: "concat(" function ("," function)* ")"
substr: "substr(" function "," SIGNED_INT ["," SIGNED_INT] ")"
boolean: TRUE | FALSE
TRUE: "true" | "True" | "TRUE"
FALSE: "false" | "False" | "FALSE"
// Embed common.lark for packaging
DIGIT: "0".."9"
HEXDIGIT: "a".."f"|"A".."F"|DIGIT
@@ -218,6 +221,17 @@ class FormatTransformer(lark.Transformer):
elif len(args) == 2:
return str(args[0])[int(args[1]) :]
def boolean(self, args):
if not args:
return True
token = args[0]
if hasattr(token, "type"):
return token.type == "TRUE"
value = str(token).lower()
if hasattr(token, "value"):
value = str(token.value).lower()
return value in ("true", "1", "yes")
def round(self, args):
value = Decimal(0.0 if args[0] == "None" else args[0] or 0.0)
nearest = Decimal(args[1])
@@ -248,13 +262,27 @@ class FormatTransformer(lark.Transformer):
if len(args) == 2:
input_unit, output_unit = "foot", "foot"
value, precision = args
else:
suppress_zero_inches = True
elif len(args) == 3:
value, precision, suppress_zero_inches = args
input_unit, output_unit = "foot", "foot"
elif len(args) == 4:
value, precision, input_unit, output_unit = args
input_unit = "inch" if input_unit == "inch" else "foot"
output_unit = "inch" if output_unit == "inch" else "foot"
suppress_zero_inches = True
else:
value, precision, input_unit, output_unit, suppress_zero_inches = args
input_unit = "inch" if input_unit == "inch" else "foot"
output_unit = "inch" if output_unit == "inch" else "foot"
return ifcopenshell.util.unit.format_length(
float(value), int(precision), unit_system="imperial", input_unit=input_unit, output_unit=output_unit
float(value),
int(precision),
suppress_zero_inches=suppress_zero_inches,
unit_system="imperial",
input_unit=input_unit,
output_unit=output_unit,
)
def int(self, args: list[str]) -> str:
+1 -1
View File
@@ -30,7 +30,7 @@ import test.bootstrap
# - .github/workflows/ci-ifcopenshell-python.yml
# - .github/workflows/ci-ifcopenshell-python-pypi.yml
# - src/ifcopenshell-python/Makefile (PYVERSION check)
SUPPORTED_PY_VERSIONS = ("39", "310", "311", "312", "313")
SUPPORTED_PY_VERSIONS = ("39", "310", "311", "312", "313", "314")
SUPPORTED_PLATFORMS = ("win64", "linux64", "macos64", "macosm164")
+87 -52
View File
@@ -35,31 +35,39 @@ ENDIF()
include(GNUInstallDirs)
INCLUDE(${SWIG_USE_FILE})
IF(NOT "$ENV{PYTHON_INCLUDE_DIR}" STREQUAL "")
SET(PYTHON_INCLUDE_DIR $ENV{PYTHON_INCLUDE_DIR} CACHE FILEPATH "Python header files")
MESSAGE(STATUS "Looking for Python header files in: ${PYTHON_INCLUDE_DIR}")
IF(NOT "${PYTHON_INCLUDE_DIR}" STREQUAL "")
# Hint for FindPython where to find Python_INCLUDE_DIRS.
set(Python_INCLUDE_DIR "${PYTHON_INCLUDE_DIR}")
MESSAGE(STATUS "Looking for Python header files in: ${Python_INCLUDE_DIR}")
ENDIF()
IF(NOT "$ENV{PYTHON_LIBRARY}" STREQUAL "")
SET(PYTHON_LIBRARY $ENV{PYTHON_LIBRARY} CACHE FILEPATH "Python library file")
MESSAGE(STATUS "Looking for Python library file in: ${PYTHON_LIBRARY}")
IF(NOT "${PYTHON_LIBRARY}" STREQUAL "")
# Hint for FindPython where to find Python_LIBRARIES.
set(Python_LIBRARY "${PYTHON_LIBRARY}")
MESSAGE(STATUS "Looking for Python library file in: ${Python_LIBRARY}")
ENDIF()
# NOTE PYTHONLIBS_FOUND and PYTHONINTERP_FOUND cannot seem to be trusted so
# we need further checks to see whether the packages were actually found or not.
FIND_PACKAGE(PythonLibs)
IF(NOT PYTHONLIBS_FOUND OR "$PYTHON_INCLUDE_DIRS}" STREQUAL "")
# Development.Module = headers on Unix, headers+libraries on Windows.
# Required variables:
# - Windows - Python_INCLUDE_DIR, Python_LIBRARY (MSVC doesn't allow undefined symbols at link time)
# - Unix - Python_INCLUDE_DIR
# Unfortunately all paths must be always provided explicitly, not by prefixing PATH.
# Otherwise FindPython will break on newer Python versions (list of supported versions is hardcoded).
find_package(Python COMPONENTS Development.Module)
IF(NOT Python_Development.Module_FOUND)
MESSAGE(FATAL_ERROR "BUILD_IFCPYTHON enabled, but unable to find Python lib or header. Disable BUILD_IFCPYTHON or fix Python paths to proceed.")
ENDIF()
INCLUDE_DIRECTORIES(${PYTHON_INCLUDE_DIRS})
# Ensure version is saved here, from wasm libraries,
# not from Python interpreter that might be unrelated to pyodide Python version.
set(_python_libs_version "${Python_VERSION_MAJOR}${Python_VERSION_MINOR}")
INCLUDE_DIRECTORIES(BEFORE ${CMAKE_CURRENT_SOURCE_DIR})
SET(CMAKE_SWIG_FLAGS ${SWIG_DEFINES})
# NOTE Workaround for most likely missing debug Python libraries on Windows (requires Python built from the source).
# Python 3.5 intaller and onwards will have an option to install the debug libraries too.
# NOTE PYTHON_DEBUG_LIBRARIES appears to be a deprecated variable
IF (WIN32 AND NOT PYTHON_DEBUG_LIBRARIES)
MESSAGE(STATUS "PYTHON_DEBUG_LIBRARIES not found, defining SWIG_PYTHON_INTERPRETER_NO_DEBUG workaround for IfcWrap.")
IF (WIN32 AND NOT Python_LIBRARY_DEBUG)
MESSAGE(STATUS "Python debug libraries were not found, defining SWIG_PYTHON_INTERPRETER_NO_DEBUG workaround for IfcWrap.")
ADD_DEFINITIONS(-DSWIG_PYTHON_INTERPRETER_NO_DEBUG)
ENDIF()
@@ -79,51 +87,80 @@ SET_PROPERTY(
utils/typemaps_out.i
)
# Debug build on Windows add required `_d` suffix to the resulting .pyd file.
# If we won't override -interface name,
# swig py wrapper will try to import `_ifcopenshell_wrapper_d` module and would fail.
set(SWIG_MODULE_ifcopenshell_wrapper_EXTRA_FLAGS "-interface" "_ifcopenshell_wrapper")
# On Windows there is '_d' prefix for debug builds - e.g. `_d.cp311-win_amd64.pyd`.
if(WIN32)
set(Python_DEBUG_POSTFIX "_d")
endif()
if (CMAKE_VERSION VERSION_GREATER_EQUAL "4.2")
# `DEBUG_POSTFIX` argument was added in 4.2.
swig_add_library(ifcopenshell_wrapper LANGUAGE python SOURCES IfcPython.i DEBUG_POSTFIX "${Python_DEBUG_POSTFIX}")
else()
# If we won't override -interface name,
# swig py wrapper will try to import `_ifcopenshell_wrapper_d` module and would fail.
set(SWIG_MODULE_ifcopenshell_wrapper_EXTRA_FLAGS "-interface" "_ifcopenshell_wrapper")
swig_add_library(ifcopenshell_wrapper LANGUAGE python SOURCES IfcPython.i)
set_target_properties(
ifcopenshell_wrapper PROPERTIES
DEBUG_POSTFIX "${Python_DEBUG_POSTFIX}"
)
endif()
swig_add_library(ifcopenshell_wrapper LANGUAGE python SOURCES IfcPython.i)
SET_PROPERTY(TARGET ${SWIG_MODULE_ifcopenshell_wrapper_REAL_NAME} PROPERTY SWIG_DEPENDS ${IFCOPENSHELL_LIBRARIES})
target_link_libraries(ifcopenshell_wrapper PRIVATE Python::Module)
SET_PROPERTY(TARGET ifcopenshell_wrapper PROPERTY SWIG_DEPENDS ${IFCOPENSHELL_LIBRARIES})
if (WASM_BUILD)
# SIDE_MODULE=1 - add to .so all symbols from linked archives (default used by pyodide).
# Since currently libIfcGeom.a seems to be linked twice it results in duplicated symbols and compilation errors.
# Possibly in the future we can clean up linked libs and try `=1`.
# SIDE_MODULE=2 - add to .so only explicitly exported functions (`-s EXPORTED_FUNCTIONS).
# Have to provide it here to override possible `=1` coming from pyodide.
if(CMAKE_GENERATOR MATCHES "Makefile")
# Accomodate bug in cmake makefile generator:
# https://gitlab.kitware.com/cmake/cmake/-/issues/27326
string(
REPLACE "SIDE_MODULE=1" "SIDE_MODULE=2"
CMAKE_MODULE_LINKER_FLAGS
"${CMAKE_MODULE_LINKER_FLAGS}"
)
endif()
target_link_options(
ifcopenshell_wrapper
PRIVATE "SHELL:-s EXPORTED_FUNCTIONS=_PyInit__ifcopenshell_wrapper -s SIDE_MODULE=2"
)
endif()
if("$ENV{LDFLAGS}" MATCHES ".undefined.suppress")
# On osx there is some state in the python dylib. With `-Wl,undefined,suppress` we can ignore the missing symbols at compile time.
SWIG_LINK_LIBRARIES(ifcopenshell_wrapper ${IFCOPENSHELL_LIBRARIES} ${OPENCASCADE_LIBRARIES} ${Boost_LIBRARIES} ${LIBSVGFILL})
target_link_libraries(ifcopenshell_wrapper PRIVATE ${IFCOPENSHELL_LIBRARIES} ${OPENCASCADE_LIBRARIES} ${Boost_LIBRARIES} ${LIBSVGFILL})
else()
SWIG_LINK_LIBRARIES(ifcopenshell_wrapper ${IFCOPENSHELL_LIBRARIES} ${PYTHON_LIBRARIES} ${LIBSVGFILL})
target_link_libraries(ifcopenshell_wrapper PRIVATE ${IFCOPENSHELL_LIBRARIES} ${LIBSVGFILL})
endif()
if ((NOT WIN32) AND BUILD_SHARED_LIBS)
SET_INSTALL_RPATHS(${SWIG_MODULE_ifcopenshell_wrapper_REAL_NAME} "${IFCDIRS};${OCC_LIBRARY_DIR}")
SET_INSTALL_RPATHS(ifcopenshell_wrapper "${IFCDIRS};${OCC_LIBRARY_DIR}")
endif()
# Try to find the Python interpreter to get the site-packages
# directory in which the wrapper can be installed.
FIND_PACKAGE(PythonInterp)
IF((PYTHONINTERP_FOUND AND NOT "${PYTHON_EXECUTABLE}" STREQUAL "") OR PYTHON_MODULE_INSTALL_DIR)
if (NOT WASM_BUILD)
# Find Python interpreter and get its version
# Not on WASM because we're cross compiling in that case
EXECUTE_PROCESS(
COMMAND ${PYTHON_EXECUTABLE} -c "import sysconfig; print(sysconfig.get_config_var('EXT_SUFFIX'))"
OUTPUT_VARIABLE PYTHON_EXTENSION_SUFFIX
OUTPUT_STRIP_TRAILING_WHITESPACE
)
# On Windows there is '_d' prefix for debug builds - e.g. `_d.cp311-win_amd64.pyd`.
if(CMAKE_SYSTEM_NAME STREQUAL "Windows")
set_target_properties(
${SWIG_MODULE_ifcopenshell_wrapper_REAL_NAME} PROPERTIES
SUFFIX ${PYTHON_EXTENSION_SUFFIX}
DEBUG_POSTFIX "_d"
)
if(NOT ${PYTHON_EXECUTABLE} STREQUAL "")
set(Python_EXECUTABLE "${PYTHON_EXECUTABLE}")
MESSAGE(STATUS "Looking for Python interpreter in: ${PYTHON_EXECUTABLE}")
endif()
FIND_PACKAGE(Python COMPONENTS Interpreter)
IF(Python_Interpreter_FOUND OR PYTHON_MODULE_INSTALL_DIR)
# Python_SOABI example value is 'cp311-win_amd64'.
if(WASM_BUILD)
# For WASM we usually don't provide interpreter,
# so `find_package` couldn't find `Python_SOABI`.
# So we just hardcode it.
# .cpython-313-wasm32-emscripten.so
set(PYTHON_EXTENSION_SUFFIX ".cpython-${_python_libs_version}-wasm32-emscripten.so")
elseif(WIN32)
set(PYTHON_EXTENSION_SUFFIX ".${Python_SOABI}.pyd")
else()
set_target_properties(
${SWIG_MODULE_ifcopenshell_wrapper_REAL_NAME} PROPERTIES
SUFFIX ${PYTHON_EXTENSION_SUFFIX}
)
set(PYTHON_EXTENSION_SUFFIX ".${Python_SOABI}.so")
endif()
endif(NOT WASM_BUILD)
set_target_properties(
ifcopenshell_wrapper PROPERTIES
SUFFIX ${PYTHON_EXTENSION_SUFFIX}
)
if (PYTHON_MODULE_INSTALL_DIR)
set(python_package_dir "${PYTHON_MODULE_INSTALL_DIR}")
else()
@@ -133,10 +170,7 @@ IF((PYTHONINTERP_FOUND AND NOT "${PYTHON_EXECUTABLE}" STREQUAL "") OR PYTHON_MOD
OUTPUT_VARIABLE python_package_dir
)
ELSE ()
EXECUTE_PROCESS(
COMMAND ${PYTHON_EXECUTABLE} -c "import sys; import sysconfig; sys.stdout.write(sysconfig.get_path('platlib'))"
OUTPUT_VARIABLE python_package_dir
)
set(python_package_dir "${Python_SITEARCH}")
ENDIF()
if (BUILD_PACKAGE)
set(python_package_dir ${CMAKE_INSTALL_LIBDIR}/python${PYTHON_VERSION_MAJOR}/dist-packages/)
@@ -145,6 +179,7 @@ IF((PYTHONINTERP_FOUND AND NOT "${PYTHON_EXECUTABLE}" STREQUAL "") OR PYTHON_MOD
IF("${python_package_dir}" STREQUAL "")
MESSAGE(WARNING "Unable to locate Python site-package directory, unable to install the Python wrapper")
ELSE()
message(STATUS "Python wrapper will be installed to '${python_package_dir}'.")
FILE(GLOB_RECURSE sourcefiles
"${CMAKE_CURRENT_SOURCE_DIR}/../ifcopenshell-python/ifcopenshell/*"
)
@@ -158,10 +193,10 @@ IF((PYTHONINTERP_FOUND AND NOT "${PYTHON_EXECUTABLE}" STREQUAL "") OR PYTHON_MOD
ENDFOREACH()
INSTALL(FILES "${CMAKE_BINARY_DIR}/ifcwrap/ifcopenshell_wrapper.py"
DESTINATION "${python_package_dir}/ifcopenshell")
INSTALL(TARGETS ${SWIG_MODULE_ifcopenshell_wrapper_REAL_NAME}
INSTALL(TARGETS ifcopenshell_wrapper
DESTINATION "${python_package_dir}/ifcopenshell")
if (MSVC)
INSTALL(FILES $<TARGET_PDB_FILE:${SWIG_MODULE_ifcopenshell_wrapper_REAL_NAME}> DESTINATION bin OPTIONAL)
INSTALL(FILES $<TARGET_PDB_FILE:ifcopenshell_wrapper> DESTINATION bin OPTIONAL)
endif()
ENDIF()
ELSE()
+105
View File
@@ -0,0 +1,105 @@
"""
It's not really a full version of nix/build-all.py for Windows,
but serves the similar purpose - build all packages during CI (though by using cmd scripts),
but also archives them to '~/outputs'.
"""
import os
import subprocess
from pathlib import Path
from zipfile import ZipFile
assert Path.cwd() == Path(__file__).parent, "Run this script from the 'win' directory."
PYTHON_VERSIONS = ["3.9.13", "3.10.3", "3.11.8", "3.12.1", "3.13.0", "3.14.0"]
REPO_PATH = Path(__file__).parent.parent
REPO_WIN = REPO_PATH / "win"
VERSION = (REPO_PATH / "VERSION").read_text().strip()
if "GITHUB_SHA" in os.environ:
SHA = os.environ["GITHUB_SHA"][:7]
else:
SHA = subprocess.check_output(["git", "rev-parse", "HEAD"], text=True).strip()
OUTPUT_DIR = Path.home() / "output"
OUTPUT_DIR.mkdir(exist_ok=True)
print("Output directory:", OUTPUT_DIR)
ZIP_TEMPLATE = f"{{package_name}}-v{VERSION}-{SHA}-win64.zip"
def run(command: list[str]) -> None:
print("Running:", command)
subprocess.check_call(command) # nosec B603
def build() -> None:
for python_version in PYTHON_VERSIONS:
os.environ["PYTHON_VERSION"] = python_version
print(f"Building for Python {python_version}...")
subprocess.run(
[str(REPO_WIN / "build-deps.cmd"), "vs2022-x64", "Release"],
check=True,
text=True,
input="y\n",
)
run(
[
str(REPO_WIN / "run-cmake.bat"),
"vs2022-x64",
"-DENABLE_BUILD_OPTIMIZATIONS=ON",
"-DGLTF_SUPPORT=ON",
"-DADD_COMMIT_SHA=ON",
"-DVERSION_OVERRIDE=ON",
"-DSCHEMA_VERSIONS=4",
]
)
run([str(REPO_WIN / "install-ifcopenshell.bat"), "vs2022-x64", "Release"])
def archive_executables() -> None:
# Typically '_installed-vs2022-x64'.
install_dir = next(d for d in REPO_PATH.iterdir() if d.is_dir() and d.name.startswith("_installed"))
for file in (install_dir / "bin").iterdir():
if file.suffix.lower() != ".exe":
continue
zip_name = ZIP_TEMPLATE.format(package_name=file.stem)
with ZipFile(OUTPUT_DIR / zip_name, "w") as zipf:
zipf.write(file, arcname=file.name)
print(f"{file} -> {zip_name}")
def archive_python_package(python_version: str, python_path: Path) -> None:
python_version_major_minor = "".join(python_version.split(".")[:2])
site_packages = python_path / "Lib" / "site-packages"
package_path = site_packages / "ifcopenshell"
# Clean cache.
for file in package_path.rglob("*.pyc"):
file.unlink()
zip_name = ZIP_TEMPLATE.format(package_name=f"ifcopenshell-python-{python_version_major_minor}")
with ZipFile(OUTPUT_DIR / zip_name, "w") as zipf:
for file in package_path.rglob("*"):
arcname = file.relative_to(site_packages)
zipf.write(file, arcname=arcname)
print(f"{package_path} -> {zip_name}")
def archive_python_packages() -> None:
deps_path = REPO_PATH / "_deps"
python_versions: list[str] = []
for d in deps_path.iterdir():
if d.is_dir() and d.name.startswith("python."):
python_version = d.name.partition(".")[2]
python_path = d / "tools"
archive_python_package(python_version, python_path)
python_versions.append(d.name.partition(".")[2])
def main() -> None:
build()
archive_executables()
archive_python_packages()
if __name__ == "__main__":
main()
+78 -38
View File
@@ -90,10 +90,6 @@ IF %VS_VER%==2008 set PATH=C:\Windows\Microsoft.NET\Framework\v3.5;%PATH%
:: User-configurable build options
IF NOT DEFINED IFCOS_INSTALL_PYTHON set IFCOS_INSTALL_PYTHON=TRUE
set PYTHON_VERSION=3.11
py -%PYTHON_VERSION% --version 2>&1>NUL
IF %ERRORLEVEL%==0 set IFCOS_INSTALL_PYTHON=EXISTS
set PYTHON_VERSION=%PYTHON_VERSION%.7
IF NOT DEFINED IFCOS_NUM_BUILD_PROCS set IFCOS_NUM_BUILD_PROCS=%NUMBER_OF_PROCESSORS%
@@ -147,12 +143,12 @@ echo Defaults to RelWithDebInfo if not specified.
IF %BUILD_CFG%==MinSizeRel call cecho.cmd 0 14 " WARNING: MinSizeRel build can suffer from a significant performance loss."
call cecho.cmd 0 13 "* Build Type`t`t= %BUILD_TYPE%"
echo - The used build type for the dependencies (Build, Rebuild, Clean).
echo Defaults to Build if not specified. Rebuild/Clean also uninstalls Python (if it was installed by this script).
echo Defaults to Build if not specified.
call cecho.cmd 0 13 "* IFCOS_INSTALL_PYTHON`t= %IFCOS_INSTALL_PYTHON%"
echo - Download and install Python.
echo Set to something other than TRUE if you wish to use an already installed version of Python.
echo EXISTS value is set automatically if same Python version is already found on the system
echo and we won't be able to install it again.
echo But then you'll need to set PYTHONHOME env variable to your Python installation before running run-cmake.bat
echo to your Python installation path.
call cecho.cmd 0 13 "* IFCOS_NUM_BUILD_PROCS`t= %IFCOS_NUM_BUILD_PROCS%"
echo - How many MSBuild.exe processes may be run in parallel.
echo Defaults to NUMBER_OF_PROCESSORS. Used also by other IfcOpenShell build scripts.
@@ -180,17 +176,17 @@ cd "%DEPS_DIR%"
:: Don't use HDF5 1.13.0, because it has a broken cmake package path.
set HDF5_VERSION=1_13_1
set OCCT_VERSION=7.8.1
:: NOTE If updating the default Python version, change PY_VER_MAJOR_MINOR accordingly in run-cmake.bat
set PYTHON_VERSION=%PYTHON_VERSION%
IF DEFINED PYTHON_VERSION (
echo Using overridden PYTHON_VERSION: '%PYTHON_VERSION%'
) else (
set PYTHON_VERSION=3.11.7
)
:: VERSION DERIVATIONS
set OCC_INCLUDE_DIR=%INSTALL_DIR%\opencascade-%OCCT_VERSION%\inc>>"%~dp0\%BUILD_DEPS_CACHE_PATH%"
set OCC_LIBRARY_DIR=%INSTALL_DIR%\opencascade-%OCCT_VERSION%\win%ARCH_BITS%\lib>>"%~dp0\%BUILD_DEPS_CACHE_PATH%"
for /f "tokens=1,2,3 delims=." %%a in ("%PYTHON_VERSION%") do (
set PY_VER_MAJOR_MINOR=%%a%%b
)
IF "%IFCOS_INSTALL_PYTHON%"=="TRUE" (
set PYTHONHOME=%INSTALL_DIR%\Python%PY_VER_MAJOR_MINOR%
set PYTHONHOME=%DEPS_DIR%\python.%PYTHON_VERSION%\tools
)
:: Cache last used CMake generator and configurable dependency dirs for other scripts to use
@@ -201,10 +197,56 @@ echo HDF5_VERSION=%HDF5_VERSION%>>"%~dp0\%BUILD_DEPS_CACHE_PATH%"
echo OCC_INCLUDE_DIR=%OCC_INCLUDE_DIR%>>"%~dp0\%BUILD_DEPS_CACHE_PATH%"
echo OCC_LIBRARY_DIR=%OCC_LIBRARY_DIR%>>"%~dp0\%BUILD_DEPS_CACHE_PATH%"
IF "%IFCOS_INSTALL_PYTHON%"=="TRUE" (
echo PY_VER_MAJOR_MINOR=%PY_VER_MAJOR_MINOR%>>"%~dp0\%BUILD_DEPS_CACHE_PATH%"
echo PYTHONHOME=%PYTHONHOME%>>"%~dp0\%BUILD_DEPS_CACHE_PATH%"
)
:nuget
set DEPENDENCY_NAME=nuget
set NUGET_VERSION=6.14.0
set NUGET_INSTALL_DIR=%DEPS_DIR%\nuget-%NUGET_VERSION%
set NUGET_EXE=%NUGET_INSTALL_DIR%\nuget.exe
where nuget >nul 2>&1
IF %ERRORLEVEL%==0 (
echo Found existing nuget in PATH. Skipping.
for /f "delims=" %%i in ('where nuget') do set "NUGET_EXE=%%i"
goto :ccache
)
IF EXIST "%NUGET_EXE%" (
echo Found existing "%DEPS_DIR%\nuget.exe", skipping
goto :ccache
)
cd %DEPS_DIR%
call :DownloadFile ^
https://dist.nuget.org/win-x86-commandline/v%NUGET_VERSION%/nuget.exe ^
"%NUGET_INSTALL_DIR%" nuget.exe
IF NOT %ERRORLEVEL%==0 GOTO :Error
:ccache
set DEPENDENCY_NAME=ccache
set CCACHE_VERSION=4.12.1
set CCACHE_INSTALL_DIR=%DEPS_DIR%\%DEPENDENCY_NAME%.%CCACHE_VERSION%\tools
set DEPENDENCY_DIR=%CCACHE_INSTALL_DIR%
where ccache >nul 2>&1
IF %ERRORLEVEL%==0 (
echo Found existing ccache in PATH. Skipping.
goto :proj
)
echo CCACHE_INSTALL_DIR=%CCACHE_INSTALL_DIR%>>"%~dp0\%BUILD_DEPS_CACHE_PATH%"
IF EXIST "%DEPENDENCY_DIR%" (
echo Found existing "%DEPENDENCY_DIR%", skipping
goto :proj
)
"%NUGET_EXE%" install ccache -Version %CCACHE_VERSION% -OutputDirectory "%DEPS_DIR%"
IF NOT %ERRORLEVEL%==0 GOTO :Error
:proj
IF EXIST "%INSTALL_DIR%\proj-9.2.1" (
@@ -506,32 +548,29 @@ set PYTHON_AMD64_POSTFIX=-amd64
IF NOT %TARGET_ARCH%==x64 set PYTHON_AMD64_POSTFIX=
set PYTHON_INSTALLER=python-%PYTHON_VERSION%%PYTHON_AMD64_POSTFIX%.exe
IF "%IFCOS_INSTALL_PYTHON%"=="TRUE" (
cd "%DEPS_DIR%"
call :DownloadFile https://www.python.org/ftp/python/%PYTHON_VERSION%/%PYTHON_INSTALLER% "%DEPS_DIR%" %PYTHON_INSTALLER%
IF NOT %ERRORLEVEL%==0 GOTO :Error
REM Uninstall if build Rebuild/Clean used
IF NOT %BUILD_TYPE%==Build (
call cecho.cmd 0 13 "Uninstalling %DEPENDENCY_NAME%. Please be patient, this will take a while."
start /w %PYTHON_INSTALLER% /quiet /uninstall
)
IF NOT EXIST "%PYTHONHOME%". (
call cecho.cmd 0 13 "Installing %DEPENDENCY_NAME%. Please be patient, this will take a while."
start /w %PYTHON_INSTALLER% /quiet TargetDir="%PYTHONHOME%"
if errorlevel 1 (
:: Standard installer doesn't support installing same Python version twice,
:: but we skip installation during IFCOS_INSTALL_PYTHON initialization.
call cecho.cmd 0 12 "Failed to install Python. Error code: !ERRORLEVEL!."
GOTO :Error
)
) ELSE (
call cecho.cmd 0 13 "%DEPENDENCY_NAME% already installed. Skipping."
)
) ELSE (
call cecho.cmd 0 13 "IFCOS_INSTALL_PYTHON not true, skipping installation of Python."
IF NOT "%IFCOS_INSTALL_PYTHON%"=="TRUE" (
call cecho.cmd 0 13 "IFCOS_INSTALL_PYTHON not 'TRUE', skipping installation of Python."
goto :SWIG
)
:: nuget doesn't support providing architecture for packages.
if NOT %TARGET_ARCH%==x64 (
call cecho.cmd 0 12 "Automatic insallation of Python for x86 builds is not supported,"
call cecho.cmd 0 12 "please install Python %PYTHON_VERSION% manually and ensure that it is available in PATH."
call cecho.cmd 0 12 "https://www.python.org/ftp/python/%PYTHON_VERSION%/%PYTHON_INSTALLER%"
goto :Error
)
if EXIST "%PYTHONHOME%" (
echo Found existing '%PYTHONHOME%', skipping installation.
goto :SWIG
)
"%NUGET_EXE%" install Python -Version %PYTHON_VERSION% -OutputDirectory "%DEPS_DIR%"
IF NOT %ERRORLEVEL%==0 GOTO :Error
:SWIG
IF EXIST "%INSTALL_DIR%\swigwin" (
@@ -753,6 +792,7 @@ exit /b %IFCOS_SCRIPT_RET%
:: DownloadFile - Downloads a file using PowerShell
:: Params: %1 url, %2 destinationDir, %3 filename
:DownloadFile
mkdir "%2"
pushd "%2"
if not exist "%~3". (
call cecho.cmd 0 13 "Downloading %DEPENDENCY_NAME% into %~2."
+8 -5
View File
@@ -91,13 +91,15 @@ set OPENCOLLADA_INSTALL_DIR=%INSTALL_DIR%\OpenCOLLADA
set LIBXML2_INCLUDE_DIR=%DEPS_DIR%\OpenCOLLADA\Externals\LibXML\include
set LIBXML2_LIBRARIES=%INSTALL_DIR%\OpenCOLLADA\lib\opencollada\xml.lib
set HDF5_INSTALL_DIR=%INSTALL_DIR%\HDF5-%HDF5_VERSION%-win%ARCH_BITS%
if not defined PY_VER_MAJOR_MINOR set PY_VER_MAJOR_MINOR=311
if not defined PYTHONHOME set PYTHONHOME=%INSTALL_DIR%\Python%PY_VER_MAJOR_MINOR%
set PYTHON_EXECUTABLE=%PYTHONHOME%\python.exe
for /f "usebackq delims=" %%v in (`
call "%PYTHON_EXECUTABLE%" -c "import sys; print(f'{sys.version_info[0]}{sys.version_info[1]}')"
`) do set "PY_VER_MAJOR_MINOR=%%v"
set PYTHON_INCLUDE_DIR=%PYTHONHOME%\include
set PYTHON_LIBRARY=%PYTHONHOME%\libs\python%PY_VER_MAJOR_MINOR%.lib
set PYTHON_EXECUTABLE=%PYTHONHOME%\python.exe
set SWIG_INSTALL_DIR=%INSTALL_DIR%\swigwin
set PATH=%PATH%;%PYTHONHOME%
set JSON_INCLUDE_DIR=%INSTALL_DIR%\json
if not defined ADD_COMMIT_SHA set ADD_COMMIT_SHA=Off
@@ -151,6 +153,7 @@ echo TBB_INCLUDE_DIR = %TBB_INCLUDE_DIR%
echo TBB_LIBRARY_DIR = %TBB_LIBRARY_DIR%
echo ROCKSDB_INSTALL_DIR = %ROCKSDB_INSTALL_DIR%
echo ZSTD_INSTALL_DIR = %ZSTD_INSTALL_DIR%
echo CCACHE_INSTALL_DIR = %CCACHE_INSTALL_DIR%
echo.
echo CMAKE_INSTALL_PREFIX = %CMAKE_INSTALL_PREFIX%
echo.
@@ -162,7 +165,7 @@ echo "Running CMake for %PROJECT_NAME%."
set CMAKE_PREFIX_PATH=%HDF5_INSTALL_DIR%;%OPENCOLLADA_INSTALL_DIR%;%SWIG_INSTALL_DIR%
set CMAKE_PREFIX_PATH=%CMAKE_PREFIX_PATH%;%ROCKSDB_INSTALL_DIR%;%ZSTD_INSTALL_DIR%
set CMAKE_PREFIX_PATH=%CMAKE_PREFIX_PATH%;%BOOST_ROOT%
set CMAKE_PREFIX_PATH=%CMAKE_PREFIX_PATH%;%BOOST_ROOT%;%CCACHE_INSTALL_DIR%
IF NOT "%VS_TOOLSET_HOST%"=="" (
cmake.exe %CMAKELISTS_DIR% -G %GENERATOR% -A %VS_PLATFORM% -T %VS_TOOLSET_HOST% ^