mirror of
https://github.com/IfcOpenShell/IfcOpenShell.git
synced 2026-08-05 23:41:44 +00:00
92c50ed3b4
Dockerfile (renamed from Dockerfile_init, Dockerfile_update removed): - Run as a non-root `builder` user matching the host UID/GID (passed as --build-arg by create() from id -u/id -g), so build output under the bind mount stays owned by the host user instead of root. - Fix CCACHE_MAXSIZE: `ccache -M 5G` wrote its limit to a config file under /ccache at image-build time, but /ccache is a volume mount point, so that file gets shadowed by the (empty) volume the moment the container actually runs - the cap never took effect. Set CCACHE_MAXSIZE=5G as an image ENV instead. - Dedupe ccache/libffi-devel, add --setopt=install_weak_deps=False --setopt=tsflags=nodocs, add `git lfs install --system`, combine the dnf update+install into one layer. - Drop Dockerfile_update: it built FROM its own previous output, so every `update` call made the image strictly larger forever (Docker layers are append-only, `dnf clean` in a later layer can't shrink an earlier one). `update` now just calls create(), which already runs `dnf update -y` FROM a clean rockylinux:9 every time. compose.yaml: pin platform: linux/amd64 so this doesn't silently run under emulation on an ARM host. ifcos_env: - Split the previously-conflated stop/down into six distinct, Compose-native lifecycle commands: up (create-or-start), down (remove), stop, start, restart (stop+start, same container), recreate (down+up, fresh container). Previously `stop` was aliased to `down`, which silently removed the container instead of pausing it. - Implement try(): copies the built wrapper into a real Blender/Bonsai install for manual testing, reading the target from a new BLENDER_USER_RESOURCE .env variable and auto-detecting the built Python version (disambiguating via PY_TGT for multi-version builds). Deliberately kept human-only - it mutates a live Blender install, so it shouldn't run unattended as part of an automated/AI workflow, which should instead copy the wrapper into the repo's own src/ifcopenshell-python/ifcopenshell/ (documented in SKILL.md). - Fix unique(): the "has .env already got a UNIQUE_ID line" check referenced an unset $FILE instead of $ENV_FILE, so it always evaluated true and appended a fresh "UNIQUE_ID=dummy" line to .env on every single `up`. - Minor: differentiate remove()'s log message from down()'s (no longer identical now that they're distinct operations), tidy help text alignment and a stray double-space typo in clean(). SKILL.md: rewritten as current-state documentation (no more "fixed in this copy" changelog framing) covering the above, plus a migration note for anyone hitting root-owned leftovers from an older image. Verified by actually building the image and driving every new lifecycle command (stop/start/restart keep the same container ID; down+up and recreate produce a new one) and try() (including the quoted-tilde BLENDER_USER_RESOURCE edge case) against the real container. Generated with the assistance of an AI coding tool.
187 lines
8.3 KiB
Markdown
187 lines
8.3 KiB
Markdown
---
|
|
name: ifcopenshell-docker-build
|
|
description: >-
|
|
Build a real ifcopenshell_wrapper (.so + .py) and IfcConvert locally via
|
|
the docker/ifcos_env toolchain, then wire them into a checkout for
|
|
running C++-dependent parts of the test suite (geometry, the SWIG
|
|
wrapper stub, the C++ parser). Use whenever a task needs to compile
|
|
IfcOpenShell's C++ core rather than just read/patch source - e.g.
|
|
reproducing or fixing a bug in src/ifcgeom, src/ifcparse, src/ifcwrap,
|
|
or validating util/scripts/validate_stub.py against the actual
|
|
generated wrapper.
|
|
---
|
|
|
|
# Building IfcOpenShell locally with docker/ifcos_env
|
|
|
|
`docker/` mirrors the project's GitHub Actions build environment locally,
|
|
in a persistent, non-root container with ccache so repeat builds are fast.
|
|
See `docker/README.md` for the design rationale. Pure-Python changes don't
|
|
need any of this - only reach for it when you need a real compiled
|
|
`_ifcopenshell_wrapper*.so` or `IfcConvert` binary.
|
|
|
|
## Placement
|
|
|
|
This `docker/` folder must live as a direct child of the repo root you want
|
|
to build (sibling of `src/`, `cmake/`, etc.) - `compose.yaml` and
|
|
`ifcos_env` resolve the repo via `../` relative to wherever `docker/`
|
|
itself sits, and bind-mount it into the container. If you're setting this
|
|
up in a fresh clone, copy the whole `docker/` directory there first.
|
|
|
|
## Setup
|
|
|
|
```bash
|
|
cd docker
|
|
./ifcos_env create # build the image (shared by name across all your clones/checkouts, so usually instant after the first time anywhere)
|
|
./ifcos_env up # create + start the container, clone/unpack the third-party dependency cache (~10GB, one-time per container)
|
|
./ifcos_env build # full build: all deps + IfcParse + IfcGeom + IfcConvert + the Python wrapper, for one Python version
|
|
```
|
|
|
|
`PY_TGT` and `UNIQUE_ID` live in `docker/.env` - `PY_TGT` (e.g. `py-311`)
|
|
restricts the build to one Python version instead of building five;
|
|
`UNIQUE_ID` is a hash of the folder path, recalculated on every `up`, so
|
|
each checkout gets its own container/volumes automatically.
|
|
|
|
A full first build takes ~1.5 hours (mostly compiling IfcOpenShell's own
|
|
C++, not the cached third-party deps). After that, ccache makes incremental
|
|
rebuilds of a couple of touched `.cpp` files **under a minute**.
|
|
|
|
## Container lifecycle
|
|
|
|
The container is long-lived (`sleep infinity`) so exec'd commands and
|
|
ccache state persist between builds. Commands map directly onto Docker
|
|
Compose's own container-vs-image distinction:
|
|
|
|
```bash
|
|
./ifcos_env up # create the container if it doesn't exist, then start it (runs ready_repo too)
|
|
./ifcos_env stop # stop the container, keep it around
|
|
./ifcos_env start # start it back up (same container, same filesystem layer)
|
|
./ifcos_env restart # stop, then start
|
|
./ifcos_env down # remove the container (and its network) entirely
|
|
./ifcos_env recreate # down, then up - a fresh container
|
|
```
|
|
|
|
Named volumes (`ccache`) and the bind-mounted repo/`build/` are unaffected
|
|
by `down`/`recreate` - only the container itself goes away, and `up`
|
|
recreates it from the image.
|
|
|
|
## Fast iteration
|
|
|
|
Pass a target to `build` to skip the parts you don't need:
|
|
|
|
```bash
|
|
./ifcos_env build IfcConvert # only the executables (IfcConvert, IfcGeomServer) - skips the Python wrapper entirely
|
|
./ifcos_env build IfcOpenShell-Python # only the SWIG Python wrapper - skips executables entirely
|
|
./ifcos_env build # no target = everything (needed the first time, or after touching shared headers)
|
|
```
|
|
|
|
Use this to keep the edit -> rebuild -> test loop fast when debugging: if
|
|
you're only touching `src/ifcgeom/`, build `IfcConvert`; if you're only
|
|
exercising the Python API, build `IfcOpenShell-Python`.
|
|
|
|
## Where the artifacts land
|
|
|
|
Build output goes to `<repo_root>/build/Linux/x86_64/install/` on the host
|
|
(bind-mounted, not just inside the container), owned by you (see
|
|
"Container user" below):
|
|
|
|
- `ifcopenshell/bin/IfcConvert` - the CLI binary
|
|
- `python-<version>/lib/python<X.Y>/site-packages/ifcopenshell/_ifcopenshell_wrapper*.so`
|
|
and `ifcopenshell_wrapper.py` - the compiled wrapper + its generated
|
|
Python glue
|
|
|
|
## Testing against a checkout (automated / AI-driven)
|
|
|
|
`_ifcopenshell_wrapper*.so` and `ifcopenshell_wrapper.py` are already
|
|
gitignored under `src/ifcopenshell-python/ifcopenshell/`, which is exactly
|
|
where a normal in-tree build would put them - copy the two files there:
|
|
|
|
```bash
|
|
SRC=build/Linux/x86_64/install/python-3.11.8/lib/python3.11/site-packages/ifcopenshell
|
|
cp "$SRC/_ifcopenshell_wrapper.cpython-311-x86_64-linux-gnu.so" src/ifcopenshell-python/ifcopenshell/
|
|
cp "$SRC/ifcopenshell_wrapper.py" src/ifcopenshell-python/ifcopenshell/
|
|
```
|
|
|
|
Then, to run the test suite against it:
|
|
|
|
```bash
|
|
export PATH="$PWD/build/Linux/x86_64/install/ifcopenshell/bin:$PATH" # for IfcConvert-dependent tests
|
|
cd src/ifcopenshell-python/test
|
|
PYTHONPATH="$PWD/.." python3.11 -m pytest -p no:pytest-blender .
|
|
```
|
|
|
|
(`-p no:pytest-blender` avoids the pytest-blender plugin trying to find a
|
|
`blender` executable and failing collection entirely, even for non-Blender
|
|
tests.) You'll need the matching Python version's `pip install`s too
|
|
(numpy, shapely, isodate, lark, tabulate, pytest, ... - whatever the
|
|
modules under test import) since this is a bare interpreter, not the
|
|
project's pixi env.
|
|
|
|
**This is the pattern to use for automated or AI-driven verification.**
|
|
Don't use `try` (below) for that - it overwrites files in a real, live
|
|
Blender installation, which isn't something an automated/AI workflow
|
|
should ever do without the human explicitly asking for it in the moment.
|
|
|
|
## Testing in Blender itself (human only)
|
|
|
|
`try` copies the built wrapper straight into your actual Blender/Bonsai
|
|
extension install, for manual in-Blender testing:
|
|
|
|
```bash
|
|
./ifcos_env try
|
|
```
|
|
|
|
It reads `BLENDER_USER_RESOURCE` from `.env` - set this to wherever
|
|
Blender's user resource folder for the Bonsai extension actually lives on
|
|
your system, which depends on your own Blender setup:
|
|
|
|
```bash
|
|
# in docker/.env
|
|
BLENDER_USER_RESOURCE=~/.config/blender/bonsai/
|
|
```
|
|
|
|
`try` figures out the built Python version from `build/.../install/`
|
|
(disambiguating with `PY_TGT` if more than one version was built) and
|
|
copies the wrapper to
|
|
`$BLENDER_USER_RESOURCE/extensions/.local/lib/python<X.Y>/site-packages/ifcopenshell/`.
|
|
|
|
## Container user
|
|
|
|
The image runs as a non-root `builder` user, UID/GID matching your host
|
|
account (passed as `--build-arg` by `create` from `id -u`/`id -g`, so it
|
|
adjusts automatically - no manual flag needed even if you're not 1000:1000).
|
|
Files the build creates under the bind mount come out owned by you, not
|
|
root. Passwordless `sudo` is available inside the container (e.g. via
|
|
`attach`) for the rare case you need root for something ad hoc.
|
|
|
|
If you're picking up an existing checkout that was previously built with
|
|
an older, root-based image, you may hit `Permission denied` the first time
|
|
you run `up`/`build` under the new image - `build/`, `.git/modules/`, the
|
|
`ccache` volume, `output/`, and `build.log` can all be left root-owned from
|
|
before. Fix it once via the container's own root (no host `sudo` needed):
|
|
|
|
```bash
|
|
docker exec -u root -w /__w/IfcOpenShell/IfcOpenShell <container-name> \
|
|
chown -R "$(id -u)":"$(id -g)" .git/modules build output build.log /ccache
|
|
```
|
|
|
|
(`<container-name>` is `ifcopenshell-<UNIQUE_ID>` - see `docker ps -a`.)
|
|
|
|
## Other things worth knowing
|
|
|
|
- **Linux x64 only.** `compose.yaml` pins `platform: linux/amd64`; on an
|
|
ARM host (e.g. Apple Silicon) this build isn't available.
|
|
- **The final "Package .zip archives" step of `build()` has a pre-existing
|
|
bash syntax error**, unrelated to compilation - the actual build already
|
|
succeeded by that point (look for `Built IfcOpenShell...` in the output),
|
|
so this is safe to ignore if you only need the raw artifacts under
|
|
`build/.../install/`, not packaged release zips.
|
|
- **`test_mmaped_stream` and similar `USE_MMAP`-dependent tests will fail**
|
|
against this build - `nix/build-all.py` is invoked with `USE_MMAP=OFF`
|
|
here. Not a bug in your code if you see it fail.
|
|
- Only the bind-mounted `<repo>/build` lives on the host filesystem your
|
|
repo is checked out on. Anything the container writes *outside* that
|
|
mount lives in the container's own writable layer under Docker's data
|
|
root (commonly `/var/lib/docker`, i.e. usually your root partition) -
|
|
keep an eye on `df -h /` if you're running several of these containers
|
|
at once.
|