mirror of
https://github.com/IfcOpenShell/IfcOpenShell.git
synced 2026-09-16 13:46:54 +00:00
Harden docker build tooling: non-root, clean lifecycle, try()
Dockerfile (renamed from Dockerfile_init, Dockerfile_update removed):
- Run as a non-root `builder` user matching the host UID/GID (passed as
--build-arg by create() from id -u/id -g), so build output under the
bind mount stays owned by the host user instead of root.
- Fix CCACHE_MAXSIZE: `ccache -M 5G` wrote its limit to a config file
under /ccache at image-build time, but /ccache is a volume mount
point, so that file gets shadowed by the (empty) volume the moment
the container actually runs - the cap never took effect. Set
CCACHE_MAXSIZE=5G as an image ENV instead.
- Dedupe ccache/libffi-devel, add --setopt=install_weak_deps=False
--setopt=tsflags=nodocs, add `git lfs install --system`, combine the
dnf update+install into one layer.
- Drop Dockerfile_update: it built FROM its own previous output, so
every `update` call made the image strictly larger forever (Docker
layers are append-only, `dnf clean` in a later layer can't shrink an
earlier one). `update` now just calls create(), which already runs
`dnf update -y` FROM a clean rockylinux:9 every time.
compose.yaml: pin platform: linux/amd64 so this doesn't silently run
under emulation on an ARM host.
ifcos_env:
- Split the previously-conflated stop/down into six distinct,
Compose-native lifecycle commands: up (create-or-start), down
(remove), stop, start, restart (stop+start, same container),
recreate (down+up, fresh container). Previously `stop` was aliased
to `down`, which silently removed the container instead of pausing
it.
- Implement try(): copies the built wrapper into a real Blender/Bonsai
install for manual testing, reading the target from a new
BLENDER_USER_RESOURCE .env variable and auto-detecting the built
Python version (disambiguating via PY_TGT for multi-version builds).
Deliberately kept human-only - it mutates a live Blender install, so
it shouldn't run unattended as part of an automated/AI workflow,
which should instead copy the wrapper into the repo's own
src/ifcopenshell-python/ifcopenshell/ (documented in SKILL.md).
- Fix unique(): the "has .env already got a UNIQUE_ID line" check
referenced an unset $FILE instead of $ENV_FILE, so it always
evaluated true and appended a fresh "UNIQUE_ID=dummy" line to .env
on every single `up`.
- Minor: differentiate remove()'s log message from down()'s (no longer
identical now that they're distinct operations), tidy help text
alignment and a stray double-space typo in clean().
SKILL.md: rewritten as current-state documentation (no more "fixed in
this copy" changelog framing) covering the above, plus a migration
note for anyone hitting root-owned leftovers from an older image.
Verified by actually building the image and driving every new
lifecycle command (stop/start/restart keep the same container ID;
down+up and recreate produce a new one) and try() (including the
quoted-tilde BLENDER_USER_RESOURCE edge case) against the real container.
Generated with the assistance of an AI coding tool.
(cherry picked from commit 92c50ed3b4)
This commit is contained in:
committed by
Dion Moult
parent
68b234f59b
commit
316dace11a
+93
-44
@@ -13,13 +13,11 @@ description: >-
|
||||
|
||||
# Building IfcOpenShell locally with docker/ifcos_env
|
||||
|
||||
`docker/` is a small toolchain (see `docker/README.md` for the original
|
||||
author's own description and design rationale - read that first for the
|
||||
*why*; this file is the practical *how*, distilled from actually driving it
|
||||
end-to-end) that mirrors the project's GitHub Actions build environment
|
||||
locally, with a persistent container and ccache so repeat builds are fast.
|
||||
Pure-Python changes don't need any of this - only reach for it when you need
|
||||
a real compiled `_ifcopenshell_wrapper*.so` or `IfcConvert` binary.
|
||||
`docker/` mirrors the project's GitHub Actions build environment locally,
|
||||
in a persistent, non-root container with ccache so repeat builds are fast.
|
||||
See `docker/README.md` for the design rationale. Pure-Python changes don't
|
||||
need any of this - only reach for it when you need a real compiled
|
||||
`_ifcopenshell_wrapper*.so` or `IfcConvert` binary.
|
||||
|
||||
## Placement
|
||||
|
||||
@@ -29,12 +27,12 @@ to build (sibling of `src/`, `cmake/`, etc.) - `compose.yaml` and
|
||||
itself sits, and bind-mount it into the container. If you're setting this
|
||||
up in a fresh clone, copy the whole `docker/` directory there first.
|
||||
|
||||
## First-time setup
|
||||
## Setup
|
||||
|
||||
```bash
|
||||
cd docker
|
||||
./ifcos_env create # build the base image (shared across all your clones/checkouts by name, so usually instant after the first time anywhere)
|
||||
./ifcos_env up # start the container, clone+unpack the third-party dependency cache (~10GB, one-time per container)
|
||||
./ifcos_env create # build the image (shared by name across all your clones/checkouts, so usually instant after the first time anywhere)
|
||||
./ifcos_env up # create + start the container, clone/unpack the third-party dependency cache (~10GB, one-time per container)
|
||||
./ifcos_env build # full build: all deps + IfcParse + IfcGeom + IfcConvert + the Python wrapper, for one Python version
|
||||
```
|
||||
|
||||
@@ -47,6 +45,25 @@ A full first build takes ~1.5 hours (mostly compiling IfcOpenShell's own
|
||||
C++, not the cached third-party deps). After that, ccache makes incremental
|
||||
rebuilds of a couple of touched `.cpp` files **under a minute**.
|
||||
|
||||
## Container lifecycle
|
||||
|
||||
The container is long-lived (`sleep infinity`) so exec'd commands and
|
||||
ccache state persist between builds. Commands map directly onto Docker
|
||||
Compose's own container-vs-image distinction:
|
||||
|
||||
```bash
|
||||
./ifcos_env up # create the container if it doesn't exist, then start it (runs ready_repo too)
|
||||
./ifcos_env stop # stop the container, keep it around
|
||||
./ifcos_env start # start it back up (same container, same filesystem layer)
|
||||
./ifcos_env restart # stop, then start
|
||||
./ifcos_env down # remove the container (and its network) entirely
|
||||
./ifcos_env recreate # down, then up - a fresh container
|
||||
```
|
||||
|
||||
Named volumes (`ccache`) and the bind-mounted repo/`build/` are unaffected
|
||||
by `down`/`recreate` - only the container itself goes away, and `up`
|
||||
recreates it from the image.
|
||||
|
||||
## Fast iteration
|
||||
|
||||
Pass a target to `build` to skip the parts you don't need:
|
||||
@@ -64,14 +81,15 @@ exercising the Python API, build `IfcOpenShell-Python`.
|
||||
## Where the artifacts land
|
||||
|
||||
Build output goes to `<repo_root>/build/Linux/x86_64/install/` on the host
|
||||
(bind-mounted, not just inside the container):
|
||||
(bind-mounted, not just inside the container), owned by you (see
|
||||
"Container user" below):
|
||||
|
||||
- `ifcopenshell/bin/IfcConvert` - the CLI binary
|
||||
- `python-<version>/lib/python<X.Y>/site-packages/ifcopenshell/_ifcopenshell_wrapper*.so`
|
||||
and `ifcopenshell_wrapper.py` - the compiled wrapper + its generated
|
||||
Python glue
|
||||
|
||||
## Wiring the build into a checkout for testing
|
||||
## Testing against a checkout (automated / AI-driven)
|
||||
|
||||
`_ifcopenshell_wrapper*.so` and `ifcopenshell_wrapper.py` are already
|
||||
gitignored under `src/ifcopenshell-python/ifcopenshell/`, which is exactly
|
||||
@@ -98,40 +116,71 @@ tests.) You'll need the matching Python version's `pip install`s too
|
||||
modules under test import) since this is a bare interpreter, not the
|
||||
project's pixi env.
|
||||
|
||||
## Known gotchas (some fixed in this copy, watch for them if you're on an
|
||||
## older/different copy of this script)
|
||||
**This is the pattern to use for automated or AI-driven verification.**
|
||||
Don't use `try` (below) for that - it overwrites files in a real, live
|
||||
Blender installation, which isn't something an automated/AI workflow
|
||||
should ever do without the human explicitly asking for it in the moment.
|
||||
|
||||
- **`try` is an unimplemented stub** - it prints a message and does
|
||||
nothing. If you want the wrapper pushed straight into a Blender
|
||||
extensions folder for manual testing, do the copy yourself (see the
|
||||
README's example path) rather than relying on `try`.
|
||||
- **`stop`/`down` removes the container**, it does not pause it (it's
|
||||
literally `docker compose down`). Named volumes (ccache) and the
|
||||
bind-mounted `build/` survive, so nothing is really lost - `up` just has
|
||||
to recreate the container - but don't expect `docker ps -a` to still
|
||||
show it afterwards.
|
||||
## Testing in Blender itself (human only)
|
||||
|
||||
`try` copies the built wrapper straight into your actual Blender/Bonsai
|
||||
extension install, for manual in-Blender testing:
|
||||
|
||||
```bash
|
||||
./ifcos_env try
|
||||
```
|
||||
|
||||
It reads `BLENDER_USER_RESOURCE` from `.env` - set this to wherever
|
||||
Blender's user resource folder for the Bonsai extension actually lives on
|
||||
your system, which depends on your own Blender setup:
|
||||
|
||||
```bash
|
||||
# in docker/.env
|
||||
BLENDER_USER_RESOURCE=~/.config/blender/bonsai/
|
||||
```
|
||||
|
||||
`try` figures out the built Python version from `build/.../install/`
|
||||
(disambiguating with `PY_TGT` if more than one version was built) and
|
||||
copies the wrapper to
|
||||
`$BLENDER_USER_RESOURCE/extensions/.local/lib/python<X.Y>/site-packages/ifcopenshell/`.
|
||||
|
||||
## Container user
|
||||
|
||||
The image runs as a non-root `builder` user, UID/GID matching your host
|
||||
account (passed as `--build-arg` by `create` from `id -u`/`id -g`, so it
|
||||
adjusts automatically - no manual flag needed even if you're not 1000:1000).
|
||||
Files the build creates under the bind mount come out owned by you, not
|
||||
root. Passwordless `sudo` is available inside the container (e.g. via
|
||||
`attach`) for the rare case you need root for something ad hoc.
|
||||
|
||||
If you're picking up an existing checkout that was previously built with
|
||||
an older, root-based image, you may hit `Permission denied` the first time
|
||||
you run `up`/`build` under the new image - `build/`, `.git/modules/`, the
|
||||
`ccache` volume, `output/`, and `build.log` can all be left root-owned from
|
||||
before. Fix it once via the container's own root (no host `sudo` needed):
|
||||
|
||||
```bash
|
||||
docker exec -u root -w /__w/IfcOpenShell/IfcOpenShell <container-name> \
|
||||
chown -R "$(id -u)":"$(id -g)" .git/modules build output build.log /ccache
|
||||
```
|
||||
|
||||
(`<container-name>` is `ifcopenshell-<UNIQUE_ID>` - see `docker ps -a`.)
|
||||
|
||||
## Other things worth knowing
|
||||
|
||||
- **Linux x64 only.** `compose.yaml` pins `platform: linux/amd64`; on an
|
||||
ARM host (e.g. Apple Silicon) this build isn't available.
|
||||
- **The final "Package .zip archives" step of `build()` has a pre-existing
|
||||
bash syntax error** unrelated to compilation - the actual build already
|
||||
bash syntax error**, unrelated to compilation - the actual build already
|
||||
succeeded by that point (look for `Built IfcOpenShell...` in the output),
|
||||
so this is safe to ignore if you only need the raw artifacts under
|
||||
`build/.../install/`, not packaged release zips.
|
||||
- **`ready_repo` originally cloned the third-party dependency cache one
|
||||
directory level too shallow** (`../build` instead of `build`, relative to
|
||||
the repo root), so `nix/build-all.py` would never find it and silently
|
||||
rebuild every dependency (boost, OCCT, CGAL, ...) from source - "did the
|
||||
build finish in ~1 minute, or is it grinding for 40+ minutes reconfiguring
|
||||
OCCT" is the tell. Fixed in this copy; if `up` seems to be building
|
||||
dependencies that should already be cached, check `ready_repo`'s `cd`
|
||||
targets first.
|
||||
- **On a brand-new `UNIQUE_ID`/folder, `up` used to fail on the very first
|
||||
run** because `ready_repo` tried to `docker exec` into the container
|
||||
before `docker compose up -d` had created it. Also fixed in this copy
|
||||
(container creation now happens first); if you see
|
||||
`Error response from daemon: No such container` right after "Getting the
|
||||
repo ready to build...", just run `up` again.
|
||||
- **Root-partition disk space**: only the bind-mounted `<repo>/build` lives
|
||||
on the host filesystem your repo is checked out on. Anything the
|
||||
container writes *outside* that mount (stray files, apt/dnf state, etc.)
|
||||
lives in the container's own writable layer under Docker's data root
|
||||
(commonly `/var/lib/docker`, i.e. usually your root partition) - keep an
|
||||
eye on `df -h /` if you're running several of these containers at once.
|
||||
- **`test_mmaped_stream` and similar `USE_MMAP`-dependent tests will fail**
|
||||
against this build - `nix/build-all.py` is invoked with `USE_MMAP=OFF`
|
||||
here. Not a bug in your code if you see it fail.
|
||||
- Only the bind-mounted `<repo>/build` lives on the host filesystem your
|
||||
repo is checked out on. Anything the container writes *outside* that
|
||||
mount lives in the container's own writable layer under Docker's data
|
||||
root (commonly `/var/lib/docker`, i.e. usually your root partition) -
|
||||
keep an eye on `df -h /` if you're running several of these containers
|
||||
at once.
|
||||
|
||||
Reference in New Issue
Block a user