Harden docker build tooling: non-root, clean lifecycle, try()

Dockerfile (renamed from Dockerfile_init, Dockerfile_update removed):
- Run as a non-root `builder` user matching the host UID/GID (passed as
  --build-arg by create() from id -u/id -g), so build output under the
  bind mount stays owned by the host user instead of root.
- Fix CCACHE_MAXSIZE: `ccache -M 5G` wrote its limit to a config file
  under /ccache at image-build time, but /ccache is a volume mount
  point, so that file gets shadowed by the (empty) volume the moment
  the container actually runs - the cap never took effect. Set
  CCACHE_MAXSIZE=5G as an image ENV instead.
- Dedupe ccache/libffi-devel, add --setopt=install_weak_deps=False
  --setopt=tsflags=nodocs, add `git lfs install --system`, combine the
  dnf update+install into one layer.
- Drop Dockerfile_update: it built FROM its own previous output, so
  every `update` call made the image strictly larger forever (Docker
  layers are append-only, `dnf clean` in a later layer can't shrink an
  earlier one). `update` now just calls create(), which already runs
  `dnf update -y` FROM a clean rockylinux:9 every time.

compose.yaml: pin platform: linux/amd64 so this doesn't silently run
under emulation on an ARM host.

ifcos_env:
- Split the previously-conflated stop/down into six distinct,
  Compose-native lifecycle commands: up (create-or-start), down
  (remove), stop, start, restart (stop+start, same container),
  recreate (down+up, fresh container). Previously `stop` was aliased
  to `down`, which silently removed the container instead of pausing
  it.
- Implement try(): copies the built wrapper into a real Blender/Bonsai
  install for manual testing, reading the target from a new
  BLENDER_USER_RESOURCE .env variable and auto-detecting the built
  Python version (disambiguating via PY_TGT for multi-version builds).
  Deliberately kept human-only - it mutates a live Blender install, so
  it shouldn't run unattended as part of an automated/AI workflow,
  which should instead copy the wrapper into the repo's own
  src/ifcopenshell-python/ifcopenshell/ (documented in SKILL.md).
- Fix unique(): the "has .env already got a UNIQUE_ID line" check
  referenced an unset $FILE instead of $ENV_FILE, so it always
  evaluated true and appended a fresh "UNIQUE_ID=dummy" line to .env
  on every single `up`.
- Minor: differentiate remove()'s log message from down()'s (no longer
  identical now that they're distinct operations), tidy help text
  alignment and a stray double-space typo in clean().

SKILL.md: rewritten as current-state documentation (no more "fixed in
this copy" changelog framing) covering the above, plus a migration
note for anyone hitting root-owned leftovers from an older image.

Verified by actually building the image and driving every new
lifecycle command (stop/start/restart keep the same container ID;
down+up and recreate produce a new one) and try() (including the
quoted-tilde BLENDER_USER_RESOURCE edge case) against the real container.

Generated with the assistance of an AI coding tool.

(cherry picked from commit 92c50ed3b4)
This commit is contained in:
Stephen Boddy
2026-07-12 20:35:01 +01:00
committed by Dion Moult
parent 68b234f59b
commit 316dace11a
8 changed files with 280 additions and 135 deletions
+93 -44
View File
@@ -13,13 +13,11 @@ description: >-
# Building IfcOpenShell locally with docker/ifcos_env
`docker/` is a small toolchain (see `docker/README.md` for the original
author's own description and design rationale - read that first for the
*why*; this file is the practical *how*, distilled from actually driving it
end-to-end) that mirrors the project's GitHub Actions build environment
locally, with a persistent container and ccache so repeat builds are fast.
Pure-Python changes don't need any of this - only reach for it when you need
a real compiled `_ifcopenshell_wrapper*.so` or `IfcConvert` binary.
`docker/` mirrors the project's GitHub Actions build environment locally,
in a persistent, non-root container with ccache so repeat builds are fast.
See `docker/README.md` for the design rationale. Pure-Python changes don't
need any of this - only reach for it when you need a real compiled
`_ifcopenshell_wrapper*.so` or `IfcConvert` binary.
## Placement
@@ -29,12 +27,12 @@ to build (sibling of `src/`, `cmake/`, etc.) - `compose.yaml` and
itself sits, and bind-mount it into the container. If you're setting this
up in a fresh clone, copy the whole `docker/` directory there first.
## First-time setup
## Setup
```bash
cd docker
./ifcos_env create # build the base image (shared across all your clones/checkouts by name, so usually instant after the first time anywhere)
./ifcos_env up # start the container, clone+unpack the third-party dependency cache (~10GB, one-time per container)
./ifcos_env create # build the image (shared by name across all your clones/checkouts, so usually instant after the first time anywhere)
./ifcos_env up # create + start the container, clone/unpack the third-party dependency cache (~10GB, one-time per container)
./ifcos_env build # full build: all deps + IfcParse + IfcGeom + IfcConvert + the Python wrapper, for one Python version
```
@@ -47,6 +45,25 @@ A full first build takes ~1.5 hours (mostly compiling IfcOpenShell's own
C++, not the cached third-party deps). After that, ccache makes incremental
rebuilds of a couple of touched `.cpp` files **under a minute**.
## Container lifecycle
The container is long-lived (`sleep infinity`) so exec'd commands and
ccache state persist between builds. Commands map directly onto Docker
Compose's own container-vs-image distinction:
```bash
./ifcos_env up # create the container if it doesn't exist, then start it (runs ready_repo too)
./ifcos_env stop # stop the container, keep it around
./ifcos_env start # start it back up (same container, same filesystem layer)
./ifcos_env restart # stop, then start
./ifcos_env down # remove the container (and its network) entirely
./ifcos_env recreate # down, then up - a fresh container
```
Named volumes (`ccache`) and the bind-mounted repo/`build/` are unaffected
by `down`/`recreate` - only the container itself goes away, and `up`
recreates it from the image.
## Fast iteration
Pass a target to `build` to skip the parts you don't need:
@@ -64,14 +81,15 @@ exercising the Python API, build `IfcOpenShell-Python`.
## Where the artifacts land
Build output goes to `<repo_root>/build/Linux/x86_64/install/` on the host
(bind-mounted, not just inside the container):
(bind-mounted, not just inside the container), owned by you (see
"Container user" below):
- `ifcopenshell/bin/IfcConvert` - the CLI binary
- `python-<version>/lib/python<X.Y>/site-packages/ifcopenshell/_ifcopenshell_wrapper*.so`
and `ifcopenshell_wrapper.py` - the compiled wrapper + its generated
Python glue
## Wiring the build into a checkout for testing
## Testing against a checkout (automated / AI-driven)
`_ifcopenshell_wrapper*.so` and `ifcopenshell_wrapper.py` are already
gitignored under `src/ifcopenshell-python/ifcopenshell/`, which is exactly
@@ -98,40 +116,71 @@ tests.) You'll need the matching Python version's `pip install`s too
modules under test import) since this is a bare interpreter, not the
project's pixi env.
## Known gotchas (some fixed in this copy, watch for them if you're on an
## older/different copy of this script)
**This is the pattern to use for automated or AI-driven verification.**
Don't use `try` (below) for that - it overwrites files in a real, live
Blender installation, which isn't something an automated/AI workflow
should ever do without the human explicitly asking for it in the moment.
- **`try` is an unimplemented stub** - it prints a message and does
nothing. If you want the wrapper pushed straight into a Blender
extensions folder for manual testing, do the copy yourself (see the
README's example path) rather than relying on `try`.
- **`stop`/`down` removes the container**, it does not pause it (it's
literally `docker compose down`). Named volumes (ccache) and the
bind-mounted `build/` survive, so nothing is really lost - `up` just has
to recreate the container - but don't expect `docker ps -a` to still
show it afterwards.
## Testing in Blender itself (human only)
`try` copies the built wrapper straight into your actual Blender/Bonsai
extension install, for manual in-Blender testing:
```bash
./ifcos_env try
```
It reads `BLENDER_USER_RESOURCE` from `.env` - set this to wherever
Blender's user resource folder for the Bonsai extension actually lives on
your system, which depends on your own Blender setup:
```bash
# in docker/.env
BLENDER_USER_RESOURCE=~/.config/blender/bonsai/
```
`try` figures out the built Python version from `build/.../install/`
(disambiguating with `PY_TGT` if more than one version was built) and
copies the wrapper to
`$BLENDER_USER_RESOURCE/extensions/.local/lib/python<X.Y>/site-packages/ifcopenshell/`.
## Container user
The image runs as a non-root `builder` user, UID/GID matching your host
account (passed as `--build-arg` by `create` from `id -u`/`id -g`, so it
adjusts automatically - no manual flag needed even if you're not 1000:1000).
Files the build creates under the bind mount come out owned by you, not
root. Passwordless `sudo` is available inside the container (e.g. via
`attach`) for the rare case you need root for something ad hoc.
If you're picking up an existing checkout that was previously built with
an older, root-based image, you may hit `Permission denied` the first time
you run `up`/`build` under the new image - `build/`, `.git/modules/`, the
`ccache` volume, `output/`, and `build.log` can all be left root-owned from
before. Fix it once via the container's own root (no host `sudo` needed):
```bash
docker exec -u root -w /__w/IfcOpenShell/IfcOpenShell <container-name> \
chown -R "$(id -u)":"$(id -g)" .git/modules build output build.log /ccache
```
(`<container-name>` is `ifcopenshell-<UNIQUE_ID>` - see `docker ps -a`.)
## Other things worth knowing
- **Linux x64 only.** `compose.yaml` pins `platform: linux/amd64`; on an
ARM host (e.g. Apple Silicon) this build isn't available.
- **The final "Package .zip archives" step of `build()` has a pre-existing
bash syntax error** unrelated to compilation - the actual build already
bash syntax error**, unrelated to compilation - the actual build already
succeeded by that point (look for `Built IfcOpenShell...` in the output),
so this is safe to ignore if you only need the raw artifacts under
`build/.../install/`, not packaged release zips.
- **`ready_repo` originally cloned the third-party dependency cache one
directory level too shallow** (`../build` instead of `build`, relative to
the repo root), so `nix/build-all.py` would never find it and silently
rebuild every dependency (boost, OCCT, CGAL, ...) from source - "did the
build finish in ~1 minute, or is it grinding for 40+ minutes reconfiguring
OCCT" is the tell. Fixed in this copy; if `up` seems to be building
dependencies that should already be cached, check `ready_repo`'s `cd`
targets first.
- **On a brand-new `UNIQUE_ID`/folder, `up` used to fail on the very first
run** because `ready_repo` tried to `docker exec` into the container
before `docker compose up -d` had created it. Also fixed in this copy
(container creation now happens first); if you see
`Error response from daemon: No such container` right after "Getting the
repo ready to build...", just run `up` again.
- **Root-partition disk space**: only the bind-mounted `<repo>/build` lives
on the host filesystem your repo is checked out on. Anything the
container writes *outside* that mount (stray files, apt/dnf state, etc.)
lives in the container's own writable layer under Docker's data root
(commonly `/var/lib/docker`, i.e. usually your root partition) - keep an
eye on `df -h /` if you're running several of these containers at once.
- **`test_mmaped_stream` and similar `USE_MMAP`-dependent tests will fail**
against this build - `nix/build-all.py` is invoked with `USE_MMAP=OFF`
here. Not a bug in your code if you see it fail.
- Only the bind-mounted `<repo>/build` lives on the host filesystem your
repo is checked out on. Anything the container writes *outside* that
mount lives in the container's own writable layer under Docker's data
root (commonly `/var/lib/docker`, i.e. usually your root partition) -
keep an eye on `df -h /` if you're running several of these containers
at once.