a2bee4b5dc07d04f52d279434908d100ca3ec05a
9 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
| 5bb24235bd |
fix(linux/build): tolerate find perm-denied in chroot scan (M1.1 iter24)
🎉 Run #4271's Build A actually produced the ISO. derivative-maker ran clean for 15:24: INFO: Script ./derivative-maker completed. Exit Code: 0. Errors Detected: 0. Execution Time: 00:15:24 '/home/user/derivative-binary/.../Kicksecure-CLI-18.1.7.4-developers-only.Intel_AMD64.iso' -> '/workspace/SilverLABS/SilverMetal/build-a/Kicksecure-CLI-18.1.7.4-developers-only.Intel_AMD64.iso' …but build-inner.sh then died on its own post-build collection step: find: '.../live-build/chroot/usr/src': Permission denied find: '.../live-build/chroot/etc/sudoers.d': Permission denied find: '.../live-build/chroot/boot': Permission denied … The chroot's standard hardened subdirs (/usr/src, /etc/sudoers.d, /etc/cron.*, /boot, /root, /run/{sudo,lvm,cryptsetup,openvpn-{client, server}}, cache/bootstrap/root) are 0700 root-owned because the live-build chroot was assembled under sudo. As `user` (uid 1000) we can't descend them. find emits Permission denied on each, exits with status 1, and `set -euo pipefail` in build-inner.sh propagates that through `xargs cp` and aborts — even though the ISO copy itself had already succeeded a few lines earlier in the same xargs stream. Fix: redirect find's stderr to /dev/null and tolerate non-zero exit on both the *.iso and *.manifest scans. build.sh already verifies an ISO landed in BUILD_DIR (exit 4 with "no ISO produced" if not), so a real miss is still caught — we just stop killing the script for the benign unreadable-chroot-subdirs case. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
| b0f1ab30f4 |
fix(linux/build): symlink /home/user/derivative-maker to checkout (M1.1 iter23)
Run #4270's Build A made it 2:40 deep — past sanity-tests, prepare- build-machine, local-deps, into 2100_create-debian-packages — then died on: + /workspace/.../genmkfile/usr/bin/genmkfile reprepro-remove running: dm-reprepro-wrapper remove local age-api + /usr/bin/dm-reprepro-wrapper: line 28: /home/user/derivative-maker/help-steps/pre: No such file or directory Earlier `dm-reprepro-wrapper includedsc/includedeb` calls succeeded because 2100_create-debian-packages invokes them by absolute path (`$source_code_folder_dist/packages/.../developer-meta-files/usr/bin/ dm-reprepro-wrapper`) — the in-repo copy resolves help-steps/pre relative to its own location. `genmkfile reprepro-remove` calls `dm-reprepro-wrapper` via PATH instead, so the system copy at /usr/bin/dm-reprepro-wrapper wins. That copy was installed by 1500_local-deps `apt install`-ing the in-repo developer-meta-files.deb into the silvermetal-builder image at runtime. The .deb's intended layout assumes the matching derivative-maker checkout lives at /home/user/derivative-maker — the upstream-blessed path. Ours is at /workspace/SilverLABS/SilverMetal/linux/build/ derivative-maker, so the relative source() at line 28 walks off into nowhere. Bridge the gap with a symlink at the start of build-inner.sh: ln -sfn "${REPO_ROOT}/linux/build/derivative-maker" \ /home/user/derivative-maker That keeps our self-referential CI bind-mount topology (we still cd into REPO_ROOT/.../derivative-maker, derivative-maker still computes paths relative to itself), but also makes the system copy of dm-reprepro-wrapper find help-steps/pre and friends. Both reprepro wrappers (in-repo and system-installed) now resolve to the same files via the symlink, so the silvermetal-reprepro-wrap.sh PATH precedence shadow at /usr/local/bin/reprepro keeps applying to both code paths. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
| 4aa59ba633 |
fix(linux/build): non-interactive mode + visible output + key import (M1.1)
Some checks failed
Build SilverMetal Linux ISO (reproducibility-gated) / build-and-verify (push) Failing after 11m33s
Run #4260 cleared every harness layer and ran for 18 minutes — past sanity-tests, prepare-build-machine, cowbuilder-setup, local-deps — into 2100_create-debian-packages, where it died on: Could not check validity of signature with '92978A6E195E4921825F7FF0F34F09744E9F5DD9' in '/home/user/derivative-binary/temp_packages_debian_sid/virtualbox_7.2.8-dfsg-1.dsc' as public key missing! …and then *also* hung the runner indefinitely because, on any error, derivative-maker's exception_handler_general detected a TTY (we passed `docker run -t`) and dropped into an interactive `read -p 'Answer? '` prompt that nothing was ever going to answer. The orphan docker run in turn orphaned the act_runner job container, blocking the runner until manual cleanup. Three coordinated fixes, validated end-to-end with docker-side smoke tests on 10.0.0.51: 1. **Non-interactive mode without losing output visibility.** The original architectural goal: keep derivative-maker out of interactive mode (`[ -t 0 ]` must be false) AND keep the build log visible to docker run / Gitea Actions (PTY needed somewhere). Resolution: - `docker run -t` is kept (required for /dev/console to be a real PTY back to docker), but no `-i`, so fd 0 stays /dev/null. - docker-entrypoint.service: `StandardInput=tty-force` → `StandardInput=null` so the service's fd 0 is /dev/null too. Verified inside the container: `[ -t 0 ]` returns false. - entrypoint.sh now wraps the user command with an explicit `> /dev/console 2>&1` redirect before writing it to /etc/docker-entrypoint-cmd. systemd's `StandardOutput=inherit` does NOT propagate PID-1's stdout to services in this PID-1- systemd-in-container topology — the service log was going nowhere visible. /dev/console under `docker run -t` IS the allocated PTY back to docker, so the redirect surfaces the log to the act_runner / Gitea Actions log. - entrypoint.sh's `[ ! -t 0 ] && exit 1` guard removed (it would now always trigger). 2. **debian-keyring for reprepro source-package signature checks.** 2100_create-debian-packages calls dm-reprepro-wrapper includedsc on every .dsc in temp_packages_debian_sid (including virtualbox_*.dsc, even for `--target iso` — see line 114 of that build step). reprepro verifies the dsc signature against the user's GPG keyring; without the maintainer keys it fails. Adds `debian-keyring` to Dockerfile.builder. build-inner.sh now imports debian-keyring.gpg / debian-maintainers.gpg / debian-nonupload.gpg into the user's keyring before running derivative-maker. 3. **BUILDER_IMAGE digest re-pinned.** Built natively on 10.0.0.51 (per memory: never on WSL/aarch64). New digest: sha256:2f680c96…f0db. Smoke-test results (against this exact image): ==> START ← user output reaches docker stdout (keyring present) ← debian-keyring imported successfully STDIN_NOT_TTY ← derivative-maker WILL stay non-interactive ==> END ← clean shutdown docker run exit: 42 ← exit code propagates correctly on failure Files: Dockerfile.builder, systemd-entrypoint/entrypoint.sh, systemd-entrypoint/docker-entrypoint.service, scripts/build.sh, scripts/build-inner.sh. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
| 9c406598e2 |
fix(linux/build): pin user_name=user, mkdir derivative-binary (M1.1)
Some checks failed
Build SilverMetal Linux ISO (reproducibility-gated) / build-and-verify (push) Failing after 18m13s
Run #4259 (the systemd-in-container debut) cleared every prior failure class, ran for 15 minutes, then died inside 1100_sanity-tests' aptgetopt_conf_add at: tee: /home/root/derivative-binary/30_derivative-maker.conf: No such file or directory last_failed_bash_command: tee --append -- "$dist_aptgetopt_file" > /dev/null Two compounding bugs: 1. **user_name resolves to "root" via $SUDO_USER** derivative-maker/help-steps/variables (lines 80-93) computes user_name with these fallbacks, in order: [ -n "$user_name" ] || user_name="$SUDO_USER" [ -n "$user_name" ] || user_name="$(logname 2>/dev/null)" if [ -z "$user_name" ] && [ "$(id -u)" != "0" ]; then user_name="$(whoami)" [ -n "$user_name" ] || user_name="$USER" fi build.sh enters the container as root (systemd's docker-entrypoint.service runs as root), then sudoes to user via `sudo --preserve-env -u user --`. sudo always sets SUDO_USER to the *calling* user (= root), regardless of --preserve-env. So variables.sh hits the first fallback and computes user_name="root", then HOMEVAR=/home/root, then binary_build_folder_dist= /home/root/derivative-binary — a directory that does not exist because root's home is /root (not /home/root). Fix: build-inner.sh now exports user_name=user before sourcing the config, satisfying the first-priority check in variables.sh and short-circuiting the SUDO_USER fallback. The comment in the script notes the failure mode for the next reader. 2. **Missing mkdir of derivative-binary** Upstream's derivative-maker-docker-start does: mkdir --parents -- "${HOME}/derivative-binary" before invoking derivative-maker. Our build-inner.sh skipped that because previous iterations didn't reach the point where it mattered. Now that we do, we replicate it. 3. **Output collection path correction** derivative-maker writes its ISO/manifest output into ${HOME}/derivative-binary (per variables.sh:109) — not into the source tree under linux/build/derivative-maker. The previous `find . -maxdepth 6 -type f -name "*.iso"` would have missed everything once we got that far. Updated to `find "${HOME}/derivative-binary" ...`. No image rebuild needed — this is a pure script-and-env change. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
| 38ac4f8a96 |
fix(linux/build): systemd-in-container build host (M1.1)
Some checks failed
Build SilverMetal Linux ISO (reproducibility-gated) / build-and-verify (push) Failing after 15m34s
Run #4258 cleared the systemctl shim only to die two seconds later on the *next* expectation derivative-maker has of a real systemd host: its sources.list points at http://127.0.0.1:9977/debian (the approx package-cache socket-activated by systemd) and apt-get update could not reach the daemon because nothing was actually started by the no-op shim: Err:1 http://127.0.0.1:9977/debian trixie InRelease Could not connect to 127.0.0.1:9977 (127.0.0.1). - connect (111: Connection refused) Whack-a-mole'ing each service derivative-maker tries to start (approx today, then journald, then systemd-logind, then who-knows-what tomorrow) is going to keep failing for a while — derivative-maker is fundamentally designed for a real systemd-managed Debian host. The container pattern upstream itself ships (linux/build/derivative-maker/docker/) runs systemd as PID 1 inside the container; this commit adopts that approach. Architecture: - PID 1 in the build container is now systemd. Upstream's vendored entrypoint.sh records the user-supplied command into /etc/docker-entrypoint-cmd, captures env into /etc/docker-entrypoint-env, masks irrelevant units, and execs systemd. systemd boots, docker-entrypoint.service runs the command, docker-entrypoint-stop.sh propagates the exit code via `systemctl exit <code>` so the container exits with the right status. - The four entrypoint files (entrypoint.sh, docker-entrypoint.service / .target, docker-entrypoint-stop.sh) are vendored at linux/build/docker/systemd-entrypoint/ rather than COPY'd from the submodule path — Docker build context can only reach below itself, and bumping is tracked in that dir's README. - Container runtime now requires --cgroupns=host, --tmpfs /run, --tmpfs /run/lock, and -v /sys/fs/cgroup:/sys/fs/cgroup:rw so systemd can manage cgroups properly. -t allocates a TTY, satisfying entrypoint.sh's `[ ! -t 0 ] && exit 1` check in CI where stdin is otherwise /dev/null. - User renamed builder → user (uid 1000, passwordless sudo) to match upstream's USER=user / HOME=/home/user convention. chown in build.sh now uses uid 1000:1000 so it's name-agnostic. - Image package list grew to match upstream's derivative-maker-docker-setup (sq stack + dbus + approx + the rest) plus our ISO toolchain (live-build / debootstrap / xorriso / squashfs-tools / etc.). Snapshot.debian.org pinning is preserved (same APT_SNAPSHOT_URL, two-phase install pattern). Verified: Smoke test on 10.0.0.51 — `docker run --rm --privileged --cgroupns=host --tmpfs /run --tmpfs /run/lock -v /sys/fs/cgroup:...:rw -t <image> /bin/bash -c 'echo OK'` — booted systemd, ran the command via docker-entrypoint.service, captured the output, shut down filesystems and exited cleanly. build.sh BUILDER_IMAGE pin → sha256:dc9dd29d…8811. Image rebuilt natively on 10.0.0.51, pushed to docker-registry.silverlabs.uk. The systemctl shim is removed by virtue of the Dockerfile rewrite — real systemd makes it unnecessary. The previous "iter6 / iter7" intermediate digests stay in the registry until we GC; the live one is m1.1-iter8-systemd. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
| 8a3cd0ba22 |
fix(linux/build): allow untagged / uncommitted submodule commits (M1.1)
Some checks failed
Build SilverMetal Linux ISO (reproducibility-gated) / build-and-verify (push) Failing after 1m24s
Run #4256 finally cleared every preceding obstacle and reached git_sanity_test's per-submodule verification phase. sq-git authenticated every commit signature in the chain — that part is working perfectly — but failed at: ERROR: Untagged commit in: qubes/qubes-template-kicksecure INFO: As a developer or advanced user you might want to use: WARNING: This can be insecure if you cannot audit the changes. --allow-untagged true --allow-uncommitted true git_sanity_test runs two orthogonal checks: 1. signatures (sq-git, verified ✅) 2. tagged-commit-only mode (verified ❌ for one submodule) The pinned upstream tag (18.1.7.4-developers-only — the name itself flags the intent) deliberately ships with some submodule pointers at intermediate / merge commits rather than release tags. parse-cmd documents `--allow-untagged true` and `--allow-uncommitted true` for exactly this case. Signatures remain verified; we're only relaxing the release-tag check, which is appropriate when we've deliberately pinned to a developer tag. If/when we move to a redistributable upstream tag in M1.10+ (signing ceremony milestone), these flags should come back out. No image rebuild needed — script-only change. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
| 4a3971cb06 |
fix(linux/build): correct derivative-maker CLI invocation (M1.1)
Some checks failed
Build SilverMetal Linux ISO (reproducibility-gated) / build-and-verify (push) Failing after 1m13s
Run #4253 finally got past all the harness failures and into derivative-maker's actual build steps, where 1100_sanity-tests rejected our invocation with: unknown option (1): '--build' The CLI we'd been passing was built from invented flag names rather than the real grammar in derivative-maker/help-steps/parse-cmd. Concretely: - `--build` is not a real option (just wrong) - `--flavour` should be `--flavor` (upstream uses American spelling) - `--dist` is not a real option; dist is implicit from `--flavor` (kicksecure-cli ⇒ bookworm) - `--config` is not a real option; the silvermetal-base.conf is sourced into env above the invocation, no flag needed - `--freedom true|false` was missing entirely; parse-cmd requires it for `--arch amd64` (line 70 in parse-cmd) — the script exits if neither is set Fix: build-inner.sh now invokes ./derivative-maker --flavor … --target … --arch … --freedom … which is the minimal valid form per parse-cmd's case-branches. Set DERIVATIVE_FREEDOM=false in silvermetal-base.conf, matching Kicksecure's own public-ISO choice — `--freedom true` would omit firmware-nonfreedom and the resulting ISO wouldn't initialise wifi / many GPUs / Intel microcode on most hardware. Privacy/functionality trade-off documented inline; the hardening overlay in M1.2+ can revisit if that conversation becomes useful. Verified: bash -n on both scripts. No image rebuild needed — pure script and config changes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
| bf55a3f81c |
fix(linux/build): mark build-inner.sh executable (M1.1)
Some checks failed
Build SilverMetal Linux ISO (reproducibility-gated) / build-and-verify (push) Failing after 1m14s
Run #4252 died at: runuser: failed to execute /workspace/SilverLABS/SilverMetal/linux/build/scripts/build-inner.sh: Permission denied The script was created on the WSL/Windows side (/mnt/c) where every file appears world-rwx regardless of git's index, so the local `chmod +x` was a no-op as far as git was concerned and the file got committed at mode 100644 like any other regular file. Sibling scripts (build.sh, verify-reproducibility.sh, diagnose-divergence.sh) all correctly carry 100755 in the index. Fix: `git update-index --chmod=+x` to set the bit in the index explicitly, independent of the working-tree perms. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
| b20e568b19 |
fix(linux/build): run derivative-maker as unprivileged builder user (M1.1)
Some checks failed
Build SilverMetal Linux ISO (reproducibility-gated) / build-and-verify (push) Failing after 1m14s
Run #4251 advanced past checkout and into derivative-maker, then died immediately: ERROR: This must NOT be run as root (sudo)! ERROR: Exiting ./derivative-maker with non-zero exit code 1. Errors Detected: 0. Execution Time: 00:00:00. Kicksecure's derivative-maker explicitly refuses to run as root — it expects a regular user with passwordless sudo and uses sudo internally for the privileged operations (debootstrap, mksquashfs, chroot mounts). Our minimal debian-slim builder image had a `builder` user (uid 1000) but no sudo, no sudoers entry, and the container ran as root. Aligns with the upstream Kicksecure container pattern at linux/build/derivative-maker/docker/derivative-maker-docker-setup (uses USER=user with `${USER} ALL=(ALL) NOPASSWD:ALL`). Changes: - Dockerfile.builder: install `sudo` (and `fakeroot` while we're here — upstream sanity-tests pulls this in via apt at build time, but having it baked avoids a snapshot.debian.org round-trip every run); add passwordless sudoers entry for builder; correct the misleading comment that claimed root was needed. - New scripts/build-inner.sh: the inner derivative-maker invocation pulled out of build.sh's heredoc. Once we needed to drop privileges via runuser, the nested-heredoc / nested-quoting situation became unmaintainable; a regular script with normal quoting is far cleaner. - build.sh: inner heredoc now just chowns the workspace to builder and runuser's into build-inner.sh. ${REPO_ROOT} and ${BUILD_DIR} continue to be forwarded into the container via -e. - build.sh: BUILDER_IMAGE digest re-pinned to sha256:f8f0db37…1bedc (rebuilt and pushed natively on 10.0.0.51 — never on the WSL/aarch64 dev box, see reference_silvermetal_runner.md memory). Verified: bash -n on both scripts; image builds and pushes cleanly. Pushing this commit triggers a fresh CI run that will exercise it. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
