SilverLABS/SilverMetal
1
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity
21 Commits 4 Branches 0 Tags
e260fe1c819e1da7d622ff72162e88f31b9e1b6d
Commit Graph

7 Commits

Author SHA1 Message Date
SysAdmin
4aa59ba633 fix(linux/build): non-interactive mode + visible output + key import (M1.1)
Some checks failed
Build SilverMetal Linux ISO (reproducibility-gated) / build-and-verify (push) Failing after 11m33s
Details 
Run #4260 cleared every harness layer and ran for 18 minutes — past
sanity-tests, prepare-build-machine, cowbuilder-setup, local-deps —
into 2100_create-debian-packages, where it died on:

    Could not check validity of signature with
    '92978A6E195E4921825F7FF0F34F09744E9F5DD9' in
    '/home/user/derivative-binary/temp_packages_debian_sid/virtualbox_7.2.8-dfsg-1.dsc'
    as public key missing!

…and then *also* hung the runner indefinitely because, on any error,
derivative-maker's exception_handler_general detected a TTY (we passed
`docker run -t`) and dropped into an interactive `read -p 'Answer? '`
prompt that nothing was ever going to answer. The orphan docker run
in turn orphaned the act_runner job container, blocking the runner
until manual cleanup.

Three coordinated fixes, validated end-to-end with docker-side smoke
tests on 10.0.0.51:

1. **Non-interactive mode without losing output visibility.**

   The original architectural goal: keep derivative-maker out of
   interactive mode (`[ -t 0 ]` must be false) AND keep the build log
   visible to docker run / Gitea Actions (PTY needed somewhere).

   Resolution:
   - `docker run -t` is kept (required for /dev/console to be a real
     PTY back to docker), but no `-i`, so fd 0 stays /dev/null.
   - docker-entrypoint.service: `StandardInput=tty-force` →
     `StandardInput=null` so the service's fd 0 is /dev/null too.
     Verified inside the container: `[ -t 0 ]` returns false.
   - entrypoint.sh now wraps the user command with an explicit
     `> /dev/console 2>&1` redirect before writing it to
     /etc/docker-entrypoint-cmd. systemd's `StandardOutput=inherit`
     does NOT propagate PID-1's stdout to services in this PID-1-
     systemd-in-container topology — the service log was going
     nowhere visible. /dev/console under `docker run -t` IS the
     allocated PTY back to docker, so the redirect surfaces the
     log to the act_runner / Gitea Actions log.
   - entrypoint.sh's `[ ! -t 0 ] && exit 1` guard removed (it
     would now always trigger).

2. **debian-keyring for reprepro source-package signature checks.**

   2100_create-debian-packages calls dm-reprepro-wrapper includedsc
   on every .dsc in temp_packages_debian_sid (including
   virtualbox_*.dsc, even for `--target iso` — see line 114 of that
   build step). reprepro verifies the dsc signature against the
   user's GPG keyring; without the maintainer keys it fails.

   Adds `debian-keyring` to Dockerfile.builder. build-inner.sh now
   imports debian-keyring.gpg / debian-maintainers.gpg /
   debian-nonupload.gpg into the user's keyring before running
   derivative-maker.

3. **BUILDER_IMAGE digest re-pinned.**

   Built natively on 10.0.0.51 (per memory: never on WSL/aarch64).
   New digest: sha256:2f680c96…f0db.

Smoke-test results (against this exact image):

    ==> START                  ← user output reaches docker stdout
    (keyring present)          ← debian-keyring imported successfully
    STDIN_NOT_TTY              ← derivative-maker WILL stay non-interactive
    ==> END                    ← clean shutdown
    docker run exit: 42        ← exit code propagates correctly on failure

Files: Dockerfile.builder, systemd-entrypoint/entrypoint.sh,
       systemd-entrypoint/docker-entrypoint.service, scripts/build.sh,
       scripts/build-inner.sh.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-07 14:05:49 +01:00
SysAdmin
9c406598e2 fix(linux/build): pin user_name=user, mkdir derivative-binary (M1.1)
Some checks failed
Build SilverMetal Linux ISO (reproducibility-gated) / build-and-verify (push) Failing after 18m13s
Details 
Run #4259 (the systemd-in-container debut) cleared every prior failure
class, ran for 15 minutes, then died inside 1100_sanity-tests'
aptgetopt_conf_add at:

    tee: /home/root/derivative-binary/30_derivative-maker.conf:
    No such file or directory
    last_failed_bash_command: tee --append -- "$dist_aptgetopt_file" > /dev/null

Two compounding bugs:

1. **user_name resolves to "root" via $SUDO_USER**

   derivative-maker/help-steps/variables (lines 80-93) computes
   user_name with these fallbacks, in order:

       [ -n "$user_name" ]   || user_name="$SUDO_USER"
       [ -n "$user_name" ]   || user_name="$(logname 2>/dev/null)"
       if [ -z "$user_name" ] && [ "$(id -u)" != "0" ]; then
          user_name="$(whoami)"
          [ -n "$user_name" ] || user_name="$USER"
       fi

   build.sh enters the container as root (systemd's
   docker-entrypoint.service runs as root), then sudoes to user via
   `sudo --preserve-env -u user --`. sudo always sets SUDO_USER to the
   *calling* user (= root), regardless of --preserve-env. So
   variables.sh hits the first fallback and computes user_name="root",
   then HOMEVAR=/home/root, then binary_build_folder_dist=
   /home/root/derivative-binary — a directory that does not exist
   because root's home is /root (not /home/root).

   Fix: build-inner.sh now exports user_name=user before sourcing the
   config, satisfying the first-priority check in variables.sh and
   short-circuiting the SUDO_USER fallback. The comment in the script
   notes the failure mode for the next reader.

2. **Missing mkdir of derivative-binary**

   Upstream's derivative-maker-docker-start does:
       mkdir --parents -- "${HOME}/derivative-binary"
   before invoking derivative-maker. Our build-inner.sh skipped that
   because previous iterations didn't reach the point where it
   mattered. Now that we do, we replicate it.

3. **Output collection path correction**

   derivative-maker writes its ISO/manifest output into
   ${HOME}/derivative-binary (per variables.sh:109) — not into the
   source tree under linux/build/derivative-maker. The previous
   `find . -maxdepth 6 -type f -name "*.iso"` would have missed
   everything once we got that far. Updated to
   `find "${HOME}/derivative-binary" ...`.

No image rebuild needed — this is a pure script-and-env change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-07 12:47:47 +01:00
SysAdmin
38ac4f8a96 fix(linux/build): systemd-in-container build host (M1.1)
Some checks failed
Build SilverMetal Linux ISO (reproducibility-gated) / build-and-verify (push) Failing after 15m34s
Details 
Run #4258 cleared the systemctl shim only to die two seconds later on
the *next* expectation derivative-maker has of a real systemd host:
its sources.list points at http://127.0.0.1:9977/debian (the approx
package-cache socket-activated by systemd) and apt-get update could
not reach the daemon because nothing was actually started by the
no-op shim:

    Err:1 http://127.0.0.1:9977/debian trixie InRelease
      Could not connect to 127.0.0.1:9977 (127.0.0.1).
      - connect (111: Connection refused)

Whack-a-mole'ing each service derivative-maker tries to start (approx
today, then journald, then systemd-logind, then who-knows-what
tomorrow) is going to keep failing for a while — derivative-maker is
fundamentally designed for a real systemd-managed Debian host. The
container pattern upstream itself ships
(linux/build/derivative-maker/docker/) runs systemd as PID 1 inside
the container; this commit adopts that approach.

Architecture:

  - PID 1 in the build container is now systemd. Upstream's vendored
    entrypoint.sh records the user-supplied command into
    /etc/docker-entrypoint-cmd, captures env into
    /etc/docker-entrypoint-env, masks irrelevant units, and execs
    systemd. systemd boots, docker-entrypoint.service runs the
    command, docker-entrypoint-stop.sh propagates the exit code via
    `systemctl exit <code>` so the container exits with the right
    status.

  - The four entrypoint files (entrypoint.sh,
    docker-entrypoint.service / .target, docker-entrypoint-stop.sh)
    are vendored at linux/build/docker/systemd-entrypoint/ rather
    than COPY'd from the submodule path — Docker build context can
    only reach below itself, and bumping is tracked in that dir's
    README.

  - Container runtime now requires --cgroupns=host, --tmpfs /run,
    --tmpfs /run/lock, and -v /sys/fs/cgroup:/sys/fs/cgroup:rw so
    systemd can manage cgroups properly. -t allocates a TTY,
    satisfying entrypoint.sh's `[ ! -t 0 ] && exit 1` check in CI
    where stdin is otherwise /dev/null.

  - User renamed builder → user (uid 1000, passwordless sudo) to
    match upstream's USER=user / HOME=/home/user convention. chown
    in build.sh now uses uid 1000:1000 so it's name-agnostic.

  - Image package list grew to match upstream's
    derivative-maker-docker-setup (sq stack + dbus + approx + the
    rest) plus our ISO toolchain (live-build / debootstrap / xorriso
    / squashfs-tools / etc.). Snapshot.debian.org pinning is
    preserved (same APT_SNAPSHOT_URL, two-phase install pattern).

Verified:

  Smoke test on 10.0.0.51 — `docker run --rm --privileged
  --cgroupns=host --tmpfs /run --tmpfs /run/lock -v /sys/fs/cgroup:...:rw
  -t <image> /bin/bash -c 'echo OK'` — booted systemd, ran the
  command via docker-entrypoint.service, captured the output, shut
  down filesystems and exited cleanly.

build.sh BUILDER_IMAGE pin → sha256:dc9dd29d…8811. Image rebuilt
natively on 10.0.0.51, pushed to docker-registry.silverlabs.uk.

The systemctl shim is removed by virtue of the Dockerfile rewrite —
real systemd makes it unnecessary. The previous "iter6 / iter7"
intermediate digests stay in the registry until we GC; the live one
is m1.1-iter8-systemd.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-07 12:06:47 +01:00
SysAdmin
8a3cd0ba22 fix(linux/build): allow untagged / uncommitted submodule commits (M1.1)
Some checks failed
Build SilverMetal Linux ISO (reproducibility-gated) / build-and-verify (push) Failing after 1m24s
Details 
Run #4256 finally cleared every preceding obstacle and reached
git_sanity_test's per-submodule verification phase. sq-git authenticated
every commit signature in the chain — that part is working perfectly —
but failed at:

    ERROR: Untagged commit in: qubes/qubes-template-kicksecure
    INFO: As a developer or advanced user you might want to use:
    WARNING: This can be insecure if you cannot audit the changes.
    --allow-untagged true --allow-uncommitted true

git_sanity_test runs two orthogonal checks:
  1. signatures (sq-git, verified ✅)
  2. tagged-commit-only mode (verified ❌ for one submodule)

The pinned upstream tag (18.1.7.4-developers-only — the name itself
flags the intent) deliberately ships with some submodule pointers at
intermediate / merge commits rather than release tags. parse-cmd
documents `--allow-untagged true` and `--allow-uncommitted true` for
exactly this case. Signatures remain verified; we're only relaxing the
release-tag check, which is appropriate when we've deliberately pinned
to a developer tag.

If/when we move to a redistributable upstream tag in M1.10+ (signing
ceremony milestone), these flags should come back out.

No image rebuild needed — script-only change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-07 11:35:27 +01:00
SysAdmin
4a3971cb06 fix(linux/build): correct derivative-maker CLI invocation (M1.1)
Some checks failed
Build SilverMetal Linux ISO (reproducibility-gated) / build-and-verify (push) Failing after 1m13s
Details 
Run #4253 finally got past all the harness failures and into
derivative-maker's actual build steps, where 1100_sanity-tests
rejected our invocation with:

    unknown option (1): '--build'

The CLI we'd been passing was built from invented flag names rather
than the real grammar in derivative-maker/help-steps/parse-cmd.
Concretely:

  - `--build`  is not a real option (just wrong)
  - `--flavour` should be `--flavor` (upstream uses American spelling)
  - `--dist`   is not a real option; dist is implicit from `--flavor`
                (kicksecure-cli ⇒ bookworm)
  - `--config` is not a real option; the silvermetal-base.conf is
                sourced into env above the invocation, no flag needed
  - `--freedom true|false` was missing entirely; parse-cmd requires it
                for `--arch amd64` (line 70 in parse-cmd) — the script
                exits if neither is set

Fix: build-inner.sh now invokes
    ./derivative-maker --flavor … --target … --arch … --freedom …
which is the minimal valid form per parse-cmd's case-branches.

Set DERIVATIVE_FREEDOM=false in silvermetal-base.conf, matching
Kicksecure's own public-ISO choice — `--freedom true` would omit
firmware-nonfreedom and the resulting ISO wouldn't initialise wifi /
many GPUs / Intel microcode on most hardware. Privacy/functionality
trade-off documented inline; the hardening overlay in M1.2+ can
revisit if that conversation becomes useful.

Verified: bash -n on both scripts. No image rebuild needed — pure
script and config changes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-07 11:18:38 +01:00
SysAdmin
bf55a3f81c fix(linux/build): mark build-inner.sh executable (M1.1)
Some checks failed
Build SilverMetal Linux ISO (reproducibility-gated) / build-and-verify (push) Failing after 1m14s
Details 
Run #4252 died at:

    runuser: failed to execute /workspace/SilverLABS/SilverMetal/linux/build/scripts/build-inner.sh:
    Permission denied

The script was created on the WSL/Windows side (/mnt/c) where every
file appears world-rwx regardless of git's index, so the local
`chmod +x` was a no-op as far as git was concerned and the file got
committed at mode 100644 like any other regular file. Sibling scripts
(build.sh, verify-reproducibility.sh, diagnose-divergence.sh) all
correctly carry 100755 in the index.

Fix: `git update-index --chmod=+x` to set the bit in the index
explicitly, independent of the working-tree perms.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-07 11:13:02 +01:00
SysAdmin
b20e568b19 fix(linux/build): run derivative-maker as unprivileged builder user (M1.1)
Some checks failed
Build SilverMetal Linux ISO (reproducibility-gated) / build-and-verify (push) Failing after 1m14s
Details 
Run #4251 advanced past checkout and into derivative-maker, then died
immediately:

    ERROR: This must NOT be run as root (sudo)!
    ERROR: Exiting ./derivative-maker with non-zero exit code 1.
           Errors Detected: 0. Execution Time: 00:00:00.

Kicksecure's derivative-maker explicitly refuses to run as root — it
expects a regular user with passwordless sudo and uses sudo internally
for the privileged operations (debootstrap, mksquashfs, chroot mounts).
Our minimal debian-slim builder image had a `builder` user (uid 1000)
but no sudo, no sudoers entry, and the container ran as root.

Aligns with the upstream Kicksecure container pattern at
linux/build/derivative-maker/docker/derivative-maker-docker-setup
(uses USER=user with `${USER} ALL=(ALL) NOPASSWD:ALL`).

Changes:
- Dockerfile.builder: install `sudo` (and `fakeroot` while we're here —
  upstream sanity-tests pulls this in via apt at build time, but having
  it baked avoids a snapshot.debian.org round-trip every run); add
  passwordless sudoers entry for builder; correct the misleading
  comment that claimed root was needed.
- New scripts/build-inner.sh: the inner derivative-maker invocation
  pulled out of build.sh's heredoc. Once we needed to drop privileges
  via runuser, the nested-heredoc / nested-quoting situation became
  unmaintainable; a regular script with normal quoting is far cleaner.
- build.sh: inner heredoc now just chowns the workspace to builder and
  runuser's into build-inner.sh. ${REPO_ROOT} and ${BUILD_DIR} continue
  to be forwarded into the container via -e.
- build.sh: BUILDER_IMAGE digest re-pinned to sha256:f8f0db37…1bedc
  (rebuilt and pushed natively on 10.0.0.51 — never on the WSL/aarch64
  dev box, see reference_silvermetal_runner.md memory).

Verified: bash -n on both scripts; image builds and pushes cleanly.
Pushing this commit triggers a fresh CI run that will exercise it.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-07 11:09:42 +01:00