fix(linux/build): cover all ISO9660 dates + locate residual byte drift (M1.1 iter34)
Some checks failed
Build SilverMetal Linux ISO (reproducibility-gated) / builder-image (push) Successful in 1s
Build SilverMetal Linux ISO (reproducibility-gated) / build-and-verify (push) Failing after 33m40s

Run #4281 cleared every layer above the ISO9660 wrapper:

    SHA256 (squashfs payload)
    caed117ca72c6c1d9204c49dd749d5f7b372f3a19cac1b2a7e66bee452a8d501  /tmp/.../a.squashfs
    caed117ca72c6c1d9204c49dd749d5f7b372f3a19cac1b2a7e66bee452a8d501  /tmp/.../b.squashfs

…squashfs is now byte-identical, ISO TOC is identical, file listing
diff is empty, but ISO SHA still differs. The remaining drift is in
the ISO9660 metadata region between the system area (first 32 KiB)
and the file payload start.

Two complementary changes:

1. xorriso post-process now sets *every* date field xorriso writes,
   not just the obvious two:

     -alter_date_r all     — atime + mtime + btime on all nodes,
                             not just mtime. ISO9660 directory
                             records carry creation+modification
                             timestamps.
     -volume_date c m x f u s — every volume-descriptor date:
       c=creation  m=modification  x=expiration  f=effective
       u=system area  s=path table
     Default for any unset volume_date is "now", which is what was
     leaking through despite us setting c+m.

2. diagnose-divergence.sh now does whole-file cmp -l (capped at 200
   lines so 1 GiB of all-different doesn't drown the report) and on
   any divergence, dumps a 128-byte xxd window from each ISO around
   the first differing byte plus a unified diff between the two
   windows. This tells us in the next failure log "first byte differs
   at offset N (LBA M), bytes around it look like X" — pinpoints the
   ISO9660 region without needing artifact download.

Workflow tail-into-log step wired up the two new files
(iso-cmp-first-200.txt, iso-around-first-diff.diff).

If iter34 still fails the gate, the new diagnostic tells us exactly
which structure (volume descriptor, path table, directory record,
boot catalog…) is still drifting.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-05-08 00:29:37 +01:00
parent 33e1501611
commit 34bc442dd8
3 changed files with 53 additions and 13 deletions

View File

@@ -237,26 +237,31 @@ post_process_for_reproducibility() {
local new_iso="${iso_file%.iso}.silvermetal-clean.iso"
sudo --non-interactive rm -f "${new_iso}"
echo "post-process: replacing /live/filesystem.squashfs in ISO"
# `-alter_date_r m` rewrites every file's mtime to the pinned epoch
# so the new /live/filesystem.squashfs and the regenerated
# /boot.catalog don't carry the wall-clock time of the post-process
# step (run #4278's TOC diff caught both: 21:27 vs 21:44).
# `-volume_date` covers the volume-descriptor-level dates that
# xorriso also stamps on -commit.
# `-alter_date_r` takes a variable-length path list and needs `--`
# to terminate it — without that, the next `-volume_date` is
# interpreted as a path, producing
# FAILURE : Cannot find path '/-volume_date' in loaded ISO image
# (caught on run #4279).
# Force every date xorriso writes into the ISO9660 structure to the
# pinned epoch.
# -alter_date_r all — atime/mtime/btime on every file & dir
# (just `m` left btime drifting in run #4281: byte-identical
# squashfs, byte-identical TOC, but still-different ISO bytes).
# -volume_date c/m/x/f/u/s — every volume-descriptor date
# (creation, modification, expiration, effective, system area,
# path table). xorriso defaults to "now" for any not-explicitly-set
# volume date.
# `--` terminates the variable-length path list of -alter_date_r;
# without it the following -volume_date is consumed as a path and
# xorriso bails with "Cannot find path '/-volume_date'" (run #4279).
sudo --non-interactive xorriso \
-return_with SORRY 0 \
-indev "${iso_file}" \
-outdev "${new_iso}" \
-boot_image any keep \
-update "${new_sqfs}" /live/filesystem.squashfs \
-alter_date_r m "=${SOURCE_DATE_EPOCH}" / -- \
-alter_date_r all "=${SOURCE_DATE_EPOCH}" / -- \
-volume_date c "=${SOURCE_DATE_EPOCH}" \
-volume_date m "=${SOURCE_DATE_EPOCH}" \
-volume_date x "=${SOURCE_DATE_EPOCH}" \
-volume_date f "=${SOURCE_DATE_EPOCH}" \
-volume_date u "=${SOURCE_DATE_EPOCH}" \
-volume_date s "=${SOURCE_DATE_EPOCH}" \
-commit
sudo --non-interactive mv -f "${new_iso}" "${iso_file}"
sudo --non-interactive rm -f "${new_sqfs}"

View File

@@ -204,8 +204,41 @@ if [[ -s "${SQFS_A}" && -s "${SQFS_B}" ]]; then
fi
fi
# --- Fallback: cmp -l on first KB of the ISOs (catches header-level drift) --
# --- Whole-file cmp + hex around first divergence ---------------------------
# When TOC + squashfs match but ISO SHA still diverges, the bytes that
# differ live in the ISO9660 structure between the system area (first
# 32 KiB) and the file payload. Limit -n to first 1 MiB scanned, capture
# first 200 differing offsets, and dump 128-byte hex windows from each
# ISO around the first divergence so the workflow log shows what region
# we're in.
if command -v cmp >/dev/null 2>&1; then
# First scan the full ISO; cap output at 200 lines so 1 GiB of all-
# different bytes can't drown the artifact.
cmp -l "${ISO_A}" "${ISO_B}" 2>/dev/null \
| head -n 200 \
> "${REPORT_DIR}/iso-cmp-first-200.txt" || true
first_diff_byte=$(awk 'NR==1 {print $1}' \
"${REPORT_DIR}/iso-cmp-first-200.txt" 2>/dev/null)
if [[ -n "${first_diff_byte}" ]]; then
# cmp -l prints 1-indexed positions; convert to 0-indexed.
first_diff_byte=$(( first_diff_byte - 1 ))
lba=$(( first_diff_byte / 2048 ))
echo "diagnose: first ISO byte difference at offset ${first_diff_byte} (LBA ${lba})"
# 128-byte window starting 32 bytes before the diff.
start=$(( first_diff_byte > 32 ? first_diff_byte - 32 : 0 ))
if command -v xxd >/dev/null 2>&1; then
dd if="${ISO_A}" bs=1 skip="${start}" count=128 2>/dev/null \
| xxd > "${REPORT_DIR}/iso-a-around-first-diff.xxd" || true
dd if="${ISO_B}" bs=1 skip="${start}" count=128 2>/dev/null \
| xxd > "${REPORT_DIR}/iso-b-around-first-diff.xxd" || true
diff -u "${REPORT_DIR}/iso-a-around-first-diff.xxd" \
"${REPORT_DIR}/iso-b-around-first-diff.xxd" \
> "${REPORT_DIR}/iso-around-first-diff.diff" 2>/dev/null || true
fi
fi
# Keep the legacy first-8KB scan around as a quick header view.
cmp -l -n 8192 "${ISO_A}" "${ISO_B}" > "${REPORT_DIR}/iso-header-cmp.txt" 2>&1 || true
fi