ECS-Public-Infra/Create-Templates
2
0
Fork 0
Code Issues 6 Pull requests 2 Projects Releases 8 Packages 2 Wiki Activity Actions

ECS01 build: etckeeper git add . busy-loops in-chroot (read(fd)=0), hangs devuan-ecs01 + debian13 #1

New issue
Open
opened 2026-06-16 23:26:16 +00:00 by hclaude · 1 comment
hclaude commented 2026-06-16 23:26:16 +00:00
Owner
Copy link

Summary

Building the ECS01 templates (devuan-ecs01, debian13-ecs01) via src/build-all-templates.sh on a non-PVE Devuan host (Hclaude01) hangs indefinitely during the ECS01 package phase. etckeeper's postinst spawns git add . in the chroot /etc and that git process pins one CPU core at 99.9% (state R) with .git/index.lock at 0 bytes — zero progress after 20+ minutes. Both ECS01 variants reproduce identically.

Root cause (strace of the live PID)

Tight loop of read(6, "", 8) = 0 — ~139,000 calls in 3 seconds. git is busy-reading a file descriptor stuck at EOF (an etckeeper hook / fsmonitor pipe inside the chroot).

What it is NOT

  • Not the /etc content and not ZFS: git add . on a copy of the exact chroot /etc completes in <1 min on both tmpfs and a fresh ZFS dir (718 files) with host git.
  • Not a template defect: the released template v1.5.6 builds fine on a real PVE host. Environmental to building these chroots on this host (in-chroot git/etckeeper interaction).

Impact

clean Devuan (no etckeeper) builds + uploads fine. devuan-ecs01 + debian13-ecs01 never produce a tarball on this host.

Proposed workaround (build-host only, does not change the released template)

Before installing etckeeper in the ECS01 phase, pre-seed /etc/etckeeper/etckeeper.conf with VCS="" in the chroot so etckeeper's postinst skips the git auto-init/commit (template still ships etckeeper; it just isn't auto-initialised, which is harmless — etckeeper init works on first use). Alternative: temporarily dpkg-divert etckeeper's init during build.

Follow-ups

  • Confirm exact fd 6 source (fsmonitor vs hook) for a precise upstream-quality note.
  • Decide whether to bake the workaround into build-devuan-template.sh guarded by a non-PVE/build-host flag, or keep it as a host-side wrapper.

Evidence captured in reports/2026-06-16-build-session.md.

## Summary Building the ECS01 templates (devuan-ecs01, debian13-ecs01) via `src/build-all-templates.sh` on a non-PVE Devuan host (Hclaude01) hangs indefinitely during the ECS01 package phase. etckeeper's postinst spawns `git add .` in the chroot `/etc` and that git process pins one CPU core at 99.9% (state R) with `.git/index.lock` at 0 bytes — zero progress after 20+ minutes. Both ECS01 variants reproduce identically. ## Root cause (strace of the live PID) Tight loop of `read(6, "", 8) = 0` — ~139,000 calls in 3 seconds. git is busy-reading a file descriptor stuck at EOF (an etckeeper hook / fsmonitor pipe inside the chroot). ## What it is NOT - Not the `/etc` content and not ZFS: `git add .` on a copy of the exact chroot `/etc` completes in <1 min on both tmpfs and a fresh ZFS dir (718 files) with host git. - Not a template defect: the released template v1.5.6 builds fine on a real PVE host. Environmental to building these chroots on this host (in-chroot git/etckeeper interaction). ## Impact clean Devuan (no etckeeper) builds + uploads fine. devuan-ecs01 + debian13-ecs01 never produce a tarball on this host. ## Proposed workaround (build-host only, does not change the released template) Before installing etckeeper in the ECS01 phase, pre-seed `/etc/etckeeper/etckeeper.conf` with `VCS=""` in the chroot so etckeeper's postinst skips the git auto-init/commit (template still ships etckeeper; it just isn't auto-initialised, which is harmless — `etckeeper init` works on first use). Alternative: temporarily `dpkg-divert` etckeeper's init during build. ## Follow-ups - Confirm exact fd 6 source (fsmonitor vs hook) for a precise upstream-quality note. - Decide whether to bake the workaround into `build-devuan-template.sh` guarded by a non-PVE/build-host flag, or keep it as a host-side wrapper. Evidence captured in `reports/2026-06-16-build-session.md`.
hclaude commented 2026-06-17 20:40:30 +00:00
Author
Owner
Copy link

Update — verified + opt-in workaround committed

  • Tested the hypothesis that the missing /dev/fd on the build host (chroot
    inherits host /dev) caused the loop. Disproved: with /dev/fd -> /proc/self/fd in place, an unmodified devuan-ecs01 build still hangs
    (git pid … >90% CPU ~90s, same read()=0 loop). The /dev/fd fix only
    resolved the separate forgejo-curl/publish failure.
  • Committed an opt-in workaround in 7c64a26: SKIP_ETCKEEPER_AUTOINIT=1
    pre-seeds etckeeper.conf VCS="" (skips the postinst git init) + passes
    --force-confold/confdef on the etckeeper-pulling install, in both ECS01
    phases. Default OFF so real-PVE builds are byte-identical. Templates
    built with the flag ship etckeeper installed but /etc VCS un-initialised
    (init at provision time).
  • The 1.5.6 ECS01 tarballs already in the registry were produced with this
    workaround.

Leaving open: still want the precise root cause of the in-chroot read(fd)=0
loop (fd 6 source) for a non-opt-in fix.

### Update — verified + opt-in workaround committed - Tested the hypothesis that the missing `/dev/fd` on the build host (chroot inherits host `/dev`) caused the loop. **Disproved:** with `/dev/fd -> /proc/self/fd` in place, an unmodified `devuan-ecs01` build **still** hangs (`git pid … >90% CPU ~90s`, same `read()=0` loop). The `/dev/fd` fix only resolved the separate `forgejo-curl`/publish failure. - Committed an **opt-in** workaround in `7c64a26`: `SKIP_ETCKEEPER_AUTOINIT=1` pre-seeds `etckeeper.conf VCS=""` (skips the postinst git init) + passes `--force-confold/confdef` on the etckeeper-pulling install, in both ECS01 phases. **Default OFF** so real-PVE builds are byte-identical. Templates built with the flag ship etckeeper installed but `/etc` VCS un-initialised (init at provision time). - The 1.5.6 ECS01 tarballs already in the registry were produced with this workaround. Leaving open: still want the precise root cause of the in-chroot `read(fd)=0` loop (fd 6 source) for a non-opt-in fix.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
master
fix/less-ifupdown-162
fix/ubuntu-backports-zfs-161
feat/ubuntu-2404
feat/vm-track-and-cloudinit-plans
fix/vm-keyring-020
VM_0.2.2
v1.6.2
VM_0.2.1
v1.6.1
VM_0.2.0
v1.6.0
v1.5.8
v1.5.7
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
ECS-Public-Infra/Create-Templates#1
No description provided.