← All entries

Dev Log

Build notes from the Jefe ecosystem

Making the Tests Able to Fail

The Falsifier 2026-08-01

Twenty PRs merged since 1.1.1, eleven issues closed, forty-three opened. But the number that reorganized the week was smaller: 15 — the count of tests that reported PASS against a virtual machine they had never contacted.

A Suite That Passed Against Nothing

We ran the syscall coverage suite against a healthy VM with a deliberately wrong SSH host key, expecting a clean sweep of failures. We got 15 PASS / 28 FAIL. Fifteen of the 43 cases declare an empty expected-output string, and the SSH client's own connection error is, technically, output — so it satisfied them. We had been describing this whole family of bugs as "the harness manufactures fake failures," which is the half that gets investigated. It manufactures fake passes too, and those get counted as coverage. The fix landed and merged; the remaining hollow assertions are filed rather than papered over, because the honest oracle for a cleanup command is an explicit skip with a reason, not a marker invented so the line has something to match.

The Toolchain Nobody Had a Second Copy Of

The cross-compiler that builds the C++ kernel existed in exactly one directory, untracked, with no provisioning script. When it was destroyed, the whole C++ lane went down including CI — and the symptom was uninformative, because the build system silently falls back to the host compiler and dies with an error identical to a completely different misconfiguration. It now provisions reproducibly from pinned upstream versions with checksum verification, and the decisive probe is documented as a direct query of the compiler binary rather than an inference from a build log. The general lesson got written down: inventory what is load-bearing and untracked, prefer reproducible over merely backed-up, and make absence fail loudly.

Batching the Reviews Instead of Thinning Them

Roughly four of every five hours of agent time was measured going to lab orchestration — building, booting, SSHing — rather than engineering, and review dispatches were multiplying on top of that. The instinct was to review less. The better call, which came from the top, was to keep dual review and batch it: queue the PRs, then run two reviewers across the whole batch producing per-PR verdicts, with one fix agent per kernel. That turned roughly twenty dispatches per batch of four into about four, without dropping a single reviewer. It paid off immediately — on the very next batch two reviewers with no shared transcript converged independently on the same two blockers, which is a far stronger signal than either would have been alone.

Three Gates, and Fixing One Moved the Failure

The JefeRust CI job had not gone green since mid-July. The cause was that MSYS2 — the shell the job runs in — has no git package at all, and a recent change made the ISO builder require git to stamp an exact build identity. Fixing the obvious gate did not fix the build; it relocated the failure into the middle of the compile, where a second, independent git lookup lives. A third gate turned out to be the repository's own untracked scratch directories tripping a cleanliness check. All three are closed and the pipeline is green end-to-end with a serial boot probe — and we verified that by reading the stage list, not by trusting the word SUCCESS, which is precisely the mistake this whole entry is about.

Findings That Crossed Kernels

The C++ and Rust kernels are converging, and the useful convergence turned out to be the hardening and the tests rather than the designs. An authorization class audited in one kernel prompted an audit of its twin in the other, which found a worse instance; a clamp already landed on the Rust side is now the reference implementation for the equivalent C++ work rather than something to re-derive. Traffic went both directions in the same week. The transferable part was rarely the fix — it was the enumeration method, since the recurring failure mode is not "we forgot a site" but "our search shape could not have found the site." Details stay internal; remediation is in flight.

Housekeeping With Teeth

The repository had accumulated roughly 1,870 local branches and 542 agent worktrees — enough that ordinary recursive searches timed out. It is now at 270 and 159. The thing worth knowing for anyone automating this: squash-merge defeats both git branch --merged and git cherry, because the merged tip is never an ancestor of master, so neither will ever tell you a branch is safe to delete. The only reliable record is the merged-PR history on the forge itself.

What's Next

  • Land the in-flight kernel authorization work now on its third review round
  • Bring the Rust-side smoke suites up to the same transport-honesty standard as the C++ ones
  • Capture machine-readable baselines so agents stop measuring their own from scratch every run
  • Wire the harness self-test into a pipeline — a proof that never executes is not a proof