A "let's clear the followup list" session turned into a drift hunt. Titan repos silently behind upstream. Jenkins pipelines stringifying JSON null as the literal string "null". PowerShell steps treating native command stderr as build failures. Four containers running on UID-mismatch workarounds. We built an anti-drift cron to catch this stuff next time. It found seven more issues on its first run.

The drift we didn't know we had

JefeLife's working tree on Titan had been dirty for weeks. Nine modified files, including a schema v2 migration and a 238-line parser rewrite. The first instinct was "someone edited on Titan in violation of the no-edit rule" — but a byte-level sha256 comparison said otherwise. Seven of eight files were byte-identical to what was already committed upstream. The eighth was a strict subset. Nobody had violated anything; Titan was just six commits behind because no one had run git pull since the prior session pushed. The fix was one git fetch, one git checkout --, one git pull. The schema v2 migration was already live in production — idempotent DDL, safe to pull onto.

The JSONNull bug that ate grocy

Grocy was supposed to be a routine standalone-to-compose migration. We wired the repo, pushed to JefeGit, scp'd the compose file to Titan, and triggered titan-deploy SERVICE=grocy. The build died at the dirty-tree stage trying to cd null. Root cause: Jenkins' readJSON returns net.sf.json.JSONNull — a truthy Object, not Groovy null, not the literal string "null". Groovy's ?: operator left it as-is, and .toString() produced the literal "null", which the downstream stage happily passed to bash. The fix was a single nullSafe closure applied to seven env var populators. One commit, one new pattern, one class of footgun eliminated.

PowerShell's stderr-as-error problem

Build #47 almost tripped a second time because docker compose pull writes layer-download progress to stderr. Jenkins' powershell step interprets any native-command stderr output as a non-terminating error that gets elevated to a step failure. Nine blocks in titan-deploy.groovy needed $ErrorActionPreference='Continue' plus an explicit $LASTEXITCODE check — a pattern already battle-tested in nightly-code-review.groovy. The Health Check stage got left alone on purpose, because its retry loop counts on the powershell step returning even when curl exits non-zero during cold start. Flagged with a code comment so no future "symmetry fix" breaks it.

Container permissions phase 2

Five containers were running on what a DBA agent called an "accidental collision" — their internal UID happened to equal host jefe at 1000, making bind-mount writes work by pure coincidence. We migrated four of them (jefeai-api, jefehealth, grocy, jefelife) to explicit group_add: ["1500"] membership in the shared jefedata group, verified with real SQLite write-lock tests. Grocy got the full lsio treatment — UMASK: "002" env var, verified via /proc/<pid>/status rather than the misleading docker exec umask which bypasses the s6-overlay init. JefeHealth explicitly declined umask widening — SQLCipher-encrypted data at 0o600 is a security feature, not a bug.

The anti-drift cron's first run

Nightly at 03:30 CT, a bash scanner fetches every git repo under /opt/jefe/repos, /opt/jefe/services, and /data/jefe, then posts a Discord alert if anything's dirty, behind, or — the red flag — ahead of upstream (which would mean someone committed directly on Titan). The first test scan found seven issues across eighteen repos: three compose files quietly modified on Titan (ourspace, planning, VibeBot4000), three repos simply behind, and one (jefenotes) where the Forgejo runner was chowning FETCH_HEAD to a user the scanner couldn't read. The whole reason this cron exists is incidents like today's JefeLife situation. It would have caught it within 24 hours instead of weeks.

What's Next

• JefeLife container rebuild — pulled code is on disk, running image is stale
• Triage the three compose-file drifts the anti-drift cron found
• Port the PowerShell stderr fix to five other pipelines flagged during the first fix
• jefeai-api umask entrypoint wrapper — feasibility confirmed, deferred for the 20-minute CUDA rebuild
• Populate /opt/jefe/config/alerts.env on Titan so the anti-drift cron can actually speak to Discord