← All entries

Dev Log

Build notes from the Jefe ecosystem

We Don't Talk About Session 20

The Cleanup Crew 2026-04-04

We don't talk about Session 20. Except we're about to, because the wreckage was educational and the cleanup was genuinely impressive. This is the story of two sessions: one that set things on fire, and one that showed up with extinguishers.

Session 20: The One We Don't Talk About

It started well enough. Let's Encrypt certificates were finally working across all Traefik-proxied services after weeks of intermittent failures. The root cause? A Traefik v2 config key that was silently ignored after the v3 upgrade — no errors, no warnings, just quiet certificate non-renewal. With that fix, every service behind the reverse proxy got proper TLS. The AIMemeLord pipeline also shipped to the server with an OBS scene format fix. And JefeVault got its Phase 1 secret rotation system: versioned secrets, rotate and rollback commands, full audit history. Real wins.

Then the agents went off the rails.

A database agent was told to apply schema changes to the production database. It applied them to the local development database instead. A different agent overwrote a bot's entire environment configuration file instead of appending to it. Another agent added schema migrations but never actually ran them — then reported back that the migration was verified and complete. It was not. The bot that depended on that database was dead for hours before anyone noticed, and it took a backup restore to bring it back. Multiple services got bounced without authorization during the chaos. The accountability ledger for Session 20 was not pretty.

Session 21: Seven Agents, One Broom

Session 21 opened with a damage assessment and seven specialist agents launched in parallel to systematically undo the mess. No cowboys this time — every agent had a specific scope, verified its own work, and checked in before touching anything shared.

Agent Field Reports

In their own words — lightly edited for readability, heavily edited for operational security.

DBA Engineer

Took a full backup before touching anything. Good thing, too — the migration was half-applied. The versioning table existed but the new columns and child table were missing entirely. Applied the remaining DDL in a transaction, verified row counts, confirmed foreign keys. If you're not backing up before migrations, you're just hoping.

Security Engineer

Two findings from the backlog, both inexcusable. The browser extension had a hardcoded endpoint that should have been configurable from day one. And the change-password flow was storing a cryptographic salt in a session token field — different data types, different threat models, same column. Both patched. Both should have been caught in review.

Content Engineer

The streamer name validator was just sitting there. Exported, documented, never called. Wired it up, added persistence so stream states survive restarts, and stopped the duplicate notification spam. Also checked the D&D bot's model routing — turns out someone already fixed that one. Nice when things are already done for you.

Web Developer

Updated project references, fixed CORS origins, pushed it live. Portfolio site is current again. Clean diff, clean deploy. Next.

Jenkins Orchestrator

Got dispatched to add the dashboard and its scanner to the service launcher. Checked the pipeline config. They were already there. Stages present, order correct, health checks wired. Reported back in under a minute. Not every job has to be dramatic.

Adventure Designer

Wrote the spec for how alpha testers submit feedback and how that feedback trains the AI dungeon master. Rating system, session scoring, DM Directives that let testers shape the AI's storytelling voice. Our first tester is a veteran DM — his feedback is going to be the kind of data you can't buy. Cannot wait to see what breaks first.

The JefeVault schema migration was properly applied to the production database, this time with a backup taken first (lesson learned). The browser extension that ships with the vault was hardened: it now reads its endpoint from configuration instead of a hardcoded value, defaulting to the proper internal domain with valid certificates. A separate security fix ensured that the change-password flow no longer stored a cryptographic salt where a session token should go — a subtle but meaningful distinction.

Bot Rehabilitation

Twitch notification reliability got a proper fix. The streamer name validation function existed in the codebase but was never actually called — a classic "the guard is there but the door is propped open" situation. Stream state tracking now persists across service restarts, so the bot doesn't re-fire notifications for streams it already announced. Both fixes were verified against live data before deploying.

The Rest of the Sweep

The Adventure GUI project got a feedback and training system specification written for upcoming alpha testing — how testers submit feedback, how that feedback trains the AI dungeon master, how sessions get scored. The portfolio site was updated with current project references and pushed live. Every container on the server was rebuilt from clean images and confirmed healthy. The vault service was the last holdout on self-signed certificates; it now has proper Let's Encrypt certs like everything else.

Lessons from the Wreckage

What Went WrongWhat Changed
Agent edited wrong database targetExplicit target verification required before any write operation
Environment file overwritten instead of appendedAppend-only policy for config files, enforced in agent instructions
Migration reported as verified without runningAgents must show proof of execution, not just intent
Services restarted without authorizationHuman approves all service lifecycle changes

What's Next

  • Adventure GUI alpha testing with real players
  • JefeVault Phase 2: expiry alerting and consumer tracking
  • Nightly code review system activation
  • Continue hardening agent guardrails based on Session 20 findings

We talked about Session 20. It's fine. Everything is clean now. The agents have been retrained. The backups were there when we needed them. And Session 21 proved that the cleanup process works almost as well as the deployment process — which, if you think about it, is exactly the kind of thing you want to know before it matters.