← All entries

Dev Log

Build notes from the Jefe ecosystem

Private CA, 24 Security Fixes, Two Websites, and a Secrets Vault

Mission Control 2026-03-27
Security Cryptography Infrastructure DevOps JefeAI Jefebots Feature

A marathon session that started with security reviews and ended with a private certificate authority issuing TLS certs across the infrastructure. In between: 24 security fixes deployed, two JefeWorks websites launched, a secrets vault proven end-to-end, NVIDIA's Nemotron models pulled, and a reverse proxy stack brought online. The kind of session where you look up and it's 2 AM.

72 Findings, 24 Fixes

Five parallel review agents swept FreeChat and Enclave: security audits, code quality reviews, and a WebSocket disconnect root-cause analysis. The reviews surfaced 72 findings across both projects. The disconnect investigation found the smoking gun — a token refresh response format mismatch that made reconnection fail 100% of the time after JWT expiry. One-line fix, but it had been silently breaking sessions for weeks. Beyond that: SSRF protection on outgoing webhooks, HMAC signature verification on incoming ones, rate limiters made atomic and fail-closed, cursor pagination crash on deleted messages fixed, admin operations wrapped in database transactions, and OAuth session creation unified so all users get the same 90-day refresh tokens regardless of auth method.

JefeVault: Wrapped Tokens Work

The secrets vault proved its end-to-end flow: generate a wrapped token from the Chrome extension, unwrap it (single-use, scoped, 10-minute TTL), read a credential from the vault, authenticate against a live service with it, and confirm the second unwrap attempt is correctly rejected as consumed. The full chain — vault stores the real credential, wrapped token provides secure scoped access, token burns after one use. This means we don't need HashiCorp Vault for secrets management. JefeVault handles that side; step-ca handles certificates.

JefeCA: Private Certificate Authority

Deployed step-ca as a private ACME CA on the GPU server. The init process fought back — the remote management flag prompted for interactive input in a headless container, volume mounts shadowed password files, and security restrictions prevented the binary from executing. Six debugging rounds later: CA generating root and intermediate certificates, ACME provisioner issuing certs, health checks passing. A TLS cipher mismatch in the health-check sidecar was the final boss — resolved by removing the TLS 1.2 cipher restriction from the CA config. The CA now issues certificates for internal service domains.

Traefik Reverse Proxy

Traefik v3.3.3 went up next, but immediately hit a Docker API version mismatch — it hardcodes API v1.24 internally while the Docker daemon requires v1.40+. The DOCKER_API_VERSION env var is ignored. The fix: an nginx reverse proxy sitting between Traefik and the Docker socket that rewrites API version paths. Unconventional, but it works. Security and network engineers reviewed the full config before deployment, catching four blocking issues: deploy phase ordering, dashboard exposure, container-to-host routing (host.docker.internal needs explicit mapping on Linux), and a port registry discrepancy. All fixed before going live.

NVIDIA Nemotron Models

Deep-dived the NVIDIA AI platform: NIM, TensorRT-LLM, Inception program, and the Nemotron model family. The standout: Nemotron-Cascade-2, a 30B MoE with only 3B active parameters that outperforms the 120B Super model on code and math benchmarks while fitting in 24GB VRAM. Both models pulled to the GPU server — Cascade-2 at 24GB and Super at 86GB. A vLLM migration plan was drafted for production inference (86x throughput improvement over the current setup at 32 concurrent users). NVIDIA Inception pitch deck ready for submission.

Two Websites Live

Rebuilt jefeworks.com as a lean AI studio page with a live chatbot, product ecosystem cards, infrastructure showcase, and a curated development log. Built jefe.works from scratch as the interactive playground — eight AI personality chatbots (Lebowski, Techbro, Ozzy, and friends), a Twitch stream embed, and product teasers. Both deployed on Cloudflare Pages. Domain transfers from Squarespace initiated for DNS consolidation. The chatbot needed its own RAG endpoint to maintain company voice without leaking personal identity from the portfolio collection.

FreeChat Stabilization

The security hardening correctly rejected a placeholder JWT secret in production, taking FreeChat offline. Rotated the secret, applied pending database migrations (message edits, DM hidden flag), and fixed a CSP policy that blocked external images and YouTube embeds. Avatars, link previews, and unfurling all restored. The v1.4.0 feature set — message edit/delete, DM close/reopen, encrypted file metadata, AI bot system, Android WebSocket resilience — was fully committed across 8 additional commits and pushed.

What's Next

  • Traefik TLS verification and Docker labels for all services
  • JefeVault accessible via internal DNS
  • vLLM Phase 1 — Qwen3-32B proof of concept on the GPU server
  • NVIDIA Developer Program + Inception signup
  • Credential rotation for remaining services
  • Headless OBS Docker for AIMemeLord always-on streaming
  • FreeChat Docker rebuild to bake in CSP fix permanently