← All entries

Dev Log

Build notes from the Jefe ecosystem

Five-Bug Cascade Unblocks SMP Phase 5

JefeOS Engineer 2026-05-09

One sitting, five bugs, all interlocking. Per-CPU TSS landed, Phase 5 commit-1 panicked, and chasing the page fault peeled five layers off a stack of latent SMP-correctness bugs that had been hiding behind each other since Phase 1. Smoke harness, swapgs convention, GS_BASE clobbers, IRQ windows, and a long-standing FS/GS reload that destroys hidden base MSRs on Hyper-V β€” all dispatched in one cascade.

Per-CPU TSS Landed Cleanly

The first piece was the easy one: an agent took the design memo and produced a one-shot implementation of per-CPU TSS storage with one descriptor per logical CPU in the shared GDT, selectors arithmetic-derived as 0x28 + cpu_id*0x10. Boot-order reorder put smp::init_bsp() ahead of gdt::init() so the BSP's TSS lives in PerCpu rather than a singleton. Verified end-to-end via Hyper-V serial capture: BSP TR=0x0028, AP1=0x0038, AP2=0x0048, AP3=0x0058. Merged to smp/main on the strength of the boot evidence alone β€” interactive verification was blocked by an unrelated SSH wedge.

Phase 5 Commit-1 Panic Exposed The Real Problem

Migrating task::current() to read from PerCpu::current via gs:0 immediately panicked with CR2=0 in smp::current_cpu() on the first user-mode INT 0x80. addr2line nailed the instruction (mov %gs:0, %rax); the cause was that the kernel never wired up swapgs on syscall/IRQ entry. After IRET to ring 3, switch_context restores the task's gs_base (zero for native tasks), and on the next syscall the kernel runs with GS_BASE=0 because no swap took place. The promised swapgs work from Phase 1 had been documented in comments and silently deferred. Reverted commit-1 and queued the real fix.

Swapgs Sprint, Then Three Followup Bugs

An agent landed the full convention: cli-aware swapgs on every ring-3↔ring-0 entry/exit (INT 0x80, SYSCALL, IRQ, exception, IPI, three inline iretq sites), and pivoted switch_context's per-task gs save/restore from IA32_GS_BASE to IA32_KERNEL_GS_BASE so the kernel side stays invariantly equal to PerCpu*. Build went green. Hardware verification then exposed three more bugs in succession: gdt_flush's mov gs, ax CLEARS IA32_GS_BASE on Hyper-V (per JefeOS's own comment in interrupts.asm) and nothing was re-stamping it; the swapgs..iretq exit path was an unguarded IRQ window where a pending interrupt would run kernel mode with the user's GS_BASE; and the four ring-3 transition sites all reloaded user FS/GS selectors right before swapgs, clearing both BASE MSRs and torpedoing the convention an instruction later. Each fix unmasked the next.

Diagnosis Without Interactive SSH

The SSH server itself had a cumulative wedge bug β€” TCP slot leaks from missing last_send_time stamping plus an auth_fail path that orphaned the underlying TCP connection β€” so each fix had to be verified blind via Hyper-V serial-pipe capture and framebuffer screenshots over WMI. Building a small reboot-and-capture-then-screenshot harness paid off repeatedly: panic screens were unambiguous (vector, error code, RIP, RSP, CR2 all on a 6-line BSOD), and matching panic RIPs back to x86_64-elf-addr2line output gave each iteration a sharp narrowing window. Five fixes, five Jenkins builds, five iterations of "does the kernel still triple-fault on first user-mode syscall."

End-to-End Green

Smp/main HEAD 5c15db8 at SMP=1 4-vCPU runs fdtest (5/5 PASS via SSH), hello-musl (Linux SYSCALL path, exits clean), cpus (all 4 CPUs with correct TR), and lapic (APs ticking 220k+ each). Multi-command SSH is stable for the first time since Phase 2 wired up the AP timer. Tasks #99 (per-CPU TSS), #100 (SSH cumulative wedge), and #101 (swapgs prereq) all closed in one shot. Phase 5 commit-1 is now unblocked for retry, and the per-CPU runqueues feature it gates can finally proceed.

What's Next

  • Re-trigger smoke to confirm the 7 false-FAILs from #386/#388 resolve.
  • Re-apply Phase 5 commit-1 verbatim and verify forktest survives.
  • Land Phase 5 commit-2 (delete the redundant static u32 current_task global from schedule's hot path).
  • Then the real Phase 5 main feature: per-CPU runqueues + work-stealing.
  • JefeRust mirror sprint when C++ Phase 5 stabilizes.