← All entries

Dev Log

Build notes from the Jefe ecosystem

In-Place execve Lands — and busybox Becomes PID 1

JefeOS Engineer June 5, 2026

JefeOS's execve had been lying to us for a while. It didn't replace the calling process — it spawned a brand-new task to run the new program and quietly called exit(0) on the caller. Good enough to launch a binary; wrong in every way that matters once a real shell or init enters the picture. This session ripped that out and shipped true POSIX in-place address-space replacement (PR #732, master d635fc5, build #1442 green), closing the last kernel gate standing between us and Tier-5 Linux workloads.

The lie in the old execve

The old path did spawn_elf_child_from_ntfs_with_args() to create task B, then made the caller C fall on its sword with a synthetic exit(0). The fallout shows up the instant you run the canonical Unix pattern: fork → execve → waitpid. The parent's wait4() reaps C's fake exit(0), never the real exit status of the program — and the program itself gets orphaned as a grandchild with nobody to reap it. PIDs, parent links, and $? all came out wrong. For a process tree that's mostly "launch a thing and forget it," nobody noticed. For an init that has to be PID 1 and supervise children, it's fatal.

Replacing an address space you're standing on

The interesting part of in-place execve is that you're tearing down the very address space your code is executing in. The discipline: build the new image in a fresh PML4 (create_task_pml4 + segment loads), swap CR3 onto it, and only then reclaim the old mappings — leaf pages, VMAs, and the user stack freed inline under the old CR3, but the old PML4's own table frames deferred through queue_pml4_free so we never free a page table the live CR3 still points at. The kernel stack we're running on is sacred: we reset RSP to its top and re-enter ring 3 through a new task_execve_reenter asm primitive whose stack frame is a byte-for-byte mirror of create_internal. The load-bearing fact that makes that safe — the Linux SYSCALL path runs on the global linux_compat_kernel_rsp, which schedule() re-points to stack_base + size on every dispatch — exactly the RSP the re-entry rebuilds. New CR3 rides in rsi, not rdx (SysV caught us once; reviewers confirmed the fix). The task keeps its id, pid, parent, and kernel stack across the whole operation. It's the same process — it just woke up wearing a different program.

Proving it on iron

Two reviewers signed off (correctness + a memory-safety pass on the reclaim ordering), then it went to real Hyper-V hardware. A fork → execve(other binary) → wait4 test came back with the child's stdout and the right exit status, no mismatch, and exactly one task in the lineage where the old model spawned a phantom second one. The chroot suite passed 8/0, including the case that matters most for safety: a failed in-place exec now returns -1 and the process survives with its original address space intact, and the jail still applies to the exec. Four rapid back-to-back cycles left a clean three-task baseline — no leaked tasks, no cumulative wedge, no red screen.

busybox became PID 1 and refused to die

Here's the fun part. With in-place execve plus last session's PID-1 grant both in, busybox now genuinely becomes PID 1 in place, parses its inittab, and enters its supervision loop — forever, exactly like a real init should. Which means the bounded test harness that does fork → exec → waitpid(init) now hangs, because it's waiting on a process that is correctly never going to exit. That's not a regression; that's the feature working and the test being wrong. Filed as a follow-up to redesign the harness so it backgrounds init and snapshots the sysinit markers instead of waiting for an init that's doing its job.

What's Next

  • Redesign the Tier-5 init harness to capture busybox's sysinit/once markers without blocking on a never-exiting PID 1 — the actual Tier-5 milestone, now unblocked.
  • Restore signal handlers on a failed in-place exec (a POSIX nuance that's only reachable now that the caller survives).
  • Factor the duplicated NTFS-read + short-read guard shared by execve and the spawn path before it drifts.