JefeOS's SSH server now completes full encrypted sessions with PuTTY 0.83. Authentication, command execution, output — the whole flow works end-to-end over chacha20-poly1305@openssh.com. This was a multi-session debug marathon that came down to one line in a header file.
The Symptoms
After key exchange completed successfully (version exchange, KEXINIT negotiation, curve25519 ECDH, NEWKEYS), the very first encrypted packet from PuTTY would fail to decrypt. The decrypted packet length was garbage (0xAE0A5530 at seq=0), which meant the encryption keys were wrong. Every session, same failure, same spot.
The Red Herrings
Multiple sessions chased several plausible theories before landing on the real cause:
- Exchange hash mismatch — extensive debug output was added to dump every component of
H = SHA256(V_C || V_S || I_C || I_S || K_S || Q_C || Q_S || K). All sizes checked out, all encodings matched the RFC. The hash computation was correct. - Buffer overflow — PuTTY 0.83 sends ~1698-byte KEXINITs. The
hash_databuffer was 2048 bytes, which overflowed with all components combined (~2266 bytes). Increased to 4096. Real bug, but not THE bug. - ChaCha20 nonce byte order — a security audit incorrectly identified OpenSSH's
POKE_U64as little-endian. It's actually big-endian. The original nonce encoding was correct all along.
The Root Cause
JefeOS advertised three ciphers: chacha20-poly1305@openssh.com,aes256-ctr,aes128-ctr but only implemented chacha20-poly1305. SSH cipher negotiation picks the first algorithm on the client's list that also appears on the server's list. PuTTY's preference order puts aes256-ctr before chacha20-poly1305. So PuTTY negotiated AES-256-CTR, while JefeOS blindly used chacha20-poly1305.
PuTTY's SSH log made it obvious: Initialised AES-256 SDCTR outbound encryption. The fix was a one-line change — only advertise the cipher we actually implement.
The Debugging Toolkit
Getting visibility into a bare-metal kernel's SSH handshake required creative approaches:
| Method | Use |
|---|---|
| Hyper-V serial pipe | Named pipe \\.\pipe\JefeOS_Serial for kernel serial output. Async PowerShell reader captures [SSH] tagged lines during connection attempts. |
PuTTY -sshlog | Full packet-level SSH log from the client side. This is what revealed the cipher mismatch — showed PuTTY using AES while we assumed chacha20. |
| Exchange hash dumping | Serial output of every hash component (V_C, V_S, I_C, I_S, K_S, Q_C, Q_S, K) with lengths and hex for external verification. |
What Ships
- Full SSH sessions: PuTTY connects, authenticates with password, executes commands, returns output
- chacha20-poly1305@openssh.com encryption with Ed25519 host keys and curve25519-sha256 key exchange
- Strict KEX support (
kex-strict-s-v00@openssh.com) with sequence number reset - State guards against TCP retransmission causing duplicate KEXINIT/ECDH_INIT processing
- Robust serial debug output for future SSH debugging
What's Next
- Continue POSIX parity sprints (file descriptors, signals, environment variables)
- Begin work on target appliances: Ecosystem Health Monitor, SSH Bastion, Secrets Vault