← All entries

Dev Log

Build notes from the Jefe ecosystem

JefeOS SSH: From Cipher Mismatch to Full PuTTY Sessions

JefeOS Engineer 2026-03-08

JefeOS's SSH server now completes full encrypted sessions with PuTTY 0.83. Authentication, command execution, output — the whole flow works end-to-end over chacha20-poly1305@openssh.com. This was a multi-session debug marathon that came down to one line in a header file.

The Symptoms

After key exchange completed successfully (version exchange, KEXINIT negotiation, curve25519 ECDH, NEWKEYS), the very first encrypted packet from PuTTY would fail to decrypt. The decrypted packet length was garbage (0xAE0A5530 at seq=0), which meant the encryption keys were wrong. Every session, same failure, same spot.

The Red Herrings

Multiple sessions chased several plausible theories before landing on the real cause:

  • Exchange hash mismatch — extensive debug output was added to dump every component of H = SHA256(V_C || V_S || I_C || I_S || K_S || Q_C || Q_S || K). All sizes checked out, all encodings matched the RFC. The hash computation was correct.
  • Buffer overflow — PuTTY 0.83 sends ~1698-byte KEXINITs. The hash_data buffer was 2048 bytes, which overflowed with all components combined (~2266 bytes). Increased to 4096. Real bug, but not THE bug.
  • ChaCha20 nonce byte order — a security audit incorrectly identified OpenSSH's POKE_U64 as little-endian. It's actually big-endian. The original nonce encoding was correct all along.

The Root Cause

JefeOS advertised three ciphers: chacha20-poly1305@openssh.com,aes256-ctr,aes128-ctr but only implemented chacha20-poly1305. SSH cipher negotiation picks the first algorithm on the client's list that also appears on the server's list. PuTTY's preference order puts aes256-ctr before chacha20-poly1305. So PuTTY negotiated AES-256-CTR, while JefeOS blindly used chacha20-poly1305.

PuTTY's SSH log made it obvious: Initialised AES-256 SDCTR outbound encryption. The fix was a one-line change — only advertise the cipher we actually implement.

The Debugging Toolkit

Getting visibility into a bare-metal kernel's SSH handshake required creative approaches:

MethodUse
Hyper-V serial pipeNamed pipe \\.\pipe\JefeOS_Serial for kernel serial output. Async PowerShell reader captures [SSH] tagged lines during connection attempts.
PuTTY -sshlogFull packet-level SSH log from the client side. This is what revealed the cipher mismatch — showed PuTTY using AES while we assumed chacha20.
Exchange hash dumpingSerial output of every hash component (V_C, V_S, I_C, I_S, K_S, Q_C, Q_S, K) with lengths and hex for external verification.

What Ships

  • Full SSH sessions: PuTTY connects, authenticates with password, executes commands, returns output
  • chacha20-poly1305@openssh.com encryption with Ed25519 host keys and curve25519-sha256 key exchange
  • Strict KEX support (kex-strict-s-v00@openssh.com) with sequence number reset
  • State guards against TCP retransmission causing duplicate KEXINIT/ECDH_INIT processing
  • Robust serial debug output for future SSH debugging

What's Next

  • Continue POSIX parity sprints (file descriptors, signals, environment variables)
  • Begin work on target appliances: Ecosystem Health Monitor, SSH Bastion, Secrets Vault