systems · netstack · terminal software
3 min readopenhsm

Auto-unsealing OpenBao with a USB HSM on an STM32U585

A tour of OpenHSM, from USB transport and secure session to key store and secure boot, and an honest list of what still stands between it and production.

The target was specific: OpenBao should auto-unseal through a standard PKCS#11 provider backed by a physical device, and the unseal keys should never exist in plaintext anywhere else. That works now, on real hardware:

./p11provision ./libopenhsm_pkcs11.dylib 123456 openbao-seal openbao-hmac
bao-hsm server -config=config.hcl
bao-hsm status   # Sealed: false

Here is what is underneath.

Hardware

An STM32U585, a Cortex-M33 with hardware AES, PKA (public-key accelerator) and a true RNG. There’s no crystal: USB runs from the internal 48 MHz oscillator, trimmed by clock recovery from USB start-of-frame packets. There’s no debugger either, so everything is flashed over DFU. The firmware fits in about 53 KB of flash and 19 KB of RAM.

Transport and session

The device is a USB vendor-class device with one bulk endpoint. Each message has a 16-byte header (command, flags, session id, counter, length, status) and up to 512 bytes of payload, reassembled from 64-byte USB packets.

A session starts with both sides sending an ephemeral X25519 key and a nonce. HKDF-SHA256 turns the shared secret into two keys, one for each direction. Every message after that is AES-256-GCM, with the header as associated data and a nonce built from direction, session id and a strictly increasing counter. A replayed or reordered message fails.

Key store

Objects are sealed with AES-256-GCM under a key-encryption key. Their immutable metadata is bound as associated data, so it can’t be swapped between objects. The mutable usage counter is kept outside the AAD on purpose.

There are 32 slots of 160 bytes in a single 8 KB flash page. Supported types are AES-256, HMAC-SHA256, Ed25519, X25519 and ECDSA P-256. There is no command that exports a plaintext key: a key can leave only wrapped, and only if it was created as exportable.

The hash peripheral story

The STM32U5 has a hardware HASH block, and the first version used it. It was removed. It mishandled message lengths that weren’t a multiple of four bytes, and it wedged on failure, so the derived KEK was not reproducible from one boot to the next. For a key store, that’s fatal.

SHA-256, HMAC and HKDF now run in software. AES-GCM still uses the AES peripheral, and P-256 uses the PKA. Ed25519 and X25519 come from Monocypher. Known-answer tests use vectors generated with libsodium, so the hardware paths are checked against an independent implementation.

Secure boot and updates

A 64 KB stage-1 bootloader verifies an Ed25519 signature over the version, length and image against a vendor key compiled into it. A high-water mark in flash rejects rollbacks.

Firmware updates arrive over USB into a staging bank. The bootloader verifies the staged image and copies it into place from a routine running in RAM. If power fails during the copy, the next boot retries from staging.

Host side

  • libopenhsm_pkcs11: all 68 PKCS#11 entry points are wired. OpenBao, OpenSC and p11-kit all work through it.
  • openhsm-ssh-agent: stock ssh -I drives P-256 keys but not Ed25519, so there’s a small agent for Ed25519.
  • openhsm-daemon: a USB-to-TCP bridge for remote or Kubernetes use.
  • openhsm-cli: provisioning, PINs, keys and audit verification.

What’s missing

The repository is plain about the gaps:

  • No device authentication. The handshake is unauthenticated ECDH, so there is no protection against a man in the middle until devices carry a certificate.
  • The KEK isn’t confidential yet. It is derived from the chip’s UID, which can be read. Fixing that needs readout protection level 2, TrustZone and a secret provisioned per device.
  • The bootloader can be replaced. Readout protection and write protection aren’t enabled, so anyone who can DFU-flash the chip can replace the bootloader.
  • No hardening reviews yet. There’s no watchdog, no fuzzing of the command parser and no constant-time review so far.

TrustZone is deferred for a practical reason. On a board without a debugger, a wrong TrustZone configuration costs a mass erase every time. The roadmap orders the remaining steps from safe to irreversible, and readout protection level 2 comes last because it can never be undone.

← all writing