Sealed OpenBao (2/2): the post-reboot checklist

After each reboot of the server hosting OpenBao, here’s the procedure I follow, in order. It takes five minutes.

1. Point the client at the right server

My OpenBao listens over HTTP on 127.0.0.1, with nginx doing TLS in front. The bao client tries HTTPS by default, hence this misleading error:

bao status
# Error checking seal status: … http: server gave HTTP response to HTTPS client
export BAO_ADDR=http://127.0.0.1:8200      # locally, on the server
# or: export BAO_ADDR=https://secrets.example.org   (through nginx)

I put that export in root’s .bashrc on that server, so I don’t have to think about it.

2. Check the state

bao status
# Sealed          true
# Total Shares    5
# Threshold       5
# Unseal Progress 0/5

Sealed true confirms it needs unsealing. Unseal Progress shows how many shares have been entered: entry can be spread over several goes, even by several people.

3. Unseal

bao operator unseal      # repeated 5 times; each share is prompted for without echo

With no argument, the command prompts for the share interactively and doesn’t display it. Never pass a share as a command-line argument: it would end up in shell history and the process list. And never paste it into a tool that keeps history, whether a chat app or an assistant.

4. Check the whole chain

An unsealed vault isn’t enough: deployments must actually be able to read their secrets. So I test from the Ansible control node, with the same AppRole identity as the playbooks:

# AppRole login, then read a secret's metadata (not its value)
bao write -field=token auth/approle/login role_id=… secret_id=… > /tmp/t
BAO_TOKEN=$(cat /tmp/t) bao kv metadata get -mount=ansible postgresql | head -3
shred -u /tmp/t

Reading metadata (version, dates) proves authentication and permissions work, without the secret’s value travelling anywhere.

5. Resume deployments

An ansible-playbook … --check on any playbook that reads a secret confirms everything is back to normal.

Don’t forget it

The biggest risk is forgetting: an overnight reboot, and nobody notices deployments are blocked until they’re needed. A Prometheus alert on the seal state would warn you as soon as it happens: the /v1/sys/health endpoint returns 503 when the vault is sealed, and a blackbox probe is enough to watch it. It’s next on my list. After every reboot of the home server, unsealing is now part of my checklist, alongside checking the services.

---

This post is part of a series of sysadmin notes (tag “sysadmin notebook”).
Series “Sealed OpenBao”, part 2/2. · ← Previous post · Version française