Virtual Signer disaster recovery
Generate the recovery key pair, configure it, and recover vaults from Virtual Signer .dr files
A Mobile Signer is recovered with a device backup file and its 24-word phrase, held by the person who owns the device. A Virtual Signer has neither: it is an unattended server-side signer with no human owner and no recovery phrase to write down.
Instead, a Virtual Signer continuously writes its own disaster recovery files (.dr files). Every time it takes part in a key generation or a reshare, it encrypts its share material to a public key that you supply and writes it to disk. The matching private key never touches the Virtual Signer. It stays offline, with your organisation, and it is the only thing that can decrypt those files later.
This means the single most important step in the whole process happens before the Virtual Signer is ever started.
The recovery key pair must exist before first startThe Virtual Signer will not start without a disaster recovery public key, and the key it starts with is sealed into its configuration on first registration. There is no supported procedure to swap it afterwards.
If the private key is lost, every
.drfile that Virtual Signer has ever written — and every one it will write in future — becomes permanently undecryptable. Generate the key pair first, verify it, and back it up offline.
Who does what
| Role | Responsibility |
|---|---|
| Product Owner (Business User) | Generates the ML-KEM-768 key pair. Stores the private key offline and safely. Verifies that the deployed Virtual Signer is configured with the matching public key, and that it starts successfully. |
| IT/DevOps Engineer | Places the public key PEM on the Broker host and points IOSIGN_SIGNER_BROKER_DR_PK_PEM_FILE at it. Configures the backup volume. Runs periodic bundle exports. Never handles the private key. |
| Recovery operator | During an actual recovery event only: runs the DR tool on an air-gapped machine with the private key and the collected bundles. Gains full access to the recovered vault keys, so must be trusted accordingly. |
The split matters. The IT/DevOps Engineer needs only the public key to run the service. Anyone holding the private key can decrypt every Virtual Signer share it protects, so it should never be stored on, or reachable from, the machine running the Virtual Signer.
Step 1 — Generate the recovery key pair
Owner: Product Owner (Business User). Do this before the Virtual Signer is deployed.
The key generation script ships with the io.finnet Disaster Recovery tool. Run it on a secure, preferably offline workstation — not on the Virtual Signer host.
# macOS / Linux
./scripts/posix/gen_mlkem768.sh# Windows
.\scripts\windows\gen_mlkem768.ps1Both scripts accept an optional base name, which defaults to mlkem768:
./scripts/posix/gen_mlkem768.sh acme-treasuryThe script produces two files:
| File | Contents | Who holds it |
|---|---|---|
mlkem768_priv.pem | ML-KEM-768 private key (PKCS#8). Written with file mode 600. | Product Owner, offline. Never goes on the Virtual Signer. |
mlkem768_pub.pem | ML-KEM-768 public key (X.509 SubjectPublicKeyInfo). | Handed to the IT/DevOps Engineer for deployment. |
OpenSSL 3.5.0 or later is requiredML-KEM support was added in OpenSSL 3.5. Check your version with
openssl version. The script stops with a clear error if the installed build cannot generate ML-KEM-768 keys. Note that theopensslbundled with Git for Windows is frequently older than 3.5.
The script refuses to overwrite an existing key pair, so it cannot silently destroy a key you are already using.
Storing the private key
Treat mlkem768_priv.pem with the same care as the 24-word phrase of a Mobile Signer, because it plays the same role: it is the one secret that makes recovery possible.
- Store it offline, on encrypted media, in a physically secure location.
- Keep more than one copy, in separate locations, so that a single fire or failed drive cannot destroy it.
- Do not store it on the Virtual Signer host, in the deployment repository, in a CI secret store, or anywhere the running service could reach it.
- Record who holds it and how it is to be retrieved during an incident. A key nobody can find during an outage is no better than a lost one.
Step 2 — Configure the public key on the Virtual Signer
Owner: IT/DevOps Engineer.
Copy only mlkem768_pub.pem to the Broker host, and point the Broker at it:
| Configuration | Description |
|---|---|
IOSIGN_SIGNER_BROKER_DR_PK_PEM_FILE | Path to the ML-KEM-768 public key PEM file used to encrypt disaster recovery files. Required — the Broker will not start without it. |
This setting is read by the Broker only. The Broker forwards the key to MPC Core with each key generation and reshare request, so MPC Core does not need its own copy.
A Docker Compose deployment mounts the key and sets the variable:
services:
vs-broker:
environment:
IOSIGN_SIGNER_BROKER_DR_PK_PEM_FILE: /etc/vsigner/dr/mlkem768_pub.pem
volumes:
- ./secrets/mlkem768_pub.pem:/etc/vsigner/dr/mlkem768_pub.pem:roStart-up validation
If the variable is missing, the Broker exits immediately with:
CONFIG_REQUIRED_MISSING: env var IOSIGN_SIGNER_BROKER_DR_PK_PEM_FILE is requiredIf the path is unreadable or the file is not a valid PEM block:
CONFIG_INVALID_PEM_FILE: env var IOSIGN_SIGNER_BROKER_DR_PK_PEM_FILE must point to a readable file containing a valid PEM block
A valid PEM is not necessarily the right keyStart-up validation checks only that the file contains a well-formed PEM block. It does not check that the key is the ML-KEM-768 public key you intended.
A Virtual Signer configured with the wrong PEM will start normally and then fail every key generation and reshare with
invalid disaster recovery public key. Worse, a Virtual Signer configured with a valid but wrong ML-KEM-768 key will run perfectly and write.drfiles that your private key cannot open — a failure you would only discover during an actual recovery.This is why Step 3 is not optional.
Step 3 — Verify the key and the first start
Owner: Product Owner (Business User).
Before the Virtual Signer is given production signing weight, confirm two things independently of whoever deployed it.
Confirm the deployed public key is yours
Compare a checksum of the key in your own custody against the one the running container is actually using:
# On your own machine, against your copy
sha256sum mlkem768_pub.pem
# Against the deployed Virtual Signer
docker exec virtual-signer-broker \
sh -c 'sha256sum "$IOSIGN_SIGNER_BROKER_DR_PK_PEM_FILE"'The two digests must match. If they do not, stop and resolve the discrepancy before the signer joins any vault — every .dr file written in the meantime would be unrecoverable with your private key.
Confirm the Virtual Signer starts successfully
Check that the Broker reaches a healthy state with that key in place:
docker compose logs vs-broker | tail -50
curl http://localhost:8182/broker/v1/health?strict=trueA successful first start with the correct key is what seals the configuration, described next.
Immutable configuration
When a Virtual Signer registers for the first time, it takes a SHA-512 hash of a small set of security-critical configuration values, signs that hash inside MPC Core's trusted execution environment, and stores both on disk. The hash covers:
- the approval mode,
- the external approval policy base URL,
- the disaster recovery public key, and
- whether the Broker's connection to MPC Core is secured.
On every subsequent start the Virtual Signer recomputes the hash and compares it with the sealed value. If it differs, the process refuses to start and logs:
IMMUTABLE_CONFIG_MISMATCH - invalid configuration hash or signature - rollback configuration to its original valuesThe purpose is to stop anyone — including someone with access to the deployment pipeline — from quietly substituting a different disaster recovery public key, which would silently redirect all future .dr files to a key your organisation does not hold. Once the Product Owner has verified the key and seen the signer start, that key is pinned for the life of the registration.
The pin cannot be changed in placeBecause the hash is sealed at initial registration and signed inside the TEE, there is no supported procedure for changing the disaster recovery public key on an existing Virtual Signer. Plan the key pair as a long-lived asset and verify it before the signer joins production vaults. If the key genuinely must change, contact your io.finnet representative.
Disaster recovery (.dr) files
Once configured, the Virtual Signer writes .dr files automatically. There is nothing to schedule and nothing for an operator to trigger.
A .dr file is written whenever the signer takes part in:
- a key generation — when it joins a new vault, or
- a reshare — whenever vault membership or weights change.
Each of those operations produces two files, one per signature scheme, under the Broker's storage volume:
{StorageBasePath}/private-dr/{vaultId}/{signerId}/{requestId}.{algo}.{curve}.drFor example:
/var/vsigner/private-dr/019f717b-.../019f2838-.../019f717b-....ecdsa.secp256k1.dr
/var/vsigner/private-dr/019f717b-.../019f2838-.../019f717b-....eddsa.ed25519.drStorageBasePath is set by IOSIGN_SIGNER_BROKER_STORAGE_BASE_PATH and defaults to /var/vsigner. Note that the leading UUID in the filename is the request ID of the operation, not the vault ID.
A vault accumulates many .dr files
Old .dr files are never deleted. Each reshare adds a new pair rather than replacing the previous one, so a vault accumulates 2 × (1 key generation + N reshares) files for each Virtual Signer that participates in it.
This is deliberate. Each file carries a previousRequestId pointer to the operation before it, forming a chain that lets the recovery tooling work out which set of shares belongs to which point in the vault's history, and which is current.
Because a recovery may need several of these files, they are collected into a single ZIP bundle before being handed to the DR tool. That is what signerctl does.
What is inside a .dr file
Each file is a JSON envelope. The identifying fields are in plaintext so that operators and tooling can tell which vault and which epoch a file belongs to without decrypting anything:
| Field | Purpose |
|---|---|
formatVersion, kemSuite | Let a reader reject a file it cannot decrypt with a clear message. Current suite is ML-KEM-768+AES-256-GCM. |
vaultId, requestId, previousRequestId | Identify the vault and position the file in the reshare chain. |
algo, curve | ECDSA/secp256k1 or EDDSA/ed25519. |
dataB64 | The encrypted share material: a fresh ML-KEM-768 encapsulation followed by an AES-256-GCM ciphertext. |
Only dataB64 is secret. The plaintext fields are diagnostic aids and are not authenticated, so the recovery tool takes all security-relevant values from inside the encrypted payload.
.drfiles carry their own protectionA
.drfile is protected by its ML-KEM envelope, independently of any encryption-at-rest configured on the storage volume.IOSIGN_SIGNER_CORE_ENCRYPTION_KEYand Azure SKR protect other Virtual Signer state; they do not encrypt.drfiles, and they are not needed to read them during a recovery.
Creating a recovery bundle with signerctl
signerctl is the Virtual Signer operator CLI. It is built into the Broker image at /usr/local/bin/signerctl and runs as a one-off command inside the Broker container, reusing the Broker's own configuration.
Configure the backup volume
Bundle creation writes to a separate volume, configured with:
| Configuration | Description |
|---|---|
IOSIGN_SIGNER_BROKER_BACKUP_BASE_PATH | Directory that receives backup ZIP bundles. Not read by the Broker at runtime; required only for signerctl share backup. |
If it is unset, the command stops with backup volume not configured: set IOSIGN_SIGNER_BROKER_BACKUP_BASE_PATH.
Export a bundle
docker exec virtual-signer-broker signerctl share backupThis collects every .dr file belonging to this signer, records an integrity manifest, writes a ZIP, then reads that ZIP back and re-verifies every entry before reporting success:
share backup — signer=aabbccdd…0001
2 vaults: 2 ok
bundle: /var/vsigner-backups/backup-aabbccdd-0000-0000-0000-000000000001-20260709T103000Z.zip (1.2 KiB, 2 files, verified)
result: OKThe bundle is named backup-<signerId>-<timestamp>.zip and contains a manifest.json at its root alongside the .dr files:
backup-<signerId>-<timestamp>.zip
├── manifest.json
└── dr/
└── {vaultId}/
└── {signerId}/
└── {requestId}.{algo}.{curve}.drThe manifest lists each file with its SHA-256 digest, byte length, and envelope metadata, and flags any file it found problems with. Nothing is decrypted during bundling — signerctl never has the private key and does not need it.
Exit codes:
| Code | Meaning |
|---|---|
0 | Clean: no findings. |
1 | Advisory findings — integrity problems recorded in the manifest. The bundle is still written. |
2 | The command could not run. No bundle is written. |
Exit code 1 is a signal to investigate, not to discard the bundle.
Related signerctl commands
| Command | Purpose |
|---|---|
signerctl signer show | Show the signer identity held on this volume and confirm it matches the backend. |
signerctl share check | Compare the shares on disk against the vaults the backend says this signer belongs to. Add -v to list every vault. |
signerctl version | Print the version. |
When to export
Export a bundle:
- after the Virtual Signer joins a new vault,
- after every reshare it participates in, and
- on a regular schedule as a safety net.
The same rule that applies to Mobile Signer backup files applies here: a bundle taken before the most recent reshare may not hold enough current shares to recover the vault.
Recovering a vault
Recovery happens on an air-gapped machine, using the io.finnet Disaster Recovery tool and the private key from Step 1.
./recovery-tool -keys /secure/mlkem768_priv.pem backup-019f2838-20260709T103000Z.zip-keys takes one or more ML-KEM-768 private key PEM paths. It is repeatable, and also accepts a comma-separated list, so a recovery spanning several Virtual Signers can supply a key for each:
./recovery-tool -keys signer1_priv.pem,signer2_priv.pem signer1-backup.zip signer2-backup.zipBundles, loose .dr files and Mobile Signer JSON backups can be combined in a single run, which is the normal case for a vault whose threshold is met by a mix of signer types:
./recovery-tool -keys /secure/mlkem768_priv.pem \
backup-019f2838-20260709T103000Z.zip \
mobile-backup.json
Virtual Signer bundles need no 24-word phrase
.drfiles and bundle ZIPs are decrypted with the ML-KEM-768 private key, so the tool will not prompt for a recovery phrase for them. Any Mobile Signer JSON backups included in the same run still require their own 24-word phrases.
A few constraints are worth knowing before an incident:
- Use version 6.5.0 or later. Support for Virtual Signer bundle ZIPs was added in v6.5.0. Check the Releases page.
- Bundle ZIPs are command-line only. The browser UI (
-web) reads loose.drfiles with a single private key, but does not handle bundle ZIPs. Use the CLI for bundles. manifest.jsonmust be at the root of the ZIP. Bundles written bysignerctl share backupalready satisfy this. If you ever re-zip an extracted bundle by hand, zip the folder's contents, not the folder itself.- Legacy exports need their own run. A legacy vault export — the older flat reshare-nonce JSON format, which carries no request IDs — cannot be combined with Virtual Signer bundles or
.drfiles. Recover it separately, with only its mnemonic.
Once the shares are combined, the tool exports the recovered private keys in the same formats used for any other vault recovery. See the Disaster recovery process for the wallet import steps.
Post-quantum security
.dr files are encrypted with ML-KEM-768, the key encapsulation mechanism standardised by NIST as FIPS 203. ML-KEM is a lattice-based scheme designed to resist attack by both classical and quantum computers, and it replaces the elliptic-curve key exchange that a sufficiently capable quantum computer would be able to break.
This matters more for disaster recovery material than for almost anything else the platform stores. .dr files are long-lived by design: they sit in backups, in archives, and on offline media for as long as a vault exists. That makes them a natural target for a harvest now, decrypt later strategy, where an adversary copies ciphertext today purely to decrypt it once quantum hardware matures. Encrypting them under ML-KEM-768 removes that path.
The share material inside each envelope is encrypted with AES-256-GCM under a key derived from a fresh ML-KEM encapsulation, performed separately for every file. AES-256 is itself considered quantum resistant, so both layers of the envelope hold up against the same threat model.
What to back up
| Item | Why | Where it lives |
|---|---|---|
| ML-KEM-768 private key | Without it, no .dr file can ever be decrypted. It is never stored by the Virtual Signer. | Offline, with the Product Owner |
signerctl share backup bundles | The portable, integrity-checked snapshot of the signer's .dr files. | IOSIGN_SIGNER_BROKER_BACKUP_BASE_PATH |
| The storage volume | Holds live shares, the keyring and the sealed configuration hash — needed to keep the existing signer running, as opposed to recovering the vault. | IOSIGN_SIGNER_BROKER_STORAGE_BASE_PATH |
| Encryption-at-rest key | If IOSIGN_SIGNER_CORE_ENCRYPTION_KEY or Azure SKR is configured, volume backups are unreadable without it. Does not affect .dr files. | Your secret store |
A bundle is not a complete signer backup. It contains .dr files only, and there is no path that restores a running Virtual Signer from one — it is an input to offline recovery tooling. Restoring service after a host loss is a separate concern, covered in Disaster recovery and backup.
Run drills
Test the whole chain before you need itThe failure modes described on this page — a wrong-but-valid public key, a private key nobody can locate, a bundle taken before the last reshare — all look identical to a healthy deployment until the day you attempt a recovery.
Perform regular air-gapped disaster recovery drills on a test vault: generate a key pair, deploy, export a bundle, and recover it end to end. Confirm that the private key can actually be retrieved from wherever your organisation stores it, by the people who would need it during an incident.
Keep a redundant set of Mobile Signers able to reach the vault threshold without the Virtual Signers, so that the loss of your server-side signing infrastructure does not by itself put a vault out of reach.
Where to get support
Please refer to our help center to review user guides and follow our support processes, or contact our team at support.iofinnet.com.
Updated about 2 months ago