AI AdminPanel Documentation

Rotate a Service Secret

Use the service's Credentials card or the Secrets page to rotate a vault-backed generated value where Rotate is available. Plan an interruption and inventory applications that use the secret before confirming. See Secrets Manager for storage and host-access limits.

Rotate

  1. Select the secret and read the confirmation warning.
  2. Confirm once and follow background progress.
  3. The worker writes a new vault version, applies the relevant database change for PostgreSQL/MySQL classes, then attempts to recreate the service's containers. Environment-only classes have no separate database change.

A request acknowledgement is not completion. Do not start overlapping manual recovery actions while a rotation job is still running.

Verify

Check the completed job, application login or database connection, and every dependent application. Revealing a vault value proves only what is stored in the vault; it does not prove which value a container or external application uses. Other services holding references may need redeployment to pick up the change.

If rotation fails

  • A rotation is refused, with nothing changed, when the secret's class cannot be rotated, when the service's image is refused, or when the database container the new password must be applied in is missing or not running. Fix the cause and rotate again.
  • A failed initial vault write changes nothing in the vault or the application.
  • An apply failure triggers a rollback attempt that writes the old value as a new version. If that write also fails, vault and application can disagree.
  • A failed recreation can leave consumers using the old value after the vault or database changed. If it failed after the old container was removed, the service shows Error.

If a rotation failed and the service is blocked

A rotation runs as one operation on the service, like a deploy. From the vault write on, any failure (including the three above) leaves the service's operation claim "in doubt". Until that claim is cleared, every operation on the service is refused with "service operation active or awaiting reconciliation". That includes another rotation, Redeploy and the job's own automatic retry. Repeatedly clicking Rotate is not a recovery procedure.

There is no button for this yet (an operator tool is tracked as AI-1061). The hosting administrator first works out which state the service is in, from the job's error and the secret's version history:

  • Nothing changed (the vault write failed, or the apply failed and the old value was written back): vault, database and containers agree on the old value.
  • The vault holds the new value and the containers still run with the old one (the recreation was refused or failed): the database already accepts the new value. This is a known state, not corruption; the containers must be rebuilt so they read the vault.
  • "vault and system now DISAGREE" in the error: the vault holds a value the database never accepted. Restore the previous version after clearing the claim.

Then clear the claim in the panel's database, on the host. On a default install open it with docker exec -it aiadminpanel_postgresql psql -U aiadminpanel.

-- 1. Inspect. One row with state 'in_doubt' is the blocking claim.
SELECT service_id, state, owner_id, acquired_at
  FROM service_operation_claims WHERE service_id = '<service id>';

-- 2. Clear it. Only an in-doubt claim; never an 'active' one.
DELETE FROM service_operation_claims
 WHERE service_id = '<service id>' AND state = 'in_doubt';

The service ID is in the address of the service's page in the panel. Before step 2, check in the panel log that no job for this service is still running; an active row means one is, and must be left alone.

  1. Then bring the service in line with the vault: press Redeploy, which rebuilds the containers with the vault's current values. If Redeploy stops at pull_images with a Compose parse error, rotate the secret again instead; that also rebuilds them. In the "nothing changed" state simply rotate again.
  2. Check application login or the database connection, and every dependent application. Keep secret values out of logs and support messages.

Previous versions and unsupported secrets

Restoring a previous version writes it forward as a new version and uses the same apply/recreate process. It can interrupt the application and needs the same verification.

A version whose value is gone cannot be restored. That is the case for a version that was deleted, for example the copy left behind when a service was moved to another customer and later moved back. The version history offers no Restore for it, and a restore request answers "version N no longer exists and cannot be restored" without changing anything. Restore never brings a deleted version back. If a service already runs with an empty credential from such a restore made before this fix, rotate that secret once: rotation writes a fresh value. Legacy services may need a vault migration before these controls appear. Redeployment reuses existing generated vault values; it is not a substitute for rotation. Change application-managed accounts inside the application using its supported recovery procedure.