Imagine a project notebook whose pages contain comments and attached photographs. After a restore, the pages appear, the comments are legible, and every photograph is a broken rectangle. The database may have returned perfectly. The application has returned only in part.
This invented example gives a recovery test a useful starting point: name the thing a person should be able to do. Opening a page and its attachment is a stronger claim than finding a backup file. It also identifies the boundary of the test. Reading an old page says little about whether someone can save a new one.
A restore produces evidence about a particular collection of data, a procedure, and an environment. Recording those conditions makes a successful rehearsal easier to interpret when the original machine, credentials or operator are unavailable.
Read what the check checks
Backup software offers checks at different levels. In restic 0.18.0, the ordinary check command examines repository structure and consistency. Adding --read-data also reads the repository's pack files to verify their integrity. That additional work can require transferring substantial data. A record saying only “backup checked” conceals which operation ran and how much evidence it gathered.
Even a thorough repository check leaves an application question open. The tool can establish that the stored objects fit its expectations without knowing whether a particular notebook page ought to have a photograph attached. That relationship belongs to the application and to whoever defines its recovery requirements.
PostgreSQL makes the boundary unusually clear in its documentation for pg_verifybackup. The tool compares a base backup with its manifest and can examine the log records needed for recovery. Its documentation nevertheless calls for test restores and inspection of the resulting databases. A running server performs checks that this verification cannot reproduce. The distinction is a reason to use both kinds of evidence: one can reveal damage before a more expensive rehearsal; the other exercises recovery itself.
For the notebook, I would record those outcomes separately. Repository integrity passed. Database recovery passed. Attachment retrieval failed. Three precise statements give the next investigation somewhere to begin.
Rebuild the surroundings
A database lives among settings, identities and other stores of information. Which of these a backup contains depends on the backup method. The word “database” does not settle the inventory.
Consider PostgreSQL's logical dumps. Its pg_dumpall reference identifies shared objects, including database roles and tablespaces, that pg_dump does not save. A rehearsal on an established server might quietly rely on roles already present there. A fresh destination can expose that dependency. The lesson is to account for shared objects when choosing a recovery procedure; it does not make every application a candidate for the same dump command.
Physical recovery has another boundary. PostgreSQL's continuous-archiving documentation explains that replaying archived write-ahead logs does not recover changes to manually edited configuration files such as postgresql.conf and pg_hba.conf. The logs carry database changes. They do not provide a history of every setting surrounding the database.
In our hypothetical notebook, the photographs live in an object store and the database contains their names. Restoring the rows would recover those names. To recover the photographs, the procedure also needs the objects and a working way for the restored application to read them. A missing photograph would reveal a gap in this application's recovery procedure, beyond the scope of its database integrity check.
Writing that dependency down changes the test. We can select a known page, follow its attachment reference, and compare the returned photograph with the expected one. That operation exercises the connection between the two stores of data.
Make missing pieces visible
A destination with familiar files and credentials is comfortable. It may also supply the very things the backup omitted. If the purpose is to rehearse recovery after losing that environment, those conveniences need to be removed or declared as assumptions.
The restic restore guide shows how to restore into a specified target directory and how to select a particular snapshot. It also warns that an interrupted restore over existing files can leave them partly restored. For the notebook rehearsal, a separate empty destination would make the result easier to inspect and keep the exercise away from the working copy. That is a proposed test design, not a guarantee supplied by the choice of directory.
The same discipline applies to access. If the rehearsal uses a password already available in a shell session, record that dependency. A second exercise might begin with a colleague who has only the documented recovery route. Any missing instruction then becomes visible while there is time to repair it.
A realistic test need not contact real recipients or modify production records. The notebook could use isolated storage and a controlled test account. Within that boundary, it should attempt a meaningful operation: open a recovered page, retrieve its photograph, save a change, and read the change back. If external delivery is excluded, the report should say so. A carefully bounded result remains useful without being enlarged into a claim about the whole service.
Give recovery two clocks
Recovery has a past to reach and a duration to endure. NIST's contingency-planning guide distinguishes the recovery point objective, concerning how far back data may need to be recovered and the associated tolerance for loss, from the recovery time objective, concerning how long a system resource can remain unavailable. It also separates checking recovered data from testing system functionality and calls for explicit test objectives and success criteria.
Suppose the notebook's chosen exercise begins with a simulated failure at 11:00. The accepted data target is 10:45, and the team wants usable service within two hours. Those are invented requirements. They let us ask two independent questions: does the restored system contain the expected work through 10:45, and when can the test user actually use it?
The file download might finish quickly while database recovery or access setup consumes the remaining time. Stopping the timer at the download would answer a smaller question. Define the start and finish before the exercise, then record intermediate stages so that a slow result can be understood.
A second run may improve because the instructions improved. It may also improve because the operator remembers an undocumented step. Having someone else follow the revised procedure helps distinguish those explanations. Keep the failed attempt, the correction and the next result together; otherwise the smoothest run can erase the evidence that made it possible.
The most useful artifact from this rehearsal may be a short record beside the backup inventory: snapshot identifier, recovered data point, destination, software versions, access assumptions, elapsed stages and the user actions that passed. Add the unresolved failures. When a colleague opens that record months later, they should be able to decide what to repeat without needing the original operator to remember what “successful” meant.
A thought worth keeping?
