Restic is a multi-platform, incremental, encrypted backup solution featuring automatic deduplication and written in Go. It’s actively developed and available for all platforms including various flavors of Linux, MacOSX, and Windows.
While most Linux distributions include various versions, it’s better to use official binaries because they often contain important fixes and enchantments. It’s also possible to build from sources using Go 1.13 or newer.
Initialization
Restic saves files in repositories and they should be initialized before they can be used. Creating an empty repository requires two things, the folder where they should be saved and an initial password used for encryption. It also supports remote file systems (more on that later) but for now, we’re going to create a local repository.
$ restic init -r test
enter password for new repository:
enter password again:
created restic repository c580929a52 at test
Please note that knowledge of your password is required to access
the repository. Losing your password means that your data is
irrecoverably lost.After a few seconds, an empty repository is created and it’s ready to be used. The password will be required for all future operations on the repository and losing it means that the data contained in the repository is irreversibly lost.
Doing our first backup
As an example, we’ll back up a folder with a few hundred files called “stuff”. Please note that all the commands contain the “-r test” parameter that sets the repository folder.
$ restic backup -r test stuff
enter password for repository:
repository c580929a opened successfully, password is correct
no parent snapshot found, will read all files
Files: 1105 new, 0 changed, 0 unmodified
Dirs: 8 new, 0 changed, 0 unmodified
Added to the repo: 46.419 MiB
processed 1105 files, 104.422 MiB in 0:00
snapshot 0141fcd7 saved
We can see that the folder contained around 104MB of data but it took only 46MB in the backup repository because of deduplication. We can verify it with the “du” command:
$ du -sh test stuff
47M test
107M stuffIt’s important to note that Restic does not support compression because it would hugely complicate operations such as deduplication but it’s often a worthy trade-off in typical usage scenarios.
Incremental backups
Let’s change a few files and do another round of backup of the same folder. In our example, this folder contains a couple of JPEG files and I simply ran “jpegoptim *jpg” to change them. Simply repeating the previous backup command will now automatically do a second round of (incremental) backup, adding updated or new files. These are called “snapshots”
$ jpegoptim stuff/jpg/*jpg
$ restic backup -r test stuff
enter password for repository:
repository c580929a opened successfully, password is correct
using parent snapshot 0141fcd7
Files: 0 new, 106 changed, 999 unmodified
Dirs: 0 new, 2 changed, 6 unmodified
Added to the repo: 15.447 MiB
processed 1105 files, 104.043 MiB in 0:00
snapshot 2ea7040e saved
Repeating the same command without changing any file will result in a new (empty) snapshot being created:
$ restic backup -r test stuff
enter password for repository:
repository c580929a opened successfully, password is correct
using parent snapshot 2ea7040e
Files: 0 new, 0 changed, 1105 unmodified
Dirs: 0 new, 0 changed, 8 unmodified
Added to the repo: 0 B
processed 1105 files, 104.043 MiB in 0:00
snapshot 1d140010 savedListing snapshots
To find out how many snapshots we have, we can use the “restic snapshots” command:
$ restic -r test snapshots
enter password for repository:
repository c580929a opened successfully, password is correct
ID Time Host Tags Paths
-----------------------------------------------------------------
0141fcd7 2022-03-08 09:18:36 test /stuff
2ea7040e 2022-03-08 09:24:48 test /stuff
1d140010 2022-03-08 09:27:00 test /stuff
------------------------------------------------------------------
3 snapshots
Each snapshot has an ID that can be used as a reference when doing operations on snapshots, such as removing them or restoring backups. Deduplication works across the whole repository so it comes in quite handy if multiple sources (or even machines) contain similar files because they’re saved only once in the backup repository.
Restoring files from backup
Restoring a file from backup works by running a “restic backup” command. It’s also possible to mount the backup (through Fuse) and browse files using a virtual filesystem. By default, it restores all the files from the backup that can be limited by using the “-i pattern” parameter, while “-t” sets the target folder to restore to. The command also requires the snapshot ID (see above) right after the “restore” parameter and “latest” is an alias to the latest backup’s snapshot ID.
$ restic -r test restore latest -t restored -i stuff/jpg
enter password for repository:
repository c580929a opened successfully, password is correct
restoring <Snapshot 1d140010 of [/stuff] at 2022-03-08 09:27:00.412616 +0100 CET by ttb@test> to restoredNow let’s try to mount the repository to demonstrate how this works:
$ mkdir mnt
$ restic mount -r test mnt
enter password for repository:
repository c580929a opened successfully, password is correct
Now serving the repository at mnt
When finished, quit with Ctrl-c or umount the mountpoint.From another console now it’s possible to browse files in all snapshots and copy files from while Resting is running the background transparently serving files. Once you’re done restoring, simply press Ctrl-c to unmount and clean up.
$ ls -1 mnt/snapshots/
2022-03-08T09:18:36+01:00
2022-03-08T09:24:48+01:00
2022-03-08T09:27:00+01:00
latest
$ du -sh mnt/snapshots/*/stuff/jpg
27M mnt/snapshots/2022-03-08T09:18:36+01:00/stuff/jpg
26M mnt/snapshots/2022-03-08T09:24:48+01:00/stuff/jpg
26M mnt/snapshots/2022-03-08T09:27:00+01:00/stuff/jpg
26M mnt/snapshots/latest/stuff/jpgIntegrity checking
The great thing about Restic is that it’s possible to have rolling backups of a given amount of days, weeks, months without having to a do full backup over and over. This is different from for example Duplicity, where it’s not possible to erase an old full backup because newer incremental ones depend on it. You can read more about Duplicity here: Safe and reliable Linux backups with Duplicity.
Each Restic snapshot works on its own and it is possible to freely delete any of the previous ones. This also means (especially with deduplication) that data corruption can have disastrous consequences so care must be taken to periodically verify backup integrity and test restoration.
Let’s check the full integrity of our backup repository using “restic check”. The “–read-data” forces all the repository data to be re-read and checked. This may be quite a costly operation with remote repositories, so “–read-data-subset X/Y” also exists that only check a small percentage or fraction of the data.
$ restic check -r test --read-data
using temporary cache in /var/folders/58/x8jjwggn3x716y_g8_n01n740000gn/T/restic-check-cache-566119768
enter password for repository:
repository c580929a opened successfully, password is correct
created new cache in /var/folders/58/x8jjwggn3x716y_g8_n01n740000gn/T/restic-check-cache-566119768
create exclusive lock for repository
load indexes
check all packs
check snapshots, trees and blobs
read all data
[0:00] 100.00% 3 / 3 snapshots
[0:00] 100.00% 19 / 19 packs
no errors were foundNo errors, all is good.
Snapshot maintenance
While backups can be done any number of times it’s often useful to remove old snapshots or at least decrease the frequency of older ones, for example, keep a daily backup for a week, then a weekly backup for a month then a monthly backup for a year.
Restic provides the “restic forget” command that removed snapshots according to a policy automatically. It’s important that the command only removes references to the data from snapshots – data should be then “pruned” so it’s really removed for good.
Let’s test this first by removing the middle snapshot in our example (ID: 2ea7040e)
$ restic forget -r test 2ea7040e
enter password for repository:
repository c580929a opened successfully, password is correct
[0:00] 100.00% 1 / 1 files deletedWe can now verify that it’s gone:
$ restic -r test snapshots
enter password for repository:
repository c580929a opened successfully, password is correct
ID Time Host Tags Paths
------------------------------------------------------------------
0141fcd7 2022-03-08 09:18:36 test /stuff
1d140010 2022-03-08 09:27:00 test /stuff
------------------------------------------------------------------
2 snapshotsThe data is still being allocated for the previously deleted snapshot – this can be deleted by running “restic prune”. Adding “–prune” to “restic forget” does this automatically in one step.
$ restic prune -r test
enter password for repository:
repository c580929a opened successfully, password is correct
loading indexes...
loading all snapshots...
finding data that is still in use for 2 snapshots
[0:00] 100.00% 2 / 2 snapshots
searching used packs...
collecting packs for deletion and repacking
[0:00] 100.00% 19 / 19 packs processed
to repack: 0 blobs / 0 B
this removes 0 blobs / 0 B
to delete: 0 blobs / 0 B
total prune: 0 blobs / 0 B
remaining: 722 blobs / 61.887 MiB
unused size after prune: 0 B (0.00% of remaining size)
doneIn our case backup snapshots #2 and #3 contained the same data hence the 0 bytes in repack/remove/delete/prune – normally space is freed when pruning real snapshots.
To remove snapshots according to a policy, “restic forget” can be told to automatically keep daily (-d), weekly (-w), monthly (-m), yearly (-y) backups. The command below will keep 7 daily and 4 weekly backups (inclusive, so one of the dailies will be considered weekly, too).
Running this in our case will leave one backup only (the latest) because of the “-d 7” parameter. The output of this is quite large, it will list snapshots it keeps/removes and automatically runs prune to release unused backup space.
$ restic forget -r test -d 7 -w 4 --prune
enter password for repository:
repository c580929a opened successfully, password is correct
Applying Policy: keep 7 daily, 4 weekly snapshots
keep 1 snapshots:
ID Time Host Tags Reasons Paths
------------------------------------------------------------------
1d140010 2022-03-08 09:27:00 test daily snapshot /stuff
weekly snapshot
------------------------------------------------------------------
1 snapshots
remove 1 snapshots:
ID Time Host Tags Paths
-------------------------------------------------------------------
0141fcd7 2022-03-08 09:18:36 test /stuff
-------------------------------------------------------------------
1 snapshots
[0:00] 100.00% 1 / 1 files deleted
1 snapshots have been removed, running prune
loading indexes...
loading all snapshots...
finding data that is still in use for 1 snapshots
[0:00] 100.00% 1 / 1 snapshots
searching used packs...
collecting packs for deletion and repacking
[0:00] 100.00% 19 / 19 packs processed
to repack: 9 blobs / 422.116 KiB
this removes 3 blobs / 83.062 KiB
to delete: 0 blobs / 0 B
total prune: 3 blobs / 83.062 KiB
remaining: 719 blobs / 61.806 MiB
unused size after prune: 0 B (0.00% of remaining size)
repacking packs
[0:00] 100.00% 1 / 1 packs repacked
rebuilding index
[0:00] 100.00% 19 / 19 packs processed
deleting obsolete index files
[0:00] 100.00% 2 / 2 files deleted
removing 1 old packs
[0:00] 100.00% 1 / 1 files deleted
doneThis should give you a good introduction to using Restic. In the next part of this article, we’ll talk about advanced uses, how to automate backups and back up / restore a complete system, how to use remote filesystems for backing up and various security considerations.
