Configuration v2
Note
The configuration file format v2 is in preview. It has been available since v0.17.0: mixins were added in v0.18.0 and group schedules in v0.29.0.
The format v1 remains the default and keeps working unchanged.
Introduction
The original file format (v1) was designed when resticprofile only supported toml. Every profile is a section at the top level of the file, next to the reserved sections global, groups and includes.
The format v2 gives each kind of declaration its own section. This makes room for features that cannot be expressed in v1:
- Groups with their own configuration: description, error handling and schedules
- Mixins to share pieces of configuration between profiles
- Predictable inheritance of lists, with a syntax to append or prepend values
- Safer defaults for
backupandretention
Selecting the version
The format is selected by the version property at the top level of the configuration file:
version | Format used |
|---|---|
| not set | v1 |
"1" | v1 |
"2" | v2 |
A value that is not a number selects the format v1.
The version can be written as a number (version: 2), but the JSON schema only accepts a string (version: "2").
File formats
The format v2 is available for:
- TOML
- YAML
- JSON
It is not available for HCL. A HCL file that declares version = 2 fails to load with the error:
HCL files can still be used with the format v1.
Structure
A v2 configuration file contains up to six top level properties:
| Property | Purpose |
|---|---|
version | Selects the file format, must be "2" |
global | Settings that apply to all profiles |
includes | Other configuration files to merge into this one |
mixins | Named pieces of configuration that profiles can use |
groups | Named groups of profiles |
profiles | Named profiles |
Warning
Any other property at the top level is ignored. A profile that is declared at the top level, as in the format v1, is not loaded and no error is reported.
Differences with v1
v1 | v2 | |
|---|---|---|
version | Optional | Required: "2" |
| File formats | TOML, YAML, JSON, HCL | TOML, YAML, JSON |
| Profiles | Sections at the top level | Nested in the profiles section |
| Groups | A list of profile names | An object: profiles, description, continue-on-error, schedules |
| Group schedules | Not available | Available |
| Mixins | Not available | mixins section and use property |
| Lists in derived profiles | Merged with the list of the parent, index by index | Replace the list of the parent, or append and prepend to it |
host in backup | Not set | Defaults to the host name of the system |
after-backup in retention | Not set | Enabled when a keep-* flag is set |
tag and host in retention | Not set | Default to the values used by backup |
Everything else is identical in both formats: the global section, the content of a profile, variables, templates, includes and schedules of a profile.
Sections
global
The global section is the same as in the format v1. See the reference for the list of properties.
profiles
All profiles are nested in the profiles section. The content of a profile is the same as in the format v1: see the reference for the list of properties.
Since profiles have their own section, a profile can use any name, including global, groups, includes, mixins and version.
groups
A group is a list of profiles that run one after the other, in the order they are listed.
In the format v1 a group is a list of profile names. In the format v2 a group is an object, and the list of profile names moves to its profiles property:
| Property | Type | Default | Purpose |
|---|---|---|---|
profiles | list of profile names | Profiles that belong to the group, in the order they run | |
description | string | Describes the group, displayed by resticprofile profiles | |
continue-on-error | true, false, auto | auto | Keep running the next profiles after a profile failed |
schedules | object | Schedules of the group, see scheduling a group |
A group is used like a profile: its name is given to the --name flag, or as a prefix of the command.
Both commands run backup on the profile root, then on the profile documents.
resticprofile profiles lists the groups with their profiles and description, and resticprofile --name full-backup show displays the configuration of a group, including its schedules.
Warning
Profiles and groups share the same names on the command line. Use a name only once: when a profile and a group have the same name, the profile is selected and resticprofile reports Duplicate profile and group names found.
Error handling
By default, a group stops at the first profile that fails and the remaining profiles do not run.
continue-on-error changes this for one group:
true: a failure is logged and the group continues with the next profilefalse: the group stops at the first failureauto(or not set): the group followsgroup-continue-on-errorfrom theglobalsection, which isfalseby default
global.group-continue-on-error is also available in the format v1. Setting it per group requires the format v2.
Scheduling a group
A group can be scheduled as a whole. When the schedule triggers, the scheduled command runs on each profile of the group in sequence, following the error handling of the group.
The schedules property of a group contains one schedule per command. The commands that can be scheduled are backup, check, forget, prune and copy.
Each schedule is an object. Only at is required: a schedule without it is ignored.
The properties are the schedule-* properties of a profile, without the schedule- prefix. They are described in schedule configuration:
| Property of a group schedule | Equivalent in a profile |
|---|---|
at | schedule |
permission | schedule-permission |
run-level | schedule-run-level |
log | schedule-log |
priority | schedule-priority |
lock-mode | schedule-lock-mode |
lock-wait | schedule-lock-wait |
capture-environment | schedule-capture-environment |
ignore-on-battery | schedule-ignore-on-battery |
ignore-on-battery-less-than | schedule-ignore-on-battery-less-than |
after-network-online | schedule-after-network-online |
hide-window | schedule-hide-window |
start-when-available | schedule-start-when-available |
A group schedule also accepts command-output and systemd-drop-in-files. All the properties are listed in the reference.
A property that is not set takes its value from schedule-defaults in the global section.
The schedule of a group is installed, removed and checked with the schedule commands and the name of the group:
resticprofile schedule --all installs the schedules of all profiles and all groups.
Note
The schedules of a group are independent of the schedules of its profiles. If the group schedules backup and one of its profiles also schedules backup, both schedules are installed and the profile is backed up by both.
mixins
Mixins are named pieces of configuration, declared in the mixins section. A profile merges a mixin into its configuration with the use property, at any level of the profile. The use property is also accepted in a group and in the global section.
Mixins are described in detail in inheritance.
includes
includes works as in the format v1, see includes. Two rules apply to the version:
- All the files must use the same version. Loading fails with
cannot include different versions of the configuration file, all files must use the same versionwhen the main file and an included file declare a differentversion. - An included file that does not declare a
versionis read with the version of the main file. Declaring the version in every file is recommended, and the JSON schema of the formatv2requires it.
An included v2 file follows the same structure as the main file: its profiles are in the profiles section. Files in TOML, YAML and JSON can be mixed, and a mixin declared in one file can be used in any other file.
Changes in profile behaviour
The content of a profile is the same in both formats, but a v2 profile is not always resolved to the same configuration as its v1 equivalent.
Tip
Use resticprofile --name <profile-name> show to display the resulting configuration of a profile, and resticprofile --dry-run --name <profile-name> backup to display the restic commands it generates.
Inheritance of lists
When a profile inherits from another profile and both define the same list (source, exclude, tag, run-before…):
- In the format
v1, the two lists are merged index by index. Withsource: [/a, /b]in the parent andsource: [/c]in the derived profile, the result is[/c, /b]. - In the format
v2, the list of the derived profile replaces the list of the parent. With the same example, the result is[/c].
To add values to the list of the parent instead of replacing it, the format v2 adds a syntax to append or prepend:
| Property | Effect |
|---|---|
<list-property>... or <list-property>__APPEND | Appends to the list of the parent |
...<list-property> or <list-property>__PREPEND | Prepends to the list of the parent |
The value of exclude in the profile documents is [".*", "~*", ".git"].
The same syntax is available in mixins.
Defaults for backup and retention
The format v2 sets a few defaults so that a backup and the retention that follows it select the same snapshots. These defaults are not applied in the format v1.
In the backup section:
hostdefaults totrue, which is replaced by the host name of the system. Sethostin thebackupsection to use another name.
In the retention section:
after-backupdefaults totruewhen at least onekeep-*flag is set and neitherafter-backupnorbefore-backupis set. Declaring a retention policy is enough to apply it after each backup.tagdefaults totruewhentagis set in thebackupsection. The retention only considers the snapshots that have the tags of the backup.hostdefaults to thehostof thebackupsection. The retention only considers the snapshots of the same host.
Setting any of these properties explicitly in the retention section overrides its default.
For example, this profile:
is resolved as:
The same profile in the format v1 only sets path in the retention section: the retention does not run after the backup, and forget is not restricted to a host or to the tags of the backup.
JSON schema
The format v2 has its own JSON schema:
- https://resticprofile.creativeprojects.tech/jsonschema/config.json detects the version of the file and validates both formats
- https://resticprofile.creativeprojects.tech/jsonschema/config-2.json only validates the format
v2
The schema of the format v2 requires the version property, as a string.
A schema can also be generated locally:
Migrating from v1
A v1 configuration is converted in a few steps:
- Convert the file to TOML, YAML or JSON if it is a HCL file.
- Add
version: "2"at the top of the file, and of every included file. - Move all the profiles into a
profilessection. Leaveglobal,groupsandincludesat the top level. - Change each group from a list of profile names to an object with a
profilesproperty. - Review the lists in the profiles that use
inherit: a list now replaces the list of the parent. Rename the property with a...suffix where values were meant to be added. - Review the
retentionsections: a section with akeep-*flag now runs after each backup, and is restricted to the host and the tags of the backup. Setafter-backup: falseto keep a retention policy that does not run after a backup. - Check the result with
resticprofile profiles, which lists all the profiles and groups, then withshowand--dry-runon each profile.
The names of the profiles and groups do not change: the schedules that are already installed keep working as long as the configuration file keeps its name.
Complete example
With this configuration:
resticprofile --name home backupbacks up/home, excluding*.tmp,**/.cacheand**/node_modules, then applies the retention policy: 60 daily and 8 weekly snapshots are kept.resticprofile --name all backupbacks upsystemthenhome, and backs uphomeeven ifsystemfailed.resticprofile --name all scheduleinstalls two schedules:backupevery day at 02:30 andcheckevery Sunday at 05:00. Both run onsystemthenhome, with the permission and the log file ofschedule-defaults.
Tip
The format v2 is discussed in this issue.