MAN page from openSUSE Tumbleweed btrbk-0.32.5-8.43.noarch.rpm

BTRBK.CONF

Section: Btrbk Manual (5)
Updated: 2022-10-23
Index

NAME

btrbk.conf - btrbk configuration file

SYNOPSIS

/etc/btrbk.conf/etc/btrbk/btrbk.conf

DESCRIPTION

The btrbk configuration file specifies which btrfs subvolumes on thefilesystem are to be processed, what target subvolumes should be usedto create the backups, and where the snapshots should begenerated. The retention policy, as well as most other options can bedefined either globally or within a section.

The options specified always apply to the last section encountered,superseding the values set in upper-level sections. This means thatglobal options must be set before any sections are defined.

Blank lines are ignored. A hash character (#) starts a commentextending until end of line.

SECTIONS

volume <volume-directory>|<url> (optional)

: Absolute path pointing to a btrfs file system containing thesource subvolume(s) to be backed up. Usually the mount point of abtrfs filesystem mounted with the subvolid=5 option.

subvolume <subvolume-name>

: Subvolume to be backed up, relative to the <volume-directory> ofthe volume section, or absolute if the volume section isomitted. Accepts wildcard character "*".
Note that if this subvolume is btrfs root (id=5), it needs to have avalid UUID, which is not the case for file systems created withbtrfs-progs < 4.16.

target [send-receive|raw] <target-directory>|<url>

: Target directory where the backup subvolumes are to becreated. The optional target type defaults to "send-receive",see TARGET TYPES below for details.
Multiple target sections are allowed, in any context: a targetdefined in volume or global context will be used for all underlyingsubvolume sections (hint: run "btrbk list" or "btrbk config print"to see the resulting configuration).

If a <url> is specified, btrbk actions (shell commands) are executedremotely via ssh, using the SSH Options describedbelow. Accepted formats are:

ssh://<hostname>[:<port>]/<directory><hostname>:<directory>

Where <hostname> is either a host name, an IPv4 address indotted-decimal form, or an IP literal encapsulated within squarebrackets (e.g. "[2001:db8::7]").

If you are connecting to virtual machines, consider configuringseveral volume sections for a <hostname>, with distinct <port>numbers for each machine.

OPTIONS

The options described here can be specified in global context aswell as volume, subvolume and target sections, unless statedotherwise.

Basic Options

timestamp_format short|long|long-iso

Timestamp format used as postfix for new snapshot subvolumenames. Defaults to "long".

short

: YYYYMMDD[_N] (e.g. "20150825", "20150825_1")

long

: YYYYMMDD<T>hhmm[_N] (e.g. "20150825T1531")

long-iso

: YYYYMMDD<T>hhmmss±hhmm[_N] (e.g. "20150825T153123+0200")

Note that a postfix "_N" is appended to the timestamp if a snapshot orbackup already exists with the timestamp of current date/time.

Use "long-iso" if you want to make sure that btrbk nevercreates ambiguous time stamps (which can happen if multiplesnapshots are created during a daylight saving time clockchange).

Note that using "long-iso" has implications on the scheduling, seeReference Time below.

snapshot_dir <directory>

: Directory in which the btrfs snapshots are created, relative to<volume-directory> of the volume section, or absolute if thevolume section is omitted. Note that btrbk does notautomatically create this directory, and the snapshot creationwill fail if it is not present.

snapshot_name <basename>

: Base name of the created snapshot (and backup). This option isonly valid in the subvolume section. Defaults to<subvolume-name>.

snapshot_create always|onchange|ondemand|no

: If set to "always", snapshots are always created. If set to"onchange", snapshots are only created if the last snapshot isnot up-to-date, i.e. the source subvolume has changed (moreprecisely: the btrfs generation has been increased) since the lastsnapshot was created. If set to "ondemand", snapshots are onlycreated if at least one target subvolume is reachable (useful ifyou are tight on disk space and you only need btrbk for backups toan external disk which is not always connected). If set to "no",the snapshots are never created (useful if another instance ofbtrbk is taking care of snapshot creation). Defaults to"always".

incremental yes|no|strict

: If set, incremental backups are created. If set to "strict",non-incremental (initial) backups are never created, andincremental backups are restricted to related parents (byparent-uuid relationship). Defaults to "yes".
Note that even if the parent-uuid chain is broken, snapshots andbackups can still share data (which is especially true for backupscreated with incremental option enabled), and are perfectly suitableas parents for incremental send-receive operations. But as btrbk cannot be certain about this, such operations are disallowed in"incremental strict" mode.

noauto yes|no

: If set, the context is skipped by all btrbk actions unlessexplicitly enabled by a matching btrbk <filter> command lineargument (e.g. "btrbk run myfilter").

Grouping Options

group <group-name> [<group-name>]...

: Add the current section (volume, subvolume or target) touser-defined groups, which can be used as filter for most btrbkcommands (see btrbk(1) FILTER STATEMENTS). This option can be setmultiple times within the same context.

Retention Policy Options

preserve_day_of_week monday|tuesday|...|sunday

: Defines on what day a snapshot/backup is considered to be a"weekly" backup. Weekly, monthly and yearly backups are preservedon this day of week (see RETENTION POLICYbelow). Defaults to "sunday".

preserve_hour_of_day [0..23]

: Defines after what time (in full hours since midnight) asnapshot/backup is considered to be a "daily" backup. Daily,weekly, monthly and yearly backups are preserved on this hour (seeRETENTION POLICY below). Ignored onsnapshots or backups without time information (timestamp_formatshort). Defaults to "0".

snapshot_preserve no|<retention_policy>

: Set retention policy for snapshots (seeRETENTION POLICY below). If set to "no",preserve snapshots according to snapshot_preserve_minonly. Defaults to "no".
Note that snapshot_preserve has no effect if snapshot_preserve_minis set to "all" (the default).

snapshot_preserve_min all|latest|<number>{h,d,w,m,y}

: Preserve all snapshots for a minimum amount of hours (h), days(d), weeks (w), months (m) or years (y), regardless of how manythere are. If set to "all", preserve all snapshots forever. Ifset to "latest", preserve latest snapshot. Defaults to "all".

target_preserve no|<retention_policy>

: Set retention policy for backups (seeRETENTION POLICY below). If set to "no",preserve backups according to target_preserve_min only. Defaultsto "no".
Note that target_preserve has no effect if target_preserve_min isset to "all" (the default).

target_preserve_min all|latest|no|<number>{h,d,w,m,y}

: Preserve all backups for a minimum amount of hours (h), days (d),weeks (w), months (m) or years (y), regardless of how many thereare. If set to "all", preserve all backups forever. If set to"latest", always preserve the latest backup (useful inconjunction with "target_preserve no", if you want to keep thelatest backup only). If set to "no", only the backups followingthe target_preserve policy are created. Defaults to "all".

archive_preserve no|<retention_policy>

: Set retention policy for archives ("btrbk archive" command), withsame semantics as target_preserve.

archive_preserve_min all|latest|no|<number>{h,d,w,m,y}

: Set retention policy for archives ("btrbk archive" command), withsame semantics as target_preserve_min.

archive_exclude <pattern>

: Exclude subvolumes matching <pattern> from archiving. The patternaccepts wildcard character "*", and is matched against the end ofthe pathname.

SSH Options

ssh_identity <file>|no

: Absolute path to a ssh identity file (private key). If not set,the ssh default is used (see ssh(1), "-i identity_file"). Notethat if the identity key is password protected and noauthentication agent is used, btrbk will prompt for user input onevery connection attempt.

ssh_user <username>|no

: Remote username for ssh. Defaults to "root". Make sure theremote user is able to run "btrfs" with root privileges (seeoption backend for details). If set to "no", the ssh defaultis used.

ssh_compression yes|no

: Enables or disables the compression of ssh connections. Defaultsto "no". Note that if stream_compress is enabled, sshcompression will always be disabled for send/receive operations.

ssh_cipher_spec default|<cipher_spec>

: Selects the cipher specification for encrypting the session(comma-separated list of ciphers in order of preference). See the"-c cipher_spec" option in ssh(1) for more information. Defaultsto "default" (the ciphers specified in ssh_config(5)).

Data Stream Options

stream_compress <compress_command>|no

: Compress the btrfs send stream before transferring it from/toremote locations. Defaults to "no". If enabled, make sure that<compress_command> is available on the source and targethosts. Supported <compress_command>: gzip, pigz, bzip2, pbzip2,xz, lzo, lz4, zstd.

stream_compress_level default|<number>

: Compression level for the specified <compress_command>. Refer tothe related man-page for details (usually [1..9], where 1 meansfastest compression). Defaults to "default" (the defaultcompression level of <compress_command>).

stream_compress_long default|<number>

: Enable long distance matching for the specified<compress_command>. Refer to the related man-page for details.Only supported for "zstd".

stream_compress_threads default|<number>

: Number of threads to use for <compress_command>. Only supportedfor "pigz", "pbzip2", "zstd" and recent versions of "xz".

stream_compress_adapt default|<number>

: Enable adaptive compression for <compress_command>. Only supportedfor "zstd" (version >= 1.3.6).

stream_buffer <size>|no

Add a buffer to the btrfs send stream (locally, on uncompresseddata), with a maximum size of <size>. This can give a speedimprovement (measured up to 20%) on both local or remoteoperations, but also increases system load. A suffix of "k", "m","g", or "%" can be added to <size> to denote kilobytes (*1024),megabytes, gigabytes, or a percentage of total physicalmemory. Defaults to "no".

If enabled, make sure that the "mbuffer" command (at least version20180505) is available on the host running btrbk. As of btrbk-0.29.0,mbuffer(1) is used for both rate_limit and stream_buffer options:

mbuffer [-m <stream_buffer>] [-r <rate_limit>]

Note that mbuffer(1) always reads defaults from "/etc/mbuffer.rc"and "~/.mbuffer.rc".

Leave this option disabled if your main concern is a stable backupprocess: while recent versions of mbuffer have proven reliable, it isoften desirable to keep things simple rather than adding anadditional, multi-threaded process to the command pipe.

stream_buffer_remote <size>|no

: Add a buffer on remote hosts (either source or target). Defaultsto "no".
Enable this if you prefer buffering on the remote side, or even onboth sides: reasons for this depend on available memory, disk and cpuperformance (btrfs send/receive, compression), as well as networkingconstraints.

rate_limit <rate>|no

: Limit the read rate of the btrfs send stream to <rate> bytes persecond (locally, on uncompressed send stream). A suffix of "k","m", "g", or "t" can be added to denote kilobytes (*1024),megabytes, and so on. Defaults to "no". Note that rate_limitimplicitly adds a stream buffer (see stream_buffer optionabove).

rate_limit_remote <rate>|no

: Add rate limit on remote hosts (either source or target). Defaultsto "no". Note that it usually does not make much sense to enableboth rate_limit and rate_limit_remote.

System Options

transaction_log <file>|no

: If set, all transactions (snapshot create, subvolume send-receive,subvolume delete) as well as abort messages are logged to <file>,in a space-separated table format: "localtime type statustarget_url source_url parent_url message".

transaction_syslog <facility>|no

: If set, all transactions (as described in transaction_log above)are logged to syslog. The program name used in the messages is"btrbk". Accepted parameters for <facility>: user, mail,daemon, auth, lpr, news, cron, authpriv, local0..local7.

lockfile <file>|no

: Create lockfile <file> on startup; checks lockfile before runningany btrfs commands (using perl "flock"), and exits if the lock isheld by another btrbk instance. Ignored on dryrun (-n,--dry-run). See also --lockfile command-line option.

backend <backend>

Backend filesystem utilities to be used for btrfs specificoperations. Available backends:

btrfs-progs

: Default backend, btrfs commands are called as specified inbtrfs(8) (e.g. "btrfs subvolume show").

btrfs-progs-btrbk

: btrfs commands are separated by a dash instead of a whitespace(e.g. "btrfs-subvolume-show" instead of "btrfs subvolumeshow"). Useful for setting suid or file capabilities (setcap) onspecific btrfs commands, as implemented in <https://github.com/digint/btrfs-progs-btrbk>.

btrfs-progs-sudo

: btrfs commands are prefixed with "sudo -n" (e.g. "sudo -n btrfssubvolume show" instead of "btrfs subvolume show"). Make sure tohave appropriate (root) permissions for the "btrfs" command groupsas well as the "readlink" and "test" commands in /etc/sudoers.

btrfs-progs-doas

: Similar to btrfs-progs-sudo, using prefix "doas -n".

If you want to set this option for local or remote hosts only, setbackend_local or backend_remote (e.g. "backend_remotebtrfs-progs-btrbk").

If you want to set this option for regular (non-root) user only, setbackend_local_user.

compat <compat-option>...

Enable compatibility options. Available compat-option:

busybox

: Use busybox compatible commands, at the expense of slight overheadwhile reading filesystem information.

ignore_receive_errors *experimental*

Tell btrfs-receive(8) to not terminate on errors by setting"--max-errors=0" option. Print warnings instead.

A known use case for this are target hosts lacking xattr support(e.g. some Synology NAS), while the send-stream contains "lsetxattr"commands. Another case is targets failing to set otime, complainingwith "ERROR: attribute 12 requested but not present".

Note that there is no guarantee that backups created with thisoption enabled can be restored at all.

If you want to set this option for local or remote hosts only, setcompat_local or compat_remote (e.g. "compat_remote busybox").

cache_dir <directory>

: If set, cache extent maps for the "btrbk extents" command.

Btrfs Specific Options

incremental_prefs <list-spec>[:<amount>]...

Specify the preferences to determine the best common (correlated)parent and clone sources for incremental backups, by choosing frompredefined candidate lists.

The list-spec defines from what candidate list the nextparent/clone-src should be appended to a result list; amount defineshow many (e.g. "sro:1 sro:1" is identical to "sro:2"), or all ifomitted. Any candidate which is already in the results is dropped.

The resulting list of subvolumes is then used as parameters for thebtrfs-send(8) command: the first for "-p <parent>", all others for"-c <clone-src>".

Available list-spec (candidate lists = filtered subsets ofcorrelated subvolumes):

sro,srn

: All from snapshot_dir matching snapshot_name, withparent_uuid relationship, sorted by btrbk timestamp (o=oldern=newer).

sao,san

: All from snapshot_dir matching snapshot_name, sortedby btrbk timestamp (o=older n=newer).

aro,arn

: All from incremental_resolve, with parent_uuidrelationship, sorted by cgen (o=older n=newer).

Defaults to "sro:1 srn:1 sao:1 san:1 aro:1 arn:1". Note that formost operations the default resolves a single parent, as there usuallyare no newer snapshots, and all "sro:1 sao:1 aro:1" resolve to thesame snapshot.

Example: "defaults,sao,san,aro,arn" takes the defaults, and adds clonesources for all (!) known candidates on the filesystem.

incremental_clones yes|no

: If enabled, btrbk adds "-c <clone-src>" to the btrfs-send(8)command for all correlated subvolumes resolved byincremental_prefs. If disabled, only "-p <parent>" isused. Defaults to "yes".

incremental_resolve mountpoint|directory

: Specifies where to search for the best common parent forincremental backups. If set to "mountpoint", use parents in thefilesystem tree below the mount point of the snapshot and targetdirectory. If set to "directory", use parents strictly belowsnapshot/target directories. Set this to "directory" if you getaccess problems (when not running btrbk as root). Defaults to"mountpoint".

btrfs_commit_delete yes|no

: If set, wait for the transaction commit at the end of eachsnapshot or backup deletion (sets --commit-each option for"btrfs subvolume delete"). Defaults to "no".

snapshot_qgroup_destroy yes|no *experimental*

target_qgroup_destroy yes|no *experimental*

archive_qgroup_destroy yes|no *experimental*

: Whenever a subvolume is deleted, also destroy correspondingdefault qgroup "0/<subvol-id>". Only useful if you have enabledbtrfs quota support. See also: <https://bugzilla.kernel.org/show_bug.cgi?id=91751>

Informative Options

warn_unknown_targets yes|no

: If set, prints a warning if btrbk encounters a target subvolume ata unknown location (i.e. not following btrbk naming scheme, oroutside the target directory). Defaults to "no".

RETENTION POLICY

Retention policies are defined individually for snapshots, backups andarchives (summarized as "backups" in the following text), using acombination of:

*_preserve_min all|latest|no|<number>{h,d,w,m,y}

: Amount of time (duration) in which all backups are preserved.

*_preserve no|<retention_policy>

: Schedule (single points in time) for which individual backups arepreserved.

Note that if "preserve_min" is set to "all" (the default), anysetting of "preserve" obviously has no effect.

The format for <retention_policy> is:

[<hourly>h] [<daily>d] [<weekly>w] [<monthly>m] [<yearly>y]

hourly

: Defines how many hours back hourly backups should bepreserved. The first backup of an hour is considered an hourlybackup.

daily

: Defines how many days back daily backups should be preserved. Thefirst backup of a day (starting at preserve_hour_of_day) isconsidered a daily backup.

weekly

: Defines how many weeks back weekly backups should bepreserved. The first daily backup created atpreserve_day_of_week (or the first backup in this week if nonewas made on the exact day) is considered as a weekly backup.

monthly

: Defines how many months back monthly backups should bepreserved. Every first weekly backup in a month is considered amonthly backup.

yearly

: Defines for how many years back yearly backups should bepreserved. Every first monthly backup in a year is considered ayearly backup.

Use an asterisk for "all" (e.g. "target_preserve 60d *m"states: "preserve daily backups for 60 days back, and all monthlybackups").

Hint: Run btrbk with the -S, --print-schedule option to get acomprehensive output of the scheduler results.

Reference Time

The local time on the host running btrbk defines the reference timefor all date/time calculations, especially for "beginning of a day",and as a consequence for the first daily, weekly, monthly or yearlybackups. The local time on remote hosts (ssh source/target) is neverused.

Unless "timestamp_format long-iso" is set, daily backups are preservedat "preserve_hour_of_day" (defaults to midnight) of the respectivetime zone (and not for "00:00 UTC", which would be "14:00" inHonolulu). This becomes relevant for setups with multiple btrbkinstances, e.g. many snapshot-only instances (spread around theworld), and a fetch-only instance on the backup server.

Caveat:

: •If "timestamp_format long-iso" is set, each btrbk instance on has adifferent interpretation of "first in day". Make sure to run btrbkwith the same time zone on every host, e.g. by setting the TZenvironment variable (see tzset(3)).

TARGET TYPES

send-receive

: Backup to a btrfs filesystem, using "btrfs send/receive". This isthe recommended (standard) target type. The <target-directory>must be an absolute path and point to a subvolume or directorywithin a btrfs file system. See btrfs-send(8), btrfs-receive(8).

raw *experimental*

Backup to a raw (filesystem independent) file from the output ofbtrfs-send(8), with optional compression and encryption.

Note that the target preserve mechanism is currently disabled forincremental raw backups (btrbk does not delete any incremental rawfiles)!

Raw backups consist of two files: the main data file containing thebtrfs send stream, and a sidecar file ".info" containing metadata:

<snapshot-name>.<timestamp>[_N].btrfs[.gz|.bz2|.xz][.gpg]<snapshot-name>.<timestamp>[_N].btrfs[.gz|.bz2|.xz][.gpg].info

For incremental backups ("incremental yes"), please note that:

: •As soon as a single incremental backup file is lost or corrupted,all later incremental backups become invalid, as there is no commonparent for the subsequent incremental images anymore. This might bea good compromise for a vacation backup plan, but for the long termmake sure that a non-incremental backup is triggered from time totime.

: •There is currently no support for rotation of incremental backups:if incremental is set, a full backup must be triggered manuallyfrom time to time in order to be able to delete old backups.

Additional options for raw targets:

raw_target_compress <compress_command>|no

: Compression algorithm to use for raw backup target. Supported<compress_command>: gzip, pigz, bzip2, pbzip2, xz, lzo, lz4,zstd.

raw_target_compress_level default|<number>

: Compression level for the specified <compress_command>.

raw_target_compress_long default|<number>

: Enable long distance matching for the specified<compress_command>.

raw_target_compress_threads default|<number>

: Number of threads to use for <compress_command>.

raw_target_split <size>|no

: Split the raw backup file into pieces of size <size>.

raw_target_block_size <number>

: Block size to use for writing the raw backup file. Defaults to"128K".

raw_target_encrypt gpg|openssl_enc|no

: If enabled, encrypt the target raw file using gpg or openssl_enc.

Additional options for "raw_target_encrypt gpg":

gpg_keyring <file>

: Keyring to use for gpg, e.g. "/etc/btrbk/gpg/pubring.kbx".

gpg_recipient <name>...

: Encrypt for user id <name> (email address).

Additional options for "raw_target_encrypt openssl_enc" (veryexperimental):

openssl_ciphername~<name>

: Defaults to "aes-256-cbc".

openssl_iv_size <size-in-bytes>|no

: Depends on selected cipher.

openssl_keyfile <file>|no

Point to a key file in hex (absolute path). Example key filecreation (256bit key):

  # dd if=/dev/urandom bs=1 count=32 \    | od -x -A n \    | tr -d "[:space:]" > /path/to/keyfile

kdf_backend <file>|no

: KDF backend to be executed,e.g. "/usr/share/btrbk/scripts/kdf_pbkdf2.py".

kdf_keysize <size-in-bytes>

: Defaults to "32".

kdf_keygen once|each

: Defaults to "once".

AVAILABILITY

Please refer to the btrbk project page <https://digint.ch/btrbk/>for further details.

AUTHOR

Axel Burri

RETENTION POLICY

Reference Time

This document was created byman2html,using the manual pages.