#language en #refresh 999999 = Backup = [[TableOfContents(4)]] Below, the term ‘back up’ refers to storage of data in such a way, that the state of that data at any given day in the past year can be recovered.[[BR]] Conversely, ‘mirroring’ or ‘synchronising’ keeps a copy of the current data. Changes to the data in the past are lost. == Backup for data less than ≈200 GB == The back up is done in two steps, i. the data is syncronised, using rsync, to your home directory on b.fluid.tuwien.ac.at, i. on b.fluid, all data in the home directories is backed up once a day. In order to use this kind of backup, a correct rsync command must be created. Then, this command is written to your crontab file, such that the rsync-command is invoked once an hour. Follow the instructions below to do this. Your home directory on b.fluid must be created by an adminstrator, Thomas Loimer or Werner Jandl. You can log in to b.fluid using the credentials of your TU e-mail account. Please note, that directories which have names ending in ".nobackup", "cache", "Cache", or contain a [http://www.bford.info/cachedir/spec.html cachedir-tag], are excluded from the backup. === Setup instructions === ==== Windows ==== In order to set up file synchronisation to `b.fluid` on your Windows client, your system needs an rsync client and a customised Windows synchronisation script. For details on how to proceed, please contact Iris Fula. ==== Common part for GNU/Linux, macOS, other Unix-like operating systems ==== On your desktop computer, please … * check that the program `rsync` is installed (should be installed by default on most systems) * save the script [attachment:sync-to-host sync-to-host] to a directory of your choice, for example `~/bin/`; further below, the path to this script must be set accordingly * make the script [attachment:sync-to-host sync-to-host] executable (e.g., `chmod +x ~/bin/sync-to-host`) * copy the following lines to a terminal, adjusting `TU_USER` according to your TU username: {{{ bash # the commands below must be executed in the bash shell export TU_USER=your_tu_username }}} * copy/paste the following lines to a terminal, adjusting `~/bin/sync-to-host` to the actual path to the script, {{{ export TARGET_HOST=b.fluid.tuwien.ac.at export TARGET_DIR=$(hostname -s) # run the backup script once (enter your TU password when asked to do so) ~/bin/sync-to-host ${TU_USER} ${TARGET_HOST} ${TARGET_DIR} }}} * Please, edit the file `~/.config/sync-to-host/exclude.txt` and set the patterns of files and directories which should be ignored by the backup. See `man rsync` under `INCLUDE/EXCLUE PATTERN RULES` for the syntax of these patterns. The top of the file [attachment:sync-to-host sync-to-host] contains some comments on how the script works. ==== Automated backups on GNU/Linux ==== In order to set up a cron job for automated client backups on GNU/Linux systems, please copy and paste the following lines into the same terminal as above. Set `SYNC_CMD` according to the path where you saved the [attachment:sync-to-host sync-to-host] script: {{{ export SYNC_CMD="${HOME}/bin/sync-to-host ${TU_USER} ${TARGET_HOST} ${TARGET_DIR}" export SYNC_TIME="$((RANDOM % 60)) * * * *" export CRON_ENTRY="${SYNC_TIME} ${SYNC_CMD}" crontab -l 2>/dev/null | { cat; echo "${CRON_ENTRY}"; } | crontab - }}} ==== Automated backups on macOS ==== In order to set up a launchd job for automated client backups on macOS systems, please copy and paste the following lines into a terminal: {{{ #!/bin/bash export BASENAME=at.ac.tuwien.fluid.b_backup-${USER} export PLIST_PATH=${HOME}/Library/LaunchAgents/${BASENAME}.plist cat > ${PLIST_PATH} < Label ${BASENAME} ProgramArguments ${HOME}/bin/sync-to-host ${TU_USER} ${TARGET_HOST} ${TARGET_DIR} RunAtLoad StartInterval3600 EOF launchctl load ${PLIST_PATH} }}} == Backup for data more than ≈200 GB == The [https://www.borgbackup.org borg] backup program is used to back up the data from the client machine. i. Install the borg backup program. Under ubuntu or debian, on the command line type `sudo apt install borgbackup`. i. Download the script [attachment:borg-user-backup borg-user-backup]. i. Make the script executable, run it and follow the instructions. Comments in the script describe what is done. i. Store the script at a convenient location and create a cron-job to invoke it once a day. For a default setup, after step (ii) above, copy the script [attachment:borg-user-backup borg-user-backup] to ~/bin/, copy the first two lines below one by one to a terminal, editing the content of SOURCE_DIRS and TU_USER according to your needs. Then, copy all the remaining lines below to your terminal and press enter. The setup script will ask for your password, see the comments in [attachment:borg-user-backup borg-user-backup] {{{ TU_USER=myusername SOURCE_DIRS="/home/myname/dir1 /home/myname/dir2 /usr/local" ~/bin/borg-user-backup $TU_USER b.fluid.tuwien.ac.at "$SOURCE_DIRS" crontab -l 2>/dev/null | { cat echo "$((RANDOM % 60)) $((RANDOM % 7 + 10)) * * * $HOME/bin/borg-user-backup $TU_USER b.fluid.tuwien.ac.at $SOURCE_DIRS" } | crontab - }}} = Restoring data = == Instructions for admins == In order to restore data from a backup, backups can be mounted to a user's home directory. Users can then access their data using an SFTP client (e.g. Nautilus on GNU/Linux, Cyberduck on macOS and WinSCP on Windows). For example, in order to expose backup data from `b.fluid` to the user `oswat`, the following commands might be used: {{{ export BACKUP_REPO=/mnt/backup/b/home/oswat.borg export RESTORE_DIR=/home/oswat/restore-$(date -Idate) mkdir ${RESTORE_DIR} borg mount -o default_permissions,allow_other ${BACKUP_REPO} ${RESTORE_DIR} ls ${RESTORE_DIR} }}} Note: As soon as users have restored their data, backups should be unmounted again: {{{ umount ${RESTORE_DIR} }}}