Backing up Shotwell photos to Box.com with WebDAV and rsync

This brief guide describes how to backup photos from a Shotwell repository to a Box.com account using rsync over a WebDAV connection. This technique was developed specifically to backup a large collection of photos to cloud storage. It can be used to automate the backup of any set of folders, not just Shotwell.

The rsync command is a powerful tool for incremental backups. The rsync command requires a file system interface. Box.com provides WebDAV access to its servers. WebDAV access would need to be mounted as a file system in order for rsync to communicate with it. The davfs2 tool provides a way to enable a file system connection to a WebDAV server.

This guide has been tested using Linux Mint and a free Box.com account. Certain commands require administrative (superuser) access. Troubleshooting is outside the scope of this guide. Please see the referenced documentation for support. This guide assumes some familiarity with the GNU-Linux command line and directory structures, and with shell scripting.

Software versions:

  • Linux Mint 16
  • Shotwell 0.15.0
  • davfs2 1.4.6
  • rsync 3.0.9

Configure WebDAV as a mountable file system

For a WebDAV server to be mounted as a file system, the davfs2 tool first must be installed. Installation is done with administrative privileges.

Next, copy the configuration and authentication files to your home directory.

Edit the following lines in the davfs2.conf file:

  • Comment out the ignore_home parameter, and
  • Enable the use_locks parameter with a value of 0 (zero).

Provide your login credentials in the secrets file. Replace loginname and password with your actual login name and password.

Add the current logged-in user (i.e., yourself) to the davfs2 group.

Reset davfs2 for the new configuration settings to be enabled.

Configure mount point

A mount point must be configured so that the WebDAV server can be recognized within the file system.

First, create and configure the mount directory.

Add the following line to the /etc/fstab file.

Finally, test the mount. This would detect any misconfigurations of davfs2, the mount point, and the fstab file.

Configure the sync script

Create a shell script that runs rsync with your desired parameters. The shell script in this guide is named mica-Pictures_sync.sh and is stored in the ~/Library/Scripts/Sync photos/ directory. (“Mica” is a nickname for my GNU-Linux laptop.)

For the local photo directory (source), use the complete path to the Pictures directory or wherever you have stored your photos. The default location would be /home/<username>/Pictures, where <username> is your actual username. For the destination directory, use /media/box/<target folder>, where <target folder> is a folder you’ve created on Box.com to store your photos. Other parameters may also be used — please see the rsync documentation.

To test the synchronization without transmitting files, use the –dry-run parameter. Disable the –dry-run parameter to enable file transmission.

A note about the rsync parameters

The following parameters are used in the rsync script.

Parameter Description
-a, ‐‐archive archive mode; equals -rlptgoD (no -H,-A,-X)
-r, ‐‐recursive recurse into directories
-l, ‐‐links copy symlinks as symlinks
-p, ‐‐perms preserve permissions
-t, ‐‐times preserve modification times
-g, ‐‐group preserve group
-o, ‐‐owner preserve owner (super-user only)
-D same as ‐‐devices ‐‐specials
‐‐devices preserve device files (super-user only)
‐‐specials preserve special files
-H, ‐‐hard-links preserve hard links
-A, ‐‐acls preserve ACLs (implies -p)
-X, ‐‐xattrs preserve extended attribute
-h, ‐‐human-readable output numbers in a human-readable format
-i, ‐‐itemize-changes output a change-summary for all updates
-v, ‐‐verbose increase verbosity
-z, ‐‐compress compress file data during the transfer
‐‐progress show progress during transfer
‐‐size-only skip files that match in size
‐‐stats give some file-transfer stats
‐‐exclude=PATTERN exclude files matching PATTERN
The pattern \”.*\” will exclude filenames starting with a dot.
-n, ‐‐dry-run perform a trial run with no changes made

Using the sync script

First, the external server must be mounted as a local drive via WebDAV. A file manager shortcut can be configured for this purpose. Otherwise, use the following command.

Once the WebDAV server is mounted, run the script with administrative privileges (sudo). For example:

A note about the synchronization process and WebDAV

This script will scan the WebDAV repository for each of the files in your local directory. If changes have been detected the changes will be queued for synchronzation. Although the script will appear to run and complete, returning you to the command prompt, uploads and synchronizations may not have been completed. According to Box.com, WebDAV uploads “are accomplished in two stages.” It will appear that rsync has concluded its work, which it has, but WebDAV takes some time to catch up. Therefore, you may need to wait and then check the server to see if all files have been synchronized, even if rsync is done.

In practice, I don’t need to interrupt my network connection so I allow the script to complete in the background.

A note about Shotwell metadata

The script described here creates only a backup of Shotwell images and their folder structure. Shotwell metadata, such as tags, is stored elsewhere and is not backed up with this script. In Shotwell 0.15.0, the version used here, collection metadata is stored at ~/.shotwell/data/photo.db.

To Do

  • The rsync –inplace parameter may be helpful in alleviating the WebDAV bottleneck described above.
  • The rsync –no-whole-file parameter informs rsync of the remote filesystem.
  • Provide a way to backup only certain folders in the Shotwell hierarchy. In particular, only newer folders may be of interest for backup.
  • Backup the Shotwell metadata.

References

Dec 4, 2016 – Added section “A note about Shotwell structure metadata.”
– Added “To Do” bullets for selecting subfolders and metadata.
– Added Shotwell version number.
See this page at https://kinasevych.ca/index.php