Automated Raspberry Pi Backup – complete image
I love my Raspberry Pi projects and I run a lot of specialist “mini” servers at home doing everything from torrent sharing of Linux distros to media streaming and media playing. But all Raspberry Pi’s and other single board computers that rely on SD-cards sooner or later comes to a point where they trash the card and doesn’t boot again.
Every time I run into that situation without remembering exactly what was running and how on the particular Raspberry Pi. I want backups, not just the backup I usually do right after installation but a last night backup or similar. So I put up an NFS share on my NAS to store the backups, it will work just as well with a USB stick connected directly to the Raspberry Pi. Here is a step by step guide how I automated the backups on all my Raspberry Pi’s. This script will create a complete image of the SD-card while the Raspberry Pi is running. You can just write that image to a new SD-card and pop it into the Pi and it will be like nothing happened!
- Create the folder /mnt/backup
- Edit fstab with
[bash]sudo /etc/fstab[/bash]
Add the share on the NAS for mounting
[bash]192.168.6.5:/nfs/Backups/raspberrypi_boot /mnt/backup nfs rsize=8192,wsize=8192,timo=14,intr 0 0[/bash]
- Test the mount
[bash]sudo mount -a[/bash]
- Download my backup script from Github
[bash]wget https://github.com/kallsbo/BASH-RaspberryPI-System-Backup/raw/master/system_backup.sh[/bash]
I placed the script on my NAS share so all the Raspberry Pi’s could use the same script file.
- Make the script file executable
[bash]chmod +x system_backup.sh[/bash]
- Add it to crontab. Since the script needs to be executed as root we need to edit the root crontab.
[bash]sudo crontab -e[/bash]
Just add the line (with correction for the path to the script)
[bash]0 3 * * * /mnt/backup/system_backup.sh[/bash]
The script is a simple bash script that you can look at and download from Github – BASH-RaspberryPI-System-Backup. The script has two variables, backup_path and rentention_days. The first is the path where to store the backup and the second is the number of days to save it. They can have defaults set in the script but you can override them by supplying command line arguments like:
[bash]sudo ./system_baskup.sh /mnt/otherpath 7[/bash]
That will give you the backup path of /mnt/otherpath and a retention of 7 days. The next few lines (until line 38) just checks if it’s running as root and checks for command line arguments. Then we touch the file /boot/forcefsck this creates a blank file at that path and forces the Raspberry Pi to check the filesystem on the next boot. This is so the backup image, if used, will be checked for filesystem inconsistencies on first boot.
Then comes the magic sauce, creating a complete copy of the SD-card:
[bash]dd if=/dev/mmcblk0 of=$backup_path/$HOSTNAME.$(date +%Y%m%d).img bs=1M[/bash]
Then the script removes the /boot/forcefsck file so the force file is only in the backup image and not on the currently running SD-card. Then it looks for backups older then the number of retention days and deletes them. Since the script includes the hostname in the filename and only looks for it’s own hostname you can have several Raspberry Pi’s backing up to the same location without being worried about them deleting each others backups.
Hi, thanks for this. i ran it from the command line and i get:
pi@pi:~ $ dd: failed to open ‘/myNAS/backup/pi.20180406.img’: No such file or directory
find: `/myNAS/backup/pi.*.img’: No such file or directory
it’s the first time it’s run so nothing will be there. it then appears to hang. any ideas or is it actually compiling the backup in the background (ileft it for hours and nothing happened). Cheers, Don
LikeLike
Make sure that the folder exists and that you can write to it. It has to be mounted. Just try writing a text file from nano in the folder to check…
LikeLike
thanks for the speedy reply. i can create files on the NAS from the Pi. It is mounted OK. i don’t know what the problem might be. cheers, Don
LikeLike
Are you running the script with the right credentials? Check message log for errors
LikeLike
Thanks for a solution, I will use it. A lot of typos in the text
timo=14 (timeo=14)
sudo ./system_baskup.sh /mnt/otherpath 7 (backup.sh)
LikeLike
Thanks for this! I’m testing it out now and it seems it is going to backup the whole 64GB sd card.. so far its backed up about 12GB and my total used space on the card is about 4.9GB. Is this correct that it will backup the empty space as well? If so, is there a way to reduce that?
LikeLike
Yes the whole SD card will be imaged. There is no way for the dd command to determine what is empty space and not. I did experiment with some empty space removal of the image it self after backup but ran into inconsistent behavior so I dropped it.
LikeLike
Hi,
Is this process of backup doable also for running applications like WordPress which using MariadDB for database. I mean is it possible to create proper image which could be also properly restored while the Raspberry Pi is running the whole time?
LikeLike
Haven’t tried with that specific combo of software but the backup should be consistent. I only tried it with a MySQL used by my Kodi boxes and it worked just fine. I recommend you try it out.
LikeLike
Thank you for this, Kristofer.
LikeLike
Thanks again for this work, Kristofer. The script worked fine to create the image on an NFS mount (NFS share from a rasPi NAS server with a hard drive). Then I split the file and cp them to my mac laptop from the NFS server. Then I cat files.* > original.img and burn the SD. Boots fine, great! I think the full dd of the SD in the rasPi probably works it more than my regular production does (rasPi email server). So, I have the script in /etc/cron.weekly and then I do file backups of the email directories more often.
(/usr/bin/split –bytes=1G mail2.YYYYMMDD.img mail2.YYYYMMDD.split)&
then, from the NAS server I copy all the files to my laptop where I do
cat mail2.YYYYMMDD.split* > mail2.YYYYMMDD.img and burn the new SD.
LikeLike
Pingback: Pi-Backup – cozmopics.de
Hi Kristofer,
Great solution. Couple of points:
1. You still have a typo in your NFS mount options (timeo, not timo)
2. You can make the backup much smaller and faster if you pipe through gzip as you go. (Gzip buffers writes, whereas dd does not)
# Perform backup and pipe through gzip to improve speed/size
dd if=/dev/mmcblk0 bs=1M | /bin/gzip > $backup_path/$HOSTNAME.$(date +%Y%m%d).img.gz
Rock on!
LikeLike
This is in my fstab:
192.168.1.5:/nfs/zalohy/hasszaloha /mnt/backup nfs rsize=8192,wsize=8192,timo=14,intr 0 0
LikeLike
Hello,
I have recently configured this backup script end it’s really awesome!
But i have a problem: I have set a retention days to 3, but the script don’t delete the oldest backup.
Do you have any suggestion for resolve this problem?
LikeLike
Does it delete any files? Do a ls -al and check the dates of the files to make sure it’s correct. If any other process touches the files the dates might be updated and cause the delete not to work.
LikeLike
No other process touches the files, in this sd card (with raspian) no other process are installed yet.
In the ls -al command i’ve found this line:
-rwxr-xr-x 1 pi pi 1182 mar 9 11:53 system_backup.sh
The file names i think are correct: Pi3b.20200315.img
I have installed this plugin on monday or thuesday this week, end the older file, at the moment, are namedPi3b.20200312.img. Before that day i have eliminated manually to create space on the usb stick i have mounted to save my bck files.
LikeLike
As far as I can see there is no .img files in the ls command? Run the ls command against the folder where you store the backups. The script will delete any file older then the specified number of days in the target directory. You have to run the script against the same folder each time for that to work. Also make sure that the time/date is correct on the pi and the process have access to the backup files.
LikeLike
Hello,
I’ve tried some thing and I noticed that the script works well deleting oldest files on the target directory, only if i launch it manually, when it run at the scheduled time with crontab don’t delete the oldest backup.
For your question the time/date is correct,
This is the ls -al command in the directory i store bck
“pi@Pi3b:/media/pi/bck/mnt/backup $ ls -al
totale 45367300
drwxrwxrwx 1 pi pi 4096 mar 23 03:00 .
drwxrwxrwx 1 pi pi 0 mar 9 11:18 ..
-rwxrwxrwx 1 pi pi 15485370368 mar 21 11:43 Pi3b.20200321.img
-rwxrwxrwx 1 pi pi 15485370368 mar 22 03:16 Pi3b.20200322.img
-rwxrwxrwx 1 pi pi 15485370368 mar 23 03:16 Pi3b.20200323.img”
and this is the ls -al command in the script backup
pi@Pi3b:/media/pi/bck $ ls -al
totale 9
drwxrwxrwx 1 pi pi 4096 mar 9 11:25 .
drwxr-x—+ 3 root root 4096 mar 19 08:59 ..
drwxrwxrwx 1 pi pi 0 mar 9 11:18 mnt
lrwxrwxrwx 1 pi pi 25 mar 9 11:22 system_backup.sh -> /home/pi/system_backup.sh
drwxrwxrwx 1 pi pi 0 mar 9 11:11 ‘System Volume Information’
LikeLike
It sounds like that the cron is executed with the wrong permissions to be able to delete the files then. You can also test to pipe the output of the script in the crontab to see any errors.
LikeLike
Hey, is it possible to implement a progress bar in script? just when I use it to manually backup some Pi
LikeLike
Yes, append status=progress to the dd command line in the script.
LikeLike
Hello I just tried to mount my nas via nfs but i get this error.
what i entered in /etc/fstab
192.168.1.200:/Multimedia/Backups/ /mnt/backup nfs rsize=8192,wsize=8192,timo=14,intr 0 0
then: sudo mount -a
Error:
mount.nfs: an incorrect mount option was specified
LikeLike
Google tour NAS and check what NFS settings it support. It differs between manufacturers and the underlying Linux flavor they used for the firmware.
LikeLike
It seems to be a good/clear tutorial to make a copy of the SD-Card. That’s what I need. But I have no NAS. Could you change the script and explain how to do it with web space and webDAV?
Thanks
LikeLike
That should work just fine. Instead of mounting the nfs share to the backup folder you have to mount the WebDAV service instead. Haven’t used WebDAV my self but something like this should work https://wouterjanson.nl/2017/04/05/webdav-mount/
LikeLike
I can not get the cron to run. I can run the system_backup.sh on a command line. I have other crons in the same folder that run with no problem. I have listed my cron below.
0 2 * * * /home/pi/system_backup.sh
LikeLike
Look at the syslog to figure out what goes wrong. As far as I can see the cron is correct, given that the file exists and are executable. This should run 02:00 each day.
LikeLike
Being a noob, I stumbled at 1.
Create a folder… Where? On the pi? On the Nas? How?
LikeLike
Step one is the mount folder on the pi it self.
LikeLike
Hello Kristofer,
I have this running on a Raspberry Pi but I get an error on execution. The error is
“system_backup.sh: 20: system_backup.sh: [[: not found.”
The backup does continue but it names the backup file .YYYYMMDD.img. It completely ignores the $HOSTNAME. I can echo the hostname in the terminal and it’s correct but it never gets appended to the output file. Any ideas? Thanks!
LikeLike
Line 20 is the root check. Do you have whoami on the pi? Are you running as root? It seems to be something wrong with the way it interprets that if statement.
LikeLike
Yes, I’m running it as root. Also, when I run sudo whoami it does return root in the terminal.
LikeLike
Try removing the other set of [] but that might mess with the not annotation.
LikeLike
That returned:
system_backup.sh: 20: [: root: unexpected operator
LikeLike
Then I have no good idea to be honest. I haven’t run into or heard anyone who did. Will try to recreate later and see if I can get the same error.
LikeLike
OK, Thanks. I was hoping it was going to be something easy but if nothing else, I’ll comment it out.
Do you have any idea why it’s not appending the $HOSTNAME to the beginning of the file name? It doesn’t give any errors in regards to creating or naming the file.
LikeLike
OK, chalk this one up to operator error. I was running it with sh instead of bash. Apparently it doesn’t work well with sh. hahaha Thanks for the help!
LikeLike