Automated Raspberry Pi Backup – complete image
I love my Raspberry Pi projects and I run a lot of specialist “mini” servers at home doing everything from torrent sharing of Linux distros to media streaming and media playing. But all Raspberry Pi’s and other single board computers that rely on SD-cards sooner or later comes to a point where they trash the card and doesn’t boot again.
Every time I run into that situation without remembering exactly what was running and how on the particular Raspberry Pi. I want backups, not just the backup I usually do right after installation but a last night backup or similar. So I put up an NFS share on my NAS to store the backups, it will work just as well with a USB stick connected directly to the Raspberry Pi. Here is a step by step guide how I automated the backups on all my Raspberry Pi’s. This script will create a complete image of the SD-card while the Raspberry Pi is running. You can just write that image to a new SD-card and pop it into the Pi and it will be like nothing happened!
- Create the folder /mnt/backup
- Edit fstab with
[bash]sudo /etc/fstab[/bash]
Add the share on the NAS for mounting
[bash]192.168.6.5:/nfs/Backups/raspberrypi_boot /mnt/backup nfs rsize=8192,wsize=8192,timo=14,intr 0 0[/bash]
- Test the mount
[bash]sudo mount -a[/bash]
- Download my backup script from Github
[bash]wget https://github.com/kallsbo/BASH-RaspberryPI-System-Backup/raw/master/system_backup.sh[/bash]
I placed the script on my NAS share so all the Raspberry Pi’s could use the same script file.
- Make the script file executable
[bash]chmod +x system_backup.sh[/bash]
- Add it to crontab. Since the script needs to be executed as root we need to edit the root crontab.
[bash]sudo crontab -e[/bash]
Just add the line (with correction for the path to the script)
[bash]0 3 * * * /mnt/backup/system_backup.sh[/bash]
The script is a simple bash script that you can look at and download from Github – BASH-RaspberryPI-System-Backup. The script has two variables, backup_path and rentention_days. The first is the path where to store the backup and the second is the number of days to save it. They can have defaults set in the script but you can override them by supplying command line arguments like:
[bash]sudo ./system_baskup.sh /mnt/otherpath 7[/bash]
That will give you the backup path of /mnt/otherpath and a retention of 7 days. The next few lines (until line 38) just checks if it’s running as root and checks for command line arguments. Then we touch the file /boot/forcefsck this creates a blank file at that path and forces the Raspberry Pi to check the filesystem on the next boot. This is so the backup image, if used, will be checked for filesystem inconsistencies on first boot.
Then comes the magic sauce, creating a complete copy of the SD-card:
[bash]dd if=/dev/mmcblk0 of=$backup_path/$HOSTNAME.$(date +%Y%m%d).img bs=1M[/bash]
Then the script removes the /boot/forcefsck file so the force file is only in the backup image and not on the currently running SD-card. Then it looks for backups older then the number of retention days and deletes them. Since the script includes the hostname in the filename and only looks for it’s own hostname you can have several Raspberry Pi’s backing up to the same location without being worried about them deleting each others backups.
Hackviking aka Kristofer Källsbo 
Hi, thanks for this. i ran it from the command line and i get:
pi@pi:~ $ dd: failed to open ‘/myNAS/backup/pi.20180406.img’: No such file or directory
find: `/myNAS/backup/pi.*.img’: No such file or directory
it’s the first time it’s run so nothing will be there. it then appears to hang. any ideas or is it actually compiling the backup in the background (ileft it for hours and nothing happened). Cheers, Don
LikeLike
Make sure that the folder exists and that you can write to it. It has to be mounted. Just try writing a text file from nano in the folder to check…
LikeLike
thanks for the speedy reply. i can create files on the NAS from the Pi. It is mounted OK. i don’t know what the problem might be. cheers, Don
LikeLike
Are you running the script with the right credentials? Check message log for errors
LikeLike
Thanks for a solution, I will use it. A lot of typos in the text
timo=14 (timeo=14)
sudo ./system_baskup.sh /mnt/otherpath 7 (backup.sh)
LikeLike
Thanks for this! I’m testing it out now and it seems it is going to backup the whole 64GB sd card.. so far its backed up about 12GB and my total used space on the card is about 4.9GB. Is this correct that it will backup the empty space as well? If so, is there a way to reduce that?
LikeLike
Yes the whole SD card will be imaged. There is no way for the dd command to determine what is empty space and not. I did experiment with some empty space removal of the image it self after backup but ran into inconsistent behavior so I dropped it.
LikeLike
Hi,
Is this process of backup doable also for running applications like WordPress which using MariadDB for database. I mean is it possible to create proper image which could be also properly restored while the Raspberry Pi is running the whole time?
LikeLike
Haven’t tried with that specific combo of software but the backup should be consistent. I only tried it with a MySQL used by my Kodi boxes and it worked just fine. I recommend you try it out.
LikeLike
Thank you for this, Kristofer.
LikeLike
Thanks again for this work, Kristofer. The script worked fine to create the image on an NFS mount (NFS share from a rasPi NAS server with a hard drive). Then I split the file and cp them to my mac laptop from the NFS server. Then I cat files.* > original.img and burn the SD. Boots fine, great! I think the full dd of the SD in the rasPi probably works it more than my regular production does (rasPi email server). So, I have the script in /etc/cron.weekly and then I do file backups of the email directories more often.
(/usr/bin/split –bytes=1G mail2.YYYYMMDD.img mail2.YYYYMMDD.split)&
then, from the NAS server I copy all the files to my laptop where I do
cat mail2.YYYYMMDD.split* > mail2.YYYYMMDD.img and burn the new SD.
LikeLike
Pingback: Pi-Backup – cozmopics.de
Hi Kristofer,
Great solution. Couple of points:
1. You still have a typo in your NFS mount options (timeo, not timo)
2. You can make the backup much smaller and faster if you pipe through gzip as you go. (Gzip buffers writes, whereas dd does not)
# Perform backup and pipe through gzip to improve speed/size
dd if=/dev/mmcblk0 bs=1M | /bin/gzip > $backup_path/$HOSTNAME.$(date +%Y%m%d).img.gz
Rock on!
LikeLiked by 1 person
This is in my fstab:
192.168.1.5:/nfs/zalohy/hasszaloha /mnt/backup nfs rsize=8192,wsize=8192,timo=14,intr 0 0
LikeLike
Hello,
I have recently configured this backup script end it’s really awesome!
But i have a problem: I have set a retention days to 3, but the script don’t delete the oldest backup.
Do you have any suggestion for resolve this problem?
LikeLike
Does it delete any files? Do a ls -al and check the dates of the files to make sure it’s correct. If any other process touches the files the dates might be updated and cause the delete not to work.
LikeLike
No other process touches the files, in this sd card (with raspian) no other process are installed yet.
In the ls -al command i’ve found this line:
-rwxr-xr-x 1 pi pi 1182 mar 9 11:53 system_backup.sh
The file names i think are correct: Pi3b.20200315.img
I have installed this plugin on monday or thuesday this week, end the older file, at the moment, are namedPi3b.20200312.img. Before that day i have eliminated manually to create space on the usb stick i have mounted to save my bck files.
LikeLike
As far as I can see there is no .img files in the ls command? Run the ls command against the folder where you store the backups. The script will delete any file older then the specified number of days in the target directory. You have to run the script against the same folder each time for that to work. Also make sure that the time/date is correct on the pi and the process have access to the backup files.
LikeLike
Hello,
I’ve tried some thing and I noticed that the script works well deleting oldest files on the target directory, only if i launch it manually, when it run at the scheduled time with crontab don’t delete the oldest backup.
For your question the time/date is correct,
This is the ls -al command in the directory i store bck
“pi@Pi3b:/media/pi/bck/mnt/backup $ ls -al
totale 45367300
drwxrwxrwx 1 pi pi 4096 mar 23 03:00 .
drwxrwxrwx 1 pi pi 0 mar 9 11:18 ..
-rwxrwxrwx 1 pi pi 15485370368 mar 21 11:43 Pi3b.20200321.img
-rwxrwxrwx 1 pi pi 15485370368 mar 22 03:16 Pi3b.20200322.img
-rwxrwxrwx 1 pi pi 15485370368 mar 23 03:16 Pi3b.20200323.img”
and this is the ls -al command in the script backup
pi@Pi3b:/media/pi/bck $ ls -al
totale 9
drwxrwxrwx 1 pi pi 4096 mar 9 11:25 .
drwxr-x—+ 3 root root 4096 mar 19 08:59 ..
drwxrwxrwx 1 pi pi 0 mar 9 11:18 mnt
lrwxrwxrwx 1 pi pi 25 mar 9 11:22 system_backup.sh -> /home/pi/system_backup.sh
drwxrwxrwx 1 pi pi 0 mar 9 11:11 ‘System Volume Information’
LikeLike
It sounds like that the cron is executed with the wrong permissions to be able to delete the files then. You can also test to pipe the output of the script in the crontab to see any errors.
LikeLike
Hey, is it possible to implement a progress bar in script? just when I use it to manually backup some Pi
LikeLike
Yes, append status=progress to the dd command line in the script.
LikeLike
Hello I just tried to mount my nas via nfs but i get this error.
what i entered in /etc/fstab
192.168.1.200:/Multimedia/Backups/ /mnt/backup nfs rsize=8192,wsize=8192,timo=14,intr 0 0
then: sudo mount -a
Error:
mount.nfs: an incorrect mount option was specified
LikeLike
Google tour NAS and check what NFS settings it support. It differs between manufacturers and the underlying Linux flavor they used for the firmware.
LikeLike
It seems to be a good/clear tutorial to make a copy of the SD-Card. That’s what I need. But I have no NAS. Could you change the script and explain how to do it with web space and webDAV?
Thanks
LikeLike
That should work just fine. Instead of mounting the nfs share to the backup folder you have to mount the WebDAV service instead. Haven’t used WebDAV my self but something like this should work https://wouterjanson.nl/2017/04/05/webdav-mount/
LikeLike
I can not get the cron to run. I can run the system_backup.sh on a command line. I have other crons in the same folder that run with no problem. I have listed my cron below.
0 2 * * * /home/pi/system_backup.sh
LikeLike
Look at the syslog to figure out what goes wrong. As far as I can see the cron is correct, given that the file exists and are executable. This should run 02:00 each day.
LikeLike
Being a noob, I stumbled at 1.
Create a folder… Where? On the pi? On the Nas? How?
LikeLike
Step one is the mount folder on the pi it self.
LikeLike
Hello Kristofer,
I have this running on a Raspberry Pi but I get an error on execution. The error is
“system_backup.sh: 20: system_backup.sh: [[: not found.”
The backup does continue but it names the backup file .YYYYMMDD.img. It completely ignores the $HOSTNAME. I can echo the hostname in the terminal and it’s correct but it never gets appended to the output file. Any ideas? Thanks!
LikeLike
Line 20 is the root check. Do you have whoami on the pi? Are you running as root? It seems to be something wrong with the way it interprets that if statement.
LikeLike
Yes, I’m running it as root. Also, when I run sudo whoami it does return root in the terminal.
LikeLike
Try removing the other set of [] but that might mess with the not annotation.
LikeLike
That returned:
system_backup.sh: 20: [: root: unexpected operator
LikeLike
Then I have no good idea to be honest. I haven’t run into or heard anyone who did. Will try to recreate later and see if I can get the same error.
LikeLike
OK, Thanks. I was hoping it was going to be something easy but if nothing else, I’ll comment it out.
Do you have any idea why it’s not appending the $HOSTNAME to the beginning of the file name? It doesn’t give any errors in regards to creating or naming the file.
LikeLike
OK, chalk this one up to operator error. I was running it with sh instead of bash. Apparently it doesn’t work well with sh. hahaha Thanks for the help!
LikeLike
Hi,
Thank you for the script. Could you let me know the steps to backup to a USB drive connected to the pi instead of a NAS?
LikeLike
Just mount the usb drive to /mnt/backup. There are a lot of examples and tutorials for mounting usb online. Just Google “mount usb raspberry pi” and you will find all the information needed.
LikeLike
Thanks for the backup script! Can you provide a script that you can use to restore the image? I’m not sure how these backups are useful without that. If the assumption is “use wind32diskimager” then please explain, although my preference would be keeping it in Linux. Thanks!
LikeLike
The created image is a complete copy of the SD-card so there is no way to restore it to a running Pi. The image can be used the same way as the base OS images and applied to an SD-card. For example you can use the Raspberry Pi OS Images available for Ubuntu to reapply the image to the SD-card.
LikeLike
We need a Linux script to restore the image file to an sd-card. Its not going to be run on “the running Pi”. You are going to take the sd card out of the Pi, put it into another computer, restore the image to the sd-card and then pop it back into the Pi. I can always burn another release image to an sd-card, but I want to restore the image we just made with your first script and using Linux/Bash.
LikeLike
If you know how to “burn another release image” to the card you know how to restore this. Why do you need a script when you still need to do the manual task of transferring the sd-card? Just do how you would normally do but “burn” the backup image to the new sd-card instead.
LikeLike
I use Win32DiskImager to burn Linux Releases in Windows, but I’ve said twice now I need a linux script to do that for these backups. If you don’t know how to create a linux script to restore the image you created in your previous scripts you could just say so and save yourself some time, instead of telling me how you think I “should” be doing this. If I could take back my PayPal donation at this point I would.
LikeLike
Are you for real? So just because I share this free of charge and you tip me $2 I’m on the hook to do this for you? If you don’t know how to image an sd-card in Linux from the cli, try to google it. But please David Cherry don’t bring your piss poor attitude to my comments. If you don’t like what I give away for free develop your own solution!
LikeLike
Remove your previous comment or you will be hearing from my lawyer.
LikeLike
Step down from your high horse David, the rest of the world doesn’t care about your lawyers.
LikeLike
Doxing is illegal, and also against WordPress terms of service. You can act tough now, while you still have a website.
LikeLike
What you don’t know could fill a warehouse! Doxing is only illegal if the information is obtained in an illegal way, you sent me the information. In addition US law don’t apply to me, I’m not in the US. Take this as a learning experience that you can’t run around acting like a spoiled brat to people. Do what you feel you need to do.
LikeLike
Gt461 is an absolute entitled Karen. I apologize on behalf of the US. Thanks for the script and help you’ve provided for FREE Hackingviking!
LikeLike
No need to apologize! Appreciate the support, thank you!
LikeLike
Doxing is the act of publishing personal information. How it is obtained is secondary, and I am not even addressing that. WordPress is an international company and will respond to US lawyers, in fact I won’t even have to use a lawyer with them to have this page pulled down for violating terms of service. I did not provide you with any information to be used on this forum. Let what happens to you be a learning experience as well.
LikeLike
Good luck with that!
LikeLike
Already submitted the complaint. They have a nice page setup for it. You should be hearing from them soon!
LikeLike
Did they have a tutorial for writing basic bash scripts for applying images to SD-cards as well? Looking forward to hearing from them. Have a great day David!
LikeLike
If it wasn’t clear previously, I am formally requesting my refund.
LikeLike
Refund for what? You donated $2 because you liked the code/article. You do realize that this code is published under the MIT license? That means no liability and no warranty. All the time you spent bitching here would have been more than enough to actually google your issue and use someone else’s free work as well to solve your problem.
https://letmegooglethat.com/?q=How+to+Write+Image+Files+to+SD+Card+with+dd+from+Command
I have already disputed your BS paypal claim for “goods not delivered”. Paypal decided to refund your money with no cost to me, so you basically scammed Paypal for $2. The paypal button clearly states that it’s a donation and no expectation of goods is more than obvious!
As they state “Due to the unique circumstances of this case”, which is the nice version of “because David Cherry is a dick”, PayPal will refund your buyer at no cost to you. Could you just please drop this and move on?
LikeLike
That is an inaccurate portrayal of what happened. They actually offered me more than the $2, but I said I didn’t want it because of the pending legal issue between you and I for you doxing me and that I needed the dispute to be by the book. I actually spoke with them.
I see that you did escalate it but that their response to me was that they found in my favor, not that “in this unique case”. Any other reasons you want to portray here are lies. I have contacted a lawyer and found that you are hosted out of California. Much to our happiness because they have about the most stringent privacy laws on the planet. It won’t be much longer before we have lots more information.
So far I have told you what you can do to resolve this and I will go away. Remove my name from your blog. Leave up everything else you’ve said if you want.
Ebay response:
“Case status Closed
The case was closed in your favor.”
LikeLike
You mean this case? https://pasteboard.co/UefC3JFFgvRu.png
If there is any doubt about the authenticity I can publish the whole thing if you like? I think you misunderstood, I don’t care if this goes away. I just want you to man-child somewhere else, through your lawyer is just fine. Because what you don’t realize is that I don’t care about your US lawyer and you might get WordPress to take down the site but it will be up straight away hosted outside of the US. I actually think I have a bash-script for the setup… 🙂
You can’t do anything to me, it was years since I moved from California and I’m not a US citizen. In my country, which has actual freedom of speech, I’m fine. All I did is called you by your name and that is not a crime. Pretty sure that isn’t a crime even in the US since I haven’t spilt any other personal details.
So spend your money on your lawyer all you want it doesn’t matter to me. All my contact details are on the blog.
LikeLike
That Paste means zero, because it looks exactly like what they sent me. My lawyer will find out the truth on it so don’t bother.
I know you want to make this about me, but no sane person goes as far as you have gone to publicly call someone out because they said they wanted their money back, while you saying you are above the law but I am the man-child? Once we have what we need on you, you WILL be hearing from us. This is my final direct communication to you.
LikeLike
I called you out for being a dick. You wanting your money back came later. Once again you misunderstand or don’t understand I’m not above the law but US law don’t apply outside of the US. Unless you guys invade Sweden it means absolutely nothing to us. I’m pretty sure that’s not gone happen since we don’t have any oil.
If we look at this from US law perspective the whole doxxing thing is ridiculous! The name Davis Cherry isn’t specific enough to identify anyone person specifically. There is more than a thousand profiles on LinkedIn and Facebook with that name and I’m pretty sure your not David Cherry the Scottis rugby player.
You just got pist because I called you out on your behavior. Properly not your first online dick move since it’s such a priority for you to hide who you are.
So once again spend all the money you want on your lawyer, it will get you nowhere. And the funny thing is that it will not take away anything from this conversation or make your behavior any better.
This is going to be a good addition to the Reddit thread about ungrateful people that pisses allowed people in open source. What did you and your lawyer ever contribute to the community?
Thank you for stopping your conversation with me!
And David… go fuck your self!
LikeLike
Worked great! Thank You
LikeLike