Automated Raspberry Pi Backup – complete image

I love my Raspberry Pi projects and I run a lot of specialist “mini” servers at home doing everything from torrent sharing of Linux distros to media streaming and media playing. But all Raspberry Pi’s and other single board computers that rely on SD-cards sooner or later comes to a point where they trash the card and doesn’t boot again.

Every time I run into that situation without remembering exactly what was running and how on the particular Raspberry Pi. I want backups, not just the backup I usually do right after installation but a last night backup or similar. So I put up an NFS share on my NAS to store the backups, it will work just as well with a USB stick connected directly to the Raspberry Pi. Here is a step by step guide how I automated the backups on all my Raspberry Pi’s. This script will create a complete image of the SD-card while the Raspberry Pi is running. You can just write that image to a new SD-card and pop it into the Pi and it will be like nothing happened!

  1. Create the folder /mnt/backup
  2. Edit fstab with

    [bash]sudo /etc/fstab[/bash]

    Add the share on the NAS for mounting

    [bash]192.168.6.5:/nfs/Backups/raspberrypi_boot /mnt/backup nfs rsize=8192,wsize=8192,timo=14,intr 0 0[/bash]

  3. Test the mount

    [bash]sudo mount -a[/bash]

  4. Download my backup script from Github

    [bash]wget https://github.com/kallsbo/BASH-RaspberryPI-System-Backup/raw/master/system_backup.sh[/bash]

    I placed the script on my NAS share so all the Raspberry Pi’s could use the same script file.

  5. Make the script file executable

    [bash]chmod +x system_backup.sh[/bash]

  6. Add it to crontab. Since the script needs to be executed as root we need to edit the root crontab.

    [bash]sudo crontab -e[/bash]

    Just add the line (with correction for the path to the script)

    [bash]0 3 * * * /mnt/backup/system_backup.sh[/bash]

The script is a simple bash script that you can look at and download from Github – BASH-RaspberryPI-System-Backup. The script has two variables, backup_path and rentention_days. The first is the path where to store the backup and the second is the number of days to save it. They can have defaults set in the script but you can override them by supplying command line arguments like:

[bash]sudo ./system_baskup.sh /mnt/otherpath 7[/bash]

That will give you the backup path of /mnt/otherpath and a retention of 7 days. The next few lines (until line 38) just checks if it’s running as root and checks for command line arguments. Then we touch the file /boot/forcefsck this creates a blank file at that path and forces the Raspberry Pi to check the filesystem on the next boot. This is so the backup image, if used, will be checked for filesystem inconsistencies on first boot.

Then comes the magic sauce, creating a complete copy of the SD-card:

[bash]dd if=/dev/mmcblk0 of=$backup_path/$HOSTNAME.$(date +%Y%m%d).img bs=1M[/bash]

Then the script removes the /boot/forcefsck file so the force file is only in the backup image and not on the currently running SD-card. Then it looks for backups older then the number of retention days and deletes them. Since the script includes the hostname in the filename and only looks for it’s own hostname you can have several Raspberry Pi’s backing up to the same location without being worried about them deleting each others backups.

63 Comments on “Automated Raspberry Pi Backup – complete image”

  1. Hi, thanks for this. i ran it from the command line and i get:

    pi@pi:~ $ dd: failed to open ‘/myNAS/backup/pi.20180406.img’: No such file or directory
    find: `/myNAS/backup/pi.*.img’: No such file or directory

    it’s the first time it’s run so nothing will be there. it then appears to hang. any ideas or is it actually compiling the backup in the background (ileft it for hours and nothing happened). Cheers, Don

    Like

  2. thanks for the speedy reply. i can create files on the NAS from the Pi. It is mounted OK. i don’t know what the problem might be. cheers, Don

    Like

  3. Thanks for a solution, I will use it. A lot of typos in the text
    timo=14 (timeo=14)
    sudo ./system_baskup.sh /mnt/otherpath 7 (backup.sh)

    Like

  4. Thanks for this! I’m testing it out now and it seems it is going to backup the whole 64GB sd card.. so far its backed up about 12GB and my total used space on the card is about 4.9GB. Is this correct that it will backup the empty space as well? If so, is there a way to reduce that?

    Like

    • Yes the whole SD card will be imaged. There is no way for the dd command to determine what is empty space and not. I did experiment with some empty space removal of the image it self after backup but ran into inconsistent behavior so I dropped it.

      Like

  5. Hi,
    Is this process of backup doable also for running applications like WordPress which using MariadDB for database. I mean is it possible to create proper image which could be also properly restored while the Raspberry Pi is running the whole time?

    Like

    • Haven’t tried with that specific combo of software but the backup should be consistent. I only tried it with a MySQL used by my Kodi boxes and it worked just fine. I recommend you try it out.

      Like

  6. Thanks again for this work, Kristofer. The script worked fine to create the image on an NFS mount (NFS share from a rasPi NAS server with a hard drive). Then I split the file and cp them to my mac laptop from the NFS server. Then I cat files.* > original.img and burn the SD. Boots fine, great! I think the full dd of the SD in the rasPi probably works it more than my regular production does (rasPi email server). So, I have the script in /etc/cron.weekly and then I do file backups of the email directories more often.

    (/usr/bin/split –bytes=1G mail2.YYYYMMDD.img mail2.YYYYMMDD.split)&
    then, from the NAS server I copy all the files to my laptop where I do
    cat mail2.YYYYMMDD.split* > mail2.YYYYMMDD.img and burn the new SD.

    Like

  7. Pingback: Pi-Backup – cozmopics.de

  8. Hi Kristofer,

    Great solution. Couple of points:

    1. You still have a typo in your NFS mount options (timeo, not timo)
    2. You can make the backup much smaller and faster if you pipe through gzip as you go. (Gzip buffers writes, whereas dd does not)

    # Perform backup and pipe through gzip to improve speed/size
    dd if=/dev/mmcblk0 bs=1M | /bin/gzip > $backup_path/$HOSTNAME.$(date +%Y%m%d).img.gz

    Rock on!

    Liked by 1 person

  9. Hello,

    I have recently configured this backup script end it’s really awesome!

    But i have a problem: I have set a retention days to 3, but the script don’t delete the oldest backup.
    Do you have any suggestion for resolve this problem?

    Like

    • Does it delete any files? Do a ls -al and check the dates of the files to make sure it’s correct. If any other process touches the files the dates might be updated and cause the delete not to work.

      Like

  10. No other process touches the files, in this sd card (with raspian) no other process are installed yet.

    In the ls -al command i’ve found this line:
    -rwxr-xr-x 1 pi pi 1182 mar 9 11:53 system_backup.sh

    The file names i think are correct: Pi3b.20200315.img

    I have installed this plugin on monday or thuesday this week, end the older file, at the moment, are namedPi3b.20200312.img. Before that day i have eliminated manually to create space on the usb stick i have mounted to save my bck files.

    Like

    • As far as I can see there is no .img files in the ls command? Run the ls command against the folder where you store the backups. The script will delete any file older then the specified number of days in the target directory. You have to run the script against the same folder each time for that to work. Also make sure that the time/date is correct on the pi and the process have access to the backup files.

      Like

  11. Hello,

    I’ve tried some thing and I noticed that the script works well deleting oldest files on the target directory, only if i launch it manually, when it run at the scheduled time with crontab don’t delete the oldest backup.

    For your question the time/date is correct,
    This is the ls -al command in the directory i store bck

    “pi@Pi3b:/media/pi/bck/mnt/backup $ ls -al
    totale 45367300
    drwxrwxrwx 1 pi pi 4096 mar 23 03:00 .
    drwxrwxrwx 1 pi pi 0 mar 9 11:18 ..
    -rwxrwxrwx 1 pi pi 15485370368 mar 21 11:43 Pi3b.20200321.img
    -rwxrwxrwx 1 pi pi 15485370368 mar 22 03:16 Pi3b.20200322.img
    -rwxrwxrwx 1 pi pi 15485370368 mar 23 03:16 Pi3b.20200323.img”

    and this is the ls -al command in the script backup

    pi@Pi3b:/media/pi/bck $ ls -al
    totale 9
    drwxrwxrwx 1 pi pi 4096 mar 9 11:25 .
    drwxr-x—+ 3 root root 4096 mar 19 08:59 ..
    drwxrwxrwx 1 pi pi 0 mar 9 11:18 mnt
    lrwxrwxrwx 1 pi pi 25 mar 9 11:22 system_backup.sh -> /home/pi/system_backup.sh
    drwxrwxrwx 1 pi pi 0 mar 9 11:11 ‘System Volume Information’

    Like

    • It sounds like that the cron is executed with the wrong permissions to be able to delete the files then. You can also test to pipe the output of the script in the crontab to see any errors.

      Like

  12. Hello I just tried to mount my nas via nfs but i get this error.

    what i entered in /etc/fstab

    192.168.1.200:/Multimedia/Backups/ /mnt/backup nfs rsize=8192,wsize=8192,timo=14,intr 0 0

    then: sudo mount -a

    Error:
    mount.nfs: an incorrect mount option was specified

    Like

  13. It seems to be a good/clear tutorial to make a copy of the SD-Card. That’s what I need. But I have no NAS. Could you change the script and explain how to do it with web space and webDAV?
    Thanks

    Like

  14. I can not get the cron to run. I can run the system_backup.sh on a command line. I have other crons in the same folder that run with no problem. I have listed my cron below.

    0 2 * * * /home/pi/system_backup.sh

    Like

  15. Hello Kristofer,
    I have this running on a Raspberry Pi but I get an error on execution. The error is

    “system_backup.sh: 20: system_backup.sh: [[: not found.”

    The backup does continue but it names the backup file .YYYYMMDD.img. It completely ignores the $HOSTNAME. I can echo the hostname in the terminal and it’s correct but it never gets appended to the output file. Any ideas? Thanks!

    Like

  16. OK, Thanks. I was hoping it was going to be something easy but if nothing else, I’ll comment it out.

    Do you have any idea why it’s not appending the $HOSTNAME to the beginning of the file name? It doesn’t give any errors in regards to creating or naming the file.

    Like

  17. OK, chalk this one up to operator error. I was running it with sh instead of bash. Apparently it doesn’t work well with sh. hahaha Thanks for the help!

    Like

  18. Hi,
    Thank you for the script. Could you let me know the steps to backup to a USB drive connected to the pi instead of a NAS?

    Like

    • Just mount the usb drive to /mnt/backup. There are a lot of examples and tutorials for mounting usb online. Just Google “mount usb raspberry pi” and you will find all the information needed.

      Like

  19. Thanks for the backup script! Can you provide a script that you can use to restore the image? I’m not sure how these backups are useful without that. If the assumption is “use wind32diskimager” then please explain, although my preference would be keeping it in Linux. Thanks!

    Like

    • The created image is a complete copy of the SD-card so there is no way to restore it to a running Pi. The image can be used the same way as the base OS images and applied to an SD-card. For example you can use the Raspberry Pi OS Images available for Ubuntu to reapply the image to the SD-card.

      Like

      • We need a Linux script to restore the image file to an sd-card. Its not going to be run on “the running Pi”. You are going to take the sd card out of the Pi, put it into another computer, restore the image to the sd-card and then pop it back into the Pi. I can always burn another release image to an sd-card, but I want to restore the image we just made with your first script and using Linux/Bash.

        Like

      • If you know how to “burn another release image” to the card you know how to restore this. Why do you need a script when you still need to do the manual task of transferring the sd-card? Just do how you would normally do but “burn” the backup image to the new sd-card instead.

        Like

      • I use Win32DiskImager to burn Linux Releases in Windows, but I’ve said twice now I need a linux script to do that for these backups. If you don’t know how to create a linux script to restore the image you created in your previous scripts you could just say so and save yourself some time, instead of telling me how you think I “should” be doing this. If I could take back my PayPal donation at this point I would.

        Like

      • Are you for real? So just because I share this free of charge and you tip me $2 I’m on the hook to do this for you? If you don’t know how to image an sd-card in Linux from the cli, try to google it. But please David Cherry don’t bring your piss poor attitude to my comments. If you don’t like what I give away for free develop your own solution!

        Like

  20. Doxing is illegal, and also against WordPress terms of service. You can act tough now, while you still have a website.

    Like

    • What you don’t know could fill a warehouse! Doxing is only illegal if the information is obtained in an illegal way, you sent me the information. In addition US law don’t apply to me, I’m not in the US. Take this as a learning experience that you can’t run around acting like a spoiled brat to people. Do what you feel you need to do.

      Like

    • Gt461 is an absolute entitled Karen. I apologize on behalf of the US. Thanks for the script and help you’ve provided for FREE Hackingviking!

      Like

  21. Doxing is the act of publishing personal information. How it is obtained is secondary, and I am not even addressing that. WordPress is an international company and will respond to US lawyers, in fact I won’t even have to use a lawyer with them to have this page pulled down for violating terms of service. I did not provide you with any information to be used on this forum. Let what happens to you be a learning experience as well.

    Like

      • Already submitted the complaint. They have a nice page setup for it. You should be hearing from them soon!

        Like

      • Refund for what? You donated $2 because you liked the code/article. You do realize that this code is published under the MIT license? That means no liability and no warranty. All the time you spent bitching here would have been more than enough to actually google your issue and use someone else’s free work as well to solve your problem.

        https://letmegooglethat.com/?q=How+to+Write+Image+Files+to+SD+Card+with+dd+from+Command

        I have already disputed your BS paypal claim for “goods not delivered”. Paypal decided to refund your money with no cost to me, so you basically scammed Paypal for $2. The paypal button clearly states that it’s a donation and no expectation of goods is more than obvious!
        As they state “Due to the unique circumstances of this case”, which is the nice version of “because David Cherry is a dick”, PayPal will refund your buyer at no cost to you. Could you just please drop this and move on?

        Like

      • That is an inaccurate portrayal of what happened. They actually offered me more than the $2, but I said I didn’t want it because of the pending legal issue between you and I for you doxing me and that I needed the dispute to be by the book. I actually spoke with them.

        I see that you did escalate it but that their response to me was that they found in my favor, not that “in this unique case”. Any other reasons you want to portray here are lies. I have contacted a lawyer and found that you are hosted out of California. Much to our happiness because they have about the most stringent privacy laws on the planet. It won’t be much longer before we have lots more information.

        So far I have told you what you can do to resolve this and I will go away. Remove my name from your blog. Leave up everything else you’ve said if you want.

        Ebay response:
        “Case status Closed

        The case was closed in your favor.”

        Like

      • You mean this case? https://pasteboard.co/UefC3JFFgvRu.png

        If there is any doubt about the authenticity I can publish the whole thing if you like? I think you misunderstood, I don’t care if this goes away. I just want you to man-child somewhere else, through your lawyer is just fine. Because what you don’t realize is that I don’t care about your US lawyer and you might get WordPress to take down the site but it will be up straight away hosted outside of the US. I actually think I have a bash-script for the setup… 🙂

        You can’t do anything to me, it was years since I moved from California and I’m not a US citizen. In my country, which has actual freedom of speech, I’m fine. All I did is called you by your name and that is not a crime. Pretty sure that isn’t a crime even in the US since I haven’t spilt any other personal details.

        So spend your money on your lawyer all you want it doesn’t matter to me. All my contact details are on the blog.

        Like

      • That Paste means zero, because it looks exactly like what they sent me. My lawyer will find out the truth on it so don’t bother.

        I know you want to make this about me, but no sane person goes as far as you have gone to publicly call someone out because they said they wanted their money back, while you saying you are above the law but I am the man-child? Once we have what we need on you, you WILL be hearing from us. This is my final direct communication to you.

        Like

      • I called you out for being a dick. You wanting your money back came later. Once again you misunderstand or don’t understand I’m not above the law but US law don’t apply outside of the US. Unless you guys invade Sweden it means absolutely nothing to us. I’m pretty sure that’s not gone happen since we don’t have any oil.

        If we look at this from US law perspective the whole doxxing thing is ridiculous! The name Davis Cherry isn’t specific enough to identify anyone person specifically. There is more than a thousand profiles on LinkedIn and Facebook with that name and I’m pretty sure your not David Cherry the Scottis rugby player.

        You just got pist because I called you out on your behavior. Properly not your first online dick move since it’s such a priority for you to hide who you are.

        So once again spend all the money you want on your lawyer, it will get you nowhere. And the funny thing is that it will not take away anything from this conversation or make your behavior any better.

        This is going to be a good addition to the Reddit thread about ungrateful people that pisses allowed people in open source. What did you and your lawyer ever contribute to the community?

        Thank you for stopping your conversation with me!

        And David… go fuck your self!

        Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: