Best solution to fully backup KVM virtual machines

Today I found the “virt-backup.pl” script from Daniel Berteaud which is IMHO the best solution to fully backup a libvirt managed virtual machine. The perl script is very flexible and allows for various configurations depending on your VM setup and backup needs, it requires ubuntu packages “libsys-virt-perl” and “libxml-simple-perl” to run.

I use it to backup running virtual machines in this way: first, suspend the virtual machine (for a short while), then create a snapshot of the LVM used as storage by the virtual machine, then resume the virtual machine so it can continue running without almost no downtime (taking a LVM snapshot is very fast!), and after that dump the snapshot to a compressed image file on a remote server (backup storage). To accomplish this just run this command (for example from a cron job):


virt-backup.pl --pre --vm=machine1 --backupdir=/mnt/remotenas/ --privatedir --compress --snapsize=5G --debug

If you use ubuntu, note that the script needs to be edited to adjust the location of lvcreate and lvremove (in /sbin instead of /usr/sbin).

This entry was posted in linux and tagged , , , , . Bookmark the permalink.

30 Responses to Best solution to fully backup KVM virtual machines

  1. TripleDES says:

    Actually, what that script does is just the best way I’ve always found to backup a KVM/Xen VM…the best thing about this script is it’s already done! and you don’t have to write a single line of code 😉

    Depending on the OS running as guest on your VM and its purpose (web, DB, etc…) you could even skip the suspend/resume steps and just do a fsck on the snapshot before dumping it.

    Great finding! and thanks for sharing!

  2. Mario Medina says:

    Good finding! But, I don’t see how to restore the backup…

    Is there any flag or another command to restore a backup?

  3. liyuan says:

    Hello

    How restore the state file?

    ./virt-backup.pl –dump –vm=vm_kiwix –state –compress

    Thank

    • pof says:

      The state file is created using ‘virsh save’ option, to restore the guest just use the command:
      virsh restore [filename]
      This restarts the saved guest, which may take some time. The guest’s name and UUID are preserved but are allocated for a new id.

  4. Dave says:

    How do I determine the snapsize, what happens if its not big enough? I just converted my VM from qcow2 to a LVM and it grew to 100G because I had allocated a stupidly large size. I left the snapsize at default of 5G and the script seemed to work although I have not attempted a restore. Well done for posting this script it has save a lot of work.
    Can anyone tell me the best way to shrink the size of the machine please?

    • pof says:

      snapsize should be large enough for the amount of diff data that can be generated in the filesystem while the backup process takes place, for most situations 5GB should be more than enough, unless you have a heavily changing filesystem in the machine you are backing up.

      To shrink the disk size you’ll have to use different tools depending on the filesystem used, for example resize2fs will work if you use ex2, ext3 or ext4 filesystems.

  5. Dave says:

    I was about to run out of disk space during a backup so aborted with a CtrlC and now there is an active snapshot. What is the “safe” way to get back to the original. Does a destroy snapshot merge it back?

    • pof says:

      why not just remove the “unfinished” snapshot with lvremove, free up some disk space and launch the backup script again (which will create a new snapshot).

      • Dave says:

        Thanks, I was a bit nervous about doing that on a live system. One of the problems was that there were 2 drives in the VM the first was the system drive and a 5G snapshot was created and that was saved fine, but the second data drive was 500G and a 500G snapshot was created by default which was a bit big. I probably need to do each individually. But the data is backed up offsite anyway, so its just the OS drive that needs saving as an image.

        Dave

  6. Joachim says:

    Hi,

    Thanks for the script. I tested the restoration of the backup, and unfortunately got a strange error.

    First I decompressed the backup and piped it to dd, like:
    $ pigz -d -c -v bitstest_hda.img.gz | dd of=/dev/raidstorage/backuptest bs=16065
    galvm-bitstest_hda.img.gz to
    791389+128655 records in
    791389+128655 records out
    12884901888 bytes (13 GB) copied, 172.781 s, 74.6 MB/s

    Next, when I created a new virtual machine in the graphical console of virt-manager with the right size, and pointed to use the /dev/raidstorage/backuptest disk: so far so good.

    When I launch the virtual machine, I got the grub menu (ok), and CentOS starts up. But during the first steps of the start up, I got…
    “Reading all physical volumes. This may take a while…
    No volume groups found.”

    And then a bunch of errors, kernel panic,… , because there must be a volume group called VolGroupSystem.

    Any idea what I have done wrong?

    Joachim

    • Joachim says:

      I works when you use virsh (after manually correcting the xml to point to the right logical volume).

      $ virsh create bitstest.xml

      Cheers,
      Joachim

  7. Enn pee says:

    We are running two ubuntu KVM based virtual servers each running on separate physical server and each KVM environment hosting a dozen of VMs without proper backup. Each physical server is running its own internal RAID 5 storage and not shared storage or networked storage. We would like to backup VMs running in 1st Ubuntu KVM on to 2nd ubuntu KVM and vice versa to backup each other. Our goal is to have proper backup and bring up the VMs in healthy physical server in the KVM environment if one of the physical servers go down.

    The method described here seems to work for shared storage / LVM environment. Any option for our environment, either opensource or commercial, preferably using some kind of GUI ?

  8. rlr219 says:

    how would I go about verifying this script worked after I run it. I use it in a cron to back up my VMs. I keep several weeks worth of backups. I want to verify the most recent back up worked before I delete my oldest ones.

    Thanks!

  9. Eddie says:

    Thanks, this seems to be the best option so far. Do you know if the script ready to run on CentOS 6 without modifications?

    • Paweł says:

      To successfully run it on CentOS 6 I had to install following dependencies: perl-XML-Simple and perl-Sys-Virt.
      Then I had to correct paths to lvcreate and lvremove inside this script:
      # lvcreate path
      $opts{lvcreate} = ‘/usr/sbin/lvcreate -c 512′;
      # lvremove path
      $opts{lvremove} = ‘/usr/sbin/lvremove';

      Above shoud be changed to /sbin/lvcreate and /sbin/lvremove.

      That’s all.

  10. Hadret says:

    Exactly the thing I was looking for. Thank you for sharing (:

  11. David Young says:

    I added this line:

    @disks = grep { $_ != $disk } @disks;

    To the bottom of the first loop in sub run_dump, to avoid the tool continually trying to dump snapshots it had already discarded, when backing up multiple VMs :)

    D

  12. Jose Ferradeira says:

    The snapshot does not work, and I don’t know why
    It takes 342.084 s to backup a 42GB VM
    Any idea?
    root@lab:~# vgs -o +vg_free_count,vg_extent_count
    VG #PV #LV #SN Attr VSize VFree Free #Ext
    backup 1 1 0 wz–n- 263.57g 13.57g 3474 67474
    lab-lvm 1 1 0 wz–n- 186.36g 16.36g 4187 47707
    lab-swap 1 1 0 wz–n- 15.83g 0 0 4052
    maquinas 1 1 0 wz–n- 465.76g 65.76g 16834 119234

  13. grezman says:

    Thank you for sharing the script. How much time does it take to execute? I am trying to take a snapshot and it takes more than 30 minutes to complete. Is this ok?

  14. dynaguy says:

    When I run:
    # ./virt-backup.pl –dump –vm=ovpn153 –debug –snapsize=8G

    It always say “Snapshoting /dev/guest_images_lvm/lvm_ovpn153 has failed (not managed by LVM, or already a snapshot ?), live backup will be disabled”. Any it suspend the guest for 96 seconds to finish the backup. What is the problem?

    Both my host and guest are CentOS 6.4. And I have:
    # virsh pool-list –all
    Name State Autostart
    —————————————–
    default active yes
    guest_images_lvm active yes

    # virsh vol-list guest_images_lvm
    Name Path
    —————————————–
    lvm_ovpn153 /dev/guest_images_lvm/lvm_ovpn153

    • charly74 says:

      Could you fix this? I’m facing the same problem.

      Looks like there is something wrong with the lvcreate command when creating the snapshot.

      I executed the lvcreate command manually and got this error:

      —————————————————————-
      Inconsistent volume group names given: “mapper” and “vg_test”
      Run `lvcreate –help’ for more information.
      —————————————————————-

      Thank you.

      • charly74 says:

        I found where the problem is, my disk/source/dev are defined with /dev/mapper/vg.._lv.. instead of /dev/vg_group/lv…

        I just changed the configuration on the XML files and works.

  15. R.C. Delaware says:

    Are there any Backup software (other than the native capability) that can be used to backup or archive the snapshots?

  16. Danno63 says:

    Hey RC – if you’re looking for a more turnkey backup and restore solution for KVM and Xen, you might want to check out ConVirt.

  17. Pruthvi Nandan Reddy says:

    Is there any additional option to be used to restore VM’s imediately after creating snapshot? Because the VM is only resuming after the entire process is completed.

  18. Mostzaa says:

    Hello

    How can i enable live backup ? when i backup it suspend my VPS

    Thank you

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>