Today I found the “virt-backup.pl” script from Daniel Berteaud which is IMHO the best solution to fully backup a libvirt managed virtual machine. The perl script is very flexible and allows for various configurations depending on your VM setup and backup needs, it requires ubuntu packages “libsys-virt-perl” and “libxml-simple-perl” to run.
I use it to backup running virtual machines in this way: first, suspend the virtual machine (for a short while), then create a snapshot of the LVM used as storage by the virtual machine, then resume the virtual machine so it can continue running without almost no downtime (taking a LVM snapshot is very fast!), and after that dump the snapshot to a compressed image file on a remote server (backup storage). To accomplish this just run this command (for example from a cron job):
virt-backup.pl --pre --vm=machine1 --backupdir=/mnt/remotenas/ --privatedir --compress --snapsize=5G --debug
If you use ubuntu, note that the script needs to be edited to adjust the location of lvcreate and lvremove (in /sbin instead of /usr/sbin).
github.com/poliva
Google+
linkedin.com/in/pauoliva
twitter.com/pof
Actually, what that script does is just the best way I’ve always found to backup a KVM/Xen VM…the best thing about this script is it’s already done! and you don’t have to write a single line of code 😉
Depending on the OS running as guest on your VM and its purpose (web, DB, etc…) you could even skip the suspend/resume steps and just do a fsck on the snapshot before dumping it.
Great finding! and thanks for sharing!
Good finding! But, I don’t see how to restore the backup…
Is there any flag or another command to restore a backup?
The backup is a raw ‘
dd‘ of the full lvm2 snapshot, so just create a new logical volume of the same size and use ‘dd‘ to restore it.Can U post any restore example??
When using gzip:
# gzip -dc backup.gz | dcfldd of=/dev/vg1/lv1
Hello
How restore the state file?
./virt-backup.pl –dump –vm=vm_kiwix –state –compress
Thank
The state file is created using ‘virsh save’ option, to restore the guest just use the command:
virsh restore [filename]This restarts the saved guest, which may take some time. The guest’s name and UUID are preserved but are allocated for a new id.
How do I determine the snapsize, what happens if its not big enough? I just converted my VM from qcow2 to a LVM and it grew to 100G because I had allocated a stupidly large size. I left the snapsize at default of 5G and the script seemed to work although I have not attempted a restore. Well done for posting this script it has save a lot of work.
Can anyone tell me the best way to shrink the size of the machine please?
snapsize should be large enough for the amount of diff data that can be generated in the filesystem while the backup process takes place, for most situations 5GB should be more than enough, unless you have a heavily changing filesystem in the machine you are backing up.
To shrink the disk size you’ll have to use different tools depending on the filesystem used, for example resize2fs will work if you use ex2, ext3 or ext4 filesystems.
I was about to run out of disk space during a backup so aborted with a CtrlC and now there is an active snapshot. What is the “safe” way to get back to the original. Does a destroy snapshot merge it back?
why not just remove the “unfinished” snapshot with lvremove, free up some disk space and launch the backup script again (which will create a new snapshot).
Thanks, I was a bit nervous about doing that on a live system. One of the problems was that there were 2 drives in the VM the first was the system drive and a 5G snapshot was created and that was saved fine, but the second data drive was 500G and a 500G snapshot was created by default which was a bit big. I probably need to do each individually. But the data is backed up offsite anyway, so its just the OS drive that needs saving as an image.
Dave
Dave, did you manage to figure out the syntax for backing up both drives without a 500G snapshot?
thanks
Hi,
Thanks for the script. I tested the restoration of the backup, and unfortunately got a strange error.
First I decompressed the backup and piped it to dd, like:
$ pigz -d -c -v bitstest_hda.img.gz | dd of=/dev/raidstorage/backuptest bs=16065
galvm-bitstest_hda.img.gz to
791389+128655 records in
791389+128655 records out
12884901888 bytes (13 GB) copied, 172.781 s, 74.6 MB/s
Next, when I created a new virtual machine in the graphical console of virt-manager with the right size, and pointed to use the /dev/raidstorage/backuptest disk: so far so good.
When I launch the virtual machine, I got the grub menu (ok), and CentOS starts up. But during the first steps of the start up, I got…
“Reading all physical volumes. This may take a while…
No volume groups found.”
And then a bunch of errors, kernel panic,… , because there must be a volume group called VolGroupSystem.
Any idea what I have done wrong?
Joachim
I works when you use virsh (after manually correcting the xml to point to the right logical volume).
$ virsh create bitstest.xml
Cheers,
Joachim
We are running two ubuntu KVM based virtual servers each running on separate physical server and each KVM environment hosting a dozen of VMs without proper backup. Each physical server is running its own internal RAID 5 storage and not shared storage or networked storage. We would like to backup VMs running in 1st Ubuntu KVM on to 2nd ubuntu KVM and vice versa to backup each other. Our goal is to have proper backup and bring up the VMs in healthy physical server in the KVM environment if one of the physical servers go down.
The method described here seems to work for shared storage / LVM environment. Any option for our environment, either opensource or commercial, preferably using some kind of GUI ?
how would I go about verifying this script worked after I run it. I use it in a cron to back up my VMs. I keep several weeks worth of backups. I want to verify the most recent back up worked before I delete my oldest ones.
Thanks!
Thanks, this seems to be the best option so far. Do you know if the script ready to run on CentOS 6 without modifications?
To successfully run it on CentOS 6 I had to install following dependencies: perl-XML-Simple and perl-Sys-Virt.
Then I had to correct paths to lvcreate and lvremove inside this script:
# lvcreate path
$opts{lvcreate} = ‘/usr/sbin/lvcreate -c 512′;
# lvremove path
$opts{lvremove} = ‘/usr/sbin/lvremove';
Above shoud be changed to /sbin/lvcreate and /sbin/lvremove.
That’s all.
Exactly the thing I was looking for. Thank you for sharing (:
I added this line:
@disks = grep { $_ != $disk } @disks;
To the bottom of the first loop in sub run_dump, to avoid the tool continually trying to dump snapshots it had already discarded, when backing up multiple VMs
D
The snapshot does not work, and I don’t know why
It takes 342.084 s to backup a 42GB VM
Any idea?
root@lab:~# vgs -o +vg_free_count,vg_extent_count
VG #PV #LV #SN Attr VSize VFree Free #Ext
backup 1 1 0 wz–n- 263.57g 13.57g 3474 67474
lab-lvm 1 1 0 wz–n- 186.36g 16.36g 4187 47707
lab-swap 1 1 0 wz–n- 15.83g 0 0 4052
maquinas 1 1 0 wz–n- 465.76g 65.76g 16834 119234
Thank you for sharing the script. How much time does it take to execute? I am trying to take a snapshot and it takes more than 30 minutes to complete. Is this ok?
When I run:
# ./virt-backup.pl –dump –vm=ovpn153 –debug –snapsize=8G
It always say “Snapshoting /dev/guest_images_lvm/lvm_ovpn153 has failed (not managed by LVM, or already a snapshot ?), live backup will be disabled”. Any it suspend the guest for 96 seconds to finish the backup. What is the problem?
Both my host and guest are CentOS 6.4. And I have:
# virsh pool-list –all
Name State Autostart
—————————————–
default active yes
guest_images_lvm active yes
# virsh vol-list guest_images_lvm
Name Path
—————————————–
lvm_ovpn153 /dev/guest_images_lvm/lvm_ovpn153
Could you fix this? I’m facing the same problem.
Looks like there is something wrong with the lvcreate command when creating the snapshot.
I executed the lvcreate command manually and got this error:
—————————————————————-
Inconsistent volume group names given: “mapper” and “vg_test”
Run `lvcreate –help’ for more information.
—————————————————————-
Thank you.
I found where the problem is, my disk/source/dev are defined with /dev/mapper/vg.._lv.. instead of /dev/vg_group/lv…
I just changed the configuration on the XML files and works.
Are there any Backup software (other than the native capability) that can be used to backup or archive the snapshots?
Hey RC – if you’re looking for a more turnkey backup and restore solution for KVM and Xen, you might want to check out ConVirt.
Is there any additional option to be used to restore VM’s imediately after creating snapshot? Because the VM is only resuming after the entire process is completed.
Hello
How can i enable live backup ? when i backup it suspend my VPS
Thank you