I’ve been a member of VMUG and user of VMware on my Dell R710 and then Dell R720XDs for almost 10 years now. It’s been interesting and valuable in helping me understand VMware.
A couple of jobs back, I was reintroduced to KVM. I’d tried it before getting into VMware and couldn’t get the hang of it but with it being my job, and my new skills with virtual machines, I had a better grasp and was even able to build terraform scripts to build sites. Cool stuff.
With Broadcom restricting access to VMUG members, and my license having expired, plus moving, had left my existing VMs out in the cold. While I have terraform scripts and documentation (and backups) for most of my systems, I do need to get some data off of some where I either failed to retrieve the data or backed up the operations but not my home directories or just verify the backups I have are complete enough.
Some of the files aren’t a big deal. Jenkins and Gitlab, I’d just reinstall and reimport or rebuild the processes. I’m not in a production or developer environment where I need to bring in all the changes over the past 10 years. Just recreate the setup, git push the files, and move forward. Heck, I’ll even have clean installations. When I first installed Jenkins, I installed everything that was suggested. With experience now, I’ll just install what I need.
The first step is to pull them off the VMware systems. I can bring them up, I just can’t start any VMs. I enabled SSH access to the systems and on my R710 KVM box, simply scp’d the ones I wanted to review over to a /opt/vms directory. I reviewed the system specs in order to properly start them and off we go.
The first step is to convert the images to the qcow2 format. Install the qemu-img package and run the following command:
qemu-img convert -f vmdk -O qcow2 /opt/vms/monkey/bldr0cuomaws1/bldr0cuomaws1.vmdk bldr0cuomaws1.qcow2
Next up is to install it into Qemu. This makes it visible to KVM in order to run the system. I used the settings I retrieved in order to properly configure the domain.
virt-install --name bldr0cuomaws1 --ram 4096 --vcpus 2 --disk bldr0cuomaws1.qcow2,format=qcow2 --import
Here’s the tricky part. For most of the systems, I just wanted to retrieve the data. Once the domain has been configured, you can try to start the new server but I wasn’t having much success. I did find I could actually just use a command called guestmount and simply mount the image to /mnt and copy the data from the system.
guestmount -d bldr0cuomaws1 --ro -i /mnt
Once done, I changed over to /mnt and simply copied the data from my home directory to a central location. After that, I didn’t really need this image any more so I deactivated it and removed it.
virsh pool-destroy bldr0cuomaws_pool
virsh pool-delete bldr0cuomaws_pool
Next up, I need to see how this will work with multiple disks.
Oh, one thing. You can make sure the image was copied and converted properly before you delete the VM from VMware.
# qemu-img info bldr0cuomrepo1.qcow2
image: bldr0cuomrepo1.qcow2
file format: qcow2
virtual size: 100 GiB (107374182400 bytes)
disk size: 16.8 GiB
cluster_size: 65536
Format specific information:
compat: 1.1
lazy refcounts: false
refcount bits: 16
corrupt: false
For a multi-disk image setup, we’ll need to convert the disks and then attach them to the primary image. They are all LVM though so you can’t really mount the entire system using guestmount. You’d only mount individual mount points.
For LVM systems, when you add the disk, you’ll need to add them as the appropriate sd. They’re added sequentially so a drive with 4 extra drives, you’d add sdb, sdc, sdd, and sde.
Once that’s done, you’ll need to look at the mount points and then mount them individually as guestmount won’t actually mount everything.
We’ll walk this through from conversion to mounting.
First off, here’s the original directory and files for this server. It basically held a bunch of linux images used when kickstarting servers. Kind of an automatic build process from the past. Do I need everything? Probably not. In this case, I’m just seeing what’s there, maybe in the home directory, and copying it off:
# ls -al
total 1656836168
drwxr-xr-x 2 root root 4096 Oct 30 23:02 .
drwxr-xr-x 6 root root 4096 Oct 31 23:30 ..
-rw-r--r-- 1 root root 1298 Oct 30 16:49 lnmt1cuomjs1-52466e53.hlog
-rw------- 1 root root 85899345920 Oct 30 17:08 lnmt1cuomjs1-flat.vmdk
-rw------- 1 root root 8684 Oct 30 23:02 lnmt1cuomjs1.nvram
-rw------- 1 root root 508 Oct 30 17:08 lnmt1cuomjs1.vmdk
-rw-r--r-- 1 root root 0 Oct 30 23:02 lnmt1cuomjs1.vmsd
-rwxr-xr-x 1 root root 4084 Oct 30 23:02 lnmt1cuomjs1.vmx
-rw------- 1 root root 536870912000 Oct 30 19:06 lnmt1cuomjs1_1-flat.vmdk
-rw------- 1 root root 511 Oct 30 19:06 lnmt1cuomjs1_1.vmdk
-rw------- 1 root root 536870912000 Oct 30 21:04 lnmt1cuomjs1_2-flat.vmdk
-rw------- 1 root root 511 Oct 30 21:04 lnmt1cuomjs1_2.vmdk
-rw------- 1 root root 536870912000 Oct 30 23:02 lnmt1cuomjs1_3-flat.vmdk
-rw------- 1 root root 457 Oct 30 23:02 lnmt1cuomjs1_3.vmdk
-rw------- 1 root root 186501 Oct 30 23:02 vmware-89.log
-rw------- 1 root root 413874 Oct 30 23:02 vmware-90.log
-rw-r--r-- 1 root root 309911 Oct 30 23:02 vmware-91.log
-rw-r--r-- 1 root root 226039 Oct 30 23:02 vmware-92.log
-rw-r--r-- 1 root root 281301 Oct 30 23:02 vmware-93.log
-rw-r--r-- 1 root root 334962 Oct 30 23:02 vmware-94.log
-rw-r--r-- 1 root root 191944 Oct 30 23:02 vmware.log
-rw------- 1 root root 85983232 Oct 30 23:02 vmx-lnmt1cuomjs1-24cae1aa722f12da9b70e188df14347036fca212-2.vswp
The files we’re interested in are just the vmdk files. These have a description of each disk, like so:
# cat lnmt1cuomjs1.vmdk
# Disk DescriptorFile
version=1
encoding="UTF-8"
CID=bbd23a17
parentCID=ffffffff
createType="vmfs"
# Extent description
RW 167772160 VMFS "lnmt1cuomjs1-flat.vmdk"
# The Disk Data Base
#DDB
ddb.adapterType = "lsilogic"
ddb.geometry.cylinders = "10443"
ddb.geometry.heads = "255"
ddb.geometry.sectors = "63"
ddb.longContentID = "876a84261b8e8ba71481a111bbd23a17"
ddb.toolsInstallType = "4"
ddb.toolsVersion = "11269"
ddb.uuid = "60 00 C2 9c 1b 27 9b c2-8a a4 da a6 3e ae eb 89"
ddb.virtualHWVersion = "11"
Honestly, to me, these don’t mean a whole lot. Once I have a list of the vmdk files (initial, 1, 2, and 3), I can convert them. First I created a directory for the files in /opt/libvirt_images which is where I have all the pool files. Then ran the qemu-img commands to convert all the disk images.
qemu-img convert -f vmdk -O qcow2 /opt/vms/morgan/lnmt1cuomjs1/lnmt1cuomjs1.vmdk lnmt1cuomjs1.qcow2
qemu-img convert -f vmdk -O qcow2 /opt/vms/morgan/lnmt1cuomjs1/lnmt1cuomjs1_1.vmdk lnmt1cuomjs1_disk1.qcow2
qemu-img convert -f vmdk -O qcow2 /opt/vms/morgan/lnmt1cuomjs1/lnmt1cuomjs1_2.vmdk lnmt1cuomjs1_disk2.qcow2
qemu-img convert -f vmdk -O qcow2 /opt/vms/morgan/lnmt1cuomjs1/lnmt1cuomjs1_3.vmdk lnmt1cuomjs1_disk3.qcow2
Once everything is converted, you’ll need to install the main qcow2 file, then add the VMs.
You’ll have to get the domain created before you can attach the disks. To do that, you use virt-install.
virt-install --name lnmt1cuomjs1 --ram 4096 --vcpus 2 --disk lnmt1cuomjs1.qcow2,format=qcow2 --import
You can run a virsh list to see the domain once it’s created. Now attach the three disks to the domain.
virsh attach-disk lnmt1cuomjs1 /opt/libvirt_images/lnmt1cuomjs1_pool/lnmt1cuomjs1_disk1.qcow2 sdb --type disk --config
virsh attach-disk lnmt1cuomjs1 /opt/libvirt_images/lnmt1cuomjs1_pool/lnmt1cuomjs1_disk2.qcow2 sdc --type disk --config
virsh attach-disk lnmt1cuomjs1 /opt/libvirt_images/lnmt1cuomjs1_pool/lnmt1cuomjs1_disk3.qcow2 sdd --type disk --config
Run the qemu-img info command to verify the integrity of the new VM.
virt-filesystems -a /opt/libvirt_images/lnmt1cuomjs1_pool/lnmt1cuomjs1.qcow2
/dev/sda1
/dev/vg00/home
/dev/vg00/opt
/dev/vg00/root
/dev/vg00/tmp
/dev/vg00/usr
/dev/vg00/var
Update: This was interesting. After attaching the disks via the command line, I’d start a system and it’d fail, sitting at the maintenance prompt. I started virt-manager after starting the X Window system and a few of the systems I’d transferred, still weren’t done so I added the converted drive in the GUI vs using the virsh attach-disk command and low and behold, the system came up. I started going through the systems one at a time. The single drive systems, I was able to start without problem other than the Solaris one (I’ll just rebuild that one). Then I used virsh edit domain and removed the CLI added disk. Then in the GUI, added the disk back in. I was finding some of the SCSI drives were still failing so I changed them to IDE and the system came up. So no all the transferred systems have come up with no problem and I can either transfer them into the Proxmox cluster or for some, just copy the data over to a fresh installation. I do want to make all the systems current as most are running CentOS 7. Plus I want to use different Linux distros for different network zones.
