Now that Xen is slated for total removal from unRaid, it was time to seriously look at migrating to KVM. The easiest solution would probably have been to start from scratch and re-create all my VMs under KVM but that would require reactivating windows VMs, downloading all the updates, re-installing and re-configuring software etc.
The more attractive route was finding a way to migrate existing VMs from Xen to KVM. Thanks to jonp from LimeTech who published a how-to guide, this become fairly straightforward. I even recorded myself following the guide. Having done my Windows 7 VM, I then set about two Windows 8.1 systems and a Netrunner (ArchLinux / Manjaro) VM.
The Windows 8.1 migrations were as straightforward as the Win7 one had been. However, the Linux one just wouldn’t work for me. Once I’d set it up in KVM, pointed to the disk image and booted, I was greeted with a Grub error indicating that the root disk UUID could not be found. A quick check indicated that the UUID was in fact the same and no amount of Google Fu revealed a solution. I ended up re-installing the OS and dealing with a world of hurt in the updates (lot’s of conflicting dependencies).
So, with 4x VMs migrated, it was time to start playing with my shiny new KVM flavour of UnRaid. With a full weekend of playing around under my belt, I can now report that some things worked well, some didn’t. Some things I liked, some not so much;
Things that went well
The new VM Manager in unRaid v6b15 is a great start. It makes setting up and managing VMs pretty straightforward and has plenty of configuration options. It’s still a work in progress, however, with a couple of manageable bugs and plans to add enhanced features such as support for TV Tuner passthrough detection.
Things that went not so well
Some or all of the following may be attributable to me being on a learning curve but they’re the issues that caused me to be stuck in from of computer screens for the weekend rather than out enjoying the decent weather, gardening, spending time with the kids, knitting etc…
out of memory issue
I came across a particularly nasty issue at one point where the entire system seemed to run out of memory and slowed to a crawl. I’d just booted into one particular VM when everything ground to a halt. VMS were unresponsive, SSH access to unRAID took ages to establish and respond, the unRaid web UI took minutes to load. In the end, I had to hard-reset the system. I suspect a memory issue as I managed to open the log and there was one error line in there relating to out of memory on the KVM plug-in. Very odd as the sum-total of VM allotted memory was well below my 32GB total RAM. This happened once only but I’ll need to keep an eye on it.
standby
I inadvertently put a VM into standby and couldn’t get it back up. No amount of poking the attached keyboard or clicking the attached mouse would work. The VM Manager showed just two options - play and save to disk. I ended up issuing a virsh destroy command and again rebooting the entire server.
losing configs
The single most frustrating aspect I encountered was the way in which changing a VM config via the ‘edit domain’ icon (pencil) would totally re-write the domain XML, obliterating any custom passthrough that had been set up.
Now, I’m aware that custom passthrough of non-listed devices is experimental and unsupported, but it’s hyper-annoying that any user-added <hostdev> blocks are erased, requiring those to be set up again and again. In the end, I found myself editing the XML directly rather than deal with the frustration.
Sound passthrough
Under Xen, I had a USB controller passed through to mu Linux VM to which I’d attached a mouse, keyboard and pair of soundsticks USB speakers. I’d usually pass USB devices through discretely but under Xen, when I did so, the performance of the speakers was poor so controller passthrough was the only option (and worked perfectly).
Under KVM, I attempted USB passthrough by nominating the devices in VM Manager but I encountered similar issues with the sound (crackles and pops) plus the mouse would not work at all, though the keyboard was fine.
I then used the non-gpu passthrough procedure to pass through the entire USB controller. This caused the mouse and keyboard to work OK (but not until after boot so I couldn’t make changes at the Grub menu) and while the speakers worked better, they were still not perfect. Xen was better for this.
Interestingly, on Xen, I could pass through just the OHCI component of the USB controller (13.0.2) but KVM insisted that both PCI IDs be included (13.0.0 / 13.0.2). Also interesting, once set up, this VM seems to boot a lot faster than it did under Xen.
IP Address
When setting up KVM and VMs, I didn’t specify a network bridge. VMS booted OK and had network and internet access. However, I like to set up static IPs on my VMs to allow me connect to services they are running and VNC into them. When I attempted to set up a strict IP on the windows VMs, it just wouldn’t work. It was necessary for me to specify my old xenbr0 bridge in settings and for each VM for this to work as expected. In the end, I got all the VMs running on static IPs.
BluRay passthrough
One of my VMs had a USB 3.0 controller passthrough to allow connection of an external BluRay drive. This worked really well under Xen but when I migrated this to KVM, the drive would show up for a couple of minutes and then disappear. Checking the Device Manager, I would find a USB Storage Controller entry with a warning icon (device could not start code 10). This is an open issue, I haven’t solved it yet.
Passthrough Woes
I have several VMs and lots of devices passed through to them (PCIe cards, PCI cards, USB controllers, usb devices). My aim was to replicate this setup under KVM. Initially, I thought I’d hit a brick wall as the first device I attempted failed with KVM informing me that really I should be passing through all 8 items in the IOMMU group. A quick chat with jonp and we determined that I needed to switch on PCIe ACS override and that freed things up a bit.
I’ve managed to pass through most of what I had before, with the exception of a bluetooth receiver which stubbornly refuses to co-operate.
Slow boot on one vm
One of my Windows 8.1 VMs is super-slow to boot and I cannot figure out why. The other one with pretty much the same config and very similar disk image is super quick. I even copied the fast-booting image and pointed the slow-booted VM to that to no avail - still super slow. This is another unresolved issue that I need to investigate further.
Creating Disk Images
In the set-up process, I attempted to use the VM UI to create a disk image (set size, format and location). In all cases, no matter what I tried, the system never created the disk image.
So, all in all, more problems than solutions. Though, in fairness, many of the problems have been resolved one way or another. I’ll probably lose another weekend to getting this all running correctly and I’m pretty confident I’ll get back to more or less exactly where I was - it’s just a pity I’ve got to sink this time into it just to stand still.
No comments:
Post a Comment