Windows GPU gaming on Linux¶
Gaming on Linux with virtual machine running Windows takes a little bit of work, specifically on how to make the GPU visible to windows. In practice, you’ll need two GPUs (one for Linux, one for the Windows virtual machine (VM)). This guide explains how make a GPU visible to the Windows VM.
GPU passthrough is in fact simple¶
I’ve read many guides, and each time something is missing or I need to install some extra stack which I don’t understand. And I thought to myself why does it have to be so hard? Eventually, after days of toying with it, I got everything to work on Mint 18. I feel I learned a lot in process and it’s not so complicated.
A minimal approach¶
Let’s just stick to the basics, the only things described here will be to configure your Linux install so that KVM/QEMU can access a GPU (or some other device) in passthrough mode.
Check hardware requirements, you need a CPU that supports virtualization (VT-d). See the links at the end for hardware help, virtual machine setup etc…
It just takes 3 steps:
Setup GRUB to enable IOMMU in your system.
Find the GPU/devices you want to linux to set aside for VM passthrough.
Configure modprobe and initramsfs
And that’s all really. So let’s start.
Configure GRUB¶
sudo vi /etc/default/grub
Add modify the line GRUB_CMDLINE_LINUX to
GRUB_CMDLINE_LINUX="intel_iommu=on iommu=pt"
Note for AMD users: use amd_iommu=on.
Now update grub:
sudo update-grub
Reboot¶
After reboot iommu groups will be available.
Identify the devices you want to passthrough¶
lspci -k | grep -i -A 3 vga
Sample output
03:00.0 VGA compatible controller: NVIDIA Corporation Device 1b06 (rev a1)
Subsystem: eVga.com. Corp. Device 6598
Kernel driver in use: nvidia
Kernel modules: nvidiafb, nouveau, nvidia_387_drm, nvidia_387
03:00.1 Audio device: NVIDIA Corporation Device 10ef (rev a1)
Subsystem: eVga.com. Corp. Device 6598
Kernel driver in use: snd_hda_intel
Kernel modules: snd_hda_intel
04:00.0 VGA compatible controller: NVIDIA Corporation Device 1b06 (rev a1)
Subsystem: eVga.com. Corp. Device 6598
Kernel driver in use: nvidia
Kernel modules: nvidiafb, nouveau, nvidia_387_drm, nvidia_387
04:00.1 Audio device: NVIDIA Corporation Device 10ef (rev a1)
Subsystem: eVga.com. Corp. Device 6598
Kernel driver in use: snd_hda_intel
Kernel modules: snd_hda_intel
Let’s say we want to pass 04:00.0 for video and 04.00.1 for nvidia audio to the VM.
Configuring modprobe and initramfs¶
Passthrough script¶
Create the file with an editor
sudo vi /sbin/vfio-pci-override-vga.sh
Then add the following content:
#!/bin/sh
# Replace with your devices below.
DEVS="04:00.0 04:00.1"
for DEV in $DEVS; do
echo "vfio-pci" > /sys/bus/pci/devices/0000:$DEV/driver_override
done
modprobe -i vfio-pci
Configure modprobe¶
Create the file with an editor
sudo vi /etc/modprobe.d/local.conf
And add the following contents to it
install vfio-pci /sbin/vfio-pci-override-vga.sh
Configuring initramfs¶
Add modules by editing the following file:
sudo vi /etc/initramfs-tools/modules
And add the following 4 lines at the end:
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd
Add the passthrough script by creating the following file:
sudo vi /etc/initramfs-tools/hooks/vfio-pci-override-vga
And add the following contents:
#! /bin/bash
. /usr/share/initramfs-tools/hook-functions
copy_exec /sbin/vfio-pci-override-vga.sh sbin/vfio-pci-override-vga.sh
And finally update initramfs
sudo update-initramfs -u
Verifying the contents¶
In my case, it generated a file called /boot/initrd.img-4.13.0-38-generic, replace with the one for you.
sudo lsinitramfs -l /boot/initrd.img-4.13.0-38-generic | grep -i vfio
The output should look like this:
-rwxr-xr-x 1 root root 158 Apr 14 22:46 sbin/vfio-pci-override-vga.sh
drwxr-xr-x 4 root root 0 Apr 15 11:26 lib/modules/4.13.0-38-generic/kernel/drivers/vfio
-rw-r--r-- 1 root root 9830 Mar 14 12:53 lib/modules/4.13.0-38-generic/kernel/drivers/vfio/vfio_virqfd.ko
-rw-r--r-- 1 root root 27838 Mar 14 12:53 lib/modules/4.13.0-38-generic/kernel/drivers/vfio/vfio_iommu_type1.ko
drwxr-xr-x 2 root root 0 Apr 15 11:26 lib/modules/4.13.0-38-generic/kernel/drivers/vfio/mdev
-rw-r--r-- 1 root root 7766 Mar 14 12:53 lib/modules/4.13.0-38-generic/kernel/drivers/vfio/mdev/vfio_mdev.ko
-rw-r--r-- 1 root root 26142 Mar 14 12:53 lib/modules/4.13.0-38-generic/kernel/drivers/vfio/mdev/mdev.ko
drwxr-xr-x 2 root root 0 Apr 15 11:26 lib/modules/4.13.0-38-generic/kernel/drivers/vfio/pci
-rw-r--r-- 1 root root 74518 Mar 14 12:53 lib/modules/4.13.0-38-generic/kernel/drivers/vfio/pci/vfio-pci.ko
-rw-r--r-- 1 root root 47182 Mar 14 12:53 lib/modules/4.13.0-38-generic/kernel/drivers/vfio/vfio.ko
Reboot¶
Now vfio is configured and your devices should be ready for passthrough, this can be verified with:
03:00.0 VGA compatible controller: NVIDIA Corporation Device 1b06 (rev a1)
Subsystem: eVga.com. Corp. Device 6598
Kernel driver in use: nvidia
Kernel modules: nvidiafb, nouveau, nvidia_387_drm, nvidia_387
03:00.1 Audio device: NVIDIA Corporation Device 10ef (rev a1)
Subsystem: eVga.com. Corp. Device 6598
Kernel driver in use: snd_hda_intel
Kernel modules: snd_hda_intel
04:00.0 VGA compatible controller: NVIDIA Corporation Device 1b06 (rev a1)
Subsystem: eVga.com. Corp. Device 6598
Kernel driver in use: vfio-pci
Kernel modules: nvidiafb, nouveau, nvidia_387_drm, nvidia_387
04:00.1 Audio device: NVIDIA Corporation Device 10ef (rev a1)
Subsystem: eVga.com. Corp. Device 6598
Kernel driver in use: vfio-pci
Kernel modules: snd_hda_intel
Note that now the value says vfio-pci for Kernel driver in use.
And voila¶
It’s really simple in the end. Setting up the virtual machine should be straightforward but find some links in the next section to help you out.
In the end, I installed Windows inside a virtual machine. With KVM/QEMU configured to use the GPU I configured to be in passthrough mode, I was able to install nVidia’s driver. Being playing The Witcher 3 at 70 FPS in Ultra details at 1440p with a GTX1080-TI, all of this in my simple little virtual machine. As to the virtual machine, I kept it entirely basic: everything was default except for passing the GPU directly of course (no hugepages, used the default qcow2 drive format, no cpu assignment).
Resources¶
I didn’t come up with everything in this guide, I just simplified what I found in various guides. These guides are more thorough and more complex, so if you encounter problem/difficulties you might find answers there. Here are the links:
powerhouse’ HOW-TO make dual-boot obsolete using kvm VGA passthrough. Discusses Mint setup, hardware requirements. And it goes in depth to optimize other aspects of VMs (such as hugepages). In addition it tells how start a VM, with a lot of options. Lot of information but it can be overwhelming.
Alex Williamson series of VFIO. This is another goldmine of information. It also goes quite in depth in a many aspect. The setup is for Fedora but shares a lot in common with Mint anyway. The author discusses hardware requirements, VM setups and advanced configurations beyong GPU passthrough (which may not be really needed). I like the screenshots of the setting up QEMU/KVM, it made it easy to follow for me.
So thanks to both these authors, I hope you’ll find this blog post useful and clear.