Sharing data between VMs is a headache every admin had once in a while. Especially when multiple virtual machines are expected to have read/write access. Besides the oldfashioned ways like NFS and the new shiny stuff like Ceph, the ‘simple’ sharing of a filesystem from host to VM fits in.

Although some solution can configure this with one click, I wanted to understand with which technology I can do this manually. I found virtiofs which allows as FUSE process on the host to directly pass filesystem access to a VIRTIO device inside the VM. This means not network protocol is involved, restricting the access to local VMs on the host. But they can use the hosts Page Cache if so chosen.

So let’s get to it:

virtiofsd is part of qemu-system-common on Debian or, if you use Proxmox like I did in this case, you can find it under /usr/lib/kvm/virtiofsd.

So we start two instances of the daemon sharing the same directory:

/usr/lib/kvm/virtiofsd --socket-path=/tmp/vm1 -o source=/root/multi-test -o cache=auto -o debug
/usr/lib/kvm/virtiofsd --socket-path=/tmp/vm2 -o source=/root/multi-test -o cache=auto -o debug

They open a socket which is now passed to the VM via the QEMU parameters. When using Proxmox, add the following lines to /etc/pve/nodes/$HOSTANME/qemu-server/$VMID.conf

args: -chardev socket,id=char0,path=/tmp/1 -device vhost-user-fs-pci,queue-size=1024,chardev=char0,tag=virtiofs_vm_1 \
      -m 4G -object memory-backend-file,id=mem,size=4G,mem-path=/dev/shm,share=on -numa node,memdev=mem

args: -chardev socket,id=char0,path=/tmp/1 -device vhost-user-fs-pci,queue-size=1024,chardev=char0,tag=virtiofs_vm_2 \
      -m 4G -object memory-backend-file,id=mem,size=4G,mem-path=/dev/shm,share=on -numa node,memdev=mem

tag=virtiofs_vm_X defines the name under which the device will be available inside the VM

See virtiofs howto for further details

Then poweroff and start the VM to enable the additional device.

If you’re running a Linux kernel >5.4 inside the VM, it supports virtiofs natively:

mount -t virtiofs virtiofs_vm_1 /mnt

Now the host filesystems is mounted to the VM.

The VM are not yet configured to access the hosts cache since this method is, yet, still experimental.