Sharing filesystems with virtiofs between multiple VMs
Sharing data between VMs is a headache every admin had once in a while. Especially when multiple virtual machines are expected to have read/write access. Besides the oldfashioned ways like NFS and the new shiny stuff like Ceph, the ‘simple’ sharing of a filesystem from host to VM fits in.
Although some solution can configure this with one click, I wanted to understand with which technology I can do this manually. I found virtiofs which allows as FUSE process on the host to directly pass filesystem access to a VIRTIO device inside the VM. This means not network protocol is involved, restricting the access to local VMs on the host. But they can use the hosts Page Cache if so choosen.
So let’s get to it:
So we start two instances of the daemon sharing the same directory:
/usr/lib/kvm/virtiofsd --socket-path=/tmp/vm1 -o source=/root/multi-test -o cache=auto -o debug /usr/lib/kvm/virtiofsd --socket-path=/tmp/vm2 -o source=/root/multi-test -o cache=auto -o debug
They open a socket which is now passed to the VM via the QEMU parameters.
When using Proxmox, add the following lines to
args: -chardev socket,id=char0,path=/tmp/1 -device vhost-user-fs-pci,queue-size=1024,chardev=char0,tag=virtiofs_vm_1 \ -m 4G -object memory-backend-file,id=mem,size=4G,mem-path=/dev/shm,share=on -numa node,memdev=mem
args: -chardev socket,id=char0,path=/tmp/1 -device vhost-user-fs-pci,queue-size=1024,chardev=char0,tag=virtiofs_vm_2 \ -m 4G -object memory-backend-file,id=mem,size=4G,mem-path=/dev/shm,share=on -numa node,memdev=mem
tag=virtiofs_vm_X defines the name under which the device will be available inside the VM
See virtiofs howto for further details
Then poweroff and start the VM to enable the additional device.
If you’re running a Linux kernel >5.4 inside the VM, it supports virtiofs natively:
mount -t virtiofs virtiofs_vm_1 /mnt
Now the host filesystems is mounted to the VM.
The VM are not yet configured to access the hosts cache since this method is, yet, still experimental.