Re: [Libguestfs] ZFS-on-NBD

It has been a few years since I've worked on it but there were a number of gotchyas we had to overcome when running block devices in userspace and mounted on the same system. The largest of which is that the userland process needs to be extremely careful with memory allocations (ideally all memory will be allocated at startup of the userland driver and memlocked) because you can quite easily cause deadlock on the system if your userland block device driver needs to allocate memory at any point during an operation and the kernel decides it first needs to reclaim memory and chooses to clean out dirty filesystem buffer pages to satisfy the malloc. This causes the kernel to again try to re-enter the filesystem and block device kernel drivers to flush those dirty pages and that causes deadlocks in the kernel as locks were already being held by the previous operation.