This was much simpler than I though, merely changing fuse_loop ->
fuse_loop_mt. FUSE itself will call our operations in parallel, so we
have to be careful about shared state.
Testing with fio shows this is about the same or a tiny bit slower
than before, which is not that surprising -- we are using the
synchronous API. It should be possible to improve performance either
by issuing commands in parallel on the same handle (complicated) or by
using multi-conn.
test.fio contains:
[global]
filename=test.img
bs=4k
direct=1
rw=read
[job1]
[job2]
[job3]
[job4]
Run the test:
$ touch test.img
$ nbdfuse test.img [ nbdkit memory 1G ] &
$ fio test.fio
$ fusermount3 -u test.img
Without this patch:
READ: bw=239MiB/s (250MB/s), 59.6MiB/s-59.7MiB/s (62.5MB/s-62.6MB/s), io=4096MiB
(4295MB), run=17166-17167msec
With this patch:
READ: bw=237MiB/s (248MB/s), 59.2MiB/s-59.2MiB/s (62.0MB/s-62.1MB/s), io=4096MiB
(4295MB), run=17303-17310msec
---
fuse/nbdfuse.c | 6 +++++-
fuse/operations.c | 7 +++++++
2 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/fuse/nbdfuse.c b/fuse/nbdfuse.c
index 3deef75..fa35080 100644
--- a/fuse/nbdfuse.c
+++ b/fuse/nbdfuse.c
@@ -151,6 +151,7 @@ main (int argc, char *argv[])
int64_t ssize;
const char *s;
struct fuse_args fuse_args = FUSE_ARGS_INIT (0, NULL);
+ struct fuse_loop_config fuse_loop_config;
FILE *fp;
for (;;) {
@@ -475,7 +476,10 @@ main (int argc, char *argv[])
}
/* Enter the main loop. */
- r = fuse_loop (fuse);
+ memset (&fuse_loop_config, 0, sizeof fuse_loop_config);
+ fuse_loop_config.clone_fd = 0;
+ fuse_loop_config.max_idle_threads = 10;
+ r = fuse_loop_mt (fuse, &fuse_loop_config);
if (r < 0) {
errno = -r;
perror ("fuse_loop");
diff --git a/fuse/operations.c b/fuse/operations.c
index 2c38c6e..4da701e 100644
--- a/fuse/operations.c
+++ b/fuse/operations.c
@@ -16,6 +16,13 @@
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*/
+/* FUSE operations invoked by the kernel.
+ *
+ * Note these may be called in parallel from multiple threads, so any
+ * shared state needs to be read-only or else protected by mutexes.
+ * libnbd calls are OK.
+ */
+
#include <config.h>
#include <stdio.h>
--
2.31.1