Multi-conn is enabled only when we know the connection is read-only: $ ./nbdkit -r curl file:/var/tmp/jammy-server-cloudimg-amd64.raw --run ' nbdinfo $uri ' | grep can_multi_conn can_multi_conn: true $ ./nbdkit curl file:/var/tmp/jammy-server-cloudimg-amd64.raw --run ' nbdinfo $uri ' | grep can_multi_conn can_multi_conn: false This improves performance. Comparing before and after this commit shows approximate doubling of performance: Benchmark 1: nbdkit -r curl file:/var/tmp/jammy-server-cloudimg-amd64.raw --run "nbdcopy -p \$uri null:" Time (mean ± σ): 943.8 ms ± 18.8 ms [User: 316.2 ms, System: 1029.7 ms] Range (min … max): 923.7 ms … 989.2 ms 10 runs Benchmark 2: ~/d/nbdkit/nbdkit -r curl file:/var/tmp/jammy-server-cloudimg-amd64.raw --run "nbdcopy -p \$uri null:" Time (mean ± σ): 455.0 ms ± 6.2 ms [User: 542.2 ms, System: 1824.7 ms] Range (min … max): 449.1 ms … 471.6 ms 10 runs Summary ' ~/d/nbdkit/nbdkit -r curl file:/var/tmp/jammy-server-cloudimg-amd64.raw --run "nbdcopy -p \$uri null:" ' ran 2.07 ± 0.05 times faster than ' nbdkit -r curl file:/var/tmp/jammy-server-cloudimg-amd64.raw --run "nbdcopy -p \$uri null:" ' See also: https://listman.redhat.com/archives/libguestfs/2023-February/030581.html --- plugins/curl/curldefs.h | 1 + plugins/curl/curl.c | 14 ++++++++++++++ 2 files changed, 15 insertions(+) diff --git a/plugins/curl/curldefs.h b/plugins/curl/curldefs.h index a2c521c50..16b6071cc 100644 --- a/plugins/curl/curldefs.h +++ b/plugins/curl/curldefs.h @@ -68,6 +68,7 @@ extern const char *user_agent; /* The per-connection handle. */ struct curl_handle { CURL *c; + int readonly; bool accept_range; int64_t exportsize; char errbuf[CURL_ERROR_SIZE]; diff --git a/plugins/curl/curl.c b/plugins/curl/curl.c index 42aba4216..2f34d1dc1 100644 --- a/plugins/curl/curl.c +++ b/plugins/curl/curl.c @@ -465,6 +465,7 @@ curl_open (int readonly) nbdkit_error ("calloc: %m"); return NULL; } + h->readonly = readonly; h->c = curl_easy_init (); if (h->c == NULL) { @@ -781,6 +782,18 @@ curl_get_size (void *handle) return h->exportsize; } +/* Multi-conn is safe for read-only connections, but HTTP does not + * have any concept of flushing so we cannot use it for read-write + * connections. + */ +static int +curl_can_multi_conn (void *handle) +{ + struct curl_handle *h = handle; + + return !! h->readonly; +} + /* NB: The terminology used by libcurl is confusing! * * WRITEFUNCTION / write_cb is used when reading from the remote server @@ -924,6 +937,7 @@ static struct nbdkit_plugin plugin = { .open = curl_open, .close = curl_close, .get_size = curl_get_size, + .can_multi_conn = curl_can_multi_conn, .pread = curl_pread, .pwrite = curl_pwrite, }; -- 2.39.0