[PATCH 0/3] Cleanups for ovirt 4.4
- Remove unneeded auth code that does not work with 4.4 - Remove trim support that never existed and not planned - Remove unsafe zero optimization Not tested yet, posting for early feedback. Nir Soffer (3): v2v: rvh-upload-plugin: Remove unneeded auth v2v: rhv-upload-plugin: Remove trim support v2v: rhv-upload-plugin: Remove unsafe highestwrite v2v/rhv-upload-plugin.py | 82 ++++++++-------------------------------- 1 file changed, 16 insertions(+), 66 deletions(-) -- 2.26.2
In ovirt < 4.2, when importing vm via the proxy, the proxy required Authorization header, with the contents of ImageTransfer.signed_ticket. In ovirt 4.4 ImageTransfer.signed_ticket was removed, and trying to access it will raise AttributeError. Remove the unneeded code. Signed-off-by: Nir Soffer <nsoffer@redhat.com> --- v2v/rhv-upload-plugin.py | 16 ---------------- 1 file changed, 16 deletions(-) diff --git a/v2v/rhv-upload-plugin.py b/v2v/rhv-upload-plugin.py index 90b3af82..4663d6cc 100644 --- a/v2v/rhv-upload-plugin.py +++ b/v2v/rhv-upload-plugin.py @@ -140,7 +140,6 @@ def open(readonly): 'can_flush': options['can_flush'], 'can_trim': options['can_trim'], 'can_zero': options['can_zero'], - 'needs_auth': options['needs_auth'], 'connection': connection, 'disk_id': disk.id, 'transfer': transfer, @@ -195,12 +194,8 @@ def request_failed(r, msg): @failing def pread(h, count, offset): http = h['http'] - transfer = h['transfer'] headers = {"Range": "bytes=%d-%d" % (offset, offset + count - 1)} - if h['needs_auth']: - headers["Authorization"] = transfer.signed_ticket - http.request("GET", h['path'], headers=headers) r = http.getresponse() @@ -216,14 +211,11 @@ def pread(h, count, offset): @failing def pwrite(h, buf, offset): http = h['http'] - transfer = h['transfer'] count = len(buf) h['highestwrite'] = max(h['highestwrite'], offset + count) http.putrequest("PUT", h['path'] + "?flush=n") - if h['needs_auth']: - http.putheader("Authorization", transfer.signed_ticket) # The oVirt server only uses the first part of the range, and the # content-length. http.putheader("Content-Range", "bytes %d-%d/*" % (offset, offset + count - 1)) @@ -277,7 +269,6 @@ def zero(h, count, offset, may_trim): def emulate_zero(h, count, offset): http = h['http'] - transfer = h['transfer'] # qemu-img convert starts by trying to zero/trim the whole device. # Since we've just created a new disk it's safe to ignore these @@ -285,8 +276,6 @@ def emulate_zero(h, count, offset): # After that we must emulate them with writes. if offset + count < h['highestwrite']: http.putrequest("PUT", h['path']) - if h['needs_auth']: - http.putheader("Authorization", transfer.signed_ticket) http.putheader("Content-Range", "bytes %d-%d/*" % (offset, offset + count - 1)) http.putheader("Content-Length", str(count)) @@ -721,8 +710,6 @@ def get_options(http, url): j = json.loads(data) features = j["features"] return { - # New imageio never used authentication. - "needs_auth": False, "can_flush": "flush" in features, "can_trim": "trim" in features, "can_zero": "zero" in features, @@ -733,9 +720,6 @@ def get_options(http, url): # Old imageio servers returned either 405 Method Not Allowed or # 204 No Content (with an empty body). return { - # Authentication was required only when using old imageio proxy. - # Can be removed when dropping support for oVirt < 4.2. - "needs_auth": not params['rhv_direct'], "can_flush": False, "can_trim": False, "can_zero": False, -- 2.26.2
imageio never supported trim and there are no plans to support it. Remove the code to detect this feature and use it. Signed-off-by: Nir Soffer <nsoffer@redhat.com> --- v2v/rhv-upload-plugin.py | 31 ++----------------------------- 1 file changed, 2 insertions(+), 29 deletions(-) diff --git a/v2v/rhv-upload-plugin.py b/v2v/rhv-upload-plugin.py index 4663d6cc..ad5642f9 100644 --- a/v2v/rhv-upload-plugin.py +++ b/v2v/rhv-upload-plugin.py @@ -131,14 +131,13 @@ def open(readonly): cancel_transfer(connection, transfer) raise - debug("imageio features: flush=%(can_flush)r trim=%(can_trim)r " + debug("imageio features: flush=%(can_flush)r " "zero=%(can_zero)r unix_socket=%(unix_socket)r" % options) # Save everything we need to make requests in the handle. return { 'can_flush': options['can_flush'], - 'can_trim': options['can_trim'], 'can_zero': options['can_zero'], 'connection': connection, 'disk_id': disk.id, @@ -152,7 +151,7 @@ def open(readonly): @failing def can_trim(h): - return h['can_trim'] + return False @failing @@ -299,30 +298,6 @@ def emulate_zero(h, count, offset): r.read() -@failing -def trim(h, count, offset): - http = h['http'] - - # Construct the JSON request for trimming. - buf = json.dumps({'op': "trim", - 'offset': offset, - 'size': count, - 'flush': False}).encode() - - headers = {"Content-Type": "application/json", - "Content-Length": str(len(buf))} - - http.request("PATCH", h['path'], body=buf, headers=headers) - - r = http.getresponse() - if r.status != 200: - request_failed(r, - "could not trim sector offset %d size %d" % - (offset, count)) - - r.read() - - @failing def flush(h): http = h['http'] @@ -711,7 +686,6 @@ def get_options(http, url): features = j["features"] return { "can_flush": "flush" in features, - "can_trim": "trim" in features, "can_zero": "zero" in features, "unix_socket": j.get('unix_socket'), } @@ -721,7 +695,6 @@ def get_options(http, url): # 204 No Content (with an empty body). return { "can_flush": False, - "can_trim": False, "can_zero": False, "unix_socket": None, } -- 2.26.2
We had a mechanism to ignore zero writes if the written area in the disk was never written. This is safe when working sparse files which are always zeroed. When working with block devices, we don't have any guarantee on the contents of the device, so we should not skip zero requests. Signed-off-by: Nir Soffer <nsoffer@redhat.com> --- v2v/rhv-upload-plugin.py | 49 +++++++++++++++++----------------------- 1 file changed, 21 insertions(+), 28 deletions(-) diff --git a/v2v/rhv-upload-plugin.py b/v2v/rhv-upload-plugin.py index ad5642f9..e261dfdb 100644 --- a/v2v/rhv-upload-plugin.py +++ b/v2v/rhv-upload-plugin.py @@ -143,7 +143,6 @@ def open(readonly): 'disk_id': disk.id, 'transfer': transfer, 'failed': False, - 'highestwrite': 0, 'http': http, 'path': destination_url.path, } @@ -212,7 +211,6 @@ def pwrite(h, buf, offset): http = h['http'] count = len(buf) - h['highestwrite'] = max(h['highestwrite'], offset + count) http.putrequest("PUT", h['path'] + "?flush=n") # The oVirt server only uses the first part of the range, and the @@ -269,33 +267,28 @@ def zero(h, count, offset, may_trim): def emulate_zero(h, count, offset): http = h['http'] - # qemu-img convert starts by trying to zero/trim the whole device. - # Since we've just created a new disk it's safe to ignore these - # requests as long as they are smaller than the highest write seen. - # After that we must emulate them with writes. - if offset + count < h['highestwrite']: - http.putrequest("PUT", h['path']) - http.putheader("Content-Range", - "bytes %d-%d/*" % (offset, offset + count - 1)) - http.putheader("Content-Length", str(count)) - http.endheaders() + http.putrequest("PUT", h['path']) + http.putheader("Content-Range", + "bytes %d-%d/*" % (offset, offset + count - 1)) + http.putheader("Content-Length", str(count)) + http.endheaders() - try: - buf = bytearray(128 * 1024) - while count > len(buf): - http.send(buf) - count -= len(buf) - http.send(memoryview(buf)[:count]) - except BrokenPipeError: - pass - - r = http.getresponse() - if r.status != 200: - request_failed(r, - "could not write zeroes offset %d size %d" % - (offset, count)) - - r.read() + try: + buf = bytearray(128 * 1024) + while count > len(buf): + http.send(buf) + count -= len(buf) + http.send(memoryview(buf)[:count]) + except BrokenPipeError: + pass + + r = http.getresponse() + if r.status != 200: + request_failed(r, + "could not write zeroes offset %d size %d" % + (offset, count)) + + r.read() @failing -- 2.26.2
ACK series, thanks. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org
On Thu, Jan 21, 2021 at 8:07 PM Nir Soffer <nirsof@gmail.com> wrote:
- Remove unneeded auth code that does not work with 4.4 - Remove trim support that never existed and not planned - Remove unsafe zero optimization
Not tested yet, posting for early feedback.
Tested now with ovirt 4.4.
Nir Soffer (3): v2v: rvh-upload-plugin: Remove unneeded auth v2v: rhv-upload-plugin: Remove trim support v2v: rhv-upload-plugin: Remove unsafe highestwrite
v2v/rhv-upload-plugin.py | 82 ++++++++-------------------------------- 1 file changed, 16 insertions(+), 66 deletions(-)
-- 2.26.2
By the way if you happen to be looking at the Python plugin, then there are two changes that may be worth using. API_VERSION = 2: https://libguestfs.org/nbdkit-python-plugin.3.html#API-versions and multi-thread support: https://libguestfs.org/nbdkit-python-plugin.3.html#Threads These were both added (by you) and are both available in nbdkit >= 1.22, which is now available in RHEL AV >= 8.3.0 so there's no reason why we can't use them now. I don't think there's any advantage to API_VERSION = 2 because the plugin is very (exclusively?) write-heavy. But a more parallel thread model is surely advantageous? For API_VERSION = 2, the changes are: * pread signature changes to: def pread(h, buf, offset, flags) -> returns nothing - You have to write the result directly into the buf parameter, avoiding the extra copy. * pwrite adds flags parameter: def pwrite(h, buf, offset, flags) -> returns nothing * flush, trim also add flags parameter. * zero changes signature completely, removing the may_trim parameter and adding a general flags parameter in its place. For threads: * Add a thread_model() function with no parameters which returns nbdkit.THREAD_MODEL_PARALLEL or nbdkit.THREAD_MODEL_SERIALIZE_REQUESTS. Of course locking will need to be introduced (especially for PARALLEL) to protect internal data structures. See nbdkit-plugin(3) for an explanation of the thread models. You'll also have to change the minimum version of nbdkit so that older versions get rejected, here: https://github.com/libguestfs/virt-v2v/blob/1d5fc257765c444644e5bfc6525e86ff... Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org
On Fri, Jan 22, 2021 at 10:49 AM Richard W.M. Jones <rjones@redhat.com> wrote:
By the way if you happen to be looking at the Python plugin, then there are two changes that may be worth using.
API_VERSION = 2:
https://libguestfs.org/nbdkit-python-plugin.3.html#API-versions
and multi-thread support:
https://libguestfs.org/nbdkit-python-plugin.3.html#Threads
These were both added (by you) and are both available in nbdkit >= 1.22, which is now available in RHEL AV >= 8.3.0 so there's no reason why we can't use them now.
Great I don't think there's any advantage to API_VERSION = 2 because the
plugin is very (exclusively?) write-heavy. But a more parallel thread model is surely advantageous?
I think we can change the rhv upload plugin to use this approach: plugins/python/examples/imageio.py Based on the tests I did with the imageio plugin this should give significant performance improvement. https://github.com/libguestfs/nbdkit/commit/78bbe95f754dd404fcbabac57940b1c0... I think the changes are basically: api version: 2 thread model: parallel open: - Create a connection pool instead of a single connection. - Based on experience wit imageio client, 4 connections are a good default pwrite/zero/pread: - Get connection from the pool, blocking for available instance - Return the connection to the pool when done or after failure flush: - send a flush request for all connections in the pool (each http connection have a separate nbd connection inside imageio) close: - close all connections in pool What do you think? For API_VERSION = 2, the changes are:
* pread signature changes to:
def pread(h, buf, offset, flags) -> returns nothing
- You have to write the result directly into the buf parameter, avoiding the extra copy.
* pwrite adds flags parameter:
def pwrite(h, buf, offset, flags) -> returns nothing
* flush, trim also add flags parameter.
* zero changes signature completely, removing the may_trim parameter and adding a general flags parameter in its place.
For threads:
* Add a thread_model() function with no parameters which returns nbdkit.THREAD_MODEL_PARALLEL or nbdkit.THREAD_MODEL_SERIALIZE_REQUESTS. Of course locking will need to be introduced (especially for PARALLEL) to protect internal data structures. See nbdkit-plugin(3) for an explanation of the thread models.
You'll also have to change the minimum version of nbdkit so that older versions get rejected, here:
https://github.com/libguestfs/virt-v2v/blob/1d5fc257765c444644e5bfc6525e86ff...
Rich.
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org
On Fri, Jan 22, 2021 at 05:11:46PM +0200, Nir Soffer wrote:
Based on the tests I did with the imageio plugin this should give significant performance improvement. https://github.com/libguestfs/nbdkit/commit/ 78bbe95f754dd404fcbabac57940b1c0e4e49c4d
I think the changes are basically:
api version: 2 thread model: parallel
open: - Create a connection pool instead of a single connection. - Based on experience wit imageio client, 4 connections are a good default
pwrite/zero/pread: - Get connection from the pool, blocking for available instance - Return the connection to the pool when done or after failure
flush: - send a flush request for all connections in the pool (each http connection have a separate nbd connection inside imageio)
close: - close all connections in pool
What do you think?
Yes. The tricky thing will be getting locking right. With the PARALLEL model you can have multiple parallel calls into the plugin even with the same handle. The Python GIL is still active, so Python code fragments won't be running at the same time, but it's still possible for one thread to block in the middle of a plugin method (eg. HTTPConnection blocking on a read), releasing the GIL, allowing another plugin method call to start with the same handle. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org
On Fri, Jan 22, 2021 at 5:42 PM Richard W.M. Jones <rjones@redhat.com> wrote:
On Fri, Jan 22, 2021 at 05:11:46PM +0200, Nir Soffer wrote:
Based on the tests I did with the imageio plugin this should give significant performance improvement. https://github.com/libguestfs/nbdkit/commit/ 78bbe95f754dd404fcbabac57940b1c0e4e49c4d
I think the changes are basically:
api version: 2 thread model: parallel
open: - Create a connection pool instead of a single connection. - Based on experience wit imageio client, 4 connections are a good default
pwrite/zero/pread: - Get connection from the pool, blocking for available instance - Return the connection to the pool when done or after failure
flush: - send a flush request for all connections in the pool (each http connection have a separate nbd connection inside imageio)
close: - close all connections in pool
What do you think?
Yes.
The tricky thing will be getting locking right. With the PARALLEL model you can have multiple parallel calls into the plugin even with the same handle. The Python GIL is still active, so Python code fragments won't be running at the same time, but it's still possible for one thread to block in the middle of a plugin method (eg. HTTPConnection blocking on a read), releasing the GIL, allowing another plugin method call to start with the same handle.
The handle is a dict, which is thread safe, and we never modify the handle during a transfer. The only change we do is set h["failed"] to True, which is thread safe and idempotent. To get a connection from the handle, we will use a queue which is thread safe. Connection are removed from the queue during use, so only one thread can access a connection. Returning connection to the queue is also thread safe. So I think no extra locking is needed.
Rich.
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org
On Fri, Jan 22, 2021 at 06:15:38PM +0200, Nir Soffer wrote:
On Fri, Jan 22, 2021 at 5:42 PM Richard W.M. Jones <rjones@redhat.com> wrote:
On Fri, Jan 22, 2021 at 05:11:46PM +0200, Nir Soffer wrote: > Based on the tests I did with the imageio plugin this should > give significant performance improvement. > https://github.com/libguestfs/nbdkit/commit/ > 78bbe95f754dd404fcbabac57940b1c0e4e49c4d > > I think the changes are basically: > > api version: 2 > thread model: parallel > > open: > - Create a connection pool instead of a single connection. > - Based on experience wit imageio client, 4 connections are a good default > > pwrite/zero/pread: > - Get connection from the pool, blocking for available instance > - Return the connection to the pool when done or after failure > > flush: > - send a flush request for all connections in the pool > (each http connection have a separate nbd connection inside imageio) > > close: > - close all connections in pool > > What do you think?
Yes.
The tricky thing will be getting locking right. With the PARALLEL model you can have multiple parallel calls into the plugin even with the same handle. The Python GIL is still active, so Python code fragments won't be running at the same time, but it's still possible for one thread to block in the middle of a plugin method (eg. HTTPConnection blocking on a read), releasing the GIL, allowing another plugin method call to start with the same handle.
The handle is a dict, which is thread safe, and we never modify the handle during a transfer.
The only change we do is set h["failed"] to True, which is thread safe and idempotent.
To get a connection from the handle, we will use a queue which is thread safe. Connection are removed from the queue during use, so only one thread can access a connection. Returning connection to the queue is also thread safe.
So I think no extra locking is needed.
Sounds good! Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW
participants (3)
-
Nir Soffer -
Nir Soffer -
Richard W.M. Jones