6:21 p.m.
On Thu, Jun 14, 2018 at 10:19 PM Richard W.M. Jones <rjones(a)redhat.com>
wrote:
On Thu, Jun 14, 2018 at 09:16:01PM +0300, Nir Soffer wrote:
> When sending request with small or no payload, it is simpler and
> possibly more efficient to use the high level HTTPSConnection.request(),
> instead of the lower level APIs.
>
> The only reason to use the lower level APIs is to avoid copying the
> payload, or on python 2, to use a bigger buffer size when streaming a
> file-like object.
Change seems quite straightforward. I'll have to test this out on my
own oVirt system first before I can properly verify it.
I tested the same change with imageio example upload script, and
it *doubled* the throughput.
See https://gerrit.ovirt.org/#/c/92275/
Thanks, Rich.
> v2v/rhv-upload-plugin.py | 35 ++++++++++++++++-------------------
> 1 file changed, 16 insertions(+), 19 deletions(-)
>
> diff --git a/v2v/rhv-upload-plugin.py b/v2v/rhv-upload-plugin.py
> index ed99cc7a9..3fad865f6 100644
> --- a/v2v/rhv-upload-plugin.py
> +++ b/v2v/rhv-upload-plugin.py
> @@ -263,12 +263,12 @@ def pread(h, count, offset):
> transfer = h['transfer']
> transfer_service = h['transfer_service']
>
> - http.putrequest("GET", h['path'])
> + headers = {"Range", "bytes=%d-%d" % (offset,
offset+count-1)}
> # Authorization is only needed for old imageio.
> if h['needs_auth']:
> - http.putheader("Authorization", transfer.signed_ticket)
> - http.putheader("Range", "bytes=%d-%d" % (offset,
offset+count-1))
> - http.endheaders()
> + headers["Authorization"] = transfer.signed_ticket
> +
> + http.request("GET", h['path'], headers=headers)
>
> r = http.getresponse()
> # 206 = HTTP Partial Content.
> @@ -319,11 +319,10 @@ def zero(h, count, offset, may_trim):
> 'size': count,
> 'flush': False}).encode()
>
> - http.putrequest("PATCH", h['path'])
> - http.putheader("Content-Type", "application/json")
> - http.putheader("Content-Length", len(buf))
> - http.endheaders()
> - http.send(buf)
> + headers = {"Content-Type": "application/json",
> + "Content-Length", str(len(buf))}
> +
> + http.request("PATCH", h['path'], body=buf, headers=headers)
>
> r = http.getresponse()
> if r.status != 200:
> @@ -368,11 +367,10 @@ def trim(h, count, offset):
> 'size': count,
> 'flush': False}).encode()
>
> - http.putrequest("PATCH", h['path'])
> - http.putheader("Content-Type", "application/json")
> - http.putheader("Content-Length", len(buf))
> - http.endheaders()
> - http.send(buf)
> + headers = {"Content-Type": "application/json",
> + "Content-Length", str(len(buf))}
> +
> + http.request("PATCH", h['path'], body=buf, headers=headers)
>
> r = http.getresponse()
> if r.status != 200:
> @@ -387,11 +385,10 @@ def flush(h):
> # Construct the JSON request for flushing.
> buf = json.dumps({'op': "flush"}).encode()
>
> - http.putrequest("PATCH", h['path'])
> - http.putheader("Content-Type", "application/json")
> - http.putheader("Content-Length", len(buf))
> - http.endheaders()
> - http.send(buf)
> + headers = {"Content-Type": "application/json",
> + "Content-Length", str(len(buf))}
> +
> + http.request("PATCH", h['path'], body=buf, headers=headers)
>
> r = http.getresponse()
> if r.status != 200:
> --
> 2.17.1
--
Richard Jones, Virtualization Group, Red Hat
http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines. Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top