7:43 a.m.
Predictably this makes no difference to the current code since nbdcopy
is using 256K requests. I will do some real testing when Nir pushes
his change here:
https://listman.redhat.com/archives/libguestfs/2022-February/msg00188.html
Nir - do you want to actually increase that back to 8M? In theory
(untested) it should be safe provided it's combined with something
like this patch.
Rich.
7:43 a.m.
We know that VDDK has problems handling very large single requests.
It may cause out of memory errors on the server unless a relatively
obscure server-side configuration change is made. With modular
virt-v2v we won't necessarily have fine control over who is connecting
to the input socket and what sized requests they may make, eg if we
delegate copying to a third party.
nbdkit-blocksize-filter offers a simple solution. It will split
requests larger than a certain size, ensuring that whatever NBD
requests come in, we shouldn't cause problems on the VMware server.
I chose 2M as the maximum request size, but this could be fine-tuned
later.
For a longer explanation, see:
https://gitlab.com/nbdkit/nbdkit/-/commit/6f4746af9ac65c779d6afda9459e9f8...
---
input/nbdkit_vddk.ml | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/input/nbdkit_vddk.ml b/input/nbdkit_vddk.ml
index 40dae9baa7..9609c4423f 100644
--- a/input/nbdkit_vddk.ml
+++ b/input/nbdkit_vddk.ml
@@ -144,6 +144,16 @@ See also the virt-v2v-input-vmware(1) manual.") libNN
*)
Nbdkit.add_filter_if_available cmd "cacheextents";
+ (* Split very large requests to avoid out of memory errors on the
+ * server. Since we're using this filter, also add minblock=512
+ * although it will make no difference.
+ *)
+ if Nbdkit.probe_filter "blocksize" then (
+ Nbdkit.add_filter cmd "blocksize";
+ Nbdkit.add_arg cmd "minblock" "512";
+ Nbdkit.add_arg cmd "maxdata" "2M"
+ );
+
(* IMPORTANT! Add the COW filter. It must be furthest away
* except for the multi-conn and rate filters.
*)
--
2.35.1
7:53 a.m.
On 02/15/22 14:43, Richard W.M. Jones wrote:
We know that VDDK has problems handling very large single requests.
It may cause out of memory errors on the server unless a relatively
obscure server-side configuration change is made. With modular
virt-v2v we won't necessarily have fine control over who is connecting
to the input socket and what sized requests they may make, eg if we
delegate copying to a third party.
nbdkit-blocksize-filter offers a simple solution. It will split
requests larger than a certain size, ensuring that whatever NBD
requests come in, we shouldn't cause problems on the VMware server.
I chose 2M as the maximum request size, but this could be fine-tuned
later.
For a longer explanation, see:
https://gitlab.com/nbdkit/nbdkit/-/commit/6f4746af9ac65c779d6afda9459e9f8...
---
input/nbdkit_vddk.ml | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/input/nbdkit_vddk.ml b/input/nbdkit_vddk.ml
index 40dae9baa7..9609c4423f 100644
--- a/input/nbdkit_vddk.ml
+++ b/input/nbdkit_vddk.ml
@@ -144,6 +144,16 @@ See also the virt-v2v-input-vmware(1) manual.") libNN
*)
Nbdkit.add_filter_if_available cmd "cacheextents";
+ (* Split very large requests to avoid out of memory errors on the
+ * server. Since we're using this filter, also add minblock=512
+ * although it will make no difference.
+ *)
+ if Nbdkit.probe_filter "blocksize" then (
+ Nbdkit.add_filter cmd "blocksize";
+ Nbdkit.add_arg cmd "minblock" "512";
+ Nbdkit.add_arg cmd "maxdata" "2M"
+ );
+
(* IMPORTANT! Add the COW filter. It must be furthest away
* except for the multi-conn and rate filters.
*)
Acked-by: Laszlo Ersek <lersek(a)redhat.com>
8:40 a.m.
On Tue, Feb 15, 2022 at 01:43:01PM +0000, Richard W.M. Jones wrote:
Predictably this makes no difference to the current code since
nbdcopy
is using 256K requests. I will do some real testing when Nir pushes
his change here:
https://listman.redhat.com/archives/libguestfs/2022-February/msg00188.html
Some test results can be found in this bug, unfortunately in private
comments. I'll reproduce relevant excerpts below. As you can see
this change + 8M block size approximately doubles the copying
performance and brings it roughly in line with virt-v2v 1.45.3.
https://bugzilla.redhat.com/show_bug.cgi?id=2039255
<quote>
From mxie Comment 31:
1. Convert a guests from ESXi7.0 to rhv(rhv-upload) via vddk7.0.2 with rhv-direct=true to
compare the performance between virt-v2v-1.45.98-1 and virt-v2v-1.45.3-3
v2v_version esx7.0-vddk7.0.2-rhv-upload-rhv-direct_true
1.45.98-1 Convert guest: 2m19s
Copying disk: 8m18s
1.45.3-3 Convert guest: 2m5s
Copying disk: 4m28s
From Richard W.M. Jones Comment 32:
virt-v2v-1.45.98-1.el9.bz2039255.3:
- Increases nbdcopy --requests to 1024
- Increases nbdcopy --request-size to 8M
- Introduces the blocksize filter to prevent VMware problems related to above changes:
https://listman.redhat.com/archives/libguestfs/2022-February/msg00211.html
From mxie Comment 33:
I think the performance has been improved by virt-v2v-1.45.98-1.el9.bz2039255.3
Test the bug with virt-v2v-1.45.98-1.el9.bz2039255.3.x86_64
Steps:
1. Convert a guests from ESXi7.0 to rhv(rhv-upload) via vddk7.0.2 with rhv-direct=true by
v2v
v2v_version esx7.0-vddk7.0.2-rhv-upload-rhv-direct_true
1.45.98-1 Convert guest: 2m15s
Copying disk: 4m54s
1.45.3-3 Convert guest: 2m5s
Copying disk: 4m28s
</quote>
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW
935
days inactive
935
days old
3 comments
2 participants
participants (2)
-
Laszlo Ersek -
Richard W.M. Jones