[PATCH v2v] vddk: Use blocksize filter to split very large requests
Predictably this makes no difference to the current code since nbdcopy is using 256K requests. I will do some real testing when Nir pushes his change here: https://listman.redhat.com/archives/libguestfs/2022-February/msg00188.html Nir - do you want to actually increase that back to 8M? In theory (untested) it should be safe provided it's combined with something like this patch. Rich.
We know that VDDK has problems handling very large single requests. It may cause out of memory errors on the server unless a relatively obscure server-side configuration change is made. With modular virt-v2v we won't necessarily have fine control over who is connecting to the input socket and what sized requests they may make, eg if we delegate copying to a third party. nbdkit-blocksize-filter offers a simple solution. It will split requests larger than a certain size, ensuring that whatever NBD requests come in, we shouldn't cause problems on the VMware server. I chose 2M as the maximum request size, but this could be fine-tuned later. For a longer explanation, see: https://gitlab.com/nbdkit/nbdkit/-/commit/6f4746af9ac65c779d6afda9459e9f862a... --- input/nbdkit_vddk.ml | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/input/nbdkit_vddk.ml b/input/nbdkit_vddk.ml index 40dae9baa7..9609c4423f 100644 --- a/input/nbdkit_vddk.ml +++ b/input/nbdkit_vddk.ml @@ -144,6 +144,16 @@ See also the virt-v2v-input-vmware(1) manual.") libNN *) Nbdkit.add_filter_if_available cmd "cacheextents"; + (* Split very large requests to avoid out of memory errors on the + * server. Since we're using this filter, also add minblock=512 + * although it will make no difference. + *) + if Nbdkit.probe_filter "blocksize" then ( + Nbdkit.add_filter cmd "blocksize"; + Nbdkit.add_arg cmd "minblock" "512"; + Nbdkit.add_arg cmd "maxdata" "2M" + ); + (* IMPORTANT! Add the COW filter. It must be furthest away * except for the multi-conn and rate filters. *) -- 2.35.1
On 02/15/22 14:43, Richard W.M. Jones wrote:
We know that VDDK has problems handling very large single requests. It may cause out of memory errors on the server unless a relatively obscure server-side configuration change is made. With modular virt-v2v we won't necessarily have fine control over who is connecting to the input socket and what sized requests they may make, eg if we delegate copying to a third party.
nbdkit-blocksize-filter offers a simple solution. It will split requests larger than a certain size, ensuring that whatever NBD requests come in, we shouldn't cause problems on the VMware server. I chose 2M as the maximum request size, but this could be fine-tuned later.
For a longer explanation, see: https://gitlab.com/nbdkit/nbdkit/-/commit/6f4746af9ac65c779d6afda9459e9f862a... --- input/nbdkit_vddk.ml | 10 ++++++++++ 1 file changed, 10 insertions(+)
diff --git a/input/nbdkit_vddk.ml b/input/nbdkit_vddk.ml index 40dae9baa7..9609c4423f 100644 --- a/input/nbdkit_vddk.ml +++ b/input/nbdkit_vddk.ml @@ -144,6 +144,16 @@ See also the virt-v2v-input-vmware(1) manual.") libNN *) Nbdkit.add_filter_if_available cmd "cacheextents";
+ (* Split very large requests to avoid out of memory errors on the + * server. Since we're using this filter, also add minblock=512 + * although it will make no difference. + *) + if Nbdkit.probe_filter "blocksize" then ( + Nbdkit.add_filter cmd "blocksize"; + Nbdkit.add_arg cmd "minblock" "512"; + Nbdkit.add_arg cmd "maxdata" "2M" + ); + (* IMPORTANT! Add the COW filter. It must be furthest away * except for the multi-conn and rate filters. *)
Acked-by: Laszlo Ersek <lersek@redhat.com>
On Tue, Feb 15, 2022 at 01:43:01PM +0000, Richard W.M. Jones wrote:
Predictably this makes no difference to the current code since nbdcopy is using 256K requests. I will do some real testing when Nir pushes his change here:
https://listman.redhat.com/archives/libguestfs/2022-February/msg00188.html
Some test results can be found in this bug, unfortunately in private comments. I'll reproduce relevant excerpts below. As you can see this change + 8M block size approximately doubles the copying performance and brings it roughly in line with virt-v2v 1.45.3. https://bugzilla.redhat.com/show_bug.cgi?id=2039255 <quote>
From mxie Comment 31:
1. Convert a guests from ESXi7.0 to rhv(rhv-upload) via vddk7.0.2 with rhv-direct=true to compare the performance between virt-v2v-1.45.98-1 and virt-v2v-1.45.3-3 v2v_version esx7.0-vddk7.0.2-rhv-upload-rhv-direct_true 1.45.98-1 Convert guest: 2m19s Copying disk: 8m18s 1.45.3-3 Convert guest: 2m5s Copying disk: 4m28s
From Richard W.M. Jones Comment 32:
virt-v2v-1.45.98-1.el9.bz2039255.3: - Increases nbdcopy --requests to 1024 - Increases nbdcopy --request-size to 8M - Introduces the blocksize filter to prevent VMware problems related to above changes: https://listman.redhat.com/archives/libguestfs/2022-February/msg00211.html
From mxie Comment 33:
I think the performance has been improved by virt-v2v-1.45.98-1.el9.bz2039255.3 Test the bug with virt-v2v-1.45.98-1.el9.bz2039255.3.x86_64 Steps: 1. Convert a guests from ESXi7.0 to rhv(rhv-upload) via vddk7.0.2 with rhv-direct=true by v2v v2v_version esx7.0-vddk7.0.2-rhv-upload-rhv-direct_true 1.45.98-1 Convert guest: 2m15s Copying disk: 4m54s 1.45.3-3 Convert guest: 2m5s Copying disk: 4m28s </quote> Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW
participants (2)
-
Laszlo Ersek -
Richard W.M. Jones