Re: [Libguestfs] [PATCH v2v] vddk: Use blocksize filter to split very large requests

We know that VDDK has problems handling very large single requests. It may cause out of memory errors on the server unless a relatively obscure server-side configuration change is made. With modular virt-v2v we won't necessarily have fine control over who is connecting to the input socket and what sized requests they may make, eg if we delegate copying to a third party. nbdkit-blocksize-filter offers a simple solution. It will split requests larger than a certain size, ensuring that whatever NBD requests come in, we shouldn't cause problems on the VMware server. I chose 2M as the maximum request size, but this could be fine-tuned later. For a longer explanation, see: https://gitlab.com/nbdkit/nbdkit/-/commit/6f4746af9ac65c779d6afda9459e9f8... --- input/nbdkit_vddk.ml | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/input/nbdkit_vddk.ml b/input/nbdkit_vddk.ml index 40dae9baa7..9609c4423f 100644 --- a/input/nbdkit_vddk.ml +++ b/input/nbdkit_vddk.ml @@ -144,6 +144,16 @@ See also the virt-v2v-input-vmware(1) manual.") libNN *) Nbdkit.add_filter_if_available cmd "cacheextents"; + (* Split very large requests to avoid out of memory errors on the + * server. Since we're using this filter, also add minblock=512 + * although it will make no difference. + *) + if Nbdkit.probe_filter "blocksize" then ( + Nbdkit.add_filter cmd "blocksize"; + Nbdkit.add_arg cmd "minblock" "512"; + Nbdkit.add_arg cmd "maxdata" "2M" + ); + (* IMPORTANT! Add the COW filter. It must be furthest away * except for the multi-conn and rate filters. *)