[PATCH v2v 0/3] Fix test-v2v-trim.sh test
Final 3 patches needed to fix "make check-slow", specifically the test-v2v-trim.sh test. This revealed a nasty bug in our sparsification if the input disk format is qcow2. Rich.
Old virt-v2v used the same output format as input format when doing disk to disk copies. Modular virt-v2v doesn't do this, it requires using -of to set the output format to the same as the input format if that is desired. --- tests/test-v2v-trim.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/test-v2v-trim.sh b/tests/test-v2v-trim.sh index 8eab4792b2..ed6c031df3 100755 --- a/tests/test-v2v-trim.sh +++ b/tests/test-v2v-trim.sh @@ -66,7 +66,7 @@ fi virt-v2v --debug-gc \ -i disk $d/fedora.qcow2 \ - -o local -os $d + -o local -of qcow2 -os $d # Test the libvirt XML metadata and a disk was created. test -f $d/fedora.xml -- 2.32.0
On 01/19/22 19:57, Richard W.M. Jones wrote:
Old virt-v2v used the same output format as input format when doing disk to disk copies. Modular virt-v2v doesn't do this, it requires using -of to set the output format to the same as the input format if that is desired. --- tests/test-v2v-trim.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tests/test-v2v-trim.sh b/tests/test-v2v-trim.sh index 8eab4792b2..ed6c031df3 100755 --- a/tests/test-v2v-trim.sh +++ b/tests/test-v2v-trim.sh @@ -66,7 +66,7 @@ fi
virt-v2v --debug-gc \ -i disk $d/fedora.qcow2 \ - -o local -os $d + -o local -of qcow2 -os $d
# Test the libvirt XML metadata and a disk was created. test -f $d/fedora.xml
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
qemu-nbd --shared (-e) flag controls how many clients can connect concurrently. 0 means unlimited. We want to allow the sockets to be queried by other processes while virt-v2v is running and it should be safe to do this. The default configuration of qemu-nbd doesn't allow this so add --shared=0. Note this does not (in current qemu) enable multi-conn because we aren't using the -r (read-only) flag. --- lib/qemuNBD.ml | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/lib/qemuNBD.ml b/lib/qemuNBD.ml index 12d083aeb7..89c93d7072 100644 --- a/lib/qemuNBD.ml +++ b/lib/qemuNBD.ml @@ -90,7 +90,11 @@ let run_unix ?socket { disk; snapshot; format } = (* Construct the qemu-nbd command line. *) let args = ref [] in List.push_back_list args - ["qemu-nbd"; "-t"; "--pid-file"; pidfile; "--socket"; socket]; + ["qemu-nbd"; + "-t"; + "--shared=0"; + "--pid-file"; pidfile; + "--socket"; socket]; (* -s adds a protective overlay. *) if snapshot then List.push_back args "-s"; -- 2.32.0
On 01/19/22 19:57, Richard W.M. Jones wrote:
qemu-nbd --shared (-e) flag controls how many clients can connect concurrently. 0 means unlimited.
We want to allow the sockets to be queried by other processes while virt-v2v is running and it should be safe to do this. The default configuration of qemu-nbd doesn't allow this so add --shared=0.
Note this does not (in current qemu) enable multi-conn because we aren't using the -r (read-only) flag. --- lib/qemuNBD.ml | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/lib/qemuNBD.ml b/lib/qemuNBD.ml index 12d083aeb7..89c93d7072 100644 --- a/lib/qemuNBD.ml +++ b/lib/qemuNBD.ml @@ -90,7 +90,11 @@ let run_unix ?socket { disk; snapshot; format } = (* Construct the qemu-nbd command line. *) let args = ref [] in List.push_back_list args - ["qemu-nbd"; "-t"; "--pid-file"; pidfile; "--socket"; socket]; + ["qemu-nbd"; + "-t"; + "--shared=0"; + "--pid-file"; pidfile; + "--socket"; socket];
(* -s adds a protective overlay. *) if snapshot then List.push_back args "-s";
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
On Wed, Jan 19, 2022 at 06:57:48PM +0000, Richard W.M. Jones wrote:
qemu-nbd --shared (-e) flag controls how many clients can connect concurrently. 0 means unlimited.
As of qemu commit 3dcf56e625 (v6.0.0). Does v2v need to cater to older qemu, or are we at the point where we can say "using this version of v2v requires at least X minimum version of qemu"?
We want to allow the sockets to be queried by other processes while virt-v2v is running and it should be safe to do this. The default configuration of qemu-nbd doesn't allow this so add --shared=0.
Note this does not (in current qemu) enable multi-conn because we aren't using the -r (read-only) flag.
Yeah, I still need to work on that for qemu 7.0 (my attempt for 6.2 garnered some interesting review comments from Kevin, which I still need to address). -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
On Thu, Jan 20, 2022 at 11:22:28AM -0600, Eric Blake wrote:
On Wed, Jan 19, 2022 at 06:57:48PM +0000, Richard W.M. Jones wrote:
qemu-nbd --shared (-e) flag controls how many clients can connect concurrently. 0 means unlimited.
As of qemu commit 3dcf56e625 (v6.0.0). Does v2v need to cater to older qemu, or are we at the point where we can say "using this version of v2v requires at least X minimum version of qemu"?
For modular virt-v2v we need a new qemu. It's not pinned down in the virt-v2v README file but we can certainly demand at least v6.0.
We want to allow the sockets to be queried by other processes while virt-v2v is running and it should be safe to do this. The default configuration of qemu-nbd doesn't allow this so add --shared=0.
Note this does not (in current qemu) enable multi-conn because we aren't using the -r (read-only) flag.
Yeah, I still need to work on that for qemu 7.0 (my attempt for 6.2 garnered some interesting review comments from Kevin, which I still need to address).
Thanks, Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org
The default for qemu-nbd is to ignore discard requests. This meant that for input files in qcow2 format the "Mapping ..." (ie. fstrim) step did nothing, all the work was ignored and we copied deleted data over to the destination. This was detected by the test-v2v-trim.sh test. Fixes: commit 255722cbf39afc0b012e2ac00d16fa6ba2f8c21f --- lib/qemuNBD.ml | 1 + 1 file changed, 1 insertion(+) diff --git a/lib/qemuNBD.ml b/lib/qemuNBD.ml index 89c93d7072..54139ce0b4 100644 --- a/lib/qemuNBD.ml +++ b/lib/qemuNBD.ml @@ -93,6 +93,7 @@ let run_unix ?socket { disk; snapshot; format } = ["qemu-nbd"; "-t"; "--shared=0"; + "--discard=unmap"; "--pid-file"; pidfile; "--socket"; socket]; -- 2.32.0
On 01/19/22 19:57, Richard W.M. Jones wrote:
The default for qemu-nbd is to ignore discard requests. This meant that for input files in qcow2 format the "Mapping ..." (ie. fstrim) step did nothing, all the work was ignored and we copied deleted data over to the destination.
This was detected by the test-v2v-trim.sh test.
Fixes: commit 255722cbf39afc0b012e2ac00d16fa6ba2f8c21f --- lib/qemuNBD.ml | 1 + 1 file changed, 1 insertion(+)
diff --git a/lib/qemuNBD.ml b/lib/qemuNBD.ml index 89c93d7072..54139ce0b4 100644 --- a/lib/qemuNBD.ml +++ b/lib/qemuNBD.ml @@ -93,6 +93,7 @@ let run_unix ?socket { disk; snapshot; format } = ["qemu-nbd"; "-t"; "--shared=0"; + "--discard=unmap"; "--pid-file"; pidfile; "--socket"; socket];
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
participants (3)
-
Eric Blake -
Laszlo Ersek -
Richard W.M. Jones