[PATCH v2] lib: Convert all drive socket parameters to an absolute path (RHBZ#1588451).
by Richard W.M. Jones
Also fix two tests which assumed that you could add a non-existent
socket.
---
fish/test-add-uri.sh | 16 ++++++++++++----
lib/drives.c | 14 +++++++++++++-
tests/disks/test-qemu-drive.sh | 19 ++++++++++++++-----
3 files changed, 39 insertions(+), 10 deletions(-)
diff --git a/fish/test-add-uri.sh b/fish/test-add-uri.sh
index 756df997b..cb4d40199 100755
--- a/fish/test-add-uri.sh
+++ b/fish/test-add-uri.sh
@@ -26,6 +26,7 @@ skip_if_skipped
rm -f test-add-uri.out
rm -f test-add-uri.img
+rm -f test-add-uri.sock
$VG guestfish sparse test-add-uri.img 10M
@@ -55,11 +56,17 @@ grep -sq 'add_drive "" "protocol:nbd" "server:tcp:example.com"' test-add-uri.out
$VG guestfish -x -a nbd://example.com:3000 </dev/null >test-add-uri.out 2>&1
grep -sq 'add_drive "" "protocol:nbd" "server:tcp:example.com:3000"' test-add-uri.out || fail
-$VG guestfish -x -a 'nbd://?socket=/sk' </dev/null >test-add-uri.out 2>&1
-grep -sq 'add_drive "" "protocol:nbd" "server:unix:/sk"' test-add-uri.out || fail
+# NBD Unix domain socket tests.
+# These require Perl to create the socket, but don't fail if it's not
+# around.
-$VG guestfish -x -a 'nbd:///export?socket=/sk' </dev/null >test-add-uri.out 2>&1
-grep -sq 'add_drive "/export" "protocol:nbd" "server:unix:/sk"' test-add-uri.out || fail
+if perl -MIO::Socket -e 'IO::Socket::UNIX->new (Local => "test-add-uri.sock", Type => SOCK_STREAM(), Listen => 1); 1'; then
+ $VG guestfish -x -a 'nbd://?socket=test-add-uri.sock' </dev/null >test-add-uri.out 2>&1
+ grep -sq 'add_drive "" "protocol:nbd" "server:unix:test-add-uri.sock"' test-add-uri.out || fail
+
+ $VG guestfish -x -a 'nbd:///export?socket=test-add-uri.sock' </dev/null >test-add-uri.out 2>&1
+ grep -sq 'add_drive "/export" "protocol:nbd" "server:unix:test-add-uri.sock"' test-add-uri.out || fail
+fi
# rbd
$VG guestfish -x -a rbd://example.com:6789/pool/disk </dev/null >test-add-uri.out 2>&1
@@ -93,3 +100,4 @@ grep -sq 'add_drive "iqn.2015-12.com.libguestfs:test2/0" "protocol:iscsi" "serve
rm test-add-uri.out
rm test-add-uri.img
+rm -f test-add-uri.sock
diff --git a/lib/drives.c b/lib/drives.c
index 82ef30093..7697f369a 100644
--- a/lib/drives.c
+++ b/lib/drives.c
@@ -28,6 +28,7 @@
#include <stdbool.h>
#include <string.h>
#include <unistd.h>
+#include <limits.h>
#include <netdb.h>
#include <arpa/inet.h>
#include <assert.h>
@@ -645,7 +646,18 @@ parse_one_server (guestfs_h *g, const char *server, struct drive_server *ret)
return -1;
}
ret->transport = drive_transport_unix;
- ret->u.socket = safe_strdup (g, server+5);
+
+ /* libvirt requires sockets to be specified as an absolute path
+ * (see RHBZ#1588451), and it's probably a good idea anyway to
+ * check the socket exists and convert it to an absolute path.
+ */
+ ret->u.socket = realpath (server+5, NULL);
+ if (ret->u.socket == NULL) {
+ perrorf (g, _("realpath: could not convert ‘%s’ to an absolute path"),
+ server+5);
+ return -1;
+ }
+
ret->port = 0;
return 0;
}
diff --git a/tests/disks/test-qemu-drive.sh b/tests/disks/test-qemu-drive.sh
index 19dd60a2f..181f21401 100755
--- a/tests/disks/test-qemu-drive.sh
+++ b/tests/disks/test-qemu-drive.sh
@@ -43,6 +43,8 @@ function fail ()
rm -f "$DEBUG_QEMU_FILE"
+rm -f test-qemu-drive.sock
+
# Ceph (RBD).
guestfish <<EOF ||:
@@ -111,13 +113,18 @@ check_output
grep -sq -- '-drive file=nbd:1.2.3.4:1234,' "$DEBUG_QEMU_FILE" || fail
rm "$DEBUG_QEMU_FILE"
-guestfish <<EOF ||:
- add "" "format:raw" "protocol:nbd" "server:unix:/socket"
+# This test requires Perl to create the socket, but don't fail if
+# it's not around.
+if perl -MIO::Socket -e 'IO::Socket::UNIX->new (Local => "test-qemu-drive.sock", Type => SOCK_STREAM(), Listen => 1); 1'; then
+
+ guestfish <<EOF ||:
+ add "" "format:raw" "protocol:nbd" "server:unix:test-qemu-drive.sock"
run
EOF
-check_output
-grep -sq -- '-drive file=nbd:unix:/socket,' "$DEBUG_QEMU_FILE" || fail
-rm "$DEBUG_QEMU_FILE"
+ check_output
+ grep -sq -- '-drive file=nbd:unix:.*/test-qemu-drive.sock,' "$DEBUG_QEMU_FILE" || fail
+ rm "$DEBUG_QEMU_FILE"
+fi
# Sheepdog.
@@ -139,3 +146,5 @@ EOF
check_output
grep -sq -- '-drive file=ssh://rich@example.com/disk.img,' "$DEBUG_QEMU_FILE" || fail
rm "$DEBUG_QEMU_FILE"
+
+rm -f test-qemu-drive.sock
--
2.16.2
6 years, 4 months
[PATCH nbdkit] main: Set umask to a known value and document it for plugins.
by Richard W.M. Jones
---
docs/nbdkit-plugin.pod.in | 4 ++++
src/main.c | 6 ++++++
2 files changed, 10 insertions(+)
diff --git a/docs/nbdkit-plugin.pod.in b/docs/nbdkit-plugin.pod.in
index b6b2e47..22ca40e 100644
--- a/docs/nbdkit-plugin.pod.in
+++ b/docs/nbdkit-plugin.pod.in
@@ -302,6 +302,10 @@ and returns C<NULL>.
The returned string must be freed by the caller.
+=head2 umask
+
+All plugins will see a L<umask(2)> of C<0022>.
+
=head1 CALLBACKS
=head2 C<.name>
diff --git a/src/main.c b/src/main.c
index 660d036..8d901cf 100644
--- a/src/main.c
+++ b/src/main.c
@@ -479,6 +479,12 @@ main (int argc, char *argv[])
exit (EXIT_FAILURE);
}
+ /* Set the umask to a known value. This makes the behaviour of
+ * plugins when creating files more predictable, and also removes an
+ * implicit dependency on umask when calling mkstemp(3).
+ */
+ umask (0022);
+
/* Initialize TLS. */
crypto_init (tls_set_on_cli);
assert (tls != -1);
--
2.16.2
6 years, 4 months
[PATCH] v2v: Set machine type explicitly for outputs which support it (RHBZ#1581428).
by Richard W.M. Jones
QEMU for x86 supports two machine types, "pc" (emulating the ancient
Intel i440FX chipset originally used by the Pentium Pro), and "q35"
(https://wiki.qemu.org/Features/Q35).
Currently virt-v2v does not set any machine type, so libvirt or the
target hypervisor will choose some default, probably i440fx. Newer
versions of libvirt and QEMU will probably switch over to defaulting
to Q35 in the near future.
None of this matters for reasonably new guests since they can boot
with either chipset. However there are some very old guests (notably
Windows XP) which cannot handle Q35.
This commit changes virt-v2v so it always tries to specify the machine
type explicitly (assuming the target supports that, and not all of
them do). We pivot around the year 2007 which happens to be the year
that Q35 hardware was first available. Any guest OS which was first
shipped earlier than 2007 will be given i440fx. Any guest OS from
2007 or later will be given Q35.
For non-x86 architectures we select the "virt" model which will
probably only work for AArch64. More work is needed for POWER.
A note about secure boot: Previous to this patch, if a guest used UEFI
and we detected that UEFI needed secure boot support, then we forced
the machine type to Q35 (which was wrong for non-x86 architectures).
In this patch I removed the code to force Q35, since any guest which
is using secure boot will be new enough that it'll be using Q35 anyway
after this patch (on x86).
---
v2v/convert_linux.ml | 20 ++++++++++++++++++++
v2v/convert_windows.ml | 10 ++++++++++
v2v/create_libvirt_xml.ml | 21 ++++++++++++++-------
v2v/create_ovf.ml | 2 ++
v2v/output_glance.ml | 5 +++++
v2v/output_qemu.ml | 14 ++++++++------
v2v/test-v2v-i-ova.xml | 2 +-
v2v/types.ml | 8 ++++++++
v2v/types.mli | 6 ++++--
9 files changed, 72 insertions(+), 16 deletions(-)
diff --git a/v2v/convert_linux.ml b/v2v/convert_linux.ml
index 02dc2fee2..4dbf2aa41 100644
--- a/v2v/convert_linux.ml
+++ b/v2v/convert_linux.ml
@@ -122,6 +122,25 @@ let convert (g : G.guestfs) inspect source output rcaps =
SELinux_relabel.relabel g;
+ (* Pivot on the year 2007. Any Linux distro from earlier than
+ * 2007 should use i440fx, anything 2007 or newer should use q35.
+ *)
+ let machine =
+ match inspect.i_arch, inspect.i_distro, inspect.i_major_version with
+ | ("i386"|"x86_64"), "fedora", _ -> Q35
+ | ("i386"|"x86_64"), ("rhel"|"centos"|"scientificlinux"|
+ "redhat-based"|"oraclelinux"), major ->
+ if major <= 4 then I440FX else Q35
+ | ("i386"|"x86_64"), ("sles"|"suse-based"|"opensuse"), major ->
+ if major < 10 then I440FX else Q35
+ | ("i386"|"x86_64"), ("debian"|"ubuntu"|"linuxmint"|
+ "kalilinux"), major ->
+ if major < 4 then I440FX else Q35
+
+ (* reasonable default for all modern Linux kernels *)
+ | ("i386"|"x86_64"), _, _ -> Q35
+ | _ -> Virt in
+
(* Return guest capabilities from the convert () function. *)
let guestcaps = {
gcaps_block_bus = block_type;
@@ -130,6 +149,7 @@ let convert (g : G.guestfs) inspect source output rcaps =
gcaps_virtio_rng = kernel.ki_supports_virtio_rng;
gcaps_virtio_balloon = kernel.ki_supports_virtio_balloon;
gcaps_isa_pvpanic = kernel.ki_supports_isa_pvpanic;
+ gcaps_machine = machine;
gcaps_arch = Utils.kvm_arch inspect.i_arch;
gcaps_acpi = acpi;
} in
diff --git a/v2v/convert_windows.ml b/v2v/convert_windows.ml
index 163319545..97882c377 100644
--- a/v2v/convert_windows.ml
+++ b/v2v/convert_windows.ml
@@ -212,6 +212,15 @@ let convert (g : G.guestfs) inspect source output rcaps =
warning (f_"this guest has Anti-Virus (AV) software and a new virtio block device driver was installed. In some circumstances, AV may prevent new drivers from working (resulting in a 7B boot error). If this happens, try disabling AV before doing the conversion.");
);
+ (* Pivot on the year 2007. Any Windows version from earlier than
+ * 2007 should use i440fx, anything 2007 or newer should use q35.
+ * Luckily this coincides almost exactly with the release of NT 6.
+ *)
+ let machine =
+ match inspect.i_arch, inspect.i_major_version with
+ | ("i386"|"x86_64"), major -> if major < 6 then I440FX else Q35
+ | _ -> Virt in
+
(* Return guest capabilities from the convert () function. *)
let guestcaps = {
gcaps_block_bus = block_driver;
@@ -220,6 +229,7 @@ let convert (g : G.guestfs) inspect source output rcaps =
gcaps_virtio_rng = virtio_rng_supported;
gcaps_virtio_balloon = virtio_ballon_supported;
gcaps_isa_pvpanic = isa_pvpanic_supported;
+ gcaps_machine = machine;
gcaps_arch = Utils.kvm_arch inspect.i_arch;
gcaps_acpi = true;
} in
diff --git a/v2v/create_libvirt_xml.ml b/v2v/create_libvirt_xml.ml
index fbe90eeaa..4b36ffb8e 100644
--- a/v2v/create_libvirt_xml.ml
+++ b/v2v/create_libvirt_xml.ml
@@ -86,10 +86,6 @@ let create_libvirt_xml ?pool source target_buses guestcaps
| Some { Uefi.flags = flags }
when List.mem Uefi.UEFI_FLAG_SECURE_BOOT_REQUIRED flags -> true
| _ -> false in
- (* Currently these are required by secure boot, but in theory they
- * might be independent properties.
- *)
- let machine_q35 = secure_boot_required in
let smm = secure_boot_required in
(* We have the machine features of the guest when it was on the
@@ -140,7 +136,18 @@ let create_libvirt_xml ?pool source target_buses guestcaps
(* The <os> section subelements. *)
let os_section =
- let machine = if machine_q35 then [ "machine", "q35" ] else [] in
+ let os = ref [] in
+
+ let machine =
+ match guestcaps.gcaps_machine with
+ | I440FX -> "pc"
+ | Q35 -> "q35"
+ | Virt -> "virt" in
+
+ List.push_back os
+ (e "type" ["arch", guestcaps.gcaps_arch;
+ "machine", machine]
+ [PCData "hvm"]);
let loader =
match uefi_firmware with
@@ -152,8 +159,8 @@ let create_libvirt_xml ?pool source target_buses guestcaps
[ PCData code ];
e "nvram" ["template", vars_template] [] ] in
- (e "type" (["arch", guestcaps.gcaps_arch] @ machine) [PCData "hvm"])
- :: loader in
+ List.push_back_list os loader;
+ !os in
List.push_back_list body [
e "os" [] os_section;
diff --git a/v2v/create_ovf.ml b/v2v/create_ovf.ml
index 9e0c772fd..81357b55e 100644
--- a/v2v/create_ovf.ml
+++ b/v2v/create_ovf.ml
@@ -602,6 +602,8 @@ let rec create_ovf source targets guestcaps inspect
source.s_vcpu memsize_mb)]
] in
+ (* XXX How to set machine type for Q35? *)
+
List.push_back virtual_hardware_section_items (
e "Item" [] ([
e "rasd:Caption" [] [PCData (sprintf "%d virtual cpu" source.s_vcpu)];
diff --git a/v2v/output_glance.ml b/v2v/output_glance.ml
index c334def42..96c31da59 100644
--- a/v2v/output_glance.ml
+++ b/v2v/output_glance.ml
@@ -86,6 +86,11 @@ object
(match guestcaps.gcaps_video with
| QXL -> "qxl"
| Cirrus -> "cirrus");
+ "hw_machine_type",
+ (match guestcaps.gcaps_machine with
+ | I440FX -> "pc"
+ | Q35 -> "q35"
+ | Virt -> "virt");
"architecture", guestcaps.gcaps_arch;
"hypervisor_type", "kvm";
"vm_mode", "hvm";
diff --git a/v2v/output_qemu.ml b/v2v/output_qemu.ml
index 952660de2..487363520 100644
--- a/v2v/output_qemu.ml
+++ b/v2v/output_qemu.ml
@@ -61,12 +61,14 @@ object
| Some { Uefi.flags }
when List.mem Uefi.UEFI_FLAG_SECURE_BOOT_REQUIRED flags -> true
| _ -> false in
- (* Currently these are required by secure boot, but in theory they
- * might be independent properties.
- *)
- let machine_q35 = secure_boot_required in
let smm = secure_boot_required in
+ let machine =
+ match guestcaps.gcaps_machine with
+ | I440FX -> "pc"
+ | Q35 -> "q35"
+ | Virt -> "virt" in
+
(* Construct the command line. Note that the [Qemuopts]
* module deals with shell and qemu comma quoting.
*)
@@ -80,8 +82,8 @@ object
flag "-no-user-config"; flag "-nodefaults";
arg "-name" source.s_name;
- arg_list "-machine" (if machine_q35 then ["q35"] else [] @
- if smm then ["smm=on"] else [] @
+ arg_list "-machine" (machine ::
+ (if smm then ["smm=on"] else []) @
["accel=kvm:tcg"]);
(match uefi_firmware with
diff --git a/v2v/test-v2v-i-ova.xml b/v2v/test-v2v-i-ova.xml
index 5a303b80a..15925ce6e 100644
--- a/v2v/test-v2v-i-ova.xml
+++ b/v2v/test-v2v-i-ova.xml
@@ -10,7 +10,7 @@
<apic/>
</features>
<os>
- <type arch='x86_64'>hvm</type>
+ <type arch='x86_64' machine='q35'>hvm</type>
</os>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
diff --git a/v2v/types.ml b/v2v/types.ml
index da1192ec3..a569759c7 100644
--- a/v2v/types.ml
+++ b/v2v/types.ml
@@ -398,6 +398,7 @@ type guestcaps = {
gcaps_virtio_rng : bool;
gcaps_virtio_balloon : bool;
gcaps_isa_pvpanic : bool;
+ gcaps_machine : guestcaps_machine;
gcaps_arch : string;
gcaps_acpi : bool;
}
@@ -409,6 +410,7 @@ and requested_guestcaps = {
and guestcaps_block_type = Virtio_blk | Virtio_SCSI | IDE
and guestcaps_net_type = Virtio_net | E1000 | RTL8139
and guestcaps_video_type = QXL | Cirrus
+and guestcaps_machine = I440FX | Q35 | Virt
let string_of_block_type = function
| Virtio_blk -> "virtio-blk"
@@ -421,17 +423,23 @@ let string_of_net_type = function
let string_of_video = function
| QXL -> "qxl"
| Cirrus -> "cirrus"
+let string_of_machine = function
+ | I440FX -> "i440fx"
+ | Q35 -> "q35"
+ | Virt -> "virt"
let string_of_guestcaps gcaps =
sprintf "\
gcaps_block_bus = %s
gcaps_net_bus = %s
gcaps_video = %s
+gcaps_machine = %s
gcaps_arch = %s
gcaps_acpi = %b
" (string_of_block_type gcaps.gcaps_block_bus)
(string_of_net_type gcaps.gcaps_net_bus)
(string_of_video gcaps.gcaps_video)
+ (string_of_machine gcaps.gcaps_machine)
gcaps.gcaps_arch
gcaps.gcaps_acpi
diff --git a/v2v/types.mli b/v2v/types.mli
index 291707b74..3eb946886 100644
--- a/v2v/types.mli
+++ b/v2v/types.mli
@@ -240,8 +240,9 @@ type guestcaps = {
gcaps_virtio_balloon : bool; (** Guest supports virtio balloon. *)
gcaps_isa_pvpanic : bool; (** Guest supports ISA pvpanic device. *)
- gcaps_arch : string; (** Architecture that KVM must emulate. *)
- gcaps_acpi : bool; (** True if guest supports acpi. *)
+ gcaps_machine : guestcaps_machine; (** Machine model. *)
+ gcaps_arch : string; (** Architecture that KVM must emulate. *)
+ gcaps_acpi : bool; (** True if guest supports acpi. *)
}
(** Guest capabilities after conversion. eg. Was virtio found or installed? *)
@@ -257,6 +258,7 @@ and requested_guestcaps = {
and guestcaps_block_type = Virtio_blk | Virtio_SCSI | IDE
and guestcaps_net_type = Virtio_net | E1000 | RTL8139
and guestcaps_video_type = QXL | Cirrus
+and guestcaps_machine = I440FX | Q35 | Virt
val string_of_guestcaps : guestcaps -> string
val string_of_requested_guestcaps : requested_guestcaps -> string
--
2.16.2
6 years, 4 months
[PATCH nbdkit 0/2] Fix a couple of problems found by Coverity.
by Richard W.M. Jones
There are a few other issues that Coverity found, but I believe
all can be ignored ... except one:
We don't set umask anywhere inside nbdkit. Coverity complains that
this is a problem where we create temporary files, since the result of
mkstemp depends implicitly on the umask value. I think we might
consider setting umask anyway (eg. to 022) just to make plugin
behaviour more predictable. What do you think?
Rich.
6 years, 4 months
[PATCH] v2v: -o rhv-upload: Optimize http request sending
by Nir Soffer
When sending request with small or no payload, it is simpler and
possibly more efficient to use the high level HTTPSConnection.request(),
instead of the lower level APIs.
The only reason to use the lower level APIs is to avoid copying the
payload, or on python 2, to use a bigger buffer size when streaming a
file-like object.
---
v2v/rhv-upload-plugin.py | 35 ++++++++++++++++-------------------
1 file changed, 16 insertions(+), 19 deletions(-)
diff --git a/v2v/rhv-upload-plugin.py b/v2v/rhv-upload-plugin.py
index ed99cc7a9..3fad865f6 100644
--- a/v2v/rhv-upload-plugin.py
+++ b/v2v/rhv-upload-plugin.py
@@ -263,12 +263,12 @@ def pread(h, count, offset):
transfer = h['transfer']
transfer_service = h['transfer_service']
- http.putrequest("GET", h['path'])
+ headers = {"Range", "bytes=%d-%d" % (offset, offset+count-1)}
# Authorization is only needed for old imageio.
if h['needs_auth']:
- http.putheader("Authorization", transfer.signed_ticket)
- http.putheader("Range", "bytes=%d-%d" % (offset, offset+count-1))
- http.endheaders()
+ headers["Authorization"] = transfer.signed_ticket
+
+ http.request("GET", h['path'], headers=headers)
r = http.getresponse()
# 206 = HTTP Partial Content.
@@ -319,11 +319,10 @@ def zero(h, count, offset, may_trim):
'size': count,
'flush': False}).encode()
- http.putrequest("PATCH", h['path'])
- http.putheader("Content-Type", "application/json")
- http.putheader("Content-Length", len(buf))
- http.endheaders()
- http.send(buf)
+ headers = {"Content-Type": "application/json",
+ "Content-Length", str(len(buf))}
+
+ http.request("PATCH", h['path'], body=buf, headers=headers)
r = http.getresponse()
if r.status != 200:
@@ -368,11 +367,10 @@ def trim(h, count, offset):
'size': count,
'flush': False}).encode()
- http.putrequest("PATCH", h['path'])
- http.putheader("Content-Type", "application/json")
- http.putheader("Content-Length", len(buf))
- http.endheaders()
- http.send(buf)
+ headers = {"Content-Type": "application/json",
+ "Content-Length", str(len(buf))}
+
+ http.request("PATCH", h['path'], body=buf, headers=headers)
r = http.getresponse()
if r.status != 200:
@@ -387,11 +385,10 @@ def flush(h):
# Construct the JSON request for flushing.
buf = json.dumps({'op': "flush"}).encode()
- http.putrequest("PATCH", h['path'])
- http.putheader("Content-Type", "application/json")
- http.putheader("Content-Length", len(buf))
- http.endheaders()
- http.send(buf)
+ headers = {"Content-Type": "application/json",
+ "Content-Length", str(len(buf))}
+
+ http.request("PATCH", h['path'], body=buf, headers=headers)
r = http.getresponse()
if r.status != 200:
--
2.17.1
6 years, 4 months
IRC question - failing tests
by Richard W.M. Jones
23:01 < nsoffer> Is this test know to fail? libguestfs/tests/c-api/.libs/lt-tests
23:01 < nsoffer> running make check on fedora 28
23:05 < nsoffer> seems like lvm issue: libguestfs: error: lvcreate: Volume group "VG1" has insufficient free space (6 extents): 13 required
23:06 < nsoffer> and xfs: libguestfs: error: xfs_admin: /dev/sda1: Cannot disable lazy-counters on V5 fs
Put the attached files into the libguestfs directory,
chmod +x localconfigure, and use ./localconfigure whenever
you would usually use ./configure.
See also:
http://libguestfs.org/guestfs-building.1.html#local-files
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines. Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top
6 years, 4 months
[PATCH] v2v: rhv-upload: Disable Nagle algorithm
by Nir Soffer
When sending a PUT request, the http header may be sent in the first
packet when calling con.endheaders(). When we send the first chunk, the
kernel may delay the send because the header packet was not acked yet.
We have seen PUT requests delayed by 40 milliseconds on the server side
during virt-v2v upload to ovirt. Here is example log from current RHEL
virt-v2v version, uploading to RHV 4.2.3:
2018-06-12 17:04:01,750 INFO (Thread-2) [images] Writing 52736 bytes
at offset 0 flush False to /path/to/image for ticket
374bec27-930d-4097-8e41-e4bc23324eb0
2018-06-12 17:04:01,790 INFO (Thread-2) [directio] Operation stats:
<Clock(total=0.04, read=0.04, write=0.00)>
The server spent 40 milliseconds reading 52736 bytes form
rhv_upload_plugin running on the same host.
This issue was fixed in python 3.5 by using the TCP_NO_DELAY option
after connecting[1]. I backported this change from python 3.5.
I tested the same change using imageio example upload script. With this
change and with optimized PATCH requests, upload time of 4G sparse image
was reduced from 7 minutes to 1 minute.
See this ovirt patch for more details:
https://gerrit.ovirt.org/#/c/92276/
This change is needed only for python 2.
[1] https://bugs.python.org/issue23302
---
v2v/rhv-upload-plugin.py | 14 +++++++++++++-
1 file changed, 13 insertions(+), 1 deletion(-)
diff --git a/v2v/rhv-upload-plugin.py b/v2v/rhv-upload-plugin.py
index ed99cc7a9..f8cd37e9f 100644
--- a/v2v/rhv-upload-plugin.py
+++ b/v2v/rhv-upload-plugin.py
@@ -23,7 +23,7 @@ import ssl
import sys
import time
-from http.client import HTTPSConnection
+from http.client import HTTPSConnection as _HTTPSConnection
from urllib.parse import urlparse
import ovirtsdk4 as sdk
@@ -38,6 +38,18 @@ timeout = 5*60
# is no formal API here.
params = None
+class HTTPSConnection(_HTTPSConnection):
+ def connect(self):
+ """
+ Using TCP_NO_DELAY avoids delays when sending small payload, such as
+ ovirt PATCH requests.
+
+ This issue was fixed in python 3.5, see:
+ https://bugs.python.org/issue23302
+ """
+ _HTTPSConnection.connect(self)
+ self.sock.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
+
def config(key, value):
global params
--
2.17.1
6 years, 4 months
[PATCH] fuse: fix build when not available
by Pino Toscano
The 'localmountpoint' variable in the handle is available only when
building with FUSE support, so guard it in a proper #ifdef block.
Fixes commit 296370fb86e96eec095d86faf6de8f532395ea54.
---
lib/handle.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/lib/handle.c b/lib/handle.c
index bc45d29b2..a47aaafab 100644
--- a/lib/handle.c
+++ b/lib/handle.c
@@ -399,7 +399,9 @@ guestfs_close (guestfs_h *g)
free (g->hv);
free (g->backend);
free (g->backend_data);
+#if HAVE_FUSE
free (g->localmountpoint);
+#endif
guestfs_int_free_string_list (g->backend_settings);
free (g->append);
guestfs_int_free_error_data_list (g);
--
2.17.1
6 years, 4 months
Re: [Libguestfs] Could libguestfs use springfield?
by Richard W.M. Jones
[Adding libguestfs mailing list]
On Thu, Jun 14, 2018 at 04:15:57PM +0200, Vojtěch Trefný wrote:
>
>
> On 06/14/2018 01:15 PM, Richard W.M. Jones wrote:
> >
> >libguestfs provides a C library and large set of tools for
> >manipulating disk images. http://libguestfs.org/
> >
> >As part of this we provide APIs to open VM disks and do things like
> >enumerate partitions or resize logical volumes. The actual way this
> >works currently is we run the external commands (eg. parted, lvresize)
> >inside a small virtual machine and pass the right command line options
> >or parse the output. In some cases we're also parsing stuff out of
> >kernel /sys/block.
> >
> >We've accumulated a large amount of code to do this (I counted 60619
> >lines of code in the current version). Here are a few examples so you
> >can see in concrete terms what I'm talking about:
> >
> >https://github.com/libguestfs/libguestfs/blob/afd1c70601c51043684a0245ce2...
> >
> >https://github.com/libguestfs/libguestfs/blob/afd1c70601c51043684a0245ce2...
> >
> >Steven W pointed me to "Project Springfield" and it sort of looks like
> >it's in the same area. Could libguestfs replace the parsing code
> >above with this?
> >
> >What might be problems: We have no python or dbus in the appliance.
> >So anything that depends on those is a non-starter.
> >
> >TBH the project webpage left me more confused than enlightened. There
> >seem to be lots of projects (subprojects?) doing stuff with odd names
> >and no unifying philosophy, and I'm not sure if Project Springfield is
> >a thing or more of an intention.
> >
> >Rich.
> >
>
> Hi, project springfield is a collection of existing projects,
> libblockdev is probably the "subproject" you are looking for. It is
> a plugin based C library -- we have plugins for working with btrfs,
> filesystems (ext, xfs, vfat and ntfs), lvm, mdraid, partitions etc.
> It mostly also uses the command line tools and in some cases also
> other existing libraries (libcryptsetup, libmount, libparted...).
>
> Libblockdev Github repo: https://github.com/storaged-project/libblockdev
> and API documentation: http://storaged.org/libblockdev/
>
> From the code you sent it looks like libblockdev covers most of the
> functionality libguestfs needs. Some functionality is missing (e.g.
> we don't support changing of uuid for lvs) but we can add these
> missing bits.
>
> --
> Vojtech Trefny
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html
6 years, 4 months
lib: Convert all drive socket parameters to an absolute path
by Richard W.M. Jones
One interesting omission is that we don't allow sockets in the
abstract namespace. The API won't let you pass these sockets because
they contain a '\0' character in the middle of the string. Therefore
this patch doesn't need to deal with those.
However we should in future allow that, probably using the '@'
character to stand in for the NUL byte, as is used in a few other
places (eg. this is used by the ‘netstat’ program).
Rich.
6 years, 4 months
