April 2018 - Libguestfs - libguestfs List Archives

[PATCH v2 0/9] v2v: -i ova: Handle OVAs containing snapshots.
by Richard W.M. Jones 25 Apr '18

25 Apr '18

https://bugzilla.redhat.com/show_bug.cgi?id=1570407 This turned into quite an in-depth refactoring of how we handle OVAs. It also fixes a potential security issue. Rich.

1 9

[PATCH 0/3] v2v: Miscellaneous refactoring and fixes.
by Richard W.M. Jones 23 Apr '18

23 Apr '18

Originally an attempt to fix: https://bugzilla.redhat.com/show_bug.cgi?id=1570407 However this isn't a complete fix. The OVA supplied doesn't even conform to VMware's own "specification" (I use the word loosely). The OVF inside the OVA references the disk.vmdk file, but the OVA doesn't contain that disk.vmdk file, only a snapshot called disk.vmdk.000000000. Therefore to really fix this bug we need some major work in input_ova.ml. Nevertheless these changes make sense on their own. Rich.

1 3

[PATCH 0/1] Install RHV tools with -o rhv-upload
by Tomáš Golembiovský 23 Apr '18

23 Apr '18

It seems we're missing this in rhv-upload. Unfortunately I'm not able to test it at the moment. Tomáš Golembiovský (1): v2v: -o rhv-upload: install RHV tools v2v/output_rhv_upload.ml | 3 +++ 1 file changed, 3 insertions(+) -- 2.17.0

2 2

[PATCH] uefi: remove last references to kraxel's old edk2 builds
by Pino Toscano 21 Apr '18

21 Apr '18

Commit 889386f8c6088e57fd68f2bb6bf85f69ecfc6f6c removed most of them already, so remove the last ones, and the supporting code. --- generator/UEFI.ml | 6 ------ v2v/utils.ml | 1 - 2 files changed, 7 deletions(-) diff --git a/generator/UEFI.ml b/generator/UEFI.ml index 5161e28ee..1ff3f84d2 100644 --- a/generator/UEFI.ml +++ b/generator/UEFI.ml @@ -30,12 +30,6 @@ open Docstrings (* Order is significant *within architectures only*. *) let firmware = [ - "i386", - "/usr/share/edk2.git/ovmf-ia32/OVMF_CODE-pure-efi.fd", - None, - "/usr/share/edk2.git/ovmf-ia32/OVMF_VARS-pure-efi.fd", - []; - "x86_64", "/usr/share/OVMF/OVMF_CODE.fd", None, diff --git a/v2v/utils.ml b/v2v/utils.ml index 58331207f..372ad8aaa 100644 --- a/v2v/utils.ml +++ b/v2v/utils.ml @@ -64,7 +64,6 @@ let find_uefi_firmware guest_arch = let files = (* The lists of firmware are actually defined in common/utils/uefi.c. *) match guest_arch with - | "i386" | "i486" | "i586" | "i686" -> Uefi.uefi_i386_firmware | "x86_64" -> Uefi.uefi_x86_64_firmware | "aarch64" -> Uefi.uefi_aarch64_firmware | arch -> -- 2.14.3

2 1

[PATCH] v2v: linux: correctly update resume=/dev/device entries (RHBZ#1532224)
by Pino Toscano 20 Apr '18

20 Apr '18

When updating the device names to the new names, handle GRUB_CMDLINE entries (typically in grub2 configuration files) correctly: loop over the whole configuration value, and replace each occurrency of resume=/dev/device found. The actual code is moved away from replace_if_device, since this function now does only the actual replacement of the device string. --- v2v/convert_linux.ml | 38 ++++++++++++++++++++++++-------------- 1 file changed, 24 insertions(+), 14 deletions(-) diff --git a/v2v/convert_linux.ml b/v2v/convert_linux.ml index b273785e6..02dc2fee2 100644 --- a/v2v/convert_linux.ml +++ b/v2v/convert_linux.ml @@ -976,7 +976,7 @@ let convert (g : G.guestfs) inspect source output rcaps = List.flatten (List.map Array.to_list (List.map g#aug_match paths)) in (* Map device names for each entry. *) - let rex_resume = PCRE.compile "^(.*resume=)(/dev/\\S+)(.*)$" + let rex_resume = PCRE.compile "^resume=(/dev/[a-z\\d]+)(.*)$" and rex_device_cciss = PCRE.compile "^/dev/(cciss/c\\d+d\\d+)(?:p(\\d+))?$" and rex_device = PCRE.compile "^/dev/([a-z]+)(\\d*)?$" in @@ -994,18 +994,7 @@ let convert (g : G.guestfs) inspect source output rcaps = device in - if String.find path "GRUB_CMDLINE" >= 0 then ( - (* Handle grub2 resume=<dev> specially. *) - if PCRE.matches rex_resume value then ( - let start = PCRE.sub 1 - and device = PCRE.sub 2 - and end_ = PCRE.sub 3 in - let device = replace_if_device path device in - start ^ device ^ end_ - ) - else value - ) - else if PCRE.matches rex_device_cciss value then ( + if PCRE.matches rex_device_cciss value then ( let device = PCRE.sub 1 and part = try PCRE.sub 2 with Not_found -> "" in "/dev/" ^ replace device ^ part @@ -1023,7 +1012,28 @@ let convert (g : G.guestfs) inspect source output rcaps = List.iter ( fun path -> let value = g#aug_get path in - let new_value = replace_if_device path value in + let new_value = + if String.find path "GRUB_CMDLINE" >= 0 then ( + (* Handle grub2 resume=<dev> specially. *) + let rec loop str = + let index = String.find str "resume=" in + if index >= 0 then ( + let part = String.sub str index (String.length str - index) in + if PCRE.matches rex_resume part then ( + let start = String.sub str 0 (index + 7 (* "resume=" *)) + and device = PCRE.sub 1 + and end_ = PCRE.sub 2 in + let device = replace_if_device path device in + start ^ device ^ loop end_ + ) + else str + ) + else str + in + loop value + ) + else + replace_if_device path value in if value <> new_value then ( g#aug_set path new_value; -- 2.14.3

2 1

[PATCH] v2v: rework handling of CPU topology
by Pino Toscano 20 Apr '18

20 Apr '18

Instead of storing the number of sockets, cores, and threads separately with the possibility to have each of them optional, store them together, so either we have a CPU topology or none. This slightly simplifies the handling of CPU topology. Most of the output modes/producers already considered a missing information of a topology as "1" when any of the other was specified, so the behaviour is unchanged. The only behaviour changes are when parsing, and creating libvirt XMLs: - libvirt requires all the three attributes to be specified, so when reading use them only if <topology> has all of them - for the same reason, write <topology> only when all of them are specified Adapt the vmx, and ova input modes to consider 1 threads when they have the other information of a CPU topology, since none of them can provide that information yet. --- v2v/create_libvirt_xml.ml | 28 ++++++++++------------------ v2v/create_ovf.ml | 26 +++++++------------------- v2v/input_disk.ml | 4 +--- v2v/input_ova.ml | 6 ++---- v2v/input_vmx.ml | 13 ++++++------- v2v/output_glance.ml | 27 +++++++++------------------ v2v/output_qemu.ml | 32 ++++++++++++-------------------- v2v/parse_libvirt_xml.ml | 11 ++++++++--- v2v/parse_ovf_from_ova.ml | 13 +++++++------ v2v/parse_ovf_from_ova.mli | 4 ++-- v2v/test-v2v-i-ova-formats.expected | 2 +- v2v/test-v2v-i-ova-gz.expected | 2 +- v2v/test-v2v-i-ova-subfolders.expected | 2 +- v2v/test-v2v-i-ova-subfolders.expected2 | 2 +- v2v/test-v2v-i-ova-tar.expected | 2 +- v2v/test-v2v-i-ova-tar.expected2 | 2 +- v2v/test-v2v-i-ova-two-disks.expected | 2 +- v2v/test-v2v-i-ova-two-disks.expected2 | 2 +- v2v/test-v2v-i-vmx-1.expected | 2 +- v2v/test-v2v-i-vmx-2.expected | 2 +- v2v/test-v2v-i-vmx-3.expected | 2 +- v2v/test-v2v-i-vmx-4.expected | 2 +- v2v/test-v2v-i-vmx-5.expected | 2 +- v2v/types.ml | 22 +++++++++++++++------- v2v/types.mli | 11 ++++++++--- v2v/v2v.ml | 25 +++++++------------------ 26 files changed, 107 insertions(+), 141 deletions(-) diff --git a/v2v/create_libvirt_xml.ml b/v2v/create_libvirt_xml.ml index cbb85cf9a..582419f00 100644 --- a/v2v/create_libvirt_xml.ml +++ b/v2v/create_libvirt_xml.ml @@ -48,8 +48,7 @@ let create_libvirt_xml ?pool source target_buses guestcaps ]; if source.s_cpu_vendor <> None || source.s_cpu_model <> None || - source.s_cpu_sockets <> None || source.s_cpu_cores <> None || - source.s_cpu_threads <> None then ( + source.s_cpu_topology <> None then ( let cpu = ref [] in (match source.s_cpu_vendor with @@ -62,22 +61,15 @@ let create_libvirt_xml ?pool source target_buses guestcaps | Some model -> List.push_back cpu (e "model" ["fallback", "allow"] [PCData model]) ); - if source.s_cpu_sockets <> None || source.s_cpu_cores <> None || - source.s_cpu_threads <> None then ( - let topology_attrs = ref [] in - (match source.s_cpu_sockets with - | None -> () - | Some v -> List.push_back topology_attrs ("sockets", string_of_int v) - ); - (match source.s_cpu_cores with - | None -> () - | Some v -> List.push_back topology_attrs ("cores", string_of_int v) - ); - (match source.s_cpu_threads with - | None -> () - | Some v -> List.push_back topology_attrs ("threads", string_of_int v) - ); - List.push_back cpu (e "topology" !topology_attrs []) + (match source.s_cpu_topology with + | None -> () + | Some { s_cpu_sockets; s_cpu_cores; s_cpu_threads } -> + let topology_attrs = [ + "sockets", string_of_int s_cpu_sockets; + "cores", string_of_int s_cpu_cores; + "threads", string_of_int s_cpu_threads; + ] in + List.push_back cpu (e "topology" topology_attrs []) ); List.push_back_list body [ e "cpu" [ "match", "minimum" ] !cpu ] diff --git a/v2v/create_ovf.ml b/v2v/create_ovf.ml index 625065214..c29b8421c 100644 --- a/v2v/create_ovf.ml +++ b/v2v/create_ovf.ml @@ -605,27 +605,15 @@ let rec create_ovf source targets guestcaps inspect e "rasd:InstanceId" [] [PCData "1"]; e "rasd:ResourceType" [] [PCData "3"] ] @ - if source.s_cpu_sockets <> None || source.s_cpu_cores <> None || - source.s_cpu_threads <> None then ( - let sockets = - match source.s_cpu_sockets with - | None -> "1" - | Some v -> string_of_int v in - let cores = - match source.s_cpu_cores with - | None -> "1" - | Some v -> string_of_int v in - let threads = - match source.s_cpu_threads with - | None -> "1" - | Some v -> string_of_int v in - [ e "rasd:num_of_sockets" [] [PCData sockets]; - e "rasd:cpu_per_socket"[] [PCData cores]; - e "rasd:threads_per_cpu"[] [PCData threads] ] - ) - else ( + (match source.s_cpu_topology with + | None -> [ e "rasd:num_of_sockets" [] [PCData "1"]; e "rasd:cpu_per_socket"[] [PCData (string_of_int source.s_vcpu)] ] + | Some { s_cpu_sockets = sockets; s_cpu_cores = cores; + s_cpu_threads = threads } -> + [ e "rasd:num_of_sockets" [] [PCData (string_of_int sockets)]; + e "rasd:cpu_per_socket"[] [PCData (string_of_int cores)]; + e "rasd:threads_per_cpu"[] [PCData (string_of_int threads)] ] ) ) ); diff --git a/v2v/input_disk.ml b/v2v/input_disk.ml index 56b9c378a..7ecd19fd3 100644 --- a/v2v/input_disk.ml +++ b/v2v/input_disk.ml @@ -83,9 +83,7 @@ class input_disk input_format disk = object s_vcpu = 1; (* 1 vCPU is a safe default *) s_cpu_vendor = None; s_cpu_model = None; - s_cpu_sockets = None; - s_cpu_cores = None; - s_cpu_threads = None; + s_cpu_topology = None; s_features = [ "acpi"; "apic"; "pae" ]; s_firmware = UnknownFirmware; (* causes virt-v2v to autodetect *) s_display = diff --git a/v2v/input_ova.ml b/v2v/input_ova.ml index 4f2480589..a909b92ed 100644 --- a/v2v/input_ova.ml +++ b/v2v/input_ova.ml @@ -245,7 +245,7 @@ object let ovf_folder = Filename.dirname ovf in (* Parse the ovf file. *) - let name, memory, vcpu, cpu_sockets, cpu_cores, firmware, + let name, memory, vcpu, cpu_topology, firmware, disks, removables, nics = parse_ovf_from_ova ovf in @@ -340,9 +340,7 @@ object s_vcpu = vcpu; s_cpu_vendor = None; s_cpu_model = None; - s_cpu_sockets = cpu_sockets; - s_cpu_cores = cpu_cores; - s_cpu_threads = None; (* XXX *) + s_cpu_topology = cpu_topology; s_features = []; (* XXX *) s_firmware = firmware; s_display = None; (* XXX *) diff --git a/v2v/input_vmx.ml b/v2v/input_vmx.ml index a8b33f66f..f79e89139 100644 --- a/v2v/input_vmx.ml +++ b/v2v/input_vmx.ml @@ -436,17 +436,18 @@ object | None -> 1 | Some i -> i in - let cpu_sockets, cpu_cores = + let cpu_topology = match Parse_vmx.get_int vmx ["cpuid"; "coresPerSocket"] with - | None -> None, None + | None -> None | Some cores_per_socket -> let sockets = vcpu / cores_per_socket in if sockets <= 0 then ( warning (f_"invalid cpuid.coresPerSocket < number of vCPUs"); - None, None + None ) else - Some sockets, Some cores_per_socket in + Some { s_cpu_sockets = sockets; s_cpu_cores = cores_per_socket; + s_cpu_threads = 1 } in let firmware = match Parse_vmx.get_string vmx ["firmware"] with @@ -486,9 +487,7 @@ object s_vcpu = vcpu; s_cpu_vendor = None; s_cpu_model = None; - s_cpu_sockets = cpu_sockets; - s_cpu_cores = cpu_cores; - s_cpu_threads = None; + s_cpu_topology = cpu_topology; s_features = []; s_firmware = firmware; s_display = None; diff --git a/v2v/output_glance.ml b/v2v/output_glance.ml index ae75ffcc7..c334def42 100644 --- a/v2v/output_glance.ml +++ b/v2v/output_glance.ml @@ -98,24 +98,15 @@ object | x -> x (* everything else is the same in libguestfs and OpenStack*) ) ] in - if source.s_cpu_sockets <> None || source.s_cpu_cores <> None || - source.s_cpu_threads <> None then ( - List.push_back properties ("hw_cpu_sockets", - match source.s_cpu_sockets with - | None -> "1" - | Some v -> string_of_int v); - List.push_back properties ("hw_cpu_cores", - match source.s_cpu_cores with - | None -> "1" - | Some v -> string_of_int v); - List.push_back properties ("hw_cpu_threads", - match source.s_cpu_threads with - | None -> "1" - | Some v -> string_of_int v); - ) - else ( - List.push_back properties ("hw_cpu_sockets", "1"); - List.push_back properties ("hw_cpu_cores", string_of_int source.s_vcpu); + (match source.s_cpu_topology with + | None -> + List.push_back properties ("hw_cpu_sockets", "1"); + List.push_back properties ("hw_cpu_cores", string_of_int source.s_vcpu); + | Some { s_cpu_sockets = sockets; s_cpu_cores = cores; + s_cpu_threads = threads } -> + List.push_back properties ("hw_cpu_sockets", string_of_int sockets); + List.push_back properties ("hw_cpu_cores", string_of_int cores); + List.push_back properties ("hw_cpu_threads", string_of_int threads); ); (match guestcaps.gcaps_block_bus with | Virtio_SCSI -> diff --git a/v2v/output_qemu.ml b/v2v/output_qemu.ml index ea94f92fd..952660de2 100644 --- a/v2v/output_qemu.ml +++ b/v2v/output_qemu.ml @@ -97,26 +97,18 @@ object arg "-m" (Int64.to_string (source.s_memory /^ 1024L /^ 1024L)); if source.s_vcpu > 1 then ( - if source.s_cpu_sockets <> None || source.s_cpu_cores <> None || - source.s_cpu_threads <> None then ( - let a = ref [] in - List.push_back a (sprintf "cpus=%d" source.s_vcpu); - List.push_back a (sprintf "sockets=%d" - (match source.s_cpu_sockets with - | None -> 1 - | Some v -> v)); - List.push_back a (sprintf "cores=%d" - (match source.s_cpu_cores with - | None -> 1 - | Some v -> v)); - List.push_back a (sprintf "threads=%d" - (match source.s_cpu_threads with - | None -> 1 - | Some v -> v)); - arg_list "-smp" !a - ) - else - arg "-smp" (string_of_int source.s_vcpu); + (match source.s_cpu_topology with + | None -> + arg "-smp" (string_of_int source.s_vcpu) + | Some { s_cpu_sockets; s_cpu_cores; s_cpu_threads } -> + let args = [ + sprintf "cpus=%d" source.s_vcpu; + sprintf "sockets=%d" s_cpu_sockets; + sprintf "cores=%d" s_cpu_cores; + sprintf "threads=%d" s_cpu_threads; + ] in + arg_list "-smp" args + ); ); let make_disk if_name i = function diff --git a/v2v/parse_libvirt_xml.ml b/v2v/parse_libvirt_xml.ml index 6f8a01e54..57e741574 100644 --- a/v2v/parse_libvirt_xml.ml +++ b/v2v/parse_libvirt_xml.ml @@ -115,6 +115,13 @@ let parse_libvirt_xml ?conn xml = and threads = Option.default 1 cpu_threads in sockets * cores * threads in + let cpu_topology = + match cpu_sockets, cpu_cores, cpu_threads with + | Some sockets, Some cores, Some threads -> + Some { s_cpu_sockets = sockets; s_cpu_cores = cores; + s_cpu_threads = threads; } + | _, _, _ -> None in + let features = let features = ref [] in let obj = Xml.xpath_eval_expression xpathctx "/domain/features/*" in @@ -484,9 +491,7 @@ let parse_libvirt_xml ?conn xml = s_vcpu = vcpu; s_cpu_vendor = cpu_vendor; s_cpu_model = cpu_model; - s_cpu_sockets = cpu_sockets; - s_cpu_cores = cpu_cores; - s_cpu_threads = cpu_threads; + s_cpu_topology = cpu_topology; s_features = features; s_firmware = UnknownFirmware; (* XXX until RHBZ#1217444 is fixed *) s_display = display; diff --git a/v2v/parse_ovf_from_ova.ml b/v2v/parse_ovf_from_ova.ml index ab4c56e0f..2ffaf7ae4 100644 --- a/v2v/parse_ovf_from_ova.ml +++ b/v2v/parse_ovf_from_ova.ml @@ -72,21 +72,22 @@ let parse_ovf_from_ova ovf_filename = * I couldn't find out how hyperthreads is specified in the OVF. *) let cores_per_socket = xpath_int "/ovf:Envelope/ovf:VirtualSystem/ovf:VirtualHardwareSection/ovf:Item[rasd:ResourceType/text()=3]/vmw:CoresPerSocket/text()" in - let cpu_sockets, cpu_cores = + let cpu_topology = match cores_per_socket with - | None -> None, None + | None -> None | Some cores_per_socket when cores_per_socket <= 0 -> warning (f_"invalid vmw:CoresPerSocket (%d) ignored") cores_per_socket; - None, None + None | Some cores_per_socket -> let sockets = vcpu / cores_per_socket in if sockets <= 0 then ( warning (f_"invalid vmw:CoresPerSocket < number of cores"); - None, None + None ) else - Some sockets, Some cores_per_socket in + Some { s_cpu_sockets = sockets; s_cpu_cores = cores_per_socket; + s_cpu_threads = 1 } in (* BIOS or EFI firmware? *) let firmware = Option.default "bios" (xpath_string "/ovf:Envelope/ovf:VirtualSystem/ovf:VirtualHardwareSection/vmw:Config[@vmw:key=\"firmware\"]/@vmw:value") in @@ -97,7 +98,7 @@ let parse_ovf_from_ova ovf_filename = | s -> error (f_"unknown Config:firmware value %s (expected \"bios\" or \"efi\")") s in - name, memory, vcpu, cpu_sockets, cpu_cores, firmware, + name, memory, vcpu, cpu_topology, firmware, parse_disks (), parse_removables (), parse_nics () (* Helper function to return the parent controller of a disk. *) diff --git a/v2v/parse_ovf_from_ova.mli b/v2v/parse_ovf_from_ova.mli index 632bd419b..39bc83d2d 100644 --- a/v2v/parse_ovf_from_ova.mli +++ b/v2v/parse_ovf_from_ova.mli @@ -29,9 +29,9 @@ type ovf_disk = { } (** A VMDK disk from a parsed OVF. *) -val parse_ovf_from_ova : string -> string option * int64 * int * int option * int option * Types.source_firmware * ovf_disk list * Types.source_removable list * Types.source_nic list +val parse_ovf_from_ova : string -> string option * int64 * int * Types.source_cpu_topology option * Types.source_firmware * ovf_disk list * Types.source_removable list * Types.source_nic list (** Parse an OVF file. The returned tuple is - [name, memory, vcpu, cpu_sockets, cpu_cores, firmware, + [name, memory, vcpu, cpu_topology, firmware, disks, removables, nics] *) diff --git a/v2v/test-v2v-i-ova-formats.expected b/v2v/test-v2v-i-ova-formats.expected index f7b79d7bc..97bce58ad 100644 --- a/v2v/test-v2v-i-ova-formats.expected +++ b/v2v/test-v2v-i-ova-formats.expected @@ -6,7 +6,7 @@ hypervisor type: vmware nr vCPUs: 1 CPU vendor: CPU model: - CPU topology: sockets: - cores/socket: - threads/core: - + CPU topology: CPU features: firmware: uefi display: diff --git a/v2v/test-v2v-i-ova-gz.expected b/v2v/test-v2v-i-ova-gz.expected index 9cf048f30..f1f79e309 100644 --- a/v2v/test-v2v-i-ova-gz.expected +++ b/v2v/test-v2v-i-ova-gz.expected @@ -6,7 +6,7 @@ hypervisor type: vmware nr vCPUs: 1 CPU vendor: CPU model: - CPU topology: sockets: - cores/socket: - threads/core: - + CPU topology: CPU features: firmware: bios display: diff --git a/v2v/test-v2v-i-ova-subfolders.expected b/v2v/test-v2v-i-ova-subfolders.expected index 44ec3b2b8..701458b8c 100644 --- a/v2v/test-v2v-i-ova-subfolders.expected +++ b/v2v/test-v2v-i-ova-subfolders.expected @@ -6,7 +6,7 @@ hypervisor type: vmware nr vCPUs: 1 CPU vendor: CPU model: - CPU topology: sockets: - cores/socket: - threads/core: - + CPU topology: CPU features: firmware: uefi display: diff --git a/v2v/test-v2v-i-ova-subfolders.expected2 b/v2v/test-v2v-i-ova-subfolders.expected2 index f42fc924f..8be8d5615 100644 --- a/v2v/test-v2v-i-ova-subfolders.expected2 +++ b/v2v/test-v2v-i-ova-subfolders.expected2 @@ -6,7 +6,7 @@ hypervisor type: vmware nr vCPUs: 1 CPU vendor: CPU model: - CPU topology: sockets: - cores/socket: - threads/core: - + CPU topology: CPU features: firmware: uefi display: diff --git a/v2v/test-v2v-i-ova-tar.expected b/v2v/test-v2v-i-ova-tar.expected index f7b79d7bc..97bce58ad 100644 --- a/v2v/test-v2v-i-ova-tar.expected +++ b/v2v/test-v2v-i-ova-tar.expected @@ -6,7 +6,7 @@ hypervisor type: vmware nr vCPUs: 1 CPU vendor: CPU model: - CPU topology: sockets: - cores/socket: - threads/core: - + CPU topology: CPU features: firmware: uefi display: diff --git a/v2v/test-v2v-i-ova-tar.expected2 b/v2v/test-v2v-i-ova-tar.expected2 index 83928c23e..7f3559339 100644 --- a/v2v/test-v2v-i-ova-tar.expected2 +++ b/v2v/test-v2v-i-ova-tar.expected2 @@ -6,7 +6,7 @@ hypervisor type: vmware nr vCPUs: 1 CPU vendor: CPU model: - CPU topology: sockets: - cores/socket: - threads/core: - + CPU topology: CPU features: firmware: uefi display: diff --git a/v2v/test-v2v-i-ova-two-disks.expected b/v2v/test-v2v-i-ova-two-disks.expected index 61eab9cf6..a21153f3f 100644 --- a/v2v/test-v2v-i-ova-two-disks.expected +++ b/v2v/test-v2v-i-ova-two-disks.expected @@ -6,7 +6,7 @@ hypervisor type: vmware nr vCPUs: 1 CPU vendor: CPU model: - CPU topology: sockets: - cores/socket: - threads/core: - + CPU topology: CPU features: firmware: bios display: diff --git a/v2v/test-v2v-i-ova-two-disks.expected2 b/v2v/test-v2v-i-ova-two-disks.expected2 index 5a218f5d5..5f0df0625 100644 --- a/v2v/test-v2v-i-ova-two-disks.expected2 +++ b/v2v/test-v2v-i-ova-two-disks.expected2 @@ -6,7 +6,7 @@ hypervisor type: vmware nr vCPUs: 1 CPU vendor: CPU model: - CPU topology: sockets: - cores/socket: - threads/core: - + CPU topology: CPU features: firmware: bios display: diff --git a/v2v/test-v2v-i-vmx-1.expected b/v2v/test-v2v-i-vmx-1.expected index 902a850c5..3314021ad 100644 --- a/v2v/test-v2v-i-vmx-1.expected +++ b/v2v/test-v2v-i-vmx-1.expected @@ -7,7 +7,7 @@ hypervisor type: vmware nr vCPUs: 1 CPU vendor: CPU model: - CPU topology: sockets: - cores/socket: - threads/core: - + CPU topology: CPU features: firmware: bios display: diff --git a/v2v/test-v2v-i-vmx-2.expected b/v2v/test-v2v-i-vmx-2.expected index 6e87862ac..d7fbafb68 100644 --- a/v2v/test-v2v-i-vmx-2.expected +++ b/v2v/test-v2v-i-vmx-2.expected @@ -7,7 +7,7 @@ hypervisor type: vmware nr vCPUs: 1 CPU vendor: CPU model: - CPU topology: sockets: - cores/socket: - threads/core: - + CPU topology: CPU features: firmware: bios display: diff --git a/v2v/test-v2v-i-vmx-3.expected b/v2v/test-v2v-i-vmx-3.expected index 0d1585b96..b4dcc6283 100644 --- a/v2v/test-v2v-i-vmx-3.expected +++ b/v2v/test-v2v-i-vmx-3.expected @@ -7,7 +7,7 @@ hypervisor type: vmware nr vCPUs: 1 CPU vendor: CPU model: - CPU topology: sockets: - cores/socket: - threads/core: - + CPU topology: CPU features: firmware: uefi display: diff --git a/v2v/test-v2v-i-vmx-4.expected b/v2v/test-v2v-i-vmx-4.expected index 06f58d7ad..5c45fca8d 100644 --- a/v2v/test-v2v-i-vmx-4.expected +++ b/v2v/test-v2v-i-vmx-4.expected @@ -7,7 +7,7 @@ hypervisor type: vmware nr vCPUs: 1 CPU vendor: CPU model: - CPU topology: sockets: - cores/socket: - threads/core: - + CPU topology: CPU features: firmware: bios display: diff --git a/v2v/test-v2v-i-vmx-5.expected b/v2v/test-v2v-i-vmx-5.expected index 192bc441e..f94306536 100644 --- a/v2v/test-v2v-i-vmx-5.expected +++ b/v2v/test-v2v-i-vmx-5.expected @@ -7,7 +7,7 @@ hypervisor type: vmware nr vCPUs: 1 CPU vendor: CPU model: - CPU topology: sockets: 1 cores/socket: 1 threads/core: - + CPU topology: sockets: 1 cores/socket: 1 threads/core: 1 CPU features: firmware: bios display: diff --git a/v2v/types.ml b/v2v/types.ml index 789ae69b9..57bbb5bd2 100644 --- a/v2v/types.ml +++ b/v2v/types.ml @@ -32,9 +32,7 @@ type source = { s_vcpu : int; s_cpu_vendor : string option; s_cpu_model : string option; - s_cpu_sockets : int option; - s_cpu_cores : int option; - s_cpu_threads : int option; + s_cpu_topology : source_cpu_topology option; s_features : string list; s_firmware : source_firmware; s_display : source_display option; @@ -102,6 +100,11 @@ and source_sound = { } and source_sound_model = AC97 | ES1370 | ICH6 | ICH9 | PCSpeaker | SB16 | USBAudio +and source_cpu_topology = { + s_cpu_sockets : int; + s_cpu_cores : int; + s_cpu_threads : int; +} let rec string_of_source s = sprintf " source name: %s @@ -110,7 +113,7 @@ hypervisor type: %s nr vCPUs: %d CPU vendor: %s CPU model: %s - CPU topology: sockets: %s cores/socket: %s threads/core: %s + CPU topology: %s CPU features: %s firmware: %s display: %s @@ -129,9 +132,9 @@ NICs: s.s_vcpu (Option.default "" s.s_cpu_vendor) (Option.default "" s.s_cpu_model) - (match s.s_cpu_sockets with None -> "-" | Some v -> string_of_int v) - (match s.s_cpu_cores with None -> "-" | Some v -> string_of_int v) - (match s.s_cpu_threads with None -> "-" | Some v -> string_of_int v) + (match s.s_cpu_topology with + | None -> "" + | Some topology -> string_of_source_cpu_topology topology) (String.concat "," s.s_features) (string_of_source_firmware s.s_firmware) (match s.s_display with @@ -273,6 +276,11 @@ and string_of_source_sound_model = function | SB16 -> "sb16" | USBAudio -> "usb" +and string_of_source_cpu_topology { s_cpu_sockets; s_cpu_cores; + s_cpu_threads } = + sprintf "sockets: %d cores/socket: %d threads/core: %d" + s_cpu_sockets s_cpu_cores s_cpu_threads + type overlay = { ov_overlay_file : string; ov_sd : string; diff --git a/v2v/types.mli b/v2v/types.mli index fa2b402b4..291707b74 100644 --- a/v2v/types.mli +++ b/v2v/types.mli @@ -66,9 +66,7 @@ type source = { s_vcpu : int; (** Number of CPUs. *) s_cpu_vendor : string option; (** Source CPU vendor. *) s_cpu_model : string option; (** Source CPU model. *) - s_cpu_sockets : int option; (** Number of sockets. *) - s_cpu_cores : int option; (** Number of cores per socket. *) - s_cpu_threads : int option; (** Number of threads per core. *) + s_cpu_topology : source_cpu_topology option; (** Source CPU topology. *) s_features : string list; (** Machine features. *) s_firmware : source_firmware; (** Firmware (BIOS or EFI). *) s_display : source_display option; (** Guest display. *) @@ -160,12 +158,19 @@ and source_sound = { and source_sound_model = AC97 | ES1370 | ICH6 | ICH9 | PCSpeaker | SB16 | USBAudio +and source_cpu_topology = { + s_cpu_sockets : int; (** Number of sockets. *) + s_cpu_cores : int; (** Number of cores per socket. *) + s_cpu_threads : int; (** Number of threads per core. *) +} + val string_of_source : source -> string val string_of_source_disk : source_disk -> string val string_of_controller : s_controller -> string val string_of_nic_model : s_nic_model -> string val string_of_source_sound_model : source_sound_model -> string val string_of_source_video : source_video -> string +val string_of_source_cpu_topology : source_cpu_topology -> string val string_of_source_hypervisor : source_hypervisor -> string val source_hypervisor_of_string : string -> source_hypervisor diff --git a/v2v/v2v.ml b/v2v/v2v.ml index ce48e99f2..4b5d25ad0 100644 --- a/v2v/v2v.ml +++ b/v2v/v2v.ml @@ -220,24 +220,13 @@ and open_source cmdline input = assert (source.s_vcpu >= 1); assert (source.s_cpu_vendor <> Some ""); assert (source.s_cpu_model <> Some ""); - (match source.s_cpu_sockets with - | None -> () - | Some i when i > 0 -> () - | _ -> assert false); - (match source.s_cpu_cores with - | None -> () - | Some i when i > 0 -> () - | _ -> assert false); - (match source.s_cpu_threads with - | None -> () - | Some i when i > 0 -> () - | _ -> assert false); - (match source.s_cpu_sockets, source.s_cpu_cores, source.s_cpu_threads with - | None, None, None -> () (* no topology specified *) - | sockets, cores, threads -> - let sockets = Option.default 1 sockets - and cores = Option.default 1 cores - and threads = Option.default 1 threads in + (match source.s_cpu_topology with + | None -> () (* no topology specified *) + | Some { s_cpu_sockets = sockets; s_cpu_cores = cores; + s_cpu_threads = threads } -> + assert (sockets > 0); + assert (cores > 0); + assert (threads > 0); let expected_vcpu = sockets * cores * threads in if expected_vcpu <> source.s_vcpu then warning (f_"source sockets * cores * threads <> number of vCPUs.\nSockets %d * cores per socket %d * threads %d = %d, but number of vCPUs = %d.\n\nThis is a problem with either the source metadata or the virt-v2v input module. In some circumstances this could stop the guest from booting on the target.") -- 2.14.3

2 1

[nbdkit PATCH v2 0/5] FUA support in Python scripts
by Eric Blake 19 Apr '18

19 Apr '18

First out of our four language bindings to add FUA support (for reference, I added 'zero' support for python, perl, and ruby back in 1.1.13, then Rich had to add it for ocaml in 1.1.20). I tested this heavily under python 2, but for now only compile tested under python 3; I plan to do further testing there and make any tweaks if necessary. I wrote patch 5 early on, but then realized I probably don't have a use for it if we fix the C code to cache for all plugins, so it's more for information than for applying. (It would matter if we ever had any per-connection state beyond the script's handle, and which would not also be cached on our behalf by the main nbdkit code). Something I thought of while writing patch 4, but haven't explored yet, would be using attribute((cleanup)) to automatically call Py_XDECREF() in a lot more places, for more compact code (it felt like a lot of boilerplate to avoid leaks of python objects on all exit paths). Eric Blake (5): python: Let zero's may_trim parameter be optional python: Expose can_zero callback python: Update internals to plugin API level 2 python: Expose FUA support RFC: python: Track and cache per-connection state in C struct plugins/python/nbdkit-python-plugin.pod | 85 ++++++- plugins/python/python.c | 388 ++++++++++++++++++++++++++------ plugins/python/example.py | 6 +- tests/test.py | 6 +- 4 files changed, 408 insertions(+), 77 deletions(-) -- 2.14.3

2 13

[nbdkit PATCH 0/2] Fix testsuite deadlocks during close
by Eric Blake 19 Apr '18

19 Apr '18

Commit 9e6d990f exposed a pre-existing deadlock between the nbd plugin as client and parallel nbdkit as server. Prior to that commit, the deadlock was "resolved" because we unloaded the .so in parallel to a .close callback that never completed (yes, it's nasty that it usually? let the testsuite pass), but now we correctly refuse to unload a plugin that has not returned from .close, which makes 'test-nbd' and 'test-parallel-nbd.sh' hang. Here are two patches which each independently break the deadlock, but where both patches should be applied so that we are robust to non-nbdkit partners that exhibit the same behaviors as our unpatched deadlock. Since testsuite hangs make automated testing less than useful, I'm going ahead and pushing them now. Eric Blake (2): nbd: Inform server of intent to quit nbd: Don't read after client sends NBD_CMD_DISC src/connections.c | 5 +++-- plugins/nbd/nbd.c | 4 +++- 2 files changed, 6 insertions(+), 3 deletions(-) -- 2.14.3

2 3

[PATCH nbdkit 0/2] connections: Protect open and close callbacks with the request lock.
by Richard W.M. Jones 18 Apr '18

18 Apr '18

I'm fairly sure that these bugs which appear in the Python plugin: https://bugzilla.redhat.com/show_bug.cgi?id=1566516 https://bugzilla.redhat.com/show_bug.cgi?id=1566522 are really bugs in the SERIALIZE_ALL_REQUESTS thread model. See the first patch for the full explanation. The second patch is a fix for a race condition which is probably nudged into being by the first patch. Now this all works, EXCEPT it quite reliably breaks either the NBD plugin or ‘tests/test-parallel-nbd.sh’. I have no idea why because as far as I know the change should not affect the NBD plugin or that test in any way. Unfortunately both are rather complex to debug. Rich.

2 3

[nbdkit PATCH v3 0/2] Add nbdkit_realpath
by Pino Toscano 17 Apr '18

17 Apr '18

Hi, this is the v3 of my series for nbdkit_realpath; this series adds an extra documentation update to nbdkit_absolute_path, documenting when it can only be used, and then adds nbdkit_realpath. Thanks, Pino Toscano (2): docs: improve documentation of nbdkit_absolute_path plugin: add and use nbdkit_realpath docs/nbdkit-plugin.pod | 24 +++++++++++++++++++++++- include/nbdkit-common.h | 1 + plugins/example2/example2.c | 2 +- plugins/file/file.c | 6 +----- plugins/gzip/gzip.c | 2 +- plugins/split/split.c | 2 +- plugins/vddk/vddk.c | 2 +- plugins/xz/xz.c | 2 +- src/plugins.c | 2 +- src/utils.c | 19 +++++++++++++++++++ 10 files changed, 50 insertions(+), 12 deletions(-) -- 2.14.3

2 4