[nbdkit] Plan for nbdkit 1.4?
by Richard W.M. Jones
I'd like to suggest that we release nbdkit 1.4 (next stable branch) soon.
The last stable branch was effectively released in January this year
(although I didn't actually implement the stable/development branch
policy until April).
There has been a lot of development since then:
- Switch to defaulting to newstyle protocol.
- On-demand ramping of thread pool.
- TRIM support in the file plugin.
- nbdkit_parse_size rewritten.
- Test IPv6 connections.
- Reworked error handling.
- New log filter.
- New blocksize filter.
- New nozero filter.
- New fua filter.
- New can_zero, can_fua methods and better handling of FUA.
- New nbdkit_realpath function.
- Better handling of shutdown.
- New ext2 plugin.
- Bash tab completion.
- New zero plugin.
- New random plugin.
- New PKG_CHECK_VAR variables.
- TLS-PSK authentication.
- New Tcl plugin (posted for review).
- Of course numerous smaller bug fixes and improvements.
I'm also going to propose that we try to get 1.4 into RHEL 8, which
means we will need to do this branching fairly quickly, in the next
few weeks.
Let me know what you think.
Thanks,
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW
6 years
[PATCH v2] v2v: rhv plugin - find suitable host
by Daniel Erez
For direct upload, a suitable host must be in status 'Up'
and belong to the same datacenter as the created disk.
Added these criteria to the host search query.
---
v2v/rhv-upload-plugin.py | 28 ++++++++++++++++++++++++----
1 file changed, 24 insertions(+), 4 deletions(-)
diff --git a/v2v/rhv-upload-plugin.py b/v2v/rhv-upload-plugin.py
index da309e288..931fcfaa2 100644
--- a/v2v/rhv-upload-plugin.py
+++ b/v2v/rhv-upload-plugin.py
@@ -69,14 +69,34 @@ def find_host(connection):
debug("hw_id = %r" % vdsm_id)
- hosts_service = connection.system_service().hosts_service()
+ system_service = connection.system_service()
+ storage_name = params['output_storage']
+ data_centers = system_service.data_centers_service().list(
+ search='storage=%s' % storage_name,
+ case_sensitive=False,
+ )
+ if len(data_centers) == 0:
+ # The storage domain is not attached to a datacenter
+ # (shouldn't happen, would fail on disk creation).
+ debug("storange domain (%s) is not attached to a DC" % storage_name)
+ return None
+
+ datacenter = data_centers[0]
+ debug("datacenter = %s" % datacenter.name)
+
+ hosts_service = system_service.hosts_service()
hosts = hosts_service.list(
- search="hw_id=%s" % vdsm_id,
+ search="hw_id=%s and datacenter=%s and status=Up" % (vdsm_id, datacenter.name),
case_sensitive=False,
)
if len(hosts) == 0:
- # This oVirt host is not registered with engine.
- debug("cannot find host with hw_id=%r, using any host" % vdsm_id)
+ # Couldn't find a host that's fulfilling the following criteria:
+ # - 'hw_id' equals to 'vdsm_id'
+ # - Its status is 'Up'
+ # - Belongs to the storage domain's datacenter
+ debug("cannot find a running host with hw_id=%r, " \
+ "that belongs to datacenter '%s', " \
+ "using any host" % (vdsm_id, datacenter.name))
return None
host = hosts[0]
--
2.17.1
6 years
Re: [Libguestfs] [PATCH] v2v: Preserve VM Generation ID (RHBZ#1598350).
by Michal Skrivanek
On 5 Jul 2018, at 16:15, Richard W.M. Jones <rjones(a)redhat.com> wrote:
On Thu, Jul 05, 2018 at 03:57:43PM +0200, Fabien Dupont wrote:
Does genid exist in oVirt ? If yes, is it already preserved or should it be
a new patch ?
This patch does not preserve it for any RHV output mode:
We are missing support currently for all OVF-based formats (hence
‘-i ova’, and all oVirt/RHV output modes). It's unclear where we
would store the genid in this format.
However it's a very good question and I defer to Tomáš and Michal,
although I believe they are both on holiday for the rest of the week.
RHV doesn’t support it at all yet.
There’s an RFE[1] but it is not targeted to any version.
Full support would need to be prioritized.
Thanks,
michal
[1] https://bugzilla.redhat.com/show_bug.cgi?id=1118825
Rich.
--
Richard Jones, Virtualization Group, Red Hat
http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines. Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v
6 years
[PATCH] v2v: Preserve VM Generation ID (RHBZ#1598350).
by Richard W.M. Jones
virt-v2v moves guests, it doesn't clone them. Therefore we should try
to preserve the VM Generation ID (genid) as much as possible.
This has the ability to read the genid from VMware VMX files and
libvirt XML (but note RHBZ#1598348). It can also write the genid to
libvirt (‘-o libvirt’, ‘-o local’) and QEMU (‘-o qemu’).
We are missing support currently for all OVF-based formats (hence
‘-i ova’, and all oVirt/RHV output modes). It's unclear where we
would store the genid in this format.
---
v2v/create_libvirt_xml.ml | 5 +++++
v2v/input_disk.ml | 1 +
v2v/input_ova.ml | 1 +
v2v/input_vmx.ml | 22 ++++++++++++++++++++++
v2v/output_qemu.ml | 7 +++++++
v2v/parse_libvirt_xml.ml | 5 +++++
v2v/test-v2v-i-ova-formats.expected | 1 +
v2v/test-v2v-i-ova-gz.expected | 1 +
v2v/test-v2v-i-ova-snapshots.expected | 1 +
v2v/test-v2v-i-ova-snapshots.expected2 | 1 +
v2v/test-v2v-i-ova-subfolders.expected | 1 +
v2v/test-v2v-i-ova-subfolders.expected2 | 1 +
v2v/test-v2v-i-ova-tar.expected | 1 +
v2v/test-v2v-i-ova-tar.expected2 | 1 +
v2v/test-v2v-i-ova-two-disks.expected | 1 +
v2v/test-v2v-i-ova-two-disks.expected2 | 1 +
v2v/test-v2v-i-vmx-1.expected | 1 +
v2v/test-v2v-i-vmx-2.expected | 1 +
v2v/test-v2v-i-vmx-3.expected | 1 +
v2v/test-v2v-i-vmx-4.expected | 1 +
v2v/test-v2v-i-vmx-5.expected | 1 +
v2v/test-v2v-print-source.expected | 1 +
v2v/types.ml | 3 +++
v2v/types.mli | 1 +
24 files changed, 61 insertions(+)
diff --git a/v2v/create_libvirt_xml.ml b/v2v/create_libvirt_xml.ml
index f5603db99..b9b248311 100644
--- a/v2v/create_libvirt_xml.ml
+++ b/v2v/create_libvirt_xml.ml
@@ -40,6 +40,11 @@ let create_libvirt_xml ?pool source target_buses guestcaps
e "name" [] [PCData source.s_name];
];
+ (match source.s_genid with
+ | None -> ()
+ | Some genid -> List.push_back body (e "genid" [] [PCData genid])
+ );
+
let memory_k = source.s_memory /^ 1024L in
List.push_back_list body [
e "memory" ["unit", "KiB"] [PCData (Int64.to_string memory_k)];
diff --git a/v2v/input_disk.ml b/v2v/input_disk.ml
index 624644532..43b901a88 100644
--- a/v2v/input_disk.ml
+++ b/v2v/input_disk.ml
@@ -80,6 +80,7 @@ class input_disk input_format disk = object
let source = {
s_hypervisor = UnknownHV;
s_name = name; s_orig_name = name;
+ s_genid = None;
s_memory = 2048L *^ 1024L *^ 1024L; (* 2048 MB *)
s_vcpu = 1; (* 1 vCPU is a safe default *)
s_cpu_vendor = None;
diff --git a/v2v/input_ova.ml b/v2v/input_ova.ml
index c4711f902..bfd72dee4 100644
--- a/v2v/input_ova.ml
+++ b/v2v/input_ova.ml
@@ -189,6 +189,7 @@ class input_ova ova = object
s_hypervisor = VMware;
s_name = name;
s_orig_name = name;
+ s_genid = None; (* XXX *)
s_memory = memory;
s_vcpu = vcpu;
s_cpu_vendor = None;
diff --git a/v2v/input_vmx.ml b/v2v/input_vmx.ml
index 1a8015545..a5caf906b 100644
--- a/v2v/input_vmx.ml
+++ b/v2v/input_vmx.ml
@@ -426,6 +426,27 @@ object
| File filename -> name_from_disk filename
| SSH uri -> name_from_disk (path_of_uri uri) in
+ let genid =
+ (* XXX NOT yet tested against VMware. *)
+ let genid = Parse_vmx.get_int64 vmx ["vm"; "genid"]
+ and genidX = Parse_vmx.get_int64 vmx ["vm"; "genidX"] in
+ match genid, genidX with
+ | None, None | Some _, None | None, Some _ -> None
+ | Some lo, Some hi ->
+ (* The actual mapping from the two integers to the UUID
+ * (as defined by qemu and used by libvirt) is very complex.
+ * This code was determined empirically. See also:
+ * https://lists.nongnu.org/archive/html/qemu-devel/2018-07/msg01505.html
+ *)
+ let sub = String.sub (sprintf "%016Lx%016Lx" lo hi) in
+ let uuid =
+ sub 8 8 ^ "-" ^
+ sub 4 4 ^ "-" ^
+ sub 0 4 ^ "-" ^
+ sub 30 2 ^ sub 28 2 ^ "-" ^
+ sub 26 2 ^ sub 24 2 ^ sub 22 2 ^ sub 20 2 ^ sub 18 2 ^ sub 16 2 in
+ Some uuid in
+
let memory_mb =
match Parse_vmx.get_int64 vmx ["memSize"] with
| None -> 32_L (* default is really 32 MB! *)
@@ -483,6 +504,7 @@ object
let source = {
s_hypervisor = VMware;
s_name = name;
+ s_genid = genid;
s_orig_name = name;
s_memory = memory;
s_vcpu = vcpu;
diff --git a/v2v/output_qemu.ml b/v2v/output_qemu.ml
index 487363520..63b411085 100644
--- a/v2v/output_qemu.ml
+++ b/v2v/output_qemu.ml
@@ -82,6 +82,13 @@ object
flag "-no-user-config"; flag "-nodefaults";
arg "-name" source.s_name;
+
+ (match source.s_genid with
+ | None -> ()
+ | Some genid ->
+ arg_list "-device" ["vmgenid"; sprintf "guid=%s" genid; "id=vmgenid0"]
+ );
+
arg_list "-machine" (machine ::
(if smm then ["smm=on"] else []) @
["accel=kvm:tcg"]);
diff --git a/v2v/parse_libvirt_xml.ml b/v2v/parse_libvirt_xml.ml
index cf6593043..36d2f66dd 100644
--- a/v2v/parse_libvirt_xml.ml
+++ b/v2v/parse_libvirt_xml.ml
@@ -90,6 +90,10 @@ let parse_libvirt_xml ?conn xml =
| None | Some "" ->
error (f_"in the libvirt XML metadata, <name> is missing or empty")
| Some s -> s in
+ let genid =
+ match xpath_string "/domain/genid/text()" with
+ | None | Some "" -> None
+ | Some _ as s -> s in
let memory =
Option.default (1024L *^ 1024L) (xpath_int64 "/domain/memory/text()") in
let memory = memory *^ 1024L in
@@ -481,6 +485,7 @@ let parse_libvirt_xml ?conn xml =
({
s_hypervisor = hypervisor;
s_name = name; s_orig_name = name;
+ s_genid = genid;
s_memory = memory;
s_vcpu = vcpu;
s_cpu_vendor = cpu_vendor;
diff --git a/v2v/test-v2v-i-ova-formats.expected b/v2v/test-v2v-i-ova-formats.expected
index 97bce58ad..697eaae51 100644
--- a/v2v/test-v2v-i-ova-formats.expected
+++ b/v2v/test-v2v-i-ova-formats.expected
@@ -2,6 +2,7 @@ Source guest information (--print-source option):
source name: 2K8R2EESP1_2_Medium
hypervisor type: vmware
+ VM genid:
memory: 1073741824 (bytes)
nr vCPUs: 1
CPU vendor:
diff --git a/v2v/test-v2v-i-ova-gz.expected b/v2v/test-v2v-i-ova-gz.expected
index f1f79e309..ec2610007 100644
--- a/v2v/test-v2v-i-ova-gz.expected
+++ b/v2v/test-v2v-i-ova-gz.expected
@@ -2,6 +2,7 @@ Source guest information (--print-source option):
source name: 2K8R2EESP1_2_Medium
hypervisor type: vmware
+ VM genid:
memory: 1073741824 (bytes)
nr vCPUs: 1
CPU vendor:
diff --git a/v2v/test-v2v-i-ova-snapshots.expected b/v2v/test-v2v-i-ova-snapshots.expected
index a64843b35..8908de03b 100644
--- a/v2v/test-v2v-i-ova-snapshots.expected
+++ b/v2v/test-v2v-i-ova-snapshots.expected
@@ -2,6 +2,7 @@ Source guest information (--print-source option):
source name: 2K8R2EESP1_2_Medium
hypervisor type: vmware
+ VM genid:
memory: 1073741824 (bytes)
nr vCPUs: 1
CPU vendor:
diff --git a/v2v/test-v2v-i-ova-snapshots.expected2 b/v2v/test-v2v-i-ova-snapshots.expected2
index 45be3cc46..58e985940 100644
--- a/v2v/test-v2v-i-ova-snapshots.expected2
+++ b/v2v/test-v2v-i-ova-snapshots.expected2
@@ -2,6 +2,7 @@ Source guest information (--print-source option):
source name: 2K8R2EESP1_2_Medium
hypervisor type: vmware
+ VM genid:
memory: 1073741824 (bytes)
nr vCPUs: 1
CPU vendor:
diff --git a/v2v/test-v2v-i-ova-subfolders.expected b/v2v/test-v2v-i-ova-subfolders.expected
index 701458b8c..6e40297d5 100644
--- a/v2v/test-v2v-i-ova-subfolders.expected
+++ b/v2v/test-v2v-i-ova-subfolders.expected
@@ -2,6 +2,7 @@ Source guest information (--print-source option):
source name: 2K8R2EESP1_2_Medium
hypervisor type: vmware
+ VM genid:
memory: 1073741824 (bytes)
nr vCPUs: 1
CPU vendor:
diff --git a/v2v/test-v2v-i-ova-subfolders.expected2 b/v2v/test-v2v-i-ova-subfolders.expected2
index 8be8d5615..e0dd1937d 100644
--- a/v2v/test-v2v-i-ova-subfolders.expected2
+++ b/v2v/test-v2v-i-ova-subfolders.expected2
@@ -2,6 +2,7 @@ Source guest information (--print-source option):
source name: 2K8R2EESP1_2_Medium
hypervisor type: vmware
+ VM genid:
memory: 1073741824 (bytes)
nr vCPUs: 1
CPU vendor:
diff --git a/v2v/test-v2v-i-ova-tar.expected b/v2v/test-v2v-i-ova-tar.expected
index 97bce58ad..697eaae51 100644
--- a/v2v/test-v2v-i-ova-tar.expected
+++ b/v2v/test-v2v-i-ova-tar.expected
@@ -2,6 +2,7 @@ Source guest information (--print-source option):
source name: 2K8R2EESP1_2_Medium
hypervisor type: vmware
+ VM genid:
memory: 1073741824 (bytes)
nr vCPUs: 1
CPU vendor:
diff --git a/v2v/test-v2v-i-ova-tar.expected2 b/v2v/test-v2v-i-ova-tar.expected2
index 7f3559339..1fa120b99 100644
--- a/v2v/test-v2v-i-ova-tar.expected2
+++ b/v2v/test-v2v-i-ova-tar.expected2
@@ -2,6 +2,7 @@ Source guest information (--print-source option):
source name: 2K8R2EESP1_2_Medium
hypervisor type: vmware
+ VM genid:
memory: 1073741824 (bytes)
nr vCPUs: 1
CPU vendor:
diff --git a/v2v/test-v2v-i-ova-two-disks.expected b/v2v/test-v2v-i-ova-two-disks.expected
index a21153f3f..1bd23af67 100644
--- a/v2v/test-v2v-i-ova-two-disks.expected
+++ b/v2v/test-v2v-i-ova-two-disks.expected
@@ -2,6 +2,7 @@ Source guest information (--print-source option):
source name: 2K8R2EESP1_2_Medium
hypervisor type: vmware
+ VM genid:
memory: 1073741824 (bytes)
nr vCPUs: 1
CPU vendor:
diff --git a/v2v/test-v2v-i-ova-two-disks.expected2 b/v2v/test-v2v-i-ova-two-disks.expected2
index 5f0df0625..92728ab6a 100644
--- a/v2v/test-v2v-i-ova-two-disks.expected2
+++ b/v2v/test-v2v-i-ova-two-disks.expected2
@@ -2,6 +2,7 @@ Source guest information (--print-source option):
source name: 2K8R2EESP1_2_Medium
hypervisor type: vmware
+ VM genid:
memory: 1073741824 (bytes)
nr vCPUs: 1
CPU vendor:
diff --git a/v2v/test-v2v-i-vmx-1.expected b/v2v/test-v2v-i-vmx-1.expected
index 3314021ad..628abb450 100644
--- a/v2v/test-v2v-i-vmx-1.expected
+++ b/v2v/test-v2v-i-vmx-1.expected
@@ -3,6 +3,7 @@ Source guest information (--print-source option):
source name: BZ1308535_21disks
hypervisor type: vmware
+ VM genid:
memory: 2147483648 (bytes)
nr vCPUs: 1
CPU vendor:
diff --git a/v2v/test-v2v-i-vmx-2.expected b/v2v/test-v2v-i-vmx-2.expected
index d7fbafb68..60fc7bd1d 100644
--- a/v2v/test-v2v-i-vmx-2.expected
+++ b/v2v/test-v2v-i-vmx-2.expected
@@ -3,6 +3,7 @@ Source guest information (--print-source option):
source name: Fedora 20
hypervisor type: vmware
+ VM genid:
memory: 2147483648 (bytes)
nr vCPUs: 1
CPU vendor:
diff --git a/v2v/test-v2v-i-vmx-3.expected b/v2v/test-v2v-i-vmx-3.expected
index b4dcc6283..beaf8ee59 100644
--- a/v2v/test-v2v-i-vmx-3.expected
+++ b/v2v/test-v2v-i-vmx-3.expected
@@ -3,6 +3,7 @@ Source guest information (--print-source option):
source name: RHEL 7.1 UEFI
hypervisor type: vmware
+ VM genid:
memory: 2147483648 (bytes)
nr vCPUs: 1
CPU vendor:
diff --git a/v2v/test-v2v-i-vmx-4.expected b/v2v/test-v2v-i-vmx-4.expected
index 5c45fca8d..fc60d1f1f 100644
--- a/v2v/test-v2v-i-vmx-4.expected
+++ b/v2v/test-v2v-i-vmx-4.expected
@@ -3,6 +3,7 @@ Source guest information (--print-source option):
source name: Windows 7 x64
hypervisor type: vmware
+ VM genid:
memory: 2147483648 (bytes)
nr vCPUs: 1
CPU vendor:
diff --git a/v2v/test-v2v-i-vmx-5.expected b/v2v/test-v2v-i-vmx-5.expected
index f94306536..1b2e9dfad 100644
--- a/v2v/test-v2v-i-vmx-5.expected
+++ b/v2v/test-v2v-i-vmx-5.expected
@@ -3,6 +3,7 @@ Source guest information (--print-source option):
source name: MSEdge - Win10_preview
hypervisor type: vmware
+ VM genid: 8312995e-57b8-f814-4013-c1ba795f05ba
memory: 2147483648 (bytes)
nr vCPUs: 1
CPU vendor:
diff --git a/v2v/test-v2v-print-source.expected b/v2v/test-v2v-print-source.expected
index 22e88b03c..76076c719 100644
--- a/v2v/test-v2v-print-source.expected
+++ b/v2v/test-v2v-print-source.expected
@@ -1,5 +1,6 @@
source name: windows
hypervisor type: kvm
+ VM genid:
memory: 1073741824 (bytes)
nr vCPUs: 64
CPU vendor: Intel
diff --git a/v2v/types.ml b/v2v/types.ml
index 672c8bf97..900275834 100644
--- a/v2v/types.ml
+++ b/v2v/types.ml
@@ -28,6 +28,7 @@ type source = {
s_hypervisor : source_hypervisor;
s_name : string;
s_orig_name : string;
+ s_genid : string option;
s_memory : int64;
s_vcpu : int;
s_cpu_vendor : string option;
@@ -109,6 +110,7 @@ and source_cpu_topology = {
let rec string_of_source s =
sprintf " source name: %s
hypervisor type: %s
+ VM genid: %s
memory: %Ld (bytes)
nr vCPUs: %d
CPU vendor: %s
@@ -128,6 +130,7 @@ NICs:
"
s.s_name
(string_of_source_hypervisor s.s_hypervisor)
+ (Option.default "" s.s_genid)
s.s_memory
s.s_vcpu
(Option.default "" s.s_cpu_vendor)
diff --git a/v2v/types.mli b/v2v/types.mli
index 2bc29fa68..71788a53f 100644
--- a/v2v/types.mli
+++ b/v2v/types.mli
@@ -69,6 +69,7 @@ type source = {
s_orig_name : string; (** Original guest name (if we rename
the guest using -on, original is
still saved here). *)
+ s_genid : string option; (** VM Generation ID. *)
s_memory : int64; (** Memory size (bytes). *)
s_vcpu : int; (** Number of CPUs. *)
s_cpu_vendor : string option; (** Source CPU vendor. *)
--
2.17.1
6 years
[PATCH] v2v: rhv plugin - find suitable host
by Daniel Erez
From: root <root(a)localhost.localdomain>
For direct upload, a suitable host must be in status 'Up'
and belong to the same datacenter as the created disk.
Added these criteria to the host search query.
---
v2v/rhv-upload-plugin.py | 18 +++++++++++++++---
1 file changed, 15 insertions(+), 3 deletions(-)
diff --git a/v2v/rhv-upload-plugin.py b/v2v/rhv-upload-plugin.py
index da309e288..c72f5e181 100644
--- a/v2v/rhv-upload-plugin.py
+++ b/v2v/rhv-upload-plugin.py
@@ -67,11 +67,23 @@ def find_host(connection):
debug("cannot read /etc/vdsm/vdsm.id, using any host: %s" % e)
return None
- debug("hw_id = %r" % vdsm_id)
+ system_service = connection.system_service()
+ storage_name = params['output_storage']
+ data_centers = system_service.data_centers_service().list(
+ search='storage=%s' % storage_name,
+ case_sensitive=False,
+ )
+ if len(data_centers) == 0:
+ # The storage domain is not attached to a datacenter
+ # (shouldn't happen, would fail on disk creation).
+ return None
+
+ datacenter = data_centers[0]
+ debug("hw_id = %r, datacenter = %s" % (vdsm_id, datacenter.name))
- hosts_service = connection.system_service().hosts_service()
+ hosts_service = system_service.hosts_service()
hosts = hosts_service.list(
- search="hw_id=%s" % vdsm_id,
+ search="hw_id=%s and datacenter=%s and status=Up" % (vdsm_id, datacenter.name),
case_sensitive=False,
)
if len(hosts) == 0:
--
2.17.1
6 years