(+libvirt-devel) On 09/29/21 21:22, Richard W.M. Jones wrote: OK. So AIUI, that means no work is needed here for Windows. AIUI the conflict avoidance logic ("no overlapping disks") is already in place. The question is how to translate device paths in /etc/fstab and similar. Please correct me if I'm wrong: at the moment, I believe virt-v2v parses and manipulates the following elements and attributes in the domain XML: <target dev='hda' bus='ide'/> <target dev='hdb' bus='ide'/> <target dev='hdc' bus='ide'/> <target dev='hdd' bus='ide'/> ^^^ ^^^ My understanding is however that the target/@dev attribute is mostly irrelevant: https://libvirt.org/formatdomain.html#hard-drives-floppy-disks-cdroms The dev attribute indicates the "logical" device name. The actual device name specified is not guaranteed to map to the device name in the guest OS. Treat it as a device ordering hint. [...] What actually matters is the target/@bus attribute, in combination with the sibling element <address>. Such as: <target dev='hda' bus='ide'/> ^^^ <address type='drive' controller='0' bus='0' target='0' unit='0'/> ^ ^ ^ <target dev='hdb' bus='ide'/> ^^^ <address type='drive' controller='0' bus='0' target='0' unit='1'/> ^ ^ ^ <target dev='hdc' bus='ide'/> ^^^ <address type='drive' controller='0' bus='1' target='0' unit='0'/> ^ ^ ^ <target dev='hdd' bus='ide'/> ^^^ <address type='drive' controller='0' bus='1' target='0' unit='1'/> ^ ^ ^ So, target/@dev should be mostly ignored; what matters is the following tuple: (target/@bus, address/@controller, address/@bus, address/@unit) Extracting just the tuples: (ide, 0, 0, 0) (ide, 0, 0, 1) (ide, 0, 1, 0) (ide, 0, 1, 1) The first two components of each tuple -- i.e., (ide, 0) -- refer to the following IDE controller: <controller type='ide' index='0'> ^^^^^^^^^^ ^^^^^^^^^ <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/> </controller> and then the rest of the components, such as (0, 0), (0, 1), (1, 0), (1, 1), identify the disk on that IDE controller. (Side comment: the PCI location of the (first) IDE controller is fixed in QEMU; if one tries to change it, libvirt complains: "Primary IDE controller must have PCI address 0:0:1.1".) (Side comment: on the QEMU command line, this maps to -device ide-cd,bus=ide.0,unit=0,... \ -device ide-cd,bus=ide.0,unit=1,... \ -device ide-cd,bus=ide.1,unit=0,... \ -device ide-cd,bus=ide.1,unit=1,... \ ) Inside the guest, /dev/hd* nodes don't even exist, so it's unlikely that /etc/fstab would refer to them. /etc/fstab can however refer to symlinks under "/dev/disk/by-id" (for example): lrwxrwxrwx. 1 root root 9 Sep 30 11:54 ata-QEMU_DVD-ROM_QM00001 -> ../../sr0 lrwxrwxrwx. 1 root root 9 Sep 30 11:54 ata-QEMU_DVD-ROM_QM00002 -> ../../sr1 lrwxrwxrwx. 1 root root 9 Sep 30 11:54 ata-QEMU_DVD-ROM_QM00003 -> ../../sr2 lrwxrwxrwx. 1 root root 9 Sep 30 11:54 ata-QEMU_DVD-ROM_QM00004 -> ../../sr3 Furthermore, we have pseudo-files (directories) such as: /sys/devices/pci0000:00/0000:00:01.1/ata1/host0/target0:0:0/0:0:0:0/block/sr0 /sys/devices/pci0000:00/0000:00:01.1/ata1/host0/target0:0:1/0:0:1:0/block/sr1 /sys/devices/pci0000:00/0000:00:01.1/ata2/host1/target1:0:0/1:0:0:0/block/sr2 /sys/devices/pci0000:00/0000:00:01.1/ata2/host1/target1:0:1/1:0:1:0/block/sr3 ^ ^ So in order to map a device path from the original guest's "/etc/fstab", such as "/dev/disk/by-id/ata-QEMU_DVD-ROM_QM00003", to the original domain XML's <disk> element, we have to do the following in the "source" appliance: NODE=$(realpath /dev/disk/by-id/ata-QEMU_DVD-ROM_QM00003) # -> /dev/sr2 NODE=${NODE#/dev/} # -> sr2 DEVPATH=$(ls -d /sys/devices/pci0000:00/0000:00:01.1/ata?/host?/target?:0:?/?:0:?:0/block/$NODE) # -> /sys/devices/pci0000:00/0000:00:01.1/ata2/host1/target1:0:0/1:0:0:0/block/sr2 And then map the "1:0:0:0" pathname component from $DEVPATH to: <target dev='hdc' bus='ide'/> <address type='drive' controller='0' bus='1' target='0' unit='0'/> ^^^^^^^ ^^^^^^^^ [1]:0:0:0 1:0:[0]:0 in the original domain XML. This tells us under what device node the original guest sees the host-side file (<source> element). After conversion, on the Q35 board, the inverse mapping is needed. We start from the domain XML, <target dev='sd*' bus='sata'/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> <target dev='sd*' bus='sata'/> <address type='drive' controller='0' bus='0' target='0' unit='1'/> <target dev='sd*' bus='sata'/> <address type='drive' controller='0' bus='0' target='0' unit='2'/> <target dev='sd*' bus='sata'/> <address type='drive' controller='0' bus='0' target='0' unit='3'/> <target dev='sd*' bus='sata'/> <address type='drive' controller='0' bus='0' target='0' unit='4'/> <target dev='sd*' bus='sata'/> <address type='drive' controller='0' bus='0' target='0' unit='5'/> <controller type='sata' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/> </controller> (Side comment: the PCI B/D/F of the SATA controller is also fixed in QEMU; otherwise libvirt complains: "Primary SATA controller must have PCI address 0:0:1f.2".) (Side comment: the QEMU command line is -device ide-cd,bus=ide.0,... \ -device ide-cd,bus=ide.1,... \ -device ide-cd,bus=ide.2,... \ -device ide-cd,bus=ide.3,... \ -device ide-cd,bus=ide.4,... \ -device ide-cd,bus=ide.5,... \ ) In the guest we have: /sys/devices/pci0000:00/0000:00:1f.2/ata1/host0/target0:0:0/0:0:0:0/block/sr0 /sys/devices/pci0000:00/0000:00:1f.2/ata2/host1/target1:0:0/1:0:0:0/block/sr1 /sys/devices/pci0000:00/0000:00:1f.2/ata3/host2/target2:0:0/2:0:0:0/block/sr2 /sys/devices/pci0000:00/0000:00:1f.2/ata4/host3/target3:0:0/3:0:0:0/block/sr3 /sys/devices/pci0000:00/0000:00:1f.2/ata5/host4/target4:0:0/4:0:0:0/block/sr4 /sys/devices/pci0000:00/0000:00:1f.2/ata6/host5/target5:0:0/5:0:0:0/block/sr5 So, assuming we mapped the original (i440fx) "/dev/disk/by-id/ata-QEMU_DVD-ROM_QM00003" guest device path to some <source> element (= host-side file) in the original domain XML, and assuming virt-v2v assigned the same <source> element to the following element on the Q35 board: <target dev='sd*' bus='sata'/> <address type='drive' controller='0' bus='0' target='0' unit='4'/> ^^^^^^^^ we find the device node in the destination appliance as follows: NODE=$(basename /sys/devices/pci0000:00/0000:00:1f.2/ata?/host?/target?:0:0/4:0:0:0/block/*) ^ unit='4' # -> sr4 and then replace "/dev/disk/by-id/ata-QEMU_DVD-ROM_QM00003" with "/dev/sr4" in "/etc/fstab". All this requires virt-v2v to parse complete <address> elements from the original domain XML, and to generate complete <address> elements in the destination domain XML. Is that feasible? The @wwn and @serial attributes don't look safe to me, because the guest could refer to them even when the original domain XML does not spell them out (eg. "QM00003" is such a @serial). So neither can we trust that a @serial is present in the original XML, nor can we just go ahead and generate a @serial if it is absent. (The generation step could immediately break references such as "QM00003" in the guest.) /dev/disk/by-label and /dev/disk/by-uuid are based on media contents, and multiple CD-ROMs may (read-only) map the same host-side file, so those are not good for mapping either, I think. Also does not cover CD-ROM devices that are empty (have no medium) at the time of conversion, but "/etc/fstab" still refers to them (potentially with "noauto"). So I think the only reliable ID is the hardware device path. ... Now if *that* needs to work when the original guest comes from a different management application than libvirt, then I have no idea. The original address (PCI B/D/F of the IDE controller, and IDE bus and unit of the drive) need to be known somehow; otherwise we cannot associate the guest-side "/dev/..." reference with the host-side file underlying that CD-ROM. Thanks, Laszlo

On Thu, Sep 30, 2021 at 12:53:51PM +0100, Richard W.M. Jones wrote: Just to put a bit of flesh on these bones, links to the source: Here's the virt-v2v code that can parse either a local or remote (over ssh) *.vmx file into virt-v2v's internal metadata representation: https://github.com/libguestfs/virt-v2v/blob/master/input/parse_domain_fro... Libvirt also has a vmx driver, but I decided not to use it because it just adds an extra translation step and probably loses fidelity in the process. In all cases, the *.vmx file is the source of truth for VMware. The libvirt driver: https://gitlab.com/libvirt/libvirt/-/tree/master/src/esx I've hacked on this one a little bit in the past, and ... it's complicated. OVF is a pseudo-standard invented by VMware, but essentially a plot so they can say they are "standard" when in fact it's just a data dump of internal VMware structures with no compatibility across software that generates or consumes OVF. Here's our OVF parser (for VMware-OVF): https://github.com/libguestfs/virt-v2v/blob/master/input/OVF.ml We can also generate OVF, but we generate the RHV-flavour of OVF which (see above) is not related except that some XML element names overlap. If you give virt-v2v just a disk then it will invent some metadata: https://github.com/libguestfs/virt-v2v/blob/5adf437bcc00586961d8aa058559f... Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top

On Fri, Oct 01, 2021 at 12:35:50PM +0200, Laszlo Ersek wrote: There are two types of variants in OCaml. The first is the normal one and it requires a type definition. eg: type t = A | B | C of int | D of string * int with values like “A” or “C 3” or “D ("hello", 1)”. These are ones we'd normally use and have the (arguable?) advantage that you have to write (and therefore, document) the full type somewhere. The second type are polymorphic variants. These are actually way more complicated than I'm going to describe below because they have a load of set theory behind them (if you really want to know see the docs that you've already linked to). They are written with backtick before the name of the variant. The brief idea is that you let the OCaml compiler work out the type for you. For example: let f = function 1 -> `A | 2 -> `B | i -> `C i creates a function f which returns the type: type pv = [ `A | `B | `C of int ] Actually if you type this into the toplevel it says: $ rlwrap ocaml OCaml version 4.12.0 # let f = function 1 -> `A | 2 -> `B | i -> `C i ;; val f : int -> [> `A | `B | `C of int ] = <fun> As you observe below there's an extra ">" character. That means that the type is "at least" that one, and there might be more cases, but the function that I wrote only contains the 3 cases. Anyway the short of this is that these open-ended, automatically resolved types don't require a type definition so they're less documented but more flexible. (And another thing: It's possible to reuse the constructors in unrelated types. eg. `A might be part of another type. But don't do this unless it's for a very good reason). In all cases the OCaml compiler enforces type safety, so although these types are less explicit, they're still safe. If I extend my example above and forget about one of the cases, the compiler will warn (in the v2v code, we've set up the flags to give a hard error): # let g j = match f j with `A -> 1 | `B -> 2 ;; Warning 8 [partial-match]: this pattern-matching is not exhaustive. Here is an example of a case that is not matched: `C _ val g : int -> int = <fun> Yes, this is all fair comment. I guess you could look at git history, but most likely it was "just because". It's a bit quicker and adding a new cases involves less code churn. I'm not opposed if you want to make the change above if you think it improves readability. In general I try to avoid tricksy features of OCaml, like functors, and this could (arguably) be one of those cases. BTW as an aside the implementation of the two types of constructors is completely different, and polymorphic variants have a slower implementation. For ordinary constructors they generally map to integers, which makes them very efficient (eg. they can be stored in registers etc). Polymorphic variants use a hash of the name instead making them slower overall. (see table in https://rwmj.wordpress.com/2009/08/05/ocaml-internals-part-2-strings-and-...) But a fun thing is you can cause hash collisions (although it is picked up by the compiler - it's still type safe!): # type collision = [`Eric_Cooper | `azdwbie];; Error: Variant tags `azdwbie and `Eric_Cooper have the same hash value. Change one of them. (https://groups.google.com/g/fa.caml/c/68oTntkRLyU) There's even a link-time pass to find hash collisions across the whole program. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-df lists disk usage of guests without needing to install any software inside the virtual machine. Supports Linux and Windows. http://people.redhat.com/~rjones/virt-df/

On Fri, Oct 01, 2021 at 12:21:29PM +0100, Richard W.M. Jones wrote: I mean C int 5 = OCaml int 2, etc. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-df lists disk usage of guests without needing to install any software inside the virtual machine. Supports Linux and Windows. http://people.redhat.com/~rjones/virt-df/

On Thu, Sep 30, 2021 at 01:12:39PM +0200, Laszlo Ersek wrote: I won't say it is irrelevant. Functionally @dev is absolutely still important, as it influences how the disk is attached to the VM. Rather I would say that the @dev attribute is misleading to users, because they mistakenly think it provides a guarantee that the disk will appear with this name inside the guest. Yes, the <address/> is what libvirt internally drivers all configuration off, but in practice application developers almost never use the <address> element directly. They will just give target/@dev and libvirt will use that to automatically populate an <address/> element, in order to reliably fixate the guest ABI thereafter. Yes, that's correct for IDE. Yep, its defined by the QEMU machine type and we just have to accept that for IDE/SATA. Yes, to map from the guest to the libvirt XML, you need to be working in terms of the hardware buses/addresses. I wouldn't recommend using any of the /dev/* devices, as those are all unstable when faced with changed guest configuration over time. /etc/fstab should really use a stable /dev/disk/ symlink, so it can be directly associated with the desired device based on the hardware topology, rather than guest OS device probe order. Yep, the symlink based on hardware topology is the only thing that can be stable and unique. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|