Hi, from time to time we get a request [1] to import appliance with crafted OVF (from Cisco or Gigamon) into oVirt. The common denominator is that some disks are missing and are supposed to be created during import of the appliance. Doing the import properly would require not only solving of the problem with missing disks, but also implementing multiple concepts -- notably concept of configurations [2] (for Cisco appliance) or concept of properties [3] (for Gigamon appliance). This would be quite complex change to oVirt as well as to virt-v2v and at the moment we don't feel that such effort is justifiable. But by solving the problem with disks we can at least provide a helping hand to those requiring the crafted appliances.

The idea here is that virt-v2v would ignore the non-existing disks and user would be required to add those manually after conversion. As for OVFs using the configurations virt-v2v would pick some settings from OVF (random from users perspective) and user would be responsible for editing the VM's configuration after conversion (CPUs, memory, etc.) to size the VM properly based on the expected use. We could further constrain this to only work with -o vdsm, but this may not be needed unless we hit some issues with implementing the feature. It is also possible that ignoring the disks may not work for some reasons that we have not yet discovered and we'll se once we try. There is one more problem with the Cisco OVA that it contains .cert file which is not specified in manifest. But the file is not needed during conversion. So this could be possibly handled by enforcing virt-v2v to use only files listed in manifest instead of complaining. Before I invest any time into this I wanted to make sure that such approach would be acceptable to the upstream. So would this be good enough?

*** The topics for discussions are above. What follows are the technical details for those interested in gain deeper understanding of the problem. You may be wondering why would we want to ignore the empty disks if we can create them for most of the output backends. The problem is, that we cannot. Either we don't know which disks are of the interest because not all should be used (configurations) or we have no idea how big the disk should be (properties). ### Configurations The OVF can define several configurations in DeploymentOptionSection. A short (simplified) example may look like this: <DeploymentOptionSection> <Configuration ovf:default="true" ovf:id="Standard" /> <Configuration ovf:id="Express" /> ... </DeploymentOptionSection> Then in the VirtualHardwareSection there can be duplicate settings distinguished only by the ovf:configuration attribute. For example 2 different vCPU configurations: <Item ovf:configuration="Express"> ... <rasd:Description>Number of Virtual CPUs</rasd:Description> <rasd:ResourceType>3</rasd:ResourceType> <rasd:VirtualQuantity>4</rasd:VirtualQuantity> </Item> <Item ovf:configuration="Standard"> ... <rasd:Description>Number of Virtual CPUs</rasd:Description> <rasd:ResourceType>3</rasd:ResourceType> <rasd:VirtualQuantity>16</rasd:VirtualQuantity> </Item> In terms of disks this means that only some of the disks get used. Specifically the Cisco Appliance has 4 disks listed in the DiskSection -- 1 system disk A and 3 empty disks B,C,D. But the created VM never has all four. It has either only A or it has A+B or A+C or A+D. Without understanding configurations we cannot tell whether to use B, C, D or none. ### Properties The OVF can define various properties in ProductSection element. Like this: <ProductSection> ... <Property ovf:key="datadisksize" ovf:qualifiers="MinValue(20),MaxValue(1000)" ovf:type="int" ovf:userConfigurable="true"> <Label>02. Size of Data Disk</Label> <Description>The size of the Data Disk in gigabytes.</Description> <Value ovf:value="40"/> </Property> ... </ProductSection Then, the ovf:key value of the property can be used as a variable on other places in the OVF. For example like this: <DiskSection> ... <Disk ovf:capacity="${datadisksize}" ovf:fileRef="" ... /> ... </DiskSection> And as before, without understanding the concept virt-v2v has no idea how to size the disk. The Value element is optional (if the property is userConfigurable) so relying on the default in OVF may not work. I can imagine some OVFs may even use a property to specify vCPU count or memory which could bring up a question how to handle those. Possibly default to 0 or 1 (where 1 may be better default in my opinion). Tomas [1] https://bugzilla.redhat.com/1705600 [2] Open Virtualization Format Specification (DSP0243) Version 2.1.1, Chapter 9.8 -- DeploymentOptionSection [3] Open Virtualization Format Specification (DSP0243) Version 2.1.1, Chapter 9.5.1 -- Property elements -- Tomáš Golembiovský <tgolembi(a)redhat.com&gt; _______________________________________________ Libguestfs mailing list Libguestfs(a)redhat.com https://listman.redhat.com/mailman/listinfo/libguestfs

On Mon, Feb 22, 2021 at 04:45:09PM +0000, Richard W.M. Jones wrote: > On Mon, Feb 22, 2021 at 02:51:32PM +0100, Tomáš Golembiovský wrote: > > Hi, > > > > from time to time we get a request [1] to import appliance with crafted > > OVF (from Cisco or Gigamon) into oVirt. The common denominator is that > > some disks are missing and are supposed to be created during import of > > the appliance. > > > > Doing the import properly would require not only solving of the problem > > with missing disks, but also implementing multiple concepts -- notably > > concept of configurations [2] (for Cisco appliance) or concept of > > properties [3] (for Gigamon appliance). This would be quite complex > > change to oVirt as well as to virt-v2v and at the moment we don't feel > > that such effort is justifiable. But by solving the problem with disks > > we can at least provide a helping hand to those requiring the crafted > > appliances. > > Some immediate questions: Can virt-v2v do anything useful for these > appliances? Probably same as for any foreign Linux VM. The Cisco appliance is RHEL 7. I don't have access to the Gigamon appliance but from metadata it looks like some CentOS. > Does libguestfs inspection find anything inside the > appliances? I am not sure what you are asking here.

> Is installing virtio drivers useful? Do they not have virtio drivers > already? It is (fairly recent) Linux, so there's no need to install anything. But if you're asking if it makes sense for the appliance to switch to virtio devices then I would say yes.

> Where are they supposed to run originally? On VMware. Tomas > > > The idea here is that virt-v2v would ignore the non-existing disks and > > user would be required to add those manually after conversion. As for > > OVFs using the configurations virt-v2v would pick some settings from OVF > > (random from users perspective) and user would be responsible for > > editing the VM's configuration after conversion (CPUs, memory, etc.) to > > size the VM properly based on the expected use. We could further > > constrain this to only work with -o vdsm, but this may not be needed > > unless we hit some issues with implementing the feature. It is also > > possible that ignoring the disks may not work for some reasons that > > we have not yet discovered and we'll se once we try. > > > > There is one more problem with the Cisco OVA that it contains .cert file > > which is not specified in manifest. But the file is not needed during > > conversion. So this could be possibly handled by enforcing virt-v2v to > > use only files listed in manifest instead of complaining. > > > > Before I invest any time into this I wanted to make sure that such > > approach would be acceptable to the upstream. So would this be good > > enough? > > Does the proposed virt-v2v split help here? > > https://listman.redhat.com/archives/libguestfs/2020-November/msg00022.html > > With this kind of split, you could set up the disks however you liked > -- for example creating dummy input disks (nbdkit-null-plugin > instances probably) -- for the missing disks. Then run just the > virt-v2v conversion component to carry out the conversion. > > Rich. > > > *** > > > > The topics for discussions are above. What follows are the technical > > details for those interested in gain deeper understanding of the > > problem. You may be wondering why would we want to ignore the empty > > disks if we can create them for most of the output backends. The problem > > is, that we cannot. Either we don't know which disks are of the interest > > because not all should be used (configurations) or we have no idea how > > big the disk should be (properties). > > > > ### Configurations > > > > The OVF can define several configurations in DeploymentOptionSection. > > A short (simplified) example may look like this: > > > > <DeploymentOptionSection> > > <Configuration ovf:default="true" ovf:id="Standard" /> > > <Configuration ovf:id="Express" /> > > ... > > </DeploymentOptionSection> > > > > Then in the VirtualHardwareSection there can be duplicate settings > > distinguished only by the ovf:configuration attribute. For example 2 different > > vCPU configurations: > > > > <Item ovf:configuration="Express"> > > ... > > <rasd:Description>Number of Virtual CPUs</rasd:Description> > > <rasd:ResourceType>3</rasd:ResourceType> > > <rasd:VirtualQuantity>4</rasd:VirtualQuantity> > > </Item> > > <Item ovf:configuration="Standard"> > > ... > > <rasd:Description>Number of Virtual CPUs</rasd:Description> > > <rasd:ResourceType>3</rasd:ResourceType> > > <rasd:VirtualQuantity>16</rasd:VirtualQuantity> > > </Item> > > > > In terms of disks this means that only some of the disks get used. Specifically > > the Cisco Appliance has 4 disks listed in the DiskSection -- 1 system disk A > > and 3 empty disks B,C,D. But the created VM never has all four. It has either > > only A or it has A+B or A+C or A+D. Without understanding configurations we > > cannot tell whether to use B, C, D or none. > > > > > > ### Properties > > > > The OVF can define various properties in ProductSection element. Like this: > > > > > > <ProductSection> > > ... > > <Property ovf:key="datadisksize" ovf:qualifiers="MinValue(20),MaxValue(1000)" ovf:type="int" ovf:userConfigurable="true"> > > <Label>02. Size of Data Disk</Label> > > <Description>The size of the Data Disk in gigabytes.</Description> > > <Value ovf:value="40"/> > > </Property> > > ... > > </ProductSection > > > > Then, the ovf:key value of the property can be used as a variable on > > other places in the OVF. For example like this: > > > > <DiskSection> > > ... > > <Disk ovf:capacity="${datadisksize}" ovf:fileRef="" ... /> > > ... > > </DiskSection> > > > > And as before, without understanding the concept virt-v2v has no idea > > how to size the disk. The Value element is optional (if the property is > > userConfigurable) so relying on the default in OVF may not work. I can > > imagine some OVFs may even use a property to specify vCPU count or > > memory which could bring up a question how to handle those. Possibly > > default to 0 or 1 (where 1 may be better default in my opinion). > > > > Tomas > > > > > > [1] https://bugzilla.redhat.com/1705600 > > [2] Open Virtualization Format Specification (DSP0243) Version 2.1.1, > > Chapter 9.8 -- DeploymentOptionSection > > [3] Open Virtualization Format Specification (DSP0243) Version 2.1.1, > > Chapter 9.5.1 -- Property elements > > > > -- > > Tomáš Golembiovský <tgolembi(a)redhat.com&gt; > > > > _______________________________________________ > > Libguestfs mailing list > > Libguestfs(a)redhat.com > > https://listman.redhat.com/mailman/listinfo/libguestfs > > -- > Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones > Read my programming and virtualization blog: http://rwmj.wordpress.com > libguestfs lets you edit virtual machines. Supports shell scripting, > bindings from many languages. http://libguestfs.org > -- Tomáš Golembiovský <tgolembi(a)redhat.com&gt;