On Fri, Sep 09, 2016 at 01:03:49PM +0200, Tomáš Golembiovský wrote: Specifically virt-v2v needs to do: qemu-img create -b <source-file-within-the-tarball> -f qcow2 overlay.qcow2 qemu-img convert overlay.qcow2 output As long as the tar file is not compressed, accessing a file within it should be trivial. I asked Kevin if there is a way to get qemu to access a disk image at an offset within another file, but there is no such feature at the moment. It's possible with `losetup', but that requires root :-( (At this point I would normally grumble about how easy this would be with a microkernel, but I won't do that now.) David Gilbert suggested looking at qemu-nbd which has an --offset option, allowing a particular offset with another file to be accessed. If we wanted to do it entirely within virt-v2v, I think this would be the way to go - the complex logic could be hidden inside v2v/input_ova.ml The second problem is to work out the right offset to use. I suspect this is something that http://www.libarchive.org/ can do, and that package is also in RHEL. We could even imagine a qemu block backend based on libarchive. http://www.cybernoia.de/software/archivemount/ is one such tool which can do this. It's not in RHEL, but it seems to be based on libarchive. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org

On Fri, Sep 09, 2016 at 01:03:49PM +0200, Tomáš Golembiovský wrote: Consider using virt-v2v --in-place, and doing your own import in whatever way is most appropriate for your scenarios. This is what we do in Virtuozzo to import VMs created in the previous version (based on proprietary hypervisor) into Virtuozzo 7 (QEMU/KVM-based). More specifically, 1) the target VM configuration is created from the source VM configuration by Virtuozzo code 2) we have two scenarios for hdd images: a) with data copying from a host running Virtuozzo 6 to a host running Virtuozzo 7. In this case the new qcow2 images are created and the data is migrated by Virtuozzo code; then they are attached directly to the new VM which is safe as the original images remain on the source host. The new images are then modified during v2v --in-place and then used by the new VM. b) without data copying (e.g. if shared storage is used). In this case qcow2 overlays are created with original images as backing files, and attached to the new VM. Once v2v --in-place is complete and the new VM starts its new life, the original images can be lazily merged into the new qcow2 files or just left as is. I think something like this can be done for your usecase, e.g. you can create qcow2 images with images from the archive as backing files. One possibility to do so is to have a loopback block device on top of the archive with appropriate offset and length (OVA are just tarballs, aren't they? So every archive member is just a contiguous chunk of data within the file.) Upon completing the conversion and starting the imported VM you can merge all the data into the new images and get rid of the source tarball. Roman.

Hi, just to keep this thread alive, here are few comments on the progress. (Mostly for those not following QEMU mailing lists.) On Fri, 9 Sep 2016 15:18:02 +0300 Roman Kagan <rkagan(a)virtuozzo.com&gt; wrote: Using the loopback device would be the easiest solution in the sense that it wouldn't require code changes outside virt-v2v. But, there's a big issue: it requires root privileges. That's why I cast this idea away from the beginning and decided to use that only as the last resort. Other suggestion was to use qemu-nbd because it already has --offset argument to set where the disk starts. It turned out that: a) The code for --offset is buggy and clients may attempt to read data outside the backing file, because qemu-nbd reports wrong size of the disk. Such request will inevitably fail. b) We also need to be able to specify the size of the disk. It cannot be left undefined and limited only by the length of underlying file. Security reasons aside, there are also some practical reasons. E.g. it turned out that VMDK format stores some footer at the end of the disk. If the size of the disk is not set properly the VMDK driver fails to find the footer and regards the disk as invalid. I fixed the issues and sent patches to QEMU [1][2]. Quickly somebody commented [3] that doing this in qemu-nbd is not the right idea and it should be done in the raw driver... ... that is something what Roman suggested from the start. It'll be more complicated than fixing qemu-nbd, but I gave it a shot. I have sent the first version [4] of that to QEMU. I would expect it going in right away, there's still some work to be done. That being said, I also have POC ready (attached) for virt-v2v. It's rather dirty as it contains some 'tar' invocations to get the necessary information from the archive. There are some OCAML modules for working with tar archives, but I didn't try any yet. Good thing is that QEMU people are not against putting the size & offset parameters into raw driver. Still it may turn out to be a task too hard. In that case, as a fallback, we can still use the NBD approach. We don't have to use qemu-nbd. We could also try to submit code for size & offset into the nbd-server in NBD tools (the defacto reference implementation). But that's something I didn't pursue yet. Tomas [1] https://lists.nongnu.org/archive/html/qemu-devel/2016-09/msg04554.html [2] https://lists.nongnu.org/archive/html/qemu-devel/2016-09/msg04556.html [3] https://lists.nongnu.org/archive/html/qemu-devel/2016-09/msg04578.html [4] https://lists.nongnu.org/archive/html/qemu-block/2016-10/msg00008.html -- Tomáš Golembiovský <tgolembi(a)redhat.com&gt;