On 2017-11-15 21:41, Gandalf Corvotempesta wrote:
> 2017-11-15 21:29 GMT+01:00 Richard W.M. Jones <rjones@redhat.com>:
>> Gandalf, is there an XVA file publically available (pref. not
>> too big) that we can look at?
>
> I can try to provide one, but it's simple:
>
> # tar tvf 20160630_124823_aa72_.xva.gz | head -n 50
> ---------- 0/0           42353 1970-01-01 01:00 ova.xml
> ---------- 0/0         1048576 1970-01-01 01:00 Ref:175/00000000
> ---------- 0/0              40 1970-01-01 01:00 Ref:175/00000000.checksum
> ---------- 0/0         1048576 1970-01-01 01:00 Ref:175/00000001
> ---------- 0/0              40 1970-01-01 01:00 Ref:175/00000001.checksum
> ---------- 0/0         1048576 1970-01-01 01:00 Ref:175/00000003
> ---------- 0/0              40 1970-01-01 01:00 Ref:175/00000003.checksum
> ---------- 0/0         1048576 1970-01-01 01:00 Ref:175/00000004
> ---------- 0/0              40 1970-01-01 01:00 Ref:175/00000004.checksum
> ---------- 0/0         1048576 1970-01-01 01:00 Ref:175/00000005
> ---------- 0/0              40 1970-01-01 01:00 Ref:175/00000005.checksum
> ---------- 0/0         1048576 1970-01-01 01:00 Ref:175/00000006
> ---------- 0/0              40 1970-01-01 01:00 Ref:175/00000006.checksum
> ---------- 0/0         1048576 1970-01-01 01:00 Ref:175/00000007
> ---------- 0/0              40 1970-01-01 01:00 Ref:175/00000007.checksum
> ---------- 0/0         1048576 1970-01-01 01:00 Ref:175/00000009
> ---------- 0/0              40 1970-01-01 01:00 Ref:175/00000009.checksum
> ---------- 0/0         1048576 1970-01-01 01:00 Ref:175/00000010
> ---------- 0/0              40 1970-01-01 01:00 Ref:175/00000010.checksum
>
>
> You can ignore the ova.xml and just use the "Ref:175" directory.
> Inside the XVA you'll fine one "Ref" directory for each virtual disk
> (ref number is different for each disk)
> Inside each Ref directory, you'll get tons of 1MB file, corrisponding
> to the RAW image.
> You have to merge these files in a single raw file with just an
> exception: numbers are not sequential.
> as you can see above, we have: 00000000, 00000001, 00000003
>
> The 00000002 is missing, because it's totally blank. XenServer doesn't
> export any empty block, thus it will skip the corrisponding 1MB file.
> When building the raw image, you have to replace empty blocks with 1MB
> full of zeros.
>
> This is (in addition to the tar extract) the most time-consuming part.
> Currently I'm rebuilding a 250GB image, with tons of files to be
> merge.
> If qemu-img can be patched to automatically convert this kind of
> format, I can save about 3 hours (30 minutes for extracting the
> tarball, and about 2 hours to merge 250-300GB image)

Hmmm...  OK, so as Rich said, you can use the raw driver with an offset
and a size.  And you can use the null-co driver to emulate holes.  And
qemu-img convert supports concatenation.

Taking all of that together, it should be possible to create an input
"file" per 1 MB block and concatenate all of those with qemu-img
convert.  The only issue is that it'll get you a *really* long command line.

(And you'll still have to gunzip the archive before, of course.)

I'll try to write a script, but of course it's hart without an example...

Max