Re: [Libguestfs] Question about mounting QCOW2 files....
by Richard W.M. Jones
[Adding libguestfs mailing list]
On Sun, Oct 14, 2018 at 11:53:17AM -0400, Raghuram Devarakonda wrote:
> Hi,
>
> My name is Raghuram Devarakonda and I am a big fan of "nbdkit". I have
> successfully used it to prototype (using Python plugin) a complex
> project in our company and since then, I have also been trying to
> understand and learn "guestfs-tools" as well. I have a question in
> this regard and I hope you don't mind my sending mail gratuitously.
>
> In our project, we deal with quite large sparse files whose total size
> runs into several hundred terabytes or even more, though actual
> allocated size on disk is much smaller. The problem is that such large
> files present issues for copying around, compression, or even for
> computing checksums. I am wondering of I can use guestfs-tools to
> mount a QCOW2 image and then use the image as sparse file. The idea is
> that actual file on the disk would be compact though to our code,
> sparse file interface is preserved.
It's not very clear to me exactly what you want, but in general yes
qcow2 is a good way to handle very large, sparse disk images.
If you can be clearer about exactly what you mean by "mount" then I
could answer the question better. For example, do you mean "mount a
filesystem in the qcow2 image"? In which case use guestmount. If you
mean "attach the qcow2 disk as a local device" then qemu-nbd can do
this.
Rich.
> Can you please let me know if this is possible? Can I mount a sparse
> file using guestfs-tools even though it is not really a disk?
>
> Thanks in advance,
> Raghu
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html
5 years, 7 months
Bug Report, .vhdx file not attaching
by Antoine KILZI
I am trying to mount a windows 10 backup .vhdx file.
I installed libguestfs through ```sudo apt-get install libguestfs-tools```
I am running Ubuntu 18.04.1 LTS and this is the installed version: ```1:1.36.13-1ubuntu3.2```
This is the output of the command I ran:
```
~$ guestmount --add Documents/8be29c38-0000-0000-0000-602200000000.vhdx --inspector --ro /media/Windows10/
libguestfs: trace: set_verbose true
libguestfs: trace: set_verbose = 0
libguestfs: create: flags = 0, handle = 0x562d44e7c570, program = guestmount
libguestfs: trace: set_recovery_proc false
libguestfs: trace: set_recovery_proc = 0
libguestfs: trace: add_drive "Documents/8be29c38-0000-0000-0000-602200000000.vhdx" "readonly:true"
libguestfs: creating COW overlay to protect original drive content
libguestfs: trace: get_tmpdir
libguestfs: trace: get_tmpdir = "/tmp"
libguestfs: trace: disk_create "/tmp/libguestfsOz0P6y/overlay1.qcow2" "qcow2" -1 "backingfile:/home/a17kilzi/Documents/8be29c38-0000-0000-0000-602200000000.vhdx"
libguestfs: command: run: qemu-img
libguestfs: command: run: \ create
libguestfs: command: run: \ -f qcow2
libguestfs: command: run: \ -o backing_file=/home/a17kilzi/Documents/8be29c38-0000-0000-0000-602200000000.vhdx
libguestfs: command: run: \ /tmp/libguestfsOz0P6y/overlay1.qcow2
Formatting '/tmp/libguestfsOz0P6y/overlay1.qcow2', fmt=qcow2 size=142622064640 backing_file=/home/a17kilzi/Documents/8be29c38-0000-0000-0000-602200000000.vhdx cluster_size=65536 lazy_refcounts=off refcount_bits=16
libguestfs: trace: disk_create = 0
libguestfs: trace: add_drive = 0
libguestfs: trace: launch
libguestfs: trace: version
libguestfs: trace: version = <struct guestfs_version = major: 1, minor: 36, release: 13, extra: , >
libguestfs: trace: get_backend
libguestfs: trace: get_backend = "direct"
libguestfs: launch: program=guestmount
libguestfs: launch: version=1.36.13
libguestfs: launch: backend registered: unix
libguestfs: launch: backend registered: uml
libguestfs: launch: backend registered: libvirt
libguestfs: launch: backend registered: direct
libguestfs: launch: backend=direct
libguestfs: launch: tmpdir=/tmp/libguestfsOz0P6y
libguestfs: launch: umask=0022
libguestfs: launch: euid=1000
libguestfs: is_openable: /dev/kvm: Permission denied
libguestfs: trace: get_backend_setting "force_tcg"
libguestfs: trace: get_backend_setting = NULL (error)
libguestfs: warning: current user is not a member of the KVM group (group ID 132). This user cannot access /dev/kvm, so libguestfs may run very slowly. It is recommended that you 'chmod 0666 /dev/kvm' or add the current user to the KVM group (you might need to log out and log in again).
libguestfs: trace: get_cachedir
libguestfs: trace: get_cachedir = "/var/tmp"
libguestfs: begin building supermin appliance
libguestfs: run supermin
libguestfs: command: run: /usr/bin/supermin
libguestfs: command: run: \ --build
libguestfs: command: run: \ --verbose
libguestfs: command: run: \ --if-newer
libguestfs: command: run: \ --lock /var/tmp/.guestfs-1000/lock
libguestfs: command: run: \ --copy-kernel
libguestfs: command: run: \ -f ext2
libguestfs: command: run: \ --host-cpu x86_64
libguestfs: command: run: \ /usr/lib/x86_64-linux-gnu/guestfs/supermin.d
libguestfs: command: run: \ -o /var/tmp/.guestfs-1000/appliance.d
supermin: version: 5.1.19
supermin: package handler: debian/dpkg
supermin: acquiring lock on /var/tmp/.guestfs-1000/lock
supermin: build: /usr/lib/x86_64-linux-gnu/guestfs/supermin.d
supermin: reading the supermin appliance
supermin: build: visiting /usr/lib/x86_64-linux-gnu/guestfs/supermin.d/base.tar.gz type gzip base image (tar)
supermin: build: visiting /usr/lib/x86_64-linux-gnu/guestfs/supermin.d/daemon.tar.gz type gzip base image (tar)
supermin: build: visiting /usr/lib/x86_64-linux-gnu/guestfs/supermin.d/excludefiles type uncompressed excludefiles
supermin: build: visiting /usr/lib/x86_64-linux-gnu/guestfs/supermin.d/hostfiles type uncompressed hostfiles
supermin: build: visiting /usr/lib/x86_64-linux-gnu/guestfs/supermin.d/init.tar.gz type gzip base image (tar)
supermin: build: visiting /usr/lib/x86_64-linux-gnu/guestfs/supermin.d/packages type uncompressed packages
supermin: build: visiting /usr/lib/x86_64-linux-gnu/guestfs/supermin.d/packages-hfsplus type uncompressed packages
supermin: build: visiting /usr/lib/x86_64-linux-gnu/guestfs/supermin.d/packages-reiserfs type uncompressed packages
supermin: build: visiting /usr/lib/x86_64-linux-gnu/guestfs/supermin.d/packages-xfs type uncompressed packages
supermin: build: visiting /usr/lib/x86_64-linux-gnu/guestfs/supermin.d/udev-rules.tar.gz type gzip base image (tar)
supermin: mapping package names to installed packages
supermin: resolving full list of package dependencies
supermin: build: 215 packages, including dependencies
supermin: build: 9953 files
supermin: build: 6727 files, after matching excludefiles
supermin: build: 6730 files, after adding hostfiles
supermin: build: 6727 files, after removing unreadable files
supermin: build: 6730 files, after munging
supermin: kernel: looking for kernel using environment variables ...
supermin: kernel: looking for kernels in /lib/modules/*/vmlinuz ...
supermin: kernel: looking for kernels in /boot ...
supermin: kernel: kernel version of /boot/vmlinuz-4.15.0-36-generic = 4.15.0-36-generic (from filename)
supermin: kernel: picked modules path /lib/modules/4.15.0-36-generic
supermin: kernel: picked vmlinuz /boot/vmlinuz-4.15.0-36-generic
supermin: kernel: kernel_version 4.15.0-36-generic
supermin: kernel: modpath /lib/modules/4.15.0-36-generic
cp: cannot open '/boot/vmlinuz-4.15.0-36-generic' for reading: Permission denied
supermin: cp -p '/boot/vmlinuz-4.15.0-36-generic' '/var/tmp/.guestfs-1000/appliance.d.ouzjasdp/kernel': command failed, see earlier errors
libguestfs: error: /usr/bin/supermin exited with error status 1, see debug messages above
libguestfs: trace: launch = -1 (error)
libguestfs: trace: close
libguestfs: closing guestfs handle 0x562d44e7c570 (state 0)
libguestfs: command: run: rm
libguestfs: command: run: \ -rf /tmp/libguestfsOz0P6y
```
And this is the output of the libguestfs-test-tool:
``` $ libguestfs-test-tool
************************************************************
* IMPORTANT NOTICE
*
* When reporting bugs, include the COMPLETE, UNEDITED
* output below in your bug report.
*
************************************************************
libguestfs: trace: set_verbose true
libguestfs: trace: set_verbose = 0
libguestfs: trace: set_verbose true
libguestfs: trace: set_verbose = 0
LIBGUESTFS_TRACE=1
LIBGUESTFS_DEBUG=1
PATH=/home/a17kilzi/bin:/home/a17kilzi/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
XDG_RUNTIME_DIR=/run/user/1000
SELinux: sh: 1: getenforce: not found
libguestfs: trace: add_drive_scratch 104857600
libguestfs: trace: get_tmpdir
libguestfs: trace: get_tmpdir = "/tmp"
libguestfs: trace: disk_create "/tmp/libguestfsS9Wa6T/scratch1.img" "raw" 104857600
libguestfs: trace: disk_create = 0
libguestfs: trace: add_drive "/tmp/libguestfsS9Wa6T/scratch1.img" "format:raw" "cachemode:unsafe"
libguestfs: trace: add_drive = 0
libguestfs: trace: add_drive_scratch = 0
libguestfs: trace: get_append
libguestfs: trace: get_append = "NULL"
guestfs_get_append: (null)
libguestfs: trace: get_autosync
libguestfs: trace: get_autosync = 1
guestfs_get_autosync: 1
libguestfs: trace: get_backend
libguestfs: trace: get_backend = "direct"
guestfs_get_backend: direct
libguestfs: trace: get_backend_settings
libguestfs: trace: get_backend_settings = []
guestfs_get_backend_settings: []
libguestfs: trace: get_cachedir
libguestfs: trace: get_cachedir = "/var/tmp"
guestfs_get_cachedir: /var/tmp
libguestfs: trace: get_direct
libguestfs: trace: get_direct = 0
guestfs_get_direct: 0
libguestfs: trace: get_hv
libguestfs: trace: get_hv = "/usr/bin/qemu-system-x86_64"
guestfs_get_hv: /usr/bin/qemu-system-x86_64
libguestfs: trace: get_memsize
libguestfs: trace: get_memsize = 500
guestfs_get_memsize: 500
libguestfs: trace: get_network
libguestfs: trace: get_network = 0
guestfs_get_network: 0
libguestfs: trace: get_path
libguestfs: trace: get_path = "/usr/lib/x86_64-linux-gnu/guestfs"
guestfs_get_path: /usr/lib/x86_64-linux-gnu/guestfs
libguestfs: trace: get_pgroup
libguestfs: trace: get_pgroup = 0
guestfs_get_pgroup: 0
libguestfs: trace: get_program
libguestfs: trace: get_program = "libguestfs-test-tool"
guestfs_get_program: libguestfs-test-tool
libguestfs: trace: get_recovery_proc
libguestfs: trace: get_recovery_proc = 1
guestfs_get_recovery_proc: 1
libguestfs: trace: get_smp
libguestfs: trace: get_smp = 1
guestfs_get_smp: 1
libguestfs: trace: get_sockdir
libguestfs: trace: get_sockdir = "/run/user/1000"
guestfs_get_sockdir: /run/user/1000
libguestfs: trace: get_tmpdir
libguestfs: trace: get_tmpdir = "/tmp"
guestfs_get_tmpdir: /tmp
libguestfs: trace: get_trace
libguestfs: trace: get_trace = 1
guestfs_get_trace: 1
libguestfs: trace: get_verbose
libguestfs: trace: get_verbose = 1
guestfs_get_verbose: 1
host_cpu: x86_64
Launching appliance, timeout set to 600 seconds.
libguestfs: trace: launch
libguestfs: trace: version
libguestfs: trace: version = <struct guestfs_version = major: 1, minor: 36, release: 13, extra: , >
libguestfs: trace: get_backend
libguestfs: trace: get_backend = "direct"
libguestfs: launch: program=libguestfs-test-tool
libguestfs: launch: version=1.36.13
libguestfs: launch: backend registered: unix
libguestfs: launch: backend registered: uml
libguestfs: launch: backend registered: libvirt
libguestfs: launch: backend registered: direct
libguestfs: launch: backend=direct
libguestfs: launch: tmpdir=/tmp/libguestfsS9Wa6T
libguestfs: launch: umask=0022
libguestfs: launch: euid=1000
libguestfs: is_openable: /dev/kvm: Permission denied
libguestfs: trace: get_backend_setting "force_tcg"
libguestfs: trace: get_backend_setting = NULL (error)
libguestfs: warning: current user is not a member of the KVM group (group ID 132). This user cannot access /dev/kvm, so libguestfs may run very slowly. It is recommended that you 'chmod 0666 /dev/kvm' or add the current user to the KVM group (you might need to log out and log in again).
libguestfs: trace: get_cachedir
libguestfs: trace: get_cachedir = "/var/tmp"
libguestfs: begin building supermin appliance
libguestfs: run supermin
libguestfs: command: run: /usr/bin/supermin
libguestfs: command: run: \ --build
libguestfs: command: run: \ --verbose
libguestfs: command: run: \ --if-newer
libguestfs: command: run: \ --lock /var/tmp/.guestfs-1000/lock
libguestfs: command: run: \ --copy-kernel
libguestfs: command: run: \ -f ext2
libguestfs: command: run: \ --host-cpu x86_64
libguestfs: command: run: \ /usr/lib/x86_64-linux-gnu/guestfs/supermin.d
libguestfs: command: run: \ -o /var/tmp/.guestfs-1000/appliance.d
supermin: version: 5.1.19
supermin: package handler: debian/dpkg
supermin: acquiring lock on /var/tmp/.guestfs-1000/lock
supermin: build: /usr/lib/x86_64-linux-gnu/guestfs/supermin.d
supermin: reading the supermin appliance
supermin: build: visiting /usr/lib/x86_64-linux-gnu/guestfs/supermin.d/base.tar.gz type gzip base image (tar)
supermin: build: visiting /usr/lib/x86_64-linux-gnu/guestfs/supermin.d/daemon.tar.gz type gzip base image (tar)
supermin: build: visiting /usr/lib/x86_64-linux-gnu/guestfs/supermin.d/excludefiles type uncompressed excludefiles
supermin: build: visiting /usr/lib/x86_64-linux-gnu/guestfs/supermin.d/hostfiles type uncompressed hostfiles
supermin: build: visiting /usr/lib/x86_64-linux-gnu/guestfs/supermin.d/init.tar.gz type gzip base image (tar)
supermin: build: visiting /usr/lib/x86_64-linux-gnu/guestfs/supermin.d/packages type uncompressed packages
supermin: build: visiting /usr/lib/x86_64-linux-gnu/guestfs/supermin.d/packages-hfsplus type uncompressed packages
supermin: build: visiting /usr/lib/x86_64-linux-gnu/guestfs/supermin.d/packages-reiserfs type uncompressed packages
supermin: build: visiting /usr/lib/x86_64-linux-gnu/guestfs/supermin.d/packages-xfs type uncompressed packages
supermin: build: visiting /usr/lib/x86_64-linux-gnu/guestfs/supermin.d/udev-rules.tar.gz type gzip base image (tar)
supermin: mapping package names to installed packages
supermin: resolving full list of package dependencies
supermin: build: 215 packages, including dependencies
supermin: build: 9953 files
supermin: build: 6727 files, after matching excludefiles
supermin: build: 6730 files, after adding hostfiles
supermin: build: 6727 files, after removing unreadable files
supermin: build: 6730 files, after munging
supermin: kernel: looking for kernel using environment variables ...
supermin: kernel: looking for kernels in /lib/modules/*/vmlinuz ...
supermin: kernel: looking for kernels in /boot ...
supermin: kernel: kernel version of /boot/vmlinuz-4.15.0-36-generic = 4.15.0-36-generic (from filename)
supermin: kernel: picked modules path /lib/modules/4.15.0-36-generic
supermin: kernel: picked vmlinuz /boot/vmlinuz-4.15.0-36-generic
supermin: kernel: kernel_version 4.15.0-36-generic
supermin: kernel: modpath /lib/modules/4.15.0-36-generic
cp: cannot open '/boot/vmlinuz-4.15.0-36-generic' for reading: Permission denied
supermin: cp -p '/boot/vmlinuz-4.15.0-36-generic' '/var/tmp/.guestfs-1000/appliance.d.mv7vaue2/kernel': command failed, see earlier errors
libguestfs: error: /usr/bin/supermin exited with error status 1, see debug messages above
libguestfs: trace: launch = -1 (error)
libguestfs: trace: close
libguestfs: closing guestfs handle 0x562c6e1c50d0 (state 0)
libguestfs: command: run: rm
libguestfs: command: run: \ -rf /tmp/libguestfsS9Wa6T
```
5 years, 7 months
Re: [Libguestfs] Provide NBD via Browser over Websockets
by Eric Blake
[adding nbdkit readers]
On 10/13/18 1:39 PM, Eric Wheeler wrote:
> Hello all,
>
> It might be neat to attach ISOs to KVM guests via websockets. Basically
> the browser would be the NBD "server" and an NBD client would run on the
> hypervisor, then use `virsh change-media vm1 hdc --insert /dev/nbd0` could
> use an ISO from my desk to boot from.
Are you using qemu as the hypervisor? If you are using something else,
like vmware, then using the kernel NBD module as the NBD client to
expose the browser-as-server via /dev/nbd0 is reasonable since
practically any hypervisor will attach to a raw block device. But if
you are using qemu, I'd suggest that you use qemu directly as the NBD
client, rather than indirecting through the kernel module (for that
matter, qemu's NBD client knows how to efficiently handle sparse files
if the server is capable, while to date the kernel nbd module does not).
Since you are using virsh, the libvirt list is a better resource for
figuring out the command line and/or XML changes to direct qemu to
connect as an NBD client.
>
> Here's an HTML5 open file example:
> https://stackoverflow.com/questions/3582671/how-to-open-a-local-disk-file...
>
> and the NBD protocol looks simple enough to implement in javascript:
> https://stackoverflow.com/questions/17295140/where-is-the-network-block-d...
>
> What do you think? Does anyone have an interest in doing this?
The nbdkit project (under the libguestfs umbrella) has already
implemented an NBD server that can bridge to an image served over http,
if you want to use that as a starting point.
Then the obvious question - do you really need to implement a solution
using websockets through your browser in order to serve up a file on
your desktop, or can you reuse existing solutions of a userspace NBD
server to directly serve the file without involving the browser in the
middle.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org
5 years, 7 months
virt-builder fail with "bridge ‘virbr0’ not found" (fixed by LIBGUESTFS_BACKEND=direct ?!)
by Nir Soffer
I'm trying to build fedora-27 image for testing uploads:
$ virt-builder fedora-27 -o /var/tmp/fedora-27.img
...
[ 25.2] Opening the new disk
virt-builder: error: libguestfs error: bridge ‘virbr0’ not found. Try
running:
brctl show
to get a list of bridges on the host, and then selecting the
bridge you wish the appliance network to connect to using:
export _SETTINGS=network_bridge=<bridge name>
I don't want to add bridges to my new fresh oVirt test host, and
remember that this used to work previously without the bridge.
So I tried the usual "fix" (I know it should not be used in production):
$ export LIBGUESTFS_BACKEND=direct
And for some reason this works now:
$ virt-builder fedora-27 -o /var/tmp/fedora-27.img
...
[ 41.2] Finishing off
Output file: /var/tmp/fedora-27.img
Output size: 6.0G
Output format: raw
Total usable space: 5.3G
Free space: 4.4G (81%)
How LIBGUESTFS_BACKEND is related to the bridge? why do we need
the bridge for building an image?
Nir
5 years, 7 months
OpenStack output workflow
by Fabien Dupont
Hi,
There has been discussion about the OpenStack output and Richard asked for
a public thread on this list, so here it is.
For v2v from VMware to RHV, there is a Python script that does some extra
steps to create the virtual machine after the disks have been converted. We
want to have the same behavior for OpenStack, i.e. have virt-v2v create the
instance once the volumes have been created.
For that, I've written a Python script [1] that takes a JSON file (sample
here [2]) as input. I expect this JSON input to be generated by virt-v2v
openstack output module, from the command line options and the volumes ids
generated during conversion.
Here are the options I think we should have for the OpenStack output:
-o openstack
-oo os-auth-url='http://controller.example.com:5000/v3'
-oo os-user-domain-name='Default'
-oo os-project-name='v2v-project'
-oo os-username='admin'
-oo os-password='secret'
-oo server-id='01234567-89ab-cdef-0123-456789abcdef'
-oo destination_project_id='01234567-89ab-cdef-0123-456789abcdef'
-oo volume_type_id='01234567-89ab-cdef-0123-456789abcdef'
-oo flavor_id='01234567-89ab-cdef-0123-456789abcdef'
-oo
security_groups_ids='01234567-89ab-cdef-0123-456789abcdef,01234567-89ab-cdef-0123-456789abcdef'
--mac 01:23:45:67:89:ab:network:01234567-89ab-cdef-0123-456789abcdef
You'll see that the --mac option is not specific to OpenStack, but it shows
how it would look like with a network id. And it should be passed to the
post-conversion script.
The translation to JSON is pretty straight forward and should not be
difficult. We simply have to agree on the JSON keys we expect and the where
the new -oo keys go. Also, the script is quite simple and relies on
OpenStack Python SDK, which is also used by the OpenStack CLI, so no
additional dependencies are required and it should be easy to maintain.
[1]
https://gist.github.com/fdupont-redhat/934b3efb6d66a991a80149235066d7d7#f...
[2]
https://gist.github.com/fdupont-redhat/934b3efb6d66a991a80149235066d7d7#f...
--
*Fabien Dupont*
PRINCIPAL SOFTWARE ENGINEER
Red Hat - Solutions Engineering
fabien(a)redhat.com M: +33 (0) 662 784 971 <+33662784971>
<http://redhat.com> *TRIED. TESTED. TRUSTED.*
Twitter: @redhatway <https://twitter.com/redhatway> | Instagram: @redhatinc
<https://www.instagram.com/redhatinc/> | Snapchat: @redhatsnaps
5 years, 7 months
[PATCH] v2v: ovf: add firmware and machine type element
by Tomáš Golembiovský
Add oVirt specific elemnt to OVF. It represents the combination of
machine type (i440fx/q35) and firmware (BIOS/UEFI).
Signed-off-by: Tomáš Golembiovský <tgolembi(a)redhat.com>
---
v2v/create_ovf.ml | 20 +++++++++++++++++++-
v2v/create_ovf.mli | 2 +-
v2v/output_rhv.ml | 6 ++----
v2v/output_rhv_upload.ml | 4 ++--
v2v/output_vdsm.ml | 6 ++----
v2v/test-v2v-o-rhv.ovf.expected | 1 +
v2v/test-v2v-o-vdsm-options.ovf.expected | 1 +
7 files changed, 28 insertions(+), 12 deletions(-)
diff --git a/v2v/create_ovf.ml b/v2v/create_ovf.ml
index 2cf610333..1cab11dfd 100644
--- a/v2v/create_ovf.ml
+++ b/v2v/create_ovf.ml
@@ -462,6 +462,22 @@ let origin_of_source_hypervisor = function
*)
| _ -> None
+let get_ovirt_biostype guestcaps target_firmware =
+ let uefi_firmware =
+ match target_firmware with
+ | TargetBIOS -> None
+ | TargetUEFI -> Some (find_uefi_firmware guestcaps.gcaps_arch) in
+ let secure_boot_required =
+ match uefi_firmware with
+ | Some { Uefi.flags = flags }
+ when List.mem Uefi.UEFI_FLAG_SECURE_BOOT_REQUIRED flags -> true
+ | _ -> false in
+ match target_firmware, secure_boot_required with
+ | TargetUEFI, true -> 3 (* q35 + UEFI + secure boot *)
+ | TargetUEFI, _ -> 2 (* q35 + UEFI *)
+ (* 1 is q35 + SeaBIOS *)
+ | _, _ -> 0 (* i440fx + SeaBIOS *)
+
(* Generate the .meta file associated with each volume. *)
let create_meta_files output_alloc sd_uuid image_uuids targets =
(* Note: Upper case in the .meta, mixed case in the OVF. *)
@@ -506,7 +522,7 @@ let create_meta_files output_alloc sd_uuid image_uuids targets =
) (List.combine targets image_uuids)
(* Create the OVF file. *)
-let rec create_ovf source targets guestcaps inspect
+let rec create_ovf source targets guestcaps inspect target_firmware
output_alloc sd_uuid image_uuids vol_uuids vm_uuid ovf_flavour =
assert (List.length targets = List.length vol_uuids);
@@ -515,6 +531,7 @@ let rec create_ovf source targets guestcaps inspect
let vmtype = get_vmtype inspect in
let vmtype = match vmtype with `Desktop -> "0" | `Server -> "1" in
let ostype = get_ostype inspect in
+ let biostype = get_ovirt_biostype guestcaps target_firmware in
let ovf : doc =
doc "ovf:Envelope" [
@@ -562,6 +579,7 @@ let rec create_ovf source targets guestcaps inspect
e "VmType" [] [PCData vmtype];
(* See https://bugzilla.redhat.com/show_bug.cgi?id=1260590#c17 *)
e "DefaultDisplayType" [] [PCData "1"];
+ e "BiosType" [] [PCData (string_of_int biostype)];
] in
(match source.s_cpu_model with
diff --git a/v2v/create_ovf.mli b/v2v/create_ovf.mli
index 8200b76f9..cb6c12690 100644
--- a/v2v/create_ovf.mli
+++ b/v2v/create_ovf.mli
@@ -43,7 +43,7 @@ val ovf_flavour_to_string : ovf_flavour -> string
create OVF for another target management system then we would need
to heavily modify or even duplicate this code. *)
-val create_ovf : Types.source -> Types.target list -> Types.guestcaps -> Types.inspect -> Types.output_allocation -> string -> string list -> string list -> string -> ovf_flavour -> DOM.doc
+val create_ovf : Types.source -> Types.target list -> Types.guestcaps -> Types.inspect -> Types.target_firmware -> Types.output_allocation -> string -> string list -> string list -> string -> ovf_flavour -> DOM.doc
(** Create the OVF file.
Actually a {!DOM} document is created, not a file. It can be written
diff --git a/v2v/output_rhv.ml b/v2v/output_rhv.ml
index 5260ab030..ce57f0309 100644
--- a/v2v/output_rhv.ml
+++ b/v2v/output_rhv.ml
@@ -115,7 +115,7 @@ object
method as_options = sprintf "-o rhv -os %s" os
- method supported_firmware = [ TargetBIOS ]
+ method supported_firmware = [ TargetBIOS; TargetUEFI ]
(* RHV doesn't support serial consoles. This causes the conversion
* step to remove it.
@@ -270,12 +270,10 @@ object
(* This is called after conversion to write the OVF metadata. *)
method create_metadata source targets _ guestcaps inspect target_firmware =
- (* See #supported_firmware above. *)
- assert (target_firmware = TargetBIOS);
(* Create the metadata. *)
let ovf = Create_ovf.create_ovf source targets guestcaps inspect
- output_alloc esd_uuid image_uuids vol_uuids vm_uuid
+ target_firmware output_alloc esd_uuid image_uuids vol_uuids vm_uuid
Create_ovf.RHVExportStorageDomain in
(* Write it to the metadata file. *)
diff --git a/v2v/output_rhv_upload.ml b/v2v/output_rhv_upload.ml
index 63fa2411a..3dcd6d952 100644
--- a/v2v/output_rhv_upload.ml
+++ b/v2v/output_rhv_upload.ml
@@ -253,7 +253,7 @@ object
sprintf " -oc %s -op %s -os %s"
output_conn output_password output_storage
- method supported_firmware = [ TargetBIOS ]
+ method supported_firmware = [ TargetBIOS; TargetUEFI ]
(* rhev-apt.exe will be installed (if available). *)
method install_rhev_apt = true
@@ -407,7 +407,7 @@ If the messages above are not sufficient to diagnose the problem then add the
(* Create the metadata. *)
let ovf =
Create_ovf.create_ovf source targets guestcaps inspect
- output_alloc
+ target_firmware output_alloc
sd_uuid image_uuids vol_uuids vm_uuid
OVirt in
let ovf = DOM.doc_to_string ovf in
diff --git a/v2v/output_vdsm.ml b/v2v/output_vdsm.ml
index 9a1b748bc..f32acedae 100644
--- a/v2v/output_vdsm.ml
+++ b/v2v/output_vdsm.ml
@@ -123,7 +123,7 @@ object
| flav -> sprintf "-oo vdsm-ovf-flavour=%s"
(Create_ovf.ovf_flavour_to_string flav))
- method supported_firmware = [ TargetBIOS ]
+ method supported_firmware = [ TargetBIOS; TargetUEFI ]
(* RHV doesn't support serial consoles. This causes the conversion
* step to remove it.
@@ -240,11 +240,9 @@ object
(* This is called after conversion to write the OVF metadata. *)
method create_metadata source targets _ guestcaps inspect target_firmware =
- (* See #supported_firmware above. *)
- assert (target_firmware = TargetBIOS);
-
(* Create the metadata. *)
let ovf = Create_ovf.create_ovf source targets guestcaps inspect
+ target_firmware
output_alloc dd_uuid
vdsm_options.image_uuids
vdsm_options.vol_uuids
diff --git a/v2v/test-v2v-o-rhv.ovf.expected b/v2v/test-v2v-o-rhv.ovf.expected
index 7bcc456c5..c6bd05c56 100644
--- a/v2v/test-v2v-o-rhv.ovf.expected
+++ b/v2v/test-v2v-o-rhv.ovf.expected
@@ -25,6 +25,7 @@
<IsStateless>False</IsStateless>
<VmType>0</VmType>
<DefaultDisplayType>1</DefaultDisplayType>
+ <BiosType>0</BiosType>
<Section ovf:id='#VM_ID#' ovf:required='false' xsi:type='ovf:OperatingSystemSection_Type'>
<Info>Microsoft Windows 7 Phony Edition</Info>
<Description>Windows7</Description>
diff --git a/v2v/test-v2v-o-vdsm-options.ovf.expected b/v2v/test-v2v-o-vdsm-options.ovf.expected
index abaf37e54..f0d418b46 100644
--- a/v2v/test-v2v-o-vdsm-options.ovf.expected
+++ b/v2v/test-v2v-o-vdsm-options.ovf.expected
@@ -25,6 +25,7 @@
<IsStateless>False</IsStateless>
<VmType>0</VmType>
<DefaultDisplayType>1</DefaultDisplayType>
+ <BiosType>0</BiosType>
<OperatingSystemSection ovf:id='VM' ovf:required='false' ovirt:id='11'>
<Info>Microsoft Windows 7 Phony Edition</Info>
<Description>Windows7</Description>
--
2.18.0
5 years, 7 months
Re: [Libguestfs] LZO compression for NBD ?
by Eric Blake
[adding libguestfs list, for nbdkit reference below]
On 10/4/18 8:39 AM, Stefan Fröberg wrote:
> Hello.
> Is it possible to improve NBD throughtput with LZO compression ?
As in, have a way for the client and server to negotiate that both
understand an LZO extension, at which point the client can request a
read or write with a flag set to mark that the data is LZO-compressed,
for less network traffic but higher CPU usage between the client and server?
The idea might have merit, you're certainly welcome to propose it as an
extension (we can help with word-smithing the extension into the NBD
specification and ideas of the best way to negotiate the feature); it is
also helpful if you also plan on showing a reference implementation of
both a server and client implementing the extension.
Would it always have to be LZO compression, or would it be smarter to
have the negotiation phase pass a list of names of compression
algorithms, where clients and servers can both advertise their list of
algorithms they support, and then settle on one that is common to both
sides? Is it better to request that ALL read/write transactions
automatically be compressed by the negotiated algorithm, or only the
specific transactions where the client requests compression (after all,
the overhead of compressing a small transaction might be worse than just
sending the data directly; while the benefits of increased CPU times for
compression/decompression are more likely to matter for large
transactions where the transmission time becomes the bottleneck).
On a different track, you may want to try some experiments with LZO
compression by using nbdkit. Assuming that the real bottleneck that you
are trying to speed up is TCP/IP traffic, while local NBD connections
over a Unix socket are much faster, you could replace:
client => uncompressed over TCP/IP => server
with
client => uncompressed over Unix => nbdkit with custom plugin =>
compressed over TCP/IP => custom wrapper => server
where you write a wrapper on the server side to perform the LZO
compression (nbdkit does not yet provide a plugin library for easy
writing of a client, but maybe it should), then a plugin on the client
side that can interpret your LZO-compressed stream on the client side.
nbdkit already comes with a plugin that can read from .xz files
(slightly different from compressing individual transactions). But the
point remains that if what you are trying to design is an alternative
network stream with compressed data while still having a traditional NBD
interface once the data is local, then using nbdkit or other
intermediaries as boundary points can prove useful.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org
5 years, 7 months
[PATCH v2] lib: Use qemu-img info -U option to avoid locking error.
by Richard W.M. Jones
https://bugs.launchpad.net/qemu/+bug/1740364
---
lib/guestfs-internal.h | 3 +++
lib/handle.c | 2 ++
lib/info.c | 39 +++++++++++++++++++++++++++++++++++++++
3 files changed, 44 insertions(+)
diff --git a/lib/guestfs-internal.h b/lib/guestfs-internal.h
index adeb9478a..c66c55e70 100644
--- a/lib/guestfs-internal.h
+++ b/lib/guestfs-internal.h
@@ -510,6 +510,9 @@ struct guestfs_h {
/* Cached features. */
struct cached_feature *features;
size_t nr_features;
+
+ /* Used by lib/info.c. -1 = not tested or error; else 0 or 1. */
+ int qemu_img_supports_U_option;
};
/**
diff --git a/lib/handle.c b/lib/handle.c
index a47aaafab..297ff6d67 100644
--- a/lib/handle.c
+++ b/lib/handle.c
@@ -101,6 +101,8 @@ guestfs_create_flags (unsigned flags, ...)
g->memsize = DEFAULT_MEMSIZE;
+ g->qemu_img_supports_U_option = -1; /* not tested, see lib/info.c */
+
/* Start with large serial numbers so they are easy to spot
* inside the protocol.
*/
diff --git a/lib/info.c b/lib/info.c
index 2eadc1c11..74e4424b8 100644
--- a/lib/info.c
+++ b/lib/info.c
@@ -57,6 +57,7 @@ cleanup_json_t_decref (void *ptr)
#endif
static json_t *get_json_output (guestfs_h *g, const char *filename);
+static int qemu_img_supports_U_option (guestfs_h *g);
static void set_child_rlimits (struct command *);
char *
@@ -149,6 +150,11 @@ get_json_output (guestfs_h *g, const char *filename)
guestfs_int_cmd_add_arg (cmd, "qemu-img");
guestfs_int_cmd_add_arg (cmd, "info");
+ switch (qemu_img_supports_U_option (g)) {
+ case -1: return NULL;
+ case 0: break;
+ default: guestfs_int_cmd_add_arg (cmd, "-U");
+ }
guestfs_int_cmd_add_arg (cmd, "--output");
guestfs_int_cmd_add_arg (cmd, "json");
if (filename[0] == '/')
@@ -218,3 +224,36 @@ set_child_rlimits (struct command *cmd)
guestfs_int_cmd_set_child_rlimit (cmd, RLIMIT_CPU, 10 /* seconds */);
#endif
}
+
+/**
+ * Test if the qemu-img info command supports the C<-U> option to
+ * disable locking. The result is memoized in the handle.
+ *
+ * Note this option was added in qemu 2.11. We can remove this test
+ * when we can assume everyone is using qemu >= 2.11.
+ */
+static int
+qemu_img_supports_U_option (guestfs_h *g)
+{
+ if (g->qemu_img_supports_U_option >= 0)
+ return g->qemu_img_supports_U_option;
+
+ CLEANUP_CMD_CLOSE struct command *cmd = guestfs_int_new_command (g);
+ int r;
+
+ guestfs_int_cmd_add_string_unquoted (cmd,
+ "qemu-img info --help | "
+ "grep -sq -- 'info.*-U'");
+ r = guestfs_int_cmd_run (cmd);
+ if (r == -1)
+ return -1;
+ if (!WIFEXITED (r)) {
+ guestfs_int_external_command_failed (g, r,
+ "qemu-img info -U option test",
+ NULL);
+ return -1;
+ }
+
+ g->qemu_img_supports_U_option = WEXITSTATUS (r) == 0;
+ return g->qemu_img_supports_U_option;
+}
--
2.19.0.rc0
5 years, 7 months
[PATCH] v2v: -it vddk: Add a warning about using vddk* options (RHBZ#1527334).
by Richard W.M. Jones
Stress these options shouldn't be used unless you know what you're
doing.
---
v2v/virt-v2v.pod | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/v2v/virt-v2v.pod b/v2v/virt-v2v.pod
index fbc5733a6..18db7206b 100644
--- a/v2v/virt-v2v.pod
+++ b/v2v/virt-v2v.pod
@@ -439,7 +439,8 @@ See L</INPUT FROM VDDK> below for details.
When using VDDK mode, these options are passed unmodified to the
L<nbdkit(1)> VDDK plugin. Please refer to L<nbdkit-vddk-plugin(1)>.
-These are all optional.
+Do not use these options unless you know what you are doing. These
+are all optional.
=item B<-ip> filename
@@ -1894,6 +1895,7 @@ Other options that you might need to add in rare circumstances include
I<-io vddk-config>, I<-io vddk-cookie>, I<-io vddk-nfchostport>,
I<-io vddk-port>, I<-io vddk-snapshot>, and I<-io vddk-transports>,
which are all explained in the L<nbdkit-vddk-plugin(1)> documentation.
+Do not use these options unless you know what you are doing.
=head2 VDDK: DEBUGGING VDDK FAILURES
--
2.19.0.rc0
5 years, 7 months
[PATCH] v2v: Remove ‘-io vimapiver’ option (RHBZ#1527334).
by Richard W.M. Jones
This option was added in error. It never had any effect and now the
nbdkit VDDK plugin ignores it. Virt-v2v users shouldn't have been
using it.
This removes the option completely (so if anyone was using it they
will now get an error).
---
v2v/cmdline.ml | 2 --
v2v/input_libvirt_vddk.ml | 4 +---
v2v/test-v2v-docs.sh | 1 -
v2v/virt-v2v.pod | 7 ++-----
4 files changed, 3 insertions(+), 11 deletions(-)
diff --git a/v2v/cmdline.ml b/v2v/cmdline.ml
index cd2d279e9..6ec076151 100644
--- a/v2v/cmdline.ml
+++ b/v2v/cmdline.ml
@@ -259,8 +259,6 @@ let parse_cmdline () =
s_"Same as ‘-io vddk-thumbprint=thumbprint’";
[ L"vddk-transports" ], Getopt.String ("transports", set_input_option_compat "vddk-transports"),
s_"Same as ‘-io vddk-transports=transports’";
- [ L"vddk-vimapiver" ], Getopt.String ("apiver", set_input_option_compat "vddk-vimapiver"),
- s_"Same as ‘-io vddk-vimapiver=apiver’";
[ L"vdsm-compat" ], Getopt.String ("0.10|1.1", set_output_option_compat "vdsm-compat"),
s_"Same as ‘-oo vdsm-compat=0.10|1.1’";
[ L"vdsm-image-uuid" ], Getopt.String ("uuid", set_output_option_compat "vdsm-image-uuid"),
diff --git a/v2v/input_libvirt_vddk.ml b/v2v/input_libvirt_vddk.ml
index f74ca05af..2ca0ef1e0 100644
--- a/v2v/input_libvirt_vddk.ml
+++ b/v2v/input_libvirt_vddk.ml
@@ -44,8 +44,7 @@ let vddk_option_keys =
"port";
"snapshot";
"thumbprint";
- "transports";
- "vimapiver" ]
+ "transports" ]
let print_input_options () =
printf (f_"Input options (-io) which can be used with -it vddk:
@@ -64,7 +63,6 @@ All other settings are optional:
VDDK snapshot moref
-io vddk-transports=MODE:MODE:..
VDDK transports
- -io vddk-vimapiver=APIVER VDDK vimapiver
Refer to nbdkit-vddk-plugin(1) and the VDDK documentation for further
information on these settings.
diff --git a/v2v/test-v2v-docs.sh b/v2v/test-v2v-docs.sh
index 64dfe5492..dfb12bb14 100755
--- a/v2v/test-v2v-docs.sh
+++ b/v2v/test-v2v-docs.sh
@@ -46,7 +46,6 @@ $top_srcdir/podcheck.pl virt-v2v.pod virt-v2v \
--vddk-snapshot,\
--vddk-thumbprint,\
--vddk-transports,\
---vddk-vimapiver,\
--vdsm-compat,\
--vdsm-image-uuid,\
--vdsm-ovf-flavour,\
diff --git a/v2v/virt-v2v.pod b/v2v/virt-v2v.pod
index e028540e2..fbc5733a6 100644
--- a/v2v/virt-v2v.pod
+++ b/v2v/virt-v2v.pod
@@ -437,8 +437,6 @@ See L</INPUT FROM VDDK> below for details.
=item B<-io vddk-transports=>MODE:MODE:...
-=item B<-io vddk-vimapiver=>APIVER
-
When using VDDK mode, these options are passed unmodified to the
L<nbdkit(1)> VDDK plugin. Please refer to L<nbdkit-vddk-plugin(1)>.
These are all optional.
@@ -1894,9 +1892,8 @@ SSL thumbprint:
Other options that you might need to add in rare circumstances include
I<-io vddk-config>, I<-io vddk-cookie>, I<-io vddk-nfchostport>,
-I<-io vddk-port>, I<-io vddk-snapshot>, I<-io vddk-transports> and
-I<-io vddk-vimapiver>, which are all explained in the
-L<nbdkit-vddk-plugin(1)> documentation.
+I<-io vddk-port>, I<-io vddk-snapshot>, and I<-io vddk-transports>,
+which are all explained in the L<nbdkit-vddk-plugin(1)> documentation.
=head2 VDDK: DEBUGGING VDDK FAILURES
--
2.19.0.rc0
5 years, 7 months