HELP
by Shashwat shagun
Hi,
i cannot import libguestfs golang lib,
go get package libguestfs.org/guestfs: unrecognized import path "
libguestfs.org/guestfs" (https fetch: Get https://libguestfs.org/
guestfs?go-get=1: dial tcp [2001:41c9:1:422::215]:443: getsockopt:
connection refused)
Any ideas?
has the url changed or something?
--
Regards,
Shashwat Shagun
6 years, 6 months
[PATCH] tests: Provide full path to Unix domain sockets.
by Richard W.M. Jones
Maybe a bug in libvirt?
https://www.redhat.com/archives/libvir-list/2018-June/msg00490.html
---
tests/test-cache.sh | 3 ++-
tests/test-cow.sh | 3 ++-
2 files changed, 4 insertions(+), 2 deletions(-)
diff --git a/tests/test-cache.sh b/tests/test-cache.sh
index e5e9e82..ac99ce2 100755
--- a/tests/test-cache.sh
+++ b/tests/test-cache.sh
@@ -32,6 +32,7 @@
# SUCH DAMAGE.
set -e
+set -x
files="cache.img cache.sock cache.pid"
rm -f $files
@@ -69,7 +70,7 @@ cleanup ()
trap cleanup INT QUIT TERM EXIT ERR
# Open the overlay and perform some operations.
-guestfish --format=raw -a 'nbd://?socket=cache.sock' <<'EOF'
+guestfish --format=raw -a "nbd://?socket=`pwd`/cache.sock" <<'EOF'
run
part-disk /dev/sda gpt
mkfs ext4 /dev/sda1
diff --git a/tests/test-cow.sh b/tests/test-cow.sh
index 443ba38..4b798c8 100755
--- a/tests/test-cow.sh
+++ b/tests/test-cow.sh
@@ -32,6 +32,7 @@
# SUCH DAMAGE.
set -e
+set -x
files="cow-base.img cow-diff.qcow2 cow.sock cow.pid"
rm -f $files
@@ -70,7 +71,7 @@ cleanup ()
trap cleanup INT QUIT TERM EXIT ERR
# Write some data into the overlay.
-guestfish --format=raw -a 'nbd://?socket=cow.sock' -m /dev/sda1 <<EOF
+guestfish --format=raw -a "nbd://?socket=`pwd`/cow.sock" -m /dev/sda1 <<EOF
fill-dir / 10000
fill-pattern "abcde" 5M /large
write /hello "hello, world"
--
2.16.2
6 years, 6 months
[PATCH nbdkit] locks: Remove debugging messages about
by Richard W.M. Jones
The messages are not really useful to us, but they do bloat the
debugging output of virt-v2v massively:
nbdkit: python[1]: debug: acquire global request lock
nbdkit: python[1]: debug: acquire per-connection request lock
nbdkit: python[1]: debug: acquire unload prevention lock
nbdkit: python[1]: debug: pwrite count=2097152 offset=4628414464 fua=0
nbdkit: python[1]: debug: release unload prevention lock
nbdkit: python[1]: debug: release per-connection request lock
nbdkit: python[1]: debug: release global request lock
We probably need some kind of level of verbosity to deal with this
sort of thing, but that's quite a large change.
Rich.
6 years, 6 months
[PATCH 0/3] v2v: Various refactorings.
by Richard W.M. Jones
Use -ip instead of --password-file, and various refactorings.
It strikes me that we should probably deprecate and eventually remove
virt-v2v-copy-to-local. With the introduction of the new SSH and VDDK
transports, and with RHEL 5 Xen becoming more irrelevant, it's no
longer needed.
Rich.
6 years, 6 months
FYI: CVE-2018-11806 Qemu: slirp: heap buffer overflow while reassembling fragmented datagrams
by Richard W.M. Jones
There's a heap overflow in qemu SLIRP which affects libguestfs,
potentially allowing a malicious filesystem to take control of the
confining qemu process and from there attack the host.
It will affect libguestfs specifically when these two conditions are
both met:
- You're using the ‘direct’ backend.
- Networking is enabled.
The direct backend is the default upstream, but not in
Fedora/RHEL/CentOS. It might also have been selected if you set the
LIBGUESTFS_BACKEND=direct environment variable or called
‘guestfs_set_backend (g, "direct")’.
Networking is enabled automatically by some tools (eg. virt-builder),
or is enabled if your code called ‘guestfs_set_network (g, 1)’ (which
is not the default).
Note that the libvirt backend is _not_ affected, for two reasons:
(1) It doesn't use SLIRP for networking (2) libvirt will use SELinux
to confine the qemu process, so even if the appliance gains full
control over qemu it is limited in what it can do.
The solution in any case is to upgrade to a qemu containing the fix.
Here's the upstream qemu patch:
https://lists.gnu.org/archive/html/qemu-devel/2018-06/msg01012.html
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine. Supports Linux and Windows.
http://people.redhat.com/~rjones/virt-df/
6 years, 6 months
[PATCH] v2v: Log full imageio response on failure.
by Richard W.M. Jones
Thanks: Nir Soffer
---
v2v/rhv-upload-plugin.py | 66 ++++++++++++++++++++++++++++--------------------
1 file changed, 39 insertions(+), 27 deletions(-)
diff --git a/v2v/rhv-upload-plugin.py b/v2v/rhv-upload-plugin.py
index 791c9e7d2..c3de7d555 100644
--- a/v2v/rhv-upload-plugin.py
+++ b/v2v/rhv-upload-plugin.py
@@ -228,6 +228,29 @@ def can_flush(h):
def get_size(h):
return params['disk_size']
+# Any unexpected HTTP response status from the server will end up
+# calling this function which logs the full error, pauses the
+# transfer, sets the failed state, and raises a RuntimeError
+# exception.
+def unexpected_response(h, r, msg):
+ # Setting the failed flag in the handle causes the disk to be
+ # cleaned up on close.
+ h['failed'] = True
+ h['transfer_service'].pause()
+
+ status = r.status
+ reason = r.reason
+ body = r.read()
+
+ # Log the full error if we're verbose.
+ debug("unexpected response from imageio server:")
+ debug(msg)
+ debug("%d: %s" % (status, reason))
+ debug(body)
+
+ # Only a short error is included in the exception.
+ raise RuntimeError("%s: %d: %s" % (msg, status, reason))
+
# For documentation see:
# https://github.com/oVirt/ovirt-imageio/blob/master/docs/random-io.md
# For examples of working code to read/write from the server, see:
@@ -248,16 +271,14 @@ def pread(h, count, offset):
r = http.getresponse()
# 206 = HTTP Partial Content.
if r.status != 206:
- h['transfer_service'].pause()
- h['failed'] = True
- raise RuntimeError("could not read sector (%d, %d): %d: %s" %
- (offset, count, r.status, r.reason))
+ unexpected_response(h, r,
+ "could not read sector offset %d size %d" %
+ (offset, count))
return r.read()
def pwrite(h, buf, offset):
http = h['http']
transfer = h['transfer']
- transfer_service = h['transfer_service']
count = len(buf)
h['highestwrite'] = max(h['highestwrite'], offset+count)
@@ -275,15 +296,13 @@ def pwrite(h, buf, offset):
r = http.getresponse()
if r.status != 200:
- transfer_service.pause()
- h['failed'] = True
- raise RuntimeError("could not write sector (%d, %d): %d: %s" %
- (offset, count, r.status, r.reason))
+ unexpected_response(h, r,
+ "could not write sector offset %d size %d" %
+ (offset, count))
def zero(h, count, offset, may_trim):
http = h['http']
transfer = h['transfer']
- transfer_service = h['transfer_service']
# Unlike the trim and flush calls, there is no 'can_zero' method
# so nbdkit could call this even if the server doesn't support
@@ -306,10 +325,9 @@ def zero(h, count, offset, may_trim):
r = http.getresponse()
if r.status != 200:
- transfer_service.pause()
- h['failed'] = True
- raise RuntimeError("could not zero sector (%d, %d): %d: %s" %
- (offset, count, r.status, r.reason))
+ unexpected_response(h, r,
+ "could not zero sector offset %d size %d" %
+ (offset, count))
def emulate_zero(h, count, offset):
# qemu-img convert starts by trying to zero/trim the whole device.
@@ -334,15 +352,13 @@ def emulate_zero(h, count, offset):
r = http.getresponse()
if r.status != 200:
- transfer_service.pause()
- h['failed'] = True
- raise RuntimeError("could not write zeroes (%d, %d): %d: %s" %
- (offset, count, r.status, r.reason))
+ unexpected_response(h, r,
+ "could not write zeroes offset %d size %d" %
+ (offset, count))
def trim(h, count, offset):
http = h['http']
transfer = h['transfer']
- transfer_service = h['transfer_service']
# Construct the JSON request for trimming.
buf = json.dumps({'op': "trim",
@@ -358,15 +374,13 @@ def trim(h, count, offset):
r = http.getresponse()
if r.status != 200:
- transfer_service.pause()
- h['failed'] = True
- raise RuntimeError("could not trim sector (%d, %d): %d: %s" %
- (offset, count, r.status, r.reason))
+ unexpected_response(h, r,
+ "could not trim sector offset %d size %d" %
+ (offset, count))
def flush(h):
http = h['http']
transfer = h['transfer']
- transfer_service = h['transfer_service']
# Construct the JSON request for flushing.
buf = json.dumps({'op': "flush"}).encode()
@@ -379,9 +393,7 @@ def flush(h):
r = http.getresponse()
if r.status != 200:
- transfer_service.pause()
- h['failed'] = True
- raise RuntimeError("could not flush: %d: %s" % (r.status, r.reason))
+ unexpected_response(h, r, "could not flush")
def delete_disk_on_failure(h):
disk_service = h['disk_service']
--
2.16.2
6 years, 6 months
virt-builder download issues
by Alex Buchanan
I'm having trouble downloading templates (ubuntu-18.04, among others) using virt-builder. The downloads get to about 50% (or 13, or 62) and then stall. wget easily downloads the files quickly.
I'd be happy with working around this by manually placing the images in the cache directory, but I can't seem to figure out the appropriate placement. virt-builder doesn't find the cached file.
Please send help,
Thanks!
6 years, 6 months
[PATCH v2] daemon: inspect: better handling windows drive mapping.
by Mykola Ivanets
I saw several Windows disk images which contains strange registry entry
for mapped drives:
"\\DosDevices\\Y:"=hex(3):00,00,00,00,00,00,00,00,00,00,00,00
Which is decoded something like diskID = 0x0, partition starts at 0
bytes offset from the start of the disk. In addition to a Windows disk
image, I have attached dummy disk and made xfs file system on a whole
device without partitioning it. I mount xfs file system to a "/" and
then mkdir and mount other found file systems inside (/fs1, /fs2 etc.).
When we decode drive mappings we are looking for a disk with ID 0x0 (it
is 4 bytes somewhere LBA0). It is appeared that dummy non-partitioned
disk with xfs file system has zeros by offset where diskID is expected
to be). So the disk is considered as a candidate to search for
partition at offset 0. part-list command (and "parted" which is used
under the hood) reports there is 1 partition on "dummy" disk which
starts exactly at offset 0. And thus dummy device name and partition
number are simply concatenated together and corresponding drive mapping
is returned: Y => /dev/sdX1. But /dev/sdX1 is not existing block
device.
No matter either it is a bug in "parted" (or it works this way
by-design), let's protect ourself from this situation: in addition we
look for msdos partition table on a disk before making any further
assumptions.
---
daemon/inspect_fs_windows.ml | 11 ++++++++---
1 file changed, 8 insertions(+), 3 deletions(-)
diff --git a/daemon/inspect_fs_windows.ml b/daemon/inspect_fs_windows.ml
index 8b2aad8d3..af52dee2d 100644
--- a/daemon/inspect_fs_windows.ml
+++ b/daemon/inspect_fs_windows.ml
@@ -329,7 +329,7 @@ and get_drive_mappings h root data =
String.is_prefix blob "DMIO:ID:" (* GPT *) then
map_registry_disk_blob_gpt (Lazy.force partitions) blob
else if String.length blob = 12 then
- map_registry_disk_blob (Lazy.force devices) blob
+ map_registry_disk_blob_mbr (Lazy.force devices) blob
else
None
)
@@ -356,13 +356,18 @@ and get_drive_mappings h root data =
* The following function maps this blob to a libguestfs partition
* name, if possible.
*)
-and map_registry_disk_blob devices blob =
+and map_registry_disk_blob_mbr devices blob =
try
(* First 4 bytes are the disk ID. Search all devices to find the
* disk with this disk ID.
*)
let diskid = String.sub blob 0 4 in
- let device = List.find (fun dev -> pread dev 4 0x01b8 = diskid) devices in
+ let device =
+ List.find (
+ fun dev ->
+ Parted.part_get_parttype dev = "msdos" &&
+ pread dev 4 0x01b8 = diskid
+ ) devices in
(* Next 8 bytes are the offset of the partition in bytes(!) given as
* a 64 bit little endian number. Luckily it's easy to get the
--
2.17.0
6 years, 7 months
[PATCH v8 0/6] daemon: list_filesystems: filter out block devices which cannot hold filesystem.
by Mykola Ivanets
v8: - Rebased on top of master.
v7: - Addresses comments after v6 series review.
v6: - Addresses comments after v5 series review.
- Large commit is splitted to more granular commits for better code review.
v5: - Addresses comments after v4 series review (part_get_mbr_part_type doesn't break original implementation in C).
- Rebased on top of master and little bit refactored for readability.
v4: - Rebased on top of master.
v3: - Rebased on top of master.
v2: - First implementation.
v1: - Initial discussion.
This patch series filters out block devices which cannot hold filesystems:
- partitioned MD devices;
- LDM partitions (only LDM volume can hold filesystem);
- Windows Logical Disk Manager data partition;
- Microsoft Reserved Partition;
- Windows Snapshot Partition;
- MBR extended partition.
Mykola Ivanets (6):
daemon: Changing the way that we detect if a device contains
partitions.
daemon: list-filesystems: Ignore partitioned MD devices.
tests: list-filesystems command ignores partitioned MD devices.
daemon: list-filesystems: Change the way we filter out LDM partitions.
daemon: list-filesystems: Filter out Microsoft Reserved and Windows
Snapshot partitions.
daemon: list-filesystems: Filter out MBR extended partitions.
daemon/listfs.ml | 132 ++++++++++++++----------
tests/md/Makefile.am | 3 +-
tests/md/test-partitioned-md-devices.sh | 79 ++++++++++++++
3 files changed, 157 insertions(+), 57 deletions(-)
create mode 100755 tests/md/test-partitioned-md-devices.sh
--
2.17.0
6 years, 7 months
