[PATCH 0/2] python: improved UTF8 decoding error handling
by Matteo Cafasso
The Python 3 bindings currently are unable to deal with non UTF8 characters.
This series continues what proposed in RHBZ#1406906.
A new function 'set_decode_error_handler' allows the User to specify
how to deal with decoding errors.
The default behaviour will be raising a UnicodeDecodeError.
If the handler is changed to 'surrogateescape', non UTF8 characters will be escaped
in a similar manner as for Python 2.
See PEP383 for reference.
This series fixes also a bug introduced in commit 9d25b4e56471f9c33ea6229a8b620fc800c240f8.
Matteo Cafasso (2):
python: return bytes when return value is RBufferOut
python: unicode decode handler error scheme setter
generator/python.ml | 19 ++++++++++++++++++-
python/handle.c | 28 ++++++++++++++++++++++++++--
python/t/test830RHBZ1406906.py | 6 ++++++
3 files changed, 50 insertions(+), 3 deletions(-)
--
2.11.0
6 years, 11 months
Re: [Libguestfs] virt-resize
by Richard W.M. Jones
On Thu, Jun 15, 2017 at 09:04:06AM +0000, JuanEsteban.Jimenez(a)mdc-berlin.de wrote:
[...]
> This is my starting point. I am using your instructions as posted
> here: http://libguestfs.org/virt-resize.1.html
>
> # virt-filesystems --long -h --all -a img-1.qcow2
> Name Type VFS Label MBR Size Parent
> /dev/sda1 filesystem xfs - - 500M -
> /dev/centos/root filesystem xfs - - 15G -
> /dev/centos/swap filesystem swap - - 820M -
> /dev/centos/root lv - - - 15G /dev/centos
> /dev/centos/swap lv - - - 820M /dev/centos
> /dev/centos vg - - - 16G /dev/sda2
> /dev/sda2 pv - - - 16G -
> /dev/sda1 partition - - 83 500M /dev/sda
> /dev/sda2 partition - - 8e 16G /dev/sda
> /dev/sda device - - - 32G -
>
> My goal is to end up with an image that has a 32G root
> filesystem. To do that I use the command:
>
> virt-resize --expand /dev/sda2 --LV-expand /dev/centos/root img-1.qcow2 img-1a.qcow2
>
> # virt-filesystems --long -h --all -a img-1a.qcow2
> Name Type VFS Label MBR Size Parent
> /dev/sda1 filesystem xfs - - 500M -
> /dev/centos/root filesystem xfs - - 15G -
The fact that /dev/centos/root (XFS filesystem) wasn't resized here
indicates that you don't have the correct XFS subpackage installed.
Virt-resize would have given you a warning about this, but in any case
you need to install the ‘libguestfs-xfs’ package on the host.
> /dev/centos/swap filesystem swap - - 820M -
> /dev/centos/root lv - - - 15G /dev/centos
> /dev/centos/swap lv - - - 820M /dev/centos
> /dev/centos vg - - - 32G /dev/sda2
> /dev/sda2 pv - - - 32G -
> /dev/sda1 partition - - 83 500M /dev/sda
> /dev/sda2 partition - - 8e 32G /dev/sda
> /dev/sda device - - - 32G -
>
> I then booted the VM and tries xfs_growfs on /dev/mapper/centos-root
> and it doesn’t do anything but show me the filesystem geometry.
You shouldn't need to resize anything in the guest, virt-resize
should do it - see above.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW
6 years, 11 months
Re: [Libguestfs] virt-customize issue/question
by Richard W.M. Jones
On Fri, Jun 09, 2017 at 08:07:57PM -0400, Richard Maloley II wrote:
> Richard,
>
> Hello - I am hoping that you can shed some light on an issue I am
> encountering. This could be due to my inexperience with KVM/Libvirtd
> and the associated tools.
>
> Scenario: I am writing a VM deployment script for my home lab. From
> my admin workstation I can connect to my KVM host and execute a
> virt-clone operation without issue. I then want to virt-customize
> the new VM. This ends up in an error.
>
> Essentially I believe it is telling me that it can’t find the disk
> file. However I can validate that the file exists on my KVM Host:
[list of images]
> My question: Is virt-customize attempting to look at my local system
> (admin jump box) instead of the libvirtd connection to KVM02? Or is
> this a bug? Or did I simply miss an option that I need to include?
When connected to a remote libvirtd, virt-customize simply asks
libvirt for a list of the domains, but then it tries to open the disk
images returned as regular files. The libvirt XML doesn't contain any
information that disk images are remote nor how to connect to remote
files.
There are however several ways to edit remote files, although none of
them are completely transparent. Probably the simplest method is to
share /var/lib/libvirt/images over NFS so it appears at the same path
on every machine.
More complex methods include NBD, iSCSI, ssh:// URLs, etc. See:
http://libguestfs.org/guestfish.1.html#adding-remote-storage
These URLs should work with virt-customize too if you use a recent
enough version.
You might also find my minimal cloud software useful, although this
does not use remote storage, but relies on the controller being able
to use passwordless ssh to connect to each cloud node:
http://git.annexia.org/?p=mclu.git;a=summary
https://rwmj.wordpress.com/2015/03/23/mini-cloudcluster-v2-0/
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines. Supports shell scripting,
bindings from many languages. http://libguestfs.org
6 years, 11 months
[PATCH] daemon: lvm: Pass --yes option to force pvresize (RHBZ#1460577).
by Richard W.M. Jones
LVM2 >= 2.02.171 requires the ‘--yes’ option to force pvresize to work
in various circumstances, eg. reducing the size of an existing PV.
Pass this flag unconditionally.
Note this does NOT break earlier versions which just ignore this flag.
---
daemon/lvm.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/daemon/lvm.c b/daemon/lvm.c
index c186a5a5e..6c57046ff 100644
--- a/daemon/lvm.c
+++ b/daemon/lvm.c
@@ -625,6 +625,7 @@ do_pvresize_size (const char *device, int64_t size)
r = command (NULL, &err,
str_lvm, "pvresize",
+ "--yes",
"--setphysicalvolumesize", buf,
device, NULL);
if (r == -1) {
--
2.12.0
6 years, 11 months
[PATCH] lib: direct, uml: Unblock SIGTERM in the hypervisor and recovery processes (RHBZ#1460338).
by Richard W.M. Jones
If SIGTERM is blocked in the main program, then it ends up still being
blocked in the subprocess after we fork. This means that we cannot
kill qemu by sending SIGTERM to it. This commit fixes the problem by
unblocking SIGTERM unconditionally after fork.
Thanks: wtfuzz on IRC for reporting and analysis.
---
lib/launch-direct.c | 15 +++++++++++++++
lib/launch-uml.c | 11 +++++++++++
2 files changed, 26 insertions(+)
diff --git a/lib/launch-direct.c b/lib/launch-direct.c
index 8558cde5e..168ee367b 100644
--- a/lib/launch-direct.c
+++ b/lib/launch-direct.c
@@ -361,6 +361,7 @@ launch_direct (guestfs_h *g, void *datav, const char *arg)
const char *cpu_model;
CLEANUP_FREE char *append = NULL;
CLEANUP_FREE_STRING_LIST char **argv = NULL;
+ sigset_t sigset;
/* At present you must add drives before starting the appliance. In
* future when we enable hotplugging you won't need to do this.
@@ -728,6 +729,13 @@ launch_direct (guestfs_h *g, void *datav, const char *arg)
close_file_descriptors (fd > 2);
}
+ /* Unblock the SIGTERM signal since we will need to send that to
+ * the subprocess (RHBZ#1460338).
+ */
+ sigemptyset (&sigset);
+ sigaddset (&sigset, SIGTERM);
+ sigprocmask (SIG_UNBLOCK, &sigset, NULL);
+
/* Dump the command line (after setting up stderr above). */
if (g->verbose)
qemuopts_to_channel (qopts, stderr);
@@ -781,6 +789,13 @@ launch_direct (guestfs_h *g, void *datav, const char *arg)
*/
close_file_descriptors (1);
+ /* Unblock the SIGTERM signal since we will need to respond to
+ * SIGTERM from the parent (RHBZ#1460338).
+ */
+ sigemptyset (&sigset);
+ sigaddset (&sigset, SIGTERM);
+ sigprocmask (SIG_UNBLOCK, &sigset, NULL);
+
/* It would be nice to be able to put this in the same process
* group as qemu (ie. setpgid (0, qemu_pid)). However this is
* not possible because we don't have any guarantee here that
diff --git a/lib/launch-uml.c b/lib/launch-uml.c
index eccc102f5..547ab5a69 100644
--- a/lib/launch-uml.c
+++ b/lib/launch-uml.c
@@ -147,6 +147,7 @@ launch_uml (guestfs_h *g, void *datav, const char *arg)
size_t i;
struct hv_param *hp;
char *term = getenv ("TERM");
+ sigset_t sigset;
if (!uml_supported (g))
return -1;
@@ -323,6 +324,11 @@ launch_uml (guestfs_h *g, void *datav, const char *arg)
close_file_descriptors (fd > 2 && fd != dsv[1]);
}
+ /* RHBZ#1460338. */
+ sigemptyset (&sigset);
+ sigaddset (&sigset, SIGTERM);
+ sigprocmask (SIG_UNBLOCK, &sigset, NULL);
+
/* Dump the command line (after setting up stderr above). */
if (g->verbose)
print_vmlinux_command_line (g, cmdline.argv);
@@ -369,6 +375,11 @@ launch_uml (guestfs_h *g, void *datav, const char *arg)
*/
close_file_descriptors (1);
+ /* RHBZ#1460338 */
+ sigemptyset (&sigset);
+ sigaddset (&sigset, SIGTERM);
+ sigprocmask (SIG_UNBLOCK, &sigset, NULL);
+
/* It would be nice to be able to put this in the same process
* group as vmlinux (ie. setpgid (0, vmlinux_pid)). However
* this is not possible because we don't have any guarantee here
--
2.13.0
6 years, 11 months
[PATCH] daemon: ldm: Use CLEANUP_FREE on list of static strings.
by Richard W.M. Jones
This likely causes a double-free in the ‘ldmtool_scan_devices’ API.
---
daemon/ldm.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/daemon/ldm.c b/daemon/ldm.c
index 8ad2fb33b..7753b0d82 100644
--- a/daemon/ldm.c
+++ b/daemon/ldm.c
@@ -316,7 +316,7 @@ do_ldmtool_scan_devices (char * const * devices)
{
char **ret;
size_t i, nr_devices;
- CLEANUP_FREE_STRING_LIST const char **argv = NULL;
+ CLEANUP_FREE const char **argv = NULL;
int r;
CLEANUP_FREE char *out = NULL, *err = NULL;
--
2.13.0
6 years, 11 months
[PATCH 00/12] Refactor utility functions.
by Richard W.M. Jones
This turned out to be rather more involved than I thought.
We have lots of utility functions, spread all over the repository,
with not a lot of structure. This moves many of them under common/
and structures them so there are clear dependencies.
This doesn't complete the job by any means. Other items I had on my
to-do list for this change were:
- Split up mllib/common_utils into:
- bindings around common/options to common/mloptions
- util functions (such as error handling) used by the virt-*
OCaml utilities to common/mltools (Tools_utils)
- Move mllib/uri* -> common/mloptions
- Other mllib/ modules used only by tools can also be moved
to common/mltools, eg. curl, JSON, regedit, ...
Rich.
6 years, 11 months
[PATCH] lib: create: Allow any [[:alnum:]]+ string as a backingfmt parameter.
by Richard W.M. Jones
If you use the libguestfs tools which open disk images read-only
(eg. virt-df), with formats such as 'vdi', then you will see an error:
error: invalid value for backingformat parameter 'vdi'
This is because opening a disk image read-only will try to create a
qcow2 file with the original image as a backing file. However the
list of permitted backing formats was very restrictive and did not
include 'vdi' (nor many other uncommon formats).
Instead of using a whitelist for backing formats, just validate that
the string is alphanumeric and short.
Thanks: Mike Goodwin for reporting the bug.
---
lib/create.c | 15 +++++++++------
1 file changed, 9 insertions(+), 6 deletions(-)
diff --git a/lib/create.c b/lib/create.c
index bd4c32ef7..fff5cf332 100644
--- a/lib/create.c
+++ b/lib/create.c
@@ -241,6 +241,14 @@ is_power_of_2 (unsigned v)
return v && ((v & (v - 1)) == 0);
}
+/**
+ * Check for valid backing format. Allow any C<^[[:alnum]]+$>
+ * (in C locale), but limit the length to something reasonable.
+ */
+#define VALID_FORMAT(format) \
+ guestfs_int_string_is_valid ((format), 1, 16, \
+ VALID_FLAG_ALPHA|VALID_FLAG_DIGIT, "")
+
static int
disk_create_qcow2 (guestfs_h *g, const char *orig_filename, int64_t size,
const char *backingfile,
@@ -267,12 +275,7 @@ disk_create_qcow2 (guestfs_h *g, const char *orig_filename, int64_t size,
if (optargs->bitmask & GUESTFS_DISK_CREATE_BACKINGFORMAT_BITMASK) {
backingformat = optargs->backingformat;
- /* Conservative whitelist. This can be extended with other
- * valid formats as required.
- */
- if (STRNEQ (backingformat, "raw") &&
- STRNEQ (backingformat, "qcow2") &&
- STRNEQ (backingformat, "vmdk")) {
+ if (!VALID_FORMAT (backingformat)) {
error (g, _("invalid value for backingformat parameter ‘%s’"),
backingformat);
return -1;
--
2.13.0
6 years, 11 months
