[PATCH] inspector: fix help text (RHBZ#1544674)
by Pino Toscano
---
inspector/inspector.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/inspector/inspector.c b/inspector/inspector.c
index 3583c61df..d608b1b63 100644
--- a/inspector/inspector.c
+++ b/inspector/inspector.c
@@ -79,8 +79,8 @@ usage (int status)
printf (_("%s: display information about a virtual machine\n"
"Copyright (C) 2010 Red Hat Inc.\n"
"Usage:\n"
- " %s [--options] -d domname file [file ...]\n"
- " %s [--options] -a disk.img [-a disk.img ...] file [file ...]\n"
+ " %s [--options] -d domname\n"
+ " %s [--options] -a disk.img [-a disk.img ...]\n"
"Options:\n"
" -a|--add image Add image\n"
" -c|--connect uri Specify libvirt URI for -d option\n"
--
2.14.3
7 years, 1 month
[PATCH] inspect: recognize the Kali Linux distribution (RHBZ#1544227)
by Pino Toscano
Read the information from os-release, and make it behave like a
Debian distribution (i.e. using dpkg, and apt).
---
daemon/inspect_fs.ml | 2 ++
daemon/inspect_fs_unix.ml | 1 +
daemon/inspect_types.ml | 2 ++
daemon/inspect_types.mli | 1 +
generator/actions_inspection.ml | 4 ++++
5 files changed, 10 insertions(+)
diff --git a/daemon/inspect_fs.ml b/daemon/inspect_fs.ml
index 2db218bb2..383e3e0a7 100644
--- a/daemon/inspect_fs.ml
+++ b/daemon/inspect_fs.ml
@@ -275,6 +275,7 @@ and check_package_format { distro } =
| Some DISTRO_SUSE_BASED ->
Some PACKAGE_FORMAT_RPM
| Some DISTRO_DEBIAN
+ | Some DISTRO_KALI_LINUX
| Some DISTRO_LINUX_MINT
| Some DISTRO_UBUNTU ->
Some PACKAGE_FORMAT_DEB
@@ -344,6 +345,7 @@ and check_package_management { distro; version } =
| Some DISTRO_ALTLINUX
| Some DISTRO_DEBIAN
+ | Some DISTRO_KALI_LINUX
| Some DISTRO_LINUX_MINT
| Some DISTRO_UBUNTU ->
Some PACKAGE_MANAGEMENT_APT
diff --git a/daemon/inspect_fs_unix.ml b/daemon/inspect_fs_unix.ml
index d0323af3f..53c86e878 100644
--- a/daemon/inspect_fs_unix.ml
+++ b/daemon/inspect_fs_unix.ml
@@ -140,6 +140,7 @@ and distro_of_os_release_id = function
| "debian" -> Some DISTRO_DEBIAN
| "fedora" -> Some DISTRO_FEDORA
| "frugalware" -> Some DISTRO_FRUGALWARE
+ | "kali" -> Some DISTRO_KALI_LINUX
| "mageia" -> Some DISTRO_MAGEIA
| "neokylin" -> Some DISTRO_NEOKYLIN
| "opensuse" -> Some DISTRO_OPENSUSE
diff --git a/daemon/inspect_types.ml b/daemon/inspect_types.ml
index 464b8c2a6..70b34c51c 100644
--- a/daemon/inspect_types.ml
+++ b/daemon/inspect_types.ml
@@ -78,6 +78,7 @@ and distro =
| DISTRO_FREEDOS
| DISTRO_FRUGALWARE
| DISTRO_GENTOO
+ | DISTRO_KALI_LINUX
| DISTRO_LINUX_MINT
| DISTRO_MAGEIA
| DISTRO_MANDRIVA
@@ -208,6 +209,7 @@ and string_of_distro = function
| DISTRO_FREEDOS -> "freedos"
| DISTRO_FRUGALWARE -> "frugalware"
| DISTRO_GENTOO -> "gentoo"
+ | DISTRO_KALI_LINUX -> "kalilinux"
| DISTRO_LINUX_MINT -> "linuxmint"
| DISTRO_MAGEIA -> "mageia"
| DISTRO_MANDRIVA -> "mandriva"
diff --git a/daemon/inspect_types.mli b/daemon/inspect_types.mli
index 78051b510..7493aa3a6 100644
--- a/daemon/inspect_types.mli
+++ b/daemon/inspect_types.mli
@@ -85,6 +85,7 @@ and distro =
| DISTRO_FREEDOS
| DISTRO_FRUGALWARE
| DISTRO_GENTOO
+ | DISTRO_KALI_LINUX
| DISTRO_LINUX_MINT
| DISTRO_MAGEIA
| DISTRO_MANDRIVA
diff --git a/generator/actions_inspection.ml b/generator/actions_inspection.ml
index 0ac282435..d8cf571c9 100644
--- a/generator/actions_inspection.ml
+++ b/generator/actions_inspection.ml
@@ -210,6 +210,10 @@ Frugalware.
Gentoo.
+=item \"kalilinux\"
+
+Kali Linux.
+
=item \"linuxmint\"
Linux Mint.
--
2.14.3
7 years, 1 month
[PATCH v2 0/1] RFC: switch from YAJL to Jansson
by Pino Toscano
Hi,
recently, there was a discussion in the development list of libvirt on
switching to a different JSON library than YAJL [1]. Since we use YAJL,
and the points there IMHO apply to libguestfs as well, I decided to give
a try in switching to Jansson [2].
The result IMHO is nice, with the additional APIs of Jansson that
simplify some of our code. Unlike with YAJL, I did not set a minimum
required version, as I did not use any feature in newer versions (simply
because there was no need for them), so even older versions should work.
Of course the test suite works for me, although it does not cover the
LDM APIs.
[1] https://www.redhat.com/archives/libvir-list/2017-November/msg00225.html
[2] http://www.digip.org/jansson/
Pino Toscano (1):
Switch from YAJL to Jansson
appliance/packagelist.in | 13 ++--
builder/Makefile.am | 4 +-
builder/yajl-c.c | 66 ++++++++++-----------
contrib/p2v/aux-scripts/do-build.sh | 8 +--
contrib/p2v/build-p2v-iso.sh | 3 +-
daemon/Makefile.am | 4 +-
daemon/ldm.c | 115 ++++++++++++++++--------------------
docs/guestfs-building.pod | 2 +-
lib/Makefile.am | 4 +-
lib/info.c | 113 ++++++++++++++---------------------
lib/qemu.c | 64 +++++++-------------
m4/guestfs-libraries.m4 | 4 +-
12 files changed, 169 insertions(+), 231 deletions(-)
--
2.14.3
7 years, 1 month
Re: [Libguestfs] [PATCH] Add a cache for iconv_t handles to hive_t
by Richard W.M. Jones
On Sat, Feb 10, 2018 at 01:40:36AM +0100, Hilko Bengen wrote:
> * Richard W.M. Jones:
>
> >> +threadlib
> >> '
> >
> > Probably better to keep these sorted.
>
> When I rebuilt from scratch, it turned out that threadlib was not
> enough. Using the "lock" module with the gl_lock_{init,lock,unlock}
> instead of glthread_lock_{init,lock,unlock} as described in lock.h seems
> correct, though. I have pushed that change.
>
> > I wonder if there's a way we can avoid hard-coding ‘4’ here, which
> > AIUI is the size of the enum type. Maybe adding an extra enum case
> > ‘nr_recode_types’?
>
> Sure, done.
Great, thanks.
I'm going to do a 1.3.15 release soonish.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines. Supports shell scripting,
bindings from many languages. http://libguestfs.org
7 years, 1 month
[PATCH] Add a cache for iconv_t handles to hive_t
by Hilko Bengen
It was brought to my attention that dumping a registry hive causes a
lot of time spent in disk I/O activity because iconv_open() and
iconv_close() are called for every key. Every iconv_open() call causes
/usr/lib/.../gconv/$ENCODING.so to be opened and mapped.
The iconv_t handles are now cached in the hive_h struct; they are
opened on-demand and re-used.
On my ~10 year old Lenovo T60, I have seen 57% savings in the overal
runtime of running
hivexregedit --export windows-8-enterprise-software.hive '\\'
---
lib/handle.c | 43 ++++++++++++++++++++++++++++++++++++++++++-
lib/hivex-internal.h | 31 ++++++++++++++++++++++---------
lib/node.c | 6 +++---
lib/utf16.c | 38 ++++++++++++++++----------------------
lib/value.c | 10 +++++-----
lib/write.c | 4 ++--
6 files changed, 90 insertions(+), 42 deletions(-)
diff --git a/lib/handle.c b/lib/handle.c
index 9dcf81d..7d715df 100644
--- a/lib/handle.c
+++ b/lib/handle.c
@@ -31,6 +31,9 @@
#include <errno.h>
#include <assert.h>
+#include <iconv.h>
+#include <pthread.h>
+
#ifdef HAVE_MMAP
#include <sys/mman.h>
#else
@@ -62,6 +65,32 @@ header_checksum (const hive_h *h)
#define HIVEX_OPEN_MSGLVL_MASK (HIVEX_OPEN_VERBOSE|HIVEX_OPEN_DEBUG)
+iconv_t *
+_hivex_get_iconv (hive_h *h, recode_type t)
+{
+ pthread_mutex_lock (&h->iconv_cache[t].mutex);
+ if (h->iconv_cache[t].handle == NULL) {
+ if (t == utf8_to_latin1)
+ h->iconv_cache[t].handle = iconv_open ("LATIN1", "UTF-8");
+ else if (t == latin1_to_utf8)
+ h->iconv_cache[t].handle = iconv_open ("UTF-8", "LATIN1");
+ else if (t == utf8_to_utf16le)
+ h->iconv_cache[t].handle = iconv_open ("UTF-16LE", "UTF-8");
+ else if (t == utf16le_to_utf8)
+ h->iconv_cache[t].handle = iconv_open ("UTF-8", "UTF-16LE");
+ } else {
+ /* reinitialize iconv context */
+ iconv (h->iconv_cache[t].handle, NULL, 0, NULL, 0);
+ }
+ return h->iconv_cache[t].handle;
+}
+
+void
+_hivex_release_iconv (hive_h *h, recode_type t)
+{
+ pthread_mutex_unlock (&h->iconv_cache[t].mutex);
+}
+
hive_h *
hivex_open (const char *filename, int flags)
{
@@ -164,11 +193,17 @@ hivex_open (const char *filename, int flags)
goto error;
}
+ for (int t=0; t<3; t++) {
+ pthread_mutex_init (&h->iconv_cache[t].mutex, NULL);
+ h->iconv_cache[t].handle = NULL;
+ }
+
/* Last modified time. */
h->last_modified = le64toh ((int64_t) h->hdr->last_modified);
if (h->msglvl >= 2) {
- char *name = _hivex_windows_utf16_to_utf8 (h->hdr->name, 64);
+ char *name = _hivex_recode (h, utf16le_to_utf8,
+ h->hdr->name, 64, NULL);
fprintf (stderr,
"hivex_open: header fields:\n"
@@ -424,6 +459,12 @@ hivex_close (hive_h *h)
else
r = 0;
free (h->filename);
+ for (int t=0; t<3; t++) {
+ if (h->iconv_cache[t].handle != NULL) {
+ iconv_close (h->iconv_cache[t].handle);
+ h->iconv_cache[t].handle = NULL;
+ }
+ }
free (h);
return r;
diff --git a/lib/hivex-internal.h b/lib/hivex-internal.h
index 9a497ed..fefdc81 100644
--- a/lib/hivex-internal.h
+++ b/lib/hivex-internal.h
@@ -23,6 +23,8 @@
#include <stddef.h>
#include <string.h>
+#include <iconv.h>
+
#include "byte_conversions.h"
#define STREQ(a,b) (strcmp((a),(b)) == 0)
@@ -35,6 +37,13 @@
#define STRCASENEQLEN(a,b,n) (strncasecmp((a),(b),(n)) != 0)
#define STRPREFIX(a,b) (strncmp((a),(b),strlen((b))) == 0)
+typedef enum {
+ utf8_to_latin1 = 0,
+ latin1_to_utf8,
+ utf8_to_utf16le,
+ utf16le_to_utf8,
+} recode_type;
+
struct hive_h {
char *filename;
int fd;
@@ -79,6 +88,11 @@ struct hive_h {
/* Internal data for mmap replacement */
void *p_winmap;
#endif
+
+ struct {
+ pthread_mutex_t mutex;
+ iconv_t *handle;
+ } iconv_cache[4];
};
/* Format of registry blocks. NB. All fields are little endian. */
@@ -282,17 +296,16 @@ extern void _hivex_free_offset_list (offset_list *list);
extern size_t * _hivex_return_offset_list (offset_list *list);
extern void _hivex_print_offset_list (offset_list *list, FILE *fp);
+/* handle.c */
+extern iconv_t * _hivex_get_iconv (hive_h *h, recode_type r);
+extern void _hivex_release_iconv (hive_h *h, recode_type r);
+
/* utf16.c */
-extern char * _hivex_recode (const char *input_encoding,
- const char *input, size_t input_len,
- const char *output_encoding, size_t *output_len);
-#define _hivex_windows_utf16_to_utf8(_input, _len) \
- _hivex_recode ("UTF-16LE", _input, _len, "UTF-8", NULL)
-#define _hivex_windows_latin1_to_utf8(_input, _len) \
- _hivex_recode ("LATIN1", _input, _len, "UTF-8", NULL)
-extern char* _hivex_encode_string(const char *str, size_t *size, int *utf16);
+extern char * _hivex_recode (hive_h *h, recode_type r,
+ const char *input, size_t input_len, size_t *output_len);
+extern char* _hivex_encode_string (hive_h *h, const char *str, size_t *size, int *utf16);
extern size_t _hivex_utf16_string_len_in_bytes_max (const char *str, size_t len);
-extern size_t _hivex_utf8_strlen (const char* str, size_t len, int utf16);
+extern size_t _hivex_utf8_strlen (hive_h *h, const char* str, size_t len, int utf16);
/* util.c */
extern void _hivex_free_strings (char **argv);
diff --git a/lib/node.c b/lib/node.c
index 36e61c4..21cd127 100644
--- a/lib/node.c
+++ b/lib/node.c
@@ -90,9 +90,9 @@ hivex_node_name (hive_h *h, hive_node_h node)
}
size_t flags = le16toh (nk->flags);
if (flags & 0x20) {
- return _hivex_windows_latin1_to_utf8 (nk->name, len);
+ return _hivex_recode (h, latin1_to_utf8, nk->name, len, NULL);
} else {
- return _hivex_windows_utf16_to_utf8 (nk->name, len);
+ return _hivex_recode (h, utf16le_to_utf8, nk->name, len, NULL);
}
}
@@ -116,7 +116,7 @@ hivex_node_name_len (hive_h *h, hive_node_h node)
return 0;
}
- return _hivex_utf8_strlen (nk->name, len, ! (le16toh (nk->flags) & 0x20));
+ return _hivex_utf8_strlen (h, nk->name, len, ! (le16toh (nk->flags) & 0x20));
}
diff --git a/lib/utf16.c b/lib/utf16.c
index 238f40a..c0f0b05 100644
--- a/lib/utf16.c
+++ b/lib/utf16.c
@@ -30,24 +30,21 @@
#include "hivex-internal.h"
char *
-_hivex_recode (const char *input_encoding, const char *input, size_t input_len,
- const char *output_encoding, size_t *output_len)
+_hivex_recode (hive_h *h, recode_type t,
+ const char *input, size_t input_len, size_t *output_len)
{
- iconv_t ic = iconv_open (output_encoding, input_encoding);
- if (ic == (iconv_t) -1)
- return NULL;
-
/* iconv(3) has an insane interface ... */
size_t outalloc = input_len;
+ iconv_t *ic = _hivex_get_iconv (h, t);
again:;
size_t inlen = input_len;
size_t outlen = outalloc;
char *out = malloc (outlen + 1);
if (out == NULL) {
int err = errno;
- iconv_close (ic);
+ _hivex_release_iconv (h, t);
errno = err;
return NULL;
}
@@ -56,18 +53,17 @@ _hivex_recode (const char *input_encoding, const char *input, size_t input_len,
size_t r = iconv (ic, (ICONV_CONST char **) &inp, &inlen, &outp, &outlen);
if (r == (size_t) -1) {
+ int err = errno;
if (errno == E2BIG) {
- int err = errno;
/* Reset errno here because we don't want to accidentally
* return E2BIG to a library caller.
*/
- errno = 0;
size_t prev = outalloc;
/* Try again with a larger output buffer. */
free (out);
outalloc *= 2;
if (outalloc < prev) {
- iconv_close (ic);
+ _hivex_release_iconv (h, t);
errno = err;
return NULL;
}
@@ -75,19 +71,17 @@ _hivex_recode (const char *input_encoding, const char *input, size_t input_len,
}
else {
/* Else some conversion failure, eg. EILSEQ, EINVAL. */
- int err = errno;
- iconv_close (ic);
+ _hivex_release_iconv (h, t);
free (out);
errno = err;
return NULL;
}
}
+ _hivex_release_iconv (h, t);
*outp = '\0';
- iconv_close (ic);
if (output_len != NULL)
*output_len = outp - out;
-
return out;
}
@@ -95,17 +89,17 @@ _hivex_recode (const char *input_encoding, const char *input, size_t input_len,
* storing in the hive file, as needed.
*/
char*
-_hivex_encode_string(const char *str, size_t *size, int *utf16)
+_hivex_encode_string (hive_h *h, const char *str, size_t *size, int *utf16)
{
char* outstr;
*utf16 = 0;
- outstr = _hivex_recode ("UTF-8", str, strlen(str),
- "LATIN1", size);
+ outstr = _hivex_recode (h, utf8_to_latin1,
+ str, strlen(str), size);
if (outstr != NULL)
return outstr;
*utf16 = 1;
- outstr = _hivex_recode ("UTF-8", str, strlen(str),
- "UTF-16LE", size);
+ outstr = _hivex_recode (h, utf8_to_utf16le,
+ str, strlen(str), size);
return outstr;
}
@@ -128,11 +122,11 @@ _hivex_utf16_string_len_in_bytes_max (const char *str, size_t len)
}
size_t
-_hivex_utf8_strlen (const char* str, size_t len, int utf16)
+_hivex_utf8_strlen (hive_h *h, const char* str, size_t len, int utf16)
{
- const char *encoding = utf16 ? "UTF-16LE" : "LATIN1";
+ recode_type t = utf16 ? utf16le_to_utf8 : latin1_to_utf8;
size_t ret = 0;
- char *buf = _hivex_recode(encoding, str, len, "UTF-8", &ret);
+ char *buf = _hivex_recode (h, t, str, len, &ret);
free(buf);
return ret;
}
diff --git a/lib/value.c b/lib/value.c
index 2dfe006..3257b53 100644
--- a/lib/value.c
+++ b/lib/value.c
@@ -209,7 +209,7 @@ hivex_value_key_len (hive_h *h, hive_value_h value)
SET_ERRNO (EFAULT, "key length is too long (%zu, %zu)", len, seg_len);
return 0;
}
- return _hivex_utf8_strlen (vk->name, len, ! (le16toh (vk->flags) & 0x01));
+ return _hivex_utf8_strlen (h, vk->name, len, ! (le16toh (vk->flags) & 0x01));
}
char *
@@ -232,9 +232,9 @@ hivex_value_key (hive_h *h, hive_value_h value)
return NULL;
}
if (flags & 0x01) {
- return _hivex_windows_latin1_to_utf8 (vk->name, len);
+ return _hivex_recode (h, latin1_to_utf8, vk->name, len, NULL);
} else {
- return _hivex_windows_utf16_to_utf8 (vk->name, len);
+ return _hivex_recode (h, utf16le_to_utf8, vk->name, len, NULL);
}
}
@@ -471,7 +471,7 @@ hivex_value_string (hive_h *h, hive_value_h value)
if (slen < len)
len = slen;
- char *ret = _hivex_windows_utf16_to_utf8 (data, len);
+ char *ret = _hivex_recode (h, utf16le_to_utf8, data, len, NULL);
free (data);
if (ret == NULL)
return NULL;
@@ -538,7 +538,7 @@ hivex_value_multiple_strings (hive_h *h, hive_value_h value)
}
ret = ret2;
- ret[nr_strings-1] = _hivex_windows_utf16_to_utf8 (p, plen);
+ ret[nr_strings-1] = _hivex_recode (h, utf16le_to_utf8, p, plen, NULL);
ret[nr_strings] = NULL;
if (ret[nr_strings-1] == NULL) {
_hivex_free_strings (ret);
diff --git a/lib/write.c b/lib/write.c
index 33b64e4..70105c9 100644
--- a/lib/write.c
+++ b/lib/write.c
@@ -610,7 +610,7 @@ hivex_node_add_child (hive_h *h, hive_node_h parent, const char *name)
size_t recoded_name_len;
int use_utf16 = 0;
char *recoded_name =
- _hivex_encode_string (name, &recoded_name_len, &use_utf16);
+ _hivex_encode_string (h, name, &recoded_name_len, &use_utf16);
if (recoded_name == NULL) {
SET_ERRNO (EINVAL, "malformed name");
return 0;
@@ -959,7 +959,7 @@ hivex_node_set_values (hive_h *h, hive_node_h node,
static const char vk_id[2] = { 'v', 'k' };
size_t recoded_name_len;
int use_utf16;
- char* recoded_name = _hivex_encode_string (values[i].key, &recoded_name_len,
+ char* recoded_name = _hivex_encode_string (h, values[i].key, &recoded_name_len,
&use_utf16);
seg_len = sizeof (struct ntreg_vk_record) + recoded_name_len;
size_t vk_offs = allocate_block (h, seg_len, vk_id);
--
2.11.0
7 years, 1 month
ANNOUNCE: libguestfs 1.38 released
by Richard W.M. Jones
I'm pleased to announce libguestfs 1.38, a library and a set of tools
for accessing and modifying virtual machine disk images.
This release represents about a year of work by many contributors.
I'd like to call out in particular substantial contributions from:
Cédric Bosdonnat, Pavel Butsykin, Matteo Cafasso, Tomáš Golembiovský,
Nikos Skalkotos, and Pino Toscano.
Virt-builder-repository is a new tool for creating virt-builder
repositories. Virt-rescue has been rewritten, implementing
implementing job control, -m and -i options, escape keys. Virt-v2v
has several new methods to pull VMs out of VMware faster. The
inspection code was rewritten and placed inside the daemon making it
much faster and more robust.
Of course dozens of other features have been added, and many bugs
fixed. See the release notes below for full details.
You can get libguestfs 1.38 from here:
Main website: http://libguestfs.org/ [not updated yet]
Source: http://libguestfs.org/download/1.38-stable/
Fedora: https://koji.fedoraproject.org/koji/packageinfo?packageID=8391
Debian/experimental: https://packages.debian.org/libguestfs0
Note for distro packagers: ocaml >= 4.01, ocaml-hivex are now
mandatory build dependencies. If using glibc >= 2.27 which dropped
support for SunRPC and crypt(3), you will need rpcgen, libtirpc and
libxcrypt.
Rich.
----------------------------------------------------------------------
Release notes for libguestfs 1.38
These are also available online at:
http://libguestfs.org/guestfs-release-notes.1.html
New features
New tools
Virt-builder-repository is a new tool allowing end users to create and
update virt-builder repositories (Cédric Bosdonnat).
Virt-rescue (while not a new tool) has been substantially rewritten,
implementing job control, -m and -i options, escape keys, etc.
New features in existing tools
Virt-builder planner has been improved so that faster and more
efficient build plans are chosen for complex cases, especially when
either the tmpdir or output is on networked storage.
New virt-builder Fedora templates (starting with Fedora 26) will have
plain partition layout and use GPT for partitions.
Virt-customize "firstboot" scripts in guests using systemd are now
installed under the "multi-user.target" instead of "default.target" so
they will only run when the system is booted normally.
Virt-customize now sets a random /etc/machine-id for Linux guests, if
one is not already set.
Virt-df now works correctly on filesystems with block sizes smaller
than 1K (Nikolay Ivanets).
Virt-dib has further compatibility enhancements with diskimage-builder
(Pino Toscano).
Virt-sysprep removes "DHCP_HOSTNAME" from ifcfg-* files.
Virt-sysprep now works on Oracle Linux (Jamie Iles).
Virt-resize now correctly copies GPT partition attributes from the
source to the destination (Cédric Bosdonnat).
Bash tab completion implemented or enhanced for: virt-win-reg,
virt-v2v-copy-to-local.
virt-v2v and virt-p2v
Virt-v2v can now read VMware VMX files directly, either from local
disk, NFS storage, or over SSH from an ESXi hypervisor.
Virt-v2v can now use VDDK as an input source.
Both virt-v2v and virt-p2v are now able to pass through the source CPU
vendor, model and topology. However unfortunately not all source and
target hypervisors are able to provide or consume this data at present
(Tomáš Golembiovský).
Virt-v2v now supports encrypted guests (Pino Toscano).
Virt-v2v can now handle VMware snapshots. Note that the snapshots are
collapsed — it does not convert the chain of snapshots into a chain of
snapshots.
Virt-v2v now installs Windows 10 / Windows Server 2016 virtio block
drivers correctly (Pavel Butsykin, Kun Wei).
Virt-v2v now installs virtio-rng, balloon and pvpanic drivers, and
correctly sets this in the target hypervisor metadata for hypervisors
which support that (Tomáš Golembiovský).
Virt-v2v now installs both legacy and modern virtio keys in the Windows
registry (Ladi Prosek).
Virt-p2v can now preserve (in some cases) the offset of the Real Time
Clock from UTC.
Virt-p2v now combines several scp commands to the conversion server
into a single command, improving conversion times.
Virt-v2v now detects the special Linux Xen PV-only kernels correctly
(Laszlo Ersek).
Virt-v2v -o glance now generates the right properties for UEFI guests
(Pino Toscano).
Virt-v2v -o null now avoids spooling the guest to a temporary file,
instead it writes to the qemu "null block device". This makes it
faster and use almost no disk space.
Virt-v2v -o rhv now supports Windows 2016 Server guest type.
Virt-v2v -i libvirtxml can now open network disks over http or https.
Virt-v2v will now give a warning about host passthrough devices (Pino
Toscano).
The virt-v2v --machine-readable output has been enhanced so it includes
"vcenter-https", "xen-ssh" and "in-place" facts (Pino Toscano).
Language bindings
Fix multiple memory leaks and other data corruption problems in the
Java bindings (Pino Toscano).
Perl %guestfs_introspection has been dropped.
Inspection
Inspection support was rewritten in OCaml and included inside the
daemon. This makes inspection considerably faster, more robust and
more easily extensible in future.
Better icon support for ALT Linux guests (Pino Toscano).
Better support for NeoKylin (Qingzheng Zhang).
Can handle OSes like Void Linux which do not include "VERSION_ID" in
/etc/os-release (Pino Toscano).
Add support for Microsoft MS-DOS (Daniel Berrangé).
Architectures and platforms
Multiple fixes for S/390 architecture. Libguestfs and all the tools
should now compile and run on this architecture.
Other
The libguestfs API is now thread-safe (although not parallel). You can
call APIs on the same handle from multiple threads without needing to
take a lock.
Security
There were multiple vulnerabilities in the icoutils "wrestool" program
which is run by libguestfs to create icons for Windows guests. Using
the latest "wrestool" is recommended.
API
New APIs
"hivex_value_string"
This replaces the deprecated "hivex_value_utf8" API, but does the
same thing.
"part_get_gpt_attributes"
"part_set_gpt_attributes"
Read and write GPT partition attribute flags (Cédric Bosdonnat).
"part_resize"
Enlarge or shrink an existing partition (Nikos Skalkotos).
"yara_destroy"
"yara_load"
"yara_scan"
Support for the Yara malware scanning engine (Matteo Cafasso).
Other API changes
APIs implemented in the daemon can now be written in either C or OCaml.
Several APIs were rewritten in OCaml, although we are not planning to
rewrite all of them.
You will now get a clear error message if you try to add too many disks
to the appliance, instead of getting a peculiar failure from qemu.
Certain APIs accidentally allowed you to use "/dev/urandom" as an input
"device", eg. "g.copy_device_to_device("/dev/urandom", "/dev/sda")".
The code has been modified to forbid this usage.
All APIs for inspecting installer CDs have been deprecated. Use
libosinfo for this task.
Build changes
A working OCaml compiler ≥ 4.01 is now required for building
libguestfs. The "./configure --disable-ocaml" option remains but is
only used to disable the OCaml language bindings.
Add "RELEASES" file which lists release dates for each version of
libguestfs. You must update this file when making a new release.
Documentation generated by "gtk-doc" has been removed. "./configure
--enable-gtk-doc" now does nothing.
Libtirpc is now used for XDR functions and rpcgen. Note that glibc has
deprecated and in most Linux distros dropped these, so for most people
this will be an extra dependency (Martin Kletzander).
Libxcrypt is now used for crypt(3). This is required if using glibc ≥
2.27.
"ocaml-hivex" is now required.
Libvirt ≥ 1.2.20 is now required.
There is now a "make check-root" target for tests which need to be run
as root (analogous to "make check-slow").
"./configure"-time check for "__attribute__((cleanup))" now works in
the cross-compilation case (Yann E. Morin).
The "AUTHORS" and "p2v/about-authors.c" files are now generated from a
single place.
Either GnuPG v1 or v2 can be used.
"./configure --with-guestfs-path" may be used to set the default
"LIBGUESTFS_PATH". In addition the way that the path is searched has
changed slightly so that all types of appliances are searched in each
path element separately (Pavel Butsykin).
"GUESTFSD_EXT_CMD" which was used to mark external commands in the
daemon has been removed. It was originally used by SUSE builds, but
they have not been using it for a while.
The output from "./configure" is now visually grouped under headings
related to what it is doing, making it much easier to scan (Pino
Toscano).
OCaml dependencies are now generated from a single script instead of
multiple not-quite-the-same Makefile fragments.
"./configure --with-distro=ID" can be used to override automatic Linux
distro detection at build time (Pino Toscano).
qemu ≥ 2.10 is supported (but not required). This adds mandatory
locking to disks and libguestfs turns this off in certain circumstances
when it is known to be safe (Lars Seipel, Peter Krempa, Daniel
Berrangé, Pino Toscano, Fam Zheng, Yongkui Guo, Václav Kadlčík).
Internals
Most common code has been moved to the common/ subdirectory, with OCaml
common code being in common/ml* directories (eg. common/visit and
common/mlvisit contain the visitor library in C and OCaml
respectively). The mllib directory has been deleted and replaced by
common/mltools.
There is now a lightweight OCaml binding for PCRE, see common/mlpcre.
Use of OCaml "Str" library has been mostly replaced with PCRE.
Add more calls to "udev_settle" to improve stability of partition code
(Dawid Zamirski).
Run "udev_settle" with --exit-if-exists option, which improves the
speed of this command (Pavel Butsykin).
Detect new locations of major(3), minor(3), makedev(3).
Actions can now be deprecated with no suggested replacement, for APIs
such as "guestfs_wait_ready" that should simply be removed from client
code.
Use gnulib "set_nonblocking_flag" wrapper instead of calling fcntl(2)
with "O_NONBLOCK" (Eric Blake). Similarly "set_cloexec_flag".
Fix memory leak in XFS version of "guestfs_vfs_minimum_size" (Pino
Toscano).
Valgrind checks now run on the virt-p2v binary.
Unicode single quotes ("‘’") and now used in place of '' or `'
throughout the code and documentation. Similarly for "’s" instead of
"'s".
The "is_zero" function has been reimplemented for greater speed (Eric
Blake).
In the direct backend, virtio-blk support has been removed. Virtio-
scsi is now the only supported way to add disks.
Generator string parameter and return types have been rationalised so
there are only two types ("String", "StringList") with many subtypes
eg. "FileIn" becomes "String (FileIn, ...)".
The appliance disk image can now be in formats other than raw (Pavel
Butsykin).
Multiple improvements to how we automatically build Debian templates
for virt-builder (Pino Toscano). Enable serial console for these
templates (Florian Klink).
In the daemon, instead of making a private copy of lvm.conf and
modifying it (eg for filters), start with an empty file since LVM
understands that to mean "all defaults" (Alasdair Kergon, Zdenek
Kabelac).
The "direct" backend can now run QMP queries against the QEMU binary,
enhancing the kinds of information we can detect. In addition the code
to query QEMU has been made more robust for handling multiple parallel
queries of different versions of QEMU.
OCaml Augeas bindings are bundled under common/mlaugeas. The long term
plan is to remove this and use system ocaml-augeas when it is more
widely available in distros (Pino Toscano).
All OCaml modules ("*.ml" files) are now required to have an interface
file ("*.mli"). If they don't export anything then the interface will
be empty except for comments.
Certain OCaml features in OCaml ≥ 4.01 are used throughout the code,
including replacing ‘{ field = field }’ with ‘{ field }’.
Virt-builder "make-template" utility now uses the "virt-install
--transient" option so that we should never need to clean up left over
domains after a crash. It also saves kickstarts and virt-install
commands, which are committed to git for future reference.
/dev/shm is now created in the appliance (Nicolas Hicher).
In verbose mode on Fedora guests, virt-customize will now use "dnf
--verbose" enabling better debugging output.
Virt-v2v input and output classes now contain a "#precheck" method
which is used to perform environmental checks before conversion starts.
Virt-p2v enables miniexpect debugging. It is written to stderr (of
virt-p2v).
Virt-v2v free space checks are more liberal especially for smaller
guests (Pino Toscano).
Bugs fixed
https://bugzilla.redhat.com/1540535
Example URI of "Convert from ESXi hypervisor over SSH to local
libvirt" is incorrect in v2v man page
https://bugzilla.redhat.com/1539395
virt-customize segfaults after upgrading to 1.37.35-3
https://bugzilla.redhat.com/1536765
Libguestfs Perl bindings can leak a small amount of memory on error
https://bugzilla.redhat.com/1536763
libguestfs Lua bindings use strerror(), which isn’t thread safe
https://bugzilla.redhat.com/1536603
man page makes no mention of using '--' when trying to change exit
on error behavior
https://bugzilla.redhat.com/1525241
virt-df displays zeros for filesystems with block size =512
https://bugzilla.redhat.com/1519204
v2v should improve the result when convert a rhel7.4 guest with no
available kernels found in the bootloader
https://bugzilla.redhat.com/1518517
virt-v2v fails with "unsupported configuration: shared access for
disk 'sdb' requires use of supported storage format"
https://bugzilla.redhat.com/1516094
Mere presence of QEMU file locking options breaks NBD (Block
protocol 'nbd' doesn't support the option 'locking')
https://bugzilla.redhat.com/1514756
./configure --disable-ocaml breaks building common/mlpcre which
breaks building daemon
https://bugzilla.redhat.com/1513884
[RFE]Should update some vddk info in v2v man page
https://bugzilla.redhat.com/1508874
virt-v2v: warning: ova disk has an unknown VMware controller type
(20)
https://bugzilla.redhat.com/1506572
virt-v2v '-i ova' is not parsing the MAC address from the source
OVF
https://bugzilla.redhat.com/1506511
virt-builder fails to parse repo file if it has blank space after
the repository identifier
https://bugzilla.redhat.com/1503958
Failed to convert the rhel5 guest with kmod-xenpv installed from
xen server by virt-v2v
https://bugzilla.redhat.com/1503497
qemu-kvm fails to open qcow2 files in read-only mode with qemu-kvm
1.5.3
https://bugzilla.redhat.com/1500673
Error info shows wrong sometimes when ssh to conversion server
using non-root user with sudo on p2v client
https://bugzilla.redhat.com/1500537
/dev/shm does not exist in the appliance environment
https://bugzilla.redhat.com/1497475
guestfish cannot list commands from interactive mode
https://bugzilla.redhat.com/1493048
Unbound constructor Hivex.OPEN_UNSAFE
https://bugzilla.redhat.com/1484957
bump debian images to use single-partition layout
https://bugzilla.redhat.com/1482737
virt-resize failed to expand swap partition for RHEL5.11 guest
image with "parsing UUID failed"
https://bugzilla.redhat.com/1477623
Running file API on a special chardev may hang forever
https://bugzilla.redhat.com/1476081
inspect-os report error: could not parse integer in version
number: V7Update2
https://bugzilla.redhat.com/1472719
[RFE]Add warning in process of v2v converting guest which has pci
passthrough device
https://bugzilla.redhat.com/1472208
virt-v2v fails on opensuse 13.2 guest with error: statns:
statns_stub: path must start with a / character
https://bugzilla.redhat.com/1469655
firstboot scripts are not correctly installed in Fedora 26
https://bugzilla.redhat.com/1466563
Libguestfs should pass copyonread flag through to the libvirt XML
https://bugzilla.redhat.com/1465665
1.36.x build failure: gtkdocize fails using newer autotools due to
missing GTK_DOC_CHECK in configure.ac
https://bugzilla.redhat.com/1460338
guestfs_shutdown hangs if main process sets signal handlers
https://bugzilla.redhat.com/1459979
guestfs_add_domain_argv fails with readonly option when vdi/vhd
disk is attached to libvirt domain
https://bugzilla.redhat.com/1451665
RFE: Virt-v2v can't convert the guest which has encrypted partition
https://bugzilla.redhat.com/1450325
document URI format for -a parameters of tools
https://bugzilla.redhat.com/1448739
RFE: Support multicore decompression for OVA files using pigz and
pxz
https://bugzilla.redhat.com/1447202
Win 2016 guest is described as Win 10 after imported to RHEVM
https://bugzilla.redhat.com/1441197
RFE: ability to convert VMware virtual machines via vmx
https://bugzilla.redhat.com/1438939
Please drop or update GnuPG (1.4.x) dependency
https://bugzilla.redhat.com/1438794
[RFE] Install Windows virtio-rng drivers on VMs imported
https://bugzilla.redhat.com/1433937
virt-inspector can't get icon info from altlinux-centaurus
https://bugzilla.redhat.com/1433577
policycoreutils setfiles >= 2.6 does .. nothing
https://bugzilla.redhat.com/1431579
Windows 8 UEFI from VMware to KVM fails to boot after conversion
https://bugzilla.redhat.com/1430680
There is error info about "No such file or directory" when convert
a guest from ova file by v2v
https://bugzilla.redhat.com/1430184
virt-dib should generate sha256 checksum instead of sha512
https://bugzilla.redhat.com/1429506
RFE: OVMF should be detected on conversion server to prevent failed
conversion
https://bugzilla.redhat.com/1429491
Should rename network name of rhv in virt-v2v man page
https://bugzilla.redhat.com/1427529
virt-sysprep should remove DHCP_HOSTNAME
https://bugzilla.redhat.com/1417306
QEMU image file locking (libguestfs)
https://bugzilla.redhat.com/1409024
[Debian] Missing db_dump abort inspection
https://bugzilla.redhat.com/1406906
Segmentation fault when reading corrupted path with Python 3
bindings
https://bugzilla.redhat.com/1379289
RFE: virt-p2v should support mnemonic operations
https://bugzilla.redhat.com/1378022
There is virt-v2v warning about <listen type='none'> during
converting a guest which has listen type='none' in XML
https://bugzilla.redhat.com/1376547
qemu-system-s390x: -device
isa-serial,chardev=charserial0,id=serial0: 'isa-serial' is not a
valid device model name
https://bugzilla.redhat.com/1374232
selinux relabel fails on RHEL 6.2 guests with "libguestfs error:
selinux_relabel: : Success"
https://bugzilla.redhat.com/1367738
Missing bash completion scripts for: virt-diff guestunmount virt-
copy-in virt-copy-out virt-customize virt-get-kernel
virt-p2v-make-disk virt-p2v-make-kickstart virt-tar-in virt-tar-out
virt-v2v-copy-to-local virt-win-reg
https://bugzilla.redhat.com/1362649
RFE: virt-sysprep does not utilize libguestfs encryption support
https://bugzilla.redhat.com/1172425
[RFE]virt-v2v failed to convert VMware ESX VM with snapshot
https://bugzilla.redhat.com/1171654
Modify a file in virt-rescue with vi on some linux terminal such as
yakuake, can lead to abnormal display in virt-rescue shell
https://bugzilla.redhat.com/1167623
Remove "If reporting bugs, run virt-v2v with debugging enabled .."
message when running virt-p2v
https://bugzilla.redhat.com/1152819
Can not end a running command in virt-rescue by press ^C or other
keys, the only way is to exit virt-rescue
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines. Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top
7 years, 1 month
[nbdkit PATCH v2 0/2] Improve nbdkit_parse_size
by Eric Blake
Take two, this time split into two patches. I liked Rich's
suggestion of unit-testing src/ files directly in src/, and
automake is a lot happier with this than with my v1 attempt
that tried to compile .c files across directories.
It's still enough of a change that I'm not pushing it right
away.
Eric Blake (2):
build: Add unit-testing of internal files
utils: Revamp nbdkit_parse_size
src/test-utils.c | 128 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
src/utils.c | 95 +++++++++++++++++++++++++++--------------
.gitignore | 1 +
src/Makefile.am | 15 ++++++-
4 files changed, 205 insertions(+), 34 deletions(-)
create mode 100644 src/test-utils.c
--
2.14.3
7 years, 1 month
[ for 1.38] ocaml: When creating compat Bytes module, create bytes.mli file.
by Richard W.M. Jones
On distros with OCaml < 4.02 we need to create a compatibility Bytes
module. However we didn't create the interface file (bytes.mli) which
would mean that dependencies wouldn't be created correctly for
parallel builds. This commit uses ‘ocaml -i’ to create an interface
file which exports everything.
---
.gitignore | 1 +
m4/guestfs-ocaml.m4 | 3 ++-
2 files changed, 3 insertions(+), 1 deletion(-)
diff --git a/.gitignore b/.gitignore
index 233a8c536..bb7026537 100644
--- a/.gitignore
+++ b/.gitignore
@@ -139,6 +139,7 @@ Makefile.in
/common/mlprogress/.depend
/common/mlstdutils/.depend
/common/mlstdutils/bytes.ml
+/common/mlstdutils/bytes.mli
/common/mlstdutils/guestfs_config.ml
/common/mlstdutils/oUnit-*
/common/mlstdutils/std_utils_tests
diff --git a/m4/guestfs-ocaml.m4 b/m4/guestfs-ocaml.m4
index 94c24bc04..e08f40a02 100644
--- a/m4/guestfs-ocaml.m4
+++ b/m4/guestfs-ocaml.m4
@@ -175,7 +175,7 @@ AM_CONDITIONAL([HAVE_OCAML_GETTEXT],
dnl Create the backwards compatibility Bytes module for OCaml < 4.02.
mkdir -p common/mlstdutils
-rm -f common/mlstdutils/bytes.ml
+rm -f common/mlstdutils/bytes.ml common/mlstdutils/bytes.mli
AS_IF([test "x$have_Bytes_module" = "xno"],[
cat > common/mlstdutils/bytes.ml <<EOF
include String
@@ -183,6 +183,7 @@ let of_string = String.copy
let to_string = String.copy
let sub_string = String.sub
EOF
+ $OCAMLC -i common/mlstdutils/bytes.ml > common/mlstdutils/bytes.mli
OCAML_BYTES_COMPAT_CMO='$(top_builddir)/common/mlstdutils/bytes.cmo'
OCAML_BYTES_COMPAT_ML='$(top_builddir)/common/mlstdutils/bytes.ml'
safe_string_option=
--
2.14.3
7 years, 1 month
libguestfs operations failing
by George Koenig
Hello all,
I'm able to add (>add-ro image) a newly created image (created with
vmbuilder) using the guestfish shell, but unable to run it. I appear to be
getting kernel panics when doing so. In trying to debug the issue, I
noticed that libguestfs-test-tool also fails with a kernel panic due to
sync errors. I had previously suspected an issue with the kernel file
permissions but this was resolved by allowing read access.
I've attached output from both running the guestfish shell, and
libguestfs-test-tool. I am running Ubuntu Xenial (16.04), and guestfish
1.32.2 installed via apt.
Thanks. Please let me know if there is anything else I can provide that
will assist.
Regards,
George Koenig
7 years, 1 month
[PATCH v2] daemon: add inspector support for MS-DOS distro
by Daniel P. Berrangé
An installation of MS-DOS has various files in a /DOS directory,
which COMMAND.COM looking like a reasonable signal that its MS-DOS
or a very close relative there-of.
This is validated with an MS-DOS 6.22 install.
Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com>
---
daemon/inspect_fs.ml | 14 ++++++++++++++
daemon/inspect_types.ml | 2 ++
daemon/inspect_types.mli | 1 +
generator/actions_inspection.ml | 4 ++++
4 files changed, 21 insertions(+)
diff --git a/daemon/inspect_fs.ml b/daemon/inspect_fs.ml
index 13ed127cd..a20de6761 100644
--- a/daemon/inspect_fs.ml
+++ b/daemon/inspect_fs.ml
@@ -220,6 +220,18 @@ and check_filesystem mountable =
*)
data.arch <- Some "i386"
)
+ (* MS-DOS *)
+ else if is_dir_nocase "/DOS" &&
+ is_file_nocase "/DOS/COMMAND.COM" then (
+ debug_matching "MS-DOS";
+ role := `Root;
+ data.os_type <- Some OS_TYPE_DOS;
+ data.distro <- Some DISTRO_MSDOS;
+ (* MS-DOS is a mix of 16 and 32 bit, but
+ * assume it requires a 32 bit i386 processor.
+ *)
+ data.arch <- Some "i386"
+ )
(* None of the above. *)
else (
debug_matching "no known OS partition"
@@ -282,6 +294,7 @@ and check_package_format { distro } =
| Some DISTRO_FREEBSD
| Some DISTRO_FREEDOS
| Some DISTRO_FRUGALWARE
+ | Some DISTRO_MSDOS
| Some DISTRO_NETBSD
| Some DISTRO_OPENBSD
| Some DISTRO_PLD_LINUX
@@ -365,6 +378,7 @@ and check_package_management { distro; version } =
| Some DISTRO_FREEBSD
| Some DISTRO_FREEDOS
| Some DISTRO_FRUGALWARE
+ | Some DISTRO_MSDOS
| Some DISTRO_NETBSD
| Some DISTRO_OPENBSD
| Some DISTRO_PLD_LINUX
diff --git a/daemon/inspect_types.ml b/daemon/inspect_types.ml
index 473939a67..464b8c2a6 100644
--- a/daemon/inspect_types.ml
+++ b/daemon/inspect_types.ml
@@ -82,6 +82,7 @@ and distro =
| DISTRO_MAGEIA
| DISTRO_MANDRIVA
| DISTRO_MEEGO
+ | DISTRO_MSDOS
| DISTRO_NEOKYLIN
| DISTRO_NETBSD
| DISTRO_OPENBSD
@@ -211,6 +212,7 @@ and string_of_distro = function
| DISTRO_MAGEIA -> "mageia"
| DISTRO_MANDRIVA -> "mandriva"
| DISTRO_MEEGO -> "meego"
+ | DISTRO_MSDOS -> "msdos"
| DISTRO_NEOKYLIN -> "neokylin"
| DISTRO_NETBSD -> "netbsd"
| DISTRO_OPENBSD -> "openbsd"
diff --git a/daemon/inspect_types.mli b/daemon/inspect_types.mli
index 6d75e5825..78051b510 100644
--- a/daemon/inspect_types.mli
+++ b/daemon/inspect_types.mli
@@ -89,6 +89,7 @@ and distro =
| DISTRO_MAGEIA
| DISTRO_MANDRIVA
| DISTRO_MEEGO
+ | DISTRO_MSDOS
| DISTRO_NEOKYLIN
| DISTRO_NETBSD
| DISTRO_OPENBSD
diff --git a/generator/actions_inspection.ml b/generator/actions_inspection.ml
index d422b6088..0ac282435 100644
--- a/generator/actions_inspection.ml
+++ b/generator/actions_inspection.ml
@@ -226,6 +226,10 @@ Mandriva.
MeeGo.
+=item \"msdos\"
+
+Microsoft DOS.
+
=item \"neokylin\"
NeoKylin.
--
2.14.3
7 years, 1 month
