[PATCH v2 0/2] change the appliance search behavior
In continuation of the topic "appliance: reorder the steps to search for appliance" ( https://www.redhat.com/archives/libguestfs/2017-April/msg00233.html ) Pavel Butsykin (2): appliance: search all types of appliances for each path separately build: add ./configure --with-guestfs-path option lib/Makefile.am | 1 - lib/appliance.c | 227 ++++++++++++++++++++++-------------------------- m4/guestfs_appliance.m4 | 11 +++ 3 files changed, 113 insertions(+), 126 deletions(-) -- 2.11.0
This patch changes appliance search using paths with multiple directories. Now all appliance checks will be done separately for each directory. For example if the path LIBGUESTFS_PATH=/a:/b:/c, then all applainces are searched first in /a, then in /b and then in /c. It allows to flexibly configure the libguestfs to interact with different appliances. Signed-off-by: Pavel Butsykin <pbutsykin@virtuozzo.com> --- lib/appliance.c | 227 +++++++++++++++++++++++++------------------------------- 1 file changed, 102 insertions(+), 125 deletions(-) diff --git a/lib/appliance.c b/lib/appliance.c index 5738e4e7f..66f91cb0a 100644 --- a/lib/appliance.c +++ b/lib/appliance.c @@ -37,18 +37,23 @@ #include "guestfs.h" #include "guestfs-internal.h" +struct appliance_files { + char *kernel; + char *initrd; + char *image; +}; + /* Old-style appliance is going to be obsoleted. */ static const char kernel_name[] = "vmlinuz." host_cpu; static const char initrd_name[] = "initramfs." host_cpu ".img"; -static int build_appliance (guestfs_h *g, char **kernel, char **initrd, char **appliance); -static int find_path (guestfs_h *g, int (*pred) (guestfs_h *g, const char *pelem, void *data), void *data, char **pelem); +static int search_appliance (guestfs_h *g, struct appliance_files *appliance); static int dir_contains_file (guestfs_h *g, const char *dir, const char *file); static int dir_contains_files (guestfs_h *g, const char *dir, ...); static int contains_old_style_appliance (guestfs_h *g, const char *path, void *data); static int contains_fixed_appliance (guestfs_h *g, const char *path, void *data); static int contains_supermin_appliance (guestfs_h *g, const char *path, void *data); -static int build_supermin_appliance (guestfs_h *g, const char *supermin_path, char **kernel, char **initrd, char **appliance); +static int build_supermin_appliance (guestfs_h *g, const char *supermin_path, struct appliance_files *appliance); static int run_supermin_build (guestfs_h *g, const char *lockfile, const char *appliancedir, const char *supermin_path); /** @@ -59,12 +64,12 @@ static int run_supermin_build (guestfs_h *g, const char *lockfile, const char *a * appliances, or using either a fixed or old-style appliance. * * The return value is C<0> = good, C<-1> = error. Returned in - * C<*kernel> will be the name of the kernel to use, C<*initrd> the - * name of the initrd, C<*appliance> the name of the ext2 root - * filesystem. C<*appliance> can be C<NULL>, meaning that we are - * using an old-style (non-ext2) appliance. All three strings must be - * freed by the caller. However the referenced files themselves must - * I<not> be deleted. + * C<appliance.kernel> will be the name of the kernel to use, + * C<appliance.initrd> the name of the initrd, C<appliance.image> the name of + * the ext2 root filesystem. C<appliance.image> can be C<NULL>, meaning that + * we are using an old-style (non-ext2) appliance. All three strings must be + * freed by the caller. However the referenced files themselves must I<not> be + * deleted. * * The process is as follows: * @@ -72,9 +77,8 @@ static int run_supermin_build (guestfs_h *g, const char *lockfile, const char *a * * =item 1. * - * Look for the first element of C<g-E<gt>path> which contains a - * supermin appliance skeleton. If no element has this, skip straight - * to step 3. + * Look in C<path> which contains a supermin appliance skeleton. If no element + * has this, skip straight to step 3. * * =item 2. * @@ -83,13 +87,12 @@ static int run_supermin_build (guestfs_h *g, const char *lockfile, const char *a * * =item 3. * - * Check each element of C<g-E<gt>path>, looking for a fixed - * appliance. If one is found, return it. + * Check C<path>, looking for a fixed appliance. If one is found, return it. * * =item 4. * - * Check each element of C<g-E<gt>path>, looking for an old-style - * appliance. If one is found, return it. + * Check C<path>, looking for an old-style appliance. If one is found, + * return it. * * =back * @@ -112,74 +115,108 @@ guestfs_int_build_appliance (guestfs_h *g, char **initrd_rtn, char **appliance_rtn) { - char *kernel = NULL, *initrd = NULL, *appliance = NULL; - if (build_appliance (g, &kernel, &initrd, &appliance) == -1) + struct appliance_files appliance; + + if (search_appliance (g, &appliance) != 1) return -1; /* Don't assign these until we know we're going to succeed, to avoid * the caller double-freeing (RHBZ#983218). */ - *kernel_rtn = kernel; - *initrd_rtn = initrd; - *appliance_rtn = appliance; + *kernel_rtn = appliance.kernel; + *initrd_rtn = appliance.initrd; + *appliance_rtn = appliance.image; return 0; } +/** + * Check C<path>, looking for one of appliances: supermin appliance, + * fixed appliance or old-style appliance. If one of the fixed appliances is + * found, return it. If the supermin appliance skeleton is found, build and + * return appliance. + * + * Return values: + * + * 1 = appliance is found, returns C<appliance>, + * 0 = appliance not found, + * -1 = error which aborts the launch process. + */ static int -build_appliance (guestfs_h *g, - char **kernel, - char **initrd, - char **appliance) +locate_or_build_appliance (guestfs_h *g, + struct appliance_files *appliance, + const char *path) { int r; - CLEANUP_FREE char *supermin_path = NULL; - CLEANUP_FREE char *path = NULL; /* Step (1). */ - r = find_path (g, contains_supermin_appliance, NULL, &supermin_path); - if (r == -1) - return -1; - - if (r == 1) + r = contains_supermin_appliance(g, path, NULL); + if (r == 1) { /* Step (2): build supermin appliance. */ - return build_supermin_appliance (g, supermin_path, - kernel, initrd, appliance); + r = build_supermin_appliance (g, path, appliance); + return r < 0 ? r : 1; + } /* Step (3). */ - r = find_path (g, contains_fixed_appliance, NULL, &path); - if (r == -1) - return -1; - + r = contains_fixed_appliance (g, path, NULL); if (r == 1) { - const size_t len = strlen (path); - *kernel = safe_malloc (g, len + 6 /* "kernel" */ + 2); - *initrd = safe_malloc (g, len + 6 /* "initrd" */ + 2); - *appliance = safe_malloc (g, len + 4 /* "root" */ + 2); - sprintf (*kernel, "%s/kernel", path); - sprintf (*initrd, "%s/initrd", path); - sprintf (*appliance, "%s/root", path); - return 0; + appliance->kernel = safe_asprintf (g, "%s/kernel", path); + appliance->initrd = safe_asprintf (g, "%s/initrd", path); + appliance->image = safe_asprintf(g, "%s/root", path); + return 1; } /* Step (4). */ - r = find_path (g, contains_old_style_appliance, NULL, &path); - if (r == -1) - return -1; - + r = contains_old_style_appliance (g, path, NULL); if (r == 1) { - const size_t len = strlen (path); - *kernel = safe_malloc (g, len + strlen (kernel_name) + 2); - *initrd = safe_malloc (g, len + strlen (initrd_name) + 2); - sprintf (*kernel, "%s/%s", path, kernel_name); - sprintf (*initrd, "%s/%s", path, initrd_name); - *appliance = NULL; - return 0; + appliance->kernel = safe_asprintf(g, "%s/%s", path, kernel_name); + appliance->initrd = safe_asprintf(g, "%s/%s", path, initrd_name); + appliance->image = NULL; + return 1; } + return 0; +} + + +/** + * Search elements of C<g-E<gt>path>, returning the first C<appliance> element + * which matches the predicate function C<locate_or_build_appliance>. + * + * Return values: + * + * 1 = a path element matched, returns C<appliance>, + * 0 = no path element matched, + * -1 = error which aborts the launch process. + */ +static int +search_appliance (guestfs_h *g, struct appliance_files *appliance) +{ + const char *pelem = g->path; + + /* Note that if g->path is an empty string, we want to check the + * current directory (for backwards compatibility with + * libguestfs < 1.5.4). + */ + do { + size_t len = strcspn (pelem, PATH_SEPARATOR); + /* Empty element or "." means current directory. */ + CLEANUP_FREE char *path = (len == 0) ? safe_strdup (g, ".") : + safe_strndup (g, pelem, len); + int r = locate_or_build_appliance (g, appliance, path); + if (r != 0) + return r; /* error or predicate matched */ + + if (pelem[len] == PATH_SEPARATOR[0]) + pelem += len + 1; + else + pelem += len; + } while (*pelem); + + /* Predicate didn't match on any path element. */ error (g, _("cannot find any suitable libguestfs supermin, fixed or old-style appliance on LIBGUESTFS_PATH (search path: %s)"), g->path); - return -1; + return 0; } static int @@ -213,8 +250,7 @@ contains_supermin_appliance (guestfs_h *g, const char *path, void *data) static int build_supermin_appliance (guestfs_h *g, const char *supermin_path, - char **kernel, char **initrd, - char **appliance) + struct appliance_files *appliance) { CLEANUP_FREE char *cachedir = NULL, *lockfile = NULL, *appliancedir = NULL; @@ -236,13 +272,13 @@ build_supermin_appliance (guestfs_h *g, debug (g, "finished building supermin appliance"); /* Return the appliance filenames. */ - *kernel = safe_asprintf (g, "%s/kernel", appliancedir); - *initrd = safe_asprintf (g, "%s/initrd", appliancedir); - *appliance = safe_asprintf (g, "%s/root", appliancedir); + appliance->kernel = safe_asprintf (g, "%s/kernel", appliancedir); + appliance->initrd = safe_asprintf (g, "%s/initrd", appliancedir); + appliance->image = safe_asprintf (g, "%s/root", appliancedir); /* Touch the files so they don't get deleted (as they are in /var/tmp). */ - (void) utimes (*kernel, NULL); - (void) utimes (*initrd, NULL); + (void) utimes (appliance->kernel, NULL); + (void) utimes (appliance->initrd, NULL); /* Checking backend != "uml" is a big hack. UML encodes the mtime * of the original backing file (in this case, the appliance) in the @@ -257,7 +293,7 @@ build_supermin_appliance (guestfs_h *g, * XXX */ if (STRNEQ (g->backend, "uml")) - (void) utimes (*appliance, NULL); + (void) utimes (appliance->image, NULL); return 0; } @@ -317,65 +353,6 @@ run_supermin_build (guestfs_h *g, } /** - * Search elements of C<g-E<gt>path>, returning the first path element - * which matches the predicate function C<pred>. - * - * Function C<pred> must return a true or false value. If it returns - * C<-1> then the entire search is aborted. - * - * Return values: - * - * 1 = a path element matched, it is returned in *pelem_ret and must be - * freed by the caller, - * 0 = no path element matched, *pelem_ret is set to NULL, or - * -1 = error which aborts the launch process - */ -static int -find_path (guestfs_h *g, - int (*pred) (guestfs_h *g, const char *pelem, void *data), - void *data, - char **pelem_ret) -{ - size_t len; - int r; - const char *pelem = g->path; - - /* Note that if g->path is an empty string, we want to check the - * current directory (for backwards compatibility with - * libguestfs < 1.5.4). - */ - do { - len = strcspn (pelem, PATH_SEPARATOR); - - /* Empty element or "." means current directory. */ - if (len == 0) - *pelem_ret = safe_strdup (g, "."); - else - *pelem_ret = safe_strndup (g, pelem, len); - - r = pred (g, *pelem_ret, data); - if (r == -1) { - free (*pelem_ret); - return -1; - } - - if (r != 0) /* predicate matched */ - return 1; - - free (*pelem_ret); - - if (pelem[len] == PATH_SEPARATOR[0]) - pelem += len + 1; - else - pelem += len; - } while (*pelem); - - /* Predicate didn't match on any path element. */ - *pelem_ret = NULL; - return 0; -} - -/** * Returns true iff C<file> is contained in C<dir>. */ static int -- 2.11.0
Looks good. I'll push this if it passes 'make check && make check-valgrind' which I'm currently running. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW
On 05.05.2017 12:27, Richard W.M. Jones wrote:
Looks good. I'll push this if it passes 'make check && make check-valgrind' which I'm currently running.
Thanks. It is not connected with the current patch-set, but I noticed a possible bug in appliance.c: static int dir_contains_files (guestfs_h *g, const char *dir, ...) { va_list args; const char *file; va_start (args, dir); while ((file = va_arg (args, const char *)) != NULL) { if (!dir_contains_file (g, dir, file)) { ... static int contains_fixed_appliance (guestfs_h *g, const char *path, void *data) { return dir_contains_files (g, path, "README.fixed", "kernel", "initrd", "root", NULL); } Passing NULL as the last argument to dir_contains_file() can leads to undefined behavior in accordance with C99. 7.15.1.1 If there is no actual next argument, or if type is not compatible with the type of the actual next argument (as promoted according to the default argument promotions), the behavior is undefined, except for the following cases: — one type is a signed integer type, the other type is the corresponding unsigned integer type, and the value is representable in both types; — one type is pointer to void and the other is a pointer to a character type There NULL is macros which can be defined as 0 or (void*)0, again in accordance with c99: 7.17 Common definitions .. The macros are NULL which expands to an implementation-defined null pointer constant; 6.3.2.3 Pointers ... 3 An integer constant expression with the value 0, or such an expression cast to type void *, is called a null pointer constant. If a null pointer constant is converted to a pointer type, the resulting pointer, called a null pointer, is guaranteed to compare unequal to a pointer to any object or function. In practice, this means that if NULL is defined as integer and we have 64 bit architecture, then here as the last argument - 4 bytes are written on the stack: dir_contains_files (g, path, "README.fixed", "kernel", "initrd", "root", NULL); But va_arg() will read 8 bytes: while ((file = va_arg (args, const char *)) != NULL) { I don't know how real can be the conditions to reproduce this, I mean the standard header where #define NULL (0), but as an extra precaution I can offer a patch (attached).
Rich.
Eric, what do you think of Pavel's analysis and/or suggested fix below? It seems all too plausible to me unfortunately :-( On Fri, May 05, 2017 at 03:46:45PM +0300, Pavel Butsykin wrote:
On 05.05.2017 12:27, Richard W.M. Jones wrote:
Looks good. I'll push this if it passes 'make check && make check-valgrind' which I'm currently running.
Thanks.
It is not connected with the current patch-set, but I noticed a possible bug in appliance.c:
static int dir_contains_files (guestfs_h *g, const char *dir, ...) { va_list args; const char *file;
va_start (args, dir); while ((file = va_arg (args, const char *)) != NULL) { if (!dir_contains_file (g, dir, file)) { ...
static int contains_fixed_appliance (guestfs_h *g, const char *path, void *data) { return dir_contains_files (g, path, "README.fixed", "kernel", "initrd", "root", NULL); }
Passing NULL as the last argument to dir_contains_file() can leads to undefined behavior in accordance with C99. 7.15.1.1 If there is no actual next argument, or if type is not compatible with the type of the actual next argument (as promoted according to the default argument promotions), the behavior is undefined, except for the following cases: — one type is a signed integer type, the other type is the corresponding unsigned integer type, and the value is representable in both types; — one type is pointer to void and the other is a pointer to a character type
There NULL is macros which can be defined as 0 or (void*)0, again in accordance with c99:
7.17 Common definitions .. The macros are NULL which expands to an implementation-defined null pointer constant;
6.3.2.3 Pointers ... 3 An integer constant expression with the value 0, or such an expression cast to type void *, is called a null pointer constant. If a null pointer constant is converted to a pointer type, the resulting pointer, called a null pointer, is guaranteed to compare unequal to a pointer to any object or function.
In practice, this means that if NULL is defined as integer and we have 64 bit architecture, then here as the last argument - 4 bytes are written on the stack: dir_contains_files (g, path, "README.fixed", "kernel", "initrd", "root", NULL);
But va_arg() will read 8 bytes: while ((file = va_arg (args, const char *)) != NULL) {
I don't know how real can be the conditions to reproduce this, I mean the standard header where #define NULL (0), but as an extra precaution I can offer a patch (attached).
Rich.
From 44164bb3ac96ecf92892bcb25aba85cf38f5769f Mon Sep 17 00:00:00 2001 From: Pavel Butsykin <pbutsykin@virtuozzo.com> Date: Fri, 5 May 2017 15:38:19 +0300 Subject: [PATCH] applaince: iterable args should be compatible with type used by va_arg()
Signed-off-by: Pavel Butsykin <pbutsykin@virtuozzo.com> --- lib/appliance.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/lib/appliance.c b/lib/appliance.c index 66f91cb0a..42591e7d7 100644 --- a/lib/appliance.c +++ b/lib/appliance.c @@ -222,7 +222,7 @@ search_appliance (guestfs_h *g, struct appliance_files *appliance) static int contains_old_style_appliance (guestfs_h *g, const char *path, void *data) { - return dir_contains_files (g, path, kernel_name, initrd_name, NULL); + return dir_contains_files (g, path, kernel_name, initrd_name, (void *) NULL); }
static int @@ -230,7 +230,7 @@ contains_fixed_appliance (guestfs_h *g, const char *path, void *data) { return dir_contains_files (g, path, "README.fixed", - "kernel", "initrd", "root", NULL); + "kernel", "initrd", "root", (void *) NULL); }
static int @@ -238,7 +238,7 @@ contains_supermin_appliance (guestfs_h *g, const char *path, void *data) { return dir_contains_files (g, path, "supermin.d/base.tar.gz", - "supermin.d/packages", NULL); + "supermin.d/packages", (void *) NULL); }
/** -- 2.11.0
Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-df lists disk usage of guests without needing to install any software inside the virtual machine. Supports Linux and Windows. http://people.redhat.com/~rjones/virt-df/
On 05/05/2017 12:07 PM, Richard W.M. Jones wrote:
Eric, what do you think of Pavel's analysis and/or suggested fix below? It seems all too plausible to me unfortunately :-(
There NULL is macros which can be defined as 0 or (void*)0, again in accordance with c99:
C99 permits: #define NULL 0 but POSIX does not. POSIX _requires_ #define NULL ((void*)0) or the equivalent, so that NULL is properly typed as a pointer to void in ALL cases (rather than the weaker C99 solution of letting the integer zero constant leak), in part _because_ of the commonality of passing NULL through varargs functions.
In practice, this means that if NULL is defined as integer and we have 64 bit architecture, then here as the last argument - 4 bytes are written on the stack: dir_contains_files (g, path, "README.fixed", "kernel", "initrd", "root", NULL);
But va_arg() will read 8 bytes: while ((file = va_arg (args, const char *)) != NULL) {
I don't know how real can be the conditions to reproduce this, I mean the standard header where #define NULL (0), but as an extra precaution I can offer a patch (attached).
Not worth worrying about in practice. All modern libc define NULL according to POSIX rules, rather than the weaker C99 rules. And gnulib guarantees the stronger definition of NULL. -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
On 05/05/2017 12:18 PM, Eric Blake wrote:
On 05/05/2017 12:07 PM, Richard W.M. Jones wrote:
Eric, what do you think of Pavel's analysis and/or suggested fix below? It seems all too plausible to me unfortunately :-(
There NULL is macros which can be defined as 0 or (void*)0, again in accordance with c99:
C99 permits: #define NULL 0
but POSIX does not. POSIX _requires_
#define NULL ((void*)0)
or the equivalent, so that NULL is properly typed as a pointer to void in ALL cases (rather than the weaker C99 solution of letting the integer zero constant leak), in part _because_ of the commonality of passing NULL through varargs functions.
I forgot to cite my reference: http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/stddef.h.html#tag_1... "NULL Null pointer constant. [CX] [Option Start] The macro shall expand to an integer constant expression with the value 0 cast to type void *. [Option End]" ... "RATIONALE The ISO C standard does not require the NULL macro to include the cast to type void * and specifies that the NULL macro be implementation-defined. POSIX.1-2008 requires the cast and therefore need not be implementation-defined." where the CX option represents requirements that POSIX makes above-and-beyond bare C99. -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
On 05/05/2017 12:18 PM, Eric Blake wrote:
On 05/05/2017 12:07 PM, Richard W.M. Jones wrote:
Eric, what do you think of Pavel's analysis and/or suggested fix below? It seems all too plausible to me unfortunately :-(
There NULL is macros which can be defined as 0 or (void*)0, again in accordance with c99: C99 permits: #define NULL 0
but POSIX does not. POSIX _requires_
#define NULL ((void*)0)
or the equivalent, so that NULL is properly typed as a pointer to void in ALL cases (rather than the weaker C99 solution of letting the integer zero constant leak), in part _because_ of the commonality of passing NULL through varargs functions. I forgot to cite my reference:
http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/stddef.h.html#tag_1...
"NULL Null pointer constant. [CX] [Option Start] The macro shall expand to an integer constant expression with the value 0 cast to type void *. [Option End]" ... "RATIONALE
The ISO C standard does not require the NULL macro to include the cast to type void * and specifies that the NULL macro be implementation-defined. POSIX.1-2008 requires the cast and therefore need not be implementation-defined."
where the CX option represents requirements that POSIX makes above-and-beyond bare C99.
Excellent, thanks for the clarification.
Signed-off-by: Pavel Butsykin <pbutsykin@virtuozzo.com> --- lib/Makefile.am | 1 - m4/guestfs_appliance.m4 | 11 +++++++++++ 2 files changed, 11 insertions(+), 1 deletion(-) diff --git a/lib/Makefile.am b/lib/Makefile.am index 063706f8f..18b6dd678 100644 --- a/lib/Makefile.am +++ b/lib/Makefile.am @@ -130,7 +130,6 @@ libguestfs_la_SOURCES = \ libguestfs.syms libguestfs_la_CPPFLAGS = \ - -DGUESTFS_DEFAULT_PATH='"$(libdir)/guestfs"' \ -DGUESTFS_WARN_DEPRECATED=1 \ -DGUESTFS_PRIVATE=1 \ -DLIBOSINFO_DB_PATH='"$(datadir)/libosinfo/db"' \ diff --git a/m4/guestfs_appliance.m4 b/m4/guestfs_appliance.m4 index 890b1999c..81c43879f 100644 --- a/m4/guestfs_appliance.m4 +++ b/m4/guestfs_appliance.m4 @@ -128,3 +128,14 @@ AC_ARG_WITH([extra-packages], [EXTRA_PACKAGES="$withval"], [EXTRA_PACKAGES=]) AC_SUBST([EXTRA_PACKAGES]) + +dnl Set guestfs default path +AC_ARG_WITH([guestfs-path], + [AS_HELP_STRING([--with-guestfs-path=PATH], + [specify guestfs path (default=$libdir/guestfs)])], + [GUESTFS_DEFAULT_PATH="$withval"], + [GUESTFS_DEFAULT_PATH="${acl_final_exec_prefix}/${acl_libdirstem}/guestfs"]) +AC_SUBST([GUESTFS_DEFAULT_PATH]) + +AC_DEFINE_UNQUOTED([GUESTFS_DEFAULT_PATH], ["$GUESTFS_DEFAULT_PATH"], + [Define guestfs default path.]) -- 2.11.0
participants (3)
-
Eric Blake -
Pavel Butsykin -
Richard W.M. Jones