September 2009 - Libguestfs - libguestfs List Archives

[PATCH REBASED] Remove main loop
by Richard W.M. Jones 14 Sep '09

14 Sep '09

This is the only patch I currently have outstanding. No changes from the previous posting, except I rebased it against the head of git. Rich. -- Richard Jones, Emerging Technologies, Red Hat http://et.redhat.com/~rjones virt-df lists disk usage of guests without needing to install any software inside the virtual machine. Supports Linux and Windows. http://et.redhat.com/~rjones/virt-df/

2 12

[PATCH] Don't enable trace in set_trace test.
by Richard W.M. Jones 14 Sep '09

14 Sep '09

-- Richard Jones, Emerging Technologies, Red Hat http://et.redhat.com/~rjones virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into Xen guests. http://et.redhat.com/~rjones/virt-p2v

2 1

Fedora Virtualization Test Day (Thursday 2009-09-17)
by Richard W.M. Jones 14 Sep '09

14 Sep '09

Mark is organizing the Fedora Virtualization Test Day, which is this Thursday, 2009-09-17. Amongst the things that we'll be testing is libguestfs, and anyone who is interested in libguestfs and can contribute a little time is welcome to come along and help us. Join #fedora-test-day on FreeNode at any time, and ping someone (eg. me: 'rwmjones'). More details here: https://fedoraproject.org/wiki/Test_Day:2009-09-17_Virtualization I will add test plans to this page before the day: https://fedoraproject.org/wiki/Test_Day:2009-09-17_Virtualization_libguestfs Rich. -- Richard Jones, Emerging Technologies, Red Hat http://et.redhat.com/~rjones libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://et.redhat.com/~rjones/libguestfs/ See what it can do: http://et.redhat.com/~rjones/libguestfs/recipes.html

1 0

[PATCH] guestfish: Enable grouping in string lists
by Matthew Booth 14 Sep '09

14 Sep '09

This change adds the ability to group entries in a string list with single quotes. So the string: "'foo bar'" becomes 1 token rather than 2. Consequently single quotes must now be escaped: "\'" resolves to a literal single quote. Incidentally, this change also alters another, probably unintentional behaviour of the previous implementation, in that tokens are separated by any amount of whitespace rather than a single whitespace character. I.e.: "a b" resolves to: 'a' 'b' rather than: 'a' '' 'b' That last syntax can be used if an empty argument is still desired. Whitespace is now also defined to include tabs. parse_string_list can also now fail if it contains an unmatched open quote. --- fish/fish.c | 131 +++++++++++++++++++++++++++++++++++++++++++++-------- guestfish.pod | 6 ++- src/generator.ml | 3 +- 3 files changed, 118 insertions(+), 22 deletions(-) diff --git a/fish/fish.c b/fish/fish.c index a4069d6..86c0dfa 100644 --- a/fish/fish.c +++ b/fish/fish.c @@ -1082,30 +1082,121 @@ is_true (const char *str) strcasecmp (str, "no") != 0; } -/* XXX We could improve list parsing. */ char ** parse_string_list (const char *str) { - char **argv; - const char *p, *pend; - int argc, i; - - argc = 1; - for (i = 0; str[i]; ++i) - if (str[i] == ' ') argc++; - - argv = malloc (sizeof (char *) * (argc+1)); - if (argv == NULL) { perror ("malloc"); exit (1); } - - p = str; - i = 0; - while (*p) { - pend = strchrnul (p, ' '); - argv[i] = strndup (p, pend-p); - i++; - p = *pend == ' ' ? pend+1 : pend; + char **argv = NULL; + size_t argv_len = 0; + + /* Current position pointer */ + const char *p = str; + + /* Token might be simple: + * Token + * or be quoted: + * 'This is a single token' + * or contain embedded single-quoted sections: + * This' is a sing'l'e to'ken + * + * The latter may seem over-complicated, but it's what a normal shell does. + * Not doing it risks surprising somebody. + * + * This outer loop is over complete tokens. + */ + while(*p) { + char *tok = NULL; + size_t tok_len = 0; + + /* Skip leading whitespace */ + p += strspn (p, " \t"); + + char in_quote = 0; + + /* This loop is over token 'fragments'. A token can be in multiple bits if + * it contains single quotes. We also treat both sides of an escaped quote + * as separate fragments because we can't just copy it: we have to remove + * the \. + */ + while (*p && (!isblank (*p) || in_quote)) { + const char *end = p; + + /* Check if the fragment starts with a quote */ + if ('\'' == *p) { + /* Toggle in_quote */ + in_quote = !in_quote; + + /* Skip the quote */ + p++; end++; + } + + /* If we're in a quote, look for an end quote */ + if (in_quote) { + end += strcspn (end, "'"); + } + + /* Otherwise, look for whitespace or a quote */ + else { + end += strcspn (end, " \t'"); + } + + /* Grow the token to accommodate the fragment */ + size_t tok_end = tok_len; + tok_len += end - p; + tok = realloc (tok, tok_len + 1); + if (NULL == tok) { perror ("realloc"); exit (1); } + + /* Check if we stopped on an escaped quote */ + if ('\'' == *end && end != p && *(end-1) == '\\') { + /* Add everything before \' to the token */ + memcpy (&tok[tok_end], p, end - p - 1); + + /* Add the quote */ + tok[tok_len-1] = '\''; + + /* Already processed the quote */ + p = end + 1; + } + + else { + /* Add the whole fragment */ + memcpy (&tok[tok_end], p, end - p); + + p = end; + } + } + + /* We've reached the end of a token. We shouldn't still be in quotes. */ + if (in_quote) { + fprintf(stderr, _("Runaway quote in string \"%s\"\n"), str); + + /* Can't use free_strings here because argv isn't NULL terminated yet */ + for (size_t i = 0; i < argv_len; i++) { + free (argv[i]); + } + free(argv); + + return NULL; + } + + /* Add this token if there is one. There might not be if there was + * whitespace at the end of the input string */ + if(tok) { + /* Add the NULL terminator */ + tok[tok_len] = '\0'; + + /* Add the argument to the argument list */ + argv_len++; + argv = realloc(argv, sizeof(*argv) * argv_len); + if (NULL == argv) { perror ("realloc"); exit (1); } + argv[argv_len-1] = tok; + } } - argv[i] = NULL; + + /* NULL terminate the argument list */ + argv_len++; + argv = realloc(argv, sizeof(*argv) * argv_len); + if (NULL == argv) { perror ("realloc"); exit (1); } + argv[argv_len-1] = NULL; return argv; } diff --git a/guestfish.pod b/guestfish.pod index d24c162..affb83b 100644 --- a/guestfish.pod +++ b/guestfish.pod @@ -250,9 +250,13 @@ quotes. For example: rm '/"' A few commands require a list of strings to be passed. For these, use -a space-separated list, enclosed in quotes. For example: +a whitespace-separated list, enclosed in quotes. Strings containing whitespace +to be passed through must be enclosed in single quotes. A literal single quote +must be escaped with a backslash. vgcreate VG "/dev/sda1 /dev/sdb1" + command "/bin/echo 'foo bar'" + command "/bin/echo \'foo\'" =head1 WILDCARDS AND GLOBBING diff --git a/src/generator.ml b/src/generator.ml index db37228..9b2600f 100755 --- a/src/generator.ml +++ b/src/generator.ml @@ -6361,7 +6361,8 @@ and generate_fish_cmds () = pr " %s = strcmp (argv[%d], \"-\") != 0 ? argv[%d] : \"/dev/stdout\";\n" name i i | StringList name | DeviceList name -> - pr " %s = parse_string_list (argv[%d]);\n" name i + pr " %s = parse_string_list (argv[%d]);\n" name i; + pr " if (%s == NULL) return -1;\n" name; | Bool name -> pr " %s = is_true (argv[%d]) ? 1 : 0;\n" name i | Int name -> -- 1.6.2.5

2 4

[PATCH] guestfish: Redirect stdout when executing remote commands
by Matthew Booth 14 Sep '09

14 Sep '09

guestfish --listen necessarily redirects its stdout to /dev/null so as not to interfere with eval. The remote protocol doesn't contain any other provision for collecting stdout for the caller, so executing guestfish --remote will never generate any output. This patch fixes that by forwarding the caller's STDOUT to the listener over the unix socket connection. The listener redirects its STDOUT to the caller's STDOUT for the duration of the command, then closes it again. --- fish/rc.c | 134 +++++++++++++++++++++++++++++++++++++++++++++++++++++--- guestfish.pod | 7 --- 2 files changed, 126 insertions(+), 15 deletions(-) diff --git a/fish/rc.c b/fish/rc.c index 5423c22..5d64c70 100644 --- a/fish/rc.c +++ b/fish/rc.c @@ -27,6 +27,7 @@ #include <sys/types.h> #include <sys/un.h> #include <signal.h> +#include <sys/socket.h> #include <rpc/types.h> #include <rpc/xdr.h> @@ -49,6 +50,124 @@ create_sockpath (pid_t pid, char *sockpath, int len, struct sockaddr_un *addr) strcpy (addr->sun_path, sockpath); } +static const socklen_t controllen = CMSG_LEN (sizeof (int)); + +static void +receive_stdout (int s) +{ + static struct cmsghdr *cmptr = NULL, *h; + struct msghdr msg; + struct iovec iov[1]; + + /* Our 1 byte buffer */ + char buf[1]; + + if (NULL == cmptr) { + cmptr = malloc (controllen); + if (NULL == cmptr) { + perror ("malloc"); + exit (1); + } + } + + /* Don't specify a source */ + msg.msg_name = NULL; + msg.msg_namelen = 0; + + /* Initialise the msghdr to receive zero byte */ + iov[0].iov_base = buf; + iov[0].iov_len = 1; + msg.msg_iov = iov; + msg.msg_iovlen = 1; + + /* Initialise the control data */ + msg.msg_control = cmptr; + msg.msg_controllen = controllen; + + /* Read a message from the socket */ + ssize_t n = recvmsg (s, &msg, 0); + if (n < 0) { + perror ("recvmsg stdout fd"); + exit (1); + } + + h = CMSG_FIRSTHDR(&msg); + if (NULL == h) { + fprintf (stderr, "didn't receive a stdout file descriptor\n"); + } + + else { + /* Extract the transferred file descriptor from the control data */ + int fd = *(int *)CMSG_DATA (h); + + /* Duplicate the received file descriptor to stdout */ + dup2 (fd, STDOUT_FILENO); + close (fd); + } +} + +static void +send_stdout (int s) +{ + static struct cmsghdr *cmptr = NULL; + struct msghdr msg; + struct iovec iov[1]; + + /* Our 1 byte dummy buffer */ + char buf[1]; + + /* Don't specify a destination */ + msg.msg_name = NULL; + msg.msg_namelen = 0; + + /* Initialise the msghdr to send zero byte */ + iov[0].iov_base = buf; + iov[0].iov_len = 1; + msg.msg_iov = iov; + msg.msg_iovlen = 1; + + /* Initialize the zero byte */ + buf[0] = 0; + + /* Initialize the control data */ + if (NULL == cmptr) { + cmptr = malloc (controllen); + if (NULL == cmptr) { + perror ("malloc"); + exit (1); + } + } + cmptr->cmsg_level = SOL_SOCKET; + cmptr->cmsg_type = SCM_RIGHTS; + cmptr->cmsg_len = controllen; + + /* Add control header to the message */ + msg.msg_control = cmptr; + msg.msg_controllen = controllen; + + /* Add STDOUT to the control data */ + *(int *)CMSG_DATA (cmptr) = STDOUT_FILENO; + + if (sendmsg (s, &msg, 0) != 1) { + perror ("sendmsg stdout fd"); + exit (1); + } +} + +static void +close_stdout (void) +{ + int fd; + + fd = open ("/dev/null", O_WRONLY); + if (fd == -1) + perror ("/dev/null"); + else { + dup2 (fd, STDOUT_FILENO); + close (fd); + } +} + /* Remote control server. */ void rc_listen (void) @@ -56,7 +175,7 @@ rc_listen (void) char sockpath[128]; pid_t pid; struct sockaddr_un addr; - int sock, s, i, fd; + int sock, s, i; FILE *fp; XDR xdr, xdr2; guestfish_hello hello; @@ -111,13 +230,7 @@ rc_listen (void) /* Now close stdout and substitute /dev/null. This is necessary * so that eval `guestfish --listen` doesn't block forever. */ - fd = open ("/dev/null", O_WRONLY); - if (fd == -1) - perror ("/dev/null"); - else { - dup2 (fd, 1); - close (fd); - } + close_stdout(); /* Read commands and execute them. */ while (!quit) { @@ -125,6 +238,8 @@ rc_listen (void) if (s == -1) perror ("accept"); else { + receive_stdout(s); + fp = fdopen (s, "r+"); xdrstdio_create (&xdr, fp, XDR_DECODE); @@ -180,6 +295,7 @@ rc_listen (void) error: xdr_destroy (&xdr); /* NB. This doesn't close 'fp'. */ fclose (fp); /* Closes the underlying socket 's'. */ + close_stdout(); /* Re-close stdout */ } } @@ -227,6 +343,8 @@ rc_remote (int pid, const char *cmd, int argc, char *argv[], return -1; } + send_stdout(sock); + /* Send the greeting. */ fp = fdopen (sock, "r+"); xdrstdio_create (&xdr, fp, XDR_ENCODE); diff --git a/guestfish.pod b/guestfish.pod index d0d6839..affb83b 100644 --- a/guestfish.pod +++ b/guestfish.pod @@ -398,13 +398,6 @@ You can have several guestfish listener processes running using: guestfish --remote=$pid1 cmd guestfish --remote=$pid2 cmd -=head2 STANDARD OUTPUT DURING REMOTE CONTROL - -Because of limitations in the C<eval> statement, stdout from the -listener is currently redirected to C</dev/null>. - -Stderr is unchanged. - =head2 REMOTE CONTROL DETAILS Remote control happens over a Unix domain socket called -- 1.6.2.5

2 4

[PATCH] Add echo_daemon command
by Matthew Booth 14 Sep '09

14 Sep '09

echo_daemon is a simple echo which can be used to test connectivity between the client and daemon. --- daemon/Makefile.am | 1 + daemon/echo_daemon.c | 66 ++++++++++++++++++++++++++++++++++++++++++++++++++ po/POTFILES.in | 1 + src/MAX_PROC_NR | 2 +- src/generator.ml | 7 +++++ 5 files changed, 76 insertions(+), 1 deletions(-) create mode 100644 daemon/echo_daemon.c diff --git a/daemon/Makefile.am b/daemon/Makefile.am index 83ee408..ae74699 100644 --- a/daemon/Makefile.am +++ b/daemon/Makefile.am @@ -36,6 +36,7 @@ guestfsd_SOURCES = \ dmesg.c \ dropcaches.c \ du.c \ + echo_daemon.c \ ext2.c \ fallocate.c \ file.c \ diff --git a/daemon/echo_daemon.c b/daemon/echo_daemon.c new file mode 100644 index 0000000..dbede2f --- /dev/null +++ b/daemon/echo_daemon.c @@ -0,0 +1,66 @@ +/* libguestfs - the guestfsd daemon + * Copyright (C) 2009 Red Hat Inc. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. + */ + +#include <config.h> + +#include <string.h> + +#include "actions.h" + +char * +do_echo_daemon (char *const *argv) +{ + char *out = NULL; + size_t out_len = 0; + + /* Iterate over argv entries until reaching the NULL terminator */ + while (*argv) { + char add_space = 0; + + /* Store the end of current output */ + size_t out_end = out_len; + + /* Calculate the new output size */ + size_t arg_len = strlen(*argv); + out_len += arg_len; + + /* We will prepend a space if this isn't the first argument added */ + if (NULL != out) { + out_len++; + add_space = 1; + } + + /* Make the output buffer big enough for the string and its terminator */ + out = realloc (out, out_len + 1); + + /* Prepend a space if required */ + if (add_space) { + out[out_end++] = ' '; + } + + /* Copy the argument to the output */ + memcpy(&out[out_end], *argv, arg_len); + + argv++; + } + + /* NULL terminate the output */ + out[out_len] = '\0'; + + return out; +} diff --git a/po/POTFILES.in b/po/POTFILES.in index 1b2f82a..44e472b 100644 --- a/po/POTFILES.in +++ b/po/POTFILES.in @@ -12,6 +12,7 @@ daemon/dir.c daemon/dmesg.c daemon/dropcaches.c daemon/du.c +daemon/echo_daemon.c daemon/ext2.c daemon/fallocate.c daemon/file.c diff --git a/src/MAX_PROC_NR b/src/MAX_PROC_NR index 205a12b..6bb2f98 100644 --- a/src/MAX_PROC_NR +++ b/src/MAX_PROC_NR @@ -1 +1 @@ -194 +195 diff --git a/src/generator.ml b/src/generator.ml index 439c9c4..f820e1f 100755 --- a/src/generator.ml +++ b/src/generator.ml @@ -3595,6 +3595,13 @@ This loads a kernel module in the appliance. The kernel module must have been whitelisted when libguestfs was built (see C<appliance/kmod.whitelist.in> in the source)."); + ("echo_daemon", (RString "output", [StringList "words"]), 195, [], + [InitNone, Always, TestRun [["echo_daemon"; "\"This is a test\""]]], + "echo arguments back to the client", + "\ +This commands works much like the unix echo command. It returns its arguments as +a string."); + ] let all_functions = non_daemon_functions @ daemon_functions -- 1.6.2.5

2 11

[PATCH] Add diffutils package
by Richard W.M. Jones 14 Sep '09

14 Sep '09

Currently guestfs_equal is broken on Fedora 12. It turns out this is because /usr/bin/cmp isn't being included in the appliance, which is because diffutils is no longer an implicit dependency of something else. This patch makes diffutils an explicit package so we avoid this problem. Rich. -- Richard Jones, Emerging Technologies, Red Hat http://et.redhat.com/~rjones virt-df lists disk usage of guests without needing to install any software inside the virtual machine. Supports Linux and Windows. http://et.redhat.com/~rjones/virt-df/

2 1

[PATCH] Remove main loop
by Richard W.M. Jones 12 Sep '09

12 Sep '09

Please note this patch depends on these earlier patches: Add command trace functionality. https://www.redhat.com/archives/libguestfs/2009-September/msg00031.html Non-daemon actions indirect through generated code. https://www.redhat.com/archives/libguestfs/2009-September/msg00022.html Rich. -- Richard Jones, Emerging Technologies, Red Hat http://et.redhat.com/~rjones New in Fedora 11: Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 70 libraries supprt'd http://fedoraproject.org/wiki/MinGW http://www.annexia.org/fedora_mingw

1 0

[PATCH] .gitignore: Ignore a couple of auto-generate m4 scripts
by Matthew Booth 11 Sep '09

11 Sep '09

--- .gitignore | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/.gitignore b/.gitignore index a6caaf9..ae3b16d 100644 --- a/.gitignore +++ b/.gitignore @@ -45,6 +45,7 @@ daemon/install-sh daemon/missing daemon/names.c daemon/stubs.c +daemon/m4/stddef_h.m4 depcomp .deps df/virt-df.1 @@ -114,6 +115,7 @@ libtool *.lo localconfigure ltmain.sh +m4/gnulib-cache.m4 m4/intmax.m4 m4/libtool.m4 m4/lt~obsolete.m4 -- 1.6.2.5

2 2

[PATCH] generator.ml: Fix string list memory leak
by Matthew Booth 11 Sep '09

11 Sep '09

Parsed string lists are allocated by malloc, but were never freed. --- src/generator.ml | 16 +++++++++++++++- 1 files changed, 15 insertions(+), 1 deletions(-) diff --git a/src/generator.ml b/src/generator.ml index 7571f95..c72c329 100755 --- a/src/generator.ml +++ b/src/generator.ml @@ -6320,7 +6320,7 @@ and generate_fish_cmds () = | OptString n | FileIn n | FileOut n -> pr " const char *%s;\n" n - | StringList n | DeviceList n -> pr " char *const *%s;\n" n + | StringList n | DeviceList n -> pr " char **%s;\n" n | Bool n -> pr " int %s;\n" n | Int n -> pr " int %s;\n" n ) (snd style); @@ -6364,6 +6364,20 @@ and generate_fish_cmds () = generate_c_call_args ~handle:"g" style; pr ";\n"; + (* XXX: There must be a cleaner way to write this *) + iteri ( + fun i -> + function + | StringList name | DeviceList name -> + pr " char **%s_i = %s;\n" name name; + pr " while(*%s_i) {\n" name; + pr " free(*%s_i);\n" name; + pr " %s_i++;\n" name; + pr " }\n"; + pr " free(%s);\n" name; + | _ -> (); + ) (snd style); + (* Check return value for errors and display command results. *) (match fst style with | RErr -> pr " return r;\n" -- 1.6.2.5

2 4