April 2013 - Libguestfs - libguestfs List Archives

[PATCH] Add support for SSH (Secure Shell) block device.
by Richard W.M. Jones 16 Apr '13

16 Apr '13

Note this patch requires a non-upstream qemu patch that I've been experimenting with. See qemu-devel list. Rich.

1 3

Internationalisation in guestconv
by Matthew Booth 15 Apr '13

15 Apr '13

I've added internationalisation to guestconv, and updated HACKING with a new section. Any thoughts/comments? Matt

4 4

[PATCH] use correct repos.d in zypp driver
by Olaf Hering 11 Apr '13

11 Apr '13

A temporary dir used during debugging was added in a previous patch. Signed-off-by: Olaf Hering <olaf(a)aepfle.de> --- src/supermin_zypp_rpm.ml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/supermin_zypp_rpm.ml b/src/supermin_zypp_rpm.ml index 4247d09..64cbb43 100644 --- a/src/supermin_zypp_rpm.ml +++ b/src/supermin_zypp_rpm.ml @@ -129,7 +129,7 @@ unset LANG ${!LC_*} zypper \ %s \ %s \ - --root %S --reposd-dir /dev/shm/supermin/zypp/repos.d \ + --root %S --reposd-dir /etc/zypp/repos.d \ --cache-dir %S \ --gpg-auto-import-keys \ --non-interactive \

2 1

[PATCH] add --with-supermin-extra-options configure option
by Olaf Hering 11 Apr '13

11 Apr '13

Useful to pass --use-installed and other options to supermin. Signed-off-by: Olaf Hering <olaf(a)aepfle.de> --- Not even compile tested!! Just copy&paste from SUPERMIN_PACKAGER_CONFIG appliance/make.sh.in | 3 +++ configure.ac | 13 +++++++++++++ 2 files changed, 16 insertions(+) diff --git a/appliance/make.sh.in b/appliance/make.sh.in index 42d6b23..8ba1fa4 100755 --- a/appliance/make.sh.in +++ b/appliance/make.sh.in @@ -33,6 +33,9 @@ exec 5<&- if [ "x@SUPERMIN_PACKAGER_CONFIG@" != "xno" ]; then extra="--yum-config @SUPERMIN_PACKAGER_CONFIG@" fi +if [ "x@SUPERMIN_EXTRA_OPTIONS@" != "xno" ]; then + extra="$extra @SUPERMIN_EXTRA_OPTIONS@" +fi echo @SUPERMIN@ -v -o supermin.d --names $(< packagelist ) $excludes $extra @SUPERMIN@ -v -o supermin.d --names $(< packagelist ) $excludes $extra diff --git a/configure.ac b/configure.ac index f94a411..87d517b 100644 --- a/configure.ac +++ b/configure.ac @@ -473,6 +473,19 @@ AC_ARG_WITH([febootstrap-packager-config], AC_SUBST([SUPERMIN_PACKAGER_CONFIG]) +dnl Pass additional supermin options. +dnl +SUPERMIN_EXTRA_OPTIONS=no +AC_MSG_CHECKING([for --with-supermin-extra-options option]) +AC_ARG_WITH([supermin-extra-options], + [AS_HELP_STRING([--with-supermin-extra-options=STRING], + [Pass additional supermin options. @<:@default=no@:>@])], + [SUPERMIN_EXTRA_OPTIONS="$withval" + AC_MSG_RESULT([$SUPERMIN_EXTRA_OPTIONS"])], + [AC_MSG_RESULT([not set])]) + +AC_SUBST([SUPERMIN_EXTRA_OPTIONS]) + if test "x$enable_appliance" = "xyes"; then test "x$SUPERMIN" = "xno" && AC_MSG_ERROR([supermin (formerly called febootstrap) must be installed])

2 1

[PATCH] remove leading tabs in zypp driver
by Olaf Hering 11 Apr '13

11 Apr '13

Signed-off-by: Olaf Hering <olaf(a)aepfle.de> --- src/supermin_zypp_rpm.ml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/src/supermin_zypp_rpm.ml b/src/supermin_zypp_rpm.ml index 1c6243d..4247d09 100644 --- a/src/supermin_zypp_rpm.ml +++ b/src/supermin_zypp_rpm.ml @@ -173,7 +173,7 @@ let rec zypp_rpm_list_files pkg = | [filename; flags; mode; size] -> let test_flag = String.contains flags in let mode = int_of_string mode in - let size = int_of_string size in + let size = int_of_string size in if test_flag 'd' then None (* ignore documentation *) else ( (* Skip unreadable files when called as non-root *) @@ -206,7 +206,7 @@ let rec zypp_rpm_list_files pkg = List.map (fun name -> name, { ft_dir = true; ft_ghost = false; ft_config = false; ft_mode = 0o40755; - ft_size = 0 }) dirs in + ft_size = 0 }) dirs in let devs = [ "/dev/null"; "/dev/full"; "/dev/zero"; "/dev/random"; "/dev/urandom"; "/dev/tty"; "/dev/console"; "/dev/ptmx"; "/dev/stdin"; "/dev/stdout"; "/dev/stderr" ] in @@ -215,7 +215,7 @@ let rec zypp_rpm_list_files pkg = List.map (fun name -> name, { ft_dir = false; ft_ghost = false; ft_config = false; ft_mode = 0o644; - ft_size = 0 }) devs in + ft_size = 0 }) devs in dirs @ devs @ files ) else files in

2 1

[PATCH] skip unreadable files in zypp_rpm_list_files
by Olaf Hering 11 Apr '13

11 Apr '13

Signed-off-by: Olaf Hering <olaf(a)aepfle.de> --- src/supermin_zypp_rpm.ml | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/src/supermin_zypp_rpm.ml b/src/supermin_zypp_rpm.ml index 720f50a..1c6243d 100644 --- a/src/supermin_zypp_rpm.ml +++ b/src/supermin_zypp_rpm.ml @@ -175,12 +175,18 @@ let rec zypp_rpm_list_files pkg = let mode = int_of_string mode in let size = int_of_string size in if test_flag 'd' then None (* ignore documentation *) - else + else ( + (* Skip unreadable files when called as non-root *) + if Unix.getuid() > 0 && + (try Unix.access filename [Unix.R_OK]; false with + Unix_error _ -> eprintf "supermin: EPERM %s\n%!" filename; true) then None + else Some (filename, { ft_dir = mode land 0o40000 <> 0; ft_ghost = test_flag 'g'; ft_config = test_flag 'c'; ft_mode = mode; ft_size = size; }) + ) | _ -> eprintf "supermin: bad output from rpm command: '%s'" line; exit 1

1 0

[PATCH] always work with empty --root in zypp driver
by Olaf Hering 11 Apr '13

11 Apr '13

The current mode of operation in the zypp driver is to download all packages into a temporary pkg-cache-dir. Installed packages in the root filesystem are ignored, otherwise zypper would download only a subset of packages, and it would not provide a list of all required dependencies. Remove the getuid check to force an empty root directory and use the configured system repositories. This works for root and non-root users. Signed-off-by: Olaf Hering <olaf(a)aepfle.de> --- src/supermin_zypp_rpm.ml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/supermin_zypp_rpm.ml b/src/supermin_zypp_rpm.ml index 51f01eb..2089837 100644 --- a/src/supermin_zypp_rpm.ml +++ b/src/supermin_zypp_rpm.ml @@ -50,7 +50,7 @@ pkg_cache_dir=%S time zypper \ %s \ %s \ - %s \ + --root %S --reposd-dir /etc/zypp/repos.d \ --cache-dir \"${cache_dir}\" \ --pkg-cache-dir \"${pkg_cache_dir}\" \ --gpg-auto-import-keys \ @@ -65,7 +65,7 @@ time zypper \ (if verbose then "--verbose --verbose" else "--quiet") (match packager_config with None -> "" | Some filename -> sprintf "--config %s" filename) - (if Unix.getuid() > 0 then sprintf "--root %S --reposd-dir /etc/zypp/repos.d" tmp_root else "") + tmp_root in run_shell sh names;

2 1

[PATCH 1/2] add run_shell helper
by Olaf Hering 10 Apr '13

10 Apr '13

The new run_shell helper is a copy of run_python, and will be used by upcoming changes. Signed-off-by: Olaf Hering <olaf(a)aepfle.de> --- src/supermin_utils.ml | 9 +++++++++ src/supermin_utils.mli | 5 +++++ 2 files changed, 14 insertions(+) diff --git a/src/supermin_utils.ml b/src/supermin_utils.ml index f98e09a..cb8a27e 100644 --- a/src/supermin_utils.ml +++ b/src/supermin_utils.ml @@ -70,6 +70,15 @@ let run_command cmd = exit 1 ) +let run_shell code args = + let cmd = sprintf "sh -c %s arg0 %s" + (Filename.quote code) + (String.concat " " (List.map Filename.quote args)) in + if Sys.command cmd <> 0 then ( + eprintf "supermin: external shell program failed, see earlier error messages\n"; + exit 1 + ) + let run_python code args = let cmd = sprintf "python -c %s %s" (Filename.quote code) diff --git a/src/supermin_utils.mli b/src/supermin_utils.mli index 54df1e8..c4a52c6 100644 --- a/src/supermin_utils.mli +++ b/src/supermin_utils.mli @@ -41,6 +41,11 @@ val run_command : string -> unit when constructing the command to properly quote any arguments (using {!Filename.quote}). *) +val run_shell : string -> string list -> unit + (** [run_shell code args] runs shell [code] with arguments [args]. + This does not return anything, but exits with an error message + if the shell code returns an error. *) + val run_python : string -> string list -> unit (** [run_python code args] runs Python [code] with arguments [args]. This does not return anything, but exits with an error message

2 2

Fwd: [Qemu-devel] debugging qemu-nbd.c
by Skippy VonDrake 10 Apr '13

10 Apr '13

---------- Forwarded message ---------- From: Skippy VonDrake <skippyvondrake(a)gmail.com> Date: Tue, Apr 9, 2013 at 11:38 AM Subject: Re: [Qemu-devel] debugging qemu-nbd.c To: "Richard W.M. Jones" <rjones(a)redhat.com> >> >> I have an app that forks a process with calls qemu-nbd with an offset >> >> into an image. >> >> The app then uses libguestfs to attach the nbd device. >> >> But it appears that qemu is failing. >> > >> > When you say "qemu is failing", do you mean qemu-nbd, or the qemu >> > subprocess that libguestfs uses? Does it print any error? >> >> I believe it is the qemu subprocess that libguestfs is using. >> I'm basing that assumption on the log file statements from both libguestfs >> and qemu, and from stderr from libguestfs. >> >> The only stderr from libguestfs is "child_cleanup:... child process died" > > Enable debugging in libguestfs to find out what's going on: > > http://libguestfs.org/guestfs-faq.1.html#debugging-libguestfs With both LIBGUESTFS_DEBUG and LIBGUESTFS_TRACE set and guestfs_set_verbose(g,1) and guestfs_set_trace(g,1). Below is the output. Some messages may look unfamiliar because I placed some statements in libguestfs just for a 'crude' tracing into the library. The initial 'nbd_client_thread' message is one I placed in the qemu sources. executing 'system(sudo rmmod nbd) executing 'system(sudo modprobe nbd max_part=16) setupNbdServer - size: 77824 setupNbdServer - system(serverCmd) string: qemu-nbd -v -n -o 77824 -t -p 3000 -c /dev/nbd0 /home/skippy/mnt/mnt0/win7x64amdHP.vdi nbd_client_thread:line 260::12806|939447:: qemu-NBD device /dev/nbd0 is NOWWW connected to /home/skippy/mnt/mnt0/win7x64amdHP.vdi libguestfs: trace: add_drive "" "format:raw" "protocol:nbd" "server:localhost:3000" libguestfs: trace: add_drive = 0 guestfs is launching the appliance libguestfs: trace: launch libguestfs: trace: get_tmpdir libguestfs: trace: get_tmpdir = "/tmp" libguestfs: launch: attach-method=appliance libguestfs: launch: tmpdir=/tmp/libguestfsscRysl libguestfs: launch: umask=0022 libguestfs: launch: euid=0 libguestfs: [00000ms] begin testing qemu features libguestfs: command: run: /home/skippy/Downloads/qemu/latest-build/install_dir/bin/qemu-system-x86_64 libguestfs: command: run: \ -nographic libguestfs: command: run: \ -help libguestfs: command: run: /home/skippy/Downloads/qemu/latest-build/install_dir/bin/qemu-system-x86_64 libguestfs: command: run: \ -nographic libguestfs: command: run: \ -version libguestfs: qemu version 1.4 libguestfs: command: run: /home/skippy/Downloads/qemu/latest-build/install_dir/bin/qemu-system-x86_64 libguestfs: command: run: \ -nographic libguestfs: command: run: \ -machine accel=kvm:tcg libguestfs: command: run: \ -device ? libguestfs: [00025ms] finished testing qemu features libguestfs: [00025ms] r from fork: 13658 \n \ libguestfs: error: appliance closed the connection unexpectedly, see earlier error messages libguestfs: child_cleanup: 0x11f4ba0: child process died libguestfs: sending SIGTERM to process 13658 libguestfs: error: /home/skippy/Downloads/qemu/latest-build/install_dir/bin/qemu-system-x86_64 exited with error status 1, see debug messages above libguestfs: error: guestfs_launch failed, see earlier error messages libguestfs: trace: launch = -1 (error) ERROR from setupGuestfs-> guestfs_launch, errnum: 0 Last error msg: guestfs_launch failed, see earlier error messages main thread exiting libguestfs: trace: close libguestfs: closing guestfs handle 0x11f4ba0 (state 0) libguestfs: command: run: rm libguestfs: command: run: \ -rf /tmp/libguestfsscRysl As you can see, the output looks similar to my previous post. At which you suggested that qemu might be failing. So that is when I started trying to pinpoint the problem in qemu. And the 'chdir' command was the final point at which my file-log statements ended. For that logging I'd open the log, write a string and close the log each time. One bit that might be of interest - my app does a fork/system command to launch the ndb server. When I give the disconnect command (after the error output above), I see this displayed from qemu: $ nbd.c:nbd_trip():L1106: From: 18446744073709551104, Len: 0, Size: 20003880960, Offset: 77824 nbd.c:nbd_trip():L1107: requested operation past EOF--bad client? The image is 20GB with a VDI header prepended to it. The size of the header is displayed with "setupNbdServer - size: 77824" and used for the offset parameter for qemu-nbd. I don't know what to make of the "From" value nor the 'Len' of zero in the last output.

2 3

Re: [Libguestfs] [Qemu-devel] debugging qemu-nbd.c
by Richard W.M. Jones 09 Apr '13

09 Apr '13

On Tue, Apr 09, 2013 at 10:41:47AM -0400, Skippy VonDrake wrote: > On Tue, Apr 9, 2013 at 10:33 AM, Richard W.M. Jones <rjones(a)redhat.com> wrote: > > On Tue, Apr 09, 2013 at 10:18:50AM -0400, Skippy VonDrake wrote: > >> Hello, > >> > >> Apologies upfront if posting to this list is inappropriate. > >> > >> I have an app that forks a process with calls qemu-nbd with an offset > >> into an image. > >> The app then uses libguestfs to attach the nbd device. > >> But it appears that qemu is failing. > > > > When you say "qemu is failing", do you mean qemu-nbd, or the qemu > > subprocess that libguestfs uses? Does it print any error? > > I believe it is the qemu subprocess that libguestfs is using. > I'm basing that assumption on the log file statements from both libguestfs > and qemu, and from stderr from libguestfs. > > The only stderr from libguestfs is "child_cleanup:... child process died" Enable debugging in libguestfs to find out what's going on: http://libguestfs.org/guestfs-faq.1.html#debugging-libguestfs > >> I'm using the latest qemu sources and placed file log statements within > >> qemu-nbd.c to pinpoint the problem. > >> > >> The statement: " if (chdir("/") < 0)" > >> appears to be the culprit. > >> > >> But the subsequent err() call is never reached. If I pull 'chdir' out of > >> the 'IF' and assign a local var to its return - execution still seems to > >> stop on the 'chdir'. So there is no opportunity to examine its return. > >> > >> What is the best approach to debugging this section of code? > > > > As a first step I would just add 'fprintf (stderr, ...)' straight into > > the qemu-nbd.c file. Or run it under gdb. > > I haven't tried this with gdb yet but the log statements I described > above were in the qemu-nbd.c file. And are successfully written > up until the chdir command. > > I'll try attaching gdb... Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming blog: http://rwmj.wordpress.com Fedora now supports 80 OCaml packages (the OPEN alternative to F#)

1 0