August 2009 - Libguestfs - libguestfs List Archives

[PATCH libguestfs] generator.ml: do not emit unused print_*_list functions
by Jim Meyering 18 Aug '09

18 Aug '09

I see the following warnings: cmds.c:934:13: warning: 'print_int_bool_list' defined but not used cmds.c:1093:13: warning: 'print_stat_list' defined but not used cmds.c:1124:13: warning: 'print_statvfs_list' defined but not used cmds.c:1171:13: warning: 'print_version_list' defined but not used This patch makes it so we don't emit the unused definitions: >From ff307d5e14b33fbbf97c411a97905127021b09fc Mon Sep 17 00:00:00 2001 From: Jim Meyering <meyering(a)redhat.com> Date: Mon, 17 Aug 2009 20:28:34 +0200 Subject: [PATCH libguestfs] generator.ml: do not emit unused print_*_list functions * src/generator.ml (emit_print_list_function): New function. Emit a function definition only if it will be used. --- src/generator.ml | 36 ++++++++++++++++++++++++------------ 1 files changed, 24 insertions(+), 12 deletions(-) diff --git a/src/generator.ml b/src/generator.ml index ac01a98..03dc52b 100755 --- a/src/generator.ml +++ b/src/generator.ml @@ -6063,6 +6063,21 @@ and generate_fish_cmds () = pr "}\n"; pr "\n"; + let emit_print_list_function typ = + pr "static void print_%s_list (struct guestfs_%s_list *%ss)\n" + typ typ typ; + pr "{\n"; + pr " int i;\n"; + pr "\n"; + pr " for (i = 0; i < %ss->len; ++i) {\n" typ; + pr " printf (\"[%%d] = {\\n\", i);\n"; + pr " print_%s_indent (&%ss->val[i], \" \");\n" typ typ; + pr " printf (\"}\\n\");\n"; + pr " }\n"; + pr "}\n"; + pr "\n"; + in + (* print_* functions *) List.iter ( fun (typ, cols) -> @@ -6119,20 +6134,17 @@ and generate_fish_cmds () = pr " print_%s_indent (%s, \"\");\n" typ typ; pr "}\n"; pr "\n"; - pr "static void print_%s_list (struct guestfs_%s_list *%ss)\n" - typ typ typ; - pr "{\n"; - pr " int i;\n"; - pr "\n"; - pr " for (i = 0; i < %ss->len; ++i) {\n" typ; - pr " printf (\"[%%d] = {\\n\", i);\n"; - pr " print_%s_indent (&%ss->val[i], \" \");\n" typ typ; - pr " printf (\"}\\n\");\n"; - pr " }\n"; - pr "}\n"; - pr "\n"; ) structs; + (* Emit a put_TYPE_list function definition only if that function is used. *) + List.iter ( + function + | typ, (RStructListOnly | RStructAndList) -> + (* generate the function for typ *) + emit_print_list_function typ + | typ, _ -> () (* empty *) + ) rstructs_used; + (* run_<action> actions *) List.iter ( fun (name, style, _, flags, _, _, _) -> -- 1.6.4.378.g88f2f

2 2

total warning-removal for daemon/
by Jim Meyering 17 Aug '09

17 Aug '09

The warnings in daemon were aggravating and risky for development (too easy to miss new ones) so I spent some time last week and today working on removing them. The first patch gets us down to almost no warnings with the original -Wall setting. That was by far the hardest part. Once I'd done that, I enabled nearly all of gcc's warnings via gnulib's warnings and manywarnings modules and the code added to configure.ac in the final patch. I did it that way so that we can still bisect without compile failure, even when using the new --enable-gcc-warnings option. I.e., fix all warnings, *and then* add the code that turns on the new -W options along with -Werror. I suggest that developers always configure with --enable-gcc-warnings. The 2nd patch is self explanatory. 0003 adds some omitted "\n"-after ";" in emitted code. The remaining changes address new warnings exposed by using all of the new -W options. Among them, there were some real bugs. 0006 is a portability bug, at least. 0007 probably doesn't matter much in practice, but I looked because of a warning about ignored write return value. 0011 is definitely a bug. It was highlighted via the sign-vs-unsigned warning. 751 0001-adjust-const-pointers-to-avoid-warnings.patch 33 0002-python-avoid-_POSIX_C_SOURCE-redefinition-warning.patch 39 0003-generator.ml-emit-slightly-prettier-code.patch 50 0004-avoid-warning-about-old-style-no-param-function-defi.patch 39 0005-command.c-avoid-shadowing-a-global-function.patch 41 0006-guestfsd.c-don-t-perform-arithmetic-on-void-pointers.patch 42 0007-daemon-zero-don-t-ignore-write-and-close-errors.patch 41 0008-wc-blockdev-avoid-warnings-about-discarding-const-qu.patch 48 0009-sfdisk.c-fallocate.c-use-a-string-literal-as-format.patch 26 0010-daemon.h-avoid-warning-about-possible-noreturn-funct.patch 27 0011-guestfsd-don-t-ignore-failed-write-to-socket.patch 35 0012-xattr.c-avoid-warning-about-comparison-between-signe.patch 125 0013-daemon-enable-Werror-and-many-gcc-warnings-when-enab.patch $ diffstat 00* b/daemon/Makefile.am | 3 - b/daemon/augeas.c | 2 b/daemon/blockdev.c | 2 b/daemon/command.c | 12 ++-- b/daemon/configure.ac | 61 ++++++++++++++++++++++++ b/daemon/daemon.h | 6 +- b/daemon/debug.c | 2 b/daemon/df.c | 4 - b/daemon/fallocate.c | 2 b/daemon/guestfsd.c | 20 ++++---- b/daemon/lvm.c | 12 +--- b/daemon/m4/gnulib-cache.m4 | 5 +- b/daemon/proto.c | 3 - b/daemon/sfdisk.c | 10 ++-- b/daemon/sync.c | 3 - b/daemon/wc.c | 3 - b/daemon/xattr.c | 11 ++-- b/daemon/zero.c | 19 +++++-- b/fish/fish.c | 6 +- b/fish/fish.h | 6 +- b/src/generator.ml | 108 ++++++++++++++++++++++++-------------------- daemon/blockdev.c | 2 daemon/command.c | 9 +-- daemon/daemon.h | 3 - daemon/guestfsd.c | 7 +- daemon/sfdisk.c | 5 -- src/generator.ml | 12 ++-- 27 files changed, 206 insertions(+), 132 deletions(-)

2 30

FYI, no more merge commits
by Jim Meyering 16 Aug '09

16 Aug '09

I've just noticed that I pushed a useless merge commit last week. Not a big deal, but I prefer not to do that, so have installed my server-side hook that prevents pushing merge commits to master.

1 0

[PATCH libguestfs] build: avoid "make sytnax-check" failure
by Jim Meyering 15 Aug '09

15 Aug '09

FYI, just pushed: >From 322ff984a39d53422b772bfeb8f69e7c648da8c0 Mon Sep 17 00:00:00 2001 From: Jim Meyering <meyering(a)redhat.com> Date: Fri, 14 Aug 2009 21:01:48 +0200 Subject: [PATCH libguestfs] build: avoid "make sytnax-check" failure * daemon/configure.ac: Change a leading TAB to 8 spaces. --- daemon/configure.ac | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/daemon/configure.ac b/daemon/configure.ac index 62c28ee..b0c7f26 100644 --- a/daemon/configure.ac +++ b/daemon/configure.ac @@ -68,7 +68,7 @@ dnl Check for libselinux (optional). AC_CHECK_HEADERS([selinux/selinux.h]) AC_CHECK_LIB([selinux],[setexeccon],[ LIBS="-lselinux $LIBS" - have_libselinux="$ac_cv_header_selinux_selinux_h" + have_libselinux="$ac_cv_header_selinux_selinux_h" AC_CHECK_FUNCS([setcon getcon]) ],[have_libselinux=no]) if test "x$have_libselinux" = "xyes"; then -- 1.6.4.357.gfd68c

2 2

Code snippet to work out which RStruct/RStructList structs are used and how
by Richard W.M. Jones 14 Aug '09

14 Aug '09

-- Richard Jones, Emerging Technologies, Red Hat http://et.redhat.com/~rjones virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into Xen guests. http://et.redhat.com/~rjones/virt-p2v

2 4

ANNOUNCE: libguestfs 1.0.67 released
by Richard W.M. Jones 13 Aug '09

13 Aug '09

I'm pleased to announce the release of libguestfs 1.0.67. Libguestfs is a library for accessing and modifying virtual machine disk images. Home page: http://libguestfs.org/ Source: http://libguestfs.org/download/ Binaries: http://libguestfs.org/FAQ.html#binaries Fedora builds aren't ready yet because of the current Koji outage. (These release notes cover all the significant changes since the last announcement which was for 1.0.64, 3 weeks ago). New features - SELinux support, for guests that use it - inotify support - Allow swapon/swapoff from a swap file - New command: fallocate - New commands to make hard and symbolic links, readlink - New command: realpath - New commands to grep files - New command: file-architecture - 'file' command can now look in compressed files automatically Bugs fixed - Big rewrite of daemon code that uses device and path parameters (Jim Meyering) - Do malloc fuzzing during tests (Jim Meyering) - Fix case where grub /boot is not a separate filesystem (Matthew Booth) - Use grub to find kernels (Matthew Booth) - Can now access disk images which are located on a tmpfs - Tons of code fixes and cleanups (Jim Meyering) - Fix segfault in guestfish tab completion - Fix CD-ROM device recognition (Matthew Booth) - Uses gnulib (Jim Meyering) - Improve speed of tests by using squashfs disks for any static test data - Improve shell quoting in the daemon using custom formatters Rich. -- Richard Jones, Emerging Technologies, Red Hat http://et.redhat.com/~rjones libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://et.redhat.com/~rjones/libguestfs/ See what it can do: http://et.redhat.com/~rjones/libguestfs/recipes.html

1 0

selinux question and answer
by Richard W.M. Jones 13 Aug '09

13 Aug '09

This is continuing/summarising a rather long discussion that happened on IRC ... We talked to some SELinux experts about what was required to make SELinux work with libguestfs, and it seems reasonably simple to load the policy from the guest filesystem. All that needs to be done is to mount the guest disks up and then run: sh "/usr/sbin/load_policy -i" That command also mounts up <sysroot>/selinux, so that solves the other problem they raised. I wasn't completely sure how to test this was actually working. My best effort was to try to run some commands that would label files. This is using a fresh Fedora 11 install that has SELinux enforcing on it: guestfish -a /dev/mapper/vg_trick-F11x64 --ro \ run : mount /dev/vg_f11x64/lv_root / : \ sh "/usr/sbin/load_policy -i" : \ sh "ls -lZ /etc/passwd" -rw-r--r--. root root system_u:object_r:etc_t:s0 /etc/passwd guestfish -a /dev/mapper/vg_trick-F11x64 --ro \ run : mount /dev/vg_f11x64/lv_root / : \ sh "/usr/sbin/load_policy -i" : \ sh "chcon user_u:object_r:file_t /etc/passwd" : \ sh "ls -lZ /etc/passwd" -rw-r--r--. root root user_u:object_r:file_t /etc/passwd So it seems that relabelling files (using chcon) works. Whether this means everything will work, I don't know. You will also get a warning when guestfish exits at the moment: libguestfs: error: umount: /sysroot: umount: /sysroot: device is busy. (In some cases useful info about processes that use the device is found by lsof(8) or fuser(1)) This happens because the load_policy command is mounting /sysroot/selinux and thus preventing /sysroot from being unmounted during the automatic shutdown phase at the end. [Note: The attached patch is also required, because at the moment we are booting the kernel with selinux=0 for other reasons. This should be made configurable]. Rich. -- Richard Jones, Emerging Technologies, Red Hat http://et.redhat.com/~rjones virt-df lists disk usage of guests without needing to install any software inside the virtual machine. Supports Linux and Windows. http://et.redhat.com/~rjones/virt-df/

5 19

[PATCH libguestfs] tests: increase likelihood that heap abuse triggers failure
by Jim Meyering 13 Aug '09

13 Aug '09

This takes advantage of glibc's MALLOC_PERTURB_. If you develop on glibc-based systems and don't yet set this via your shell's init scripts, add something like this to e.g., .bashrc: export MALLOC_PERTURB_=$(($RANDOM % 255 + 1)) This change ensures that it's enabled for anyone running "make check" and in particular for koji. There's probably documentation somewhere, but I don't know where off hand, so this will have to do: http://udrepper.livejournal.com/11429.html >From e7ee94be7e929d8aa94e3e360d1d1ca3711dd01e Mon Sep 17 00:00:00 2001 From: Jim Meyering <meyering(a)redhat.com> Date: Thu, 13 Aug 2009 18:29:59 +0200 Subject: [PATCH libguestfs] tests: increase likelihood that heap abuse triggers failure * regressions/Makefile.am (TESTS_ENVIRONMENT): Always set MALLOC_PERTURB_ to a random value in 1..255. --- regressions/Makefile.am | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/regressions/Makefile.am b/regressions/Makefile.am index a51c76b..3279f95 100644 --- a/regressions/Makefile.am +++ b/regressions/Makefile.am @@ -40,7 +40,10 @@ SKIPPED_TESTS = \ FAILING_TESTS = \ test-qemudie-launchfail.sh +random_val := $(shell awk 'BEGIN{srand(); print 1+int(255*rand())}' < /dev/null) + TESTS_ENVIRONMENT = \ + MALLOC_PERTURB_=$(random_val) \ LD_LIBRARY_PATH=$(top_builddir)/src/.libs \ LIBGUESTFS_PATH=$(top_builddir)/appliance \ NOEXEC_CHECK="$(top_builddir)/src/.libs/libguestfs.so $(top_builddir)/daemon/guestfsd" -- 1.6.4.357.gfd68c

2 1

Summary of current test failures
by Richard W.M. Jones 13 Aug '09

13 Aug '09

CentOS 5.3 and RHEL 5.4 both fail on these 3 tests only: 82/212 test_zero_0 umount: /dev/sda1: umount: /dev/sda1: not found test_zero_0 FAILED 83/212 test_fsck_0 umount: /dev/sda1: umount: /dev/sda1: not found test_fsck_0 FAILED 84/212 test_fsck_1 umount: /dev/sda1: umount: /dev/sda1: not found test_fsck_1 FAILED Debian fails on 57 / 212 tests, but they all seem to be of the same type: 2/212 test_inotify_init_0 mount: /dev/sdd on /: mount: wrong fs type, bad option, bad superblock on /dev/sdd, missing codepage or helper program, or other error In some cases useful info is found in syslog - try dmesg | tail or so test_inotify_init_0 FAILED I'm still investigating the above. (I don't think any of them are related to my or Jim's patches - I think they have existed for some days). Rich. -- Richard Jones, Emerging Technologies, Red Hat http://et.redhat.com/~rjones virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into Xen guests. http://et.redhat.com/~rjones/virt-p2v

1 1

[PATCH] Enable device name translation for umount /dev
by Richard W.M. Jones 13 Aug '09

13 Aug '09

OK, I believe this is the cause of the 3 related failures on CentOS / RHEL. Calls to guestfs_umount ("/dev/...") didn't have the device path being rewritten. Rich. -- Richard Jones, Emerging Technologies, Red Hat http://et.redhat.com/~rjones New in Fedora 11: Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 70 libraries supprt'd http://fedoraproject.org/wiki/MinGW http://www.annexia.org/fedora_mingw

1 1