August 2012 - Libguestfs - libguestfs List Archives

Can't locate object method "html_charset" via package "Podwrapper::XHTML"
by Richard W.M. Jones 28 Aug '12

28 Aug '12

[Another one reported on IRC today ...] http://paste.pound-python.org/show/25268/ -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://et.redhat.com/~rjones/virt-top

1 1

[PATCH] python: Fixed syntax errors in python/guestfs-py.c
by Masami HIRATA 25 Aug '12

25 Aug '12

Signed-off-by: Masami HIRATA <msmhrt(a)gmail.com> --- generator/generator_python.ml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/generator/generator_python.ml b/generator/generator_python.ml index dc9cbca..b11df6e 100644 --- a/generator/generator_python.ml +++ b/generator/generator_python.ml @@ -391,7 +391,7 @@ free_strings (char **argv) pr " optargs_s.%s = PyString_AsString (py_%s);\n" n n; pr "#else\n"; pr " PyObject *bytes;\n"; - pr " bytes = PyUnicode_AsUTF8String (py_%s));\n" n; + pr " bytes = PyUnicode_AsUTF8String (py_%s);\n" n; pr " optargs_s.%s = PyBytes_AS_STRING (bytes);\n" n; pr "#endif\n"; | OStringList _ -> -- 1.7.9.6 (Apple Git-31.1)

2 1

[libguestfs] Options for hotplugging
by Richard W.M. Jones 21 Aug '12

21 Aug '12

libguestfs recently added support for virtio-scsi and libvirt, and when these are both available this lets us relatively easily add hotplugging of drives. This email is about how we would present that through the libguestfs API. (a) The current API Currently you have to add drive(s) via guestfs_add_drive* and then call guestfs_launch, ie. your program must look something like this: guestfs_h *g = guestfs_create (); guestfs_add_drive (g, "/tmp/disk1.img"); guestfs_add_drive (g, "/tmp/disk2.img"); guestfs_launch (g); After guestfs_launch [the qemu backend is running] you are not allowed to add more drives. The API specifies that you refer to drives in other commands in one of two ways. Either you're allowed to use names like "/dev/sda", "/dev/sdb" etc to refer to the first, second etc drive you added, in the same order that you added them. Or you can call guestfs_list_devices which returns a list of device names, opaque strings that you pass to other functions. In the first case (using "/dev/sdX" names), some magic already happens translating these to the real names underneath, but currently that magic is just "/dev/sdX" -> "/dev/vdX" for the virtio case. Note that we cannot change this API or break existing programs. (b) The hidden appliance drive The libguestfs appliance has to have its own root drive. Currently this is added after the user-added drives. For example, if the user adds two drives, then the appliance will appear as /dev/sdc (or /dev/vdc or whatever). Some magic in the bootloader causes the last drive to be used as the root filesystem. This hidden drive doesn't appear in the API -- for example it is suppressed when we generate the result of guestfs_list_devices. (c) /dev/null drives It's always been possible to add "/dev/null" as a drive via guestfs_add_drive*. This is mainly useful for testing, or if you want to access just those parts of the API which don't require a disk image (for various reasons we force you to add one drive, so if you don't have any drive to add, you can use "/dev/null"). Current libguestfs treats "/dev/null" as a magic string and (because of bugs in qemu) substitutes a non-zero sized temporary file. (d) Maximum number of drives With virtio-scsi, this maximum is pretty large -- currently 255 (256 targets less the hidden appliance), but if we used LUNs or even multiple controllers then it'd be almost unlimited. We actually test up to 255, and virt-df will use as many slots as it can. (e) For libguestfs you can assume the latest of everything, qemu, guest kernel, host kernel, libvirt, tools. Any suggestions based on very new features are fine, even proposed features provided there's a working implementation which is likely to go upstream. - - - - Here are some ideas about how we might add hotplugging without breaking existing clients. (1) The "raw libvirt" option In this one we'd simply provide thin wrappers around virDomainAttachDevice and virDomainDetactDevice, and leave it up to the user to know what they're doing. The problem with this is the hidden appliance disk. We certainly don't want the user to accidentally detach that(!) It's also undesirable for there to be a "hole" in the naming scheme so that you'd have: /dev/sda <- your normal drives /dev/sdb <- [/dev/sdc # sorry, you can't use this, we won't tell you why] /dev/sdd <- your first hotplugged device As far as I know, the kernel assigns /dev/sdX names on a first-free basis, so there's no way to permanently put the appliance at /dev/sdzzz (if there is, please let me know!) (2) The "slots" option In this option you'd have to use null devices to reserve the maximum number of drive slots that you're going to use in the libguestfs handle before launch. Then after launching you'd be allowed to hotplug only those slots. So for example: guestfs_add_drive (g, "/dev/null"); # reserves /dev/sda guestfs_add_drive (g, "/dev/null"); # reserves /dev/sdb guestfs_add_drive (g, "/dev/null"); # reserves /dev/sdc guestfs_launch (g); guestfs_hotplug (g, 1, "/tmp/foo"); # replaces index 1 == /dev/sdb guestfs_hotplug (g, 3, "/tmp/foo"); # error! Although ugly, in some ways this is quite attractive. It maps easily into guestfish scripts. You have contiguous device naming. You often know how many drives you'll need in advance, and if you don't then you can reserve up to max_disks-1. (3) The "serial numbers" option This was Dan's suggestion. Hotplugged drives are known only by their serial number. ie. We hotplug them via libvirt using the <serial/> field, and then they are accessed using /dev/disk/by-id/serial. This is tempting, but unfortunately it doesn't quite work in stock udev, because the actual name used is: /dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_SERIAL We could add a custom udev rule to get the path we wanted. (4) The "rewriting device names" option Since we already have the infrastructure to rewrite device names, we could do some complicated and hairy device name rewriting to make names appear continguous, even though there's an hidden appliance drive. This is my least favourite option, mainly because of the complexity, and complexity is bound to lead to bugs. (5) Your idea here ... As usual, comments and suggestions welcome. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://et.redhat.com/~rjones/virt-top

3 4

[PATCH] xfs: add new api xfs_admin
by Wanlong Gao 20 Aug '12

20 Aug '12

Add new api xfs_admin to change parameters of an XFS filesystem. Signed-off-by: Wanlong Gao <gaowanlong(a)cn.fujitsu.com> --- daemon/xfs.c | 78 ++++++++++++++++++++++++++++++++++++++++++ generator/generator_actions.ml | 21 ++++++++++++ gobject/Makefile.inc | 6 ++-- guestfs-release-notes.txt | 1 + po/POTFILES | 1 + src/MAX_PROC_NR | 2 +- 6 files changed, 106 insertions(+), 3 deletions(-) diff --git a/daemon/xfs.c b/daemon/xfs.c index b331b9b..b32d256 100644 --- a/daemon/xfs.c +++ b/daemon/xfs.c @@ -460,3 +460,81 @@ error: if (out) free (out); return NULL; } + +char * +do_xfs_admin (const char *device, + int extunwritten, int imgfile, int v2log, + int printlabel, int projid32bit, int printuuid, + int lazycounter, const char *label, const char *uuid) +{ + int r; + char *out = NULL, *err = NULL; + const char *argv[MAX_ARGS]; + size_t i = 0; + + ADD_ARG (argv, i, "xfs_admin"); + + /* Optional arguments */ + if (!(optargs_bitmask & GUESTFS_XFS_ADMIN_EXTUNWRITTEN_BITMASK)) + extunwritten = 0; + if (!(optargs_bitmask & GUESTFS_XFS_ADMIN_IMGFILE_BITMASK)) + imgfile = 0; + if (!(optargs_bitmask & GUESTFS_XFS_ADMIN_V2LOG_BITMASK)) + v2log = 0; + if (!(optargs_bitmask & GUESTFS_XFS_ADMIN_PRINTLABEL_BITMASK)) + printlabel = 0; + if (!(optargs_bitmask & GUESTFS_XFS_ADMIN_PRINTUUID_BITMASK)) + printuuid = 0; + if (!(optargs_bitmask & GUESTFS_XFS_ADMIN_PROJID32BIT_BITMASK)) + projid32bit = 0; + + if (extunwritten) + ADD_ARG (argv, i, "-e"); + if (imgfile) + ADD_ARG (argv, i, "-f"); + if (v2log) + ADD_ARG (argv, i, "-j"); + if (printlabel) + ADD_ARG (argv, i, "-l"); + if (printuuid) + ADD_ARG (argv, i, "-u"); + if (projid32bit) + ADD_ARG (argv, i, "-p"); + + if (optargs_bitmask & GUESTFS_XFS_ADMIN_LAZYCOUNTER_BITMASK) { + if (lazycounter) { + ADD_ARG (argv, i, "-c"); + ADD_ARG (argv, i, "1"); + } else { + ADD_ARG (argv, i, "-c"); + ADD_ARG (argv, i, "0"); + } + } + + if (optargs_bitmask & GUESTFS_XFS_ADMIN_LABEL_BITMASK) { + ADD_ARG (argv, i, "-L"); + ADD_ARG (argv, i, label); + } + + if (optargs_bitmask & GUESTFS_XFS_ADMIN_UUID_BITMASK) { + ADD_ARG (argv, i, "-U"); + ADD_ARG (argv, i, uuid); + } + + ADD_ARG (argv, i, device); + ADD_ARG (argv, i, NULL); + + r = commandv (&out, &err, argv); + if (r == -1) { + reply_with_error ("%s: %s", device, err); + goto error; + } + + free (err); + return out; + +error: + if (err) free (err); + if (out) free (out); + return NULL; +} diff --git a/generator/generator_actions.ml b/generator/generator_actions.ml index d45558d..5173503 100644 --- a/generator/generator_actions.ml +++ b/generator/generator_actions.ml @@ -9432,6 +9432,27 @@ empty files in the directory C<dir> with names C<00000000> through C<nr-1> (ie. each file name is 8 digits long padded with zeroes)." }; + { defaults with + name = "xfs_admin"; + style = RString "info", [Device "device"], [OBool "extunwritten"; OBool "imgfile"; OBool "v2log"; OBool "printlabel"; OBool "projid32bit"; OBool "printuuid"; OBool "lazycounter"; OString "label"; OString "uuid"]; + proc_nr = Some 349; + optional = Some "xfs"; + tests = [ + InitEmpty, IfAvailable "xfs", TestOutputStruct ( + [["part_disk"; "/dev/sda"; "mbr"]; + ["mkfs"; "xfs"; "/dev/sda1"; ""; "NOARG"; ""; ""]; + ["xfs_admin"; "/dev/sda1"; ""; ""; ""; ""; ""; ""; "false"; ""; ""]; + ["mount"; "/dev/sda1"; "/"]; + ["xfs_info"; "/"]], + [CompareWithInt ("xfs_lazycount", 0); + ]) + ]; + shortdesc = "change parameters of an XFS filesystem"; + longdesc = "\ +Devices that are mounted cannot be modified. Administrators must unmount +filesystems before C<xfs_admin> can convert parameters. A number of parameters +of a mounted filesystem can be examined and modified using the C<xfs_growfs>." }; + ] (* Non-API meta-commands available only in guestfish. diff --git a/gobject/Makefile.inc b/gobject/Makefile.inc index 49f9538..9f81cd2 100644 --- a/gobject/Makefile.inc +++ b/gobject/Makefile.inc @@ -72,7 +72,8 @@ guestfs_gobject_headers= \ include/guestfs-gobject/optargs-xfs_growfs.h \ include/guestfs-gobject/optargs-rsync.h \ include/guestfs-gobject/optargs-rsync_in.h \ - include/guestfs-gobject/optargs-rsync_out.h + include/guestfs-gobject/optargs-rsync_out.h \ + include/guestfs-gobject/optargs-xfs_admin.h guestfs_gobject_sources= \ src/session.c \ @@ -126,4 +127,5 @@ guestfs_gobject_sources= \ src/optargs-xfs_growfs.c \ src/optargs-rsync.c \ src/optargs-rsync_in.c \ - src/optargs-rsync_out.c + src/optargs-rsync_out.c \ + src/optargs-xfs_admin.c diff --git a/guestfs-release-notes.txt b/guestfs-release-notes.txt index 8fe0103..3c7d02b 100644 --- a/guestfs-release-notes.txt +++ b/guestfs-release-notes.txt @@ -89,6 +89,7 @@ RELEASE NOTES FOR LIBGUESTFS 1.20 For further information, see https://bugzilla.redhat.com/show_bug.cgi?id=788642 + <https://bugzilla.redhat.com/show_bug.cgi?id=788642> New APIs diff --git a/po/POTFILES b/po/POTFILES index 60887dc..d961ac1 100644 --- a/po/POTFILES +++ b/po/POTFILES @@ -167,6 +167,7 @@ gobject/src/optargs-tar_out.c gobject/src/optargs-tune2fs.c gobject/src/optargs-umount.c gobject/src/optargs-umount_local.c +gobject/src/optargs-xfs_admin.c gobject/src/optargs-xfs_growfs.c gobject/src/session.c gobject/src/struct-application.c diff --git a/src/MAX_PROC_NR b/src/MAX_PROC_NR index 71627d7..aef2e27 100644 --- a/src/MAX_PROC_NR +++ b/src/MAX_PROC_NR @@ -1 +1 @@ -348 +349 -- 1.7.12

2 7

[PATCH 0/4] Add customization capabilities to virt-sysprep
by Richard W.M. Jones 16 Aug '12

16 Aug '12

In the TODO file there's a discussion of perhaps writing a new 'virt-customize' tool. I think it's probably better (or at any rate, easier) to just add this functionality into virt-sysprep. That is what this small series of patches aims to achieve. Note these are not very well tested at the moment. The first patch adds a generic and useful '--firstboot' flag. The intended use is to add programs/scripts that run at first boot, for example: virt-sysprep -d some_guest \ --firstboot ./yum-update.sh \ --firstboot ./add-users.sh This is implemented by installing a systemd or sysvinit service into the guest which runs the scripts when the guest boots, deleting them after they have run. [Dan: Is the systemd unit correct?] Patches 2/4 and 3/4 implement another customization: setting the login screen background image. When I actually started investigating how to do this, I realized it is fearsomely complicated because basically every guest type has its own method to do this, and they even change frequently between versions (eg. gconf -> dconf, GNOME vs KDE etc). Since the --firstboot parameter effectively gives this capability to system administrators (albeit they need to work out how to do it themselves), I think it may be a good idea to invest the time making sure --firstboot is bullet-proof, and *not* implement specific customizations like this. What do people think? Patch 4/4 updates the man page to reflect the new capabilities of virt-sysprep. Comments welcome. Rich.

1 5

[PATCH] Even on Debian, the package containing the diff binary it has been diffutils for two years.
by Hilko Bengen 15 Aug '12

15 Aug '12

There had been a virtual package "diff" that depended on diffutils, but that's gone in wheezy/sid, too. --- appliance/packagelist.in | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/appliance/packagelist.in b/appliance/packagelist.in index b26ef23..4830962 100644 --- a/appliance/packagelist.in +++ b/appliance/packagelist.in @@ -23,7 +23,6 @@ btrfs-progs cryptsetup cryptsetup-luks /* old name used before Fedora 17 */ - diffutils e2fsprogs /* e4fsprogs only exists on RHEL 5, will be ignored everywhere else. */ e4fsprogs @@ -51,7 +50,6 @@ bsdmainutils btrfs-tools cryptsetup - diff e2fsprogs gfs-tools gfs2-tools @@ -77,7 +75,6 @@ vim btrfs-progs-unstable cryptsetup - diffutils augeas zfs-fuse e2fsprogs @@ -98,6 +95,7 @@ binutils bzip2 coreutils cpio +diffutils dosfstools file findutils -- 1.7.10.4

2 1

[PATCH 0/7] Add tar compress, numericowner, excludes flags.
by Richard W.M. Jones 14 Aug '12

14 Aug '12

https://bugzilla.redhat.com/show_bug.cgi?id=847880 https://bugzilla.redhat.com/show_bug.cgi?id=847881 This patch series adds various optional arguments to the tar-in and tar-out commands. Firstly (1/7) an optional "compress" flag is added to select compression. This makes the calls tgz-in/tgz-out/txz-in/txz-out deprecated, and expands the range of compression types available. Secondly (2/7) we add an optional numericowner boolean to tar-out, which generates numeric UIDs/GIDs in the resulting tar file. The remaining patches allow us to encode the '--exclude=PATTERN' argument to tar. We add an OStringList type to the generator for passing optional lists of strings. Then we modify tar-out so that this can be translated into zero or more '--exclude' arguments to tar. Note that OStringList is not yet implemented for GObject bindings. Rich.

1 7

[PATCH] sysprep: remove the process accounting log files
by Wanlong Gao 11 Aug '12

11 Aug '12

We just remove the process accounting files previously without touch a empty file, this will cause psacct runs error. Restart the service can't help us create this file auto. couldn't open file '/var/account/pacct': No such file or directory Signed-off-by: Wanlong Gao <gaowanlong(a)cn.fujitsu.com> --- po/POTFILES-ml | 1 + sysprep/Makefile.am | 4 +-- sysprep/sysprep_operation_logfiles.ml | 1 - sysprep/sysprep_operation_pacct_log.ml | 60 ++++++++++++++++++++++++++++++++++ 4 files changed, 63 insertions(+), 3 deletions(-) create mode 100644 sysprep/sysprep_operation_pacct_log.ml diff --git a/po/POTFILES-ml b/po/POTFILES-ml index 11aab05..7f75dc8 100644 --- a/po/POTFILES-ml +++ b/po/POTFILES-ml @@ -26,6 +26,7 @@ sysprep/sysprep_operation_lvm_uuids.ml sysprep/sysprep_operation_machine_id.ml sysprep/sysprep_operation_mail_spool.ml sysprep/sysprep_operation_net_hwaddr.ml +sysprep/sysprep_operation_pacct_log.ml sysprep/sysprep_operation_package_manager_cache.ml sysprep/sysprep_operation_pam_data.ml sysprep/sysprep_operation_puppet_data_log.ml diff --git a/sysprep/Makefile.am b/sysprep/Makefile.am index c52a65f..50c6e11 100644 --- a/sysprep/Makefile.am +++ b/sysprep/Makefile.am @@ -42,8 +42,8 @@ operations = \ abrt_data bash_history blkid_tab ca_certificates cron_spool \ dhcp_client_state dhcp_server_state dovecot_data flag_reconfiguration \ hostname kerberos_data lvm_uuids logfiles machine_id mail_spool \ - net_hwaddr package_manager_cache pam_data puppet_data_log random_seed \ - rhn_systemid samba_db_log script smolt_uuid ssh_hostkeys \ + net_hwaddr pacct_log package_manager_cache pam_data puppet_data_log \ + random_seed rhn_systemid samba_db_log script smolt_uuid ssh_hostkeys \ ssh_userdir sssd_db_log udev_persistent_net user_account \ utmp yum_uuid diff --git a/sysprep/sysprep_operation_logfiles.ml b/sysprep/sysprep_operation_logfiles.ml index 333c317..5e6ce7f 100644 --- a/sysprep/sysprep_operation_logfiles.ml +++ b/sysprep/sysprep_operation_logfiles.ml @@ -54,7 +54,6 @@ let globs = List.sort compare [ "/var/log/libvirt/uml/*.log"; "/var/named/data/named.run"; "/var/log/ppp/connect-errors"; - "/var/account/pacct"; "/var/log/setroubleshoot/*.log"; "/var/log/squid/*.log"; (* And the status file of logrotate *) diff --git a/sysprep/sysprep_operation_pacct_log.ml b/sysprep/sysprep_operation_pacct_log.ml new file mode 100644 index 0000000..aebed29 --- /dev/null +++ b/sysprep/sysprep_operation_pacct_log.ml @@ -0,0 +1,60 @@ +(* virt-sysprep + * Copyright (C) 2012 Fujitsu Limited. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License along + * with this program; if not, write to the Free Software Foundation, Inc., + * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. + *) + +open Sysprep_operation +open Sysprep_gettext.Gettext + +module G = Guestfs + +let pacct_log_perform g root = + let typ = g#inspect_get_type root in + let distro = g#inspect_get_distro root in + match typ, distro with + | "linux", ("fedora"|"rhel"|"centos"|"scientificlinux"|"redhat-based") -> + let files = g#glob_expand "/var/account/pacct*" in + Array.iter ( + fun file -> + try g#rm file with G.Error _ -> () + ) files; + (try g#touch "/var/account/pacct" with G.Error _ -> ()); + [ `Created_files ] + + | "linux", ("debian"|"ubuntu") -> + let files = g#glob_expand "/var/log/account/pacct*" in + Array.iter ( + fun file -> + try g#rm file with G.Error _ -> () + ) files; + (try g#touch "/var/log/account/pacct" with G.Error _ -> ()); + [ `Created_files ] + + | _ -> [] + +let pacct_log_op = { + name = "pacct-log"; + enabled_by_default = true; + heading = s_"Remove the process accounting log files"; + pod_description = Some (s_"\ +The system wide process accounting will store to the pacct +log files if the process accounting is on."); + extra_args = []; + perform_on_filesystems = Some pacct_log_perform; + perform_on_devices = None; +} + +let () = register_operation pacct_log_op -- 1.7.12.rc1

2 1

[PATCH] xfs: fix a possible memory leak
by Wanlong Gao 11 Aug '12

11 Aug '12

free out when failure. Signed-off-by: Wanlong Gao <gaowanlong(a)cn.fujitsu.com> --- daemon/xfs.c | 1 + 1 file changed, 1 insertion(+) diff --git a/daemon/xfs.c b/daemon/xfs.c index 73ba791..b331b9b 100644 --- a/daemon/xfs.c +++ b/daemon/xfs.c @@ -457,5 +457,6 @@ do_xfs_growfs (const char *path, error: if (buf) free (buf); if (err) free (err); + if (out) free (out); return NULL; } -- 1.7.12.rc1

2 1

[PATCH] RELEASE-NOTES: fix typo
by Wanlong Gao 11 Aug '12

11 Aug '12

Fix typo. Signed-off-by: Wanlong Gao <gaowanlong(a)cn.fujitsu.com> --- RELEASE-NOTES | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/RELEASE-NOTES b/RELEASE-NOTES index 6407cba..9242d34 100644 --- a/RELEASE-NOTES +++ b/RELEASE-NOTES @@ -33,7 +33,7 @@ New features - virt-sysprep enhancements: * generate new UUIDs for PVs and VGs - * remote the local machine ID (Wanlong Gao) + * remove the local machine ID (Wanlong Gao) * remove ABRT data (Wanlong Gao) * remove puppet keys and log files (Wanlong Gao) -- 1.7.12.rc1

2 1