March 2016 - Libguestfs - libguestfs List Archives

[PATCH] inspect: improve UsrMove detection (RHBZ#1186935)
by Pino Toscano 17 Mar '16

17 Mar '16

In case /usr is a symlink to /usr/bin, then we cannot rely on /usr/bin to exist, since /usr might be in a different partition. Thus, in case /bin is a symlink, check it points to "usr/bin". --- src/inspect-fs.c | 20 ++++++++++++++++++-- 1 file changed, 18 insertions(+), 2 deletions(-) diff --git a/src/inspect-fs.c b/src/inspect-fs.c index 02fdb2a..0714ae1 100644 --- a/src/inspect-fs.c +++ b/src/inspect-fs.c @@ -43,6 +43,7 @@ static int check_filesystem (guestfs_h *g, const char *mountable, int whole_device); static void extend_fses (guestfs_h *g); static int get_partition_context (guestfs_h *g, const char *partition, int *partnum_ret, int *nr_partitions_ret); +static int is_symlink_to (guestfs_h *g, const char *file, const char *wanted_target); /* Find out if 'device' contains a filesystem. If it does, add * another entry in g->fses. @@ -215,8 +216,7 @@ check_filesystem (guestfs_h *g, const char *mountable, /* Linux root? */ else if (is_dir_etc && (is_dir_bin || - (guestfs_is_symlink (g, "/bin") > 0 && - guestfs_is_dir (g, "/usr/bin") > 0)) && + is_symlink_to (g, "/bin", "usr/bin") > 0) && guestfs_is_file (g, "/etc/fstab") > 0) { fs->is_root = 1; fs->format = OS_FORMAT_INSTALLED; @@ -366,6 +366,22 @@ get_partition_context (guestfs_h *g, const char *partition, return 0; } +static int +is_symlink_to (guestfs_h *g, const char *file, const char *wanted_target) +{ + CLEANUP_FREE char *target = NULL; + + if (guestfs_is_symlink (g, file) == 0) + return 0; + + target = guestfs_readlink (g, file); + /* This should not fail, but play safe. */ + if (target == NULL) + return 0; + + return STREQ (target, wanted_target); +} + int guestfs_int_is_file_nocase (guestfs_h *g, const char *path) { -- 2.5.0

2 2

[virt-builder] XFS metadata corruption in Fedora 23 template images (on resize)
by Kashyap Chamarthy 17 Mar '16

17 Mar '16

Running this: $ virt-builder fedora-23 -o vm1.qcow2 --format qcow2 \ --root-password password:123456 --selinux-relabel --size 100G And, importing vm1.qcow2 into libvirt results in these "corrupted metadata buffer" errors, in a continuous loop, on the serial console: [...] [ 51.687988] XFS (vda3): First 64 bytes of corrupted metadata buffer: [ 51.688938] ffff8800bad08a00: 58 41 47 46 00 00 00 01 00 00 00 4f 00 04 e9 00 XAGF.......O.... [ 51.690212] ffff8800bad08a10: 00 00 00 01 00 00 00 02 00 00 00 00 00 00 00 01 ................ [ 51.691486] ffff8800bad08a20: 00 00 00 01 00 00 00 00 00 00 00 00 00 00 00 76 ...............v [ 51.692767] ffff8800bad08a30: 00 00 00 00 00 04 e8 fb 00 04 e8 fb 00 00 00 00 ................ [ 51.694054] XFS (vda3): metadata I/O error: block 0xc1f3801 ("xfs_trans_read_buf_map") error 117 numblks 1 [ 51.695814] XFS (vda3): Metadata corruption detected at xfs_agf_read_verify+0x70/0x120 [xfs], block 0xc468001 [ 51.697312] XFS (vda3): Unmount and run xfs_repair [ 51.698057] XFS (vda3): First 64 bytes of corrupted metadata buffer: [ 51.699012] ffff8800bad08a00: 58 41 47 46 00 00 00 01 00 00 00 50 00 02 d2 c0 XAGF.......P.... [ 51.700296] ffff8800bad08a10: 00 00 00 01 00 00 00 02 00 00 00 00 00 00 00 01 ................ [ 51.701581] ffff8800bad08a20: 00 00 00 01 00 00 00 00 00 00 00 00 00 00 00 76 ...............v [ 51.702872] ffff8800bad08a30: 00 00 00 00 00 02 d2 bb 00 02 d2 bb 00 00 00 00 ................ [ 51.704166] XFS (vda3): metadata I/O error: block 0xc468001 ("xfs_trans_read_buf_map") error 117 numblks 1 [ 51.705779] systemd-journald[327]: /dev/kmsg buffer overrun, some messages lost. [ 51.717069] XFS (vda3): Metadata corruption detected at xfs_agf_read_verify+0x70/0x120 [xfs], block 0x29b [...] I could reproduce it twice so far with QCOW2 image _and_ raw image formats. Notably, these errors occur only when the image is resized (as is the case above). I confirmed this by testing both the default raw & QCOW2 Fedora-23 image (_without_ the "--size 100G"), there are no such corrupted metadata errors in either case, and I get the login prompt on the serial console just fine. I'm using these on the host: libguestfs-tools-c-1.32.3-2.fc23.x86_64 libguestfs-xfs-1.32.3-2.fc23.x86_64 qemu-system-x86-2.4.1-7.fc23.x86_64 kernel-4.4.4-301.fc23.x86_64 -- /kashyap

2 1

[PATCH 0/2] blkls API to extract unallocated blocks
by Matteo Cafasso 16 Mar '16

16 Mar '16

The blkls API downloads on the host a range of unallocated blocks on the virtual disk image. This allows to recover deleted data on filesystems where icat fails. Example: guestfish --ro -a /home/noxdafox/ubuntu.qcow2 ><fs> run ><fs> mount /dev/sda1 / ><fs> write /test.txt "$foo$bar$" ><fs> rm /test.txt ><fs> umount / ><fs> blkls /dev/sda1 0 8192 blocks.bin $ strings -t d blocks.bin 516096 $foo$bar$ A more complete example can be found here: http://wiki.sleuthkit.org/index.php?title=FS_Analysis Matteo Cafasso (2): added blkls API added blkls API tests daemon/tsk.c | 27 +++++++++++++++++++++++ generator/actions.ml | 19 ++++++++++++++++ src/MAX_PROC_NR | 2 +- tests/tsk/Makefile.am | 3 ++- tests/tsk/test-blkls.sh | 58 +++++++++++++++++++++++++++++++++++++++++++++++++ 5 files changed, 107 insertions(+), 2 deletions(-) create mode 100755 tests/tsk/test-blkls.sh -- 2.7.0

1 2

Improving supermin appliance startup time (lkvm/qboot)
by Török Edwin 16 Mar '16

16 Mar '16

Hi, I remembered reading about Intel Clear Containers [1], and Hyper/Qboot [2] that support startup times measured in the hundreds of miliseconds range. On an AMD FX(tm)-8350 'guestfish -a /dev/null run' takes ~4s when run the 2nd/3rd time: real 0m4.152s user 0m2.120s sys 0m0.564s Are there any plans on applying these improvements to the supermin appliance? I did some quick tests on what would be possible, see below: If I try to use virtio 9P instead it becomes a little faster (instead of root fs on block device): real 0m3.299s user 0m2.084s sys 0m1.828s To do this we need to load just the virtio and 9p modules: let kmods = [ "virtio*.ko*"; "9p*.ko*"; ] Modify init.c to mount the 9p root: https://gist.github.com/anonymous/cb32986fd9b85c315ae4 And wrap qemu to create the 9p device (the mount is just there for a quick test, in reality supermin itself should probably leave the unpacker dir around): https://gist.github.com/anonymous/5cdbc18974f88d6ea9e0 Next step: kvmtool [3]. Avoids some delays introduced by legacy BIOS boot, and device probing, however lkvm currently lacks support for additional virtio-serial channels, so - although the supermin appliance would boot - guestfsd wouldn't be able to communicate with the host: https://gist.github.com/anonymous/e6b0a12e1da811f24e5b I tried QBoot [4], which requires that you first strip down your kernel+initrd to fit into 8128k, and uses an unmodified Qemu otherwise. But then I haven't been able to get it to actually launch guestfsd, it hangs somewhere earlier: https://gist.github.com/anonymous/52fc2890a0884230e4f8 Maybe I removed too many modules here? Another possibility would be to run guestfsd with rkt+lkvm stage1, or hyper.sh; but I don't see a way to forward a unix socket or a virtio-serial channel through them, would guestfsd be able to work over a TCP socket or the main virtio console? (its probably preferable to avoid having any networking in the guestfsd appliance) [1] https://lwn.net/Articles/644675/ [2] https://hyper.sh/faq.html [3] https://git.kernel.org/cgit/linux/kernel/git/will/kvmtool.git/tree/README [4] https://github.com/bonzini/qboot Best regards, -- Edwin Török | Co-founder and Lead Developer Skylable open-source object storage: reliable, fast, secure http://www.skylable.com

2 3

[PATCH 0/2] blkcat API to extract device data units
by Matteo Cafasso 14 Mar '16

14 Mar '16

blkcat allows to extract files given their data units (clusters for NTFS, fragments for ExtX). This API allows to recover files which icat cannot. For example icat often fails retrieving deleted files on Ext3/Ext4 filesystems. As this API works at data unit level, some garbage at the beginning or at the end of the files is expected. An example of blkcat can be found here: http://wiki.sleuthkit.org/index.php?title=FS_Analysis Matteo Cafasso (2): added blkcat API added blkcat API tests daemon/tsk.c | 27 ++++++++++++++++++++++++ generator/actions.ml | 18 ++++++++++++++++ src/MAX_PROC_NR | 2 +- tests/tsk/Makefile.am | 3 ++- tests/tsk/test-blkcat.sh | 53 ++++++++++++++++++++++++++++++++++++++++++++++++ 5 files changed, 101 insertions(+), 2 deletions(-) create mode 100755 tests/tsk/test-blkcat.sh -- 2.7.0

1 2

virt-sysprep and immutable files
by Nicholas Westlake 12 Mar '16

12 Mar '16

Greetings, I attempted to use virt-sysprep however it failed because the .ssh/authorized_keys file of a user had been (purposely) set immutable. Would this be considered a bug? An obvious work around would be not to clone vms with such files of course. Many thanks, Nick

2 1

[PATCH] sleuthkit availability check renamed
by Matteo Cafasso 08 Mar '16

08 Mar '16

>From 'available icat' to 'available sleuthkit'. Signed-off-by: Matteo Cafasso <noxdafox(a)gmail.com> --- daemon/tsk.c | 6 +++--- generator/actions.ml | 2 +- tests/tsk/test-icat.sh | 2 +- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/daemon/tsk.c b/daemon/tsk.c index 647d1e7..0fe1250 100644 --- a/daemon/tsk.c +++ b/daemon/tsk.c @@ -31,12 +31,12 @@ static int file_out (const char *cmd); -GUESTFSD_EXT_CMD(str_tsk_probe, icat); +GUESTFSD_EXT_CMD(str_sleuthkit_probe, icat); int -optgroup_icat_available (void) +optgroup_sleuthkit_available (void) { - return prog_exists (str_tsk_probe); + return prog_exists (str_sleuthkit_probe); } int diff --git a/generator/actions.ml b/generator/actions.ml index 51f2295..ff72cfe 100644 --- a/generator/actions.ml +++ b/generator/actions.ml @@ -12949,7 +12949,7 @@ otherwise the call will fail." }; name = "icat"; added = (1, 33, 14); style = RErr, [Mountable "device"; Int64 "inode"; FileOut "filename"], []; proc_nr = Some 464; - optional = Some "icat"; + optional = Some "sleuthkit"; progress = true; cancellable = true; shortdesc = "download a file to the local machine given its inode"; longdesc = "\ diff --git a/tests/tsk/test-icat.sh b/tests/tsk/test-icat.sh index e1fdc5a..3b0fca4 100755 --- a/tests/tsk/test-icat.sh +++ b/tests/tsk/test-icat.sh @@ -28,7 +28,7 @@ fi rm -f test-mft.bin # Skip if TSK is not supported by the appliance. -if ! guestfish add /dev/null : run : available "icat"; then +if ! guestfish add /dev/null : run : available "sleuthkit"; then echo "$0: skipped because TSK is not available in the appliance" exit 77 fi -- 2.7.0

2 1

[PATCH] fish: reset the console on ^Z RHBZ#1213844
by Maros Zatko 08 Mar '16

08 Mar '16

Patch registers SIGTSTP hook where it sends reset terminal color control sequence using write and fsync. Handler is installed only if signal is not being ignored. Patch uses rl_free_line_state and rl_cleanup_after_signal to unhook readline from terminal, then it calls original TSTP handler using approach in URL below and again hooks readline using rl_reset_after_signal. Handling is based on code from: http://man7.org/tlpi/code/online/dist/pgsjc/handling_SIGTSTP.c.html This approach seems to mostly work. User is sometimes able to get readline into state when it doesn't correctly respond to newline. ^Z and subsequent fg helps there as reset command helps broken terminal. I haven't noticed this using previous approach copied from vi. If you know what is going on I'll be more than happy to fix it. Maros Maros Zatko (1): fish: reset the console on ^Z RHBZ#1213844 fish/fish.c | 76 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 76 insertions(+) -- 2.5.0

1 1

[PATCH v3 0/3] btrfs subvolumes display fix
by Cédric Bosdonnat 08 Mar '16

08 Mar '16

Hi there, Latest and greatest version including all new remarks from both Pino and Rich. Cédric Bosdonnat (3): configure: handle older version of ncurses api: add mountable_device and mountable_subvolume fish: fix btrfs subvolumes display in error case fish/options.c | 28 ++++++++++++++++++++++++++- generator/actions.ml | 26 +++++++++++++++++++++++++ m4/guestfs_libraries.m4 | 9 ++++++++- po/POTFILES | 1 + src/Makefile.am | 1 + src/mountable.c | 51 +++++++++++++++++++++++++++++++++++++++++++++++++ 6 files changed, 114 insertions(+), 2 deletions(-) create mode 100644 src/mountable.c -- 2.6.2

2 5

[PATCH v2 0/3] btrfs subvolumes display fix
by Cédric Bosdonnat 08 Mar '16

08 Mar '16

Hi all, Here is version 2 of the patch series, including the changes for Pino's remarks. Cédric Bosdonnat (3): configure: handle older version of ncurses api: add mountable_device and mountable_subvolume fish: fix btrfs subvolumes display in error case fish/options.c | 28 ++++++++++++++++++++++++++- generator/actions.ml | 26 +++++++++++++++++++++++++ m4/guestfs_libraries.m4 | 12 +++++++++++- po/POTFILES | 1 + src/Makefile.am | 1 + src/mountable.c | 51 +++++++++++++++++++++++++++++++++++++++++++++++++ 6 files changed, 117 insertions(+), 2 deletions(-) create mode 100644 src/mountable.c -- 2.6.2

3 7