[PATCH 0/3] Add JSON output for virt-builder
by Pino Toscano
Hi,
This small patch serie adds a JSON output for virt-builder.
This way it is possible to parse the list of available templates,
with no need to parse the unstructured and possibly changing short and
long outputs of virt-builder.
Pino Toscano (3):
builder: small refactor of the list output
builder: add --list-format
builder: add a JSON output for --list
builder/builder.ml | 4 +-
builder/cmdline.ml | 19 +++--
builder/list_entries.ml | 142 ++++++++++++++++++++++++++++----------
builder/list_entries.mli | 2 +-
builder/test-virt-builder-list.sh | 67 ++++++++++++++++++
builder/virt-builder.pod | 34 ++++++++-
6 files changed, 222 insertions(+), 46 deletions(-)
--
1.8.3.1
10 years, 4 months
[PATCH 1/2] hivex: ruby: find files to install in correct directories when building out-of-tree
by Hilko Bengen
---
ruby/Makefile.am | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/ruby/Makefile.am b/ruby/Makefile.am
index b6654d3..e78ab59 100644
--- a/ruby/Makefile.am
+++ b/ruby/Makefile.am
@@ -51,7 +51,7 @@ RUBY_SITEARCH := $(shell $(RUBY) -rrbconfig -e "puts RbConfig::CONFIG['sitearchd
install:
$(MKDIR_P) $(DESTDIR)$(RUBY_SITELIB)
$(MKDIR_P) $(DESTDIR)$(RUBY_SITEARCH)
- $(INSTALL) -p -m 0644 lib/hivex.rb $(DESTDIR)$(RUBY_SITELIB)
- $(INSTALL) -p -m 0755 ext/hivex/_hivex.so $(DESTDIR)$(RUBY_SITEARCH)
+ $(INSTALL) -p -m 0644 ${srcdir}/lib/hivex.rb $(DESTDIR)$(RUBY_SITELIB)
+ $(INSTALL) -p -m 0755 ${builddir}/ext/hivex/_hivex.so $(DESTDIR)$(RUBY_SITEARCH)
endif
--
1.8.5.2
10 years, 4 months
Mounting Snapshots
by Thorsten Sick
Hello
I am doing a research project (http://www.ites-project.org/) and want to
be able to mount VM images (all kinds of VMs, starting with VirtualBox).
For that libguestfs seems perfect. I am already able to mount the base
HD, but I was not able to additionally mount the overlaying Snapshot files.
Is there a way to mount several HD files onto each other ?
Thanks
Thorsten Sick
--
Thorsten Sick
Research
Direct: +49 7542-500 4252
Email: thorsten.sick(a)avira.com
--
Avira Operations GmbH & Co. KG
Kaplaneiweg 1 | 88069 Tettnang | Deutschland / Germany
Telefon / Telephone: +49 7542-500 0
Telefax / Facsimile: +49 7542-500 3000
Registergericht: Amtsgericht Ulm, HRA 722586 | USt.-IdNr.: DE 815289569 | Pers. haftende Gesellschafterin: Avira OP GmbH | Firmensitz: Tettnang | Registergericht: Amtsgericht Ulm, HRB 726712 | Geschäftsführer: Travis Witteveen
Commercial Register: Amtsgericht Ulm, HRA 722586 | VAT-ID: DE 815289569 | Personally Liable Partner: Avira OP GmbH | Headquarters: Tettnang | Commercial Register: Amtsgericht Ulm, HRB 726712 | Chief Executive Officer (CEO): Travis Witteveen
10 years, 4 months
[PATCH] fuse: provide a stub "flush" implementation (RHBZ#660687).
by Pino Toscano
It seems that FUSE can invoke flush to make sure the pending changes
(e.g. to the attributes) of a file are set. Since a missing flush
implementation is handled as if it were returning ENOSYS, this can cause
issues later.
To overcome this, just provide a stub implementation which does nothing,
since we have nothing to do and don't want to have FUSE error out.
Furthermore, uncomment the timestamp checks in test-fuse.sh, since now
they should be working fine.
---
fuse/test-fuse.sh | 23 +++++++++++------------
src/fuse.c | 15 +++++++++++++++
2 files changed, 26 insertions(+), 12 deletions(-)
diff --git a/fuse/test-fuse.sh b/fuse/test-fuse.sh
index f1e03d0..30b3c31 100755
--- a/fuse/test-fuse.sh
+++ b/fuse/test-fuse.sh
@@ -220,18 +220,17 @@ if truncate --help >/dev/null 2>&1; then
rm -f truncated
fi
-# Disabled because of RHBZ#660687 on Debian.
-# stage Checking utimens and timestamps
-# for ts in 12345 1234567 987654321; do
-# # NB: It's not possible to set the ctime with touch.
-# touch -a -d @$ts timestamp
-# [ "$(stat -c %X timestamp)" -eq $ts ]
-# touch -m -d @$ts timestamp
-# [ "$(stat -c %Y timestamp)" -eq $ts ]
-# touch -d @$ts timestamp
-# [ "$(stat -c %X timestamp)" -eq $ts ]
-# [ "$(stat -c %Y timestamp)" -eq $ts ]
-# done
+stage Checking utimens and timestamps
+for ts in 12345 1234567 987654321; do
+ # NB: It's not possible to set the ctime with touch.
+ touch -a -d @$ts timestamp
+ [ "$(stat -c %X timestamp)" -eq $ts ]
+ touch -m -d @$ts timestamp
+ [ "$(stat -c %Y timestamp)" -eq $ts ]
+ touch -d @$ts timestamp
+ [ "$(stat -c %X timestamp)" -eq $ts ]
+ [ "$(stat -c %Y timestamp)" -eq $ts ]
+done
stage Checking writes
cp hello.txt copy.txt
diff --git a/src/fuse.c b/src/fuse.c
index 967a744..748b933 100644
--- a/src/fuse.c
+++ b/src/fuse.c
@@ -876,6 +876,20 @@ mount_local_removexattr(const char *path, const char *name)
return 0;
}
+static int
+mount_local_flush(const char *path, struct fuse_file_info *fi)
+{
+ DECL_G ();
+ DEBUG_CALL ("%s", path);
+
+ /* Just a stub. This method is called whenever FUSE wants to flush the
+ * pending changes (f.ex. to attributes) to a file. Since we don't have
+ * anything to do and don't want FUSE to think something went badly,
+ * just return 0.
+ */
+ return 0;
+}
+
static struct fuse_operations mount_local_operations = {
.getattr = mount_local_getattr,
.access = mount_local_access,
@@ -902,6 +916,7 @@ static struct fuse_operations mount_local_operations = {
.getxattr = mount_local_getxattr,
.listxattr = mount_local_listxattr,
.removexattr = mount_local_removexattr,
+ .flush = mount_local_flush,
};
int
--
1.8.3.1
10 years, 4 months
[PATCH 1/4] hivex: Python 2.6 does not have sysconfig.
by Hilko Bengen
---
configure.ac | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/configure.ac b/configure.ac
index 6785037..203f34f 100644
--- a/configure.ac
+++ b/configure.ac
@@ -329,8 +329,8 @@ AS_IF([test "x$enable_python" != "xno"],
AC_MSG_CHECKING([for Python extension suffix (PEP-3149)])
if test -z "$PYTHON_EXT_SUFFIX"; then
- python_ext_suffix=`$PYTHON -c "import sysconfig; \
- print (sysconfig.get_config_var('EXT_SUFFIX') or sysconfig.get_config_var('SO'))"`
+ python_ext_suffix=`$PYTHON -c "import distutils.sysconfig; \
+ print (distutils.sysconfig.get_config_var('EXT_SUFFIX') or distutils.sysconfig.get_config_var('SO'))"`
PYTHON_EXT_SUFFIX=$python_ext_suffix
fi
AC_MSG_RESULT([$PYTHON_EXT_SUFFIX])
--
1.8.5.2
10 years, 4 months
[PATCH] fuse: remove extra trailing \n in debug messages
by Pino Toscano
debug() adds it already.
---
src/fuse.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/fuse.c b/src/fuse.c
index d684c84..288c02a 100644
--- a/src/fuse.c
+++ b/src/fuse.c
@@ -68,7 +68,7 @@ gl_lock_define_initialized (static, mount_local_lock);
#define DEBUG_CALL(fs,...) \
if (g->ml_debug_calls) { \
debug (g, \
- "%s: %s (" fs ")\n", \
+ "%s: %s (" fs ")", \
g->localmountpoint, __func__, ## __VA_ARGS__); \
}
--
1.8.3.1
10 years, 4 months
Double fault panic in L2 upon v2v conversion
by Rom Freiman
Hi everybody,
Wanted to hear your opinion and to receive a smart advice.
I'm trying to use virt-v2v in order to convert ova image (exported from
vmware) to run on libvirt/kvm - all this inside a VM of fedora.
The converted image is also a fedora.
During the conversion process, in some point of libguestfs activity, I get
double fault panic from L2 (printed as part of libguest output) and the
conversion process fails - no errors appear neither in L0 not in L1 message
logs.
This is more detailed explanation:
Description of problem:
During v2v conversion for ova fedora image in a VM there is double
fault panic happens and L2 crashes during libguestfs conversion.
The crash happens in different stages, but usually upon memory pressure in L0.
There are no error logs in L1 and cannot find strong correlation to
patches that were added to L0-KVM to avoid L0 crash upon nested vm
with high memory pressure -
http://git.kernel.org/cgit/virt/kvm/kvm.git/patch/arch/x86/kvm/mmu.c?id=9....
Command within L1 to perform the conversion (fedora.ova - vmdk image
of fedora - was placed in advance on the VM):
LIBGUESTFS_TRACE=1 LIBGUESTFS_DEBUG=1 /usr/bin/virt-v2v -i ova -os
default -oc qemu:///system -of qcow2 -n default
/var/tmp/fedora-v2v.ova
Version-Release number of selected component (if applicable):
L0:
Kernel: 3.11.8-200.fc19 + nested crash patches
libvritd: 1.0.5.8
qemu: 1.6.1
libguestfs-test-tool 1.22.7fedora=19,release=1.fc19,libvirt
L1:
Kernel: 3.11.8-200..fc19.x86_64
libvirtd: 1.0.5.8
qemu: 1.6.1 + v2v patch (skip vmdk version verification)
libguestfs-test-tool 1.22.7fedora=19,release=2.fc19,libvirt
virt-v2v 0.9.0
L2:
Kernel: 3.11.10-301.fc20
How reproducible:
LIBGUESTFS_TRACE=1 LIBGUESTFS_DEBUG=1 /usr/bin/virt-v2v -i ova -os
default -oc qemu:///system -of qcow2 -n default
/var/tmp/fedora-v2v.ova
Steps to Reproduce:
1. Upload ova image to VM
2. Run v2v to perform the conversion
3. Add some memory pressure on L0 (dd if=/dev/urandom of=/tmp/bigfile count=6M)
Actual results:
libguestfs: checksum of existing appliance:
63a0a4c8acb7f789c7a725bebbb16e64a3bc0999fdca61931babda50292d83b8
libguestfs: trace: get_cachedir
libguestfs: trace: get_cachedir = "/var/tmp"
libguestfs: [00337ms] begin testing qemu features
libguestfs: command: run: /usr/bin/qemu-kvm
libguestfs: command: run: \ -nographic
libguestfs: command: run: \ -help
libguestfs: command: run: /usr/bin/qemu-kvm
libguestfs: command: run: \ -nographic
libguestfs: command: run: \ -version
libguestfs: qemu version 1.6
libguestfs: command: run: /usr/bin/qemu-kvm
libguestfs: command: run: \ -nographic
libguestfs: command: run: \ -machine accel=kvm:tcg
libguestfs: command: run: \ -device ?
libguestfs: [00661ms] finished testing qemu features
[00663ms] /usr/bin/qemu-kvm \
-global virtio-blk-pci.scsi=off \
-nodefconfig \
-nodefaults \
-nographic \
-machine accel=kvm:tcg \
-m 500 \
-no-reboot \
-no-hpet \
-kernel /var/tmp/.guestfs-0/kernel.1416 \
-initrd /var/tmp/.guestfs-0/initrd.1416 \
-device virtio-scsi-pci,id=scsi \
-drive file=/var/lib/libvirt/images/fedora-v2v-disk1.vmdk,cache=none,format=qcow2,id=hd0,if=ide
\
-drive file=/tmp/3pEC1jAXQs,snapshot=on,format=raw,id=hd1,if=ide \
-drive file=/var/tmp/.guestfs-0/root.1416,snapshot=on,id=appliance,if=none,cache=unsafe
\
-device scsi-hd,drive=appliance \
-device virtio-serial \
-serial stdio \
-device sga \
-chardev socket,path=/tmp/libguestfsKr0x6j/guestfsd.sock,id=channel0 \
-device virtserialport,chardev=channel0,name=org.libguestfs.channel.0 \
-netdev user,id=usernet,net=169.254.0.0/16 \
-device virtio-net-pci,netdev=usernet \
-append 'panic=1 console=ttyS0 udevtimeout=600 no_timer_check
acpi=off printk.time=1 cgroup_disable=memory root=/dev/sdc selinux=0
guestfs_verbose=1 TERM=dumb'
\x1b[1;256r\x1b[256;256H\x1b[6n
Google, Inc.
Serial Graphics Adapter 02/18/13
SGABIOS $Id: sgabios.S 8 2010-04-22 00:03:40Z nlaredo $ (mockbuild@)
Mon Feb 18 08:29:02 UTC 2013
Term: 80x24
4 0
SeaBIOS (version ?-20131118_112708-)
iPXE (http://ipxe.org) 00:04.0 C100 PCI2.10 PnP PMM+1F3C7C60+1F327C60 C100
Booting from ROM...
Probing EDD (edd=off to disable)... ok
\x1b[2J[ 0.000000] Initializing cgroup subsys cpuset
[ 0.000000] Initializing cgroup subsys cpu
[ 0.000000] Initializing cgroup subsys cpuacct
[ 0.000000] Linux version
3.11.8-200.strato0002.fc19.strato.c3850ae03e9d.x86_64
(mockbuild@ellaria) (gcc version 4.8.2 20131017 (Red Hat 4.8.2-1)
(GCC) ) #1 SMP Mon Nov 18 17:54:34 IST 2013
[ 0.000000] Command line: panic=1 console=ttyS0 udevtimeout=600
no_timer_check acpi=off printk.time=1 cgroup_disable=memory
root=/dev/sdc selinux=0 guestfs_verbose=1 TERM=dumb
[ 0.000000] e820: BIOS-provided physical RAM map:
[ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
............................................
.........More logs here............
.............................................
guestfsd: main_loop: new request, len 0x3c
mount -o ro /dev/sdb /sysroot/
[ 12.645305] PANIC: double fault, error_code: 0x0
[ 12.645305] CPU: 0 PID: 141 Comm: mount Not tainted
3.11.8-200.strato0002.fc19.strato.c3850ae03e9d.x86_64 #1
[ 12.645305] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011
[ 12.645305] task: ffff88001cc816e0 ti: ffff88001cde6000 task.ti:
ffff88001cde6000
[ 12.645305] RIP: 0033:[<00007fa602c5b99b>] [<00007fa602c5b99b>]
0x7fa602c5b99a
[ 12.645305] RSP: 002b:00007fff4f5884a0 EFLAGS: 00010216
[ 12.645305] RAX: 00007fa602008ff8 RBX: 00007fa601ff0000 RCX: 00007fa601ff0000
[ 12.645305] RDX: 00000000003b7068 RSI: 00007fff4f588560 RDI: 00007fa601ff3d18
[ 12.645305] RBP: 00007fff4f5885d0 R08: 00007fa60200f310 R09: 0000000000000000
[ 12.645305] R10: 0000000000000022 R11: 00007fa60200f310 R12: 00007fa60200e9b0
[ 12.645305] R13: 0000000000000000 R14: 0000000000000000 R15: 00007fa602e6e990
[ 12.645305] FS: 00007fa602e69880(0000) GS:ffff88001f000000(0000)
knlGS:0000000000000000
[ 12.645305] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 12.645305] CR2: 0000000000000000 CR3: 000000001d7fb000 CR4: 00000000000006f0
[ 12.645305]
[ 12.645305] Kernel panic - not syncing: Machine halted.
[ 12.645305] CPU: 0 PID: 141 Comm: mount Not tainted
3.11.8-200.strato0002.fc19.strato.c3850ae03e9d.x86_64 #1
[ 12.645305] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011
[ 12.645305] ffff88001f005f58 ffff88001f005e90 ffffffff8164024b
ffffffff819e89dc
[ 12.645305] ffff88001f005f08 ffffffff8163c272 0000000000000008
ffff88001f005f18
[ 12.645305] ffff88001f005eb8 ffffffff8163c8e5 0000000000000046
00000000000000b1
[ 12.645305] Call Trace:
[ 12.645305] <#DF> [<ffffffff8164024b>] dump_stack+0x45/0x56
[ 12.645305] [<ffffffff8163c272>] panic+0xc8/0x1d7
[ 12.645305] [<ffffffff8163c8e5>] ? printk+0x67/0x69
[ 12.645305] [<ffffffff81048ae1>] df_debug+0x31/0x40
[ 12.645305] [<ffffffff810132ed>] do_double_fault+0x5d/0x80
[ 12.645305] [<ffffffff81650b88>] double_fault+0x28/0x30
[ 12.645305] <<EOE>>
[ 12.645305] Rebooting in 1 seconds..libguestfs: child_cleanup:
0x3a05f50: child process died
libguestfs: sending SIGTERM to process 1526
libguestfs: trace: mount_ro = -1 (error)
libguestfs: trace: vfs_type "/dev/sda1"
libguestfs: trace: vfs_type = NULL (error)
libguestfs: check_for_filesystem_on: /dev/sda1 (failed to get vfs type)
libguestfs: trace: internal_parse_mountable "/dev/sda1"
libguestfs: trace: internal_parse_mountable = NULL (error)
libguestfs: trace: inspect_os = NULL (error)
libguestfs: trace: close
Additional info:
The same crash also happens when L0 is 3.11.9 (with kvm patch to avoid
L0 crash in nested environment -
http://git.kernel.org/cgit/virt/kvm/kvm.git/patch/arch/x86/kvm/mmu.c?id=9...)
but more rarely.
Thanks a lot,
Rom
10 years, 4 months
[PATCH] hivex: Make empty strings in REG_MULTI_SZ values available.
by Hilko Bengen
---
lib/value.c | 26 +++++++++++++++++++++++---
1 file changed, 23 insertions(+), 3 deletions(-)
diff --git a/lib/value.c b/lib/value.c
index db6396f..c4e21ec 100644
--- a/lib/value.c
+++ b/lib/value.c
@@ -462,7 +462,27 @@ hivex_value_string (hive_h *h, hive_value_h value)
return ret;
}
-/* http://blogs.msdn.com/oldnewthing/archive/2009/10/08/9904646.aspx */
+/* Even though
+ * http://msdn.microsoft.com/en-us/library/windows/desktop/ms724884.aspx
+ * and
+ * http://blogs.msdn.com/oldnewthing/archive/2009/10/08/9904646.aspx
+ * claim that it is not possible to store empty strings in MULTI_SZ
+ * string lists, such lists are used by Windows itself:
+ *
+ * The MoveFileEx function can schedule files to be renamed (or
+ * removed) at restart time by storing pairs of filenames in the
+ * HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\PendingFileRenameOperations
+ * value.
+ *
+ * The documentation for MoveFileEx
+ * (http://msdn.microsoft.com/en-us/library/windows/desktop/aa365240)
+ * states that "[i]f dwFlags specifies MOVEFILE_DELAY_UNTIL_REBOOT,
+ * and lpNewFileName is NULL, MoveFileEx registers the
+ * lpExistingFileName file to be deleted when the system restarts."
+ *
+ * For scheduled removals, the second file name of any pair stored in
+ * PendingFileRenameOperations is an empty string.
+ */
char **
hivex_value_multiple_strings (hive_h *h, hive_value_h value)
{
@@ -490,8 +510,8 @@ hivex_value_multiple_strings (hive_h *h, hive_value_h value)
char *p = data;
size_t plen;
- while (p < data + len &&
- (plen = _hivex_utf16_string_len_in_bytes_max (p, data + len - p)) > 0) {
+ while (p < data + len) {
+ plen = _hivex_utf16_string_len_in_bytes_max (p, data + len - p);
nr_strings++;
char **ret2 = realloc (ret, (1 + nr_strings) * sizeof (char *));
if (ret2 == NULL) {
--
1.8.5.2
10 years, 4 months
hivex: Make node names and value names with embedded null characters accessible
by Hilko Bengen
On Windows, there exist at least two APIs for dealing with the
Registry: The Win32 API (RegCreateKeyA, RegCreateKeyW, etc.) works
with null-terminated ASCII or UTF-16 strings. The native API
(ZwCreateKey, etc.), on the other hand works with UTF-16 strings that
are stored as buffers+length and may contain null characters. Malware
authors have been relying on the Win32 API's inability to properly
work with such names for several years.
These changes make such names accessible from hivex.
10 years, 4 months
Re: [Libguestfs] Livecd-creator is disabling selinux
by Richard W.M. Jones
[Moving this to the libguestfs mailing list]
On Mon, Jan 13, 2014 at 03:05:14PM -0500, Daniel J Walsh wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 01/13/2014 11:49 AM, Richard W.M. Jones wrote:
> > On Mon, Jan 13, 2014 at 10:20:22AM -0500, Daniel J Walsh wrote:
> >> Secondly we prevent even unconfined_t from putting down labels on the
> >> file system that the kernel does not understand. IE If I am building a
> >> F21 image on a RHEL6 box, it would blow up in enforcing mode if run as
> >> unconfined_t. We added a special policy called livecd_t that is allowed
> >> to put down labels which the kernel does not understand, and unconfined_t
> >> will transition to this domain.
> >
> > Slightly off-topic, but this (in-)ability to label files with labels which
> > the kernel doesn't know about affects libguestfs negatively too. Is there
> > some reason why it's bad?
>
> Well we could add a label to libguestfs to allow these labels. From
> the running kernel point of view you end up with unlabeled_t.
I think it's more complex with libguestfs because of the appliance,
and because libguestfs normally runs with SELinux disabled inside the
appliance.
[Note to peanut gallery: libguestfs uses and endorses SELinux and
sVirt to implement containment of qemu *in the host*. The above
applies only to the appliance.]
I know you wrote a good blog posting and we have a (very very old)
open bug to fix libguestfs's SELinux support:
https://bugzilla.redhat.com/show_bug.cgi?id=554829
After thinking about this for some years and Pino Toscano implementing
a new & useful copy-attributes API in libguestfs, I think what we
really want is to copy security.selinux xattr from one file to
another. This gives us almost all we need, doesn't require us to run
loadpolicy, and should work independent of guest policy.
Luckily for us ... it works!
$ virt-builder fedora-20
$ guestfish -a fedora-20.img -i
><fs> getxattrs /etc/shadow
[0] = {
attrname: security.selinux
attrval: system_u:object_r:shadow_t:s0\x00
}
><fs> cp /etc/shadow /etc/shadow.copy
><fs> setxattr security.selinux system_u:object_r:shadow_t:s0 29 /etc/shadow.copy
><fs> getxattrs /etc/shadow.copy [0] = {
attrname: security.selinux
attrval: system_u:object_r:shadow_t:s0
}
Is this expected? Are we bypassing the kernel/SELinux in some way?
Note that the libguestfs appliance kernel doesn't (AFAIK) know about
the system_u:object_r:shadow_t:s0 label.
> We also would like to prevent users from making mistakes like
> assigning httpd_t to a file when it is a process type.
>
> SELinux is going to check before you put the label down unless you have the
> mac_admin capability.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW
10 years, 4 months