Re: libguestfs performance
by Richard W.M. Jones
On Tue, Dec 01, 2009 at 04:08:38PM +0100, Pierre Riteau wrote:
> For a project of mine I would like to create VMs from tarballs of
> root filesystems. I would like to use libguestfs since this is
> exactly what the tar2vm.sh guestfish recipe does.
>
> However, I need to extract tarballs that are in the hundreds of
> megabytes, but my tests showed that performance is very low, both
> with tgz-in and with guestmount (sorry I don't have the numbers any
> more, but it was something like more than one hour to extract a
> tarball containing a Debian Lenny installation).
>
> Is there any way to drastically improve libguestfs' performance?
First of all, read:
http://libguestfs.org/FAQ.html#slow
Using tar-in/tgz-in to copy the tarball into the appliance is always
going to be slow. I would think you should get better performance if
you write the tarball into an ISO first, and attach that. Something
like:
mkisofs -JR -o cd.iso the_tarball.tar.gz
guestfish -a guest.img -a cd.iso
[...]
><fs> mkmountpoint /guest
><fs> mkmountpoint /cd
[...]
><fs> mount /dev/sdb /cd
[...]
><fs> debug sh "tar xcf /cd/the_tarball.tar.gz /guest"
However we found experimentally that the fastest method of all for
creating distributions was to prebuild an ext2/3/4 image, and 'dd' it
over to the target partition, then resize it to the final (larger)
size. This method is extremely quick, enabling us to build Fedora
images in just a minute or two:
http://rwmj.wordpress.com/2009/11/23/prebuilt-distributions-part-2/#content
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
libguestfs lets you edit virtual machines. Supports shell scripting,
bindings from many languages. http://et.redhat.com/~rjones/libguestfs/
See what it can do: http://et.redhat.com/~rjones/libguestfs/recipes.html
14 years, 5 months
Re: Windows port of daemon?
by Szabolcs Szakacsits
On Thu, Nov 19, 2009 at 03:40:32PM +0000, Richard W.M. Jones wrote:
> The reason to want a Windows appliance at all is twofold: (1) better
> support for NTFS filesystems and Windows-native filesystem features
> (attributes, volume management etc)
Could you please tell more technical details? We support NTFS attributes,
volume management, etc for years. Thanks.
Szaka
14 years, 5 months
Attempts to install a Windows driver from WinPE
by Matthew Booth
One of the things we would really like to be able to do for V2V is to
install a new driver in a Windows guest. There are a couple of reasons
for this:
* The guest may not be bootable without the driver installed, for
example because the underlying virtual hardware has changed from vmscsi
to virtio.
* If the guest can boot, the alternative is to modify the guest to run a
script on next boot. This requires making assumptions about supporting
software being installed and working correctly on the guest. Certain
environments, particularly heavily locked-down environments, make this
an unsafe assumption.
The Windows PE environment looks perfect for this task. It gives you a
very lightweight Windows OS which can be customised with additional
tools. It is specifically for doing installations. I spent Friday trying
to use it to install a driver in a guest. Here's what I tried and why it
didn't work.
Installing a driver in Windows is 'driven' by a .inf file. From my
(admittedly limited) understanding, this broadly describes:
* The files which need to be installed
* The hardware the driver is compatible with
The files, including the .inf file itself must be copied in to the
correct places. In addition, information from the .inf file must be
written to the registry. It is this last part which causes problems.
From reading documentation, it appears that a driver would normally be
installed using the SetupCopyOEMInf() library call. I wrote a simple
wrapper round this and installed it in the Windows PE image, along with
the VirtIO drivers. I booted into Windows PE and attempted to install
the driver. As you might expect, the drivers were installed into the
Windows PE image rather than the guest. I then tried setting
%systemdrive% and %systemroot% to the guest image. This appears to have
no effect. This is what makes me suspect that the process is primarily
registry driven.
I started looking around for ways of using a different registry. I
discovered the Registry Editor PE plugin to BartPE
(https://sourceforge.net/projects/regeditpe/) which allows editing the
registry of a guest. Looking at how it does this, it uses reg.exe to
load the guests's hives. I confirmed that you can do this. Unfortunately
you don't seem to be able to replace the default hives. The new hives
are loaded in a different part of the tree, and are therefore ignored.
This is as far as I've got. Still on my list are:
* Ask on various Windows mailing lists how to do this
* Investigate if the packaged .msi containing the drivers is more flexible.
* Look for other, possibly lower level, ways of replacing making a
process use a different registry.
Any and all suggestions are gratefully received,
Matt
--
Matthew Booth, RHCA, RHCSS
Red Hat Engineering, Virtualisation Team
M: +44 (0)7977 267231
GPG ID: D33C3490
GPG FPR: 3733 612D 2D05 5458 8A8A 1600 3441 EA19 D33C 3490
14 years, 5 months
[PATCH libguestfs] accommodate leading "+" in git submodule output
by Jim Meyering
With the very latest git, "git submodule status" output may now
start with a "+". This change makes it so that is also stripped
away before comparing current and stored SHA1 values.
>From 1bc2c5def80b8a0860a1980551d433f1fdce9a50 Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering(a)redhat.com>
Date: Wed, 2 Dec 2009 15:06:48 +0100
Subject: [PATCH libguestfs] accommodate leading "+" in git submodule output
* cfg.mk (_submodule_hash): Also filter out "+".
* autogen.sh: Likewise.
---
autogen.sh | 2 +-
cfg.mk | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/autogen.sh b/autogen.sh
index 166ca56..6e89c84 100755
--- a/autogen.sh
+++ b/autogen.sh
@@ -24,7 +24,7 @@ set -v
# Ensure that whenever we pull in a gnulib update or otherwise change to a
# different version (i.e., when switching branches), we also rerun ./bootstrap.
curr_status=.git-module-status
-t=$(git submodule status|sed 's/^[ -]//;s/ .*//')
+t=$(git submodule status|sed 's/^[ +-]//;s/ .*//')
if test "$t" = "$(cat $curr_status 2>/dev/null)"; then
: # good, it's up to date
else
diff --git a/cfg.mk b/cfg.mk
index 8811d0d..d45ec88 100644
--- a/cfg.mk
+++ b/cfg.mk
@@ -134,7 +134,7 @@ ifeq (0,$(MAKELEVEL))
# b653eda3ac4864de205419d9f41eec267cb89eeb .gnulib (v0.0-2286-gb653eda)
# $ cat .git-module-status
# b653eda3ac4864de205419d9f41eec267cb89eeb
- _submodule_hash = sed 's/^[ -]//;s/ .*//'
+ _submodule_hash = sed 's/^[ +-]//;s/ .*//'
_update_required := $(shell \
actual=$$(git submodule status | $(_submodule_hash)); \
stamp="$$($(_submodule_hash) $(_curr_status) 2>/dev/null)"; \
--
1.6.6.rc0.324.gb5bf2
14 years, 5 months
RE: Attempts to install a Windows driver from WinPE
by Yuval Kashtan
I have some knowledge with windows and might be able to help
SetupCopyOEMInf is the right API
And it should be possible to track it's whereabouts using filemon and
regmon.
I would have also investigate more into pkgmgr.exe which sounds as a
possibly possible solution (from its description)
Another possible source of info is through the administration guide. I
know it's possible to sysprep a machine with additional drivers for later
discovery by the PnP mechanism and the same mechanism used there might be
useful here as well
BTW - storage pose additional complexity if you want to boot directly from
the new driver (like in the case of moving to virtio), there are
additional required changes (other than just installing the driver, you
have to tell it to load it before other components (something to do with
the CriticalDeviceDatabse
http://technet.microsoft.com/en-us/library/cc976055.aspx)
> -----Original Message-----
> From: Itamar Heim [mailto:iheim@redhat.com]
> Sent: Monday, November 30, 2009 2:33 PM
> To: 'Yuval Kashtan'; 'Yaniv Kaul'
> Subject: FW: Attempts to install a Windows driver from WinPE
>
> fyi
>
> -----Original Message-----
> From: Matthew Booth [mailto:mbooth@redhat.com]
> Sent: Monday, November 30, 2009 5:52 AM
> To: libguestfs(a)redhat.com
> Cc: 'Ayal Baron'; Itamar Heim
> Subject: Attempts to install a Windows driver from WinPE
>
> One of the things we would really like to be able to do for V2V is to
> install a new driver in a Windows guest. There are a couple of reasons
> for this:
>
> * The guest may not be bootable without the driver installed, for
> example because the underlying virtual hardware has changed from vmscsi
> to virtio.
>
> * If the guest can boot, the alternative is to modify the guest to run
> a
> script on next boot. This requires making assumptions about supporting
> software being installed and working correctly on the guest. Certain
> environments, particularly heavily locked-down environments, make this
> an unsafe assumption.
>
> The Windows PE environment looks perfect for this task. It gives you a
> very lightweight Windows OS which can be customised with additional
> tools. It is specifically for doing installations. I spent Friday
> trying
> to use it to install a driver in a guest. Here's what I tried and why
> it
> didn't work.
>
> Installing a driver in Windows is 'driven' by a .inf file. From my
> (admittedly limited) understanding, this broadly describes:
>
> * The files which need to be installed
> * The hardware the driver is compatible with
>
> The files, including the .inf file itself must be copied in to the
> correct places. In addition, information from the .inf file must be
> written to the registry. It is this last part which causes problems.
>
> From reading documentation, it appears that a driver would normally be
> installed using the SetupCopyOEMInf() library call. I wrote a simple
> wrapper round this and installed it in the Windows PE image, along with
> the VirtIO drivers. I booted into Windows PE and attempted to install
> the driver. As you might expect, the drivers were installed into the
> Windows PE image rather than the guest. I then tried setting
> %systemdrive% and %systemroot% to the guest image. This appears to have
> no effect. This is what makes me suspect that the process is primarily
> registry driven.
>
> I started looking around for ways of using a different registry. I
> discovered the Registry Editor PE plugin to BartPE
> (https://sourceforge.net/projects/regeditpe/) which allows editing the
> registry of a guest. Looking at how it does this, it uses reg.exe to
> load the guests's hives. I confirmed that you can do this.
> Unfortunately
> you don't seem to be able to replace the default hives. The new hives
> are loaded in a different part of the tree, and are therefore ignored.
>
> This is as far as I've got. Still on my list are:
>
> * Ask on various Windows mailing lists how to do this
> * Investigate if the packaged .msi containing the drivers is more
> flexible.
> * Look for other, possibly lower level, ways of replacing making a
> process use a different registry.
>
> Any and all suggestions are gratefully received,
>
> Matt
> --
> Matthew Booth, RHCA, RHCSS
> Red Hat Engineering, Virtualisation Team
>
> M: +44 (0)7977 267231
> GPG ID: D33C3490
> GPG FPR: 3733 612D 2D05 5458 8A8A 1600 3441 EA19 D33C 3490
14 years, 5 months
