5:18 a.m.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming blog: http://rwmj.wordpress.com
Fedora now supports 80 OCaml packages (the OPEN alternative to F#)
http://cocan.org/getting_started_with_ocaml_on_red_hat_and_fedora
6:24 a.m.
New subject: [PATCH v2] New APIs: gzip-out, xz-out, gzip-device-out, xz-device-out.
This updates the existing patch by removing 'compress'. It turns out
this program isn't installed in the appliance, and in any case the
compress format is long obsolete.
Below are some timings. Note that even for plain gzip compression
there is quite a large time penalty. Virtio-serial / the libguestfs
protocol is quite efficient at copying large files around
(75 megabytes/sec, only a fraction slower than 'dd' on the host).
There is no multithread support in gzip or xz at the moment, so adding
SMP support to the appliance probably won't make much difference,
although it's worth adding it anyway.
I might play with snappy and a simple RLE-based compressor next.
Rich.
# Newly created, empty, 100MB ext2 filesystem:
$ ./run ./fish/guestfish -N fs time download /dev/vda1 >(wc -c)
elapsed time: 0.65 seconds
104792576
$ ./run ./fish/guestfish -N fs time gzip-device-out /dev/vda1 >(wc -c)
elapsed time: 1.08 seconds
106799
$ ./run ./fish/guestfish -N fs time xz-device-out /dev/vda1 >(wc -c)
elapsed time: 9.83 seconds
18168
# On a real filesystem that is too large to fit into cache:
# Filesystem Size Used Available Use%
# F16x64:/dev/vg_f16x64/lv_root 5.5G 3.2G 2.2G 59%
$ ./run ./fish/guestfish --ro -a /dev/vg_pin/F16x64 run : time download
/dev/vg_f16x64/lv_root >(wc -c)
elapsed time: 78.74 seconds
5972688896
$ ./run ./fish/guestfish --ro -a /dev/vg_pin/F16x64 run : time gzip-device-out
/dev/vg_f16x64/lv_root >(wc -c)
elapsed time: 274.11 seconds
1927316373
[xz case took more than 10 minutes, I didn't wait for it to finish]
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine. Supports Linux and Windows.
http://et.redhat.com/~rjones/virt-df/
7:51 a.m.
New subject: [PATCH v3] New APIs: compress-out, compress-device-out.
This third version changes the API again to allow you to select both
the compression program (from {compress,gzip,bzip2,xz,lzop}) and
optionally the compression level.
Some more timings are below. 'lzop' is generally much faster than
other methods, although there is still > 50% penalty for using lzop on
a fairly full filesystem.
Interestingly, 'lzop -1' which is supposed to be fast, is slower than
the default level.
Rich.
100 MB empty filesystem.
Straight download was 0.65 seconds.
$ ./run ./fish/guestfish -N fs \
time compress-device-out gzip /dev/vda1 >(wc -c) level:1
elapsed time: 0.92 seconds
462390
$ ./run ./fish/guestfish -N fs \
time compress-device-out gzip /dev/vda1 >(wc -c) level:9
elapsed time: 1.14 seconds
107903
$ ./run ./fish/guestfish -N fs \
time compress-device-out lzop /dev/vda1 >(wc -c)
elapsed time: 0.73 seconds
428023
$ ./run ./fish/guestfish -N fs \
time compress-device-out lzop /dev/vda1 >(wc -c) level:1
elapsed time: 0.84 seconds
428125
$ ./run ./fish/guestfish -N fs \
time compress-device-out bzip2 /dev/vda1 >(wc -c)
elapsed time: 1.80 seconds
2801
$ ./run ./fish/guestfish -N fs \
time compress-device-out xz /dev/vda1 >(wc -c)
elapsed time: 9.17 seconds
18164
Real filesystem image ~5.5GB in size, 59% in use.
Straight download was 78.74 seconds.
$ ./run ./fish/guestfish --ro -a /dev/vg_pin/F16x64 \
run : time compress-device-out lzop /dev/vg_f16x64/lv_root >(wc -c)
elapsed time: 123.17 seconds
2488500746
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
libguestfs lets you edit virtual machines. Supports shell scripting,
bindings from many languages. http://libguestfs.org
8:59 a.m.
New subject: [PATCH v3] New APIs: compress-out, compress-device-out.
On Wed, Sep 28, 2011 at 01:51:18PM +0100, Richard W.M. Jones wrote:
[...]
I have pushed this patch, and a further patch to enable SMP in the
appliance (optional, default is still UP).
Interestingly, enabling more virtual CPUs in the appliance causes a
substantial slowdown in lzop performance. The following test:
Real filesystem image ~5.5GB in size, 59% in use.
Straight download was 78.74 seconds.
$ ./run ./fish/guestfish --ro -a /dev/vg_pin/F16x64 \
run : time compress-device-out lzop /dev/vg_f16x64/lv_root >(wc -c)
elapsed time: 123.17 seconds
2488500746
takes 148 seconds with "qemu-kvm -smp 4".
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
virt-p2v converts physical machines to virtual machines. Boot with a
live CD or over the network (PXE) and turn machines into Xen guests.
http://et.redhat.com/~rjones/virt-p2v
4837
days inactive
4837
days old
3 comments
1 participants
participants (1)
-
Richard W.M. Jones