8:39 a.m.
The nbdkit tests take a long time to run especially on armv7. To try
to get an idea of which tests take the most time, and since I happen
to have a qemu aarch64 *TCG* guest available, I ran the tests on that
guest.
A TCG guest is about the worst case for libguestfs, used by some
tests, since it uses double-nested TCG, and as you can see below there
are tests taking over 6 hours.
In fact I didn't complete the run yet. test-cow.sh is still running
(after 2+ hours) and there may be further tests to come.
Results:
06:43:00 PASS: test-cache.sh
06:37:48 PASS: test-cache-on-read.sh
[ test-cow.sh possibly here ]
00:35:41 PASS: test-partitioning3.sh
00:34:29 PASS: test-partitioning5.sh
00:34:03 PASS: test-linuxdisk-copy-out.sh
00:33:58 PASS: test-floppy.sh
00:30:45 PASS: test-partitioning2.sh
00:28:41 PASS: test-linuxdisk.sh
00:28:34 PASS: test-iso.sh
00:02:59 PASS: test-cache-max-size.sh
00:02:58 PASS: test-partitioning6.sh
00:02:55 PASS: test-partitioning4.sh
00:02:51 PASS: test-eflags.sh
00:01:11 PASS: test-export-name.sh
00:01:01 PASS: test-parallel-sh.sh
00:00:54 PASS: test-python.sh
00:00:44 PASS: test-sh-errors.sh
00:00:44 PASS: test-cacheextents.sh
00:00:41 PASS: test-sh-extents.sh
00:00:35 PASS: test-info-base64.sh
Ignoring the ones which take mere minutes (those run in seconds on
more reasonable hardware), here is a summary of how the tests are
implemented:
test-cache.sh Gx2 F D
test-cache-on-read.sh Gx2 F D
test-cow.sh Gx3 F
test-partitioning3.sh G
test-partitioning5.sh G F
test-linuxdisk-copy-out.sh G Dx4
test-floppy.sh G Dx2
test-partitioning2.sh G F
test-linuxdisk.sh G D
test-iso.sh G
D = using libguestfs download or similar APIs
F = using libguestfs "fill" APIs which write lots of data
G = guestfish
xN = runs multiple times
So there's quite a strong correlation between multiple runs of
guestfish, and using those two APIs, and time taken.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html
2:51 a.m.
I pushed a change which should reduce the time taken in the
test-cache-* and test-cow.sh tests. Again note this is double TCG
which is the worst case for these tests and explains the excessively
long times which wouldn't be seen on a normal build system.
Updated results:
01:04:18 PASS: test-readahead
01:02:32 PASS: test-cow.sh
00:37:25 PASS: test-floppy.sh
00:34:57 PASS: test-shell
00:34:53 PASS: test-linuxdisk-copy-out.sh
00:33:57 PASS: test-partitioning3.sh
00:33:46 PASS: test-perl
00:33:38 PASS: test-partitioning5.sh
00:33:38 PASS: test-memory
00:33:28 PASS: test-ocaml
00:33:11 PASS: test-lua
00:32:33 PASS: test-tcl
00:32:16 PASS: test-offset
00:31:48 PASS: test-xz
00:31:47 PASS: test-xz-curl
00:30:19 PASS: test-partitioning2.sh
00:28:57 PASS: test-linuxdisk.sh
00:28:46 PASS: test-data
00:28:24 PASS: test-gzip
00:27:57 PASS: test-ext2
00:27:49 PASS: test-curl
00:27:16 PASS: test-iso.sh
00:25:56 PASS: test-nbd
[all others < 7 mins]
Analysis of what these tests do as in the previous email:
test-readahead G Fx2
test-cow.sh Gx2
test-floppy.sh G Dx2
test-shell G
test-linuxdisk-copy-out.sh G Dx4
test-partitioning3.sh G
test-perl G
test-partitioning5.sh G F
test-memory G
test-ocaml G
test-lua G
test-tcl G
test-offset G
test-xz G
test-xz-curl G W
test-partitioning2.sh G F
test-linuxdisk.sh G D
test-data G
test-gzip G
test-ext2 G
test-curl G W
test-iso.sh G
test-nbd G
D = using libguestfs download or similar APIs
F = using libguestfs "fill" APIs which write lots of data
G = guestfish/libguestfs
W = curl/webserver
xN = runs multiple times
test-readahead needs more work. test-cow.sh is obdurately difficult
to test. The others are good tests so there's probably not much we
can or should do to them.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines. Supports shell scripting,
bindings from many languages. http://libguestfs.org
9:09 a.m.
I pushed these two patches which together reduce the build time in
Koji from 7 hours down to 2 hours - almost all of that spent on the
armv7 builder, the other architectures take only a few minutes.
https://github.com/libguestfs/nbdkit/commit/fdbd0ef69f0c222d0cdc35d3c4829...
https://github.com/libguestfs/nbdkit/commit/ce8357a883167b75ced16ea9f1488...
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines. Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top
1806
days inactive
1807
days old
2 comments
1 participants
participants (1)
-
Richard W.M. Jones