Re: [Libguestfs] [PATCH nbdkit] file: Implement cache=none and fadvise=normal|random|sequential.

The setup is an 8G file called /var/tmp/random8 of random data which is cached. Then we create a new sparse file called /var/tmp/out and copy random8 -> out. With default settings: $ rm -f /var/tmp/out; truncate -s 8G /var/tmp/out; free -m; time ./nbdkit file /var/tmp/out --run 'qemu-img convert -n -p -m 16 -W /var/tmp/random8 $nbd'; free -m ; cachestats /var/tmp/out total used free shared buff/cache available Mem: 32083 1181 6730 1 24171 30439 Swap: 16135 16 16119 (100.00/100%) real 0m6.267s user 0m0.229s sys 0m12.904s total used free shared buff/cache available Mem: 32083 1184 966 1 29932 30444 Swap: 16135 16 16119 pages in cache: 2038342/2097152 (97.2%) [filesize=8388608.0K, pagesize=4K] Notice after the run the new file is almost entirely in memory. With fadvise=sequential cache=none: $ rm -f /var/tmp/out; truncate -s 8G /var/tmp/out; free -m; time ./nbdkit file /var/tmp/out fadvise=sequential cache=none --run 'qemu-img convert -n -p -m 16 -W /var/tmp/random8 $nbd'; free -m ; cachestats /var/tmp/out total used free shared buff/cache available Mem: 32083 1178 6737 1 24167 30442 Swap: 16135 16 16119 (100.00/100%) real 0m8.791s user 0m0.361s sys 0m29.766s total used free shared buff/cache available Mem: 32083 1182 6716 1 24184 30438 Swap: 16135 16 16119 pages in cache: 3075/2097152 (0.1%) [filesize=8388608.0K, pagesize=4K] The overhead is large - about 40% - which I think is to be expected given how we're doing this. But it does end up with almost none of the file in the cache. So it works. I think there's room for improvement here by using double buffering or a background thread or something like that. Hard to implement this in the current plugin. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org