Re: [Libguestfs] [PATCH libnbd v3 3/3] copy: Evict pages from the page cache when writing to local files.
On Thu, Feb 25, 2021 at 03:48:19PM -0600, Eric Blake wrote:
On 2/25/21 11:34 AM, Richard W.M. Jones wrote:
> When writing to a file or block device, we are always writing new
> (ie. previously uncached) data. This commit ensures that very little
> of that data will be in the page cache after nbdcopy finishes by
> evicting it as we go along. This ensures that the host page cache is
> largely unchanged for other host processes.
>
> This uses Linus's technique described here:
> https://stackoverflow.com/a/3756466
> but instead of using 2 windows, it uses a configurable larger number
> of windows (in this case 8).
Here you state configurable...
I mean configurable in a figurative sense that we can configure it by
adjusting the #define :-)
It's tricky to implement this using the interfaces available. Ideally
the kernel would offer an interface to write the page cache out and
evict it afterwards, but AFAICT there is no such interface.
I suspect that is because we end up waiting longer for flushing
actions
to complete before evicting things from cache. Do we want this to be an
opt-in/out knob on the command line? If so, which way should we lean
for the default value of that knob?
This is the right reason. If we make the window depth configurable
the problem is what would an end-user set this knob to? It's affected
by things like the speed and contention on the local disk.
We might allow end users to opt out completely, in the case where they
know they will consume the file immediately afterwards, and RAM > size
of file.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines. Supports shell scripting,
bindings from many languages. http://libguestfs.org