From IRC channel:

<StenaviN> Can someone confirm cat/test-virt-tail.sh works in 'master'?

<StenaviN> I get https://pastebin.com/GBkg7Vtw

<rwmjones> StenaviN: yes it works for me; the error is not very helpful, you'll need to set LIBGUESTFS_DEBUG=1 LIBGUESTFS_TRACE=1

<StenaviN> https://pastebin.com/yABgCHwV

<rwmjones> I think the error is:

<rwmjones> libguestfs: trace: statns "/tail"

<rwmjones> guestfsd: => mount_options (0x4a) took 0.00 secs

<rwmjones> guestfsd: <= statns (0x1a5) request length 52 bytes

<rwmjones> [    0.930738] EXT2-fs (sda1): error: ext2_lookup: deleted inode referenced: 12

<rwmjones> guestfsd: error: /tail: Input/output error

<rwmjones> but I don't know exactly why

<StenaviN> Yes, I see. Trying to figure out...

<rwmjones> actually no, that's not the problem

<rwmjones> for some reason two instances of ‘guestfish --remote exit’ run at the same time, but according to the test script only one should run

<rwmjones> notice how the cleanup() function is called twice

<rwmjones> afaik that should never happen

<StenaviN> and I saw two qemu/guestfish processes running. Continue investigating...

And here is what I found:

Second copy of 'guestfish --listen' process is a child "recovery process" (https://github.com/libguestfs/libguestfs/blob/master/lib/launch-direct.c#L777) and that is OK.

'cleanup' was called twice because:

1. call to virt-tail returns non-zero exit code (due to Input/Output error. About this later.) and we trap ERR signal which cause to run 'cleanup' once

2. in 'cleanup' we do 'exit $statuscode' and we trap EXIT and 'cleanup' is called once again

It might look confusing but not end of the life. At least there is an explanation if I didn't miss something.

Now about failing test case with virt-tail.

Jumping ahead, adding extra 'guestfish --remote sync' after 'guestfish --remote rm /tail' in 'cat/test-virt-tail.sh' fixes the test case.

virt-tail re-creates overlay image each time it trying to access the file and calls guestfs_statns for the file(s) it watching.

guestfs_statns in turn returns NULL indicating an error with exit code EIO instead of ENOENT: 'EXT2-fs (sda1): error: ext2_lookup: deleted inode referenced: 12'. (see pastebin posted in discussion above).

So I suspect that changes on original disk made through 'guestfish --remote rm /tail' call were not fully flushed which confirms by proposed patch.

It is hard to explain why it works on your system but it might be because of number of factors:

1. Different QEMU caching policy

2. Different caching policy of underlying OS/filesytem

3. etc.

--
    Nikolay Ivanets
    Mobile: +380979184774
    Skype: n_ivanets