Re: [Libguestfs] Thoughts on nbdkit automatic reconnection
On Thu, Sep 19, 2019 at 01:42:25PM +0200, Nenad Peric wrote:
I have an update on the networking issue:
- After the deep dive into the logs of the firewall by customer's security
team, it turns out that even though there were some disconnections, the
time-stamps do not match.
I wouldn't necessarily expect them to match, although it would depend
on how exactly the disconnect happens. For example:
- Firewall sends TCP FIN packets to both ends:
=> Our client would immediately respond to this, so timestamps
should be very close (within seconds).
- Firewall drops the entry in its connections table but does not send
anything to either side:
=> Our client which is sending TCP keepalives (or rather, the
Linux kernel is) would notice the next time it sends a
keepalive, which would be bounced back and cause the connection
to close.
=> Delay could be as long as the keepalive period, so that depends
on a bunch of stuff such as VDDK client settings and kernel
settings.
This means that we got the disconnected by something else (ESXi or
conversion host perhaps)
- As we mentioned in the chat briefly, there could be general keep-alive
issues on both RHEL (conversion host) and ESXi side.
We changed the keep-alive settings in RHEL, but could not find the
equvalent in VMware as of yet.
- I found on a few spots that there are some vddk (vixDiskLib.nfc*)
settings which can configure NFC keep-alives and timeouts, but I do not
understand it deeply enough to see if anything would help.
Yes there's a VDDK setting (vixDiskLib.nfc.ReadTimeoutMs). We don't
set it at the moment so AIUI it should default to between 6 and 45
seconds depending on the version of VDDK.
Whatever may be the cause, a retry filter would most likely solve
the
problem.
Yes I think so too.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine. Supports Linux and Windows.
http://people.redhat.com/~rjones/virt-df/