On Thu, Mar 29, 2018 at 3:07 PM Richard W.M. Jones <rjones@redhat.com> wrote:
And another problem ...

nbdkit: python[1]: error: /home/rjones/d/libguestfs/tmp/rhvupload.riX9kG/rhv-upload-plugin.py: callback failed: pwrite
Traceback (most recent call last):
  File "/home/rjones/d/libguestfs/tmp/rhvupload.riX9kG/rhv-upload-plugin.py", line 268, in pwrite
  File "/usr/lib64/python3.6/http/client.py", line 986, in send
  File "/usr/lib64/python3.6/ssl.py", line 965, in sendall
    v = self.send(data[count:])
  File "/usr/lib64/python3.6/ssl.py", line 935, in send
    return self._sslobj.write(data)
  File "/usr/lib64/python3.6/ssl.py", line 636, in write
    return self._sslobj.write(data)
BrokenPipeError: [Errno 32] Broken pipe
qemu-img: error while writing sector 25343488: Input/output error

On the server side we see it's something to do with the ticket

2018-03-29 12:59:24,215 INFO    (Thread-18701) [web] START [] PUT /images/4e8567f6-939d-471f-8ea8-e64d172c0a26
2018-03-29 12:59:24,216 INFO    (Thread-18701) [images] Writing 458752 bytes at offset 12975407104 to /rhev/data-center/mnt/ham0.home.annexia.org:_mnt_ovirt-data/7b4ae32d-bc79-4368-b683-3e0b58b64bf6/images/b4c31e0c-16c0-47d0-a162-9db4357a2dc5/89d70696-2075-4d30-b30e-86465060f677 for ticket 4e8567f6-939d-471f-8ea8-e64d172c0a26
2018-03-29 12:59:24,272 INFO    (Thread-18701) [directio] Operation stats: <Clock(total=0.06, read=0.04, write=0.01, sync=0.00)>
2018-03-29 12:59:24,272 INFO    (Thread-18701) [web] FINISH [] PUT /images/4e8567f6-939d-471f-8ea8-e64d172c0a26: [200] 0 (0.05s)
2018-03-29 12:59:24,296 INFO    (Thread-18702) [web] START [] PUT /images/4e8567f6-939d-471f-8ea8-e64d172c0a26
2018-03-29 12:59:24,296 WARNING (Thread-18702) [web] ERROR [] PUT /images/4e8567f6-939d-471f-8ea8-e64d172c0a26: [403] Ticket u'4e8567f6-939d-471f-8ea8-e64d172c0a26' expired (0.00s)

We had suspected this before but now I think I've proven that
it's a problem.

I believe the timeout was after 30 minutes (but we were writing
constantly the whole time).  Is this a bug in imageio?

The ticket has expired. Engine is monitoring download/upload activity for tickets
and extend the ticket when needed. We need complete daemon and engine logs
to check why the ticket was not extended.