On Wed, Nov 20, 2019 at 1:19 AM Nir Soffer <nirsof@gmail.com> wrote:
Fix NameError introduced by inocomplete change to extract create_transfer() by extracting cancel_transfer() function.
Finally extract finallize_transfer() function, dealing with the poorly documented and over complicated oVirt SDK.
Tested with: libguestfs-1.40.2-4.fc30.x86_64 nbdkit-1.12.8-1.fc30.x86_64
Tested flows: - Successful import - Error in close() - by injecting long sleep in vdsm verifaction step - Error in close() - by injecting error in vdsm verification step - Error in open() - by injecting error in imageio OPTIONS handler - Error in zero() - by injecting error in imageio PATCH handler - Error in flush() - by injecting error in imageio PATCH handler
Here is output from the tests runs: ## Simulate verification error by injecting fake error in vdsm. ... nbdkit: python[1]: debug: client sent NBD_CMD_DISC, closing connection nbdkit: python[1]: debug: close finalizing transfer 01aecf9a-2629-40e4-8528-f310a4ad6c82 virtual copying rate: 3368.6 M bits/sec canceling transfer 01aecf9a-2629-40e4-8528-f310a4ad6c82 nbdkit: python[1]: error: /home/nsoffer/src/virt-v2v/tmp/rhvupload.SZBNp8/rhv-upload-plugin.py: close: error: ['Traceback (most recent call last):\n', ' File "/home/nsoffer/src/virt-v2v/tmp/rhvupload.SZBNp8/rhv-upload-plugin.py", line 332, in close\n finalize_transfer(connection, transfer, disk_id)\n', ' File "/home/nsoffer/src/virt-v2v/tmp/rhvupload.SZBNp8/rhv-upload-plugin.py", line 561, in finalize_transfer\n "transfer %s failed: disk is ILLEGAL" % transfer.id)\n', 'RuntimeError: transfer 01aecf9a-2629-40e4-8528-f310a4ad6c82 failed: disk is ILLEGAL\n'] ## Simulate inaccessible storage by adding long sleep in vdsm. ... nbdkit: python[1]: debug: client sent NBD_CMD_DISC, closing connection nbdkit: python[1]: debug: close finalizing transfer c05c578e-df99-4fb5-80f0-6944051dbee0 virtual copying rate: 2281.9 M bits/sec canceling transfer c05c578e-df99-4fb5-80f0-6944051dbee0 nbdkit: python[1]: error: /home/nsoffer/src/virt-v2v/tmp/rhvupload.YKpW3U/rhv-upload-plugin.py: close: error: ['Traceback (most recent call last):\n', ' File "/home/nsoffer/src/virt-v2v/tmp/rhvupload.YKpW3U/rhv-upload-plugin.py", line 332, in close\n finalize_transfer(connection, transfer, disk_id)\n', ' File "/home/nsoffer/src/virt-v2v/tmp/rhvupload.YKpW3U/rhv-upload-plugin.py", line 561, in finalize_transfer\n "transfer %s failed: disk is ILLEGAL" % transfer.id)\n', 'RuntimeError: transfer c05c578e-df99-4fb5-80f0-6944051dbee0 failed: disk is ILLEGAL\n'] ## Simulate error after transfer was started by returning invalid json in imageio OPTIONS handler ... disk.id = 'a2bad783-0736-42a5-92d1-d76d133453bf' transfer.id = 'fd297a0d-1c9b-49ac-9104-829358127f23' canceling transfer fd297a0d-1c9b-49ac-9104-829358127f23 nbdkit: python[1]: error: /home/nsoffer/src/virt-v2v/tmp/rhvupload.nejhOD/rhv-upload-plugin.py: open: error: ['Traceback (most recent call last):\n', ' File "/home/nsoffer/src/virt-v2v/tmp/rhvupload.nejhOD/rhv-upload-plugin.py", line 94, in open\n options = get_options(http, destination_url)\n', ' File "/home/nsoffer/src/virt-v2v/tmp/rhvupload.nejhOD/rhv-upload-plugin.py", line 618, in get_options\n j = json.loads(data)\n', ' File "/usr/lib64/python3.7/json/__init__.py", line 348, in loads\n return _default_decoder.decode(s)\n', ' File "/usr/lib64/python3.7/json/decoder.py", line 337, in decode\n obj, end = self.raw_decode(s, idx=_w(s, 0).end())\n', ' File "/usr/lib64/python3.7/json/decoder.py", line 355, in raw_decode\n raise JSONDecodeError("Expecting value", s, err.value) from None\n', 'json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)\n'] qemu-img: Could not open 'json:{ "file.driver": "nbd", "file.path": "/home/nsoffer/src/virt-v2v/tmp/rhvupload.nejhOD/nbdkit0.sock", "file.export": "/" }': Failed to read option reply: Unexpected end-of-file before all bytes were read ## Simulate error in zero by injecting error in imageio PATCH/zero ... nbdkit: python[1]: debug: zero count=458752 offset=6441926656 may_trim=1 fua=0 unexpected response from imageio server: could not zero sector offset 6441926656 size 458752 500: Internal Server Error b'Server failed to perform the request, check logs' nbdkit: python[1]: error: /home/nsoffer/src/virt-v2v/tmp/rhvupload.3rv9Nk/rhv-upload-plugin.py: zero: error: ['Traceback (most recent call last):\n', ' File "/home/nsoffer/src/virt-v2v/tmp/rhvupload.3rv9Nk/rhv-upload-plugin.py", line 229, in zero\n (offset, count))\n', ' File "/home/nsoffer/src/virt-v2v/tmp/rhvupload.3rv9Nk/rhv-upload-plugin.py", line 149, in request_failed\n raise RuntimeError("%s: %d %s: %r" % (msg, status, reason, body[:200]))\n', "RuntimeError: could not zero sector offset 6441926656 size 458752: 500 Internal Server Error: b'Server failed to perform the request, check logs'\n"] nbdkit: python[1]: debug: sending error reply: Input/output error nbdkit: python[1]: debug: pwrite count=458752 offset=6441926656 fua=0 nbdkit: python[1]: debug: pwrite count=65536 offset=6442385408 fua=0 (100.00/100%) nbdkit: python[1]: debug: flush nbdkit: python[1]: debug: flush nbdkit: python[1]: debug: client sent NBD_CMD_DISC, closing connection nbdkit: python[1]: debug: close canceling transfer 86af4ae7-e1dc-4514-ad95-d7587678f946 virtual copying rate: 1213.8 M bits/sec ## Simulate error in flush by injecting error in imageio PATCH/flush ... nbdkit: python[1]: debug: pwrite count=65536 offset=6442385408 fua=0 (100.00/100%) nbdkit: python[1]: debug: flush unexpected response from imageio server: could not flush 500: Internal Server Error b'Server failed to perform the request, check logs' nbdkit: python[1]: error: /home/nsoffer/src/virt-v2v/tmp/rhvupload.vPoylv/rhv-upload-plugin.py: flush: error: ['Traceback (most recent call last):\n', ' File "/home/nsoffer/src/virt-v2v/tmp/rhvupload.vPoylv/rhv-upload-plugin.py", line 302, in flush\n request_failed(h, r, "could not flush")\n', ' File "/home/nsoffer/src/virt-v2v/tmp/rhvupload.vPoylv/rhv-upload-plugin.py", line 149, in request_failed\n raise RuntimeError("%s: %d %s: %r" % (msg, status, reason, body[:200]))\n', "RuntimeError: could not flush: 500 Internal Server Error: b'Server failed to perform the request, check logs'\n"] nbdkit: python[1]: debug: sending error reply: Input/output error nbdkit: python[1]: debug: flush unexpected response from imageio server: could not flush 500: Internal Server Error b'Server failed to perform the request, check logs' nbdkit: python[1]: error: /home/nsoffer/src/virt-v2v/tmp/rhvupload.vPoylv/rhv-upload-plugin.py: flush: error: ['Traceback (most recent call last):\n', ' File "/home/nsoffer/src/virt-v2v/tmp/rhvupload.vPoylv/rhv-upload-plugin.py", line 302, in flush\n request_failed(h, r, "could not flush")\n', ' File "/home/nsoffer/src/virt-v2v/tmp/rhvupload.vPoylv/rhv-upload-plugin.py", line 149, in request_failed\n raise RuntimeError("%s: %d %s: %r" % (msg, status, reason, body[:200]))\n', "RuntimeError: could not flush: 500 Internal Server Error: b'Server failed to perform the request, check logs'\n"] nbdkit: python[1]: debug: sending error reply: Input/output error nbdkit: python[1]: debug: client sent NBD_CMD_DISC, closing connection nbdkit: python[1]virtual copying rate: 2227.0 M bits/sec : debug: close canceling transfer 6508e39c-6956-4333-8f27-1f2b55377986 virt-v2v: error: transfer of disk 1/1 failed, see earlier error messages Nir