On Fri, Aug 9, 2019 at 11:20 AM Richard W.M. Jones <rjones@redhat.com> wrote:
On Fri, Aug 09, 2019 at 02:58:18AM +0300, Nir Soffer wrote:
> On Thu, Aug 8, 2019 at 5:05 PM Richard W.M. Jones <rjones@redhat.com> wrote:
>
> > > +                if disk.status == types.DiskStatus.OK:
> > > +                    debug("finalized after %s seconds", time.time() -
> > start)
> >
>
> Maybe this was the real error? (using , instead of %)

I fixed this in a later build.  However the error we're talking about
here seems to be unrelated to it.

...
> > ', '  File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py",
> > line 37334, in remove
> >     self._internal_remove(headers, query, wait)
> > ', '  File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line
> > 271, in _internal_remove
> >     return future.wait() if wait else future
> > ', '  File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line
> > 55, in wait
> >     return self._code(response)
> > ', '  File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line
> > 268, in callback
> >     self._check_fault(response)
> > ', '  File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line
> > 132, in _check_fault
> >     self._raise_error(response, body)
> > ', '  File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line
> > 118, in _raise_error
> >     raise error
> > ', 'Error: Fault reason is "Operation Failed". Fault detail is "[Cannot
> > remove Virtual Disk. Related operation is currently in progress. Please try
> > again later.]". HTTP response code is 409.
> > ']
> >
>
> I think this error means that the disk is currently busy, probably because
> of the real error.
>
> Ilanit, can you share complete virt-v2v and daemon logs?

These are quite large so I uploaded them here:

http://oirase.annexia.org/tmp/v2v-import-20190808T125134-439694.log

I think what we see here is that upload was successful:

nbdkit: python[1]: debug: flush
nbdkit: python[1]: debug: flush
nbdkit: python[1]: debug: client sent disconnect command, closing connection
nbdkit: python[1]: debug: close
...
virtual copying rate: 649.6 M bits/sec
...
[1729.6] Creating output metadata
virt-v2v: error: transfer of disk 1/1 failed, see earlier error messages
...
nbdkit: python[1]: error: /var/tmp/v2v.sZm0my/rhv-upload-plugin.py: close: error: ['Traceback (most recent call last):\n', ' File "/var/tmp/v2v.sZm0my/rhv-upload-plugin.py", line 577, in close\n', ' File "/var/tmp/v2v.sZm0my/rhv-upload-plugin.py", line 516, in delete_disk_on_failure\n', ' File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line 37334, in remove\n self._internal_remove(headers, query, wait)\n', ' File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 271, in _internal_remove\n return future.wait() if wait else future\n', ' File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 55, in wait\n return self._code(response)\n', ' File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 268, in callback\n self._check_fault(response)\n', ' File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 132, in _check_fault\n self._raise_error(response, body)\n', ' File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 118, in _raise_error\n raise error\n', 'Error: Fault reason is "Operation Failed". Fault detail is "[Cannot remove Virtual Disk. Related operation is currently in progress. Please try again later.]". HTTP response code is 409.\n']

But we had a bug in the logging code:

    debug("fmt", value)

This raised when we discovered that the upload was successful, and we tried to cleanup
after errors by deleting the disk. This failed because the disk is still being used by engine
at this point.

Base on what Ilanit wrote in the other thread, after fixing the debug() call this works.
 

http://oirase.annexia.org/tmp/v2v-import-20190808T125134-439694-wrapper.log

Rich.

--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html