Re: [Libguestfs] [Qemu-devel] How to emulate block I/O timeout on qemu side?
On 11/2/18 3:11 AM, Dongli Zhang wrote:
Hi,
Is there any way to emulate I/O timeout on qemu side (not fault injection in VM kernel) without modifying qemu source code?
You may be interested in Rich's work on nbdkit. If you don't mind the overhead of the host connecting through NBD, then you can use nbdkit's delay and fault-injection filters for inserting delays or even run-time-controllable failures to investigate how the guest reacts to those situations
For instance, I would like to observe/study/debug the I/O timeout handling of nvme, scsi, virtio-blk (not supported) of VM kernel.
Is there a way to trigger this on purpose on qemu side?
Thank you very much!
Dongli Zhang
-- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
On 11/06/2018 01:49 AM, Eric Blake wrote:
On 11/2/18 3:11 AM, Dongli Zhang wrote:
Hi,
Is there any way to emulate I/O timeout on qemu side (not fault injection in VM kernel) without modifying qemu source code?
You may be interested in Rich's work on nbdkit. If you don't mind the overhead of the host connecting through NBD, then you can use nbdkit's delay and fault-injection filters for inserting delays or even run-time-controllable failures to investigate how the guest reacts to those situations
Thank you all very much for the suggestions. I will take a look on nbdkit. So far I am reproducing the issue with NFS (by shutdown the link to NFS where the image is placed on purpose) but it did not work well.
For instance, I would like to observe/study/debug the I/O timeout handling of nvme, scsi, virtio-blk (not supported) of VM kernel.
Is there a way to trigger this on purpose on qemu side?
Thank you very much!
Dongli Zhang
Dongli Zhang
On Tue, Nov 06, 2018 at 02:17:46PM +0800, Dongli Zhang wrote:
On 11/06/2018 01:49 AM, Eric Blake wrote:
On 11/2/18 3:11 AM, Dongli Zhang wrote:
Hi,
Is there any way to emulate I/O timeout on qemu side (not fault injection in VM kernel) without modifying qemu source code?
You may be interested in Rich's work on nbdkit. If you don't mind the overhead of the host connecting through NBD, then you can use nbdkit's delay and fault-injection filters for inserting delays or even run-time-controllable failures to investigate how the guest reacts to those situations
Thank you all very much for the suggestions. I will take a look on nbdkit.
These links should help: https://rwmj.wordpress.com/2018/09/04/nbdkit-for-loopback-pt-2-injecting-err... https://rwmj.wordpress.com/2018/09/06/nbdkit-for-loopback-pt-7-a-slow-disk/ This link shows how to combine delay and error filters together: https://rwmj.wordpress.com/2018/11/04/nbd-graphical-viewer/ Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW
On Tue, Nov 06, 2018 at 09:14:57AM +0000, Richard W.M. Jones wrote:
This link shows how to combine delay and error filters together:
Oops, that's in a forthcoming blog post not this one. Not enough caffeine this morning. Combining the filters is easy however: nbdkit --filter=error --filter=delay \ memory size=$size \ rdelay=$delay wdelay=$delay \ error-rate=100% error-file=/tmp/error Then touching /tmp/error will inject errors, and removing /tmp/error will stop injecting errors. The documentation says you should be able to write error-rate=1 instead of error-rate=100%, but in fact that was broken until recently, and fixed in: https://github.com/libguestfs/nbdkit/commit/ee2d3b4fea6d4b7618262f85f882374c... Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v
On Tue, Nov 06, 2018 at 09:43:06AM +0000, Richard W.M. Jones wrote:
On Tue, Nov 06, 2018 at 09:14:57AM +0000, Richard W.M. Jones wrote:
This link shows how to combine delay and error filters together:
Oops, that's in a forthcoming blog post not this one. Not enough caffeine this morning.
Up now: https://rwmj.wordpress.com/2018/11/06/nbd-graphical-viewer-raid-5-edition/ Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org
participants (3)
-
Dongli Zhang -
Eric Blake -
Richard W.M. Jones