Re: [Libguestfs] Regression testing inspection code

On Mon, 2013-12-02 at 13:23 +0000, Richard W.M. Jones wrote: > [Continuing a discussion we had on IRC last week ...] > > How to create meaningful regression tests for the inspection code? > We'd like developers to be able to test that changes to the inspection > code don't break inspection of existing guests. We might not be able > to run these tests routinely, and we might have to accept that they > have to download hundreds of megabytes of test data. > > > Idea #1: Run virt-inspector on cloud images > ---------------------------------------------------------------------- > > eg: > > virt-inspector -a http://download.fedoraproject.org/pub/fedora/linux/releases/19/Images/x86... > > Unfortunately this doesn't work for a couple of reasons. Firstly the > Curl driver in qemu is broken (RHBZ#971790). Secondly even when I > implemented a workaround, the command above doesn't work reliably when > connecting to a slow mirror, because of SCSI timeouts inside the > appliance. > > > Idea #2: Make "reduced" test images available > ---------------------------------------------------------------------- > > Delete bits of real test images so they are small enough to > distribute. Inspection doesn't look at the vast majority of files in > a test image. The hard bit is working out which bits inspection > *does* need to look at, either now or in the future. > > I wrote a small program to test this. Unfortunately it doesn't reduce > images that much. We'd still be talking about ~50-100MB to host + > distribute per test case, and we'd ideally want >20 test cases so > that's still multiple gigabytes of test data. > > I've attached the program to this email in case anyone wants to look > at this further. > > > Any other ideas ...? guestconv builds several RHEL and Fedora based images for automated testing. The build process is separate from the test process, and tests are skipped if their image hasn't been built. It's very flexible, and works for guests we can build automatically.