9:49 p.m.
Hello,
I am running into problems with virt-builder on my Ubuntu-14.04 install.
Please find the checklist below:
1. What are you trying to achieve?
I'm trying to build a standard Ubuntu 14.04 disk image.
2. What exact commands did you run?
I ran "virt-builder ubuntu-14.04"
3. What was the precise error/output of these commands?
The process gets stuck indefinitely at "Opening the new disk." I even
left it over night and it did not get any farther.
4. Enable debugging, run the commands again, and capture the complete
output.
The file attached contains the output from the command with debugging
enabled.
5. Versions of installed software:
libguestfs: 1.28.10 compiled from source tarball but not installed
supermin: 5.1.13 compiled from source tarball but not installed, used
instructions from
https://rwmj.wordpress.com/2014/03/08/tip-old-supermin-new-libguestfs-and...
to compile libguestfs with supermin5
operating system: Ubuntu 14.04.2 x64
kernel: 3.13.0-55-generic #94-Ubuntu SMP Thu Jun 18 00:27:10 UTC 2015
x86_64 x86_64 x86_64 GNU/Linux
Additional information:
This is a new issue, I used to use libguestfs 1.28.6 (compiled from source)
and had no issues in the past until this weekend. So I compiled the newest
version (1.28.10) and still have this issue.
Additionally, I tried compiling and running it on a virtual machine (Ubuntu
14.04 image created months ago with virt-builder when it was working) and
that works as expected. Therefore I believe that it is an issue with
something on my host, but I can't seem to narrow it down.
I can still use other libguestfs tools such as virt-df, and virt-inspector
(those are the only two that I've tried)
Any help is appreciated.
Thanks,
Shawn
3:06 a.m.
On Mon, Jun 29, 2015 at 08:49:26PM -0600, Shawn Greig wrote:
[...]
Thanks for the detailed bug report.
The issue is indeed something strange with your host or KVM or your
host kernel (3.13.0-55-generic). As you can see from the end of the
messages:
[ 0.371018] EXT4-fs (sdb): mounting ext2 file system using the
ext4 subsystem
[ 0.393840] EXT4-fs (sdb): mounted filesystem without journal. Opts:
supermin: chroot
Starting /init script ...
ln: failed to create symbolic link '/etc/mtab': File exists
/init: 55: /init: systemd-tmpfiles: not found
[ 1.687079] systemd-udevd[91]: starting version 204
[ 1.808148] random: nonblocking pool is initialized
[ 1.886029] ACPI Exception: AE_BAD_PARAMETER, Thread 487102464 could not acquire Mutex
[0x1] (20131115/utmutex-285)
[ 1.886591] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0xb100, revision 0
/init: 79: /init: cannot create /sys/block/{h,s,ub,v}d*/queue/scheduler: Directory
nonexistent
/sbin/dhclient-script: line 31: /etc/fstab: No such file or directory
[ 2.935872] input: ImExPS/2 Generic Explorer Mouse as
/devices/platform/i8042/serio1/input/input3
... the kernel boots quite far, and then hangs in the init script (see
appliance/init).
Some things you can try:
- Force qemu to use TCG (no KVM) mode to see if that makes a difference:
export LIBGUESTFS_BACKEND_SETTINGS=force_tcg
- Use 'libguestfs-test-tool' for testing to see if that makes a
difference. In the build directory, run:
make quickcheck
- Try a different appliance kernel by setting these variables, but
choose different kernel versions by looking in your /boot directory
first:
export SUPERMIN_KERNEL_VERSION=3.13.0-55-generic
export SUPERMIN_KERNEL=/boot/vmlinuz-$SUPERMIN_KERNEL_VERSION
export SUPERMIN_MODULES=/lib/modules/$SUPERMIN_KERNEL_VERSION
rm -rf tmp/.guestfs-*
make quickcheck
- Add debugging 'set -x' to the top of appliance/init in the source
tree to see if it fails while running a specific command.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines. Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top
10:07 a.m.
- I tried forcing TCG and it gets to the same spot and stops.
- running make quickcheck passes the tests (see attached)
- I tried a different kernel (3.13.0-49-generic) and again it passes the
quickcheck but virt-builder hangs.
- I added 'set -x' to the top of the init script and I don't see any
additional output (see attached)
Shawn
On Tue, Jun 30, 2015 at 2:06 AM, Richard W.M. Jones <rjones(a)redhat.com>
wrote:
On Mon, Jun 29, 2015 at 08:49:26PM -0600, Shawn Greig wrote:
[...]
Thanks for the detailed bug report.
The issue is indeed something strange with your host or KVM or your
host kernel (3.13.0-55-generic). As you can see from the end of the
messages:
> [ 0.371018] EXT4-fs (sdb): mounting ext2 file system using the ext4
subsystem
> [ 0.393840] EXT4-fs (sdb): mounted filesystem without journal. Opts:
> supermin: chroot
> Starting /init script ...
> ln: failed to create symbolic link '/etc/mtab': File exists
> /init: 55: /init: systemd-tmpfiles: not found
> [ 1.687079] systemd-udevd[91]: starting version 204
> [ 1.808148] random: nonblocking pool is initialized
> [ 1.886029] ACPI Exception: AE_BAD_PARAMETER, Thread 487102464 could
not acquire Mutex [0x1] (20131115/utmutex-285)
> [ 1.886591] piix4_smbus 0000:00:01.3: SMBus Host Controller at
0xb100, revision 0
> /init: 79: /init: cannot create /sys/block/{h,s,ub,v}d*/queue/scheduler:
Directory nonexistent
> /sbin/dhclient-script: line 31: /etc/fstab: No such file or directory
> [ 2.935872] input: ImExPS/2 Generic Explorer Mouse as
/devices/platform/i8042/serio1/input/input3
... the kernel boots quite far, and then hangs in the init script (see
appliance/init).
Some things you can try:
- Force qemu to use TCG (no KVM) mode to see if that makes a difference:
export LIBGUESTFS_BACKEND_SETTINGS=force_tcg
- Use 'libguestfs-test-tool' for testing to see if that makes a
difference. In the build directory, run:
make quickcheck
- Try a different appliance kernel by setting these variables, but
choose different kernel versions by looking in your /boot directory
first:
export SUPERMIN_KERNEL_VERSION=3.13.0-55-generic
export SUPERMIN_KERNEL=/boot/vmlinuz-$SUPERMIN_KERNEL_VERSION
export SUPERMIN_MODULES=/lib/modules/$SUPERMIN_KERNEL_VERSION
rm -rf tmp/.guestfs-*
make quickcheck
- Add debugging 'set -x' to the top of appliance/init in the source
tree to see if it fails while running a specific command.
Rich.
--
Richard Jones, Virtualization Group, Red Hat
http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines. Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top
10:17 a.m.
On Tue, Jun 30, 2015 at 09:07:25AM -0600, Shawn Greig wrote:
- I tried forcing TCG and it gets to the same spot and stops.
That excludes the possibility that it's a KVM or kernel bug
(probably).
- running make quickcheck passes the tests (see attached)
- I tried a different kernel (3.13.0-49-generic) and again it passes the
quickcheck but virt-builder hangs.
Add 'virt-builder -v -x' flags and see where it's hanging.
- I added 'set -x' to the top of the init script and I
don't see any
additional output (see attached)
Somehow this didn't have any effect. You probably didn't do a 'make'
before the 'make quickcheck'. It would still be useful to see the
'set -x' output to see if a particular command is hanging.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine. Supports Linux and Windows.
http://people.redhat.com/~rjones/virt-df/
10:41 a.m.
Rich you're right, I forgot to run make. I added 'set -x' and reran make
and then ran 'virt-builder -v -x' and it seems to hang at dhclient (see
attached log).
Shawn
On Tue, Jun 30, 2015 at 9:17 AM, Richard W.M. Jones <rjones(a)redhat.com>
wrote:
On Tue, Jun 30, 2015 at 09:07:25AM -0600, Shawn Greig wrote:
> - I tried forcing TCG and it gets to the same spot and stops.
That excludes the possibility that it's a KVM or kernel bug
(probably).
> - running make quickcheck passes the tests (see attached)
>
> - I tried a different kernel (3.13.0-49-generic) and again it passes the
> quickcheck but virt-builder hangs.
Add 'virt-builder -v -x' flags and see where it's hanging.
> - I added 'set -x' to the top of the init script and I don't see any
> additional output (see attached)
Somehow this didn't have any effect. You probably didn't do a 'make'
before the 'make quickcheck'. It would still be useful to see the
'set -x' output to see if a particular command is hanging.
Rich.
--
Richard Jones, Virtualization Group, Red Hat
http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine. Supports Linux and Windows.
http://people.redhat.com/~rjones/virt-df/
10:53 a.m.
On Tue, Jun 30, 2015 at 09:41:45AM -0600, Shawn Greig wrote:
Rich you're right, I forgot to run make. I added 'set -x'
and reran make
and then ran 'virt-builder -v -x' and it seems to hang at dhclient (see
attached log).
+ dhclient --version
+ dhclient
/sbin/dhclient-script: line 31: /etc/fstab: No such file or directory
Yup, dhclient has been an ongoing problem. Most recently:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=775514
https://bugzilla.redhat.com/show_bug.cgi?id=1224795
It may help to run dhclient in the appliance under strace to see where
exactly it has stopped. The appliance should contain strace (check
for it in appliance/packagelist.in and appliance/supermin.d/packages),
and if it does you can just edit the appliance/init script like this:
- dhclient
+ strace dhclient
The other option is to use dhcpcd, but that also requires patching the
init script.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW
12:50 p.m.
Yeah, it's definitely an issue with dhclient. I patched the init script to
use dhcpcd instead and everything is working as expected now.
I've attached a log containing the strace of dhclient if that helps you or
someone else in the future.
Thanks for all your help Rich.
Shawn
On Tue, Jun 30, 2015 at 9:53 AM, Richard W.M. Jones <rjones(a)redhat.com>
wrote:
On Tue, Jun 30, 2015 at 09:41:45AM -0600, Shawn Greig wrote:
> Rich you're right, I forgot to run make. I added 'set -x' and reran
make
> and then ran 'virt-builder -v -x' and it seems to hang at dhclient (see
> attached log).
> + dhclient --version
> + dhclient
> /sbin/dhclient-script: line 31: /etc/fstab: No such file or directory
Yup, dhclient has been an ongoing problem. Most recently:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=775514
https://bugzilla.redhat.com/show_bug.cgi?id=1224795
It may help to run dhclient in the appliance under strace to see where
exactly it has stopped. The appliance should contain strace (check
for it in appliance/packagelist.in and appliance/supermin.d/packages),
and if it does you can just edit the appliance/init script like this:
- dhclient
+ strace dhclient
The other option is to use dhcpcd, but that also requires patching the
init script.
Rich.
--
Richard Jones, Virtualization Group, Red Hat
http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW
1:03 p.m.
On Tue, Jun 30, 2015 at 11:50:57AM -0600, Shawn Greig wrote:
Yeah, it's definitely an issue with dhclient. I patched the init
script to
use dhcpcd instead and everything is working as expected now.
/sbin/dhclient-script: line 31: /etc/fstab: No such file or directory
So it seems to actually hang when /etc/fstab isn't present. Depending
on the base OS, the libguestfs appliance may or may not have
/etc/fstab -- it's not really required.
It's a bug in whatever provides dhclient-script if dhclient-script
hangs when this file is not present.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html
3463
days inactive
3463
days old
7 comments
2 participants
participants (2)
-
Richard W.M. Jones -
Shawn Greig