OK.  I hacked some extra eprintfs into febootstrap.ml (learning a bit of ocaml in the process).  I can confirm that the unpacked debian file has restrictive permissions:

tmp/febootstrape193191fefe36c245623bc48e5b150a2.tmp/bash_4.1-2ubuntu3_amd64.deb.d$ ls -l bin
total 920
-rwxr-x--- 1 dek redacted 934336 Apr 18  2010 bash
lrwxrwxrwx 1 dek redacted      4 Aug 10 11:43 rbash -> bash
lrwxrwxrwx 1 dek redacted      4 Aug 10 11:43 sh -> bash

which is what is causing the permission check in febootstrap.ml to fail:
      else if mode land 0o004 = 0 then
        warn_unreadable := path :: !warn_unreadable

It looks like when I extract a package using dpkg-deb directly, it has permissive permissions,

$ dpkg-deb --extract /tmp/febootstrap02946e2e929e78a0efa20670d79db1ab.tmp/bash_4.1-2ubuntu3_amd64.deb /tmp/shoop
$ ls -l /tmp/shoop
total 920
-rwxr-xr-x 1 dek redacted 934336 Apr 18  2010 bash
lrwxrwxrwx 1 dek redacted      4 Aug 10 09:39 rbash -> bash
lrwxrwxrwx 1 dek redacted      4 Aug 10 09:39 sh -> bash


but, febootstrap runs this command:
dpkg-deb --fsys-tarfile /tmp/febootstrape193191fefe36c245623bc48e5b150a2.tmp/bash_4.1-2ubuntu3_amd64.deb | (cd /tmp/febootstrape193191fefe36c245623bc48e5b150a2.tmp/bash_4.1-2ubuntu3_amd64.deb.d && tar xf -

which creates a tar file that has this:
-rwxr-xr-x root/root    934336 2010-04-18 19:16 ./bin/bash

which unpacks to:
-rwxr-x--- 1 dek redacted 934336 Apr 18  2010 bash

I think this was happening due to my umask being set to '0027'.  I set it to 0000, and I don't get the warning any more, and the generated appliance has a /bin/bash.

Sorry about the noise.  Maybe febootstrap should set a reasonable umask internally?




On Wed, Aug 10, 2011 at 11:27 AM, David Konerding <dek@konerding.com> wrote:


On Wed, Aug 10, 2011 at 10:46 AM, Richard W.M. Jones <rjones@redhat.com> wrote:
On Wed, Aug 10, 2011 at 09:42:10AM -0700, David Konerding wrote:
> BTW, it's actually a list of ~8000 files (looks like all the unpacked files
> from all the .debs that got downloadeD).
>
> I stopped feboostrap during its download after it downloaded the .deb for
> bash (there seems to be no option to force febootstrap to not clean up
> /tmp/febootstrap* after running).
>
> I see no reason why feboostrap wouldn't be able to read these files:
> 2318338    4 drwxr-xr-x   2 dek      redacted          4096 Aug 10 09:39
> /tmp/bashpkg/bin
> 2318444    0 lrwxrwxrwx   1 dek      redacted             4 Aug 10 09:39
> /tmp/bashpkg/bin/sh -> bash
> 2318443    0 lrwxrwxrwx   1 dek      redacted             4 Aug 10 09:39
> /tmp/bashpkg/bin/rbash -> bash
> 2318339 920 -rwxr-xr-x   1 dek      redacted        934336 Apr 18  2010
> /tmp/bashpkg/bin/bash

I would agree this certainly looks like a bug in febootstrap.  I can't
imagine what it would be that wouldn't also affect our builds on
Ubuntu 11.04.

Here is a way you could try reproducing the issue, in a smaller and
more manageable context.

You can run all of the commands below as non-root from /tmp or another
temporary directory.

First:

$ febootstrap --verbose --names bash

then examine the debug messages and the output files created in the
current directory ('base.img' and 'hostfiles').


This fetches bash, dash, and a number of other packages.  It reports:
1719 files and directories
febootstrap: warning: some host files are unreadable by non-root
febootstrap: warning: get your distro to fix these files:
        /bin/bash
        /bin/cat
        /bin/chgrp
        /bin/chmod
...

There are 1361 files in the list.
 
If base.img and hostfiles look reasonable, try building an appliance
from them:


Looking at hostfiles, there is no /bin/bash, but there is /bin/sh and /bin/rbash (BTW, Ubuntu uses dash as the main /bin/sh, not bash, but I assume that's not relevant).  sh is a symlink to bash, as is rbash.

I extracted base.img, and it doesn't include any files in /bin.  It looks like all the files in that archive are text/config files or symlinks. 

$ febootstrap-supermin-helper -f ext2 -v \
 base.img hostfiles $(uname -m) kernel initrd appliance

(Note that 'kernel', 'initrd' and 'appliance' are all output files
that will be created in the current directory).

Now you've got a chicken-and-egg problem, but 'appliance' is an ext2
disk image which can be examined using guestfish:

$ guestfish --ro -a appliance -m /dev/sda


Right.  I just mounted it as a loopback image instead... /bin contains only this:
$ ls -l  /mnt/appliance/bin
total 8
lrwxrwxrwx 1 root root 4 Sep 22  2010 rbash -> bash
lrwxrwxrwx 1 root root 9 Sep 22  2010 sh -> /bin/bash

Running strace on febootstrap, I see that it stats the /bin/bash in the unpacked bash deb pkg at least once successfully:
7928  lstat("/tmp/febootstrape193191fefe36c245623bc48e5b150a2.tmp/bash_4.1-2ubuntu3_amd64.deb.d//bin/bash", {st_mode=S_IFREG|0750, st_size=934336, ...}) = 0
7928  write(2, "febootstrap: warning: some host files are unreadable by non-root\nfebootstrap: warning: get your distro to fix these files:\n\t/bin/bash\n", 134) = 134

 
Welcome to guestfish, the libguestfs filesystem interactive shell for
editing virtual machine filesystems.

Type: 'help' for help on commands
     'man' to read the manual
     'quit' to quit the shell

><fs> ll /bin/
total 2668
drwxr-xr-x  2 1000 1000   4096 Aug 10 17:40 .
drwxr-xr-x 17 1000 1000   4096 Aug 10 17:41 ..
-rwxr-xr-x  1    0    0 954896 Mar 31 21:20 bash
-rwxr-xr-x  1    0    0  47696 Feb 23 14:24 cat
-rwxr-xr-x  1    0    0  55864 Feb 23 14:24 chgrp
-rwxr-xr-x  1    0    0  51704 Feb 23 14:24 chmod
-rwxr-xr-x  1    0    0  55880 Feb 23 14:24 chown
-rwxr-xr-x  1    0    0 117808 Feb 23 14:24 cp
-rwxr-xr-x  1    0    0 109736 Nov 15  2010 dash
-rwxr-xr-x  1    0    0  55832 Feb 23 14:24 date
-rwxr-xr-x  1    0    0  51840 Feb 23 14:24 dd
-rwxr-xr-x  1    0    0  64208 Feb 23 14:24 df
-rwxr-xr-x  1    0    0 105776 Feb 23 14:24 dir
-rwxr-xr-x  1    0    0  26968 Feb 23 14:24 echo
-rwxr-xr-x  1    0    0  22856 Feb 23 14:24 false
-rwxr-xr-x  1    0    0  47680 Feb 23 14:24 ln
-rwxr-xr-x  1    0    0 105776 Feb 23 14:24 ls
-rwxr-xr-x  1    0    0  47624 Feb 23 14:24 mkdir
-rwxr-xr-x  1    0    0  31112 Feb 23 14:24 mknod
-rwxr-xr-x  1    0    0  35352 Feb 23 14:24 mktemp
-rwxr-xr-x  1    0    0 109584 Feb 23 14:24 mv
-rwxr-xr-x  1    0    0  31200 Feb 23 14:24 pwd
lrwxrwxrwx  1    0    0      4 Jul 20 21:19 rbash -> bash
-rwxr-xr-x  1    0    0  35272 Feb 23 14:24 readlink
-rwxr-xr-x  1    0    0  55872 Feb 23 14:24 rm
-rwxr-xr-x  1    0    0  39320 Feb 23 14:24 rmdir
-rwxr-xr-x  1    0    0  15048 Jan 20  2011 run-parts
lrwxrwxrwx  1    0    0      4 Jul 20 21:19 sh -> dash
-rwxr-xr-x  1    0    0  26984 Feb 23 14:24 sleep
-rwxr-xr-x  1    0    0  63992 Feb 23 14:24 stty
-rwxr-xr-x  1    0    0  22872 Feb 23 14:24 sync
-rwxr-xr-x  1    0    0  10312 Jan 20  2011 tempfile
-rwxr-xr-x  1    0    0  47640 Feb 23 14:24 touch
-rwxr-xr-x  1    0    0  22856 Feb 23 14:24 true
-rwxr-xr-x  1    0    0  27016 Feb 23 14:24 uname
-rwxr-xr-x  1    0    0 105776 Feb 23 14:24 vdir
-rwxr-xr-x  1    0    0    946 Jan 20  2011 which

(The above output was from Ubuntu 11.04)

Rich.

--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org