On Mon, Oct 08, Richard W.M. Jones wrote:
On Mon, Oct 08, 2012 at 08:50:47PM +0200, Olaf Hering wrote:
On Mon, Oct 08, Richard W.M. Jones wrote:
But out of interest, how are you creating this appliance? I thought we'd ripped out support for old-style appliances from the rest of the toolchain ...
I use mkinitrd to pull in all tools and run guestfsd instead of /sysroot/sbin/init. This allows me to build the package automated as unprivileged user in the buildsystem. Creating a loop file requires root, which is cumbersome in the openSuSE buildservice.
BTW, febootstrap (which is what we use) runs unprivileged and should be able to process SuSE rpms. However it does require yum, not zypper, but that's probably easy to fix if zypper has a way to take a list of package names and resolve all the dependencies without needing root and without needing to install any packages.
I just looked at this. Does febootstrap do some sort of unrpm? Does it call %pre/%post install scripts? zypper alone can not be used because it calls rpm, which does chroot. It can however download the required rpms and place them into a directory: zypper \ --verbose \ --verbose \ --gpg-auto-import-keys \ --root /dev/shm/${LOGNAME} \ --reposd-dir /etc/zypp/repos.d \ --pkg-cache-dir /dev/shm/${LOGNAME}--pkg-cache-dir \ --non-interactive \ install \ --auto-agree-with-licenses \ --no-recommends \ --download-only \ kernel-default So what do you suggest to do on a system with zypper/libzypp? Olaf
On Thu, Nov 29, 2012 at 09:18:23PM +0100, Olaf Hering wrote:
On Mon, Oct 08, Richard W.M. Jones wrote:
On Mon, Oct 08, 2012 at 08:50:47PM +0200, Olaf Hering wrote:
On Mon, Oct 08, Richard W.M. Jones wrote:
But out of interest, how are you creating this appliance? I thought we'd ripped out support for old-style appliances from the rest of the toolchain ...
I use mkinitrd to pull in all tools and run guestfsd instead of /sysroot/sbin/init. This allows me to build the package automated as unprivileged user in the buildsystem. Creating a loop file requires root, which is cumbersome in the openSuSE buildservice.
BTW, febootstrap (which is what we use) runs unprivileged and should be able to process SuSE rpms. However it does require yum, not zypper, but that's probably easy to fix if zypper has a way to take a list of package names and resolve all the dependencies without needing root and without needing to install any packages.
I just looked at this.
Does febootstrap do some sort of unrpm? Does it call %pre/%post install scripts?
Not sure what unrpm is, but it does unpack the RPM (using rpm2cpio). It does *not* call %pre or %post scripts.
zypper alone can not be used because it calls rpm, which does chroot. It can however download the required rpms and place them into a directory:
zypper \ --verbose \ --verbose \ --gpg-auto-import-keys \ --root /dev/shm/${LOGNAME} \ --reposd-dir /etc/zypp/repos.d \ --pkg-cache-dir /dev/shm/${LOGNAME}--pkg-cache-dir \ --non-interactive \ install \ --auto-agree-with-licenses \ --no-recommends \ --download-only \ kernel-default
So what do you suggest to do on a system with zypper/libzypp?
This sounds fine. This is basically the same as what this horrible bit of Python code does: https://github.com/libguestfs/febootstrap/blob/master/src/febootstrap_yum_rp... followed by the call to yumdownloader. As long as the above zypper command doesn't need root, it should all just work fine. My suggestion is to take the current febootstrap_yum_rpm.ml file, rename it and modify it. The 'yum_rpm_resolve_dependencies_and_download' function (renamed) should end up being considerably shorter. BTW we found it was useful to be able to pass a yum configuration file through to the yum-rpm package handler. That's the purpose of the yum_config global variable. It is set in febootstrap_cmdline.ml. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming blog: http://rwmj.wordpress.com Fedora now supports 80 OCaml packages (the OPEN alternative to F#) http://cocan.org/getting_started_with_ocaml_on_red_hat_and_fedora
On Thu, Nov 29, 2012 at 09:02:34PM +0000, Richard W.M. Jones wrote:
zypper \ --verbose \ --verbose \ --gpg-auto-import-keys \ --root /dev/shm/${LOGNAME} \ --reposd-dir /etc/zypp/repos.d \ --pkg-cache-dir /dev/shm/${LOGNAME}--pkg-cache-dir \ --non-interactive \ install \ --auto-agree-with-licenses \ --no-recommends \ --download-only \ kernel-default
By the way, does this download everything (even, for example, packages which are already installed at the latest version)? Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones New in Fedora 11: Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 70 libraries supprt'd http://fedoraproject.org/wiki/MinGW http://www.annexia.org/fedora_mingw
On Thu, Nov 29, Richard W.M. Jones wrote:
On Thu, Nov 29, 2012 at 09:02:34PM +0000, Richard W.M. Jones wrote:
zypper \ --verbose \ --verbose \ --gpg-auto-import-keys \ --root /dev/shm/${LOGNAME} \ --reposd-dir /etc/zypp/repos.d \ --pkg-cache-dir /dev/shm/${LOGNAME}--pkg-cache-dir \ --non-interactive \ install \ --auto-agree-with-licenses \ --no-recommends \ --download-only \ kernel-default
By the way, does this download everything (even, for example, packages which are already installed at the latest version)?
It will download kernel-default from the configured repos in /etc/zypp/repos.d into --pkg-cache-dir. Installed packaged will not be considered. Does febootstrap work with installed packages? Olaf
On Thu, Nov 29, 2012 at 10:24:54PM +0100, Olaf Hering wrote:
On Thu, Nov 29, Richard W.M. Jones wrote:
On Thu, Nov 29, 2012 at 09:02:34PM +0000, Richard W.M. Jones wrote:
zypper \ --verbose \ --verbose \ --gpg-auto-import-keys \ --root /dev/shm/${LOGNAME} \ --reposd-dir /etc/zypp/repos.d \ --pkg-cache-dir /dev/shm/${LOGNAME}--pkg-cache-dir \ --non-interactive \ install \ --auto-agree-with-licenses \ --no-recommends \ --download-only \ kernel-default
By the way, does this download everything (even, for example, packages which are already installed at the latest version)?
It will download kernel-default from the configured repos in /etc/zypp/repos.d into --pkg-cache-dir. Installed packaged will not be considered. Does febootstrap work with installed packages?
The idea of febootstrap is to create new instances of the operating system, so essentially whatever is installed on the host we don't care about. ie: febootstrap --names bash will download 'bash' and all its dependencies and create a supermin appliance out of them, whether or not bash + dependencies happen to be installed on the host. Of course "normal" yum commands and suchlike consider what's installed on the host already and only install or upgrade packages. We don't want that behaviour -- hence the complicated Python script (see link in previous email). If there's a way to do all this from the zypper C API, then we could use that instead. Calling out to C from OCaml is simple -- in fact we already do it in libguestfs. See example resize/progress-c.c, or for the real documentation read: http://caml.inria.fr/pub/docs/manual-ocaml-4.00/manual033.html In short, we need a way to download all RPMs, starting at a set of root packages (or root package names), without considering what is installed already. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into Xen guests. http://et.redhat.com/~rjones/virt-p2v
On Thu, Nov 29, 2012 at 10:24:54PM +0100, Olaf Hering wrote:
On Thu, Nov 29, Richard W.M. Jones wrote:
On Thu, Nov 29, 2012 at 09:02:34PM +0000, Richard W.M. Jones wrote:
zypper \ --verbose \ --verbose \ --gpg-auto-import-keys \ --root /dev/shm/${LOGNAME} \ --reposd-dir /etc/zypp/repos.d \ --pkg-cache-dir /dev/shm/${LOGNAME}--pkg-cache-dir \ --non-interactive \ install \ --auto-agree-with-licenses \ --no-recommends \ --download-only \ kernel-default
By the way, does this download everything (even, for example, packages which are already installed at the latest version)?
It will download kernel-default from the configured repos in /etc/zypp/repos.d into --pkg-cache-dir. Installed packaged will not be considered.
I was reading the last sentence again this morning. Did I misunderstand it? Does this zypper command not consider installed packages and download all of them regardless of what is installed on the host? Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming blog: http://rwmj.wordpress.com Fedora now supports 80 OCaml packages (the OPEN alternative to F#) http://cocan.org/getting_started_with_ocaml_on_red_hat_and_fedora
On Fri, Nov 30, Richard W.M. Jones wrote:
On Thu, Nov 29, 2012 at 10:24:54PM +0100, Olaf Hering wrote:
On Thu, Nov 29, Richard W.M. Jones wrote:
On Thu, Nov 29, 2012 at 09:02:34PM +0000, Richard W.M. Jones wrote:
zypper \ --verbose \ --verbose \ --gpg-auto-import-keys \ --root /dev/shm/${LOGNAME} \ --reposd-dir /etc/zypp/repos.d \ --pkg-cache-dir /dev/shm/${LOGNAME}--pkg-cache-dir \ --non-interactive \ install \ --auto-agree-with-licenses \ --no-recommends \ --download-only \ kernel-default
By the way, does this download everything (even, for example, packages which are already installed at the latest version)?
It will download kernel-default from the configured repos in /etc/zypp/repos.d into --pkg-cache-dir. Installed packaged will not be considered.
I was reading the last sentence again this morning. Did I misunderstand it? Does this zypper command not consider installed packages and download all of them regardless of what is installed on the host?
I think it does. With the --root option above it considers only packages within that directory. Other than that, I think only --pkg-cache-dir needs to be set because an unprivileged user has no write permissions in /var/cache/zypp. So I think if febootstrap takes files from the installed system and in addition looks at the possibly downloaded packages then all should be fine. Olaf
On Fri, Nov 30, 2012 at 08:38:24PM +0100, Olaf Hering wrote:
I think it does. With the --root option above it considers only packages within that directory. Other than that, I think only --pkg-cache-dir needs to be set because an unprivileged user has no write permissions in /var/cache/zypp.
That all sounds good, so ignore what I said in the other reply.
So I think if febootstrap takes files from the installed system and in addition looks at the possibly downloaded packages then all should be fine.
At boot time, febootstrap-supermin-helper takes files from the installed system. Except for files that could have changed (/etc files, or more specifically, files marked with %config). Those files come from the RPMs. We ensure the non-%config files exist on the host by using dependencies: http://pkgs.fedoraproject.org/cgit/libguestfs.git/tree/libguestfs.spec#n94 It's actually fairly easy to understand how it all works if you run febootstrap on a system like Fedora that already has a package handler. None of these commands need root, or will damage your system. $ cd /tmp; febootstrap --names bash febootstrap: warning: some host files are unreadable by non-root febootstrap: warning: get your distro to fix these files: /usr/sbin/build-locale-archive /usr/sbin/glibc_post_upgrade.x86_64 [after a few minutes, two files are created in the current directory] $ ls -l -rw-rw-r--. 1 rjones rjones 466258 Nov 30 19:56 hostfiles -rw-rw-r--. 1 rjones rjones 2732544 Nov 30 19:56 base.img $ wc -l hostfiles 17552 hostfiles $ grep /bin/bash hostfiles /usr/bin/bash /usr/bin/bashbug-64 $ cpio -itv < base.img [...] -rw-r--r-- 1 rjones rjones 158 Nov 30 19:56 etc/hosts -rw-r--r-- 1 rjones rjones 370 Nov 30 19:56 etc/hosts.allow -rw-r--r-- 1 rjones rjones 460 Nov 30 19:56 etc/hosts.deny -rw-r--r-- 1 rjones rjones 942 Nov 30 19:56 etc/inputrc -rw-r--r-- 1 rjones rjones 0 Nov 30 19:56 etc/ld.so.cache -rw-r--r-- 1 rjones rjones 28 Nov 30 19:56 etc/ld.so.conf drwxr-xr-x 2 rjones rjones 0 Nov 30 19:56 etc/ld.so.conf.d -rw-r--r-- 1 rjones rjones 0 Nov 30 19:56 etc/motd -rw-r--r-- 1 rjones rjones 1706 Nov 30 19:56 etc/nsswitch.conf drwxr-xr-x 2 rjones rjones 0 Nov 30 19:56 etc/opt -rw-r--r-- 1 rjones rjones 609 Nov 30 19:56 etc/passwd [...] 'base.img' contains the directory outline and configuration files, while 'hostfiles' is a list of files that will be pulled in from the host filesystem. 'base.img' + 'hostfiles' is what we call the "supermin appliance". To reconstruct a real appliance (as is done when libguestfs runs), do: $ mkdir output $ febootstrap-supermin-helper --format ext2 \ base.img hostfiles `uname -m` \ ouput/kernel output/initrd output/appliance $ ls -l output/ total 283112 -rw-r--r--. 1 rjones rjones 4294967296 Nov 30 20:00 appliance -rw-rw-r--. 1 rjones rjones 1220608 Nov 30 20:00 initrd lrwxrwxrwx. 1 rjones rjones 33 Nov 30 20:00 kernel -> /boot/vmlinuz-3.6.2-2.fc18.x86_64 And that's all there is to it. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://et.redhat.com/~rjones/virt-top
participants (2)
-
Olaf Hering -
Richard W.M. Jones