Re: [Libguestfs] Rethinking appliance building
On Sun, Dec 05, 2010 at 10:09:58AM -0700, Thomas S Hatch wrote:
So I am in the process of Adding libguestfs to ArchLinux, should I use febootstrap 3 to build a supermin appliance for Arch? And if so where exactly should I start, do I need to submit patches for febootstrap for Arch support?
Yes please! This brings me to the second part of the email that I didn't write ... To support a new package manager, the package manager has to support some simple operations. (1) There has to be a way to detect that the package manager should be used, eg. by looking for /etc/arch-release (2) There must be a way to take a list of package names, and fully resolve all dependencies (recursively as necessary) to get a complete list of packages that must be installed to bootstrap. (3) Must be a way to download a package. (4) Must be able to list out the files in a downloaded package. (5) Must be able to extract a single file from a downloaded package. (1)-(5) have to work as non-root. You can use the RPM and Debian plugins for comparison: http://git.annexia.org/?p=febootstrap.git;a=blob;f=febootstrap_debian.ml;h=c... http://git.annexia.org/?p=febootstrap.git;a=blob;f=febootstrap_yum_rpm.ml;h=... The precise API is described here: http://git.annexia.org/?p=febootstrap.git;a=blob;f=febootstrap_package_handl... The git repo is down at the moment, but the contents of the tarball are identical to git right now. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-df lists disk usage of guests without needing to install any software inside the virtual machine. Supports Linux and Windows. http://et.redhat.com/~rjones/virt-df/
To make this even more explicit:
(1) There has to be a way to detect that the package manager should be used, eg. by looking for /etc/arch-release
RPM: check for existence of /etc/redhat-release Debian: check for existence of /etc/debian_version
(2) There must be a way to take a list of package names, and fully resolve all dependencies (recursively as necessary) to get a complete list of packages that must be installed to bootstrap.
RPM: [lots of Yum API magic] Debian: apt-cache depends --recurse -i <list of package names>
(3) Must be a way to download a package.
RPM: yumdownloader <name of package> Debian: aptitude download <name of package>
(4) Must be able to list out the files in a downloaded package.
RPM: rpm -qlp foo.rpm Debian: dpkg-deb -c foo.deb
(5) Must be able to extract a single file from a downloaded package.
RPM: rpm2cpio foo.rpm | cpio -id Debian: dpkg-deb --fsys-tarfile foo.deb | tar xf - So it's almost enough for you to tell me the equivalent of those commands for pacman, although of course I'd prefer a patch to febootstrap! Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into Xen guests. http://et.redhat.com/~rjones/virt-p2v
On Sun, Dec 5, 2010 at 10:56 AM, Richard W.M. Jones <rjones@redhat.com>wrote:
To make this even more explicit:
(1) There has to be a way to detect that the package manager should be used, eg. by looking for /etc/arch-release
RPM: check for existence of /etc/redhat-release Debian: check for existence of /etc/debian_version
Arch: check for existence of /etc/arch-release Arch: Or check the kernel version for "ARCH" with a uname -r
(2) There must be a way to take a list of package names, and fully resolve all dependencies (recursively as necessary) to get a complete list of packages that must be installed to bootstrap.
RPM: [lots of Yum API magic] Debian: apt-cache depends --recurse -i <list of package names>
Arch: pactree -l <list of package names> | sort -u pactree is part of the pacman-contrib package
(3) Must be a way to download a package.
RPM: yumdownloader <name of package> Debian: aptitude download <name of package>
Arch: pacman -Sw --cachedir=<Where to download> <name of package(s)> This will download the package to the cachedir: /var/cache/pacman/pkg
(4) Must be able to list out the files in a downloaded package.
RPM: rpm -qlp foo.rpm Debian: dpkg-deb -c foo.deb
pacman -Qlp foo.pkg.tar.xz
(5) Must be able to extract a single file from a downloaded package.
RPM: rpm2cpio foo.rpm | cpio -id Debian: dpkg-deb --fsys-tarfile foo.deb | tar xf -
Arch: tar xf foo.pkg.tar.xz Arch packages are just tarballs with a PKGINFO file in the root directory which has package metadata
So it's almost enough for you to tell me the equivalent of those commands for pacman, although of course I'd prefer a patch to febootstrap!
Rich.
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into Xen guests. http://et.redhat.com/~rjones/virt-p2v
Here are the answers, and all commands work as non-root, before running any commands the package database would need to be updated by running "pacman -Sy" I will grab a git checkout of febootstrap and start looking into prepping a patch, but if this is enough info for you to whip up support then let me know and I will test it and fix any arch specific things. Thanks! -Tom Hatch
Ok, here is where I am so far, but my ocaml is VERY weak, so it is slow going, I also added the right stuff to config.ml.in and Makefile.ml to make it work. it seems to work, but when I run febootstrap --names bash coreutils dcron dhcpcd e2fsprogs file filesystem findutils gawk grep grub gzip initscripts iputils kernel26 less libusb logrotate net-tools pacman sed shadow sysvinit tar tcp_wrappers udev util-linux-ng it only processes bash. I am sure I am missing something obvious, but if you have any suggestions that would be great, learning ocaml is taking some time :) -Tom Hatch On Sun, Dec 5, 2010 at 4:28 PM, Thomas S Hatch <thatch45@gmail.com> wrote:
On Sun, Dec 5, 2010 at 10:56 AM, Richard W.M. Jones <rjones@redhat.com>wrote:
To make this even more explicit:
(1) There has to be a way to detect that the package manager should be used, eg. by looking for /etc/arch-release
RPM: check for existence of /etc/redhat-release Debian: check for existence of /etc/debian_version
Arch: check for existence of /etc/arch-release Arch: Or check the kernel version for "ARCH" with a uname -r
(2) There must be a way to take a list of package names, and fully resolve all dependencies (recursively as necessary) to get a complete list of packages that must be installed to bootstrap.
RPM: [lots of Yum API magic] Debian: apt-cache depends --recurse -i <list of package names>
Arch: pactree -l <list of package names> | sort -u pactree is part of the pacman-contrib package
(3) Must be a way to download a package.
RPM: yumdownloader <name of package> Debian: aptitude download <name of package>
Arch: pacman -Sw --cachedir=<Where to download> <name of package(s)> This will download the package to the cachedir: /var/cache/pacman/pkg
(4) Must be able to list out the files in a downloaded package.
RPM: rpm -qlp foo.rpm Debian: dpkg-deb -c foo.deb
pacman -Qlp foo.pkg.tar.xz
(5) Must be able to extract a single file from a downloaded package.
RPM: rpm2cpio foo.rpm | cpio -id Debian: dpkg-deb --fsys-tarfile foo.deb | tar xf -
Arch: tar xf foo.pkg.tar.xz Arch packages are just tarballs with a PKGINFO file in the root directory which has package metadata
So it's almost enough for you to tell me the equivalent of those commands for pacman, although of course I'd prefer a patch to febootstrap!
Rich.
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into Xen guests. http://et.redhat.com/~rjones/virt-p2v
Here are the answers, and all commands work as non-root, before running any commands the package database would need to be updated by running "pacman -Sy"
I will grab a git checkout of febootstrap and start looking into prepping a patch, but if this is enough info for you to whip up support then let me know and I will test it and fix any arch specific things.
Thanks!
-Tom Hatch
On Sun, Dec 05, 2010 at 11:16:01PM -0700, Thomas S Hatch wrote:
Ok, here is where I am so far, but my ocaml is VERY weak, so it is slow going, I also added the right stuff to config.ml.in and Makefile.ml to make it work.
I've pushed it, untested of course, with some minor tidy up to comments: http://git.annexia.org/?p=febootstrap.git;a=commitdiff;h=3e156a27f7f67c9e05f...
it seems to work, but when I run febootstrap --names bash coreutils dcron dhcpcd e2fsprogs file filesystem findutils gawk grep grub gzip initscripts iputils kernel26 less libusb logrotate net-tools pacman sed shadow sysvinit tar tcp_wrappers udev util-linux-ng it only processes bash.
let pacman_resolve_dependencies_and_download names = let cmd = sprintf "pactree -u %s | sort -u" (String.concat " " (List.map Filename.quote names)) in The command that this generates would be something like pactree -u 'bash' 'coreutils' 'dcron' [etc...] | sort -u Does pactree itself support multiple names here? Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://et.redhat.com/~rjones/libguestfs/ See what it can do: http://et.redhat.com/~rjones/libguestfs/recipes.html
On Sun, Dec 05, 2010 at 04:28:39PM -0700, Thomas S Hatch wrote:
On Sun, Dec 5, 2010 at 10:56 AM, Richard W.M. Jones <rjones@redhat.com>wrote:
(4) Must be able to list out the files in a downloaded package.
RPM: rpm -qlp foo.rpm Debian: dpkg-deb -c foo.deb
pacman -Qlp foo.pkg.tar.xz
RPM stores per-file metadata: basically if each file is a config file, documentation and a few other things. Is this available in pacman? For Debian we have to synthesize this (eg. everything in /etc is marked as a config file). It is not very satisfactory. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into Xen guests. http://et.redhat.com/~rjones/virt-p2v
participants (2)
-
Richard W.M. Jones -
Thomas S Hatch