Re: [Libguestfs] Fwd: Re: febootstrap in Debian
[CC also to libguestfs mailing list] On Tue, May 17, 2011 at 08:11:10PM +0200, Guido Günther wrote:
As to binary debs: I wonder if we can't just get away by adding the needed packages as build-depends (so they end up installed on the buildd before the package built) and modify febootstrap to look at the _installed_ files instead of at the downloaded debs.
Looking at febootstrap_debian.ml this looks possible but very likely I'm missing something. I'm not very good at OCaml though.
I guess the (non-obvious) thing you're missing is that we can't use installed configuration files, because the sysadmin may have changed them. We need the original of all files, and we get those from the *.debs. In the special case (and only in this special case) where we are building on the Debian builders, there's no sysadmin and so no one could have modified any configuration or other files. However there may still be other problems I haven't seen. How to get the list of files in each package (should be OK: dpkg -L). How to resolve the dependencies so we know we have all packages / files? With the Fedora builders, we have the *.rpm files left around and we use them. Is that not the same in the Debian case?
Is the insmod.static issue somehow resolved? Maybe filing a wishlist bug would help?
Still a bug AFAIK. In Fedora we build module-init-tools from source and we don't do anything special[1] to get insmod.static, so I'm not sure why it doesn't exist on Debian too.
Having febootstrap/libguestfs in Debian would be great, I've met so many people asking about it's status! Roland, do you plan to work on libguestfs? If so I'd be happy to add you to the pkg-libvirt team.
Hope you can get something worked out. Let me know if you need any more help. libguestfs is legendarily hard to build ... Rich. [1] http://pkgs.fedoraproject.org/gitweb/?p=module-init-tools.git;a=blob;f=modul... -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into Xen guests. http://et.redhat.com/~rjones/virt-p2v
Hi Richard, On Tue, May 17, 2011 at 07:32:17PM +0100, Richard W.M. Jones wrote:
[CC also to libguestfs mailing list]
On Tue, May 17, 2011 at 08:11:10PM +0200, Guido Günther wrote:
As to binary debs: I wonder if we can't just get away by adding the needed packages as build-depends (so they end up installed on the buildd before the package built) and modify febootstrap to look at the _installed_ files instead of at the downloaded debs.
Looking at febootstrap_debian.ml this looks possible but very likely I'm missing something. I'm not very good at OCaml though.
I guess the (non-obvious) thing you're missing is that we can't use installed configuration files, because the sysadmin may have changed them. We need the original of all files, and we get those from the *.debs.
In the special case (and only in this special case) where we are building on the Debian builders, there's no sysadmin and so no one could have modified any configuration or other files.
Sure. I was only refering to the build servers here.
However there may still be other problems I haven't seen. How to get the list of files in each package (should be OK: dpkg -L). How to resolve the dependencies so we know we have all packages / files?
dpkg -L should do the trick. For dependencies it seems you're using "apt-cache depends --recurse -i %s" but I didn't look closely.
With the Fedora builders, we have the *.rpm files left around and we use them. Is that not the same in the Debian case?
I don't think we can assume we have all the debs around if we build-depend on them. The packages might be already installed on the system and therefore might be already removed from /var/cache/apt/archives/. Could we build the appliance on the target system and only update it on package upgrades?
Is the insmod.static issue somehow resolved? Maybe filing a wishlist bug would help?
Still a bug AFAIK.
In Fedora we build module-init-tools from source and we don't do anything special[1] to get insmod.static, so I'm not sure why it doesn't exist on Debian too.
It just didn't get built and installed. I've filed a bug so we can at least update febootstrap in Debian to 3.0. (Closes: #627353)
Having febootstrap/libguestfs in Debian would be great, I've met so many people asking about it's status! Roland, do you plan to work on libguestfs? If so I'd be happy to add you to the pkg-libvirt team.
Hope you can get something worked out. Let me know if you need any more help.
libguestfs is legendarily hard to build ...
In fact I find it fun to build but unfortunately lack the time to work on it. Cheers, -- Guido
Rich.
[1] http://pkgs.fedoraproject.org/gitweb/?p=module-init-tools.git;a=blob;f=modul...
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into Xen guests. http://et.redhat.com/~rjones/virt-p2v
On Thu, May 19, 2011 at 10:15:38PM +0200, Guido Günther wrote:
Could we build the appliance on the target system and only update it on package upgrades?
While you could do this, I don't think it's a good idea. The main part of building libguestfs in Fedora is we run the whole test suite on the appliance, thus ensuring it has a high chance of actually working when downloaded by the end user. We often find errors too -- for example only two days ago we ran across a bug in the upstream 'file' program. Since it's a supermin appliance, distributing it is not an issue because it's only a couple of megabytes compressed. I leave it up to your discretion though. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into Xen guests. http://et.redhat.com/~rjones/virt-p2v
By the me, I think this is not correct to use febootstrap in Debian, we have better application - debootstrap. And we should build image file with clean chroot environment. For now I see it downloaded packages with the _same_ version that is installed on the host, but by the me I use some packages from testing and unstable. And this try to download these packages from stable repo. This fails. And also it simply extracts contents from deb files, and do not run configure scripts. On 20 May 2011 01:28, Richard W.M. Jones <rjones@redhat.com> wrote:
On Thu, May 19, 2011 at 10:15:38PM +0200, Guido Günther wrote:
Could we build the appliance on the target system and only update it on package upgrades?
While you could do this, I don't think it's a good idea. The main part of building libguestfs in Fedora is we run the whole test suite on the appliance, thus ensuring it has a high chance of actually working when downloaded by the end user. We often find errors too -- for example only two days ago we ran across a bug in the upstream 'file' program.
Since it's a supermin appliance, distributing it is not an issue because it's only a couple of megabytes compressed.
I leave it up to your discretion though.
Rich.
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into Xen guests. http://et.redhat.com/~rjones/virt-p2v
_______________________________________________ Libguestfs mailing list Libguestfs@redhat.com https://www.redhat.com/mailman/listinfo/libguestfs
-- Nikita A Menkovich http://libc6.org/ JID: menkovich@gmail.com Tel: +7 (921) 423-96-48
On Sat, May 21, 2011 at 09:40:54PM +0400, Nikita A Menkovich wrote:
By the me, I think this is not correct to use febootstrap in Debian, we have better application - debootstrap. And we should build image file with clean chroot environment.
The thing is, febootstrap is no longer like debootstrap. It used to be (febootstrap 2.x), hence the name, but nowadays it works very differently, and it is far more simple, and works across distros.
For now I see it downloaded packages with the _same_ version that is installed on the host, but by the me I use some packages from testing and unstable. And this try to download these packages from stable repo. This fails.
And also it simply extracts contents from deb files, and do not run configure scripts.
... which is in fact the whole point of it. Using debirf or (old 2.x) febootstrap was a constant source of pain, and I'm glad we got rid of both. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into Xen guests. http://et.redhat.com/~rjones/virt-p2v
Maybe it will be better to create separate package, that will provides appliances. And this will provide OS appliance created right way for current OS. For Debian/Ubuntu - debootstra for example. So we will write script for current OS, and this won't be a hack On 21 May 2011 22:07, Richard W.M. Jones <rjones@redhat.com> wrote:
On Sat, May 21, 2011 at 09:40:54PM +0400, Nikita A Menkovich wrote:
By the me, I think this is not correct to use febootstrap in Debian, we have better application - debootstrap. And we should build image file with clean chroot environment.
The thing is, febootstrap is no longer like debootstrap. It used to be (febootstrap 2.x), hence the name, but nowadays it works very differently, and it is far more simple, and works across distros.
For now I see it downloaded packages with the _same_ version that is installed on the host, but by the me I use some packages from testing and unstable. And this try to download these packages from stable repo. This fails.
And also it simply extracts contents from deb files, and do not run configure scripts.
... which is in fact the whole point of it.
Using debirf or (old 2.x) febootstrap was a constant source of pain, and I'm glad we got rid of both.
Rich.
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into Xen guests. http://et.redhat.com/~rjones/virt-p2v
-- Nikita A Menkovich http://libc6.org/ JID: menkovich@gmail.com Tel: +7 (921) 423-96-48
On Sat, May 21, 2011 at 10:17:18PM +0400, Nikita A Menkovich wrote:
Maybe it will be better to create separate package, that will provides appliances.
And this will provide OS appliance created right way for current OS. For Debian/Ubuntu - debootstra for example. So we will write script for current OS, and this won't be a hack
Such a package exists -- debirf on Debian. Guido made a heroic effort to port libguestfs to use debirf already. The problem is that the debirf/debootstrap combination simply does not work. It was constantly full of bugs and does not work more often than it does work. It was because I constantly had to deal with these bugs that I removed debirf support and replaced it with the cross-platform febootstrap 3.x that we are using now. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org
What the difference between appliances built with febootstrap and raw image, that could be built with debootstrap, and putting contents of debootstrap directory to image? On 23 May 2011 11:39, Richard W.M. Jones <rjones@redhat.com> wrote:
On Sat, May 21, 2011 at 10:17:18PM +0400, Nikita A Menkovich wrote:
Maybe it will be better to create separate package, that will provides appliances.
And this will provide OS appliance created right way for current OS. For Debian/Ubuntu - debootstra for example. So we will write script for current OS, and this won't be a hack
Such a package exists -- debirf on Debian. Guido made a heroic effort to port libguestfs to use debirf already.
The problem is that the debirf/debootstrap combination simply does not work. It was constantly full of bugs and does not work more often than it does work.
It was because I constantly had to deal with these bugs that I removed debirf support and replaced it with the cross-platform febootstrap 3.x that we are using now.
Rich.
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org
-- Nikita A Menkovich http://libc6.org/ JID: menkovich@gmail.com Tel: +7 (921) 423-96-48
On Mon, May 23, 2011 at 12:57:13PM +0400, Nikita A Menkovich wrote:
What the difference between appliances built with febootstrap and raw image, that could be built with debootstrap, and putting contents of debootstrap directory to image?
Apart from the difference that debootstrap is often broken, how do you propose to get the directory into an image w/o using either libguestfs or needing (real not fake-) root? Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones New in Fedora 11: Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 70 libraries supprt'd http://fedoraproject.org/wiki/MinGW http://www.annexia.org/fedora_mingw
On Thu, May 19, 2011 at 10:15:38PM +0200, Guido Günther wrote: [..snip..]
In Fedora we build module-init-tools from source and we don't do anything special[1] to get insmod.static, so I'm not sure why it doesn't exist on Debian too.
It just didn't get built and installed. I've filed a bug so we can at least update febootstrap in Debian to 3.0. (Closes: #627353)
Is klibc's insmod an alternative? It's in the klibc-utils package: http://packages.debian.org/sid/ia64/klibc-utils/filelist Cheers, -- Guido
On Sun, May 22, 2011 at 09:58:43PM +0200, Guido Günther wrote:
On Thu, May 19, 2011 at 10:15:38PM +0200, Guido Günther wrote: [..snip..]
In Fedora we build module-init-tools from source and we don't do anything special[1] to get insmod.static, so I'm not sure why it doesn't exist on Debian too.
It just didn't get built and installed. I've filed a bug so we can at least update febootstrap in Debian to 3.0. (Closes: #627353)
Is klibc's insmod an alternative? It's in the klibc-utils package: http://packages.debian.org/sid/ia64/klibc-utils/filelist
Not the ia64/Itanic package :-) Yes, this looks perfect. In fact at 1608 bytes [sic] this is pretty amazing. I checked, and the binary does appear to work. # ls -l /usr/lib/klibc/bin/insmod -rwxr-xr-x 1 root root 1608 May 18 14:27 /usr/lib/klibc/bin/insmod # ldd /usr/lib/klibc/bin/insmod not a dynamic executable # file /usr/lib/klibc/bin/insmod /usr/lib/klibc/bin/insmod: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked (uses shared libs), stripped Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming blog: http://rwmj.wordpress.com Fedora now supports 80 OCaml packages (the OPEN alternative to F#) http://cocan.org/getting_started_with_ocaml_on_red_hat_and_fedora
participants (3)
-
Guido Günther -
Nikita A Menkovich -
Richard W.M. Jones