Weird results from g.sh()
Hi, I'm using libguestfs 1.42, When I run the following python3 commands on a rhel7.8: print(g.ls('/sys')) print(g.sh('ls /sys')) I get: [] block bus class dev devices firmware fs hypervisor kernel module power It seems that g.ls('/sys') is chrooted correctly. But g.sh('ls /sys') isn't. I came across this behavior when I used g.command(['grub2-mkconfig', '-o', '....']) on rhel7.8 and it didn't produce the correct linuxefi/initrdefi entries in grub, since in rhel7.8 the uefi detection algorithm is checking in bash '[ -d /sys/firmware/efi ]'. https://src.fedoraproject.org/rpms/grub2/blob/71e0fb4ea646071ec86e4a65bb2424... I thought I could simply g.mkdir_p('/sys/firmware/efi') - but that created a fake dir in my chrooted environment, where grub2-mkconfig somehow was inspecting the real /sys as demonstrated in the above 'ls' example. Any ideas on why sh jailbreaks the chroot? is /sys set through some environment variable? Thanks!
On Thu, Aug 27, 2020 at 10:50:12AM +0300, Sam Eiderman wrote:
Hi,
I'm using libguestfs 1.42,
When I run the following python3 commands on a rhel7.8:
print(g.ls('/sys')) []
This lists the files in /sys in the filesystem, which in most disk images is likely to be an empty directory.
print(g.sh('ls /sys')) block bus class dev devices firmware fs hypervisor kernel module power
sh bind-mounts /sys from the appliance into the filesystem temporarily in order to make it possible to run some commands: https://github.com/libguestfs/libguestfs/blob/fce82fe55a2b64a1a7e494858aa272... However this /sys has nothing to do with /sys as it may appear in the guest when it is running. Libguestfs edits filesystems, it doesn't run the guest.
It seems that g.ls('/sys') is chrooted correctly. But g.sh('ls /sys') isn't.
I came across this behavior when I used g.command(['grub2-mkconfig', '-o', '....']) on rhel7.8 and it didn't produce the correct linuxefi/initrdefi entries in grub, since in rhel7.8 the uefi detection algorithm is checking in bash '[ -d /sys/firmware/efi ]'. https://src.fedoraproject.org/rpms/grub2/blob/71e0fb4ea646071ec86e4a65bb2424...
It's pretty unlikely that running grub can work. The libguestfs appliance is sufficiently different from how the guest boots that things like BIOS mappings are not related, and libguestfs doesn't use EFI at all. There's unfortunately no good way to run grub. syslinux/extlinux is the only option since it is designed to make modifications to offline filesystems, and grub is not. Rich.
I thought I could simply g.mkdir_p('/sys/firmware/efi') - but that created a fake dir in my chrooted environment, where grub2-mkconfig somehow was inspecting the real /sys as demonstrated in the above 'ls' example.
Any ideas on why sh jailbreaks the chroot? is /sys set through some environment variable?
Thanks!
_______________________________________________ Libguestfs mailing list Libguestfs@redhat.com https://www.redhat.com/mailman/listinfo/libguestfs
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v
On Thu, Aug 27, 2020 at 11:08 AM Richard W.M. Jones <rjones@redhat.com> wrote:
On Thu, Aug 27, 2020 at 10:50:12AM +0300, Sam Eiderman wrote:
Hi,
I'm using libguestfs 1.42,
When I run the following python3 commands on a rhel7.8:
print(g.ls('/sys')) []
This lists the files in /sys in the filesystem, which in most disk images is likely to be an empty directory.
print(g.sh('ls /sys')) block bus class dev devices firmware fs hypervisor kernel module power
sh bind-mounts /sys from the appliance into the filesystem temporarily in order to make it possible to run some commands:
https://github.com/libguestfs/libguestfs/blob/fce82fe55a2b64a1a7e494858aa272...
However this /sys has nothing to do with /sys as it may appear in the guest when it is running. Libguestfs edits filesystems, it doesn't run the guest.
Yes, I understand that, I am just surprised to know that we bind /sys for g.sh() commands. However, "g.command([self.mkconfig, '-o', self.config])" shouldn't have the appliances /sys available (I looked briefly in command.c and I don't see the same bind-mount code), or maybe I'm missing something.
It seems that g.ls('/sys') is chrooted correctly. But g.sh('ls /sys') isn't.
I came across this behavior when I used g.command(['grub2-mkconfig', '-o', '....']) on rhel7.8 and it didn't produce the correct linuxefi/initrdefi entries in grub, since in rhel7.8 the uefi detection algorithm is checking in bash '[ -d /sys/firmware/efi ]'. https://src.fedoraproject.org/rpms/grub2/blob/71e0fb4ea646071ec86e4a65bb2424...
It's pretty unlikely that running grub can work. The libguestfs appliance is sufficiently different from how the guest boots that things like BIOS mappings are not related, and libguestfs doesn't use EFI at all.
There's unfortunately no good way to run grub. syslinux/extlinux is the only option since it is designed to make modifications to offline filesystems, and grub is not.
Rich.
Well, at the moment in my code in and in virt-v2v, that's what we do: https://github.com/libguestfs/virt-v2v/blob/master/v2v/linux_bootloaders.ml#... I thought I could use my g.mkdir_p('/sys/firmware/efi') hack, but since it didn't work, I'm now resort to explicitly handling this failure after running the mkconfig command: def _fix_grub2_config_efi(self, g): # On rhel7.8 the grub config is set to non efi commands due to # a weak uefi detection algorithm which doesn't work from a chrooted # environment (checks for [ -d /sys/firmware/efi ]) if (self.root.family, self.boot_type) != (LinuxFamily.RHEL, BootType.UEFI): return grub_config = g.read_file(self.config).decode('utf-8') write = False for old, new in [('linux16', 'linuxefi'), ('initrd16', 'initrdefi')]: if f'\t{old} ' in grub_config: debug(f'Replacing {old} with {new} in {self.config}') grub_config = grub_config.replace(f'\t{old} ', f'\t{new} ') write = True if write: g.write(self.config, grub_config.encode('utf-8')) The problem is easily reproduced by using the 7.8 iso available now from rhel, installed on VMware as UEFI. Should probably be reproduced in virt-v2v too. The problematic file is 10_linux which belongs to: grub2-tools-2.02-0.81.el8.x86_64 Sam
I thought I could simply g.mkdir_p('/sys/firmware/efi') - but that created a fake dir in my chrooted environment, where grub2-mkconfig somehow was inspecting the real /sys as demonstrated in the above 'ls' example.
Any ideas on why sh jailbreaks the chroot? is /sys set through some environment variable?
Thanks!
_______________________________________________ Libguestfs mailing list Libguestfs@redhat.com https://www.redhat.com/mailman/listinfo/libguestfs
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v
On Thu, Aug 27, 2020 at 11:24:57AM +0300, Sam Eiderman wrote:
Yes, I understand that, I am just surprised to know that we bind /sys for g.sh() commands.
However, "g.command([self.mkconfig, '-o', self.config])" shouldn't have the appliances /sys available (I looked briefly in command.c and I don't see the same bind-mount code), or maybe I'm missing something.
guestfs_sh, guestfs_sh_lines, guestfs_command, guestfs_command_lines all attempt to bind-mount /sys and other directories into the chroot while running the command. (See daemon/sh.c) Anyway I misunderstood before - I thought you were trying to run grub to install the bootloader. Running grub2-mkconfig as done in virt-v2v should be fine since AIUI it's just concatenating some configuration files together.
I thought I could use my g.mkdir_p('/sys/firmware/efi') hack, but since it didn't work, I'm now resort to explicitly handling this failure after running the mkconfig command:
The mkdir didn't work since that directory will be covered by the bind mount when the command is running. I'm not sure there's a good answer here, maybe something like: g.sh (" mkdir -p /sys/firmware/efi # create some files in /sys/firmware/efi to trick grub grub2-mkconfig -o ... ") although this may modify the appliance's real /sys. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org
I see, thanks for the clarification. On Thu, Aug 27, 2020 at 12:05 PM Richard W.M. Jones <rjones@redhat.com> wrote:
On Thu, Aug 27, 2020 at 11:24:57AM +0300, Sam Eiderman wrote:
Yes, I understand that, I am just surprised to know that we bind /sys for g.sh() commands.
However, "g.command([self.mkconfig, '-o', self.config])" shouldn't have the appliances /sys available (I looked briefly in command.c and I don't see the same bind-mount code), or maybe I'm missing something.
guestfs_sh, guestfs_sh_lines, guestfs_command, guestfs_command_lines all attempt to bind-mount /sys and other directories into the chroot while running the command. (See daemon/sh.c)
Anyway I misunderstood before - I thought you were trying to run grub to install the bootloader. Running grub2-mkconfig as done in virt-v2v should be fine since AIUI it's just concatenating some configuration files together.
I thought I could use my g.mkdir_p('/sys/firmware/efi') hack, but since it didn't work, I'm now resort to explicitly handling this failure after running the mkconfig command:
The mkdir didn't work since that directory will be covered by the bind mount when the command is running.
I'm not sure there's a good answer here, maybe something like:
g.sh (" mkdir -p /sys/firmware/efi # create some files in /sys/firmware/efi to trick grub grub2-mkconfig -o ... ")
although this may modify the appliance's real /sys.
Rich.
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org
I ended up with the following logic (which is pretty close to what you suggested, had to unmount sysfs first to create directories there) def update(self, g): if self.boot_type == BootType.UEFI: # On rhel7 and sles12/15 the grub config is set to non efi commands # due to a weak uefi detection algorithm which doesn't work from a # chrooted environment (checks for [ -d /sys/firmware/efi ]). # libguestfs bind-mounts the sysfs of the appliance to /sys when # invoking g.sh()/command() - Therefore, /sys/firmware/efi does # not exist. Hack around this behavior by unmount /sys and # creating /sys/firmware/efi dir before running grub2's mkconfig. g.sh(f'umount /sys && mkdir -p /sys/firmware/efi && ' f'{self.mkconfig} -o {self.config}') else: g.command([self.mkconfig, '-o', self.config]) return True I'm pretty sure that should be fixed in virt-v2v too (although I'm a bit lazy do write that OCaml equivalent of this and test it). So just a heads up on the mailing list. Sam On Thu, Aug 27, 2020 at 12:28 PM Sam Eiderman <sameid@google.com> wrote:
I see, thanks for the clarification.
On Thu, Aug 27, 2020 at 12:05 PM Richard W.M. Jones <rjones@redhat.com> wrote:
On Thu, Aug 27, 2020 at 11:24:57AM +0300, Sam Eiderman wrote:
Yes, I understand that, I am just surprised to know that we bind /sys for g.sh() commands.
However, "g.command([self.mkconfig, '-o', self.config])" shouldn't have the appliances /sys available (I looked briefly in command.c and I don't see the same bind-mount code), or maybe I'm missing something.
guestfs_sh, guestfs_sh_lines, guestfs_command, guestfs_command_lines all attempt to bind-mount /sys and other directories into the chroot while running the command. (See daemon/sh.c)
Anyway I misunderstood before - I thought you were trying to run grub to install the bootloader. Running grub2-mkconfig as done in virt-v2v should be fine since AIUI it's just concatenating some configuration files together.
I thought I could use my g.mkdir_p('/sys/firmware/efi') hack, but since it didn't work, I'm now resort to explicitly handling this failure after running the mkconfig command:
The mkdir didn't work since that directory will be covered by the bind mount when the command is running.
I'm not sure there's a good answer here, maybe something like:
g.sh (" mkdir -p /sys/firmware/efi # create some files in /sys/firmware/efi to trick grub grub2-mkconfig -o ... ")
although this may modify the appliance's real /sys.
Rich.
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org
participants (2)
-
Richard W.M. Jones -
Sam Eiderman