SELinux relabeling: do it by default
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1554735 Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2075718 I'm going to post four patches or patch-sets in response to this email. Due to how the libguestfs-common module is organized & consumed, and how the generator in libguestfs works, this work is very awkward. (See more below.) The idea is to *replace* "--selinux-relabel" with "--no-selinux-relabel", and to invert the SELinux relabeling choice: that is, to do it by default, and allow users to prevent it with "--no-selinux-relabel" if they desire so. This is being requested in the above two BZs. I've intentionally avoided introducing "--no-selinux-relabel" *in addition* to "--selinux-relabel". While some utilities support a similar dual form (such as virt-builder's "--network" and "--no-network"), with one being the default, those options are special in that they are *not shared* between different utilities, and they are not generated by the generator in libguestfs. The key difference is that the *non-shared* options use Getopt.Set and Getopt.Clear on the *same* boolean reference cell, whereas the generator introduces a *separate* boolean reference cell for each option it generates (and then it uses *either* Getopt.Clear *or* Getopt.Set when the option is passed on the command line, dependent on the default value of the reference cell). This means that "--no-selinux-relabel" and "--selinux-relabel", if they both existed, would work on different booleans, and that would be the source of a lot of fun (priority? command line order? documentation? etc etc). So, nope to that. Back to the structuring of these patches / patch sets. The generator lives and runs solely in libguestfs. However, it generates such code as well that is owned by libguestfs-common. Normally we don't notice, because the generator overwrites "common" submodule contents with identical files; thus, "git" does not complain about the submodule checkout being modified locally. This no longer holds with these patches. Therefore: - as first step, libguestfs needs to be modified - the generator is run as a part of "make", which creates a local diff in the "common" submodule checkout under the libguestfs worktree - that diff is reflected to, and captured as a commit, in libguestfs-common - this returns libguestfs to an "everything in sync" state, but more importantly - it exposes the new stuff to virt-v2v and guestfs-tools, - virt-v2v and guestfs-tools need to be updated to consider the disappearance of "--selinux-relabel". The fact that documentation and test cases are shared in various ways only makes this more complicated. For example, the virt-builder(1) manual speaks words on SELinux in the auto-generated (and shared), and the private (non-shared) sections *both*. One thing to note is that libguestfs itself does not *consume* the particular "common" contents that it generates. Therefore we don't have a reference loop in practice. What we have is this dependency graph: libguestfs (generator) | v libguestfs-common (generated content) / \ v v guestfs-tools virt-v2v Because of that, the usual "update common submodule" hunk *need not* be squashed into the libguestfs (generator) patches, when merging this. However, said "update common submodule" hunk does have to be squashed into the (single) guestfs-tools and virt-v2v patches, when merging. I meticulously tested this stuff: - libguestfs: - "make check" and "make check-slow" complete fine - There is no documentation (under the "website/" subdir) that is updated by the patches. - guestfs-tools: - Checked the rendered documentation regarding "--no-selinux-relabel" that comes from "common": virt-builder.1.html virt-customize.1.html virt-sysprep.1.html - Checked the rendered documentation changes that come from guestfs-tools itself: virt-builder.1.html - Checked the "--help" output of: virt-builder virt-customize virt-sysprep - "make check" completes OK. - "make check-slow" completes OK: - PASS for test-firstboot-*.sh (Linux guests -- Windows guests are SKIPped), - same for test-settings-*.sh - except for "test-settings-ubuntu-18.04.sh". It fails for an independent reason: "libguestfs: error: download: /etc/sysconfig/network: No such file or directory" - PASS for test-selinuxrelabel.sh - "test-console-ubuntu-20.04.sh" fails for an independent reason: "didn't see login banner in serial console output" -- but no serial output was actually shown in the log. - virt-v2v: - "make check" completes OK. - "make check-slow" completes fine - in particular, PASS for test-v2v-conversion-of-*.sh (Linux guests -- Windows guests are SKIPped) Thanks, Laszlo
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1554735 Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2075718 Thanks, Laszlo Laszlo Ersek (2): generator/customize: document that "--selinux-relabel" checks for SELinux generator/customize: invert SELinux relabeling default generator/customize.ml | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) -- 2.19.1.3.g30247aa5d201
The documentation currently says that the user should avoid passing "--selinux-relabel" on the command line if the guest does not support SELinux. However, the "is_selinux_guest" helper function in "common/mlcustomize/SELinux_relabel.ml" already turns "--selinux-relabel" into a no-op if some key SELinux files are absent from the guest, so there is no need to caution the user. This change is relevant because the subsequent patches will turn on "--selinux-relabel" by default, and therefore "is_selinux_guest" will grow in importance. Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1554735 Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2075718 Signed-off-by: Laszlo Ersek <lersek@redhat.com> --- generator/customize.ml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/generator/customize.ml b/generator/customize.ml index 0b256e2d2351..3b3eec6d2370 100644 --- a/generator/customize.ml +++ b/generator/customize.ml @@ -575,7 +575,7 @@ This will attempt to relabel files immediately, but if the operation fails this will instead touch F</.autorelabel> on the image to schedule a relabel operation for the next time the image boots. -You should only use this option for guests which support SELinux."; +This option is a no-op for guests that do not support SELinux."; }; { flag_name = "sm-credentials"; -- 2.19.1.3.g30247aa5d201
Replace the "--selinux-relabel" option with "--no-selinux-relabel", inverting the default behavior (for guests with SELinux support, that is -- relabeling is always skipped for guests that don't support SELinux.) Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1554735 Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2075718 Signed-off-by: Laszlo Ersek <lersek@redhat.com> --- generator/customize.ml | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/generator/customize.ml b/generator/customize.ml index 3b3eec6d2370..9634dad85a64 100644 --- a/generator/customize.ml +++ b/generator/customize.ml @@ -564,18 +564,21 @@ to modify C</etc/sysconfig/authconfig> (Fedora, RHEL) or C</etc/pam.d/common-password> (Debian, Ubuntu)."; }; - { flag_name = "selinux-relabel"; + { flag_name = "no-selinux-relabel"; flag_type = FlagBool false (* XXX - the default in virt-builder *); - flag_ml_var = "selinux_relabel"; - flag_shortdesc = "Relabel files with correct SELinux labels"; + flag_ml_var = "no_selinux_relabel"; + flag_shortdesc = "Do not relabel files with correct SELinux labels"; flag_pod_longdesc = "\ -Relabel files in the guest so that they have the correct SELinux label. +Do not attempt to correct the SELinux labels of files in the guest. -This will attempt to relabel files immediately, but if the operation fails -this will instead touch F</.autorelabel> on the image to schedule a -relabel operation for the next time the image boots. +In such guests that support SELinux, customization automatically +relabels files so that they have the correct SELinux label. (The +relabeling is performed immediately, but if the operation fails, +customization will instead touch F</.autorelabel> on the image to +schedule a relabel operation for the next time the image boots.) This +option disables the automatic relabeling. -This option is a no-op for guests that do not support SELinux."; +The option is a no-op for guests that do not support SELinux."; }; { flag_name = "sm-credentials"; -- 2.19.1.3.g30247aa5d201
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1554735 Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2075718 Thanks Laszlo Laszlo Ersek (2): mlcustomize: refresh generated files remove non-generated "--selinux-relabel" options mlcustomize/customize-options.pod | 23 +++++++++++--------- mlcustomize/customize-synopsis.pod | 2 +- mlcustomize/customize_cmdline.ml | 16 +++++++------- mlcustomize/customize_cmdline.mli | 4 ++-- mlcustomize/test-firstboot.sh | 3 --- mlcustomize/test-selinuxrelabel.sh | 5 ++--- 6 files changed, 26 insertions(+), 27 deletions(-) -- 2.19.1.3.g30247aa5d201
Refresh the generated mlcustomize files after libguestfs patches - generator/customize: document that "--selinux-relabel" checks for SELinux - generator/customize: invert SELinux relabeling default Refer to commit 4e3a44ed1c07 ("common/{mlcustomize,mlv2v}: Add generated files to git.", 2019-10-14) for background. Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1554735 Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2075718 Signed-off-by: Laszlo Ersek <lersek@redhat.com> --- mlcustomize/customize_cmdline.mli | 4 ++-- mlcustomize/customize_cmdline.ml | 16 +++++++------- mlcustomize/customize-options.pod | 23 +++++++++++--------- mlcustomize/customize-synopsis.pod | 2 +- 4 files changed, 24 insertions(+), 21 deletions(-) diff --git a/mlcustomize/customize_cmdline.mli b/mlcustomize/customize_cmdline.mli index 14eda49eb1fb..7ee882a6499b 100644 --- a/mlcustomize/customize_cmdline.mli +++ b/mlcustomize/customize_cmdline.mli @@ -101,8 +101,8 @@ and flags = { (* --no-logfile *) password_crypto : Password.password_crypto option; (* --password-crypto md5|sha256|sha512 *) - selinux_relabel : bool; - (* --selinux-relabel *) + no_selinux_relabel : bool; + (* --no-selinux-relabel *) sm_credentials : Subscription_manager.sm_credentials option; (* --sm-credentials SELECTOR *) } diff --git a/mlcustomize/customize_cmdline.ml b/mlcustomize/customize_cmdline.ml index 9326baa00f75..5d404e84490a 100644 --- a/mlcustomize/customize_cmdline.ml +++ b/mlcustomize/customize_cmdline.ml @@ -109,8 +109,8 @@ and flags = { (* --no-logfile *) password_crypto : Password.password_crypto option; (* --password-crypto md5|sha256|sha512 *) - selinux_relabel : bool; - (* --selinux-relabel *) + no_selinux_relabel : bool; + (* --no-selinux-relabel *) sm_credentials : Subscription_manager.sm_credentials option; (* --sm-credentials SELECTOR *) } @@ -121,7 +121,7 @@ let rec argspec () = let ops = ref [] in let scrub_logfile = ref false in let password_crypto = ref None in - let selinux_relabel = ref false in + let no_selinux_relabel = ref false in let sm_credentials = ref None in let rec get_ops () = { @@ -131,7 +131,7 @@ let rec argspec () = and get_flags () = { scrub_logfile = !scrub_logfile; password_crypto = !password_crypto; - selinux_relabel = !selinux_relabel; + no_selinux_relabel = !no_selinux_relabel; sm_credentials = !sm_credentials; } in @@ -459,11 +459,11 @@ let rec argspec () = ), Some "md5|sha256|sha512", "When the virt tools change or set a password in the guest, this\noption sets the password encryption of that password to\nC<md5>, C<sha256> or C<sha512>.\n\nC<sha256> and C<sha512> require glibc E<ge> 2.7 (check crypt(3) inside\nthe guest).\n\nC<md5> will work with relatively old Linux guests (eg. RHEL 3), but\nis not secure against modern attacks.\n\nThe default is C<sha512> unless libguestfs detects an old guest that\ndidn't have support for SHA-512, in which case it will use C<md5>.\nYou can override libguestfs by specifying this option.\n\nNote this does not change the default password encryption used\nby the guest when you create new user accounts inside the guest.\nIf you want to do that, then you should use the I<--edit> option\nto modify C</etc/sysconfig/authconfig> (Fedora, RHEL) or\nC</etc/pam.d/common-password> (Debian, Ubuntu)."; ( - [ L"selinux-relabel" ], - Getopt.Set selinux_relabel, - s_"Relabel files with correct SELinux labels" + [ L"no-selinux-relabel" ], + Getopt.Set no_selinux_relabel, + s_"Do not relabel files with correct SELinux labels" ), - None, "Relabel files in the guest so that they have the correct SELinux label.\n\nThis will attempt to relabel files immediately, but if the operation fails\nthis will instead touch F</.autorelabel> on the image to schedule a\nrelabel operation for the next time the image boots.\n\nYou should only use this option for guests which support SELinux."; + None, "Do not attempt to correct the SELinux labels of files in the guest.\n\nIn such guests that support SELinux, customization automatically\nrelabels files so that they have the correct SELinux label. (The\nrelabeling is performed immediately, but if the operation fails,\ncustomization will instead touch F</.autorelabel> on the image to\nschedule a relabel operation for the next time the image boots.) This\noption disables the automatic relabeling.\n\nThe option is a no-op for guests that do not support SELinux."; ( [ L"sm-credentials" ], Getopt.String ( diff --git a/mlcustomize/customize-options.pod b/mlcustomize/customize-options.pod index 71b545da42f9..a83c80a567d0 100644 --- a/mlcustomize/customize-options.pod +++ b/mlcustomize/customize-options.pod @@ -206,6 +206,19 @@ the image was built, use this option. See also: L</LOG FILE>. +=item B<--no-selinux-relabel> + +Do not attempt to correct the SELinux labels of files in the guest. + +In such guests that support SELinux, customization automatically +relabels files so that they have the correct SELinux label. (The +relabeling is performed immediately, but if the operation fails, +customization will instead touch F</.autorelabel> on the image to +schedule a relabel operation for the next time the image boots.) This +option disables the automatic relabeling. + +The option is a no-op for guests that do not support SELinux. + =item B<--password> USER:SELECTOR Set the password for C<USER>. (Note this option does I<not> @@ -297,16 +310,6 @@ It cannot delete directories, only regular files. =back -=item B<--selinux-relabel> - -Relabel files in the guest so that they have the correct SELinux label. - -This will attempt to relabel files immediately, but if the operation fails -this will instead touch F</.autorelabel> on the image to schedule a -relabel operation for the next time the image boots. - -You should only use this option for guests which support SELinux. - =item B<--sm-attach> SELECTOR Attach to a pool using C<subscription-manager>. diff --git a/mlcustomize/customize-synopsis.pod b/mlcustomize/customize-synopsis.pod index 5f185408978c..252085380a99 100644 --- a/mlcustomize/customize-synopsis.pod +++ b/mlcustomize/customize-synopsis.pod @@ -12,5 +12,5 @@ [--truncate-recursive PATH] [--timezone TIMEZONE] [--touch FILE] [--uninstall PKG,PKG..] [--update] [--upload FILE:DEST] [--write FILE:CONTENT] [--no-logfile] - [--password-crypto md5|sha256|sha512] [--selinux-relabel] + [--password-crypto md5|sha256|sha512] [--no-selinux-relabel] [--sm-credentials SELECTOR] -- 2.19.1.3.g30247aa5d201
Two test cases pass "--selinux-relabel" explicitly. Remove this (now obsolete / default) option. Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1554735 Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2075718 Signed-off-by: Laszlo Ersek <lersek@redhat.com> --- mlcustomize/test-firstboot.sh | 3 --- mlcustomize/test-selinuxrelabel.sh | 5 ++--- 2 files changed, 2 insertions(+), 6 deletions(-) diff --git a/mlcustomize/test-firstboot.sh b/mlcustomize/test-firstboot.sh index b906997515db..24c67f3e1339 100755 --- a/mlcustomize/test-firstboot.sh +++ b/mlcustomize/test-firstboot.sh @@ -61,9 +61,6 @@ case "$guestname" in extra[${#extra[*]}]='/etc/inittab: s,^#([1-9].*respawn.*/sbin/getty.*),$1,' ;; - fedora*|rhel*|centos*) - extra[${#extra[*]}]='--selinux-relabel' - ;; *) ;; esac diff --git a/mlcustomize/test-selinuxrelabel.sh b/mlcustomize/test-selinuxrelabel.sh index 86278c63b200..caf75211346a 100755 --- a/mlcustomize/test-selinuxrelabel.sh +++ b/mlcustomize/test-selinuxrelabel.sh @@ -41,13 +41,12 @@ virt-builder "$guestname" --quiet -o "$disk" # Test #1: relabel with the default configuration works. rm -f "$disk_overlay" guestfish -- disk-create "$disk_overlay" qcow2 -1 backingfile:"$disk" -virt-customize -a "$disk" --selinux-relabel +virt-customize -a "$disk" # Test #2: relabel with no SELINUXTYPE in the configuration. rm -f "$disk_overlay" guestfish -- disk-create "$disk_overlay" qcow2 -1 backingfile:"$disk" virt-customize -a "$disk" \ - --edit /etc/selinux/config:"s,^SELINUXTYPE=,#&,g" \ - --selinux-relabel + --edit /etc/selinux/config:"s,^SELINUXTYPE=,#&,g" rm "$disk" "$disk_overlay" -- 2.19.1.3.g30247aa5d201
Remove "--selinux-relabel" options. Do not add any "--no-selinux-relabel" options; rely on the internal check for SELinux support instead ("is_selinux_guest" in "common/mlcustomize/SELinux_relabel.ml"). "--no-selinux-relabel" becomes a real option for virt-sysprep now. (Again?) Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1554735 Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2075718 Signed-off-by: Laszlo Ersek <lersek@redhat.com> --- builder/templates/make-template.ml | 8 +------- customize/customize_run.ml | 2 +- sysprep/main.ml | 2 -- builder/virt-builder.pod | 20 ++++---------------- customize/test-settings.sh | 3 --- sysprep/test-virt-sysprep-docs.sh | 2 +- 6 files changed, 7 insertions(+), 30 deletions(-) diff --git a/builder/templates/make-template.ml b/builder/templates/make-template.ml index d87349404ee4..58603242670b 100755 --- a/builder/templates/make-template.ml +++ b/builder/templates/make-template.ml @@ -256,8 +256,7 @@ let rec main () = printf "Sysprepping ...\n%!"; let cmd = sprintf "virt-sysprep --quiet -a %s%s" - (quote tmpout) - (if is_selinux_os os then " --selinux-relabel" else "") in + (quote tmpout) in if Sys.command cmd <> 0 then exit 1 ); @@ -480,11 +479,6 @@ and can_sysprep_os = function | Debian _ | Ubuntu _ -> true | FreeBSD _ | Windows _ -> false -and is_selinux_os = function - | RHEL _ | Alma _ | CentOS _ | CentOSStream _ | Fedora _ -> true - | Debian _ | Ubuntu _ - | FreeBSD _ | Windows _ -> false - and needs_uefi os arch = match os, arch with | Fedora _, Armv7 diff --git a/customize/customize_run.ml b/customize/customize_run.ml index f2ee20413ece..99b5fe14d849 100644 --- a/customize/customize_run.ml +++ b/customize/customize_run.ml @@ -415,7 +415,7 @@ let run (g : G.guestfs) root (ops : ops) = warning (f_"passwords could not be set for this type of guest") ); - if ops.flags.selinux_relabel then ( + if not ops.flags.no_selinux_relabel then ( message (f_"SELinux relabelling"); SELinux_relabel.relabel g ); diff --git a/sysprep/main.ml b/sysprep/main.ml index 087d1a17f3e8..b760618ad58a 100644 --- a/sysprep/main.ml +++ b/sysprep/main.ml @@ -132,8 +132,6 @@ let main () = [ L"mount-options" ], Getopt.Set_string (s_"opts", mount_opts), s_"Set mount options (eg /:noatime;/var:rw,noatime)"; [ L"network" ], Getopt.Set network, s_"Enable appliance network"; [ L"no-network" ], Getopt.Clear network, s_"Disable appliance network (default)"; - [ L"no-selinux-relabel" ], Getopt.Unit (fun () -> ()), - s_"Compatibility option, does nothing"; [ L"operation"; L"operations" ], Getopt.String (s_"operations", set_operations), s_"Enable/disable specific operations"; ] in let args = basic_args @ Sysprep_operation.extra_args () in diff --git a/builder/virt-builder.pod b/builder/virt-builder.pod index f7dd6cdad533..aeb505296887 100644 --- a/builder/virt-builder.pod +++ b/builder/virt-builder.pod @@ -131,12 +131,6 @@ To update the installed packages to the latest version: virt-builder debian-7 --update -For guests which use SELinux, like Fedora and Red Hat Enterprise -Linux, you may need to do SELinux relabelling after installing or -updating packages (see L</SELINUX> below): - - virt-builder fedora-27 --update --selinux-relabel - =head2 Customizing the installation There are many options that let you customize the installation. These @@ -972,7 +966,7 @@ command line. =item * -SELinux relabelling is done (I<--selinux-relabel>). +SELinux relabelling is done unless disabled with I<--no-selinux-relabel>. =back @@ -1072,8 +1066,7 @@ A typical virt-builder command would be: --install puppet \ --append-line '/etc/puppet/puppet.conf:[agent]' \ --append-line '/etc/puppet/puppet.conf:server = puppetmaster.example.com/' \ - --run-command 'systemctl enable puppet' \ - --selinux-relabel + --run-command 'systemctl enable puppet' The precise instructions vary according to the Linux distro. For further information see: @@ -1753,14 +1746,14 @@ two possible strategies it can use to ensure correct labelling: =over 4 -=item Using I<--selinux-relabel> +=item Automatic relabeling This runs L<setfiles(8)> just before finalizing the guest, which sets SELinux labels correctly in the disk image. This is the recommended method. -=item I<--touch> F</.autorelabel> +=item Using I<--no-selinux-relabel> I<--touch> F</.autorelabel> Guest templates may already contain a file called F</.autorelabel> or you may touch it. @@ -1771,11 +1764,6 @@ them, which is normal and harmless. =back -Please note that if your guest uses SELinux, and you are doing operations -on it which might create new files or change existing ones, you are -recommended to use I<--selinux-relabel>. This will help in making sure -that files have the right SELinux labels. - =head1 MACHINE READABLE OUTPUT The I<--machine-readable> option can be used to make the output more diff --git a/customize/test-settings.sh b/customize/test-settings.sh index ed4c90f2eb37..e8b492dd15be 100755 --- a/customize/test-settings.sh +++ b/customize/test-settings.sh @@ -61,9 +61,6 @@ case "$guestname" in extra[${#extra[*]}]='/etc/inittab: s,^#([1-9].*respawn.*/sbin/getty.*),$1,' ;; - fedora*|rhel*|centos*) - extra[${#extra[*]}]='--selinux-relabel' - ;; *) ;; esac diff --git a/sysprep/test-virt-sysprep-docs.sh b/sysprep/test-virt-sysprep-docs.sh index 51500b5e9799..9d0298d68557 100755 --- a/sysprep/test-virt-sysprep-docs.sh +++ b/sysprep/test-virt-sysprep-docs.sh @@ -25,4 +25,4 @@ $top_srcdir/podcheck.pl "$srcdir/virt-sysprep.pod" virt-sysprep \ --path $top_srcdir/common/options \ --insert sysprep-extra-options.pod:__EXTRA_OPTIONS__ \ --insert sysprep-operations.pod:__OPERATIONS__ \ - --ignore=--dryrun,--dump-pod,--dump-pod-options,--no-selinux-relabel + --ignore=--dryrun,--dump-pod,--dump-pod-options -- 2.19.1.3.g30247aa5d201
Remove "--selinux-relabel" options. Do not add any "--no-selinux-relabel" options; rely on the internal check for SELinux support instead ("is_selinux_guest" in "common/mlcustomize/SELinux_relabel.ml"). Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1554735 Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2075718 Signed-off-by: Laszlo Ersek <lersek@redhat.com> --- tests/test-v2v-conversion-of.sh | 7 ------- 1 file changed, 7 deletions(-) diff --git a/tests/test-v2v-conversion-of.sh b/tests/test-v2v-conversion-of.sh index 5a974d1b74c5..5c5cae7cd62a 100755 --- a/tests/test-v2v-conversion-of.sh +++ b/tests/test-v2v-conversion-of.sh @@ -53,13 +53,6 @@ fi # Some guests need special virt-builder parameters. # See virt-builder --notes "$guestname" declare -a extra -case "$guestname" in - fedora*|rhel*|centos*) - extra[${#extra[*]}]='--selinux-relabel' - ;; - *) - ;; -esac # Don't try to update Windows versions. case "$guestname" in -- 2.19.1.3.g30247aa5d201
On Tue, May 10, 2022 at 12:09:38PM +0200, Laszlo Ersek wrote:
Back to the structuring of these patches / patch sets. The generator lives and runs solely in libguestfs. However, it generates such code as well that is owned by libguestfs-common. Normally we don't notice, because the generator overwrites "common" submodule contents with identical files; thus, "git" does not complain about the submodule checkout being modified locally.
Right, this was a known compromise to make splitting up the libguestfs repo tractable. Originally the generator could write into the common/ subdirectory freely because there was only a single repo. When splitting, I chose to commit the generated files, and since they change infrequently this hasn't been a problem.
This no longer holds with these patches. Therefore:
- as first step, libguestfs needs to be modified
- the generator is run as a part of "make", which creates a local diff in the "common" submodule checkout under the libguestfs worktree
- that diff is reflected to, and captured as a commit, in libguestfs-common
- this returns libguestfs to an "everything in sync" state, but more importantly
- it exposes the new stuff to virt-v2v and guestfs-tools,
- virt-v2v and guestfs-tools need to be updated to consider the disappearance of "--selinux-relabel".
This is the right way to do it.
The fact that documentation and test cases are shared in various ways only makes this more complicated. For example, the virt-builder(1) manual speaks words on SELinux in the auto-generated (and shared), and the private (non-shared) sections *both*.
One thing to note is that libguestfs itself does not *consume* the particular "common" contents that it generates. Therefore we don't have a reference loop in practice. What we have is this dependency graph:
libguestfs (generator) | v libguestfs-common (generated content) / \ v v guestfs-tools virt-v2v
Because of that, the usual "update common submodule" hunk *need not* be squashed into the libguestfs (generator) patches, when merging this. However, said "update common submodule" hunk does have to be squashed into the (single) guestfs-tools and virt-v2v patches, when merging.
To be clear, while there isn't a separate "update common submodule" commit in libguestfs, the commit hash of libguestfs -> common submodule *will* still change (in the same commit that changes the generator)?
I meticulously tested this stuff:
- libguestfs:
- "make check" and "make check-slow" complete fine
- There is no documentation (under the "website/" subdir) that is updated by the patches.
- guestfs-tools:
- Checked the rendered documentation regarding "--no-selinux-relabel" that comes from "common":
virt-builder.1.html virt-customize.1.html virt-sysprep.1.html
- Checked the rendered documentation changes that come from guestfs-tools itself:
virt-builder.1.html
- Checked the "--help" output of:
virt-builder virt-customize virt-sysprep
- "make check" completes OK.
- "make check-slow" completes OK:
- PASS for test-firstboot-*.sh (Linux guests -- Windows guests are SKIPped),
- same for test-settings-*.sh
- except for "test-settings-ubuntu-18.04.sh". It fails for an independent reason: "libguestfs: error: download: /etc/sysconfig/network: No such file or directory"
- PASS for test-selinuxrelabel.sh
- "test-console-ubuntu-20.04.sh" fails for an independent reason: "didn't see login banner in serial console output" -- but no serial output was actually shown in the log.
- virt-v2v:
- "make check" completes OK.
- "make check-slow" completes fine
- in particular, PASS for test-v2v-conversion-of-*.sh (Linux guests -- Windows guests are SKIPped)
I looked at the patches through the mailing list archives and everything looks fine to me so you might as well push it all. If there are any problems I'm sure we'll find out soon enough. For the whole series: Acked-by: Richard W.M. Jones <rjones@redhat.com> (You don't need to add this because I guess it will mess up your carefully curated git commit hashes!) Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-builder quickly builds VMs from scratch http://libguestfs.org/virt-builder.1.html
On 05/10/22 13:10, Richard W.M. Jones wrote:
On Tue, May 10, 2022 at 12:09:38PM +0200, Laszlo Ersek wrote:
One thing to note is that libguestfs itself does not *consume* the particular "common" contents that it generates. Therefore we don't have a reference loop in practice. What we have is this dependency graph:
libguestfs (generator) | v libguestfs-common (generated content) / \ v v guestfs-tools virt-v2v
Because of that, the usual "update common submodule" hunk *need not* be squashed into the libguestfs (generator) patches, when merging this. However, said "update common submodule" hunk does have to be squashed into the (single) guestfs-tools and virt-v2v patches, when merging.
To be clear, while there isn't a separate "update common submodule" commit in libguestfs, the commit hash of libguestfs -> common submodule *will* still change (in the same commit that changes the generator)?
I'll merge the libguestfs-common patch set, and the libguestfs patch set, in unspecified order. Then there *will* be a separate commit in libguestfs that updates the submodule reference. It's just that this change will be an entirely stand-alone commit -- the submodule update hunk need not be squashed into any of the other -- actual patch -- libguestfs commits. In other words, in the git history, there will be two stages where the generator will output such content under "common" that actually differs from the then-checked-out submodule content -- but that's fine, because libguestfs itself does not "consume back" the same content. So this (temporary) difference is harmless. For guestfs-tools and virt-v2v however, the difference is not tolerable. Therefore, in each of those superprojects, I will extend the one patch for that superproject, with the submodule update. So that the submodule checkout and the dependent code advance in sync.
I looked at the patches through the mailing list archives and everything looks fine to me so you might as well push it all. If there are any problems I'm sure we'll find out soon enough.
Thanks!
For the whole series:
Acked-by: Richard W.M. Jones <rjones@redhat.com>
(You don't need to add this because I guess it will mess up your carefully curated git commit hashes!)
I will add it -- first because I have to extend the guestfs-tools and virt-v2v patches for merging, like described above, anyway; second, I need to be prepared for libguestfs-common moving forward mid-review in any case. (I shouldn't expect actual conflicts, but the commit hash of the HEAD could change.) I'll report back with the actual commit ranges. Thanks! Laszlo
On Tue, May 10, 2022 at 01:53:26PM +0200, Laszlo Ersek wrote:
On 05/10/22 13:10, Richard W.M. Jones wrote:
On Tue, May 10, 2022 at 12:09:38PM +0200, Laszlo Ersek wrote:
One thing to note is that libguestfs itself does not *consume* the particular "common" contents that it generates. Therefore we don't have a reference loop in practice. What we have is this dependency graph:
libguestfs (generator) | v libguestfs-common (generated content) / \ v v guestfs-tools virt-v2v
Because of that, the usual "update common submodule" hunk *need not* be squashed into the libguestfs (generator) patches, when merging this. However, said "update common submodule" hunk does have to be squashed into the (single) guestfs-tools and virt-v2v patches, when merging.
To be clear, while there isn't a separate "update common submodule" commit in libguestfs, the commit hash of libguestfs -> common submodule *will* still change (in the same commit that changes the generator)?
I'll merge the libguestfs-common patch set, and the libguestfs patch set, in unspecified order.
Then there *will* be a separate commit in libguestfs that updates the submodule reference. It's just that this change will be an entirely stand-alone commit -- the submodule update hunk need not be squashed into any of the other -- actual patch -- libguestfs commits.
Right, I totally missed that the hunk "*need not* be squashed". Anyway it's all good.
In other words, in the git history, there will be two stages where the generator will output such content under "common" that actually differs from the then-checked-out submodule content -- but that's fine, because libguestfs itself does not "consume back" the same content. So this (temporary) difference is harmless.
For guestfs-tools and virt-v2v however, the difference is not tolerable. Therefore, in each of those superprojects, I will extend the one patch for that superproject, with the submodule update. So that the submodule checkout and the dependent code advance in sync.
I looked at the patches through the mailing list archives and everything looks fine to me so you might as well push it all. If there are any problems I'm sure we'll find out soon enough.
Thanks!
For the whole series:
Acked-by: Richard W.M. Jones <rjones@redhat.com>
(You don't need to add this because I guess it will mess up your carefully curated git commit hashes!)
I will add it -- first because I have to extend the guestfs-tools and virt-v2v patches for merging, like described above, anyway; second, I need to be prepared for libguestfs-common moving forward mid-review in any case. (I shouldn't expect actual conflicts, but the commit hash of the HEAD could change.)
I'll report back with the actual commit ranges.
Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-builder quickly builds VMs from scratch http://libguestfs.org/virt-builder.1.html
On 05/10/22 14:21, Richard W.M. Jones wrote:
On Tue, May 10, 2022 at 01:53:26PM +0200, Laszlo Ersek wrote:
On 05/10/22 13:10, Richard W.M. Jones wrote:
On Tue, May 10, 2022 at 12:09:38PM +0200, Laszlo Ersek wrote:
One thing to note is that libguestfs itself does not *consume* the particular "common" contents that it generates. Therefore we don't have a reference loop in practice. What we have is this dependency graph:
libguestfs (generator) | v libguestfs-common (generated content) / \ v v guestfs-tools virt-v2v
Because of that, the usual "update common submodule" hunk *need not* be squashed into the libguestfs (generator) patches, when merging this. However, said "update common submodule" hunk does have to be squashed into the (single) guestfs-tools and virt-v2v patches, when merging.
To be clear, while there isn't a separate "update common submodule" commit in libguestfs, the commit hash of libguestfs -> common submodule *will* still change (in the same commit that changes the generator)?
I'll merge the libguestfs-common patch set, and the libguestfs patch set, in unspecified order.
Then there *will* be a separate commit in libguestfs that updates the submodule reference. It's just that this change will be an entirely stand-alone commit -- the submodule update hunk need not be squashed into any of the other -- actual patch -- libguestfs commits.
Right, I totally missed that the hunk "*need not* be squashed". Anyway it's all good.
In other words, in the git history, there will be two stages where the generator will output such content under "common" that actually differs from the then-checked-out submodule content -- but that's fine, because libguestfs itself does not "consume back" the same content. So this (temporary) difference is harmless.
For guestfs-tools and virt-v2v however, the difference is not tolerable. Therefore, in each of those superprojects, I will extend the one patch for that superproject, with the submodule update. So that the submodule checkout and the dependent code advance in sync.
I looked at the patches through the mailing list archives and everything looks fine to me so you might as well push it all. If there are any problems I'm sure we'll find out soon enough.
Thanks!
For the whole series:
Acked-by: Richard W.M. Jones <rjones@redhat.com>
(You don't need to add this because I guess it will mess up your carefully curated git commit hashes!)
I will add it -- first because I have to extend the guestfs-tools and virt-v2v patches for merging, like described above, anyway; second, I need to be prepared for libguestfs-common moving forward mid-review in any case. (I shouldn't expect actual conflicts, but the commit hash of the HEAD could change.)
I'll report back with the actual commit ranges.
libguestfs commit range 00b9ef239342..08c4ac90f5a3 libguestfs-common commit range 81f86a0058a9..48527b8768d7 guestfs-tools commit 19de3d1c8d4e virt-v2v commit 0c24fc6015ce Thanks! Laszlo
So I didn't realise that these commits break --selinux-relabel, eg: $ rpm -q guestfs-tools guestfs-tools-1.49.1-1.fc37.x86_64 $ virt-builder fedora-36 --selinux-relabel virt-builder: unrecognized option '--selinux-relabel' Try ‘virt-builder --help’ or consult virt-builder(1) for more information. This is kind of bad since it breaks existing scripts. We just had a bug report about this from someone who is testing CentOS Stream 9.1 (where I backported the patches already). This option should continue to exist, but do nothing. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com nbdkit - Flexible, fast NBD server with plugins https://gitlab.com/nbdkit/nbdkit
On Tue, May 24, 2022 at 11:04:49AM +0100, Richard W.M. Jones wrote:
So I didn't realise that these commits break --selinux-relabel, eg:
$ rpm -q guestfs-tools guestfs-tools-1.49.1-1.fc37.x86_64 $ virt-builder fedora-36 --selinux-relabel virt-builder: unrecognized option '--selinux-relabel' Try ‘virt-builder --help’ or consult virt-builder(1) for more information.
This is kind of bad since it breaks existing scripts. We just had a bug report about this from someone who is testing CentOS Stream 9.1 (where I backported the patches already).
This option should continue to exist, but do nothing.
RHEL 9.1 bug: https://bugzilla.redhat.com/show_bug.cgi?id=2089748 Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com nbdkit - Flexible, fast NBD server with plugins https://gitlab.com/nbdkit/nbdkit
On 05/24/22 12:04, Richard W.M. Jones wrote:
So I didn't realise that these commits break --selinux-relabel, eg:
$ rpm -q guestfs-tools guestfs-tools-1.49.1-1.fc37.x86_64 $ virt-builder fedora-36 --selinux-relabel virt-builder: unrecognized option '--selinux-relabel' Try ‘virt-builder --help’ or consult virt-builder(1) for more information.
This is kind of bad since it breaks existing scripts.
I carefully highlighted it in the cover letter: https://listman.redhat.com/archives/libguestfs/2022-May/028826.html
I've intentionally avoided introducing "--no-selinux-relabel" *in addition* to "--selinux-relabel". While some utilities support a similar dual form (such as virt-builder's "--network" and "--no-network"), with one being the default, those options are special in that they are *not shared* between different utilities, and they are not generated by the generator in libguestfs. The key difference is that the *non-shared* options use Getopt.Set and Getopt.Clear on the *same* boolean reference cell, whereas the generator introduces a *separate* boolean reference cell for each option it generates (and then it uses *either* Getopt.Clear *or* Getopt.Set when the option is passed on the command line, dependent on the default value of the reference cell). This means that "--no-selinux-relabel" and "--selinux-relabel", if they both existed, would work on different booleans, and that would be the source of a lot of fun (priority? command line order? documentation? etc etc). So, nope to that.
On 05/24/22 12:04, Richard W.M. Jones wrote:
We just had a bug report about this from someone who is testing CentOS Stream 9.1 (where I backported the patches already).
This option should continue to exist, but do nothing.
The problems I outlined above remain. What happens if someone passes both options? Are we supposed to continue documenting "--selinux-relabel"? Thanks Laszlo
On Tue, May 24, 2022 at 03:59:24PM +0200, Laszlo Ersek wrote:
On 05/24/22 12:04, Richard W.M. Jones wrote:
So I didn't realise that these commits break --selinux-relabel, eg:
$ rpm -q guestfs-tools guestfs-tools-1.49.1-1.fc37.x86_64 $ virt-builder fedora-36 --selinux-relabel virt-builder: unrecognized option '--selinux-relabel' Try ‘virt-builder --help’ or consult virt-builder(1) for more information.
This is kind of bad since it breaks existing scripts.
I carefully highlighted it in the cover letter:
https://listman.redhat.com/archives/libguestfs/2022-May/028826.html
I didn't carefully read it enough :-(
I've intentionally avoided introducing "--no-selinux-relabel" *in addition* to "--selinux-relabel". While some utilities support a similar dual form (such as virt-builder's "--network" and "--no-network"), with one being the default, those options are special in that they are *not shared* between different utilities, and they are not generated by the generator in libguestfs. The key difference is that the *non-shared* options use Getopt.Set and Getopt.Clear on the *same* boolean reference cell, whereas the generator introduces a *separate* boolean reference cell for each option it generates (and then it uses *either* Getopt.Clear *or* Getopt.Set when the option is passed on the command line, dependent on the default value of the reference cell). This means that "--no-selinux-relabel" and "--selinux-relabel", if they both existed, would work on different booleans, and that would be the source of a lot of fun (priority? command line order? documentation? etc etc). So, nope to that.
On 05/24/22 12:04, Richard W.M. Jones wrote:
We just had a bug report about this from someone who is testing CentOS Stream 9.1 (where I backported the patches already).
This option should continue to exist, but do nothing.
The problems I outlined above remain.
What happens if someone passes both options?
Are we supposed to continue documenting "--selinux-relabel"?
AIUI this option should remain in the code, but do nothing. If it can never do anything[0] then we are allowed to drop it from documentation. [0] = I'm assuming here that even for a non-SELinux guest this option would do nothing. It wouldn't do some doomed attempt to relabel it. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org
On 05/24/22 15:59, Laszlo Ersek wrote:
On 05/24/22 12:04, Richard W.M. Jones wrote:
So I didn't realise that these commits break --selinux-relabel, eg:
$ rpm -q guestfs-tools guestfs-tools-1.49.1-1.fc37.x86_64 $ virt-builder fedora-36 --selinux-relabel virt-builder: unrecognized option '--selinux-relabel' Try ‘virt-builder --help’ or consult virt-builder(1) for more information.
This is kind of bad since it breaks existing scripts.
I carefully highlighted it in the cover letter:
https://listman.redhat.com/archives/libguestfs/2022-May/028826.html
I've intentionally avoided introducing "--no-selinux-relabel" *in addition* to "--selinux-relabel". While some utilities support a similar dual form (such as virt-builder's "--network" and "--no-network"), with one being the default, those options are special in that they are *not shared* between different utilities, and they are not generated by the generator in libguestfs. The key difference is that the *non-shared* options use Getopt.Set and Getopt.Clear on the *same* boolean reference cell, whereas the generator introduces a *separate* boolean reference cell for each option it generates (and then it uses *either* Getopt.Clear *or* Getopt.Set when the option is passed on the command line, dependent on the default value of the reference cell). This means that "--no-selinux-relabel" and "--selinux-relabel", if they both existed, would work on different booleans, and that would be the source of a lot of fun (priority? command line order? documentation? etc etc). So, nope to that.
On 05/24/22 12:04, Richard W.M. Jones wrote:
We just had a bug report about this from someone who is testing CentOS Stream 9.1 (where I backported the patches already).
This option should continue to exist, but do nothing.
The problems I outlined above remain.
What happens if someone passes both options?
Are we supposed to continue documenting "--selinux-relabel"?
... is it perhaps enough if I I do not reintroduce "--selinux-relabel" to the generator (in libguestfs), only add it to guestfs-tools, namely in: - "basic_args" in "sysprep/main.ml", - to the "--ignore=" option in "sysprep/test-virt-sysprep-docs.sh"? Basically: git show --color 19de3d1c8d4e -- \ sysprep/main.ml \ sysprep/test-virt-sysprep-docs.sh And restore those hunks, except replace "--no-selinux-relabel" with "--selinux-relabel"? Does that suffice? Thanks Laszlo
On 05/25/22 07:24, Laszlo Ersek wrote:
On 05/24/22 15:59, Laszlo Ersek wrote:
On 05/24/22 12:04, Richard W.M. Jones wrote:
So I didn't realise that these commits break --selinux-relabel, eg:
$ rpm -q guestfs-tools guestfs-tools-1.49.1-1.fc37.x86_64 $ virt-builder fedora-36 --selinux-relabel virt-builder: unrecognized option '--selinux-relabel' Try ‘virt-builder --help’ or consult virt-builder(1) for more information.
This is kind of bad since it breaks existing scripts.
I carefully highlighted it in the cover letter:
https://listman.redhat.com/archives/libguestfs/2022-May/028826.html
I've intentionally avoided introducing "--no-selinux-relabel" *in addition* to "--selinux-relabel". While some utilities support a similar dual form (such as virt-builder's "--network" and "--no-network"), with one being the default, those options are special in that they are *not shared* between different utilities, and they are not generated by the generator in libguestfs. The key difference is that the *non-shared* options use Getopt.Set and Getopt.Clear on the *same* boolean reference cell, whereas the generator introduces a *separate* boolean reference cell for each option it generates (and then it uses *either* Getopt.Clear *or* Getopt.Set when the option is passed on the command line, dependent on the default value of the reference cell). This means that "--no-selinux-relabel" and "--selinux-relabel", if they both existed, would work on different booleans, and that would be the source of a lot of fun (priority? command line order? documentation? etc etc). So, nope to that.
On 05/24/22 12:04, Richard W.M. Jones wrote:
We just had a bug report about this from someone who is testing CentOS Stream 9.1 (where I backported the patches already).
This option should continue to exist, but do nothing.
The problems I outlined above remain.
What happens if someone passes both options?
Are we supposed to continue documenting "--selinux-relabel"?
... is it perhaps enough if I I do not reintroduce "--selinux-relabel" to the generator (in libguestfs), only add it to guestfs-tools, namely in:
- "basic_args" in "sysprep/main.ml", - to the "--ignore=" option in "sysprep/test-virt-sysprep-docs.sh"?
Basically:
git show --color 19de3d1c8d4e -- \ sysprep/main.ml \ sysprep/test-virt-sysprep-docs.sh
And restore those hunks, except replace "--no-selinux-relabel" with "--selinux-relabel"?
Does that suffice?
Sigh, I don't think that this will work. :( If I put the compat option in the generator, then both "virt-sysprep" and "virt-customize" will get the option. If I only put the compat option into "sysprep/main.ml", as described above, then virt-customize will *not* get the option. This can easily be tested with virt-customize and virt-sysprep binaries that *predate* this particular patch set (that is, binaries that predate guestfs-tools commit 19de3d1c8d4e): $ virt-sysprep --no-selinux-relabel virt-sysprep: error: you must give either -a or -d options. Read virt-sysprep(1) man page for further information. ... $ virt-customize --no-selinux-relabel virt-customize: unrecognized option '--no-selinux-relabel' That is, "virt-sysprep" recognizes the historical compat option "--no-selinux-relabel", but "virt-customize" doesn't. This means that I cannot put the now-necessary "--selinux-relabel" option into just "sysprep/main.ml", it needs to go into the generator. Dang. Laszlo
participants (2)
-
Laszlo Ersek -
Richard W.M. Jones