A common problem that people have with virt-builder and virt-sysprep is which guests that use SELinux, like Fedora and RHEL. In both cases we touch /.autorelabel in the guest, which means the guest has to reboot once during its first boot. Recap: SELinux file labels -------------------------- SELinux requires that files have labels. Access to a file is controlled by the label on that file. For example: $ ls -lZ /etc/passwd -rw-r--r--. root root system_u:object_r:passwd_file_t:s0 /etc/passwd <----------- label -----------> The SELinux policy allows (eg) /usr/sbin/passwd to write to this file, based on the process label and the target file label. (The path "/etc/passwd" is not considered.) On a running system, files created from scratch are labelled by a daemon (restorecond, I think). Also they could be labelled by programs that create files such as 'rpm'. The label is actually stored in an extended attribute (xattr): $ getfattr -d --match=.* /etc/passwd getfattr: Removing leading '/' from absolute path names # file: etc/passwd security.selinux="system_u:object_r:passwd_file_t:s0" Libguestfs & SELinux labelling ------------------------------ Unfortunately SELinux requires a policy to be loaded into the kernel in order to know how to label files. There is no default labelling scheme for files, it depends entirely on the policy. Since the policy is guest specific, libguestfs cannot really load it (for a start, it wouldn't necessarily know where to load it from, and it might not be safe to load an untrusted policy). However libguestfs can do one of four things: (1) It can set the security.selinux xattr to some value. It has to know or be able to guess what value to set it to. Note there is no policy, so we have to infer this based on other things we know about the guest such as inspection data. (2) It can copy the security.selinux xattr from another file. (3) It can update the file in such a way that the xattr doesn't change, eg. appending to the file. (4) It can touch '/.autorelabel' which causes an SELinux enabled guest to do a full filesystem relabel at first boot (followed by a reboot). Virt-sysprep labelling ---------------------- As you may have guessed, currently virt-sysprep takes the easy option (4) above. Virt-sysprep tracks if any file has been created, and if so and the guest looks like an SELinux distro, it touches '/.autorelabel'. Since the 'hostname' operation is (a) enabled by default and (b) always creates the hostname file with default content even if the 'virt-sysprep --hostname' option was not used, almost any SELinux-enabled guest that virt-sysprep touches is going to have /.autorelabel at the end. Virt-builder labelling ---------------------- Virt-builder does even less than virt-sysprep. In particular it doesn't touch /.autorelabel. However the prepared templates that we ship with virt-builder were all virt-sysprepped before being uploaded, and so all have /.autorelabel in them. What files virt-sysprep creates or modifies ------------------------------------------- This is my analysis of what files virt-sysprep creates or modifies: firstboot: - creates scripts and more (brand new files) - shared with virt-builder reconfiguration: - creates /.unconfigured (brand new) hostname: - creates various files depending on distro (eg /etc/hostname) - files probably exist already, but might not - shared with virt-builder machine_id: - creates empty /etc/machine-id - probably this file exists already net_hostname: - modifies /etc/sysconfig/network-scripts/ifcfg-* net_hwaddr: - modifies /etc/sysconfig/network-scripts/ifcfg-* pacct_log: - touches /var/account/pacct or /var/log/account/pacct password: - edits /etc/shadow directly - shared with virt-builder random_seed: - creates seed file - seed file may or may not exist before - shared with virt-builder timezone: - creates/updates /etc/localtime symlink - file almost certainly exists before, but might be a regular file - shared with virt-builder user_account: - uses Augeas to edit /etc/passwd, /etc/shadow, /etc/group - Augeas does not preserve or set SELinux labels [in this case] What files virt-builder creates or modifies ------------------------------------------- edit option: - edits a file - copies xattrs from old file to new file (thanks Pino!) link, mkdir, upload, write options: - creates new links/directories/files under user control - currently no way to specify the SELinux label of the new file - would user know how to specify it? install, update options: - complicated: see longer section below Plus, shared modules with virt-sysprep, see notes above: - firstboot - hostname - password - random_seed - timezone Virt-builder --install and --update options ------------------------------------------- These options allow you to install new packages, or update existing packages. They run 'yum' in the context of the libguestfs appliance. The appliance normally has SELinux disabled and no policy loaded. It is plausible that RPM packages could contain the right SELinux labels, and that yum could be setting labels correctly (by setting the security.selinux xattr) even though SELinux is disabled. However I could not say for sure, so I did the following test instead: $ virt-builder fedora-20 --install /usr/sbin/tcpdump $ guestfish --ro -a fedora-20.img -i [no output] It seems from this that RPM is not setting labels, instead it relies on restorecond/etc to do the labelling of files that it creates. Conclusion ---------- I don't think there is any plausible way we can make --install or --update work without requiring a full filesystem relabel on first boot. However I think we can try harder to copy attributes from old files to new files, which will make many cases above work (especially for virt-sysprep which seems tractable). Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org