ANNOUNCE: libguestfs 1.48 and guestfs-tools 1.48 released
by Richard W.M. Jones
I'm pleased to announce the simultaneous releases of libguestfs 1.48
and guestfs-tools 1.48. These are a library and a set of convenient
tools for accessing and modifying filesystems inside virtual machines
and disk images, securely and without needing root access.
You can download both projects here:
https://download.libguestfs.org/1.48-stable/
https://download.libguestfs.org/guestfs-tools/1.48-stable/
Release notes are attached below or online here:
https://libguestfs.org/guestfs-release-notes-1.48.1.html
https://libguestfs.org/guestfs-tools-release-notes-1.48.1.html
Rich.
----------------------------------------------------------------------
RELEASE NOTES FOR LIBGUESTFS 1.48
These are the release notes for libguestfs stable release 1.48. This
describes the major changes since 1.46.
Libguestfs 1.48.0 was released on 14 March 2022.
Some features removed
The following little used, experimental or unsafe features have been
removed:
"Libguestfs live" was an experimental feature that allowed you (in
theory) to connect to an existing instance of the libguestfs daemon in
a running guest. If you want to do this we recommend installing qemu-
guest-agent in the guest instead and using the APIs provided by it.
The related "unix:" backend has been removed. The guestfish(1) and
guestmount(1) --live option now prints an error.
The User-mode Linux ("uml") backend has been removed. It was an
alternate hypervisor that could be used instead of qemu. It had many
limitations including lack of network support, and lack of qemu native
features like qcow2 or network-backed disks. In addition it was slower
than KVM and particular features like bulk upload/download were
painfully slow. Also the UML feature of Linux that it was based on
often does not work.
The 9pfs APIs "guestfs_list_9p" and "guestfs_mount_9p" have been
deprecated and now return errors. This was related to an experimental
feature for passing through host filesystems to the appliance. You had
to hand-hack the qemu command line for this to work and it did not fit
the libguestfs model very well.
"guestfs_add_drive" no longer supports hotplugging (adding a drive
after launch), and "guestfs_remove_drive" has been deprecated and now
returns an error. Hotplugging was an unsuccessful feature which was
complicated to implement and maintain, and was slow and insecure. It
also only worked with the non-default libvirt backend, and was never
implemented in the default "direct" backend.
Language bindings
Fix allocation and deallocation of string lists in golang bindings
(Laszlo Ersek).
Multiple fixes to the OCaml bindings: Fix warnings, move minimum
version to OCaml 4.04, replace various deprecated OCaml features with
modern OCaml features.
Running "make distclean" in the rust directory now completely cleans
the cargo temporary files.
Inspection
Fix inspection of modern Pardus GNU/Linux (Laszlo Ersek).
Add support for Kylin (Laszlo Ersek).
Add support for Rocky Linux (Neil Hanlon).
Inspection of guests which use LUKS encryption on top of LVM logical
volumes should now work (Laszlo Ersek).
API
New APIs
No new APIs were added in libguestfs 1.48.
Other API changes
"guestfs_list_9p" and "guestfs_mount_9p" are deprecated and now return
errors.
"guestfs_remove_drive" has been deprecated and now returns an error.
In "guestfs_xfs_admin" the "lazycounter" parameter is deprecated
because it is no longer supported in recent versions of XFS.
Partition APIs now cope with the broken MBR partition tables created by
dosfstools ≥ 4.2 (Laszlo Ersek).
In "guestfs_md_create" the "chunk" parameter should not be used with
RAID 1. Previously it was allowed but ignored, but now it will give an
error (Laszlo Ersek).
Build changes
Multiple compiler warnings fixed and/or analyzed and suppressed (Laszlo
Ersek).
Fix detection of optional Yara library using pkg-config instead of
checking for library/header file (Laszlo Ersek).
Update valgrind suppressions for glibc 2.35, OCaml 4.13 and libvirt
7.7. "make check-valgrind" works again.
If using libvirt ≥ 3.8.0, we now use <interface> to create the
appliance network interface instead of adding qemu command line
parameters (Laszlo Ersek).
The guestfs-release-notes(1) page was a broken link, but it has now
been restored as a new page that links to the release notes from each
stable release.
Internals
The libc API strerror_r(3) is difficult to use correctly because of
differing signatures in glibc and other libc. A wrapper
"guestfs_int_strerror" has been created to hide the differences.
Bugs fixed
https://bugzilla.redhat.com/2030709
[RFE] libguestfs: Add Rocky Linux support
https://bugzilla.redhat.com/2030396
Fix usage of strerror_r which caused corrupted error messages
https://bugzilla.redhat.com/2024022
xfs-admin: Cannot disable lazy-counters on V5 fs
https://bugzilla.redhat.com/1995391
RFE: virt-inspector fails to detect the distro and applications on
Kylin-Desktop-V10 image
https://bugzilla.redhat.com/1993842
RFE: virt-inspector fails to detect the distro on Pardus-19.5 image
https://bugzilla.redhat.com/1931821
work around mkfs.vfat writes corrupted filesystem/partition table
when used on whole block device
https://bugzilla.redhat.com/1451665
RFE: Virt-v2v can't convert the guest which has encrypted partition
SEE ALSO
guestfs-examples(1), guestfs-faq(1), guestfs-performance(1),
guestfs-recipes(1), guestfs-testing(1), guestfs(3), guestfish(1),
http://libguestfs.org/
AUTHORS
Laszlo Ersek
Neil Hanlon
Richard W.M. Jones
COPYRIGHT
Copyright (C) 2009-2022 Red Hat Inc.
----------------------------------------------------------------------
RELEASE NOTES FOR GUESTFS TOOLS 1.48
These are the release notes for guestfs tools stable release 1.48.
This describes the major changes since 1.46.
Guestfs tools 1.48.0 was released on 14 March 2022.
virt-builder
New templates: "alma-8.5", "centosstream-8", "debian-11", "fedora-34",
"fedora-34" (armv7l), "fedora-35" "fedora-35" (aarch64) (Lars Kaiser,
Stef Walter).
New fedora-34 (armv7l) template is built to use UEFI.
virt-customize
The yescrypt password hashing method is now supported (Björn Esser).
virt-inspector
Recognise "kalilinux", "kylin" and "msdos" operating systems / Linux
distros (Laszlo Ersek).
Inspection of LUKS encrypted filesystems placed directly on logical
volumes is now supported, but you will need libguestfs ≥ 1.48 (Laszlo
Ersek).
virt-sysprep
Add a new default operation called "net-nmconn" which removes
NetworkManager connection profiles (Laszlo Ersek).
virt-win-reg
"virt-win-reg --version" now prints both the version of guestfs tools
and the libguestfs library that it is linked with.
Common changes
All OCaml tools now suppress line wrapping when output is redirected to
a file (to make it easier to parse logs with external programs). There
is a new --wrap option which can be used to reenable line wrapping to
files if needed (Laszlo Ersek).
Build changes
Multiple fixes to make it easier to build against either the system
installed libguestfs or a locally built libguestfs (Laszlo Ersek).
Fix multiple OCaml warnings. Remove some deprecated OCaml features.
We now require OCaml ≥ 4.04.
Multiple fixes to valgrind support for glibc 2.35, OCaml 4.13 and
libvirt 7.7. "make check-valgrind" should now work.
Internals
New OCaml "On_exit" module (from libguestfs-common) is used instead of
custom on-exit and signal handling.
Bugs fixed
https://bugzilla.redhat.com/2003706
virt-install armv7 guest on aarch64 host doesn't work, no LPAE
support and no PCI devices
https://bugzilla.redhat.com/1995391
RFE: virt-inspector fails to detect the distro and applications on
Kylin-Desktop-V10 image
https://bugzilla.redhat.com/1980922
virt-sysprep doesn't cleanup NetworkManager connection files
https://bugzilla.redhat.com/1961160
The version is incorrect when running 'virt-win-reg --version'
https://bugzilla.redhat.com/1820221
Log lines wrapping make log parsing difficult
https://bugzilla.redhat.com/1658126
Virt-v2v can't convert guest using LUKS-on-LV
https://bugzilla.redhat.com/1633328
Armv7 guest fails to boot with qemu-3.0.0-1
SEE ALSO
http://libguestfs.org/
AUTHORS
Björn Esser
Lars Kaiser
Laszlo Ersek
Richard W.M. Jones
Stef Walter
COPYRIGHT
Copyright (C) 2009-2022 Red Hat Inc.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines. Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top
2 years, 8 months
Running lcitool in nbdkit
by Richard W.M. Jones
Am I doing this wrong? [see log below] This is using the upstream
libvirt-ci checked out just now from gitlab, but it seems as if nbdkit
has to be listed as a project under that repo.
"./lcitool projects" shows libnbd but not nbdkit.
A second question, since the libvirt-ci project does a bunch of stuff
with ansible & containers, regenerating the files like this doesn't
require ansible & containers? (I don't want to run those on my
development machine.)
$ ../libvirt-ci/lcitool manifest ci/manifest.yml
Generating ci/containers/almalinux-8.Dockerfile
[ERROR]: An unexpected error occurred
Traceback (most recent call last):
File "/home/rjones/d/libvirt-ci/guests/lcitool/lcitool/projects.py", line 121, in get_packages
obj = self.projects[proj]
KeyError: 'nbdkit'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/rjones/d/libvirt-ci/guests/lcitool/bin/lcitool", line 36, in <module>
Application().run(args)
File "/home/rjones/d/libvirt-ci/guests/lcitool/lcitool/application.py", line 392, in run
args.func(self, args)
File "/home/rjones/d/libvirt-ci/guests/lcitool/lcitool/application.py", line 387, in _action_manifest
manifest.generate(args.dry_run)
File "/home/rjones/d/libvirt-ci/guests/lcitool/lcitool/manifest.py", line 149, in generate
generated = self._generate_containers(dryrun)
File "/home/rjones/d/libvirt-ci/guests/lcitool/lcitool/manifest.py", line 200, in _generate_containers
return self._generate_formatter(dryrun,
File "/home/rjones/d/libvirt-ci/guests/lcitool/lcitool/manifest.py", line 191, in _generate_formatter
payload = formatter.format(target,
File "/home/rjones/d/libvirt-ci/guests/lcitool/lcitool/formatters.py", line 482, in format
facts, cross_arch, varmap = self._generator_prepare(target,
File "/home/rjones/d/libvirt-ci/guests/lcitool/lcitool/formatters.py", line 138, in _generator_prepare
varmap = self._generator_build_varmap(facts,
File "/home/rjones/d/libvirt-ci/guests/lcitool/lcitool/formatters.py", line 163, in _generator_build_varmap
self)._generator_build_varmap(facts,
File "/home/rjones/d/libvirt-ci/guests/lcitool/lcitool/formatters.py", line 87, in _generator_build_varmap
pkgs.update(projects.get_packages(selected_projects, facts, cross_arch))
File "/home/rjones/d/libvirt-ci/guests/lcitool/lcitool/projects.py", line 123, in get_packages
obj = self.internal_projects[proj]
KeyError: 'nbdkit'
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines. Supports shell scripting,
bindings from many languages. http://libguestfs.org
2 years, 8 months
Re: [Libguestfs] guestfish with support Microsoft ReFS
by Richard W.M. Jones
[Please always CC the mailing list]
On Mon, Mar 14, 2022 at 09:24:08AM +0100, Gottschalk wrote:
> Hello Rich,
>
> i installed the ReFS driver on a Testsystem.
>
> But guestfish doesn't find the ReFS driver which is a Kernelmodul.
>
> To mount a ReFS Partition i have to type the following command:
>
> mount -t urefs /dev/sdb2 /mnt
>
> Can you solve the problem?
There is probably a package of userspace tooling containing
'mount.urefs' and similar programs. You'll need to add that to the
libguestfs appliance, by adding the package name to
appliance/packagelist.in (if building from source) or
$libdir/guestfs/supermin.d/packages (if using the system libguestfs
package).
Note this will only work correctly if the userspace tooling is
packaged and installed as a distro package, ie. an RPM or .deb. If
they are doing stupid stuff then you'll need to list the files
separately in 'hostfiles' which is more complicated.
You can check whether or not the right tools get included in the
appliance using virt-rescue.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html
2 years, 8 months
[PATCH libnbd 0/3] Disablble pread buffer initiialization
by Nir Soffer
Disbaling pread initialization is atually measurable and gives small speedup in
nbdcopy and some examples.
Some cases are safer; in copy-libev example, we allocate all buffers with
calloc() and resuse them for the entire copy, so the initialization is
completly useless.
In nbdcopy and Go aio_copy example, we allocate new buffer using malloc(), so
when working with bad server that does not return all data chunks in structured
reply, we can leak uninitialized memory to the destination server. I think this
issue should be solved by libnbd; it should verfify that the server return all
the expected chunks and fail the request if not.
Nir Soffer (3):
golang: examples: Do not initialize pread buffer
examples: copy-libev: Do not initialize pread buffer
copy: Do not initialize read buffer
copy/nbd-ops.c | 1 +
examples/copy-libev.c | 6 ++++++
golang/examples/aio_copy/aio_copy.go | 5 +++++
golang/examples/simple_copy/simple_copy.go | 5 +++++
4 files changed, 17 insertions(+)
--
2.35.1
2 years, 8 months
Re: [Libguestfs] libguestfs question
by Richard W.M. Jones
On Wed, Mar 09, 2022 at 08:24:27PM -0500, Lonnie Cumberland wrote:
[...]
> The second part of this is that once the application creates the
> virtual disk and adds the database files, I want to keep the handle
> open to those files so that as they are used in the application then
> the database files (which are in the virtual disk) will grow as data
> is added so that when the application closes then all that remains
> to do is to close the virtual disk.
No, disk images don't work in this way. You will simply get data
corruption if you try to do this.
If you want to share files between the host and the guest then you
could try using a network filesystem like NFS; or virtiofsd:
https://qemu.readthedocs.io/en/latest/tools/virtiofsd.html
Ric.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html
2 years, 8 months
[libguestfs-common PATCH] mlcustomize/firstboot: take an optional priority in "add_firstboot_script"
by Laszlo Ersek
Enable callers of "add_firstboot_script" to pass an (optional) priority
parameter. With this, groups of scripts can be formed with well-defined
(lexicographical) ordering between the groups. Within a group, the usual
order of registration persists.
Suggested-by: Richard W.M. Jones <rjones(a)redhat.com>
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1788823
Signed-off-by: Laszlo Ersek <lersek(a)redhat.com>
---
mlcustomize/firstboot.mli | 16 ++++++++++++----
mlcustomize/firstboot.ml | 9 +++++++--
2 files changed, 19 insertions(+), 6 deletions(-)
diff --git a/mlcustomize/firstboot.mli b/mlcustomize/firstboot.mli
index fb30886184cc..e6a0d1f70102 100644
--- a/mlcustomize/firstboot.mli
+++ b/mlcustomize/firstboot.mli
@@ -16,16 +16,24 @@
* 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
*)
-val add_firstboot_script : Guestfs.guestfs -> string -> string -> string -> unit
- (** [add_firstboot_script g root name content] adds a firstboot
- script called [name] containing [content].
+val default_prio : int
+
+val add_firstboot_script : Guestfs.guestfs -> string -> ?prio:int -> string ->
+ string -> unit
+ (** [add_firstboot_script g root prio name content] adds a firstboot
+ script called [name] containing [content] with priority [prio].
[content] is the contents of the script, {b not} a filename.
The actual name of the script on the guest filesystem is made of [name]
with all characters but alphanumeric replaced with dashes.
- The scripts are run in the order they are registered.
+ Within a given priority, the scripts are run in the order they are
+ registered. A group of scripts with a numerically lower priority is run
+ before a group with a numerically greater priority. If [prio] is omitted,
+ it is taken as [default_prio], that is, 5000. If [prio] is smaller than 0
+ or greater than 9999, an Assert_failure is raised (the [prio] parameter
+ is not expected to depend on external data).
For Linux guests using SELinux you should make sure the
filesystem is relabelled after calling this. *)
diff --git a/mlcustomize/firstboot.ml b/mlcustomize/firstboot.ml
index c3ebfd910d0b..18dc04efa2e6 100644
--- a/mlcustomize/firstboot.ml
+++ b/mlcustomize/firstboot.ml
@@ -345,12 +345,17 @@ end
let script_count = ref 0
-let add_firstboot_script (g : Guestfs.guestfs) root name content =
+let default_prio = 5000
+
+let add_firstboot_script (g : Guestfs.guestfs) root ?(prio = default_prio) name
+ content =
+ assert (prio >= 0 && prio <= 9999);
let typ = g#inspect_get_type root in
let distro = g#inspect_get_distro root in
let major = g#inspect_get_major_version root in
incr script_count;
- let filename = sprintf "%04d-%s" !script_count (sanitize_name name) in
+ let filename = sprintf "%04d-%04d-%s" prio !script_count
+ (sanitize_name name) in
match typ, distro with
| "linux", _ ->
Linux.install_service g root distro major;
base-commit: 60efc5407552c12688a5e99b7a54d23d5990aa8f
--
2.19.1.3.g30247aa5d201
2 years, 8 months
[PATCH 0/4] Remove some experimental features from libguestfs 1.48
by Richard W.M. Jones
I think we should take the opportunity of libguestfs 1.48 to remove
some features which are experimental, inadvisable or both. This will
also reduce the delta between Fedora and RHEL.
The commit messages outline why in my opinion it's right to remove
each feature.
This doesn't break API/ABI (as in, compiling and linking will still
work). Of course if you were using one of these features then that
sucks because the APIs will return errors instead of doing what you
expected. None of our existing tools use any of these features.
(I wouldn't necessarily bother with detailed reviews of each patch --
all the tests pass and if there's something I've overlooked we can
remove that later. It's more about do we agree on this as a general
direction of travel.)
Rich.
2 years, 8 months
Re: [Libguestfs] libnbd | Failed pipeline for master | 0cd77478
by Richard W.M. Jones
On Sun, Mar 06, 2022 at 08:28:09PM +0000, GitLab wrote:
> GitLab
> ✖ Pipeline #485634933 has failed!
>
> Project nbdkit / libnbd
> Branch ● master
> Commit ● 0cd77478
> copy: Minor cleanups Minor fixes suggested by ...
> Commit Author ● Nir Soffer
>
> Pipeline #485634933 triggered by ● Nir Soffer
> had 2 failed jobs.
> Failed jobs
> ✖ builds x86_64-centos-8
> ✖ builds x86_64-ubuntu-2004
I'll fix this. In brief it happened because centos:8 is no longer a
thing (sadly), we have to replace it with almalinux; and the Ubuntu
problem looks like a temporary failure.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW
2 years, 8 months
