November 2021 - Libguestfs - libguestfs List Archives

[PATCH v2v v2] convert/convert_windows.ml: Handle date formats with dots instead of /
by Richard W.M. Jones 12 Nov '21

12 Nov '21

If the ShortDatePattern is yy.M.d (as can happen for US locale) we may not always reformat the date for schtasks.exe correctly. For explanation and testing see: https://bugzilla.redhat.com/show_bug.cgi?id=1895323#c46 (- #c50) Thanks: Xiaodai Wang for testing and bug fix suggestions Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1895323 --- convert/convert_windows.ml | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/convert/convert_windows.ml b/convert/convert_windows.ml index 31e16723ba..913eb0f4f1 100644 --- a/convert/convert_windows.ml +++ b/convert/convert_windows.ml @@ -400,12 +400,15 @@ popd and configure_qemu_ga files = List.iter ( fun msi_path -> - (* Windows is a trashfire. https://stackoverflow.com/a/18730884 *) + (* Windows is a trashfire. + * https://stackoverflow.com/a/18730884 + * https://bugzilla.redhat.com/show_bug.cgi?id=1895323 + *) let fb_script = sprintf "\ echo Removing any previously scheduled qemu-ga installation schtasks.exe /Delete /TN Firstboot-qemu-ga /F echo Scheduling delayed installation of qemu-ga from %s -powershell.exe -command \"$d = (get-date).AddSeconds(120); $FormatHack = ($([System.Globalization.DateTimeFormatInfo]::CurrentInfo.ShortDatePattern) -replace 'M+/', 'MM/') -replace 'd+/', 'dd/'; schtasks.exe /Create /SC ONCE /ST $d.ToString('HH:mm') /SD $d.ToString($FormatHack) /RU SYSTEM /TN Firstboot-qemu-ga /TR \\\"C:\\%s /forcerestart /qn /l+*vx C:\\%s.log\\\"\" +powershell.exe -command \"$d = (get-date).AddSeconds(120); $FormatHack = ($([System.Globalization.DateTimeFormatInfo]::CurrentInfo.ShortDatePattern) -replace 'y+', 'yyyy') -replace 'M+', 'MM') -replace 'd+', 'dd'; schtasks.exe /Create /SC ONCE /ST $d.ToString('HH:mm') /SD $d.ToString($FormatHack) /RU SYSTEM /TN Firstboot-qemu-ga /TR \\\"C:\\%s /forcerestart /qn /l+*vx C:\\%s.log\\\"\" " msi_path msi_path msi_path in Firstboot.add_firstboot_script g inspect.i_root -- 2.32.0

2 2

Re: [Libguestfs] [libguestfs/libguestfs] Provide a new "appliance" build based on 1.45.x (Issue #74)
by Richard W.M. Jones 12 Nov '21

12 Nov '21

On Fri, Nov 12, 2021 at 01:51:39AM -0800, Cédric Jeanneret wrote: > Hello there, > > It would be really nice getting a new appliance build in order to > support newest CentOS Stream 9 virtual machines running on CentOS > Stream 8. Currently, there are issues with the filesystem - the same > kind of things already happened with "8 on 7" environment some years > ago. I don't have a way to build a C9S appliance, but a Fedora one ought to work since the new XFS tools there support the two new features in RHEL 9 (bigtime and inobtcount). I uploaded a Fedora 35-based appliance here: https://download.libguestfs.org/binaries/appliance/ Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top

1 0

[PATCH v2v] convert/convert_windows.ml: Handle date formats with dots instead of /
by Richard W.M. Jones 12 Nov '21

12 Nov '21

If the ShortDatePattern is yy.M.d (as can happen for US locale) we may not always reformat the date for schtasks.exe correctly. For explanation and testing see: https://bugzilla.redhat.com/show_bug.cgi?id=1895323#c46 https://bugzilla.redhat.com/show_bug.cgi?id=1895323#c47 https://bugzilla.redhat.com/show_bug.cgi?id=1895323#c48 Thanks: Xiaodai Wang for testing and bug fix suggestions Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1895323 --- convert/convert_windows.ml | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/convert/convert_windows.ml b/convert/convert_windows.ml index 31e16723ba..eb614c8e95 100644 --- a/convert/convert_windows.ml +++ b/convert/convert_windows.ml @@ -400,12 +400,15 @@ popd and configure_qemu_ga files = List.iter ( fun msi_path -> - (* Windows is a trashfire. https://stackoverflow.com/a/18730884 *) + (* Windows is a trashfire. + * https://stackoverflow.com/a/18730884 + * https://bugzilla.redhat.com/show_bug.cgi?id=1895323 + *) let fb_script = sprintf "\ echo Removing any previously scheduled qemu-ga installation schtasks.exe /Delete /TN Firstboot-qemu-ga /F echo Scheduling delayed installation of qemu-ga from %s -powershell.exe -command \"$d = (get-date).AddSeconds(120); $FormatHack = ($([System.Globalization.DateTimeFormatInfo]::CurrentInfo.ShortDatePattern) -replace 'M+/', 'MM/') -replace 'd+/', 'dd/'; schtasks.exe /Create /SC ONCE /ST $d.ToString('HH:mm') /SD $d.ToString($FormatHack) /RU SYSTEM /TN Firstboot-qemu-ga /TR \\\"C:\\%s /forcerestart /qn /l+*vx C:\\%s.log\\\"\" +powershell.exe -command \"$d = (get-date).AddSeconds(120); $FormatHack = ($([System.Globalization.DateTimeFormatInfo]::CurrentInfo.ShortDatePattern) -replace 'M+', 'MM') -replace 'd+', 'dd'; schtasks.exe /Create /SC ONCE /ST $d.ToString('HH:mm') /SD $d.ToString($FormatHack) /RU SYSTEM /TN Firstboot-qemu-ga /TR \\\"C:\\%s /forcerestart /qn /l+*vx C:\\%s.log\\\"\" " msi_path msi_path msi_path in Firstboot.add_firstboot_script g inspect.i_root -- 2.32.0

1 0

[PATCH libnbd] common/include/checked-overflow.h: Simplify
by Nir Soffer 11 Nov '21

11 Nov '21

We don't need separate macros for size_t and uint64_t since the compiler builtins are generic. If we need to port to a complier that does not have generic builtins, it is likely that we can do this in a generic way. Signed-off-by: Nir Soffer <nsoffer(a)redhat.com> --- common/include/checked-overflow.h | 18 ++++-------------- common/utils/vector.c | 10 +++++----- 2 files changed, 9 insertions(+), 19 deletions(-) diff --git a/common/include/checked-overflow.h b/common/include/checked-overflow.h index a84b82d..ddc4b48 100644 --- a/common/include/checked-overflow.h +++ b/common/include/checked-overflow.h @@ -46,24 +46,14 @@ #error "this file may need to be ported to your compiler" #endif -/* Add two uint64_t values. *r = a + b +/* Add two values. *r = a + b * Returns true if overflow happened. */ -#define ADD_UINT64_T_OVERFLOW(a, b, r) __builtin_add_overflow((a), (b), (r)) +#define ADD_OVERFLOW(a, b, r) __builtin_add_overflow((a), (b), (r)) -/* Multiply two uint64_t values. *r = a * b +/* Multiply two values. *r = a * b * Returns true if overflow happened. */ -#define MUL_UINT64_T_OVERFLOW(a, b, r) __builtin_mul_overflow((a), (b), (r)) - -/* Add two size_t values. *r = a + b - * Returns true if overflow happened. - */ -#define ADD_SIZE_T_OVERFLOW(a, b, r) __builtin_add_overflow((a), (b), (r)) - -/* Multiply two size_t values. *r = a * b - * Returns true if overflow happened. - */ -#define MUL_SIZE_T_OVERFLOW(a, b, r) __builtin_mul_overflow((a), (b), (r)) +#define MUL_OVERFLOW(a, b, r) __builtin_mul_overflow((a), (b), (r)) #endif /* NBDKIT_CHECKED_OVERFLOW_H */ diff --git a/common/utils/vector.c b/common/utils/vector.c index 550e624..ee75915 100644 --- a/common/utils/vector.c +++ b/common/utils/vector.c @@ -49,8 +49,8 @@ generic_vector_reserve (struct generic_vector *v, size_t n, size_t itemsize) * reqcap = v->cap + n * reqbytes = reqcap * itemsize */ - if (ADD_SIZE_T_OVERFLOW (v->cap, n, &reqcap) || - MUL_SIZE_T_OVERFLOW (reqcap, itemsize, &reqbytes)) { + if (ADD_OVERFLOW (v->cap, n, &reqcap) || + MUL_OVERFLOW (reqcap, itemsize, &reqbytes)) { errno = ENOMEM; return -1; } @@ -60,9 +60,9 @@ generic_vector_reserve (struct generic_vector *v, size_t n, size_t itemsize) * newcap = v->cap + (v->cap + 1) / 2 * newbytes = newcap * itemsize */ - if (ADD_SIZE_T_OVERFLOW (v->cap, 1, &t) || - ADD_SIZE_T_OVERFLOW (v->cap, t/2, &newcap) || - MUL_SIZE_T_OVERFLOW (newcap, itemsize, &newbytes) || + if (ADD_OVERFLOW (v->cap, 1, &t) || + ADD_OVERFLOW (v->cap, t/2, &newcap) || + MUL_OVERFLOW (newcap, itemsize, &newbytes) || newbytes < reqbytes) { /* If that either overflows or is less than the minimum requested, * fall back to the requested capacity. -- 2.31.1

3 4

[PATCH libnbd] common/utils/vector.c: Improve whitespace
by Nir Soffer 11 Nov '21

11 Nov '21

- Indent the pseudo code to separate it from the comment text - Add missing blank line after return to make the flow more clear Signed-off-by: Nir Soffer <nsoffer(a)redhat.com> --- common/utils/vector.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/common/utils/vector.c b/common/utils/vector.c index b145e7d..550e624 100644 --- a/common/utils/vector.c +++ b/common/utils/vector.c @@ -46,8 +46,8 @@ generic_vector_reserve (struct generic_vector *v, size_t n, size_t itemsize) size_t reqcap, reqbytes, newcap, newbytes, t; /* New capacity requested. We must allocate this minimum (or fail). - * reqcap = v->cap + n - * reqbytes = reqcap * itemsize + * reqcap = v->cap + n + * reqbytes = reqcap * itemsize */ if (ADD_SIZE_T_OVERFLOW (v->cap, n, &reqcap) || MUL_SIZE_T_OVERFLOW (reqcap, itemsize, &reqbytes)) { @@ -57,8 +57,8 @@ generic_vector_reserve (struct generic_vector *v, size_t n, size_t itemsize) /* However for the sake of optimization, scale buffer by 3/2 so that * repeated reservations don't call realloc often. - * newcap = v->cap + (v->cap + 1) / 2 - * newbytes = newcap * itemsize + * newcap = v->cap + (v->cap + 1) / 2 + * newbytes = newcap * itemsize */ if (ADD_SIZE_T_OVERFLOW (v->cap, 1, &t) || ADD_SIZE_T_OVERFLOW (v->cap, t/2, &newcap) || @@ -74,6 +74,7 @@ generic_vector_reserve (struct generic_vector *v, size_t n, size_t itemsize) newptr = realloc (v->ptr, newbytes); if (newptr == NULL) return -1; + v->ptr = newptr; v->cap = newcap; return 0; -- 2.31.1

2 3

Re: [Libguestfs] libguestfs language bindings for CentOS/RHEL 9
by Richard W.M. Jones 11 Nov '21

11 Nov '21

On Wed, Nov 10, 2021 at 08:46:06AM -0500, Neal Gompa wrote: > Hey Richard, > > I'm looking to use the libguestfs Python, PHP, and Go bindings on > CentOS Stream 9, but it seems that these are not available right > now. Could they be made available in CentOS/RHEL 9, if not, what was > the reason for not shipping them? I believe these are shipped in > Fedora today. Hi Neal, answer depends on the particular bindings. For python3-libguestfs, we have a BZ to move these back to AppStream, so they should be usable. This should be done pretty soon: https://bugzilla.redhat.com/show_bug.cgi?id=2019914 For PHP, the bindings are currently broken upstream. Patches welcome etc ... For Golang I think most people are anticipating that you'll "vendor" (ie. bundle) the bindings in your code. While this is stupid, the language effectively forces you to do that. A longer term plan may look more like what we recently did with the libnbd bindings, but we're not there yet. Search "golang" here: https://listman.redhat.com/archives/libguestfs/2021-November/thread.html Rich. > Thanks in advance and best regards, > > -- > Neal Gompa > Senior DevOps Engineer > Datto, Inc. > www.datto.com -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-builder quickly builds VMs from scratch http://libguestfs.org/virt-builder.1.html

3 3

[PATCH nbdkit 0/2] common: Add checked-overflow macros
by Richard W.M. Jones 10 Nov '21

10 Nov '21

In common/vector/vector.c use GCC/Clang built-in overflow operators. The first patch is a neutral change which adds comments. The second patch is the actual change. Add a new header "checked-overflow.h" which has the purpose of isolating the use of the built-ins to one file (in case we need to add a new compiler later). Then use this in generic_vector_reserve. I tested this with GCC 11 and Clang 13. I verified by disassembly that "jo" (jump overflow) / "jno" is used where it was not used previously, by both compilers. Rich.

4 13

[libnbd PATCH] common/utils/vector: Better overflow handling
by Eric Blake 10 Nov '21

10 Nov '21

Check newcap * itemsize for overflow prior to calling realloc, so that we don't accidentally truncate an existing array. Set errno to ENOMEM on all failure paths, rather than leaving it indeterminate on overflow. The patch works by assuming a prerequisite that v->cap*itemsize is always smaller than size_t. Fixes: 985dfa72ae (common/utils/vector.c: Optimize vector append) --- Unless you see problems in this, I'll push this and also port it to nbdkit. common/utils/vector.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/common/utils/vector.c b/common/utils/vector.c index a4b43ce..c37f0c3 100644 --- a/common/utils/vector.c +++ b/common/utils/vector.c @@ -1,5 +1,5 @@ /* nbdkit - * Copyright (C) 2018-2020 Red Hat Inc. + * Copyright (C) 2018-2021 Red Hat Inc. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions are @@ -44,12 +44,14 @@ generic_vector_reserve (struct generic_vector *v, size_t n, size_t itemsize) size_t reqcap, newcap; reqcap = v->cap + n; - if (reqcap < v->cap) + if (reqcap * itemsize < v->cap * itemsize) { + errno = ENOMEM; return -1; /* overflow */ + } newcap = (v->cap * 3 + 1) / 2; - if (newcap < reqcap) + if (newcap * itemsize < reqcap * itemsize) newcap = reqcap; newptr = realloc (v->ptr, newcap * itemsize); -- 2.33.1

4 8

Minimum OCaml compiler version (2021/2022 edition)
by Richard W.M. Jones 10 Nov '21

10 Nov '21

Previously: https://listman.redhat.com/archives/libguestfs/2020-March/msg00063.html https://listman.redhat.com/archives/libguestfs/2017-September/msg00203.html Our current minimum version across projects is 4.03. We still use "noalloc" in a few places which causes this warning: ocamlopt.opt -warn-error +A-3 -c NBDKit.ml -o NBDKit.cmx File "NBDKit.ml", line 155, characters 0-70: 155 | external set_name : string -> unit = "ocaml_nbdkit_set_name" "noalloc" ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Alert deprecated: [@@noalloc] should be used instead of "noalloc" I just noticed now that this change was made in 4.03 -- I will go ahead and fix this everywhere today. Should we move to a newer minimum version? If we moved to 4.07 then we could also get rid of the warnings about Pervasives (replaced by Stdlib), eg: File "std_utils.ml", line 329, characters 26-44: 329 | let sort_uniq ?(cmp = Pervasives.compare) xs = ^^^^^^^^^^^^^^^^^^ Alert deprecated: module Stdlib.Pervasives Use Stdlib instead. However 4.07 was only released in 2018, and this would mean removing RHEL 7 compatibility (officially -- it's sort of unofficially not supported already). More irritatingly, FreeBSD is stuck on 4.05. There are no particularly compelling new features at the moment. Here are the common distros and versions (< 4.07 marked with '*'): Arch (Extra) OCaml 4.12 Debian stable OCaml 4.11 Debian testing OCaml 4.11 Fedora 31 OCaml 4.08 Fedora 35 OCaml 4.12 FreeBSD (ports) OCaml 4.05 * OpenSUSE OCaml 4.13 RHEL 7 OCaml 4.05 * RHEL 8 OCaml 4.07 RHEL 9 OCaml 4.11 Ubuntu 16.04 OCaml 4.02 * Ubuntu 18.04 OCaml 4.05 * Ubuntu 20.04 OCaml 4.08 Ubuntu 21.04 OCaml 4.11 And here are the release dates of the OCaml compiler: OCaml version Release date 4.02 2014-08 4.03 2016-04 4.04 2016-11 4.05 2017-07 4.06 2017-11 4.07 2018-07 4.08 2019-06 4.09 2019-09 4.10 2020-02 4.11 2020-08 4.12 2021-02 4.13 2021-09 Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v

4 5

libnbd | Failed pipeline for master | 3b9b3b18
by GitLab 09 Nov '21

09 Nov '21

Pipeline #405686805 has failed! Project: libnbd ( https://gitlab.com/nbdkit/libnbd ) Branch: master ( https://gitlab.com/nbdkit/libnbd/-/commits/master ) Commit: 3b9b3b18 ( https://gitlab.com/nbdkit/libnbd/-/commit/3b9b3b1836bae5bb8e0e688d3e2c1a32f… ) Commit Message: common/utils/test-vector.c: Add a simple test o... Commit Author: Richard W_M_ Jones ( https://gitlab.com/rwmjones ) Pipeline #405686805 ( https://gitlab.com/nbdkit/libnbd/-/pipelines/405686805 ) triggered by Richard W_M_ Jones ( https://gitlab.com/rwmjones ) had 4 failed jobs. Job #1766436429 ( https://gitlab.com/nbdkit/libnbd/-/jobs/1766436429/raw ) Stage: builds Name: x86_64-freebsd-13 Job #1766436430 ( https://gitlab.com/nbdkit/libnbd/-/jobs/1766436430/raw ) Stage: builds Name: x86_64-freebsd-current Job #1766436426 ( https://gitlab.com/nbdkit/libnbd/-/jobs/1766436426/raw ) Stage: builds Name: x86_64-freebsd-12 Job #1766436432 ( https://gitlab.com/nbdkit/libnbd/-/jobs/1766436432/raw ) Stage: builds Name: x86_64-macos-11 -- You're receiving this email because of your account on gitlab.com.

1 0