[libnbd PATCH 00/23] continue wrapping generated C code harder
by Laszlo Ersek
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2172516
Repo: https://gitlab.com/lersek/libnbd.git
Branch: wrap-2172516-wave4
This series is a continuation of the one noted in
<https://bugzilla.redhat.com/show_bug.cgi?id=2172516#c15>. It reduces
the widths of the following generated C files:
file width before width after
------------------ ------------ -----------
include/libnbd.h 81 79
lib/api.c 206 79
ocaml/nbd-c.c 108 79
python/libnbdmod.c 130 79
python/methods.h 104 80
The following generated C files remain for the next wave:
file width
------------------ -----
lib/states-run.c 102
lib/states.c 116
lib/states.h 123
Quite a few patches in the series are easier to review with "git show
-b" or "git show --color-words", due to code reindentation or other
relatively mechanical changes. (And I've just found out about "git show
--color-moved", which works really well on patch #08, 'generator/utils:
factor out "pr_buf"', for example.) Most patches where such display
tweaks are beneficial mention that fact.
These formatting tweaks are also the reason why I've pushed the branch
to my libnbd fork on gitlab (see URL & branch name above). The posted
patches should be possible to apply with git-am, for merging (thus, they
couldn't be formatted with the "-b" option, for example), but for
review, it may be easier to just fetch them (or to fetch them in
addition) and to show them locally.
Thanks for reviewing,
Laszlo
Laszlo Ersek (23):
generator: split LIBNBD_ATTRIBUTE_NONNULL replacement text
generator: wrap "python/methods.h" at 80 characters
generator: wrap "python/libnbdmod.c" at 80 characters
generator: split "int64_from_uint32_array" calls in "ocaml/nbd-c.c"
generator: wrap nbd_internal_ocaml_raise_closed() calls in
"ocaml/nbd-c.c"
generator: factor out "num_args" in "print_ocaml_binding"
generator: wrap byte-code compat function bodies in "ocaml/nbd-c.c"
generator/utils: factor out "pr_buf"
generator/utils: comment on "maxcol" handling in "pr_wrap"
generator/utils: add "pr_wrap_cstr"
generator/C: print_trace_enter: break "enter" to a common new line
generator/C: print_trace_enter: expand "function" shorthand in value
iters
generator/C: print_trace_enter: print Closure values uniformly with
others
generator/C: print_trace_enter: generate commas as suffixes, not
prefixes
generator/C: print_trace_enter: strip spaces around PRI* macros
generator/C: print_trace_enter: wrap "enter" debug() / debug_direct()
call
generator/C: print_wrapper: break apart the "ready" & "processing"
states
generator/C: print_wrapper: fold permitted states error message
generator/C: print_trace_leave: separate arg list from function
designator
generator/C: print_trace_leave: wrap "leave" debug() / debug_direct()
call
generator/C: print_flags_check: break guard condition to a new line
generator/C: print_wrapper: use helper variable for permitted state
check
generator/C: lib/api.c: indent arg list 2 spaces relative to function
name
generator/C.ml | 195 ++++++++++++--------
generator/OCaml.ml | 27 ++-
generator/Python.ml | 13 +-
generator/utils.ml | 85 ++++++++-
generator/utils.mli | 1 +
ocaml/helpers.c | 2 +-
ocaml/nbd-c.h | 3 +-
7 files changed, 225 insertions(+), 101 deletions(-)
base-commit: 0744f748ed900fb0537da9a5b6532538f3c78e83
1 year, 7 months
[PATCH nbdkit 0/2] allocators: sparse: Split the highly contended mutex
by Richard W.M. Jones
nbdkit-memory-plugin is a RAM disk, but it is quite a slow one because
of the lock we must acquire on the sparse array. Try to improve
performance by changing this mutex for a read-write lock.
The results are somewhat mixed.
Note that in the real world, workloads which constantly update the
metadata are assumed to be fairly rare. The test program added in the
first patch exercises metadata updates a lot more aggressively than
would be expected.
Rich.
1 year, 7 months
[PATCH v2] daemon/tar: support more compression methods.
by Zixun LI
Add support for lzma, zstd and auto detection of compression methods.
Removed "auto" setting and updated documentation since v1.
Signed-off-by: Zixun LI <admin(a)hifiphile.com>
---
daemon/tar.c | 8 ++++++++
generator/actions_core.ml | 12 ++++++------
2 files changed, 14 insertions(+), 6 deletions(-)
diff --git a/daemon/tar.c b/daemon/tar.c
index 57b5b37ff..ff29fe500 100644
--- a/daemon/tar.c
+++ b/daemon/tar.c
@@ -159,6 +159,10 @@ do_tar_in (const char *dir, const char *compress, int xattrs, int selinux, int a
filter = " --xz";
else if (STREQ (compress, "lzop"))
filter = " --lzop";
+ else if (STREQ (compress, "lzma"))
+ filter = " --lzma";
+ else if (STREQ (compress, "zstd"))
+ filter = " --zstd";
else {
reply_with_error ("unknown compression type: %s", compress);
return -1;
@@ -312,6 +316,10 @@ do_tar_out (const char *dir, const char *compress, int numericowner,
filter = " --xz";
else if (STREQ (compress, "lzop"))
filter = " --lzop";
+ else if (STREQ (compress, "lzma"))
+ filter = " --lzma";
+ else if (STREQ (compress, "zstd"))
+ filter = " --zstd";
else {
reply_with_error ("unknown compression type: %s", compress);
return -1;
diff --git a/generator/actions_core.ml b/generator/actions_core.ml
index c8d9949ba..273ea1a7f 100644
--- a/generator/actions_core.ml
+++ b/generator/actions_core.ml
@@ -2710,9 +2710,9 @@ This command uploads and unpacks local file C<tarfile> into F<directory>.
The optional C<compress> flag controls compression. If not given,
then the input should be an uncompressed tar file. Otherwise one
of the following strings may be given to select the compression
-type of the input file: C<compress>, C<gzip>, C<bzip2>, C<xz>, C<lzop>.
-(Note that not all builds of libguestfs will support all of these
-compression types).
+type of the input file: C<compress>, C<gzip>, C<bzip2>, C<xz>, C<lzop>,
+C<lzma>, C<zstd>. (Note that not all builds of libguestfs will support
+all of these compression types).
The other optional arguments are:
@@ -2745,9 +2745,9 @@ it to local file C<tarfile>.
The optional C<compress> flag controls compression. If not given,
then the output will be an uncompressed tar file. Otherwise one
of the following strings may be given to select the compression
-type of the output file: C<compress>, C<gzip>, C<bzip2>, C<xz>, C<lzop>.
-(Note that not all builds of libguestfs will support all of these
-compression types).
+type of the output file: C<compress>, C<gzip>, C<bzip2>, C<xz>, C<lzop>,
+C<lzma>, C<zstd>. (Note that not all builds of libguestfs will support
+all of these compression types).
The other optional arguments are:
--
2.40.1
1 year, 7 months
[PATCH] daemon/tar: support more compression methods.
by Zixun LI
Add support for lzma, zstd and auto detection of compression methods.
Signed-off-by: Zixun LI <admin(a)hifiphile.com>
---
daemon/tar.c | 12 ++++++++++++
1 file changed, 12 insertions(+)
diff --git a/daemon/tar.c b/daemon/tar.c
index 57b5b37ff..347add341 100644
--- a/daemon/tar.c
+++ b/daemon/tar.c
@@ -159,6 +159,12 @@ do_tar_in (const char *dir, const char *compress, int xattrs, int selinux, int a
filter = " --xz";
else if (STREQ (compress, "lzop"))
filter = " --lzop";
+ else if (STREQ (compress, "lzma"))
+ filter = " --lzma";
+ else if (STREQ (compress, "zstd"))
+ filter = " --zstd";
+ else if (STREQ (compress, "auto"))
+ filter = " --auto-compress";
else {
reply_with_error ("unknown compression type: %s", compress);
return -1;
@@ -312,6 +318,12 @@ do_tar_out (const char *dir, const char *compress, int numericowner,
filter = " --xz";
else if (STREQ (compress, "lzop"))
filter = " --lzop";
+ else if (STREQ (compress, "lzma"))
+ filter = " --lzma";
+ else if (STREQ (compress, "zstd"))
+ filter = " --zstd";
+ else if (STREQ (compress, "auto"))
+ filter = " --auto-compress";
else {
reply_with_error ("unknown compression type: %s", compress);
return -1;
--
2.40.1
1 year, 7 months
Using nested partition with LUKS+GPT
by Philippe Midol-Monnet
Hi
I have already fulfill a an issue, but it looks like it's not the
recommended process.
We are using a LUKS partition containing a GPT table and several partions.
With libguestfs we can open the LUKS partition and the
|/dev/mapper/<mapname>| device become available, but nested partition
are not seen by the kernel. |partprobe| need to be run (or may be
kpartx) to make partitions available in |/dev/mapper|.
AFAIK there are no simple solution.
As a workaround we use |guestfs_debug| but it's not a long term solution.
We can see two solutions:
* adding a udev rule that will automatically run partpobe when a new
device appeared in /dev/mapper
* adding a new API in libguestfs |guestfs_partprobe(device)| that can
be call by the user when needed.
What is the best best solution from your point of view?
We can help implementing it.
Regards
Philippe Midol-Monnet
--
*This email and any attachment contains EasyMile’s confidential
information, and must not be modified or circulated without EasyMile’s
prior written consent. It is intended exclusively for their recipient.s. If
you received this message by mistake, please notify us promptly and
immediately delete this email and any of its attachments.*
*As data
controller, EasyMile processes personal data, in compliance with the
European GDPR (EU) 2016/679 of 27 April 2016. To exercise your rights, you
can contact EasyMile at privacy(a)easymile.com, and if necessary, you may
contact the local competent supervisory authority.”*
*Cet e-mail et toute
pièce jointe contiennent des informations confidentielles d'EasyMile et ne
doivent pas être modifiés ou diffusés sans le consentement écrit préalable
d'EasyMile. Ils sont destinés exclusivement à leur destinataire.s. Si vous
avez reçu ce message par erreur, veuillez nous en informer rapidement et
supprimer immédiatement cet e-mail et toutes ses pièces jointes. *
*En
tant que responsable de traitement, EasyMile traite des données
personnelles, conformément au RGPD européen (UE) 2016/679 du 27 avril 2016.
Pour exercer vos droits, vous pouvez contacter EasyMile à
privacy(a)easymile.com <mailto:privacy@easymile.com>, et si nécessaire, vous
pouvez contacter l'autorité de contrôle locale compétente.*
1 year, 8 months
[PATCH v3 0/6] NBD 64-bit extensions (spec only)
by Eric Blake
v2 was here:
https://lists.debian.org/nbd/2022/11/msg00030.html
The bulk of the changes since then are:
- forbid NBD_OPT_EXPORT_NAME once extended headers are negotiated (Wouter)
- consistently use 'maximum payload', rather than a haphazard mix of
'maximum block payload' (Vladimir)
At this point, I want to make sure we are happy with the spec before
re-posting patches for qemu and libnbd (there were some slight tweaks
to the spec compared to v2, which will be easy enough to code, but I
want to make sure there won't be any other tweaks needed based on
review comments to this proposal).
I could still be persuaded that if extended headers are negotiated,
then the server should ONLY send back 64-bit block status replies (as
written here, and as I implemented in v2 patches for qemu and libnbd,
I specified that the server can choose either 32- or 64-bit replies,
based on what fits best, and regardless of whether the client made a
request with effect length less than 32 bits; clients must therefore
be prepared to handle both types of replies).
I'm still not sure whether patch 6 is desirable; it didn't get any
feedback in my v2 posting.
In relation to v2, the subject line for patch 2 changed (the 'git
backport-diff' tool doesn't handle that well), and the interdiff
changes look bigger than they really are due to reflowed paragraphs.
[----] : patches are identical
[####] : number of functional differences between upstream/downstream patch
[down] : patch is downstream-only
The flags [FC] indicate (F)unctional and (C)ontextual differences, respectively
001/6:[----] [-C] 'spec: Recommend cap on NBD_REPLY_TYPE_BLOCK_STATUS length'
002/6:[down] 'spec: Change maximum block size to maximum payload size'
003/6:[0237] [FC] 'spec: Add NBD_OPT_EXTENDED_HEADERS'
004/6:[----] [-C] 'spec: Allow 64-bit block status results'
005/6:[----] [-C] 'spec: Introduce NBD_FLAG_BLOCK_STATUS_PAYLOAD'
006/6:[0062] [FC] 'RFC: spec: Introduce NBD_REPLY_TYPE_OFFSET_HOLE_EXT'
Eric Blake (6):
spec: Recommend cap on NBD_REPLY_TYPE_BLOCK_STATUS length
spec: Change maximum block size to maximum payload size
spec: Add NBD_OPT_EXTENDED_HEADERS
spec: Allow 64-bit block status results
spec: Introduce NBD_FLAG_BLOCK_STATUS_PAYLOAD
RFC: spec: Introduce NBD_REPLY_TYPE_OFFSET_HOLE_EXT
doc/proto.md | 738 ++++++++++++++++++++++++++++++++++++++-------------
1 file changed, 549 insertions(+), 189 deletions(-)
base-commit: fd2c5a60e36ef0ea0d7fd6f8a6bd908d24d5801d
--
2.39.2
1 year, 8 months
[PATCH] spec: Document extended headers extension branch
by Eric Blake
Branch extension-ext-header now exists to capture the specs reviewed
on list and tentatively implemented in both qemu (server and client)
and libnbd (client) code. I'm still working on implementing the
extension in nbdkit (server) before promoting the extension branch to
mainline.
Meanwhile, structured replies and block status have been incorporated
into mainline for some time now (see commit 6e896bca in 2018, v3.17);
and while block status is still not implemented in the kernel module
or in nbd-server, we did just recently implement structured replies in
nbd-server.
Signed-off-by: Eric Blake <eblake(a)redhat.com>
---
Given the review on my v3 spec proposal for extended headers, I've
pushed patch 1 and 2 of that series to mainline along with this patch,
while patches 3-5 of that series are tweaked per review comments and
pushed on the new extension-ext-header branch. My next step will be
posting v3 patches to qemu and libnbd which rebases my earlier
proof-of-concept patches to those projects to match the spec as now
live (and/or tweak the extension spec if something insurmountable
comes up).
doc/proto.md | 25 ++++++++++++++++++++++---
1 file changed, 22 insertions(+), 3 deletions(-)
diff --git a/doc/proto.md b/doc/proto.md
index 460823b..8589e22 100644
--- a/doc/proto.md
+++ b/doc/proto.md
@@ -1137,6 +1137,9 @@ The field has the following format:
will be faster than a regular write). Clients MUST NOT set the
`NBD_CMD_FLAG_FAST_ZERO` request flag unless this transmission flag
is set.
+- bit 12, `NBD_FLAG_BLOCK_STATUS_PAYLOAD`; defined by the experimental
+ `EXTENDED_HEADERS`
+ [extension](https://github.com/NetworkBlockDevice/nbd/blob/extension-ext-h....
Clients SHOULD ignore unknown flags.
@@ -1500,6 +1503,11 @@ of the newstyle negotiation.
option does not select any metadata context, provided the client
then does not attempt to issue `NBD_CMD_BLOCK_STATUS` commands.
+* `NBD_OPT_EXTENDED_HEADERS` (11)
+
+ Defined by the experimental `EXTENDED_HEADERS`
+ [extension](https://github.com/NetworkBlockDevice/nbd/blob/extension-ext-h....
+
#### Option reply types
These values are used in the "reply type" field, sent by the server
@@ -1680,6 +1688,11 @@ case that data is an error message string suitable for display to the user.
The request or the reply is too large to process.
+* `NBD_REP_ERR_EXT_HEADER_REQD` (2^31 + 10)
+
+ Defined by the experimental `EXTENDED_HEADERS`
+ [extension](https://github.com/NetworkBlockDevice/nbd/blob/extension-ext-h....
+
### Transmission phase
#### Flag fields
@@ -1733,6 +1746,9 @@ valid may depend on negotiation during the handshake phase.
`NBD_CMD_WRITE`, then the server MUST fail quickly with an error of
`NBD_ENOTSUP`. The client MUST NOT set this unless the server advertised
`NBD_FLAG_SEND_FAST_ZERO`.
+- bit 5, `NBD_CMD_FLAG_PAYLOAD_LEN`; defined by the experimental
+ `EXTENDED_HEADERS`
+ [extension](https://github.com/NetworkBlockDevice/nbd/blob/extension-ext-h....
##### Structured reply flags
@@ -1853,6 +1869,11 @@ small amount of fixed-length overhead inherent in the chunk type).
extent information at the first offset not covered by a
reduced-length reply.
+* `NBD_REPLY_TYPE_BLOCK_STATUS_EXT` (6)
+
+ Defined by the experimental `EXTENDED_HEADERS`
+ [extension](https://github.com/NetworkBlockDevice/nbd/blob/extension-ext-h....
+
All error chunk types have bit 15 set, and begin with the same
*error*, *message length*, and optional *message* fields as
`NBD_REPLY_TYPE_ERROR`. If nonzero, *message length* indicates
@@ -2336,9 +2357,7 @@ with names starting with the word 'extension'.
Currently known are:
-* The `STRUCTURED_REPLY` [extension](https://github.com/NetworkBlockDevice/nbd/blob/extension-struc....
-
-* The `BLOCK_STATUS` [extension](https://github.com/NetworkBlockDevice/nbd/blob/extension-block... (based on the `STRUCTURED_REPLY` extension).
+* The `EXTENDED_HEADER` [extension](https://github.com/NetworkBlockDevice/nbd/blob/extension-ext-h....
* The `RESIZE` [extension](https://github.com/NetworkBlockDevice/nbd/blob/extension-resiz....
base-commit: daf3fce989b23cc8c9023f28e0eec2f074c9f1bd
--
2.40.0
1 year, 8 months
Specify partition when using virt-customize --copy-in, copy file to boot partition
by Ingo Weinmann
Using virt-customize of libguestfs, I can copy a file from the host to
the root partition of an image like this:
virt-customize -a 2023-02-21-raspios-bullseye-arm64-lite.img
--copy-in cmdline.txt:/boot/
But the image also has a boot partition (FAT) and I want to copy that
file to that partition instead. Is that possible using virt-customize?
1 year, 8 months
[COMMON PATCH 0/1] mlcustomize: skip SELinux relabeling if it's disabled
by Andrey Drobyshev
This patch effectively limits the number of cases when we would want to
do a complete SELinux relabeling on Linux guest conversion.
This was brought to my attention as we've recently had a support case
when the conversion was taking too much time mostly because of
relabeling performed with "setfiles -F".
Although this patch might be worthy of taking as it is, I'd also like to
clarify, do we really need relabeling of the entire file system during
conversion? What exactly might go wrong here if we don't do that?
Since this process might take hours on VMs big enough, it would be
beneficial to be able to limit number of such cases even further, if
possible. Unfortunately I couldn't find any hints in the libguestfs commit
history as the relabeling code goes back to 0131d6f666 ("New tool: virt-v2v.").
Roman Kagan (1):
customize: skip SELinux relabeling if it's disabled
mlcustomize/SELinux_relabel.ml | 16 +++++++++++++---
1 file changed, 13 insertions(+), 3 deletions(-)
--
2.31.1
1 year, 8 months
