[PATCH nbdkit 0/2] curl: Fallback to GET if HEAD not supported
by Richard W.M. Jones
AWS S3 servers are mad. You can do an unclear "pre-signing" operation
on them, but only for a single method (eg. just GET). This means that
the way we get the size of the export, by first querying the headers
with HEAD and then using GET for partial data transfers, does not work.
In theory we can get around this by falling back to using GET if we
notice that the HEAD operation failed, and then dropping the
connection when the server starts sending data. I restricted the
fallback behaviour so it only happens when we see the expected
403 Forbidden error from AWS.
I haven't been able to test it against a real server, but I modified
our internal test web server so it can behave like this and added a
test.
Fixes: https://github.com/kubevirt/containerized-data-importer/issues/2737
Rich.
11 months, 2 weeks
[PATCH v3 00/14] qemu patches for 64-bit NBD extensions
by Eric Blake
v2 was here:
https://lists.gnu.org/archive/html/qemu-devel/2022-11/msg02340.html
Since then:
- upstream NBD has accepted the extension on a branch; once multiple
implementations interoperate based on that spec, it will be promoted
to mainline (my plan: qemu with this series, libnbd nearly ready to
go, nbdkit a bit further out)
- rebase to block changes in meantime
- drop RFC patches for 64-bit NBD_CMD_READ (NBD spec did not take them)
- per upstream spec decision, extended headers now mandates use of
NBD_REPLY_TYPE_BLOCK_STATUS_EXT rather than server choice based on
reply size, which in turn required rearranging server patches a bit
- other changes that I noticed while testing with parallel changes
being added to libnbd (link to those patches to follow in the next
week or so)
Eric Blake (14):
nbd/client: Use smarter assert
nbd/client: Add safety check on chunk payload length
nbd/server: Prepare for alternate-size headers
nbd: Prepare for 64-bit request effect lengths
nbd: Add types for extended headers
nbd/server: Refactor handling of request payload
nbd/server: Refactor to pass full request around
nbd/server: Support 64-bit block status
nbd/server: Initial support for extended headers
nbd/client: Initial support for extended headers
nbd/client: Accept 64-bit block status chunks
nbd/client: Request extended headers during negotiation
nbd/server: Prepare for per-request filtering of BLOCK_STATUS
nbd/server: Add FLAG_PAYLOAD support to CMD_BLOCK_STATUS
docs/interop/nbd.txt | 1 +
include/block/nbd.h | 165 +++--
nbd/nbd-internal.h | 8 +-
block/nbd.c | 86 ++-
nbd/client-connection.c | 4 +-
nbd/client.c | 143 ++--
nbd/common.c | 10 +-
nbd/server.c | 653 ++++++++++++------
qemu-nbd.c | 4 +
block/trace-events | 1 +
nbd/trace-events | 11 +-
tests/qemu-iotests/223.out | 18 +-
tests/qemu-iotests/233.out | 5 +
tests/qemu-iotests/241.out | 3 +
tests/qemu-iotests/307.out | 15 +-
.../tests/nbd-qemu-allocation.out | 3 +-
16 files changed, 797 insertions(+), 333 deletions(-)
base-commit: 18b6727083acceac5d76ea0b8cb6f5cdef6858a7
--
2.40.1
11 months, 2 weeks
[PATCH libguestfs 1/2] ocaml/implicit_close test: collect all currently unreachable blocks
by Jürgen Hötzel
Fixes failing implice_close test on OCaml 5.
---
ocaml/t/guestfs_065_implicit_close.ml | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/ocaml/t/guestfs_065_implicit_close.ml b/ocaml/t/guestfs_065_implicit_close.ml
index 567e550b4..5e00c21ac 100644
--- a/ocaml/t/guestfs_065_implicit_close.ml
+++ b/ocaml/t/guestfs_065_implicit_close.ml
@@ -30,8 +30,8 @@ let () =
*)
(* This should cause the GC to close the handle. *)
-let () = Gc.compact ()
+let () = Gc.full_major ()
let () = assert (!close_invoked = 1)
-let () = Gc.compact ()
+let () = Gc.full_major ()
--
2.40.1
11 months, 2 weeks
Workaround to get supermin to work on Slackware
by shiftag
Hello,
I'm experimenting with a Software using supermin. However, it look like
supermin cannot work because of Slackware package manager.
$ supermin --list-drivers
arch/pacman not-detected
debian/dpkg not-detected
openmandriva/rpm not-detected
mageia/rpm not-detected
opensuse/rpm not-detected
ibm_powerkvm/rpm not-detected
fedora/rpm not-detected
Is there any workaround to make the following command works :
$ supermin --prepare bash coreutils ethtool iproute iputils kmod
net-tools pciutils perf procps-ng python strace strace systemd
systemd-udev util-linux vim -o supermin.d
Thanks.
11 months, 3 weeks
Patch review request for Veritas Filesystem(VxFS) and Volume Manager(VxVM) support to libguestfs library
by Ravi Singh
Hi Team,
This mail is regarding these attached patches which will add Veritas Filesystem(VxFS) and Volume Manager(VxVM) support to libguestfs. Since we are modifying the libguestfs library code for commercial purpose, to honor the LGPL license we are sending our changes for review and request to merge these changes to upstream libguestfs git repository.
Changes:
appliance/init:
+# Create dev directory for all Veritas (VxVM) device files.
+mkdir -p /dev/vx
appliance/hostfiles.in: All the files mentioned in this file will be copied to appliance.
+/etc/vx/vxfs
+/etc/vx/veki
+/etc/vx/vxvm-startup
+/opt/VRTS/bin/fstyp
daemon/vm.ml: This new OCAML file has functions, which are used to identify the Veritas volumes inside VMDKs. Similar to lvm/lvm2 in libguestfs. These functions are called Veritas volume manager related commands.
daemon/vxvm_type.ml : This new OCAML file has a function defined to determine the filesystem type on Veritas volume.
generator/actions_core.ml : This is an already existing file used to call functions defined in OCAML *.ml files. In case of Veritas volume, it calls functions from <daemon/vm.ml> and <daemon/vxvm_type.ml>
daemon/vxfs.c : This new C file has functions to identify and start Veritas filesystem(VxFS) on host appliance. Similar to XFS, EXT3/4 and BTRFS in libguestfs.
Attached zip file contains two patches:
Patch for upstream:
0001-Master-VxFS-Filesystem-support-to-libguestfs.patch
Patch for rhel8(libguestfs-1.44):
0024-RHEL8-VxFS-filesystem-support-to-libguestfs.patch
We would really appreciate your feedback on these changes.
Thanks,
Ravi
11 months, 3 weeks
[PATCH libguestfs 2/2] Only leave/enter blocking_section when OCaml lock is not held
by Jürgen Hötzel
Fixes deadlocks on OCaml5 when trying to get the lock that is already
held:
Fatal error during lock: Resource deadlock avoided
---
ocaml/guestfs-c.c | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/ocaml/guestfs-c.c b/ocaml/guestfs-c.c
index 3888c9456..bcf8e6ab3 100644
--- a/ocaml/guestfs-c.c
+++ b/ocaml/guestfs-c.c
@@ -395,12 +395,16 @@ event_callback_wrapper (guestfs_h *g,
/* Ensure we are holding the GC lock before any GC operations are
* possible. (RHBZ#725824)
*/
- caml_leave_blocking_section ();
+ bool in_blocking_section = (caml_state == NULL);
+
+ if (in_blocking_section)
+ caml_leave_blocking_section ();
event_callback_wrapper_locked (g, data, event, event_handle, flags,
buf, buf_len, array, array_len);
- caml_enter_blocking_section ();
+ if (in_blocking_section)
+ caml_enter_blocking_section ();
}
value
--
2.40.1
11 months, 3 weeks
[libnbd PATCH] internal: Tweak use of attribute packed in union sbuf
by Eric Blake
While analyzing 'union sbuf' in preparation to add more members to the
union, I noticed several things related to __attribute__((packed))
that can be improved. It helps to note that that the bulk of the
members of 'union sbuf' are already marked packed structures from
nbd-protocol.h, where we don't need to repeat that annotation in
internal.h but where it does factor into sbuf alignment.
First, rather than open-coding the attribute name in internal.h (in
some places with inconsistent whitespace), we can reuse
NBD_ATTRIBUTE_PACKED already present from nbd-protocol.h.
Second, note that two of the union members are themselves substructs
with two parts: both sbuf.or and sbuf.sr need distinct storage for a
header (a packed structure) and a payload (a union of various items;
currently each member of both of those unions are already packed),
where the choice of union branch and overall size of the payload to be
read (if any) is determined by information in the header. Later
states then refer to information from both the header and payload, so
we need to keep the sub-structs (rather than hoisting the two parts of
the sub-struct into being directly-overlapping top-level members of
union sbuf proper). But because we don't know the payload size until
after the header is read, the state machine uses separate recv() calls
into the two parts of sbuf.or and sbuf.sr; and there is no specific
reason that the two reads need to occur into adjacent memory. Thus,
these two packed annotations buy us nothing at this layer and can be
safely elided.
Finally, there are benefits to naturally aligning uint64_t members,
whether or not hardware supports unaligned access. Even though we are
using attribute packed to match wire format (and some NBD messages do
not have any natural alignment), the judicious use of a non-packed
uint64_t member to various unions gives the compiler permission to
insert padding that is otherwise not possible when a packed struct
occurs immediately before a union containing only packed members. In
particular, with structured replies, it is worth ensuring that
sbuf.sr.payload.offset_data.offset falls on a 64-bit alignment.
Note that sbuf.or does not need this latter treatment (currently,
(sbuf.or.payload.export.exportsize is the only uint64_t in that
particular payload union, but does not occur at a natural alignment to
begin with); but it is also worth remembering that option negotiation
is not in the hot path the way sbuf.sr is.
Actually testing the alignment change from adding align_ members is
harder to do, especially without C11's <stdalign.h>, in part because
on 32-bit platforms where uint64_t is only 4-byte aligned (without gcc
-malign-double), there is no change to layout. But in my
investigation under gdb on x86_64, inserting the otherwise-unused
align_* members changed both offsetof(struct nbd_handle, sbuf) % 8 and
(offsetof(struct nbd_handle, sbuf.sr.payload) - offsetof(struct
nbd_handle, sbuf)) % 8 from 4 to 0, which is what I wanted.
Not touched here: gcc's -Wpacked says several (but not all) of our
structures in that file are already naturally aligned, where adding a
packed notation can actually cause slower code when embedding that
type in a larger structure (exactly what we're doing when combining
those structs into sbuf), when the compiler has to prepare for
unaligned access even if the struct would otherwise be aligned.
Signed-off-by: Eric Blake <eblake(a)redhat.com>
---
Fallout from review of the 64-bit v3 2/22 patch.
lib/internal.h | 14 ++++++++------
1 file changed, 8 insertions(+), 6 deletions(-)
diff --git a/lib/internal.h b/lib/internal.h
index b155681d..a8c49e16 100644
--- a/lib/internal.h
+++ b/lib/internal.h
@@ -212,6 +212,7 @@ struct nbd_handle {
* and commands.
*/
union {
+ uint64_t align_; /* Start sbuf on an 8-byte alignment where useful */
struct nbd_old_handshake old_handshake;
struct nbd_new_handshake new_handshake;
struct nbd_new_option option;
@@ -221,34 +222,35 @@ struct nbd_handle {
struct {
struct nbd_fixed_new_option_reply_server server;
char str[NBD_MAX_STRING * 2 + 1]; /* name, description, NUL */
- } __attribute__ ((packed)) server;
+ } NBD_ATTRIBUTE_PACKED server;
struct nbd_fixed_new_option_reply_info_export export;
struct nbd_fixed_new_option_reply_info_block_size block_size;
struct {
struct nbd_fixed_new_option_reply_info_name_or_desc info;
char str[NBD_MAX_STRING];
- } __attribute__ ((packed)) name_desc;
+ } NBD_ATTRIBUTE_PACKED name_desc;
struct {
struct nbd_fixed_new_option_reply_meta_context context;
char str[NBD_MAX_STRING];
- } __attribute__ ((packed)) context;
+ } NBD_ATTRIBUTE_PACKED context;
char err_msg[NBD_MAX_STRING];
} payload;
- } __attribute__ ((packed)) or;
+ } or;
struct nbd_export_name_option_reply export_name_reply;
struct nbd_simple_reply simple_reply;
struct {
struct nbd_structured_reply structured_reply;
union {
+ uint64_t align_; /* Start sr.payload on an 8-byte alignment */
struct nbd_structured_reply_offset_data offset_data;
struct nbd_structured_reply_offset_hole offset_hole;
struct {
struct nbd_structured_reply_error error;
char msg[NBD_MAX_STRING]; /* Common to all error types */
uint64_t offset; /* Only used for NBD_REPLY_TYPE_ERROR_OFFSET */
- } __attribute__ ((packed)) error;
+ } NBD_ATTRIBUTE_PACKED error;
} payload;
- } __attribute__ ((packed)) sr;
+ } sr;
uint16_t gflags;
uint32_t cflags;
uint32_t len;
--
2.40.1
11 months, 3 weeks
