July 2019 - Libguestfs - libguestfs List Archives

[PATCH 0/5] Split virt-p2v in own repository
by Pino Toscano 10 Jul '19

10 Jul '19

Hi, as it was already discussed on this list, here it is my attempt in splitting virt-p2v in an own repository. Sadly there are things that must be copied from libguestfs, as it cannot be avoided. The approach taken was to run a script (will send separately) to just get the "p2v" subdirectory with its history as own repository, and then add in few followup commits all the bits needed to build. The split repository I have is: https://github.com/ptoscano/virt-p2v-split And follow here in this series all the patches to make virt-p2v build, work, and with also "make dist" usable. Let me know what you think -- any kind of feedback is welcome. Pino Toscano (5): Import miniexpect library Import gnulib Copy C sources from libguestfs Add/adapt all the files to build Improve gitignore .gitignore | 17 +- .gitmodules | 3 + .gnulib | 1 + AUTHORS | 16 + Makefile.am | 54 ++- autogen.sh | 51 +++ bootstrap | 124 ++++++ configure.ac | 109 +++++ libguestfs/README | 4 + libguestfs/cleanups.c | 96 ++++ libguestfs/cleanups.h | 74 ++++ libguestfs/guestfs-utils.c | 247 +++++++++++ libguestfs/guestfs-utils.h | 85 ++++ libguestfs/libxml2-cleanups.c | 94 ++++ libguestfs/libxml2-writer-macros.h | 200 +++++++++ m4/p2v-c.m4 | 179 ++++++++ m4/p2v-libraries.m4 | 79 ++++ m4/p2v-progs.m4 | 63 +++ m4/p2v-tests.m4 | 45 ++ miniexpect/README | 31 ++ miniexpect/miniexpect.c | 489 +++++++++++++++++++++ miniexpect/miniexpect.h | 110 +++++ podcheck.pl | 273 ++++++++++++ podwrapper.pl.in | 678 +++++++++++++++++++++++++++++ run.in | 118 +++++ subdir-rules.mk | 60 +++ test-functions.sh | 120 +++++ 27 files changed, 3401 insertions(+), 19 deletions(-) create mode 100644 .gitmodules create mode 160000 .gnulib create mode 100644 AUTHORS create mode 100755 autogen.sh create mode 100755 bootstrap create mode 100644 configure.ac create mode 100644 libguestfs/README create mode 100644 libguestfs/cleanups.c create mode 100644 libguestfs/cleanups.h create mode 100644 libguestfs/guestfs-utils.c create mode 100644 libguestfs/guestfs-utils.h create mode 100644 libguestfs/libxml2-cleanups.c create mode 100644 libguestfs/libxml2-writer-macros.h create mode 100644 m4/p2v-c.m4 create mode 100644 m4/p2v-libraries.m4 create mode 100644 m4/p2v-progs.m4 create mode 100644 m4/p2v-tests.m4 create mode 100644 miniexpect/README create mode 100644 miniexpect/miniexpect.c create mode 100644 miniexpect/miniexpect.h create mode 100755 podcheck.pl create mode 100755 podwrapper.pl.in create mode 100755 run.in create mode 100644 subdir-rules.mk create mode 100755 test-functions.sh -- 2.21.0

2 9

Questions on libguestfs-java
by Arik Hadas 09 Jul '19

09 Jul '19

Hi, When using the libguestfs-java bindings, java.library.path should point to the place the JNI stuff resides in. In [1] it was suggested to set it to /usr/local/lib but apparently, it is deployed elsewhere in other distributions, right? so I wonder what is the recommended way of setting it in a Java application that could execute on various distributions. Additionally, the libguestfs-java bindings for Fedora 30 seem to be missing - is it a known issue? [1] https://www.redhat.com/archives/libguestfs/2012-September/msg00057.html

2 2

[PATCH 0/4] p2v: more bits of independence (#3)
by Pino Toscano 08 Jul '19

08 Jul '19

These are more small bits to make p2v more independent, and they will help after the split. Pino Toscano (4): p2v: clean also the test images p2v: consider p2v-config.h as generated source p2v: fix tests with srcdir!=builddir p2v: split appliance tests in own variable p2v/Makefile.am | 13 +++++++++---- p2v/test-virt-p2v-docs.sh | 2 +- p2v/test-virt-p2v-nbdkit.sh | 4 ++-- p2v/test-virt-p2v.sh | 4 ++-- 4 files changed, 14 insertions(+), 9 deletions(-) -- 2.21.0

2 5

[libnbd PATCH] RFC: Add bindings for Rust language
by Martin Kletzander 08 Jul '19

08 Jul '19

This is just a basic ugly support, not meant to be pushed (at least for now). There is quite a lot missing, but there is an example which shows that it really works. And valgrind reports that all allocations were freed. The way the code is generated is also not nice, I wish there was more code actually written in some files and not generated by the generator (as much hard-coded static strings as possible), maybe similarly to the states.c, I don't know. It follows the convention of `-sys` packages: https://doc.rust-lang.org/cargo/reference/build-scripts.html#a-sys-packages But that's about it, I hadn't really had much more time to do anything, I had only limited free time during the weekend to push this as far as possible and I got stuck couple of times. But I think the hardest parts are solved and Also up for discussion is whether the libnbd crate should be separate since the higher-level functionality it should provide will not be tightly coupled with libnbd itself and the releases (especially the numbers) do not need to happen in sync. Maybe even the libnbd-sys crate should be in separate repo which has libnbd as a submodule (in order to use the valuable structuring of API data that is not installed in the system). Signed-off-by: Martin Kletzander <mkletzan(a)redhat.com> --- .gitignore | 5 + Makefile.am | 1 + configure.ac | 16 ++ generator/generator | 301 ++++++++++++++++++++++++++++++ run.in | 9 + rust/Cargo.toml | 6 + rust/Makefile.am | 42 +++++ rust/libnbd-sys/Cargo.toml.in | 10 + rust/libnbd-sys/build.rs | 9 + rust/libnbd-sys/src/.gitkeep | 0 rust/libnbd/Cargo.toml.in | 9 + rust/libnbd/examples/hello-nbd.rs | 7 + rust/libnbd/src/lib.rs | 4 + rust/libnbd/src/nbd_error.rs | 31 +++ rust/run-tests | 4 + 15 files changed, 454 insertions(+) create mode 100644 rust/Cargo.toml create mode 100644 rust/Makefile.am create mode 100644 rust/libnbd-sys/Cargo.toml.in create mode 100644 rust/libnbd-sys/build.rs create mode 100644 rust/libnbd-sys/src/.gitkeep create mode 100644 rust/libnbd/Cargo.toml.in create mode 100644 rust/libnbd/examples/hello-nbd.rs create mode 100644 rust/libnbd/src/lib.rs create mode 100644 rust/libnbd/src/nbd_error.rs create mode 100755 rust/run-tests diff --git a/.gitignore b/.gitignore index d4828fae028b..58ed9d04df09 100644 --- a/.gitignore +++ b/.gitignore @@ -86,6 +86,11 @@ Makefile.in /python/nbd.py /python/run-python-tests /run +/rust/**/Cargo.lock +/rust/*/Cargo.toml +/rust/libnbd/src/glue.rs +/rust/libnbd-sys/src/lib.rs +/rust/target /sh/nbdsh /sh/nbdsh.1 /stamp-h1 diff --git a/Makefile.am b/Makefile.am index 6a0f4bdfc653..f28f828f6b7d 100644 --- a/Makefile.am +++ b/Makefile.am @@ -38,6 +38,7 @@ SUBDIRS = \ ocaml \ ocaml/examples \ interop \ + rust \ $(NULL) noinst_SCRIPTS = run diff --git a/configure.ac b/configure.ac index 91d08341d732..339073cf6660 100644 --- a/configure.ac +++ b/configure.ac @@ -258,6 +258,19 @@ AS_IF([test "x$enable_python" != "xno"],[ AM_CONDITIONAL([HAVE_PYTHON], [test "x$PYTHON" != "xno" && test "x$have_python_module" = "x1" ]) +dnl Rust, optional, just runs tests +AC_ARG_ENABLE([rust], + AS_HELP_STRING([--disable-rust], [disable Rust language binding tests]), + [], + [enable_rust=yes]) +AS_IF([test "x$enable_rust" != "xno"], + [AC_CHECK_PROG([CARGO],[cargo],[cargo],[no])]) +AS_IF([test "$CARGO" = "no"], + [AC_MSG_ERROR([cargo not found])]) + +AM_CONDITIONAL([HAVE_RUST], + [test "$CARGO" != "no" && test "$enable_rust" = "yes" ]) + dnl Produce output files. AC_CONFIG_HEADERS([config.h]) @@ -282,6 +295,9 @@ AC_CONFIG_FILES([Makefile ocaml/META ocaml/examples/Makefile python/Makefile + rust/Makefile + rust/libnbd-sys/Cargo.toml + rust/libnbd/Cargo.toml sh/Makefile tests/Makefile tests/functions.sh diff --git a/generator/generator b/generator/generator index 9192988af4b3..969d0af0c247 100755 --- a/generator/generator +++ b/generator/generator @@ -2841,6 +2841,9 @@ let () = | name, { permitted_states = (_::_); may_set_error = false } -> failwithf "%s: if may_set_error is false, permitted_states must be empty (any permitted state)" name + | name, { ret = RErr; may_set_error = false } -> + failwithf "%s: if ret is RErr, then may_set_error cannot be false" + name | _ -> () ) handle_calls @@ -4652,6 +4655,302 @@ let generate_ocaml_nbd_c () = List.iter print_ocaml_binding handle_calls +let rec pr_indent = function + | 0 -> () + | n -> pr " "; pr_indent (n - 1) + +let rec print_rust_ffi_arg_list ?(handle = false) indent args = + let pri () = pr_indent indent in + let pri2 () = pr_indent (indent + 1) in + pr "(\n"; + if handle then ( + pri (); pr "h: *mut nbd_handle,\n"; + ); + List.iter ( + fun arg -> + (match arg with + | ArrayAndLen (UInt32 n, len) -> + pri (); pr "%s: *mut u32,\n" n; + pri (); pr "%s: usize,\n" len + | ArrayAndLen _ -> assert false + | Bool n -> pri (); pr "%s: bool,\n" n + | BytesIn (n, len) + | BytesPersistIn (n, len) -> + pri (); pr "%s: *const c_void,\n" n; + pri (); pr "%s: usize,\n" len + | BytesOut (n, len) + | BytesPersistOut (n, len) -> + pri (); pr "%s: *mut c_void,\n" n; + pri (); pr "%s: usize,\n" len + | Callback (n, args) + | CallbackPersist (n, args) -> + pri (); pr "%s: Option<\n" n; + pri2 (); pr "unsafe extern \"C\" fn"; + print_rust_ffi_arg_list (indent + 2) args; + pr " -> c_int,\n"; + pri (); pr ">,\n" + | Flags n -> pri (); pr "%s: u32,\n" n + | Int n -> pri (); pr "%s: c_int,\n" n + | Int64 n -> pri (); pr "%s: i64,\n" n + | Mutable (Int n) -> pri (); pr "%s: *mut c_int,\n" n + | Mutable arg -> assert false + | Opaque n -> pri (); pr "%s: *mut c_void,\n" n + | Path n + | String n -> pri (); pr "%s: *const c_char,\n" n + | StringList n -> pri (); pr "%s: *mut *mut c_char,\n" n + | SockAddrAndLen (n, len) -> + pri (); pr "%s: *const libc::sockaddr,\n" n; + pri (); pr "%s: libc::socklen_t,\n" len + | UInt n -> pri (); pr "%s: c_uint,\n" n + | UInt32 n -> pri (); pr "%s: u32,\n" n + | UInt64 n -> pri (); pr "%s: u64,\n" n + ); + ) args; + pr_indent (indent - 1); pr ")" + +let print_rust_arg_list ?(handle = false) indent args = + let pri () = pr_indent indent in + let pri2 () = pr_indent (indent + 1) in + pr "(\n"; + if handle then ( + pri (); pr "&self,\n"; + ); + List.iter ( + fun arg -> + (* TODO: rewrite to use Rust types *) + (match arg with + | ArrayAndLen (UInt32 n, len) -> + pr "%s: *mut u32,\n" n; pri (); + pr "%s: usize,\n" len; pri () + | ArrayAndLen _ -> assert false + | Bool n -> pri (); pr "%s: bool,\n" n + | BytesIn (n, len) + | BytesPersistIn (n, len) -> + pri (); pr "%s: *const c_void,\n" n; + pri (); pr "%s: usize,\n" len + | BytesOut (n, len) + | BytesPersistOut (n, len) -> + pri (); pr "%s: *mut c_void,\n" n; + pri (); pr "%s: usize,\n" len + | Callback (n, args) + | CallbackPersist (n, args) -> + pri (); pr "%s: Option<\n" n; + pri2 (); pr "unsafe extern \"C\" fn"; + print_rust_ffi_arg_list (indent + 2) args; + pr " -> c_int,\n"; + pri (); pr ">,\n" + | Flags n -> pri (); pr "%s: u32,\n" n + | Int n -> pri (); pr "%s: c_int,\n" n + | Int64 n -> pri (); pr "%s: i64,\n" n + | Mutable (Int n) -> pri (); pr "%s: *mut c_int,\n" n + | Mutable arg -> assert false + | Opaque n -> pri (); pr "%s: *mut c_void,\n" n + | Path n + | String n -> pri (); pr "%s: *const c_char,\n" n + | StringList n -> pri (); pr "%s: *mut *mut c_char,\n" n + | SockAddrAndLen (n, len) -> + pri (); pr "%s: *const libc::sockaddr,\n" n; + pri (); pr "%s: libc::socklen_t" len + | UInt n -> pri (); pr "%s: c_uint,\n" n + | UInt32 n -> pri (); pr "%s: u32,\n" n + | UInt64 n -> pri (); pr "%s: u64,\n" n + ); + ) args; + pr_indent (indent - 1); pr ")" + +let generate_rust_sys_lib_rs () = + let print_extern (name, {args; ret; _ }) = + let ret_rs_type = + match ret with + | RBool + | RErr + | RFd + | RInt -> "c_int" + | RConstString -> "*const c_char" + | RInt64 -> "i64" + | RString -> "*mut c_char" + in + + (* TODO: print shortdesc and longdesc *) + pr " pub fn nbd_%s" name; + print_rust_ffi_arg_list ~handle:true 2 args; + pr " -> %s;\n" ret_rs_type + in + + generate_header CStyle; + + (* TODO: global documentation *) + + pr "#[allow(unused_imports)]\n"; + pr "use std::os::raw::{c_char, c_int, c_uint, c_void};\n"; + pr "\n"; + + (* TODO: print constants *) + (* TODO: print constants for metadata namespaces *) + + pr "#[link(name = \"nbd\")]"; + pr "extern \"C\" {\n"; + pr " pub fn nbd_create() -> *mut nbd_handle;\n"; + pr " pub fn nbd_close(handle: *mut nbd_handle);\n"; + pr " pub fn nbd_add_close_callback(\n"; + pr " handle: *mut nbd_handle,\n"; + pr " callback: Option<unsafe extern \"C\" fn(*mut c_void)>,\n"; + pr " ) -> c_int;\n"; + pr ""; + pr " pub fn nbd_get_error() -> *const c_char;\n"; + pr " pub fn nbd_get_errno() -> c_int;\n"; + pr "\n"; + List.iter print_extern handle_calls; + pr "}\n"; + pr "\n"; + pr "#[repr(C)]\n"; + pr "#[derive(Debug, Copy, Clone)]\n"; + pr "pub struct nbd_handle {\n"; + pr " _unused: [u8; 0],\n"; + pr "}\n" + +let generate_rust_glue_rs () = + let print_wrapper (name, {args; ret; permitted_states; + is_locked; may_set_error}) = + let ret_rs_type = + let typ = + match ret with + | RBool -> "bool" + | RErr -> "()" + | RFd -> "RawFd" + | RInt -> "i32" + | RConstString -> "&'static str" + | RInt64 -> "i64" + | RString -> "String" in + if may_set_error then "Result<" ^ typ ^ ", NbdError>" else typ + in + + (* TODO: at least shortdesc *) + pr "\n pub fn %s" name; + print_rust_arg_list ~handle:true 2 args; + pr " -> %s {\n" ret_rs_type; + + let num = ref 0 in + let arg_transform arg num = + (* TODO: transform Rust types to C types *) + match arg with + | ArrayAndLen (UInt32 n, len) -> [""], 2 + | ArrayAndLen _ -> assert false + | Bool n -> [], 0 + | BytesIn (n, _) | BytesPersistIn (n, _) -> [""], 2 + | BytesPersistOut (n, _) -> [""], 2 + | BytesOut (_, count) -> [""], 2 + | Callback (n, _) | CallbackPersist (n, _) -> [""], 2 + | Flags n -> [""], 1 + | Int n -> [""], 1 + | Int64 n -> [""], 1 + | Mutable arg -> [""], 1 + | Opaque n -> [""], 1 + | Path n -> [""], 1 + | SockAddrAndLen (n, _) -> [""], 2 + | String n -> [""], 1 + | StringList n -> [""], 1 + | UInt n -> [""], 1 + | UInt32 n -> [""], 1 + | UInt64 n -> [""], 1 + in + + let before = List.flatten (List.map (fun arg -> + let code, n = arg_transform arg num in + num := !num + n; + code + ) args) + in + + let ret_transform ret = + let err_chk = + [(match ret with + | RBool + | RErr + | RFd + | RInt + | RInt64 -> "if ret == -1 {" + | RConstString + | RString -> "if ret.is_null() {"); + " return Err(NbdError::from_libnbd());"; + "}"; + ] + in + let trans = + match ret with + | RBool + | RErr + | RFd + | RInt + | RInt64 -> [] + | RConstString -> [ + "let ret = unsafe { CStr::from_ptr(ret) };"; + "let ret = ret.to_str().unwrap();"; + ] + | RString -> [ + "let c_str = unsafe { CStr::from_ptr(ret as *const c_char) };"; + "let ret = c_str.to_string_lossy().into_owned();"; + "unsafe { libc::free(c_str.as_ptr() as *mut c_void) }; "; + ] + in + let real_ret = + match ret with + | RBool -> "ret != 0"; + | RErr -> "()" + | RFd + | RInt + | RInt64 + | RConstString + | RString -> "ret" + in + (if may_set_error then err_chk else []) @ + trans @ + [(if may_set_error then "Ok(" ^ real_ret ^ ")" else real_ret)] + in + + List.iter (fun str -> pr " %s\n" str) before; + pr " let ret = unsafe { nbd_%s (self.handle" name; + let argnames = List.flatten (List.map name_of_arg args) in + List.iter (pr ", %s") argnames; + pr ") };\n"; + List.iter (fun str -> pr " %s\n" str) (ret_transform ret); + pr " }\n"; + in + + generate_header CStyle; + + pr "#[allow(unused_imports)]\n"; + pr "use std::os::raw::{c_char, c_int, c_uint, c_void};\n"; + pr "use std::os::unix::io::RawFd;\n"; + pr "use std::ffi::CStr;\n"; + pr "use libnbd_sys::*;\n"; + pr "use libc;\n"; + pr "\n"; + pr "pub use super::nbd_error::*;\n"; + pr "\n"; + + pr "pub struct Nbd {\n"; + pr " handle: *mut nbd_handle,\n"; + pr "}\n"; + pr "\n"; + pr "impl Nbd {\n"; + pr " pub fn create() -> Result<Self, NbdError> {\n"; + pr " let handle = unsafe { nbd_create() };\n"; + pr " if handle.is_null() {\n"; + pr " return Err(NbdError::from_libnbd());\n"; + pr " }\n"; + pr " Ok(Self { handle: handle })\n"; + pr " }\n"; + + List.iter print_wrapper handle_calls; + pr "}\n"; + + pr "impl Drop for Nbd {\n"; + pr " fn drop(&mut self) {\n"; + pr " unsafe { nbd_close(self.handle);\n"; + pr " }\n"; + pr "}\n" + (*----------------------------------------------------------------------*) (* Write the output files. *) @@ -4670,3 +4969,5 @@ let () = output_to "ocaml/NBD.mli" generate_ocaml_nbd_mli; output_to "ocaml/NBD.ml" generate_ocaml_nbd_ml; output_to "ocaml/nbd-c.c" generate_ocaml_nbd_c; + output_to "rust/libnbd-sys/src/lib.rs" generate_rust_sys_lib_rs; + output_to "rust/libnbd/src/glue.rs" generate_rust_glue_rs; diff --git a/run.in b/run.in index 56ed70346922..117ab59d46b8 100755 --- a/run.in +++ b/run.in @@ -106,5 +106,14 @@ fi export GNOME_KEYRING_CONTROL= export GNOME_KEYRING_PID= +# For rust +export RUST="@RUST@" +if [ -z "$RUSTFLAGS" ]; then + RUSTFLAGS="-C link-args=-L$b/lib/.libs" +else + RUSTFLAGS="$RUSTFLAGS -C link-args=-L$b/lib/.libs" +fi +export RUSTFLAGS + # Run the program. exec $libtool $valgrind "$@" diff --git a/rust/Cargo.toml b/rust/Cargo.toml new file mode 100644 index 000000000000..877446074a6b --- /dev/null +++ b/rust/Cargo.toml @@ -0,0 +1,6 @@ +[workspace] + +members = [ + "libnbd-sys", + "libnbd", +] diff --git a/rust/Makefile.am b/rust/Makefile.am new file mode 100644 index 000000000000..ca2974e09aa2 --- /dev/null +++ b/rust/Makefile.am @@ -0,0 +1,42 @@ +# nbd client library in userspace +# Copyright (C) 2013-2019 Red Hat Inc. +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + +include $(top_srcdir)/subdir-rules.mk + +clean-local: + -cargo clean + +generator_built = \ + src/ffi.rs + +EXTRA_DIST = \ + $(generator_built) \ + Cargo.toml.in + +if HAVE_RUST + +all: target/release/libnbd-sys.rlib target/release/examples/hello-nbd + +target/release/libnbd-sys.rlib: + $(top_builddir)/run cargo build --release + +target/release/examples/hello-nbd: target/release/libnbd-sys.rlib + $(top_builddir)/run cargo build --release --example hello-nbd + +TESTS = run-tests + +endif HAVE_RUST diff --git a/rust/libnbd-sys/Cargo.toml.in b/rust/libnbd-sys/Cargo.toml.in new file mode 100644 index 000000000000..684510fca14b --- /dev/null +++ b/rust/libnbd-sys/Cargo.toml.in @@ -0,0 +1,10 @@ +[package] +name = "libnbd-sys" +version = "@VERSION@" +authors = ["Martin Kletzander <mkletzan(a)redhat.com>"] +edition = "2018" +links = "nbd" +build = "build.rs" + +[dependencies] +libc = "^0.2.58" diff --git a/rust/libnbd-sys/build.rs b/rust/libnbd-sys/build.rs new file mode 100644 index 000000000000..acfdf8e57245 --- /dev/null +++ b/rust/libnbd-sys/build.rs @@ -0,0 +1,9 @@ +/* + * We have to have a build.rs script in order to use `links` in Cargo.toml. + * Ideally this should figure out whether we can build with a library installed + * in the system and if not, it should add C files to the build and build it + * locally or basically build it without the library being installed in the + * system. + */ + +fn main() {} diff --git a/rust/libnbd-sys/src/.gitkeep b/rust/libnbd-sys/src/.gitkeep new file mode 100644 index 000000000000..e69de29bb2d1 diff --git a/rust/libnbd/Cargo.toml.in b/rust/libnbd/Cargo.toml.in new file mode 100644 index 000000000000..6ba12963fc08 --- /dev/null +++ b/rust/libnbd/Cargo.toml.in @@ -0,0 +1,9 @@ +[package] +name = "libnbd" +version = "@VERSION@" +authors = ["Martin Kletzander <mkletzan(a)redhat.com>"] +edition = "2018" + +[dependencies] +libc = "^0.2.58" +libnbd-sys = { version = "^@VERSION@", path = "../libnbd-sys" } diff --git a/rust/libnbd/examples/hello-nbd.rs b/rust/libnbd/examples/hello-nbd.rs new file mode 100644 index 000000000000..5d6279088bc7 --- /dev/null +++ b/rust/libnbd/examples/hello-nbd.rs @@ -0,0 +1,7 @@ +use libnbd::*; + +fn main() -> Result<(), Box<dyn std::error::Error>> { + let nbd = Nbd::create()?; + println!("libnbd version is {}", nbd.get_version()); + Ok(()) +} diff --git a/rust/libnbd/src/lib.rs b/rust/libnbd/src/lib.rs new file mode 100644 index 000000000000..b69b7c1a1863 --- /dev/null +++ b/rust/libnbd/src/lib.rs @@ -0,0 +1,4 @@ +mod nbd_error; + +mod glue; +pub use glue::*; diff --git a/rust/libnbd/src/nbd_error.rs b/rust/libnbd/src/nbd_error.rs new file mode 100644 index 000000000000..0a7f667d5ab7 --- /dev/null +++ b/rust/libnbd/src/nbd_error.rs @@ -0,0 +1,31 @@ +use libnbd_sys::{nbd_get_errno, nbd_get_error}; +use std::ffi::CStr; +use std::fmt; + +#[derive(Debug, Copy, Clone)] +pub struct NbdError { + errno: i32, + strerr: &'static CStr, +} + +impl NbdError { + pub fn from_libnbd() -> Self { + Self { + errno: unsafe { nbd_get_errno() }, + strerr: unsafe { CStr::from_ptr(nbd_get_error()) }, + } + } +} + +impl std::error::Error for NbdError {} + +impl fmt::Display for NbdError { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + write!( + f, + "NBD Error (errno = {}): {}", + self.errno, + self.strerr.to_string_lossy() + ) + } +} diff --git a/rust/run-tests b/rust/run-tests new file mode 100755 index 000000000000..d1e7b58113e6 --- /dev/null +++ b/rust/run-tests @@ -0,0 +1,4 @@ +#!/bin/sh +set -e + +$CARGO test -- 2.22.0

2 5

[nbdkit] [filter/nozero] large binary size with GCC 9
by Thomas Weißschuh 05 Jul '19

05 Jul '19

Hi all, It seems GCC 9 does not put read-only zero-initialized variables into the BSS section anymore; instead it is put into RODATA. (See the thread at [0], especially [1]) In filter/nozero a 64M large, static, zeroed, read-only array is declared. The new behavior of GCC puts this array as-is into the binary inflating the size by a factor of around 10000. (Clang 8 and older GCCs work "correctly") It would be possible to remove the "const" qualifier or use some compiler attribute trickery to get the previous behavior back. Thanks, Thomas [0] https://gcc.gnu.org/ml/gcc/2019-04/msg00034.html [1] https://gcc.gnu.org/ml/gcc/2019-04/msg00049.html

3 7

[PATCH] v2v: Allow Windows virtio ISO to be a block device as well as a regular file.
by Richard W.M. Jones 05 Jul '19

05 Jul '19

Thanks: Steven Rosenberg --- v2v/windows_virtio.ml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/v2v/windows_virtio.ml b/v2v/windows_virtio.ml index 59b0bf493..56c7a6757 100644 --- a/v2v/windows_virtio.ml +++ b/v2v/windows_virtio.ml @@ -338,7 +338,7 @@ and copy_from_virtio_win g inspect srcdir destdir filter missing = ) paths ) ) - else if is_regular_file virtio_win then ( + else if is_regular_file virtio_win || is_block_device virtio_win then ( debug "windows: copy_from_virtio_win: guest tools source ISO %s" virtio_win; try -- 2.22.0

3 3

[PATCH 0/6] p2v: make it more independent (part #2)
by Pino Toscano 04 Jul '19

04 Jul '19

As preliminary steps in splitting virt-p2v to an own repository, continue making p2v more independent within libguestfs. This is accomplished by the following changes: - have only the authors in the about dialog, and read them from a local AUTHORS file - few more cleanups This is still not complete, although I believe most of the work needed is done, and it still makes p2v usable within libguestfs. Pino Toscano (6): p2v: generate an AUTHORS file p2v: remove non-author credits p2v: ship generate-p2v-config.pl p2v: generate C about data authors from AUTHORS file contrib: move p2v files within p2v p2v: stop including guestfs.h Makefile.am | 5 -- docs/C_SOURCE_FILES | 1 - generator/authors.ml | 59 +++---------------- generator/authors.mli | 2 +- generator/main.ml | 4 +- p2v/.gitignore | 1 + p2v/Makefile.am | 17 +++++- .../contrib}/aux-scripts/do-build.sh | 0 {contrib/p2v => p2v/contrib}/build-p2v-iso.sh | 0 ...BLE-AUTOMATIC-REMOTE-PORT-ALLOCATION.patch | 0 ...-5-ONLY-QEMU-NBD-1.4-HAS-NO-f-OPTION.patch | 0 {contrib/p2v => p2v/contrib}/test-p2v-iso.sh | 0 p2v/generate-p2v-authors.pl | 54 +++++++++++++++++ p2v/gui-gtk3-compat.h | 5 -- p2v/gui.c | 13 ---- p2v/p2v.h | 9 --- 16 files changed, 80 insertions(+), 90 deletions(-) rename {contrib/p2v => p2v/contrib}/aux-scripts/do-build.sh (100%) rename {contrib/p2v => p2v/contrib}/build-p2v-iso.sh (100%) rename {contrib/p2v => p2v/contrib}/patches/0001-RHEL-5-ONLY-DISABLE-AUTOMATIC-REMOTE-PORT-ALLOCATION.patch (100%) rename {contrib/p2v => p2v/contrib}/patches/0002-RHEL-5-ONLY-QEMU-NBD-1.4-HAS-NO-f-OPTION.patch (100%) rename {contrib/p2v => p2v/contrib}/test-p2v-iso.sh (100%) create mode 100755 p2v/generate-p2v-authors.pl -- 2.21.0

2 7

[PATCH libnbd 0/2] Two patches to make libnbd work on FreeBSD.
by Richard W.M. Jones 04 Jul '19

04 Jul '19

Two simple patches which make libnbd compile on FreeBSD. Are we OK to copy common/include/byte-swapping.h from nbdkit? There is no license issue that I know of. Should we put it in lib/ or create a common/ directory? The header file is actually also needed by the tests (follow up patch for that) so putting it in common/ might make more sense. Some notes if you want to compile on FreeBSD: - OCaml is too old to run the generator, so you can't easily build from git. Tarball builds should work, or you can copy the generated files across from a Linux machine. - Running the tests is difficult because there is no nbdkit package for FreeBSD. With a local build of nbdkit you can play with PATH and PKG_CONFIG_PATH to get the tests to work. Rich.

3 9

[libnbd PATCH] tests: Make errors more robust under load
by Eric Blake 04 Jul '19

04 Jul '19

When run under valgrind, the 'errors' test would sometimes fail because a single NBD_CMD_WRITE managed to actually send() the entire packet to the server without blocking the state machine. To make things more robust, switch to a server which is serialized (memory is parallel, but sh is serial), and which intentionally does not read a second command until the first is processed, then switch the client to send two commands rather than one (whether or not the first loses the race with blocking, we can now guarantee the second will block). Since we are now writing a shell script, it is also easy to use that script to add another improvement: check that a server error propagates back to the client as expected. Surprisingly, this uncovered an nbdkit bug (now fixed) - it was very difficult to coerce nbdkit sh into failing with anything other than EIO, because it mistakenly used strcmp() for output ending in a trailing space but not a trailing newline. --- I'm pushing this now, because it at least solved the valgrind failure for me (the failure wansn't 100% pre-patch, but common enough at least once in 10 runs, where now I went 50 runs without the failure). tests/errors.c | 95 +++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 91 insertions(+), 4 deletions(-) diff --git a/tests/errors.c b/tests/errors.c index faa1488..b4ff665 100644 --- a/tests/errors.c +++ b/tests/errors.c @@ -24,6 +24,8 @@ #include <stdlib.h> #include <string.h> #include <errno.h> +#include <unistd.h> +#include <sys/stat.h> #include <libnbd.h> @@ -53,15 +55,73 @@ check (int experr, const char *prefix) } } +static char script[] = "/tmp/libnbd-errors-scriptXXXXXX"; +static char witness[] = "/tmp/libnbd-errors-witnessXXXXXX"; +static int script_fd = -1, witness_fd = -1; + +static void +cleanup (void) +{ + if (script_fd != -1) { + if (script_fd >= 0) + close (script_fd); + unlink (script); + } + if (witness_fd >= 0) { + close (witness_fd); + unlink (witness); + } +} + int main (int argc, char *argv[]) { struct nbd_handle *nbd; - const char *cmd[] = { "nbdkit", "-s", "--exit-with-parent", "memory", - "size=128m", NULL }; + /* We will connect to a custom nbdkit sh plugin which always fails + * on reads (with a precise spelling required for older nbdkit), and + * which delays responding to writes until a witness file no longer + * exists. + */ + const char *cmd[] = { "nbdkit", "-s", "--exit-with-parent", "sh", + script, NULL }; progname = argv[0]; + if (atexit (cleanup) != 0) { + perror ("atexit"); + exit (EXIT_FAILURE); + } + if ((script_fd = mkstemp (script)) == -1) { + perror ("mkstemp"); + exit (EXIT_FAILURE); + } + if ((witness_fd = mkstemp (witness)) == -1) { + perror ("mkstemp"); + exit (EXIT_FAILURE); + } + + if (dprintf (script_fd, "case $1 in\n" + " get_size) echo 128m || exit 1 ;;\n" + " pread) printf 'ENOMEM ' >&2; exit 1 ;;\n" + " can_write) exit 0 ;;\n" + " pwrite)\n" + " while test -e %s; do sleep 1; done\n" + " exit 0;;\n" + " *) exit 2 ;;\n" + "esac\n", witness) < 0) { + perror ("dprintf"); + exit (EXIT_FAILURE); + } + if (fchmod (script_fd, 0700) == -1) { + perror ("fchmod"); + exit (EXIT_FAILURE); + } + if (close (script_fd) == -1) { /* Unlinked later during atexit */ + perror ("close"); + exit (EXIT_FAILURE); + } + script_fd = -2; + nbd = nbd_create (); if (nbd == NULL) { fprintf (stderr, "%s\n", nbd_get_error ()); @@ -167,13 +227,29 @@ main (int argc, char *argv[]) } check (ERANGE, "nbd_aio_pwrite: "); - /* Queue up a write command so large that we block on POLLIN, then queue - * multiple disconnects. + /* Send a read that the nbdkit sh plugin will fail. */ + if (nbd_pread (nbd, buf, 512, 0, 0) != -1) { + fprintf (stderr, "%s: test failed: " + "nbd_pread did not report server failure\n", + argv[0]); + exit (EXIT_FAILURE); + } + check (ENOMEM, "nbd_pread: "); + + /* Queue up two write commands so large that we block on POLLIN (the + * first might not block when under load, such as valgrind, but the + * second definitely will, since the nbdkit sh plugin reads only one + * command at a time and stalls on the first), then queue multiple + * disconnects. */ if (nbd_aio_pwrite (nbd, buf, 2 * 1024 * 1024, 0, 0) == -1) { fprintf (stderr, "%s: %s\n", argv[0], nbd_get_error ()); exit (EXIT_FAILURE); } + if (nbd_aio_pwrite (nbd, buf, 2 * 1024 * 1024, 0, 0) == -1) { + fprintf (stderr, "%s: %s\n", argv[0], nbd_get_error ()); + exit (EXIT_FAILURE); + } if ((nbd_aio_get_direction (nbd) & LIBNBD_AIO_DIRECTION_WRITE) == 0) { fprintf (stderr, "%s: test failed: " "expect to be blocked on write\n", @@ -192,6 +268,17 @@ main (int argc, char *argv[]) } check (EINVAL, "nbd_aio_disconnect: "); + /* Unblock the nbdkit sh plugin */ + if (close (witness_fd) == -1) { + perror ("close"); + exit (EXIT_FAILURE); + } + witness_fd = -1; + if (unlink (witness) == -1) { + perror ("unlink"); + exit (EXIT_FAILURE); + } + /* Flush the queue (whether this one fails is a race with how fast * the server shuts down, so don't enforce status), then try to send * another command while CLOSED/DEAD -- 2.20.1

2 1

[nbdkit PATCH] sh: Parse a larger number of error messages
by Eric Blake 04 Jul '19

04 Jul '19

In order to trigger a non-EIO failure, a script had to output _exactly_ 'EINVAL ' or similar, with one trailing space and no message, because we forgot to limit the length of the string comparison. Fix things to tolerate an error name without a message, as well as to be case-insensitive. Also, parse EOVERFLOW (missed in commit 6f8c8084). Signed-off-by: Eric Blake <eblake(a)redhat.com> --- I'm also working on adding tests/test-sh-errors.h to ensure we don't regress, I'll push that along with this once I get it working. plugins/sh/nbdkit-sh-plugin.pod | 4 +-- plugins/sh/call.c | 45 +++++++++++++++++++++++---------- 2 files changed, 33 insertions(+), 16 deletions(-) diff --git a/plugins/sh/nbdkit-sh-plugin.pod b/plugins/sh/nbdkit-sh-plugin.pod index f108051f..55ee6c69 100644 --- a/plugins/sh/nbdkit-sh-plugin.pod +++ b/plugins/sh/nbdkit-sh-plugin.pod @@ -85,5 +85,5 @@ The method was executed successfully. =item 1 and 8-127 -There was an error. The script may print on stderr an errno and a -message, for example: +There was an error. The script may print on stderr an errno name, +optionally followed by whitespace and a message, for example: ENOSPC Out of space diff --git a/plugins/sh/call.c b/plugins/sh/call.c index f69523c1..585bd32c 100644 --- a/plugins/sh/call.c +++ b/plugins/sh/call.c @@ -42,6 +42,7 @@ #include <poll.h> #include <sys/types.h> #include <sys/wait.h> +#include <ctype.h> #include <nbdkit-plugin.h> @@ -267,32 +268,36 @@ static void handle_script_error (char *ebuf, size_t len) { int err; - size_t skip; + size_t skip = 0; char *p; - if (strcmp (ebuf, "EPERM ") == 0) { + if (strncasecmp (ebuf, "EPERM", 5) == 0) { err = EPERM; - skip = 6; + skip = 5; } - else if (strcmp (ebuf, "EIO ") == 0) { + else if (strncasecmp (ebuf, "EIO", 3) == 0) { err = EIO; - skip = 4; + skip = 3; } - else if (strcmp (ebuf, "ENOMEM ") == 0) { + else if (strncasecmp (ebuf, "ENOMEM", 6) == 0) { err = ENOMEM; - skip = 7; + skip = 6; } - else if (strcmp (ebuf, "EINVAL ") == 0) { + else if (strncasecmp (ebuf, "EINVAL", 6) == 0) { err = EINVAL; - skip = 7; + skip = 6; } - else if (strcmp (ebuf, "ENOSPC ") == 0) { + else if (strncasecmp (ebuf, "ENOSPC", 6) == 0) { err = ENOSPC; skip = 7; } - else if (strcmp (ebuf, "ESHUTDOWN ") == 0) { + else if (strncasecmp (ebuf, "EOVERFLOW", 9) == 0) { + err = EOVERFLOW; + skip = 9; + } + else if (strncasecmp (ebuf, "ESHUTDOWN", 9) == 0) { err = ESHUTDOWN; - skip = 10; + skip = 9; } else { /* Default to EIO. */ @@ -300,6 +305,18 @@ handle_script_error (char *ebuf, size_t len) skip = 0; } + if (skip && ebuf[skip]) { + if (!isspace ((unsigned char) ebuf[skip])) { + /* Treat 'EINVALID' as EIO, not EINVAL */ + err = EIO; + skip = 0; + } + else + do + skip++; + while (isspace ((unsigned char) ebuf[skip])); + } + while (len > 0 && ebuf[len-1] == '\n') ebuf[--len] = '\0'; @@ -311,8 +328,8 @@ handle_script_error (char *ebuf, size_t len) /* ... but truncate it for the error message below. */ *p = '\0'; } - if (strlen (ebuf) >= skip) - ebuf += skip; + assert (strlen (ebuf) >= skip); + ebuf += skip; nbdkit_error ("%s: %s", script, ebuf); } else -- 2.20.1

2 1