Publishing the libnbd go module
by Nir Soffer
I'm playing with libnbd go module, planning to use it in a new command[1]
The biggest obstacle for me is that the module is not published in a way that
Go developers expect.
The module is listed in:
https://pkg.go.dev/github.com/libguestfs/libnbd/golang
But the module actually lives in:
https://github.com/libguestfs/libnbd/tree/master/golang/src/libguestfs.or...
So the pkg.go.dev page is broken, .e.g no there is no documation or license, and
the suggested import is wrong.
The module name is "libguestfs.org/libnbd". But if you try to use it,
for example
in the (improved) example from libnbd-golang.pod:
$ cat test.go
package main
import "fmt"
import "libguestfs.org/libnbd"
func main() {
h, err := libnbd.Create()
if err != nil {
panic(err)
}
defer h.Close()
uri := "nbd://localhost"
err = h.ConnectUri(uri)
if err != nil {
panic(err)
}
size, err := h.GetSize()
if err != nil {
panic(err)
}
fmt.Printf("size of %s = %d\n", uri, size)
}
$ go mod init example/test
go: creating new go.mod: module example/test
go: to add module requirements and sums:
go mod tidy
$ go mod tidy
go: finding module for package libguestfs.org/libnbd
example/test imports
libguestfs.org/libnbd: cannot find module providing package
libguestfs.org/libnbd: unrecognized import path
"libguestfs.org/libnbd": reading
https://libguestfs.org/libnbd?go-get=1: 404 Not Found
If we use libguestfs.org, https://libguestfs.org/libnbd?go-get=1
should return the expected
metadata instead of 404.
But even if "libguestfs.org/libnbd" would work, we cannot use the
module from the source
since the source is missing the generated files (wrappers.go, binding.go, ...).
It looks like the only ways to use the module are:
- Vendor the go code from the tarball.
I did not try this since I don't want to copy libnbd into source into
my project.
- Clone and build libnbd locally, and replace libguestfs.org with the
path to your local source
$ go mod edit -replace
libguestfs.org/libnbd=../../src/libnbd/golang/src/libguestfs.org/libnbd
$ go mod tidy
go: found libguestfs.org/libnbd in libguestfs.org/libnbd
v0.0.0-00010101000000-000000000000
$ cat go.mod
module example/test
go 1.16
replace libguestfs.org/libnbd =>
../../src/libnbd/golang/src/libguestfs.org/libnbd
require libguestfs.org/libnbd v0.0.0-00010101000000-000000000000
But the version is wrong - it should be v1.10.0.
I think the issue is missing tag:
https://golang.org/ref/mod#vcs-version
If a module is defined in a subdirectory within the repository,
that is, the module subdirectory
portion of the module path is not empty, then each tag name must
be prefixed with the module
subdirectory, followed by a slash. For example, the module
golang.org/x/tools/gopls is defined
in the gopls subdirectory of the repository with root path
golang.org/x/tools. The version v0.4.0
of that module must have the tag named gopls/v0.4.0 in that repository.
So the linbd project needs a tag like:
golang/src/libguestfs.org/libnbd/v1.10.0
Removing the "src/libguestfs.org" directories will clean things up:
golang/libnbd/v1.10.0
I hope we can solve this issue. Making the go binding easy to use for
developers is
important for making libnbd more popular in the Go community.
[1] https://gerrit.ovirt.org/c/ovirt-imageio/+/117277
Nir
2 years, 6 months
[PATCH v3 0/5] qapi: Add feature flags to enum members
by Markus Armbruster
PATCH 1+2 add feature flags to enum members. Awkward due to an
introspection design mistake; see PATCH 1 for details.
PATCH 3+4 implement policy deprecated-input={reject,crash} for enum
values.
Policy deprecated-output=hide is not implemented, because we can't
hide a value without hiding the entire member, which is almost
certainly more than the requester of this policy bargained for.
Perhaps we want a new policy deprecated-output=hide-or-else-crash to
help us catch unwanted use of deprecated enum values. Perhaps we want
deprecated-output=hide to behave that way together with
deprecated-input=crash. Or even always. Thoughts?
PATCH 5 puts the new feature flags to use. It's RFC because it makes
sense only on top of Vladimir's deprecation of drive-backup. See its
commit message for a reference.
I prefer to commit new features together with a use outside tests/.
PATCH 5 adds such a use, but it's RFC, because it depends on
Vladimir's work. Perhaps another use pops up. I can delay this work
in the hope of a use becoming ready, but the feature flags work I have
in the pipeline will eventually force my hand.
v3:
* PATCH 1+2: Update qapi-code-gen.rst [Kevin, Eric]
* PATCH 4: Commit message typo [Eric], doc update moved to PATCH 2
* PATCH 5: Doc comment FIXME resolved [Kevin]
v2:
* Rebased with straightforward conflicts.
* PATCH 1-4: No longer RFC.
* PATCH 1: "Since" information fixed [Eric]. Commit message updated
to reflect feedback.
* PATCH 2: Commit message amended to point out special feature flag
'deprecated' is ignored at this stage.
* PATCH 4: Documentation updated. Commit message tweaked.
Markus Armbruster (5):
qapi: Enable enum member introspection to show more than name
qapi: Add feature flags to enum members
qapi: Move compat policy from QObject to generic visitor
qapi: Implement deprecated-input={reject,crash} for enum values
block: Deprecate transaction type drive-backup
docs/devel/qapi-code-gen.rst | 29 ++++++++++++++-----
qapi/compat.json | 3 ++
qapi/introspect.json | 24 +++++++++++++--
qapi/transaction.json | 6 +++-
include/qapi/qobject-input-visitor.h | 4 ---
include/qapi/qobject-output-visitor.h | 4 ---
include/qapi/util.h | 6 +++-
include/qapi/visitor-impl.h | 3 ++
include/qapi/visitor.h | 9 ++++++
qapi/qapi-visit-core.c | 27 +++++++++++++++--
qapi/qmp-dispatch.c | 4 +--
qapi/qobject-input-visitor.c | 14 +--------
qapi/qobject-output-visitor.c | 14 +--------
scripts/qapi/expr.py | 3 +-
scripts/qapi/introspect.py | 19 +++++++++---
scripts/qapi/schema.py | 22 ++++++++++++--
scripts/qapi/types.py | 17 ++++++++++-
tests/qapi-schema/doc-good.json | 5 +++-
tests/qapi-schema/doc-good.out | 3 ++
tests/qapi-schema/doc-good.txt | 3 ++
.../qapi-schema/enum-dict-member-unknown.err | 2 +-
tests/qapi-schema/qapi-schema-test.json | 3 +-
tests/qapi-schema/qapi-schema-test.out | 1 +
tests/qapi-schema/test-qapi.py | 1 +
24 files changed, 164 insertions(+), 62 deletions(-)
--
2.31.1
2 years, 6 months
Re: [Libguestfs] [libguestfs/nbdkit] nbdkit server hangs during negotiation when running in background, works in foreground (Issue #16)
by Richard W.M. Jones
On Sun, Oct 24, 2021 at 01:07:19AM -0700, Nathan Shearer wrote:
> With my current setup, the issue is reproducible. If I start the
> nbdkit server by placing a startup script in /etc/local.d to start
> on boot, it will start, but the nbd-client will always hang during
> negotiation. Killing and restarting the server and/or the client
> does not resolve the deadlock.
>
> However, if I run the exact same script manually, by logging into
> the server, then nbd-client is able to successfully connect to the
> nbdkit server and everything works as expected.
>
> During my tests, I found that when I killed nbdkit, it would see
> errors about their being insufficient "magic" or an invalid
> password. I am not using any passwords, and both the nbdkit server
> and nbd-client are on the same host, connecting through 127.0.0.1
I'm a bit unclear - how does it hang? What happens if you enable
verbose messages (-v), what messages are sent to the log?
Also try connecting with
LIBNBD_DEBUG=1 nbdinfo nbd://server
and see where it hangs in negotiation.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines. Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top
2 years, 6 months
[virt-v2v PATCH wave 1] lib/types: remove "source video: QXL" constructor
by Laszlo Ersek
* Background:
We intend to stop virt-v2v from outputting converted domains with a QXL
display device [1]. The reason is that QXL is a complex device model with
a large security attack surface, while at the same time, better device
models exist with regard to either guest compatibility, or graphics
acceleration. Namely, if compatibility takes priority, standard VGA is at
least as widely supported by guests [2] [3] [4], with lesser security
risks. And if graphics performance takes priority, then 2D acceleration is
generally considered obsolete, and virtio-vga is where new 3D features are
being developed [5].
[1] https://bugzilla.redhat.com/show_bug.cgi?id=1961107#c1
[2] https://bugzilla.redhat.com/show_bug.cgi?id=1961107#c2
[3] https://www.kraxel.org/blog/2019/09/display-devices-in-qemu/#qxl-vga
[4] https://www.kraxel.org/blog/2019/09/display-devices-in-qemu/#VGA
[5] https://www.kraxel.org/blog/2019/09/display-devices-in-qemu/#virtio-vga
For virt-v2v in particular, compatibility is what matters, so standard VGA
will ultimately supplant QXL in newly converted domains.
* About this patch:
In order to get the ball rolling, remove the "Source_QXL" constructor of
the "source_video" union type.
"Source_QXL" was originally introduced in commit [6], and then put to use
in commit [7], specifically for the sake of in-place conversions, so that
the "input" QXL display device serve as a distinguished hint for the
"output" domain (after conversion).
With the modularization of virt-v2v in commit [8], in-place conversion has
been (temporarily) removed. Therefore, "Source_QXL" is functionally dead
code, at the time of writing.
[6] 04af2bc3cd2c ("v2v: collect source network and video adapter types",
2016-03-24)
[7] 3c21ad036296 ("v2v: in-place: request caps based on source config",
2016-03-24)
[8] 255722cbf39a ("v2v: Modular virt-v2v", 2021-09-07)
"Functionally dead" means that, mentions of the QXL display device in the
*source* domain descriptions of the test suite can be satisfied by the
parametric "Source_other_video of string" constructor just as well; there
is no actual functionality attached to "Source_QXL" at the moment. And
once we reimplement in-place conversion for modular virt-v2v, we'll still
want to flip domains that use QXL to standard VGA (see the background
above).
(Further patches are expected for RHBZ#1961107.)
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1961107
Signed-off-by: Laszlo Ersek <lersek(a)redhat.com>
---
lib/types.ml | 4 +---
lib/types.mli | 2 +-
2 files changed, 2 insertions(+), 4 deletions(-)
diff --git a/lib/types.ml b/lib/types.ml
index 27f488255fd7..72d10d1ec774 100644
--- a/lib/types.ml
+++ b/lib/types.ml
@@ -90,7 +90,7 @@ and s_display_listen =
| LNone
and source_video = Source_other_video of string |
- Source_Cirrus | Source_QXL
+ Source_Cirrus
and source_sound = {
s_sound_model : source_sound_model;
@@ -260,12 +260,10 @@ and string_of_source_display { s_display_type = typ;
)
and string_of_source_video = function
- | Source_QXL -> "qxl"
| Source_Cirrus -> "cirrus"
| Source_other_video video -> video
and source_video_of_string = function
- | "qxl" -> Source_QXL
| "cirrus" -> Source_Cirrus
| video -> Source_other_video video
diff --git a/lib/types.mli b/lib/types.mli
index 2af5871fd54b..0bae445d8393 100644
--- a/lib/types.mli
+++ b/lib/types.mli
@@ -147,7 +147,7 @@ and s_display_listen =
(** Video adapter model. *)
and source_video = Source_other_video of string |
- Source_Cirrus | Source_QXL
+ Source_Cirrus
and source_sound = {
s_sound_model : source_sound_model; (** Sound model. *)
base-commit: 7ce5df273be304ec3ba87e46319cf78275eb4990
--
2.19.1.3.g30247aa5d201
2 years, 7 months
[PATCH v2 0/5] qapi: Add feature flags to enum members
by Markus Armbruster
PATCH 1+2 add feature flags to enum members. Awkward due to an
introspection design mistake; see PATCH 1 for details.
PATCH 3+4 implement policy deprecated-input={reject,crash} for enum
values.
Policy deprecated-output=hide is not implemented, because we can't
hide a value without hiding the entire member, which is almost
certainly more than the requester of this policy bargained for.
Perhaps we want a new policy deprecated-output=hide-or-else-crash to
help us catch unwanted use of deprecated enum values. Perhaps we want
deprecated-output=hide to behave that way together with
deprecated-input=crash. Or even always. Thoughts?
PATCH 5 puts the new feature flags to use. It's RFC because it makes
sense only on top of Vladimir's deprecation of drive-backup. See its
commit message for a reference.
I prefer to commit new features together with a use outside tests/.
PATCH 5 adds such a use, but it's RFC, because it depends on
Vladimir's work. Perhaps another use pops up. I can delay this work
in the hope of a use becoming ready, but the feature flags work I have
in the pipeline will eventually force my hand.
v2:
* Rebased with straightforward conflicts.
* PATCH 1-4: No longer RFC.
* PATCH 1: "Since" information fixed [Eric]. Commit message updated
to reflect feedback.
* PATCH 2: Commit message amended to point out special feature flag
'deprecated' is ignored at this stage.
* PATCH 4: Documentation updated. Commit message tweaked.
Markus Armbruster (5):
qapi: Enable enum member introspection to show more than name
qapi: Add feature flags to enum members
qapi: Move compat policy from QObject to generic visitor
qapi: Implement deprecated-input={reject,crash} for enum values
block: Deprecate transaction type drive-backup
docs/devel/qapi-code-gen.rst | 10 ++++---
qapi/compat.json | 3 +++
qapi/introspect.json | 24 +++++++++++++++--
qapi/transaction.json | 5 +++-
include/qapi/qobject-input-visitor.h | 4 ---
include/qapi/qobject-output-visitor.h | 4 ---
include/qapi/util.h | 6 ++++-
include/qapi/visitor-impl.h | 3 +++
include/qapi/visitor.h | 9 +++++++
qapi/qapi-visit-core.c | 27 ++++++++++++++++---
qapi/qmp-dispatch.c | 4 +--
qapi/qobject-input-visitor.c | 14 +---------
qapi/qobject-output-visitor.c | 14 +---------
scripts/qapi/expr.py | 3 ++-
scripts/qapi/introspect.py | 19 ++++++++++---
scripts/qapi/schema.py | 22 +++++++++++++--
scripts/qapi/types.py | 17 +++++++++++-
tests/qapi-schema/doc-good.json | 5 +++-
tests/qapi-schema/doc-good.out | 3 +++
tests/qapi-schema/doc-good.txt | 3 +++
.../qapi-schema/enum-dict-member-unknown.err | 2 +-
tests/qapi-schema/qapi-schema-test.json | 3 ++-
tests/qapi-schema/qapi-schema-test.out | 1 +
tests/qapi-schema/test-qapi.py | 1 +
24 files changed, 149 insertions(+), 57 deletions(-)
--
2.31.1
2 years, 7 months
[PATCH nbdkit v2 0/2] New filter: nbdkit-retry-request-filter
by Richard W.M. Jones
I believe this addresses everything mentioned in the previous reviews
(sorry if I missed anything). In addition I have extended
web-server.c to implement a fairly realistic test scenario.
Alexander: It would be useful if you could test this filter in your
code as well so we can be sure it really fixes your problem.
Rich.
2 years, 7 months
[PATCH nbdkit] curl: Add effective-url flag
by Richard W.M. Jones
Probably best to read this first:
https://bugzilla.redhat.com/show_bug.cgi?id=2013000
This adds an effective-url=true|false flag to nbdkit-curl-plugin. If
true, the first time we fetch the URL, we fetch the "effective" URL
(ie. the URL after all redirects were resolved) and use that for all
future connections within the current nbdkit instance.
The idea behind this patch is to do with the Fedora mirror system
which is flakey. Some mirrors return errors or 404s. And the mirror
system will give you a new redirect each time (within your
geographical region). This results in transfers sometimes failing
because a single read went to a bad mirror.
After implementing this patch I'm not very happy with it. It is
incomplete because we pass the original URL to cookie-script and
header-script, so plugin/curl/scripts.c will also need to be modified.
Once those modifications are done the change becomes quite invasive.
Also I'm not convinced that it really solves any problem. In the
manual change I wrote:
Note use of this feature in long-lived nbdkit instances can cause
subtle problems:
• The effective URL persists across connections for the lifetime
of the nbdkit instance. If nbdkit is used for a long time then
it is possible for the redirected URL to become stale.
• It will defeat some mirror load-balancing techniques.
• If the mirror service sometimes redirects to a broken URL and
it happens that the URL you fetch first is broken then nbdkit
will no longer recover on subsequent connections (instead you
will need to restart nbdkit).
I suggested another way to solve this by using curl APIs to fetch the
effective URL up front and passing that URL to nbdkit (see
https://bugzilla.redhat.com/show_bug.cgi?id=2013000#c1), but
apparently that solution isn't acceptable for unclear reasons.
I can't think of any other way to solve this in the context of nbdkit
(maybe have it detect when redirection is happening and retry the
redirection on error?). So here's the patch.
Rich.
2 years, 7 months
