Publishing the libnbd go module
I'm playing with libnbd go module, planning to use it in a new command[1] The biggest obstacle for me is that the module is not published in a way that Go developers expect. The module is listed in: https://pkg.go.dev/github.com/libguestfs/libnbd/golang But the module actually lives in: https://github.com/libguestfs/libnbd/tree/master/golang/src/libguestfs.org/l... So the pkg.go.dev page is broken, .e.g no there is no documation or license, and the suggested import is wrong. The module name is "libguestfs.org/libnbd". But if you try to use it, for example in the (improved) example from libnbd-golang.pod: $ cat test.go package main import "fmt" import "libguestfs.org/libnbd" func main() { h, err := libnbd.Create() if err != nil { panic(err) } defer h.Close() uri := "nbd://localhost" err = h.ConnectUri(uri) if err != nil { panic(err) } size, err := h.GetSize() if err != nil { panic(err) } fmt.Printf("size of %s = %d\n", uri, size) } $ go mod init example/test go: creating new go.mod: module example/test go: to add module requirements and sums: go mod tidy $ go mod tidy go: finding module for package libguestfs.org/libnbd example/test imports libguestfs.org/libnbd: cannot find module providing package libguestfs.org/libnbd: unrecognized import path "libguestfs.org/libnbd": reading https://libguestfs.org/libnbd?go-get=1: 404 Not Found If we use libguestfs.org, https://libguestfs.org/libnbd?go-get=1 should return the expected metadata instead of 404. But even if "libguestfs.org/libnbd" would work, we cannot use the module from the source since the source is missing the generated files (wrappers.go, binding.go, ...). It looks like the only ways to use the module are: - Vendor the go code from the tarball. I did not try this since I don't want to copy libnbd into source into my project. - Clone and build libnbd locally, and replace libguestfs.org with the path to your local source $ go mod edit -replace libguestfs.org/libnbd=../../src/libnbd/golang/src/libguestfs.org/libnbd $ go mod tidy go: found libguestfs.org/libnbd in libguestfs.org/libnbd v0.0.0-00010101000000-000000000000 $ cat go.mod module example/test go 1.16 replace libguestfs.org/libnbd => ../../src/libnbd/golang/src/libguestfs.org/libnbd require libguestfs.org/libnbd v0.0.0-00010101000000-000000000000 But the version is wrong - it should be v1.10.0. I think the issue is missing tag: https://golang.org/ref/mod#vcs-version If a module is defined in a subdirectory within the repository, that is, the module subdirectory portion of the module path is not empty, then each tag name must be prefixed with the module subdirectory, followed by a slash. For example, the module golang.org/x/tools/gopls is defined in the gopls subdirectory of the repository with root path golang.org/x/tools. The version v0.4.0 of that module must have the tag named gopls/v0.4.0 in that repository. So the linbd project needs a tag like: golang/src/libguestfs.org/libnbd/v1.10.0 Removing the "src/libguestfs.org" directories will clean things up: golang/libnbd/v1.10.0 I hope we can solve this issue. Making the go binding easy to use for developers is important for making libnbd more popular in the Go community. [1] https://gerrit.ovirt.org/c/ovirt-imageio/+/117277 Nir
[Adding Matthew] On Mon, Oct 25, 2021 at 04:45:03PM +0300, Nir Soffer wrote:
I'm playing with libnbd go module, planning to use it in a new command[1]
The biggest obstacle for me is that the module is not published in a way that Go developers expect.
The module is listed in: https://pkg.go.dev/github.com/libguestfs/libnbd/golang
But the module actually lives in: https://github.com/libguestfs/libnbd/tree/master/golang/src/libguestfs.org/l...
So the pkg.go.dev page is broken, .e.g no there is no documation or license, and the suggested import is wrong.
The module name is "libguestfs.org/libnbd". But if you try to use it, for example in the (improved) example from libnbd-golang.pod:
$ cat test.go package main
import "fmt" import "libguestfs.org/libnbd"
func main() { h, err := libnbd.Create() if err != nil { panic(err) } defer h.Close() uri := "nbd://localhost" err = h.ConnectUri(uri) if err != nil { panic(err) } size, err := h.GetSize() if err != nil { panic(err) } fmt.Printf("size of %s = %d\n", uri, size) }
$ go mod init example/test go: creating new go.mod: module example/test go: to add module requirements and sums: go mod tidy
$ go mod tidy go: finding module for package libguestfs.org/libnbd example/test imports libguestfs.org/libnbd: cannot find module providing package libguestfs.org/libnbd: unrecognized import path "libguestfs.org/libnbd": reading https://libguestfs.org/libnbd?go-get=1: 404 Not Found
That website is entirely static so if it involves fetching stuff from there it's probably not going to work.
If we use libguestfs.org, https://libguestfs.org/libnbd?go-get=1 should return the expected metadata instead of 404.
But even if "libguestfs.org/libnbd" would work, we cannot use the module from the source since the source is missing the generated files (wrappers.go, binding.go, ...).
I believe that Matthew hit this problem too ...
It looks like the only ways to use the module are:
- Vendor the go code from the tarball.
I did not try this since I don't want to copy libnbd into source into my project.
- Clone and build libnbd locally, and replace libguestfs.org with the path to your local source
$ go mod edit -replace libguestfs.org/libnbd=../../src/libnbd/golang/src/libguestfs.org/libnbd $ go mod tidy go: found libguestfs.org/libnbd in libguestfs.org/libnbd v0.0.0-00010101000000-000000000000
$ cat go.mod module example/test
go 1.16
replace libguestfs.org/libnbd => ../../src/libnbd/golang/src/libguestfs.org/libnbd
require libguestfs.org/libnbd v0.0.0-00010101000000-000000000000
But the version is wrong - it should be v1.10.0. I think the issue is missing tag: https://golang.org/ref/mod#vcs-version
If a module is defined in a subdirectory within the repository, that is, the module subdirectory portion of the module path is not empty, then each tag name must be prefixed with the module subdirectory, followed by a slash. For example, the module golang.org/x/tools/gopls is defined in the gopls subdirectory of the repository with root path golang.org/x/tools. The version v0.4.0 of that module must have the tag named gopls/v0.4.0 in that repository.
So the linbd project needs a tag like: golang/src/libguestfs.org/libnbd/v1.10.0
I'm not sure I understand what this all means. Is that the literal name of the git tag (ie. git tag golang/src/libguestfs.org/libnbd/v1.10.0 )?
Removing the "src/libguestfs.org" directories will clean things up: golang/libnbd/v1.10.0
I hope we can solve this issue. Making the go binding easy to use for developers is important for making libnbd more popular in the Go community.
Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org
On Mon, Oct 25, 2021 at 5:01 PM Richard W.M. Jones <rjones@redhat.com> wrote:
[Adding Matthew]
On Mon, Oct 25, 2021 at 04:45:03PM +0300, Nir Soffer wrote:
I'm playing with libnbd go module, planning to use it in a new command[1]
The biggest obstacle for me is that the module is not published in a way that Go developers expect.
The module is listed in: https://pkg.go.dev/github.com/libguestfs/libnbd/golang
But the module actually lives in: https://github.com/libguestfs/libnbd/tree/master/golang/src/libguestfs.org/l...
So the pkg.go.dev page is broken, .e.g no there is no documation or license, and the suggested import is wrong.
The module name is "libguestfs.org/libnbd". But if you try to use it, for example in the (improved) example from libnbd-golang.pod:
$ cat test.go package main
import "fmt" import "libguestfs.org/libnbd"
func main() { h, err := libnbd.Create() if err != nil { panic(err) } defer h.Close() uri := "nbd://localhost" err = h.ConnectUri(uri) if err != nil { panic(err) } size, err := h.GetSize() if err != nil { panic(err) } fmt.Printf("size of %s = %d\n", uri, size) }
$ go mod init example/test go: creating new go.mod: module example/test go: to add module requirements and sums: go mod tidy
$ go mod tidy go: finding module for package libguestfs.org/libnbd example/test imports libguestfs.org/libnbd: cannot find module providing package libguestfs.org/libnbd: unrecognized import path "libguestfs.org/libnbd": reading https://libguestfs.org/libnbd?go-get=1: 404 Not Found
That website is entirely static so if it involves fetching stuff from there it's probably not going to work.
So the import path should be: gitlab.com/libnbd/golang/src/libguestfs.org/libnbd Since gitlab (or github) already supports go tools. ...
$ go mod edit -replace libguestfs.org/libnbd=../../src/libnbd/golang/src/libguestfs.org/libnbd $ go mod tidy go: found libguestfs.org/libnbd in libguestfs.org/libnbd v0.0.0-00010101000000-000000000000
$ cat go.mod module example/test
go 1.16
replace libguestfs.org/libnbd => ../../src/libnbd/golang/src/libguestfs.org/libnbd
require libguestfs.org/libnbd v0.0.0-00010101000000-000000000000
But the version is wrong - it should be v1.10.0. I think the issue is missing tag: https://golang.org/ref/mod#vcs-version
If a module is defined in a subdirectory within the repository, that is, the module subdirectory portion of the module path is not empty, then each tag name must be prefixed with the module subdirectory, followed by a slash. For example, the module golang.org/x/tools/gopls is defined in the gopls subdirectory of the repository with root path golang.org/x/tools. The version v0.4.0 of that module must have the tag named gopls/v0.4.0 in that repository.
So the linbd project needs a tag like: golang/src/libguestfs.org/libnbd/v1.10.0
I'm not sure I understand what this all means. Is that the literal name of the git tag (ie. git tag golang/src/libguestfs.org/libnbd/v1.10.0 )?
Yes, this is how I understand the docs. I did not try it yet. Nir
On Mon, Oct 25, 2021 at 05:22:28PM +0300, Nir Soffer wrote:
On Mon, Oct 25, 2021 at 5:01 PM Richard W.M. Jones <rjones@redhat.com> wrote:
[Adding Matthew]
On Mon, Oct 25, 2021 at 04:45:03PM +0300, Nir Soffer wrote:
I'm playing with libnbd go module, planning to use it in a new command[1]
The biggest obstacle for me is that the module is not published in a way that Go developers expect.
The module is listed in: https://pkg.go.dev/github.com/libguestfs/libnbd/golang
But the module actually lives in: https://github.com/libguestfs/libnbd/tree/master/golang/src/libguestfs.org/l...
So the pkg.go.dev page is broken, .e.g no there is no documation or license, and the suggested import is wrong.
The module name is "libguestfs.org/libnbd". But if you try to use it, for example in the (improved) example from libnbd-golang.pod:
$ cat test.go package main
import "fmt" import "libguestfs.org/libnbd"
func main() { h, err := libnbd.Create() if err != nil { panic(err) } defer h.Close() uri := "nbd://localhost" err = h.ConnectUri(uri) if err != nil { panic(err) } size, err := h.GetSize() if err != nil { panic(err) } fmt.Printf("size of %s = %d\n", uri, size) }
$ go mod init example/test go: creating new go.mod: module example/test go: to add module requirements and sums: go mod tidy
$ go mod tidy go: finding module for package libguestfs.org/libnbd example/test imports libguestfs.org/libnbd: cannot find module providing package libguestfs.org/libnbd: unrecognized import path "libguestfs.org/libnbd": reading https://libguestfs.org/libnbd?go-get=1: 404 Not Found
That website is entirely static so if it involves fetching stuff from there it's probably not going to work.
So the import path should be:
gitlab.com/libnbd/golang/src/libguestfs.org/libnbd
Since gitlab (or github) already supports go tools.
Depends if you want applications to be susceptible to future changes in where the code is located. A libguestfs.org/libnbd import would mean you can move git hosting without impacting apps. It is fairly simple to configure - you just need the web address https://libguestfs.org/libnbd to serve up an HTML document that contains a magic <meta> tag in the page header <meta name="go-import" content="libguestfs.org/libnbd git https://gitlab.com/path/to/real/git/repo/location"/> Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On Mon, Oct 25, 2021 at 10:01 AM Richard W.M. Jones <rjones@redhat.com> wrote:
But even if "libguestfs.org/libnbd" would work, we cannot use the module from the source since the source is missing the generated files (wrappers.go, binding.go, ...).
I believe that Matthew hit this problem too ...
(I forgot to reply to all...) I ended up creating a new repository with the generated files: https://github.com/mrnold/go-libnbd In my case it sort of made sense because I also needed to change it to use LDFLAGS instead of pkg-config, since Bazel ignores pkg-config. But I would be happy to get rid of this if there's a better way. On Mon, Oct 25, 2021 at 10:01 AM Richard W.M. Jones <rjones@redhat.com> wrote:
[Adding Matthew]
I'm playing with libnbd go module, planning to use it in a new command[1]
The biggest obstacle for me is that the module is not published in a way
On Mon, Oct 25, 2021 at 04:45:03PM +0300, Nir Soffer wrote: that
Go developers expect.
The module is listed in: https://pkg.go.dev/github.com/libguestfs/libnbd/golang
But the module actually lives in:
https://github.com/libguestfs/libnbd/tree/master/golang/src/libguestfs.org/l...
So the pkg.go.dev page is broken, .e.g no there is no documation or
license, and
the suggested import is wrong.
The module name is "libguestfs.org/libnbd". But if you try to use it, for example in the (improved) example from libnbd-golang.pod:
$ cat test.go package main
import "fmt" import "libguestfs.org/libnbd"
func main() { h, err := libnbd.Create() if err != nil { panic(err) } defer h.Close() uri := "nbd://localhost" err = h.ConnectUri(uri) if err != nil { panic(err) } size, err := h.GetSize() if err != nil { panic(err) } fmt.Printf("size of %s = %d\n", uri, size) }
$ go mod init example/test go: creating new go.mod: module example/test go: to add module requirements and sums: go mod tidy
$ go mod tidy go: finding module for package libguestfs.org/libnbd example/test imports libguestfs.org/libnbd: cannot find module providing package libguestfs.org/libnbd: unrecognized import path "libguestfs.org/libnbd": reading https://libguestfs.org/libnbd?go-get=1: 404 Not Found
That website is entirely static so if it involves fetching stuff from there it's probably not going to work.
If we use libguestfs.org, https://libguestfs.org/libnbd?go-get=1 should return the expected metadata instead of 404.
But even if "libguestfs.org/libnbd" would work, we cannot use the module from the source since the source is missing the generated files (wrappers.go, binding.go, ...).
I believe that Matthew hit this problem too ...
It looks like the only ways to use the module are:
- Vendor the go code from the tarball.
I did not try this since I don't want to copy libnbd into source into my project.
- Clone and build libnbd locally, and replace libguestfs.org with the path to your local source
$ go mod edit -replace libguestfs.org/libnbd=../../src/libnbd/golang/src/libguestfs.org/libnbd $ go mod tidy go: found libguestfs.org/libnbd in libguestfs.org/libnbd v0.0.0-00010101000000-000000000000
$ cat go.mod module example/test
go 1.16
replace libguestfs.org/libnbd => ../../src/libnbd/golang/src/libguestfs.org/libnbd
require libguestfs.org/libnbd v0.0.0-00010101000000-000000000000
But the version is wrong - it should be v1.10.0. I think the issue is missing tag: https://golang.org/ref/mod#vcs-version
If a module is defined in a subdirectory within the repository, that is, the module subdirectory portion of the module path is not empty, then each tag name must be prefixed with the module subdirectory, followed by a slash. For example, the module golang.org/x/tools/gopls is defined in the gopls subdirectory of the repository with root path golang.org/x/tools. The version v0.4.0 of that module must have the tag named gopls/v0.4.0 in that repository.
So the linbd project needs a tag like: golang/src/libguestfs.org/libnbd/v1.10.0
I'm not sure I understand what this all means. Is that the literal name of the git tag (ie. git tag golang/src/libguestfs.org/libnbd/v1.10.0 )?
Removing the "src/libguestfs.org" directories will clean things up: golang/libnbd/v1.10.0
I hope we can solve this issue. Making the go binding easy to use for developers is important for making libnbd more popular in the Go community.
Rich.
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org
On Mon, Oct 25, 2021 at 04:45:03PM +0300, Nir Soffer wrote:
I'm playing with libnbd go module, planning to use it in a new command[1]
The biggest obstacle for me is that the module is not published in a way that Go developers expect.
The module is listed in: https://pkg.go.dev/github.com/libguestfs/libnbd/golang
But the module actually lives in: https://github.com/libguestfs/libnbd/tree/master/golang/src/libguestfs.org/l...
So the pkg.go.dev page is broken, .e.g no there is no documation or license, and the suggested import is wrong.
The module name is "libguestfs.org/libnbd". But if you try to use it, for example in the (improved) example from libnbd-golang.pod:
$ cat test.go package main
import "fmt" import "libguestfs.org/libnbd"
func main() { h, err := libnbd.Create() if err != nil { panic(err) } defer h.Close() uri := "nbd://localhost" err = h.ConnectUri(uri) if err != nil { panic(err) } size, err := h.GetSize() if err != nil { panic(err) } fmt.Printf("size of %s = %d\n", uri, size) }
$ go mod init example/test go: creating new go.mod: module example/test go: to add module requirements and sums: go mod tidy
$ go mod tidy go: finding module for package libguestfs.org/libnbd example/test imports libguestfs.org/libnbd: cannot find module providing package libguestfs.org/libnbd: unrecognized import path "libguestfs.org/libnbd": reading https://libguestfs.org/libnbd?go-get=1: 404 Not Found
If we use libguestfs.org, https://libguestfs.org/libnbd?go-get=1 should return the expected metadata instead of 404.
But even if "libguestfs.org/libnbd" would work, we cannot use the module from the source since the source is missing the generated files (wrappers.go, binding.go, ...).
It looks like the only ways to use the module are:
- Vendor the go code from the tarball.
I did not try this since I don't want to copy libnbd into source into my project.
- Clone and build libnbd locally, and replace libguestfs.org with the path to your local source
$ go mod edit -replace libguestfs.org/libnbd=../../src/libnbd/golang/src/libguestfs.org/libnbd $ go mod tidy go: found libguestfs.org/libnbd in libguestfs.org/libnbd v0.0.0-00010101000000-000000000000
$ cat go.mod module example/test
go 1.16
replace libguestfs.org/libnbd => ../../src/libnbd/golang/src/libguestfs.org/libnbd
require libguestfs.org/libnbd v0.0.0-00010101000000-000000000000
But the version is wrong - it should be v1.10.0. I think the issue is missing tag: https://golang.org/ref/mod#vcs-version
If a module is defined in a subdirectory within the repository, that is, the module subdirectory portion of the module path is not empty, then each tag name must be prefixed with the module subdirectory, followed by a slash. For example, the module golang.org/x/tools/gopls is defined in the gopls subdirectory of the repository with root path golang.org/x/tools. The version v0.4.0 of that module must have the tag named gopls/v0.4.0 in that repository.
So the linbd project needs a tag like: golang/src/libguestfs.org/libnbd/v1.10.0
Removing the "src/libguestfs.org" directories will clean things up: golang/libnbd/v1.10.0
Go modules are a bit annoying in that they're effectively designed around the use of semver for versioning. Anything from v2.xxxx onwards means you need to put the module code in sub-directory named after the major version number. This means all your apps need to update their imports any time you change major version. In libvirt's go modules we had to re-create the git repos with new tags based on semver instead of calver, to get out of the ever changing sub-directory trap. libnbd is lucky that it is still on v1.xxxx, so isn't in the sub-dir naming trap yet. I feel like as a general rule life is simpler if the Go code is all in a dedicated repo, completely separate from any other non-Go code, and any auto-generated code is committed too, and wired up such that "go gen" will trigger whatever command is needed to re-generate. Assuming you don't want to change libnbd build system at all though, you could have a separate git repo where you import the generated Go binding at time of each release. It'll still get a little odd when people want to submit bugs/patches, as they will need to know to submit to the main repo, not this import-only repo. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On Mon, Oct 25, 2021 at 5:04 PM Daniel P. Berrangé <berrange@redhat.com> wrote:
On Mon, Oct 25, 2021 at 04:45:03PM +0300, Nir Soffer wrote:
I'm playing with libnbd go module, planning to use it in a new command[1]
The biggest obstacle for me is that the module is not published in a way that Go developers expect.
The module is listed in: https://pkg.go.dev/github.com/libguestfs/libnbd/golang
But the module actually lives in: https://github.com/libguestfs/libnbd/tree/master/golang/src/libguestfs.org/l...
So the pkg.go.dev page is broken, .e.g no there is no documation or license, and the suggested import is wrong.
The module name is "libguestfs.org/libnbd". But if you try to use it, for example in the (improved) example from libnbd-golang.pod:
$ cat test.go package main
import "fmt" import "libguestfs.org/libnbd"
func main() { h, err := libnbd.Create() if err != nil { panic(err) } defer h.Close() uri := "nbd://localhost" err = h.ConnectUri(uri) if err != nil { panic(err) } size, err := h.GetSize() if err != nil { panic(err) } fmt.Printf("size of %s = %d\n", uri, size) }
$ go mod init example/test go: creating new go.mod: module example/test go: to add module requirements and sums: go mod tidy
$ go mod tidy go: finding module for package libguestfs.org/libnbd example/test imports libguestfs.org/libnbd: cannot find module providing package libguestfs.org/libnbd: unrecognized import path "libguestfs.org/libnbd": reading https://libguestfs.org/libnbd?go-get=1: 404 Not Found
If we use libguestfs.org, https://libguestfs.org/libnbd?go-get=1 should return the expected metadata instead of 404.
But even if "libguestfs.org/libnbd" would work, we cannot use the module from the source since the source is missing the generated files (wrappers.go, binding.go, ...).
It looks like the only ways to use the module are:
- Vendor the go code from the tarball.
I did not try this since I don't want to copy libnbd into source into my project.
- Clone and build libnbd locally, and replace libguestfs.org with the path to your local source
$ go mod edit -replace libguestfs.org/libnbd=../../src/libnbd/golang/src/libguestfs.org/libnbd $ go mod tidy go: found libguestfs.org/libnbd in libguestfs.org/libnbd v0.0.0-00010101000000-000000000000
$ cat go.mod module example/test
go 1.16
replace libguestfs.org/libnbd => ../../src/libnbd/golang/src/libguestfs.org/libnbd
require libguestfs.org/libnbd v0.0.0-00010101000000-000000000000
But the version is wrong - it should be v1.10.0. I think the issue is missing tag: https://golang.org/ref/mod#vcs-version
If a module is defined in a subdirectory within the repository, that is, the module subdirectory portion of the module path is not empty, then each tag name must be prefixed with the module subdirectory, followed by a slash. For example, the module golang.org/x/tools/gopls is defined in the gopls subdirectory of the repository with root path golang.org/x/tools. The version v0.4.0 of that module must have the tag named gopls/v0.4.0 in that repository.
So the linbd project needs a tag like: golang/src/libguestfs.org/libnbd/v1.10.0
Removing the "src/libguestfs.org" directories will clean things up: golang/libnbd/v1.10.0
Go modules are a bit annoying in that they're effectively designed around the use of semver for versioning. Anything from v2.xxxx onwards means you need to put the module code in sub-directory named after the major version number. This means all your apps need to update their imports any time you change major version.
In libvirt's go modules we had to re-create the git repos with new tags based on semver instead of calver, to get out of the ever changing sub-directory trap.
libnbd is lucky that it is still on v1.xxxx, so isn't in the sub-dir naming trap yet.
I feel like as a general rule life is simpler if the Go code is all in a dedicated repo, completely separate from any other non-Go code, and any auto-generated code is committed too, and wired up such that "go gen" will trigger whatever command is needed to re-generate.
Assuming you don't want to change libnbd build system at all though, you could have a separate git repo where you import the generated Go binding at time of each release. It'll still get a little odd when people want to submit bugs/patches, as they will need to know to submit to the main repo, not this import-only repo.
It seems like Go tools are optimized to have each module in its own repo. For my case, I really want to have the go modules in the imageio repo, since the modules are a client library and client command line, and having them inside the repo with the server makes it easier to test. For example I can write tests start the server from source, and testing the client against the server. This ensures that changes in the server will never break the client when all this happens in the CI. Nir
On Mon, Oct 25, 2021 at 03:04:47PM +0100, Daniel P. Berrangé wrote:
On Mon, Oct 25, 2021 at 04:45:03PM +0300, Nir Soffer wrote:
I'm playing with libnbd go module, planning to use it in a new command[1]
The biggest obstacle for me is that the module is not published in a way that Go developers expect.
The module is listed in: https://pkg.go.dev/github.com/libguestfs/libnbd/golang
But the module actually lives in: https://github.com/libguestfs/libnbd/tree/master/golang/src/libguestfs.org/l...
So the pkg.go.dev page is broken, .e.g no there is no documation or license, and the suggested import is wrong.
The module name is "libguestfs.org/libnbd". But if you try to use it, for example in the (improved) example from libnbd-golang.pod:
$ cat test.go package main
import "fmt" import "libguestfs.org/libnbd"
func main() { h, err := libnbd.Create() if err != nil { panic(err) } defer h.Close() uri := "nbd://localhost" err = h.ConnectUri(uri) if err != nil { panic(err) } size, err := h.GetSize() if err != nil { panic(err) } fmt.Printf("size of %s = %d\n", uri, size) }
$ go mod init example/test go: creating new go.mod: module example/test go: to add module requirements and sums: go mod tidy
$ go mod tidy go: finding module for package libguestfs.org/libnbd example/test imports libguestfs.org/libnbd: cannot find module providing package libguestfs.org/libnbd: unrecognized import path "libguestfs.org/libnbd": reading https://libguestfs.org/libnbd?go-get=1: 404 Not Found
If we use libguestfs.org, https://libguestfs.org/libnbd?go-get=1 should return the expected metadata instead of 404.
But even if "libguestfs.org/libnbd" would work, we cannot use the module from the source since the source is missing the generated files (wrappers.go, binding.go, ...).
It looks like the only ways to use the module are:
- Vendor the go code from the tarball.
I did not try this since I don't want to copy libnbd into source into my project.
- Clone and build libnbd locally, and replace libguestfs.org with the path to your local source
$ go mod edit -replace libguestfs.org/libnbd=../../src/libnbd/golang/src/libguestfs.org/libnbd $ go mod tidy go: found libguestfs.org/libnbd in libguestfs.org/libnbd v0.0.0-00010101000000-000000000000
$ cat go.mod module example/test
go 1.16
replace libguestfs.org/libnbd => ../../src/libnbd/golang/src/libguestfs.org/libnbd
require libguestfs.org/libnbd v0.0.0-00010101000000-000000000000
But the version is wrong - it should be v1.10.0. I think the issue is missing tag: https://golang.org/ref/mod#vcs-version
If a module is defined in a subdirectory within the repository, that is, the module subdirectory portion of the module path is not empty, then each tag name must be prefixed with the module subdirectory, followed by a slash. For example, the module golang.org/x/tools/gopls is defined in the gopls subdirectory of the repository with root path golang.org/x/tools. The version v0.4.0 of that module must have the tag named gopls/v0.4.0 in that repository.
So the linbd project needs a tag like: golang/src/libguestfs.org/libnbd/v1.10.0
Removing the "src/libguestfs.org" directories will clean things up: golang/libnbd/v1.10.0
Go modules are a bit annoying in that they're effectively designed around the use of semver for versioning. Anything from v2.xxxx onwards means you need to put the module code in sub-directory named after the major version number. This means all your apps need to update their imports any time you change major version.
In libvirt's go modules we had to re-create the git repos with new tags based on semver instead of calver, to get out of the ever changing sub-directory trap.
libnbd is lucky that it is still on v1.xxxx, so isn't in the sub-dir naming trap yet.
I feel like as a general rule life is simpler if the Go code is all in a dedicated repo, completely separate from any other non-Go code, and any auto-generated code is committed too, and wired up such that "go gen" will trigger whatever command is needed to re-generate.
Assuming you don't want to change libnbd build system at all though, you could have a separate git repo where you import the generated Go binding at time of each release. It'll still get a little odd when people want to submit bugs/patches, as they will need to know to submit to the main repo, not this import-only repo.
Hopefully Matthew will chime in because I'm pretty sure we've been here before with someone from the KubeVirt team. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-df lists disk usage of guests without needing to install any software inside the virtual machine. Supports Linux and Windows. http://people.redhat.com/~rjones/virt-df/
Couple of months back I was playing with GitHub Actions so that the libnbd-go is regenerated after each commit and the changes pushed into a separate repository, tagging it if tag was pushed as well. Unfortunately more pressing things came my way and I did not finish this work. Now I am making terribly slow progress on the CI in GitLab, but nevertheless the aforementioned approach could still work. AFAIU we would just need to figure out the versioning. If anyone wants to take a stab at it, it might help, or is my understanding of go modules flawed in this respect? Thanks for any corrections. Have a nice day Martin On Mon, Oct 25, 2021 at 04:45:03PM +0300, Nir Soffer wrote:
I'm playing with libnbd go module, planning to use it in a new command[1]
The biggest obstacle for me is that the module is not published in a way that Go developers expect.
The module is listed in: https://pkg.go.dev/github.com/libguestfs/libnbd/golang
But the module actually lives in: https://github.com/libguestfs/libnbd/tree/master/golang/src/libguestfs.org/l...
So the pkg.go.dev page is broken, .e.g no there is no documation or license, and the suggested import is wrong.
The module name is "libguestfs.org/libnbd". But if you try to use it, for example in the (improved) example from libnbd-golang.pod:
$ cat test.go package main
import "fmt" import "libguestfs.org/libnbd"
func main() { h, err := libnbd.Create() if err != nil { panic(err) } defer h.Close() uri := "nbd://localhost" err = h.ConnectUri(uri) if err != nil { panic(err) } size, err := h.GetSize() if err != nil { panic(err) } fmt.Printf("size of %s = %d\n", uri, size) }
$ go mod init example/test go: creating new go.mod: module example/test go: to add module requirements and sums: go mod tidy
$ go mod tidy go: finding module for package libguestfs.org/libnbd example/test imports libguestfs.org/libnbd: cannot find module providing package libguestfs.org/libnbd: unrecognized import path "libguestfs.org/libnbd": reading https://libguestfs.org/libnbd?go-get=1: 404 Not Found
If we use libguestfs.org, https://libguestfs.org/libnbd?go-get=1 should return the expected metadata instead of 404.
But even if "libguestfs.org/libnbd" would work, we cannot use the module from the source since the source is missing the generated files (wrappers.go, binding.go, ...).
It looks like the only ways to use the module are:
- Vendor the go code from the tarball.
I did not try this since I don't want to copy libnbd into source into my project.
- Clone and build libnbd locally, and replace libguestfs.org with the path to your local source
$ go mod edit -replace libguestfs.org/libnbd=../../src/libnbd/golang/src/libguestfs.org/libnbd $ go mod tidy go: found libguestfs.org/libnbd in libguestfs.org/libnbd v0.0.0-00010101000000-000000000000
$ cat go.mod module example/test
go 1.16
replace libguestfs.org/libnbd => ../../src/libnbd/golang/src/libguestfs.org/libnbd
require libguestfs.org/libnbd v0.0.0-00010101000000-000000000000
But the version is wrong - it should be v1.10.0. I think the issue is missing tag: https://golang.org/ref/mod#vcs-version
If a module is defined in a subdirectory within the repository, that is, the module subdirectory portion of the module path is not empty, then each tag name must be prefixed with the module subdirectory, followed by a slash. For example, the module golang.org/x/tools/gopls is defined in the gopls subdirectory of the repository with root path golang.org/x/tools. The version v0.4.0 of that module must have the tag named gopls/v0.4.0 in that repository.
So the linbd project needs a tag like: golang/src/libguestfs.org/libnbd/v1.10.0
Removing the "src/libguestfs.org" directories will clean things up: golang/libnbd/v1.10.0
I hope we can solve this issue. Making the go binding easy to use for developers is important for making libnbd more popular in the Go community.
[1] https://gerrit.ovirt.org/c/ovirt-imageio/+/117277
Nir
_______________________________________________ Libguestfs mailing list Libguestfs@redhat.com https://listman.redhat.com/mailman/listinfo/libguestfs
On Mon, Oct 25, 2021 at 4:45 PM Nir Soffer <nsoffer@redhat.com> wrote:
I'm playing with libnbd go module, planning to use it in a new command[1]
The biggest obstacle for me is that the module is not published in a way that Go developers expect.
The module is listed in: https://pkg.go.dev/github.com/libguestfs/libnbd/golang
But the module actually lives in: https://github.com/libguestfs/libnbd/tree/master/golang/src/libguestfs.org/l...
So the pkg.go.dev page is broken, .e.g no there is no documation or license, and the suggested import is wrong.
The module name is "libguestfs.org/libnbd". But if you try to use it, for example in the (improved) example from libnbd-golang.pod:
$ cat test.go package main
import "fmt" import "libguestfs.org/libnbd"
func main() { h, err := libnbd.Create() if err != nil { panic(err) } defer h.Close() uri := "nbd://localhost" err = h.ConnectUri(uri) if err != nil { panic(err) } size, err := h.GetSize() if err != nil { panic(err) } fmt.Printf("size of %s = %d\n", uri, size) }
$ go mod init example/test go: creating new go.mod: module example/test go: to add module requirements and sums: go mod tidy
$ go mod tidy go: finding module for package libguestfs.org/libnbd example/test imports libguestfs.org/libnbd: cannot find module providing package libguestfs.org/libnbd: unrecognized import path "libguestfs.org/libnbd": reading https://libguestfs.org/libnbd?go-get=1: 404 Not Found
If we use libguestfs.org, https://libguestfs.org/libnbd?go-get=1 should return the expected metadata instead of 404.
But even if "libguestfs.org/libnbd" would work, we cannot use the module from the source since the source is missing the generated files (wrappers.go, binding.go, ...).
It looks like the only ways to use the module are:
- Vendor the go code from the tarball.
I did not try this since I don't want to copy libnbd into source into my project.
- Clone and build libnbd locally, and replace libguestfs.org with the path to your local source
$ go mod edit -replace libguestfs.org/libnbd=../../src/libnbd/golang/src/libguestfs.org/libnbd $ go mod tidy go: found libguestfs.org/libnbd in libguestfs.org/libnbd v0.0.0-00010101000000-000000000000
$ cat go.mod module example/test
go 1.16
replace libguestfs.org/libnbd => ../../src/libnbd/golang/src/libguestfs.org/libnbd
require libguestfs.org/libnbd v0.0.0-00010101000000-000000000000
But the version is wrong - it should be v1.10.0. I think the issue is missing tag: https://golang.org/ref/mod#vcs-version
If a module is defined in a subdirectory within the repository, that is, the module subdirectory portion of the module path is not empty, then each tag name must be prefixed with the module subdirectory, followed by a slash. For example, the module golang.org/x/tools/gopls is defined in the gopls subdirectory of the repository with root path golang.org/x/tools. The version v0.4.0 of that module must have the tag named gopls/v0.4.0 in that repository.
So the linbd project needs a tag like: golang/src/libguestfs.org/libnbd/v1.10.0
Removing the "src/libguestfs.org" directories will clean things up: golang/libnbd/v1.10.0
I hope we can solve this issue. Making the go binding easy to use for developers is important for making libnbd more popular in the Go community.
Another issue, what is the license of the libnbd Go module? The project license is LGPL 2.1, but how can tools find the license when we point them to a subdirectory?, and the license is not using a standard file name? Some modules have a GPL header, and some LGPL: $ git grep -n 'GNU General Public' golang | wc -l 66 $ git grep -n 'GNU Lesser General Public' golang | wc -l 6 I guess using LGPL is the right license? I think the directory with the module should have a LICENSE file to make this clear, and libnbd-golang.pod should have a LICENSE section. Nir
On Mon, Oct 25, 2021 at 08:05:47PM +0300, Nir Soffer wrote:
On Mon, Oct 25, 2021 at 4:45 PM Nir Soffer <nsoffer@redhat.com> wrote:
I'm playing with libnbd go module, planning to use it in a new command[1]
The biggest obstacle for me is that the module is not published in a way that Go developers expect.
The module is listed in: https://pkg.go.dev/github.com/libguestfs/libnbd/golang
But the module actually lives in: https://github.com/libguestfs/libnbd/tree/master/golang/src/libguestfs.org/l...
So the pkg.go.dev page is broken, .e.g no there is no documation or license, and the suggested import is wrong.
The module name is "libguestfs.org/libnbd". But if you try to use it, for example in the (improved) example from libnbd-golang.pod:
$ cat test.go package main
import "fmt" import "libguestfs.org/libnbd"
func main() { h, err := libnbd.Create() if err != nil { panic(err) } defer h.Close() uri := "nbd://localhost" err = h.ConnectUri(uri) if err != nil { panic(err) } size, err := h.GetSize() if err != nil { panic(err) } fmt.Printf("size of %s = %d\n", uri, size) }
$ go mod init example/test go: creating new go.mod: module example/test go: to add module requirements and sums: go mod tidy
$ go mod tidy go: finding module for package libguestfs.org/libnbd example/test imports libguestfs.org/libnbd: cannot find module providing package libguestfs.org/libnbd: unrecognized import path "libguestfs.org/libnbd": reading https://libguestfs.org/libnbd?go-get=1: 404 Not Found
If we use libguestfs.org, https://libguestfs.org/libnbd?go-get=1 should return the expected metadata instead of 404.
But even if "libguestfs.org/libnbd" would work, we cannot use the module from the source since the source is missing the generated files (wrappers.go, binding.go, ...).
It looks like the only ways to use the module are:
- Vendor the go code from the tarball.
I did not try this since I don't want to copy libnbd into source into my project.
- Clone and build libnbd locally, and replace libguestfs.org with the path to your local source
$ go mod edit -replace libguestfs.org/libnbd=../../src/libnbd/golang/src/libguestfs.org/libnbd $ go mod tidy go: found libguestfs.org/libnbd in libguestfs.org/libnbd v0.0.0-00010101000000-000000000000
$ cat go.mod module example/test
go 1.16
replace libguestfs.org/libnbd => ../../src/libnbd/golang/src/libguestfs.org/libnbd
require libguestfs.org/libnbd v0.0.0-00010101000000-000000000000
But the version is wrong - it should be v1.10.0. I think the issue is missing tag: https://golang.org/ref/mod#vcs-version
If a module is defined in a subdirectory within the repository, that is, the module subdirectory portion of the module path is not empty, then each tag name must be prefixed with the module subdirectory, followed by a slash. For example, the module golang.org/x/tools/gopls is defined in the gopls subdirectory of the repository with root path golang.org/x/tools. The version v0.4.0 of that module must have the tag named gopls/v0.4.0 in that repository.
So the linbd project needs a tag like: golang/src/libguestfs.org/libnbd/v1.10.0
Removing the "src/libguestfs.org" directories will clean things up: golang/libnbd/v1.10.0
I hope we can solve this issue. Making the go binding easy to use for developers is important for making libnbd more popular in the Go community.
Another issue, what is the license of the libnbd Go module?
The project license is LGPL 2.1, but how can tools find the license when we point them to a subdirectory?, and the license is not using a standard file name?
Some modules have a GPL header, and some LGPL:
$ git grep -n 'GNU General Public' golang | wc -l 66 $ git grep -n 'GNU Lesser General Public' golang | wc -l 6
I guess using LGPL is the right license?
The license is supposed to be LGPLv2+ (see COPYING.LIB in the main source directory). However two of the files do have an incorrect GPL license statement in them, which I'll fix shortly. Note that the tests have the correct (GPL) license - these should not be linked into the end user's programs so GPL is fine for those. Rich.
I think the directory with the module should have a LICENSE file to make this clear, and libnbd-golang.pod should have a LICENSE section.
Nir
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top
participants (5)
-
Daniel P. Berrangé -
Martin Kletzander -
Matthew Arnold -
Nir Soffer -
Richard W.M. Jones