11:17 a.m.
Two simple patches which make libnbd compile on FreeBSD.
Are we OK to copy common/include/byte-swapping.h from nbdkit? There
is no license issue that I know of. Should we put it in lib/ or
create a common/ directory? The header file is actually also needed
by the tests (follow up patch for that) so putting it in common/ might
make more sense.
Some notes if you want to compile on FreeBSD:
- OCaml is too old to run the generator, so you can't easily build
from git. Tarball builds should work, or you can copy the
generated files across from a Linux machine.
- Running the tests is difficult because there is no nbdkit package
for FreeBSD. With a local build of nbdkit you can play with PATH
and PKG_CONFIG_PATH to get the tests to work.
Rich.
11:17 a.m.
New subject: [PATCH libnbd 1/2] lib: Use byte-swapping.h from nbdkit.
This allows functions such as beto64h to be used on FreeBSD.
---
configure.ac | 6 +++
lib/Makefile.am | 1 +
lib/byte-swapping.h | 96 +++++++++++++++++++++++++++++++++++++++++++++
lib/internal.h | 1 +
4 files changed, 104 insertions(+)
diff --git a/configure.ac b/configure.ac
index 50e83a7..8b6d99a 100644
--- a/configure.ac
+++ b/configure.ac
@@ -59,6 +59,12 @@ if test "x$gcc_warnings" = "xyes"; then
AC_SUBST([WARNINGS_CFLAGS])
fi
+dnl Check for other headers, all optional.
+AC_CHECK_HEADERS([\
+ byteswap.h \
+ endian.h \
+ sys/endian.h])
+
dnl Check for GnuTLS (optional, for TLS support).
AC_ARG_WITH([gnutls],
[AS_HELP_STRING([--without-gnutls],
diff --git a/lib/Makefile.am b/lib/Makefile.am
index 7939375..9d31d92 100644
--- a/lib/Makefile.am
+++ b/lib/Makefile.am
@@ -36,6 +36,7 @@ BUILT_SOURCES = $(generator_built)
libnbd_la_SOURCES = \
aio.c \
api.c \
+ byte-swapping.h \
connect.c \
crypto.c \
debug.c \
diff --git a/lib/byte-swapping.h b/lib/byte-swapping.h
new file mode 100644
index 0000000..0dd1754
--- /dev/null
+++ b/lib/byte-swapping.h
@@ -0,0 +1,96 @@
+/* nbdkit
+ * Copyright (C) 2013-2018 Red Hat Inc.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are
+ * met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ *
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * * Neither the name of Red Hat nor the names of its contributors may be
+ * used to endorse or promote products derived from this software without
+ * specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
+ * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
+ * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+ * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+ * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+ * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+/* The job of this header is to define macros (or functions) called
+ * things like 'htobe32' and 'le64toh' which byte swap N-bit integers
+ * between host representation, and little and big endian. The core
+ * code and plugins in nbdkit uses these names and relies on this
+ * header to provide the platform-specific implementation. On Linux
+ * these are defined in <endian.h> but other platforms have other
+ * requirements.
+ */
+
+#ifndef NBDKIT_BYTE_SWAPPING_H
+#define NBDKIT_BYTE_SWAPPING_H
+
+#ifdef __HAIKU__
+#define _BSD_SOURCE
+#endif
+
+#ifdef HAVE_BYTESWAP_H
+#include <byteswap.h>
+#endif
+
+#ifdef HAVE_ENDIAN_H
+#include <endian.h>
+#endif
+
+#ifdef HAVE_SYS_ENDIAN_H
+#include <sys/endian.h>
+#endif
+
+#ifndef htobe32
+# if __BYTE_ORDER == __LITTLE_ENDIAN
+# define htobe16(x) __bswap_16 (x)
+# define htole16(x) (x)
+# define be16toh(x) __bswap_16 (x)
+# define le16toh(x) (x)
+
+# define htobe32(x) __bswap_32 (x)
+# define htole32(x) (x)
+# define be32toh(x) __bswap_32 (x)
+# define le32toh(x) (x)
+
+# define htobe64(x) __bswap_64 (x)
+# define htole64(x) (x)
+# define be64toh(x) __bswap_64 (x)
+# define le64toh(x) (x)
+
+# else
+# define htobe16(x) (x)
+# define htole16(x) __bswap_16 (x)
+# define be16toh(x) (x)
+# define le16toh(x) __bswap_16 (x)
+
+# define htobe32(x) (x)
+# define htole32(x) __bswap_32 (x)
+# define be32toh(x) (x)
+# define le32toh(x) __bswap_32 (x)
+
+# define htobe64(x) (x)
+# define htole64(x) __bswap_64 (x)
+# define be64toh(x) (x)
+# define le64toh(x) __bswap_64 (x)
+# endif
+#endif
+
+#endif /* NBDKIT_BYTE_SWAPPING_H */
diff --git a/lib/internal.h b/lib/internal.h
index 0bb8411..0b4364a 100644
--- a/lib/internal.h
+++ b/lib/internal.h
@@ -30,6 +30,7 @@
#include "libnbd.h"
#include "nbd-protocol.h"
+#include "byte-swapping.h"
#include "states.h"
#include "unlocked.h"
--
2.22.0
11:17 a.m.
New subject: [PATCH libnbd 2/2] freebsd: Ignore missing MSG_MORE on older versions of FreeBSD.
As MSG_MORE is only an optimization, define it to 0 on platforms which
lack it, in particular older versions of FreeBSD.
---
lib/internal.h | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/lib/internal.h b/lib/internal.h
index 0b4364a..a954169 100644
--- a/lib/internal.h
+++ b/lib/internal.h
@@ -34,6 +34,11 @@
#include "states.h"
#include "unlocked.h"
+/* MSG_MORE is an optimization. If not present, ignore it. */
+#ifndef MSG_MORE
+#define MSG_MORE 0
+#endif
+
/* XXX This is the same as nbdkit, but probably it should be detected
* from the server (NBD_INFO_BLOCK_SIZE) or made configurable.
*/
--
2.22.0
11:24 a.m.
On 7/3/19 11:17 AM, Richard W.M. Jones wrote:
Two simple patches which make libnbd compile on FreeBSD.
Are we OK to copy common/include/byte-swapping.h from nbdkit? There
is no license issue that I know of.
Agreed. Going from BSD -> LGPL never hurts (it's the reverse direction
where you have to be careful).
Should we put it in lib/ or
create a common/ directory? The header file is actually also needed
by the tests (follow up patch for that) so putting it in common/ might
make more sense.
We don't have a common/ yet, but I don't see any problem with adding
one, the way you did in nbdkit.
Some notes if you want to compile on FreeBSD:
- OCaml is too old to run the generator, so you can't easily build
from git. Tarball builds should work, or you can copy the
generated files across from a Linux machine.
- Running the tests is difficult because there is no nbdkit package
for FreeBSD. With a local build of nbdkit you can play with PATH
and PKG_CONFIG_PATH to get the tests to work.
Rich.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
11:33 a.m.
On Wed, Jul 03, 2019 at 11:24:31AM -0500, Eric Blake wrote:
On 7/3/19 11:17 AM, Richard W.M. Jones wrote:
> Two simple patches which make libnbd compile on FreeBSD.
>
> Are we OK to copy common/include/byte-swapping.h from nbdkit? There
> is no license issue that I know of.
Agreed. Going from BSD -> LGPL never hurts (it's the reverse direction
where you have to be careful).
> Should we put it in lib/ or
> create a common/ directory? The header file is actually also needed
> by the tests (follow up patch for that) so putting it in common/ might
> make more sense.
We don't have a common/ yet, but I don't see any problem with adding
one, the way you did in nbdkit.
Thanks. After trying to compile the tests I need further changes
including using byte-swapping.h from the tests. So I'll withdraw this
particular patch series and come up with something that adds
common/include instead.
Rich.
>
> Some notes if you want to compile on FreeBSD:
>
> - OCaml is too old to run the generator, so you can't easily build
> from git. Tarball builds should work, or you can copy the
> generated files across from a Linux machine.
>
> - Running the tests is difficult because there is no nbdkit package
> for FreeBSD. With a local build of nbdkit you can play with PATH
> and PKG_CONFIG_PATH to get the tests to work.
>
> Rich.
>
>
>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine. Supports Linux and Windows.
http://people.redhat.com/~rjones/virt-df/
2:45 a.m.
On Wed, Jul 03, 2019 at 05:17:42PM +0100, Richard W.M. Jones wrote:
Two simple patches which make libnbd compile on FreeBSD.
Are we OK to copy common/include/byte-swapping.h from nbdkit? There
is no license issue that I know of. Should we put it in lib/ or
create a common/ directory? The header file is actually also needed
by the tests (follow up patch for that) so putting it in common/ might
make more sense.
Some notes if you want to compile on FreeBSD:
- OCaml is too old to run the generator, so you can't easily build
from git. Tarball builds should work, or you can copy the
generated files across from a Linux machine.
Are you sure? I checked repology.org and it says the FreeBSD Ports have 4.05.0,
which I managed to make work by a simple patch some time ago. I'm setting up a
FreeBSD machine to try it out.
- Running the tests is difficult because there is no nbdkit package
for FreeBSD. With a local build of nbdkit you can play with PATH
and PKG_CONFIG_PATH to get the tests to work.
Rich.
_______________________________________________
Libguestfs mailing list
Libguestfs(a)redhat.com
https://www.redhat.com/mailman/listinfo/libguestfs
3:14 a.m.
On Thu, Jul 04, 2019 at 09:45:26AM +0200, Martin Kletzander wrote:
On Wed, Jul 03, 2019 at 05:17:42PM +0100, Richard W.M. Jones wrote:
>Two simple patches which make libnbd compile on FreeBSD.
>
>Are we OK to copy common/include/byte-swapping.h from nbdkit? There
>is no license issue that I know of. Should we put it in lib/ or
>create a common/ directory? The header file is actually also needed
>by the tests (follow up patch for that) so putting it in common/ might
>make more sense.
>
>Some notes if you want to compile on FreeBSD:
>
>- OCaml is too old to run the generator, so you can't easily build
> from git. Tarball builds should work, or you can copy the
> generated files across from a Linux machine.
>
Are you sure? I checked repology.org and it says the FreeBSD Ports have 4.05.0,
which I managed to make work by a simple patch some time ago. I'm setting up a
FreeBSD machine to try it out.
Probably because I'm not running the latest FreeBSD. I think I was
using 11.2.
Rich.
>- Running the tests is difficult because there is no nbdkit
package
> for FreeBSD. With a local build of nbdkit you can play with PATH
> and PKG_CONFIG_PATH to get the tests to work.
>
>Rich.
>
>
>_______________________________________________
>Libguestfs mailing list
>Libguestfs(a)redhat.com
>https://www.redhat.com/mailman/listinfo/libguestfs
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines. Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v
3:25 a.m.
On Thu, Jul 04, 2019 at 09:14:13AM +0100, Richard W.M. Jones wrote:
On Thu, Jul 04, 2019 at 09:45:26AM +0200, Martin Kletzander wrote:
> On Wed, Jul 03, 2019 at 05:17:42PM +0100, Richard W.M. Jones wrote:
> >Two simple patches which make libnbd compile on FreeBSD.
> >
> >Are we OK to copy common/include/byte-swapping.h from nbdkit? There
> >is no license issue that I know of. Should we put it in lib/ or
> >create a common/ directory? The header file is actually also needed
> >by the tests (follow up patch for that) so putting it in common/ might
> >make more sense.
> >
> >Some notes if you want to compile on FreeBSD:
> >
> >- OCaml is too old to run the generator, so you can't easily build
> > from git. Tarball builds should work, or you can copy the
> > generated files across from a Linux machine.
> >
>
> Are you sure? I checked repology.org and it says the FreeBSD Ports have 4.05.0,
> which I managed to make work by a simple patch some time ago. I'm setting up a
> FreeBSD machine to try it out.
Probably because I'm not running the latest FreeBSD. I think I was
using 11.2.
The Ports should be the same across releases. But maybe it was not updated?
Anyway, I tried finding that in freebsd-11.1 (because that is the one available
from virt-builder) and we found two issues:
- The template suggests there is /dev/sda5, which it definitely looks like it
from virt-rescue, but all following operations (parted, sgdisk, etc.) fail
because the GPT is invalid, so they load the MBR which only has /dev/sda1
(the MRB extended partition on which /dev/sda5 and /dev/sda6 live), but they
do not see any other partitions and fail. I thought I could fix it locally
by recreating the table based on what the kernel found (no idea how, but it
knows about /dev/sda5 and /dev/sda6). The partition types were unknown to
me, but I somehow managed to make it so that it does not report invalid GPT.
- The second issue is that the libguestfs-appliance-1.40.1 which I am using
(prebuilt upstream) does not have a kernel module for UFS, so it cannot even
inspect the filesystem.
Pino told me to report both of these to you, hence the above. If you want me to
write them down in a bug report somewhere, let me know.
Rich.
> >- Running the tests is difficult because there is no nbdkit package
> > for FreeBSD. With a local build of nbdkit you can play with PATH
> > and PKG_CONFIG_PATH to get the tests to work.
> >
> >Rich.
> >
> >
> >_______________________________________________
> >Libguestfs mailing list
> >Libguestfs(a)redhat.com
> >https://www.redhat.com/mailman/listinfo/libguestfs
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines. Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v
3:40 a.m.
On Thu, Jul 04, 2019 at 10:25:03AM +0200, Martin Kletzander wrote:
On Thu, Jul 04, 2019 at 09:14:13AM +0100, Richard W.M. Jones wrote:
>On Thu, Jul 04, 2019 at 09:45:26AM +0200, Martin Kletzander wrote:
>>On Wed, Jul 03, 2019 at 05:17:42PM +0100, Richard W.M. Jones wrote:
>>>Two simple patches which make libnbd compile on FreeBSD.
>>>
>>>Are we OK to copy common/include/byte-swapping.h from nbdkit? There
>>>is no license issue that I know of. Should we put it in lib/ or
>>>create a common/ directory? The header file is actually also needed
>>>by the tests (follow up patch for that) so putting it in common/ might
>>>make more sense.
>>>
>>>Some notes if you want to compile on FreeBSD:
>>>
>>>- OCaml is too old to run the generator, so you can't easily build
>>> from git. Tarball builds should work, or you can copy the
>>> generated files across from a Linux machine.
>>>
>>
>>Are you sure? I checked repology.org and it says the FreeBSD Ports have 4.05.0,
>>which I managed to make work by a simple patch some time ago. I'm setting up
a
>>FreeBSD machine to try it out.
>
>Probably because I'm not running the latest FreeBSD. I think I was
>using 11.2.
>
The Ports should be the same across releases. But maybe it was not updated?
Anyway, I tried finding that in freebsd-11.1 (because that is the one available
from virt-builder) and we found two issues:
- The template suggests there is /dev/sda5, which it definitely looks like it
from virt-rescue, but all following operations (parted, sgdisk, etc.) fail
because the GPT is invalid, so they load the MBR which only has /dev/sda1
(the MRB extended partition on which /dev/sda5 and /dev/sda6 live), but they
do not see any other partitions and fail. I thought I could fix it locally
by recreating the table based on what the kernel found (no idea how, but it
knows about /dev/sda5 and /dev/sda6). The partition types were unknown to
me, but I somehow managed to make it so that it does not report invalid GPT.
- The second issue is that the libguestfs-appliance-1.40.1 which I am using
(prebuilt upstream) does not have a kernel module for UFS, so it cannot even
inspect the filesystem.
Pino told me to report both of these to you, hence the above. If you want me to
write them down in a bug report somewhere, let me know.
I think the FreeBSD template needs some work, but no time to do that
now.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines. Supports shell scripting,
bindings from many languages. http://libguestfs.org
5:36 a.m.
On Thu, Jul 04, 2019 at 09:40:55AM +0100, Richard W.M. Jones wrote:
On Thu, Jul 04, 2019 at 10:25:03AM +0200, Martin Kletzander wrote:
> On Thu, Jul 04, 2019 at 09:14:13AM +0100, Richard W.M. Jones wrote:
> >On Thu, Jul 04, 2019 at 09:45:26AM +0200, Martin Kletzander wrote:
> >>On Wed, Jul 03, 2019 at 05:17:42PM +0100, Richard W.M. Jones wrote:
> >>>Two simple patches which make libnbd compile on FreeBSD.
> >>>
> >>>Are we OK to copy common/include/byte-swapping.h from nbdkit? There
> >>>is no license issue that I know of. Should we put it in lib/ or
> >>>create a common/ directory? The header file is actually also needed
> >>>by the tests (follow up patch for that) so putting it in common/ might
> >>>make more sense.
> >>>
> >>>Some notes if you want to compile on FreeBSD:
> >>>
> >>>- OCaml is too old to run the generator, so you can't easily build
> >>> from git. Tarball builds should work, or you can copy the
> >>> generated files across from a Linux machine.
> >>>
> >>
> >>Are you sure? I checked repology.org and it says the FreeBSD Ports have
4.05.0,
> >>which I managed to make work by a simple patch some time ago. I'm
setting up a
> >>FreeBSD machine to try it out.
> >
> >Probably because I'm not running the latest FreeBSD. I think I was
> >using 11.2.
> >
>
> The Ports should be the same across releases. But maybe it was not updated?
> Anyway, I tried finding that in freebsd-11.1 (because that is the one available
> from virt-builder) and we found two issues:
>
> - The template suggests there is /dev/sda5, which it definitely looks like it
> from virt-rescue, but all following operations (parted, sgdisk, etc.) fail
> because the GPT is invalid, so they load the MBR which only has /dev/sda1
> (the MRB extended partition on which /dev/sda5 and /dev/sda6 live), but they
> do not see any other partitions and fail. I thought I could fix it locally
> by recreating the table based on what the kernel found (no idea how, but it
> knows about /dev/sda5 and /dev/sda6). The partition types were unknown to
> me, but I somehow managed to make it so that it does not report invalid GPT.
>
> - The second issue is that the libguestfs-appliance-1.40.1 which I am using
> (prebuilt upstream) does not have a kernel module for UFS, so it cannot even
> inspect the filesystem.
>
> Pino told me to report both of these to you, hence the above. If you want me to
> write them down in a bug report somewhere, let me know.
I think the FreeBSD template needs some work, but no time to do that
now.
Oh, sure, it's nothing pressing, I just wanted to say that so that I do not
forget and also to do what I promised to Pino. Also when talking on our way for
lunch we figured out it's not a wrong image, it's just that FreeBSD uses BSD
disklabel and slicing instead of normal or extended partitions. That's it, I
can forget about it and move on with other things now =)
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines. Supports shell scripting,
bindings from many languages. http://libguestfs.org
1969
days inactive
1970
days old
9 comments
3 participants
participants (3)
-
Eric Blake -
Martin Kletzander -
Richard W.M. Jones