[PATCH nbdinfo 1/2] common/utils: Add function to convert sizes to human-readable
For example 1024 is returned as "1K". This does not attempt to handle decimals or SI units. If the number isn't some multiple of a power of 1024 then it is returned as bytes (a flag is available to indicate this). I looked at both the gnulib and qemu versions of this function. The gnulib version is not under a license which is compatible with libnbd and is also really complicated, although it does handle fractions and SI units. The qemu version is essentially just frexp + sprintf and doesn't attempt to convert to the human-readable version reversibly. --- .gitignore | 1 + common/utils/Makefile.am | 10 +++- common/utils/human-size.c | 54 +++++++++++++++++++++ common/utils/human-size.h | 49 +++++++++++++++++++ common/utils/test-human-size.c | 89 ++++++++++++++++++++++++++++++++++ 5 files changed, 201 insertions(+), 2 deletions(-) diff --git a/.gitignore b/.gitignore index 2aa1fd99..5fc59677 100644 --- a/.gitignore +++ b/.gitignore @@ -31,6 +31,7 @@ Makefile.in /bash/nbdcopy /bash/nbdfuse /bash/nbdinfo +/common/utils/test-human-size /common/utils/test-vector /compile /config.cache diff --git a/common/utils/Makefile.am b/common/utils/Makefile.am index 1ca4a370..b273ada1 100644 --- a/common/utils/Makefile.am +++ b/common/utils/Makefile.am @@ -34,6 +34,8 @@ include $(top_srcdir)/common-rules.mk noinst_LTLIBRARIES = libutils.la libutils_la_SOURCES = \ + human-size.c \ + human-size.h \ vector.c \ vector.h \ version.c \ @@ -50,8 +52,12 @@ libutils_la_LIBADD = \ # Unit tests. -TESTS = test-vector -check_PROGRAMS = test-vector +TESTS = test-human-size test-vector +check_PROGRAMS = test-human-size test-vector + +test_human_size_SOURCES = test-human-size.c human-size.c human-size.h +test_human_size_CPPFLAGS = -I$(srcdir) +test_human_size_CFLAGS = $(WARNINGS_CFLAGS) test_vector_SOURCES = test-vector.c vector.c vector.h test_vector_CPPFLAGS = -I$(srcdir) diff --git a/common/utils/human-size.c b/common/utils/human-size.c new file mode 100644 index 00000000..5cff722b --- /dev/null +++ b/common/utils/human-size.c @@ -0,0 +1,54 @@ +/* nbd client library in userspace + * Copyright (C) 2020-2021 Red Hat Inc. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include <config.h> + +#include <stdio.h> +#include <stdlib.h> +#include <stdbool.h> +#include <stdint.h> +#include <inttypes.h> + +#include "human-size.h" + +char * +human_size (char *buf, uint64_t bytes, bool *human) +{ + static const char *ext[] = { "E", "P", "T", "G", "M", "K", "" }; + size_t i; + + if (buf == NULL) { + buf = malloc (HUMAN_SIZE_LONGEST); + if (buf == NULL) + return NULL; + } + + /* Set to false if we're not going to add a human-readable extension. */ + if (human) + *human = bytes != 0 && (bytes & 1023) == 0; + + /* Work out which extension to use, if any. */ + for (i = 6; i >= 0; --i) { + if (bytes == 0 || (bytes & 1023) != 0) + break; + bytes /= 1024; + } + + snprintf (buf, HUMAN_SIZE_LONGEST-1, "%" PRIu64 "%s", bytes, ext[i]); + return buf; +} diff --git a/common/utils/human-size.h b/common/utils/human-size.h new file mode 100644 index 00000000..9ee78803 --- /dev/null +++ b/common/utils/human-size.h @@ -0,0 +1,49 @@ +/* nbd client library in userspace + * Copyright (C) 2020-2021 Red Hat Inc. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef LIBNBD_HUMAN_SIZE_H +#define LIBNBD_HUMAN_SIZE_H + +#include <stdbool.h> +#include <stdint.h> + +/* If you allocate a buffer of at least this length in bytes and pass + * it as the first parameter to human_size, then it will not overrun. + */ +#define HUMAN_SIZE_LONGEST 64 + +/* Convert bytes to a human-readable string. + * + * This is roughly the opposite of nbdkit_parse_size. It will convert + * multiples of powers of 1024 to the appropriate human size with the + * right extension like 'M' or 'G'. Anything that cannot be converted + * is returned as bytes. The *human flag is set to true if the output + * was abbreviated to a human-readable size, or false if it is just + * bytes. + * + * If buf == NULL, a buffer is allocated and returned. In this case + * the returned buffer must be freed. + * + * buf may also be allocated by the caller, in which case it must be + * at least HUMAN_SIZE_LONGEST bytes. + * + * On error the function returns an error and sets errno. + */ +extern char *human_size (char *buf, uint64_t bytes, bool *human); + +#endif /* LIBNBD_HUMAN_SIZE_H */ diff --git a/common/utils/test-human-size.c b/common/utils/test-human-size.c new file mode 100644 index 00000000..d35a21bf --- /dev/null +++ b/common/utils/test-human-size.c @@ -0,0 +1,89 @@ +/* nbd client library in userspace + * Copyright (C) 2020-2021 Red Hat Inc. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include <config.h> + +#include <stdio.h> +#include <stdlib.h> +#include <stdbool.h> +#include <stdint.h> +#include <inttypes.h> +#include <string.h> + +#include "human-size.h" + +static unsigned errors = 0; + +static void +test (uint64_t bytes, const char *expected, bool expected_human_flag) +{ + char actual[HUMAN_SIZE_LONGEST]; + bool actual_human_flag; + + human_size (actual, bytes, &actual_human_flag); + + if (strcmp (actual, expected) == 0 || + actual_human_flag != expected_human_flag) { + printf ("test-human-size: %" PRIu64 " -> \"%s\" (%s) OK\n", + bytes, actual, actual_human_flag ? "true" : "false"); + fflush (stdout); + } + else { + fprintf (stderr, + "test-human-size: error: test case %" PRIu64 + "expected \"%s\" (%s) " + "but returned \"%s\" (%s)\n", + bytes, + expected, expected_human_flag ? "true" : "false", + actual, actual_human_flag ? "true" : "false"); + errors++; + } +} + +int +main (int argc, char *argv[]) +{ + test (0, "0", false); + test (1, "1", false); + test (512, "512", false); + test (1023, "1023", false); + test (1024, "1K", true); + test (1025, "1025", false); + test (2047, "2047", false); + test (2048, "2K", true); + test (3 * 1024, "3K", true); + + test (1023 * 1024, "1023K", true); + test (1048575, "1048575", false); + test (1048576, "1M", true); + test (1048577, "1048577", false); + + test (UINT64_C(1073741824), "1G", true); + + test (UINT64_C(1099511627776), "1T", true); + test (UINT64_C(1099511627777), "1099511627777", false); + test (UINT64_C(1099511627776) + 1024, "1073741825K", true); + + test (UINT64_C(1125899906842624), "1P", true); + + test ((uint64_t)INT64_MAX+1, "8E", true); + test (UINT64_MAX-1023, "18014398509481983K", true); + test (UINT64_MAX, "18446744073709551615", false); + + exit (errors == 0 ? EXIT_SUCCESS : EXIT_FAILURE); +} -- 2.32.0
For example: $ nbdkit null 1G --run 'nbdinfo "$uri"' protocol: newstyle-fixed without TLS export="": export-size: 1073741824 (1G) If the value cannot be abbreviated then the output doesn't include the part in parentheses: $ nbdkit null 1023 --run 'nbdinfo "$uri"' protocol: newstyle-fixed without TLS export="": export-size: 1023 The human-readable output changes, but callers shouldn't be parsing it (use --size and/or --json instead). --- info/nbdinfo.pod | 9 +++++---- info/show.c | 13 +++++++++++-- 2 files changed, 16 insertions(+), 6 deletions(-) diff --git a/info/nbdinfo.pod b/info/nbdinfo.pod index fbd0ef20..f64fe213 100644 --- a/info/nbdinfo.pod +++ b/info/nbdinfo.pod @@ -29,7 +29,7 @@ L<https://github.com/NetworkBlockDevice/nbd/blob/master/doc/uri.md>): $ nbdinfo nbd://localhost protocol: newstyle-fixed without TLS export="": - export-size: 1048576 + export-size: 1048576 (1M) content: data uri: nbd://localhost:10809/ is_rotational: false @@ -78,7 +78,8 @@ the I<--json> parameter: "block_size_minimum": 1, "block_size_preferred": 4096, "block_size_maximum": 33554432, - "export-size": 2125119488 + "export-size": 2125119488, + "export-size-str": "2075312K" } ] } @@ -222,11 +223,11 @@ For example: $ nbdkit file dir=. --run 'nbdinfo --list "$uri"' protocol: newstyle-fixed without TLS export="Fedora-Workstation-Live-x86_64-29-1.2.iso": - export-size: 1931476992 + export-size: 1931476992 (1842M) uri: nbd://localhost:10809/Fedora-Workstation-Live-x86_64-29-1.2.iso [...] export="debian-10.4.0-amd64-DVD-1.iso": - export-size: 3955556352 + export-size: 3955556352 (3862848K) uri: nbd://localhost:10809/debian-10.4.0-amd64-DVD-1.iso [...] diff --git a/info/show.c b/info/show.c index 893178cb..ff241a83 100644 --- a/info/show.c +++ b/info/show.c @@ -28,6 +28,7 @@ #include <libnbd.h> +#include "human-size.h" #include "vector.h" #include "nbdinfo.h" @@ -45,6 +46,8 @@ show_one_export (struct nbd_handle *nbd, const char *desc, bool first, bool last) { int64_t i, size; + char size_str[HUMAN_SIZE_LONGEST]; + bool human_size_flag; char *export_name = NULL; char *export_desc = NULL; char *content = NULL; @@ -75,6 +78,8 @@ show_one_export (struct nbd_handle *nbd, const char *desc, exit (EXIT_FAILURE); } + human_size (size_str, size, &human_size_flag); + uri = nbd_get_uri (nbd); /* Prefer the server's version of the name, if available */ @@ -116,7 +121,10 @@ show_one_export (struct nbd_handle *nbd, const char *desc, fprintf (fp, ":\n"); if (desc && *desc) fprintf (fp, "\tdescription: %s\n", desc); - fprintf (fp, "\texport-size: %" PRIi64 "\n", size); + if (human_size_flag) + fprintf (fp, "\texport-size: %" PRIi64 " (%s)\n", size, size_str); + else + fprintf (fp, "\texport-size: %" PRIi64 "\n", size); if (content) fprintf (fp, "\tcontent: %s\n", content); if (uri) @@ -239,7 +247,8 @@ show_one_export (struct nbd_handle *nbd, const char *desc, block_maximum); /* Put this one at the end because of the stupid comma thing in JSON. */ - fprintf (fp, "\t\"export-size\": %" PRIi64 "\n", size); + fprintf (fp, "\t\"export-size\": %" PRIi64 ",\n", size); + fprintf (fp, "\t\"export-size-str\": \"%s\"\n", size_str); if (last) fprintf (fp, "\t} ]\n"); -- 2.32.0
On Sat, Sep 18, 2021 at 03:38:52PM +0100, Richard W.M. Jones wrote:
For example 1024 is returned as "1K".
This does not attempt to handle decimals or SI units. If the number isn't some multiple of a power of 1024 then it is returned as bytes (a flag is available to indicate this).
I looked at both the gnulib and qemu versions of this function. The gnulib version is not under a license which is compatible with libnbd and is also really complicated, although it does handle fractions and SI units. The qemu version is essentially just frexp + sprintf and doesn't attempt to convert to the human-readable version reversibly. --- .gitignore | 1 + common/utils/Makefile.am | 10 +++- common/utils/human-size.c | 54 +++++++++++++++++++++ common/utils/human-size.h | 49 +++++++++++++++++++ common/utils/test-human-size.c | 89 ++++++++++++++++++++++++++++++++++ 5 files changed, 201 insertions(+), 2 deletions(-)
diff --git a/.gitignore b/.gitignore index 2aa1fd99..5fc59677 100644 --- a/.gitignore +++ b/.gitignore @@ -31,6 +31,7 @@ Makefile.in /bash/nbdcopy /bash/nbdfuse /bash/nbdinfo +/common/utils/test-human-size /common/utils/test-vector /compile /config.cache diff --git a/common/utils/Makefile.am b/common/utils/Makefile.am index 1ca4a370..b273ada1 100644 --- a/common/utils/Makefile.am +++ b/common/utils/Makefile.am @@ -34,6 +34,8 @@ include $(top_srcdir)/common-rules.mk noinst_LTLIBRARIES = libutils.la
libutils_la_SOURCES = \ + human-size.c \ + human-size.h \ vector.c \ vector.h \ version.c \ @@ -50,8 +52,12 @@ libutils_la_LIBADD = \
# Unit tests.
-TESTS = test-vector -check_PROGRAMS = test-vector +TESTS = test-human-size test-vector +check_PROGRAMS = test-human-size test-vector + +test_human_size_SOURCES = test-human-size.c human-size.c human-size.h +test_human_size_CPPFLAGS = -I$(srcdir) +test_human_size_CFLAGS = $(WARNINGS_CFLAGS)
test_vector_SOURCES = test-vector.c vector.c vector.h test_vector_CPPFLAGS = -I$(srcdir) diff --git a/common/utils/human-size.c b/common/utils/human-size.c new file mode 100644 index 00000000..5cff722b --- /dev/null +++ b/common/utils/human-size.c @@ -0,0 +1,54 @@ +/* nbd client library in userspace + * Copyright (C) 2020-2021 Red Hat Inc. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include <config.h> + +#include <stdio.h> +#include <stdlib.h> +#include <stdbool.h> +#include <stdint.h> +#include <inttypes.h> + +#include "human-size.h" + +char * +human_size (char *buf, uint64_t bytes, bool *human) +{ + static const char *ext[] = { "E", "P", "T", "G", "M", "K", "" }; + size_t i; + + if (buf == NULL) { + buf = malloc (HUMAN_SIZE_LONGEST); + if (buf == NULL) + return NULL; + } + + /* Set to false if we're not going to add a human-readable extension. */ + if (human) + *human = bytes != 0 && (bytes & 1023) == 0; +
It would be more readable (with some de-duplication even) to remove this part ^^ ...
+ /* Work out which extension to use, if any. */ + for (i = 6; i >= 0; --i) { + if (bytes == 0 || (bytes & 1023) != 0) + break; + bytes /= 1024; + } +
And just set here: *human = i == 6; Even better make it more future-proof and use (sizeof(ext) / sizeof(*ext)) instead of just "6". Disclaimer: Written by hand, not tested.
+ snprintf (buf, HUMAN_SIZE_LONGEST-1, "%" PRIu64 "%s", bytes, ext[i]);
The '-1' seems unnecessary here as snprintf() counts the zero terminator as part the supplied size and will not overrun @size-sized buffers. But even without the modifications both patches look fine. Martin
On Mon, Sep 20, 2021 at 12:50:47PM +0200, Martin Kletzander wrote:
And just set here:
*human = i == 6;
Even better make it more future-proof and use (sizeof(ext) / sizeof(*ext)) instead of just "6". Disclaimer: Written by hand, not tested.
In my v2 I will change it to: + /* Set to flag to true if we're going to add a human-readable extension. */ + if (human) + *human = ext[i][0] != '\0';
+ snprintf (buf, HUMAN_SIZE_LONGEST-1, "%" PRIu64 "%s", bytes, ext[i]);
The '-1' seems unnecessary here as snprintf() counts the zero terminator as part the supplied size and will not overrun @size-sized buffers.
Oops, that's left over from when I thought I was going to save one byte for adding the extension. Will fix in v2. Thanks, Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW
participants (2)
-
Martin Kletzander -
Richard W.M. Jones