On Sat, Jun 26, 2021 at 05:19:47PM +0100, Richard W.M. Jones wrote:
This is similar to "qemu-img measure". Some examples:
$ nbdkit -r file fedora-33.img --run 'nbdinfo --map --totals $uri'
1226113024 19.0 0 data
5216337920 81.0 3 hole,zero
Is it worth a human-legible header line? Or maybe a % character in
the second column?
$ nbdkit -r file fedora-33.img --run 'nbdinfo --map --totals --json $uri | jq'
[
{
"size": 1226113024,
"percent": 19.0318,
"type": 0,
"description": "data"
},
{
"size": 5216337920,
"percent": 80.9682,
"type": 3,
"description": "hole,zero"
}
]
Output looks nice.
$ nbdkit sparse-random 6G --run 'nbdinfo --map --totals $uri'
941551616 14.6 0 data
5500899328 85.4 3 hole,zero
---
...
+++ b/info/info-map-totals-json.sh
+
+test $( jq -r '.[0].size' < $out ) -eq 65536
+test $( jq -r '.[0].percent' < $out ) = "6.25"
+test $( jq -r '.[0].type' < $out ) -eq 0
+test $( jq -r '.[0].description' < $out ) = "data"
+
+test $( jq -r '.[1].size' < $out ) -eq 983040
+test $( jq -r '.[1].percent' < $out ) = "93.75"
+test $( jq -r '.[1].type' < $out ) -eq 3
+test $( jq -r '.[1].description' < $out ) = "hole,zero"
...
diff --git a/info/info-map-totals.sh b/info/info-map-totals.sh
new file mode 100755
index 0000000..12c1263
--- /dev/null
+++ b/info/info-map-totals.sh
@@ -0,0 +1,43 @@
+
+cat $out
+
+if [ "$(tr -s ' ' < $out)" != " 65536 6.2 0 data
+ 983040 93.8 3 hole,zero" ]; then
So this is the human-readable counterpart of info-map-totals-json.sh.
I'm trying to figure out why 6.25 rounded down and 93.75 rounded up.
I guess it's related to round-to-even? But it's also fine if the
rounded values don't sum back up to 100%; I don't know if it's worth a
documentation disclaimer.
+++ b/info/map.c
+/* --map --totals suboption */
+static void
+print_totals (uint32_vector *entries, int64_t size)
+{
+ uint32_t type;
+ bool comma = false;
+
+ if (json_output) fprintf (fp, "[\n");
+
+ /* In the outer loop assume we have already printed all entries with
+ * entry type < type. Count all instances of type and at the same
+ * time find the next type that exists > type.
+ */
+ type = 0;
+ for (;;) {
+ uint64_t next_type = (uint64_t)UINT32_MAX + 1;
+ uint64_t c = 0;
+ size_t i;
+
+ for (i = 0; i < entries->size; i += 2) {
+ uint32_t t = entries->ptr[i+1];
+
+ if (t == type)
+ c += entries->ptr[i];
+ else if (type < t && t < next_type)
+ next_type = t;
+ }
I see what you mean about a malicious server being able to force us to
do a LOT of work, but I think this is fine for a first cut (we can
worry about rewriting it to do a single pass utilizing a hash table if
it proves to be noticeable in practice).
+++ b/info/nbdinfo.pod
@@ -8,7 +8,7 @@ nbdinfo - display information and metadata about NBD servers and exports
nbdinfo --size [--json] NBD-URI
- nbdinfo --map [--json] NBD-URI
+ nbdinfo --map [--totals] [--json] NBD-URI
nbdinfo -L|--list [--json] NBD-URI
@@ -119,6 +119,35 @@ other maps too:
For more information on NBD maps, see I<Metadata querying> in the NBD
protocol.
+=head2 Map totals
+
+Using S<I<--map --totals>> performs the same operation as I<--map>
but
+displays a summary of the total size of each type of allocation, in
+bytes and as a percentage (of the virtual size of the export). This
+is useful for estimating how much real storage is used on the server,
+or might be required when copying a sparse image with L<nbdcopy(1)>.
+
+In the example below, half (50.0%) of the disk is allocated data and
+half is unallocated:
+
+ $ nbdinfo --map --totals nbd://localhost/
+ 1048576 50.0 0 data
+ 1048576 50.0 3 hole,zero
+
+The fields are: total size in bytes, percentage of the virtual size,
+type, description (optional).
+
+You can also get the same information in parseable form using I<--json>:
+
+ $ nbdinfo --map --totals --json nbd://localhost/
+ [{ "size": 1048576, "percent": 50,
+ "type": 0, "description": "data" },
+ { "size": 1048576, "percent": 50,
+ "type": 3, "description": "hole,zero" }]
+
+As with the I<--map> option, by default this shows the
+C<"base:allocation"> map, but you can show the summary for other maps.
I'd still like to someday have a mode where we can read all advertised
maps at once, rather than making the command-line user have to
manually invoke nbdinfo once per map spelled by the user, but that can
still go on top of this.
Overall, looks good to me!
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization:
qemu.org |
libvirt.org