On Sat, Jun 26, 2021 at 05:19:47PM +0100, Richard W.M. Jones wrote:
This is similar to "qemu-img measure". Some examples:
$ nbdkit -r file fedora-33.img --run 'nbdinfo --map --totals $uri' 1226113024 19.0 0 data 5216337920 81.0 3 hole,zero
Is it worth a human-legible header line? Or maybe a % character in the second column?
$ nbdkit -r file fedora-33.img --run 'nbdinfo --map --totals --json $uri | jq' [ { "size": 1226113024, "percent": 19.0318, "type": 0, "description": "data" }, { "size": 5216337920, "percent": 80.9682, "type": 3, "description": "hole,zero" } ]
Output looks nice.
$ nbdkit sparse-random 6G --run 'nbdinfo --map --totals $uri' 941551616 14.6 0 data 5500899328 85.4 3 hole,zero ---
...
+++ b/info/info-map-totals-json.sh + +test $( jq -r '.[0].size' < $out ) -eq 65536 +test $( jq -r '.[0].percent' < $out ) = "6.25" +test $( jq -r '.[0].type' < $out ) -eq 0 +test $( jq -r '.[0].description' < $out ) = "data" + +test $( jq -r '.[1].size' < $out ) -eq 983040 +test $( jq -r '.[1].percent' < $out ) = "93.75" +test $( jq -r '.[1].type' < $out ) -eq 3 +test $( jq -r '.[1].description' < $out ) = "hole,zero" ... diff --git a/info/info-map-totals.sh b/info/info-map-totals.sh new file mode 100755 index 0000000..12c1263 --- /dev/null +++ b/info/info-map-totals.sh @@ -0,0 +1,43 @@
+ +cat $out + +if [ "$(tr -s ' ' < $out)" != " 65536 6.2 0 data + 983040 93.8 3 hole,zero" ]; then
So this is the human-readable counterpart of info-map-totals-json.sh. I'm trying to figure out why 6.25 rounded down and 93.75 rounded up. I guess it's related to round-to-even? But it's also fine if the rounded values don't sum back up to 100%; I don't know if it's worth a documentation disclaimer.
+++ b/info/map.c
+/* --map --totals suboption */ +static void +print_totals (uint32_vector *entries, int64_t size) +{ + uint32_t type; + bool comma = false; + + if (json_output) fprintf (fp, "[\n"); + + /* In the outer loop assume we have already printed all entries with + * entry type < type. Count all instances of type and at the same + * time find the next type that exists > type. + */ + type = 0; + for (;;) { + uint64_t next_type = (uint64_t)UINT32_MAX + 1; + uint64_t c = 0; + size_t i; + + for (i = 0; i < entries->size; i += 2) { + uint32_t t = entries->ptr[i+1]; + + if (t == type) + c += entries->ptr[i]; + else if (type < t && t < next_type) + next_type = t; + }
I see what you mean about a malicious server being able to force us to do a LOT of work, but I think this is fine for a first cut (we can worry about rewriting it to do a single pass utilizing a hash table if it proves to be noticeable in practice).
+++ b/info/nbdinfo.pod @@ -8,7 +8,7 @@ nbdinfo - display information and metadata about NBD servers and exports
nbdinfo --size [--json] NBD-URI
- nbdinfo --map [--json] NBD-URI + nbdinfo --map [--totals] [--json] NBD-URI
nbdinfo -L|--list [--json] NBD-URI
@@ -119,6 +119,35 @@ other maps too: For more information on NBD maps, see I<Metadata querying> in the NBD protocol.
+=head2 Map totals + +Using S<I<--map --totals>> performs the same operation as I<--map> but +displays a summary of the total size of each type of allocation, in +bytes and as a percentage (of the virtual size of the export). This +is useful for estimating how much real storage is used on the server, +or might be required when copying a sparse image with L<nbdcopy(1)>. + +In the example below, half (50.0%) of the disk is allocated data and +half is unallocated: + + $ nbdinfo --map --totals nbd://localhost/ + 1048576 50.0 0 data + 1048576 50.0 3 hole,zero + +The fields are: total size in bytes, percentage of the virtual size, +type, description (optional). + +You can also get the same information in parseable form using I<--json>: + + $ nbdinfo --map --totals --json nbd://localhost/ + [{ "size": 1048576, "percent": 50, + "type": 0, "description": "data" }, + { "size": 1048576, "percent": 50, + "type": 3, "description": "hole,zero" }] + +As with the I<--map> option, by default this shows the +C<"base:allocation"> map, but you can show the summary for other maps.
I'd still like to someday have a mode where we can read all advertised maps at once, rather than making the command-line user have to manually invoke nbdinfo once per map spelled by the user, but that can still go on top of this. Overall, looks good to me! -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org