Hi,

Following up on my last post on this topic (since there seemed to be at least some interest): after evaluating the setup for a few weeks, I have decided that this is not as good a solution as I had hoped.

In particular, the expected benefits of splitting data between a special vdev backed by a bucket with small object size and a normal vdev backed by a bucket with much larger object size did not materialize.

More details (including histograms) are available at https://www.rath.org/zfs-on-nbd-my-verdict.html

Best,
-Nikolaus