Re: [Libguestfs] Extending the nbdkit python plugin
On 11/21/19 7:57 AM, Richard W.M. Jones wrote:
We have an nbdkit plugin that lets you write NBD servers in Python.
An example of an existing Python plugin is here:
https://github.com/libguestfs/nbdkit/blob/master/plugins/python/example.p...
This morning I tried to modify the plugin to use the newer nbdkit API
(version 2). One of the things that would change would be passing
flags parameters to some functions, eg:
def pwrite (h, buf, offset):
might become one of these possibilities (where flags is a bitmask):
def pwrite (h, buf, offset, flags):
def pwrite (h, buf, offset, flags=0):
The problem is if we did this it would break all existing Python
plugins. While we don't guarantee the nbdkit API for non-C languages,
we do nevertheless have Python plugins that we care about such as the
rhv-upload-plugin used by virt-v2v. Having a flag day which breaks
all existing plugins is very awkward.
At one point, I tried using Python introspection to determine which
signature the user's function has, to determine if the user wrote
against the v1 protocol. It may also be possible to use python's keyword
argument parsing to pass all optional arguments like flags by key=value
pairs (allowing future addition of additional flags). I'll have to
revive my patches.
I tried to simply pass the extra arguments from the C code to the
Python code, and existing plugins break with:
nbdkit: python[1]: error: ./test.py: pwrite: error: pwrite() takes 3 positional arguments
but 4 were given
One possibility is that we could introspect the plugin to find out how
many parameters it takes. This is possible, but very difficult from
C. (See https://stackoverflow.com/a/41188411 for how to do it from
Python).
Another possibility is we could encourage existing Python plugins to
add **kwargs to all functions where we might plausibly add extra
parameters in future, ie. the above would become:
def pwrite (h, buf, offset, **kwargs):
This still requires all Python plugins to change, but at least they
would remain backwards compatible with old and new nbdkit.
Yes, that's along the lines of the work I had attempted previously. But
I attempted it back when we were using Python2, so I'd have to rework it
for Python3.
However I
couldn't actually work out how to make this work because:
>>> def test(a, **kwargs):
... pass
...
>>> test(1)
>>> test(1,2)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: test() takes 1 positional argument but 2 were given
Yet another possibility is the Python plugin itself should declare
which version of the API it wants, and the C code can then pass the
correct parameters. (This is in fact how nbdkit C plugins work).
This pushes a bunch of work into the C code, but I guess we can deal
with that.
So really this is a question directed at Python experts. What's the
right way to go about this?
I'll try and revive my patches; I may be able to save you some time
based on what I already have in my tree.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org