On Thu, May 28, 2020 at 12:24:22AM +0000, Eric Wheeler wrote: > On Mon, 15 Oct 2018, Nir Soffer wrote: > > On Sat, Oct 13, 2018 at 9:45 PM Eric Wheeler <nbd(a)lists.ewheeler.net&gt; wrote: > > Hello all, > > > > It might be neat to attach ISOs to KVM guests via websockets.  Basically > > the  browser would be the NBD "server" and an NBD client would run on the > > hypervisor, then use `virsh change-media vm1 hdc --insert /dev/nbd0` could > > use an ISO from my desk to boot from. > > > > Here's an HTML5 open file example: > > https://stackoverflow.com/questions/3582671/how-to-open-a-local-disk-file... > > > > and the NBD protocol looks simple enough to implement in javascript: > > https://stackoverflow.com/questions/17295140/where-is-the-network-block-d... > > > > What do you think? Does anyone have an interest in doing this? > > > > > > HTML File API is very limited: > > - you cannot access any file except file provided by the user interactively > > - no support for sparseness or underlying disk block size > > > > So it will be a pretty bad backend for NBD server. > > > > What are you trying to do? > > Hi Nir and Eric, > > I hope you are well! > > (I'm resurecting this old thread, not sure how I missed the replies.) > > We are interested in attaching a local ISO to a remote VM over http (maybe > .qcow's, but ISO is the primary interest). So this bit is simple, and something we do regularly. Assuming the "remote VM" is using qemu as a hypervisor, then qemu has a curl driver integrated so you can mount remote http:// or https:// resources as disks on the VM directly. eg: $ qemu-system-x86_64 -m 2048 -cdrom https://download.fedoraproject.org/pub/fedora/linux/releases/32/Server/x8... (A more realistic example would use libvirt, see the https protocol in: https://libvirt.org/formatdomain.html#elementsDisks )

You can also bounce the connection through nbdkit-curl-plugin which gives you a bit more flexibility, eg the ability to transparently uncompress the remote ISO. You would be using the NBD client in qemu to connect (eg. over a Unix domain socket) to a local nbdkit, and nbdkit's curl plugin to connect to the remote ISO. More info and a worked example in these pages: http://libguestfs.org/nbdkit-curl-plugin.1.html https://rwmj.wordpress.com/2018/11/23/nbdkit-xz-curl/

> This is common for remote KVM (iDRAC/iLO/iKVM/CIMC), so wondering about > an http front-end for qemu to do the same. > Combining that with a spice JS client or noVNC for VM console access would > be neat. I'm a bit confused by what you mean by "http front-end for qemu".

> I also like Eric Blake's idea of direct NBD client integration with qemu > instead of using /dev/nbd0. You shouldn't need to use a loop device for this scenario, assuming I've understood what you want.

Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v

Going back to the original email from 2018: > It might be neat to attach ISOs to KVM guests via websockets.  Basically > the  browser would be the NBD "server" and an NBD client would run on the > hypervisor, then use `virsh change-media vm1 hdc --insert /dev/nbd0` could > use an ISO from my desk to boot from. > > Here's an HTML5 open file example: > https://stackoverflow.com/questions/3582671/how-to-open-a-local-disk-file... > > and the NBD protocol looks simple enough to implement in javascript: > https://stackoverflow.com/questions/17295140/where-is-the-network-block-d... So I think what you mean here is that in a browser you'd open a local (eg) ISO, and then that ISO could be shared with a remote VM. The browser runs a Javascript NBD server. The remote VM is qemu. Between the two is a WebSocket. I've seen this being done with an HP blade server of some kind and IIRC the client side used a Java applet. No idea what the protocol was but likely something proprietary. It was nevertheless a useful feature, eg to boot the server from an install CD that you have locally. I guess the problem is two-fold: (1) You need to write an NBD server in Javascript. Not especially difficult, particularly if you avoid any complicated features, and I guess you only need read support.

(2) You need to persuade qemu's NBD client to read from a WebSocket. I didn't really know anything about WebSockets until today but it seems as if they are a full-duplex protocol layered on top of HTTP [a]. Is there a WebSocket proxy that turns WS into plain TCP (a bit like stunnel)? Google suggests [b]. [a] https://en.wikipedia.org/wiki/WebSocket#Protocol_handshake [b] https://github.com/novnc/websockify

... > When qemu is running headless using a VNC or Spice display we can access > the display of https+websockets using things like noVNC---which is out of > scope to this converstation---but I'm just saying that such an existing > web front-end for the display could be extended to have an "Insert CDROM" > button and use the javascript file IO for the user to reference a local > file and pass it to qemu over nbd, perhaps via nbdkit. I'm not sure how nbdkit would be involved, unless it was compiled to WASM or something like that. But the plan above sounds feasible, albeit a chunk of work. Rich.

>> (2) You need to persuade qemu's NBD client to read from a WebSocket. >> I didn't really know anything about WebSockets until today but it >> seems as if they are a full-duplex protocol layered on top of HTTP [a]. >> Is there a WebSocket proxy that turns WS into plain TCP (a bit like >> stunnel)? Google suggests [b]. >> >> [a] https://en.wikipedia.org/wiki/WebSocket#Protocol_handshake >> [b] https://github.com/novnc/websockify > > qemu already knows how to connect as a client to websockets; Dan Berrange > knows more about that setup. I suspect it would not be too difficult to > teach the qemu NBD client code to use a WebSocket instead of a Unix or TCP > socket as its data source. Actually the inverse. The QIOChannelWebsocket impl is only the server side of the problem, as used by QEMU's VNC server. We've never implemented the client side. There is nothing especially stopping us doing that - just needs someone motivated with time to work on it.

On Fri, May 29, 2020 at 08:58:06AM -0500, Eric Blake wrote: > On 5/29/20 8:50 AM, Daniel P. Berrangé wrote: > > >>>(2) You need to persuade qemu's NBD client to read from a WebSocket. > >>>I didn't really know anything about WebSockets until today but it > >>>seems as if they are a full-duplex protocol layered on top of HTTP [a]. > >>>Is there a WebSocket proxy that turns WS into plain TCP (a bit like > >>>stunnel)? Google suggests [b]. > >>> > >>>[a] https://en.wikipedia.org/wiki/WebSocket#Protocol_handshake > >>>[b] https://github.com/novnc/websockify > >> > >>qemu already knows how to connect as a client to websockets; Dan Berrange > >>knows more about that setup. I suspect it would not be too difficult to > >>teach the qemu NBD client code to use a WebSocket instead of a Unix or TCP > >>socket as its data source. > > > >Actually the inverse. The QIOChannelWebsocket impl is only the server > >side of the problem, as used by QEMU's VNC server. We've never implemented > >the client side. There is nothing especially stopping us doing that - just > >needs someone motivated with time to work on it. > > In the meantime, you may still be able to set up something like: > > local machine: > iso -> NBD server -> Unix socket -> websockify -> WebSocket I guess the idea is to have a zero-install solution for the browser. As I said in the email earlier this is very common for IPMI-type remote access to blade servers and in my experience is implemented using a Java applet and a proprietary protocol terminated at the BMC (which then emulates a virtual CDROM to the server). There are some HP blade servers on Red Hat's internal Beaker instance where you can play with this. For qemu we wouldn't need to invent a new protocol when NBD is available and already implemented (albeit not yet on top of WebSockets). The NBD server must run inside the browser and therefore be either written from scratch in Javascript, or an existing server cross-compiled to WASM (if that is possible - I don't really know).

> remote machine: > WebSocket -> websockify -> Unix socket -> qemu NBD client > > Adding websocket client support into qemu would reduce the length of > the chain slightly (for less data copying) by getting rid of a > websockify proxy middleman, but would not necessarily improve > performance (it's hard to say where the latency bottlenecks will be > in the chain). Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org

On Fri, May 29, 2020 at 09:08:29PM +0000, Eric Wheeler wrote: > On Fri, 29 May 2020, Richard W.M. Jones wrote: > > On Fri, May 29, 2020 at 08:58:06AM -0500, Eric Blake wrote: > > > On 5/29/20 8:50 AM, Daniel P. Berrang�© wrote: > > > > > > >>>(2) You need to persuade qemu's NBD client to read from a WebSocket. > > > >>>I didn't really know anything about WebSockets until today but it > > > >>>seems as if they are a full-duplex protocol layered on top of HTTP [a]. > > > >>>Is there a WebSocket proxy that turns WS into plain TCP (a bit like > > > >>>stunnel)? Google suggests [b]. > > > >>> > > > >>>[a] https://en.wikipedia.org/wiki/WebSocket#Protocol_handshake > > > >>>[b] https://github.com/novnc/websockify > > > >> > > > >>qemu already knows how to connect as a client to websockets; Dan Berrange > > > >>knows more about that setup. I suspect it would not be too difficult to > > > >>teach the qemu NBD client code to use a WebSocket instead of a Unix or TCP > > > >>socket as its data source. > > > > > > > >Actually the inverse. The QIOChannelWebsocket impl is only the server > > > >side of the problem, as used by QEMU's VNC server. We've never implemented > > > >the client side. There is nothing especially stopping us doing that - just > > > >needs someone motivated with time to work on it. > > > > > > In the meantime, you may still be able to set up something like: > > > > > > local machine: > > > iso -> NBD server -> Unix socket -> websockify -> WebSocket > > > > I guess the idea is to have a zero-install solution for the browser. > > As I said in the email earlier this is very common for IPMI-type > > remote access to blade servers and in my experience is implemented > > using a Java applet and a proprietary protocol terminated at the BMC > > (which then emulates a virtual CDROM to the server). There are some > > HP blade servers on Red Hat's internal Beaker instance where you can > > play with this. For qemu we wouldn't need to invent a new protocol > > when NBD is available and already implemented (albeit not yet on top > > of WebSockets). > > > > The NBD server must run inside the browser and therefore be either > > written from scratch in Javascript, or an existing server > > cross-compiled to WASM (if that is possible - I don't really know). > > Interesting idea about WASM. I'll see if I can build one of the simple > nbd servers that are around. Not sure how to link it to the JS file IO, > however. After reading a bit about compiling to WebSockets it sounds like you can cross-compile a C program, but there's no library support at all. IOW to port an existing server you'd have to implement enough of POSIX to make it work. nbdkit has a liberal license deliberately to make it possible to chop it up and incorporate it into completely forked codebases (nbdkit is a plot to make NBD more popular). But since NBD is pretty simple, a fresh Javascript server might be easier, especially if you stick to only implementing reads.

Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top