On Fri, Mar 13, 2020 at 09:41:37PM +0000, JP Zhang wrote:

Hi Richard,

Thanks for the great work you did on libguestfs.

I try to call libguestfs lib API directly in C to "add drive" by using a Azure blob storage access SAS URI, which has shared access token/secret on the URL.

The URL likes like this: 'https://<account>.blob.core.windows.net/<container>/<VMDiskImage>?sv=2017-04-17&sr=b&si=<guid>&sig=<secret>' I try to use this API:

int guestfs_add_drive_opts (guestfs_h *g, const char *filename, ...);

My code looks like this: char const *servers[] = { (char const *)"<account>.blob.core.windows.net", NULL }; int addDriveRet = guestfs_add_drive_opts (g, "/<container>/<VMDiskImage>?sv=2017-04-17&sr=b&si=<guid>&sig=<secret>", GUESTFS_ADD_DRIVE_OPTS_FORMAT, "raw", GUESTFS_ADD_DRIVE_OPTS_PROTOCOL, "https", GUESTFS_ADD_DRIVE_OPTS_SERVER, servers, GUESTFS_ADD_DRIVE_OPTS_READONLY, 1, -1);

Since you say you got it to work with guestfish, one trick is to run ‘guestfish -x’ (or the ‘trace true’ command, or set LIBGUESTFS_TRACE=1) and it will actually print the full API call you should use. I believe the problem may be to do with URI %-quoting of the filename (2nd) parameter.

I read the documentation and samples on libguestfs.org in depth. I could not find a good sample for "https" with SAS secret on URL. I tried many different ways to break up the URL into "server", "protocol", and secret. It looks like "secret" option is only for "rbd". From the debug log/printouts from libguestfs, it seemed to reassemble the parts of original URL back to a URL that matched the original one. But I got http 403 error( auth error). I am very sure the URL is correct because I can download the disk image using Curl. The libguestfs debug log printout displayed the URL with ? being turned into %3F and % into %25. I am not sure whether libguestfs URL-encoded printout was just for printout or whether it was the actual URL string being used to make the https server call. If it was, then the secret has been changed due to URL encoding again. This is just a guess. Maybe not related at all.

BTW, I can run libguestfs-test-tool successfully. and I have another program in Python that call through /run.sh -> GuestFish -> libguestfs lib, it works with the same original SAS URL.

So you can contact the SAS URL through this? If so I would try the trace trick above to see the actual protocol call being made. If that doesn't work, another way to do it would be to forward the connection through nbdkit, ie: rm -f /tmp/socket nbdkit -f -U /tmp/socket curl 'https://...' & and connect to the the NBD socket using the add_drive_opts API as documented in the manual. (in guestfish you'd use: guestfish -a 'nbd://?socket=/tmp/socket') nbdkit can also be run captive from your program, see http://libguestfs.org/nbdkit-captive.1.html

also, on my dev machine, I can do "guestfs_add_drive" on the local disk image successfully and mount the device, and list files after I downloaded the same disk image to my local machine.

guestfs_add_drive(g, "/pathto/vmdiskimage");

Could you share how I should fill in various params in "guest_add_drive_opts()? like "PROTOCOL", "SERVER", "filename"( the 2nd param), or USERNAME ( not seem to apply here, I also tried with putting "secret" in SECRET, "sig" in USERNAME, the rebuilt URL from libguestfs looks missing these parts in final URL, it probably put these in HTTP header.)

I also try to call guestfs_add_drive() with the original URL below. it didn't work either.

guestfs_add_drive(g, "https://<account>.blob.core.windows.net/<container>/<VMDiskImage>?sv=2017-04-17&sr=b&si=<guid>&sig=<secret>");

Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW