On Thu, Nov 22, 2018 at 10:00 AM Nikolay Ivanets <stenavin@gmail.com> wrote:
Let me share my thoughts.

Thanks! :)
 
 

> However, as one can see in the video, it takes a pretty long time to inspect the disk (in order to find the operating system) and to mount the file systems.

Do you really need to inspect disk image BEFORE mounting file
system(s)? You can do inspection on demand if you want/need to provide
some metadata about installed OS etc. And definitely inspection might
take some time. Pay attention that it is better to provide all images
to libguestfs if original installation was spread across multiple
disks. And while it is not strongly required but is good idea to
provide all disk images in the same order.

Well, I don't need any information about the operating system itself but ideally, when browsing the disk image the user sees the file system(s) as if the user would have connected (e.g., via ssh) to the guest. So let's say that I browse a disk image with two partitions, the root that is mounted to "/" and another one that is mounted to "/boot". I think it makes sense to show the content of the disk in a way that the second partition is mounted to "/boot", and it seems that this mapping is only returned by inspect-os, right?
 

I think you can use list-filesystems, mount, ll and friends to
navigate file system on demand as user drills down disk image(s).
E.g.:

1. User: selects disk image, double-click. You: launch libguestfs
instance, call list-filesystems and display them.

Right, adding another level of file-systems is possible. The downside though is that I then stop seeing the directory-structure from the guest point of view, right?
Maybe it's not that bad if it significantely improves the responsiveness...
 
2. User: selects some file system, double-click. You: mount file
system, call ll (or friends) and display top-level file system
hierarchy.
3. User: selects directory, double-click. You: call ll (or friends) to
discover and display next level. And so on.
4. You can umount current file system when the user goes back to a
very first level (list of file systems).
5. And you can terminate current libguestfs instance when user goes
out of the current disk image on you UI. 

About terminating the libguestfs instance, note that muCommander caches the browsed files internally to improve responsiveness.
For example, imagine that the user has several open tabs - it makes sense for muCommander to keep their current location cached because of the likelihood the user will switch between those tabs.
So I would rather rely on the framework to decide when to release resources and not to control that within the plugin.


Again, pay attention that with LVM, Windows dynamic disks, MD arrays
you might need to provide all involved disk images to a guestfs to
make things to work as expected.

But on the other hand, I do want to enable the user to browse a single disk image even if it was attached to the VM along with few others.
What would be the risk in inspecting a single disk image in that case? shouldn't each disk image be a stand alone entity?
 

Another thing to keep in mind that original disk image might be
altered if you launched guestfs instance in RW mode even it seemed
you/user didn't make any modifications. That is because some file
systems will apply journal on mount.

Oh, that might explain some troubles I'm facing:
When implemented an extension for RAR files, I used the file modification time in order to check whether the file has changed and therefore I need to re-inspect the file.
But with libguestfs, I've noticed that even when not changing anything on the disk, it's modification time changes. This could explain why that happens (so currently I assume that the disk is not modified by other tool while being browsed by muCommander).
 

> So it takes me back to the filesystem_walk function as I read in [4] that it allows inspecting the metadata without mounting the disk partition(s). Unfortunately, I didn't manage to try that function due to the following error I get when trying to use it (tried on Fedora 28 & 29):
>
> Exception in thread "main" com.redhat.et.libguestfs.LibGuestFSException: internal_filesystem_walk: feature 'libtsk' is not available in this
> build of libguestfs.  Read 'AVAILABILITY' in the guestfs(3) man page for
> how to check for the availability of features.
>     at com.redhat.et.libguestfs.GuestFS._filesystem_walk(Native Method)
>     at com.redhat.et.libguestfs.GuestFS.filesystem_walk(GuestFS.java:6010)
>     at test.Main.main(Main.java:27)
>
> Is that right that the filesystem_walk doesn't require mounting the disk partition(s)? If that's true, can we publish a version that supports that in the Fedora repo?

Do you think filesystem_walk will help you? It might take even longer.
I think it will return you a list of ALL the files and directories.
And you have to build directory tree yourself to present it to a user
on your UI. But if you want to: just check libtsk feature availability
before using guestfs functionality (like filesystem_walk) which
depends on it.

Yeah, I didn't want to make the original post too long so I dropped the part that could explain why that's exactly what I'm doing at the moment (inspecting the whole directory structure) :)

Long story short, there are three type of files in mCcommander:
- Local files
- Protocol files, e.g., FTP, SMB.
- Archive files, e.g., ZIP, RAR, TAR.

Protocol files require the user to authenticate a remote place and then we browse the destination in a lazy way, i.e., as you wrote before - inspecting a directory when the user asks to browse it.
In contrast, archieve files were assumed local and easier to inspect and so their whole structure is retrieved when you "enter the file". muCommander then caches the whole tree structure and refreshes it only when needed.

Virtual disks don't really fit either of them. On the one hand, they cannot be considered protocol files as they reside within the disk. On the other hand, they should be inspected in a lazy way because they are typically quite big.

As I wrote in [1], some of the assumptions we took 10-15 years ago might be irrelevant anymore and need to change. One of them is probably to start inspecting archieve files in a lazy way as well but that requires massive changes in the framework and so I would not want to do it right now.
 

You will always depends on guestfs available on a host system if you
don't build and provide your own build alongside with muCommander. And
you never know how guestfs was built and what features are available.
So you should always check feature availability before using it.
Moreover a lot of functionality of libguestfs very depends on a host
kernel and available modules. My experience with libguestfs forces me
to build it myself. I also build my custom guestfs appliance to not
depend on a host kernel etc.

You can consider using mount_local but that is another story...

That's a very good point. I touched the deployment aspect also in [1] - by making it a plugin I can provide it only on linux and declare libguestfs as a dependency *of the plugin*. As the functionality I need from libguestfs is pretty basic, I really hope I could use the default build of libguestfs :)

 

--
  Mykola Ivanets