missing btrfs subvol support
Is btrfs subvol support failing just for me? Looks like nothing adds the required '@/' string. virt-ls uses the first variant of the command:
<rescue> mount -vo subvol=var/spool,ro /dev/sda2 /sysroot/ [ 113.852047] BTRFS info (device sda2): disk space caching is enabled [ 113.852869] BTRFS: has skinny extents mount: mount(2) failed: No such file or directory <rescue> mount -vo subvol=@var/spool,ro /dev/sda2 /sysroot/ [ 116.082642] BTRFS info (device sda2): disk space caching is enabled [ 116.083476] BTRFS: has skinny extents mount: mount(2) failed: No such file or directory <rescue> mount -vo subvol=@/var/spool,ro /dev/sda2 /sysroot/ [ 123.995343] BTRFS info (device sda2): disk space caching is enabled [ 123.996238] BTRFS: has skinny extents mount: /dev/sda2 mounted on /sysroot. <rescue> exit
fstab is: UUID=d538a81f-9778-424d-96cc-e48dd2d4a323 swap swap defaults 0 0 UUID=65b72f12-eccb-4cf2-a4f4-9c4bb462456f / btrfs defaults 0 0 UUID=65b72f12-eccb-4cf2-a4f4-9c4bb462456f /var/spool btrfs subvol=@/var/spool 0 0 Olaf
On Fri, Oct 10, 2014 at 01:47:01PM +0200, Olaf Hering wrote:
Is btrfs subvol support failing just for me? Looks like nothing adds the required '@/' string. virt-ls uses the first variant of the command:
<rescue> mount -vo subvol=var/spool,ro /dev/sda2 /sysroot/ [ 113.852047] BTRFS info (device sda2): disk space caching is enabled [ 113.852869] BTRFS: has skinny extents mount: mount(2) failed: No such file or directory <rescue> mount -vo subvol=@var/spool,ro /dev/sda2 /sysroot/ [ 116.082642] BTRFS info (device sda2): disk space caching is enabled [ 116.083476] BTRFS: has skinny extents mount: mount(2) failed: No such file or directory <rescue> mount -vo subvol=@/var/spool,ro /dev/sda2 /sysroot/ [ 123.995343] BTRFS info (device sda2): disk space caching is enabled [ 123.996238] BTRFS: has skinny extents mount: /dev/sda2 mounted on /sysroot. <rescue> exit
fstab is: UUID=d538a81f-9778-424d-96cc-e48dd2d4a323 swap swap defaults 0 0 UUID=65b72f12-eccb-4cf2-a4f4-9c4bb462456f / btrfs defaults 0 0 UUID=65b72f12-eccb-4cf2-a4f4-9c4bb462456f /var/spool btrfs subvol=@/var/spool 0 0
I wonder if this is an Augeas thing? We use Augeas to parse the /etc/fstab btrfs entries, see src/inspect-fs-unix.c: check_fstab: https://github.com/libguestfs/libguestfs/blob/master/src/inspect-fs-unix.c#L... And we then pass the subvol back to the mount command in daemon/mount.c (which doesn't alter it). https://github.com/libguestfs/libguestfs/blob/master/daemon/mount.c#L154 You could try: cd /tmp mkdir etc cat > etc/fstab <<EOF UUID=123 swap swap defaults 0 0 UUID=456 / btrfs defaults 0 0 UUID=789 /var/spool btrfs subvol=@/var/spool 0 0 EOF augtool -r /tmp In augtool: augtool> ls /files/etc/fstab/3/vfstype augtool> get /files/etc/fstab/3/vfstype /files/etc/fstab/3/vfstype = btrfs augtool> get /files/etc/fstab/3/spec /files/etc/fstab/3/spec = UUID=789 augtool> get /files/etc/fstab/3/file /files/etc/fstab/3/file = /var/spool augtool> get /files/etc/fstab/3/dump /files/etc/fstab/3/dump = 0 augtool> get /files/etc/fstab/3/opt/value /files/etc/fstab/3/opt/value = @/var/spool My guess is that the last one (opt/value) will be different for you. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v
On Fri, Oct 10, Richard W.M. Jones wrote:
My guess is that the last one (opt/value) will be different for you.
Thanks for the pointers. I will poke around. At least augtool on the host seems to behave correctly: olaf@bax:~ $ cd /dev/shm/$$ bash: cd: /dev/shm/3570: Datei oder Verzeichnis nicht gefunden olaf@bax:~ $ mkdir $_ olaf@bax:~ $ cd $_ olaf@bax:/dev/shm/3570 $ mkdir etc olaf@bax:/dev/shm/3570 $ cat > etc/fstab UUID=d538a81f-9778-424d-96cc-e48dd2d4a323 swap swap defaults 0 0 UUID=65b72f12-eccb-4cf2-a4f4-9c4bb462456f / btrfs defaults 0 0 UUID=65b72f12-eccb-4cf2-a4f4-9c4bb462456f /var/spool btrfs subvol=@/var/spool 0 0 olaf@bax:/dev/shm/3570 $ augtool -r $PWD augtool> ls /files/etc/fstab/3/vfstype augtool> get /files/etc/fstab/3/vfstype /files/etc/fstab/3/vfstype = btrfs augtool> get /files/etc/fstab/3/opt/value /files/etc/fstab/3/opt/value = @/var/spool augtool> I will poke around to see what I find. Olaf
On Friday, October 10, 2014 03:07:51 PM Olaf Hering wrote:
Thanks for the pointers. I will poke around. At least augtool on the host seems to behave correctly:
I don't think this is related to Augeas. Instead, I think it's the naming convention SUSE uses for the root btrfs subvolume (@). I encountered this same issue in virt-v2v and posted a question here: https://www.redhat.com/archives/libguestfs/2014-April/msg00071.html At the time, the problem only seemed to break v2v in SUSE environments, so I created a local patch to strip out the duplicate root in the v2v process. As other tools are running into this, we should probably deal with this in guestfs itself. -Mike
On Fri, Oct 10, Mike Latimer wrote:
On Friday, October 10, 2014 03:07:51 PM Olaf Hering wrote:
Thanks for the pointers. I will poke around. At least augtool on the host seems to behave correctly:
I don't think this is related to Augeas. Instead, I think it's the naming convention SUSE uses for the root btrfs subvolume (@).
It does not look like its the naming convention. Insteads its the fact that a given subvolume is for the entire root filesystem. So both sda2 and also the 'whatever' subvolume get ->is_root=1. I can probably add a check somewhere to catch the whatever=="@". A better fix would be to check if a given subvolume is for the entire partition. Furthermore there are appearent bugs in the subvolume parser. Will send a patch to fix one of them shortly. Olaf
On Thu, Oct 16, Olaf Hering wrote:
I can probably add a check somewhere to catch the whatever=="@". A better fix would be to check if a given subvolume is for the entire partition.
So after talking to David Sterba there is no way to tell if a given subvolume is for the entire partition. Also my attempt to "fix" it by using the -o option is unreliable. Richard, so what should be done with the root-partition-listed-twice issue mentioned in the URL above? Just check for a plain '@', assume thats the entire thing, and skip it to not set is_root=1? The other bug is that mounting subvolumes in such setup fails anyway because the subvol= argument lacks the absolute path. It was suggested to use the "subvolume list" option -a and tweak the resulting list. There is a <FS_TREE> prefix, which shouldnt be there according to David. Olaf
On Friday 17 October 2014 11:25:03 Olaf Hering wrote:
On Thu, Oct 16, Olaf Hering wrote:
I can probably add a check somewhere to catch the whatever=="@". A better fix would be to check if a given subvolume is for the entire partition.
So after talking to David Sterba there is no way to tell if a given subvolume is for the entire partition. Also my attempt to "fix" it by using the -o option is unreliable.
Richard, so what should be done with the root-partition-listed-twice issue mentioned in the URL above? Just check for a plain '@', assume thats the entire thing, and skip it to not set is_root=1?
Is '@' a particular marker for this situation? Or, from what I get, there can be any label for this?
The other bug is that mounting subvolumes in such setup fails anyway because the subvol= argument lacks the absolute path. It was suggested to use the "subvolume list" option -a and tweak the resulting list. There is a <FS_TREE> prefix, which shouldnt be there according to David.
Do you have a way to reproduce the above situation (and possibly the root-partition-listed-twice situation too)? I'm not too familiar with btrfs yet, and my couple of simple installation of the latest beta of openSUSE 13.2 didn't get me that. Thanks, -- Pino Toscano
On Fri, Oct 17, Pino Toscano wrote:
On Friday 17 October 2014 11:25:03 Olaf Hering wrote:
On Thu, Oct 16, Olaf Hering wrote:
I can probably add a check somewhere to catch the whatever=="@". A better fix would be to check if a given subvolume is for the entire partition.
So after talking to David Sterba there is no way to tell if a given subvolume is for the entire partition. Also my attempt to "fix" it by using the -o option is unreliable.
Richard, so what should be done with the root-partition-listed-twice issue mentioned in the URL above? Just check for a plain '@', assume thats the entire thing, and skip it to not set is_root=1?
Is '@' a particular marker for this situation? Or, from what I get, there can be any label for this?
Its my understanding that the subvolume name can be everything.
The other bug is that mounting subvolumes in such setup fails anyway because the subvol= argument lacks the absolute path. It was suggested to use the "subvolume list" option -a and tweak the resulting list. There is a <FS_TREE> prefix, which shouldnt be there according to David.
Do you have a way to reproduce the above situation (and possibly the root-partition-listed-twice situation too)? I'm not too familiar with btrfs yet, and my couple of simple installation of the latest beta of openSUSE 13.2 didn't get me that.
Right, because there is no subvolume for /. Only SLE12 is using that, and appearently Ubuntu if I understand the conversion back in April correctly. I was told that it should be possible to have subvolumes named like "os1", "os2", "osN" where each one would be a root filesystem. Now the interesting case is what happens with the entire partition. Does it get its own subvolume or will each of those installations go into their own subvolume right away? If they do the partition will have no files that would trigger is_root=1. I guess all these subvolumes will be discovered anyway and each one will get the is_root flag set, leading to the mutli-boot case. Thats fine because the given disk/partition is indeed multi-boot on purpose. So the bottom line is that there is room for improvement if a given btrfs partition has indeed multiple root subvolumes. I think to cover these cases the list command has to be called with -a. And there has to be a way to specify which subvolume to work with. Maybe guestfish can already do that, no idea. However, it would be nice if the common case of single subvolume "@" would be handled out of the box, so that virt-ls and others work out of the box. Olaf
On Fri, Oct 10, Richard W.M. Jones wrote:
We use Augeas to parse the /etc/fstab btrfs entries, see src/inspect-fs-unix.c: check_fstab:
https://github.com/libguestfs/libguestfs/blob/master/src/inspect-fs-unix.c#L...
And we then pass the subvol back to the mount command in daemon/mount.c (which doesn't alter it).
https://github.com/libguestfs/libguestfs/blob/master/daemon/mount.c#L154
Something between add_fstab_entry and the actual mount call removes the '@/'. My debug() shows: add_fstab_entry(1048): fstab: mountable=btrfsvol:/dev/sda2/@/var/spool mountpoint=/var/spool libguestfs: check_for_filesystem_on: btrfsvol:/dev/sda2/var/spool (btrfs) So 'fs->fstab[].mountable' gets tweaked somewhere. Olaf
On Fri, Oct 10, Olaf Hering wrote:
Is btrfs subvol support failing just for me?
This is an old thread, and it seems the underlying bug is not fixed because guestfs takes the wrong approach, IMO.
From how I understand btrfs, it has the concept of subvolumes. Each one is an entry point for a separate filesystem. In addition to that it has a default subvol which is used if no specific subvolume is specified. This turns the whole btrfs partition (or assembled blockdevice) into a container of individual filesystems.
But guestfs, as far as I understand it, just mounts the btrfs filesystem with the default subvolume and ends up somewhere. Then it tries to make sense of the other subvolumes based on info from a /etc/fstab, which may or may not exist. This looks like the wrong approach. Shouldnt it instead handle the whole thing as a container, just as it does for an individual disk? First do something like mount subvolid=0, poke around there and collect the items (sub list -a). Then use all this info to see which one might be one of the many root filesystems? Olaf
On Wed, Sep 09, 2015 at 07:14:05PM +0200, Olaf Hering wrote:
On Fri, Oct 10, Olaf Hering wrote:
Is btrfs subvol support failing just for me?
This is an old thread, and it seems the underlying bug is not fixed because guestfs takes the wrong approach, IMO.
From how I understand btrfs, it has the concept of subvolumes. Each one is an entry point for a separate filesystem. In addition to that it has a default subvol which is used if no specific subvolume is specified. This turns the whole btrfs partition (or assembled blockdevice) into a container of individual filesystems.
But guestfs, as far as I understand it, just mounts the btrfs filesystem with the default subvolume and ends up somewhere. Then it tries to make sense of the other subvolumes based on info from a /etc/fstab, which may or may not exist. This looks like the wrong approach.
Shouldnt it instead handle the whole thing as a container, just as it does for an individual disk? First do something like mount subvolid=0, poke around there and collect the items (sub list -a). Then use all this info to see which one might be one of the many root filesystems?
Matt? Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-builder quickly builds VMs from scratch http://libguestfs.org/virt-builder.1.html
participants (4)
-
Mike Latimer -
Olaf Hering -
Pino Toscano -
Richard W.M. Jones