On Tue, Apr 05, 2022 at 07:45:05AM -0700, Alli wrote:
On Mon, 2022-04-04 at 19:45 +0100, Richard W.M. Jones wrote:
On Sat, Apr 02, 2022 at 08:18:52AM -0700, Alli wrote:
Hi all, I hope I am in the right place, I found this email by following the link in the project README. We are collecting use cases for xfs dev projects, one of which would be enhancing xfs protofiles, which is a type of script that facilitates filesystem generation. I am not very familiar with libguestfs, but a coworker suggested that this might be of interest to your project. More specifically, the task would be to add attributes to the files generated by the protofile script. Does this sound like something that might be of use to libguestfs? I did take a quick glance through the libguestfs code, and didnt notice any current usage of xfs protofiles, but I thought it was worth asking if protofiles might become of interest if we did add this feature. Thanks!
Hello Allison,
I have to admit I'm not sure what any of this means. Also the information about protofiles is a bit thin. There is some information in the mkfs.xfs(8) man page, but even that is a bit unclear. Yes, there's just that bit under the "-p protofile" option
Is it like the (very useful) "-d" option of mke2fs?
Not exactly. You would use the -p flag and pass it a text file with the protofile script in it. So here is example of using the protofile from the man page:
root@garnet:/home/achender/work_area/study_group_demos# cat proto.txt /stand/diskboot 4872 110 d--777 3 1 usr d--777 3 1 sh ---755 3 1 /bin/sh ken d--755 6 1 $ b0 b--644 3 1 0 0 c0 c--644 3 1 0 0 fifo p--644 3 1 slink l--644 3 1 /a/symbolic/link : This is a comment line $ $
FWIW, protofiles date back to Unix 4th Edition from November 1973. IIRC the original reason for creating mkfs (with protofiles) was to make it easier for Ken Thompson to "deploy" Unix onto the disk packs that people sent him, since before that he had to write the superblock and the root directory by hand, and being able to copy *some* of his computer was quite the time saver. They're documented on page 274 here: https://dspinellis.github.io/unix-v4man/v4man.pdf XFS (and JFS, and AIX and HPUX) still use the protofile format as a means of copying files into a newly formatted filesystem. I'm not sure why e2fsprogs went their own way, though I'd imagine that modern deploy image strategies create a chroot and import everything within that part of the directory tree instead of cherry-picking files off an existing filesystem. Possibly the answer here is that protofiles have have their day, and mkfs.xfs should just grow the means to import a directory tree into the filesystem so that you can run the image creation tools as non-root. OTOH, I /do/ have a protofile generator up my sleeve, and as far as XFS goes it would be a bit less churn to add the one missing piece instead of rewriting a ton of mkfs code: https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git/commit/?h=metadir&id=8c11be97ed81bc6ae49c066c422b9d3fc662f232 If it would be more generally useful for libguestfs to be able to format and populate filesystems, we could talk about committing xfs_protofile into the upstream xfsprogs distribution, as well as enhancing it to pick up extended attributes and fsxattrs from the files it's importing? --D
root@garnet:/home/achender/work_area/study_group_demos# dd if=/dev/zero of=image.bin bs=1048576 count=1024 1024+0 records in 1024+0 records out 1073741824 bytes (1.1 GB, 1.0 GiB) copied, 2.60282 s, 413 MB/s root@garnet:/home/achender/work_area/study_group_demos# mkfs.xfs -p proto.txt -d name=image.bin meta-data=image.bin isize=512 agcount=4, agsize=65536 blks = sectsz=512 attr=2, projid32bit=1 = crc=1 finobt=1, sparse=1, rmapbt=0 = reflink=1 bigtime=1 inobtcount=1 data = bsize=4096 blocks=262144, imaxpct=25 = sunit=0 swidth=0 blks naming =version 2 bsize=4096 ascii-ci=0, ftype=1 log =internal log bsize=4096 blocks=2560, version=2 = sectsz=512 sunit=0 blks, lazy- count=1 realtime =none extsz=4096 blocks=0, rtextents=0 root@garnet:/home/achender/work_area/study_group_demos# mount image.bin mntPt/ root@garnet:/home/achender/work_area/study_group_demos# ls mntPt/ usr root@garnet:/home/achender/work_area/study_group_demos# ls mntPt/usr/ b0 c0 fifo ken sh slink root@garnet:/home/achender/work_area/study_group_demos#
Libguestfs does not currently provide a binding around mkfs.xfs. Instead we rely on generic mkfs support and that only encodes a few baseline mkfs features like blocksize:
https://urldefense.com/v3/__https://github.com/libguestfs/libguestfs/blob/31... https://urldefense.com/v3/__https://github.com/libguestfs/libguestfs/blob/31...
This is unusual. We do support other mkfs-like tools more directly, eg mke2fs:
https://urldefense.com/v3/__https://github.com/libguestfs/libguestfs/blob/31... https://urldefense.com/v3/__https://github.com/libguestfs/libguestfs/blob/31...
And we also currently have support for other xfs_* tools like xfs_info, xfs_growfs. Just not (yet) any support for mkfs.xfs.
I see, generally protofiles are of interest when you want to create an image with a particular root file system in it. Since there's no journaling, it's a lot faster than creating and populating an empty fs.
It looks like there's a couple areas in libguestfs where a folder is created, though not a huge tree. Perhaps not enough to be motivating right now, since this would only apply to an xfs type, but maybe something to keep in mind should the use case arise :-)
I've copied Eric Sandeen in on this email as he usually knows a lot more than I do.
Sure, Eric may have some more ideas about use cases for this as well. Thanks for the feedback!
Allison
Rich.