11:06 p.m.
On Tue, Apr 05, 2022 at 07:45:05AM -0700, Alli wrote:
On Mon, 2022-04-04 at 19:45 +0100, Richard W.M. Jones wrote:
> On Sat, Apr 02, 2022 at 08:18:52AM -0700, Alli wrote:
> > Hi all,
> > I hope I am in the right place, I found this email by following the
> > link in the project README. We are collecting use cases for xfs
> > dev
> > projects, one of which would be enhancing xfs protofiles, which is
> > a
> > type of script that facilitates filesystem generation. I am not
> > very familiar with libguestfs, but a coworker suggested that this
> > might be of interest to your project. More specifically, the task
> > would be to add attributes to the files generated by the protofile
> > script. Does this sound like something that might be of use to
> > libguestfs? I did take a quick glance through the libguestfs code,
> > and didnt notice any current usage of xfs protofiles, but I thought
> > it was worth asking if protofiles might become of interest if we
> > did
> > add this feature. Thanks!
>
> Hello Allison,
>
> I have to admit I'm not sure what any of this means. Also the
> information about protofiles is a bit thin. There is some
> information
> in the mkfs.xfs(8) man page, but even that is a bit unclear.
Yes, there's just that bit under the "-p protofile" option
>
> Is it like the (very useful) "-d" option of mke2fs?
Not exactly. You would use the -p flag and pass it a text file with
the protofile script in it. So here is example of using the protofile
from the man page:
root@garnet:/home/achender/work_area/study_group_demos# cat proto.txt
/stand/diskboot
4872 110
d--777 3 1
usr d--777 3 1
sh ---755 3 1 /bin/sh
ken d--755 6 1
$
b0 b--644 3 1 0 0
c0 c--644 3 1 0 0
fifo p--644 3 1
slink l--644 3 1 /a/symbolic/link
: This is a comment line
$
$
FWIW, protofiles date back to Unix 4th Edition from November 1973. IIRC
the original reason for creating mkfs (with protofiles) was to make it
easier for Ken Thompson to "deploy" Unix onto the disk packs that people
sent him, since before that he had to write the superblock and the root
directory by hand, and being able to copy *some* of his computer was
quite the time saver.
They're documented on page 274 here:
https://dspinellis.github.io/unix-v4man/v4man.pdf
XFS (and JFS, and AIX and HPUX) still use the protofile format as a
means of copying files into a newly formatted filesystem. I'm not sure
why e2fsprogs went their own way, though I'd imagine that modern deploy
image strategies create a chroot and import everything within that part
of the directory tree instead of cherry-picking files off an existing
filesystem.
Possibly the answer here is that protofiles have have their day, and
mkfs.xfs should just grow the means to import a directory tree into the
filesystem so that you can run the image creation tools as non-root.
OTOH, I /do/ have a protofile generator up my sleeve, and as far as XFS
goes it would be a bit less churn to add the one missing piece instead
of rewriting a ton of mkfs code:
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git/c...
If it would be more generally useful for libguestfs to be able to format
and populate filesystems, we could talk about committing xfs_protofile
into the upstream xfsprogs distribution, as well as enhancing it to pick
up extended attributes and fsxattrs from the files it's importing?
--D
root@garnet:/home/achender/work_area/study_group_demos# dd
if=/dev/zero
of=image.bin bs=1048576 count=1024
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 2.60282 s, 413 MB/s
root@garnet:/home/achender/work_area/study_group_demos# mkfs.xfs -p
proto.txt -d name=image.bin
meta-data=image.bin isize=512 agcount=4, agsize=65536
blks
= sectsz=512 attr=2, projid32bit=1
= crc=1 finobt=1, sparse=1,
rmapbt=0
= reflink=1 bigtime=1 inobtcount=1
data = bsize=4096 blocks=262144, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0, ftype=1
log =internal log bsize=4096 blocks=2560, version=2
= sectsz=512 sunit=0 blks, lazy-
count=1
realtime =none extsz=4096 blocks=0, rtextents=0
root@garnet:/home/achender/work_area/study_group_demos# mount image.bin
mntPt/
root@garnet:/home/achender/work_area/study_group_demos# ls mntPt/
usr
root@garnet:/home/achender/work_area/study_group_demos# ls mntPt/usr/
b0 c0 fifo ken sh slink
root@garnet:/home/achender/work_area/study_group_demos#
>
> Libguestfs does not currently provide a binding around mkfs.xfs.
> Instead we rely on generic mkfs support and that only encodes a few
> baseline mkfs features like blocksize:
>
>
https://urldefense.com/v3/__https://github.com/libguestfs/libguestfs/blob...
>
https://urldefense.com/v3/__https://github.com/libguestfs/libguestfs/blob...
>
> This is unusual. We do support other mkfs-like tools more directly,
> eg mke2fs:
>
>
https://urldefense.com/v3/__https://github.com/libguestfs/libguestfs/blob...
>
https://urldefense.com/v3/__https://github.com/libguestfs/libguestfs/blob...
>
> And we also currently have support for other xfs_* tools like
> xfs_info, xfs_growfs. Just not (yet) any support for mkfs.xfs.
I see, generally protofiles are of interest when you want to create an
image with a particular root file system in it. Since there's no
journaling, it's a lot faster than creating and populating an empty fs.
It looks like there's a couple areas in libguestfs where a folder is
created, though not a huge tree. Perhaps not enough to be motivating
right now, since this would only apply to an xfs type, but maybe
something to keep in mind should the use case arise :-)
>
> I've copied Eric Sandeen in on this email as he usually knows a lot
> more than I do.
Sure, Eric may have some more ideas about use cases for this as well.
Thanks for the feedback!
Allison
>
> Rich.
>