10:58 p.m.
hi guys,
happy new year!!!
I encountered a problem when we used virt-df command to display my
guest disks useage.
the virt-df command from pkg
libguestfs-tools-c-1.36.3-6.el7_4.3.x86_64,version is virt-df --version
: virt-df 1.36.3rhel=7,release=6.el7_4.3,libvirt
the output in guest with the method referred in
http://libguestfs.org/virt-df.1.html#STATVFS NUMBERS
is:
[root@localhost ~]# df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mapper/centos-root 23544596 4335172 19209424 19% /
devtmpfs 931220 0 931220 0% /dev
tmpfs 941860 0 941860 0% /dev/shm
tmpfs 941860 8612 933248 1% /run
tmpfs 941860 0 941860 0% /sys/fs/cgroup
/dev/sda1 508588 133328 375260 27% /boot
tmpfs 188376 0 188376 0% /run/user/0
using scripts:
python -c 'import os; s = os.statvfs ("/"); print s'
posix.statvfs_result(f_bsize=4096, f_frsize=4096, f_blocks=5886149,
f_bfree=4802342, f_bavail=4802342, f_files=23556096, f_ffree=23435372,
f_favail=23435372, f_flag=4096, f_namemax=255)
python -c 'import os; s = os.statvfs ("/boot"); print s'
posix.statvfs_result(f_bsize=4096, f_frsize=4096, f_blocks=127147,
f_bfree=93815, f_bavail=93815, f_files=512000, f_ffree=511626,
f_favail=511626, f_flag=4096, f_namemax=255)
and then on host using virt-df showing:
# sudo virt-df rpm-build-for-7.2
Filesystem 1K-blocks Used Available Use%
rpm-build-for-7.2:/dev/sda1 508588 107728 400860 22%
rpm-build-for-7.2:/dev/centos/root 23544596 4298384 19246212 19%
using scripts:
# guestfish --ro -d rpm-build-for-7.2 -i statvfs /
bsize: 4096
frsize: 4096
blocks: 5886149
bfree: 4810534
bavail: 4810534
files: 23556096
ffree: 23435372
favail: 23435372
fsid: 64513
flag: 4097
namemax: 255
# sudo guestfish --ro -d rpm-build-for-7.2 -i statvfs /boot
bsize: 4096
frsize: 4096
blocks: 127147
bfree: 100215
bavail: 100215
files: 512000
ffree: 511626
favail: 511626
fsid: 2049
flag: 4097
namemax: 255
we can saw that in guest the statvfs get the value was the same with the
"df" command output. but the bfree/bavail values are different between
using guestfish statvfs on guest and
using statvfs in guest. I want to know whether is this a bug? or Does
same reason cause this appearance？
Thanks,
Chen
5:41 a.m.
On Thu, Jan 04, 2018 at 12:58:40PM +0800, Chen Fan wrote:
[In guest]
python -c 'import os; s = os.statvfs ("/"); print
s'
posix.statvfs_result(f_bsize=4096, f_frsize=4096, f_blocks=5886149,
f_bfree=4802342, f_bavail=4802342, f_files=23556096,
f_ffree=23435372, f_favail=23435372, f_flag=4096, f_namemax=255)
python -c 'import os; s = os.statvfs ("/boot"); print s'
posix.statvfs_result(f_bsize=4096, f_frsize=4096, f_blocks=127147,
f_bfree=93815, f_bavail=93815, f_files=512000, f_ffree=511626,
f_favail=511626, f_flag=4096, f_namemax=255)
[From the host via libguestfs]
# guestfish --ro -d rpm-build-for-7.2 -i statvfs /
bsize: 4096
frsize: 4096
blocks: 5886149
bfree: 4810534
bavail: 4810534
files: 23556096
ffree: 23435372
favail: 23435372
fsid: 64513
flag: 4097
namemax: 255
# sudo guestfish --ro -d rpm-build-for-7.2 -i statvfs /boot
bsize: 4096
frsize: 4096
blocks: 127147
bfree: 100215
bavail: 100215
files: 512000
ffree: 511626
favail: 511626
fsid: 2049
flag: 4097
namemax: 255
Was the guest quiescent and fully sync()d before you ran the python
commands?
In any case all that libguestfs is doing is executing the
statvfs(2) system call on a copy of the disk:
https://github.com/libguestfs/libguestfs/blob/e6e23c4d77ee15779f833fd4fbe...
So if there's any difference then it's something to do with the guest
kernel vs the host kernel calculations.
I wouldn't expect the numbers to be exactly the same, but libguestfs
doesn't alter the numbers, it just passes them through.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW
9:21 p.m.
hi Jones, Thanks for your reply.
At 2018-01-04 19:41:27, "Richard W.M. Jones" <rjones(a)redhat.com> wrote:
On Thu, Jan 04, 2018 at 12:58:40PM +0800, Chen Fan wrote:
[In guest]
> python -c 'import os; s = os.statvfs ("/"); print s'
> posix.statvfs_result(f_bsize=4096, f_frsize=4096, f_blocks=5886149,
> f_bfree=4802342, f_bavail=4802342, f_files=23556096,
> f_ffree=23435372, f_favail=23435372, f_flag=4096, f_namemax=255)
>
> python -c 'import os; s = os.statvfs ("/boot"); print s'
> posix.statvfs_result(f_bsize=4096, f_frsize=4096, f_blocks=127147,
> f_bfree=93815, f_bavail=93815, f_files=512000, f_ffree=511626,
> f_favail=511626, f_flag=4096, f_namemax=255)
[From the host via libguestfs]
> # guestfish --ro -d rpm-build-for-7.2 -i statvfs /
>
> bsize: 4096
> frsize: 4096
> blocks: 5886149
> bfree: 4810534
> bavail: 4810534
> files: 23556096
> ffree: 23435372
> favail: 23435372
> fsid: 64513
> flag: 4097
> namemax: 255
>
> # sudo guestfish --ro -d rpm-build-for-7.2 -i statvfs /boot
> bsize: 4096
> frsize: 4096
> blocks: 127147
> bfree: 100215
> bavail: 100215
> files: 512000
> ffree: 511626
> favail: 511626
> fsid: 2049
> flag: 4097
> namemax: 255
Was the guest quiescent and fully sync()d before you ran the python
commands?
yes, almost the guest is quiescent, and after invoke sync(), the results have no
different.
In any case all that libguestfs is doing is executing the
statvfs(2) system call on a copy of the disk:
https://github.com/libguestfs/libguestfs/blob/e6e23c4d77ee15779f833fd4fbe...
So if there's any difference then it's something to do with the guest
kernel vs the host kernel calculations.
I saw that when doing virt-df, virt-df created a temporary VM with the host kernel, so I
recreate a temporary QEMU VM with command line:
/usr/libexec/qemu-kvm -cpu host -m 1024 -nographic -kernel
/var/tmp/.guestfs-0/appliance.d/kernel -initrd /var/tmp/.guestfs-0/appliance.d/initrd
-append "console=ttyS0 root=/dev/vda guestfs_rescue=1" -device
virtio-blk-pci,drive=disk0 -drive
file=/var/tmp/.guestfs-0/appliance.d/root,id=disk0,if=none -device virtio-serial -chardev
socket,id=charchannel0,path=/tmp/a,server,nowait -device
virtserialport,nr=1,chardev=charchannel0,id=channel0,name=org.libguestfs.channel.0 -device
virtio-blk-pci,drive=disk1 -drive file=/srv/imgs/centos-7.2.qcow2,if=none,id=disk1
then the console showing:
<rescue> mount /dev/vdb1 /sysroot/
[ 72.552185] SGI XFS
with ACLs, security attributes, no debug enabled
[ 72.568377] XFS (vdb1): Mounting V4 Filesystem
[ 73.379679] XFS (vdb1): Starting recovery (logdev: internal)
[ 73.406398] XFS (vdb1): Ending recovery (logdev: internal)
<rescue> statvfs
bash: statvfs: command not found
<rescue> stat -f /sysroot
File: "/sysroot"
ID: fd1100000000 Namelen: 255 Type: xfs
Block size: 4096 Fundamental block size: 4096
Blocks: Total: 127147 Free: 93815 Available: 93815
Inodes: Total: 512000 Free: 511626
here the mount /dev/vdb1 device is the /boot partition from my guest, we can saw that
the free blocks are the same as python command, so I think the host kernel does not affect
the result.
if I am wrong, pls correct me :)
Thanks,
Chen
I wouldn't expect the numbers to be exactly the same, but libguestfs
doesn't alter the numbers, it just passes them through.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW
10:22 a.m.
To summarise:
[In guest]
python -c 'import os; s = os.statvfs ("/boot"); print s'
posix.statvfs_result(f_bsize=4096, f_frsize=4096, f_blocks=127147,
f_bfree=93815, f_bavail=93815, f_files=512000, f_ffree=511626,
f_favail=511626, f_flag=4096, f_namemax=255)
[From the host via libguestfs]
# sudo guestfish --ro -d rpm-build-for-7.2 -i statvfs /boot
bsize: 4096
frsize: 4096
blocks: 127147
bfree: 100215
bavail: 100215
files: 512000
ffree: 511626
favail: 511626
fsid: 2049
flag: 4097
namemax: 255
[In virt-rescue]
Block size: 4096 Fundamental block size: 4096
Blocks: Total: 127147 Free: 93815 Available: 93815
Inodes: Total: 512000 Free: 511626
The block stats are the same as the "in guest" results above,
but slightly different from libguestfs.
- * - * - * -
I wanted to see if I could reproduce this with a very simple and
reproducible example. I made the assumption that this guest is using XFS.
(1) Make an XFS filesystem and use libguestfs to query the statvfs:
$ guestfish -N fs:xfs -m /dev/sda1 statvfs /
bsize: 4096
frsize: 4096
blocks: 24713
bfree: 23347
bavail: 23347
files: 51136
ffree: 51133
favail: 51133
fsid: 2049
flag: 4096
namemax: 255
(2) Open the same disk image in virt-rescue and examine with stat
command:
$ virt-rescue -a test1.img
<rescue> mount /dev/sda1 /sysroot
<rescue> stat -f /sysroot
File: "/sysroot"
ID: 80100000000 Namelen: 255 Type: xfs
Block size: 4096 Fundamental block size: 4096
Blocks: Total: 24713 Free: 23347 Available: 23347
Inodes: Total: 51136 Free: 51133
In this case the numbers are the same ...
I would really like a bit more information though:
(a) Are you using XFS or another filesystem?
(b) Can you make a small independent reproducer like mine above?
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines. Supports shell scripting,
bindings from many languages. http://libguestfs.org
8:11 p.m.
hi Jones,
Sorry for the late reply. when I used your method to execute command:
$guestfish -N fs:xfs -m /dev/sda1 statvfs /
libguestfs: error: mkfs: xfs: /dev/sda1: mkfs.xfs: No such file or directory
guestfish: error creating prepared disk image 'fs:xfs' on 'test1.img':
failed to create filesystem (xfs): mkfs: xfs: /dev/sda1: mkfs.xfs: No such file or
directory
does I need to rebuild the iniramfs to include mkfs.xfs, is this the root reason that
cause the different numbers ?
then I used virt-rescure method , the filesystem type was ext2/ext3:
<rescue> mount /dev/sda1 /sysroot/
mount: /dev/sda1 is
write-protected, mounting read-only
mount: unknown filesystem type '(null)'
<rescue> stat -f /sysroot
File: "/sysroot"
ID: ab0d9f3277e010fd Namelen: 255 Type: ext2/ext3
Block size: 4096 Fundamental block size: 4096
Blocks: Total: 1032112 Free: 941307 Available: 888879
Inodes: Total: 262144 Free: 250996
<rescue>
Thanks,
Chen
At 2018-01-05 00:22:46, "Richard W.M. Jones" <rjones(a)redhat.com> wrote:
>To summarise:
>
>>[In guest]
>> python -c 'import os; s = os.statvfs ("/boot"); print s'
>> posix.statvfs_result(f_bsize=4096, f_frsize=4096, f_blocks=127147,
>> f_bfree=93815, f_bavail=93815, f_files=512000, f_ffree=511626,
>> f_favail=511626, f_flag=4096, f_namemax=255)
>
>>[From the host via libguestfs]
>> # sudo guestfish --ro -d rpm-build-for-7.2 -i statvfs /boot
>> bsize: 4096
>> frsize: 4096
>> blocks: 127147
>> bfree: 100215
>> bavail: 100215
>> files: 512000
>> ffree: 511626
>> favail: 511626
>> fsid: 2049
>> flag: 4097
>> namemax: 255
>
>[In virt-rescue]
>> Block size: 4096 Fundamental block size: 4096
>> Blocks: Total: 127147 Free: 93815 Available: 93815
>> Inodes: Total: 512000 Free: 511626
>
>The block stats are the same as the "in guest" results above,
>but slightly different from libguestfs.
>
> - * - * - * -
>
>I wanted to see if I could reproduce this with a very simple and
>reproducible example. I made the assumption that this guest is using XFS.
>
>(1) Make an XFS filesystem and use libguestfs to query the statvfs:
>
>$ guestfish -N fs:xfs -m /dev/sda1 statvfs /
>bsize: 4096
>frsize: 4096
>blocks: 24713
>bfree: 23347
>bavail: 23347
>files: 51136
>ffree: 51133
>favail: 51133
>fsid: 2049
>flag: 4096
>namemax: 255
>
>(2) Open the same disk image in virt-rescue and examine with stat
>command:
>
>$ virt-rescue -a test1.img
>><rescue> mount /dev/sda1 /sysroot
><rescue> stat -f /sysroot
> File:
"/sysroot"
> ID: 80100000000 Namelen: 255 Type: xfs
>Block size: 4096 Fundamental block size: 4096
>Blocks: Total: 24713 Free: 23347 Available: 23347
>Inodes: Total: 51136 Free: 51133
>
>
>In this case the numbers are the same ...
>
>I would really like a bit more information though:
>
>(a) Are you using XFS or another filesystem?
>
>(b) Can you make a small independent reproducer like mine above?
>
>Rich.
>
>--
>Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
>Read my programming and virtualization blog: http://rwmj.wordpress.com
>libguestfs lets you edit virtual machines. Supports shell scripting,
>bindings from many languages. http://libguestfs.org
6:22 a.m.
On Sun, Jan 07, 2018 at 08:11:58PM +0800, 陈帆 wrote:
Sorry for the late reply. when I used your method to execute
command:
$guestfish -N fs:xfs -m /dev/sda1 statvfs /
libguestfs: error: mkfs: xfs: /dev/sda1: mkfs.xfs: No such file or directory
guestfish: error creating prepared disk image 'fs:xfs' on 'test1.img':
failed to create filesystem (xfs): mkfs: xfs: /dev/sda1: mkfs.xfs: No such file or
directory
You need to install ‘libguestfs-xfs’.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines. Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v
9:06 p.m.
after install libguestfs_xfs, all results are:
[using guestfish]
guestfish -N fs:xfs -m /dev/sda1 statvfs /
bsize: 4096
frsize: 4096
blocks: 24713
bfree: 23391
bavail: 23391
files: 51136
ffree: 51133
favail: 51133
fsid: 2049
flag: 4096
namemax: 255
[using virt-rescure]
virt-rescue -a test1.img
<rescue> mount /dev/sda1 /sysroot
<rescue> stat -f /sysroot
File: "/sysroot"
ID: 80100000000 Namelen: 255 Type: xfs
Block size: 4096 Fundamental block size: 4096
Blocks: Total: 24713 Free: 23391 Available: 23391
Inodes: Total: 51136 Free: 51133
the free numbers are the same. but the "virt-df" output is not changed.
Thanks,
Chen
在 2018-01-07 20:22:28，"Richard W.M. Jones" <rjones(a)redhat.com> 写道：
On Sun, Jan 07, 2018 at 08:11:58PM +0800, 陈帆 wrote:
> Sorry for the late reply. when I used your method to execute command:
> $guestfish -N fs:xfs -m /dev/sda1 statvfs /
> libguestfs: error: mkfs: xfs: /dev/sda1: mkfs.xfs: No such file or directory
> guestfish: error creating prepared disk image 'fs:xfs' on
'test1.img': failed to create filesystem (xfs): mkfs: xfs: /dev/sda1: mkfs.xfs: No
such file or directory
You need to install ‘libguestfs-xfs’.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines. Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v
2:23 a.m.
There are some strange things going on with the kernel XFS driver.
Opening a filesystem for write or read-only makes a difference to the
stats returned.
With bit-for-bit identical filesystems:
<rescue> mount /dev/sda1 /sysroot
<rescue> stat -f /sysroot
File: "/sysroot"
ID: 80100000000 Namelen: 255 Type: xfs
Block size: 4096 Fundamental block size: 4096
Blocks: Total: 24713 Free: 23347 Available: 23347
Inodes: Total: 51136 Free: 51133
vs:
<rescue> mount -o ro /dev/sda1 /sysroot
<rescue> stat -f /sysroot
File: "/sysroot"
ID: 80100000000 Namelen: 255 Type: xfs
Block size: 4096 Fundamental block size: 4096
Blocks: Total: 24713 Free: 24653 Available: 24653
Inodes: Total: 51136 Free: 51133
virt-df always uses the ‘-o ro’ option.
Despite this virt-df itself is working fine:
$ virt-df -a test1.img
Filesystem 1K-blocks Used Available Use%
test1.img:/dev/sda1 98852 240 98612 1%
Using the calculations from here:
https://github.com/libguestfs/libguestfs/blob/ab0a3e027665696502141319438...
blocks * bsize / 1024 = 24713 * 4096 / 1024 = 98852
(blocks - bfree) * bsize / 1024 = (24713-24653) * 4096 / 1024 = 240
bavail * bsize / 1024 = 24653 * 4096 / 1024 = 98612
So everything appears to be working fine, besides possibly odd stuff
going on in the kernel XFS driver.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW
6:09 a.m.
The answer is attached. In summary it's not a bug in virt-df,
nor df, nor the kernel.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine. Supports Linux and Windows.
http://people.redhat.com/~rjones/virt-df/
2491
days inactive
2495
days old
9 comments
3 participants
participants (3)
-
Chen Fan -
Richard W.M. Jones -
陈帆