RFC: OSTree support
by Pino Toscano
Hi,
currently, libguestfs does not support the inspection of OSTree/Atomic
disk images [1].
While this means that certain tools which use inspection to
automatically mount all the partitions available (virt-customize,
virt-builder, virt-sysprep, virt-diff, guestfish -i, and the default
modes of virt-cat, virt-log, virt-edit) will fail, although it's still
possible to manually mount partitions as usual.
Let's start from an example, the Fedora 21 cloud/atomic image,
Fedora-Cloud-Atomic-20141203-21.x86_64.qcow2 (or .raw.xz):
><fs> list-filesystems
/dev/sda1: ext4
/dev/atomicos/root: xfs
/dev/sda1 is a "shared /boot", while /dev/atomicos/root contains the
root and all the deploys:
/
+- /ostree/deploy/fedora-atomic
+- deploy/ba7ee9475c462c9265517ab1e5fb548524c01a71709539bbe744e5fdccf6288b.0
so there is a single deploy, "fedora-atomic", containing one OS in it,
ba7ee9475...etc (having revision 0).
Considering I would prefer to not lose information of any of the bits
in the image (root and OSTree trees), so far I see two ways to properly
represent these information within libguestfs:
A) Single OS, multiple levels of mount points
This would mean representing OSTree images like:
/
+ deploy-A
| + checksum-1 (mount point / of it)
| | + mountpoint-1
| | + ...
| | + mountpoint-N
| + ...
| + checksum-N
+ ...
+ deploy-N
Pro:
- single OS found by inspection
- dynamically represent all the deploys
Contra:
- multi-level mount points are not supported
- tools would need to mount stuff in a better way than "get the list
of mount points, sort them, and mount them"
- need to proper "remap" the mount points in each checksum to the
actual directory of the checksum itself, so e.g. /boot found in the
fstab would be /ostree/deploy/$DEPLOY/deploy/$CHECKSUM/boot
- need to support bind mounts, so it's easier to just mount one
checksum to / or within its directory in the "global" root
B) Multiple roots
This would mean representing the "global" root and each "checksum root"
as different "OS", i.e. different roots returned by the inspection, so:
inspection:
+ /
+ checksum-1 (mount point / of it)
| + mountpoint-1
| + ...
| + mountpoint-N
+ ...
+ checksum-N
Pro:
- each OS is handled as separate OS
- single level of mount points for each OS, like done currently
Contra:
- tools would need to support multiple roots, and possibly let users
select which one
So far that's the not-in-depth summary of what's needed to be done,
at least from what I could say.
Thoughts?
[1] https://bugzilla.redhat.com/show_bug.cgi?id=1102241
Thanks,
--
Pino Toscano
9 years, 11 months
Re: [Libguestfs] Virt-copy-in Doubt
by Richard W.M. Jones
On Tue, Jan 13, 2015 at 02:49:03PM +0530, Ramani Kothadia wrote:
[A question about virt-copy-in & disk corruption, not reproduced
because it was sent privately]
libguestfs modifies the disk image directly. If the same disk image
is also being used by a running guest, then the guest kernel will not
realize that changes to filesystem metadata and data are being made
behind its back. It might make conflicting changes, or cache copies
of old disk blocks that libguestfs has changed.
It's a bit like connecting two host machines to a single disk drive,
and then expecting them to be able to write to the same filesystem on
that drive.
This is not a theoretical problem either: if you try it you really
will get disk corruption.
There are filesystems that can cope with being used in a cluster
(eg. RHCS GFS), but common filesystems like ext4 certainly cannot be
used this way.
libguestfs can access live filesystems in a special read-only mode, in
which case libguestfs does not write to the filesystem so cannot
interfere with the guest. This is still not completely error-free as
you may see unexpected/strange stuff on the libguestfs side. But it
is safe, and in practice works fine.
Yes, the virtio-serial device is used to implement virt-copy-in.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines. Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top
9 years, 11 months
[Query] Access procfs of guest VM using libguestfs
by Surojit Pathak
Hi all,
I am using libvirt:qemu-kvm and I want to access(RO) the procfs of the
guest vm.
I was trying to use guestfs for the same purpose.
I can access /tmp of the guest but not the /proc
Is it a known issue? If so, how can I overcome the same? Any pointers?
Snippet>
[root@hv285 scoreboards]# virsh list
Id Name State
----------------------------------------------------
80 instance-00000121 running
[root@hv285 scoreboards]# virt-cat -d instance-00000121 /tmp/uname
Linux 2.6.32-431.11.2.el6.XXXXX.20140407.x86_64 #1 SMP Mon Apr 7
11:31:51 CDT 2014 x86_64 x86_64 x86_64 GNU/Linux
[root@hv285 scoreboards]# guestfish --version
guestfish 1.20.11rhel=6,release=2.el6
[root@hv285 scoreboards]# uname
Linux
[root@hv285 scoreboards]# virt-cat -d instance-00000121 /proc/meminfo
libguestfs: error: download: /proc/meminfo: No such file or directory
[root@hv285 scoreboards]#
--
Regards,
SURO
9 years, 11 months
Installing Libguestfs
by Priyanka Naik
I want to transfer files between host and guest. What are the ways to
transfer these files? Is libguestfs the only way to transfer files between
host and guest?
Regards,
Priyanka Naik
9 years, 11 months
Windows 2003 BSOD, 0X0000007B bug
by Nicolas Ecarnot
Hello,
In last November, Richard helped me and succeed in P2V-converting a
windows 2003 server towards an oVirt/RHEV image.
Recent upgrades solved many ssh issues
https://bugzilla.redhat.com/show_bug.cgi?id=1167774, and conversion
seems OK now.
The server I converted is now booting and running fine every time.
I used the very same procedure on two others 2003 servers (one in x86
and one in 64 bits), and both are showing BSOD and the very well known
0x...7B bug.
As it is a very known bug, I tried all that is available on earth to try
to cope with it :
- before P2V, log as local admin, merge the mergeide.reg into the
registry, reboot, then halt and P2V
- before P2V, boot and install the different libraries viostor, net,
ballooning, and so on... then reboot, then P2V
- in the P2V iso GUI, disabled the conversion of every device except the
disk (remove floppy, NICs, CDROM)
- once converted and imported into oVirt, set the disk driver as virtio,
ide, iscsi...
BSOD remains...
I could give you every version of every parts used in the process, but
to make it short, let say that everything is up to date, nothing may be
more aged than two months.
I don't know what I could try next, as I think I tried everything I read
about.
--
Nicolas Ecarnot
9 years, 11 months
Access procfs of guest VM using libguestfs
by Surojit Pathak
Hi all,
I am using libvirt:qemu-kvm and I want to access(RO) the procfs of the
guest vm.
I was trying to use guestfs for the same purpose.
I can access /tmp of the guest but not the /proc
Is it a known issue? If so, how can I overcome the same? Any pointers?
Snippet>
[root@hv285 scoreboards]# virsh list
Id Name State
----------------------------------------------------
80 instance-00000121 running
[root@hv285 scoreboards]# virt-cat -d instance-00000121 /tmp/uname
Linux 2.6.32-431.11.2.el6.XXXXX.20140407.x86_64 #1 SMP Mon Apr 7
11:31:51 CDT 2014 x86_64 x86_64 x86_64 GNU/Linux
[root@hv285 scoreboards]# guestfish --version
guestfish 1.20.11rhel=6,release=2.el6
[root@hv285 scoreboards]# uname
Linux
[root@hv285 scoreboards]# virt-cat -d instance-00000121 /proc/meminfo
libguestfs: error: download: /proc/meminfo: No such file or directory
[root@hv285 scoreboards]#
--
Regards,
SURO
9 years, 11 months
(no subject)
by Gabriele Cerami
Hi, we needed these changes when we had to build a guest image
compatible with a starting guest image but not backed by it in any way•
We needed some tool to check our progress, comparing original and•
rebuilt (from scratch) images, and virt-diff seemed the best option, but•
we had to soften the comparison to reduce the noise in the output. I
added some options to ignore certain informations when comparing. I
built and tested it, and now for example, one can ignore file permission
with --no-compare-perms, and still see permissions shown when files
differ by other criterias.
9 years, 11 months
[PATCH] virt-diff: add additional ignore options
by Gabriele Cerami
--compare-xattrs
--compare-extra-stats
--compare-perms
--compare-uids
--compare-times
to ignore, when set to no, specified files informations when comparing.
The current strategy to disable comparison on file informations is to
flatten data structure so they return the same 0/NULL value on
comparison and be, in fact, ignored to determine if the files differ.
This patch preserve original information on copies. Comparison is still
made on flattened structures, except for xattrs, but preserved
structure are used when showing file informations.
---
diff/diff.c | 96 ++++++++++++++++++++++++++++++++++++++++++------------
diff/virt-diff.pod | 20 ++++++++++++
2 files changed, 95 insertions(+), 21 deletions(-)
diff --git a/diff/diff.c b/diff/diff.c
index 6a374af..72e95a1 100644
--- a/diff/diff.c
+++ b/diff/diff.c
@@ -58,6 +58,11 @@ const char *libvirt_uri = NULL;
int inspector = 1;
static int atime = 0;
+static int compare_extra_stats = 1;
+static int compare_perms = 1;
+static int compare_times = 1;
+static int compare_uids = 1;
+static int compare_xattrs = 1;
static int csv = 0;
static int dir_links = 0;
static int dir_times = 0;
@@ -104,6 +109,11 @@ usage (int status)
" --atime Don't ignore access time changes\n"
" -A image Add image from second guest\n"
" --checksum[=...] Use checksum of file content\n"
+ " --compare-extra-stats (yes|no) Compare stats (default: yes)\n"
+ " --compare-perms (yes|no) Compare permissions (default: yes)\n"
+ " --compare-times (yes|no) Compare times (Default: yes)\n"
+ " --compare-uids (yes|no) Compare uid and gid (default: yes)\n"
+ " --compare-xattrs (yes|no) Compare extended attributes (default: yes)\n"
" -c|--connect uri Specify libvirt URI for -d option\n"
" --csv Comma-Separated Values output\n"
" --dir-links Don't ignore directory nlink changes\n"
@@ -179,6 +189,11 @@ main (int argc, char *argv[])
{ "version", 0, 0, 'V' },
{ "xattr", 0, 0, 0 },
{ "xattrs", 0, 0, 0 },
+ { "compare-extra-stats", 1, 0, 0 },
+ { "compare-perms", 1, 0, 0 },
+ { "compare-times", 1, 0, 0 },
+ { "compare-uids", 1, 0, 0 },
+ { "compare-xattrs", 1, 0, 0 },
{ 0, 0, 0, 0 }
};
struct drv *drvs = NULL; /* First guest. */
@@ -260,6 +275,21 @@ main (int argc, char *argv[])
} else if (STREQ (long_options[option_index].name, "xattr") ||
STREQ (long_options[option_index].name, "xattrs")) {
enable_xattrs = 1;
+ } else if (STREQ (long_options[option_index].name, "compare-extra-stats")) {
+ if (STREQ (optarg, "no"))
+ compare_extra_stats = 0;
+ } else if (STREQ (long_options[option_index].name, "compare-perms")) {
+ if (STREQ (optarg, "no"))
+ compare_perms = 0;
+ } else if (STREQ (long_options[option_index].name, "compare-times")) {
+ if (STREQ (optarg, "no"))
+ compare_times = 0;
+ } else if (STREQ (long_options[option_index].name, "compare-uids")) {
+ if (STREQ (optarg, "no"))
+ compare_uids = 0;
+ } else if (STREQ (long_options[option_index].name, "compare-xattrs")) {
+ if (STREQ (optarg, "no"))
+ compare_xattrs = 0;
} else {
fprintf (stderr, _("%s: unknown long option: %s (%d)\n"),
program_name, long_options[option_index].name, option_index);
@@ -404,6 +434,8 @@ struct file {
char *path;
struct guestfs_statns *stat;
struct guestfs_xattr_list *xattrs;
+ struct guestfs_statns *stat_orig;
+ struct guestfs_xattr_list *xattrs_orig;
char *csum; /* Checksum. If NULL, use file times and size. */
};
@@ -416,6 +448,8 @@ free_tree (struct tree *t)
free (t->files[i].path);
guestfs_free_statns (t->files[i].stat);
guestfs_free_xattr_list (t->files[i].xattrs);
+ guestfs_free_statns (t->files[i].stat_orig);
+ guestfs_free_xattr_list (t->files[i].xattrs_orig);
free (t->files[i].csum);
}
@@ -466,6 +500,8 @@ visit_entry (const char *dir, const char *name,
char *path = NULL, *csum = NULL;
struct guestfs_statns *stat = NULL;
struct guestfs_xattr_list *xattrs = NULL;
+ struct guestfs_statns *stat_copy = NULL;
+ struct guestfs_xattr_list *xattrs_copy = NULL;
size_t i;
path = full_path (dir, name);
@@ -474,16 +510,31 @@ visit_entry (const char *dir, const char *name,
* free them after we return.
*/
stat = guestfs_copy_statns (stat_orig);
+ stat_copy = guestfs_copy_statns (stat_orig);
if (stat == NULL) {
perror ("guestfs_copy_stat");
goto error;
}
+ if (!compare_perms)
+ stat->st_mode &= 0170000;
+
+ if (!compare_extra_stats)
+ stat->st_dev = stat->st_ino = stat->st_nlink = stat->st_rdev =
+ stat->st_blocks = 0;
+
+ if (!compare_uids)
+ stat->st_uid = stat->st_gid = 0;
+
+ if (!compare_times)
+ stat->st_atime_sec = stat->st_mtime_sec = stat->st_ctime_sec =
+ stat->st_atime_nsec = stat->st_mtime_nsec = stat->st_ctime_nsec = 0;
+
xattrs = guestfs_copy_xattr_list (xattrs_orig);
+ xattrs_copy = guestfs_copy_xattr_list (xattrs_orig);
if (xattrs == NULL) {
perror ("guestfs_copy_xattr_list");
goto error;
}
-
if (checksum && is_reg (stat->st_mode)) {
csum = guestfs_checksum (t->g, checksum, path);
if (!csum)
@@ -534,6 +585,8 @@ visit_entry (const char *dir, const char *name,
t->files[i].stat = stat;
t->files[i].xattrs = xattrs;
t->files[i].csum = csum;
+ t->files[i].stat_orig = stat_copy;
+ t->files[i].xattrs_orig = xattrs_copy;
return 0;
@@ -631,10 +684,11 @@ compare_stats (struct file *file1, struct file *file2)
if (r != 0)
return r;
- r = guestfs_compare_xattr_list (file1->xattrs, file2->xattrs);
- if (r != 0)
- return r;
-
+ if (compare_xattrs) {
+ r = guestfs_compare_xattr_list (file1->xattrs, file2->xattrs);
+ if (r != 0)
+ return r;
+ }
return 0;
}
@@ -778,29 +832,29 @@ output_file (guestfs_h *g, struct file *file)
filetype = "u";
output_string (filetype);
- output_int64_perms (file->stat->st_mode & 07777);
+ output_int64_perms (file->stat_orig->st_mode & 07777);
- output_int64_size (file->stat->st_size);
+ output_int64_size (file->stat_orig->st_size);
/* Display extra fields when enabled. */
if (enable_uids) {
- output_int64_uid (file->stat->st_uid);
- output_int64_uid (file->stat->st_gid);
+ output_int64_uid (file->stat_orig->st_uid);
+ output_int64_uid (file->stat_orig->st_gid);
}
if (enable_times) {
if (atime)
- output_int64_time (file->stat->st_atime_sec, file->stat->st_atime_nsec);
- output_int64_time (file->stat->st_mtime_sec, file->stat->st_mtime_nsec);
- output_int64_time (file->stat->st_ctime_sec, file->stat->st_ctime_nsec);
+ output_int64_time (file->stat_orig->st_atime_sec, file->stat_orig->st_atime_nsec);
+ output_int64_time (file->stat_orig->st_mtime_sec, file->stat_orig->st_mtime_nsec);
+ output_int64_time (file->stat_orig->st_ctime_sec, file->stat_orig->st_ctime_nsec);
}
if (enable_extra_stats) {
- output_int64_dev (file->stat->st_dev);
- output_int64 (file->stat->st_ino);
- output_int64 (file->stat->st_nlink);
- output_int64_dev (file->stat->st_rdev);
- output_int64 (file->stat->st_blocks);
+ output_int64_dev (file->stat_orig->st_dev);
+ output_int64 (file->stat_orig->st_ino);
+ output_int64 (file->stat_orig->st_nlink);
+ output_int64_dev (file->stat_orig->st_rdev);
+ output_int64 (file->stat_orig->st_blocks);
}
if (file->csum)
@@ -816,10 +870,10 @@ output_file (guestfs_h *g, struct file *file)
}
if (enable_xattrs) {
- for (i = 0; i < file->xattrs->len; ++i) {
- output_string (file->xattrs->val[i].attrname);
- output_binary (file->xattrs->val[i].attrval,
- file->xattrs->val[i].attrval_len);
+ for (i = 0; i < file->xattrs_orig->len; ++i) {
+ output_string (file->xattrs_orig->val[i].attrname);
+ output_binary (file->xattrs_orig->val[i].attrval,
+ file->xattrs_orig->val[i].attrval_len);
}
}
}
diff --git a/diff/virt-diff.pod b/diff/virt-diff.pod
index e1d67f3..898155d 100644
--- a/diff/virt-diff.pod
+++ b/diff/virt-diff.pod
@@ -214,6 +214,26 @@ Enable tracing of libguestfs API calls.
Display extended attributes.
+=item B<--compare-extra-stats (yes|no)>
+
+If set to no, when comparing, do not consider files different if extra stats (like inode number) differ. Default is yes.
+
+=item B<--compare-perms (yes|no)>
+
+If set to no, when comparing, do not consider files different if permissions differ. Default is yes.
+
+=item B<--compare-times (yes|no)>
+
+If set to no, when comparing, do not consider files different if any of the times differ. Default is yes.
+
+=item B<--compare-uids (yes|no)>
+
+If set to no, when comparing, do not consider files different if uid and gid differ. Default is yes.
+
+=item B<--compare-xattrs (yes|no)>
+
+If set to no, when comparing, do not consider files different if extended attributes differ. Default is yes.
+
=back
=head1 NOTE ABOUT CSV FORMAT
--
1.9.3
9 years, 11 months
