July 2017 - Libguestfs - libguestfs List Archives

[PATCH] common/mlstdutils: Always build bytecode version of this library.
by Richard W.M. Jones 13 Jul '17

13 Jul '17

Even if ocamlopt is available, always build a bytecode version of ‘common/mlstdutils’. The generator is always built as bytecode. The generator depends on ‘../common/mlstdutils/guestfs_config.cmo’ and ‘../common/mlstdutils/std_utils.cmo’, but if these have not been built already then the generator races to build the .cmi files. Since the generator doesn't have the dependencies covering for the ‘common/mlstdutils’ directory you can get a broken link on fast machines: File "../common/mlstdutils/std_utils.ml", line 1: Error: Corrupted compiled interface ../common/mlstdutils/guestfs_config.cmi make[2]: *** [Makefile:1993: ../common/mlstdutils/std_utils.cmo] Error 2 make[2]: *** Waiting for unfinished jobs.... make[2]: Leaving directory '/builddir/build/BUILD/libguestfs-1.37.17/generator' --- common/mlstdutils/Makefile.am | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/common/mlstdutils/Makefile.am b/common/mlstdutils/Makefile.am index 9e0b34d42..d965f433e 100644 --- a/common/mlstdutils/Makefile.am +++ b/common/mlstdutils/Makefile.am @@ -48,7 +48,9 @@ else MLSTDUTILS_CMA = mlstdutils.cmxa endif -noinst_DATA = $(MLSTDUTILS_CMA) +# Just for this library, we need to build both bytecode and native +# code because the generator always requires the bytecode version. +noinst_DATA = mlstdutils.cma mlstdutils.cmxa libmlstdutils_a_SOURCES = dummy.c libmlstdutils_a_CPPFLAGS = \ @@ -79,8 +81,11 @@ endif libmlstdutils_a_DEPENDENCIES = $(OBJECTS) -$(MLSTDUTILS_CMA): $(OBJECTS) - $(OCAMLFIND) mklib $(OCAMLPACKAGES) $(OBJECTS) -o mlstdutils +mlstdutils.cma: $(BOBJECTS) + $(OCAMLFIND) mklib $(OCAMLPACKAGES) $^ -o mlstdutils + +mlstdutils.cmxa: $(XOBJECTS) + $(OCAMLFIND) mklib $(OCAMLPACKAGES) $^ -o mlstdutils # This OCaml module has to be generated by make (configure will put # unexpanded prefix macro in). -- 2.13.2

2 1

compression method for qcow2?
by 陳培泓 13 Jul '17

13 Jul '17

I found guestfs_compress_out do the compression to gzip, bzip2, compress...etc. I want to compress qcow2, and I found details about qcow2 compression in https://people.gnome.org/~markmc/qcow-image-format.html The QCOW format supports compression by allowing each cluster to be > independently compressed with zlib. Can I do that with guestfs_compress_out?

2 1

ATTN packagers: Fix to supermin is required for Linux kernel >= 4.13
by Richard W.M. Jones 13 Jul '17

13 Jul '17

A fix to supermin is required if you want to use supermin and/or libguestfs with Linux kernel >= 4.13. It does not affect the recently released kernel 4.12. Supermin incorrectly created symlinks in the appliance filesystem, and although this works with earlier kernels a recent change breaks these (incorrect) filesystems. The symptoms you will see are described in this bug report: https://bugzilla.redhat.com/show_bug.cgi?id=1470157 The fix is included in supermin 5.1.18: http://download.libguestfs.org/supermin/ If you prefer to backport the commit which is fairly small and well-contained, it is: https://github.com/libguestfs/supermin/commit/158854e3ba4be7f6b8d81f662ddad… There is no need to upgrade libguestfs. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org

1 0

[PATCH supermin v2] ext2: Create symlinks properly (RHBZ#1470157).
by Richard W.M. Jones 13 Jul '17

13 Jul '17

The ext2 filesystem on disk format has two ways to store symlinks. For symlinks >= 60 bytes in length, they are stored as files (so-called "slow symlinks"). For shorter symlinks the symlink is stored in the inode ("fast symlinks"). Previously we only created slow symlinks even if they are shorter than 60 bytes. This didn't matter until recently, when a change went into the upstream kernel which assumes that symlinks shorter than 60 bytes are always stored in the inode, thus breaking the filesystems that we created before: https://bugzilla.redhat.com/show_bug.cgi?id=1470157#c4 This changes the code to use the ext2fs_symlink function instead which creates fast and slow symlinks properly. It also removes use of PATH_MAX so this should work with symbolic link targets of any length. This fix is required if you use supermin with any Linux kernel >= 4.13. Thanks: Eric Sandeen --- src/ext2fs-c.c | 59 ++++++++++++---------------------------------------------- 1 file changed, 12 insertions(+), 47 deletions(-) diff --git a/src/ext2fs-c.c b/src/ext2fs-c.c index 2743da7..e8ab972 100644 --- a/src/ext2fs-c.c +++ b/src/ext2fs-c.c @@ -191,7 +191,6 @@ supermin_ext2fs_read_bitmaps (value fsv) static void ext2_mkdir (ext2_filsys fs, ext2_ino_t dir_ino, const char *dirname, const char *basename, mode_t mode, uid_t uid, gid_t gid, time_t ctime, time_t atime, time_t mtime); static void ext2_empty_inode (ext2_filsys fs, ext2_ino_t dir_ino, const char *dirname, const char *basename, mode_t mode, uid_t uid, gid_t gid, time_t ctime, time_t atime, time_t mtime, int major, int minor, int dir_ft, ext2_ino_t *ino_ret); -static void ext2_write_file (ext2_filsys fs, ext2_ino_t ino, const char *buf, size_t size, const char *filename); static void ext2_write_host_file (ext2_filsys fs, ext2_ino_t ino, const char *src, const char *filename); static void ext2_link (ext2_filsys fs, ext2_ino_t dir_ino, const char *basename, ext2_ino_t ino, int dir_ft); static void ext2_clean_path (ext2_filsys fs, ext2_ino_t dir_ino, const char *dirname, const char *basename, int isdir); @@ -468,49 +467,9 @@ ext2_empty_inode (ext2_filsys fs, *ino_ret = ino; } -/* You must create the file first with ext2_empty_inode. */ -static void -ext2_write_file (ext2_filsys fs, - ext2_ino_t ino, const char *buf, size_t size, - const char *filename) -{ - errcode_t err; - ext2_file_t file; - err = ext2fs_file_open2 (fs, ino, NULL, EXT2_FILE_WRITE, &file); - if (err != 0) - ext2_error_to_exception ("ext2fs_file_open2", err, filename); - - /* ext2fs_file_write cannot deal with partial writes. You have - * to write the entire file in a single call. - */ - unsigned int written; - err = ext2fs_file_write (file, buf, size, &written); - if (err != 0) - ext2_error_to_exception ("ext2fs_file_write", err, filename); - if ((size_t) written != size) - caml_failwith ("ext2fs_file_write: file size != bytes written"); - - err = ext2fs_file_flush (file); - if (err != 0) - ext2_error_to_exception ("ext2fs_file_flush", err, filename); - err = ext2fs_file_close (file); - if (err != 0) - ext2_error_to_exception ("ext2fs_file_close", err, filename); - - /* Update the true size in the inode. */ - struct ext2_inode inode; - err = ext2fs_read_inode (fs, ino, &inode); - if (err != 0) - ext2_error_to_exception ("ext2fs_read_inode", err, filename); - inode.i_size = size; - err = ext2fs_write_inode (fs, ino, &inode); - if (err != 0) - ext2_error_to_exception ("ext2fs_write_inode", err, filename); -} - -/* Same as ext2_write_file, but it copies the file contents from the - * host. You must create the file first with ext2_empty_inode, and - * the host file must be a regular file. +/* Copies the file contents from the host. You must create the file + * first with ext2_empty_inode, and the host file must be a regular + * file. */ static void ext2_write_host_file (ext2_filsys fs, @@ -829,11 +788,17 @@ ext2_copy_file (struct ext2_data *data, const char *src, const char *dest) statbuf.st_ctime, statbuf.st_atime, statbuf.st_mtime, 0, 0, EXT2_FT_SYMLINK, &ino); - char buf[PATH_MAX+1]; - ssize_t r = readlink (src, buf, sizeof buf); + char *buf = malloc (statbuf.st_size+1); + if (buf == NULL) + caml_raise_out_of_memory (); + ssize_t r = readlink (src, buf, statbuf.st_size); if (r == -1) unix_error (errno, (char *) "readlink", caml_copy_string (src)); - ext2_write_file (data->fs, ino, buf, r, dest); + if (r > statbuf.st_size) + r = statbuf.st_size; + buf[r] = '\0'; + ext2fs_symlink (data->fs, dir_ino, ino, dest, buf); + free (buf); } /* Create directory. */ else if (S_ISDIR (statbuf.st_mode)) -- 2.13.1

3 3

[PATCH supermin] ext2: Create symlinks properly (RHBZ#1470157).
by Richard W.M. Jones 13 Jul '17

13 Jul '17

The ext2 filesystem on disk format has two ways to store symlinks. For symlinks >= 60 bytes in length, they are stored as files (so-called "slow symlinks"). For shorter symlinks the symlink is stored in the inode ("fast symlinks"). Previously we only created slow symlinks even if there are shorter than 60 bytes. This didn't matter until recently, when a change went upstream which assumes that symlinks shorter than 60 bytes are always stored in the inode, thus breaking the filesystems that we created before: https://bugzilla.redhat.com/show_bug.cgi?id=1470157#c4 This changes the code to use the ext2fs_symlink function instead which creates fast and slow symlinks properly. This fix is required if you use supermin with any Linux kernel >= 4.13. The actual fix is two lines replacing ext2_write_file with a simpler call to ext2fs_symlink. The majority of this fix is removing the ext2_write_file function which is no longer referenced after that change. Thanks: Eric Sandeen --- src/ext2fs-c.c | 50 +++++--------------------------------------------- 1 file changed, 5 insertions(+), 45 deletions(-) diff --git a/src/ext2fs-c.c b/src/ext2fs-c.c index 2743da7..0b3b29e 100644 --- a/src/ext2fs-c.c +++ b/src/ext2fs-c.c @@ -191,7 +191,6 @@ supermin_ext2fs_read_bitmaps (value fsv) static void ext2_mkdir (ext2_filsys fs, ext2_ino_t dir_ino, const char *dirname, const char *basename, mode_t mode, uid_t uid, gid_t gid, time_t ctime, time_t atime, time_t mtime); static void ext2_empty_inode (ext2_filsys fs, ext2_ino_t dir_ino, const char *dirname, const char *basename, mode_t mode, uid_t uid, gid_t gid, time_t ctime, time_t atime, time_t mtime, int major, int minor, int dir_ft, ext2_ino_t *ino_ret); -static void ext2_write_file (ext2_filsys fs, ext2_ino_t ino, const char *buf, size_t size, const char *filename); static void ext2_write_host_file (ext2_filsys fs, ext2_ino_t ino, const char *src, const char *filename); static void ext2_link (ext2_filsys fs, ext2_ino_t dir_ino, const char *basename, ext2_ino_t ino, int dir_ft); static void ext2_clean_path (ext2_filsys fs, ext2_ino_t dir_ino, const char *dirname, const char *basename, int isdir); @@ -468,49 +467,9 @@ ext2_empty_inode (ext2_filsys fs, *ino_ret = ino; } -/* You must create the file first with ext2_empty_inode. */ -static void -ext2_write_file (ext2_filsys fs, - ext2_ino_t ino, const char *buf, size_t size, - const char *filename) -{ - errcode_t err; - ext2_file_t file; - err = ext2fs_file_open2 (fs, ino, NULL, EXT2_FILE_WRITE, &file); - if (err != 0) - ext2_error_to_exception ("ext2fs_file_open2", err, filename); - - /* ext2fs_file_write cannot deal with partial writes. You have - * to write the entire file in a single call. - */ - unsigned int written; - err = ext2fs_file_write (file, buf, size, &written); - if (err != 0) - ext2_error_to_exception ("ext2fs_file_write", err, filename); - if ((size_t) written != size) - caml_failwith ("ext2fs_file_write: file size != bytes written"); - - err = ext2fs_file_flush (file); - if (err != 0) - ext2_error_to_exception ("ext2fs_file_flush", err, filename); - err = ext2fs_file_close (file); - if (err != 0) - ext2_error_to_exception ("ext2fs_file_close", err, filename); - - /* Update the true size in the inode. */ - struct ext2_inode inode; - err = ext2fs_read_inode (fs, ino, &inode); - if (err != 0) - ext2_error_to_exception ("ext2fs_read_inode", err, filename); - inode.i_size = size; - err = ext2fs_write_inode (fs, ino, &inode); - if (err != 0) - ext2_error_to_exception ("ext2fs_write_inode", err, filename); -} - -/* Same as ext2_write_file, but it copies the file contents from the - * host. You must create the file first with ext2_empty_inode, and - * the host file must be a regular file. +/* Copies the file contents from the host. You must create the file + * first with ext2_empty_inode, and the host file must be a regular + * file. */ static void ext2_write_host_file (ext2_filsys fs, @@ -833,7 +792,8 @@ ext2_copy_file (struct ext2_data *data, const char *src, const char *dest) ssize_t r = readlink (src, buf, sizeof buf); if (r == -1) unix_error (errno, (char *) "readlink", caml_copy_string (src)); - ext2_write_file (data->fs, ino, buf, r, dest); + buf[r] = '\0'; + ext2fs_symlink (data->fs, dir_ino, ino, dest, buf); } /* Create directory. */ else if (S_ISDIR (statbuf.st_mode)) -- 2.13.1

3 3

[virt-v2v] no installed kernel packages were found Converting CentOS OVA to KVM
by Zach Seils (seils) 12 Jul '17

12 Jul '17

While trying to convert an OVA exported from VMware vSphere to libvirt using virt-v2v, I get the following error: virt-v2v: error: no installed kernel packages were found. Environment info: Base OS: 16.04.2 LTS (Xenial Xerus) Source VM OS: CentOS 7 libguestfs: 1.37.16 (built from source) Relevant logs are attached. Thanks, Zach

3 6

any read/write raw image function or API in Python?
by 陳培泓 12 Jul '17

12 Jul '17

I know there's lots of functions that can access/modify the internal fs of img or qcow2 through libguestfs. I want to know is there any library can modify the raw/qcow2 image directly in Python? For example, there's function called read_qcow2(), and I can know the content of tmp.qcow2 through the function read_qcow2().

3 3

questions about python examples
by 陳培泓 12 Jul '17

12 Jul '17

I followed the examples 1 and 2 in http://libguestfs.org/guestfs-python.3.html I create disk img by example 1 and use example 2 to inspect the img. I got error in the line: > g.inspect_os() : no operating systems found. how could fix it? any examples to create os in python?

2 1

[PATCH 1/2] builder: fix paths to mlstdutils & mlutils
by Pino Toscano 11 Jul '17

11 Jul '17

Followup/fix of commit 61d4891ef48df171a27873efe90aab51a9b711ef. --- builder/Makefile.am | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/builder/Makefile.am b/builder/Makefile.am index 09ae4ae..e64c899 100644 --- a/builder/Makefile.am +++ b/builder/Makefile.am @@ -311,7 +311,7 @@ depend: .depend .depend: $(wildcard $(abs_srcdir)/*.mli) $(wildcard $(abs_srcdir)/*.ml) rm -f $@ $@-t - $(OCAMLFIND) ocamldep -I ../ocaml -I $(abs_srcdir) -I $(abs_top_builddir)/mlstdutils -I $(abs_top_builddir)/mlutils -I $(abs_top_builddir)/mllib -I $(abs_top_builddir)/customize $^ | \ + $(OCAMLFIND) ocamldep -I ../ocaml -I $(abs_srcdir) -I $(abs_top_builddir)/common/mlstdutils -I $(abs_top_builddir)/common/mlutils -I $(abs_top_builddir)/mllib -I $(abs_top_builddir)/customize $^ | \ $(SED) 's/ *$$//' | \ $(SED) -e :a -e '/ *\\$$/N; s/ *\\\n */ /; ta' | \ $(SED) -e 's,$(abs_srcdir)/,$(builddir)/,g' | \ -- 2.9.4

2 2

[PATCH v3 0/5] threads: Add support for thread-safe handle.
by Richard W.M. Jones 11 Jul '17

11 Jul '17

Previously posted in 2015: v1: https://www.redhat.com/archives/libguestfs/2015-June/msg00048.html v2: https://www.redhat.com/archives/libguestfs/2015-June/msg00118.html I have rebased and tidied up the patches, fixing a few spelling mistakes, but they are broadly the same as before. I also ran all the tests, which pass. As with the previous versions, this makes a change to the API, where you can no longer pass a handle between threads and expect guestfs_last_error() to work. I'm somewhat more sanguine about this change, since using the API like that is abstruse and no one should be expecting that to have worked. Rich.

3 11