[PATCH] builder: small code simplification
by Pino Toscano
No actual behaviour changes, just remove extra match statements.
---
builder/list_entries.ml | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)
diff --git a/builder/list_entries.ml b/builder/list_entries.ml
index 742e43b..0d3d9b2 100644
--- a/builder/list_entries.ml
+++ b/builder/list_entries.ml
@@ -98,13 +98,11 @@ and list_entries_json ~sources index =
| c -> String.make 1 c)
done;
!res in
- let json_optional_printf_string key value =
- match value with
+ let json_optional_printf_string key = function
| None -> ()
| Some str ->
printf " \"%s\": \"%s\",\n" key (json_string_escape str) in
- let json_optional_printf_int64 key value =
- match value with
+ let json_optional_printf_int64 key = function
| None -> ()
| Some n ->
printf " \"%s\": \"%Ld\",\n" key n in
--
1.8.3.1
10 years, 10 months
[PATCH] builder: fix small regression in subkey parsing
by Pino Toscano
Introduced in 5cbdf35d651b6c730d62d9af4876039faa122efc, it caused the
first character of the value to be skipped if the key has a subkey.
---
builder/index-scan.l | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/builder/index-scan.l b/builder/index-scan.l
index 7a9618f..e43f82e 100644
--- a/builder/index-scan.l
+++ b/builder/index-scan.l
@@ -65,7 +65,7 @@ extern void yyerror (const char *);
if (yytext[i] == '[') {
size_t j = strcspn (yytext+i+1, "]");
yylval.field->subkey = strndup (yytext+i+1, j);
- i += 1+j+2;
+ i += 1+j+1;
} else {
yylval.field->subkey = NULL;
}
--
1.8.3.1
10 years, 10 months
virt-builder & virt-sysprep: Avoiding SELinux relabelling
by Richard W.M. Jones
A common problem that people have with virt-builder and virt-sysprep
is which guests that use SELinux, like Fedora and RHEL. In both cases
we touch /.autorelabel in the guest, which means the guest has to
reboot once during its first boot.
Recap: SELinux file labels
--------------------------
SELinux requires that files have labels. Access to a file is
controlled by the label on that file.
For example:
$ ls -lZ /etc/passwd
-rw-r--r--. root root system_u:object_r:passwd_file_t:s0 /etc/passwd
<----------- label ----------->
The SELinux policy allows (eg) /usr/sbin/passwd to write to this file,
based on the process label and the target file label. (The path
"/etc/passwd" is not considered.)
On a running system, files created from scratch are labelled by a
daemon (restorecond, I think). Also they could be labelled by
programs that create files such as 'rpm'.
The label is actually stored in an extended attribute (xattr):
$ getfattr -d --match=.* /etc/passwd
getfattr: Removing leading '/' from absolute path names
# file: etc/passwd
security.selinux="system_u:object_r:passwd_file_t:s0"
Libguestfs & SELinux labelling
------------------------------
Unfortunately SELinux requires a policy to be loaded into the kernel
in order to know how to label files. There is no default labelling
scheme for files, it depends entirely on the policy.
Since the policy is guest specific, libguestfs cannot really load it
(for a start, it wouldn't necessarily know where to load it from, and
it might not be safe to load an untrusted policy).
However libguestfs can do one of four things:
(1) It can set the security.selinux xattr to some value. It has to
know or be able to guess what value to set it to. Note there is
no policy, so we have to infer this based on other things we know
about the guest such as inspection data.
(2) It can copy the security.selinux xattr from another file.
(3) It can update the file in such a way that the xattr doesn't
change, eg. appending to the file.
(4) It can touch '/.autorelabel' which causes an SELinux enabled guest
to do a full filesystem relabel at first boot (followed by a
reboot).
Virt-sysprep labelling
----------------------
As you may have guessed, currently virt-sysprep takes the easy option
(4) above. Virt-sysprep tracks if any file has been created, and if
so and the guest looks like an SELinux distro, it touches
'/.autorelabel'.
Since the 'hostname' operation is (a) enabled by default and (b)
always creates the hostname file with default content even if the
'virt-sysprep --hostname' option was not used, almost any
SELinux-enabled guest that virt-sysprep touches is going to have
/.autorelabel at the end.
Virt-builder labelling
----------------------
Virt-builder does even less than virt-sysprep. In particular it
doesn't touch /.autorelabel. However the prepared templates that we
ship with virt-builder were all virt-sysprepped before being uploaded,
and so all have /.autorelabel in them.
What files virt-sysprep creates or modifies
-------------------------------------------
This is my analysis of what files virt-sysprep creates or modifies:
firstboot:
- creates scripts and more (brand new files)
- shared with virt-builder
reconfiguration:
- creates /.unconfigured (brand new)
hostname:
- creates various files depending on distro (eg /etc/hostname)
- files probably exist already, but might not
- shared with virt-builder
machine_id:
- creates empty /etc/machine-id
- probably this file exists already
net_hostname:
- modifies /etc/sysconfig/network-scripts/ifcfg-*
net_hwaddr:
- modifies /etc/sysconfig/network-scripts/ifcfg-*
pacct_log:
- touches /var/account/pacct or /var/log/account/pacct
password:
- edits /etc/shadow directly
- shared with virt-builder
random_seed:
- creates seed file
- seed file may or may not exist before
- shared with virt-builder
timezone:
- creates/updates /etc/localtime symlink
- file almost certainly exists before, but might be a regular file
- shared with virt-builder
user_account:
- uses Augeas to edit /etc/passwd, /etc/shadow, /etc/group
- Augeas does not preserve or set SELinux labels [in this case]
What files virt-builder creates or modifies
-------------------------------------------
edit option:
- edits a file
- copies xattrs from old file to new file (thanks Pino!)
link, mkdir, upload, write options:
- creates new links/directories/files under user control
- currently no way to specify the SELinux label of the new file
- would user know how to specify it?
install, update options:
- complicated: see longer section below
Plus, shared modules with virt-sysprep, see notes above:
- firstboot
- hostname
- password
- random_seed
- timezone
Virt-builder --install and --update options
-------------------------------------------
These options allow you to install new packages, or update existing
packages. They run 'yum' in the context of the libguestfs appliance.
The appliance normally has SELinux disabled and no policy loaded.
It is plausible that RPM packages could contain the right SELinux
labels, and that yum could be setting labels correctly (by setting the
security.selinux xattr) even though SELinux is disabled.
However I could not say for sure, so I did the following test instead:
$ virt-builder fedora-20 --install /usr/sbin/tcpdump
$ guestfish --ro -a fedora-20.img -i
><fs> getxattrs /usr/sbin/tcpdump
[no output]
It seems from this that RPM is not setting labels, instead it relies
on restorecond/etc to do the labelling of files that it creates.
Conclusion
----------
I don't think there is any plausible way we can make --install or
--update work without requiring a full filesystem relabel on first
boot.
However I think we can try harder to copy attributes from old files to
new files, which will make many cases above work (especially for
virt-sysprep which seems tractable).
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
libguestfs lets you edit virtual machines. Supports shell scripting,
bindings from many languages. http://libguestfs.org
10 years, 10 months
[PATCH] builder: proper consider subkeys in index files
by Pino Toscano
The index files already allowed the 'key[subkey]=...' syntax for keys,
but considering such string as whole key. Proper split the parsing and
the handling of the subkeys, so they can be searched a bit easier.
This causes no actual behaviour changes, it is just internal
micro-refactoring.
---
builder/index-parser-c.c | 8 +++++---
builder/index-scan.l | 9 ++++++++-
builder/index-struct.c | 1 +
builder/index-struct.h | 1 +
builder/index_parser.ml | 44 ++++++++++++++++++++++++++------------------
5 files changed, 41 insertions(+), 22 deletions(-)
diff --git a/builder/index-parser-c.c b/builder/index-parser-c.c
index 17e680b..baf2b62 100644
--- a/builder/index-parser-c.c
+++ b/builder/index-parser-c.c
@@ -83,11 +83,13 @@ virt_builder_parse_index (value filenamev)
for (j = 0, fields = sections->fields; fields != NULL;
j++, fields = fields->next) {
- v = caml_alloc_tuple (2);
+ v = caml_alloc_tuple (3);
sv = caml_copy_string (fields->key);
- Store_field (v, 0, sv); /* (key, value) */
- sv = caml_copy_string (fields->value);
+ Store_field (v, 0, sv); /* (key, subkey, value) */
+ sv = caml_copy_string (fields->subkey ? fields->subkey : "");
Store_field (v, 1, sv);
+ sv = caml_copy_string (fields->value);
+ Store_field (v, 2, sv);
Store_field (fv, j, v); /* assign to return array of fields */
}
diff --git a/builder/index-scan.l b/builder/index-scan.l
index 9a6a0e3..7a9618f 100644
--- a/builder/index-scan.l
+++ b/builder/index-scan.l
@@ -58,10 +58,17 @@ extern void yyerror (const char *);
/* field=value or field[subfield]=value */
^[A-Za-z0-9_.]+("["[A-Za-z0-9_,.]+"]")?"=".*\n {
- size_t i = strcspn (yytext, "=");
+ size_t i = strcspn (yytext, "=[");
yylval.field = malloc (sizeof (struct field));
yylval.field->next = NULL;
yylval.field->key = strndup (yytext, i);
+ if (yytext[i] == '[') {
+ size_t j = strcspn (yytext+i+1, "]");
+ yylval.field->subkey = strndup (yytext+i+1, j);
+ i += 1+j+2;
+ } else {
+ yylval.field->subkey = NULL;
+ }
/* Note we chop the final \n off here. */
yylval.field->value = strndup (yytext+i+1, yyleng-(i+2));
return FIELD;
diff --git a/builder/index-struct.c b/builder/index-struct.c
index 26bed24..fe5b0e3 100644
--- a/builder/index-struct.c
+++ b/builder/index-struct.c
@@ -52,6 +52,7 @@ free_field (struct field *field)
if (field) {
free_field (field->next);
free (field->key);
+ free (field->subkey);
free (field->value);
free (field);
}
diff --git a/builder/index-struct.h b/builder/index-struct.h
index ac8a3dd..f92e01d 100644
--- a/builder/index-struct.h
+++ b/builder/index-struct.h
@@ -32,6 +32,7 @@ struct section {
struct field {
struct field *next;
char *key;
+ char *subkey;
char *value;
};
diff --git a/builder/index_parser.ml b/builder/index_parser.ml
index 453a3a1..4b070cb 100644
--- a/builder/index_parser.ml
+++ b/builder/index_parser.ml
@@ -101,7 +101,7 @@ let print_entry chan (name, { printable_name = printable_name;
type sections = section array
and section = string * fields (* [name] + fields *)
and fields = field array
-and field = string * string (* key + value *)
+and field = string * string * string (* key + subkey + value *)
(* Calls yyparse in the C code. *)
external parse_index : string -> sections = "virt_builder_parse_index"
@@ -149,12 +149,13 @@ let get_index ~prog ~debug ~downloader ~sigchecker source =
fun (n, fields) ->
let fseen = Hashtbl.create 13 in
List.iter (
- fun (field, _) ->
- if Hashtbl.mem fseen field then (
+ fun (field, subkey, _) ->
+ let hashkey = (field, subkey) in
+ if Hashtbl.mem fseen hashkey then (
eprintf (f_"virt-builder: index is corrupt: %s: field '%s' appears two or more times\n") n field;
corrupt_file ()
);
- Hashtbl.add fseen field true
+ Hashtbl.add fseen hashkey true
) fields
) sections;
@@ -162,25 +163,32 @@ let get_index ~prog ~debug ~downloader ~sigchecker source =
let entries =
List.map (
fun (n, fields) ->
+ let find_elem key subkey fields =
+ match List.filter (
+ fun (iterkey, itersubkey, itervalue) ->
+ iterkey = key && itersubkey = subkey
+ ) fields with
+ | [] -> raise Not_found
+ | (_, _, value) :: _ -> value in
let printable_name =
- try Some (List.assoc "name" fields) with Not_found -> None in
+ try Some (find_elem "name" "" fields) with Not_found -> None in
let osinfo =
- try Some (List.assoc "osinfo" fields) with Not_found -> None in
+ try Some (find_elem "osinfo" "" fields) with Not_found -> None in
let file_uri =
- try make_absolute_uri (List.assoc "file" fields)
+ try make_absolute_uri (find_elem "file" "" fields)
with Not_found ->
eprintf (f_"virt-builder: no 'file' (URI) entry for '%s'\n") n;
corrupt_file () in
let signature_uri =
- try Some (make_absolute_uri (List.assoc "sig" fields))
+ try Some (make_absolute_uri (find_elem "sig" "" fields))
with Not_found -> None in
let checksum_sha512 =
- try Some (List.assoc "checksum[sha512]" fields)
+ try Some (find_elem "checksum" "sha512" fields)
with Not_found ->
- try Some (List.assoc "checksum" fields)
+ try Some (find_elem "checksum" "" fields)
with Not_found -> None in
let revision =
- try int_of_string (List.assoc "revision" fields)
+ try int_of_string (find_elem "revision" "" fields)
with
| Not_found -> 1
| Failure "int_of_string" ->
@@ -188,9 +196,9 @@ let get_index ~prog ~debug ~downloader ~sigchecker source =
n;
corrupt_file () in
let format =
- try Some (List.assoc "format" fields) with Not_found -> None in
+ try Some (find_elem "format" "" fields) with Not_found -> None in
let size =
- try Int64.of_string (List.assoc "size" fields)
+ try Int64.of_string (find_elem "size" "" fields)
with
| Not_found ->
eprintf (f_"virt-builder: no 'size' field for '%s'\n") n;
@@ -200,7 +208,7 @@ let get_index ~prog ~debug ~downloader ~sigchecker source =
n;
corrupt_file () in
let compressed_size =
- try Some (Int64.of_string (List.assoc "compressed_size" fields))
+ try Some (Int64.of_string (find_elem "compressed_size" "" fields))
with
| Not_found ->
None
@@ -209,13 +217,13 @@ let get_index ~prog ~debug ~downloader ~sigchecker source =
n;
corrupt_file () in
let expand =
- try Some (List.assoc "expand" fields) with Not_found -> None in
+ try Some (find_elem "expand" "" fields) with Not_found -> None in
let lvexpand =
- try Some (List.assoc "lvexpand" fields) with Not_found -> None in
+ try Some (find_elem "lvexpand" "" fields) with Not_found -> None in
let notes =
- try Some (List.assoc "notes" fields) with Not_found -> None in
+ try Some (find_elem "notes" "" fields) with Not_found -> None in
let hidden =
- try bool_of_string (List.assoc "hidden" fields)
+ try bool_of_string (find_elem "hidden" "" fields)
with
| Not_found -> false
| Failure "bool_of_string" ->
--
1.8.3.1
10 years, 10 months
[PATCH] builder: allow more empty lines in index files
by Pino Toscano
Improve the index grammar tp allow more than one empty line between
sections, and to allow any number of empty lines at the end of the file
(after the last section).
---
builder/index-parse.y | 12 +++++++++---
1 file changed, 9 insertions(+), 3 deletions(-)
diff --git a/builder/index-parse.y b/builder/index-parse.y
index f5e551b..a8d2f62 100644
--- a/builder/index-parse.y
+++ b/builder/index-parse.y
@@ -80,10 +80,10 @@ index:
{ parsed_index = $2; }
sections:
- section
+ section emptylines
{ $$ = $1; }
- | section EMPTY_LINE sections
- { $$ = $1; $$->next = $3; }
+ | section EMPTY_LINE emptylines sections
+ { $$ = $1; $$->next = $4; }
section:
SECTION_HEADER fields
@@ -113,6 +113,12 @@ continuations:
free ($1);
free ($2); }
+emptylines:
+ /* empty */
+ {}
+ | EMPTY_LINE emptylines
+ {}
+
%%
void
--
1.8.3.1
10 years, 10 months
[PATCH 1/2] sysprep: Update comments.
by Richard W.M. Jones
---
sysprep/sysprep_operation.mli | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/sysprep/sysprep_operation.mli b/sysprep/sysprep_operation.mli
index 61dde72..eb89db4 100644
--- a/sysprep/sysprep_operation.mli
+++ b/sysprep/sysprep_operation.mli
@@ -16,14 +16,16 @@
* 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
*)
-(** Structure used to describe sysprep operations. *)
+(** Defines the interface between the main program and sysprep operations. *)
val prog : string
type flag = [ `Created_files ]
type callback = Guestfs.guestfs -> string -> flag list
+(** [callback g root] is called to do work. *)
+(** Structure used to describe sysprep operations. *)
type operation = {
name : string;
(** Operation name, also used to enable the operation on the command
--
1.8.4.2
10 years, 10 months
Libguestfs can't launch with one of the disk images in the RHEV cluster
by Исаев Виталий Анатольевич
Dear Libguestfs developers,
I faced with an unexpected behavior of the quite old version of libguestfs (1.16.34) running on the RHEV 3.1 hypervisor which is part of the RHEV cluster with SCSI storage. My issue is described in the attachment.
Unfortunately, updating is not suitable for the project I'm working on due to the licensing and security reasons.
I am going to send a request to the RH Customer Support, but I prefer to ask the community first.
Thank you in advance.
Vitaly Isaev
Software engineer
Information security department
Fintech JSC, Moscow, Russia
10 years, 10 months
LVM mounting issue
by Исаев Виталий Анатольевич
Hello, It's me again with one more misunderstanding of libguestfs usage...
Guestfish (as well as python libguestfs module) refuse to mount logical volume containing the root partition of the VM's filesystem.
According to this short guide<http://rwmj.wordpress.com/2009/04/07/libguestfs-lvm-support/> I am trying to mount /dev/vg_kojit/lv_root as "/", but nothing works:
[root@rhevh1 integrity]#
[root@rhevh1 integrity]#
[root@rhevh1 integrity]#
[root@rhevh1 integrity]# guestfish
Welcome to guestfish, the libguestfs filesystem interactive shell for
editing virtual machine filesystems.
Type: 'help' for help on commands
'man' to read the manual
'quit' to quit the shell
><fs> add /dev/dm-40
><fs> run
><fs> inspect_os
><fs> pvs
/dev/vda2
unknown device
><fs> lvs
/dev/vg_kojit/lv_root
/dev/vg_kojit/lv_swap
><fs> mount /dev/vg_kojit/lv_root /
libguestfs: error: mount: mount_stub: /dev/vg_kojit/lv_root: No such file or directory
><fs>
><fs>
The actual information about VM's block devices can be seen on the screenshot attached to this message. I would be grateful for your help
Sincerely,
Vitaly Isaev
Software engineer
Information security department
Fintech JSC, Moscow, Russia
10 years, 10 months
newlines with write-append
by Olaf Hering
Silly bash scripts have stuff like below to get things done, but equally
silly guestfish scripts fail to add the required newline. Why is that?
echo "$dev1 $mnt1 $fs $opts 1 2" >> /etc/fstab
echo "$dev2 $mnt2 $fs $opts 1 2" >> /etc/fstab
write-append /etc/fstab "$dev1 $mnt1 $fs $opts 1 2" : \
write-append /etc/fstab "$dev2 $mnt2 $fs $opts 1 2" : \
Even adding variants of \n does not help.
Simple testcase:
write /etc/fstab "#\n" : \
write-append /etc/fstab '#1\n' : \
write-append /etc/fstab '#2\\n' : \
cat /etc/fstab : \
quit
...
libguestfs: trace: cat = "#\n#1\n#2\\n"
#\n#1\n#2\\n
libguestfs: trace: shutdown
...
I guess using "sh" will be my workaround.
Olaf
10 years, 10 months
