[libguestfs-common PATCH 0/2] recognize "/dev/mapper/VG-LV" with "--key"
https://bugzilla.redhat.com/show_bug.cgi?id=2168506 The commit message of the second patch explains the idea. The test suites in libguestfs, guestfs-tools and virt-v2v will have to be extended so they cover this feature; I'm soon going to post the virt-v2v update. I plan to cover the libguestfs and guestfs-tools test suites later. Thanks for reviewing, Laszlo Laszlo Ersek (2): options/keys: key_store_import_key(): un-constify "key" parameter options/keys: introduce unescape_device_mapper_lvm() options/keys.c | 88 +++++++++++++++++++- options/options.h | 3 +- 2 files changed, 89 insertions(+), 2 deletions(-)
The key_store_import_key() function is called from both C-language utilities -- via key_store_add_from_selector() -- and OCaml-language ones -- via guestfs_int_mllib_inspect_decrypt(). We currently declare the function's second parameter as const struct key_store_key *key That is however a superficial, if not outright false, promise: in key_store_import_key(), we take ownership of all three string fields within "key", as evidenced by free_key_store(), where we free() all three strings. With the same effort, we might as well modify the contents of those strings at once. Drop "const". (None of the callers care, but let's be honest.) Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2168506 Signed-off-by: Laszlo Ersek <lersek@redhat.com> --- options/options.h | 3 ++- options/keys.c | 2 +- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/options/options.h b/options/options.h index 94573ee063bb..94e8b9eef43e 100644 --- a/options/options.h +++ b/options/options.h @@ -169,7 +169,8 @@ extern struct matching_key *get_keys (struct key_store *ks, const char *device, const char *uuid, size_t *nr_matches); extern void free_keys (struct matching_key *keys, size_t nr_matches); extern struct key_store *key_store_add_from_selector (struct key_store *ks, const char *selector); -extern struct key_store *key_store_import_key (struct key_store *ks, const struct key_store_key *key); +extern struct key_store *key_store_import_key (struct key_store *ks, + struct key_store_key *key); extern bool key_store_requires_network (const struct key_store *ks); extern void free_key_store (struct key_store *ks); diff --git a/options/keys.c b/options/keys.c index 48f1bc7c7c47..bc7459749276 100644 --- a/options/keys.c +++ b/options/keys.c @@ -261,7 +261,7 @@ key_store_add_from_selector (struct key_store *ks, const char *selector) } struct key_store * -key_store_import_key (struct key_store *ks, const struct key_store_key *key) +key_store_import_key (struct key_store *ks, struct key_store_key *key) { struct key_store_key *new_keys;
On Mon, May 15, 2023 at 07:49:23PM +0200, Laszlo Ersek wrote:
The key_store_import_key() function is called from both C-language utilities -- via key_store_add_from_selector() -- and OCaml-language ones -- via guestfs_int_mllib_inspect_decrypt(). We currently declare the function's second parameter as
const struct key_store_key *key
That is however a superficial, if not outright false, promise: in key_store_import_key(), we take ownership of all three string fields within "key", as evidenced by free_key_store(), where we free() all three strings. With the same effort, we might as well modify the contents of those strings at once. Drop "const". (None of the callers care, but let's be honest.)
I'm not completely certain what the rules are here. Can't you have a const struct containing non-const strings? However I agree it looks confusing so that's a decent reason to change it, so: Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2168506 Signed-off-by: Laszlo Ersek <lersek@redhat.com> --- options/options.h | 3 ++- options/keys.c | 2 +- 2 files changed, 3 insertions(+), 2 deletions(-)
diff --git a/options/options.h b/options/options.h index 94573ee063bb..94e8b9eef43e 100644 --- a/options/options.h +++ b/options/options.h @@ -169,7 +169,8 @@ extern struct matching_key *get_keys (struct key_store *ks, const char *device, const char *uuid, size_t *nr_matches); extern void free_keys (struct matching_key *keys, size_t nr_matches); extern struct key_store *key_store_add_from_selector (struct key_store *ks, const char *selector); -extern struct key_store *key_store_import_key (struct key_store *ks, const struct key_store_key *key); +extern struct key_store *key_store_import_key (struct key_store *ks, + struct key_store_key *key); extern bool key_store_requires_network (const struct key_store *ks); extern void free_key_store (struct key_store *ks);
diff --git a/options/keys.c b/options/keys.c index 48f1bc7c7c47..bc7459749276 100644 --- a/options/keys.c +++ b/options/keys.c @@ -261,7 +261,7 @@ key_store_add_from_selector (struct key_store *ks, const char *selector) }
struct key_store * -key_store_import_key (struct key_store *ks, const struct key_store_key *key) +key_store_import_key (struct key_store *ks, struct key_store_key *key) { struct key_store_key *new_keys;
Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v
On 5/16/23 14:14, Richard W.M. Jones wrote:
On Mon, May 15, 2023 at 07:49:23PM +0200, Laszlo Ersek wrote:
The key_store_import_key() function is called from both C-language utilities -- via key_store_add_from_selector() -- and OCaml-language ones -- via guestfs_int_mllib_inspect_decrypt(). We currently declare the function's second parameter as
const struct key_store_key *key
That is however a superficial, if not outright false, promise: in key_store_import_key(), we take ownership of all three string fields within "key", as evidenced by free_key_store(), where we free() all three strings. With the same effort, we might as well modify the contents of those strings at once. Drop "const". (None of the callers care, but let's be honest.)
I'm not completely certain what the rules are here. Can't you have a const struct containing non-const strings?
Indeed you can have a const struct containing non-const strings, and the next patch would still work fine if we dropped this patch. The problem is with the "messaging" to the programmer. If a structure is taken (via a pointer-to-const) by a function, that's effectively a promise to the caller that neither the structure, nor anything reachable from it, is going to be modified by the function. C doesn't have a way to express the "anything reachable from it" part. "const" is technically not as effective as it should arguably be, but it carries a message, and that message is not factual in this particular case (even before patch#2) -- hence patch#1. Slightly related: <https://c-faq.com/ansi/constmismatch.html>.
However I agree it looks confusing so that's a decent reason to change it, so:
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Thanks! Laszlo
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2168506 Signed-off-by: Laszlo Ersek <lersek@redhat.com> --- options/options.h | 3 ++- options/keys.c | 2 +- 2 files changed, 3 insertions(+), 2 deletions(-)
diff --git a/options/options.h b/options/options.h index 94573ee063bb..94e8b9eef43e 100644 --- a/options/options.h +++ b/options/options.h @@ -169,7 +169,8 @@ extern struct matching_key *get_keys (struct key_store *ks, const char *device, const char *uuid, size_t *nr_matches); extern void free_keys (struct matching_key *keys, size_t nr_matches); extern struct key_store *key_store_add_from_selector (struct key_store *ks, const char *selector); -extern struct key_store *key_store_import_key (struct key_store *ks, const struct key_store_key *key); +extern struct key_store *key_store_import_key (struct key_store *ks, + struct key_store_key *key); extern bool key_store_requires_network (const struct key_store *ks); extern void free_key_store (struct key_store *ks);
diff --git a/options/keys.c b/options/keys.c index 48f1bc7c7c47..bc7459749276 100644 --- a/options/keys.c +++ b/options/keys.c @@ -261,7 +261,7 @@ key_store_add_from_selector (struct key_store *ks, const char *selector) }
struct key_store * -key_store_import_key (struct key_store *ks, const struct key_store_key *key) +key_store_import_key (struct key_store *ks, struct key_store_key *key) { struct key_store_key *new_keys;
Rich.
Introduce unescape_device_mapper_lvm() for turning /dev/mapper/VG-LV key IDs into /dev/VG/LV ones, unescaping doubled hyphens to single hyphens in both VG and LV in the process. Libguestfs uses the /dev/VG/LV format internally, for identifying devices, but the user might want to specify the /dev/mapper/VG-LV ID format with the "--key ID:..." options. Call unescape_device_mapper_lvm() from key_store_import_key(). That is, translate the ID as soon as the "--key" option is processed -- let the keystore only know about the usual /dev/VG/LV format, for later matching. Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2168506 Signed-off-by: Laszlo Ersek <lersek@redhat.com> --- Notes: regcomp() must definitely allocate memory dynamically, and we (intentionally) never free that -- we never call regfree(). I assume valgrind will catch this as a leak, so we might have to extend "valgrind-suppressions" in each dependent superproject. However, I'm unable to run "make check-valgrind" successfully in e.g. virt-v2v even before these patches; see also <https://listman.redhat.com/archives/libguestfs/2023-May/031496.html>. options/keys.c | 86 ++++++++++++++++++++ 1 file changed, 86 insertions(+) diff --git a/options/keys.c b/options/keys.c index bc7459749276..f0164a6ed987 100644 --- a/options/keys.c +++ b/options/keys.c @@ -27,6 +27,7 @@ #include <errno.h> #include <error.h> #include <assert.h> +#include <regex.h> #include "guestfs.h" @@ -260,6 +261,90 @@ key_store_add_from_selector (struct key_store *ks, const char *selector) return key_store_import_key (ks, &key); } +/* A /dev/mapper/ escaped character is: + * - either any character except a slash or a hyphen, + * - or a double hyphen. + * + * Note that parens are deliberately not included here -- they will be included + * in the full pattern, for making them more easily countable. + * + * Also note that the bracket expression below does not contain a range + * expression, therefore it should not be locale-sensitive. + */ +#define ESC_CH "[^-/]|--" + +/* Turn /dev/mapper/VG-LV into /dev/VG/LV. */ +static void +unescape_device_mapper_lvm (char *id) +{ + static bool compiled; + int rc; + static regex_t regex; + regmatch_t match[5]; + char *output; + const char *vg_start, *vg_end, *lv_start, *lv_end; + const char *input; + + if (!compiled) { + /* Recognize /dev/mapper/VG-LV pathnames, where VG and LV don't contain + * slashes, plus any original hyphens in them are doubled for escaping. + */ + static const char pattern[] = "^/dev/(" + "mapper/" + "((" ESC_CH ")+)" + "-" + "((" ESC_CH ")+)" + ")$"; + + rc = regcomp (®ex, pattern, REG_EXTENDED); + if (rc != 0) { + char errbuf[256]; + + /* When regcomp() fails, the contents of "regex" are undefined, so we + * cannot pass "regex" to regerror(). + */ + (void)regerror (rc, NULL, errbuf, sizeof errbuf); + error (EXIT_FAILURE, 0, + _("%s: Failed to compile pattern (%s). Please report a bug for " + "libguestfs -- refer to guestfs(3) section BUGS."), + __func__, errbuf); + } + compiled = true; + } + + rc = regexec (®ex, id, 5, match, 0); + + /* If there's no match, just leave "id" alone. */ + if (rc != 0) + return; + + /* Start outputting after "/dev/". */ + output = id + match[1].rm_so; + vg_start = id + match[2].rm_so; + vg_end = id + match[2].rm_eo; + lv_start = id + match[4].rm_so; + lv_end = id + match[4].rm_eo; + + /* Each of the following two loops produces at most as many bytes as it + * consumes, so we unescape "id" in-place. + */ + input = vg_start; + while (input < vg_end) + if ((*output++ = *input++) == '-') + input++; + + *output++ = '/'; + + input = lv_start; + while (input < lv_end) + if ((*output++ = *input++) == '-') + input++; + + *output++ = '\0'; +} + +#undef ESC_CH + struct key_store * key_store_import_key (struct key_store *ks, struct key_store_key *key) { @@ -278,6 +363,7 @@ key_store_import_key (struct key_store *ks, struct key_store_key *key) error (EXIT_FAILURE, errno, "realloc"); ks->keys = new_keys; + unescape_device_mapper_lvm (key->id); ks->keys[ks->nr_keys] = *key; ++ks->nr_keys;
On Mon, May 15, 2023 at 07:49:24PM +0200, Laszlo Ersek wrote:
Introduce unescape_device_mapper_lvm() for turning
/dev/mapper/VG-LV
key IDs into
/dev/VG/LV
ones, unescaping doubled hyphens to single hyphens in both VG and LV in the process. Libguestfs uses the /dev/VG/LV format internally, for identifying devices, but the user might want to specify the /dev/mapper/VG-LV ID format with the "--key ID:..." options.
Call unescape_device_mapper_lvm() from key_store_import_key(). That is, translate the ID as soon as the "--key" option is processed -- let the keystore only know about the usual /dev/VG/LV format, for later matching.
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2168506 Signed-off-by: Laszlo Ersek <lersek@redhat.com> ---
Notes: regcomp() must definitely allocate memory dynamically, and we (intentionally) never free that -- we never call regfree(). I assume valgrind will catch this as a leak, so we might have to extend "valgrind-suppressions" in each dependent superproject. However, I'm unable to run "make check-valgrind" successfully in e.g. virt-v2v even before these patches; see also <https://listman.redhat.com/archives/libguestfs/2023-May/031496.html>.
options/keys.c | 86 ++++++++++++++++++++ 1 file changed, 86 insertions(+)
diff --git a/options/keys.c b/options/keys.c index bc7459749276..f0164a6ed987 100644 --- a/options/keys.c +++ b/options/keys.c @@ -27,6 +27,7 @@ #include <errno.h> #include <error.h> #include <assert.h> +#include <regex.h>
#include "guestfs.h"
@@ -260,6 +261,90 @@ key_store_add_from_selector (struct key_store *ks, const char *selector) return key_store_import_key (ks, &key); }
+/* A /dev/mapper/ escaped character is: + * - either any character except a slash or a hyphen, + * - or a double hyphen. + * + * Note that parens are deliberately not included here -- they will be included + * in the full pattern, for making them more easily countable. + * + * Also note that the bracket expression below does not contain a range + * expression, therefore it should not be locale-sensitive. + */ +#define ESC_CH "[^-/]|--" + +/* Turn /dev/mapper/VG-LV into /dev/VG/LV. */ +static void +unescape_device_mapper_lvm (char *id) +{ + static bool compiled; + int rc; + static regex_t regex; + regmatch_t match[5]; + char *output; + const char *vg_start, *vg_end, *lv_start, *lv_end; + const char *input; + + if (!compiled) { + /* Recognize /dev/mapper/VG-LV pathnames, where VG and LV don't contain + * slashes, plus any original hyphens in them are doubled for escaping. + */ + static const char pattern[] = "^/dev/(" + "mapper/" + "((" ESC_CH ")+)" + "-" + "((" ESC_CH ")+)" + ")$"; + + rc = regcomp (®ex, pattern, REG_EXTENDED); + if (rc != 0) { + char errbuf[256]; + + /* When regcomp() fails, the contents of "regex" are undefined, so we + * cannot pass "regex" to regerror(). + */ + (void)regerror (rc, NULL, errbuf, sizeof errbuf); + error (EXIT_FAILURE, 0, + _("%s: Failed to compile pattern (%s). Please report a bug for " + "libguestfs -- refer to guestfs(3) section BUGS."), + __func__, errbuf); + } + compiled = true; + }
Any reason not to use PCRE and the COMPILE_REGEXP macro? We also have macros like CLEANUP_PCRE2_MATCH_DATA_FREE to free match data automaticaly. In general PCRE regexps are a lot more usable than the POSIX stuff. Rich.
+ rc = regexec (®ex, id, 5, match, 0); + + /* If there's no match, just leave "id" alone. */ + if (rc != 0) + return; + + /* Start outputting after "/dev/". */ + output = id + match[1].rm_so; + vg_start = id + match[2].rm_so; + vg_end = id + match[2].rm_eo; + lv_start = id + match[4].rm_so; + lv_end = id + match[4].rm_eo; + + /* Each of the following two loops produces at most as many bytes as it + * consumes, so we unescape "id" in-place. + */ + input = vg_start; + while (input < vg_end) + if ((*output++ = *input++) == '-') + input++; + + *output++ = '/'; + + input = lv_start; + while (input < lv_end) + if ((*output++ = *input++) == '-') + input++; + + *output++ = '\0'; +} + +#undef ESC_CH + struct key_store * key_store_import_key (struct key_store *ks, struct key_store_key *key) { @@ -278,6 +363,7 @@ key_store_import_key (struct key_store *ks, struct key_store_key *key) error (EXIT_FAILURE, errno, "realloc");
ks->keys = new_keys; + unescape_device_mapper_lvm (key->id); ks->keys[ks->nr_keys] = *key; ++ks->nr_keys;
_______________________________________________ Libguestfs mailing list Libguestfs@redhat.com https://listman.redhat.com/mailman/listinfo/libguestfs
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org
On 5/16/23 14:17, Richard W.M. Jones wrote:
On Mon, May 15, 2023 at 07:49:24PM +0200, Laszlo Ersek wrote:
Introduce unescape_device_mapper_lvm() for turning
/dev/mapper/VG-LV
key IDs into
/dev/VG/LV
ones, unescaping doubled hyphens to single hyphens in both VG and LV in the process. Libguestfs uses the /dev/VG/LV format internally, for identifying devices, but the user might want to specify the /dev/mapper/VG-LV ID format with the "--key ID:..." options.
Call unescape_device_mapper_lvm() from key_store_import_key(). That is, translate the ID as soon as the "--key" option is processed -- let the keystore only know about the usual /dev/VG/LV format, for later matching.
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2168506 Signed-off-by: Laszlo Ersek <lersek@redhat.com> ---
Notes: regcomp() must definitely allocate memory dynamically, and we (intentionally) never free that -- we never call regfree(). I assume valgrind will catch this as a leak, so we might have to extend "valgrind-suppressions" in each dependent superproject. However, I'm unable to run "make check-valgrind" successfully in e.g. virt-v2v even before these patches; see also <https://listman.redhat.com/archives/libguestfs/2023-May/031496.html>.
options/keys.c | 86 ++++++++++++++++++++ 1 file changed, 86 insertions(+)
diff --git a/options/keys.c b/options/keys.c index bc7459749276..f0164a6ed987 100644 --- a/options/keys.c +++ b/options/keys.c @@ -27,6 +27,7 @@ #include <errno.h> #include <error.h> #include <assert.h> +#include <regex.h>
#include "guestfs.h"
@@ -260,6 +261,90 @@ key_store_add_from_selector (struct key_store *ks, const char *selector) return key_store_import_key (ks, &key); }
+/* A /dev/mapper/ escaped character is: + * - either any character except a slash or a hyphen, + * - or a double hyphen. + * + * Note that parens are deliberately not included here -- they will be included + * in the full pattern, for making them more easily countable. + * + * Also note that the bracket expression below does not contain a range + * expression, therefore it should not be locale-sensitive. + */ +#define ESC_CH "[^-/]|--" + +/* Turn /dev/mapper/VG-LV into /dev/VG/LV. */ +static void +unescape_device_mapper_lvm (char *id) +{ + static bool compiled; + int rc; + static regex_t regex; + regmatch_t match[5]; + char *output; + const char *vg_start, *vg_end, *lv_start, *lv_end; + const char *input; + + if (!compiled) { + /* Recognize /dev/mapper/VG-LV pathnames, where VG and LV don't contain + * slashes, plus any original hyphens in them are doubled for escaping. + */ + static const char pattern[] = "^/dev/(" + "mapper/" + "((" ESC_CH ")+)" + "-" + "((" ESC_CH ")+)" + ")$"; + + rc = regcomp (®ex, pattern, REG_EXTENDED); + if (rc != 0) { + char errbuf[256]; + + /* When regcomp() fails, the contents of "regex" are undefined, so we + * cannot pass "regex" to regerror(). + */ + (void)regerror (rc, NULL, errbuf, sizeof errbuf); + error (EXIT_FAILURE, 0, + _("%s: Failed to compile pattern (%s). Please report a bug for " + "libguestfs -- refer to guestfs(3) section BUGS."), + __func__, errbuf); + } + compiled = true; + }
Any reason not to use PCRE and the COMPILE_REGEXP macro?
I didn't even think of PCRE -- regexes are standard (i.e., in POSIX). This is a relatively simple regex at that.
We also have macros like CLEANUP_PCRE2_MATCH_DATA_FREE to free match data automaticaly.
That's different, I think. If I understand "THE MATCH DATA BLOCK" section <https://www.pcre.org/current/doc/html/pcre2api.html#SEC26> correctly, it implies that dynamic memory needs to be allocated for each separate matching attempt, not just for compiling the pattern. Here we only allocate memory when compiling the pattern; regexec() needs no dynamic memory. (POSIX says that if it fails, it can only fail with REG_NOMATCH, and not (for example) with REG_ESPACE.) Furthermore, the offsets of the subexpressions are placed in the local variable "match", which doesn't even provide enough slots for all the subexpressions -- we don't care for the last subexpression.
In general PCRE regexps are a lot more usable than the POSIX stuff.
I think PCRE is much more capable than POSIX regexes, regarding parsing power / expressivity. On the other hand, I find the PCRE API way more complex than the POSIX one -- I have never *not* struggled with the PCRE documentation. POSIX is just one specification page: https://pubs.opengroup.org/onlinepubs/9699919799/functions/regcomp.html and for this RE, it is sufficient. If we're supposed to use pcre2 in common/ for consistency with libguestfs, that sounds like a valid argument (consistency is important). However, in that case, I'm tempted to rewrite this without regular expressions in the first place. That's possible; I just found regexes more comfortable, specifically due to having been used to the POSIX RE API. The argument went like, "POSIX regex is easy, and then the loops become even easier, relying on the regex-pre-check". However, the PCRE2 API is something I have no experience with, and I find the documentation baroque... So that kind of defeats the entire original argument. After some digging in libguestfs... my point is illustrated by the fact that we have the macro COMPILE_REGEXP(), and functions like guestfs_int_match2() :( Let me take a stab at a rewrite without a regular expression. I wonder if I'll dislike that enough to sweeten pcre2 to me. Thanks Laszlo
Rich.
+ rc = regexec (®ex, id, 5, match, 0); + + /* If there's no match, just leave "id" alone. */ + if (rc != 0) + return; + + /* Start outputting after "/dev/". */ + output = id + match[1].rm_so; + vg_start = id + match[2].rm_so; + vg_end = id + match[2].rm_eo; + lv_start = id + match[4].rm_so; + lv_end = id + match[4].rm_eo; + + /* Each of the following two loops produces at most as many bytes as it + * consumes, so we unescape "id" in-place. + */ + input = vg_start; + while (input < vg_end) + if ((*output++ = *input++) == '-') + input++; + + *output++ = '/'; + + input = lv_start; + while (input < lv_end) + if ((*output++ = *input++) == '-') + input++; + + *output++ = '\0'; +} + +#undef ESC_CH + struct key_store * key_store_import_key (struct key_store *ks, struct key_store_key *key) { @@ -278,6 +363,7 @@ key_store_import_key (struct key_store *ks, struct key_store_key *key) error (EXIT_FAILURE, errno, "realloc");
ks->keys = new_keys; + unescape_device_mapper_lvm (key->id); ks->keys[ks->nr_keys] = *key; ++ks->nr_keys;
_______________________________________________ Libguestfs mailing list Libguestfs@redhat.com https://listman.redhat.com/mailman/listinfo/libguestfs
participants (2)
-
Laszlo Ersek -
Richard W.M. Jones