[Hivex] [PATCH v3] Report last-modified time of hive root and nodes
The infrastructure for modified-time reporting has been essentially unused. These changes report the registry time by treating the time fields as Windows filetime fields stored in little-Endian (which means they can be treated as a single 64-bit little-Endian integer). This patch adds to the hivex ABI: * int64_t hivex_last_modified (hive_h *) * int64_t hivex_node_timestamp (hive_h *, hive_node_h) These two functions return the hive's last-modified time and a particular node's last-modified time, respectively. Credit to Richard Jones for the ABI suggestion, and for the tip on Microsoft's filetime time span. hivexml employs these two functions to produce mtime elements for a hive and all of its nodes, producing ISO-8601 formatted time. Signed-off-by: Alex Nelson <ajnelson@cs.ucsc.edu> --- generator/generator.ml | 20 +++++++++++++ lib/hivex.c | 46 +++++++++++++++++++++++++++++-- xml/hivexml.c | 71 +++++++++++++++++++++++++++++++++++++++++++++++- 3 files changed, 133 insertions(+), 4 deletions(-) diff --git a/generator/generator.ml b/generator/generator.ml index de911f1..370472f 100755 --- a/generator/generator.ml +++ b/generator/generator.ml @@ -158,6 +158,13 @@ but instead are lost. See L<hivex(3)/WRITING TO HIVE FILES>."; "\ Return root node of the hive. All valid hives must contain a root node."; + "last_modified", (RInt64, [AHive]), + "return the modification time of the root node of the hive", + "\ +Return the modification time of the root node of the hive. The +returned value is equivalent in type and caveats as +hivex_node_timestamp."; + "node_name", (RString, [AHive; ANode "node"]), "return the name of the node", "\ @@ -170,6 +177,16 @@ only know the \"real\" name of the root node by knowing which registry file this hive originally comes from, which is knowledge that is outside the scope of this library."; + "node_timestamp", (RInt64, [AHive; ANode "node"]), + "return the modification time of the node", + "\ +Return the modification time of the node. Output is an Endian-correct +64-bit signed number. + +Though Windows' filetime struct looks like it should be an unsigned +64-bit int, Microsoft only guarantees FILETIME through 9999-12-31, due +to conversion limitations with .Net's DateTime."; + "node_children", (RNodeList, [AHive; ANode "node"]), "return children of node", "\ @@ -708,6 +725,9 @@ typedef size_t hive_value_h; # define HIVEX_NO_KEY ENOENT #endif +#include <time.h> +#define TIMESTAMP_BUF_LEN 32 + /* Pre-defined types. */ enum hive_type { "; diff --git a/lib/hivex.c b/lib/hivex.c index fedbb6c..1e77831 100644 --- a/lib/hivex.c +++ b/lib/hivex.c @@ -93,6 +93,7 @@ struct hive_h { /* Fields from the header, extracted from little-endianness hell. */ size_t rootoffs; /* Root key offset (always an nk-block). */ size_t endpages; /* Offset of end of pages. */ + int64_t last_modified; /* mtime of base block. */ /* For writing. */ size_t endblocks; /* Offset to next block allocation (0 @@ -104,7 +105,7 @@ struct ntreg_header { char magic[4]; /* "regf" */ uint32_t sequence1; uint32_t sequence2; - char last_modified[8]; + int64_t last_modified; uint32_t major_ver; /* 1 */ uint32_t minor_ver; /* 3 */ uint32_t unknown5; /* 0 */ @@ -173,7 +174,7 @@ struct ntreg_nk_record { int32_t seg_len; /* length (always -ve because used) */ char id[2]; /* "nk" */ uint16_t flags; - char timestamp[8]; + int64_t timestamp; uint32_t unknown1; uint32_t parent; /* offset of owner/parent */ uint32_t nr_subkeys; /* number of subkeys */ @@ -359,6 +360,9 @@ hivex_open (const char *filename, int flags) goto error; } + /* Last-modified time. */ + h->last_modified = le64toh ((int64_t) h->hdr->last_modified); + if (h->msglvl >= 2) { char *name = windows_utf16_to_utf8 (h->hdr->name, 64); @@ -367,6 +371,8 @@ hivex_open (const char *filename, int flags) " file version %" PRIu32 ".%" PRIu32 "\n" " sequence nos %" PRIu32 " %" PRIu32 "\n" " (sequences nos should match if hive was synched at shutdown)\n" + " last modified \n" + " (decimal, 100 ns) %" PRIu64 "\n" " original file name %s\n" " (only 32 chars are stored, name is probably truncated)\n" " root offset 0x%x + 0x1000\n" @@ -374,6 +380,7 @@ hivex_open (const char *filename, int flags) " checksum 0x%x (calculated 0x%x)\n", major_ver, le32toh (h->hdr->minor_ver), le32toh (h->hdr->sequence1), le32toh (h->hdr->sequence2), + h->last_modified, name ? name : "(conversion failed)", le32toh (h->hdr->offset), le32toh (h->hdr->blocks), h->size, @@ -608,6 +615,39 @@ hivex_node_name (hive_h *h, hive_node_h node) return ret; } +int64_t +hivex_timestamp_check (hive_h *h, hive_node_h node, int64_t timestamp) +{ + if (timestamp < 0) { + fprintf (stderr, "Negative time reported at %z: %" PRIi64 "\n", node, timestamp); + errno = EINVAL; + return -1; + } + return timestamp; +} + +int64_t +hivex_last_modified (hive_h *h) +{ + return hivex_timestamp_check (h, 0, h->last_modified); +} + +int64_t +hivex_node_timestamp (hive_h *h, hive_node_h node) +{ + int64_t ret; + + if (!IS_VALID_BLOCK (h, node) || !BLOCK_ID_EQ (h, node, "nk")) { + errno = EINVAL; + return -1; + } + + struct ntreg_nk_record *nk = (struct ntreg_nk_record *) (h->addr + node); + + ret = le64toh (nk->timestamp); + return hivex_timestamp_check (h, node, ret); +} + #if 0 /* I think the documentation for the sk and classname fields in the nk * record is wrong, or else the offset field is in the wrong place. @@ -2264,7 +2304,7 @@ hivex_node_add_child (hive_h *h, hive_node_h parent, const char *name) nk->sk = htole32 (parent_sk_offset - 0x1000); /* Inherit parent timestamp. */ - memcpy (nk->timestamp, parent_nk->timestamp, sizeof (parent_nk->timestamp)); + nk->timestamp = parent_nk->timestamp; /* What I found out the hard way (not documented anywhere): the * subkeys in lh-records must be kept sorted. If you just add a diff --git a/xml/hivexml.c b/xml/hivexml.c index 90cb22b..c68a3d2 100644 --- a/xml/hivexml.c +++ b/xml/hivexml.c @@ -64,6 +64,8 @@ static struct hivex_visitor visitor = { .value_other = value_other }; +char * filetime_to_8601 (int64_t windows_ticks); + #define XML_CHECK(proc, args) \ do { \ if ((proc args) == -1) { \ @@ -124,6 +126,20 @@ main (int argc, char *argv[]) XML_CHECK (xmlTextWriterStartDocument, (writer, NULL, "utf-8", NULL)); XML_CHECK (xmlTextWriterStartElement, (writer, BAD_CAST "hive")); + int64_t hive_mtime = hivex_last_modified (h); + if (hive_mtime < 0) + goto skip_mtime; + char *timebuf = filetime_to_8601 (hive_mtime); + if (timebuf == NULL) + goto skip_mtime; + XML_CHECK (xmlTextWriterStartElement, (writer, BAD_CAST "mtime")); + XML_CHECK (xmlTextWriterWriteString, (writer, BAD_CAST timebuf)); + XML_CHECK (xmlTextWriterEndElement, (writer)); + free (timebuf); + timebuf = NULL; + +skip_mtime: + if (hivex_visit (h, &visitor, sizeof visitor, writer, visit_flags) == -1) { perror (argv[optind]); exit (EXIT_FAILURE); @@ -141,13 +157,66 @@ main (int argc, char *argv[]) exit (EXIT_SUCCESS); } +#define WINDOWS_TICK 10000000LL +#define SEC_TO_UNIX_EPOCH 11644473600LL +/** + * Convert Windows filetime to ISO 8601 format. + * Source for filetime->time_t conversion: http://stackoverflow.com/questions/6161776/convert-windows-filetime-to-secon... + * Source for time_t->char* conversion: Fiwalk version 0.6.14's fiwalk.cpp. + * @param windows_ticks Expected to not have any remaining Endian issues. + * + * Caller is responsible for freeing non-null returned buffer. + */ +char * +filetime_to_8601 (int64_t windows_ticks) +{ + char *ret = calloc (1 + TIMESTAMP_BUF_LEN, sizeof (char)); + if (ret == NULL) { + goto error_other; + } + uint64_t nanos = windows_ticks % WINDOWS_TICK; + time_t tt = (windows_ticks / WINDOWS_TICK - SEC_TO_UNIX_EPOCH); + struct tm time_tm; + if (gmtime_r (&tt, &time_tm) == NULL) { + fprintf (stderr, "filetime_to_8601: Error running gmtime_r on timestamp (decimal hundreds of ns: %" PRIu64 ").\n", windows_ticks); + goto error_cleanup; + } + if (strftime (ret, TIMESTAMP_BUF_LEN, "%FT%TZ", &time_tm) == 0) + goto error_cleanup; + return ret; + +error_cleanup: + free (ret); + ret = NULL; +error_other: + return NULL; +} + static int node_start (hive_h *h, void *writer_v, hive_node_h node, const char *name) { + int ret = 0; + int64_t last_modified; + char *timebuf; + xmlTextWriterPtr writer = (xmlTextWriterPtr) writer_v; XML_CHECK (xmlTextWriterStartElement, (writer, BAD_CAST "node")); XML_CHECK (xmlTextWriterWriteAttribute, (writer, BAD_CAST "name", BAD_CAST name)); - return 0; + + last_modified = hivex_node_timestamp (h, node); + if (last_modified < 0) + goto skip_mtime; + timebuf = filetime_to_8601 (last_modified); + if (!timebuf) + goto skip_mtime; + XML_CHECK (xmlTextWriterStartElement, (writer, BAD_CAST "mtime")); + XML_CHECK (xmlTextWriterWriteString, (writer, BAD_CAST timebuf)); + XML_CHECK (xmlTextWriterEndElement, (writer)); + free (timebuf); + timebuf = NULL; + +skip_mtime: + return ret; } static int -- 1.7.6
Looks good. I'll apply it later on today. Thanks, Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://et.redhat.com/~rjones/virt-top
I have modified the code quite a lot before committing it. Please take a look: http://git.annexia.org/?p=hivex.git;a=commitdiff;h=e0b21193257f9784b28e93152... Thanks for your contribution, Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org
Hi Richard, all, The modifications make sense. I'm happier with the nested if tests instead of the gotos. Thank you. I noticed while I was producing the patch that I had to have the filetime_to_8601 in xml/hivexml.c due to a linking error. I would've liked to have the function in lib/hivex.c, but I hit a strange linking error when I had the function and its prototype placed as in the patch below. It's strange because the extern doesn't appear to be in a strange place, and there is no static restriction on the function definition, so the function seems to me like it should behave like any of the other API functions. Could it be a linking eror against an old library? The last few lines of `make V=1` in a Fedora 15 VM are: Making all in xml make[2]: Entering directory `/home/alex/local/src/hivex/xml' /bin/sh ../libtool --tag=CC --mode=link gcc -std=gnu99 -DLOCALEBASEDIR=\""/usr/local/share/locale"\" -I../gnulib/lib -I../lib -I/usr/include/libxml2 -g -O2 -o hivexml hivexml-hivexml.o ../lib/libhivex.la -lxml2 -lz -lm libtool: link: gcc -std=gnu99 -DLOCALEBASEDIR=\"/usr/local/share/locale\" -I../gnulib/lib -I../lib -I/usr/include/libxml2 -g -O2 -o .libs/hivexml hivexml-hivexml.o ../lib/.libs/libhivex.so -lxml2 -lz -lm -Wl,-rpath -Wl,/usr/local/lib hivexml-hivexml.o: In function `node_start': /home/alex/local/src/hivex/xml/hivexml.c:167: undefined reference to `filetime_to_8601' hivexml-hivexml.o: In function `main': /home/alex/local/src/hivex/xml/hivexml.c:129: undefined reference to `filetime_to_8601' collect2: ld returned 1 exit status make[2]: *** [hivexml] Error 1 make[2]: Leaving directory `/home/alex/local/src/hivex/xml' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/home/alex/local/src/hivex' make: *** [all] Error 2 I think I've found a related issue while trying to get hivexml running from the GIt source on a fresh Ubuntu 11.04 VM. I asked about the linking error because Ubuntu warns about relinking, though it did compile (without the below patch moving filetime_to_8601 around). Environment: Completely new virtual machine, with this command ran for dependencies: sudo apt-get install git libtool autopoint ocaml autoconf libxml2-dev python-dev To build, I cd'd into the git-clone'd directory and ran: ./autogen.sh && make V=1 && sudo make install At the 'sudo make install' phase, this warning appeared: libtool: install: warning: relinking 'libhivexmod.la' The relinking warning remained after running 'sudo make uninstall' and 'make clean' in succession, and even removing the cloned folder and starting over again. Also, running hivexml raised this error: hivexml: error while loading shared libraries: libhivex.so.0: cannot open shared object file: No such file or directory I needed this line in my ~/.bashrc to get the shared library working: export LD_LIBRARY_PATH="/usr/local/lib:$LD_LIBRARY_PATH" If that's a complaint about Ubuntu in general, though, than I guess it's not a problem for a Redhat mailing list. --Alex --- generator/generator.ml | 6 ++++++ lib/hivex.c | 34 ++++++++++++++++++++++++++++++++++ xml/hivexml.c | 42 ------------------------------------------ 3 files changed, 40 insertions(+), 42 deletions(-) diff --git a/generator/generator.ml b/generator/generator.ml index 943039c..47ddc31 100755 --- a/generator/generator.ml +++ b/generator/generator.ml @@ -710,6 +710,12 @@ let rec generate_c_header () = extern \"C\" { #endif +#include <time.h> +#define WINDOWS_TICK 10000000LL +#define SEC_TO_UNIX_EPOCH 11644473600LL +#define TIMESTAMP_BUF_LEN 32 +extern char *filetime_to_8601 (int64_t windows_ticks); + /* NOTE: This API is documented in the man page hivex(3). */ /* Hive handle. */ diff --git a/lib/hivex.c b/lib/hivex.c index dceea73..e1a2c87 100644 --- a/lib/hivex.c +++ b/lib/hivex.c @@ -266,6 +266,40 @@ header_checksum (const hive_h *h) return sum; } +/* Convert Windows filetime to ISO 8601 format. + * http://stackoverflow.com/questions/6161776/convert-windows-filetime-to-secon... + * + * Source for time_t->char* conversion: Fiwalk version 0.6.14's + * fiwalk.cpp. + * + * The caller should free the returned buffer. + */ +char * +filetime_to_8601 (int64_t windows_ticks) +{ + char *ret; + time_t t; + struct tm *tm; + + t = windows_ticks / WINDOWS_TICK - SEC_TO_UNIX_EPOCH; + tm = gmtime (&t); + if (tm == NULL) + return NULL; + + ret = malloc (TIMESTAMP_BUF_LEN); + if (ret == NULL) { + perror ("malloc"); + exit (EXIT_FAILURE); + } + + if (strftime (ret, TIMESTAMP_BUF_LEN, "%FT%TZ", tm) == 0) { + perror ("strftime"); + exit (EXIT_FAILURE); + } + + return ret; +} + #define HIVEX_OPEN_MSGLVL_MASK (HIVEX_OPEN_VERBOSE|HIVEX_OPEN_DEBUG) hive_h * diff --git a/xml/hivexml.c b/xml/hivexml.c index 2967ac9..a846799 100644 --- a/xml/hivexml.c +++ b/xml/hivexml.c @@ -25,7 +25,6 @@ #include <inttypes.h> #include <unistd.h> #include <errno.h> -#include <time.h> #include <libxml/xmlwriter.h> @@ -40,8 +39,6 @@ //#define N_(str) str #endif -static char *filetime_to_8601 (int64_t windows_ticks); - /* Callback functions. */ static int node_start (hive_h *, void *, hive_node_h, const char *name); static int node_end (hive_h *, void *, hive_node_h, const char *name); @@ -155,45 +152,6 @@ main (int argc, char *argv[]) exit (EXIT_SUCCESS); } -/* Convert Windows filetime to ISO 8601 format. - * http://stackoverflow.com/questions/6161776/convert-windows-filetime-to-secon... - * - * Source for time_t->char* conversion: Fiwalk version 0.6.14's - * fiwalk.cpp. - * - * The caller should free the returned buffer. - */ - -#define WINDOWS_TICK 10000000LL -#define SEC_TO_UNIX_EPOCH 11644473600LL -#define TIMESTAMP_BUF_LEN 32 - -static char * -filetime_to_8601 (int64_t windows_ticks) -{ - char *ret; - time_t t; - struct tm *tm; - - t = windows_ticks / WINDOWS_TICK - SEC_TO_UNIX_EPOCH; - tm = gmtime (&t); - if (tm == NULL) - return NULL; - - ret = malloc (TIMESTAMP_BUF_LEN); - if (ret == NULL) { - perror ("malloc"); - exit (EXIT_FAILURE); - } - - if (strftime (ret, TIMESTAMP_BUF_LEN, "%FT%TZ", tm) == 0) { - perror ("strftime"); - exit (EXIT_FAILURE); - } - - return ret; -} - static int node_start (hive_h *h, void *writer_v, hive_node_h node, const char *name) { -- 1.7.6 On Aug 13, 2011, at 01:46 , Richard W.M. Jones wrote:
I have modified the code quite a lot before committing it. Please take a look:
http://git.annexia.org/?p=hivex.git;a=commitdiff;h=e0b21193257f9784b28e93152...
Thanks for your contribution,
Rich.
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org
On Sat, Aug 13, 2011 at 03:49:13PM -0700, Alex Nelson wrote:
I noticed while I was producing the patch that I had to have the filetime_to_8601 in xml/hivexml.c due to a linking error. I would've liked to have the function in lib/hivex.c, but I hit a strange linking error when I had the function and its prototype placed as in the patch below. It's strange because the extern doesn't appear to be in a strange place, and there is no static restriction on the function definition, so the function seems to me like it should behave like any of the other API functions. Could it be a linking eror against an old library?
When the libhivex.so library is built, we whitelist the symbols that it exports. This is to prevent unintentional exports of symbols. In any case, you can't have filetime_to_8601 in hivex.c. It's solely required by hivexml, so it should be there. It isn't namespaced properly. Because it's not listed in the generator, no bindings would be made for it in other languages, and for the same reason the generator doesn't add it to the whitelist. Also <hivex.h> is the public API of the hivex library. One of the consumers happens to be hivexml, but there are many others. This file isn't a dumping ground for random symbols and #includes that are only needed by hivexml. So, NACK. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-df lists disk usage of guests without needing to install any software inside the virtual machine. Supports Linux and Windows. http://et.redhat.com/~rjones/virt-df/
participants (2)
-
Alex Nelson -
Richard W.M. Jones