September 2010 - Libguestfs - libguestfs List Archives

Proposed timetable for libguestfs 1.6
by Richard W.M. Jones 14 Oct '10

14 Oct '10

The development branch (1.5) contains some major performance improvements over the current stable branch (1.4). Furthermore nearly 3 months have passed since we branched 1.4, which is about the same time that separated 1.4 from 1.2. Thus I think around the first or second week of October we ought to start the new stable and development branches (1.6 and 1.7 resp.) At this time we should decide if all the new APIs and features we have added in 1.5 are supportable in the long term, since we only guarantee the C API is stable for stable branches. Below are the commits we have added (so far) in 1.5, and you can read the whole API on the website. I will be compiling a summary of major features in the release notes file in the source: http://git.annexia.org/?p=libguestfs.git;a=blob;f=RELEASE-NOTES;hb=HEAD Any objections or problems with a commit, please follow up. Rich. d75a2bf fish: Implement 'hexedit' command. 5d4ae18 New APIs: upload-offset and download-offset 8c5a4d9 leak: Clear history before exiting guestfish. 8ea62c8 leak: Free list of drives and mountpoints in guestfish. e7f6274 leak: Free PCRE regexps when library is unloaded. 6d276da leak: Appliance name was leaked during guestfs_launch. 0132890 Add more exclusions to .gitignore. d1c2287 Fix appliance build dependency problem ff4ae86 fish: Add --echo-keys option to allow passphrases/keys to be echoed. 8328649 df: Add --one-per-guest option for using one appliance per guest. 17e7cb9 Fix error launching libguestfs when euid != uid. 6123abc todo: Suggest removing repo name from appliance name. 0003ea2 generator: Generate guestfish-only commands. 585fceb fish: In guestfish(1) turn command references into links. 3eb7655 fish: Correction for online help for 'edit' and 'more' commands. a37ec22 Version 1.5.16. 6b2ae9a configure: Make "fedora-13" the default repository. dd093a7 fish: If -m option fails, suggest a mountpoint. 325587d todo: Remove section since we now have list-filesystems API. d4ba3d3 Version 1.5.15. 000c4a8 New API: list-filesystems: list filesystems ffd4820 New API: part-to-dev: Convert partition name to device name. ff38fea generator: Add TestOutputDevice. 43d5ea6 todo: More use of libblkid. c98f073 fish: In usage message use new-style -i option syntax. 42b62b8 fish: Update copyright dates in usage message. 31a2c00 fish: Remove extraneous space from usage message. 7c87dd1 todo: More ideas. 5a7c430 Version 1.5.14. d3c6e50 configure: Check for virtio-serial support in qemu. 9cab7e3 pardus: Check for cpio in configure. 17b50a3 Update Spanish translation (RHBZ#633357). a403e74 build: Add run-test-tool-locally to EXTRA_DIST. 2b1e36d Version 1.5.13. 7419335 appliance: Disable setting scheduler to noop. 3deebe5 ubuntu: Remove bogus debirf file. 75f0a92 ubuntu: Add linux-image to the packagelist. 93ae3fa ubuntu: /proc can be a symlink da107cf build: 'make quickcheck' rule now uses new run-test-tool-locally script. 9f653ed build: hivex is required. f565e71 Version 1.5.12. e903739 build: require Augeas for library. 67fd3a7 generator: Provide no-op generator if no OCaml compiler. 2fbac04 build: Don't distribute src/generator.ml, no longer exists. b42262c generator: Calculate MD5 of test.iso at runtime. 451a283 generator: Don't use real uuidgen for UUIDs. 04d8209 Split generator into separate source files. 264629b syntax: Replace -a and -o with && and || for portability. 431503d syntax: Use exit (EXIT_SUCCESS) instead of hard-coded number. 5133632 syntax: Remove unused assert.h header. 21aac86 syntax: Remove unused ignore-value.h header. 0e422c0 syntax: Remove unused signal.h header. a0b7045 syntax: Remove unused c-ctype.h header. c1b6352 syntax: Fully bracket m4 macro arguments. b3d94d6 syntax: Remove trailing spaces. b4a3aec syntax: Use spaces instead of tabs for indentation. 3d994f3 syntax: Replace _prohibit_regexp with _sc_search_regexp. c359347 fish: glob should only print commands when trace mode is enabled. b033b55 fish: Add regression test for copy-in and copy-out. 3161cb5 README: Document virtio-serial is now the only vmchannel. b1e6580 fish: const-correctness fixes in copy.c f6246e9 Version 1.5.11. 0255835 Update OCaml dependencies. 646813e Update PO files. 878c8c7 fish: Fix 'copy-out' command when local directory is "/foo". 3578f17 fish: Fix typo in documentation of copy-out. 8902b33 Version 1.5.10. f383ac1 todo: Remove discussion of copy-in/copy-out. 2635a9c fish: Implement copy-in and copy-out commands. 43eed09 New APIs: is-chardev, is-blockdev, is-fifo, is-symlink, is-socket 3a99114 daemon: Move 'exists', 'is-file' and 'is-dir' to separate file. 55b6e18 generator: Fix incorrect shortdesc in docs for 'is-dir' command. 22aa926 generator: Fix documentation for 'is-file' command. 639ca18 fish: Fix 'more' command to work with any file. b5c287b fish: Fix 'edit' command to work with any file. 13be761 guestfs: Reference guestfs-browser architecture in threads documentation. b1454e3 guestfs: Document progress notification messages in protocol. e2ef068 guestfs: More accurate documentation for initial message. 6d9f8f5 guestfs: Remove traces of documentation for non-existent 'low-level API' f3c05da guestfs: Fix typo in man page. 979bcc5 todo: Notes on virt copy command. 6a87e0a Bring TODO file up to date. c51ade2 Version 1.5.9. d5c8d3b fish: Add guestfish -N bootroot and -N bootrootlv for creating boot+root disks. 574df37 fish: Add guestfish -N lvfs for creating formatted LVs. 422a8d8 fish: Add guestfish -N lv for creating disks with LVs. 45f72c8 fish: Improve appearance of guestfish -N help output. fa918b1 fish: Allow guestfish -N help for listing prepared disk image help. 60cdd02 fish: Generate list of prepared disk image types. 06c9061 test-tool: Add a 'run-test-tool-locally' script. b762848 appliance: init script does 'ls -lR /dev' (verbose only) aa96881 daemon: Don't warn about 'long long' usage. bd77c8d php: Remove 'make clean' rule in subdirectory. 56da696 Version 1.5.8. 2c61e04 PHP bindings. 2d8fd7d Define LIBGUESTFS_HAVE_<shortname> for C API functions. 5fc69ce build: guestfs-structs.h was missing from libguestfs_la_SOURCES. 8ad79a7 Add full docs pot file. 8c48f5a Allow manual pages and POD files to be translated. 1193df7 Add Dutch translation (RHBZ#629593). 25521f1 ruby: Add Guestfs::Guestfs.new() method. b8b0c99 perl: Document handle is a hashref. 0f24424 perl: Add documentation about testing availability of methods and features. 71f5cc5 build: Link static -ltinfo into guestfish.static binary. d8daebf Version 1.5.7. 2310514 fish: Add missing header file to sources. d1485e0 ocaml: Add test for progress notification callbacks. 8c37961 debug: Add 'debug progress' command. 99679fe debug: Arrange prototypes in alphabetical order. 83f381f daemon: Enable debug command by default. 867319e Consistent use of 'void *opaque' to refer to opaque pointer in C API. c9bc865 resize: Add progress bar to virt-resize. 8ff17e1 perl: bindings to progress callback. 3978a30 ocaml: bindings to progress callback. 4ea0abf Implement private data area. 54837f6 fish: Implement progress bars in guestfish. 3003df6 fish: Detect UTF-8 output and open termcap/terminfo database. 6551096 Add progress messages to download command. 133a92b Add progress messages to zero-device command. 88ab203 Add progress messages to zero command. 5593840 Add progress messages to fill-pattern command. 7f1ecfc Add progress messages to fill command. 41512f4 Add progress messages to copy-size command. e776a46 Implement progress messages in the daemon and library. a8a44ce ocaml: Remove old entry from .gitignore file. 377926b Version 1.5.6. d46230f Requires febootstrap >= 2.9. d89a88d daemon: Set O_CLOEXEC flag on the virtio-serial file descriptor. 4932fdc build: Don't add version extra string to the version number. 1c52376 Update BUGS, PO files. b59aff0 Update Spanish translations (RHBZ#627556). 3f7965b Updated Spanish translations (RHBZ#626843). 9bc72e2 Update Polish translation (RHBZ#502533). 92876a4 Prepare for version 1.5.5. daead56 rescue: Fix typo in comment. c3194e4 Ignore launch() error in virt-rescue. (RHBZ#618556) d3fc7e1 Shut down the appliance cleanly c0b38fb Call sync after guestfsd exits a45302c Add a core_pattern debug command a0d514f Include statically linked binaries in the binary distribution. 27ef6f9 Rename global 'xdr_str'. 6d15d4e Add -nodefconfig command line option to qemu. e503b31 Version 1.5.4. 4963be8 New APIs: set-network and get-network to enable network support. 10d1aa3 build: Add 'bindist' rule for building binary distribution. 5c1346d Don't print debug messages when not in verbose mode. 831b1fc Change protocol to send Linux errno from daemon to library. 90d06e2 Raise error message max size to 64K. 866ec00 Use virtio-serial, remove other vmchannel methods. 5c31f61 Change to using ext2-based, cached supermin appliance. 4b753c6 Make print_timestamped_message into a cross-module function. a2d4a8b Factor out code for locating the temporary directory. 4d2f163 Whitespace change: Add blank line between structures and functions. c56fa58 appliance: Remove some obsolete testing rules from Makefile.am 83c2217 Version 1.5.3 ad373a4 Remove old ocaml-inspector code. 4440e22 fish: Reimplement -i option using new C-based inspection. 1a9aa56 fish: Add -c/--connect and -d/--domain options. 8289aa1 New APIs for guest inspection. 65e9ac4 New APIs: findfs-label and findfs-uuid ad4cff2 New API: file-architecture 3cd272f generator: No need to redefine safe_* macros. 3905cc7 Add safe_strndup call. 64d7022 Send trace output to stderr. 8340779 Python: Use new PyCapsule API where supported. 2992524 generator: Fix typo in error message for RConstOptString. 6280ac9 New API: is-lv: check if a block device is a logical volume (RHBZ#619793) 737181b Rename internal functions. 41f25ab Rearrange library code into separate files. a617f52 Rename guestfs-{actions,bindtests}.c to {actions,bindtests}.c d30176d TODO: Implement inspector code in C. b42d34b TODO: Implement recursive upload/download in guestfish. 4a7d8cd TODO: Add link to discussion of progress bars. cd773b0 Version 1.5.2. e7ee6ee regressions: Don't print misleading 'Expect error ...' lines. 945e569 New APIs: Support for creating LUKS and managing keys. 2fd8c25 Move variable initialization close to variable use. 799d52b Revert "add_drive_ro adds readonly=on option if available." (RHBZ#617200). 2a286f1 generator: Make documentation inside guestfish match man page. e67298c Version 1.5.1. 637f8df New APIs: Support for opening LUKS-encrypted disks. 581a796 generator: Add 'Key' parameter type. 2e7da2a generator: Remove unnecessary parameter. 45a4cd7 df: Minimize the number of times we launch the libguestfs appliance. d2cf9a1 New APIs: lvm-set-filter and lvm-clear-filter. 321ca1e Use an unsigned type (size_t) for all loop iterators. aac5194 generator: Don't hard-code name in DeviceList check. 0c09764 build: Don't warn about 'long long'. 5b77be7 doc: Add guestfish 'lvcreate 1M' gotcha. 70d27f6 tar: Remove redundant use statement. 10ea14a edit: Clean up temporary files. fed8714 edit: Add -b (backup) option and make uploading more robust. 8f6d8b0 edit: Add -e 'expr' option to non-interactively apply expression to the file. 58667f2 Prepare for new development branch, starting at 1.5.0. 49a71a4 Make tmp directory world readable (RHBZ#610880). Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into Xen guests. http://et.redhat.com/~rjones/virt-p2v

1 3

[PATCH 0/4] Fixes for virt-resize
by Richard W.M. Jones 01 Oct '10

01 Oct '10

Fixes for both of these bugs: https://bugzilla.redhat.com/show_bug.cgi?id=633096 virt-resize calculates block device size incorrectly, doesn't work with qcow2 target https://bugzilla.redhat.com/show_bug.cgi?id=633766 virt-resize --shrink fails I'm still doing testing on these, but the patches seem good enough to review. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://et.redhat.com/~rjones/virt-top

1 7

[PATCH] Fix error launching libguestfs when euid != uid
by Matthew Booth 30 Sep '10

30 Sep '10

When writing to a RHEV target, virt-v2v launches the libguestfs appliance with euid:egid = 36:36, which is required to write to an NFS target using root_squash. Since the update to use an febootstrap cached appliance, this causes an error on startup as the cached files are owned by root, but the cache directory is owned by 36:36. The reason for this is that execve() resets euid and egid to uid and gid respectively, so when febootstrap-supermin-helper is executed, it runs as root:root. The cache directory, however, is created by libguestfs directly without exec()ing another program, so it has the correct ownership. This patch fixes this issue by setting real uid and gid to euid and egid respectively before exec()ing the helper program. --- src/appliance.c | 134 +++++++++++++++++++++++++++++++++++++++++++++++++++---- 1 files changed, 125 insertions(+), 9 deletions(-) diff --git a/src/appliance.c b/src/appliance.c index 3c3279b..be5e167 100644 --- a/src/appliance.c +++ b/src/appliance.c @@ -27,6 +27,7 @@ #include <time.h> #include <sys/stat.h> #include <sys/select.h> +#include <sys/wait.h> #include <utime.h> #ifdef HAVE_SYS_TYPES_H @@ -49,6 +50,7 @@ static int contains_ordinary_appliance (guestfs_h *g, const char *path, void *da static char *calculate_supermin_checksum (guestfs_h *g, const char *supermin_path); static int check_for_cached_appliance (guestfs_h *g, const char *supermin_path, const char *checksum, char **kernel, char **initrd, char **appliance); static int build_supermin_appliance (guestfs_h *g, const char *supermin_path, const char *checksum, char **kernel, char **initrd, char **appliance); +static pid_t fork_helper(guestfs_h *g); /* Locate or build the appliance. * @@ -170,19 +172,59 @@ calculate_supermin_checksum (guestfs_h *g, const char *supermin_path) if (g->verbose) guestfs___print_timestamped_message (g, "%s", cmd); - /* Errors here are non-fatal, so we don't need to call error(). */ - FILE *pp = popen (cmd, "r"); - if (pp == NULL) + int fds[2]; + if (pipe(fds) == -1) { + error (g, _("pipe failed: %m")); return NULL; + } + + int pid = fork_helper(g); + if (pid == -1) { + return NULL; + } + + /* exec febootstrap-supermin-helper in the child */ + if (pid == 0) { + /* Close the read end */ + if(close(fds[0]) == -1) { + perror("close read end"); + exit(1); + } + + /* dup2 the write end to stdout */ + if(dup2(fds[1], STDOUT_FILENO) == -1) { + perror("dup2"); + exit(1); + } + + /* Close the write end */ + if(close(fds[1]) == -1) { + perror("close write end"); + exit(1); + } + + char *const argv[] = { strdup("/bin/sh"), strdup("-c"), cmd, NULL }; + if (execv("/bin/sh", argv) == -1) { + perror("execv"); + exit(1); + } + } + + FILE *pp = fdopen(fds[0], "r"); + if (pp == NULL) { + perror("fdopen"); + goto helper_error; + } char checksum[256]; if (fgets (checksum, sizeof checksum, pp) == NULL) { - pclose (pp); - return NULL; + fclose (pp); + goto helper_error; } - if (pclose (pp) == -1) { - perror ("pclose"); + fclose (pp); + if (waitpid(pid, NULL, 0) == -1) { + perror ("waitpid"); return NULL; } @@ -197,6 +239,18 @@ calculate_supermin_checksum (guestfs_h *g, const char *supermin_path) checksum[--len] = '\0'; return safe_strndup (g, checksum, len); + + int dummy[128]; +helper_error: + /* Consume any remaining output, ignoring errors */ + while (read(fds[0], dummy, sizeof(dummy) > 0)) + ; + + if (waitpid(pid, NULL, 0) == -1) { + perror("reaping feboostrap-supermin-helper"); + } + + return NULL; } /* Check for cached appliance in $TMPDIR/$checksum. Check it exists @@ -357,8 +411,28 @@ build_supermin_appliance (guestfs_h *g, cachedir, cachedir, cachedir); if (g->verbose) guestfs___print_timestamped_message (g, "%s", cmd); - int r = system (cmd); - if (r == -1 || WEXITSTATUS (r) != 0) { + + pid_t pid = fork_helper(g); + if (pid == -1) { + return -1; + } + + /* exec febootstrap-supermin-helper in the child */ + if (pid == 0) { + char * const argv[] = { strdup("/bin/sh"), strdup("-c"), cmd, NULL }; + if (execv("/bin/sh", argv) == -1) { + perror("execv"); + exit(1); + } + } + + int r; + if (waitpid(pid, &r, 0) == -1) { + error (g, _("error waiting for command: %s (%m)"), cmd); + return -1; + } + + if (WEXITSTATUS (r) != 0) { error (g, _("external command failed: %s"), cmd); return -1; } @@ -463,3 +537,45 @@ dir_contains_files (const char *dir, ...) va_end (args); return 1; } + +/* Launch may be called by a seteuid/setegid process (virt-v2v does this). + * Unfortunately, execve resets EGID/EUID to GID/UID. This means that files + * created by a subprocess will have the wrong ownership. To work round this, + * we set the real GID/UID first before exec. */ +static pid_t fork_helper(guestfs_h *g) +{ + pid_t pid = fork(); + if (pid == -1) { + error (g, _("fork failed: %m")); + return -1; + } + + /* Set euid/egid in the child. + * Note that the child just needs to exit with an error, not return + */ + if (pid == 0) { + if (getuid() == 0) { + int egid = getegid(); + int euid = geteuid(); + + if (egid != 0 || euid != 0) { + if (seteuid(0) == -1) { + perror("seteuid"); + exit(1); + } + + if (setgid(egid) == -1) { + perror("setgid"); + exit(1); + } + + if (setuid(euid) == -1) { + perror("setuid"); + exit(1); + } + } + } + } + + return pid; +} -- 1.7.2.3

2 8

Re: [Libguestfs] [parted-devel] "Error: Can't have a partition outside the disk!"
by Richard W.M. Jones 29 Sep '10

29 Sep '10

On Wed, Sep 29, 2010 at 05:28:33PM +0200, Karel Zak wrote: > On Wed, Sep 29, 2010 at 02:55:59PM +0100, Richard W.M. Jones wrote: > > On Wed, Sep 29, 2010 at 11:11:48AM +0200, Karel Zak wrote: > > > On Sun, Sep 26, 2010 at 05:42:21PM +0100, Richard W.M. Jones wrote: > > > > In any case I'll probably write a small utility which just zaps the > > > > partition table without touching the rest of the bootloader. > > > > > > Maybe we can add this functionality to wipefs(8), libblkid is already > > > able to detect and parse almost all partition table formats. > > > > Coincidentally I tried using wipefs within libguestfs (but not > > successfully yet). It really only wipes filesystems, which I guess > > should be obvious from the name. I think it should be extended to > > wipe other stuff to: LVM, partition tables, RAID metadata being three > > obvious ones. > > The latest version (F-14) also supports RAIDs and LVM. > > Karel > > > > $ wipefs /home/images/filesystems/mdraid.img > > offset type > ---------------------------------------------------------------- > 0x9f0000 linux_raid_member [raid] > UUID: 37c76b91-011a-05c5-d30c-1fd4c5c3dbbc > > > $ wipefs /home/images/filesystems/lvm2.img > offset type > ---------------------------------------------------------------- > 0x218 LVM2_member [raid] > UUID: Vynv4k-APH8-xQER-HSBb-8VJ3-SvFF-PB5O1U OK, that's good to know. I'll add a wipefs wrapper to libguestfs at some point. It seems a more reliable way to do it than our current method: http://libguestfs.org/guestfs.3.html#guestfs_zero Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones New in Fedora 11: Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 70 libraries supprt'd http://fedoraproject.org/wiki/MinGW http://www.annexia.org/fedora_mingw

1 0

[PATCH 0/2] Remove troublesome devices when outputting to libvirt
by Matthew Booth 27 Sep '10

27 Sep '10

These 2 patches remove devices which can prevent a guest from starting after conversion if they aren't supported by the target. This solution isn't ideal. We would preferrably query the target for supported models and update if necessary, however there's no API for this yet. Matthew Booth (2): Remove converted SCSI controllers Remove sound cards when outputting to libvirt lib/Sys/VirtV2V/Target/LibVirt.pm | 22 ++++++++++++++++++++++ 1 files changed, 22 insertions(+), 0 deletions(-) -- 1.7.2.3

2 4

[PATCH 0/2] Add pwrite-device API call
by Richard W.M. Jones 26 Sep '10

26 Sep '10

These two very simple patches add a pwrite-device API call. I have been conservative and not yet added the equivalent pread-device call, although that could be added in future. The motivation for this is in virt-resize: We need a way to zap the partition table cleanly[1], and obviously parted isn't working out for us[2]. Using this call we can zap MBR and GPT partition tables by overwriting them with zeroes (or zeroes + signature), and I have shown that this works at least for MBR. Although the semantics of pwrite and pwrite-device are not guaranteed full write, they almost certainly do this now and we could add this extra constraint in future. Rich. [1] https://bugzilla.redhat.com/show_bug.cgi?id=633766 [2] http://lists.alioth.debian.org/pipermail/parted-devel/2010-September/003748… -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-df lists disk usage of guests without needing to install any software inside the virtual machine. Supports Linux and Windows. http://et.redhat.com/~rjones/virt-df/

1 3

[PREVIEW ONLY] Refactor data transfer code
by Matthew Booth 22 Sep '10

22 Sep '10

This patch refactors the data transfer code with several goals: * Have a common read(source)/write(target) loop so that common processing can happen in the middle of it, e.g. format change/progress bar * Provide volume metadata to transfers to allow smarter reading/writing, e.g. of sparse files * Simplify the data transfer code The patch *isn't* NFC because there are some minor behaviour changes, but it isn't intended to provide any new features either. For example, although sparse info is passed around, nothing actually uses it yet. This patch *may* allow qcow2->qcow2 conversions, although I haven't tested this. A later patch will enable qcow2->raw and raw->qcow2 conversions. Metadata sources where previously 'Connection's. Targets now become 'Connection's too. They both supply Volume objects. New objects are: Volume Holds metadata on a specific object, and a handle to a Transfer object which can read/write it. Transfer Provides a read or write stream to an underlying volume. This will also be extended to provide a file interface. ReadStream/WriteStream Provides read/write streaming access to a volume's data. --- MANIFEST | 12 +- lib/Sys/VirtV2V/Connection/LibVirt.pm | 193 ++------ lib/Sys/VirtV2V/Connection/LibVirtSource.pm | 247 +++++++++ .../LibVirt.pm => Connection/LibVirtTarget.pm} | 228 +++------ .../{LibVirtXML.pm => LibVirtXMLSource.pm} | 81 ++- .../{Target/RHEV.pm => Connection/RHEVTarget.pm} | 520 +++++++++++-------- .../{Connection.pm => Connection/Source.pm} | 116 ++++-- lib/Sys/VirtV2V/Connection/Volume.pm | 179 +++++++ lib/Sys/VirtV2V/Transfer/ESX.pm | 533 ++++++++++++-------- lib/Sys/VirtV2V/Transfer/Local.pm | 238 +++++++++ lib/Sys/VirtV2V/Transfer/LocalCopy.pm | 150 ------ lib/Sys/VirtV2V/Transfer/SSH.pm | 396 ++++++++++----- lib/Sys/VirtV2V/Util.pm | 41 ++- v2v/virt-v2v.pl | 92 ++-- 14 files changed, 1905 insertions(+), 1121 deletions(-) create mode 100644 lib/Sys/VirtV2V/Connection/LibVirtSource.pm rename lib/Sys/VirtV2V/{Target/LibVirt.pm => Connection/LibVirtTarget.pm} (70%) rename lib/Sys/VirtV2V/Connection/{LibVirtXML.pm => LibVirtXMLSource.pm} (57%) rename lib/Sys/VirtV2V/{Target/RHEV.pm => Connection/RHEVTarget.pm} (75%) rename lib/Sys/VirtV2V/{Connection.pm => Connection/Source.pm} (60%) create mode 100644 lib/Sys/VirtV2V/Connection/Volume.pm create mode 100644 lib/Sys/VirtV2V/Transfer/Local.pm delete mode 100644 lib/Sys/VirtV2V/Transfer/LocalCopy.pm diff --git a/MANIFEST b/MANIFEST index 4c8c5c2..a5c72be 100644 --- a/MANIFEST +++ b/MANIFEST @@ -5,8 +5,12 @@ COPYING COPYING.LIB lib/Sys/VirtV2V/Config.pm lib/Sys/VirtV2V/Connection/LibVirt.pm -lib/Sys/VirtV2V/Connection/LibVirtXML.pm -lib/Sys/VirtV2V/Connection.pm +lib/Sys/VirtV2V/Connection/LibVirtSource.pm +lib/Sys/VirtV2V/Connection/LibVirtTarget.pm +lib/Sys/VirtV2V/Connection/LibVirtXMLSource.pm +lib/Sys/VirtV2V/Connection/RHEVTarget.pm +lib/Sys/VirtV2V/Connection/Source.pm +lib/Sys/VirtV2V/Connection/Volume.pm lib/Sys/VirtV2V/Converter/Linux.pm lib/Sys/VirtV2V/Converter.pm lib/Sys/VirtV2V/Converter/Windows.pm @@ -15,10 +19,8 @@ lib/Sys/VirtV2V/GuestfsHandle.pm lib/Sys/VirtV2V/GuestOS.pm lib/Sys/VirtV2V/GuestOS/RedHat.pm lib/Sys/VirtV2V.pm -lib/Sys/VirtV2V/Target/LibVirt.pm -lib/Sys/VirtV2V/Target/RHEV.pm lib/Sys/VirtV2V/Transfer/ESX.pm -lib/Sys/VirtV2V/Transfer/LocalCopy.pm +lib/Sys/VirtV2V/Transfer/Local.pm lib/Sys/VirtV2V/Transfer/SSH.pm lib/Sys/VirtV2V/Util.pm MANIFEST.SKIP diff --git a/lib/Sys/VirtV2V/Connection/LibVirt.pm b/lib/Sys/VirtV2V/Connection/LibVirt.pm index 51331da..7d634ff 100644 --- a/lib/Sys/VirtV2V/Connection/LibVirt.pm +++ b/lib/Sys/VirtV2V/Connection/LibVirt.pm @@ -1,5 +1,5 @@ # Sys::VirtV2V::Connection::LibVirt -# Copyright (C) 2009 Red Hat Inc. +# Copyright (C) 2009,2010 Red Hat Inc. # # This library is free software; you can redistribute it and/or # modify it under the terms of the GNU Lesser General Public @@ -20,8 +20,6 @@ package Sys::VirtV2V::Connection::LibVirt; use strict; use warnings; -use Sys::VirtV2V::Connection; - use Net::Netrc; use URI; use XML::DOM; @@ -29,6 +27,11 @@ use XML::DOM; use Sys::Virt; use Sys::VirtV2V; +use Sys::VirtV2V::Connection; +use Sys::VirtV2V::Connection::Volume; +use Sys::VirtV2V::Transfer::ESX; +use Sys::VirtV2V::Transfer::SSH; +use Sys::VirtV2V::Transfer::Local; use Sys::VirtV2V::Util qw(user_message); use Locale::TextDomain 'virt-v2v'; @@ -39,70 +42,47 @@ use Locale::TextDomain 'virt-v2v'; =head1 NAME -Sys::VirtV2V::Connection::LibVirt - Read libvirt metadata from libvirtd - -=head1 SYNOPSIS - - use Sys::VirtV2V::Connection::LibVirt; - - $conn = Sys::VirtV2V::Connection::LibVirt->new - ("xen+ssh://xenserver.example.com/", $name, $target); - $dom = $conn->get_dom(); +Sys::VirtV2V::Connection::LibVirt - Access storage and metadata from libvirt =head1 DESCRIPTION -Sys::VirtV2V::Connection::LibVirt is an implementation of -Sys::VirtV2V::Connection which reads a guest's libvirt XML directly from a -libvirt connection. - -=head1 METHODS - -=over - -=item new(uri, name, target) - -Create a new Sys::VirtV2V::Connection::LibVirt. Domain I<name> will be -obtained from I<uri>. Remote storage will be create on I<target>. +Do not use C<Sys::VirtV2V::Connection::LibVirt> directly. Instead use either +C<Sys::VirtV2V::Connection::LibVirtSource> or +C<Sys::VirtV2V::Connection::LibVirtTarget>. =cut -sub new +sub _libvirt_new { my $class = shift; - - my ($uri, $name, $target) = @_; + my ($uri) = @_; my $self = {}; - bless($self, $class); - $self->{uri} = URI->new($uri); - $self->{name} = $name; - - # Check that the guest doesn't already exist on the target - die(user_message(__x("Domain {name} already exists on the target.", - name => $name))) if ($target->guest_exists($name)); + $self->{uri} = $uri = URI->new($uri); # Parse uri authority for hostname and username - $self->{uri}->authority() =~ /^(?:([^:]*)(?::([^@]*))?@)?(.*)$/ + $uri->authority() =~ /^(?:([^:]*)(?::([^@]*))?@)?(.*)$/ or die(user_message(__x("Unable to parse URI authority: {auth}", - auth => $self->{uri}->authority()))); - - my $username = $self->{username} = $1; - my $hostname = $self->{hostname} = $3; + auth => $uri->authority()))); warn user_message(__"WARNING: Specifying a password in the connection URI ". - "is not supported. It has been ignored.") if (defined($2)); + "is not supported. It has been ignored.") + if (defined($2)); + + $self->{username} = $1; + $self->{hostname} = $3; # Look for credentials in .netrc if the URI contains a hostname - if (defined($hostname)) { - if (defined($username)) { - my $mach = Net::Netrc->lookup($hostname, $username); + if (defined($self->{hostname})) { + if (defined($self->{username})) { + my $mach = Net::Netrc->lookup($self->{hostname}, $self->{username}); $self->{password} = $mach->password if (defined($mach)); } else { - my $mach = Net::Netrc->lookup($hostname); + my $mach = Net::Netrc->lookup($self->{hostname}); if (defined($mach)) { $self->{username} = $mach->login; @@ -111,11 +91,10 @@ sub new } } - my $sourcevmm; + my $vmm; eval { - $sourcevmm = Sys::Virt->new( + $vmm = Sys::Virt->new( uri => $uri, - readonly => 1, auth => 1, credlist => [ Sys::Virt::CRED_AUTHNAME, @@ -142,117 +121,41 @@ sub new uri => $uri, error => $@->stringify()))) if ($@); - $self->{sourcevmm} = $sourcevmm; - - $self->_check_shutdown(); - - $self->_get_dom(); - - my $transfer; - if ($self->{uri}->scheme eq "esx") { - $transfer = "Sys::VirtV2V::Transfer::ESX"; - } - - elsif ($self->{uri}->scheme =~ /\+ssh$/) { - $transfer = "Sys::VirtV2V::Transfer::SSH"; - } - - # Default to LocalCopy - # XXX: Need transfer methods for remote libvirt connections, e.g. scp - else { - $transfer = "Sys::VirtV2V::Transfer::LocalCopy"; - } - - $self->_storage_iterate($transfer, $target); + $self->{vmm} = $vmm; return $self; } -sub _check_shutdown +sub _get_transfer { my $self = shift; + my ($path, $is_sparse) = @_; - my $vmm = $self->{vmm}; - my $domain = $self->_get_domain(); + my $uri = $self->{uri}; - # Check the domain is shutdown - die(user_message(__x("Guest {name} is currently {state}. It must be ". - "shut down first.", - state => _state_string($domain->get_info()->{state}), - name => $self->{name}))) - unless ($domain->get_info()->{state} == - Sys::Virt::Domain::STATE_SHUTOFF); -} + if ($uri->scheme eq "esx") { + my %query = $uri->query_form; + my $noverify = $query{no_verify} eq "1" ? 1 : 0; -sub _state_string -{ - my ($state) = @_; - - if ($state == Sys::Virt::Domain::STATE_NOSTATE) { - return __"idle"; - } elsif ($state == Sys::Virt::Domain::STATE_RUNNING) { - return __"running"; - } elsif ($state == Sys::Virt::Domain::STATE_BLOCKED) { - return __"blocked"; - } elsif ($state == Sys::Virt::Domain::STATE_PAUSED) { - return __"paused"; - } elsif ($state == Sys::Virt::Domain::STATE_SHUTDOWN) { - return __"shutting down"; - } elsif ($state == Sys::Virt::Domain::STATE_SHUTOFF) { - return __"shut down"; - } elsif ($state == Sys::Virt::Domain::STATE_CRASHED) { - return __"crashed"; - } else { - return "unknown state ($state)"; + return new Sys::VirtV2V::Transfer::ESX($path, + $self->{hostname}, + $self->{username}, + $self->{password}, + $noverify, + $is_sparse); } -} - -sub _get_domain -{ - my $self = shift; - - return $self->{domain} if (defined($self->{domain})); - - my $vmm = $self->{sourcevmm}; - my $name = $self->{name}; - - # Lookup the domain - my $domain; - eval { - $domain = $vmm->get_domain_by_name($name); - }; - die($@) if ($@); - - # Check we found it - die(user_message(__x("{name} isn't a valid guest name", name => $name))) - unless($domain); - $self->{domain} = $domain; - - return $domain; -} - -sub _get_dom -{ - my $self = shift; - - my $vmm = $self->{vmm}; - my $name = $self->{name}; - - # Lookup the domain - my $domain = $self->_get_domain(); - - # Warn and exit if we didn't find it - return undef unless(defined($domain)); - - my $xml = $domain->get_xml_description(); + elsif ($uri->scheme =~ /\+ssh$/) { + return new Sys::VirtV2V::Transfer::SSH($path, + $self->{hostname}, + $self->{username}, + $is_sparse); + } - my $dom = new XML::DOM::Parser->parse($xml); - $self->{dom} = $dom; + # Default to Local + return new Sys::VirtV2V::Transfer::Local($path, $is_sparse); } -=back - =head1 COPYRIGHT Copyright (C) 2009,2010 Red Hat Inc. @@ -264,6 +167,8 @@ Please see the file COPYING.LIB for the full license. =head1 SEE ALSO L<Sys::VirtV2V::Connection(3)>, +L<Sys::VirtV2V::Connection::LibVirtSource(3)>, +L<Sys::VirtV2V::Connection::LibVirtTarget(3)>, L<virt-v2v(1)>, L<http://libguestfs.org/>. diff --git a/lib/Sys/VirtV2V/Connection/LibVirtSource.pm b/lib/Sys/VirtV2V/Connection/LibVirtSource.pm new file mode 100644 index 0000000..701d8f2 --- /dev/null +++ b/lib/Sys/VirtV2V/Connection/LibVirtSource.pm @@ -0,0 +1,247 @@ +# Sys::VirtV2V::Connection::LibVirt +# Copyright (C) 2009,2010 Red Hat Inc. +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + +package Sys::VirtV2V::Connection::LibVirtSource; + +use strict; +use warnings; + +use URI; +use XML::DOM; + +use Sys::Virt; + +use Sys::VirtV2V; +use Sys::VirtV2V::Connection::Source; +use Sys::VirtV2V::Connection::LibVirt; +use Sys::VirtV2V::Transfer::ESX; +use Sys::VirtV2V::Util qw(user_message parse_libvirt_volinfo); + +use Locale::TextDomain 'virt-v2v'; + +@Sys::VirtV2V::Connection::LibVirtSource::ISA = + qw(Sys::VirtV2V::Connection::Source Sys::VirtV2V::Connection::LibVirt); + +=pod + +=head1 NAME + +Sys::VirtV2V::Connection::LibVirtSource - Get storage and metadata from libvirt + +=head1 SYNOPSIS + + use Sys::VirtV2V::Connection::LibVirtSource; + + $conn = Sys::VirtV2V::Connection::LibVirtSource->new + ("xen+ssh://xenserver.example.com/", $name); + $dom = $conn->get_dom(); + +=head1 DESCRIPTION + +Sys::VirtV2V::Connection::LibVirtSource reads a guest's libvirt XML directly +from a libvirt connection. It accesses the guest's storage over the same +transport as the libvirt connection. + +=head1 METHODS + +=over + +=item new(uri, name) + +Create a new libvirt source connection. Domain I<name> will be obtained from +I<uri>. + +=cut + +sub new +{ + my $class = shift; + my ($uri, $name) = @_; + + my $self = $class->SUPER::_libvirt_new($uri); + + $self->{name} = $name; + + $self->_check_shutdown(); + $self->_get_dom(); + + return $self; +} + +=item get_name + +Return the name of the domain. + +=cut + +sub get_name +{ + return shift->{name}; +} + +=item get_volume(path) + +Return a volume object for I<path>, where I<path> is the path of a volume on the +connected hypervisor. + +=cut + +sub get_volume +{ + my $self = shift; + my ($path) = @_; + + my $uri = $self->{uri}; + + my ($name, $format, $size, $is_sparse, $is_block); + + # The libvirt storage APIs aren't yet reliably implemented for ESX, so we + # need to get volume metadata some other way + my $transfer; + if ($uri->scheme eq "esx") { + $transfer = $self->_get_transfer($path, 0); + + $name = $transfer->esx_get_name(); + $format = "raw"; + $size = $transfer->esx_get_size(); + $is_sparse = 0; + $is_block = 0; + } + + else { + my $vol; + eval { + $vol = $self->{vmm}->get_storage_volume_by_path($path); + }; + die(user_message(__x("Failed to retrieve storage volume {path}:". + "{error}", + path => $path, + error => $@->stringify()))) if($@); + + ($name, $format, $size, $is_sparse, $is_block) = + parse_libvirt_volinfo($vol); + + $transfer = $self->_get_transfer($path, $is_sparse); + } + + return new Sys::VirtV2V::Connection::Volume($name, $format, $path, $size, + $is_sparse, $is_block, + $transfer); +} + +sub _check_shutdown +{ + my $self = shift; + + my $vmm = $self->{vmm}; + my $domain = $self->_get_domain(); + + # Check the domain is shutdown + die(user_message(__x("Guest {name} is currently {state}. It must be ". + "shut down first.", + state => _state_string($domain->get_info()->{state}), + name => $self->{name}))) + unless ($domain->get_info()->{state} == + Sys::Virt::Domain::STATE_SHUTOFF); +} + +sub _state_string +{ + my ($state) = @_; + + if ($state == Sys::Virt::Domain::STATE_NOSTATE) { + return __"idle"; + } elsif ($state == Sys::Virt::Domain::STATE_RUNNING) { + return __"running"; + } elsif ($state == Sys::Virt::Domain::STATE_BLOCKED) { + return __"blocked"; + } elsif ($state == Sys::Virt::Domain::STATE_PAUSED) { + return __"paused"; + } elsif ($state == Sys::Virt::Domain::STATE_SHUTDOWN) { + return __"shutting down"; + } elsif ($state == Sys::Virt::Domain::STATE_SHUTOFF) { + return __"shut down"; + } elsif ($state == Sys::Virt::Domain::STATE_CRASHED) { + return __"crashed"; + } else { + return "unknown state ($state)"; + } +} + +sub _get_domain +{ + my $self = shift; + + return $self->{domain} if (defined($self->{domain})); + + my $vmm = $self->{vmm}; + my $name = $self->{name}; + + # Lookup the domain + my $domain; + eval { + $domain = $vmm->get_domain_by_name($name); + }; + die($@) if ($@); + + # Check we found it + die(user_message(__x("{name} isn't a valid guest name", name => $name))) + unless($domain); + + $self->{domain} = $domain; + + return $domain; +} + +sub _get_dom +{ + my $self = shift; + + my $vmm = $self->{vmm}; + my $name = $self->{name}; + + # Lookup the domain + my $domain = $self->_get_domain(); + + # Warn and exit if we didn't find it + return undef unless(defined($domain)); + + my $xml = $domain->get_xml_description(); + + my $dom = new XML::DOM::Parser->parse($xml); + $self->{dom} = $dom; +} + +=back + +=head1 COPYRIGHT + +Copyright (C) 2009,2010 Red Hat Inc. + +=head1 LICENSE + +Please see the file COPYING.LIB for the full license. + +=head1 SEE ALSO + +L<Sys::VirtV2V::Connection::Source(3)>, +L<virt-v2v(1)>, +L<http://libguestfs.org/>. + +=cut + +1; diff --git a/lib/Sys/VirtV2V/Target/LibVirt.pm b/lib/Sys/VirtV2V/Connection/LibVirtTarget.pm similarity index 70% rename from lib/Sys/VirtV2V/Target/LibVirt.pm rename to lib/Sys/VirtV2V/Connection/LibVirtTarget.pm index 279e93d..9a3b1c5 100644 --- a/lib/Sys/VirtV2V/Target/LibVirt.pm +++ b/lib/Sys/VirtV2V/Connection/LibVirtTarget.pm @@ -1,4 +1,4 @@ -# Sys::VirtV2V::Target::LibVirt +# Sys::VirtV2V::Connection::LibVirtTarget # Copyright (C) 2010 Red Hat Inc. # # This library is free software; you can redistribute it and/or @@ -18,161 +18,38 @@ use strict; use warnings; -package Sys::VirtV2V::Target::LibVirt::Vol; +package Sys::VirtV2V::Connection::LibVirtTarget; -use POSIX; - -use Sys::VirtV2V::Util qw(user_message); +use Sys::VirtV2V::Connection::LibVirt; +use Sys::VirtV2V::Util qw(user_message parse_libvirt_volinfo); use Locale::TextDomain 'virt-v2v'; -sub _new -{ - my $class = shift; - my ($vol) = @_; - - my $self = {}; - bless($self, $class); - - $self->{vol} = $vol; - - return $self; -} - -sub _create -{ - my $class = shift; - my ($pool, $name, $size) = @_; - - my $vol_xml = " - <volume> - <name>$name</name> - <capacity>$size</capacity> - </volume> - "; - - my $vol; - eval { - $vol = $pool->create_volume($vol_xml); - }; - die(user_message(__x("Failed to create storage volume: {error}", - error => $@->stringify()))) if ($@); - - return $class->_new($vol); -} - -sub _get -{ - my $class = shift; - my ($pool, $name) = @_; - - my $vol; - eval { - $vol = $pool->get_volume_by_name($name); - }; - die(user_message(__x("Failed to get storage volume: {error}", - error => $@->stringify()))) if ($@); - - return $class->_new($vol); -} - -sub get_path -{ - my $self = shift; - - return $self->{vol}->get_path(); -} - -sub get_format -{ - my $self = shift; - - my $vol = $self->{vol}; - my $voldom = new XML::DOM::Parser->parse($vol->get_xml_description()); - - my ($format) = $voldom->findnodes('/volume/target/format/@type'); - $format = $format->getValue() if (defined($format)); - $format ||= 'auto'; - - return $format; -} - -sub is_block -{ - my $self = shift; - - my $type = $self->{vol}->get_info()->{type}; - return $type == Sys::Virt::StorageVol::TYPE_BLOCK; -} - -sub open -{ - my $self = shift; - - my $path = $self->get_path(); - - # We want to open the existing volume without truncating it - sysopen(my $fd, $path, O_WRONLY) - or die(user_message(__x("Error opening storage volume {path} ". - "for writing: {error}", error => $!))); - - $self->{fd} = $fd; -} - -sub write -{ - my $self = shift; - my ($data) = @_; - - defined($self->{fd}) or die("write called without open"); - - syswrite($self->{fd}, $data) - or die(user_message(__x("Error writing to {path}: {error}", - path => $self->get_path(), - error => $!))); -} - -sub close -{ - my $self = shift; - - close($self->{fd}) - or die(user_message(__x("Error closing volume handle: {error}", - error => $!))); - - delete($self->{fd}); -} - -package Sys::VirtV2V::Target::LibVirt; - -use Sys::Virt; -use Sys::Virt::Error; - -use Sys::VirtV2V::Util qw(user_message); -use Locale::TextDomain 'virt-v2v'; +@Sys::VirtV2V::Connection::LibVirtTarget::ISA = + qw(Sys::VirtV2V::Connection::LibVirt); =head1 NAME -Sys::VirtV2V::Target::LibVirt - Output to libvirt +Sys::VirtV2V::Connection::LibVirtTarget - Output to libvirt =head1 SYNOPSIS - use Sys::VirtV2V::Target::LibVirt; + use Sys::VirtV2V::Connection::LibVirtTarget; - my $target = new Sys::VirtV2V::Target::LibVirt($uri, $poolname); + my $target = new Sys::VirtV2V::Connection::LibVirtTarget($uri, $poolname); =head1 DESCRIPTION -Sys::VirtV2V::Target::LibVirt creates a new libvirt domain using the given -target URI. New storage will be created in the target pool. +Sys::VirtV2V::Connection::LibVirtTarget creates a new libvirt domain using the +given target URI. New storage will be created in the target pool. =head1 METHODS =over -=item Sys::VirtV2V::Target::LibVirt->new(uri, poolname) +=item Sys::VirtV2V::Connection::LibVirtTarget->new(uri, poolname) -Create a new Sys::VirtV2V::Target::LibVirt object. +Create a new LibVirtTarget object. =over @@ -193,20 +70,14 @@ sub new my $class = shift; my ($uri, $poolname) = @_; - my $self = {}; - bless($self, $class); - - $self->{vmm} = Sys::Virt->new(auth => 1, uri => $uri); + my $self = $class->SUPER::_libvirt_new($uri); eval { $self->{pool} = $self->{vmm}->get_storage_pool_by_name($poolname); }; - - if ($@) { - die(user_message(__x("Output pool {poolname} is not a valid ". - "storage pool", - poolname => $poolname))); - } + die(user_message(__x("Output pool {poolname} is not a valid ". + "storage pool", + poolname => $poolname))) if ($@); return $self; } @@ -221,23 +92,56 @@ Create a new volume in the pool whose name was passed to new(). The name of the volume which is being created. +=item format + +The file format of the new volume. + =item size The size of the volume which is being created in bytes. +=item sparse + +1 if the output should be sparse if possible, 0 otherwise. + =back -create_volume() returns a Sys::VirtV2V::Target::LibVirt::Vol object. +create_volume() returns a Sys::VirtV2V::Connection::Volume object. =cut sub create_volume { my $self = shift; - my ($name, $size) = @_; + my ($name, $format, $size, $sparse) = @_; + + my $allocation = $sparse ? 0 : $size; + my $vol_xml = " + <volume> + <name>$name</name> + <capacity>$size</capacity> + <allocation>$allocation</allocation> + <target> + <format>$format</format> + </target> + </volume> + "; - return Sys::VirtV2V::Target::LibVirt::Vol->_create($self->{pool}, - $name, $size); + my $vol; + eval { + $vol = $self->{pool}->create_volume($vol_xml); + }; + die(user_message(__x("Failed to create storage volume: {error}", + error => $@->stringify()))) if ($@); + + my $info = $vol->get_info(); + my $is_block = $info->{type} == Sys::Virt::StorageVol::TYPE_BLOCK ? 1 : 0; + + my $transfer = $self->_get_transfer($vol->get_path(), $sparse); + return new Sys::VirtV2V::Connection::Volume($name, $format, + $vol->get_path(), $size, + $sparse, $is_block, + $transfer); } =item volume_exists (name) @@ -278,7 +182,8 @@ sub volume_exists =item get_volume (name) -Get a reference to an existing volume. See L<create_volume> for return value. +Get a reference to an existing volume. get_volume returns a +Sys::VirtV2V::Connection::Volume object. =cut @@ -287,7 +192,24 @@ sub get_volume my $self = shift; my ($name) = @_; - return Sys::VirtV2V::Target::LibVirt::Vol->_get($self->{pool}, $name); + my $uri = $self->{uri}; + my $pool = $self->{pool}; + + my $vol; + eval { + $vol = $pool->get_volume_by_name($name); + }; + die(user_message(__x("Failed to get storage volume: {error}", + error => $@->stringify()))) if ($@); + + my (undef, $format, $size, $is_sparse, $is_block) = + parse_libvirt_volinfo($vol); + + my $transfer = $self->_get_transfer($vol->get_path(), $is_sparse); + return new Sys::VirtV2V::Connection::Volume($name, $format, + $vol->get_path(), $size, + $is_sparse, $is_block, + $transfer); } =item guest_exists(name) @@ -317,7 +239,7 @@ sub guest_exists return 1; } -=item create_guest(dom) +=item create_guest(desc, dom, guestcaps) Create the guest in the target diff --git a/lib/Sys/VirtV2V/Connection/LibVirtXML.pm b/lib/Sys/VirtV2V/Connection/LibVirtXMLSource.pm similarity index 57% rename from lib/Sys/VirtV2V/Connection/LibVirtXML.pm rename to lib/Sys/VirtV2V/Connection/LibVirtXMLSource.pm index 7dc122f..25d705a 100644 --- a/lib/Sys/VirtV2V/Connection/LibVirtXML.pm +++ b/lib/Sys/VirtV2V/Connection/LibVirtXMLSource.pm @@ -1,5 +1,5 @@ -# Sys::VirtV2V::Connection::LibVirtXML -# Copyright (C) 2009 Red Hat Inc. +# Sys::VirtV2V::Connection::LibVirtXMLSource +# Copyright (C) 2009,2010 Red Hat Inc. # # This library is free software; you can redistribute it and/or # modify it under the terms of the GNU Lesser General Public @@ -15,48 +15,38 @@ # License along with this library; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA -package Sys::VirtV2V::Connection::LibVirtXML; +package Sys::VirtV2V::Connection::LibVirtXMLSource; use strict; use warnings; +use Sys::Virt; use XML::DOM; use XML::DOM::XPath; -use Sys::VirtV2V::Connection; -use Sys::VirtV2V::Util qw(user_message); +use Sys::VirtV2V::Connection::Source; +use Sys::VirtV2V::Transfer::Local; +use Sys::VirtV2V::Util qw(user_message parse_libvirt_volinfo); use Locale::TextDomain 'virt-v2v'; -@Sys::VirtV2V::Connection::LibVirtXML::ISA = qw(Sys::VirtV2V::Connection); +@Sys::VirtV2V::Connection::Source::LibVirtXMLSource::ISA = + qw(Sys::VirtV2V::Connection::Source); =pod =head1 NAME -Sys::VirtV2V::Connection::LibVirtXML - Read libvirt XML from a file - -=head1 SYNOPSIS - - use Sys::VirtV2V::Connection::LibVirtXML; - - $conn = Sys::VirtV2V::Connection::LibVirtXML->new($path, $target); - $dom = $conn->get_dom(); - -=head1 DESCRIPTION - -Sys::VirtV2V::Connection::LibVirtXML is an implementation of -Sys::VirtV2V::Connection which reads libvirt XML guest descriptions from a -file. +Sys::VirtV2V::Connection::Source::LibVirtXMLSource - Read domain XML from a file =head1 METHODS =over -=item new(path, target) +=item new(path) -Create a new LibVirtXML connection. The metadata itself is read from I<path>. -Storage will be copied to I<target>. +Create a new LibVirtXMLSource connection. The metadata itself is read from +I<path>. =cut @@ -73,12 +63,23 @@ sub new $self->_get_dom($path); - # Only support LocalCopy for libvirtxml - $self->_storage_iterate("Sys::VirtV2V::Transfer::LocalCopy", $target); - return $self; } +=item get_name + +Return the name of the domain. + +=cut + +sub get_name +{ + my $dom = shift->{dom}; + + my ($name) = $dom->findnodes('/domain/name'); + return $name; +} + sub _get_dom { my $self = shift; @@ -102,6 +103,32 @@ sub _get_dom path => $self->{path}))) unless (defined($dummy)); } +=item get_volume(path) + +Return a Sys::VirtV2V::Connection::Volume object for I<path>, where I<path> is +the path to a locally available volume. + +=cut + +sub get_volume +{ + my $self = shift; + my ($path) = @_; + + # Use a libvirt session connection to inspect local volumes + my $vmm = Sys::Virt->new(uri => 'qemu:///session'); + my $vol = $vmm->get_storage_volume_by_path($path); + + my ($name, $format, $size, $is_sparse, $is_block) = + parse_libvirt_volinfo($vol, $path); + + my $transfer = new Sys::VirtV2V::Transfer::Local($path, $is_sparse); + + return new Sys::VirtV2V::Connection::Volume($name, $format, $path, + $size, $is_sparse, $is_block, + $transfer); +} + =back =head1 COPYRIGHT @@ -114,7 +141,7 @@ Please see the file COPYING.LIB for the full license. =head1 SEE ALSO -L<Sys::VirtV2V::Connection(3pm>, +L<Sys::VirtV2V::Connection::Source(3pm>, L<virt-v2v(1)>, L<http://libguestfs.org/>. diff --git a/lib/Sys/VirtV2V/Target/RHEV.pm b/lib/Sys/VirtV2V/Connection/RHEVTarget.pm similarity index 75% rename from lib/Sys/VirtV2V/Target/RHEV.pm rename to lib/Sys/VirtV2V/Connection/RHEVTarget.pm index b865c56..765a035 100644 --- a/lib/Sys/VirtV2V/Target/RHEV.pm +++ b/lib/Sys/VirtV2V/Connection/RHEVTarget.pm @@ -1,4 +1,4 @@ -# Sys::VirtV2V::Target::RHEV +# Sys::VirtV2V::Connection::RHEVTarget # Copyright (C) 2010 Red Hat Inc. # # This library is free software; you can redistribute it and/or @@ -18,7 +18,15 @@ use strict; use warnings; -package Sys::VirtV2V::Target::RHEV::UUIDHelper; +package rhev_util; + +use Exporter 'import'; +our @EXPORT = qw(nfs_helper get_uuid); + +sub nfs_helper +{ + return Sys::VirtV2V::Connection::RHEVTarget::NFSHelper->new(@_); +} sub get_uuid { @@ -35,14 +43,13 @@ sub get_uuid return $uuid; } -package Sys::VirtV2V::Target::RHEV::NFSHelper; + +package Sys::VirtV2V::Connection::RHEVTarget::NFSHelper; use Carp; use File::Temp qw(tempfile); use POSIX qw(:sys_wait_h setuid setgid); -use Sys::VirtV2V::Util qw(user_message); - use Locale::TextDomain 'virt-v2v'; sub new @@ -83,13 +90,14 @@ sub new close($tochild_read); close($fromchild_write); - # Set EUID and EGID to RHEV magic values 36:36 + # Set EUID and EGID to RHEV magic values # execute the wrapped function, trapping errors eval { setgid(36) or die("setgid failed: $!"); setuid(36) or die("setuid failed: $!"); - &$sub(); + # Print out the values returned, 1 per line + print join("\n", &$sub()); }; # Don't exit, which would cause destructors to be called in the child. @@ -113,10 +121,28 @@ sub new return $self; } +sub values +{ + my $self = shift; + + my @values; + my $fromchild = $self->{fromchild}; + while(<$fromchild>) { + chomp; + push(@values, $_); + } + + $self->check_exit(); + return @values; +} + sub check_exit { my $self = shift; + # Make sure it's not waiting on more input + close($self->{tochild}); + my $ret = waitpid($self->{pid}, 0); # If the process terminated normally, check the exit status and stderr @@ -158,215 +184,282 @@ sub DESTROY $? = $retval; } -package Sys::VirtV2V::Target::RHEV::Vol; -use File::Path; -use File::Temp qw(tempdir); -use POSIX; +package Sys::VirtV2V::Connection::RHEVTarget::WriteStream; -use Sys::VirtV2V::Util qw(user_message); +use File::Spec::Functions qw(splitpath); +use Sys::VirtV2V::Util qw(user_message); use Locale::TextDomain 'virt-v2v'; -our %vols_by_path; -our @vols; -our $tmpdir; - -sub _new +sub new { my $class = shift; - my ($mountdir, $domainuuid, $insize) = @_; + my ($volume) = @_; my $self = {}; bless($self, $class); - $self->{insize} = $insize; - # RHEV needs disks to be a multiple of 512 in size. Additionally, SIZE in - # the disk meta file has units of kilobytes. To ensure everything matches up - # exactly, we will pad to to a 1024 byte boundary. - $self->{outsize} = ceil($insize/1024) * 1024; + $self->{volume} = $volume; + $self->{writer} = rhev_util::nfs_helper(sub { + my $path = $self->{volume}->get_path(); - my $imageuuid = Sys::VirtV2V::Target::RHEV::UUIDHelper::get_uuid(); - my $voluuid = Sys::VirtV2V::Target::RHEV::UUIDHelper::get_uuid(); - $self->{imageuuid} = $imageuuid; - $self->{voluuid} = $voluuid; - $self->{domainuuid} = $domainuuid; + # Create the output directory + my (undef, $dir, undef) = splitpath($path); + mkdir($dir) + or die(user_message(__x("Failed to create directory {dir}: {error}", + dir => $dir, + error => $!))); - my $root = "$mountdir/$domainuuid"; - unless (defined($tmpdir)) { - my $nfs = Sys::VirtV2V::Target::RHEV::NFSHelper->new(sub { - print tempdir("v2v.XXXXXXXX", DIR => $root); - }); - my $fromchild = $nfs->{fromchild}; - ($tmpdir) = <$fromchild>; - $nfs->check_exit(); - } + # Write data using dd in 2MB chunks + # XXX - mbooth(a)redhat.com 06/04/2010 (Fedora 12 writing to RHEL 5 NFS) + # Use direct IO as writing a large amount of data to NFS regularly + # crashes my machine. Using direct io doesn't. + exec('dd', 'obs=2M', 'oflag=direct', 'of='.$path) + or die("Unable to execute dd: $!"); + }); + $self->{written} = 0; - $self->{dir} = "$root/images/$imageuuid"; - $self->{tmpdir} = "$tmpdir/$imageuuid"; + return $self; +} - $self->{path} = $self->{tmpdir}."/$voluuid"; +sub _write_metadata +{ + my $self = shift; - $self->{creation} = time(); + my $nfs = rhev_util::nfs_helper(sub { + my $volume = $self->{volume}; - $vols_by_path{$self->{path}} = $self; - push(@vols, $self); + my $path = $volume->get_path().'.meta'; - return $self; -} + my $sizek = $self->{written} / 1024; -sub _get_by_path -{ - my $class = shift; - my ($path) = @_; + # Write out the .meta file + my $meta; + open($meta, '>', $path) + or die(user_message(__x("Unable to open {path} for writing: ". + "{error}", + path => $path, + error => $!))); - return $vols_by_path{$path}; + print $meta "DOMAIN=".$volume->_get_domainuuid()."\n"; + print $meta "VOLTYPE=LEAF\n"; + print $meta "CTIME=".$volume->_get_creation()."\n"; + print $meta "FORMAT=".$volume->_get_rhev_format()."\n"; + print $meta "IMAGE=".$volume->_get_imageuuid()."\n"; + print $meta "DISKTYPE=1\n"; + print $meta "PUUID=00000000-0000-0000-0000-000000000000\n"; + print $meta "LEGALITY=LEGAL\n"; + print $meta "MTIME=".$volume->_get_creation()."\n"; + print $meta "POOL_UUID=00000000-0000-0000-0000-000000000000\n"; + print $meta "SIZE=$sizek\n"; + print $meta "TYPE=".uc($volume->_get_rhev_type())."\n"; + print $meta "DESCRIPTION=Exported by virt-v2v\n"; + print $meta "EOF\n"; + + close($meta) + or die(user_message(__x("Error closing {path}: {error}", + path => $path, + error => $!))); + }); + $nfs->check_exit(); } -sub _get_size +sub write { my $self = shift; + my ($buf) = @_; + + print { $self->{writer}->{tochild} } $buf + or die(user_message(__x("Writing to {path} failed: {error}", + path => $self->{volume}->get_path(), + error => $!))); - return $self->{outsize}; + $self->{written} += length($buf); } -sub _get_imageuuid +sub close { my $self = shift; - return $self->{imageuuid}; + # Pad the file up to a 1K boundary + my $pad = (1024 - ($self->{written} % 1024)) % 1024; + $self->write("\0" x $pad) if ($pad); + + $self->{writer}->check_exit(); + $self->_write_metadata(); } -sub _get_voluuid +sub DESTROY { my $self = shift; - return $self->{voluuid}; + $self->close() if (exists($self->{pid})); } -sub _get_creation -{ - my $self = shift; - return $self->{creation}; -} +package Sys::VirtV2V::Connection::RHEVTarget::Transfer; -sub get_path +use Carp; +use Locale::TextDomain 'virt-v2v'; + +sub new { - my $self = shift; + my $class = shift; + my ($volume) = @_; - return $self->{path}; + my $self = {}; + bless($self, $class); + + $self->{volume} = $volume; + + return $self; } -sub get_format +sub local_path { - my $self = shift; + return shift->{volume}->get_path(); +} - return "raw"; +sub get_read_stream +{ + die(user_message(__"Unable to read data from RHEV")); } -sub is_block +sub get_write_stream { - my $self = shift; + my $volume = shift->{volume}; + return new Sys::VirtV2V::Connection::RHEVTarget::WriteStream($volume); +} - return 0; +sub DESTROY +{ + # Remove circular reference + delete(shift->{volume}); } -sub open + +package Sys::VirtV2V::Connection::RHEVTarget::Vol; + +use File::Path; +use File::Spec::Functions; +use File::Temp qw(tempdir); +use POSIX; + +use Sys::VirtV2V::Util qw(user_message); +use Locale::TextDomain 'virt-v2v'; + +our %vols_by_path; +our @vols; +our $tmpdir; + +@Sys::VirtV2V::Connection::RHEVTarget::Vol::ISA = + qw(Sys::VirtV2V::Connection::Volume); + +sub new { - my $self = shift; + my $class = shift; + my ($mountdir, $domainuuid, $format, $insize, $sparse) = @_; - my $now = $self->{creation}; - $self->{written} = 0; + my $root = catdir($mountdir, $domainuuid); - $self->{writer} = Sys::VirtV2V::Target::RHEV::NFSHelper->new(sub { - my $dir = $self->{tmpdir}; - my $path = $self->{path}; + # Initialise the package-wide temp directory if required + unless (defined($tmpdir)) { + my $nfs = rhev_util::nfs_helper(sub { + return tempdir("v2v.XXXXXXXX", DIR => $root); + }); + ($tmpdir) = $nfs->values(); + } - mkdir($dir) - or die(user_message(__x("Failed to create directory {dir}: {error}", - dir => $dir, - error => $!))); + my $imageuuid = rhev_util::get_uuid(); + my $voluuid = rhev_util::get_uuid(); - # Write out the .meta file - my $meta; - open($meta, '>', "$path.meta") - or die(user_message(__x("Unable to open {path} for writing: ". - "{error}", - path => "$path.meta", - error => $!))); + my $imagedir = catdir($root, 'images', $imageuuid); + my $imagetmpdir = catdir($tmpdir, $imageuuid); + my $volpath = catfile($imagetmpdir, $voluuid); - print $meta "DOMAIN=".$self->{domainuuid}."\n"; - print $meta "VOLTYPE=LEAF\n"; - print $meta "CTIME=$now\n"; - print $meta "FORMAT=RAW\n"; - print $meta "IMAGE=".$self->{imageuuid}."\n"; - print $meta "DISKTYPE=1\n"; - print $meta "PUUID=00000000-0000-0000-0000-000000000000\n"; - print $meta "LEGALITY=LEGAL\n"; - print $meta "MTIME=$now\n"; - print $meta "POOL_UUID=00000000-0000-0000-0000-000000000000\n"; - print $meta "SIZE=".($self->{outsize} / 1024)."\n"; - print $meta "TYPE=SPARSE\n"; - print $meta "DESCRIPTION=Exported by virt-v2v\n"; - print $meta "EOF\n"; + # RHEV needs disks to be a multiple of 512 in size. Additionally, SIZE in + # the disk meta file has units of kilobytes. To ensure everything matches up + # exactly, we will pad to to a 1024 byte boundary. + my $outsize = ceil($insize/1024) * 1024; - close($meta) - or die(user_message(__x("Error closing {path}: {error}", - path => "$path.meta", - error => $!))); + my $creation = time(); - # Write the remainder of the data using dd in 2MB chunks - # XXX - mbooth(a)redhat.com 06/04/2010 (Fedora 12 writing to RHEL 5 NFS) - # Use direct IO as writing a large amount of data to NFS regularly - # crashes my machine. Using direct io crashes less. - exec('dd', 'obs='.1024*1024*2, 'oflag=direct', 'of='.$path) - or die("Unable to execute dd: $!"); - }); -} + my $self = $class->SUPER::new($imageuuid, $format, $volpath, $outsize, + $sparse, 0); + $self->{transfer} = + new Sys::VirtV2V::Connection::RHEVTarget::Transfer($self); -sub write -{ - my $self = shift; - my ($data) = @_; + $self->{insize} = $insize; + $self->{outsize} = $outsize; + + $self->{imageuuid} = $imageuuid; + $self->{voluuid} = $voluuid; + $self->{domainuuid} = $domainuuid; - defined($self->{writer}) or die("write called without open"); + $self->{imagedir} = $imagedir; + $self->{imagetmpdir} = $imagetmpdir; - unless(print {$self->{writer}->{tochild}} $data) { - # This should only have failed if there was an error from the helper - $self->{writer}->check_exit(); + $self->{creation} = $creation; - # die() explicitly in case the above didn't - die("Error writing to helper: $!"); + # Convert format into something RHEV understands + my $rhev_format; + if ($format eq 'raw') { + $self->{rhev_format} = 'RAW'; + } elsif ($format eq 'qcow2') { + $self->{rhev_format} = 'COW'; + } else { + die(user_message(__x("RHEV cannot handle volumes of format {format}", + format => $format))); } - $self->{written} += length($data); + # Generate the RHEV type + # N.B. This must be in mixed case in the OVF, but in upper case in the .meta + # file. We store it in mixed case and convert to upper when required. + $self->{rhev_type} = $sparse ? 'Sparse' : 'Preallocated'; + + $vols_by_path{$volpath} = $self; + push(@vols, $self); + + return $self; } -sub close +sub _get_by_path { - my $self = shift; + my $class = shift; + my ($path) = @_; - # Check we wrote the full file - die(user_message(__x("Didn't write full volume. Expected {expected} ". - "bytes, wrote {actual} bytes.", - expected => $self->{insize}, - actual => $self->{written}))) - unless ($self->{written} == $self->{insize}); + return $vols_by_path{$path}; +} - # Pad the output up to outsize - my $pad = $self->{outsize} - $self->{insize}; - $self->write("\0" x $pad) if ($pad); +sub _get_domainuuid +{ + return shift->{domainuuid}; +} - # Close the writer pipe, which will cause the child to exit - close($self->{writer}->{tochild}) - or die("Error closing tochild pipe"); +sub _get_imageuuid +{ + return shift->{imageuuid}; +} - # Wait for the child to exit - $self->{writer}->check_exit(); +sub _get_voluuid +{ + return shift->{voluuid}; +} + +sub _get_creation +{ + return shift->{creation}; +} - delete($self->{writer}); - delete($self->{written}); +sub _get_rhev_format +{ + return shift->{rhev_format}; +} + +sub _get_rhev_type +{ + return shift->{rhev_type}; } sub _move_vols @@ -374,11 +467,11 @@ sub _move_vols my $class = shift; foreach my $vol (@vols) { - rename($vol->{tmpdir}, $vol->{dir}) + rename($vol->{imagetmpdir}, $vol->{imagedir}) or die(user_message(__x("Unable to move volume from temporary ". "location {tmpdir} to {dir}", - tmpdir => $vol->{tmpdir}, - dir => $vol->{dir}))); + tmpdir => $vol->{imagetmpdir}, + dir => $vol->{imagedir}))); } $class->_cleanup(); @@ -405,9 +498,10 @@ sub _cleanup $tmpdir = undef; } -package Sys::VirtV2V::Target::RHEV; +package Sys::VirtV2V::Connection::RHEVTarget; use File::Temp qw(tempdir); +use File::Spec::Functions; use Time::gmtime; use Sys::VirtV2V::ExecHelper; @@ -417,26 +511,15 @@ use Locale::TextDomain 'virt-v2v'; =head1 NAME -Sys::VirtV2V::Target::RHEV - Output to a RHEV Export storage domain - -=head1 SYNOPSIS - - use Sys::VirtV2V::Target::RHEV; - - my $target = new Sys::VirtV2V::Target::RHEV($domain_path); - -=head1 DESCRIPTION - -Sys::VirtV2V::Target::RHEV write the converted guest to a RHEV Export storage -domain. This can later be imported to RHEV by the user. +Sys::VirtV2V::Connection::RHEVTarget - Output to a RHEV Export storage domain =head1 METHODS =over -=item Sys::VirtV2V::Target::RHEV->new(domain_path) +=item Sys::VirtV2V::Connection::RHEVTarget->new(domain_path) -Create a new Sys::VirtV2V::Target::RHEV object. +Create a new Sys::VirtV2V::Connection::RHEVTarget object. =over @@ -479,60 +562,46 @@ sub new output => $eh->output()))); } - my $nfs = Sys::VirtV2V::Target::RHEV::NFSHelper->new(sub { + my $nfs = rhev_util::nfs_helper(sub { opendir(my $dir, $mountdir) or die(user_message(__x("Unable to open {mountdir}: {error}", mountdir => $mountdir, error => $!))); + my @entries; foreach my $entry (readdir($dir)) { # return entries which look like uuids - print "$entry\n" + push(@entries, $entry) if ($entry =~ /^[0-9a-z]{8}-(?:[0-9a-z]{4}-){3}[0-9a-z]{12}$/); } + + return @entries; }); + my @entries = $nfs->values(); - # Get the UUID of the storage domain - my $domainuuid; - my $fromchild = $nfs->{fromchild}; - while (<$fromchild>) { - if (defined($domainuuid)) { - die(user_message(__x("{domain_path} contains multiple possible ". - "domains. It may only contain one.", - domain_path => $domain_path))); - } - chomp; - $domainuuid = $_; - } - $nfs->check_exit(); + die(user_message(__x("{domain_path} contains multiple possible ". + "domains. It may only contain one.", + domain_path => $domain_path))) if (@entries > 1); - if (!defined($domainuuid)) { - die(user_message(__x("{domain_path} does not contain an initialised ". - "storage domain", - domain_path => $domain_path))); - } + my ($domainuuid) = @entries; + die(user_message(__x("{domain_path} does not contain an initialised ". + "storage domain", + domain_path => $domain_path))) + unless (defined($domainuuid)); $self->{domainuuid} = $domainuuid; # Check that the domain has been attached to a Data Center by checking that # the master/vms directory exists - my $vms_rel = $domainuuid.'/master/vms'; - my $vms_abs = $mountdir.'/'.$vms_rel; - $nfs = Sys::VirtV2V::Target::RHEV::NFSHelper->new(sub { - if (-d $vms_abs) { - print "1\n"; - } else { - print "0\n"; - } + my $vms_rel = catdir($domainuuid, 'master', 'vms'); + my $vms_abs = catdir($mountdir, $vms_rel); + $nfs = rhev_util::nfs_helper(sub { + return -d $vms_abs ? 1 : 0; }); - $fromchild = $nfs->{fromchild}; - while (<$fromchild>) { - chomp; - die(user_message(__x("{domain_path} has not been attached to a RHEV ". - "data center ({path} does not exist).", - domain_path => $domain_path, - path => $vms_rel))) if ($_ eq "0"); - } - $nfs->check_exit(); + my ($attached) = $nfs->values(); + die(user_message(__x("{domain_path} has not been attached to a RHEV ". + "data center ({path} does not exist).", + domain_path => $domain_path, + path => $vms_rel))) unless ($attached); return $self; } @@ -547,8 +616,8 @@ sub DESTROY my $retval = $?; eval { - my $nfs = Sys::VirtV2V::Target::RHEV::NFSHelper->new(sub { - Sys::VirtV2V::Target::RHEV::Vol->_cleanup(); + my $nfs = rhev_util::nfs_helper(sub { + Sys::VirtV2V::Connection::RHEVTarget::Vol->_cleanup(); }); $nfs->check_exit(); }; @@ -577,7 +646,7 @@ sub DESTROY $? = $retval; } -=item create_volume(name, size) +=item create_volume(name, format, size, is_sparse) Create a new volume in the export storage domain @@ -587,27 +656,37 @@ Create a new volume in the export storage domain The name of the volume which is being created. +=item format + +The file format of the target volume, as returned by qemu. + =item size The size of the volume which is being created in bytes. +=item is_sparse + +1 if the target volume is sparse, 0 otherwise. + =back -create_volume() returns a Sys::VirtV2V::Target::RHEV::Vol object. +create_volume() returns a Sys::VirtV2V::Connection::RHEVTarget::Vol object. =cut sub create_volume { my $self = shift; - my ($name, $size) = @_; + my ($name, $format, $size, $is_sparse) = @_; - return Sys::VirtV2V::Target::RHEV::Vol->_new($self->{mountdir}, - $self->{domainuuid}, - $size); + return Sys::VirtV2V::Connection::RHEVTarget::Vol->new($self->{mountdir}, + $self->{domainuuid}, + $format, + $size, + $is_sparse); } -=item volume_exists (name) +=item volume_exists(name) Check if volume I<name> exists in the target storage domain. @@ -617,13 +696,10 @@ Always returns 0, as RHEV storage domains don't have names sub volume_exists { - my $self = shift; - my ($name) = @_; - return 0; } -=item get_volume (name) +=item get_volume(name) Not defined for RHEV output @@ -675,7 +751,7 @@ sub create_guest # Generate a creation date my $vmcreation = _format_time(gmtime()); - my $vmuuid = Sys::VirtV2V::Target::RHEV::UUIDHelper::get_uuid(); + my $vmuuid = rhev_util::get_uuid(); my $ostype = _get_os_type($desc); @@ -755,20 +831,20 @@ EOF $self->_disks($ovf, $dom); $self->_networks($ovf, $dom); - my $nfs = Sys::VirtV2V::Target::RHEV::NFSHelper->new(sub { + my $nfs = rhev_util::nfs_helper(sub { my $mountdir = $self->{mountdir}; my $domainuuid = $self->{domainuuid}; - my $dir = $mountdir.'/'.$domainuuid.'/master/vms/'.$vmuuid; + my $dir = catdir($mountdir, $domainuuid, 'master', 'vms', $vmuuid); mkdir($dir) or die(user_message(__x("Failed to create directory {dir}: {error}", dir => $dir, error => $!))); - Sys::VirtV2V::Target::RHEV::Vol->_move_vols(); + Sys::VirtV2V::Connection::RHEVTarget::Vol->_move_vols(); my $vm; - my $ovfpath = $dir.'/'.$vmuuid.'.ovf'; + my $ovfpath = catfile($dir, $vmuuid.'.ovf'); open($vm, '>', $ovfpath) or die(user_message(__x("Unable to open {path} for writing: ". "{error}", @@ -958,13 +1034,13 @@ sub _disks my ($bus) = $disk->findnodes('target/@bus'); $bus = $bus->getNodeValue(); - my $vol = Sys::VirtV2V::Target::RHEV::Vol->_get_by_path($path); + my $vol = Sys::VirtV2V::Connection::RHEVTarget::Vol->_get_by_path($path); die("dom contains path not written by virt-v2v: $path\n". $dom->toString()) unless (defined($vol)); - my $fileref = $vol->_get_imageuuid().'/'.$vol->_get_voluuid(); - my $size_gb = int($vol->_get_size()/1024/1024/1024); + my $fileref = catdir($vol->_get_imageuuid(), $vol->_get_voluuid()); + my $size_gb = int($vol->get_size()/1024/1024/1024); # Add disk to References my $file = $ovf->createElement("File"); @@ -972,7 +1048,7 @@ sub _disks $file->setAttribute('ovf:href', $fileref); $file->setAttribute('ovf:id', $vol->_get_voluuid()); - $file->setAttribute('ovf:size', $vol->_get_size()); + $file->setAttribute('ovf:size', $vol->get_size()); $file->setAttribute('ovf:description', 'imported by virt-v2v'); # Add disk to DiskSection @@ -986,8 +1062,8 @@ sub _disks $diske->setAttribute('ovf:parentRef', ''); $diske->setAttribute('ovf:vm_snapshot_id', '00000000-0000-0000-0000-000000000000'); - $diske->setAttribute('ovf:volume-format', 'RAW'); - $diske->setAttribute('ovf:volume-type', 'Sparse'); + $diske->setAttribute('ovf:volume-format', $vol->_get_rhev_format()); + $diske->setAttribute('ovf:volume-type', $vol->_get_rhev_type()); $diske->setAttribute('ovf:format', 'http://en.wikipedia.org/wiki/Byte'); # IDE = 0, SCSI = 1, VirtIO = 2 $diske->setAttribute('ovf:disk-interface', $bus eq 'virtio' ? 2 : 0); diff --git a/lib/Sys/VirtV2V/Connection.pm b/lib/Sys/VirtV2V/Connection/Source.pm similarity index 60% rename from lib/Sys/VirtV2V/Connection.pm rename to lib/Sys/VirtV2V/Connection/Source.pm index 8029230..71c5f1f 100644 --- a/lib/Sys/VirtV2V/Connection.pm +++ b/lib/Sys/VirtV2V/Connection/Source.pm @@ -1,5 +1,5 @@ -# Sys::VirtV2V::Connection -# Copyright (C) 2009 Red Hat Inc. +# Sys::VirtV2V::Connection::Source +# Copyright (C) 2009,2010 Red Hat Inc. # # This library is free software; you can redistribute it and/or # modify it under the terms of the GNU Lesser General Public @@ -15,16 +15,13 @@ # License along with this library; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA -package Sys::VirtV2V::Connection; +package Sys::VirtV2V::Connection::Source; use strict; use warnings; use Sys::Virt; -use Sys::VirtV2V::Transfer::ESX; -use Sys::VirtV2V::Transfer::LocalCopy; -use Sys::VirtV2V::Transfer::SSH; use Sys::VirtV2V::Util qw(user_message); use Locale::TextDomain 'virt-v2v'; @@ -37,21 +34,21 @@ Sys::VirtV2V::Connection - Obtain domain metadata =head1 SYNOPSIS - use Sys::VirtV2V::Connection::LibVirt; + use Sys::VirtV2V::Connection::LibVirtSource; - $conn = Sys::VirtV2V::Connection::LibVirt->new($uri, $name, $target); + $conn = Sys::VirtV2V::Connection::LibVirtSource->new($uri, $name, $target); $dom = $conn->get_dom(); $storage = $conn->get_storage_paths(); $devices = $conn->get_storage_devices(); =head1 DESCRIPTION -Sys::VirtV2V::Connection describes a connection to a, possibly remote, source of -guest metadata and storage. It is a virtual superclass and can't be instantiated -directly. Use one of the subclasses: +Sys::VirtV2V::Source provides access to a source of guest metadata and storage. +It is a virtual superclass and can't be instantiated directly. Use one of the +subclasses: - Sys::VirtV2V::Connection::LibVirt - Sys::VirtV2V::Connection::LibVirtXML + Sys::VirtV2V::Target::LibVirtSource + Sys::VirtV2V::Source::LibVirtXML =head1 METHODS @@ -102,14 +99,58 @@ sub get_dom return $self->{dom}; } +sub _volume_copy +{ + my ($src, $dst) = @_; + + my $src_s; + my $dst_s; + + # Can we just do a straight copy? + if ($src->get_format() eq $dst->get_format()) { + $src_s = $src->get_read_stream(); + $dst_s = $dst->get_write_stream(); + } + + else { + die("Different formats"); + } + + # Copy the contents of the source stream to the destination stream + my $total = 0; + for (;;) { + my $buf = $src_s->read(4 * 1024 * 1024); + last if (length($buf) == 0); + + $total += length($buf); + + $dst_s->write($buf); + } + + # This would be closed implicitly, but we want to report read/write errors + # before checking for a short volume + $dst_s->close(); -# Iterate over returned storage. Transfer it and update DOM as necessary. To be -# called by subclasses. -sub _storage_iterate + die(user_message(__x("Didn't receive full volume. Received {received} ". + "of {total} bytes.", + received => $total, + total => $src->get_size()))) + unless ($total == $src->get_size()); + + return $dst; +} + +=item copy_storage(target) + +Copy all of a guests storage devices to I<target>. Update the guest metadata to +reflect their new locations and properties. + +=cut + +sub copy_storage { my $self = shift; - - my ($transfer, $target) = @_; + my ($target) = @_; my $dom = $self->get_dom(); @@ -118,7 +159,8 @@ sub _storage_iterate # A list of libvirt target device names my @devices; - foreach my $disk ($dom->findnodes("/domain/devices/disk[\@device='disk']")) { + foreach my $disk ($dom->findnodes("/domain/devices/disk[\@device='disk']")) + { my ($source_e) = $disk->findnodes('source'); my ($source) = $source_e->findnodes('@file | @dev'); @@ -129,17 +171,30 @@ sub _storage_iterate defined($dev) or die("disk does not have a target device: \n". $dom->toString()); - my $path = $source->getValue(); + my $src = $self->get_volume($source->getValue()); + my $dst; + if ($target->volume_exists($src->get_name())) { + warn user_message(__x("WARNING: storage volume {name} already ". + "exists on the target. NOT copying it ". + "again. Delete the volume and retry to ". + "copy again.", + name => $src->get_name())); + $dst = $target->get_volume($src->get_name());; + } else { + $dst = $target->create_volume($src->get_name(), + $src->get_format(), + $src->get_size(), + $src->is_sparse()); + } - # Die if transfer required and no output target - die (user_message(__"No output target was specified")) - unless (defined($target)); + # This will die if libguestfs can't use the result directly, so we do it + # before copying all the data. + push(@paths, $dst->get_local_path()); - # Fetch the remote storage - my $vol = $transfer->transfer($self, $path, $target); + _volume_copy($src, $dst); # Export the new path - $path = $vol->get_path(); + my $path = $dst->get_path(); # Find any existing driver element. my ($driver) = $disk->findnodes('driver'); @@ -151,13 +206,13 @@ sub _storage_iterate $disk->appendChild($driver); } $driver->setAttribute('name', 'qemu'); - $driver->setAttribute('type', $vol->get_format()); + $driver->setAttribute('type', $dst->get_format()); # Remove the @file or @dev attribute before adding a new one $source_e->removeAttributeNode($source); # Set @file or @dev as appropriate - if ($vol->is_block()) { + if ($dst->is_block()) { $disk->setAttribute('type', 'block'); $source_e->setAttribute('dev', $path); } else { @@ -165,7 +220,6 @@ sub _storage_iterate $source_e->setAttribute('file', $path); } - push(@paths, $path); push(@devices, $dev->getNodeValue()); } @@ -205,8 +259,8 @@ Please see the file COPYING.LIB for the full license. =head1 SEE ALSO -L<Sys::VirtV2V::Connection::LibVirt(3pm)>, -L<Sys::VirtV2V::Connection::LibVirtXML(3pm)>, +L<Sys::VirtV2V::Source::LibVirt(3pm)>, +L<Sys::VirtV2V::Source::LibVirtXML(3pm)>, L<virt-v2v(1)>, L<http://libguestfs.org/>. diff --git a/lib/Sys/VirtV2V/Connection/Volume.pm b/lib/Sys/VirtV2V/Connection/Volume.pm new file mode 100644 index 0000000..7961bd5 --- /dev/null +++ b/lib/Sys/VirtV2V/Connection/Volume.pm @@ -0,0 +1,179 @@ +# Sys::VirtV2V::Connection::Volume +# Copyright (C) 2010 Red Hat Inc. +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + +package Sys::VirtV2V::Connection::Volume; + +use strict; +use warnings; + +use Data::Dumper; + +use Sys::VirtV2V::Util qw(user_message); + +use Locale::TextDomain 'virt-v2v'; + +=pod + +=head1 NAME + +Sys::VirtV2V::Connection::Volume - Read and write storage volumes + +=head1 METHODS + +=over + +=item new(name, format, path, size, is_block, transfer) + +Create a new Volume which returns the given metadata, and uses I<transfer> for +transferring data. + +=cut + +sub new +{ + my $class = shift; + my ($name, $format, $path, $size, $is_sparse, $is_block, $transfer) = @_; + + my $self = {}; + bless($self, $class); + + $self->{name} = $name; + $self->{format} = $format; + $self->{path} = $path; + $self->{size} = $size; + $self->{is_sparse} = $is_block; + $self->{is_block} = $is_block; + $self->{transfer} = $transfer; + + return $self; +} + +=item get_size + +Return size for this Volume. + +=cut + +sub get_size +{ + return shift->{size}; +} + +=item get_format + +Return the on-disk format of this Volume. + +=cut + +sub get_format +{ + return shift->{format}; +} + +=item get_name + +Return the name of this Volume. + +=cut + +sub get_name +{ + return shift->{name}; +} + +=item get_path + +Return the native path of this Volume. + +=cut + +sub get_path +{ + return shift->{path}; +} + +=item get_local_path + +=cut + +sub get_local_path +{ + return shift->{transfer}->local_path(); +} + +=item is_block + +Return 1 if the volume is stored directly on a block device, 0 otherwise. + +=cut + +sub is_block +{ + return shift->{is_block}; +} + +=item is_sparse + +Return 1 if the volume is not fully allocated, 0 otherwise. + +=cut + +sub is_sparse +{ + return shift->{is_sparse}; +} + +=item get_read_stream + +Return a ReadStream for this volume. + +=cut + +sub get_read_stream +{ + return shift->{transfer}->get_read_stream(); +} + +=item get_write_stream + +Return a WriteStream for this volume. + +=cut + +sub get_write_stream +{ + return shift->{transfer}->get_write_stream(); +} + +=back + +=head1 COPYRIGHT + +Copyright (C) 2010 Red Hat Inc. + +=head1 LICENSE + +Please see the file COPYING.LIB for the full license. + +=head1 SEE ALSO + +L<virt-v2v(1)>, +L<http://libguestfs.org/>. + +=cut + +1; diff --git a/lib/Sys/VirtV2V/Transfer/ESX.pm b/lib/Sys/VirtV2V/Transfer/ESX.pm index 798285b..5628a9b 100644 --- a/lib/Sys/VirtV2V/Transfer/ESX.pm +++ b/lib/Sys/VirtV2V/Transfer/ESX.pm @@ -15,17 +15,16 @@ # License along with this library; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA -package Sys::VirtV2V::Transfer::ESX::UA; - use strict; use warnings; -use Sys::Virt::Error; +package Sys::VirtV2V::Transfer::ESX::UA; -use Sys::VirtV2V; +use DateTime; +use MIME::Base64; +use Sys::VirtV2V; use Sys::VirtV2V::Util qw(user_message); - use Locale::TextDomain 'virt-v2v'; # This is a gross hack to bring sanity to Net::HTTPS's SSL handling. Net::HTTPS @@ -37,37 +36,24 @@ use Locale::TextDomain 'virt-v2v'; # will silently do nothing. # # To try to fix this situation, we hardcode here that we want Net::SSL. In the -# _new constructor, we check that Net::SSL was actually used, and die() if it -# wasn't. We subsequently only include configuration for Net::SSL. +# constructor, we check that Net::SSL was actually used, and die() if it wasn't. +# We subsequently only include configuration for Net::SSL. BEGIN { use Net::HTTPS; $Net::HTTPS::SSL_SOCKET_CLASS = "Net::SSL"; } -use LWP::UserAgent; -@Sys::VirtV2V::Transfer::ESX::UA::ISA = qw(LWP::UserAgent); - -our %handles; - sub new { my $class = shift; + my ($username, $password, $noverify) = @_; - my ($server, $username, $password, $target, $noverify) = @_; - - my $self = $class->SUPER::new( - agent => 'virt-v2v/'.$Sys::VirtV2V::VERSION, - protocols_allowed => [ 'https' ] - ); + my $self = {}; + bless($self, $class); - # Older versions of LWP::UserAgent don't support show_progress - $self->show_progress(1) if ($self->can('show_progress')); - - $self->{_v2v_server} = $server; - $self->{_v2v_target} = $target; - $self->{_v2v_username} = $username; - $self->{_v2v_password} = $password; - $self->{_v2v_noverify} = $noverify; + $self->{noverify} = $noverify; + $self->{agent} = 'virt-v2v/'.$Sys::VirtV2V::VERSION; + $self->{auth} = 'Basic '.encode_base64("$username:$password"); if ($noverify) { # Unset HTTPS_CA_DIR if it is already set @@ -79,213 +65,211 @@ sub new { $ENV{HTTPS_CA_DIR} = "" unless (exists($ENV{HTTPS_CA_DIR})); } - die("Invalid configuration of Net::HTTPS") + die('Invalid configuration of Net::HTTPS') unless(Net::HTTPS->isa('Net::SSL')); return $self; } -sub get_volume +sub _request { my $self = shift; + my ($method, $uri) = @_; - my ($path) = @_; - - # Need to turn this: - # [yellow:storage1] win2k3r2-32/win2k3r2-32.vmdk - # into this: - # https://yellow.rhev.marston/folder/win2k3r2-32/win2k3r2-32-flat.vmdk? \ - # dcPath=ha-datacenter&dsName=yellow:storage1 - - $path =~ /^\[(.*)\]\s+(.*)\.vmdk$/ - or die("Failed to parse ESX path: $path"); - my $datastore = $1; - my $vmdk = $2; - - my $url = _get_vol_url($self->{_v2v_server}, $vmdk, $datastore); + my $base = URI->new($uri->scheme.'://'.$uri->host); + my $conn = new Net::HTTPS(Host => $uri->host, + MaxLineLength => 0) + or die(user_message(__x("Failed to connect to {host}: {error}", + host => $uri->host, + error => $@))); - # Replace / with _ so the vmdk name can be used as a volume name - my $volname = $vmdk; - $volname =~ s,/,_,g; - $self->{_v2v_volname} = $volname; + $conn->write_request($method => '/'.$uri->rel($base), + 'User-Agent' => $self->{agent}, + 'Authorization' => $self->{auth}) + or die(user_message(__x("Failed to send request to {host}: {error}", + host => $uri->host, + error => $@))); - my $target = $self->{_v2v_target}; - if ($target->volume_exists($volname)) { - warn user_message(__x("WARNING: storage volume {name} already exists ". - "on the target. NOT fetching it again. Delete ". - "the volume and retry to download again.", - name => $volname)); - return $target->get_volume($volname); - } + my ($code, $msg, %h) = $conn->read_response_headers(); + die([$code, $msg]) unless ($code == 200); - # Head request to get the size and create the volume - # We could do this with a single GET request. The problem with this is that - # you have to create the volume before writing to it. If the volume creation - # takes a very long time, the transfer may fail in the mean time. - my $retried = 0; - SIZE: for(;;) { - my $r = $self->head($url); - if ($r->is_success) { - $self->verify_certificate($r) unless ($self->{_v2v_noverify}); - $self->create_volume($r); - last SIZE; - } - - # If a disk is actually a snapshot image it will have '-00000n' - # appended to its name, e.g.: - # [yellow:storage1] RHEL4-X/RHEL4-X-000003.vmdk - # The flat storage is still called RHEL4-X-flat, however. - # If we got a 404 and the vmdk name looks like it might be a snapshot, - # try again without the snapshot suffix. - # XXX: We're in flaky heuristic territory here. When the libvirt ESX - # driver implements the volume apis we should look for this information - # there instead. - elsif ($r->code == 404 && $retried == 0) { - $retried = 1; - if ($vmdk =~ /^(.*)-\d+$/) { - $vmdk = $1; - $url = _get_vol_url($self->{_v2v_server}, $vmdk, $datastore); - } - } + $self->_verify_certificate($conn, $uri->host) unless ($self->{noverify}); - else { - $self->report_error($r); - } - } - - $self->{_v2v_received} = 0; - my $r = $self->get($url, - ':content_cb' => sub { $self->handle_data(@_); }, - ':read_size_hint' => 64 * 1024); - - if ($r->is_success) { - # It reports success even if one of the callbacks died - my $died = $r->header('X-Died'); - die($died) if (defined($died)); - - $self->verify_certificate($r) unless ($self->{_v2v_noverify}); - - # It reports success even if we didn't receive the whole file - die(user_message(__x("Didn't receive full volume. Received {received} ". - "of {total} bytes.", - received => $self->{_v2v_received}, - total => $self->{_v2v_volsize}))) - unless ($self->{_v2v_received} == $self->{_v2v_volsize}); - - my $vol = $self->{_v2v_vol}; - $vol->close(); - return $vol; - } - - $self->report_error($r); + return ($conn, \%h); } -sub _get_vol_url +sub get_content_length { - my ($server, $vmdk, $datastore) = @_; + my $self = shift; + my ($uri) = @_; - my $url = URI->new("https://".$server); - $url->path("/folder/$vmdk-flat.vmdk"); - $url->query_form(dcPath => "ha-datacenter", dsName => $datastore); + my ($conn, $h) = $self->_request('HEAD', $uri); - return $url; + my $length = $h->{'Content-Length'}; + die(user_message(__x("ESX Server didn't return content length ". + "for {uri}", + uri => $uri))) + unless (defined($length)); + + return $length; } -sub report_error +sub request_content { my $self = shift; - my ($r) = @_; + my ($uri) = @_; - if ($r->code == 401) { - die(user_message(__x("Authentication error connecting to ". - "{server}. Used credentials for {username} ". - "from .netrc.", - server => $self->{_v2v_server}, - username => $self->{_v2v_username}))) - } + my ($conn) = $self->_request('GET', $uri); - die(user_message(__x("Failed to connect to ESX server: {error}", - error => $r->status_line))); + $self->{conn} = $conn; + $self->{hostname} = $uri->host; } -sub get_basic_credentials +sub read_content { my $self = shift; + my ($size) = @_; + + my $conn = $self->{conn}; + die("read_content called without request_content") unless (defined($conn)); + + my $buf; + my $rv; + do { + $rv = $conn->read_entity_body($buf, $size); + } while (defined($rv) && $rv == -1); + # We want to clean up and exit immediately on signals, and we don't set + # nonblocking on any socket, so EINTR and EAGAIN don't need to be handled + # here - my ($realm, $uri, $isproxy) = @_; # Not interested in any of these things - # because we only ever contact a single - # server in a single context + die(user_message(__x("Error reading data from {host}", + host => $self->{hostname}))) unless (defined($rv)); - return ($self->{_v2v_username}, $self->{_v2v_password}); + return $buf; } -sub handle_data +sub _verify_certificate { my $self = shift; + my ($conn, $hostname) = @_; - my ($data, $response) = @_; + my $cert = $conn->get_peer_certificate(); - # Verify the certificate of the get request the first time we're called - if ($self->{_v2v_received} == 0) { - $self->verify_certificate($response) unless ($self->{_v2v_noverify}); + my $cn; + foreach my $i (split(/\//, $cert->subject_name)) { + next unless(length($i) > 0); + my ($key, $value) = split(/=/, $i); + $cn = lc($value) if (lc($key) eq 'cn'); + } + die(user_message(__x("SSL Certificate Subject from {host} doesn't contain ". + "a CN", host => $hostname))) unless (defined($cn)); + + $hostname = lc($hostname); + die(user_message(__x("Server {server} presented an SSL certificate ". + "for {commonname}", + server => $hostname, + commonname => $cn))) + unless ($hostname eq $cn or $hostname !~ /\Q.$cn\E$/); + + my $not_before = _parse_nottime($cert->not_before); + my $not_after = _parse_nottime($cert->not_after); + + my $now = DateTime->now; + if (DateTime->compare($now, $not_before) < 0) { + die(user_message(__x("SSL Certificate presented by {host} will not ". + "be valid until {date}", + host => $hostname, + date => $not_before))); } - $self->{_v2v_received} += length($data); - $self->{_v2v_vol}->write($data); + if (DateTime->compare($now, $not_after) > 0) { + die(user_message(__x("SSL Certificate present by {host} expired on ". + "{date}", + host => $hostname, + date => $not_after))); + } + + # This should never happen, because we should instead get an SSL connection + # error Net::HTTPS + if (!$conn->get_peer_verify()) { + die("SSL Certificate not verified"); + } } -sub create_volume +sub _parse_nottime { - my $self = shift; + my ($date) = @_; + + $date =~ /^\s*(\d{4})-(\d\d)-(\d\d)\s+(\d\d):(\d\d):(\d\d)\s+(\S+)\s*$/ + or die("Unrecognised date format: $date"); + + return new DateTime( + year => $1, + month => $2, + day => $3, + hour => $4, + minute => $5, + second => $6, + time_zone => $7 + ); +} - my ($response) = @_; - my $target = $self->{_v2v_target}; +package Sys::VirtV2V::Transfer::ESX::ReadStream; - my $name = $self->{_v2v_volname}; - die("create_volume called, but _v2v_volname is not set") - unless (defined($name)); +sub new +{ + my $class = shift; + my ($uri, $username, $password, $noverify, $error) = @_; - my $size = $response->content_length(); - $self->{_v2v_volsize} = $size; + my $self = {}; + bless($self, $class); + + $self->{ua} = new Sys::VirtV2V::Transfer::ESX::UA( + $username, + $password, + $noverify + ); - my $vol = $target->create_volume($name, $size); - $vol->open(); - $self->{_v2v_vol} = $vol; + eval { + $self->{ua}->request_content($uri); + }; + if ($@) { + if (ref($@) eq 'ARRAY') { + &$error($@->[0], $@->[1]); + } else { + die($@); + } + } + + return $self; } -sub verify_certificate +sub read { my $self = shift; + my ($size) = @_; - my ($r) = @_; - - # No point in trying to verify headers if the request failed anyway - return unless ($r->is_success); - - my $subject = $r->header('Client-SSL-Cert-Subject'); - die(user_message(__"Server response didn't include an SSL subject")) - unless ($subject); + return $self->{ua}->read_content($size); +} - $subject =~ /\/CN=([^\/]*)/ - or die(user_message(__x("SSL Certification Subject doesn't contain a ". - "common name: {subject}", - subject => $subject))); - my $cn = $1; +sub close +{ + # Nothing required +} - $self->{_v2v_server} =~ /(^|\.)\Q$cn\E$/ - or die(user_message(__x("Server {server} presented an SSL certificate ". - "for {commonname}", - server => $self->{_v2v_server}, - commonname => $cn))); +sub DESTROY +{ + shift->close(); } + package Sys::VirtV2V::Transfer::ESX; use Sys::Virt; +use URI; use Sys::VirtV2V::Util qw(user_message); - use Locale::TextDomain 'virt-v2v'; =pod @@ -294,66 +278,198 @@ use Locale::TextDomain 'virt-v2v'; Sys::VirtV2V::Transfer::ESX - Transfer guest storage from an ESX server -=head1 SYNOPSIS +=head1 METHODS - use Sys::VirtV2V::Transfer::ESX; +=over - $vol = Sys::VirtV2V::Transfer::ESX->transfer($conn, $path, $target); +=item new(path, hostname, username, password, noverify, is_sparse) -=head1 DESCRIPTION +Return a new ESX Transfer object -Sys::VirtV2V::Transfer::ESX retrieves guest storage devices from an ESX server. +=cut -=head1 METHODS +sub new +{ + my $class = shift; + my ($path, $hostname, $username, $password, $noverify, $is_sparse) = @_; -=over + die(user_message(__x("Authentication is required to connect to ". + "{server} and no credentials were found in ". + ".netrc.", + server => $hostname))) + unless (defined($username)); + + my $self = {}; + bless($self, $class); + + $self->{hostname} = $hostname; + $self->{username} = $username; + $self->{password} = $password; + $self->{noverify} = $noverify; + + my $ua = new Sys::VirtV2V::Transfer::ESX::UA( + $username, + $password, + $noverify + ); + + # ESX path looks like this: + # [yellow:storage1] win2k3r2-32/win2k3r2-32.vmdk + + # Strip out datastore and vmdk name + $path =~ /^\[(.*)\]\s+(.*)\.vmdk$/ + or die("Failed to parse ESX path: $path"); + my $datastore = $1; + my $vmdk = $2; + + $self->{uri} = _get_vol_uri($hostname, $vmdk, $datastore); + + # Get the size of the volume. At the same time, verify we have the correct + # URI. + my $retried = 0; + for(;;) { + eval { + $self->{size} = $ua->get_content_length($self->{uri}); + }; + + last if defined($self->{size}); + + if ($@) { + # Re-throw an unstructured error + die ($@) if (ref($@) ne 'ARRAY'); + + my $code = $@->[0]; + my $msg = $@->[1]; + + my $r = $@; + # If a disk is actually a snapshot image it will have '-00000n' + # appended to its name, e.g.: + # [yellow:storage1] RHEL4-X/RHEL4-X-000003.vmdk + # The flat storage is still called RHEL4-X-flat, however. If we got + # a 404 and the vmdk name looks like it might be a snapshot, try + # again without the snapshot suffix. + # XXX: We're in flaky heuristic territory here. When the libvirt ESX + # driver implements the volume apis we should look for this + # information there instead. + if ($code == 404 && !$retried && $vmdk =~ /^(.*)-\d+$/) { + $vmdk = $1; + $self->{uri} = _get_vol_uri($hostname, $vmdk, $datastore); + } + + else { + $self->_report_error($code, $msg); + } + + $retried = 1; + } + } -=item transfer(conn, path, target) + # Create a libvirt-friendly volume name + $self->{name} = $vmdk; + $self->{name} =~ s,/,_,g; -Transfer <path> from a remote ESX server. Server and authentication details will -be taken from <conn>. Storage will be created using <target>. + return $self; +} + +# Volume path looks like this: +# https://yellow.rhev.marston/folder/win2k3r2-32/win2k3r2-32-flat.vmdk? \ +# dcPath=ha-datacenter&dsName=yellow:storage1 +sub _get_vol_uri +{ + my ($server, $vmdk, $datastore) = @_; + + my $uri = URI->new("https://".$server); + $uri->path("/folder/$vmdk-flat.vmdk"); + $uri->query_form(dcPath => "ha-datacenter", dsName => $datastore); + + return $uri; +} + +=item local_path + +ESX cannot return a local path. This function will die(). =cut -sub transfer +sub local_path { - my $class = shift; + die(user_message(__"virt-v2v cannot write to an ESX connection")); +} - my ($conn, $path, $target) = @_; +=item get_read_stream - my $uri = $conn->{uri}; - my $username = $conn->{username}; - my $password = $conn->{password}; +Get a read stream for this volume. - die("URI not defined for connection") unless (defined($uri)); +=cut - die(user_message(__x("Authentication is required to connect to ". - "{server} and no credentials were found in ". - ".netrc.", - server => $conn->{hostname}))) - unless (defined($username)); +sub get_read_stream +{ + my $self = shift; + return new Sys::VirtV2V::Transfer::ESX::ReadStream( + $self->{uri}, + $self->{username}, + $self->{password}, + $self->{noverify}, + sub { $self->_report_error(@_) } + ); +} - # Look for no_verify in the URI - my %query = $uri->query_form; +=item get_write_stream - my $noverify = 0; - $noverify = 1 if (exists($query{no_verify}) && $query{no_verify} eq "1"); +get_write_stream is not implemented for ESX. This function will die with an +error message if called. - # Initialise a user agent - my $ua = Sys::VirtV2V::Transfer::ESX::UA->new($conn->{hostname}, - $username, - $password, - $target, - $noverify); +=cut + +sub get_write_stream +{ + die(user_message(__"Unable to write to an ESX connection")); +} + +=item esx_get_name - return $ua->get_volume($path); +Return a libvirt-friendly name for this ESX path. + +=cut + +sub esx_get_name +{ + return shift->{name}; +} + +=item esx_get_size + +Return the size of the volume which will be returned. + +=cut + +sub esx_get_size +{ + return shift->{size}; +} + +sub _report_error +{ + my $self = shift; + my ($code, $msg) = @_; + + if ($code == 401) { + die(user_message(__x("Authentication error connecting to ". + "{server}. Used credentials for {username} ". + "from .netrc.", + server => $self->{hostname}, + username => $self->{username}))) + } + + die(user_message(__x("Failed to connect to ESX server: {error}", + error => $msg))); } =back =head1 COPYRIGHT -Copyright (C) 2009,2010 Red Hat Inc. +Copyright (C) 2010 Red Hat Inc. =head1 LICENSE @@ -361,7 +477,6 @@ Please see the file COPYING.LIB for the full license. =head1 SEE ALSO -L<Sys::VirtV2V::Converter(3pm)>, L<virt-v2v(1)>, L<http://libguestfs.org/>. diff --git a/lib/Sys/VirtV2V/Transfer/Local.pm b/lib/Sys/VirtV2V/Transfer/Local.pm new file mode 100644 index 0000000..3207c4f --- /dev/null +++ b/lib/Sys/VirtV2V/Transfer/Local.pm @@ -0,0 +1,238 @@ +# Sys::VirtV2V::Transfer::Local +# Copyright (C) 2010 Red Hat Inc. +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + +use strict; +use warnings; + +package Sys::VirtV2V::Transfer::Local::ReadStream; + +use Sys::VirtV2V::Util qw(user_message); +use Locale::TextDomain 'virt-v2v'; + +sub new +{ + my $class = shift; + my ($path, $is_sparse) = @_; + + my $self = {}; + bless($self, $class); + + my $fh; + open($fh, '<', $path) + or die(user_message(__x("Unable to open {path} for reading: {error}", + path => $path, + error => $!))); + + $self->{fh} = $fh; + $self->{is_sparse} = $is_sparse; + + return $self; +} + +sub read +{ + my $self = shift; + my ($size) = @_; + + my $buf; + my $in = read($self->{fh}, $buf, $size); + $self->_read_error($!) unless (defined($in)); + + return "" if ($in == 0); + return $buf; +} + +sub close +{ + my $self = shift; + close($self->{fh}) or $self->_read_error($!); +} + +sub DESTROY +{ + my $self = shift; + $self->close(); +} + +sub _read_error +{ + my $self = shift; + my ($error) = @_; + + die(user_message(__x("Error reading from {path}: {error}", + path => $self->{path}, + error => $error))); +} + + +package Sys::VirtV2V::Transfer::Local::WriteStream; + +use Sys::VirtV2V::Util qw(user_message); +use Locale::TextDomain 'virt-v2v'; + +sub new +{ + my $class = shift; + my ($path, $is_sparse) = @_; + + my $self = {}; + bless($self, $class); + + my $fh; + open($fh, '>', $path) + or die(user_message(__x("Unable to open {path} for writing: {error}", + path => $path, + error => $!))); + + $self->{fh} = $fh; + $self->{is_sparse} = $is_sparse; + + return $self; +} + +sub write +{ + my $self = shift; + my ($buf) = @_; + + print { $self->{fh} } $buf or $self->_write_error($!); +} + +sub close +{ + my $self = shift; + close($self->{fh}) or $self->_write_error($!); +} + +sub _write_error +{ + my $self = shift; + my ($error) = @_; + + die(user_message(__x("Error writing to {path}: {error}", + path => $self->{path}, + error => $error))); +} + +sub DESTROY +{ + my $self = shift; + $self->close(); +} + + +package Sys::VirtV2V::Transfer::Local; + +use POSIX; +use File::Spec; +use File::stat; + +use Sys::VirtV2V::Util qw(user_message); +use Locale::TextDomain 'virt-v2v'; + +=pod + +=head1 NAME + +Sys::VirtV2V::Transfer::Local - Access local storage + +=head1 METHODS + +=over + +=item new(path, is_sparse) + +Create a new Local transfer object. + +=cut + +sub new +{ + my $class = shift; + my ($path, $is_sparse) = @_; + + my $self = {}; + bless($self, $class); + + $self->{path} = $path; + $self->{is_sparse} = $is_sparse; + + return $self; +} + +=item local_path + +Return a local path to the file. + +=cut + +sub local_path +{ + return shift->{path}; +} + +=item get_read_stream + +Get a stream to receive data from a local file. + +=cut + +sub get_read_stream +{ + my $self = shift; + + return new Sys::VirtV2V::Transfer::Local::ReadStream( + $self->{path}, + $self->{is_sparse} + ); +} + +=item get_write_stream + +Get a stream to write data to a local file. + +=cut + +sub get_write_stream +{ + my $self = shift; + + return new Sys::VirtV2V::Transfer::Local::WriteStream( + $self->{path}, + $self->{is_sparse} + ); +} + +=back + +=head1 COPYRIGHT + +Copyright (C) 2010 Red Hat Inc. + +=head1 LICENSE + +Please see the file COPYING.LIB for the full license. + +=head1 SEE ALSO + +L<Sys::VirtV2V::Converter(3pm)>, +L<virt-v2v(1)>, +L<http://libguestfs.org/>. + +=cut + +1; diff --git a/lib/Sys/VirtV2V/Transfer/LocalCopy.pm b/lib/Sys/VirtV2V/Transfer/LocalCopy.pm deleted file mode 100644 index cdff97e..0000000 --- a/lib/Sys/VirtV2V/Transfer/LocalCopy.pm +++ /dev/null @@ -1,150 +0,0 @@ -# Sys::VirtV2V::Transfer::LocalCopy -# Copyright (C) 2010 Red Hat Inc. -# -# This library is free software; you can redistribute it and/or -# modify it under the terms of the GNU Lesser General Public -# License as published by the Free Software Foundation; either -# version 2 of the License, or (at your option) any later version. -# -# This library is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU -# Lesser General Public License for more details. -# -# You should have received a copy of the GNU Lesser General Public -# License along with this library; if not, write to the Free Software -# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA - -package Sys::VirtV2V::Transfer::LocalCopy; - -use POSIX; -use File::Spec; -use File::stat; - -use Sys::VirtV2V::Util qw(user_message); - -use Locale::TextDomain 'virt-v2v'; - -=pod - -=head1 NAME - -Sys::VirtV2V::Transfer::LocalCopy - Copy a guest's local storage - -=head1 SYNOPSIS - - use Sys::VirtV2V::Transfer::LocalCopy; - - $vol = Sys::VirtV2V::Transfer::LocalCopy->transfer($conn, $path, $target); - -=head1 DESCRIPTION - -Sys::VirtV2V::Transfer::LocalCopy retrieves guest storage devices from local -storage. - -=head1 METHODS - -=over - -=item transfer(conn, path, target) - -Transfer <path> from local storage. Storage will be created using <target>. - -=cut - -sub transfer -{ - my $class = shift; - - my ($conn, $path, $target) = @_; - - my (undef, undef, $name) = File::Spec->splitpath($path); - - if ($target->volume_exists($name)) { - warn user_message(__x("WARNING: storage volume {name} already exists ". - "on the target. NOT copying it again. Delete ". - "the volume and retry to copy again.", - name => $name)); - return $target->get_volume($name); - } - - my $fh; - open($fh, '<', $path) - or die(user_message(__x("Unable to open {path} for reading: {error}", - path => $path, - error => $!))); - - my $st = stat($fh) - or die(user_message(__x("Unable to stat {path}: {error}", - path => $path, - error => $!))); - - my $size; - - # If it's a block device, use the output of blockdev command - if (S_ISBLK($st->mode)) { - my $blockdev; - open($blockdev, '-|', 'blockdev', '--getsize64', $path) - or die("Unable to execute blockdev: $!"); - - while (<$blockdev>) { - if (defined($size)) { - my $error = "blockdev returned multiple output lines:\n$size\n"; - $error .= $_; - while(<$blockdev>) { - $error .= $_; - } - die($error); - } - chomp; - $size = $_; - } - - close($blockdev) or die("blockdev returned an error: $size"); - } - - # Otherwise use the size of the file directly - else { - $size = $st->size; - } - - my $vol = $target->create_volume($name, $size); - $vol->open(); - - for (;;) { - my $buffer; - # Transfer in block chunks - my $in = sysread($fh, $buffer, $st->blksize); - die(user_message(__x("Error reading data from {path}: {error}", - path => $path, - error => $!))) if (!defined($in)); - - last if ($in == 0); - - $vol->write($buffer); - } - - $vol->close(); - - return $vol; -} - -=back - -=head1 COPYRIGHT - -Copyright (C) 2010 Red Hat Inc. - -=head1 LICENSE - -Please see the file COPYING.LIB for the full license. - -=head1 SEE ALSO - -L<Sys::VirtV2V::Converter(3pm)>, -L<virt-v2v(1)>, -L<http://libguestfs.org/>. - -=cut - -1; diff --git a/lib/Sys/VirtV2V/Transfer/SSH.pm b/lib/Sys/VirtV2V/Transfer/SSH.pm index 5f54f0e..39eaa66 100644 --- a/lib/Sys/VirtV2V/Transfer/SSH.pm +++ b/lib/Sys/VirtV2V/Transfer/SSH.pm @@ -15,114 +15,21 @@ # License along with this library; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA -package Sys::VirtV2V::Transfer::SSH; +use strict; +use warnings; -use POSIX; -use File::Spec; -use File::stat; +package Sys::VirtV2V::Transfer::SSH::Stream; use Sys::VirtV2V::Util qw(user_message); - use Locale::TextDomain 'virt-v2v'; -=pod - -=head1 NAME - -Sys::VirtV2V::Transfer::SSH - Copy a remote guest's storage via ssh - -=head1 SYNOPSIS - - use Sys::VirtV2V::Transfer::SSH; - - $vol = Sys::VirtV2V::Transfer::SSH->transfer($conn, $path, $target); - -=head1 DESCRIPTION - -Sys::VirtV2V::Transfer::SSH retrieves guest storage devices from a remote server -via SSH. - -=head1 METHODS - -=over - -=item transfer(conn, path, target) - -Transfer <path> from the remove server. Storage will be created using <target>. - -=cut - -sub transfer +sub new { my $class = shift; + my ($hostname, $username, $command) = @_; - my ($conn, $path, $target) = @_; - - my (undef, undef, $name) = File::Spec->splitpath($path); - - if ($target->volume_exists($name)) { - warn user_message(__x("WARNING: storage volume {name} already exists ". - "on the target. NOT copying it again. Delete ". - "the volume and retry to copy again.", - name => $name)); - return $target->get_volume($name); - } - - my $uri = $conn->{uri}; - my $username = $conn->{username}; - my $password = $conn->{password}; - my $host = $conn->{hostname}; - - die("URI not defined for connection") unless (defined($uri)); - - my ($pid, $size, $fh, $error) = - _connect($host, $username, $path); - - my $vol = $target->create_volume($name, $size); - $vol->open(); - - my $written = 0; - for (;;) { - my $buffer; - # Transfer in 8k chunks - my $in = read($fh, $buffer, 8 * 1024); - die(user_message(__x("Error reading data from {path}: {error}", - path => $path, - error => $!))) if (!defined($in)); - - last if ($in == 0); - - $vol->write($buffer); - $written += length($buffer); - } - - $vol->close(); - - die(user_message(__x("Didn't receive full volume. Received {received} ". - "of {total} bytes.", - received => $written, - total => $size))) unless ($written == $size); - - waitpid($pid, 0) == $pid or die("error reaping child: $!"); - # If the child returned an error, check for anything on its stderr - if ($? != 0) { - my $msg = ""; - while (<$error>) { - $msg .= $_; - } - die(user_message(__x("Unexpected error copying {path} from {host}. ". - "Command output: {output}", - path => $path, - host => $uri->host, - output => $msg))); - } - - return $vol; -} - -sub _connect -{ - my ($host, $username, $path) = @_; + my $self = {}; + bless($self, $class); my ($stdin_read, $stdin_write); my ($stdout_read, $stdout_write); @@ -137,22 +44,8 @@ sub _connect my @command; push(@command, 'ssh'); push(@command, '-l', $username) if (defined($username)); - push(@command, $host); - - # Return the size of the remote path on the first line, followed by its - # contents. - # The bit arithmetic with the output of stat is a translation into shell - # of the S_ISBLK macro. If the remote device is a block device, stat - # will simply return the size of the block device inode. In this case, - # we use the output of blockdev --getsize64 instead. - push(@command, - "dev=$path; ". - 'if [[ $(((0x$(stat -L -c %f $dev)&0170000)>>12)) == 6 ]]; then '. - 'blockdev --getsize64 $dev; '. - 'else '. - 'stat -L -c %s $dev; '. - 'fi; '. - 'cat $dev'); + push(@command, $hostname); + push(@command, $command); # Close the ends of the pipes we don't need close($stdin_write); @@ -176,36 +69,271 @@ sub _connect close($stdout_write); close($stderr_write); - # Check that we don't get output on stderr before we read the file size + $self->{pid} = $pid; + $self->{stdin} = $stdin_write; + $self->{stdout} = $stdout_read; + $self->{stderr} = $stderr_read; + + $self->{hostname} = $hostname; + + return $self; +} + +sub close +{ + my $self = shift; + + # Nothing to do if it's already closed. + return unless (exists($self->{pid})); + + my $pid = $self->{pid}; + my $stderr = $self->{stderr}; + + # Must close stdin before waitpid, or process will not exit when writing + close($self->{stdin}); + close($self->{stdout}); + + waitpid($pid, 0) == $pid or die("error reaping child: $!"); + # If the child returned an error, check for anything on its stderr + if ($? != 0) { + my $msg = ""; + while (<$stderr>) { + $msg .= $_; + } + die(user_message(__x("Unexpected error copying {path} from {host}. ". + "Command output: {output}", + path => $self->{path}, + host => $self->{hostname}, + output => $msg))); + } + + close($self->{stderr}); + + delete($self->{pid}); + delete($self->{stdin}); + delete($self->{stdout}); + delete($self->{stderr}); +} + +sub DESTROY +{ + my $self = shift; + $self->close(); +} + +sub _check_stderr +{ + my $self = shift; + my ($rw, $errfun) = @_; + for(;;) { - my ($rin, $rout); + my ($rin, $rout, $win, $wout); $rin = ''; - vec($rin, fileno($stdout_read), 1) = 1; - vec($rin, fileno($stderr_read), 1) = 1; + vec($rin, fileno($self->{stderr}), 1) = 1; + + # Waiting to read from stdout + if ($rw == 0) { + $win = undef; + vec($rin, fileno($self->{stdout}), 1) = 1; + } + + # Waiting to write to stdin + else { + $win = ''; + vec($win, fileno($self->{stdin}), 1) = 1; + } - my $nfound = select($rout=$rin, undef, undef, undef); + my $nfound = select($rout=$rin, $wout=$win, undef, undef); die("select failed: $!") if ($nfound < 0); - if (vec($rout, fileno($stderr_read), 1) == 1) { - my $stderr = ''; - while(<$stderr_read>) { - $stderr .= $_; + my $stderr = $self->{stderr}; + if (vec($rout, fileno($stderr), 1) == 1) { + my $error = ''; + while(<$stderr>) { + $error .= $_; } - die(user_message(__x("Unexpected error getting {path}: ". - "{output}", - path => $path, output => $stderr))); + &$errfun($error); } - if (vec($rout, fileno($stdout_read), 1) == 1) { - last; - } + last if (vec($rout, fileno($self->{stdout}), 1) == 1 || + vec($wout, fileno($self->{stdin}), 1) == 1); } +} + +package Sys::VirtV2V::Transfer::SSH::ReadStream; - # First line returned is the output of stat - my $size = <$stdout_read>; +use Sys::VirtV2V::Util qw(user_message); +use Locale::TextDomain 'virt-v2v'; - return ($pid, $size, $stdout_read, $stderr_read); +@Sys::VirtV2V::Transfer::SSH::ReadStream::ISA = + qw(Sys::VirtV2V::Transfer::SSH::Stream); + +sub new +{ + my $class = shift; + my ($path, $hostname, $username, $is_sparse) = @_; + + my $self = $class->SUPER::new($hostname, $username, "dd if=$path"); + + $self->{path} = $path; + + # Check that the stream becomes readable without anything on stderr + $self->SUPER::_check_stderr(0, sub { $self->_read_error(@_) } ); + + return $self; +} + +sub read +{ + my $self = shift; + my ($size) = @_; + + my $buf; + my $in = read($self->{stdout}, $buf, $size); + $self->_read_error($!) unless (defined($in)); + + return "" if ($in == 0); + return $buf; +} + +sub _read_error +{ + my $self = shift; + my ($error) = @_; + + die(user_message(__x("Error reading from {path}: {error}", + path => $self->{path}, + error => $error))); +} + +package Sys::VirtV2V::Transfer::SSH::WriteStream; + +use Sys::VirtV2V::Util qw(user_message); +use Locale::TextDomain 'virt-v2v'; + +@Sys::VirtV2V::Transfer::SSH::WriteStream::ISA = + qw(Sys::VirtV2V::Transfer::SSH::Stream); + +sub new +{ + my $class = shift; + my ($path, $hostname, $username, $is_sparse) = @_; + + my $self = $class->SUPER::new($hostname, $username, "dd of=$path"); + + $self->{path} = $path; + + # Check that the stream becomes writable without anything on stderr + $self->SUPER::_check_stderr(1, sub { $self->_write_error(@_) }); + + return $self; +} + +sub write +{ + my $self = shift; + my ($buf) = @_; + + print { $self->{stdin} } $buf or $self->_write_error($!); +} + +sub _write_error +{ + my $self = shift; + my ($error) = @_; + + die(user_message(__x("Error writing data to {path}: {error}", + path => $self->{path}, + error => $error))); +} + +package Sys::VirtV2V::Transfer::SSH; + +use POSIX; +use File::Spec; +use File::stat; + +use Sys::VirtV2V::Util qw(user_message); +use Locale::TextDomain 'virt-v2v'; + +=pod + +=head1 NAME + +Sys::VirtV2V::Transfer::SSH - Transfer data over an SSH connection + +=head1 METHODS + +=over + +=item new(path, hostname, username, is_sparse) + +Create a new SSH transfer object. + +=cut + +sub new +{ + my $class = shift; + my ($path, $hostname, $username, $is_sparse) = @_; + + my $self = {}; + bless($self, $class); + + $self->{path} = $path; + $self->{hostname} = $hostname; + $self->{username} = $username; + $self->{is_sparse} = $is_sparse; + + return $self; +} + +=item local_path + +SSH cannot currently return a local path. This function will die(). + +=cut + +sub local_path +{ + die(user_message(__"virt-v2v cannot yet write to an SSH connection")); +} + +=item get_read_stream + +Get a stream to read from a file over an SSH connection. + +=cut + +sub get_read_stream +{ + my $self = shift; + + return new Sys::VirtV2V::Transfer::SSH::ReadStream( + $self->{path}, + $self->{hostname}, + $self->{username}, + $self->{is_sparse} + ); +} + +=item get_write_stream + +Get a stream to write to a file over an SSH connection. + +=cut + +sub get_write_stream +{ + my $self = shift; + + return new Sys::VirtV2V::Transfer::SSH::WriteStream( + $self->{path}, + $self->{hostname}, + $self->{username}, + $self->{is_sparse} + ); } =back diff --git a/lib/Sys/VirtV2V/Util.pm b/lib/Sys/VirtV2V/Util.pm index 699899a..4aab8af 100644 --- a/lib/Sys/VirtV2V/Util.pm +++ b/lib/Sys/VirtV2V/Util.pm @@ -20,13 +20,16 @@ package Sys::VirtV2V::Util; use strict; use warnings; +use Sys::Virt; +use XML::DOM; + use Locale::TextDomain 'virt-v2v'; require Exporter; use vars qw(@EXPORT_OK @ISA); @ISA = qw(Exporter); -@EXPORT_OK = qw(augeas_error user_message); +@EXPORT_OK = qw(augeas_error user_message parse_libvirt_volinfo); =pod @@ -128,6 +131,42 @@ sub user_message return __x("virt-v2v: {message}\n", message => $msg); } +=item parse_libvirt_volinfo(vol) + +Return name, format, size, is_sparse, is_block for a given a libvirt volume. + +=cut + +sub parse_libvirt_volinfo +{ + my ($vol) = @_; + + my $voldom = new XML::DOM::Parser->parse($vol->get_xml_description()); + + my ($name, $format, $size, $is_sparse, $is_block); + + ($name) = $voldom->findnodes('/volume/name/text()'); + $name = $name->getData(); + + ($format) = $voldom->findnodes('/volume/target/format/@type'); + $format = $format->getValue(); + + my $info = $vol->get_info(); + + $size = $info->{capacity}; + + my $allocation = $info->{allocation}; + if ($allocation < $size) { + $is_sparse = 1; + } else { + $is_sparse = 0; + } + + $is_block = $info->{type} == Sys::Virt::StorageVol::TYPE_BLOCK ? 1 : 0; + + return ($name, $format, $size, $is_sparse, $is_block); +} + =back =head1 COPYRIGHT diff --git a/v2v/virt-v2v.pl b/v2v/virt-v2v.pl index 93bfcd5..c6a2e55 100755 --- a/v2v/virt-v2v.pl +++ b/v2v/virt-v2v.pl @@ -32,10 +32,10 @@ use Sys::Guestfs::Lib qw(get_partitions inspect_all_partitions use Sys::VirtV2V; use Sys::VirtV2V::Config; use Sys::VirtV2V::Converter; -use Sys::VirtV2V::Connection::LibVirt; -use Sys::VirtV2V::Connection::LibVirtXML; -use Sys::VirtV2V::Target::LibVirt; -use Sys::VirtV2V::Target::RHEV; +use Sys::VirtV2V::Connection::LibVirtSource; +use Sys::VirtV2V::Connection::LibVirtTarget; +use Sys::VirtV2V::Connection::LibVirtXMLSource; +use Sys::VirtV2V::Connection::RHEVTarget; use Sys::VirtV2V::ExecHelper; use Sys::VirtV2V::GuestfsHandle; use Sys::VirtV2V::GuestOS; @@ -282,7 +282,8 @@ if ($output_method eq "libvirt") { -exitval => 1 }) unless (defined($output_pool)); - $target = new Sys::VirtV2V::Target::LibVirt($output_uri, $output_pool); + $target = new Sys::VirtV2V::Connection::LibVirtTarget($output_uri, + $output_pool); } elsif ($output_method eq "rhev") { @@ -291,7 +292,7 @@ elsif ($output_method eq "rhev") { -exitval => 1 }) unless (defined($output_storage_domain)); - $target = new Sys::VirtV2V::Target::RHEV($output_storage_domain); + $target = new Sys::VirtV2V::Connection::RHEVTarget($output_storage_domain); } else { @@ -299,48 +300,41 @@ else { output => $output_method))); } -# Get an appropriate Connection -my $conn; -eval { - if ($input_method eq "libvirtxml") { - my $path = shift(@ARGV) or - pod2usage({ -message => user_message(__"You must specify a filename"), - -exitval => 1 }); - - # Warn if we were given more than 1 argument - if(scalar(@_) > 0) { - print STDERR user_message - (__x("WARNING: {modulename} only takes a single filename.", - modulename => 'libvirtxml')); - } - - $conn = Sys::VirtV2V::Connection::LibVirtXML->new($path, $target); +# Get an appropriate Source +my $source; +if ($input_method eq "libvirtxml") { + my $path = shift(@ARGV) or + pod2usage({ -message => user_message(__"You must specify a filename"), + -exitval => 1 }); + + # Warn if we were given more than 1 argument + if(scalar(@_) > 0) { + warn user_message + (__x("WARNING: {modulename} only takes a single filename.", + modulename => 'libvirtxml')); } - elsif ($input_method eq "libvirt") { - my $name = shift(@ARGV) or - pod2usage({ -message => user_message(__"You must specify a guest"), - -exitval => 1 }); + $source = Sys::VirtV2V::Connection::LibVirtXMLSource->new($path); +} - $conn = Sys::VirtV2V::Connection::LibVirt->new($input_uri, $name, - $target); +elsif ($input_method eq "libvirt") { + my $name = shift(@ARGV) or + pod2usage({ -message => user_message(__"You must specify a guest"), + -exitval => 1 }); - # Warn if we were given more than 1 argument - if(scalar(@_) > 0) { - print STDERR user_message - (__x("WARNING: {modulename} only takes a single domain name.", - modulename => 'libvirt')); - } - } + $source = Sys::VirtV2V::Connection::LibVirtSource->new($input_uri, $name); - else { - print STDERR user_message __x("{input} is not a valid input method", - input => $input_method); - exit(1); + # Warn if we were given more than 1 argument + if(scalar(@_) > 0) { + warn user_message + (__x("WARNING: {modulename} only takes a single domain name.", + modulename => 'libvirt')); } -}; -if ($@) { - print STDERR $@; +} + +else { + warn user_message(__x("{input} is not a valid input method", + input => $input_method)); exit(1); } @@ -348,12 +342,20 @@ if ($@) { ############################################################################### ## Start of processing +# Check that the guest doesn't already exist on the target +die(user_message(__x("Domain {name} already exists on the target.", + name => $source->get_name))) + if ($target->guest_exists($source->get_name())); + +# Copy source storage to target +$source->copy_storage($target); + # Get a libvirt configuration for the guest -my $dom = $conn->get_dom(); +my $dom = $source->get_dom(); exit(1) unless(defined($dom)); # Get a list of the guest's transfered storage devices -my $storage = $conn->get_storage_paths(); +my $storage = $source->get_storage_paths(); # Create the transfer iso if required my $transferiso; @@ -375,7 +377,7 @@ eval { # Modify the guest and its metadata $guestcaps = Sys::VirtV2V::Converter->convert($g, $guestos, $config, $dom, $os, - $conn->get_storage_devices()); + $source->get_storage_devices()); }; # If any of the above commands result in failure, we need to ensure that the -- 1.7.2.3

2 5

[PATCH] Add more exclusions to .gitignore
by Matthew Booth 21 Sep '10

21 Sep '10

--- .gitignore | 6 +++++- 1 files changed, 5 insertions(+), 1 deletions(-)

2 3

[PATCH 0/2] *** SUBJECT HERE ***
by Matthew Booth 21 Sep '10

21 Sep '10

*** BLURB HERE *** Matthew Booth (2): New APIs: hopen-device hopen-file hread hwrite hseek hclose hclose-all Update pwrite to write a full buffer daemon/.gitignore | 1 + daemon/Makefile.am | 1 + daemon/file.c | 20 ++- daemon/hfile.c | 249 ++++++++++++++++++++++++++++++++++++++++ daemon/m4/gnulib-cache.m4 | 3 +- generator/generator_actions.ml | 120 ++++++++++++++++++- po/POTFILES.in | 2 + regressions/Makefile.am | 1 + regressions/test-hfile.pl | 236 +++++++++++++++++++++++++++++++++++++ src/MAX_PROC_NR | 2 +- 10 files changed, 620 insertions(+), 15 deletions(-) create mode 100644 daemon/hfile.c create mode 100755 regressions/test-hfile.pl -- 1.7.2.3

1 2

[PATCH] Fix appliance build dependency problem
by Matthew Booth 21 Sep '10

21 Sep '10

The appliance was being completely rebuilt every time guestfsd was updated. This was because make.sh depended on guestfsd, which it had to do because it called update.sh to install guestfsd. This fix removes the call to update.sh in make.sh, and therefore the dependency on guestfsd. The Makefile already includes a rule to run update.sh when guestfsd is updated, so this was unnecessary. --- appliance/Makefile.am | 2 +- appliance/make.sh.in | 4 ---- 2 files changed, 1 insertions(+), 5 deletions(-) diff --git a/appliance/Makefile.am b/appliance/Makefile.am index bbf3c24..172f0cf 100644 --- a/appliance/Makefile.am +++ b/appliance/Makefile.am @@ -53,7 +53,7 @@ VMLINUZ = vmlinuz.$(REPO).$(host_cpu) # This is for building the normal appliance: $(INITRAMFSIMG) $(VMLINUZ): $(top_builddir)/initramfs/fakeroot.log -$(top_builddir)/initramfs/fakeroot.log: make.sh kmod.whitelist packagelist $(top_builddir)/daemon/guestfsd +$(top_builddir)/initramfs/fakeroot.log: make.sh kmod.whitelist packagelist mv $(INITRAMFSIMG) $(INITRAMFSIMG).bak 2>/dev/null; : mv $(VMLINUZ) $(VMLINUZ).bak 2>/dev/null; : if ! bash make.sh; then rm -f $@; exit 1; fi diff --git a/appliance/make.sh.in b/appliance/make.sh.in index 9150f55..ccaa757 100755 --- a/appliance/make.sh.in +++ b/appliance/make.sh.in @@ -168,10 +168,6 @@ __EOF__ ls -lh $koutput - # Now directly run the update script to copy/update the daemon in the - # initramfs. - cd appliance && bash update.sh - elif [ "@DIST@" = "DEBIAN" ]; then cd @top_builddir@/appliance debirf make -n debian -- 1.7.2.3

2 1