[PATCH FOR DISCUSSION ONLY 0/2] v2v: Copy static IP address information over for Windows guests (RHBZ#1626503).
This patch is just for discussion. There are still a couple of issues that I'm trying to fix. One is that all of the test guests I have, even ones with static IPs, have multiple interfaces, some using DHCP, so the conditions for adding the Powershell script don't kick in. This makes testing very awkward. However a bigger issue is that I think the premise is wrong. In some registries I've found that the MAC address _is_ stored (in \CurrentControlSet\Control\NetworkSetup2\Interfaces\{GUID} key PermanentAddress). Rich.
Extend the inspection data to include information about source guest network interfaces. Collect network interface information for Windows guests (only). For a test guest using DHCP this returned [my formatting added]: i_interfaces = [ { if_name="Local Area Connection* 1", if_default_gateway=, if_ip_address=, if_enable_dhcp=true, if_subnet_mask= }, { if_name="Ethernet", if_default_gateway=, if_ip_address=, if_enable_dhcp=true, if_subnet_mask= } ] Another guest which had a static IP returned: i_interfaces = [ { if_name="Ethernet1", if_default_gateway=, if_ip_address=, if_enable_dhcp=true, if_subnet_mask= }, { if_name="Ethernet", if_default_gateway=, if_ip_address=, if_enable_dhcp=true, if_subnet_mask= }, { if_name="Local Area Connection* 1", if_default_gateway=, if_ip_address=, if_enable_dhcp=true, if_subnet_mask= }, { if_name="Ethernet0", if_default_gateway=10.19.2.1, if_ip_address=10.19.2.99, if_enable_dhcp=false, if_subnet_mask=255.255.255.128 } ] --- common/mltools/registry.ml | 19 ++++++ common/mltools/registry.mli | 4 ++ v2v/inspect_source.ml | 113 +++++++++++++++++++++++++++++++++++- v2v/types.ml | 32 ++++++++-- v2v/types.mli | 23 ++++++-- 5 files changed, 180 insertions(+), 11 deletions(-) diff --git a/common/mltools/registry.ml b/common/mltools/registry.ml index 1ed465910..141917db4 100644 --- a/common/mltools/registry.ml +++ b/common/mltools/registry.ml @@ -96,3 +96,22 @@ let decode_utf16le str = Bytes.unsafe_set copy i cl done; Bytes.to_string copy + +let rec split_multi_sz ss = + let n = String.length ss in + List.rev (_split_multi_sz ss 0 n []) + +and _split_multi_sz ss i n acc = + let lenbytes = _utf16_length_in_bytes ss i n in + if lenbytes = 0 then acc (* base case *) + else ( + (* Get the next string without \0\0. *) + let r = String.sub ss i (lenbytes-2) in + _split_multi_sz ss (i+lenbytes) n (r::acc) + ) + +(* Find the length of the string in bytes including terminating \0\0. *) +and _utf16_length_in_bytes ss i n = + if i+1 >= n then 0 + else if ss.[i] = '\000' && ss.[i+1] = '\000' then 2 + else 2 + _utf16_length_in_bytes ss (i+2) n diff --git a/common/mltools/registry.mli b/common/mltools/registry.mli index 5cbcacf5e..226552809 100644 --- a/common/mltools/registry.mli +++ b/common/mltools/registry.mli @@ -52,3 +52,7 @@ val encode_utf16le : string -> string val decode_utf16le : string -> string (** Helper: Take a UTF-16LE string and decode it to UTF-8. *) + +val split_multi_sz : string -> string list +(** Helper: Split up a multiple string (type = REG_MULTI_SZ = 7). + This does {i not} decode the strings. *) diff --git a/v2v/inspect_source.ml b/v2v/inspect_source.ml index c1a7e5737..b05a5b5e2 100644 --- a/v2v/inspect_source.ml +++ b/v2v/inspect_source.ml @@ -98,6 +98,14 @@ let rec inspect_source root_choice g = | _ -> "", "", "", "" in + (* For Windows only get the network interfaces of the source. *) + let ifs = + match typ with + | "windows" -> + get_network_interfaces g root system_hive current_cs + | _ -> + [] in + let inspect = { i_root = root; i_type = typ; @@ -113,11 +121,12 @@ let rec inspect_source root_choice g = i_mountpoints = mps; i_apps = apps; i_apps_map = apps_map; - i_firmware = get_firmware_bootable_device g; i_windows_systemroot = systemroot; i_windows_software_hive = software_hive; i_windows_system_hive = system_hive; i_windows_current_control_set = current_cs; + i_firmware = get_firmware_bootable_device g; + i_interfaces = ifs; } in debug "%s" (string_of_inspect inspect); @@ -218,6 +227,108 @@ and get_firmware_bootable_device g = | [] -> I_BIOS | partitions -> I_UEFI partitions +(* For Windows only get the network interfaces of the source. + * + * We start at \CurrentControlSet\Control\Network. Under this + * node is the Network Adapter class (with the specific class GUID + * defined below). Under here is a list of network adapter GUIDs + * which we can cross reference with + * \CurrentControlSet\Services\Tcpip\Parameters\Interfaces\{GUID} + * to find the DHCP configuration, IP address etc. + *) +and get_network_interfaces g root system_hive current_cs = + with_return (fun {return} -> + Registry.with_hive_readonly g system_hive (fun reg -> + (* https://docs.microsoft.com/en-us/windows-hardware/drivers/install/system-def... *) + let network_adapter_guid = "{4D36E972-E325-11CE-BFC1-08002BE10318}" in + let path = [ current_cs; "Control"; "Network"; + network_adapter_guid ] in + let node = + match Registry.get_node reg path with + | Some node -> node + | None -> return [] in + + let interfaces = g#hivex_node_children node in + let interfaces = Array.to_list interfaces in + + (* Get the node name (GUID). If it exists then the + * <node>\Connection\Name key is the interface name such + * as "Ethernet0". + * + * The GUID can also be cross-referenced under + * CurrentControlSet\Services\Tcpip\Parameters\Interfaces + * (see below). + *) + let interfaces = List.filter_map ( + fun { G.hivex_node_h = node } -> + try + let guid = g#hivex_node_name node in + let connection_node = g#hivex_node_get_child node "Connection" in + let if_name_v = g#hivex_node_get_value connection_node "Name" in + let if_name = g#hivex_value_string if_name_v in + Some (guid, if_name) + with + Not_found | G.Error _ -> None + ) interfaces in + + (* Cross reference GUID. *) + let interfaces = List.filter_map ( + fun (guid, if_name) -> + let path = [ current_cs; "Services"; "Tcpip"; + "Parameters"; "Interfaces"; guid ] in + match Registry.get_node reg path with + | Some node -> Some (node, guid, if_name) + | None -> None + ) interfaces in + + (* Get the fields we are interested in. *) + let interfaces = List.map ( + fun (node, guid, if_name) -> + let values = g#hivex_node_values node in + let values = Array.to_list values in + + (* Convert to list of pairs (key, value). *) + let values = + List.map (fun { G.hivex_value_h = v } -> + String.lowercase_ascii (g#hivex_value_key v), v) + values in + + (* Some of the registry fields are REG_MULTI_SZ (= 7) and we + * only want the first string. + *) + let first_string_of_multi_sz v = + let t = g#hivex_value_type v in + if t <> 7_L then raise Not_found; + let data = g#hivex_value_value v in + let strs = Registry.split_multi_sz data in + if strs = [] then raise Not_found; + let str = List.hd strs in + Registry.decode_utf16le str + in + + let if_default_gateway = + try first_string_of_multi_sz (List.assoc "defaultgateway" values) + with Not_found -> "" in + let if_ip_address = + try first_string_of_multi_sz (List.assoc "ipaddress" values) + with Not_found -> "" in + let if_enable_dhcp = + try + let v = List.assoc "enabledhcp" values in + int_of_le32 (g#hivex_value_value v) <> 0_L + with Not_found -> false in + let if_subnet_mask = + try first_string_of_multi_sz (List.assoc "subnetmask" values) + with Not_found -> "" in + + { if_name; if_default_gateway; if_ip_address; if_enable_dhcp; + if_subnet_mask } + ) interfaces in + + interfaces + ) + ) + (* If some inspection fields are "unknown", then that indicates a * failure in inspection, and we shouldn't continue. For an example * of this, see RHBZ#1278371. However don't "assert" here, since diff --git a/v2v/types.ml b/v2v/types.ml index 5c4f3c8ec..d94fad3cc 100644 --- a/v2v/types.ml +++ b/v2v/types.ml @@ -361,14 +361,23 @@ type inspect = { i_mountpoints : (string * string) list; i_apps : Guestfs.application2 list; i_apps_map : Guestfs.application2 list StringMap.t; - i_firmware : i_firmware; i_windows_systemroot : string; i_windows_software_hive : string; i_windows_system_hive : string; i_windows_current_control_set : string; + i_firmware : i_firmware; + i_interfaces : i_interface list; } -let string_of_inspect inspect = +and i_interface = { + if_name : string; + if_default_gateway : string; + if_ip_address : string; + if_enable_dhcp : bool; + if_subnet_mask : string; +} + +let rec string_of_inspect inspect = sprintf "\ i_root = %s i_type = %s @@ -381,11 +390,12 @@ i_package_format = %s i_package_management = %s i_product_name = %s i_product_variant = %s -i_firmware = %s i_windows_systemroot = %s i_windows_software_hive = %s i_windows_system_hive = %s i_windows_current_control_set = %s +i_firmware = %s +i_interfaces = [%s] " inspect.i_root inspect.i_type inspect.i_distro @@ -397,13 +407,23 @@ i_windows_current_control_set = %s inspect.i_package_management inspect.i_product_name inspect.i_product_variant - (match inspect.i_firmware with - | I_BIOS -> "BIOS" - | I_UEFI devices -> sprintf "UEFI [%s]" (String.concat ", " devices)) inspect.i_windows_systemroot inspect.i_windows_software_hive inspect.i_windows_system_hive inspect.i_windows_current_control_set + (match inspect.i_firmware with + | I_BIOS -> "BIOS" + | I_UEFI devices -> sprintf "UEFI [%s]" (String.concat ", " devices)) + (string_of_inspect_interfaces inspect.i_interfaces) + +and string_of_inspect_interfaces ifs = + String.concat ", " (List.map string_of_inspect_interface ifs) + +and string_of_inspect_interface { if_name; if_default_gateway; + if_ip_address; if_enable_dhcp; + if_subnet_mask } = + sprintf "{if_name=%s, if_default_gateway=%s, if_ip_address=%s, if_enable_dhcp=%b, if_subnet_mask=%s}" + if_name if_default_gateway if_ip_address if_enable_dhcp if_subnet_mask type guestcaps = { gcaps_block_bus : guestcaps_block_type; diff --git a/v2v/types.mli b/v2v/types.mli index 6f7a0b5d2..d6ff1b061 100644 --- a/v2v/types.mli +++ b/v2v/types.mli @@ -325,7 +325,9 @@ val string_of_target_buses : target_buses -> string type inspect = { i_root : string; (** Root device. *) - i_type : string; (** Usual inspection fields. *) + + (** Usual guestfs inspection fields. *) + i_type : string; i_distro : string; i_osinfo : string; i_arch : string; @@ -341,16 +343,29 @@ type inspect = { (** This is a map from the app name to the application object. Since RPM allows multiple packages with the same name to be installed, the value is a list. *) - i_firmware : i_firmware; - (** The list of EFI system partitions for the guest with UEFI, - otherwise the BIOS identifier. *) + i_windows_systemroot : string; i_windows_software_hive : string; i_windows_system_hive : string; i_windows_current_control_set : string; + + (** The following fields are the result of additional guest + inspection done by virt-v2v. *) + i_firmware : i_firmware; (** The list of EFI system partitions for the + guest with UEFI, otherwise [BIOS]. *) + i_interfaces : i_interface list; (** List of network interfaces, + currently Windows only. *) } (** Inspection information. *) +and i_interface = { + if_name : string; + if_default_gateway : string; (* "" = no data *) + if_ip_address : string; (* "" = no data *) + if_enable_dhcp : bool; + if_subnet_mask : string; (* "" = no data *) +} + val string_of_inspect : inspect -> string (** {2 Command line parameters} *) -- 2.19.0.rc0
For Linux the guest itself remembers the IP address associated with each MAC address. Thus it doesn't matter if the interface type changes (ie. to virtio-net), because as long as we preserve the MAC address the guest will use the same IP address or the same DHCP configuration. However on Windows this association is not maintained by MAC address. In fact the MAC address isn't saved anywhere in the guest registry. (It seems instead this is likely done through PCI device type and address which we don't record at the moment and is almost impossible to preserve.) When a guest which doesn't use DHCP is migrated, the guest sees the brand new virtio-net devices and doesn't know what to do with them, and meanwhile the right static IPs are still associated with the old and now-defunct interfaces in the registry. This commit is an interim fix which, in limited situations described below, copies the static IP address of the old interface to the new virtio-net (netkvm.sys) interface. The installation of the old IP address on the new interface is done using a Powershell script. - Only works for IPv4 addresses. - Likely only works properly if the guest has a single physical interface. - Only works if the guest is getting virtio drivers. - Only works for Windows >= 7 (because of the Powershell dependency). A longer term fix for this will likely involve trying to decode the PCI address information (assuming my guess above is even correct) and associate that with source hypervisor devices, get the same information as now from the registry, and use a similar technique as this to copy to the new interface, but do it based on target MAC address. Thanks: Brett Thurber for diagnosing the problem and suggesting paths towards a fix. --- common/mlstdutils/std_utils.ml | 7 +++ common/mlstdutils/std_utils.mli | 5 ++ v2v/convert_windows.ml | 92 +++++++++++++++++++++++++++++++++ 3 files changed, 104 insertions(+) diff --git a/common/mlstdutils/std_utils.ml b/common/mlstdutils/std_utils.ml index df443058f..e273faae4 100644 --- a/common/mlstdutils/std_utils.ml +++ b/common/mlstdutils/std_utils.ml @@ -836,3 +836,10 @@ let read_first_line_from_file filename = let is_regular_file path = (* NB: follows symlinks. *) try (Unix.stat path).Unix.st_kind = Unix.S_REG with Unix.Unix_error _ -> false + +let rec ipv4_prefix_length mask = + if mask = 0_l then 0 + else if mask >= 0x8000_0000_l (* top bit set *) then + 1 + ipv4_prefix_length (Int32.shift_left mask 1) + else (* mask is invalid if top bit is not set but other bits are set *) + invalid_arg "ipv4_prefix_length" diff --git a/common/mlstdutils/std_utils.mli b/common/mlstdutils/std_utils.mli index 62cb8e9ff..aea0ec4d4 100644 --- a/common/mlstdutils/std_utils.mli +++ b/common/mlstdutils/std_utils.mli @@ -447,3 +447,8 @@ val read_first_line_from_file : string -> string val is_regular_file : string -> bool (** Checks whether the file is a regular file. *) + +val ipv4_prefix_length : int32 -> int +(** Calculate the prefix length of the given IPv4 network mask + (given in host byte order). Raises [Invalid_argument _] if + not a network mask. *) diff --git a/v2v/convert_windows.ml b/v2v/convert_windows.ml index 2d2b6adfe..e0b889a06 100644 --- a/v2v/convert_windows.ml +++ b/v2v/convert_windows.ml @@ -38,10 +38,23 @@ module G = Guestfs * time the Windows VM is booted on KVM. *) +let ipv4_re = PCRE.compile "^(\\d+)\\.(\\d+)\\.(\\d+)\\.(\\d+)$" + let convert (g : G.guestfs) inspect source output rcaps = (*----------------------------------------------------------------------*) (* Inspect the Windows guest. *) + (* All physical network interfaces. *) + let all_physical_interfaces = + List.filter (fun { if_name } -> String.is_prefix if_name "Ethernet") + inspect.i_interfaces in + + (* All physical network interfaces have static IP addresses? *) + let all_physical_interfaces_are_static = + all_physical_interfaces <> [] && + List.for_all (fun { if_enable_dhcp } -> if_enable_dhcp = false) + all_physical_interfaces in + (* If the Windows guest appears to be using group policy. *) let has_group_policy = Registry.with_hive_readonly g inspect.i_windows_software_hive @@ -220,6 +233,16 @@ let convert (g : G.guestfs) inspect source output rcaps = Registry.with_hive_write g inspect.i_windows_software_hive update_software_hive; + (* If we have only static interfaces then we need to copy the + * IP addresses from the old interfaces to the new virtio-net + * interfaces. Only works for Win7 and above. (RHBZ#1626503) + *) + if net_driver = Virtio_net && + all_physical_interfaces_are_static && + (inspect.i_major_version, inspect.i_minor_version) >= (6, 1) + then + configure_ip_address_transfer_at_firstboot (); + fix_ntfs_heads (); fix_win_esp (); @@ -595,6 +618,75 @@ if errorlevel 3010 exit /b 0 | None -> warning (f_"could not find registry key HKLM\\SOFTWARE\\Microsoft\\Windows\\CurrentVersion") + and configure_ip_address_transfer_at_firstboot () = + (* For guests which don't use DHCP we will need to copy network + * information from the first of the old interfaces to the + * first of the virtio-net interfaces. Unfortunately Windows + * doesn't save information about MAC address so we have no way + * to associate which original interface corresponds to which + * target interface. XXX (RHBZ#1626503) + *) + + assert (all_physical_interfaces <> []); (* checked by caller *) + let { if_default_gateway; if_ip_address; if_subnet_mask } = + List.hd all_physical_interfaces in + + (* Parameters of New-NetIPAddress. + * https://docs.microsoft.com/en-us/powershell/module/nettcpip/new-netipaddress... + * Only works for IPv4 right now. We need to find an example + * of a Windows guest using IPv6 to see what the registry + * contains. XXX + *) + let params = ref "" in + if if_ip_address <> "" then + params := !params ^ sprintf " -IPAddress %s" if_ip_address; + if if_default_gateway <> "" then + params := !params ^ sprintf " -DefaultGateway %s" if_default_gateway; + if PCRE.matches ipv4_re if_subnet_mask then ( + let a = Int32.of_string (PCRE.sub 0) + and b = Int32.of_string (PCRE.sub 1) + and c = Int32.of_string (PCRE.sub 2) + and d = Int32.of_string (PCRE.sub 3) in + (* Can we calculate a mask? *) + if a >= 0_l && a <= 255_l && b >= 0_l && b <= 255_l && + c >= 0_l && c <= 255_l && d >= 0_l && d <= 255_l then ( + let a = Int32.shift_left a 24 + and b = Int32.shift_left b 16 + and c = Int32.shift_left c 8 in + let mask = Int32.logor (Int32.logor (Int32.logor a b) c) d in + try + let prefix_length = ipv4_prefix_length mask in + params := !params ^ sprintf " -PrefixLength %d" prefix_length + with Invalid_argument _ -> () + ) + ); + let params = !params in + + let tempdir = sprintf "%s/Temp" inspect.i_windows_systemroot in + let psh_filename = "v2v-ip-address-transfer.ps1" in + let psh = sprintf "\ +# Wait for the netkvm (virtio-net) driver to become active. +$adapter = \"\" +While (-Not $adapter) { + Start-Sleep -Seconds 5 + $adapter = (Get-NetAdapter | Where DriverFileName -eq \"netkvm.sys\").InterfaceAlias +} +New-NetIPAddress -InterfaceAlias $adapter%s +" params in + g#mkdir_p tempdir; + g#write psh (tempdir ^ "/" ^ psh_filename); + + (* Unfortunately Powershell scripts cannot be directly executed + * (unless some system config changes are made which for other + * reasons we don't want to do) and so we have to run this via + * a regular batch file. + *) + let fb = + sprintf "%s\\System32\\WindowsPowerShell\\v1.0\\powershell.exe -ExecutionPolicy ByPass -file C:%s\\%s" + inspect.i_windows_systemroot + (String.replace_char tempdir '/' '\\') psh_filename in + Firstboot.add_firstboot_script g inspect.i_root "ip address transfer" fb + and fix_ntfs_heads () = (* NTFS hardcodes the number of heads on the drive which created it in the filesystem header. Modern versions of Windows -- 2.19.0.rc0
On Tue, Dec 04, 2018 at 05:29:31PM +0000, Richard W.M. Jones wrote:
This patch is just for discussion. There are still a couple of issues that I'm trying to fix.
Sorry if I missed the original discussion on that, but why is copying the static IP configuraition even considered to be a good thing? For one, it's unlikely that the IP address will remain valid in the target environment. For two, in a typical cloud setup static IP addresses are configured by some sort of a cloud-specific management agent, and bypassing it is likely to cause conflicts. For three, network configuration is a notoriously complex thing. I think I can recall a dozen of network configuration systems in Linux that claimed to provide the ultimate solution to all problems and none that delivered to the promise. I guess in Windows the situation is hardly much simpler. Creating yet another universal solution doesn't appear to be in scope for v2v (if realistic at all); and I'm not sure that having a very limited solution for a very basic use case only is better than explicitly telling the user not to rely on v2v for network configuration. Roman.
On Wed, Dec 05, 2018 at 09:01:16AM +0000, Roman Kagan wrote:
On Tue, Dec 04, 2018 at 05:29:31PM +0000, Richard W.M. Jones wrote:
This patch is just for discussion. There are still a couple of issues that I'm trying to fix.
Sorry if I missed the original discussion on that, but why is copying the static IP configuraition even considered to be a good thing?
For one, it's unlikely that the IP address will remain valid in the target environment.
For our v2v conversions (generally from VMware to oVirt or OpenStack) we're trying to maintain a near-identical network configuration on both sides. So for example a guest will have the same MAC addresses before and after and, if using DHCP, will pick up the exact same IP address (probably served by the same DHCP server) and be able to access the same set of services etc. Also we're doing like 1000s of VMs in one go, so manual reconfiguration of anything is a non-starter. However let me describe the problem we're seeing, as it is a bit more general and might even apply in some cloud scenarios. If you have a Windows guest which starts out with a static IP address (no DHCP) it might be configured like this on VMware: 00:50:56:01:02:03 192.168.128.10 vmxnet3 "Ethernet0" EnableDHCP=0 After conversion we preserve the MAC address and add virtio drivers, but the new netkvm.sys driver just adds a new network interface so it'll end up like this: (no hardware) 192.168.128.10 vmxnet3 "Ethernet0" EnableDHCP=0 00:50:56:01:02:03 (no address) netkvm "Ethernet1" EnableDHCP=1 It has no address in the second case because this is a non-DHCP network. (The DHCP situation is better because the second network will be assigned the right IP address because of the preserved MAC address, although it still has the unnecessary network adapter.) Note that Windows < 10 doesn't have a concept of persistent network names like we do in Linux which mostly solves this problem. (Windows 10 on the other hand does save the MAC address in the Registry, but I haven't fully investigated yet what it's doing.) Anyway if you have another suggestion how to solve this, I'm all ears ... For other areas we'd fix things up using an Ansible post-copy script, but in this case we can't do that since Ansible cannot reconfigure a guest that has no working network.
For two, in a typical cloud setup static IP addresses are configured by some sort of a cloud-specific management agent, and bypassing it is likely to cause conflicts.
For three, network configuration is a notoriously complex thing. I think I can recall a dozen of network configuration systems in Linux that claimed to provide the ultimate solution to all problems and none that delivered to the promise. I guess in Windows the situation is hardly much simpler. Creating yet another universal solution doesn't appear to be in scope for v2v (if realistic at all); and I'm not sure that having a very limited solution for a very basic use case only is better than explicitly telling the user not to rely on v2v for network configuration.
Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v
On Wed, Dec 5, 2018 at 12:01 PM Richard W.M. Jones <rjones@redhat.com> wrote:
On Wed, Dec 05, 2018 at 09:01:16AM +0000, Roman Kagan wrote:
On Tue, Dec 04, 2018 at 05:29:31PM +0000, Richard W.M. Jones wrote:
This patch is just for discussion. There are still a couple of issues that I'm trying to fix.
Sorry if I missed the original discussion on that, but why is copying the static IP configuraition even considered to be a good thing?
For one, it's unlikely that the IP address will remain valid in the target environment.
For our v2v conversions (generally from VMware to oVirt or OpenStack) we're trying to maintain a near-identical network configuration on both sides. So for example a guest will have the same MAC addresses before and after and, if using DHCP, will pick up the exact same IP address (probably served by the same DHCP server) and be able to access the same set of services etc.
Also we're doing like 1000s of VMs in one go, so manual reconfiguration of anything is a non-starter.
However let me describe the problem we're seeing, as it is a bit more general and might even apply in some cloud scenarios. If you have a Windows guest which starts out with a static IP address (no DHCP) it might be configured like this on VMware:
00:50:56:01:02:03 192.168.128.10 vmxnet3 "Ethernet0" EnableDHCP=0
After conversion we preserve the MAC address and add virtio drivers, but the new netkvm.sys driver just adds a new network interface so it'll end up like this:
(no hardware) 192.168.128.10 vmxnet3 "Ethernet0" EnableDHCP=0 00:50:56:01:02:03 (no address) netkvm "Ethernet1" EnableDHCP=1
It has no address in the second case because this is a non-DHCP network. (The DHCP situation is better because the second network will be assigned the right IP address because of the preserved MAC address, although it still has the unnecessary network adapter.)
Note that Windows < 10 doesn't have a concept of persistent network names like we do in Linux which mostly solves this problem. (Windows 10 on the other hand does save the MAC address in the Registry, but I haven't fully investigated yet what it's doing.)
Anyway if you have another suggestion how to solve this, I'm all ears ... For other areas we'd fix things up using an Ansible post-copy script, but in this case we can't do that since Ansible cannot reconfigure a guest that has no working network.
As you mention, using Ansible means that the IP address of the VM is correctly set. In another discussion, you mentioned using transient DHCP, but would mean that we reconfigure one of the NICs before conversion to use DHCP. That would alter the original VM. And in the case of multiple NICs, how do we know which one to reconfigure. That would be quite specific to each customer, meaning customized playbook / role. And IIRC, there's a good chance that it breaks WinRM configuration with regards to certificates, meaning reconfiguring WinRM after DHCP and after conversion. But maybe I'm too pessimistic ;)
For two, in a typical cloud setup static IP addresses are configured by some sort of a cloud-specific management agent, and bypassing it is likely to cause conflicts.
In the case of OpenStack, we keep the MAC address by creating the network ports with it and the IP address pre-allocated. This allows neutron to have working security groups. As Richard says, we're looking at the case of a massive migration where adapting the VM is out of scope. It's a mainly a 1:1 relationship. Anyway, we're still looking at more complex scenarios with tenant networks and floating IPs. It promises to be fun :)
For three, network configuration is a notoriously complex thing. I think I can recall a dozen of network configuration systems in Linux that claimed to provide the ultimate solution to all problems and none that delivered to the promise. I guess in Windows the situation is hardly much simpler. Creating yet another universal solution doesn't appear to be in scope for v2v (if realistic at all); and I'm not sure that having a very limited solution for a very basic use case only is better than explicitly telling the user not to rely on v2v for network configuration.
Rich.
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v
_______________________________________________ Libguestfs mailing list Libguestfs@redhat.com https://www.redhat.com/mailman/listinfo/libguestfs
-- *Fabien Dupont* PRINCIPAL SOFTWARE ENGINEER Red Hat - Solutions Engineering fabien@redhat.com M: +33 (0) 662 784 971 <+33662784971> <http://redhat.com> *TRIED. TESTED. TRUSTED.* Twitter: @redhatway <https://twitter.com/redhatway> | Instagram: @redhatinc <https://www.instagram.com/redhatinc/> | Snapchat: @redhatsnaps
On Wed, Dec 05, 2018 at 01:18:49PM +0100, Fabien Dupont wrote:
On Wed, Dec 5, 2018 at 12:01 PM Richard W.M. Jones <rjones@redhat.com> wrote:
On Wed, Dec 05, 2018 at 09:01:16AM +0000, Roman Kagan wrote:
On Tue, Dec 04, 2018 at 05:29:31PM +0000, Richard W.M. Jones wrote:
This patch is just for discussion. There are still a couple of issues that I'm trying to fix.
Sorry if I missed the original discussion on that, but why is copying the static IP configuraition even considered to be a good thing?
For one, it's unlikely that the IP address will remain valid in the target environment.
For our v2v conversions (generally from VMware to oVirt or OpenStack) we're trying to maintain a near-identical network configuration on both sides. So for example a guest will have the same MAC addresses before and after and, if using DHCP, will pick up the exact same IP address (probably served by the same DHCP server) and be able to access the same set of services etc.
Also we're doing like 1000s of VMs in one go, so manual reconfiguration of anything is a non-starter.
However let me describe the problem we're seeing, as it is a bit more general and might even apply in some cloud scenarios. If you have a Windows guest which starts out with a static IP address (no DHCP) it might be configured like this on VMware:
00:50:56:01:02:03 192.168.128.10 vmxnet3 "Ethernet0" EnableDHCP=0
After conversion we preserve the MAC address and add virtio drivers, but the new netkvm.sys driver just adds a new network interface so it'll end up like this:
(no hardware) 192.168.128.10 vmxnet3 "Ethernet0" EnableDHCP=0 00:50:56:01:02:03 (no address) netkvm "Ethernet1" EnableDHCP=1
It has no address in the second case because this is a non-DHCP network. (The DHCP situation is better because the second network will be assigned the right IP address because of the preserved MAC address, although it still has the unnecessary network adapter.)
Note that Windows < 10 doesn't have a concept of persistent network names like we do in Linux which mostly solves this problem. (Windows 10 on the other hand does save the MAC address in the Registry, but I haven't fully investigated yet what it's doing.)
Anyway if you have another suggestion how to solve this, I'm all ears ... For other areas we'd fix things up using an Ansible post-copy script, but in this case we can't do that since Ansible cannot reconfigure a guest that has no working network.
As you mention, using Ansible means that the IP address of the VM is correctly set. In another discussion, you mentioned using transient DHCP, but would mean that we reconfigure one of the NICs before conversion to use DHCP. That would alter the original VM.
No I don't think so. The newly added netkvm NIC seems to come up with EnableDHCP=1 (although NB I did not test in the end to end case whether this means the Windows guest will get an address if a DHCP server is available).
And in the case of multiple NICs, how do we know which one to reconfigure.
The current patch has the same problem. In the multiple NIC case for Windows < 10 we don't know how the NICs were assigned to guest hardware on the source and it seems hard to know without looking at PCI address assignments which opens up a whole can of worms. In Windows >= 10 it looks as if Windows is finally storing the hardware MAC in the registry associated with each interface, so maybe this problem will disappear in future.
That would be quite specific to each customer, meaning customized playbook / role.
This is correct.
And IIRC, there's a good chance that it breaks WinRM configuration with regards to certificates, meaning reconfiguring WinRM after DHCP and after conversion.
What is “WinRM configuration”?
But maybe I'm too pessimistic ;)
For two, in a typical cloud setup static IP addresses are configured by some sort of a cloud-specific management agent, and bypassing it is likely to cause conflicts.
In the case of OpenStack, we keep the MAC address by creating the network ports with it and the IP address pre-allocated. This allows neutron to have working security groups. As Richard says, we're looking at the case of a massive migration where adapting the VM is out of scope. It's a mainly a 1:1 relationship. Anyway, we're still looking at more complex scenarios with tenant networks and floating IPs. It promises to be fun :)
For three, network configuration is a notoriously complex thing. I think I can recall a dozen of network configuration systems in Linux that claimed to provide the ultimate solution to all problems and none that delivered to the promise. I guess in Windows the situation is hardly much simpler. Creating yet another universal solution doesn't appear to be in scope for v2v (if realistic at all); and I'm not sure that having a very limited solution for a very basic use case only is better than explicitly telling the user not to rely on v2v for network configuration.
Rich.
Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top
On Wed, Dec 5, 2018 at 2:50 PM Richard W.M. Jones <rjones@redhat.com> wrote:
On Wed, Dec 05, 2018 at 01:18:49PM +0100, Fabien Dupont wrote:
On Wed, Dec 5, 2018 at 12:01 PM Richard W.M. Jones <rjones@redhat.com> wrote:
On Wed, Dec 05, 2018 at 09:01:16AM +0000, Roman Kagan wrote:
On Tue, Dec 04, 2018 at 05:29:31PM +0000, Richard W.M. Jones wrote:
This patch is just for discussion. There are still a couple of issues that I'm trying to fix.
Sorry if I missed the original discussion on that, but why is copying the static IP configuraition even considered to be a good thing?
For one, it's unlikely that the IP address will remain valid in the target environment.
For our v2v conversions (generally from VMware to oVirt or OpenStack) we're trying to maintain a near-identical network configuration on both sides. So for example a guest will have the same MAC addresses before and after and, if using DHCP, will pick up the exact same IP address (probably served by the same DHCP server) and be able to access the same set of services etc.
Also we're doing like 1000s of VMs in one go, so manual reconfiguration of anything is a non-starter.
However let me describe the problem we're seeing, as it is a bit more general and might even apply in some cloud scenarios. If you have a Windows guest which starts out with a static IP address (no DHCP) it might be configured like this on VMware:
00:50:56:01:02:03 192.168.128.10 vmxnet3 "Ethernet0" EnableDHCP=0
After conversion we preserve the MAC address and add virtio drivers, but the new netkvm.sys driver just adds a new network interface so it'll end up like this:
(no hardware) 192.168.128.10 vmxnet3 "Ethernet0" EnableDHCP=0 00:50:56:01:02:03 (no address) netkvm "Ethernet1" EnableDHCP=1
It has no address in the second case because this is a non-DHCP network. (The DHCP situation is better because the second network will be assigned the right IP address because of the preserved MAC address, although it still has the unnecessary network adapter.)
Note that Windows < 10 doesn't have a concept of persistent network names like we do in Linux which mostly solves this problem. (Windows 10 on the other hand does save the MAC address in the Registry, but I haven't fully investigated yet what it's doing.)
Anyway if you have another suggestion how to solve this, I'm all ears ... For other areas we'd fix things up using an Ansible post-copy script, but in this case we can't do that since Ansible cannot reconfigure a guest that has no working network.
As you mention, using Ansible means that the IP address of the VM is correctly set. In another discussion, you mentioned using transient DHCP, but would mean that we reconfigure one of the NICs before conversion to use DHCP. That would alter the original VM.
No I don't think so. The newly added netkvm NIC seems to come up with EnableDHCP=1 (although NB I did not test in the end to end case whether this means the Windows guest will get an address if a DHCP server is available).
And in the case of multiple NICs, how do we know which one to reconfigure.
The current patch has the same problem. In the multiple NIC case for Windows < 10 we don't know how the NICs were assigned to guest hardware on the source and it seems hard to know without looking at PCI address assignments which opens up a whole can of worms.
In Windows >= 10 it looks as if Windows is finally storing the hardware MAC in the registry associated with each interface, so maybe this problem will disappear in future.
That would be quite specific to each customer, meaning customized playbook / role.
This is correct.
And IIRC, there's a good chance that it breaks WinRM configuration with regards to certificates, meaning reconfiguring WinRM after DHCP and after conversion.
What is “WinRM configuration”?
I have limited experience with WinRM, but AFAIK it works with SSL certs and they are generated when service is first configured. So, I suspect that if we change the IP address the certificate might become invalid. To be confirmed.
But maybe I'm too pessimistic ;)
For two, in a typical cloud setup static IP addresses are configured
by
some sort of a cloud-specific management agent, and bypassing it is likely to cause conflicts.
In the case of OpenStack, we keep the MAC address by creating the network ports with it and the IP address pre-allocated. This allows neutron to have working security groups. As Richard says, we're looking at the case of a massive migration where adapting the VM is out of scope. It's a mainly a 1:1 relationship. Anyway, we're still looking at more complex scenarios with tenant networks and floating IPs. It promises to be fun :)
For three, network configuration is a notoriously complex thing. I think I can recall a dozen of network configuration systems in Linux that claimed to provide the ultimate solution to all problems and none that delivered to the promise. I guess in Windows the situation is hardly much simpler. Creating yet another universal solution doesn't appear to be in scope for v2v (if realistic at all); and I'm not sure that having a very limited solution for a very basic use case only is better than explicitly telling the user not to rely on v2v for network configuration.
Rich.
Rich.
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top
-- *Fabien Dupont* PRINCIPAL SOFTWARE ENGINEER Red Hat - Solutions Engineering fabien@redhat.com M: +33 (0) 662 784 971 <+33662784971> <http://redhat.com> *TRIED. TESTED. TRUSTED.* Twitter: @redhatway <https://twitter.com/redhatway> | Instagram: @redhatinc <https://www.instagram.com/redhatinc/> | Snapchat: @redhatsnaps
On Wed, Dec 05, 2018 at 11:01:43AM +0000, Richard W.M. Jones wrote:
On Wed, Dec 05, 2018 at 09:01:16AM +0000, Roman Kagan wrote:
On Tue, Dec 04, 2018 at 05:29:31PM +0000, Richard W.M. Jones wrote:
This patch is just for discussion. There are still a couple of issues that I'm trying to fix.
Sorry if I missed the original discussion on that, but why is copying the static IP configuraition even considered to be a good thing?
For one, it's unlikely that the IP address will remain valid in the target environment.
For our v2v conversions (generally from VMware to oVirt or OpenStack) we're trying to maintain a near-identical network configuration on both sides. So for example a guest will have the same MAC addresses before and after and, if using DHCP, will pick up the exact same IP address (probably served by the same DHCP server) and be able to access the same set of services etc.
This doesn't look like an assumption that can be made in general.
However let me describe the problem we're seeing, as it is a bit more general and might even apply in some cloud scenarios. If you have a Windows guest which starts out with a static IP address (no DHCP) it might be configured like this on VMware:
00:50:56:01:02:03 192.168.128.10 vmxnet3 "Ethernet0" EnableDHCP=0
After conversion we preserve the MAC address and add virtio drivers, but the new netkvm.sys driver just adds a new network interface so it'll end up like this:
(no hardware) 192.168.128.10 vmxnet3 "Ethernet0" EnableDHCP=0 00:50:56:01:02:03 (no address) netkvm "Ethernet1" EnableDHCP=1
It has no address in the second case because this is a non-DHCP network. (The DHCP situation is better because the second network will be assigned the right IP address because of the preserved MAC address, although it still has the unnecessary network adapter.)
... unless the dhcp client uses unique client-id for each device...
Note that Windows < 10 doesn't have a concept of persistent network names like we do in Linux which mostly solves this problem. (Windows 10 on the other hand does save the MAC address in the Registry, but I haven't fully investigated yet what it's doing.)
Anyway if you have another suggestion how to solve this, I'm all ears ... For other areas we'd fix things up using an Ansible post-copy script, but in this case we can't do that since Ansible cannot reconfigure a guest that has no working network.
No I don't have a general solution to this. But ...
For two, in a typical cloud setup static IP addresses are configured by some sort of a cloud-specific management agent, and bypassing it is likely to cause conflicts.
In particular, in Virtuozzo the guests are often configured to get their IP configuration from the management layer via a guest agent. This is of course limited in the variety of net configurations that can be created in the guest (IIRC only static and dhcp are supported), and also allows to apply IP filtering on the hypervisor side. Then, if desired, on v2v the configuration is obtained from the source hypervisor (Virtuozzo 6 in our case) and applied on the target (QEMU/KVM-based Virtuozzo 7). IOW in this case the network configuration is *owned* by the management/orchestration layer, not the guest, and it would be a mistake to ignore that. Dunno if it can be applied in other scenarios, though. Thanks, Roman.
On Wed, Dec 05, 2018 at 12:52:21PM +0000, Roman Kagan wrote:
On Wed, Dec 05, 2018 at 11:01:43AM +0000, Richard W.M. Jones wrote:
On Wed, Dec 05, 2018 at 09:01:16AM +0000, Roman Kagan wrote:
On Tue, Dec 04, 2018 at 05:29:31PM +0000, Richard W.M. Jones wrote:
This patch is just for discussion. There are still a couple of issues that I'm trying to fix.
Sorry if I missed the original discussion on that, but why is copying the static IP configuraition even considered to be a good thing?
For one, it's unlikely that the IP address will remain valid in the target environment.
For our v2v conversions (generally from VMware to oVirt or OpenStack) we're trying to maintain a near-identical network configuration on both sides. So for example a guest will have the same MAC addresses before and after and, if using DHCP, will pick up the exact same IP address (probably served by the same DHCP server) and be able to access the same set of services etc.
This doesn't look like an assumption that can be made in general.
I agree this is a somewhat specialized scenario, but our interest in virt-v2v is liberating VMs off VMware on to our open source products and the idea is that you get a drop-in replacement without major changes to how VMs conceptually run. IOW the result looks more like vSphere and not much like a public cloud.
However let me describe the problem we're seeing, as it is a bit more general and might even apply in some cloud scenarios. If you have a Windows guest which starts out with a static IP address (no DHCP) it might be configured like this on VMware:
00:50:56:01:02:03 192.168.128.10 vmxnet3 "Ethernet0" EnableDHCP=0
After conversion we preserve the MAC address and add virtio drivers, but the new netkvm.sys driver just adds a new network interface so it'll end up like this:
(no hardware) 192.168.128.10 vmxnet3 "Ethernet0" EnableDHCP=0 00:50:56:01:02:03 (no address) netkvm "Ethernet1" EnableDHCP=1
It has no address in the second case because this is a non-DHCP network. (The DHCP situation is better because the second network will be assigned the right IP address because of the preserved MAC address, although it still has the unnecessary network adapter.)
... unless the dhcp client uses unique client-id for each device...
This seems a niche configuration though. I've only used Linux DHCP clients in anger and client ID would be specified by machine (although looking at the docs, I see that you _can_ set it per interface if you're that crazy :-))
Note that Windows < 10 doesn't have a concept of persistent network names like we do in Linux which mostly solves this problem. (Windows 10 on the other hand does save the MAC address in the Registry, but I haven't fully investigated yet what it's doing.)
Anyway if you have another suggestion how to solve this, I'm all ears ... For other areas we'd fix things up using an Ansible post-copy script, but in this case we can't do that since Ansible cannot reconfigure a guest that has no working network.
No I don't have a general solution to this. But ...
For two, in a typical cloud setup static IP addresses are configured by some sort of a cloud-specific management agent, and bypassing it is likely to cause conflicts.
In particular, in Virtuozzo the guests are often configured to get their IP configuration from the management layer via a guest agent. This is of course limited in the variety of net configurations that can be created in the guest (IIRC only static and dhcp are supported), and also allows to apply IP filtering on the hypervisor side. Then, if desired, on v2v the configuration is obtained from the source hypervisor (Virtuozzo 6 in our case) and applied on the target (QEMU/KVM-based Virtuozzo 7).
IOW in this case the network configuration is *owned* by the management/orchestration layer, not the guest, and it would be a mistake to ignore that.
Dunno if it can be applied in other scenarios, though.
We do have a guest agent running, although only after conversion. Unfortunately without a working network it cannot collect the required network configuration. One possibility then is that we modify the guest agent so it can read out the required IP address from a temporary file and set up the network that way, although that it not very different from the patch I proposed, except using an agent instead of a hacky Powershell script. I'm also now hearing that some customers regard DHCP as insecure. Aarrggh. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v
participants (3)
-
Fabien Dupont -
Richard W.M. Jones -
Roman Kagan