Re: [Libguestfs] Fwd: [PATCH] v2v: virtio-win: include *.dll too
On Tue, 2015-10-27 at 11:00 -0400, Amnon Ilan wrote:
Hi Jeff,
There were some questions on the libguestfs mailing list regarding virtio-win iso an packaging (see below). Could you answer it on the list? (just send it to libguestfs@redhat.com , no need to register, keep us CCed) The people asking are from Virtuozzo, they are not customers or official partners, just good contributors upstream. Note that this mailing list is public.
Thanks, Amnon
----- Forwarded Message -----
From: "Richard W.M. Jones" <rjones@redhat.com> To: "Roman Kagan" <rkagan@virtuozzo.com>, libguestfs@redhat.com, "Denis Lunev" <den@virtuozzo.com> Cc: "Amnon Ilan" <ailan@redhat.com> Sent: Tuesday, October 27, 2015 4:11:10 PM Subject: Re: [PATCH] v2v: virtio-win: include *.dll too
On Tue, Oct 27, 2015 at 04:45:00PM +0300, Roman Kagan wrote:
On Tue, Oct 27, 2015 at 12:02:40PM +0000, Richard W.M. Jones wrote:
On Tue, Oct 27, 2015 at 02:08:42PM +0300, Roman Kagan wrote:
On Tue, Oct 27, 2015 at 09:12:41AM +0000, Richard W.M. Jones wrote:
On Mon, Oct 26, 2015 at 09:00:03PM +0300, Roman Kagan wrote: > Windows QXL drivers include also qxldd.dll which used to get > filtered > out and not copied over into the guest. As a result QXL driver > failed > to install due to a missing file. > (* Skip files without specific extensions. *) > - let extensions = ["cat"; "inf"; "pdb"; "sys"] in > + let extensions = ["cat"; "dll"; "inf"; "pdb"; "sys"] in
Actually no, this part is (may be) wrong.
This causes WdfCoInstaller*.dll from the ISO to be copied in. It's not clear to me whether these files are needed - I suspect not.
Having WdfCoInstaller* is absolutely necessary for successful installation any of our WDF based drivers (serial, balloon, and rng).
Yes it is, see the corresponding *.inf where it's mentioned in the corresponding CopyFiles directive (and yes, I just verified that with that file missing the balloon driver installation fails with 'file not found' error).
Maybe we should be looking at the CopyFiles directive? (As you may be able to guess, I know next to nothing about how Windows drivers work).
So do I, and parsing the *.inf files is the last thing I'd like to do here; I'm not sure it can be made reliable (I mean, not the parsing, but making any sense out of the parsed data).
However I'm now facing a more serious problem: when I actually went ahead and looked into the virio-win iso packaged in the rhel virtio-win rpm, I found out that its contents differed significantly from what was packaged in that very same rpm as a driver directory hierarchy.
Namely,
- there are no qxl drivers on the iso, but they are present in the rpm
- there are no balloon, qemupciserial, viorng, vioserial in the rpm, but they are present on the iso
- qemupciserial is an inf-only 'driver' which matches all windows versions; it's just one copy at the sub-toplevel directory, so it won't pass the virtio_iso_path_matches_guest_os() criteria
- there are a lot of duplicates between files for different windows flavors
It's due to historical reasons mostly. The best way would be having a set of separate distribution images packaged on per-platform base. Vadim.
To sum up, the packaging and naming policy of the virtio-win rpm and the virtio-win iso therein are different and neither is clear. Hardcoding the policy in v2v without actually knowing it appears risky at best.
We need guidance here from someone who knows how that virtio-win stuff is packaged and how different it is across distros. Any idea whom to contact on the matter?
Amnon -- CC'd -- I guess?
I'm aware that there are at least some differences in paths, and the current virt-v2v code should be able to cope for the two drivers that we really care about - viostor and virtio-net. Of course it can't install drivers that don't exist on either the ISO or the rpm.
Rich.
Anyhow, can you fix up the test at least.
Sure, but I'd like first to figure out how to fix up the code ;)
Roman.
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-df lists disk usage of guests without needing to install any software inside the virtual machine. Supports Linux and Windows. http://people.redhat.com/~rjones/virt-df/
On Wed, Oct 28, 2015 at 12:10:19PM +1100, Vadim Rozenfeld wrote:
To sum up, the packaging and naming policy of the virtio-win rpm and the virtio-win iso therein are different and neither is clear. Hardcoding the policy in v2v without actually knowing it appears risky at best.
It's due to historical reasons mostly. The best way would be having a set of separate distribution images packaged on per-platform base.
Let me try to get the libguestfs requirements straight: given a set of Windows drivers, it should be able to identify the ones appropriate for the particular Windows flavor, in order to 1) tell which devices can be configured 2) offline-"install" the storage driver and thus enable the guest too boot 3) copy over the matching drivers into the guest and allow it to pick them up on the first boot Obviously virtio-win driver packaging and libguestfs must agree on how to deal with this. Could you please provide any guidance on how to address this problem? Thanks, Roman.
On 28 Oct 2015, at 13:53, Roman Kagan <rkagan@virtuozzo.com> wrote:
On Wed, Oct 28, 2015 at 12:10:19PM +1100, Vadim Rozenfeld wrote:
To sum up, the packaging and naming policy of the virtio-win rpm and the virtio-win iso therein are different and neither is clear. Hardcoding the policy in v2v without actually knowing it appears risky at best.
It's due to historical reasons mostly. The best way would be having a set of separate distribution images packaged on per-platform base.
Let me try to get the libguestfs requirements straight:
given a set of Windows drivers, it should be able to identify the ones appropriate for the particular Windows flavor, in order to
1) tell which devices can be configured
2) offline-"install" the storage driver and thus enable the guest too boot
3) copy over the matching drivers into the guest and allow it to pick them up on the first boot
Obviously virtio-win driver packaging and libguestfs must agree on how to deal with this.
Could you please provide any guidance on how to address this problem?
As Vadim said: "The best way would be having a set of separate distribution images packaged on per-platform base.” Otherwise you will have to maintain the knowledge of binary compatibility between Windows platforms, which is different according to the driver type.
Thanks, Roman.
On Wed, Oct 28, 2015 at 04:15:06PM +0200, Yan Vugenfirer wrote:
Otherwise you will have to maintain the knowledge of binary compatibility between Windows platforms, which is different according to the driver type.
What does this mean? Could you give an example? Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW
On Wed, Oct 28, 2015 at 04:15:06PM +0200, Yan Vugenfirer wrote:
On 28 Oct 2015, at 13:53, Roman Kagan <rkagan@virtuozzo.com> wrote:
On Wed, Oct 28, 2015 at 12:10:19PM +1100, Vadim Rozenfeld wrote:
To sum up, the packaging and naming policy of the virtio-win rpm and the virtio-win iso therein are different and neither is clear. Hardcoding the policy in v2v without actually knowing it appears risky at best.
It's due to historical reasons mostly. The best way would be having a set of separate distribution images packaged on per-platform base.
Let me try to get the libguestfs requirements straight:
given a set of Windows drivers, it should be able to identify the ones appropriate for the particular Windows flavor, in order to
1) tell which devices can be configured
2) offline-"install" the storage driver and thus enable the guest too boot
3) copy over the matching drivers into the guest and allow it to pick them up on the first boot
Obviously virtio-win driver packaging and libguestfs must agree on how to deal with this.
Could you please provide any guidance on how to address this problem?
As Vadim said: "The best way would be having a set of separate distribution images packaged on per-platform base.”
Great, but apparently this style of packaging is supposed to happen on virtio-win side of things, and libguestfs would just need to adapt. Presumably there supposed to be a separate iso or floppy image containing all the drivers for each Windows flavor, and some stable naming convention for such images making it easy to map the guest OS to the image. Is there any work on that being done or planned?
Otherwise you will have to maintain the knowledge of binary compatibility between Windows platforms, which is different according to the driver type.
Indeed, this doesn't look realistic within libguestfs. Thanks, Roman.
Hi, I am the guy who puts together the virtio-win package for Red Hat. I recently took over this role so I'm still in learning mode. On Wed, Oct 28, 2015 at 04:15:06PM +0200, Yan Vugenfirer wrote:
On 28 Oct 2015, at 13:53, Roman Kagan <rkagan@virtuozzo.com> wrote: On Wed, Oct 28, 2015 at 12:10:19PM +1100, Vadim Rozenfeld wrote:
To sum up, the packaging and naming policy of the virtio-win rpm and the virtio-win iso therein are different and neither is clear. Hardcoding the policy in v2v without actually knowing it appears risky at best.
It's due to historical reasons mostly. The best way would be having a set of separate distribution images packaged on per-platform base.
Today, the virtio-win rpm contains: * Individual driver files, installed onto the host contained in the directory hierarchy: /usr/share/virtio-win/drivers/<arch>/<os>/<file> * an .iso file which is intended to be used within a Windows guest. The .iso directory hierarchy is: /<driver>/<os>/<arch>/<file> * two vfd files--one for amd64 (x86_64), the other for x86 (i386)-- that are floppy images, intended to be used as a Windows guest install-time alternative to the .iso. The vfd files contain essential drivers needed during installation: (block, scsi, net and qxl drivers).
Let me try to get the libguestfs requirements straight:
given a set of Windows drivers, it should be able to identify the ones appropriate for the particular Windows flavor, in order to
1) tell which devices can be configured
2) offline-"install" the storage driver and thus enable the guest too boot
3) copy over the matching drivers into the guest and allow it to pick them up on the first boot
Obviously virtio-win driver packaging and libguestfs must agree on how to deal with this.
Could you please provide any guidance on how to address this problem?
As Vadim said: "The best way would be having a set of separate distribution images packaged on per-platform base.”
Otherwise you will have to maintain the knowledge of binary compatibility between Windows platforms, which is different according to the driver type.
BZ 1223668 suggests reorganizing the directory structure to allow for automatic driver discovery. This would change the iso directory structure from: /<driver>/<os>/<arch>/<file> to: /<arch>/<os>/<file> which is the same structure used in /usr/share/virtio-win/drivers and in the vfd files. Would this work? -Jeff
Thanks, Roman.
On Wed, Oct 28, 2015 at 02:54:00PM -0400, Jeff Nelson wrote:
BZ 1223668 suggests reorganizing the directory structure to allow for automatic driver discovery. This would change the iso directory structure from:
/<driver>/<os>/<arch>/<file>
to:
/<arch>/<os>/<file>
which is the same structure used in /usr/share/virtio-win/drivers and in the vfd files.
Would this work?
I don't think this would make a difference to virt-v2v, since (currently) v2v matches bits of the path to try to determine which driver(s) to apply. Code is here: https://github.com/libguestfs/libguestfs/blob/master/v2v/utils.ml#L185-L226 As Vadim pointed out in the other email, this information is already encoded in the .inf files, and that seems like a better place to get it from. If we do that, then the exact path will matter even less. - - - I think the specific problem Roman was saying was that the directory and the ISO contain different, overlapping subsets of the total set of drivers. Which seems strange ... virt-v2v can pull drivers out of either the ISO or the filesystem, but (currently) not both. So if you point virt-v2v at the ISO you'll get a different subset of the drivers from pointing v2v at the filesystem. See also discussion of "VIRTIO_WIN" environment variable here: http://libguestfs.org/virt-v2v.1.html#environment-variables Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org
On Thu, 2015-10-29 at 08:34 +0000, Richard W.M. Jones wrote:
On Wed, Oct 28, 2015 at 02:54:00PM -0400, Jeff Nelson wrote:
BZ 1223668 suggests reorganizing the directory structure to allow for automatic driver discovery. This would change the iso directory structure from:
/<driver>/<os>/<arch>/<file>
to:
/<arch>/<os>/<file>
which is the same structure used in /usr/share/virtio-win/drivers and in the vfd files.
Would this work?
I don't think this would make a difference to virt-v2v, since (currently) v2v matches bits of the path to try to determine which driver(s) to apply. Code is here:
https://github.com/libguestfs/libguestfs/blob/master/v2v/utils.ml#L185-L226
As Vadim pointed out in the other email, this information is already encoded in the .inf files, and that seems like a better place to get it from. If we do that, then the exact path will matter even less.
The key point here is "automatic driver discovery". Couple of years ago we tried to come up with some sort of "universal" iso layout which will work for all supported (at that time) Windows platforms from WinXP to Win8/WS2012. Unfortunately it doesn't work, mostly because MS changed the naming conversion pattern during the time. It's why I said that maintaining a separate per-OS iso image will be the easiest (an probably the only) solution. Best regards, Vadim.
- - -
I think the specific problem Roman was saying was that the directory and the ISO contain different, overlapping subsets of the total set of drivers. Which seems strange ...
virt-v2v can pull drivers out of either the ISO or the filesystem, but (currently) not both. So if you point virt-v2v at the ISO you'll get a different subset of the drivers from pointing v2v at the filesystem. See also discussion of "VIRTIO_WIN" environment variable here:
http://libguestfs.org/virt-v2v.1.html#environment-variables
Rich.
On Wed, 2015-10-28 at 14:53 +0300, Roman Kagan wrote:
On Wed, Oct 28, 2015 at 12:10:19PM +1100, Vadim Rozenfeld wrote:
To sum up, the packaging and naming policy of the virtio-win rpm and the virtio-win iso therein are different and neither is clear. Hardcoding the policy in v2v without actually knowing it appears risky at best.
It's due to historical reasons mostly. The best way would be having a set of separate distribution images packaged on per-platform base.
Let me try to get the libguestfs requirements straight:
given a set of Windows drivers, it should be able to identify the ones appropriate for the particular Windows flavor, in order to
1) tell which devices can be configured Not sure that I fully understated your question. but if you are going to create some sort of off-line automatic virtio-win drivers update utility, then it shouldn't be too complicated. Firs of all you will need to obtain the Windows kernel version by reading the following Registry key - "HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion". Let's say it 6.3, which means that it is Win8.1 or WS2012R2, parsing "BuildLabEx" string from the same hive will give you information about the platform bitness. Next you need to go through inf files, and find "DriverVer" string, like this one . taken from from vioscsi.inf
DriverVer=08/01/2015,62.72.104.10800 This string contains build time and version stamps. The version stamp looks as follow "62.72.104.10800" where 62 means a target Windows kernel version multiplied by 10. In this case it is 6.2 which means Win8 or WS2012 72 - the target host platform version multiplied bu 10 (was RHEL7.2) 104 just a magic number, but it can be changed, don't make any assumption based on this number. 10800 our internal build number (build 108) multiplied by 100 If you found an inf file with the matching minor OS (6 in our case) version and matching or less but close minor version number (2 vs 3) then you are in the right directory.
2) offline-"install" the storage driver and thus enable the guest too boot
This is not so easy. You can do the same thing that Rich does for v2v an it should work fine. But in some cases, when a guest already has at least one third party storport miniport device driver intalled, the boot can fail
3) copy over the matching drivers into the guest and allow it to pick them up on the first boot. It should be pretty the same what v2v does. Some sort of emulation of what "software first" installation procedure does.
Cheers, Vadim.
Obviously virtio-win driver packaging and libguestfs must agree on how to deal with this.
Could you please provide any guidance on how to address this problem?
Thanks, Roman.
On Thu, Oct 29, 2015 at 11:05:42AM +1100, Vadim Rozenfeld wrote:
1) tell which devices can be configured Not sure that I fully understated your question. but if you are going to create some sort of off-line automatic virtio-win drivers update utility, then it shouldn't be too complicated.
Roman is talking about what virt-v2v already does. This is good information:
First of all you will need to obtain the Windows kernel version by reading the following Registry key - "HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion". Let's say it 6.3, which means that it is Win8.1 or WS2012R2, parsing "BuildLabEx" string from the same hive will give you information about the platform bitness. Next you need to go through inf files, and find "DriverVer" string, like this one . taken from from vioscsi.inf
DriverVer=08/01/2015,62.72.104.10800
This string contains build time and version stamps. The version stamp looks as follow "62.72.104.10800" where 62 means a target Windows kernel version multiplied by 10. In this case it is 6.2 which means Win8 or WS2012 72 - the target host platform version multiplied bu 10 (was RHEL7.2) 104 just a magic number, but it can be changed, don't make any assumption based on this number. 10800 our internal build number (build 108) multiplied by 100 If you found an inf file with the matching minor OS (6 in our case) version and matching or less but close minor version number (2 vs 3) then you are in the right directory.
Seems like looking at this field in the .inf file is better than our current approach of matching path names. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW
On 29 Oct 2015, at 10:28, Richard W.M. Jones <rjones@redhat.com> wrote:
On Thu, Oct 29, 2015 at 11:05:42AM +1100, Vadim Rozenfeld wrote:
1) tell which devices can be configured Not sure that I fully understated your question. but if you are going to create some sort of off-line automatic virtio-win drivers update utility, then it shouldn't be too complicated.
Roman is talking about what virt-v2v already does.
This is good information:
First of all you will need to obtain the Windows kernel version by reading the following Registry key - "HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion". Let's say it 6.3, which means that it is Win8.1 or WS2012R2, parsing "BuildLabEx" string from the same hive will give you information about the platform bitness. Next you need to go through inf files, and find "DriverVer" string, like this one . taken from from vioscsi.inf
DriverVer=08/01/2015,62.72.104.10800
This string contains build time and version stamps. The version stamp looks as follow "62.72.104.10800" where 62 means a target Windows kernel version multiplied by 10. In this case it is 6.2 which means Win8 or WS2012 72 - the target host platform version multiplied bu 10 (was RHEL7.2) 104 just a magic number, but it can be changed, don't make any assumption based on this number. 10800 our internal build number (build 108) multiplied by 100 If you found an inf file with the matching minor OS (6 in our case) version and matching or less but close minor version number (2 vs 3) then you are in the right directory.
Seems like looking at this field in the .inf file is better than our current approach of matching path names.
You just need to remember that we have binary compatible versions of drivers. For example virtio-net driver for Windows 2012 (Windows 8) (6.2) will be used for Windows 2012R2 (Windows 8.1) and Windows Server 2016 (Windows 10). So you are getting the lowest compatible version of the OS from INF file.
Rich.
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW
On Thu, Oct 29, 2015 at 10:33:57AM +0200, Yan Vugenfirer wrote:
On 29 Oct 2015, at 10:28, Richard W.M. Jones <rjones@redhat.com> wrote: On Thu, Oct 29, 2015 at 11:05:42AM +1100, Vadim Rozenfeld wrote:
Seems like looking at this field in the .inf file is better than our current approach of matching path names.
You just need to remember that we have binary compatible versions of drivers. For example virtio-net driver for Windows 2012 (Windows 8) (6.2) will be used for Windows 2012R2 (Windows 8.1) and Windows Server 2016 (Windows 10). So you are getting the lowest compatible version of the OS from INF file.
That seems to match Vadim's description:
If you found an inf file with the matching minor OS (6 in our case) version and matching or less but close minor version number (2 vs 3) then you are in the right directory.
and it's OK as it's an algorithm that can be coded and made to work reliably. However, do I get it right that this is just a convention currently followed in virtio-win drivers? I.e. there's nothing in the Windows driver specifications mandating this versioning scheme? I'm trying to figure out how dependable it is. Besides, ATM there's at least one driver -- qemupciserial -- which doesn't fit in this scheme (or any scheme at all :). There are other peculiarities which we don't know what to do: - the netkvm driver directory on virtio-win.iso contains netkvmco.dll (I'm assuming it's coinstaller dll). However, netkvm.inf doesn't mention it, and the driver installs just fine without it. - the balloon driver directory on virtio-win.iso contains blnsvr.exe (and .pdb) which is apparently a usermode service to report memory stats to host via balloon virtio queue. However nothing seems to automatically trigger installation of that service. Thanks, Roman.
On 29 Oct 2015, at 11:53, Roman Kagan <rkagan@virtuozzo.com> wrote:
On Thu, Oct 29, 2015 at 10:33:57AM +0200, Yan Vugenfirer wrote:
On 29 Oct 2015, at 10:28, Richard W.M. Jones <rjones@redhat.com> wrote: On Thu, Oct 29, 2015 at 11:05:42AM +1100, Vadim Rozenfeld wrote:
Seems like looking at this field in the .inf file is better than our current approach of matching path names.
You just need to remember that we have binary compatible versions of drivers. For example virtio-net driver for Windows 2012 (Windows 8) (6.2) will be used for Windows 2012R2 (Windows 8.1) and Windows Server 2016 (Windows 10). So you are getting the lowest compatible version of the OS from INF file.
That seems to match Vadim's description:
If you found an inf file with the matching minor OS (6 in our case) version and matching or less but close minor version number (2 vs 3) then you are in the right directory.
and it's OK as it's an algorithm that can be coded and made to work reliably.
However, do I get it right that this is just a convention currently followed in virtio-win drivers? I.e. there's nothing in the Windows driver specifications mandating this versioning scheme? I'm trying to figure out how dependable it is.
This scheme of driver versioning is not mandated by MS specification. It was an internal decision made several years ago and we intend to continue following it.
Besides, ATM there's at least one driver -- qemupciserial -- which doesn't fit in this scheme (or any scheme at all :).
It is single INF for all OSes. And we can change the versioning to comply to virtio-win convention. Also qxl (Spice driver) has similar versioning scheme. In qxl INF the version will be 5.1.0.10018 for XP and 6.1.0.10018 for Windows 7 for example.
There are other peculiarities which we don't know what to do:
- the netkvm driver directory on virtio-win.iso contains netkvmco.dll (I'm assuming it's coinstaller dll). However, netkvm.inf doesn't mention it, and the driver installs just fine without it.
It can be used as co-installer, but eventually it was decided not to use it in this way and allow the user manual activation of the functionality in netkvmco.dll. MS required that command line tools should be provided by the NDIS miniport drivers that have GUI configuration (advanced tab in device manager). The netkvmco is essentially netsh plugin that can be used to change NetKVM parameters (or actually parameters of any network driver in the system as the parameters are saved in MS mandated format). The documentation for the netkvmco.dll is found in readyme.doc file (once again this is MS requirement).
- the balloon driver directory on virtio-win.iso contains blnsvr.exe (and .pdb) which is apparently a usermode service to report memory stats to host via balloon virtio queue. However nothing seems to automatically trigger installation of that service.
Thanks, Roman.
On Thu, 2015-10-29 at 12:53 +0300, Roman Kagan wrote:
On Thu, Oct 29, 2015 at 10:33:57AM +0200, Yan Vugenfirer wrote:
On 29 Oct 2015, at 10:28, Richard W.M. Jones <rjones@redhat.com> wrote: On Thu, Oct 29, 2015 at 11:05:42AM +1100, Vadim Rozenfeld wrote:
Seems like looking at this field in the .inf file is better than our current approach of matching path names.
You just need to remember that we have binary compatible versions of drivers. For example virtio-net driver for Windows 2012 (Windows 8) (6.2) will be used for Windows 2012R2 (Windows 8.1) and Windows Server 2016 (Windows 10). So you are getting the lowest compatible version of the OS from INF file.
That seems to match Vadim's description:
If you found an inf file with the matching minor OS (6 in our case) version and matching or less but close minor version number (2 vs 3) then you are in the right directory.
and it's OK as it's an algorithm that can be coded and made to work reliably.
However, do I get it right that this is just a convention currently followed in virtio-win drivers? I.e. there's nothing in the Windows driver specifications mandating this versioning scheme? I'm trying to figure out how dependable it is.
No restrictions from MS side, OEM or SW vendor can implement they own versioning scheme. Only two rules - no zeroed fields in the driver version section (to make WHQL inf check test happy), keep driver version increasing from build to build (to simplify driver update procedure).
Besides, ATM there's at least one driver -- qemupciserial -- which doesn't fit in this scheme (or any scheme at all :).
There are other peculiarities which we don't know what to do:
- the netkvm driver directory on virtio-win.iso contains netkvmco.dll (I'm assuming it's coinstaller dll). However, netkvm.inf doesn't mention it, and the driver installs just fine without it.
- the balloon driver directory on virtio-win.iso contains blnsvr.exe (and .pdb) which is apparently a usermode service to report memory stats to host via balloon virtio queue. However nothing seems to automatically trigger installation of that service.
Not sure that it will happen soon. The reason is simple - Windows allows to install any binary as a part of driver installation procedure. It only needs to be mentioned in inf file. But then if we change and rebuild that binary - we need to regenerate cat file, re-WHQL that driver and re-submit cat file to Microsoft. Even though that the balloon service is stable enough, I wouldn't recommend making it as a part of automated balloon driver installation procedure. At least not now. Best regards, Vadim.
Thanks, Roman.
In terms of turning what Vadim wrote into code ...
First of all you will need to obtain the Windows kernel version by reading the following Registry key - "HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion". Let's say it 6.3, which means that it is Win8.1 or WS2012R2,
We already access this field in the registry and present it through inspection, so: http://libguestfs.org/guestfs.3.html#guestfs_inspect_get_major_version In virt-v2v this is present in the Types.inspect.i_major_version and Types.inspect.i_minor_version fields.
parsing "BuildLabEx" string from the same hive will give you information about the platform bitness.
Currently we can determine the bitness of a Windows VM by examining some of the binaries found in %systemroot%. We present this through inspection: http://libguestfs.org/guestfs.3.html#guestfs_inspect_get_arch (either "i386" or "x86-64") and this is present in virt-v2v in the Types.inspect.i_arch field. I looked at BuildLabEx, which is a field we've not looked at before.
From Windows 7:
"BuildLabEx"="7601.18247.amd64fre.win7sp1_gdr.130828-1532" Apparently the first four numbers are the build number, next five are the revision number, and after that is either ".amd64" or ".x86" (for x86-64 or i386 respectively).
Next you need to go through inf files, and find "DriverVer" string, like this one . taken from from vioscsi.inf
DriverVer=08/01/2015,62.72.104.10800
This string contains build time and version stamps. The version stamp looks as follow "62.72.104.10800" where 62 means a target Windows kernel version multiplied by 10. In this case it is 6.2 which means Win8 or WS2012 72 - the target host platform version multiplied bu 10 (was RHEL7.2) 104 just a magic number, but it can be changed, don't make any assumption based on this number. 10800 our internal build number (build 108) multiplied by 100 If you found an inf file with the matching minor OS (6 in our case) version and matching or less but close minor version number (2 vs 3) then you are in the right directory.
This should be easy enough to parse out. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v
On Thu, Oct 29, 2015 at 11:05:42AM +1100, Vadim Rozenfeld wrote:
On Wed, 2015-10-28 at 14:53 +0300, Roman Kagan wrote:
1) tell which devices can be configured Not sure that I fully understated your question. but if you are going to create some sort of off-line automatic virtio-win drivers update utility, then it shouldn't be too complicated. Firs of all you will need to obtain the Windows kernel version by reading the following Registry key - "HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion". Let's say it 6.3, which means that it is Win8.1 or WS2012R2, parsing "BuildLabEx" string from the same hive will give you information about the platform bitness. Next you need to go through inf files, and find "DriverVer" string, like this one . taken from from vioscsi.inf
DriverVer=08/01/2015,62.72.104.10800
This string contains build time and version stamps. The version stamp looks as follow "62.72.104.10800" where 62 means a target Windows kernel version multiplied by 10. In this case it is 6.2 which means Win8 or WS2012 72 - the target host platform version multiplied bu 10 (was RHEL7.2) 104 just a magic number, but it can be changed, don't make any assumption based on this number. 10800 our internal build number (build 108) multiplied by 100
Hi Vadim, I've almost finished implementing this, but I have a couple of technical questions: (1) How can we tell from the .inf file what architecture (x86 or amd64) the driver is for? There is a [Manufacturer] section which looks hopeful, eg: [Manufacturer] %RHEL% = RHEL,NTAMD64 but the contents are not very consistent across all the .inf files I have access to. (2) If we have a directory containing multiple drivers, can we assume that if any .inf file found matches the guest OS, then all those drivers can be installed? So far this appears to be true. If not then we have to start associating drivers with .inf files, which seems a bit complicated. (3) You write:
If you found an inf file with the matching minor OS (6 in our case) version and matching or less but close minor version number (2 vs 3) then you are in the right directory.
When you say "close minor version number" is there a particular definition you have in mind? eg. minor or minor-1 is OK? Or do we have to consider all .inf files in totality and choose the nearest one? Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-builder quickly builds VMs from scratch http://libguestfs.org/virt-builder.1.html
On Thu, 2015-11-05 at 21:46 +0000, Richard W.M. Jones wrote:
On Thu, Oct 29, 2015 at 11:05:42AM +1100, Vadim Rozenfeld wrote:
On Wed, 2015-10-28 at 14:53 +0300, Roman Kagan wrote:
1) tell which devices can be configured Not sure that I fully understated your question. but if you are going to create some sort of off-line automatic virtio-win drivers update utility, then it shouldn't be too complicated. Firs of all you will need to obtain the Windows kernel version by reading the following Registry key - "HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion". Let's say it 6.3, which means that it is Win8.1 or WS2012R2, parsing "BuildLabEx" string from the same hive will give you information about the platform bitness. Next you need to go through inf files, and find "DriverVer" string, like this one . taken from from vioscsi.inf
DriverVer=08/01/2015,62.72.104.10800
This string contains build time and version stamps. The version stamp looks as follow "62.72.104.10800" where 62 means a target Windows kernel version multiplied by 10. In this case it is 6.2 which means Win8 or WS2012 72 - the target host platform version multiplied bu 10 (was RHEL7.2) 104 just a magic number, but it can be changed, don't make any assumption based on this number. 10800 our internal build number (build 108) multiplied by 100
Hi Vadim,
I've almost finished implementing this, but I have a couple of technical questions:
(1) How can we tell from the .inf file what architecture (x86 or amd64) the driver is for? There is a [Manufacturer] section which looks hopeful, eg:
[Manufacturer] %RHEL% = RHEL,NTAMD64
but the contents are not very consistent across all the .inf files I have access to.
Hi Richard, It should be consistent. We do specify target architecture (amd64 or x86) when stamping inf/inx files. IIRC, qemupciserial.inf is the only exception, but it goes without binaries. In any case, if you are going to use existing distribution media, you can rely on the media layout - for example, if "amd64" is a part of your current path - you are in 64-bit directory.
(2) If we have a directory containing multiple drivers, can we assume that if any .inf file found matches the guest OS, then all those drivers can be installed? So far this appears to be true.
Yes, we will never mix 32 and 64 bit drivers into the same directory. I swear:)
If not then we have to start associating drivers with .inf files, which seems a bit complicated.
(3) You write:
If you found an inf file with the matching minor OS (6 in our case) version and matching or less but close minor version number (2 vs 3) then you are in the right directory.
When you say "close minor version number" is there a particular definition you have in mind? eg. minor or minor-1 is OK? Or do we have to consider all .inf files in totality and choose the nearest one? Ideally, we should try finding the exact match. If not, we have to find one with the same major version number and nearest minor number which is smaller than the target one. For example, let's say we are looking for a driver for Win8.1/WS2012R2, we know that both of them have kernel version 6.3 . So, ideally we should to find inf file with the following driver version string:
DriverVer=XX/YY/ZZZZ,63.AA.BBB.CCCCC if we cannot find an inf file with such criteria, we should try finding another one with the OS version 62 (which means that a driver was build for kernel 6.2 Win8/WS2012 and technically should be compatible with 6.3). So, we are looking for DriverVer which looks like this one: DriverVer=XX/YY/ZZZZ,62.AA.BBB.CCCCC Still no luck - decrease the target OS version to 61 and repeat. For WDF drivers like balloon, vioserial, and rng you can go as far as you can trying to find an appropriate driver. For miniport drivers like netkvm ( Yan, please confirm ), viostor and vioscsi I wouldn't go too far. For example drivers designed to fork in specific stacks at kernel version 5.1 (WinXP) can refuse working with kernel 6.2 (Win8). However, Win10 will be a problem. At least as long as we do not release virtio-win drivers built for this particular OS version, and keep using drivers designed for Win8/WS2012 and Win8.1/WS2012R2 platforms. Win10 has kernel version 10.0 (guys, who has Win10 RTM running - please confirm). It used to be 6.4 for RC and Beta versions, but IIRC Microsoft decided to change it to 10.0 for RTM and final release. Microsoft promised that drivers designed for Win8/WS2012 and Win8.1/WS2012R2 platforms will continue working, at least for the nearest future. Vadim.
Rich.
On 06 Nov 2015, at 08:51, Vadim Rozenfeld <vrozenfe@redhat.com> wrote:
On Thu, 2015-11-05 at 21:46 +0000, Richard W.M. Jones wrote:
On Thu, Oct 29, 2015 at 11:05:42AM +1100, Vadim Rozenfeld wrote:
On Wed, 2015-10-28 at 14:53 +0300, Roman Kagan wrote:
1) tell which devices can be configured Not sure that I fully understated your question. but if you are going to create some sort of off-line automatic virtio-win drivers update utility, then it shouldn't be too complicated. Firs of all you will need to obtain the Windows kernel version by reading the following Registry key - "HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion". Let's say it 6.3, which means that it is Win8.1 or WS2012R2, parsing "BuildLabEx" string from the same hive will give you information about the platform bitness. Next you need to go through inf files, and find "DriverVer" string, like this one . taken from from vioscsi.inf
DriverVer=08/01/2015,62.72.104.10800
This string contains build time and version stamps. The version stamp looks as follow "62.72.104.10800" where 62 means a target Windows kernel version multiplied by 10. In this case it is 6.2 which means Win8 or WS2012 72 - the target host platform version multiplied bu 10 (was RHEL7.2) 104 just a magic number, but it can be changed, don't make any assumption based on this number. 10800 our internal build number (build 108) multiplied by 100
Hi Vadim,
I've almost finished implementing this, but I have a couple of technical questions:
(1) How can we tell from the .inf file what architecture (x86 or amd64) the driver is for? There is a [Manufacturer] section which looks hopeful, eg:
[Manufacturer] %RHEL% = RHEL,NTAMD64
but the contents are not very consistent across all the .inf files I have access to.
Hi Richard,
It should be consistent. We do specify target architecture (amd64 or x86) when stamping inf/inx files. IIRC, qemupciserial.inf is the only exception, but it goes without binaries. In any case, if you are going to use existing distribution media, you can rely on the media layout - for example, if "amd64" is a part of your current path - you are in 64-bit directory.
(2) If we have a directory containing multiple drivers, can we assume that if any .inf file found matches the guest OS, then all those drivers can be installed? So far this appears to be true.
Yes, we will never mix 32 and 64 bit drivers into the same directory. I swear:)
If not then we have to start associating drivers with .inf files, which seems a bit complicated.
(3) You write:
If you found an inf file with the matching minor OS (6 in our case) version and matching or less but close minor version number (2 vs 3) then you are in the right directory.
When you say "close minor version number" is there a particular definition you have in mind? eg. minor or minor-1 is OK? Or do we have to consider all .inf files in totality and choose the nearest one? Ideally, we should try finding the exact match. If not, we have to find one with the same major version number and nearest minor number which is smaller than the target one. For example, let's say we are looking for a driver for Win8.1/WS2012R2, we know that both of them have kernel version 6.3 . So, ideally we should to find inf file with the following driver version string:
DriverVer=XX/YY/ZZZZ,63.AA.BBB.CCCCC
if we cannot find an inf file with such criteria, we should try finding another one with the OS version 62 (which means that a driver was build for kernel 6.2 Win8/WS2012 and technically should be compatible with 6.3). So, we are looking for DriverVer which looks like this one: DriverVer=XX/YY/ZZZZ,62.AA.BBB.CCCCC
Still no luck - decrease the target OS version to 61 and repeat.
For WDF drivers like balloon, vioserial, and rng you can go as far as you can trying to find an appropriate driver.
For miniport drivers like netkvm ( Yan, please confirm ), viostor and vioscsi I wouldn't go too far. For example drivers designed to fork in specific stacks at kernel version 5.1 (WinXP) can refuse working with kernel 6.2 (Win8).
While NDIS miniport designed for NDIS 5.1 (XP) still can run on latest Windows, I wouldn’t do that. For NDIS 6.x (Windows 2008 and newer), we can use older drivers on new OS. But installing a driver for a newer OS on older OS might lead to driver miss-behaviour.
However, Win10 will be a problem. At least as long as we do not release virtio-win drivers built for this particular OS version, and keep using drivers designed for Win8/WS2012 and Win8.1/WS2012R2 platforms. Win10 has kernel version 10.0 (guys, who has Win10 RTM running - please confirm). It used to be 6.4 for RC and Beta versions, but IIRC Microsoft decided to change it to 10.0 for RTM and final release. Microsoft promised that drivers designed for Win8/WS2012 and Win8.1/WS2012R2 platforms will continue working, at least for the nearest future.
Vadim.
Rich.
On Sun, Nov 08, 2015 at 10:15:31AM +0200, Yan Vugenfirer wrote:
On 06 Nov 2015, at 08:51, Vadim Rozenfeld <vrozenfe@redhat.com> wrote:
On Thu, 2015-11-05 at 21:46 +0000, Richard W.M. Jones wrote:
On Thu, Oct 29, 2015 at 11:05:42AM +1100, Vadim Rozenfeld wrote:
On Wed, 2015-10-28 at 14:53 +0300, Roman Kagan wrote:
1) tell which devices can be configured Not sure that I fully understated your question. but if you are going to create some sort of off-line automatic virtio-win drivers update utility, then it shouldn't be too complicated. Firs of all you will need to obtain the Windows kernel version by reading the following Registry key - "HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion". Let's say it 6.3, which means that it is Win8.1 or WS2012R2, parsing "BuildLabEx" string from the same hive will give you information about the platform bitness. Next you need to go through inf files, and find "DriverVer" string, like this one . taken from from vioscsi.inf
DriverVer=08/01/2015,62.72.104.10800
This string contains build time and version stamps. The version stamp looks as follow "62.72.104.10800" where 62 means a target Windows kernel version multiplied by 10. In this case it is 6.2 which means Win8 or WS2012 72 - the target host platform version multiplied bu 10 (was RHEL7.2) 104 just a magic number, but it can be changed, don't make any assumption based on this number. 10800 our internal build number (build 108) multiplied by 100
Hi Vadim,
I've almost finished implementing this, but I have a couple of technical questions:
(1) How can we tell from the .inf file what architecture (x86 or amd64) the driver is for? There is a [Manufacturer] section which looks hopeful, eg:
[Manufacturer] %RHEL% = RHEL,NTAMD64
but the contents are not very consistent across all the .inf files I have access to.
Hi Richard,
It should be consistent. We do specify target architecture (amd64 or x86) when stamping inf/inx files. IIRC, qemupciserial.inf is the only exception, but it goes without binaries. In any case, if you are going to use existing distribution media, you can rely on the media layout - for example, if "amd64" is a part of your current path - you are in 64-bit directory.
(2) If we have a directory containing multiple drivers, can we assume that if any .inf file found matches the guest OS, then all those drivers can be installed? So far this appears to be true.
Yes, we will never mix 32 and 64 bit drivers into the same directory. I swear:)
If not then we have to start associating drivers with .inf files, which seems a bit complicated.
(3) You write:
If you found an inf file with the matching minor OS (6 in our case) version and matching or less but close minor version number (2 vs 3) then you are in the right directory.
When you say "close minor version number" is there a particular definition you have in mind? eg. minor or minor-1 is OK? Or do we have to consider all .inf files in totality and choose the nearest one? Ideally, we should try finding the exact match. If not, we have to find one with the same major version number and nearest minor number which is smaller than the target one. For example, let's say we are looking for a driver for Win8.1/WS2012R2, we know that both of them have kernel version 6.3 . So, ideally we should to find inf file with the following driver version string:
DriverVer=XX/YY/ZZZZ,63.AA.BBB.CCCCC
if we cannot find an inf file with such criteria, we should try finding another one with the OS version 62 (which means that a driver was build for kernel 6.2 Win8/WS2012 and technically should be compatible with 6.3). So, we are looking for DriverVer which looks like this one: DriverVer=XX/YY/ZZZZ,62.AA.BBB.CCCCC
Still no luck - decrease the target OS version to 61 and repeat.
For WDF drivers like balloon, vioserial, and rng you can go as far as you can trying to find an appropriate driver.
For miniport drivers like netkvm ( Yan, please confirm ), viostor and vioscsi I wouldn't go too far. For example drivers designed to fork in specific stacks at kernel version 5.1 (WinXP) can refuse working with kernel 6.2 (Win8).
While NDIS miniport designed for NDIS 5.1 (XP) still can run on latest Windows, I wouldn’t do that. For NDIS 6.x (Windows 2008 and newer), we can use older drivers on new OS. But installing a driver for a newer OS on older OS might lead to driver miss-behaviour.
I'm wondering how people who don't rely on virt-v2v are supposed to discover the virtio drivers for their guests? This includes both those who do it programmatically and those who locate the appropriate drivers manually. I mean, both manual and programmatic procedures should resolve to the same set of drivers, and ideally the procedure should be the same, declared official and documented. I see no problem implementing any particular algorithm, no matter how complex, in v2v, but I'd like to make sure that my colleagues who, say, write a tool for unattended deployment of Windows guests, or others, who manually set up their Windows VMs, arrive at the same configuration. I tend to think that your original idea of sorting that out on the driver build side, and storing the resulting drivers grouped by Windows arch/version following a well-defined and stable naming convention is a more reliable and universal solution. Thanks, Roman.
On Mon, 2015-11-09 at 18:08 +0300, Roman Kagan wrote:
On Sun, Nov 08, 2015 at 10:15:31AM +0200, Yan Vugenfirer wrote:
On 06 Nov 2015, at 08:51, Vadim Rozenfeld <vrozenfe@redhat.com> wrote:
On Thu, 2015-11-05 at 21:46 +0000, Richard W.M. Jones wrote:
On Thu, Oct 29, 2015 at 11:05:42AM +1100, Vadim Rozenfeld wrote:
On Wed, 2015-10-28 at 14:53 +0300, Roman Kagan wrote:
1) tell which devices can be configured Not sure that I fully understated your question. but if you are going to create some sort of off-line automatic virtio-win drivers update utility, then it shouldn't be too complicated. Firs of all you will need to obtain the Windows kernel version by reading the following Registry key - "HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion". Let's say it 6.3, which means that it is Win8.1 or WS2012R2, parsing "BuildLabEx" string from the same hive will give you information about the platform bitness. Next you need to go through inf files, and find "DriverVer" string, like this one . taken from from vioscsi.inf
DriverVer=08/01/2015,62.72.104.10800
This string contains build time and version stamps. The version stamp looks as follow "62.72.104.10800" where 62 means a target Windows kernel version multiplied by 10. In this case it is 6.2 which means Win8 or WS2012 72 - the target host platform version multiplied bu 10 (was RHEL7.2) 104 just a magic number, but it can be changed, don't make any assumption based on this number. 10800 our internal build number (build 108) multiplied by 100
Hi Vadim,
I've almost finished implementing this, but I have a couple of technical questions:
(1) How can we tell from the .inf file what architecture (x86 or amd64) the driver is for? There is a [Manufacturer] section which looks hopeful, eg:
[Manufacturer] %RHEL% = RHEL,NTAMD64
but the contents are not very consistent across all the .inf files I have access to.
Hi Richard,
It should be consistent. We do specify target architecture (amd64 or x86) when stamping inf/inx files. IIRC, qemupciserial.inf is the only exception, but it goes without binaries. In any case, if you are going to use existing distribution media, you can rely on the media layout - for example, if "amd64" is a part of your current path - you are in 64-bit directory.
(2) If we have a directory containing multiple drivers, can we assume that if any .inf file found matches the guest OS, then all those drivers can be installed? So far this appears to be true.
Yes, we will never mix 32 and 64 bit drivers into the same directory. I swear:)
If not then we have to start associating drivers with .inf files, which seems a bit complicated.
(3) You write:
If you found an inf file with the matching minor OS (6 in our case) version and matching or less but close minor version number (2 vs 3) then you are in the right directory.
When you say "close minor version number" is there a particular definition you have in mind? eg. minor or minor-1 is OK? Or do we have to consider all .inf files in totality and choose the nearest one? Ideally, we should try finding the exact match. If not, we have to find one with the same major version number and nearest minor number which is smaller than the target one. For example, let's say we are looking for a driver for Win8.1/WS2012R2, we know that both of them have kernel version 6.3 . So, ideally we should to find inf file with the following driver version string:
DriverVer=XX/YY/ZZZZ,63.AA.BBB.CCCCC
if we cannot find an inf file with such criteria, we should try finding another one with the OS version 62 (which means that a driver was build for kernel 6.2 Win8/WS2012 and technically should be compatible with 6.3). So, we are looking for DriverVer which looks like this one: DriverVer=XX/YY/ZZZZ,62.AA.BBB.CCCCC
Still no luck - decrease the target OS version to 61 and repeat.
For WDF drivers like balloon, vioserial, and rng you can go as far as you can trying to find an appropriate driver.
For miniport drivers like netkvm ( Yan, please confirm ), viostor and vioscsi I wouldn't go too far. For example drivers designed to fork in specific stacks at kernel version 5.1 (WinXP) can refuse working with kernel 6.2 (Win8).
While NDIS miniport designed for NDIS 5.1 (XP) still can run on latest Windows, I wouldn’t do that. For NDIS 6.x (Windows 2008 and newer), we can use older drivers on new OS. But installing a driver for a newer OS on older OS might lead to driver miss-behaviour.
I'm wondering how people who don't rely on virt-v2v are supposed to discover the virtio drivers for their guests? This includes both those who do it programmatically and those who locate the appropriate drivers manually.
I mean, both manual and programmatic procedures should resolve to the same set of drivers, and ideally the procedure should be the same, declared official and documented.
I see no problem implementing any particular algorithm, no matter how complex, in v2v, but I'd like to make sure that my colleagues who, say, write a tool for unattended deployment of Windows guests, or others, who manually set up their Windows VMs, arrive at the same configuration.
I tend to think that your original idea of sorting that out on the driver build side, and storing the resulting drivers grouped by Windows arch/version following a well-defined and stable naming convention is a more reliable and universal solution.
It definitely shouldn't be so complicated. Packaging drivers on per-platform base can solve most of the problems. One single iso image, containing both 32 and 64 bit drivers compatible to run on Win8/Win8.1/WS2012/WS2012R2, and exposing an appropriate media layout "understandable" by all these platforms, is what we need. After all, the information provided n this thread, should be enough for writing a script that can split our single iso image into several properly build ones. In this case, the only thing you should worry about, is that <os>\<architecture> naming pattern is not the same among the various Windows platforms. IIRC there were some changes around Vista/Longhorn when Microsoft did it. Cheers, Vadim.
Thanks, Roman.
+Cole. There are some packaging scripts and proposed BZs for making the iso and rpm contents rational. Adding Cole, since he is owner of at least one of the BZs. -Jeff On Tue, Nov 10, 2015 at 09:17:48AM +1100, Vadim Rozenfeld wrote:
On Mon, 2015-11-09 at 18:08 +0300, Roman Kagan wrote:
On Sun, Nov 08, 2015 at 10:15:31AM +0200, Yan Vugenfirer wrote:
On 06 Nov 2015, at 08:51, Vadim Rozenfeld <vrozenfe@redhat.com> wrote:
On Thu, 2015-11-05 at 21:46 +0000, Richard W.M. Jones wrote:
On Thu, Oct 29, 2015 at 11:05:42AM +1100, Vadim Rozenfeld wrote:
On Wed, 2015-10-28 at 14:53 +0300, Roman Kagan wrote: > 1) tell which devices can be configured Not sure that I fully understated your question. but if you are going to create some sort of off-line automatic virtio-win drivers update utility, then it shouldn't be too complicated. Firs of all you will need to obtain the Windows kernel version by reading the following Registry key - "HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion". Let's say it 6.3, which means that it is Win8.1 or WS2012R2, parsing "BuildLabEx" string from the same hive will give you information about the platform bitness. Next you need to go through inf files, and find "DriverVer" string, like this one . taken from from vioscsi.inf
DriverVer=08/01/2015,62.72.104.10800
This string contains build time and version stamps. The version stamp looks as follow "62.72.104.10800" where 62 means a target Windows kernel version multiplied by 10. In this case it is 6.2 which means Win8 or WS2012 72 - the target host platform version multiplied bu 10 (was RHEL7.2) 104 just a magic number, but it can be changed, don't make any assumption based on this number. 10800 our internal build number (build 108) multiplied by 100
Hi Vadim,
I've almost finished implementing this, but I have a couple of technical questions:
(1) How can we tell from the .inf file what architecture (x86 or amd64) the driver is for? There is a [Manufacturer] section which looks hopeful, eg:
[Manufacturer] %RHEL% = RHEL,NTAMD64
but the contents are not very consistent across all the .inf files I have access to.
Hi Richard,
It should be consistent. We do specify target architecture (amd64 or x86) when stamping inf/inx files. IIRC, qemupciserial.inf is the only exception, but it goes without binaries. In any case, if you are going to use existing distribution media, you can rely on the media layout - for example, if "amd64" is a part of your current path - you are in 64-bit directory.
(2) If we have a directory containing multiple drivers, can we assume that if any .inf file found matches the guest OS, then all those drivers can be installed? So far this appears to be true.
Yes, we will never mix 32 and 64 bit drivers into the same directory. I swear:)
If not then we have to start associating drivers with .inf files, which seems a bit complicated.
(3) You write:
If you found an inf file with the matching minor OS (6 in our case) version and matching or less but close minor version number (2 vs 3) then you are in the right directory.
When you say "close minor version number" is there a particular definition you have in mind? eg. minor or minor-1 is OK? Or do we have to consider all .inf files in totality and choose the nearest one? Ideally, we should try finding the exact match. If not, we have to find one with the same major version number and nearest minor number which is smaller than the target one. For example, let's say we are looking for a driver for Win8.1/WS2012R2, we know that both of them have kernel version 6.3 . So, ideally we should to find inf file with the following driver version string:
DriverVer=XX/YY/ZZZZ,63.AA.BBB.CCCCC
if we cannot find an inf file with such criteria, we should try finding another one with the OS version 62 (which means that a driver was build for kernel 6.2 Win8/WS2012 and technically should be compatible with 6.3). So, we are looking for DriverVer which looks like this one: DriverVer=XX/YY/ZZZZ,62.AA.BBB.CCCCC
Still no luck - decrease the target OS version to 61 and repeat.
For WDF drivers like balloon, vioserial, and rng you can go as far as you can trying to find an appropriate driver.
For miniport drivers like netkvm ( Yan, please confirm ), viostor and vioscsi I wouldn't go too far. For example drivers designed to fork in specific stacks at kernel version 5.1 (WinXP) can refuse working with kernel 6.2 (Win8).
While NDIS miniport designed for NDIS 5.1 (XP) still can run on latest Windows, I wouldn’t do that. For NDIS 6.x (Windows 2008 and newer), we can use older drivers on new OS. But installing a driver for a newer OS on older OS might lead to driver miss-behaviour.
I'm wondering how people who don't rely on virt-v2v are supposed to discover the virtio drivers for their guests? This includes both those who do it programmatically and those who locate the appropriate drivers manually.
I mean, both manual and programmatic procedures should resolve to the same set of drivers, and ideally the procedure should be the same, declared official and documented.
I see no problem implementing any particular algorithm, no matter how complex, in v2v, but I'd like to make sure that my colleagues who, say, write a tool for unattended deployment of Windows guests, or others, who manually set up their Windows VMs, arrive at the same configuration.
I tend to think that your original idea of sorting that out on the driver build side, and storing the resulting drivers grouped by Windows arch/version following a well-defined and stable naming convention is a more reliable and universal solution.
It definitely shouldn't be so complicated. Packaging drivers on per-platform base can solve most of the problems. One single iso image, containing both 32 and 64 bit drivers compatible to run on Win8/Win8.1/WS2012/WS2012R2, and exposing an appropriate media layout "understandable" by all these platforms, is what we need. After all, the information provided n this thread, should be enough for writing a script that can split our single iso image into several properly build ones. In this case, the only thing you should worry about, is that <os>\<architecture> naming pattern is not the same among the various Windows platforms. IIRC there were some changes around Vista/Longhorn when Microsoft did it.
Cheers, Vadim.
Thanks, Roman.
I think one - maybe final? - problem. How can I tell the difference between drivers for "client" versions of Windows (eg. Windows 7) and server versions of Windows (eg. Windows 2008 Server)? It seems in many or most cases the drivers are identical, eg: $ md5sum viostor/2k12/amd64/* viostor/w8/amd64/* bbe250c13bf891fd7292ccab9908a63a viostor/2k12/amd64/viostor.cat c2fdfc6a47fdc9dd25242441cdf1a68e viostor/2k12/amd64/viostor.inf 5ca2cadca2214b90196755dab06676f4 viostor/2k12/amd64/viostor.pdb 80c5ea60a69f37e965cb277383abd067 viostor/2k12/amd64/viostor.sys bbe250c13bf891fd7292ccab9908a63a viostor/w8/amd64/viostor.cat c2fdfc6a47fdc9dd25242441cdf1a68e viostor/w8/amd64/viostor.inf 5ca2cadca2214b90196755dab06676f4 viostor/w8/amd64/viostor.pdb 80c5ea60a69f37e965cb277383abd067 viostor/w8/amd64/viostor.sys $ md5sum Balloon/2k12/amd64/* Balloon/w8/amd64/* d4104f08e674ab9def49d086d960349d Balloon/2k12/amd64/balloon.cat 0325b7e0552e1567292eb6c1068fee18 Balloon/2k12/amd64/balloon.inf ac2116f04fd678d0e264f34e3bf90d1e Balloon/2k12/amd64/balloon.pdb c975501dd5b43ade3717722b81acbc1f Balloon/2k12/amd64/balloon.sys 56f63afdd992a5395a4d8da6a52e559e Balloon/2k12/amd64/blnsvr.exe c539f2056fb196da377d14180fda195c Balloon/2k12/amd64/blnsvr.pdb d10864c1730172780c2d4be633b9220a Balloon/2k12/amd64/WdfCoInstaller01011.dll d4104f08e674ab9def49d086d960349d Balloon/w8/amd64/balloon.cat 0325b7e0552e1567292eb6c1068fee18 Balloon/w8/amd64/balloon.inf ac2116f04fd678d0e264f34e3bf90d1e Balloon/w8/amd64/balloon.pdb c975501dd5b43ade3717722b81acbc1f Balloon/w8/amd64/balloon.sys 56f63afdd992a5395a4d8da6a52e559e Balloon/w8/amd64/blnsvr.exe c539f2056fb196da377d14180fda195c Balloon/w8/amd64/blnsvr.pdb d10864c1730172780c2d4be633b9220a Balloon/w8/amd64/WdfCoInstaller01011.dll (completely identical) $ md5sum viostor/2k8/amd64/* viostor/w7/amd64/*d1132fbd27c98f25ea71a6eb4037b2f8 viostor/2k8/amd64/viostor.cat defccbe48aca1cd97fee4d7b16053b72 viostor/2k8/amd64/viostor.inf 6da6120368c097806e4ac7fc6b940810 viostor/2k8/amd64/viostor.pdb 63b05edfb75d9dc60e82ab3c0bfee701 viostor/2k8/amd64/viostor.sys af0cca02244fcfedf3209aef56610464 viostor/w7/amd64/viostor.cat defccbe48aca1cd97fee4d7b16053b72 viostor/w7/amd64/viostor.inf 6da6120368c097806e4ac7fc6b940810 viostor/w7/amd64/viostor.pdb 63b05edfb75d9dc60e82ab3c0bfee701 viostor/w7/amd64/viostor.sys (almost identical, *.cat files are different) In any case the *.inf files don't seem to have any distinguishing feature which would allow us to check this. Maybe this doesn't matter? Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-df lists disk usage of guests without needing to install any software inside the virtual machine. Supports Linux and Windows. http://people.redhat.com/~rjones/virt-df/
There was a copy and paste error. It should have read: $ md5sum viostor/2k8/amd64/* viostor/w7/amd64/* d1132fbd27c98f25ea71a6eb4037b2f8 viostor/2k8/amd64/viostor.cat defccbe48aca1cd97fee4d7b16053b72 viostor/2k8/amd64/viostor.inf 6da6120368c097806e4ac7fc6b940810 viostor/2k8/amd64/viostor.pdb 63b05edfb75d9dc60e82ab3c0bfee701 viostor/2k8/amd64/viostor.sys af0cca02244fcfedf3209aef56610464 viostor/w7/amd64/viostor.cat defccbe48aca1cd97fee4d7b16053b72 viostor/w7/amd64/viostor.inf 6da6120368c097806e4ac7fc6b940810 viostor/w7/amd64/viostor.pdb 63b05edfb75d9dc60e82ab3c0bfee701 viostor/w7/amd64/viostor.sys (almost identical, *.cat files are different) Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top
On Tue, Nov 17, 2015 at 08:49:54PM +0000, Richard W.M. Jones wrote:
There was a copy and paste error. It should have read:
$ md5sum viostor/2k8/amd64/* viostor/w7/amd64/* d1132fbd27c98f25ea71a6eb4037b2f8 viostor/2k8/amd64/viostor.cat defccbe48aca1cd97fee4d7b16053b72 viostor/2k8/amd64/viostor.inf 6da6120368c097806e4ac7fc6b940810 viostor/2k8/amd64/viostor.pdb 63b05edfb75d9dc60e82ab3c0bfee701 viostor/2k8/amd64/viostor.sys af0cca02244fcfedf3209aef56610464 viostor/w7/amd64/viostor.cat defccbe48aca1cd97fee4d7b16053b72 viostor/w7/amd64/viostor.inf 6da6120368c097806e4ac7fc6b940810 viostor/w7/amd64/viostor.pdb 63b05edfb75d9dc60e82ab3c0bfee701 viostor/w7/amd64/viostor.sys
(almost identical, *.cat files are different)
2k8 == vista == 6.0 2k8r2 == w7 == 6.1 You need to compare w7 against 2k8r2. AFAICS they are the same. Roman.
On Wed, Nov 18, 2015 at 09:12:42PM +0300, Roman Kagan wrote:
On Tue, Nov 17, 2015 at 08:49:54PM +0000, Richard W.M. Jones wrote:
There was a copy and paste error. It should have read:
$ md5sum viostor/2k8/amd64/* viostor/w7/amd64/* d1132fbd27c98f25ea71a6eb4037b2f8 viostor/2k8/amd64/viostor.cat defccbe48aca1cd97fee4d7b16053b72 viostor/2k8/amd64/viostor.inf 6da6120368c097806e4ac7fc6b940810 viostor/2k8/amd64/viostor.pdb 63b05edfb75d9dc60e82ab3c0bfee701 viostor/2k8/amd64/viostor.sys af0cca02244fcfedf3209aef56610464 viostor/w7/amd64/viostor.cat defccbe48aca1cd97fee4d7b16053b72 viostor/w7/amd64/viostor.inf 6da6120368c097806e4ac7fc6b940810 viostor/w7/amd64/viostor.pdb 63b05edfb75d9dc60e82ab3c0bfee701 viostor/w7/amd64/viostor.sys
(almost identical, *.cat files are different)
2k8 == vista == 6.0 2k8r2 == w7 == 6.1
You need to compare w7 against 2k8r2. AFAICS they are the same.
Fair enough. Yes, they're the same too: $ md5sum viostor/2k8R2/amd64/* viostor/w7/amd64/* af0cca02244fcfedf3209aef56610464 viostor/2k8R2/amd64/viostor.cat defccbe48aca1cd97fee4d7b16053b72 viostor/2k8R2/amd64/viostor.inf 6da6120368c097806e4ac7fc6b940810 viostor/2k8R2/amd64/viostor.pdb 63b05edfb75d9dc60e82ab3c0bfee701 viostor/2k8R2/amd64/viostor.sys af0cca02244fcfedf3209aef56610464 viostor/w7/amd64/viostor.cat defccbe48aca1cd97fee4d7b16053b72 viostor/w7/amd64/viostor.inf 6da6120368c097806e4ac7fc6b940810 viostor/w7/amd64/viostor.pdb 63b05edfb75d9dc60e82ab3c0bfee701 viostor/w7/amd64/viostor.sys Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org
On Tue, 2015-11-17 at 19:44 +0000, Richard W.M. Jones wrote:
I think one - maybe final? - problem. How can I tell the difference between drivers for "client" versions of Windows (eg. Windows 7) and server versions of Windows (eg. Windows 2008 Server)?
It seems in many or most cases the drivers are identical, eg:
$ md5sum viostor/2k12/amd64/* viostor/w8/amd64/* bbe250c13bf891fd7292ccab9908a63a viostor/2k12/amd64/viostor.cat c2fdfc6a47fdc9dd25242441cdf1a68e viostor/2k12/amd64/viostor.inf 5ca2cadca2214b90196755dab06676f4 viostor/2k12/amd64/viostor.pdb 80c5ea60a69f37e965cb277383abd067 viostor/2k12/amd64/viostor.sys bbe250c13bf891fd7292ccab9908a63a viostor/w8/amd64/viostor.cat c2fdfc6a47fdc9dd25242441cdf1a68e viostor/w8/amd64/viostor.inf 5ca2cadca2214b90196755dab06676f4 viostor/w8/amd64/viostor.pdb 80c5ea60a69f37e965cb277383abd067 viostor/w8/amd64/viostor.sys
$ md5sum Balloon/2k12/amd64/* Balloon/w8/amd64/* d4104f08e674ab9def49d086d960349d Balloon/2k12/amd64/balloon.cat 0325b7e0552e1567292eb6c1068fee18 Balloon/2k12/amd64/balloon.inf ac2116f04fd678d0e264f34e3bf90d1e Balloon/2k12/amd64/balloon.pdb c975501dd5b43ade3717722b81acbc1f Balloon/2k12/amd64/balloon.sys 56f63afdd992a5395a4d8da6a52e559e Balloon/2k12/amd64/blnsvr.exe c539f2056fb196da377d14180fda195c Balloon/2k12/amd64/blnsvr.pdb d10864c1730172780c2d4be633b9220a Balloon/2k12/amd64/WdfCoInstaller01011.dll d4104f08e674ab9def49d086d960349d Balloon/w8/amd64/balloon.cat 0325b7e0552e1567292eb6c1068fee18 Balloon/w8/amd64/balloon.inf ac2116f04fd678d0e264f34e3bf90d1e Balloon/w8/amd64/balloon.pdb c975501dd5b43ade3717722b81acbc1f Balloon/w8/amd64/balloon.sys 56f63afdd992a5395a4d8da6a52e559e Balloon/w8/amd64/blnsvr.exe c539f2056fb196da377d14180fda195c Balloon/w8/amd64/blnsvr.pdb d10864c1730172780c2d4be633b9220a Balloon/w8/amd64/WdfCoInstaller01011.dll
(completely identical)
$ md5sum viostor/2k8/amd64/* viostor/w7/amd64/*d1132fbd27c98f25ea71a6eb4037b2f8 viostor/2k8/amd64/viostor.cat defccbe48aca1cd97fee4d7b16053b72 viostor/2k8/amd64/viostor.inf 6da6120368c097806e4ac7fc6b940810 viostor/2k8/amd64/viostor.pdb 63b05edfb75d9dc60e82ab3c0bfee701 viostor/2k8/amd64/viostor.sys af0cca02244fcfedf3209aef56610464 viostor/w7/amd64/viostor.cat defccbe48aca1cd97fee4d7b16053b72 viostor/w7/amd64/viostor.inf 6da6120368c097806e4ac7fc6b940810 viostor/w7/amd64/viostor.pdb 63b05edfb75d9dc60e82ab3c0bfee701 viostor/w7/amd64/viostor.sys
(almost identical, *.cat files are different)
In any case the *.inf files don't seem to have any distinguishing feature which would allow us to check this.
Maybe this doesn't matter?
Let me explain how it works: We don't make any difference between server or client platforms when building our drivers. Initially, all the files, mentioned above (inf, cat, sys, and pdb), are absolutely the same. Then the files go to QE for WHQL testing. QE tests the same drivers on both, desktop and server platforms, and creates two different submissions files for sending them to WHQL dashboard togather with the original "cat" file. As the result, we have two different cat files created from the same original one. As far as I remember, there shouldn't be any problem to swap these two cat files, and use them interchangeable. But it will be better get some conformation from QE team. Best regards, Vadim.
Rich.
+Li Jin ----- Original Message -----
From: "Vadim Rozenfeld" <vrozenfe@redhat.com> To: "Richard W.M. Jones" <rjones@redhat.com> Cc: "Roman Kagan" <rkagan@virtuozzo.com>, libguestfs@redhat.com, "Amnon Ilan" <ailan@redhat.com>, "Jeff Nelson" <jenelson@redhat.com>, "Yan Vugenfirer" <yvugenfi@redhat.com>, "Denis Lunev" <den@virtuozzo.com> Sent: Wednesday, November 18, 2015 6:00:12 AM Subject: Re: Fwd: [PATCH] v2v: virtio-win: include *.dll too
On Tue, 2015-11-17 at 19:44 +0000, Richard W.M. Jones wrote:
I think one - maybe final? - problem. How can I tell the difference between drivers for "client" versions of Windows (eg. Windows 7) and server versions of Windows (eg. Windows 2008 Server)?
It seems in many or most cases the drivers are identical, eg:
$ md5sum viostor/2k12/amd64/* viostor/w8/amd64/* bbe250c13bf891fd7292ccab9908a63a viostor/2k12/amd64/viostor.cat c2fdfc6a47fdc9dd25242441cdf1a68e viostor/2k12/amd64/viostor.inf 5ca2cadca2214b90196755dab06676f4 viostor/2k12/amd64/viostor.pdb 80c5ea60a69f37e965cb277383abd067 viostor/2k12/amd64/viostor.sys bbe250c13bf891fd7292ccab9908a63a viostor/w8/amd64/viostor.cat c2fdfc6a47fdc9dd25242441cdf1a68e viostor/w8/amd64/viostor.inf 5ca2cadca2214b90196755dab06676f4 viostor/w8/amd64/viostor.pdb 80c5ea60a69f37e965cb277383abd067 viostor/w8/amd64/viostor.sys
$ md5sum Balloon/2k12/amd64/* Balloon/w8/amd64/* d4104f08e674ab9def49d086d960349d Balloon/2k12/amd64/balloon.cat 0325b7e0552e1567292eb6c1068fee18 Balloon/2k12/amd64/balloon.inf ac2116f04fd678d0e264f34e3bf90d1e Balloon/2k12/amd64/balloon.pdb c975501dd5b43ade3717722b81acbc1f Balloon/2k12/amd64/balloon.sys 56f63afdd992a5395a4d8da6a52e559e Balloon/2k12/amd64/blnsvr.exe c539f2056fb196da377d14180fda195c Balloon/2k12/amd64/blnsvr.pdb d10864c1730172780c2d4be633b9220a Balloon/2k12/amd64/WdfCoInstaller01011.dll d4104f08e674ab9def49d086d960349d Balloon/w8/amd64/balloon.cat 0325b7e0552e1567292eb6c1068fee18 Balloon/w8/amd64/balloon.inf ac2116f04fd678d0e264f34e3bf90d1e Balloon/w8/amd64/balloon.pdb c975501dd5b43ade3717722b81acbc1f Balloon/w8/amd64/balloon.sys 56f63afdd992a5395a4d8da6a52e559e Balloon/w8/amd64/blnsvr.exe c539f2056fb196da377d14180fda195c Balloon/w8/amd64/blnsvr.pdb d10864c1730172780c2d4be633b9220a Balloon/w8/amd64/WdfCoInstaller01011.dll
(completely identical)
$ md5sum viostor/2k8/amd64/* viostor/w7/amd64/*d1132fbd27c98f25ea71a6eb4037b2f8 viostor/2k8/amd64/viostor.cat defccbe48aca1cd97fee4d7b16053b72 viostor/2k8/amd64/viostor.inf 6da6120368c097806e4ac7fc6b940810 viostor/2k8/amd64/viostor.pdb 63b05edfb75d9dc60e82ab3c0bfee701 viostor/2k8/amd64/viostor.sys af0cca02244fcfedf3209aef56610464 viostor/w7/amd64/viostor.cat defccbe48aca1cd97fee4d7b16053b72 viostor/w7/amd64/viostor.inf 6da6120368c097806e4ac7fc6b940810 viostor/w7/amd64/viostor.pdb 63b05edfb75d9dc60e82ab3c0bfee701 viostor/w7/amd64/viostor.sys
(almost identical, *.cat files are different)
In any case the *.inf files don't seem to have any distinguishing feature which would allow us to check this.
Maybe this doesn't matter?
Let me explain how it works: We don't make any difference between server or client platforms when building our drivers. Initially, all the files, mentioned above (inf, cat, sys, and pdb), are absolutely the same. Then the files go to QE for WHQL testing. QE tests the same drivers on both, desktop and server platforms, and creates two different submissions files for sending them to WHQL dashboard togather with the original "cat" file. As the result, we have two different cat files created from the same original one.
As far as I remember, there shouldn't be any problem to swap these two cat files, and use them interchangeable. But it will be better get some conformation from QE team.
Li Jin, this is a discussion we have on a public mailing-list regarding the files structure of our virtio-win distribution. Could you confirm Vadim's statement above? Thanks, Amnon
Best regards, Vadim.
Rich.
----- Original Message -----
From: "Amnon Ilan" <ailan@redhat.com> To: "Vadim Rozenfeld" <vrozenfe@redhat.com>, "Li Jin" <lijin@redhat.com> Cc: "Richard W.M. Jones" <rjones@redhat.com>, "Roman Kagan" <rkagan@virtuozzo.com>, libguestfs@redhat.com, "Jeff Nelson" <jenelson@redhat.com>, "Yan Vugenfirer" <yvugenfi@redhat.com>, "Denis Lunev" <den@virtuozzo.com> Sent: Wednesday, November 18, 2015 6:29:11 PM Subject: Re: [PATCH] v2v: virtio-win: include *.dll too
+Li Jin
----- Original Message -----
From: "Vadim Rozenfeld" <vrozenfe@redhat.com> To: "Richard W.M. Jones" <rjones@redhat.com> Cc: "Roman Kagan" <rkagan@virtuozzo.com>, libguestfs@redhat.com, "Amnon Ilan" <ailan@redhat.com>, "Jeff Nelson" <jenelson@redhat.com>, "Yan Vugenfirer" <yvugenfi@redhat.com>, "Denis Lunev" <den@virtuozzo.com> Sent: Wednesday, November 18, 2015 6:00:12 AM Subject: Re: Fwd: [PATCH] v2v: virtio-win: include *.dll too
On Tue, 2015-11-17 at 19:44 +0000, Richard W.M. Jones wrote:
I think one - maybe final? - problem. How can I tell the difference between drivers for "client" versions of Windows (eg. Windows 7) and server versions of Windows (eg. Windows 2008 Server)?
It seems in many or most cases the drivers are identical, eg:
$ md5sum viostor/2k12/amd64/* viostor/w8/amd64/* bbe250c13bf891fd7292ccab9908a63a viostor/2k12/amd64/viostor.cat c2fdfc6a47fdc9dd25242441cdf1a68e viostor/2k12/amd64/viostor.inf 5ca2cadca2214b90196755dab06676f4 viostor/2k12/amd64/viostor.pdb 80c5ea60a69f37e965cb277383abd067 viostor/2k12/amd64/viostor.sys bbe250c13bf891fd7292ccab9908a63a viostor/w8/amd64/viostor.cat c2fdfc6a47fdc9dd25242441cdf1a68e viostor/w8/amd64/viostor.inf 5ca2cadca2214b90196755dab06676f4 viostor/w8/amd64/viostor.pdb 80c5ea60a69f37e965cb277383abd067 viostor/w8/amd64/viostor.sys
$ md5sum Balloon/2k12/amd64/* Balloon/w8/amd64/* d4104f08e674ab9def49d086d960349d Balloon/2k12/amd64/balloon.cat 0325b7e0552e1567292eb6c1068fee18 Balloon/2k12/amd64/balloon.inf ac2116f04fd678d0e264f34e3bf90d1e Balloon/2k12/amd64/balloon.pdb c975501dd5b43ade3717722b81acbc1f Balloon/2k12/amd64/balloon.sys 56f63afdd992a5395a4d8da6a52e559e Balloon/2k12/amd64/blnsvr.exe c539f2056fb196da377d14180fda195c Balloon/2k12/amd64/blnsvr.pdb d10864c1730172780c2d4be633b9220a Balloon/2k12/amd64/WdfCoInstaller01011.dll d4104f08e674ab9def49d086d960349d Balloon/w8/amd64/balloon.cat 0325b7e0552e1567292eb6c1068fee18 Balloon/w8/amd64/balloon.inf ac2116f04fd678d0e264f34e3bf90d1e Balloon/w8/amd64/balloon.pdb c975501dd5b43ade3717722b81acbc1f Balloon/w8/amd64/balloon.sys 56f63afdd992a5395a4d8da6a52e559e Balloon/w8/amd64/blnsvr.exe c539f2056fb196da377d14180fda195c Balloon/w8/amd64/blnsvr.pdb d10864c1730172780c2d4be633b9220a Balloon/w8/amd64/WdfCoInstaller01011.dll
(completely identical)
$ md5sum viostor/2k8/amd64/* viostor/w7/amd64/*d1132fbd27c98f25ea71a6eb4037b2f8 viostor/2k8/amd64/viostor.cat defccbe48aca1cd97fee4d7b16053b72 viostor/2k8/amd64/viostor.inf 6da6120368c097806e4ac7fc6b940810 viostor/2k8/amd64/viostor.pdb 63b05edfb75d9dc60e82ab3c0bfee701 viostor/2k8/amd64/viostor.sys af0cca02244fcfedf3209aef56610464 viostor/w7/amd64/viostor.cat defccbe48aca1cd97fee4d7b16053b72 viostor/w7/amd64/viostor.inf 6da6120368c097806e4ac7fc6b940810 viostor/w7/amd64/viostor.pdb 63b05edfb75d9dc60e82ab3c0bfee701 viostor/w7/amd64/viostor.sys
(almost identical, *.cat files are different)
In any case the *.inf files don't seem to have any distinguishing feature which would allow us to check this.
Maybe this doesn't matter?
Let me explain how it works: We don't make any difference between server or client platforms when building our drivers. Initially, all the files, mentioned above (inf, cat, sys, and pdb), are absolutely the same. Then the files go to QE for WHQL testing. QE tests the same drivers on both, desktop and server platforms, and creates two different submissions files for sending them to WHQL dashboard togather with the original "cat" file. As the result, we have two different cat files created from the same original one.
As far as I remember, there shouldn't be any problem to swap these two cat files, and use them interchangeable. But it will be better get some conformation from QE team.
Li Jin, this is a discussion we have on a public mailing-list regarding the files structure of our virtio-win distribution. Could you confirm Vadim's statement above?
Yes, I agree with Vadim's statement. The *.cat file is a signature file: Before whql testing,the driver is only signed by redhat,when install driver in guest,windows OS will prompt a security warning to confirm if customer want to install a driver not signed by msft. After whql testing and submit to msft,we will get a *.cat file signed by microsoft,which will make driver installation smoothly(no extra warning). The original *.cat file is same,whether the *.cat files are still same afer whql submission depends on whether they are submit in one submission. for example: win8-64 and win2012 *.cat files are same because QE submit them in one submission(win8-64 and win2012 whql test both run on hck,results can be merged into one package); win7-64 and win2008-64 *.cat files are different because we submit them in different submissions(win7-64 whql test run on hck,win2008-64 run wlk,hck and wlk must submit differently according to msft's strategy). I tried swap cat files,drivers still can be installed correctly.
Thanks, Amnon
Best regards, Vadim.
Rich.
On Wed, Nov 18, 2015 at 11:31:17PM -0500, Li Jin wrote:
In any case the *.inf files don't seem to have any distinguishing feature which would allow us to check this.
Maybe this doesn't matter?
Let me explain how it works: We don't make any difference between server or client platforms when building our drivers. Initially, all the files, mentioned above (inf, cat, sys, and pdb), are absolutely the same. Then the files go to QE for WHQL testing. QE tests the same drivers on both, desktop and server platforms, and creates two different submissions files for sending them to WHQL dashboard togather with the original "cat" file. As the result, we have two different cat files created from the same original one.
As far as I remember, there shouldn't be any problem to swap these two cat files, and use them interchangeable. But it will be better get some conformation from QE team.
Li Jin, this is a discussion we have on a public mailing-list regarding the files structure of our virtio-win distribution. Could you confirm Vadim's statement above?
Yes, I agree with Vadim's statement. The *.cat file is a signature file: Before whql testing,the driver is only signed by redhat,when install driver in guest,windows OS will prompt a security warning to confirm if customer want to install a driver not signed by msft. After whql testing and submit to msft,we will get a *.cat file signed by microsoft,which will make driver installation smoothly(no extra warning).
The original *.cat file is same,whether the *.cat files are still same afer whql submission depends on whether they are submit in one submission. for example: win8-64 and win2012 *.cat files are same because QE submit them in one submission(win8-64 and win2012 whql test both run on hck,results can be merged into one package); win7-64 and win2008-64 *.cat files are different because we submit them in different submissions(win7-64 whql test run on hck,win2008-64 run wlk,hck and wlk must submit differently according to msft's strategy).
I tried swap cat files,drivers still can be installed correctly.
I went ahead and dug into this a little, and also talked to a colleague who is involved in WHQL certification of drivers. Here's what I've got (sorry if it's obvious for anyone; I failed to find any description of it with just enough details). Catalog files are pkcs7 containers with multi-layered nested ASN.1 structure; on Linux they are best viewed with with dumpasn1 (which, unlike openssl asn1parse, recognizes most of the Microsoft objects). They contain, in addition to signatures, a number of name:value pairs with properties of the whole package, and a list of entries describing every file in the package, with their own set of name:value pairs representing the properties of those entries. In particular, for every file a filename and an embedded (for PE images: exe, sys, dll) or detached (for other files, e.g. inf) digest is stored, to match the file to the record in the catalog. For every file in the catalog theres an attribute named OSAttr; its value is a string containing comma-separted list of Windows versions, e.g. "2:5.1,2:5.2,2:6.0,2:6.1". Besides, there's a global attribute "OS", containing comma-separated list of Windows versions in a different format, e.g. "XPX64,Server2003X64,VistaX64,Server2008X64,7X64,Server2008R2X64" These are presumably the ones that have to be matched against the version of the Windows the driver is installed into. Both per-file OSAttr and global OS are populated when the catalog is generated by inf2cat tool from Microsoft WDK (https://msdn.microsoft.com/en-us/library/windows/hardware/ff553618(v=vs.85)....), which takes the list of values in its /os command-line parameter. When the package is submitted for WHQL certification, MS regenerates the the catalog with its own signature and leaving only those OSes in OSAttr and OS for which certification is granted, and sends it back to the submitter. So it looks like yes, there's certain data in there which can be used to match it against the exact Windows version; not sure what to do with all this, though. Roman.
On Thu, Nov 19, 2015 at 09:45:36PM +0300, Roman Kagan wrote:
On Wed, Nov 18, 2015 at 11:31:17PM -0500, Li Jin wrote:
In any case the *.inf files don't seem to have any distinguishing feature which would allow us to check this.
Catalog files are pkcs7 containers with multi-layered nested ASN.1 structure [...] So it looks like yes, there's certain data in there which can be used to match it against the exact Windows version; not sure what to do with all this, though.
Having given it some more thought and having discussed that with colleagues who do various things with the drivers, like generating unattended install helpers or just manually installing drivers into guests, I'm now of the opinion that neither libguestfs nor any other user of the drivers should look inside the driver files in order to match them against the guest OS, because it's hard for programs and unrealistic for humans. I tend to believe it should be the responsibility of the driver packages to lay out the drivers in a directory structure that both humans and programs could grok easily and that wouldn't change once established; in a sense this would be both an API and a human interface. If I were to lay it out I'd group the stuff by guest OS, so that for a particular guest a single entity -- a directory tree or a pre-created ISO or floppy image -- would contain all that's needed for this guest and nothing else. This would allow separation of concerns, so that actors who bring the stuff into the guest wouldn't need to care about what's inside, and actors involved with its installation wouldn't need to know how to find the right one. E.g. from this POV the way drivers are stored in RHEL7 virtio-win package on the *filesystem* looks OK, the way they are stored in the *ISO image* in the same package doesn't. As for naming one can stick with what's being used in that virtio-win rpm, i.e. {amd64,i386}/Win{XP,2003,2008,2008R2,2012,2012R2,7,8,8.1} or switch to the names passed to inf2cat in its /os parameter, i.e. XP_X86,Server2003_X86,...,8_X64,Server8_X64,6_3_X86,6_3_X64,Server6_3_X64 (https://msdn.microsoft.com/en-us/library/windows/hardware/ff547089(v=vs.85)....) Whatever convention is chosen it needs to be followed thereafter. The main problem with this proposal is that I don't know how to implement it: there are multiple vendors of virtio-win packages, and, for licensing reasons, they don't even share the source code, so it's unclear who could own this convention and enforce it on others. I'd appreciate any thoughts or suggestions on how to proceed from that and whether it's worth it at all. Thanks, Roman.
On Mon, Nov 23, 2015 at 08:26:10PM +0300, Roman Kagan wrote:
On Thu, Nov 19, 2015 at 09:45:36PM +0300, Roman Kagan wrote:
On Wed, Nov 18, 2015 at 11:31:17PM -0500, Li Jin wrote:
In any case the *.inf files don't seem to have any distinguishing feature which would allow us to check this.
Catalog files are pkcs7 containers with multi-layered nested ASN.1 structure [...] So it looks like yes, there's certain data in there which can be used to match it against the exact Windows version; not sure what to do with all this, though.
Having given it some more thought and having discussed that with colleagues who do various things with the drivers, like generating unattended install helpers or just manually installing drivers into guests, I'm now of the opinion that neither libguestfs nor any other user of the drivers should look inside the driver files in order to match them against the guest OS, because it's hard for programs and unrealistic for humans.
I tend to believe it should be the responsibility of the driver packages to lay out the drivers in a directory structure that both humans and programs could grok easily and that wouldn't change once established; in a sense this would be both an API and a human interface.
If I were to lay it out I'd group the stuff by guest OS, so that for a particular guest a single entity -- a directory tree or a pre-created ISO or floppy image -- would contain all that's needed for this guest and nothing else. This would allow separation of concerns, so that actors who bring the stuff into the guest wouldn't need to care about what's inside, and actors involved with its installation wouldn't need to know how to find the right one.
The good news is that this is what we do right now.
E.g. from this POV the way drivers are stored in RHEL7 virtio-win package on the *filesystem* looks OK, the way they are stored in the *ISO image* in the same package doesn't.
As for naming one can stick with what's being used in that virtio-win rpm, i.e.
{amd64,i386}/Win{XP,2003,2008,2008R2,2012,2012R2,7,8,8.1}
or switch to the names passed to inf2cat in its /os parameter, i.e.
XP_X86,Server2003_X86,...,8_X64,Server8_X64,6_3_X86,6_3_X64,Server6_3_X64 (https://msdn.microsoft.com/en-us/library/windows/hardware/ff547089(v=vs.85)....)
Whatever convention is chosen it needs to be followed thereafter.
Yup - very important that once we decide what the naming convention is going to be, we stick with it. Moving drivers around or using ad-hoc naming schemes causes churn in virt-v2v and probably other projects too.
The main problem with this proposal is that I don't know how to implement it: there are multiple vendors of virtio-win packages, and, for licensing reasons, they don't even share the source code, so it's unclear who could own this convention and enforce it on others.
I guess that virt-v2v will have to deal with this with more special rules.
I'd appreciate any thoughts or suggestions on how to proceed from that and whether it's worth it at all.
Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW
On Tue, Nov 24, 2015 at 01:07:03PM +0000, Richard W.M. Jones wrote:
On Mon, Nov 23, 2015 at 08:26:10PM +0300, Roman Kagan wrote:
Having given it some more thought and having discussed that with colleagues who do various things with the drivers, like generating unattended install helpers or just manually installing drivers into guests, I'm now of the opinion that neither libguestfs nor any other user of the drivers should look inside the driver files in order to match them against the guest OS, because it's hard for programs and unrealistic for humans.
I tend to believe it should be the responsibility of the driver packages to lay out the drivers in a directory structure that both humans and programs could grok easily and that wouldn't change once established; in a sense this would be both an API and a human interface.
If I were to lay it out I'd group the stuff by guest OS, so that for a particular guest a single entity -- a directory tree or a pre-created ISO or floppy image -- would contain all that's needed for this guest and nothing else. This would allow separation of concerns, so that actors who bring the stuff into the guest wouldn't need to care about what's inside, and actors involved with its installation wouldn't need to know how to find the right one.
The good news is that this is what we do right now.
E.g. from this POV the way drivers are stored in RHEL7 virtio-win package on the *filesystem* looks OK, the way they are stored in the *ISO image* in the same package doesn't.
As for naming one can stick with what's being used in that virtio-win rpm, i.e.
{amd64,i386}/Win{XP,2003,2008,2008R2,2012,2012R2,7,8,8.1}
or switch to the names passed to inf2cat in its /os parameter, i.e.
XP_X86,Server2003_X86,...,8_X64,Server8_X64,6_3_X86,6_3_X64,Server6_3_X64 (https://msdn.microsoft.com/en-us/library/windows/hardware/ff547089(v=vs.85)....)
Whatever convention is chosen it needs to be followed thereafter.
Yup - very important that once we decide what the naming convention is going to be, we stick with it. Moving drivers around or using ad-hoc naming schemes causes churn in virt-v2v and probably other projects too.
The main problem with this proposal is that I don't know how to implement it: there are multiple vendors of virtio-win packages, and, for licensing reasons, they don't even share the source code, so it's unclear who could own this convention and enforce it on others.
I guess that virt-v2v will have to deal with this with more special rules.
I'd appreciate any thoughts or suggestions on how to proceed from that and whether it's worth it at all.
Apparently I failed to inspire anybody who can affect the driver packaging layout :) Let me try again. Do I get it right that the way virtio-win is packaged for Fedora and RHEL is driven by the scripts at https://github.com/crobinso/virtio-win-pkg-scripts and I should be submitting patches or pull requests to it? Or should the layout be determined only by the build machinery at https://github.com/YanVugenfirer/kvm-guest-drivers-windows and I should be direct my submissions that way? Or any combination of the above? Also does anybody know this is done in SUSE? Any help would be appreciated. Thanks, Roman.
On Thu, Nov 26, 2015 at 08:12:41PM +0300, Roman Kagan wrote:
On Tue, Nov 24, 2015 at 01:07:03PM +0000, Richard W.M. Jones wrote:
On Mon, Nov 23, 2015 at 08:26:10PM +0300, Roman Kagan wrote:
Having given it some more thought and having discussed that with colleagues who do various things with the drivers, like generating unattended install helpers or just manually installing drivers into guests, I'm now of the opinion that neither libguestfs nor any other user of the drivers should look inside the driver files in order to match them against the guest OS, because it's hard for programs and unrealistic for humans.
I tend to believe it should be the responsibility of the driver packages to lay out the drivers in a directory structure that both humans and programs could grok easily and that wouldn't change once established; in a sense this would be both an API and a human interface.
If I were to lay it out I'd group the stuff by guest OS, so that for a particular guest a single entity -- a directory tree or a pre-created ISO or floppy image -- would contain all that's needed for this guest and nothing else. This would allow separation of concerns, so that actors who bring the stuff into the guest wouldn't need to care about what's inside, and actors involved with its installation wouldn't need to know how to find the right one.
The good news is that this is what we do right now.
E.g. from this POV the way drivers are stored in RHEL7 virtio-win package on the *filesystem* looks OK, the way they are stored in the *ISO image* in the same package doesn't.
As for naming one can stick with what's being used in that virtio-win rpm, i.e.
{amd64,i386}/Win{XP,2003,2008,2008R2,2012,2012R2,7,8,8.1}
or switch to the names passed to inf2cat in its /os parameter, i.e.
XP_X86,Server2003_X86,...,8_X64,Server8_X64,6_3_X86,6_3_X64,Server6_3_X64 (https://msdn.microsoft.com/en-us/library/windows/hardware/ff547089(v=vs.85)....)
Whatever convention is chosen it needs to be followed thereafter.
Yup - very important that once we decide what the naming convention is going to be, we stick with it. Moving drivers around or using ad-hoc naming schemes causes churn in virt-v2v and probably other projects too.
The main problem with this proposal is that I don't know how to implement it: there are multiple vendors of virtio-win packages, and, for licensing reasons, they don't even share the source code, so it's unclear who could own this convention and enforce it on others.
I guess that virt-v2v will have to deal with this with more special rules.
I'd appreciate any thoughts or suggestions on how to proceed from that and whether it's worth it at all.
Apparently I failed to inspire anybody who can affect the driver packaging layout :) Let me try again.
Do I get it right that the way virtio-win is packaged for Fedora and RHEL is driven by the scripts at
https://github.com/crobinso/virtio-win-pkg-scripts
and I should be submitting patches or pull requests to it?
Yes, that's what I use to package the drivers for RHEL.
Or should the layout be determined only by the build machinery at
https://github.com/YanVugenfirer/kvm-guest-drivers-windows
and I should be direct my submissions that way?
I don't know, but if you work strictly with virtio-win-pkg-scripts it shouldn't matter.
Or any combination of the above?
Also does anybody know this is done in SUSE?
No idea.
Any help would be appreciated.
Thanks, Roman.
-Jeff
On Thu, Nov 26, 2015 at 06:29:45PM -0600, Jeff Nelson wrote:
On Thu, Nov 26, 2015 at 08:12:41PM +0300, Roman Kagan wrote:
Do I get it right that the way virtio-win is packaged for Fedora and RHEL is driven by the scripts at
https://github.com/crobinso/virtio-win-pkg-scripts
and I should be submitting patches or pull requests to it?
Yes, that's what I use to package the drivers for RHEL.
Or should the layout be determined only by the build machinery at
https://github.com/YanVugenfirer/kvm-guest-drivers-windows
and I should be direct my submissions that way?
I don't know, but if you work strictly with virtio-win-pkg-scripts it shouldn't matter.
Thanks a lot, I'll look into it early next week. I guess there's no mailing list, so I'll have to do pull requests on github, right? Thanks, Roman.
On Fri, Nov 27, 2015 at 08:16:46PM +0300, Roman Kagan wrote:
On Thu, Nov 26, 2015 at 06:29:45PM -0600, Jeff Nelson wrote:
On Thu, Nov 26, 2015 at 08:12:41PM +0300, Roman Kagan wrote:
Do I get it right that the way virtio-win is packaged for Fedora and RHEL is driven by the scripts at
https://github.com/crobinso/virtio-win-pkg-scripts
and I should be submitting patches or pull requests to it?
Yes, that's what I use to package the drivers for RHEL.
Or should the layout be determined only by the build machinery at
https://github.com/YanVugenfirer/kvm-guest-drivers-windows
and I should be direct my submissions that way?
I don't know, but if you work strictly with virtio-win-pkg-scripts it shouldn't matter.
Thanks a lot, I'll look into it early next week.
I guess there's no mailing list, so I'll have to do pull requests on github, right?
I guess for general discussion, the virt-tools mailing list is the right one, but use pull requests on github for patches I suppose. https://www.redhat.com/mailman/listinfo/virt-tools-list Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top
On 11/27/2015 12:16 PM, Roman Kagan wrote:
On Thu, Nov 26, 2015 at 06:29:45PM -0600, Jeff Nelson wrote:
On Thu, Nov 26, 2015 at 08:12:41PM +0300, Roman Kagan wrote:
Do I get it right that the way virtio-win is packaged for Fedora and RHEL is driven by the scripts at
https://github.com/crobinso/virtio-win-pkg-scripts
and I should be submitting patches or pull requests to it?
Yes, that's what I use to package the drivers for RHEL.
Or should the layout be determined only by the build machinery at
https://github.com/YanVugenfirer/kvm-guest-drivers-windows
and I should be direct my submissions that way?
I don't know, but if you work strictly with virtio-win-pkg-scripts it shouldn't matter.
Thanks a lot, I'll look into it early next week.
Sorry I'm only getting to this thread now :/ I'd like to try and summarize the interrelated packaging changes that were mentioned in this thread: - The iso layout is not optimal for programmatic consumption - The iso layout is not compatible with windows driver media autodetect conventions - The iso and RPM host contents do not match - The reasoning behind the iso/RPM layout is not clear To all this I say... preach, my brother :) https://bugzilla.redhat.com/show_bug.cgi?id=1217658 https://bugzilla.redhat.com/show_bug.cgi?id=1167941 The ISO and RPM layout is largely the result of organic extensions and no one ever tried to streamline things, so I fully support any work in that direction. But the things to consider: - There are many consumers of the iso layout as it is right now. Any changes in the short to medium term need to be entirely additive so we can get the new layout right without breaking every consumer. So keep the existing iso layout of $drivername/$weird-os/$arch, but _add_ the $arch/$standard-os/* layout, and avoid duplicates by hardlinking identical files (the scripts on github already have code that handles the last bit). In fact it might be a good starting point to write a script that takes the existing iso, extracts it, and rearranges the contents to match the new format. Then I could help integrate that with the scripts and my own testing. - The layout should use the latest windows conventions for arch/osname. If this means we have to diverge from the naming convention we currently use for RPM host installed drivers, that's fine, since we can maintain back compat there with symlinks. However I've tried searching for this info (googling 'windows driver cd layout/hierarchy/format') but I can't find anything definitive... anyone have a pointer? Especially WRT the current standard vs. the pre-windows-7 standard that vadim pointed out. - Anything regarding a separate iso with a separate layout to appease < win7 windows vintage should be handled separately, if at all IMO
I guess there's no mailing list, so I'll have to do pull requests on github, right?
Let's stick with email and virt-tools-list@redhat.com since there may be lots of discussion... Thanks, Cole
On Mon, 2015-11-30 at 17:56 -0500, Cole Robinson wrote:
On 11/27/2015 12:16 PM, Roman Kagan wrote:
On Thu, Nov 26, 2015 at 06:29:45PM -0600, Jeff Nelson wrote:
On Thu, Nov 26, 2015 at 08:12:41PM +0300, Roman Kagan wrote:
Do I get it right that the way virtio-win is packaged for Fedora and RHEL is driven by the scripts at
https://github.com/crobinso/virtio-win-pkg-scripts
and I should be submitting patches or pull requests to it?
Yes, that's what I use to package the drivers for RHEL.
Or should the layout be determined only by the build machinery at
https://github.com/YanVugenfirer/kvm-guest-drivers-windows
and I should be direct my submissions that way?
I don't know, but if you work strictly with virtio-win-pkg-scripts it shouldn't matter.
Thanks a lot, I'll look into it early next week.
Sorry I'm only getting to this thread now :/
I'd like to try and summarize the interrelated packaging changes that were mentioned in this thread:
- The iso layout is not optimal for programmatic consumption - The iso layout is not compatible with windows driver media autodetect conventions - The iso and RPM host contents do not match - The reasoning behind the iso/RPM layout is not clear
To all this I say... preach, my brother :)
https://bugzilla.redhat.com/show_bug.cgi?id=1217658 https://bugzilla.redhat.com/show_bug.cgi?id=1167941
The ISO and RPM layout is largely the result of organic extensions and no one ever tried to streamline things, so I fully support any work in that direction. But the things to consider:
- There are many consumers of the iso layout as it is right now. Any changes in the short to medium term need to be entirely additive so we can get the new layout right without breaking every consumer. So keep the existing iso layout of $drivername/$weird-os/$arch, but _add_ the $arch/$standard-os/* layout, and avoid duplicates by hardlinking identical files (the scripts on github already have code that handles the last bit). In fact it might be a good starting point to write a script that takes the existing iso, extracts it, and rearranges the contents to match the new format. Then I could help integrate that with the scripts and my own testing.
- The layout should use the latest windows conventions for arch/osname. If this means we have to diverge from the naming convention we currently use for RPM host installed drivers, that's fine, since we can maintain back compat there with symlinks. However I've tried searching for this info (googling 'windows driver cd layout/hierarchy/format') but I can't find anything definitive... anyone have a pointer? Especially WRT the current standard vs. the pre-windows-7 standard that vadim pointed out.
I don't remember all details, but there are some traces left in bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=708930 https://bugzilla.redhat.com/show_bug.cgi?id=700118#c31 Vadim.
- Anything regarding a separate iso with a separate layout to appease < win7 windows vintage should be handled separately, if at all IMO
I guess there's no mailing list, so I'll have to do pull requests on github, right?
Let's stick with email and virt-tools-list@redhat.com since there may be lots of discussion...
Thanks, Cole
On Mon, Nov 30, 2015 at 05:56:48PM -0500, Cole Robinson wrote:
- There are many consumers of the iso layout as it is right now. Any changes in the short to medium term need to be entirely additive so we can get the new layout right without breaking every consumer. So keep the existing iso layout of $drivername/$weird-os/$arch, but _add_ the $arch/$standard-os/* layout, and avoid duplicates by hardlinking identical files (the scripts on github already have code that handles the last bit).
May also be worth asking who the current consumers are. Virt-v2v is one of them. Are there others? (GNOME Boxes?) Virt-v2v uses a very simple scheme where it looks at each path element for magic strings (eg. "/win2003/" or "/x86/") and from that it works out what Windows OS + arch the drivers in that directory are intended for. It's flexible enough that we can easily change it, but only if we're told what the new convention will be. https://github.com/libguestfs/libguestfs/blob/master/v2v/windows.ml#L132-L16... Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-df lists disk usage of guests without needing to install any software inside the virtual machine. Supports Linux and Windows. http://people.redhat.com/~rjones/virt-df/
On 12/01/2015 04:49 AM, Richard W.M. Jones wrote:
On Mon, Nov 30, 2015 at 05:56:48PM -0500, Cole Robinson wrote:
- There are many consumers of the iso layout as it is right now. Any changes in the short to medium term need to be entirely additive so we can get the new layout right without breaking every consumer. So keep the existing iso layout of $drivername/$weird-os/$arch, but _add_ the $arch/$standard-os/* layout, and avoid duplicates by hardlinking identical files (the scripts on github already have code that handles the last bit).
May also be worth asking who the current consumers are.
Virt-v2v is one of them.
Are there others? (GNOME Boxes?)
* ovirt, as part of building their own iso and guest installer * spice-guest-tools.exe builds * boxes/libosinfo get drivers from https://zeenix.fedorapeople.org/drivers/win-tools/preinst/ which is just a copy from a tested virtio-win release, but they control the filesystem layout. Explicitly documenting this is one of the things on my todo list. But there may be far more... who knows what RHEL customers are doing with the drivers. - Cole
On Mon, Nov 30, 2015 at 05:56:48PM -0500, Cole Robinson wrote:
This one is restricted access so I can't see it. The links and suggestions are very appreciated, thanks.
I guess there's no mailing list, so I'll have to do pull requests on github, right?
Let's stick with email and virt-tools-list@redhat.com since there may be lots of discussion...
OK. Then I'll start posting patches there once I have something ready. I guess everybody interested is subscribed; if not please speak up so that I cc you. Thanks, Roman.
On Tue, Dec 01, 2015 at 03:42:05PM +0300, Roman Kagan wrote:
On Mon, Nov 30, 2015 at 05:56:48PM -0500, Cole Robinson wrote:
This one is restricted access so I can't see it.
I've opened it up - try again now. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org
+Lev (oVirt guest installer) ----- Original Message -----
From: "Cole Robinson" <crobinso@redhat.com> To: "Roman Kagan" <rkagan@virtuozzo.com>, "Jeff Nelson" <jenelson@redhat.com>, "Richard W.M. Jones" <rjones@redhat.com>, "Denis Lunev" <den@virtuozzo.com>, "Yan Vugenfirer" <yvugenfi@redhat.com>, "Vadim Rozenfeld" <vrozenfe@redhat.com>, michen@redhat.com, "Amnon Ilan" <ailan@redhat.com>, "Jin" <lijin@redhat.com>, libguestfs@redhat.com, "Alexander Burluka" <aburluka@virtuozzo.com> Sent: Tuesday, December 1, 2015 12:56:48 AM Subject: Re: packaging virtio-win
On 11/27/2015 12:16 PM, Roman Kagan wrote:
On Thu, Nov 26, 2015 at 06:29:45PM -0600, Jeff Nelson wrote:
On Thu, Nov 26, 2015 at 08:12:41PM +0300, Roman Kagan wrote:
Do I get it right that the way virtio-win is packaged for Fedora and RHEL is driven by the scripts at
https://github.com/crobinso/virtio-win-pkg-scripts
and I should be submitting patches or pull requests to it?
Yes, that's what I use to package the drivers for RHEL.
Or should the layout be determined only by the build machinery at
https://github.com/YanVugenfirer/kvm-guest-drivers-windows
and I should be direct my submissions that way?
I don't know, but if you work strictly with virtio-win-pkg-scripts it shouldn't matter.
Thanks a lot, I'll look into it early next week.
Sorry I'm only getting to this thread now :/
I'd like to try and summarize the interrelated packaging changes that were mentioned in this thread:
- The iso layout is not optimal for programmatic consumption - The iso layout is not compatible with windows driver media autodetect conventions - The iso and RPM host contents do not match - The reasoning behind the iso/RPM layout is not clear
To all this I say... preach, my brother :)
https://bugzilla.redhat.com/show_bug.cgi?id=1217658 https://bugzilla.redhat.com/show_bug.cgi?id=1167941
The ISO and RPM layout is largely the result of organic extensions and no one ever tried to streamline things, so I fully support any work in that direction. But the things to consider:
- There are many consumers of the iso layout as it is right now. Any changes in the short to medium term need to be entirely additive so we can get the new layout right without breaking every consumer. So keep the existing iso layout of $drivername/$weird-os/$arch, but _add_ the $arch/$standard-os/* layout, and avoid duplicates by hardlinking identical files (the scripts on github already have code that handles the last bit). In fact it might be a good starting point to write a script that takes the existing iso, extracts it, and rearranges the contents to match the new format. Then I could help integrate that with the scripts and my own testing.
- The layout should use the latest windows conventions for arch/osname. If this means we have to diverge from the naming convention we currently use for RPM host installed drivers, that's fine, since we can maintain back compat there with symlinks. However I've tried searching for this info (googling 'windows driver cd layout/hierarchy/format') but I can't find anything definitive... anyone have a pointer? Especially WRT the current standard vs. the pre-windows-7 standard that vadim pointed out.
- Anything regarding a separate iso with a separate layout to appease < win7 windows vintage should be handled separately, if at all IMO
I guess there's no mailing list, so I'll have to do pull requests on github, right?
Let's stick with email and virt-tools-list@redhat.com since there may be lots of discussion...
Thanks, Cole
participants (8)
-
Amnon Ilan -
Cole Robinson -
Jeff Nelson -
Li Jin -
Richard W.M. Jones -
Roman Kagan -
Vadim Rozenfeld -
Yan Vugenfirer