On 28 Oct 2015, at 13:53, Roman Kagan <rkagan(a)virtuozzo.com&gt; wrote: On Wed, Oct 28, 2015 at 12:10:19PM +1100, Vadim Rozenfeld wrote: >>>> To sum up, the packaging and naming policy of the virtio-win rpm and the >>>> virtio-win iso therein are different and neither is clear. Hardcoding >>>> the policy in v2v without actually knowing it appears risky at best. > > It's due to historical reasons mostly. The best way would be having a set of separate > distribution images packaged on per-platform base. Let me try to get the libguestfs requirements straight: given a set of Windows drivers, it should be able to identify the ones appropriate for the particular Windows flavor, in order to 1) tell which devices can be configured 2) offline-"install" the storage driver and thus enable the guest too boot 3) copy over the matching drivers into the guest and allow it to pick them up on the first boot Obviously virtio-win driver packaging and libguestfs must agree on how to deal with this. Could you please provide any guidance on how to address this problem?

Thanks, Roman.

> On 28 Oct 2015, at 13:53, Roman Kagan <rkagan(a)virtuozzo.com&gt; wrote: > > On Wed, Oct 28, 2015 at 12:10:19PM +1100, Vadim Rozenfeld wrote: >>>>> To sum up, the packaging and naming policy of the virtio-win rpm and the >>>>> virtio-win iso therein are different and neither is clear. Hardcoding >>>>> the policy in v2v without actually knowing it appears risky at best. >> >> It's due to historical reasons mostly. The best way would be having a set of separate >> distribution images packaged on per-platform base. > > Let me try to get the libguestfs requirements straight: > > given a set of Windows drivers, it should be able to identify the ones > appropriate for the particular Windows flavor, in order to > > 1) tell which devices can be configured > > 2) offline-"install" the storage driver and thus enable the guest too > boot > > 3) copy over the matching drivers into the guest and allow it to pick > them up on the first boot > > > Obviously virtio-win driver packaging and libguestfs must agree on how > to deal with this. > > Could you please provide any guidance on how to address this problem? As Vadim said: "The best way would be having a set of separate distribution images packaged on per-platform base.”

Otherwise you will have to maintain the knowledge of binary compatibility between Windows platforms, which is different according to the driver type.

BZ 1223668 suggests reorganizing the directory structure to allow for automatic driver discovery. This would change the iso directory structure from: /<driver>/<os>/<arch>/<file> to: /<arch>/<os>/<file> which is the same structure used in /usr/share/virtio-win/drivers and in the vfd files. Would this work?

On Wed, Oct 28, 2015 at 12:10:19PM +1100, Vadim Rozenfeld wrote: > > > > To sum up, the packaging and naming policy of the virtio-win rpm and the > > > > virtio-win iso therein are different and neither is clear. Hardcoding > > > > the policy in v2v without actually knowing it appears risky at best. > > It's due to historical reasons mostly. The best way would be having a set of separate > distribution images packaged on per-platform base. Let me try to get the libguestfs requirements straight: given a set of Windows drivers, it should be able to identify the ones appropriate for the particular Windows flavor, in order to 1) tell which devices can be configured

2) offline-"install" the storage driver and thus enable the guest too boot

3) copy over the matching drivers into the guest and allow it to pick them up on the first boot.

Obviously virtio-win driver packaging and libguestfs must agree on how to deal with this. Could you please provide any guidance on how to address this problem? Thanks, Roman.

On 29 Oct 2015, at 10:28, Richard W.M. Jones <rjones(a)redhat.com&gt; wrote: On Thu, Oct 29, 2015 at 11:05:42AM +1100, Vadim Rozenfeld wrote: >> 1) tell which devices can be configured > Not sure that I fully understated your question. but if you are going to > create some sort of off-line automatic virtio-win drivers update > utility, then it shouldn't be too complicated. Roman is talking about what virt-v2v already does. This is good information: > First of all you will need > to obtain the Windows kernel version by reading the following Registry > key - "HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion". Let's say it > 6.3, which means that it is Win8.1 or WS2012R2, parsing "BuildLabEx" > string from the same hive will give you information about the platform > bitness. Next you need to go through inf files, and find "DriverVer" > string, like this one . taken from from vioscsi.inf > > DriverVer=08/01/2015,62.72.104.10800 > > This string contains build time and version stamps. The version stamp > looks as follow "62.72.104.10800" > where 62 means a target Windows kernel version multiplied by 10. In this > case it is 6.2 which means Win8 or WS2012 > 72 - the target host platform version multiplied bu 10 (was RHEL7.2) > 104 just a magic number, but it can be changed, don't make any > assumption based on this number. > 10800 our internal build number (build 108) multiplied by 100 > If you found an inf file with the matching minor OS (6 in our case) > version and matching or less but close minor version number (2 vs 3) > then you are in the right directory. Seems like looking at this field in the .inf file is better than our current approach of matching path names.

Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW

On 29 Oct 2015, at 11:53, Roman Kagan <rkagan(a)virtuozzo.com&gt; wrote: On Thu, Oct 29, 2015 at 10:33:57AM +0200, Yan Vugenfirer wrote: >> On 29 Oct 2015, at 10:28, Richard W.M. Jones <rjones(a)redhat.com&gt; wrote: >> On Thu, Oct 29, 2015 at 11:05:42AM +1100, Vadim Rozenfeld wrote: >> >> Seems like looking at this field in the .inf file is better than our >> current approach of matching path names. > > You just need to remember that we have binary compatible versions of > drivers. For example virtio-net driver for Windows 2012 (Windows 8) > (6.2) will be used for Windows 2012R2 (Windows 8.1) and Windows Server > 2016 (Windows 10). So you are getting the lowest compatible version of > the OS from INF file. That seems to match Vadim's description: >>> If you found an inf file with the matching minor OS (6 in our case) >>> version and matching or less but close minor version number (2 vs 3) >>> then you are in the right directory. and it's OK as it's an algorithm that can be coded and made to work reliably. However, do I get it right that this is just a convention currently followed in virtio-win drivers? I.e. there's nothing in the Windows driver specifications mandating this versioning scheme? I'm trying to figure out how dependable it is.

Besides, ATM there's at least one driver -- qemupciserial -- which doesn't fit in this scheme (or any scheme at all :).

There are other peculiarities which we don't know what to do: - the netkvm driver directory on virtio-win.iso contains netkvmco.dll (I'm assuming it's coinstaller dll). However, netkvm.inf doesn't mention it, and the driver installs just fine without it.

- the balloon driver directory on virtio-win.iso contains blnsvr.exe (and .pdb) which is apparently a usermode service to report memory stats to host via balloon virtio queue. However nothing seems to automatically trigger installation of that service. Thanks, Roman.

First of all you will need to obtain the Windows kernel version by reading the following Registry key - "HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion". Let's say it 6.3, which means that it is Win8.1 or WS2012R2,

parsing "BuildLabEx" string from the same hive will give you information about the platform bitness.

From Windows 7:

Next you need to go through inf files, and find "DriverVer" string, like this one . taken from from vioscsi.inf DriverVer=08/01/2015,62.72.104.10800 This string contains build time and version stamps. The version stamp looks as follow "62.72.104.10800" where 62 means a target Windows kernel version multiplied by 10. In this case it is 6.2 which means Win8 or WS2012 72 - the target host platform version multiplied bu 10 (was RHEL7.2) 104 just a magic number, but it can be changed, don't make any assumption based on this number. 10800 our internal build number (build 108) multiplied by 100 If you found an inf file with the matching minor OS (6 in our case) version and matching or less but close minor version number (2 vs 3) then you are in the right directory.

On Thu, Oct 29, 2015 at 11:05:42AM +1100, Vadim Rozenfeld wrote: > On Wed, 2015-10-28 at 14:53 +0300, Roman Kagan wrote: > > 1) tell which devices can be configured > Not sure that I fully understated your question. but if you are going to > create some sort of off-line automatic virtio-win drivers update > utility, then it shouldn't be too complicated. Firs of all you will need > to obtain the Windows kernel version by reading the following Registry > key - "HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion". Let's say it > 6.3, which means that it is Win8.1 or WS2012R2, parsing "BuildLabEx" > string from the same hive will give you information about the platform > bitness. Next you need to go through inf files, and find "DriverVer" > string, like this one . taken from from vioscsi.inf > > DriverVer=08/01/2015,62.72.104.10800 > > This string contains build time and version stamps. The version stamp > looks as follow "62.72.104.10800" > where 62 means a target Windows kernel version multiplied by 10. In this > case it is 6.2 which means Win8 or WS2012 > 72 - the target host platform version multiplied bu 10 (was RHEL7.2) > 104 just a magic number, but it can be changed, don't make any > assumption based on this number. > 10800 our internal build number (build 108) multiplied by 100 Hi Vadim, I've almost finished implementing this, but I have a couple of technical questions: (1) How can we tell from the .inf file what architecture (x86 or amd64) the driver is for? There is a [Manufacturer] section which looks hopeful, eg: [Manufacturer] %RHEL% = RHEL,NTAMD64 but the contents are not very consistent across all the .inf files I have access to.

(2) If we have a directory containing multiple drivers, can we assume that if any .inf file found matches the guest OS, then all those drivers can be installed? So far this appears to be true.

If not then we have to start associating drivers with .inf files, which seems a bit complicated. (3) You write: > If you found an inf file with the matching minor OS (6 in our case) > version and matching or less but close minor version number (2 vs 3) > then you are in the right directory. When you say "close minor version number" is there a particular definition you have in mind? eg. minor or minor-1 is OK? Or do we have to consider all .inf files in totality and choose the nearest one?

Rich.

> On 06 Nov 2015, at 08:51, Vadim Rozenfeld <vrozenfe(a)redhat.com&gt; wrote: > > On Thu, 2015-11-05 at 21:46 +0000, Richard W.M. Jones wrote: >> On Thu, Oct 29, 2015 at 11:05:42AM +1100, Vadim Rozenfeld wrote: >>> On Wed, 2015-10-28 at 14:53 +0300, Roman Kagan wrote: >>>> 1) tell which devices can be configured >>> Not sure that I fully understated your question. but if you are going to >>> create some sort of off-line automatic virtio-win drivers update >>> utility, then it shouldn't be too complicated. Firs of all you will need >>> to obtain the Windows kernel version by reading the following Registry >>> key - "HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion". Let's say it >>> 6.3, which means that it is Win8.1 or WS2012R2, parsing "BuildLabEx" >>> string from the same hive will give you information about the platform >>> bitness. Next you need to go through inf files, and find "DriverVer" >>> string, like this one . taken from from vioscsi.inf >>> >>> DriverVer=08/01/2015,62.72.104.10800 >>> >>> This string contains build time and version stamps. The version stamp >>> looks as follow "62.72.104.10800" >>> where 62 means a target Windows kernel version multiplied by 10. In this >>> case it is 6.2 which means Win8 or WS2012 >>> 72 - the target host platform version multiplied bu 10 (was RHEL7.2) >>> 104 just a magic number, but it can be changed, don't make any >>> assumption based on this number. >>> 10800 our internal build number (build 108) multiplied by 100 >> >> Hi Vadim, >> >> I've almost finished implementing this, but I have a couple of >> technical questions: >> >> (1) How can we tell from the .inf file what architecture (x86 or >> amd64) the driver is for? There is a [Manufacturer] section which >> looks hopeful, eg: >> >> [Manufacturer] >> %RHEL% = RHEL,NTAMD64 >> >> but the contents are not very consistent across all the .inf files I >> have access to. > > Hi Richard, > > It should be consistent. We do specify target architecture (amd64 or > x86) when stamping inf/inx files. IIRC, qemupciserial.inf is the only > exception, but it goes without binaries. In any case, if you are going > to use existing distribution media, you can rely on the media layout - > for example, if "amd64" is a part of your current path - you are in > 64-bit directory. >> >> (2) If we have a directory containing multiple drivers, can we assume >> that if any .inf file found matches the guest OS, then all those >> drivers can be installed? So far this appears to be true. > Yes, we will never mix 32 and 64 bit drivers into the same directory. > I swear:) > >> If not >> then we have to start associating drivers with .inf files, which seems >> a bit complicated. >> >> (3) You write: >> >>> If you found an inf file with the matching minor OS (6 in our case) >>> version and matching or less but close minor version number (2 vs 3) >>> then you are in the right directory. >> >> When you say "close minor version number" is there a particular >> definition you have in mind? eg. minor or minor-1 is OK? Or do we >> have to consider all .inf files in totality and choose the nearest >> one? > Ideally, we should try finding the exact match. If not, we have to find > one with the same major version number and nearest minor number which is > smaller than the target one. For example, let's say we are looking for a > driver for Win8.1/WS2012R2, we know that both of them have kernel > version 6.3 . So, ideally we should to find inf file with the following > driver version string: > > DriverVer=XX/YY/ZZZZ,63.AA.BBB.CCCCC > > if we cannot find an inf file with such criteria, we should try finding > another one with the OS version 62 (which means that a driver was build > for kernel 6.2 Win8/WS2012 and technically should be compatible with > 6.3). So, we are looking for DriverVer which looks like this one: > DriverVer=XX/YY/ZZZZ,62.AA.BBB.CCCCC > > Still no luck - decrease the target OS version to 61 and repeat. > > For WDF drivers like balloon, vioserial, and rng you can go as far as > you can trying to find an appropriate driver. > > For miniport drivers like netkvm ( Yan, please confirm ), viostor and > vioscsi I wouldn't go too far. For example drivers designed to fork in > specific stacks at kernel version 5.1 (WinXP) can refuse working with > kernel 6.2 (Win8). While NDIS miniport designed for NDIS 5.1 (XP) still can run on latest Windows, I wouldn’t do that. For NDIS 6.x (Windows 2008 and newer), we can use older drivers on new OS. But installing a driver for a newer OS on older OS might lead to driver miss-behaviour.

On Mon, 2015-11-09 at 18:08 +0300, Roman Kagan wrote: > On Sun, Nov 08, 2015 at 10:15:31AM +0200, Yan Vugenfirer wrote: > > > > > On 06 Nov 2015, at 08:51, Vadim Rozenfeld <vrozenfe(a)redhat.com&gt; wrote: > > > > > > On Thu, 2015-11-05 at 21:46 +0000, Richard W.M. Jones wrote: > > >> On Thu, Oct 29, 2015 at 11:05:42AM +1100, Vadim Rozenfeld wrote: > > >>> On Wed, 2015-10-28 at 14:53 +0300, Roman Kagan wrote: > > >>>> 1) tell which devices can be configured > > >>> Not sure that I fully understated your question. but if you are going to > > >>> create some sort of off-line automatic virtio-win drivers update > > >>> utility, then it shouldn't be too complicated. Firs of all you will need > > >>> to obtain the Windows kernel version by reading the following Registry > > >>> key - "HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion". Let's say it > > >>> 6.3, which means that it is Win8.1 or WS2012R2, parsing "BuildLabEx" > > >>> string from the same hive will give you information about the platform > > >>> bitness. Next you need to go through inf files, and find "DriverVer" > > >>> string, like this one . taken from from vioscsi.inf > > >>> > > >>> DriverVer=08/01/2015,62.72.104.10800 > > >>> > > >>> This string contains build time and version stamps. The version stamp > > >>> looks as follow "62.72.104.10800" > > >>> where 62 means a target Windows kernel version multiplied by 10. In this > > >>> case it is 6.2 which means Win8 or WS2012 > > >>> 72 - the target host platform version multiplied bu 10 (was RHEL7.2) > > >>> 104 just a magic number, but it can be changed, don't make any > > >>> assumption based on this number. > > >>> 10800 our internal build number (build 108) multiplied by 100 > > >> > > >> Hi Vadim, > > >> > > >> I've almost finished implementing this, but I have a couple of > > >> technical questions: > > >> > > >> (1) How can we tell from the .inf file what architecture (x86 or > > >> amd64) the driver is for? There is a [Manufacturer] section which > > >> looks hopeful, eg: > > >> > > >> [Manufacturer] > > >> %RHEL% = RHEL,NTAMD64 > > >> > > >> but the contents are not very consistent across all the .inf files I > > >> have access to. > > > > > > Hi Richard, > > > > > > It should be consistent. We do specify target architecture (amd64 or > > > x86) when stamping inf/inx files. IIRC, qemupciserial.inf is the only > > > exception, but it goes without binaries. In any case, if you are going > > > to use existing distribution media, you can rely on the media layout - > > > for example, if "amd64" is a part of your current path - you are in > > > 64-bit directory. > > >> > > >> (2) If we have a directory containing multiple drivers, can we assume > > >> that if any .inf file found matches the guest OS, then all those > > >> drivers can be installed? So far this appears to be true. > > > Yes, we will never mix 32 and 64 bit drivers into the same directory. > > > I swear:) > > > > > >> If not > > >> then we have to start associating drivers with .inf files, which seems > > >> a bit complicated. > > >> > > >> (3) You write: > > >> > > >>> If you found an inf file with the matching minor OS (6 in our case) > > >>> version and matching or less but close minor version number (2 vs 3) > > >>> then you are in the right directory. > > >> > > >> When you say "close minor version number" is there a particular > > >> definition you have in mind? eg. minor or minor-1 is OK? Or do we > > >> have to consider all .inf files in totality and choose the nearest > > >> one? > > > Ideally, we should try finding the exact match. If not, we have to find > > > one with the same major version number and nearest minor number which is > > > smaller than the target one. For example, let's say we are looking for a > > > driver for Win8.1/WS2012R2, we know that both of them have kernel > > > version 6.3 . So, ideally we should to find inf file with the following > > > driver version string: > > > > > > DriverVer=XX/YY/ZZZZ,63.AA.BBB.CCCCC > > > > > > if we cannot find an inf file with such criteria, we should try finding > > > another one with the OS version 62 (which means that a driver was build > > > for kernel 6.2 Win8/WS2012 and technically should be compatible with > > > 6.3). So, we are looking for DriverVer which looks like this one: > > > DriverVer=XX/YY/ZZZZ,62.AA.BBB.CCCCC > > > > > > Still no luck - decrease the target OS version to 61 and repeat. > > > > > > For WDF drivers like balloon, vioserial, and rng you can go as far as > > > you can trying to find an appropriate driver. > > > > > > For miniport drivers like netkvm ( Yan, please confirm ), viostor and > > > vioscsi I wouldn't go too far. For example drivers designed to fork in > > > specific stacks at kernel version 5.1 (WinXP) can refuse working with > > > kernel 6.2 (Win8). > > > > While NDIS miniport designed for NDIS 5.1 (XP) still can run on latest Windows, I wouldn’t do that. For NDIS 6.x (Windows 2008 and newer), we can use older drivers on new OS. > > But installing a driver for a newer OS on older OS might lead to driver miss-behaviour. > > I'm wondering how people who don't rely on virt-v2v are supposed to > discover the virtio drivers for their guests? This includes both those > who do it programmatically and those who locate the appropriate drivers > manually. > > I mean, both manual and programmatic procedures should resolve to the > same set of drivers, and ideally the procedure should be the same, > declared official and documented. > > I see no problem implementing any particular algorithm, no matter how > complex, in v2v, but I'd like to make sure that my colleagues who, say, > write a tool for unattended deployment of Windows guests, or others, who > manually set up their Windows VMs, arrive at the same configuration. > > I tend to think that your original idea of sorting that out on the > driver build side, and storing the resulting drivers grouped by > Windows arch/version following a well-defined and stable naming > convention is a more reliable and universal solution. > It definitely shouldn't be so complicated. Packaging drivers on per-platform base can solve most of the problems. One single iso image, containing both 32 and 64 bit drivers compatible to run on Win8/Win8.1/WS2012/WS2012R2, and exposing an appropriate media layout "understandable" by all these platforms, is what we need. After all, the information provided n this thread, should be enough for writing a script that can split our single iso image into several properly build ones. In this case, the only thing you should worry about, is that <os>\<architecture> naming pattern is not the same among the various Windows platforms. IIRC there were some changes around Vista/Longhorn when Microsoft did it. Cheers, Vadim. > Thanks, > Roman.

On Tue, Nov 17, 2015 at 08:49:54PM +0000, Richard W.M. Jones wrote: > There was a copy and paste error. It should have read: > > $ md5sum viostor/2k8/amd64/* viostor/w7/amd64/* > d1132fbd27c98f25ea71a6eb4037b2f8 viostor/2k8/amd64/viostor.cat > defccbe48aca1cd97fee4d7b16053b72 viostor/2k8/amd64/viostor.inf > 6da6120368c097806e4ac7fc6b940810 viostor/2k8/amd64/viostor.pdb > 63b05edfb75d9dc60e82ab3c0bfee701 viostor/2k8/amd64/viostor.sys > af0cca02244fcfedf3209aef56610464 viostor/w7/amd64/viostor.cat > defccbe48aca1cd97fee4d7b16053b72 viostor/w7/amd64/viostor.inf > 6da6120368c097806e4ac7fc6b940810 viostor/w7/amd64/viostor.pdb > 63b05edfb75d9dc60e82ab3c0bfee701 viostor/w7/amd64/viostor.sys > > (almost identical, *.cat files are different) 2k8 == vista == 6.0 2k8r2 == w7 == 6.1 You need to compare w7 against 2k8r2. AFAICS they are the same.

From: "Vadim Rozenfeld" <vrozenfe(a)redhat.com&gt; To: "Richard W.M. Jones" <rjones(a)redhat.com&gt; Cc: "Roman Kagan" <rkagan(a)virtuozzo.com&gt;, libguestfs(a)redhat.com, "Amnon Ilan" <ailan(a)redhat.com&gt;, "Jeff Nelson" <jenelson(a)redhat.com&gt;, "Yan Vugenfirer" <yvugenfi(a)redhat.com&gt;, "Denis Lunev" <den(a)virtuozzo.com&gt; Sent: Wednesday, November 18, 2015 6:00:12 AM Subject: Re: Fwd: [PATCH] v2v: virtio-win: include *.dll too On Tue, 2015-11-17 at 19:44 +0000, Richard W.M. Jones wrote: > I think one - maybe final? - problem. How can I tell the difference > between drivers for "client" versions of Windows (eg. Windows 7) > and server versions of Windows (eg. Windows 2008 Server)? > > It seems in many or most cases the drivers are identical, eg: > > $ md5sum viostor/2k12/amd64/* viostor/w8/amd64/* > bbe250c13bf891fd7292ccab9908a63a viostor/2k12/amd64/viostor.cat > c2fdfc6a47fdc9dd25242441cdf1a68e viostor/2k12/amd64/viostor.inf > 5ca2cadca2214b90196755dab06676f4 viostor/2k12/amd64/viostor.pdb > 80c5ea60a69f37e965cb277383abd067 viostor/2k12/amd64/viostor.sys > bbe250c13bf891fd7292ccab9908a63a viostor/w8/amd64/viostor.cat > c2fdfc6a47fdc9dd25242441cdf1a68e viostor/w8/amd64/viostor.inf > 5ca2cadca2214b90196755dab06676f4 viostor/w8/amd64/viostor.pdb > 80c5ea60a69f37e965cb277383abd067 viostor/w8/amd64/viostor.sys > > $ md5sum Balloon/2k12/amd64/* Balloon/w8/amd64/* > d4104f08e674ab9def49d086d960349d Balloon/2k12/amd64/balloon.cat > 0325b7e0552e1567292eb6c1068fee18 Balloon/2k12/amd64/balloon.inf > ac2116f04fd678d0e264f34e3bf90d1e Balloon/2k12/amd64/balloon.pdb > c975501dd5b43ade3717722b81acbc1f Balloon/2k12/amd64/balloon.sys > 56f63afdd992a5395a4d8da6a52e559e Balloon/2k12/amd64/blnsvr.exe > c539f2056fb196da377d14180fda195c Balloon/2k12/amd64/blnsvr.pdb > d10864c1730172780c2d4be633b9220a > Balloon/2k12/amd64/WdfCoInstaller01011.dll > d4104f08e674ab9def49d086d960349d Balloon/w8/amd64/balloon.cat > 0325b7e0552e1567292eb6c1068fee18 Balloon/w8/amd64/balloon.inf > ac2116f04fd678d0e264f34e3bf90d1e Balloon/w8/amd64/balloon.pdb > c975501dd5b43ade3717722b81acbc1f Balloon/w8/amd64/balloon.sys > 56f63afdd992a5395a4d8da6a52e559e Balloon/w8/amd64/blnsvr.exe > c539f2056fb196da377d14180fda195c Balloon/w8/amd64/blnsvr.pdb > d10864c1730172780c2d4be633b9220a Balloon/w8/amd64/WdfCoInstaller01011.dll > > (completely identical) > > $ md5sum viostor/2k8/amd64/* > viostor/w7/amd64/*d1132fbd27c98f25ea71a6eb4037b2f8 > viostor/2k8/amd64/viostor.cat > defccbe48aca1cd97fee4d7b16053b72 viostor/2k8/amd64/viostor.inf > 6da6120368c097806e4ac7fc6b940810 viostor/2k8/amd64/viostor.pdb > 63b05edfb75d9dc60e82ab3c0bfee701 viostor/2k8/amd64/viostor.sys > af0cca02244fcfedf3209aef56610464 viostor/w7/amd64/viostor.cat > defccbe48aca1cd97fee4d7b16053b72 viostor/w7/amd64/viostor.inf > 6da6120368c097806e4ac7fc6b940810 viostor/w7/amd64/viostor.pdb > 63b05edfb75d9dc60e82ab3c0bfee701 viostor/w7/amd64/viostor.sys > > (almost identical, *.cat files are different) > > In any case the *.inf files don't seem to have any distinguishing > feature which would allow us to check this. > > Maybe this doesn't matter? > Let me explain how it works: We don't make any difference between server or client platforms when building our drivers. Initially, all the files, mentioned above (inf, cat, sys, and pdb), are absolutely the same. Then the files go to QE for WHQL testing. QE tests the same drivers on both, desktop and server platforms, and creates two different submissions files for sending them to WHQL dashboard togather with the original "cat" file. As the result, we have two different cat files created from the same original one. As far as I remember, there shouldn't be any problem to swap these two cat files, and use them interchangeable. But it will be better get some conformation from QE team.

Best regards, Vadim. > Rich. >

> > > In any case the *.inf files don't seem to have any distinguishing > > > feature which would allow us to check this. > > > > > > Maybe this doesn't matter? > > > > Let me explain how it works: > > We don't make any difference between server or client platforms when > > building our drivers. Initially, all the files, mentioned above (inf, > > cat, sys, and pdb), are absolutely the same. Then the files go to QE for > > WHQL testing. QE tests the same drivers on both, desktop and server > > platforms, and creates two different submissions files for sending them > > to WHQL dashboard togather with the original "cat" file. > > As the result, we have two different cat files created from the same > > original one. > > > > As far as I remember, there shouldn't be any problem to swap these two > > cat files, and use them interchangeable. But it will be better get some > > conformation from QE team. > > Li Jin, this is a discussion we have on a public mailing-list regarding the > files structure > of our virtio-win distribution. > Could you confirm Vadim's statement above? Yes, I agree with Vadim's statement. The *.cat file is a signature file: Before whql testing,the driver is only signed by redhat,when install driver in guest,windows OS will prompt a security warning to confirm if customer want to install a driver not signed by msft. After whql testing and submit to msft,we will get a *.cat file signed by microsoft,which will make driver installation smoothly(no extra warning). The original *.cat file is same,whether the *.cat files are still same afer whql submission depends on whether they are submit in one submission. for example: win8-64 and win2012 *.cat files are same because QE submit them in one submission(win8-64 and win2012 whql test both run on hck,results can be merged into one package); win7-64 and win2008-64 *.cat files are different because we submit them in different submissions(win7-64 whql test run on hck,win2008-64 run wlk,hck and wlk must submit differently according to msft's strategy). I tried swap cat files,drivers still can be installed correctly.

On Thu, Nov 19, 2015 at 09:45:36PM +0300, Roman Kagan wrote: > On Wed, Nov 18, 2015 at 11:31:17PM -0500, Li Jin wrote: > > > > > In any case the *.inf files don't seem to have any distinguishing > > > > > feature which would allow us to check this. > > Catalog files are pkcs7 containers with multi-layered nested ASN.1 > structure [...] > So it looks like yes, there's certain data in there which can be used to > match it against the exact Windows version; not sure what to do with all > this, though. Having given it some more thought and having discussed that with colleagues who do various things with the drivers, like generating unattended install helpers or just manually installing drivers into guests, I'm now of the opinion that neither libguestfs nor any other user of the drivers should look inside the driver files in order to match them against the guest OS, because it's hard for programs and unrealistic for humans. I tend to believe it should be the responsibility of the driver packages to lay out the drivers in a directory structure that both humans and programs could grok easily and that wouldn't change once established; in a sense this would be both an API and a human interface. If I were to lay it out I'd group the stuff by guest OS, so that for a particular guest a single entity -- a directory tree or a pre-created ISO or floppy image -- would contain all that's needed for this guest and nothing else. This would allow separation of concerns, so that actors who bring the stuff into the guest wouldn't need to care about what's inside, and actors involved with its installation wouldn't need to know how to find the right one.

E.g. from this POV the way drivers are stored in RHEL7 virtio-win package on the *filesystem* looks OK, the way they are stored in the *ISO image* in the same package doesn't. As for naming one can stick with what's being used in that virtio-win rpm, i.e. {amd64,i386}/Win{XP,2003,2008,2008R2,2012,2012R2,7,8,8.1} or switch to the names passed to inf2cat in its /os parameter, i.e. XP_X86,Server2003_X86,...,8_X64,Server8_X64,6_3_X86,6_3_X64,Server6_3_X64 (https://msdn.microsoft.com/en-us/library/windows/hardware/ff547089(v=vs.8...) Whatever convention is chosen it needs to be followed thereafter.

The main problem with this proposal is that I don't know how to implement it: there are multiple vendors of virtio-win packages, and, for licensing reasons, they don't even share the source code, so it's unclear who could own this convention and enforce it on others.

I'd appreciate any thoughts or suggestions on how to proceed from that and whether it's worth it at all.

On Tue, Nov 24, 2015 at 01:07:03PM +0000, Richard W.M. Jones wrote: > On Mon, Nov 23, 2015 at 08:26:10PM +0300, Roman Kagan wrote: > > Having given it some more thought and having discussed that with > > colleagues who do various things with the drivers, like generating > > unattended install helpers or just manually installing drivers into > > guests, I'm now of the opinion that neither libguestfs nor any other > > user of the drivers should look inside the driver files in order to > > match them against the guest OS, because it's hard for programs and > > unrealistic for humans. > > > > I tend to believe it should be the responsibility of the driver packages > > to lay out the drivers in a directory structure that both humans and > > programs could grok easily and that wouldn't change once established; in > > a sense this would be both an API and a human interface. > > > > If I were to lay it out I'd group the stuff by guest OS, so that for a > > particular guest a single entity -- a directory tree or a pre-created > > ISO or floppy image -- would contain all that's needed for this guest > > and nothing else. This would allow separation of concerns, so that > > actors who bring the stuff into the guest wouldn't need to care about > > what's inside, and actors involved with its installation wouldn't need > > to know how to find the right one. > > The good news is that this is what we do right now. > > > E.g. from this POV the way drivers are stored in RHEL7 virtio-win > > package on the *filesystem* looks OK, the way they are stored in the > > *ISO image* in the same package doesn't. > > > > As for naming one can stick with what's being used in that virtio-win > > rpm, i.e. > > > > {amd64,i386}/Win{XP,2003,2008,2008R2,2012,2012R2,7,8,8.1} > > > > or switch to the names passed to inf2cat in its /os parameter, i.e. > > > > XP_X86,Server2003_X86,...,8_X64,Server8_X64,6_3_X86,6_3_X64,Server6_3_X64 > > (https://msdn.microsoft.com/en-us/library/windows/hardware/ff547089(v=vs.8...) > > > > Whatever convention is chosen it needs to be followed thereafter. > > Yup - very important that once we decide what the naming convention is > going to be, we stick with it. Moving drivers around or using ad-hoc > naming schemes causes churn in virt-v2v and probably other projects > too. > > > The main problem with this proposal is that I don't know how to > > implement it: there are multiple vendors of virtio-win packages, and, > > for licensing reasons, they don't even share the source code, so it's > > unclear who could own this convention and enforce it on others. > > I guess that virt-v2v will have to deal with this with more special > rules. > > > I'd appreciate any thoughts or suggestions on how to proceed from that > > and whether it's worth it at all. Apparently I failed to inspire anybody who can affect the driver packaging layout :) Let me try again. Do I get it right that the way virtio-win is packaged for Fedora and RHEL is driven by the scripts at https://github.com/crobinso/virtio-win-pkg-scripts and I should be submitting patches or pull requests to it?

Or should the layout be determined only by the build machinery at https://github.com/YanVugenfirer/kvm-guest-drivers-windows and I should be direct my submissions that way?

Or any combination of the above? Also does anybody know this is done in SUSE?

Any help would be appreciated. Thanks, Roman.

On Thu, Nov 26, 2015 at 06:29:45PM -0600, Jeff Nelson wrote: > On Thu, Nov 26, 2015 at 08:12:41PM +0300, Roman Kagan wrote: > >Do I get it right that the way virtio-win is packaged for Fedora and > >RHEL is driven by the scripts at > > > >https://github.com/crobinso/virtio-win-pkg-scripts > > > >and I should be submitting patches or pull requests to it? > > Yes, that's what I use to package the drivers for RHEL. > > > >Or should the layout be determined only by the build machinery at > > > >https://github.com/YanVugenfirer/kvm-guest-drivers-windows > > > >and I should be direct my submissions that way? > > I don't know, but if you work strictly with virtio-win-pkg-scripts it > shouldn't matter. Thanks a lot, I'll look into it early next week. I guess there's no mailing list, so I'll have to do pull requests on github, right?

On 11/27/2015 12:16 PM, Roman Kagan wrote: > On Thu, Nov 26, 2015 at 06:29:45PM -0600, Jeff Nelson wrote: >> On Thu, Nov 26, 2015 at 08:12:41PM +0300, Roman Kagan wrote: >>> Do I get it right that the way virtio-win is packaged for Fedora and >>> RHEL is driven by the scripts at >>> >>> https://github.com/crobinso/virtio-win-pkg-scripts >>> >>> and I should be submitting patches or pull requests to it? >> >> Yes, that's what I use to package the drivers for RHEL. >> >> >>> Or should the layout be determined only by the build machinery at >>> >>> https://github.com/YanVugenfirer/kvm-guest-drivers-windows >>> >>> and I should be direct my submissions that way? >> >> I don't know, but if you work strictly with virtio-win-pkg-scripts it >> shouldn't matter. > > Thanks a lot, I'll look into it early next week. > Sorry I'm only getting to this thread now :/ I'd like to try and summarize the interrelated packaging changes that were mentioned in this thread: - The iso layout is not optimal for programmatic consumption - The iso layout is not compatible with windows driver media autodetect conventions - The iso and RPM host contents do not match - The reasoning behind the iso/RPM layout is not clear To all this I say... preach, my brother :) https://bugzilla.redhat.com/show_bug.cgi?id=1217658 https://bugzilla.redhat.com/show_bug.cgi?id=1167941 The ISO and RPM layout is largely the result of organic extensions and no one ever tried to streamline things, so I fully support any work in that direction. But the things to consider: - There are many consumers of the iso layout as it is right now. Any changes in the short to medium term need to be entirely additive so we can get the new layout right without breaking every consumer. So keep the existing iso layout of $drivername/$weird-os/$arch, but _add_ the $arch/$standard-os/* layout, and avoid duplicates by hardlinking identical files (the scripts on github already have code that handles the last bit). In fact it might be a good starting point to write a script that takes the existing iso, extracts it, and rearranges the contents to match the new format. Then I could help integrate that with the scripts and my own testing. - The layout should use the latest windows conventions for arch/osname. If this means we have to diverge from the naming convention we currently use for RPM host installed drivers, that's fine, since we can maintain back compat there with symlinks. However I've tried searching for this info (googling 'windows driver cd layout/hierarchy/format') but I can't find anything definitive... anyone have a pointer? Especially WRT the current standard vs. the pre-windows-7 standard that vadim pointed out.

- Anything regarding a separate iso with a separate layout to appease < win7 windows vintage should be handled separately, if at all IMO > I guess there's no mailing list, so I'll have to do pull requests on > github, right? Let's stick with email and virt-tools-list(a)redhat.com since there may be lots of discussion... Thanks, Cole

On Mon, Nov 30, 2015 at 05:56:48PM -0500, Cole Robinson wrote: > - There are many consumers of the iso layout as it is right now. Any > changes in the short to medium term need to be entirely additive so > we can get the new layout right without breaking every consumer. So > keep the existing iso layout of $drivername/$weird-os/$arch, but > _add_ the $arch/$standard-os/* layout, and avoid duplicates by > hardlinking identical files (the scripts on github already have code > that handles the last bit). May also be worth asking who the current consumers are. Virt-v2v is one of them. Are there others? (GNOME Boxes?)

On Mon, Nov 30, 2015 at 05:56:48PM -0500, Cole Robinson wrote: > https://bugzilla.redhat.com/show_bug.cgi?id=1167941 This one is restricted access so I can't see it.