[ipxe-devel] Two problems/bugs in iPXE related to multiple NICs

[kevin at fantaplay.com]

Hi Andrew and Robin,

Sorry for the tardy reply; gamedev has me busy.

-------- Original Message --------
From: Andrew Bobulsky <rulerof at gmail.com>
> I'm finding myself thoroughly confused here, because I'm booting
> Windows 8 from iSCSI on an Intel NIC.... I haven't had any of the
> problems you're referring to, Kevin. I'm chain loading iPXE.pxe from
> the onboard Realtek ROM PXE. I know Windows 8 isn't Server 2012, but
> I'd expect similar behavior on this issue.

(I had read iSCSI boot was only available in Windows Server but I guess
not!)

I assume the Intel NIC you're using is dual-ported also?  If so this
would seem to be a problem specific to Broadcom's driver.  It seems a
bit strange, though, that the NIC driver should even be aware of the
iBFT.

It would be interesting to see how my server would react to having a
different NIC added in as Robin suggested.  Unfortunately I don't have
much opportunity to mess with that system further at the moment.

Also, do you have two disparate networks connected to the two ports?  My
ports were connected to physically isolated networks on entirely
different IP ranges.  The SAN was not reachable at all from the other
NIC port.

> Could you share your installation procedure, Kevin? I can try to
> duplicate your problem, if you like.

At the moment I don't actually need a fix as I was able to work around
the problem, but you can always have a go at it, in the name of
improving the program.  :)  I'll be most interested to hear your results
if you do.

I tested with StarWind's iSCSI SAN Free Edition, which you can find
here, as the iSCSI target and PXE server:

http://www.starwindsoftware.com/starwind-free

I had this on a Server 2008 R2 VM with the MS DHCP server bound only to
the SAN.  StarWind interacts with the MS DHCP to do all the PXE-specific
modifications to DHCP automatically.

I only had to set up a TFTP server (Open TFTP Server, MT), put the (it
so turns out) gPXE binary there, and tell StarWind to use it as the PXE
boot image.

I used the UNDI only version, as otherwise the info that got written to
the iBFT was jumbled up (confirmed by dumps) and caused Windows to
crash.  That was on the 1.0.0 gPXE, so that particular glitch might have
been fixed in iPXE but I didn't check, as at the time I had no idea how
your releases worked.

The first time booting, I used the gPXE command prompt in order to
invoke keep-san and sanboot, and then let BIOS fall back to the local
DVD-ROM with the Server 2012 RTM disc.  The LU looked like a BIOS disk
to the installer.

For subsequent boots I allowed gPXE to go through the auto boot.

2012 would boot ok but would show the LAN NIC (the one opposite the one
it was booting from) as "not working properly."

Replacing the regular gPXE with one modified to provide a second NIC
section in the iBFT fixed that, provided the SAN was on the first NIC
and also in the first iBFT NIC section, among other arbitrary
requirements (see my serverfault.com posts).

Let me know if you'd like any other details on the procedure (hopefully
I can remember).

> I'm curious as to what that "responsibility" actually is. I speculate
> here, but several internal/undocumented things have changed regarding
> Server 2012 and Windows 8 boot device drivers; for example, the
> CriticalDeviceDatabase is just gone. Perhaps there's something about
> initializing the entire NDIS stack(?) at this early point that some...
> older drivers just don't like? In addition, while you could normally
> restart many portions of Windows networking to initialize a driver...
> I don't think that it will let you do that when the system volume is
> behind it. Perhaps that's the source of device manager's woes :P

I have admittedly very poor insight here.  While I used to do low-level
programming routinely, I haven't dealt with these particular aspects of
any modern OS.  So usually I am just assuming how they work based on how
I'd have designed them, and more often than not they end up defying
those assumptions in often very strange ways!

The really odd point which seems to elude explanation is how it is that
the same driver can bring up one port but not the other, when both use
the same chipset.  And then work fine if the iBFT is filled in better...

Cheers,
Kevin