[ipxe-devel] Issues with Broadcom NICs in Dell R7625

[Thomas Walker]

On Thu, Jan 19, 2023 at 11:16:22PM +0000, Michael Brown wrote:
> On 19/01/2023 16:48, Thomas Walker via ipxe-devel wrote:
> > First off, I want to say thank you, the work done 2 years ago (https://github.com/ipxe/ipxe/commit/988d2c13cdf0f0b4140685af35ced70ac5b3283c) to fix walking through multiple instances of EFI_PCI_ROOT_BRIDGE_IO_PROTOCOL worked like a charm on another system with a funky bus layout (this is the next generation of the system that I encountered that problem on).  All appears to be working on that front.  But I'm encontering 2 problems (on master from a couple of weeks back, 7147532c):
> > 
> > The 1G LOM is a Broadcom 5720, nearly identical to the previous generation server that works fine.  Both good and bad are vendor:dev 14e4:165f and subvendor 1028.  The older (working) system has a subdevice of 08ff whereas the newer (sort of working) one has a subdevice of 0a6b.
> > 
> > While ipxe does appear to "work" with this adapter, it prints:
> > 
> > WARNING: Using legacy NIC wrapper on 79:79:79:79:79:79
> > 
> > Where that MAC obviously bears no relation to the actual MAC on the device.
> 
> That will be a false positive detection of an ISA NIC.
> 
> > This system also has an OCP slot fitted with a Broadcom 57414 (vendor:device 14e4:16d7 and subvendor:subdevice 1407:4145) and manages to dhcp but not much more.  Enabling debugging (DEBUG=bnxt:2) shows that, when "Configuring", it starts spewing:
> > 
> > assert(!iob) failed at drivers/net/bnxt/bnxt.c line 468
> > 
> > over and over until it finally fails and ipxe exits.  In fact, I had to compile bnxt out completely in order to boot the system off of the 1G LOM.
> > 
> > Any suggestions for either of these?
> 
> I'm slightly confused by the report: are you saying that you _are_ able to
> boot from the LOM, but only if you compile out bnxt (or use a build that
> contains only the relevant LOM driver, such as bin-x86_64-efi/tg3.efi)?
> 
> Thanks,
> 
> Michael
> 
> 

Correct.  Without disabling the bnxt driver, iPXE fails.
When I emailed yesterday, I thought that there were 3 problems here:
- The assertion failure in bnxt
- That this assertion failure causes iPXE to fail completely (i.e. not simply ignore the bnxt devices and proceed with the tg3)
- This "legacy NIC wrapper" message related to the tg3 device (perhaps simply cosmetic, as it still succeeds if bnxt isn't present?)

Because the bnxt assertion failure isn't visible without debugging enabled, I spent quite a while thinking that the "legacy NIC wrapper" was actually fatal in this case.  On closer examination, it became clear that the bnxt device was "functional enough" (link came up, dhcp suceeded) that ipxe proceeded running the rest of our embedded script (which then fetches vmlinuz/initrd over http) using the bnxt device but repeatedly hit the assertion, wasn't successfully passing traffic, and eventually timed out.  So the 2nd point above isn't really valid.  ipxe had already "chosen" a NIC at that point, it just didn't work fully.

Anyway, I can move forward here.  I have no intention of ever needing to PXE off of bnxt, so I can leave it out.  And the "legacy NIC wrapper" warning appears to be cosmetic.  But I'm willing to debug further if that is helpful since I know you don't have the hardware in question.  It is also worth noting that this hardware was just released, and everything is on "the one and only" firmware versions available.  So it is quite possible that these issues will all clear up as firmware matures.

Thanks!