[ipxe-devel] multiple NICs using same driver and boot order

[Alex Williamson]

Hi,

A question on boot order, if I launch QEMU as:

qemu-kvm -m 1G -nodefaults -device sga -serial stdio -display none -boot menu=on \
  -netdev user,id=hostnet0 -device e1000,netdev=hostnet0,id=net0,mac=ba:db:ad:ba:db:ad,bootindex=2,addr=4 \
  -netdev user,id=hostnet1 -device e1000,netdev=hostnet1,id=net1,mac=60:0d:60:0d:60:0d,bootindex=1,addr=5

This gives me two NICs, a "bad" emulated e1000 at PCI 00:04.0 and a
"good" emulated e1000 at PCI 00:05.0.  The bootindex directives
prioritize good over bad.  The boot goes as follows:

        SeaBIOS (version seabios-1.7.2.2)
        
        iPXE (http://ipxe.org) 00:04.0 C100 PCI2.10 PnP PMM+3FFC9A70+3FF29A70 C100
        
        iPXE (http://ipxe.org) 00:05.0 C200 PCI2.10 PnP PMM 3FFC9A70 3FF29A70 C200
        
        Press F12 for boot menu.
        
        Select boot device:
        
        1. iPXE (PCI 00:05.0)
        2. iPXE (PCI 00:04.0)
        3. Legacy option rom
        <select 1 or let timeout>
        
        Booting from ROM...
        iPXE (PCI 00:05.0) starting execution...ok
        iPXE initialising devices...ok
        
        iPXE 1.0.0+ (017e6) -- Open Source Network Boot Firmware -- http://ipxe.org
        Features: HTTP iSCSI DNS TFTP AoE bzImage ELF MBOOT PXE PXEXT Menu
        
        net0: ba:db:ad:ba:db:ad using 82540em on PCI00:04.0 (open)
          [Link:up, TX:0 TXE:0 RX:0 RXE:0]
        Configuring (net0 ba:db:ad:ba:db:ad)...... ok
        net0: 10.0.2.15/255.255.255.0 gw 10.0.2.2
        Nothing to boot: No such file or directory (http://ipxe.org/2d03e13b)
        net1: 60:0d:60:0d:60:0d using 82540em on PCI00:05.0 (open)
          [Link:up, TX:0 TXE:0 RX:0 RXE:0]
        Configuring (net1 60:0d:60:0d:60:0d)...... ok
        net0: 10.0.2.15/255.255.255.0 gw 10.0.2.2 (inaccessible)
        net1: 10.0.2.15/255.255.255.0 gw 10.0.2.2
        Nothing to boot: No such file or directory (http://ipxe.org/2d03e13b)
        No more network devices

So SeaBIOS indicates that 00:05.0 is the first boot device, iPXE
initializes 00:05.0, but then tries to boot from 00:04.0.  The only way
I can get the "good" NIC to boot is to put it at a lower PCI device
address than the "bad" NIC.  SeaBIOS seems to be telling iPXE the
correct device to use, but at some point iPXE decides to just use the
first device supported by the driver.  This can also happen with
assigned devices, for instance a physical 82576 uses the same driver as
an emulated e1000.

It's also not necessarily desirable that iPXE will attempt to boot from
all devices supported by the driver before returning and letting the
BIOS pick the next boot device.  For instance, I may setup 3 NICs,
e1000, rtl8139, e1000 where I want to attempt boot in that order.  There
seems to be no way to do this as the iPXE ROM with intel driver will
attempt to boot both e1000 NICs, then the iPXE ROM on the rtl NIC will
attempt to boot.  The 3rd boot slot retries boot on both e1000 NICs.

Is there any way we can get this working like it's supposed to?  Thanks,

Alex