[ipxe-devel] virtnet_open_legacy() fails to find virtqueues on Google Compute Engine

[Akihiro Suda]

Hi Christian,

Thank you for the suggestion, I'll try to report this to google.

I found that the cause of this issue is because GCE's VIRTIO_PCI_QUEUE_NUM
is 4096, which is larger than iPXE's MAX_QUEUE_NUM (256).

https://git.ipxe.org/ipxe.git/blob/26050fd4c87c50503d5bd573b2ec91703676e211:/src/drivers/bus/virtio-pci.c#l43
https://git.ipxe.org/ipxe.git/blob/26050fd4c87c50503d5bd573b2ec91703676e211:/src/include/ipxe/virtio-ring.h#l24


IIUC this issue will be resolved if GCE supports virtio 1.0, which would
allow iPXE to use custom queue num smaller than the virtio device's max
value.

However, it would be great if iPXE can support setting MAX_QUEUE_NUM to
4096 (via `config/general.h`, probably).

Actually I tried to set MAX_QUEUE_NUM to 4096 and got virtnet_open_legacy()
succeeded, but it still doesn't work:

    iPXE> dhcp
    DHCP 0x120e44 entering discovery state
    Configuring (net0 42:01:0a:92:00:0a)...DHCP 0x120e44 DHCPDISCOVER
    DHCP 0x120e44 DHCPOFFER from 169.254.169.254:67 for 10.146.0.10
    .DHCP 0x120e44 DHCPDISCOVER
    ..DHCP 0x120e44 entering request state
    DHCP 0x120e44 DHCPREQUEST to 169.254.169.254:67 for 10.146.0.10
    DHCP 0x120e44 DHCPREQUEST to 169.254.169.254:67 for 10.146.0.10
    .DHCP 0x120e44 DHCPREQUEST to 169.254.169.254:67 for 10.146.0.10
    .DHCP 0x120e44 DHCPREQUEST to 169.254.169.254:67 for 10.146.0.10
    ...DHCP 0x120e44 DHCPREQUEST to 169.254.169.254:67 for 10.146.0.10
    ....... No configuration methods succeeded (http://ipxe.org/040ee119)
    iPXE>

I haven't captured DHCP packets (because I'm unsure how to do so in GCE's
fully managed DHCP server), but I guess DHCPACK is actually sent from the
DHCP server but somehow iPXE missed it.

I'd appreciate if anyone can support me to set MAX_QUEUE_NUM to 4096.


2016-12-14 16:19 GMT+09:00 Christian Nilsson <nikize at gmail.com>:

> You might want to report this to google as well if you haven't already.
>
> On Wed, Dec 14, 2016 at 7:16 AM, Akihiro Suda <suda.kyoto at gmail.com>
> wrote:
> > Hello,
> >
> > I tried to boot iPXE on a Google Compute Engine instance, but the virtio
> > driver doesn't work because virtnet_open_legacy() fails to find
> virtqueues.
> >
> > Please let me know what I can do for providing further information to
> debug
> > the issue.
> >
> > The serial console output with "DEBUG=virtio-net" is as follows:
> >
> >     iPXE initialising devices...Common virtio capability not found!
> >     VIRTIO-NET 0xdf584 busaddr=0000:00:04.0 ioaddr=0xc040 irq=11
> >     VIRTIO-NET 0xdf584 mac=42:01:0a:92:00:0a
> >     ok
> >
> >     iPXE 1.0.0+ (26050) -- Open Source Network Boot Firmware --
> > http://ipxe.org
> >     Features: DNS HTTP iSCSI TFTP SRP AoE ELF MBOOT PXE bzImage Menu
> PXEXT
> >     iPXE> ifstat
> >     net0: 42:01:0a:92:00:0a using virtio-net on 0000:00:04.0 (closed)
> >       [Link:up, TX:0 TXE:0 RX:0 RXE:0]
> >     iPXE> dhcp
> >     VIRTIO-NET 0xdf584 cannot register queue 0
> >     Could not open net0: No such file or directory
> > (http://ipxe.org/2d5c403b)
> >     iPXE>
> >
> >
> > This ENOENT is raised from virtnet_open_legacy():
> > https://git.ipxe.org/ipxe.git/blob/26050fd4c87c50503d5bd573b2ec91
> 703676e211:/src/drivers/net/virtio-net.c#l205
> >
> >
> > I also tested some previous commits, but all of them failed, so the
> issue is
> > not a regression in some recent commit:
> >
> >  * 26050fd (Dec  8, 2016): the last commit when I tested
> >  * 7b499f8 (Apr 11, 2016): the last commit without support for virtio 1.0
> >  * e4419ff (Jul  2, 2010):  the first commit with the "native iPXE
> driver"
> > (The commit date is older than 232c208, but actually newer than 232c208)
> >  * 232c208 (Jul 11, 2010): the last commit without the "native iPXE
> driver".
> > The error message is "ERROR: queue size 4096 > 512", "Cannot register
> queue
> > #0"
> >
> >
> > Regards,
> >
> > Akihiro Suda
> >
> >
> > _______________________________________________
> > ipxe-devel mailing list
> > ipxe-devel at lists.ipxe.org
> > https://lists.ipxe.org/mailman/listinfo.cgi/ipxe-devel
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ipxe.org/pipermail/ipxe-devel/attachments/20161214/acaeaeca/attachment.htm>