[ipxe-devel] iPXE tftp multicast help requested (connection timeouts)

[Erik Jacobson]

I'm hoping someone can help me out here.

I have run in to a problem with getting tftp working in multicast mode
from iPXE. I'm talking here tftp multicast and not mtftp. The transfer
times out.

 - I'm using the latest iPXE from your git repo (as of yesterday).
 - On the same node, if I boot linux and use atftp client in multicast mode,
   the file transfers OK
 - atftpd on the server is in verbose mode and shows a multicast tftp transfer
   is attempted.
 - Both iPXE and the tftpd server show timeouts for the transfer

Here is the iPXE command I use from the command line environment. It happens
to be grub2 core.0 but that's just a handy test file I have around.

iPXE> dhcp                                                                      
Configuring (net0 00:25:90:0c:da:a6)...... ok                                   
iPXE> imgfetch tftm://172.23.0.1/boot/grub2/i386-pc/core.0                      
tftm://172.23.0.1/boot/grub2/i386-pc/core.0... Connection timed out (http://ipxe
.org/4c126035)                                                               

If I switch to normal tftp instead of tftm, all seems well (but it isn't
multicast).

I have tried chainloading both undionly.kpxe and ipxe.pxe (with the
drivers integrated). The behavior is the same.

Here are some lines from the server side atftpd server showing the failed
attempt:

Dec  9 14:39:15 icicle atftpd[12659]: Serving /boot/grub2/i386-pc/core.0 to 172.23.0.9:58592
Dec  9 14:39:15 icicle atftpd[12659]: tsize option -> 255088
Dec  9 14:39:15 icicle atftpd[12659]: blksize option -> 1432
Dec  9 14:39:15 icicle atftpd[12659]: multicast option -> 239.255.0.0,1758,1
Dec  9 14:39:25 icicle atftpd[12659]: timeout: retrying...
Dec  9 14:39:30 icicle atftpd[12659]: timeout: retrying...


I'm hoping I'm missing something obvious here.

As I write this, we're starting to work on collecting more data on the
problem, including port mirroring on the switch and collecting traffic in
dumps. I'll begin taking a closer look at the code soon (and will get a
second pair of eyes as network code isn't my current strong suit). We believe
we've disabled igmp on the switch and should be in multicast flood mode.

Thanks again for taking the time to look at this. Any help appreciated.


Here are some general comments on what I'm up to in case you're curious.
Optional reading.

First, I wanted to thank you for your work on iPXE. We use this in a
cluster manager product which means iPXE has a role in some of the largest
clusters in the world. I'm considering to expand it's role to something
that will include all nodes in a future release.

In other parts of the system (not iPXE), we have experience at using multicast
including tftp multicast (not mtftp) and udpcast to aid in diskless booting,
provisioning, etc. I'm considering changing our system to always chainload
to iPXE and boot up a yet-to-be-developed miniroot environment with our
toolchain. Using multicast tftp should allow fetching of this environment,
via an initramfs/initrd, on thousands of nodes without causing terrible
thrashing.

Note: We have EFI and legacy-boot nodes. I haven't gotten to check if the
EFI iPXE support works. I hope to get legacy working first and then move on
to see how EFI iPXE tftp works for us.