[ipxe-devel] [PATCH 0/1] [intelxvf] Reset all vf settings before config

[Vishvananda Ishaya Abrams]

Iscsi booting off a virtual function using ixgbevf works correctly during the
first boot, but after booting into a guest os running linux and hard rebooting,
the guest will no longer iscsi boot. I tracked the issue down to an issue with
the reset of the virtual function. Reset does not actually reset the memory
addresses for the vf queues.  Since the linux driver sets up multiple
send/receive queues, and the ipxe code only sets up one, the packets get copied
to random memory addresses that ipxe does not know about.

You can see in the code that  handles a vf reset[1] in the ixgbe driver that it
doesn't actually reset the send and receive queues. Additionally there isn't a
reliable way to reset these settings from userspace on the host side. The best
I could do was force the adapter to reset completely using an ethtool
command[2], which clearly is not ideal since it resets other vfs as well. I
briefly considered modifying the ixgbe driver in linux to expose a command
through syfs to completely reset one of its vfs but I decided against it.
(incidentally it is also impossible to "unset" the mac address of a vf from
linux. It starts out as 00:00:00:00:00:00, but once you set it to a value using
iproute2, you cannot return it to unset).

Ultimately I settled on making the vf completely reset its registers to default
values from the ipxe ixgbevf code, which is what the patch does. I found all of
the registers that were getting modified via ethregs[3], and I reset them. The
testing showed that in addition to resetting rx and tx queues, I also needed to
reset packet split receive type register as well as reset some flags in the
SRRCTL register to get things back to the default. It is possible that some
future version of linux will set other values that I missed that will also need
to be reset, but this does the trick for now.

Vish

[1] http://lxr.free-electrons.com/source/drivers/net/ethernet/intel/ixgbe/ixgbe_sriov.c#L791
[2] http://lxr.free-electrons.com/source/drivers/net/ethernet/intel/ixgbe/ixgbe_ethtool.c#L986
[3] https://sourceforge.net/projects/e1000/files/Ethregs%20-%20Register%20Dump%20Tool/1.20.1/


Vishvananda Ishaya (1):
  [intelxvf] Reset all vf settings before config

 src/drivers/net/intel.c    | 25 +++++++++++++++++++++++++
 src/drivers/net/intel.h    |  2 ++
 src/drivers/net/intelxvf.c | 17 ++++++++++++++++-
 src/drivers/net/intelxvf.h |  9 ++++++++-
 4 files changed, 51 insertions(+), 2 deletions(-)

-- 
2.5.0