<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<font size="+1"><font face="FreeSerif">Hi Maxime,<br>
<br>
I don't know which version Ladi uses, we use OVS v2.6.1<br>
<br>
BR,<br>
Zoltan<br>
</font></font><br>
<div class="moz-cite-prefix">On 11/29/2017 02:06 PM, Maxime Coquelin
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:8bc5207f-ab5e-d505-b7c2-bccf4c7b0fa3@redhat.com">
<br>
<br>
On 11/29/2017 11:42 AM, Ladi Prosek wrote:
<br>
<blockquote type="cite">On Wed, Nov 29, 2017 at 9:57 AM, Maxime
Coquelin
<br>
<a class="moz-txt-link-rfc2396E" href="mailto:maxime.coquelin@redhat.com"><maxime.coquelin@redhat.com></a> wrote:
<br>
<blockquote type="cite">Hi Ladi,
<br>
<br>
Sorry for the late reply.
<br>
<br>
On 11/27/2017 05:01 PM, Ladi Prosek wrote:
<br>
<blockquote type="cite">
<br>
I think I understand what's going on. DPDK simply won't
consider the
<br>
interface 'ready' until after all queues have been
initialized.
<br>
<br>
<a class="moz-txt-link-freetext" href="http://dpdk.org/browse/dpdk/tree/lib/librte_vhost/vhost_user.c#n713">http://dpdk.org/browse/dpdk/tree/lib/librte_vhost/vhost_user.c#n713</a>
<br>
<br>
It looks like Maxime is the right person to bug about this.
One of his
<br>
recent commits appears to be somewhat related:
<br>
<a class="moz-txt-link-freetext" href="http://dpdk.org/browse/dpdk/commit/?id=eefac9536a">http://dpdk.org/browse/dpdk/commit/?id=eefac9536a</a>
<br>
<br>
Maxime, iPXE has a simple virtio-net driver that never
negotiates the
<br>
VIRTIO_NET_F_MQ feature and never initializes more than one
queue.
<br>
This makes it incompatible with vhost-user configured with
mq=on, as
<br>
Rafael and Zoltan have discovered.
<br>
<br>
Is there any chance DPDK can be made aware of the
VIRTIO_NET_F_MQ
<br>
feature bit being acked by the guest driver, and
successfully operate
<br>
with one queue in case it was not acked? There's some
context below in
<br>
this email. I can provide instructions on how to build iPXE
and launch
<br>
QEMU to test this if you're interested.
<br>
</blockquote>
<br>
<br>
I think I get your problem. I'm interested in instructions to
reproduce
<br>
the issue.
<br>
</blockquote>
<br>
Here it is, let me know if you run into any issues:
<br>
<br>
$ git clone git://git.ipxe.org/ipxe.git
<br>
$ cd ipxe/src
<br>
$ make bin/1af41000.rom DEBUG=virtio-net:2
<br>
$ ln -s bin/1af41000.rom efi-virtio.rom
<br>
<br>
Then run QEMU without changing the current directory (i.e.
should
<br>
still be .../ipxe/src):
<br>
<br>
qemu-system-x86_64 \
<br>
-machine pc,accel=kvm -m 128M -boot strict=on -device cirrus-vga
\
<br>
-monitor stdio \
<br>
-object
memory-backend-file,id=mem,size=128M,mem-path=/dev/hugepages,share=on
\
<br>
-numa node,memdev=mem \
<br>
-chardev socket,id=char1,path=/var/run/openvswitch/vhost-user0 \
<br>
-netdev type=vhost-user,id=mynet1,chardev=char1,vhostforce \
<br>
-device
virtio-net-pci,mac=00:00:00:00:00:01,netdev=mynet1,bootindex=0
<br>
<br>
You'll see a bunch of "enqueuing iobuf" debug messages on the
screen,
<br>
followed by at least "tx complete". Maybe also "rx complete"
depending
<br>
on what /var/run/openvswitch/vhost-user0 is connected to.
<br>
<br>
Now if you enable multiqueue by replacing the last two lines
with:
<br>
<br>
-netdev
type=vhost-user,id=mynet1,chardev=char1,vhostforce,queues=16 \
<br>
-device
virtio-net-pci,mac=00:00:00:00:00:01,netdev=mynet1,mq=on,vectors=34,bootindex=0<br>
<br>
you'll see only "enqueuing iobuf" without any completion,
indicating
<br>
that the host is not processing packets placed in the tx
virtqueue by
<br>
iPXE.
<br>
</blockquote>
<br>
<br>
Thanks, just tested with DPDK v16.11 & DPDK v17.11 using
testpmd instead
<br>
of OVS.
<br>
<br>
In my case, the packets send by iPXE are well received both with
and
<br>
without mq=on.
<br>
<br>
I don't think there is an issue with the virtio_is_ready() code
you mentioned. Indeed, nr_vrings gets incremented only when
receiving
<br>
vhost-user protocol requests for a new ring. This code has changed
<br>
between v16.11 and v17.11 but idea remains the same.
<br>
<br>
In he case of iPXE, it only sends requests for queues 0 & 1,
so
<br>
nr_vrings is two.
<br>
<br>
I will try with OVS to try to reproduce your issue, what OVS
version are
<br>
you using?
<br>
<br>
Thanks,
<br>
Maxime
<br>
<blockquote type="cite">Thanks!
<br>
Ladi
<br>
<br>
</blockquote>
</blockquote>
<br>
</body>
</html>