[ipxe-devel] iPXE boot fails if multique is enabled in Openstack

Zoltan Kanizsai zoltan.kanizsai at ericsson.com
Wed Nov 22 16:06:32 UTC 2017 

Hi Ladi,

Thanks for the effort!

The main problem is not with incoming DHCP Replies, but that DHCP 
Request packets cannot be sent out from the VM by the iPXE image: TX 
errors present.
And I can see this also when using ovs-tcpdump on the vhost-user socket: 
nothing comes out from the VM trying to pxe boot.

BR,
Zoltan

On 11/22/2017 04:57 PM, Ladi Prosek wrote:
> Hi Zoltan,
>
> I'll work on setting this up on my host in the upcoming days. Adding
> Victor to cc in case this is a known issue or he knows off-hand why
> this doesn't work.
>
> In summary, this is vhost-user networking configured with mq=on but
> the simple iPXE driver uses only the first queue. Maybe incoming DHCP
> packets are delivered to one of the other queues, which is why the
> driver doesn't see them.
>
> Thanks,
> Ladi
>
> On Mon, Nov 20, 2017 at 4:44 PM, Zoltan Kanizsai
> <zoltan.kanizsai at ericsson.com> wrote:
>> Hi Ladi,
>>
>> I'm in the same team with Rafael. Here are the information you asked for:
>>
>> Qemu related package versions:
>>
>> root at blade0-7:~# apt list|grep qemu
>>
>> WARNING: apt does not have a stable CLI interface yet. Use with caution in
>> scripts.
>>
>> ipxe-qemu/trusty-updates,now 1.0.0+git-20131111.c3d1e78-2ubuntu1.1 all
>> [installed,automatic]
>> nova-compute-qemu/mos9.0-updates 2:13.1.4-7~u14.04+mos38 all
>> qemu/mos9.0-updates 1:2.5+dfsg-5ubuntu6~u1404+mos4 amd64
>> qemu-block-extra/mos9.0-updates,now 1:2.5+dfsg-5ubuntu6~u1404+mos4 amd64
>> [installed,automatic]
>> qemu-guest-agent/mos9.0-updates 1:2.5+dfsg-5ubuntu6~u1404+mos4 amd64
>> qemu-kvm/mos9.0-updates,now 1:2.5+dfsg-5ubuntu6~u1404+mos4 amd64 [installed]
>> qemu-slof/trusty-updates 20140630+dfsg-1ubuntu1~14.04 all
>> qemu-system/mos9.0-updates 1:2.5+dfsg-5ubuntu6~u1404+mos4 amd64
>> qemu-system-arm/mos9.0-updates 1:2.5+dfsg-5ubuntu6~u1404+mos4 amd64
>> qemu-system-common/mos9.0-updates,now 1:2.5+dfsg-5ubuntu6~u1404+mos4 amd64
>> [installed,automatic]
>> qemu-system-mips/mos9.0-updates 1:2.5+dfsg-5ubuntu6~u1404+mos4 amd64
>> qemu-system-misc/mos9.0-updates 1:2.5+dfsg-5ubuntu6~u1404+mos4 amd64
>> qemu-system-ppc/mos9.0-updates 1:2.5+dfsg-5ubuntu6~u1404+mos4 amd64
>> qemu-system-sparc/mos9.0-updates 1:2.5+dfsg-5ubuntu6~u1404+mos4 amd64
>> qemu-system-x86/mos9.0-updates,now 1:2.5+dfsg-5ubuntu6~u1404+mos4 amd64
>> [installed,automatic]
>> qemu-user/mos9.0-updates 1:2.5+dfsg-5ubuntu6~u1404+mos4 amd64
>> qemu-user-binfmt/mos9.0-updates 1:2.5+dfsg-5ubuntu6~u1404+mos4 amd64
>> qemu-user-static/mos9.0-updates 1:2.5+dfsg-5ubuntu6~u1404+mos4 amd64
>> qemu-utils/mos9.0-updates,now 1:2.5+dfsg-5ubuntu6~u1404+mos4 amd64
>> [installed,automatic]
>>
>>
>> Qemu command line with multiqueue enabled:
>>
>> root at blade0-7:~# ps aux|grep qemu
>> libvirt+ 18516  100  0.1 64700976 75052 ?      Sl   15:38   1:41
>> qemu-system-x86_64 -enable-kvm -name instance-00000018 -S -machine
>> pc-i440fx-wily,accel=kvm,usb=off -cpu host -m 59392 -realtime mlock=off -smp
>> 16,sockets=8,cores=1,threads=2 -object
>> memory-backend-file,id=ram-node0,prealloc=yes,mem-path=/mnt/hugepages_1GB/libvirt/qemu,share=yes,size=62277025792,host-nodes=0,policy=bind
>> -numa node,nodeid=0,cpus=0-15,memdev=ram-node0 -uuid
>> 40f79f06-fd66-47f1-a952-cb1366117c15 -smbios type=1,manufacturer=OpenStack
>> Foundation,product=OpenStack
>> Nova,version=13.1.4,serial=db887cbe-039b-453f-9eb9-4443a3ac48e5,uuid=40f79f06-fd66-47f1-a952-cb1366117c15,family=Virtual
>> Machine -no-user-config -nodefaults -chardev
>> socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-instance-00000018/monitor.sock,server,nowait
>> -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew
>> -global kvm-pit.lost_tick_policy=discard -no-hpet -no-shutdown -boot
>> strict=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive
>> file=rbd:compute/40f79f06-fd66-47f1-a952-cb1366117c15_disk:id=compute:key=AQC8Z/hZiuytBRAAS9nrU5BkLQyVnngO/1az2A==:auth_supported=cephx\;none:mon_host=192.168.1.1\:6789,format=raw,if=none,id=drive-virtio-disk0,cache=writeback
>> -device
>> virtio-blk-pci,scsi=off,bus=pci.0,addr=0x8,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1
>> -drive
>> file=rbd:compute/40f79f06-fd66-47f1-a952-cb1366117c15_disk.config:id=compute:key=AQC8Z/hZiuytBRAAS9nrU5BkLQyVnngO/1az2A==:auth_supported=cephx\;none:mon_host=192.168.1.1\:6789,format=raw,if=none,id=drive-virtio-disk25,cache=writeback
>> -device
>> virtio-blk-pci,scsi=off,bus=pci.0,addr=0x9,drive=drive-virtio-disk25,id=virtio-disk25
>> -chardev socket,id=charnet0,path=/var/run/openvswitch/vhu5ff7901d-17 -netdev
>> type=vhost-user,id=hostnet0,chardev=charnet0,queues=16 -device
>> virtio-net-pci,mq=on,vectors=34,netdev=hostnet0,id=net0,mac=02:10:20:10:03:00,bus=pci.0,addr=0x3
>> -chardev socket,id=charnet1,path=/var/run/openvswitch/vhu6137cefb-a5 -netdev
>> type=vhost-user,id=hostnet1,chardev=charnet1,queues=16 -device
>> virtio-net-pci,mq=on,vectors=34,netdev=hostnet1,id=net1,mac=02:10:20:10:03:02,bus=pci.0,addr=0x4
>> -chardev socket,id=charnet2,path=/var/run/openvswitch/vhu98d81e14-60 -netdev
>> type=vhost-user,id=hostnet2,chardev=charnet2,queues=16 -device
>> virtio-net-pci,mq=on,vectors=34,netdev=hostnet2,id=net2,mac=02:10:20:10:03:04,bus=pci.0,addr=0x5
>> -chardev socket,id=charnet3,path=/var/run/openvswitch/vhu38230b89-15 -netdev
>> type=vhost-user,id=hostnet3,chardev=charnet3,queues=16 -device
>> virtio-net-pci,mq=on,vectors=34,netdev=hostnet3,id=net3,mac=02:10:20:10:03:05,bus=pci.0,addr=0x6
>> -chardev socket,id=charnet4,path=/var/run/openvswitch/vhu173ca317-4f -netdev
>> type=vhost-user,id=hostnet4,chardev=charnet4,queues=16 -device
>> virtio-net-pci,mq=on,vectors=34,netdev=hostnet4,id=net4,mac=02:10:20:10:03:06,bus=pci.0,addr=0x7
>> -chardev
>> file,id=charserial0,path=/var/lib/nova/instances/40f79f06-fd66-47f1-a952-cb1366117c15/console.log
>> -device isa-serial,chardev=charserial0,id=serial0 -chardev
>> pty,id=charserial1 -device isa-serial,chardev=charserial1,id=serial1 -device
>> usb-tablet,id=input0 -vnc 0.0.0.0:0 -k en-us -device
>> cirrus-vga,id=video0,bus=pci.0,addr=0x2 -device
>> i6300esb,id=watchdog0,bus=pci.0,addr=0xb -watchdog-action reset -device
>> virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0xa -msg timestamp=on
>> root at blade0-7:~#
>>
>>
>> Without multiqueue:
>>
>> root at blade0-7:~# ps aux|grep qemu
>> libvirt+ 12808  277  0.3 65044396 201752 ?     Sl   Nov16 17003:49
>> qemu-system-x86_64 -enable-kvm -name instance-00000018 -S -machine
>> pc-i440fx-wily,accel=kvm,usb=off -cpu host -m 59392 -realtime mlock=off -smp
>> 16,sockets=8,cores=1,threads=2 -object
>> memory-backend-file,id=ram-node0,prealloc=yes,mem-path=/mnt/hugepages_1GB/libvirt/qemu,share=yes,size=62277025792,host-nodes=0,policy=bind
>> -numa node,nodeid=0,cpus=0-15,memdev=ram-node0 -uuid
>> 40f79f06-fd66-47f1-a952-cb1366117c15 -smbios type=1,manufacturer=OpenStack
>> Foundation,product=OpenStack
>> Nova,version=13.1.4,serial=db887cbe-039b-453f-9eb9-4443a3ac48e5,uuid=40f79f06-fd66-47f1-a952-cb1366117c15,family=Virtual
>> Machine -no-user-config -nodefaults -chardev
>> socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-instance-00000018/monitor.sock,server,nowait
>> -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew
>> -global kvm-pit.lost_tick_policy=discard -no-hpet -no-shutdown -boot
>> strict=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive
>> file=rbd:compute/40f79f06-fd66-47f1-a952-cb1366117c15_disk:id=compute:key=AQC8Z/hZiuytBRAAS9nrU5BkLQyVnngO/1az2A==:auth_supported=cephx\;none:mon_host=192.168.1.1\:6789,format=raw,if=none,id=drive-virtio-disk0,cache=writeback
>> -device
>> virtio-blk-pci,scsi=off,bus=pci.0,addr=0x8,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1
>> -drive
>> file=rbd:compute/40f79f06-fd66-47f1-a952-cb1366117c15_disk.config:id=compute:key=AQC8Z/hZiuytBRAAS9nrU5BkLQyVnngO/1az2A==:auth_supported=cephx\;none:mon_host=192.168.1.1\:6789,format=raw,if=none,id=drive-virtio-disk25,cache=writeback
>> -device
>> virtio-blk-pci,scsi=off,bus=pci.0,addr=0x9,drive=drive-virtio-disk25,id=virtio-disk25
>> -chardev socket,id=charnet0,path=/var/run/openvswitch/vhu5ff7901d-17 -netdev
>> type=vhost-user,id=hostnet0,chardev=charnet0 -device
>> virtio-net-pci,netdev=hostnet0,id=net0,mac=02:10:20:10:03:00,bus=pci.0,addr=0x3
>> -chardev socket,id=charnet1,path=/var/run/openvswitch/vhu6137cefb-a5 -netdev
>> type=vhost-user,id=hostnet1,chardev=charnet1 -device
>> virtio-net-pci,netdev=hostnet1,id=net1,mac=02:10:20:10:03:02,bus=pci.0,addr=0x4
>> -chardev socket,id=charnet2,path=/var/run/openvswitch/vhu98d81e14-60 -netdev
>> type=vhost-user,id=hostnet2,chardev=charnet2 -device
>> virtio-net-pci,netdev=hostnet2,id=net2,mac=02:10:20:10:03:04,bus=pci.0,addr=0x5
>> -chardev socket,id=charnet3,path=/var/run/openvswitch/vhu38230b89-15 -netdev
>> type=vhost-user,id=hostnet3,chardev=charnet3 -device
>> virtio-net-pci,netdev=hostnet3,id=net3,mac=02:10:20:10:03:05,bus=pci.0,addr=0x6
>> -chardev socket,id=charnet4,path=/var/run/openvswitch/vhu173ca317-4f -netdev
>> type=vhost-user,id=hostnet4,chardev=charnet4 -device
>> virtio-net-pci,netdev=hostnet4,id=net4,mac=02:10:20:10:03:06,bus=pci.0,addr=0x7
>> -chardev
>> file,id=charserial0,path=/var/lib/nova/instances/40f79f06-fd66-47f1-a952-cb1366117c15/console.log
>> -device isa-serial,chardev=charserial0,id=serial0 -chardev
>> pty,id=charserial1 -device isa-serial,chardev=charserial1,id=serial1 -device
>> usb-tablet,id=input0 -vnc 0.0.0.0:0 -k en-us -device
>> cirrus-vga,id=video0,bus=pci.0,addr=0x2 -device
>> i6300esb,id=watchdog0,bus=pci.0,addr=0xb -watchdog-action reset -device
>> virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0xa -msg timestamp=on
>> root at blade0-7:~#
>>
>> BR,
>> Zoltan
>>
>>
>> ________________________________
>> From: Ladi Prosek <lprosek at redhat.com>
>> Sent: Tuesday, November 14, 2017 7:58 AM
>> To: Rafael Gellert
>> Cc: ipxe-devel at lists.ipxe.org
>> Subject: Re: [ipxe-devel] iPXE boot fails if multique is enabled in
>> Openstack
>>
>> Hi,
>>
>> On Mon, Nov 13, 2017 at 8:33 AM, Rafael Gellert
>> <rafael.gellert at ericsson.com> wrote:
>>> Hi,
>>>
>>> Could you please help me resolve this issue:
>>> http://forum.ipxe.org/showthread.php?tid=10521
>> Can you please post the QEMU command line with and without the
>> problematic hw_vif_multiqueue_enabled="true" option. Also QEMU version
>> running on the host.
>>
>> Thank you,
>> Ladi
>>
>>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ipxe.org/pipermail/ipxe-devel/attachments/20171122/0ac8c18c/attachment.htm>

More information about the ipxe-devel mailing list