Re: [dpdk-dev] [PATCH v5 resend 03/12] vhost: vring queue setup for multiple queue support

[Marcel Apfelbaum <marcel@redhat.com>]

On 09/22/2015 10:31 AM, Yuanhan Liu wrote:
> On Mon, Sep 21, 2015 at 08:56:30PM +0300, Marcel Apfelbaum wrote:
[...]
>>>
>>> Hi,
>>>
>>> I have made 4 cleanup patches few weeks before, including the patch
>>> to define kickfd and callfd as int type, and they have already got
>>> the ACK from Huawei Xie, and Chuangchun Ouyang. It's likely that
>>> they will be merged, hence I made this patchset based on them.
>>>
>>> This will also answer the question from your another email: can't
>>> apply.
>>
>> Hi,
>> Thank you for the response, it makes sense now.
>>
>> T have another issue, maybe you can help.
>> I have some problems making it work with OVS/DPDK backend and virtio-net driver in guest.
>>
>> I am using a simple setup:
>>      http://wiki.qemu.org/Features/vhost-user-ovs-dpdk
>> that connects 2 VMs using OVS's dpdkvhostuser ports (regular virtio-net driver in guest, not the PMD driver).
>>
>> The setup worked fine with the prev DPDK MQ implementation (V4), however on this one the traffic stops
>> once I set queues=n in guest.
>
> Hi,
>
> Could you be more specific about that? It also would be helpful if you
> could tell me the steps, besides those setup steps you mentioned in the
> qemu wiki and this email, you did for testing.
>

Hi,
Thank you for your help.

I am sorry the wiki is not enough, I'll be happy to add all the missing parts.
In the meantime maybe you can tell me where the problem is, I also suggest to
post here the output of journalctl command.

We only need a regular machine and we want traffic between 2 VMs. I'll try to summarize the steps:

1. Be sure you have enough hugepages enabled (2M pages are enough) and mounted.
2. Configure and start OVS following the wiki
    - we only want one bridge with 2 dpdkvhostuser ports.
3. Start VMs using the wiki command line
    - check journalctl for possible errors. You can use
         journalctl  --since `date +%T --date="-10 minutes"`
      to see only last 10 minutes.
4. Configure the guests IPs.
    - Disable the Network Manager as described bellow in the mail.
5. At this point you should be able to ping between guests.

Please let me know if you have any problem until this point.
I'll be happy to help. Please point any special steps you made that
are not in the WIKI. The journalctl logs would also help.

Does the ping between VMS work now?

If yes, please let me know and I'll go over MQ enabling.

> I had a very rough testing based on your test guides, I indeed found
> an issue: the IP address assigned by "ifconfig" disappears soon in the
> first few times and after about 2 or 3 times reset, it never changes.
>
> (well, I saw that quite few times before while trying different QEMU
> net devices. So, it might be a system configuration issue, or something
> else?)
>

You are right, this is a guest config issue, I think you should disable NetworkManager
for static IP addresses. Please use only the virtio-net device.

You cant try this:
sudo systemctl stop NetworkManager
sudo systemctl disable NetworkManager


> Besides that, it works, say, I can wget a big file from host.
>

The target here is traffic between 2 VMs.
We want to be able to ping (for example) between VMS when MQ > 1 is enabled on both guests:
- ethtool -L eth0 combined <queues nr, the same as QEMU>

Thank you again for the involvement, this is very much appreciated!
Marcel

> 	--yliu
>
>> (virtio-net uses only one queue when the guest starts, even if QEMU has multiple queues).
>>
>> Two steps are required in order to enable multiple queues in OVS.
>> 1. Apply the following patch:
>>   - https://www.mail-archive.com/dev@openvswitch.org/msg49198.html
>>   - It needs merging (I think)
>> 2. Configure ovs for multiqueue:
>>   - ovs-vsctl set Open_vSwitch . other_config:n-dpdk-rxqs=<queues nr, the same as QEMU>
>>   - ovs-vsctl set Open_vSwitch . other_config:pmd-cpu-mask=<cpu mask for rx queues, say 0xff00>
>> 3. In order to set queues=n in guest use:
>>   - ethtool -L eth0 combined <queues nr, the same as QEMU>
>>
>> Any pointers/ideas would be appreciated.
>>
>> Thank you,
>> Marcel
>>
>>
>>
>>>
[...]