From: Marcel Apfelbaum <marcel@redhat.com>
To: Yuanhan Liu <yuanhan.liu@linux.intel.com>
Cc: dev@dpdk.org, "Michael S. Tsirkin" <mst@redhat.com>
Subject: Re: [dpdk-dev] [PATCH v5 resend 03/12] vhost: vring queue setup for multiple queue support
Date: Tue, 22 Sep 2015 11:47:34 +0300 [thread overview]
Message-ID: <560115A5.2020602@redhat.com> (raw)
In-Reply-To: <20150922083416.GW2339@yliu-dev.sh.intel.com>
On 09/22/2015 11:34 AM, Yuanhan Liu wrote:
> On Tue, Sep 22, 2015 at 11:10:13AM +0300, Marcel Apfelbaum wrote:
>> On 09/22/2015 10:31 AM, Yuanhan Liu wrote:
>>> On Mon, Sep 21, 2015 at 08:56:30PM +0300, Marcel Apfelbaum wrote:
>> [...]
>>>>>
>>>>> Hi,
>>>>>
>>>>> I have made 4 cleanup patches few weeks before, including the patch
>>>>> to define kickfd and callfd as int type, and they have already got
>>>>> the ACK from Huawei Xie, and Chuangchun Ouyang. It's likely that
>>>>> they will be merged, hence I made this patchset based on them.
>>>>>
>>>>> This will also answer the question from your another email: can't
>>>>> apply.
>>>>
>>>> Hi,
>>>> Thank you for the response, it makes sense now.
>>>>
>>>> T have another issue, maybe you can help.
>>>> I have some problems making it work with OVS/DPDK backend and virtio-net driver in guest.
>>>>
>>>> I am using a simple setup:
>>>> http://wiki.qemu.org/Features/vhost-user-ovs-dpdk
>>>> that connects 2 VMs using OVS's dpdkvhostuser ports (regular virtio-net driver in guest, not the PMD driver).
>>>>
>>>> The setup worked fine with the prev DPDK MQ implementation (V4), however on this one the traffic stops
>>>> once I set queues=n in guest.
>>>
>>> Hi,
>>>
>>> Could you be more specific about that? It also would be helpful if you
>>> could tell me the steps, besides those setup steps you mentioned in the
>>> qemu wiki and this email, you did for testing.
>>>
>>
>> Hi,
>> Thank you for your help.
>>
>> I am sorry the wiki is not enough, I'll be happy to add all the missing parts.
>> In the meantime maybe you can tell me where the problem is, I also suggest to
>> post here the output of journalctl command.
>>
>> We only need a regular machine and we want traffic between 2 VMs. I'll try to summarize the steps:
>>
>> 1. Be sure you have enough hugepages enabled (2M pages are enough) and mounted.
>> 2. Configure and start OVS following the wiki
>> - we only want one bridge with 2 dpdkvhostuser ports.
>> 3. Start VMs using the wiki command line
>> - check journalctl for possible errors. You can use
>> journalctl --since `date +%T --date="-10 minutes"`
>> to see only last 10 minutes.
>> 4. Configure the guests IPs.
>> - Disable the Network Manager as described bellow in the mail.
>> 5. At this point you should be able to ping between guests.
>>
>> Please let me know if you have any problem until this point.
>> I'll be happy to help. Please point any special steps you made that
>> are not in the WIKI. The journalctl logs would also help.
>>
>> Does the ping between VMS work now?
>
> Yes, it works, too. I can ping the other vm inside a vm.
>
> [root@dpdk-kvm ~]# ethtool -l eth0
> Channel parameters for eth0:
> Pre-set maximums:
> RX: 0
> TX: 0
> Other: 0
> Combined: 2
> Current hardware settings:
> RX: 0
> TX: 0
> Other: 0
> Combined: 2
>
> [root@dpdk-kvm ~]# ifconfig eth0
> eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
> inet 192.168.100.11 netmask 255.255.255.0 broadcast 192.168.100.255
> inet6 fe80::5054:ff:fe12:3459 prefixlen 64 scopeid 0x20<link>
> ether 52:54:00:12:34:59 txqueuelen 1000 (Ethernet)
> RX packets 56 bytes 5166 (5.0 KiB)
> RX errors 0 dropped 0 overruns 0 frame 0
> TX packets 84 bytes 8303 (8.1 KiB)
> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
>
> [root@dpdk-kvm ~]# ping 192.168.100.10
> PING 192.168.100.10 (192.168.100.10) 56(84) bytes of data.
> 64 bytes from 192.168.100.10: icmp_seq=1 ttl=64 time=0.213 ms
> 64 bytes from 192.168.100.10: icmp_seq=2 ttl=64 time=0.094 ms
> 64 bytes from 192.168.100.10: icmp_seq=3 ttl=64 time=0.246 ms
> 64 bytes from 192.168.100.10: icmp_seq=4 ttl=64 time=0.153 ms
> 64 bytes from 192.168.100.10: icmp_seq=5 ttl=64 time=0.104 ms
> ^C
>>
>> If yes, please let me know and I'll go over MQ enabling.
>
> I'm just wondering why it doesn't work on your side.
Hi,
This is working also for me, but without enabling the MQ. (ethtool -L eth0 combined n (n>1) )
The problem starts when I am applying the patches and I enable MQ. (Need a slightly different QEMU commandline)
>
>>
>>> I had a very rough testing based on your test guides, I indeed found
>>> an issue: the IP address assigned by "ifconfig" disappears soon in the
>>> first few times and after about 2 or 3 times reset, it never changes.
>>>
>>> (well, I saw that quite few times before while trying different QEMU
>>> net devices. So, it might be a system configuration issue, or something
>>> else?)
>>>
>>
>> You are right, this is a guest config issue, I think you should disable NetworkManager
>
> Yeah, I figured it out by my self, and it worked when I hardcoded it at
> /etc/sysconfig/network-scripts/ifcfg-eth0.
>
>> for static IP addresses. Please use only the virtio-net device.
>>
>> You cant try this:
>> sudo systemctl stop NetworkManager
>> sudo systemctl disable NetworkManager
>
> Thanks for the info and tip!
>
>>
>>> Besides that, it works, say, I can wget a big file from host.
>>>
>>
>> The target here is traffic between 2 VMs.
>> We want to be able to ping (for example) between VMS when MQ > 1 is enabled on both guests:
>> - ethtool -L eth0 combined <queues nr, the same as QEMU>
>
> As you can see from my command log, I did so and it worked :)
>
Let me understand, it worked after applying MQ patches on all 3 projects (DPDK, QEMU and OVS)?
It worked with MQ enabled? MQ >1 ?
You can be sure by using the following command in one of the VMs:
cat /proc/interrupts | grep virtio
and see that you have interrupts for all virtio0-input.0/1/...
Thanks,
Marcel
>>
>> Thank you again for the involvement, this is very much appreciated!
>
> Welcome! I need fix it if there is a bug.
>
> --yliu
>
next prev parent reply other threads:[~2015-09-22 8:47 UTC|newest]
Thread overview: 47+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-09-18 15:10 [dpdk-dev] [PATCH v5 resend 00/12] vhost-user multiple queues enabling Yuanhan Liu
2015-09-18 15:10 ` [dpdk-dev] [PATCH v5 resend 01/12] vhost-user: add protocol features support Yuanhan Liu
2015-09-18 15:10 ` [dpdk-dev] [PATCH v5 resend 02/12] vhost-user: add VHOST_USER_GET_QUEUE_NUM message Yuanhan Liu
2015-09-18 15:10 ` [dpdk-dev] [PATCH v5 resend 03/12] vhost: vring queue setup for multiple queue support Yuanhan Liu
2015-09-20 13:58 ` Marcel Apfelbaum
2015-09-21 2:06 ` Yuanhan Liu
2015-09-21 17:56 ` Marcel Apfelbaum
2015-09-22 7:31 ` Yuanhan Liu
2015-09-22 8:10 ` Marcel Apfelbaum
2015-09-22 8:34 ` Yuanhan Liu
2015-09-22 8:47 ` Marcel Apfelbaum [this message]
2015-09-22 9:21 ` Yuanhan Liu
2015-09-22 10:06 ` Marcel Apfelbaum
2015-09-22 14:22 ` Yuanhan Liu
2015-09-22 14:51 ` Marcel Apfelbaum
2015-09-23 3:46 ` Yuanhan Liu
2015-09-24 9:51 ` Marcel Apfelbaum
2015-09-21 9:07 ` Michael S. Tsirkin
2015-09-18 15:10 ` [dpdk-dev] [PATCH v5 resend 04/12] vhost: rxtx: prepare work " Yuanhan Liu
2015-09-20 9:29 ` Michael S. Tsirkin
2015-09-21 2:25 ` Yuanhan Liu
2015-09-21 9:04 ` Michael S. Tsirkin
2015-09-22 2:54 ` Yuanhan Liu
2015-09-18 15:10 ` [dpdk-dev] [PATCH v5 resend 05/12] vhost: add VHOST_USER_SET_VRING_ENABLE message Yuanhan Liu
2015-09-20 9:37 ` Michael S. Tsirkin
2015-09-21 2:22 ` Yuanhan Liu
2015-09-21 9:02 ` Michael S. Tsirkin
2015-09-22 2:21 ` Yuanhan Liu
2015-09-23 8:43 ` Yuanhan Liu
2015-09-18 15:10 ` [dpdk-dev] [PATCH v5 resend 06/12] vhost-user: handle VHOST_USER_RESET_OWNER correctly Yuanhan Liu
2015-09-18 15:10 ` [dpdk-dev] [PATCH v5 resend 07/12] virtio: resolve for control queue Yuanhan Liu
2015-09-20 9:21 ` Michael S. Tsirkin
2015-09-21 6:36 ` Yuanhan Liu
2015-10-08 15:32 ` Nikita Kalyazin
2015-10-08 20:51 ` Steffen Bauch
2015-10-09 7:11 ` Nikita Kalyazin
2015-10-12 8:39 ` Yuanhan Liu
2015-10-12 9:31 ` Xie, Huawei
2015-10-12 9:56 ` Xie, Huawei
2015-10-12 20:58 ` Steffen Bauch
2015-10-13 9:54 ` Yuanhan Liu
2015-09-18 15:10 ` [dpdk-dev] [PATCH v5 resend 08/12] vhost-user: enable vhost-user multiple queue Yuanhan Liu
2015-09-18 15:10 ` [dpdk-dev] [PATCH v5 resend 09/12] vhost: add API bind a virtq to a specific core Yuanhan Liu
2015-09-18 15:10 ` [dpdk-dev] [PATCH v5 resend 10/12] ixgbe: support VMDq RSS in non-SRIOV environment Yuanhan Liu
2015-09-18 15:11 ` [dpdk-dev] [PATCH v5 resend 11/12] examples/vhost: demonstrate the usage of vhost mq feature Yuanhan Liu
2015-09-18 15:11 ` [dpdk-dev] [PATCH v5 resend 12/12] examples/vhost: add per queue stats Yuanhan Liu
2015-09-20 11:48 ` [dpdk-dev] [PATCH v5 resend 00/12] vhost-user multiple queues enabling Marcel Apfelbaum
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=560115A5.2020602@redhat.com \
--to=marcel@redhat.com \
--cc=dev@dpdk.org \
--cc=mst@redhat.com \
--cc=yuanhan.liu@linux.intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).