Re: [dpdk-dev] [PATCH v4] vhost: Add indirect descriptors support to the TX path

[Maxime Coquelin <maxime.coquelin@redhat.com>]

On 11/04/2016 12:36 PM, Yuanhan Liu wrote:
> On Fri, Nov 04, 2016 at 12:22:47PM +0100, Maxime Coquelin wrote:
>>
>>
>>>>>> Hi Maxime,
>>>>>>
>>>>>> I did a little more macswap test and found out more stuff here:
>>>>> Thanks for doing more tests.
>>>>>
>>>>>>
>>>>>> 1. I did loopback test on another HSW machine with the same H/W,
>>>>>>    and indirect_desc on and off seems have close perf
>>>>>>
>>>>>> 2. So I checked the gcc version:
>>>>>>
>>>>>>     *  Previous: gcc version 6.2.1 20160916 (Fedora 24)
>>>>>>
>>>>>>     *  New: gcc version 5.4.0 20160609 (Ubuntu 16.04.1 LTS)
>>>>>
>>>>> On my side, I tested with RHEL7.3:
>>>>> - gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-11)
>>>>>
>>>>> It certainly contains some backports from newer GCC versions.
>>>>>
>>>>>>
>>>>>>    On previous one indirect_desc has 20% drop
>>>>>>
>>>>>> 3. Then I compiled binary on Ubuntu and scp to Fedora, and as
>>>>>>    expected I got the same perf as on Ubuntu, and the perf gap
>>>>>>    disappeared, so gcc is definitely one factor here
>>>>>>
>>>>>> 4. Then I use the Ubuntu binary on Fedora for PVP test, then the
>>>>>>    perf gap comes back again and the same with the Fedora binary
>>>>>>    results, indirect_desc causes about 20% drop
>>>>>
>>>>> Let me know if I understand correctly:
>>>
>>> Yes, and it's hard to breakdown further at this time.
>>>
>>> Also we may need to check whether it's caused by certain NIC
>>> model. Unfortunately I don't have the right setup right now.
>>>
>>>>> Loopback test with macswap:
>>>>> - gcc version 6.2.1 : 20% perf drop
>>>>> - gcc version 5.4.0 : No drop
>>>>>
>>>>> PVP test with macswap:
>>>>> - gcc version 6.2.1 : 20% perf drop
>>>>> - gcc version 5.4.0 : 20% perf drop
>>>>
>>>> I forgot to ask, did you recompile only host, or both host and guest
>>>> testmpd's in your test?
>>
>>> Both.
>>
>> I recompiled testpmd on a Fedora 24 machine using GCC6:
>
> Have you built host DPDK with gcc6 as well?

Yes, I use the same build based on GCC6 on both sides.

>
> 	--yliu
>
>> gcc (GCC) 6.1.1 20160621 (Red Hat 6.1.1-3)
>> Testing loopback with macswap on my Haswell RHEL7.3 machine gives me the
>> following results:
>>  - indirect on: 7.75Mpps
>>  - indirect off: 7.35Mpps
>>
>> Surprisingly, I get better results with indirect on my setup (I
>> reproduced the tests multiple times).
>>
>> Do you have a document explaining the tuning/config you apply to both the
>> host and the guest (isolation, HT, hugepage size, ...) in your
>> setup?
>>
>> Regards,
>> Maxime