Re: 21.11.4 patches review and test

[Kevin Traynor <ktraynor@redhat.com>]

On 04/05/2023 03:13, Xu, HailinX wrote:
>> -----Original Message-----
>> From: Kevin Traynor <ktraynor@redhat.com>
>> Sent: Tuesday, May 2, 2023 5:35 PM
>> To: Xu, HailinX <hailinx.xu@intel.com>; stable@dpdk.org
>> Cc: Stokes, Ian <ian.stokes@intel.com>; Mcnamara, John
>> <john.mcnamara@intel.com>; Luca Boccassi <bluca@debian.org>; Xu, Qian Q
>> <qian.q.xu@intel.com>; Thomas Monjalon <thomas@monjalon.net>; Peng,
>> Yuan <yuan.peng@intel.com>; Chen, Zhaoyan <zhaoyan.chen@intel.com>;
>> dev@dpdk.org
>> Subject: Re: 21.11.4 patches review and test
>>
>> On 20/04/2023 11:32, Kevin Traynor wrote:
>>> On 20/04/2023 03:40, Xu, HailinX wrote:
>>>>> -----Original Message-----
>>>>> From: Xu, HailinX <hailinx.xu@intel.com>
>>>>> Sent: Thursday, April 13, 2023 2:13 PM
>>>>> To: Kevin Traynor <ktraynor@redhat.com>; stable@dpdk.org
>>>>> Cc: dev@dpdk.org; Abhishek Marathe
>> <Abhishek.Marathe@microsoft.com>;
>>>>> Ali Alnubani <alialnu@nvidia.com>; Walker, Benjamin
>>>>> <benjamin.walker@intel.com>; David Christensen
>>>>> <drc@linux.vnet.ibm.com>; Hemant Agrawal
>> <hemant.agrawal@nxp.com>;
>>>>> Stokes, Ian <ian.stokes@intel.com>; Jerin Jacob
>>>>> <jerinj@marvell.com>; Mcnamara, John <john.mcnamara@intel.com>;
>>>>> Ju-Hyoung Lee <juhlee@microsoft.com>; Luca Boccassi
>>>>> <bluca@debian.org>; Pei Zhang <pezhang@redhat.com>; Xu, Qian Q
>>>>> <qian.q.xu@intel.com>; Raslan Darawsheh <rasland@nvidia.com>;
>> Thomas
>>>>> Monjalon <thomas@monjalon.net>; yanghliu@redhat.com; Peng, Yuan
>>>>> <yuan.peng@intel.com>; Chen, Zhaoyan <zhaoyan.chen@intel.com>
>>>>> Subject: RE: 21.11.4 patches review and test
>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Kevin Traynor <ktraynor@redhat.com>
>>>>>> Sent: Thursday, April 6, 2023 7:38 PM
>>>>>> To: stable@dpdk.org
>>>>>> Cc: dev@dpdk.org; Abhishek Marathe
>>>>>> <Abhishek.Marathe@microsoft.com>; Ali Alnubani
>>>>>> <alialnu@nvidia.com>; Walker, Benjamin <benjamin.walker@intel.com>;
>>>>>> David Christensen <drc@linux.vnet.ibm.com>; Hemant Agrawal
>>>>>> <hemant.agrawal@nxp.com>; Stokes, Ian <ian.stokes@intel.com>; Jerin
>>>>>> Jacob <jerinj@marvell.com>; Mcnamara, John
>>>>>> <john.mcnamara@intel.com>; Ju-Hyoung Lee <juhlee@microsoft.com>;
>>>>>> Kevin Traynor <ktraynor@redhat.com>; Luca Boccassi
>>>>>> <bluca@debian.org>; Pei Zhang <pezhang@redhat.com>; Xu, Qian Q
>>>>>> <qian.q.xu@intel.com>; Raslan Darawsheh <rasland@nvidia.com>;
>>>>> Thomas
>>>>>> Monjalon <thomas@monjalon.net>; yanghliu@redhat.com; Peng, Yuan
>>>>>> <yuan.peng@intel.com>; Chen, Zhaoyan <zhaoyan.chen@intel.com>
>>>>>> Subject: 21.11.4 patches review and test
>>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> Here is a list of patches targeted for stable release 21.11.4.
>>>>>>
>>>>>> The planned date for the final release is 25th April.
>>>>>>
>>>>>> Please help with testing and validation of your use cases and
>>>>>> report any issues/results with reply-all to this mail. For the
>>>>>> final release the fixes and reported validations will be added to the
>> release notes.
>>>>>>
>>>>>> A release candidate tarball can be found at:
>>>>>>
>>>>>>        https://dpdk.org/browse/dpdk-stable/tag/?id=v21.11.4-rc1
>>>>>>
>>>>>> These patches are located at branch 21.11 of dpdk-stable repo:
>>>>>>        https://dpdk.org/browse/dpdk-stable/
>>>>>>
>>>>>> Thanks.
>>>>>>
>>>>>> Kevin
>>>>>
>>>>> HI All,
>>>>>
>>>>> Update the test status for Intel part. Till now dpdk21.11.4-rc1
>>>>> validation test rate is 85%. No critical issue is found.
>>>>> 2 new bugs are found, 1 new issue is under confirming by Intel Dev.
>>>>> New bugs:   --20.11.8-rc1 also has these two issues
>>>>>      1.
>> pvp_qemu_multi_paths_port_restart:perf_pvp_qemu_vector_rx_mac:
>>>>> performance drop about 23.5% when send small packets
>>>>> 	https://bugs.dpdk.org/show_bug.cgi?id=1212    -- no fix yet
>>>>>      2. some of the virtio tests are failing:    -- Intel dev is under
>> investigating
>>>>> # Basic Intel(R) NIC testing
>>>>> * Build & CFLAG compile: cover the build test combination with
>>>>> latest GCC/Clang version and the popular OS revision such as
>>>>>      Ubuntu20.04, Ubuntu22.04, Fedora35, Fedora37, RHEL8.6, RHEL8.4,
>>>>> FreeBSD13.1, SUSE15, CentOS7.9, etc.
>>>>> - All test done. No new dpdk issue is found.
>>>>> * PF(i40e, ixgbe): test scenarios including
>>>>> RTE_FLOW/TSO/Jumboframe/checksum offload/VLAN/VXLAN, etc.
>>>>> - All test done. No new dpdk issue is found.
>>>>> * VF(i40e, ixgbe): test scenarios including
>>>>> VF-RTE_FLOW/TSO/Jumboframe/checksum offload/VLAN/VXLAN, etc.
>>>>> - All test done. No new dpdk issue is found.
>>>>> * PF/VF(ice): test scenarios including Switch features/Package
>>>>> Management/Flow Director/Advanced Tx/Advanced RSS/ACL/DCF/Flexible
>>>>> Descriptor, etc.
>>>>> - All test done. No new dpdk issue is found.
>>>>> * Intel NIC single core/NIC performance: test scenarios including
>>>>> PF/VF single core performance test, etc.
>>>>> - All test done. No new dpdk issue is found.
>>>>> * IPsec: test scenarios including ipsec/ipsec-gw/ipsec library basic
>>>>> test - QAT&SW/FIB library, etc.
>>>>> - On going.
>>>>>
>>>>> # Basic cryptodev and virtio testing
>>>>> * Virtio: both function and performance test are covered. Such as
>>>>> PVP/Virtio_loopback/virtio-user loopback/virtio-net VM2VM perf
>>>>> testing/VMAWARE ESXI 8.0, etc.
>>>>> - All test done. found bug1.
>>>>> * Cryptodev:
>>>>>      *Function test: test scenarios including Cryptodev API
>>>>> testing/CompressDev ISA-L/QAT/ZLIB PMD Testing/FIPS, etc.
>>>>>        - Execution rate is 90%. found bug2.
>>>>>      *Performance test: test scenarios including Thoughput
>>>>> Performance/Cryptodev Latency, etc.
>>>>>        - All test done. No new dpdk issue is found.
>>>>>
>>>>> Regards,
>>>>> Xu, Hailin
>>>> Update the test status for Intel part. completed dpdk21.11.4-rc1 all
>> validation. No critical issue is found.
>>>
>>> Hi. Thanks for testing.
>>>
>>>> 2 new bugs are found, 1 new issue is under confirming by Intel Dev.
>>>> New bugs: --20.11.8-rc1 also has these two issues
>>>>      1.
>> pvp_qemu_multi_paths_port_restart:perf_pvp_qemu_vector_rx_mac:
>> performance drop about 23.5% when send small packets
>>>> 	https://bugs.dpdk.org/show_bug.cgi?id=1212  --not fix yet, Only the
>>>> specified platform exists
>>>
>>> Do you know which patch caaused the regression? I'm not fully clear
>>> from the Bz for 20.11. The backported patch ID'd as root cause [0] in
>>> 20.11 is in the previous releases of 20.11 (and 21.11).
>>>
>>> Trying to understand because then it would have shown in testing for
>>> previous releases. Or is this a new test introduced for latest LTS
>>> releases? and if so, what is the baseline performance based on?
>>>
>>> [0]
>>> commit 1c9a7fba5c90e0422b517404499ed106f647bcff
>>> Author: Mattias Rönnblom <mattias.ronnblom@ericsson.com>
>>> Date:   Mon Jul 11 14:11:32 2022 +0200
>>>
>>>        net: accept unaligned data in checksum routines
>>>
>>>>      2. some of the virtio tests are failing: -- Intel dev is under
>>>> investigating
>>>
>>> ok, thank you.
>>>
>>
>> Hi, it was mentioned in a separate mail that the performance drop issue was
>> not the same as 20.11.
>>
>> Is there any update on that and the failing virtio tests ? Is there a regression
>> introduced since in 21.11.4 ?
> 
> Found bad commit id:
> commit c41493361c87e730459ead9311c68528eb0874aa
> Author: Boleslav Stankevich <boleslav.stankevich@oktetlabs.ru>
> Date:   Fri Mar 3 14:19:29 2023 +0300
>      net/virtio: deduce IP length for TSO checksum
> 
> We try this issue on 3 different platforms
> Performance drop only found on Intel(R) Xeon(R) Platinum 8280M CPU @ 2.70GHz CPU.
> 

Ok, thanks for reporting. This commit was also in 23.03 and 20.11.8 so 
not sure if they were tested on this platform?

This commit fixes a functional issue, so I don't think it should be 
removed unless it is critical issue.

There's no update on other performance report. Release is already 1 week 
overdue, is there critical issues that we should hold release for?

thanks,
Kevin.

> Regards,
> Xu, Hailin
> 
>>
>> thanks,
>> Kevin,
>>
>>> Kevin.
>>>
>>>> # Basic Intel(R) NIC testing
>>>> * Build & CFLAG compile: cover the build test combination with latest
>> GCC/Clang version and the popular OS revision such as
>>>>      Ubuntu20.04, Ubuntu22.04, Fedora35, Fedora37, RHEL8.6, RHEL8.4,
>> FreeBSD13.1, SUSE15, CentOS7.9, etc.
>>>> - All test done. No new dpdk issue is found.
>>>> * PF(i40e, ixgbe): test scenarios including
>> RTE_FLOW/TSO/Jumboframe/checksum offload/VLAN/VXLAN, etc.
>>>> - All test done. No new dpdk issue is found.
>>>> * VF(i40e, ixgbe): test scenarios including
>> VF-RTE_FLOW/TSO/Jumboframe/checksum offload/VLAN/VXLAN, etc.
>>>> - All test done. No new dpdk issue is found.
>>>> * PF/VF(ice): test scenarios including Switch features/Package
>> Management/Flow Director/Advanced Tx/Advanced RSS/ACL/DCF/Flexible
>> Descriptor, etc.
>>>> - All test done. No new dpdk issue is found.
>>>> * Intel NIC single core/NIC performance: test scenarios including PF/VF
>> single core performance test, etc.
>>>> - All test done. No new dpdk issue is found.
>>>> * IPsec: test scenarios including ipsec/ipsec-gw/ipsec library basic test -
>> QAT&SW/FIB library, etc.
>>>> - All test done. No new dpdk issue is found.
>>>>
>>>> # Basic cryptodev and virtio testing
>>>> * Virtio: both function and performance test are covered. Such as
>> PVP/Virtio_loopback/virtio-user loopback/virtio-net VM2VM perf
>> testing/VMAWARE ESXI 8.0, etc.
>>>> - All test done. found bug1.
>>>> * Cryptodev:
>>>>      *Function test: test scenarios including Cryptodev API
>> testing/CompressDev ISA-L/QAT/ZLIB PMD Testing/FIPS, etc.
>>>>        - All test done. found bug2.
>>>>      *Performance test: test scenarios including Thoughput
>> Performance/Cryptodev Latency, etc.
>>>>        - All test done. No new dpdk issue is found.
>>>>
>>>> Regards,
>>>> Xu, Hailin
>>>>
>>>
>