Re: [dpdk-dev] [EXTERNAL] Re: 18.11.6 (LTS) patches review and test

[Ju-Hyoung Lee <juhlee@microsoft.com>]

I agree with Kevin. We should investigate the previous performance issue. Lili, please add yourself to the list.
Thanks,

Ju

-----Original Message-----
From: Kevin Traynor <ktraynor@redhat.com> 
Sent: Thursday, January 9, 2020 6:03 AM
To: Lili Deng <Lili.Deng@microsoft.com>
Cc: stable@dpdk.org; dev@dpdk.org; Abhishek Marathe <Abhishek.Marathe@microsoft.com>; Akhil Goyal <akhil.goyal@nxp.com>; Ali Alnubani <alialnu@mellanox.com>; benjamin walker <benjamin.walker@intel.com>; David Christensen <drc@linux.vnet.ibm.com>; Hemant Agrawal <hemant.agrawal@nxp.com>; Ian Stokes <ian.stokes@intel.com>; Jerin Jacob <jerinj@marvell.com>; John McNamara <john.mcnamara@intel.com>; Ju-Hyoung Lee <juhlee@microsoft.com>; Luca Boccassi <bluca@debian.org>; pingx yu <pingx.yu@intel.com>; qian q xu <qian.q.xu@intel.com>; Raslan Darawsheh <rasland@mellanox.com>; Thomas Monjalon <thomas@monjalon.net>; yuan peng <yuan.peng@intel.com>; zhaoyan chen <zhaoyan.chen@intel.com>
Subject: [EXTERNAL] Re: 18.11.6 (LTS) patches review and test

On 30/12/2019 08:56, Lili Deng wrote:
> Hi Kevin,
> 
> 
> 
> I'd like to sign off validation dpdk-stable-18.11.6-rc1 against Azure gallery images.
> 
> Version used - 
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.
> dpdk.org%2Fdpdk-stable%2Fsnapshot%2Fdpdk-stable-18.11.6-rc1.tar.xz&amp
> ;data=02%7C01%7Cjuhlee%40microsoft.com%7C11305b9ae1784038fc5108d7950cb
> 93e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637141754201490209&am
> p;sdata=Mn2CcjMLsDhIo8tPkwrcly5sJPIugOu7L0%2FDi%2FszHDU%3D&amp;reserve
> d=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2F
> git.dpdk.org%2Fdpdk-stable%2Fsnapshot%2Fdpdk-stable-18.11.6-rc1.tar.xz
> &amp;data=02%7C01%7Cjuhlee%40microsoft.com%7C11305b9ae1784038fc5108d79
> 50cb93e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C63714175420149020
> 9&amp;sdata=Mn2CcjMLsDhIo8tPkwrcly5sJPIugOu7L0%2FDi%2FszHDU%3D&amp;res
> erved=0>
> 
> 
> 
> Below are test matrix and results -
> 
> Tests
> 
> Ubuntu 16.04
> 
> Ubuntu 18.04
> 
> Ubuntu 19.10
> 
> RHEL 7-RAW
> 
> SLES 15
> 
> RHEL 7.5
> 
> CentOS 7.5
> 
> RHEL 8
> 
> CentOS 8
> 
> VERIFY-DPDK-FAILSAFE-DURING-TRAFFIC
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PASS
> 
> VERIFY-DPDK-BUILD-AND-TESTPMD-TEST
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PASS
> 
> VERIFY-SRIOV-FAILSAFE-FOR-DPDK
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PASS
> 
> VERIFY-DPDK-COMPLIANCE
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PASS
> 
> VERIFY-DPDK-OVS
> 
> PASS
> 
> PASS
> 
> PASS
> 
> SKIPPED
> 
> SKIPPED
> 
> SKIPPED
> 
> SKIPPED
> 
> SKIPPED
> 
> SKIPPED
> 
> VERIFY-DPDK-RING-LATENCY
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PERF-DPDK-FWD-PPS-DS15
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PERF-DPDK-SINGLE-CORE-PPS-DS4
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PERF-DPDK-SINGLE-CORE-PPS-DS15
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PERF-DPDK-MULTICORE-PPS-DS15
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PASS
> 
> PERF-DPDK-MULTICORE-PPS-F32
> 
> PASS
> 
> PASS
> 
> PASS
> 
> FAIL
> 
> PASS
> 
> PASS
> 
> FAIL
> 
> PASS
> 
> FAIL
> 
> 
> 
> FAIL reason for PERF-DPDK-MULTICORE-PPS-F32 - the performance is not expected in some rate.
> 
> For example, when run 3 times, only 1/3 pass, maybe the size Standard_F32s_v2 on Azure is unstable.
> 
> 

Thanks Lili. I think this is ok. It seems this test has been reporting fails in previous testing cycles too. Perhaps in future the thresholds could be changed so it will check if there are regressions?

fyi - if you want to be added to the list of validation contacts for dpdk stable (https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.dpdk.org%2Ftools%2Fstable-scripts%2Ftree%2Fvalidation-contacts&amp;data=02%7C01%7Cjuhlee%40microsoft.com%7C11305b9ae1784038fc5108d7950cb93e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637141754201490209&amp;sdata=KPG2Jl2C3cDNDHmR3QN5DGurbQ9O%2BtjEN4R5D0DQvUE%3D&amp;reserved=0),
let me know. Not a requirement, just to make it easier for being notified about an RC's etc.

Kevin.

> 
> Thanks,
> 
> Lili
> 


<snip>