Re: [dpdk-dev] [dpdk-announce] release candidate 19.11-rc4

["Xu, Qian Q" <qian.q.xu@intel.com>]

Hi, all
Below is the report from Intel validation team. 
Summary: 
1.  Totally 9852 test cases for 19.11 RC4, 9852 cases are executed, 9819 test cases passed, 33 test cases failed. Execution rate is 100%, pass rate is 99.67%.  
2.  All Intel build tests(30 different configurations) passed. No critical/high issues for RC4. 
3.   Totally 405 new tests are added for the 19.11 new features. 

Coverage for the different OS/NIC/configurations. 
Please check the full test plan at http://doc.dpdk.org/dts/test_plans/  and automation test suites at https://git.dpdk.org/tools/dts/tree/tests

NIC PMD: Covered Intel NICs. Below are some test list for reference. All test 
TestSuite_blacklist.py
TestSuite_checksum_offload.py
TestSuite_cmdline.py
TestSuite_ddp_gtp.py
TestSuite_ddp_gtp_qregion.py
TestSuite_ddp_mpls.py
TestSuite_ddp_ppp_l2tp.py
TestSuite_dual_vlan.py
TestSuite_dynamic_config.py
TestSuite_dynamic_flowtype.py
TestSuite_dynamic_queue.py
TestSuite_flow_classify.py
TestSuite_flow_filtering.py
TestSuite_fortville_rss_granularity_config.py
TestSuite_generic_filter.py
TestSuite_generic_flow_api.py
TestSuite_hotplug.py
TestSuite_iavf.py
TestSuite_jumboframes.py
TestSuite_link_flowctrl.py
TestSuite_link_status_interrupt.py
TestSuite_mac_filter.py
TestSuite_mdd.py
TestSuite_multicast.py
TestSuite_veb_switch.py
TestSuite_vf_interrupt_pmd.py
TestSuite_vf_jumboframe.py
TestSuite_vf_kernel.py
TestSuite_vf_l3fwd.py
TestSuite_vf_macfilter.py
TestSuite_vf_offload.py
TestSuite_vf_packet_rxtx.py
TestSuite_vf_port_start_stop.py
TestSuite_vf_rss.py
TestSuite_vf_vlan.py

Vhost/Virtio regression test is passed, below is the major test list. 
pvp_multi_paths_vhost_single_core_performance
pvp_multi_paths_virtio_single_core_performance
pvp_multi_paths_performance
pvp_qemu_multi_paths_port_restart
pvp_diff_qemu_version
vhost_multi_queue_qemu
vhost_virtio_pmd_interrupt
virtio_event_idx_interrupt
vhost_dequeue_zero_copy
pvp_virtio_user_multi_queues

Cryptodev/Compression regression test is passed, below is the major test list.
unit_tests_cryptodev_func
ipsec_gw_cryptodev_func
fips_cryptodev
l2fwd-crypto_cryptodev_func
crypto-perf_cryptodev_perf
virtio_ipsec_cryptodev_func
virtio_unit_cryptodev_func
virtio_perf_cryptodev_func
QAT compression PMD
ISA_L Compression PMD
Zlib Compression PMD
Compressdev QAT 

Other Packet processing libraries test such as Power, Metrics, HQoS, are also tested. 
DPDK PMD for AF_XDP_test
ntb_test_plan
cbdma_test_plan
rte_security AESNI 
rte_security AESNI 
IPsec: Security Association Database (SAD)
IPsec: Support multiple crypto processing modes for the same SA
Qos_sched
Ip_pipeline
softnic


> -----Original Message-----
> From: dev <dev-bounces@dpdk.org> On Behalf Of Luca Boccassi
> Sent: Thursday, November 28, 2019 5:51 PM
> To: dev@dpdk.org
> Cc: thomas@monjalon.net; Ju-Hyoung Lee <juhlee@microsoft.com>; Abhishek
> Marathe <Abhishek.Marathe@microsoft.com>
> Subject: Re: [dpdk-dev] [dpdk-announce] release candidate 19.11-rc4
> 
> Hi,
> 
> Here's the report from the automated Jenkins pipeline at MSFT - QA team will be
> out for Thanksgiving this week, they can chip in next week for more details, but I
> do not see any blocking issue being highlighted - just the usual small
> performance issue that is already being investigated separately.
> 
> 
> DPDK v19.11-rc4 was validated on Azure for Canonical UbuntuServer 16.04-LTS
> latest, Canonical UbuntuServer 18.04-DAILY-LTS latest, RedHat RHEL 7-RAW
> latest, RedHat RHEL 7.5 latest, Openlogic CentOS 7.5 latest, SUSE SLES 15 latest.
> Tested with Mellanox and netvsc poll-mode drivers.
> The tests were executed using LISAv2 framework
> ( https://github.com/LIS/LISAv2 ).
> 
> Test case description:
> 
> * VERIFY-DPDK-COMPLIANCE - verifies kernel is supported and that the build is
> successful
> * VERIFY-DPDK-BUILD-AND-TESTPMD-TEST - verifies using testpmd that packets
> can be sent from a VM to another VM
> * VERIFY-SRIOV-FAILSAFE-FOR-DPDK - disables/enables Accelerated Networking
> for the NICs under test and makes sure DPDK works in both scenarios
> * VERIFY-DPDK-FAILSAFE-DURING-TRAFFIC - disables/enables Accelerated
> Networking for the NICs while generating traffic using testpmd
> 
> * PERF-DPDK-FWD-PPS-DS15 - verifies DPDK forwarding performance using
> testpmd on 2, 4, 8 cores, rx and io mode on size Standard_DS15_v2
> * PERF-DPDK-SINGLE-CORE-PPS-DS4 - verifies DPDK performance using testpmd
> on 1 core, rx and io mode on size Standard_DS4_v2
> * PERF-DPDK-SINGLE-CORE-PPS-DS15 - verifies DPDK performance using
> testpmd on 1 core, rx and io mode on size Standard_DS15_v2
> * PERF-DPDK-MULTICORE-PPS-DS15 - verifies DPDK performance using testpmd
> on 2, 4, 8 cores, rx and io mode on size Standard_DS15_v2
> * PERF-DPDK-MULTICORE-PPS-F32 - verifies DPDK performance using testpmd
> on 2, 4, 8, 16 cores, rx and io mode on size Standard_F32s_v2
> 
> * DPDK-RING-LATENCY - verifies DPDK CPU latency using
> https://github.com/shemminger/dpdk-ring-ping
> 
> * VERIFY-DPDK-PRIMARY-SECONDARY-PROCESSES - verifies primary / secondary
> processes support for DPDK. Runs only on RHEL and Ubuntu distros with Linux
> kernel >= 4.20
> 
> * VERIFY-DPDK-OVS - builds OVS with DPDK support and tests if the OVS DPDK
> ports can be created. Runs only on Ubuntu distro.
> * VERIFY-DPDK-VPP - builds VPP with DPDK support and tests if the VPP ports are
> present. Runs only on RHEL and Ubuntu distros.
> * VERIFY-DPDK-NFF-GO - builds NFF-GO with DPDK support and runs the
> functional tests from the NFF-GO repository. Runs only on Ubuntu distro.
> 
>  DPDK job exited with status: UNSTABLE.
> 
> Test results for DPDK 'v19.11-rc4' and Azure image: 'Canonical UbuntuServer
> 16.04-LTS latest':
> 
> * PERF-DPDK-SINGLE-CORE-PPS-DS4: FAILED
> * VERIFY-DPDK-OVS: PASSED
> * VERIFY-DPDK-BUILD-AND-TESTPMD-TEST: PASSED
> * VERIFY-SRIOV-FAILSAFE-FOR-DPDK: PASSED
> * PERF-DPDK-MULTICORE-PPS-F32: PASSED
> * VERIFY-DPDK-FAILSAFE-DURING-TRAFFIC: PASSED
> 
> Test results for DPDK 'v19.11-rc4' and Azure image: 'Canonical UbuntuServer
> 18.04-DAILY-LTS latest':
> 
> * PERF-DPDK-SINGLE-CORE-PPS-DS4: PASSED
> * VERIFY-DPDK-OVS: PASSED
> * VERIFY-DPDK-BUILD-AND-TESTPMD-TEST: PASSED
> * VERIFY-SRIOV-FAILSAFE-FOR-DPDK: PASSED
> * PERF-DPDK-MULTICORE-PPS-F32: FAILED
> * VERIFY-DPDK-FAILSAFE-DURING-TRAFFIC: PASSED
> 
> Test results for DPDK 'v19.11-rc4' and Azure image: 'RedHat RHEL 7-RAW latest':
> 
> * PERF-DPDK-SINGLE-CORE-PPS-DS4: PASSED
> * VERIFY-DPDK-OVS: SKIPPED
> * VERIFY-DPDK-BUILD-AND-TESTPMD-TEST: PASSED
> * VERIFY-SRIOV-FAILSAFE-FOR-DPDK: PASSED
> * PERF-DPDK-MULTICORE-PPS-F32: FAILED
> * VERIFY-DPDK-FAILSAFE-DURING-TRAFFIC: PASSED
> 
> Test results for DPDK 'v19.11-rc4' and Azure image: 'RedHat RHEL 7.5 latest':
> 
> * PERF-DPDK-SINGLE-CORE-PPS-DS4: PASSED
> * VERIFY-DPDK-OVS: SKIPPED
> * VERIFY-DPDK-BUILD-AND-TESTPMD-TEST: PASSED
> * VERIFY-SRIOV-FAILSAFE-FOR-DPDK: PASSED
> * PERF-DPDK-MULTICORE-PPS-F32: PASSED
> * VERIFY-DPDK-FAILSAFE-DURING-TRAFFIC: PASSED
> 
> Test results for DPDK 'v19.11-rc4' and Azure image: 'Openlogic CentOS 7.5
> latest':
> 
> * PERF-DPDK-SINGLE-CORE-PPS-DS4: PASSED
> * VERIFY-DPDK-OVS: SKIPPED
> * VERIFY-DPDK-BUILD-AND-TESTPMD-TEST: PASSED
> * VERIFY-SRIOV-FAILSAFE-FOR-DPDK: PASSED
> * PERF-DPDK-MULTICORE-PPS-F32: FAILED
> * VERIFY-DPDK-FAILSAFE-DURING-TRAFFIC: PASSED
> 
> Test results for DPDK 'v19.11-rc4' and Azure image: 'SUSE SLES 15 latest':
> 
> * PERF-DPDK-SINGLE-CORE-PPS-DS4: FAILED
> * VERIFY-DPDK-OVS: SKIPPED
> * VERIFY-DPDK-BUILD-AND-TESTPMD-TEST: PASSED
> * VERIFY-SRIOV-FAILSAFE-FOR-DPDK: PASSED
> * PERF-DPDK-MULTICORE-PPS-F32: PASSED
> * VERIFY-DPDK-FAILSAFE-DURING-TRAFFIC: PASSED
> 
> On Wed, 2019-11-27 at 00:56 +0100, Thomas Monjalon wrote:
> > A new DPDK release candidate is ready for testing:
> >
> > https://git.dpdk.org/dpdk/tag/?id=v19.11-rc4
> >
> > 93 patches were integrated.
> >
> > The release notes so far:
> >
> > http://doc.dpdk.org/guides/rel_notes/release_19_11.html
> >
> > It should be completed with a list of tested hardware.
> >
> > This is the last release candidate for DPDK 19.11.
> > After -rc4 only critical bug fixes and doc updates should be accepted.
> >
> > The major release should be done in a couple of days (Thanksgiving?).
> > Please share some release validation results by replying to this
> > message (at dev@dpdk.org ).
> >
> > If you are preparing the next release cycle, please send your v1
> > patches before the 20.02 proposal deadline, which will happen on
> > December 6th.
> > It is also time to build an estimated roadmap for the next cycles.
> >
> > Thank you everyone
> >
> >
> >
> --
> Kind regards,
> Luca Boccassi