From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 30CAAA046B for ; Wed, 24 Jul 2019 05:17:05 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id C35981C0C2; Wed, 24 Jul 2019 05:17:04 +0200 (CEST) Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by dpdk.org (Postfix) with ESMTP id 8DEE41C0B3 for ; Wed, 24 Jul 2019 05:17:02 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga107.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 23 Jul 2019 20:17:01 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.64,300,1559545200"; d="scan'208,217";a="177534815" Received: from fmsmsx105.amr.corp.intel.com ([10.18.124.203]) by FMSMGA003.fm.intel.com with ESMTP; 23 Jul 2019 20:17:01 -0700 Received: from fmsmsx601.amr.corp.intel.com (10.18.126.81) by FMSMSX105.amr.corp.intel.com (10.18.124.203) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 23 Jul 2019 20:17:00 -0700 Received: from fmsmsx601.amr.corp.intel.com (10.18.126.81) by fmsmsx601.amr.corp.intel.com (10.18.126.81) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Tue, 23 Jul 2019 20:17:00 -0700 Received: from shsmsx107.ccr.corp.intel.com (10.239.4.96) by fmsmsx601.amr.corp.intel.com (10.18.126.81) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.1713.5 via Frontend Transport; Tue, 23 Jul 2019 20:16:59 -0700 Received: from shsmsx103.ccr.corp.intel.com ([169.254.4.109]) by SHSMSX107.ccr.corp.intel.com ([169.254.9.162]) with mapi id 14.03.0439.000; Wed, 24 Jul 2019 11:16:58 +0800 From: "Liu, Yong" To: "Chen, Zhaoyan" , "dts@dpdk.org" CC: "Chen, Zhaoyan" Thread-Topic: DTS Enhancement Plan Thread-Index: AdVBx0Km+VjG6je5QSm08Jia/aHreAABSGyA Date: Wed, 24 Jul 2019 03:16:57 +0000 Message-ID: <86228AFD5BCD8E4EBFD2B90117B5E81E6334186C@SHSMSX103.ccr.corp.intel.com> References: <9DEEADBC57E43F4DA73B571777FECECA41D52001@SHSMSX104.ccr.corp.intel.com> In-Reply-To: <9DEEADBC57E43F4DA73B571777FECECA41D52001@SHSMSX104.ccr.corp.intel.com> Accept-Language: zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiYjExYTg3MjMtNWYyMC00YmVjLWFjZjUtOGVjODM4NGFmOGU1IiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjEwLjE4MDQuNDkiLCJUcnVzdGVkTGFiZWxIYXNoIjoiVDJybngwOGt1SHFmUWtQOXUyeHlmeHdHZFpXcDZpK0xiVlBKcXUzZ1gxQkF3WFdxWlc1Z1VvdUFVak5USlM2VCJ9 x-ctpclassification: CTP_NT dlp-product: dlpe-windows dlp-version: 11.2.0.6 dlp-reaction: no-action x-originating-ip: [10.239.127.40] Content-Type: multipart/alternative; boundary="_000_86228AFD5BCD8E4EBFD2B90117B5E81E6334186CSHSMSX103ccrcor_" MIME-Version: 1.0 Subject: Re: [dts] DTS Enhancement Plan X-BeenThere: dts@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: test suite reviews and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dts-bounces@dpdk.org Sender: "dts" --_000_86228AFD5BCD8E4EBFD2B90117B5E81E6334186CSHSMSX103ccrcor_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi Zhaoyan, This is great overall idea. It will be nice to see the roadmap of DTS enhan= cement work, since all works can't be done in one day:) One comment for "All-in-one configuration file". It is nice at first glance= , but configuration file may will be too huge for reading and modification = after add things continuously. And json format has been supported in DTS, is there any special reason for = replacing it? In my mind "Global resource management" will manage all the resources for t= est cases. We also need to add test binary into consideration, as it is als= o shared between suites. Regards, Marvin From: dts [mailto:dts-bounces@dpdk.org] On Behalf Of Chen, Zhaoyan Sent: Wednesday, July 24, 2019 10:52 AM To: dts@dpdk.org Cc: Chen, Zhaoyan Subject: [dts] DTS Enhancement Plan Hi, DTS developers/users, With DPDK development rapidly, more and more test suites were developed in = DTS. (150+ test suites and 3k+ test cases in DTS). Meanwhile, paralleling e= xecution tests for NICs/crypto/other device on same DUT become important fo= r reducing execution time for covering all test cases. So we plan to enhanc= e the DTS, for * Improve test suite quality * Unified test capability/function into framework * Paralleling execution So we draft a DTS development plan, and share with you. Looking forward to hear your thoughts, ideas and suggestions. /Zhaoyan DTS framework enhancement =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Unified packet sending for functionality test * Implement all packet sending features in Packet Class * Remove others sending functions in DTS framework * Sending packet from Tester only Generic EAL options generator for DPDK based application * Generate EAL options for DPDK app, and consider the assigne= d hardware resource for test, instead of whole DUT * Test suite can overwrite the options, in restricted range * Define hardware resource in crb.cfg, port.cfg and pktgen.cf= g Global resource management * Global driver load or unload, vfio-pci, igb_uio, uio_pci_ge= neric * Define a parameter to decside if insmod/rmmod, if the modul= e updated. otherwise, keep this module on host. * External_memory setup * Define DUT's cores mask in crb configuration * Define DUT's ports mask * Define physical ports in crb config Clean up unused APIs, unused imported libraries, and unused APIs * plotting.py, plotgraph.py, numpy, matplotlib * e.g. self.tester.run_rfc2544, has been replaced by pktgen c= lass * Replace private texttable.py with public version All-in-one configuration file * Support configure all information in one configuration file= for easy using * yaml file is better Prepare to move to Python 3, make code be compatible with py2 and py3 Generate auto-script for all DTS dependencies * Update requirements.txt * Generate script for binary dependencies for compilation pyt= hon packages Release DTS container version Test Suites cleanness =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Move to new packet sending API * Update all tests, which generate packets and send packets Move to generic EAL generator API * Update all tests which use DPDK based application with new = EAL string Clean up "sleep" * Replace "sleep" with a stable checking mechanism Replace "pktgen app" with "testpmd app" for testing * pktgen app needs be built for special DPDK version and too = heavy for a simple function test, replace it in test case with testpmd app Clean up absolute path * Replace "/tmp" with "/tmp//", or relative p= ath "tmp". * Replace specific OS path in test case Clean up test case dependencies, (files, libraries or tools) * Check all test suites, and add dependencies into auto-scrip= t or requirements.txt --_000_86228AFD5BCD8E4EBFD2B90117B5E81E6334186CSHSMSX103ccrcor_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Hi Zhaoyan,

This is great overall idea. It will be nice to see t= he roadmap of DTS enhancement work, since all works can’t be done in = one dayJ

 

One comment for “All-in-one configuration file= ”. It is nice at first glance, but configuration file may will be too= huge for reading and modification after add things continuously.

And json format has been supported in DTS, is there = any special reason for replacing it?

 

In my mind “Global resource management” = will manage all the resources for test cases. We also need to add test bina= ry into consideration, as it is also shared between suites.

 

Regards,

Marvin

 

From: dts= [mailto:dts-bounces@dpdk.org] On Behalf Of Chen, Zhaoyan
Sent: Wednesday, July 24, 2019 10:52 AM
To: dts@dpdk.org
Cc: Chen, Zhaoyan <zhaoyan.chen@intel.com>
Subject: [dts] DTS Enhancement Plan

 

Hi, DTS developers/users,

 

With DPDK development rapidly, more and more test su= ites were developed in DTS. (150+ test suites and 3k+ test cases in= DTS). Meanwhile, paralleling execution tests for NICs/crypto/other device = on same DUT become important for reducing execution time for covering all test cases. So we plan to enhance the DTS,= for

·        Improve test suite quality

·        Unified test capability/function into framew= ork

·        Paralleling execution

 

So we draft a DTS development plan, and share with y= ou.

 

Looking forward to hear your thoughts, ideas and sug= gestions.

 

/Zhaoyan

 

DTS framework enhancement

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D

 

Unified packet sending for functionality test

        &nbs= p;     * Implement all packet sending features in Packe= t Class

        &nbs= p;     * Remove others sending functions in DTS framewo= rk

        &nbs= p;     * Sending packet from Tester only

 

Generic EAL options generator for DPDK based applica= tion

        &nbs= p;     * Generate EAL options for DPDK app, and conside= r the assigned hardware resource for test, instead of whole DUT<= /p>

        &nbs= p;     * Test suite can overwrite the options, in restr= icted range

        &nbs= p;     * Define hardware resource in crb.cfg, port.cfg = and pktgen.cfg

 

Global resource management

        &nbs= p;     * Global driver load or unload, vfio-pci, igb_ui= o, uio_pci_generic

        &nbs= p;     * Define a parameter to decside if insmod/rmmod,= if the module updated. otherwise, keep this module on host.

        &nbs= p;     * External_memory setup

        &nbs= p;     * Define DUT's cores mask in crb configuration

        &nbs= p;     * Define DUT's ports mask

        &nbs= p;     * Define physical ports in crb config=

 

Clean up unused APIs, unused imported libraries, and= unused APIs

        &nbs= p;     * plotting.py, plotgraph.py, numpy, matplotlib

        &nbs= p;     * e.g. self.tester.run_rfc2544, has been replace= d by pktgen class

        &nbs= p;     * Replace private texttable.py with public versi= on

 

All-in-one configuration file

        &nbs= p;     * Support configure all information in one confi= guration file for easy using

        &nbs= p;     * yaml file is better

 

Prepare to move to Python 3, make code be compatible= with py2 and py3

 

Generate auto-script for all DTS dependencies

        &nbs= p;     * Update requirements.txt

        &nbs= p;     * Generate script for binary dependencies for co= mpilation python packages

 

Release DTS container version

 

 

Test Suites cleanness

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D<= o:p>

 

Move to new packet sending API

        &nbs= p;     * Update all tests, which generate packets and s= end packets

 

Move to generic EAL generator API

        &nbs= p;     * Update all tests which use DPDK based applicat= ion with new EAL string

 

Clean up "sleep"

        &nbs= p;     * Replace "sleep" with a stable checki= ng mechanism

 

Replace "pktgen app" with "testpmd ap= p" for testing

        &nbs= p;     * pktgen app needs be built for special DPDK ver= sion and too heavy for a simple function test, replace it in test case with= testpmd app

 

Clean up absolute path

        &nbs= p;     * Replace "/tmp" with "/tmp/<d= ts_pid_random>/", or relative path "tmp".

        &nbs= p;     * Replace specific OS path in test case

 

Clean up test case dependencies, (files, libraries o= r tools)

        &nbs= p;     * Check all test suites, and add dependencies in= to auto-script or requirements.txt

 

--_000_86228AFD5BCD8E4EBFD2B90117B5E81E6334186CSHSMSX103ccrcor_--