From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id D9018A2E1B for ; Wed, 4 Sep 2019 07:34:48 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 7F9121EBB7; Wed, 4 Sep 2019 07:34:48 +0200 (CEST) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id DE1811EBAC for ; Wed, 4 Sep 2019 07:34:46 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 03 Sep 2019 22:34:45 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.64,465,1559545200"; d="scan'208";a="185001789" Received: from fmsmsx105.amr.corp.intel.com ([10.18.124.203]) by orsmga003.jf.intel.com with ESMTP; 03 Sep 2019 22:34:45 -0700 Received: from fmsmsx122.amr.corp.intel.com (10.18.125.37) by FMSMSX105.amr.corp.intel.com (10.18.124.203) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 3 Sep 2019 22:34:45 -0700 Received: from shsmsx152.ccr.corp.intel.com (10.239.6.52) by fmsmsx122.amr.corp.intel.com (10.18.125.37) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 3 Sep 2019 22:34:45 -0700 Received: from shsmsx101.ccr.corp.intel.com ([169.254.1.92]) by SHSMSX152.ccr.corp.intel.com ([169.254.6.132]) with mapi id 14.03.0439.000; Wed, 4 Sep 2019 13:34:44 +0800 From: "Tu, Lijuan" To: "Mo, YufengX" , "Zeng, XiaoxiaoX" , "dts@dpdk.org" CC: "Zeng, XiaoxiaoX" Thread-Topic: [dts] [PATCH V1] tests/TestSuite_multiprocess: add case test_perf_multiprocess_performance Thread-Index: AQHVYTkW5bSZusDUbkymIKYMFUN/A6cbAmsQ Date: Wed, 4 Sep 2019 05:34:43 +0000 Message-ID: <8CE3E05A3F976642AAB0F4675D0AD20E0BB1C0CA@SHSMSX101.ccr.corp.intel.com> References: <20190830204208.7111-1-xiaoxiaox.zeng@intel.com> In-Reply-To: Accept-Language: zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-version: 11.2.0.6 dlp-reaction: no-action x-ctpclassification: CTP_NT x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiZGI5OWMwMzItM2JhMS00YzU0LWEzY2EtOWU1MGZlMjFhOTZmIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjEwLjE4MDQuNDkiLCJUcnVzdGVkTGFiZWxIYXNoIjoiYncyQlZDN1FKRERrUXZKK1BkTE4yU1N4VWVzU2dlZzV1RFAwNW5tazU1OERWaW5lQnNLWlVvTDBlV292T3BBQSJ9 x-originating-ip: [10.239.127.40] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dts] [PATCH V1] tests/TestSuite_multiprocess: add case test_perf_multiprocess_performance X-BeenThere: dts@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: test suite reviews and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dts-bounces@dpdk.org Sender: "dts" Thanks yufeng's review > -----Original Message----- > From: dts [mailto:dts-bounces@dpdk.org] On Behalf Of Mo, YufengX > Sent: Monday, September 2, 2019 10:49 AM > To: Zeng, XiaoxiaoX ; dts@dpdk.org > Cc: Zeng, XiaoxiaoX > Subject: Re: [dts] [PATCH V1] tests/TestSuite_multiprocess: add case > test_perf_multiprocess_performance >=20 > Hi, zeng,xiaoxiao >=20 > some questions about your test case `test_perf_multiprocess_performance` > *. self.dut.kill_all() is used to kill dpdk revelant process, your test c= ase hasn't > run any dpdk process before it. > What's your target? > *. You use self.dut.new_session() to create new session in a for loop, bu= t you > haven't close these sessions. And this suite > has created one `session_secondary` session in set_up_all method, why > you want to create new one? > *. When you run symmetric_mp, you haven't run server process, is it the t= est > plan required? > *. After you boot up first symmetric_mp, you haven't close it and run oth= er > one, is it the test plan required? > *. Could you update this test case's test plan at the same time? >=20 >=20 > > -----Original Message----- > > From: dts [mailto:dts-bounces@dpdk.org] On Behalf Of Xiaoxiao Zeng > > Sent: Saturday, August 31, 2019 4:42 AM > > To: dts@dpdk.org > > Cc: Zeng, XiaoxiaoX > > Subject: [dts] [PATCH V1] tests/TestSuite_multiprocess: add case > > test_perf_multiprocess_performance > > > > add testcase test_perf_multiprocess_performance > > *.according to testplan, add test_perf_multiprocess_performance code > > > > Signed-off-by: Xiaoxiao Zeng > > --- > > tests/TestSuite_multiprocess.py | 127 > > ++++++++++++++++++++++++++------ > > 1 file changed, 103 insertions(+), 24 deletions(-) > > > > diff --git a/tests/TestSuite_multiprocess.py > > b/tests/TestSuite_multiprocess.py index b690d81..4a4743e 100644 > > --- a/tests/TestSuite_multiprocess.py > > +++ b/tests/TestSuite_multiprocess.py > > @@ -37,6 +37,7 @@ Multi-process Test. > > import utils > > import time > > import os > > + > > executions =3D [] > > from test_case import TestCase > > from pktgen import PacketGeneratorHelper @@ -68,16 +69,17 @@ class > > TestMultiprocess(TestCase): > > executions.append({'nprocs': 4, 'cores': '1S/2C/2T', 'pps': 0}= ) > > executions.append({'nprocs': 4, 'cores': '1S/4C/1T', 'pps': 0}= ) > > executions.append({'nprocs': 8, 'cores': '1S/4C/2T', 'pps': 0}= ) > > - self.dut.alt_session.send_expect("cd dpdk","# ",5) > > + self.dut.alt_session.send_expect("cd dpdk", "# ", 5) > > > > # start new session to run secondary > > self.session_secondary =3D self.dut.new_session() > > + > > # get dts output path > > if self.logger.log_path.startswith(os.sep): > > self.output_path =3D self.logger.log_path > > else: > > cur_path =3D os.path.dirname( > > - os.path.dirname(os.path.realpath(__fil= e__))) > > + os.path.dirname(os.path.realpath(__file__))) > > self.output_path =3D os.sep.join([cur_path, self.logger.lo= g_path]) > > # create an instance to set stream field setting > > self.pktgen_helper =3D PacketGeneratorHelper() @@ -95,23 +97,2= 8 > > @@ class TestMultiprocess(TestCase): > > # Send message from secondary to primary > > cores =3D self.dut.get_core_list('1S/2C/1T') > > coremask =3D utils.create_mask(cores) > > - > self.dut.send_expect("./examples/multi_process/simple_mp/%s/simple_mp - > n 1 -c %s --proc-type=3Dprimary" % (self.target, > > coremask), "Finished Process Init", 100) > > + > > + > self.dut.send_expect("./examples/multi_process/simple_mp/%s/simple_m > > + p -n 1 -c %s --proc-type=3Dprimary" % (self.target, > > coremask), > > + "Finished Process Init", 100) > > time.sleep(20) > > coremask =3D hex(int(coremask, 16) * 0x10).rstrip("L") > > - > self.session_secondary.send_expect("./examples/multi_process/simple_mp/ > %s/simple_mp -n 1 -c %s --proc-type=3Dsecondary" % > > (self.target, coremask), "Finished Process Init", 100) > > + self.session_secondary.send_expect( > > + "./examples/multi_process/simple_mp/%s/simple_mp -n 1 -c > > + %s --proc-type=3Dsecondary" % (self.target, coremask), "Finished > > Process Init", > > + 100) > > > > self.session_secondary.send_expect("send hello_primary", ">") > > out =3D self.dut.get_session_output() > > self.session_secondary.send_expect("quit", "# ") > > - self.dut.send_expect("quit","# ") > > + self.dut.send_expect("quit", "# ") > > self.verify("Received 'hello_primary'" in out, "Message not re= ceived > on primary process") > > # Send message from primary to secondary > > cores =3D self.dut.get_core_list('1S/2C/1T') > > coremask =3D utils.create_mask(cores) > > - > self.session_secondary.send_expect("./examples/multi_process/simple_mp/ > %s/simple_mp -n 1 -c %s --proc-type=3Dprimary " % > > (self.target, coremask), "Finished Process Init", 100) > > + self.session_secondary.send_expect( > > + "./examples/multi_process/simple_mp/%s/simple_mp -n 1 -c > > + %s --proc-type=3Dprimary " % (self.target, coremask), "Finished > > Process Init", 100) > > time.sleep(20) > > coremask =3D hex(int(coremask, 16) * 0x10).rstrip("L") > > - > self.dut.send_expect("./examples/multi_process/simple_mp/%s/simple_mp - > n 1 -c %s --proc-type=3Dsecondary" % (self.target, > > coremask), "Finished Process Init", 100) > > + > > + > self.dut.send_expect("./examples/multi_process/simple_mp/%s/simple_m > > + p -n 1 -c %s --proc-type=3Dsecondary" % (self.target, > > coremask), > > + "Finished Process Init", 100) > > self.session_secondary.send_expect("send hello_secondary", ">"= ) > > out =3D self.dut.get_session_output() > > self.session_secondary.send_expect("quit", "# ") @@ -127,10 > > +134,12 @@ class TestMultiprocess(TestCase): > > > > cores =3D self.dut.get_core_list('1S/2C/1T') > > coremask =3D utils.create_mask(cores) > > - > self.session_secondary.send_expect("./examples/multi_process/simple_mp/ > %s/simple_mp -n 1 -c %s --proc-type=3Dprimary" % > > (self.target, coremask), "Finished Process Init", 100) > > + > > + self.session_secondary.send_expect("./examples/multi_process/simple_ > > + mp/%s/simple_mp -n 1 -c %s --proc-type=3Dprimary" % > > (self.target, coremask), > > + "Finished Process Init", > > + 100) > > time.sleep(20) > > coremask =3D hex(int(coremask, 16) * 0x10).rstrip("L") > > - > self.dut.send_expect("./examples/multi_process/simple_mp/%s/simple_mp - > n 1 -c %s --proc-type=3Dsecondary" % (self.target, > > coremask), "Finished Process Init", 100) > > + > > + > self.dut.send_expect("./examples/multi_process/simple_mp/%s/simple_m > > + p -n 1 -c %s --proc-type=3Dsecondary" % (self.target, > > coremask), > > + "Finished Process Init", 100) > > stringsSent =3D 0 > > for line in open('/usr/share/dict/words', 'r').readlines(): > > line =3D line.split('\n')[0] @@ -151,11 +160,13 @@ class > > TestMultiprocess(TestCase): > > # Send message from secondary to primary (auto process type) > > cores =3D self.dut.get_core_list('1S/2C/1T') > > coremask =3D utils.create_mask(cores) > > - out =3D > self.dut.send_expect("./examples/multi_process/simple_mp/%s/simple_mp - > n 1 -c %s --proc-type=3Dauto " % (self.target, > > coremask), "Finished Process Init", 100) > > + out =3D > > + > self.dut.send_expect("./examples/multi_process/simple_mp/%s/simple_m > > + p -n 1 -c %s --proc-type=3Dauto " % (self.target, > > coremask), > > + "Finished Process Init", 100) > > self.verify("EAL: Auto-detected process type: PRIMARY" in out,= "The > type of process (PRIMARY) was not detected properly") > > time.sleep(20) > > coremask =3D hex(int(coremask, 16) * 0x10).rstrip("L") > > - out =3D > self.session_secondary.send_expect("./examples/multi_process/simple_mp/ > %s/simple_mp -n 1 -c %s --proc-type=3Dauto" % > > (self.target, coremask), "Finished Process Init", 100) > > + out =3D self.session_secondary.send_expect( > > + "./examples/multi_process/simple_mp/%s/simple_mp -n 1 -c > > + %s --proc-type=3Dauto" % (self.target, coremask), "Finished Process > > Init", 100) > > self.verify("EAL: Auto-detected process type: SECONDARY" in ou= t, > > "The type of process (SECONDARY) was not detected > > properly") > > > > @@ -168,13 +179,15 @@ class TestMultiprocess(TestCase): > > # Send message from primary to secondary (auto process type) > > cores =3D self.dut.get_core_list('1S/2C/1T') > > coremask =3D utils.create_mask(cores) > > - out =3D > self.session_secondary.send_expect("./examples/multi_process/simple_mp/ > %s/simple_mp -n 1 -c %s --proc-type=3Dauto" % > > (self.target, coremask), "Finished Process Init", 100) > > + out =3D self.session_secondary.send_expect( > > + "./examples/multi_process/simple_mp/%s/simple_mp -n 1 -c > > + %s --proc-type=3Dauto" % (self.target, coremask), "Finished Process > > Init", 100) > > self.verify("EAL: Auto-detected process type: PRIMARY" in out,= "The > type of process (PRIMARY) was not detected properly") > > time.sleep(20) > > coremask =3D hex(int(coremask, 16) * 0x10).rstrip("L") > > - out =3D > self.dut.send_expect("./examples/multi_process/simple_mp/%s/simple_mp - > n 1 -c %s --proc-type=3Dauto" % (self.target, > > coremask), "Finished Process Init", 100) > > + out =3D > > + > self.dut.send_expect("./examples/multi_process/simple_mp/%s/simple_m > > + p -n 1 -c %s --proc-type=3Dauto" % (self.target, > > coremask), > > + "Finished Process Init", 100) > > self.verify("EAL: Auto-detected process type: SECONDARY" in ou= t, > "The type of process (SECONDARY) was not detected properly") > > - self.session_secondary.send_expect("send hello_secondary", ">"= ,100) > > + self.session_secondary.send_expect("send hello_secondary", > > + ">", 100) > > out =3D self.dut.get_session_output() > > self.session_secondary.send_expect("quit", "# ") > > self.dut.send_expect("quit", "# ") @@ -189,7 +202,8 @@ class > > TestMultiprocess(TestCase): > > > > cores =3D self.dut.get_core_list('1S/2C/1T') > > coremask =3D utils.create_mask(cores) > > - > self.session_secondary.send_expect("./examples/multi_process/simple_mp/ > %s/simple_mp -n 1 -c %s -m 64" % (self.target, > > coremask), "Finished Process Init", 100) > > + > > + self.session_secondary.send_expect("./examples/multi_process/simple_ > > + mp/%s/simple_mp -n 1 -c %s -m 64" % (self.target, > > coremask), > > + "Finished Process Init", > > + 100) > > coremask =3D hex(int(coremask, 16) * 0x10).rstrip("L") > > out =3D > > > self.dut.send_expect("./examples/multi_process/simple_mp/%s/simple_mp > > -n 1 -c %s" % (self.target, coremask), "# ", 100) > > > > @@ -198,6 +212,66 @@ class TestMultiprocess(TestCase): > > > > self.session_secondary.send_expect("quit", "# ") > > > > + def test_perf_multiprocess_performance(self): > > + """ > > + Benchmark Multiprocess performance. > > + """ > > + packet_count =3D 16 > > + self.dut.kill_all() > > + self.dut.send_expect("fg", "# ") > > + dutPorts =3D self.dut.get_ports() > > + txPort =3D self.tester.get_local_port(dutPorts[0]) > > + rxPort =3D self.tester.get_local_port(dutPorts[1]) > > + mac =3D self.tester.get_mac(txPort) > > + dmac =3D self.dut.get_mac_address(dutPorts[0]) > > + tgenInput =3D [] > > + > > + for i in range(packet_count): > > + package =3D r'flows =3D [Ether(src=3D"%s", > dst=3D"%s")/IP(src=3D"192.168.1.%d", dst=3D"192.168.1.%d")/("X"*26)]' % (= mac, > dmac, i + 1, i + 2) > > + self.tester.scapy_append(package) > > + pcap =3D os.sep.join([self.output_path, "test_%d.pcap"%i]) > > + self.tester.scapy_append('wrpcap("%s", flows)' % pcap) > > + tgenInput.append([txPort, rxPort, pcap]) > > + self.tester.scapy_execute() > > + > > + validExecutions =3D [] > > + for execution in executions: > > + if len(self.dut.get_core_list(execution['cores'])) =3D=3D > execution['nprocs']: > > + validExecutions.append(execution) > > + > > + portMask =3D utils.create_mask([dutPorts[0], dutPorts[1]]) > > + > > + for n in range(len(validExecutions)): > > + execution =3D validExecutions[n] > > + coreList =3D self.dut.get_core_list(execution['cores']) > > + coreMask =3D > utils.create_mask(self.dut.get_core_list(execution['cores'])) > > + time.sleep(5) > > + self.dut_session =3D self.dut.new_session() > > + self.dut_session.send_expect( > > + > > + "./examples/multi_process/symmetric_mp/%s/symmetric_mp -c %s > > + --proc-type=3Dauto -- -p %s --num-procs=3D%d --proc-id=3D%d" % > > ( > > + self.target, coreMask, portMask, execution['nprocs= '], n), > "Finished Process Init") > > + # clear streams before add new streams > > + self.tester.pktgen.clear_streams() > > + # run packet generator > > + streams =3D > self.pktgen_helper.prepare_stream_from_tginput(tgenInput, 100, > > + N= one, self.tester.pktgen) > > + _, pps =3D > > + self.tester.pktgen.measure_throughput(stream_ids=3Dstreams) > > + > > + execution['pps'] =3D pps > > + #self.dut.kill_all() > > + time.sleep(5) > > + > > + for n in range(len(executions)): > > + self.verify(executions[n]['pps'] is not 0, "No traffic > > + detected") > > + > > + self.result_table_create(['Num-procs', > > + 'Sockets/Cores/Threads', 'Num Ports', 'Frame Size', '%-age Line > > + Rate', 'Packet Rate(mpps)']) > > + > > + for execution in validExecutions: > > + self.result_table_add( > > + [execution['nprocs'], execution['cores'], 2, 64, > > + execution['pps'] / float(100000000 / (8 * 84)), execution['pps'] / > > + float(1000000)]) > > + > > + self.result_table_print() > > + > > def test_perf_multiprocess_client_serverperformance(self): > > """ > > Benchmark Multiprocess client-server performance. > > @@ -227,14 +301,16 @@ class TestMultiprocess(TestCase): > > > > coreMask =3D utils.create_mask(self.dut.get_core_list('1S/= 1C/1T')) > > portMask =3D utils.create_mask([dutPorts[0], dutPorts[1]]) > > - > self.dut.send_expect("./examples/multi_process/client_server_mp/mp_serve > r/%s/mp_server -n %d -c %s -- -p %s -n %d" % > > (self.target, self.dut.get_memory_channels(), "0xA0", portMask, > > execution['nprocs']), "Finished Process Init", 20) > > + > self.dut.send_expect("./examples/multi_process/client_server_mp/mp_serve > r/%s/mp_server -n %d -c %s -- -p %s -n %d" % ( > > + self.target, self.dut.get_memory_channels(), "0xA0", > > + portMask, execution['nprocs']), "Finished Process Init", 20) > > self.dut.send_expect("^Z", "\r\n") > > self.dut.send_expect("bg", "# ") > > > > for n in range(execution['nprocs']): > > time.sleep(5) > > coreMask =3D utils.create_mask([coreList[n]]) > > - > self.dut.send_expect("./examples/multi_process/client_server_mp/mp_clien > t/%s/mp_client -n %d -c %s --proc-type=3Dsecondary > > -- -n %d" % (self.target, self.dut.get_memory_channels(), coreMask, > > n), "Finished Process Init") > > + > > + self.dut.send_expect("./examples/multi_process/client_server_mp/mp_c > > + lient/%s/mp_client -n %d -c %s --proc-type=3Dsecondary > > -- -n %d" % ( > > + self.target, self.dut.get_memory_channels(), > > + coreMask, n), "Finished Process Init") > > self.dut.send_expect("^Z", "\r\n") > > self.dut.send_expect("bg", "# ") > > > > @@ -245,7 +321,7 @@ class TestMultiprocess(TestCase): > > self.tester.pktgen.clear_streams() > > # run packet generator > > streams =3D > self.pktgen_helper.prepare_stream_from_tginput(tgenInput, 100, > > - None, self.tester.pktg= en) > > + > > + None, self.tester.pktgen) > > _, pps =3D > > self.tester.pktgen.measure_throughput(stream_ids=3Dstreams) > > > > execution['pps'] =3D pps > > @@ -255,20 +331,23 @@ class TestMultiprocess(TestCase): > > for n in range(len(executions)): > > self.verify(executions[n]['pps'] is not 0, "No traffic > > detected") > > > > - self.result_table_create(['Server threads', 'Server Cores/Thre= ads', > 'Num-procs', 'Sockets/Cores/Threads', 'Num Ports', 'Frame Size', > > '%-age Line Rate', 'Packet Rate(mpps)']) > > + self.result_table_create( > > + ['Server threads', 'Server Cores/Threads', 'Num-procs', > 'Sockets/Cores/Threads', 'Num Ports', 'Frame Size', '%-age Line Rate', > > + 'Packet Rate(mpps)']) > > > > for execution in validExecutions: > > - self.result_table_add([1, '1S/1C/1T', execution['nprocs'], > execution['cores'], 2, 64, execution['pps'] / float(100000000 / (8 * 84))= , > > execution['pps'] / float(1000000)]) > > + self.result_table_add([1, '1S/1C/1T', execution['nprocs'], > execution['cores'], 2, 64, execution['pps'] / float(100000000 / (8 * 84))= , > > + execution['pps'] / > > + float(1000000)]) > > > > self.result_table_print() > > > > def set_fields(self): > > ''' set ip protocol field behavior ''' > > fields_config =3D { > > - 'ip': { > > - 'src': {'range': 64, 'action': 'inc'}, > > - 'dst': {'range': 64, 'action': 'inc'}, > > - },} > > + 'ip': { > > + 'src': {'range': 64, 'action': 'inc'}, > > + 'dst': {'range': 64, 'action': 'inc'}, > > + }, } > > > > return fields_config > > > > -- > > 2.17.0