From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by dpdk.org (Postfix) with ESMTP id 7D7F1201 for ; Wed, 28 Dec 2016 06:56:18 +0100 (CET) Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orsmga105.jf.intel.com with ESMTP; 27 Dec 2016 21:56:17 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.33,420,1477983600"; d="scan'208";a="47083432" Received: from fmsmsx108.amr.corp.intel.com ([10.18.124.206]) by orsmga005.jf.intel.com with ESMTP; 27 Dec 2016 21:56:17 -0800 Received: from shsmsx104.ccr.corp.intel.com (10.239.4.70) by FMSMSX108.amr.corp.intel.com (10.18.124.206) with Microsoft SMTP Server (TLS) id 14.3.248.2; Tue, 27 Dec 2016 21:56:16 -0800 Received: from shsmsx102.ccr.corp.intel.com ([169.254.2.88]) by SHSMSX104.ccr.corp.intel.com ([10.239.4.70]) with mapi id 14.03.0248.002; Wed, 28 Dec 2016 13:56:15 +0800 From: "Yao, Lei A" To: "Liu, Yong" , "dts@dpdk.org" Thread-Topic: [dts] [PATCH V1] tests: add test for vhost/virtio loopback performance Thread-Index: AQHSYMswDc92HQd4OkSzP+rE2/Ih8aEc2t8w Date: Wed, 28 Dec 2016 05:56:14 +0000 Message-ID: <2DBBFF226F7CF64BAFCA79B681719D953A149C77@shsmsx102.ccr.corp.intel.com> References: <1482377343-9957-1-git-send-email-lei.a.yao@intel.com> <86228AFD5BCD8E4EBFD2B90117B5E81E62D341AA@SHSMSX103.ccr.corp.intel.com> In-Reply-To: <86228AFD5BCD8E4EBFD2B90117B5E81E62D341AA@SHSMSX103.ccr.corp.intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.239.127.40] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dts] [PATCH V1] tests: add test for vhost/virtio loopback performance X-BeenThere: dts@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: test suite reviews and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Dec 2016 05:56:19 -0000 Some update. Thanks. > -----Original Message----- > From: Liu, Yong > Sent: Wednesday, December 28, 2016 1:28 PM > To: Yao, Lei A ; dts@dpdk.org > Cc: Yao, Lei A > Subject: RE: [dts] [PATCH V1] tests: add test for vhost/virtio loopback > performance >=20 > Lei, some comments below. >=20 > > -----Original Message----- > > From: dts [mailto:dts-bounces@dpdk.org] On Behalf Of lei,yao > > Sent: Thursday, December 22, 2016 11:29 AM > > To: dts@dpdk.org > > Cc: Yao, Lei A > > Subject: [dts] [PATCH V1] tests: add test for vhost/virtio loopback > > performance > > > > From: lei yao > > > > This test won't use qemu, but will use two testpmd thread to launch vho= st- > > user and > > virtio-user to simulate the qemu user case. > > > > Signed-off-by: lei yao > > --- > > ...Suite_vhost_loopback_performance_virtio_user.py | 256 > > +++++++++++++++++++++ > > 1 file changed, 256 insertions(+) > > create mode 100644 > > tests/TestSuite_vhost_loopback_performance_virtio_user.py > > > > diff --git a/tests/TestSuite_vhost_loopback_performance_virtio_user.py > > b/tests/TestSuite_vhost_loopback_performance_virtio_user.py > > new file mode 100644 > > index 0000000..46d2110 > > --- /dev/null > > +++ b/tests/TestSuite_vhost_loopback_performance_virtio_user.py > > @@ -0,0 +1,256 @@ > > +# > > +# BSD LICENSE > > +# > > +# Copyright(c) 2010-2016 Intel Corporation. All rights reserved. > > +# All rights reserved. > > +# > > +# Redistribution and use in source and binary forms, with or without > > +# modification, are permitted provided that the following conditions > > +# are met: > > +# > > +# * Redistributions of source code must retain the above copyright > > +# notice, this list of conditions and the following disclaimer. > > +# * Redistributions in binary form must reproduce the above copyrigh= t > > +# notice, this list of conditions and the following disclaimer in > > +# the documentation and/or other materials provided with the > > +# distribution. > > +# * Neither the name of Intel Corporation nor the names of its > > +# contributors may be used to endorse or promote products derived > > +# from this software without specific prior written permission. > > +# > > +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND > CONTRIBUTORS > > +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT > NOT > > +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND > FITNESS FOR > > +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE > COPYRIGHT > > +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, > INCIDENTAL, > > +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT > NOT > > +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS > OF USE, > > +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED > AND ON ANY > > +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR > TORT > > +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT > OF THE USE > > +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH > DAMAGE. > > + > > + > > +""" > > +DPDK Test suite. > > +Test Vhost loopback performance for Mergeable, normal , vector Path > > +""" > > +import string > > +import utils > > +import time > > +import re > > +from test_case import TestCase > > + > > + > > +class TestVhostLoopback(TestCase): > > + > > + def set_up_all(self): > > + """ > > + Run at the start of each test suite. > > + """ > > + # Clean the execution environment > > + self.dut.send_expect("rm -rf ./vhost.out", "#") > > + self.dut.send_expect("rm -rf ./vhost-net*", "#") > > + self.dut.send_expect("killall -s INT vhost-switch", "#") > > + self.dut.send_expect("killall -s INT testpmd", "#") >=20 > Dpdk application will be automated killed by framework. There's no need t= o > kill dpdk application in set_up_all function. >=20 So both vhost-switch and testpmd will be cleaned by framework, right? > > + self.header_row =3D ["FrameSize(B)", "Mode", "Throughput(Mpps)= ", > > "Virtio Version"] > > + self.frame_sizes =3D [64, 128, 260, 520, 1024, 1500] > > + self.test_cycles =3D {'Mpps': {}, 'pct': {}} > > + > > + def set_up(self): > > + """ > > + Run before each test case. > > + """ > > + self.dut.restore_interfaces() > > + pass >=20 > This is critical function which will bind all ports to default driver. > You can use simple code like "port.bind_driver()" to bind port to default > driver and "port.bind_driver(driver=3Dself.drivername)" to igb_uio driver= . > Just in this suite, I can't see any requirement to do this. >=20 Because this test cases don't need any NIC binned to pmd driver. So I use t= his function=20 to clean it.=20 > > + > > + def test_perf_vhost_mergeable_loopback(self): > > + """ > > + Benchmark performance for frame_sizes. > > + """ > > + > > + # Modify the driver file to disbale the Mergeable, then re- > > compile the DPDK and back up the original driver file > > + for frame_size in self.frame_sizes: > > + # Back up the original driver file > > + > > self.dut.send_expect("cp ./drivers/net/virtio/virtio_ethdev.h ./", "#",= 30) > > + # Change the packet size sent by the testpmd > > + self.change_testpmd_size =3D "sed -i -e 's/#define > > TXONLY_DEF_PACKET_LEN.*$/#define > TXONLY_DEF_PACKET_LEN %d/' ./app/test- > > pmd/testpmd.h" % frame_size >=20 > If changed any code in testpmd, please restore it back for other suites m= ay > depend on that. Sure. I will restore the testpmd for every test in V2. > > + self.dut.send_expect(self.change_testpmd_size, "#", 30) > > + self.dut.build_install_dpdk(self.dut.target) > > + > > + # Start the vhost user side > > + cmd =3D self.target + "/app/testpmd -n 4 -c 0x03 " + \ > > + "-m 2048 --file-prefix=3Dvhost --vdev > > 'net_vhost0,iface=3Dvhost-net,queues=3D1,client=3D0' -- -i" > > + self.dut.send_expect(cmd, "testpmd>", 120) > > + # Start the virtio_user side > > + vhost_user =3D self.dut.new_session() > > + command_line_user =3D self.target + "/app/testpmd -n 4 -c = 0x0c > > " + \ > > + " -m 2048 --no-pci --file-prefix=3Dvir= tio " > > + \ > > + " -- > > vdev=3Dnet_virtio_user0,mac=3D00:01:02:03:04:05,path=3D./vhost-net" + \ > > + " -- -i --txqflags=3D0xf01 --disable-h= w- > > vlan-filter" > > + > > + vhost_user.send_expect(command_line_user, "testpmd>", 120) > > + > > + self.dut.send_expect("set fwd mac retry", "testpmd>", 60) > > + self.dut.send_expect("start tx_first 32", "testpmd>", 60) > > + vhost_user.send_expect("start tx_first 32", "testpmd> ", 1= 20) > > + results =3D 0.0 > > + out =3D self.dut.send_expect("show port stats all", "testp= md>", > > 60) > > + time.sleep(5) > > + # Get throughput 10 times and calculate the average > > throughput > > + for i in range(10): > > + out =3D self.dut.send_expect("show port stats all", > > "testpmd>", 60) > > + time.sleep(5) > > + lines =3D re.search("Rx-pps:\s*(\d*)", out) > > + result =3D lines.group(1) > > + results +=3D float(result) > > + Mpps =3D results / (1000000 * 10) > > + self.result_table_create(self.header_row) > > + > > + self.dut.send_expect("quit", "#", 60) > > + vhost_user.send_expect("quit", "#", 60) > > + # Restore the driver file > > + self.dut.send_expect("rm - > > rf ./drivers/net/virtio/virtio_ethdev.h", "#", 30) > > + > > self.dut.send_expect("mv ./virtio_ethdev.h ./drivers/net/virtio/", "#",= 30) > > + self.test_cycles['Mpps'][frame_size] =3D Mpps > > + self.test_cycles['pct'][frame_size] =3D "Virtio 0.95" > > + > > + for frame_size in self.frame_sizes: > > + results_row =3D [frame_size] > > + results_row.append("Mergeable on") > > + results_row.append(self.test_cycles['Mpps'][frame_size]) > > + results_row.append(self.test_cycles['pct'][frame_size]) > > + self.result_table_add(results_row) > > + > > + self.result_table_print() > > + > > + def test_perf_vhost_vector_loopback(self): > > + """ > > + Benchmark performance for frame_sizes. > > + """ > > + for frame_size in self.frame_sizes: > > + # Modify the driver file to disbale the Mergeable, then re= - > > compile the DPDK and back up the original driver file > > + > > self.dut.send_expect("cp ./drivers/net/virtio/virtio_ethdev.h ./", "#",= 30) > > + self.dut.send_expect("sed -i > > '/VIRTIO_NET_F_MRG_RXBUF/d' ./drivers/net/virtio/virtio_ethdev.h", "#", > 30) > > + self.change_testpmd_size =3D "sed -i -e 's/#define > > TXONLY_DEF_PACKET_LEN .*$/#define > TXONLY_DEF_PACKET_LEN %d/' ./app/test- > > pmd/testpmd.h" % frame_size >=20 > Same comments as previous. >=20 > > + self.dut.send_expect(self.change_testpmd_size, "#", 30) > > + self.dut.build_install_dpdk(self.dut.target) > > + > > + # Start the vhost user side > > + cmd =3D self.target + "/app/testpmd -n 4 -c 0x03 " + \ > > + "-m 2048 --file-prefix=3Dvhost --vdev > > 'net_vhost0,iface=3Dvhost-net,queues=3D1,client=3D0' -- -i" > > + self.dut.send_expect(cmd, "testpmd>", 120) > > + # Start the virtio_user side > > + vhost_user =3D self.dut.new_session() > > + command_line_user =3D self.target + "/app/testpmd -n 4 -c = 0x0c > > " + \ > > + " -m 2048 --no-pci --file-prefix=3Dvir= tio " > > + \ > > + " -- > > vdev=3Dnet_virtio_user0,mac=3D00:01:02:03:04:05,path=3D./vhost-net " + = \ > > + " -- -i --txqflags=3D0xf01 --disable-h= w- > > vlan-filter" > > + > > + vhost_user.send_expect(command_line_user, "testpmd>", 120) > > + > > + self.dut.send_expect("set fwd mac retry", "testpmd>", 60) > > + self.dut.send_expect("start tx_first 32", "testpmd>", 60) > > + vhost_user.send_expect("start tx_first 32", "testpmd> ", 1= 20) > > + results =3D 0.0 > > + out =3D self.dut.send_expect("show port stats all", "testp= md>", > > 60) > > + time.sleep(5) > > + # Get throughput 10 times and calculate the average > > throughput > > + for i in range(10): > > + out =3D self.dut.send_expect("show port stats all", > > "testpmd>", 60) > > + time.sleep(5) > > + lines =3D re.search("Rx-pps:\s*(\d*)", out) > > + result =3D lines.group(1) > > + results +=3D float(result) > > + Mpps =3D results / (1000000 * 10) > > + self.result_table_create(self.header_row) >=20 > Code look like not aligned here, please check again. >=20 > > + > > + self.dut.send_expect("quit", "#", 60) > > + vhost_user.send_expect("quit", "#", 60) > > + # Restore the driver file > > + self.dut.send_expect("rm - > > rf ./drivers/net/virtio/virtio_ethdev.h", "#", 30) > > + > > self.dut.send_expect("mv ./virtio_ethdev.h ./drivers/net/virtio/", "#",= 30) > > + self.test_cycles['Mpps'][frame_size] =3D Mpps > > + self.test_cycles['pct'][frame_size] =3D "Virtio 0.95" > > + > > + for frame_size in self.frame_sizes: > > + results_row =3D [frame_size] > > + results_row.append("Vector on") > > + results_row.append(self.test_cycles['Mpps'][frame_size]) > > + results_row.append(self.test_cycles['pct'][frame_size]) > > + self.result_table_add(results_row) > > + > > + self.result_table_print() > > + > > + def test_perf_vhost_normal_loopback(self): > > + """ > > + Benchmark performance for frame_sizes. > > + """ > > + > > + self.result_table_create(self.header_row) > > + for frame_size in self.frame_sizes: > > + # Modify the driver file to disbale the Mergeable, then re= - > > compile the DPDK and back up the original driver file > > + > > self.dut.send_expect("cp ./drivers/net/virtio/virtio_ethdev.h ./", "#",= 30) > > + self.dut.send_expect("sed -i > > '/VIRTIO_NET_F_MRG_RXBUF/d' ./drivers/net/virtio/virtio_ethdev.h", "#", > 30) > > + self.change_testpmd_size =3D "sed -i -e 's/#define > > TXONLY_DEF_PACKET_LEN .*$/#define > TXONLY_DEF_PACKET_LEN %d/' ./app/test- > > pmd/testpmd.h" % frame_size > > + self.dut.send_expect(self.change_testpmd_size, "#", 30) > > + self.dut.build_install_dpdk(self.dut.target) > > + > > + # Start the vhost user side > > + cmd =3D self.target + "/app/testpmd -n 4 -c 0x03 " + \ > > + "-m 2048 --file-prefix=3Dvhost --vdev > > 'net_vhost0,iface=3Dvhost-net,queues=3D1,client=3D0' -- -i" > > + self.dut.send_expect(cmd, "testpmd>", 120) > > + # Start the virtio_user side > > + vhost_user =3D self.dut.new_session() > > + command_line_user =3D self.target + "/app/testpmd -n 4 -c = 0x0c > > " + \ > > + " -m 2048 --no-pci --file-prefix=3Dvir= tio " > > + \ > > + " -- > > vdev=3Dnet_virtio_user0,mac=3D00:01:02:03:04:05,path=3D./vhost-net" + \ > > + " -- -i --txqflags=3D0xf00 --disable-h= w- > > vlan-filter" > > + > > + vhost_user.send_expect(command_line_user, "testpmd>", 120) > > + > > + self.dut.send_expect("set fwd mac retry", "testpmd>", 60) > > + self.dut.send_expect("start tx_first 32", "testpmd>", 60) > > + vhost_user.send_expect("start tx_first 32", "testpmd> ", 1= 20) > > + results =3D 0.0 > > + out =3D self.dut.send_expect("show port stats all", "testp= md>", > > 60) > > + time.sleep(5) > > + # Get throughput 10 times and calculate the average > > throughput > > + for i in range(10): > > + out =3D self.dut.send_expect("show port stats all", > > "testpmd>", 60) > > + time.sleep(5) > > + lines =3D re.search("Rx-pps:\s*(\d*)", out) > > + result =3D lines.group(1) > > + results +=3D float(result) > > + Mpps =3D results / (1000000 * 10) > > + self.result_table_create(self.header_row) > > + > > + self.dut.send_expect("quit", "#", 60) > > + vhost_user.send_expect("quit", "#", 60) > > + # Restore the driver file > > + self.dut.send_expect("rm - > > rf ./drivers/net/virtio/virtio_ethdev.h", "#", 30) > > + > > self.dut.send_expect("mv ./virtio_ethdev.h ./drivers/net/virtio/", "#",= 30) > > + self.test_cycles['Mpps'][frame_size] =3D Mpps > > + self.test_cycles['pct'][frame_size] =3D "Virtio 0.95" > > + > > + for frame_size in self.frame_sizes: > > + results_row =3D [frame_size] > > + results_row.append("Normal") > > + results_row.append(self.test_cycles['Mpps'][frame_size]) > > + results_row.append(self.test_cycles['pct'][frame_size]) > > + self.result_table_add(results_row) > > + > > + self.result_table_print() > > + > > + def tear_down(self): > > + """ > > + Run after each test case. > > + """ > > + time.sleep(2) > > + > > + def tear_down_all(self): > > + """ > > + Run after each test suite. > > + """ > > + # Recompile the dpdk because we change the source code during = the > > test >=20 > Here need to restore default Tx packet size. Sure. I will add it. >=20 > > + self.dut.build_install_dpdk(self.dut.target) > > -- > > 2.7.4