From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 262D7A2E1B for ; Wed, 4 Sep 2019 07:07:58 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id DAB751EC1D; Wed, 4 Sep 2019 07:07:57 +0200 (CEST) Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by dpdk.org (Postfix) with ESMTP id 53A431E9CF for ; Wed, 4 Sep 2019 07:07:55 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga107.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 03 Sep 2019 22:07:55 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.64,465,1559545200"; d="scan'208";a="184995861" Received: from fmsmsx103.amr.corp.intel.com ([10.18.124.201]) by orsmga003.jf.intel.com with ESMTP; 03 Sep 2019 22:07:54 -0700 Received: from fmsmsx122.amr.corp.intel.com (10.18.125.37) by FMSMSX103.amr.corp.intel.com (10.18.124.201) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 3 Sep 2019 22:07:54 -0700 Received: from shsmsx154.ccr.corp.intel.com (10.239.6.54) by fmsmsx122.amr.corp.intel.com (10.18.125.37) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 3 Sep 2019 22:07:54 -0700 Received: from shsmsx101.ccr.corp.intel.com ([169.254.1.92]) by SHSMSX154.ccr.corp.intel.com ([169.254.7.195]) with mapi id 14.03.0439.000; Wed, 4 Sep 2019 13:07:52 +0800 From: "Tu, Lijuan" To: "Zhu, ShuaiX" , "dts@dpdk.org" CC: "Wang, Yinan" , "Zhu, ShuaiX" Thread-Topic: [dts] [PATCH V1] tests/dpdk_hugetlbfs_mount_size: add new testsuite dpdk_hugetlbfs_mount_size Thread-Index: AQHVXKMsfJPWGp1nLU67xbWREH7UJqcbBBbQ Date: Wed, 4 Sep 2019 05:07:51 +0000 Message-ID: <8CE3E05A3F976642AAB0F4675D0AD20E0BB1BEAF@SHSMSX101.ccr.corp.intel.com> References: <1566888644-70444-1-git-send-email-shuaix.zhu@intel.com> In-Reply-To: <1566888644-70444-1-git-send-email-shuaix.zhu@intel.com> Accept-Language: zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-version: 11.2.0.6 dlp-reaction: no-action x-ctpclassification: CTP_NT x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiNzU1NzE5NDMtNDFjNS00MmY5LTllZDMtMmQ5MDVhODBjYTAzIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjEwLjE4MDQuNDkiLCJUcnVzdGVkTGFiZWxIYXNoIjoidFQwRGwzekx0STU0eENFZEdiRUlvUXJLUlJKMFBLbEM0V0RlSXZVcFNYWWdkc1hqWmd0RXJqVk9WOHMwQ01rRiJ9 x-originating-ip: [10.239.127.40] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dts] [PATCH V1] tests/dpdk_hugetlbfs_mount_size: add new testsuite dpdk_hugetlbfs_mount_size X-BeenThere: dts@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: test suite reviews and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dts-bounces@dpdk.org Sender: "dts" Applied, thanks > -----Original Message----- > From: dts [mailto:dts-bounces@dpdk.org] On Behalf Of zhu,shuai > Sent: Tuesday, August 27, 2019 2:51 PM > To: dts@dpdk.org > Cc: Wang, Yinan ; Zhu, ShuaiX > > Subject: [dts] [PATCH V1] tests/dpdk_hugetlbfs_mount_size: add new > testsuite dpdk_hugetlbfs_mount_size >=20 > add new testsuite dpdk_hugetlbfs_mount_size. >=20 > Signed-off-by: zhu,shuai > --- > tests/TestSuite_dpdk_hugetlbfs_mount_size.py | 273 +++++++++++++++++++ > 1 file changed, 273 insertions(+) > create mode 100644 tests/TestSuite_dpdk_hugetlbfs_mount_size.py >=20 > diff --git a/tests/TestSuite_dpdk_hugetlbfs_mount_size.py > b/tests/TestSuite_dpdk_hugetlbfs_mount_size.py > new file mode 100644 > index 0000000..8ce94b2 > --- /dev/null > +++ b/tests/TestSuite_dpdk_hugetlbfs_mount_size.py > @@ -0,0 +1,273 @@ > +#.. Copyright (c) <2019>, Intel Corporation > +# All rights reserved. > +# Redistribution and use in source and binary forms, with or without > +# modification, are permitted provided that the following conditions > +# are met: > + > +# - Redistributions of source code must retain the above copyright > +# notice, this list of conditions and the following disclaimer. > + > +# - Redistributions in binary form must reproduce the above copyright > +# notice, this list of conditions and the following disclaimer in > +# the documentation and/or other materials provided with the > +# distribution. > + > +# - Neither the name of Intel Corporation nor the names of its > +# contributors may be used to endorse or promote products derived > +# from this software without specific prior written permission. > + > +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND > CONTRIBUTORS > +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT > NOT > +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND > FITNESS > +# FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE > +# COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, > INDIRECT, > +# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES > +# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE > GOODS OR > +# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) > +# HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN > CONTRACT, > +# STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) > +# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF > ADVISED > +# OF THE POSSIBILITY OF SUCH DAMAGE. > + > +""" > +This feature is to limit DPDK to use the exact size which is the mounted > hugepage size. > +""" > + > +import re > +import utils > +import time > +from test_case import TestCase > + > +DEFAULT_MNT =3D '/mnt/huge' > +MNT_PATH =3D ["/mnt/huge1", "/mnt/huge2", "/mnt/huge3"] vhost_name =3D > +["vhost1", "vhost2", "vhost3"] > + > + > +class DpdkHugetlbfsMountSize(TestCase): > + > + def set_up_all(self): > + """ > + Run at the start of each test suite. > + """ > + self.packet_num =3D 100 > + self.mem_channels =3D self.dut.get_memory_channels() > + self.dut_ports =3D self.dut.get_ports(self.nic) > + self.verify(len(self.dut_ports) >=3D 2, "Insufficient ports for = testing") > + self.ports_socket =3D self.dut.get_numa_id(self.dut_ports[0]) > + cores =3D self.dut.get_core_list("1S/6C/1T", socket=3Dself.ports= _socket) > + self.verify(len(cores) >=3D 6, "Insufficient cores for speed tes= ting") > + self.core_list1 =3D ','.join(str(i) for i in cores[0:2]) > + self.core_list2 =3D ','.join(str(i) for i in cores[2:4]) > + self.core_list3 =3D ','.join(str(i) for i in cores[4:6]) > + self.pci_info_0 =3D self.dut.ports_info[0]['pci'] > + self.pci_info_1 =3D self.dut.ports_info[1]['pci'] > + self.numa_id =3D self.dut.get_numa_id(self.dut_ports[0]) > + self.create_folder([MNT_PATH[0], MNT_PATH[1], MNT_PATH[2]]) > + if self.numa_id =3D=3D 0: > + self.socket_mem =3D "1024,0" > + self.socket_mem2 =3D "2048,0" > + else: > + self.socket_mem =3D "0,1024" > + self.socket_mem2 =3D "0,2048" > + self.umount_huge([DEFAULT_MNT]) > + > + > + def set_up(self): > + """ > + Run before each test case. > + """ > + pass > + > + def close_session(self): > + self.dut.close_session(self.session_first) > + self.dut.close_session(self.session_secondary) > + > + def send_pkg(self, port_id): > + tx_interface =3D > self.tester.get_interface(self.tester.get_local_port(self.dut_ports[port_= id])) > + mac =3D self.dut.get_mac_address(self.dut_ports[port_id]) > + cmd =3D 'sendp([Ether(dst=3D"%s")/IP()/("X"*64)], iface=3D"%s", = count=3D%d)' > + excute_cmd =3D cmd % (mac, tx_interface, self.packet_num) > + self.tester.scapy_append(excute_cmd) > + self.tester.scapy_execute() > + > + def verify_result(self, session): > + out =3D session.send_expect("show port stats all", "testpmd> ", = 120) > + self.result_first =3D re.findall(r'RX-packets: (\w+)', out) > + self.result_secondary =3D re.findall(r'TX-packets: (\w+)', out) > + self.verify(int(self.result_first[0]) =3D=3D self.packet_num and > + int(self.result_secondary[0]) =3D=3D self.packet_num, "forward packets = no > + correctly") > + > + def create_folder(self, huges=3D[]): > + for huge in huges: > + cmd =3D ('mkdir -p %s' % huge) > + self.dut.send_expect(cmd, "#", 15) > + > + def del_folder(self, huges=3D[]): > + for huge in huges: > + cmd =3D ('rm -rf %s' % huge) > + self.dut.send_expect(cmd, "#", 15) > + > + def umount_huge(self, huges=3D[]): > + for huge in huges: > + cmd =3D ('umount %s' % huge) > + self.dut.send_expect(cmd, "#", 15) > + > + def test_default_hugepage_size(self): > + # Bind one nic port to igb_uio driver, launch testpmd > + self.dut.send_expect("mount -t hugetlbfs hugetlbfs %s" % > MNT_PATH[0], "#", 15) > + self.logger.info("test default hugepage size start testpmd witho= ut > numa") > + ttd =3D './%s/app/testpmd -l %s -n %d --huge-dir %s --file-prefi= x=3D%s -w %s > -- -i' > + launch_ttd =3D ttd % (self.target, self.core_list1, self.mem_cha= nnels, > MNT_PATH[0], vhost_name[0], self.pci_info_0) > + self.dut.send_expect(launch_ttd, "testpmd> ", 120) > + self.dut.send_expect("set promisc all off", "testpmd> ", 120) > + self.dut.send_expect("start", "testpmd> ", 120) > + self.dut.send_expect("clear port stats all", "testpmd> ", 120) > + self.send_pkg(0) > + self.verify_result(self.dut) > + self.dut.send_expect("quit", "#", 15) > + > + # resart testpmd with numa support > + self.logger.info("test default hugepage size start testpmd with = numa") > + ttd_secondary =3D './%s/app/testpmd -l %s -n %d --huge-dir %s --= file- > prefix=3D%s -w %s -- -i --numa' > + launch_ttd_secondary =3D ttd_secondary % (self.target, self.core= _list1, > self.mem_channels, MNT_PATH[0], vhost_name[0], self.pci_info_0) > + self.dut.send_expect(launch_ttd_secondary, "testpmd> ", 120) > + self.dut.send_expect("set promisc all off", "testpmd> ", 120) > + self.dut.send_expect("start", "testpmd> ", 120) > + self.dut.send_expect("clear port stats all", "testpmd> ", 120) > + self.send_pkg(0) > + self.verify_result(self.dut) > + self.dut.send_expect("quit", "#", 15) > + self.umount_huge([MNT_PATH[0]]) > + > + def > test_mount_size_exactly_match_hugepage_size_two_mount_points(self): > + # Bind two nic ports to igb_uio driver, launch testpmd with numa= ctl > + self.session_first =3D self.dut.new_session(suite=3D"session_fir= st") > + self.session_secondary =3D > self.dut.new_session(suite=3D"session_secondary") > + self.dut.send_expect("mount -t hugetlbfs -o size=3D4G hugetlbfs = %s" % > MNT_PATH[0], "#", 15) > + self.dut.send_expect("mount -t hugetlbfs -o size=3D4G hugetlbfs > + %s" % MNT_PATH[1], "#", 15) > + > + self.logger.info("start first testpmd") > + ttd =3D 'numactl --membind=3D%d ./%s/app/testpmd -l %s -n %d --l= egacy- > mem --socket-mem %s' \ > + ' --huge-dir %s --file-prefix=3D%s -w %s -- -i --socke= t-num=3D%d --no- > numa' > + launch_ttd =3D ttd % (self.numa_id, self.target, self.core_list1= , > self.mem_channels, self.socket_mem2, MNT_PATH[0], vhost_name[0], > self.pci_info_0, self.numa_id) > + self.session_first.send_expect(launch_ttd, "testpmd> ", 120) > + self.session_first.send_expect("set promisc all off", "testpmd> = ", 120) > + self.session_first.send_expect("start", "testpmd> ", 120) > + self.session_first.send_expect("clear port stats all", > + "testpmd> ", 120) > + > + self.logger.info("start secondary testpmd") > + ttd_secondary =3D 'numactl --membind=3D%d ./%s/app/testpmd -l %s= - > n %d --legacy-mem --socket-mem %s' \ > + ' --huge-dir %s --file-prefix=3D%s -w %s -- = -i --socket-num=3D%d - > -no-numa' > + launch_ttd_secondary =3D ttd_secondary % (self.numa_id, self.tar= get, > self.core_list2, self.mem_channels, self.socket_mem2, MNT_PATH[1], > vhost_name[1], self.pci_info_1, self.numa_id) > + self.session_secondary.send_expect(launch_ttd_secondary, "testpm= d> > ", 120) > + self.session_secondary.send_expect("set promisc all off", "testp= md> ", > 120) > + self.session_secondary.send_expect("start", "testpmd> ", 120) > + self.session_secondary.send_expect("clear port stats all", "test= pmd> ", > 120) > + self.send_pkg(0) > + self.send_pkg(1) > + self.verify_result(self.session_first) > + self.verify_result(self.session_secondary) > + self.session_first.send_expect("quit", "#", 15) > + self.session_secondary.send_expect("quit", "#", 15) > + self.close_session() > + self.umount_huge([MNT_PATH[0], MNT_PATH[1]]) > + > + def > test_mount_size_greater_than_hugepage_size_single_mount_point(self): > + # Bind one nic port to igb_uio driver > + self.dut.send_expect("mount -t hugetlbfs -o size=3D9G hugetlbfs = %s" % > MNT_PATH[0], "#", 15) > + ttd =3D './%s/app/testpmd -l %s -n %d --legacy-mem --huge-dir %s= --file- > prefix=3D%s -w %s -- -i' > + launch_ttd =3D ttd % (self.target, self.core_list1, self.mem_cha= nnels, > MNT_PATH[0], vhost_name[0], self.pci_info_0) > + self.dut.send_expect(launch_ttd, "testpmd> ", 120) > + self.dut.send_expect("set promisc all off", "testpmd> ", 120) > + self.dut.send_expect("start", "testpmd> ", 120) > + self.dut.send_expect("clear port stats all", "testpmd> ", 120) > + self.send_pkg(0) > + self.verify_result(self.dut) > + self.dut.send_expect("quit", "#", 15) > + self.umount_huge([MNT_PATH[0]]) > + > + def > test_mount_size_greater_than_hugepage_size_multiple_mount_points(self): > + # Bind one nic port to igb_uio driver, launch testpmd > + self.session_first =3D self.dut.new_session(suite=3D"session_fir= st") > + self.session_secondary =3D > self.dut.new_session(suite=3D"session_secondary") > + self.dut.send_expect("mount -t hugetlbfs -o size=3D4G hugetlbfs = %s" % > MNT_PATH[0], "#", 15) > + self.dut.send_expect("mount -t hugetlbfs -o size=3D4G hugetlbfs = %s" % > MNT_PATH[1], "#", 15) > + self.dut.send_expect("mount -t hugetlbfs -o size=3D1G hugetlbfs = %s" % > MNT_PATH[2], "#", 15) > + # launch first testpmd > + self.logger.info("launch first testpmd") > + ttd =3D 'numactl --membind=3D%d ./%s/app/testpmd -l %s -n %d --= legacy- > mem --socket-mem %s --huge-dir %s' \ > + ' --file-prefix=3D%s -w %s -- -i --socket-num=3D%d --no-n= uma' > + launch_ttd =3D ttd % (self.numa_id, self.target, self.core_list1= , > self.mem_channels, self.socket_mem2, MNT_PATH[0], vhost_name[0], > self.pci_info_0, self.numa_id) > + self.session_first.send_expect(launch_ttd, "testpmd> ", 120) > + self.session_first.send_expect("set promisc all off", "testpmd> = ", 120) > + self.session_first.send_expect("start", "testpmd> ", 120) > + self.session_first.send_expect("start", "testpmd> ", 120) > + > + # launch secondary testpmd > + self.logger.info("launch secondary testpmd") > + ttd_secondary =3D 'numactl --membind=3D%d ./%s/app/testpmd -l %s= - > n %d --legacy-mem --socket-mem %s --huge-dir' \ > + ' %s --file-prefix=3D%s -w %s -- -i --socket= -num=3D%d --no-numa' > + launch_ttd_secondary =3D ttd_secondary % (self.numa_id, self.tar= get, > self.core_list2, self.mem_channels, self.socket_mem2, MNT_PATH[1], > vhost_name[1], self.pci_info_0, self.numa_id) > + self.session_secondary.send_expect(launch_ttd_secondary, "testpm= d> > ", 120) > + self.session_secondary.send_expect("set promisc all off", "testp= md> ", > 120) > + self.session_secondary.send_expect("start", "testpmd> ", 120) > + self.session_secondary.send_expect("start", "testpmd> ", 120) > + > + # launch third testpmd > + self.logger.info("launch third testpmd") > + ttd_third =3D 'numactl --membind=3D%d ./%s/app/testpmd -l %s -n = %d -- > legacy-mem --socket-mem %s --huge-dir' \ > + ' %s --file-prefix=3D%s -w %s -- -i --socket-num= =3D%d --no-numa' > + launch_ttd_third =3D ttd_third % (self.numa_id, self.target, sel= f.core_list3, > self.mem_channels, self.socket_mem, MNT_PATH[2], vhost_name[2], > self.pci_info_0, self.numa_id) > + expect_str =3D 'Not enough memory available on socket' > + self.dut.get_session_output(timeout=3D2) > + try: > + self.dut.send_expect(launch_ttd_third, expect_str, 120) > + except Exception as e: > + print(e) > + self.dut.send_expect("quit", "#", 15) > + self.session_first.send_expect("quit", "#", 15) > + self.session_secondary.send_expect("quit", "#", 15) > + self.umount_huge([MNT_PATH[0], MNT_PATH[1], MNT_PATH[2]]) > + self.verify(0, 'the expect str: %s ,not in output info' % ex= pect_str) > + self.logger.info('the third testpmd start failed as expect : %s'= % > expect_str) > + result =3D self.dut.get_session_output(timeout=3D2) > + print result > + > + # start send packet and verify the session can receive the packe= t. > + self.send_pkg(0) > + self.verify_result(self.session_first) > + self.verify_result(self.session_secondary) > + self.session_first.send_expect("quit", "#", 15) > + self.session_secondary.send_expect("quit", "#", 15) > + self.close_session() > + self.umount_huge([MNT_PATH[0], MNT_PATH[1], MNT_PATH[2]]) > + > + def test_run_dpdk_app_limited_hugepages_controlled_by_cgroup(self): > + # Bind one nic port to igb_uio driver, launch testpmd in limited > hugepages > + self.dut.send_expect("mount -t hugetlbfs nodev %s" % MNT_PATH[0]= , > "#", 15) > + self.dut.send_expect("cgcreate -g hugetlb:/test-subgroup", "# ",= 15) > + self.dut.send_expect("cgset -r hugetlb.1GB.limit_in_bytes=3D2147= 483648 > test-subgroup", "#", 15) > + ttd =3D 'cgexec -g hugetlb:test-subgroup numactl - > m %d ./%s/app/testpmd -l %s -n %d -w %s -- -i --socket-num=3D%d --no-numa= ' > + launch_ttd =3D ttd % (self.numa_id, self.target, self.core_list1= , > self.mem_channels, self.pci_info_0, self.numa_id) > + self.dut.send_expect(launch_ttd, "testpmd> ", 120) > + self.dut.send_expect("set promisc all off", "testpmd> ", 120) > + self.dut.send_expect("start", "testpmd> ", 120) > + self.dut.send_expect("clear port stats all", "testpmd> ", 120) > + self.send_pkg(0) > + self.verify_result(self.dut) > + self.dut.send_expect("quit", "#", 15) > + self.umount_huge([MNT_PATH[0]]) > + > + def tear_down(self): > + """ > + Run after each test case. > + """ > + self.dut.kill_all() > + time.sleep(2) > + > + def tear_down_all(self): > + """ > + Run after each test suite. > + """ > + self.umount_huge([MNT_PATH[0], MNT_PATH[1], MNT_PATH[2]]) > + self.del_folder([MNT_PATH[0], MNT_PATH[1], MNT_PATH[2]]) > + self.dut.send_expect("mount -t hugetlbfs nodev %s" % > + DEFAULT_MNT, "#", 15) > -- > 2.17.2