From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR04-VI1-obe.outbound.protection.outlook.com (mail-eopbgr80054.outbound.protection.outlook.com [40.107.8.54]) by dpdk.org (Postfix) with ESMTP id 88BE720BD for ; Fri, 21 Sep 2018 05:22:03 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector1-arm-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=60g7uUWYckfmaar790M+OdA2woA9de+2e6WlziwZZoI=; b=fblxcq3HQFYA140zoHVQse3eRDPgkWQrI4dzxxOuCtX3j2/5Ou0Lb3pLqB0aCsNqKF/Dnz8CtqKS66EATY4pCNUMmF8J5YtSl9GdYO0Ra8++6BNZmCHJwenmcp7Qr6iGyh4elDzfP4fcI9Z8UdP9VX5zqQwK2GII90EG00mnSRY= Received: from DB7PR08MB3385.eurprd08.prod.outlook.com (20.176.238.90) by DB7PR08MB3177.eurprd08.prod.outlook.com (52.134.110.151) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1143.18; Fri, 21 Sep 2018 03:22:01 +0000 Received: from DB7PR08MB3385.eurprd08.prod.outlook.com ([fe80::f0e6:1c3d:9285:472f]) by DB7PR08MB3385.eurprd08.prod.outlook.com ([fe80::f0e6:1c3d:9285:472f%3]) with mapi id 15.20.1143.017; Fri, 21 Sep 2018 03:21:58 +0000 From: "Phil Yang (Arm Technology China)" To: "Tu, Lijuan" , "Phil@dpdk.org" , "dts@dpdk.org" CC: nd , "Liu, Yong" Thread-Topic: [dts] [PATCH v5 01/22] framework/packet: support packet sniffer to specify running machine Thread-Index: AQHUUViHHgNaAboMz0qWfdwdCXAIRaT6D6Mg Date: Fri, 21 Sep 2018 03:21:58 +0000 Message-ID: References: <1523526771-7437-1-git-send-email-phil.yang@arm.com> <1536049595-22271-1-git-send-email-phil.yang@arm.com> <8CE3E05A3F976642AAB0F4675D0AD20EEC707E@SHSMSX101.ccr.corp.intel.com> In-Reply-To: <8CE3E05A3F976642AAB0F4675D0AD20EEC707E@SHSMSX101.ccr.corp.intel.com> Accept-Language: zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=Phil.Yang@arm.com; x-originating-ip: [113.29.88.7] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; DB7PR08MB3177; 6:KXXkwCEkyphwH/fRJ0xJBEvPplR7e3hwAg14K7Z87oLTKNVDEg/S5svR+Ok9Ghggd6R0w2qK9l45etYhlXyVqARWlmt6tmYwAsxOmeYsoZYNIlSIKSyWZx1LSZJ+XpNB2ypuPWWjJ4J77kz7G9vFYUsS5KE1ohNG0SOzHiJ5y7vTXdjubhRytpwbW+BPYYXHNnZoAMcBSH+B6PAoF6lSKXnC4eh2WdkY00aCCT1yPWe+lq6+ELq6ubJMPHQBE51jP0DgcGdDClQ0UlfinPYzuUv5rT1W0wCErlSbtfnq3Qc+WFbbVGuCqywvQkoZmyAA9O35ANh+YfRu0bCVZHu2+N7JJ+tM/I475EVMrGXVnzwppvOZX9zojrjm71HYKE9tsZwq6mwofP2V+vMIePmsa2pWWY91JB5AGpV5e6S7IEtt+vBBqMHrjE5pA7jJ2zLBeLdrrzOB5D3MNBNiYHkerw==; 5:uDUgw1/KbdefuFPTbYhMAZqCPhfWqf5YaeqNpw+1t32nMlrlh8pgFn331kTe3HshnwLZF48zI3817i9J4SICspAYTWvK5Ljg/c98R6eI34XTqRWs7tapqeJMFuz9MgF2+JYlpDebz+jA7NXPb/UoYxX/pI4ZaOQqygwa6KORO6k=; 7:16nAARLzaG39derto+yQu9OzaHfjky+iXO8mBu52hjYvz/uMB9x+HmzeqYHrCBvt9xDylO0xBaihW6+xdkgqGaZImwYrjgrULtxTbn56l4PUjsh0ociA632aFpA3vBfu1MSkB3jqcuqyrkR5nxQRjOZjt1gHilbtSGFUjAlZfdnOdFIet3Uh80zMAfwEyOccsNE39JSkPWJWzWjfIDA/S1sTFlpzDg3jzU2fNpHMGhs/aTZFZv2QxMwFjifeo/M4 x-ms-exchange-antispam-srfa-diagnostics: SOS; x-ms-office365-filtering-correlation-id: 58f35d6a-e4da-410d-cb71-08d61f71640e x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(7020095)(4652040)(8989299)(4534165)(7168020)(4627221)(201703031133081)(201702281549075)(8990200)(5600074)(711020)(4618075)(2017052603328)(7153060)(7193020); SRVR:DB7PR08MB3177; x-ms-traffictypediagnostic: DB7PR08MB3177: nodisclaimer: True x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(158342451672863)(180628864354917)(228905959029699); x-ms-exchange-senderadcheck: 1 x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(10201501046)(3002001)(3231355)(944501410)(52105095)(93006095)(93001095)(6055026)(149027)(150027)(6041310)(20161123562045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123560045)(20161123564045)(20161123558120)(201708071742011)(7699051); SRVR:DB7PR08MB3177; BCL:0; PCL:0; RULEID:; SRVR:DB7PR08MB3177; x-forefront-prvs: 0802ADD973 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(366004)(136003)(396003)(346002)(39860400002)(376002)(13464003)(199004)(189003)(9686003)(5660300001)(110136005)(229853002)(2906002)(2900100001)(256004)(186003)(55236004)(6506007)(966005)(26005)(5250100002)(102836004)(551544002)(105586002)(14454004)(2501003)(53546011)(11346002)(476003)(53936002)(4326008)(316002)(25786009)(305945005)(6246003)(6436002)(8936002)(6306002)(76176011)(7736002)(55016002)(33656002)(486006)(81156014)(66066001)(14444005)(106356001)(74316002)(72206003)(86362001)(2201001)(21490400003)(68736007)(99286004)(54906003)(3846002)(97736004)(6116002)(478600001)(81166006)(7696005)(446003)(71200400001)(71190400001); DIR:OUT; SFP:1101; SCL:1; SRVR:DB7PR08MB3177; H:DB7PR08MB3385.eurprd08.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: C9qrJ1paXI9UdHDCmqK2Kxg0KBw/ZCj7I2tTFaGGjC3VPZZHOi1VUvMEm+u4FFlYf69JuZW+tSlqbgRZWaeOaT/YwtDemShIYDxxjf8J6ilrpBbFoHFqL1k32EB3XIrghDrS4C36ou6nLDRbS8tlP6btijhqLNUZgFkNa7wWnX+03MNMl42xGJPZwoG6OgCU3rCO+zNiWwonidHhLID25EQHMtQvl41Dka0fB9Uc8upJEhS13ayeobzkT4QfGqVXKek8kEZdkLQaKSlPpZrBiJET+0GEENhjHSOwsCL+mZCSsX9OxyLOvDJMYAAxLEYxtOat5S5estY+vsRDYikJoBEhWQ9/F2xHGyKojsLlzCs= spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-Network-Message-Id: 58f35d6a-e4da-410d-cb71-08d61f71640e X-MS-Exchange-CrossTenant-originalarrivaltime: 21 Sep 2018 03:21:58.8559 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB7PR08MB3177 Subject: Re: [dts] [PATCH v5 01/22] framework/packet: support packet sniffer to specify running machine X-BeenThere: dts@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: test suite reviews and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Sep 2018 03:22:03 -0000 Hi Lijuan, It also need to follow the DTS setup steps in https://doc.dpdk.org/dts/gsg/= sys_reqs.html#authorized-login-session . Since you know where are your DUT and Tester, so you can use tool ssh-copy-= id to save local available keys on DUT and Tester. Thanks, Phil Yang > -----Original Message----- > From: Tu, Lijuan > Sent: Friday, September 21, 2018 11:09 AM > To: Phil@dpdk.org; dts@dpdk.org > Cc: nd ; Liu, Yong ; Phil Yang (Arm > Technology China) > Subject: RE: [dts] [PATCH v5 01/22] framework/packet: support packet snif= fer to > specify running machine >=20 > One comments, when we ssh to a server, it might request a password, but w= e > don't handle it. >=20 >=20 > > -----Original Message----- > > From: dts [mailto:dts-bounces@dpdk.org] On Behalf Of Phil@dpdk.org > > Sent: Tuesday, September 4, 2018 4:26 PM > > To: dts@dpdk.org > > Cc: nd@arm.com; Liu, Yong ; Phil Yang > > > > Subject: [dts] [PATCH v5 01/22] framework/packet: support packet > > sniffer to specify running machine > > > > From: Phil Yang > > > > By default, the Tester is supposed to be the server which running DTS > > and the packet sniff methods are running on it. > > However, if DTS was not running on the Tester and the Tester is > > another remote server, so packet sniff methods cannot sniff Tester's > > packets correctly. > > > > This patch adds support for packet sniff methods to specify running > > machine and implements sniff packet methods in class tester. > > > > 1. Add parameter to sniff_packet and load_sniff_pcap methods to > > specify the running machine. > > 2. Remove load_sniff_packets method in packet.py. > > 3. Add tcpdump_sniff_packets, load_tcpdump_sniff_pcap and > > load_tcpdump_sniff_packets for class tester with the new sniff_packet A= PI. > > 4. Update tester.check_random_pkts method with > > tester.tcpdump_sniff_packets and tester.load_tcpdump_sniff_packets to > > make it execution on the Tester. > > > > Signed-off-by: Phil Yang > > Suggested-by: Marvin Liu > > --- > > framework/packet.py | 86 > > +++++++++++++++++++++++++---------------------------- > > framework/tester.py | 48 +++++++++++++++++++++++++++--- > > 2 files changed, 84 insertions(+), 50 deletions(-) > > > > diff --git a/framework/packet.py b/framework/packet.py index > > 976b82b..1f3f07d 100755 > > --- a/framework/packet.py > > +++ b/framework/packet.py > > @@ -812,15 +812,31 @@ def get_filter_cmd(filters=3D[]): > > return "" > > > > > > -def sniff_packets(intf, count=3D0, timeout=3D5, filters=3D[]): > > +def sniff_packets(intf, count=3D0, timeout=3D5, filters=3D[], target= =3D[]): > > """ > > sniff all packets for certain port in certain seconds > > """ > > param =3D "" > > direct_param =3D r"(\s+)\[ (\S+) in\|out\|inout \]" > > - tcpdump_help =3D subprocess.check_output("tcpdump -h; echo 0", > > - > > stderr=3Dsubprocess.STDOUT, > > - shell=3DTrue) > > + remote_terminate =3D 0 > > + > > + # target[] contain the remote machine info for ssh connection > > + # target[0]: username > > + # target[1]: ip address > > + # target[2]: pass word > > + if target: > > + tcpdump_help_pipe =3D subprocess.Popen(["ssh", > > + "%s@%s" % (target[0], target[1]), > > + "tcpdump -h"], > > + stderr=3Dsubprocess.PIPE, > > + stdout=3Dsubprocess.PIPE, > > + shell=3DFalse) > [Lijuan] Action will be blocked if SSH need password. > > + tcpdump_help =3D > > "".join(tuple(tcpdump_help_pipe.communicate())) > > + tcpdump_help_pipe.wait() > > + else: > > + tcpdump_help =3D subprocess.check_output("tcpdump -h; echo 0", > > + stderr=3Dsubprocess.STDOUT, > > + shell=3DTrue) > > + > > for line in tcpdump_help.split('\n'): > > m =3D re.match(direct_param, line) > > if m: > > @@ -850,22 +866,29 @@ def sniff_packets(intf, count=3D0, timeout=3D5, > > filters=3D[]): > > else: > > cmd =3D sniff_cmd % options > > > > - args =3D shlex.split(cmd) > > + if target: > > + pipe =3D subprocess.Popen(['ssh', > > + '%s@%s' % (target[0], target[1]), cmd], > > + stdin=3Dsubprocess.PIPE, > > + shell=3DFalse) > > + remote_terminate =3D 1 > > + else: > > + args =3D shlex.split(cmd) > > + pipe =3D subprocess.Popen(args) > > > > - pipe =3D subprocess.Popen(args) > > index =3D str(time.time()) > > - SNIFF_PIDS[index] =3D (pipe, intf, timeout) > > + SNIFF_PIDS[index] =3D (pipe, intf, timeout, remote_terminate) > > time.sleep(1) > > return index > > > > > > -def load_sniff_pcap(index=3D''): > > +def load_sniff_pcap(index=3D'', target=3D[]): > > """ > > Stop sniffer and return pcap file > > """ > > child_exit =3D False > > if index in SNIFF_PIDS.keys(): > > - pipe, intf, timeout =3D SNIFF_PIDS[index] > > + pipe, intf, timeout, remote_terminate =3D SNIFF_PIDS[index] > > time_elapse =3D int(time.time() - float(index)) > > while time_elapse < timeout: > > if pipe.poll() is not None: > > @@ -876,6 +899,14 @@ def load_sniff_pcap(index=3D''): > > time_elapse +=3D 1 > > > > if not child_exit: > > + if remote_terminate =3D=3D 1: > > + stop_tcpdump_pipe =3D subprocess.Popen(['ssh', > > + '%s@%s' % (target[0], > > target[1]), > > + 'kill -2 $(pidof tcpdump)'], > > + stderr=3Dsubprocess.PIPE, > > + shell=3DFalse) > > + stop_tcpdump_pipe.wait() > > + > > pipe.send_signal(signal.SIGINT) > > pipe.wait() > > > > @@ -886,42 +917,6 @@ def load_sniff_pcap(index=3D''): > > return "" > > > > > > -def load_sniff_packets(index=3D''): > > - """ > > - Stop sniffer and return packet objects > > - """ > > - pkts =3D [] > > - child_exit =3D False > > - if index in SNIFF_PIDS.keys(): > > - pipe, intf, timeout =3D SNIFF_PIDS[index] > > - time_elapse =3D int(time.time() - float(index)) > > - while time_elapse < timeout: > > - if pipe.poll() is not None: > > - child_exit =3D True > > - break > > - > > - time.sleep(1) > > - time_elapse +=3D 1 > > - > > - if not child_exit: > > - pipe.send_signal(signal.SIGINT) > > - pipe.wait() > > - > > - # wait pcap file ready > > - time.sleep(1) > > - try: > > - cap_pkts =3D rdpcap("/tmp/sniff_%s.pcap" % intf) > > - for pkt in cap_pkts: > > - # packet gen should be scapy > > - packet =3D Packet(tx_port=3Dintf) > > - packet.pktgen.assign_pkt(pkt) > > - pkts.append(packet) > > - except: > > - pass > > - > > - return pkts > > - > > - > > def load_pcapfile(filename=3D""): > > pkts =3D [] > > try: > > @@ -983,7 +978,6 @@ if __name__ =3D=3D "__main__": > > inst =3D sniff_packets("lo", timeout=3D5) > > pkt =3D Packet(pkt_type=3D'UDP') > > pkt.send_pkt(tx_port=3D'lo') > > - pkts =3D load_sniff_packets(inst) > > > > pkt =3D Packet(pkt_type=3D'UDP', pkt_len=3D1500, ran_payload=3DTru= e) > > pkt.send_pkt(tx_port=3D'lo') > > diff --git a/framework/tester.py b/framework/tester.py index > > a775f68..c5b705d 100755 > > --- a/framework/tester.py > > +++ b/framework/tester.py > > @@ -35,6 +35,7 @@ Interface for bulk traffic generators. > > > > import re > > import subprocess > > +import os > > from time import sleep > > from settings import NICS, load_global_setting, PERF_SETTING from > > crb import Crb @@ -560,8 +561,6 @@ class Tester(Crb): > > module =3D __import__("packet") > > pkt_c =3D getattr(module, "Packet") > > send_f =3D getattr(module, "send_packets") > > - sniff_f =3D getattr(module, "sniff_packets") > > - load_f =3D getattr(module, "load_sniff_packets") > > compare_f =3D getattr(module, "compare_pktload") > > strip_f =3D getattr(module, "strip_pktload") > > save_f =3D getattr(module, "save_packets") @@ -606,7 +605,7 @@ > > class Tester(Crb): > > > > # send and sniff packets > > save_f(pkts=3Dpkts, filename=3D"/tmp/%s_tx.pcap" % txIntf) > > - inst =3D sniff_f(intf=3DrxIntf, count=3Dpktnum, timeout=3D= timeout, > > filters=3D > > + inst =3D self.tcpdump_sniff_packets(intf=3DrxIntf, > > + count=3Dpktnum, timeout=3Dtimeout, filters=3D > > [{'layer': 'network', 'config': {'srcport': '65535'}}, > > {'layer': 'network', 'config': {'dstport': '65535'}}]= ) > > rx_inst[rxport] =3D inst > > @@ -627,7 +626,7 @@ class Tester(Crb): > > # Verify all packets > > prev_id =3D -1 > > for txport, rxport in portList: > > - recv_pkts =3D load_f(rx_inst[rxport]) > > + recv_pkts =3D > > + self.load_tcpdump_sniff_packets(rx_inst[rxport]) > > > > # only report when recevied number not matched > > if len(tx_pkts[txport]) > len(recv_pkts): > > @@ -704,6 +703,47 @@ class Tester(Crb): > > self.proc.kill() > > self.proc =3D None > > > > + def tcpdump_sniff_packets(self, intf, count=3D0, timeout=3D5, filt= ers=3D[]): > > + """ > > + Wrapper for packet module sniff_packets > > + """ > > + # load functions in packet module > > + module =3D __import__("packet") > > + sniff_f =3D getattr(module, "sniff_packets") > > + > > + target=3D[] > > + target.append(self.get_username()) > > + target.append(self.get_ip_address()) > > + target.append(self.get_password()) > > + return sniff_f(intf, count, timeout, filters, target) > > + > > + def load_tcpdump_sniff_pcap(self, index=3D''): > > + """ > > + Wrapper for packet module load_sniff_pcap > > + """ > > + # load functions in packet module > > + module =3D __import__("packet") > > + load_pcap_f =3D getattr(module, "load_sniff_pcap") > > + > > + target=3D[] > > + target.append(self.get_username()) > > + target.append(self.get_ip_address()) > > + target.append(self.get_password()) > > + pcap =3D load_pcap_f(index, target) > > + self.session.copy_file_from(pcap) > > + > > + return pcap.split(os.sep)[-1] > > + > > + def load_tcpdump_sniff_packets(self, index=3D''): > > + """ > > + Wrapper for packet module load_pcapfile > > + """ > > + # load functions in packet module > > + packet =3D __import__("packet") > > + file =3D self.load_tcpdump_sniff_pcap(index) > > + > > + return packet.load_pcapfile(file) > > + > > def kill_all(self, killall=3DFalse): > > """ > > Kill all scapy process or DPDK application on tester. > > -- > > 2.7.4