From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR01-DB5-obe.outbound.protection.outlook.com (mail-db5eur01on0048.outbound.protection.outlook.com [104.47.2.48]) by dpdk.org (Postfix) with ESMTP id 859572B9A for ; Mon, 17 Sep 2018 07:12:45 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector1-arm-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=gOM0UwWTu9ImML4YjUZt50JX24x7uirmyBKANI5wW60=; b=VpCmRHGoxrfSU780gDKBvUYUodZEt+tIIP/eSsIcHz6f8VnMZsgpWV9661HkKTRB4qUEd4ASsJ77E4Oq0xhLNfaH1Yr8BjnB2LLLxUSKlmbaqJLqLFAE1DgOAvRC7Ek7I8ICyAq1+VQHRWEyEyVhU18FpR24gZajxfMh7N7uZ2c= Received: from DB7PR08MB3385.eurprd08.prod.outlook.com (20.176.238.90) by DB7PR08MB3401.eurprd08.prod.outlook.com (20.176.238.94) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1122.21; Mon, 17 Sep 2018 05:12:44 +0000 Received: from DB7PR08MB3385.eurprd08.prod.outlook.com ([fe80::f0e6:1c3d:9285:472f]) by DB7PR08MB3385.eurprd08.prod.outlook.com ([fe80::f0e6:1c3d:9285:472f%3]) with mapi id 15.20.1143.017; Mon, 17 Sep 2018 05:12:44 +0000 From: "Phil Yang (Arm Technology China)" To: "Phil Yang (Arm Technology China)" , "dts@dpdk.org" CC: nd , "yong.liu@intel.com" Thread-Topic: [PATCH v5 01/22] framework/packet: support packet sniffer to specify running machine Thread-Index: AQHURClyYx6W8TPe2kKHrENIhrEnnKT0Aa0g Date: Mon, 17 Sep 2018 05:12:44 +0000 Message-ID: References: <1523526771-7437-1-git-send-email-phil.yang@arm.com> <1536049595-22271-1-git-send-email-phil.yang@arm.com> In-Reply-To: <1536049595-22271-1-git-send-email-phil.yang@arm.com> Accept-Language: zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=Phil.Yang@arm.com; x-originating-ip: [113.29.88.7] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; DB7PR08MB3401; 6:owxFrpFRMLhIWqTobIG2iXU0mqqv6pHiicO15YjV0V7aJuVxHsBfwGmppxQWBpZWACdG4GB93oHwW6QFwjv0SROL0zKYNrqWU1gDcZhy4/urnXpLP3bNm0Qa3pq9ZD1dQC/t7ZbmX7rxLJnEr03a2q3OCYsbkzOqWB6bBLOh7PalMZVCh107GB4tVMPjpRaN5MzuzqIsE4dEsu6piPr84gz+dQjfBRyiuCLgcvlYIQEthQuFz6COcX2n5Yb/NucdsMpcHIUyt+Tg4IhbNY7N6/7CMTmNEKFJ6Pi++4nhWGOiBdptz8HA0EAwxqQvbo9jKyHQzd6O3hNUfOC92OmXptfsQThOucUyGtNtDNpp+nXLzO8aMTouOaSNeYOzRzCTXiEGnEvABN/avW2PVHWxRwp5Ua2T+y0kx2tQS971XzxCGDc+twaVVVFLwXvjI2+ZvV1R090Owfhq5lxFS1187g==; 5:c9WdzwPSciK4/bpRHejiUDYWCJC7m6j0EfLSOhSeVFU2Qh8id1HvwZEJfnMhby7OGHMTv2IkwZ72xzls/TlGZ+SInSNwAhOrNKb3oXA3cD6nP/HxAyaKWUKrdD3roFqAIXnX8Am5QY1tdWgeyu4pUkbQWbvUe5v3oPOs5TTC9dc=; 7:CwvDZ6PODPKs5tDbxgJzZvZ00r8ipVn94hD8CldPWBp3Fwn5IdVIXvaFoCINj9twuSzKz2/qBf/6cIX/m36ZYbGfmJmpJ0Msb6NzFvxdM9b1wIK5fEXITYCc4hBittj4l08/7dhgtdbAVkn9jrvbn5T25i3s384dg0Ltwk+hZe+ntKAhJKb529CBcB/K1rsIy5r4I4D3P2EENyXWL/XkGi045lc0/oddGdNxy2LmeMdIFB8v51grIDYpXvAiexkL x-ms-exchange-antispam-srfa-diagnostics: SOS;SOR; x-ms-office365-filtering-correlation-id: 60ca0cfb-d052-4aa8-60f8-08d61c5c335b x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(7020095)(4652040)(8989137)(5600074)(711020)(4618075)(2017052603328)(7153060)(7193020); SRVR:DB7PR08MB3401; x-ms-traffictypediagnostic: DB7PR08MB3401: nodisclaimer: True x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(158342451672863)(180628864354917)(228905959029699); x-ms-exchange-senderadcheck: 1 x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(3231355)(944501410)(52105095)(10201501046)(93006095)(93001095)(3002001)(6055026)(149027)(150027)(6041310)(20161123558120)(20161123564045)(20161123560045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123562045)(201708071742011)(7699050); SRVR:DB7PR08MB3401; BCL:0; PCL:0; RULEID:; SRVR:DB7PR08MB3401; x-forefront-prvs: 0798146F16 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(366004)(39850400004)(346002)(396003)(136003)(376002)(13464003)(199004)(189003)(53936002)(5250100002)(2501003)(551544002)(105586002)(14454004)(21490400003)(5660300001)(106356001)(229853002)(33656002)(316002)(8676002)(76176011)(4326008)(3846002)(6116002)(256004)(54906003)(110136005)(72206003)(2906002)(14444005)(478600001)(25786009)(53546011)(74316002)(305945005)(8936002)(99286004)(9686003)(81166006)(7696005)(6436002)(66066001)(486006)(2900100001)(11346002)(476003)(55016002)(446003)(81156014)(6506007)(6246003)(68736007)(86362001)(26005)(7736002)(102836004)(97736004); DIR:OUT; SFP:1101; SCL:1; SRVR:DB7PR08MB3401; H:DB7PR08MB3385.eurprd08.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: TQiUivZ3UD68GHey7iwefhdp7q0ZgI3eWfOk04+uKvPwXBPTm2vRZyVWlhQws+t9Zh9FOjYXCtNKPdldYD03XfmHWACfNKA5hDXtlwPhLQEpb5ZITJ+Sf9IaZWK2b/dviGUiFRlzQoesSUS/OIXckaWKQGNPDB6IXlptw8kf9d+11+h2RUcy0En4y5ZqkeJwV+9AW0vFO/9LaTY3up8ilC1XVbX9rDl1U9KmTZajfYiJ5bQ5h0MWfmeiZ2bMu+z0o6+K8lHzXHXgLlxwHtVD1ezB/RO9YzBZwd0dqwAKurDUtUJAR70jDKDxJEb3n9kHKvy9yF26N/Czlq4F6nuEZeyNqshR+9N20n9hFe3TD2g= spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-Network-Message-Id: 60ca0cfb-d052-4aa8-60f8-08d61c5c335b X-MS-Exchange-CrossTenant-originalarrivaltime: 17 Sep 2018 05:12:44.2908 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB7PR08MB3401 Subject: Re: [dts] [PATCH v5 01/22] framework/packet: support packet sniffer to specify running machine X-BeenThere: dts@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: test suite reviews and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Sep 2018 05:12:45 -0000 Hi, Any comments for those patches? Thanks, Phil Yang > -----Original Message----- > From: Phil Yang > Sent: Tuesday, September 4, 2018 4:26 PM > To: dts@dpdk.org > Cc: nd ; yong.liu@intel.com; Phil Yang (Arm Technology China) > > Subject: [PATCH v5 01/22] framework/packet: support packet sniffer to spe= cify > running machine >=20 > From: Phil Yang >=20 > By default, the Tester is supposed to be the server which running DTS and= the > packet sniff methods are running on it. > However, if DTS was not running on the Tester and the Tester is another r= emote > server, so packet sniff methods cannot sniff Tester's packets correctly. >=20 > This patch adds support for packet sniff methods to specify running machi= ne and > implements sniff packet methods in class tester. >=20 > 1. Add parameter to sniff_packet and load_sniff_pcap methods to specify t= he > running machine. > 2. Remove load_sniff_packets method in packet.py. > 3. Add tcpdump_sniff_packets, load_tcpdump_sniff_pcap and > load_tcpdump_sniff_packets for class tester with the new sniff_packet API= . > 4. Update tester.check_random_pkts method with > tester.tcpdump_sniff_packets and tester.load_tcpdump_sniff_packets to mak= e > it execution on the Tester. >=20 > Signed-off-by: Phil Yang > Suggested-by: Marvin Liu > --- > framework/packet.py | 86 +++++++++++++++++++++++++----------------------= ----- > - > framework/tester.py | 48 +++++++++++++++++++++++++++--- > 2 files changed, 84 insertions(+), 50 deletions(-) >=20 > diff --git a/framework/packet.py b/framework/packet.py index > 976b82b..1f3f07d 100755 > --- a/framework/packet.py > +++ b/framework/packet.py > @@ -812,15 +812,31 @@ def get_filter_cmd(filters=3D[]): > return "" >=20 >=20 > -def sniff_packets(intf, count=3D0, timeout=3D5, filters=3D[]): > +def sniff_packets(intf, count=3D0, timeout=3D5, filters=3D[], target=3D[= ]): > """ > sniff all packets for certain port in certain seconds > """ > param =3D "" > direct_param =3D r"(\s+)\[ (\S+) in\|out\|inout \]" > - tcpdump_help =3D subprocess.check_output("tcpdump -h; echo 0", > - stderr=3Dsubprocess.STDOUT, > - shell=3DTrue) > + remote_terminate =3D 0 > + > + # target[] contain the remote machine info for ssh connection > + # target[0]: username > + # target[1]: ip address > + # target[2]: pass word > + if target: > + tcpdump_help_pipe =3D subprocess.Popen(["ssh", > + "%s@%s" % (target[0], target[1]), > + "tcpdump -h"], > + stderr=3Dsubprocess.PIPE, > + stdout=3Dsubprocess.PIPE, > + shell=3DFalse) > + tcpdump_help =3D "".join(tuple(tcpdump_help_pipe.communicate())) > + tcpdump_help_pipe.wait() > + else: > + tcpdump_help =3D subprocess.check_output("tcpdump -h; echo 0", > + stderr=3Dsubprocess.STDOUT, > + shell=3DTrue) > + > for line in tcpdump_help.split('\n'): > m =3D re.match(direct_param, line) > if m: > @@ -850,22 +866,29 @@ def sniff_packets(intf, count=3D0, timeout=3D5, fil= ters=3D[]): > else: > cmd =3D sniff_cmd % options >=20 > - args =3D shlex.split(cmd) > + if target: > + pipe =3D subprocess.Popen(['ssh', > + '%s@%s' % (target[0], target[1]), cmd], > + stdin=3Dsubprocess.PIPE, > + shell=3DFalse) > + remote_terminate =3D 1 > + else: > + args =3D shlex.split(cmd) > + pipe =3D subprocess.Popen(args) >=20 > - pipe =3D subprocess.Popen(args) > index =3D str(time.time()) > - SNIFF_PIDS[index] =3D (pipe, intf, timeout) > + SNIFF_PIDS[index] =3D (pipe, intf, timeout, remote_terminate) > time.sleep(1) > return index >=20 >=20 > -def load_sniff_pcap(index=3D''): > +def load_sniff_pcap(index=3D'', target=3D[]): > """ > Stop sniffer and return pcap file > """ > child_exit =3D False > if index in SNIFF_PIDS.keys(): > - pipe, intf, timeout =3D SNIFF_PIDS[index] > + pipe, intf, timeout, remote_terminate =3D SNIFF_PIDS[index] > time_elapse =3D int(time.time() - float(index)) > while time_elapse < timeout: > if pipe.poll() is not None: > @@ -876,6 +899,14 @@ def load_sniff_pcap(index=3D''): > time_elapse +=3D 1 >=20 > if not child_exit: > + if remote_terminate =3D=3D 1: > + stop_tcpdump_pipe =3D subprocess.Popen(['ssh', > + '%s@%s' % (target[0], target[1]), > + 'kill -2 $(pidof tcpdump)'], > + stderr=3Dsubprocess.PIPE, > + shell=3DFalse) > + stop_tcpdump_pipe.wait() > + > pipe.send_signal(signal.SIGINT) > pipe.wait() >=20 > @@ -886,42 +917,6 @@ def load_sniff_pcap(index=3D''): > return "" >=20 >=20 > -def load_sniff_packets(index=3D''): > - """ > - Stop sniffer and return packet objects > - """ > - pkts =3D [] > - child_exit =3D False > - if index in SNIFF_PIDS.keys(): > - pipe, intf, timeout =3D SNIFF_PIDS[index] > - time_elapse =3D int(time.time() - float(index)) > - while time_elapse < timeout: > - if pipe.poll() is not None: > - child_exit =3D True > - break > - > - time.sleep(1) > - time_elapse +=3D 1 > - > - if not child_exit: > - pipe.send_signal(signal.SIGINT) > - pipe.wait() > - > - # wait pcap file ready > - time.sleep(1) > - try: > - cap_pkts =3D rdpcap("/tmp/sniff_%s.pcap" % intf) > - for pkt in cap_pkts: > - # packet gen should be scapy > - packet =3D Packet(tx_port=3Dintf) > - packet.pktgen.assign_pkt(pkt) > - pkts.append(packet) > - except: > - pass > - > - return pkts > - > - > def load_pcapfile(filename=3D""): > pkts =3D [] > try: > @@ -983,7 +978,6 @@ if __name__ =3D=3D "__main__": > inst =3D sniff_packets("lo", timeout=3D5) > pkt =3D Packet(pkt_type=3D'UDP') > pkt.send_pkt(tx_port=3D'lo') > - pkts =3D load_sniff_packets(inst) >=20 > pkt =3D Packet(pkt_type=3D'UDP', pkt_len=3D1500, ran_payload=3DTrue) > pkt.send_pkt(tx_port=3D'lo') > diff --git a/framework/tester.py b/framework/tester.py index a775f68..c5b= 705d > 100755 > --- a/framework/tester.py > +++ b/framework/tester.py > @@ -35,6 +35,7 @@ Interface for bulk traffic generators. >=20 > import re > import subprocess > +import os > from time import sleep > from settings import NICS, load_global_setting, PERF_SETTING from crb i= mport > Crb @@ -560,8 +561,6 @@ class Tester(Crb): > module =3D __import__("packet") > pkt_c =3D getattr(module, "Packet") > send_f =3D getattr(module, "send_packets") > - sniff_f =3D getattr(module, "sniff_packets") > - load_f =3D getattr(module, "load_sniff_packets") > compare_f =3D getattr(module, "compare_pktload") > strip_f =3D getattr(module, "strip_pktload") > save_f =3D getattr(module, "save_packets") @@ -606,7 +605,7 @@ c= lass > Tester(Crb): >=20 > # send and sniff packets > save_f(pkts=3Dpkts, filename=3D"/tmp/%s_tx.pcap" % txIntf) > - inst =3D sniff_f(intf=3DrxIntf, count=3Dpktnum, timeout=3Dti= meout, filters=3D > + inst =3D self.tcpdump_sniff_packets(intf=3DrxIntf, > + count=3Dpktnum, timeout=3Dtimeout, filters=3D > [{'layer': 'network', 'config': {'srcport': '65535'}}, > {'layer': 'network', 'config': {'dstport': '65535'}}]) > rx_inst[rxport] =3D inst > @@ -627,7 +626,7 @@ class Tester(Crb): > # Verify all packets > prev_id =3D -1 > for txport, rxport in portList: > - recv_pkts =3D load_f(rx_inst[rxport]) > + recv_pkts =3D > + self.load_tcpdump_sniff_packets(rx_inst[rxport]) >=20 > # only report when recevied number not matched > if len(tx_pkts[txport]) > len(recv_pkts): > @@ -704,6 +703,47 @@ class Tester(Crb): > self.proc.kill() > self.proc =3D None >=20 > + def tcpdump_sniff_packets(self, intf, count=3D0, timeout=3D5, filter= s=3D[]): > + """ > + Wrapper for packet module sniff_packets > + """ > + # load functions in packet module > + module =3D __import__("packet") > + sniff_f =3D getattr(module, "sniff_packets") > + > + target=3D[] > + target.append(self.get_username()) > + target.append(self.get_ip_address()) > + target.append(self.get_password()) > + return sniff_f(intf, count, timeout, filters, target) > + > + def load_tcpdump_sniff_pcap(self, index=3D''): > + """ > + Wrapper for packet module load_sniff_pcap > + """ > + # load functions in packet module > + module =3D __import__("packet") > + load_pcap_f =3D getattr(module, "load_sniff_pcap") > + > + target=3D[] > + target.append(self.get_username()) > + target.append(self.get_ip_address()) > + target.append(self.get_password()) > + pcap =3D load_pcap_f(index, target) > + self.session.copy_file_from(pcap) > + > + return pcap.split(os.sep)[-1] > + > + def load_tcpdump_sniff_packets(self, index=3D''): > + """ > + Wrapper for packet module load_pcapfile > + """ > + # load functions in packet module > + packet =3D __import__("packet") > + file =3D self.load_tcpdump_sniff_pcap(index) > + > + return packet.load_pcapfile(file) > + > def kill_all(self, killall=3DFalse): > """ > Kill all scapy process or DPDK application on tester. > -- > 2.7.4 >=20