From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR03-DB5-obe.outbound.protection.outlook.com (mail-eopbgr40057.outbound.protection.outlook.com [40.107.4.57]) by dpdk.org (Postfix) with ESMTP id 899221B549 for ; Fri, 19 Oct 2018 11:29:28 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector1-arm-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=DsjQG8n4nL3QUKHq4DPA3KtmgGm9mHLiOH8X0h1SDNg=; b=bhf9PejoBOPlvqJf4bUN1PZrjLAzkMhpyPyNEsUggl8dY8QI0m7pNCZnAoN4euKv8xiyXCFW8S5jp19SkrL8FzTRrpNEUCA2NfPGw5BV6iTiipFE/muOJqF9pQv+HwH/85hGyfhShDIzwdppJhfErDT4Zl1WCnnWqBXR7VrZ52g= Received: from DB7PR08MB3385.eurprd08.prod.outlook.com (20.176.238.90) by DB7PR08MB3498.eurprd08.prod.outlook.com (20.177.120.32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1228.25; Fri, 19 Oct 2018 09:29:25 +0000 Received: from DB7PR08MB3385.eurprd08.prod.outlook.com ([fe80::eda0:7bf6:a891:3ae7]) by DB7PR08MB3385.eurprd08.prod.outlook.com ([fe80::eda0:7bf6:a891:3ae7%6]) with mapi id 15.20.1250.020; Fri, 19 Oct 2018 09:29:25 +0000 From: "Phil Yang (Arm Technology China)" To: "Phil Yang (Arm Technology China)" , "Tu, Lijuan" , "dts@dpdk.org" CC: nd Thread-Topic: [PATCH v6 01/22] framework/packet: support packet sniffer to specify running machine Thread-Index: AQHUZsMU8GWetv7zyU2kz/v4zZl2kaUkvMMQgAGRH/A= Date: Fri, 19 Oct 2018 09:29:24 +0000 Message-ID: References: <1536049595-22271-1-git-send-email-phil.yang@arm.com> <1539670802-15603-1-git-send-email-phil.yang@arm.com> <8CE3E05A3F976642AAB0F4675D0AD20E0B97B7E5@SHSMSX101.ccr.corp.intel.com> In-Reply-To: Accept-Language: zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=Phil.Yang@arm.com; x-originating-ip: [113.29.88.7] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; DB7PR08MB3498; 6:Jwav+5PluiIG4gFszJhetEBhYz75FS55LNr54DVyLr/ZfjCl34cPeBu6mau4hvFfBSaV3wVRbpgq66o6f6jzij9bbpNcGjADQb+hIoNgkMuKsrS1yVl6RIbT6M7LN2QHdzT2MRmIAR0K4+Wq3DSpFoxeUxE1B4PX23jNdDJAxY0FPxLKB2WJpPZOHAbf8ZKagK8yXaJJ5Ccjeb+F3D7PBf4gR1W3X8XQ9KumobKML1zerw3u2W1eqRm41dYj7KJ615UjjvppL+KiAdt18ijjtLR52XlxMCYVfZEPIZDfrhl+DoYsMXdtVD3rtnF6SerlbTgAaMWAL2EDU8b+1yfuIvK2F46vEnFJNdIlduAazMZC4lga7uew6xyX5l1onzCTSxKV0s4JTzOX2XGfSbhgkDmhc8LbeLGZ13gviosF5RnOnkPPPmRPi3HsY3+JYJtT6/czIMcAi8ez8QTsXtIGjA==; 5:ZfdPoiLDn3Lgq5tYRaGyY4u4RAcb8u/8D0KPcjRsySgps8TEiNgn2IZ2xVE4Ck43rDq0SE1Tsq3vO9S649rno1dbfG875RMifZRVGuwrCp1cSwthMuB9uDrO0ZYWm4R0+Eh7A+QRlrM+c7KjMFeep98tcRY2EI0s4/uQ6ZIJS9I=; 7:E6lP5ZQZIEwYyfl1NIIWZO6MMf/RSHYY9MUxj4C2Y2RnE/fUEIlNjbegyGuj8DcxSO2WYyhWAVa4D+tImiSq+wSROEZja36gtzYxCry8WiTKwfAHMIpkVE9JOfN8bGhwnuUhodMwX1jRlymrWUVUT6/Fu/wDIjmSGDnAJU6P9AZ8/WC1wM72f9FerIWeR3vso/MRPvl5CmAiw8QdAPXaDMdErjbhwxHwLgko//MRCvn0DBUmIKKtsASkulSBpQww x-ms-exchange-antispam-srfa-diagnostics: SOS;SOR; x-ms-office365-filtering-correlation-id: 68c6a54b-c341-4b10-6c56-08d635a55c1f x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(7020095)(4652040)(8989299)(5600074)(711020)(4618075)(4534185)(7168020)(4627221)(201703031133081)(201702281549075)(8990200)(2017052603328)(7153060)(7193020); SRVR:DB7PR08MB3498; x-ms-traffictypediagnostic: DB7PR08MB3498: nodisclaimer: True x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(180628864354917)(158342451672863)(228905959029699); x-ms-exchange-senderadcheck: 1 x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(5005006)(8121501046)(93006095)(93001095)(3002001)(10201501046)(3231355)(944501410)(52105095)(6055026)(149066)(150057)(6041310)(20161123560045)(20161123562045)(20161123564045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123558120)(201708071742011)(7699051)(76991095); SRVR:DB7PR08MB3498; BCL:0; PCL:0; RULEID:; SRVR:DB7PR08MB3498; x-forefront-prvs: 0830866D19 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(396003)(39860400002)(376002)(366004)(346002)(136003)(199004)(189003)(13464003)(486006)(446003)(476003)(66066001)(72206003)(11346002)(86362001)(9686003)(6506007)(102836004)(53546011)(55236004)(4744004)(256004)(14444005)(97736004)(2501003)(305945005)(478600001)(5250100002)(106356001)(53936002)(7736002)(14454004)(110136005)(55016002)(105586002)(71200400001)(8936002)(25786009)(71190400001)(4326008)(316002)(33656002)(8676002)(93886005)(6246003)(2906002)(2900100001)(21490400003)(81166006)(81156014)(5660300001)(6436002)(186003)(26005)(99286004)(74316002)(551544002)(76176011)(7696005)(229853002)(3846002)(6116002)(68736007); DIR:OUT; SFP:1101; SCL:1; SRVR:DB7PR08MB3498; H:DB7PR08MB3385.eurprd08.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: y1EJMA+/AnH2Z94DbzsjltBlBOMOcZIxrE5p+1XUwJHaxsuG6G4w9jcaCVEMon7DKkX/E2XxTl7eWtJb9Aug7R4vBA/3zJwfs1iM7xz4QWENsN523XUpcFN5rAKp9u+dOHLJE/qUBFR0IZTDdKoTSRZD6wTtqgaJH8QpVIcgeZ6eTHus8GB+gBvomAoNyOpvjV0D3BlBt15bKaXR4bvB5ArQ7601ZMtMyVaJ5cO4GPlasHa3Ersf4Qc6ektMeO4SCo8sZZ1kWiI9RFeKZGDRrPkTTzNXS+MrLQma0wssVdb7rFP9CPz/0StSYTqo4A4Q8J9YNBU1IQKQMgfwm0hzMLYApjo4RBu4cPV83UV6Jho= spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-Network-Message-Id: 68c6a54b-c341-4b10-6c56-08d635a55c1f X-MS-Exchange-CrossTenant-originalarrivaltime: 19 Oct 2018 09:29:24.9592 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB7PR08MB3498 Subject: Re: [dts] [PATCH v6 01/22] framework/packet: support packet sniffer to specify running machine X-BeenThere: dts@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: test suite reviews and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Oct 2018 09:29:28 -0000 Hi Lijuan, I have sent out the 7th version of this single patch. Other patches in this= series kept on the 6th version. Please review it. Thanks, Phil Yang > -----Original Message----- > From: dts On Behalf Of Phil Yang (Arm Technology > China) > Sent: Thursday, October 18, 2018 6:00 PM > To: Tu, Lijuan ; dts@dpdk.org > Cc: nd > Subject: Re: [dts] [PATCH v6 01/22] framework/packet: support packet snif= fer to > specify running machine >=20 > Hi Lijuan, >=20 > Agree. >=20 > I can sanity the patch in the new version. >=20 > Thanks, > Phil Yang >=20 > > -----Original Message----- > > From: Tu, Lijuan > > Sent: Thursday, October 18, 2018 5:15 PM > > To: Phil Yang (Arm Technology China) ; dts@dpdk.org > > Cc: nd > > Subject: RE: [PATCH v6 01/22] framework/packet: support packet sniffer > > to specify running machine > > > > Hi Phil, > > Poll is an attribute of Popen, but not pxssh. > > > > To make sniff function more clear, I prefer to remove the old code > > instead of keeping them. > > As in dts, we always pass the parameter target, the tester > > information, we could raise exception when finding target is None. > > > > > -----Original Message----- > > > From: phil.yang@arm.com [mailto:phil.yang@arm.com] > > > Sent: Tuesday, October 16, 2018 2:20 PM > > > To: dts@dpdk.org > > > Cc: nd@arm.com; Tu, Lijuan > > > Subject: [PATCH v6 01/22] framework/packet: support packet sniffer > > > to specify running machine > > > > > > By default, the Tester is supposed to be the server which running > > > DTS and the packet sniff methods are running on it. > > > However, if DTS was not running on the Tester and the Tester is > > > another remote server, so packet sniff methods cannot sniff Tester's > > > packets correctly. > > > > > > This patch adds support for packet sniff methods to specify running > > > machine and implements sniff packet methods in class tester. > > > > > > 1. Add parameter to sniff_packet and load_sniff_pcap methods to > > > specify the running machine. > > > 2. Remove load_sniff_packets method in packet.py. > > > 3. Add tcpdump_sniff_packets, load_tcpdump_sniff_pcap and > > > load_tcpdump_sniff_packets for class tester with the new sniff_packet= API. > > > 4. Update tester.check_random_pkts method with > > > tester.tcpdump_sniff_packets and tester.load_tcpdump_sniff_packets > > > to make it execution on the Tester. > > > > > > Signed-off-by: Phil Yang > > > Suggested-by: Marvin Liu > > > Suggested-by: Tu Lijuan > > > --- > > > framework/packet.py | 100 > > > +++++++++++++++++++++++++++------------------------- > > > framework/tester.py | 48 ++++++++++++++++++++++--- > > > 2 files changed, 95 insertions(+), 53 deletions(-) > > > > > > diff --git a/framework/packet.py b/framework/packet.py index > > > 976b82b..517c93d 100755 > > > --- a/framework/packet.py > > > +++ b/framework/packet.py > > > @@ -33,7 +33,6 @@ > > > Generic packet create, transmit and analyze module Base on > > > scapy(python program for packet manipulation) """ > > > - > > > import os > > > import time > > > import sys > > > @@ -61,6 +60,7 @@ from scapy.route import * from scapy.packet > > > import bind_layers, Raw from scapy.sendrecv import sendp from > > > scapy.arch import get_if_hwaddr > > > +from pexpect import pxssh > > > > > > # load extension layers > > > exec_file =3D os.path.realpath(__file__) @@ -812,15 +812,31 @@ def > > > get_filter_cmd(filters=3D[]): > > > return "" > > > > > > > > > -def sniff_packets(intf, count=3D0, timeout=3D5, filters=3D[]): > > > +def sniff_packets(intf, count=3D0, timeout=3D5, filters=3D[], target= =3D[]): > > > """ > > > sniff all packets for certain port in certain seconds > > > """ > > > param =3D "" > > > direct_param =3D r"(\s+)\[ (\S+) in\|out\|inout \]" > > > - tcpdump_help =3D subprocess.check_output("tcpdump -h; echo 0", > > > - > > > stderr=3Dsubprocess.STDOUT, > > > - shell=3DTrue) > > > + remote_terminate =3D 0 # session remote terminate flag > > > + > > > + # target[] contain the remote machine info for ssh connection > > > + # target[0]: username > > > + # target[1]: ip address > > > + # target[2]: pass word > > > + if target: > > > + tcpdump_help_session=3Dpxssh.pxssh() > > > + tcpdump_help_session.login(server=3Dtarget[1], username=3Dtarget[0]= , > > > + password=3Dtarget[2], > > > original_prompt=3D'[$#>]', > > > + login_timeout=3D20) > > > + tcpdump_help_session.sendline('tcpdump -h') > > > + tcpdump_help_session.prompt() > > > + tcpdump_help=3Dtcpdump_help_session.before > > > + tcpdump_help_session.logout() > > > + else: > > > + tcpdump_help =3D subprocess.check_output("tcpdump -h; echo 0= ", > > > + stderr=3Dsubprocess.STDOUT, > > > +shell=3DTrue) > > > + > > > for line in tcpdump_help.split('\n'): > > > m =3D re.match(direct_param, line) > > > if m: > > > @@ -850,22 +866,31 @@ def sniff_packets(intf, count=3D0, timeout=3D5, > > > filters=3D[]): > > > else: > > > cmd =3D sniff_cmd % options > > > > > > - args =3D shlex.split(cmd) > > > + if target: > > > + tcpdump_session=3Dpxssh.pxssh() > > > + tcpdump_session.login(server=3Dtarget[1], username=3Dtarget[= 0], > > > + password=3Dtarget[2], > > > original_prompt=3D'[$#>]', > > > + login_timeout=3D20) > > > + tcpdump_session.sendline(cmd) > > > + pipe =3D tcpdump_session > > > + remote_terminate =3D 1 > > > + else: > > > + args =3D shlex.split(cmd) > > > + pipe =3D subprocess.Popen(args) > > > > > > - pipe =3D subprocess.Popen(args) > > > index =3D str(time.time()) > > > - SNIFF_PIDS[index] =3D (pipe, intf, timeout) > > > + SNIFF_PIDS[index] =3D (pipe, intf, timeout, remote_terminate) > > > time.sleep(1) > > > return index > > > > > > > > > -def load_sniff_pcap(index=3D''): > > > +def load_sniff_pcap(index=3D'', target=3D[]): > > > """ > > > Stop sniffer and return pcap file > > > """ > > > child_exit =3D False > > > if index in SNIFF_PIDS.keys(): > > > - pipe, intf, timeout =3D SNIFF_PIDS[index] > > > + pipe, intf, timeout, remote_terminate =3D SNIFF_PIDS[index] > > > time_elapse =3D int(time.time() - float(index)) > > > while time_elapse < timeout: > > > if pipe.poll() is not None: > > [Lijuan] Exception happened here because poll is not an attribute of px= ssh. > > > @@ -876,8 +901,22 @@ def load_sniff_pcap(index=3D''): > > > time_elapse +=3D 1 > > > > > > if not child_exit: > > > - pipe.send_signal(signal.SIGINT) > > > - pipe.wait() > > > + if remote_terminate =3D=3D 1: > > > + tcpdump_quit_session=3Dpxssh.pxssh() > > > + tcpdump_quit_session.login(server=3Dtarget[1], > > > username=3Dtarget[0], > > > + password=3Dtarget[2], > > > original_prompt=3D'[$#>]', > > > + login_timeout=3D20) > > > + tcpdump_quit_session.sendline('kill -2 $(pidof > > > tcpdump)') > > > + tcpdump_quit_session.prompt() > > > + tcpdump_quit_session.logout() > > > + # Teminate the tcpdump_session > > > + pipe.prompt() > > > + pipe.logout() > > > + pipe =3D None > > > + > > > + if pipe is not None: > > > + pipe.send_signal(signal.SIGINT) > > > + pipe.wait() > > > > > > # wait pcap file ready > > > time.sleep(1) > > > @@ -886,42 +925,6 @@ def load_sniff_pcap(index=3D''): > > > return "" > > > > > > > > > -def load_sniff_packets(index=3D''): > > > - """ > > > - Stop sniffer and return packet objects > > > - """ > > > - pkts =3D [] > > > - child_exit =3D False > > > - if index in SNIFF_PIDS.keys(): > > > - pipe, intf, timeout =3D SNIFF_PIDS[index] > > > - time_elapse =3D int(time.time() - float(index)) > > > - while time_elapse < timeout: > > > - if pipe.poll() is not None: > > > - child_exit =3D True > > > - break > > > - > > > - time.sleep(1) > > > - time_elapse +=3D 1 > > > - > > > - if not child_exit: > > > - pipe.send_signal(signal.SIGINT) > > > - pipe.wait() > > > - > > > - # wait pcap file ready > > > - time.sleep(1) > > > - try: > > > - cap_pkts =3D rdpcap("/tmp/sniff_%s.pcap" % intf) > > > - for pkt in cap_pkts: > > > - # packet gen should be scapy > > > - packet =3D Packet(tx_port=3Dintf) > > > - packet.pktgen.assign_pkt(pkt) > > > - pkts.append(packet) > > > - except: > > > - pass > > > - > > > - return pkts > > > - > > > - > > > def load_pcapfile(filename=3D""): > > > pkts =3D [] > > > try: > > > @@ -983,7 +986,6 @@ if __name__ =3D=3D "__main__": > > > inst =3D sniff_packets("lo", timeout=3D5) > > > pkt =3D Packet(pkt_type=3D'UDP') > > > pkt.send_pkt(tx_port=3D'lo') > > > - pkts =3D load_sniff_packets(inst) > > > > > > pkt =3D Packet(pkt_type=3D'UDP', pkt_len=3D1500, ran_payload=3DT= rue) > > > pkt.send_pkt(tx_port=3D'lo') > > > diff --git a/framework/tester.py b/framework/tester.py index > > > a775f68..c5b705d 100755 > > > --- a/framework/tester.py > > > +++ b/framework/tester.py > > > @@ -35,6 +35,7 @@ Interface for bulk traffic generators. > > > > > > import re > > > import subprocess > > > +import os > > > from time import sleep > > > from settings import NICS, load_global_setting, PERF_SETTING from > > > crb import Crb @@ -560,8 +561,6 @@ class Tester(Crb): > > > module =3D __import__("packet") > > > pkt_c =3D getattr(module, "Packet") > > > send_f =3D getattr(module, "send_packets") > > > - sniff_f =3D getattr(module, "sniff_packets") > > > - load_f =3D getattr(module, "load_sniff_packets") > > > compare_f =3D getattr(module, "compare_pktload") > > > strip_f =3D getattr(module, "strip_pktload") > > > save_f =3D getattr(module, "save_packets") @@ -606,7 +605,7 > > > @@ class Tester(Crb): > > > > > > # send and sniff packets > > > save_f(pkts=3Dpkts, filename=3D"/tmp/%s_tx.pcap" % txInt= f) > > > - inst =3D sniff_f(intf=3DrxIntf, count=3Dpktnum, timeout= =3Dtimeout, > > > filters=3D > > > + inst =3D self.tcpdump_sniff_packets(intf=3DrxIntf, > > > + count=3Dpktnum, timeout=3Dtimeout, filters=3D > > > [{'layer': 'network', 'config': {'srcport': '65535'}= }, > > > {'layer': 'network', 'config': {'dstport': '65535'}= }]) > > > rx_inst[rxport] =3D inst > > > @@ -627,7 +626,7 @@ class Tester(Crb): > > > # Verify all packets > > > prev_id =3D -1 > > > for txport, rxport in portList: > > > - recv_pkts =3D load_f(rx_inst[rxport]) > > > + recv_pkts =3D > > > + self.load_tcpdump_sniff_packets(rx_inst[rxport]) > > > > > > # only report when recevied number not matched > > > if len(tx_pkts[txport]) > len(recv_pkts): > > > @@ -704,6 +703,47 @@ class Tester(Crb): > > > self.proc.kill() > > > self.proc =3D None > > > > > > + def tcpdump_sniff_packets(self, intf, count=3D0, timeout=3D5, fi= lters=3D[]): > > > + """ > > > + Wrapper for packet module sniff_packets > > > + """ > > > + # load functions in packet module > > > + module =3D __import__("packet") > > > + sniff_f =3D getattr(module, "sniff_packets") > > > + > > > + target=3D[] > > > + target.append(self.get_username()) > > > + target.append(self.get_ip_address()) > > > + target.append(self.get_password()) > > > + return sniff_f(intf, count, timeout, filters, target) > > > + > > > + def load_tcpdump_sniff_pcap(self, index=3D''): > > > + """ > > > + Wrapper for packet module load_sniff_pcap > > > + """ > > > + # load functions in packet module > > > + module =3D __import__("packet") > > > + load_pcap_f =3D getattr(module, "load_sniff_pcap") > > > + > > > + target=3D[] > > > + target.append(self.get_username()) > > > + target.append(self.get_ip_address()) > > > + target.append(self.get_password()) > > > + pcap =3D load_pcap_f(index, target) > > > + self.session.copy_file_from(pcap) > > > + > > > + return pcap.split(os.sep)[-1] > > > + > > > + def load_tcpdump_sniff_packets(self, index=3D''): > > > + """ > > > + Wrapper for packet module load_pcapfile > > > + """ > > > + # load functions in packet module > > > + packet =3D __import__("packet") > > > + file =3D self.load_tcpdump_sniff_pcap(index) > > > + > > > + return packet.load_pcapfile(file) > > > + > > > def kill_all(self, killall=3DFalse): > > > """ > > > Kill all scapy process or DPDK application on tester. > > > -- > > > 2.7.4