From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by dpdk.org (Postfix) with ESMTP id 9A3271B19 for ; Fri, 21 Sep 2018 05:41:46 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga106.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 20 Sep 2018 20:41:45 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.54,283,1534834800"; d="scan'208";a="264482035" Received: from fmsmsx103.amr.corp.intel.com ([10.18.124.201]) by fmsmga005.fm.intel.com with ESMTP; 20 Sep 2018 20:41:45 -0700 Received: from fmsmsx123.amr.corp.intel.com (10.18.125.38) by FMSMSX103.amr.corp.intel.com (10.18.124.201) with Microsoft SMTP Server (TLS) id 14.3.319.2; Thu, 20 Sep 2018 20:41:44 -0700 Received: from shsmsx152.ccr.corp.intel.com (10.239.6.52) by fmsmsx123.amr.corp.intel.com (10.18.125.38) with Microsoft SMTP Server (TLS) id 14.3.319.2; Thu, 20 Sep 2018 20:41:44 -0700 Received: from shsmsx101.ccr.corp.intel.com ([169.254.1.39]) by SHSMSX152.ccr.corp.intel.com ([169.254.6.37]) with mapi id 14.03.0319.002; Fri, 21 Sep 2018 11:41:42 +0800 From: "Tu, Lijuan" To: "Phil Yang (Arm Technology China)" , "Phil@dpdk.org" , "dts@dpdk.org" CC: nd , "Liu, Yong" Thread-Topic: [dts] [PATCH v5 01/22] framework/packet: support packet sniffer to specify running machine Thread-Index: AQHURCl5x+6dVZMr7UqEB4Fw65lVOKT6JatA//+AZwCAAIegEA== Date: Fri, 21 Sep 2018 03:41:41 +0000 Message-ID: <8CE3E05A3F976642AAB0F4675D0AD20EEC70F7@SHSMSX101.ccr.corp.intel.com> References: <1523526771-7437-1-git-send-email-phil.yang@arm.com> <1536049595-22271-1-git-send-email-phil.yang@arm.com> <8CE3E05A3F976642AAB0F4675D0AD20EEC707E@SHSMSX101.ccr.corp.intel.com> In-Reply-To: Accept-Language: zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-version: 11.0.400.15 dlp-reaction: no-action x-ctpclassification: CTP_NT x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiMDg4NDMzM2UtZGM4My00YWM5LWE0Y2ItOGZhMTcyZmQxMWU1IiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjEwLjE4MDQuNDkiLCJUcnVzdGVkTGFiZWxIYXNoIjoiMzBNRTlIVmNXVHVLV1V2VXQ5WERaZ0s1YU5GeVZMbkQ5dWcwak1aaFdBXC9PT0tYUnBwK1o2bDJ2OEFjdDZIUjgifQ== x-originating-ip: [10.239.127.40] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dts] [PATCH v5 01/22] framework/packet: support packet sniffer to specify running machine X-BeenThere: dts@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: test suite reviews and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Sep 2018 03:41:47 -0000 DTS supports either login server by password or utilize authorized login. For logging by password, we could get some codes like " self.session =3D SS= HPexpect(host, username, password, dut_id) " in SSHConnection module. and w= e have ip, username, password information in crb.cfg I think we don't want to force user to utilize authorized login. > -----Original Message----- > From: Phil Yang (Arm Technology China) [mailto:Phil.Yang@arm.com] > Sent: Friday, September 21, 2018 11:22 AM > To: Tu, Lijuan ; Phil@dpdk.org; dts@dpdk.org > Cc: nd ; Liu, Yong > Subject: RE: [dts] [PATCH v5 01/22] framework/packet: support packet > sniffer to specify running machine >=20 > Hi Lijuan, >=20 > It also need to follow the DTS setup steps in > https://doc.dpdk.org/dts/gsg/sys_reqs.html#authorized-login-session . >=20 > Since you know where are your DUT and Tester, so you can use tool > ssh-copy-id to save local available keys on DUT and Tester. >=20 > Thanks, > Phil Yang >=20 > > -----Original Message----- > > From: Tu, Lijuan > > Sent: Friday, September 21, 2018 11:09 AM > > To: Phil@dpdk.org; dts@dpdk.org > > Cc: nd ; Liu, Yong ; Phil Yang (Arm > > Technology China) > > Subject: RE: [dts] [PATCH v5 01/22] framework/packet: support packet > > sniffer to specify running machine > > > > One comments, when we ssh to a server, it might request a password, > > but we don't handle it. > > > > > > > -----Original Message----- > > > From: dts [mailto:dts-bounces@dpdk.org] On Behalf Of Phil@dpdk.org > > > Sent: Tuesday, September 4, 2018 4:26 PM > > > To: dts@dpdk.org > > > Cc: nd@arm.com; Liu, Yong ; Phil Yang > > > > > > Subject: [dts] [PATCH v5 01/22] framework/packet: support packet > > > sniffer to specify running machine > > > > > > From: Phil Yang > > > > > > By default, the Tester is supposed to be the server which running > > > DTS and the packet sniff methods are running on it. > > > However, if DTS was not running on the Tester and the Tester is > > > another remote server, so packet sniff methods cannot sniff Tester's > > > packets correctly. > > > > > > This patch adds support for packet sniff methods to specify running > > > machine and implements sniff packet methods in class tester. > > > > > > 1. Add parameter to sniff_packet and load_sniff_pcap methods to > > > specify the running machine. > > > 2. Remove load_sniff_packets method in packet.py. > > > 3. Add tcpdump_sniff_packets, load_tcpdump_sniff_pcap and > > > load_tcpdump_sniff_packets for class tester with the new sniff_packet > API. > > > 4. Update tester.check_random_pkts method with > > > tester.tcpdump_sniff_packets and tester.load_tcpdump_sniff_packets > > > to make it execution on the Tester. > > > > > > Signed-off-by: Phil Yang > > > Suggested-by: Marvin Liu > > > --- > > > framework/packet.py | 86 > > > +++++++++++++++++++++++++---------------------------- > > > framework/tester.py | 48 +++++++++++++++++++++++++++--- > > > 2 files changed, 84 insertions(+), 50 deletions(-) > > > > > > diff --git a/framework/packet.py b/framework/packet.py index > > > 976b82b..1f3f07d 100755 > > > --- a/framework/packet.py > > > +++ b/framework/packet.py > > > @@ -812,15 +812,31 @@ def get_filter_cmd(filters=3D[]): > > > return "" > > > > > > > > > -def sniff_packets(intf, count=3D0, timeout=3D5, filters=3D[]): > > > +def sniff_packets(intf, count=3D0, timeout=3D5, filters=3D[], target= =3D[]): > > > """ > > > sniff all packets for certain port in certain seconds > > > """ > > > param =3D "" > > > direct_param =3D r"(\s+)\[ (\S+) in\|out\|inout \]" > > > - tcpdump_help =3D subprocess.check_output("tcpdump -h; echo 0", > > > - > > > stderr=3Dsubprocess.STDOUT, > > > - shell=3DTrue) > > > + remote_terminate =3D 0 > > > + > > > + # target[] contain the remote machine info for ssh connection > > > + # target[0]: username > > > + # target[1]: ip address > > > + # target[2]: pass word > > > + if target: > > > + tcpdump_help_pipe =3D subprocess.Popen(["ssh", > > > + "%s@%s" % (target[0], target[1]), > > > + "tcpdump -h"], > > > + stderr=3Dsubprocess.PIPE, > > > + stdout=3Dsubprocess.PIPE, > > > + shell=3DFalse) > > [Lijuan] Action will be blocked if SSH need password. > > > + tcpdump_help =3D > > > "".join(tuple(tcpdump_help_pipe.communicate())) > > > + tcpdump_help_pipe.wait() > > > + else: > > > + tcpdump_help =3D subprocess.check_output("tcpdump -h; echo > 0", > > > + > stderr=3Dsubprocess.STDOUT, > > > + shell=3DTrue) > > > + > > > for line in tcpdump_help.split('\n'): > > > m =3D re.match(direct_param, line) > > > if m: > > > @@ -850,22 +866,29 @@ def sniff_packets(intf, count=3D0, timeout=3D5, > > > filters=3D[]): > > > else: > > > cmd =3D sniff_cmd % options > > > > > > - args =3D shlex.split(cmd) > > > + if target: > > > + pipe =3D subprocess.Popen(['ssh', > > > + '%s@%s' % (target[0], target[1]), cmd], > > > + stdin=3Dsubprocess.PIPE, > > > + shell=3DFalse) > > > + remote_terminate =3D 1 > > > + else: > > > + args =3D shlex.split(cmd) > > > + pipe =3D subprocess.Popen(args) > > > > > > - pipe =3D subprocess.Popen(args) > > > index =3D str(time.time()) > > > - SNIFF_PIDS[index] =3D (pipe, intf, timeout) > > > + SNIFF_PIDS[index] =3D (pipe, intf, timeout, remote_terminate) > > > time.sleep(1) > > > return index > > > > > > > > > -def load_sniff_pcap(index=3D''): > > > +def load_sniff_pcap(index=3D'', target=3D[]): > > > """ > > > Stop sniffer and return pcap file > > > """ > > > child_exit =3D False > > > if index in SNIFF_PIDS.keys(): > > > - pipe, intf, timeout =3D SNIFF_PIDS[index] > > > + pipe, intf, timeout, remote_terminate =3D SNIFF_PIDS[index] > > > time_elapse =3D int(time.time() - float(index)) > > > while time_elapse < timeout: > > > if pipe.poll() is not None: > > > @@ -876,6 +899,14 @@ def load_sniff_pcap(index=3D''): > > > time_elapse +=3D 1 > > > > > > if not child_exit: > > > + if remote_terminate =3D=3D 1: > > > + stop_tcpdump_pipe =3D subprocess.Popen(['ssh', > > > + '%s@%s' % (target[0], > > > target[1]), > > > + 'kill -2 $(pidof tcpdump)'], > > > + stderr=3Dsubprocess.PIPE, > > > + shell=3DFalse) > > > + stop_tcpdump_pipe.wait() > > > + > > > pipe.send_signal(signal.SIGINT) > > > pipe.wait() > > > > > > @@ -886,42 +917,6 @@ def load_sniff_pcap(index=3D''): > > > return "" > > > > > > > > > -def load_sniff_packets(index=3D''): > > > - """ > > > - Stop sniffer and return packet objects > > > - """ > > > - pkts =3D [] > > > - child_exit =3D False > > > - if index in SNIFF_PIDS.keys(): > > > - pipe, intf, timeout =3D SNIFF_PIDS[index] > > > - time_elapse =3D int(time.time() - float(index)) > > > - while time_elapse < timeout: > > > - if pipe.poll() is not None: > > > - child_exit =3D True > > > - break > > > - > > > - time.sleep(1) > > > - time_elapse +=3D 1 > > > - > > > - if not child_exit: > > > - pipe.send_signal(signal.SIGINT) > > > - pipe.wait() > > > - > > > - # wait pcap file ready > > > - time.sleep(1) > > > - try: > > > - cap_pkts =3D rdpcap("/tmp/sniff_%s.pcap" % intf) > > > - for pkt in cap_pkts: > > > - # packet gen should be scapy > > > - packet =3D Packet(tx_port=3Dintf) > > > - packet.pktgen.assign_pkt(pkt) > > > - pkts.append(packet) > > > - except: > > > - pass > > > - > > > - return pkts > > > - > > > - > > > def load_pcapfile(filename=3D""): > > > pkts =3D [] > > > try: > > > @@ -983,7 +978,6 @@ if __name__ =3D=3D "__main__": > > > inst =3D sniff_packets("lo", timeout=3D5) > > > pkt =3D Packet(pkt_type=3D'UDP') > > > pkt.send_pkt(tx_port=3D'lo') > > > - pkts =3D load_sniff_packets(inst) > > > > > > pkt =3D Packet(pkt_type=3D'UDP', pkt_len=3D1500, ran_payload=3DT= rue) > > > pkt.send_pkt(tx_port=3D'lo') > > > diff --git a/framework/tester.py b/framework/tester.py index > > > a775f68..c5b705d 100755 > > > --- a/framework/tester.py > > > +++ b/framework/tester.py > > > @@ -35,6 +35,7 @@ Interface for bulk traffic generators. > > > > > > import re > > > import subprocess > > > +import os > > > from time import sleep > > > from settings import NICS, load_global_setting, PERF_SETTING from > > > crb import Crb @@ -560,8 +561,6 @@ class Tester(Crb): > > > module =3D __import__("packet") > > > pkt_c =3D getattr(module, "Packet") > > > send_f =3D getattr(module, "send_packets") > > > - sniff_f =3D getattr(module, "sniff_packets") > > > - load_f =3D getattr(module, "load_sniff_packets") > > > compare_f =3D getattr(module, "compare_pktload") > > > strip_f =3D getattr(module, "strip_pktload") > > > save_f =3D getattr(module, "save_packets") @@ -606,7 +605,7 > > > @@ class Tester(Crb): > > > > > > # send and sniff packets > > > save_f(pkts=3Dpkts, filename=3D"/tmp/%s_tx.pcap" % txInt= f) > > > - inst =3D sniff_f(intf=3DrxIntf, count=3Dpktnum, > timeout=3Dtimeout, > > > filters=3D > > > + inst =3D self.tcpdump_sniff_packets(intf=3DrxIntf, > > > + count=3Dpktnum, timeout=3Dtimeout, filters=3D > > > [{'layer': 'network', 'config': {'srcport': '65535'}= }, > > > {'layer': 'network', 'config': {'dstport': '65535'}= }]) > > > rx_inst[rxport] =3D inst > > > @@ -627,7 +626,7 @@ class Tester(Crb): > > > # Verify all packets > > > prev_id =3D -1 > > > for txport, rxport in portList: > > > - recv_pkts =3D load_f(rx_inst[rxport]) > > > + recv_pkts =3D > > > + self.load_tcpdump_sniff_packets(rx_inst[rxport]) > > > > > > # only report when recevied number not matched > > > if len(tx_pkts[txport]) > len(recv_pkts): > > > @@ -704,6 +703,47 @@ class Tester(Crb): > > > self.proc.kill() > > > self.proc =3D None > > > > > > + def tcpdump_sniff_packets(self, intf, count=3D0, timeout=3D5, > filters=3D[]): > > > + """ > > > + Wrapper for packet module sniff_packets > > > + """ > > > + # load functions in packet module > > > + module =3D __import__("packet") > > > + sniff_f =3D getattr(module, "sniff_packets") > > > + > > > + target=3D[] > > > + target.append(self.get_username()) > > > + target.append(self.get_ip_address()) > > > + target.append(self.get_password()) > > > + return sniff_f(intf, count, timeout, filters, target) > > > + > > > + def load_tcpdump_sniff_pcap(self, index=3D''): > > > + """ > > > + Wrapper for packet module load_sniff_pcap > > > + """ > > > + # load functions in packet module > > > + module =3D __import__("packet") > > > + load_pcap_f =3D getattr(module, "load_sniff_pcap") > > > + > > > + target=3D[] > > > + target.append(self.get_username()) > > > + target.append(self.get_ip_address()) > > > + target.append(self.get_password()) > > > + pcap =3D load_pcap_f(index, target) > > > + self.session.copy_file_from(pcap) > > > + > > > + return pcap.split(os.sep)[-1] > > > + > > > + def load_tcpdump_sniff_packets(self, index=3D''): > > > + """ > > > + Wrapper for packet module load_pcapfile > > > + """ > > > + # load functions in packet module > > > + packet =3D __import__("packet") > > > + file =3D self.load_tcpdump_sniff_pcap(index) > > > + > > > + return packet.load_pcapfile(file) > > > + > > > def kill_all(self, killall=3DFalse): > > > """ > > > Kill all scapy process or DPDK application on tester. > > > -- > > > 2.7.4