From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by dpdk.org (Postfix) with ESMTP id 4C8C54F90 for ; Thu, 18 Oct 2018 11:15:11 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga107.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 18 Oct 2018 02:15:10 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.54,395,1534834800"; d="scan'208";a="272335041" Received: from fmsmsx105.amr.corp.intel.com ([10.18.124.203]) by fmsmga005.fm.intel.com with ESMTP; 18 Oct 2018 02:15:10 -0700 Received: from fmsmsx119.amr.corp.intel.com (10.18.124.207) by FMSMSX105.amr.corp.intel.com (10.18.124.203) with Microsoft SMTP Server (TLS) id 14.3.319.2; Thu, 18 Oct 2018 02:15:09 -0700 Received: from shsmsx103.ccr.corp.intel.com (10.239.4.69) by FMSMSX119.amr.corp.intel.com (10.18.124.207) with Microsoft SMTP Server (TLS) id 14.3.319.2; Thu, 18 Oct 2018 02:15:09 -0700 Received: from shsmsx101.ccr.corp.intel.com ([169.254.1.60]) by SHSMSX103.ccr.corp.intel.com ([169.254.4.245]) with mapi id 14.03.0319.002; Thu, 18 Oct 2018 17:15:07 +0800 From: "Tu, Lijuan" To: "phil.yang@arm.com" , "dts@dpdk.org" CC: "nd@arm.com" Thread-Topic: [PATCH v6 01/22] framework/packet: support packet sniffer to specify running machine Thread-Index: AQHUZTVtoj1k6AQS4E+Dh5dmc5FxQaUkjncg Date: Thu, 18 Oct 2018 09:15:06 +0000 Message-ID: <8CE3E05A3F976642AAB0F4675D0AD20E0B97B7E5@SHSMSX101.ccr.corp.intel.com> References: <1536049595-22271-1-git-send-email-phil.yang@arm.com> <1539670802-15603-1-git-send-email-phil.yang@arm.com> In-Reply-To: <1539670802-15603-1-git-send-email-phil.yang@arm.com> Accept-Language: zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-version: 11.0.400.15 dlp-reaction: no-action x-ctpclassification: CTP_NT x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiNDAzZDEwZTgtNGQ1NC00MzU1LTg1ODktMzhiNWU4NjdkODk3IiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjEwLjE4MDQuNDkiLCJUcnVzdGVkTGFiZWxIYXNoIjoiK0Y2N1V3YTQyWDlrVWNqRThVVGlsOU5RUVwvK3pzZDNQRTZxTDE3QVNtd2hmMmwxWk1FeXdrZ2FXVTl4aWxnTnoifQ== x-originating-ip: [10.239.127.40] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dts] [PATCH v6 01/22] framework/packet: support packet sniffer to specify running machine X-BeenThere: dts@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: test suite reviews and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Oct 2018 09:15:12 -0000 Hi Phil, Poll is an attribute of Popen, but not pxssh. To make sniff function more clear, I prefer to remove the old code instead = of keeping them. As in dts, we always pass the parameter target, the tester information, we = could raise exception when finding target is None. > -----Original Message----- > From: phil.yang@arm.com [mailto:phil.yang@arm.com] > Sent: Tuesday, October 16, 2018 2:20 PM > To: dts@dpdk.org > Cc: nd@arm.com; Tu, Lijuan > Subject: [PATCH v6 01/22] framework/packet: support packet sniffer to > specify running machine >=20 > By default, the Tester is supposed to be the server which running DTS and= the > packet sniff methods are running on it. > However, if DTS was not running on the Tester and the Tester is another > remote server, so packet sniff methods cannot sniff Tester's packets > correctly. >=20 > This patch adds support for packet sniff methods to specify running machi= ne > and implements sniff packet methods in class tester. >=20 > 1. Add parameter to sniff_packet and load_sniff_pcap methods to specify > the running machine. > 2. Remove load_sniff_packets method in packet.py. > 3. Add tcpdump_sniff_packets, load_tcpdump_sniff_pcap and > load_tcpdump_sniff_packets for class tester with the new sniff_packet API= . > 4. Update tester.check_random_pkts method with > tester.tcpdump_sniff_packets and tester.load_tcpdump_sniff_packets to > make it execution on the Tester. >=20 > Signed-off-by: Phil Yang > Suggested-by: Marvin Liu > Suggested-by: Tu Lijuan > --- > framework/packet.py | 100 > +++++++++++++++++++++++++++------------------------- > framework/tester.py | 48 ++++++++++++++++++++++--- > 2 files changed, 95 insertions(+), 53 deletions(-) >=20 > diff --git a/framework/packet.py b/framework/packet.py index > 976b82b..517c93d 100755 > --- a/framework/packet.py > +++ b/framework/packet.py > @@ -33,7 +33,6 @@ > Generic packet create, transmit and analyze module Base on > scapy(python program for packet manipulation) """ > - > import os > import time > import sys > @@ -61,6 +60,7 @@ from scapy.route import * from scapy.packet import > bind_layers, Raw from scapy.sendrecv import sendp from scapy.arch > import get_if_hwaddr > +from pexpect import pxssh >=20 > # load extension layers > exec_file =3D os.path.realpath(__file__) > @@ -812,15 +812,31 @@ def get_filter_cmd(filters=3D[]): > return "" >=20 >=20 > -def sniff_packets(intf, count=3D0, timeout=3D5, filters=3D[]): > +def sniff_packets(intf, count=3D0, timeout=3D5, filters=3D[], target=3D[= ]): > """ > sniff all packets for certain port in certain seconds > """ > param =3D "" > direct_param =3D r"(\s+)\[ (\S+) in\|out\|inout \]" > - tcpdump_help =3D subprocess.check_output("tcpdump -h; echo 0", > - > stderr=3Dsubprocess.STDOUT, > - shell=3DTrue) > + remote_terminate =3D 0 # session remote terminate flag > + > + # target[] contain the remote machine info for ssh connection > + # target[0]: username > + # target[1]: ip address > + # target[2]: pass word > + if target: > + tcpdump_help_session=3Dpxssh.pxssh() > + tcpdump_help_session.login(server=3Dtarget[1], username=3Dtarget[0], > + password=3Dtarget[2], > original_prompt=3D'[$#>]', > + login_timeout=3D20) > + tcpdump_help_session.sendline('tcpdump -h') > + tcpdump_help_session.prompt() > + tcpdump_help=3Dtcpdump_help_session.before > + tcpdump_help_session.logout() > + else: > + tcpdump_help =3D subprocess.check_output("tcpdump -h; echo 0", > + stderr=3Dsubprocess.STDOUT, > +shell=3DTrue) > + > for line in tcpdump_help.split('\n'): > m =3D re.match(direct_param, line) > if m: > @@ -850,22 +866,31 @@ def sniff_packets(intf, count=3D0, timeout=3D5, > filters=3D[]): > else: > cmd =3D sniff_cmd % options >=20 > - args =3D shlex.split(cmd) > + if target: > + tcpdump_session=3Dpxssh.pxssh() > + tcpdump_session.login(server=3Dtarget[1], username=3Dtarget[0], > + password=3Dtarget[2], > original_prompt=3D'[$#>]', > + login_timeout=3D20) > + tcpdump_session.sendline(cmd) > + pipe =3D tcpdump_session > + remote_terminate =3D 1 > + else: > + args =3D shlex.split(cmd) > + pipe =3D subprocess.Popen(args) >=20 > - pipe =3D subprocess.Popen(args) > index =3D str(time.time()) > - SNIFF_PIDS[index] =3D (pipe, intf, timeout) > + SNIFF_PIDS[index] =3D (pipe, intf, timeout, remote_terminate) > time.sleep(1) > return index >=20 >=20 > -def load_sniff_pcap(index=3D''): > +def load_sniff_pcap(index=3D'', target=3D[]): > """ > Stop sniffer and return pcap file > """ > child_exit =3D False > if index in SNIFF_PIDS.keys(): > - pipe, intf, timeout =3D SNIFF_PIDS[index] > + pipe, intf, timeout, remote_terminate =3D SNIFF_PIDS[index] > time_elapse =3D int(time.time() - float(index)) > while time_elapse < timeout: > if pipe.poll() is not None: [Lijuan] Exception happened here because poll is not an attribute of pxssh. > @@ -876,8 +901,22 @@ def load_sniff_pcap(index=3D''): > time_elapse +=3D 1 >=20 > if not child_exit: > - pipe.send_signal(signal.SIGINT) > - pipe.wait() > + if remote_terminate =3D=3D 1: > + tcpdump_quit_session=3Dpxssh.pxssh() > + tcpdump_quit_session.login(server=3Dtarget[1], > username=3Dtarget[0], > + password=3Dtarget[2], > original_prompt=3D'[$#>]', > + login_timeout=3D20) > + tcpdump_quit_session.sendline('kill -2 $(pidof > tcpdump)') > + tcpdump_quit_session.prompt() > + tcpdump_quit_session.logout() > + # Teminate the tcpdump_session > + pipe.prompt() > + pipe.logout() > + pipe =3D None > + > + if pipe is not None: > + pipe.send_signal(signal.SIGINT) > + pipe.wait() >=20 > # wait pcap file ready > time.sleep(1) > @@ -886,42 +925,6 @@ def load_sniff_pcap(index=3D''): > return "" >=20 >=20 > -def load_sniff_packets(index=3D''): > - """ > - Stop sniffer and return packet objects > - """ > - pkts =3D [] > - child_exit =3D False > - if index in SNIFF_PIDS.keys(): > - pipe, intf, timeout =3D SNIFF_PIDS[index] > - time_elapse =3D int(time.time() - float(index)) > - while time_elapse < timeout: > - if pipe.poll() is not None: > - child_exit =3D True > - break > - > - time.sleep(1) > - time_elapse +=3D 1 > - > - if not child_exit: > - pipe.send_signal(signal.SIGINT) > - pipe.wait() > - > - # wait pcap file ready > - time.sleep(1) > - try: > - cap_pkts =3D rdpcap("/tmp/sniff_%s.pcap" % intf) > - for pkt in cap_pkts: > - # packet gen should be scapy > - packet =3D Packet(tx_port=3Dintf) > - packet.pktgen.assign_pkt(pkt) > - pkts.append(packet) > - except: > - pass > - > - return pkts > - > - > def load_pcapfile(filename=3D""): > pkts =3D [] > try: > @@ -983,7 +986,6 @@ if __name__ =3D=3D "__main__": > inst =3D sniff_packets("lo", timeout=3D5) > pkt =3D Packet(pkt_type=3D'UDP') > pkt.send_pkt(tx_port=3D'lo') > - pkts =3D load_sniff_packets(inst) >=20 > pkt =3D Packet(pkt_type=3D'UDP', pkt_len=3D1500, ran_payload=3DTrue) > pkt.send_pkt(tx_port=3D'lo') > diff --git a/framework/tester.py b/framework/tester.py index > a775f68..c5b705d 100755 > --- a/framework/tester.py > +++ b/framework/tester.py > @@ -35,6 +35,7 @@ Interface for bulk traffic generators. >=20 > import re > import subprocess > +import os > from time import sleep > from settings import NICS, load_global_setting, PERF_SETTING from crb > import Crb @@ -560,8 +561,6 @@ class Tester(Crb): > module =3D __import__("packet") > pkt_c =3D getattr(module, "Packet") > send_f =3D getattr(module, "send_packets") > - sniff_f =3D getattr(module, "sniff_packets") > - load_f =3D getattr(module, "load_sniff_packets") > compare_f =3D getattr(module, "compare_pktload") > strip_f =3D getattr(module, "strip_pktload") > save_f =3D getattr(module, "save_packets") @@ -606,7 +605,7 @@ > class Tester(Crb): >=20 > # send and sniff packets > save_f(pkts=3Dpkts, filename=3D"/tmp/%s_tx.pcap" % txIntf) > - inst =3D sniff_f(intf=3DrxIntf, count=3Dpktnum, timeout=3Dti= meout, > filters=3D > + inst =3D self.tcpdump_sniff_packets(intf=3DrxIntf, > + count=3Dpktnum, timeout=3Dtimeout, filters=3D > [{'layer': 'network', 'config': {'srcport': '65535'}}, > {'layer': 'network', 'config': {'dstport': '65535'}}]) > rx_inst[rxport] =3D inst > @@ -627,7 +626,7 @@ class Tester(Crb): > # Verify all packets > prev_id =3D -1 > for txport, rxport in portList: > - recv_pkts =3D load_f(rx_inst[rxport]) > + recv_pkts =3D > + self.load_tcpdump_sniff_packets(rx_inst[rxport]) >=20 > # only report when recevied number not matched > if len(tx_pkts[txport]) > len(recv_pkts): > @@ -704,6 +703,47 @@ class Tester(Crb): > self.proc.kill() > self.proc =3D None >=20 > + def tcpdump_sniff_packets(self, intf, count=3D0, timeout=3D5, filter= s=3D[]): > + """ > + Wrapper for packet module sniff_packets > + """ > + # load functions in packet module > + module =3D __import__("packet") > + sniff_f =3D getattr(module, "sniff_packets") > + > + target=3D[] > + target.append(self.get_username()) > + target.append(self.get_ip_address()) > + target.append(self.get_password()) > + return sniff_f(intf, count, timeout, filters, target) > + > + def load_tcpdump_sniff_pcap(self, index=3D''): > + """ > + Wrapper for packet module load_sniff_pcap > + """ > + # load functions in packet module > + module =3D __import__("packet") > + load_pcap_f =3D getattr(module, "load_sniff_pcap") > + > + target=3D[] > + target.append(self.get_username()) > + target.append(self.get_ip_address()) > + target.append(self.get_password()) > + pcap =3D load_pcap_f(index, target) > + self.session.copy_file_from(pcap) > + > + return pcap.split(os.sep)[-1] > + > + def load_tcpdump_sniff_packets(self, index=3D''): > + """ > + Wrapper for packet module load_pcapfile > + """ > + # load functions in packet module > + packet =3D __import__("packet") > + file =3D self.load_tcpdump_sniff_pcap(index) > + > + return packet.load_pcapfile(file) > + > def kill_all(self, killall=3DFalse): > """ > Kill all scapy process or DPDK application on tester. > -- > 2.7.4