From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by dpdk.org (Postfix) with ESMTP id 2FE051B20E for ; Wed, 16 May 2018 08:37:45 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga101.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 15 May 2018 23:37:43 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.49,404,1520924400"; d="scan'208";a="40296113" Received: from fmsmsx105.amr.corp.intel.com ([10.18.124.203]) by fmsmga008.fm.intel.com with ESMTP; 15 May 2018 23:37:42 -0700 Received: from fmsmsx118.amr.corp.intel.com (10.18.116.18) by FMSMSX105.amr.corp.intel.com (10.18.124.203) with Microsoft SMTP Server (TLS) id 14.3.319.2; Tue, 15 May 2018 23:37:41 -0700 Received: from shsmsx152.ccr.corp.intel.com (10.239.6.52) by fmsmsx118.amr.corp.intel.com (10.18.116.18) with Microsoft SMTP Server (TLS) id 14.3.319.2; Tue, 15 May 2018 23:37:41 -0700 Received: from shsmsx103.ccr.corp.intel.com ([169.254.4.210]) by SHSMSX152.ccr.corp.intel.com ([169.254.6.6]) with mapi id 14.03.0319.002; Wed, 16 May 2018 14:37:26 +0800 From: "Liu, Yong" To: "Phil.Yang@arm.com" , "dts@dpdk.org" CC: nd Thread-Topic: [PATCH 1/2] framework/qemu_kvm: fix vm control session failure Thread-Index: AQHT7DSBi0Wl09Qwok+Ui9XLqGyXvqQxkrKA//+iSwCAAJdZoP//gQyAgACZmdA= Date: Wed, 16 May 2018 06:37:25 +0000 Message-ID: <86228AFD5BCD8E4EBFD2B90117B5E81E63041D27@SHSMSX103.ccr.corp.intel.com> References: <1526378779-10530-1-git-send-email-phil.yang@arm.com> <86228AFD5BCD8E4EBFD2B90117B5E81E63041953@SHSMSX103.ccr.corp.intel.com> <86228AFD5BCD8E4EBFD2B90117B5E81E63041AA3@SHSMSX103.ccr.corp.intel.com> In-Reply-To: Accept-Language: zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiZjVkMjk4Y2EtYTM2MS00N2VjLWIyYWQtOGE2Mjc3MTMwZDhkIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjIuNS4xOCIsIlRydXN0ZWRMYWJlbEhhc2giOiIxOE5yT2QzWGE4bVJqK1BNWjdMVzJYT3F5VmlkV3IzeDdROFRBczRPSElsZ3llcGEwaUY3VHVJd3N0WTdiRlQxIn0= x-ctpclassification: CTP_NT dlp-product: dlpe-windows dlp-version: 11.0.200.100 dlp-reaction: no-action x-originating-ip: [10.239.127.40] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dts] [PATCH 1/2] framework/qemu_kvm: fix vm control session failure X-BeenThere: dts@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: test suite reviews and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 May 2018 06:37:45 -0000 Phil, As I known, this command will only send to self.host_dut which should not b= e possible to occur below error. Is your dut running some other processes? [10.169.40.174:vm0] not ready for login dut.10.169.40.174: Can't login [vm0] on [10.169.40.174], retry= 1 times!!! dut.10.169.40.174: lsof -i:7002 | grep telnet | awk '{print $2= }' Exception happened in [lsof -i:7002 | grep telnet | awk '{print $2}'] and o= utput is [lsof -i:7002 | grep telnet | awk '{print $2}' Command not found] Thanks, Marvin > -----Original Message----- > From: Phil.Yang@arm.com [mailto:Phil.Yang@arm.com] > Sent: Wednesday, May 16, 2018 1:23 PM > To: Liu, Yong ; dts@dpdk.org > Cc: nd > Subject: RE: [PATCH 1/2] framework/qemu_kvm: fix vm control session > failure >=20 > Hi Marvin, >=20 > The DUT had installed lsof. >=20 > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > phil@net-arm-c2400_01:~$ sudo lsof -i:7000 | grep telnet > telnet 24489 root 3u IPv4 256487 0t0 TCP localhost:58776- > >localhost:afs3-fileserver (ESTABLISHED) > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >=20 > Thanks, > Phil Yang >=20 > > -----Original Message----- > > From: Liu, Yong > > Sent: Wednesday, May 16, 2018 1:01 PM > > To: Phil Yang ; dts@dpdk.org > > Cc: nd > > Subject: RE: [PATCH 1/2] framework/qemu_kvm: fix vm control session > failure > > > > Hi Phil, > > From the error log, issue was caused by lsof command not installed. Qem= u > > module will check whether connection has been established by this tool. > > Could you please install it and try again? If that fix the issue, I > think only shell > > regression need to be changed. > > > > Thanks, > > Marvin > > > > > -----Original Message----- > > > From: Phil.Yang@arm.com [mailto:Phil.Yang@arm.com] > > > Sent: Wednesday, May 16, 2018 11:56 AM > > > To: Liu, Yong ; dts@dpdk.org > > > Cc: nd > > > Subject: RE: [PATCH 1/2] framework/qemu_kvm: fix vm control session > > > failure > > > > > > Hi Marvin, > > > > > > For telnet vm control I've encountered serial connection got no repos= e > > > after first time login failure. Below is the log. Please check it. > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D > > > m] Started Load/Save Random Seed.[\x1b[0;32m OK \x1b[0m] Started > > > Apply Kernel Variables.[\x1b[0;32m OK \x1b[0m] Mounted FUSE Control > > > File System.[\x1b[0;32m OK \x1b[0m] Mounted Kernel Configuration > > > File System.[\x1b[0;32m OK \x1b[0m] Started Create Static Device > Nodes in > > > /dev. Starting udev Kernel Device Manager...[\x1b[0;32m OK > > > \x1b[0m] Reached target Local File Systems (Pre).[\x1b[0;32m OK > > > \x1b[0m] Started udev Kernel Device Manager.[\x1b[0;32m OK \x1b[0m] > > > Started Dispatch Password Requests to Console Directory > > > Watch.[\x1b[0;32m OK \x1b[0m] Reached target Local Encrypted > > > Volumes.[\x1b[0;32m OK \x1b[0m] Activated swap /swapfile.[\x1b[0;32= m > > > OK \x1b[0m] Reached target Swap.[\x1b[0;32m OK \x1b[0m] Started > > > Flush Journal to Persistent Storage." > > > (Pdb) c > > > [10.169.40.174:vm0] not ready for login > > > dut.10.169.40.174: Can't login [vm0] on [10.169.40.174], > > > retry 1 times!!! > > > dut.10.169.40.174: lsof -i:7002 | grep telnet | awk > > > '{print $2}' > > > Exception happened in [lsof -i:7002 | grep telnet | awk '{print $2}'] > > > and output is [lsof -i:7002 | grep telnet | awk '{print $2}' > > > Command not found] > > > Traceback (most recent call last): > > > File "/root/dpdk-dts/framework/virt_base.py", line 284, in start > > > self._start_vm() > > > File "/root/dpdk-dts/framework/qemu_kvm.py", line 1269, in _start_v= m > > > self.__wait_vm_ready() > > > TimeoutException: TIMEOUT on lsof -i:7002 | grep telnet | awk '{print > $2}' > > > Unhandled expection > > > TestVhostPmdXstats: Failure for Set up VM ENV failed > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > > > > > So I added a delay to flush the useless output. Which can resolve thi= s > > > issue. > > > BTW, only telnet connection has this defect. This fix is kind of > > > workaround, but It works. > > > > > > Thanks, > > > Phil Yang > > > > > > > -----Original Message----- > > > > From: Liu, Yong > > > > Sent: Wednesday, May 16, 2018 9:38 AM > > > > To: Phil Yang ; dts@dpdk.org > > > > Cc: nd > > > > Subject: RE: [PATCH 1/2] framework/qemu_kvm: fix vm control session > > > failure > > > > > > > > Thanks, Phil. I'm fine with the changes, just one question about th= e > > > issue you > > > > met. > > > > Qemu module should has taken care of the problem that VM maybe can'= t > > > login > > > > for the first time. > > > > If login action can't be done continuously, I think there may be > > > > some > > > kind of > > > > issue here. > > > > Could you please check the output of serial session? It may help us > > > > to > > > figure out > > > > the problem. > > > > > > > > Regards, > > > > Marvin > > > > > > > > > -----Original Message----- > > > > > From: phil.yang@arm.com [mailto:phil.yang@arm.com] > > > > > Sent: Tuesday, May 15, 2018 6:06 PM > > > > > To: dts@dpdk.org > > > > > Cc: nd@arm.com; phil.yang@arm.com; Liu, Yong > > > > > Subject: [PATCH 1/2] framework/qemu_kvm: fix vm control session > > > > > failure > > > > > > > > > > For telnet vm control, it will continuously fail to login to vm > > > > > when it doesn't read login prompt during first time connection. S= o > > > > > extend the connection time to get more output each time. > > > > > > > > > > For socket vm control, added LOGIN_PROMPT into the login status > > > > > check for other platform. > > > > > Keep the connection longer to avoid failure and reconnection. > > > > > > > > > > Signed-off-by: Phil Yang > > > > > --- > > > > > framework/qemu_kvm.py | 11 ++++++----- > > > > > 1 file changed, 6 insertions(+), 5 deletions(-) > > > > > > > > > > diff --git a/framework/qemu_kvm.py b/framework/qemu_kvm.py index > > > > > ec33669..57d16f6 100644 > > > > > --- a/framework/qemu_kvm.py > > > > > +++ b/framework/qemu_kvm.py > > > > > @@ -979,7 +979,7 @@ class QEMUKvm(VirtBase): > > > > > Connect to serial port and return connected session for > usage > > > > > if connected failed will return None > > > > > """ > > > > > - shell_reg =3D r"(\s*)\[(.*)\]# " > > > > > + shell_reg =3D r"(.*)# " > > > > > try: > > > > > if getattr(self, 'control_session', None) is None: > > > > > self.control_session =3D self.host_session @@ > > > > > -987,7 > > > > > +987,7 @@ class QEMUKvm(VirtBase): > > > > > self.control_session.send_command("nc -U %s" % > > > > > self.serial_path) > > > > > > > > > > # login message not ouput if timeout too small > > > > > - out =3D self.control_session.send_command("", > > > > > timeout=3D5).replace('\r', '').replace('\n', '') > > > > > + out =3D self.control_session.send_command("", > > > > > timeout=3D15).replace('\r', '').replace('\n', '') > > > > > > > > > > if len(out) =3D=3D 0: > > > > > raise StartVMFailedException("Can't get output > > > > > from [%s:%s]" % (self.host_dut.crb['My IP'], self.vm_name)) @@ > > > > > -1025,7 > > > > > +1025,7 @@ class QEMUKvm(VirtBase): > > > > > Connect to serial port and return connected session for > usage > > > > > if connected failed will return None > > > > > """ > > > > > - shell_reg =3D r"(\s*)\[(.*)\]# " > > > > > + shell_reg =3D r"(.*)# " > > > > > scan_cmd =3D "lsof -i:%d | grep telnet | awk '{print $2}= '" > > > > > % self.serial_port > > > > > > > > > > try: > > > > > @@ -1034,9 +1034,10 @@ class QEMUKvm(VirtBase): > > > > > self.control_session =3D self.host_session > > > > > > > > > > self.control_session.send_expect("telnet > > > > > localhost %d" % self.serial_port, "Connected to localhost", > > > > > timeout=3Dself.OPERATION_TIMEOUT) > > > > > + time.sleep(5) > > > > > > > > > > # output will be empty if timeout too small > > > > > - out =3D self.control_session.send_command("", > > > > > timeout=3D5).replace('\r', '').replace('\n', '') > > > > > + out =3D self.control_session.send_command("", > > > > > timeout=3D10).replace('\r', '').replace('\n', '') > > > > > > > > > > # if no output from serial port, either connection > > > > > close or system hang > > > > > if len(out) =3D=3D 0: > > > > > @@ -1058,7 +1059,7 @@ class QEMUKvm(VirtBase): > > > > > return True > > > > > > > > > > # login into Redhat os, not sure can work on all > > > > > distributions > > > > > - if "x86_64 on an x86_64" not in out: > > > > > + if ("x86_64 on an x86_64" not in out) and > > > > > + (self.LOGIN_PROMPT > > > > > not in out): > > > > > print RED("[%s:%s] not ready for login" % > > > > > (self.host_dut.crb['My IP'], self.vm_name)) > > > > > return False > > > > > else: > > > > > -- > > > > > 2.7.4