From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 808B7A04AF for ; Thu, 20 Aug 2020 23:07:40 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 787E11C021; Thu, 20 Aug 2020 23:07:40 +0200 (CEST) Received: from mail-oi1-f182.google.com (mail-oi1-f182.google.com [209.85.167.182]) by dpdk.org (Postfix) with ESMTP id 816C91BEC4 for ; Thu, 20 Aug 2020 23:07:39 +0200 (CEST) Received: by mail-oi1-f182.google.com with SMTP id k4so3148115oik.2 for ; Thu, 20 Aug 2020 14:07:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=5cg8HQ9glMrFYfnCOt2/g633nONMTbcGGEpYcLGfPcc=; b=EQ3ZLyGnBKvP6ufeGob5+B4WbRI0WUoiAPuNQFvDrvziwiS99NhiSxrdIZ46nCR6So GR2rY4Lqjkpwdq3UCIKhNEQD96zxCRvpfxlADN6vGd9AnFjjTzA6wJCLVoKC7+SOsLLe 1gjxG0yHHOQBChvxxUnyD+SOwHrYv0+yt5ByA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=5cg8HQ9glMrFYfnCOt2/g633nONMTbcGGEpYcLGfPcc=; b=eTzlJnxw/yFGT2sIa8XEYM89iUbe0vcfQEiKMgSjYy//sYGsZNkWpkwprrzLaQKj5F 6bHnPG9gEGoWeW2irf9kc90S/MGYnESXDz1ohBEvWbP1Q5mXy19LHLfDOI+VpIuf6RCo MJZtj5E/7ZhfhT3yoKkCNGKYoFYYOc88ngAjMW2shAAdPSKFYIR2r737bCYViUSqxRSg zZ6amB7r7XfVxVl+EFtFBn/nnsqykULzoBGU4m6aXfOgnOoX1Oy1ImKx5rxTdBtf/+Ay cFBDMn4UpBpj1xiZvK6Qa3r3ejttbxKFaQTyOy69QdxQoSY3Cv2+0uvS2bOQcZwA8MQS u60g== X-Gm-Message-State: AOAM53352Npvh39Kv6L9uq/4nz5KMOjB0r53YgtEnL1g33p221rgHTCS mUiBJ1WnfmFlLoez8onB5RYN5l9sMbQpxh7CGLWBhQ== X-Google-Smtp-Source: ABdhPJyRxOUC0Uqsb+LJouMY2ciCsBnVyzdo6a8eZndAownfIbo+poitqpObdkiOrtkfg3s6XwuCEE6jHVsQKXIE+ho= X-Received: by 2002:a54:4f85:: with SMTP id g5mr177244oiy.27.1597957658525; Thu, 20 Aug 2020 14:07:38 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Ajit Khaparde Date: Thu, 20 Aug 2020 14:07:22 -0700 Message-ID: To: Brandon Lo Cc: ci@dpdk.org, dpdklab Content-Type: multipart/alternative; boundary="0000000000009aa94f05ad558233" Subject: Re: [dpdk-ci] New 100G Broadcom NIC Ubuntu X-BeenThere: ci@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK CI discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ci-bounces@dpdk.org Sender: "ci" --0000000000009aa94f05ad558233 Content-Type: text/plain; charset="UTF-8" Hi Brandon, I do see some issues while running trex on this setup. I will have to dig further. I will let you know once I find something or may be a fix. Thanks Ajit On Thu, Aug 20, 2020 at 8:39 AM Brandon Lo wrote: > Hi Ajit, > > I believe DTS/TREX is run on the io machine as a tester. Rhea runs > testpmd (through an ssh session from io --> rhea) to catch information > as a DUT. > > The only commands that I run (on io) are: > cd /opt/dts > export DTS_CFG_FOLDER='conf_100g' > ./dts -s > > The rest is managed by DTS itself. > The issue occurs when DTS is trying to send packets to rhea from io on > the new 100G NIC. > It seems to happen after it tries to measure throughput using TREX's API. > > Thanks, > Brandon > > On Thu, Aug 20, 2020 at 12:29 AM Ajit Khaparde > wrote: > > > > Hi Brandon, > > I was trying to see what exactly is happening on the setup and what is > causing the problem. > > But I will need your help to use the proper code and commands. > > > > I believe you run the testpmd command on io. And the trex is executed on > rhea? > > Can you point me to the location of the code and the steps you are > following while running the test? > > > > Thanks > > Ajit > > > > On Tue, Aug 11, 2020 at 10:47 AM Ajit Khaparde < > ajit.khaparde@broadcom.com> wrote: > >> > >> Hi Brandon, > >> I haven't. I tried to login as well. But I had some issues doing it > from the office. > >> I just have to remind myself to try it again once I get home before I > connect to the company VPN. > >> Thanks for checking in. I will try to update you as soon as I can. > >> > >> Thanks > >> Ajit > >> > >> On Tue, Aug 11, 2020 at 10:46 AM Brandon Lo wrote: > >>> > >>> Hi Ajit, > >>> > >>> I'm just checking in; have you heard of any updates on this issue? > >>> > >>> Thanks, > >>> Brandon > >>> > >>> On Tue, Aug 4, 2020 at 1:42 PM Brandon Lo wrote: > >>>> > >>>> Hi Ajit, > >>>> > >>>> Yes, I believe the issue is coming from the trex/tester system with > the 100G NIC. > >>>> I'm not sure what causes this issue; if I run trex using the command > "cd /opt/v2.82;./t-rex-64 -i --cfg /etc/trex_cfg_100g.yaml -c 7", which is > the same command used in DTS, it seems to launch without failing. > >>>> > >>>> If you want to replicate it, here are the steps that I ran: > >>>> > >>>> (on io) cd /opt/dts > >>>> export DTS_CFG_FOLDER='conf_100g' > >>>> > >>>> conf_100g has the new configuration files to use the new PCI id and > pktgen config file > >>>> > >>>> ./dts -s > >>>> > >>>> > >>>> Thanks for your help, > >>>> Brandon > >>>> > >>>> On Tue, Aug 4, 2020 at 12:38 PM Ajit Khaparde < > ajit.khaparde@broadcom.com> wrote: > >>>>> > >>>>> Hi Brandon, > >>>>> No, I haven't seen or heard this before. > >>>>> But I will try to have someone run it again. > >>>>> > >>>>> Just to make sure - > >>>>> You are running trex on the 100G NIC and the problem is encountered > on that setup? > >>>>> Or is it the system that is running testpmd where you are running > into the issue? > >>>>> > >>>>> Thanks > >>>>> Ajit > >>>>> > >>>>> On Tue, Aug 4, 2020 at 9:16 AM Brandon Lo wrote: > >>>>>> > >>>>>> Hi Ajit, > >>>>>> > >>>>>> I'm running into a problem with trying to run nic_single_core_perf > on the new NIC. > >>>>>> The current configuration uses trex version v2.82. > >>>>>> However, I'm running into an error when it tries to actually do a > test case in the nic_single_core_perf. > >>>>>> > >>>>>> The output looks like this when it reaches a test case: > >>>>>> > >>>>>>> TestNicSingleCorePerf: Test running at parameters: framesize: 64, > rxd/txd: 512 > >>>>>>> dut.rhea: > ./x86_64-native-linuxapp-gcc/app/testpmd -l 16,17 -n 4 -w 0000:81:00.0 -w > 0000:81:00.1 --file-prefix=dpdk_11307_20200804160513 -- -i > --portmask=0x3 --txd=512 --rxd=512 > >>>>>>> dut.rhea: start > >>>>>>> TestNicSingleCorePerf: Test Case > test_perf_nic_single_core Result ERROR: Traceback (most recent call last): > >>>>>>> File "/opt/dts/framework/test_case.py", line 316, in > _execute_test_case > >>>>>>> case_obj() > >>>>>>> File "tests/TestSuite_nic_single_core_perf.py", line 198, in > test_perf_nic_single_core > >>>>>>> self.perf_test(self.nb_ports) > >>>>>>> File "tests/TestSuite_nic_single_core_perf.py", line 259, in > perf_test > >>>>>>> _, packets_received = > self.tester.pktgen.measure_throughput(stream_ids=streams, > options=traffic_opt) > >>>>>>> File "/opt/dts/framework/pktgen_base.py", line 245, in > measure_throughput > >>>>>>> self._prepare_transmission(stream_ids=stream_ids) > >>>>>>> File "/opt/dts/framework/pktgen_trex.py", line 779, in > _prepare_transmission > >>>>>>> self._conn.reset(ports=self._ports) > >>>>>>> File > "/opt/v2.82/automation/trex_control_plane/interactive/trex/common/trex_api_annotators.py", > line 51, in wrap2 > >>>>>>> ret = f(*args, **kwargs) > >>>>>>> File > "/opt/v2.82/automation/trex_control_plane/interactive/trex/stl/trex_stl_client.py", > line 339, in reset > >>>>>>> self.clear_stats(ports) > >>>>>>> File > "/opt/v2.82/automation/trex_control_plane/interactive/trex/common/trex_api_annotators.py", > line 51, in wrap2 > >>>>>>> ret = f(*args, **kwargs) > >>>>>>> File > "/opt/v2.82/automation/trex_control_plane/interactive/trex/stl/trex_stl_client.py", > line 1467, in clear_stats > >>>>>>> self._clear_stats_common(ports, clear_global, clear_xstats) > >>>>>>> File > "/opt/v2.82/automation/trex_control_plane/interactive/trex/common/trex_client.py", > line 2840, in _clear_stats_common > >>>>>>> raise TRexError(rc) > >>>>>>> trex.common.trex_exceptions.TRexError: *** [RPC] - Failed to get > server response from tcp://127.0.0.1:4501 > >>>>>> > >>>>>> > >>>>>> I have found one similar case on the github repository for trex, > but the solution was vendor-specific: > https://github.com/cisco-system-traffic-generator/trex-core/issues/147. > >>>>>> Have you ran into this issue before? > >>>>>> > >>>>>> Thanks, > >>>>>> Brandon > >>>> > >>>> > >>>> > >>>> -- > >>>> > >>>> Brandon Lo > >>>> > >>>> UNH InterOperability Laboratory > >>>> > >>>> 21 Madbury Rd, Suite 100, Durham, NH 03824 > >>>> > >>>> blo@iol.unh.edu > >>>> > >>>> www.iol.unh.edu > >>> > >>> > >>> > >>> -- > >>> > >>> Brandon Lo > >>> > >>> UNH InterOperability Laboratory > >>> > >>> 21 Madbury Rd, Suite 100, Durham, NH 03824 > >>> > >>> blo@iol.unh.edu > >>> > >>> www.iol.unh.edu > > > > -- > > Brandon Lo > > UNH InterOperability Laboratory > > 21 Madbury Rd, Suite 100, Durham, NH 03824 > > blo@iol.unh.edu > > www.iol.unh.edu > --0000000000009aa94f05ad558233 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi Brandon,
I do see some issues while running trex on this setup= .
I will have to dig further. I will let you know once I find something or m= ay be a fix.

Thanks
Ajit

On Thu, Aug 20, 2020 at 8:39 AM Brandon Lo = <blo@iol.unh.edu> wrote:
Hi Ajit,

I believe DTS/TREX is run on the io machine as a tester. Rhea runs
testpmd (through an ssh session from io --> rhea) to catch information as a DUT.

The only commands that I run (on io) are:
cd /opt/dts
export DTS_CFG_FOLDER=3D'conf_100g'
./dts -s

The rest is managed by DTS itself.
The issue occurs when DTS is trying to send packets to rhea from io on
the new 100G NIC.
It seems to happen after it tries to measure throughput using TREX's AP= I.

Thanks,
Brandon

On Thu, Aug 20, 2020 at 12:29 AM Ajit Khaparde
<ajit.kh= aparde@broadcom.com> wrote:
>
> Hi Brandon,
> I was trying to see what exactly is happening on the setup and what is= causing the problem.
> But I will need your help to use the proper code and commands.
>
> I believe you run the testpmd command on io. And the trex is executed = on rhea?
> Can you point me to the location of the code and the steps you are fol= lowing while running the test?
>
> Thanks
> Ajit
>
> On Tue, Aug 11, 2020 at 10:47 AM Ajit Khaparde <ajit.khaparde@broadcom.com= > wrote:
>>
>> Hi Brandon,
>> I haven't. I tried to login as well. But I had some issues doi= ng it from the office.
>> I just have to remind myself to try it again once I get home befor= e I connect to the company VPN.
>> Thanks for checking in. I will try to update you as soon as I can.=
>>
>> Thanks
>> Ajit
>>
>> On Tue, Aug 11, 2020 at 10:46 AM Brandon Lo <blo@iol.unh.edu> wrote:
>>>
>>> Hi Ajit,
>>>
>>> I'm just checking in; have you heard of any updates on thi= s issue?
>>>
>>> Thanks,
>>> Brandon
>>>
>>> On Tue, Aug 4, 2020 at 1:42 PM Brandon Lo <blo@iol.unh.edu> wrote:
>>>>
>>>> Hi Ajit,
>>>>
>>>> Yes, I believe the issue is coming from the trex/tester sy= stem with the 100G NIC.
>>>> I'm not sure what causes this issue; if I run trex usi= ng the command "cd /opt/v2.82;./t-rex-64 -i --cfg /etc/trex_cfg_100g.y= aml -c 7", which is the same command used in DTS, it seems to launch w= ithout failing.
>>>>
>>>> If you want to replicate it, here are the steps that I ran= :
>>>>
>>>> (on io) cd /opt/dts
>>>> export DTS_CFG_FOLDER=3D'conf_100g'
>>>>
>>>> conf_100g has the new configuration files to use the new P= CI id and pktgen config file
>>>>
>>>> ./dts -s
>>>>
>>>>
>>>> Thanks for your help,
>>>> Brandon
>>>>
>>>> On Tue, Aug 4, 2020 at 12:38 PM Ajit Khaparde <ajit.khaparde@broa= dcom.com> wrote:
>>>>>
>>>>> Hi Brandon,
>>>>> No, I haven't seen or heard this before.
>>>>> But I will try to have someone run it again.
>>>>>
>>>>> Just to make sure -
>>>>> You are running trex on the 100G NIC and the problem i= s encountered on that setup?
>>>>> Or is it the system that is running testpmd where you = are running into the issue?
>>>>>
>>>>> Thanks
>>>>> Ajit
>>>>>
>>>>> On Tue, Aug 4, 2020 at 9:16 AM Brandon Lo <blo@iol.unh.edu> wrote= :
>>>>>>
>>>>>> Hi Ajit,
>>>>>>
>>>>>> I'm running into a problem with trying to run = nic_single_core_perf on the new NIC.
>>>>>> The current configuration uses trex version v2.82.=
>>>>>> However, I'm running into an error when it tri= es to actually do a test case in the nic_single_core_perf.
>>>>>>
>>>>>> The output looks like this when it reaches a test = case:
>>>>>>
>>>>>>> TestNicSingleCorePerf: Test running at paramet= ers: framesize: 64, rxd/txd: 512
>>>>>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0dut.rhea: ./x86_64-native-linuxapp-gc= c/app/testpmd -l 16,17 -n 4 -w 0000:81:00.0 -w 0000:81:00.1=C2=A0 --file-pr= efix=3Ddpdk_11307_20200804160513=C2=A0 =C2=A0 -- -i=C2=A0 --portmask=3D0x3 = --txd=3D512 --rxd=3D512
>>>>>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0dut.rhea: start
>>>>>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 TestNicSingl= eCorePerf: Test Case test_perf_nic_single_core Result ERROR: Traceback (mos= t recent call last):
>>>>>>>=C2=A0 =C2=A0File "/opt/dts/framework/test= _case.py", line 316, in _execute_test_case
>>>>>>>=C2=A0 =C2=A0 =C2=A0case_obj()
>>>>>>>=C2=A0 =C2=A0File "tests/TestSuite_nic_sin= gle_core_perf.py", line 198, in test_perf_nic_single_core
>>>>>>>=C2=A0 =C2=A0 =C2=A0self.perf_test(self.nb_port= s)
>>>>>>>=C2=A0 =C2=A0File "tests/TestSuite_nic_sin= gle_core_perf.py", line 259, in perf_test
>>>>>>>=C2=A0 =C2=A0 =C2=A0_, packets_received =3D sel= f.tester.pktgen.measure_throughput(stream_ids=3Dstreams, options=3Dtraffic_= opt)
>>>>>>>=C2=A0 =C2=A0File "/opt/dts/framework/pktg= en_base.py", line 245, in measure_throughput
>>>>>>>=C2=A0 =C2=A0 =C2=A0self._prepare_transmission(= stream_ids=3Dstream_ids)
>>>>>>>=C2=A0 =C2=A0File "/opt/dts/framework/pktg= en_trex.py", line 779, in _prepare_transmission
>>>>>>>=C2=A0 =C2=A0 =C2=A0self._conn.reset(ports=3Dse= lf._ports)
>>>>>>>=C2=A0 =C2=A0File "/opt/v2.82/automation/t= rex_control_plane/interactive/trex/common/trex_api_annotators.py", lin= e 51, in wrap2
>>>>>>>=C2=A0 =C2=A0 =C2=A0ret =3D f(*args, **kwargs)<= br> >>>>>>>=C2=A0 =C2=A0File "/opt/v2.82/automation/t= rex_control_plane/interactive/trex/stl/trex_stl_client.py", line 339, = in reset
>>>>>>>=C2=A0 =C2=A0 =C2=A0self.clear_stats(ports)
>>>>>>>=C2=A0 =C2=A0File "/opt/v2.82/automation/t= rex_control_plane/interactive/trex/common/trex_api_annotators.py", lin= e 51, in wrap2
>>>>>>>=C2=A0 =C2=A0 =C2=A0ret =3D f(*args, **kwargs)<= br> >>>>>>>=C2=A0 =C2=A0File "/opt/v2.82/automation/t= rex_control_plane/interactive/trex/stl/trex_stl_client.py", line 1467,= in clear_stats
>>>>>>>=C2=A0 =C2=A0 =C2=A0self._clear_stats_common(po= rts, clear_global, clear_xstats)
>>>>>>>=C2=A0 =C2=A0File "/opt/v2.82/automation/t= rex_control_plane/interactive/trex/common/trex_client.py", line 2840, = in _clear_stats_common
>>>>>>>=C2=A0 =C2=A0 =C2=A0raise TRexError(rc)
>>>>>>> trex.common.trex_exceptions.TRexError: *** [RP= C] - Failed to get server response from tcp://127.0.0.1:4501
>>>>>>
>>>>>>
>>>>>> I have found one similar case on the github reposi= tory for trex, but the solution was vendor-specific: https://github.com/cisco-system-traffic-generator/tr= ex-core/issues/147.
>>>>>> Have you ran into this issue before?
>>>>>>
>>>>>> Thanks,
>>>>>> Brandon
>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> Brandon Lo
>>>>
>>>> UNH InterOperability Laboratory
>>>>
>>>> 21 Madbury Rd, Suite 100, Durham, NH 03824
>>>>
>>>> blo@i= ol.unh.edu
>>>>
>>>> www.iol.unh.edu
>>>
>>>
>>>
>>> --
>>>
>>> Brandon Lo
>>>
>>> UNH InterOperability Laboratory
>>>
>>> 21 Madbury Rd, Suite 100, Durham, NH 03824
>>>
>>> blo@iol.u= nh.edu
>>>
>>> www.iol.unh.edu



--

Brandon Lo

UNH InterOperability Laboratory

21 Madbury Rd, Suite 100, Durham, NH 03824

blo@iol.unh.edu
www= .iol.unh.edu
--0000000000009aa94f05ad558233--