From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id EC408A04B1 for ; Wed, 26 Aug 2020 21:58:14 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id C16FB1F1C; Wed, 26 Aug 2020 21:58:14 +0200 (CEST) Received: from mail-ot1-f42.google.com (mail-ot1-f42.google.com [209.85.210.42]) by dpdk.org (Postfix) with ESMTP id 46677255 for ; Wed, 26 Aug 2020 21:58:14 +0200 (CEST) Received: by mail-ot1-f42.google.com with SMTP id f42so2517136otf.13 for ; Wed, 26 Aug 2020 12:58:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=W8ajtGlZS4BoRqyAox2nmFu2+fCvEpZRhhIYsvwCnxg=; b=SO9PUmZ6OFjMNnfV2nFlrdkl4ChXuKAFMjjrBh2H3jqT+hEkX6yTAwCMS3LQ58sHz2 TgKtRI6EJFpUlo+LWBicqpewV01Zi2O+lz2Jk4qQpatzZg0wzkkpwxUY2db8k8PUgJVC 7Kr8ZHmUSnV1tQJ+1S5F+3u2Fi+3hsEpVyD1Y= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=W8ajtGlZS4BoRqyAox2nmFu2+fCvEpZRhhIYsvwCnxg=; b=sofrLPD4w+lCtsp3NFbRbZ+1L/c0KwY0IdzfqRI5qXFY1N50yNA0S1XkLAc/7udimM D8Feq+rUKHSEZC77C5btV74W3BTyrrIzrywsDxrsJJiRR/kuCIxP8esse6WhKqrIJG5z lD6EcVy4Xpzu9QggznQvbYcd4U3tlAT4vb7lmevtEpKpbK6GnScEprJKar6b+3oiQ+Fd EKK+hs49P9TvdikNoB4yHMMvjHKTZel2Mxw+KemYTfWd8QNGBLgDVMOsjOJaiim+a6DP m3HV4WwXsVEv9rGiAgijNVwyhdJfklomvQVWValiykHt5AmLQ3Ac1uEXWnf9nsgRlo16 rsKA== X-Gm-Message-State: AOAM531xHq++0HxUqX1KU5LvxPaiCbN0T5Fj80cXylDeSXg5biFXREsD Lu/rj1/5Lyi+3bXLtoCYsejUu69yrYhg8VR3KaKvFw== X-Google-Smtp-Source: ABdhPJys9uqemAnrvt+gSLM87DnS/fKwMct7Cr5i+2yi1SgYZvu2iGC1lNGaG+2/qL/w9G0p5OYIkUsP3xd4HYs1IXU= X-Received: by 2002:a9d:1905:: with SMTP id j5mr8496963ota.283.1598471892388; Wed, 26 Aug 2020 12:58:12 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Ajit Khaparde Date: Wed, 26 Aug 2020 12:57:56 -0700 Message-ID: To: Brandon Lo Cc: ci@dpdk.org, dpdklab Content-Type: multipart/alternative; boundary="00000000000054bae805adcd3d31" Subject: Re: [dpdk-ci] New 100G Broadcom NIC Ubuntu X-BeenThere: ci@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK CI discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ci-bounces@dpdk.org Sender: "ci" --00000000000054bae805adcd3d31 Content-Type: text/plain; charset="UTF-8" Not yet Brandon. But I believe I might have to look at what trex is doing to make progress. I will let you know if I need any help. Thanks Ajit On Tue, Aug 25, 2020 at 1:13 PM Brandon Lo wrote: > Hi Ajit, > > Did you find anything of interest on the machine? > I could help out if needed. > > Thanks, > Brandon > > On Thu, Aug 20, 2020 at 5:07 PM Ajit Khaparde > wrote: > > > > Hi Brandon, > > I do see some issues while running trex on this setup. > > I will have to dig further. I will let you know once I find something or > may be a fix. > > > > Thanks > > Ajit > > > > On Thu, Aug 20, 2020 at 8:39 AM Brandon Lo wrote: > >> > >> Hi Ajit, > >> > >> I believe DTS/TREX is run on the io machine as a tester. Rhea runs > >> testpmd (through an ssh session from io --> rhea) to catch information > >> as a DUT. > >> > >> The only commands that I run (on io) are: > >> cd /opt/dts > >> export DTS_CFG_FOLDER='conf_100g' > >> ./dts -s > >> > >> The rest is managed by DTS itself. > >> The issue occurs when DTS is trying to send packets to rhea from io on > >> the new 100G NIC. > >> It seems to happen after it tries to measure throughput using TREX's > API. > >> > >> Thanks, > >> Brandon > >> > >> On Thu, Aug 20, 2020 at 12:29 AM Ajit Khaparde > >> wrote: > >> > > >> > Hi Brandon, > >> > I was trying to see what exactly is happening on the setup and what > is causing the problem. > >> > But I will need your help to use the proper code and commands. > >> > > >> > I believe you run the testpmd command on io. And the trex is executed > on rhea? > >> > Can you point me to the location of the code and the steps you are > following while running the test? > >> > > >> > Thanks > >> > Ajit > >> > > >> > On Tue, Aug 11, 2020 at 10:47 AM Ajit Khaparde < > ajit.khaparde@broadcom.com> wrote: > >> >> > >> >> Hi Brandon, > >> >> I haven't. I tried to login as well. But I had some issues doing it > from the office. > >> >> I just have to remind myself to try it again once I get home before > I connect to the company VPN. > >> >> Thanks for checking in. I will try to update you as soon as I can. > >> >> > >> >> Thanks > >> >> Ajit > >> >> > >> >> On Tue, Aug 11, 2020 at 10:46 AM Brandon Lo wrote: > >> >>> > >> >>> Hi Ajit, > >> >>> > >> >>> I'm just checking in; have you heard of any updates on this issue? > >> >>> > >> >>> Thanks, > >> >>> Brandon > >> >>> > >> >>> On Tue, Aug 4, 2020 at 1:42 PM Brandon Lo wrote: > >> >>>> > >> >>>> Hi Ajit, > >> >>>> > >> >>>> Yes, I believe the issue is coming from the trex/tester system > with the 100G NIC. > >> >>>> I'm not sure what causes this issue; if I run trex using the > command "cd /opt/v2.82;./t-rex-64 -i --cfg /etc/trex_cfg_100g.yaml -c 7", > which is the same command used in DTS, it seems to launch without failing. > >> >>>> > >> >>>> If you want to replicate it, here are the steps that I ran: > >> >>>> > >> >>>> (on io) cd /opt/dts > >> >>>> export DTS_CFG_FOLDER='conf_100g' > >> >>>> > >> >>>> conf_100g has the new configuration files to use the new PCI id > and pktgen config file > >> >>>> > >> >>>> ./dts -s > >> >>>> > >> >>>> > >> >>>> Thanks for your help, > >> >>>> Brandon > >> >>>> > >> >>>> On Tue, Aug 4, 2020 at 12:38 PM Ajit Khaparde < > ajit.khaparde@broadcom.com> wrote: > >> >>>>> > >> >>>>> Hi Brandon, > >> >>>>> No, I haven't seen or heard this before. > >> >>>>> But I will try to have someone run it again. > >> >>>>> > >> >>>>> Just to make sure - > >> >>>>> You are running trex on the 100G NIC and the problem is > encountered on that setup? > >> >>>>> Or is it the system that is running testpmd where you are running > into the issue? > >> >>>>> > >> >>>>> Thanks > >> >>>>> Ajit > >> >>>>> > >> >>>>> On Tue, Aug 4, 2020 at 9:16 AM Brandon Lo > wrote: > >> >>>>>> > >> >>>>>> Hi Ajit, > >> >>>>>> > >> >>>>>> I'm running into a problem with trying to run > nic_single_core_perf on the new NIC. > >> >>>>>> The current configuration uses trex version v2.82. > >> >>>>>> However, I'm running into an error when it tries to actually do > a test case in the nic_single_core_perf. > >> >>>>>> > >> >>>>>> The output looks like this when it reaches a test case: > >> >>>>>> > >> >>>>>>> TestNicSingleCorePerf: Test running at parameters: framesize: > 64, rxd/txd: 512 > >> >>>>>>> dut.rhea: > ./x86_64-native-linuxapp-gcc/app/testpmd -l 16,17 -n 4 -w 0000:81:00.0 -w > 0000:81:00.1 --file-prefix=dpdk_11307_20200804160513 -- -i > --portmask=0x3 --txd=512 --rxd=512 > >> >>>>>>> dut.rhea: start > >> >>>>>>> TestNicSingleCorePerf: Test Case > test_perf_nic_single_core Result ERROR: Traceback (most recent call last): > >> >>>>>>> File "/opt/dts/framework/test_case.py", line 316, in > _execute_test_case > >> >>>>>>> case_obj() > >> >>>>>>> File "tests/TestSuite_nic_single_core_perf.py", line 198, in > test_perf_nic_single_core > >> >>>>>>> self.perf_test(self.nb_ports) > >> >>>>>>> File "tests/TestSuite_nic_single_core_perf.py", line 259, in > perf_test > >> >>>>>>> _, packets_received = > self.tester.pktgen.measure_throughput(stream_ids=streams, > options=traffic_opt) > >> >>>>>>> File "/opt/dts/framework/pktgen_base.py", line 245, in > measure_throughput > >> >>>>>>> self._prepare_transmission(stream_ids=stream_ids) > >> >>>>>>> File "/opt/dts/framework/pktgen_trex.py", line 779, in > _prepare_transmission > >> >>>>>>> self._conn.reset(ports=self._ports) > >> >>>>>>> File > "/opt/v2.82/automation/trex_control_plane/interactive/trex/common/trex_api_annotators.py", > line 51, in wrap2 > >> >>>>>>> ret = f(*args, **kwargs) > >> >>>>>>> File > "/opt/v2.82/automation/trex_control_plane/interactive/trex/stl/trex_stl_client.py", > line 339, in reset > >> >>>>>>> self.clear_stats(ports) > >> >>>>>>> File > "/opt/v2.82/automation/trex_control_plane/interactive/trex/common/trex_api_annotators.py", > line 51, in wrap2 > >> >>>>>>> ret = f(*args, **kwargs) > >> >>>>>>> File > "/opt/v2.82/automation/trex_control_plane/interactive/trex/stl/trex_stl_client.py", > line 1467, in clear_stats > >> >>>>>>> self._clear_stats_common(ports, clear_global, clear_xstats) > >> >>>>>>> File > "/opt/v2.82/automation/trex_control_plane/interactive/trex/common/trex_client.py", > line 2840, in _clear_stats_common > >> >>>>>>> raise TRexError(rc) > >> >>>>>>> trex.common.trex_exceptions.TRexError: *** [RPC] - Failed to > get server response from tcp://127.0.0.1:4501 > >> >>>>>> > >> >>>>>> > >> >>>>>> I have found one similar case on the github repository for trex, > but the solution was vendor-specific: > https://github.com/cisco-system-traffic-generator/trex-core/issues/147. > >> >>>>>> Have you ran into this issue before? > >> >>>>>> > >> >>>>>> Thanks, > >> >>>>>> Brandon > >> >>>> > >> >>>> > >> >>>> > >> >>>> -- > >> >>>> > >> >>>> Brandon Lo > >> >>>> > >> >>>> UNH InterOperability Laboratory > >> >>>> > >> >>>> 21 Madbury Rd, Suite 100, Durham, NH 03824 > >> >>>> > >> >>>> blo@iol.unh.edu > >> >>>> > >> >>>> www.iol.unh.edu > >> >>> > >> >>> > >> >>> > >> >>> -- > >> >>> > >> >>> Brandon Lo > >> >>> > >> >>> UNH InterOperability Laboratory > >> >>> > >> >>> 21 Madbury Rd, Suite 100, Durham, NH 03824 > >> >>> > >> >>> blo@iol.unh.edu > >> >>> > >> >>> www.iol.unh.edu > >> > >> > >> > >> -- > >> > >> Brandon Lo > >> > >> UNH InterOperability Laboratory > >> > >> 21 Madbury Rd, Suite 100, Durham, NH 03824 > >> > >> blo@iol.unh.edu > >> > >> www.iol.unh.edu > > > > -- > > Brandon Lo > > UNH InterOperability Laboratory > > 21 Madbury Rd, Suite 100, Durham, NH 03824 > > blo@iol.unh.edu > > www.iol.unh.edu > --00000000000054bae805adcd3d31 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Not yet Brandon.
But I believe I might have to look at what trex = is doing to make progress.
I will let you know if I need any help.

=
Thanks=
= Ajit

On Tue, Aug 25, 2020 at 1:13 PM Brandon Lo <blo@iol.unh.edu> wrote:
Hi Ajit,

Did you find anything of interest on the machine?
I could help out if needed.

Thanks,
Brandon

On Thu, Aug 20, 2020 at 5:07 PM Ajit Khaparde
<ajit.kh= aparde@broadcom.com> wrote:
>
> Hi Brandon,
> I do see some issues while running trex on this setup.
> I will have to dig further. I will let you know once I find something = or may be a fix.
>
> Thanks
> Ajit
>
> On Thu, Aug 20, 2020 at 8:39 AM Brandon Lo <blo@iol.unh.edu> wrote:
>>
>> Hi Ajit,
>>
>> I believe DTS/TREX is run on the io machine as a tester. Rhea runs=
>> testpmd (through an ssh session from io --> rhea) to catch info= rmation
>> as a DUT.
>>
>> The only commands that I run (on io) are:
>> cd /opt/dts
>> export DTS_CFG_FOLDER=3D'conf_100g'
>> ./dts -s
>>
>> The rest is managed by DTS itself.
>> The issue occurs when DTS is trying to send packets to rhea from i= o on
>> the new 100G NIC.
>> It seems to happen after it tries to measure throughput using TREX= 's API.
>>
>> Thanks,
>> Brandon
>>
>> On Thu, Aug 20, 2020 at 12:29 AM Ajit Khaparde
>> <ajit.khaparde@broadcom.com> wrote:
>> >
>> > Hi Brandon,
>> > I was trying to see what exactly is happening on the setup an= d what is causing the problem.
>> > But I will need your help to use the proper code and commands= .
>> >
>> > I believe you run the testpmd command on io. And the trex is = executed on rhea?
>> > Can you point me to the location of the code and the steps yo= u are following while running the test?
>> >
>> > Thanks
>> > Ajit
>> >
>> > On Tue, Aug 11, 2020 at 10:47 AM Ajit Khaparde <ajit.khaparde@broadco= m.com> wrote:
>> >>
>> >> Hi Brandon,
>> >> I haven't. I tried to login as well. But I had some i= ssues doing it from the office.
>> >> I just have to remind myself to try it again once I get h= ome before I connect to the company VPN.
>> >> Thanks for checking in. I will try to update you as soon = as I can.
>> >>
>> >> Thanks
>> >> Ajit
>> >>
>> >> On Tue, Aug 11, 2020 at 10:46 AM Brandon Lo <blo@iol.unh.edu> wrote= :
>> >>>
>> >>> Hi Ajit,
>> >>>
>> >>> I'm just checking in; have you heard of any updat= es on this issue?
>> >>>
>> >>> Thanks,
>> >>> Brandon
>> >>>
>> >>> On Tue, Aug 4, 2020 at 1:42 PM Brandon Lo <blo@iol.unh.edu> wrote= :
>> >>>>
>> >>>> Hi Ajit,
>> >>>>
>> >>>> Yes, I believe the issue is coming from the trex/= tester system with the 100G NIC.
>> >>>> I'm not sure what causes this issue; if I run= trex using the command "cd /opt/v2.82;./t-rex-64 -i --cfg /etc/trex_c= fg_100g.yaml -c 7", which is the same command used in DTS, it seems to= launch without failing.
>> >>>>
>> >>>> If you want to replicate it, here are the steps t= hat I ran:
>> >>>>
>> >>>> (on io) cd /opt/dts
>> >>>> export DTS_CFG_FOLDER=3D'conf_100g'
>> >>>>
>> >>>> conf_100g has the new configuration files to use = the new PCI id and pktgen config file
>> >>>>
>> >>>> ./dts -s
>> >>>>
>> >>>>
>> >>>> Thanks for your help,
>> >>>> Brandon
>> >>>>
>> >>>> On Tue, Aug 4, 2020 at 12:38 PM Ajit Khaparde <= ;ajit.khapa= rde@broadcom.com> wrote:
>> >>>>>
>> >>>>> Hi Brandon,
>> >>>>> No, I haven't seen or heard this before.<= br> >> >>>>> But I will try to have someone run it again.<= br> >> >>>>>
>> >>>>> Just to make sure -
>> >>>>> You are running trex on the 100G NIC and the = problem is encountered on that setup?
>> >>>>> Or is it the system that is running testpmd w= here you are running into the issue?
>> >>>>>
>> >>>>> Thanks
>> >>>>> Ajit
>> >>>>>
>> >>>>> On Tue, Aug 4, 2020 at 9:16 AM Brandon Lo <= ;blo@iol.unh.edu&g= t; wrote:
>> >>>>>>
>> >>>>>> Hi Ajit,
>> >>>>>>
>> >>>>>> I'm running into a problem with tryin= g to run nic_single_core_perf on the new NIC.
>> >>>>>> The current configuration uses trex versi= on v2.82.
>> >>>>>> However, I'm running into an error wh= en it tries to actually do a test case in the nic_single_core_perf.
>> >>>>>>
>> >>>>>> The output looks like this when it reache= s a test case:
>> >>>>>>
>> >>>>>>> TestNicSingleCorePerf: Test running a= t parameters: framesize: 64, rxd/txd: 512
>> >>>>>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0dut.rhea: ./x86_64-native-linu= xapp-gcc/app/testpmd -l 16,17 -n 4 -w 0000:81:00.0 -w 0000:81:00.1=C2=A0 --= file-prefix=3Ddpdk_11307_20200804160513=C2=A0 =C2=A0 -- -i=C2=A0 --portmask= =3D0x3 --txd=3D512 --rxd=3D512
>> >>>>>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0dut.rhea: start
>> >>>>>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Tes= tNicSingleCorePerf: Test Case test_perf_nic_single_core Result ERROR: Trace= back (most recent call last):
>> >>>>>>>=C2=A0 =C2=A0File "/opt/dts/frame= work/test_case.py", line 316, in _execute_test_case
>> >>>>>>>=C2=A0 =C2=A0 =C2=A0case_obj()
>> >>>>>>>=C2=A0 =C2=A0File "tests/TestSuit= e_nic_single_core_perf.py", line 198, in test_perf_nic_single_core
>> >>>>>>>=C2=A0 =C2=A0 =C2=A0self.perf_test(sel= f.nb_ports)
>> >>>>>>>=C2=A0 =C2=A0File "tests/TestSuit= e_nic_single_core_perf.py", line 259, in perf_test
>> >>>>>>>=C2=A0 =C2=A0 =C2=A0_, packets_receive= d =3D self.tester.pktgen.measure_throughput(stream_ids=3Dstreams, options= =3Dtraffic_opt)
>> >>>>>>>=C2=A0 =C2=A0File "/opt/dts/frame= work/pktgen_base.py", line 245, in measure_throughput
>> >>>>>>>=C2=A0 =C2=A0 =C2=A0self._prepare_tran= smission(stream_ids=3Dstream_ids)
>> >>>>>>>=C2=A0 =C2=A0File "/opt/dts/frame= work/pktgen_trex.py", line 779, in _prepare_transmission
>> >>>>>>>=C2=A0 =C2=A0 =C2=A0self._conn.reset(p= orts=3Dself._ports)
>> >>>>>>>=C2=A0 =C2=A0File "/opt/v2.82/aut= omation/trex_control_plane/interactive/trex/common/trex_api_annotators.py&q= uot;, line 51, in wrap2
>> >>>>>>>=C2=A0 =C2=A0 =C2=A0ret =3D f(*args, *= *kwargs)
>> >>>>>>>=C2=A0 =C2=A0File "/opt/v2.82/aut= omation/trex_control_plane/interactive/trex/stl/trex_stl_client.py", l= ine 339, in reset
>> >>>>>>>=C2=A0 =C2=A0 =C2=A0self.clear_stats(p= orts)
>> >>>>>>>=C2=A0 =C2=A0File "/opt/v2.82/aut= omation/trex_control_plane/interactive/trex/common/trex_api_annotators.py&q= uot;, line 51, in wrap2
>> >>>>>>>=C2=A0 =C2=A0 =C2=A0ret =3D f(*args, *= *kwargs)
>> >>>>>>>=C2=A0 =C2=A0File "/opt/v2.82/aut= omation/trex_control_plane/interactive/trex/stl/trex_stl_client.py", l= ine 1467, in clear_stats
>> >>>>>>>=C2=A0 =C2=A0 =C2=A0self._clear_stats_= common(ports, clear_global, clear_xstats)
>> >>>>>>>=C2=A0 =C2=A0File "/opt/v2.82/aut= omation/trex_control_plane/interactive/trex/common/trex_client.py", li= ne 2840, in _clear_stats_common
>> >>>>>>>=C2=A0 =C2=A0 =C2=A0raise TRexError(rc= )
>> >>>>>>> trex.common.trex_exceptions.TRexError= : *** [RPC] - Failed to get server response from tcp://127.0.0.1:4501
>> >>>>>>
>> >>>>>>
>> >>>>>> I have found one similar case on the gith= ub repository for trex, but the solution was vendor-specific: https://github.com/cisco-system-traffic-g= enerator/trex-core/issues/147.
>> >>>>>> Have you ran into this issue before?
>> >>>>>>
>> >>>>>> Thanks,
>> >>>>>> Brandon
>> >>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>>
>> >>>> Brandon Lo
>> >>>>
>> >>>> UNH InterOperability Laboratory
>> >>>>
>> >>>> 21 Madbury Rd, Suite 100, Durham, NH 03824
>> >>>>
>> >>>> blo@iol.unh.edu
>> >>>>
>> >>>> www.iol.unh.edu
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>>
>> >>> Brandon Lo
>> >>>
>> >>> UNH InterOperability Laboratory
>> >>>
>> >>> 21 Madbury Rd, Suite 100, Durham, NH 03824
>> >>>
>> >>> = blo@iol.unh.edu
>> >>>
>> >>> www.iol.unh.edu
>>
>>
>>
>> --
>>
>> Brandon Lo
>>
>> UNH InterOperability Laboratory
>>
>> 21 Madbury Rd, Suite 100, Durham, NH 03824
>>
>> blo@iol.unh.e= du
>>
>> www.iol.unh.edu



--

Brandon Lo

UNH InterOperability Laboratory

21 Madbury Rd, Suite 100, Durham, NH 03824

blo@iol.unh.edu
www= .iol.unh.edu
--00000000000054bae805adcd3d31--