From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id DEABDA04AF for ; Thu, 20 Aug 2020 17:39:56 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id AB9071C027; Thu, 20 Aug 2020 17:39:56 +0200 (CEST) Received: from mail-il1-f169.google.com (mail-il1-f169.google.com [209.85.166.169]) by dpdk.org (Postfix) with ESMTP id 040981BEC4 for ; Thu, 20 Aug 2020 17:39:54 +0200 (CEST) Received: by mail-il1-f169.google.com with SMTP id t4so2032310iln.1 for ; Thu, 20 Aug 2020 08:39:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=reHrojqQYQYTavuI3l5H2L1gSBrr0Cy2mdBtVjy2Zg8=; b=BKY4X9k/ZVKi/gxEmLRBnYM2CcLZhU/2DGPbaz3bFI7Prtg0a+lZDDCiaK/jxyAf8e DO7oy6QYpWdUQiyKZ18Dge6pFVS3C0/HEU3joASuJepYmCZKwlYKN3f5DzxEKlv/IpeT KPj0UuTkhxZjZEDPq2mvMRto2FevypZW0saV4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=reHrojqQYQYTavuI3l5H2L1gSBrr0Cy2mdBtVjy2Zg8=; b=iL8ZDy1iMhfDYje1U/WmRjUXyWAxnKAzFaKwyI1lVGRqtNa3e1vAUCQ9ii+6wr3IY2 GQNYODMXBZaiTiltlSt33j3KnFhTiGo2Z48Lq5Yv6EQWkYk2Q8eDGG3YbViQJIE31jpg UHvcUE7T/GaGcF+I3CaKI0WXaFgpcyN8jGlKRLaXnPySrQiKyRjv4kmyAhKINN0KXfK3 CLKS66/1W6mJeydul3V1oO2gfLobBUvKbxyjxYqsxGpyHVlFMvq1CYvMxp/vkLlCgc0p ATipyJVpNFrwTaW5QHZQd1B81PShTLvGkEg5e7g8Ej2jIP2BbJQ6xEKYi69wZb+CWHSF t1ww== X-Gm-Message-State: AOAM530T9be1A4JLVNb92QOdIB0aKibQhhu/TVUrEYz6MBNjTNOfF3dX EQyKoQMM/ITMaLDJDdXzMryrIelFGPpK9bs6l8qllw== X-Google-Smtp-Source: ABdhPJy6xd9zBol1YViNgE7Ef31cAW60mfZgbdQszjOOB+cZt32Q+uAhXmEnhpmwXA9MpIwbzjxnwf7OcP6sDvsivr4= X-Received: by 2002:a05:6e02:cb0:: with SMTP id 16mr2993030ilg.147.1597937994137; Thu, 20 Aug 2020 08:39:54 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Brandon Lo Date: Thu, 20 Aug 2020 11:39:18 -0400 Message-ID: To: Ajit Khaparde Cc: ci@dpdk.org, dpdklab Content-Type: text/plain; charset="UTF-8" Subject: Re: [dpdk-ci] New 100G Broadcom NIC Ubuntu X-BeenThere: ci@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK CI discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ci-bounces@dpdk.org Sender: "ci" Hi Ajit, I believe DTS/TREX is run on the io machine as a tester. Rhea runs testpmd (through an ssh session from io --> rhea) to catch information as a DUT. The only commands that I run (on io) are: cd /opt/dts export DTS_CFG_FOLDER='conf_100g' ./dts -s The rest is managed by DTS itself. The issue occurs when DTS is trying to send packets to rhea from io on the new 100G NIC. It seems to happen after it tries to measure throughput using TREX's API. Thanks, Brandon On Thu, Aug 20, 2020 at 12:29 AM Ajit Khaparde wrote: > > Hi Brandon, > I was trying to see what exactly is happening on the setup and what is causing the problem. > But I will need your help to use the proper code and commands. > > I believe you run the testpmd command on io. And the trex is executed on rhea? > Can you point me to the location of the code and the steps you are following while running the test? > > Thanks > Ajit > > On Tue, Aug 11, 2020 at 10:47 AM Ajit Khaparde wrote: >> >> Hi Brandon, >> I haven't. I tried to login as well. But I had some issues doing it from the office. >> I just have to remind myself to try it again once I get home before I connect to the company VPN. >> Thanks for checking in. I will try to update you as soon as I can. >> >> Thanks >> Ajit >> >> On Tue, Aug 11, 2020 at 10:46 AM Brandon Lo wrote: >>> >>> Hi Ajit, >>> >>> I'm just checking in; have you heard of any updates on this issue? >>> >>> Thanks, >>> Brandon >>> >>> On Tue, Aug 4, 2020 at 1:42 PM Brandon Lo wrote: >>>> >>>> Hi Ajit, >>>> >>>> Yes, I believe the issue is coming from the trex/tester system with the 100G NIC. >>>> I'm not sure what causes this issue; if I run trex using the command "cd /opt/v2.82;./t-rex-64 -i --cfg /etc/trex_cfg_100g.yaml -c 7", which is the same command used in DTS, it seems to launch without failing. >>>> >>>> If you want to replicate it, here are the steps that I ran: >>>> >>>> (on io) cd /opt/dts >>>> export DTS_CFG_FOLDER='conf_100g' >>>> >>>> conf_100g has the new configuration files to use the new PCI id and pktgen config file >>>> >>>> ./dts -s >>>> >>>> >>>> Thanks for your help, >>>> Brandon >>>> >>>> On Tue, Aug 4, 2020 at 12:38 PM Ajit Khaparde wrote: >>>>> >>>>> Hi Brandon, >>>>> No, I haven't seen or heard this before. >>>>> But I will try to have someone run it again. >>>>> >>>>> Just to make sure - >>>>> You are running trex on the 100G NIC and the problem is encountered on that setup? >>>>> Or is it the system that is running testpmd where you are running into the issue? >>>>> >>>>> Thanks >>>>> Ajit >>>>> >>>>> On Tue, Aug 4, 2020 at 9:16 AM Brandon Lo wrote: >>>>>> >>>>>> Hi Ajit, >>>>>> >>>>>> I'm running into a problem with trying to run nic_single_core_perf on the new NIC. >>>>>> The current configuration uses trex version v2.82. >>>>>> However, I'm running into an error when it tries to actually do a test case in the nic_single_core_perf. >>>>>> >>>>>> The output looks like this when it reaches a test case: >>>>>> >>>>>>> TestNicSingleCorePerf: Test running at parameters: framesize: 64, rxd/txd: 512 >>>>>>> dut.rhea: ./x86_64-native-linuxapp-gcc/app/testpmd -l 16,17 -n 4 -w 0000:81:00.0 -w 0000:81:00.1 --file-prefix=dpdk_11307_20200804160513 -- -i --portmask=0x3 --txd=512 --rxd=512 >>>>>>> dut.rhea: start >>>>>>> TestNicSingleCorePerf: Test Case test_perf_nic_single_core Result ERROR: Traceback (most recent call last): >>>>>>> File "/opt/dts/framework/test_case.py", line 316, in _execute_test_case >>>>>>> case_obj() >>>>>>> File "tests/TestSuite_nic_single_core_perf.py", line 198, in test_perf_nic_single_core >>>>>>> self.perf_test(self.nb_ports) >>>>>>> File "tests/TestSuite_nic_single_core_perf.py", line 259, in perf_test >>>>>>> _, packets_received = self.tester.pktgen.measure_throughput(stream_ids=streams, options=traffic_opt) >>>>>>> File "/opt/dts/framework/pktgen_base.py", line 245, in measure_throughput >>>>>>> self._prepare_transmission(stream_ids=stream_ids) >>>>>>> File "/opt/dts/framework/pktgen_trex.py", line 779, in _prepare_transmission >>>>>>> self._conn.reset(ports=self._ports) >>>>>>> File "/opt/v2.82/automation/trex_control_plane/interactive/trex/common/trex_api_annotators.py", line 51, in wrap2 >>>>>>> ret = f(*args, **kwargs) >>>>>>> File "/opt/v2.82/automation/trex_control_plane/interactive/trex/stl/trex_stl_client.py", line 339, in reset >>>>>>> self.clear_stats(ports) >>>>>>> File "/opt/v2.82/automation/trex_control_plane/interactive/trex/common/trex_api_annotators.py", line 51, in wrap2 >>>>>>> ret = f(*args, **kwargs) >>>>>>> File "/opt/v2.82/automation/trex_control_plane/interactive/trex/stl/trex_stl_client.py", line 1467, in clear_stats >>>>>>> self._clear_stats_common(ports, clear_global, clear_xstats) >>>>>>> File "/opt/v2.82/automation/trex_control_plane/interactive/trex/common/trex_client.py", line 2840, in _clear_stats_common >>>>>>> raise TRexError(rc) >>>>>>> trex.common.trex_exceptions.TRexError: *** [RPC] - Failed to get server response from tcp://127.0.0.1:4501 >>>>>> >>>>>> >>>>>> I have found one similar case on the github repository for trex, but the solution was vendor-specific: https://github.com/cisco-system-traffic-generator/trex-core/issues/147. >>>>>> Have you ran into this issue before? >>>>>> >>>>>> Thanks, >>>>>> Brandon >>>> >>>> >>>> >>>> -- >>>> >>>> Brandon Lo >>>> >>>> UNH InterOperability Laboratory >>>> >>>> 21 Madbury Rd, Suite 100, Durham, NH 03824 >>>> >>>> blo@iol.unh.edu >>>> >>>> www.iol.unh.edu >>> >>> >>> >>> -- >>> >>> Brandon Lo >>> >>> UNH InterOperability Laboratory >>> >>> 21 Madbury Rd, Suite 100, Durham, NH 03824 >>> >>> blo@iol.unh.edu >>> >>> www.iol.unh.edu -- Brandon Lo UNH InterOperability Laboratory 21 Madbury Rd, Suite 100, Durham, NH 03824 blo@iol.unh.edu www.iol.unh.edu