From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A42E443348; Thu, 16 Nov 2023 21:01:28 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9D8A04027D; Thu, 16 Nov 2023 21:01:28 +0100 (CET) Received: from mail-oi1-f171.google.com (mail-oi1-f171.google.com [209.85.167.171]) by mails.dpdk.org (Postfix) with ESMTP id 9333940150 for ; Thu, 16 Nov 2023 21:01:26 +0100 (CET) Received: by mail-oi1-f171.google.com with SMTP id 5614622812f47-3b40d5ea323so765285b6e.0 for ; Thu, 16 Nov 2023 12:01:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; t=1700164886; x=1700769686; darn=dpdk.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=RaVoW7KzkJy08cEsQbu+JIuU1ADaQFZOMflTOALUh6E=; b=Y2Fg+wRn2jTBB6k6+2vlGsFs4U8H2R/i0oH7XKbayMF6MGfaAgeh5rR13lb+7iMC3V aZ9mQmIrA67gOW8oGl34XljAUtjIM+hWMeac5fTpnIvgtLeP3clg4Xo7ekyjkm68GBxw KeV+xN12Mgfx4Ps3V5sL+C4TzST6ApK2pD6Vo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700164886; x=1700769686; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=RaVoW7KzkJy08cEsQbu+JIuU1ADaQFZOMflTOALUh6E=; b=CS3HxoNsFbCGUDHX5MUrpSpA/r0ix0MFo7Gt9Q3zGWqmexetnbrtnXORk/ui6IRCcW Kw5kuLHv2aVCoK3ooz7jJfXfhHFVFC9Zx+gtKwq1uafdu34KS5Uxayx7kpPaxssaqUBE ThhdvTa8nQLDNtVROASPTfGxbCTxaJgVawBxMOGFYOvY9OANQSonA4UIHMvokkdnf6rk oaOZZGdfHGWanNjHV1aUn1mz1TSkg6brRCx7c84i5AaBe0B1miJ9+tP/uuaUluFMHVtC EBm5F8hpSEoIFVuJ1J2xag8cX01170/Ew8EO3puUed6CRib8weMtPDwig66wwt1A2Soz IkaQ== X-Gm-Message-State: AOJu0YzKNKwMLZz02AewmptIgwozQ1xD3Zzw3AVFLtrW3N3RIs1he7ri BiYGT5n/dYf6/zZqI0iRnZ1BVN06Oq4KD27cfaC/rg== X-Google-Smtp-Source: AGHT+IEnX3OY6XCjRFyVRh2pkl+01wKp9PWO99UzXNK/sDGAX3JTVPU9nUxtiUHKMEgtKlzRqXjaUcy4dAXi5lpDTO4= X-Received: by 2002:a05:6808:112:b0:3b2:e577:7339 with SMTP id b18-20020a056808011200b003b2e5777339mr19555227oie.41.1700164885903; Thu, 16 Nov 2023 12:01:25 -0800 (PST) MIME-Version: 1.0 References: <7788077c-4e0b-f68f-2e09-3994a49ae715@oktetlabs.ru> <1f53aade-73a7-baaf-aecb-2b9a33ab6682@oktetlabs.ru> <4979713b-8e5f-417b-b1af-21f54130eab7@oktetlabs.ru> <8e26b8e6-8d8b-4925-9b30-3fbc5e103e18@oktetlabs.ru> <0cd6a9fb-bc39-4b69-be5d-3470e2374016@oktetlabs.ru> In-Reply-To: <0cd6a9fb-bc39-4b69-be5d-3470e2374016@oktetlabs.ru> From: Adam Hassick Date: Thu, 16 Nov 2023 15:03:35 -0500 Message-ID: Subject: Re: Setting up DPDK PMD Test Suite To: Andrew Rybchenko Cc: Konstantin Ushakov , Patrick Robb , ci@dpdk.org Content-Type: multipart/alternative; boundary="00000000000015916e060a4a7c74" X-BeenThere: ci@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK CI discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ci-bounces@dpdk.org --00000000000015916e060a4a7c74 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi Andrew, If you use copy of dpdk-ethdev-ts has > 398e272495143884274f5a53c6fe0cc16df41052, you don't need to pass --trc-ta= g=3Dpci-8086-1572 > any more since corresponding changeset updates expectations to have the > same for pci-8086-1583. I'll try this for the next run. Sorry, but I've failed to find what's wrong there. That if statement works if using the traditional single-bracket conditional, or it needs to be rewritten as "[[ -z "${test_log}" ]] || [[ ! -r "${test_log}" ]]". The latter is the change I made, but both work. As far as I can see LLDP packets spoil testing results: > > https://ts-factory.io/bublik/v2/log/362398?focusId=3D362760&mode=3DtreeAn= dinfoAndlog&experimental=3Dtrue&lineNumber=3D1_63 > > As far as I can see main prologue disables FW LLDP on Tester > > https://ts-factory.io/bublik/v2/log/362398?focusId=3D362400&mode=3DtreeAn= dinfoAndlog&experimental=3Dtrue&lineNumber=3D1_80 > but I guess it could be still enabled on DUT side and DPDK do not provide > means to disable it as far as I know. I vaguely remember that Intel > provides FW configuration tools which can do it. > It is interesting since DPDK gets unexpected LLDP packets but may be > packets sent by FW go via loopback and visible to PF as well. > Other possible source of LLDP packet is a switch if NICs are connected vi= a > switch. If so, LLDP should be disabled on corresponding switch ports. > > As far as I can see fixing the problem should make results much closer. > However, I already see some differences in behaviour which should be simp= ly > fixed in TRC. For example, X710 gets 9 packets less than configuration > number of Rx descriptors, but XL710 gets 10 packets less. I have the "disable-fw-lldp" private flag set on both of the XL710 ports on the DUT machine. Very strange how there are still LLDP packets appearing in there. These systems are not connected to any switch, so maybe a service on the DUT itself is sending them. I'm not sure how that could be happening though, because I don't have the LLDP daemon installed on either system. Also I see that performance tests are not run because of failed prologue: > > https://ts-factory.io/bublik/v2/log/362398?focusId=3D369564&mode=3DtreeAn= dinfoAndlog&experimental=3Dtrue > I'll investigate it, but I guess the source of difference is that we > always run tests on single interface. Just add -p0 (--cfg=3Diol-dts-xl710= -p0) > to your configuration name. You don't need to change ts-rigs for it since > the suffix is handled by generic code. It simply comments the second > instance and forces take the first interface only into account. Initially > it was introduced to run independent tests on different ports to be able = to > share configuration, but I guess right now it has limitations for some > packages like representors which require entire NIC. I can try that and will see if it works. Thanks, Adam On Wed, Nov 8, 2023 at 2:20=E2=80=AFAM Andrew Rybchenko < andrew.rybchenko@oktetlabs.ru> wrote: > Hi Adam, > > On 11/7/23 23:30, Adam Hassick wrote: > > Hi Andrew, > > The runner machine was missing a dependency for one of the scripts, > "pixz". After installing that, it appears to have worked. I can see the > results listed on the ts-factory Bublik instance. > > > If you use copy of dpdk-ethdev-ts has > 398e272495143884274f5a53c6fe0cc16df41052, you don't need to pass --trc-ta= g=3Dpci-8086-1572 > any more since corresponding changeset updates expectations to have the > same for pci-8086-1583. > > In the latest revision of ts-rigs, there appears to be a syntax error at > line 42 within the script located at > "ts-rigs/scripts/publish_logs/prj/ts-factory/publish", within the if > condition. I fixed it locally to get it to run. > > > Sorry, but I've failed to find what's wrong there. > > Taking a quick look at a comparison against your most recent X710 run, it > looks like we're NOK on around ~400 more test cases. By percentage of > tests, we're 1% off, however, it looks like whole subsets of the test sui= te > that contain low numbers of tests are failing. I wonder if this is due to > differences between the Intel X710 and XL710 or issues in our dev testbed= . > > > As far as I can see LLDP packets spoil testing results: > > https://ts-factory.io/bublik/v2/log/362398?focusId=3D362760&mode=3DtreeAn= dinfoAndlog&experimental=3Dtrue&lineNumber=3D1_63 > > As far as I can see main prologue disables FW LLDP on Tester > > https://ts-factory.io/bublik/v2/log/362398?focusId=3D362400&mode=3DtreeAn= dinfoAndlog&experimental=3Dtrue&lineNumber=3D1_80 > but I guess it could be still enabled on DUT side and DPDK do not provide > means to disable it as far as I know. I vaguely remember that Intel > provides FW configuration tools which can do it. > It is interesting since DPDK gets unexpected LLDP packets but may be > packets sent by FW go via loopback and visible to PF as well. > Other possible source of LLDP packet is a switch if NICs are connected vi= a > switch. If so, LLDP should be disabled on corresponding switch ports. > > As far as I can see fixing the problem should make results much closer. > However, I already see some differences in behaviour which should be simp= ly > fixed in TRC. For example, X710 gets 9 packets less than configuration > number of Rx descriptors, but XL710 gets 10 packets less. > > Also I see that performance tests are not run because of failed prologue: > > https://ts-factory.io/bublik/v2/log/362398?focusId=3D369564&mode=3DtreeAn= dinfoAndlog&experimental=3Dtrue > I'll investigate it, but I guess the source of difference is that we > always run tests on single interface. Just add -p0 (--cfg=3Diol-dts-xl710= -p0) > to your configuration name. You don't need to change ts-rigs for it since > the suffix is handled by generic code. It simply comments the second > instance and forces take the first interface only into account. Initially > it was introduced to run independent tests on different ports to be able = to > share configuration, but I guess right now it has limitations for some > packages like representors which require entire NIC. > > Regards, > Andrew. > > Thanks, > Adam > > > (dropped history, to keep mail size small) > --00000000000015916e060a4a7c74 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi Andrew,

If you use copy of dpdk-ethdev-ts has 398e272495143884274f5a53c6fe0cc16df41052, you don't need to pass --trc-tag=3Dpci-8086-1572 any more since corresponding changeset updates expectations to have the same for pci-8086-1583.

I'll try this for the next run.

Sorry, but I've failed to find what&#= 39;s wrong there.

That if statement works i= f using the traditional single-bracket conditional, or it needs to be rewri= tten as "[[ -z "${test_log}" ]] || [[ ! -r "${test_log}= " ]]". The latter is the change I made, but both work.

As far as I can see LLDP pac= kets spoil testing results:
https://ts-factory.io/bublik/= v2/log/362398?focusId=3D362760&mode=3DtreeAndinfoAndlog&experimenta= l=3Dtrue&lineNumber=3D1_63

As far as I can see main prologue disables FW LLDP on Tester
https://ts-factory.io/bublik/v2/log/362398?focusId=3D362400&mod= e=3DtreeAndinfoAndlog&experimental=3Dtrue&lineNumber=3D1_80
but I guess it could be still enabled on DUT side and DPDK do not provide means to disable it as far as I know. I vaguely remember that Intel provides FW configuration tools which can do it.
It is interesting since DPDK gets unexpected LLDP packets but may be packets sent by FW go via loopback and visible to PF as well.
Other = possible source of LLDP packet is a switch if NICs are connected via switch. If so, LLDP should be disabled on corresponding switch ports.

As far as I can see fixing the problem should make results much closer. However, I already see some differences in behaviour which should be simply fixed in TRC. For example, X710 gets 9 packets less than configuration number of Rx descriptors, but XL710 gets 10 packets less.

I have the "disable-= fw-lldp" private flag set on both of the XL710 ports on the DUT machin= e. Very strange how there are still LLDP packets appearing in there.
These systems are not connected to any switch, so maybe a service o= n the DUT itself is sending them. I'm not sure how that could be happen= ing though, because I don't have the LLDP daemon installed on either sy= stem.

Also I see t= hat performance tests are not run because of failed prologue:
https://ts-factory.io/bublik/v2/log/362398?focusId=3D369564&= ;mode=3DtreeAndinfoAndlog&experimental=3Dtrue
I'll investigate it, but I guess the source of difference is that w= e always run tests on single interface. Just add -p0 (--cfg=3Diol-d= ts-xl710-p0) to your configuration name. You don't need to change ts-rigs for it since the suffix is handled by generic code. It simply comments the second instance and forces take the first interface only into account. Initially it was introduced to run independent tests on different ports to be able to share configuration, but I guess right now it has limitations for some packages like representors which require entire NIC.

I can try that and = will see if it works.

Thanks,
Adam
=20 =20

On Wed, Nov 8, 2023 at 2:20=E2=80=AFAM Andrew Rybchenko &l= t;andrew= .rybchenko@oktetlabs.ru> wrote:
=20 =20 =20
Hi Adam,

On 11/7/23 23:30, Adam Hassick wrote:
=20
Hi Andrew,

The runner machine was missing a dependency for one of the scripts, "pixz". After installing that, it ap= pears to have worked. I can see the results listed on the ts-factory Bublik instance.

If you use copy of dpdk-ethdev-ts has 398e272495143884274f5a53c6fe0cc16df41052, you don't need to pass --trc-tag=3Dpci-8086-1572 any more since corresponding changeset updates expectations to have the same for pci-8086-1583.

In the latest revision of ts-rigs, there appears to be a syntax error at line 42 within the script located at "ts-rigs/scripts/publish_logs/prj/ts-factory/publish&quo= t;, within the if condition. I fixed it locally to get it to run.

Sorry, but I've failed to find what's wrong there.

Taking a quick look at a comparison against your most recent X710 run, it looks like we're NOK on around ~400 mor= e test cases. By percentage of tests, we're 1% off, however, it looks like whole subsets of the test suite that contain low numbers of tests are failing. I wonder if this is due to differences between the Intel X710 and XL710 or issues in our dev testbed.

As far as I can see LLDP packets spoil testing results:
https://ts-factory.io/bublik/v2/log/362398?focusId=3D362760&= amp;mode=3DtreeAndinfoAndlog&experimental=3Dtrue&lineNumber=3D1_63<= /a>

As far as I can see main prologue disables FW LLDP on Tester
https://ts-factory.io/bublik/v2/log/362398?focusId=3D362400&= amp;mode=3DtreeAndinfoAndlog&experimental=3Dtrue&lineNumber=3D1_80<= /a>
but I guess it could be still enabled on DUT side and DPDK do not provide means to disable it as far as I know. I vaguely remember that Intel provides FW configuration tools which can do it.
It is interesting since DPDK gets unexpected LLDP packets but may be packets sent by FW go via loopback and visible to PF as well.
Other possible source of LLDP packet is a switch if NICs are connected via switch. If so, LLDP should be disabled on corresponding switch ports.

As far as I can see fixing the problem should make results much closer. However, I already see some differences in behaviour which should be simply fixed in TRC. For example, X710 gets 9 packets less than configuration number of Rx descriptors, but XL710 gets 10 packets less.

Also I see that performance tests are not run because of failed prologue:
https:/= /ts-factory.io/bublik/v2/log/362398?focusId=3D369564&mode=3DtreeAndinfo= Andlog&experimental=3Dtrue
I'll investigate it, but I guess the source of difference is that w= e always run tests on single interface. Just add -p0 (--cfg=3Diol-d= ts-xl710-p0) to your configuration name. You don't need to change ts-rigs for it since the suffix is handled by generic code. It simply comments the second instance and forces take the first interface only into account. Initially it was introduced to run independent tests on different ports to be able to share configuration, but I guess right now it has limitations for some packages like representors which require entire NIC.

Regards,
Andrew.

Thanks,
Adam

(dropped history, to keep mail size small)
--00000000000015916e060a4a7c74--