Hi Adam, On 11/16/23 23:03, Adam Hassick wrote: > Hi Andrew, > > If you use copy of dpdk-ethdev-ts has > 398e272495143884274f5a53c6fe0cc16df41052, you don't need to pass > --trc-tag=pci-8086-1572 any more since corresponding changeset > updates expectations to have the same for pci-8086-1583. > > > I'll try this for the next run. > > Sorry, but I've failed to find what's wrong there. > > > That if statement works if using the traditional single-bracket > conditional, or it needs to be rewritten as "[[ -z "${test_log}" ]] || > [[ ! -r "${test_log}" ]]". The latter is the change I made, but both work. Thanks a lot. Hopefully fixed. > > As far as I can see LLDP packets spoil testing results: > https://ts-factory.io/bublik/v2/log/362398?focusId=362760&mode=treeAndinfoAndlog&experimental=true&lineNumber=1_63 > > > As far as I can see main prologue disables FW LLDP on Tester > https://ts-factory.io/bublik/v2/log/362398?focusId=362400&mode=treeAndinfoAndlog&experimental=true&lineNumber=1_80 > > but I guess it could be still enabled on DUT side and DPDK do not > provide means to disable it as far as I know. I vaguely remember > that Intel provides FW configuration tools which can do it. > It is interesting since DPDK gets unexpected LLDP packets but may > be packets sent by FW go via loopback and visible to PF as well. > Other possible source of LLDP packet is a switch if NICs are > connected via switch. If so, LLDP should be disabled on > corresponding switch ports. > > As far as I can see fixing the problem should make results much > closer. However, I already see some differences in behaviour which > should be simply fixed in TRC. For example, X710 gets 9 packets > less than configuration number of Rx descriptors, but XL710 gets > 10 packets less. > > > I have the "disable-fw-lldp" private flag set on both of the XL710 > ports on the DUT machine. Very strange how there are still LLDP > packets appearing in there. Me too. Corresponding packet has source MAC from Peer/Tester machine NIC. It is really strange since prologue disabled LLDP there as well. I'll try to play with it locally more, but have no good ideas in fact. > These systems are not connected to any switch, so maybe a service on > the DUT itself is sending them. I'm not sure how that could be > happening though, because I don't have the LLDP daemon installed on > either system. > > Also I see that performance tests are not run because of failed > prologue: > https://ts-factory.io/bublik/v2/log/362398?focusId=369564&mode=treeAndinfoAndlog&experimental=true > > I'll investigate it, but I guess the source of difference is that > we always run tests on single interface. Just add -p0 > (--cfg=iol-dts-xl710-p0) to your configuration name. You don't > need to change ts-rigs for it since the suffix is handled by > generic code. It simply comments the second instance and forces > take the first interface only into account. Initially it was > introduced to run independent tests on different ports to be able > to share configuration, but I guess right now it has limitations > for some packages like representors which require entire NIC. > > > I can try that and will see if it works. This problem is fixed in fresh TE and dpdk-ethdev-ts published on GitHub. Regards, Andrew. > > Thanks, > Adam > > On Wed, Nov 8, 2023 at 2:20 AM Andrew Rybchenko > wrote: > > Hi Adam, > > On 11/7/23 23:30, Adam Hassick wrote: >> Hi Andrew, >> >> The runner machine was missing a dependency for one of the >> scripts, "pixz". After installing that, it appears to have >> worked. I can see the results listed on the ts-factory Bublik >> instance. > > If you use copy of dpdk-ethdev-ts has > 398e272495143884274f5a53c6fe0cc16df41052, you don't need to pass > --trc-tag=pci-8086-1572 any more since corresponding changeset > updates expectations to have the same for pci-8086-1583. > >> In the latest revision of ts-rigs, there appears to be a syntax >> error at line 42 within the script located at >> "ts-rigs/scripts/publish_logs/prj/ts-factory/publish", within the >> if condition. I fixed it locally to get it to run. > > Sorry, but I've failed to find what's wrong there. > >> Taking a quick look at a comparison against your most recent X710 >> run, it looks like we're NOK on around ~400 more test cases. By >> percentage of tests, we're 1% off, however, it looks like whole >> subsets of the test suite that contain low numbers of tests are >> failing. I wonder if this is due to differences between the Intel >> X710 and XL710 or issues in our dev testbed. > > As far as I can see LLDP packets spoil testing results: > https://ts-factory.io/bublik/v2/log/362398?focusId=362760&mode=treeAndinfoAndlog&experimental=true&lineNumber=1_63 > > > As far as I can see main prologue disables FW LLDP on Tester > https://ts-factory.io/bublik/v2/log/362398?focusId=362400&mode=treeAndinfoAndlog&experimental=true&lineNumber=1_80 > > but I guess it could be still enabled on DUT side and DPDK do not > provide means to disable it as far as I know. I vaguely remember > that Intel provides FW configuration tools which can do it. > It is interesting since DPDK gets unexpected LLDP packets but may > be packets sent by FW go via loopback and visible to PF as well. > Other possible source of LLDP packet is a switch if NICs are > connected via switch. If so, LLDP should be disabled on > corresponding switch ports. > > As far as I can see fixing the problem should make results much > closer. However, I already see some differences in behaviour which > should be simply fixed in TRC. For example, X710 gets 9 packets > less than configuration number of Rx descriptors, but XL710 gets > 10 packets less. > > Also I see that performance tests are not run because of failed > prologue: > https://ts-factory.io/bublik/v2/log/362398?focusId=369564&mode=treeAndinfoAndlog&experimental=true > > I'll investigate it, but I guess the source of difference is that > we always run tests on single interface. Just add -p0 > (--cfg=iol-dts-xl710-p0) to your configuration name. You don't > need to change ts-rigs for it since the suffix is handled by > generic code. It simply comments the second instance and forces > take the first interface only into account. Initially it was > introduced to run independent tests on different ports to be able > to share configuration, but I guess right now it has limitations > for some packages like representors which require entire NIC. > > Regards, > Andrew. > >> Thanks, >> Adam > > (dropped history, to keep mail size small) >