From: Bill Michalowski <bmichalo@redhat.com>
To: Lance Richardson <lance.richardson@broadcom.com>
Cc: dev@dpdk.org, users@dpdk.org
Subject: Re: [dpdk-users] [dpdk-dev] testpmd / SR-IOV RX packets, but TX-errors
Date: Thu, 17 Oct 2019 08:33:18 -0400 [thread overview]
Message-ID: <CAO8xzOsL0rTrawV=E20UnbEiO=C0b2br1Aobn7bOuTbRYDXQWw@mail.gmail.com> (raw)
In-Reply-To: <CAO8xzOsSTdZCEkOEGXX3QEw8L6uO3YUYVw+zZtA4da73hGBP1A@mail.gmail.com>
Solved... had to disable spoof checking:
ip link set p7p1 vf 0 spoofchk off
ip link set p7p2 vf 0 spoofchk off
Thank you!
On Tue, Oct 15, 2019 at 11:03 AM Bill Michalowski <bmichalo@redhat.com>
wrote:
> Thank you for the suggestion Lance, but that didn't correct the problem.
>
> That being said, I would think that testpmd in "io" mode would just
> forward the packets in the SR-IOV case just as it successfully does with
> testpmd across the PFs (non SR-IOV) - but I could be wrong.
>
> On Tue, Oct 15, 2019 at 10:26 AM Lance Richardson <
> lance.richardson@broadcom.com> wrote:
>
>> Hi Bill,
>>
>> This is just a guess, but perhaps the transmit drops are due to MAC
>> anti-spoofing checks?
>>
>> You could probably prove/disprove by making the packet source MAC
>> match the transmit VF's MAC.
>>
>> Lance
>>
>>
>> On Tue, Oct 15, 2019 at 10:01 AM Bill Michalowski <bmichalo@redhat.com>
>> wrote:
>> >
>> > I'm trying run testpmd with SR-IOV, specifically across two VFs. I see
>> > packets being received successfully by the application but they are not
>> > being transmitted. They are dropped with the 'TX-errors' counters
>> > incrementing.
>> >
>> > However, if I generate the same traffic and the DUT isn't programmed
>> with
>> > SR-IOV, but with testpmd just across PFs, the packets are transmitted
>> just
>> > fine and I don't have any TX-errors. I'm hoping someone can help me
>> figure
>> > out why the TX packets are being dropped in the SR-IOV case.
>> >
>> > testpmd is running in the default "io" mode.
>> >
>> > The DUT is a two port Intel xxv710 25G card. To simplify things, I'm
>> not
>> > actually using a guest, just testpmd across VFs.
>> >
>> > *# rpm -qa | grep dpdk*
>> > dpdk-18.11.2-1.el7.x86_64
>> >
>> > *# uname -r*
>> > 3.10.0-1062.1.2.el7.x86_64
>> >
>> > *# cat /etc/redhat-release *
>> > Red Hat Enterprise Linux Server release 7.7 (Maipo)
>> >
>> > PFs:
>> >
>> > #* ethtool -i p7p1*
>> > driver: i40e
>> > version: 2.10.19.30
>> > firmware-version: 6.80 0x80003d17 18.8.9
>> > expansion-rom-version:
>> > bus-info: 0000:86:00.0
>> >
>> > *# ethtool -i p7p2*
>> > driver: i40e
>> > version: 2.10.19.30
>> > firmware-version: 6.80 0x80003d17 18.8.9
>> > expansion-rom-version:
>> > bus-info: 0000:86:00.1
>> >
>> > Procedure:
>> >
>> > Create VFs:
>> >
>> > *#echo 1 > /sys/class/net/p7p1/device/sriov_numvfs*
>> >
>> > *#echo 1 > /sys/class/net/p7p2/device/sriov_numvfs*
>> >
>> > *# ip l*
>> > 4: p7p1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP
>> mode
>> > DEFAULT group default qlen 1000
>> > link/ether 3c:fd:fe:b9:c8:a0 brd ff:ff:ff:ff:ff:ff
>> > vf 0 MAC 00:00:00:00:00:00, spoof checking on, link-state auto,
>> trust
>> > off
>> > 5: p7p2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP
>> mode
>> > DEFAULT group default qlen 1000
>> > link/ether 3c:fd:fe:b9:c8:a1 brd ff:ff:ff:ff:ff:ff
>> > vf 0 MAC 00:00:00:00:00:00, spoof checking on, link-state auto,
>> trust
>> > off
>> > 13: p7p1_0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
>> > DEFAULT group default qlen 1000
>> > link/ether 6a:ff:a3:55:23:4e brd ff:ff:ff:ff:ff:ff
>> > 14: p7p2_0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
>> > DEFAULT group default qlen 1000
>> > link/ether da:a1:1c:c4:6c:e7 brd ff:ff:ff:ff:ff:ff
>> >
>> > *# ethtool -i p7p1_0*
>> > driver: iavf
>> > version: 3.7.53
>> > firmware-version: N/A
>> > expansion-rom-version:
>> > bus-info: 0000:86:02.0
>> >
>> > *# ethtool -i p7p2_0*
>> > driver: iavf
>> > version: 3.7.53
>> > firmware-version: N/A
>> > expansion-rom-version:
>> > bus-info: 0000:86:0a.0
>> >
>> > Set MAC addresses:
>> >
>> > *#ip link set p7p1 vf 0 mac a0:b0:c0:d0:e0:f0*
>> > *#ip link set p7p2 vf 0 mac 66:55:44:33:22:11*
>> >
>> > Set trust mode:
>> >
>> >
>> >
>> > *#ip link set p7p1 vf 0 trust on#ip link set p7p2 vf 0 trust on*
>> >
>> > Unbind iavf from VFs and bind vfio-pci to them:
>> >
>> > *# modprobe vfio-pci*
>> > *# dpdk-devbind -u 0000:86:02.0 0000:86:0a.0*
>> > *# dpdk-devbind -b vfio-pci 0000:86:02.0 0000:86:0a.0*
>> > *# dpdk-devbind -s*
>> >
>> > Network devices using DPDK-compatible driver
>> > ============================================
>> > 0000:86:02.0 'Ethernet Virtual Function 700 Series 154c' drv=vfio-pci
>> > unused=iavf
>> > 0000:86:0a.0 'Ethernet Virtual Function 700 Series 154c' drv=vfio-pci
>> > unused=iavf
>> >
>> > Start testpmd:
>> >
>> > #*/usr/bin/testpmd -l 1,3,35 --socket-mem 1024,1024 -n 4 --proc-type
>> auto
>> > --file-prefix testpmd0 -w 0000:86:02.0 -w 0000:86:0a.0 -- --nb-cores=2
>> > --nb-ports=2 --portmask=3 --auto-start --rxq=1 --txq=1 --rxd=2048
>> > --txd=2048 -i*
>> >
>> > Starting traffic at the far end, packets are RX successfully on the DUT
>> but
>> > not TX, with many TX-errors:
>> >
>> > *testpmd> show port stats all*
>> >
>> > ######################## NIC statistics for port 0
>> > ########################
>> > RX-packets: 144132155 RX-missed: 0 RX-bytes: 9224457088
>> > RX-errors: 0
>> > RX-nombuf: 0
>> > TX-packets: 0 TX-errors: 137664400 TX-bytes: 0
>> >
>> > Throughput (since last show)
>> > Rx-pps: 19000875
>> > Tx-pps: 0
>> >
>> >
>> ############################################################################
>> >
>> > ######################## NIC statistics for port 1
>> > ########################
>> > RX-packets: 144324307 RX-missed: 0 RX-bytes: 9236754880
>> > RX-errors: 0
>> > RX-nombuf: 0
>> > TX-packets: 0 TX-errors: 137873118 TX-bytes: 0
>> >
>> > Throughput (since last show)
>> > Rx-pps: 19000772
>> > Tx-pps: 0
>> >
>> >
>> ############################################################################
>> >
>> > Thanks for any help.
>> >
>> > - Bill
>>
>
prev parent reply other threads:[~2019-10-17 12:33 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-10-15 14:01 [dpdk-users] " Bill Michalowski
2019-10-15 14:26 ` [dpdk-users] [dpdk-dev] " Lance Richardson
2019-10-15 15:03 ` Bill Michalowski
2019-10-17 12:33 ` Bill Michalowski [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAO8xzOsL0rTrawV=E20UnbEiO=C0b2br1Aobn7bOuTbRYDXQWw@mail.gmail.com' \
--to=bmichalo@redhat.com \
--cc=dev@dpdk.org \
--cc=lance.richardson@broadcom.com \
--cc=users@dpdk.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).