mlx5: Keeping packets with invalid CRC/FCS

mlx5: Keeping packets with invalid CRC/FCS

[Pfau, Johannes (ITIV)]

[-- Attachment #1: Type: text/plain, Size: 2225 bytes --]

Hi all,

We're currently developing a low-cost DAQ system to record high-bandwidth
data from FPGA systems. We're using DPDK with Mellanox Connect-X5 cards, the
drivers and DPDK installed from the standard RHEL 8 repositories. We capture
raw ethernet (no L3) on 1:1 links to the FPGA devices to avoid all possible
overhead. So far this setup works great, we can handle 100 Gbit/s of traffic
even on a single core, but we can also distribute packets to multiple cores
depending on the ether type if required.

To save a few more bits in the transmission, we'd like to avoid encoding
packet counters into the data stream. In that case we have to make sure we
never miss any packets in recording though, even if the FCS is invalid.
There are two aspects involved, leading to two questions:

First, we need to store the CRC as well so that we can detect the invalid
packets later on in offline processing. Using RTE_ETH_RX_OFFLOAD_KEEP_CRC is
working fine, but it first confused me a lot: Both rte_pktmbuf_pkt_len and
rte_pktmbuf_data_len still report the length without CRC. If I just read 4
bytes more than announced in these functions, I can read the correct CRC. Is
it intentional that the 4 CRC bytes are not included in these counts?

Second, and this is a larger issue: We also need to receive packets with
invalid FCS. We don't really have a an idea how to actually inject packets
with invalid FCS for testing, but from the documentation I assume the mlx5
driver in default setup would drop invalid packets? There was an mailing
list discussing this for intel NICS
(https://mails.dpdk.org/archives/users/2021-June/005651.html), but I
couldn't find anything for mlx5. Does anybody know if the mlx5 driver also
offers an option to keep invalid packets?

Best regards,
Johannes

-- 
Karlsruhe Institute of Technology (KIT)
Institut für Technik der Informationsverarbeitung (ITIV)

M.Sc. Johannes Pfau
Research Associate

Engesserstr. 5
Building 30.10, Room 218.1
76131 Karlsruhe, Germany

Phone: +49 721 608-41939
E-mail: johannes.pfau@kit.edu 


Registered office:
Kaiserstraße 12, 76131 Karlsruhe, Germany


KIT – The Research University in the Helmholtz Association


[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 6301 bytes --]

Re: mlx5: Keeping packets with invalid CRC/FCS

[Erez Ferber]

[-- Attachment #1: Type: text/plain, Size: 2859 bytes --]

Hi,

Yes, that should be considered as an expected behaviour.
The PMD will reduce 4 bytes from pkt_len when KEEP_CRC is configured. In
that case the  user should check the CRC after the pkt_data in the mbuf to
read its value.
2.Try allowing the NIC to receive packets with bad FCS by enabling this
capability using ethtool (general NIC feature, applies to all frameworks)
"ethtool -K ens2f0 rx-all on"

Regards,
Erez

On Thu, 14 Jul 2022 at 16:52, Pfau, Johannes (ITIV) <johannes.pfau@kit.edu>
wrote:

> Hi all,
>
> We're currently developing a low-cost DAQ system to record high-bandwidth
> data from FPGA systems. We're using DPDK with Mellanox Connect-X5 cards,
> the
> drivers and DPDK installed from the standard RHEL 8 repositories. We
> capture
> raw ethernet (no L3) on 1:1 links to the FPGA devices to avoid all possible
> overhead. So far this setup works great, we can handle 100 Gbit/s of
> traffic
> even on a single core, but we can also distribute packets to multiple cores
> depending on the ether type if required.
>
> To save a few more bits in the transmission, we'd like to avoid encoding
> packet counters into the data stream. In that case we have to make sure we
> never miss any packets in recording though, even if the FCS is invalid.
> There are two aspects involved, leading to two questions:
>
> First, we need to store the CRC as well so that we can detect the invalid
> packets later on in offline processing. Using RTE_ETH_RX_OFFLOAD_KEEP_CRC
> is
> working fine, but it first confused me a lot: Both rte_pktmbuf_pkt_len and
> rte_pktmbuf_data_len still report the length without CRC. If I just read 4
> bytes more than announced in these functions, I can read the correct CRC.
> Is
> it intentional that the 4 CRC bytes are not included in these counts?
>
> Second, and this is a larger issue: We also need to receive packets with
> invalid FCS. We don't really have a an idea how to actually inject packets
> with invalid FCS for testing, but from the documentation I assume the mlx5
> driver in default setup would drop invalid packets? There was an mailing
> list discussing this for intel NICS
> (https://mails.dpdk.org/archives/users/2021-June/005651.html), but I
> couldn't find anything for mlx5. Does anybody know if the mlx5 driver also
> offers an option to keep invalid packets?
>
> Best regards,
> Johannes
>
> --
> Karlsruhe Institute of Technology (KIT)
> Institut für Technik der Informationsverarbeitung (ITIV)
>
> M.Sc. Johannes Pfau
> Research Associate
>
> Engesserstr. 5
> Building 30.10, Room 218.1
> 76131 Karlsruhe, Germany
>
> Phone: +49 721 608-41939
> E-mail: johannes.pfau@kit.edu
>
>
> Registered office:
> Kaiserstraße 12, 76131 Karlsruhe, Germany
>
>
> KIT – The Research University in the Helmholtz Association
>
>

[-- Attachment #2: Type: text/html, Size: 3489 bytes --]