From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0E415A0032 for ; Mon, 18 Jul 2022 12:14:49 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D40B64069F; Mon, 18 Jul 2022 12:14:48 +0200 (CEST) Received: from mail-qt1-f176.google.com (mail-qt1-f176.google.com [209.85.160.176]) by mails.dpdk.org (Postfix) with ESMTP id 434C340041 for ; Mon, 18 Jul 2022 12:14:48 +0200 (CEST) Received: by mail-qt1-f176.google.com with SMTP id w29so7624420qtv.9 for ; Mon, 18 Jul 2022 03:14:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=KAPdHJedeMbheg1XsYQ4+zEQsGKDJycGoxKCLCkXJb8=; b=MLD/xPXWJcbZdRpa3Nu/FyLNDgehdTWBw5SfdZT7U8N8Y3+qdVXBuiu1+UpH82H4Jk y1bdLzND85zSyqof2F4l5k72IuHXiEKvphbMTwjDVITrVwNeYE9MJM6+3jyjW7uHxfvH jSoOLnydqi5/mHlI6TAzN44su+HaHC8N95nrXJvJlMQU5Xd53CAhXViyu0S7PMxWnvRs ABw4/DeGf2nGfjPOSC+IA4b2rLi1i1xmf1mWJeyJ03suyHDAeSvJ7SbimEy4g69wsDrx G/bWqJAXk6NlVQStdlEhSjnim27J7jdKTfwZso0bqsvcPAGkFmI8IYCH4kwWxlUFJpaC iZAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=KAPdHJedeMbheg1XsYQ4+zEQsGKDJycGoxKCLCkXJb8=; b=pqvUHvRnPPlcSgdWZ8EtEYjqnlGHgB3M9V1Nx2ZxN7t+Bkf1KKnXnKofzlbHN0Hyww SuKVcP8Q3F9ZjZ3+IFag6179mT5v/x9VyIFBL08GmYcVkTh/ql52osJisA7pnxhBgLsi edavyU5nu0vl03aQgmu3+Yu+I07g7X1wfSP567Vj4OTIMTK5H70IwxB8A3aWtXfjLf1R MjzIMWFBNB3My4uYJnmiAw0+ypvbKpE6rWZZ2tDRV/p8xdkugOJQlRmhSz3IxaSdyL5Y 2Llx08McqujqQTAdQRGwijj37bk97QJNeuq4+6AmqBLdcSIhNYJfp9T640SsRjOo3SO4 +kFQ== X-Gm-Message-State: AJIora/1BlxL6yZoHKayfWSuIdCNh1vWEXmu6zuqwsKEhog/bgvu7+YH pxJ+n0z0mssIU8wzc0v19UMmw/wHoPAmr+vtXu1i+ChUKLI= X-Google-Smtp-Source: AGRyM1vDIvLA8ju6vp22UqhBiSoFasMxK+7fRY6o1n0eLOFw0i9QcRzi6F4Z/K4YJmzX1AOCazANPZZIvIMYt6YTCl0= X-Received: by 2002:a05:622a:609:b0:31e:f083:7dff with SMTP id z9-20020a05622a060900b0031ef0837dffmr2672296qta.38.1658139287627; Mon, 18 Jul 2022 03:14:47 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Erez Ferber Date: Mon, 18 Jul 2022 13:14:36 +0300 Message-ID: Subject: Re: mlx5: Keeping packets with invalid CRC/FCS To: "Pfau, Johannes (ITIV)" Cc: "users@dpdk.org" Content-Type: multipart/alternative; boundary="0000000000003a2b2205e411a396" X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org --0000000000003a2b2205e411a396 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi, Yes, that should be considered as an expected behaviour. The PMD will reduce 4 bytes from pkt_len when KEEP_CRC is configured. In that case the user should check the CRC after the pkt_data in the mbuf to read its value. 2.Try allowing the NIC to receive packets with bad FCS by enabling this capability using ethtool (general NIC feature, applies to all frameworks) "ethtool -K ens2f0 rx-all on" Regards, Erez On Thu, 14 Jul 2022 at 16:52, Pfau, Johannes (ITIV) wrote: > Hi all, > > We're currently developing a low-cost DAQ system to record high-bandwidth > data from FPGA systems. We're using DPDK with Mellanox Connect-X5 cards, > the > drivers and DPDK installed from the standard RHEL 8 repositories. We > capture > raw ethernet (no L3) on 1:1 links to the FPGA devices to avoid all possib= le > overhead. So far this setup works great, we can handle 100 Gbit/s of > traffic > even on a single core, but we can also distribute packets to multiple cor= es > depending on the ether type if required. > > To save a few more bits in the transmission, we'd like to avoid encoding > packet counters into the data stream. In that case we have to make sure w= e > never miss any packets in recording though, even if the FCS is invalid. > There are two aspects involved, leading to two questions: > > First, we need to store the CRC as well so that we can detect the invalid > packets later on in offline processing. Using RTE_ETH_RX_OFFLOAD_KEEP_CRC > is > working fine, but it first confused me a lot: Both rte_pktmbuf_pkt_len an= d > rte_pktmbuf_data_len still report the length without CRC. If I just read = 4 > bytes more than announced in these functions, I can read the correct CRC. > Is > it intentional that the 4 CRC bytes are not included in these counts? > > Second, and this is a larger issue: We also need to receive packets with > invalid FCS. We don't really have a an idea how to actually inject packet= s > with invalid FCS for testing, but from the documentation I assume the mlx= 5 > driver in default setup would drop invalid packets? There was an mailing > list discussing this for intel NICS > (https://mails.dpdk.org/archives/users/2021-June/005651.html), but I > couldn't find anything for mlx5. Does anybody know if the mlx5 driver als= o > offers an option to keep invalid packets? > > Best regards, > Johannes > > -- > Karlsruhe Institute of Technology (KIT) > Institut f=C3=BCr Technik der Informationsverarbeitung (ITIV) > > M.Sc. Johannes Pfau > Research Associate > > Engesserstr. 5 > Building 30.10, Room 218.1 > 76131 Karlsruhe, Germany > > Phone: +49 721 608-41939 > E-mail: johannes.pfau@kit.edu > > > Registered office: > Kaiserstra=C3=9Fe 12, 76131 Karlsruhe, Germany > > > KIT =E2=80=93 The Research University in the Helmholtz Association > > --0000000000003a2b2205e411a396 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi,

Yes, that should be considered as an expected b= ehaviour.=C2=A0
The PMD will reduce 4 bytes from pkt_len when KEEP_CRC = is configured. In that case the=C2=A0 user should check the CRC after the p= kt_data in the mbuf to read its value.
2.Try allowing the NIC to receive= packets with bad FCS by enabling this capability using ethtool (general NI= C feature, applies to all frameworks)
"ethtool -K ens2f0 rx-all on&= quot;=C2=A0

Regards,
Erez
On Thu, 1= 4 Jul 2022 at 16:52, Pfau, Johannes (ITIV) <johannes.pfau@kit.edu> wrote:
Hi all,

We're currently developing a low-cost DAQ system to record high-bandwid= th
data from FPGA systems. We're using DPDK with Mellanox Connect-X5 cards= , the
drivers and DPDK installed from the standard RHEL 8 repositories. We captur= e
raw ethernet (no L3) on 1:1 links to the FPGA devices to avoid all possible=
overhead. So far this setup works great, we can handle 100 Gbit/s of traffi= c
even on a single core, but we can also distribute packets to multiple cores=
depending on the ether type if required.

To save a few more bits in the transmission, we'd like to avoid encodin= g
packet counters into the data stream. In that case we have to make sure we<= br> never miss any packets in recording though, even if the FCS is invalid.
There are two aspects involved, leading to two questions:

First, we need to store the CRC as well so that we can detect the invalid packets later on in offline processing. Using RTE_ETH_RX_OFFLOAD_KEEP_CRC i= s
working fine, but it first confused me a lot: Both rte_pktmbuf_pkt_len and<= br> rte_pktmbuf_data_len still report the length without CRC. If I just read 4<= br> bytes more than announced in these functions, I can read the correct CRC. I= s
it intentional that the 4 CRC bytes are not included in these counts?

Second, and this is a larger issue: We also need to receive packets with invalid FCS. We don't really have a an idea how to actually inject pack= ets
with invalid FCS for testing, but from the documentation I assume the mlx5<= br> driver in default setup would drop invalid packets? There was an mailing list discussing this for intel NICS
(https://mails.dpdk.org/archives/users/20= 21-June/005651.html), but I
couldn't find anything for mlx5. Does anybody know if the mlx5 driver a= lso
offers an option to keep invalid packets?

Best regards,
Johannes

--
Karlsruhe Institute of Technology (KIT)
Institut f=C3=BCr Technik der Informationsverarbeitung (ITIV)

M.Sc. Johannes Pfau
Research Associate

Engesserstr. 5
Building 30.10, Room 218.1
76131 Karlsruhe, Germany

Phone: +49 721 608-41939
E-mail: johannes= .pfau@kit.edu


Registered office:
Kaiserstra=C3=9Fe 12, 76131 Karlsruhe, Germany


KIT =E2=80=93 The Research University in the Helmholtz Association

--0000000000003a2b2205e411a396--