From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B8008A04A9; Wed, 26 Jan 2022 18:40:00 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 7416242747; Wed, 26 Jan 2022 18:39:59 +0100 (CET) Received: from NAM04-BN8-obe.outbound.protection.outlook.com (mail-bn8nam08on2045.outbound.protection.outlook.com [40.107.100.45]) by mails.dpdk.org (Postfix) with ESMTP id DA48142738; Wed, 26 Jan 2022 18:39:57 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ENBXlySNMQmvyi52PJNl7MbXd4hUONErJky6rPS05MwHC7k9h77c0I9QZRtY2xMdHBG5P8i0+g2BLaq+OWZHmeKE0h0hEnrtn/1Ptg/SSB/Ui6y8pGLS32Js5bf1NuAhZUf5JmeEQ0uWsEG4R0p2bitrQ2dJXhrMqBexj6LEYxSJfrvs9RV7BsY4NjnlVzo5Aw3xFdFIyYH9Qs8X9RqPC4Jqy+GfRXoUBpIvNe2OtAnevt9UZyYT2WPZLTef2u0kA3/paM37wl3NEdUsVma73g4LGLblI2L5/F2WZBceYZyOeOUoWP/dm2ExkMHkqXwaq49WuCdE1tCKBRZd5NsKvw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=qqvfOo3VQDnldGJKhWt2+yRstIHKl8hBqhsRFX4XRBo=; b=kZB4Bm9x6isNPt3yc33yUVyjKBvP6NFru+gCvEt69xeJM/q668Y13abf06ZrjtV+ntN/8vtmAM+dFQh2/XJgyWfT0fv11IshYG7jO/9lBVwrGGbby7ZjVlI2LU4Eibg1E/lLApRNM8AGRBSqlbI1u2HxKzzdaINDTYeAfIJy96oCDokGfP828IM4X5TyFK7Xb1ZLGF6l77HekQyrTmYU9x9cAkG4hFxMCN1u4Wsj3Xbgr7CtiNDub0z09Y6uM9bUu49EAleR7vQUt+CB9sOtczhJf2wS7+QLxYfdvnm/DC3SIV+rAUxaQZ5gKk+T+ES+qcnP9B9FIb9AkS3VxNUYgQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=qqvfOo3VQDnldGJKhWt2+yRstIHKl8hBqhsRFX4XRBo=; b=PWgqZlZ+N8+B8+y/9Q+D/bBxumQmHsI5SU3T1SaRMcwlPtqKu6QtDLRpgaMBCRNLestgnMqn3FStEsVqxpY14HTxw6uwINQpal7OqoJieTg1/GPKBhKC89AdmwfvtEQmsEBgZpI1nO0TYD3qirU3MbSKWWxy52juN71dX8QVJLmMlBA+uvC6ahzMHnCm2K1tmqUizNet1nnjUxsGW9tUF/O5srMEa46NROjzRy7Cr/nTJMpNV3lVe8/YQo2BMP3hdy//6MdqcqyQgfcebdhyPZyVV0WYrOr+5FpBuLtR5dZg8nHeONX3Mqo7yLYw1NGtadybezu+NqW2z+YO6SeKog== Received: from DM5PR1201MB2555.namprd12.prod.outlook.com (2603:10b6:3:ea::14) by DM6PR12MB4985.namprd12.prod.outlook.com (2603:10b6:5:16d::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4909.12; Wed, 26 Jan 2022 17:39:55 +0000 Received: from DM5PR1201MB2555.namprd12.prod.outlook.com ([fe80::e4c8:ad94:bd2f:3fdd]) by DM5PR1201MB2555.namprd12.prod.outlook.com ([fe80::e4c8:ad94:bd2f:3fdd%5]) with mapi id 15.20.4909.019; Wed, 26 Jan 2022 17:39:55 +0000 From: Asaf Penso To: "bugzilla@dpdk.org" , "dev@dpdk.org" Subject: Re: [Bug 924] Mellanox ConnectX6 DPDK dpdk-testpmd Receive error len error checksum UDP packet performance is very low! Thread-Topic: [Bug 924] Mellanox ConnectX6 DPDK dpdk-testpmd Receive error len error checksum UDP packet performance is very low! Thread-Index: AQHYDdxdSn4HPyR9C0CeaKFZ1pcBVax1m30o Date: Wed, 26 Jan 2022 17:39:55 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: b3bf9352-02aa-4308-ea1d-08d9e0f2ddb5 x-ms-traffictypediagnostic: DM6PR12MB4985:EE_ x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:1002; x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: dRn/uTMwFX8pvMNz+Vw4YWmvdrbR7SOlLBqWSarXCPg/4ZZQvFCsIJX+7FXqFNHfpR/t/lNUhW6rvKp1AElUsuUoCRkUndxSrnrioAOAWLISE3TBVdk0BnOSL1mwpGY6QE5QeQV4Un/I53oEzPMav2RIZYchNVJLgbdAG8IqlRZ7/LQrgPy7GTEiHhZEfkFuYTl0CPBcfsZbqjBZWYla3DSL5zrK3Ah4QvmIGJjOlt+qEQJ4nBUQ+gyP3ZAStsafzIRbQuR2LEqHHZlkx08DczCZ2xD4Wnar7zGf2lkMGagqGgmVrOtiTAWNDH91wHjzy9gcpemIYlFXmDecV2elZLXE7iucugKYAWj1xlyFMJSDGnFeUp6EFtWhpU3wSSd43nTMlRVfX1Z6HtTwLsH7V87XK2OAOBdkwcJ9/I6EMSnNLtxJZq2zck+0N+tnl/ay/hU7uZ1ksTHMERcMlMoNJiuWjU2VfeuuL+qSsqp7vcR+2YKpy2S+/SHjUwZIvhAHmtxqfl89WHK5sweNcNbkwtBNHxWgHtQMtgnzv6Lml9SsV/aAsEZWq1GEpOO0DmFf4T+Ko2ULEfFVPg6BE0lvedzUVdl3ShGgu8mwROyy52i4PVK8SKrEAewee2sMGfujvHDtnhImiLN+uUtf/1AHEmpW5cW4w4+ZX71MzOb54tHXUhvazUj57fhVwnywxZjEW1KfAsOXOKUCXbOWwKE9a0I9+F1kKBnWtox3GHDSYIvvd5IpmVhufU/JdZ2cMb64Iftsb1gauVoh6/B5hSpg+7LJHEa7ubeX6ZyeWBR8XZQ= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM5PR1201MB2555.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(366004)(83380400001)(5660300002)(8936002)(38100700002)(6506007)(52536014)(166002)(86362001)(122000001)(53546011)(7696005)(316002)(33656002)(9686003)(8676002)(450100002)(2906002)(26005)(91956017)(66946007)(71200400001)(508600001)(186003)(55016003)(64756008)(66476007)(66446008)(66556008)(76116006)(110136005)(38070700005)(966005); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?Vcq2lP+0OVOqNGZx/hq2ibaIMqELR2TJ+m+rSz/FUc0S7y3KCfrn36n4fO1j?= =?us-ascii?Q?Ba2ETPLx7HEX378wW0jxY/1Uv3vr8V7zPdcOfTvGDHqix53JqBi7yjmCvwnO?= =?us-ascii?Q?yVY0qdRtD8kekd3/wvhYKp9n/z/XauFvl+yZLNOkwQSVSXIbrRrboqjtZzgC?= =?us-ascii?Q?dJcrSTxORw2zmb00Ve9WGwIZMp84QbCOxlC5/dczqWSyckLEB6lZyQYNmgbE?= =?us-ascii?Q?8LdJvxrNX13JCW0qYnbanclEoRAQMju7hgzjV2uay7S8rYF63y8t/AW/3LIM?= =?us-ascii?Q?AxUHTLG0b332DjcScCc7DCiCkH+kE9rWbsc+Go0aA6X61de4hQyi0ugsNljU?= =?us-ascii?Q?atY+sYqpQSby1BjcrKg+8eFDZSUO7Q5G78m93J8ylTdynEF4/O0DT0L0u6MI?= =?us-ascii?Q?RBUHDxcLbrVaZ+UuzGzzAPV9udP70lhXZLRqB1nCwbFfyPEGsaWsYZPqeoln?= =?us-ascii?Q?mdd5bMuC/6T4q/LLW815apoXDnBM8NzMSfaBgXPDrCV3Wh/woFGYHR0F5KTx?= =?us-ascii?Q?/u2v+NqEhi/sfFfCHBIW50UcrzD7YOS40GMw1kOkCK3GZrUfCpfN/67LvllN?= =?us-ascii?Q?psAUM8x0szGgkFXhN3EfYTnHr5+78xXyozqHpjJ6Y83MZHbT2mmkXWMHtSoz?= =?us-ascii?Q?eIgq7nbaQvAsSDsTpV6SiD4Z6/RZhMplXgg1Bt/96zDGAS2ddxrvxrrKbDrO?= =?us-ascii?Q?Xz6+stWtEpbWBQHeHcFxCwp/RtVaoPWdCtxMdrLeCXipQc+623p9QbQIkjml?= =?us-ascii?Q?5Jyf/DpPGen/1qOZ8djSKLYoJr7dPoR6NdQ/0uI1aqvciRraJnOUzJGp1/7e?= =?us-ascii?Q?IRHQv4MdqJ82c7T7XJ4gV3/Jg2bxUiieLpzzP0MPeyyNiSKxV5fF4ALmxegr?= =?us-ascii?Q?30vcKjLiKjLqG5LbrWN6JB6ytkmEAbpqvHQm02uSSwDV6/bOcW99roxq+YCS?= =?us-ascii?Q?vgM/dOet0lb0Oq6XKAseLjqor8Og3iZqHlKe3YMibwZeu7M/QzszUimi6HTN?= =?us-ascii?Q?xOawrXKV1CgT5ib2sEd06Qryl0W9IHgDyO8G4H5U3OMSw7mE3enSAVqXn/EC?= =?us-ascii?Q?hn3j577ZFYBYKhAp/d8kpg16cMrbeQbBQ5Vi3g3568vh0J4mfVnAqf9enKmf?= =?us-ascii?Q?PyBVQJeG2Epc23i0rrlJg9+9v2pW53tWQp81fE7bQcoYAWEysu8QVJeHLbF5?= =?us-ascii?Q?x3RQ9C8Qb09aU6AXNh+yU7/v4HQEVBzMhM8VTfhau3F8dsU0NxE0fE+VQ+kc?= =?us-ascii?Q?uSNq3pei963phC/+5pRQNWUwTx9TwgMur+/mOqRa5BJyeMqShaI9Dvv/CaqM?= =?us-ascii?Q?NmrS2ntBYe4e/5DqoSk2u6zTT+BhypOZ1dRjKmMpTSnXBAJu9q1yGBiw0CqK?= =?us-ascii?Q?lbOXnJcRrvN65KFE2dam+RlfxAEenrp6rIDCCHZxylUV7k4VNE7vikxuhLQw?= =?us-ascii?Q?e8k4QCwZpzWjxOORVr7+VGo8EJk+44dHdem2iRAdtLekXAbOEVfZq0To9bx7?= =?us-ascii?Q?WOmBYxb0hLRvJYKCA0WBNuoTFeOUN95Lbb/3qHBGRhUFPFHyt3Fwq+xN6hsQ?= =?us-ascii?Q?JQW40/92/kfAejbyXKdWRKcYqBsOXOCoP6dlHz6BDAyJQNPSUbtXCHoR0Q40?= =?us-ascii?Q?LT6OyNa7ilsped6N7c1DVdHIbGuza6jFXgjH3AUj2c6e?= Content-Type: multipart/alternative; boundary="_000_DM5PR1201MB25557E1FD15849CF8B763C96CD209DM5PR1201MB2555_" MIME-Version: 1.0 X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM5PR1201MB2555.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: b3bf9352-02aa-4308-ea1d-08d9e0f2ddb5 X-MS-Exchange-CrossTenant-originalarrivaltime: 26 Jan 2022 17:39:55.4780 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: PjARvhbZjuZ39ZrmKvVJfvq1QMaisu5u9rZ1ad4d3NtZD7trk2Q1fYX1ga3bgI6Dl5tU/Vx0mn/98D3zth3QVA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4985 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org --_000_DM5PR1201MB25557E1FD15849CF8B763C96CD209DM5PR1201MB2555_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable You mention flow2 is with the errors, but also show the performance is good= . is this correct? Do you mean when you have errors you get perf drop? Regards, Asaf Penso ________________________________ From: bugzilla@dpdk.org Sent: Thursday, January 20, 2022 11:01:43 AM To: dev@dpdk.org Subject: [Bug 924] Mellanox ConnectX6 DPDK dpdk-testpmd Receive error len e= rror checksum UDP packet performance is very low! https://bugs.dpdk.org/show_bug.cgi?id=3D924 Bug ID: 924 Summary: Mellanox ConnectX6 DPDK dpdk-testpmd Receive error len error checksum UDP packet performance is very low! Product: DPDK Version: 21.11 Hardware: x86 OS: Linux Status: UNCONFIRMED Severity: normal Priority: Normal Component: ethdev Assignee: dev@dpdk.org Reporter: 188989801@qq.com Target Milestone: --- I use Ixia to construct two streams flow1 udp 64size small packet flow2 udp 64size small packet error len error checksum Send per second 30G bps 57 million pps ./dpdk-testpmd -l 4-22 -n 8 -- -i --rxq 19 --txq 19 --nb-cores 18 --rxd 204= 8 --txd 2048 --portmask 0xff set fwd rxonly start ######################## NIC statistics for port 6 #####################= ### RX-packets: 178423861761 RX-missed: 646877 RX-bytes: 11419127182586 RX-errors: 0 RX-nombuf: 0 TX-packets: 0 TX-errors: 0 TX-bytes: 0 Throughput (since last show) Rx-pps: 5904277 Rx-bps: 3022990248 Tx-pps: 0 Tx-bps: 0 #########################################################################= ### Recive per second 3G bps 5 million pps ,Only about one tenth was received rx_discards_phy drop [root@localhost ~]# ethtool -S enp202s0f0 |grep dis rx_discards_phy: 48832790759 tx_discards_phy: 0 rx_prio0_discards: 48832470594 rx_prio1_discards: 0 rx_prio2_discards: 0 rx_prio3_discards: 0 rx_prio4_discards: 0 rx_prio5_discards: 0 rx_prio6_discards: 0 rx_prio7_discards: 0 When I change the flow2 to no problem udp, It's all normal. ######################## NIC statistics for port 6 #####################= ### RX-packets: 179251451823 RX-missed: 660384 RX-bytes: 11472092947106 RX-errors: 0 RX-nombuf: 0 TX-packets: 0 TX-errors: 0 TX-bytes: 0 Throughput (since last show) Rx-pps: 57553942 Rx-bps: 29467618576 Tx-pps: 0 Tx-bps: 0 #########################################################################= ### Recive per second 29.4G bps 57 million pps [root@localhost ~]# mlxfwmanager Querying Mellanox devices firmware ... Device #1: ---------- Device Type: ConnectX6 Part Number: MCX653106A-ECA_Ax Description: ConnectX-6 VPI adapter card; H100Gb/s (HDR100; EDR IB a= nd 100GbE); dual-port QSFP56; PCIe3.0 x16; tall bracket; ROHS R6 PSID: MT_0000000224 PCI Device Name: 0000:ca:00.0 Base MAC: 08c0eb204e5a Versions: Current Available FW 20.30.1004 20.32.1010 PXE 3.6.0301 3.6.0502 UEFI 14.23.0017 14.25.0017 Status: Update required -- You are receiving this mail because: You are the assignee for the bug. --_000_DM5PR1201MB25557E1FD15849CF8B763C96CD209DM5PR1201MB2555_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable
You mention flow2 is with the errors, but also show the p= erformance is good. is this correct?
Do you mean when you have errors you get perf drop?

Regards,
Asaf Penso
From: bugzilla@dpdk.org <= ;bugzilla@dpdk.org>
Sent: Thursday, January 20, 2022 11:01:43 AM
To: dev@dpdk.org <dev@dpdk.org>
Subject: [Bug 924] Mellanox ConnectX6 DPDK dpdk-testpmd Receive erro= r len error checksum UDP packet performance is very low!
 
https://bugs.dpdk.org/show_bug.cgi?id=3D924

            Bug ID: = 924
           Summary: Mella= nox ConnectX6 DPDK dpdk-testpmd Receive error len
            &nb= sp;       error checksum UDP packet performan= ce is very low!
           Product: DPDK<= br>            Version: 21.11=
          Hardware: x86
            &nb= sp;   OS: Linux
            Status: = UNCONFIRMED
          Severity: normal
          Priority: Normal
         Component: ethdev
          Assignee: dev@dpdk.o= rg
          Reporter: 188989801@= qq.com
  Target Milestone: ---

I use Ixia to construct two streams  
flow1 udp 64size small packet
flow2 udp 64size small packet error len error checksum
Send per second 30G bps  57 million pps


./dpdk-testpmd -l 4-22 -n 8 -- -i --rxq 19 --txq 19 --nb-cores 18 --rxd 204= 8
--txd 2048 --portmask 0xff

set fwd rxonly
start

  ######################## NIC statistics for port 6  ###########= #############
  RX-packets: 178423861761 RX-missed: 646877     R= X-bytes:  11419127182586
  RX-errors: 0
  RX-nombuf:  0         <= br>   TX-packets: 0          = TX-errors: 0          TX-bytes= :  0

  Throughput (since last show)
  Rx-pps:      5904277    = ;      Rx-bps:   3022990248
  Tx-pps:          &= nbsp; 0          Tx-bps: =            0
  ####################################################################= ########
Recive per second 3G bps 5 million pps ,Only about one tenth was received
rx_discards_phy drop
[root@localhost ~]#  ethtool -S enp202s0f0 |grep dis
     rx_discards_phy: 48832790759
     tx_discards_phy: 0
     rx_prio0_discards: 48832470594
     rx_prio1_discards: 0
     rx_prio2_discards: 0
     rx_prio3_discards: 0
     rx_prio4_discards: 0
     rx_prio5_discards: 0
     rx_prio6_discards: 0
     rx_prio7_discards: 0


When I change the flow2 to no problem udp, It's all normal.

  ######################## NIC statistics for port 6  ###########= #############
  RX-packets: 179251451823 RX-missed: 660384     R= X-bytes:  11472092947106
  RX-errors: 0
  RX-nombuf:  0         <= br>   TX-packets: 0          = TX-errors: 0          TX-bytes= :  0

  Throughput (since last show)
  Rx-pps:     57553942    &nbs= p;     Rx-bps:  29467618576
  Tx-pps:          &= nbsp; 0          Tx-bps: =            0
  ####################################################################= ########

Recive per second 29.4G bps 57 million pps



[root@localhost ~]# mlxfwmanager
Querying Mellanox devices firmware ...

Device #1:
----------

  Device Type:      ConnectX6
  Part Number:      MCX653106A-ECA_Ax
  Description:      ConnectX-6 VPI adapter ca= rd; H100Gb/s (HDR100; EDR IB and
100GbE); dual-port QSFP56; PCIe3.0 x16; tall bracket; ROHS R6
  PSID:          &nb= sp;  MT_0000000224
  PCI Device Name:  0000:ca:00.0
  Base MAC:         08c0eb204e= 5a
  Versions:         Current&nb= sp;       Available     <= br>      FW        =      20.30.1004     20.32.1010 = ;  
     PXE        = ;    3.6.0301       3.6.0502&n= bsp;    
     UEFI       &nbs= p;   14.23.0017     14.25.0017  &nb= sp;

  Status:           = Update required

--
You are receiving this mail because:
You are the assignee for the bug.
--_000_DM5PR1201MB25557E1FD15849CF8B763C96CD209DM5PR1201MB2555_--