From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C2F19A09E4; Thu, 28 Jan 2021 14:21:01 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 88B504067A; Thu, 28 Jan 2021 14:21:01 +0100 (CET) Received: from inbox.dpdk.org (xvm-172-178.dc0.ghst.net [95.142.172.178]) by mails.dpdk.org (Postfix) with ESMTP id 5E58040395 for ; Thu, 28 Jan 2021 14:21:00 +0100 (CET) Received: by inbox.dpdk.org (Postfix, from userid 33) id 3B3F9A0A0B; Thu, 28 Jan 2021 14:21:00 +0100 (CET) From: bugzilla@dpdk.org To: dev@dpdk.org Date: Thu, 28 Jan 2021 13:21:00 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: DPDK X-Bugzilla-Component: ethdev X-Bugzilla-Version: 20.11 X-Bugzilla-Keywords: X-Bugzilla-Severity: major X-Bugzilla-Who: martin.weiser@allegro-packets.com X-Bugzilla-Status: UNCONFIRMED X-Bugzilla-Resolution: X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: dev@dpdk.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform op_sys bug_status bug_severity priority component assigned_to reporter target_milestone Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: http://bugs.dpdk.org/ Auto-Submitted: auto-generated X-Auto-Response-Suppress: All MIME-Version: 1.0 Subject: [dpdk-dev] [Bug 627] mlx5 / DPDK 20.11: severe RSS misbehavior and performance drop with libibverbs 29 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" https://bugs.dpdk.org/show_bug.cgi?id=3D627 Bug ID: 627 Summary: mlx5 / DPDK 20.11: severe RSS misbehavior and performance drop with libibverbs 29 Product: DPDK Version: 20.11 Hardware: x86 OS: Linux Status: UNCONFIRMED Severity: major Priority: Normal Component: ethdev Assignee: dev@dpdk.org Reporter: martin.weiser@allegro-packets.com Target Milestone: --- When updating to DPDK v20.11 we noticed a dramatic performance drop and RSS misbehavior with our ConnectX-5 NICs. Some of the rx queues do not receive any traffic while others only receive a fraction of what they should receive. Overall the receive performance dropped by around 90%. We then were able to identify that this only happens when the default mlx5_rx_burst path is used and the issue does not appear when a vectorized rx function is used. We are forced to the default path since we use multi-segment mbufs to handle jumbo frames which automatically drops you to the default non-vectorized path in mlx5. After digging a bit deeper we could identify the following commit to cause = the issue: 54c2d46b160f8ad0bff0977812bf871ca5dd8241: net/mlx5: support flow tag and pa= cket header miniCQEs Reverting this commit restored the original performance. But on a test system running testpmd we were not able to replicate this behavior. It took us quite some time to identify the reason for the difference: On the test system we have an installation of libibverbs 33 and this version has the symbol mlx5dv_dr_action_create_dest_devx_tir in infiniband/mlx5dv.h. This causes t= he meson build to set HAVE_MLX5DV_DR_ACTION_DEST_DEVX_TIR during compilation which in turn causes config->dest_tir to be set in the driver. It seems that the whole driver behaves quite differently if this fl= ag is set. On our regular system we have an installation of libibverbs 29 which lacks = the aforementioned symbol and if we compile testpmd on that system it also shows the misbehavior. But only if y= ou force it to use the non-vectorized mlx5_rx_burst path. We used the following testpmd command line to replicate the behavior: `./app/dpdk-testpmd -n 4 --legacy-mem -w 81:00.0 -w 81:00.1 -l 1,2,3,4,5 -- --total-num-mbufs=3D2000000 --nb-cores=3D4 --rxq=3D4 --txq=3D4 --max-pkt-le= n=3D15360 --rx-offloads=3D0x12800 --mbuf-size=3D2331 --rxd=3D4096` --=20 You are receiving this mail because: You are the assignee for the bug.=