From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id DA7FB45DD0; Fri, 29 Nov 2024 21:46:12 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id BE54E4027D; Fri, 29 Nov 2024 21:46:12 +0100 (CET) Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2054.outbound.protection.outlook.com [40.107.237.54]) by mails.dpdk.org (Postfix) with ESMTP id 62E0440150; Fri, 29 Nov 2024 21:46:11 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=v1to4vSDAnzuPwauMQ6UJE9b1yny1WSatmbuj12ThNnUAyRe42FvqLlu6b3pmDAHiqA7OhWiVjHUgQgSqSGzLyei6oH/O/+eqIXLSj8C7iSeJ9xqCl7gicF/mHUm7rnPpCZwCe8JkWjc692hj6Gm0g6O5ZPxr8F+jhz6rUJaFGnWRzyymrCAz6PKZs1be0QCLvv9utagQo9qFijNxf6otKkau/xqOtxF+ycqARWGETS1l5rmx/MFyJQcAFV8kUxc0pAXhIzMJBCp/WSCG3NguaqwpL6FyG2AZyTjn+WXt7fhl+uM2ks2Tu2h7oJ2gg1GIz9JpevLYIgH+ySOWyDY5w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=JqsBeEKM8a/bVL17okNN2XmH89tEMJxx5m4zqBMp2Hg=; b=eNNgBE9ME0gKJi6Bex6HPpCqNBBXzBR1nz4hXGzymxD3Dy/pbRdT3yKbUrlviCmxMdJnYx1kNlR+y9mzV7oG+T5x1ffsqhP2w0KLTGIMmNRAAQtSwPu7KNzzAiKF7NddnvTJpfK0BokUCTCjSdNPPr8OcfHos4hxep0SF/cmeDVrJZEsno0nhPPAu3rgtVmXdC9byqn1HT6vwvEZlqCZUJs1bVFvhzunhcbdm7lDkppCzQ13/faJ+UywPzmbVYjqZ94mOV1w8nSwMMIZhs/QTGYAd4OUQpVq1gy1XDctya+BJGgyMmjzlMHFdWIymt8C17rje12xf0jkHoFqamEKIQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=JqsBeEKM8a/bVL17okNN2XmH89tEMJxx5m4zqBMp2Hg=; b=W3JZ3UJ+JpEmI4AJxsT+qAvxRC2Wqu+O0koGTGmEcd9grSdv0VjnDDa4T78D/hNjKo+3AN3mpk2o6pMysEUpGNZh87OFEL6eLNn8FLfzK/v6ofG/tG/NlmdY0047OFPNUQGtMcUwMmRrQMmU4EoiJ65SlJ5F7sGNRtLBtt0GHnjIVjXEyguNNTyoHtPPMpYdMQaPD2/QSSBk7PiI27WkDXiO9h6kZcUaYSGXYk1PLEGFZGzC06w9XfFCxEzgAGkMi7S0sEPSIIDoyAUjdleGJxldPvuj5yMO79J6rYyNjTU1IiNrfOF+9y+V+TQoBX6L6+V0DnTcwOKBZscOvaVJWg== Received: from SA9PR13CA0164.namprd13.prod.outlook.com (2603:10b6:806:28::19) by IA0PR12MB8839.namprd12.prod.outlook.com (2603:10b6:208:493::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8207.13; Fri, 29 Nov 2024 20:46:06 +0000 Received: from SA2PEPF000015C8.namprd03.prod.outlook.com (2603:10b6:806:28:cafe::6b) by SA9PR13CA0164.outlook.office365.com (2603:10b6:806:28::19) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8230.5 via Frontend Transport; Fri, 29 Nov 2024 20:46:06 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by SA2PEPF000015C8.mail.protection.outlook.com (10.167.241.198) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8207.12 via Frontend Transport; Fri, 29 Nov 2024 20:46:05 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Fri, 29 Nov 2024 12:46:00 -0800 Received: from pegasus01.mtr.labs.mlnx (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Fri, 29 Nov 2024 12:45:57 -0800 From: Alexander Kozyrev To: CC: , , , , , , Subject: [PATCH v2] net/mlx5: fix RSS hash for non-RSS CQE zipping Date: Fri, 29 Nov 2024 22:44:59 +0200 Message-ID: <20241129204540.2159168-1-akozyrev@nvidia.com> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20241121133231.914815-1-akozyrev@nvidia.com> References: <20241121133231.914815-1-akozyrev@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [10.126.231.35] X-ClientProxiedBy: rnnvmail202.nvidia.com (10.129.68.7) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SA2PEPF000015C8:EE_|IA0PR12MB8839:EE_ X-MS-Office365-Filtering-Correlation-Id: 3cabbeb6-fa9e-4aa1-e351-08dd10b6d88a X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|1800799024|376014|36860700013|82310400026; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?We1lR9KZFwjCS5i1sKshm11ruwXQ2/BWTHU6YkEtEi6TBA6fearmIiwwWKA8?= =?us-ascii?Q?ONsWVLHYqZRTfDkI1cKOzyNOPykG4CwMoiz3G8/hRIAgZvxojVU5c0SE0Zqb?= =?us-ascii?Q?CzOwQ2KugtdL4A3mKra31IQmjcDma2s01+TfMMxL121ISbu2GycxkOI+SjAp?= =?us-ascii?Q?iyZ3Iqzz5g8hf910FeAPdZG0tlDdYZ0vNpjYWl7zkxZQ+17o0k4kTq+4GCq0?= =?us-ascii?Q?Q97K4fh7glG1wRg86dxt0Au85c6L0k57/rGVuf8N/Awg3yRztuE+5IKsoMTo?= =?us-ascii?Q?P7WTJqOlIAO5LkpApF5DOPkiogkhD1q0TMCpxy2Ov8FOmCaXn0F10cufYuZS?= =?us-ascii?Q?LwJp/TYMj8wOP3qd845qSRfYgAYsDI3uzFmdIUiVE1G0CwoMfdj364+wo2ex?= =?us-ascii?Q?rKlYevj2W3gu0cmKqUIT4cD0m6Mm7kpqZQYN6FmOm7PqcYwH3gs4/Ie81sde?= =?us-ascii?Q?jil/uqt0LWgCe2uKItRT2E81GPecVtath4WIaQhjVc2fIepQvP9OHuuge2UW?= =?us-ascii?Q?u78TpLvDOc0K4YUm9qiBZ3mlyyLzHdByEBqQockotPOL1wprltQ8jozpI6NB?= =?us-ascii?Q?84KjxVuRGydtCmDCfYSQzduBGP0DLc3/VWHoCYPUWWvuUE3yL1+ps/K5E0Q5?= =?us-ascii?Q?L35+xHzyAaWgMY8+h/KSw3vVyMET1Zud2aRUOLdrzpe2zMjH+5wAnTKa1gs6?= =?us-ascii?Q?s4vCyDbglht73SZ13BBu9F1CUDbaVML3qwIZZrGAftF+WYo3NxVxXC0Xuy3i?= =?us-ascii?Q?2RkPEOBflU9KtGS1Em4ICx+jmkx4hv75SDm4aivZ6kDsvHWP7j66auHZvHVH?= =?us-ascii?Q?Jn8QTeDnGMUQbOJT/t6oMpj0+ECQOxorQaLB972N2+YLZ5ydt3qefI+0vGPE?= =?us-ascii?Q?tQINWqO1IgEvqz01gdPjCSUvdhQlgUi2/wv3UkPL6xcr7k1NKDjANQHp5Rg6?= =?us-ascii?Q?tOChPuWrZInXT5B3+11AyzMwvXdfxGRaOqomraY+pet1toB3IKCYZxgMVVUh?= =?us-ascii?Q?jKl92EHhhR9O6J38lxuTfwgKRrykf3i8pvZr71muX8yXwq1CivMH0UywgBjx?= =?us-ascii?Q?vaygEj6HE4cgvwSGdxpzH5kbNQWw6G+N4HdvW3g7pYjVfw+G1GaZhPIrM7j1?= =?us-ascii?Q?hnQyb4oiger1VKW9ZdClwYrN9+nU4NGP2HAhJ0v7WQe1EGa1NPfb9yg0OuiT?= =?us-ascii?Q?ATfUo+dm2iA2fKw/UqXwg55AKSvPp7ulYCHVnKCgV5go4kDqkejQFcyXLdJH?= =?us-ascii?Q?U00/nL4RzF5ij6orzETfF7RMv6cedfk0psAmiwVfQsn0wG8h8Y0ST5ieM8W9?= =?us-ascii?Q?QmWN8ymx+lsyeDbG/YODbni56OprzHxCzaASx49KDXkhlvt1Rn/KywSWtaOS?= =?us-ascii?Q?nJ/BjJD/J4Gvyefe4cOwaAuRaNrzQhrCJlMX5T0LxEJ8UhljlWQ3LDN9a1wp?= =?us-ascii?Q?RRpyVUViyE6C0heoAa61I1ScNWqJkBng?= X-Forefront-Antispam-Report: CIP:216.228.117.161; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge2.nvidia.com; CAT:NONE; SFS:(13230040)(1800799024)(376014)(36860700013)(82310400026); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Nov 2024 20:46:05.7989 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 3cabbeb6-fa9e-4aa1-e351-08dd10b6d88a X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.161]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SA2PEPF000015C8.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR12MB8839 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Take the RSS hash and flow tag values from the title packet before they get overwritten by the decompressing routine. Set the RSS hash flag in the packet mbuf if RSS is enabled in case of non-RSS CQE zipping format. Signed-off-by: Alexander Kozyrev --- drivers/net/mlx5/mlx5_rxtx_vec_altivec.h | 23 +++++++++++++---------- drivers/net/mlx5/mlx5_rxtx_vec_neon.h | 15 ++++++++------- drivers/net/mlx5/mlx5_rxtx_vec_sse.h | 15 ++++++++------- 3 files changed, 29 insertions(+), 24 deletions(-) diff --git a/drivers/net/mlx5/mlx5_rxtx_vec_altivec.h b/drivers/net/mlx5/mlx5_rxtx_vec_altivec.h index 240987d03d..3b2f33d138 100644 --- a/drivers/net/mlx5/mlx5_rxtx_vec_altivec.h +++ b/drivers/net/mlx5/mlx5_rxtx_vec_altivec.h @@ -82,6 +82,8 @@ rxq_cq_decompress_v(struct mlx5_rxq_data *rxq, volatile struct mlx5_cqe *cq, (void *)&(cq + !rxq->cqe_comp_layout)->pkt_info; /* Title packet is pre-built. */ struct rte_mbuf *t_pkt = rxq->cqe_comp_layout ? &rxq->title_pkt : elts[0]; + const uint32_t hash_rss = t_pkt->hash.rss * rxq->rss_hash; + const uint32_t flow_tag = t_pkt->hash.fdir.hi; const __vector unsigned char zero = (__vector unsigned char){0}; /* Mask to shuffle from extracted mini CQE to mbuf. */ const __vector unsigned char shuf_mask1 = (__vector unsigned char){ @@ -266,8 +268,6 @@ rxq_cq_decompress_v(struct mlx5_rxq_data *rxq, volatile struct mlx5_cqe *cq, if (rxq->mark) { if (rxq->mcqe_format != MLX5_CQE_RESP_FORMAT_FTAG_STRIDX) { - const uint32_t flow_tag = t_pkt->hash.fdir.hi; - /* E.1 store flow tag (rte_flow mark). */ elts[pos]->hash.fdir.hi = flow_tag; elts[pos + 1]->hash.fdir.hi = flow_tag; @@ -442,10 +442,10 @@ rxq_cq_decompress_v(struct mlx5_rxq_data *rxq, volatile struct mlx5_cqe *cq, } const __vector unsigned char hash_mask = (__vector unsigned char)(__vector unsigned int) { - RTE_MBUF_F_RX_RSS_HASH, - RTE_MBUF_F_RX_RSS_HASH, - RTE_MBUF_F_RX_RSS_HASH, - RTE_MBUF_F_RX_RSS_HASH}; + rxq->rss_hash * RTE_MBUF_F_RX_RSS_HASH, + rxq->rss_hash * RTE_MBUF_F_RX_RSS_HASH, + rxq->rss_hash * RTE_MBUF_F_RX_RSS_HASH, + rxq->rss_hash * RTE_MBUF_F_RX_RSS_HASH}; const __vector unsigned char rearm_flags = (__vector unsigned char)(__vector unsigned int) { (uint32_t)t_pkt->ol_flags, @@ -456,6 +456,9 @@ rxq_cq_decompress_v(struct mlx5_rxq_data *rxq, volatile struct mlx5_cqe *cq, ol_flags_mask = (__vector unsigned char) vec_or((__vector unsigned long)ol_flags_mask, (__vector unsigned long)hash_mask); + ol_flags = (__vector unsigned char) + vec_or((__vector unsigned long)ol_flags, + (__vector unsigned long)hash_mask); ol_flags = (__vector unsigned char) vec_or((__vector unsigned long)ol_flags, (__vector unsigned long) @@ -470,10 +473,10 @@ rxq_cq_decompress_v(struct mlx5_rxq_data *rxq, volatile struct mlx5_cqe *cq, ((__vector unsigned int)ol_flags)[2]; elts[pos + 3]->ol_flags = ((__vector unsigned int)ol_flags)[3]; - elts[pos]->hash.rss = 0; - elts[pos + 1]->hash.rss = 0; - elts[pos + 2]->hash.rss = 0; - elts[pos + 3]->hash.rss = 0; + elts[pos]->hash.rss = hash_rss; + elts[pos + 1]->hash.rss = hash_rss; + elts[pos + 2]->hash.rss = hash_rss; + elts[pos + 3]->hash.rss = hash_rss; } if (rxq->dynf_meta) { int32_t offs = rxq->flow_meta_offset; diff --git a/drivers/net/mlx5/mlx5_rxtx_vec_neon.h b/drivers/net/mlx5/mlx5_rxtx_vec_neon.h index dc1d30753d..58e3918ef4 100644 --- a/drivers/net/mlx5/mlx5_rxtx_vec_neon.h +++ b/drivers/net/mlx5/mlx5_rxtx_vec_neon.h @@ -78,6 +78,8 @@ rxq_cq_decompress_v(struct mlx5_rxq_data *rxq, volatile struct mlx5_cqe *cq, (void *)&(cq + !rxq->cqe_comp_layout)->pkt_info; /* Title packet is pre-built. */ struct rte_mbuf *t_pkt = rxq->cqe_comp_layout ? &rxq->title_pkt : elts[0]; + const uint32_t hash_rss = t_pkt->hash.rss * rxq->rss_hash; + const uint32_t flow_tag = t_pkt->hash.fdir.hi; unsigned int pos; unsigned int i; unsigned int inv = 0; @@ -211,8 +213,6 @@ rxq_cq_decompress_v(struct mlx5_rxq_data *rxq, volatile struct mlx5_cqe *cq, if (rxq->mark) { if (rxq->mcqe_format != MLX5_CQE_RESP_FORMAT_FTAG_STRIDX) { - const uint32_t flow_tag = t_pkt->hash.fdir.hi; - /* E.1 store flow tag (rte_flow mark). */ elts[pos]->hash.fdir.hi = flow_tag; elts[pos + 1]->hash.fdir.hi = flow_tag; @@ -327,21 +327,22 @@ rxq_cq_decompress_v(struct mlx5_rxq_data *rxq, volatile struct mlx5_cqe *cq, } } const uint32x4_t hash_flags = - vdupq_n_u32(RTE_MBUF_F_RX_RSS_HASH); + vdupq_n_u32(rxq->rss_hash * RTE_MBUF_F_RX_RSS_HASH); const uint32x4_t rearm_flags = vdupq_n_u32((uint32_t)t_pkt->ol_flags); ol_flags_mask = vorrq_u32(ol_flags_mask, hash_flags); + ol_flags = vorrq_u32(ol_flags, hash_flags); ol_flags = vorrq_u32(ol_flags, vbicq_u32(rearm_flags, ol_flags_mask)); elts[pos]->ol_flags = vgetq_lane_u32(ol_flags, 3); elts[pos + 1]->ol_flags = vgetq_lane_u32(ol_flags, 2); elts[pos + 2]->ol_flags = vgetq_lane_u32(ol_flags, 1); elts[pos + 3]->ol_flags = vgetq_lane_u32(ol_flags, 0); - elts[pos]->hash.rss = 0; - elts[pos + 1]->hash.rss = 0; - elts[pos + 2]->hash.rss = 0; - elts[pos + 3]->hash.rss = 0; + elts[pos]->hash.rss = hash_rss; + elts[pos + 1]->hash.rss = hash_rss; + elts[pos + 2]->hash.rss = hash_rss; + elts[pos + 3]->hash.rss = hash_rss; } if (rxq->dynf_meta) { int32_t offs = rxq->flow_meta_offset; diff --git a/drivers/net/mlx5/mlx5_rxtx_vec_sse.h b/drivers/net/mlx5/mlx5_rxtx_vec_sse.h index 81a177fce7..8a83a0e59d 100644 --- a/drivers/net/mlx5/mlx5_rxtx_vec_sse.h +++ b/drivers/net/mlx5/mlx5_rxtx_vec_sse.h @@ -78,6 +78,8 @@ rxq_cq_decompress_v(struct mlx5_rxq_data *rxq, volatile struct mlx5_cqe *cq, volatile struct mlx5_mini_cqe8 *mcq = (void *)(cq + !rxq->cqe_comp_layout); /* Title packet is pre-built. */ struct rte_mbuf *t_pkt = rxq->cqe_comp_layout ? &rxq->title_pkt : elts[0]; + const uint32_t hash_rss = t_pkt->hash.rss * rxq->rss_hash; + const uint32_t flow_tag = t_pkt->hash.fdir.hi; unsigned int pos; unsigned int i; unsigned int inv = 0; @@ -194,8 +196,6 @@ rxq_cq_decompress_v(struct mlx5_rxq_data *rxq, volatile struct mlx5_cqe *cq, if (rxq->mark) { if (rxq->mcqe_format != MLX5_CQE_RESP_FORMAT_FTAG_STRIDX) { - const uint32_t flow_tag = t_pkt->hash.fdir.hi; - /* E.1 store flow tag (rte_flow mark). */ elts[pos]->hash.fdir.hi = flow_tag; elts[pos + 1]->hash.fdir.hi = flow_tag; @@ -311,11 +311,12 @@ rxq_cq_decompress_v(struct mlx5_rxq_data *rxq, volatile struct mlx5_cqe *cq, } } const __m128i hash_flags = - _mm_set1_epi32(RTE_MBUF_F_RX_RSS_HASH); + _mm_set1_epi32(rxq->rss_hash * RTE_MBUF_F_RX_RSS_HASH); const __m128i rearm_flags = _mm_set1_epi32((uint32_t)t_pkt->ol_flags); ol_flags_mask = _mm_or_si128(ol_flags_mask, hash_flags); + ol_flags = _mm_or_si128(ol_flags, hash_flags); ol_flags = _mm_or_si128(ol_flags, _mm_andnot_si128(ol_flags_mask, rearm_flags)); elts[pos]->ol_flags = @@ -326,10 +327,10 @@ rxq_cq_decompress_v(struct mlx5_rxq_data *rxq, volatile struct mlx5_cqe *cq, _mm_extract_epi32(ol_flags, 2); elts[pos + 3]->ol_flags = _mm_extract_epi32(ol_flags, 3); - elts[pos]->hash.rss = 0; - elts[pos + 1]->hash.rss = 0; - elts[pos + 2]->hash.rss = 0; - elts[pos + 3]->hash.rss = 0; + elts[pos]->hash.rss = hash_rss; + elts[pos + 1]->hash.rss = hash_rss; + elts[pos + 2]->hash.rss = hash_rss; + elts[pos + 3]->hash.rss = hash_rss; } if (rxq->dynf_meta) { int32_t offs = rxq->flow_meta_offset; -- 2.43.5