From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 32208A00C4 for ; Wed, 8 Dec 2021 15:52:16 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 1F8114118F; Wed, 8 Dec 2021 15:52:16 +0100 (CET) Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2050.outbound.protection.outlook.com [40.107.243.50]) by mails.dpdk.org (Postfix) with ESMTP id F35DF406FF; Wed, 8 Dec 2021 15:52:13 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=OFGQbQqN2VCBDpOvBQocwhZ0LITQQYqg4sWzeogLS7jxr/CYvTfDL8BNbjKDinmRhhCTuTCD8uMQ01Bb1lH/iaikuNPzqUum8ejJRgnfHjJ4kyxhNM7CbJM1DQV5gaM0dDxY9j354zWuxKJIkBeTqjRoWlXvfFDg2ZFr5dPeRzqZMhe1wvidcpTzWfJSp2TqDJwwh8fCMsVnhHG03Mq6UMzD3+z+RX1KZ64vALLesyAq9eB9zKypkM01mAg9b6cSqXc1kC7/I1ld9oBxmBN4r8AbimblfHVVP1HFml0O6940dUI19ItJyYmFAuzEI9onbYQnYdfGt0ldjuOgzUi/Ow== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=V2CGvMcvIfeplrgqSTLh0QsClqB6FLE/Hx4jqk5iiNo=; b=XLIE9+Un/bzZHbiOrdZ1NPOOyG4As5LD/gIpCL10iv0r6ot8YtMShIfdb9Q/NNCgXyHhYFK3sF4o/psirhehTfkxlNgkYliOQwZDgz+gQsj85l3L/bh4TtB0u3Xay/Yn001ViYNsxyU76nQhMD0EhYESbPh0UtXid+Jx+7709+QE/W9uGuxn0Z4fqbOR3jorCtRed/EE7GO+KOL+sPpL1JTdRCKsZxEJF8dFBB6WcnhOYdkWY8IoMayKlhiGgRurGmeQjLatX0+VML+WJMv8pF5/yye+Dbj32ti0nTuPJ6JIaoIIylBNPke8dbctcR9+WAQ6tqniVquR47PQkDnh7A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 203.18.50.14) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=V2CGvMcvIfeplrgqSTLh0QsClqB6FLE/Hx4jqk5iiNo=; b=DgYGYeoibEAAwizdd5tJ3/btZPCAwWqN33KLPlEx9K5w9xu73g2bO7NugGbvgZQajGi4WhlMZVX/jyEk24ekJ7mFIfaZnSbJGZoFOIyLyxnljrCzKKiBZRf1/5WAjX6jSfU7cxTW1Tp8A1Kk0+oyEurNAoJwdk9vR4PypqAsWg1gEWTF9WEGiGTD5CIjPUFfde21/TyrySjBZjf+x9lcJEEPBLro1RHOlThGyWTN3IEekdIopNrxUh2JfbjkHGskUS9QZnJQkBxdLsilUaUw/rVEpGu521SnBZHxh8cVeQQAXUsVunf/QrKTnYg6PMiSnsbtvCHur5TJzd1q1KxH3Q== Received: from BN8PR07CA0008.namprd07.prod.outlook.com (2603:10b6:408:ac::21) by DM6PR12MB4300.namprd12.prod.outlook.com (2603:10b6:5:21a::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4755.20; Wed, 8 Dec 2021 14:52:12 +0000 Received: from BN8NAM11FT046.eop-nam11.prod.protection.outlook.com (2603:10b6:408:ac:cafe::40) by BN8PR07CA0008.outlook.office365.com (2603:10b6:408:ac::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4778.11 via Frontend Transport; Wed, 8 Dec 2021 14:52:11 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 203.18.50.14) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 203.18.50.14 as permitted sender) receiver=protection.outlook.com; client-ip=203.18.50.14; helo=mail.nvidia.com; Received: from mail.nvidia.com (203.18.50.14) by BN8NAM11FT046.mail.protection.outlook.com (10.13.177.127) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4755.13 via Frontend Transport; Wed, 8 Dec 2021 14:52:11 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by HKMAIL103.nvidia.com (10.18.16.12) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Wed, 8 Dec 2021 14:52:00 +0000 Received: from nvidia.com (172.20.187.5) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.986.9; Wed, 8 Dec 2021 06:51:56 -0800 From: To: CC: Matan Azrad , Raslan Darawsheh , Viacheslav Ovsiienko , Michael Baum , Subject: [PATCH] net/mlx5: fix the memory socket selection in ASO management Date: Wed, 8 Dec 2021 16:51:34 +0200 Message-ID: <20211208145134.171137-1-michaelba@nvidia.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [172.20.187.5] X-ClientProxiedBy: HQMAIL101.nvidia.com (172.20.187.10) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 82e39c78-eedb-48a0-2950-08d9ba5a5109 X-MS-TrafficTypeDiagnostic: DM6PR12MB4300:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:669; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: cNd/ynf1AsdCXW/tMKPpnnvjV9VRckO7Mk9kDemMbQ3PsV6U/mTfElFF/tOZW5mMdksPHuXLt08ir+Pz9gTj9C/rzSUfP6Lr4iP8WlXjAUj+4t4jnTKyVx3uhTnZfsQ0V40qZFTYPxwDPEE8USgS6nQFpRAx+/S0F1T2XAX9pDGPUycosA+LDDwvokGx6CfTAEaouCMLzzwZadEDD1bvSqkLlca6guH8k4C/MHhCXAwYYYo9ah4A5x5iTdSZY//aGuGBI3xsIFuDj8kUumjDbyYkHDLbXzH4aGHfn9aBctrmtyXjAeAghQksdJDPX+ngKvm+qrOETh/hR9bPNBsaZojUMe8spY8o8LTc9rstkdW0FxWX32tAHUyg04munLXIPq7DKDKd38Lcqub/oTVMBazL7nQzsxCZZB57DGXDsYy98/MWaJGT1snABcbGv6JSW02jUDAFG6V3zUglz0X2XUntO5/qp2M3ot54ByhurXR0XOn1w976GCQoX8JAJJ/DMy5Q2T4wHuRS/SLrXE5uqPw2S/A+Oof9qR9+wNZleV2dWH3jYj+rEgdeybAOOBUlDWpOrowdZF388BfqxM8NKrZzUITnFX/ad8cfifDJqYHJd8q8yuPZIdkT1JKRbvSzD6wC0jeYotHYwZOSCtsdL4g/UMr++Vao6ukVOj342r/qPvQ6/d5s+vAzQsurmvN839i19Ef8c8rrPWPuQ5P17+ApjkQyjA78x/sk8f4HRHt+sXyTH4ozj1UREWXn5rl7RAO/AY4mkS4j8UfaJgN69STqbqcuoDkELTsdWBQ8MtQ= X-Forefront-Antispam-Report: CIP:203.18.50.14; CTRY:HK; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:hkhybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(46966006)(36840700001)(40470700001)(70206006)(6916009)(36860700001)(8676002)(356005)(70586007)(7636003)(5660300002)(1076003)(336012)(47076005)(426003)(2876002)(186003)(16526019)(2616005)(508600001)(40460700001)(83380400001)(26005)(7696005)(2906002)(34070700002)(86362001)(4326008)(36756003)(55016003)(316002)(6286002)(6666004)(450100002)(8936002)(82310400004)(54906003); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Dec 2021 14:52:11.4092 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 82e39c78-eedb-48a0-2950-08d9ba5a5109 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[203.18.50.14]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT046.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4300 X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: stable-bounces@dpdk.org From: Michael Baum In ASO objects creation (WQE, CQE and MR), socket number is given as a parameter. The selection was wrongly socket 0 hardcoded even if the user didn't configure memory for this socket. This patch replaces the selection to default socket (SOCKET_ID_ANY). Fixes: f935ed4b645a ("net/mlx5: support flow hit action for aging") Cc: stable@dpdk.org Signed-off-by: Michael Baum Acked-by: Matan Azrad --- drivers/net/mlx5/mlx5_flow_aso.c | 117 +++++++++---------------------- 1 file changed, 32 insertions(+), 85 deletions(-) diff --git a/drivers/net/mlx5/mlx5_flow_aso.c b/drivers/net/mlx5/mlx5_flow_aso.c index ddf4328dec..eb7fc43da3 100644 --- a/drivers/net/mlx5/mlx5_flow_aso.c +++ b/drivers/net/mlx5/mlx5_flow_aso.c @@ -13,50 +13,6 @@ #include "mlx5.h" #include "mlx5_flow.h" -/** - * Destroy Completion Queue used for ASO access. - * - * @param[in] cq - * ASO CQ to destroy. - */ -static void -mlx5_aso_cq_destroy(struct mlx5_aso_cq *cq) -{ - if (cq->cq_obj.cq) - mlx5_devx_cq_destroy(&cq->cq_obj); - memset(cq, 0, sizeof(*cq)); -} - -/** - * Create Completion Queue used for ASO access. - * - * @param[in] ctx - * Context returned from mlx5 open_device() glue function. - * @param[in/out] cq - * Pointer to CQ to create. - * @param[in] log_desc_n - * Log of number of descriptors in queue. - * @param[in] socket - * Socket to use for allocation. - * @param[in] uar_page_id - * UAR page ID to use. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. - */ -static int -mlx5_aso_cq_create(void *ctx, struct mlx5_aso_cq *cq, uint16_t log_desc_n, - int socket, int uar_page_id) -{ - struct mlx5_devx_cq_attr attr = { - .uar_page_id = uar_page_id, - }; - - cq->log_desc_n = log_desc_n; - cq->cq_ci = 0; - return mlx5_devx_cq_create(ctx, &cq->cq_obj, log_desc_n, &attr, socket); -} - /** * Free MR resources. * @@ -84,21 +40,18 @@ mlx5_aso_dereg_mr(struct mlx5_common_device *cdev, struct mlx5_pmd_mr *mr) * Size of MR buffer. * @param[in/out] mr * Pointer to MR to create. - * @param[in] socket - * Socket to use for allocation. * * @return * 0 on success, a negative errno value otherwise and rte_errno is set. */ static int mlx5_aso_reg_mr(struct mlx5_common_device *cdev, size_t length, - struct mlx5_pmd_mr *mr, int socket) + struct mlx5_pmd_mr *mr) { - int ret; mr->addr = mlx5_malloc(MLX5_MEM_RTE | MLX5_MEM_ZERO, length, 4096, - socket); + SOCKET_ID_ANY); if (!mr->addr) { DRV_LOG(ERR, "Failed to create ASO bits mem for MR."); return -1; @@ -122,7 +75,7 @@ static void mlx5_aso_destroy_sq(struct mlx5_aso_sq *sq) { mlx5_devx_sq_destroy(&sq->sq_obj); - mlx5_aso_cq_destroy(&sq->cq); + mlx5_devx_cq_destroy(&sq->cq.cq_obj); memset(sq, 0, sizeof(*sq)); } @@ -226,35 +179,31 @@ mlx5_aso_ct_init_sq(struct mlx5_aso_sq *sq) /** * Create Send Queue used for ASO access. * - * @param[in] ctx - * Context returned from mlx5 open_device() glue function. + * @param[in] cdev + * Pointer to the mlx5 common device. * @param[in/out] sq * Pointer to SQ to create. - * @param[in] socket - * Socket to use for allocation. * @param[in] uar * User Access Region object. - * @param[in] pdn - * Protection Domain number to use. - * @param[in] log_desc_n - * Log of number of descriptors in queue. - * @param[in] ts_format - * timestamp format supported by the queue. * * @return * 0 on success, a negative errno value otherwise and rte_errno is set. */ static int -mlx5_aso_sq_create(void *ctx, struct mlx5_aso_sq *sq, int socket, void *uar, - uint32_t pdn, uint16_t log_desc_n, uint32_t ts_format) +mlx5_aso_sq_create(struct mlx5_common_device *cdev, struct mlx5_aso_sq *sq, + void *uar) { - struct mlx5_devx_create_sq_attr attr = { + struct mlx5_devx_cq_attr cq_attr = { + .uar_page_id = mlx5_os_get_devx_uar_page_id(uar), + }; + struct mlx5_devx_create_sq_attr sq_attr = { .user_index = 0xFFFF, .wq_attr = (struct mlx5_devx_wq_attr){ - .pd = pdn, + .pd = cdev->pdn, .uar_page = mlx5_os_get_devx_uar_page_id(uar), }, - .ts_format = mlx5_ts_format_conv(ts_format), + .ts_format = + mlx5_ts_format_conv(cdev->config.hca_attr.sq_ts_format), }; struct mlx5_devx_modify_sq_attr modify_attr = { .state = MLX5_SQC_STATE_RDY, @@ -262,14 +211,18 @@ mlx5_aso_sq_create(void *ctx, struct mlx5_aso_sq *sq, int socket, void *uar, uint16_t log_wqbb_n; int ret; - if (mlx5_aso_cq_create(ctx, &sq->cq, log_desc_n, socket, - mlx5_os_get_devx_uar_page_id(uar))) + if (mlx5_devx_cq_create(cdev->ctx, &sq->cq.cq_obj, + MLX5_ASO_QUEUE_LOG_DESC, &cq_attr, + SOCKET_ID_ANY)) goto error; - sq->log_desc_n = log_desc_n; - attr.cqn = sq->cq.cq_obj.cq->id; + sq->cq.cq_ci = 0; + sq->cq.log_desc_n = MLX5_ASO_QUEUE_LOG_DESC; + sq->log_desc_n = MLX5_ASO_QUEUE_LOG_DESC; + sq_attr.cqn = sq->cq.cq_obj.cq->id; /* for mlx5_aso_wqe that is twice the size of mlx5_wqe */ - log_wqbb_n = log_desc_n + 1; - ret = mlx5_devx_sq_create(ctx, &sq->sq_obj, log_wqbb_n, &attr, socket); + log_wqbb_n = sq->log_desc_n + 1; + ret = mlx5_devx_sq_create(cdev->ctx, &sq->sq_obj, log_wqbb_n, &sq_attr, + SOCKET_ID_ANY); if (ret) { DRV_LOG(ERR, "Can't create SQ object."); rte_errno = ENOMEM; @@ -313,34 +266,28 @@ mlx5_aso_queue_init(struct mlx5_dev_ctx_shared *sh, switch (aso_opc_mod) { case ASO_OPC_MOD_FLOW_HIT: if (mlx5_aso_reg_mr(cdev, (MLX5_ASO_AGE_ACTIONS_PER_POOL / 8) * - sq_desc_n, &sh->aso_age_mng->aso_sq.mr, 0)) + sq_desc_n, &sh->aso_age_mng->aso_sq.mr)) return -1; - if (mlx5_aso_sq_create(cdev->ctx, &sh->aso_age_mng->aso_sq, 0, - sh->tx_uar.obj, cdev->pdn, - MLX5_ASO_QUEUE_LOG_DESC, - cdev->config.hca_attr.sq_ts_format)) { + if (mlx5_aso_sq_create(cdev, &sh->aso_age_mng->aso_sq, + sh->tx_uar.obj)) { mlx5_aso_dereg_mr(cdev, &sh->aso_age_mng->aso_sq.mr); return -1; } mlx5_aso_age_init_sq(&sh->aso_age_mng->aso_sq); break; case ASO_OPC_MOD_POLICER: - if (mlx5_aso_sq_create(cdev->ctx, &sh->mtrmng->pools_mng.sq, 0, - sh->tx_uar.obj, cdev->pdn, - MLX5_ASO_QUEUE_LOG_DESC, - cdev->config.hca_attr.sq_ts_format)) + if (mlx5_aso_sq_create(cdev, &sh->mtrmng->pools_mng.sq, + sh->tx_uar.obj)) return -1; mlx5_aso_mtr_init_sq(&sh->mtrmng->pools_mng.sq); break; case ASO_OPC_MOD_CONNECTION_TRACKING: /* 64B per object for query. */ if (mlx5_aso_reg_mr(cdev, 64 * sq_desc_n, - &sh->ct_mng->aso_sq.mr, 0)) + &sh->ct_mng->aso_sq.mr)) return -1; - if (mlx5_aso_sq_create(cdev->ctx, &sh->ct_mng->aso_sq, 0, - sh->tx_uar.obj, cdev->pdn, - MLX5_ASO_QUEUE_LOG_DESC, - cdev->config.hca_attr.sq_ts_format)) { + if (mlx5_aso_sq_create(cdev, &sh->ct_mng->aso_sq, + sh->tx_uar.obj)) { mlx5_aso_dereg_mr(cdev, &sh->ct_mng->aso_sq.mr); return -1; } -- 2.25.1