From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B7AC8A0C43; Tue, 16 Nov 2021 08:38:53 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 852F440141; Tue, 16 Nov 2021 08:38:53 +0100 (CET) Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2069.outbound.protection.outlook.com [40.107.237.69]) by mails.dpdk.org (Postfix) with ESMTP id CA68B40040 for ; Tue, 16 Nov 2021 08:38:52 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=kQMqV3pswclkd8ATHL2dVXOZaKst9hEFIzN/D5iacGVq7LnS5N3tUgxbAGANnR8Zl8MyvaYWBnE54/nZTUn8yMuBb2B4f50lyQ9YaVmFpkkBEsnVMC0T7vtxc03W3op3K0PD4TtUOJ1ugP8dvDNyVVGxqUyon0UAuMBahCeckb25qpS2qWuU5odTKgly9OeqWAUQ6Twwx+7u8pWZsAgryHq4oCRQ6LwTruuXMy7jMNRmdt66SO/cqedH//XXIwCqUIOu32RhjRQJOMyGN2Piu6N0eeksrLGq8vpN7qM5xX5BK/LwAR7/VwqA74f4omLjWyLYeLF/IMKinsraiGi3GQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=/GxyQOC6NxHLh7PrfLqzvq0EKrdV7EiJnkB8F1zlfHM=; b=aFiC2HZMwu7OGvHOhbWnEc1t1INC7ZIPhKMi6CbyGfJm9CkffTA123O7Tw6/cBDtr9YxjVrcLzKmpuzOpmvBP3xBKK21CmtcnHne2C/FJNPw8aOua0DpRHKbYscb5teqIRizs9x243Ri7kK13fPbgzU8dUQPMSOQ0JmJPydiIBJbG7VbA2bvTVCn4oOnSSsvp3ow3HyTOY62VQRZiwe9PDqHmSrgeceyPxTNMZXlM8gQiyH2srjHxmcK1h2tbg0U4TZ+N+m/VMKVNFfzneO2bbywyT0hkDF3VOvIQLV9xhoPxcRSCv3s0EICt5gTiE7a+ahHShRbZ4bYqd+wbusGYw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/GxyQOC6NxHLh7PrfLqzvq0EKrdV7EiJnkB8F1zlfHM=; b=GvGzYFSL2OU3sv5WoZQYAJg2e/NcBrOrTPklntB5n3wyB37IsrlM6osv4yZIVjUJFdwHNfWt26a4StzUdz38eK3i2+buf17rUapxf+f+aV0E++3GT3Obc5WWXbV/fGKoPjZUZ6i97xQ2RxUXki+f2/V/fzZp5Cfr2bE9KcT7lGYlvPWDSDqeQeXAFV2lHi0qDkwF8veTTRXqq9ILKFqqBSyVqxaZdNt3tIXpZ1ttJj65SWNyJNP21yApqEujDDe3T+2MTq+EnoKgCZcNzUg0RUIisVqREeiwmmwdoF0pvfsx9pPv6jHzgr0gVd+RnWGQ5ph0UqealmZRVHzKJ08JTw== Received: from DM6PR06CA0038.namprd06.prod.outlook.com (2603:10b6:5:54::15) by MN2PR12MB4534.namprd12.prod.outlook.com (2603:10b6:208:24f::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4690.27; Tue, 16 Nov 2021 07:38:51 +0000 Received: from DM6NAM11FT039.eop-nam11.prod.protection.outlook.com (2603:10b6:5:54:cafe::99) by DM6PR06CA0038.outlook.office365.com (2603:10b6:5:54::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4690.17 via Frontend Transport; Tue, 16 Nov 2021 07:38:51 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by DM6NAM11FT039.mail.protection.outlook.com (10.13.172.83) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4690.15 via Frontend Transport; Tue, 16 Nov 2021 07:38:50 +0000 Received: from nvidia.com (172.20.187.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Tue, 16 Nov 2021 07:38:48 +0000 From: Dmitry Kozlyuk To: CC: Matan Azrad , Ori Kam , "Raslan Darawsheh" , , Viacheslav Ovsiienko Subject: [PATCH] net/mlx5: fix keeping indirect RSS non-isolated mode Date: Tue, 16 Nov 2021 09:38:34 +0200 Message-ID: <20211116073834.2413952-1-dkozlyuk@nvidia.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL111.nvidia.com (172.20.187.18) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: dc9c9eaf-6245-4e02-5cf9-08d9a8d4222e X-MS-TrafficTypeDiagnostic: MN2PR12MB4534: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:2000; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 4Jq1Ca/H1TcSWHLIFv6IOUzNATcj27/qpy5tgUfBVrUkkGCTMkuokU9htGHnhdD7eddGaBMnQ/CJgm4GxNb84ZvO8sye2sgciTgy1Y6orie4D5wYZb3GmjUzXqAbOKRXoTaf+JtlDzOS1zhrKAToX/IApsZE9aZ2UGO1lqzBr5dZb88M1a9/oXrOFt23MujvQjAbfj4KjDpbKowq6BXCn0pa0WaUQb1wCPB8Bc5l4FIcMOfc9u7u+VWdrsXYtbfsmHFjc02gICuSZTwuc+nTus0KUG8IisnJxpa0fYxAH1xOAxutwlV375X4YIUBxTF6JwAMsNjR2V1blUXBrkBoVuUgy9U+xyaXzD8AveRoKOTGrS54u+lwN3EMoGmzj9oxI/wjbnZY6/bflmmSr9isdZExy9AYChXA3tMOzfoJHYoe7/Vcpll4Ui1+u+9V5ycBR1vKX+cMHxVAlITXJAepek0Ot89muhSZir026rtZ3WeoZIZsbMMDLEhScnba0acJfJPc4ZNEiP1ZY0c5hFpUq+h+ix6B07Ywj2H9pJ6M7x33Qu61Ur7isuOlP7GFXE2rBO0x3WAquvZDyqdHTU+vvcSR5Cyg2wOz+CQamL1xh+FnmQHN9H2eOcFwfuE+OQoW9EQxeeNsTEdv6J9+YmQgBoQ/dZwb9hvJsGGIbyDD66pM8MY1EmgWNyarozFSOtXUn6AfBBqK8ceHFNMXpKS0VA== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(36840700001)(46966006)(8676002)(70586007)(356005)(107886003)(7636003)(55016002)(508600001)(82310400003)(86362001)(47076005)(36756003)(70206006)(8936002)(336012)(316002)(6916009)(5660300002)(7696005)(6666004)(1076003)(2906002)(4326008)(16526019)(36860700001)(2616005)(6286002)(54906003)(186003)(83380400001)(426003)(26005); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Nov 2021 07:38:50.8643 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: dc9c9eaf-6245-4e02-5cf9-08d9a8d4222e X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT039.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4534 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org When a port starts in non-isolated mode, an internal indirect RSS is created that includes all configured queues and a flow rule is created that references this indirect RSS. If before switching to non-isolated mode an indirect RSS was created that includes the same set of queues, it would be reused at this point. However, because the port had been stopped (or not yet started), the TIR for this indirect RSS had been destroyed (or not yet created). The flow rule could not be created and the port start failed. Creation of TIRs is moved before configuring non-isolated mode flows, but it is not enough because of the following issue. Commit 0cedf34da78f ("net/mlx5: move Rx queue reference count") changed mlx5_rxq_get() not to increment RxQ control structure reference count, mlx5_rxq_ref() was introduced for this purpose. mlx5_ind_table_obj_attach() was not updated to use the new function, so when the port was stopped, the control structure reference count of an RxQ used in RSS reached zero and the structure was destroyed. Use mlx5_rxq_ref() to keep RxQ control structure needed for indirect RSS persistence across port restart. Fixes: ec4e11d41d12 ("net/mlx5: preserve indirect actions on restart") Fixes: 0cedf34da78f ("net/mlx5: move Rx queue reference count") Cc: xuemingl@nvidia.com Signed-off-by: Dmitry Kozlyuk Acked-by: Viacheslav Ovsiienko --- drivers/net/mlx5/mlx5_rxq.c | 2 +- drivers/net/mlx5/mlx5_trigger.c | 19 +++++++++++-------- 2 files changed, 12 insertions(+), 9 deletions(-) diff --git a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c index 52b95d7070..d5a7155392 100644 --- a/drivers/net/mlx5/mlx5_rxq.c +++ b/drivers/net/mlx5/mlx5_rxq.c @@ -2438,7 +2438,7 @@ mlx5_ind_table_obj_attach(struct rte_eth_dev *dev, return ret; } for (i = 0; i < ind_tbl->queues_n; i++) - mlx5_rxq_get(dev, ind_tbl->queues[i]); + mlx5_rxq_ref(dev, ind_tbl->queues[i]); return 0; } diff --git a/drivers/net/mlx5/mlx5_trigger.c b/drivers/net/mlx5/mlx5_trigger.c index 1952d68444..65caa5ac14 100644 --- a/drivers/net/mlx5/mlx5_trigger.c +++ b/drivers/net/mlx5/mlx5_trigger.c @@ -1172,6 +1172,17 @@ mlx5_dev_start(struct rte_eth_dev *dev) goto error; } mlx5_os_stats_init(dev); + /* + * Attach indirection table objects detached on port stop. + * They may be needed to create RSS in non-isolated mode. + */ + ret = mlx5_action_handle_attach(dev); + if (ret) { + DRV_LOG(ERR, + "port %u failed to attach indirect actions: %s", + dev->data->port_id, rte_strerror(rte_errno)); + goto error; + } ret = mlx5_traffic_enable(dev); if (ret) { DRV_LOG(ERR, "port %u failed to set defaults flows", @@ -1184,14 +1195,6 @@ mlx5_dev_start(struct rte_eth_dev *dev) mlx5_rxq_timestamp_set(dev); /* Set a mask and offset of scheduling on timestamp into Tx queues. */ mlx5_txq_dynf_timestamp_set(dev); - /* Attach indirection table objects detached on port stop. */ - ret = mlx5_action_handle_attach(dev); - if (ret) { - DRV_LOG(ERR, - "port %u failed to attach indirect actions: %s", - dev->data->port_id, rte_strerror(rte_errno)); - goto error; - } /* * In non-cached mode, it only needs to start the default mreg copy * action and no flow created by application exists anymore. -- 2.25.1