From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 383AAA034F for ; Wed, 10 Nov 2021 07:52:31 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2E93D40142; Wed, 10 Nov 2021 07:52:31 +0100 (CET) Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2084.outbound.protection.outlook.com [40.107.94.84]) by mails.dpdk.org (Postfix) with ESMTP id 5310840E03 for ; Wed, 10 Nov 2021 07:52:30 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=EzNWwV11WE+VrLpF3sOsY0ivkM+BltR+shHXLSYMp15DiE172MWlMErpcBTnLUNccj/j3OSbo1D49P5OwwE9ELpwQYNVjh0JppBN0V4mWmH+slkNMoxiXz1VOfOZPnefLg4qw8kGcD3uIXAwiiyyeR8+Gl9lp4nduru+x4A1tCFp1M7vsoCxr0FKoEDpPPIn1ubwjWwAkVe5zP0iXM0thn21lDY+ef3PtFc1E1EPI20GBCnSpagkyrUmQQ9lmTRhxUqW+KsgP3gIireltmy10hCFNzD1DhrxJHkB7D+mUv21YGFwl6HGhXMamWgmVY1qSKcY+5JKzA0jF07BNFfivg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=PZKsflURvlbfp4CLURsbHqk7AaUYuv/k1qeFqUoBmYk=; b=Hw0FE5NmQjhIK5J/ByvaCDKzoPWQMY4fPMUO3kMn16NW0qoWE0rPhOZxNGfMpq1OVPanqq7ywwDRDA52AfOiFnSuHE6jqUrlgllxYIBTK3+oUJXr9ZxX2T91/84e0sGdBP3/MWplasWXFLqEfxMhWutDw7X8UNFZZhkc31tKcd/8LnpfJ8I1medq/7kBp6mFf48dS03QjtwCi8uqDtYKMo21dzP0wN9n6bcpAZG+v+NUM+nCtsK1hhYy0aWsDd0k0Wrt/8FQNE+oWhREIiPTOJ0qE+b7GbICnx3Cc50j/5jg0WAMsz5SV2mIw9DDIlJXmeYhUymwwImLVPEeS+v2pA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=PZKsflURvlbfp4CLURsbHqk7AaUYuv/k1qeFqUoBmYk=; b=jcgHHvVDCqCHjBhl2Ew9Efl7SfTm7raQLDkktyj2DQqe4N7bbl4xpoKx/QjoaMEiUoaOGWT8bGmcnf7zFD0V85rj2bkML2siqweEwZKHZnzCIfNxz3+4ktgLJo/U1VSCmpAQwVG1XxtyK5ABB1R5FlBKzTTO6WOqO/56gYIq8CMv+reZFlTONbldPtRMHnX5am4pOmNsFwxQHBDf6ncnxbVctsJFTvzg28KI122Ly3b6lKQ2lxaroiQMt4O5dFj/mmRmERvMX/6O7/nPbnR8D7Q7HB49ly0X1aWO3WxkFiEC2dTUM96fYD9Yb1sul70iVbLxR8Bw/5chcmbE17wqgA== Received: from BN0PR04CA0038.namprd04.prod.outlook.com (2603:10b6:408:e8::13) by SA0PR12MB4414.namprd12.prod.outlook.com (2603:10b6:806:9a::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4669.11; Wed, 10 Nov 2021 06:52:28 +0000 Received: from BN8NAM11FT068.eop-nam11.prod.protection.outlook.com (2603:10b6:408:e8:cafe::82) by BN0PR04CA0038.outlook.office365.com (2603:10b6:408:e8::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4690.15 via Frontend Transport; Wed, 10 Nov 2021 06:52:28 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by BN8NAM11FT068.mail.protection.outlook.com (10.13.177.69) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4690.15 via Frontend Transport; Wed, 10 Nov 2021 06:52:27 +0000 Received: from nvidia.com (172.20.187.5) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Wed, 10 Nov 2021 06:52:15 +0000 From: Xueming Li To: Xueming Li CC: Luca Boccassi , Matan Azrad , "Maxime Coquelin" , dpdk stable Date: Wed, 10 Nov 2021 14:31:03 +0800 Message-ID: <20211110063216.2744012-180-xuemingl@nvidia.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20211110063216.2744012-1-xuemingl@nvidia.com> References: <20211110063216.2744012-1-xuemingl@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [172.20.187.5] X-ClientProxiedBy: HQMAIL101.nvidia.com (172.20.187.10) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 2aabbdd6-ad3a-4d7b-1b53-08d9a416a8cf X-MS-TrafficTypeDiagnostic: SA0PR12MB4414: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:9508; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: zgbFZd1GFmEAKHYfSgYJJJL2UNprC0bMFX9UBcreMfad79Bmflf8Bonr6d5vsaL4nJxJJcrnQoq+46TL/B1eq3yaJhRlJ+B0kNk4kbJiYXwdB8UfrIkI/15mQf8HWCZDHN9oWGOfrNatg8+zeHEPyW9FNfVdKudgtibe8z+yY2onE3rCU5lgL1ljUBQjAxlcw9c4sj983A4aw7XbE3mCaSpNr3TZ6hxZDodMZfwM+EzD6YLiyt/KRpkQuEwnLAhm2IHsV0n2wEO0Pe7lIq/7K6gFiBCi70QXs3dO2l0ssW2TR062eN2eIXof9GKmpMb9N4BImDJBc2Aa2J0AQsdEkgK+/4J8/UR2WSKLpjK3m+989MS9y8g2/f3CwCIvCyHvoApkALQwI3So/sds1S56w1vVzpvG2hGqbc8ap9BP/djT7r5OWq2mYtUvU2RNh9VYXLUFuwv0PzMy23xkI6WhfroOQ6YcssA+zWuXzxorLtCfhHExgS6izVJWzOfnblTiLCfaBvK+y+6qvG7WuiYR/lwqDq1SvLDebNwdaTkrrl/1AUKe13ikkXXBKyyuXR6jdhpxmScDy0LkM7NRkelOg/tLy1Fxb5RYUYIO2hvXJ6gsfZvDokX1i3Y1JlzfuoOh80uR3gwTKRYBuiwomTE+okGqHyaCjy4M17GQt8Cm61qn5MU/HzV/RqwT2Kf61qY0k/Ocg1rEshx+u5EMrLzLl19WpbRPEIfpCQ/HSuh2IKTT+8ltVGtaC2tOYyzOIDr+CKDdXdCYZzcfI4m4mznvsYN9qRgXKss1EY+luT1xTO53BZu6Rwe9vH1FjwGjg0BhRvih0Ak5Q6wMd61QLKRtTg== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(36840700001)(46966006)(7696005)(53546011)(55016002)(186003)(16526019)(36860700001)(47076005)(86362001)(36756003)(54906003)(4001150100001)(5660300002)(356005)(336012)(6666004)(6862004)(4326008)(6286002)(26005)(6200100001)(8936002)(1076003)(83380400001)(7636003)(82310400003)(316002)(8676002)(966005)(508600001)(70206006)(426003)(2906002)(2616005)(7049001)(70586007)(37006003); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Nov 2021 06:52:27.6317 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 2aabbdd6-ad3a-4d7b-1b53-08d9a416a8cf X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT068.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR12MB4414 Subject: [dpdk-stable] patch 'vdpa/mlx5: retry VAR allocation during vDPA restart' has been queued to stable release 20.11.4 X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: stable-bounces@dpdk.org Sender: "stable" Hi, FYI, your patch has been queued to stable release 20.11.4 Note it hasn't been pushed to http://dpdk.org/browse/dpdk-stable yet. It will be pushed if I get no objections before 11/12/21. So please shout if anyone has objections. Also note that after the patch there's a diff of the upstream commit vs the patch applied to the branch. This will indicate if there was any rebasing needed to apply to the stable branch. If there were code changes for rebasing (ie: not only metadata diffs), please double check that the rebase was correctly done. Queued patches are on a temporary branch at: https://github.com/steevenlee/dpdk This queued commit can be viewed at: https://github.com/steevenlee/dpdk/commit/5497fdbe2b4a9b452b42e7805b0ef7d8395ae86b Thanks. Xueming Li --- >From 5497fdbe2b4a9b452b42e7805b0ef7d8395ae86b Mon Sep 17 00:00:00 2001 From: Xueming Li Date: Fri, 15 Oct 2021 23:05:45 +0800 Subject: [PATCH] vdpa/mlx5: retry VAR allocation during vDPA restart Cc: Xueming Li [ upstream commit 8011a09adda4d99121bcc0fe34c98356559eb32f ] VAR is the device memory space for the virtio queues doorbells, Qemu could mmap it to directly to speed up doorbell push. On a busy system, Qemu takes time to release VAR resources during driver shutdown. If vdpa restarted quickly, the VAR allocation failed with error 28 since the VAR is singleton resource per device. This patch adds retry mechanism for VAR allocation. Fixes: 4cae722c1b06 ("vdpa/mlx5: move virtual doorbell alloc to probe") Signed-off-by: Xueming Li Reviewed-by: Matan Azrad Reviewed-by: Maxime Coquelin --- drivers/vdpa/mlx5/mlx5_vdpa.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/drivers/vdpa/mlx5/mlx5_vdpa.c b/drivers/vdpa/mlx5/mlx5_vdpa.c index 6029cfc3a8..38a93d041d 100644 --- a/drivers/vdpa/mlx5/mlx5_vdpa.c +++ b/drivers/vdpa/mlx5/mlx5_vdpa.c @@ -686,6 +686,7 @@ mlx5_vdpa_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, struct mlx5_vdpa_priv *priv = NULL; struct ibv_context *ctx = NULL; struct mlx5_hca_attr attr; + int retry; int ret; ibv = mlx5_vdpa_get_ib_device_match(&pci_dev->addr); @@ -740,7 +741,14 @@ mlx5_vdpa_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, priv->num_lag_ports = 1; priv->ctx = ctx; priv->pci_dev = pci_dev; - priv->var = mlx5_glue->dv_alloc_var(ctx, 0); + for (retry = 0; retry < 7; retry++) { + priv->var = mlx5_glue->dv_alloc_var(priv->ctx, 0); + if (priv->var != NULL) + break; + DRV_LOG(WARNING, "Failed to allocate VAR, retry %d.\n", retry); + /* Wait Qemu release VAR during vdpa restart, 0.1 sec based. */ + usleep(100000U << retry); + } if (!priv->var) { DRV_LOG(ERR, "Failed to allocate VAR %u.\n", errno); goto error; -- 2.33.0 --- Diff of the applied patch vs upstream commit (please double-check if non-empty: --- --- - 2021-11-10 14:17:09.880726918 +0800 +++ 0179-vdpa-mlx5-retry-VAR-allocation-during-vDPA-restart.patch 2021-11-10 14:17:01.990745109 +0800 @@ -1 +1 @@ -From 8011a09adda4d99121bcc0fe34c98356559eb32f Mon Sep 17 00:00:00 2001 +From 5497fdbe2b4a9b452b42e7805b0ef7d8395ae86b Mon Sep 17 00:00:00 2001 @@ -4,0 +5,3 @@ +Cc: Xueming Li + +[ upstream commit 8011a09adda4d99121bcc0fe34c98356559eb32f ] @@ -16 +18,0 @@ -Cc: stable@dpdk.org @@ -26 +28 @@ -index 3971f2e335..9a6f64797b 100644 +index 6029cfc3a8..38a93d041d 100644 @@ -29,2 +31 @@ -@@ -506,6 +506,7 @@ mlx5_vdpa_dev_probe(struct mlx5_common_device *cdev) - { +@@ -686,6 +686,7 @@ mlx5_vdpa_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, @@ -32 +33,2 @@ - struct mlx5_hca_attr *attr = &cdev->config.hca_attr; + struct ibv_context *ctx = NULL; + struct mlx5_hca_attr attr; @@ -33,0 +36 @@ + int ret; @@ -35,4 +38,2 @@ - if (!attr->vdpa.valid || !attr->vdpa.max_num_virtio_queues) { - DRV_LOG(ERR, "Not enough capabilities to support vdpa, maybe " -@@ -530,7 +531,14 @@ mlx5_vdpa_dev_probe(struct mlx5_common_device *cdev) - if (attr->num_lag_ports == 0) + ibv = mlx5_vdpa_get_ib_device_match(&pci_dev->addr); +@@ -740,7 +741,14 @@ mlx5_vdpa_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, @@ -40,2 +41,3 @@ - priv->cdev = cdev; -- priv->var = mlx5_glue->dv_alloc_var(priv->cdev->ctx, 0); + priv->ctx = ctx; + priv->pci_dev = pci_dev; +- priv->var = mlx5_glue->dv_alloc_var(ctx, 0); @@ -43 +45 @@ -+ priv->var = mlx5_glue->dv_alloc_var(priv->cdev->ctx, 0); ++ priv->var = mlx5_glue->dv_alloc_var(priv->ctx, 0); @@ -51 +53 @@ - DRV_LOG(ERR, "Failed to allocate VAR %u.", errno); + DRV_LOG(ERR, "Failed to allocate VAR %u.\n", errno);