From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id F310FA0548 for ; Wed, 10 Nov 2021 07:54:27 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E827540E28; Wed, 10 Nov 2021 07:54:27 +0100 (CET) Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2067.outbound.protection.outlook.com [40.107.94.67]) by mails.dpdk.org (Postfix) with ESMTP id D53B840683 for ; Wed, 10 Nov 2021 07:54:25 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Y774wSHqbsOGOVEEV6VUNcl20fdVjJZox85Sn1KjP28p6QQg7+DLQYI9Gh8g2wgnGqGldHy02EUWt/wh6kCKnFdW6DBbU9Nl9TZLAluzQOSikXzxcMJvM7+7JBQfJuptZgSqaUgjowW8PWuTR4DrAMbERY0KOQJVb18R9+SyvwpWnAEXJxvwDcie1p2eN+zCJydR8p6vXdb0YRm2bOKQQzArFZQMZ9IySfvlwFYXcRhLtYBwZi1mChy3cyrbqkOsF9p7waMC/UGnr/LhO+3tswMLlGNMSTM+B23Wx9+vL2heSIHE/bgR2xKwX9BwuM9IG+clRKB2RHJhxZETE2WCMA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=rlQJebBQCXLcuAaryz6B9XFsctT/CA/tWu1XeMAXmwU=; b=QTgtaa7Qe33T0lpzgHOcK5LfufrTJCrMZuhxiTfx2cbDx0ndGFHfR58Ev2U7LrK3xjR2WwDCZbEJ2W09mX79+oRuDOw5JKtrNhvutRXgXkK6hSo51uplDlIQpvgxZX+IJjMeV4CdUhrXE4sFxp5TNBaCs5/aLramx5nwY2zcj3SjCrclHacwAg5PutWScCVvqZncW3SJGDUSZY3NTkpvB/puK5okNDyB3UCY3TW8D8TzjrZzxML0jVUE4A7RqN3ygr4UW0TntqD0T/Fg2SFP8pYvuod6A6NrEcs36bE7Q5GsiLyvjHwWiBI986JG7Ps3A4SSaRk+tIZDM3WYGxpPMw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=rlQJebBQCXLcuAaryz6B9XFsctT/CA/tWu1XeMAXmwU=; b=ioTwY7jMBsoiobBtRWygsWyRtewxskEDVd3vvoMbLbkVBb3iizSA0fDqBjhJ3rghs5SZoakhmIgBx3o7GrDEKCxShnRS3BzU6W4IZ6h5u3DfouRVij9YHHFIipnzspIosiD5FpmoC+hHYk4GQgOGwvZXDYqQ6y6/A8crJJs5MyyzHhvhEVik1AFa+hmjC60uHZZtgCjsxICR/bdx4+DsUu6CBmpBf4Fiw+BxwaRLtYn6nZ042KQm9qjUOjIXfPVSBBVE1SWQYh50gw9HYvkeP13DaqRMHCmIaG5Gr+PkGLZfpRmOD0BMdEobB3Ik4l4Qgj8oI1dtwnuZD/2uCAkATw== Received: from BN9PR03CA0271.namprd03.prod.outlook.com (2603:10b6:408:f5::6) by BYAPR12MB3237.namprd12.prod.outlook.com (2603:10b6:a03:13a::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4669.13; Wed, 10 Nov 2021 06:54:22 +0000 Received: from BN8NAM11FT058.eop-nam11.prod.protection.outlook.com (2603:10b6:408:f5:cafe::cb) by BN9PR03CA0271.outlook.office365.com (2603:10b6:408:f5::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4690.15 via Frontend Transport; Wed, 10 Nov 2021 06:54:22 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by BN8NAM11FT058.mail.protection.outlook.com (10.13.177.58) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4690.15 via Frontend Transport; Wed, 10 Nov 2021 06:54:21 +0000 Received: from nvidia.com (172.20.187.5) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Wed, 10 Nov 2021 06:54:18 +0000 From: Xueming Li To: Dmitry Kozlyuk CC: Luca Boccassi , Tal Shnaiderman , Pallavi Kadam , dpdk stable Date: Wed, 10 Nov 2021 14:31:20 +0800 Message-ID: <20211110063216.2744012-197-xuemingl@nvidia.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20211110063216.2744012-1-xuemingl@nvidia.com> References: <20211110063216.2744012-1-xuemingl@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [172.20.187.5] X-ClientProxiedBy: HQMAIL101.nvidia.com (172.20.187.10) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 4085e58c-b3ba-40d2-a325-08d9a416eceb X-MS-TrafficTypeDiagnostic: BYAPR12MB3237: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:10000; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: mWsTbzFbZEy84kwIpczKgYsEc6fAi2rAr370TgaLkc6bhkOTwiO9wMazB7lxRNMEHGhX9gLLanDBGwqvtvd+jugx93O63Er2pETEvqfv6neoa2zEO2a8G/x/AdyFRbVQp/2oAGYXqVJS2ZbO2sEHpxa58YtPn/74+zzQdQmNJCjepl+jgC6NMCKZv7xMjLXVFWAevcKNR5uxd3+whqUNjZ6Z0ergIQ/CRn0ILRU2BXz2H/aByclAaSpEiLsoCQFyNqUnV4ii4UH1Xs6zbz2tCfLg7Qoon3inCbheCvjyyI++a/jJn2o3ibmBz2lK59PfborWH6BFT9TQ7n/NsP+qPL3Q9cUDipzmLXyaZm5GCN/RA5lYklwgDuW2n87Nx2hHSiYTGKtZxbEmGfhxQ3I+lo6S7cxxDjxJGgMatfUclWePC+e/NIRjNZhEvW1kI8nQrnT5TmSL4BKyOQg2K+T3/Ly69RVRmkB0qRyZpuHoRQqW1llxi8V2eGZGmzulnvNzUbH9+A55N4IF4g7m/hOyNDeZhNSSGjN1NdGpj3CNt+X/vQbrPds8IF4LQd0SJSWtAH1Jrcv9E1cBtFzxHl0bOOHYlEHzlms909Kvb6Hev1a7E+y2IrSwxhrc4OcpGEGrLN84KsA3gv/0DmNvMx0Qxu5HA1rJj4PcGELNIoLK67ylURaB3joCKyG/Ad8ADnD2F6iGvGGO2d96Yl7t1Qew//WIOFGH0c/a6qQ5j4TUqkms+TqmSsTmnTUYxcq2hpGLih3uPllG491T0lUKYvFGQoIJDa7cmD7J0fJyThXuvx9OljDwtxKu79u+g8/GlEawIhSPpXR3PeFdEfFpwrWqJQ== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(36840700001)(46966006)(36860700001)(54906003)(5660300002)(70206006)(70586007)(55016002)(82310400003)(7696005)(4001150100001)(426003)(37006003)(316002)(6636002)(4326008)(8676002)(2906002)(26005)(356005)(86362001)(7636003)(6666004)(53546011)(47076005)(36756003)(1076003)(336012)(508600001)(83380400001)(6862004)(186003)(966005)(16526019)(6286002)(2616005)(8936002); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Nov 2021 06:54:21.8988 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 4085e58c-b3ba-40d2-a325-08d9a416eceb X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT058.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR12MB3237 Subject: [dpdk-stable] patch 'eal/windows: fix IOVA mode detection and handling' has been queued to stable release 20.11.4 X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: stable-bounces@dpdk.org Sender: "stable" Hi, FYI, your patch has been queued to stable release 20.11.4 Note it hasn't been pushed to http://dpdk.org/browse/dpdk-stable yet. It will be pushed if I get no objections before 11/12/21. So please shout if anyone has objections. Also note that after the patch there's a diff of the upstream commit vs the patch applied to the branch. This will indicate if there was any rebasing needed to apply to the stable branch. If there were code changes for rebasing (ie: not only metadata diffs), please double check that the rebase was correctly done. Queued patches are on a temporary branch at: https://github.com/steevenlee/dpdk This queued commit can be viewed at: https://github.com/steevenlee/dpdk/commit/6ddd5388c2713f9a451450d3dd83d41dec81e700 Thanks. Xueming Li --- >From 6ddd5388c2713f9a451450d3dd83d41dec81e700 Mon Sep 17 00:00:00 2001 From: Dmitry Kozlyuk Date: Mon, 25 Oct 2021 15:20:52 +0300 Subject: [PATCH] eal/windows: fix IOVA mode detection and handling Cc: Xueming Li [ upstream commit 0c8fc83a71f37107ad0b205b2a176fde9188390e ] Windows EAL did not detect IOVA mode and worked incorrectly if physical addresses could not be obtained (if virt2phys driver was missing or inaccessible). In this case, rte_mem_virt2iova() reported RTE_BAD_IOVA for any address. Inability to obtain IOVA, be it PA or VA, should cause a failure for the DPDK allocator, but it was hidden by the implementation, so allocations did not fail when they should. The mode when DPDK cannot obtain PA but can work is IOVA-as-VA mode. However, rte_eal_iova_mode() always returned RTE_IOVA_DC (while it should only ever return RTE_IOVA_PA or RTE_IOVA_VA), because IOVA mode detection was not implemented. Implement IOVA mode detection: 1. Always allow to force --iova-mode=va. 2. Allow to force --iova-mode=pa only if virt2phys is available. 3. If no mode is forced and virt2phys is available, select the mode according to bus requests, default to PA. 4. If no mode is forced but virt2phys is unavailable, default to VA. Fix rte_mem_virt2iova() by returning VA when using IOVA-as-VA. Fix rte_eal_iova_mode() by returning the selected mode. Fixes: 2a5d547a4a9b ("eal/windows: implement basic memory management") Reported-by: Tal Shnaiderman Signed-off-by: Dmitry Kozlyuk Tested-by: Pallavi Kadam Acked-by: Pallavi Kadam --- lib/librte_eal/windows/eal.c | 63 +++++++++++++++++++-------- lib/librte_eal/windows/eal_memalloc.c | 15 +++---- lib/librte_eal/windows/eal_memory.c | 6 +-- 3 files changed, 51 insertions(+), 33 deletions(-) diff --git a/lib/librte_eal/windows/eal.c b/lib/librte_eal/windows/eal.c index 5b6d13481a..abe4c4038d 100644 --- a/lib/librte_eal/windows/eal.c +++ b/lib/librte_eal/windows/eal.c @@ -269,6 +269,8 @@ rte_eal_init(int argc, char **argv) const struct rte_config *config = rte_eal_get_configuration(); struct internal_config *internal_conf = eal_get_internal_configuration(); + bool has_phys_addr; + enum rte_iova_mode iova_mode; int ret; rte_eal_log_init(NULL, 0); @@ -315,18 +317,59 @@ rte_eal_init(int argc, char **argv) internal_conf->memory = MEMSIZE_IF_NO_HUGE_PAGE; } + if (rte_eal_intr_init() < 0) { + rte_eal_init_alert("Cannot init interrupt-handling thread"); + return -1; + } + + if (rte_eal_timer_init() < 0) { + rte_eal_init_alert("Cannot init TSC timer"); + rte_errno = EFAULT; + return -1; + } + + bscan = rte_bus_scan(); + if (bscan < 0) { + rte_eal_init_alert("Cannot scan the buses"); + rte_errno = ENODEV; + return -1; + } + if (eal_mem_win32api_init() < 0) { rte_eal_init_alert("Cannot access Win32 memory management"); rte_errno = ENOTSUP; return -1; } + has_phys_addr = true; if (eal_mem_virt2iova_init() < 0) { /* Non-fatal error if physical addresses are not required. */ - RTE_LOG(WARNING, EAL, "Cannot access virt2phys driver, " + RTE_LOG(DEBUG, EAL, "Cannot access virt2phys driver, " "PA will not be available\n"); + has_phys_addr = false; } + iova_mode = internal_conf->iova_mode; + if (iova_mode == RTE_IOVA_PA && !has_phys_addr) { + rte_eal_init_alert("Cannot use IOVA as 'PA' since physical addresses are not available"); + rte_errno = EINVAL; + return -1; + } + if (iova_mode == RTE_IOVA_DC) { + RTE_LOG(DEBUG, EAL, "Specific IOVA mode is not requested, autodetecting\n"); + if (has_phys_addr) { + RTE_LOG(DEBUG, EAL, "Selecting IOVA mode according to bus requests\n"); + iova_mode = rte_bus_get_iommu_class(); + if (iova_mode == RTE_IOVA_DC) + iova_mode = RTE_IOVA_PA; + } else { + iova_mode = RTE_IOVA_VA; + } + } + RTE_LOG(DEBUG, EAL, "Selected IOVA mode '%s'\n", + iova_mode == RTE_IOVA_PA ? "PA" : "VA"); + rte_eal_get_configuration()->iova_mode = iova_mode; + if (rte_eal_memzone_init() < 0) { rte_eal_init_alert("Cannot init memzone"); rte_errno = ENODEV; @@ -351,27 +394,9 @@ rte_eal_init(int argc, char **argv) return -1; } - if (rte_eal_intr_init() < 0) { - rte_eal_init_alert("Cannot init interrupt-handling thread"); - return -1; - } - - if (rte_eal_timer_init() < 0) { - rte_eal_init_alert("Cannot init TSC timer"); - rte_errno = EFAULT; - return -1; - } - __rte_thread_init(config->main_lcore, &lcore_config[config->main_lcore].cpuset); - bscan = rte_bus_scan(); - if (bscan < 0) { - rte_eal_init_alert("Cannot init PCI"); - rte_errno = ENODEV; - return -1; - } - RTE_LCORE_FOREACH_WORKER(i) { /* diff --git a/lib/librte_eal/windows/eal_memalloc.c b/lib/librte_eal/windows/eal_memalloc.c index d8cae3ebc1..d1e49f0be8 100644 --- a/lib/librte_eal/windows/eal_memalloc.c +++ b/lib/librte_eal/windows/eal_memalloc.c @@ -100,16 +100,11 @@ alloc_seg(struct rte_memseg *ms, void *requested_addr, int socket_id, */ *(volatile int *)addr = *(volatile int *)addr; - /* Only try to obtain IOVA if it's available, so that applications - * that do not need IOVA can use this allocator. - */ - if (rte_eal_using_phys_addrs()) { - iova = rte_mem_virt2iova(addr); - if (iova == RTE_BAD_IOVA) { - RTE_LOG(DEBUG, EAL, - "Cannot get IOVA of allocated segment\n"); - goto error; - } + iova = rte_mem_virt2iova(addr); + if (iova == RTE_BAD_IOVA) { + RTE_LOG(DEBUG, EAL, + "Cannot get IOVA of allocated segment\n"); + goto error; } /* Only "Ex" function can handle hugepages. */ diff --git a/lib/librte_eal/windows/eal_memory.c b/lib/librte_eal/windows/eal_memory.c index 71741fc07e..2fd37d9708 100644 --- a/lib/librte_eal/windows/eal_memory.c +++ b/lib/librte_eal/windows/eal_memory.c @@ -225,19 +225,17 @@ rte_mem_virt2phy(const void *virt) return phys.QuadPart; } -/* Windows currently only supports IOVA as PA. */ rte_iova_t rte_mem_virt2iova(const void *virt) { phys_addr_t phys; - if (virt2phys_device == INVALID_HANDLE_VALUE) - return RTE_BAD_IOVA; + if (rte_eal_iova_mode() == RTE_IOVA_VA) + return (rte_iova_t)virt; phys = rte_mem_virt2phy(virt); if (phys == RTE_BAD_PHYS_ADDR) return RTE_BAD_IOVA; - return (rte_iova_t)phys; } -- 2.33.0 --- Diff of the applied patch vs upstream commit (please double-check if non-empty: --- --- - 2021-11-10 14:17:10.602349960 +0800 +++ 0196-eal-windows-fix-IOVA-mode-detection-and-handling.patch 2021-11-10 14:17:02.004078333 +0800 @@ -1 +1 @@ -From 0c8fc83a71f37107ad0b205b2a176fde9188390e Mon Sep 17 00:00:00 2001 +From 6ddd5388c2713f9a451450d3dd83d41dec81e700 Mon Sep 17 00:00:00 2001 @@ -4,0 +5,3 @@ +Cc: Xueming Li + +[ upstream commit 0c8fc83a71f37107ad0b205b2a176fde9188390e ] @@ -28 +30,0 @@ -Cc: stable@dpdk.org @@ -35,3 +37,3 @@ - lib/eal/windows/eal.c | 63 ++++++++++++++++++++++++---------- - lib/eal/windows/eal_memalloc.c | 15 +++----- - lib/eal/windows/eal_memory.c | 6 ++-- + lib/librte_eal/windows/eal.c | 63 +++++++++++++++++++-------- + lib/librte_eal/windows/eal_memalloc.c | 15 +++---- + lib/librte_eal/windows/eal_memory.c | 6 +-- @@ -40,5 +42,5 @@ -diff --git a/lib/eal/windows/eal.c b/lib/eal/windows/eal.c -index 3d8c520412..f7ce1b6671 100644 ---- a/lib/eal/windows/eal.c -+++ b/lib/eal/windows/eal.c -@@ -276,6 +276,8 @@ rte_eal_init(int argc, char **argv) +diff --git a/lib/librte_eal/windows/eal.c b/lib/librte_eal/windows/eal.c +index 5b6d13481a..abe4c4038d 100644 +--- a/lib/librte_eal/windows/eal.c ++++ b/lib/librte_eal/windows/eal.c +@@ -269,6 +269,8 @@ rte_eal_init(int argc, char **argv) @@ -52,2 +54,2 @@ - eal_log_init(NULL, 0); -@@ -322,18 +324,59 @@ rte_eal_init(int argc, char **argv) + rte_eal_log_init(NULL, 0); +@@ -315,18 +317,59 @@ rte_eal_init(int argc, char **argv) @@ -114 +116 @@ -@@ -358,27 +401,9 @@ rte_eal_init(int argc, char **argv) +@@ -351,27 +394,9 @@ rte_eal_init(int argc, char **argv) @@ -142,5 +144,5 @@ -diff --git a/lib/eal/windows/eal_memalloc.c b/lib/eal/windows/eal_memalloc.c -index 4459d59b1a..55d6dcc71c 100644 ---- a/lib/eal/windows/eal_memalloc.c -+++ b/lib/eal/windows/eal_memalloc.c -@@ -99,16 +99,11 @@ alloc_seg(struct rte_memseg *ms, void *requested_addr, int socket_id, +diff --git a/lib/librte_eal/windows/eal_memalloc.c b/lib/librte_eal/windows/eal_memalloc.c +index d8cae3ebc1..d1e49f0be8 100644 +--- a/lib/librte_eal/windows/eal_memalloc.c ++++ b/lib/librte_eal/windows/eal_memalloc.c +@@ -100,16 +100,11 @@ alloc_seg(struct rte_memseg *ms, void *requested_addr, int socket_id, @@ -168 +170 @@ -diff --git a/lib/eal/windows/eal_memory.c b/lib/eal/windows/eal_memory.c +diff --git a/lib/librte_eal/windows/eal_memory.c b/lib/librte_eal/windows/eal_memory.c @@ -170,2 +172,2 @@ ---- a/lib/eal/windows/eal_memory.c -+++ b/lib/eal/windows/eal_memory.c +--- a/lib/librte_eal/windows/eal_memory.c ++++ b/lib/librte_eal/windows/eal_memory.c