From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C33C4A0C43; Fri, 22 Oct 2021 09:59:59 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 88E0141103; Fri, 22 Oct 2021 09:59:59 +0200 (CEST) Received: from AZHDRRW-EX02.NVIDIA.COM (azhdrrw-ex02.nvidia.com [20.64.145.131]) by mails.dpdk.org (Postfix) with ESMTP id B21834069D for ; Fri, 22 Oct 2021 09:59:57 +0200 (CEST) Received: from NAM11-DM6-obe.outbound.protection.outlook.com (104.47.57.171) by mxs.oss.nvidia.com (10.13.234.37) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.858.15; Fri, 22 Oct 2021 00:59:56 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=NHVLKDXrOegK/Euz/aDfLK3s5Hv3W8nP1vsnV7pGsVua+FaHJu1833zuPDXMPnPh4Y77prXYVlBiR67gthyKZfiCOfg3+UJj3VtkzUTPGNXH2LYsGazUDiNxwzTJ4mcFBpaZQ+ZMHbcSfTfroxQav3IeqzKrqWfhvca5SvjNlsKY9aqLG80waW6TJ0e0LJ0a+oyyzHfZ6uV2JokY5cMSolb3pvGSEQt3KprCgBWz7E/Za0GEgt3QYUeFRg5olNo/yvJbOlwmJaGSmaX5x+lZqhp5wx6I+wCFymbFaYGh13Ldq+YCcUYawyn5dJOtLz28ZWIrTqt6UaoNTzwLw9Dfww== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=k3tpRn26QrNeiFTPbZAWOwDwC1YpzYNIAbjLxpiwWfc=; b=n13NvkurOLwqkMFWVYml5c87VvIi2xpwMNHIQJoEs9VvQqeisArYMR3LCBlaxzTiUtKqTCT1rCDvNTWqrg6lv9+dwCsO3UdvFJ+fQyxlWfGfm50hgh8UIKMHN8qRYOsN131XQQvnhMCJXHoO4FnOF61iBaGuaciTqg/goRhQCJt25ZGrLKtkEB1fGQP2k8iQE1Ozq5g6LlsGyMDjCoEK8hNBMos0wEaOQXkTOoi5TX6DnC3hnYKodqcOhW/LIgiQAEfbeKBbnIeyhYXZzeCEmuBrP0BJEn/gdnK0gpRsL9Qr6j1GegIon3p9jdyqKBybiEdf5hbXbe2MWjQCRBx1gQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=oktetlabs.ru smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=k3tpRn26QrNeiFTPbZAWOwDwC1YpzYNIAbjLxpiwWfc=; b=CPzD1aCWjwvyMxvdpTL53zNCmvjKYrAVl6Ac8Ei9TrerEeuhItmESc73cCvDWm40kjJQNcJzPmDwIM82MSikpvR0PZZB7d0eWzlRrGRSyUdQxiG5H2WOpKpbsZGVkAukFBZeoLdu9pQMeYojR1YY9iUiEcs8/HSDVkvjZupOU8proAokaSelK/ChlHVKPxBYQIEysOcRbQ5u3POhqBOENkcR5HRYGK2BfTVLHYIYVv8HJuvfHwae9rrmNuVSddwfjqMCPvJXHmUjCHgU9R45h8fboblopxkVOZZ3rVZS/htbF42JLwnFkNCOusKwz1offgVdp7DK41luRS9uFbnsKg== Received: from MW4P222CA0005.NAMP222.PROD.OUTLOOK.COM (2603:10b6:303:114::10) by BN6PR1201MB2513.namprd12.prod.outlook.com (2603:10b6:404:ae::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4608.18; Fri, 22 Oct 2021 07:59:55 +0000 Received: from CO1NAM11FT010.eop-nam11.prod.protection.outlook.com (2603:10b6:303:114:cafe::13) by MW4P222CA0005.outlook.office365.com (2603:10b6:303:114::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4628.18 via Frontend Transport; Fri, 22 Oct 2021 07:59:54 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; oktetlabs.ru; dkim=none (message not signed) header.d=none;oktetlabs.ru; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by CO1NAM11FT010.mail.protection.outlook.com (10.13.175.88) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4628.16 via Frontend Transport; Fri, 22 Oct 2021 07:59:53 +0000 Received: from nvidia.com (172.20.187.5) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Fri, 22 Oct 2021 07:59:51 +0000 From: Dmitry Kozlyuk To: CC: Tal Shnaiderman , Olivier Matz , Andrew Rybchenko Date: Fri, 22 Oct 2021 10:59:37 +0300 Message-ID: <20211022075937.52983-1-dkozlyuk@nvidia.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [172.20.187.5] X-ClientProxiedBy: HQMAIL111.nvidia.com (172.20.187.18) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: b0c72e31-be13-45f6-0cde-08d99531ee5d X-MS-TrafficTypeDiagnostic: BN6PR1201MB2513: X-Microsoft-Antispam-PRVS: X-MS-Exchange-Transport-Forked: True X-MS-Oob-TLC-OOBClassifiers: OLM:4941; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 84bIaLQykfqPfyyklau5qBvZ+f3JrOJN/YgJYxXKeptxwt9hCDoPqYmMNbc1jAJL1Myr9egsNsfml4ppxgVf+Namn+pR/+NNeYcAohzjOiZu4g2uvZFHjHPzf4f8TVDLjED1ouNDkiAinbyLzKnu6qgu6AlrKoThMkedtqYrkDP0hy3d6c3aFp8oAWz4FPdfkbUdjVoYoJSLjPPxDz9AJ4oinWi11LltOp8BqVx64BfoksBvfDVECDtX90+XSujKrjPT4rCJKj/QXZtkcaf47CvUR33obxoT0vJHGX5BK+vaFiI930fCcjVcR4YL3faq3RyMnefMUYzit+DhybWV06kDSQ58Ise+LXq+QYWyHfyPKQKzFm/zEV/LZcJFBDwSDA4Zukq9lw6GSOXFvLSl1K19+Vv1HgKIzfyvksMvQ8F6f9wUSEIe7H+z4HpCIq2iSsh8x7lcYMfhO/lO3cX2ee+x2i9MM/sd8VgOiSXqV6kGdC1LPtGZfXGn+763x8Vp8S5NLGB8BxTSlJJKlxPuMbe+Hd8Av3baLUD9rC5UlgBndFDYSeYOMhTpROotZfCzvETJ+sySIvjoqTtbv1oMA6CFaQXgCuNnFkuIzpROY/uY2Fqal9DJRqgNqte+Uk56McaOaF5KOsSdlOtyl3nge7hrME02TGANY9YDp0clRo5SvwklIUJPST20beVmq6BGEfVuzW6OtIYXNznPtuSTXg== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(36840700001)(46966006)(2616005)(426003)(16526019)(54906003)(70586007)(36756003)(2906002)(186003)(36860700001)(336012)(86362001)(6666004)(7696005)(8676002)(356005)(82310400003)(6286002)(316002)(508600001)(36906005)(5660300002)(47076005)(7636003)(107886003)(6916009)(1076003)(55016002)(83380400001)(4326008)(26005)(70206006)(8936002); DIR:OUT; SFP:1101; X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Oct 2021 07:59:53.3508 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b0c72e31-be13-45f6-0cde-08d99531ee5d X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT010.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR1201MB2513 Subject: [dpdk-dev] [PATCH] mempool: fix non-IO flag inference X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" When mempool had been created with RTE_MEMPOOL_F_NO_IOVA_CONTIG flag but later populated with valid IOVA, RTE_MEMPOOL_F_NON_IO was set, while it should not be. The unit test did not catch this because rte_mempool_populate_default() it used was populating with RTE_BAD_IOVA. When IOVA mode is IOVA-as-VA, RTE_MEMPOOL_NON_IO flag cannot be reliably inferred unless RTE_MEMPOOL_F_ON_IOVA_CONTIG is set. However, rte_mempool_populate_iova() was not checkig for IOVA mode. Keep setting RTE_MEMPOOL_NON_IO at an empty mempool creation and add an assert for it in the unit test (remove a separate test case). Reset the flag using the following logic: * Never reset if RTE_MEMPOOL_F_ON_IOVA_CONTIG is set. * Otherwise, always reset if IOVA mode is IOVA-as-VA. * Otherwise, reset if IOVA is RTE_BAD_IOVA. The unit test temporarily rewrites IOVA mode in EAL configuration before populationg mempools. While it theoretically can affect rte_malloc() inside rte_mempool_populate_iova() if it triggers a new hugepage mapping, it is unlikely to do so, because the allocation size is small. In exchange, the test run with any IOVA mode will check both modes. Fixes: 11541c5c81dd ("mempool: add non-IO flag") Reported-by: Tal Shnaiderman Signed-off-by: Dmitry Kozlyuk --- app/test/test_mempool.c | 88 ++++++++++++++++++++++++--------------- lib/mempool/rte_mempool.c | 7 +++- 2 files changed, 59 insertions(+), 36 deletions(-) diff --git a/app/test/test_mempool.c b/app/test/test_mempool.c index 4f399b461d..ae1692e6aa 100644 --- a/app/test/test_mempool.c +++ b/app/test/test_mempool.c @@ -28,6 +28,7 @@ #include #include #include +#include "eal_private.h" #include "test.h" @@ -738,36 +739,57 @@ test_mempool_events_safety(void) } while (0) static int -test_mempool_flag_non_io_set_when_no_iova_contig_set(void) +test_mempool_flag_non_io_with_no_iova_contig_set(enum rte_iova_mode mode) { + void *virt = NULL; + rte_iova_t iova; + size_t size = MEMPOOL_ELT_SIZE * 16; struct rte_mempool *mp = NULL; + enum rte_iova_mode saved_mode = rte_eal_iova_mode(); int ret; + virt = rte_malloc("test_mempool", size, rte_mem_page_size()); + RTE_TEST_ASSERT_NOT_NULL(virt, "Cannot allocate memory"); + iova = rte_mem_virt2iova(virt); + RTE_TEST_ASSERT_NOT_EQUAL(iova, RTE_BAD_IOVA, "Cannot get IOVA"); mp = rte_mempool_create_empty("empty", MEMPOOL_SIZE, MEMPOOL_ELT_SIZE, 0, 0, SOCKET_ID_ANY, RTE_MEMPOOL_F_NO_IOVA_CONTIG); RTE_TEST_ASSERT_NOT_NULL(mp, "Cannot create mempool: %s", rte_strerror(rte_errno)); rte_mempool_set_ops_byname(mp, rte_mbuf_best_mempool_ops(), NULL); - ret = rte_mempool_populate_default(mp); + + RTE_TEST_ASSERT(mp->flags & RTE_MEMPOOL_F_NON_IO, + "NON_IO flag is not set on an empty mempool"); + + /* + * Always use valid IOVA so that populate() has no other reason + * to infer that the mempool cannot be used for IO. + */ + rte_eal_get_configuration()->iova_mode = mode; + ret = rte_mempool_populate_iova(mp, virt, iova, size, NULL, NULL); RTE_TEST_ASSERT(ret > 0, "Failed to populate mempool: %s", rte_strerror(-ret)); RTE_TEST_ASSERT(mp->flags & RTE_MEMPOOL_F_NON_IO, "NON_IO flag is not set when NO_IOVA_CONTIG is set"); ret = TEST_SUCCESS; exit: + rte_eal_get_configuration()->iova_mode = saved_mode; rte_mempool_free(mp); + rte_free(virt); return ret; } static int -test_mempool_flag_non_io_unset_when_populated_with_valid_iova(void) +test_mempool_flag_non_io_when_populated(enum rte_iova_mode mode) { void *virt = NULL; rte_iova_t iova; size_t total_size = MEMPOOL_ELT_SIZE * MEMPOOL_SIZE; size_t block_size = total_size / 3; struct rte_mempool *mp = NULL; + bool using_pa = mode == RTE_IOVA_PA; + enum rte_iova_mode saved_mode = rte_eal_iova_mode(); int ret; /* @@ -785,55 +807,51 @@ test_mempool_flag_non_io_unset_when_populated_with_valid_iova(void) RTE_TEST_ASSERT_NOT_NULL(mp, "Cannot create mempool: %s", rte_strerror(rte_errno)); + RTE_TEST_ASSERT(mp->flags & RTE_MEMPOOL_F_NON_IO, + "NON_IO flag is not set on an empty mempool"); + + rte_eal_get_configuration()->iova_mode = mode; + ret = rte_mempool_populate_iova(mp, RTE_PTR_ADD(virt, 1 * block_size), RTE_BAD_IOVA, block_size, NULL, NULL); RTE_TEST_ASSERT(ret > 0, "Failed to populate mempool: %s", rte_strerror(-ret)); - RTE_TEST_ASSERT(mp->flags & RTE_MEMPOOL_F_NON_IO, - "NON_IO flag is not set when mempool is populated with only RTE_BAD_IOVA"); + if (using_pa) + RTE_TEST_ASSERT(mp->flags & RTE_MEMPOOL_F_NON_IO, + "NON_IO flag is not set when mempool is populated with only RTE_BAD_IOVA"); + else + RTE_TEST_ASSERT(!(mp->flags & RTE_MEMPOOL_F_NON_IO), + "NON_IO flag is set on a mempool with contiguous objects with IOVA-as-VA"); ret = rte_mempool_populate_iova(mp, virt, iova, block_size, NULL, NULL); RTE_TEST_ASSERT(ret > 0, "Failed to populate mempool: %s", rte_strerror(-ret)); - RTE_TEST_ASSERT(!(mp->flags & RTE_MEMPOOL_F_NON_IO), - "NON_IO flag is not unset when mempool is populated with valid IOVA"); + if (using_pa) + RTE_TEST_ASSERT(!(mp->flags & RTE_MEMPOOL_F_NON_IO), + "NON_IO flag is not unset when mempool is populated with valid IOVA"); + else + RTE_TEST_ASSERT(!(mp->flags & RTE_MEMPOOL_F_NON_IO), + "NON_IO flag is set when mempool is populated with valid IOVA with IOVA-as-VA"); ret = rte_mempool_populate_iova(mp, RTE_PTR_ADD(virt, 2 * block_size), RTE_BAD_IOVA, block_size, NULL, NULL); RTE_TEST_ASSERT(ret > 0, "Failed to populate mempool: %s", rte_strerror(-ret)); - RTE_TEST_ASSERT(!(mp->flags & RTE_MEMPOOL_F_NON_IO), - "NON_IO flag is set even when some objects have valid IOVA"); + if (using_pa) + RTE_TEST_ASSERT(!(mp->flags & RTE_MEMPOOL_F_NON_IO), + "NON_IO flag is set even when some objects have valid IOVA"); + else + RTE_TEST_ASSERT(!(mp->flags & RTE_MEMPOOL_F_NON_IO), + "NON_IO flag is set for a mempool with contiguous objects with IOVA-as-VA"); ret = TEST_SUCCESS; exit: + rte_eal_get_configuration()->iova_mode = saved_mode; rte_mempool_free(mp); rte_free(virt); return ret; } -static int -test_mempool_flag_non_io_unset_by_default(void) -{ - struct rte_mempool *mp; - int ret; - - mp = rte_mempool_create_empty("empty", MEMPOOL_SIZE, - MEMPOOL_ELT_SIZE, 0, 0, - SOCKET_ID_ANY, 0); - RTE_TEST_ASSERT_NOT_NULL(mp, "Cannot create mempool: %s", - rte_strerror(rte_errno)); - ret = rte_mempool_populate_default(mp); - RTE_TEST_ASSERT_EQUAL(ret, (int)mp->size, "Failed to populate mempool: %s", - rte_strerror(-ret)); - RTE_TEST_ASSERT(!(mp->flags & RTE_MEMPOOL_F_NON_IO), - "NON_IO flag is set by default"); - ret = TEST_SUCCESS; -exit: - rte_mempool_free(mp); - return ret; -} - #pragma pop_macro("RTE_TEST_TRACE_FAILURE") static int @@ -1022,11 +1040,13 @@ test_mempool(void) GOTO_ERR(ret, err); /* test NON_IO flag inference */ - if (test_mempool_flag_non_io_unset_by_default() < 0) + if (test_mempool_flag_non_io_with_no_iova_contig_set(RTE_IOVA_PA) < 0) + GOTO_ERR(ret, err); + if (test_mempool_flag_non_io_with_no_iova_contig_set(RTE_IOVA_VA) < 0) GOTO_ERR(ret, err); - if (test_mempool_flag_non_io_set_when_no_iova_contig_set() < 0) + if (test_mempool_flag_non_io_when_populated(RTE_IOVA_PA) < 0) GOTO_ERR(ret, err); - if (test_mempool_flag_non_io_unset_when_populated_with_valid_iova() < 0) + if (test_mempool_flag_non_io_when_populated(RTE_IOVA_VA) < 0) GOTO_ERR(ret, err); rte_mempool_list_dump(stdout); diff --git a/lib/mempool/rte_mempool.c b/lib/mempool/rte_mempool.c index b75d26c82a..ba81db6f67 100644 --- a/lib/mempool/rte_mempool.c +++ b/lib/mempool/rte_mempool.c @@ -327,6 +327,7 @@ rte_mempool_populate_iova(struct rte_mempool *mp, char *vaddr, unsigned i = 0; size_t off; struct rte_mempool_memhdr *memhdr; + bool always_io, never_io; int ret; ret = mempool_ops_alloc_once(mp); @@ -372,8 +373,10 @@ rte_mempool_populate_iova(struct rte_mempool *mp, char *vaddr, STAILQ_INSERT_TAIL(&mp->mem_list, memhdr, next); mp->nb_mem_chunks++; - /* At least some objects in the pool can now be used for IO. */ - if (iova != RTE_BAD_IOVA) + /* Check if at least some objects in the pool are now usable for IO. */ + never_io = mp->flags & RTE_MEMPOOL_F_NO_IOVA_CONTIG; + always_io = rte_eal_iova_mode() == RTE_IOVA_VA; + if (!never_io && (always_io || iova != RTE_BAD_IOVA)) mp->flags &= ~RTE_MEMPOOL_F_NON_IO; /* Report the mempool as ready only when fully populated. */ -- 2.25.1