From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id CA96D4326E; Thu, 2 Nov 2023 12:32:18 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 6A89140282; Thu, 2 Nov 2023 12:32:18 +0100 (CET) Received: from EUR03-AM7-obe.outbound.protection.outlook.com (mail-am7eur03on2049.outbound.protection.outlook.com [40.107.105.49]) by mails.dpdk.org (Postfix) with ESMTP id 8E50A40298 for ; Wed, 1 Nov 2023 22:21:24 +0100 (CET) ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass; b=QDpOrF9QlRKXbieLTcYGe0NLH6H0rw/FxJCa9QfLRJ28atMWwKnWZ1rBNwjiZEiyFX0Z1WeSJpOCgjEUN2v1pLmXmYLbvq4PjGxQcpVurUz9BCEvP4MsyTdUZBFOZQ5wQVaDMf4C47XWBo4G2lXsUPVex1NWLAW2ruOfZtjYU7aYIqK8hhKGVs+ibMCMdW2rM6n2sZnmYz3DLOAhGGQ7hLVwg1ZgcyKH6EOooOvjh7/3+UbG0o3ZRCDxJhO3kObEPQRPZGh5WCrkpqpuCaL3v4daSVaetdKSWtkfFoMmpXt5Vb+C1ltuG6KUP+zQeQxQQqJJhmmUaWqk42RTco6bzw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=0VFm3teRHLdp+FAxf5pSxeqCopmi6Hhf6MfefxjbQ28=; b=KtpnlKIdO6H/K/QqRhVh46hwZbFsnUvhzh0H+0/YPiFQRtTs0dhRl6NEeCh2h4C3Spgd6eavJTiMkNlaOsFJjO7AEv/Pzi6hwXc/6vVAb2BYSOzE5fuasz+gJ0tdIMN1PKfy4/PXORJ9bHnqY9gGZMUapkPmnhgm3JFlDJsvCTqFqv/8g5WBnGtePq9ix0ezeTSCd/U+/F5Un/HO15+yvWoQpK5QtU1dLctQrT3Ty8mDKomzwvPBHPLic2Y8IZE8eN/XP0hnFiHMh5MeYexqzGSrQ5BPHDEJgkJ9aGpiQDgywCpJcN1X+qFYgg40uzNd0guhiqCv1Q0w/QMvrNCknA== ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is 51.104.160.93) smtp.rcpttodomain=nvidia.com smtp.mailfrom=astron.nl; dmarc=pass (p=none sp=none pct=100) action=none header.from=astron.nl; dkim=fail (signature did not verify) header.d=astron.nl; arc=pass (0 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=astron.nl] dkim=[1,1,header.d=astron.nl] dmarc=[1,1,header.from=astron.nl]) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=astron.nl; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=0VFm3teRHLdp+FAxf5pSxeqCopmi6Hhf6MfefxjbQ28=; b=IrNi4EqzepCRklNocnR0I22+xBPZXmgKwaFShOyN0Zf2w+Ls5gT8sV2hgh/4GVDh2+ov8QhYYfKYk5dmFu0GzPtrIL/04XkOJ3FFsGb1UiV0FbJf/wxwCQ6uWGdxcwzf13u2O7Cw9nkeBiovaGvQMYhvcwcp2nwuwmBAqBtgWx8= Received: from AS9PR01CA0015.eurprd01.prod.exchangelabs.com (2603:10a6:20b:540::28) by DU0PR10MB5582.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:10:31f::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6933.29; Wed, 1 Nov 2023 21:21:22 +0000 Received: from AMS0EPF0000019E.eurprd05.prod.outlook.com (2603:10a6:20b:540:cafe::b8) by AS9PR01CA0015.outlook.office365.com (2603:10a6:20b:540::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6954.19 via Frontend Transport; Wed, 1 Nov 2023 21:21:21 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 51.104.160.93) smtp.mailfrom=astron.nl; dkim=fail (signature did not verify) header.d=astron.nl;dmarc=pass action=none header.from=astron.nl; Received-SPF: Pass (protection.outlook.com: domain of astron.nl designates 51.104.160.93 as permitted sender) receiver=protection.outlook.com; client-ip=51.104.160.93; helo=xwaresmtpservice-58f757bb85-2c8nf.xware-eu-k1-north-1.o365.crossware.co.nz; pr=C Received: from xwaresmtpservice-58f757bb85-2c8nf.xware-eu-k1-north-1.o365.crossware.co.nz (51.104.160.93) by AMS0EPF0000019E.mail.protection.outlook.com (10.167.16.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6954.19 via Frontend Transport; Wed, 1 Nov 2023 21:21:21 +0000 Received: from EUR01-DB5-obe.outbound.protection.outlook.com (104.47.2.51) by xwaresmtpservice-58f757bb85-2c8nf.xware-eu-k1-north-1.o365.crossware.co.nz with Crossware for Office365 id 0154ef42-2d5c-4a16-a3b3-45de52acd0d1; Wed, 1 Nov 2023 21:21:21 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=d91KEPPt73jyBrRjzoW2m0/ijOzSO/roovGaINAY56X/j9RBucptbRlnKCTxGJwGIzVHzXjGsy9piLCT0NPj3c+OPNhUd5EQP7dXVqN0I+bFm5Dpe18ktD/CzBrPNZFWWDah5BsqM5G4GTpI8tDTaHpGfEUYcoLPEg6LxqoJu5dCzrOIBGgOwYBYS3CH8gKxkI+wEWfN0h5NHtFJup5RzZbmRp6dxJXwJPqixAQU5gYtDmnZgXZlOWu9JoRzpYnoc2jAnZup0PmOD+afGwysESDtEgtF+FHMZpTU/piq9RSnMMDwebkW3pwNl19fRMwvk/nwhZ9VpdoRoHUiT8QKzw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=0VFm3teRHLdp+FAxf5pSxeqCopmi6Hhf6MfefxjbQ28=; b=G7v8EC5JY+5S5B6xDSbMOVu9FosuWozaa97B/rZDFUDnlX2GPBT63UMoho4NmoUb2J+5NV/R72vBJz44mFPKBgI45jR0K4HPKQa3cbNDV8F0WtmJBSRn9eJaqFGPg2x+DcSBhM+LW5TvOpa3/l7ONP7D6iSakyxytNhN0DjJc0LUxHaGk5TRruH9jjwBpCxr4Cuob4WIFHmOt9EFiyO6hnb5MCtxHXC19NiYpJeFb+Xu451ukaMkt+y2hkgYgZkLYXV5ptqeyMOaTA8ntc7Bk2f9ii61SKG3LcgJrQGMYa8PBfHL/rJm2eIDSaD5C7swDPBCztHwdaPekc2IuVukyA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=astron.nl; dmarc=pass action=none header.from=astron.nl; dkim=pass header.d=astron.nl; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=astron.nl; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=0VFm3teRHLdp+FAxf5pSxeqCopmi6Hhf6MfefxjbQ28=; b=IrNi4EqzepCRklNocnR0I22+xBPZXmgKwaFShOyN0Zf2w+Ls5gT8sV2hgh/4GVDh2+ov8QhYYfKYk5dmFu0GzPtrIL/04XkOJ3FFsGb1UiV0FbJf/wxwCQ6uWGdxcwzf13u2O7Cw9nkeBiovaGvQMYhvcwcp2nwuwmBAqBtgWx8= Authentication-Results-Original: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=astron.nl; Received: from VI1PR10MB1935.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:803:35::32) by PAXPR10MB5591.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:102:243::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6933.29; Wed, 1 Nov 2023 21:21:19 +0000 Received: from VI1PR10MB1935.EURPRD10.PROD.OUTLOOK.COM ([fe80::4e62:a1c7:d95d:ff4d]) by VI1PR10MB1935.EURPRD10.PROD.OUTLOOK.COM ([fe80::4e62:a1c7:d95d:ff4d%4]) with mapi id 15.20.6954.019; Wed, 1 Nov 2023 21:21:19 +0000 Content-Type: multipart/alternative; boundary="------------YWJHDW6krOKyCdxx5XibV3jq" Message-ID: <05b8ccc9-a159-4141-b323-924f7ec15baf@astron.nl> Date: Wed, 1 Nov 2023 22:21:16 +0100 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v2] common/mlx5: Optimize mlx5 mempool get extmem Content-Language: en-US, nl To: Slava Ovsiienko , Aaron Conole , "dev@dpdk.org" Cc: Raslan Darawsheh , Elena Agostini , Dmitry Kozlyuk , Matan Azrad , Ori Kam , Suanming Mou References: <20231010143800.102459-1-aconole@redhat.com> From: John Romein In-Reply-To: X-ClientProxiedBy: AS4P190CA0038.EURP190.PROD.OUTLOOK.COM (2603:10a6:20b:5d1::17) To VI1PR10MB1935.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:803:35::32) MIME-Version: 1.0 X-MS-TrafficTypeDiagnostic: VI1PR10MB1935:EE_|PAXPR10MB5591:EE_|AMS0EPF0000019E:EE_|DU0PR10MB5582:EE_ X-MS-Office365-Filtering-Correlation-Id: dfa0fbca-a218-4746-0313-08dbdb207ee5 X-MAIL_SIG_ADMINUNIT: none X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: 3NEflLMmwfcWwK95gT7maTXzwqbmXsGg8GO8qcr+GYwRgE81b19wgupBD4TI7OeE1whdXJBHZkNSX+8C0AAgW8nypWVuJlN6v2NWDw1rJr57LLXnkH/XKXk8dNMmj1Y3/U0jUZHqvwdmt4T1DLT3wx1CavvADu/LtBa0QcZOk0Qq91lYGKgQmEGw9fyt+gTnLv4D3eTQM2ci55irywIin3wSdAFaPeTTIxUapKP38epYYB+E2Ben+8N0pgbMd5YCQ/vuL/JoSvKWjfFmMwDrdFodZranNad03HKFT9NQltg+onP6I1pDRQhugB50DMjkbtbEDjHoWVUWoDon/+iqyma3uyiMax2+5r8vGqhXfj7zTuCfK+NaWUANfMy1IJpXErwJkpxLXrA+1z7Hpx6bMj0QdJr+2aoijut6n57kbhvRVYzT6geSpaL+4CsKcSMhs9uly5TecYoC6pyRkj5ZwFMvOL9fkUIxYUga7mKMVd2g0wvyCfj9WZFZ7UDlZNh60An8wrNoWEERw6/bdDlzKd8GTNroMaAQve7LkvHC1zZNBXXS643/BXW/9R1XZ5pAasP1Fhqf0GTmHWDjVAbYvw/J1UL5XY6VFHKEAxm2sS458Oxoqkpd5/EFCxAzj+J0u9YlJJE7yeZ1WjU71k5zGgQVN54PU4Gx0McFuU+4ZJHYXPtk7YFgfm7voKwjcQD9 X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:VI1PR10MB1935.EURPRD10.PROD.OUTLOOK.COM; PTR:; CAT:NONE; SFS:(13230031)(376002)(136003)(366004)(396003)(39850400004)(346002)(230173577357003)(230273577357003)(230922051799003)(1800799009)(64100799003)(186009)(451199024)(8936002)(2906002)(31686004)(8676002)(4326008)(5660300002)(30864003)(6666004)(6486002)(41300700001)(478600001)(786003)(66556008)(110136005)(66476007)(316002)(54906003)(66946007)(53546011)(6512007)(2616005)(6506007)(26005)(83380400001)(33964004)(86362001)(31696002)(38100700002)(36756003)(45980500001)(43740500002); DIR:OUT; SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR10MB5591 X-CWesigProcessed: Y X-MAIL_SIG_VERSION: 0.2.341 X-MAIL_SIG_SERVER: xwaresmtpservice-58f757bb85-2c8nf.xware-eu-k1-north-1.o365.crossware.co.nz X-MAIL_SIG_CACHEID: 13045 X-MAIL_SIG_CONFIGNAME: 1 - No signature X-MAIL_SIG_CONFIGNAMEPLIED: 1 - No signature X-EOPAttributedMessage: 0 X-CrossPremisesHeadersPromoted: AMS0EPF0000019E.eurprd05.prod.outlook.com X-CrossPremisesHeadersFiltered: AMS0EPF0000019E.eurprd05.prod.outlook.com X-MS-Exchange-Transport-CrossTenantHeadersStripped: AMS0EPF0000019E.eurprd05.prod.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 39328a72-6fbe-4489-a4a3-08dbdb207d2c X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: AWPkB4DstQucGAXA43Hpo4QpOO1qtX+t+614xSp7c1inLgib0pvc6g7SDDZqxj/kJtSHYyxcw5YxeT2de2LrkmpcBApH/msQ3PSVtaTCZUDiiZFM8IUsW6sJ/YBgau3nh1exHx6CxgL2DnRC0ADf1Wb9fSw1PTQQ/P2twDbckjrDBseB8WNF+KI8ypx2BKegw9atjCUaJhy6x72679AkVCy5fdRHCXhV4p6DjilH8M7Y/KU1qI907MXXILG8Df28JZP7kcRozcRIbo2SKPDeOCTFB8nd38vk+MTxQtIvFvS255AsN2ZEkl+zdYGIYCRCjEAMyZRrvafS81ifEyr5SkZ9s6aTvmQSHK6Ot8hj1ygqX0/v3/T+d+BWTNArSjuynZceAQvTev+bboCN81nQdU3cLrBgZpZeSvV1SmnuL2PHdPXmAQFtLNezigi7zKtOBgwbaxu5SW+ECoYgD8Cn6FdJMpJNaJQsmZ9bhEKI/cIvTDGu0G39t4gHgmQ56RqVYv+x3c/8zhEWVrFyFGFJmvtrezIxkg7CNwz6rbzEb2BcaQPQZGy238L2NZygHJX9pVlFGZe0u0PGgSbg1J4ZnsIPEZaSoORTAAPiuBtXqzkbDvd2AIQcU4yuJPJv/7msqBPB3DJo51AuUVTCl222FKVLbJ+8POEsnjpTVwi5Uf5Sy0srwiD4YM/qU3HmVEJIjNwGLFjx7jM6o1PpyupCDJtnVbnXRX4EznfDKgRK9NvBOvIYk+KjKtvKFGvD+EFMwBr1/Vu3Uea4lXLP0iYHVGw5Nt7E1tUlYbGAZZiSXsWFWm370e42yxWrC/M4eDm4 X-Forefront-Antispam-Report: CIP:51.104.160.93; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:xwaresmtpservice-58f757bb85-2c8nf.xware-eu-k1-north-1.o365.crossware.co.nz; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230031)(4636009)(346002)(136003)(376002)(39850400004)(396003)(230173577357003)(230922051799003)(230273577357003)(82310400011)(1800799009)(451199024)(64100799003)(186009)(46966006)(36840700001)(36756003)(86362001)(31696002)(82740400003)(31686004)(40480700001)(2906002)(33964004)(6486002)(478600001)(6666004)(30864003)(4326008)(41300700001)(8936002)(8676002)(53546011)(6512007)(47076005)(83380400001)(26005)(2616005)(70206006)(110136005)(316002)(786003)(5660300002)(54906003)(36860700001)(356005)(81166007)(336012)(6506007)(43740500002)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: astron.nl X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Nov 2023 21:21:21.5712 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: dfa0fbca-a218-4746-0313-08dbdb207ee5 X-MS-Exchange-CrossTenant-AuthSource: AMS0EPF0000019E.eurprd05.prod.outlook.com X-MS-Exchange-CrossTenant-Id: 03e9259c-3262-4694-ad93-4dce7c9fa403 X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=03e9259c-3262-4694-ad93-4dce7c9fa403; Ip=[51.104.160.93]; Helo=[xwaresmtpservice-58f757bb85-2c8nf.xware-eu-k1-north-1.o365.crossware.co.nz] X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR10MB5582 X-Mailman-Approved-At: Thu, 02 Nov 2023 12:32:17 +0100 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org --------------YWJHDW6krOKyCdxx5XibV3jq Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Dear Slava, Thank you for looking at the patch.  With the original code, I saw that the application spent literally hours in this function during program start up, if tens of gigabytes of GPU memory are registered.  This was due to qsort being invoked for every new added item (to keep the list sorted).  So I tried to write equivalent code that sorts the list only once, after all items were added.  At least for our application, this works well and is /much/ faster, as the complexity decreased from n^2 log(n) to n log(n).  But I must admit that I have no idea /what/ is being sorted, or why; I only understand this isolated piece of code (or at least I think so).  So if you think there are better ways to initialize the list, then I am sure you will be absolutely right.  But I will not be able to implement this, as I do not understand the full context of the code. Kind Regards,  John On 01-11-2023 09:29, Slava Ovsiienko wrote: > Hi, > > Thank you for this optimizing patch. > My concern is this line: >> + heap = malloc(mp->size * sizeof(struct mlx5_range)); > The pool size can be huge and it might cause the large memory allocation > (on host CPU side). > > What is the reason causing "hours" of registering? Reallocs per each pool element? > The mp struct has "struct rte_mempool_memhdr_list mem_list" member. > I think we should consider populating this list with data from > "struct rte_pktmbuf_extmem *ext_mem" on pool creation. > > Because of it seems the rte_mempool_mem_iter() functionality is > completely broken for the pools with external memory, and that's why > mlx5 implemented the dedicated branch to handle their registration. > > With best regards, > Slava > >> -----Original Message----- >> From: Aaron Conole >> Sent: Tuesday, October 10, 2023 5:38 PM >> To:dev@dpdk.org >> Cc: John Romein; Raslan Darawsheh >> ; Elena Agostini; Dmitry >> Kozlyuk; Matan Azrad; Slava >> Ovsiienko; Ori Kam; >> Suanming Mou >> Subject: [PATCH v2] common/mlx5: Optimize mlx5 mempool get extmem >> >> From: John Romein >> >> This patch reduces the time to allocate and register tens of gigabytes of GPU >> memory from hours to seconds, by sorting the heap only once instead of for >> each object in the mempool. >> >> Fixes: 690b2a88c2f7 ("common/mlx5: add mempool registration facilities") >> >> Signed-off-by: John Romein >> --- >> drivers/common/mlx5/mlx5_common_mr.c | 69 ++++++++-------------------- >> 1 file changed, 20 insertions(+), 49 deletions(-) >> >> diff --git a/drivers/common/mlx5/mlx5_common_mr.c >> b/drivers/common/mlx5/mlx5_common_mr.c >> index 40ff9153bd..77b66e444b 100644 >> --- a/drivers/common/mlx5/mlx5_common_mr.c >> +++ b/drivers/common/mlx5/mlx5_common_mr.c >> @@ -1389,63 +1389,23 @@ mlx5_mempool_get_chunks(struct >> rte_mempool *mp, struct mlx5_range **out, >> return 0; >> } >> >> -struct mlx5_mempool_get_extmem_data { >> - struct mlx5_range *heap; >> - unsigned int heap_size; >> - int ret; >> -}; >> - >> static void >> mlx5_mempool_get_extmem_cb(struct rte_mempool *mp, void *opaque, >> void *obj, unsigned int obj_idx) >> { >> - struct mlx5_mempool_get_extmem_data *data = opaque; >> + struct mlx5_range *heap = opaque; >> struct rte_mbuf *mbuf = obj; >> uintptr_t addr = (uintptr_t)mbuf->buf_addr; >> - struct mlx5_range *seg, *heap; >> struct rte_memseg_list *msl; >> size_t page_size; >> uintptr_t page_start; >> - unsigned int pos = 0, len = data->heap_size, delta; >> >> RTE_SET_USED(mp); >> - RTE_SET_USED(obj_idx); >> - if (data->ret < 0) >> - return; >> - /* Binary search for an already visited page. */ >> - while (len > 1) { >> - delta = len / 2; >> - if (addr < data->heap[pos + delta].start) { >> - len = delta; >> - } else { >> - pos += delta; >> - len -= delta; >> - } >> - } >> - if (data->heap != NULL) { >> - seg = &data->heap[pos]; >> - if (seg->start <= addr && addr < seg->end) >> - return; >> - } >> - /* Determine the page boundaries and remember them. */ >> - heap = realloc(data->heap, sizeof(heap[0]) * (data->heap_size + 1)); >> - if (heap == NULL) { >> - free(data->heap); >> - data->heap = NULL; >> - data->ret = -1; >> - return; >> - } >> - data->heap = heap; >> - data->heap_size++; >> - seg = &heap[data->heap_size - 1]; >> msl = rte_mem_virt2memseg_list((void *)addr); >> page_size = msl != NULL ? msl->page_sz : rte_mem_page_size(); >> page_start = RTE_PTR_ALIGN_FLOOR(addr, page_size); >> - seg->start = page_start; >> - seg->end = page_start + page_size; >> - /* Maintain the heap order. */ >> - qsort(data->heap, data->heap_size, sizeof(heap[0]), >> - mlx5_range_compare_start); >> + heap[obj_idx].start = page_start; >> + heap[obj_idx].end = page_start + page_size; >> } >> >> /** >> @@ -1457,15 +1417,26 @@ static int >> mlx5_mempool_get_extmem(struct rte_mempool *mp, struct mlx5_range >> **out, >> unsigned int *out_n) >> { >> - struct mlx5_mempool_get_extmem_data data; >> + unsigned int out_size = 1; >> + struct mlx5_range *heap; >> >> DRV_LOG(DEBUG, "Recovering external pinned pages of mempool >> %s", >> mp->name); >> - memset(&data, 0, sizeof(data)); >> - rte_mempool_obj_iter(mp, mlx5_mempool_get_extmem_cb, >> &data); >> - *out = data.heap; >> - *out_n = data.heap_size; >> - return data.ret; >> + heap = malloc(mp->size * sizeof(struct mlx5_range)); >> + if (heap == NULL) >> + return -1; >> + rte_mempool_obj_iter(mp, mlx5_mempool_get_extmem_cb, heap); >> + qsort(heap, mp->size, sizeof(heap[0]), mlx5_range_compare_start); >> + /* remove duplicates */ >> + for (unsigned int i = 1; i < mp->size; i++) >> + if (heap[out_size - 1].start != heap[i].start) >> + heap[out_size++] = heap[i]; >> + heap = realloc(heap, out_size * sizeof(struct mlx5_range)); >> + if (heap == NULL) >> + return -1; >> + *out = heap; >> + *out_n = out_size; >> + return 0; >> } >> >> /** >> -- >> 2.41.0 --------------YWJHDW6krOKyCdxx5XibV3jq Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 8bit
Dear Slava,

Thank you for looking at the patch.  With the original code, I saw that the application spent literally hours in this function during program start up, if tens of gigabytes of GPU memory are registered.  This was due to qsort being invoked for every new added item (to keep the list sorted).  So I tried to write equivalent code that sorts the list only once, after all items were added.  At least for our application, this works well and is much faster, as the complexity decreased from n^2 log(n) to n log(n).  But I must admit that I have no idea what is being sorted, or why; I only understand this isolated piece of code (or at least I think so).  So if you think there are better ways to initialize the list, then I am sure you will be absolutely right.  But I will not be able to implement this, as I do not understand the full context of the code.

Kind Regards,  John

On 01-11-2023 09:29, Slava Ovsiienko wrote:
Hi,

Thank you for this optimizing patch.
My concern is this line:

+	heap = malloc(mp->size * sizeof(struct mlx5_range));

The pool size can be huge and it might cause the large memory allocation 
(on host CPU side).

What is the reason causing "hours" of registering? Reallocs per each pool element?
The mp struct has "struct rte_mempool_memhdr_list mem_list" member.
I think we should consider populating this list with data from
"struct rte_pktmbuf_extmem *ext_mem" on pool creation.

Because of it seems the rte_mempool_mem_iter() functionality is
completely broken for the pools with external memory, and that's why
mlx5 implemented the dedicated branch to handle their registration.

With best regards,
Slava


-----Original Message-----
From: Aaron Conole <aconole@redhat.com>
Sent: Tuesday, October 10, 2023 5:38 PM
To: dev@dpdk.org
Cc: John Romein <romein@astron.nl>; Raslan Darawsheh
<rasland@nvidia.com>; Elena Agostini <eagostini@nvidia.com>; Dmitry
Kozlyuk <dkozlyuk@nvidia.com>; Matan Azrad <matan@nvidia.com>; Slava
Ovsiienko <viacheslavo@nvidia.com>; Ori Kam <orika@nvidia.com>;
Suanming Mou <suanmingm@nvidia.com>
Subject: [PATCH v2] common/mlx5: Optimize mlx5 mempool get extmem

From: John Romein <romein@astron.nl>

This patch reduces the time to allocate and register tens of gigabytes of GPU
memory from hours to seconds, by sorting the heap only once instead of for
each object in the mempool.

Fixes: 690b2a88c2f7 ("common/mlx5: add mempool registration facilities")

Signed-off-by: John Romein <romein@astron.nl>
---
 drivers/common/mlx5/mlx5_common_mr.c | 69 ++++++++--------------------
 1 file changed, 20 insertions(+), 49 deletions(-)

diff --git a/drivers/common/mlx5/mlx5_common_mr.c
b/drivers/common/mlx5/mlx5_common_mr.c
index 40ff9153bd..77b66e444b 100644
--- a/drivers/common/mlx5/mlx5_common_mr.c
+++ b/drivers/common/mlx5/mlx5_common_mr.c
@@ -1389,63 +1389,23 @@ mlx5_mempool_get_chunks(struct
rte_mempool *mp, struct mlx5_range **out,
 	return 0;
 }

-struct mlx5_mempool_get_extmem_data {
-	struct mlx5_range *heap;
-	unsigned int heap_size;
-	int ret;
-};
-
 static void
 mlx5_mempool_get_extmem_cb(struct rte_mempool *mp, void *opaque,
 			   void *obj, unsigned int obj_idx)
 {
-	struct mlx5_mempool_get_extmem_data *data = opaque;
+	struct mlx5_range *heap = opaque;
 	struct rte_mbuf *mbuf = obj;
 	uintptr_t addr = (uintptr_t)mbuf->buf_addr;
-	struct mlx5_range *seg, *heap;
 	struct rte_memseg_list *msl;
 	size_t page_size;
 	uintptr_t page_start;
-	unsigned int pos = 0, len = data->heap_size, delta;

 	RTE_SET_USED(mp);
-	RTE_SET_USED(obj_idx);
-	if (data->ret < 0)
-		return;
-	/* Binary search for an already visited page. */
-	while (len > 1) {
-		delta = len / 2;
-		if (addr < data->heap[pos + delta].start) {
-			len = delta;
-		} else {
-			pos += delta;
-			len -= delta;
-		}
-	}
-	if (data->heap != NULL) {
-		seg = &data->heap[pos];
-		if (seg->start <= addr && addr < seg->end)
-			return;
-	}
-	/* Determine the page boundaries and remember them. */
-	heap = realloc(data->heap, sizeof(heap[0]) * (data->heap_size + 1));
-	if (heap == NULL) {
-		free(data->heap);
-		data->heap = NULL;
-		data->ret = -1;
-		return;
-	}
-	data->heap = heap;
-	data->heap_size++;
-	seg = &heap[data->heap_size - 1];
 	msl = rte_mem_virt2memseg_list((void *)addr);
 	page_size = msl != NULL ? msl->page_sz : rte_mem_page_size();
 	page_start = RTE_PTR_ALIGN_FLOOR(addr, page_size);
-	seg->start = page_start;
-	seg->end = page_start + page_size;
-	/* Maintain the heap order. */
-	qsort(data->heap, data->heap_size, sizeof(heap[0]),
-	      mlx5_range_compare_start);
+	heap[obj_idx].start = page_start;
+	heap[obj_idx].end = page_start + page_size;
 }

 /**
@@ -1457,15 +1417,26 @@ static int
 mlx5_mempool_get_extmem(struct rte_mempool *mp, struct mlx5_range
**out,
 			unsigned int *out_n)
 {
-	struct mlx5_mempool_get_extmem_data data;
+	unsigned int out_size = 1;
+	struct mlx5_range *heap;

 	DRV_LOG(DEBUG, "Recovering external pinned pages of mempool
%s",
 		mp->name);
-	memset(&data, 0, sizeof(data));
-	rte_mempool_obj_iter(mp, mlx5_mempool_get_extmem_cb,
&data);
-	*out = data.heap;
-	*out_n = data.heap_size;
-	return data.ret;
+	heap = malloc(mp->size * sizeof(struct mlx5_range));
+	if (heap == NULL)
+		return -1;
+	rte_mempool_obj_iter(mp, mlx5_mempool_get_extmem_cb, heap);
+	qsort(heap, mp->size, sizeof(heap[0]), mlx5_range_compare_start);
+	/* remove duplicates */
+	for (unsigned int i = 1; i < mp->size; i++)
+		if (heap[out_size - 1].start != heap[i].start)
+			heap[out_size++] = heap[i];
+	heap = realloc(heap, out_size * sizeof(struct mlx5_range));
+	if (heap == NULL)
+		return -1;
+	*out = heap;
+	*out_n = out_size;
+	return 0;
 }

 /**
--
2.41.0


    

    

  


--------------YWJHDW6krOKyCdxx5XibV3jq--