From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9B8C642C98; Mon, 12 Jun 2023 15:53:34 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 39D4C41138; Mon, 12 Jun 2023 15:53:34 +0200 (CEST) Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2088.outbound.protection.outlook.com [40.107.237.88]) by mails.dpdk.org (Postfix) with ESMTP id 810CE40698 for ; Mon, 12 Jun 2023 15:53:32 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=l7kUTC5tPU7B5JYO9jjIJhPAMHlNzPr+0tvLk0FqZbFFp8drUCDDu9KEnPr4WzAcYbtfcvBVnnWos26f3sqoL/jIjEm9J/9tWFl4K6b4rSRGjlThPO+xohT2ub4AvShm0E5j1gYEQDAobCcTUg5CVNG1cy8BEqxDDyOZXbNBOioHW5a1EDieL2nRCMx6t+BLc6LFVQElNzJyz1ZzJnllhTaMMDiM2k4r90nkNfKlQdiWow5VQPrf6arg1xz3ALo+Z/XOJEwrbpTE1+/yWnlsUhnSK9jj8+qFW3z9yfJnNER02RBwDvGcdTnsaLg1vaKx2v2/exW7Uvyp/bg+IsUvRw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=LyKcxHclp5Y6hFOdd0NnrKErFhzKu/Wr5Mwhw/iTunQ=; b=RoBXAwIVCI16OM6QqLwv/7sOkPXReDG/CayLRTvc5TFfjQTzOV8/O7bfcITyg+yUYEomocKkgRGbPGUbhv/Ib31+QfVd9AYsOoFPT5PA6HDCqgvZiysRqp8rAKtQt5hpcfC5AZTzaaDsLotHluLoM2P9zVdnvb73upMj+hSj2wVgyx5iuCuIyBM1XRTO6TyWjEVxwUOuGRxYsFctI7A0cVvemIqfaEqmgLZdVPrU2OiSC7LaJrJMB3RBfZ5xsYY+g59qpBmsAjW/mf+8/iCPG8Xhwg7xIv39vAjbWV3MylzpX77g1ZEtHNZL+FDNai3PqVtbOb5gktMRJGff5SVHGQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=LyKcxHclp5Y6hFOdd0NnrKErFhzKu/Wr5Mwhw/iTunQ=; b=ljriDJh0qyGy39s6UazS37rVV6EgWQ1t6rMLeiG6hdeltATTM3xLFhuiu2tss4/iNGRygZEdQXPLwnGzH/VgMrBDavaKHKpDcTpZ3HdvoTJtUQro8PR2ihJeZrSVTtmnOkeR9wxyqvAALLJtLhBHvlV9wbT6IrRe7RiqiRoSYOQ= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=amd.com; Received: from CH2PR12MB4294.namprd12.prod.outlook.com (2603:10b6:610:a9::11) by DS7PR12MB5957.namprd12.prod.outlook.com (2603:10b6:8:7c::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6455.44; Mon, 12 Jun 2023 13:53:28 +0000 Received: from CH2PR12MB4294.namprd12.prod.outlook.com ([fe80::16e3:326c:5c2a:be42]) by CH2PR12MB4294.namprd12.prod.outlook.com ([fe80::16e3:326c:5c2a:be42%3]) with mapi id 15.20.6477.028; Mon, 12 Jun 2023 13:53:28 +0000 Message-ID: <8137a547-da19-7f10-7bb8-f2deef7fc64c@amd.com> Date: Mon, 12 Jun 2023 14:53:21 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.12.0 Content-Language: en-US To: Chengwen Feng , thomas@monjalon.net, david.marchand@redhat.com Cc: dev@dpdk.org, mb@smartsharesystems.com, anatoly.burakov@intel.com, dmitry.kozliuk@gmail.com, jerinjacobk@gmail.com, hofors@lysator.liu.se, stephen@networkplumber.org References: <20220721044648.6817-1-fengchengwen@huawei.com> <20230209063610.35501-1-fengchengwen@huawei.com> From: Ferruh Yigit Subject: Re: [PATCH v14 0/6] introduce memarea library In-Reply-To: <20230209063610.35501-1-fengchengwen@huawei.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-ClientProxiedBy: LO4P123CA0029.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:151::16) To CH2PR12MB4294.namprd12.prod.outlook.com (2603:10b6:610:a9::11) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR12MB4294:EE_|DS7PR12MB5957:EE_ X-MS-Office365-Filtering-Correlation-Id: e5c8c786-bd1c-488b-8a26-08db6b4c663a X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: wFIHZD5PZSqmu8jTVpVbmI91MN20NpzM+PACg8G6dTp98GDmV9Ya+18bFclrzhS2ds8QsK6/4bYBd5WnHyuZRnK8B/7Nv2FgMgw33wKksYPb61sxjYB4geZhuPIazP0zpXkYkgqaLHEINjhgMeir1m33pRr81awFRlYgUixoIqxk8ZvymDPf3LjkxVXWLU1OaAt0onI3REpgix8hExShUB7myIMLR2moNoluv10zi3lwOHWPdzFUjQSWiTZWFw8KdVFZvpTd3UPBWt1FrXHoUjjFmkncAp4kUZqIumjy7T+nRQQDR4aLwxwkoDogLNPMOGQOLCzdTwp1SxmQjwsMuF8WeZL5+epcKdySOmpcrxozr78jyWsEQopIsYgOqkEk50J9Mnsx9HGXmvRyA41V1uMqS0+wcXkGGyi1BlHTQohDoxUZJp6vrQpFs59+QOPy3BhXWedhRtKPznPB4I3R4Ime9UQxkUKDlxvlgYMercHMET3jxiqacbxbDlpi6J1JomevQQG60bw9LEXwchlbo45SJq8GyKHdgXXRzQHsn92MWl5JjZNHYJ5s12FJBiwZhbErmQv6GA1CsrAKsYFyzbKyv/n6OtX6+wvGwzfTlEQ= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR12MB4294.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230028)(4636009)(136003)(39860400002)(396003)(376002)(346002)(366004)(451199021)(36756003)(31696002)(86362001)(2906002)(44832011)(7416002)(31686004)(966005)(6666004)(186003)(83380400001)(53546011)(6506007)(26005)(6512007)(6486002)(66946007)(4326008)(66476007)(316002)(66556008)(2616005)(38100700002)(478600001)(8936002)(8676002)(41300700001)(5660300002)(45980500001)(43740500002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?R2xkeFJMK1M4RzhtbkI2azZ4d0hPOURBUjJHT2ttbjdDTDRodUZDWjdJTGdW?= =?utf-8?B?ZHVnQURMQ3RwSDlpd281WnBOSG5jVGtIbTQwcmdzRDNxWFlpdlhjNkt5NUUr?= =?utf-8?B?K3hEaVpQdi9Fd05ZSzh4eEoxR0VUcyswZjZRMnhMa3lYMUJtV2R2WitGd1RZ?= =?utf-8?B?cS9aUjdSb0N0SnBuTElSd0hhQWN3UDVzOTNZL3FvZ2djSk1yMzQwMWZpUy9i?= =?utf-8?B?SkU2by8veHB3K1h6V1R1dWVxSWFVUGtZNmhEWGFLcjA1T3VJeGsvNHpBbEF0?= =?utf-8?B?aVZuWTBRaVJrRGN3NU5lYUhLbjh0a0x3aVpHUy9nL0NqeEkwS1NFVHYwSi9h?= =?utf-8?B?dGdxaDVNSmdtSmovZG82aFkxSnNBeFE1TWFmU29rd2poeUM2RFhidTFUWmgx?= =?utf-8?B?N2RLU2lub0lPQlBKdDArYUl5QmhLOFcxV3VEZUV6dDBrMm5ZRzg5cXlwdUE4?= =?utf-8?B?c3JBZ3J4bjMvZnhzdG0xYmtkb1NUQ3FwenFvN1Q4L1M2cWkxRmRzbm1PaWtp?= =?utf-8?B?Y0xlaXlZa3FsaUo4NkszWS9KQTE4c2M4Y2xXckgvcG4zQkg3MER5dnJsSUwx?= =?utf-8?B?bUdBV3NwY2lFNDdsZ1BJeTBONVE5MkNFcWVOOUFsdDZGVk9CS3c5Lzh6UVhS?= =?utf-8?B?TXhScy9iaEhJUy9tcXE3U3JkV20rZnA2YTlCdmxhTUtKQVJ4dDlvcGd5QllM?= =?utf-8?B?clVLZ08zdE8va0FMOFIyVzZDT1AreTFaOFdiODRGTFQrbyszY3NxU0YvcHds?= =?utf-8?B?RzYzM0FEVzJsYWZhNWg3VTA1WDdJNEVVbC9ENkhTc1lxU3kxblBUTjE0R1ZU?= =?utf-8?B?RlNjSFkwNGpZYk9MaXJDbEU0eTlrU3l4M0VjSjB4ZGEvOVVsMzRmbS93U0R5?= =?utf-8?B?RlRtajZJOGV6QVpidFhYUStFUUxkTzI1eVh5OWdIUGZIRVZaL0NPU2xoQ00r?= =?utf-8?B?NHRKNG9hZWlnekRySHhXY2JZSUROb1QrcjZMR3RLMjNvaWVaRGd4Sm8zZTVp?= =?utf-8?B?S3JwSjNpekhGWXFCc2RXaldTVnpiT2cvUjZqOWI2bzlWeU81NkVYWWNGVTh2?= =?utf-8?B?KzFGSHlzQ0hHQkRYUHdpNWZDNEhnV2YvY0J6Vm16cGR6T1lEWjM1ZGZIc081?= =?utf-8?B?eVUxS2t5U2FKZWVWbmF1UlR0UDhiZGt4QzFlTGM1N2RNTmVqQTRXNjU5UVNm?= =?utf-8?B?ZC9VMVc5WlpqcjV1QWVxR3ZHYTJiRWxsWEVHUXkrOEU1elJLRXFoSkVJS1lQ?= =?utf-8?B?N1Jma1ErdGZ5Sm5aalMweExEZjJtMy8ybkJHTFlQaExZWWhxS2d3SXhJYU5y?= =?utf-8?B?am1jMHR3WGZqZlhZSkFGamZqTFloZkMzeFQ3czZXUGNkWUFYWTZSa0xzSlZl?= =?utf-8?B?TThKNzBtNmNLM2ZSQ2N0N3V2VFJqNVlNSWc3OVc4cGkvQWNoTnJkTEs3NlJk?= =?utf-8?B?cXZBN0dTS0JrcUtaRklRSDRSQS9lMGNhUUJ1WFY2WjczZHpyVFFCdEg2b056?= =?utf-8?B?ZWZ3WWpBbFhIa1krck8rTjFraW9iZHYrZUpBUWoybFZDbEhwNjUvN1VNaUdv?= =?utf-8?B?V213QUVPcFJCV050aG9pTG01TjduWnZkcFBnaVNiai9VbFE2Z2htM0VFM2RO?= =?utf-8?B?QUxIYWlKOFBGU2NQc2p0cis1cldyS3NrQW1mTkpncnFDcXVzRWFZR1MyUXBz?= =?utf-8?B?K24rUDlQZEJkVFRWOXIvcXpzcHdKWE1tdFRCNTZOSnFobUQrbTl0dDAxeGdO?= =?utf-8?B?ZWNzbSt2TiszNm51YU01cnBBcjU5SVI2S0RkK1JuVUZvTWw0UmZRQUNHMmtV?= =?utf-8?B?S0dJc2RnVmdNZlNoc2xXU1FvTngycXg0YnN2WndrS0h4b3JqdlU2cHZtVDBT?= =?utf-8?B?UHRJRUFtT1BXN3g5UkZidlB0dnp0S3FiQ3Rwd1NvcFNBVFUvM05oNXNKdE8z?= =?utf-8?B?UjI5U2tqV2FWNmhFc2xOVUpFZDM3c1JBbDMyZkhyN21NYzByVmRjR1Y2UFp1?= =?utf-8?B?L0V6YjdrLzZ5VnRxQ0RNenFxNUs4SmlaY2ljU0xBVytnakFvUWE5eHNPUUFq?= =?utf-8?B?NXFQUFJ5cnJnRjYwZlJwL25jNmFmYk9yVW53cHhvR3cxWVN0clQzUVNCT1ph?= =?utf-8?Q?+vCUZpU4lE8S93rkxlyMTHXfC?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: e5c8c786-bd1c-488b-8a26-08db6b4c663a X-MS-Exchange-CrossTenant-AuthSource: CH2PR12MB4294.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Jun 2023 13:53:28.3757 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: hKg/9Nql50925MaJFCFAU4wSuYVxS0PBGCzBPDT2jmxv+v+BxhMWhBhujvQcdjmB X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR12MB5957 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 2/9/2023 6:36 AM, Chengwen Feng wrote: > The memarea library is an allocator of variable-size object which based > on a memory region. The main features are as follows: > > - The memory region can be initialized from the following memory > sources: > 1. HEAP: e.g. invoke rte_malloc_socket. > 2. LIBC: e.g. invoke posix_memalign. > 3. Another memarea: it can be from another memarea. > > - It supports MT-safe as long as it's specified at creation time. > > Note: > a) The memarea is oriented towards the application layer, which could > provides 'region-based memory management' [1] function. > b) The eal library also provide memory zone/heap management, but these > are tied to huge pages management. > > [1] https://en.wikipedia.org/wiki/Region-based_memory_management > > Chengwen Feng (6): > memarea: introduce memarea library > test/memarea: support memarea test > memarea: support alloc and free API > test/memarea: support alloc and free API test > memarea: support dump API > test/memarea: support dump API test > Hi Chengwen, I am aware that patchset it out for a while now, and library has a good feature set, but I am curious what problem this library solves that existing APIs can't. Can you please describe more what is the usecase and motivation of the feature/library? I can see it is easy/cheap to destroy a memory region, so is your problem memory free performance? Mostly mempool is used to solve this issue, are you planning to use these APIs on datapath for variable size buffers? Or is the motivation to be able to run memory management on top of malloc() allocated memory, if so why? Hugepages are used for performance reason, if allocated memory is used for datapath, won't this cause performance degradation. And if this is the main motivation, can it be an option to add regular page support to memsegments, this way existing APIs can be used with a flag, to make it easier for users. Thanks, Ferruh > --- > v14: > * address Stephen's comments: RTE_MEMAREA_LOG use easy impl and add > __func__ print. > v13: > * address Morten's comments: make debug cookies optional, controlled > by RTE_LIBRTE_MEMAREA_DEBUG; disabled by default. > * reduce management data overhead. > v12: > * remove rte_memarea_refcnt_update() API which address Dongdong's > comments. > * refine the variable naming. > * fix some bugs. > v11: > * rebase 23.03 > * remove "app/test: add memarea to malloc-perf-autotest" because the > two algorithm are not comparable which also address previous > comments. > v10: > * support windows platform. > * add rte_memarea.libc perftest to malloc-perf-autotest. > v9: > * address Dmitry's comments. > * drop features of SOURCE_USER and backup memarea mechanism. > * rename rte_memarea_update_refcnt to rte_memarea_refcnt_update > to keep with rte_mbuf_refcnt_update name style. > * fix memarea perftest compile failed at windows platform. > * fix spell warning. > v8: > * address Mattias's comments (rename ALG_DEFAULT with ALG_NEXTFIT). > * small feature patches are combined. > * enhanced backup memory mechanism. > * add memarea to malloc-perf-autotest. > * other tiny naming optimize. > v7: > * repost patches as there are spread over different series in patchwork. > v6: > * address Mattias's comments. > v5: > * fix 09/10 patch spell warning. > v4: > * repost patches as there are spread over different series in patchwork. > v3: > * add memory source of RTE memory. > * add algorithm field to facilitate the introduction of new algorithms. > * fix memarea log don't output problem. > v2: > * fix compile issues reported by dpdk-test-report. > * address Dimitry and Jerin's comments. > * add no MT-safe test. > > MAINTAINERS | 6 + > app/test/meson.build | 2 + > app/test/test_memarea.c | 297 ++++++++++++++++ > doc/api/doxy-api-index.md | 3 +- > doc/api/doxy-api.conf.in | 1 + > doc/guides/prog_guide/index.rst | 1 + > doc/guides/prog_guide/memarea_lib.rst | 55 +++ > doc/guides/rel_notes/release_23_03.rst | 6 + > lib/memarea/memarea_private.h | 58 ++++ > lib/memarea/meson.build | 10 + > lib/memarea/rte_memarea.c | 448 +++++++++++++++++++++++++ > lib/memarea/rte_memarea.h | 185 ++++++++++ > lib/memarea/version.map | 15 + > lib/meson.build | 1 + > 14 files changed, 1087 insertions(+), 1 deletion(-) > create mode 100644 app/test/test_memarea.c > create mode 100644 doc/guides/prog_guide/memarea_lib.rst > create mode 100644 lib/memarea/memarea_private.h > create mode 100644 lib/memarea/meson.build > create mode 100644 lib/memarea/rte_memarea.c > create mode 100644 lib/memarea/rte_memarea.h > create mode 100644 lib/memarea/version.map >