From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM01-BN3-obe.outbound.protection.outlook.com (mail-bn3nam01on0068.outbound.protection.outlook.com [104.47.33.68]) by dpdk.org (Postfix) with ESMTP id CE55CA48B for ; Wed, 17 Jan 2018 16:55:58 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=m/1W6S9KmBmn37KVzb+dJDRDnVXpc8hRLySB1GOiFkQ=; b=Ct7F0owhuHu8jJZD0YLL5iXQ9jOxbQUB5E8S1NBqCqbO7juakVujY4WbBYksZ3pT69FoKSoSuMiPTLY1n6ohIS8EfHY3SgboAoxQPSca45rYv9WJ6FRHaqgzquleoNkWSyLW4rfKg7jgRntq1yXA3n6vOqkoyHrQdw+gAbrWQ8Q= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Santosh.Shukla@cavium.com; Received: from [192.168.0.105] (103.76.57.2) by MWHPR07MB3101.namprd07.prod.outlook.com (10.172.95.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.407.7; Wed, 17 Jan 2018 15:55:55 +0000 To: Andrew Rybchenko , Olivier MATZ Cc: dev@dpdk.org, "Artem V. Andreev" References: <1511539591-20966-1-git-send-email-arybchenko@solarflare.com> <1511539591-20966-3-git-send-email-arybchenko@solarflare.com> <20171214133721.b7z2i7yccwpath37@platinum> From: santosh Message-ID: <69c425a0-7339-3e7a-a0d2-e2987a644cf2@caviumnetworks.com> Date: Wed, 17 Jan 2018 21:25:43 +0530 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.5.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Content-Language: en-US X-Originating-IP: [103.76.57.2] X-ClientProxiedBy: HK2PR02CA0172.apcprd02.prod.outlook.com (10.171.30.32) To MWHPR07MB3101.namprd07.prod.outlook.com (10.172.95.7) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: f8ba1896-4397-447a-0a08-08d55dc2cc1c X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(5600026)(4604075)(4534125)(4602075)(4627221)(201703031133081)(201702281549075)(2017052603307)(7153060)(7193020); SRVR:MWHPR07MB3101; X-Microsoft-Exchange-Diagnostics: 1; MWHPR07MB3101; 3:hLc70phbZASU8wNmgr9u40Zj4Ll7NadaRyRBdBX44XOz413Yxlr1yirGEgB3QfIJhvJPsnxA69INJzul1u4t7dC/GMPmV5n69Lls4k53fhcWmDZcVCJTmwm08zuSmnJdgUfywVfvmuY37nhvlMN0vFoa7hNlXm3xiUMO07n+cCUm1IvMHr8zOcxavT6rZPjycdiagi2VBXHYAkgcUj0tyFGqIBsrOxtHoLaZe+9FnmY73kxI5GXFxCDriV4FWOca; 25:RvaBxmlUBZsac9ScPcw1hcx9lka5mCE4AFASQemo5cyULwQmYLQ5R2vmKRtD3soy1/rLuAMIEhiBVjTUJcdW0ljOGoP8q4LTh2eAFGeTgA40LdcNqVOBxmTOVdDjWMhz0K9zbOYqGJyKJ9jfCA4G7zH4opz7JtXM/HLT06KwnHB+ZPqr576BxEKWQp6VvlORNm5TtSGIHBLioRZWLPh+GsmYSH80ltbGyFXKQ2AthIOGMEqraAeE2nCfZjnNNRzJjU7+IOdOGLd2binFU3N7j5dmXmL54247gpDWN4jmQyqL3XxEerekZ60XeIuamBANk8cXmmW28MNRsT544QgOSg==; 31:Ok/rTpCntFIAsPH0zATJZlqoRiMWIEtvC8/JyEVnZVwpqnY0v5Z9f4WW+YXZbveMQhDo3cTHTbPof8CLcork9SNn1hqbylQGTfue4KnDJAcaABMNQwZjVjr8OsKh8oyKnakuj/wUpj/tsLnYBHnCeXAeEW/xZWUPVm3/yupWDQVPJT95jAYbA9FII67bRJdbWTcBs5GnG7nQ3o4XjjPDQRCIhtK/iMP0QAuLSoYPsQc= X-MS-TrafficTypeDiagnostic: MWHPR07MB3101: X-Microsoft-Exchange-Diagnostics: 1; MWHPR07MB3101; 20:DzmPSC7PDVf+4OqJjJuhZvbo4ccCKT762JM6OeBJ8ztgNmA0u1bs1h8x119dqfqd8RW+9yt1XHYwFXWo1858Ucx0z/jJk3TYp+SMTl5C6x4igqZBsin36ianCdZaQNpsdhRWl2habpLp4wugWBrqje4QG+a1YjBODKvJ6kHRFajdXiQCFUXfYAJI5SuUcQ8Y41C29ge3EqA2U2Fuwh9iaaE0o5GmxPT0Ya8ftZcULeq/nM7stBWDsQ0QcdrnhjVZRqsDudTPgfBLrutgXJFzvh5avt2bEbF1dNxvroJJTs6/Xn4Pq+Bx6B5WH8W2FGEeZz8JsKyChqJ6YYyl+c/ofUsYxwyQ9985yqxpX7Rww6x1wrC8Ld8ekegW6M6vjkFwxflLRqJcwosd0CoT5FaBeCidt7Engp1AtuYwJiepx0lJzgmlJg3NNe6djT3bcxkPCDledifmvojaqoSl1AI3+io4BH6iFHkM9sEJIZj8B7BJ3kIUvZ60J9MYLO/IVudfoNPqXWjPHkdzvz6viQ86KIR+SpgafQ3CNU6DbGknb9JI5+5zrKUqJ5wNGGY1Z2emwX2pwSc3i/yoCDsi98EdzFc4pwQrwba0nXOHfHKENUU=; 4:9Eh/FfmWkE+L/vLQ/1BwQaDKuH9OfFaAgUx/o8w0+61OyjvuqZRGepkJR7NKWVzMeLMd4j8OltuBQvRM5+zRkMd5mPay3G0V9ZLrgE/xGxA3NT9jg/ixmRgbK8inH0nClpOwWRPP548wYx3Z1M8DinoMCTsNiyYp3k1Qe2qdUPUB23ei+TjqYgkwIcLUuz/IYcK0F7diABfeTDH178JkFxVGDeDa+tIlsGD5quQQLh3lDrmKKMCcPX4APmh+MD2pYZi1b9IkYDjbQV2a3GBlsRk1vnUtT4I/zPouPcNQ2SmprxjV8njRoqpx3mFmO/Nq X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(17755550239193); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040470)(2401047)(5005006)(8121501046)(3231023)(2400048)(944501161)(10201501046)(93006095)(3002001)(6041268)(20161123564045)(20161123560045)(20161123562045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123558120)(6072148)(201708071742011); SRVR:MWHPR07MB3101; BCL:0; PCL:0; RULEID:(100000803101)(100110400095); SRVR:MWHPR07MB3101; X-Forefront-PRVS: 0555EC8317 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(6049001)(396003)(39860400002)(366004)(39380400002)(376002)(346002)(189003)(199004)(24454002)(3846002)(68736007)(31696002)(110136005)(6666003)(2486003)(47776003)(52146003)(52116002)(23676004)(65956001)(66066001)(65806001)(65826007)(4326008)(36756003)(76176011)(31686004)(42882006)(2950100002)(97736004)(5660300001)(8936002)(77096006)(53936002)(316002)(386003)(58126008)(6486002)(16526018)(53546011)(229853002)(8676002)(93886005)(26005)(81166006)(6306002)(86152003)(90366009)(81156014)(16576012)(2906002)(105586002)(478600001)(83506002)(50466002)(6246003)(966005)(53376002)(4477795004)(6116002)(117156002)(2870700001)(72206003)(7736002)(305945005)(106356001)(25786009)(64126003)(217873001); DIR:OUT; SFP:1101; SCL:1; SRVR:MWHPR07MB3101; H:[192.168.0.105]; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; Received-SPF: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtNV0hQUjA3TUIzMTAxOzIzOm1IRGNaYXVYaGRZS3gxUXdZUlpFVU5CSndY?= =?utf-8?B?ZSsvTmltNW9ja1hZVkYrc2I2cXNOVmxBWXREZlNDNHVMODAvNHljRm45MU1D?= =?utf-8?B?cWpwZDJ6MnpJUzIxM0lMODIvT0pOZTFIWGVHRzF4UVNabWtUb3dLYjFiZ09w?= =?utf-8?B?cUZBV1ltREdKSGZ0NjFiZEhMak1FeDkxZk9vTUwrUVRLSHNtREFmc0ptNjFz?= =?utf-8?B?akRpWGZLZTRlbWlqTkNTN25PSGE4aWxkTmIydnVtdW1GQ24rb3JJWnVqTmpR?= =?utf-8?B?YUhkeHZYS1J0L29scWhrS2lzaHQvQ2NBOVFFbDRtanllQTlkYS81bDJDV2tx?= =?utf-8?B?TWxoVHVxYmZRbEZabVVkU3ptZitwdWR1QkRHQkpKc2lFTm05Y01UdmRndURT?= =?utf-8?B?RVNZeHMrUytTdlVVM0luU1hoaTdRZXFRbklSenc5THRvZEtUZkRvSm11VW5L?= =?utf-8?B?TStEZUJPT2QwYXZndE53Uk1NNEszU2FoaTZXRDFWR2dZdWllczNPV2cwSzR0?= =?utf-8?B?ditHOHdGMkRZNHpUbm4zMS9wUEErZkI0VURhZTJJTkVLa0NYeWtuajdxUytO?= =?utf-8?B?UUd2d2NCWE82YW40SXlyNnNlSHU4Z3VmWkpBb1RzSVZ5S3dYcmpKcDhlclFw?= =?utf-8?B?cTU1NlM2ZUloU0JPbzVIWTlPZmJCd2d4Tk5FM0xkSEdrOHlHUWd3czI0czVa?= =?utf-8?B?ZzM2TjNJbmNtek9aTFhNam8zNkNsWFpYa0UzUDFPUmFONWNjYjZnQjRpSEhy?= =?utf-8?B?TVBhTFJlZmpFL2lpRHhtNjFrVmh5UTJwOHBUYWREeFdTL3JVL243RWNIK1U0?= =?utf-8?B?Z0IvVUhvT01YSG0wOTBBbFh5WUZOaDRUQ3VLVENmL2tqcU1YQzVwbVNqUGN3?= =?utf-8?B?ZTQrV1Y5M01iemxIT1hraC9XaWpZaEMvbHFqMkNLUTRXOUpjRkFsY1I2NDhX?= =?utf-8?B?RnZzWGZwVjFPOGVjNjFlYzNEZDc3Q2tLRmh6eWFmenpDc3RjaE9jTFN6Rmtn?= =?utf-8?B?R0hFY2k1aE5MNUhrdVh0dFN2TmhQOERheDg4MGV5YmFHcDNwQWJpK2VqTlN6?= =?utf-8?B?VitjcnVRMlkvbmNxOGlGQ0lUQ0ZqWFQxeGczWXI5bERHUTZZalVoemtnQWR4?= =?utf-8?B?YWRZQTNBdVY5R0dod1RtaUNWdXVOcExlTG9ucWxReEdCUjNpWEJ2eWM2NC9p?= =?utf-8?B?VWZrTHF4MDBlVDhjVE10eTNsV0owa05EQXJCYWxkZzc3TFdHRGZCZU4wUDFH?= =?utf-8?B?bW5wRkRyRzhzeXRBR0UwN3J1bi9vWUpKTEY4aUZVTm9ORzd0eXlDMnd0dEhC?= =?utf-8?B?TmtoemNWWUVaMmxLNVZPUThaOVNzWDBIZFRrVVZQUDFmVlNqUWk5a3FqVzBX?= =?utf-8?B?ZS9ZNHVDcHNGVlJVbFJjM1B1Z0dHR3oycGtWV2JmbFI2RWpkQWcrRTdyalRD?= =?utf-8?B?bHQzOFE1RnNSQmFjZ0ZmK0h4VU03d1hNZHRNWkgrM0dNTy9qekRpZGVEbHBl?= =?utf-8?B?Z2dJS1ZtUGlBWDZrdklialJ0REJoRTBGaVhmM3dlQUY3a215eXQxYjBYbWdR?= =?utf-8?B?K0tpTTY3cVhKQUVidFZWNEQ0aFNjYk91Q3ZVS0JSUmc4MTM3aEJMQU04NFRs?= =?utf-8?B?MEV3WUVMNHBUQ1Bvd1djSEU0c2RWY0RqeDFacHdLajQ5amY5ekpTVm5KclQy?= =?utf-8?B?Uko0OXFseGtoL2NTZ2kyMk9OaUNJeGRFbnVkRENxNFFkMDdnYkVzK2tMQk9z?= =?utf-8?B?RHE3NDZjdnlWVVZCUUxBRTg1aWQ5eE96NUJudGNNTDZNNXZaTkpyR0xlaEZZ?= =?utf-8?B?bk1VOWFKdkJqbmdzQjlLQ0x3SzgwSWpJczR0WXlpbnArckVKWjBMZXRoZEFp?= =?utf-8?B?VDJLY2g5dHhQb21EUkNlbXI2bkxsREpHaUNZWGQyNHBCdHpjY3FCb3VKQ3Vu?= =?utf-8?B?ejZVRXFVM1pVYk1henZJTWRXU0xqOWhvbXJNa3NYU0FqQ0JwMXhGblhqOFFs?= =?utf-8?B?bkxnU21KTkpacDA3UGFxZVlmdjJzNlJKa0lJS09QK3J6a1dWVk9ISmYwVUxU?= =?utf-8?B?TVVkYlBzV2pOTmtjb2VBQ3pqR0JwZUpNcGo3TnNaeXlrbm8wWUoxeWF3c0JX?= =?utf-8?Q?+k++1K9UOyg8+oUnzy5gSt4=3D?= X-Microsoft-Exchange-Diagnostics: 1; MWHPR07MB3101; 6:ZxOhf28xRBnbRoJwcVAudt3LuTc6JjVVtKBchRvfdPP8fCe/ihj59XopCfGaLhjujajj7+tjn5UThuv72YmcpcxcBCMAV3p4pI7RC1gZCLK/aBNZabG743eT/UUSeZqnuIPAnGyI6sVPQr9Ld4nZhL7kqwvmI1SnjQD0y2fgMRWaNmgICZRQur1svQbnVj387oThxa6Qw6PPSVf9ubiSf0OOHOdM3AwYWGjh1/+Eg4+8/CEfhv10jTn2L8z2I1yFJwiWYN397h2i3apvM9QZK7R44zOvpzYDepTgaxLfWV35yQlxOBCxCxzYX7cO4ox4FdoGwPxPcUpb+eC0TvlCTrtq57wZAOXqsLANFZoayhI=; 5:hkBM71x2ThPozoEl+BOEtxZZa4LF6waZV/qCcon+LWDkBIrsd3p95cN6daNuVFMN+rAqabBWjCny3zUJNMCl+zHb0KryYGQNhJ7P7HKXePaImu6yyLyBYSr1U7JdfRkwa3MQ53WSSj/a3JKeDd1rWrUQzB/xBnUhPWFVukv9bNM=; 24:r5B4Iu9qlGnBi6yiRS8ZVGWsGU+ljTnzxmspvlVAnzqsS7noIvlBWpp/KMQkMBryajSOLfXJUgdS2VYdSpZKzr0zbW8bWzAa9cniQapvtoI=; 7:7LUqDrTD755Pqjgcf4SS2UDtfza5Z49zscby0/xMM9QpkC/L1Dy5C8DOax9bAZAMhU94sE/qded9GMnoCp/18Z9DFpUGQQBV6nODB/ai+cszQ3JQpIVxLHNMDt35jfVYEjW0WXP2JVbTbdM9iqEOXidavSKRV+5KPSdruSFz3W6dhV3wF6XJOCHMKkGsapk0HYCP0i8sjI8w9fWXZO+ue+RAMjujh+NBoiDkN4r+hoF1u5NiCnkmvcsNIhVInOG/ SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Jan 2018 15:55:55.7690 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: f8ba1896-4397-447a-0a08-08d55dc2cc1c X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR07MB3101 Subject: Re: [dpdk-dev] [RFC PATCH 2/6] mempool: implement clustered object allocation X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jan 2018 15:56:00 -0000 On Wednesday 17 January 2018 08:33 PM, Andrew Rybchenko wrote: > On 12/14/2017 04:37 PM, Olivier MATZ wrote: >> On Fri, Nov 24, 2017 at 04:06:27PM +0000, Andrew Rybchenko wrote: >>> From: "Artem V. Andreev" >>> >>> Clustered allocation is required to simplify packaging objects into >>> buckets and search of the bucket control structure by an object. >>> >>> Signed-off-by: Artem V. Andreev >>> Signed-off-by: Andrew Rybchenko >>> --- >>>   lib/librte_mempool/rte_mempool.c | 39 +++++++++++++++++++++++++++++++++++---- >>>   lib/librte_mempool/rte_mempool.h | 23 +++++++++++++++++++++-- >>>   test/test/test_mempool.c         |  2 +- >>>   3 files changed, 57 insertions(+), 7 deletions(-) >>> >>> diff --git a/lib/librte_mempool/rte_mempool.c b/lib/librte_mempool/rte_mempool.c >>> index d50dba4..43455a3 100644 >>> --- a/lib/librte_mempool/rte_mempool.c >>> +++ b/lib/librte_mempool/rte_mempool.c >>> @@ -239,7 +239,8 @@ rte_mempool_calc_obj_size(uint32_t elt_size, uint32_t flags, >>>    */ >>>   size_t >>>   rte_mempool_xmem_size(uint32_t elt_num, size_t total_elt_sz, uint32_t pg_shift, >>> -              unsigned int flags) >>> +              unsigned int flags, >>> +              const struct rte_mempool_info *info) >>>   { >>>       size_t obj_per_page, pg_num, pg_sz; >>>       unsigned int mask; >>> @@ -252,6 +253,17 @@ rte_mempool_xmem_size(uint32_t elt_num, size_t total_elt_sz, uint32_t pg_shift, >>>       if (total_elt_sz == 0) >>>           return 0; >>>   +    if (flags & MEMPOOL_F_CAPA_ALLOCATE_IN_CLUSTERS) { >>> +        unsigned int align_shift = >>> +            rte_bsf32( >>> +                rte_align32pow2(total_elt_sz * >>> +                        info->cluster_size)); >>> +        if (pg_shift < align_shift) { >>> +            return ((elt_num / info->cluster_size) + 2) >>> +                << align_shift; >>> +        } >>> +    } >>> + >> +Cc Santosh for this >> >> To be honnest, that was my fear when introducing >> MEMPOOL_F_CAPA_BLK_ALIGNED_OBJECTS and MEMPOOL_F_CAPA_PHYS_CONTIG to see more >> and more specific flags in generic code. >> >> I feel that the hidden meaning of these flags is more "if driver == foo", >> which shows that something is wrong is the current design. >> >> We have to think about another way to do. Let me try to propose >> something (to be deepen). >> >> The standard way to create a mempool is: >> >>    mp = create_empty(...) >>    set_ops_by_name(mp, "my-driver")    // optional >>    populate_default(mp)                // or populate_*() >>    obj_iter(mp, callback, arg)         // optional, to init objects >>    // and optional local func to init mempool priv >> >> First, we can consider deprecating some APIs like: >>   - rte_mempool_xmem_create() >>   - rte_mempool_xmem_size() >>   - rte_mempool_xmem_usage() >>   - rte_mempool_populate_iova_tab() >> >> These functions were introduced for xen, which was recently >> removed. They are complex to use, and are not used anywhere else in >> DPDK. >> >> Then, instead of having flags (quite hard to understand without knowing >> the underlying driver), we can let the mempool drivers do the >> populate_default() operation. For that we can add a populate_default >> field in mempool ops. Same for populate_virt(), populate_anon(), and >> populate_phys() which can return -ENOTSUP if this is not >> implemented/implementable on a specific driver, or if flags >> (NO_CACHE_ALIGN, NO_SPREAD, ...) are not supported. If the function >> pointer is NULL, use the generic function. >> >> Thanks to this, the generic code would remain understandable and won't >> have to care about how memory should be allocated for a specific driver. >> >> Thoughts? > > Yes, I agree. This week we'll provide updated version of the RFC which > covers it including transition of the mempool/octeontx. I think it is sufficient > to introduce two new ops: >  1. To calculate memory space required to store specified number of objects >  2. To populate objects in the provided memory chunk (the op will be called >      from rte_mempool_populate_iova() which is a leaf function for all >      rte_mempool_populate_*() calls. > It will allow to avoid duplication and keep memchunks housekeeping inside > mempool library. > There is also a downside of letting mempool driver to populate, which was raised in other thread. http://dpdk.org/dev/patchwork/patch/31943/ Thanks.