From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 283ECA31F3 for ; Sat, 19 Oct 2019 14:28:15 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 547F91BEC1; Sat, 19 Oct 2019 14:28:14 +0200 (CEST) Received: from mail-il1-f169.google.com (mail-il1-f169.google.com [209.85.166.169]) by dpdk.org (Postfix) with ESMTP id E028A1BE84 for ; Sat, 19 Oct 2019 14:28:12 +0200 (CEST) Received: by mail-il1-f169.google.com with SMTP id q24so799733ili.7 for ; Sat, 19 Oct 2019 05:28:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=2gZZYYseRTJLhYoYoEH2ZGYYPD+gx3dENceh+eT7bZI=; b=sfapUCVbK0tsXR4Q1Gbdgo+a1+lmwoN9e6nbBNl0rAwWbyDlL75/69jsLzVE7I2TK0 1oPp2ANFucroGt4ICKEVoUOBLenVxE+YhZ47r91L8ybSgeZxd1X1Ja14K5pFNJMlnSgB /3HiTEH8/QP01orTwQJ3XDOQEqElw/ZztdZ+wwwrMmRbkNnNSxXGJrwWnYUck9ZT1VZ7 fpYfXpftEFbQR1afA87iwBWXyzai/HAWhhhuyF1yba71Zd/rj4vG5UyTw0jfsA0ZUuCo nmGFsa5wjttxjhey59G63NTr9ok/GkV4SYIJzgHqRfMHy0suh6JzWvnKt/YMyPMMocm3 mYpg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=2gZZYYseRTJLhYoYoEH2ZGYYPD+gx3dENceh+eT7bZI=; b=nsNHeloMqgtXEttYgLo6zkEygRlj5nE04IRguSeQfKwxNNXdTSo1zef/zMZaqRi8Gj 1usC+PGKGBQmHDvhjjvTRwNZW7WFzvlxS5d+0TswAIKMUP+qGWiam9rAMmLOx+XPQuke nc5ytVhyZclFGtYoGy3scjcg6PJX0Ogkvv/uV5iw0kgG1QGmMnXcRamFkvQ50sRjNyJz vt/98gYOHbZQ1tIn0ITrqnCuZvOA8MQxtBMZ4Ko6uuKo9zhgABzW3D2nbES4UvFQXGz6 NgpCxqJOmtst/TDB61touRNIJJyYEmUWhzlon16/iDWxMIYaD24lRVH+Y3S/4RDeTQto 02wA== X-Gm-Message-State: APjAAAUDQxQG0CHFnqdksUXdIheFylWwGKs5/LFu1oPnyE16sSzoK/H1 Zte9oMq13J7K4Tly8QLNgAhag6kJ8kK1A11YoEJE8UJNXoU= X-Google-Smtp-Source: APXvYqxS50LEBTSCCddlQJcUkOcQQFonooyeqgIR3vJqILPIhi46GuOzel1TasSuPJRIr9QuBzmYtCi3S9ozAX7l4cQ= X-Received: by 2002:a92:2c03:: with SMTP id t3mr15886319ile.271.1571488091878; Sat, 19 Oct 2019 05:28:11 -0700 (PDT) MIME-Version: 1.0 References: <1571295301-25911-1-git-send-email-xuemingl@mellanox.com> In-Reply-To: From: Jerin Jacob Date: Sat, 19 Oct 2019 17:58:00 +0530 Message-ID: To: Xueming Li Cc: Olivier Matz , Andrew Rybchenko , dpdk-dev , Asaf Penso , Ori Kam , Stephen Hemminger Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-dev] [RFC] mempool: introduce indexed memory pool X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Fri, 18 Oct, 2019, 3:40 pm Xueming(Steven) Li, wrote: > > -----Original Message----- > > From: Jerin Jacob > > Sent: Friday, October 18, 2019 12:41 AM > > To: Xueming(Steven) Li > > Cc: Olivier Matz ; Andrew Rybchenko > > ; dpdk-dev ; Asaf Penso > > ; Ori Kam ; Stephen > > Hemminger > > Subject: Re: [dpdk-dev] [RFC] mempool: introduce indexed memory pool > > > > On Thu, Oct 17, 2019 at 6:43 PM Xueming(Steven) Li > > wrote: > > > > > > > -----Original Message----- > > > > From: Jerin Jacob > > > > Sent: Thursday, October 17, 2019 3:14 PM > > > > To: Xueming(Steven) Li > > > > Cc: Olivier Matz ; Andrew Rybchenko > > > > ; dpdk-dev ; Asaf Penso > > > > ; Ori Kam > > > > Subject: Re: [dpdk-dev] [RFC] mempool: introduce indexed memory pool > > > > > > > > On Thu, Oct 17, 2019 at 12:25 PM Xueming Li > > wrote: > > > > > > > > > > Indexed memory pool manages memory entries by index, allocation > > > > > from pool returns both memory pointer and index(ID). users save ID > > > > > as u32 or less(u16) instead of traditional 8 bytes pointer. Memory > > > > > could be retrieved from pool or returned to pool later by index. > > > > > > > > > > Pool allocates backend memory in chunk on demand, pool size grows > > > > > dynamically. Bitmap is used to track entry usage in chunk, thus > > > > > management overhead is one bit per entry. > > > > > > > > > > Standard rte_malloc demands malloc overhead(64B) and minimal data > > > > > size(64B). This pool aims to such cost saving also pointer size. > > > > > For scenario like creating millions of rte_flows each consists of > > > > > small pieces of memories, the difference is huge. > > > > > > > > > > Like standard memory pool, this lightweight pool only support > > > > > fixed size memory allocation. Pools should be created for each > > > > > different size. > > > > > > > > > > To facilitate memory allocated by index, a set of ILIST_XXX macro > > > > > defined to operate entries as regular LIST. > > > > > > > > > > By setting entry size to zero, pool can be used as ID generator. > > > > > > > > > > Signed-off-by: Xueming Li > > > > > --- > > > > > lib/librte_mempool/Makefile | 3 +- > > > > > lib/librte_mempool/rte_indexed_pool.c | 289 > > +++++++++++++++++++++ > > > > > lib/librte_mempool/rte_indexed_pool.h | 224 ++++++++++++++++ > > > > > > > > Can this be abstracted over the driver interface instead of creating > a new > > APIS? > > > > ie using drivers/mempool/ > > > > > > The driver interface manage memory entries with pointers, while this > api > > uses u32 index as key... > > > > I see. As a use case, it makes sense to me. > > > Have you checked the possibility reusing/extending > > lib/librte_eal/common/include/rte_bitmap.h for bitmap management, > > instead of rolling a new one? > > Yes, the rte_bitmap designed for fixed bitmap size, to grow, have to copy > almost entire bitmap(array1+array2). > This pool distribute array2 into each trunk, and the trunk array actually > plays the array1 role. > When growing, just grow array1 which is smaller, no touch to existing > array2 in each trunk. > IMO, Growing bit map is generic problem so moving bitmap management logic to common place will be usefull for other libraries in future. My suggestion would be to enchanse rte_bitmap to support dynamic bitmap through new APIs. > The map_xxx() naming might confused people, I'll make following change in > next version: > map_get()/map_set(): only used once and the code is simple, move > code into caller. > map_is_empty()/map_clear()/ : unused, remove > map_clear_any(): relative simple, embed into caller. >