From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id ABE60A32A4 for ; Fri, 25 Oct 2019 18:28:48 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 68F991D50C; Fri, 25 Oct 2019 18:28:47 +0200 (CEST) Received: from mail-io1-f46.google.com (mail-io1-f46.google.com [209.85.166.46]) by dpdk.org (Postfix) with ESMTP id 2E1B91D44F for ; Fri, 25 Oct 2019 18:28:45 +0200 (CEST) Received: by mail-io1-f46.google.com with SMTP id r144so3079457iod.8 for ; Fri, 25 Oct 2019 09:28:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=Rx0EUgFcEJ2MTLRUK9/4czc4zvICTqaGcOx0vXJMxuU=; b=O8KIYpCmyRf2k99+QfzOKXHSiqQLEVRzeL8eFUaeu5B+BBCQa/fFmYWewIIkPwwmb4 9Irec7QNAmluXk7tU0lPa1f62/aWNXJmSLcwMm5Y88ci5znnvRT4cnXD+kNYkDVnMLlH SZ5QuycegRRbawXrLpWL+ZeTGOax8mZYGnHTnjE+HbJXHZE4HnhS0AyQIFY4X++0zDYi dhslinw0FLIroJQ5cGqXTBdfBuBT5zQkiygscqvIU5Cb44WmOBwHUy0y24rADw+0DkgT Wp3RuLqmLnLpaTI4X0XtfStyjYSZbpE2mAuCZpbYDTN3V+GgyrGNDvedh3IlmkYMcpH9 0FPA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=Rx0EUgFcEJ2MTLRUK9/4czc4zvICTqaGcOx0vXJMxuU=; b=kuvaiDvh0A0AEwPOyjpd+51MhIbcpeZ5jLL5PN3NY0zt/uASCDTiuPBSeIGiJ1MloK Lr03ycofFIAKuhllg9UMDgM5aKK9dQlxLpBAln9wMpxAgD9QzCffUR6qHA0j9hdd7pDg +h4OoRBvw/7d1h+x6zJjMOBLyjyyS9ddTwZQjD+gv66Th4z0r+ApWBxVQ/yhCTOwnq3u QXiTppraKOoR4H5LESUb+JvcWzFHKyz/Rt//PnVpdiYb9GRue0bkKcv9f3fuQGeKESug wCt/jQpUohbRcH2SYafnnFAhfLrIO7gW9TOVuvPEqRgplhAD2OOZTRPUP7F94hLbPEVc 9YdA== X-Gm-Message-State: APjAAAVCuL9PSPOVExQpHQQxQdfXUmcxiuZTLlN6i1i+2sP+z+8ZH0WU Lwv2p2eNkOAlSA7ynC1+dePBhzq04HG6Ucwz5Y8= X-Google-Smtp-Source: APXvYqx0mcJsD/leo9YuRgXQ2XEhd8vUboOkuXyDeIHmGUN4PYm70rQz3BzYgsRrmEjWFETxy4re664EyzlYeTZJ1cQ= X-Received: by 2002:a6b:c701:: with SMTP id x1mr4582487iof.162.1572020924282; Fri, 25 Oct 2019 09:28:44 -0700 (PDT) MIME-Version: 1.0 References: <1571295301-25911-1-git-send-email-xuemingl@mellanox.com> In-Reply-To: From: Jerin Jacob Date: Fri, 25 Oct 2019 21:58:28 +0530 Message-ID: To: "Xueming(Steven) Li" Cc: Olivier Matz , Andrew Rybchenko , dpdk-dev , Asaf Penso , Ori Kam , Stephen Hemminger , cristian.dumitrescu@intel.com Content-Type: text/plain; charset="UTF-8" Subject: Re: [dpdk-dev] [RFC] mempool: introduce indexed memory pool X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Fri, Oct 25, 2019 at 8:59 PM Xueming(Steven) Li wrote: > > > >From: Jerin Jacob > >Sent: Saturday, October 19, 2019 8:28 PM > > > >On Fri, 18 Oct, 2019, 3:40 pm Xueming(Steven) Li, wrote: > >> -----Original Message----- > >> From: Jerin Jacob > >> Sent: Friday, October 18, 2019 12:41 AM > >> To: Xueming(Steven) Li > >> Cc: Olivier Matz ; Andrew Rybchenko > >> ; dpdk-dev ; Asaf Penso > >> ; Ori Kam ; Stephen > >> Hemminger > >> Subject: Re: [dpdk-dev] [RFC] mempool: introduce indexed memory pool > >> > >> On Thu, Oct 17, 2019 at 6:43 PM Xueming(Steven) Li > >> wrote: > >> > > >> > > -----Original Message----- > >> > > From: Jerin Jacob > >> > > Sent: Thursday, October 17, 2019 3:14 PM > >> > > To: Xueming(Steven) Li > >> > > Cc: Olivier Matz ; Andrew Rybchenko > >> > > ; dpdk-dev ; Asaf Penso > >> > > ; Ori Kam > >> > > Subject: Re: [dpdk-dev] [RFC] mempool: introduce indexed memory pool > >> > > > >> > > On Thu, Oct 17, 2019 at 12:25 PM Xueming Li > >> wrote: > >> > > > > >> > > > Indexed memory pool manages memory entries by index, allocation > >> > > > from pool returns both memory pointer and index(ID). users save ID > >> > > > as u32 or less(u16) instead of traditional 8 bytes pointer. Memory > >> > > > could be retrieved from pool or returned to pool later by index. > >> > > > > >> > > > Pool allocates backend memory in chunk on demand, pool size grows > >> > > > dynamically. Bitmap is used to track entry usage in chunk, thus > >> > > > management overhead is one bit per entry. > >> > > > > >> > > > Standard rte_malloc demands malloc overhead(64B) and minimal data > >> > > > size(64B). This pool aims to such cost saving also pointer size. > >> > > > For scenario like creating millions of rte_flows each consists of > >> > > > small pieces of memories, the difference is huge. > >> > > > > >> > > > Like standard memory pool, this lightweight pool only support > >> > > > fixed size memory allocation. Pools should be created for each > >> > > > different size. > >> > > > > >> > > > To facilitate memory allocated by index, a set of ILIST_XXX macro > >> > > > defined to operate entries as regular LIST. > >> > > > > >> > > > By setting entry size to zero, pool can be used as ID generator. > >> > > > > >> > > > Signed-off-by: Xueming Li > >> > > > --- > >> > > > lib/librte_mempool/Makefile | 3 +- > >> > > > lib/librte_mempool/rte_indexed_pool.c | 289 > >> +++++++++++++++++++++ > >> > > > lib/librte_mempool/rte_indexed_pool.h | 224 ++++++++++++++++ > >> > > > >> > > Can this be abstracted over the driver interface instead of creating a new > >> APIS? > >> > > ie using drivers/mempool/ > >> > > >> > The driver interface manage memory entries with pointers, while this api > >> uses u32 index as key... > >> > >> I see. As a use case, it makes sense to me. > > > >> Have you checked the possibility reusing/extending > >> lib/librte_eal/common/include/rte_bitmap.h for bitmap management, > >> instead of rolling a new one? > > > >Yes, the rte_bitmap designed for fixed bitmap size, to grow, have to copy almost entire bitmap(array1+array2). > >This pool distribute array2 into each trunk, and the trunk array actually plays the array1 role. > >When growing, just grow array1 which is smaller, no touch to existing array2 in each trunk. > > > >IMO, Growing bit map is generic problem so moving bitmap management logic to common place will be usefull for other libraries in future. My suggestion would be to enchanse rte_bitmap to support dynamic bitmap through new APIs. > > > Interesting that people always think this api a bitmap, now start to realize it meaningful, memory just an optional attachment storage to each bit :) > I'll append missing api like set bitmap by index, then move it to eal common folder, the header file should be rte_bitmap2.h? + cristian.dumitrescu@intel.com(rte_bitmap.h maintainer) for any comments. rte_bitmap2.h may not be an intuitive name. One option could behave separate APIs for the new case in rte_bitmap.h. No strong opinions on the code organization in eal. > > > > > > >The map_xxx() naming might confused people, I'll make following change in next version: > > map_get()/map_set(): only used once and the code is simple, move code into caller. > > map_is_empty()/map_clear()/ : unused, remove > > map_clear_any(): relative simple, embed into caller. > >