From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR01-HE1-obe.outbound.protection.outlook.com (mail-eopbgr130088.outbound.protection.outlook.com [40.107.13.88]) by dpdk.org (Postfix) with ESMTP id 58925288C for ; Fri, 18 Jan 2019 06:05:04 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector1-arm-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zFYVKbueKRTwgQS4qSER28AwdVpvH16cpSBAK21uYRE=; b=kwHFGQzaxT3q50iIs0SHegI61s8ea36oxg506wZm+m2amxitXipT2RghKpKurD8w++L6a/MEH9pC+90+Fe8N2gzknh4ula+ObPso0EFBN+Xsp5txtrAd3O898cVxJ8ZVqtqLMuPNl1ARf3nieE5dEvgOsKora2k0snWx6p5ZgYs= Received: from AM6PR08MB3672.eurprd08.prod.outlook.com (20.177.115.76) by AM6PR08MB3239.eurprd08.prod.outlook.com (52.135.164.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1537.26; Fri, 18 Jan 2019 05:05:02 +0000 Received: from AM6PR08MB3672.eurprd08.prod.outlook.com ([fe80::25ec:2db7:d268:2b7b]) by AM6PR08MB3672.eurprd08.prod.outlook.com ([fe80::25ec:2db7:d268:2b7b%2]) with mapi id 15.20.1537.018; Fri, 18 Jan 2019 05:05:02 +0000 From: Honnappa Nagarahalli To: Gage Eads , "dev@dpdk.org" CC: "olivier.matz@6wind.com" , "arybchenko@solarflare.com" , "bruce.richardson@intel.com" , "konstantin.ananyev@intel.com" , "Gavin Hu (Arm Technology China)" , nd , Honnappa Nagarahalli , nd Thread-Topic: [dpdk-dev] [PATCH v4 2/2] mempool/nb_stack: add non-blocking stack mempool Thread-Index: AQHUrnqu5jX8tbxycUCltqyIk7tCgKWzwdxQ Date: Fri, 18 Jan 2019 05:05:02 +0000 Message-ID: References: <20190116151835.22424-1-gage.eads@intel.com> <20190117153659.28477-1-gage.eads@intel.com> <20190117153659.28477-3-gage.eads@intel.com> In-Reply-To: <20190117153659.28477-3-gage.eads@intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=Honnappa.Nagarahalli@arm.com; x-originating-ip: [217.140.103.75] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; AM6PR08MB3239; 6:3SJ/Pv16HLg/8GZGNVrtQguOpSwHsRQtn1X5G3yG4w4QER25doumIS5mEwmzs9FMXDjRW+YXspbF34/jRmA6RxcD3FuU+yMB3KNXcWnBv6XtNK1o6ErFgRnwfMme8XKJoP0CZRPQKnM8lQhu4OcALyMsxUkSwc/hYCylE5o2a+zFHsie/U3YPHvv3Kgcsq9/cFaIPlW/3FqP2HJY3xacRC8p5TxI7ekHtPZ7UIOJ7sgDUNi9ceN2F+nj4k3iSQiYlVrF5+rSgiP1Z3IhC3uLfCd72NSMW8Uw7Gop/nFHoVc0sLFlWlgjvf1nYbdgz1QHVnEn+oCM19KxyKehJyByRQPkl3XoL/rrarxeDh693k5pbAsIPpq//M+rgiV8NCn2ZURYqGPbRIK19SmZMFHNcESdcRLpV1ksNKSf4pKuwRSNmy/ajXFWmQ+/05quxhXS8bbPNgFAFjNc+awlJqEgTQ==; 5:Cho9PRMu5jEHMCx7gP5pILbDgJ6kmYMd4PkwAAID9Kr9l63xAPWIcQX/lj4HXCnHhODtmUIWiPJ9w+ffamviESX7sGWFCw4ITHmIOg3Px4l9RhQZzhct+5C0+Sfad+ZgBRaTtzth9rNcELhAQi8McswCVobI/r591L1xm+FMKd2ABTYMy9U5qa1if5Np3hEg0ogJWf7sQb7dxhToQOie3w==; 7:L5gjWvfoqsLsCKO3VP91FHUL3wKiosT+ZYE9iZ6VDsKJ95bzVmMljc67MhAPfyqVNJqNZ/HXj7a/YI8tXNdLA45SQFzCJjNU3tjB/r0ZFgkI0tXOrEBe+QcJTIJKh3uL8f2yt+2yjomj7b6n7JAG7w== x-ms-exchange-antispam-srfa-diagnostics: SOS;SOR; x-ms-office365-filtering-correlation-id: d03e6e87-7004-44ea-1857-08d67d028121 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600109)(711020)(4618075)(2017052603328)(7153060)(7193020); SRVR:AM6PR08MB3239; x-ms-traffictypediagnostic: AM6PR08MB3239: nodisclaimer: True x-microsoft-antispam-prvs: x-forefront-prvs: 0921D55E4F x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(396003)(346002)(136003)(39860400002)(376002)(366004)(199004)(189003)(229853002)(5660300001)(106356001)(345774005)(53946003)(105586002)(86362001)(76176011)(54906003)(25786009)(53936002)(72206003)(99286004)(74316002)(316002)(14454004)(68736007)(7696005)(71200400001)(30864003)(110136005)(71190400001)(11346002)(8936002)(102836004)(6506007)(55016002)(26005)(81166006)(186003)(81156014)(66066001)(446003)(478600001)(256004)(33656002)(14444005)(6116002)(486006)(2906002)(3846002)(7736002)(97736004)(6436002)(4326008)(2501003)(9686003)(6246003)(305945005)(476003)(21314003); DIR:OUT; SFP:1101; SCL:1; SRVR:AM6PR08MB3239; H:AM6PR08MB3672.eurprd08.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: p39p9x+jcfIxPxCbiWedXRR2uhumRUCRAlC63zdoPgxoHPyr5aMm8PaqemrsiBRxSvgHYrNEXKE6EpBMVmYlvX5Gllf6Xk799tPrw0M8UXCWP56xVgGeJdA6ao59hjsmeWSFzpweJiMBQLMm3tAhQsoMB3QnhsQGFGypJNJ6fmMoPBsygDm1WmRL5npE3wHuvRIB0tVxFCvi23lfCMOT9bJ40g+FrOXzHu3rlbvQOi8JCFOK6Jxat0w5GIRcYU8Nl97oKI4hRv+t7WjCuFkEk1ASBB3UiEEqI2xTyiBWcp7x+eZ7SQDhssbz8RLyW676Zf4aufi6xC9zJvLKtv10a3uP7dqN+vEQBUwbT6K8q1uKmpJ2xNOsMrkMCRyxXJFZGjleC5oxUjlyZ1PjNyD1T/2yNdatDT1SrZ2Kv5DAa+E= spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-Network-Message-Id: d03e6e87-7004-44ea-1857-08d67d028121 X-MS-Exchange-CrossTenant-originalarrivaltime: 18 Jan 2019 05:05:02.8399 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB3239 Subject: Re: [dpdk-dev] [PATCH v4 2/2] mempool/nb_stack: add non-blocking stack mempool X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Jan 2019 05:05:04 -0000 Hi Gage, Thank you for your contribution on non-blocking data structures. I thi= nk they are important to extend DPDK into additional use cases. I am wondering if it makes sense to decouple the NB stack data structure fr= om mempool driver (similar to rte_ring)? I see that stack based mempool imp= lements the stack data structure in the driver. But, NB stack might not be = such a trivial data structure. It might be useful for the applications or o= ther use cases as well. I also suggest that we use C11 __atomic_xxx APIs for memory operations. The= rte_atomic64_xxx APIs use __sync_xxx APIs which do not provide the capabil= ity to express memory orderings. Please find few comments inline. >=20 > This commit adds support for non-blocking (linked list based) stack > mempool handler. The stack uses a 128-bit compare-and-swap instruction, > and thus is limited to x86_64. The 128-bit CAS atomically updates the sta= ck > top pointer and a modification counter, which protects against the ABA > problem. >=20 > In mempool_perf_autotest the lock-based stack outperforms the non- > blocking handler*, however: > - For applications with preemptible pthreads, a lock-based stack's > worst-case performance (i.e. one thread being preempted while > holding the spinlock) is much worse than the non-blocking stack's. > - Using per-thread mempool caches will largely mitigate the performance > difference. >=20 > *Test setup: x86_64 build with default config, dual-socket Xeon E5-2699 v= 4, > running on isolcpus cores with a tickless scheduler. The lock-based stack= 's > rate_persec was 1x-3.5x the non-blocking stack's. >=20 > Signed-off-by: Gage Eads > Acked-by: Andrew Rybchenko > --- > MAINTAINERS | 4 + > config/common_base | 1 + > doc/guides/prog_guide/env_abstraction_layer.rst | 5 + > drivers/mempool/Makefile | 3 + > drivers/mempool/meson.build | 3 +- > drivers/mempool/nb_stack/Makefile | 23 ++++ > drivers/mempool/nb_stack/meson.build | 6 + > drivers/mempool/nb_stack/nb_lifo.h | 147 > +++++++++++++++++++++ > drivers/mempool/nb_stack/rte_mempool_nb_stack.c | 125 > ++++++++++++++++++ > .../nb_stack/rte_mempool_nb_stack_version.map | 4 + > mk/rte.app.mk | 7 +- > 11 files changed, 325 insertions(+), 3 deletions(-) create mode 100644 > drivers/mempool/nb_stack/Makefile create mode 100644 > drivers/mempool/nb_stack/meson.build > create mode 100644 drivers/mempool/nb_stack/nb_lifo.h > create mode 100644 drivers/mempool/nb_stack/rte_mempool_nb_stack.c > create mode 100644 > drivers/mempool/nb_stack/rte_mempool_nb_stack_version.map >=20 > diff --git a/MAINTAINERS b/MAINTAINERS > index 470f36b9c..5519d3323 100644 > --- a/MAINTAINERS > +++ b/MAINTAINERS > @@ -416,6 +416,10 @@ M: Artem V. Andreev > M: Andrew Rybchenko > F: drivers/mempool/bucket/ >=20 > +Non-blocking stack memory pool > +M: Gage Eads > +F: drivers/mempool/nb_stack/ > + >=20 > Bus Drivers > ----------- > diff --git a/config/common_base b/config/common_base index > 964a6956e..8a51f36b1 100644 > --- a/config/common_base > +++ b/config/common_base > @@ -726,6 +726,7 @@ CONFIG_RTE_LIBRTE_MEMPOOL_DEBUG=3Dn # > CONFIG_RTE_DRIVER_MEMPOOL_BUCKET=3Dy > CONFIG_RTE_DRIVER_MEMPOOL_BUCKET_SIZE_KB=3D64 > +CONFIG_RTE_DRIVER_MEMPOOL_NB_STACK=3Dy > CONFIG_RTE_DRIVER_MEMPOOL_RING=3Dy > CONFIG_RTE_DRIVER_MEMPOOL_STACK=3Dy >=20 > diff --git a/doc/guides/prog_guide/env_abstraction_layer.rst > b/doc/guides/prog_guide/env_abstraction_layer.rst > index 929d76dba..9497b879c 100644 > --- a/doc/guides/prog_guide/env_abstraction_layer.rst > +++ b/doc/guides/prog_guide/env_abstraction_layer.rst > @@ -541,6 +541,11 @@ Known Issues >=20 > 5. It MUST not be used by multi-producer/consumer pthreads, whose > scheduling policies are SCHED_FIFO or SCHED_RR. >=20 > + Alternatively, x86_64 applications can use the non-blocking stack > mempool handler. When considering this handler, note that: > + > + - it is limited to the x86_64 platform, because it uses an instruction= (16- > byte compare-and-swap) that is not available on other platforms. ^^^^^^^^^^^^^^^^^^^^^^^^^^= ^^^^^^ Arm architecture supports similar instructions. I suggest to simplify this = statement to indicate that 'nb_stack feature is available for x86_64 platfo= rms currently' > + - it has worse average-case performance than the non-preemptive > rte_ring, but software caching (e.g. the mempool cache) can mitigate this= by > reducing the number of handler operations. > + > + rte_timer >=20 > Running ``rte_timer_manage()`` on a non-EAL pthread is not allowed. > However, resetting/stopping the timer from a non-EAL pthread is allowed. > diff --git a/drivers/mempool/Makefile b/drivers/mempool/Makefile index > 28c2e8360..895cf8a34 100644 > --- a/drivers/mempool/Makefile > +++ b/drivers/mempool/Makefile > @@ -10,6 +10,9 @@ endif > ifeq ($(CONFIG_RTE_EAL_VFIO)$(CONFIG_RTE_LIBRTE_FSLMC_BUS),yy) > DIRS-$(CONFIG_RTE_LIBRTE_DPAA2_MEMPOOL) +=3D dpaa2 endif > +ifeq ($(CONFIG_RTE_ARCH_X86_64),y) > +DIRS-$(CONFIG_RTE_DRIVER_MEMPOOL_NB_STACK) +=3D nb_stack endif > DIRS-$(CONFIG_RTE_DRIVER_MEMPOOL_RING) +=3D ring > DIRS-$(CONFIG_RTE_DRIVER_MEMPOOL_STACK) +=3D stack > DIRS-$(CONFIG_RTE_LIBRTE_OCTEONTX_MEMPOOL) +=3D octeontx diff --git > a/drivers/mempool/meson.build b/drivers/mempool/meson.build index > 4527d9806..220cfaf63 100644 > --- a/drivers/mempool/meson.build > +++ b/drivers/mempool/meson.build > @@ -1,7 +1,8 @@ > # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2017 Intel > Corporation >=20 > -drivers =3D ['bucket', 'dpaa', 'dpaa2', 'octeontx', 'ring', 'stack'] > +drivers =3D ['bucket', 'dpaa', 'dpaa2', 'nb_stack', 'octeontx', 'ring', > +'stack'] > + > std_deps =3D ['mempool'] > config_flag_fmt =3D 'RTE_LIBRTE_@0@_MEMPOOL' > driver_name_fmt =3D 'rte_mempool_@0@' > diff --git a/drivers/mempool/nb_stack/Makefile > b/drivers/mempool/nb_stack/Makefile > new file mode 100644 > index 000000000..318b18283 > --- /dev/null > +++ b/drivers/mempool/nb_stack/Makefile > @@ -0,0 +1,23 @@ > +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2019 Intel > +Corporation > + > +include $(RTE_SDK)/mk/rte.vars.mk > + > +# > +# library name > +# > +LIB =3D librte_mempool_nb_stack.a > + > +CFLAGS +=3D -O3 > +CFLAGS +=3D $(WERROR_FLAGS) > + > +# Headers > +LDLIBS +=3D -lrte_eal -lrte_mempool > + > +EXPORT_MAP :=3D rte_mempool_nb_stack_version.map > + > +LIBABIVER :=3D 1 > + > +SRCS-$(CONFIG_RTE_DRIVER_MEMPOOL_NB_STACK) +=3D > rte_mempool_nb_stack.c > + > +include $(RTE_SDK)/mk/rte.lib.mk > diff --git a/drivers/mempool/nb_stack/meson.build > b/drivers/mempool/nb_stack/meson.build > new file mode 100644 > index 000000000..7dec72242 > --- /dev/null > +++ b/drivers/mempool/nb_stack/meson.build > @@ -0,0 +1,6 @@ > +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2019 Intel > +Corporation > + > +build =3D dpdk_conf.has('RTE_ARCH_X86_64') > + > +sources =3D files('rte_mempool_nb_stack.c') > diff --git a/drivers/mempool/nb_stack/nb_lifo.h > b/drivers/mempool/nb_stack/nb_lifo.h > new file mode 100644 > index 000000000..ad4a3401f > --- /dev/null > +++ b/drivers/mempool/nb_stack/nb_lifo.h > @@ -0,0 +1,147 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(c) 2019 Intel Corporation > + */ > + > +#ifndef _NB_LIFO_H_ > +#define _NB_LIFO_H_ > + > +struct nb_lifo_elem { > + void *data; > + struct nb_lifo_elem *next; > +}; > + > +struct nb_lifo_head { > + struct nb_lifo_elem *top; /**< Stack top */ > + uint64_t cnt; /**< Modification counter */ }; Minor comment, mentioning ABA problem in the comments for 'cnt' will be hel= pful. > + > +struct nb_lifo { > + volatile struct nb_lifo_head head __rte_aligned(16); > + rte_atomic64_t len; > +} __rte_cache_aligned; > + > +static __rte_always_inline void > +nb_lifo_init(struct nb_lifo *lifo) > +{ > + memset(lifo, 0, sizeof(*lifo)); > + rte_atomic64_set(&lifo->len, 0); > +} > + > +static __rte_always_inline unsigned int nb_lifo_len(struct nb_lifo > +*lifo) { > + /* nb_lifo_push() and nb_lifo_pop() do not update the list's > contents > + * and lifo->len atomically, which can cause the list to appear > shorter > + * than it actually is if this function is called while other threads > + * are modifying the list. > + * > + * However, given the inherently approximate nature of the > get_count > + * callback -- even if the list and its size were updated atomically, > + * the size could change between when get_count executes and > when the > + * value is returned to the caller -- this is acceptable. > + * > + * The lifo->len updates are placed such that the list may appear to > + * have fewer elements than it does, but will never appear to have > more > + * elements. If the mempool is near-empty to the point that this is a > + * concern, the user should consider increasing the mempool size. > + */ > + return (unsigned int)rte_atomic64_read(&lifo->len); > +} > + > +static __rte_always_inline void > +nb_lifo_push(struct nb_lifo *lifo, > + struct nb_lifo_elem *first, > + struct nb_lifo_elem *last, > + unsigned int num) > +{ > + while (1) { > + struct nb_lifo_head old_head, new_head; > + > + old_head =3D lifo->head; > + > + /* Swing the top pointer to the first element in the list and > + * make the last element point to the old top. > + */ > + new_head.top =3D first; > + new_head.cnt =3D old_head.cnt + 1; > + > + last->next =3D old_head.top; > + > + if (rte_atomic128_cmpset((volatile uint64_t *)&lifo->head, > + (uint64_t *)&old_head, > + (uint64_t *)&new_head)) > + break; > + } Minor comment, this can be a do-while loop (for ex: similar to the one in _= _rte_ring_move_prod_head) > + > + rte_atomic64_add(&lifo->len, num); > +} > + > +static __rte_always_inline void > +nb_lifo_push_single(struct nb_lifo *lifo, struct nb_lifo_elem *elem) { > + nb_lifo_push(lifo, elem, elem, 1); > +} > + > +static __rte_always_inline struct nb_lifo_elem * nb_lifo_pop(struct > +nb_lifo *lifo, > + unsigned int num, > + void **obj_table, > + struct nb_lifo_elem **last) > +{ > + struct nb_lifo_head old_head; > + > + /* Reserve num elements, if available */ > + while (1) { > + uint64_t len =3D rte_atomic64_read(&lifo->len); > + > + /* Does the list contain enough elements? */ > + if (len < num) > + return NULL; > + > + if (rte_atomic64_cmpset((volatile uint64_t *)&lifo->len, > + len, len - num)) > + break; > + } > + > + /* Pop num elements */ > + while (1) { > + struct nb_lifo_head new_head; > + struct nb_lifo_elem *tmp; > + unsigned int i; > + > + old_head =3D lifo->head; > + > + tmp =3D old_head.top; > + > + /* Traverse the list to find the new head. A next pointer will > + * either point to another element or NULL; if a thread > + * encounters a pointer that has already been popped, the > CAS > + * will fail. > + */ > + for (i =3D 0; i < num && tmp !=3D NULL; i++) { > + if (obj_table) This 'if' check can be outside the for loop. May be use RTE_ASSERT in the b= eginning of the function? > + obj_table[i] =3D tmp->data; > + if (last) > + *last =3D tmp; > + tmp =3D tmp->next; > + } > + > + /* If NULL was encountered, the list was modified while > + * traversing it. Retry. > + */ > + if (i !=3D num) > + continue; > + > + new_head.top =3D tmp; > + new_head.cnt =3D old_head.cnt + 1; > + > + if (rte_atomic128_cmpset((volatile uint64_t *)&lifo->head, > + (uint64_t *)&old_head, > + (uint64_t *)&new_head)) > + break; > + } > + > + return old_head.top; > +} > + > +#endif /* _NB_LIFO_H_ */ > diff --git a/drivers/mempool/nb_stack/rte_mempool_nb_stack.c > b/drivers/mempool/nb_stack/rte_mempool_nb_stack.c > new file mode 100644 > index 000000000..1818a2cfa > --- /dev/null > +++ b/drivers/mempool/nb_stack/rte_mempool_nb_stack.c > @@ -0,0 +1,125 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(c) 2019 Intel Corporation > + */ > + > +#include > +#include > +#include > + > +#include "nb_lifo.h" > + > +struct rte_mempool_nb_stack { > + uint64_t size; > + struct nb_lifo used_lifo; /**< LIFO containing mempool pointers */ > + struct nb_lifo free_lifo; /**< LIFO containing unused LIFO elements > */ > +}; > + > +static int > +nb_stack_alloc(struct rte_mempool *mp) > +{ > + struct rte_mempool_nb_stack *s; > + struct nb_lifo_elem *elems; > + unsigned int n =3D mp->size; > + unsigned int size, i; > + > + size =3D sizeof(*s) + n * sizeof(struct nb_lifo_elem); IMO, the allocation of the stack elements can be moved under nb_lifo_init A= PI, it would make the nb stack code modular. > + > + /* Allocate our local memory structure */ > + s =3D rte_zmalloc_socket("mempool-nb_stack", > + size, > + RTE_CACHE_LINE_SIZE, > + mp->socket_id); > + if (s =3D=3D NULL) { > + RTE_LOG(ERR, MEMPOOL, "Cannot allocate nb_stack!\n"); > + return -ENOMEM; > + } > + > + s->size =3D n; > + > + nb_lifo_init(&s->used_lifo); > + nb_lifo_init(&s->free_lifo); > + > + elems =3D (struct nb_lifo_elem *)&s[1]; > + for (i =3D 0; i < n; i++) > + nb_lifo_push_single(&s->free_lifo, &elems[i]); This also can be added to nb_lifo_init API. > + > + mp->pool_data =3D s; > + > + return 0; > +} > + > +static int > +nb_stack_enqueue(struct rte_mempool *mp, void * const *obj_table, > + unsigned int n) > +{ > + struct rte_mempool_nb_stack *s =3D mp->pool_data; > + struct nb_lifo_elem *first, *last, *tmp; > + unsigned int i; > + > + if (unlikely(n =3D=3D 0)) > + return 0; > + > + /* Pop n free elements */ > + first =3D nb_lifo_pop(&s->free_lifo, n, NULL, NULL); > + if (unlikely(first =3D=3D NULL)) > + return -ENOBUFS; > + > + /* Prepare the list elements */ > + tmp =3D first; > + for (i =3D 0; i < n; i++) { > + tmp->data =3D obj_table[i]; > + last =3D tmp; > + tmp =3D tmp->next; > + } > + > + /* Enqueue them to the used list */ > + nb_lifo_push(&s->used_lifo, first, last, n); > + > + return 0; > +} > + > +static int > +nb_stack_dequeue(struct rte_mempool *mp, void **obj_table, > + unsigned int n) > +{ > + struct rte_mempool_nb_stack *s =3D mp->pool_data; > + struct nb_lifo_elem *first, *last; > + > + if (unlikely(n =3D=3D 0)) > + return 0; > + > + /* Pop n used elements */ > + first =3D nb_lifo_pop(&s->used_lifo, n, obj_table, &last); > + if (unlikely(first =3D=3D NULL)) > + return -ENOENT; > + > + /* Enqueue the list elements to the free list */ > + nb_lifo_push(&s->free_lifo, first, last, n); > + > + return 0; > +} > + > +static unsigned > +nb_stack_get_count(const struct rte_mempool *mp) { > + struct rte_mempool_nb_stack *s =3D mp->pool_data; > + > + return nb_lifo_len(&s->used_lifo); > +} > + > +static void > +nb_stack_free(struct rte_mempool *mp) > +{ > + rte_free(mp->pool_data); > +} > + > +static struct rte_mempool_ops ops_nb_stack =3D { > + .name =3D "nb_stack", > + .alloc =3D nb_stack_alloc, > + .free =3D nb_stack_free, > + .enqueue =3D nb_stack_enqueue, > + .dequeue =3D nb_stack_dequeue, > + .get_count =3D nb_stack_get_count > +}; > + > +MEMPOOL_REGISTER_OPS(ops_nb_stack); > diff --git a/drivers/mempool/nb_stack/rte_mempool_nb_stack_version.map > b/drivers/mempool/nb_stack/rte_mempool_nb_stack_version.map > new file mode 100644 > index 000000000..fc8c95e91 > --- /dev/null > +++ b/drivers/mempool/nb_stack/rte_mempool_nb_stack_version.map > @@ -0,0 +1,4 @@ > +DPDK_19.05 { > + > + local: *; > +}; > diff --git a/mk/rte.app.mk b/mk/rte.app.mk index 02e8b6f05..d4b4aaaf6 > 100644 > --- a/mk/rte.app.mk > +++ b/mk/rte.app.mk > @@ -131,8 +131,11 @@ endif > ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),n) > # plugins (link only if static libraries) >=20 > -_LDLIBS-$(CONFIG_RTE_DRIVER_MEMPOOL_BUCKET) +=3D - > lrte_mempool_bucket > -_LDLIBS-$(CONFIG_RTE_DRIVER_MEMPOOL_STACK) +=3D - > lrte_mempool_stack > +_LDLIBS-$(CONFIG_RTE_DRIVER_MEMPOOL_BUCKET) +=3D - > lrte_mempool_bucket > +ifeq ($(CONFIG_RTE_ARCH_X86_64),y) > +_LDLIBS-$(CONFIG_RTE_DRIVER_MEMPOOL_NB_STACK) +=3D - > lrte_mempool_nb_stack > +endif > +_LDLIBS-$(CONFIG_RTE_DRIVER_MEMPOOL_STACK) +=3D - > lrte_mempool_stack > ifeq ($(CONFIG_RTE_LIBRTE_DPAA_BUS),y) > _LDLIBS-$(CONFIG_RTE_LIBRTE_DPAA_MEMPOOL) +=3D -lrte_mempool_dpaa > endif > -- > 2.13.6