From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 739B7A034E; Thu, 6 Jan 2022 13:24:14 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E88E64114E; Thu, 6 Jan 2022 13:24:12 +0100 (CET) Received: from smartserver.smartsharesystems.com (smartserver.smartsharesystems.com [77.243.40.215]) by mails.dpdk.org (Postfix) with ESMTP id E069F4114D for ; Thu, 6 Jan 2022 13:24:10 +0100 (CET) Received: from dkrd2.smartsharesys.local ([192.168.4.12]) by smartserver.smartsharesystems.com with Microsoft SMTPSVC(6.0.3790.4675); Thu, 6 Jan 2022 13:24:05 +0100 From: =?UTF-8?q?Morten=20Br=C3=B8rup?= To: olivier.matz@6wind.com, andrew.rybchenko@oktetlabs.ru Cc: dev@dpdk.org, =?UTF-8?q?Morten=20Br=C3=B8rup?= Subject: [PATCH] mempool: optimize incomplete cache handling Date: Thu, 6 Jan 2022 13:23:45 +0100 Message-Id: <20220106122345.82740-1-mb@smartsharesystems.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <98CBD80474FA8B44BF855DF32C47DC35D86DB2@smartserver.smartshare.dk> References: <98CBD80474FA8B44BF855DF32C47DC35D86DB2@smartserver.smartshare.dk> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-OriginalArrivalTime: 06 Jan 2022 12:24:05.0006 (UTC) FILETIME=[4B8C4AE0:01D802F8] X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org A flush threshold for the mempool cache was introduced in DPDK version 1.3, but rte_mempool_do_generic_get() was not completely updated back then. The incompleteness did not cause any functional bugs, so this patch could be considered refactoring for the purpose of cleaning up. This patch completes the update of rte_mempool_do_generic_get() as follows: 1. A few comments were malplaced or no longer correct. Some comments have been updated/added/corrected. 2. The code that initially screens the cache request was not updated. The initial screening compared the request length to the cache size, which was correct before, but became irrelevant with the introduction of the flush threshold. E.g. the cache can hold up to flushthresh objects, which is more than its size, so some requests were not served from the cache, even though they could be. The initial screening has now been corrected to match the initial screening in rte_mempool_do_generic_put(), which verifies that a cache is present, and that the length of the request does not overflow the memory allocated for the cache. 3. The code flow for satisfying the request from the cache was weird. The likely code path where the objects are simply served from the cache was treated as unlikely; now it is treated as likely. And in the code path where the cache was backfilled first, numbers were added and subtracted from the cache length; now this code path simply sets the cache length to its final value. 4. The objects were returned in reverse order. Returning the objects in reverse order is not necessary, so rte_memcpy() is now used instead. This patch also updates/adds/corrects some comments in rte_mempool_do_generic_put(). Signed-off-by: Morten Brørup --- lib/mempool/rte_mempool.h | 36 +++++++++++++++++++----------------- 1 file changed, 19 insertions(+), 17 deletions(-) diff --git a/lib/mempool/rte_mempool.h b/lib/mempool/rte_mempool.h index 1e7a3c1527..4c36ad6dd1 100644 --- a/lib/mempool/rte_mempool.h +++ b/lib/mempool/rte_mempool.h @@ -1353,12 +1353,12 @@ rte_mempool_do_generic_put(struct rte_mempool *mp, void * const *obj_table, * cache flush threshold) is flushed to the ring. */ - /* Add elements back into the cache */ + /* Add the objects to the cache */ rte_memcpy(&cache_objs[0], obj_table, sizeof(void *) * n); - cache->len += n; if (cache->len >= cache->flushthresh) { + /* Flush excess objects in the cache to the ring */ rte_mempool_ops_enqueue_bulk(mp, &cache->objs[cache->size], cache->len - cache->size); cache->len = cache->size; @@ -1368,7 +1368,7 @@ rte_mempool_do_generic_put(struct rte_mempool *mp, void * const *obj_table, ring_enqueue: - /* push remaining objects in ring */ + /* Put the objects into the ring */ #ifdef RTE_LIBRTE_MEMPOOL_DEBUG if (rte_mempool_ops_enqueue_bulk(mp, obj_table, n) < 0) rte_panic("cannot put objects in mempool\n"); @@ -1460,21 +1460,25 @@ rte_mempool_do_generic_get(struct rte_mempool *mp, void **obj_table, unsigned int n, struct rte_mempool_cache *cache) { int ret; - uint32_t index, len; void **cache_objs; - /* No cache provided or cannot be satisfied from cache */ - if (unlikely(cache == NULL || n >= cache->size)) + /* No cache provided or if get would overflow mem allocated for cache */ + if (unlikely(cache == NULL || n > RTE_MEMPOOL_CACHE_MAX_SIZE)) goto ring_dequeue; cache_objs = cache->objs; - /* Can this be satisfied from the cache? */ - if (cache->len < n) { - /* No. Backfill the cache first, and then fill from it */ + /* Can the request be satisfied from the cache? */ + if (n <= cache->len) { + /* Yes. Simply decrease the cache length */ + cache->len -= n; + } else { + /* No. Backfill the cache from the ring first */ + + /* Number required to fill the cache + n */ uint32_t req = n + (cache->size - cache->len); - /* How many do we require i.e. number to fill the cache + the request */ + /* Backfill the cache from the ring */ ret = rte_mempool_ops_dequeue_bulk(mp, &cache->objs[cache->len], req); if (unlikely(ret < 0)) { @@ -1487,14 +1491,12 @@ rte_mempool_do_generic_get(struct rte_mempool *mp, void **obj_table, goto ring_dequeue; } - cache->len += req; + /* Set the length of the backfilled cache - n */ + cache->len = cache->size; } - /* Now fill in the response ... */ - for (index = 0, len = cache->len - 1; index < n; ++index, len--, obj_table++) - *obj_table = cache_objs[len]; - - cache->len -= n; + /* Get the objects from the cache, at the already decreased offset */ + rte_memcpy(obj_table, &cache_objs[cache->len], sizeof(void *) * n); RTE_MEMPOOL_STAT_ADD(mp, get_success_bulk, 1); RTE_MEMPOOL_STAT_ADD(mp, get_success_objs, n); @@ -1503,7 +1505,7 @@ rte_mempool_do_generic_get(struct rte_mempool *mp, void **obj_table, ring_dequeue: - /* get remaining objects from ring */ + /* Get the objects from the ring */ ret = rte_mempool_ops_dequeue_bulk(mp, obj_table, n); if (ret < 0) { -- 2.17.1