From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D90EA43EEA; Tue, 23 Apr 2024 15:50:08 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 6D97E42EEA; Tue, 23 Apr 2024 15:50:08 +0200 (CEST) Received: from mail-ot1-f43.google.com (mail-ot1-f43.google.com [209.85.210.43]) by mails.dpdk.org (Postfix) with ESMTP id B6436402C7 for ; Tue, 23 Apr 2024 15:50:03 +0200 (CEST) Received: by mail-ot1-f43.google.com with SMTP id 46e09a7af769-6ed06982819so1790006a34.0 for ; Tue, 23 Apr 2024 06:50:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1713880203; x=1714485003; darn=dpdk.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=JCGgAgMwgrPZqTefK5bfHA0Kve9Xy81NofuymPaXyRI=; b=Of/VmaXNIAqkGFCfVA+smiGkOPjyT2mYnyX15Umq3u6J1Ds7ttLZxB49h+4E7fk+UR owB3DjNtMpG3Ek7n/vCxNBPWABgLwcr/BwpoPpToc03UJhmMmZmSUyM+AiE69+RGRZvl WtGd9Y1Yerid4XKwdUUWeBDpI4NbMoQDBbTQ9rmZMvYmLHiKhjWF7QZKlywtc8b2RXlp dJY3I9JZT30U+FWAwEeUOO9+Q8e1kGd8jHltRUmzglF6i69HeK1SmEKSaHA3kkxQ8G15 Kkl8p8hS4jMd7nd3JnJcGqBWyEA95vwEttLJtxfEnMZCR5+YixqS84V1hjxurw1duALB EUuw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713880203; x=1714485003; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=JCGgAgMwgrPZqTefK5bfHA0Kve9Xy81NofuymPaXyRI=; b=nul1l9t3o/AvBNtNLlYbq8p0rh3ebudV2KrjisOwOY2whr7JLB2LEmYMsVBT+Spf9w XGHS1iYuvqAl9BnFC8608VxgLNFWPNictu6VsKdn/Vy7BzBHVprGR7lHtZni25nkcQtP THDbXh9+YKznPx/ut+1fj1U1u3UKd8zSf69iYEIAnOkpbh/UFswvL3RWD0K82EygAcuX vMllh3NAUp1JEGgSDNuiUpWl5fw0OpDF9FTJBKU5aH4yBr4bXtKDH22J05PVoxYNXibn oiWHqtmLY8mdXgf0kJTUW/pc/rynmo7+Yq+qJ0N09wm8zF53KNMnXt81iqFvT4ezhsXG FugQ== X-Gm-Message-State: AOJu0Yxa27HM56udxZP/TtucX2zowLmHUI7p/yN2x0bWYSTbOVHnPLEo XgRce5Ici7YFtSCuVTxCepdnjBBwNfbuX1b6wif/M47IRU56s/Gy1sSNoCHhxITztV1qPQQt9t3 u8XPAWmXQlVJ9N6jBKlAjn7HkVwWN2Meu X-Google-Smtp-Source: AGHT+IHCI1qmukBPvbWEqzK7zvDuZqndcvWaMavNuMXVHwHU5WYqxmWRQm3pXEpzFjsR4wYI1KVSLpbmxF8c4ruvn+o= X-Received: by 2002:a05:6871:58a7:b0:229:ce58:477a with SMTP id ok39-20020a05687158a700b00229ce58477amr16271415oac.19.1713880202787; Tue, 23 Apr 2024 06:50:02 -0700 (PDT) MIME-Version: 1.0 References: <20240415112602.690972-1-aomeryamac@gmail.com> In-Reply-To: <20240415112602.690972-1-aomeryamac@gmail.com> From: =?UTF-8?B?QWJkdWxsYWggw5ZtZXIgWWFtYcOn?= Date: Tue, 23 Apr 2024 16:49:51 +0300 Message-ID: Subject: Re: [PATCH v4] lib/hash: add defer queue reclaim API To: dev@dpdk.org Cc: Honnappa Nagarahalli , Thomas Monjalon Content-Type: multipart/alternative; boundary="000000000000acc5b90616c3d4b9" X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org --000000000000acc5b90616c3d4b9 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hello, is there any other comment on this patch? Thanks On Mon, Apr 15, 2024 at 2:26=E2=80=AFPM Abdullah =C3=96mer Yama=C3=A7 wrote: > This patch adds a new feature to the hash library to allow the user to > reclaim the defer queue. This is useful when the user wants to force > reclaim resources that are not being used. This API is only available > if the RCU is enabled. > > Signed-off-by: Abdullah =C3=96mer Yama=C3=A7 > --- > app/test/test_hash.c | 97 ++++++++++++++++++++++++++++++++++++++ > lib/hash/rte_cuckoo_hash.c | 23 +++++++++ > lib/hash/rte_hash.h | 24 ++++++++++ > lib/hash/version.map | 7 +++ > 4 files changed, 151 insertions(+) > > diff --git a/app/test/test_hash.c b/app/test/test_hash.c > index d586878a22..ebeda8c322 100644 > --- a/app/test/test_hash.c > +++ b/app/test/test_hash.c > @@ -2183,6 +2183,100 @@ test_hash_rcu_qsbr_sync_mode(uint8_t ext_bkt) > > } > > +/* > + * rte_hash_rcu_qsbr_dq_reclaim unit test. > + */ > +static int > +test_hash_rcu_qsbr_dq_reclaim(void) > +{ > + size_t sz; > + int32_t status; > + unsigned int total_entries =3D 8; > + unsigned int freed, pending, available; > + uint32_t reclaim_keys[8] =3D {10, 11, 12, 13, 14, 15, 16, 17}; > + struct rte_hash_rcu_config rcu_cfg =3D {0}; > + struct rte_hash_parameters hash_params =3D { > + .name =3D "test_hash_rcu_qsbr_dq_reclaim", > + .entries =3D total_entries, > + .key_len =3D sizeof(uint32_t), > + .hash_func =3D NULL, > + .hash_func_init_val =3D 0, > + .socket_id =3D 0, > + }; > + > + hash_params.extra_flag =3D RTE_HASH_EXTRA_FLAGS_RW_CONCURRENCY_LF= ; > + > + g_qsv =3D NULL; > + g_handle =3D NULL; > + > + printf("\n# Running RCU QSBR DQ mode, reclaim defer queue > functional test\n"); > + > + g_handle =3D rte_hash_create(&hash_params); > + RETURN_IF_ERROR_RCU_QSBR(g_handle =3D=3D NULL, "Hash creation fai= led"); > + > + /* Create RCU QSBR variable */ > + sz =3D rte_rcu_qsbr_get_memsize(RTE_MAX_LCORE); > + g_qsv =3D (struct rte_rcu_qsbr *)rte_zmalloc_socket(NULL, sz, > + RTE_CACHE_LINE_SIZE, > SOCKET_ID_ANY); > + RETURN_IF_ERROR_RCU_QSBR(g_qsv =3D=3D NULL, > + "RCU QSBR > variable creation failed"); > + > + status =3D rte_rcu_qsbr_init(g_qsv, RTE_MAX_LCORE); > + RETURN_IF_ERROR_RCU_QSBR(status !=3D 0, > + "RCU QSBR > variable initialization failed"); > + > + rcu_cfg.v =3D g_qsv; > + rcu_cfg.dq_size =3D total_entries; > + rcu_cfg.mode =3D RTE_HASH_QSBR_MODE_DQ; > + > + /* Attach RCU QSBR to hash table */ > + status =3D rte_hash_rcu_qsbr_add(g_handle, &rcu_cfg); > + RETURN_IF_ERROR_RCU_QSBR(status !=3D 0, > + "Attach RCU QSBR > to hash table failed"); > + > + /* Register pseudo reader */ > + status =3D rte_rcu_qsbr_thread_register(g_qsv, 0); > + RETURN_IF_ERROR_RCU_QSBR(status !=3D 0, > + "RCU QSBR thread > registration failed"); > + rte_rcu_qsbr_thread_online(g_qsv, 0); > + > + /* Fill half of the hash table */ > + for (size_t i =3D 0; i < total_entries / 2; i++) > + status =3D rte_hash_add_key(g_handle, &reclaim_keys[i]); > + > + /* Lookup inserted elements*/ > + for (size_t i =3D 0; i < total_entries / 2; i++) > + rte_hash_lookup(g_handle, &reclaim_keys[i]); > + > + /* Try to put these elements into the defer queue*/ > + for (size_t i =3D 0; i < total_entries / 2; i++) > + rte_hash_del_key(g_handle, &reclaim_keys[i]); > + > + /* Reader quiescent */ > + rte_rcu_qsbr_quiescent(g_qsv, 0); > + > + status =3D rte_hash_add_key(g_handle, &reclaim_keys[0]); > + RETURN_IF_ERROR_RCU_QSBR(status < 0, > + "failed to add > key (pos[%u]=3D%d)", 0, > + status); > + > + /* This should be (total_entries / 2) + 1 (last add) */ > + unsigned int hash_size =3D rte_hash_count(g_handle); > + > + /* Freed size should be (total_entries / 2) */ > + rte_hash_rcu_qsbr_dq_reclaim(g_handle, &freed, &pending, > &available); > + > + rte_hash_free(g_handle); > + rte_free(g_qsv); > + > + if (hash_size !=3D (total_entries / 2 + 1) || freed !=3D > (total_entries / 2)) { > + printf("Failed to reclaim defer queue\n"); > + return -1; > + } > + > + return 0; > +} > + > /* > * Do all unit and performance tests. > */ > @@ -2261,6 +2355,9 @@ test_hash(void) > if (test_hash_rcu_qsbr_sync_mode(1) < 0) > return -1; > > + if (test_hash_rcu_qsbr_dq_reclaim() < 0) > + return -1; > + > return 0; > } > > diff --git a/lib/hash/rte_cuckoo_hash.c b/lib/hash/rte_cuckoo_hash.c > index 9cf94645f6..4a44aadd9a 100644 > --- a/lib/hash/rte_cuckoo_hash.c > +++ b/lib/hash/rte_cuckoo_hash.c > @@ -1588,6 +1588,29 @@ rte_hash_rcu_qsbr_add(struct rte_hash *h, struct > rte_hash_rcu_config *cfg) > return 0; > } > > +int > +rte_hash_rcu_qsbr_dq_reclaim(struct rte_hash *h, unsigned int *freed, > + unsigned int *pending, unsigned > int *available) > +{ > + int ret; > + > + if (h =3D=3D NULL || h->hash_rcu_cfg =3D=3D NULL) { > + rte_errno =3D EINVAL; > + return 1; > + } > + > + ret =3D rte_rcu_qsbr_dq_reclaim(h->dq, > h->hash_rcu_cfg->max_reclaim_size, > + freed, > pending, available); > + if (ret !=3D 0) { > + HASH_LOG(ERR, > + "%s: could not reclaim the defer queue i= n > hash table", > + __func__); > + return 1; > + } > + > + return 0; > +} > + > static inline void > remove_entry(const struct rte_hash *h, struct rte_hash_bucket *bkt, > unsigned int i) > diff --git a/lib/hash/rte_hash.h b/lib/hash/rte_hash.h > index 7ecc021111..edfa262aca 100644 > --- a/lib/hash/rte_hash.h > +++ b/lib/hash/rte_hash.h > @@ -674,6 +674,30 @@ rte_hash_iterate(const struct rte_hash *h, const voi= d > **key, void **data, uint32 > */ > int rte_hash_rcu_qsbr_add(struct rte_hash *h, struct rte_hash_rcu_config > *cfg); > > +/** > + * Reclaim resources from the defer queue. > + * This API reclaim the resources from the defer queue if rcu is enabled= . > + * > + * @param h > + * The hash object to reclaim resources. > + * @param freed > + * Number of resources that were freed. > + * @param pending > + * Number of resources pending on the defer queue. > + * This number might not be accurate if multi-thread safety is > configured. > + * @param available > + * Number of resources that can be added to the defer queue. > + * This number might not be accurate if multi-thread safety is > configured. > + * @return > + * On success - 0 > + * On error - 1 with error code set in rte_errno. > + * Possible rte_errno codes are: > + * - EINVAL - invalid pointer > + */ > +__rte_experimental > +int rte_hash_rcu_qsbr_dq_reclaim(struct rte_hash *h, unsigned int *freed= , > + unsigned int *pending, unsigned int *available); > + > #ifdef __cplusplus > } > #endif > diff --git a/lib/hash/version.map b/lib/hash/version.map > index 6f4bcdb71b..d348dd9196 100644 > --- a/lib/hash/version.map > +++ b/lib/hash/version.map > @@ -53,3 +53,10 @@ INTERNAL { > rte_thash_gfni_stub; > rte_thash_gfni_bulk_stub; > }; > + > +EXPERIMENTAL { > + global: > + > + # added in 24.07 > + rte_hash_rcu_qsbr_dq_reclaim; > +}; > -- > 2.34.1 > > --000000000000acc5b90616c3d4b9 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hello, is there any other comment on this patch? Thanks=C2= =A0

On Mon, Apr 15, 2024 at 2:26=E2=80=AFPM Abdullah =C3=96mer Yama=C3=A7 &l= t;aomeryamac@gmail.com> wrot= e:
This patch ad= ds a new feature to the hash library to allow the user to
reclaim the defer queue. This is useful when the user wants to force
reclaim resources that are not being used. This API is only available
if the RCU is enabled.

Signed-off-by: Abdullah =C3=96mer Yama=C3=A7 <aomeryamac@gmail.com>
---
=C2=A0app/test/test_hash.c=C2=A0 =C2=A0 =C2=A0 =C2=A0| 97 +++++++++++++++++= +++++++++++++++++++++
=C2=A0lib/hash/rte_cuckoo_hash.c | 23 +++++++++
=C2=A0lib/hash/rte_hash.h=C2=A0 =C2=A0 =C2=A0 =C2=A0 | 24 ++++++++++
=C2=A0lib/hash/version.map=C2=A0 =C2=A0 =C2=A0 =C2=A0|=C2=A0 7 +++
=C2=A04 files changed, 151 insertions(+)

diff --git a/app/test/test_hash.c b/app/test/test_hash.c
index d586878a22..ebeda8c322 100644
--- a/app/test/test_hash.c
+++ b/app/test/test_hash.c
@@ -2183,6 +2183,100 @@ test_hash_rcu_qsbr_sync_mode(uint8_t ext_bkt)

=C2=A0}

+/*
+ * rte_hash_rcu_qsbr_dq_reclaim unit test.
+ */
+static int
+test_hash_rcu_qsbr_dq_reclaim(void)
+{
+=C2=A0 =C2=A0 =C2=A0 =C2=A0size_t sz;
+=C2=A0 =C2=A0 =C2=A0 =C2=A0int32_t status;
+=C2=A0 =C2=A0 =C2=A0 =C2=A0unsigned int total_entries =3D 8;
+=C2=A0 =C2=A0 =C2=A0 =C2=A0unsigned int freed, pending, available;
+=C2=A0 =C2=A0 =C2=A0 =C2=A0uint32_t reclaim_keys[8] =3D {10, 11, 12, 13, 1= 4, 15, 16, 17};
+=C2=A0 =C2=A0 =C2=A0 =C2=A0struct rte_hash_rcu_config rcu_cfg =3D {0};
+=C2=A0 =C2=A0 =C2=A0 =C2=A0struct rte_hash_parameters hash_params =3D { +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0.name =3D "tes= t_hash_rcu_qsbr_dq_reclaim",
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0.entries =3D total_= entries,
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0.key_len =3D sizeof= (uint32_t),
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0.hash_func =3D NULL= ,
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0.hash_func_init_val= =3D 0,
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0.socket_id =3D 0, +=C2=A0 =C2=A0 =C2=A0 =C2=A0};
+
+=C2=A0 =C2=A0 =C2=A0 =C2=A0hash_params.extra_flag =3D RTE_HASH_EXTRA_FLAGS= _RW_CONCURRENCY_LF;
+
+=C2=A0 =C2=A0 =C2=A0 =C2=A0g_qsv =3D NULL;
+=C2=A0 =C2=A0 =C2=A0 =C2=A0g_handle =3D NULL;
+
+=C2=A0 =C2=A0 =C2=A0 =C2=A0printf("\n# Running RCU QSBR DQ mode, recl= aim defer queue functional test\n");
+
+=C2=A0 =C2=A0 =C2=A0 =C2=A0g_handle =3D rte_hash_create(&hash_params);=
+=C2=A0 =C2=A0 =C2=A0 =C2=A0RETURN_IF_ERROR_RCU_QSBR(g_handle =3D=3D NULL, = "Hash creation failed");
+
+=C2=A0 =C2=A0 =C2=A0 =C2=A0/* Create RCU QSBR variable */
+=C2=A0 =C2=A0 =C2=A0 =C2=A0sz =3D rte_rcu_qsbr_get_memsize(RTE_MAX_LCORE);=
+=C2=A0 =C2=A0 =C2=A0 =C2=A0g_qsv =3D (struct rte_rcu_qsbr *)rte_zmalloc_so= cket(NULL, sz,
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0RTE_CACHE= _LINE_SIZE, SOCKET_ID_ANY);
+=C2=A0 =C2=A0 =C2=A0 =C2=A0RETURN_IF_ERROR_RCU_QSBR(g_qsv =3D=3D NULL,
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 "RCU QSBR variable cr= eation failed");
+
+=C2=A0 =C2=A0 =C2=A0 =C2=A0status =3D rte_rcu_qsbr_init(g_qsv, RTE_MAX_LCO= RE);
+=C2=A0 =C2=A0 =C2=A0 =C2=A0RETURN_IF_ERROR_RCU_QSBR(status !=3D 0,
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 "RCU QSBR variable in= itialization failed");
+
+=C2=A0 =C2=A0 =C2=A0 =C2=A0rcu_cfg.v =3D g_qsv;
+=C2=A0 =C2=A0 =C2=A0 =C2=A0rcu_cfg.dq_size =3D total_entries;
+=C2=A0 =C2=A0 =C2=A0 =C2=A0rcu_cfg.mode =3D RTE_HASH_QSBR_MODE_DQ;
+
+=C2=A0 =C2=A0 =C2=A0 =C2=A0/* Attach RCU QSBR to hash table */
+=C2=A0 =C2=A0 =C2=A0 =C2=A0status =3D rte_hash_rcu_qsbr_add(g_handle, &= ;rcu_cfg);
+=C2=A0 =C2=A0 =C2=A0 =C2=A0RETURN_IF_ERROR_RCU_QSBR(status !=3D 0,
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 "Attach RCU QSBR to h= ash table failed");
+
+=C2=A0 =C2=A0 =C2=A0 =C2=A0/* Register pseudo reader */
+=C2=A0 =C2=A0 =C2=A0 =C2=A0status =3D rte_rcu_qsbr_thread_register(g_qsv, = 0);
+=C2=A0 =C2=A0 =C2=A0 =C2=A0RETURN_IF_ERROR_RCU_QSBR(status !=3D 0,
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 "RCU QSBR thread regi= stration failed");
+=C2=A0 =C2=A0 =C2=A0 =C2=A0rte_rcu_qsbr_thread_online(g_qsv, 0);
+
+=C2=A0 =C2=A0 =C2=A0 =C2=A0/* Fill half of the hash table */
+=C2=A0 =C2=A0 =C2=A0 =C2=A0for (size_t i =3D 0; i < total_entries / 2; = i++)
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0status =3D rte_hash= _add_key(g_handle, &reclaim_keys[i]);
+
+=C2=A0 =C2=A0 =C2=A0 =C2=A0/* Lookup inserted elements*/
+=C2=A0 =C2=A0 =C2=A0 =C2=A0for (size_t i =3D 0; i < total_entries / 2; = i++)
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0rte_hash_lookup(g_h= andle, &reclaim_keys[i]);
+
+=C2=A0 =C2=A0 =C2=A0 =C2=A0/* Try to put these elements into the defer que= ue*/
+=C2=A0 =C2=A0 =C2=A0 =C2=A0for (size_t i =3D 0; i < total_entries / 2; = i++)
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0rte_hash_del_key(g_= handle, &reclaim_keys[i]);
+
+=C2=A0 =C2=A0 =C2=A0 =C2=A0/* Reader quiescent */
+=C2=A0 =C2=A0 =C2=A0 =C2=A0rte_rcu_qsbr_quiescent(g_qsv, 0);
+
+=C2=A0 =C2=A0 =C2=A0 =C2=A0status =3D rte_hash_add_key(g_handle, &recl= aim_keys[0]);
+=C2=A0 =C2=A0 =C2=A0 =C2=A0RETURN_IF_ERROR_RCU_QSBR(status < 0,
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 "failed to add key (p= os[%u]=3D%d)", 0,
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 status);
+
+=C2=A0 =C2=A0 =C2=A0 =C2=A0/* This should be (total_entries / 2) + 1 (last= add) */
+=C2=A0 =C2=A0 =C2=A0 =C2=A0unsigned int hash_size =3D rte_hash_count(g_han= dle);
+
+=C2=A0 =C2=A0 =C2=A0 =C2=A0/* Freed size should be (total_entries / 2) */<= br> +=C2=A0 =C2=A0 =C2=A0 =C2=A0rte_hash_rcu_qsbr_dq_reclaim(g_handle, &fre= ed, &pending, &available);
+
+=C2=A0 =C2=A0 =C2=A0 =C2=A0rte_hash_free(g_handle);
+=C2=A0 =C2=A0 =C2=A0 =C2=A0rte_free(g_qsv);
+
+=C2=A0 =C2=A0 =C2=A0 =C2=A0if (hash_size !=3D (total_entries / 2 + 1) || f= reed !=3D (total_entries / 2)) {
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0printf("Failed= to reclaim defer queue\n");
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return -1;
+=C2=A0 =C2=A0 =C2=A0 =C2=A0}
+
+=C2=A0 =C2=A0 =C2=A0 =C2=A0return 0;
+}
+
=C2=A0/*
=C2=A0 * Do all unit and performance tests.
=C2=A0 */
@@ -2261,6 +2355,9 @@ test_hash(void)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 if (test_hash_rcu_qsbr_sync_mode(1) < 0)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 return -1;

+=C2=A0 =C2=A0 =C2=A0 =C2=A0if (test_hash_rcu_qsbr_dq_reclaim() < 0)
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return -1;
+
=C2=A0 =C2=A0 =C2=A0 =C2=A0 return 0;
=C2=A0}

diff --git a/lib/hash/rte_cuckoo_hash.c b/lib/hash/rte_cuckoo_hash.c
index 9cf94645f6..4a44aadd9a 100644
--- a/lib/hash/rte_cuckoo_hash.c
+++ b/lib/hash/rte_cuckoo_hash.c
@@ -1588,6 +1588,29 @@ rte_hash_rcu_qsbr_add(struct rte_hash *h, struct rte= _hash_rcu_config *cfg)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 return 0;
=C2=A0}

+int
+rte_hash_rcu_qsbr_dq_reclaim(struct rte_hash *h, unsigned int *freed,
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0unsigned = int *pending, unsigned int *available)
+{
+=C2=A0 =C2=A0 =C2=A0 =C2=A0int ret;
+
+=C2=A0 =C2=A0 =C2=A0 =C2=A0if (h =3D=3D NULL || h->hash_rcu_cfg =3D=3D = NULL) {
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0rte_errno =3D EINVA= L;
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return 1;
+=C2=A0 =C2=A0 =C2=A0 =C2=A0}
+
+=C2=A0 =C2=A0 =C2=A0 =C2=A0ret =3D rte_rcu_qsbr_dq_reclaim(h->dq, h->= ;hash_rcu_cfg->max_reclaim_size,
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0freed, pending, available);
+=C2=A0 =C2=A0 =C2=A0 =C2=A0if (ret !=3D 0) {
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0HASH_LOG(ERR,
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 "%s: could not reclaim the defe= r queue in hash table",
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 __func__);
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return 1;
+=C2=A0 =C2=A0 =C2=A0 =C2=A0}
+
+=C2=A0 =C2=A0 =C2=A0 =C2=A0return 0;
+}
+
=C2=A0static inline void
=C2=A0remove_entry(const struct rte_hash *h, struct rte_hash_bucket *bkt, =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 unsigned int i)
diff --git a/lib/hash/rte_hash.h b/lib/hash/rte_hash.h
index 7ecc021111..edfa262aca 100644
--- a/lib/hash/rte_hash.h
+++ b/lib/hash/rte_hash.h
@@ -674,6 +674,30 @@ rte_hash_iterate(const struct rte_hash *h, const void = **key, void **data, uint32
=C2=A0 */
=C2=A0int rte_hash_rcu_qsbr_add(struct rte_hash *h, struct rte_hash_rcu_con= fig *cfg);

+/**
+ * Reclaim resources from the defer queue.
+ * This API reclaim the resources from the defer queue if rcu is enabled.<= br> + *
+ * @param h
+ *=C2=A0 =C2=A0The hash object to reclaim resources.
+ * @param freed
+ *=C2=A0 =C2=A0Number of resources that were freed.
+ * @param pending
+ *=C2=A0 =C2=A0Number of resources pending on the defer queue.
+ *=C2=A0 =C2=A0This number might not be accurate if multi-thread safety is= configured.
+ * @param available
+ *=C2=A0 =C2=A0Number of resources that can be added to the defer queue. + *=C2=A0 =C2=A0This number might not be accurate if multi-thread safety is= configured.
+ * @return
+ *=C2=A0 =C2=A0On success - 0
+ *=C2=A0 =C2=A0On error - 1 with error code set in rte_errno.
+ *=C2=A0 =C2=A0Possible rte_errno codes are:
+ *=C2=A0 =C2=A0- EINVAL - invalid pointer
+ */
+__rte_experimental
+int rte_hash_rcu_qsbr_dq_reclaim(struct rte_hash *h, unsigned int *freed,<= br> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0unsigned int *pendi= ng, unsigned int *available);
+
=C2=A0#ifdef __cplusplus
=C2=A0}
=C2=A0#endif
diff --git a/lib/hash/version.map b/lib/hash/version.map
index 6f4bcdb71b..d348dd9196 100644
--- a/lib/hash/version.map
+++ b/lib/hash/version.map
@@ -53,3 +53,10 @@ INTERNAL {
=C2=A0 =C2=A0 =C2=A0 =C2=A0 rte_thash_gfni_stub;
=C2=A0 =C2=A0 =C2=A0 =C2=A0 rte_thash_gfni_bulk_stub;
=C2=A0};
+
+EXPERIMENTAL {
+=C2=A0 =C2=A0 =C2=A0 =C2=A0global:
+
+=C2=A0 =C2=A0 =C2=A0 =C2=A0# added in 24.07
+=C2=A0 =C2=A0 =C2=A0 =C2=A0rte_hash_rcu_qsbr_dq_reclaim;
+};
--
2.34.1

--000000000000acc5b90616c3d4b9--