From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6576D469EB; Wed, 18 Jun 2025 11:26:55 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 522C6427E8; Wed, 18 Jun 2025 11:26:55 +0200 (CEST) Received: from mail-pg1-f177.google.com (mail-pg1-f177.google.com [209.85.215.177]) by mails.dpdk.org (Postfix) with ESMTP id D8E3440BA4 for ; Tue, 17 Jun 2025 16:52:20 +0200 (CEST) Received: by mail-pg1-f177.google.com with SMTP id 41be03b00d2f7-b26df8f44e6so6338527a12.2 for ; Tue, 17 Jun 2025 07:52:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1750171940; x=1750776740; darn=dpdk.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=AtAvO7b+XPavZYXh3oMXxg517f51y8+rYYZmALmj5go=; b=Y39hpzWRZgTWwCx/sn7HdxwWHkN22en7Oil9/5DHKBpyfFTYbawWxHrUMNzOj8J1w+ IsSDv4UJm8KMNPRTMkq94bHrVRgvisYmrSGEdToM+aociXbOBig13H4KFeEAXFEFYNs0 bK/vqdDNh4BjueVQ2J0b6rDIr3hvOicKb3WZauZvuQwslVvmkjuJkBm0IFKYNTS6Iabw 1lCwu2lkKpXODv3FNXoub2jwGEOs+vKmXbT6PniNCA/tWgn4n5rUTrL9y/DREvzS/uHa m/bcLzK9QFDI5DcM7hJX7nZMB28z4q+yVcoLEvhlNe5fbFCPLaoEOog7hf20BgpymyvB QZgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1750171940; x=1750776740; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=AtAvO7b+XPavZYXh3oMXxg517f51y8+rYYZmALmj5go=; b=amdiHAl9YbVAPRvwcQVET58gZXjP5tfspxKB75JtELiVXcGDBu3ZxWClg4E6p390Rq RW98ooKx4UU8WAYMg0Tv8O8JxtCG2cLZTD+sYY3Pi9dTXp4XwH/2eOs+7EyxPapFhd1/ VuTPhYRR0mrB2S+GSWpCnGcdZwV6o1B9L9IAGHec4Hr082SC/BNeG/jonC2rrr6lK3RC 440FksWStnW//pYnMlCgUm3lh3lETc/NXfyi1ZTlQNZOAHHSZKAVT/i37kYdkC83jOul 7Li7MDn4JWRa8InJ816e4QUc46BEUj5f62UFYtJE/G9L3d9ZdE6lkeLtOfIAr0nB6zqh D9UA== X-Gm-Message-State: AOJu0YyBPV2GnNs5T2XMJ69zsYVIXKJ4QIBZy8SCry9EfXw3yd9wZQFi Z3quzRbP3jxaq0p+yl0PonM3f8sZWvIdrCzXK4BRw5iUF+sx8EMqn1FDR4K54Lo0zVX9EIft/7b gi9INsMzfYOjVZpEaKT9EfnWOfVk0ep/D3HYm X-Gm-Gg: ASbGncugVojmiMkDHOF55MDzGCmRBo2Q2Fc5T5idJJr7VPHbnsYrsn7ZAbgN488oDv1 EvFaJXeJXMolU0iY+MMTamOWZPV/AEL/6qClVjfljkNxaT72lcf45YtrBWx8KatvUL2t7pZOU3H LMzTNhPU+6Nh3iNtTj8eEZVXAp+zFU1vF2UX7pbmqeU5Zq9JM9ifXDSg== X-Google-Smtp-Source: AGHT+IGmgZW2vi7d5TqvJCTMKD266SY8j5p71Qhe3p7p05isCo8gn4wGSXtNhT1BzhVuyQy6uTT34LJNMJuv8W23OAQ= X-Received: by 2002:a05:6a21:e8a:b0:1f5:8f7f:8f19 with SMTP id adf61e73a8af0-21fbd569482mr21361514637.10.1750171939826; Tue, 17 Jun 2025 07:52:19 -0700 (PDT) MIME-Version: 1.0 References: <20250608084708.71377b9f@hermes.local> In-Reply-To: From: venkatesh bs Date: Tue, 17 Jun 2025 20:22:03 +0530 X-Gm-Features: AX0GCFukEJBLm3ACtQ6AcZHha5JPyhx-PHkH1b7JqJbDCAlrfoiyviwoPboAqL4 Message-ID: Subject: Re: Question regarding rte_hash_hash and rte_hash_add_key_with_hash_data To: Stephen Hemminger Cc: dev@dpdk.org Content-Type: multipart/alternative; boundary="000000000000c4f25d0637c5a885" X-Mailman-Approved-At: Wed, 18 Jun 2025 11:26:54 +0200 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org --000000000000c4f25d0637c5a885 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi @Stephen Hemminger , We debugged further on this issue .. We see that position and entry_size(position * key_entry_size) crosses above int32_t it overflows and it will corrupt random index of hash table. in Nut shell hash table is only supporting upto 2topower32 size entries on the table. Below mentioned snapshot of the code change. Thanks, Venkatesh. struct rte_hash_key *k, *keys =3D h->key_store; Add comment63Plus - k =3D (struct rte_hash_key *) ((char *) keys + (positi= on + 1) * Add comment64Plus - h->key_entry_size); Add comment65Plus + k =3D (struct rte_hash_key *) ((char *) keys + (uint64_t) (position + 1) * Add comment66Plus + (uint64_t) h->key_entry_size); Add comment67Plus *key =3D k->key; Add comment 68 On Mon, Jun 16, 2025 at 9:40=E2=80=AFPM venkatesh bs = wrote: > Hi @Stephen Hemminger , > > We debugged further on this issue .. > > We see that position and entry_size(position * key_entry_size) crosses > above int32_t it overflows and it will corrupt random index of hash table= . > > in Nut shell hash table is only supporting upto 2topower32 size entries o= n > the table. > > Below mentioned snapshot of the code change. > > Thanks, > Venkatesh. > > > struct rte_hash_key *k, *keys =3D h->key_store; > Add comment63Plus - k =3D (struct rte_hash_key *) ((char *) keys + > (position + 1) * > Add comment64Plus - h->key_entry_size); > Add comment65Plus + k =3D (struct rte_hash_key *) ((char *) keys + > (uint64_t) (position + 1) * > Add comment66Plus + (uint64_t) h->key_entry_size); > Add comment67Plus *key =3D k->key; > Add comment68 > > On Tue, Jun 10, 2025 at 12:19=E2=80=AFPM venkatesh bs wrote: > >> Hi @Stephen Hemminger , >> >> Thanks for the reply , please find the details below. >> >> DPDK version : 20.11.6 >> >> #define LOADBAL_HASH_ENTRIES_MAX (1024*1024*36) >> #define LOADBAL_HASH_TABLE_SIZE_MULTIPLIER 2 >> >> hashSizeMultiplier =3D LOADBAL_HASH_TABLE_SIZE_MULTIPLIER; >> >> struct rte_hash_parameters loadbal_hash_params =3D { >> .name =3D NULL, >> .entries =3D LOADBAL_HASH_ENTRIES_MAX * >> hashSizeMultiplier, >> .key_len =3D sizeof(flow_key_t), >> .hash_func =3D app_hash_crc(internally it calls >> rte_hash_crc_4byte for v4/v6) >> .hash_func_init_val =3D 0, >> }; >> >> First, what is the return value, which error? >> we captured only the return value, and will check what error it is >> returning. >> >> IPv4 Load-Bal Flow hash table: >> numInsertions: 998214247 >> numInsertionsFailures: 4252 >> numRemovals: 997197485 >> numRemovalFailures: 214902 >> numObjects: 1016762 >> NumBuckets: 75497472 >> TableCapacity: 75497472 >> LoadFactor: 1% >> NumLookupSuccess: 389165605 >> NumLookupFails: 2704814006 >> Failure Analysis: >> TableFull(>=3D95%): 0 >> HighLoad(75-95%): 0 >> MediumLoad(50-75%): 0 >> LowLoad(<50%): 4252 >> >> We are try to analyze the code and find out the details, >> Initially there will be no issues, insertion starts failing after 24 HRS >> or so. >> >> Thanks, >> Venkatesh. >> >> >> On Sun, Jun 8, 2025 at 9:17=E2=80=AFPM Stephen Hemminger < >> stephen@networkplumber.org> wrote: >> >>> On Sun, 8 Jun 2025 00:36:02 +0530 >>> venkatesh bs wrote: >>> >>> > Hi @dpdk community, >>> > >>> > In my application, I am using the dpdk hash table as below. >>> > >>> > 1. calculate the hash value. >>> > 2. using this signature add the key and data into a table. >>> > 3. lock is used in the 2nd call (add). >>> > >>> > Below is the code sniffer for the same. >>> > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >>> > hash_sig_t sig =3D rte_hash_hash(hash_tablele, (void *) &new_key= ); >>> > >>> > pthread_mutex_lock(&lock); >>> > int32_t ret =3D rte_hash_add_key_with_hash_data(hash_table, >>> > (void *)&new_key, >>> > sig, >>> > info); >>> > >>> > if (ret < 0) >>> > { >>> > pthread_mutex_unlock(lock); >>> > return TOS_E_FAIL; >>> > } >>> > pthread_mutex_unlock(lock); >>> > >>> > return OK.. >>> > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >>> > >>> > My application is having a lot of threads and when run with heavy loa= d >>> , I >>> > am getting a lot of insertion failure, i felt the reason could be >>> > calculating and adding the has value in 2 api's as opposed to >>> > rte_hash_add() that is safe under a lock.. >>> > >>> > >>> > Please let me know your thoughts. >>> >>> Which architecture and DPDK version? >>> What flags did you use during hash creation? >>> >>> As always with open source, the first thing to do is look at the source >>> and see what is really happening, rather than just relying on the >>> documentation. >>> >>> I assume you are using the default hash function which is CRC. >>> >>> First what is the return value, which error? >>> It might just be key collisions. How big is the table and how many >>> inserts? >>> >>> Also, the current DPDK hash flags with better locking and RCU. >>> This would be faster than simple pthread mutex. >>> >>> --000000000000c4f25d0637c5a885 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
=C2=A0 Hi=C2=A0@Stephen Hemminger=C2=A0,

We debugge= d further on this issue ..

We see that position=C2= =A0and entry_size(position * key_entry_size) crosses above int32_t it overf= lows and it will corrupt random index of hash table.

in Nut shell=C2=A0hash table is only supporting upto 2topower32=C2=A0siz= e entries on the table.=C2=A0

Below mentioned snap= shot of the code change.

Thanks,
Venkate= sh.


struct rte_hash_key *k, *ke= ys =3D h->key_store;
Add comment63Plus=C2=A0=C2=A0= - k =3D (struct rte_hash_key *) ((cha= r *) keys + (position + 1) *
Add comment64Plus=C2=A0= =C2=A0- h->key_entry_size);=
Add comment<= /span>65Plus=C2=A0=C2=A0= + k =3D (struct rte_hash_key *) ((char *) keys + (uint64_t) (= position + 1) *
Add comment66<= /span>Plus=C2=A0=C2=A0<= span aria-hidden=3D"true" style=3D"box-sizing:inherit">+ (uint64_t) h->key_entry_size);
Add comment67Plus=C2=A0=C2=A0= *key =3D k->key;
Add comment 68
=

On Mon, Jun 16, 2025 at 9:40= =E2=80=AFPM venkatesh bs <venki.b= sv@gmail.com> wrote:
Hi=C2=A0@Stephen Hemminger=C2=A0,

We de= bugged further on this issue ..

We see that positi= on=C2=A0and entry_size(position * key_entry_size) crosses above int32_t it = overflows and it will corrupt random index of hash table.

in Nut shell=C2=A0hash table is only supporting upto 2topower32=C2= =A0size entries on the table.=C2=A0

Below mentione= d snapshot of the code change.

Thanks,
V= enkatesh.


= struct rte_hash_key *k, *keys =3D h->key_store;
Add comment63Plus=C2=A0=C2=A0= - k =3D (struct rte_hash_key *) ((char *) keys + (position + = 1) *
Add comment64Plus=C2=A0=C2= =A0- h->key_entry_size);
Add comment65Plus=C2=A0=C2=A0+ k =3D (struct rte_hash_key *) ((char *) ke= ys + (uint64_t) (position + 1) *
Add comment66Plus=C2=A0=C2=A0+ = (uint64_t) h->key_entry_size);
Add comment67= Plus=C2=A0=C2=A0 *key= =3D k->key;
Add comment68

On Tue, Jun 10, 2025 at 12:19=E2=80=AFPM venkatesh bs &l= t;venki.bsv@gmail.= com> wrote:
Hi=C2=A0@Stephen Hemminger= =C2=A0,

Thanks for the reply , please find the detai= ls below.

DPDK version : 20.11.6

#de= fine LOADBAL_HASH_ENTRIES_MAX (1024*1024*36)
#define LOADBAL_HASH_TABLE_= SIZE_MULTIPLIER 2

hashSizeMultiplier =3D LOADBAL_HASH_TABLE_SIZE_MUL= TIPLIER;

=C2=A0 struct rte_hash_parameters loadbal_hash_params =3D {=
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 .name = =3D NULL,
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= .entries =3D LOADBAL_HASH_ENTRIES_MAX * hashSizeMultiplier,
=C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 .key_len =3D sizeof(fl= ow_key_t),
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 .hash_func =3D app_hash_crc(internally it calls rte_hash_crc_4byte for = v4/v6)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 .h= ash_func_init_val =3D 0,
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 };

First, what is the return value, which error?
we captured= only the return value, and will check what =C2=A0error it is returning.
IPv4 Load-Bal Flow hash table:
=C2=A0 =C2=A0 numInsertions: =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0998214247
=C2=A0 =C2=A0 numInsertionsFailures= : =C2=A04252
=C2=A0 =C2=A0 numRemovals: =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0997197485
=C2=A0 =C2=A0 numRemovalFailures: =C2=A0 =C2=A0 2149= 02
=C2=A0 =C2=A0 numObjects: =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 1= 016762
=C2=A0 =C2=A0 NumBuckets: =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 75497472
=C2=A0 =C2=A0 TableCapacity: =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A075497472
=C2=A0 =C2=A0 LoadFactor: =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 1%
=C2=A0 =C2=A0 NumLookupSuccess: =C2=A0 =C2=A0 =C2=A0 38916= 5605
=C2=A0 =C2=A0 NumLookupFails: =C2=A0 =C2=A0 =C2=A0 =C2=A0 270481400= 6
=C2=A0 =C2=A0 Failure Analysis:
=C2=A0 =C2=A0 =C2=A0 TableFull(>= =3D95%): =C2=A0 =C2=A00
=C2=A0 =C2=A0 =C2=A0 HighLoad(75-95%): =C2=A0 = =C2=A00
=C2=A0 =C2=A0 =C2=A0 MediumLoad(50-75%): =C2=A00
=C2=A0 =C2= =A0 =C2=A0 LowLoad(<50%): =C2=A0 =C2=A0 =C2=A0 4252
=C2=A0 =C2=A0
We are try to analyze the code and find out=C2=A0the details,
I= nitially there will be no issues, insertion starts failing after 24 HRS or = so.

Thanks,
Venkatesh.


On S= un, Jun 8, 2025 at 9:17=E2=80=AFPM Stephen Hemminger <stephen@networkplumber.org> wrote:
On = Sun, 8 Jun 2025 00:36:02 +0530
venkatesh bs <v= enki.bsv@gmail.com> wrote:

> Hi @dpdk community,
>
> In my application, I am using the dpdk hash table as below.
>
> 1. calculate the hash value.
> 2. using this signature add the key and data into=C2=A0 a table.
> 3. lock is used in the 2nd call (add).
>
> Below is the code sniffer for the same.
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>=C2=A0 =C2=A0 =C2=A0 hash_sig_t sig =3D rte_hash_hash(hash_tablele, (vo= id *) &new_key);
>
>=C2=A0 =C2=A0 =C2=A0 pthread_mutex_lock(&lock);
>=C2=A0 =C2=A0 =C2=A0 =C2=A0int32_t ret =3D rte_hash_add_key_with_hash_d= ata(hash_table,
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0(void *)&new_key,
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 sig,
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 info);
>
> if (ret < 0)
> {
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 pthread_mutex_unlock(lock);
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 return TOS_E_FAIL;
> }
> pthread_mutex_unlock(lock);
>
> return OK..
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>
> My application is having a lot of threads and when run with heavy load= , I
> am getting a lot of insertion failure, i felt the reason could be
> calculating and adding the has value in 2 api's as opposed to
> rte_hash_add() that is safe under a lock..
>
>
> Please let me know your thoughts.

Which architecture and DPDK version?
What flags did you use during hash creation?

As always with open source, the first thing to do is look at the source
and see what is really happening, rather than just relying on the documenta= tion.

I assume you are using the default hash function which is CRC.

First what is the return value, which error?
It might just be key collisions. How big is the table and how many inserts?=

Also, the current DPDK hash flags with better locking and RCU.
This would be faster than simple pthread mutex.

--000000000000c4f25d0637c5a885--