Bihash Support in DPDK

Bihash Support in DPDK

[rajesh goel]

[-- Attachment #1: Type: text/plain, Size: 168 bytes --]

Hi All,
Need info if DPDK hash library supports bihash table where for multi-thread
and multi-process we can update/del/lookup entries per bucket level.

Thanks
Rajesh

[-- Attachment #2: Type: text/html, Size: 233 bytes --]

Re: Bihash Support in DPDK

[Ferruh Yigit]

On 8/22/2024 8:51 AM, rajesh goel wrote:
> Hi All,
> Need info if DPDK hash library supports bihash table where for multi-
> thread and multi-process we can update/del/lookup entries per bucket level.
> 
> 

+ hash library maintainers.

Re: Bihash Support in DPDK

[rajesh goel]

[-- Attachment #1: Type: text/plain, Size: 396 bytes --]

Hi All,
Can we get some reply.

Thanks
Rajesh

On Thu, Aug 22, 2024 at 9:32 PM Ferruh Yigit <ferruh.yigit@amd.com> wrote:

> On 8/22/2024 8:51 AM, rajesh goel wrote:
> > Hi All,
> > Need info if DPDK hash library supports bihash table where for multi-
> > thread and multi-process we can update/del/lookup entries per bucket
> level.
> >
> >
>
> + hash library maintainers.
>

[-- Attachment #2: Type: text/html, Size: 740 bytes --]

RE: Bihash Support in DPDK

[Medvedkin, Vladimir]

[-- Attachment #1: Type: text/plain, Size: 1068 bytes --]

Hi Rajesh,

Please clarify what do you mean by “bihash”? Bidirectional? Bounded index?

As for concurrent lookup/updates, yes, DPDK hash table supports multi-process/multi-thread, please see the documentation:
https://doc.dpdk.org/guides/prog_guide/hash_lib.html#multi-process-support


From: rajesh goel <rgoel.bangalore@gmail.com>
Sent: Tuesday, August 27, 2024 7:04 AM
To: Ferruh Yigit <ferruh.yigit@amd.com>
Cc: Wang, Yipeng1 <yipeng1.wang@intel.com>; Gobriel, Sameh <sameh.gobriel@intel.com>; Richardson, Bruce <bruce.richardson@intel.com>; Medvedkin, Vladimir <vladimir.medvedkin@intel.com>; dev@dpdk.org
Subject: Re: Bihash Support in DPDK

Hi All,
Can we get some reply.

Thanks
Rajesh

On Thu, Aug 22, 2024 at 9:32 PM Ferruh Yigit <ferruh.yigit@amd.com<mailto:ferruh.yigit@amd.com>> wrote:
On 8/22/2024 8:51 AM, rajesh goel wrote:
> Hi All,
> Need info if DPDK hash library supports bihash table where for multi-
> thread and multi-process we can update/del/lookup entries per bucket level.
>
>

+ hash library maintainers.

[-- Attachment #2: Type: text/html, Size: 4715 bytes --]

Re: Bihash Support in DPDK

[rajesh goel]

[-- Attachment #1: Type: text/plain, Size: 1756 bytes --]

Thanks for the reply.

Bihash I mean bounded index what Vpp supports.

Iam looking for the bucket level lock support. Currently Iam using hash
table shared by multiple process or multiple core/threads. So I have to
take the write lock by single core and then read lock by multiple cores to
read the value wrote in this hash table. Multiple readers are getting
blocked due to this. I want to avoid this to increase performance.

Let me know your thoughts on this.

Regards
Rajesh

On Tue, 27 Aug, 2024, 14:44 Medvedkin, Vladimir, <
vladimir.medvedkin@intel.com> wrote:

> Hi Rajesh,
>
>
>
> Please clarify what do you mean by “bihash”? Bidirectional? Bounded index?
>
>
>
> As for concurrent lookup/updates, yes, DPDK hash table supports
> multi-process/multi-thread, please see the documentation:
>
> https://doc.dpdk.org/guides/prog_guide/hash_lib.html#multi-process-support
>
>
>
>
>
> *From:* rajesh goel <rgoel.bangalore@gmail.com>
> *Sent:* Tuesday, August 27, 2024 7:04 AM
> *To:* Ferruh Yigit <ferruh.yigit@amd.com>
> *Cc:* Wang, Yipeng1 <yipeng1.wang@intel.com>; Gobriel, Sameh <
> sameh.gobriel@intel.com>; Richardson, Bruce <bruce.richardson@intel.com>;
> Medvedkin, Vladimir <vladimir.medvedkin@intel.com>; dev@dpdk.org
> *Subject:* Re: Bihash Support in DPDK
>
>
>
> Hi All,
>
> Can we get some reply.
>
>
>
> Thanks
>
> Rajesh
>
>
>
> On Thu, Aug 22, 2024 at 9:32 PM Ferruh Yigit <ferruh.yigit@amd.com> wrote:
>
> On 8/22/2024 8:51 AM, rajesh goel wrote:
> > Hi All,
> > Need info if DPDK hash library supports bihash table where for multi-
> > thread and multi-process we can update/del/lookup entries per bucket
> level.
> >
> >
>
> + hash library maintainers.
>
>

[-- Attachment #2: Type: text/html, Size: 4767 bytes --]

RE: Bihash Support in DPDK

[Medvedkin, Vladimir]

[-- Attachment #1: Type: text/plain, Size: 2475 bytes --]

Hi Rajesh,

rte_hash does not support per bucket locks, instead it uses global rwlock.
But you can try lock free mode (see documentation, in particular RTE_HASH_EXTRA_FLAGS_RW_CONCURRENCY_LF flag)


From: rajesh goel <rgoel.bangalore@gmail.com>
Sent: Tuesday, August 27, 2024 4:57 PM
To: Medvedkin, Vladimir <vladimir.medvedkin@intel.com>
Cc: Ferruh Yigit <ferruh.yigit@amd.com>; Wang, Yipeng1 <yipeng1.wang@intel.com>; Gobriel, Sameh <sameh.gobriel@intel.com>; Richardson, Bruce <bruce.richardson@intel.com>; dev@dpdk.org
Subject: Re: Bihash Support in DPDK

Thanks for the reply.

Bihash I mean bounded index what Vpp supports.

Iam looking for the bucket level lock support. Currently Iam using hash table shared by multiple process or multiple core/threads. So I have to take the write lock by single core and then read lock by multiple cores to read the value wrote in this hash table. Multiple readers are getting blocked due to this. I want to avoid this to increase performance.

Let me know your thoughts on this.

Regards
Rajesh

On Tue, 27 Aug, 2024, 14:44 Medvedkin, Vladimir, <vladimir.medvedkin@intel.com<mailto:vladimir.medvedkin@intel.com>> wrote:
Hi Rajesh,

Please clarify what do you mean by “bihash”? Bidirectional? Bounded index?

As for concurrent lookup/updates, yes, DPDK hash table supports multi-process/multi-thread, please see the documentation:
https://doc.dpdk.org/guides/prog_guide/hash_lib.html#multi-process-support


From: rajesh goel <rgoel.bangalore@gmail.com<mailto:rgoel.bangalore@gmail.com>>
Sent: Tuesday, August 27, 2024 7:04 AM
To: Ferruh Yigit <ferruh.yigit@amd.com<mailto:ferruh.yigit@amd.com>>
Cc: Wang, Yipeng1 <yipeng1.wang@intel.com<mailto:yipeng1.wang@intel.com>>; Gobriel, Sameh <sameh.gobriel@intel.com<mailto:sameh.gobriel@intel.com>>; Richardson, Bruce <bruce.richardson@intel.com<mailto:bruce.richardson@intel.com>>; Medvedkin, Vladimir <vladimir.medvedkin@intel.com<mailto:vladimir.medvedkin@intel.com>>; dev@dpdk.org<mailto:dev@dpdk.org>
Subject: Re: Bihash Support in DPDK

Hi All,
Can we get some reply.

Thanks
Rajesh

On Thu, Aug 22, 2024 at 9:32 PM Ferruh Yigit <ferruh.yigit@amd.com<mailto:ferruh.yigit@amd.com>> wrote:
On 8/22/2024 8:51 AM, rajesh goel wrote:
> Hi All,
> Need info if DPDK hash library supports bihash table where for multi-
> thread and multi-process we can update/del/lookup entries per bucket level.
>
>

+ hash library maintainers.

[-- Attachment #2: Type: text/html, Size: 9111 bytes --]

Re: Bihash Support in DPDK

[rajesh goel]

[-- Attachment #1: Type: text/plain, Size: 2681 bytes --]

Thanks Vladimir for the confirmation.
Is there any plan to support bucket level lock support in dpdk hash.

Thanks
Rajesh

On Wed, Aug 28, 2024 at 2:33 PM Medvedkin, Vladimir <
vladimir.medvedkin@intel.com> wrote:

> Hi Rajesh,
>
>
>
> rte_hash does not support per bucket locks, instead it uses global rwlock.
>
> But you can try lock free mode (see documentation, in particular
> RTE_HASH_EXTRA_FLAGS_RW_CONCURRENCY_LF flag)
>
>
>
>
>
> *From:* rajesh goel <rgoel.bangalore@gmail.com>
> *Sent:* Tuesday, August 27, 2024 4:57 PM
> *To:* Medvedkin, Vladimir <vladimir.medvedkin@intel.com>
> *Cc:* Ferruh Yigit <ferruh.yigit@amd.com>; Wang, Yipeng1 <
> yipeng1.wang@intel.com>; Gobriel, Sameh <sameh.gobriel@intel.com>;
> Richardson, Bruce <bruce.richardson@intel.com>; dev@dpdk.org
> *Subject:* Re: Bihash Support in DPDK
>
>
>
> Thanks for the reply.
>
>
>
> Bihash I mean bounded index what Vpp supports.
>
>
>
> Iam looking for the bucket level lock support. Currently Iam using hash
> table shared by multiple process or multiple core/threads. So I have to
> take the write lock by single core and then read lock by multiple cores to
> read the value wrote in this hash table. Multiple readers are getting
> blocked due to this. I want to avoid this to increase performance.
>
>
>
> Let me know your thoughts on this.
>
>
>
> Regards
>
> Rajesh
>
>
>
> On Tue, 27 Aug, 2024, 14:44 Medvedkin, Vladimir, <
> vladimir.medvedkin@intel.com> wrote:
>
> Hi Rajesh,
>
>
>
> Please clarify what do you mean by “bihash”? Bidirectional? Bounded index?
>
>
>
> As for concurrent lookup/updates, yes, DPDK hash table supports
> multi-process/multi-thread, please see the documentation:
>
> https://doc.dpdk.org/guides/prog_guide/hash_lib.html#multi-process-support
>
>
>
>
>
> *From:* rajesh goel <rgoel.bangalore@gmail.com>
> *Sent:* Tuesday, August 27, 2024 7:04 AM
> *To:* Ferruh Yigit <ferruh.yigit@amd.com>
> *Cc:* Wang, Yipeng1 <yipeng1.wang@intel.com>; Gobriel, Sameh <
> sameh.gobriel@intel.com>; Richardson, Bruce <bruce.richardson@intel.com>;
> Medvedkin, Vladimir <vladimir.medvedkin@intel.com>; dev@dpdk.org
> *Subject:* Re: Bihash Support in DPDK
>
>
>
> Hi All,
>
> Can we get some reply.
>
>
>
> Thanks
>
> Rajesh
>
>
>
> On Thu, Aug 22, 2024 at 9:32 PM Ferruh Yigit <ferruh.yigit@amd.com> wrote:
>
> On 8/22/2024 8:51 AM, rajesh goel wrote:
> > Hi All,
> > Need info if DPDK hash library supports bihash table where for multi-
> > thread and multi-process we can update/del/lookup entries per bucket
> level.
> >
> >
>
> + hash library maintainers.
>
>

[-- Attachment #2: Type: text/html, Size: 7577 bytes --]

RE: Bihash Support in DPDK

[Medvedkin, Vladimir]

[-- Attachment #1: Type: text/plain, Size: 3356 bytes --]

I am not aware of such plans.

From: rajesh goel <rgoel.bangalore@gmail.com>
Sent: Wednesday, August 28, 2024 11:22 AM
To: Medvedkin, Vladimir <vladimir.medvedkin@intel.com>
Cc: Ferruh Yigit <ferruh.yigit@amd.com>; Wang, Yipeng1 <yipeng1.wang@intel.com>; Gobriel, Sameh <sameh.gobriel@intel.com>; Richardson, Bruce <bruce.richardson@intel.com>; dev@dpdk.org
Subject: Re: Bihash Support in DPDK

Thanks Vladimir for the confirmation.
Is there any plan to support bucket level lock support in dpdk hash.

Thanks
Rajesh

On Wed, Aug 28, 2024 at 2:33 PM Medvedkin, Vladimir <vladimir.medvedkin@intel.com<mailto:vladimir.medvedkin@intel.com>> wrote:
Hi Rajesh,

rte_hash does not support per bucket locks, instead it uses global rwlock.
But you can try lock free mode (see documentation, in particular RTE_HASH_EXTRA_FLAGS_RW_CONCURRENCY_LF flag)


From: rajesh goel <rgoel.bangalore@gmail.com<mailto:rgoel.bangalore@gmail.com>>
Sent: Tuesday, August 27, 2024 4:57 PM
To: Medvedkin, Vladimir <vladimir.medvedkin@intel.com<mailto:vladimir.medvedkin@intel.com>>
Cc: Ferruh Yigit <ferruh.yigit@amd.com<mailto:ferruh.yigit@amd.com>>; Wang, Yipeng1 <yipeng1.wang@intel.com<mailto:yipeng1.wang@intel.com>>; Gobriel, Sameh <sameh.gobriel@intel.com<mailto:sameh.gobriel@intel.com>>; Richardson, Bruce <bruce.richardson@intel.com<mailto:bruce.richardson@intel.com>>; dev@dpdk.org<mailto:dev@dpdk.org>
Subject: Re: Bihash Support in DPDK

Thanks for the reply.

Bihash I mean bounded index what Vpp supports.

Iam looking for the bucket level lock support. Currently Iam using hash table shared by multiple process or multiple core/threads. So I have to take the write lock by single core and then read lock by multiple cores to read the value wrote in this hash table. Multiple readers are getting blocked due to this. I want to avoid this to increase performance.

Let me know your thoughts on this.

Regards
Rajesh

On Tue, 27 Aug, 2024, 14:44 Medvedkin, Vladimir, <vladimir.medvedkin@intel.com<mailto:vladimir.medvedkin@intel.com>> wrote:
Hi Rajesh,

Please clarify what do you mean by “bihash”? Bidirectional? Bounded index?

As for concurrent lookup/updates, yes, DPDK hash table supports multi-process/multi-thread, please see the documentation:
https://doc.dpdk.org/guides/prog_guide/hash_lib.html#multi-process-support


From: rajesh goel <rgoel.bangalore@gmail.com<mailto:rgoel.bangalore@gmail.com>>
Sent: Tuesday, August 27, 2024 7:04 AM
To: Ferruh Yigit <ferruh.yigit@amd.com<mailto:ferruh.yigit@amd.com>>
Cc: Wang, Yipeng1 <yipeng1.wang@intel.com<mailto:yipeng1.wang@intel.com>>; Gobriel, Sameh <sameh.gobriel@intel.com<mailto:sameh.gobriel@intel.com>>; Richardson, Bruce <bruce.richardson@intel.com<mailto:bruce.richardson@intel.com>>; Medvedkin, Vladimir <vladimir.medvedkin@intel.com<mailto:vladimir.medvedkin@intel.com>>; dev@dpdk.org<mailto:dev@dpdk.org>
Subject: Re: Bihash Support in DPDK

Hi All,
Can we get some reply.

Thanks
Rajesh

On Thu, Aug 22, 2024 at 9:32 PM Ferruh Yigit <ferruh.yigit@amd.com<mailto:ferruh.yigit@amd.com>> wrote:
On 8/22/2024 8:51 AM, rajesh goel wrote:
> Hi All,
> Need info if DPDK hash library supports bihash table where for multi-
> thread and multi-process we can update/del/lookup entries per bucket level.
>
>

+ hash library maintainers.

[-- Attachment #2: Type: text/html, Size: 15272 bytes --]

Re: Bihash Support in DPDK

[Stephen Hemminger]

On Wed, 28 Aug 2024 15:54:27 +0000
"Medvedkin, Vladimir" <vladimir.medvedkin@intel.com> wrote:

> Thanks for the reply.
> 
> Bihash I mean bounded index what Vpp supports.
> 
> Iam looking for the bucket level lock support. Currently Iam using hash table shared by multiple process or multiple core/threads. So I have to take the write lock by single core and then read lock by multiple cores to read the value wrote in this hash table. Multiple readers are getting blocked due to this. I want to avoid this to increase performance.
> 
> Let me know your thoughts on this.
> 
> Regards
> Rajesh

RCU is always faster than reader/writer locks.
Reader/Writer locks are slower than simple spin lock unless reader holds for a long time.

Re: Bihash Support in DPDK

[rajesh goel]

[-- Attachment #1: Type: text/plain, Size: 1030 bytes --]

Thanks Stephen , Vladimir,
Let me explore DPDK RCU more, if it helps my application.

Will get back with my results.

Regards.
Rajesh

On Wed, Aug 28, 2024 at 10:53 PM Stephen Hemminger <
stephen@networkplumber.org> wrote:

> On Wed, 28 Aug 2024 15:54:27 +0000
> "Medvedkin, Vladimir" <vladimir.medvedkin@intel.com> wrote:
>
> > Thanks for the reply.
> >
> > Bihash I mean bounded index what Vpp supports.
> >
> > Iam looking for the bucket level lock support. Currently Iam using hash
> table shared by multiple process or multiple core/threads. So I have to
> take the write lock by single core and then read lock by multiple cores to
> read the value wrote in this hash table. Multiple readers are getting
> blocked due to this. I want to avoid this to increase performance.
> >
> > Let me know your thoughts on this.
> >
> > Regards
> > Rajesh
>
> RCU is always faster than reader/writer locks.
> Reader/Writer locks are slower than simple spin lock unless reader holds
> for a long time.
>

[-- Attachment #2: Type: text/html, Size: 1504 bytes --]