[dpdk-users] Sharing tables among pipelines

[dpdk-users] Sharing tables among pipelines

[Nidhia Varghese]

Hi,

For my pipeline application, I want to share same table between two
different pipeline. Is that possible? If yes, how can I do it?

Thanks for your reply and help.

-- 

Regards,
Nidhia Varghese

[dpdk-users] Fwd: Sharing tables among pipelines

[Nidhia Varghese]

Hi,

Any comments on this question?

Thanks,
Nidhia

---------- Forwarded message ----------
From: Nidhia Varghese <nidhiavarghese93@gmail.com>
Date: Thu, May 4, 2017 at 3:43 PM
Subject: Sharing tables among pipelines
To: dev@dpdk.org, users@dpdk.org


Hi,

For my pipeline application, I want to share same table between two
different pipeline. Is that possible? If yes, how can I do it?

Thanks for your reply and help.

-- 

Regards,
Nidhia Varghese



-- 

Regards,
Nidhia Varghese

Re: [dpdk-users] Sharing tables among pipelines

[zhilong zheng]

Hi Nidia,

I have tried something similar to yours. Actually, I share some hash tables among different/independent processes.
For my application, I extend the codebase from DPDK Sample Applications 23.1.4(http://dpdk.org/doc/guides-16.07/sample_app_ug/multi_process.html <http://dpdk.org/doc/guides-16.07/sample_app_ug/multi_process.html> Client-Server Multi-process Example). The API "rte_memzone_lookup(const char *name)" contributes to the availability.

I hope this will help you.

Regards,
Zhilong

> 在 2017年5月8日，12:54，Nidhia Varghese <nidhiavarghese93@gmail.com> 写道：
> 
> Hi,
> 
> Any comments on this question?
> 
> Thanks,
> Nidhia
> 
> ---------- Forwarded message ----------
> From: Nidhia Varghese <nidhiavarghese93@gmail.com>
> Date: Thu, May 4, 2017 at 3:43 PM
> Subject: Sharing tables among pipelines
> To: dev@dpdk.org, users@dpdk.org
> 
> 
> Hi,
> 
> For my pipeline application, I want to share same table between two
> different pipeline. Is that possible? If yes, how can I do it?
> 
> Thanks for your reply and help.
> 
> -- 
> 
> Regards,
> Nidhia Varghese
> 
> 
> 
> -- 
> 
> Regards,
> Nidhia Varghese

Re: [dpdk-users] [dpdk-dev] Fwd: Sharing tables among pipelines

[Singh, Jasvinder]

Hi Nidhia,


For my pipeline application, I want to share same table between two different pipeline. Is that possible? If yes, how can I do it?

[Jasvinder] -  In the current ip pipeline application, we don’t have such illustration of sharing a table between two pipelines. Therefore, you need to tweak the code to implement it for your application.

Thanks,
Jasvinder

Re: [dpdk-users] [dpdk-dev] Fwd: Sharing tables among pipelines

[Dumitrescu, Cristian]

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Singh, Jasvinder
> Sent: Monday, May 8, 2017 11:37 AM
> To: Nidhia Varghese <nidhiavarghese93@gmail.com>; dev@dpdk.org;
> users@dpdk.org
> Subject: Re: [dpdk-dev] Fwd: Sharing tables among pipelines
> 
> Hi Nidhia,
> 
> 
> For my pipeline application, I want to share same table between two
> different pipeline. Is that possible? If yes, how can I do it?
> 

This is usually a bad idea due to the implications: locks required for each table record, performance penalty due to cache line ping-pong between cores.

See link below for a quick discussion:
http://www.dpdk.org/doc/guides/prog_guide/packet_framework.html#shared-data-structures

Usually we split the application-level table into several smaller tables that are sections of the big table and each one is handled by a different pipeline instance (of e.g. the flow classification pipeline type), with each instance potentially running on different CPU core; we make sure that packets reach the right pipeline instance through flow affinity schemes like RSS or SW steering.

> [Jasvinder] -  In the current ip pipeline application, we don’t have such
> illustration of sharing a table between two pipelines. Therefore, you need to
> tweak the code to implement it for your application.
> 
> Thanks,
> Jasvinder

Re: [dpdk-users] [dpdk-dev] Fwd: Sharing tables among pipelines

[Nidhia Varghese]

Hi all,

Thanks for your reply.

As given in the link(3rd point), what I need is single writer
thread(pipeline) performing table entry add/delete operations and another
thread(which is in turn another pipeline) that performs lookup operations
to the same table entries.


Thanks,
Nidhia Varghese

Re: [dpdk-users] [dpdk-dev] Fwd: Sharing tables among pipelines

[Shyam Shrivastav]

Something I have gathered reading through code if it helps, ip_pipeline is
designed such that user side cli (_fe) runs in master pipeline and sends
conf requests to various other packet processing pipelines (_be) which
handle such requests at intervals. These can run on same or different
threads(cores) as per conf, no locking requirement as table-updation &
table-reading (packet processing) always happens in same pipeline so always
same thread. Master pipeline can be configured to run on a separate core &
it should be the case, but table updation happens in pipeline thread only
at lower priority.



On Tue, May 9, 2017 at 11:29 PM, Nidhia Varghese <nidhiavarghese93@gmail.com
> wrote:

> Hi all,
>
> Thanks for your reply.
>
> As given in the link(3rd point), what I need is single writer
> thread(pipeline) performing table entry add/delete operations and another
> thread(which is in turn another pipeline) that performs lookup operations
> to the same table entries.
>
>
> Thanks,
> Nidhia Varghese
>

Re: [dpdk-users] [dpdk-dev] Fwd: Sharing tables among pipelines

[Dumitrescu, Cristian]

Thanks Shyam, great to see that people get the idea behind this design! Regards, Cristian

From: Shyam Shrivastav [mailto:shrivastav.shyam@gmail.com]
Sent: Wednesday, May 10, 2017 5:14 PM
To: Nidhia Varghese <nidhiavarghese93@gmail.com>
Cc: Dumitrescu, Cristian <cristian.dumitrescu@intel.com>; Singh, Jasvinder <jasvinder.singh@intel.com>; dev@dpdk.org; users@dpdk.org
Subject: Re: [dpdk-users] [dpdk-dev] Fwd: Sharing tables among pipelines

Something I have gathered reading through code if it helps, ip_pipeline is designed such that user side cli (_fe) runs in master pipeline and sends conf requests to various other packet processing pipelines (_be) which handle such requests at intervals. These can run on same or different threads(cores) as per conf, no locking requirement as table-updation & table-reading (packet processing) always happens in same pipeline so always same thread. Master pipeline can be configured to run on a separate core & it should be the case, but table updation happens in pipeline thread only at lower priority.


On Tue, May 9, 2017 at 11:29 PM, Nidhia Varghese <nidhiavarghese93@gmail.com<mailto:nidhiavarghese93@gmail.com>> wrote:
Hi all,

Thanks for your reply.

As given in the link(3rd point), what I need is single writer
thread(pipeline) performing table entry add/delete operations and another
thread(which is in turn another pipeline) that performs lookup operations
to the same table entries.


Thanks,
Nidhia Varghese