[dpdk-dev] [ 2nd try ] Lookup mechanim in DPDK HASH table.

[dpdk-dev] [ 2nd try ] Lookup mechanim in DPDK HASH table.

[Yeddula, Avinash]

Any comments on this question ?

Thanks
-Avinash

-----Original Message-----
From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Yeddula, Avinash
Sent: Wednesday, August 12, 2015 3:04 PM
To: dev@dpdk.org
Subject: [dpdk-dev] Lookup mechanim in DPDK HASH table.

Hello All,

I'm using DPDK extendable hash tables. This question is with respect to the lookup aspect of the hash table.
I see that there is just one "t->key_offset" that is pre-defined for the hash table. I also understand that the frame needs to carry the "lookup_key / keys" in the meta data.

Here is my question:  How to support more than one lookup with different keys on the same frame on the same table.
Use case: Src mac  lookup and dst mac lookup on the same mac table.

Thanks
-Avinash

Re: [dpdk-dev] [ 2nd try ] Lookup mechanim in DPDK HASH table.

[Bruce Richardson]

On Thu, Aug 13, 2015 at 05:37:21PM -0400, Yeddula, Avinash wrote:
> Any comments on this question ?
> 
> Thanks
> -Avinash
> 
> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Yeddula, Avinash
> Sent: Wednesday, August 12, 2015 3:04 PM
> To: dev@dpdk.org
> Subject: [dpdk-dev] Lookup mechanim in DPDK HASH table.
> 
> Hello All,
> 
> I'm using DPDK extendable hash tables. This question is with respect to the lookup aspect of the hash table.
> I see that there is just one "t->key_offset" that is pre-defined for the hash table. I also understand that the frame needs to carry the "lookup_key / keys" in the meta data.
> 
> Here is my question:  How to support more than one lookup with different keys on the same frame on the same table.
> Use case: Src mac  lookup and dst mac lookup on the same mac table.
> 
> Thanks
> -Avinash

Just to confirm: this is using the extensible bucket hash in the rte_table library of packet framework,
rather than the standalone rte_hash library, right?

/Bruce

Re: [dpdk-dev] [ 2nd try ] Lookup mechanim in DPDK HASH table.

[Yeddula, Avinash]

That's correct Bruce.

Thanks
-Avinash

-----Original Message-----
From: Bruce Richardson [mailto:bruce.richardson@intel.com] 
Sent: Friday, August 14, 2015 2:25 AM
To: Yeddula, Avinash
Cc: dev@dpdk.org
Subject: Re: [dpdk-dev] [ 2nd try ] Lookup mechanim in DPDK HASH table.

On Thu, Aug 13, 2015 at 05:37:21PM -0400, Yeddula, Avinash wrote:
> Any comments on this question ?
> 
> Thanks
> -Avinash
> 
> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Yeddula, Avinash
> Sent: Wednesday, August 12, 2015 3:04 PM
> To: dev@dpdk.org
> Subject: [dpdk-dev] Lookup mechanim in DPDK HASH table.
> 
> Hello All,
> 
> I'm using DPDK extendable hash tables. This question is with respect to the lookup aspect of the hash table.
> I see that there is just one "t->key_offset" that is pre-defined for the hash table. I also understand that the frame needs to carry the "lookup_key / keys" in the meta data.
> 
> Here is my question:  How to support more than one lookup with different keys on the same frame on the same table.
> Use case: Src mac  lookup and dst mac lookup on the same mac table.
> 
> Thanks
> -Avinash

Just to confirm: this is using the extensible bucket hash in the rte_table library of packet framework, rather than the standalone rte_hash library, right?

/Bruce

Re: [dpdk-dev] [ 2nd try ] Lookup mechanim in DPDK HASH table.

[Singh, Jasvinder]

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Bruce Richardson
> Sent: Friday, August 14, 2015 10:25 AM
> To: Yeddula, Avinash
> Cc: dev@dpdk.org
> Subject: Re: [dpdk-dev] [ 2nd try ] Lookup mechanim in DPDK HASH table.
> 
> On Thu, Aug 13, 2015 at 05:37:21PM -0400, Yeddula, Avinash wrote:
> > Any comments on this question ?
> >
> > Thanks
> > -Avinash
> >
> > -----Original Message-----
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Yeddula, Avinash
> > Sent: Wednesday, August 12, 2015 3:04 PM
> > To: dev@dpdk.org
> > Subject: [dpdk-dev] Lookup mechanim in DPDK HASH table.
> >
> > Hello All,
> >
> > I'm using DPDK extendable hash tables. This question is with respect to the
> lookup aspect of the hash table.
> > I see that there is just one "t->key_offset" that is pre-defined for the hash
> table. I also understand that the frame needs to carry the "lookup_key /
> keys" in the meta data.
> >
> > Here is my question:  How to support more than one lookup with different
> keys on the same frame on the same table.
> > Use case: Src mac  lookup and dst mac lookup on the same mac table.
> >
> > Thanks
> > -Avinash
> 
> Just to confirm: this is using the extensible bucket hash in the rte_table
> library of packet framework, rather than the standalone rte_hash library,
> right?
> 
> /Bruce

Could you share detail on the two different keys used for lookups. In case if you are considering bidirectional packet flow between the source and destination, symmetric hash can be used-  http://www.ndsl.kaist.edu/~kyoungsoo/papers/TR-symRSS.pdf 

Jasvinder

Re: [dpdk-dev] [ 2nd try ] Lookup mechanim in DPDK HASH table.

[Yeddula, Avinash]

+ Mike ( My team mate)
Hi Jasvinder, It's not a bidirectional packet flow. The pipeline looks something like this.

Ingress port-----Table 1 ----Table-2 ----- Mac_Table ----- Table4 ---- Egress port.

Before the frame goes reaches table 4, we do 2 lookups at the mac table.
1. src lookup ( To learn the MAC on the bridge)
2. dst lookup ( Flooding if dst MAC look up fails else Unicast/forward if dst lookup success).

Here are the keys we are using.
Src lookup key  -  Src MAC (src MAC in the frame) + Bridge ID ( Bridge on which it arrived).
Dst lookup key  - Dst MAC (dst MAC in the frame) + Bridge ID ( Bridge on which it arrived)


Thanks
-Avinash

-----Original Message-----
From: Singh, Jasvinder [mailto:jasvinder.singh@intel.com] 
Sent: Monday, August 17, 2015 7:00 AM
To: Richardson, Bruce; Yeddula, Avinash
Cc: dev@dpdk.org
Subject: RE: [dpdk-dev] [ 2nd try ] Lookup mechanim in DPDK HASH table.



> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Bruce Richardson
> Sent: Friday, August 14, 2015 10:25 AM
> To: Yeddula, Avinash
> Cc: dev@dpdk.org
> Subject: Re: [dpdk-dev] [ 2nd try ] Lookup mechanim in DPDK HASH table.
> 
> On Thu, Aug 13, 2015 at 05:37:21PM -0400, Yeddula, Avinash wrote:
> > Any comments on this question ?
> >
> > Thanks
> > -Avinash
> >
> > -----Original Message-----
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Yeddula, 
> > Avinash
> > Sent: Wednesday, August 12, 2015 3:04 PM
> > To: dev@dpdk.org
> > Subject: [dpdk-dev] Lookup mechanim in DPDK HASH table.
> >
> > Hello All,
> >
> > I'm using DPDK extendable hash tables. This question is with respect 
> > to the
> lookup aspect of the hash table.
> > I see that there is just one "t->key_offset" that is pre-defined for 
> > the hash
> table. I also understand that the frame needs to carry the "lookup_key 
> / keys" in the meta data.
> >
> > Here is my question:  How to support more than one lookup with 
> > different
> keys on the same frame on the same table.
> > Use case: Src mac  lookup and dst mac lookup on the same mac table.
> >
> > Thanks
> > -Avinash
> 
> Just to confirm: this is using the extensible bucket hash in the 
> rte_table library of packet framework, rather than the standalone 
> rte_hash library, right?
> 
> /Bruce

Could you share detail on the two different keys used for lookups. In case if you are considering bidirectional packet flow between the source and destination, symmetric hash can be used-  http://www.ndsl.kaist.edu/~kyoungsoo/papers/TR-symRSS.pdf 

Jasvinder

Re: [dpdk-dev] [ 2nd try ] Lookup mechanim in DPDK HASH table.

[Venkateswara Rao Thummala]

Hi Avinash,

I think, you can use the same table by just updating the packet meta data
based on the Look UP.
In the first lookup, you can populate the meta data [key offset] with the
source MAC and in the second lookup, you can populate the same meta data
with the destination lookup. I think this should work.

Thanks
Venkat
OneHop Networks

On 17 August 2015 at 22:05, Yeddula, Avinash <ayeddula@ciena.com> wrote:

> + Mike ( My team mate)
> Hi Jasvinder, It's not a bidirectional packet flow. The pipeline looks
> something like this.
>
> Ingress port-----Table 1 ----Table-2 ----- Mac_Table ----- Table4 ----
> Egress port.
>
> Before the frame goes reaches table 4, we do 2 lookups at the mac table.
> 1. src lookup ( To learn the MAC on the bridge)
> 2. dst lookup ( Flooding if dst MAC look up fails else Unicast/forward if
> dst lookup success).
>
> Here are the keys we are using.
> Src lookup key  -  Src MAC (src MAC in the frame) + Bridge ID ( Bridge on
> which it arrived).
> Dst lookup key  - Dst MAC (dst MAC in the frame) + Bridge ID ( Bridge on
> which it arrived)
>
>
> Thanks
> -Avinash
>
> -----Original Message-----
> From: Singh, Jasvinder [mailto:jasvinder.singh@intel.com]
> Sent: Monday, August 17, 2015 7:00 AM
> To: Richardson, Bruce; Yeddula, Avinash
> Cc: dev@dpdk.org
> Subject: RE: [dpdk-dev] [ 2nd try ] Lookup mechanim in DPDK HASH table.
>
>
>
> > -----Original Message-----
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Bruce Richardson
> > Sent: Friday, August 14, 2015 10:25 AM
> > To: Yeddula, Avinash
> > Cc: dev@dpdk.org
> > Subject: Re: [dpdk-dev] [ 2nd try ] Lookup mechanim in DPDK HASH table.
> >
> > On Thu, Aug 13, 2015 at 05:37:21PM -0400, Yeddula, Avinash wrote:
> > > Any comments on this question ?
> > >
> > > Thanks
> > > -Avinash
> > >
> > > -----Original Message-----
> > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Yeddula,
> > > Avinash
> > > Sent: Wednesday, August 12, 2015 3:04 PM
> > > To: dev@dpdk.org
> > > Subject: [dpdk-dev] Lookup mechanim in DPDK HASH table.
> > >
> > > Hello All,
> > >
> > > I'm using DPDK extendable hash tables. This question is with respect
> > > to the
> > lookup aspect of the hash table.
> > > I see that there is just one "t->key_offset" that is pre-defined for
> > > the hash
> > table. I also understand that the frame needs to carry the "lookup_key
> > / keys" in the meta data.
> > >
> > > Here is my question:  How to support more than one lookup with
> > > different
> > keys on the same frame on the same table.
> > > Use case: Src mac  lookup and dst mac lookup on the same mac table.
> > >
> > > Thanks
> > > -Avinash
> >
> > Just to confirm: this is using the extensible bucket hash in the
> > rte_table library of packet framework, rather than the standalone
> > rte_hash library, right?
> >
> > /Bruce
>
> Could you share detail on the two different keys used for lookups. In case
> if you are considering bidirectional packet flow between the source and
> destination, symmetric hash can be used-
> http://www.ndsl.kaist.edu/~kyoungsoo/papers/TR-symRSS.pdf
>
> Jasvinder
>

Re: [dpdk-dev] [ 2nd try ] Lookup mechanim in DPDK HASH table.

[Bly, Mike]

Venkat,

While that does “solve” the issue, I find the approach of tying the meta-data-key-offset to a specific table highly limiting. IMO, it would be far more interesting to provide the offset as part of the lookup call itself. For example, I might have an optimized code sequence that generates a series of “keys” in meta data, which then triggers a series of lookups, where one or more lookups are different keys, but in the same table(s). As it stands now, the proposed solution requires an iterative updating of the same key location in meta data between lookups. This also prevents multiple tables from sharing the same meta data “space” for overlapping key values, in that I am forced to iteratively change/reload said key data.

Another thing to consider would be the ability to provide offset/size per sub-field for a given key. The MAC table in question is a great example. I’ll add IP addresses here as well to make it a bit more interesting:

Meta-data = DA | (SA << 6*8) | (BRIDGE_ID << 12*8) | (DIP << 14*8) | (SIP << 20*8)

If we were allowed to use per field offset/size values, I could use the above meta-data for two MAC lookups and two FIB/RIB lookups and perhaps a combo L2/L3 ACL lookup. However, the current limitation requires me to modify the “meta-data” before each individual lookup, which means my frame parsing is iterative and NOT necessarily optimized.

-MikeB

From: Venkateswara Rao Thummala [mailto:venkat.thummala.1978@gmail.com]
Sent: Monday, August 17, 2015 11:06 AM
To: Yeddula, Avinash
Cc: Singh, Jasvinder; Richardson, Bruce; dev@dpdk.org; Bly, Mike
Subject: Re: [dpdk-dev] [ 2nd try ] Lookup mechanim in DPDK HASH table.

Hi Avinash,
I think, you can use the same table by just updating the packet meta data based on the Look UP.
In the first lookup, you can populate the meta data [key offset] with the source MAC and in the second lookup, you can populate the same meta data with the destination lookup. I think this should work.
Thanks
Venkat
OneHop Networks

On 17 August 2015 at 22:05, Yeddula, Avinash <ayeddula@ciena.com<mailto:ayeddula@ciena.com>> wrote:
+ Mike ( My team mate)
Hi Jasvinder, It's not a bidirectional packet flow. The pipeline looks something like this.

Ingress port-----Table 1 ----Table-2 ----- Mac_Table ----- Table4 ---- Egress port.

Before the frame goes reaches table 4, we do 2 lookups at the mac table.
1. src lookup ( To learn the MAC on the bridge)
2. dst lookup ( Flooding if dst MAC look up fails else Unicast/forward if dst lookup success).

Here are the keys we are using.
Src lookup key  -  Src MAC (src MAC in the frame) + Bridge ID ( Bridge on which it arrived).
Dst lookup key  - Dst MAC (dst MAC in the frame) + Bridge ID ( Bridge on which it arrived)


Thanks
-Avinash

-----Original Message-----
From: Singh, Jasvinder [mailto:jasvinder.singh@intel.com<mailto:jasvinder.singh@intel.com>]
Sent: Monday, August 17, 2015 7:00 AM
To: Richardson, Bruce; Yeddula, Avinash
Cc: dev@dpdk.org<mailto:dev@dpdk.org>
Subject: RE: [dpdk-dev] [ 2nd try ] Lookup mechanim in DPDK HASH table.



> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org<mailto:dev-bounces@dpdk.org>] On Behalf Of Bruce Richardson
> Sent: Friday, August 14, 2015 10:25 AM
> To: Yeddula, Avinash
> Cc: dev@dpdk.org<mailto:dev@dpdk.org>
> Subject: Re: [dpdk-dev] [ 2nd try ] Lookup mechanim in DPDK HASH table.
>
> On Thu, Aug 13, 2015 at 05:37:21PM -0400, Yeddula, Avinash wrote:
> > Any comments on this question ?
> >
> > Thanks
> > -Avinash
> >
> > -----Original Message-----
> > From: dev [mailto:dev-bounces@dpdk.org<mailto:dev-bounces@dpdk.org>] On Behalf Of Yeddula,
> > Avinash
> > Sent: Wednesday, August 12, 2015 3:04 PM
> > To: dev@dpdk.org<mailto:dev@dpdk.org>
> > Subject: [dpdk-dev] Lookup mechanim in DPDK HASH table.
> >
> > Hello All,
> >
> > I'm using DPDK extendable hash tables. This question is with respect
> > to the
> lookup aspect of the hash table.
> > I see that there is just one "t->key_offset" that is pre-defined for
> > the hash
> table. I also understand that the frame needs to carry the "lookup_key
> / keys" in the meta data.
> >
> > Here is my question:  How to support more than one lookup with
> > different
> keys on the same frame on the same table.
> > Use case: Src mac  lookup and dst mac lookup on the same mac table.
> >
> > Thanks
> > -Avinash
>
> Just to confirm: this is using the extensible bucket hash in the
> rte_table library of packet framework, rather than the standalone
> rte_hash library, right?
>
> /Bruce

Could you share detail on the two different keys used for lookups. In case if you are considering bidirectional packet flow between the source and destination, symmetric hash can be used-  http://www.ndsl.kaist.edu/~kyoungsoo/papers/TR-symRSS.pdf

Jasvinder

Re: [dpdk-dev] [ 2nd try ] Lookup mechanim in DPDK HASH table.

[Dumitrescu, Cristian]

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Venkateswara Rao
> Thummala
> Sent: Monday, August 17, 2015 7:06 PM
> To: Yeddula, Avinash
> Cc: dev@dpdk.org; Bly, Mike
> Subject: Re: [dpdk-dev] [ 2nd try ] Lookup mechanim in DPDK HASH table.
> 
> Hi Avinash,
> 
> I think, you can use the same table by just updating the packet meta data
> based on the Look UP.
> In the first lookup, you can populate the meta data [key offset] with the
> source MAC and in the second lookup, you can populate the same meta data
> with the destination lookup. I think this should work.
> 
> Thanks
> Venkat
> OneHop Networks

Yes, should work, although probably this approach is not really required by the specific use-case. The two lookups in the same table are really one lookup and one add/update operation.

> 
> On 17 August 2015 at 22:05, Yeddula, Avinash <ayeddula@ciena.com> wrote:
> 
> > + Mike ( My team mate)
> > Hi Jasvinder, It's not a bidirectional packet flow. The pipeline looks
> > something like this.
> >
> > Ingress port-----Table 1 ----Table-2 ----- Mac_Table ----- Table4 ----
> > Egress port.
> >
> > Before the frame goes reaches table 4, we do 2 lookups at the mac table.
> > 1. src lookup ( To learn the MAC on the bridge)
> > 2. dst lookup ( Flooding if dst MAC look up fails else Unicast/forward if
> > dst lookup success).
> >
> > Here are the keys we are using.
> > Src lookup key  -  Src MAC (src MAC in the frame) + Bridge ID ( Bridge on
> > which it arrived).
> > Dst lookup key  - Dst MAC (dst MAC in the frame) + Bridge ID ( Bridge on
> > which it arrived)
> >
> >
> > Thanks
> > -Avinash
> >
> > -----Original Message-----
> > From: Singh, Jasvinder [mailto:jasvinder.singh@intel.com]
> > Sent: Monday, August 17, 2015 7:00 AM
> > To: Richardson, Bruce; Yeddula, Avinash
> > Cc: dev@dpdk.org
> > Subject: RE: [dpdk-dev] [ 2nd try ] Lookup mechanim in DPDK HASH table.
> >
> >
> >
> > > -----Original Message-----
> > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Bruce
> Richardson
> > > Sent: Friday, August 14, 2015 10:25 AM
> > > To: Yeddula, Avinash
> > > Cc: dev@dpdk.org
> > > Subject: Re: [dpdk-dev] [ 2nd try ] Lookup mechanim in DPDK HASH table.
> > >
> > > On Thu, Aug 13, 2015 at 05:37:21PM -0400, Yeddula, Avinash wrote:
> > > > Any comments on this question ?
> > > >
> > > > Thanks
> > > > -Avinash
> > > >
> > > > -----Original Message-----
> > > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Yeddula,
> > > > Avinash
> > > > Sent: Wednesday, August 12, 2015 3:04 PM
> > > > To: dev@dpdk.org
> > > > Subject: [dpdk-dev] Lookup mechanim in DPDK HASH table.
> > > >
> > > > Hello All,
> > > >
> > > > I'm using DPDK extendable hash tables. This question is with respect
> > > > to the
> > > lookup aspect of the hash table.
> > > > I see that there is just one "t->key_offset" that is pre-defined for
> > > > the hash
> > > table. I also understand that the frame needs to carry the "lookup_key
> > > / keys" in the meta data.
> > > >
> > > > Here is my question:  How to support more than one lookup with
> > > > different
> > > keys on the same frame on the same table.
> > > > Use case: Src mac  lookup and dst mac lookup on the same mac table.
> > > >
> > > > Thanks
> > > > -Avinash
> > >
> > > Just to confirm: this is using the extensible bucket hash in the
> > > rte_table library of packet framework, rather than the standalone
> > > rte_hash library, right?
> > >
> > > /Bruce
> >
> > Could you share detail on the two different keys used for lookups. In case
> > if you are considering bidirectional packet flow between the source and
> > destination, symmetric hash can be used-
> > http://www.ndsl.kaist.edu/~kyoungsoo/papers/TR-symRSS.pdf
> >
> > Jasvinder
> >

Re: [dpdk-dev] [ 2nd try ] Lookup mechanim in DPDK HASH table.

[Singh, Jasvinder]

Hi,

> -----Original Message-----
> From: Yeddula, Avinash [mailto:ayeddula@ciena.com]
> Sent: Monday, August 17, 2015 5:35 PM
> To: Singh, Jasvinder; Richardson, Bruce
> Cc: dev@dpdk.org; Bly, Mike
> Subject: RE: [dpdk-dev] [ 2nd try ] Lookup mechanim in DPDK HASH table.
> 
> + Mike ( My team mate)
> Hi Jasvinder, It's not a bidirectional packet flow. The pipeline looks something
> like this.
> 
> Ingress port-----Table 1 ----Table-2 ----- Mac_Table ----- Table4 ---- Egress
> port.
> 
> Before the frame goes reaches table 4, we do 2 lookups at the mac table.
> 1. src lookup ( To learn the MAC on the bridge) 2. dst lookup ( Flooding if dst
> MAC look up fails else Unicast/forward if dst lookup success).
> 
> Here are the keys we are using.
> Src lookup key  -  Src MAC (src MAC in the frame) + Bridge ID ( Bridge on
> which it arrived).
> Dst lookup key  - Dst MAC (dst MAC in the frame) + Bridge ID ( Bridge on
> which it arrived)
> 

There is as such no mechanism to support double lookup on the same table for the packet. However, alternative approach could be-    first perform destination lookup and  invoke action handler ( in both table hit & miss case). In action handler, Src MAC + bridge ID key can be added to the table (rte_pipeline_table_entry_add) .  If a new entry is successfully added to the hash for the specified key, or there is already an entry in the hash for the specified key, then the position of the entry is returned. In this way, table entries will be updated with incoming packets.

 
> Thanks
> -Avinash
> 
> -----Original Message-----
> From: Singh, Jasvinder [mailto:jasvinder.singh@intel.com]
> Sent: Monday, August 17, 2015 7:00 AM
> To: Richardson, Bruce; Yeddula, Avinash
> Cc: dev@dpdk.org
> Subject: RE: [dpdk-dev] [ 2nd try ] Lookup mechanim in DPDK HASH table.
> 
> 
> 
> > -----Original Message-----
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Bruce Richardson
> > Sent: Friday, August 14, 2015 10:25 AM
> > To: Yeddula, Avinash
> > Cc: dev@dpdk.org
> > Subject: Re: [dpdk-dev] [ 2nd try ] Lookup mechanim in DPDK HASH table.
> >
> > On Thu, Aug 13, 2015 at 05:37:21PM -0400, Yeddula, Avinash wrote:
> > > Any comments on this question ?
> > >
> > > Thanks
> > > -Avinash
> > >
> > > -----Original Message-----
> > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Yeddula,
> > > Avinash
> > > Sent: Wednesday, August 12, 2015 3:04 PM
> > > To: dev@dpdk.org
> > > Subject: [dpdk-dev] Lookup mechanim in DPDK HASH table.
> > >
> > > Hello All,
> > >
> > > I'm using DPDK extendable hash tables. This question is with respect
> > > to the
> > lookup aspect of the hash table.
> > > I see that there is just one "t->key_offset" that is pre-defined for
> > > the hash
> > table. I also understand that the frame needs to carry the "lookup_key
> > / keys" in the meta data.
> > >
> > > Here is my question:  How to support more than one lookup with
> > > different
> > keys on the same frame on the same table.
> > > Use case: Src mac  lookup and dst mac lookup on the same mac table.
> > >
> > > Thanks
> > > -Avinash
> >
> > Just to confirm: this is using the extensible bucket hash in the
> > rte_table library of packet framework, rather than the standalone
> > rte_hash library, right?
> >
> > /Bruce
> 
> Could you share detail on the two different keys used for lookups. In case if
> you are considering bidirectional packet flow between the source and
> destination, symmetric hash can be used-
> http://www.ndsl.kaist.edu/~kyoungsoo/papers/TR-symRSS.pdf
> 
> Jasvinder

Re: [dpdk-dev] [ 2nd try ] Lookup mechanim in DPDK HASH table.

[Dumitrescu, Cristian]

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Singh, Jasvinder
> Sent: Monday, August 17, 2015 7:14 PM
> To: Yeddula, Avinash; Richardson, Bruce
> Cc: dev@dpdk.org; Bly, Mike
> Subject: Re: [dpdk-dev] [ 2nd try ] Lookup mechanim in DPDK HASH table.
> 
> Hi,
> 
> > -----Original Message-----
> > From: Yeddula, Avinash [mailto:ayeddula@ciena.com]
> > Sent: Monday, August 17, 2015 5:35 PM
> > To: Singh, Jasvinder; Richardson, Bruce
> > Cc: dev@dpdk.org; Bly, Mike
> > Subject: RE: [dpdk-dev] [ 2nd try ] Lookup mechanim in DPDK HASH table.
> >
> > + Mike ( My team mate)
> > Hi Jasvinder, It's not a bidirectional packet flow. The pipeline looks
> something
> > like this.
> >
> > Ingress port-----Table 1 ----Table-2 ----- Mac_Table ----- Table4 ---- Egress
> > port.
> >
> > Before the frame goes reaches table 4, we do 2 lookups at the mac table.
> > 1. src lookup ( To learn the MAC on the bridge) 2. dst lookup ( Flooding if
> dst
> > MAC look up fails else Unicast/forward if dst lookup success).
> >
> > Here are the keys we are using.
> > Src lookup key  -  Src MAC (src MAC in the frame) + Bridge ID ( Bridge on
> > which it arrived).
> > Dst lookup key  - Dst MAC (dst MAC in the frame) + Bridge ID ( Bridge on
> > which it arrived)
> >
> 
> There is as such no mechanism to support double lookup on the same table
> for the packet. However, alternative approach could be-    first perform
> destination lookup and  invoke action handler ( in both table hit & miss case).
> In action handler, Src MAC + bridge ID key can be added to the table
> (rte_pipeline_table_entry_add) .  If a new entry is successfully added to the
> hash for the specified key, or there is already an entry in the hash for the
> specified key, then the position of the entry is returned. In this way, table
> entries will be updated with incoming packets.
> 

Yes, the two lookups are really one lookup and one add/update operation.

>
> > Thanks
> > -Avinash
> >
> > -----Original Message-----
> > From: Singh, Jasvinder [mailto:jasvinder.singh@intel.com]
> > Sent: Monday, August 17, 2015 7:00 AM
> > To: Richardson, Bruce; Yeddula, Avinash
> > Cc: dev@dpdk.org
> > Subject: RE: [dpdk-dev] [ 2nd try ] Lookup mechanim in DPDK HASH table.
> >
> >
> >
> > > -----Original Message-----
> > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Bruce
> Richardson
> > > Sent: Friday, August 14, 2015 10:25 AM
> > > To: Yeddula, Avinash
> > > Cc: dev@dpdk.org
> > > Subject: Re: [dpdk-dev] [ 2nd try ] Lookup mechanim in DPDK HASH table.
> > >
> > > On Thu, Aug 13, 2015 at 05:37:21PM -0400, Yeddula, Avinash wrote:
> > > > Any comments on this question ?
> > > >
> > > > Thanks
> > > > -Avinash
> > > >
> > > > -----Original Message-----
> > > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Yeddula,
> > > > Avinash
> > > > Sent: Wednesday, August 12, 2015 3:04 PM
> > > > To: dev@dpdk.org
> > > > Subject: [dpdk-dev] Lookup mechanim in DPDK HASH table.
> > > >
> > > > Hello All,
> > > >
> > > > I'm using DPDK extendable hash tables. This question is with respect
> > > > to the
> > > lookup aspect of the hash table.
> > > > I see that there is just one "t->key_offset" that is pre-defined for
> > > > the hash
> > > table. I also understand that the frame needs to carry the "lookup_key
> > > / keys" in the meta data.
> > > >
> > > > Here is my question:  How to support more than one lookup with
> > > > different
> > > keys on the same frame on the same table.
> > > > Use case: Src mac  lookup and dst mac lookup on the same mac table.
> > > >
> > > > Thanks
> > > > -Avinash
> > >
> > > Just to confirm: this is using the extensible bucket hash in the
> > > rte_table library of packet framework, rather than the standalone
> > > rte_hash library, right?
> > >
> > > /Bruce
> >
> > Could you share detail on the two different keys used for lookups. In case if
> > you are considering bidirectional packet flow between the source and
> > destination, symmetric hash can be used-
> > http://www.ndsl.kaist.edu/~kyoungsoo/papers/TR-symRSS.pdf
> >
> > Jasvinder

Re: [dpdk-dev] [ 2nd try ] Lookup mechanim in DPDK HASH table.

[Dumitrescu, Cristian]

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Yeddula, Avinash
> Sent: Thursday, August 13, 2015 10:37 PM
> To: dev@dpdk.org
> Subject: [dpdk-dev] [ 2nd try ] Lookup mechanim in DPDK HASH table.
> 
> Any comments on this question ?
> 
> Thanks
> -Avinash
> 

Sorry for my delay, Avinash, I was out of office for a few days.

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Yeddula, Avinash
> Sent: Wednesday, August 12, 2015 3:04 PM
> To: dev@dpdk.org
> Subject: [dpdk-dev] Lookup mechanim in DPDK HASH table.
> 
> Hello All,
> 
> I'm using DPDK extendable hash tables. This question is with respect to the
> lookup aspect of the hash table.
> I see that there is just one "t->key_offset" that is pre-defined for the hash
> table.

Just to avoid any confusion here implied by "pre-defined" statement, the key offset is definitely not hardcoded at build time; the key offset is configurable per each hash table instance when the hash table instance is initialized. Once set up for a particular hash table instance, it cannot be changed for that instance. Different hash table instances can have different values for this parameter.

> I also understand that the frame needs to carry the "lookup_key / keys" in the meta data.
> 
> Here is my question:  How to support more than one lookup with different
> keys on the same frame on the same table.

I agree with Venkat that one way of doing this is to do additional work of extracting the lookup key from the packet (which can have different format, depending on the point in the processing pipeline) and placing it at a fixed offset within the packet meta-data. This work can be done by: 
- the action handler defined for the input ports of the same pipeline that current table is part of
- the action handler of a table in the same pipeline (located before the current table)
- an action handler (of input port/output port/table) from a different pipeline instance (located before the current pipeline in the processing chain)

I also agree with Mike Bly this is not the best way of doing things, and I don't think you actually need it, based on your use-case. Please see below for my suggestion.

> Use case: Src mac  lookup and dst mac lookup on the same mac table.

Your use-case looks like classical L2 bridging. My understanding is: 
- you need to lookup the MAC destination in the MAC forwarding table: on lookup hit, unicast forward the frame on the port read from the hit table entry; on lookup hit, broadcast/flood the frame on all ports. 
- you also need to learn the MAC source, i.e. make sure you add the MAC source to the same MAC forwarding table to make the association of MAC address and the port where it is located for future lookups. As Jasvinder is pointing out, you do not really need to do a lookup of the MAC source in the table, what you need is to add the MAC source to the table.

So one suggestion is to:
- have a single lookup operation in the MAC forwarding table (based on MAC destination)
- have the table action handler (or the input port action handler, or the output port action handler) to perform the add operation to the MAC forwarding table (add the MAC source as a new key to the table); the add operation is really an add/update, meaning that when the key is already present in the table, only the data associated with the key (i.e. the port where to forward the frame) is modified, which can be handy to pick up automatically the corner case of one station being moved from port A to port B (so one MAC address that previously showed up as being sourced on port A is not sourced by port B)

You can also optimize things a bit to reduce the rate of add operations to the table, so you don't need to perform an add operation per frame:
-have single lookup operation to table 1( MAC forwarding table), using the MAC destination as the lookup key
-have single lookup operation to table 2 (MAC learning cache table, which can be a small LRU-based table used to record the most frequently MAC addresses encountered lately), using the MAC source as the lookup key: only add the current MAC source to table 1 (MAC forwarding table) on lookup miss in table 2

I am sure that other people have even better ideas for optimizations.

> 
> Thanks
> -Avinash