[dpdk-dev] DPDK hash function related question

[dpdk-dev] DPDK hash function related question

[Yeddula, Avinash]

Hello All,

I'm DPDK extensible bucket hash in the rte_table library of packet framework. My question is related to the actual hash function that computes the hash signature.

All the available examples have initialized it to test_hash.   I do not see any hash function available in rte_table library , that computes the actual signature



struct rte_table_hash_ext_params   hash_table_params = {

    .key_size = TABLE_ENTRY_KEY_SIZE,

    .n_keys = TABLE_MAX_SIZE,

    .n_buckets = TABLE_MAX_BUCKET_COUNT,

    .n_buckets_ext = TABLE_MAX_EXT_BUCKET_COUNT,

    .f_hash = test_hash,

    .seed = 0,

    .signature_offset = 0;

    .key_offset = __builtin_offsetof(struct metadata_t, tbl_key),

};



So, I wanted to use hash functions from DPDK rte_hash library. This is what I'm doing and looking at the code this looks ok to me.

I'm at least a week or 2 away from testing this part of the code. I wanted to confirm that, there is no fundamental flaw in using the DPDK rte_hash library and rte_table library like this. Could someone confirm this please ?



#define DEFAULT_HASH_FUNC rte_hash_crc



struct rte_table_hash_ext_params   hash_table_params = {

    .key_size = TABLE_ENTRY_KEY_SIZE,

    .n_keys = TABLE_MAX_SIZE,

    .n_buckets = TABLE_MAX_BUCKET_COUNT,

    .n_buckets_ext = TABLE_MAX_EXT_BUCKET_COUNT,

    .f_hash = DEFAULT_HASH_FUNC ,

    .seed = 0,

    .signature_offset = 0;

    .key_offset = __builtin_offsetof(struct metadata_t, tbl_key),

};



Thanks

-Avinash

Re: [dpdk-dev] DPDK hash function related question

[Dumitrescu, Cristian]

Hi Avinash,

Yes, the hash function is configurable.

Are you using a DPDK release older than 2.1? In DPDK we moved away from test_hash to CRC-based hashes. Please take a look at DPDK release 2.1 examples/ip_pipeline application: in pipeline_flow_classification_be.c, we use CRC-based hash functions defined in file hash_func.h from the same folder.

Regards,
Cristian

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Yeddula, Avinash
> Sent: Tuesday, September 22, 2015 1:34 AM
> To: dev@dpdk.org; Bly, Mike
> Subject: [dpdk-dev] DPDK hash function related question
> 
> Hello All,
> 
> I'm DPDK extensible bucket hash in the rte_table library of packet
> framework. My question is related to the actual hash function that computes
> the hash signature.
> 
> All the available examples have initialized it to test_hash.   I do not see any
> hash function available in rte_table library , that computes the actual
> signature
> 
> 
> 
> struct rte_table_hash_ext_params   hash_table_params = {
> 
>     .key_size = TABLE_ENTRY_KEY_SIZE,
> 
>     .n_keys = TABLE_MAX_SIZE,
> 
>     .n_buckets = TABLE_MAX_BUCKET_COUNT,
> 
>     .n_buckets_ext = TABLE_MAX_EXT_BUCKET_COUNT,
> 
>     .f_hash = test_hash,
> 
>     .seed = 0,
> 
>     .signature_offset = 0;
> 
>     .key_offset = __builtin_offsetof(struct metadata_t, tbl_key),
> 
> };
> 
> 
> 
> So, I wanted to use hash functions from DPDK rte_hash library. This is what
> I'm doing and looking at the code this looks ok to me.
> 
> I'm at least a week or 2 away from testing this part of the code. I wanted to
> confirm that, there is no fundamental flaw in using the DPDK rte_hash library
> and rte_table library like this. Could someone confirm this please ?
> 
> 
> 
> #define DEFAULT_HASH_FUNC rte_hash_crc
> 
> 
> 
> struct rte_table_hash_ext_params   hash_table_params = {
> 
>     .key_size = TABLE_ENTRY_KEY_SIZE,
> 
>     .n_keys = TABLE_MAX_SIZE,
> 
>     .n_buckets = TABLE_MAX_BUCKET_COUNT,
> 
>     .n_buckets_ext = TABLE_MAX_EXT_BUCKET_COUNT,
> 
>     .f_hash = DEFAULT_HASH_FUNC ,
> 
>     .seed = 0,
> 
>     .signature_offset = 0;
> 
>     .key_offset = __builtin_offsetof(struct metadata_t, tbl_key),
> 
> };
> 
> 
> 
> Thanks
> 
> -Avinash
> 

Re: [dpdk-dev] DPDK hash function related question

[Thomas Monjalon]

2015-09-22 10:05, Dumitrescu, Cristian:
> Are you using a DPDK release older than 2.1?
> In DPDK we moved away from test_hash to CRC-based hashes.
> Please take a look at DPDK release 2.1 examples/ip_pipeline application:
> in pipeline_flow_classification_be.c, we use CRC-based hash functions
> defined in file hash_func.h from the same folder.

Has it already been said that these functions should be hosted elsewhere?
e.g in librte_hash ;)

Re: [dpdk-dev] DPDK hash function related question

[Dumitrescu, Cristian]

> -----Original Message-----
> From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com]
> Sent: Tuesday, September 22, 2015 4:41 PM
> To: Dumitrescu, Cristian
> Cc: dev@dpdk.org; Yeddula, Avinash; Bly, Mike
> Subject: Re: [dpdk-dev] DPDK hash function related question
> 
> 2015-09-22 10:05, Dumitrescu, Cristian:
> > Are you using a DPDK release older than 2.1?
> > In DPDK we moved away from test_hash to CRC-based hashes.
> > Please take a look at DPDK release 2.1 examples/ip_pipeline application:
> > in pipeline_flow_classification_be.c, we use CRC-based hash functions
> > defined in file hash_func.h from the same folder.
> 
> Has it already been said that these functions should be hosted elsewhere?
> e.g in librte_hash ;)

It is fine with me. Pablo, feel free to look at these functions and move them to librte_hash if useful.

We are looking to augment these functions to with some more for symmetric hashing during 2.2 (hopefully) or 2.3 development, might be better to wait for this until that time for a more comprehensible solution.

Regards,
Cristian

Re: [dpdk-dev] DPDK hash function related question

[Yeddula, Avinash]

Hi Cristian,
I have configured the hash function and it compile fine with "warnings". Since librte_hash vs librte_table is 32bit vs 64bit.

librte_hash library :
/** Type of function that can be used for calculating the hash value. */
typedef uint32_t (*rte_hash_function)(const void *key, uint32_t key_len, uint32_t init_val);

librte_table library:
typedef uint64_t (*rte_table_hash_op_hash) (void *key,	uint32_t key_size, uint64_t seed);

I could use one of these hash functions. This is one option, but our first priority is  to use crc hash or cukoo hash.
https://github.com/scylladb/dpdk/blob/master/examples/ip_pipeline/pipeline/hash_func.h

We do not want to have those warning in our code. What do you suggest ?

Thanks
-Avinash

-----Original Message-----
From: Dumitrescu, Cristian [mailto:cristian.dumitrescu@intel.com] 
Sent: Tuesday, September 22, 2015 3:05 AM
To: Yeddula, Avinash; dev@dpdk.org; Bly, Mike
Subject: RE: DPDK hash function related question

Hi Avinash,

Yes, the hash function is configurable.

Are you using a DPDK release older than 2.1? In DPDK we moved away from test_hash to CRC-based hashes. Please take a look at DPDK release 2.1 examples/ip_pipeline application: in pipeline_flow_classification_be.c, we use CRC-based hash functions defined in file hash_func.h from the same folder.

Regards,
Cristian

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Yeddula, Avinash
> Sent: Tuesday, September 22, 2015 1:34 AM
> To: dev@dpdk.org; Bly, Mike
> Subject: [dpdk-dev] DPDK hash function related question
> 
> Hello All,
> 
> I'm DPDK extensible bucket hash in the rte_table library of packet 
> framework. My question is related to the actual hash function that 
> computes the hash signature.
> 
> All the available examples have initialized it to test_hash.   I do not see any
> hash function available in rte_table library , that computes the 
> actual signature
> 
> 
> 
> struct rte_table_hash_ext_params   hash_table_params = {
> 
>     .key_size = TABLE_ENTRY_KEY_SIZE,
> 
>     .n_keys = TABLE_MAX_SIZE,
> 
>     .n_buckets = TABLE_MAX_BUCKET_COUNT,
> 
>     .n_buckets_ext = TABLE_MAX_EXT_BUCKET_COUNT,
> 
>     .f_hash = test_hash,
> 
>     .seed = 0,
> 
>     .signature_offset = 0;
> 
>     .key_offset = __builtin_offsetof(struct metadata_t, tbl_key),
> 
> };
> 
> 
> 
> So, I wanted to use hash functions from DPDK rte_hash library. This is 
> what I'm doing and looking at the code this looks ok to me.
> 
> I'm at least a week or 2 away from testing this part of the code. I 
> wanted to confirm that, there is no fundamental flaw in using the DPDK 
> rte_hash library and rte_table library like this. Could someone confirm this please ?
> 
> 
> 
> #define DEFAULT_HASH_FUNC rte_hash_crc
> 
> 
> 
> struct rte_table_hash_ext_params   hash_table_params = {
> 
>     .key_size = TABLE_ENTRY_KEY_SIZE,
> 
>     .n_keys = TABLE_MAX_SIZE,
> 
>     .n_buckets = TABLE_MAX_BUCKET_COUNT,
> 
>     .n_buckets_ext = TABLE_MAX_EXT_BUCKET_COUNT,
> 
>     .f_hash = DEFAULT_HASH_FUNC ,
> 
>     .seed = 0,
> 
>     .signature_offset = 0;
> 
>     .key_offset = __builtin_offsetof(struct metadata_t, tbl_key),
> 
> };
> 
> 
> 
> Thanks
> 
> -Avinash
> 

Re: [dpdk-dev] DPDK hash function related question

[De Lara Guarch, Pablo]

Hi Avinash,

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Yeddula, Avinash
> Sent: Monday, October 12, 2015 6:03 PM
> To: Dumitrescu, Cristian; dev@dpdk.org; Bly, Mike
> Subject: Re: [dpdk-dev] DPDK hash function related question
> 
> Hi Cristian,
> I have configured the hash function and it compile fine with "warnings". Since
> librte_hash vs librte_table is 32bit vs 64bit.
> 
> librte_hash library :
> /** Type of function that can be used for calculating the hash value. */
> typedef uint32_t (*rte_hash_function)(const void *key, uint32_t key_len,
> uint32_t init_val);
> 
> librte_table library:
> typedef uint64_t (*rte_table_hash_op_hash) (void *key,	uint32_t
> key_size, uint64_t seed);
> 
> I could use one of these hash functions. This is one option, but our first
> priority is  to use crc hash or cukoo hash.

Mind that cuckoo hash is not a hash function, but a method for resolving collisions in a hash table.

> https://github.com/scylladb/dpdk/blob/master/examples/ip_pipeline/pipeli
> ne/hash_func.h
> 
> We do not want to have those warning in our code. What do you suggest ?
> 
> Thanks
> -Avinash
> 
> -----Original Message-----
> From: Dumitrescu, Cristian [mailto:cristian.dumitrescu@intel.com]
> Sent: Tuesday, September 22, 2015 3:05 AM
> To: Yeddula, Avinash; dev@dpdk.org; Bly, Mike
> Subject: RE: DPDK hash function related question
> 
> Hi Avinash,
> 
> Yes, the hash function is configurable.
> 
> Are you using a DPDK release older than 2.1? In DPDK we moved away from
> test_hash to CRC-based hashes. Please take a look at DPDK release 2.1
> examples/ip_pipeline application: in pipeline_flow_classification_be.c, we
> use CRC-based hash functions defined in file hash_func.h from the same
> folder.
> 
> Regards,
> Cristian
> 
> > -----Original Message-----
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Yeddula, Avinash
> > Sent: Tuesday, September 22, 2015 1:34 AM
> > To: dev@dpdk.org; Bly, Mike
> > Subject: [dpdk-dev] DPDK hash function related question
> >
> > Hello All,
> >
> > I'm DPDK extensible bucket hash in the rte_table library of packet
> > framework. My question is related to the actual hash function that
> > computes the hash signature.
> >
> > All the available examples have initialized it to test_hash.   I do not see any
> > hash function available in rte_table library , that computes the
> > actual signature
> >
> >
> >
> > struct rte_table_hash_ext_params   hash_table_params = {
> >
> >     .key_size = TABLE_ENTRY_KEY_SIZE,
> >
> >     .n_keys = TABLE_MAX_SIZE,
> >
> >     .n_buckets = TABLE_MAX_BUCKET_COUNT,
> >
> >     .n_buckets_ext = TABLE_MAX_EXT_BUCKET_COUNT,
> >
> >     .f_hash = test_hash,
> >
> >     .seed = 0,
> >
> >     .signature_offset = 0;
> >
> >     .key_offset = __builtin_offsetof(struct metadata_t, tbl_key),
> >
> > };
> >
> >
> >
> > So, I wanted to use hash functions from DPDK rte_hash library. This is
> > what I'm doing and looking at the code this looks ok to me.
> >
> > I'm at least a week or 2 away from testing this part of the code. I
> > wanted to confirm that, there is no fundamental flaw in using the DPDK
> > rte_hash library and rte_table library like this. Could someone confirm this
> please ?
> >
> >
> >
> > #define DEFAULT_HASH_FUNC rte_hash_crc
> >
> >
> >
> > struct rte_table_hash_ext_params   hash_table_params = {
> >
> >     .key_size = TABLE_ENTRY_KEY_SIZE,
> >
> >     .n_keys = TABLE_MAX_SIZE,
> >
> >     .n_buckets = TABLE_MAX_BUCKET_COUNT,
> >
> >     .n_buckets_ext = TABLE_MAX_EXT_BUCKET_COUNT,
> >
> >     .f_hash = DEFAULT_HASH_FUNC ,
> >
> >     .seed = 0,
> >
> >     .signature_offset = 0;
> >
> >     .key_offset = __builtin_offsetof(struct metadata_t, tbl_key),
> >
> > };
> >
> >
> >
> > Thanks
> >
> > -Avinash
> >
> 

Re: [dpdk-dev] DPDK hash function related question

[Dumitrescu, Cristian]

> -----Original Message-----
> From: Yeddula, Avinash [mailto:ayeddula@ciena.com]
> Sent: Monday, October 12, 2015 6:03 PM
> To: Dumitrescu, Cristian; dev@dpdk.org; Bly, Mike
> Subject: RE: DPDK hash function related question
> 
> Hi Cristian,
> I have configured the hash function and it compile fine with "warnings". Since
> librte_hash vs librte_table is 32bit vs 64bit.
> 
> librte_hash library :
> /** Type of function that can be used for calculating the hash value. */
> typedef uint32_t (*rte_hash_function)(const void *key, uint32_t key_len,
> uint32_t init_val);
> 
> librte_table library:
> typedef uint64_t (*rte_table_hash_op_hash) (void *key,	uint32_t
> key_size, uint64_t seed);
> 
> I could use one of these hash functions. This is one option, but our first
> priority is  to use crc hash or cukoo hash.
> https://github.com/scylladb/dpdk/blob/master/examples/ip_pipeline/pipeli
> ne/hash_func.h
> 
> We do not want to have those warning in our code. What do you suggest ?

Would function pointer conversion work?

> 
> Thanks
> -Avinash
> 
> -----Original Message-----
> From: Dumitrescu, Cristian [mailto:cristian.dumitrescu@intel.com]
> Sent: Tuesday, September 22, 2015 3:05 AM
> To: Yeddula, Avinash; dev@dpdk.org; Bly, Mike
> Subject: RE: DPDK hash function related question
> 
> Hi Avinash,
> 
> Yes, the hash function is configurable.
> 
> Are you using a DPDK release older than 2.1? In DPDK we moved away from
> test_hash to CRC-based hashes. Please take a look at DPDK release 2.1
> examples/ip_pipeline application: in pipeline_flow_classification_be.c, we
> use CRC-based hash functions defined in file hash_func.h from the same
> folder.
> 
> Regards,
> Cristian
> 
> > -----Original Message-----
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Yeddula, Avinash
> > Sent: Tuesday, September 22, 2015 1:34 AM
> > To: dev@dpdk.org; Bly, Mike
> > Subject: [dpdk-dev] DPDK hash function related question
> >
> > Hello All,
> >
> > I'm DPDK extensible bucket hash in the rte_table library of packet
> > framework. My question is related to the actual hash function that
> > computes the hash signature.
> >
> > All the available examples have initialized it to test_hash.   I do not see any
> > hash function available in rte_table library , that computes the
> > actual signature
> >
> >
> >
> > struct rte_table_hash_ext_params   hash_table_params = {
> >
> >     .key_size = TABLE_ENTRY_KEY_SIZE,
> >
> >     .n_keys = TABLE_MAX_SIZE,
> >
> >     .n_buckets = TABLE_MAX_BUCKET_COUNT,
> >
> >     .n_buckets_ext = TABLE_MAX_EXT_BUCKET_COUNT,
> >
> >     .f_hash = test_hash,
> >
> >     .seed = 0,
> >
> >     .signature_offset = 0;
> >
> >     .key_offset = __builtin_offsetof(struct metadata_t, tbl_key),
> >
> > };
> >
> >
> >
> > So, I wanted to use hash functions from DPDK rte_hash library. This is
> > what I'm doing and looking at the code this looks ok to me.
> >
> > I'm at least a week or 2 away from testing this part of the code. I
> > wanted to confirm that, there is no fundamental flaw in using the DPDK
> > rte_hash library and rte_table library like this. Could someone confirm this
> please ?
> >
> >
> >
> > #define DEFAULT_HASH_FUNC rte_hash_crc
> >
> >
> >
> > struct rte_table_hash_ext_params   hash_table_params = {
> >
> >     .key_size = TABLE_ENTRY_KEY_SIZE,
> >
> >     .n_keys = TABLE_MAX_SIZE,
> >
> >     .n_buckets = TABLE_MAX_BUCKET_COUNT,
> >
> >     .n_buckets_ext = TABLE_MAX_EXT_BUCKET_COUNT,
> >
> >     .f_hash = DEFAULT_HASH_FUNC ,
> >
> >     .seed = 0,
> >
> >     .signature_offset = 0;
> >
> >     .key_offset = __builtin_offsetof(struct metadata_t, tbl_key),
> >
> > };
> >
> >
> >
> > Thanks
> >
> > -Avinash
> >
>