[dpdk-dev] [PATCH] hash: fix memcmp function pointer in multi-process

[dpdk-dev] [PATCH] hash: fix memcmp function pointer in multi-process

[Dhananjaya Reddy Eadala]

Hi

We found a problem in dpdk-2.2 using under multi-process environment.
Here is the brief description how we are using the dpdk:

We have two processes proc1, proc2 using dpdk. These proc1 and proc2 are
two different compiled binaries.
proc1 is started as primary process and proc2 as secondary process.

proc1:
Calls srcHash = rte_hash_create("src_hash_name") to create rte_hash
structure.
As part of this, this api initalized the rte_hash structure and set the
srcHash->rte_hash_cmp_eq to the address of memcmp() from proc1 address
space.

proc2:
calls srcHash =  rte_hash_find_existing("src_hash_name"). This returns the
rte_hash created by proc1.
This srcHash->rte_hash_cmp_eq still points to the address of memcmp() from
proc1 address space.
Later proc2  calls rte_hash_lookup_with_hash(srcHash, (const void*) &key,
key.sig);
Under the hood, rte_hash_lookup_with_hash() invokes
__rte_hash_lookup_with_hash(), which in turn calls h->rte_hash_cmp_eq(key,
k->key, h->key_len).
This leads to a crash as h->rte_hash_cmp_eq is an address from proc1
address space and is invalid address in proc2 address space.

We found, from dpdk documentation, that
"
 The use of function pointers between multiple processes running based of
different compiled
 binaries is not supported, since the location of a given function in one
process may be different to
 its location in a second. This prevents the librte_hash library from
behaving properly as in a  multi-
 threaded instance, since it uses a pointer to the hash function internally.


 To work around this issue, it is recommended that multi-process
applications perform the hash
 calculations by directly calling the hashing function from the code and
then using the
 rte_hash_add_with_hash()/rte_hash_lookup_with_hash() functions instead of
the functions which do
 the hashing internally, such as rte_hash_add()/rte_hash_lookup().
"

We did follow the recommended steps by invoking rte_hash_lookup_with_hash().
It was no issue up to and including dpdk-2.0. In later releases started
crashing because rte_hash_cmp_eq is introduced in dpdk-2.1

We fixed it with the following patch and would like to submit the patch to
dpdk.org.
Patch is created such that, if anyone wanted to use dpdk in multi-process
environment with function pointers not shared, they need to
define RTE_LIB_MP_NO_FUNC_PTR in their Makefile. Without defining this flag
in Makefile, it works as it is now.


Please find here attached is the patch file.

Thanks
Dhana

Re: [dpdk-dev] [PATCH] hash: fix memcmp function pointer in multi-process

[Qiu, Michael]

On 3/2/2016 2:57 AM, Dhananjaya Reddy Eadala wrote:
> Hi
>
> We found a problem in dpdk-2.2 using under multi-process environment.
> Here is the brief description how we are using the dpdk:
>
> We have two processes proc1, proc2 using dpdk. These proc1 and proc2 are
> two different compiled binaries.
> proc1 is started as primary process and proc2 as secondary process.
>
> proc1:
> Calls srcHash = rte_hash_create("src_hash_name") to create rte_hash
> structure.
> As part of this, this api initalized the rte_hash structure and set the
> srcHash->rte_hash_cmp_eq to the address of memcmp() from proc1 address
> space.
>
> proc2:
> calls srcHash =  rte_hash_find_existing("src_hash_name"). This returns the
> rte_hash created by proc1.
> This srcHash->rte_hash_cmp_eq still points to the address of memcmp() from
> proc1 address space.
> Later proc2  calls rte_hash_lookup_with_hash(srcHash, (const void*) &key,
> key.sig);
> Under the hood, rte_hash_lookup_with_hash() invokes
> __rte_hash_lookup_with_hash(), which in turn calls h->rte_hash_cmp_eq(key,
> k->key, h->key_len).
> This leads to a crash as h->rte_hash_cmp_eq is an address from proc1
> address space and is invalid address in proc2 address space.
>
> We found, from dpdk documentation, that
> "
>  The use of function pointers between multiple processes running based of
> different compiled
>  binaries is not supported, since the location of a given function in one
> process may be different to
>  its location in a second. This prevents the librte_hash library from
> behaving properly as in a  multi-
>  threaded instance, since it uses a pointer to the hash function internally.
>
>
>  To work around this issue, it is recommended that multi-process
> applications perform the hash
>  calculations by directly calling the hashing function from the code and
> then using the
>  rte_hash_add_with_hash()/rte_hash_lookup_with_hash() functions instead of
> the functions which do
>  the hashing internally, such as rte_hash_add()/rte_hash_lookup().
> "
>
> We did follow the recommended steps by invoking rte_hash_lookup_with_hash().
> It was no issue up to and including dpdk-2.0. In later releases started
> crashing because rte_hash_cmp_eq is introduced in dpdk-2.1
>
> We fixed it with the following patch and would like to submit the patch to
> dpdk.org.

Could you send the patch in the mail?

Learn how to send a patch:

http://www.dpdk.org/dev

Thanks,
Michael
> Patch is created such that, if anyone wanted to use dpdk in multi-process
> environment with function pointers not shared, they need to
> define RTE_LIB_MP_NO_FUNC_PTR in their Makefile. Without defining this flag
> in Makefile, it works as it is now.
>
>
> Please find here attached is the patch file.
>
> Thanks
> Dhana
>

Re: [dpdk-dev] [PATCH] hash: fix memcmp function pointer in multi-process

[Dhananjaya Reddy Eadala]

Michael

Please find the attached is the patch file generated from "git format-patch
-1"


Thanks
Dhana


On Tue, Mar 1, 2016 at 9:29 PM, Qiu, Michael <michael.qiu@intel.com> wrote:

> On 3/2/2016 2:57 AM, Dhananjaya Reddy Eadala wrote:
> > Hi
> >
> > We found a problem in dpdk-2.2 using under multi-process environment.
> > Here is the brief description how we are using the dpdk:
> >
> > We have two processes proc1, proc2 using dpdk. These proc1 and proc2 are
> > two different compiled binaries.
> > proc1 is started as primary process and proc2 as secondary process.
> >
> > proc1:
> > Calls srcHash = rte_hash_create("src_hash_name") to create rte_hash
> > structure.
> > As part of this, this api initalized the rte_hash structure and set the
> > srcHash->rte_hash_cmp_eq to the address of memcmp() from proc1 address
> > space.
> >
> > proc2:
> > calls srcHash =  rte_hash_find_existing("src_hash_name"). This returns
> the
> > rte_hash created by proc1.
> > This srcHash->rte_hash_cmp_eq still points to the address of memcmp()
> from
> > proc1 address space.
> > Later proc2  calls rte_hash_lookup_with_hash(srcHash, (const void*) &key,
> > key.sig);
> > Under the hood, rte_hash_lookup_with_hash() invokes
> > __rte_hash_lookup_with_hash(), which in turn calls
> h->rte_hash_cmp_eq(key,
> > k->key, h->key_len).
> > This leads to a crash as h->rte_hash_cmp_eq is an address from proc1
> > address space and is invalid address in proc2 address space.
> >
> > We found, from dpdk documentation, that
> > "
> >  The use of function pointers between multiple processes running based of
> > different compiled
> >  binaries is not supported, since the location of a given function in one
> > process may be different to
> >  its location in a second. This prevents the librte_hash library from
> > behaving properly as in a  multi-
> >  threaded instance, since it uses a pointer to the hash function
> internally.
> >
> >
> >  To work around this issue, it is recommended that multi-process
> > applications perform the hash
> >  calculations by directly calling the hashing function from the code and
> > then using the
> >  rte_hash_add_with_hash()/rte_hash_lookup_with_hash() functions instead
> of
> > the functions which do
> >  the hashing internally, such as rte_hash_add()/rte_hash_lookup().
> > "
> >
> > We did follow the recommended steps by invoking
> rte_hash_lookup_with_hash().
> > It was no issue up to and including dpdk-2.0. In later releases started
> > crashing because rte_hash_cmp_eq is introduced in dpdk-2.1
> >
> > We fixed it with the following patch and would like to submit the patch
> to
> > dpdk.org.
>
> Could you send the patch in the mail?
>
> Learn how to send a patch:
>
> http://www.dpdk.org/dev
>
> Thanks,
> Michael
> > Patch is created such that, if anyone wanted to use dpdk in multi-process
> > environment with function pointers not shared, they need to
> > define RTE_LIB_MP_NO_FUNC_PTR in their Makefile. Without defining this
> flag
> > in Makefile, it works as it is now.
> >
> >
> > Please find here attached is the patch file.
> >
> > Thanks
> > Dhana
> >
>
>

Re: [dpdk-dev] [PATCH] hash: fix memcmp function pointer in multi-process

[De Lara Guarch, Pablo]

Hi Dhana,

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Dhananjaya Reddy
> Eadala
> Sent: Wednesday, March 02, 2016 2:02 PM
> To: Qiu, Michael
> Cc: dev@dpdk.org
> Subject: Re: [dpdk-dev] [PATCH] hash: fix memcmp function pointer in multi-
> process
> 
> Michael
> 
> Please find the attached is the patch file generated from "git format-patch
> -1"

What Michael means is that you should use git send-email and not attach the patch
to the mail.

You can use:

git send-email -1 --to dev@dpdk.org

or

git send-email your-patch.patch --to dev@dpdk.org

Take a look at dpdk.org/dev for more details and how to configure git with your smtp server details.

Thanks,
Pablo

> 
> 
> Thanks
> Dhana
> 
> 
> On Tue, Mar 1, 2016 at 9:29 PM, Qiu, Michael <michael.qiu@intel.com>
> wrote:
> 
> > On 3/2/2016 2:57 AM, Dhananjaya Reddy Eadala wrote:
> > > Hi
> > >
> > > We found a problem in dpdk-2.2 using under multi-process environment.
> > > Here is the brief description how we are using the dpdk:
> > >
> > > We have two processes proc1, proc2 using dpdk. These proc1 and proc2
> are
> > > two different compiled binaries.
> > > proc1 is started as primary process and proc2 as secondary process.
> > >
> > > proc1:
> > > Calls srcHash = rte_hash_create("src_hash_name") to create rte_hash
> > > structure.
> > > As part of this, this api initalized the rte_hash structure and set the
> > > srcHash->rte_hash_cmp_eq to the address of memcmp() from proc1
> address
> > > space.
> > >
> > > proc2:
> > > calls srcHash =  rte_hash_find_existing("src_hash_name"). This returns
> > the
> > > rte_hash created by proc1.
> > > This srcHash->rte_hash_cmp_eq still points to the address of memcmp()
> > from
> > > proc1 address space.
> > > Later proc2  calls rte_hash_lookup_with_hash(srcHash, (const void*)
> &key,
> > > key.sig);
> > > Under the hood, rte_hash_lookup_with_hash() invokes
> > > __rte_hash_lookup_with_hash(), which in turn calls
> > h->rte_hash_cmp_eq(key,
> > > k->key, h->key_len).
> > > This leads to a crash as h->rte_hash_cmp_eq is an address from proc1
> > > address space and is invalid address in proc2 address space.
> > >
> > > We found, from dpdk documentation, that
> > > "
> > >  The use of function pointers between multiple processes running based
> of
> > > different compiled
> > >  binaries is not supported, since the location of a given function in one
> > > process may be different to
> > >  its location in a second. This prevents the librte_hash library from
> > > behaving properly as in a  multi-
> > >  threaded instance, since it uses a pointer to the hash function
> > internally.
> > >
> > >
> > >  To work around this issue, it is recommended that multi-process
> > > applications perform the hash
> > >  calculations by directly calling the hashing function from the code and
> > > then using the
> > >  rte_hash_add_with_hash()/rte_hash_lookup_with_hash() functions
> instead
> > of
> > > the functions which do
> > >  the hashing internally, such as rte_hash_add()/rte_hash_lookup().
> > > "
> > >
> > > We did follow the recommended steps by invoking
> > rte_hash_lookup_with_hash().
> > > It was no issue up to and including dpdk-2.0. In later releases started
> > > crashing because rte_hash_cmp_eq is introduced in dpdk-2.1
> > >
> > > We fixed it with the following patch and would like to submit the patch
> > to
> > > dpdk.org.
> >
> > Could you send the patch in the mail?
> >
> > Learn how to send a patch:
> >
> > http://www.dpdk.org/dev
> >
> > Thanks,
> > Michael
> > > Patch is created such that, if anyone wanted to use dpdk in multi-process
> > > environment with function pointers not shared, they need to
> > > define RTE_LIB_MP_NO_FUNC_PTR in their Makefile. Without defining
> this
> > flag
> > > in Makefile, it works as it is now.
> > >
> > >
> > > Please find here attached is the patch file.
> > >
> > > Thanks
> > > Dhana
> > >
> >
> >

Re: [dpdk-dev] [PATCH] hash: fix memcmp function pointer in multi-process

[Dhananjaya Reddy Eadala]

Hi Pablo

Thanks for the clarification. I just sent the patch using "git send-email".
Probably that will show up as different mail thread.

Thanks
Dhana

On Wed, Mar 2, 2016 at 9:22 AM, De Lara Guarch, Pablo <
pablo.de.lara.guarch@intel.com> wrote:

> Hi Dhana,
>
> > -----Original Message-----
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Dhananjaya Reddy
> > Eadala
> > Sent: Wednesday, March 02, 2016 2:02 PM
> > To: Qiu, Michael
> > Cc: dev@dpdk.org
> > Subject: Re: [dpdk-dev] [PATCH] hash: fix memcmp function pointer in
> multi-
> > process
> >
> > Michael
> >
> > Please find the attached is the patch file generated from "git
> format-patch
> > -1"
>
> What Michael means is that you should use git send-email and not attach
> the patch
> to the mail.
>
> You can use:
>
> git send-email -1 --to dev@dpdk.org
>
> or
>
> git send-email your-patch.patch --to dev@dpdk.org
>
> Take a look at dpdk.org/dev for more details and how to configure git
> with your smtp server details.
>
> Thanks,
> Pablo
>
> >
> >
> > Thanks
> > Dhana
> >
> >
> > On Tue, Mar 1, 2016 at 9:29 PM, Qiu, Michael <michael.qiu@intel.com>
> > wrote:
> >
> > > On 3/2/2016 2:57 AM, Dhananjaya Reddy Eadala wrote:
> > > > Hi
> > > >
> > > > We found a problem in dpdk-2.2 using under multi-process environment.
> > > > Here is the brief description how we are using the dpdk:
> > > >
> > > > We have two processes proc1, proc2 using dpdk. These proc1 and proc2
> > are
> > > > two different compiled binaries.
> > > > proc1 is started as primary process and proc2 as secondary process.
> > > >
> > > > proc1:
> > > > Calls srcHash = rte_hash_create("src_hash_name") to create rte_hash
> > > > structure.
> > > > As part of this, this api initalized the rte_hash structure and set
> the
> > > > srcHash->rte_hash_cmp_eq to the address of memcmp() from proc1
> > address
> > > > space.
> > > >
> > > > proc2:
> > > > calls srcHash =  rte_hash_find_existing("src_hash_name"). This
> returns
> > > the
> > > > rte_hash created by proc1.
> > > > This srcHash->rte_hash_cmp_eq still points to the address of memcmp()
> > > from
> > > > proc1 address space.
> > > > Later proc2  calls rte_hash_lookup_with_hash(srcHash, (const void*)
> > &key,
> > > > key.sig);
> > > > Under the hood, rte_hash_lookup_with_hash() invokes
> > > > __rte_hash_lookup_with_hash(), which in turn calls
> > > h->rte_hash_cmp_eq(key,
> > > > k->key, h->key_len).
> > > > This leads to a crash as h->rte_hash_cmp_eq is an address from proc1
> > > > address space and is invalid address in proc2 address space.
> > > >
> > > > We found, from dpdk documentation, that
> > > > "
> > > >  The use of function pointers between multiple processes running
> based
> > of
> > > > different compiled
> > > >  binaries is not supported, since the location of a given function
> in one
> > > > process may be different to
> > > >  its location in a second. This prevents the librte_hash library from
> > > > behaving properly as in a  multi-
> > > >  threaded instance, since it uses a pointer to the hash function
> > > internally.
> > > >
> > > >
> > > >  To work around this issue, it is recommended that multi-process
> > > > applications perform the hash
> > > >  calculations by directly calling the hashing function from the code
> and
> > > > then using the
> > > >  rte_hash_add_with_hash()/rte_hash_lookup_with_hash() functions
> > instead
> > > of
> > > > the functions which do
> > > >  the hashing internally, such as rte_hash_add()/rte_hash_lookup().
> > > > "
> > > >
> > > > We did follow the recommended steps by invoking
> > > rte_hash_lookup_with_hash().
> > > > It was no issue up to and including dpdk-2.0. In later releases
> started
> > > > crashing because rte_hash_cmp_eq is introduced in dpdk-2.1
> > > >
> > > > We fixed it with the following patch and would like to submit the
> patch
> > > to
> > > > dpdk.org.
> > >
> > > Could you send the patch in the mail?
> > >
> > > Learn how to send a patch:
> > >
> > > http://www.dpdk.org/dev
> > >
> > > Thanks,
> > > Michael
> > > > Patch is created such that, if anyone wanted to use dpdk in
> multi-process
> > > > environment with function pointers not shared, they need to
> > > > define RTE_LIB_MP_NO_FUNC_PTR in their Makefile. Without defining
> > this
> > > flag
> > > > in Makefile, it works as it is now.
> > > >
> > > >
> > > > Please find here attached is the patch file.
> > > >
> > > > Thanks
> > > > Dhana
> > > >
> > >
> > >
>