[dpdk-dev] free a memzone

[dpdk-dev] free a memzone

[Mahdi Dashtbozorgi]

Hi,

I have two processes, which uses DPDK multi-process feature to communicate.
Master process captures packets from NIC and put them to a ring buffer,
which is shared between master and slave process.
The slave process looks up the shared ring buffer using rte_ring_lookup
function and reads the packets.
The slave process needs a memory pool, too. Therefore, it creates a mempool
using rte_mempool_create. But If the slave process crashes during its
processing and runs again, rte_mempool_create function fails and tells that
there is a memory zone with that name.
If I use rte_mempool_lookup in this case, the memory pool is not a clean
memory pool. Because the previous run of slave process did not terminate
gracefully and did not return all the objects to the pool.
Is there any function to free an existing memory zone, which I call before
rte_mempool_create to ensure that previous memory pool does not exists
anymore?

Best Regards,
Mahdi.

Re: [dpdk-dev] free a memzone

[Mahdi Dashtbozorgi]

Hi guys,

Is there any suggestion to free the previously allocated memzone?
I really need help in this issue.
Any help is appreciated.

Best Regards,
Mahdi.



On Tue, Jul 22, 2014 at 4:03 PM, Mahdi Dashtbozorgi <mdasht@gmail.com>
wrote:

> Hi,
>
> I have two processes, which uses DPDK multi-process feature to communicate.
> Master process captures packets from NIC and put them to a ring buffer,
> which is shared between master and slave process.
> The slave process looks up the shared ring buffer using rte_ring_lookup
> function and reads the packets.
> The slave process needs a memory pool, too. Therefore, it creates a
> mempool using rte_mempool_create. But If the slave process crashes during
> its processing and runs again, rte_mempool_create function fails and tells
> that there is a memory zone with that name.
> If I use rte_mempool_lookup in this case, the memory pool is not a clean
> memory pool. Because the previous run of slave process did not terminate
> gracefully and did not return all the objects to the pool.
> Is there any function to free an existing memory zone, which I call before
> rte_mempool_create to ensure that previous memory pool does not exists
> anymore?
>
> Best Regards,
> Mahdi.
>

Re: [dpdk-dev] free a memzone

[Richardson, Bruce]

Rather than freeing the previously allocated memzone, could you not just re-initialize the mempool using something like rte_mempool_xmem_create?

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Mahdi Dashtbozorgi
> Sent: Wednesday, July 23, 2014 2:05 AM
> To: dev@dpdk.org
> Subject: Re: [dpdk-dev] free a memzone
> 
> Hi guys,
> 
> Is there any suggestion to free the previously allocated memzone?
> I really need help in this issue.
> Any help is appreciated.
> 
> Best Regards,
> Mahdi.
> 
> 
> 
> On Tue, Jul 22, 2014 at 4:03 PM, Mahdi Dashtbozorgi <mdasht@gmail.com>
> wrote:
> 
> > Hi,
> >
> > I have two processes, which uses DPDK multi-process feature to communicate.
> > Master process captures packets from NIC and put them to a ring buffer,
> > which is shared between master and slave process.
> > The slave process looks up the shared ring buffer using rte_ring_lookup
> > function and reads the packets.
> > The slave process needs a memory pool, too. Therefore, it creates a
> > mempool using rte_mempool_create. But If the slave process crashes during
> > its processing and runs again, rte_mempool_create function fails and tells
> > that there is a memory zone with that name.
> > If I use rte_mempool_lookup in this case, the memory pool is not a clean
> > memory pool. Because the previous run of slave process did not terminate
> > gracefully and did not return all the objects to the pool.
> > Is there any function to free an existing memory zone, which I call before
> > rte_mempool_create to ensure that previous memory pool does not exists
> > anymore?
> >
> > Best Regards,
> > Mahdi.
> >

Re: [dpdk-dev] free a memzone

[Mahdi Dashtbozorgi]

Hi Bruce,

Thank you for the response. That's a great Idea!
But I do not understand the last four parameters of this function. (vaddr,
paddr, pg_num, pg_shift)
I guess vaddr is the virtual address of the previously allocated mempool,
paddr is calculated using function call rte_mem_virt2phy(vaddr), am I
right? what about pg_num and pg_shift? how can I pass them correctly?

Best Regards,
Mahdi.


On Thu, Jul 24, 2014 at 9:48 AM, Mahdi Dashtbozorgi <mdasht@gmail.com>
wrote:

> Hi Bruce,
>
> Thank you for the response. That's a great Idea!
> But I do not understand the last four parameters of this function. (vaddr,
> paddr, pg_num, pg_shift)
> I guess vaddr is the virtual address of the previously allocated mempool,
> paddr is calculated using function call rte_mem_virt2phy(vaddr), am I
> right? what about pg_num and pg_shift? how can I pass them correctly?
>
> Best Regards,
> Mahdi.
>
>
> On Wed, Jul 23, 2014 at 11:09 PM, Richardson, Bruce <
> bruce.richardson@intel.com> wrote:
>
>> Rather than freeing the previously allocated memzone, could you not just
>> re-initialize the mempool using something like rte_mempool_xmem_create?
>>
>> > -----Original Message-----
>> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Mahdi Dashtbozorgi
>> > Sent: Wednesday, July 23, 2014 2:05 AM
>> > To: dev@dpdk.org
>> > Subject: Re: [dpdk-dev] free a memzone
>> >
>> > Hi guys,
>> >
>> > Is there any suggestion to free the previously allocated memzone?
>> > I really need help in this issue.
>> > Any help is appreciated.
>> >
>> > Best Regards,
>> > Mahdi.
>> >
>> >
>> >
>> > On Tue, Jul 22, 2014 at 4:03 PM, Mahdi Dashtbozorgi <mdasht@gmail.com>
>> > wrote:
>> >
>> > > Hi,
>> > >
>> > > I have two processes, which uses DPDK multi-process feature to
>> communicate.
>> > > Master process captures packets from NIC and put them to a ring
>> buffer,
>> > > which is shared between master and slave process.
>> > > The slave process looks up the shared ring buffer using
>> rte_ring_lookup
>> > > function and reads the packets.
>> > > The slave process needs a memory pool, too. Therefore, it creates a
>> > > mempool using rte_mempool_create. But If the slave process crashes
>> during
>> > > its processing and runs again, rte_mempool_create function fails and
>> tells
>> > > that there is a memory zone with that name.
>> > > If I use rte_mempool_lookup in this case, the memory pool is not a
>> clean
>> > > memory pool. Because the previous run of slave process did not
>> terminate
>> > > gracefully and did not return all the objects to the pool.
>> > > Is there any function to free an existing memory zone, which I call
>> before
>> > > rte_mempool_create to ensure that previous memory pool does not exists
>> > > anymore?
>> > >
>> > > Best Regards,
>> > > Mahdi.
>> > >
>>
>
>

Re: [dpdk-dev] free a memzone

[Ananyev, Konstantin]

Hi Mahdi,

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Mahdi Dashtbozorgi
> Sent: Thursday, July 24, 2014 6:20 AM
> To: dev@dpdk.org
> Subject: Re: [dpdk-dev] free a memzone
> 
> Hi Bruce,
> 
> Thank you for the response. That's a great Idea!
> But I do not understand the last four parameters of this function. (vaddr,
> paddr, pg_num, pg_shift)
> I guess vaddr is the virtual address of the previously allocated mempool,

yes

> paddr is calculated using function call rte_mem_virt2phy(vaddr), am I
> right?

yes

 >what about pg_num and pg_shift? how can I pass them correctly?

>From rte_mempool.h:
"* @param pg_num
 *   Number of elements in the paddr array.
 * @param pg_shift
 *   LOG2 of the physical pages size."

If you are using memzone as externally allocated memory - it will be already physically continuos.
So in your case pg_num = MEMPOOL_PG_NUM_DEFAULT,  pg_shift = MEMPOOL_PG_SHIFT_MAX.

Though, I don't think rte_mempool_xmem_create() will help you in any way.
Again from rte_mempool.h:
"* Creates a new mempool named *name* in memory.
 *
 * This function uses ``memzone_reserve()`` to allocate memory. The
 * pool contains n elements of elt_size. Its size is set to n.
 * Depending on the input parameters, mempool elements can be either allocated
 * together with the mempool header, or an externally provided memory buffer
 * could be used to store mempool objects. In later case, that external
 * memory buffer can consist of set of disjoint phyiscal pages."

So xmem_create would still create a new ring, reserve a new memzone of mempool's metadata, etc.
The only difference - it can use externally allocated memory to store mempool elements.

As I understand what you need is sort of mempool_reset(): a function that would re-init mempool to just created state
(all elements are free, lcores caches are empty, etc).
Right now we don't have such function,  but I suppose something like that should do
(note that I didn't run or even build it):

If ((mp = rte_mempool_lookup(name)) != NULL {
   
char ring_name[RTE_RING_NAMESIZE];

/* save mp ring name. */
memcpy(ring_name, mp->ring->name, sizeof ring_name);

/* reset the ring. */
  rte_ring_init(mp->ring, ring_name,  rte_align32pow2(mp->size+1), mp->ring->flags);

/*repopulate mempool and reinit all its elements. */
mempool_populate(mp, mp->size, 1, rte_pktmbuf_init, NULL);

/* reset all lcore caches. */
 memset(mp->local_cache, 0, sizeof(local_cahce));

  /* reset statistics if needed. */
} else {
  /* create new mempool. */
}

Ideally such function should be in the librte_mempool of course, but if you are in a hurry - you probably can give it a try.

Note that I assume that no other process, except failed/restarting secondary are using this mempool.
If primary or some other secondary do, then first you need to stop them using this mempool and wait till they finish all their packet processing activity.

Konstantin

> Best Regards,
> Mahdi.
> 
> 
> On Thu, Jul 24, 2014 at 9:48 AM, Mahdi Dashtbozorgi <mdasht@gmail.com>
> wrote:
> 
> > Hi Bruce,
> >
> > Thank you for the response. That's a great Idea!
> > But I do not understand the last four parameters of this function. (vaddr,
> > paddr, pg_num, pg_shift)
> > I guess vaddr is the virtual address of the previously allocated mempool,
> > paddr is calculated using function call rte_mem_virt2phy(vaddr), am I
> > right? what about pg_num and pg_shift? how can I pass them correctly?
> >
> > Best Regards,
> > Mahdi.
> >
> >
> > On Wed, Jul 23, 2014 at 11:09 PM, Richardson, Bruce <
> > bruce.richardson@intel.com> wrote:
> >
> >> Rather than freeing the previously allocated memzone, could you not just
> >> re-initialize the mempool using something like rte_mempool_xmem_create?
> >>
> >> > -----Original Message-----
> >> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Mahdi Dashtbozorgi
> >> > Sent: Wednesday, July 23, 2014 2:05 AM
> >> > To: dev@dpdk.org
> >> > Subject: Re: [dpdk-dev] free a memzone
> >> >
> >> > Hi guys,
> >> >
> >> > Is there any suggestion to free the previously allocated memzone?
> >> > I really need help in this issue.
> >> > Any help is appreciated.
> >> >
> >> > Best Regards,
> >> > Mahdi.
> >> >
> >> >
> >> >
> >> > On Tue, Jul 22, 2014 at 4:03 PM, Mahdi Dashtbozorgi <mdasht@gmail.com>
> >> > wrote:
> >> >
> >> > > Hi,
> >> > >
> >> > > I have two processes, which uses DPDK multi-process feature to
> >> communicate.
> >> > > Master process captures packets from NIC and put them to a ring
> >> buffer,
> >> > > which is shared between master and slave process.
> >> > > The slave process looks up the shared ring buffer using
> >> rte_ring_lookup
> >> > > function and reads the packets.
> >> > > The slave process needs a memory pool, too. Therefore, it creates a
> >> > > mempool using rte_mempool_create. But If the slave process crashes
> >> during
> >> > > its processing and runs again, rte_mempool_create function fails and
> >> tells
> >> > > that there is a memory zone with that name.
> >> > > If I use rte_mempool_lookup in this case, the memory pool is not a
> >> clean
> >> > > memory pool. Because the previous run of slave process did not
> >> terminate
> >> > > gracefully and did not return all the objects to the pool.
> >> > > Is there any function to free an existing memory zone, which I call
> >> before
> >> > > rte_mempool_create to ensure that previous memory pool does not exists
> >> > > anymore?
> >> > >
> >> > > Best Regards,
> >> > > Mahdi.
> >> > >
> >>
> >
> >

Re: [dpdk-dev] free a memzone

[Mahdi Dashtbozorgi]

Hi Konstantin,

Thank you very much. Your solution fixed my problem.
Is there a solution like this for resetting the memory zone, which is used
by rte_malloc function?
Because if I use rte_malloc instead of malloc, in the case of application
crash, the memory zone, which was used by rte_malloc in the previous run
would be unusable for the next run of slave process.

Best Regards,
Mahdi.




On Mon, Jul 28, 2014 at 4:27 PM, Ananyev, Konstantin <
konstantin.ananyev@intel.com> wrote:

> Hi Mahdi,
>
> > -----Original Message-----
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Mahdi Dashtbozorgi
> > Sent: Thursday, July 24, 2014 6:20 AM
> > To: dev@dpdk.org
> > Subject: Re: [dpdk-dev] free a memzone
> >
> > Hi Bruce,
> >
> > Thank you for the response. That's a great Idea!
> > But I do not understand the last four parameters of this function.
> (vaddr,
> > paddr, pg_num, pg_shift)
> > I guess vaddr is the virtual address of the previously allocated mempool,
>
> yes
>
> > paddr is calculated using function call rte_mem_virt2phy(vaddr), am I
> > right?
>
> yes
>
>  >what about pg_num and pg_shift? how can I pass them correctly?
>
> From rte_mempool.h:
> "* @param pg_num
>  *   Number of elements in the paddr array.
>  * @param pg_shift
>  *   LOG2 of the physical pages size."
>
> If you are using memzone as externally allocated memory - it will be
> already physically continuos.
> So in your case pg_num = MEMPOOL_PG_NUM_DEFAULT,  pg_shift =
> MEMPOOL_PG_SHIFT_MAX.
>
> Though, I don't think rte_mempool_xmem_create() will help you in any way.
> Again from rte_mempool.h:
> "* Creates a new mempool named *name* in memory.
>  *
>  * This function uses ``memzone_reserve()`` to allocate memory. The
>  * pool contains n elements of elt_size. Its size is set to n.
>  * Depending on the input parameters, mempool elements can be either
> allocated
>  * together with the mempool header, or an externally provided memory
> buffer
>  * could be used to store mempool objects. In later case, that external
>  * memory buffer can consist of set of disjoint phyiscal pages."
>
> So xmem_create would still create a new ring, reserve a new memzone of
> mempool's metadata, etc.
> The only difference - it can use externally allocated memory to store
> mempool elements.
>
> As I understand what you need is sort of mempool_reset(): a function that
> would re-init mempool to just created state
> (all elements are free, lcores caches are empty, etc).
> Right now we don't have such function,  but I suppose something like that
> should do
> (note that I didn't run or even build it):
>
> If ((mp = rte_mempool_lookup(name)) != NULL {
>
> char ring_name[RTE_RING_NAMESIZE];
>
> /* save mp ring name. */
> memcpy(ring_name, mp->ring->name, sizeof ring_name);
>
> /* reset the ring. */
>   rte_ring_init(mp->ring, ring_name,  rte_align32pow2(mp->size+1),
> mp->ring->flags);
>
> /*repopulate mempool and reinit all its elements. */
> mempool_populate(mp, mp->size, 1, rte_pktmbuf_init, NULL);
>
> /* reset all lcore caches. */
>  memset(mp->local_cache, 0, sizeof(local_cahce));
>
>   /* reset statistics if needed. */
> } else {
>   /* create new mempool. */
> }
>
> Ideally such function should be in the librte_mempool of course, but if
> you are in a hurry - you probably can give it a try.
>
> Note that I assume that no other process, except failed/restarting
> secondary are using this mempool.
> If primary or some other secondary do, then first you need to stop them
> using this mempool and wait till they finish all their packet processing
> activity.
>
> Konstantin
>
> > Best Regards,
> > Mahdi.
> >
> >
> > On Thu, Jul 24, 2014 at 9:48 AM, Mahdi Dashtbozorgi <mdasht@gmail.com>
> > wrote:
> >
> > > Hi Bruce,
> > >
> > > Thank you for the response. That's a great Idea!
> > > But I do not understand the last four parameters of this function.
> (vaddr,
> > > paddr, pg_num, pg_shift)
> > > I guess vaddr is the virtual address of the previously allocated
> mempool,
> > > paddr is calculated using function call rte_mem_virt2phy(vaddr), am I
> > > right? what about pg_num and pg_shift? how can I pass them correctly?
> > >
> > > Best Regards,
> > > Mahdi.
> > >
> > >
> > > On Wed, Jul 23, 2014 at 11:09 PM, Richardson, Bruce <
> > > bruce.richardson@intel.com> wrote:
> > >
> > >> Rather than freeing the previously allocated memzone, could you not
> just
> > >> re-initialize the mempool using something like
> rte_mempool_xmem_create?
> > >>
> > >> > -----Original Message-----
> > >> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Mahdi
> Dashtbozorgi
> > >> > Sent: Wednesday, July 23, 2014 2:05 AM
> > >> > To: dev@dpdk.org
> > >> > Subject: Re: [dpdk-dev] free a memzone
> > >> >
> > >> > Hi guys,
> > >> >
> > >> > Is there any suggestion to free the previously allocated memzone?
> > >> > I really need help in this issue.
> > >> > Any help is appreciated.
> > >> >
> > >> > Best Regards,
> > >> > Mahdi.
> > >> >
> > >> >
> > >> >
> > >> > On Tue, Jul 22, 2014 at 4:03 PM, Mahdi Dashtbozorgi <
> mdasht@gmail.com>
> > >> > wrote:
> > >> >
> > >> > > Hi,
> > >> > >
> > >> > > I have two processes, which uses DPDK multi-process feature to
> > >> communicate.
> > >> > > Master process captures packets from NIC and put them to a ring
> > >> buffer,
> > >> > > which is shared between master and slave process.
> > >> > > The slave process looks up the shared ring buffer using
> > >> rte_ring_lookup
> > >> > > function and reads the packets.
> > >> > > The slave process needs a memory pool, too. Therefore, it creates
> a
> > >> > > mempool using rte_mempool_create. But If the slave process crashes
> > >> during
> > >> > > its processing and runs again, rte_mempool_create function fails
> and
> > >> tells
> > >> > > that there is a memory zone with that name.
> > >> > > If I use rte_mempool_lookup in this case, the memory pool is not a
> > >> clean
> > >> > > memory pool. Because the previous run of slave process did not
> > >> terminate
> > >> > > gracefully and did not return all the objects to the pool.
> > >> > > Is there any function to free an existing memory zone, which I
> call
> > >> before
> > >> > > rte_mempool_create to ensure that previous memory pool does not
> exists
> > >> > > anymore?
> > >> > >
> > >> > > Best Regards,
> > >> > > Mahdi.
> > >> > >
> > >>
> > >
> > >
>

Re: [dpdk-dev] free a memzone

[Ananyev, Konstantin]

Hi Mahdi,

>Hi Konstantin,
>Thank you very much. Your solution fixed my problem.
>Is there a solution like this for resetting the memory zone, which is used by rte_malloc function? 
>Because if I use rte_malloc instead of malloc, in the case of application crash, the memory zone, which was used by rte_malloc in the previous run would be >unusable for the next run of slave process. 

Without significant modification inside librte_eal and/or librte_malloc - nothing comes on top of my head.
If that is such a big problem to you, might be it is possible to change your whole process model a bit:

In the parent process:
- allocate memory/init run-time strcutures
L1:
- fork();
- wait till child terminates
- if it terminated abnormally, then free memory/re-init run-time structures.
Goto L1

Do actual packet processing inside child process.

Would that help somehow?
Konstantin

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Mahdi Dashtbozorgi
> Sent: Thursday, July 24, 2014 6:20 AM
> To: dev@dpdk.org
> Subject: Re: [dpdk-dev] free a memzone
>
> Hi Bruce,
>
> Thank you for the response. That's a great Idea!
> But I do not understand the last four parameters of this function. (vaddr,
> paddr, pg_num, pg_shift)
> I guess vaddr is the virtual address of the previously allocated mempool,
yes

> paddr is calculated using function call rte_mem_virt2phy(vaddr), am I
> right?
yes

 >what about pg_num and pg_shift? how can I pass them correctly?
>From rte_mempool.h:
"* @param pg_num
 *   Number of elements in the paddr array.
 * @param pg_shift
 *   LOG2 of the physical pages size."

If you are using memzone as externally allocated memory - it will be already physically continuos.
So in your case pg_num = MEMPOOL_PG_NUM_DEFAULT,  pg_shift = MEMPOOL_PG_SHIFT_MAX.

Though, I don't think rte_mempool_xmem_create() will help you in any way.
Again from rte_mempool.h:
"* Creates a new mempool named *name* in memory.
 *
 * This function uses ``memzone_reserve()`` to allocate memory. The
 * pool contains n elements of elt_size. Its size is set to n.
 * Depending on the input parameters, mempool elements can be either allocated
 * together with the mempool header, or an externally provided memory buffer
 * could be used to store mempool objects. In later case, that external
 * memory buffer can consist of set of disjoint phyiscal pages."

So xmem_create would still create a new ring, reserve a new memzone of mempool's metadata, etc.
The only difference - it can use externally allocated memory to store mempool elements.

As I understand what you need is sort of mempool_reset(): a function that would re-init mempool to just created state
(all elements are free, lcores caches are empty, etc).
Right now we don't have such function,  but I suppose something like that should do
(note that I didn't run or even build it):

If ((mp = rte_mempool_lookup(name)) != NULL {

char ring_name[RTE_RING_NAMESIZE];

/* save mp ring name. */
memcpy(ring_name, mp->ring->name, sizeof ring_name);

/* reset the ring. */
  rte_ring_init(mp->ring, ring_name,  rte_align32pow2(mp->size+1), mp->ring->flags);

/*repopulate mempool and reinit all its elements. */
mempool_populate(mp, mp->size, 1, rte_pktmbuf_init, NULL);

/* reset all lcore caches. */
 memset(mp->local_cache, 0, sizeof(local_cahce));

  /* reset statistics if needed. */
} else {
  /* create new mempool. */
}

Ideally such function should be in the librte_mempool of course, but if you are in a hurry - you probably can give it a try.

Note that I assume that no other process, except failed/restarting secondary are using this mempool.
If primary or some other secondary do, then first you need to stop them using this mempool and wait till they finish all their packet processing activity.

Konstantin

> Best Regards,
> Mahdi.
>
>
> On Thu, Jul 24, 2014 at 9:48 AM, Mahdi Dashtbozorgi <mdasht@gmail.com>
> wrote:
>
> > Hi Bruce,
> >
> > Thank you for the response. That's a great Idea!
> > But I do not understand the last four parameters of this function. (vaddr,
> > paddr, pg_num, pg_shift)
> > I guess vaddr is the virtual address of the previously allocated mempool,
> > paddr is calculated using function call rte_mem_virt2phy(vaddr), am I
> > right? what about pg_num and pg_shift? how can I pass them correctly?
> >
> > Best Regards,
> > Mahdi.
> >
> >
> > On Wed, Jul 23, 2014 at 11:09 PM, Richardson, Bruce <
> > bruce.richardson@intel.com> wrote:
> >
> >> Rather than freeing the previously allocated memzone, could you not just
> >> re-initialize the mempool using something like rte_mempool_xmem_create?
> >>
> >> > -----Original Message-----
> >> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Mahdi Dashtbozorgi
> >> > Sent: Wednesday, July 23, 2014 2:05 AM
> >> > To: dev@dpdk.org
> >> > Subject: Re: [dpdk-dev] free a memzone
> >> >
> >> > Hi guys,
> >> >
> >> > Is there any suggestion to free the previously allocated memzone?
> >> > I really need help in this issue.
> >> > Any help is appreciated.
> >> >
> >> > Best Regards,
> >> > Mahdi.
> >> >
> >> >
> >> >
> >> > On Tue, Jul 22, 2014 at 4:03 PM, Mahdi Dashtbozorgi <mdasht@gmail.com>
> >> > wrote:
> >> >
> >> > > Hi,
> >> > >
> >> > > I have two processes, which uses DPDK multi-process feature to
> >> communicate.
> >> > > Master process captures packets from NIC and put them to a ring
> >> buffer,
> >> > > which is shared between master and slave process.
> >> > > The slave process looks up the shared ring buffer using
> >> rte_ring_lookup
> >> > > function and reads the packets.
> >> > > The slave process needs a memory pool, too. Therefore, it creates a
> >> > > mempool using rte_mempool_create. But If the slave process crashes
> >> during
> >> > > its processing and runs again, rte_mempool_create function fails and
> >> tells
> >> > > that there is a memory zone with that name.
> >> > > If I use rte_mempool_lookup in this case, the memory pool is not a
> >> clean
> >> > > memory pool. Because the previous run of slave process did not
> >> terminate
> >> > > gracefully and did not return all the objects to the pool.
> >> > > Is there any function to free an existing memory zone, which I call
> >> before
> >> > > rte_mempool_create to ensure that previous memory pool does not exists
> >> > > anymore?
> >> > >
> >> > > Best Regards,
> >> > > Mahdi.
> >> > >
> >>
> >
> >

Re: [dpdk-dev] free a memzone

[Mahdi Dashtbozorgi]

Hi Konstantin,

Thank you for your response.
Your Idea seems great! I will implement it this way.

Best Regards,
Mahdi.



On Mon, Jul 28, 2014 at 9:39 PM, Ananyev, Konstantin <
konstantin.ananyev@intel.com> wrote:

> Hi Mahdi,
>
> >Hi Konstantin,
> >Thank you very much. Your solution fixed my problem.
> >Is there a solution like this for resetting the memory zone, which is
> used by rte_malloc function?
> >Because if I use rte_malloc instead of malloc, in the case of application
> crash, the memory zone, which was used by rte_malloc in the previous run
> would be >unusable for the next run of slave process.
>
> Without significant modification inside librte_eal and/or librte_malloc -
> nothing comes on top of my head.
> If that is such a big problem to you, might be it is possible to change
> your whole process model a bit:
>
> In the parent process:
> - allocate memory/init run-time strcutures
> L1:
> - fork();
> - wait till child terminates
> - if it terminated abnormally, then free memory/re-init run-time
> structures.
> Goto L1
>
> Do actual packet processing inside child process.
>
> Would that help somehow?
> Konstantin
>
> > -----Original Message-----
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Mahdi Dashtbozorgi
> > Sent: Thursday, July 24, 2014 6:20 AM
> > To: dev@dpdk.org
> > Subject: Re: [dpdk-dev] free a memzone
> >
> > Hi Bruce,
> >
> > Thank you for the response. That's a great Idea!
> > But I do not understand the last four parameters of this function.
> (vaddr,
> > paddr, pg_num, pg_shift)
> > I guess vaddr is the virtual address of the previously allocated mempool,
> yes
>
> > paddr is calculated using function call rte_mem_virt2phy(vaddr), am I
> > right?
> yes
>
>  >what about pg_num and pg_shift? how can I pass them correctly?
> From rte_mempool.h:
> "* @param pg_num
>  *   Number of elements in the paddr array.
>  * @param pg_shift
>  *   LOG2 of the physical pages size."
>
> If you are using memzone as externally allocated memory - it will be
> already physically continuos.
> So in your case pg_num = MEMPOOL_PG_NUM_DEFAULT,  pg_shift =
> MEMPOOL_PG_SHIFT_MAX.
>
> Though, I don't think rte_mempool_xmem_create() will help you in any way.
> Again from rte_mempool.h:
> "* Creates a new mempool named *name* in memory.
>  *
>  * This function uses ``memzone_reserve()`` to allocate memory. The
>  * pool contains n elements of elt_size. Its size is set to n.
>  * Depending on the input parameters, mempool elements can be either
> allocated
>  * together with the mempool header, or an externally provided memory
> buffer
>  * could be used to store mempool objects. In later case, that external
>  * memory buffer can consist of set of disjoint phyiscal pages."
>
> So xmem_create would still create a new ring, reserve a new memzone of
> mempool's metadata, etc.
> The only difference - it can use externally allocated memory to store
> mempool elements.
>
> As I understand what you need is sort of mempool_reset(): a function that
> would re-init mempool to just created state
> (all elements are free, lcores caches are empty, etc).
> Right now we don't have such function,  but I suppose something like that
> should do
> (note that I didn't run or even build it):
>
> If ((mp = rte_mempool_lookup(name)) != NULL {
>
> char ring_name[RTE_RING_NAMESIZE];
>
> /* save mp ring name. */
> memcpy(ring_name, mp->ring->name, sizeof ring_name);
>
> /* reset the ring. */
>   rte_ring_init(mp->ring, ring_name,  rte_align32pow2(mp->size+1),
> mp->ring->flags);
>
> /*repopulate mempool and reinit all its elements. */
> mempool_populate(mp, mp->size, 1, rte_pktmbuf_init, NULL);
>
> /* reset all lcore caches. */
>  memset(mp->local_cache, 0, sizeof(local_cahce));
>
>   /* reset statistics if needed. */
> } else {
>   /* create new mempool. */
> }
>
> Ideally such function should be in the librte_mempool of course, but if
> you are in a hurry - you probably can give it a try.
>
> Note that I assume that no other process, except failed/restarting
> secondary are using this mempool.
> If primary or some other secondary do, then first you need to stop them
> using this mempool and wait till they finish all their packet processing
> activity.
>
> Konstantin
>
> > Best Regards,
> > Mahdi.
> >
> >
> > On Thu, Jul 24, 2014 at 9:48 AM, Mahdi Dashtbozorgi <mdasht@gmail.com>
> > wrote:
> >
> > > Hi Bruce,
> > >
> > > Thank you for the response. That's a great Idea!
> > > But I do not understand the last four parameters of this function.
> (vaddr,
> > > paddr, pg_num, pg_shift)
> > > I guess vaddr is the virtual address of the previously allocated
> mempool,
> > > paddr is calculated using function call rte_mem_virt2phy(vaddr), am I
> > > right? what about pg_num and pg_shift? how can I pass them correctly?
> > >
> > > Best Regards,
> > > Mahdi.
> > >
> > >
> > > On Wed, Jul 23, 2014 at 11:09 PM, Richardson, Bruce <
> > > bruce.richardson@intel.com> wrote:
> > >
> > >> Rather than freeing the previously allocated memzone, could you not
> just
> > >> re-initialize the mempool using something like
> rte_mempool_xmem_create?
> > >>
> > >> > -----Original Message-----
> > >> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Mahdi
> Dashtbozorgi
> > >> > Sent: Wednesday, July 23, 2014 2:05 AM
> > >> > To: dev@dpdk.org
> > >> > Subject: Re: [dpdk-dev] free a memzone
> > >> >
> > >> > Hi guys,
> > >> >
> > >> > Is there any suggestion to free the previously allocated memzone?
> > >> > I really need help in this issue.
> > >> > Any help is appreciated.
> > >> >
> > >> > Best Regards,
> > >> > Mahdi.
> > >> >
> > >> >
> > >> >
> > >> > On Tue, Jul 22, 2014 at 4:03 PM, Mahdi Dashtbozorgi <
> mdasht@gmail.com>
> > >> > wrote:
> > >> >
> > >> > > Hi,
> > >> > >
> > >> > > I have two processes, which uses DPDK multi-process feature to
> > >> communicate.
> > >> > > Master process captures packets from NIC and put them to a ring
> > >> buffer,
> > >> > > which is shared between master and slave process.
> > >> > > The slave process looks up the shared ring buffer using
> > >> rte_ring_lookup
> > >> > > function and reads the packets.
> > >> > > The slave process needs a memory pool, too. Therefore, it creates
> a
> > >> > > mempool using rte_mempool_create. But If the slave process crashes
> > >> during
> > >> > > its processing and runs again, rte_mempool_create function fails
> and
> > >> tells
> > >> > > that there is a memory zone with that name.
> > >> > > If I use rte_mempool_lookup in this case, the memory pool is not a
> > >> clean
> > >> > > memory pool. Because the previous run of slave process did not
> > >> terminate
> > >> > > gracefully and did not return all the objects to the pool.
> > >> > > Is there any function to free an existing memory zone, which I
> call
> > >> before
> > >> > > rte_mempool_create to ensure that previous memory pool does not
> exists
> > >> > > anymore?
> > >> > >
> > >> > > Best Regards,
> > >> > > Mahdi.
> > >> > >
> > >>
> > >
> > >
>
>