* [dpdk-dev] RTE Ring removing @ 2014-05-06 9:05 Igor Ryzhov 2014-05-07 7:54 ` Igor Ryzhov 0 siblings, 1 reply; 10+ messages in thread From: Igor Ryzhov @ 2014-05-06 9:05 UTC (permalink / raw) To: dev Hello. For what reason RTE Rings can not be removed once created? In my application I want to use many rings with different names so I think there may be a problem with memory because of many ring that already not in use, but allocated. Or DPDK has a mechanism of reusing memory if rings are not in use? Best regards, Igor Ryzhov ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [dpdk-dev] RTE Ring removing 2014-05-06 9:05 [dpdk-dev] RTE Ring removing Igor Ryzhov @ 2014-05-07 7:54 ` Igor Ryzhov 2014-05-07 11:39 ` Olivier MATZ 0 siblings, 1 reply; 10+ messages in thread From: Igor Ryzhov @ 2014-05-07 7:54 UTC (permalink / raw) To: dev Hello again. I did some investigation on the code. I learned that RTE Ring creation function uses functions related to RTE Memzone to reserve memory (rte_memzone_reserve). Documentation states that once reserved memzone can not be unreserved. I decided to find out why it is so. I noticed that in Memzone realization there is a special global variable "free_memseg" containing pointers on free memory segments. An memzone reserve function just finst the best segment for allocation from this "free_memseg" variable. So I think there is a possibility to unreserve already reserved memory back to "free_memseg", and impossibility of unreserving memory is just because there is no function for that, not because it is impossible in principle. Am I right? Or there are any restrictions? Best regards, Igor Ryzhov 06.05.2014 13:05, Igor Ryzhov пишет: > Hello. > > For what reason RTE Rings can not be removed once created? > In my application I want to use many rings with different names so I > think there may be a problem with memory because of many ring that > already not in use, but allocated. > Or DPDK has a mechanism of reusing memory if rings are not in use? > > Best regards, > Igor Ryzhov ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [dpdk-dev] RTE Ring removing 2014-05-07 7:54 ` Igor Ryzhov @ 2014-05-07 11:39 ` Olivier MATZ 2014-05-07 12:42 ` Igor Ryzhov 2014-05-07 14:01 ` Venkatesan, Venky 0 siblings, 2 replies; 10+ messages in thread From: Olivier MATZ @ 2014-05-07 11:39 UTC (permalink / raw) To: Igor Ryzhov, dev Hi Igor, On 05/07/2014 09:54 AM, Igor Ryzhov wrote: > I noticed that in Memzone realization there is a special global variable > "free_memseg" containing pointers on free memory segments. > An memzone reserve function just finst the best segment for allocation > from this "free_memseg" variable. > > So I think there is a possibility to unreserve already reserved memory > back to "free_memseg", and impossibility of unreserving memory is just > because there is no function for that, not because it is impossible in > principle. > Am I right? Or there are any restrictions? I think that implementing a freeing of memory segment is feasible, but it would require some work to properly merge freed zones to avoid memory fragmentation. Another solution is to allocate/free rings in standard memory (malloc for instance) instead of rte_memzones. Let me know if the patches I've just sent on the mailing list solves your issue. By the way, I plan to do the same thing for mempools in the coming weeks but there is much more work. Regards, Olivier ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [dpdk-dev] RTE Ring removing 2014-05-07 11:39 ` Olivier MATZ @ 2014-05-07 12:42 ` Igor Ryzhov 2014-05-07 13:08 ` Olivier MATZ 2014-05-07 14:01 ` Venkatesan, Venky 1 sibling, 1 reply; 10+ messages in thread From: Igor Ryzhov @ 2014-05-07 12:42 UTC (permalink / raw) To: Olivier MATZ, dev It seems to be a good idea, thank you, Olivier! But a few questions: 1. Will this changes affect performance? 2. In PATCH 2/2 you have a small bug: In file rte_ring.h, in comments describing rte_ring_init function you have: + * @param name + * The size of the ring. But it is name of the ring, not size. Best regards, Igor Ryzhov 07.05.2014 15:39, Olivier MATZ пишет: > Hi Igor, > > On 05/07/2014 09:54 AM, Igor Ryzhov wrote: >> I noticed that in Memzone realization there is a special global variable >> "free_memseg" containing pointers on free memory segments. >> An memzone reserve function just finst the best segment for allocation >> from this "free_memseg" variable. >> >> So I think there is a possibility to unreserve already reserved memory >> back to "free_memseg", and impossibility of unreserving memory is just >> because there is no function for that, not because it is impossible in >> principle. >> Am I right? Or there are any restrictions? > > I think that implementing a freeing of memory segment is feasible, but > it would require some work to properly merge freed zones to avoid memory > fragmentation. > > Another solution is to allocate/free rings in standard memory (malloc > for instance) instead of rte_memzones. Let me know if the patches I've > just sent on the mailing list solves your issue. > > By the way, I plan to do the same thing for mempools in the coming > weeks but there is much more work. > > Regards, > Olivier > ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [dpdk-dev] RTE Ring removing 2014-05-07 12:42 ` Igor Ryzhov @ 2014-05-07 13:08 ` Olivier MATZ 0 siblings, 0 replies; 10+ messages in thread From: Olivier MATZ @ 2014-05-07 13:08 UTC (permalink / raw) To: Igor Ryzhov, dev Hi Igor, On 05/07/2014 02:42 PM, Igor Ryzhov wrote: > But a few questions: > 1. Will this changes affect performance? It should not. If you have many rings, you may allocate them in huge pages to avoid TLB misses. > 2. In PATCH 2/2 you have a small bug: > > In file rte_ring.h, in comments describing rte_ring_init function you have: > > + * @param name > + * The size of the ring. > > But it is name of the ring, not size. Thank you for this comment, I'll fix it in the v2. Regards, Olivier ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [dpdk-dev] RTE Ring removing 2014-05-07 11:39 ` Olivier MATZ 2014-05-07 12:42 ` Igor Ryzhov @ 2014-05-07 14:01 ` Venkatesan, Venky 2014-05-07 15:09 ` Olivier MATZ 2014-05-07 15:36 ` Rogers, Gerald 1 sibling, 2 replies; 10+ messages in thread From: Venkatesan, Venky @ 2014-05-07 14:01 UTC (permalink / raw) To: Olivier MATZ, Igor Ryzhov, dev Olivier, We should look at how to make the memseg capable of doing alloc/free (including re-assembly of fragments) after the 1.7 release. Is that something you are considering doing (or are there any other DPDKers considering this), or should I look at putting together a patch for that? Regards, -Venky -----Original Message----- From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Olivier MATZ Sent: Wednesday, May 07, 2014 4:39 AM To: Igor Ryzhov; dev@dpdk.org Subject: Re: [dpdk-dev] RTE Ring removing Hi Igor, On 05/07/2014 09:54 AM, Igor Ryzhov wrote: > I noticed that in Memzone realization there is a special global > variable "free_memseg" containing pointers on free memory segments. > An memzone reserve function just finst the best segment for allocation > from this "free_memseg" variable. > > So I think there is a possibility to unreserve already reserved memory > back to "free_memseg", and impossibility of unreserving memory is just > because there is no function for that, not because it is impossible in > principle. > Am I right? Or there are any restrictions? I think that implementing a freeing of memory segment is feasible, but it would require some work to properly merge freed zones to avoid memory fragmentation. Another solution is to allocate/free rings in standard memory (malloc for instance) instead of rte_memzones. Let me know if the patches I've just sent on the mailing list solves your issue. By the way, I plan to do the same thing for mempools in the coming weeks but there is much more work. Regards, Olivier ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [dpdk-dev] RTE Ring removing 2014-05-07 14:01 ` Venkatesan, Venky @ 2014-05-07 15:09 ` Olivier MATZ 2014-05-07 15:19 ` Ananyev, Konstantin 2014-05-07 15:36 ` Rogers, Gerald 1 sibling, 1 reply; 10+ messages in thread From: Olivier MATZ @ 2014-05-07 15:09 UTC (permalink / raw) To: Venkatesan, Venky, Igor Ryzhov, dev Hi Venky, On 05/07/2014 04:01 PM, Venkatesan, Venky wrote: > We should look at how to make the memseg capable of doing alloc/free > (including re-assembly of fragments) after the 1.7 release. Is that > something you are considering doing (or are there any other DPDKers > considering this), or should I look at putting together a patch for > that? No, that's not something I'm working on today. On this topic, I have some work in progress in the rte_mempool code. I'll submit it here as soon as it is ready, I'm not sure it will be finished before the end of the 1.7.0 integration window. Regards, Olivier ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [dpdk-dev] RTE Ring removing 2014-05-07 15:09 ` Olivier MATZ @ 2014-05-07 15:19 ` Ananyev, Konstantin 2014-05-07 15:37 ` Olivier MATZ 0 siblings, 1 reply; 10+ messages in thread From: Ananyev, Konstantin @ 2014-05-07 15:19 UTC (permalink / raw) To: Olivier MATZ, Venkatesan, Venky, Igor Ryzhov, dev Hi Oliver, Just to clarify about mempool - I suppose you are talking about ability to place internal ring and mempool metadata inside externally allocated memory? It is already possible to keep mempool elements inside externally allocated memory (rte_mempool_xmem_create()). Konstantin -----Original Message----- From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Olivier MATZ Sent: Wednesday, May 07, 2014 4:10 PM To: Venkatesan, Venky; Igor Ryzhov; dev@dpdk.org Subject: Re: [dpdk-dev] RTE Ring removing Hi Venky, On 05/07/2014 04:01 PM, Venkatesan, Venky wrote: > We should look at how to make the memseg capable of doing alloc/free > (including re-assembly of fragments) after the 1.7 release. Is that > something you are considering doing (or are there any other DPDKers > considering this), or should I look at putting together a patch for > that? No, that's not something I'm working on today. On this topic, I have some work in progress in the rte_mempool code. I'll submit it here as soon as it is ready, I'm not sure it will be finished before the end of the 1.7.0 integration window. Regards, Olivier ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [dpdk-dev] RTE Ring removing 2014-05-07 15:19 ` Ananyev, Konstantin @ 2014-05-07 15:37 ` Olivier MATZ 0 siblings, 0 replies; 10+ messages in thread From: Olivier MATZ @ 2014-05-07 15:37 UTC (permalink / raw) To: Ananyev, Konstantin, Venkatesan, Venky, Igor Ryzhov, dev Hi Konstantin, On 05/07/2014 05:19 PM, Ananyev, Konstantin wrote: > Just to clarify about mempool - I suppose you are talking about > ability to place internal ring and mempool metadata inside externally > allocated memory? Yes, exactly. > It is already possible to keep mempool elements inside externally > allocated memory (rte_mempool_xmem_create()). You are right, but I think the current API is a bit too complex. For instance the function rte_mempool_xmem_create() has 15 arguments which is probably too much. Anyway, as soon as I'll have a patch to show, I'll send it to the mailing list so we can discuss it. Regards, Olivier ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [dpdk-dev] RTE Ring removing 2014-05-07 14:01 ` Venkatesan, Venky 2014-05-07 15:09 ` Olivier MATZ @ 2014-05-07 15:36 ` Rogers, Gerald 1 sibling, 0 replies; 10+ messages in thread From: Rogers, Gerald @ 2014-05-07 15:36 UTC (permalink / raw) To: Venkatesan, Venky, Olivier MATZ, Igor Ryzhov, dev Venky, This also applies to mbuf pools. Inside of the openvswitch.org patches we allocate mbuf pools for a port, but we are unable to free them back when the port is removed. One other request (maybe it is there, and I¹m unaware), is the ability to dynamically add / remove a physical port to DPDK. Basically we should be able to reassign on the fly a port from the kernel to DPDK, and vice versa (of course with the caveat that all structures be released in both environments and a port reinitialized). Gerald On 5/7/14, 7:01 AM, "Venkatesan, Venky" <venky.venkatesan@intel.com> wrote: >Olivier, > >We should look at how to make the memseg capable of doing alloc/free >(including re-assembly of fragments) after the 1.7 release. Is that >something you are considering doing (or are there any other DPDKers >considering this), or should I look at putting together a patch for that? > >Regards, >-Venky > >-----Original Message----- >From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Olivier MATZ >Sent: Wednesday, May 07, 2014 4:39 AM >To: Igor Ryzhov; dev@dpdk.org >Subject: Re: [dpdk-dev] RTE Ring removing > >Hi Igor, > >On 05/07/2014 09:54 AM, Igor Ryzhov wrote: >> I noticed that in Memzone realization there is a special global >> variable "free_memseg" containing pointers on free memory segments. >> An memzone reserve function just finst the best segment for allocation >> from this "free_memseg" variable. >> >> So I think there is a possibility to unreserve already reserved memory >> back to "free_memseg", and impossibility of unreserving memory is just >> because there is no function for that, not because it is impossible in >> principle. >> Am I right? Or there are any restrictions? > >I think that implementing a freeing of memory segment is feasible, but it >would require some work to properly merge freed zones to avoid memory >fragmentation. > >Another solution is to allocate/free rings in standard memory (malloc for >instance) instead of rte_memzones. Let me know if the patches I've just >sent on the mailing list solves your issue. > >By the way, I plan to do the same thing for mempools in the coming weeks >but there is much more work. > >Regards, >Olivier > ^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2014-05-07 15:37 UTC | newest] Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2014-05-06 9:05 [dpdk-dev] RTE Ring removing Igor Ryzhov 2014-05-07 7:54 ` Igor Ryzhov 2014-05-07 11:39 ` Olivier MATZ 2014-05-07 12:42 ` Igor Ryzhov 2014-05-07 13:08 ` Olivier MATZ 2014-05-07 14:01 ` Venkatesan, Venky 2014-05-07 15:09 ` Olivier MATZ 2014-05-07 15:19 ` Ananyev, Konstantin 2014-05-07 15:37 ` Olivier MATZ 2014-05-07 15:36 ` Rogers, Gerald
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).