* [dpdk-stable] [PATCH] malloc: notify primary process about hotplug in secondary [not found] <20181204170610.250124-1-seth.howell@intel.com> @ 2018-12-07 20:06 ` Seth Howell 2018-12-07 20:10 ` [dpdk-stable] [PATCH v2] " Seth Howell 1 sibling, 0 replies; 7+ messages in thread From: Seth Howell @ 2018-12-07 20:06 UTC (permalink / raw) To: anatoly.burakov; +Cc: dev, stable, Seth Howell, Darek Stojaczyk When secondary process hotplugs memory, it sends a request to primary, which then performs the real mmap() and sends sync requests to all secondary processes. Upon receiving such sync request, each secondary process will notify the upper layers of hotplugged memory (and will call all locally registered event callbacks). In the end we'll end up with memory event callbacks fired in all the processes except the primary, which is a bug. This gets critical if memory is hotplugged while a VFIO device is attached, as the VFIO memory registration - which is done from a memory event callback present in the primary process only - is never called. After this patch, a primary process fires memory event callbacks before secondary processes start their synchronizations - both for hotplug and hotremove. Fixes: 07dcbfe0101f ("malloc: support multiprocess memory hotplug") Signed-off-by: Seth Howell <seth.howell@intel.com> Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com> --- lib/librte_eal/common/malloc_mp.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/lib/librte_eal/common/malloc_mp.c b/lib/librte_eal/common/malloc_mp.c index 5f2d4e0be..f3a13353b 100644 --- a/lib/librte_eal/common/malloc_mp.c +++ b/lib/librte_eal/common/malloc_mp.c @@ -209,6 +209,8 @@ handle_alloc_request(const struct malloc_mp_req *m, map_addr = ms[0]->addr; + eal_memalloc_mem_event_notify(RTE_MEM_EVENT_ALLOC, map_addr, alloc_sz); + /* we have succeeded in allocating memory, but we still need to sync * with other processes. however, since DPDK IPC is single-threaded, we * send an asynchronous request and exit this callback. @@ -258,6 +260,9 @@ handle_request(const struct rte_mp_msg *msg, const void *peer __rte_unused) if (m->t == REQ_TYPE_ALLOC) { ret = handle_alloc_request(m, entry); } else if (m->t == REQ_TYPE_FREE) { + eal_memalloc_mem_event_notify(RTE_MEM_EVENT_FREE, + m->free_req.addr, m->free_req.len); + ret = malloc_heap_free_pages(m->free_req.addr, m->free_req.len); } else { @@ -436,6 +441,9 @@ handle_sync_response(const struct rte_mp_msg *request, memset(&rb_msg, 0, sizeof(rb_msg)); /* we've failed to sync, so do a rollback */ + eal_memalloc_mem_event_notify(RTE_MEM_EVENT_FREE, + state->map_addr, state->map_len); + rollback_expand_heap(state->ms, state->ms_len, state->elem, state->map_addr, state->map_len); -- 2.17.2 ^ permalink raw reply [flat|nested] 7+ messages in thread
* [dpdk-stable] [PATCH v2] malloc: notify primary process about hotplug in secondary [not found] <20181204170610.250124-1-seth.howell@intel.com> 2018-12-07 20:06 ` [dpdk-stable] [PATCH] malloc: notify primary process about hotplug in secondary Seth Howell @ 2018-12-07 20:10 ` Seth Howell 2018-12-07 20:30 ` Stojaczyk, Dariusz 2018-12-10 10:49 ` Burakov, Anatoly 1 sibling, 2 replies; 7+ messages in thread From: Seth Howell @ 2018-12-07 20:10 UTC (permalink / raw) To: anatoly.burakov; +Cc: dev, stable, Seth Howell, Darek Stojaczyk When secondary process hotplugs memory, it sends a request to primary, which then performs the real mmap() and sends sync requests to all secondary processes. Upon receiving such sync request, each secondary process will notify the upper layers of hotplugged memory (and will call all locally registered event callbacks). In the end we'll end up with memory event callbacks fired in all the processes except the primary, which is a bug. This gets critical if memory is hotplugged while a VFIO device is attached, as the VFIO memory registration - which is done from a memory event callback present in the primary process only - is never called. After this patch, a primary process fires memory event callbacks before secondary processes start their synchronizations - both for hotplug and hotremove. Fixes: 07dcbfe0101f ("malloc: support multiprocess memory hotplug") Signed-off-by: Seth Howell <seth.howell@intel.com> Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com> --- lib/librte_eal/common/malloc_mp.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/lib/librte_eal/common/malloc_mp.c b/lib/librte_eal/common/malloc_mp.c index 5f2d4e0be..f3a13353b 100644 --- a/lib/librte_eal/common/malloc_mp.c +++ b/lib/librte_eal/common/malloc_mp.c @@ -209,6 +209,8 @@ handle_alloc_request(const struct malloc_mp_req *m, map_addr = ms[0]->addr; + eal_memalloc_mem_event_notify(RTE_MEM_EVENT_ALLOC, map_addr, alloc_sz); + /* we have succeeded in allocating memory, but we still need to sync * with other processes. however, since DPDK IPC is single-threaded, we * send an asynchronous request and exit this callback. @@ -258,6 +260,9 @@ handle_request(const struct rte_mp_msg *msg, const void *peer __rte_unused) if (m->t == REQ_TYPE_ALLOC) { ret = handle_alloc_request(m, entry); } else if (m->t == REQ_TYPE_FREE) { + eal_memalloc_mem_event_notify(RTE_MEM_EVENT_FREE, + m->free_req.addr, m->free_req.len); + ret = malloc_heap_free_pages(m->free_req.addr, m->free_req.len); } else { @@ -436,6 +441,9 @@ handle_sync_response(const struct rte_mp_msg *request, memset(&rb_msg, 0, sizeof(rb_msg)); /* we've failed to sync, so do a rollback */ + eal_memalloc_mem_event_notify(RTE_MEM_EVENT_FREE, + state->map_addr, state->map_len); + rollback_expand_heap(state->ms, state->ms_len, state->elem, state->map_addr, state->map_len); -- 2.17.2 ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [dpdk-stable] [PATCH v2] malloc: notify primary process about hotplug in secondary 2018-12-07 20:10 ` [dpdk-stable] [PATCH v2] " Seth Howell @ 2018-12-07 20:30 ` Stojaczyk, Dariusz 2018-12-08 17:02 ` Kevin Traynor 2018-12-20 14:24 ` Thomas Monjalon 2018-12-10 10:49 ` Burakov, Anatoly 1 sibling, 2 replies; 7+ messages in thread From: Stojaczyk, Dariusz @ 2018-12-07 20:30 UTC (permalink / raw) To: Kevin Traynor; +Cc: stable, Howell, Seth, Burakov, Anatoly Hi Kevin, is the merge window for DPDK 18.08.1 and 18.05.2 still open? This fix is critical for multi-process memory hotplug and you might want to pull it in. D. > -----Original Message----- > From: Howell, Seth > Sent: Friday, December 7, 2018 9:11 PM > To: Burakov, Anatoly <anatoly.burakov@intel.com> > Cc: dev@dpdk.org; stable@dpdk.org; Howell, Seth > <seth.howell@intel.com>; Stojaczyk, Dariusz <dariusz.stojaczyk@intel.com> > Subject: [PATCH v2] malloc: notify primary process about hotplug in > secondary > > When secondary process hotplugs memory, it sends a request > to primary, which then performs the real mmap() and sends > sync requests to all secondary processes. Upon receiving > such sync request, each secondary process will notify the > upper layers of hotplugged memory (and will call all > locally registered event callbacks). > > In the end we'll end up with memory event callbacks fired > in all the processes except the primary, which is a bug. > > This gets critical if memory is hotplugged while a VFIO > device is attached, as the VFIO memory registration - > which is done from a memory event callback present in the > primary process only - is never called. > > After this patch, a primary process fires memory event > callbacks before secondary processes start their > synchronizations - both for hotplug and hotremove. > > Fixes: 07dcbfe0101f ("malloc: support multiprocess memory hotplug") > > Signed-off-by: Seth Howell <seth.howell@intel.com> > Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com> > --- > lib/librte_eal/common/malloc_mp.c | 8 ++++++++ > 1 file changed, 8 insertions(+) > > diff --git a/lib/librte_eal/common/malloc_mp.c > b/lib/librte_eal/common/malloc_mp.c > index 5f2d4e0be..f3a13353b 100644 > --- a/lib/librte_eal/common/malloc_mp.c > +++ b/lib/librte_eal/common/malloc_mp.c > @@ -209,6 +209,8 @@ handle_alloc_request(const struct malloc_mp_req > *m, > > map_addr = ms[0]->addr; > > + eal_memalloc_mem_event_notify(RTE_MEM_EVENT_ALLOC, > map_addr, alloc_sz); > + > /* we have succeeded in allocating memory, but we still need to sync > * with other processes. however, since DPDK IPC is single-threaded, > we > * send an asynchronous request and exit this callback. > @@ -258,6 +260,9 @@ handle_request(const struct rte_mp_msg *msg, > const void *peer __rte_unused) > if (m->t == REQ_TYPE_ALLOC) { > ret = handle_alloc_request(m, entry); > } else if (m->t == REQ_TYPE_FREE) { > + eal_memalloc_mem_event_notify(RTE_MEM_EVENT_FREE, > + m->free_req.addr, m->free_req.len); > + > ret = malloc_heap_free_pages(m->free_req.addr, > m->free_req.len); > } else { > @@ -436,6 +441,9 @@ handle_sync_response(const struct rte_mp_msg > *request, > memset(&rb_msg, 0, sizeof(rb_msg)); > > /* we've failed to sync, so do a rollback */ > + eal_memalloc_mem_event_notify(RTE_MEM_EVENT_FREE, > + state->map_addr, state->map_len); > + > rollback_expand_heap(state->ms, state->ms_len, state- > >elem, > state->map_addr, state->map_len); > > -- > 2.17.2 ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [dpdk-stable] [PATCH v2] malloc: notify primary process about hotplug in secondary 2018-12-07 20:30 ` Stojaczyk, Dariusz @ 2018-12-08 17:02 ` Kevin Traynor 2018-12-20 14:24 ` Thomas Monjalon 1 sibling, 0 replies; 7+ messages in thread From: Kevin Traynor @ 2018-12-08 17:02 UTC (permalink / raw) To: Stojaczyk, Dariusz Cc: stable, Howell, Seth, Burakov, Anatoly, Christian Ehrhardt On 12/07/2018 12:30 PM, Stojaczyk, Dariusz wrote: > Hi Kevin, is the merge window for DPDK 18.08.1 and 18.05.2 still open? This fix is critical for multi-process memory hotplug and you might want to pull it in. > D. > Hi Dariusz, I can still take patches for 18.08.1 but the patch must be applied in the main tree before it can be backported. DPDK 18.05 is not maintained anymore. >> -----Original Message----- >> From: Howell, Seth >> Sent: Friday, December 7, 2018 9:11 PM >> To: Burakov, Anatoly <anatoly.burakov@intel.com> >> Cc: dev@dpdk.org; stable@dpdk.org; Howell, Seth >> <seth.howell@intel.com>; Stojaczyk, Dariusz <dariusz.stojaczyk@intel.com> >> Subject: [PATCH v2] malloc: notify primary process about hotplug in >> secondary >> >> When secondary process hotplugs memory, it sends a request >> to primary, which then performs the real mmap() and sends >> sync requests to all secondary processes. Upon receiving >> such sync request, each secondary process will notify the >> upper layers of hotplugged memory (and will call all >> locally registered event callbacks). >> >> In the end we'll end up with memory event callbacks fired >> in all the processes except the primary, which is a bug. >> >> This gets critical if memory is hotplugged while a VFIO >> device is attached, as the VFIO memory registration - >> which is done from a memory event callback present in the >> primary process only - is never called. >> >> After this patch, a primary process fires memory event >> callbacks before secondary processes start their >> synchronizations - both for hotplug and hotremove. >> >> Fixes: 07dcbfe0101f ("malloc: support multiprocess memory hotplug") In this case I know because you mailed, but for future, 'Cc: stable@dpdk.org' tag should also be inserted here in the commit message so it can be found by scripts. Kevin. >> >> Signed-off-by: Seth Howell <seth.howell@intel.com> >> Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com> >> --- >> lib/librte_eal/common/malloc_mp.c | 8 ++++++++ >> 1 file changed, 8 insertions(+) >> >> diff --git a/lib/librte_eal/common/malloc_mp.c >> b/lib/librte_eal/common/malloc_mp.c >> index 5f2d4e0be..f3a13353b 100644 >> --- a/lib/librte_eal/common/malloc_mp.c >> +++ b/lib/librte_eal/common/malloc_mp.c >> @@ -209,6 +209,8 @@ handle_alloc_request(const struct malloc_mp_req >> *m, >> >> map_addr = ms[0]->addr; >> >> + eal_memalloc_mem_event_notify(RTE_MEM_EVENT_ALLOC, >> map_addr, alloc_sz); >> + >> /* we have succeeded in allocating memory, but we still need to sync >> * with other processes. however, since DPDK IPC is single-threaded, >> we >> * send an asynchronous request and exit this callback. >> @@ -258,6 +260,9 @@ handle_request(const struct rte_mp_msg *msg, >> const void *peer __rte_unused) >> if (m->t == REQ_TYPE_ALLOC) { >> ret = handle_alloc_request(m, entry); >> } else if (m->t == REQ_TYPE_FREE) { >> + eal_memalloc_mem_event_notify(RTE_MEM_EVENT_FREE, >> + m->free_req.addr, m->free_req.len); >> + >> ret = malloc_heap_free_pages(m->free_req.addr, >> m->free_req.len); >> } else { >> @@ -436,6 +441,9 @@ handle_sync_response(const struct rte_mp_msg >> *request, >> memset(&rb_msg, 0, sizeof(rb_msg)); >> >> /* we've failed to sync, so do a rollback */ >> + eal_memalloc_mem_event_notify(RTE_MEM_EVENT_FREE, >> + state->map_addr, state->map_len); >> + >> rollback_expand_heap(state->ms, state->ms_len, state- >>> elem, >> state->map_addr, state->map_len); >> >> -- >> 2.17.2 > ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [dpdk-stable] [PATCH v2] malloc: notify primary process about hotplug in secondary 2018-12-07 20:30 ` Stojaczyk, Dariusz 2018-12-08 17:02 ` Kevin Traynor @ 2018-12-20 14:24 ` Thomas Monjalon 1 sibling, 0 replies; 7+ messages in thread From: Thomas Monjalon @ 2018-12-20 14:24 UTC (permalink / raw) To: Stojaczyk, Dariusz, Howell, Seth; +Cc: stable, Kevin Traynor, Burakov, Anatoly 07/12/2018 21:30, Stojaczyk, Dariusz: > Hi Kevin, is the merge window for DPDK 18.08.1 and 18.05.2 still open? This fix is critical for multi-process memory hotplug and you might want to pull it in. First thing to do is to make sure stable@dpdk.org is Cc'ed in the patch, below Fixes:. > > -----Original Message----- > > From: Howell, Seth > > Sent: Friday, December 7, 2018 9:11 PM > > To: Burakov, Anatoly <anatoly.burakov@intel.com> > > Cc: dev@dpdk.org; stable@dpdk.org; Howell, Seth > > <seth.howell@intel.com>; Stojaczyk, Dariusz <dariusz.stojaczyk@intel.com> > > Subject: [PATCH v2] malloc: notify primary process about hotplug in > > secondary > > > > When secondary process hotplugs memory, it sends a request > > to primary, which then performs the real mmap() and sends > > sync requests to all secondary processes. Upon receiving > > such sync request, each secondary process will notify the > > upper layers of hotplugged memory (and will call all > > locally registered event callbacks). > > > > In the end we'll end up with memory event callbacks fired > > in all the processes except the primary, which is a bug. > > > > This gets critical if memory is hotplugged while a VFIO > > device is attached, as the VFIO memory registration - > > which is done from a memory event callback present in the > > primary process only - is never called. > > > > After this patch, a primary process fires memory event > > callbacks before secondary processes start their > > synchronizations - both for hotplug and hotremove. > > > > Fixes: 07dcbfe0101f ("malloc: support multiprocess memory hotplug") > > > > Signed-off-by: Seth Howell <seth.howell@intel.com> > > Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com> > > --- > > lib/librte_eal/common/malloc_mp.c | 8 ++++++++ > > 1 file changed, 8 insertions(+) > > > > diff --git a/lib/librte_eal/common/malloc_mp.c > > b/lib/librte_eal/common/malloc_mp.c > > index 5f2d4e0be..f3a13353b 100644 > > --- a/lib/librte_eal/common/malloc_mp.c > > +++ b/lib/librte_eal/common/malloc_mp.c > > @@ -209,6 +209,8 @@ handle_alloc_request(const struct malloc_mp_req > > *m, > > > > map_addr = ms[0]->addr; > > > > + eal_memalloc_mem_event_notify(RTE_MEM_EVENT_ALLOC, > > map_addr, alloc_sz); > > + > > /* we have succeeded in allocating memory, but we still need to sync > > * with other processes. however, since DPDK IPC is single-threaded, > > we > > * send an asynchronous request and exit this callback. > > @@ -258,6 +260,9 @@ handle_request(const struct rte_mp_msg *msg, > > const void *peer __rte_unused) > > if (m->t == REQ_TYPE_ALLOC) { > > ret = handle_alloc_request(m, entry); > > } else if (m->t == REQ_TYPE_FREE) { > > + eal_memalloc_mem_event_notify(RTE_MEM_EVENT_FREE, > > + m->free_req.addr, m->free_req.len); > > + > > ret = malloc_heap_free_pages(m->free_req.addr, > > m->free_req.len); > > } else { > > @@ -436,6 +441,9 @@ handle_sync_response(const struct rte_mp_msg > > *request, > > memset(&rb_msg, 0, sizeof(rb_msg)); > > > > /* we've failed to sync, so do a rollback */ > > + eal_memalloc_mem_event_notify(RTE_MEM_EVENT_FREE, > > + state->map_addr, state->map_len); > > + > > rollback_expand_heap(state->ms, state->ms_len, state- > > >elem, > > state->map_addr, state->map_len); > > > > -- > > 2.17.2 > ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [dpdk-stable] [PATCH v2] malloc: notify primary process about hotplug in secondary 2018-12-07 20:10 ` [dpdk-stable] [PATCH v2] " Seth Howell 2018-12-07 20:30 ` Stojaczyk, Dariusz @ 2018-12-10 10:49 ` Burakov, Anatoly 2018-12-20 14:26 ` Thomas Monjalon 1 sibling, 1 reply; 7+ messages in thread From: Burakov, Anatoly @ 2018-12-10 10:49 UTC (permalink / raw) To: Seth Howell; +Cc: dev, stable, Darek Stojaczyk On 07-Dec-18 8:10 PM, Seth Howell wrote: > When secondary process hotplugs memory, it sends a request > to primary, which then performs the real mmap() and sends > sync requests to all secondary processes. Upon receiving > such sync request, each secondary process will notify the > upper layers of hotplugged memory (and will call all > locally registered event callbacks). > > In the end we'll end up with memory event callbacks fired > in all the processes except the primary, which is a bug. > > This gets critical if memory is hotplugged while a VFIO > device is attached, as the VFIO memory registration - > which is done from a memory event callback present in the > primary process only - is never called. > > After this patch, a primary process fires memory event > callbacks before secondary processes start their > synchronizations - both for hotplug and hotremove. > > Fixes: 07dcbfe0101f ("malloc: support multiprocess memory hotplug") > > Signed-off-by: Seth Howell <seth.howell@intel.com> > Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com> > --- Lost my review tag... Reviewed-by: Anatoly Burakov <anatoly.burakov@intel.com> -- Thanks, Anatoly ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [dpdk-stable] [PATCH v2] malloc: notify primary process about hotplug in secondary 2018-12-10 10:49 ` Burakov, Anatoly @ 2018-12-20 14:26 ` Thomas Monjalon 0 siblings, 0 replies; 7+ messages in thread From: Thomas Monjalon @ 2018-12-20 14:26 UTC (permalink / raw) To: Seth Howell; +Cc: stable, Burakov, Anatoly, dev, Darek Stojaczyk, ktraynor 10/12/2018 11:49, Burakov, Anatoly: > On 07-Dec-18 8:10 PM, Seth Howell wrote: > > When secondary process hotplugs memory, it sends a request > > to primary, which then performs the real mmap() and sends > > sync requests to all secondary processes. Upon receiving > > such sync request, each secondary process will notify the > > upper layers of hotplugged memory (and will call all > > locally registered event callbacks). > > > > In the end we'll end up with memory event callbacks fired > > in all the processes except the primary, which is a bug. > > > > This gets critical if memory is hotplugged while a VFIO > > device is attached, as the VFIO memory registration - > > which is done from a memory event callback present in the > > primary process only - is never called. > > > > After this patch, a primary process fires memory event > > callbacks before secondary processes start their > > synchronizations - both for hotplug and hotremove. > > > > Fixes: 07dcbfe0101f ("malloc: support multiprocess memory hotplug") > > > > Signed-off-by: Seth Howell <seth.howell@intel.com> > > Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com> > > --- > > Lost my review tag... > > Reviewed-by: Anatoly Burakov <anatoly.burakov@intel.com> +Cc: stable@dpdk.org Applied, thanks ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2018-12-20 14:26 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <20181204170610.250124-1-seth.howell@intel.com>
2018-12-07 20:06 ` [dpdk-stable] [PATCH] malloc: notify primary process about hotplug in secondary Seth Howell
2018-12-07 20:10 ` [dpdk-stable] [PATCH v2] " Seth Howell
2018-12-07 20:30 ` Stojaczyk, Dariusz
2018-12-08 17:02 ` Kevin Traynor
2018-12-20 14:24 ` Thomas Monjalon
2018-12-10 10:49 ` Burakov, Anatoly
2018-12-20 14:26 ` Thomas Monjalon
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).