From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by dpdk.org (Postfix) with ESMTP id 9694D1B277 for ; Tue, 4 Dec 2018 18:07:04 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga104.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 04 Dec 2018 09:07:03 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,314,1539673200"; d="scan'208";a="96067521" Received: from sethhowe-desk.ch.intel.com ([143.182.137.120]) by orsmga007.jf.intel.com with ESMTP; 04 Dec 2018 09:07:00 -0800 From: Seth Howell To: anatoly.burakov@intel.com Cc: dev@dpdk.org, Seth Howell , Darek Stojaczyk Date: Tue, 4 Dec 2018 10:06:10 -0700 Message-Id: <20181204170610.250124-1-seth.howell@intel.com> X-Mailer: git-send-email 2.17.2 Subject: [dpdk-dev] [PATCH] malloc: notify primary process about hotplug in secondary X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Dec 2018 17:07:05 -0000 When secondary process hotplugs memory, it sends a request to primary, which then performs the real mmap() and sends sync requests to all secondary processes. Upon receiving such sync request, each secondary process will notify the upper layers of hotplugged memory (and will call all locally registered event callbacks). In the end we'll end up with memory event callbacks fired in all the processes except the primary, which is a bug. This gets critical if memory is hotplugged while a VFIO device is attached, as the VFIO memory registration - which is done from a memory event callback present in the primary process only - is never called. After this patch, a primary process fires memory event callbacks before secondary processes start their synchronizations - both for hotplug and hotremove. Change-Id: I60de33913f58bc2454069c3844826c92cb043fff Signed-off-by: Seth Howell Signed-off-by: Darek Stojaczyk --- lib/librte_eal/common/malloc_mp.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/lib/librte_eal/common/malloc_mp.c b/lib/librte_eal/common/malloc_mp.c index 5f2d4e0be..f3a13353b 100644 --- a/lib/librte_eal/common/malloc_mp.c +++ b/lib/librte_eal/common/malloc_mp.c @@ -209,6 +209,8 @@ handle_alloc_request(const struct malloc_mp_req *m, map_addr = ms[0]->addr; + eal_memalloc_mem_event_notify(RTE_MEM_EVENT_ALLOC, map_addr, alloc_sz); + /* we have succeeded in allocating memory, but we still need to sync * with other processes. however, since DPDK IPC is single-threaded, we * send an asynchronous request and exit this callback. @@ -258,6 +260,9 @@ handle_request(const struct rte_mp_msg *msg, const void *peer __rte_unused) if (m->t == REQ_TYPE_ALLOC) { ret = handle_alloc_request(m, entry); } else if (m->t == REQ_TYPE_FREE) { + eal_memalloc_mem_event_notify(RTE_MEM_EVENT_FREE, + m->free_req.addr, m->free_req.len); + ret = malloc_heap_free_pages(m->free_req.addr, m->free_req.len); } else { @@ -436,6 +441,9 @@ handle_sync_response(const struct rte_mp_msg *request, memset(&rb_msg, 0, sizeof(rb_msg)); /* we've failed to sync, so do a rollback */ + eal_memalloc_mem_event_notify(RTE_MEM_EVENT_FREE, + state->map_addr, state->map_len); + rollback_expand_heap(state->ms, state->ms_len, state->elem, state->map_addr, state->map_len); -- 2.17.2