From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by dpdk.org (Postfix) with ESMTP id C07DEA3; Fri, 3 May 2019 13:50:53 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga106.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 03 May 2019 04:50:53 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.60,425,1549958400"; d="scan'208";a="321118206" Received: from silpixa00399498.ir.intel.com (HELO silpixa00399498.ger.corp.intel.com) ([10.237.223.125]) by orsmga005.jf.intel.com with ESMTP; 03 May 2019 04:50:51 -0700 From: Anatoly Burakov To: dev@dpdk.org Cc: John McNamara , Marko Kovacevic , stable@dpdk.org Date: Fri, 3 May 2019 12:50:49 +0100 Message-Id: X-Mailer: git-send-email 2.17.1 In-Reply-To: References: In-Reply-To: References: Subject: [dpdk-dev] [PATCH 2/3] ipc: add warnings about not using ipc with memory API's X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 May 2019 11:50:54 -0000 IPC and memory-related API's should not be mixed because memory relies on IPC internally. Add explicit warnings to IPC API and to the documentation about this. Cc: stable@dpdk.org Signed-off-by: Anatoly Burakov --- doc/guides/prog_guide/env_abstraction_layer.rst | 8 ++++++++ doc/guides/prog_guide/multi_proc_support.rst | 5 +++++ lib/librte_eal/common/include/rte_eal.h | 7 +++++++ 3 files changed, 20 insertions(+) diff --git a/doc/guides/prog_guide/env_abstraction_layer.rst b/doc/guides/prog_guide/env_abstraction_layer.rst index c27f730c7..f15bcd976 100644 --- a/doc/guides/prog_guide/env_abstraction_layer.rst +++ b/doc/guides/prog_guide/env_abstraction_layer.rst @@ -147,6 +147,14 @@ A default validator callback is provided by EAL, which can be enabled with a ``--socket-limit`` command-line option, for a simple way to limit maximum amount of memory that can be used by DPDK application. +.. warning:: + Memory subsystem uses DPDK IPC internally, so memory allocations/callbacks + and IPC must not be mixed: it is not safe to allocate/free memory inside + memory-related or IPC callbacks, and it is not safe to use IPC inside + memory-related callbacks. See chapter + :ref:`Multi-process Support ` for more details about + DPDK IPC. + + Legacy memory mode This mode is enabled by specifying ``--legacy-mem`` command-line switch to the diff --git a/doc/guides/prog_guide/multi_proc_support.rst b/doc/guides/prog_guide/multi_proc_support.rst index 1e78a1f02..665474a4c 100644 --- a/doc/guides/prog_guide/multi_proc_support.rst +++ b/doc/guides/prog_guide/multi_proc_support.rst @@ -318,6 +318,11 @@ supported. However, since sending messages (not requests) does not involve an IPC thread, sending messages while processing another message or request is supported. +Since the memory sybsystem uses IPC internally, memory allocations and IPC must +not be mixed: it is not safe to use IPC inside a memory-related callback, nor is +it safe to allocate/free memory inside IPC callbacks. Attempting to do so may +lead to a deadlock. + Asynchronous request callbacks may be triggered either from IPC thread or from interrupt thread, depending on whether the request has timed out. It is therefore suggested to avoid waiting for interrupt-based events (such as alarms) diff --git a/lib/librte_eal/common/include/rte_eal.h b/lib/librte_eal/common/include/rte_eal.h index 0603eaf25..7db022532 100644 --- a/lib/librte_eal/common/include/rte_eal.h +++ b/lib/librte_eal/common/include/rte_eal.h @@ -225,6 +225,8 @@ struct rte_mp_reply { * * As we create socket channel for primary/secondary communication, use * this function typedef to register action for coming messages. + * + * @note No memory allocations should take place inside the callback. */ typedef int (*rte_mp_t)(const struct rte_mp_msg *msg, const void *peer); @@ -234,6 +236,8 @@ typedef int (*rte_mp_t)(const struct rte_mp_msg *msg, const void *peer); * As we create socket channel for primary/secondary communication, use * this function typedef to register action for coming responses to asynchronous * requests. + * + * @note No memory allocations should take place inside the callback. */ typedef int (*rte_mp_async_reply_t)(const struct rte_mp_msg *request, const struct rte_mp_reply *reply); @@ -308,6 +312,9 @@ rte_mp_sendmsg(struct rte_mp_msg *msg); * * @note The caller is responsible to free reply->replies. * + * @note This API must not be used inside memory-related or IPC callbacks, and + * no memory allocations should take place inside such callback. + * * @param req * The req argument contains the customized request message. * -- 2.17.1 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by dpdk.space (Postfix) with ESMTP id 9F10EA0AC5 for ; Fri, 3 May 2019 13:51:07 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 63BD34CC7; Fri, 3 May 2019 13:50:57 +0200 (CEST) Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by dpdk.org (Postfix) with ESMTP id C07DEA3; Fri, 3 May 2019 13:50:53 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga106.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 03 May 2019 04:50:53 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.60,425,1549958400"; d="scan'208";a="321118206" Received: from silpixa00399498.ir.intel.com (HELO silpixa00399498.ger.corp.intel.com) ([10.237.223.125]) by orsmga005.jf.intel.com with ESMTP; 03 May 2019 04:50:51 -0700 From: Anatoly Burakov To: dev@dpdk.org Cc: John McNamara , Marko Kovacevic , stable@dpdk.org Date: Fri, 3 May 2019 12:50:49 +0100 Message-Id: X-Mailer: git-send-email 2.17.1 In-Reply-To: References: In-Reply-To: References: Subject: [dpdk-dev] [PATCH 2/3] ipc: add warnings about not using ipc with memory API's X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Content-Type: text/plain; charset="UTF-8" Message-ID: <20190503115049.Y3R7YlDfzfE9CeU2eG2Vwa2IzxgSznwhPci0a6pFaaQ@z> IPC and memory-related API's should not be mixed because memory relies on IPC internally. Add explicit warnings to IPC API and to the documentation about this. Cc: stable@dpdk.org Signed-off-by: Anatoly Burakov --- doc/guides/prog_guide/env_abstraction_layer.rst | 8 ++++++++ doc/guides/prog_guide/multi_proc_support.rst | 5 +++++ lib/librte_eal/common/include/rte_eal.h | 7 +++++++ 3 files changed, 20 insertions(+) diff --git a/doc/guides/prog_guide/env_abstraction_layer.rst b/doc/guides/prog_guide/env_abstraction_layer.rst index c27f730c7..f15bcd976 100644 --- a/doc/guides/prog_guide/env_abstraction_layer.rst +++ b/doc/guides/prog_guide/env_abstraction_layer.rst @@ -147,6 +147,14 @@ A default validator callback is provided by EAL, which can be enabled with a ``--socket-limit`` command-line option, for a simple way to limit maximum amount of memory that can be used by DPDK application. +.. warning:: + Memory subsystem uses DPDK IPC internally, so memory allocations/callbacks + and IPC must not be mixed: it is not safe to allocate/free memory inside + memory-related or IPC callbacks, and it is not safe to use IPC inside + memory-related callbacks. See chapter + :ref:`Multi-process Support ` for more details about + DPDK IPC. + + Legacy memory mode This mode is enabled by specifying ``--legacy-mem`` command-line switch to the diff --git a/doc/guides/prog_guide/multi_proc_support.rst b/doc/guides/prog_guide/multi_proc_support.rst index 1e78a1f02..665474a4c 100644 --- a/doc/guides/prog_guide/multi_proc_support.rst +++ b/doc/guides/prog_guide/multi_proc_support.rst @@ -318,6 +318,11 @@ supported. However, since sending messages (not requests) does not involve an IPC thread, sending messages while processing another message or request is supported. +Since the memory sybsystem uses IPC internally, memory allocations and IPC must +not be mixed: it is not safe to use IPC inside a memory-related callback, nor is +it safe to allocate/free memory inside IPC callbacks. Attempting to do so may +lead to a deadlock. + Asynchronous request callbacks may be triggered either from IPC thread or from interrupt thread, depending on whether the request has timed out. It is therefore suggested to avoid waiting for interrupt-based events (such as alarms) diff --git a/lib/librte_eal/common/include/rte_eal.h b/lib/librte_eal/common/include/rte_eal.h index 0603eaf25..7db022532 100644 --- a/lib/librte_eal/common/include/rte_eal.h +++ b/lib/librte_eal/common/include/rte_eal.h @@ -225,6 +225,8 @@ struct rte_mp_reply { * * As we create socket channel for primary/secondary communication, use * this function typedef to register action for coming messages. + * + * @note No memory allocations should take place inside the callback. */ typedef int (*rte_mp_t)(const struct rte_mp_msg *msg, const void *peer); @@ -234,6 +236,8 @@ typedef int (*rte_mp_t)(const struct rte_mp_msg *msg, const void *peer); * As we create socket channel for primary/secondary communication, use * this function typedef to register action for coming responses to asynchronous * requests. + * + * @note No memory allocations should take place inside the callback. */ typedef int (*rte_mp_async_reply_t)(const struct rte_mp_msg *request, const struct rte_mp_reply *reply); @@ -308,6 +312,9 @@ rte_mp_sendmsg(struct rte_mp_msg *msg); * * @note The caller is responsible to free reply->replies. * + * @note This API must not be used inside memory-related or IPC callbacks, and + * no memory allocations should take place inside such callback. + * * @param req * The req argument contains the customized request message. * -- 2.17.1