From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by dpdk.org (Postfix) with ESMTP id 27F004CB5 for ; Tue, 27 Feb 2018 15:35:44 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga104.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 27 Feb 2018 06:35:43 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.47,401,1515484800"; d="scan'208";a="21168565" Received: from irvmail001.ir.intel.com ([163.33.26.43]) by orsmga008.jf.intel.com with ESMTP; 27 Feb 2018 06:35:42 -0800 Received: from sivswdev01.ir.intel.com (sivswdev01.ir.intel.com [10.237.217.45]) by irvmail001.ir.intel.com (8.14.3/8.13.6/MailSET/Hub) with ESMTP id w1REZfCv011879; Tue, 27 Feb 2018 14:35:41 GMT Received: from sivswdev01.ir.intel.com (localhost [127.0.0.1]) by sivswdev01.ir.intel.com with ESMTP id w1REZfUS019121; Tue, 27 Feb 2018 14:35:41 GMT Received: (from aburakov@localhost) by sivswdev01.ir.intel.com with LOCAL id w1REZfQI019116; Tue, 27 Feb 2018 14:35:41 GMT From: Anatoly Burakov To: dev@dpdk.org Cc: jianfeng.tan@intel.com Date: Tue, 27 Feb 2018 14:35:38 +0000 Message-Id: <3903de6f3824e5063d49936e743e22dd819bad09.1519740527.git.anatoly.burakov@intel.com> X-Mailer: git-send-email 1.7.0.7 In-Reply-To: <3f4fab71b0d17fb9d0b15e3634f4fe7d12dd3e92.1519740527.git.anatoly.burakov@intel.com> References: <3f4fab71b0d17fb9d0b15e3634f4fe7d12dd3e92.1519740527.git.anatoly.burakov@intel.com> In-Reply-To: <31f6d9ef676fb1eb0a664c06d62d66f32876dcb6.1519672713.git.anatoly.burakov@intel.com> References: <31f6d9ef676fb1eb0a664c06d62d66f32876dcb6.1519672713.git.anatoly.burakov@intel.com> Subject: [dpdk-dev] [PATCH v3 2/5] eal: don't process IPC messages before init finished X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Feb 2018 14:35:45 -0000 It is not possible for a primary process to receive any messages while initializing, because RTE_MAGIC value is not set in the shared config, and hence no secondary process can ever spin up during that time. However, it is possible for a secondary process to receive messages from the primary during initialization. We can't just drop the messages as they may be important, and also we might need to process replies to our own requests (e.g. VFIO) during initialization. Therefore, add a tailq for incoming messages, and queue them up until initialization is complete, and process them in order they arrived. Signed-off-by: Anatoly Burakov --- Notes: v3: check for init_complete after receiving message v2: no changes lib/librte_eal/common/eal_common_proc.c | 52 +++++++++++++++++++++++++++++---- 1 file changed, 47 insertions(+), 5 deletions(-) diff --git a/lib/librte_eal/common/eal_common_proc.c b/lib/librte_eal/common/eal_common_proc.c index 3a1088e..a6e24e6 100644 --- a/lib/librte_eal/common/eal_common_proc.c +++ b/lib/librte_eal/common/eal_common_proc.c @@ -25,6 +25,7 @@ #include #include #include +#include #include "eal_private.h" #include "eal_filesystem.h" @@ -58,6 +59,18 @@ struct mp_msg_internal { struct rte_mp_msg msg; }; +struct message_queue_entry { + TAILQ_ENTRY(message_queue_entry) next; + struct mp_msg_internal msg; + struct sockaddr_un sa; +}; + +/** Double linked list of received messages. */ +TAILQ_HEAD(message_queue, message_queue_entry); + +static struct message_queue message_queue = + TAILQ_HEAD_INITIALIZER(message_queue); + struct sync_request { TAILQ_ENTRY(sync_request) next; int reply_received; @@ -276,12 +289,41 @@ process_msg(struct mp_msg_internal *m, struct sockaddr_un *s) static void * mp_handle(void *arg __rte_unused) { - struct mp_msg_internal msg; - struct sockaddr_un sa; - + struct message_queue_entry *cur_msg, *next_msg, *new_msg = NULL; while (1) { - if (read_msg(&msg, &sa) == 0) - process_msg(&msg, &sa); + /* we want to process all messages in order of their arrival, + * but status of init_complete may change while we're iterating + * the tailq. so, store it here and check once every iteration. + */ + int init_complete; + + if (new_msg == NULL) + new_msg = malloc(sizeof(*new_msg)); + if (read_msg(&new_msg->msg, &new_msg->sa) == 0) { + /* we successfully read the message, so enqueue it */ + TAILQ_INSERT_TAIL(&message_queue, new_msg, next); + new_msg = NULL; + } /* reuse new_msg for next message if we couldn't read_msg */ + + init_complete = internal_config.init_complete; + + /* tailq only accessed here, so no locking needed */ + TAILQ_FOREACH_SAFE(cur_msg, &message_queue, next, next_msg) { + /* secondary process should not process any incoming + * requests until its initialization is complete, but + * it is allowed to process replies to its own queries. + */ + if (rte_eal_process_type() == RTE_PROC_SECONDARY && + !init_complete && + cur_msg->msg.type != MP_REP) + continue; + + TAILQ_REMOVE(&message_queue, cur_msg, next); + + process_msg(&cur_msg->msg, &cur_msg->sa); + + free(cur_msg); + } } return NULL; -- 2.7.4