From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by dpdk.org (Postfix) with ESMTP id E26D04C88 for ; Wed, 28 Feb 2018 02:09:30 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 27 Feb 2018 17:09:28 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.47,403,1515484800"; d="scan'208";a="34652316" Received: from fmsmsx104.amr.corp.intel.com ([10.18.124.202]) by orsmga001.jf.intel.com with ESMTP; 27 Feb 2018 17:09:27 -0800 Received: from fmsmsx117.amr.corp.intel.com (10.18.116.17) by fmsmsx104.amr.corp.intel.com (10.18.124.202) with Microsoft SMTP Server (TLS) id 14.3.319.2; Tue, 27 Feb 2018 17:09:05 -0800 Received: from shsmsx101.ccr.corp.intel.com (10.239.4.153) by fmsmsx117.amr.corp.intel.com (10.18.116.17) with Microsoft SMTP Server (TLS) id 14.3.319.2; Tue, 27 Feb 2018 17:09:05 -0800 Received: from shsmsx103.ccr.corp.intel.com ([169.254.4.116]) by SHSMSX101.ccr.corp.intel.com ([169.254.1.253]) with mapi id 14.03.0319.002; Wed, 28 Feb 2018 09:09:03 +0800 From: "Tan, Jianfeng" To: "Burakov, Anatoly" , "dev@dpdk.org" Thread-Topic: [PATCH v3 2/5] eal: don't process IPC messages before init finished Thread-Index: AQHTr9hBqvNgpcB3XkS/Y0yxePkT66O5AIWg Date: Wed, 28 Feb 2018 01:09:03 +0000 Message-ID: References: <31f6d9ef676fb1eb0a664c06d62d66f32876dcb6.1519672713.git.anatoly.burakov@intel.com> <3903de6f3824e5063d49936e743e22dd819bad09.1519740527.git.anatoly.burakov@intel.com> In-Reply-To: <3903de6f3824e5063d49936e743e22dd819bad09.1519740527.git.anatoly.burakov@intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.239.127.40] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH v3 2/5] eal: don't process IPC messages before init finished X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Feb 2018 01:09:31 -0000 > -----Original Message----- > From: Burakov, Anatoly > Sent: Tuesday, February 27, 2018 10:36 PM > To: dev@dpdk.org > Cc: Tan, Jianfeng > Subject: [PATCH v3 2/5] eal: don't process IPC messages before init finis= hed >=20 > It is not possible for a primary process to receive any messages > while initializing, because RTE_MAGIC value is not set in the > shared config, and hence no secondary process can ever spin up > during that time. >=20 > However, it is possible for a secondary process to receive messages > from the primary during initialization. We can't just drop the > messages as they may be important, and also we might need to process > replies to our own requests (e.g. VFIO) during initialization. >=20 > Therefore, add a tailq for incoming messages, and queue them up > until initialization is complete, and process them in order they > arrived. >=20 > Signed-off-by: Anatoly Burakov > --- >=20 > Notes: > v3: check for init_complete after receiving message >=20 > v2: no changes >=20 > lib/librte_eal/common/eal_common_proc.c | 52 > +++++++++++++++++++++++++++++---- > 1 file changed, 47 insertions(+), 5 deletions(-) >=20 > diff --git a/lib/librte_eal/common/eal_common_proc.c > b/lib/librte_eal/common/eal_common_proc.c > index 3a1088e..a6e24e6 100644 > --- a/lib/librte_eal/common/eal_common_proc.c > +++ b/lib/librte_eal/common/eal_common_proc.c > @@ -25,6 +25,7 @@ > #include > #include > #include > +#include >=20 > #include "eal_private.h" > #include "eal_filesystem.h" > @@ -58,6 +59,18 @@ struct mp_msg_internal { > struct rte_mp_msg msg; > }; >=20 > +struct message_queue_entry { > + TAILQ_ENTRY(message_queue_entry) next; > + struct mp_msg_internal msg; > + struct sockaddr_un sa; > +}; > + > +/** Double linked list of received messages. */ > +TAILQ_HEAD(message_queue, message_queue_entry); > + > +static struct message_queue message_queue =3D > + TAILQ_HEAD_INITIALIZER(message_queue); > + > struct sync_request { > TAILQ_ENTRY(sync_request) next; > int reply_received; > @@ -276,12 +289,41 @@ process_msg(struct mp_msg_internal *m, struct > sockaddr_un *s) > static void * > mp_handle(void *arg __rte_unused) > { > - struct mp_msg_internal msg; > - struct sockaddr_un sa; > - > + struct message_queue_entry *cur_msg, *next_msg, *new_msg =3D > NULL; > while (1) { > - if (read_msg(&msg, &sa) =3D=3D 0) > - process_msg(&msg, &sa); > + /* we want to process all messages in order of their arrival, > + * but status of init_complete may change while we're > iterating > + * the tailq. so, store it here and check once every iteration. > + */ > + int init_complete; > + > + if (new_msg =3D=3D NULL) > + new_msg =3D malloc(sizeof(*new_msg)); > + if (read_msg(&new_msg->msg, &new_msg->sa) =3D=3D 0) { Suppose a case that: req and msg received but init not completed, so we enq= ueue all of them in the tailq; and from now on, no req/rep/msg comes. Then = mp thread will hang here for reading new message. In such a case, we might need the master thread to signal mp thread to wake= up when init is completed? Thanks, Jianfeng > + /* we successfully read the message, so enqueue it > */ > + TAILQ_INSERT_TAIL(&message_queue, new_msg, > next); > + new_msg =3D NULL; > + } /* reuse new_msg for next message if we couldn't > read_msg */ > + > + init_complete =3D internal_config.init_complete; > + > + /* tailq only accessed here, so no locking needed */ > + TAILQ_FOREACH_SAFE(cur_msg, &message_queue, next, > next_msg) { > + /* secondary process should not process any > incoming > + * requests until its initialization is complete, but > + * it is allowed to process replies to its own queries. > + */ > + if (rte_eal_process_type() =3D=3D > RTE_PROC_SECONDARY && > + !init_complete && > + cur_msg->msg.type !=3D MP_REP) > + continue; > + > + TAILQ_REMOVE(&message_queue, cur_msg, next); > + > + process_msg(&cur_msg->msg, &cur_msg->sa); > + > + free(cur_msg); > + } > } >=20 > return NULL; > -- > 2.7.4