From: "Tan, Jianfeng" <jianfeng.tan@intel.com>
To: "Burakov, Anatoly" <anatoly.burakov@intel.com>,
"dev@dpdk.org" <dev@dpdk.org>
Subject: Re: [dpdk-dev] [PATCH v3 2/5] eal: don't process IPC messages before init finished
Date: Wed, 28 Feb 2018 01:09:03 +0000 [thread overview]
Message-ID: <ED26CBA2FAD1BF48A8719AEF02201E3651455398@SHSMSX103.ccr.corp.intel.com> (raw)
In-Reply-To: <3903de6f3824e5063d49936e743e22dd819bad09.1519740527.git.anatoly.burakov@intel.com>
> -----Original Message-----
> From: Burakov, Anatoly
> Sent: Tuesday, February 27, 2018 10:36 PM
> To: dev@dpdk.org
> Cc: Tan, Jianfeng
> Subject: [PATCH v3 2/5] eal: don't process IPC messages before init finished
>
> It is not possible for a primary process to receive any messages
> while initializing, because RTE_MAGIC value is not set in the
> shared config, and hence no secondary process can ever spin up
> during that time.
>
> However, it is possible for a secondary process to receive messages
> from the primary during initialization. We can't just drop the
> messages as they may be important, and also we might need to process
> replies to our own requests (e.g. VFIO) during initialization.
>
> Therefore, add a tailq for incoming messages, and queue them up
> until initialization is complete, and process them in order they
> arrived.
>
> Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
> ---
>
> Notes:
> v3: check for init_complete after receiving message
>
> v2: no changes
>
> lib/librte_eal/common/eal_common_proc.c | 52
> +++++++++++++++++++++++++++++----
> 1 file changed, 47 insertions(+), 5 deletions(-)
>
> diff --git a/lib/librte_eal/common/eal_common_proc.c
> b/lib/librte_eal/common/eal_common_proc.c
> index 3a1088e..a6e24e6 100644
> --- a/lib/librte_eal/common/eal_common_proc.c
> +++ b/lib/librte_eal/common/eal_common_proc.c
> @@ -25,6 +25,7 @@
> #include <rte_errno.h>
> #include <rte_lcore.h>
> #include <rte_log.h>
> +#include <rte_tailq.h>
>
> #include "eal_private.h"
> #include "eal_filesystem.h"
> @@ -58,6 +59,18 @@ struct mp_msg_internal {
> struct rte_mp_msg msg;
> };
>
> +struct message_queue_entry {
> + TAILQ_ENTRY(message_queue_entry) next;
> + struct mp_msg_internal msg;
> + struct sockaddr_un sa;
> +};
> +
> +/** Double linked list of received messages. */
> +TAILQ_HEAD(message_queue, message_queue_entry);
> +
> +static struct message_queue message_queue =
> + TAILQ_HEAD_INITIALIZER(message_queue);
> +
> struct sync_request {
> TAILQ_ENTRY(sync_request) next;
> int reply_received;
> @@ -276,12 +289,41 @@ process_msg(struct mp_msg_internal *m, struct
> sockaddr_un *s)
> static void *
> mp_handle(void *arg __rte_unused)
> {
> - struct mp_msg_internal msg;
> - struct sockaddr_un sa;
> -
> + struct message_queue_entry *cur_msg, *next_msg, *new_msg =
> NULL;
> while (1) {
> - if (read_msg(&msg, &sa) == 0)
> - process_msg(&msg, &sa);
> + /* we want to process all messages in order of their arrival,
> + * but status of init_complete may change while we're
> iterating
> + * the tailq. so, store it here and check once every iteration.
> + */
> + int init_complete;
> +
> + if (new_msg == NULL)
> + new_msg = malloc(sizeof(*new_msg));
> + if (read_msg(&new_msg->msg, &new_msg->sa) == 0) {
Suppose a case that: req and msg received but init not completed, so we enqueue all of them in the tailq; and from now on, no req/rep/msg comes. Then mp thread will hang here for reading new message.
In such a case, we might need the master thread to signal mp thread to wake up when init is completed?
Thanks,
Jianfeng
> + /* we successfully read the message, so enqueue it
> */
> + TAILQ_INSERT_TAIL(&message_queue, new_msg,
> next);
> + new_msg = NULL;
> + } /* reuse new_msg for next message if we couldn't
> read_msg */
> +
> + init_complete = internal_config.init_complete;
> +
> + /* tailq only accessed here, so no locking needed */
> + TAILQ_FOREACH_SAFE(cur_msg, &message_queue, next,
> next_msg) {
> + /* secondary process should not process any
> incoming
> + * requests until its initialization is complete, but
> + * it is allowed to process replies to its own queries.
> + */
> + if (rte_eal_process_type() ==
> RTE_PROC_SECONDARY &&
> + !init_complete &&
> + cur_msg->msg.type != MP_REP)
> + continue;
> +
> + TAILQ_REMOVE(&message_queue, cur_msg, next);
> +
> + process_msg(&cur_msg->msg, &cur_msg->sa);
> +
> + free(cur_msg);
> + }
> }
>
> return NULL;
> --
> 2.7.4
next prev parent reply other threads:[~2018-02-28 1:09 UTC|newest]
Thread overview: 57+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-02-22 18:21 [dpdk-dev] [PATCH 1/3] eal: add internal flag indicating init has completed Anatoly Burakov
2018-02-22 18:21 ` [dpdk-dev] [PATCH 2/3] eal: don't process IPC messages before init finished Anatoly Burakov
2018-02-22 18:21 ` [dpdk-dev] [PATCH 3/3] eal: use locks to determine if secondary process is active Anatoly Burakov
2018-03-02 15:14 ` [dpdk-dev] [PATCH v4 1/5] eal: add internal flag indicating init has completed Anatoly Burakov
2018-03-07 16:56 ` [dpdk-dev] [PATCH v5 0/6] Improvements for DPDK IPC Anatoly Burakov
2018-03-13 17:42 ` [dpdk-dev] [PATCH v6 " Anatoly Burakov
2018-03-21 17:43 ` Thomas Monjalon
2018-03-13 17:42 ` [dpdk-dev] [PATCH v6 1/6] eal: add internal flag indicating init has completed Anatoly Burakov
2018-03-13 20:59 ` Bruce Richardson
2018-03-13 17:42 ` [dpdk-dev] [PATCH v6 2/6] eal: abstract away IPC socket path generation Anatoly Burakov
2018-03-13 17:42 ` [dpdk-dev] [PATCH v6 3/6] eal: don't hardcode socket filter value in IPC Anatoly Burakov
2018-03-13 17:42 ` [dpdk-dev] [PATCH v6 4/6] eal: lock IPC directory on init and send Anatoly Burakov
2018-03-13 17:42 ` [dpdk-dev] [PATCH v6 5/6] eal: simplify IPC sync request timeout code Anatoly Burakov
2018-03-13 17:42 ` [dpdk-dev] [PATCH v6 6/6] eal: ignore messages until init is complete Anatoly Burakov
2018-03-07 16:56 ` [dpdk-dev] [PATCH v5 1/6] eal: add internal flag indicating init has completed Anatoly Burakov
2018-03-07 16:56 ` [dpdk-dev] [PATCH v5 2/6] eal: abstract away IPC socket path generation Anatoly Burakov
2018-03-11 9:02 ` Tan, Jianfeng
2018-03-07 16:56 ` [dpdk-dev] [PATCH v5 3/6] eal: don't hardcode socket filter value in IPC Anatoly Burakov
2018-03-07 16:56 ` [dpdk-dev] [PATCH v5 4/6] eal: lock IPC directory on init and send Anatoly Burakov
2018-03-11 9:14 ` Tan, Jianfeng
2018-03-07 16:56 ` [dpdk-dev] [PATCH v5 5/6] eal: simplify IPC sync request timeout code Anatoly Burakov
2018-03-11 9:25 ` Tan, Jianfeng
2018-03-07 16:56 ` [dpdk-dev] [PATCH v5 6/6] eal: ignore messages until init is complete Anatoly Burakov
2018-03-12 1:42 ` Tan, Jianfeng
2018-03-12 8:58 ` Burakov, Anatoly
2018-03-02 15:14 ` [dpdk-dev] [PATCH v4 2/5] eal: use file to check if secondary process is ready Anatoly Burakov
2018-03-06 11:03 ` Burakov, Anatoly
2018-03-02 15:14 ` [dpdk-dev] [PATCH v4 3/5] eal: prevent secondary process init while sending messages Anatoly Burakov
2018-03-02 15:14 ` [dpdk-dev] [PATCH v4 4/5] eal: don't hardcode socket filter value in IPC Anatoly Burakov
2018-03-02 15:14 ` [dpdk-dev] [PATCH v4 5/5] eal: simplify IPC sync request timeout code Anatoly Burakov
2018-02-22 18:32 ` [dpdk-dev] [PATCH 1/3] eal: add internal flag indicating init has completed Burakov, Anatoly
2018-02-27 13:23 ` [dpdk-dev] [PATCH v2 1/5] " Anatoly Burakov
2018-02-27 14:35 ` [dpdk-dev] [PATCH v3 " Anatoly Burakov
2018-02-28 2:12 ` Tan, Jianfeng
2018-02-28 9:43 ` Burakov, Anatoly
2018-02-27 14:35 ` [dpdk-dev] [PATCH v3 2/5] eal: don't process IPC messages before init finished Anatoly Burakov
2018-02-28 1:09 ` Tan, Jianfeng [this message]
2018-02-28 9:45 ` Burakov, Anatoly
2018-02-28 4:00 ` Wiles, Keith
2018-02-28 9:47 ` Burakov, Anatoly
2018-02-27 14:35 ` [dpdk-dev] [PATCH v3 3/5] eal: use locks to determine if secondary process is active Anatoly Burakov
2018-02-28 1:26 ` Tan, Jianfeng
2018-02-28 10:15 ` Burakov, Anatoly
2018-02-28 4:17 ` Wiles, Keith
2018-02-28 10:17 ` Burakov, Anatoly
2018-02-27 14:35 ` [dpdk-dev] [PATCH v3 4/5] eal: prevent secondary process init while sending messages Anatoly Burakov
2018-02-28 1:58 ` Tan, Jianfeng
2018-02-28 10:19 ` Burakov, Anatoly
2018-02-28 15:49 ` Tan, Jianfeng
2018-02-27 14:35 ` [dpdk-dev] [PATCH v3 5/5] eal: don't hardcode socket filter value in IPC Anatoly Burakov
2018-02-28 1:52 ` Tan, Jianfeng
2018-02-28 10:21 ` Burakov, Anatoly
2018-02-28 15:01 ` Tan, Jianfeng
2018-02-27 13:23 ` [dpdk-dev] [PATCH v2 2/5] eal: don't process IPC messages before init finished Anatoly Burakov
2018-02-27 13:23 ` [dpdk-dev] [PATCH v2 3/5] eal: use locks to determine if secondary process is active Anatoly Burakov
2018-02-27 13:23 ` [dpdk-dev] [PATCH v2 4/5] eal: prevent secondary process init while sending messages Anatoly Burakov
2018-02-27 13:23 ` [dpdk-dev] [PATCH v2 5/5] eal: don't hardcode socket filter value in IPC Anatoly Burakov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ED26CBA2FAD1BF48A8719AEF02201E3651455398@SHSMSX103.ccr.corp.intel.com \
--to=jianfeng.tan@intel.com \
--cc=anatoly.burakov@intel.com \
--cc=dev@dpdk.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).