From: "Van Haaren, Harry" <harry.van.haaren@intel.com>
To: "Tan, Jianfeng" <jianfeng.tan@intel.com>,
Thomas Monjalon <thomas@monjalon.net>
Cc: "Burakov, Anatoly" <anatoly.burakov@intel.com>,
"dev@dpdk.org" <dev@dpdk.org>,
"Ananyev, Konstantin" <konstantin.ananyev@intel.com>
Subject: Re: [dpdk-dev] [PATCH v6 2/2] eal: add asynchronous request API to DPDK IPC
Date: Wed, 28 Mar 2018 09:10:12 +0000 [thread overview]
Message-ID: <E923DB57A917B54B9182A2E928D00FA65E018361@IRSMSX101.ger.corp.intel.com> (raw)
In-Reply-To: <944ebaf4-6dc3-5069-1d7e-2ee7bbcd8adc@intel.com>
> -----Original Message-----
> From: Tan, Jianfeng
> Sent: Wednesday, March 28, 2018 9:55 AM
> To: Van Haaren, Harry <harry.van.haaren@intel.com>; Thomas Monjalon
> <thomas@monjalon.net>
> Cc: Burakov, Anatoly <anatoly.burakov@intel.com>; dev@dpdk.org; Ananyev,
> Konstantin <konstantin.ananyev@intel.com>
> Subject: Re: [dpdk-dev] [PATCH v6 2/2] eal: add asynchronous request API to
> DPDK IPC
>
> Hi Thomas and Harry,
Hey,
> On 3/28/2018 4:22 PM, Van Haaren, Harry wrote:
> >> From: Thomas Monjalon [mailto:thomas@monjalon.net]
> >> Sent: Wednesday, March 28, 2018 8:30 AM
> >> To: Tan, Jianfeng <jianfeng.tan@intel.com>
> >> Cc: Burakov, Anatoly <anatoly.burakov@intel.com>; dev@dpdk.org; Ananyev,
> >> Konstantin <konstantin.ananyev@intel.com>; Van Haaren, Harry
> >> <harry.van.haaren@intel.com>
> >> Subject: Re: [dpdk-dev] [PATCH v6 2/2] eal: add asynchronous request API
> to
> >> DPDK IPC
> >>
> >> 28/03/2018 04:08, Tan, Jianfeng:
> >>> Hi Thomas ,
> >>>
> >>> From: Thomas Monjalon [mailto:thomas@monjalon.net]
> >>>> 27/03/2018 15:59, Anatoly Burakov:
> >>>>> Under the hood, we create a separate thread to deal with replies to
> >>>>> asynchronous requests, that will just wait to be notified by the
> >>>>> main thread, or woken up on a timer.
> >>>> I really don't like that a library is creating a thread.
> >>>> We don't even know where the thread is created (which core).
> >>>> Can it be a rte_service? or in the interrupt thread?
> >>> Agree that we'd better not adding so many threads in a library.
> >>>
> >>> I was considering to merge all the threads into the interrupt thread,
> >> however, we don't have an interrupt thread in freebsd. Further, we don't
> >> implement alarm API in freebsd. That's why I tend to current
> implementation,
> >> and optimize it later.
> >>
> >> I would prefer we improve the current code now instead of polluting more
> >> with more uncontrolled threads.
> >>
> >>> For rte_service, it may be not a good idea to reply on it as it needs
> >> explicit API calls to setup.
> >>
> >> I don't see the issue of the explicit API.
> >> The IPC is a new service.
>
> My concern is that not every DPDK application sets up rte_service, but
> IPC will be used for very fundamental functions, like memory allocation.
> We could not possibly ask all DPDK applications to add rte_service now.
>
> And also take Harry's comments below into consideration, most likely, we
> will move these threads into interrupt thread now by adding
I don't suggest moving everything into interrupt thread.
We need to ensure the interrupt thread (aka, link status interrupt thread) is not
busy for too long. Certain work may cause un-acceptable delay in the interrupt
thread, hence perhaps we should have 2 core DPDK threads:
1) EAL interrupt thread: handles only interrupts, so is ~always blocked on some FDs
2) EAL "core" thread: handles the IPC work here, and any other irregular work that
can take some time to complete. Adding all core DPDK work to this thread enables
us to refactor and create a scalable / coherent design.
> > Although I do like to see new services, if we want to enable "core" dpdk
> functionality with Services, we need a proper designed solution for that.
> Service cores is not intended for "occasional" work - there is no method to
> block and sleep on a specific service until work becomes available, so this
> would imply a busy-polling. Using a service (hence busy polling) for
> rte_malloc()-based memory mapping requests is inefficient, and total
> overkill :)
> >
> > For this patch I suggest to use some blocking-read capable mechanism.
>
> The problem here is that we add too many threads; blocking-read does not
> decrease # of threads.
To correct my statement just above - some method capable of blocking reads (as opposed to
busy polling). In my mind, this method *must* allow waiting on multiple FDs,
as to *require* only 1 thread. We could use more if it makes sense to do so.
> > The above said, in the longer term it would be good to have a design that
> allows new file-descriptors to be added to a "dpdk core" thread, which
> performs occasional lengthy work if the FD has data available.
>
> Interrupt thread vs rte_service, which is the direction to go? We
> actually have some others threads, in vhost and even virtio-user; we can
> also avoid those threads if we have a clear direction.
I don't think that service is the correct hammer for this problem.
As to interrupt thread or creating a new thread, what makes more sense given
the current codebase? Is the current implementation an acceptable short-term
solution, that gets reworked to be more generic in future?
next prev parent reply other threads:[~2018-03-28 9:10 UTC|newest]
Thread overview: 40+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-02-27 14:59 [dpdk-dev] [PATCH] " Anatoly Burakov
2018-02-28 10:22 ` Burakov, Anatoly
2018-03-02 18:06 ` [dpdk-dev] [PATCH v2] " Anatoly Burakov
2018-03-07 16:57 ` [dpdk-dev] [PATCH v3] " Anatoly Burakov
2018-03-13 17:42 ` [dpdk-dev] [PATCH v4] " Anatoly Burakov
2018-03-23 15:38 ` Tan, Jianfeng
2018-03-23 18:21 ` Burakov, Anatoly
2018-03-24 13:22 ` Burakov, Anatoly
2018-03-24 12:46 ` [dpdk-dev] [PATCH v5 1/2] eal: rename IPC sync request to pending request Anatoly Burakov
2018-03-26 7:31 ` Tan, Jianfeng
2018-03-27 13:59 ` [dpdk-dev] [PATCH v6 " Anatoly Burakov
2018-03-27 16:27 ` Thomas Monjalon
2018-03-28 9:15 ` Burakov, Anatoly
2018-03-28 10:08 ` Thomas Monjalon
2018-03-28 10:57 ` Burakov, Anatoly
2018-03-31 17:06 ` [dpdk-dev] [PATCH v7 1/3] " Anatoly Burakov
2018-03-31 17:06 ` [dpdk-dev] [PATCH v7 2/3] eal: rename mp_request to mp_request_sync Anatoly Burakov
2018-04-02 5:09 ` Tan, Jianfeng
2018-03-31 17:06 ` [dpdk-dev] [PATCH v7 3/3] eal: add asynchronous request API to DPDK IPC Anatoly Burakov
2018-04-04 22:15 ` Thomas Monjalon
2018-03-27 13:59 ` [dpdk-dev] [PATCH v6 2/2] " Anatoly Burakov
2018-03-27 16:33 ` Thomas Monjalon
2018-03-28 2:08 ` Tan, Jianfeng
2018-03-28 7:29 ` Thomas Monjalon
2018-03-28 8:22 ` Van Haaren, Harry
2018-03-28 8:55 ` Tan, Jianfeng
2018-03-28 9:10 ` Van Haaren, Harry [this message]
2018-03-28 9:21 ` Burakov, Anatoly
2018-03-28 9:53 ` Thomas Monjalon
2018-03-28 10:42 ` Burakov, Anatoly
2018-03-28 11:26 ` Thomas Monjalon
2018-03-28 12:21 ` Burakov, Anatoly
2018-03-28 9:11 ` Bruce Richardson
2018-03-24 12:46 ` [dpdk-dev] [PATCH v5 " Anatoly Burakov
2018-03-26 14:15 ` Tan, Jianfeng
2018-03-26 14:28 ` Burakov, Anatoly
2018-03-02 18:48 ` [dpdk-dev] [PATCH] " Stephen Hemminger
2018-03-03 12:29 ` Burakov, Anatoly
2018-03-02 18:51 ` Stephen Hemminger
2018-03-03 13:44 ` Burakov, Anatoly
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=E923DB57A917B54B9182A2E928D00FA65E018361@IRSMSX101.ger.corp.intel.com \
--to=harry.van.haaren@intel.com \
--cc=anatoly.burakov@intel.com \
--cc=dev@dpdk.org \
--cc=jianfeng.tan@intel.com \
--cc=konstantin.ananyev@intel.com \
--cc=thomas@monjalon.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).