From: "Zhang, Qi Z" <qi.z.zhang@intel.com>
To: "Burakov, Anatoly" <anatoly.burakov@intel.com>,
"dev@dpdk.org" <dev@dpdk.org>
Cc: "Ananyev, Konstantin" <konstantin.ananyev@intel.com>,
"thomas@monjalon.net" <thomas@monjalon.net>,
"Richardson, Bruce" <bruce.richardson@intel.com>
Subject: Re: [dpdk-dev] [PATCH 0/8] Remove IPC threads
Date: Tue, 26 Jun 2018 07:03:01 +0000 [thread overview]
Message-ID: <039ED4275CED7440929022BC67E706115323E2E4@SHSMSX103.ccr.corp.intel.com> (raw)
In-Reply-To: <cover.1529071026.git.anatoly.burakov@intel.com>
> -----Original Message-----
> From: Zhang, Qi Z
> Sent: Tuesday, June 26, 2018 9:19 AM
> To: 'Anatoly Burakov' <anatoly.burakov@intel.com>; dev@dpdk.org
> Cc: Ananyev, Konstantin <konstantin.ananyev@intel.com>;
> thomas@monjalon.net; Richardson, Bruce <bruce.richardson@intel.com>
> Subject: RE: [dpdk-dev] [PATCH 0/8] Remove IPC threads
>
> Hi Anatoly and Thomas:
>
> Sorry for raise this late, but seems merge mp thread into interrupt thread gives
> problem to enable hotplug on secondary [1].
>
> The issue is, when secondary want to attach a share device, it send request to
> primary Then primary is running in mp callback (mp thread) to attach device, it
> will call rte_malloc which get chance to increase heap that will do sync IPC, You
> know, this is the limitation we can't do sync IPC in mp thread itself. so the
> solution is try to move real work to a separate thread which has no limitation
> to do sync IPC, and interrupt thread is the good candidate, because we just
> need to call rte_eal_set_alarm and we don't need to worry about the
> execution sequence.
>
> But if we merge mp thread into interrupt thread, the solution will not work, we
> may need to create specific temporal thread to handle callback, but this looks
> like some re-build which we already have.
Ok, actually this method looks good to me :), I will send v4 to have this,
But please let me know if you have better idea.
> So I think we need to revisit if we need to merge the thread before we have a
> good solution for such kind of issue.
>
> Thanks
> Qi
>
> [1] https://mails.dpdk.org/archives/dev/2018-June/105018.html
>
>
> > -----Original Message-----
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Anatoly Burakov
> > Sent: Friday, June 15, 2018 10:25 PM
> > To: dev@dpdk.org
> > Cc: Ananyev, Konstantin <konstantin.ananyev@intel.com>;
> > thomas@monjalon.net; Richardson, Bruce <bruce.richardson@intel.com>
> > Subject: [dpdk-dev] [PATCH 0/8] Remove IPC threads
> >
> > As previously discussed [1], IPC threads need to be removed and their
> > workload moved to interrupt thread.
> >
> > FreeBSD did not have an interrupt thread, nor did it support alarm
> > API. This patchset adds support for both on FreeBSD. FreeBSD interrupt
> > thread is based on kevent, FreeBSD's native event multiplexing
> > mechanism similar to Linux's epoll.
> >
> > The patchset makes FreeBSD's interrupts and alarm work just enough to
> > suffice for purposes of IPC, however there are really weird problems
> observed.
> > Specifically, FreeBSD's kevent timers are really slow to trigger for
> > some reason, sleeping on a 10ms timer as much as 200ms before waking
> > up. Interrupt handling on fd's is also a bit flaky.
> >
> > It has also been observed that both problems go away if we do not
> > affinitize master lcore (by commenting relevant code out [2]). It is
> > not known why these problems are observed, nor it is clear what a solution
> might entail.
> >
> > For the purposes of making IPC work and having rudimentary support for
> > alarm and interrupt API's, this patchset works fine. However, because
> > of the above described issues, documentation will not be updated to
> > indicate support for interrupts on FreeBSD at this time.
> >
> > [1] http://dpdk.org/dev/patchwork/patch/36579/
> > [2]
> > http://dpdk.org/browse/dpdk/tree/lib/librte_eal/bsdapp/eal/eal.c#n729
> >
> > Anatoly Burakov (4):
> > ipc: remove IPC thread for async requests
> > eal/bsdapp: add interrupt thread
> > eal/bsdapp: add alarm support
> > ipc: remove main IPC thread
> >
> > Jianfeng Tan (4):
> > eal/linux: use glibc malloc in alarm
> > eal/linux: use glibc malloc in interrupt handling
> > eal: bring forward init of interrupt handling
> > eal: add IPC type for interrupt thread
> >
> > lib/librte_eal/bsdapp/eal/eal.c | 10 +-
> > lib/librte_eal/bsdapp/eal/eal_alarm.c | 299 +++++++++++-
> > lib/librte_eal/bsdapp/eal/eal_alarm_private.h | 19 +
> > lib/librte_eal/bsdapp/eal/eal_interrupts.c | 460 +++++++++++++++++-
> > lib/librte_eal/common/eal_common_proc.c | 243 ++++-----
> > .../common/include/rte_eal_interrupts.h | 1 +
> > lib/librte_eal/linuxapp/eal/eal.c | 10 +-
> > lib/librte_eal/linuxapp/eal/eal_alarm.c | 9 +-
> > lib/librte_eal/linuxapp/eal/eal_interrupts.c | 19 +-
> > test/test/test_interrupts.c | 29 +-
> > 10 files changed, 899 insertions(+), 200 deletions(-) create mode
> > 100644 lib/librte_eal/bsdapp/eal/eal_alarm_private.h
> >
> > --
> > 2.17.1
next prev parent reply other threads:[~2018-06-26 7:03 UTC|newest]
Thread overview: 25+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-06-15 14:25 Anatoly Burakov
2018-06-15 14:25 ` [dpdk-dev] [PATCH 1/8] eal/linux: use glibc malloc in alarm Anatoly Burakov
2018-06-15 14:25 ` [dpdk-dev] [PATCH 2/8] eal/linux: use glibc malloc in interrupt handling Anatoly Burakov
2018-06-15 14:25 ` [dpdk-dev] [PATCH 3/8] ipc: remove IPC thread for async requests Anatoly Burakov
2018-06-15 14:25 ` [dpdk-dev] [PATCH 4/8] eal: bring forward init of interrupt handling Anatoly Burakov
2018-06-15 14:25 ` [dpdk-dev] [PATCH 5/8] eal: add IPC type for interrupt thread Anatoly Burakov
2018-06-15 14:25 ` [dpdk-dev] [PATCH 6/8] eal/bsdapp: add " Anatoly Burakov
2018-06-15 14:25 ` [dpdk-dev] [PATCH 7/8] eal/bsdapp: add alarm support Anatoly Burakov
2018-06-15 14:25 ` [dpdk-dev] [PATCH 8/8] ipc: remove main IPC thread Anatoly Burakov
2018-06-26 1:19 ` [dpdk-dev] [PATCH 0/8] Remove IPC threads Zhang, Qi Z
2018-06-26 7:03 ` Zhang, Qi Z [this message]
2018-06-26 10:53 ` [dpdk-dev] [PATCH v2 0/7] Remove asynchronous IPC thread Anatoly Burakov
2018-07-13 10:44 ` Thomas Monjalon
2018-06-26 10:53 ` [dpdk-dev] [PATCH v2 1/7] eal/linux: use glibc malloc in alarm Anatoly Burakov
2018-06-26 10:53 ` [dpdk-dev] [PATCH v2 2/7] eal/linux: use glibc malloc in interrupt handling Anatoly Burakov
2018-06-26 10:53 ` [dpdk-dev] [PATCH v2 3/7] eal/bsdapp: add interrupt thread Anatoly Burakov
2018-06-26 10:53 ` [dpdk-dev] [PATCH v2 4/7] eal/bsdapp: add alarm support Anatoly Burakov
2018-06-26 10:53 ` [dpdk-dev] [PATCH v2 5/7] eal: bring forward init of interrupt handling Anatoly Burakov
2018-07-12 22:36 ` Thomas Monjalon
2018-07-13 7:41 ` Burakov, Anatoly
2018-07-13 8:09 ` David Marchand
2018-07-13 9:10 ` Tiwei Bie
2018-07-13 11:28 ` David Marchand
2018-06-26 10:53 ` [dpdk-dev] [PATCH v2 6/7] ipc: remove IPC thread for async requests Anatoly Burakov
2018-06-26 10:53 ` [dpdk-dev] [PATCH v2 7/7] doc: document IPC callback limitations Anatoly Burakov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=039ED4275CED7440929022BC67E706115323E2E4@SHSMSX103.ccr.corp.intel.com \
--to=qi.z.zhang@intel.com \
--cc=anatoly.burakov@intel.com \
--cc=bruce.richardson@intel.com \
--cc=dev@dpdk.org \
--cc=konstantin.ananyev@intel.com \
--cc=thomas@monjalon.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).