Re: [dpdk-dev] [PATCH 0/8] Remove IPC threads

["Zhang, Qi Z" <qi.z.zhang@intel.com>]

Hi Anatoly and Thomas:

Sorry for raise this late, but seems merge mp thread into interrupt thread gives problem to enable hotplug on secondary [1].

The issue is, when secondary want to attach a share device, it send request to primary
Then primary is running in mp callback (mp thread) to attach device, it will call rte_malloc which get chance to increase heap that will do sync IPC,
You know, this is the limitation we can't do sync IPC in mp thread itself. so the solution is try to move real work to a separate thread which has no limitation to do sync IPC, 
and interrupt thread is the good candidate, because we just need to call rte_eal_set_alarm and we don't need to worry about the execution sequence.

But if we merge mp thread into interrupt thread, the solution will not work, we may need to create specific temporal thread to handle callback, but this looks like some re-build which we already have.
So I think we need to revisit if we need to merge the thread before we have a good solution for such kind of issue.

Thanks
Qi

[1] https://mails.dpdk.org/archives/dev/2018-June/105018.html


> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Anatoly Burakov
> Sent: Friday, June 15, 2018 10:25 PM
> To: dev@dpdk.org
> Cc: Ananyev, Konstantin <konstantin.ananyev@intel.com>;
> thomas@monjalon.net; Richardson, Bruce <bruce.richardson@intel.com>
> Subject: [dpdk-dev] [PATCH 0/8] Remove IPC threads
> 
> As previously discussed [1], IPC threads need to be removed and their
> workload moved to interrupt thread.
> 
> FreeBSD did not have an interrupt thread, nor did it support alarm API. This
> patchset adds support for both on FreeBSD. FreeBSD interrupt thread is based
> on kevent, FreeBSD's native event multiplexing mechanism similar to Linux's
> epoll.
> 
> The patchset makes FreeBSD's interrupts and alarm work just enough to
> suffice for purposes of IPC, however there are really weird problems observed.
> Specifically, FreeBSD's kevent timers are really slow to trigger for some reason,
> sleeping on a 10ms timer as much as 200ms before waking up. Interrupt
> handling on fd's is also a bit flaky.
> 
> It has also been observed that both problems go away if we do not affinitize
> master lcore (by commenting relevant code out [2]). It is not known why these
> problems are observed, nor it is clear what a solution might entail.
> 
> For the purposes of making IPC work and having rudimentary support for
> alarm and interrupt API's, this patchset works fine. However, because of the
> above described issues, documentation will not be updated to indicate
> support for interrupts on FreeBSD at this time.
> 
> [1] http://dpdk.org/dev/patchwork/patch/36579/
> [2] http://dpdk.org/browse/dpdk/tree/lib/librte_eal/bsdapp/eal/eal.c#n729
> 
> Anatoly Burakov (4):
>   ipc: remove IPC thread for async requests
>   eal/bsdapp: add interrupt thread
>   eal/bsdapp: add alarm support
>   ipc: remove main IPC thread
> 
> Jianfeng Tan (4):
>   eal/linux: use glibc malloc in alarm
>   eal/linux: use glibc malloc in interrupt handling
>   eal: bring forward init of interrupt handling
>   eal: add IPC type for interrupt thread
> 
>  lib/librte_eal/bsdapp/eal/eal.c               |  10 +-
>  lib/librte_eal/bsdapp/eal/eal_alarm.c         | 299 +++++++++++-
>  lib/librte_eal/bsdapp/eal/eal_alarm_private.h |  19 +
>  lib/librte_eal/bsdapp/eal/eal_interrupts.c    | 460 +++++++++++++++++-
>  lib/librte_eal/common/eal_common_proc.c       | 243 ++++-----
>  .../common/include/rte_eal_interrupts.h       |   1 +
>  lib/librte_eal/linuxapp/eal/eal.c             |  10 +-
>  lib/librte_eal/linuxapp/eal/eal_alarm.c       |   9 +-
>  lib/librte_eal/linuxapp/eal/eal_interrupts.c  |  19 +-
>  test/test/test_interrupts.c                   |  29 +-
>  10 files changed, 899 insertions(+), 200 deletions(-)  create mode 100644
> lib/librte_eal/bsdapp/eal/eal_alarm_private.h
> 
> --
> 2.17.1