* graph: dispatch mode with libevent for control plane
@ 2024-12-09 9:27 Robin Jarry
2024-12-16 8:10 ` [EXTERNAL] " Jerin Jacob
0 siblings, 1 reply; 2+ messages in thread
From: Robin Jarry @ 2024-12-09 9:27 UTC (permalink / raw)
To: Nitin Saxena, dev, grout
Cc: Jerin Jacob, David Marchand, Christophe Fontaine, Nitin Saxena
Hi folks,
I had a look at the graph dispatch mode and wondered if it could be
adapted to be used with event loops for "slow path" processing.
The use case I have in mind is having lcores dedicated for control plane
operations which are not latency sensitive. It would be nice to allow
data plane workers to send packets to these lcores while keeping the
packets in the graph.
The problem I have with this solution for now, is that all CPUs
processing the graph with rte_graph_walk_mcore_dispatch() are supposed
to do busy polling. This is not compatible with select/epoll based event
loops.
Would it be possible to have a way for busy-polling threads to signal
non-busy-polling threads that there are packets waiting for them in the
graph ring? In grout, we implemented something outside of the graph
using pthread_cond_signal().
https://github.com/DPDK/grout/blob/main/modules/infra/datapath/control_output.c
https://github.com/DPDK/grout/blob/main/modules/infra/control/control_output.c
Unless I am mistaken, this does not perform any syscall and does not
require acquiring any lock. Only the "slow" thread needs to acquire
locks and wait for the data path threads to call pthread_cond_signal().
This solution can only be used for busy-polling -> epoll-based
communication, for the other direction epoll-based -> busy-polling there
would not need any specific signaling required. Only posting the packets
to the graph ring would be enough.
What do you think? How would you integrate this into rte_graph? What
would the API look like in your opinion?
Thanks!
--
Robin
^ permalink raw reply [flat|nested] 2+ messages in thread
* RE: [EXTERNAL] graph: dispatch mode with libevent for control plane
2024-12-09 9:27 graph: dispatch mode with libevent for control plane Robin Jarry
@ 2024-12-16 8:10 ` Jerin Jacob
0 siblings, 0 replies; 2+ messages in thread
From: Jerin Jacob @ 2024-12-16 8:10 UTC (permalink / raw)
To: Robin Jarry, Nitin Saxena, dev, grout
Cc: David Marchand, Christophe Fontaine, Nitin Saxena, Zhirun Yan,
Pavan Nikhilesh Bhagavatula
> -----Original Message-----
> From: Robin Jarry <rjarry@redhat.com>
> Sent: Monday, December 9, 2024 2:58 PM
> To: Nitin Saxena <nsaxena@marvell.com>; dev@dpdk.org; grout@dpdk.org
> Cc: Jerin Jacob <jerinj@marvell.com>; David Marchand
> <david.marchand@redhat.com>; Christophe Fontaine <cfontain@redhat.com>;
> Nitin Saxena <nsaxena16@gmail.com>
> Subject: [EXTERNAL] graph: dispatch mode with libevent for control plane
>
> Hi folks, I had a look at the graph dispatch mode and wondered if it could be
> adapted to be used with event loops for "slow path" processing. The use case I
> have in mind is having lcores dedicated for control plane operations which are
> not latency
> Hi folks,
>
> I had a look at the graph dispatch mode and wondered if it could be adapted to
> be used with event loops for "slow path" processing.
>
> The use case I have in mind is having lcores dedicated for control plane
> operations which are not latency sensitive. It would be nice to allow data plane
> workers to send packets to these lcores while keeping the packets in the graph.
>
> The problem I have with this solution for now, is that all CPUs processing the
> graph with rte_graph_walk_mcore_dispatch() are supposed to do busy polling.
> This is not compatible with select/epoll based event loops.
>
> Would it be possible to have a way for busy-polling threads to signal non-busy-
> polling threads that there are packets waiting for them in the graph ring? In
> grout, we implemented something outside of the graph using
> pthread_cond_signal().
Sorry for delay, I was on business travel.
Adding yanzhirun_163@163.com(mcore model maintainer).
If I understand it correctly, we need three operations.
a)wait object creation.
b)wait object waiting instead of polling.
c)signal wait object from fastpath to wake up (b)
I see two options.
Option A:
In application (without graph change)
1)call (a)
2)
while(1)
{
Call (b)
rte_graph_walk_mcore_dispatch()
}
3)Call (c) from fp nodes(Assuming it is out of tree nodes)
Option B
1)Call (a) in model create or so
2)Add an option in rte_graph_walk_mcore_dispatch() to call (b)
3)Update rte_.._enqueue() to call (c) after ring enqueue
Option B is need if we need inbuilt fp nodes.
>
> https://urldefense.proofpoint.com/v2/url?u=https-
> 3A__github.com_DPDK_grout_blob_main_modules_infra_datapath_control-
> 5Foutput.c&d=DwIFaQ&c=nKjWec2b6R0mOyPaz7xtfQ&r=1DGob4H4rxz6H8uIT
> ozGOCa0s5f4wCNtTa4UUKvcsvI&m=sctQnRalWyQEGrI4o2UpcCXD8n8qveibIe4C
> clcMOUmS7Oi0S7MoVryikpo_ZDh1&s=Zm-
> i7a_D6MFxZO362gYHHe9PtKO_Kr6JeFNlmEHpMCg&e=
> https://urldefense.proofpoint.com/v2/url?u=https-
> 3A__github.com_DPDK_grout_blob_main_modules_infra_control_control-
> 5Foutput.c&d=DwIFaQ&c=nKjWec2b6R0mOyPaz7xtfQ&r=1DGob4H4rxz6H8uIT
> ozGOCa0s5f4wCNtTa4UUKvcsvI&m=sctQnRalWyQEGrI4o2UpcCXD8n8qveibIe4C
> clcMOUmS7Oi0S7MoVryikpo_ZDh1&s=09ea7Sa47B5SPe7SjP5zBNMzqem0XzI3
> HnlU-k73N2w&e=
>
> Unless I am mistaken, this does not perform any syscall and does not require
> acquiring any lock. Only the "slow" thread needs to acquire locks and wait for
> the data path threads to call pthread_cond_signal().
>
> This solution can only be used for busy-polling -> epoll-based communication,
> for the other direction epoll-based -> busy-polling there would not need any
> specific signaling required. Only posting the packets to the graph ring would be
> enough.
>
> What do you think? How would you integrate this into rte_graph? What would
> the API look like in your opinion?
>
> Thanks!
>
> --
> Robin
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2024-12-16 8:11 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-12-09 9:27 graph: dispatch mode with libevent for control plane Robin Jarry
2024-12-16 8:10 ` [EXTERNAL] " Jerin Jacob
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).