RE: DPDK community: RTE_FLOW support for P4-programmable devices

["Dumitrescu, Cristian" <cristian.dumitrescu@intel.com>]

> -----Original Message-----
> From: Ori Kam <orika@nvidia.com>
> Sent: Friday, September 1, 2023 7:58 AM
> To: Jerin Jacob <jerinjacobk@gmail.com>
> Cc: Ferruh Yigit <ferruh.yigit@amd.com>; Dumitrescu, Cristian
> <cristian.dumitrescu@intel.com>; Morten Brørup
> <mb@smartsharesystems.com>; Zhang, Qi Z <qi.z.zhang@intel.com>; NBU-
> Contact-Thomas Monjalon (EXTERNAL) <thomas@monjalon.net>;
> david.marchand@redhat.com; Richardson, Bruce <bruce.richardson@intel.com>;
> jerinj@marvell.com; techboard@dpdk.org; Mcnamara, John
> <john.mcnamara@intel.com>; Zhang, Helin <helin.zhang@intel.com>;
> dev@dpdk.org
> Subject: RE: DPDK community: RTE_FLOW support for P4-programmable devices
> 
> 
> 
> > -----Original Message-----
> > From: Jerin Jacob <jerinjacobk@gmail.com>
> > Sent: Friday, September 1, 2023 5:07 AM
> >
> > On Thu, Aug 31, 2023 at 4:02 PM Ori Kam <orika@nvidia.com> wrote:
> > >
> > > Hi
> >
> > > > >>
> > > > >> 3. Everybody on the call agreed that the P4-programmable devices from
> > > > Intel,
> > > > >> AMD and others need to be fully supported by DPDK and that there are
> > > > some
> > > > >> gaps in RTE_FLOW to be fixed for supporting these devices.
> > > > >
> > > > > Personally, It makes sense to me to have normative DPDK API to send p4
> > > > > runtime message to the
> > > > > ethdev so that we have "vendor neutral + DPDK based" p4 runtime
> > backend.
> > > > >
> > > > > I prefer to have specialized ethdev ops for this due to the following
> > reasons.
> > > > >
> > > > > # If the ethdev has both real TCAM based HW(for existing rte_flow
> > > > > patterns and actions) and SW agent to receive P4 runtime message etc.
> > > > > Typically, it needs to take a different path in driver to talk. Assume, if you
> > > > > have cascaded patterns/actions, One is targeted for TCAM and other for
> > > > > SW agent for p4, one
> > > > > need to have serious amount checking for dispatching.It complicates
> > > > > the driver and forbid to have
> > > > > driver optimization especially cases for templates etc. if user making
> > > > > rules for both category of HW.
> > > > >
> > > >
> > > > Indeed I am not against dedicated APIs for P4 runtime backend.
> > > >
> > > > But assuming there is a dedicated rte_flow item for P4, how it is
> > > > different than dedicated API in above scenario?
> > > > If driver detects P4 runtime specific rule, it can bail it out to SW agent.
> > > > Can you please elaborate the complexity it introduces?
> >
> > Assume, normal existing rte-flow programming include a bunch of
> > register writes and
> > p4 runtime backend is more of SW ring. If a template has both patterns
> > and actions
> > as cascaded, it will be difficult for driver to optimize the template.
> >
> >
> > > >
> > > > > # All we need "char buffer//string" based communication ethdev <-> app.
> > > > >
> > > >
> > > > Yes, and both a dedicated API or dedicated rte_flow item can provide
> > > > medium for this communication.
> > > >
> > > > rte_flow one has flexibility & extensibility advantages, but maybe not
> > > > as straightforward as an API.
> > >
> > > I think not using the rte_flow will also require duplication of all the rule
> > handling functions and table creations, for example aync rule create/destroy
> > query ......
> >
> > Yes. That is a fair point. I am OK with exposing as rte_flow.
> > As a driver implementation note, to get rid of the above problem,
> > driver can choose to have pseudo ethdev devices for p4 if needed(if
> > driver finds difficult to combine TCAM based on HW rules and p4
> > runtime rule).
> >
> 
> What about the following concept:
> The p4 compiler can generate the translation to known PMD objects in rte_flow,
> then when a command is sent from the p4 agent to the offload using GRPC or any
> other way, the DPDK will convert from
> p4 protocol to rte_flow commands (this should be very fast since the commands
> are known and the mapping is already
> defined).
> 
> To support the above idea we need to add two new functions to rte_flow (each
> function will be implemented in PMD level)
> Rte_flow_register_p4(void *p4_info, void *p4_blob)
> {
> 	Creates the static structures/objects
> 	Internal register the p4 commands to PMD translation table.
> }
> 
> Rte_flow_p4_runtime(p4 command based on the p4 spec)
> {
> 	Based on the registered mapping, translate the command to rte_flow
> commands.
> 	Rte_flow_xxx() calls
> }
> 
> As far as I see, the above code will also allow PMD to implement internal logic if
> needed, while from DPDK API,
> we will only add two new functions.

Hi Ori,

It would be great if we (all the interested folks) could meet at the DPDK summit
in the next few days and brainstorm on this topic!

I don't think the compiler idea fixes the problem at all, unfortunately:

1. The P4 compiler is for the data path, not for the control path. You seem to 
suggest a second compiler focused on the control path, which would be a
completely new tool of high complexity or even tasked to do the impossible
(explained below), which is a path to failure. Therefore,  it is definitely not OK
to make his translator/compiler a mandatory requirement for RTE_FLOW
support of existing HW!

2. This compiler is expected to magically translate an infinite list of possible
user-defined action to one of the existing finite list of RTE_FLOW actions,
and this is obviously not possible. See the example P4 program from the
slides I presented, the user defined P4 actions are a sort of user-defined
subroutines (mixing assignments, arithmetic operations and branches on
metadata and header fields, target dependent intrinsics, etc) as opposed
to simple RTE_FLOW primitives.

3. The translation of user-defined actions to one of the existing RTE_FLOW
actions is sometimes possible, but this takes place only for the most trivial
cases. And even in such cases it typically needs a hint from the user to testify
the mapping as opposed to the mapping being unequivocally identified by a
compiler. Even the simplest action of packet drop can be done in so many
different ways:
	a) straight call to a target dependent drop intrinsic (unknown to the
	   "compiler" unless user provides a hint);
	b) set a meta-data flag that is tested at a later stage for packet drop;
	c) send the packet to a predefined output port that is either dropping
	   or logging/counting the packet;
	d) etc.

4. The P4 actions are simply user-defined actions that are built into the HW
through the program, so the control path to configure them it needs to know
just their ID and their arguments that have to be provided. These actions are
not portable between programs, as their life span is the time the P4 program
is loaded on the target; once a new program is loaded, the old actions cease
to exist, therefore it makes less sense to standardize some "ephemeral"
actions, and it makes more sense to provide a registration mechanism in
RTE_FLOW for these user-defined actions to be registered when the device
is initialized.

What do people think?

Regards,
Cristian