RE: [RFC] lib/ethdev: introduce table driven APIs

["Zhang, Qi Z" <qi.z.zhang@intel.com>]

> -----Original Message-----
> From: Jerin Jacob <jerinjacobk@gmail.com>
> Sent: Thursday, June 15, 2023 4:38 PM
> To: Zhang, Qi Z <qi.z.zhang@intel.com>
> Cc: Ori Kam <orika@nvidia.com>; NBU-Contact-Thomas Monjalon (EXTERNAL)
> <thomas@monjalon.net>; david.marchand@redhat.com; Richardson, Bruce
> <bruce.richardson@intel.com>; jerinj@marvell.com; ferruh.yigit@amd.com;
> Mcnamara, John <john.mcnamara@intel.com>; Zhang, Helin
> <helin.zhang@intel.com>; techboard@dpdk.org; dev@dpdk.org; Ivan Malov
> <ivan.malov@arknetworks.am>
> Subject: Re: [RFC] lib/ethdev: introduce table driven APIs
> 
> On Thu, Jun 15, 2023 at 1:12 PM Zhang, Qi Z <qi.z.zhang@intel.com> wrote:
> >
> >
> >
> > > -----Original Message-----
> > > From: Jerin Jacob <jerinjacobk@gmail.com>
> > > Sent: Thursday, June 15, 2023 2:21 PM
> > > To: Zhang, Qi Z <qi.z.zhang@intel.com>
> > > Cc: Ori Kam <orika@nvidia.com>; NBU-Contact-Thomas Monjalon
> > > (EXTERNAL) <thomas@monjalon.net>; david.marchand@redhat.com;
> > > Richardson, Bruce <bruce.richardson@intel.com>; jerinj@marvell.com;
> > > ferruh.yigit@amd.com; Mcnamara, John <john.mcnamara@intel.com>;
> > > Zhang, Helin <helin.zhang@intel.com>; techboard@dpdk.org;
> > > dev@dpdk.org; Ivan Malov <ivan.malov@arknetworks.am>
> > > Subject: Re: [RFC] lib/ethdev: introduce table driven APIs
> > >
> > > On Thu, Jun 15, 2023 at 11:33 AM Zhang, Qi Z <qi.z.zhang@intel.com>
> wrote:
> > > >
> > > > Hi Jerin:
> > >
> > > Hi Qi
> > >
> > > >
> > > > > -----Original Message-----
> > > > > From: Jerin Jacob <jerinjacobk@gmail.com>
> > > > > Sent: Thursday, June 15, 2023 12:58 PM
> > > > > To: Zhang, Qi Z <qi.z.zhang@intel.com>
> > > > > Cc: Ori Kam <orika@nvidia.com>; NBU-Contact-Thomas Monjalon
> > > > > (EXTERNAL) <thomas@monjalon.net>; david.marchand@redhat.com;
> > > > > Richardson, Bruce <bruce.richardson@intel.com>;
> > > > > jerinj@marvell.com; ferruh.yigit@amd.com; Mcnamara, John
> > > > > <john.mcnamara@intel.com>; Zhang, Helin <helin.zhang@intel.com>;
> > > > > techboard@dpdk.org; dev@dpdk.org; Ivan Malov
> > > > > <ivan.malov@arknetworks.am>
> > > > > Subject: Re: [RFC] lib/ethdev: introduce table driven APIs
> > > > >
> > > > > On Thu, Jun 15, 2023 at 7:55 AM Zhang, Qi Z
> > > > > <qi.z.zhang@intel.com>
> > > wrote:
> > > > > >
> > > > > > Hi Ori:
> > > > > >
> > > > > >         Thank you for your review!
> > > > > >         Comment inline.
> > > > > >         Please let me know if anything I missed.
> > > > > >
> > > > > > Thanks
> > > > > > Qi
> > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: Ori Kam <orika@nvidia.com>
> > > > > > > Sent: Thursday, June 15, 2023 2:31 AM
> > > > > > > To: Zhang, Qi Z <qi.z.zhang@intel.com>; NBU-Contact-Thomas
> > > > > > > Monjalon
> > > > > > > (EXTERNAL) <thomas@monjalon.net>;
> david.marchand@redhat.com;
> > > > > > > Richardson, Bruce <bruce.richardson@intel.com>;
> > > > > > > jerinj@marvell.com; ferruh.yigit@amd.com
> > > > > > > Cc: Mcnamara, John <john.mcnamara@intel.com>; Zhang, Helin
> > > > > > > <helin.zhang@intel.com>; techboard@dpdk.org; dev@dpdk.org
> > > > > > > Subject: RE: [RFC] lib/ethdev:
> > > > > > >
> > > > > > > Hi Qi,
> > > > > > >
> > > > > > >
> > > > > > > 1. it may be useful to get some general calling flow what
> > > > > > > comes from the application, what comes from the compiler.
> > > > > > > Simple example will be good.
> > > > > >
> > > > > > An example of decap VXLAN TCP flow is explained in problem
> > > > > > statement
> > > > > > (http://mails.dpdk.org/archives/dev/2023-May/267719.html)
> > > > > > covering the following information.
> > > > > >
> > > > > > 1. the p4 source code, the definition of the table and actions 2.
> > > > > > the table / action hints generated by the compiler, details to
> > > > > > each
> > > fields.
> > > > > > 3. How the Control Plane Application utilizes the P4 Runtime
> > > > > > API to program the rule with the respective table and action
> > > > > > IDs
> > > > > >
> > > > > > The DPDK PMD is responsible for loading the hints generated by
> > > > > > the
> > > > > compiler.
> > > > > > This enables the PMD to accept requests from the P4 Runtime
> > > > > > and reject
> > > > > any incompatible request.
> > > > >
> > > > > I see two different types of device/system category
> > > > >
> > > > > 1) HW + SW/FW combination that really understands p4 structures
> > > > > and job of the driver to is to give work to HW/SW as p4
> > > > > structure generated from vendor specific compiler and runtime
> > > > > gRPC message
> > > > > 2) Existing HW and SW drivers implements rte-flow driver.
> > > > >
> > > > > For item (1), if end user application is using P4 program and P4
> > > > > runtime and this is _API contract_ to application, Not sure why
> > > > > end user care it is DPDK PMD or not?
> > > >
> > > > That's true. DPDK as a platform that manage the hardware, it is
> > > > required to
> > > provide a channel that connects applications with the hardware
> > > responsible for implementing the contract.
> > > > In this context, the PMD (ethdev) serves as the conduit that can
> > > > fulfill this
> > > requirement.
> > >
> > > I meant vdev + rawdev combo can be used to talk to FW.
> >
> > OK, I will comment this together when vdev + rawdev be mentioned again
> at below.
> >
> > >
> > > > > If driver writer care about using DPDK for driver framework for
> > > > > EAL services, simply using vdev etc would be enough. Right?
> > > >
> > > > I may not fully understand this, a vdev should have a device type,
> > > > I didn't
> > > see any issue for a ethdev vdev to implement the table-driven APIs.
> > >
> > > See above.
> > >
> > > There is a lot of overlap between rte_flow and table driven API is the
> issue.
> > > To make things worst, there is also lib/table/ API.
> >
> > I assume this is just the concern about naming? At least, they are target to
> different usage.
> 
> Not the naming of library. Duplicate functional APIs to express a specific use
> case for HW.
> 
> >
> > >
> > > >
> > > > >
> > > > > For item (2), I think, interest is how to offload p4 workload to
> > > > > rte_flow. So that _existing_ drivers implements rte_flow can
> > > > > support
> > > > > p4 naturally in addition to existing rte_flow API. If that is
> > > > > direction, then we need to the following.
> > > >
> > > > While the idea of offloading P4 to rte_flow is certainly
> > > > interesting, it
> > > doesn't seem to directly address our initial problem statement.
> > > > The primary objective is to find a solution for offloading
> > > > rte_flow into a P4-
> > > based pipeline.
> > >
> > > Isn't same? If not, Please elaborate on "P4 to rte_flow mapping" vs
> > > "offloading rte_flow into a P4-based pipeline"
> >
> > OK I guess the gap here is I may not fully understand is how we
> > defined the case of item(2) Existing HW and SW drivers implements rte-
> flow driver.
> >
> > If we assume that the application is not P4-aware, it will consume existing
> rte_flow API for flow offloading. In this case, all we need to do is implement
> it in the PMD, which will be a highly hardware-specific task. Do you propose
> generalizing this common part?
> >
> > On the other hand, if the application is P4-aware, we can assume that
> there won't be a need for translation between P4 tokens and rte_flow
> protocols in the PMD.
> 
> I agree, Translation is BAD. There are two elements to that.
> 1)if it is p4 aware application, why bother with DPDK abstraction?
> 2)Can we use compiler techniques to avoid the cost of translation if P4-
> aware path is needed in DPDK. Rather than creating yet another library. In
> this context, that would translate to some of your compiler and FW work
> making as generic so that _any_ other rte_flow based driver can use and
> improve it.


Ok, I would like to gain a better understanding. Below is my current understanding:

There are no plans to introduce any new API from DPDK. However, your proposal suggests the creation of a tool, such as a compiler, which would assist in generating a translation layer from P4 table/actions to rte_flow for user application like p4 runtime backend that based on DPDK.

Could you provide more details about the design? Specifically, I would like to know what the input for the compiler is and who is responsible for generating that input, as well as the process involved.

I apologize if I have not grasped the complete picture, but I would appreciate your patience.

> 
> >
> > >
> > >
> > > >
> > > > We have identified two distinct use cases:
> > > >
> > > > P4-Aware Applications:
> > > >
> > > > For applications that are already P4 aware, the proposal suggests
> > > > the
> > > introduction of a new set of APIs to rte_flow.
> > > > These APIs aim to facilitate seamless integration between DPDK and
> > > > P4
> > > aware applications.
> > >
> > > Counter argument for that is, If the P4 is API contract then why
> > > bother with DPDK abstraction use vdev +  rawdev talk to FW as PMD is
> > > just passing message to FW. FW is doing the heavy lifting anyway.
> >
> > We are attempting to generalize the common aspects, considering that P4
> Runtime is a standard API. It appears worthwhile to expose certain APIs that
> can assist its backend implementation.
> 
> I agree. If backend is built on top rte_flow and some compiler bits. I think,
> multiple consumer can use it.
> Otherwise, we are making the API for a p4 FW backend is needed.
> 
> 
> > I may need some time to understand the concept of vdev +rawdev combo
> solution, currently one question in my mind is: in this solution, is above
> consideration covered?
> >
> > Thanks
> > Qi
> >
> > >
> > >
> > > >
> > > > Non-P4 Aware Applications:
> > > >
> > > > In the case, our focus is on bridging the existing rte_flow API to
> > > > the
> > > underlying P4 pipeline.
> > > > Currently, we haven't identified any significant gaps in the DPDK APIs.
> > > > The key challenge lies in handling the translation process within
> > > > the PMD
> > > >
> > > > Thanks
> > > > Qi
> > > >
> > > > >
> > > > > a)Improve p4-dpdk compiler backend or add new compiler DPDK
> > > backend
> > > > > to understand the rte_flow and have helper library in DPDK to
> > > > > understand the compiler spec file to translate to rte_flow
> > > > > objects b)Similar case for runtime API. i.e Have helper
> > > > > functions to translate
> > > > > p4 MatchField name etc to appropriate rte_flow objects.
> > > > > c)Enhance base rte_flow specification if there are any
> > > > > fundamental gaps to express the new pattern or actions (which is
> > > > > not specific to
> > > > > p4 and applicable for any flow matching use case)
> > > > >
> > > > > If we introduce compiler in the pipeline, a lot of translation
> > > > > will get in the slowpath. And for runtime API, the translation
> > > > > primarily will be name to rte_flow object lookup (which is not
> > > > > that costly) and using rte_flow_template etc. to amortize the cost by
> making it burst.
> > > > >
> > > > >  Just my 2c.
> >