Re: [dpdk-dev] [PATCH v3 0/4] add AESNI-MB rawdev for multi-function processing

["Coyle, David" <david.coyle@intel.com>]

Thank you Thomas for your input.

We would like to request that the Tech-Board (CC'ed) also review the proposal to help us reach a consensus.
 
If the current proposal is not acceptable, we would welcome feedback from the board on how to rework our
proposal to something that would be acceptable.
 
For the benefit of the Tech-Board here is the back-ground to our proposal for Rawdev-based multi-function
processing:
- The primary objective is to support the AESNI MB combined Crypto-CRC processing capability in DPDK and
   in future to add support for combined Crypto-CRC support in QAT.
- The cryptodev API was considered unsuitable because CRC is not a cryptographic operation, and this would
   also preclude other non-crypto operations in the future such as compression.
- The rte_security API was also not considered suitable for chaining of non-crypto operations such as CRC,
   as Declan pointed out below.
- A new Accelerator API was proposed as an RFC but was not pursued due to community feedback that a
   new API would not be welcome for a single use-case.
- Using Rawdev for multi-function processing was then proposed and, initially, as there was no opposition
   we implemented a patch-set for this approach.
 
It was considered that a Rawdev-based multi-function approach would be suitable for the following reasons:
1) Multi-function processing for Crypto-CRC cases is not a good fit for any of the existing DPDK classes.
2) Rawdev was intended for such specialized acceleration processing that are not a good fit for existing DPDK
     classes.
3) Rawdev was also intended as somewhere that new use-cases like this could be prototyped and developed,
     such as Declan mentions below
4) The Rawdev-based multi-function proposal is extensible and we would hope that it can evolve to support
     new use-cases and target new devices in the future with the communities involvement.


> -----Original Message-----
> From: Doherty, Declan <declan.doherty@intel.com>
> Sent: Tuesday, April 21, 2020 5:46 PM
> 
> On 15/04/2020 11:33 PM, Thomas Monjalon wrote:
> > 16/04/2020 00:19, Doherty, Declan:
> >> On 14/04/2020 3:44 PM, Thomas Monjalon wrote:
> >>> 14/04/2020 16:02, Trahe, Fiona:
> >>>> From: Thomas Monjalon <thomas@monjalon.net>
> >>>>> 14/04/2020 15:04, Trahe, Fiona:
> >>>>>>> 14/04/2020 12:21, Ferruh Yigit:
> >>>>>>>
> >>>>>
> http://inbox.dpdk.org/dev/MN2PR11MB35507D4B96677A41E66440C5E3C30
> @M
> >>>>> N2PR11MB3550.na
> >>>>>>> mprd11.prod.outlook.com/
> >>>>>>>
> >>>>>>> I am not convinced.
> >>>>>>> I don't like rawdev in general.
> >>>>>>> Rawdev is good only for hardware support which cannot be generic
> >>>>>>> like SoC, FPGA management or DMA engine.
> >>>>>>
> >>>>>> [Fiona] CRC and BIP are not crypto algorithms, they are error
> detection processes.
> >>>>>> So there is no class in DPDK that these readily fit into.
> >>>>>> There was resistance to adding another xxxddev, and even if one
> >>>>>> had been added for error_detection_dev, there would still have
> >>>>>> been another layer needed to couple this with cryptodev. Various
> >>>>>> proposals for this have been discussed on the ML in RFC and recent
> patches, there doesn't seem to be an appetite for this as a generic API.
> >>>>>> So it seems that only Intel has software and hardware engines
> >>>>>> that provide this specialised feature coupling. In that case
> >>>>>> rawdev seems like the most appropriate vehicle to expose this.
> >>>>>
> >>>>> Adding some vendor-specific API is not a good answer.
> >>>>> It will work in some cases, but it won't make DPDK better.
> >>>>> What's the purpose of DPDK if it's not solving a common problem
> >>>>> for different hardware?
> >>>>
> >> The current proposal in rawdev could easily be supported by any
> >> hardware which supports chaining multiple functions/services into a
> >> single operation, in this case symmetric crypto and error detection,
> >> but it could conceivably support chaining symmetric/asymmetric crypto
> >> operations or chaining symmetric crypto and compression operations.
> >>
> >>>> [Fiona] Based on that logic rawdev should be deprecated.
> >>>> But the community has agreed that it has a place.
> >>>
> >>> No, as I said above, rawdev is good for SoC, FPGA management or DMA
> engine.
> >>
> >> I distinctly remember when rawdev was being proposed one of the uses
> >> cases proposed was that a new classes of APIs could be prototyped and
> >> developed under rawdev and when a solid consensus was reached then
> >> migrated to a mainstream DPDK library. I think every effort has been
> >> made here to engage the community to develop a generic approach. As
> >> Fiona notes there hasn't really been much of an appetite for this.
> >>
> >> Therefore I think the option to use rawdev makes sense, it allows an
> >> initial proposal to be deployed,  without a generic solution
> >> agreement, it will also give others in the community to see how this
> >> approach can work and hopefully lead to more engagement on a generic
> >> solution. Also as APIs in rawdev are essentially treated as private
> >> APIs the onus is on Intel to support this going forward.
> >
> > Because hardware support is pending,
> > we should accept an Intel-only "temporary" solution, opening the door
> > to more vendor-specific APIs?
> >
> > What is the benefit for the DPDK project?
> >
> Sorry I don't agree with this sentiment, David has made every attempt to
> solicit feedback and to engage the community in this.
> 
> I also don't agree in classifying this as a "temporary solution" as this is a solid
> proposal for an approach to chaining multiple operations together, but I
> guess the fact remains that we only currently have a single use-case, but it is
> difficult to generate a generic solution in this case.
> 
> While there is only a single use case it is targeting two devices so that drove
> the need for a common interface within rawdev.
> 
> The advantage of using rawdev is that it allows this to be consumed through
> DPDK, which enables DPDK project consumers, but also leaves the door open
> to other contributors to have their say on how this should evolve. For
> example this exact process seems to be occurring with DMA engines in
> rawdev today, with a critical mass of implementations which now is giving the
> impetus to create a generic solution, as we would hope can occur here too in
> the future.
> 
> 
> >>>> And the common problem here is device exposure.
> >>>> With a specialised service on top.
> >>>>
> >>>>
> >>>>>>> Here the intent is to use rawdev because we don't find a good API.
> >>>>>>> API defeat is a no-go.
> >>>>>>
> >>>>>> [Fiona] It's not that we haven't found a good API, but that there
> >>>>>> doesn't seem to be a general requirement for such a specialised
> >>>>>> API
> >>>>>
> >>>>> There is a requirement to combine features of different classes.
> >>>>
> >>>> [Fiona] Can you point me to that requirement please?
> >>>
> >>> Yes, rte_security is trying to address this exact issue.
> >>>
> >>
> >> I don't agree rte_security addresses the problem of different device
> >> types supporting the same services. The problem being addressed here
> >> is a single device which supports the chaining of multiple services
> >> (sym crypto & error detection)
> >
> > Doing IPsec processing in Rx or Tx of a NIC is not chaining?
> >
> I wouldn't consider an inline crypto offload or full IPsec offload a chained
> operation in the vein being proposed here where completely independent
> services (in the view of DPDK which are currently on independent devices
> and APIs) are linked together.
> 
> We did look at using rte_security here but it wasn't considered suitable for a
> chaining of non-crypto operations such as CRC or possibly compression in the
> future, as it would still run into the issue of having to use the cryptodev
> enq/deq API in the lookaside offload case.
> 
> 
> >>>> We suggested it, but did not get community engagement and have
> >>>> dropped our generic API requirement, instead focussing on this
> specialised case.
> >>>
> >>> I did not see such generic proposal, sorry.
> >>>
> >>>>> In the past, rte_security was an "answer" to this issue with crypto and
> ethdev.
> >>>>> This is a real topic, please let's find a generic elegant solution.
> >
> >
> >