Re: [dpdk-dev] [PATCH v3 0/4] add AESNI-MB rawdev for multi-function processing

[Kevin Traynor <ktraynor@redhat.com>]

Hi David,

On 21/04/2020 18:23, Coyle, David wrote:
> Thank you Thomas for your input.
> 
> We would like to request that the Tech-Board (CC'ed) also review the proposal to help us reach a consensus.
>  

The discussion on the mailing list still looks active and I think that's
where it should continue until there is no reasonable hope of consensus.
I'm not sure discussing over irc at TB will find a better technical
solution.

> If the current proposal is not acceptable, we would welcome feedback from the board on how to rework our
> proposal to something that would be acceptable.
>  
> For the benefit of the Tech-Board here is the back-ground to our proposal for Rawdev-based multi-function
> processing:
> - The primary objective is to support the AESNI MB combined Crypto-CRC processing capability in DPDK and
>    in future to add support for combined Crypto-CRC support in QAT.
> - The cryptodev API was considered unsuitable because CRC is not a cryptographic operation, and this would
>    also preclude other non-crypto operations in the future such as compression.
> - The rte_security API was also not considered suitable for chaining of non-crypto operations such as CRC,
>    as Declan pointed out below.
> - A new Accelerator API was proposed as an RFC but was not pursued due to community feedback that a
>    new API would not be welcome for a single use-case.
> - Using Rawdev for multi-function processing was then proposed and, initially, as there was no opposition
>    we implemented a patch-set for this approach.
>  
> It was considered that a Rawdev-based multi-function approach would be suitable for the following reasons:
> 1) Multi-function processing for Crypto-CRC cases is not a good fit for any of the existing DPDK classes.
> 2) Rawdev was intended for such specialized acceleration processing that are not a good fit for existing DPDK
>      classes.
> 3) Rawdev was also intended as somewhere that new use-cases like this could be prototyped and developed,
>      such as Declan mentions below
> 4) The Rawdev-based multi-function proposal is extensible and we would hope that it can evolve to support
>      new use-cases and target new devices in the future with the communities involvement.
> 

This is a useful summary and explaining your approach but it doesn't
mention the counter arguments, so it doesn't seem balanced. Of course
people can read that in the ML thread.

Kevin.

> 
>> -----Original Message-----
>> From: Doherty, Declan <declan.doherty@intel.com>
>> Sent: Tuesday, April 21, 2020 5:46 PM
>>
>> On 15/04/2020 11:33 PM, Thomas Monjalon wrote:
>>> 16/04/2020 00:19, Doherty, Declan:
>>>> On 14/04/2020 3:44 PM, Thomas Monjalon wrote:
>>>>> 14/04/2020 16:02, Trahe, Fiona:
>>>>>> From: Thomas Monjalon <thomas@monjalon.net>
>>>>>>> 14/04/2020 15:04, Trahe, Fiona:
>>>>>>>>> 14/04/2020 12:21, Ferruh Yigit:
>>>>>>>>>
>>>>>>>
>> http://inbox.dpdk.org/dev/MN2PR11MB35507D4B96677A41E66440C5E3C30
>> @M
>>>>>>> N2PR11MB3550.na
>>>>>>>>> mprd11.prod.outlook.com/
>>>>>>>>>
>>>>>>>>> I am not convinced.
>>>>>>>>> I don't like rawdev in general.
>>>>>>>>> Rawdev is good only for hardware support which cannot be generic
>>>>>>>>> like SoC, FPGA management or DMA engine.
>>>>>>>>
>>>>>>>> [Fiona] CRC and BIP are not crypto algorithms, they are error
>> detection processes.
>>>>>>>> So there is no class in DPDK that these readily fit into.
>>>>>>>> There was resistance to adding another xxxddev, and even if one
>>>>>>>> had been added for error_detection_dev, there would still have
>>>>>>>> been another layer needed to couple this with cryptodev. Various
>>>>>>>> proposals for this have been discussed on the ML in RFC and recent
>> patches, there doesn't seem to be an appetite for this as a generic API.
>>>>>>>> So it seems that only Intel has software and hardware engines
>>>>>>>> that provide this specialised feature coupling. In that case
>>>>>>>> rawdev seems like the most appropriate vehicle to expose this.
>>>>>>>
>>>>>>> Adding some vendor-specific API is not a good answer.
>>>>>>> It will work in some cases, but it won't make DPDK better.
>>>>>>> What's the purpose of DPDK if it's not solving a common problem
>>>>>>> for different hardware?
>>>>>>
>>>> The current proposal in rawdev could easily be supported by any
>>>> hardware which supports chaining multiple functions/services into a
>>>> single operation, in this case symmetric crypto and error detection,
>>>> but it could conceivably support chaining symmetric/asymmetric crypto
>>>> operations or chaining symmetric crypto and compression operations.
>>>>
>>>>>> [Fiona] Based on that logic rawdev should be deprecated.
>>>>>> But the community has agreed that it has a place.
>>>>>
>>>>> No, as I said above, rawdev is good for SoC, FPGA management or DMA
>> engine.
>>>>
>>>> I distinctly remember when rawdev was being proposed one of the uses
>>>> cases proposed was that a new classes of APIs could be prototyped and
>>>> developed under rawdev and when a solid consensus was reached then
>>>> migrated to a mainstream DPDK library. I think every effort has been
>>>> made here to engage the community to develop a generic approach. As
>>>> Fiona notes there hasn't really been much of an appetite for this.
>>>>
>>>> Therefore I think the option to use rawdev makes sense, it allows an
>>>> initial proposal to be deployed,  without a generic solution
>>>> agreement, it will also give others in the community to see how this
>>>> approach can work and hopefully lead to more engagement on a generic
>>>> solution. Also as APIs in rawdev are essentially treated as private
>>>> APIs the onus is on Intel to support this going forward.
>>>
>>> Because hardware support is pending,
>>> we should accept an Intel-only "temporary" solution, opening the door
>>> to more vendor-specific APIs?
>>>
>>> What is the benefit for the DPDK project?
>>>
>> Sorry I don't agree with this sentiment, David has made every attempt to
>> solicit feedback and to engage the community in this.
>>
>> I also don't agree in classifying this as a "temporary solution" as this is a solid
>> proposal for an approach to chaining multiple operations together, but I
>> guess the fact remains that we only currently have a single use-case, but it is
>> difficult to generate a generic solution in this case.
>>
>> While there is only a single use case it is targeting two devices so that drove
>> the need for a common interface within rawdev.
>>
>> The advantage of using rawdev is that it allows this to be consumed through
>> DPDK, which enables DPDK project consumers, but also leaves the door open
>> to other contributors to have their say on how this should evolve. For
>> example this exact process seems to be occurring with DMA engines in
>> rawdev today, with a critical mass of implementations which now is giving the
>> impetus to create a generic solution, as we would hope can occur here too in
>> the future.
>>
>>
>>>>>> And the common problem here is device exposure.
>>>>>> With a specialised service on top.
>>>>>>
>>>>>>
>>>>>>>>> Here the intent is to use rawdev because we don't find a good API.
>>>>>>>>> API defeat is a no-go.
>>>>>>>>
>>>>>>>> [Fiona] It's not that we haven't found a good API, but that there
>>>>>>>> doesn't seem to be a general requirement for such a specialised
>>>>>>>> API
>>>>>>>
>>>>>>> There is a requirement to combine features of different classes.
>>>>>>
>>>>>> [Fiona] Can you point me to that requirement please?
>>>>>
>>>>> Yes, rte_security is trying to address this exact issue.
>>>>>
>>>>
>>>> I don't agree rte_security addresses the problem of different device
>>>> types supporting the same services. The problem being addressed here
>>>> is a single device which supports the chaining of multiple services
>>>> (sym crypto & error detection)
>>>
>>> Doing IPsec processing in Rx or Tx of a NIC is not chaining?
>>>
>> I wouldn't consider an inline crypto offload or full IPsec offload a chained
>> operation in the vein being proposed here where completely independent
>> services (in the view of DPDK which are currently on independent devices
>> and APIs) are linked together.
>>
>> We did look at using rte_security here but it wasn't considered suitable for a
>> chaining of non-crypto operations such as CRC or possibly compression in the
>> future, as it would still run into the issue of having to use the cryptodev
>> enq/deq API in the lookaside offload case.
>>
>>
>>>>>> We suggested it, but did not get community engagement and have
>>>>>> dropped our generic API requirement, instead focussing on this
>> specialised case.
>>>>>
>>>>> I did not see such generic proposal, sorry.
>>>>>
>>>>>>> In the past, rte_security was an "answer" to this issue with crypto and
>> ethdev.
>>>>>>> This is a real topic, please let's find a generic elegant solution.
>>>
>>>
>>>