Re: [dpdk-dev] [RFC] ipsec: new library for IPsec data-path processing

[Jerin Jacob <jerin.jacob@caviumnetworks.com>]

-----Original Message-----
> Date: Sat, 15 Sep 2018 22:36:18 +0530
> From: "Joseph, Anoob" <Anoob.Joseph@caviumnetworks.com>
> To: "Ananyev, Konstantin" <konstantin.ananyev@intel.com>, "dev@dpdk.org"
>  <dev@dpdk.org>
> Cc: "Awal, Mohammad Abdul" <mohammad.abdul.awal@intel.com>, "Doherty,
>  Declan" <declan.doherty@intel.com>, "Jerin Jacob
>  (jerin.jacob@caviumnetworks.com)" <jerin.jacob@caviumnetworks.com>,
>  Narayana Prasad <narayanaprasad.athreya@caviumnetworks.com>
> Subject: Re: [dpdk-dev] [RFC] ipsec: new library for IPsec data-path
>  processing
> User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101
>  Thunderbird/52.9.1
> 
> > 
> > Anyway, let's pretend we found some smart way to distribute inbound packets for the same SA to multiple HW queues/CPU cores.
> > To make ipsec processing for such case to work correctly just atomicity on check/update segn/replay_window is not enough.
> > I think it would require some extra synchronization:
> > make sure that we do final packet processing (seq check/update) at the same order as we received the packets
> > (packets entered ipsec processing).
> > I don't really like to introduce such heavy mechanisms on SA level,  after all it supposed to be light and simple.
> > Though we plan CTX level API to support such scenario.
> > What I think would be useful addition for SA level API - have an ability to do one update seqn/replay_window and multiple checks concurrently.
> > 
> > > In case of ingress also, the same problem exists. We will not be able to use RSS and spread the traffic to multiple cores. Considering
> > > IPsec being CPU intensive, this would limit the net output of the chip.
> > That's true - but from other side implementation can offload heavy part
> > (encrypt/decrypt, auth) to special HW (cryptodev).
> > In that case single core might be enough for SA and extra synchronization would just slowdown things.
> > That's why I think it should be configurable  what behavior (ST or MT) to use.
> I do agree that these are the issues that we need to address to make the
> library MT safe. Whether the extra synchronization would slow down things is
> a very subjective question and will heavily depend on the platform. The
> library should have enough provisions to be able to support MT without
> causing overheads to ST. Right now, the library assumes ST.


I agree with Anoob here.

I have two concerns with librte_ipsec as a separate library

1) There is an overlap with rte_security and new proposed library.
For IPsec, If an application needs to use rte_security for HW
implementation and and application needs to use librte_ipsec for
 SW implementation then it is bad and a lot duplication of work on 
he slow path too.

The rte_security spec can support both inline and look-aside IPSec
protocol support.

2) This library is tuned for fat CPU core in mind like single SA on core
etc. Which is fine for x86 servers and arm64 server category of machines
but it does not work very well with NPU class of SoC or FPGA.

As there  are the different ways to implement the IPSec, For instance,
use of eventdev can help in situation for handling millions of SA and 
equence number of update and anti reply check can be done by leveraging 
some of the HW specific features like 
ORDERED, ATOMIC schedule type(mapped as eventdev feature)in HW with PIPELINE model.

# Issues with having one SA one core,
- In the outbound side, there could be multiple flows using the same SA.
  Multiple flows could be processed parallel on different lcores,
but tying one SA to one core would mean we won't be able to do that.

- In the inbound side, we will have a fat flow hitting one core. If
  IPsec library assumes single core, we will not be able to to spread
fat flow to multiple cores. And one SA-one core would mean all ports on
which we would expect IPsec traffic has to be handled by that core.

I have made a simple presentation. This presentation details ONE WAY to
implement the IPSec with HW support on NPU.

https://docs.google.com/presentation/d/1e3IDf9R7ZQB8FN16Nvu7KINuLSWMdyKEw8_0H05rjj4/edit?usp=sharing

I am not saying this should be the ONLY way to do as it does not work
very well with non NPU/FPGA class of SoC.

So how about making the proposed IPSec library as plugin/driver to
rte_security.
This would give flexibly for each vendor/platform choose to different
IPse implementation based on HW support WITHOUT CHANGING THE APPLICATION
INTERFACE.

IMO, rte_security IPsec look aside support can be simply added by
creating the virtual crypto device(i.e move the proposed code to the virtual crypto device)
likewise inline support
can be added by the virtual ethdev device. This would avoid the need for
updating ipsec-gw application as well i.e unified interface to application.

If you don't like the above idea, any scheme of plugin based
implementation would be fine so that vendor or platform can choose its own implementation.
It can be based on partial HW implement too. i.e SA look can be used in SW, remaining stuff in HW
(for example IPsec inline case)

# For protocols like UDP, it makes sense to create librte_udp as there
no much HW specific offload other than ethdev provides.

# PDCP could be another library to offload to HW, So talking
rte_security path makes more sense in that case too.

Jerin