Re: [dpdk-dev] [PATCH v3 0/3] add fallback session

[Anoob Joseph <anoobj@marvell.com>]

Hi Konstantin,

Please see inline.

Thanks,
Anoob

> -----Original Message-----
> From: Ananyev, Konstantin <konstantin.ananyev@intel.com>
> Sent: Thursday, September 26, 2019 6:08 PM
> To: Anoob Joseph <anoobj@marvell.com>; Smoczynski, MarcinX
> <marcinx.smoczynski@intel.com>; akhil.goyal@nxp.com
> Cc: dev@dpdk.org; Jerin Jacob Kollanukkaran <jerinj@marvell.com>; Narayana
> Prasad Raju Athreya <pathreya@marvell.com>; Archana Muniganti
> <marchana@marvell.com>
> Subject: RE: [dpdk-dev] [PATCH v3 0/3] add fallback session
> 
> 
> Hi Anoob,
> Answers/comments inline.
> Marcin, please correct me, if I missed something.
> Konstantin
> 
> >
> > Hi Marcin, Konstantin,
> >
> > I've few more observations regarding the proposed feature.
> >
> > 1. From what I understood, if an ESP packet ends up on an unprotected
> > interface and doesn't have 'PKT_RX_SEC_OFFLOAD' bit set, then the packet
> would be looked up to see the associated SA and then fallback session is figured
> out and then further processing is done.
> >
> > Can you confirm if I understood the sequence correctly? If yes, then
> > aren't we doing an extra lookup in the s/w? The packet may be looked
> > by the h/w using rte_flow and that information could be used to determine the
> SA. Also, if the ESP packet is expected to be forwarded, then the above logic will
> add an unnecessary lookup even after your h/w has detected that the packet
> need not be security processed.
> 
> Not sure I understood your whole statement above.
> AFAIK, right now (with dpdk master) for unprotected iface it works like that:
> 
> 1.  slit incoming traffic into 3 groups: ESP packets, IPv4 packets, IPv6 packets.
> For ESP packets:
> 2. perform SAD lookup
>     a. if it fails (non SA found for that SPI), then drop the packet.
>     b. otherwise (SA found) process the packet using found SA
> 
> What fall-back patch adds:
> Before step 2.b check:
> does that SA has its primary session  with type INLINE-CRYPTO and
> does HW fail to do IPsec realted processing for it (by some reason)?
> If yes, then mark this packet to be processed by fall-back session.
> if (MBUF_NO_SEC_OFFLOAD(pkt) && sa->fallback_sessions > 0) {
>                 uintptr_t intsa = (uintptr_t)sa;
>                 intsa |= IPSEC_SA_OFFLOAD_FALLBACK_FLAG;
>                 result_sa = (void *)intsa;  }
> 
> So from my perspective, no additional lookup where introduced.

[Anoob] For inline processing, h/w does a lookup and figures out the security processing to be done on the packet. "rte_security_get_userdata()" allows s/w to further segregate that info. In this approach, I believe we don't consider how such info from h/w can be used.
 
> Also AFAIK, right now there is no possibility to configure ipsec-secgw to BYPASS
> some ESP traffic.
> Should we do it (to conform to ipsec RFC) - that's probably subject of another
> discussion.

[Anoob] The approach (choice of flags) forces a software-based SA lookup for packets that need to be bypassed instead of leveraging possible hardware SA lookup. I don't think what ipsec-secgw does matters here.

For example, ESN was not supported by ipsec-secgw (before library mode was introduced), but every single library change and application change was added keeping in mind that ESN is a valid feature for ipsec. So it is one thing to say ipsec-secgw doesn't support ESP bypass and saying the solution doesn't consider a possibility of ESP bypass. 
 
> 
> >
> > 2. The solution proposed here seems like adding the handling in
> > ipsec-secgw instead of ipsec library. In other words, this feature is not getting
> added in ipsec library, which was supposed to simplify the whole ipsec usage in
> DPDK, but fails to handle the case of fragmentation.
> 
> What we have right now with ipsec library is SA (low) level API.
> It can handle multi-segment packets properly, but expects someone else to do
> other steps (fragmentation/reassembly).
> ipsec-secgw demonstrates how librte_ip_frag and librte_ipsec can be used
> together to deal with fragmented IPsec traffic in a proper manner.
> Probably in future we'll consider adding some high-level API that will be able to
> do whole processing under hood (SPD/SAD lookup, fragment/reassembly, actual
> IPsec packet processing, matching inbound selectors, etc.), but right now it is
> not the case.
> 
> > Also, since the fallback feature is entirely done in the application, it begs the
> question why the same feature is omitted for legacy use case.
> 
> Because legacy mode doesn't support multi-seg packets at first place.
> Also it is an extra overhead to support 2 code-paths (legacy and library) for the
> same app, so we'd like in future to deprecate and later remove legacy code-
> path.
> As a first step we propose to make library code-path a default one:
> http://patches.dpdk.org/cover/58247/

[Anoob] Even if we need the application to do the reassembly, it would look simple if application uses the same "rte_ipsec_session" to process the fallback packet. In that case, the processing required to handle the other packet will be hidden from the application. Here application has to make sure it chooses the correct session, which I believe should be moved to ipsec library.

> 
> >
> > 3. It seems like ordering won't be maintained once this processing is
> > done. Again, this is the sequence I understood. Please correct me if I missed
> something,
> >        a. Application receives a bunch of packets (let's say 6
> > packets), in which few are fragmented (P3 & P4) and the rest can be inline
> processed.
> >        b. Application receives P1->P2->P3->P4->P5->P6 (in this, P1, P2, P5, P6 are
> inline processed successfully) and P4 & P5 are the fragments
> >        c. Application groups packets. P1->P2->P5->P6 becomes one group and P3-
> >P4 becomes another and goes for fallback processing.
> > Now how is ordering maintained? I couldn't figure out how that is done in this
> case.
> 
> You right, fallback session can introduce packet reordering.
> At least till we'll have ability to process packets in sync mode too.
> See our presentation at dpdk userspace (slides 17, 18):
> https://static.sched.com/hosted_files/dpdkbordeaux2019/8f/DPDK-IPSECu9.pdf
> Right now the only way to deal with it - have replay window big enough to
> sustain reordering and async processing latency.
> That's actually another reason why we add this feature into ipsec-secgw sample
> app:
> so people can evaluate it on their platforms, determine what replay window size
> would be needed, what issues/slowdowns it might cause, etc.

[Anoob] This is something I had noticed while going through the code flow. The ordering info is lost because of the approach. As we dig deeper, the feature looks hardly complete. The concerning part is the approach doesn't seem conducive to fixing any of these issues in the future.

Also, if the new ipsec related features are introduced via ipsec-secgw than via rte_ipsec, then it raises doubts over the utility of rte_ipsec library.
 
> 
> >
> > Thanks,
> > Anoob
> >
> > > -----Original Message-----
> > > From: dev <dev-bounces@dpdk.org> On Behalf Of Marcin Smoczynski
> > > Sent: Monday, September 23, 2019 5:14 PM
> > > To: Anoob Joseph <anoobj@marvell.com>; akhil.goyal@nxp.com;
> > > konstantin.ananyev@intel.com
> > > Cc: dev@dpdk.org; Marcin Smoczynski <marcinx.smoczynski@intel.com>
> > > Subject: [dpdk-dev] [PATCH v3 0/3] add fallback session
> > >
> > > Add fallback session feature allowing to process packets that inline
> > > processor is unable to handle (e.g. fragmented traffic). Processing
> > > takes place in a secondary session defined for SA in a configuration file.
> > >
> > > This feature is limited to ingress IPsec traffic only. IPsec
> > > anti-replay window and ESN are supported in conjunction with
> > > fallback session when following conditions are met:
> > >  * primary session is 'inline-crypto-offload,
> > >  * fallback sessions is 'lookaside-none'.
> > >
> > > v2 to v3 changes:
> > >  - doc and commit log update - explicitly state feature limitations
> > >
> > > v1 to v2 changes:
> > >  - disable fallback offload for outbound SAs
> > >  - add test scripts
> > >
> > > Marcin Smoczynski (3):
> > >   examples/ipsec-secgw: ipsec_sa structure cleanup
> > >   examples/ipsec-secgw: add fallback session feature
> > >   examples/ipsec-secgw: add offload fallback tests
> > >
> > >  doc/guides/sample_app_ug/ipsec_secgw.rst      |  20 ++-
> > >  examples/ipsec-secgw/esp.c                    |  35 ++--
> > >  examples/ipsec-secgw/ipsec-secgw.c            |  16 +-
> > >  examples/ipsec-secgw/ipsec.c                  |  99 ++++++-----
> > >  examples/ipsec-secgw/ipsec.h                  |  61 +++++--
> > >  examples/ipsec-secgw/ipsec_process.c          | 113 +++++++-----
> > >  examples/ipsec-secgw/sa.c                     | 164 +++++++++++++-----
> > >  .../test/trs_aesgcm_common_defs.sh            |   4 +-
> > >  .../trs_aesgcm_inline_crypto_fallback_defs.sh |   5 +
> > >  .../test/tun_aesgcm_common_defs.sh            |   6 +-
> > >  .../tun_aesgcm_inline_crypto_fallback_defs.sh |   5 +
> > >  11 files changed, 361 insertions(+), 167 deletions(-)  create mode
> > > 100644
> > > examples/ipsec-secgw/test/trs_aesgcm_inline_crypto_fallback_defs.sh
> > >  create mode 100644 examples/ipsec-
> > > secgw/test/tun_aesgcm_inline_crypto_fallback_defs.sh
> > >
> > > --
> > > 2.17.1