Re: [dpdk-dev] [PATCH v2 00/12] add eventmode to ipsec-secgw

["Ananyev, Konstantin" <konstantin.ananyev@intel.com>]

> 
> Hi Akhil, Konstantin,
> 
> Do you have any further comments?

Will try to have a proper look today/tomorrow.
Sorry for delay.
Konstantin

> 
> Thanks,
> Anoob
> 
> > -----Original Message-----
> > From: dev <dev-bounces@dpdk.org> On Behalf Of Anoob Joseph
> > Sent: Monday, January 20, 2020 7:15 PM
> > To: Akhil Goyal <akhil.goyal@nxp.com>; Radu Nicolau
> > <radu.nicolau@intel.com>; Thomas Monjalon <thomas@monjalon.net>
> > Cc: Anoob Joseph <anoobj@marvell.com>; Jerin Jacob Kollanukkaran
> > <jerinj@marvell.com>; Narayana Prasad Raju Athreya
> > <pathreya@marvell.com>; Ankur Dwivedi <adwivedi@marvell.com>;
> > Archana Muniganti <marchana@marvell.com>; Tejasree Kondoj
> > <ktejasree@marvell.com>; Vamsi Krishna Attunuru
> > <vattunuru@marvell.com>; Lukas Bartosik <lbartosik@marvell.com>;
> > Konstantin Ananyev <konstantin.ananyev@intel.com>; dev@dpdk.org
> > Subject: [dpdk-dev] [PATCH v2 00/12] add eventmode to ipsec-secgw
> >
> > This series introduces event-mode additions to ipsec-secgw. This effort is
> > parallel to the similar changes in l2fwd (l2fwd-event app) & l3fwd.
> >
> > With this series, ipsec-secgw would be able to run in eventmode. The worker
> > thread (executing loop) would be receiving events and would be submitting
> > it back to the eventdev after the processing. This way, multicore scaling and
> > h/w assisted scheduling is achieved by making use of the eventdev
> > capabilities.
> >
> > Since the underlying event device would be having varying capabilities, the
> > worker thread could be drafted differently to maximize performance.
> > This series introduces usage of multiple worker threads, among which the
> > one to be used will be determined by the operating conditions and the
> > underlying device capabilities.
> >
> > For example, if an event device - eth device pair has Tx internal port, then
> > application can do tx_adapter_enqueue() instead of regular
> > event_enqueue(). So a thread making an assumption that the device pair has
> > internal port will not be the right solution for another pair. The infrastructure
> > added with these patches aims to help application to have multiple worker
> > threads, there by extracting maximum performance from every device
> > without affecting existing paths/use cases.
> >
> > The eventmode configuration is predefined. All packets reaching one eth
> > port will hit one event queue. All event queues will be mapped to all event
> > ports. So all cores will be able to receive traffic from all ports.
> > When schedule_type is set as RTE_SCHED_TYPE_ORDERED/ATOMIC, event
> > device will ensure the ordering. Ordering would be lost when tried in
> > PARALLEL.
> >
> > Following command line options are introduced,
> >
> > --transfer-mode: to choose between poll mode & event mode
> > --schedule-type: to specify the scheduling type
> >                  (RTE_SCHED_TYPE_ORDERED/
> >                   RTE_SCHED_TYPE_ATOMIC/
> >                   RTE_SCHED_TYPE_PARALLEL)
> >
> > Additionally the event mode introduces two modes of processing packets:
> >
> > Driver-mode: This mode will have bare minimum changes in the application
> >              to support ipsec. There woudn't be any lookup etc done in
> >              the application. And for inline-protocol use case, the
> >              thread would resemble l2fwd as the ipsec processing would be
> >              done entirely in the h/w. This mode can be used to benchmark
> >              the raw performance of the h/w. All the application side
> >              steps (like lookup) can be redone based on the requirement
> >              of the end user. Hence the need for a mode which would
> >              report the raw performance.
> >
> > App-mode: This mode will have all the features currently implemented with
> >           ipsec-secgw (non librte_ipsec mode). All the lookups etc
> >           would follow the existing methods and would report numbers
> >           that can be compared against regular ipsec-secgw benchmark
> >           numbers.
> >
> > The driver mode is selected with existing --single-sa option (used also by poll
> > mode). When --single-sa option is used in conjution with event mode then
> > index passed to --single-sa is ignored.
> >
> > Example commands to execute ipsec-secgw in various modes on OCTEON
> > TX2 platform,
> >
> > #Inbound and outbound app mode
> > ipsec-secgw -w 0002:02:00.0,ipsec_in_max_spi=128 -w
> > 0002:03:00.0,ipsec_in_max_spi=128 -w 0002:0e:00.0 -w 0002:10:00.1 --log-
> > level=8 -c 0x1 -- -P -p 0x3 -u 0x1 --config "(1,0,0),(0,0,0)" -f aes-gcm.cfg --
> > transfer-mode event --schedule-type parallel
> >
> > #Inbound and outbound driver mode
> > ipsec-secgw -w 0002:02:00.0,ipsec_in_max_spi=128 -w
> > 0002:03:00.0,ipsec_in_max_spi=128 -w 0002:0e:00.0 -w 0002:10:00.1 --log-
> > level=8 -c 0x1 -- -P -p 0x3 -u 0x1 --config "(1,0,0),(0,0,0)" -f aes-gcm.cfg --
> > transfer-mode event --schedule-type parallel --single-sa 0
> >
> > This series adds non burst tx internal port workers only. It provides
> > infrastructure for non internal port workers, however does not define any.
> > Also, only inline ipsec protocol mode is supported by the worker threads
> > added.
> >
> > Following are planned features,
> > 1. Add burst mode workers.
> > 2. Add non internal port workers.
> > 3. Verify support for Rx core (the support is added but lack of h/w to verify).
> > 4. Add lookaside protocol support.
> >
> > Following are features that Marvell won't be attempting.
> > 1. Inline crypto support.
> > 2. Lookaside crypto support.
> >
> > For the features that Marvell won't be attempting, new workers can be
> > introduced by the respective stake holders.
> >
> > This series is tested on Marvell OCTEON TX2.
> >
> > Changes in v2:
> > * Remove --process-dir option. Instead use existing unprotected port mask
> >   option (-u) to decide wheter port handles inbound or outbound traffic.
> > * Remove --process-mode option. Instead use existing --single-sa option
> >   to select between app and driver modes.
> > * Add handling of PKT_RX_SEC_OFFLOAD_FAIL result in app worker thread.
> > * Fix passing of req_rx_offload flags to create_default_ipsec_flow().
> > * Move destruction of flows to a location where eth ports are stopped
> >   and closed.
> > * Print error and exit when event mode --schedule-type option is used
> >   in poll mode.
> > * Reduce number of goto statements replacing them with loop constructs.
> > * Remove sec_session_fixed table and replace it with locally build
> >   table in driver worker thread. Table is indexed by port identifier
> >   and holds first inline session pointer found for a given port.
> > * Print error and exit when sessions other than inline are configured
> >   in event mode.
> > * When number of event queues is less than number of eth ports then
> >   map all eth ports to one event queue.
> > * Cleanup and minor improvements in code as suggested by Konstantin
> >
> > Deferred to v3:
> > * The final patch updates the hardcoded number of buffers in a pool.
> >   Also, there was a discussion on the update of number of qp. Both the
> >   above can be handled properly, if we can remove the logic which limits
> >   one core to only use one crypto qp. If we can allow one qp per
> >   lcore_param, every eth queue can have it's own crypto qp and that would
> >   solve the requirements with OCTEON TX2 inline ipsec support as well.
> >
> >   Patch with the mentioned change,
> >   https://urldefense.proofpoint.com/v2/url?u=http-
> > 3A__patches.dpdk.org_patch_64408_&d=DwIDAg&c=nKjWec2b6R0mOyPaz
> > 7xtfQ&r=BPcGOOudUMrTDQ9YbgKcOkO5ChYiUPPlPNIEvTOhjNE&m=rg71UQ
> > 1CwRYPFy30QuJQZd1Lam_kwYg15N2h5GN2iD4&s=yHzfRBRuunl4JWV97vufk
> > 7aycUc472ahPVnQ9Tt6SeY&e=
> >
> > * Update ipsec-secgw documentation to describe the new options as well as
> >   event mode support.
> >
> > This series depends on the PMD changes submitted in the following set,
> > https://urldefense.proofpoint.com/v2/url?u=http-
> > 3A__patches.dpdk.org_project_dpdk_list_-3Fseries-
> > 3D8203&d=DwIDAg&c=nKjWec2b6R0mOyPaz7xtfQ&r=BPcGOOudUMrTDQ9Y
> > bgKcOkO5ChYiUPPlPNIEvTOhjNE&m=rg71UQ1CwRYPFy30QuJQZd1Lam_kwY
> > g15N2h5GN2iD4&s=g2wtO9tOQTYHa9os1ECz5uwgpz9JmjTlGbEl-
> > Cp6WAw&e=
> >
> > Ankur Dwivedi (1):
> >   examples/ipsec-secgw: add default rte_flow for inline Rx
> >
> > Anoob Joseph (5):
> >   examples/ipsec-secgw: add framework for eventmode helper
> >   examples/ipsec-secgw: add eventdev port-lcore link
> >   examples/ipsec-secgw: add Rx adapter support
> >   examples/ipsec-secgw: add Tx adapter support
> >   examples/ipsec-secgw: add routines to display config
> >
> > Lukasz Bartosik (6):
> >   examples/ipsec-secgw: add routines to launch workers
> >   examples/ipsec-secgw: add support for internal ports
> >   examples/ipsec-secgw: add eventmode to ipsec-secgw
> >   examples/ipsec-secgw: add driver mode worker
> >   examples/ipsec-secgw: add app mode worker
> >   examples/ipsec-secgw: add cmd line option for bufs
> >
> >  examples/ipsec-secgw/Makefile       |    2 +
> >  examples/ipsec-secgw/event_helper.c | 1714
> > +++++++++++++++++++++++++++++++++++
> >  examples/ipsec-secgw/event_helper.h |  312 +++++++  examples/ipsec-
> > secgw/ipsec-secgw.c  |  502 ++++++++--
> >  examples/ipsec-secgw/ipsec-secgw.h  |   86 ++
> >  examples/ipsec-secgw/ipsec.c        |    7 +
> >  examples/ipsec-secgw/ipsec.h        |   36 +-
> >  examples/ipsec-secgw/ipsec_worker.c |  656 ++++++++++++++
> >  examples/ipsec-secgw/ipsec_worker.h |   39 +
> >  examples/ipsec-secgw/meson.build    |    4 +-
> >  examples/ipsec-secgw/sa.c           |   11 -
> >  11 files changed, 3275 insertions(+), 94 deletions(-)  create mode 100644
> > examples/ipsec-secgw/event_helper.c
> >  create mode 100644 examples/ipsec-secgw/event_helper.h
> >  create mode 100644 examples/ipsec-secgw/ipsec-secgw.h
> >  create mode 100644 examples/ipsec-secgw/ipsec_worker.c
> >  create mode 100644 examples/ipsec-secgw/ipsec_worker.h
> >
> > --
> > 2.7.4