From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from dpdk.org (dpdk.org [92.243.14.124])
	by inbox.dpdk.org (Postfix) with ESMTP id CC710A04B3;
	Tue, 28 Jan 2020 06:02:36 +0100 (CET)
Received: from [92.243.14.124] (localhost [127.0.0.1])
	by dpdk.org (Postfix) with ESMTP id 5B7E61C012;
	Tue, 28 Jan 2020 06:02:35 +0100 (CET)
Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com
 [67.231.148.174]) by dpdk.org (Postfix) with ESMTP id 7AEE32952
 for <dev@dpdk.org>; Tue, 28 Jan 2020 06:02:33 +0100 (CET)
Received: from pps.filterd (m0045849.ppops.net [127.0.0.1])
 by mx0a-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id
 00S52En7019791; Mon, 27 Jan 2020 21:02:32 -0800
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com;
 h=from : to : cc :
 subject : date : message-id : references : in-reply-to : content-type :
 content-transfer-encoding : mime-version; s=pfpt0818;
 bh=HBevC1G9hUkCW7L4lGwA6CNBZzP3lX7ZXAbfIdC6oyU=;
 b=kdx0RSJ41VQzzJwffjay35Kpl+NLOSj//mltNsEOWHHwKZBk7wCtP+gnnrm4MTDUXYEE
 8sgGBX+JypD8RP66joh4ObuFFCM6n4GL509k+BrP3uC+yBcTpjrl9pvOMoiwRiFzpvH3
 zKOCNP9Xg+4cq/au1hw8QTN4+cRzhyFutHHOXzK/GMdezGSKuD9sD3zX2UF1ZLdknu37
 Ks3p2CRASyqwJWVYFfRjWAfA/RhoTS1GkdjI/AEwoD3/LnGNMELogK28WO0lkRSWFmDq
 fmgz3Uwd/Pt8vbxrW22BSuhuic+bjTGvZdywczQpXWhipi87YPbyIMk6ePUQIupftyiX 0g== 
Received: from sc-exch04.marvell.com ([199.233.58.184])
 by mx0a-0016f401.pphosted.com with ESMTP id 2xt6bsss16-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT);
 Mon, 27 Jan 2020 21:02:32 -0800
Received: from SC-EXCH01.marvell.com (10.93.176.81) by SC-EXCH04.marvell.com
 (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 27 Jan
 2020 21:02:30 -0800
Received: from NAM12-DM6-obe.outbound.protection.outlook.com (104.47.59.169)
 by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server (TLS) id
 15.0.1497.2 via Frontend Transport; Mon, 27 Jan 2020 21:02:31 -0800
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=M+5El6A3ROv4P5jiAq/xzek5fqwoUx68Dki4RKeVcArqtTOVas2UdZpXXrQLbq2F1W0/d8sFilX1/B+I2Dob9MsUvWk4bgI0bDMVqOa68N1dtSy1eBuxnkRnadC7Z7donjm4nd14Ff3zjTXRjEWjU0XqT3DoXgS2gykOrsyiNaI7y1ZMdY72EWHSMJbwQlgzWvhNQpM2ojpXO+ESO7Nsej61fUVBzlGteR9FuNv1w0DTFr5R4dzzI9HAJ1h4vypjG7MyTSflOoIgHsVxA3ZYeYfNKw1npGUYpK3QWHX9zq3oAF1iFJD1rey5x+aQ+w1CFGGbnOGy/SZphdLRRRmtMA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; 
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=HBevC1G9hUkCW7L4lGwA6CNBZzP3lX7ZXAbfIdC6oyU=;
 b=D/aPP9bS9UWRNkDad+vo3b0hqn5M2hYBlR5vJI7kdeQztHDTKDiZq0lY/KZt7JSnmrycJ+YrCmbpMWNij+iXEFie9+XS+dUlatWOK8oIRPgkf5L+zCeoaZZtVnHR4VW+3uSMkA0gaGh5qtipsJaB5QLxRELuDM0OkAOtNuVhJJeg+zzoQMfUWr+IATby2Vnv4UdvtYTW870ZCqGVRSBQs2N3jB+qlvPUXDKUs+gJ7wrAOPnWqoanTpNIx9JBmWfkUgB2p+4M1tcZ+R0NEz0dCxFa9R5TQtB+XIQKIP6F556/bM+ydCXppfpf/cPAZ7J2G2P9QP5kdWS9WCjOV+NeMQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com;
 dkim=pass header.d=marvell.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=HBevC1G9hUkCW7L4lGwA6CNBZzP3lX7ZXAbfIdC6oyU=;
 b=QFjyxDyLq0xtBNRmaR1acZz5bjPLorHzGhR8joNSFzwYoEOjrLyvZ/uDoOYW/7ylVB00BKzumu7SwqMb+1GDQcG/OOHnyNQSL/u2oKdegV6iC/GiQ1hHlOSXofXMq17G4BAtanAUVePXbr8VKSyd8AvzWnB+VtpR6zQC/6HBODw=
Received: from MN2PR18MB2877.namprd18.prod.outlook.com (20.179.20.218) by
 MN2PR18MB2896.namprd18.prod.outlook.com (20.179.22.75) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.2665.20; Tue, 28 Jan 2020 05:02:28 +0000
Received: from MN2PR18MB2877.namprd18.prod.outlook.com
 ([fe80::e48d:494:fc46:3572]) by MN2PR18MB2877.namprd18.prod.outlook.com
 ([fe80::e48d:494:fc46:3572%7]) with mapi id 15.20.2665.026; Tue, 28 Jan 2020
 05:02:28 +0000
From: Anoob Joseph <anoobj@marvell.com>
To: Akhil Goyal <akhil.goyal@nxp.com>, "konstantin.ananyev@intel.com"
 <konstantin.ananyev@intel.com>
CC: Jerin Jacob Kollanukkaran <jerinj@marvell.com>, "Narayana Prasad Raju
 Athreya" <pathreya@marvell.com>, Ankur Dwivedi <adwivedi@marvell.com>,
 "Archana Muniganti" <marchana@marvell.com>, Tejasree Kondoj
 <ktejasree@marvell.com>, Vamsi Krishna Attunuru <vattunuru@marvell.com>,
 Lukas Bartosik <lbartosik@marvell.com>, Konstantin Ananyev
 <konstantin.ananyev@intel.com>, "dev@dpdk.org" <dev@dpdk.org>, Thomas
 Monjalon <thomas@monjalon.net>, "Radu Nicolau" <radu.nicolau@intel.com>,
 Anoob Joseph <anoobj@marvell.com>
Thread-Topic: [dpdk-dev] [PATCH v2 00/12] add eventmode to ipsec-secgw
Thread-Index: AQHVz5fzks5kwLxWl0Wby+6wad3ZSKf/kJMQ
Date: Tue, 28 Jan 2020 05:02:28 +0000
Message-ID: <MN2PR18MB2877182843F7818B3A36D485DF0A0@MN2PR18MB2877.namprd18.prod.outlook.com>
References: <1575808249-31135-1-git-send-email-anoobj@marvell.com>
 <1579527918-360-1-git-send-email-anoobj@marvell.com>
In-Reply-To: <1579527918-360-1-git-send-email-anoobj@marvell.com>
Accept-Language: en-IN, en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [14.140.231.66]
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: acdb6ca3-593c-4d0c-2efc-08d7a3af4624
x-ms-traffictypediagnostic: MN2PR18MB2896:
x-ms-exchange-transport-forked: True
x-microsoft-antispam-prvs: <MN2PR18MB289669403D62E9376865D84ADF0A0@MN2PR18MB2896.namprd18.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:6790;
x-forefront-prvs: 029651C7A1
x-forefront-antispam-report: SFV:NSPM;
 SFS:(10009020)(4636009)(136003)(39850400004)(346002)(396003)(376002)(366004)(189003)(199004)(966005)(2906002)(55016002)(478600001)(33656002)(9686003)(53546011)(316002)(107886003)(7696005)(6506007)(54906003)(4326008)(55236004)(26005)(186003)(110136005)(8676002)(81166006)(8936002)(52536014)(19627235002)(5660300002)(81156014)(66446008)(66556008)(64756008)(66476007)(86362001)(71200400001)(76116006)(66946007);
 DIR:OUT; SFP:1101; SCL:1; SRVR:MN2PR18MB2896;
 H:MN2PR18MB2877.namprd18.prod.outlook.com; FPR:; SPF:None; LANG:en;
 PTR:InfoNoRecords; MX:1; A:1; 
received-spf: None (protection.outlook.com: marvell.com does not designate
 permitted sender hosts)
x-ms-exchange-senderadcheck: 1
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: pAnIhqT8Tga2xFEiP/+nE0njUOhq6DBRC3x3c6ntZT3RX8DX/oO+2uw9PYwyVsDa5AEDMy1IhOabZQoSpCim+YERKwjjJybDVRJSHLzlP+mBi+dKO61/rcT3dzb5pCtvZ2/nEWkp711gPDsBDf2GjBebjm3SdRWboWsvI99Poevz02U25K9InWkebdy0AP3OotsuuB0KapR8mR9LdOABSL0Pot0gkSDVJyfMt3O1Kf9yU6e3D8+1EdL+MXpIMulevGCNGEoqOnzYTPgCA4yJ0pkuRJQK1SscmmKqgwRHJCU6H9UHhiE9nPDF8vP/eFbTINsc0sTgoURaO2OxrWmnVzItFz+toAxpOJns2LlRx6DfvaHm3twMPgbhSv2eXQfIUnqNZztpSE0CKTMbaVlUfWL3Ozq97G6jXc7GcWe/EVe9PFwUeGCNqA9Bk6Gly8qPhFHvhQSaSPhxs3FRhzo8SgWoVzzUNvuoigo2YnnNeVLufj6fdOiNDH+kZFG74fF1ZS5lwdfhDvBR+tLua95Qfw==
x-ms-exchange-antispam-messagedata: coRAGNGG+diw4hZF8/wnLnY/3fWeX1XCsI9ShXPAj57rkpQcKz11tOTElS4xK8aU0MCYq6D64GKTQzlj9RXrWt17+e6lDNTTVVg2a5LsFWT8ZJPoebif9L9vvPFYLU6BwOPPSj3rEp/bk1q7jcbqDw==
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-Network-Message-Id: acdb6ca3-593c-4d0c-2efc-08d7a3af4624
X-MS-Exchange-CrossTenant-originalarrivaltime: 28 Jan 2020 05:02:28.4736 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: PK9PhVhm4gNPxcVzVj4rQFo5/Ye9kLq2iyNeZLED0MPXvftO582mrxRzpGqqNWIJ6NXE4uuzM24GXnfoA12sHg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR18MB2896
X-OriginatorOrg: marvell.com
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138, 18.0.572
 definitions=2020-01-27_08:2020-01-24,
 2020-01-27 signatures=0
Subject: Re: [dpdk-dev] [PATCH v2 00/12] add eventmode to ipsec-secgw
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Sender: "dev" <dev-bounces@dpdk.org>

Hi Akhil, Konstantin,

Do you have any further comments?

Thanks,
Anoob

> -----Original Message-----
> From: dev <dev-bounces@dpdk.org> On Behalf Of Anoob Joseph
> Sent: Monday, January 20, 2020 7:15 PM
> To: Akhil Goyal <akhil.goyal@nxp.com>; Radu Nicolau
> <radu.nicolau@intel.com>; Thomas Monjalon <thomas@monjalon.net>
> Cc: Anoob Joseph <anoobj@marvell.com>; Jerin Jacob Kollanukkaran
> <jerinj@marvell.com>; Narayana Prasad Raju Athreya
> <pathreya@marvell.com>; Ankur Dwivedi <adwivedi@marvell.com>;
> Archana Muniganti <marchana@marvell.com>; Tejasree Kondoj
> <ktejasree@marvell.com>; Vamsi Krishna Attunuru
> <vattunuru@marvell.com>; Lukas Bartosik <lbartosik@marvell.com>;
> Konstantin Ananyev <konstantin.ananyev@intel.com>; dev@dpdk.org
> Subject: [dpdk-dev] [PATCH v2 00/12] add eventmode to ipsec-secgw
>=20
> This series introduces event-mode additions to ipsec-secgw. This effort i=
s
> parallel to the similar changes in l2fwd (l2fwd-event app) & l3fwd.
>=20
> With this series, ipsec-secgw would be able to run in eventmode. The work=
er
> thread (executing loop) would be receiving events and would be submitting
> it back to the eventdev after the processing. This way, multicore scaling=
 and
> h/w assisted scheduling is achieved by making use of the eventdev
> capabilities.
>=20
> Since the underlying event device would be having varying capabilities, t=
he
> worker thread could be drafted differently to maximize performance.
> This series introduces usage of multiple worker threads, among which the
> one to be used will be determined by the operating conditions and the
> underlying device capabilities.
>=20
> For example, if an event device - eth device pair has Tx internal port, t=
hen
> application can do tx_adapter_enqueue() instead of regular
> event_enqueue(). So a thread making an assumption that the device pair ha=
s
> internal port will not be the right solution for another pair. The infras=
tructure
> added with these patches aims to help application to have multiple worker
> threads, there by extracting maximum performance from every device
> without affecting existing paths/use cases.
>=20
> The eventmode configuration is predefined. All packets reaching one eth
> port will hit one event queue. All event queues will be mapped to all eve=
nt
> ports. So all cores will be able to receive traffic from all ports.
> When schedule_type is set as RTE_SCHED_TYPE_ORDERED/ATOMIC, event
> device will ensure the ordering. Ordering would be lost when tried in
> PARALLEL.
>=20
> Following command line options are introduced,
>=20
> --transfer-mode: to choose between poll mode & event mode
> --schedule-type: to specify the scheduling type
>                  (RTE_SCHED_TYPE_ORDERED/
>                   RTE_SCHED_TYPE_ATOMIC/
>                   RTE_SCHED_TYPE_PARALLEL)
>=20
> Additionally the event mode introduces two modes of processing packets:
>=20
> Driver-mode: This mode will have bare minimum changes in the application
>              to support ipsec. There woudn't be any lookup etc done in
>              the application. And for inline-protocol use case, the
>              thread would resemble l2fwd as the ipsec processing would be
>              done entirely in the h/w. This mode can be used to benchmark
>              the raw performance of the h/w. All the application side
>              steps (like lookup) can be redone based on the requirement
>              of the end user. Hence the need for a mode which would
>              report the raw performance.
>=20
> App-mode: This mode will have all the features currently implemented with
>           ipsec-secgw (non librte_ipsec mode). All the lookups etc
>           would follow the existing methods and would report numbers
>           that can be compared against regular ipsec-secgw benchmark
>           numbers.
>=20
> The driver mode is selected with existing --single-sa option (used also b=
y poll
> mode). When --single-sa option is used in conjution with event mode then
> index passed to --single-sa is ignored.
>=20
> Example commands to execute ipsec-secgw in various modes on OCTEON
> TX2 platform,
>=20
> #Inbound and outbound app mode
> ipsec-secgw -w 0002:02:00.0,ipsec_in_max_spi=3D128 -w
> 0002:03:00.0,ipsec_in_max_spi=3D128 -w 0002:0e:00.0 -w 0002:10:00.1 --log=
-
> level=3D8 -c 0x1 -- -P -p 0x3 -u 0x1 --config "(1,0,0),(0,0,0)" -f aes-gc=
m.cfg --
> transfer-mode event --schedule-type parallel
>=20
> #Inbound and outbound driver mode
> ipsec-secgw -w 0002:02:00.0,ipsec_in_max_spi=3D128 -w
> 0002:03:00.0,ipsec_in_max_spi=3D128 -w 0002:0e:00.0 -w 0002:10:00.1 --log=
-
> level=3D8 -c 0x1 -- -P -p 0x3 -u 0x1 --config "(1,0,0),(0,0,0)" -f aes-gc=
m.cfg --
> transfer-mode event --schedule-type parallel --single-sa 0
>=20
> This series adds non burst tx internal port workers only. It provides
> infrastructure for non internal port workers, however does not define any=
.
> Also, only inline ipsec protocol mode is supported by the worker threads
> added.
>=20
> Following are planned features,
> 1. Add burst mode workers.
> 2. Add non internal port workers.
> 3. Verify support for Rx core (the support is added but lack of h/w to ve=
rify).
> 4. Add lookaside protocol support.
>=20
> Following are features that Marvell won't be attempting.
> 1. Inline crypto support.
> 2. Lookaside crypto support.
>=20
> For the features that Marvell won't be attempting, new workers can be
> introduced by the respective stake holders.
>=20
> This series is tested on Marvell OCTEON TX2.
>=20
> Changes in v2:
> * Remove --process-dir option. Instead use existing unprotected port mask
>   option (-u) to decide wheter port handles inbound or outbound traffic.
> * Remove --process-mode option. Instead use existing --single-sa option
>   to select between app and driver modes.
> * Add handling of PKT_RX_SEC_OFFLOAD_FAIL result in app worker thread.
> * Fix passing of req_rx_offload flags to create_default_ipsec_flow().
> * Move destruction of flows to a location where eth ports are stopped
>   and closed.
> * Print error and exit when event mode --schedule-type option is used
>   in poll mode.
> * Reduce number of goto statements replacing them with loop constructs.
> * Remove sec_session_fixed table and replace it with locally build
>   table in driver worker thread. Table is indexed by port identifier
>   and holds first inline session pointer found for a given port.
> * Print error and exit when sessions other than inline are configured
>   in event mode.
> * When number of event queues is less than number of eth ports then
>   map all eth ports to one event queue.
> * Cleanup and minor improvements in code as suggested by Konstantin
>=20
> Deferred to v3:
> * The final patch updates the hardcoded number of buffers in a pool.
>   Also, there was a discussion on the update of number of qp. Both the
>   above can be handled properly, if we can remove the logic which limits
>   one core to only use one crypto qp. If we can allow one qp per
>   lcore_param, every eth queue can have it's own crypto qp and that would
>   solve the requirements with OCTEON TX2 inline ipsec support as well.
>=20
>   Patch with the mentioned change,
>   https://urldefense.proofpoint.com/v2/url?u=3Dhttp-
> 3A__patches.dpdk.org_patch_64408_&d=3DDwIDAg&c=3DnKjWec2b6R0mOyPaz
> 7xtfQ&r=3DBPcGOOudUMrTDQ9YbgKcOkO5ChYiUPPlPNIEvTOhjNE&m=3Drg71UQ
> 1CwRYPFy30QuJQZd1Lam_kwYg15N2h5GN2iD4&s=3DyHzfRBRuunl4JWV97vufk
> 7aycUc472ahPVnQ9Tt6SeY&e=3D
>=20
> * Update ipsec-secgw documentation to describe the new options as well as
>   event mode support.
>=20
> This series depends on the PMD changes submitted in the following set,
> https://urldefense.proofpoint.com/v2/url?u=3Dhttp-
> 3A__patches.dpdk.org_project_dpdk_list_-3Fseries-
> 3D8203&d=3DDwIDAg&c=3DnKjWec2b6R0mOyPaz7xtfQ&r=3DBPcGOOudUMrTDQ9Y
> bgKcOkO5ChYiUPPlPNIEvTOhjNE&m=3Drg71UQ1CwRYPFy30QuJQZd1Lam_kwY
> g15N2h5GN2iD4&s=3Dg2wtO9tOQTYHa9os1ECz5uwgpz9JmjTlGbEl-
> Cp6WAw&e=3D
>=20
> Ankur Dwivedi (1):
>   examples/ipsec-secgw: add default rte_flow for inline Rx
>=20
> Anoob Joseph (5):
>   examples/ipsec-secgw: add framework for eventmode helper
>   examples/ipsec-secgw: add eventdev port-lcore link
>   examples/ipsec-secgw: add Rx adapter support
>   examples/ipsec-secgw: add Tx adapter support
>   examples/ipsec-secgw: add routines to display config
>=20
> Lukasz Bartosik (6):
>   examples/ipsec-secgw: add routines to launch workers
>   examples/ipsec-secgw: add support for internal ports
>   examples/ipsec-secgw: add eventmode to ipsec-secgw
>   examples/ipsec-secgw: add driver mode worker
>   examples/ipsec-secgw: add app mode worker
>   examples/ipsec-secgw: add cmd line option for bufs
>=20
>  examples/ipsec-secgw/Makefile       |    2 +
>  examples/ipsec-secgw/event_helper.c | 1714
> +++++++++++++++++++++++++++++++++++
>  examples/ipsec-secgw/event_helper.h |  312 +++++++  examples/ipsec-
> secgw/ipsec-secgw.c  |  502 ++++++++--
>  examples/ipsec-secgw/ipsec-secgw.h  |   86 ++
>  examples/ipsec-secgw/ipsec.c        |    7 +
>  examples/ipsec-secgw/ipsec.h        |   36 +-
>  examples/ipsec-secgw/ipsec_worker.c |  656 ++++++++++++++
>  examples/ipsec-secgw/ipsec_worker.h |   39 +
>  examples/ipsec-secgw/meson.build    |    4 +-
>  examples/ipsec-secgw/sa.c           |   11 -
>  11 files changed, 3275 insertions(+), 94 deletions(-)  create mode 10064=
4
> examples/ipsec-secgw/event_helper.c
>  create mode 100644 examples/ipsec-secgw/event_helper.h
>  create mode 100644 examples/ipsec-secgw/ipsec-secgw.h
>  create mode 100644 examples/ipsec-secgw/ipsec_worker.c
>  create mode 100644 examples/ipsec-secgw/ipsec_worker.h
>=20
> --
> 2.7.4