From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7B029A04B3; Tue, 28 Jan 2020 14:01:10 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 21F691C2A0; Tue, 28 Jan 2020 14:01:09 +0100 (CET) Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by dpdk.org (Postfix) with ESMTP id 2C1A91C2A0 for ; Tue, 28 Jan 2020 14:01:06 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga107.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 28 Jan 2020 05:01:06 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.70,373,1574150400"; d="scan'208";a="401651326" Received: from orsmsx102.amr.corp.intel.com ([10.22.225.129]) by orsmga005.jf.intel.com with ESMTP; 28 Jan 2020 05:01:05 -0800 Received: from orsmsx609.amr.corp.intel.com (10.22.229.22) by ORSMSX102.amr.corp.intel.com (10.22.225.129) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 28 Jan 2020 05:01:05 -0800 Received: from orsmsx609.amr.corp.intel.com (10.22.229.22) by ORSMSX609.amr.corp.intel.com (10.22.229.22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Tue, 28 Jan 2020 05:01:05 -0800 Received: from ORSEDG002.ED.cps.intel.com (10.7.248.5) by orsmsx609.amr.corp.intel.com (10.22.229.22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.1713.5 via Frontend Transport; Tue, 28 Jan 2020 05:01:05 -0800 Received: from NAM02-BL2-obe.outbound.protection.outlook.com (104.47.38.56) by edgegateway.intel.com (134.134.137.101) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 28 Jan 2020 05:01:03 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=KmXFZbncLV3xX0agRIndpJzDrkEdHrKjf2WTHi5DsGwRxdJ7PQet6mAyn58dXRb8ReVD5BjB+tfb+9Wo2VvnSmpXIQznv6NVaP6IwXupYrMxz1TmRehOskk31/rIpo8JvHK57mNp3Q/g8VcbGzk+UAdinHX46VK4lV+4dTQdHGI6GC4FOaQIa5QMCwCNFQ54wiNNj8ckChWP1sX0zPQ17calXrZCitFLe8bBFQGmLbtdLY+Dm7f7a/LDK6kUqqElJvd261AHMY45jU5Hk6AWLFm142vVfpdD2GxeJYGLrNj5jGAK4Rza1JrCMRqDgrNADNi5H427rDlTLY5OPixG9w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4JdOxDejzcMaWc4M+gsAOyoegVbsErFRsMACBDzfPTk=; b=ocgBX6QJdsk8EEsGQ6pxlR/R61gpU5GIN0S7VWKGUB+vJfKj+WbEjjdsLHSaGbMgSrJL881KH8n3Aim6R6RVa5kq5/8PX+zWuPZP52pDSew3N7BGYaY4MX6hL9y1V+QHvVKJgGFkNANj7H+OOhRqIJPsaLuqcQ6HqCocrrqtJApMPUMtv8HHsmR0ND48t9P9s6EidsqTVeBVEDvGtuRrOq7XiGp+Xtb8pN4c7KqtIkYuXHzQyRH++gDM7Fl39REmyno4HP/4BacnfYFFCDPGuHVJqWUfyW84yjV4XkACHSLy3ffA9qJNu9THsVbjp0vGXyTNt05IE/pEsaE6WmjlYw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4JdOxDejzcMaWc4M+gsAOyoegVbsErFRsMACBDzfPTk=; b=gZZPLBLsgnGo7SHMRnYHRkBrW8jhKN+jnSYGZMiaEuB6HxKhZCjqezi+fvx5MWu8nc0t7QZaGUxb3AtFGcqLcbtKov7g+CqQmUc+wtF8TfhvYKGWPNrzxdsdEUKcCnYaRmUSJsN4MZbQqnrX3xxKNgxZMqcw7kvlXEFFTqoUszM= Received: from SN6PR11MB2558.namprd11.prod.outlook.com (52.135.94.19) by SN6PR11MB2559.namprd11.prod.outlook.com (52.135.91.15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2665.23; Tue, 28 Jan 2020 13:00:58 +0000 Received: from SN6PR11MB2558.namprd11.prod.outlook.com ([fe80::4d86:362a:13c3:8386]) by SN6PR11MB2558.namprd11.prod.outlook.com ([fe80::4d86:362a:13c3:8386%7]) with mapi id 15.20.2665.026; Tue, 28 Jan 2020 13:00:58 +0000 From: "Ananyev, Konstantin" To: Anoob Joseph , Akhil Goyal CC: Jerin Jacob Kollanukkaran , "Narayana Prasad Raju Athreya" , Ankur Dwivedi , Archana Muniganti , Tejasree Kondoj , Vamsi Krishna Attunuru , Lukas Bartosik , "dev@dpdk.org" , Thomas Monjalon , "Nicolau, Radu" Thread-Topic: [dpdk-dev] [PATCH v2 00/12] add eventmode to ipsec-secgw Thread-Index: AQHVz5f/1aEFuAiAFUah0Jcw4LFomaf/kRYAgABnj+A= Date: Tue, 28 Jan 2020 13:00:57 +0000 Message-ID: References: <1575808249-31135-1-git-send-email-anoobj@marvell.com> <1579527918-360-1-git-send-email-anoobj@marvell.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiNmQ0NTQ2MzctMDBmZS00YTViLWFhMzUtYjhlNmQyYzM2OGIzIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjEwLjE4MDQuNDkiLCJUcnVzdGVkTGFiZWxIYXNoIjoiOHVXQStQUTNxQkNuQlVvTlJVNDk0bXE4aSs4RmoxbnhnK2RVODZvYXNmTDVZd25idTlKVVwvUElrMmNlVlwvTmRuIn0= dlp-product: dlpe-windows dlp-reaction: no-action dlp-version: 11.2.0.6 x-ctpclassification: CTP_NT authentication-results: spf=none (sender IP is ) smtp.mailfrom=konstantin.ananyev@intel.com; x-originating-ip: [192.198.151.187] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 1b1db30f-0406-4f14-8768-08d7a3f21e43 x-ms-traffictypediagnostic: SN6PR11MB2559: x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:6790; x-forefront-prvs: 029651C7A1 x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(136003)(346002)(376002)(366004)(396003)(39860400002)(199004)(189003)(186003)(5660300002)(55016002)(9686003)(7696005)(2906002)(19627235002)(76116006)(7416002)(8936002)(66556008)(66946007)(66476007)(64756008)(66446008)(52536014)(6506007)(478600001)(81166006)(71200400001)(8676002)(53546011)(26005)(966005)(81156014)(4326008)(33656002)(54906003)(316002)(86362001)(110136005)(107886003); DIR:OUT; SFP:1102; SCL:1; SRVR:SN6PR11MB2559; H:SN6PR11MB2558.namprd11.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: tXUR2OBkxOKhldTeGTBie6KJV3TlxcI42+u/q5fNlvIeDdOLDHN5Tjl6ezM/xrr3Weam1jTyK5DV1bz05pyRqzxPrBPcYRAyPLzqjqUTANvmSd17Dnggod4MccEUd0sFRGF/Rlbqza+kaKH3uI74yPE0ZvFNzX0vmKFn0CrYPOATmQJudAce6fELPHX7Y/Tjj5xX+oHSxKmiRRfVMJ3eJ8BEQdnRhTUbyJcnA1fOmrkJ9HSuxLmQ6zInz3hsGvbB0wpi1dtdHKoYHauV1eGJbRhn28NuhNXgYN63sTJcCKTJISpWbwFzecKLNot3pjv/HIAfStA0g2cRLa1Nv+TpIoBsvwXkua0To0DvtoTEoW+WtEj8L37KWN0h4mpdJZmepa7JDn9yY4gHdGjRZzYSfWspPdXP4l04VnFxTi7OoDHHbp7wZfcWCqvBb2N+83K2agadh6Gl3TOCoVvixv5RjpEskVGM6GLM084d0ZjVPk4JtzCx8Ph1bJOHbQJ5uwzOioCbZFPcuHZBvJTVDzABog== x-ms-exchange-antispam-messagedata: S31g1cmsoBsEQgGwKVHjS+GYC78pMSGOde+i27xyvdvM6d41zKV5wT8hOJHfijojLgjFtEoiRDNhsno4GXGiOl83OVhh8jhYr38m+2NqlNCAd7yTJ3p638mnnKqNxnifJ6hfjPJlfQyfzutLux8p7g== Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: 1b1db30f-0406-4f14-8768-08d7a3f21e43 X-MS-Exchange-CrossTenant-originalarrivaltime: 28 Jan 2020 13:00:57.9420 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: c3DETMxmaZ5Cm1ZIHfSEBkmhncmlt448htw8Q6gI6cNtgVvUCfDbI6QvwolKVKxPTmlwOR0RI3U4yb5P1HZT93QXJJ5GjKgN8QaBNkopNwc= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR11MB2559 X-OriginatorOrg: intel.com Subject: Re: [dpdk-dev] [PATCH v2 00/12] add eventmode to ipsec-secgw X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" >=20 > Hi Akhil, Konstantin, >=20 > Do you have any further comments? Will try to have a proper look today/tomorrow. Sorry for delay. Konstantin >=20 > Thanks, > Anoob >=20 > > -----Original Message----- > > From: dev On Behalf Of Anoob Joseph > > Sent: Monday, January 20, 2020 7:15 PM > > To: Akhil Goyal ; Radu Nicolau > > ; Thomas Monjalon > > Cc: Anoob Joseph ; Jerin Jacob Kollanukkaran > > ; Narayana Prasad Raju Athreya > > ; Ankur Dwivedi ; > > Archana Muniganti ; Tejasree Kondoj > > ; Vamsi Krishna Attunuru > > ; Lukas Bartosik ; > > Konstantin Ananyev ; dev@dpdk.org > > Subject: [dpdk-dev] [PATCH v2 00/12] add eventmode to ipsec-secgw > > > > This series introduces event-mode additions to ipsec-secgw. This effort= is > > parallel to the similar changes in l2fwd (l2fwd-event app) & l3fwd. > > > > With this series, ipsec-secgw would be able to run in eventmode. The wo= rker > > thread (executing loop) would be receiving events and would be submitti= ng > > it back to the eventdev after the processing. This way, multicore scali= ng and > > h/w assisted scheduling is achieved by making use of the eventdev > > capabilities. > > > > Since the underlying event device would be having varying capabilities,= the > > worker thread could be drafted differently to maximize performance. > > This series introduces usage of multiple worker threads, among which th= e > > one to be used will be determined by the operating conditions and the > > underlying device capabilities. > > > > For example, if an event device - eth device pair has Tx internal port,= then > > application can do tx_adapter_enqueue() instead of regular > > event_enqueue(). So a thread making an assumption that the device pair = has > > internal port will not be the right solution for another pair. The infr= astructure > > added with these patches aims to help application to have multiple work= er > > threads, there by extracting maximum performance from every device > > without affecting existing paths/use cases. > > > > The eventmode configuration is predefined. All packets reaching one eth > > port will hit one event queue. All event queues will be mapped to all e= vent > > ports. So all cores will be able to receive traffic from all ports. > > When schedule_type is set as RTE_SCHED_TYPE_ORDERED/ATOMIC, event > > device will ensure the ordering. Ordering would be lost when tried in > > PARALLEL. > > > > Following command line options are introduced, > > > > --transfer-mode: to choose between poll mode & event mode > > --schedule-type: to specify the scheduling type > > (RTE_SCHED_TYPE_ORDERED/ > > RTE_SCHED_TYPE_ATOMIC/ > > RTE_SCHED_TYPE_PARALLEL) > > > > Additionally the event mode introduces two modes of processing packets: > > > > Driver-mode: This mode will have bare minimum changes in the applicatio= n > > to support ipsec. There woudn't be any lookup etc done in > > the application. And for inline-protocol use case, the > > thread would resemble l2fwd as the ipsec processing would = be > > done entirely in the h/w. This mode can be used to benchma= rk > > the raw performance of the h/w. All the application side > > steps (like lookup) can be redone based on the requirement > > of the end user. Hence the need for a mode which would > > report the raw performance. > > > > App-mode: This mode will have all the features currently implemented wi= th > > ipsec-secgw (non librte_ipsec mode). All the lookups etc > > would follow the existing methods and would report numbers > > that can be compared against regular ipsec-secgw benchmark > > numbers. > > > > The driver mode is selected with existing --single-sa option (used also= by poll > > mode). When --single-sa option is used in conjution with event mode the= n > > index passed to --single-sa is ignored. > > > > Example commands to execute ipsec-secgw in various modes on OCTEON > > TX2 platform, > > > > #Inbound and outbound app mode > > ipsec-secgw -w 0002:02:00.0,ipsec_in_max_spi=3D128 -w > > 0002:03:00.0,ipsec_in_max_spi=3D128 -w 0002:0e:00.0 -w 0002:10:00.1 --l= og- > > level=3D8 -c 0x1 -- -P -p 0x3 -u 0x1 --config "(1,0,0),(0,0,0)" -f aes-= gcm.cfg -- > > transfer-mode event --schedule-type parallel > > > > #Inbound and outbound driver mode > > ipsec-secgw -w 0002:02:00.0,ipsec_in_max_spi=3D128 -w > > 0002:03:00.0,ipsec_in_max_spi=3D128 -w 0002:0e:00.0 -w 0002:10:00.1 --l= og- > > level=3D8 -c 0x1 -- -P -p 0x3 -u 0x1 --config "(1,0,0),(0,0,0)" -f aes-= gcm.cfg -- > > transfer-mode event --schedule-type parallel --single-sa 0 > > > > This series adds non burst tx internal port workers only. It provides > > infrastructure for non internal port workers, however does not define a= ny. > > Also, only inline ipsec protocol mode is supported by the worker thread= s > > added. > > > > Following are planned features, > > 1. Add burst mode workers. > > 2. Add non internal port workers. > > 3. Verify support for Rx core (the support is added but lack of h/w to = verify). > > 4. Add lookaside protocol support. > > > > Following are features that Marvell won't be attempting. > > 1. Inline crypto support. > > 2. Lookaside crypto support. > > > > For the features that Marvell won't be attempting, new workers can be > > introduced by the respective stake holders. > > > > This series is tested on Marvell OCTEON TX2. > > > > Changes in v2: > > * Remove --process-dir option. Instead use existing unprotected port ma= sk > > option (-u) to decide wheter port handles inbound or outbound traffic= . > > * Remove --process-mode option. Instead use existing --single-sa option > > to select between app and driver modes. > > * Add handling of PKT_RX_SEC_OFFLOAD_FAIL result in app worker thread. > > * Fix passing of req_rx_offload flags to create_default_ipsec_flow(). > > * Move destruction of flows to a location where eth ports are stopped > > and closed. > > * Print error and exit when event mode --schedule-type option is used > > in poll mode. > > * Reduce number of goto statements replacing them with loop constructs. > > * Remove sec_session_fixed table and replace it with locally build > > table in driver worker thread. Table is indexed by port identifier > > and holds first inline session pointer found for a given port. > > * Print error and exit when sessions other than inline are configured > > in event mode. > > * When number of event queues is less than number of eth ports then > > map all eth ports to one event queue. > > * Cleanup and minor improvements in code as suggested by Konstantin > > > > Deferred to v3: > > * The final patch updates the hardcoded number of buffers in a pool. > > Also, there was a discussion on the update of number of qp. Both the > > above can be handled properly, if we can remove the logic which limit= s > > one core to only use one crypto qp. If we can allow one qp per > > lcore_param, every eth queue can have it's own crypto qp and that wou= ld > > solve the requirements with OCTEON TX2 inline ipsec support as well. > > > > Patch with the mentioned change, > > https://urldefense.proofpoint.com/v2/url?u=3Dhttp- > > 3A__patches.dpdk.org_patch_64408_&d=3DDwIDAg&c=3DnKjWec2b6R0mOyPaz > > 7xtfQ&r=3DBPcGOOudUMrTDQ9YbgKcOkO5ChYiUPPlPNIEvTOhjNE&m=3Drg71UQ > > 1CwRYPFy30QuJQZd1Lam_kwYg15N2h5GN2iD4&s=3DyHzfRBRuunl4JWV97vufk > > 7aycUc472ahPVnQ9Tt6SeY&e=3D > > > > * Update ipsec-secgw documentation to describe the new options as well = as > > event mode support. > > > > This series depends on the PMD changes submitted in the following set, > > https://urldefense.proofpoint.com/v2/url?u=3Dhttp- > > 3A__patches.dpdk.org_project_dpdk_list_-3Fseries- > > 3D8203&d=3DDwIDAg&c=3DnKjWec2b6R0mOyPaz7xtfQ&r=3DBPcGOOudUMrTDQ9Y > > bgKcOkO5ChYiUPPlPNIEvTOhjNE&m=3Drg71UQ1CwRYPFy30QuJQZd1Lam_kwY > > g15N2h5GN2iD4&s=3Dg2wtO9tOQTYHa9os1ECz5uwgpz9JmjTlGbEl- > > Cp6WAw&e=3D > > > > Ankur Dwivedi (1): > > examples/ipsec-secgw: add default rte_flow for inline Rx > > > > Anoob Joseph (5): > > examples/ipsec-secgw: add framework for eventmode helper > > examples/ipsec-secgw: add eventdev port-lcore link > > examples/ipsec-secgw: add Rx adapter support > > examples/ipsec-secgw: add Tx adapter support > > examples/ipsec-secgw: add routines to display config > > > > Lukasz Bartosik (6): > > examples/ipsec-secgw: add routines to launch workers > > examples/ipsec-secgw: add support for internal ports > > examples/ipsec-secgw: add eventmode to ipsec-secgw > > examples/ipsec-secgw: add driver mode worker > > examples/ipsec-secgw: add app mode worker > > examples/ipsec-secgw: add cmd line option for bufs > > > > examples/ipsec-secgw/Makefile | 2 + > > examples/ipsec-secgw/event_helper.c | 1714 > > +++++++++++++++++++++++++++++++++++ > > examples/ipsec-secgw/event_helper.h | 312 +++++++ examples/ipsec- > > secgw/ipsec-secgw.c | 502 ++++++++-- > > examples/ipsec-secgw/ipsec-secgw.h | 86 ++ > > examples/ipsec-secgw/ipsec.c | 7 + > > examples/ipsec-secgw/ipsec.h | 36 +- > > examples/ipsec-secgw/ipsec_worker.c | 656 ++++++++++++++ > > examples/ipsec-secgw/ipsec_worker.h | 39 + > > examples/ipsec-secgw/meson.build | 4 +- > > examples/ipsec-secgw/sa.c | 11 - > > 11 files changed, 3275 insertions(+), 94 deletions(-) create mode 100= 644 > > examples/ipsec-secgw/event_helper.c > > create mode 100644 examples/ipsec-secgw/event_helper.h > > create mode 100644 examples/ipsec-secgw/ipsec-secgw.h > > create mode 100644 examples/ipsec-secgw/ipsec_worker.c > > create mode 100644 examples/ipsec-secgw/ipsec_worker.h > > > > -- > > 2.7.4