From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6A2BDA0571; Tue, 3 Mar 2020 19:00:53 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 8F6891C02A; Tue, 3 Mar 2020 19:00:23 +0100 (CET) Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by dpdk.org (Postfix) with ESMTP id 562F81C00F for ; Tue, 3 Mar 2020 19:00:21 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga105.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 03 Mar 2020 10:00:20 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.70,511,1574150400"; d="scan'208";a="274308124" Received: from fmsmsx108.amr.corp.intel.com ([10.18.124.206]) by fmsmga002.fm.intel.com with ESMTP; 03 Mar 2020 10:00:19 -0800 Received: from fmsmsx151.amr.corp.intel.com (10.18.125.4) by FMSMSX108.amr.corp.intel.com (10.18.124.206) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 3 Mar 2020 10:00:19 -0800 Received: from FMSEDG002.ED.cps.intel.com (10.1.192.134) by FMSMSX151.amr.corp.intel.com (10.18.125.4) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 3 Mar 2020 10:00:19 -0800 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (104.47.58.169) by edgegateway.intel.com (192.55.55.69) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 3 Mar 2020 10:00:19 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=g9S8VGawQ+KyIplUHBciZ1skY30H5LbKJeM3sTy3MZbRT34tSNjBPJwwWiB+tS9M3/bxXDJtWfIWsvlt5GUCFrHz6mID7OrHtDGdH9u9XzDMw8lwzesSd6nXJX9e5ry2d63Fvzi6Lzrc8rFfR+66JcL/6Q0V+0asLWShbPSoOVgxfk3TJ4iE42QRElnLekr24k1InhLe6ittf3AYghEA5trzA6q8wWfG4LUvF131lmmdLlO0kN74VUNR5Hnemg6LY9MXeYdCc9Ih3x4iHOUyeIcngJd8m+LyjY1kqEE0WT5vpw76AfMmhWnIVuO9ryRGrct7J34ivBlLWD0IKrrj0A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=QUMuPQqjiC+1RR/ZSpdQUNufWwdszqmrelc1a+uFk8k=; b=eS5tL/vNfMUvoyFMpTDxwOQ+G9r7M/23uwt8X8sGBkXutSYXNQ4vw15Sx4f5zZqjXjnMloLehSWpFrxPQRNThehV1ZuScmoxafpc+irlvMc20o2O+kn1Xn3ooqSPoB1yXqMxyLjIKZ6AJ9l3k6fy8Op9fBaChbWsxVUO4A41vEPvkshqCQ0Fyxe4VV3hUPyuKNtBmMJY26ODdvB4qvGOwUqDJ4zmB9bC2XVrHkb9oNS0Mo9AvJ5pCxVGndvjQSzeN/O94yXBu2znmk+eCpQf0WBheHjnS0acaZir5GbBDfEoq8gG4BmV/NlIxdOYaoxA6SW/SXEPNGt19ACScsJ8kQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=QUMuPQqjiC+1RR/ZSpdQUNufWwdszqmrelc1a+uFk8k=; b=Hk5D0JLIwhJaT5NJYDuleQ51vgyhwiIfYHl/vNFzuyipXgF6hQu97H0e0xRWh46lEkcUXBZKwNlGGeFSwrz4trHV7r05oGaQSiks43DgLX82ndGDD3QJAPGmKW6b2pftzJT00nSDgXXFrI5Z5XutrRgYZfn8WYO0GWsszhXlM3A= Received: from SN6PR11MB2558.namprd11.prod.outlook.com (2603:10b6:805:5d::19) by SN6PR11MB3488.namprd11.prod.outlook.com (2603:10b6:805:b8::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2772.14; Tue, 3 Mar 2020 18:00:17 +0000 Received: from SN6PR11MB2558.namprd11.prod.outlook.com ([fe80::395e:eb75:6ab7:2ba5]) by SN6PR11MB2558.namprd11.prod.outlook.com ([fe80::395e:eb75:6ab7:2ba5%3]) with mapi id 15.20.2772.019; Tue, 3 Mar 2020 18:00:17 +0000 From: "Ananyev, Konstantin" To: Lukasz Bartosik , Akhil Goyal , "Nicolau, Radu" , "Thomas Monjalon" CC: Jerin Jacob , Narayana Prasad , Ankur Dwivedi , Anoob Joseph , Archana Muniganti , Tejasree Kondoj , Vamsi Attunuru , "dev@dpdk.org" Thread-Topic: [PATCH v5 00/15] add eventmode to ipsec-secgw Thread-Index: AQHV7YmuehPaKXxw20up80y8/M6ikKg3L1iA Date: Tue, 3 Mar 2020 18:00:16 +0000 Message-ID: References: <1582185727-6749-1-git-send-email-lbartosik@marvell.com> <1582820317-7333-1-git-send-email-lbartosik@marvell.com> In-Reply-To: <1582820317-7333-1-git-send-email-lbartosik@marvell.com> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiNWNjNWQ0YWUtOTViNS00MTRkLTgwMjYtNmRlMTM3NzgxZWVlIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjEwLjE4MDQuNDkiLCJUcnVzdGVkTGFiZWxIYXNoIjoic0dIazZCZDRwR3prOFl6YjdnZFdZVGdYb1Rhd2hnYUlSS2grT013eUtrOUFWVEhsODY2Zit4aXlVQ1VGSWtBVSJ9 dlp-product: dlpe-windows dlp-reaction: no-action dlp-version: 11.2.0.6 x-ctpclassification: CTP_NT authentication-results: spf=none (sender IP is ) smtp.mailfrom=konstantin.ananyev@intel.com; x-originating-ip: [192.198.151.164] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 134f528c-de49-4593-82aa-08d7bf9cbb2a x-ms-traffictypediagnostic: SN6PR11MB3488: x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:2582; x-forefront-prvs: 03319F6FEF x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(39860400002)(136003)(346002)(366004)(396003)(376002)(189003)(199004)(26005)(186003)(8936002)(33656002)(30864003)(316002)(52536014)(54906003)(2906002)(478600001)(7696005)(110136005)(81166006)(81156014)(7416002)(71200400001)(55016002)(9686003)(66556008)(64756008)(5660300002)(6506007)(66946007)(4326008)(86362001)(8676002)(66476007)(76116006)(66446008); DIR:OUT; SFP:1102; SCL:1; SRVR:SN6PR11MB3488; H:SN6PR11MB2558.namprd11.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: D9s1UW4an4WByBUxdrDmuip0icwFA6wQ+B5V3VXK0Xk6VWxN5Y6QSDL48PDG2EiP4EODFi40E9uTDk2yi7d4tBnJ/fUYzKEnM8pmE8m9Ql6EB/fBV68fipusQ/xc0H1Tt0mIoXi1FMjxnQZ+T5y17IgZuEBhmEHy/q72UERBcaPTOUVzQqgggJfIkHQCfVH6yxDG9tWR3WpQla/NgEpboPLJ4Rf7Nkdcuml10Nwci+/51jqfboEO8FSDP8bzDxPzPakQ1fXL2GU+LdM/CPxmrPD3vMekKIPYaoeJPP8G5Wn8CIeSwSBt7BUHAWMEL1tfNOj/6KY/jh+c2Gcls2CAycf7Rkkk0UwFKtuYhZoHi7br4u6a70vdIYIAiUftmMJJT4TqpyklE01BC2/lyz3aOCI0UhWKVfNVQgcAeUM7UH1Oa68GVAtjH9venCM58sj2 x-ms-exchange-antispam-messagedata: mpqpZkVA/2SuTBWJZX57s3UahA+fnbMY+vUMzImGbZNfbxPIh6ZSPbYESz1UaSzWOdojMONjjqjonkAM1pl4w0lFMXDR+P5iWza5DNKPwJRsfhVLkUYvxoZqQRZFXmt2+At2hyQ+JVLia9eKx+A9nQ== Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: 134f528c-de49-4593-82aa-08d7bf9cbb2a X-MS-Exchange-CrossTenant-originalarrivaltime: 03 Mar 2020 18:00:17.0204 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: dQefor7tQ0VFdb6Gu5b+QS+DTqI2hqqajPAS4nTQX74k8DXtQkf5GQCgh2MX2RMOUNzbTMIw5xk5NLw83e0ix4QLQvohmy9k8B//LXgHv0I= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR11MB3488 X-OriginatorOrg: intel.com Subject: Re: [dpdk-dev] [PATCH v5 00/15] add eventmode to ipsec-secgw X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" >=20 > This series introduces event-mode additions to ipsec-secgw. >=20 > With this series, ipsec-secgw would be able to run in eventmode. The > worker thread (executing loop) would be receiving events and would be > submitting it back to the eventdev after the processing. This way, > multicore scaling and h/w assisted scheduling is achieved by making use > of the eventdev capabilities. >=20 > Since the underlying event device would be having varying capabilities, > the worker thread could be drafted differently to maximize performance. > This series introduces usage of multiple worker threads, among which the > one to be used will be determined by the operating conditions and the > underlying device capabilities. >=20 > For example, if an event device - eth device pair has Tx internal port, > then application can do tx_adapter_enqueue() instead of regular > event_enqueue(). So a thread making an assumption that the device pair > has internal port will not be the right solution for another pair. The > infrastructure added with these patches aims to help application to have > multiple worker threads, there by extracting maximum performance from > every device without affecting existing paths/use cases. >=20 > The eventmode configuration is predefined. All packets reaching one eth > port will hit one event queue. All event queues will be mapped to all > event ports. So all cores will be able to receive traffic from all ports. > When schedule_type is set as RTE_SCHED_TYPE_ORDERED/ATOMIC, event device > will ensure the ordering. Ordering would be lost when tried in PARALLEL. >=20 > Following command line options are introduced, >=20 > --transfer-mode: to choose between poll mode & event mode > --event-schedule-type: to specify the scheduling type > (RTE_SCHED_TYPE_ORDERED/ > RTE_SCHED_TYPE_ATOMIC/ > RTE_SCHED_TYPE_PARALLEL) >=20 > Additionally the event mode introduces two modes of processing packets: >=20 > Driver-mode: This mode will have bare minimum changes in the application > to support ipsec. There woudn't be any lookup etc done in > the application. And for inline-protocol use case, the > thread would resemble l2fwd as the ipsec processing would be > done entirely in the h/w. This mode can be used to benchmark > the raw performance of the h/w. All the application side > steps (like lookup) can be redone based on the requirement > of the end user. Hence the need for a mode which would > report the raw performance. >=20 > App-mode: This mode will have all the features currently implemented with > ipsec-secgw (non librte_ipsec mode). All the lookups etc > would follow the existing methods and would report numbers > that can be compared against regular ipsec-secgw benchmark > numbers. >=20 > The driver mode is selected with existing --single-sa option > (used also by poll mode). When --single-sa option is used > in conjution with event mode then index passed to --single-sa > is ignored. >=20 > Example commands to execute ipsec-secgw in various modes on OCTEON TX2 pl= atform, >=20 > #Inbound and outbound app mode > ipsec-secgw -w 0002:02:00.0,ipsec_in_max_spi=3D128 -w 0002:03:00.0,ipsec_= in_max_spi=3D128 -w 0002:0e:00.0 -w 0002:10:00.1 --log-level=3D8 > -c 0x1 -- -P -p 0x3 -u 0x1 -f aes-gcm.cfg --transfer-mode event --event-s= chedule-type parallel >=20 > #Inbound and outbound driver mode > ipsec-secgw -w 0002:02:00.0,ipsec_in_max_spi=3D128 -w 0002:03:00.0,ipsec_= in_max_spi=3D128 -w 0002:0e:00.0 -w 0002:10:00.1 --log-level=3D8 > -c 0x1 -- -P -p 0x3 -u 0x1 -f aes-gcm.cfg --transfer-mode event --event-s= chedule-type parallel --single-sa 0 >=20 > This series adds non burst tx internal port workers only. It provides inf= rastructure > for non internal port workers, however does not define any. Also, only in= line ipsec > protocol mode is supported by the worker threads added. >=20 > Following are planned features, > 1. Add burst mode workers. > 2. Add non internal port workers. > 3. Verify support for Rx core (the support is added but lack of h/w to ve= rify). > 4. Add lookaside protocol support. >=20 > Following are features that Marvell won't be attempting. > 1. Inline crypto support. > 2. Lookaside crypto support. >=20 > For the features that Marvell won't be attempting, new workers can be > introduced by the respective stake holders. >=20 > This series is tested on Marvell OCTEON TX2. > This series is targeted for 20.05 release. >=20 > Changes in v5: > * Rename function check_params() to check_poll_mode_params() and > check_eh_conf() to check_event_mode_params() in order to make it clear > what is their purpose. > * Forbid usage of --config option in event mode. > * Replace magic numbers on return with enum values in process_ipsec_ev_in= bound() > and process_ipsec_ev_outbound() functions. > * Add session_priv_pool for both inbound and outbound configuration in > ipsec_wrkr_non_burst_int_port_app_mode worker. > * Add check of event type in ipsec_wrkr_non_burst_int_port_app_mode worke= r. > * Update description of --config option in both ipsec-secgw help and docu= mentation. >=20 > Changes in v4: > * Update ipsec-secgw documentation to describe the new options as well as > event mode support. > * In event mode reserve number of crypto queues equal to number of eth po= rts > in order to meet inline protocol offload requirements. > * Add calculate_nb_mbufs() function to calculate number of mbufs in a poo= l > and include fragments table size into the calculation. > * Move structures ipsec_xf and ipsec_sad to ipsec.h and remove static key= word > from sa_out, nb_sa_out, sa_in and nb_sa_in in sa.c. > * Update process_ipsec_ev_inbound(), process_ipsec_ev_outbound(), check_s= p() > and prepare_out_sessions_tbl() functions as a result of changes introdu= ced > by SAD feature. > * Remove setting sa->cdev_id_qp to 0 in create_inline_session as sa_ctx > is created with rte_zmalloc. > * Minor cleanup enhancements: > - In eh_set_default_conf_eventdev() function in event_helper.c put defini= tion > of int local vars in one line, remove invalid comment, put > "eventdev_config->ev_queue_mode =3D RTE_EVENT_QUEUE_CFG_ALL_TYPES" in o= ne line > instead of two. > - Remove extern "C" from event_helper.h. > - Put local vars in reverse xmas tree order in eh_dev_has_rx_internal_por= t() and > eh_dev_has_tx_internal_port() functions in event_helper.c. > - Put #include in alphabetical order in ipsec-secgw.c. > - Move "extern volatile bool force_quit" and "#include " to ip= sec-secgw.h, > remove #include . > - Remove not needed includes in ipsec_worker.c. > - Remove expired todo from ipsec_worker.h. >=20 > Changes in v3: > * Move eh_conf_init() and eh_conf_uninit() functions to event_helper.c > including minor rework. > * Rename --schedule-type option to --event-schedule-type. > * Replace macro UNPROTECTED_PORT with static inline function > is_unprotected_port(). > * Move definitions of global variables used by multiple modules > to .c files and add externs in .h headers. > * Add eh_check_conf() which validates ipsec-secgw configuration > for event mode. > * Add dynamic calculation of number of buffers in a pool based > on number of cores, ports and crypto queues. > * Fix segmentation fault in event mode driver worker which happens > when there are no inline outbound sessions configured. > * Remove change related to updating number of crypto queues > in cryptodevs_init(). The update of crypto queues will be handled > in a separate patch. > * Fix compilation error on 32-bit platforms by using userdata instead > of udata64 from rte_mbuf. >=20 > Changes in v2: > * Remove --process-dir option. Instead use existing unprotected port mask > option (-u) to decide wheter port handles inbound or outbound traffic. > * Remove --process-mode option. Instead use existing --single-sa option > to select between app and driver modes. > * Add handling of PKT_RX_SEC_OFFLOAD_FAIL result in app worker thread. > * Fix passing of req_rx_offload flags to create_default_ipsec_flow(). > * Move destruction of flows to a location where eth ports are stopped > and closed. > * Print error and exit when event mode --schedule-type option is used > in poll mode. > * Reduce number of goto statements replacing them with loop constructs. > * Remove sec_session_fixed table and replace it with locally build > table in driver worker thread. Table is indexed by port identifier > and holds first inline session pointer found for a given port. > * Print error and exit when sessions other than inline are configured > in event mode. > * When number of event queues is less than number of eth ports then > map all eth ports to one event queue. > * Cleanup and minor improvements in code as suggested by Konstantin >=20 > Ankur Dwivedi (1): > examples/ipsec-secgw: add default rte flow for inline Rx >=20 > Anoob Joseph (5): > examples/ipsec-secgw: add framework for eventmode helper > examples/ipsec-secgw: add eventdev port-lcore link > examples/ipsec-secgw: add Rx adapter support > examples/ipsec-secgw: add Tx adapter support > examples/ipsec-secgw: add routines to display config >=20 > Lukasz Bartosik (9): > examples/ipsec-secgw: add routines to launch workers > examples/ipsec-secgw: add support for internal ports > examples/ipsec-secgw: add event helper config init/uninit > examples/ipsec-secgw: add eventmode to ipsec-secgw > examples/ipsec-secgw: add driver mode worker > examples/ipsec-secgw: add app mode worker > examples/ipsec-secgw: make number of buffers dynamic > doc: add event mode support to ipsec-secgw > examples/ipsec-secgw: reserve crypto queues in event mode >=20 > doc/guides/sample_app_ug/ipsec_secgw.rst | 135 ++- > examples/ipsec-secgw/Makefile | 2 + > examples/ipsec-secgw/event_helper.c | 1812 ++++++++++++++++++++++++= ++++++ > examples/ipsec-secgw/event_helper.h | 327 ++++++ > examples/ipsec-secgw/ipsec-secgw.c | 506 +++++++-- > examples/ipsec-secgw/ipsec-secgw.h | 88 ++ > examples/ipsec-secgw/ipsec.c | 5 +- > examples/ipsec-secgw/ipsec.h | 53 +- > examples/ipsec-secgw/ipsec_worker.c | 649 +++++++++++ > examples/ipsec-secgw/ipsec_worker.h | 41 + > examples/ipsec-secgw/meson.build | 6 +- > examples/ipsec-secgw/sa.c | 21 +- > examples/ipsec-secgw/sad.h | 5 - > 13 files changed, 3516 insertions(+), 134 deletions(-) > create mode 100644 examples/ipsec-secgw/event_helper.c > create mode 100644 examples/ipsec-secgw/event_helper.h > create mode 100644 examples/ipsec-secgw/ipsec-secgw.h > create mode 100644 examples/ipsec-secgw/ipsec_worker.c > create mode 100644 examples/ipsec-secgw/ipsec_worker.h >=20 > -- Have to say I didn't look extensively on event mode. My primary concern was poll-mode and common code changes. >From that perspective - LGTM. Series Acked-by: Konstantin Ananyev > 2.7.4