From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4B113A0524; Mon, 24 Feb 2020 06:20:11 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 417E31F1C; Mon, 24 Feb 2020 06:20:10 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by dpdk.org (Postfix) with ESMTP id 82F841DBF for ; Mon, 24 Feb 2020 06:20:08 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 01O5JKRZ018666; Sun, 23 Feb 2020 21:20:07 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-transfer-encoding : mime-version; s=pfpt0818; bh=X7n7MC6Fs9yeEAzsrv9hpUJ7+N4VBMFdz9loNhtjQNc=; b=F3ryaiHYLxuj2PgJ48t2cbt8XVBmx1S2Uh2kV5KnUmTcxorh/jMTBDW9sV+E9wrdzm6M nYJ9DuoMEhcVNbujc2b3iVsYTQRzvsVb51FtYpFmeBpN5ScXdq457AYpaMclfEs3ahI2 iKmOhrhmz4dHoRzduz6KrH3U665UqVQDMDw+LjKUM26h1cyBGshLICdlHykKANn34MYU l96RhJD6mG+gscCDnlwrZWaCoqc815F8to7T7ANJiRAJxQDdaiB58UNcB+htswcFFtTw 4lZPLy5CVxOibkansnu8GyV26u41n/KvBM2sMwOic6nVQbcsie0leV4IrwYB+I1Kl81S VA== Received: from sc-exch02.marvell.com ([199.233.58.182]) by mx0b-0016f401.pphosted.com with ESMTP id 2yb4pt4vms-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Sun, 23 Feb 2020 21:20:07 -0800 Received: from SC-EXCH02.marvell.com (10.93.176.82) by SC-EXCH02.marvell.com (10.93.176.82) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sun, 23 Feb 2020 21:20:04 -0800 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (104.47.58.168) by SC-EXCH02.marvell.com (10.93.176.82) with Microsoft SMTP Server (TLS) id 15.0.1497.2 via Frontend Transport; Sun, 23 Feb 2020 21:20:04 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=AuxPCE5Nm9lxkyq1d2yTh1ZmELCE7BiSoO+ZnivwNA1jQx+dqTPVfqqunjc/wfRs9LWU1ZW7+t4W0CZNb5M5S7JG6BwwCdyEVHMTDL9CxmUeEPAIcpRUaQYsrTBSFqCLJWvnY5emPhTQr+yyIIZWNh+8yCZ0MfgpKa1uvebMQBduubrrh7K3I5EUONto6sXKaM9fvhvjcuHxU2KuXSKnBAk/Ewox/qiOns5L4p/Z1KHy+OwY9XgMkFm5R6Z9iczi0IuAQf5n1yO+qzZVYqko+vqnXRz4KV3zMAcR19A2KVR+nwbOZNACrbpsP1kFNyuM+DAaH2O5jphzRseEIduGrQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=X7n7MC6Fs9yeEAzsrv9hpUJ7+N4VBMFdz9loNhtjQNc=; b=Ep4fq9w9dvK6Xr8IJvRk/cMktzLzPCsIPvoepnFmirni2zdTcgki4M82Wlj45CLWbenxPP+YJqSQuDbYd0fWODFTjqtM+PHgYyG8nUzBIM4eTsevWqH2EvSioCgXvwBMa9xAWzWFoH5ZKa6KNYj1M8JtvmDG3up3DzMb0LuCiK3IRtLY671CHXXX/IPdKkpW1TWQHny9wrZxGOZO8HbeZyVjqySzNI1jm8QvhaFSASlvfJoGJKUb4r70VGuDo9PwgyJzZ21caY7Fmk7Ch+pqGyByO5MIZLalGIaxE9U9hZWe07aS7QcujRFFZsRZQqruCH2xQ4XrDvQu6BY4KMO1Wg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=X7n7MC6Fs9yeEAzsrv9hpUJ7+N4VBMFdz9loNhtjQNc=; b=VTJbA9F6X7U2yVh8UZNA3tV/7n7pR1792T5jsx9Q+1jQXXqLTC/FWXj8XTO66aexzJh2Drd5ByY50fXPchXkQd2aNG5bjpc0Z2rk93DW968Agel/iLkzXIkLdw3AmPe+T0vOt7Z/sUXEjhBrvFMLC2+Kg9fayWjI1KFvHKMVCcI= Received: from MN2PR18MB2877.namprd18.prod.outlook.com (2603:10b6:208:3b::26) by MN2PR18MB2560.namprd18.prod.outlook.com (2603:10b6:208:102::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2750.17; Mon, 24 Feb 2020 05:20:03 +0000 Received: from MN2PR18MB2877.namprd18.prod.outlook.com ([fe80::648f:e7fa:f95e:191b]) by MN2PR18MB2877.namprd18.prod.outlook.com ([fe80::648f:e7fa:f95e:191b%2]) with mapi id 15.20.2750.021; Mon, 24 Feb 2020 05:20:03 +0000 From: Anoob Joseph To: Konstantin Ananyev , Akhil Goyal CC: Jerin Jacob Kollanukkaran , "Narayana Prasad Raju Athreya" , Ankur Dwivedi , "Archana Muniganti" , Tejasree Kondoj , Vamsi Krishna Attunuru , "dev@dpdk.org" , Lukas Bartosik , Radu Nicolau , Thomas Monjalon Thread-Topic: [PATCH v4 00/15] add eventmode to ipsec-secgw Thread-Index: AQHV58QWLtKpUPM/WkKXkz4RPRcS06gp0+bQ Date: Mon, 24 Feb 2020 05:20:03 +0000 Message-ID: References: <1580824721-21527-1-git-send-email-lbartosik@marvell.com> <1582185727-6749-1-git-send-email-lbartosik@marvell.com> In-Reply-To: <1582185727-6749-1-git-send-email-lbartosik@marvell.com> Accept-Language: en-IN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [137.97.2.15] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 10782a09-e7d8-4e11-2973-08d7b8e933db x-ms-traffictypediagnostic: MN2PR18MB2560: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:4125; x-forefront-prvs: 032334F434 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(4636009)(346002)(376002)(396003)(136003)(39850400004)(366004)(189003)(199004)(5660300002)(4326008)(71200400001)(81156014)(9686003)(26005)(478600001)(6506007)(53546011)(110136005)(52536014)(8936002)(33656002)(55016002)(81166006)(8676002)(86362001)(66946007)(76116006)(54906003)(2906002)(64756008)(66476007)(316002)(66446008)(186003)(7696005)(66556008)(30864003); DIR:OUT; SFP:1101; SCL:1; SRVR:MN2PR18MB2560; H:MN2PR18MB2877.namprd18.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: marvell.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: En8hv67wMw7tZ/jUUTSzbyH79I+KQeB7QT0WG4oJ2UYPfkCduSZrgGByPryChE7X40jKnYqpg43TB1lPfNgRNEUGk2sG61lDMXWNoky0VDW1Xk4QKO7xp7eBMh+FpcUqOGL/CiIckJUNLXhlUPJhpHcIhyT0R+slQO19wC0m7Dvo8umxr4HH7z7j3wTjKiY34AWzP5Vj2kNxWlbea6M5Qggs9UFdzt7a80PGRTzE65sMtzlFABNNl3at7toC1hLYriDBCno4f0+eZodPxqWvdDtKNpQMYoQzn3GtKIXzgtypGfK4+h09/mereGdJFV+tB11BgCBvjEj+OcxbkZ/Ayu1X+KK8EqcTXNlwnodREvSP2Ewwttw1HCPeD50wXVqi0CNQYb289iLac8kpyQb5nZf59l26WGBg6IclYYrTktOxbefjE/twkEHFWo6GCbpM x-ms-exchange-antispam-messagedata: RfM18SpPkS2U87DGJ6yp4YFekG2+FG81L0BRJU2X7gyT2XbJwswl5mNMcik3b7HNqZmzbofeKSISQdPIUpydcLlxFKrkfuZQ9LBxwLl8Eo1h3k1BM4Wzs2zga3+smoAJnexQArwEHYSGfazTrLsJLw== Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: 10782a09-e7d8-4e11-2973-08d7b8e933db X-MS-Exchange-CrossTenant-originalarrivaltime: 24 Feb 2020 05:20:03.1727 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: Ozo+GI+8c/T+26YwVgnvLuuvHDw2GXNJWpW1K8EeS8hBXMjn8IPGuy7n1Y58uAuPHts6vTWDNVGQyyXbtu40aA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR18MB2560 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138, 18.0.572 definitions=2020-02-23_07:2020-02-21, 2020-02-23 signatures=0 Subject: Re: [dpdk-dev] [PATCH v4 00/15] add eventmode to ipsec-secgw X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Konstantin, Akhil, Do you have any further comments on this series? Regarding the config parameter change which was discussed in the last submi= ssion, we have deferred it and went back to Konstantin's suggestion of upda= ting qp only for our case. Hope now you can Ack the series. Thanks, Anoob > -----Original Message----- > From: Lukasz Bartosik > Sent: Thursday, February 20, 2020 1:32 PM > To: Akhil Goyal ; Radu Nicolau > ; Thomas Monjalon > Cc: Jerin Jacob Kollanukkaran ; Narayana Prasad Raju > Athreya ; Ankur Dwivedi ; > Anoob Joseph ; Archana Muniganti > ; Tejasree Kondoj ; Vamsi > Krishna Attunuru ; Konstantin Ananyev > ; dev@dpdk.org > Subject: [PATCH v4 00/15] add eventmode to ipsec-secgw >=20 > This series introduces event-mode additions to ipsec-secgw. >=20 > With this series, ipsec-secgw would be able to run in eventmode. The work= er > thread (executing loop) would be receiving events and would be submitting= it > back to the eventdev after the processing. This way, multicore scaling an= d h/w > assisted scheduling is achieved by making use of the eventdev capabilitie= s. >=20 > Since the underlying event device would be having varying capabilities, t= he > worker thread could be drafted differently to maximize performance. > This series introduces usage of multiple worker threads, among which the = one to > be used will be determined by the operating conditions and the underlying > device capabilities. >=20 > For example, if an event device - eth device pair has Tx internal port, t= hen > application can do tx_adapter_enqueue() instead of regular event_enqueue(= ). So > a thread making an assumption that the device pair has internal port will= not be > the right solution for another pair. The infrastructure added with these = patches > aims to help application to have multiple worker threads, there by extrac= ting > maximum performance from every device without affecting existing paths/us= e > cases. >=20 > The eventmode configuration is predefined. All packets reaching one eth p= ort > will hit one event queue. All event queues will be mapped to all event po= rts. So > all cores will be able to receive traffic from all ports. > When schedule_type is set as RTE_SCHED_TYPE_ORDERED/ATOMIC, event > device will ensure the ordering. Ordering would be lost when tried in PAR= ALLEL. >=20 > Following command line options are introduced, >=20 > --transfer-mode: to choose between poll mode & event mode > --event-schedule-type: to specify the scheduling type > (RTE_SCHED_TYPE_ORDERED/ > RTE_SCHED_TYPE_ATOMIC/ > RTE_SCHED_TYPE_PARALLEL) >=20 > Additionally the event mode introduces two modes of processing packets: >=20 > Driver-mode: This mode will have bare minimum changes in the application > to support ipsec. There woudn't be any lookup etc done in > the application. And for inline-protocol use case, the > thread would resemble l2fwd as the ipsec processing would be > done entirely in the h/w. This mode can be used to benchmark > the raw performance of the h/w. All the application side > steps (like lookup) can be redone based on the requirement > of the end user. Hence the need for a mode which would > report the raw performance. >=20 > App-mode: This mode will have all the features currently implemented with > ipsec-secgw (non librte_ipsec mode). All the lookups etc > would follow the existing methods and would report numbers > that can be compared against regular ipsec-secgw benchmark > numbers. >=20 > The driver mode is selected with existing --single-sa option (used also b= y poll > mode). When --single-sa option is used in conjution with event mode then = index > passed to --single-sa is ignored. >=20 > Example commands to execute ipsec-secgw in various modes on OCTEON TX2 > platform, >=20 > #Inbound and outbound app mode > ipsec-secgw -w 0002:02:00.0,ipsec_in_max_spi=3D128 -w > 0002:03:00.0,ipsec_in_max_spi=3D128 -w 0002:0e:00.0 -w 0002:10:00.1 --log= - > level=3D8 -c 0x1 -- -P -p 0x3 -u 0x1 --config "(1,0,0),(0,0,0)" -f aes-gc= m.cfg -- > transfer-mode event --event-schedule-type parallel >=20 > #Inbound and outbound driver mode > ipsec-secgw -w 0002:02:00.0,ipsec_in_max_spi=3D128 -w > 0002:03:00.0,ipsec_in_max_spi=3D128 -w 0002:0e:00.0 -w 0002:10:00.1 --log= - > level=3D8 -c 0x1 -- -P -p 0x3 -u 0x1 --config "(1,0,0),(0,0,0)" -f aes-gc= m.cfg -- > transfer-mode event --event-schedule-type parallel --single-sa 0 >=20 > This series adds non burst tx internal port workers only. It provides inf= rastructure > for non internal port workers, however does not define any. Also, only in= line > ipsec protocol mode is supported by the worker threads added. >=20 > Following are planned features, > 1. Add burst mode workers. > 2. Add non internal port workers. > 3. Verify support for Rx core (the support is added but lack of h/w to ve= rify). > 4. Add lookaside protocol support. >=20 > Following are features that Marvell won't be attempting. > 1. Inline crypto support. > 2. Lookaside crypto support. >=20 > For the features that Marvell won't be attempting, new workers can be > introduced by the respective stake holders. >=20 > This series is tested on Marvell OCTEON TX2. > This series is targeted for 20.05 release. >=20 > Changes in v4: > * Update ipsec-secgw documentation to describe the new options as well as > event mode support. > * In event mode reserve number of crypto queues equal to number of eth po= rts > in order to meet inline protocol offload requirements. > * Add calculate_nb_mbufs() function to calculate number of mbufs in a poo= l > and include fragments table size into the calculation. > * Move structures ipsec_xf and ipsec_sad to ipsec.h and remove static key= word > from sa_out, nb_sa_out, sa_in and nb_sa_in in sa.c. > * Update process_ipsec_ev_inbound(), process_ipsec_ev_outbound(), > check_sp() > and prepare_out_sessions_tbl() functions as a result of changes introdu= ced > by SAD feature. > * Remove setting sa->cdev_id_qp to 0 in create_inline_session as sa_ctx > is created with rte_zmalloc. > * Minor cleanup enhancements: > - In eh_set_default_conf_eventdev() function in event_helper.c put defini= tion > of int local vars in one line, remove invalid comment, put > "eventdev_config->ev_queue_mode =3D RTE_EVENT_QUEUE_CFG_ALL_TYPES" > in one line > instead of two. > - Remove extern "C" from event_helper.h. > - Put local vars in reverse xmas tree order in eh_dev_has_rx_internal_por= t() and > eh_dev_has_tx_internal_port() functions in event_helper.c. > - Put #include in alphabetical order in ipsec-secgw.c. > - Move "extern volatile bool force_quit" and "#include " to ip= sec- > secgw.h, > remove #include . > - Remove not needed includes in ipsec_worker.c. > - Remove expired todo from ipsec_worker.h. >=20 > Changes in v3: > * Move eh_conf_init() and eh_conf_uninit() functions to event_helper.c > including minor rework. > * Rename --schedule-type option to --event-schedule-type. > * Replace macro UNPROTECTED_PORT with static inline function > is_unprotected_port(). > * Move definitions of global variables used by multiple modules > to .c files and add externs in .h headers. > * Add eh_check_conf() which validates ipsec-secgw configuration > for event mode. > * Add dynamic calculation of number of buffers in a pool based > on number of cores, ports and crypto queues. > * Fix segmentation fault in event mode driver worker which happens > when there are no inline outbound sessions configured. > * Remove change related to updating number of crypto queues > in cryptodevs_init(). The update of crypto queues will be handled > in a separate patch. > * Fix compilation error on 32-bit platforms by using userdata instead > of udata64 from rte_mbuf. >=20 > Changes in v2: > * Remove --process-dir option. Instead use existing unprotected port mask > option (-u) to decide wheter port handles inbound or outbound traffic. > * Remove --process-mode option. Instead use existing --single-sa option > to select between app and driver modes. > * Add handling of PKT_RX_SEC_OFFLOAD_FAIL result in app worker thread. > * Fix passing of req_rx_offload flags to create_default_ipsec_flow(). > * Move destruction of flows to a location where eth ports are stopped > and closed. > * Print error and exit when event mode --schedule-type option is used > in poll mode. > * Reduce number of goto statements replacing them with loop constructs. > * Remove sec_session_fixed table and replace it with locally build > table in driver worker thread. Table is indexed by port identifier > and holds first inline session pointer found for a given port. > * Print error and exit when sessions other than inline are configured > in event mode. > * When number of event queues is less than number of eth ports then > map all eth ports to one event queue. > * Cleanup and minor improvements in code as suggested by Konstantin >=20 > Ankur Dwivedi (1): > examples/ipsec-secgw: add default rte flow for inline Rx >=20 > Anoob Joseph (5): > examples/ipsec-secgw: add framework for eventmode helper > examples/ipsec-secgw: add eventdev port-lcore link > examples/ipsec-secgw: add Rx adapter support > examples/ipsec-secgw: add Tx adapter support > examples/ipsec-secgw: add routines to display config >=20 > Lukasz Bartosik (9): > examples/ipsec-secgw: add routines to launch workers > examples/ipsec-secgw: add support for internal ports > examples/ipsec-secgw: add event helper config init/uninit > examples/ipsec-secgw: add eventmode to ipsec-secgw > examples/ipsec-secgw: add driver mode worker > examples/ipsec-secgw: add app mode worker > examples/ipsec-secgw: make number of buffers dynamic > doc: add event mode support to ipsec-secgw > examples/ipsec-secgw: reserve crypto queues in event mode >=20 > doc/guides/sample_app_ug/ipsec_secgw.rst | 138 ++- > examples/ipsec-secgw/Makefile | 2 + > examples/ipsec-secgw/event_helper.c | 1812 > ++++++++++++++++++++++++++++++ > examples/ipsec-secgw/event_helper.h | 327 ++++++ > examples/ipsec-secgw/ipsec-secgw.c | 463 ++++++-- > examples/ipsec-secgw/ipsec-secgw.h | 88 ++ > examples/ipsec-secgw/ipsec.c | 5 +- > examples/ipsec-secgw/ipsec.h | 53 +- > examples/ipsec-secgw/ipsec_worker.c | 638 +++++++++++ > examples/ipsec-secgw/ipsec_worker.h | 35 + > examples/ipsec-secgw/meson.build | 6 +- > examples/ipsec-secgw/sa.c | 21 +- > examples/ipsec-secgw/sad.h | 5 - > 13 files changed, 3464 insertions(+), 129 deletions(-) create mode 1006= 44 > examples/ipsec-secgw/event_helper.c > create mode 100644 examples/ipsec-secgw/event_helper.h > create mode 100644 examples/ipsec-secgw/ipsec-secgw.h > create mode 100644 examples/ipsec-secgw/ipsec_worker.c > create mode 100644 examples/ipsec-secgw/ipsec_worker.h >=20 > -- > 2.7.4