From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7179EA04F7; Tue, 7 Jan 2020 05:32:19 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 9D4FE1D8EB; Tue, 7 Jan 2020 05:32:18 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by dpdk.org (Postfix) with ESMTP id 9B3761D8E7 for ; Tue, 7 Jan 2020 05:32:17 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 0074UpEZ032458; Mon, 6 Jan 2020 20:32:16 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-transfer-encoding : mime-version; s=pfpt0818; bh=oAjvML8KV8di6yb2eR1/CfZx5vO1ASslUGiIvesjbts=; b=QATcrzR617cJ+FKNoZyw7oM0KHM4nSMLTkqx7viuhnZJY1qmS9F9nziiuJZFYi+PMIa5 NrJFr4D5zu6C2Mcx9jfX8+rpGRFkMdzH2pvpDfSoMCaKMbv1878mZ7gaG5Q3sbvK3JYh NnCm9Whe+B2Ow2VOHvDmD6dY7aqfjev8LkeZv4bRAfF1wh29FdGJZPX2i5iglx5DBWd/ 6QsLuTCpDv8itX8/BF86Qv69hSo2L9ZNdyEexPB3/FHrTpHta2GphKnr3WjRh54HgM9h T2VufQb+sL5/YUUp1B/XUYhxKDEv6FTZxz0HIigZwHiycA6DnpWONwPib4FHom+Vs2rH qA== Received: from sc-exch01.marvell.com ([199.233.58.181]) by mx0a-0016f401.pphosted.com with ESMTP id 2xarxv91vj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Mon, 06 Jan 2020 20:32:16 -0800 Received: from SC-EXCH04.marvell.com (10.93.176.84) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 6 Jan 2020 20:32:15 -0800 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (104.47.55.105) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1497.2 via Frontend Transport; Mon, 6 Jan 2020 20:32:15 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=GThc+cgRQaLUUmx55J5xWTm1Ah7qUIVT3SHd4SRuHMGbeJ5EHu6nddnZ8dFkbjmnx4E6SUd7/6YMIDRsSx23+jfaz/M97/gl9374Axxr7xlbiec4b7GOLVS/51kt+nmw/pBxoEhNmvSS0Vwwrh5Rj4Uot8nal/9wqDwnrJ8Pqmz3yK+HkLe9hOn3l99QFTyPulNmaHvEbh7KU/zBtVmHvTqnzEoj8W9KF658gMqwl2RR59+Y5/pPGvBFMG3S5Vw+yEk6f9F4qHz4R/tZ+P+FENdmIA5ir/zSZX79DzohsbqJGfm25z0TfB5N6IwE3JZZ4Tfsbx46+swU+WvR/wvupA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=oAjvML8KV8di6yb2eR1/CfZx5vO1ASslUGiIvesjbts=; b=UYfwd22tgKjnk3es7NiYkNtK1EnyvwK1qCV0Qy+7hZWCgbZkOFI2vrc+ZEjd/bB/wiMVfjD3QdjYynRZjuESssd/RnY3HcmYpuLUPUWP+Sbw4orNvITcujH0olmz3Q7qJmwMeuzQa2VTfwmoOrlrwaDuKWr09XPR7dzA80mIak8qoeq/WFC2NuZb1cY0o9qYTVtn+pGw0gRLt5aO/XF4g0nYO6oeJ0SeGpFI52pmpKBviLG5zvqILGSQ752QipJduwa9eMYmV0Q5VQqH49NKWvK9T2qhiaRXPedgiLQkcRBFYMZaVJJOaT4CnqGVXaFCSFhi1snk6jCvnmo9ej/P7A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=oAjvML8KV8di6yb2eR1/CfZx5vO1ASslUGiIvesjbts=; b=gWMGeEqViu/pYm576j1pppmE6HQQQ+5w66GN05yApxOJKgWxkJIh+gObtoEP9XYKmsYkOfir24D4hF98r9UwiN4QvEVAbK0eja3/W9L9D9vjzeLXcgi5cQqwS4tDgEwIZBFMRW2ySGiC2eMAFpUaeqWug+maXD3vMdOYfTr6vH0= Received: from MN2PR18MB2877.namprd18.prod.outlook.com (20.179.20.218) by MN2PR18MB2479.namprd18.prod.outlook.com (20.179.83.155) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2602.12; Tue, 7 Jan 2020 04:32:13 +0000 Received: from MN2PR18MB2877.namprd18.prod.outlook.com ([fe80::5db5:d179:8a01:4636]) by MN2PR18MB2877.namprd18.prod.outlook.com ([fe80::5db5:d179:8a01:4636%7]) with mapi id 15.20.2602.016; Tue, 7 Jan 2020 04:32:13 +0000 From: Anoob Joseph To: "Ananyev, Konstantin" , Akhil Goyal , "Nicolau, Radu" , "Thomas Monjalon" CC: Ankur Dwivedi , Jerin Jacob Kollanukkaran , Narayana Prasad Raju Athreya , Archana Muniganti , Tejasree Kondoj , Vamsi Krishna Attunuru , Lukas Bartosik , "dev@dpdk.org" Thread-Topic: [dpdk-dev] [PATCH 12/14] examples/ipsec-secgw: add driver outbound worker Thread-Index: AQHVxLk7B/fmVycoWU253bHe0DTTpafema5g Date: Tue, 7 Jan 2020 04:32:13 +0000 Message-ID: References: <1575808249-31135-1-git-send-email-anoobj@marvell.com> <1575808249-31135-13-git-send-email-anoobj@marvell.com> In-Reply-To: Accept-Language: en-IN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [115.113.156.3] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 193b3f46-c756-4161-9f1c-08d7932a9195 x-ms-traffictypediagnostic: MN2PR18MB2479: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:8882; x-forefront-prvs: 027578BB13 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(4636009)(396003)(376002)(346002)(366004)(39850400004)(136003)(13464003)(199004)(189003)(6506007)(7696005)(2906002)(53546011)(52536014)(55016002)(30864003)(76116006)(66446008)(186003)(9686003)(66556008)(66476007)(64756008)(66946007)(55236004)(33656002)(5660300002)(81156014)(4326008)(26005)(8936002)(478600001)(110136005)(71200400001)(81166006)(316002)(86362001)(54906003); DIR:OUT; SFP:1101; SCL:1; SRVR:MN2PR18MB2479; H:MN2PR18MB2877.namprd18.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: marvell.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: GZBnGKrcS5n52H4ginFyVQM8nyn3tS0VqMVW1OoSCDo22QwOqZMpB6J9oBD6u1FJzYGW1oTSoR1HteewSHUwispIgaaYGx1370Wo0GrAQ3J/dZNIHb6Y1JLYaEsy0O3oJmBJGQi4mFFRBEf0oV0Aa0F1aAe5mqCYWmnmpqCnQFjgCjV3dc5eGCzbGSPS5PTGMzPmu9wbLGgA3VBCmO6R8lzrGmCDDDAlw1ZVI08lxzAf3WDt1vF/DiCu5mbTZji3cFIRrtSkUDR7JmJei/j5q4AuAUmpHt7tePY4WOQJHciPQFMLumzK+9f11igF+fXWc7bMG4zdbTg4iQplLN4CK2araJNrgNy5OD3SZhrXuhcP07z9DIT22TNzFjs0J4EWOAvbgud6K/vAknrkHQpI54CQzH+AhnBz5tfpsZ6VDXKPkXFEzAFadURDNAOwpsE7ZRiG1zX98fEjyJ01SS84VtZSTfACrouRvLMktwXGYhVgbtMmzj8YQ5cXaz1GJkeU Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: 193b3f46-c756-4161-9f1c-08d7932a9195 X-MS-Exchange-CrossTenant-originalarrivaltime: 07 Jan 2020 04:32:13.1808 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: /5XE6uVPh/qUl4TU0qgZXUqIGPchiRaRJKWgmr4QQsqq8GMMAVbMbqLvxmYjR/YCkTFr8jnEgq9mkIhy/S7g1g== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR18MB2479 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,18.0.572 definitions=2020-01-06_08:2020-01-06,2020-01-06 signatures=0 Subject: Re: [dpdk-dev] [PATCH 12/14] examples/ipsec-secgw: add driver outbound worker X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Konstantin, Please see inline. Thanks, Anoob > -----Original Message----- > From: dev On Behalf Of Ananyev, Konstantin > Sent: Monday, January 6, 2020 11:16 PM > To: Anoob Joseph ; Akhil Goyal ; > Nicolau, Radu ; Thomas Monjalon > > Cc: Ankur Dwivedi ; Jerin Jacob Kollanukkaran > ; Narayana Prasad Raju Athreya > ; Archana Muniganti ; > Tejasree Kondoj ; Vamsi Krishna Attunuru > ; Lukas Bartosik ; > dev@dpdk.org > Subject: Re: [dpdk-dev] [PATCH 12/14] examples/ipsec-secgw: add driver > outbound worker >=20 > > > > This patch adds the driver outbound worker thread for ipsec-secgw. > > > > In this mode the security session is a fixed one and sa update is > > > > not done. > > > > > > > > Signed-off-by: Ankur Dwivedi > > > > Signed-off-by: Anoob Joseph > > > > Signed-off-by: Lukasz Bartosik > > > > --- > > > > examples/ipsec-secgw/ipsec-secgw.c | 12 +++++ > > > > examples/ipsec-secgw/ipsec.c | 9 ++++ > > > > examples/ipsec-secgw/ipsec_worker.c | 90 > > > > ++++++++++++++++++++++++++++++++++++- > > > > 3 files changed, 110 insertions(+), 1 deletion(-) > > > > > > > > diff --git a/examples/ipsec-secgw/ipsec-secgw.c > > > > b/examples/ipsec-secgw/ipsec-secgw.c > > > > index 2e7d4d8..76719f2 100644 > > > > --- a/examples/ipsec-secgw/ipsec-secgw.c > > > > +++ b/examples/ipsec-secgw/ipsec-secgw.c > > > > @@ -2011,6 +2011,18 @@ cryptodevs_init(void) > > > > i++; > > > > } > > > > > > > > + /* > > > > + * Set the queue pair to at least the number of ethernet > > > > + * devices for inline outbound. > > > > + */ > > > > + qp =3D RTE_MAX(rte_eth_dev_count_avail(), qp); > > > > > > > > > Not sure, what for? > > > Why we can't process packets from several eth devs on the same > > > crypto-dev queue? > > > > [Anoob] This is because of a limitation in our hardware. In our > > hardware, it's the crypto queue pair which would be submitting to the > > ethernet queue for Tx. But in DPDK spec, the security processing is > > done by the ethernet PMD Tx routine alone. We manage to do this by shar= ing > the crypto queue internally. The crypto queues initialized during > crypto_configure() gets mapped to various ethernet ports. Because of this= , we > need to have atleast as many crypto queues as the number of eth ports. >=20 > Ok, but that breaks current behavior. > Right now in poll-mode it is possible to map traffic from N eth-devs to M= crypto- > devs (N>=3D M, by using M lcores). > Would prefer to keep this functionality in place. [Anoob] Understood. I don't think that functionality is broken. If the numb= er of qps available is lower than the number of eth devs, then only the one= s available would be enabled. Inline protocol session for the other eth dev= s would fail for us. Currently, the app assumes that for one core, it needs only one qp (and for= M core, M qp). Is there any harm in enabling all qps available? If such a = change can be done, that would also work for us.=20 >=20 > > > > The above change is required because here we limit the number of > > crypto qps based on the number of cores etc. So when tried on single co= re, the > qps get limited to 1, which causes session_create() to fail for all ports= other than > the first one. > > > > > > > > > + > > > > + /* > > > > + * The requested number of queues should never exceed > > > > + * the max available > > > > + */ > > > > + qp =3D RTE_MIN(qp, max_nb_qps); > > > > + > > > > if (qp =3D=3D 0) > > > > continue; > > > > > > > > diff --git a/examples/ipsec-secgw/ipsec.c > > > > b/examples/ipsec-secgw/ipsec.c index e529f68..9ff8a63 100644 > > > > --- a/examples/ipsec-secgw/ipsec.c > > > > +++ b/examples/ipsec-secgw/ipsec.c > > > > @@ -141,6 +141,10 @@ create_lookaside_session(struct ipsec_ctx > > > *ipsec_ctx, struct ipsec_sa *sa, > > > > return 0; > > > > } > > > > > > > > +uint16_t sa_no; > > > > +#define MAX_FIXED_SESSIONS 10 > > > > +struct rte_security_session > > > > +*sec_session_fixed[MAX_FIXED_SESSIONS]; > > > > + > > > > int > > > > create_inline_session(struct socket_ctx *skt_ctx, struct ipsec_sa = *sa, > > > > struct rte_ipsec_session *ips) > > > > @@ -401,6 +405,11 @@ create_inline_session(struct socket_ctx > > > > *skt_ctx, struct ipsec_sa *sa, > > > > > > > > ips->security.ol_flags =3D sec_cap->ol_flags; > > > > ips->security.ctx =3D sec_ctx; > > > > + if (sa_no < MAX_FIXED_SESSIONS) { > > > > + sec_session_fixed[sa_no] =3D > > > > + ipsec_get_primary_session(sa)- > > > >security.ses; > > > > + sa_no++; > > > > + } > > > > } > > > > > > Totally lost what is the purpose of these changes... > > > Why first 10 inline-proto are special and need to be saved inside > > > global array (sec_session_fixed)? > > > Why later, in ipsec_worker.c this array is referenced by eth port_id? > > > What would happen if number of inline-proto sessions is less than > > > number of eth ports? > > > > [Anoob] This is required for the outbound driver mode. The 'driver > > mode' is more like 'single_sa' mode of the existing application. The > > idea is to skip all the lookups etc done in the s/w and perform ipsec > > processing fully in h/w. In outbound, following is roughly what we > > should do for driver mode, > > > > pkt =3D rx_burst(); > > > > /* set_pkt_metadata() */ > > pkt-> udata64 =3D session; > > > > tx_burst(pkt); > > > > The session is created on eth ports. And so, if we have single SA, > > then the entire traffic will have to be forwarded on the same port. The= above > change is to make sure we could send traffic on all ports. > > > > Currently we just use the first 10 SAs and save it in the array. So > > the user has to set the conf properly and make sure the SAs are > > distributed such. Will update this to save the first parsed outbound SA= for a > port in the array. That way the size of the array will be RTE_MAX_ETHPORT= S. >=20 > Ok, then if it is for specific case (event-mode + sing-sa mode) then in > create_inline_session we probably shouldn't do it always, but only when t= his > mode is selected. [Anoob] Will make that change. =20 > Also wouldn't it better to reuse current single-sa cmd-line option and l= ogic? > I.E. whe event-mode and single-sa is selected, go though all eth-devs and= for > each do create_inline_session() with for sa that corresponds to sing_sa_i= dx? > Then, I think create_inline_session() can be kept intact. [Anoob] No disagreement. Current single_sa uses single_sa universally. The = driver mode intends to use single_sa per port. Technically, just single_sa = (universally) will result in the eth port being the bottleneck. So I can fi= x the single sa and we can use single_sa option in eventmode as you have de= scribed. =20 >=20 > > > > Is the above approach fine? > > > > > > > > > set_cdev_id: > > > > diff --git a/examples/ipsec-secgw/ipsec_worker.c > > > > b/examples/ipsec-secgw/ipsec_worker.c > > > > index 2af9475..e202277 100644 > > > > --- a/examples/ipsec-secgw/ipsec_worker.c > > > > +++ b/examples/ipsec-secgw/ipsec_worker.c > > > > @@ -263,7 +263,7 @@ process_ipsec_ev_inbound(struct ipsec_ctx > > > > *ctx, > > > struct route_table *rt, > > > > */ > > > > > > > > /* Workers registered */ > > > > -#define IPSEC_EVENTMODE_WORKERS 2 > > > > +#define IPSEC_EVENTMODE_WORKERS 3 > > > > > > > > /* > > > > * Event mode worker > > > > @@ -423,6 +423,84 @@ > > > ipsec_wrkr_non_burst_int_port_app_mode_inb(struct eh_event_link_info > > > *links, > > > > return; > > > > } > > > > > > > > +/* > > > > + * Event mode worker > > > > + * Operating parameters : non-burst - Tx internal port - driver > > > > +mode > > > > +- outbound */ extern struct rte_security_session > > > > +*sec_session_fixed[]; static void > > > > +ipsec_wrkr_non_burst_int_port_drvr_mode_outb(struct > > > eh_event_link_info *links, > > > > + uint8_t nb_links) > > > > +{ > > > > + unsigned int nb_rx =3D 0; > > > > + struct rte_mbuf *pkt; > > > > + unsigned int port_id; > > > > + struct rte_event ev; > > > > + uint32_t lcore_id; > > > > + > > > > + /* Check if we have links registered for this lcore */ > > > > + if (nb_links =3D=3D 0) { > > > > + /* No links registered - exit */ > > > > + goto exit; > > > > + } > > > > + > > > > + /* Get core ID */ > > > > + lcore_id =3D rte_lcore_id(); > > > > + > > > > + RTE_LOG(INFO, IPSEC, > > > > + "Launching event mode worker (non-burst - Tx internal port - > > > " > > > > + "driver mode - outbound) on lcore %d\n", lcore_id); > > > > + > > > > + /* We have valid links */ > > > > + > > > > + /* Check if it's single link */ > > > > + if (nb_links !=3D 1) { > > > > + RTE_LOG(INFO, IPSEC, > > > > + "Multiple links not supported. Using first link\n"); > > > > + } > > > > + > > > > + RTE_LOG(INFO, IPSEC, " -- lcoreid=3D%u event_port_id=3D%u\n", > > > lcore_id, > > > > + links[0].event_port_id); > > > > + while (!force_quit) { > > > > + /* Read packet from event queues */ > > > > + nb_rx =3D rte_event_dequeue_burst(links[0].eventdev_id, > > > > + links[0].event_port_id, > > > > + &ev, /* events */ > > > > + 1, /* nb_events */ > > > > + 0 /* timeout_ticks */); > > > > + > > > > + if (nb_rx =3D=3D 0) > > > > + continue; > > > > + > > > > + port_id =3D ev.queue_id; > > > > + pkt =3D ev.mbuf; > > > > + > > > > + rte_prefetch0(rte_pktmbuf_mtod(pkt, void *)); > > > > + > > > > + /* Process packet */ > > > > + ipsec_event_pre_forward(pkt, port_id); > > > > + > > > > + pkt->udata64 =3D (uint64_t) sec_session_fixed[port_id]; > > > > + > > > > + /* Mark the packet for Tx security offload */ > > > > + pkt->ol_flags |=3D PKT_TX_SEC_OFFLOAD; > > > > + > > > > + /* > > > > + * Since tx internal port is available, events can be > > > > + * directly enqueued to the adapter and it would be > > > > + * internally submitted to the eth device. > > > > + */ > > > > + rte_event_eth_tx_adapter_enqueue(links[0].eventdev_id, > > > > + links[0].event_port_id, > > > > + &ev, /* events */ > > > > + 1, /* nb_events */ > > > > + 0 /* flags */); > > > > + } > > > > + > > > > +exit: > > > > + return; > > > > +} > > > > + > > > > static uint8_t > > > > ipsec_eventmode_populate_wrkr_params(struct > > > eh_app_worker_params > > > > *wrkrs) { @@ -449,6 +527,16 @@ > > > > ipsec_eventmode_populate_wrkr_params(struct eh_app_worker_params > > > *wrkrs) > > > > wrkr->cap.ipsec_dir =3D EH_IPSEC_DIR_TYPE_INBOUND; > > > > wrkr->worker_thread =3D > > > ipsec_wrkr_non_burst_int_port_app_mode_inb; > > > > > > > > + wrkr++; > > > > + nb_wrkr_param++; > > > > + > > > > + /* Non-burst - Tx internal port - driver mode - outbound */ > > > > + wrkr->cap.burst =3D EH_RX_TYPE_NON_BURST; > > > > + wrkr->cap.tx_internal_port =3D EH_TX_TYPE_INTERNAL_PORT; > > > > + wrkr->cap.ipsec_mode =3D EH_IPSEC_MODE_TYPE_DRIVER; > > > > + wrkr->cap.ipsec_dir =3D EH_IPSEC_DIR_TYPE_OUTBOUND; > > > > + wrkr->worker_thread =3D > > > ipsec_wrkr_non_burst_int_port_drvr_mode_outb; > > > > + > > > > nb_wrkr_param++; > > > > return nb_wrkr_param; > > > > } > > > > -- > > > > 2.7.4