From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id BDDA3A04F1; Mon, 6 Jan 2020 18:46:22 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 1E5F21D6DE; Mon, 6 Jan 2020 18:46:22 +0100 (CET) Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by dpdk.org (Postfix) with ESMTP id C87ED1D6BE for ; Mon, 6 Jan 2020 18:46:20 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga106.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 06 Jan 2020 09:46:19 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.69,403,1571727600"; d="scan'208";a="210851520" Received: from fmsmsx104.amr.corp.intel.com ([10.18.124.202]) by orsmga007.jf.intel.com with ESMTP; 06 Jan 2020 09:46:19 -0800 Received: from fmsmsx157.amr.corp.intel.com (10.18.116.73) by fmsmsx104.amr.corp.intel.com (10.18.124.202) with Microsoft SMTP Server (TLS) id 14.3.439.0; Mon, 6 Jan 2020 09:46:19 -0800 Received: from FMSEDG001.ED.cps.intel.com (10.1.192.133) by FMSMSX157.amr.corp.intel.com (10.18.116.73) with Microsoft SMTP Server (TLS) id 14.3.439.0; Mon, 6 Jan 2020 09:46:19 -0800 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (104.47.58.100) by edgegateway.intel.com (192.55.55.68) with Microsoft SMTP Server (TLS) id 14.3.439.0; Mon, 6 Jan 2020 09:46:18 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ed71zTO4uab6JyGpfB4eQfuEIyLCT9A5UWmzr5yztxCN6P+z4xfx9OYiF5NT+Xwl0CJdHvkai1dQqBbLILcvSOUuW9IQPOAMsykeUw74sh53tBKb4BdSoN3kKDeZycgjYDXLV9I/IEGmCcnV7pWeXgNfVajOrLXKNi3MN4ZvS6zBYRr8FleGx1JvrHNQ/aeY2gsSslg9nDZJB1F+sOjs2cW3gAY+RONcmUBIytg2j8z8qOYB7OdLCn2OLdrjy1zwkmag7QX8l90QTW4/0f87yJvGJ7+FewusZCV+xk1fA/u/EDjnPBeCmduKrz7NDRJ8NEbbozUYdHYNjCzUJihVeQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Ra34ntAzSuzRidJy6OTu1BhNTSeS12AKBUYY9FXE2ww=; b=KZTLcEOLO0ZArARq8Cd+xPjUY4AfyEWF8KujKp9sKvV60OWfn7C2ZcEsW167H1apq4F7A7Y4NZXdi6pq0HERo2dayCzP3cUwvpeKM8F7sP+MH2RGCWXuzy1dh29JqgCql8eq2qjk7IWwMXorVtx5E3rySmLcJImPj6vcmWDP5RyD2XMg71NRSvacfK05a26MIAw+SEAjDxrlR6d/zX11e0wceXNuf1VurIxR767z69FK1/TCmoG3vFs7eAG9lQZvw1UV/fV6yCThgbQEUMZcrHE6zCdYkS21YyGCZm7OW5iG3nKr148dnTwvk7IchCXG3CDmO/7jR5ZVJghbddNo3A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Ra34ntAzSuzRidJy6OTu1BhNTSeS12AKBUYY9FXE2ww=; b=BkoLaJL3ZOuT23XE8oEp6SJTcEUTQQmytqLylXzi/sJQPIi1w5yz04RaUTPtme27SXumQ7p25IxNy+XeaKJXrCmk8hrTHYQEfyHJ6msV6FdYH8Wl35kDy7Y92AK55F/Vib1kmjXzmbEqSuWvTJLPvNuvRTvya6C6dw6RiPiYw+Y= Received: from SN6PR11MB2558.namprd11.prod.outlook.com (52.135.94.19) by SN6PR11MB3423.namprd11.prod.outlook.com (52.135.127.214) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2602.13; Mon, 6 Jan 2020 17:46:17 +0000 Received: from SN6PR11MB2558.namprd11.prod.outlook.com ([fe80::4d86:362a:13c3:8386]) by SN6PR11MB2558.namprd11.prod.outlook.com ([fe80::4d86:362a:13c3:8386%7]) with mapi id 15.20.2602.015; Mon, 6 Jan 2020 17:46:17 +0000 From: "Ananyev, Konstantin" To: Anoob Joseph , Akhil Goyal , "Nicolau, Radu" , Thomas Monjalon CC: Ankur Dwivedi , Jerin Jacob Kollanukkaran , Narayana Prasad Raju Athreya , Archana Muniganti , Tejasree Kondoj , Vamsi Krishna Attunuru , Lukas Bartosik , "dev@dpdk.org" Thread-Topic: [PATCH 12/14] examples/ipsec-secgw: add driver outbound worker Thread-Index: AQHVrcObmnDTU18PTEqxffjPihYmT6fIDP4AgBJzZQCAA4bpYA== Date: Mon, 6 Jan 2020 17:46:17 +0000 Message-ID: References: <1575808249-31135-1-git-send-email-anoobj@marvell.com> <1575808249-31135-13-git-send-email-anoobj@marvell.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiNTJlOTVhMzAtNDJkMi00ODIzLWI5ZDItYjJhNjkxZjdiZTEzIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjEwLjE4MDQuNDkiLCJUcnVzdGVkTGFiZWxIYXNoIjoiXC83TGU2b2RZQ3Zkc3dtRTlFYkVwXC9RcmQ4Z2luOE9LWkI3dGRRTzV3Z3MyZkNDV0pZOHIxUjBtb2Y0Z04rYTVFIn0= dlp-product: dlpe-windows dlp-reaction: no-action dlp-version: 11.2.0.6 x-ctpclassification: CTP_NT authentication-results: spf=none (sender IP is ) smtp.mailfrom=konstantin.ananyev@intel.com; x-originating-ip: [192.198.151.163] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 20b9910f-cd49-45a9-a8f5-08d792d054fc x-ms-traffictypediagnostic: SN6PR11MB3423: x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:7691; x-forefront-prvs: 0274272F87 x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(346002)(376002)(136003)(39860400002)(366004)(396003)(199004)(189003)(64756008)(66446008)(66946007)(7696005)(66476007)(66556008)(55016002)(76116006)(9686003)(186003)(4326008)(6506007)(26005)(33656002)(478600001)(54906003)(110136005)(7416002)(316002)(5660300002)(71200400001)(81166006)(81156014)(2906002)(8676002)(52536014)(8936002)(86362001); DIR:OUT; SFP:1102; SCL:1; SRVR:SN6PR11MB3423; H:SN6PR11MB2558.namprd11.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: e0iPWH0E351CEipCQWusp72S7P/G5Gz1w/LK07jZpynGL33h6jDHpKqYC8LJSELFhsxUOBTF0hCXEFjjFpNxTAIz5BjEIh89mdYOEwHQTAwpsjCOwznBs7oU8222zKnp9diR7qDT4xs0m+cCr7H4zUXOM8eGUk/4CdDyJFhKE1LhoSnJn21VpwNpZGmybBzQHdgNvB11rwRuIflBomPhkdtajpS1v845Jy3zLvRWzAdX80vqHJB++j+7iE+tgMfL04KtCeDOQ/iAy8tl9OaNUauqCgzXq9Vpp+a8bGPsyJHhchWXISnHzkoFFlLT4mr6u6m8EDI9KGAj2TJ9KboouN3Xa8unpU8qDwMIvYqwt7sH9xCPHdLu19O7ssEo2W2G2yyFRKYe1tL4eLuEHp4bgnMUSzad6W04Kekt+D14U43F9rT1o+bQJaGQbGr70fhp Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: 20b9910f-cd49-45a9-a8f5-08d792d054fc X-MS-Exchange-CrossTenant-originalarrivaltime: 06 Jan 2020 17:46:17.1852 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: /rp/1UPJxSrfB0ML+QY5e84VBmhxSl8rxXG5ZXD8Y+v7DP1piOMg3GNIR6k5EvNWT0kGCOL+s3Jhnn6vayQRLI0dFJuIBWhNCG1pU0mlRC4= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR11MB3423 X-OriginatorOrg: intel.com Subject: Re: [dpdk-dev] [PATCH 12/14] examples/ipsec-secgw: add driver outbound worker X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > > > This patch adds the driver outbound worker thread for ipsec-secgw. > > > In this mode the security session is a fixed one and sa update is not > > > done. > > > > > > Signed-off-by: Ankur Dwivedi > > > Signed-off-by: Anoob Joseph > > > Signed-off-by: Lukasz Bartosik > > > --- > > > examples/ipsec-secgw/ipsec-secgw.c | 12 +++++ > > > examples/ipsec-secgw/ipsec.c | 9 ++++ > > > examples/ipsec-secgw/ipsec_worker.c | 90 > > > ++++++++++++++++++++++++++++++++++++- > > > 3 files changed, 110 insertions(+), 1 deletion(-) > > > > > > diff --git a/examples/ipsec-secgw/ipsec-secgw.c > > > b/examples/ipsec-secgw/ipsec-secgw.c > > > index 2e7d4d8..76719f2 100644 > > > --- a/examples/ipsec-secgw/ipsec-secgw.c > > > +++ b/examples/ipsec-secgw/ipsec-secgw.c > > > @@ -2011,6 +2011,18 @@ cryptodevs_init(void) > > > i++; > > > } > > > > > > + /* > > > + * Set the queue pair to at least the number of ethernet > > > + * devices for inline outbound. > > > + */ > > > + qp =3D RTE_MAX(rte_eth_dev_count_avail(), qp); > > > > > > Not sure, what for? > > Why we can't process packets from several eth devs on the same crypto-d= ev > > queue? >=20 > [Anoob] This is because of a limitation in our hardware. In our hardware,= it's the crypto queue pair which would be submitting to the > ethernet queue for Tx. But in DPDK spec, the security processing is done = by the ethernet PMD Tx routine alone. We manage to do this by > sharing the crypto queue internally. The crypto queues initialized during= crypto_configure() gets mapped to various ethernet ports. Because > of this, we need to have atleast as many crypto queues as the number of e= th ports. Ok, but that breaks current behavior. Right now in poll-mode it is possible to map traffic from N eth-devs to M c= rypto-devs (N>=3D M, by using M lcores). Would prefer to keep this functionality in place.=20 >=20 > The above change is required because here we limit the number of crypto q= ps based on the number of cores etc. So when tried on single > core, the qps get limited to 1, which causes session_create() to fail for= all ports other than the first one. >=20 > > > > > + > > > + /* > > > + * The requested number of queues should never exceed > > > + * the max available > > > + */ > > > + qp =3D RTE_MIN(qp, max_nb_qps); > > > + > > > if (qp =3D=3D 0) > > > continue; > > > > > > diff --git a/examples/ipsec-secgw/ipsec.c > > > b/examples/ipsec-secgw/ipsec.c index e529f68..9ff8a63 100644 > > > --- a/examples/ipsec-secgw/ipsec.c > > > +++ b/examples/ipsec-secgw/ipsec.c > > > @@ -141,6 +141,10 @@ create_lookaside_session(struct ipsec_ctx > > *ipsec_ctx, struct ipsec_sa *sa, > > > return 0; > > > } > > > > > > +uint16_t sa_no; > > > +#define MAX_FIXED_SESSIONS 10 > > > +struct rte_security_session *sec_session_fixed[MAX_FIXED_SESSIONS]; > > > + > > > int > > > create_inline_session(struct socket_ctx *skt_ctx, struct ipsec_sa *s= a, > > > struct rte_ipsec_session *ips) > > > @@ -401,6 +405,11 @@ create_inline_session(struct socket_ctx *skt_ctx= , > > > struct ipsec_sa *sa, > > > > > > ips->security.ol_flags =3D sec_cap->ol_flags; > > > ips->security.ctx =3D sec_ctx; > > > + if (sa_no < MAX_FIXED_SESSIONS) { > > > + sec_session_fixed[sa_no] =3D > > > + ipsec_get_primary_session(sa)- > > >security.ses; > > > + sa_no++; > > > + } > > > } > > > > Totally lost what is the purpose of these changes... > > Why first 10 inline-proto are special and need to be saved inside globa= l array > > (sec_session_fixed)? > > Why later, in ipsec_worker.c this array is referenced by eth port_id? > > What would happen if number of inline-proto sessions is less than numbe= r of > > eth ports? >=20 > [Anoob] This is required for the outbound driver mode. The 'driver mode' = is more like 'single_sa' mode of the existing application. The idea > is to skip all the lookups etc done in the s/w and perform ipsec processi= ng fully in h/w. In outbound, following is roughly what we should do > for driver mode, >=20 > pkt =3D rx_burst(); >=20 > /* set_pkt_metadata() */ > pkt-> udata64 =3D session; >=20 > tx_burst(pkt); >=20 > The session is created on eth ports. And so, if we have single SA, then t= he entire traffic will have to be forwarded on the same port. The > above change is to make sure we could send traffic on all ports. >=20 > Currently we just use the first 10 SAs and save it in the array. So the u= ser has to set the conf properly and make sure the SAs are distributed > such. Will update this to save the first parsed outbound SA for a port in= the array. That way the size of the array will be > RTE_MAX_ETHPORTS. Ok, then if it is for specific case (event-mode + sing-sa mode) then in cre= ate_inline_session we probably shouldn't do it always, but only when this mode is selected. Also wouldn't it better to reuse current single-sa cmd-line option and log= ic? I.E. whe event-mode and single-sa is selected, go though all eth-devs and f= or each do create_inline_session() with for sa that corresponds to sing_sa_idx= ? Then, I think create_inline_session() can be kept intact. =20 >=20 > Is the above approach fine? >=20 > > > > > set_cdev_id: > > > diff --git a/examples/ipsec-secgw/ipsec_worker.c > > > b/examples/ipsec-secgw/ipsec_worker.c > > > index 2af9475..e202277 100644 > > > --- a/examples/ipsec-secgw/ipsec_worker.c > > > +++ b/examples/ipsec-secgw/ipsec_worker.c > > > @@ -263,7 +263,7 @@ process_ipsec_ev_inbound(struct ipsec_ctx *ctx, > > struct route_table *rt, > > > */ > > > > > > /* Workers registered */ > > > -#define IPSEC_EVENTMODE_WORKERS 2 > > > +#define IPSEC_EVENTMODE_WORKERS 3 > > > > > > /* > > > * Event mode worker > > > @@ -423,6 +423,84 @@ > > ipsec_wrkr_non_burst_int_port_app_mode_inb(struct eh_event_link_info > > *links, > > > return; > > > } > > > > > > +/* > > > + * Event mode worker > > > + * Operating parameters : non-burst - Tx internal port - driver mode > > > +- outbound */ extern struct rte_security_session > > > +*sec_session_fixed[]; static void > > > +ipsec_wrkr_non_burst_int_port_drvr_mode_outb(struct > > eh_event_link_info *links, > > > + uint8_t nb_links) > > > +{ > > > + unsigned int nb_rx =3D 0; > > > + struct rte_mbuf *pkt; > > > + unsigned int port_id; > > > + struct rte_event ev; > > > + uint32_t lcore_id; > > > + > > > + /* Check if we have links registered for this lcore */ > > > + if (nb_links =3D=3D 0) { > > > + /* No links registered - exit */ > > > + goto exit; > > > + } > > > + > > > + /* Get core ID */ > > > + lcore_id =3D rte_lcore_id(); > > > + > > > + RTE_LOG(INFO, IPSEC, > > > + "Launching event mode worker (non-burst - Tx internal port - > > " > > > + "driver mode - outbound) on lcore %d\n", lcore_id); > > > + > > > + /* We have valid links */ > > > + > > > + /* Check if it's single link */ > > > + if (nb_links !=3D 1) { > > > + RTE_LOG(INFO, IPSEC, > > > + "Multiple links not supported. Using first link\n"); > > > + } > > > + > > > + RTE_LOG(INFO, IPSEC, " -- lcoreid=3D%u event_port_id=3D%u\n", > > lcore_id, > > > + links[0].event_port_id); > > > + while (!force_quit) { > > > + /* Read packet from event queues */ > > > + nb_rx =3D rte_event_dequeue_burst(links[0].eventdev_id, > > > + links[0].event_port_id, > > > + &ev, /* events */ > > > + 1, /* nb_events */ > > > + 0 /* timeout_ticks */); > > > + > > > + if (nb_rx =3D=3D 0) > > > + continue; > > > + > > > + port_id =3D ev.queue_id; > > > + pkt =3D ev.mbuf; > > > + > > > + rte_prefetch0(rte_pktmbuf_mtod(pkt, void *)); > > > + > > > + /* Process packet */ > > > + ipsec_event_pre_forward(pkt, port_id); > > > + > > > + pkt->udata64 =3D (uint64_t) sec_session_fixed[port_id]; > > > + > > > + /* Mark the packet for Tx security offload */ > > > + pkt->ol_flags |=3D PKT_TX_SEC_OFFLOAD; > > > + > > > + /* > > > + * Since tx internal port is available, events can be > > > + * directly enqueued to the adapter and it would be > > > + * internally submitted to the eth device. > > > + */ > > > + rte_event_eth_tx_adapter_enqueue(links[0].eventdev_id, > > > + links[0].event_port_id, > > > + &ev, /* events */ > > > + 1, /* nb_events */ > > > + 0 /* flags */); > > > + } > > > + > > > +exit: > > > + return; > > > +} > > > + > > > static uint8_t > > > ipsec_eventmode_populate_wrkr_params(struct > > eh_app_worker_params > > > *wrkrs) { @@ -449,6 +527,16 @@ > > > ipsec_eventmode_populate_wrkr_params(struct eh_app_worker_params > > *wrkrs) > > > wrkr->cap.ipsec_dir =3D EH_IPSEC_DIR_TYPE_INBOUND; > > > wrkr->worker_thread =3D > > ipsec_wrkr_non_burst_int_port_app_mode_inb; > > > > > > + wrkr++; > > > + nb_wrkr_param++; > > > + > > > + /* Non-burst - Tx internal port - driver mode - outbound */ > > > + wrkr->cap.burst =3D EH_RX_TYPE_NON_BURST; > > > + wrkr->cap.tx_internal_port =3D EH_TX_TYPE_INTERNAL_PORT; > > > + wrkr->cap.ipsec_mode =3D EH_IPSEC_MODE_TYPE_DRIVER; > > > + wrkr->cap.ipsec_dir =3D EH_IPSEC_DIR_TYPE_OUTBOUND; > > > + wrkr->worker_thread =3D > > ipsec_wrkr_non_burst_int_port_drvr_mode_outb; > > > + > > > nb_wrkr_param++; > > > return nb_wrkr_param; > > > } > > > -- > > > 2.7.4