From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by dpdk.org (Postfix) with ESMTP id AEB40A84E for ; Tue, 20 Feb 2018 19:55:11 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga103.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 20 Feb 2018 10:55:10 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.46,540,1511856000"; d="scan'208";a="205621712" Received: from fmsmsx104.amr.corp.intel.com ([10.18.124.202]) by fmsmga005.fm.intel.com with ESMTP; 20 Feb 2018 10:55:10 -0800 Received: from fmsmsx108.amr.corp.intel.com ([169.254.9.125]) by fmsmsx104.amr.corp.intel.com ([169.254.3.66]) with mapi id 14.03.0319.002; Tue, 20 Feb 2018 10:55:09 -0800 From: "Vangati, Narender" To: Jerin Jacob , "Gujjar, Abhinandan S" CC: "dev@dpdk.org" , "Rao, Nikhil" , "Eads, Gage" , "hemant.agrawal@nxp.com" , "akhil.goyal@nxp.com" , "narayanaprasad.athreya@cavium.com" , "nidadavolu.murthy@cavium.com" , "nithin.dabilpuram@cavium.com" Thread-Topic: [RFC v2, 2/2] eventdev: add crypto adapter API header Thread-Index: AQHTje8wfckl01Eyo0mJHZpDEv65JqOoJGCAgAQmTwCAAcWSgP//tPsQ Date: Tue, 20 Feb 2018 18:55:09 +0000 Message-ID: <3B5C8B5D2B969C4CBE78502867F1AD9A3E2FFDA1@FMSMSX108.amr.corp.intel.com> References: <1516013630-146114-1-git-send-email-abhinandan.gujjar@intel.com> <20180216193348.GA8882@jerin> <5612CB344B05EE4F95FC5B729939F7807069B737@PGSMSX102.gar.corp.intel.com> <20180220135920.GA23970@jerin> In-Reply-To: <20180220135920.GA23970@jerin> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiMDQxZmQ3M2ItMmU1Ny00YjJhLTg0YTItMmUwZjE2YjI4ZTI2IiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE2LjUuOS4zIiwiVHJ1c3RlZExhYmVsSGFzaCI6IlJEUlU5alo4S21ORmhHbXJBWHlBSWtKcTFra2UrUmU0ZmlST0FtVVhFNFU9In0= x-ctpclassification: CTP_NT dlp-product: dlpe-windows dlp-version: 11.0.0.116 dlp-reaction: no-action x-originating-ip: [10.1.200.108] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [RFC v2, 2/2] eventdev: add crypto adapter API header X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Feb 2018 18:55:12 -0000 Jerin, I see the "ENQ" part of the adapter a little differently. I think there is = value to offloading cryptodev_enqueue to an adapter service, even when the = h/w natively supports DEQ.=20 When the same queue pair is being used by the workers to enqueue requests (= this could be where the pre crypto stage was ordered scheduled and the cryp= todev enqueues need to be ordered), you would need some synchronization. Go= ing thru eventdev to an adapter which enqueues on worker behalf provides th= at synchronization and ordering. In that sense, the ENQ-DEQ mode is an application choice and defines its pr= ogramming model. Based on that, a service would be created that does ENQ. F= or the DEQ part, perhaps the adapter can tell whether the h/w supports DEQ = natively or needs to be done in s/w - that way you can have a single adapte= r. Application model for DEQ mode - call cryptodev_enqueue directly. DEQ is h/= w or s/w based on capability Application model for ENQ-DEQ mode - use event_enqueue to offload cryptodev= _enqueue to adapter. DEQ is h/w or s/w based on capability vnr --- -----Original Message----- From: Jerin Jacob [mailto:jerin.jacob@caviumnetworks.com]=20 Sent: Tuesday, February 20, 2018 7:59 AM To: Gujjar, Abhinandan S Cc: dev@dpdk.org; Vangati, Narender ; Rao, Nikh= il ; Eads, Gage ; hemant.agrawal= @nxp.com; akhil.goyal@nxp.com; narayanaprasad.athreya@cavium.com; nidadavol= u.murthy@cavium.com; nithin.dabilpuram@cavium.com Subject: Re: [RFC v2, 2/2] eventdev: add crypto adapter API header -----Original Message----- > Date: Mon, 19 Feb 2018 10:55:58 +0000 > From: "Gujjar, Abhinandan S" > To: Jerin Jacob > CC: "dev@dpdk.org" , "Vangati, Narender" > , "Rao, Nikhil" ,=20 > "Eads, Gage" , "hemant.agrawal@nxp.com" > , "akhil.goyal@nxp.com"=20 > , "narayanaprasad.athreya@cavium.com"=20 > , > "nidadavolu.murthy@cavium.com" , =20 > "nithin.dabilpuram@cavium.com" > Subject: RE: [RFC v2, 2/2] eventdev: add crypto adapter API header >=20 > Hi Jerin, Hi Abhinandan, >=20 > Thanks for the review. Please find few comments inline. >=20 > > -----Original Message----- > > From: Jerin Jacob [mailto:jerin.jacob@caviumnetworks.com] > > Sent: Saturday, February 17, 2018 1:04 AM > > To: Gujjar, Abhinandan S > > Cc: dev@dpdk.org; Vangati, Narender ;=20 > > Rao, Nikhil ; Eads, Gage=20 > > ; hemant.agrawal@nxp.com; akhil.goyal@nxp.com;=20 > > narayanaprasad.athreya@cavium.com; nidadavolu.murthy@cavium.com;=20 > > nithin.dabilpuram@cavium.com > > Subject: Re: [RFC v2, 2/2] eventdev: add crypto adapter API header > >=20 > > -----Original Message----- > > > Date: Mon, 15 Jan 2018 16:23:50 +0530 > > > From: Abhinandan Gujjar > > > To: jerin.jacob@caviumnetworks.com > > > CC: dev@dpdk.org, narender.vangati@intel.com, Abhinandan Gujjar=20 > > > , Nikhil Rao ,=20 > > > Gage Eads > > > Subject: [RFC v2, 2/2] eventdev: add crypto adapter API header > > > X-Mailer: git-send-email 1.9.1 > > > > > > + > > > +/** > > > + * This adapter adds support to enqueue crypto completions to event = device. > > > + * The packet flow from cryptodev to the event device can be=20 > > > +accomplished > > > + * using both SW and HW based transfer mechanisms. > > > + * The adapter uses a EAL service core function for SW based=20 > > > +packet transfer > > > + * and uses the eventdev PMD functions to configure HW based=20 > > > +packet transfer > > > + * between the cryptodev and the event device. > > > + * > > > + * In the case of SW based transfers, application can choose to=20 > > > +submit a > >=20 > > I think, we can remove "In the case of SW based transfers" as it=20 > > should be applicable for HW case too > Ok. In that case, adapter will detect the presence of HW connection=20 > between cryptodev & eventdev and will not dequeue crypto completions. I would say presence of "specific capability" instead of HW. >=20 > >=20 > > > + * crypto operation directly to cryptodev or send it to the=20 > > > + cryptodev > > > + * adapter via eventdev, the cryptodev adapter then submits the=20 > > > + crypto > > > + * operation to the crypto device. The first mode is known as the > >=20 > > The first mode (DEQ) is very clear. In the second mode(ENQ_DEQ), > > - How does "worker" submits the crypto work through crypto-adapter? > > If I understand it correctly, "workers" always deals with only=20 > > cryptodev's > > rte_cryptodev_enqueue_burst() API and "service" function in crypto=20 > > adapter would be responsible for dequeue() from cryptodev and enqueue t= o eventdev? > >=20 > > I understand the need for OP_NEW vs OP_FWD mode difference in both mode= s. > > Other than that, What makes ENQ_DEQ different? Could you share the=20 > > flow for ENQ_DEQ mode with APIs. >=20 > /* > Application changes for ENQ_DEQ mode: > ------------------------------------------------- > /* In ENQ_DEQ mode, to enqueue to adapter app > * has to fill out following details. > */ > struct rte_event_crypto_request *req; > struct rte_crypto_op *op =3D rte_crypto_op_alloc(); > =09 > /* fill request info */ > req =3D (void *)((char *)op + op.private_data_offset); > req->cdev_id =3D 1; > req->queue_pair_id =3D 1; >=20 > /* fill response info */ > ... >=20 > /* send event to crypto adapter */ > ev->event_ptr =3D op; > ev->queue_id =3D dst_event_qid; > ev->priority =3D dst_priority; > ev->sched_type =3D dst_sched_type; > ev->event_type =3D RTE_EVENT_TYPE_CRYPTODEV; > ev->sub_event_type =3D sub_event_type; > ev->flow_id =3D dst_flow_id; > ret =3D rte_event_enqueue_burst(event_dev_id, event_port_id, ev, 1); >=20 >=20 > Adapter in ENQ_DEQ mode, submitting crypto ops to cryptodev: > -------------------------------------------------------------------------= ---- > n =3D rte_event_dequeue_burst(event_dev_id, event_port_id, ev, BATCH_SIZ= E, time_out); > struct rte_crypto_op *op =3D ev->event_ptr; > struct rte_event_crypto_request *req =3D (void *)op + op.private_data_of= fset; > cdev_id =3D req->cdev_id; > qp_id =3D req->queue_pair_id >=20 > ret =3D rte_cryptodev_enqueue_burst(cdev_id, qp_id, op, 1); This mode wont work for the HW implementations that I know. As in HW implem= entations, The Adapter is embedded in HW. The DEQ mode works. But, This would call for to have two separate applicati= on logic for DEQ and ENQ_DEQ mode. I think, it is unavoidable as SW scheme has better performance with ENQ_DEQ= MODE. If you think, there is no option other than introducing a capability in ada= pter then please create capability in Rx adapter to inform the adapter capa= bility to the application.=20 Do we think, it possible to have scheme with ENQ_DEQ mode, Where applicatio= n still enqueue to cryptodev like DEQ mode but using cryptodev. ie. Adapter= patches the cryptodev dev->enqueue_burst() to "eventdev enqueue burst" fol= lowed by "exiting dev->enqueue_burst". Something like exiting ethdev rx_burst callback scheme. This will enable application to have unified flow IMO. Any thoughts from NXP folks? > */ > >=20 > > > + * dequeue only (DEQ) mode and the second as the enqueue -=20 > > > + dequeue > >=20 > > extra space between "mode" and "and" > Ok > >=20 > > > + * (ENQ_DEQ) mode. The choice of mode can be specified when=20 > > > + creating > > > + * the adapter. > > > + * In the latter choice, the cryptodev adapter is able to use > > > + * RTE_OP_FORWARD as the event dev enqueue type, this has a=20 > > > + performance > > > + * advantage in "closed system" eventdevs like the eventdev SW=20 > > > + PMD and > >=20