From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id B6815A04CC; Mon, 21 Sep 2020 14:28:17 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 9346F1D97C; Mon, 21 Sep 2020 14:28:17 +0200 (CEST) Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by dpdk.org (Postfix) with ESMTP id E586C1D96C for ; Mon, 21 Sep 2020 14:28:15 +0200 (CEST) IronPort-SDR: cLFqNIpF4ij/sL/zWYIjs8wJ6l+xloeGVy24ddmyErvuUI61riE85RLv7wos/as81Udu06Q5zR IVs6sUJxddVQ== X-IronPort-AV: E=McAfee;i="6000,8403,9750"; a="148114413" X-IronPort-AV: E=Sophos;i="5.77,286,1596524400"; d="scan'208";a="148114413" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Sep 2020 05:28:11 -0700 IronPort-SDR: DjbzhQY199zYYNl++PPiN6HpDNOeFTz5J/OTioa8UleHZlb3j5x0kerWhee/OesE9r4NeDIN2+ zp3mmQk3EJbg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.77,286,1596524400"; d="scan'208";a="289931345" Received: from fmsmsx606.amr.corp.intel.com ([10.18.126.86]) by fmsmga007.fm.intel.com with ESMTP; 21 Sep 2020 05:28:10 -0700 Received: from fmsmsx603.amr.corp.intel.com (10.18.126.83) by fmsmsx606.amr.corp.intel.com (10.18.126.86) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Mon, 21 Sep 2020 05:28:09 -0700 Received: from fmsedg601.ED.cps.intel.com (10.1.192.135) by fmsmsx603.amr.corp.intel.com (10.18.126.83) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5 via Frontend Transport; Mon, 21 Sep 2020 05:28:09 -0700 Received: from NAM04-BN3-obe.outbound.protection.outlook.com (104.47.46.56) by edgegateway.intel.com (192.55.55.70) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.1713.5; Mon, 21 Sep 2020 05:28:09 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=hQHp1d3DGViJQzzpY4AM7Pc2qklAC0Q+iXEJAJ6wnQNkUiNWOMR2EBfEfIqGoowDKpwGuMAOP6PC6USZicodEY2eDoDVnTAnZJ4kktJ2cQazSm5ozZg+0QHprxl6snhYwJ3w3DXb+K0oljmk578+QXsoQl/CLdkyBIvQZXHKIeiCV0jYnOUaqGsz0/NvcfCMVdKTOv/Eh1qUmwW1ejT4KHlgDiADI+nnCSJQPUMF277O1aT/AAYdKWFPkBGo10Zu1Xk5TWRTT9CymBp9hg+BgAw1fBZRRl466m4i2yfHc0zC+I+yTsuJ0OfM9GhFcLDJk/ZPNdRfyKDsyYFHYIMejw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=TX0H5hT855cF6iXvU8+UmVyCkfbU16YJxhqz78Tjito=; b=Eczeb5S2DtyeoSov2Kgw468BRK9zS7ssPht5rHSSPmdbeO4e23/O1J74GICdSiiYYzq9bFuPOTwC/9JFe3XP6D5969HFS7zv6w4nSVj+ex4u8RMXDpNnlbb/0wMY1QvcvaatGbiLxCXSz0lvLNF4Jh9WlZqrhBeKGdeA5eCvaUZ1uQKua2qOxa3AD0lX3vpMWvHgEmVEZebG7TwiseHerxOKRUZ9o4ObPSbOc5BvCI5zcmvUFrZ1eMlzauOlP2HgObGbPf4PBHi90JLiKePdwi9JieYtZ3qMHzLch5ZoOHSZyK8Prcsx6pmZ7Kdch6j3GHYS/iS4KrFkpprX8UBwow== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=TX0H5hT855cF6iXvU8+UmVyCkfbU16YJxhqz78Tjito=; b=cN7A/2tpZqC0PM8RJT/zQy7igzi0GIjDgQD0wsSmRN9WG82lA3/f41Ee2aLhCvjxz+KCCWlGkCYXxZvFnLIm833HwCA7JhhDY25Z8wc2xBPda6M+mgwLom/ofmcV5fmvfFEVAETdeRPeYiIaso9S/EF0KfmirNhmDWLyn80mozI= Received: from BL0PR11MB3043.namprd11.prod.outlook.com (2603:10b6:208:33::19) by BL0PR11MB3395.namprd11.prod.outlook.com (2603:10b6:208:6c::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3391.15; Mon, 21 Sep 2020 12:28:08 +0000 Received: from BL0PR11MB3043.namprd11.prod.outlook.com ([fe80::11fa:a7fe:329d:9239]) by BL0PR11MB3043.namprd11.prod.outlook.com ([fe80::11fa:a7fe:329d:9239%5]) with mapi id 15.20.3391.024; Mon, 21 Sep 2020 12:28:08 +0000 From: "Zhang, Roy Fan" To: Akhil Goyal , "dev@dpdk.org" CC: "Trahe, Fiona" , "Kusztal, ArkadiuszX" , "Dybkowski, AdamX" , Anoob Joseph Thread-Topic: [dpdk-dev v9 4/4] doc: add cryptodev service APIs guide Thread-Index: AQHWhbweiYo7E6UpO0WHhA+t0mS6Oalu7AQAgAQQIlA= Date: Mon, 21 Sep 2020 12:28:07 +0000 Message-ID: References: <20200904152539.20608-1-roy.fan.zhang@intel.com> <20200908084253.81022-1-roy.fan.zhang@intel.com> <20200908084253.81022-5-roy.fan.zhang@intel.com> In-Reply-To: Accept-Language: zh-Hans-HK, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-version: 11.5.1.3 dlp-reaction: no-action dlp-product: dlpe-windows authentication-results: nxp.com; dkim=none (message not signed) header.d=none;nxp.com; dmarc=none action=none header.from=intel.com; x-originating-ip: [95.44.220.85] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: d26e130b-52f6-4924-54d8-08d85e29cbe6 x-ms-traffictypediagnostic: BL0PR11MB3395: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 8PcjWu2FaGowM4hUq4drCdLZNmCrgJm3B5lQRnbOXFtyn5HlUZRcWleHydipDKQFhor0npJVFllA/pigH7i+oc3u9YKG/thY3cflaVjcAfeRo82UIOs7FXTQtsJfoNMfNQahnfFKpFiEKOpxUyqKXKZuOJQN+EbZIIVy9Bl5Fd2OgTt2JJdzY5q8psnuTKsYwgcKUf0fV+RP8NgKlT/YwuZbedmrAjVNEci4afhvPd2+pTyo6vnbhv9LeD6TJNDVFIzrYsupHCIGYFNqgpKSJuxlzuRvL0hUn5ihDyeeAa/85h+A78vDem/ClBtyoI/VOZTE0sbKgqKApRW10irl/A== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BL0PR11MB3043.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(136003)(39860400002)(396003)(346002)(376002)(366004)(33656002)(9686003)(4326008)(83380400001)(55016002)(5660300002)(478600001)(30864003)(52536014)(86362001)(76116006)(64756008)(66556008)(66476007)(66946007)(71200400001)(66446008)(316002)(26005)(53546011)(2906002)(7696005)(110136005)(8676002)(186003)(54906003)(8936002)(6506007); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata: sJWNndHF59zfrRyLslb0f1sjR9g4X92ixlGC14shSags0opAIC7BhX/JNGX5prmepsF3pQ+ZA7cHPQzKmlBE+SQXEFPSALM2ghWWxF897pQIWRcpKO1jzYC81q1mTmVttJKPnvC0XrSrj4EOvuh7GnmUfNg6cByDjY/PaAdpGe9Zp0kwhUzy0XyhN5rARYMLhzm/T2oNmnvd9KTZf80iJKVuwcXy1l5v1yR39qsLFzKTQKoQ+XTcxwYjp/q2F/G+rE82S2/cBp64+u7sJkc+/ZCeqiYKsBQTjI4MU3XjDOCH8pRPFIv8ERP7pk4jMmJLWCQQm1iliO3MlRhh1klRNUs5Er7/OBVMOBs4fny127syx4ZmhPS7QNMFuXhLQGODdmQ85y3OkmhMQAJouQXnHvMUnd4hOLG94gAgM/lbzdRkS4H6VSrrHHxIwUgAb9gkSC3xM2J0O3BrQrtCtnmAOszvRUc2ogN7lewR6bUVTSFyMB6JXkgMaBgJ/I6L6ZXl9l15vKssixvxQpeS9VXqfbKUIWgjzcUyVVsUGM2zk0dzxxjFXTCPrI4hNrViAiIk8GhYJ9RHYG0sS8zSDVZG3pjyDTJHiTyknf2z4/tlt2Ob8cXdCR5riBJpXrhWbEyNN4HwXe+Iv20IJM995nZh0g== Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: BL0PR11MB3043.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: d26e130b-52f6-4924-54d8-08d85e29cbe6 X-MS-Exchange-CrossTenant-originalarrivaltime: 21 Sep 2020 12:28:07.4668 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: Y4bUU/dFlySYj/wjeCCnTNaqv7zi3o8fs93LuE5uK/gXpY9+QcyUuBsFrWl7FoAI/gtlkyfgan+eG0FrImqb0A== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL0PR11MB3395 X-OriginatorOrg: intel.com Subject: Re: [dpdk-dev] [dpdk-dev v9 4/4] doc: add cryptodev service APIs guide X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Akhil, > -----Original Message----- > From: Akhil Goyal > Sent: Friday, September 18, 2020 9:39 PM > To: Zhang, Roy Fan ; dev@dpdk.org > Cc: Trahe, Fiona ; Kusztal, ArkadiuszX > ; Dybkowski, AdamX > ; Anoob Joseph > Subject: RE: [dpdk-dev v9 4/4] doc: add cryptodev service APIs guide >=20 > Hi Fan, >=20 > > Subject: [dpdk-dev v9 4/4] doc: add cryptodev service APIs guide > > > > This patch updates programmer's guide to demonstrate the usage > > and limitations of cryptodev symmetric crypto data-path service > > APIs. > > > > Signed-off-by: Fan Zhang > > --- > > doc/guides/prog_guide/cryptodev_lib.rst | 90 > +++++++++++++++++++++++++ > > doc/guides/rel_notes/release_20_11.rst | 7 ++ > > 2 files changed, 97 insertions(+) >=20 > We generally do not take separate patches for documentation. Squash it th= e > patches which > Implement the feature. >=20 > > > > diff --git a/doc/guides/prog_guide/cryptodev_lib.rst > > b/doc/guides/prog_guide/cryptodev_lib.rst > > index c14f750fa..1321e4c5d 100644 > > --- a/doc/guides/prog_guide/cryptodev_lib.rst > > +++ b/doc/guides/prog_guide/cryptodev_lib.rst > > @@ -631,6 +631,96 @@ a call argument. Status different than zero must b= e > > treated as error. > > For more details, e.g. how to convert an mbuf to an SGL, please refer = to an > > example usage in the IPsec library implementation. > > > > +Cryptodev Direct Data-path Service API > > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >=20 > What do you mean by Direct here. It should be referenced as raw APIs? > Moreover, service keyword can also be dropped. We normally use it for > Software implementation of a feature which is normally done in hardware. >=20 You are right, raw API is a good name. Will remove service keyword. >=20 > > + > > +Direct crypto data-path service are a set of APIs that especially prov= ided > for > > +the external libraries/applications who want to take advantage of the = rich > > +features provided by cryptodev, but not necessarily depend on cryptode= v > > +operations, mempools, or mbufs in the their data-path implementations. >=20 > Raw crypto data path is a set of APIs which can be used by external > libraries/applications to take advantage of the rich features provided by > cryptodev, > but not necessarily depend on cryptodev operations, mempools, or mbufs in > the > data-path implementations. >=20 Thanks. > > + > > +The direct crypto data-path service has the following advantages: > > +- Supports raw data pointer and physical addresses as input. > > +- Do not require specific data structure allocated from heap, such as > > + cryptodev operation. > > +- Enqueue in a burst or single operation. The service allow enqueuing = in > > + a burst similar to ``rte_cryptodev_enqueue_burst`` operation, or onl= y > > + enqueue one job at a time but maintaining necessary context data loc= ally > for > > + next single job enqueue operation. The latter method is especially > helpful > > + when the user application's crypto operations are clustered into a b= urst. > > + Allowing enqueue one operation at a time helps reducing one addition= al > loop > > + and also reduced the cache misses during the double "looping" situat= ion. > > +- Customerizable dequeue count. Instead of dequeue maximum possible > > operations > > + as same as ``rte_cryptodev_dequeue_burst`` operation, the service > allows the > > + user to provide a callback function to decide how many operations to= be > > + dequeued. This is especially helpful when the expected dequeue count > is > > + hidden inside the opaque data stored during enqueue. The user can > provide > > + the callback function to parse the opaque data structure. > > +- Abandon enqueue and dequeue anytime. One of the drawbacks of > > + ``rte_cryptodev_enqueue_burst`` and ``rte_cryptodev_dequeue_burst`` > > + operations are: once an operation is enqueued/dequeued there is no > way to > > + undo the operation. The service make the operation abandon possible > by > > + creating a local copy of the queue operation data in the service con= text > > + data. The data will be written back to driver maintained operation d= ata > > + when enqueue or dequeue done function is called. > > + >=20 > The language in the above test need to be re-written. Some sentences does > not > Make complete sense and has grammatical errors. >=20 > I suggest to have an internal review within Intel first before sending th= e next > version. >=20 > > +The cryptodev data-path service uses > > + > > +Cryptodev PMDs who supports this feature will have > > +``RTE_CRYPTODEV_FF_SYM_HW_DIRECT_API`` feature flag presented. To > use >=20 > RTE_CRYPTODEV_FF_SYM_HW_RAW_DP looks better. >=20 > > this > > +feature the function ``rte_cryptodev_get_dp_service_ctx_data_size`` > should >=20 > Can be renamed as rte_cryptodev_get_raw_dp_ctx_size >=20 > > +be called to get the data path service context data size. The user sho= uld > > +creates a local buffer at least this size long and initialize it using >=20 > The user should create a local buffer of at least this size and initializ= e it using >=20 > > +``rte_cryptodev_dp_configure_service`` function call. >=20 > rte_cryptodev_raw_dp_configure or rte_cryptodev_configure _raw_dp can > be used here. >=20 > > + > > +The ``rte_cryptodev_dp_configure_service`` function call initialize or > > +updates the ``struct rte_crypto_dp_service_ctx`` buffer, in which cont= ains > the > > +driver specific queue pair data pointer and service context buffer, an= d a > > +set of function pointers to enqueue and dequeue different algorithms' > > +operations. The ``rte_cryptodev_dp_configure_service`` should be calle= d > when: > > + > > +- Before enqueuing or dequeuing starts (set ``is_update`` parameter to= 0). > > +- When different cryptodev session, security session, or session-less > xform > > + is used (set ``is_update`` parameter to 1). >=20 > The use of is_update is not clear with above text. IMO, we do not need th= is > flag. > Whenever an update is required, we change the session information and cal= l > the > Same API again and driver can copy all information blindly without checki= ng. >=20 The flag is very important - When flag is not set much queue specific data will be write into driver s= pecific Data field in the service data. This data is written back to the driver whe= n=20 "submit_done" or "dequeue_done" function is called. - When flag is set the above step is not executed, driver only updates the = function handlers attached the service data for different algorithm used. Will updat= e this into the description. > > + > > +Two different enqueue functions are provided. > > + > > +- ``rte_cryptodev_dp_sym_submit_vec``: submit a burst of operations > stored in > > + the ``rte_crypto_sym_vec`` structure. > > +- ``rte_cryptodev_dp_submit_single_job``: submit single operation. >=20 > What is the meaning of single job here? Can we use multiple buffers/vecto= rs > of same session > In a single job? Or we can submit only a single buffer/vector in a job? >=20 Same as CPU crypto, a sym vec will contains one or more jobs with same sess= ion. But it is not an inline function as it assumes we are bursting multiple ops= up. Single job - the sole purpose is to use it as an inline function that pushe= s one job into the queue, but not kicking the HW to start processing. When the raw APIs are used in external application/lib that also works in b= urst, such as VPP, single job submission is very useful to reduce the cycle cost.= Since they also have data structure translation to their specific data structure = and also work in burst, you don't want the application loops a burst of 32 ops to translate into a burst of DPDK crypto sym vec first, passing to the = driver, the driver then loops the jobs the second time to write to the HW one by one. U= se of inline "submit single" API can help reducing the cycles and cache misses, e= specially when the burst size is 256.=20 > > + > > +Either enqueue functions will not command the crypto device to start > > processing > > +until ``rte_cryptodev_dp_submit_done`` function is called. Before then > the user > > +shall expect the driver only stores the necessory context data in the > > +``rte_crypto_dp_service_ctx`` buffer for the next enqueue operation. I= f > the > > user > > +wants to abandon the submitted operations, simply call > > +``rte_cryptodev_dp_configure_service`` function instead with the > parameter > > +``is_update`` set to 0. The driver will recover the service context da= ta to > > +the previous state. >=20 > Can you explain a use case where this is actually being used? This looks = fancy > but > Do we have this type of requirement in any protocol stacks/specifications= ? > I believe it to be an extra burden on the application writer if it is not= a > protocol requirement. It is not protocol stack specific.=20 For now all crypto ops need to be translated into HW device/queue operation data in a looped manner. The context between last and current ops is stored= in the driver and is updated within enqueue burst function (e.g. for QAT such context is the shadow copy of the QAT queue tail/head). As mentioned earlier such burst operation introduce more cycles and cache Misses. So we want to introduce single job enqueue in DPDK so we don't loop the same burst twice. However we have to take care of the context between last and current submit single function. The idea is let the user allocate = the buffer for that. The "is_update" param is used to tell the driver if the it needs to write the context to the driver (is_update =3D=3D 0), or only upda= te context unrelated data inside dp_service buffer (updating function handler to the a= lgo etc). When the burst is processed the user can call submit_done function so the u= ser maintained context is written back to the driver. So in next burst enqueue = the user should call ``rte_cryptodev_dp_configure_service`` with "is_update" =3D 0 b= efore submit any jobs into the driver, or when the session is changed use "is_update" =3D 1 = to only update the function pointer. >=20 > > + > > +To dequeue the operations the user also have two operations: > > + > > +- ``rte_cryptodev_dp_sym_dequeue``: fully customizable deuqueue > operation. > > The > > + user needs to provide the callback function for the driver to get th= e > > + dequeue count and perform post processing such as write the status > field. > > +- ``rte_cryptodev_dp_sym_dequeue_single_job``: dequeue single job. > > + > > +Same as enqueue, the function ``rte_cryptodev_dp_dequeue_done`` is > used to > > +merge user's local service context data with the driver's queue operat= ion > > +data. Also to abandon the dequeue operation (still keep the operations= in > the > > +queue), the user shall avoid ``rte_cryptodev_dp_dequeue_done`` > function call > > +but calling ``rte_cryptodev_dp_configure_service`` function with the > parameter > > +``is_update`` set to 0. > > + > > +There are a few limitations to the data path service: > > + > > +* Only support in-place operations. > > +* APIs are NOT thread-safe. > > +* CANNOT mix the direct API's enqueue with > rte_cryptodev_enqueue_burst, or > > + vice versa. > > + > > +See *DPDK API Reference* for details on each API definitions. > > + > > Sample code > > ----------- > > > > diff --git a/doc/guides/rel_notes/release_20_11.rst > > b/doc/guides/rel_notes/release_20_11.rst > > index df227a177..159823345 100644 > > --- a/doc/guides/rel_notes/release_20_11.rst > > +++ b/doc/guides/rel_notes/release_20_11.rst > > @@ -55,6 +55,13 @@ New Features > > Also, make sure to start the actual text at the margin. > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D > > > > + * **Added data-path APIs for cryptodev library.** > > + > > + Cryptodev is added data-path APIs to accelerate external librarie= s or > > + applications those want to avail fast cryptodev enqueue/dequeue > > + operations but does not necessarily depends on mbufs and cryptode= v > > + operation mempool. > > + > > > > Removed Items > > ------------- >=20 >=20 > Regards, > Akhil