From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id EDA64A052A; Sat, 25 Jul 2020 08:59:02 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 798F51C025; Sat, 25 Jul 2020 08:59:01 +0200 (CEST) Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by dpdk.org (Postfix) with ESMTP id D3D0CE07 for ; Sat, 25 Jul 2020 08:58:59 +0200 (CEST) IronPort-SDR: Tg22a/pj/tg9epIVOD2N4AmsgRR9618Zy20DGfN89Ba4UWwKbLcBlKVcdwDuaerXkg8X0VXCPM DDppEGV2xUKw== X-IronPort-AV: E=McAfee;i="6000,8403,9692"; a="150808776" X-IronPort-AV: E=Sophos;i="5.75,392,1589266800"; d="scan'208";a="150808776" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Jul 2020 23:58:58 -0700 IronPort-SDR: +O32/3tY9eeslRHNxQ7SnK2I38F92qmXOwc2kW6R8LFEAARju4+3LXj9BvFjPi0Uv9SOYf8H/W uskqrgw1oPrA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.75,392,1589266800"; d="scan'208";a="488969918" Received: from orsmsx601.amr.corp.intel.com ([10.22.229.14]) by fmsmga006.fm.intel.com with ESMTP; 24 Jul 2020 23:58:58 -0700 Received: from orsmsx604.amr.corp.intel.com (10.22.229.17) by ORSMSX601.amr.corp.intel.com (10.22.229.14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Fri, 24 Jul 2020 23:58:57 -0700 Received: from ORSEDG002.ED.cps.intel.com (10.7.248.5) by orsmsx604.amr.corp.intel.com (10.22.229.17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.1713.5 via Frontend Transport; Fri, 24 Jul 2020 23:58:57 -0700 Received: from NAM04-BN3-obe.outbound.protection.outlook.com (104.47.46.56) by edgegateway.intel.com (134.134.137.101) with Microsoft SMTP Server (TLS) id 14.3.439.0; Fri, 24 Jul 2020 23:58:56 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jSRo/RaTm5ydoDpQbcu1px35XyBzxKxGjN07qSIJ4v9LHtA7IFHUhzTjpSzWEaqHP0z1WhtadrCF7TiS88ejChyn7xhjtZv13M3+SIf8JRlPLanw90JjDm9VIdxeHdW5mOTd+5E9L0zcxh4wBg3hmHtoC5Wgn6E6Pds9nJxclVHM0GINU0K1ecis8M8DdrOkU7z00xRvxElB+2i9t9P351GBjkNU+vBYDEG9teQSrfBhb/OcBkPshlnLl2h/FAz7W4hUKLlzItBfXddfN4LzXJIiHut2hQWrDh7zqKkDW/JFGiXv04hELXMhSO2SuL6IxroQtAcIOsBcZfCksrMniA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=yrz25Ap3U7zgiHO6NS62kY3z3t3Xs6HsEaJ8tZ6AHho=; b=Ci6a0Y0lMJ0yOAhjx9Nfm02gMIC8TsEsDL2vUDnEbrK7PgRPB0Z6Ye9gtKNphLz2zmIsisA0WufgDbc8vFBR/t/l/t9Zoo6PDsKVtUrtNELHJzFjXZZ1HKUkMiTyK/hnzSV6Dn+dbsCVQSPrYRyISsSji50EjGpoq7+gCKy/RVOEjDlSpAPpDz/lV+4lSZCr9wmTae0yf8BhyMfuWzWNEZPwIq5Zx+kAz2yF2FiGHCQlvhCQaGSjo6YhwvJJmOpIHj7Ftvl/Iyd/bt6B4WFoCQOOf1qugb98VMVDbOll8Y/PrtZ1XBPmrbWYH0u6AVZIWQPUHTSuwHkBJoCn9jl4cQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=yrz25Ap3U7zgiHO6NS62kY3z3t3Xs6HsEaJ8tZ6AHho=; b=TNTMKeGJXka9OxocRkp4v8yh6DcLsy+iK3Xa8p8rFX3y6wMthuRDN9argsj9uFfbTJ9uLzYnuHwRQoDi3lSfI39vrFAVavUcl+jHDdyu0WjAcOfcGGZZ9yE+Rd/wOYmrV1NIhCB7FchhJUWDo2oSODWyvwBTQqHvqEHxqqIZZ4I= Received: from MN2PR11MB4063.namprd11.prod.outlook.com (2603:10b6:208:13f::22) by MN2PR11MB3854.namprd11.prod.outlook.com (2603:10b6:208:f0::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3216.23; Sat, 25 Jul 2020 06:58:54 +0000 Received: from MN2PR11MB4063.namprd11.prod.outlook.com ([fe80::b898:36f5:61cb:42ca]) by MN2PR11MB4063.namprd11.prod.outlook.com ([fe80::b898:36f5:61cb:42ca%7]) with mapi id 15.20.3216.026; Sat, 25 Jul 2020 06:58:54 +0000 From: "Xia, Chenbo" To: "Fu, Patrick" , "dev@dpdk.org" , "maxime.coquelin@redhat.com" Thread-Topic: [PATCH v2 1/2] doc: update guides for vhost async APIs Thread-Index: AQHWYDl74saNmoRvQEmmoc45a8R8uKkX4RmQ Date: Sat, 25 Jul 2020 06:58:54 +0000 Message-ID: References: <20200722105741.3421255-1-patrick.fu@intel.com> <20200722150153.3422450-1-patrick.fu@intel.com> <20200722150153.3422450-2-patrick.fu@intel.com> In-Reply-To: <20200722150153.3422450-2-patrick.fu@intel.com> Accept-Language: en-US, zh-CN Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-reaction: no-action dlp-version: 11.2.0.6 authentication-results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=intel.com; x-originating-ip: [192.198.147.208] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 19cf4859-bdd4-40a1-d656-08d8306831f8 x-ms-traffictypediagnostic: MN2PR11MB3854: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:9508; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: hVA/UAGo+KknlEJE8fCfU0Ton2AV6qDnHBSWSR/ArNfZLXjfrrdNh+9EWPtlhW0/tRZdh3zMLUXP/Tp0eWe44gslZ12o+14lHd/qBDt1sijrHMKYNSrFkQnkGgJDbSyg4cSCjlVqC7D/ARe4yz8peB+WWcSYG/z68PWlL1TyNAx3zrEoOBUHheuuxsx7iFn5h+/dwcKz9pwgEcXPHQQY9Mg35p+9aBGemfJiwDuXzc8V9sOARTUBYWuP3iimHaLLutPC4azIRook5gBm56uPGekp88KFyNXwhkreOreB7r8hHJp6CDngBxAsFp+INQBrzPsfRYtgeU76WzjhCwVY4Q== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MN2PR11MB4063.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(346002)(39860400002)(366004)(136003)(376002)(396003)(316002)(66446008)(55016002)(9686003)(53546011)(26005)(76116006)(7696005)(83380400001)(6506007)(86362001)(66476007)(66946007)(64756008)(66556008)(71200400001)(186003)(5660300002)(52536014)(33656002)(8936002)(478600001)(15650500001)(2906002)(8676002)(110136005); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata: QTpFDlP1HDfBJyXO2h0iqjW85Mf7XLWdekG0kxX3Tk70hPxAeTuPjYIcNNz5D5Fpf8Sw1on8qTaYDH70fvwrHz4Vr/R4EPZnllyO83HfWhXgZrBngTUiu0TfnF6TY0QlYlHs6UjCnuWhFzOe34VhI2YeDrHx6Sh2DYZpmm80MTrph5dn+0eyPXkpoHcGNTQJ2wvG8ITsbGLACAEbIOr6VWzdRgG+9zcSQ+KYaojc4uos7XvfKDoVrn7EZ9fSc+38w9jOdLln0OakJGRZFDw92kZZcezTmnltcuRsokvs0O2B+TlsTbtsB8fu9teURBMAynosMLLVGjRyA+z79ApZgoa7/kAEelFVl77qD3BRpNzb8510gDrX4436KU5Yt0XZiSn0Mp85Wd23oioKEO5Wh6Vmfy7BNitPhhCPsQUAM/g7CwZLpkzrmUe9MGSbCY5c9NKDjMZrKY8UCTlmbJkQP9BQyZ7/sR5iXnhL394jerqOfvDNBokshBy3CnGIsDaU Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MN2PR11MB4063.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 19cf4859-bdd4-40a1-d656-08d8306831f8 X-MS-Exchange-CrossTenant-originalarrivaltime: 25 Jul 2020 06:58:54.1421 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: Wp7Vpm4Fq2svRcmzRx9adN27yC1jPGQ0Op6SyNcC3Q5lqwoV56FFxWI7n+p9jLdQDatbH7SxNZN5fQqkQqlQ3Q== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR11MB3854 X-OriginatorOrg: intel.com Subject: Re: [dpdk-dev] [PATCH v2 1/2] doc: update guides for vhost async APIs X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > -----Original Message----- > From: Fu, Patrick > Sent: Wednesday, July 22, 2020 11:02 PM > To: dev@dpdk.org; maxime.coquelin@redhat.com; Xia, Chenbo > > Cc: Fu, Patrick > Subject: [PATCH v2 1/2] doc: update guides for vhost async APIs >=20 > From: Patrick Fu >=20 > Update vhost guides to document vhost async APIs >=20 > Signed-off-by: Patrick Fu > --- > doc/guides/prog_guide/vhost_lib.rst | 86 ++++++++++++++++++++++++++--- > 1 file changed, 77 insertions(+), 9 deletions(-) >=20 > diff --git a/doc/guides/prog_guide/vhost_lib.rst > b/doc/guides/prog_guide/vhost_lib.rst > index db921f922..b892eec67 100644 > --- a/doc/guides/prog_guide/vhost_lib.rst > +++ b/doc/guides/prog_guide/vhost_lib.rst > @@ -147,6 +147,21 @@ The following is an overview of some key Vhost API > functions: >=20 > It is disabled by default. >=20 > + - ``RTE_VHOST_USER_ASYNC_COPY`` > + > + Asynchronous data path will be enabled when this flag is set. Async = data > + path allows applications to register async copy devices (typically > + hardware DMA channels) to the vhost queues. Vhost leverages the copy > + device registered to free CPU from memory copy operations. A set of > + async data path APIs are defined for DPDK applications to make use o= f > + the async capability. Only packets enqueued/dequeued by async APIs a= re > + processed through the async data path. > + > + Currently this feature is only implemented on split ring enqueue dat= a > + path. > + > + It is disabled by default. > + > * ``rte_vhost_driver_set_features(path, features)`` >=20 > This function sets the feature bits the vhost-user driver supports. Th= e @@ - > 235,6 +250,59 @@ The following is an overview of some key Vhost API > functions: >=20 > Enable or disable zero copy feature of the vhost crypto backend. >=20 > +* ``rte_vhost_async_channel_register(vid, queue_id, features, ops)`` > + > + Register a vhost queue with async copy device channel. > + Following device ``features`` must be specified together with the > + registration: > + > + * ``async_inorder`` > + > + Async copy device can guarantee the ordering of copy completion > + sequence. Copies are completed in the same order with that at > + the submission time. > + > + Currently, only ``async_inorder`` capable device is supported by vho= st. > + > + * ``async_threshold`` > + > + The copy length (in bytes) below which CPU copy will be used even if > + applications call async vhost APIs to enqueue/dequeue data. > + > + Typical value is 512~1024 depending on the async device capability. > + > + Applications must provide following ``ops`` callbacks for vhost lib > + to work with the async copy devices: > + > + * ``transfer_data(vid, queue_id, descs, opaque_data, count)`` > + > + vhost invokes this function to submit copy data to the async devices= . > + For non-async_inorder capable devices, ``opaque_data`` could be used > + for identifying the completed packets. > + > + * ``check_completed_copies(vid, queue_id, opaque_data, max_packets)`` > + > + vhost invokes this function to get the copy data completed by async > + devices. > + > +* ``rte_vhost_async_channel_unregister(vid, queue_id)`` > + > + Unregister the async copy device channel from a vhost queue. > + > +* ``rte_vhost_submit_enqueue_burst(vid, queue_id, pkts, count)`` > + > + Submit an enqueue request to transmit ``count`` packets from host to > + guest by async data path. Enqueue is not guaranteed to finish upon > + the return of this API call. > + > + Applications must not free the packets submitted for enqueue until > + the packets are completed. > + > +* ``rte_vhost_poll_enqueue_completed(vid, queue_id, pkts, count)`` > + > + Poll enqueue completion status from async data path. Completed > + packets are returned to applications through ``pkts``. > + > Vhost-user Implementations > -------------------------- >=20 > @@ -294,16 +362,16 @@ Guest memory requirement >=20 > * Memory pre-allocation >=20 > - For non-zerocopy, guest memory pre-allocation is not a must. This can = help > - save of memory. If users really want the guest memory to be pre-alloca= ted > - (e.g., for performance reason), we can add option ``-mem-prealloc`` wh= en > - starting QEMU. Or, we can lock all memory at vhost side which will for= ce > - memory to be allocated when mmap at vhost side; option --mlockall in > - ovs-dpdk is an example in hand. > + For non-zerocopy non-async data path, guest memory pre-allocation is > + not a must. This can help save of memory. If users really want the > + guest memory to be pre-allocated (e.g., for performance reason), we > + can add option ``-mem-prealloc`` when starting QEMU. Or, we can lock > + all memory at vhost side which will force memory to be allocated when > + mmap at vhost side; option --mlockall in ovs-dpdk is an example in han= d. >=20 > - For zerocopy, we force the VM memory to be pre-allocated at vhost lib = when > - mapping the guest memory; and also we need to lock the memory to preve= nt > - pages being swapped out to disk. > + For async and zerocopy data path, we force the VM memory to be > + pre-allocated at vhost lib when mapping the guest memory; and also we > + need to lock the memory to prevent pages being swapped out to disk. >=20 > * Memory sharing >=20 > -- > 2.18.4 Reviewed-by: Chenbo Xia