From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 60A86A0526; Wed, 22 Jul 2020 13:21:40 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 9DAFB1C012; Wed, 22 Jul 2020 13:21:38 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by dpdk.org (Postfix) with ESMTP id E4D1F1BFFF for ; Wed, 22 Jul 2020 13:21:36 +0200 (CEST) IronPort-SDR: b6DiYD6nHMAqVXQAmS5t9QPWTIjZxloa5XhJuUX5wsl0k7mIK6FiZxoxnTOiMe6+yhViuvtAvb a0yIytppLNNg== X-IronPort-AV: E=McAfee;i="6000,8403,9689"; a="151634973" X-IronPort-AV: E=Sophos;i="5.75,381,1589266800"; d="scan'208";a="151634973" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Jul 2020 04:21:34 -0700 IronPort-SDR: sL9OZvUVlENm3Xf0dGI1IOcWnZA/WKAUOL6z482D6zSIjrwcZIRqs3Tpqn2lLSetjh+4bXlG5Q tZWsWPVyMp0A== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.75,381,1589266800"; d="scan'208";a="318637305" Received: from orsmsx106.amr.corp.intel.com ([10.22.225.133]) by orsmga008.jf.intel.com with ESMTP; 22 Jul 2020 04:21:34 -0700 Received: from orsmsx122.amr.corp.intel.com (10.22.225.227) by ORSMSX106.amr.corp.intel.com (10.22.225.133) with Microsoft SMTP Server (TLS) id 14.3.439.0; Wed, 22 Jul 2020 04:21:34 -0700 Received: from ORSEDG001.ED.cps.intel.com (10.7.248.4) by ORSMSX122.amr.corp.intel.com (10.22.225.227) with Microsoft SMTP Server (TLS) id 14.3.439.0; Wed, 22 Jul 2020 04:21:34 -0700 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (104.47.55.108) by edgegateway.intel.com (134.134.137.100) with Microsoft SMTP Server (TLS) id 14.3.439.0; Wed, 22 Jul 2020 04:21:29 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=MfoYW2zJiNDmbzD0s18po4MhXvnjnYcH/7xvoL7SgC5FNYc74kmEyMmQu673NzDRXopZ1LU9x6ScReBcRu8yfK19OR5wBuWtrtSIv2xQtjJxfFnp7QuQjitbRIJqqrA6rgKzMIExgr7xa5cSe7nD4KkndUhHLEdhYIXUBx/yfUXrKBhK81ogD2dkLsvwr6g7Xbyzj8P1FPAeKqm3HDtqely114uGSThjvHRmPm1PA9mSpV4gJLB91Xm6Kv80EkZNE0npHybcLeA2skCybWdtvnEmKbHH5sNy6OIMCzzcXRB/sdQTa8KSEIDZzZ/O+51HWKp2593loOBk3IVa9m9Dsw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=JOq0ftvLzUxEv+k6xAQB+LP4RxMUJcxCNDCab6N2Agw=; b=mB/andX0IHGWwoP9PqxJ28H/tzfPyQAl/zK8sRiPRyExC3Cn6TdIjl2L4YcDFLW+UKGlLZdxffOcKMCl4lMlKm/Ss4bFWJZpdQqnFXedWwMMRSaPtbnvPV4GCugsCEbjQO6ysj7wEctFBd6WKL/hqBy1yR2IoxaZRD18HqOoAWkxBqvUYsmq4cTfIsvyGNUheFTGPeg+fSPUFeWCV9MdzqTaa6snCZhgehYqI505AvLPOgboyi3FyF3URXkadNcFgiiyLaCNYFYWb4nhfxHX62WjAnBvSljOOjS3mNoJ+U7sVoBIB+q+53phYqo/RZCk7R7ZREA2+UmVi7JCOwoWEg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=JOq0ftvLzUxEv+k6xAQB+LP4RxMUJcxCNDCab6N2Agw=; b=cLUsU8mEs2F12D3+3kSYR5447q8v0T1rqu8CRV32Twvqvei2nXFp1sWr7UaFlU0NaitnwPDYA9LuWFoaH0CG/jJA4Yl0LdArVr+mSsR7ijY9LpLTAx3QulRKzKSlZCUL3ARQihazBf93IS6Dmg+VQRK4HpW0wGboPTILoe0uBLU= Received: from MN2PR11MB4063.namprd11.prod.outlook.com (2603:10b6:208:13f::22) by MN2PR11MB3600.namprd11.prod.outlook.com (2603:10b6:208:fa::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3216.23; Wed, 22 Jul 2020 11:21:27 +0000 Received: from MN2PR11MB4063.namprd11.prod.outlook.com ([fe80::7cde:8326:5010:c47e]) by MN2PR11MB4063.namprd11.prod.outlook.com ([fe80::7cde:8326:5010:c47e%7]) with mapi id 15.20.3195.024; Wed, 22 Jul 2020 11:21:27 +0000 From: "Xia, Chenbo" To: "Fu, Patrick" , "dev@dpdk.org" , "maxime.coquelin@redhat.com" Thread-Topic: [PATCH v1 1/2] doc: update guides for vhost async APIs Thread-Index: AQHWYBdT226mcYFOV0WzNGnqdhGyRqkTchGQ Date: Wed, 22 Jul 2020 11:21:27 +0000 Message-ID: References: <20200722105741.3421255-1-patrick.fu@intel.com> <20200722105741.3421255-2-patrick.fu@intel.com> In-Reply-To: <20200722105741.3421255-2-patrick.fu@intel.com> Accept-Language: en-US, zh-CN Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=intel.com; x-originating-ip: [192.102.204.36] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 66f3bd11-b590-480e-beed-08d82e31601b x-ms-traffictypediagnostic: MN2PR11MB3600: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: ZM90CoqMdEp2pobeXwiKtvA1W3RiPBaMPTNPBjFthti5aufgrIQF27O67d45Rg5dKHAnNuodrBT9iBU9TAuP31821w7hR+OBEptIeuHvTAw3ARiKkElnmENjqZ0BUcMJ/0AOkq1CWyHFQ+ziscLqmcTcJy88CfNDntKqCPPz1SmUQcUXnSFECD/Df5YziFICvaKIO6craqcEcEYHwprHzeVlcVshn8RRjvrQKp52PnklxzYgA500yMshklN9B0zpu8GEHg+J3wj0Q7YssuGIQs2yco/DaYl39jSqqNTloPML/z5jTQW8HKvrpNNFz/5Gv0hyC51TsevTtEKQme+cEA== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MN2PR11MB4063.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(396003)(39860400002)(136003)(366004)(346002)(376002)(66476007)(64756008)(66556008)(76116006)(66446008)(66946007)(26005)(186003)(316002)(5660300002)(9686003)(52536014)(110136005)(55016002)(8676002)(86362001)(8936002)(478600001)(2906002)(15650500001)(53546011)(7696005)(71200400001)(6506007)(33656002)(83380400001); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata: UhjmmRgA7JTovu3v7vd0ZkUvo/y76q/jZby8+NmF5C+yjt7eMCoImpiZoa5hRYu9ChzhRbef7zAQmctM7+CY6SFNom/t52YwE8yk96uAbufA9D1Wxi8mMnPXiirRLEcKZaGWCpsjer/i2vxBkb98nk0xsGp9ZRk1HbGqanZVxwT8g18HZO/qpklXwSVRSwuo4bC6X+tVxIU9P0ZD85336woWlOCiokJxfUaY93QK0N2qSfdWpxZzEawkijfJu7YU9xNALvqDqg/WYmrvwGwFOJLNH3BdYv01R4PJPGaOylRVEbR+3J6HDGhu7NqhWmF/EIStXx0p8tRMyeioPRC6T/CD8scyqiWRIsO4cTNruKQ+3oN6ozbm4irRd0mV/EIYe0GCeRb2GXbYCBZ3xfHTGLZqZL6Cc8sANzh3g+CqpZK0VrOZCeHOoShlYsK7l4niiUDni50kh1MuqKFT7/CR9Jx2ZbrMRiKRACX9NtMUHNa3+vlXH81ffpjYHanAf6wU Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MN2PR11MB4063.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 66f3bd11-b590-480e-beed-08d82e31601b X-MS-Exchange-CrossTenant-originalarrivaltime: 22 Jul 2020 11:21:27.2811 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: glAldbEa0bhxbnNOa42YjoWiI+JcUcFNXXyFFzz0NPwFlKzPWsTwADhrQeZ1qCZKYhDm0fxxNWMTueDfxjNgPA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR11MB3600 X-OriginatorOrg: intel.com Subject: Re: [dpdk-dev] [PATCH v1 1/2] doc: update guides for vhost async APIs X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Patrick, > -----Original Message----- > From: Fu, Patrick > Sent: Wednesday, July 22, 2020 6:58 PM > To: dev@dpdk.org; maxime.coquelin@redhat.com; Xia, Chenbo > > Cc: Fu, Patrick > Subject: [PATCH v1 1/2] doc: update guides for vhost async APIs >=20 > From: Patrick Fu >=20 > Update vhost guides to document vhost async APIs >=20 > Signed-off-by: Patrick Fu > --- > doc/guides/prog_guide/vhost_lib.rst | 86 ++++++++++++++++++++++++++--- > 1 file changed, 77 insertions(+), 9 deletions(-) >=20 > diff --git a/doc/guides/prog_guide/vhost_lib.rst > b/doc/guides/prog_guide/vhost_lib.rst > index db921f922..cce8b6ae7 100644 > --- a/doc/guides/prog_guide/vhost_lib.rst > +++ b/doc/guides/prog_guide/vhost_lib.rst > @@ -147,6 +147,21 @@ The following is an overview of some key Vhost API > functions: >=20 > It is disabled by default. >=20 > + - ``RTE_VHOST_USER_ASYNC_COPY`` > + > + Asynchronous data path will be enabled when this flag is set. Async = data > + path allows applications to register async copy devices (typically > + hardware DMA channels) to the vhost queues. Vhost leverages the copy > + device registered to offload CPU from memory copy operations. A set = of You mean 'free' CPU from memory copy? > + async data path APIs are defined for DPDK applications to make use o= f > + the async capability. Only packets enqueued/dequeued by async APIs a= re > + processed through the async data path. > + > + Currently this feature is only implemented on split ring enqueue dat= a > + path. > + > + It is disabled by default. > + > * ``rte_vhost_driver_set_features(path, features)`` >=20 > This function sets the feature bits the vhost-user driver supports. Th= e @@ - > 235,6 +250,59 @@ The following is an overview of some key Vhost API > functions: >=20 > Enable or disable zero copy feature of the vhost crypto backend. >=20 > +* ``rte_vhost_async_channel_register(vid, queue_id, features, ops)`` > + > + Register a vhost queue with async copy device channel. > + Following device ``features`` must be specified together with the > + registration: > + > + * ``async_inorder`` > + > + Async copy device can guarantee the ordering of copy completion > + sequence. Copies are completed in the same order with that at > + the submission time. > + > + Currently, only ``async_inorder`` capable device is supported by vho= st. > + > + * ``async_threshold`` > + > + The copy length (in bytes) below which CPU copy will be used even if > + applications call async vhost APIs to enqueue/dequeue data. > + > + Typical value is 512~1024 depending on the async device capability. > + > + Applications must provide following ``ops`` callbacks for vhost lib > + to work with the async copye devices: s/copye/copy > + > + * ``transfer_data(vid, queue_id, descs, opaque_data, count)`` > + > + vhost invokes this function to submit copy data to the async devices= . > + For non-async_inorder capable devices, ``opaque_data`` could be used > + for identifying the completed packets. > + > + * ``check_completed_copies(vid, queue_id, opaque_data, max_packets)`` > + > + vhost invokes this function to get the copy data completed by async > + devices. > + > +* ``rte_vhost_async_channel_unregister(vid, queue_id)`` > + > + Unregister the async copy device from a vhost queue. 'Copy device channel' may be more accurate?=20 > + > +* ``rte_vhost_submit_enqueue_burst(vid, queue_id, pkts, count)`` > + > + Submit an enqueue request to transmit ``count`` packets from host to > + guest by async data path. Enqueue is not guaranteed to finish upon > + the return of this API call. > + > + Applications must not free the packets submitted for enqueue until > + the packets are completed. > + > +* ``rte_vhost_poll_enqueue_completed(vid, queue_id, pkts, count)`` > + > + Poll enqueue completion status from async data path. Completed > + packets are returned to applications through ``pkts``. > + > Vhost-user Implementations > -------------------------- >=20 > @@ -294,16 +362,16 @@ Guest memory requirement >=20 > * Memory pre-allocation >=20 > - For non-zerocopy, guest memory pre-allocation is not a must. This can = help > - save of memory. If users really want the guest memory to be pre-alloca= ted > - (e.g., for performance reason), we can add option ``-mem-prealloc`` wh= en > - starting QEMU. Or, we can lock all memory at vhost side which will for= ce > - memory to be allocated when mmap at vhost side; option --mlockall in > - ovs-dpdk is an example in hand. > + For non-zerocopy non-async data path, guest memory pre-allocation is > + not a must. This can help save of memory. If users really want the > + guest memory to be pre-allocated (e.g., for performance reason), we > + can add option ``-mem-prealloc`` when starting QEMU. Or, we can lock > + all memory at vhost side which will force memory to be allocated when > + mmap at vhost side; option --mlockall in ovs-dpdk is an example in han= d. >=20 > - For zerocopy, we force the VM memory to be pre-allocated at vhost lib = when > - mapping the guest memory; and also we need to lock the memory to preve= nt > - pages being swapped out to disk. > + For async data path or zerocopy, we force the VM memory to be 'For async or zerocopy data path' may be better? Thanks! Chenbo > + pre-allocated at vhost lib when mapping the guest memory; and also we > + need to lock the memory to prevent pages being swapped out to disk. >=20 > * Memory sharing >=20 > -- > 2.18.4