From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0C3C6A0526; Wed, 22 Jul 2020 17:06:31 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id ED9161BFBB; Wed, 22 Jul 2020 17:06:30 +0200 (CEST) Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by dpdk.org (Postfix) with ESMTP id 0F2E82B86 for ; Wed, 22 Jul 2020 17:06:29 +0200 (CEST) IronPort-SDR: n4gXOhUDQEZpkjVNw441IhbP5RYddwht4dqS22gA7ZF+nTXt+UXRw4IqBgO66woz3/1aCyYlos xccIvYJ6NVYg== X-IronPort-AV: E=McAfee;i="6000,8403,9689"; a="214978932" X-IronPort-AV: E=Sophos;i="5.75,383,1589266800"; d="scan'208";a="214978932" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Jul 2020 08:06:23 -0700 IronPort-SDR: kTZGeRva7GO7j5rd9y1l0T5MCYCUDa3GY+bFKGb370ZyAMeQbNwW+ioTJmOpYxwkXQxz3Cmz4n ayTET030vCMw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.75,383,1589266800"; d="scan'208";a="326713549" Received: from orsmsx102.amr.corp.intel.com ([10.22.225.129]) by FMSMGA003.fm.intel.com with ESMTP; 22 Jul 2020 08:06:23 -0700 Received: from orsmsx114.amr.corp.intel.com (10.22.240.10) by ORSMSX102.amr.corp.intel.com (10.22.225.129) with Microsoft SMTP Server (TLS) id 14.3.439.0; Wed, 22 Jul 2020 08:06:22 -0700 Received: from ORSEDG002.ED.cps.intel.com (10.7.248.5) by ORSMSX114.amr.corp.intel.com (10.22.240.10) with Microsoft SMTP Server (TLS) id 14.3.439.0; Wed, 22 Jul 2020 08:06:22 -0700 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (104.47.55.170) by edgegateway.intel.com (134.134.137.101) with Microsoft SMTP Server (TLS) id 14.3.439.0; Wed, 22 Jul 2020 08:06:22 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=kHcB0Dx2R0Vdm7QK3S6fY3JTakWfLhhQ60gXfroRqXyDOF5/+Zn3UPSphE8h/XxMtHzb42RuvwtDqMJHQ7qvYJWOZ23CMzP/pxNIp+iBCoYygcV8hjsC7VmYiBSjzTxBNHrF4zlSCfUEvGlBpac+UqWIcfiHhXsDDdEC9n1VxZQ4yzx8LdrTbrpuA3SFVa0upGsGGgCHMTNZBr50bjW4EuvOHZlgsPGZ7laeMqP/NoOyM5aah9fAHlbOFEQJiOn6b4nW8h0ML2ZlrqlWLkloBuEZDw8MAR0uP/NIHySc2gMTSh88qaxcsoj3lQk1wUG7UIcAEnPTUJIb9q6DeIbB/w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zEHMaZ0aPgbnhzBBmq7VAWfxI7miOayW7lTSzJcYaM4=; b=IgTsIJubnVeM7bWYKBJ1qKhb5NQo7iF8V+e5mpPKdE/U+eyUBH/XSxeF59EKt+9RtGCBKSUu6Ze3vZAwYxiYZEiw9gxPvRev6i+IpvXvDI5C+bnNL56dkJyX4XwUG7n+fmkRtJrHiqNbXC4Qe5/T5Jk3STOPJPtgUBiNeLvs5rHcajTZz3wlT7+hrqSXXpval2CLn+vQ6dhxrIbMhrTeTgnaq69yT1cFdi1KrmgK8PBzgfH4xiXx/i0j9WbYwcfZw+bNDoAGwrSSjy+RjtAYYJeGWUnk88EpD9G+ZZopH0GFtG1lMj2BszRJ56OcWSKpekjfiqSnG00Twv2VyuJLBA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zEHMaZ0aPgbnhzBBmq7VAWfxI7miOayW7lTSzJcYaM4=; b=ds09IH/LxuFDplt1RX9IU2dnF7Htm7tTtftG20gkn1mOlJrDPK9+Ou3uj6jwLSUXWpo5bdylFqJms4dDx6ugNNUCT0PbsoHi8mzLcLU3TqwEVZ5xtl5L6s6FglHL4WP7aV6BnrI8Pw18OBE/t+16hkJqBxXo9D/Cf6deQetWsVM= Received: from BYAPR11MB3735.namprd11.prod.outlook.com (2603:10b6:a03:b4::31) by BYAPR11MB3526.namprd11.prod.outlook.com (2603:10b6:a03:88::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3195.25; Wed, 22 Jul 2020 15:06:19 +0000 Received: from BYAPR11MB3735.namprd11.prod.outlook.com ([fe80::2571:24e3:140b:d78c]) by BYAPR11MB3735.namprd11.prod.outlook.com ([fe80::2571:24e3:140b:d78c%7]) with mapi id 15.20.3216.020; Wed, 22 Jul 2020 15:06:19 +0000 From: "Fu, Patrick" To: "Xia, Chenbo" , "dev@dpdk.org" , "maxime.coquelin@redhat.com" Thread-Topic: [PATCH v1 1/2] doc: update guides for vhost async APIs Thread-Index: AQHWYBdTwCMx+EEls0Wesfm3jrqaPakTdEyAgAA+OzA= Date: Wed, 22 Jul 2020 15:06:19 +0000 Message-ID: References: <20200722105741.3421255-1-patrick.fu@intel.com> <20200722105741.3421255-2-patrick.fu@intel.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-version: 11.2.0.6 dlp-product: dlpe-windows dlp-reaction: no-action authentication-results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=intel.com; x-originating-ip: [192.198.147.200] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 2382c6ad-7109-4367-e2e6-08d82e50ca4c x-ms-traffictypediagnostic: BYAPR11MB3526: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: lm25eh3FOrZnQE0QEH0x5uRDJtZlWodGBeIJMR+s6KVPGavWnUXoEpKXpqvVUktyuq8yOzJIeJdmH5Kw5z6ZnTKMk+yTTK8O4v0y2fKDiEWKFetppzXB6WWVXaLNOl6o88GyEpdIvFvbHbp+Y0cO8ZaN1MLFwao6z4uuVjI9hoEbieRNOc/vmOR3KUjJ+F3Vy1tKqhRCWAkBxkkAVCmPEAd8vDB0bvLyHygdZmM4Cpkx2WEJ7xRthdJRRVHuw6WeWBX4ZMrKqPjcMsv709/siOkpzshwtDXK0v+yTlk3BrWNKylBa08TFfsQCVyzW58bxOCQ9fl0NVpKuU2Ht6Rleg== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BYAPR11MB3735.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(366004)(376002)(346002)(396003)(39860400002)(136003)(53546011)(6506007)(26005)(478600001)(76116006)(186003)(7696005)(5660300002)(66476007)(66446008)(64756008)(316002)(66556008)(66946007)(52536014)(86362001)(8936002)(33656002)(110136005)(15650500001)(83380400001)(71200400001)(8676002)(2906002)(55016002)(9686003); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata: m+4M6gLrcxmaY0S4b3+FkrknvSSE9eN1cdJgWRxxWPn1f/lGRbSd0atJL+OSDQZWzipNXWQairYM5P3MC4+7JkuAVFSv3e03UW/wycIeUAckeyrlB6CMg317LYZXEtGFYj3R7A7fr/7764VKppp6u4RfFJuohIOTV1FNZotOSDZ2hPFUh9BaTC5FymJpzby77iz3Z0GpKtRiHyij6uIA5kzTsHj5RCpN0eAjRRqy5S80jZRZhnXNdUZgOrAZdRUkcxYKtzq8UQdB9LL94Yv/1urxaSdf36d+H6HWc75FhrHaQgX3zmMcYiaGKURtYY4Yi6o2Ogzuj+BmxbL+nYSaKHdGietKgtK4BNRElVKiHERQW0aZquX2mNky3P8VfbJMcEMGD5YvLQ4L3ckS2IjTurEmmD6KlHDCCGX7jcDOTrRjaJUgmqP8lY8JWkBwTTeAWuOJgZSDTFhKbpIZ4WzoXjWlX3lse3msinmr1uVCR+ylOQxttGX6gfNm2kqsncdz Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: BYAPR11MB3735.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2382c6ad-7109-4367-e2e6-08d82e50ca4c X-MS-Exchange-CrossTenant-originalarrivaltime: 22 Jul 2020 15:06:19.7294 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: jdFvg1+AiiIyYcADXYdYNFiq8vnsXRxQM/Ohy4qqNig0mMr7+OW1078oUdkt6I+5BvYZJixMVE1m1GjBVDq5nA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR11MB3526 X-OriginatorOrg: intel.com Subject: Re: [dpdk-dev] [PATCH v1 1/2] doc: update guides for vhost async APIs X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Archived-At: List-Archive: Thanks for comments. v2 patch sent with all the changes suggested. Thanks, Patrick > -----Original Message----- > From: Xia, Chenbo > Sent: Wednesday, July 22, 2020 7:21 PM > To: Fu, Patrick ; dev@dpdk.org; > maxime.coquelin@redhat.com > Subject: RE: [PATCH v1 1/2] doc: update guides for vhost async APIs >=20 > Hi Patrick, >=20 > > -----Original Message----- > > From: Fu, Patrick > > Sent: Wednesday, July 22, 2020 6:58 PM > > To: dev@dpdk.org; maxime.coquelin@redhat.com; Xia, Chenbo > > > > Cc: Fu, Patrick > > Subject: [PATCH v1 1/2] doc: update guides for vhost async APIs > > > > From: Patrick Fu > > > > Update vhost guides to document vhost async APIs > > > > Signed-off-by: Patrick Fu > > --- > > doc/guides/prog_guide/vhost_lib.rst | 86 > > ++++++++++++++++++++++++++--- > > 1 file changed, 77 insertions(+), 9 deletions(-) > > > > diff --git a/doc/guides/prog_guide/vhost_lib.rst > > b/doc/guides/prog_guide/vhost_lib.rst > > index db921f922..cce8b6ae7 100644 > > --- a/doc/guides/prog_guide/vhost_lib.rst > > +++ b/doc/guides/prog_guide/vhost_lib.rst > > @@ -147,6 +147,21 @@ The following is an overview of some key Vhost > > API > > functions: > > > > It is disabled by default. > > > > + - ``RTE_VHOST_USER_ASYNC_COPY`` > > + > > + Asynchronous data path will be enabled when this flag is set. Asyn= c > data > > + path allows applications to register async copy devices (typically > > + hardware DMA channels) to the vhost queues. Vhost leverages the co= py > > + device registered to offload CPU from memory copy operations. A > > + set of >=20 > You mean 'free' CPU from memory copy? >=20 > > + async data path APIs are defined for DPDK applications to make use= of > > + the async capability. Only packets enqueued/dequeued by async APIs > are > > + processed through the async data path. > > + > > + Currently this feature is only implemented on split ring enqueue d= ata > > + path. > > + > > + It is disabled by default. > > + > > * ``rte_vhost_driver_set_features(path, features)`` > > > > This function sets the feature bits the vhost-user driver supports. > > The @@ - > > 235,6 +250,59 @@ The following is an overview of some key Vhost API > > functions: > > > > Enable or disable zero copy feature of the vhost crypto backend. > > > > +* ``rte_vhost_async_channel_register(vid, queue_id, features, ops)`` > > + > > + Register a vhost queue with async copy device channel. > > + Following device ``features`` must be specified together with the > > + registration: > > + > > + * ``async_inorder`` > > + > > + Async copy device can guarantee the ordering of copy completion > > + sequence. Copies are completed in the same order with that at > > + the submission time. > > + > > + Currently, only ``async_inorder`` capable device is supported by v= host. > > + > > + * ``async_threshold`` > > + > > + The copy length (in bytes) below which CPU copy will be used even = if > > + applications call async vhost APIs to enqueue/dequeue data. > > + > > + Typical value is 512~1024 depending on the async device capability= . > > + > > + Applications must provide following ``ops`` callbacks for vhost lib > > + to work with the async copye devices: >=20 > s/copye/copy >=20 > > + > > + * ``transfer_data(vid, queue_id, descs, opaque_data, count)`` > > + > > + vhost invokes this function to submit copy data to the async devic= es. > > + For non-async_inorder capable devices, ``opaque_data`` could be us= ed > > + for identifying the completed packets. > > + > > + * ``check_completed_copies(vid, queue_id, opaque_data, > > + max_packets)`` > > + > > + vhost invokes this function to get the copy data completed by asyn= c > > + devices. > > + > > +* ``rte_vhost_async_channel_unregister(vid, queue_id)`` > > + > > + Unregister the async copy device from a vhost queue. >=20 > 'Copy device channel' may be more accurate? >=20 > > + > > +* ``rte_vhost_submit_enqueue_burst(vid, queue_id, pkts, count)`` > > + > > + Submit an enqueue request to transmit ``count`` packets from host > > + to guest by async data path. Enqueue is not guaranteed to finish > > + upon the return of this API call. > > + > > + Applications must not free the packets submitted for enqueue until > > + the packets are completed. > > + > > +* ``rte_vhost_poll_enqueue_completed(vid, queue_id, pkts, count)`` > > + > > + Poll enqueue completion status from async data path. Completed > > + packets are returned to applications through ``pkts``. > > + > > Vhost-user Implementations > > -------------------------- > > > > @@ -294,16 +362,16 @@ Guest memory requirement > > > > * Memory pre-allocation > > > > - For non-zerocopy, guest memory pre-allocation is not a must. This > > can help > > - save of memory. If users really want the guest memory to be > > pre-allocated > > - (e.g., for performance reason), we can add option ``-mem-prealloc`` > > when > > - starting QEMU. Or, we can lock all memory at vhost side which will > > force > > - memory to be allocated when mmap at vhost side; option --mlockall > > in > > - ovs-dpdk is an example in hand. > > + For non-zerocopy non-async data path, guest memory pre-allocation > > + is not a must. This can help save of memory. If users really want > > + the guest memory to be pre-allocated (e.g., for performance > > + reason), we can add option ``-mem-prealloc`` when starting QEMU. > > + Or, we can lock all memory at vhost side which will force memory to > > + be allocated when mmap at vhost side; option --mlockall in ovs-dpdk = is > an example in hand. > > > > - For zerocopy, we force the VM memory to be pre-allocated at vhost > > lib when > > - mapping the guest memory; and also we need to lock the memory to > > prevent > > - pages being swapped out to disk. > > + For async data path or zerocopy, we force the VM memory to be >=20 > 'For async or zerocopy data path' may be better? >=20 > Thanks! > Chenbo >=20 > > + pre-allocated at vhost lib when mapping the guest memory; and also > > + we need to lock the memory to prevent pages being swapped out to dis= k. > > > > * Memory sharing > > > > -- > > 2.18.4