From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from dpdk.org (dpdk.org [92.243.14.124])
	by inbox.dpdk.org (Postfix) with ESMTP id 0C3C6A0526;
	Wed, 22 Jul 2020 17:06:31 +0200 (CEST)
Received: from [92.243.14.124] (localhost [127.0.0.1])
	by dpdk.org (Postfix) with ESMTP id ED9161BFBB;
	Wed, 22 Jul 2020 17:06:30 +0200 (CEST)
Received: from mga07.intel.com (mga07.intel.com [134.134.136.100])
 by dpdk.org (Postfix) with ESMTP id 0F2E82B86
 for <dev@dpdk.org>; Wed, 22 Jul 2020 17:06:29 +0200 (CEST)
IronPort-SDR: n4gXOhUDQEZpkjVNw441IhbP5RYddwht4dqS22gA7ZF+nTXt+UXRw4IqBgO66woz3/1aCyYlos
 xccIvYJ6NVYg==
X-IronPort-AV: E=McAfee;i="6000,8403,9689"; a="214978932"
X-IronPort-AV: E=Sophos;i="5.75,383,1589266800"; d="scan'208";a="214978932"
X-Amp-Result: SKIPPED(no attachment in message)
X-Amp-File-Uploaded: False
Received: from fmsmga003.fm.intel.com ([10.253.24.29])
 by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;
 22 Jul 2020 08:06:23 -0700
IronPort-SDR: kTZGeRva7GO7j5rd9y1l0T5MCYCUDa3GY+bFKGb370ZyAMeQbNwW+ioTJmOpYxwkXQxz3Cmz4n
 ayTET030vCMw==
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="5.75,383,1589266800"; d="scan'208";a="326713549"
Received: from orsmsx102.amr.corp.intel.com ([10.22.225.129])
 by FMSMGA003.fm.intel.com with ESMTP; 22 Jul 2020 08:06:23 -0700
Received: from orsmsx114.amr.corp.intel.com (10.22.240.10) by
 ORSMSX102.amr.corp.intel.com (10.22.225.129) with Microsoft SMTP Server (TLS)
 id 14.3.439.0; Wed, 22 Jul 2020 08:06:22 -0700
Received: from ORSEDG002.ED.cps.intel.com (10.7.248.5) by
 ORSMSX114.amr.corp.intel.com (10.22.240.10) with Microsoft SMTP Server (TLS)
 id 14.3.439.0; Wed, 22 Jul 2020 08:06:22 -0700
Received: from NAM12-BN8-obe.outbound.protection.outlook.com (104.47.55.170)
 by edgegateway.intel.com (134.134.137.101) with Microsoft SMTP Server (TLS)
 id 14.3.439.0; Wed, 22 Jul 2020 08:06:22 -0700
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=kHcB0Dx2R0Vdm7QK3S6fY3JTakWfLhhQ60gXfroRqXyDOF5/+Zn3UPSphE8h/XxMtHzb42RuvwtDqMJHQ7qvYJWOZ23CMzP/pxNIp+iBCoYygcV8hjsC7VmYiBSjzTxBNHrF4zlSCfUEvGlBpac+UqWIcfiHhXsDDdEC9n1VxZQ4yzx8LdrTbrpuA3SFVa0upGsGGgCHMTNZBr50bjW4EuvOHZlgsPGZ7laeMqP/NoOyM5aah9fAHlbOFEQJiOn6b4nW8h0ML2ZlrqlWLkloBuEZDw8MAR0uP/NIHySc2gMTSh88qaxcsoj3lQk1wUG7UIcAEnPTUJIb9q6DeIbB/w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; 
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=zEHMaZ0aPgbnhzBBmq7VAWfxI7miOayW7lTSzJcYaM4=;
 b=IgTsIJubnVeM7bWYKBJ1qKhb5NQo7iF8V+e5mpPKdE/U+eyUBH/XSxeF59EKt+9RtGCBKSUu6Ze3vZAwYxiYZEiw9gxPvRev6i+IpvXvDI5C+bnNL56dkJyX4XwUG7n+fmkRtJrHiqNbXC4Qe5/T5Jk3STOPJPtgUBiNeLvs5rHcajTZz3wlT7+hrqSXXpval2CLn+vQ6dhxrIbMhrTeTgnaq69yT1cFdi1KrmgK8PBzgfH4xiXx/i0j9WbYwcfZw+bNDoAGwrSSjy+RjtAYYJeGWUnk88EpD9G+ZZopH0GFtG1lMj2BszRJ56OcWSKpekjfiqSnG00Twv2VyuJLBA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com;
 dkim=pass header.d=intel.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; 
 s=selector2-intel-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=zEHMaZ0aPgbnhzBBmq7VAWfxI7miOayW7lTSzJcYaM4=;
 b=ds09IH/LxuFDplt1RX9IU2dnF7Htm7tTtftG20gkn1mOlJrDPK9+Ou3uj6jwLSUXWpo5bdylFqJms4dDx6ugNNUCT0PbsoHi8mzLcLU3TqwEVZ5xtl5L6s6FglHL4WP7aV6BnrI8Pw18OBE/t+16hkJqBxXo9D/Cf6deQetWsVM=
Received: from BYAPR11MB3735.namprd11.prod.outlook.com (2603:10b6:a03:b4::31)
 by BYAPR11MB3526.namprd11.prod.outlook.com (2603:10b6:a03:88::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3195.25; Wed, 22 Jul
 2020 15:06:19 +0000
Received: from BYAPR11MB3735.namprd11.prod.outlook.com
 ([fe80::2571:24e3:140b:d78c]) by BYAPR11MB3735.namprd11.prod.outlook.com
 ([fe80::2571:24e3:140b:d78c%7]) with mapi id 15.20.3216.020; Wed, 22 Jul 2020
 15:06:19 +0000
From: "Fu, Patrick" <patrick.fu@intel.com>
To: "Xia, Chenbo" <chenbo.xia@intel.com>, "dev@dpdk.org" <dev@dpdk.org>,
 "maxime.coquelin@redhat.com" <maxime.coquelin@redhat.com>
Thread-Topic: [PATCH v1 1/2] doc: update guides for vhost async APIs
Thread-Index: AQHWYBdTwCMx+EEls0Wesfm3jrqaPakTdEyAgAA+OzA=
Date: Wed, 22 Jul 2020 15:06:19 +0000
Message-ID: <BYAPR11MB37357C2A1B323B4F2041017A84790@BYAPR11MB3735.namprd11.prod.outlook.com>
References: <20200722105741.3421255-1-patrick.fu@intel.com>
 <20200722105741.3421255-2-patrick.fu@intel.com>
 <MN2PR11MB406305F1F8A75DFA46A3F3C79C790@MN2PR11MB4063.namprd11.prod.outlook.com>
In-Reply-To: <MN2PR11MB406305F1F8A75DFA46A3F3C79C790@MN2PR11MB4063.namprd11.prod.outlook.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
dlp-version: 11.2.0.6
dlp-product: dlpe-windows
dlp-reaction: no-action
authentication-results: intel.com; dkim=none (message not signed)
 header.d=none;intel.com; dmarc=none action=none header.from=intel.com;
x-originating-ip: [192.198.147.200]
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 2382c6ad-7109-4367-e2e6-08d82e50ca4c
x-ms-traffictypediagnostic: BYAPR11MB3526:
x-ms-exchange-transport-forked: True
x-microsoft-antispam-prvs: <BYAPR11MB3526BB1EC850D6AB6737C9C884790@BYAPR11MB3526.namprd11.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: lm25eh3FOrZnQE0QEH0x5uRDJtZlWodGBeIJMR+s6KVPGavWnUXoEpKXpqvVUktyuq8yOzJIeJdmH5Kw5z6ZnTKMk+yTTK8O4v0y2fKDiEWKFetppzXB6WWVXaLNOl6o88GyEpdIvFvbHbp+Y0cO8ZaN1MLFwao6z4uuVjI9hoEbieRNOc/vmOR3KUjJ+F3Vy1tKqhRCWAkBxkkAVCmPEAd8vDB0bvLyHygdZmM4Cpkx2WEJ7xRthdJRRVHuw6WeWBX4ZMrKqPjcMsv709/siOkpzshwtDXK0v+yTlk3BrWNKylBa08TFfsQCVyzW58bxOCQ9fl0NVpKuU2Ht6Rleg==
x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:;
 IPV:NLI; SFV:NSPM; H:BYAPR11MB3735.namprd11.prod.outlook.com; PTR:; CAT:NONE;
 SFTY:;
 SFS:(4636009)(366004)(376002)(346002)(396003)(39860400002)(136003)(53546011)(6506007)(26005)(478600001)(76116006)(186003)(7696005)(5660300002)(66476007)(66446008)(64756008)(316002)(66556008)(66946007)(52536014)(86362001)(8936002)(33656002)(110136005)(15650500001)(83380400001)(71200400001)(8676002)(2906002)(55016002)(9686003);
 DIR:OUT; SFP:1102; 
x-ms-exchange-antispam-messagedata: m+4M6gLrcxmaY0S4b3+FkrknvSSE9eN1cdJgWRxxWPn1f/lGRbSd0atJL+OSDQZWzipNXWQairYM5P3MC4+7JkuAVFSv3e03UW/wycIeUAckeyrlB6CMg317LYZXEtGFYj3R7A7fr/7764VKppp6u4RfFJuohIOTV1FNZotOSDZ2hPFUh9BaTC5FymJpzby77iz3Z0GpKtRiHyij6uIA5kzTsHj5RCpN0eAjRRqy5S80jZRZhnXNdUZgOrAZdRUkcxYKtzq8UQdB9LL94Yv/1urxaSdf36d+H6HWc75FhrHaQgX3zmMcYiaGKURtYY4Yi6o2Ogzuj+BmxbL+nYSaKHdGietKgtK4BNRElVKiHERQW0aZquX2mNky3P8VfbJMcEMGD5YvLQ4L3ckS2IjTurEmmD6KlHDCCGX7jcDOTrRjaJUgmqP8lY8JWkBwTTeAWuOJgZSDTFhKbpIZ4WzoXjWlX3lse3msinmr1uVCR+ylOQxttGX6gfNm2kqsncdz
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BYAPR11MB3735.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 2382c6ad-7109-4367-e2e6-08d82e50ca4c
X-MS-Exchange-CrossTenant-originalarrivaltime: 22 Jul 2020 15:06:19.7294 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: jdFvg1+AiiIyYcADXYdYNFiq8vnsXRxQM/Ohy4qqNig0mMr7+OW1078oUdkt6I+5BvYZJixMVE1m1GjBVDq5nA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR11MB3526
X-OriginatorOrg: intel.com
Subject: Re: [dpdk-dev] [PATCH v1 1/2] doc: update guides for vhost async
	APIs
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Sender: "dev" <dev-bounces@dpdk.org>

Thanks for comments. v2 patch sent with all the changes suggested.

Thanks,

Patrick


> -----Original Message-----
> From: Xia, Chenbo <chenbo.xia@intel.com>
> Sent: Wednesday, July 22, 2020 7:21 PM
> To: Fu, Patrick <patrick.fu@intel.com>; dev@dpdk.org;
> maxime.coquelin@redhat.com
> Subject: RE: [PATCH v1 1/2] doc: update guides for vhost async APIs
>=20
> Hi Patrick,
>=20
> > -----Original Message-----
> > From: Fu, Patrick <patrick.fu@intel.com>
> > Sent: Wednesday, July 22, 2020 6:58 PM
> > To: dev@dpdk.org; maxime.coquelin@redhat.com; Xia, Chenbo
> > <chenbo.xia@intel.com>
> > Cc: Fu, Patrick <patrick.fu@intel.com>
> > Subject: [PATCH v1 1/2] doc: update guides for vhost async APIs
> >
> > From: Patrick Fu <patrick.fu@intel.com>
> >
> > Update vhost guides to document vhost async APIs
> >
> > Signed-off-by: Patrick Fu <patrick.fu@intel.com>
> > ---
> >  doc/guides/prog_guide/vhost_lib.rst | 86
> > ++++++++++++++++++++++++++---
> >  1 file changed, 77 insertions(+), 9 deletions(-)
> >
> > diff --git a/doc/guides/prog_guide/vhost_lib.rst
> > b/doc/guides/prog_guide/vhost_lib.rst
> > index db921f922..cce8b6ae7 100644
> > --- a/doc/guides/prog_guide/vhost_lib.rst
> > +++ b/doc/guides/prog_guide/vhost_lib.rst
> > @@ -147,6 +147,21 @@ The following is an overview of some key Vhost
> > API
> > functions:
> >
> >      It is disabled by default.
> >
> > +  - ``RTE_VHOST_USER_ASYNC_COPY``
> > +
> > +    Asynchronous data path will be enabled when this flag is set. Asyn=
c
> data
> > +    path allows applications to register async copy devices (typically
> > +    hardware DMA channels) to the vhost queues. Vhost leverages the co=
py
> > +    device registered to offload CPU from memory copy operations. A
> > + set of
>=20
> You mean 'free' CPU from memory copy?
>=20
> > +    async data path APIs are defined for DPDK applications to make use=
 of
> > +    the async capability. Only packets enqueued/dequeued by async APIs
> are
> > +    processed through the async data path.
> > +
> > +    Currently this feature is only implemented on split ring enqueue d=
ata
> > +    path.
> > +
> > +    It is disabled by default.
> > +
> >  * ``rte_vhost_driver_set_features(path, features)``
> >
> >    This function sets the feature bits the vhost-user driver supports.
> > The @@ -
> > 235,6 +250,59 @@ The following is an overview of some key Vhost API
> > functions:
> >
> >    Enable or disable zero copy feature of the vhost crypto backend.
> >
> > +* ``rte_vhost_async_channel_register(vid, queue_id, features, ops)``
> > +
> > +  Register a vhost queue with async copy device channel.
> > +  Following device ``features`` must be specified together with the
> > +  registration:
> > +
> > +  * ``async_inorder``
> > +
> > +    Async copy device can guarantee the ordering of copy completion
> > +    sequence. Copies are completed in the same order with that at
> > +    the submission time.
> > +
> > +    Currently, only ``async_inorder`` capable device is supported by v=
host.
> > +
> > +  * ``async_threshold``
> > +
> > +    The copy length (in bytes) below which CPU copy will be used even =
if
> > +    applications call async vhost APIs to enqueue/dequeue data.
> > +
> > +    Typical value is 512~1024 depending on the async device capability=
.
> > +
> > +  Applications must provide following ``ops`` callbacks for vhost lib
> > + to  work with the async copye devices:
>=20
> s/copye/copy
>=20
> > +
> > +  * ``transfer_data(vid, queue_id, descs, opaque_data, count)``
> > +
> > +    vhost invokes this function to submit copy data to the async devic=
es.
> > +    For non-async_inorder capable devices, ``opaque_data`` could be us=
ed
> > +    for identifying the completed packets.
> > +
> > +  * ``check_completed_copies(vid, queue_id, opaque_data,
> > + max_packets)``
> > +
> > +    vhost invokes this function to get the copy data completed by asyn=
c
> > +    devices.
> > +
> > +* ``rte_vhost_async_channel_unregister(vid, queue_id)``
> > +
> > +  Unregister the async copy device from a vhost queue.
>=20
> 'Copy device channel' may be more accurate?
>=20
> > +
> > +* ``rte_vhost_submit_enqueue_burst(vid, queue_id, pkts, count)``
> > +
> > +  Submit an enqueue request to transmit ``count`` packets from host
> > + to guest  by async data path. Enqueue is not guaranteed to finish
> > + upon the return of  this API call.
> > +
> > +  Applications must not free the packets submitted for enqueue until
> > + the  packets are completed.
> > +
> > +* ``rte_vhost_poll_enqueue_completed(vid, queue_id, pkts, count)``
> > +
> > +  Poll enqueue completion status from async data path. Completed
> > + packets  are returned to applications through ``pkts``.
> > +
> >  Vhost-user Implementations
> >  --------------------------
> >
> > @@ -294,16 +362,16 @@ Guest memory requirement
> >
> >  * Memory pre-allocation
> >
> > -  For non-zerocopy, guest memory pre-allocation is not a must. This
> > can help
> > -  save of memory. If users really want the guest memory to be
> > pre-allocated
> > -  (e.g., for performance reason), we can add option ``-mem-prealloc``
> > when
> > -  starting QEMU. Or, we can lock all memory at vhost side which will
> > force
> > -  memory to be allocated when mmap at vhost side; option --mlockall
> > in
> > -  ovs-dpdk is an example in hand.
> > +  For non-zerocopy non-async data path, guest memory pre-allocation
> > + is not a  must. This can help save of memory. If users really want
> > + the guest memory  to be pre-allocated (e.g., for performance
> > + reason), we can add option  ``-mem-prealloc`` when starting QEMU.
> > + Or, we can lock all memory at vhost  side which will force memory to
> > + be allocated when mmap at vhost side;  option --mlockall in ovs-dpdk =
is
> an example in hand.
> >
> > -  For zerocopy, we force the VM memory to be pre-allocated at vhost
> > lib when
> > -  mapping the guest memory; and also we need to lock the memory to
> > prevent
> > -  pages being swapped out to disk.
> > +  For async data path or zerocopy, we force the VM memory to be
>=20
> 'For async or zerocopy data path' may be better?
>=20
> Thanks!
> Chenbo
>=20
> > + pre-allocated  at vhost lib when mapping the guest memory; and also
> > + we need to lock the  memory to prevent pages being swapped out to dis=
k.
> >
> >  * Memory sharing
> >
> > --
> > 2.18.4