From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id BE837A0613 for ; Wed, 25 Sep 2019 08:51:54 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id A170D1B05; Wed, 25 Sep 2019 08:51:53 +0200 (CEST) Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by dpdk.org (Postfix) with ESMTP id ED51911A2 for ; Wed, 25 Sep 2019 08:51:50 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga107.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 24 Sep 2019 23:51:50 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.64,547,1559545200"; d="scan'208";a="189589366" Received: from fmsmsx108.amr.corp.intel.com ([10.18.124.206]) by fmsmga007.fm.intel.com with ESMTP; 24 Sep 2019 23:51:50 -0700 Received: from FMSMSX110.amr.corp.intel.com (10.18.116.10) by FMSMSX108.amr.corp.intel.com (10.18.124.206) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 24 Sep 2019 23:51:49 -0700 Received: from shsmsx101.ccr.corp.intel.com (10.239.4.153) by fmsmsx110.amr.corp.intel.com (10.18.116.10) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 24 Sep 2019 23:51:48 -0700 Received: from shsmsx103.ccr.corp.intel.com ([169.254.4.140]) by SHSMSX101.ccr.corp.intel.com ([169.254.1.92]) with mapi id 14.03.0439.000; Wed, 25 Sep 2019 14:51:47 +0800 From: "Liu, Yong" To: "'Gavin Hu (Arm Technology China)'" , "'maxime.coquelin@redhat.com'" , "Bie, Tiwei" , "Wang, Zhihong" CC: "'dev@dpdk.org'" , 'nd' Thread-Topic: [dpdk-dev] [PATCH v2 08/16] vhost: add flush function for burst enqueue Thread-Index: AQHVc1KjqgwH11j1rE+YIdQmsLJnj6c73v9QgAAUyVA= Date: Wed, 25 Sep 2019 06:51:46 +0000 Message-ID: <86228AFD5BCD8E4EBFD2B90117B5E81E633B0C2B@SHSMSX103.ccr.corp.intel.com> References: <20190905161421.55981-2-yong.liu@intel.com> <20190919163643.24130-1-yong.liu@intel.com> <20190919163643.24130-9-yong.liu@intel.com> <86228AFD5BCD8E4EBFD2B90117B5E81E633B08DB@SHSMSX103.ccr.corp.intel.com> In-Reply-To: <86228AFD5BCD8E4EBFD2B90117B5E81E633B08DB@SHSMSX103.ccr.corp.intel.com> Accept-Language: zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiZmQzMWZhZGQtYjkxNC00ZDZkLWE0NTktYjY3NDI0OGYzOTYyIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjEwLjE4MDQuNDkiLCJUcnVzdGVkTGFiZWxIYXNoIjoiWjBCZFVVZk9vamRVc25nN3R2SDVZYWczZ1pqblwvMVwvRTdyUkg0bnZYcWlHRlZMZ0g4aWhDd1BHU1JOdTN2K1ZJIn0= x-ctpclassification: CTP_NT dlp-product: dlpe-windows dlp-version: 11.2.0.6 dlp-reaction: no-action x-originating-ip: [10.239.127.40] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH v2 08/16] vhost: add flush function for burst enqueue X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > -----Original Message----- > From: Liu, Yong > Sent: Wednesday, September 25, 2019 1:38 PM > To: Gavin Hu (Arm Technology China) ; > maxime.coquelin@redhat.com; Bie, Tiwei ; Wang, Zhiho= ng > > Cc: dev@dpdk.org; nd > Subject: RE: [dpdk-dev] [PATCH v2 08/16] vhost: add flush function for > burst enqueue >=20 >=20 >=20 > > -----Original Message----- > > From: Gavin Hu (Arm Technology China) [mailto:Gavin.Hu@arm.com] > > Sent: Wednesday, September 25, 2019 11:38 AM > > To: Liu, Yong ; maxime.coquelin@redhat.com; Bie, > Tiwei > > ; Wang, Zhihong > > Cc: dev@dpdk.org; nd > > Subject: RE: [dpdk-dev] [PATCH v2 08/16] vhost: add flush function for > > burst enqueue > > > > Hi Marvin, > > > > One typo and one comment about the barrier. > > > > /Gavin > > > > > -----Original Message----- > > > From: dev On Behalf Of Marvin Liu > > > Sent: Friday, September 20, 2019 12:37 AM > > > To: maxime.coquelin@redhat.com; tiwei.bie@intel.com; > > > zhihong.wang@intel.com > > > Cc: dev@dpdk.org; Marvin Liu > > > Subject: [dpdk-dev] [PATCH v2 08/16] vhost: add flush function for > burst > > > enqueue > > > > > > Flush used flags when burst enqueue function is finished. Descriptor'= s > > > flags are pre-calculated as them will be reset by vhost. > > s/them/they > > >=20 > Thanks. >=20 > > > > > > Signed-off-by: Marvin Liu > > > > > > diff --git a/lib/librte_vhost/vhost.h b/lib/librte_vhost/vhost.h > > > index 000648dd4..9c42c7db0 100644 > > > --- a/lib/librte_vhost/vhost.h > > > +++ b/lib/librte_vhost/vhost.h > > > @@ -39,6 +39,9 @@ > > > > > > #define VHOST_LOG_CACHE_NR 32 > > > > > > +#define VIRTIO_RX_USED_FLAG (0ULL | VRING_DESC_F_AVAIL | > > > VRING_DESC_F_USED \ > > > + | VRING_DESC_F_WRITE) > > > +#define VIRTIO_RX_USED_WRAP_FLAG (VRING_DESC_F_WRITE) > > > #define PACKED_DESCS_BURST (RTE_CACHE_LINE_SIZE / \ > > > sizeof(struct vring_packed_desc)) > > > > > > diff --git a/lib/librte_vhost/virtio_net.c > > b/lib/librte_vhost/virtio_net.c > > > index e2787b72e..8e4036204 100644 > > > --- a/lib/librte_vhost/virtio_net.c > > > +++ b/lib/librte_vhost/virtio_net.c > > > @@ -169,6 +169,51 @@ update_shadow_packed(struct vhost_virtqueue > > > *vq, > > > vq->shadow_used_packed[i].count =3D count; > > > } > > > > > > +static __rte_always_inline void > > > +flush_burst_packed(struct virtio_net *dev, struct vhost_virtqueue *v= q, > > > + uint64_t *lens, uint16_t *ids, uint16_t flags) > > > +{ > > > + uint16_t i; > > > + > > > + UNROLL_PRAGMA(PRAGMA_PARAM) > > > + for (i =3D 0; i < PACKED_DESCS_BURST; i++) { > > > + vq->desc_packed[vq->last_used_idx + i].id =3D ids[i]; > > > + vq->desc_packed[vq->last_used_idx + i].len =3D lens[i]; > > > + } > > > + > > > + UNROLL_PRAGMA(PRAGMA_PARAM) > > > + for (i =3D 0; i < PACKED_DESCS_BURST; i++) { > > > + rte_smp_wmb(); > > Should this rte_smp_wmb() be moved above the loop? It guarantees the > > orderings of updates of id, len happens before the flags, > > But all the flags of different descriptors should not be ordered. > > > Hi Gavin, > For each descriptor, virtio driver will first check flags and then check > read barrier, at the last driver will read id and length. > So wmb here is to guarantee that id and length are updated before flags. > And afterwards wmb is to guarantee the sequence. >=20 Gavin, Checked with master branch, flags store sequence is not needed. But in my environment, performance will be a litter better if ordered flags= store. I think it may be harmless to place wmb here. How about your idea? > Thanks, > Marvin >=20 > > > + vq->desc_packed[vq->last_used_idx + i].flags =3D flags; > > > + } > > > + > > > + vhost_log_cache_used_vring(dev, vq, vq->last_used_idx * > > > + sizeof(struct vring_packed_desc), > > > + sizeof(struct vring_packed_desc) * > > > + PACKED_DESCS_BURST); > > > + vhost_log_cache_sync(dev, vq); > > > + > > > + vq->last_used_idx +=3D PACKED_DESCS_BURST; > > > + if (vq->last_used_idx >=3D vq->size) { > > > + vq->used_wrap_counter ^=3D 1; > > > + vq->last_used_idx -=3D vq->size; > > > + } > > > +} > > > + > > > +static __rte_always_inline void > > > +flush_enqueue_burst_packed(struct virtio_net *dev, struct > > > vhost_virtqueue *vq, > > > + uint64_t *lens, uint16_t *ids) > > > +{ > > > + uint16_t flags =3D 0; > > > + > > > + if (vq->used_wrap_counter) > > > + flags =3D VIRTIO_RX_USED_FLAG; > > > + else > > > + flags =3D VIRTIO_RX_USED_WRAP_FLAG; > > > + > > > + flush_burst_packed(dev, vq, lens, ids, flags); > > > +} > > > + > > > static __rte_always_inline void > > > update_enqueue_shadow_packed(struct vhost_virtqueue *vq, uint16_t > > > desc_idx, > > > uint32_t len, uint16_t count) > > > @@ -950,6 +995,7 @@ virtio_dev_rx_burst_packed(struct virtio_net *dev= , > > > struct vhost_virtqueue *vq, > > > struct virtio_net_hdr_mrg_rxbuf *hdrs[PACKED_DESCS_BURST]; > > > uint32_t buf_offset =3D dev->vhost_hlen; > > > uint64_t lens[PACKED_DESCS_BURST]; > > > + uint16_t ids[PACKED_DESCS_BURST]; > > > > > > uint16_t i; > > > > > > @@ -1013,6 +1059,12 @@ virtio_dev_rx_burst_packed(struct virtio_net > > > *dev, struct vhost_virtqueue *vq, > > > pkts[i]->pkt_len); > > > } > > > > > > + UNROLL_PRAGMA(PRAGMA_PARAM) > > > + for (i =3D 0; i < PACKED_DESCS_BURST; i++) > > > + ids[i] =3D descs[avail_idx + i].id; > > > + > > > + flush_enqueue_burst_packed(dev, vq, lens, ids); > > > + > > > return 0; > > > } > > > > > > -- > > > 2.17.1