From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 09956A0C45; Mon, 25 Oct 2021 09:22:47 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 897BD407FF; Mon, 25 Oct 2021 09:22:47 +0200 (CEST) Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by mails.dpdk.org (Postfix) with ESMTP id 987394003E for ; Mon, 25 Oct 2021 09:22:46 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10147"; a="253106411" X-IronPort-AV: E=Sophos;i="5.87,179,1631602800"; d="scan'208";a="253106411" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Oct 2021 00:22:45 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.87,179,1631602800"; d="scan'208";a="485557812" Received: from irsmsx603.ger.corp.intel.com ([163.33.146.9]) by orsmga007.jf.intel.com with ESMTP; 25 Oct 2021 00:22:44 -0700 Received: from shsmsx606.ccr.corp.intel.com (10.109.6.216) by irsmsx603.ger.corp.intel.com (163.33.146.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2242.12; Mon, 25 Oct 2021 08:22:42 +0100 Received: from shsmsx606.ccr.corp.intel.com ([10.109.6.216]) by SHSMSX606.ccr.corp.intel.com ([10.109.6.216]) with mapi id 15.01.2242.012; Mon, 25 Oct 2021 15:22:39 +0800 From: "Hu, Jiayu" To: Maxime Coquelin , "dev@dpdk.org" , "Xia, Chenbo" , "Wang, YuanX" , "Ma, WenwuX" , "Richardson, Bruce" , "Mcnamara, John" , "david.marchand@redhat.com" Thread-Topic: [PATCH v1 08/14] vhost: improve IO vector logic Thread-Index: AQHXxCCjfTZu2w7WJkyCNnXRPTJxqavjUNag Date: Mon, 25 Oct 2021 07:22:39 +0000 Message-ID: <9239770ed6c74f67975524bd081af39a@intel.com> References: <20211018130229.308694-1-maxime.coquelin@redhat.com> <20211018130229.308694-9-maxime.coquelin@redhat.com> In-Reply-To: <20211018130229.308694-9-maxime.coquelin@redhat.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-reaction: no-action dlp-version: 11.6.200.16 dlp-product: dlpe-windows x-originating-ip: [10.239.127.36] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH v1 08/14] vhost: improve IO vector logic X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Maxime, > -----Original Message----- > From: Maxime Coquelin > Sent: Monday, October 18, 2021 9:02 PM > To: dev@dpdk.org; Xia, Chenbo ; Hu, Jiayu > ; Wang, YuanX ; Ma, > WenwuX ; Richardson, Bruce > ; Mcnamara, John > ; david.marchand@redhat.com > Cc: Maxime Coquelin > Subject: [PATCH v1 08/14] vhost: improve IO vector logic >=20 > IO vectors and their iterators arrays were part of the async metadata but= not > their indexes. >=20 > In order to makes this more consistent, the patch adds the indexes to the > async metadata. Doing that, we can avoid triggering DMA transfer within t= he > loop as it IO vector index overflow is now prevented in the > async_mbuf_to_desc() function. >=20 > Note that previous detection mechanism was broken since the overflow > already happened when detected, so OOB memory access would already > have happened. >=20 > With this changes done, virtio_dev_rx_async_submit_split() > and virtio_dev_rx_async_submit_packed() can be further simplified. >=20 > Signed-off-by: Maxime Coquelin > --- > lib/vhost/vhost.h | 2 + > lib/vhost/virtio_net.c | 291 ++++++++++++++++++----------------------- > 2 files changed, 131 insertions(+), 162 deletions(-) >=20 > diff --git a/lib/vhost/vhost.h b/lib/vhost/vhost.h index > dae9a1ac2d..812d4c55a5 100644 > --- a/lib/vhost/vhost.h > +++ b/lib/vhost/vhost.h > @@ -134,6 +134,8 @@ struct vhost_async { >=20 > struct rte_vhost_iov_iter iov_iter[VHOST_MAX_ASYNC_IT]; > struct rte_vhost_iovec iovec[VHOST_MAX_ASYNC_VEC]; > + uint16_t iter_idx; > + uint16_t iovec_idx; >=20 > /* data transfer status */ > struct async_inflight_info *pkts_info; diff --git a/lib/vhost/virtio_ne= t.c > b/lib/vhost/virtio_net.c index ae7dded979..c80823a8de 100644 > --- a/lib/vhost/virtio_net.c > +++ b/lib/vhost/virtio_net.c > @@ -924,33 +924,86 @@ copy_mbuf_to_desc(struct virtio_net *dev, struct > vhost_virtqueue *vq, > return error; > } >=20 > +static __rte_always_inline int > +async_iter_initialize(struct vhost_async *async) { > + struct rte_vhost_iov_iter *iter; > + > + if (unlikely(async->iovec_idx >=3D VHOST_MAX_ASYNC_VEC)) { > + VHOST_LOG_DATA(ERR, "no more async iovec available\n"); > + return -1; > + } > + > + iter =3D async->iov_iter + async->iter_idx; > + iter->iov =3D async->iovec + async->iovec_idx; > + iter->nr_segs =3D 0; > + > + return 0; > +} > + > +static __rte_always_inline int > +async_iter_add_iovec(struct vhost_async *async, void *src, void *dst, > +size_t len) { > + struct rte_vhost_iov_iter *iter; > + struct rte_vhost_iovec *iovec; > + > + if (unlikely(async->iovec_idx >=3D VHOST_MAX_ASYNC_VEC)) { > + VHOST_LOG_DATA(ERR, "no more async iovec available\n"); > + return -1; > + } For large packets, like 64KB in iperf test, async_iter_add_iovec() frequent= ly reports the log above, as we run out of iovecs. I think it's better to chan= ge the log from ERR to DEBUG. In addition, the size of iovec is too small. For burst 32 and 64KB pkts, it= 's easy to run out of iovecs and we will drop the pkts to enqueue if it happen= s, which hurts performance. Enlarging the array is a choice to mitigate the issue, but another solution is to reallocate iovec once we run out of it. H= ow do you think? Thanks, Jiayu > + > + iter =3D async->iov_iter + async->iter_idx; > + iovec =3D async->iovec + async->iovec_idx; > + > + iovec->src_addr =3D src; > + iovec->dst_addr =3D dst; > + iovec->len =3D len; > + > + iter->nr_segs++; > + async->iovec_idx++; > + > + return 0; > +}