From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <zhiyong.yang@intel.com>
Received: from mga05.intel.com (mga05.intel.com [192.55.52.43])
by dpdk.org (Postfix) with ESMTP id E441E1B29B
for <dev@dpdk.org>; Wed, 17 Jan 2018 02:37:04 +0100 (CET)
X-Amp-Result: SKIPPED(no attachment in message)
X-Amp-File-Uploaded: False
Received: from orsmga007.jf.intel.com ([10.7.209.58])
by fmsmga105.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384;
16 Jan 2018 17:37:03 -0800
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="5.46,369,1511856000"; d="scan'208";a="10291180"
Received: from fmsmsx106.amr.corp.intel.com ([10.18.124.204])
by orsmga007.jf.intel.com with ESMTP; 16 Jan 2018 17:37:03 -0800
Received: from fmsmsx157.amr.corp.intel.com (10.18.116.73) by
FMSMSX106.amr.corp.intel.com (10.18.124.204) with Microsoft SMTP Server (TLS)
id 14.3.319.2; Tue, 16 Jan 2018 17:37:03 -0800
Received: from bgsmsx152.gar.corp.intel.com (10.224.48.50) by
FMSMSX157.amr.corp.intel.com (10.18.116.73) with Microsoft SMTP Server (TLS)
id 14.3.319.2; Tue, 16 Jan 2018 17:37:02 -0800
Received: from bgsmsx101.gar.corp.intel.com ([169.254.1.245]) by
BGSMSX152.gar.corp.intel.com ([169.254.6.92]) with mapi id 14.03.0319.002;
Wed, 17 Jan 2018 07:07:00 +0530
From: "Yang, Zhiyong" <zhiyong.yang@intel.com>
To: "Chen, Junjie J" <junjie.j.chen@intel.com>, "yliu@fridaylinux.org"
<yliu@fridaylinux.org>, "maxime.coquelin@redhat.com"
<maxime.coquelin@redhat.com>
CC: "dev@dpdk.org" <dev@dpdk.org>
Thread-Topic: [dpdk-dev] [PATCH] vhost: do deep copy while reallocate vq
Thread-Index: AQHTjbQ/5KsxoduuNUyWcernZcX9X6N0oo6Q//+oaYCAAVyD8IAAGviAgAGJBLA=
Date: Wed, 17 Jan 2018 01:36:58 +0000
Message-ID: <E182254E98A5DA4EB1E657AC7CB9BD2A8B0268DA@BGSMSX101.gar.corp.intel.com>
References: <1516015939-11266-1-git-send-email-junjie.j.chen@intel.com>
<E182254E98A5DA4EB1E657AC7CB9BD2A8B0250EC@BGSMSX101.gar.corp.intel.com>
<AA85A5A5E706C44BACB0BEFD5AC08BF63132C026@SHSMSX101.ccr.corp.intel.com>
<E182254E98A5DA4EB1E657AC7CB9BD2A8B025827@BGSMSX101.gar.corp.intel.com>
<AA85A5A5E706C44BACB0BEFD5AC08BF63132C4EF@SHSMSX101.ccr.corp.intel.com>
In-Reply-To: <AA85A5A5E706C44BACB0BEFD5AC08BF63132C4EF@SHSMSX101.ccr.corp.intel.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
dlp-product: dlpe-windows
dlp-version: 11.0.0.116
dlp-reaction: no-action
x-originating-ip: [10.223.10.10]
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
Subject: Re: [dpdk-dev] [PATCH] vhost: do deep copy while reallocate vq
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://dpdk.org/ml/options/dev>,
<mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://dpdk.org/ml/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://dpdk.org/ml/listinfo/dev>,
<mailto:dev-request@dpdk.org?subject=subscribe>
X-List-Received-Date: Wed, 17 Jan 2018 01:37:05 -0000
> -----Original Message-----
> From: Chen, Junjie J
> Sent: Tuesday, January 16, 2018 3:39 PM
> To: Yang, Zhiyong <zhiyong.yang@intel.com>; yliu@fridaylinux.org;
> maxime.coquelin@redhat.com
> Cc: dev@dpdk.org
> Subject: RE: [dpdk-dev] [PATCH] vhost: do deep copy while reallocate vq
>=20
> Hi
> > > > > @@ -227,6 +227,7 @@ vhost_user_set_vring_num(struct virtio_net
> > *dev,
> > > > > "zero copy is force disabled\n");
> > > > > dev->dequeue_zero_copy =3D 0;
> > > > > }
> > > > > + TAILQ_INIT(&vq->zmbuf_list);
> > > > > }
> > > > >
> > > > > vq->shadow_used_ring =3D rte_malloc(NULL, @@ -261,6
> +262,9
> > @@
> > > > > numa_realloc(struct virtio_net *dev, int index)
> > > > > int oldnode, newnode;
> > > > > struct virtio_net *old_dev;
> > > > > struct vhost_virtqueue *old_vq, *vq;
> > > > > + struct zcopy_mbuf *new_zmbuf;
> > > > > + struct vring_used_elem *new_shadow_used_ring;
> > > > > + struct batch_copy_elem *new_batch_copy_elems;
> > > > > int ret;
> > > > >
> > > > > old_dev =3D dev;
> > > > > @@ -285,6 +289,33 @@ numa_realloc(struct virtio_net *dev, int
> > index)
> > > > > return dev;
> > > > >
> > > > > memcpy(vq, old_vq, sizeof(*vq));
> > > > > + TAILQ_INIT(&vq->zmbuf_list);
> > > > > +
> > > > > + new_zmbuf =3D rte_malloc_socket(NULL, vq-
> >zmbuf_size *
> > > > > + sizeof(struct zcopy_mbuf), 0, newnode);
> > > > > + if (new_zmbuf) {
> > > > > + rte_free(vq->zmbufs);
> > > > > + vq->zmbufs =3D new_zmbuf;
> > > > > + }
> > > >
> > > > You need to consider how to handle the case ( rte_malloc_socket
> > > > return NULL).
> > >
> > > If it failed to allocate new_zmbuf, it uses old zmbufs, so as to
> > > keep vhost alive.
> >
> > It sounds reasonable, another question is, for the 3 blocks of memory
> > being allocated, If some succeed , others fails, Does it mean that
> > the code will run on different socket? What's the perf impact if it ha=
ppens.
>=20
> The original code doesn't do deep copy and thus access memory on differen=
t
> socket, this patch is to mitigate this situation. It does access remote m=
emory
> when one of above allocation failed.
>=20
> I saw some performance improvement (24.8Gbits/s -> 26.1Gbit/s) on my dev
> machine when only reallocate for zmbufs, while I didn't see significant
> performance difference when allocating vring_used_elem and
> batch_copy_elem.
Great,=20
Reviewed-by: Zhiyong Yang <zhiyong.yang@intel.com>=20
Thanks
Zhiyong