From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by dpdk.org (Postfix) with ESMTP id 00E7412001 for ; Tue, 16 Jan 2018 08:38:45 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga104.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 15 Jan 2018 23:38:44 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.46,367,1511856000"; d="scan'208";a="10890360" Received: from fmsmsx103.amr.corp.intel.com ([10.18.124.201]) by orsmga006.jf.intel.com with ESMTP; 15 Jan 2018 23:38:44 -0800 Received: from shsmsx152.ccr.corp.intel.com (10.239.6.52) by FMSMSX103.amr.corp.intel.com (10.18.124.201) with Microsoft SMTP Server (TLS) id 14.3.319.2; Mon, 15 Jan 2018 23:38:44 -0800 Received: from shsmsx101.ccr.corp.intel.com ([169.254.1.159]) by SHSMSX152.ccr.corp.intel.com ([169.254.6.93]) with mapi id 14.03.0319.002; Tue, 16 Jan 2018 15:38:42 +0800 From: "Chen, Junjie J" To: "Yang, Zhiyong" , "yliu@fridaylinux.org" , "maxime.coquelin@redhat.com" CC: "dev@dpdk.org" Thread-Topic: [dpdk-dev] [PATCH] vhost: do deep copy while reallocate vq Thread-Index: AQHTjbQyopx1w1avvUa06Po/lYrCYqN0HoiAgACH5+CAAIHnAIAA8cmQ Date: Tue, 16 Jan 2018 07:38:41 +0000 Message-ID: References: <1516015939-11266-1-git-send-email-junjie.j.chen@intel.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-version: 11.0.0.116 dlp-reaction: no-action x-ctpclassification: CTP_NT x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiODg4MzRhNTctYTE4YS00ZmRhLWFmOTktNmNiZWRlZTA5ZmIzIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjIuNS4xOCIsIlRydXN0ZWRMYWJlbEhhc2giOiJEYk5DeDFGZVNnV045NGx0WmVhS1ZnMkVKRldXXC9mV3F0d1N0WElLVHM5Z0pDeXJoWTA1QmVtbjQxd041VmRkdCJ9 x-originating-ip: [10.239.127.40] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH] vhost: do deep copy while reallocate vq X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jan 2018 07:38:46 -0000 Hi > > > > @@ -227,6 +227,7 @@ vhost_user_set_vring_num(struct virtio_net > *dev, > > > > "zero copy is force disabled\n"); > > > > dev->dequeue_zero_copy =3D 0; > > > > } > > > > + TAILQ_INIT(&vq->zmbuf_list); > > > > } > > > > > > > > vq->shadow_used_ring =3D rte_malloc(NULL, @@ -261,6 +262,9 > @@ > > > > numa_realloc(struct virtio_net *dev, int index) > > > > int oldnode, newnode; > > > > struct virtio_net *old_dev; > > > > struct vhost_virtqueue *old_vq, *vq; > > > > + struct zcopy_mbuf *new_zmbuf; > > > > + struct vring_used_elem *new_shadow_used_ring; > > > > + struct batch_copy_elem *new_batch_copy_elems; > > > > int ret; > > > > > > > > old_dev =3D dev; > > > > @@ -285,6 +289,33 @@ numa_realloc(struct virtio_net *dev, int > index) > > > > return dev; > > > > > > > > memcpy(vq, old_vq, sizeof(*vq)); > > > > + TAILQ_INIT(&vq->zmbuf_list); > > > > + > > > > + new_zmbuf =3D rte_malloc_socket(NULL, vq->zmbuf_size * > > > > + sizeof(struct zcopy_mbuf), 0, newnode); > > > > + if (new_zmbuf) { > > > > + rte_free(vq->zmbufs); > > > > + vq->zmbufs =3D new_zmbuf; > > > > + } > > > > > > You need to consider how to handle the case ( rte_malloc_socket > > > return NULL). > > > > If it failed to allocate new_zmbuf, it uses old zmbufs, so as to keep > > vhost alive. >=20 > It sounds reasonable, another question is, for the 3 blocks of memory bei= ng > allocated, If some succeed , others fails, Does it mean that the code w= ill > run on different socket? What's the perf impact if it happens. The original code doesn't do deep copy and thus access memory on different = socket, this patch is to mitigate this situation. It does access remote mem= ory when one of above allocation failed.=20 I saw some performance improvement (24.8Gbits/s -> 26.1Gbit/s) on my dev ma= chine when only reallocate for zmbufs, while I didn't see significant perfo= rmance difference when allocating vring_used_elem=20 and batch_copy_elem.