From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7E041A00BE; Tue, 28 Apr 2020 17:38:44 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 0C67E1D609; Tue, 28 Apr 2020 17:38:44 +0200 (CEST) Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by dpdk.org (Postfix) with ESMTP id F38EB1D5FC for ; Tue, 28 Apr 2020 17:38:41 +0200 (CEST) IronPort-SDR: 7RXvgTfYaOIvAX/Fd7MbtaV5ikkYx7LlNUFuOzs/zs5+CV05bfd8gHlIwqFuxc4uGGe1WETjnC m7SthpBYuUyg== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Apr 2020 08:38:40 -0700 IronPort-SDR: S5qJW9J//XAee/4mJHkzn/1dfHblksAuPqH+jOq+4+yi1BkowzCN7/qJL2DGv11qgVYgPll535 rnRy7Js0OYeQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.73,328,1583222400"; d="scan'208";a="257651315" Received: from fmsmsx103.amr.corp.intel.com ([10.18.124.201]) by orsmga003.jf.intel.com with ESMTP; 28 Apr 2020 08:38:39 -0700 Received: from fmsmsx152.amr.corp.intel.com (10.18.125.5) by FMSMSX103.amr.corp.intel.com (10.18.124.201) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 28 Apr 2020 08:38:38 -0700 Received: from shsmsx154.ccr.corp.intel.com (10.239.6.54) by FMSMSX152.amr.corp.intel.com (10.18.125.5) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 28 Apr 2020 08:38:38 -0700 Received: from shsmsx103.ccr.corp.intel.com ([169.254.4.146]) by SHSMSX154.ccr.corp.intel.com ([169.254.7.214]) with mapi id 14.03.0439.000; Tue, 28 Apr 2020 23:38:36 +0800 From: "Liu, Yong" To: Maxime Coquelin , "Ye, Xiaolong" , "Wang, Zhihong" CC: "dev@dpdk.org" Thread-Topic: [PATCH v3 2/2] vhost: binary search address mapping table Thread-Index: AQHWHT1jo6XWYj2yGEKH6brJq3nbHKiOIq6AgACGcUA= Date: Tue, 28 Apr 2020 15:38:35 +0000 Message-ID: <86228AFD5BCD8E4EBFD2B90117B5E81E6354779B@SHSMSX103.ccr.corp.intel.com> References: <20200316153353.112897-1-yong.liu@intel.com> <20200428091359.11297-1-yong.liu@intel.com> <20200428091359.11297-2-yong.liu@intel.com> <0328ee15-d26a-3a21-035b-077361417191@redhat.com> In-Reply-To: <0328ee15-d26a-3a21-035b-077361417191@redhat.com> Accept-Language: zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-version: 11.2.0.6 dlp-reaction: no-action x-originating-ip: [10.239.127.40] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH v3 2/2] vhost: binary search address mapping table X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > -----Original Message----- > From: Maxime Coquelin > Sent: Tuesday, April 28, 2020 11:28 PM > To: Liu, Yong ; Ye, Xiaolong ; > Wang, Zhihong > Cc: dev@dpdk.org > Subject: Re: [PATCH v3 2/2] vhost: binary search address mapping table >=20 >=20 >=20 > On 4/28/20 11:13 AM, Marvin Liu wrote: > > If Tx zero copy enabled, gpa to hpa mapping table is updated one by > > one. This will harm performance when guest memory backend using 2M > > hugepages. Now utilize binary search to find the entry in mapping > > table, meanwhile set threshold to 256 entries for linear search. > > > > Signed-off-by: Marvin Liu > > > > diff --git a/lib/librte_vhost/Makefile b/lib/librte_vhost/Makefile > > index e592795f2..8769afaad 100644 > > --- a/lib/librte_vhost/Makefile > > +++ b/lib/librte_vhost/Makefile > > @@ -10,7 +10,7 @@ EXPORT_MAP :=3D rte_vhost_version.map > > > > CFLAGS +=3D $(WERROR_FLAGS) -I$(SRCDIR) -O3 > > CFLAGS +=3D -I vhost_user > > -CFLAGS +=3D -fno-strict-aliasing > > +CFLAGS +=3D -fno-strict-aliasing -Wno-maybe-uninitialized > > LDLIBS +=3D -lpthread > > > > ifeq ($(RTE_TOOLCHAIN), gcc) > > diff --git a/lib/librte_vhost/vhost.h b/lib/librte_vhost/vhost.h > > index 507dbf214..a0fee39d5 100644 > > --- a/lib/librte_vhost/vhost.h > > +++ b/lib/librte_vhost/vhost.h > > @@ -546,20 +546,46 @@ extern int vhost_data_log_level; > > #define MAX_VHOST_DEVICE 1024 > > extern struct virtio_net *vhost_devices[MAX_VHOST_DEVICE]; > > > > +#define VHOST_BINARY_SEARCH_THRESH 256 > > +static int guest_page_addrcmp(const void *p1, const void *p2) > > +{ > > + const struct guest_page *page1 =3D (const struct guest_page *)p1; > > + const struct guest_page *page2 =3D (const struct guest_page *)p2; > > + > > + if (page1->guest_phys_addr > page2->guest_phys_addr) > > + return 1; > > + if (page1->guest_phys_addr < page2->guest_phys_addr) > > + return -1; > > + > > + return 0; > > +} > > + > > /* Convert guest physical address to host physical address */ > > static __rte_always_inline rte_iova_t > > gpa_to_hpa(struct virtio_net *dev, uint64_t gpa, uint64_t size) > > { > > uint32_t i; > > struct guest_page *page; > > - > > - for (i =3D 0; i < dev->nr_guest_pages; i++) { > > - page =3D &dev->guest_pages[i]; > > - > > - if (gpa >=3D page->guest_phys_addr && > > - gpa + size < page->guest_phys_addr + page->size) { > > - return gpa - page->guest_phys_addr + > > - page->host_phys_addr; > > + struct guest_page key; > > + > > + if (dev->nr_guest_pages >=3D VHOST_BINARY_SEARCH_THRESH) { >=20 > I would have expected the binary search to be more efficient for much > smaller number of pages. Have you done some tests to define this > threshold value? >=20 Maxime, In my unit test, binary search will be better when size over 16. But it won= 't be the case with real VM. I have tested around 128 to 1024 pages, the benefit can be seen around 256= . So threshold is set to it. Thanks, Marvin > > + key.guest_phys_addr =3D gpa; > > + page =3D bsearch(&key, dev->guest_pages, dev- > >nr_guest_pages, > > + sizeof(struct guest_page), guest_page_addrcmp); > > + if (page) { > > + if (gpa + size < page->guest_phys_addr + page->size) > > + return gpa - page->guest_phys_addr + > > + page->host_phys_addr; > > + } >=20 > Is all the generated code inlined? >=20 The compare function hasn't been inlined. Will inline it in next version. > I see that in the elf file: > 2386: 0000000000874f70 16 FUNC LOCAL DEFAULT 13 > guest_page_addrcmp >=20 > > + } else { > > + for (i =3D 0; i < dev->nr_guest_pages; i++) { > > + page =3D &dev->guest_pages[i]; > > + > > + if (gpa >=3D page->guest_phys_addr && > > + gpa + size < page->guest_phys_addr + > > + page->size) > > + return gpa - page->guest_phys_addr + > > + page->host_phys_addr; > > } > > } > > > > diff --git a/lib/librte_vhost/vhost_user.c b/lib/librte_vhost/vhost_use= r.c > > index 79fcb9d19..15e50d27d 100644 > > --- a/lib/librte_vhost/vhost_user.c > > +++ b/lib/librte_vhost/vhost_user.c > > @@ -965,6 +965,12 @@ add_guest_pages(struct virtio_net *dev, struct > rte_vhost_mem_region *reg, > > reg_size -=3D size; > > } > > > > + /* sort guest page array if over binary search threshold */ > > + if (dev->nr_guest_pages >=3D VHOST_BINARY_SEARCH_THRESH) { > > + qsort((void *)dev->guest_pages, dev->nr_guest_pages, > > + sizeof(struct guest_page), guest_page_addrcmp); > > + } > > + > > return 0; > > } > > > >