From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9083DA057B; Wed, 1 Apr 2020 12:07:27 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 6A803375B; Wed, 1 Apr 2020 12:07:27 +0200 (CEST) Received: from EUR05-VI1-obe.outbound.protection.outlook.com (mail-vi1eur05on2073.outbound.protection.outlook.com [40.107.21.73]) by dpdk.org (Postfix) with ESMTP id 3B73F2C15 for ; Wed, 1 Apr 2020 12:07:26 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2yuN6/n0JOSMfDkZx1zWhP41yFdLC7qAi53+7E3olZ8=; b=rEe50CnMvrHayoJhWTRRckKiRBhnY6fB6GFdI+FfThFdo9AwWCqeCq5DLB+xGlV5L09r8V61cYA884cyVbflinpBhoYsKTepQPnuv7P9QLXtRNjPGkCOkCwjdOfwLDpK0sLKUt2kb2ZxlpIJXnjMgcLBCNNzlOma99D2V0yK9ic= Received: from DB6P18901CA0019.EURP189.PROD.OUTLOOK.COM (2603:10a6:4:16::29) by DBBPR08MB4695.eurprd08.prod.outlook.com (2603:10a6:10:db::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2856.20; Wed, 1 Apr 2020 10:07:24 +0000 Received: from DB5EUR03FT059.eop-EUR03.prod.protection.outlook.com (2603:10a6:4:16:cafe::7e) by DB6P18901CA0019.outlook.office365.com (2603:10a6:4:16::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2856.19 via Frontend Transport; Wed, 1 Apr 2020 10:07:24 +0000 Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dpdk.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dpdk.org; dmarc=bestguesspass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by DB5EUR03FT059.mail.protection.outlook.com (10.152.21.175) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2856.17 via Frontend Transport; Wed, 1 Apr 2020 10:07:23 +0000 Received: ("Tessian outbound e2c88df8bbbe:v50"); Wed, 01 Apr 2020 10:07:22 +0000 X-CR-MTA-TID: 64aa7808 Received: from ff6afdd92f40.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id BF43CC4F-FA08-4A72-9CAF-2E173775402A.1; Wed, 01 Apr 2020 10:07:17 +0000 Received: from EUR04-DB3-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id ff6afdd92f40.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Wed, 01 Apr 2020 10:07:17 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=atm3WOrOmEqAx4jk+eBTaF+sey1eafBV2qslZ5jyZqsiMt9yvCTVoD29zPhwyFyW1RfIxj38A9VSLyqPPB0HU7bhE2XfMx5F132oTpJMWo0aJ47xinU5xwyXP8I5wFjdN14SGcTbPpXNgtaf9/lah0ugi/RJbfa2J7vUwBUg56nInBrlUl5IbPlHB/iy4dYCTdJukQeGeLY/Zs/5LwYC2TRsGbSUvIlilQcxhEyE7+Cmt48S82yfIJ+zCN4j9rRb0qoOtZIGL2R8TtdHajsAzqZKCDxKa55VtCD3Ml6Ft93RkBhZEBtNTsa+XpBgLN7TZeTm4agfOKsjIPjNoS3EAw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2yuN6/n0JOSMfDkZx1zWhP41yFdLC7qAi53+7E3olZ8=; b=c4POB96DN3d1Kvvyoa8EB3DORMO+2SXnebYow0sg5S8OoyN4m8nr3Zn3yAopZiFc4w2zKqbS4GLudwpc8FhgeLrIXbUeTG9tysII3r5othpvzFNibIjjd5cGk3eCFtAiQ6PaRPp96qS2I2QU/PtpMqsJ9T8Gl2EBs9LnhzgJ2QnPjF+NNBg7werSVwxqKkBh8yWYyP38htc7SlzIX6qF9T4ugZtdyteLwKvIUJq71AdIYCdIoB4lBs+0R0nmGiVXLgp5iGmM7xgKAGetwP052BWSZLH2rFNeyQAZvizE2ETJuHKriHrDBzFStTmkbupADgVACjUlsyo3vqKkBSkeOg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2yuN6/n0JOSMfDkZx1zWhP41yFdLC7qAi53+7E3olZ8=; b=rEe50CnMvrHayoJhWTRRckKiRBhnY6fB6GFdI+FfThFdo9AwWCqeCq5DLB+xGlV5L09r8V61cYA884cyVbflinpBhoYsKTepQPnuv7P9QLXtRNjPGkCOkCwjdOfwLDpK0sLKUt2kb2ZxlpIJXnjMgcLBCNNzlOma99D2V0yK9ic= Received: from VI1PR08MB5376.eurprd08.prod.outlook.com (10.255.196.79) by VI1PR08MB3999.eurprd08.prod.outlook.com (20.178.204.154) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2856.20; Wed, 1 Apr 2020 10:07:15 +0000 Received: from VI1PR08MB5376.eurprd08.prod.outlook.com ([fe80::a0e2:2a9f:be7b:4b15]) by VI1PR08MB5376.eurprd08.prod.outlook.com ([fe80::a0e2:2a9f:be7b:4b15%3]) with mapi id 15.20.2856.019; Wed, 1 Apr 2020 10:07:15 +0000 From: Gavin Hu To: Marvin Liu , "maxime.coquelin@redhat.com" , "xiaolong.ye@intel.com" , "zhihong.wang@intel.com" CC: "dev@dpdk.org" , nd Thread-Topic: [dpdk-dev] [PATCH v2 2/2] vhost: cache gpa to hpa translation Thread-Index: AQHWB/UyN8J3fCX6WkKxI4Ky3yKHxKhkCX5Q Date: Wed, 1 Apr 2020 10:07:15 +0000 Message-ID: References: <20200316153353.112897-1-yong.liu@intel.com> <20200401145011.67357-1-yong.liu@intel.com> <20200401145011.67357-2-yong.liu@intel.com> In-Reply-To: <20200401145011.67357-2-yong.liu@intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: 514322f0-08a1-4bca-91f6-98081e345c99.0 x-checkrecipientchecked: true Authentication-Results-Original: spf=none (sender IP is ) smtp.mailfrom=Gavin.Hu@arm.com; x-originating-ip: [221.239.170.106] x-ms-publictraffictype: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: 69667404-a9bf-4996-5a79-08d7d6247963 x-ms-traffictypediagnostic: VI1PR08MB3999:|DBBPR08MB4695: X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true nodisclaimer: true x-ms-oob-tlc-oobclassifiers: OLM:4941;OLM:4941; x-forefront-prvs: 03607C04F0 X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:VI1PR08MB5376.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(10009020)(4636009)(396003)(39860400002)(136003)(346002)(376002)(366004)(316002)(86362001)(55016002)(5660300002)(26005)(66476007)(186003)(8676002)(9686003)(76116006)(66446008)(6506007)(7696005)(52536014)(64756008)(54906003)(110136005)(8936002)(66556008)(4326008)(53546011)(81166006)(2906002)(81156014)(66946007)(33656002)(478600001)(71200400001); DIR:OUT; SFP:1101; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: 97AAgDc0cCtvDGDQe1ZVVglY1Hk0Q5lAbMjmSfDosdfv8kNX6YLwYQKRGvh5jitAGNqATy7gsMifBsqZ5zepSPIShRW4Xeet1C7mh8dNqbzGWAjQ1lUGDIfMtLImDRG1Ev8kV6/ixBCCzT712RYxRlI84tANcUS4o46PU1/dkiGQYcXMmZsa4bKfX+2XZW56C5safNNhZGOnJosAeBb5y/69HxhcahpslS5A4ze1xaVVvPn7sxyxJH39bxsif3WbU0vLw8Z4vXYljvaIR/ClJilLp99EQKKUiI5Rljhztu5jglMrY1CQTv9A940+qI0uSXWCY2kJZT3ZVJJw3MOFFkDG+L+ZIUtpAAu4UL/PM6wpGqo0z2YeGOmuAOVbaf50yJxv8CsETxSg9dQjkdPRoNddVuyOp4MXXbcxS+Gq9Ut99CMVDz4q5ru2qTOe5kk8 x-ms-exchange-antispam-messagedata: kP2lqInJ9w5D+p/3tdn7jpTzWYSDTaRI1G81my4XfnmALQlaxS4FofeN9+ORMUaF7OhtNB3MIm4CWfVABuSJh5LrLjVeOKcRniyfTTDglkgN3AgxFywSZ13i6w6Yw+O6NaaeXYjXH09KWnHmUwqAig== x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB3999 Original-Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Gavin.Hu@arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: DB5EUR03FT059.eop-EUR03.prod.protection.outlook.com X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFTY:; SFS:(10009020)(4636009)(376002)(136003)(39860400002)(396003)(346002)(46966005)(9686003)(81156014)(186003)(52536014)(336012)(8936002)(5660300002)(70586007)(81166006)(110136005)(316002)(54906003)(478600001)(6506007)(356004)(26826003)(33656002)(8676002)(2906002)(7696005)(82740400003)(47076004)(70206006)(4326008)(26005)(86362001)(53546011)(55016002); DIR:OUT; SFP:1101; X-MS-Office365-Filtering-Correlation-Id-Prvs: 9ea267a2-fc31-4453-2738-08d7d6247465 X-Forefront-PRVS: 03607C04F0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 9qruDJWJIpS26rhhZt4VXyYByVl9IHOm9VhjfFbvgjsfzRSIakDqnI76QD2VyseYqNGKEI1E3BH1Zqq9OrLii/8ifF85Tb7+s+n371YtnuqLSvOGQ0Ykdvsi8Gpm+ARJscKPD3gcpa+T0R9FEXH7Rb8fvVodAnwCSJSEklV6nUc17uSnWcD8KytwL+FR/ccgjboB9qNG+On7yBwaNPoUrv1gnYFymUB3Hf1avTFBbShZrrYhOlRDPOsfdMfwMDM/TW6wJt9b6uFr6kGoLPbNkiiySWg+SMDkoCQoc9Qi0FQUluw/pAEFQ43AsJGh39k1lbDBXCrh6tOUM3sIV6Y67xFSTWHFUzsBL8Uff6pILgOmA3QjuXzmrMjq+6KjhW4J+EendK4usbI+Fjw4A5mltGZwD1j8zhGjVKYuZ//MfHuIzhHx7aSFYgYHcrhf/q998Rr3KbZoaml5IMdtww/J/n3PbF93oMocDwHwbcMvUgZArEnLh6n4vFdGUXqrGbD8sjktrsrcSYbCDdwsstS6jQ== X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Apr 2020 10:07:23.9845 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 69667404-a9bf-4996-5a79-08d7d6247963 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR08MB4695 Subject: Re: [dpdk-dev] [PATCH v2 2/2] vhost: cache gpa to hpa translation X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Marvin, > -----Original Message----- > From: dev On Behalf Of Marvin Liu > Sent: Wednesday, April 1, 2020 10:50 PM > To: maxime.coquelin@redhat.com; xiaolong.ye@intel.com; > zhihong.wang@intel.com > Cc: dev@dpdk.org; Marvin Liu > Subject: [dpdk-dev] [PATCH v2 2/2] vhost: cache gpa to hpa translation >=20 > If Tx zero copy enabled, gpa to hpa mapping table is updated one by > one. This will harm performance when guest memory backend using 2M > hugepages. Now add cached mapping table which will sorted by using > sequence. Address translation will first check cached mapping table, > then check unsorted mapping table if no match found. >=20 > Signed-off-by: Marvin Liu >=20 > diff --git a/lib/librte_vhost/vhost.h b/lib/librte_vhost/vhost.h > index 2087d1400..5cb0e83dd 100644 > --- a/lib/librte_vhost/vhost.h > +++ b/lib/librte_vhost/vhost.h > @@ -368,7 +368,9 @@ struct virtio_net { > struct vhost_device_ops const *notify_ops; >=20 > uint32_t nr_guest_pages; > + uint32_t nr_cached_guest_pages; > uint32_t max_guest_pages; > + struct guest_page *cached_guest_pages; > struct guest_page *guest_pages; >=20 > int slave_req_fd; > @@ -553,12 +555,25 @@ gpa_to_hpa(struct virtio_net *dev, uint64_t gpa, > uint64_t size) > { > uint32_t i; > struct guest_page *page; > + uint32_t cached_pages =3D dev->nr_cached_guest_pages; > + > + for (i =3D 0; i < cached_pages; i++) { > + page =3D &dev->cached_guest_pages[i]; > + if (gpa >=3D page->guest_phys_addr && > + gpa + size < page->guest_phys_addr + page->size) { > + return gpa - page->guest_phys_addr + > + page->host_phys_addr; > + } > + } Sorry, I did not see any speedup with cached guest pages in comparison to t= he old code below. Is it not a simple copy?=20 Is it a better idea to use hash instead to speed up the translation?=20 /Gavin >=20 > for (i =3D 0; i < dev->nr_guest_pages; i++) { > page =3D &dev->guest_pages[i]; >=20 > if (gpa >=3D page->guest_phys_addr && > gpa + size < page->guest_phys_addr + page->size) { > + rte_memcpy(&dev- > >cached_guest_pages[cached_pages], > + page, sizeof(struct guest_page)); > + dev->nr_cached_guest_pages++; > return gpa - page->guest_phys_addr + > page->host_phys_addr; > } > diff --git a/lib/librte_vhost/vhost_user.c b/lib/librte_vhost/vhost_user.= c > index 79fcb9d19..1bae1fddc 100644 > --- a/lib/librte_vhost/vhost_user.c > +++ b/lib/librte_vhost/vhost_user.c > @@ -192,7 +192,9 @@ vhost_backend_cleanup(struct virtio_net *dev) > } >=20 > rte_free(dev->guest_pages); > + rte_free(dev->cached_guest_pages); > dev->guest_pages =3D NULL; > + dev->cached_guest_pages =3D NULL; >=20 > if (dev->log_addr) { > munmap((void *)(uintptr_t)dev->log_addr, dev->log_size); > @@ -898,7 +900,7 @@ add_one_guest_page(struct virtio_net *dev, > uint64_t guest_phys_addr, > uint64_t host_phys_addr, uint64_t size) > { > struct guest_page *page, *last_page; > - struct guest_page *old_pages; > + struct guest_page *old_pages, *old_cached_pages; >=20 > if (dev->nr_guest_pages =3D=3D dev->max_guest_pages) { > dev->max_guest_pages *=3D 2; > @@ -906,9 +908,19 @@ add_one_guest_page(struct virtio_net *dev, > uint64_t guest_phys_addr, > dev->guest_pages =3D rte_realloc(dev->guest_pages, > dev->max_guest_pages * > sizeof(*page), > RTE_CACHE_LINE_SIZE); > - if (dev->guest_pages =3D=3D NULL) { > + old_cached_pages =3D dev->cached_guest_pages; > + dev->cached_guest_pages =3D rte_realloc(dev- > >cached_guest_pages, > + dev->max_guest_pages * > + sizeof(*page), > + RTE_CACHE_LINE_SIZE); > + dev->nr_cached_guest_pages =3D 0; > + if (dev->guest_pages =3D=3D NULL || > + dev->cached_guest_pages =3D=3D NULL) { > VHOST_LOG_CONFIG(ERR, "cannot realloc > guest_pages\n"); > rte_free(old_pages); > + rte_free(old_cached_pages); > + dev->guest_pages =3D NULL; > + dev->cached_guest_pages =3D NULL; > return -1; > } > } > @@ -1078,6 +1090,20 @@ vhost_user_set_mem_table(struct virtio_net > **pdev, struct VhostUserMsg *msg, > } > } >=20 > + if (dev->cached_guest_pages =3D=3D NULL) { > + dev->cached_guest_pages =3D rte_zmalloc(NULL, > + dev->max_guest_pages * > + sizeof(struct guest_page), > + RTE_CACHE_LINE_SIZE); > + if (dev->cached_guest_pages =3D=3D NULL) { > + VHOST_LOG_CONFIG(ERR, > + "(%d) failed to allocate memory " > + "for dev->cached_guest_pages\n", > + dev->vid); > + return RTE_VHOST_MSG_RESULT_ERR; > + } > + } > + > dev->mem =3D rte_zmalloc("vhost-mem-table", sizeof(struct > rte_vhost_memory) + > sizeof(struct rte_vhost_mem_region) * memory->nregions, > 0); > if (dev->mem =3D=3D NULL) { > -- > 2.17.1