From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id DC8D8A04E6; Wed, 9 Dec 2020 15:16:29 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id C5D96BE75; Wed, 9 Dec 2020 15:16:21 +0100 (CET) Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by dpdk.org (Postfix) with ESMTP id 9F18AC988 for ; Wed, 9 Dec 2020 15:16:18 +0100 (CET) IronPort-SDR: MBz6805YztUje9ExBOWIASjRaCGG81gL0NZ8A9Cmjohn39bYNz2B0PZIufGn0xChlenetBV6XO mflNkcBIBPXg== X-IronPort-AV: E=McAfee;i="6000,8403,9829"; a="192384053" X-IronPort-AV: E=Sophos;i="5.78,405,1599548400"; d="scan'208";a="192384053" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Dec 2020 06:16:16 -0800 IronPort-SDR: EHIqdob/kV/fSRSH8ts/58b27s6fpABsJjngTtdsP9aw4Z3sKwalPeeizmtD4/t54xlj+hahYP rxMxTwfdPWvw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.78,405,1599548400"; d="scan'208";a="437803375" Received: from orsmsx602.amr.corp.intel.com ([10.22.229.15]) by fmsmga001.fm.intel.com with ESMTP; 09 Dec 2020 06:16:16 -0800 Received: from orsmsx610.amr.corp.intel.com (10.22.229.23) by ORSMSX602.amr.corp.intel.com (10.22.229.15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Wed, 9 Dec 2020 06:16:16 -0800 Received: from orsmsx605.amr.corp.intel.com (10.22.229.18) by ORSMSX610.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Wed, 9 Dec 2020 06:16:15 -0800 Received: from ORSEDG602.ED.cps.intel.com (10.7.248.7) by orsmsx605.amr.corp.intel.com (10.22.229.18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5 via Frontend Transport; Wed, 9 Dec 2020 06:16:15 -0800 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (104.47.56.172) by edgegateway.intel.com (134.134.137.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.1713.5; Wed, 9 Dec 2020 06:16:15 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=fPSNnIj690fmdSmrAm3LC//8sEaKXzjYxGkQ6taGhnjH/RmHH8gAhEYXMBwyL56ibDVqJJHDtWFc5o0PZk4/frXBZJmyTZt4LN8m+LzC66IDbMVGY82kvQu9MIS8fxrn6PhOcnvKwmPOSeRGLtwUfWFk1NAXvYwkHKPSmTfjYIn0Pz1WFFEU6p0Hy++mhNJqFSwyXF8RZwpbbezc3T7/6sSQQ14tcMMAvHajiy27hZmXkXaJZYyYHe5gG3ZvjsV2YT8uvnABy60yCqmeWsITJ8JBtah0l6CprT14y6E3/pmky6f8+Fqnw+XsOId8/Dhu9YfzdQyffQkIjL9/xS9Kww== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=SaCLqnyaY2Xt6Wg7FLxYfXRtJyIIgR2320yYLoLQIUY=; b=XPzNl41ukO/iDqsBsWE7t79l4igmN7eUcIMdOn1yuPCKgvr76VseZgBMgtviCUqeR62KGeoFSUBydcOlC5yMTue5M5XR+QOQEjyOJwroPgxn8DP4j+grpRqaEpmquo6huCPEiwETO3Lj96JnMThoyhECscucxdwTQzBnIC09t9tpDRCSSe/13czKjv52oO/RWJnihhowztmEMGf4A1lT3M7kqQ4QcgEOJ2bK/zBb9EaXt4zVEO0iwD93XVZTb3nDTruVIa4Jd4NMcAdJ5rUO7sSsiUWZtn3MAg8OZk3JwzkrV23EjoPFQuhMwPlP10+rSNDPGAWz+vFBnW9tBm8Efw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=SaCLqnyaY2Xt6Wg7FLxYfXRtJyIIgR2320yYLoLQIUY=; b=v0h+ue46NRdmIZtxuOHxUvH/VC9QOeJ19M7XCRZU+CHYLZPDz7ceO6NYOa5toOwWTz2kl0dHBpN6sR6lNiNAYEJb7lDC8zDZmyOHkQMCsaIbif4/BlBAhuGx2jF/a7SZoWP3BpW1cBW9Mo9FTvv217UNQaI2Q0p9hCTcMl+xzxg= Received: from MN2PR11MB4063.namprd11.prod.outlook.com (2603:10b6:208:13f::22) by MN2PR11MB3581.namprd11.prod.outlook.com (2603:10b6:208:f0::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3654.12; Wed, 9 Dec 2020 14:16:14 +0000 Received: from MN2PR11MB4063.namprd11.prod.outlook.com ([fe80::74f2:ba7f:cb22:5272]) by MN2PR11MB4063.namprd11.prod.outlook.com ([fe80::74f2:ba7f:cb22:5272%7]) with mapi id 15.20.3632.023; Wed, 9 Dec 2020 14:16:14 +0000 From: "Xia, Chenbo" To: Maxime Coquelin , "dev@dpdk.org" , "Ding, Xuan" Thread-Topic: [PATCH 21.02 3/3] vhost: refactor memory regions mapping Thread-Index: AQHWvAzE4hi2uifx8UKxxXgL/yh08anu7RJQ Date: Wed, 9 Dec 2020 14:16:14 +0000 Message-ID: References: <20201116113620.587073-1-maxime.coquelin@redhat.com> <20201116113620.587073-4-maxime.coquelin@redhat.com> In-Reply-To: <20201116113620.587073-4-maxime.coquelin@redhat.com> Accept-Language: en-US, zh-CN Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-reaction: no-action dlp-version: 11.5.1.3 authentication-results: redhat.com; dkim=none (message not signed) header.d=none;redhat.com; dmarc=none action=none header.from=intel.com; x-originating-ip: [192.198.147.218] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: d138ea41-9f06-4de2-c9cf-08d89c4cfcf5 x-ms-traffictypediagnostic: MN2PR11MB3581: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:9508; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: urJcu2Vn73aAv7DrdfLWOnYztPEQSxjXZ+91O6O5qiWj5S6T1ssZcvfUxiwgQwZ01WSINYW03zQsQm9S4GDrd3V9fuI45o8o9Ydu5LH+JtwwB3+991vbdT7ZNK9WsKGB0AH6hocU8X6pei1Cb79O/guoFcRZOWohzjAUmmDoB3H1SsdLngteWGiGIk2ep4XTKUeTWmWMae1e3OnwHZ31SQdPSQD34IRYwkYOcT7juXGNXk+lN8Z72UaghYLcTB+ZPs9pJj+0xrq6dXNXiIk2V5gWKbtzF/FiEdiHBX+IZnu5TELAbQj8EeNZDOKulBs4vq6peKy5HEZGhIfmRVEnkg== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MN2PR11MB4063.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(366004)(346002)(376002)(136003)(55016002)(33656002)(6636002)(9686003)(2906002)(71200400001)(8676002)(8936002)(6506007)(53546011)(64756008)(66476007)(5660300002)(86362001)(7696005)(66946007)(26005)(76116006)(83380400001)(110136005)(52536014)(186003)(508600001)(66556008)(66446008); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata: =?us-ascii?Q?P2S90iKAAmvU2LVFOzwJLKt0hjvHEWci+v0l1mGu4bUKtpaQL1YmrOyXwRw2?= =?us-ascii?Q?5F8wiQZZmcF1G7UQHvA0JwqGNozec3w2JX/UScaiB3wMFY8WcJq5gkp5zrR8?= =?us-ascii?Q?SoWxOJd1svhbVkoJBXlbVUab8O02nLEAXrKYsfBlMegoqoD7iRwG4GZ96pFi?= =?us-ascii?Q?uaALfmdBDtXqpx53FEyXwpAFay6IaNVx0oqiObkv9ZMjdoW21l/KCK/KKQP1?= =?us-ascii?Q?09fEoPHezHh+HD0ObPOKi+E/6aw9l6jdqljhPbO5Sie76rgSuYuC703T4Xrs?= =?us-ascii?Q?1y6dYCkzmSQ2Im+/BLY8KihLpYOxP2mP4ZHreWBKDzM+QRZ2NixRpS8Hypze?= =?us-ascii?Q?Twrj5VviuLLwrqaxl0gmSFqddvGC7MIUWx8cqjKQkgQ3lURiQilCNpEFeehB?= =?us-ascii?Q?egziNNX67bgNkgeDrnvorQUJAPrVNTiYQoyXya9KMUNCHKshnc+g9/bNUT7r?= =?us-ascii?Q?Q79zITEDXksH8ijLAaguXpO/IbJ2TqTWTQ1+rTSqeNwaDqVhkziVTUN6MaQK?= =?us-ascii?Q?05X1QuRzEmy6mgxixZJLNq2fTdnyQvIpQpyTXoRv9EA6d0mW4LwDG3OzhG6z?= =?us-ascii?Q?dHos3zY3iFXCPCbhrN7DJp/CQq8amVXBTRekgJxuHHBUgVCql5Ercer19izR?= =?us-ascii?Q?paPqZF3/ISHYVxP/QmKnlseTGb/fGMg+DNEf/8ONXbQxkqFclhGwqoML+jyc?= =?us-ascii?Q?zAzIbdRpq6vHF7D7/QC61SeSfibjCikP5IT0Yj5ZuwcjNWUuBiJ/oF3HwXCM?= =?us-ascii?Q?81fnbfzWhur6nXKMMh1NVenkkfkvIoeVLJGXqkT+ayXijpJlc99zWlq2nnal?= =?us-ascii?Q?Nb8AofZ49HgdRNv9axV2t5liddiV8M8XALL8ESr7XYJPyTk2S6eZ78z61Tuq?= =?us-ascii?Q?L5JBNfQuC2ftSBekHEBX2wC6wssr0sNys/1vE8YDVI4qy9/VI0ObbbkyNXeo?= =?us-ascii?Q?SnsY6D2/jkZQkCF0Bq6jtre6XLztbmYcJL5YnPN9nx8=3D?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MN2PR11MB4063.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: d138ea41-9f06-4de2-c9cf-08d89c4cfcf5 X-MS-Exchange-CrossTenant-originalarrivaltime: 09 Dec 2020 14:16:14.7619 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: CpZLxx0I8isibb1Ol6xxXitB/0B2Ph+aQRsjNFEHyQ/A7WXwoYzTTY1AqVpXsjd632mRkFGVSFN8iSJjpb6uzw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR11MB3581 X-OriginatorOrg: intel.com Subject: Re: [dpdk-dev] [PATCH 21.02 3/3] vhost: refactor memory regions mapping X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > -----Original Message----- > From: Maxime Coquelin > Sent: Monday, November 16, 2020 7:36 PM > To: dev@dpdk.org; Xia, Chenbo ; Ding, Xuan > > Cc: Maxime Coquelin > Subject: [PATCH 21.02 3/3] vhost: refactor memory regions mapping >=20 > This patch moves memory region mmaping and related > preparation in a dedicated function in order to simplify > VHOST_USER_SET_MEM_TABLE request handling function. >=20 > Signed-off-by: Maxime Coquelin > --- > lib/librte_vhost/vhost_user.c | 178 ++++++++++++++++++---------------- > 1 file changed, 94 insertions(+), 84 deletions(-) >=20 > diff --git a/lib/librte_vhost/vhost_user.c b/lib/librte_vhost/vhost_user.= c > index 2ee22ef76d..24109bd4c1 100644 > --- a/lib/librte_vhost/vhost_user.c > +++ b/lib/librte_vhost/vhost_user.c > @@ -1097,6 +1097,96 @@ vhost_user_postcopy_register(struct virtio_net *de= v, > int main_fd, > return 0; > } >=20 > +static int > +vhost_user_mmap_region(struct virtio_net *dev, > + struct rte_vhost_mem_region *region, > + uint64_t mmap_offset) > +{ > + void *mmap_addr; > + uint64_t mmap_size; > + uint64_t alignment; > + int populate; > + > + /* Check for memory_size + mmap_offset overflow */ > + if (mmap_offset >=3D -region->size) { > + VHOST_LOG_CONFIG(ERR, > + "mmap_offset (%#"PRIx64") and memory_size " > + "(%#"PRIx64") overflow\n", > + mmap_offset, region->size); > + return -1; > + } > + > + mmap_size =3D region->size + mmap_offset; > + > + /* mmap() without flag of MAP_ANONYMOUS, should be called with > length > + * argument aligned with hugepagesz at older longterm version Linux, > + * like 2.6.32 and 3.2.72, or mmap() will fail with EINVAL. > + * > + * To avoid failure, make sure in caller to keep length aligned. > + */ > + alignment =3D get_blk_size(region->fd); > + if (alignment =3D=3D (uint64_t)-1) { > + VHOST_LOG_CONFIG(ERR, > + "couldn't get hugepage size through fstat\n"); > + return -1; > + } > + mmap_size =3D RTE_ALIGN_CEIL(mmap_size, alignment); > + if (mmap_size =3D=3D 0) { > + /* > + * It could happen if initial mmap_size + alignment overflows > + * the sizeof uint64, which could happen if either mmap_size > or > + * alignment value is wrong. > + * > + * mmap() kernel implementation would return an error, but > + * better catch it before and provide useful info in the logs. > + */ > + VHOST_LOG_CONFIG(ERR, "mmap size (0x%" PRIx64 ") " > + "or alignment (0x%" PRIx64 ") is invalid\n", > + region->size + mmap_offset, alignment); > + return -1; > + } > + > + populate =3D dev->async_copy ? MAP_POPULATE : 0; > + mmap_addr =3D mmap(NULL, mmap_size, PROT_READ | PROT_WRITE, > + MAP_SHARED | populate, region->fd, 0); > + > + if (mmap_addr =3D=3D MAP_FAILED) { > + VHOST_LOG_CONFIG(ERR, "mmap failed (%s).\n", strerror(errno)); > + return -1; > + } > + > + region->mmap_addr =3D mmap_addr; > + region->mmap_size =3D mmap_size; > + region->host_user_addr =3D (uint64_t)(uintptr_t)mmap_addr + > mmap_offset; > + > + if (dev->async_copy) > + if (add_guest_pages(dev, region, alignment) < 0) { > + VHOST_LOG_CONFIG(ERR, > + "adding guest pages to region failed.\n"); > + return -1; > + } > + > + VHOST_LOG_CONFIG(INFO, > + "guest memory region size: 0x%" PRIx64 "\n" > + "\t guest physical addr: 0x%" PRIx64 "\n" > + "\t guest virtual addr: 0x%" PRIx64 "\n" > + "\t host virtual addr: 0x%" PRIx64 "\n" > + "\t mmap addr : 0x%" PRIx64 "\n" > + "\t mmap size : 0x%" PRIx64 "\n" > + "\t mmap align: 0x%" PRIx64 "\n" > + "\t mmap off : 0x%" PRIx64 "\n", > + region->size, > + region->guest_phys_addr, > + region->guest_user_addr, > + region->host_user_addr, > + (uint64_t)(uintptr_t)mmap_addr, > + mmap_size, > + alignment, > + mmap_offset); > + > + return 0; > +} > + > static int > vhost_user_set_mem_table(struct virtio_net **pdev, struct VhostUserMsg > *msg, > int main_fd) > @@ -1104,12 +1194,9 @@ vhost_user_set_mem_table(struct virtio_net **pdev, > struct VhostUserMsg *msg, > struct virtio_net *dev =3D *pdev; > struct VhostUserMemory *memory =3D &msg->payload.memory; > struct rte_vhost_mem_region *reg; > - void *mmap_addr; > - uint64_t mmap_size; > + > uint64_t mmap_offset; > - uint64_t alignment; > uint32_t i; > - int populate; >=20 > if (validate_msg_fds(msg, memory->nregions) !=3D 0) > return RTE_VHOST_MSG_RESULT_ERR; > @@ -1171,7 +1258,6 @@ vhost_user_set_mem_table(struct virtio_net **pdev, > struct VhostUserMsg *msg, > dev->vid); > goto free_guest_pages; > } > - dev->mem->nregions =3D memory->nregions; >=20 > for (i =3D 0; i < memory->nregions; i++) { > reg =3D &dev->mem->regions[i]; > @@ -1189,88 +1275,12 @@ vhost_user_set_mem_table(struct virtio_net **pdev= , > struct VhostUserMsg *msg, >=20 > mmap_offset =3D memory->regions[i].mmap_offset; >=20 > - /* Check for memory_size + mmap_offset overflow */ > - if (mmap_offset >=3D -reg->size) { > - VHOST_LOG_CONFIG(ERR, > - "mmap_offset (%#"PRIx64") and memory_size " > - "(%#"PRIx64") overflow\n", > - mmap_offset, reg->size); > - goto free_mem_table; > - } > - > - mmap_size =3D reg->size + mmap_offset; > - > - /* mmap() without flag of MAP_ANONYMOUS, should be called > - * with length argument aligned with hugepagesz at older > - * longterm version Linux, like 2.6.32 and 3.2.72, or > - * mmap() will fail with EINVAL. > - * > - * to avoid failure, make sure in caller to keep length > - * aligned. > - */ > - alignment =3D get_blk_size(reg->fd); > - if (alignment =3D=3D (uint64_t)-1) { > - VHOST_LOG_CONFIG(ERR, > - "couldn't get hugepage size through fstat\n"); > - goto free_mem_table; > - } > - mmap_size =3D RTE_ALIGN_CEIL(mmap_size, alignment); > - if (mmap_size =3D=3D 0) { > - /* > - * It could happen if initial mmap_size + alignment > - * overflows the sizeof uint64, which could happen if > - * either mmap_size or alignment value is wrong. > - * > - * mmap() kernel implementation would return an error, > - * but better catch it before and provide useful info > - * in the logs. > - */ > - VHOST_LOG_CONFIG(ERR, "mmap size (0x%" PRIx64 ") " > - "or alignment (0x%" PRIx64 ") is invalid\n", > - reg->size + mmap_offset, alignment); > - goto free_mem_table; > - } > - > - populate =3D dev->async_copy ? MAP_POPULATE : 0; > - mmap_addr =3D mmap(NULL, mmap_size, PROT_READ | PROT_WRITE, > - MAP_SHARED | populate, reg->fd, 0); > - > - if (mmap_addr =3D=3D MAP_FAILED) { > - VHOST_LOG_CONFIG(ERR, > - "mmap region %u failed.\n", i); > + if (vhost_user_mmap_region(dev, reg, mmap_offset) < 0) { > + VHOST_LOG_CONFIG(ERR, "Failed to mmap region %u\n", i); > goto free_mem_table; > } >=20 > - reg->mmap_addr =3D mmap_addr; > - reg->mmap_size =3D mmap_size; > - reg->host_user_addr =3D (uint64_t)(uintptr_t)mmap_addr + > - mmap_offset; > - > - if (dev->async_copy) > - if (add_guest_pages(dev, reg, alignment) < 0) { > - VHOST_LOG_CONFIG(ERR, > - "adding guest pages to region %u failed.\n", > - i); > - goto free_mem_table; > - } > - > - VHOST_LOG_CONFIG(INFO, > - "guest memory region %u, size: 0x%" PRIx64 "\n" > - "\t guest physical addr: 0x%" PRIx64 "\n" > - "\t guest virtual addr: 0x%" PRIx64 "\n" > - "\t host virtual addr: 0x%" PRIx64 "\n" > - "\t mmap addr : 0x%" PRIx64 "\n" > - "\t mmap size : 0x%" PRIx64 "\n" > - "\t mmap align: 0x%" PRIx64 "\n" > - "\t mmap off : 0x%" PRIx64 "\n", > - i, reg->size, > - reg->guest_phys_addr, > - reg->guest_user_addr, > - reg->host_user_addr, > - (uint64_t)(uintptr_t)mmap_addr, > - mmap_size, > - alignment, > - mmap_offset); > + dev->mem->nregions++; > } >=20 > if (vhost_user_postcopy_register(dev, main_fd, msg) < 0) > -- > 2.26.2 Reviewed-by: Chenbo Xia