From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR02-HE1-obe.outbound.protection.outlook.com (mail-eopbgr10053.outbound.protection.outlook.com [40.107.1.53]) by dpdk.org (Postfix) with ESMTP id 040F549E0; Wed, 27 Mar 2019 06:36:05 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3he9jJbouMxB6EUq/1dSVUA37bbzqJfpKMqlgarKCuw=; b=tjj0E6hP+ETHbMTPXE7tWRSQRA+YRCkq2o+yZRwpdQmZTJ4vBfD94snfz18eNxwbVH7xQQKUcDyh3Bn/oeOuEkncDl+Lnqo6ie6256rNvr7xL9nSZdThrHhdDW6vw/fGk3cj1KHFm9FpG6UsHzGH2JvuVpfhsVgws8vB2vBHifE= Received: from AM0PR0502MB3795.eurprd05.prod.outlook.com (52.133.45.150) by AM0PR0502MB3619.eurprd05.prod.outlook.com (52.133.46.143) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1730.19; Wed, 27 Mar 2019 05:36:04 +0000 Received: from AM0PR0502MB3795.eurprd05.prod.outlook.com ([fe80::84f3:7e92:7a51:1003]) by AM0PR0502MB3795.eurprd05.prod.outlook.com ([fe80::84f3:7e92:7a51:1003%2]) with mapi id 15.20.1750.014; Wed, 27 Mar 2019 05:36:04 +0000 From: Shahaf Shuler To: Shahaf Shuler , "dev@dpdk.org" , "anatoly.burakov@intel.com" CC: Thomas Monjalon , "stable@dpdk.org" , "alejandro.lucero@netronome.com" Thread-Topic: [dpdk-dev] [PATCH] mem: don't use address hint for mapping unless requested Thread-Index: AQHU4CPOgqO8BD/REEimp1u6JauxrKYe/YZg Date: Wed, 27 Mar 2019 05:36:04 +0000 Message-ID: References: <20190321202156.117496-1-shahafs@mellanox.com> In-Reply-To: <20190321202156.117496-1-shahafs@mellanox.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=shahafs@mellanox.com; x-originating-ip: [31.154.10.105] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 00474137-4205-4e69-749f-08d6b2761ade x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600127)(711020)(4605104)(4618075)(2017052603328)(7153060)(7193020); SRVR:AM0PR0502MB3619; x-ms-traffictypediagnostic: AM0PR0502MB3619: x-ld-processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr x-microsoft-antispam-prvs: x-forefront-prvs: 0989A7979C x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(136003)(346002)(39850400004)(396003)(376002)(366004)(199004)(189003)(256004)(71200400001)(25786009)(71190400001)(5660300002)(6246003)(105586002)(26005)(66066001)(2201001)(446003)(55016002)(6506007)(81156014)(4326008)(53936002)(68736007)(81166006)(11346002)(106356001)(86362001)(54906003)(110136005)(8936002)(316002)(476003)(486006)(33656002)(7736002)(99286004)(6436002)(14454004)(3846002)(186003)(229853002)(76176011)(9686003)(478600001)(7696005)(2906002)(305945005)(2501003)(6116002)(74316002)(97736004)(102836004)(52536014); DIR:OUT; SFP:1101; SCL:1; SRVR:AM0PR0502MB3619; H:AM0PR0502MB3795.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: FNVzmQPmXoxhfFqGTiKEF8WE/F0onskOK8dC7KP1NTOG4kaFOJ6XV0sv8+iOr9U4rYUFef9NNcawl/Fay1t6ktl+FjZ1OADA16qbt479djq9zOd1jCXkhZZmJdjx8MUWcEaZhra2v6pEjbykLVviFUvK9lIHPpOtJ3JTXMvctjn0btG4e4ewfSrBxEfBLepig7Oq/e28Abt8K0TdLb+robvWbC1sDE9+G4uz6ByxhQB2DJpMGJ2Y5sNm0DXgYopbHg2vfGHzkD3IndY+5/lntNT9T8U9CHCSt2Lmy7Bi8vLoHImD6hb7t7ka8rnDV6JHy9yjJN+HEjaaSAEIFfovM5M82DTXj/KEsEkwiNljlQu//0Vk0BxBPXKYjjHwV7kuLhVXEH0Gx6Yf78wWXj4BVAcNqXwJ04JLwb4EELwFWNo= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 00474137-4205-4e69-749f-08d6b2761ade X-MS-Exchange-CrossTenant-originalarrivaltime: 27 Mar 2019 05:36:04.4704 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR0502MB3619 Subject: Re: [dpdk-dev] [PATCH] mem: don't use address hint for mapping unless requested X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Mar 2019 05:36:06 -0000 Anatoly, Alejandro, Any comments on this one? Thursday, March 21, 2019 10:22 PM, Shahaf Shuler: > Subject: [dpdk-dev] [PATCH] mem: don't use address hint for mapping > unless requested >=20 > patch[1] added an address hint as starting address for 64 bit systems in = case > an explicit base virtual address was not set by the user. >=20 > The justification for such hint was to help devices that work in VA mode = and > has a address range limitation to work smoothly with the eal memory > subsystem. >=20 > While the base address value selected may work fine for the eal initializ= ation, > it easily breaks when trying to register external memory using > rte_extmem_register API. >=20 > Trying to register anonymous memory on RH x86_64 machine took several > minutes, during them the function eal_get_virtual_area repeatedly scanned > for a good VA candidate. >=20 > The attempt to guess which VA address will be free for mapping will alway= s > result in not portable, error prone code: > * different application may use different libraries along w/ DPDK. One > can never guess which library was called first and how much virtual > memory it consumed. > * external memory can be registered at any time in the application run > time. >=20 > This patch removes the default address hint and use the address returned = by > mmap. > devices with address limitations should suggest to their users a proper b= ase- > virtaddr (EAL arg) to use. >=20 > Fixes: 1df21702873d ("mem: use address hint for mapping hugepages") > Cc: stable@dpdk.org > Cc: alejandro.lucero@netronome.com >=20 > [1] commit 1df21702873d ("mem: use address hint for mapping hugepages") >=20 > Signed-off-by: Shahaf Shuler > --- > lib/librte_eal/common/eal_common_memory.c | 22 ---------------------- > 1 file changed, 22 deletions(-) >=20 > diff --git a/lib/librte_eal/common/eal_common_memory.c > b/lib/librte_eal/common/eal_common_memory.c > index c9da69b164..09108f7a32 100644 > --- a/lib/librte_eal/common/eal_common_memory.c > +++ b/lib/librte_eal/common/eal_common_memory.c > @@ -39,23 +39,6 @@ > static void *next_baseaddr; > static uint64_t system_page_sz; >=20 > -#ifdef RTE_ARCH_64 > -/* > - * Linux kernel uses a really high address as starting address for servi= ng > - * mmaps calls. If there exists addressing limitations and IOVA mode is = VA, > - * this starting address is likely too high for those devices. However, = it > - * is possible to use a lower address in the process virtual address spa= ce > - * as with 64 bits there is a lot of available space. > - * > - * Current known limitations are 39 or 40 bits. Setting the starting add= ress > - * at 4GB implies there are 508GB or 1020GB for mapping the available > - * hugepages. This is likely enough for most systems, although a device = with > - * addressing limitations should call rte_mem_check_dma_mask for > ensuring all > - * memory is within supported range. > - */ > -static uint64_t baseaddr =3D 0x100000000; -#endif > - > void * > eal_get_virtual_area(void *requested_addr, size_t *size, > size_t page_sz, int flags, int mmap_flags) @@ -79,11 +62,6 > @@ eal_get_virtual_area(void *requested_addr, size_t *size, > rte_eal_process_type() =3D=3D RTE_PROC_PRIMARY) > next_baseaddr =3D (void *) internal_config.base_virtaddr; >=20 > -#ifdef RTE_ARCH_64 > - if (next_baseaddr =3D=3D NULL && internal_config.base_virtaddr =3D=3D 0= && > - rte_eal_process_type() =3D=3D RTE_PROC_PRIMARY) > - next_baseaddr =3D (void *) baseaddr; > -#endif > if (requested_addr =3D=3D NULL && next_baseaddr !=3D NULL) { > requested_addr =3D next_baseaddr; > requested_addr =3D RTE_PTR_ALIGN(requested_addr, > page_sz); > -- > 2.12.0 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by dpdk.space (Postfix) with ESMTP id 50730A05D3 for ; Wed, 27 Mar 2019 06:36:09 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 883B15689; Wed, 27 Mar 2019 06:36:07 +0100 (CET) Received: from EUR02-HE1-obe.outbound.protection.outlook.com (mail-eopbgr10053.outbound.protection.outlook.com [40.107.1.53]) by dpdk.org (Postfix) with ESMTP id 040F549E0; Wed, 27 Mar 2019 06:36:05 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3he9jJbouMxB6EUq/1dSVUA37bbzqJfpKMqlgarKCuw=; b=tjj0E6hP+ETHbMTPXE7tWRSQRA+YRCkq2o+yZRwpdQmZTJ4vBfD94snfz18eNxwbVH7xQQKUcDyh3Bn/oeOuEkncDl+Lnqo6ie6256rNvr7xL9nSZdThrHhdDW6vw/fGk3cj1KHFm9FpG6UsHzGH2JvuVpfhsVgws8vB2vBHifE= Received: from AM0PR0502MB3795.eurprd05.prod.outlook.com (52.133.45.150) by AM0PR0502MB3619.eurprd05.prod.outlook.com (52.133.46.143) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1730.19; Wed, 27 Mar 2019 05:36:04 +0000 Received: from AM0PR0502MB3795.eurprd05.prod.outlook.com ([fe80::84f3:7e92:7a51:1003]) by AM0PR0502MB3795.eurprd05.prod.outlook.com ([fe80::84f3:7e92:7a51:1003%2]) with mapi id 15.20.1750.014; Wed, 27 Mar 2019 05:36:04 +0000 From: Shahaf Shuler To: Shahaf Shuler , "dev@dpdk.org" , "anatoly.burakov@intel.com" CC: Thomas Monjalon , "stable@dpdk.org" , "alejandro.lucero@netronome.com" Thread-Topic: [dpdk-dev] [PATCH] mem: don't use address hint for mapping unless requested Thread-Index: AQHU4CPOgqO8BD/REEimp1u6JauxrKYe/YZg Date: Wed, 27 Mar 2019 05:36:04 +0000 Message-ID: References: <20190321202156.117496-1-shahafs@mellanox.com> In-Reply-To: <20190321202156.117496-1-shahafs@mellanox.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=shahafs@mellanox.com; x-originating-ip: [31.154.10.105] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 00474137-4205-4e69-749f-08d6b2761ade x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600127)(711020)(4605104)(4618075)(2017052603328)(7153060)(7193020); SRVR:AM0PR0502MB3619; x-ms-traffictypediagnostic: AM0PR0502MB3619: x-ld-processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr x-microsoft-antispam-prvs: x-forefront-prvs: 0989A7979C x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(136003)(346002)(39850400004)(396003)(376002)(366004)(199004)(189003)(256004)(71200400001)(25786009)(71190400001)(5660300002)(6246003)(105586002)(26005)(66066001)(2201001)(446003)(55016002)(6506007)(81156014)(4326008)(53936002)(68736007)(81166006)(11346002)(106356001)(86362001)(54906003)(110136005)(8936002)(316002)(476003)(486006)(33656002)(7736002)(99286004)(6436002)(14454004)(3846002)(186003)(229853002)(76176011)(9686003)(478600001)(7696005)(2906002)(305945005)(2501003)(6116002)(74316002)(97736004)(102836004)(52536014); DIR:OUT; SFP:1101; SCL:1; SRVR:AM0PR0502MB3619; H:AM0PR0502MB3795.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: FNVzmQPmXoxhfFqGTiKEF8WE/F0onskOK8dC7KP1NTOG4kaFOJ6XV0sv8+iOr9U4rYUFef9NNcawl/Fay1t6ktl+FjZ1OADA16qbt479djq9zOd1jCXkhZZmJdjx8MUWcEaZhra2v6pEjbykLVviFUvK9lIHPpOtJ3JTXMvctjn0btG4e4ewfSrBxEfBLepig7Oq/e28Abt8K0TdLb+robvWbC1sDE9+G4uz6ByxhQB2DJpMGJ2Y5sNm0DXgYopbHg2vfGHzkD3IndY+5/lntNT9T8U9CHCSt2Lmy7Bi8vLoHImD6hb7t7ka8rnDV6JHy9yjJN+HEjaaSAEIFfovM5M82DTXj/KEsEkwiNljlQu//0Vk0BxBPXKYjjHwV7kuLhVXEH0Gx6Yf78wWXj4BVAcNqXwJ04JLwb4EELwFWNo= Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 00474137-4205-4e69-749f-08d6b2761ade X-MS-Exchange-CrossTenant-originalarrivaltime: 27 Mar 2019 05:36:04.4704 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR0502MB3619 Subject: Re: [dpdk-dev] [PATCH] mem: don't use address hint for mapping unless requested X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Message-ID: <20190327053604.636YY7URbuRiMulVjk9Wrizc0R2aGGtxiYAxRdtHF7I@z> Anatoly, Alejandro, Any comments on this one? Thursday, March 21, 2019 10:22 PM, Shahaf Shuler: > Subject: [dpdk-dev] [PATCH] mem: don't use address hint for mapping > unless requested >=20 > patch[1] added an address hint as starting address for 64 bit systems in = case > an explicit base virtual address was not set by the user. >=20 > The justification for such hint was to help devices that work in VA mode = and > has a address range limitation to work smoothly with the eal memory > subsystem. >=20 > While the base address value selected may work fine for the eal initializ= ation, > it easily breaks when trying to register external memory using > rte_extmem_register API. >=20 > Trying to register anonymous memory on RH x86_64 machine took several > minutes, during them the function eal_get_virtual_area repeatedly scanned > for a good VA candidate. >=20 > The attempt to guess which VA address will be free for mapping will alway= s > result in not portable, error prone code: > * different application may use different libraries along w/ DPDK. One > can never guess which library was called first and how much virtual > memory it consumed. > * external memory can be registered at any time in the application run > time. >=20 > This patch removes the default address hint and use the address returned = by > mmap. > devices with address limitations should suggest to their users a proper b= ase- > virtaddr (EAL arg) to use. >=20 > Fixes: 1df21702873d ("mem: use address hint for mapping hugepages") > Cc: stable@dpdk.org > Cc: alejandro.lucero@netronome.com >=20 > [1] commit 1df21702873d ("mem: use address hint for mapping hugepages") >=20 > Signed-off-by: Shahaf Shuler > --- > lib/librte_eal/common/eal_common_memory.c | 22 ---------------------- > 1 file changed, 22 deletions(-) >=20 > diff --git a/lib/librte_eal/common/eal_common_memory.c > b/lib/librte_eal/common/eal_common_memory.c > index c9da69b164..09108f7a32 100644 > --- a/lib/librte_eal/common/eal_common_memory.c > +++ b/lib/librte_eal/common/eal_common_memory.c > @@ -39,23 +39,6 @@ > static void *next_baseaddr; > static uint64_t system_page_sz; >=20 > -#ifdef RTE_ARCH_64 > -/* > - * Linux kernel uses a really high address as starting address for servi= ng > - * mmaps calls. If there exists addressing limitations and IOVA mode is = VA, > - * this starting address is likely too high for those devices. However, = it > - * is possible to use a lower address in the process virtual address spa= ce > - * as with 64 bits there is a lot of available space. > - * > - * Current known limitations are 39 or 40 bits. Setting the starting add= ress > - * at 4GB implies there are 508GB or 1020GB for mapping the available > - * hugepages. This is likely enough for most systems, although a device = with > - * addressing limitations should call rte_mem_check_dma_mask for > ensuring all > - * memory is within supported range. > - */ > -static uint64_t baseaddr =3D 0x100000000; -#endif > - > void * > eal_get_virtual_area(void *requested_addr, size_t *size, > size_t page_sz, int flags, int mmap_flags) @@ -79,11 +62,6 > @@ eal_get_virtual_area(void *requested_addr, size_t *size, > rte_eal_process_type() =3D=3D RTE_PROC_PRIMARY) > next_baseaddr =3D (void *) internal_config.base_virtaddr; >=20 > -#ifdef RTE_ARCH_64 > - if (next_baseaddr =3D=3D NULL && internal_config.base_virtaddr =3D=3D 0= && > - rte_eal_process_type() =3D=3D RTE_PROC_PRIMARY) > - next_baseaddr =3D (void *) baseaddr; > -#endif > if (requested_addr =3D=3D NULL && next_baseaddr !=3D NULL) { > requested_addr =3D next_baseaddr; > requested_addr =3D RTE_PTR_ALIGN(requested_addr, > page_sz); > -- > 2.12.0