From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 294CEA046B for ; Tue, 23 Jul 2019 07:37:38 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id C7D501BF66; Tue, 23 Jul 2019 07:37:36 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by dpdk.org (Postfix) with ESMTP id EE7821BF63 for ; Tue, 23 Jul 2019 07:37:34 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id x6N5ZaOh011968; Mon, 22 Jul 2019 22:37:34 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-transfer-encoding : mime-version; s=pfpt0818; bh=/q9Lvu5k5eL0meDmniyvR9ZoT9hcWTDsA1soKCAorto=; b=BS/dPkaVZEIdfffjkVQzF+PMKjy4+7NMs2bG9mfFrT8PvuaM6s8ZR2umTZEunWd5t+2O 6GL9zuS83pym95u5W2f+oHSPEjxafvA44wz0aBd/VIlbeTtjhQqevoYmzzeMe63xgpCu +ijCmCvKxShuKWnuxfCWUPpZkzIXbZp1k7fKDTJ3yWIjJTl9m4NhvT1G0Nf+ik8eJJ6X FhCtrhgpd+LlVE7qS4fqQC0zp9DJjf9Uq0RZDRiZMVy45tE/ZmWyvHUVtLCXNqHUaAsI PvvE/D9c0n9ZR0R9zsCutjJwrB0RkEqfLDVr2SqppDKxHS2oBa7w+PPhELAisukcRVDl sg== Received: from sc-exch04.marvell.com ([199.233.58.184]) by mx0b-0016f401.pphosted.com with ESMTP id 2tv2fj9gt0-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Mon, 22 Jul 2019 22:37:33 -0700 Received: from SC-EXCH02.marvell.com (10.93.176.82) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Mon, 22 Jul 2019 22:37:31 -0700 Received: from NAM03-DM3-obe.outbound.protection.outlook.com (104.47.41.50) by SC-EXCH02.marvell.com (10.93.176.82) with Microsoft SMTP Server (TLS) id 15.0.1367.3 via Frontend Transport; Mon, 22 Jul 2019 22:37:30 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=aV4k4pMmcD2iZTVmpsjUGeQxpFiMzS/aIOhsgZVk253hVcLXJ5rg6cYcQ3xXTiY4rwUt16UvruwA6qgcHo9mAiBl1M6edWUPz3yD9SblG6vdNxdnObf5cYNnMnwaykwy6unLRjFrFvYC6fsasX6JH56aQY3R47mMKC1OUwTTlZTnGPyoA0DYEXVD1G45N+E9cJ5aerYhZyG6RYYn4poXAGgV2F3FhdnAuU7UFZoNOLR7ZhlO1PhD45W5UA2P7x36T7DA8AOH003C6o2SKiU00pMemT60iNPhUaigtSR4zRLrVGZbAKSlYT9hNU81C0+JKsXS6WVQ2BBbjxHXoMe60Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/q9Lvu5k5eL0meDmniyvR9ZoT9hcWTDsA1soKCAorto=; b=nklzrnJK9iySmVX7TfKFyv7PA7xzRmCJ87D6O+infmv/ZADTZdIpxiE6EnPeDGv+/I8DDP/4AvJ8VbtR+Dk6v1sUTQytu+kKJHUaGOgNaW2C6N56VDD3KSss2wzRG10qFXyhGxuo1D2u4Efq/FLe283pIE6CQVEPpHRIsxOG5hRjYIGijsciNqORvq5+W0JsA99qrUBwhYwGlQI0RQYqNHvLzm40S8BdsRi2326Nl1L+IlbLJv3D0HT2I80gYxH6xospvqMzlz6MRet5fVT0RejqKsO6lGUYtD+tuZg4vL1GXzkH9EaXto3oV7k1JqyxReyyQpLzofBwrEKv1qCf5Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1;spf=pass smtp.mailfrom=marvell.com;dmarc=pass action=none header.from=marvell.com;dkim=pass header.d=marvell.com;arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector2-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/q9Lvu5k5eL0meDmniyvR9ZoT9hcWTDsA1soKCAorto=; b=DRlsf5VzI96HjO3cYybiLEc+oIu11ybYC8RfPiQ5njOgjp5jvm94NfZZ8hMFWkqphphcZ8myhoyh7LZsi2bwGArlsNbeX24XpIA8oIkzNvHjkE25CpvQfmIlCr9Pfaz6mpFvEAKrHz7U1ImIrjJZROkPYSZMO0Sx1LtFEX2Bv6s= Received: from CH2PR18MB3381.namprd18.prod.outlook.com (52.132.246.204) by CH2PR18MB3350.namprd18.prod.outlook.com (52.132.247.218) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2094.16; Tue, 23 Jul 2019 05:37:29 +0000 Received: from CH2PR18MB3381.namprd18.prod.outlook.com ([fe80::189c:3889:b207:8922]) by CH2PR18MB3381.namprd18.prod.outlook.com ([fe80::189c:3889:b207:8922%5]) with mapi id 15.20.2073.012; Tue, 23 Jul 2019 05:37:28 +0000 From: Vamsi Krishna Attunuru To: Olivier Matz , "dev@dpdk.org" CC: Andrew Rybchenko , Thomas Monjalon , Anatoly Burakov , "Jerin Jacob Kollanukkaran" , Kiran Kumar Kokkilagadda , Ferruh Yigit Thread-Topic: [RFC 0/4] mempool: avoid objects allocations across pages Thread-Index: AQHVPjdXtsERg9JzJki7qd0INSTdVabXsr5w Date: Tue, 23 Jul 2019 05:37:28 +0000 Message-ID: References: <20190719133845.32432-1-olivier.matz@6wind.com> In-Reply-To: <20190719133845.32432-1-olivier.matz@6wind.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [14.140.231.66] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: a5aa8020-622e-4f05-2097-08d70f2fd9e5 x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(7168020)(4627221)(201703031133081)(201702281549075)(8990200)(5600148)(711020)(4605104)(1401327)(2017052603328)(7193020); SRVR:CH2PR18MB3350; x-ms-traffictypediagnostic: CH2PR18MB3350: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-forefront-prvs: 0107098B6C x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(4636009)(376002)(136003)(396003)(346002)(366004)(39860400002)(199004)(189003)(52314003)(13464003)(54906003)(66066001)(2906002)(316002)(256004)(110136005)(74316002)(6436002)(102836004)(229853002)(53546011)(7736002)(76176011)(305945005)(86362001)(2501003)(486006)(26005)(99286004)(186003)(478600001)(55236004)(6506007)(11346002)(6116002)(25786009)(446003)(14444005)(33656002)(7696005)(14454004)(66476007)(8676002)(66446008)(3846002)(64756008)(8936002)(4326008)(52536014)(76116006)(81156014)(81166006)(66556008)(476003)(66946007)(5660300002)(53936002)(9686003)(68736007)(71200400001)(71190400001)(55016002)(6246003); DIR:OUT; SFP:1101; SCL:1; SRVR:CH2PR18MB3350; H:CH2PR18MB3381.namprd18.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: marvell.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: 3zO1CJga57Y4r151rOqPZaPgK6IJfKcwvasbCNDtlo7OyZOJzQaFztQ6bej/5atiBuJ+USBC8nIoSBTf3tYUcl/9sxHzmRYH/S1hpxeuaXhrEk97Pa0bPvuURCUKAy1RR57gaqsHrQLDTz41sq4iZIDygCMNf1u0A87GoI4nrSzBc88w6sH/9Q8RVvow+UxRUyNE/h5Y9MnfDIVUlGVVe4fE8qWF7pIaK4jJCyeHKMxvKUFB+Qnw4NDRnzMqE15BgZIQRieITvwchi8UAK2TqRR+08spwnS7nRmjl+jLDcbYx9goR7ljrJfRe70ytVx8HfH1yzeDJaq/yl8RrIIT4fZAYi5C6bGXuYBQEGzlXlh1/Y9U0b72ZPfoiIBhqCMN5g7H1cLdQJnODazIOfAmr1vEShl746U2tBYc8MAiPR0= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: a5aa8020-622e-4f05-2097-08d70f2fd9e5 X-MS-Exchange-CrossTenant-originalarrivaltime: 23 Jul 2019 05:37:28.8316 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: vattunuru@marvell.com X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR18MB3350 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:5.22.84,1.0.8 definitions=2019-07-23_03:2019-07-23,2019-07-23 signatures=0 Subject: Re: [dpdk-dev] [RFC 0/4] mempool: avoid objects allocations across pages X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > -----Original Message----- > From: Olivier Matz > Sent: Friday, July 19, 2019 7:09 PM > To: Vamsi Krishna Attunuru ; dev@dpdk.org > Cc: Andrew Rybchenko ; Thomas Monjalon > ; Anatoly Burakov ; Jerin > Jacob Kollanukkaran ; Kiran Kumar Kokkilagadda > ; Ferruh Yigit > Subject: [RFC 0/4] mempool: avoid objects allocations across pages >=20 > When IOVA mode is VA, a mempool can be created with objects that are not > physically contiguous, which breaks KNI. >=20 > To solve this, this patchset changes the default behavior of mempool popu= late > function, to prevent objects from being located across pages. >=20 > Olivier Matz (4): > mempool: clarify default populate function > mempool: unalign size when calculating required mem amount > mempool: introduce function to get mempool page size > mempool: prevent objects from being across pages >=20 > lib/librte_mempool/rte_mempool.c | 106 +++++++++++----------= ------ > lib/librte_mempool/rte_mempool.h | 8 +- > lib/librte_mempool/rte_mempool_ops.c | 4 +- > lib/librte_mempool/rte_mempool_ops_default.c | 39 +++++++++- > 4 files changed, 90 insertions(+), 67 deletions(-) >=20 > --- >=20 > Hi, >=20 > > @Olivier, > > Any suggestions..? >=20 > I took some time to go a bit deeper. I still think we can change the defa= ult > behavior to avoid objects to be located accross pages. But it is more com= plex > that I expected. >=20 > I made a draft patchset, that, in short: > - cleans/renames variables > - removes the optimistic full iova contiguous allocation > - changes return value of calc_mem_size to return the unaligned size, > therefore the allocation is smaller in case of big hugepages > - changes rte_mempool_op_populate_default() to prevent allocation > of objects accross multiple pages >=20 > Andrew, Anatoly, did I miss something? > Vamsi, can you check if it solves your issue? >=20 Thanks Olivier for your time, it solves our problem with the KNI in VA mode= .=20 Since the change set could not go in 19.08 and as you suggested,=20 implemented mempool populate without a flag and used the same for kni appli= cation.=20 Sending v8 with these changes. > Anyway, even if validate the patchset it and make it work, I'm afraid thi= s is not > something that could go in 19.08. >=20 > The only alternative I see is a specific mempool allocation function when= used in > iova=3Dva mode + kni, as you proposed previously. >=20 > It can probably be implemented without adding a flag, starting from > rte_mempool_create(), and replacing rte_mempool_populate_default(mp) by > something else: allocate pages one by one, and call > rte_mempool_populate_iova() for each of them. >=20 > Hope it helps. Unfortunately, I may not have too much time to spend on it= in the > coming days. >=20 > Regards, > Olivier