From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from dpdk.org (dpdk.org [92.243.14.124])
	by inbox.dpdk.org (Postfix) with ESMTP id 90D78A0471
	for <public@inbox.dpdk.org>; Fri, 19 Jul 2019 15:39:00 +0200 (CEST)
Received: from [92.243.14.124] (localhost [127.0.0.1])
	by dpdk.org (Postfix) with ESMTP id 78E3B322C;
	Fri, 19 Jul 2019 15:39:00 +0200 (CEST)
Received: from proxy.6wind.com (host.76.145.23.62.rev.coltfrance.com
 [62.23.145.76]) by dpdk.org (Postfix) with ESMTP id BEF122BA8
 for <dev@dpdk.org>; Fri, 19 Jul 2019 15:38:59 +0200 (CEST)
Received: from glumotte.dev.6wind.com. (unknown [10.16.0.195])
 by proxy.6wind.com (Postfix) with ESMTP id 92E712ED324;
 Fri, 19 Jul 2019 15:38:59 +0200 (CEST)
From: Olivier Matz <olivier.matz@6wind.com>
To: Vamsi Krishna Attunuru <vattunuru@marvell.com>,
	dev@dpdk.org
Cc: Andrew Rybchenko <arybchenko@solarflare.com>,
 Thomas Monjalon <thomas@monjalon.net>,
 Anatoly Burakov <anatoly.burakov@intel.com>,
 Jerin Jacob Kollanukkaran <jerinj@marvell.com>,
 Kokkilagadda <kirankumark@marvell.com>,
 Ferruh Yigit <ferruh.yigit@intel.com>
Date: Fri, 19 Jul 2019 15:38:41 +0200
Message-Id: <20190719133845.32432-1-olivier.matz@6wind.com>
X-Mailer: git-send-email 2.11.0
In-Reply-To: <CH2PR18MB338160CD8EF16EEB45EED387A6C80@CH2PR18MB3381.namprd18.prod.outlook.com>
References: <CH2PR18MB338160CD8EF16EEB45EED387A6C80@CH2PR18MB3381.namprd18.prod.outlook.com>
Subject: [dpdk-dev] [RFC 0/4] mempool: avoid objects allocations across pages
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Sender: "dev" <dev-bounces@dpdk.org>

When IOVA mode is VA, a mempool can be created with objects that
are not physically contiguous, which breaks KNI.

To solve this, this patchset changes the default behavior of mempool
populate function, to prevent objects from being located across pages.

Olivier Matz (4):
  mempool: clarify default populate function
  mempool: unalign size when calculating required mem amount
  mempool: introduce function to get mempool page size
  mempool: prevent objects from being across pages

 lib/librte_mempool/rte_mempool.c             | 106 +++++++++++----------------
 lib/librte_mempool/rte_mempool.h             |   8 +-
 lib/librte_mempool/rte_mempool_ops.c         |   4 +-
 lib/librte_mempool/rte_mempool_ops_default.c |  39 +++++++++-
 4 files changed, 90 insertions(+), 67 deletions(-)

---

Hi,

> @Olivier,
> Any suggestions..?

I took some time to go a bit deeper. I still think we can change the
default behavior to avoid objects to be located accross pages. But
it is more complex that I expected.

I made a draft patchset, that, in short:
- cleans/renames variables
- removes the optimistic full iova contiguous allocation
- changes return value of calc_mem_size to return the unaligned size,
  therefore the allocation is smaller in case of big hugepages
- changes rte_mempool_op_populate_default() to prevent allocation
  of objects accross multiple pages

Andrew, Anatoly, did I miss something?
Vamsi, can you check if it solves your issue?

Anyway, even if validate the patchset it and make it work, I'm afraid
this is not something that could go in 19.08.

The only alternative I see is a specific mempool allocation function
when used in iova=va mode + kni, as you proposed previously.

It can probably be implemented without adding a flag, starting from
rte_mempool_create(), and replacing rte_mempool_populate_default(mp) by
something else: allocate pages one by one, and call
rte_mempool_populate_iova() for each of them.

Hope it helps. Unfortunately, I may not have too much time to spend on
it in the coming days.

Regards,
Olivier