From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 32A97A0562; Sat, 4 Apr 2020 06:27:24 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 3552C4C8A; Sat, 4 Apr 2020 06:27:23 +0200 (CEST) Received: from integrity.niometrics.com (integrity.niometrics.com [42.61.70.122]) by dpdk.org (Postfix) with ESMTP id 93F6DFFA; Sat, 4 Apr 2020 06:27:21 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by integrity.niometrics.com (Postfix) with ESMTP id 6B475409CBAA; Sat, 4 Apr 2020 12:27:19 +0800 (+08) DKIM-Filter: OpenDKIM Filter v2.11.0 integrity.niometrics.com 6B475409CBAA DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=niometrics.com; s=default; t=1585974440; bh=Mqu20HRe3lf1M7+07tznYBFmAfoP2gQ1Kz1hR7OS/Hk=; h=From:Subject:Date:Cc:To:From; b=MoW/VptZnTl4l32cfMOQvSR920DfrKhyFyENrvXRqPi8er1d1rQFXT0rVR52lQJyZ YKFAzIsW7AN8861LS4Px7cIlwLfpUnQvJBcOAgvooiI9Y8EahvTJhiztIXaR3qWHyR C/YFKvYd5phfLQyZYbKh2JsMz0UHK5HsqMHu8XaI= X-Virus-Scanned: amavisd-new at niometrics.com Received: from integrity.niometrics.com ([127.0.0.1]) by localhost (integrity.niometrics.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id q0LbU7AjIekH; Sat, 4 Apr 2020 12:27:19 +0800 (+08) Received: from [10.15.0.122] (unknown [10.15.0.122]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by integrity.niometrics.com (Postfix) with ESMTPSA id 25C81409CBA9; Sat, 4 Apr 2020 12:27:17 +0800 (+08) DMARC-Filter: OpenDMARC Filter v1.3.2 integrity.niometrics.com 25C81409CBA9 DKIM-Filter: OpenDKIM Filter v2.11.0 integrity.niometrics.com 25C81409CBA9 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=niometrics.com; s=default; t=1585974439; bh=Mqu20HRe3lf1M7+07tznYBFmAfoP2gQ1Kz1hR7OS/Hk=; h=From:Subject:Date:Cc:To:From; b=GZJCSZRG/R8bR95UwREcXIcH1ySBKzjc17h6pDCkJUGebQlxAWDikjqXij2MxQmfC 4au0J6kfbkWGlLxrowvau4Z4fj5OhR29voF8pr3n9g+/tI8jrOfcSNNjh1TSWlROiP /x6Irlhiute/Rj5mcjQiK1pauK7rJHZk/VlnuCOg= From: Bao-Long Tran Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.80.23.2.2\)) Message-Id: <0B83F781-A4DB-4775-95CA-E6C2A6838D72@niometrics.com> Date: Sat, 4 Apr 2020 12:27:16 +0800 Cc: dev@dpdk.org, users@dpdk.org To: anatoly.burakov@intel.com, olivier.matz@6wind.com, arybchenko@solarflare.com X-Mailer: Apple Mail (2.3608.80.23.2.2) X-Spam-Status: No, score=-1.3 required=3.5 tests=ALL_TRUSTED, AWL, DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU autolearn=disabled version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on integrity.niometrics.com Subject: [dpdk-dev] Memory requirement calculation to create mempools with external memory X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi, My goal is to create mbuf pools using external memory with = rte_malloc_heap_*(). The initial implementation goes like this: - Create an empty heap rte_malloc_heap_create() - Calculate memory required - Provision and add the calculated single memory chunk to=20 the heap rte_malloc_heap_memory_add() - Create mbuf pool from the heap The problematic step is the calculation step. AFAIK there's no generic = way to=20 know how much memory is required to create a particular mempool. The = example=20 I found in testpmd (see below) is quite pessimistic as stated, which = leads to=20 up to 1G overprovision if you're using 1G hugepages. I wonder under what circumstances can I make this memory requirement=20 calculation deterministic. I.e., let's say I have total control over the=20= underlying hugepages, including page size, iova, alignment, = contiguousness=20 (both iova and va), etc, how can I know exactly how much memory to = allocate? Anatoly, I don't quite understand your comment in the code below about = the=20 hdr_mem provision in particular. Why do we need to give hdr_mem special=20= treatment? Further explanation is much appreciated. Thanks, BL /* extremely pessimistic estimation of memory required to create a = mempool */ static int calc_mem_size(uint32_t nb_mbufs, uint32_t mbuf_sz, size_t pgsz, size_t = *out) { unsigned int n_pages, mbuf_per_pg, leftover; uint64_t total_mem, mbuf_mem, obj_sz; /* there is no good way to predict how much space the mempool = will * occupy because it will allocate chunks on the fly, and some = of those * will come from default DPDK memory while some will come from = our * external memory, so just assume 128MB will be enough for = everyone. */ uint64_t hdr_mem =3D 128 << 20; /* account for possible non-contiguousness */ obj_sz =3D rte_mempool_calc_obj_size(mbuf_sz, 0, NULL); if (obj_sz > pgsz) { TESTPMD_LOG(ERR, "Object size is bigger than page = size\n"); return -1; } mbuf_per_pg =3D pgsz / obj_sz; leftover =3D (nb_mbufs % mbuf_per_pg) > 0; n_pages =3D (nb_mbufs / mbuf_per_pg) + leftover; mbuf_mem =3D n_pages * pgsz; total_mem =3D RTE_ALIGN(hdr_mem + mbuf_mem, pgsz); if (total_mem > SIZE_MAX) { TESTPMD_LOG(ERR, "Memory size too big\n"); return -1; } *out =3D (size_t)total_mem; return 0; }