From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <users-bounces@dpdk.org>
Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124])
	by inbox.dpdk.org (Postfix) with ESMTP id 6D3F7A034C
	for <public@inbox.dpdk.org>; Sun, 30 Jan 2022 02:23:13 +0100 (CET)
Received: from [217.70.189.124] (localhost [127.0.0.1])
	by mails.dpdk.org (Postfix) with ESMTP id E73584069F;
	Sun, 30 Jan 2022 02:23:12 +0100 (CET)
Received: from mail-lf1-f50.google.com (mail-lf1-f50.google.com
 [209.85.167.50]) by mails.dpdk.org (Postfix) with ESMTP id 2F2EC40041
 for <users@dpdk.org>; Sun, 30 Jan 2022 02:23:11 +0100 (CET)
Received: by mail-lf1-f50.google.com with SMTP id p27so19470831lfa.1
 for <users@dpdk.org>; Sat, 29 Jan 2022 17:23:11 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
 h=date:from:to:cc:subject:message-id:in-reply-to:references
 :mime-version:content-transfer-encoding;
 bh=vPKAxXtA1rt/sDXAtRLg0W9RAgEhmcQr6DMOv1fNi+s=;
 b=CYBzT/rO+w9Zq6ms99hzSeNyaMgv+PJF6FzivdgnNA+Vz8+XxZ2FmVZpd+JP/Ga206
 MfsEV0RI/HoDaD7ScK3ne7xhccOd5bRTDhy2mj+nzfg34qazsSE5zAdAxiROMuvYQxfB
 6HUsHRwFRzHI1ilaH81SL92Hk0CO5YYIGEhXUUZ8kusGRzuDbvEwbhXRSLktFX7DsmFC
 Z3ewLF4c4aDOcpIp1KyclIb1MS3PFJC4ng839q/WlYbcV3AR29z3tLhYXDZ07ogjeUID
 nV9D+6Anoi0fo1ZA6x7jvrBVZYhJmHyra9o9ZteGpKMg9J77BdyAEGGoBGfTjJyZ2DFc
 HnVQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20210112;
 h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to
 :references:mime-version:content-transfer-encoding;
 bh=vPKAxXtA1rt/sDXAtRLg0W9RAgEhmcQr6DMOv1fNi+s=;
 b=XfF4HkJYBA4XYhpfkZiUoAnHt7O/ORUKNFDpchvZ/uXBdnWnlcGp33dSxFPCcZ55G/
 diC8OX227m0Fj6beZnBByHAZR+TYKQc+ef6jbvI+6WlloKZ8eUkviIZRToQiZhU8JLI/
 98wC7hOpss7cTLCcd8G7X7Slm5BQaHZW6pH6zD1TPc3oMN9VdZdzAeunaverAOiQZsAy
 huFqTaHE0n0OLmQeYVejxskUOJZ2QqGVEFecJqGuUe20CqLq3oevEHRe6QALjVO1OFNJ
 hQf9RVhGj6IB1Y6Ript7zc0ub/wBdjQTNUtdttw29bJXMXxmsbGZeh0gk/PC9Q76FXup
 Q0Uw==
X-Gm-Message-State: AOAM533sYmbNm3wuQI3THlhj0ITw4Kz9mlX7VndDpPlSzJhYu5boKPoW
 sTVeBcEH01wv9fgp9dknmUc=
X-Google-Smtp-Source: ABdhPJzleObv4e85adjBkhTqbpW8l3jaxYHBWGM5V0oPvmhSkcHqhMKQkbAj5MbKIkxfLE49o07DYw==
X-Received: by 2002:a05:6512:6cb:: with SMTP id
 u11mr10586919lff.1.1643505790532; 
 Sat, 29 Jan 2022 17:23:10 -0800 (PST)
Received: from sovereign (broadband-37-110-65-23.ip.moscow.rt.ru.
 [37.110.65.23])
 by smtp.gmail.com with ESMTPSA id u22sm3075355lfc.80.2022.01.29.17.23.09
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Sat, 29 Jan 2022 17:23:10 -0800 (PST)
Date: Sun, 30 Jan 2022 04:23:09 +0300
From: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
To: fwefew 4t4tg <7532yahoo@gmail.com>
Cc: users@dpdk.org
Subject: Re: allocating a mempool w/ rte_pktmbuf_pool_create()
Message-ID: <20220130042309.5e590857@sovereign>
In-Reply-To: <CA+Tq66X8X=azqjnH_81XMtAkEx+rkp5a2zvW3_HnvK7quU7fVQ@mail.gmail.com>
References: <CA+Tq66X8X=azqjnH_81XMtAkEx+rkp5a2zvW3_HnvK7quU7fVQ@mail.gmail.com>
X-Mailer: Claws Mail 3.18.0 (GTK+ 2.24.33; x86_64-pc-linux-gnu)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
X-BeenThere: users@dpdk.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: DPDK usage discussions <users.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/users>,
 <mailto:users-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/users/>
List-Post: <mailto:users@dpdk.org>
List-Help: <mailto:users-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/users>,
 <mailto:users-request@dpdk.org?subject=subscribe>
Errors-To: users-bounces@dpdk.org

2022-01-29 18:46 (UTC-0500), fwefew 4t4tg:
[...]
> 1. Does cache_size include or exclude data_room_size?
> 2. Does cache_size include or exclude sizeof(struct rtre_mbuf)?
> 3. Does cache size include or exclude RTE_PKTMBUF_HEADROOM?

Cache size is measured in the number of elements, irrelevant of their size.
It is not a memory size, so the questions above are not really meaningful.

> 4. What lcore is the allocated memory pinned to? 

Memory is associated with a NUMA node (DPDK calls it "socket"), not an lcore.
Each lcore belongs to one NUMA node, see rte_lcore_to_socket_id().

> The lcore of the caller
> when this method is run? The answer here is important. If it's the lcore of
> the caller when called, this routine should be called in the lcore's entry
> point so it's on the right lcore the memory is intended. Calling it on the
> lcore that happens to be running main, for example, could have a bad side
> effect if it's different from where the memory will be ultimately used.

The NUMA node is controlled by "socket_id" parameter.
Your considerations are correct, often you should create separate mempools
for each NUMA node to avoid this performance issue. (You should also
consider which NUMA node each device belongs to.)

> 5. Which one of the formal arguments represents tail room indicated in
> https://doc.dpdk.org/guides/prog_guide/mbuf_lib.html#figure-mbuf1
[...]
> 5. Unknown. Perhaps if you want private data which corresponds to tail room
> in the diagram above one has to call rte_mempool_create() instead and focus
> on private_data_size.

Incorrect; tail room is simply an unused part at the end of the data room.
Private data is for the entire mempool, not for individual mbufs.
 
> Mempool creation is like malloc: you request the total number of absolute
> bytes required. The API will not add or remove bytes to the number you
> specify. Therefore the number you give must be inclusive of all needs
> including your payload, any DPDK overheader, headroom, tailroom, and so on.
> DPDK is not adding to the number you give for its own purposes. Clearer?
> Perhaps ... but what needs? Read on ...

On the contrary: rte_pktmbuf_pool_create() takes the amount
of usable memory (dataroom) and adds space for rte_mbuf and the headroom.
Furthermore, the underlying rte_mempool_create() ensures element (mbuf)
alignment, may spread the elements between pages, etc.

[...]
> No. I might not. I might have half my TXQ and RXQs dealing with tiny
> mbufs/packets, and the other half dealing with completely different traffic
> of a completely different size and structure. So I might want memory pool
> allocation to be done on a smaller scale e.g. per RXQ/TXQ/lcore. DPDK
> doesn't seem to permit this.

You can create different mempools for each purpose
and specify the proper mempool to rte_eth_rx_queue_setup().
When creating them, you can and should also take NUMA into account.
Take a look at init_mem() function of examples/l3fwd.