DPDK patches and discussions
 help / color / mirror / Atom feed
From: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>
To: "jerinj@marvell.com" <jerinj@marvell.com>, "dev@dpdk.org" <dev@dpdk.org>
Cc: "thomas@monjalon.net" <thomas@monjalon.net>,
	"olivier.matz@6wind.com" <olivier.matz@6wind.com>,
	"arybchenko@solarflare.com" <arybchenko@solarflare.com>,
	"bruce.richardson@intel.com" <bruce.richardson@intel.com>,
	"konstantin.ananyev@intel.com" <konstantin.ananyev@intel.com>,
	"hemant.agrawal@nxp.com" <hemant.agrawal@nxp.com>,
	"shahafs@mellanox.com" <shahafs@mellanox.com>,
	Gavin Hu <Gavin.Hu@arm.com>,
	"viktorin@rehivetech.com" <viktorin@rehivetech.com>,
	"drc@linux.vnet.ibm.com" <drc@linux.vnet.ibm.com>,
	 "anatoly.burakov@intel.com" <anatoly.burakov@intel.com>,
	"jerinj@marvell.com" <jerinj@marvell.com>,
	"stable@dpdk.org" <stable@dpdk.org>, nd <nd@arm.com>,
	nd <nd@arm.com>
Subject: Re: [dpdk-dev] [PATCH] mempool: fix mempool obj alignment for non x86
Date: Fri, 20 Dec 2019 15:55:37 +0000	[thread overview]
Message-ID: <VE1PR08MB5149C1827E0EA0E4B21FEDF8982D0@VE1PR08MB5149.eurprd08.prod.outlook.com> (raw)
In-Reply-To: <20191219134227.3841799-1-jerinj@marvell.com>

<snip>

> 
> From: Jerin Jacob <jerinj@marvell.com>
> 
> The exiting optimize_object_size() function address the memory object
> alignment constraint on x86 for better performance.
> 
> Different (Mirco) architecture may have different memory alignment
> constraint for better performance and it not same as the existing
> optimize_object_size() function. Some use, XOR(kind of CRC) scheme to
> enable DRAM channel distribution based on the address and some may have
> a different formula.
If I understand correctly, address interleaving is the characteristic of the memory controller and not the CPU.
For ex: different SoCs using the same Arm architecture might have different memory controllers. So, the solution should not be architecture specific, but SoC specific.

> 
> Introducing arch_mem_object_align() function to abstract the differences in
> different (mirco) architectures and avoid wasting memory for mempool
> object alignment for the architecture the existing optimize_object_size() is
> not valid.
> 
> Additional details:
> https://www.mail-archive.com/dev@dpdk.org/msg149157.html
> 
> Fixes: af75078fece3 ("first public release")
> Cc: stable@dpdk.org
> 
> Signed-off-by: Jerin Jacob <jerinj@marvell.com>
> ---
>  doc/guides/prog_guide/mempool_lib.rst |  6 +++---
>  lib/librte_mempool/rte_mempool.c      | 17 +++++++++++++----
>  2 files changed, 16 insertions(+), 7 deletions(-)
> 
> diff --git a/doc/guides/prog_guide/mempool_lib.rst
> b/doc/guides/prog_guide/mempool_lib.rst
> index 3bb84b0a6..eea7a2906 100644
> --- a/doc/guides/prog_guide/mempool_lib.rst
> +++ b/doc/guides/prog_guide/mempool_lib.rst
> @@ -27,10 +27,10 @@ In debug mode
> (CONFIG_RTE_LIBRTE_MEMPOOL_DEBUG is enabled),  statistics about get
> from/put in the pool are stored in the mempool structure.
>  Statistics are per-lcore to avoid concurrent access to statistics counters.
> 
> -Memory Alignment Constraints
> -----------------------------
> +Memory Alignment Constraints on X86 architecture
> +------------------------------------------------
> 
> -Depending on hardware memory configuration, performance can be greatly
> improved by adding a specific padding between objects.
> +Depending on hardware memory configuration on X86 architecture,
> performance can be greatly improved by adding a specific padding between
> objects.
>  The objective is to ensure that the beginning of each object starts on a
> different channel and rank in memory so that all channels are equally loaded.
> 
>  This is particularly true for packet buffers when doing L3 forwarding or flow
> classification.
> diff --git a/lib/librte_mempool/rte_mempool.c
> b/lib/librte_mempool/rte_mempool.c
> index 78d8eb941..871894525 100644
> --- a/lib/librte_mempool/rte_mempool.c
> +++ b/lib/librte_mempool/rte_mempool.c
> @@ -45,6 +45,7 @@ EAL_REGISTER_TAILQ(rte_mempool_tailq)
>  #define CALC_CACHE_FLUSHTHRESH(c)	\
>  	((typeof(c))((c) * CACHE_FLUSHTHRESH_MULTIPLIER))
> 
> +#if defined(RTE_ARCH_X86)
>  /*
>   * return the greatest common divisor between a and b (fast algorithm)
>   *
> @@ -74,12 +75,13 @@ static unsigned get_gcd(unsigned a, unsigned b)  }
> 
>  /*
> - * Depending on memory configuration, objects addresses are spread
> + * Depending on memory configuration on x86 arch, objects addresses are
> + spread
>   * between channels and ranks in RAM: the pool allocator will add
>   * padding between objects. This function return the new size of the
>   * object.
>   */
> -static unsigned optimize_object_size(unsigned obj_size)
> +static unsigned
> +arch_mem_object_align(unsigned obj_size)
>  {
>  	unsigned nrank, nchan;
>  	unsigned new_obj_size;
> @@ -99,6 +101,13 @@ static unsigned optimize_object_size(unsigned
> obj_size)
>  		new_obj_size++;
>  	return new_obj_size * RTE_MEMPOOL_ALIGN;  }
> +#else
This applies to add Arm (PPC as well) SoCs which might have different schemes depending on the memory controller. IMO, this should not be architecture specific.

> +static unsigned
> +arch_mem_object_align(unsigned obj_size) {
> +	return obj_size;
> +}
> +#endif
> 
>  struct pagesz_walk_arg {
>  	int socket_id;
> @@ -234,8 +243,8 @@ rte_mempool_calc_obj_size(uint32_t elt_size,
> uint32_t flags,
>  	 */
>  	if ((flags & MEMPOOL_F_NO_SPREAD) == 0) {
>  		unsigned new_size;
> -		new_size = optimize_object_size(sz->header_size + sz-
> >elt_size +
> -			sz->trailer_size);
> +		new_size = arch_mem_object_align
> +			    (sz->header_size + sz->elt_size + sz->trailer_size);
>  		sz->trailer_size = new_size - sz->header_size - sz->elt_size;
>  	}
> 
> --
> 2.24.1


  parent reply	other threads:[~2019-12-20 15:55 UTC|newest]

Thread overview: 18+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-12-19 13:42 jerinj
2019-12-20  3:26 ` Gavin Hu
2019-12-20  3:45   ` Jerin Jacob
2019-12-20 10:54     ` [dpdk-dev] [PATCH] mempool: fix mempool obj alignment for nonx86 Morten Brørup
2019-12-20 15:55 ` Honnappa Nagarahalli [this message]
2019-12-20 16:55   ` [dpdk-dev] [PATCH] mempool: fix mempool obj alignment for non x86 Jerin Jacob
2019-12-20 21:07     ` Honnappa Nagarahalli
2019-12-21  5:06       ` Jerin Jacob
2019-12-27 15:54         ` Olivier Matz
2020-01-11 13:34 ` [dpdk-dev] [PATCH v2] " jerinj
2020-01-11 17:41   ` Stephen Hemminger
2020-01-13  6:49   ` [dpdk-dev] [PATCH v3] " jerinj
2020-01-13  9:46     ` [dpdk-dev] [dpdk-stable] " David Marchand
2020-01-13 11:46       ` [dpdk-dev] [EXT] " Jerin Jacob Kollanukkaran
2020-01-14 21:06     ` [dpdk-dev] [PATCH v4] mempool: remove memory wastage on " jerinj
2020-01-16 13:10       ` Jerin Jacob
2020-01-20 12:24         ` Olivier Matz
2020-01-20 12:29           ` Thomas Monjalon

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=VE1PR08MB5149C1827E0EA0E4B21FEDF8982D0@VE1PR08MB5149.eurprd08.prod.outlook.com \
    --to=honnappa.nagarahalli@arm.com \
    --cc=Gavin.Hu@arm.com \
    --cc=anatoly.burakov@intel.com \
    --cc=arybchenko@solarflare.com \
    --cc=bruce.richardson@intel.com \
    --cc=dev@dpdk.org \
    --cc=drc@linux.vnet.ibm.com \
    --cc=hemant.agrawal@nxp.com \
    --cc=jerinj@marvell.com \
    --cc=konstantin.ananyev@intel.com \
    --cc=nd@arm.com \
    --cc=olivier.matz@6wind.com \
    --cc=shahafs@mellanox.com \
    --cc=stable@dpdk.org \
    --cc=thomas@monjalon.net \
    --cc=viktorin@rehivetech.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).