patches for DPDK stable branches
 help / color / mirror / Atom feed
From: Jerin Jacob <jerinjacobk@gmail.com>
To: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>
Cc: "jerinj@marvell.com" <jerinj@marvell.com>,
	"dev@dpdk.org" <dev@dpdk.org>,
	 "thomas@monjalon.net" <thomas@monjalon.net>,
	"olivier.matz@6wind.com" <olivier.matz@6wind.com>,
	 "arybchenko@solarflare.com" <arybchenko@solarflare.com>,
	 "bruce.richardson@intel.com" <bruce.richardson@intel.com>,
	 "konstantin.ananyev@intel.com" <konstantin.ananyev@intel.com>,
	 "hemant.agrawal@nxp.com" <hemant.agrawal@nxp.com>,
	"shahafs@mellanox.com" <shahafs@mellanox.com>,
	 Gavin Hu <Gavin.Hu@arm.com>,
	"viktorin@rehivetech.com" <viktorin@rehivetech.com>,
	 "drc@linux.vnet.ibm.com" <drc@linux.vnet.ibm.com>,
	 "anatoly.burakov@intel.com" <anatoly.burakov@intel.com>,
	"stable@dpdk.org" <stable@dpdk.org>, nd <nd@arm.com>
Subject: Re: [dpdk-stable] [dpdk-dev] [PATCH] mempool: fix mempool obj alignment for non x86
Date: Fri, 20 Dec 2019 22:25:23 +0530	[thread overview]
Message-ID: <CALBAE1My=eL8hHYWZ+a_vty0DSeWeS0gCC3i+vyvc=ufMS6pdA@mail.gmail.com> (raw)
In-Reply-To: <VE1PR08MB5149C1827E0EA0E4B21FEDF8982D0@VE1PR08MB5149.eurprd08.prod.outlook.com>

On Fri, Dec 20, 2019 at 9:25 PM Honnappa Nagarahalli
<Honnappa.Nagarahalli@arm.com> wrote:
>
> <snip>
>
> >
> > From: Jerin Jacob <jerinj@marvell.com>
> >
> > The exiting optimize_object_size() function address the memory object
> > alignment constraint on x86 for better performance.
> >
> > Different (Mirco) architecture may have different memory alignment
> > constraint for better performance and it not same as the existing
> > optimize_object_size() function. Some use, XOR(kind of CRC) scheme to
> > enable DRAM channel distribution based on the address and some may have
> > a different formula.
> If I understand correctly, address interleaving is the characteristic of the memory controller and not the CPU.
> For ex: different SoCs using the same Arm architecture might have different memory controllers. So, the solution should not be architecture specific, but SoC specific.

Yes.  See below.

> > -static unsigned optimize_object_size(unsigned obj_size)
> > +static unsigned
> > +arch_mem_object_align(unsigned obj_size)
> >  {
> >       unsigned nrank, nchan;
> >       unsigned new_obj_size;
> > @@ -99,6 +101,13 @@ static unsigned optimize_object_size(unsigned
> > obj_size)
> >               new_obj_size++;
> >       return new_obj_size * RTE_MEMPOOL_ALIGN;  }
> > +#else
> This applies to add Arm (PPC as well) SoCs which might have different schemes depending on the memory controller. IMO, this should not be architecture specific.

I agree in principle.
I will summarize the
https://www.mail-archive.com/dev@dpdk.org/msg149157.html feedback:

1) For x86 arch, it is architecture-specific
2) For power PC arch, It is architecture-specific
3) For the ARM case, it will be the memory controller specific.
4) For the ARM case, The memory controller is not using the existing
x86 arch formula.
5) If it is memory/arch-specific, Can userspace code find the optimal
alignment? In the case of octeontx2/arm64,
the memory controller does  XOR on PA address which userspace code
doesn't have much control.

This patch address the known case of (1), (2),  (4) and (5). (2) can
be added to this framework when POWER9 folks want it.

We can extend this patch to address (3) if there is a case. Without
the actual requirement(If some can share the formula of alignment
which is the memory
controller specific and it does not come under (4))) then we can
create extra layer for the memory controller and abstraction to probe
it.
Again there is no standard way of probing the memory controller in
userspace and we need platform #define, which won't work for
distribution build.
So solution needs to be arch-specific and then fine-tune to memory
controller if possible.

I can work on creating an extra layer of code if some can provide the
details of the memory controller and probing mechanism or this
patch be extended to support such case if it arises in future.

Thoughts?

>
> > +static unsigned
> > +arch_mem_object_align(unsigned obj_size) {
> > +     return obj_size;
> > +}
> > +#endif
> >
> >  struct pagesz_walk_arg {
> >       int socket_id;
> > @@ -234,8 +243,8 @@ rte_mempool_calc_obj_size(uint32_t elt_size,
> > uint32_t flags,
> >        */
> >       if ((flags & MEMPOOL_F_NO_SPREAD) == 0) {
> >               unsigned new_size;
> > -             new_size = optimize_object_size(sz->header_size + sz-
> > >elt_size +
> > -                     sz->trailer_size);
> > +             new_size = arch_mem_object_align
> > +                         (sz->header_size + sz->elt_size + sz->trailer_size);
> >               sz->trailer_size = new_size - sz->header_size - sz->elt_size;
> >       }
> >
> > --
> > 2.24.1
>

  reply	other threads:[~2019-12-20 16:55 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-12-19 13:42 jerinj
2019-12-20  3:26 ` Gavin Hu
2019-12-20  3:45   ` Jerin Jacob
2019-12-20 10:54     ` [dpdk-stable] [dpdk-dev] [PATCH] mempool: fix mempool obj alignment for nonx86 Morten Brørup
2019-12-20 15:55 ` [dpdk-stable] [dpdk-dev] [PATCH] mempool: fix mempool obj alignment for non x86 Honnappa Nagarahalli
2019-12-20 16:55   ` Jerin Jacob [this message]
2019-12-20 21:07     ` Honnappa Nagarahalli
2019-12-21  5:06       ` Jerin Jacob
2019-12-27 15:54         ` Olivier Matz
2020-01-11 13:34 ` [dpdk-stable] [dpdk-dev] [PATCH v2] " jerinj
2020-01-11 17:41   ` Stephen Hemminger
2020-01-13  6:49   ` [dpdk-stable] [dpdk-dev] [PATCH v3] " jerinj
2020-01-13  9:46     ` David Marchand
2020-01-13 11:46       ` [dpdk-stable] [EXT] " Jerin Jacob Kollanukkaran

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CALBAE1My=eL8hHYWZ+a_vty0DSeWeS0gCC3i+vyvc=ufMS6pdA@mail.gmail.com' \
    --to=jerinjacobk@gmail.com \
    --cc=Gavin.Hu@arm.com \
    --cc=Honnappa.Nagarahalli@arm.com \
    --cc=anatoly.burakov@intel.com \
    --cc=arybchenko@solarflare.com \
    --cc=bruce.richardson@intel.com \
    --cc=dev@dpdk.org \
    --cc=drc@linux.vnet.ibm.com \
    --cc=hemant.agrawal@nxp.com \
    --cc=jerinj@marvell.com \
    --cc=konstantin.ananyev@intel.com \
    --cc=nd@arm.com \
    --cc=olivier.matz@6wind.com \
    --cc=shahafs@mellanox.com \
    --cc=stable@dpdk.org \
    --cc=thomas@monjalon.net \
    --cc=viktorin@rehivetech.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).