patches for DPDK stable branches
 help / color / mirror / Atom feed
From: Jerin Jacob <jerinjacobk@gmail.com>
To: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>
Cc: "jerinj@marvell.com" <jerinj@marvell.com>,
	"dev@dpdk.org" <dev@dpdk.org>,
	 "thomas@monjalon.net" <thomas@monjalon.net>,
	"olivier.matz@6wind.com" <olivier.matz@6wind.com>,
	 "arybchenko@solarflare.com" <arybchenko@solarflare.com>,
	 "bruce.richardson@intel.com" <bruce.richardson@intel.com>,
	 "konstantin.ananyev@intel.com" <konstantin.ananyev@intel.com>,
	 "hemant.agrawal@nxp.com" <hemant.agrawal@nxp.com>,
	"shahafs@mellanox.com" <shahafs@mellanox.com>,
	 Gavin Hu <Gavin.Hu@arm.com>,
	"viktorin@rehivetech.com" <viktorin@rehivetech.com>,
	 "drc@linux.vnet.ibm.com" <drc@linux.vnet.ibm.com>,
	 "anatoly.burakov@intel.com" <anatoly.burakov@intel.com>,
	"stable@dpdk.org" <stable@dpdk.org>, nd <nd@arm.com>
Subject: Re: [dpdk-stable] [dpdk-dev] [PATCH] mempool: fix mempool obj alignment for non x86
Date: Sat, 21 Dec 2019 10:36:15 +0530	[thread overview]
Message-ID: <CALBAE1MnmfGuNN86K3czpsTrEns2X7m0rd5+DcwzBzicpeTR5A@mail.gmail.com> (raw)
In-Reply-To: <VE1PR08MB5149E7D8F7B385C14F01C14A982D0@VE1PR08MB5149.eurprd08.prod.outlook.com>

On Sat, Dec 21, 2019 at 2:37 AM Honnappa Nagarahalli
<Honnappa.Nagarahalli@arm.com> wrote:
>
> <snip>
> > > > From: Jerin Jacob <jerinj@marvell.com>
> > > >
> > > > The exiting optimize_object_size() function address the memory
> > > > object alignment constraint on x86 for better performance.
> > > >
> > > > Different (Mirco) architecture may have different memory alignment
> > > > constraint for better performance and it not same as the existing
> > > > optimize_object_size() function. Some use, XOR(kind of CRC) scheme
> > > > to enable DRAM channel distribution based on the address and some
> > > > may have a different formula.
> > > If I understand correctly, address interleaving is the characteristic of the
> > memory controller and not the CPU.
> > > For ex: different SoCs using the same Arm architecture might have different
> > memory controllers. So, the solution should not be architecture specific, but
> > SoC specific.
> >
> > Yes.  See below.
> >
> > > > -static unsigned optimize_object_size(unsigned obj_size)
> > > > +static unsigned
> > > > +arch_mem_object_align(unsigned obj_size)
> > > >  {
> > > >       unsigned nrank, nchan;
> > > >       unsigned new_obj_size;
> > > > @@ -99,6 +101,13 @@ static unsigned optimize_object_size(unsigned
> > > > obj_size)
> > > >               new_obj_size++;
> > > >       return new_obj_size * RTE_MEMPOOL_ALIGN;  }
> > > > +#else
> > > This applies to add Arm (PPC as well) SoCs which might have different
> > schemes depending on the memory controller. IMO, this should not be
> > architecture specific.
> >
> > I agree in principle.
> > I will summarize the
> > https://www.mail-archive.com/dev@dpdk.org/msg149157.html feedback:
> >
> > 1) For x86 arch, it is architecture-specific
> > 2) For power PC arch, It is architecture-specific
> > 3) For the ARM case, it will be the memory controller specific.
> > 4) For the ARM case, The memory controller is not using the existing
> > x86 arch formula.
> > 5) If it is memory/arch-specific, Can userspace code find the optimal
> > alignment? In the case of octeontx2/arm64, the memory controller does  XOR
> > on PA address which userspace code doesn't have much control.
> >
> > This patch address the known case of (1), (2),  (4) and (5). (2) can be added to
> > this framework when POWER9 folks want it.
> >
> > We can extend this patch to address (3) if there is a case. Without the actual
> > requirement(If some can share the formula of alignment which is the
> > memory controller specific and it does not come under (4))) then we can
> > create extra layer for the memory controller and abstraction to probe it.
> > Again there is no standard way of probing the memory controller in
> > userspace and we need platform #define, which won't work for distribution
> > build.
> > So solution needs to be arch-specific and then fine-tune to memory controller
> > if possible.
> >
> > I can work on creating an extra layer of code if some can provide the details
> > of the memory controller and probing mechanism or this patch be extended
> Inputs for BlueField, DPAAx, ThunderX2 would be helpful.

Yes. Probably memory controller used in n1sdp SoC also.

>
> > to support such case if it arises in future.
> >
> > Thoughts?
> How much memory will this save for your platform? Is it affecting performance?

No performance difference.

The existing code adding the tailer for each objs.
Additional space/Trailer space will be function of number of objects
in mempool  and its obj_size, its alignment and selected
rte_memory_get_nchannel() and rte_memory_get_nrank()

I will wait for inputs from Bluefield, DPAAx, ThunderX2 and n1sdp(if
any) for any rework on the patch.

>
> >
> > >
> > > > +static unsigned
> > > > +arch_mem_object_align(unsigned obj_size) {
> > > > +     return obj_size;
> > > > +}
> > > > +#endif
> > > >
> > > >  struct pagesz_walk_arg {
> > > >       int socket_id;
> > > > @@ -234,8 +243,8 @@ rte_mempool_calc_obj_size(uint32_t elt_size,
> > > > uint32_t flags,
> > > >        */
> > > >       if ((flags & MEMPOOL_F_NO_SPREAD) == 0) {
> > > >               unsigned new_size;
> > > > -             new_size = optimize_object_size(sz->header_size + sz-
> > > > >elt_size +
> > > > -                     sz->trailer_size);
> > > > +             new_size = arch_mem_object_align
> > > > +                         (sz->header_size + sz->elt_size +
> > > > + sz->trailer_size);
> > > >               sz->trailer_size = new_size - sz->header_size - sz->elt_size;
> > > >       }
> > > >
> > > > --
> > > > 2.24.1
> > >

  reply	other threads:[~2019-12-21  5:06 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-12-19 13:42 jerinj
2019-12-20  3:26 ` Gavin Hu
2019-12-20  3:45   ` Jerin Jacob
2019-12-20 10:54     ` [dpdk-stable] [dpdk-dev] [PATCH] mempool: fix mempool obj alignment for nonx86 Morten Brørup
2019-12-20 15:55 ` [dpdk-stable] [dpdk-dev] [PATCH] mempool: fix mempool obj alignment for non x86 Honnappa Nagarahalli
2019-12-20 16:55   ` Jerin Jacob
2019-12-20 21:07     ` Honnappa Nagarahalli
2019-12-21  5:06       ` Jerin Jacob [this message]
2019-12-27 15:54         ` Olivier Matz
2020-01-11 13:34 ` [dpdk-stable] [dpdk-dev] [PATCH v2] " jerinj
2020-01-11 17:41   ` Stephen Hemminger
2020-01-13  6:49   ` [dpdk-stable] [dpdk-dev] [PATCH v3] " jerinj
2020-01-13  9:46     ` David Marchand
2020-01-13 11:46       ` [dpdk-stable] [EXT] " Jerin Jacob Kollanukkaran

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CALBAE1MnmfGuNN86K3czpsTrEns2X7m0rd5+DcwzBzicpeTR5A@mail.gmail.com \
    --to=jerinjacobk@gmail.com \
    --cc=Gavin.Hu@arm.com \
    --cc=Honnappa.Nagarahalli@arm.com \
    --cc=anatoly.burakov@intel.com \
    --cc=arybchenko@solarflare.com \
    --cc=bruce.richardson@intel.com \
    --cc=dev@dpdk.org \
    --cc=drc@linux.vnet.ibm.com \
    --cc=hemant.agrawal@nxp.com \
    --cc=jerinj@marvell.com \
    --cc=konstantin.ananyev@intel.com \
    --cc=nd@arm.com \
    --cc=olivier.matz@6wind.com \
    --cc=shahafs@mellanox.com \
    --cc=stable@dpdk.org \
    --cc=thomas@monjalon.net \
    --cc=viktorin@rehivetech.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).