From: Konstantin Ananyev <konstantin.ananyev@huawei.com>
To: Bruce Richardson <bruce.richardson@intel.com>,
"Varghese, Vipin" <Vipin.Varghese@amd.com>
Cc: "Mattias Rönnblom" <hofors@lysator.liu.se>,
"Yigit, Ferruh" <Ferruh.Yigit@amd.com>,
"dev@dpdk.org" <dev@dpdk.org>
Subject: RE: [RFC 0/2] introduce LLC aware functions
Date: Wed, 11 Sep 2024 22:25:41 +0000 [thread overview]
Message-ID: <5b0a3e6e91a3404d90224420e1cd758d@huawei.com> (raw)
In-Reply-To: <ZuG-8MiGBrHI4k-S@bricha3-mobl1.ger.corp.intel.com>
> > > >>> Thank you Mattias for the comments and question, please let me try
> > > >>> to explain the same below
> > > >>>
> > > >>>> We shouldn't have a separate CPU/cache hierarchy API instead?
> > > >>>
> > > >>> Based on the intention to bring in CPU lcores which share same L3
> > > >>> (for better cache hits and less noisy neighbor) current API focuses
> > > >>> on using
> > > >>>
> > > >>> Last Level Cache. But if the suggestion is `there are SoC where L2
> > > >>> cache are also shared, and the new API should be provisioned`, I am
> > > >>> also
> > > >>>
> > > >>> comfortable with the thought.
> > > >>>
> > > >>
> > > >> Rather than some AMD special case API hacked into <rte_lcore.h>, I
> > > >> think we are better off with no DPDK API at all for this kind of functionality.
> > > >
> > > > Hi Mattias, as shared in the earlier email thread, this is not a AMD special
> > > case at all. Let me try to explain this one more time. One of techniques used to
> > > increase cores cost effective way to go for tiles of compute complexes.
> > > > This introduces a bunch of cores in sharing same Last Level Cache (namely
> > > L2, L3 or even L4) depending upon cache topology architecture.
> > > >
> > > > The API suggested in RFC is to help end users to selectively use cores under
> > > same Last Level Cache Hierarchy as advertised by OS (irrespective of the BIOS
> > > settings used). This is useful in both bare-metal and container environment.
> > > >
> > >
> > > I'm pretty familiar with AMD CPUs and the use of tiles (including the
> > > challenges these kinds of non-uniformities pose for work scheduling).
> > >
> > > To maximize performance, caring about core<->LLC relationship may well not
> > > be enough, and more HT/core/cache/memory topology information is
> > > required. That's what I meant by special case. A proper API should allow
> > > access to information about which lcores are SMT siblings, cores on the same
> > > L2, and cores on the same L3, to name a few things. Probably you want to fit
> > > NUMA into the same API as well, although that is available already in
> > > <rte_lcore.h>.
> >
> > Thank you Mattias for the information, as shared by in the reply with Anatoly we want expose a new API `rte_get_next_lcore_ex`
> which intakes a extra argument `u32 flags`.
> > The flags can be RTE_GET_LCORE_L1 (SMT), RTE_GET_LCORE_L2, RTE_GET_LCORE_L3, RTE_GET_LCORE_BOOST_ENABLED,
> RTE_GET_LCORE_BOOST_DISABLED.
> >
>
> For the naming, would "rte_get_next_sibling_core" (or lcore if you prefer)
> be a clearer name than just adding "ex" on to the end of the existing
> function?
>
> Looking logically, I'm not sure about the BOOST_ENABLED and BOOST_DISABLED
> flags you propose - in a system with multiple possible standard and boost
> frequencies what would those correspond to? What's also missing is a define
> for getting actual NUMA siblings i.e. those sharing common memory but not
> an L3 or anything else.
>
> My suggestion would be to have the function take just an integer-type e.g.
> uint16_t parameter which defines the memory/cache hierarchy level to use, 0
> being lowest, 1 next, and so on. Different systems may have different
> numbers of cache levels so lets just make it a zero-based index of levels,
> rather than giving explicit defines (except for memory which should
> probably always be last). The zero-level will be for "closest neighbour"
> whatever that happens to be, with as many levels as is necessary to express
> the topology, e.g. without SMT, but with 3 cache levels, level 0 would be
> an L2 neighbour, level 1 an L3 neighbour. If the L3 was split within a
> memory NUMA node, then level 2 would give the NUMA siblings. We'd just need
> an API to return the max number of levels along with the iterator.
Sounds like a neat idea to me.
next prev parent reply other threads:[~2024-09-11 22:25 UTC|newest]
Thread overview: 53+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-08-27 15:10 Vipin Varghese
2024-08-27 15:10 ` [RFC 1/2] eal: add llc " Vipin Varghese
2024-08-27 17:36 ` Stephen Hemminger
2024-09-02 0:27 ` Varghese, Vipin
2024-08-27 20:56 ` Wathsala Wathawana Vithanage
2024-08-29 3:21 ` 答复: " Feifei Wang
2024-09-02 1:20 ` Varghese, Vipin
2024-09-03 17:54 ` Wathsala Wathawana Vithanage
2024-09-04 8:18 ` Bruce Richardson
2024-09-06 11:59 ` Varghese, Vipin
2024-09-12 16:58 ` Wathsala Wathawana Vithanage
2024-08-27 15:10 ` [RFC 2/2] eal/lcore: add llc aware for each macro Vipin Varghese
2024-08-27 21:23 ` [RFC 0/2] introduce LLC aware functions Mattias Rönnblom
2024-09-02 0:39 ` Varghese, Vipin
2024-09-04 9:30 ` Mattias Rönnblom
2024-09-04 14:37 ` Stephen Hemminger
2024-09-11 3:13 ` Varghese, Vipin
2024-09-11 3:53 ` Stephen Hemminger
2024-09-12 1:11 ` Varghese, Vipin
2024-09-09 14:22 ` Varghese, Vipin
2024-09-09 14:52 ` Mattias Rönnblom
2024-09-11 3:26 ` Varghese, Vipin
2024-09-11 15:55 ` Mattias Rönnblom
2024-09-11 17:04 ` Honnappa Nagarahalli
2024-09-12 1:33 ` Varghese, Vipin
2024-09-12 6:38 ` Mattias Rönnblom
2024-09-12 7:02 ` Mattias Rönnblom
2024-09-12 11:23 ` Varghese, Vipin
2024-09-12 12:12 ` Mattias Rönnblom
2024-09-12 15:50 ` Stephen Hemminger
2024-09-12 11:17 ` Varghese, Vipin
2024-09-12 11:59 ` Mattias Rönnblom
2024-09-12 13:30 ` Bruce Richardson
2024-09-12 16:32 ` Mattias Rönnblom
2024-09-12 2:28 ` Varghese, Vipin
2024-09-11 16:01 ` Bruce Richardson
2024-09-11 22:25 ` Konstantin Ananyev [this message]
2024-09-12 2:38 ` Varghese, Vipin
2024-09-12 2:19 ` Varghese, Vipin
2024-09-12 9:17 ` Bruce Richardson
2024-09-12 11:50 ` Varghese, Vipin
2024-09-13 14:15 ` Burakov, Anatoly
2024-09-12 13:18 ` Mattias Rönnblom
2024-08-28 8:38 ` Burakov, Anatoly
2024-09-02 1:08 ` Varghese, Vipin
2024-09-02 14:17 ` Burakov, Anatoly
2024-09-02 15:33 ` Varghese, Vipin
2024-09-03 8:50 ` Burakov, Anatoly
2024-09-05 13:05 ` Ferruh Yigit
2024-09-05 14:45 ` Burakov, Anatoly
2024-09-05 15:34 ` Ferruh Yigit
2024-09-06 8:44 ` Burakov, Anatoly
2024-09-09 14:14 ` Varghese, Vipin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=5b0a3e6e91a3404d90224420e1cd758d@huawei.com \
--to=konstantin.ananyev@huawei.com \
--cc=Ferruh.Yigit@amd.com \
--cc=Vipin.Varghese@amd.com \
--cc=bruce.richardson@intel.com \
--cc=dev@dpdk.org \
--cc=hofors@lysator.liu.se \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).