RE: [RFC v3 1/3] eal/lcore: add topology based functions

["Varghese, Vipin" <Vipin.Varghese@amd.com>]

Snipped

> > >
> > > I recall from the Cache Stashing community call... There is some
> > > ACPI
> > function to
> > > get the (opaque) "location IDs" of various parts in the system, to
> > > be
> > used for setting
> > > the Cache Stashing hints.
> > > Is there only one "ACPI location ID" (I don't know the correct name)
> > shared by the
> > > L3 cache and the v-cache in AMD EPYC, or do they have each their own?
> >
> > At least on AMD EPYC, the stashing ID updated to either MSI-X table or
> > Device Specific Mode is core-id.
> 
> Are you saying that on AMD EPYC only L2 caches have a Stashing ID, so no other
> CPU caches can be stashed into?
On AMD EPYC zen4 (limited Operating Part Number) & zen5, Cache Stashing is done on L2 cache level.
This is done by passing core-id as the steering tag (opaque value)

> If yes, then it's a non-issue for Cache Stashing, since it doesn't need to care about
> L3 cache or v-cache.
Yes, that is what I am trying to imply irrespective of the platform (Arm, powerpc, riscv, Intel or AMD) cache
stashing based on PCIe SIG is based on each vendor implementation. On AMD EPYC currently this is on based 
on core-id (which is the hint to platform to put it into L2 cache). 

Other platforms might support putting to L1, L2 or L3. And for these I agree that cache id might be used steering tag.

> 
> >
> >
> > > If they are not exposed as one ID, but two separate IDs, the
> > > Topology
> > API might
> > > need to reflect this, so it can be used in the Cache Stashing API.
> >
> > I have different view on the same and had shared this with Ajit
> > (Broadcom) and others. To my understanding, use of rte_ethdev API used
> > for caching hints should be inline to rte_lcore. Depending upon the
> > platform (ARM's specific implementation, the lcore gets translated to
> > L2 or L3 cache ID within the PMD.
> 
> The rte_ethdev API for cache stashing provides a higher level of abstraction, yes.
> 
> But the layer below it - the Stashing API used by the PMDs to obtain Stashing ID
> from "location ID" - could use the "location ID" structure type defined by the
> Topology library's lower layer.

Yes, consume the lcore-id from the user via ethdev API, internally the PMD translates this..
Based on my current understanding this can be done in two ways

1. the translation is done using hwloc library API calls
2. using rte_topology structure during probing, also probe for cache id for L1, L2, L3 and L4.

To do achieve this, one has to add `unsigned int cache_id` to `struct core_domain_mapping`.
This allows `rte_eal_topology_init` probe to store the cache_id.

> 
> >
> > Note:  the current patch introduces of Topology aware grouping, which
> > helps to run application better or tiles or chiplets sharing same
> > L2|L3 or IO domain.
> 
> Both libraries (Topology and Cache Stashing) need to have detailed information
> about the hardware, although they use the information for two different purposes.
> 
> Maybe they could share a common lower layer for the system topology, perhaps
> just a few header files. Or maybe the Cache Stashing library should depend on the
> Topology library as its "lower layer" to provide the hardware information it needs.
> 
> I'm not saying that it must be so. I'm only saying that I suppose these two libraries
> have a lot in common, and they could try to take advantage of this, to provide a
> more uniform API at their lower layers.

Yes I agree, my only case here let us first get rte_topology into dpdk eco-system.
Since the ` struct core_domain_mapping` can be easily updated this can be done in next step.

Note: assuming we can merge this for release 25.03, and we all concur or `cache stashing API` we get it tested on AMD EPYC and other platforms alike.

I will share v4 by today.