Re: [dpdk-dev] [dpdk-stable] [PATCH v4] lib/table: fix cache alignment issue

["Dumitrescu, Cristian" <cristian.dumitrescu@intel.com>]

> -----Original Message-----
> From: David Marchand <david.marchand@redhat.com>
> Sent: Wednesday, July 29, 2020 3:00 PM
> To: Dumitrescu, Cristian <cristian.dumitrescu@intel.com>
> Cc: Xu, Ting <ting.xu@intel.com>; dev <dev@dpdk.org>; dpdk stable
> <stable@dpdk.org>; Kevin Traynor <ktraynor@redhat.com>; Luca Boccassi
> <bluca@debian.org>
> Subject: Re: [dpdk-stable] [dpdk-dev] [PATCH v4] lib/table: fix cache
> alignment issue
> 
> On Wed, Jul 29, 2020 at 3:54 PM Dumitrescu, Cristian
> <cristian.dumitrescu@intel.com> wrote:
> > > -----Original Message-----
> > > From: David Marchand <david.marchand@redhat.com>
> > > Sent: Wednesday, July 29, 2020 2:28 PM
> > > To: Dumitrescu, Cristian <cristian.dumitrescu@intel.com>
> > > Cc: Xu, Ting <ting.xu@intel.com>; dev <dev@dpdk.org>; dpdk stable
> > > <stable@dpdk.org>; Kevin Traynor <ktraynor@redhat.com>; Luca
> Boccassi
> > > <bluca@debian.org>
> > > Subject: Re: [dpdk-stable] [dpdk-dev] [PATCH v4] lib/table: fix cache
> > > alignment issue
> > >
> > > On Wed, Jul 29, 2020 at 3:14 PM Dumitrescu, Cristian
> > > <cristian.dumitrescu@intel.com> wrote:
> > > > > Please correct me if I am wrong, but it simply means this part of the
> > > > > table library never worked for 32-bit.
> > > > > It seems more adding 32-bit support rather than a fix and then I
> > > > > wonder if it has its place in rc3.
> > > > >
> > > >
> > > > Functionally. the code works, but performance is affected.
> > > >
> > > > The only thing that prevents the code from working is the check in the
> > > table create function that checks the size of the above structure is 64
> bytes,
> > > which caught this issue.
> > >
> > > Yes, and that's my point.
> > > It was not working.
> > > It was not tested.
> > >
> > >
> >
> > Not sure when this code was last tested on 32-bit systems, I'll let the
> validation folks comment on this, but I cannot rule out a change in compiler
> behavior either.
> >
> > This is a low complexity and low impact change, hence low risk IMO.
> 
> Risk is to be evaluated when there is a need.
> I got pinged on this, like it was the end of the times.
> 
> Then I find something that is not worth looking at, hence I am a bit irritated.
> 

I got pinged as well, and I also had to allocate time on this patch. It probably means it is important for somebody.

> And please, for the 2nd time, can you look at my comment below?
> 
Sorry, I missed it first.

> 
> > > > > > diff --git a/lib/librte_table/rte_table_hash_key16.c
> > > > > b/lib/librte_table/rte_table_hash_key16.c
> > > > > > index 2cca1c924..c4384b114 100644
> > > > > > --- a/lib/librte_table/rte_table_hash_key16.c
> > > > > > +++ b/lib/librte_table/rte_table_hash_key16.c
> > > > > > @@ -33,6 +33,7 @@
> > > > > >
> > > > > >  #endif
> > > > > >
> > > > > > +#ifdef RTE_ARCH_64
> > > > > >  struct rte_bucket_4_16 {
> > > > > >         /* Cache line 0 */
> > > > > >         uint64_t signature[4 + 1];
> > > > > > @@ -46,6 +47,22 @@ struct rte_bucket_4_16 {
> > > > > >         /* Cache line 2 */
> > > > > >         uint8_t data[0];
> > > > > >  };
> > > > > > +#else
> > > > > > +struct rte_bucket_4_16 {
> > > > > > +       /* Cache line 0 */
> > > > > > +       uint64_t signature[4 + 1];
> > > > > > +       uint64_t lru_list;
> > > > > > +       struct rte_bucket_4_16 *next;
> > > > > > +       uint32_t pad;
> > > > > > +       uint64_t next_valid;
> > > > > > +
> > > > > > +       /* Cache line 1 */
> > > > > > +       uint64_t key[4][2];
> > > > > > +
> > > > > > +       /* Cache line 2 */
> > > > > > +       uint8_t data[0];
> > > > > > +};
> > > > > > +#endif
> > > > >
> > > > > The change could simply be:
> > > > >
> > > > > @@ -38,6 +38,9 @@ struct rte_bucket_4_16 {
> > > > >         uint64_t signature[4 + 1];
> > > > >         uint64_t lru_list;
> > > > >         struct rte_bucket_4_16 *next;
> > > > > +#ifndef RTE_ARCH_64
> > > > > +       uint32_t pad;
> > > > > +#endif
> > > > >         uint64_t next_valid;
> > > > >
> > > > >         /* Cache line 1 */
> > > > >
> > > > > It avoids duplicating the whole structure definition (we could miss
> > > > > updating one side of the #ifdef later).
> > > > > Idem for the other "8" and "32" structures.
> > >
> > >
> > > What about this comment?
> 
> What about this comment?
> 

You might suspect I also thought about this option. My preference is for the option in the patch for the reasons that IMO it is easier to read and understand the reason for the difference, even though the code is slightly larger. It also leaves the 64-bit code untouched, so it is easier to remove when we finally decide at some point to drop the 32-bit support.

But I can live with the option you describe as well. Thanks for the input.

For me, it would be great if somebody on this list could indicate why the 4-byte padding was not inserted by the compiler automatically, and hence the need for this fix.

> 
> --
> David Marchand