Re: [dpdk-dev] [dpdk-stable] [PATCH v4] lib/table: fix cache alignment issue

["Xu, Ting" <ting.xu@intel.com>]

Hi, all,

> -----Original Message-----
> From: Dumitrescu, Cristian <cristian.dumitrescu@intel.com>
> Sent: Wednesday, July 29, 2020 10:53 PM
> To: David Marchand <david.marchand@redhat.com>
> Cc: Xu, Ting <ting.xu@intel.com>; dev <dev@dpdk.org>; dpdk stable
> <stable@dpdk.org>; Kevin Traynor <ktraynor@redhat.com>; Luca Boccassi
> <bluca@debian.org>
> Subject: RE: [dpdk-stable] [dpdk-dev] [PATCH v4] lib/table: fix cache alignment
> issue
> 
> 
> 
> > -----Original Message-----
> > From: David Marchand <david.marchand@redhat.com>
> > Sent: Wednesday, July 29, 2020 3:00 PM
> > To: Dumitrescu, Cristian <cristian.dumitrescu@intel.com>
> > Cc: Xu, Ting <ting.xu@intel.com>; dev <dev@dpdk.org>; dpdk stable
> > <stable@dpdk.org>; Kevin Traynor <ktraynor@redhat.com>; Luca Boccassi
> > <bluca@debian.org>
> > Subject: Re: [dpdk-stable] [dpdk-dev] [PATCH v4] lib/table: fix cache
> > alignment issue
> >
> > On Wed, Jul 29, 2020 at 3:54 PM Dumitrescu, Cristian
> > <cristian.dumitrescu@intel.com> wrote:
> > > > -----Original Message-----
> > > > From: David Marchand <david.marchand@redhat.com>
> > > > Sent: Wednesday, July 29, 2020 2:28 PM
> > > > To: Dumitrescu, Cristian <cristian.dumitrescu@intel.com>
> > > > Cc: Xu, Ting <ting.xu@intel.com>; dev <dev@dpdk.org>; dpdk stable
> > > > <stable@dpdk.org>; Kevin Traynor <ktraynor@redhat.com>; Luca
> > Boccassi
> > > > <bluca@debian.org>
> > > > Subject: Re: [dpdk-stable] [dpdk-dev] [PATCH v4] lib/table: fix
> > > > cache alignment issue
> > > >
> > > > On Wed, Jul 29, 2020 at 3:14 PM Dumitrescu, Cristian
> > > > <cristian.dumitrescu@intel.com> wrote:
> > > > > > Please correct me if I am wrong, but it simply means this part
> > > > > > of the table library never worked for 32-bit.
> > > > > > It seems more adding 32-bit support rather than a fix and then
> > > > > > I wonder if it has its place in rc3.
> > > > > >
> > > > >
> > > > > Functionally. the code works, but performance is affected.
> > > > >
> > > > > The only thing that prevents the code from working is the check
> > > > > in the
> > > > table create function that checks the size of the above structure
> > > > is 64
> > bytes,
> > > > which caught this issue.
> > > >
> > > > Yes, and that's my point.
> > > > It was not working.
> > > > It was not tested.
> > > >
> > > >
> > >
> > > Not sure when this code was last tested on 32-bit systems, I'll let
> > > the
> > validation folks comment on this, but I cannot rule out a change in
> > compiler behavior either.
> > >
> > > This is a low complexity and low impact change, hence low risk IMO.
> >
> > Risk is to be evaluated when there is a need.
> > I got pinged on this, like it was the end of the times.
> >
> > Then I find something that is not worth looking at, hence I am a bit irritated.
> >
> 
> I got pinged as well, and I also had to allocate time on this patch. It probably
> means it is important for somebody.
> 
> > And please, for the 2nd time, can you look at my comment below?
> >
> Sorry, I missed it first.
> 
> >
> > > > > > > diff --git a/lib/librte_table/rte_table_hash_key16.c
> > > > > > b/lib/librte_table/rte_table_hash_key16.c
> > > > > > > index 2cca1c924..c4384b114 100644
> > > > > > > --- a/lib/librte_table/rte_table_hash_key16.c
> > > > > > > +++ b/lib/librte_table/rte_table_hash_key16.c
> > > > > > > @@ -33,6 +33,7 @@
> > > > > > >
> > > > > > >  #endif
> > > > > > >
> > > > > > > +#ifdef RTE_ARCH_64
> > > > > > >  struct rte_bucket_4_16 {
> > > > > > >         /* Cache line 0 */
> > > > > > >         uint64_t signature[4 + 1]; @@ -46,6 +47,22 @@ struct
> > > > > > > rte_bucket_4_16 {
> > > > > > >         /* Cache line 2 */
> > > > > > >         uint8_t data[0];
> > > > > > >  };
> > > > > > > +#else
> > > > > > > +struct rte_bucket_4_16 {
> > > > > > > +       /* Cache line 0 */
> > > > > > > +       uint64_t signature[4 + 1];
> > > > > > > +       uint64_t lru_list;
> > > > > > > +       struct rte_bucket_4_16 *next;
> > > > > > > +       uint32_t pad;
> > > > > > > +       uint64_t next_valid;
> > > > > > > +
> > > > > > > +       /* Cache line 1 */
> > > > > > > +       uint64_t key[4][2];
> > > > > > > +
> > > > > > > +       /* Cache line 2 */
> > > > > > > +       uint8_t data[0];
> > > > > > > +};
> > > > > > > +#endif
> > > > > >
> > > > > > The change could simply be:
> > > > > >
> > > > > > @@ -38,6 +38,9 @@ struct rte_bucket_4_16 {
> > > > > >         uint64_t signature[4 + 1];
> > > > > >         uint64_t lru_list;
> > > > > >         struct rte_bucket_4_16 *next;
> > > > > > +#ifndef RTE_ARCH_64
> > > > > > +       uint32_t pad;
> > > > > > +#endif
> > > > > >         uint64_t next_valid;
> > > > > >
> > > > > >         /* Cache line 1 */
> > > > > >
> > > > > > It avoids duplicating the whole structure definition (we could
> > > > > > miss updating one side of the #ifdef later).
> > > > > > Idem for the other "8" and "32" structures.
> > > >
> > > >
> > > > What about this comment?
> >
> > What about this comment?
> >
> 
> You might suspect I also thought about this option. My preference is for the
> option in the patch for the reasons that IMO it is easier to read and
> understand the reason for the difference, even though the code is slightly
> larger. It also leaves the 64-bit code untouched, so it is easier to remove when
> we finally decide at some point to drop the 32-bit support.
> 
> But I can live with the option you describe as well. Thanks for the input.
> 
> For me, it would be great if somebody on this list could indicate why the 4-
> byte padding was not inserted by the compiler automatically, and hence the
> need for this fix.
> 

Thanks for your help and additional works on this patch.
The validation team tested this case in a 32-bit environment, besides, there are a series of similar tests in 32-bit environment as well. There might be some practical needs for this.
Therefore, before we decide to drop 32-bit support formally, I think such modification is OK, if we cannot fix the compiler issue directly.

Shall I update the patch as David suggested to make it simpler?

> >
> > --
> > David Marchand