From: "Dumitrescu, Cristian" <cristian.dumitrescu@intel.com>
To: "Xu, Ting" <ting.xu@intel.com>, "dev@dpdk.org" <dev@dpdk.org>
Cc: "stable@dpdk.org" <stable@dpdk.org>
Subject: Re: [dpdk-dev] [PATCH v3] lib/table: fix cache alignment issue
Date: Tue, 21 Jul 2020 21:16:44 +0000 [thread overview]
Message-ID: <BYAPR11MB2935CB2AA722EA1C8D1B2386EB780@BYAPR11MB2935.namprd11.prod.outlook.com> (raw)
In-Reply-To: <CY4PR1101MB2310C59B19F5CD8C2E1D60FFF8780@CY4PR1101MB2310.namprd11.prod.outlook.com>
> -----Original Message-----
> From: Xu, Ting <ting.xu@intel.com>
> Sent: Tuesday, July 21, 2020 6:16 AM
> To: Dumitrescu, Cristian <cristian.dumitrescu@intel.com>; dev@dpdk.org
> Cc: stable@dpdk.org
> Subject: RE: [PATCH v3] lib/table: fix cache alignment issue
>
> Hi, Cristian
>
> > -----Original Message-----
> > From: Dumitrescu, Cristian <cristian.dumitrescu@intel.com>
> > Sent: Monday, July 20, 2020 10:38 PM
> > To: Xu, Ting <ting.xu@intel.com>; dev@dpdk.org
> > Cc: stable@dpdk.org
> > Subject: RE: [PATCH v3] lib/table: fix cache alignment issue
> >
> >
> >
> > > -----Original Message-----
> > > From: Xu, Ting <ting.xu@intel.com>
> > > Sent: Thursday, July 9, 2020 2:48 AM
> > > To: dev@dpdk.org
> > > Cc: Dumitrescu, Cristian <cristian.dumitrescu@intel.com>; Xu, Ting
> > > <ting.xu@intel.com>; stable@dpdk.org
> > > Subject: [PATCH v3] lib/table: fix cache alignment issue
> > >
> > > When create softnic hash table with 16 keys, it failed on 32bit
> > > environment because of the structure rte_bucket_4_16 alignment issue.
> > > Add __rte_cache_aligned to ensure correct cache align.
> > >
> > > Fixes: 8aa327214c ("table: hash")
> > > Cc: stable@dpdk.org
> > >
> > > Signed-off-by: Ting Xu <ting.xu@intel.com>
> > >
> > > ---
> > > v2->v3: Rebase
> > > v1->v2: Correct patch time
> > > ---
> > > lib/librte_table/rte_table_hash_key16.c | 2 +-
> > > 1 file changed, 1 insertion(+), 1 deletion(-)
> > >
> > > diff --git a/lib/librte_table/rte_table_hash_key16.c
> > > b/lib/librte_table/rte_table_hash_key16.c
> > > index 2cca1c924..5e1665c15 100644
> > > --- a/lib/librte_table/rte_table_hash_key16.c
> > > +++ b/lib/librte_table/rte_table_hash_key16.c
> > > @@ -44,7 +44,7 @@ struct rte_bucket_4_16 {
> > > uint64_t key[4][2];
> > >
> > > /* Cache line 2 */
> > > - uint8_t data[0];
> > > + uint8_t data[0] __rte_cache_aligned;
> > > };
> > >
> > > struct rte_table_hash {
> > > --
> > > 2.17.1
> >
> > Hi Ting,
> >
> > This fix is breaking the execution for systems with cache line of 128 bytes,
> as
> > typically (on 64-bit systems) this structure would be 64-byte in size and
> > adding the __rte_cache_aligned would force doubling the size of this
> > structure through padding enforced by the compiler.
> >
> > Can you please confirm this is caused by check below failing in the table
> > create function:
> > sizeof(struct rte_bucket_4_16) % 64) != 0
> >
>
> The result of sizeof(struct rte_bucket_4_16) is 124 byte in this case, and this
> is the direct reason causing this failure.
>
> > Since all the other fields in this data structure are explicitly declared as 64-
> bit
> > fields, due to the alignment rules I was expecting the compiler to add a 32-
> bit
> > padding field after the "next" field, which is a pointer that would only take
> 32
> > bits on 32-bit systems. I am not sure why this did not take place in your
> case,
> > any thoughts?
> >
>
> It shows that the size of the field struct rte_bucket_4_16 *next in struct
> rte_bucket_4_16 is only 32 bits. And there is no padding added by the
> compiler in my and the reporter's case.
> I tried to add a 32 bits pad field after the field next manually, and the result is
> correct then.
> Is it because in 32-bit system, the compiler will not extend the 32 bits pointer
> to 64 bits, since the 32 bits size has already matched the cache line?
>
> > Not sure why we would run Soft NIC on 32-bit systems, might be better to
> > disable Soft NIC for 32-bit systems.
> >
>
My proposed solution, which IMO provides the cleanest and most readable way to fix / maintain this code:
#ifdef RTE_ARCH_64
struct rte_bucket_4_16 {
//current definition of this struct
};
#else
struct rte_bucket_4_16 {
//definition with padding fields for the 32-bit pointers to keep this struct to a multiple of 64 bytes
};
#endif
We need to apply the same for 8-byte key and 32-byte key hash functions from the same folder.
What do you think, Ting?
> To be honest, I do not know why we should run softnic on 32-bit system, I
> was just assigned this specific bug. It seems there is a complete test case for
> validation team to test softnic in 32-bit system.
> I am not sure is it OK to tell the validation team that we should disable softnic
> in 32-bit system now. Or we should fix this issue this time and discuss about
> the problem later?
>
> Thanks!
>
> > Thanks,
> > Cristian
next prev parent reply other threads:[~2020-07-21 21:16 UTC|newest]
Thread overview: 26+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-06-16 16:27 [dpdk-dev] [PATCH v1] " Ting Xu
2020-06-17 5:43 ` [dpdk-dev] [PATCH v2] " Ting Xu
2020-07-02 8:06 ` Zhou, JunX W
2020-07-09 1:48 ` [dpdk-dev] [PATCH v3] " Ting Xu
2020-07-20 14:37 ` Dumitrescu, Cristian
2020-07-21 5:15 ` Xu, Ting
2020-07-21 21:16 ` Dumitrescu, Cristian [this message]
2020-07-22 2:16 ` Xu, Ting
2020-07-22 2:16 ` [dpdk-dev] [PATCH v4] " Ting Xu
2020-07-22 8:26 ` Dumitrescu, Cristian
2020-07-22 8:30 ` Xu, Ting
2020-07-22 8:49 ` Dumitrescu, Cristian
2020-07-22 8:48 ` Dumitrescu, Cristian
2020-07-29 12:01 ` David Marchand
2020-07-29 13:13 ` Dumitrescu, Cristian
2020-07-29 13:28 ` [dpdk-dev] [dpdk-stable] " David Marchand
2020-07-29 13:54 ` Dumitrescu, Cristian
2020-07-29 13:59 ` David Marchand
2020-07-29 14:53 ` Dumitrescu, Cristian
2020-07-30 6:57 ` Xu, Ting
2020-07-30 10:35 ` Kevin Traynor
2020-09-09 6:18 ` Xu, Ting
2020-09-15 8:03 ` David Marchand
2020-10-14 8:26 ` Xu, Ting
2020-10-14 13:53 ` [dpdk-dev] " David Marchand
2020-07-09 1:44 [dpdk-dev] [PATCH v3] " Ting Xu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=BYAPR11MB2935CB2AA722EA1C8D1B2386EB780@BYAPR11MB2935.namprd11.prod.outlook.com \
--to=cristian.dumitrescu@intel.com \
--cc=dev@dpdk.org \
--cc=stable@dpdk.org \
--cc=ting.xu@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).