[dpdk-dev] [PATCH] net/mlx5: fix linkage error for glue lib

[dpdk-dev] [PATCH] net/mlx5: fix linkage error for glue lib

[Shahaf Shuler]

From: Yaroslav Brustinov <ybrustin@cisco.com>

Compiling with gcc 4.7.2 introduced the linkage error

"bin/ld: Warning: alignment 8 of symbol `mlx5_glue' in
src/dpdk/drivers/net/mlx5/mlx5_glue.c.21.o is smaller than 16 in
src/dpdk/drivers/net/mlx5/mlx5_rxq.c.21.o"

Fix it be forcing the alignment of the glue lib.

Fixes: 0e83b8e536c1 ("net/mlx5: move rdma-core calls to separate file")
Cc: stable@dpdk.org
Cc: nelio.laranjeiro@6wind.com
Cc: adrien.mazarguil@6wind.com

Signed-off-by: Yaroslav Brustinov <ybrustin@cisco.com>
Signed-off-by: Shahaf Shuler <shahafs@mellanox.com>
---
 drivers/net/mlx5/mlx5_glue.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/drivers/net/mlx5/mlx5_glue.c b/drivers/net/mlx5/mlx5_glue.c
index c7965e51fe..c56c69bb13 100644
--- a/drivers/net/mlx5/mlx5_glue.c
+++ b/drivers/net/mlx5/mlx5_glue.c
@@ -343,7 +343,9 @@ mlx5_glue_dv_create_qp(struct ibv_context *context,
 #endif
 }
 
-const struct mlx5_glue *mlx5_glue = &(const struct mlx5_glue){
+const struct mlx5_glue *mlx5_glue __attribute__((__aligned__(64))) =
+						&(const struct mlx5_glue)
+{
 	.version = MLX5_GLUE_VERSION,
 	.fork_init = mlx5_glue_fork_init,
 	.alloc_pd = mlx5_glue_alloc_pd,
-- 
2.12.0

Re: [dpdk-dev] [PATCH] net/mlx5: fix linkage error for glue lib

[Adrien Mazarguil]

On Tue, Jul 24, 2018 at 11:54:45AM +0300, Shahaf Shuler wrote:
> From: Yaroslav Brustinov <ybrustin@cisco.com>
> 
> Compiling with gcc 4.7.2 introduced the linkage error
> 
> "bin/ld: Warning: alignment 8 of symbol `mlx5_glue' in
> src/dpdk/drivers/net/mlx5/mlx5_glue.c.21.o is smaller than 16 in
> src/dpdk/drivers/net/mlx5/mlx5_rxq.c.21.o"
> 
> Fix it be forcing the alignment of the glue lib.
> 
> Fixes: 0e83b8e536c1 ("net/mlx5: move rdma-core calls to separate file")
> Cc: stable@dpdk.org
> Cc: nelio.laranjeiro@6wind.com
> Cc: adrien.mazarguil@6wind.com
> 
> Signed-off-by: Yaroslav Brustinov <ybrustin@cisco.com>
> Signed-off-by: Shahaf Shuler <shahafs@mellanox.com>

Odd, how could this happen assuming both files are compiled during the same
run using identical flags? Looks like a compiler issue.

Anyway, may I suggest an alignment constraint on the structure type in
mlx5_glue.h instead, so that all users inherit it. E.g. using C11 syntax:

 #include <stdalign.h>
 #include <stddef.h>

 [...]
 alignas(max_align_t)
 struct mlx5_glue {
     [...]
 };

> ---
>  drivers/net/mlx5/mlx5_glue.c | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/net/mlx5/mlx5_glue.c b/drivers/net/mlx5/mlx5_glue.c
> index c7965e51fe..c56c69bb13 100644
> --- a/drivers/net/mlx5/mlx5_glue.c
> +++ b/drivers/net/mlx5/mlx5_glue.c
> @@ -343,7 +343,9 @@ mlx5_glue_dv_create_qp(struct ibv_context *context,
>  #endif
>  }
>  
> -const struct mlx5_glue *mlx5_glue = &(const struct mlx5_glue){
> +const struct mlx5_glue *mlx5_glue __attribute__((__aligned__(64))) =
> +						&(const struct mlx5_glue)
> +{
>  	.version = MLX5_GLUE_VERSION,
>  	.fork_init = mlx5_glue_fork_init,
>  	.alloc_pd = mlx5_glue_alloc_pd,
> -- 
> 2.12.0
> 

-- 
Adrien Mazarguil
6WIND

Re: [dpdk-dev] [PATCH] net/mlx5: fix linkage error for glue lib

[Adrien Mazarguil]

On Tue, Jul 24, 2018 at 02:00:48PM +0200, Adrien Mazarguil wrote:
> On Tue, Jul 24, 2018 at 11:54:45AM +0300, Shahaf Shuler wrote:
> > From: Yaroslav Brustinov <ybrustin@cisco.com>
> > 
> > Compiling with gcc 4.7.2 introduced the linkage error
> > 
> > "bin/ld: Warning: alignment 8 of symbol `mlx5_glue' in
> > src/dpdk/drivers/net/mlx5/mlx5_glue.c.21.o is smaller than 16 in
> > src/dpdk/drivers/net/mlx5/mlx5_rxq.c.21.o"
> > 
> > Fix it be forcing the alignment of the glue lib.
> > 
> > Fixes: 0e83b8e536c1 ("net/mlx5: move rdma-core calls to separate file")
> > Cc: stable@dpdk.org
> > Cc: nelio.laranjeiro@6wind.com
> > Cc: adrien.mazarguil@6wind.com
> > 
> > Signed-off-by: Yaroslav Brustinov <ybrustin@cisco.com>
> > Signed-off-by: Shahaf Shuler <shahafs@mellanox.com>
> 
> Odd, how could this happen assuming both files are compiled during the same
> run using identical flags? Looks like a compiler issue.
> 
> Anyway, may I suggest an alignment constraint on the structure type in
> mlx5_glue.h instead, so that all users inherit it. E.g. using C11 syntax:
> 
>  #include <stdalign.h>
>  #include <stddef.h>
> 
>  [...]
>  alignas(max_align_t)
>  struct mlx5_glue {
>      [...]
>  };

My bad, this is not a correct use for alignas(), it doesn't work on types.
How about this instead:

 alignas(max_align_t)
 const struct mlx5_glue *mlx5_glue;

> 
> > ---
> >  drivers/net/mlx5/mlx5_glue.c | 4 +++-
> >  1 file changed, 3 insertions(+), 1 deletion(-)
> > 
> > diff --git a/drivers/net/mlx5/mlx5_glue.c b/drivers/net/mlx5/mlx5_glue.c
> > index c7965e51fe..c56c69bb13 100644
> > --- a/drivers/net/mlx5/mlx5_glue.c
> > +++ b/drivers/net/mlx5/mlx5_glue.c
> > @@ -343,7 +343,9 @@ mlx5_glue_dv_create_qp(struct ibv_context *context,
> >  #endif
> >  }
> >  
> > -const struct mlx5_glue *mlx5_glue = &(const struct mlx5_glue){
> > +const struct mlx5_glue *mlx5_glue __attribute__((__aligned__(64))) =
> > +						&(const struct mlx5_glue)
> > +{
> >  	.version = MLX5_GLUE_VERSION,
> >  	.fork_init = mlx5_glue_fork_init,
> >  	.alloc_pd = mlx5_glue_alloc_pd,
> > -- 
> > 2.12.0
> > 
> 
> -- 
> Adrien Mazarguil
> 6WIND

-- 
Adrien Mazarguil
6WIND

Re: [dpdk-dev] [PATCH] net/mlx5: fix linkage error for glue lib

[Shahaf Shuler]

Yaroslav, 

Tuesday, July 24, 2018 3:06 PM, Adrien Mazarguil:
> Subject: Re: [PATCH] net/mlx5: fix linkage error for glue lib
> > Odd, how could this happen assuming both files are compiled during the
> > same run using identical flags? Looks like a compiler issue.
> >
> > Anyway, may I suggest an alignment constraint on the structure type in
> > mlx5_glue.h instead, so that all users inherit it. E.g. using C11 syntax:
> >
> >  #include <stdalign.h>
> >  #include <stddef.h>
> >
> >  [...]
> >  alignas(max_align_t)
> >  struct mlx5_glue {
> >      [...]
> >  };
> 
> My bad, this is not a correct use for alignas(), it doesn't work on types.
> How about this instead:
> 
>  alignas(max_align_t)
>  const struct mlx5_glue *mlx5_glue;

Can you confirm the above suggestion fixes your issue? 

> 
> >

Re: [dpdk-dev] [PATCH] net/mlx5: fix linkage error for glue lib

[Yaroslav Brustinov (ybrustin)]

Hi,

Are you OK with C11?
I saw in file dpdk/drivers/net/mlx4/mlx4_utils.c:

     * C11 code would include stdalign.h and use alignof(max_align_t) however
     * we'll stick with C99 for the time being.

:)

As far as I understand, the issue is alignment of the pointer itself, not the struct.
I'm not familiar with this command: "alignof(max_align_t)".
Applying this on the struct:

    struct mlx5_glue {
    ...
    } __attribute__((__aligned__(64)));

    struct __attribute__((__aligned__(64))) mlx5_glue {
    ...
    };

...still keeps the linker unhappy.

Thanks,
Yaroslav.


-----Original Message-----
From: Shahaf Shuler [mailto:shahafs@mellanox.com] 
Sent: Tuesday, July 24, 2018 16:51
To: Adrien Mazarguil <adrien.mazarguil@6wind.com>
Cc: Yongseok Koh <yskoh@mellanox.com>; dev@dpdk.org; Yaroslav Brustinov (ybrustin) <ybrustin@cisco.com>; stable@dpdk.org; Nélio Laranjeiro <nelio.laranjeiro@6wind.com>
Subject: RE: [PATCH] net/mlx5: fix linkage error for glue lib

Yaroslav, 

Tuesday, July 24, 2018 3:06 PM, Adrien Mazarguil:
> Subject: Re: [PATCH] net/mlx5: fix linkage error for glue lib
> > Odd, how could this happen assuming both files are compiled during 
> > the same run using identical flags? Looks like a compiler issue.
> >
> > Anyway, may I suggest an alignment constraint on the structure type 
> > in mlx5_glue.h instead, so that all users inherit it. E.g. using C11 syntax:
> >
> >  #include <stdalign.h>
> >  #include <stddef.h>
> >
> >  [...]
> >  alignas(max_align_t)
> >  struct mlx5_glue {
> >      [...]
> >  };
> 
> My bad, this is not a correct use for alignas(), it doesn't work on types.
> How about this instead:
> 
>  alignas(max_align_t)
>  const struct mlx5_glue *mlx5_glue;

Can you confirm the above suggestion fixes your issue? 

> 
> >

Re: [dpdk-dev] [PATCH] net/mlx5: fix linkage error for glue lib

[Adrien Mazarguil]

On Tue, Jul 24, 2018 at 02:24:12PM +0000, Yaroslav Brustinov (ybrustin) wrote:
> Hi,
> 
> Are you OK with C11?
> I saw in file dpdk/drivers/net/mlx4/mlx4_utils.c:
> 
>      * C11 code would include stdalign.h and use alignof(max_align_t) however
>      * we'll stick with C99 for the time being.
> 
> :)

Hah, this code was originally intended to become a generic EAL function
hence the C99 but got rejected. Mellanox PMDs otherwise rely on C11 features
already.

> As far as I understand, the issue is alignment of the pointer itself, not the struct.
> I'm not familiar with this command: "alignof(max_align_t)".
> Applying this on the struct:
> 
>     struct mlx5_glue {
>     ...
>     } __attribute__((__aligned__(64)));
> 
>     struct __attribute__((__aligned__(64))) mlx5_glue {
>     ...
>     };
> 
> ...still keeps the linker unhappy.

Right, this was my first (wrong) suggestion that doesn't work on types. How
about the second one instead? Here's how the diff on mlx5_glue.h should look
like:

 +#include <stdalign.h>
  #include <stddef.h>
  #include <stdint.h>

 [...]
  
 +alignas(max_align_t)
  const struct mlx5_glue *mlx5_glue;

Another comment regarding this patch, commit log should probably mention it
addresses a GCC bug that cannot be reproduced with latter versions. Keep in
mind DPDK recommends to use at least GCC version 4.9.

> -----Original Message-----
> From: Shahaf Shuler [mailto:shahafs@mellanox.com] 
> Sent: Tuesday, July 24, 2018 16:51
> To: Adrien Mazarguil <adrien.mazarguil@6wind.com>
> Cc: Yongseok Koh <yskoh@mellanox.com>; dev@dpdk.org; Yaroslav Brustinov (ybrustin) <ybrustin@cisco.com>; stable@dpdk.org; Nélio Laranjeiro <nelio.laranjeiro@6wind.com>
> Subject: RE: [PATCH] net/mlx5: fix linkage error for glue lib
> 
> Yaroslav, 
> 
> Tuesday, July 24, 2018 3:06 PM, Adrien Mazarguil:
> > Subject: Re: [PATCH] net/mlx5: fix linkage error for glue lib
> > > Odd, how could this happen assuming both files are compiled during 
> > > the same run using identical flags? Looks like a compiler issue.
> > >
> > > Anyway, may I suggest an alignment constraint on the structure type 
> > > in mlx5_glue.h instead, so that all users inherit it. E.g. using C11 syntax:
> > >
> > >  #include <stdalign.h>
> > >  #include <stddef.h>
> > >
> > >  [...]
> > >  alignas(max_align_t)
> > >  struct mlx5_glue {
> > >      [...]
> > >  };
> > 
> > My bad, this is not a correct use for alignas(), it doesn't work on types.
> > How about this instead:
> > 
> >  alignas(max_align_t)
> >  const struct mlx5_glue *mlx5_glue;
> 
> Can you confirm the above suggestion fixes your issue? 
> 
> > 
> > >

-- 
Adrien Mazarguil
6WIND

Re: [dpdk-dev] [PATCH] net/mlx5: fix linkage error for glue lib

[Yaroslav Brustinov (ybrustin)]

+Hanoch

I've added -std=c11 flag to our compilation to check.

Without alignas(max_align_t):
/usr/bin/ld: Warning: alignment 8 of symbol `mlx5_glue' in src/dpdk/drivers/net/mlx5/mlx5_glue.c.11.o is smaller than 32 in src/dpdk/drivers/net/mlx5/mlx5_rxq.c.11.o

With alignas(max_align_t):
/usr/bin/ld: Warning: alignment 16 of symbol `mlx5_glue' in src/dpdk/drivers/net/mlx5/mlx5_glue.c.11.o is smaller than 32 in src/dpdk/drivers/net/mlx5/mlx5_rxq.c.11.o

Using alignas(64) does not produce linker warning.

Thanks,
Yaroslav.


-----Original Message-----
From: Adrien Mazarguil [mailto:adrien.mazarguil@6wind.com] 
Sent: Tuesday, July 24, 2018 19:04
To: Yaroslav Brustinov (ybrustin) <ybrustin@cisco.com>
Cc: Shahaf Shuler <shahafs@mellanox.com>; Yongseok Koh <yskoh@mellanox.com>; dev@dpdk.org; stable@dpdk.org; Nélio Laranjeiro <nelio.laranjeiro@6wind.com>
Subject: Re: [PATCH] net/mlx5: fix linkage error for glue lib

On Tue, Jul 24, 2018 at 02:24:12PM +0000, Yaroslav Brustinov (ybrustin) wrote:
> Hi,
> 
> Are you OK with C11?
> I saw in file dpdk/drivers/net/mlx4/mlx4_utils.c:
> 
>      * C11 code would include stdalign.h and use alignof(max_align_t) however
>      * we'll stick with C99 for the time being.
> 
> :)

Hah, this code was originally intended to become a generic EAL function hence the C99 but got rejected. Mellanox PMDs otherwise rely on C11 features already.

> As far as I understand, the issue is alignment of the pointer itself, not the struct.
> I'm not familiar with this command: "alignof(max_align_t)".
> Applying this on the struct:
> 
>     struct mlx5_glue {
>     ...
>     } __attribute__((__aligned__(64)));
> 
>     struct __attribute__((__aligned__(64))) mlx5_glue {
>     ...
>     };
> 
> ...still keeps the linker unhappy.

Right, this was my first (wrong) suggestion that doesn't work on types. How about the second one instead? Here's how the diff on mlx5_glue.h should look
like:

 +#include <stdalign.h>
  #include <stddef.h>
  #include <stdint.h>

 [...]
  
 +alignas(max_align_t)
  const struct mlx5_glue *mlx5_glue;

Another comment regarding this patch, commit log should probably mention it addresses a GCC bug that cannot be reproduced with latter versions. Keep in mind DPDK recommends to use at least GCC version 4.9.

> -----Original Message-----
> From: Shahaf Shuler [mailto:shahafs@mellanox.com]
> Sent: Tuesday, July 24, 2018 16:51
> To: Adrien Mazarguil <adrien.mazarguil@6wind.com>
> Cc: Yongseok Koh <yskoh@mellanox.com>; dev@dpdk.org; Yaroslav 
> Brustinov (ybrustin) <ybrustin@cisco.com>; stable@dpdk.org; Nélio 
> Laranjeiro <nelio.laranjeiro@6wind.com>
> Subject: RE: [PATCH] net/mlx5: fix linkage error for glue lib
> 
> Yaroslav,
> 
> Tuesday, July 24, 2018 3:06 PM, Adrien Mazarguil:
> > Subject: Re: [PATCH] net/mlx5: fix linkage error for glue lib
> > > Odd, how could this happen assuming both files are compiled during 
> > > the same run using identical flags? Looks like a compiler issue.
> > >
> > > Anyway, may I suggest an alignment constraint on the structure 
> > > type in mlx5_glue.h instead, so that all users inherit it. E.g. using C11 syntax:
> > >
> > >  #include <stdalign.h>
> > >  #include <stddef.h>
> > >
> > >  [...]
> > >  alignas(max_align_t)
> > >  struct mlx5_glue {
> > >      [...]
> > >  };
> > 
> > My bad, this is not a correct use for alignas(), it doesn't work on types.
> > How about this instead:
> > 
> >  alignas(max_align_t)
> >  const struct mlx5_glue *mlx5_glue;
> 
> Can you confirm the above suggestion fixes your issue? 
> 
> > 
> > >

--
Adrien Mazarguil
6WIND

Re: [dpdk-dev] [PATCH] net/mlx5: fix linkage error for glue lib

[Adrien Mazarguil]

On Wed, Jul 25, 2018 at 07:38:39AM +0000, Yaroslav Brustinov (ybrustin) wrote:
> +Hanoch
> 
> I've added -std=c11 flag to our compilation to check.
> 
> Without alignas(max_align_t):
> /usr/bin/ld: Warning: alignment 8 of symbol `mlx5_glue' in src/dpdk/drivers/net/mlx5/mlx5_glue.c.11.o is smaller than 32 in src/dpdk/drivers/net/mlx5/mlx5_rxq.c.11.o
> 
> With alignas(max_align_t):
> /usr/bin/ld: Warning: alignment 16 of symbol `mlx5_glue' in src/dpdk/drivers/net/mlx5/mlx5_glue.c.11.o is smaller than 32 in src/dpdk/drivers/net/mlx5/mlx5_rxq.c.11.o
> 
> Using alignas(64) does not produce linker warning.

OK, let's forget max_align_t. Even better, how about
alignas(RTE_CACHE_LINE_SIZE) just in case the same GCC version complains
about the lack of a 128 byte alignment on architectures like IBM POWER8.

(remember to #include <rte_config.h> for RTE_CACHE_LINE_SIZE)

> -----Original Message-----
> From: Adrien Mazarguil [mailto:adrien.mazarguil@6wind.com] 
> Sent: Tuesday, July 24, 2018 19:04
> To: Yaroslav Brustinov (ybrustin) <ybrustin@cisco.com>
> Cc: Shahaf Shuler <shahafs@mellanox.com>; Yongseok Koh <yskoh@mellanox.com>; dev@dpdk.org; stable@dpdk.org; Nélio Laranjeiro <nelio.laranjeiro@6wind.com>
> Subject: Re: [PATCH] net/mlx5: fix linkage error for glue lib
> 
> On Tue, Jul 24, 2018 at 02:24:12PM +0000, Yaroslav Brustinov (ybrustin) wrote:
> > Hi,
> > 
> > Are you OK with C11?
> > I saw in file dpdk/drivers/net/mlx4/mlx4_utils.c:
> > 
> >      * C11 code would include stdalign.h and use alignof(max_align_t) however
> >      * we'll stick with C99 for the time being.
> > 
> > :)
> 
> Hah, this code was originally intended to become a generic EAL function hence the C99 but got rejected. Mellanox PMDs otherwise rely on C11 features already.
> 
> > As far as I understand, the issue is alignment of the pointer itself, not the struct.
> > I'm not familiar with this command: "alignof(max_align_t)".
> > Applying this on the struct:
> > 
> >     struct mlx5_glue {
> >     ...
> >     } __attribute__((__aligned__(64)));
> > 
> >     struct __attribute__((__aligned__(64))) mlx5_glue {
> >     ...
> >     };
> > 
> > ...still keeps the linker unhappy.
> 
> Right, this was my first (wrong) suggestion that doesn't work on types. How about the second one instead? Here's how the diff on mlx5_glue.h should look
> like:
> 
>  +#include <stdalign.h>
>   #include <stddef.h>
>   #include <stdint.h>
> 
>  [...]
>   
>  +alignas(max_align_t)
>   const struct mlx5_glue *mlx5_glue;
> 
> Another comment regarding this patch, commit log should probably mention it addresses a GCC bug that cannot be reproduced with latter versions. Keep in mind DPDK recommends to use at least GCC version 4.9.
> 
> > -----Original Message-----
> > From: Shahaf Shuler [mailto:shahafs@mellanox.com]
> > Sent: Tuesday, July 24, 2018 16:51
> > To: Adrien Mazarguil <adrien.mazarguil@6wind.com>
> > Cc: Yongseok Koh <yskoh@mellanox.com>; dev@dpdk.org; Yaroslav 
> > Brustinov (ybrustin) <ybrustin@cisco.com>; stable@dpdk.org; Nélio 
> > Laranjeiro <nelio.laranjeiro@6wind.com>
> > Subject: RE: [PATCH] net/mlx5: fix linkage error for glue lib
> > 
> > Yaroslav,
> > 
> > Tuesday, July 24, 2018 3:06 PM, Adrien Mazarguil:
> > > Subject: Re: [PATCH] net/mlx5: fix linkage error for glue lib
> > > > Odd, how could this happen assuming both files are compiled during 
> > > > the same run using identical flags? Looks like a compiler issue.
> > > >
> > > > Anyway, may I suggest an alignment constraint on the structure 
> > > > type in mlx5_glue.h instead, so that all users inherit it. E.g. using C11 syntax:
> > > >
> > > >  #include <stdalign.h>
> > > >  #include <stddef.h>
> > > >
> > > >  [...]
> > > >  alignas(max_align_t)
> > > >  struct mlx5_glue {
> > > >      [...]
> > > >  };
> > > 
> > > My bad, this is not a correct use for alignas(), it doesn't work on types.
> > > How about this instead:
> > > 
> > >  alignas(max_align_t)
> > >  const struct mlx5_glue *mlx5_glue;
> > 
> > Can you confirm the above suggestion fixes your issue? 
> > 
> > > 
> > > >
> 
> --
> Adrien Mazarguil
> 6WIND

-- 
Adrien Mazarguil
6WIND

Re: [dpdk-dev] [PATCH] net/mlx5: fix linkage error for glue lib

[Yaroslav Brustinov (ybrustin)]

Hi,

Sounds good to me.

Thanks,
Yaroslav.


-----Original Message-----
From: Adrien Mazarguil [mailto:adrien.mazarguil@6wind.com] 
Sent: Wednesday, July 25, 2018 12:25
To: Yaroslav Brustinov (ybrustin) <ybrustin@cisco.com>
Cc: Shahaf Shuler <shahafs@mellanox.com>; Yongseok Koh <yskoh@mellanox.com>; dev@dpdk.org; stable@dpdk.org; Nélio Laranjeiro <nelio.laranjeiro@6wind.com>; Hanoch Haim (hhaim) <hhaim@cisco.com>
Subject: Re: [PATCH] net/mlx5: fix linkage error for glue lib

On Wed, Jul 25, 2018 at 07:38:39AM +0000, Yaroslav Brustinov (ybrustin) wrote:
> +Hanoch
> 
> I've added -std=c11 flag to our compilation to check.
> 
> Without alignas(max_align_t):
> /usr/bin/ld: Warning: alignment 8 of symbol `mlx5_glue' in 
> src/dpdk/drivers/net/mlx5/mlx5_glue.c.11.o is smaller than 32 in 
> src/dpdk/drivers/net/mlx5/mlx5_rxq.c.11.o
> 
> With alignas(max_align_t):
> /usr/bin/ld: Warning: alignment 16 of symbol `mlx5_glue' in 
> src/dpdk/drivers/net/mlx5/mlx5_glue.c.11.o is smaller than 32 in 
> src/dpdk/drivers/net/mlx5/mlx5_rxq.c.11.o
> 
> Using alignas(64) does not produce linker warning.

OK, let's forget max_align_t. Even better, how about
alignas(RTE_CACHE_LINE_SIZE) just in case the same GCC version complains about the lack of a 128 byte alignment on architectures like IBM POWER8.

(remember to #include <rte_config.h> for RTE_CACHE_LINE_SIZE)

> -----Original Message-----
> From: Adrien Mazarguil [mailto:adrien.mazarguil@6wind.com]
> Sent: Tuesday, July 24, 2018 19:04
> To: Yaroslav Brustinov (ybrustin) <ybrustin@cisco.com>
> Cc: Shahaf Shuler <shahafs@mellanox.com>; Yongseok Koh 
> <yskoh@mellanox.com>; dev@dpdk.org; stable@dpdk.org; Nélio Laranjeiro 
> <nelio.laranjeiro@6wind.com>
> Subject: Re: [PATCH] net/mlx5: fix linkage error for glue lib
> 
> On Tue, Jul 24, 2018 at 02:24:12PM +0000, Yaroslav Brustinov (ybrustin) wrote:
> > Hi,
> > 
> > Are you OK with C11?
> > I saw in file dpdk/drivers/net/mlx4/mlx4_utils.c:
> > 
> >      * C11 code would include stdalign.h and use alignof(max_align_t) however
> >      * we'll stick with C99 for the time being.
> > 
> > :)
> 
> Hah, this code was originally intended to become a generic EAL function hence the C99 but got rejected. Mellanox PMDs otherwise rely on C11 features already.
> 
> > As far as I understand, the issue is alignment of the pointer itself, not the struct.
> > I'm not familiar with this command: "alignof(max_align_t)".
> > Applying this on the struct:
> > 
> >     struct mlx5_glue {
> >     ...
> >     } __attribute__((__aligned__(64)));
> > 
> >     struct __attribute__((__aligned__(64))) mlx5_glue {
> >     ...
> >     };
> > 
> > ...still keeps the linker unhappy.
> 
> Right, this was my first (wrong) suggestion that doesn't work on 
> types. How about the second one instead? Here's how the diff on 
> mlx5_glue.h should look
> like:
> 
>  +#include <stdalign.h>
>   #include <stddef.h>
>   #include <stdint.h>
> 
>  [...]
>   
>  +alignas(max_align_t)
>   const struct mlx5_glue *mlx5_glue;
> 
> Another comment regarding this patch, commit log should probably mention it addresses a GCC bug that cannot be reproduced with latter versions. Keep in mind DPDK recommends to use at least GCC version 4.9.
> 
> > -----Original Message-----
> > From: Shahaf Shuler [mailto:shahafs@mellanox.com]
> > Sent: Tuesday, July 24, 2018 16:51
> > To: Adrien Mazarguil <adrien.mazarguil@6wind.com>
> > Cc: Yongseok Koh <yskoh@mellanox.com>; dev@dpdk.org; Yaroslav 
> > Brustinov (ybrustin) <ybrustin@cisco.com>; stable@dpdk.org; Nélio 
> > Laranjeiro <nelio.laranjeiro@6wind.com>
> > Subject: RE: [PATCH] net/mlx5: fix linkage error for glue lib
> > 
> > Yaroslav,
> > 
> > Tuesday, July 24, 2018 3:06 PM, Adrien Mazarguil:
> > > Subject: Re: [PATCH] net/mlx5: fix linkage error for glue lib
> > > > Odd, how could this happen assuming both files are compiled 
> > > > during the same run using identical flags? Looks like a compiler issue.
> > > >
> > > > Anyway, may I suggest an alignment constraint on the structure 
> > > > type in mlx5_glue.h instead, so that all users inherit it. E.g. using C11 syntax:
> > > >
> > > >  #include <stdalign.h>
> > > >  #include <stddef.h>
> > > >
> > > >  [...]
> > > >  alignas(max_align_t)
> > > >  struct mlx5_glue {
> > > >      [...]
> > > >  };
> > > 
> > > My bad, this is not a correct use for alignas(), it doesn't work on types.
> > > How about this instead:
> > > 
> > >  alignas(max_align_t)
> > >  const struct mlx5_glue *mlx5_glue;
> > 
> > Can you confirm the above suggestion fixes your issue? 
> > 
> > > 
> > > >
> 
> --
> Adrien Mazarguil
> 6WIND

--
Adrien Mazarguil
6WIND

[dpdk-dev] [PATCH v2] net/mlx5: fix linkage error for glue lib

[Shahaf Shuler]

From: Yaroslav Brustinov <ybrustin@cisco.com>

addressing a gcc 4.7.2 bug that cannot be reproduced with latter
versions:

"bin/ld: Warning: alignment 8 of symbol `mlx5_glue' in
src/dpdk/drivers/net/mlx5/mlx5_glue.c.21.o is smaller than 16 in
src/dpdk/drivers/net/mlx5/mlx5_rxq.c.21.o"

Fix it be forcing the alignment of the glue lib.

Fixes: 0e83b8e536c1 ("net/mlx5: move rdma-core calls to separate file")
Cc: stable@dpdk.org
Cc: nelio.laranjeiro@6wind.com
Cc: adrien.mazarguil@6wind.com

Signed-off-by: Yaroslav Brustinov <ybrustin@cisco.com>
Signed-off-by: Shahaf Shuler <shahafs@mellanox.com>
---
On v2:
 - Forced alignment using alignas to the size of the cacheline.

---
 drivers/net/mlx5/mlx5_glue.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/drivers/net/mlx5/mlx5_glue.c b/drivers/net/mlx5/mlx5_glue.c
index c7965e51fe..705232f614 100644
--- a/drivers/net/mlx5/mlx5_glue.c
+++ b/drivers/net/mlx5/mlx5_glue.c
@@ -6,6 +6,7 @@
 #include <errno.h>
 #include <stddef.h>
 #include <stdint.h>
+#include <stdalign.h>
 
 /*
  * Not needed by this file; included to work around the lack of off_t
@@ -23,6 +24,8 @@
 #pragma GCC diagnostic error "-Wpedantic"
 #endif
 
+#include <rte_config.h>
+
 #include "mlx5_autoconf.h"
 #include "mlx5_glue.h"
 
@@ -343,7 +346,8 @@ mlx5_glue_dv_create_qp(struct ibv_context *context,
 #endif
 }
 
-const struct mlx5_glue *mlx5_glue = &(const struct mlx5_glue){
+alignas(RTE_CACHE_LINE_SIZE)
+const struct mlx5_glue *mlx5_glue = &(const struct mlx5_glue) {
 	.version = MLX5_GLUE_VERSION,
 	.fork_init = mlx5_glue_fork_init,
 	.alloc_pd = mlx5_glue_alloc_pd,
-- 
2.12.0

Re: [dpdk-dev] [PATCH v2] net/mlx5: fix linkage error for glue lib

[Adrien Mazarguil]

On Wed, Jul 25, 2018 at 04:18:54PM +0300, Shahaf Shuler wrote:
> From: Yaroslav Brustinov <ybrustin@cisco.com>
> 
> addressing a gcc 4.7.2 bug that cannot be reproduced with latter
> versions:
> 
> "bin/ld: Warning: alignment 8 of symbol `mlx5_glue' in
> src/dpdk/drivers/net/mlx5/mlx5_glue.c.21.o is smaller than 16 in
> src/dpdk/drivers/net/mlx5/mlx5_rxq.c.21.o"
> 
> Fix it be forcing the alignment of the glue lib.
> 
> Fixes: 0e83b8e536c1 ("net/mlx5: move rdma-core calls to separate file")
> Cc: stable@dpdk.org
> Cc: nelio.laranjeiro@6wind.com
> Cc: adrien.mazarguil@6wind.com
> 
> Signed-off-by: Yaroslav Brustinov <ybrustin@cisco.com>
> Signed-off-by: Shahaf Shuler <shahafs@mellanox.com>

Except for a couple of minor nits below,

Acked-by: Adrien Mazarguil <adrien.mazarguil@6wind.com>

> ---
> On v2:
>  - Forced alignment using alignas to the size of the cacheline.
> 
> ---
>  drivers/net/mlx5/mlx5_glue.c | 6 +++++-
>  1 file changed, 5 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/net/mlx5/mlx5_glue.c b/drivers/net/mlx5/mlx5_glue.c
> index c7965e51fe..705232f614 100644
> --- a/drivers/net/mlx5/mlx5_glue.c
> +++ b/drivers/net/mlx5/mlx5_glue.c
> @@ -6,6 +6,7 @@
>  #include <errno.h>
>  #include <stddef.h>
>  #include <stdint.h>
> +#include <stdalign.h>

You should keep alphabetical order.

>  
>  /*
>   * Not needed by this file; included to work around the lack of off_t
> @@ -23,6 +24,8 @@
>  #pragma GCC diagnostic error "-Wpedantic"
>  #endif
>  
> +#include <rte_config.h>
> +
>  #include "mlx5_autoconf.h"
>  #include "mlx5_glue.h"
>  
> @@ -343,7 +346,8 @@ mlx5_glue_dv_create_qp(struct ibv_context *context,
>  #endif
>  }
>  
> -const struct mlx5_glue *mlx5_glue = &(const struct mlx5_glue){
> +alignas(RTE_CACHE_LINE_SIZE)
> +const struct mlx5_glue *mlx5_glue = &(const struct mlx5_glue) {

Extra space added before opening brace :)

>  	.version = MLX5_GLUE_VERSION,
>  	.fork_init = mlx5_glue_fork_init,
>  	.alloc_pd = mlx5_glue_alloc_pd,
> -- 
> 2.12.0
> 

-- 
Adrien Mazarguil
6WIND

Re: [dpdk-dev] [PATCH v2] net/mlx5: fix linkage error for glue lib

[Shahaf Shuler]

Wednesday, July 25, 2018 4:23 PM, Adrien Mazarguil:
> Subject: Re: [PATCH v2] net/mlx5: fix linkage error for glue lib
> 
> On Wed, Jul 25, 2018 at 04:18:54PM +0300, Shahaf Shuler wrote:
> > From: Yaroslav Brustinov <ybrustin@cisco.com>
> >
> > addressing a gcc 4.7.2 bug that cannot be reproduced with latter
> > versions:
> >
> > "bin/ld: Warning: alignment 8 of symbol `mlx5_glue' in
> > src/dpdk/drivers/net/mlx5/mlx5_glue.c.21.o is smaller than 16 in
> > src/dpdk/drivers/net/mlx5/mlx5_rxq.c.21.o"
> >
> > Fix it be forcing the alignment of the glue lib.
> >
> > Fixes: 0e83b8e536c1 ("net/mlx5: move rdma-core calls to separate
> > file")
> > Cc: stable@dpdk.org
> > Cc: nelio.laranjeiro@6wind.com
> > Cc: adrien.mazarguil@6wind.com
> >
> > Signed-off-by: Yaroslav Brustinov <ybrustin@cisco.com>
> > Signed-off-by: Shahaf Shuler <shahafs@mellanox.com>
> 
> Except for a couple of minor nits below,
> 
> Acked-by: Adrien Mazarguil <adrien.mazarguil@6wind.com>

Applied to next-net-mlx with the needed fixes. Thanks.