[dpdk-dev] [PATCH] ethdev: fix dpdk gcc build on Arm

[dpdk-dev] [PATCH] ethdev: fix dpdk gcc build on Arm

[Juraj Linkeš]

Directive #include <file> in gcc implementation searches for files
in a standard list of system directories, which leads to a sporadici
build error on Taishan arm machines:
/tmp/openvpp-testing/dpdk/lib/librte_ethdev/rte_ethdev.h:4287:10:
fatal error: rte_ethdev_core.h:
No such file or directory #include <rte_ethdev_core.h>

Fix this by replacing the directive with #include "file" which searches
for files first in the directory containing the current file and other
directories only after that.

Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
 lib/librte_ethdev/rte_ethdev.h        | 2 +-
 lib/librte_ethdev/rte_ethdev_driver.h | 2 +-
 lib/librte_ethdev/rte_ethdev_pci.h    | 2 +-
 lib/librte_ethdev/rte_ethdev_vdev.h   | 2 +-
 4 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h
index a49242bcd..f04d42368 100644
--- a/lib/librte_ethdev/rte_ethdev.h
+++ b/lib/librte_ethdev/rte_ethdev.h
@@ -4302,7 +4302,7 @@ __rte_experimental
 int rte_eth_dev_hairpin_capability_get(uint16_t port_id,
 				       struct rte_eth_hairpin_cap *cap);
 
-#include <rte_ethdev_core.h>
+#include "rte_ethdev_core.h"
 
 /**
  *
diff --git a/lib/librte_ethdev/rte_ethdev_driver.h b/lib/librte_ethdev/rte_ethdev_driver.h
index 99d4cd6cd..496c77fb5 100644
--- a/lib/librte_ethdev/rte_ethdev_driver.h
+++ b/lib/librte_ethdev/rte_ethdev_driver.h
@@ -15,7 +15,7 @@
  *
  */
 
-#include <rte_ethdev.h>
+#include "rte_ethdev.h"
 
 #ifdef __cplusplus
 extern "C" {
diff --git a/lib/librte_ethdev/rte_ethdev_pci.h b/lib/librte_ethdev/rte_ethdev_pci.h
index a999602fd..72cdb8e5a 100644
--- a/lib/librte_ethdev/rte_ethdev_pci.h
+++ b/lib/librte_ethdev/rte_ethdev_pci.h
@@ -36,7 +36,7 @@
 #include <rte_pci.h>
 #include <rte_bus_pci.h>
 #include <rte_config.h>
-#include <rte_ethdev_driver.h>
+#include "rte_ethdev_driver.h"
 
 /**
  * Copy pci device info to the Ethernet device data.
diff --git a/lib/librte_ethdev/rte_ethdev_vdev.h b/lib/librte_ethdev/rte_ethdev_vdev.h
index b3ef12398..05be36362 100644
--- a/lib/librte_ethdev/rte_ethdev_vdev.h
+++ b/lib/librte_ethdev/rte_ethdev_vdev.h
@@ -35,7 +35,7 @@
 #include <rte_config.h>
 #include <rte_malloc.h>
 #include <rte_bus_vdev.h>
-#include <rte_ethdev_driver.h>
+#include "rte_ethdev_driver.h"
 
 /**
  * @internal
-- 
2.20.1

Re: [dpdk-dev] [PATCH] ethdev: fix dpdk gcc build on Arm

[Thomas Monjalon]

03/06/2020 11:48, Juraj Linkeš:
> Directive #include <file> in gcc implementation searches for files
> in a standard list of system directories, which leads to a sporadici
> build error on Taishan arm machines:
> /tmp/openvpp-testing/dpdk/lib/librte_ethdev/rte_ethdev.h:4287:10:
> fatal error: rte_ethdev_core.h:
> No such file or directory #include <rte_ethdev_core.h>

Would be interesting to know why nobody else hit such error?


> Fix this by replacing the directive with #include "file" which searches
> for files first in the directory containing the current file and other
> directories only after that.
> 
> Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>

What about #include <rte_ethdev_trace.h> ?

Except this,
Acked-by: Thomas Monjalon <thomas@monjalon.net>


I would change the title to be more generic (nothing specific to gcc or Arm here):
	ethdev: fix local include

It needs to be backported, so these lines are required:
	Fixes: 7f0a669e7b04 ("ethdev: add allocation helper for virtual drivers")
	Fixes: dcd5c8112bc3 ("ethdev: add PCI driver helpers")
	Fixes: ffc905f3b856 ("ethdev: separate driver APIs")
	Fixes: 331c447ad913 ("ethdev: separate internal structures into own header")
	Cc: stable@dpdk.org

Re: [dpdk-dev] [PATCH] ethdev: fix dpdk gcc build on Arm

[Ananyev, Konstantin]

> > Directive #include <file> in gcc implementation searches for files
> > in a standard list of system directories, which leads to a sporadici
> > build error on Taishan arm machines:
> > /tmp/openvpp-testing/dpdk/lib/librte_ethdev/rte_ethdev.h:4287:10:
> > fatal error: rte_ethdev_core.h:
> > No such file or directory #include <rte_ethdev_core.h>
> 
> Would be interesting to know why nobody else hit such error?

Same question.
Again why it happens for rte_ethdev library only?
We do have multiple headers in other libs too.

> 
> 
> > Fix this by replacing the directive with #include "file" which searches
> > for files first in the directory containing the current file and other
> > directories only after that.
> >
> > Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
> 
> What about #include <rte_ethdev_trace.h> ?
> 
> Except this,
> Acked-by: Thomas Monjalon <thomas@monjalon.net>
> 
> 
> I would change the title to be more generic (nothing specific to gcc or Arm here):
> 	ethdev: fix local include
> 
> It needs to be backported, so these lines are required:
> 	Fixes: 7f0a669e7b04 ("ethdev: add allocation helper for virtual drivers")
> 	Fixes: dcd5c8112bc3 ("ethdev: add PCI driver helpers")
> 	Fixes: ffc905f3b856 ("ethdev: separate driver APIs")
> 	Fixes: 331c447ad913 ("ethdev: separate internal structures into own header")
> 	Cc: stable@dpdk.org
> 
> 

Re: [dpdk-dev] [PATCH] ethdev: fix dpdk gcc build on Arm

[Juraj Linkeš]

-----Original Message-----
From: Thomas Monjalon <thomas@monjalon.net> 
Sent: Wednesday, June 3, 2020 12:17 PM
To: Juraj Linkeš <juraj.linkes@pantheon.tech>
Cc: ferruh.yigit@intel.com; arybchenko@solarflare.com; dev@dpdk.org
Subject: Re: [PATCH] ethdev: fix dpdk gcc build on Arm

03/06/2020 11:48, Juraj Linkeš:
> Directive #include <file> in gcc implementation searches for files in 
> a standard list of system directories, which leads to a sporadici 
> build error on Taishan arm machines:
> /tmp/openvpp-testing/dpdk/lib/librte_ethdev/rte_ethdev.h:4287:10:
> fatal error: rte_ethdev_core.h:
> No such file or directory #include <rte_ethdev_core.h>

Would be interesting to know why nobody else hit such error?


> Fix this by replacing the directive with #include "file" which 
> searches for files first in the directory containing the current file 
> and other directories only after that.
> 
> Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>

What about #include <rte_ethdev_trace.h> ?

I only changed the .h files since that was enough to fix the issue, but I'll add that include as well.

Except this,
Acked-by: Thomas Monjalon <thomas@monjalon.net>


I would change the title to be more generic (nothing specific to gcc or Arm here):
	ethdev: fix local include

According to my research, the handling of include directives could be differenct across compilers so I made it specific, as I only looked at how gcc does it. The implementations are probably basically the same, so I'll change to wording so it's generic.

It needs to be backported, so these lines are required:
	Fixes: 7f0a669e7b04 ("ethdev: add allocation helper for virtual drivers")
	Fixes: dcd5c8112bc3 ("ethdev: add PCI driver helpers")
	Fixes: ffc905f3b856 ("ethdev: separate driver APIs")
	Fixes: 331c447ad913 ("ethdev: separate internal structures into own header")
	Cc: stable@dpdk.org

I didn't know which patches this fixes, so I'll add this. How should I figure this out? Just look at which patch(es) changed the lines I'm modifying?

Re: [dpdk-dev] [PATCH] ethdev: fix dpdk gcc build on Arm

[Thomas Monjalon]

03/06/2020 12:47, Juraj Linkeš:
> From: Thomas Monjalon <thomas@monjalon.net> 
> What about #include <rte_ethdev_trace.h> ?
> 
> I only changed the .h files since that was enough to fix the issue, but I'll add that include as well.
[...]
> It needs to be backported, so these lines are required:
> 	Fixes: 7f0a669e7b04 ("ethdev: add allocation helper for virtual drivers")
> 	Fixes: dcd5c8112bc3 ("ethdev: add PCI driver helpers")
> 	Fixes: ffc905f3b856 ("ethdev: separate driver APIs")
> 	Fixes: 331c447ad913 ("ethdev: separate internal structures into own header")
> 	Cc: stable@dpdk.org
> 
> I didn't know which patches this fixes, so I'll add this. How should I figure this out? Just look at which patch(es) changed the lines I'm modifying?

You can use any git tool to search in history.
I like using gitk.
I searched when the lines you changed were introduced.
This kind of search is called pickaxe in git docs.

When you found the SHA1, you can use this git alias to generate the Fixes line:
	fixline = log -1 --abbrev=12 --format='Fixes: %h (\"%s\")%nCc: %ae'

Or even more complex to add the Cc: stable@dpdk.org line automatically:
	fixline = "!f () { git log -1 --abbrev=12 --format='Fixes: %h (\"%s\")%nCc: %ae' $1 ; ( git tag -l --contains $1 | head -n1 | sed 's,^v,,' | sed 's,-rc.*,,' ; for i in 1 2 ; do make showversion | cut -d'.' -f-2 ; done ) | uniq -u | head -n1 | sed 's,.*,Cc: stable@dpdk.org,' ;}; f"


PS: your email was not prefixing my words with "> ".
Please check your mail config to make sure we clearly distinguish
questions and answers. Thanks

Re: [dpdk-dev] [PATCH] ethdev: fix dpdk gcc build on Arm

[Ferruh Yigit]

On 6/3/2020 11:16 AM, Thomas Monjalon wrote:
> 03/06/2020 11:48, Juraj Linkeš:
>> Directive #include <file> in gcc implementation searches for files
>> in a standard list of system directories, which leads to a sporadici
>> build error on Taishan arm machines:
>> /tmp/openvpp-testing/dpdk/lib/librte_ethdev/rte_ethdev.h:4287:10:
>> fatal error: rte_ethdev_core.h:
>> No such file or directory #include <rte_ethdev_core.h>
> 
> Would be interesting to know why nobody else hit such error?


I can't see why this is happening, in the 'mk/rte.lib.mk' we have following:

"
install: _preinstall build _postinstall
build: _preinstall
"

Which should cause the library header files installed before building .c files
in that library.
So when compiling 'rte_class_eth.c', the header files should be already in
install folder.


I can see how/why changing to "" fixes the issue but I am not sure about this fix.
"rte_ethdev.h" is a public header file, that applications will include it in
their applications. In the public library it is more proper to have other
includes from system folder, using format <>.
Again, I can't see why it is failing but I believe we should find another
solution for _internal_ build error.


A very simple solution can be following, but that is also not good, since it
solves the issue by creating a dependency to the order of the header includes:
 diff --git a/lib/librte_ethdev/rte_class_eth.c b/lib/librte_ethdev/rte_class_eth.c
 index 6338355e25..3030c49020 100644
 --- a/lib/librte_ethdev/rte_class_eth.c
 +++ b/lib/librte_ethdev/rte_class_eth.c
 @@ -10,8 +10,8 @@
  #include <rte_kvargs.h>
  #include <rte_log.h>

 -#include "rte_ethdev.h"
  #include "rte_ethdev_core.h"
 +#include "rte_ethdev.h"
  #include "rte_ethdev_driver.h"
  #include "ethdev_private.h"



> 
> 
>> Fix this by replacing the directive with #include "file" which searches
>> for files first in the directory containing the current file and other
>> directories only after that.
>>
>> Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
> 
> What about #include <rte_ethdev_trace.h> ?
> 
> Except this,
> Acked-by: Thomas Monjalon <thomas@monjalon.net>
> 
> 
> I would change the title to be more generic (nothing specific to gcc or Arm here):
> 	ethdev: fix local include
> 
> It needs to be backported, so these lines are required:
> 	Fixes: 7f0a669e7b04 ("ethdev: add allocation helper for virtual drivers")
> 	Fixes: dcd5c8112bc3 ("ethdev: add PCI driver helpers")
> 	Fixes: ffc905f3b856 ("ethdev: separate driver APIs")
> 	Fixes: 331c447ad913 ("ethdev: separate internal structures into own header")
> 	Cc: stable@dpdk.org
> 
> 
> 

Re: [dpdk-dev] [PATCH] ethdev: fix dpdk gcc build on Arm

[Juraj Linkeš]

> -----Original Message-----
> From: Ferruh Yigit <ferruh.yigit@intel.com>
> Sent: Wednesday, June 3, 2020 1:41 PM
> To: Thomas Monjalon <thomas@monjalon.net>; Juraj Linkeš
> <juraj.linkes@pantheon.tech>
> Cc: arybchenko@solarflare.com; dev@dpdk.org
> Subject: Re: [PATCH] ethdev: fix dpdk gcc build on Arm
> 
> On 6/3/2020 11:16 AM, Thomas Monjalon wrote:
> > 03/06/2020 11:48, Juraj Linkeš:
> >> Directive #include <file> in gcc implementation searches for files in
> >> a standard list of system directories, which leads to a sporadici
> >> build error on Taishan arm machines:
> >> /tmp/openvpp-testing/dpdk/lib/librte_ethdev/rte_ethdev.h:4287:10:
> >> fatal error: rte_ethdev_core.h:
> >> No such file or directory #include <rte_ethdev_core.h>
> >
> > Would be interesting to know why nobody else hit such error?
> 
> 
> I can't see why this is happening, in the 'mk/rte.lib.mk' we have following:
> 
> "
> install: _preinstall build _postinstall
> build: _preinstall
> "
> 
> Which should cause the library header files installed before building .c files in
> that library.
> So when compiling 'rte_class_eth.c', the header files should be already in install
> folder.
> 
> 
> I can see how/why changing to "" fixes the issue but I am not sure about this fix.
> "rte_ethdev.h" is a public header file, that applications will include it in their
> applications. In the public library it is more proper to have other includes from
> system folder, using format <>.
> Again, I can't see why it is failing but I believe we should find another solution
> for _internal_ build error.
> 
> 
> A very simple solution can be following, but that is also not good, since it solves
> the issue by creating a dependency to the order of the header includes:
>  diff --git a/lib/librte_ethdev/rte_class_eth.c b/lib/librte_ethdev/rte_class_eth.c
>  index 6338355e25..3030c49020 100644
>  --- a/lib/librte_ethdev/rte_class_eth.c
>  +++ b/lib/librte_ethdev/rte_class_eth.c
>  @@ -10,8 +10,8 @@
>   #include <rte_kvargs.h>
>   #include <rte_log.h>
> 
>  -#include "rte_ethdev.h"
>   #include "rte_ethdev_core.h"
>  +#include "rte_ethdev.h"
>   #include "rte_ethdev_driver.h"
>   #include "ethdev_private.h"
> 

Thomas, Ferruh, what should be the solution? I'm not an expert on this and I can't really offer anything better, but I'd like that this gets fixed.

> >
> >
> >> Fix this by replacing the directive with #include "file" which
> >> searches for files first in the directory containing the current file
> >> and other directories only after that.
> >>
> >> Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
> >
> > What about #include <rte_ethdev_trace.h> ?
> >
> > Except this,
> > Acked-by: Thomas Monjalon <thomas@monjalon.net>
> >
> >
> > I would change the title to be more generic (nothing specific to gcc or Arm
> here):
> > 	ethdev: fix local include
> >
> > It needs to be backported, so these lines are required:
> > 	Fixes: 7f0a669e7b04 ("ethdev: add allocation helper for virtual drivers")
> > 	Fixes: dcd5c8112bc3 ("ethdev: add PCI driver helpers")
> > 	Fixes: ffc905f3b856 ("ethdev: separate driver APIs")
> > 	Fixes: 331c447ad913 ("ethdev: separate internal structures into own
> header")
> > 	Cc: stable@dpdk.org
> >
> >
> >

Re: [dpdk-dev] [PATCH] ethdev: fix dpdk gcc build on Arm

[Ferruh Yigit]

On 6/4/2020 11:36 AM, Juraj Linkeš wrote:
> 
> 
>> -----Original Message-----
>> From: Ferruh Yigit <ferruh.yigit@intel.com>
>> Sent: Wednesday, June 3, 2020 1:41 PM
>> To: Thomas Monjalon <thomas@monjalon.net>; Juraj Linkeš
>> <juraj.linkes@pantheon.tech>
>> Cc: arybchenko@solarflare.com; dev@dpdk.org
>> Subject: Re: [PATCH] ethdev: fix dpdk gcc build on Arm
>>
>> On 6/3/2020 11:16 AM, Thomas Monjalon wrote:
>>> 03/06/2020 11:48, Juraj Linkeš:
>>>> Directive #include <file> in gcc implementation searches for files in
>>>> a standard list of system directories, which leads to a sporadici
>>>> build error on Taishan arm machines:
>>>> /tmp/openvpp-testing/dpdk/lib/librte_ethdev/rte_ethdev.h:4287:10:
>>>> fatal error: rte_ethdev_core.h:
>>>> No such file or directory #include <rte_ethdev_core.h>
>>>
>>> Would be interesting to know why nobody else hit such error?
>>
>>
>> I can't see why this is happening, in the 'mk/rte.lib.mk' we have following:
>>
>> "
>> install: _preinstall build _postinstall
>> build: _preinstall
>> "
>>
>> Which should cause the library header files installed before building .c files in
>> that library.
>> So when compiling 'rte_class_eth.c', the header files should be already in install
>> folder.
>>
>>
>> I can see how/why changing to "" fixes the issue but I am not sure about this fix.
>> "rte_ethdev.h" is a public header file, that applications will include it in their
>> applications. In the public library it is more proper to have other includes from
>> system folder, using format <>.
>> Again, I can't see why it is failing but I believe we should find another solution
>> for _internal_ build error.
>>
>>
>> A very simple solution can be following, but that is also not good, since it solves
>> the issue by creating a dependency to the order of the header includes:
>>  diff --git a/lib/librte_ethdev/rte_class_eth.c b/lib/librte_ethdev/rte_class_eth.c
>>  index 6338355e25..3030c49020 100644
>>  --- a/lib/librte_ethdev/rte_class_eth.c
>>  +++ b/lib/librte_ethdev/rte_class_eth.c
>>  @@ -10,8 +10,8 @@
>>   #include <rte_kvargs.h>
>>   #include <rte_log.h>
>>
>>  -#include "rte_ethdev.h"
>>   #include "rte_ethdev_core.h"
>>  +#include "rte_ethdev.h"
>>   #include "rte_ethdev_driver.h"
>>   #include "ethdev_private.h"
>>
> 
> Thomas, Ferruh, what should be the solution? I'm not an expert on this and I can't really offer anything better, but I'd like that this gets fixed.

First we need to root cause this before trying to solve it. Honnappa also said
he can reproduce this but our CI builds can't (we are talking about tens of
builds daily on various platforms), need to understand why.
Also from Makefile I can't see how this is happening, I am feeling uneasy to fix
something before figuring out how/why it is failing.

Can you please try to collect more data on when/how this happens, initial
questions I can think of:
- Can you reproduce this with meson build?
- Is it bare DPDK build, or build part of other project (I guess I saw fd.io on
the link)
- - If this is not bare DPDK build what changes has been done to build system?
- Do you see this with fresh build (new clone) or rebuild of existing clone?
- Can you confirm you have correct RTE_SDK and RTE_TARGET environment variables?
- Can you please share your build command?

Thanks,
ferruh

> 
>>>
>>>
>>>> Fix this by replacing the directive with #include "file" which
>>>> searches for files first in the directory containing the current file
>>>> and other directories only after that.
>>>>
>>>> Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
>>>
>>> What about #include <rte_ethdev_trace.h> ?
>>>
>>> Except this,
>>> Acked-by: Thomas Monjalon <thomas@monjalon.net>
>>>
>>>
>>> I would change the title to be more generic (nothing specific to gcc or Arm
>> here):
>>> 	ethdev: fix local include
>>>
>>> It needs to be backported, so these lines are required:
>>> 	Fixes: 7f0a669e7b04 ("ethdev: add allocation helper for virtual drivers")
>>> 	Fixes: dcd5c8112bc3 ("ethdev: add PCI driver helpers")
>>> 	Fixes: ffc905f3b856 ("ethdev: separate driver APIs")
>>> 	Fixes: 331c447ad913 ("ethdev: separate internal structures into own
>> header")
>>> 	Cc: stable@dpdk.org
>>>
>>>
>>>
> 

Re: [dpdk-dev] [PATCH] ethdev: fix dpdk gcc build on Arm

[Ferruh Yigit]

On 6/4/2020 1:22 PM, Ferruh Yigit wrote:
> On 6/4/2020 11:36 AM, Juraj Linkeš wrote:
>>
>>
>>> -----Original Message-----
>>> From: Ferruh Yigit <ferruh.yigit@intel.com>
>>> Sent: Wednesday, June 3, 2020 1:41 PM
>>> To: Thomas Monjalon <thomas@monjalon.net>; Juraj Linkeš
>>> <juraj.linkes@pantheon.tech>
>>> Cc: arybchenko@solarflare.com; dev@dpdk.org
>>> Subject: Re: [PATCH] ethdev: fix dpdk gcc build on Arm
>>>
>>> On 6/3/2020 11:16 AM, Thomas Monjalon wrote:
>>>> 03/06/2020 11:48, Juraj Linkeš:
>>>>> Directive #include <file> in gcc implementation searches for files in
>>>>> a standard list of system directories, which leads to a sporadici
>>>>> build error on Taishan arm machines:
>>>>> /tmp/openvpp-testing/dpdk/lib/librte_ethdev/rte_ethdev.h:4287:10:
>>>>> fatal error: rte_ethdev_core.h:
>>>>> No such file or directory #include <rte_ethdev_core.h>
>>>>
>>>> Would be interesting to know why nobody else hit such error?
>>>
>>>
>>> I can't see why this is happening, in the 'mk/rte.lib.mk' we have following:
>>>
>>> "
>>> install: _preinstall build _postinstall
>>> build: _preinstall
>>> "
>>>
>>> Which should cause the library header files installed before building .c files in
>>> that library.
>>> So when compiling 'rte_class_eth.c', the header files should be already in install
>>> folder.
>>>
>>>
>>> I can see how/why changing to "" fixes the issue but I am not sure about this fix.
>>> "rte_ethdev.h" is a public header file, that applications will include it in their
>>> applications. In the public library it is more proper to have other includes from
>>> system folder, using format <>.
>>> Again, I can't see why it is failing but I believe we should find another solution
>>> for _internal_ build error.
>>>
>>>
>>> A very simple solution can be following, but that is also not good, since it solves
>>> the issue by creating a dependency to the order of the header includes:
>>>  diff --git a/lib/librte_ethdev/rte_class_eth.c b/lib/librte_ethdev/rte_class_eth.c
>>>  index 6338355e25..3030c49020 100644
>>>  --- a/lib/librte_ethdev/rte_class_eth.c
>>>  +++ b/lib/librte_ethdev/rte_class_eth.c
>>>  @@ -10,8 +10,8 @@
>>>   #include <rte_kvargs.h>
>>>   #include <rte_log.h>
>>>
>>>  -#include "rte_ethdev.h"
>>>   #include "rte_ethdev_core.h"
>>>  +#include "rte_ethdev.h"
>>>   #include "rte_ethdev_driver.h"
>>>   #include "ethdev_private.h"
>>>
>>
>> Thomas, Ferruh, what should be the solution? I'm not an expert on this and I can't really offer anything better, but I'd like that this gets fixed.
> 
> First we need to root cause this before trying to solve it. Honnappa also said
> he can reproduce this but our CI builds can't (we are talking about tens of
> builds daily on various platforms), need to understand why.
> Also from Makefile I can't see how this is happening, I am feeling uneasy to fix
> something before figuring out how/why it is failing.
> 
> Can you please try to collect more data on when/how this happens, initial
> questions I can think of:
> - Can you reproduce this with meson build?
> - Is it bare DPDK build, or build part of other project (I guess I saw fd.io on
> the link)
> - - If this is not bare DPDK build what changes has been done to build system?
> - Do you see this with fresh build (new clone) or rebuild of existing clone?
> - Can you confirm you have correct RTE_SDK and RTE_TARGET environment variables?
> - Can you please share your build command?

+ Can you please build with "V=1" argument, this should give more detail on
where header files are installed, this will let you know if the header files are
installed to different path than you expected which may lead header file not to
be found.

> 
> Thanks,
> ferruh
> 
>>
>>>>
>>>>
>>>>> Fix this by replacing the directive with #include "file" which
>>>>> searches for files first in the directory containing the current file
>>>>> and other directories only after that.
>>>>>
>>>>> Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
>>>>
>>>> What about #include <rte_ethdev_trace.h> ?
>>>>
>>>> Except this,
>>>> Acked-by: Thomas Monjalon <thomas@monjalon.net>
>>>>
>>>>
>>>> I would change the title to be more generic (nothing specific to gcc or Arm
>>> here):
>>>> 	ethdev: fix local include
>>>>
>>>> It needs to be backported, so these lines are required:
>>>> 	Fixes: 7f0a669e7b04 ("ethdev: add allocation helper for virtual drivers")
>>>> 	Fixes: dcd5c8112bc3 ("ethdev: add PCI driver helpers")
>>>> 	Fixes: ffc905f3b856 ("ethdev: separate driver APIs")
>>>> 	Fixes: 331c447ad913 ("ethdev: separate internal structures into own
>>> header")
>>>> 	Cc: stable@dpdk.org
>>>>
>>>>
>>>>
>>
> 

Re: [dpdk-dev] [PATCH] ethdev: fix dpdk gcc build on Arm

[Juraj Linkeš]

> -----Original Message-----
> From: Ferruh Yigit <ferruh.yigit@intel.com>
> Sent: Thursday, June 4, 2020 2:23 PM
> To: Juraj Linkeš <juraj.linkes@pantheon.tech>; Thomas Monjalon
> <thomas@monjalon.net>
> Cc: arybchenko@solarflare.com; dev@dpdk.org; Honnappa Nagarahalli
> <Honnappa.Nagarahalli@arm.com>
> Subject: Re: [PATCH] ethdev: fix dpdk gcc build on Arm
> 
> On 6/4/2020 11:36 AM, Juraj Linkeš wrote:
> >
> >
> >> -----Original Message-----
> >> From: Ferruh Yigit <ferruh.yigit@intel.com>
> >> Sent: Wednesday, June 3, 2020 1:41 PM
> >> To: Thomas Monjalon <thomas@monjalon.net>; Juraj Linkeš
> >> <juraj.linkes@pantheon.tech>
> >> Cc: arybchenko@solarflare.com; dev@dpdk.org
> >> Subject: Re: [PATCH] ethdev: fix dpdk gcc build on Arm
> >>
> >> On 6/3/2020 11:16 AM, Thomas Monjalon wrote:
> >>> 03/06/2020 11:48, Juraj Linkeš:
> >>>> Directive #include <file> in gcc implementation searches for files
> >>>> in a standard list of system directories, which leads to a
> >>>> sporadici build error on Taishan arm machines:
> >>>> /tmp/openvpp-testing/dpdk/lib/librte_ethdev/rte_ethdev.h:4287:10:
> >>>> fatal error: rte_ethdev_core.h:
> >>>> No such file or directory #include <rte_ethdev_core.h>
> >>>
> >>> Would be interesting to know why nobody else hit such error?
> >>
> >>
> >> I can't see why this is happening, in the 'mk/rte.lib.mk' we have following:
> >>
> >> "
> >> install: _preinstall build _postinstall
> >> build: _preinstall
> >> "
> >>
> >> Which should cause the library header files installed before building
> >> .c files in that library.
> >> So when compiling 'rte_class_eth.c', the header files should be
> >> already in install folder.
> >>
> >>
> >> I can see how/why changing to "" fixes the issue but I am not sure about this
> fix.
> >> "rte_ethdev.h" is a public header file, that applications will
> >> include it in their applications. In the public library it is more
> >> proper to have other includes from system folder, using format <>.
> >> Again, I can't see why it is failing but I believe we should find
> >> another solution for _internal_ build error.
> >>
> >>
> >> A very simple solution can be following, but that is also not good,
> >> since it solves the issue by creating a dependency to the order of the header
> includes:
> >>  diff --git a/lib/librte_ethdev/rte_class_eth.c
> >> b/lib/librte_ethdev/rte_class_eth.c
> >>  index 6338355e25..3030c49020 100644
> >>  --- a/lib/librte_ethdev/rte_class_eth.c
> >>  +++ b/lib/librte_ethdev/rte_class_eth.c
> >>  @@ -10,8 +10,8 @@
> >>   #include <rte_kvargs.h>
> >>   #include <rte_log.h>
> >>
> >>  -#include "rte_ethdev.h"
> >>   #include "rte_ethdev_core.h"
> >>  +#include "rte_ethdev.h"
> >>   #include "rte_ethdev_driver.h"
> >>   #include "ethdev_private.h"
> >>
> >
> > Thomas, Ferruh, what should be the solution? I'm not an expert on this and I
> can't really offer anything better, but I'd like that this gets fixed.
> 
> First we need to root cause this before trying to solve it. Honnappa also said he
> can reproduce this but our CI builds can't (we are talking about tens of builds
> daily on various platforms), need to understand why.
> Also from Makefile I can't see how this is happening, I am feeling uneasy to fix
> something before figuring out how/why it is failing.
> 
> Can you please try to collect more data on when/how this happens, initial
> questions I can think of:
> - Can you reproduce this with meson build?
> - Is it bare DPDK build, or build part of other project (I guess I saw fd.io on the
> link)
> - - If this is not bare DPDK build what changes has been done to build system?
> - Do you see this with fresh build (new clone) or rebuild of existing clone?
> - Can you confirm you have correct RTE_SDK and RTE_TARGET environment
> variables?
> - Can you please share your build command?
> 

I sent an e-mail to dpdk dev a few days back asking for help where I outlined what we're doing:
We're not doing anything special, just downloading and extracting the archive, then setting CONFIG_RTE_LIBRTE_MLX5_PMD and CONFIG_RTE_LIBRTE_I40E_16BYTE_RX_DESC to y in config/common_base and then running make install T=arm64-armv8a-linuxapp-gcc -j. As mentioned in the subject, the build server is a Taishan ARM server.

We're doing a fresh rebuild everytime. The error doesn't happen everytime, just sometimes - it seems to be random.

We don't set RTE_SDK nor RTE_TARGET since https://doc.dpdk.org/guides/linux_gsg/build_dpdk.html#installation-of-dpdk-target-environment-using-make doesn't mention those.

I'll try Meson build a few times. How can I enable those two config options in Meson?

> Thanks,
> ferruh
> 
> >
> >>>
> >>>
> >>>> Fix this by replacing the directive with #include "file" which
> >>>> searches for files first in the directory containing the current
> >>>> file and other directories only after that.
> >>>>
> >>>> Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
> >>>
> >>> What about #include <rte_ethdev_trace.h> ?
> >>>
> >>> Except this,
> >>> Acked-by: Thomas Monjalon <thomas@monjalon.net>
> >>>
> >>>
> >>> I would change the title to be more generic (nothing specific to gcc
> >>> or Arm
> >> here):
> >>> 	ethdev: fix local include
> >>>
> >>> It needs to be backported, so these lines are required:
> >>> 	Fixes: 7f0a669e7b04 ("ethdev: add allocation helper for virtual drivers")
> >>> 	Fixes: dcd5c8112bc3 ("ethdev: add PCI driver helpers")
> >>> 	Fixes: ffc905f3b856 ("ethdev: separate driver APIs")
> >>> 	Fixes: 331c447ad913 ("ethdev: separate internal structures into own
> >> header")
> >>> 	Cc: stable@dpdk.org
> >>>
> >>>
> >>>
> >

Re: [dpdk-dev] [PATCH] ethdev: fix dpdk gcc build on Arm

[Ferruh Yigit]

On 6/4/2020 1:55 PM, Juraj Linkeš wrote:
>> -----Original Message-----
>> From: Ferruh Yigit <ferruh.yigit@intel.com>
>> Sent: Thursday, June 4, 2020 2:23 PM
>> To: Juraj Linkeš <juraj.linkes@pantheon.tech>; Thomas Monjalon
>> <thomas@monjalon.net>
>> Cc: arybchenko@solarflare.com; dev@dpdk.org; Honnappa Nagarahalli
>> <Honnappa.Nagarahalli@arm.com>
>> Subject: Re: [PATCH] ethdev: fix dpdk gcc build on Arm
>>
>> On 6/4/2020 11:36 AM, Juraj Linkeš wrote:
>>>
>>>
>>>> -----Original Message-----
>>>> From: Ferruh Yigit <ferruh.yigit@intel.com>
>>>> Sent: Wednesday, June 3, 2020 1:41 PM
>>>> To: Thomas Monjalon <thomas@monjalon.net>; Juraj Linkeš
>>>> <juraj.linkes@pantheon.tech>
>>>> Cc: arybchenko@solarflare.com; dev@dpdk.org
>>>> Subject: Re: [PATCH] ethdev: fix dpdk gcc build on Arm
>>>>
>>>> On 6/3/2020 11:16 AM, Thomas Monjalon wrote:
>>>>> 03/06/2020 11:48, Juraj Linkeš:
>>>>>> Directive #include <file> in gcc implementation searches for files
>>>>>> in a standard list of system directories, which leads to a
>>>>>> sporadici build error on Taishan arm machines:
>>>>>> /tmp/openvpp-testing/dpdk/lib/librte_ethdev/rte_ethdev.h:4287:10:
>>>>>> fatal error: rte_ethdev_core.h:
>>>>>> No such file or directory #include <rte_ethdev_core.h>
>>>>>
>>>>> Would be interesting to know why nobody else hit such error?
>>>>
>>>>
>>>> I can't see why this is happening, in the 'mk/rte.lib.mk' we have following:
>>>>
>>>> "
>>>> install: _preinstall build _postinstall
>>>> build: _preinstall
>>>> "
>>>>
>>>> Which should cause the library header files installed before building
>>>> .c files in that library.
>>>> So when compiling 'rte_class_eth.c', the header files should be
>>>> already in install folder.
>>>>
>>>>
>>>> I can see how/why changing to "" fixes the issue but I am not sure about this
>> fix.
>>>> "rte_ethdev.h" is a public header file, that applications will
>>>> include it in their applications. In the public library it is more
>>>> proper to have other includes from system folder, using format <>.
>>>> Again, I can't see why it is failing but I believe we should find
>>>> another solution for _internal_ build error.
>>>>
>>>>
>>>> A very simple solution can be following, but that is also not good,
>>>> since it solves the issue by creating a dependency to the order of the header
>> includes:
>>>>  diff --git a/lib/librte_ethdev/rte_class_eth.c
>>>> b/lib/librte_ethdev/rte_class_eth.c
>>>>  index 6338355e25..3030c49020 100644
>>>>  --- a/lib/librte_ethdev/rte_class_eth.c
>>>>  +++ b/lib/librte_ethdev/rte_class_eth.c
>>>>  @@ -10,8 +10,8 @@
>>>>   #include <rte_kvargs.h>
>>>>   #include <rte_log.h>
>>>>
>>>>  -#include "rte_ethdev.h"
>>>>   #include "rte_ethdev_core.h"
>>>>  +#include "rte_ethdev.h"
>>>>   #include "rte_ethdev_driver.h"
>>>>   #include "ethdev_private.h"
>>>>
>>>
>>> Thomas, Ferruh, what should be the solution? I'm not an expert on this and I
>> can't really offer anything better, but I'd like that this gets fixed.
>>
>> First we need to root cause this before trying to solve it. Honnappa also said he
>> can reproduce this but our CI builds can't (we are talking about tens of builds
>> daily on various platforms), need to understand why.
>> Also from Makefile I can't see how this is happening, I am feeling uneasy to fix
>> something before figuring out how/why it is failing.
>>
>> Can you please try to collect more data on when/how this happens, initial
>> questions I can think of:
>> - Can you reproduce this with meson build?
>> - Is it bare DPDK build, or build part of other project (I guess I saw fd.io on the
>> link)
>> - - If this is not bare DPDK build what changes has been done to build system?
>> - Do you see this with fresh build (new clone) or rebuild of existing clone?
>> - Can you confirm you have correct RTE_SDK and RTE_TARGET environment
>> variables?
>> - Can you please share your build command?
>>
> 
> I sent an e-mail to dpdk dev a few days back asking for help where I outlined what we're doing:
> We're not doing anything special, just downloading and extracting the archive, then setting CONFIG_RTE_LIBRTE_MLX5_PMD and CONFIG_RTE_LIBRTE_I40E_16BYTE_RX_DESC to y in config/common_base and then running make install T=arm64-armv8a-linuxapp-gcc -j. As mentioned in the subject, the build server is a Taishan ARM server.
> 
> We're doing a fresh rebuild everytime. The error doesn't happen everytime, just sometimes - it seems to be random.
> 
> We don't set RTE_SDK nor RTE_TARGET since https://doc.dpdk.org/guides/linux_gsg/build_dpdk.html#installation-of-dpdk-target-environment-using-make doesn't mention those.

Can you please try with "V=1",
"make install T=arm64-armv8a-linuxapp-gcc -j V=1",
this will generate verbose output.

And _before_ lines compiling the 'rte_class_eth.c' you should see something like
following lines, can you please check:
"
SYMLINK-FILE include/rte_ethdev_core.h

ln -nsf `.../buildtools/relpath.sh .../lib/librte_ethdev/rte_ethdev_core.h
.../x86_64-native-linux-gcc/include` .../x86_64-native-linux-gcc/include
"

Also for the failed case, after fail can you please check if
a) '.../arm64-armv8a-linuxapp-gcc/build/lib/librte_ethdev/_preinstall' file exists
b) '.../x86_64-native-linux-gcc/include/rte_ethdev_core.h' exists and it linked
to correct file '../../lib/librte_ethdev/rte_ethdev_core.h'


> 
> I'll try Meson build a few times. How can I enable those two config options in Meson?
> 
>> Thanks,
>> ferruh
>>
>>>
>>>>>
>>>>>
>>>>>> Fix this by replacing the directive with #include "file" which
>>>>>> searches for files first in the directory containing the current
>>>>>> file and other directories only after that.
>>>>>>
>>>>>> Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
>>>>>
>>>>> What about #include <rte_ethdev_trace.h> ?
>>>>>
>>>>> Except this,
>>>>> Acked-by: Thomas Monjalon <thomas@monjalon.net>
>>>>>
>>>>>
>>>>> I would change the title to be more generic (nothing specific to gcc
>>>>> or Arm
>>>> here):
>>>>> 	ethdev: fix local include
>>>>>
>>>>> It needs to be backported, so these lines are required:
>>>>> 	Fixes: 7f0a669e7b04 ("ethdev: add allocation helper for virtual drivers")
>>>>> 	Fixes: dcd5c8112bc3 ("ethdev: add PCI driver helpers")
>>>>> 	Fixes: ffc905f3b856 ("ethdev: separate driver APIs")
>>>>> 	Fixes: 331c447ad913 ("ethdev: separate internal structures into own
>>>> header")
>>>>> 	Cc: stable@dpdk.org
>>>>>
>>>>>
>>>>>
>>>
> 

Re: [dpdk-dev] [PATCH] ethdev: fix dpdk gcc build on Arm

[Juraj Linkeš]

> -----Original Message-----
> From: Ferruh Yigit <ferruh.yigit@intel.com>
> Sent: Thursday, June 4, 2020 3:30 PM
> To: Juraj Linkeš <juraj.linkes@pantheon.tech>; Thomas Monjalon
> <thomas@monjalon.net>
> Cc: arybchenko@solarflare.com; dev@dpdk.org; Honnappa Nagarahalli
> <Honnappa.Nagarahalli@arm.com>
> Subject: Re: [PATCH] ethdev: fix dpdk gcc build on Arm
> 
> On 6/4/2020 1:55 PM, Juraj Linkeš wrote:
> >> -----Original Message-----
> >> From: Ferruh Yigit <ferruh.yigit@intel.com>
> >> Sent: Thursday, June 4, 2020 2:23 PM
> >> To: Juraj Linkeš <juraj.linkes@pantheon.tech>; Thomas Monjalon
> >> <thomas@monjalon.net>
> >> Cc: arybchenko@solarflare.com; dev@dpdk.org; Honnappa Nagarahalli
> >> <Honnappa.Nagarahalli@arm.com>
> >> Subject: Re: [PATCH] ethdev: fix dpdk gcc build on Arm
> >>
> >> On 6/4/2020 11:36 AM, Juraj Linkeš wrote:
> >>>
> >>>
> >>>> -----Original Message-----
> >>>> From: Ferruh Yigit <ferruh.yigit@intel.com>
> >>>> Sent: Wednesday, June 3, 2020 1:41 PM
> >>>> To: Thomas Monjalon <thomas@monjalon.net>; Juraj Linkeš
> >>>> <juraj.linkes@pantheon.tech>
> >>>> Cc: arybchenko@solarflare.com; dev@dpdk.org
> >>>> Subject: Re: [PATCH] ethdev: fix dpdk gcc build on Arm
> >>>>
> >>>> On 6/3/2020 11:16 AM, Thomas Monjalon wrote:
> >>>>> 03/06/2020 11:48, Juraj Linkeš:
> >>>>>> Directive #include <file> in gcc implementation searches for
> >>>>>> files in a standard list of system directories, which leads to a
> >>>>>> sporadici build error on Taishan arm machines:
> >>>>>> /tmp/openvpp-testing/dpdk/lib/librte_ethdev/rte_ethdev.h:4287:10:
> >>>>>> fatal error: rte_ethdev_core.h:
> >>>>>> No such file or directory #include <rte_ethdev_core.h>
> >>>>>
> >>>>> Would be interesting to know why nobody else hit such error?
> >>>>
> >>>>
> >>>> I can't see why this is happening, in the 'mk/rte.lib.mk' we have following:
> >>>>
> >>>> "
> >>>> install: _preinstall build _postinstall
> >>>> build: _preinstall
> >>>> "
> >>>>
> >>>> Which should cause the library header files installed before
> >>>> building .c files in that library.
> >>>> So when compiling 'rte_class_eth.c', the header files should be
> >>>> already in install folder.
> >>>>
> >>>>
> >>>> I can see how/why changing to "" fixes the issue but I am not sure
> >>>> about this
> >> fix.
> >>>> "rte_ethdev.h" is a public header file, that applications will
> >>>> include it in their applications. In the public library it is more
> >>>> proper to have other includes from system folder, using format <>.
> >>>> Again, I can't see why it is failing but I believe we should find
> >>>> another solution for _internal_ build error.
> >>>>
> >>>>
> >>>> A very simple solution can be following, but that is also not good,
> >>>> since it solves the issue by creating a dependency to the order of
> >>>> the header
> >> includes:
> >>>>  diff --git a/lib/librte_ethdev/rte_class_eth.c
> >>>> b/lib/librte_ethdev/rte_class_eth.c
> >>>>  index 6338355e25..3030c49020 100644
> >>>>  --- a/lib/librte_ethdev/rte_class_eth.c
> >>>>  +++ b/lib/librte_ethdev/rte_class_eth.c
> >>>>  @@ -10,8 +10,8 @@
> >>>>   #include <rte_kvargs.h>
> >>>>   #include <rte_log.h>
> >>>>
> >>>>  -#include "rte_ethdev.h"
> >>>>   #include "rte_ethdev_core.h"
> >>>>  +#include "rte_ethdev.h"
> >>>>   #include "rte_ethdev_driver.h"
> >>>>   #include "ethdev_private.h"
> >>>>
> >>>
> >>> Thomas, Ferruh, what should be the solution? I'm not an expert on
> >>> this and I
> >> can't really offer anything better, but I'd like that this gets fixed.
> >>
> >> First we need to root cause this before trying to solve it. Honnappa
> >> also said he can reproduce this but our CI builds can't (we are
> >> talking about tens of builds daily on various platforms), need to understand
> why.
> >> Also from Makefile I can't see how this is happening, I am feeling
> >> uneasy to fix something before figuring out how/why it is failing.
> >>
> >> Can you please try to collect more data on when/how this happens,
> >> initial questions I can think of:
> >> - Can you reproduce this with meson build?
> >> - Is it bare DPDK build, or build part of other project (I guess I
> >> saw fd.io on the
> >> link)
> >> - - If this is not bare DPDK build what changes has been done to build system?
> >> - Do you see this with fresh build (new clone) or rebuild of existing clone?
> >> - Can you confirm you have correct RTE_SDK and RTE_TARGET environment
> >> variables?
> >> - Can you please share your build command?
> >>
> >
> > I sent an e-mail to dpdk dev a few days back asking for help where I outlined
> what we're doing:
> > We're not doing anything special, just downloading and extracting the archive,
> then setting CONFIG_RTE_LIBRTE_MLX5_PMD and
> CONFIG_RTE_LIBRTE_I40E_16BYTE_RX_DESC to y in config/common_base and
> then running make install T=arm64-armv8a-linuxapp-gcc -j. As mentioned in the
> subject, the build server is a Taishan ARM server.
> >
> > We're doing a fresh rebuild everytime. The error doesn't happen everytime,
> just sometimes - it seems to be random.
> >
> > We don't set RTE_SDK nor RTE_TARGET since
> https://doc.dpdk.org/guides/linux_gsg/build_dpdk.html#installation-of-dpdk-
> target-environment-using-make doesn't mention those.
> 
> Can you please try with "V=1",
> "make install T=arm64-armv8a-linuxapp-gcc -j V=1", this will generate verbose
> output.
> 
> And _before_ lines compiling the 'rte_class_eth.c' you should see something like
> following lines, can you please check:
> "
> SYMLINK-FILE include/rte_ethdev_core.h
> 
> ln -nsf `.../buildtools/relpath.sh .../lib/librte_ethdev/rte_ethdev_core.h
> .../x86_64-native-linux-gcc/include` .../x86_64-native-linux-gcc/include
> "
> 
> Also for the failed case, after fail can you please check if
> a) '.../arm64-armv8a-linuxapp-gcc/build/lib/librte_ethdev/_preinstall' file exists
> b) '.../x86_64-native-linux-gcc/include/rte_ethdev_core.h' exists and it linked to
> correct file '../../lib/librte_ethdev/rte_ethdev_core.h'
> 

Thanks, I'll try with V=1. In the meantine, I tried a lot of meson builds and I didn't hit this, but I'll do some more testing to be sure.

> 
> >
> > I'll try Meson build a few times. How can I enable those two config options in
> Meson?
> >
> >> Thanks,
> >> ferruh
> >>
> >>>
> >>>>>
> >>>>>
> >>>>>> Fix this by replacing the directive with #include "file" which
> >>>>>> searches for files first in the directory containing the current
> >>>>>> file and other directories only after that.
> >>>>>>
> >>>>>> Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
> >>>>>
> >>>>> What about #include <rte_ethdev_trace.h> ?
> >>>>>
> >>>>> Except this,
> >>>>> Acked-by: Thomas Monjalon <thomas@monjalon.net>
> >>>>>
> >>>>>
> >>>>> I would change the title to be more generic (nothing specific to
> >>>>> gcc or Arm
> >>>> here):
> >>>>> 	ethdev: fix local include
> >>>>>
> >>>>> It needs to be backported, so these lines are required:
> >>>>> 	Fixes: 7f0a669e7b04 ("ethdev: add allocation helper for virtual drivers")
> >>>>> 	Fixes: dcd5c8112bc3 ("ethdev: add PCI driver helpers")
> >>>>> 	Fixes: ffc905f3b856 ("ethdev: separate driver APIs")
> >>>>> 	Fixes: 331c447ad913 ("ethdev: separate internal structures into
> >>>>> own
> >>>> header")
> >>>>> 	Cc: stable@dpdk.org
> >>>>>
> >>>>>
> >>>>>
> >>>
> >

Re: [dpdk-dev] [PATCH] ethdev: fix dpdk gcc build on Arm

[Juraj Linkeš]

> -----Original Message-----
> From: Ferruh Yigit <ferruh.yigit@intel.com>
> Sent: Thursday, June 4, 2020 3:30 PM
> To: Juraj Linkeš <juraj.linkes@pantheon.tech>; Thomas Monjalon
> <thomas@monjalon.net>
> Cc: arybchenko@solarflare.com; dev@dpdk.org; Honnappa Nagarahalli
> <Honnappa.Nagarahalli@arm.com>
> Subject: Re: [PATCH] ethdev: fix dpdk gcc build on Arm
> 
> On 6/4/2020 1:55 PM, Juraj Linkeš wrote:
> >> -----Original Message-----
> >> From: Ferruh Yigit <ferruh.yigit@intel.com>
> >> Sent: Thursday, June 4, 2020 2:23 PM
> >> To: Juraj Linkeš <juraj.linkes@pantheon.tech>; Thomas Monjalon
> >> <thomas@monjalon.net>
> >> Cc: arybchenko@solarflare.com; dev@dpdk.org; Honnappa Nagarahalli
> >> <Honnappa.Nagarahalli@arm.com>
> >> Subject: Re: [PATCH] ethdev: fix dpdk gcc build on Arm
> >>
> >> On 6/4/2020 11:36 AM, Juraj Linkeš wrote:
> >>>
> >>>
> >>>> -----Original Message-----
> >>>> From: Ferruh Yigit <ferruh.yigit@intel.com>
> >>>> Sent: Wednesday, June 3, 2020 1:41 PM
> >>>> To: Thomas Monjalon <thomas@monjalon.net>; Juraj Linkeš
> >>>> <juraj.linkes@pantheon.tech>
> >>>> Cc: arybchenko@solarflare.com; dev@dpdk.org
> >>>> Subject: Re: [PATCH] ethdev: fix dpdk gcc build on Arm
> >>>>
> >>>> On 6/3/2020 11:16 AM, Thomas Monjalon wrote:
> >>>>> 03/06/2020 11:48, Juraj Linkeš:
> >>>>>> Directive #include <file> in gcc implementation searches for
> >>>>>> files in a standard list of system directories, which leads to a
> >>>>>> sporadici build error on Taishan arm machines:
> >>>>>> /tmp/openvpp-testing/dpdk/lib/librte_ethdev/rte_ethdev.h:4287:10:
> >>>>>> fatal error: rte_ethdev_core.h:
> >>>>>> No such file or directory #include <rte_ethdev_core.h>
> >>>>>
> >>>>> Would be interesting to know why nobody else hit such error?
> >>>>
> >>>>
> >>>> I can't see why this is happening, in the 'mk/rte.lib.mk' we have following:
> >>>>
> >>>> "
> >>>> install: _preinstall build _postinstall
> >>>> build: _preinstall
> >>>> "
> >>>>
> >>>> Which should cause the library header files installed before
> >>>> building .c files in that library.
> >>>> So when compiling 'rte_class_eth.c', the header files should be
> >>>> already in install folder.
> >>>>
> >>>>
> >>>> I can see how/why changing to "" fixes the issue but I am not sure
> >>>> about this
> >> fix.
> >>>> "rte_ethdev.h" is a public header file, that applications will
> >>>> include it in their applications. In the public library it is more
> >>>> proper to have other includes from system folder, using format <>.
> >>>> Again, I can't see why it is failing but I believe we should find
> >>>> another solution for _internal_ build error.
> >>>>
> >>>>
> >>>> A very simple solution can be following, but that is also not good,
> >>>> since it solves the issue by creating a dependency to the order of
> >>>> the header
> >> includes:
> >>>>  diff --git a/lib/librte_ethdev/rte_class_eth.c
> >>>> b/lib/librte_ethdev/rte_class_eth.c
> >>>>  index 6338355e25..3030c49020 100644
> >>>>  --- a/lib/librte_ethdev/rte_class_eth.c
> >>>>  +++ b/lib/librte_ethdev/rte_class_eth.c
> >>>>  @@ -10,8 +10,8 @@
> >>>>   #include <rte_kvargs.h>
> >>>>   #include <rte_log.h>
> >>>>
> >>>>  -#include "rte_ethdev.h"
> >>>>   #include "rte_ethdev_core.h"
> >>>>  +#include "rte_ethdev.h"
> >>>>   #include "rte_ethdev_driver.h"
> >>>>   #include "ethdev_private.h"
> >>>>
> >>>
> >>> Thomas, Ferruh, what should be the solution? I'm not an expert on
> >>> this and I
> >> can't really offer anything better, but I'd like that this gets fixed.
> >>
> >> First we need to root cause this before trying to solve it. Honnappa
> >> also said he can reproduce this but our CI builds can't (we are
> >> talking about tens of builds daily on various platforms), need to understand
> why.
> >> Also from Makefile I can't see how this is happening, I am feeling
> >> uneasy to fix something before figuring out how/why it is failing.
> >>
> >> Can you please try to collect more data on when/how this happens,
> >> initial questions I can think of:
> >> - Can you reproduce this with meson build?
> >> - Is it bare DPDK build, or build part of other project (I guess I
> >> saw fd.io on the
> >> link)
> >> - - If this is not bare DPDK build what changes has been done to build system?
> >> - Do you see this with fresh build (new clone) or rebuild of existing clone?
> >> - Can you confirm you have correct RTE_SDK and RTE_TARGET environment
> >> variables?
> >> - Can you please share your build command?
> >>
> >
> > I sent an e-mail to dpdk dev a few days back asking for help where I outlined
> what we're doing:
> > We're not doing anything special, just downloading and extracting the archive,
> then setting CONFIG_RTE_LIBRTE_MLX5_PMD and
> CONFIG_RTE_LIBRTE_I40E_16BYTE_RX_DESC to y in config/common_base and
> then running make install T=arm64-armv8a-linuxapp-gcc -j. As mentioned in the
> subject, the build server is a Taishan ARM server.
> >
> > We're doing a fresh rebuild everytime. The error doesn't happen everytime,
> just sometimes - it seems to be random.
> >
> > We don't set RTE_SDK nor RTE_TARGET since
> https://doc.dpdk.org/guides/linux_gsg/build_dpdk.html#installation-of-dpdk-
> target-environment-using-make doesn't mention those.
> 
> Can you please try with "V=1",
> "make install T=arm64-armv8a-linuxapp-gcc -j V=1", this will generate verbose
> output.
> 
> And _before_ lines compiling the 'rte_class_eth.c' you should see something like
> following lines, can you please check:
> "
> SYMLINK-FILE include/rte_ethdev_core.h
> 
> ln -nsf `.../buildtools/relpath.sh .../lib/librte_ethdev/rte_ethdev_core.h
> .../x86_64-native-linux-gcc/include` .../x86_64-native-linux-gcc/include
> "
> 

This is the log with V=1 build (a failed build):
== Build lib/librte_ethdev
== Build lib/librte_cmdline
ar crDs librte_lpm.a rte_lpm.o rte_lpm6.o
  INSTALL-LIB librte_lpm.a
ar crDs librte_member.a rte_member.o rte_member_ht.o rte_member_vbf.o
  INSTALL-LIB librte_member.a
ar crDs librte_cryptodev.a rte_cryptodev.o rte_cryptodev_pmd.o
  INSTALL-LIB librte_cryptodev.a
ar crDs librte_bbdev.a rte_bbdev.o
ar crDs librte_compressdev.a rte_compressdev.o rte_compressdev_pmd.o rte_comp.o
  INSTALL-LIB librte_compressdev.a
  INSTALL-LIB librte_bbdev.a
  SYMLINK-FILE include/rte_ethdev_pci.h
  SYMLINK-FILE include/rte_tm.h
  SYMLINK-FILE include/rte_tm_driver.h
  SYMLINK-FILE include/rte_eth_ctrl.h
  SYMLINK-FILE include/rte_flow_driver.h
  SYMLINK-FILE include/rte_ethdev_vdev.h
  SYMLINK-FILE include/rte_mtr_driver.h
  SYMLINK-FILE include/rte_ethdev_core.h
  SYMLINK-FILE include/rte_flow.h
  SYMLINK-FILE include/rte_mtr.h
  SYMLINK-FILE include/rte_ethdev_driver.h
  SYMLINK-FILE include/rte_ethdev.h
  SYMLINK-FILE include/rte_dev_info.h
gcc -Wp,-MD,./.ethdev_private.o.d.tmp   -pthread -I/tmp/dpdk-20.02/lib/librte_eal/linux/eal/include  -march=armv8-a+crc -DRTE_MACHINE_CPUFLAG_NEON -DRTE_MACHINE_CPUFLAG_CRC32  -I/tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include -DRTE_USE_FUNCTION_VERSIONING -include /tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include/rte_config.h -D_GNU_SOURCE -DALLOW_EXPERIMENTAL_API -O3 -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wmissing-declarations -Wold-style-definition -Wpointer-arith -Wcast-align -Wnested-externs -Wcast-qual -Wformat-nonliteral -Wformat-security -Wundef -Wwrite-strings -Wdeprecated -Wno-missing-field-initializers -Wimplicit-fallthrough=2 -Wno-format-truncation -Wno-address-of-packed-member    -o ethdev_private.o -c /tmp/dpdk-20.02/lib/librte_ethdev/ethdev_private.c
gcc -Wp,-MD,./.rte_ethdev.o.d.tmp   -pthread -I/tmp/dpdk-20.02/lib/librte_eal/linux/eal/include  -march=armv8-a+crc -DRTE_MACHINE_CPUFLAG_NEON -DRTE_MACHINE_CPUFLAG_CRC32  -I/tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include -DRTE_USE_FUNCTION_VERSIONING -include /tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include/rte_config.h -D_GNU_SOURCE -DALLOW_EXPERIMENTAL_API -O3 -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wmissing-declarations -Wold-style-definition -Wpointer-arith -Wcast-align -Wnested-externs -Wcast-qual -Wformat-nonliteral -Wformat-security -Wundef -Wwrite-strings -Wdeprecated -Wno-missing-field-initializers -Wimplicit-fallthrough=2 -Wno-format-truncation -Wno-address-of-packed-member    -o rte_ethdev.o -c /tmp/dpdk-20.02/lib/librte_ethdev/rte_ethdev.c
gcc -Wp,-MD,./.rte_class_eth.o.d.tmp   -pthread -I/tmp/dpdk-20.02/lib/librte_eal/linux/eal/include  -march=armv8-a+crc -DRTE_MACHINE_CPUFLAG_NEON -DRTE_MACHINE_CPUFLAG_CRC32  -I/tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include -DRTE_USE_FUNCTION_VERSIONING -include /tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include/rte_config.h -D_GNU_SOURCE -DALLOW_EXPERIMENTAL_API -O3 -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wmissing-declarations -Wold-style-definition -Wpointer-arith -Wcast-align -Wnested-externs -Wcast-qual -Wformat-nonliteral -Wformat-security -Wundef -Wwrite-strings -Wdeprecated -Wno-missing-field-initializers -Wimplicit-fallthrough=2 -Wno-format-truncation -Wno-address-of-packed-member    -o rte_class_eth.o -c /tmp/dpdk-20.02/lib/librte_ethdev/rte_class_eth.c
gcc -Wp,-MD,./.rte_flow.o.d.tmp   -pthread -I/tmp/dpdk-20.02/lib/librte_eal/linux/eal/include  -march=armv8-a+crc -DRTE_MACHINE_CPUFLAG_NEON -DRTE_MACHINE_CPUFLAG_CRC32  -I/tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include -DRTE_USE_FUNCTION_VERSIONING -include /tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include/rte_config.h -D_GNU_SOURCE -DALLOW_EXPERIMENTAL_API -O3 -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wmissing-declarations -Wold-style-definition -Wpointer-arith -Wcast-align -Wnested-externs -Wcast-qual -Wformat-nonliteral -Wformat-security -Wundef -Wwrite-strings -Wdeprecated -Wno-missing-field-initializers -Wimplicit-fallthrough=2 -Wno-format-truncation -Wno-address-of-packed-member    -o rte_flow.o -c /tmp/dpdk-20.02/lib/librte_ethdev/rte_flow.c
gcc -Wp,-MD,./.rte_tm.o.d.tmp   -pthread -I/tmp/dpdk-20.02/lib/librte_eal/linux/eal/include  -march=armv8-a+crc -DRTE_MACHINE_CPUFLAG_NEON -DRTE_MACHINE_CPUFLAG_CRC32  -I/tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include -DRTE_USE_FUNCTION_VERSIONING -include /tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include/rte_config.h -D_GNU_SOURCE -DALLOW_EXPERIMENTAL_API -O3 -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wmissing-declarations -Wold-style-definition -Wpointer-arith -Wcast-align -Wnested-externs -Wcast-qual -Wformat-nonliteral -Wformat-security -Wundef -Wwrite-strings -Wdeprecated -Wno-missing-field-initializers -Wimplicit-fallthrough=2 -Wno-format-truncation -Wno-address-of-packed-member    -o rte_tm.o -c /tmp/dpdk-20.02/lib/librte_ethdev/rte_tm.c
  SYMLINK-FILE include/cmdline.h
  SYMLINK-FILE include/cmdline_parse.h
  SYMLINK-FILE include/cmdline_parse_num.h
gcc -Wp,-MD,./.ethdev_profile.o.d.tmp   -pthread -I/tmp/dpdk-20.02/lib/librte_eal/linux/eal/include  -march=armv8-a+crc -DRTE_MACHINE_CPUFLAG_NEON -DRTE_MACHINE_CPUFLAG_CRC32  -I/tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include -DRTE_USE_FUNCTION_VERSIONING -include /tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include/rte_config.h -D_GNU_SOURCE -DALLOW_EXPERIMENTAL_API -O3 -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wmissing-declarations -Wold-style-definition -Wpointer-arith -Wcast-align -Wnested-externs -Wcast-qual -Wformat-nonliteral -Wformat-security -Wundef -Wwrite-strings -Wdeprecated -Wno-missing-field-initializers -Wimplicit-fallthrough=2 -Wno-format-truncation -Wno-address-of-packed-member    -o ethdev_profile.o -c /tmp/dpdk-20.02/lib/librte_ethdev/ethdev_profile.c
gcc -Wp,-MD,./.rte_mtr.o.d.tmp   -pthread -I/tmp/dpdk-20.02/lib/librte_eal/linux/eal/include  -march=armv8-a+crc -DRTE_MACHINE_CPUFLAG_NEON -DRTE_MACHINE_CPUFLAG_CRC32  -I/tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include -DRTE_USE_FUNCTION_VERSIONING -include /tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include/rte_config.h -D_GNU_SOURCE -DALLOW_EXPERIMENTAL_API -O3 -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wmissing-declarations -Wold-style-definition -Wpointer-arith -Wcast-align -Wnested-externs -Wcast-qual -Wformat-nonliteral -Wformat-security -Wundef -Wwrite-strings -Wdeprecated -Wno-missing-field-initializers -Wimplicit-fallthrough=2 -Wno-format-truncation -Wno-address-of-packed-member    -o rte_mtr.o -c /tmp/dpdk-20.02/lib/librte_ethdev/rte_mtr.c
  SYMLINK-FILE include/rte_sched.h
  SYMLINK-FILE include/rte_red.h
  SYMLINK-FILE include/rte_approx.h
In file included from /tmp/dpdk-20.02/lib/librte_ethdev/rte_ethdev.c:43:
/tmp/dpdk-20.02/lib/librte_ethdev/rte_ethdev_driver.h:18:10: fatal error: rte_ethdev.h: No such file or directory
 #include <rte_ethdev.h>
          ^~~~~~~~~~~~~~
compilation terminated.
/tmp/dpdk-20.02/mk/internal/rte.compile-pre.mk:114: recipe for target 'rte_ethdev.o' failed

> Also for the failed case, after fail can you please check if
> a) '.../arm64-armv8a-linuxapp-gcc/build/lib/librte_ethdev/_preinstall' file exists
> b) '.../x86_64-native-linux-gcc/include/rte_ethdev_core.h' exists and it linked to
> correct file '../../lib/librte_ethdev/rte_ethdev_core.h'
> 

a) it doesn't exist
b) all of these exist:
rte_ethdev_core.h -> ../../lib/librte_ethdev/rte_ethdev_core.h
rte_ethdev_driver.h -> ../../lib/librte_ethdev/rte_ethdev_driver.h
rte_ethdev.h -> ../../lib/librte_ethdev/rte_ethdev.h
rte_ethdev_pci.h -> ../../lib/librte_ethdev/rte_ethdev_pci.h
rte_ethdev_vdev.h -> ../../lib/librte_ethdev/rte_ethdev_vdev.h

I was able to reproduce the failure after 7 successful builds.

> 
> >
> > I'll try Meson build a few times. How can I enable those two config options in
> Meson?
> >
> >> Thanks,
> >> ferruh
> >>
> >>>
> >>>>>
> >>>>>
> >>>>>> Fix this by replacing the directive with #include "file" which
> >>>>>> searches for files first in the directory containing the current
> >>>>>> file and other directories only after that.
> >>>>>>
> >>>>>> Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
> >>>>>
> >>>>> What about #include <rte_ethdev_trace.h> ?
> >>>>>
> >>>>> Except this,
> >>>>> Acked-by: Thomas Monjalon <thomas@monjalon.net>
> >>>>>
> >>>>>
> >>>>> I would change the title to be more generic (nothing specific to
> >>>>> gcc or Arm
> >>>> here):
> >>>>> 	ethdev: fix local include
> >>>>>
> >>>>> It needs to be backported, so these lines are required:
> >>>>> 	Fixes: 7f0a669e7b04 ("ethdev: add allocation helper for virtual drivers")
> >>>>> 	Fixes: dcd5c8112bc3 ("ethdev: add PCI driver helpers")
> >>>>> 	Fixes: ffc905f3b856 ("ethdev: separate driver APIs")
> >>>>> 	Fixes: 331c447ad913 ("ethdev: separate internal structures into
> >>>>> own
> >>>> header")
> >>>>> 	Cc: stable@dpdk.org
> >>>>>
> >>>>>
> >>>>>
> >>>
> >

Re: [dpdk-dev] [PATCH] ethdev: fix dpdk gcc build on Arm

[Ferruh Yigit]

On 6/8/2020 9:17 AM, Juraj Linkeš wrote:
> 
> 
>> -----Original Message-----
>> From: Ferruh Yigit <ferruh.yigit@intel.com>
>> Sent: Thursday, June 4, 2020 3:30 PM
>> To: Juraj Linkeš <juraj.linkes@pantheon.tech>; Thomas Monjalon
>> <thomas@monjalon.net>
>> Cc: arybchenko@solarflare.com; dev@dpdk.org; Honnappa Nagarahalli
>> <Honnappa.Nagarahalli@arm.com>
>> Subject: Re: [PATCH] ethdev: fix dpdk gcc build on Arm
>>
>> On 6/4/2020 1:55 PM, Juraj Linkeš wrote:
>>>> -----Original Message-----
>>>> From: Ferruh Yigit <ferruh.yigit@intel.com>
>>>> Sent: Thursday, June 4, 2020 2:23 PM
>>>> To: Juraj Linkeš <juraj.linkes@pantheon.tech>; Thomas Monjalon
>>>> <thomas@monjalon.net>
>>>> Cc: arybchenko@solarflare.com; dev@dpdk.org; Honnappa Nagarahalli
>>>> <Honnappa.Nagarahalli@arm.com>
>>>> Subject: Re: [PATCH] ethdev: fix dpdk gcc build on Arm
>>>>
>>>> On 6/4/2020 11:36 AM, Juraj Linkeš wrote:
>>>>>
>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Ferruh Yigit <ferruh.yigit@intel.com>
>>>>>> Sent: Wednesday, June 3, 2020 1:41 PM
>>>>>> To: Thomas Monjalon <thomas@monjalon.net>; Juraj Linkeš
>>>>>> <juraj.linkes@pantheon.tech>
>>>>>> Cc: arybchenko@solarflare.com; dev@dpdk.org
>>>>>> Subject: Re: [PATCH] ethdev: fix dpdk gcc build on Arm
>>>>>>
>>>>>> On 6/3/2020 11:16 AM, Thomas Monjalon wrote:
>>>>>>> 03/06/2020 11:48, Juraj Linkeš:
>>>>>>>> Directive #include <file> in gcc implementation searches for
>>>>>>>> files in a standard list of system directories, which leads to a
>>>>>>>> sporadici build error on Taishan arm machines:
>>>>>>>> /tmp/openvpp-testing/dpdk/lib/librte_ethdev/rte_ethdev.h:4287:10:
>>>>>>>> fatal error: rte_ethdev_core.h:
>>>>>>>> No such file or directory #include <rte_ethdev_core.h>
>>>>>>>
>>>>>>> Would be interesting to know why nobody else hit such error?
>>>>>>
>>>>>>
>>>>>> I can't see why this is happening, in the 'mk/rte.lib.mk' we have following:
>>>>>>
>>>>>> "
>>>>>> install: _preinstall build _postinstall
>>>>>> build: _preinstall
>>>>>> "
>>>>>>
>>>>>> Which should cause the library header files installed before
>>>>>> building .c files in that library.
>>>>>> So when compiling 'rte_class_eth.c', the header files should be
>>>>>> already in install folder.
>>>>>>
>>>>>>
>>>>>> I can see how/why changing to "" fixes the issue but I am not sure
>>>>>> about this
>>>> fix.
>>>>>> "rte_ethdev.h" is a public header file, that applications will
>>>>>> include it in their applications. In the public library it is more
>>>>>> proper to have other includes from system folder, using format <>.
>>>>>> Again, I can't see why it is failing but I believe we should find
>>>>>> another solution for _internal_ build error.
>>>>>>
>>>>>>
>>>>>> A very simple solution can be following, but that is also not good,
>>>>>> since it solves the issue by creating a dependency to the order of
>>>>>> the header
>>>> includes:
>>>>>>  diff --git a/lib/librte_ethdev/rte_class_eth.c
>>>>>> b/lib/librte_ethdev/rte_class_eth.c
>>>>>>  index 6338355e25..3030c49020 100644
>>>>>>  --- a/lib/librte_ethdev/rte_class_eth.c
>>>>>>  +++ b/lib/librte_ethdev/rte_class_eth.c
>>>>>>  @@ -10,8 +10,8 @@
>>>>>>   #include <rte_kvargs.h>
>>>>>>   #include <rte_log.h>
>>>>>>
>>>>>>  -#include "rte_ethdev.h"
>>>>>>   #include "rte_ethdev_core.h"
>>>>>>  +#include "rte_ethdev.h"
>>>>>>   #include "rte_ethdev_driver.h"
>>>>>>   #include "ethdev_private.h"
>>>>>>
>>>>>
>>>>> Thomas, Ferruh, what should be the solution? I'm not an expert on
>>>>> this and I
>>>> can't really offer anything better, but I'd like that this gets fixed.
>>>>
>>>> First we need to root cause this before trying to solve it. Honnappa
>>>> also said he can reproduce this but our CI builds can't (we are
>>>> talking about tens of builds daily on various platforms), need to understand
>> why.
>>>> Also from Makefile I can't see how this is happening, I am feeling
>>>> uneasy to fix something before figuring out how/why it is failing.
>>>>
>>>> Can you please try to collect more data on when/how this happens,
>>>> initial questions I can think of:
>>>> - Can you reproduce this with meson build?
>>>> - Is it bare DPDK build, or build part of other project (I guess I
>>>> saw fd.io on the
>>>> link)
>>>> - - If this is not bare DPDK build what changes has been done to build system?
>>>> - Do you see this with fresh build (new clone) or rebuild of existing clone?
>>>> - Can you confirm you have correct RTE_SDK and RTE_TARGET environment
>>>> variables?
>>>> - Can you please share your build command?
>>>>
>>>
>>> I sent an e-mail to dpdk dev a few days back asking for help where I outlined
>> what we're doing:
>>> We're not doing anything special, just downloading and extracting the archive,
>> then setting CONFIG_RTE_LIBRTE_MLX5_PMD and
>> CONFIG_RTE_LIBRTE_I40E_16BYTE_RX_DESC to y in config/common_base and
>> then running make install T=arm64-armv8a-linuxapp-gcc -j. As mentioned in the
>> subject, the build server is a Taishan ARM server.
>>>
>>> We're doing a fresh rebuild everytime. The error doesn't happen everytime,
>> just sometimes - it seems to be random.
>>>
>>> We don't set RTE_SDK nor RTE_TARGET since
>> https://doc.dpdk.org/guides/linux_gsg/build_dpdk.html#installation-of-dpdk-
>> target-environment-using-make doesn't mention those.
>>
>> Can you please try with "V=1",
>> "make install T=arm64-armv8a-linuxapp-gcc -j V=1", this will generate verbose
>> output.
>>
>> And _before_ lines compiling the 'rte_class_eth.c' you should see something like
>> following lines, can you please check:
>> "
>> SYMLINK-FILE include/rte_ethdev_core.h
>>
>> ln -nsf `.../buildtools/relpath.sh .../lib/librte_ethdev/rte_ethdev_core.h
>> .../x86_64-native-linux-gcc/include` .../x86_64-native-linux-gcc/include
>> "
>>
> 
> This is the log with V=1 build (a failed build):
> == Build lib/librte_ethdev
> == Build lib/librte_cmdline
> ar crDs librte_lpm.a rte_lpm.o rte_lpm6.o
>   INSTALL-LIB librte_lpm.a
> ar crDs librte_member.a rte_member.o rte_member_ht.o rte_member_vbf.o
>   INSTALL-LIB librte_member.a
> ar crDs librte_cryptodev.a rte_cryptodev.o rte_cryptodev_pmd.o
>   INSTALL-LIB librte_cryptodev.a
> ar crDs librte_bbdev.a rte_bbdev.o
> ar crDs librte_compressdev.a rte_compressdev.o rte_compressdev_pmd.o rte_comp.o
>   INSTALL-LIB librte_compressdev.a
>   INSTALL-LIB librte_bbdev.a
>   SYMLINK-FILE include/rte_ethdev_pci.h
>   SYMLINK-FILE include/rte_tm.h
>   SYMLINK-FILE include/rte_tm_driver.h
>   SYMLINK-FILE include/rte_eth_ctrl.h
>   SYMLINK-FILE include/rte_flow_driver.h
>   SYMLINK-FILE include/rte_ethdev_vdev.h
>   SYMLINK-FILE include/rte_mtr_driver.h
>   SYMLINK-FILE include/rte_ethdev_core.h
>   SYMLINK-FILE include/rte_flow.h
>   SYMLINK-FILE include/rte_mtr.h
>   SYMLINK-FILE include/rte_ethdev_driver.h
>   SYMLINK-FILE include/rte_ethdev.h
>   SYMLINK-FILE include/rte_dev_info.h
> gcc -Wp,-MD,./.ethdev_private.o.d.tmp   -pthread -I/tmp/dpdk-20.02/lib/librte_eal/linux/eal/include  -march=armv8-a+crc -DRTE_MACHINE_CPUFLAG_NEON -DRTE_MACHINE_CPUFLAG_CRC32  -I/tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include -DRTE_USE_FUNCTION_VERSIONING -include /tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include/rte_config.h -D_GNU_SOURCE -DALLOW_EXPERIMENTAL_API -O3 -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wmissing-declarations -Wold-style-definition -Wpointer-arith -Wcast-align -Wnested-externs -Wcast-qual -Wformat-nonliteral -Wformat-security -Wundef -Wwrite-strings -Wdeprecated -Wno-missing-field-initializers -Wimplicit-fallthrough=2 -Wno-format-truncation -Wno-address-of-packed-member    -o ethdev_private.o -c /tmp/dpdk-20.02/lib/librte_ethdev/ethdev_private.c
> gcc -Wp,-MD,./.rte_ethdev.o.d.tmp   -pthread -I/tmp/dpdk-20.02/lib/librte_eal/linux/eal/include  -march=armv8-a+crc -DRTE_MACHINE_CPUFLAG_NEON -DRTE_MACHINE_CPUFLAG_CRC32  -I/tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include -DRTE_USE_FUNCTION_VERSIONING -include /tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include/rte_config.h -D_GNU_SOURCE -DALLOW_EXPERIMENTAL_API -O3 -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wmissing-declarations -Wold-style-definition -Wpointer-arith -Wcast-align -Wnested-externs -Wcast-qual -Wformat-nonliteral -Wformat-security -Wundef -Wwrite-strings -Wdeprecated -Wno-missing-field-initializers -Wimplicit-fallthrough=2 -Wno-format-truncation -Wno-address-of-packed-member    -o rte_ethdev.o -c /tmp/dpdk-20.02/lib/librte_ethdev/rte_ethdev.c
> gcc -Wp,-MD,./.rte_class_eth.o.d.tmp   -pthread -I/tmp/dpdk-20.02/lib/librte_eal/linux/eal/include  -march=armv8-a+crc -DRTE_MACHINE_CPUFLAG_NEON -DRTE_MACHINE_CPUFLAG_CRC32  -I/tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include -DRTE_USE_FUNCTION_VERSIONING -include /tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include/rte_config.h -D_GNU_SOURCE -DALLOW_EXPERIMENTAL_API -O3 -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wmissing-declarations -Wold-style-definition -Wpointer-arith -Wcast-align -Wnested-externs -Wcast-qual -Wformat-nonliteral -Wformat-security -Wundef -Wwrite-strings -Wdeprecated -Wno-missing-field-initializers -Wimplicit-fallthrough=2 -Wno-format-truncation -Wno-address-of-packed-member    -o rte_class_eth.o -c /tmp/dpdk-20.02/lib/librte_ethdev/rte_class_eth.c
> gcc -Wp,-MD,./.rte_flow.o.d.tmp   -pthread -I/tmp/dpdk-20.02/lib/librte_eal/linux/eal/include  -march=armv8-a+crc -DRTE_MACHINE_CPUFLAG_NEON -DRTE_MACHINE_CPUFLAG_CRC32  -I/tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include -DRTE_USE_FUNCTION_VERSIONING -include /tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include/rte_config.h -D_GNU_SOURCE -DALLOW_EXPERIMENTAL_API -O3 -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wmissing-declarations -Wold-style-definition -Wpointer-arith -Wcast-align -Wnested-externs -Wcast-qual -Wformat-nonliteral -Wformat-security -Wundef -Wwrite-strings -Wdeprecated -Wno-missing-field-initializers -Wimplicit-fallthrough=2 -Wno-format-truncation -Wno-address-of-packed-member    -o rte_flow.o -c /tmp/dpdk-20.02/lib/librte_ethdev/rte_flow.c
> gcc -Wp,-MD,./.rte_tm.o.d.tmp   -pthread -I/tmp/dpdk-20.02/lib/librte_eal/linux/eal/include  -march=armv8-a+crc -DRTE_MACHINE_CPUFLAG_NEON -DRTE_MACHINE_CPUFLAG_CRC32  -I/tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include -DRTE_USE_FUNCTION_VERSIONING -include /tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include/rte_config.h -D_GNU_SOURCE -DALLOW_EXPERIMENTAL_API -O3 -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wmissing-declarations -Wold-style-definition -Wpointer-arith -Wcast-align -Wnested-externs -Wcast-qual -Wformat-nonliteral -Wformat-security -Wundef -Wwrite-strings -Wdeprecated -Wno-missing-field-initializers -Wimplicit-fallthrough=2 -Wno-format-truncation -Wno-address-of-packed-member    -o rte_tm.o -c /tmp/dpdk-20.02/lib/librte_ethdev/rte_tm.c
>   SYMLINK-FILE include/cmdline.h
>   SYMLINK-FILE include/cmdline_parse.h
>   SYMLINK-FILE include/cmdline_parse_num.h
> gcc -Wp,-MD,./.ethdev_profile.o.d.tmp   -pthread -I/tmp/dpdk-20.02/lib/librte_eal/linux/eal/include  -march=armv8-a+crc -DRTE_MACHINE_CPUFLAG_NEON -DRTE_MACHINE_CPUFLAG_CRC32  -I/tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include -DRTE_USE_FUNCTION_VERSIONING -include /tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include/rte_config.h -D_GNU_SOURCE -DALLOW_EXPERIMENTAL_API -O3 -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wmissing-declarations -Wold-style-definition -Wpointer-arith -Wcast-align -Wnested-externs -Wcast-qual -Wformat-nonliteral -Wformat-security -Wundef -Wwrite-strings -Wdeprecated -Wno-missing-field-initializers -Wimplicit-fallthrough=2 -Wno-format-truncation -Wno-address-of-packed-member    -o ethdev_profile.o -c /tmp/dpdk-20.02/lib/librte_ethdev/ethdev_profile.c
> gcc -Wp,-MD,./.rte_mtr.o.d.tmp   -pthread -I/tmp/dpdk-20.02/lib/librte_eal/linux/eal/include  -march=armv8-a+crc -DRTE_MACHINE_CPUFLAG_NEON -DRTE_MACHINE_CPUFLAG_CRC32  -I/tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include -DRTE_USE_FUNCTION_VERSIONING -include /tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include/rte_config.h -D_GNU_SOURCE -DALLOW_EXPERIMENTAL_API -O3 -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wmissing-declarations -Wold-style-definition -Wpointer-arith -Wcast-align -Wnested-externs -Wcast-qual -Wformat-nonliteral -Wformat-security -Wundef -Wwrite-strings -Wdeprecated -Wno-missing-field-initializers -Wimplicit-fallthrough=2 -Wno-format-truncation -Wno-address-of-packed-member    -o rte_mtr.o -c /tmp/dpdk-20.02/lib/librte_ethdev/rte_mtr.c
>   SYMLINK-FILE include/rte_sched.h
>   SYMLINK-FILE include/rte_red.h
>   SYMLINK-FILE include/rte_approx.h
> In file included from /tmp/dpdk-20.02/lib/librte_ethdev/rte_ethdev.c:43:
> /tmp/dpdk-20.02/lib/librte_ethdev/rte_ethdev_driver.h:18:10: fatal error: rte_ethdev.h: No such file or directory
>  #include <rte_ethdev.h>
>           ^~~~~~~~~~~~~~
> compilation terminated.
> /tmp/dpdk-20.02/mk/internal/rte.compile-pre.mk:114: recipe for target 'rte_ethdev.o' failed
> 
>> Also for the failed case, after fail can you please check if
>> a) '.../arm64-armv8a-linuxapp-gcc/build/lib/librte_ethdev/_preinstall' file exists
>> b) '.../x86_64-native-linux-gcc/include/rte_ethdev_core.h' exists and it linked to
>> correct file '../../lib/librte_ethdev/rte_ethdev_core.h'
>>
> 
> a) it doesn't exist
> b) all of these exist:
> rte_ethdev_core.h -> ../../lib/librte_ethdev/rte_ethdev_core.h
> rte_ethdev_driver.h -> ../../lib/librte_ethdev/rte_ethdev_driver.h
> rte_ethdev.h -> ../../lib/librte_ethdev/rte_ethdev.h
> rte_ethdev_pci.h -> ../../lib/librte_ethdev/rte_ethdev_pci.h
> rte_ethdev_vdev.h -> ../../lib/librte_ethdev/rte_ethdev_vdev.h
> 
> I was able to reproduce the failure after 7 successful builds.

Thanks for details.

This time it seems build complaining about not able to find "rte_ethdev.h", so
your patch won't fix this case and build still will fail, looks like we need
different fix.

And in the build command for 'rte_ethdev.c', as you can see there is
'-I/tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include' paramter, and you
mentioned above "rte_ethdev.h -> ../../lib/librte_ethdev/rte_ethdev.h" link
exist under that filder, so still not clear why compiler can't find the files.

There is a dependency of build for the '_preinstall' file, so buildling .c files
in that library shouldn't start before all header files are installed and
'_preinstall' file is created. This is also not clear how '_preinstall' can be
missing.

Can you please double check above links are valid, I mean you can access to the
files from links?

> 
>>
>>>
>>> I'll try Meson build a few times. How can I enable those two config options in
>> Meson?
>>>
>>>> Thanks,
>>>> ferruh
>>>>
>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> Fix this by replacing the directive with #include "file" which
>>>>>>>> searches for files first in the directory containing the current
>>>>>>>> file and other directories only after that.
>>>>>>>>
>>>>>>>> Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
>>>>>>>
>>>>>>> What about #include <rte_ethdev_trace.h> ?
>>>>>>>
>>>>>>> Except this,
>>>>>>> Acked-by: Thomas Monjalon <thomas@monjalon.net>
>>>>>>>
>>>>>>>
>>>>>>> I would change the title to be more generic (nothing specific to
>>>>>>> gcc or Arm
>>>>>> here):
>>>>>>> 	ethdev: fix local include
>>>>>>>
>>>>>>> It needs to be backported, so these lines are required:
>>>>>>> 	Fixes: 7f0a669e7b04 ("ethdev: add allocation helper for virtual drivers")
>>>>>>> 	Fixes: dcd5c8112bc3 ("ethdev: add PCI driver helpers")
>>>>>>> 	Fixes: ffc905f3b856 ("ethdev: separate driver APIs")
>>>>>>> 	Fixes: 331c447ad913 ("ethdev: separate internal structures into
>>>>>>> own
>>>>>> header")
>>>>>>> 	Cc: stable@dpdk.org
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>
>>>
> 

Re: [dpdk-dev] [PATCH] ethdev: fix dpdk gcc build on Arm

[Juraj Linkeš]

> -----Original Message-----
> From: Ferruh Yigit <ferruh.yigit@intel.com>
> Sent: Monday, June 8, 2020 1:39 PM
> To: Juraj Linkeš <juraj.linkes@pantheon.tech>; Thomas Monjalon
> <thomas@monjalon.net>
> Cc: arybchenko@solarflare.com; dev@dpdk.org; Honnappa Nagarahalli
> <Honnappa.Nagarahalli@arm.com>
> Subject: Re: [dpdk-dev] [PATCH] ethdev: fix dpdk gcc build on Arm
> 
> On 6/8/2020 9:17 AM, Juraj Linkeš wrote:
> >
> >
> >> -----Original Message-----
> >> From: Ferruh Yigit <ferruh.yigit@intel.com>
> >> Sent: Thursday, June 4, 2020 3:30 PM
> >> To: Juraj Linkeš <juraj.linkes@pantheon.tech>; Thomas Monjalon
> >> <thomas@monjalon.net>
> >> Cc: arybchenko@solarflare.com; dev@dpdk.org; Honnappa Nagarahalli
> >> <Honnappa.Nagarahalli@arm.com>
> >> Subject: Re: [PATCH] ethdev: fix dpdk gcc build on Arm
> >>
> >> On 6/4/2020 1:55 PM, Juraj Linkeš wrote:
> >>>> -----Original Message-----
> >>>> From: Ferruh Yigit <ferruh.yigit@intel.com>
> >>>> Sent: Thursday, June 4, 2020 2:23 PM
> >>>> To: Juraj Linkeš <juraj.linkes@pantheon.tech>; Thomas Monjalon
> >>>> <thomas@monjalon.net>
> >>>> Cc: arybchenko@solarflare.com; dev@dpdk.org; Honnappa Nagarahalli
> >>>> <Honnappa.Nagarahalli@arm.com>
> >>>> Subject: Re: [PATCH] ethdev: fix dpdk gcc build on Arm
> >>>>
> >>>> On 6/4/2020 11:36 AM, Juraj Linkeš wrote:
> >>>>>
> >>>>>
> >>>>>> -----Original Message-----
> >>>>>> From: Ferruh Yigit <ferruh.yigit@intel.com>
> >>>>>> Sent: Wednesday, June 3, 2020 1:41 PM
> >>>>>> To: Thomas Monjalon <thomas@monjalon.net>; Juraj Linkeš
> >>>>>> <juraj.linkes@pantheon.tech>
> >>>>>> Cc: arybchenko@solarflare.com; dev@dpdk.org
> >>>>>> Subject: Re: [PATCH] ethdev: fix dpdk gcc build on Arm
> >>>>>>
> >>>>>> On 6/3/2020 11:16 AM, Thomas Monjalon wrote:
> >>>>>>> 03/06/2020 11:48, Juraj Linkeš:
> >>>>>>>> Directive #include <file> in gcc implementation searches for
> >>>>>>>> files in a standard list of system directories, which leads to
> >>>>>>>> a sporadici build error on Taishan arm machines:
> >>>>>>>> /tmp/openvpp-testing/dpdk/lib/librte_ethdev/rte_ethdev.h:4287:10:
> >>>>>>>> fatal error: rte_ethdev_core.h:
> >>>>>>>> No such file or directory #include <rte_ethdev_core.h>
> >>>>>>>
> >>>>>>> Would be interesting to know why nobody else hit such error?
> >>>>>>
> >>>>>>
> >>>>>> I can't see why this is happening, in the 'mk/rte.lib.mk' we have
> following:
> >>>>>>
> >>>>>> "
> >>>>>> install: _preinstall build _postinstall
> >>>>>> build: _preinstall
> >>>>>> "
> >>>>>>
> >>>>>> Which should cause the library header files installed before
> >>>>>> building .c files in that library.
> >>>>>> So when compiling 'rte_class_eth.c', the header files should be
> >>>>>> already in install folder.
> >>>>>>
> >>>>>>
> >>>>>> I can see how/why changing to "" fixes the issue but I am not
> >>>>>> sure about this
> >>>> fix.
> >>>>>> "rte_ethdev.h" is a public header file, that applications will
> >>>>>> include it in their applications. In the public library it is
> >>>>>> more proper to have other includes from system folder, using format <>.
> >>>>>> Again, I can't see why it is failing but I believe we should find
> >>>>>> another solution for _internal_ build error.
> >>>>>>
> >>>>>>
> >>>>>> A very simple solution can be following, but that is also not
> >>>>>> good, since it solves the issue by creating a dependency to the
> >>>>>> order of the header
> >>>> includes:
> >>>>>>  diff --git a/lib/librte_ethdev/rte_class_eth.c
> >>>>>> b/lib/librte_ethdev/rte_class_eth.c
> >>>>>>  index 6338355e25..3030c49020 100644
> >>>>>>  --- a/lib/librte_ethdev/rte_class_eth.c
> >>>>>>  +++ b/lib/librte_ethdev/rte_class_eth.c
> >>>>>>  @@ -10,8 +10,8 @@
> >>>>>>   #include <rte_kvargs.h>
> >>>>>>   #include <rte_log.h>
> >>>>>>
> >>>>>>  -#include "rte_ethdev.h"
> >>>>>>   #include "rte_ethdev_core.h"
> >>>>>>  +#include "rte_ethdev.h"
> >>>>>>   #include "rte_ethdev_driver.h"
> >>>>>>   #include "ethdev_private.h"
> >>>>>>
> >>>>>
> >>>>> Thomas, Ferruh, what should be the solution? I'm not an expert on
> >>>>> this and I
> >>>> can't really offer anything better, but I'd like that this gets fixed.
> >>>>
> >>>> First we need to root cause this before trying to solve it.
> >>>> Honnappa also said he can reproduce this but our CI builds can't
> >>>> (we are talking about tens of builds daily on various platforms),
> >>>> need to understand
> >> why.
> >>>> Also from Makefile I can't see how this is happening, I am feeling
> >>>> uneasy to fix something before figuring out how/why it is failing.
> >>>>
> >>>> Can you please try to collect more data on when/how this happens,
> >>>> initial questions I can think of:
> >>>> - Can you reproduce this with meson build?
> >>>> - Is it bare DPDK build, or build part of other project (I guess I
> >>>> saw fd.io on the
> >>>> link)
> >>>> - - If this is not bare DPDK build what changes has been done to build
> system?
> >>>> - Do you see this with fresh build (new clone) or rebuild of existing clone?
> >>>> - Can you confirm you have correct RTE_SDK and RTE_TARGET
> >>>> environment variables?
> >>>> - Can you please share your build command?
> >>>>
> >>>
> >>> I sent an e-mail to dpdk dev a few days back asking for help where I
> >>> outlined
> >> what we're doing:
> >>> We're not doing anything special, just downloading and extracting
> >>> the archive,
> >> then setting CONFIG_RTE_LIBRTE_MLX5_PMD and
> >> CONFIG_RTE_LIBRTE_I40E_16BYTE_RX_DESC to y in config/common_base
> and
> >> then running make install T=arm64-armv8a-linuxapp-gcc -j. As
> >> mentioned in the subject, the build server is a Taishan ARM server.
> >>>
> >>> We're doing a fresh rebuild everytime. The error doesn't happen
> >>> everytime,
> >> just sometimes - it seems to be random.
> >>>
> >>> We don't set RTE_SDK nor RTE_TARGET since
> >> https://doc.dpdk.org/guides/linux_gsg/build_dpdk.html#installation-of
> >> -dpdk- target-environment-using-make doesn't mention those.
> >>
> >> Can you please try with "V=1",
> >> "make install T=arm64-armv8a-linuxapp-gcc -j V=1", this will generate
> >> verbose output.
> >>
> >> And _before_ lines compiling the 'rte_class_eth.c' you should see
> >> something like following lines, can you please check:
> >> "
> >> SYMLINK-FILE include/rte_ethdev_core.h
> >>
> >> ln -nsf `.../buildtools/relpath.sh
> >> .../lib/librte_ethdev/rte_ethdev_core.h
> >> .../x86_64-native-linux-gcc/include`
> >> .../x86_64-native-linux-gcc/include
> >> "
> >>
> >
> > This is the log with V=1 build (a failed build):
> > == Build lib/librte_ethdev
> > == Build lib/librte_cmdline
> > ar crDs librte_lpm.a rte_lpm.o rte_lpm6.o
> >   INSTALL-LIB librte_lpm.a
> > ar crDs librte_member.a rte_member.o rte_member_ht.o rte_member_vbf.o
> >   INSTALL-LIB librte_member.a
> > ar crDs librte_cryptodev.a rte_cryptodev.o rte_cryptodev_pmd.o
> >   INSTALL-LIB librte_cryptodev.a
> > ar crDs librte_bbdev.a rte_bbdev.o
> > ar crDs librte_compressdev.a rte_compressdev.o rte_compressdev_pmd.o
> rte_comp.o
> >   INSTALL-LIB librte_compressdev.a
> >   INSTALL-LIB librte_bbdev.a
> >   SYMLINK-FILE include/rte_ethdev_pci.h
> >   SYMLINK-FILE include/rte_tm.h
> >   SYMLINK-FILE include/rte_tm_driver.h
> >   SYMLINK-FILE include/rte_eth_ctrl.h
> >   SYMLINK-FILE include/rte_flow_driver.h
> >   SYMLINK-FILE include/rte_ethdev_vdev.h
> >   SYMLINK-FILE include/rte_mtr_driver.h
> >   SYMLINK-FILE include/rte_ethdev_core.h
> >   SYMLINK-FILE include/rte_flow.h
> >   SYMLINK-FILE include/rte_mtr.h
> >   SYMLINK-FILE include/rte_ethdev_driver.h
> >   SYMLINK-FILE include/rte_ethdev.h
> >   SYMLINK-FILE include/rte_dev_info.h
> > gcc -Wp,-MD,./.ethdev_private.o.d.tmp   -pthread -I/tmp/dpdk-
> 20.02/lib/librte_eal/linux/eal/include  -march=armv8-a+crc -
> DRTE_MACHINE_CPUFLAG_NEON -DRTE_MACHINE_CPUFLAG_CRC32  -
> I/tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include -
> DRTE_USE_FUNCTION_VERSIONING -include /tmp/dpdk-20.02/arm64-armv8a-
> linuxapp-gcc/include/rte_config.h -D_GNU_SOURCE -
> DALLOW_EXPERIMENTAL_API -O3 -W -Wall -Wstrict-prototypes -Wmissing-
> prototypes -Wmissing-declarations -Wold-style-definition -Wpointer-arith -
> Wcast-align -Wnested-externs -Wcast-qual -Wformat-nonliteral -Wformat-
> security -Wundef -Wwrite-strings -Wdeprecated -Wno-missing-field-initializers -
> Wimplicit-fallthrough=2 -Wno-format-truncation -Wno-address-of-packed-
> member    -o ethdev_private.o -c /tmp/dpdk-
> 20.02/lib/librte_ethdev/ethdev_private.c
> > gcc -Wp,-MD,./.rte_ethdev.o.d.tmp   -pthread -I/tmp/dpdk-
> 20.02/lib/librte_eal/linux/eal/include  -march=armv8-a+crc -
> DRTE_MACHINE_CPUFLAG_NEON -DRTE_MACHINE_CPUFLAG_CRC32  -
> I/tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include -
> DRTE_USE_FUNCTION_VERSIONING -include /tmp/dpdk-20.02/arm64-armv8a-
> linuxapp-gcc/include/rte_config.h -D_GNU_SOURCE -
> DALLOW_EXPERIMENTAL_API -O3 -W -Wall -Wstrict-prototypes -Wmissing-
> prototypes -Wmissing-declarations -Wold-style-definition -Wpointer-arith -
> Wcast-align -Wnested-externs -Wcast-qual -Wformat-nonliteral -Wformat-
> security -Wundef -Wwrite-strings -Wdeprecated -Wno-missing-field-initializers -
> Wimplicit-fallthrough=2 -Wno-format-truncation -Wno-address-of-packed-
> member    -o rte_ethdev.o -c /tmp/dpdk-20.02/lib/librte_ethdev/rte_ethdev.c
> > gcc -Wp,-MD,./.rte_class_eth.o.d.tmp   -pthread -I/tmp/dpdk-
> 20.02/lib/librte_eal/linux/eal/include  -march=armv8-a+crc -
> DRTE_MACHINE_CPUFLAG_NEON -DRTE_MACHINE_CPUFLAG_CRC32  -
> I/tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include -
> DRTE_USE_FUNCTION_VERSIONING -include /tmp/dpdk-20.02/arm64-armv8a-
> linuxapp-gcc/include/rte_config.h -D_GNU_SOURCE -
> DALLOW_EXPERIMENTAL_API -O3 -W -Wall -Wstrict-prototypes -Wmissing-
> prototypes -Wmissing-declarations -Wold-style-definition -Wpointer-arith -
> Wcast-align -Wnested-externs -Wcast-qual -Wformat-nonliteral -Wformat-
> security -Wundef -Wwrite-strings -Wdeprecated -Wno-missing-field-initializers -
> Wimplicit-fallthrough=2 -Wno-format-truncation -Wno-address-of-packed-
> member    -o rte_class_eth.o -c /tmp/dpdk-
> 20.02/lib/librte_ethdev/rte_class_eth.c
> > gcc -Wp,-MD,./.rte_flow.o.d.tmp   -pthread -I/tmp/dpdk-
> 20.02/lib/librte_eal/linux/eal/include  -march=armv8-a+crc -
> DRTE_MACHINE_CPUFLAG_NEON -DRTE_MACHINE_CPUFLAG_CRC32  -
> I/tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include -
> DRTE_USE_FUNCTION_VERSIONING -include /tmp/dpdk-20.02/arm64-armv8a-
> linuxapp-gcc/include/rte_config.h -D_GNU_SOURCE -
> DALLOW_EXPERIMENTAL_API -O3 -W -Wall -Wstrict-prototypes -Wmissing-
> prototypes -Wmissing-declarations -Wold-style-definition -Wpointer-arith -
> Wcast-align -Wnested-externs -Wcast-qual -Wformat-nonliteral -Wformat-
> security -Wundef -Wwrite-strings -Wdeprecated -Wno-missing-field-initializers -
> Wimplicit-fallthrough=2 -Wno-format-truncation -Wno-address-of-packed-
> member    -o rte_flow.o -c /tmp/dpdk-20.02/lib/librte_ethdev/rte_flow.c
> > gcc -Wp,-MD,./.rte_tm.o.d.tmp   -pthread -I/tmp/dpdk-
> 20.02/lib/librte_eal/linux/eal/include  -march=armv8-a+crc -
> DRTE_MACHINE_CPUFLAG_NEON -DRTE_MACHINE_CPUFLAG_CRC32  -
> I/tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include -
> DRTE_USE_FUNCTION_VERSIONING -include /tmp/dpdk-20.02/arm64-armv8a-
> linuxapp-gcc/include/rte_config.h -D_GNU_SOURCE -
> DALLOW_EXPERIMENTAL_API -O3 -W -Wall -Wstrict-prototypes -Wmissing-
> prototypes -Wmissing-declarations -Wold-style-definition -Wpointer-arith -
> Wcast-align -Wnested-externs -Wcast-qual -Wformat-nonliteral -Wformat-
> security -Wundef -Wwrite-strings -Wdeprecated -Wno-missing-field-initializers -
> Wimplicit-fallthrough=2 -Wno-format-truncation -Wno-address-of-packed-
> member    -o rte_tm.o -c /tmp/dpdk-20.02/lib/librte_ethdev/rte_tm.c
> >   SYMLINK-FILE include/cmdline.h
> >   SYMLINK-FILE include/cmdline_parse.h
> >   SYMLINK-FILE include/cmdline_parse_num.h
> > gcc -Wp,-MD,./.ethdev_profile.o.d.tmp   -pthread -I/tmp/dpdk-
> 20.02/lib/librte_eal/linux/eal/include  -march=armv8-a+crc -
> DRTE_MACHINE_CPUFLAG_NEON -DRTE_MACHINE_CPUFLAG_CRC32  -
> I/tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include -
> DRTE_USE_FUNCTION_VERSIONING -include /tmp/dpdk-20.02/arm64-armv8a-
> linuxapp-gcc/include/rte_config.h -D_GNU_SOURCE -
> DALLOW_EXPERIMENTAL_API -O3 -W -Wall -Wstrict-prototypes -Wmissing-
> prototypes -Wmissing-declarations -Wold-style-definition -Wpointer-arith -
> Wcast-align -Wnested-externs -Wcast-qual -Wformat-nonliteral -Wformat-
> security -Wundef -Wwrite-strings -Wdeprecated -Wno-missing-field-initializers -
> Wimplicit-fallthrough=2 -Wno-format-truncation -Wno-address-of-packed-
> member    -o ethdev_profile.o -c /tmp/dpdk-
> 20.02/lib/librte_ethdev/ethdev_profile.c
> > gcc -Wp,-MD,./.rte_mtr.o.d.tmp   -pthread -I/tmp/dpdk-
> 20.02/lib/librte_eal/linux/eal/include  -march=armv8-a+crc -
> DRTE_MACHINE_CPUFLAG_NEON -DRTE_MACHINE_CPUFLAG_CRC32  -
> I/tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include -
> DRTE_USE_FUNCTION_VERSIONING -include /tmp/dpdk-20.02/arm64-armv8a-
> linuxapp-gcc/include/rte_config.h -D_GNU_SOURCE -
> DALLOW_EXPERIMENTAL_API -O3 -W -Wall -Wstrict-prototypes -Wmissing-
> prototypes -Wmissing-declarations -Wold-style-definition -Wpointer-arith -
> Wcast-align -Wnested-externs -Wcast-qual -Wformat-nonliteral -Wformat-
> security -Wundef -Wwrite-strings -Wdeprecated -Wno-missing-field-initializers -
> Wimplicit-fallthrough=2 -Wno-format-truncation -Wno-address-of-packed-
> member    -o rte_mtr.o -c /tmp/dpdk-20.02/lib/librte_ethdev/rte_mtr.c
> >   SYMLINK-FILE include/rte_sched.h
> >   SYMLINK-FILE include/rte_red.h
> >   SYMLINK-FILE include/rte_approx.h
> > In file included from /tmp/dpdk-20.02/lib/librte_ethdev/rte_ethdev.c:43:
> > /tmp/dpdk-20.02/lib/librte_ethdev/rte_ethdev_driver.h:18:10: fatal
> > error: rte_ethdev.h: No such file or directory  #include <rte_ethdev.h>
> >           ^~~~~~~~~~~~~~
> > compilation terminated.
> > /tmp/dpdk-20.02/mk/internal/rte.compile-pre.mk:114: recipe for target
> > 'rte_ethdev.o' failed
> >
> >> Also for the failed case, after fail can you please check if
> >> a)
> >> '.../arm64-armv8a-linuxapp-gcc/build/lib/librte_ethdev/_preinstall'
> >> file exists
> >> b) '.../x86_64-native-linux-gcc/include/rte_ethdev_core.h' exists and
> >> it linked to correct file '../../lib/librte_ethdev/rte_ethdev_core.h'
> >>
> >
> > a) it doesn't exist
> > b) all of these exist:
> > rte_ethdev_core.h -> ../../lib/librte_ethdev/rte_ethdev_core.h
> > rte_ethdev_driver.h -> ../../lib/librte_ethdev/rte_ethdev_driver.h
> > rte_ethdev.h -> ../../lib/librte_ethdev/rte_ethdev.h
> > rte_ethdev_pci.h -> ../../lib/librte_ethdev/rte_ethdev_pci.h
> > rte_ethdev_vdev.h -> ../../lib/librte_ethdev/rte_ethdev_vdev.h
> >
> > I was able to reproduce the failure after 7 successful builds.
> 
> Thanks for details.
> 
> This time it seems build complaining about not able to find "rte_ethdev.h", so
> your patch won't fix this case and build still will fail, looks like we need different
> fix.
> 

Right, Thomas already pointed this out - I haven't submitted a new patch version since it didn't make much sense before we hash this all out.

> And in the build command for 'rte_ethdev.c', as you can see there is '-
> I/tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include' paramter, and you
> mentioned above "rte_ethdev.h -> ../../lib/librte_ethdev/rte_ethdev.h" link exist
> under that filder, so still not clear why compiler can't find the files.
> 
> There is a dependency of build for the '_preinstall' file, so buildling .c files in that
> library shouldn't start before all header files are installed and '_preinstall' file is
> created. This is also not clear how '_preinstall' can be missing.
> 
> Can you please double check above links are valid, I mean you can access to the
> files from links?
> 

The links are pointing to existing files:
testuser@s17-t33-sut1:/tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include$ ll rte_ethdev.h
lrwxrwxrwx 1 testuser testuser 36 Jun  5 11:01 rte_ethdev.h -> ../../lib/librte_ethdev/rte_ethdev.h
testuser@s17-t33-sut1:/tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include$ ll ../../lib/librte_ethdev/rte_ethdev.h
-rw-rw-r-- 1 testuser testuser 171636 Feb 25 15:46 ../../lib/librte_ethdev/rte_ethdev.h

This is a complete mystery to me. Maybe there are some implementation details in the arm version of gcc or make?

> >
> >>
> >>>
> >>> I'll try Meson build a few times. How can I enable those two config
> >>> options in
> >> Meson?
> >>>
> >>>> Thanks,
> >>>> ferruh
> >>>>
> >>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>> Fix this by replacing the directive with #include "file" which
> >>>>>>>> searches for files first in the directory containing the
> >>>>>>>> current file and other directories only after that.
> >>>>>>>>
> >>>>>>>> Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
> >>>>>>>
> >>>>>>> What about #include <rte_ethdev_trace.h> ?
> >>>>>>>
> >>>>>>> Except this,
> >>>>>>> Acked-by: Thomas Monjalon <thomas@monjalon.net>
> >>>>>>>
> >>>>>>>
> >>>>>>> I would change the title to be more generic (nothing specific to
> >>>>>>> gcc or Arm
> >>>>>> here):
> >>>>>>> 	ethdev: fix local include
> >>>>>>>
> >>>>>>> It needs to be backported, so these lines are required:
> >>>>>>> 	Fixes: 7f0a669e7b04 ("ethdev: add allocation helper for virtual
> drivers")
> >>>>>>> 	Fixes: dcd5c8112bc3 ("ethdev: add PCI driver helpers")
> >>>>>>> 	Fixes: ffc905f3b856 ("ethdev: separate driver APIs")
> >>>>>>> 	Fixes: 331c447ad913 ("ethdev: separate internal structures into
> >>>>>>> own
> >>>>>> header")
> >>>>>>> 	Cc: stable@dpdk.org
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>
> >>>
> >
> 

Re: [dpdk-dev] [PATCH] ethdev: fix dpdk gcc build on Arm

[Ferruh Yigit]

On 6/8/2020 1:03 PM, Juraj Linkeš wrote:
> 
> 
>> -----Original Message-----
>> From: Ferruh Yigit <ferruh.yigit@intel.com>
>> Sent: Monday, June 8, 2020 1:39 PM
>> To: Juraj Linkeš <juraj.linkes@pantheon.tech>; Thomas Monjalon
>> <thomas@monjalon.net>
>> Cc: arybchenko@solarflare.com; dev@dpdk.org; Honnappa Nagarahalli
>> <Honnappa.Nagarahalli@arm.com>
>> Subject: Re: [dpdk-dev] [PATCH] ethdev: fix dpdk gcc build on Arm
>>
>> On 6/8/2020 9:17 AM, Juraj Linkeš wrote:
>>>
>>>
>>>> -----Original Message-----
>>>> From: Ferruh Yigit <ferruh.yigit@intel.com>
>>>> Sent: Thursday, June 4, 2020 3:30 PM
>>>> To: Juraj Linkeš <juraj.linkes@pantheon.tech>; Thomas Monjalon
>>>> <thomas@monjalon.net>
>>>> Cc: arybchenko@solarflare.com; dev@dpdk.org; Honnappa Nagarahalli
>>>> <Honnappa.Nagarahalli@arm.com>
>>>> Subject: Re: [PATCH] ethdev: fix dpdk gcc build on Arm
>>>>
>>>> On 6/4/2020 1:55 PM, Juraj Linkeš wrote:
>>>>>> -----Original Message-----
>>>>>> From: Ferruh Yigit <ferruh.yigit@intel.com>
>>>>>> Sent: Thursday, June 4, 2020 2:23 PM
>>>>>> To: Juraj Linkeš <juraj.linkes@pantheon.tech>; Thomas Monjalon
>>>>>> <thomas@monjalon.net>
>>>>>> Cc: arybchenko@solarflare.com; dev@dpdk.org; Honnappa Nagarahalli
>>>>>> <Honnappa.Nagarahalli@arm.com>
>>>>>> Subject: Re: [PATCH] ethdev: fix dpdk gcc build on Arm
>>>>>>
>>>>>> On 6/4/2020 11:36 AM, Juraj Linkeš wrote:
>>>>>>>
>>>>>>>
>>>>>>>> -----Original Message-----
>>>>>>>> From: Ferruh Yigit <ferruh.yigit@intel.com>
>>>>>>>> Sent: Wednesday, June 3, 2020 1:41 PM
>>>>>>>> To: Thomas Monjalon <thomas@monjalon.net>; Juraj Linkeš
>>>>>>>> <juraj.linkes@pantheon.tech>
>>>>>>>> Cc: arybchenko@solarflare.com; dev@dpdk.org
>>>>>>>> Subject: Re: [PATCH] ethdev: fix dpdk gcc build on Arm
>>>>>>>>
>>>>>>>> On 6/3/2020 11:16 AM, Thomas Monjalon wrote:
>>>>>>>>> 03/06/2020 11:48, Juraj Linkeš:
>>>>>>>>>> Directive #include <file> in gcc implementation searches for
>>>>>>>>>> files in a standard list of system directories, which leads to
>>>>>>>>>> a sporadici build error on Taishan arm machines:
>>>>>>>>>> /tmp/openvpp-testing/dpdk/lib/librte_ethdev/rte_ethdev.h:4287:10:
>>>>>>>>>> fatal error: rte_ethdev_core.h:
>>>>>>>>>> No such file or directory #include <rte_ethdev_core.h>
>>>>>>>>>
>>>>>>>>> Would be interesting to know why nobody else hit such error?
>>>>>>>>
>>>>>>>>
>>>>>>>> I can't see why this is happening, in the 'mk/rte.lib.mk' we have
>> following:
>>>>>>>>
>>>>>>>> "
>>>>>>>> install: _preinstall build _postinstall
>>>>>>>> build: _preinstall
>>>>>>>> "
>>>>>>>>
>>>>>>>> Which should cause the library header files installed before
>>>>>>>> building .c files in that library.
>>>>>>>> So when compiling 'rte_class_eth.c', the header files should be
>>>>>>>> already in install folder.
>>>>>>>>
>>>>>>>>
>>>>>>>> I can see how/why changing to "" fixes the issue but I am not
>>>>>>>> sure about this
>>>>>> fix.
>>>>>>>> "rte_ethdev.h" is a public header file, that applications will
>>>>>>>> include it in their applications. In the public library it is
>>>>>>>> more proper to have other includes from system folder, using format <>.
>>>>>>>> Again, I can't see why it is failing but I believe we should find
>>>>>>>> another solution for _internal_ build error.
>>>>>>>>
>>>>>>>>
>>>>>>>> A very simple solution can be following, but that is also not
>>>>>>>> good, since it solves the issue by creating a dependency to the
>>>>>>>> order of the header
>>>>>> includes:
>>>>>>>>  diff --git a/lib/librte_ethdev/rte_class_eth.c
>>>>>>>> b/lib/librte_ethdev/rte_class_eth.c
>>>>>>>>  index 6338355e25..3030c49020 100644
>>>>>>>>  --- a/lib/librte_ethdev/rte_class_eth.c
>>>>>>>>  +++ b/lib/librte_ethdev/rte_class_eth.c
>>>>>>>>  @@ -10,8 +10,8 @@
>>>>>>>>   #include <rte_kvargs.h>
>>>>>>>>   #include <rte_log.h>
>>>>>>>>
>>>>>>>>  -#include "rte_ethdev.h"
>>>>>>>>   #include "rte_ethdev_core.h"
>>>>>>>>  +#include "rte_ethdev.h"
>>>>>>>>   #include "rte_ethdev_driver.h"
>>>>>>>>   #include "ethdev_private.h"
>>>>>>>>
>>>>>>>
>>>>>>> Thomas, Ferruh, what should be the solution? I'm not an expert on
>>>>>>> this and I
>>>>>> can't really offer anything better, but I'd like that this gets fixed.
>>>>>>
>>>>>> First we need to root cause this before trying to solve it.
>>>>>> Honnappa also said he can reproduce this but our CI builds can't
>>>>>> (we are talking about tens of builds daily on various platforms),
>>>>>> need to understand
>>>> why.
>>>>>> Also from Makefile I can't see how this is happening, I am feeling
>>>>>> uneasy to fix something before figuring out how/why it is failing.
>>>>>>
>>>>>> Can you please try to collect more data on when/how this happens,
>>>>>> initial questions I can think of:
>>>>>> - Can you reproduce this with meson build?
>>>>>> - Is it bare DPDK build, or build part of other project (I guess I
>>>>>> saw fd.io on the
>>>>>> link)
>>>>>> - - If this is not bare DPDK build what changes has been done to build
>> system?
>>>>>> - Do you see this with fresh build (new clone) or rebuild of existing clone?
>>>>>> - Can you confirm you have correct RTE_SDK and RTE_TARGET
>>>>>> environment variables?
>>>>>> - Can you please share your build command?
>>>>>>
>>>>>
>>>>> I sent an e-mail to dpdk dev a few days back asking for help where I
>>>>> outlined
>>>> what we're doing:
>>>>> We're not doing anything special, just downloading and extracting
>>>>> the archive,
>>>> then setting CONFIG_RTE_LIBRTE_MLX5_PMD and
>>>> CONFIG_RTE_LIBRTE_I40E_16BYTE_RX_DESC to y in config/common_base
>> and
>>>> then running make install T=arm64-armv8a-linuxapp-gcc -j. As
>>>> mentioned in the subject, the build server is a Taishan ARM server.
>>>>>
>>>>> We're doing a fresh rebuild everytime. The error doesn't happen
>>>>> everytime,
>>>> just sometimes - it seems to be random.
>>>>>
>>>>> We don't set RTE_SDK nor RTE_TARGET since
>>>> https://doc.dpdk.org/guides/linux_gsg/build_dpdk.html#installation-of
>>>> -dpdk- target-environment-using-make doesn't mention those.
>>>>
>>>> Can you please try with "V=1",
>>>> "make install T=arm64-armv8a-linuxapp-gcc -j V=1", this will generate
>>>> verbose output.
>>>>
>>>> And _before_ lines compiling the 'rte_class_eth.c' you should see
>>>> something like following lines, can you please check:
>>>> "
>>>> SYMLINK-FILE include/rte_ethdev_core.h
>>>>
>>>> ln -nsf `.../buildtools/relpath.sh
>>>> .../lib/librte_ethdev/rte_ethdev_core.h
>>>> .../x86_64-native-linux-gcc/include`
>>>> .../x86_64-native-linux-gcc/include
>>>> "
>>>>
>>>
>>> This is the log with V=1 build (a failed build):
>>> == Build lib/librte_ethdev
>>> == Build lib/librte_cmdline
>>> ar crDs librte_lpm.a rte_lpm.o rte_lpm6.o
>>>   INSTALL-LIB librte_lpm.a
>>> ar crDs librte_member.a rte_member.o rte_member_ht.o rte_member_vbf.o
>>>   INSTALL-LIB librte_member.a
>>> ar crDs librte_cryptodev.a rte_cryptodev.o rte_cryptodev_pmd.o
>>>   INSTALL-LIB librte_cryptodev.a
>>> ar crDs librte_bbdev.a rte_bbdev.o
>>> ar crDs librte_compressdev.a rte_compressdev.o rte_compressdev_pmd.o
>> rte_comp.o
>>>   INSTALL-LIB librte_compressdev.a
>>>   INSTALL-LIB librte_bbdev.a
>>>   SYMLINK-FILE include/rte_ethdev_pci.h
>>>   SYMLINK-FILE include/rte_tm.h
>>>   SYMLINK-FILE include/rte_tm_driver.h
>>>   SYMLINK-FILE include/rte_eth_ctrl.h
>>>   SYMLINK-FILE include/rte_flow_driver.h
>>>   SYMLINK-FILE include/rte_ethdev_vdev.h
>>>   SYMLINK-FILE include/rte_mtr_driver.h
>>>   SYMLINK-FILE include/rte_ethdev_core.h
>>>   SYMLINK-FILE include/rte_flow.h
>>>   SYMLINK-FILE include/rte_mtr.h
>>>   SYMLINK-FILE include/rte_ethdev_driver.h
>>>   SYMLINK-FILE include/rte_ethdev.h
>>>   SYMLINK-FILE include/rte_dev_info.h
>>> gcc -Wp,-MD,./.ethdev_private.o.d.tmp   -pthread -I/tmp/dpdk-
>> 20.02/lib/librte_eal/linux/eal/include  -march=armv8-a+crc -
>> DRTE_MACHINE_CPUFLAG_NEON -DRTE_MACHINE_CPUFLAG_CRC32  -
>> I/tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include -
>> DRTE_USE_FUNCTION_VERSIONING -include /tmp/dpdk-20.02/arm64-armv8a-
>> linuxapp-gcc/include/rte_config.h -D_GNU_SOURCE -
>> DALLOW_EXPERIMENTAL_API -O3 -W -Wall -Wstrict-prototypes -Wmissing-
>> prototypes -Wmissing-declarations -Wold-style-definition -Wpointer-arith -
>> Wcast-align -Wnested-externs -Wcast-qual -Wformat-nonliteral -Wformat-
>> security -Wundef -Wwrite-strings -Wdeprecated -Wno-missing-field-initializers -
>> Wimplicit-fallthrough=2 -Wno-format-truncation -Wno-address-of-packed-
>> member    -o ethdev_private.o -c /tmp/dpdk-
>> 20.02/lib/librte_ethdev/ethdev_private.c
>>> gcc -Wp,-MD,./.rte_ethdev.o.d.tmp   -pthread -I/tmp/dpdk-
>> 20.02/lib/librte_eal/linux/eal/include  -march=armv8-a+crc -
>> DRTE_MACHINE_CPUFLAG_NEON -DRTE_MACHINE_CPUFLAG_CRC32  -
>> I/tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include -
>> DRTE_USE_FUNCTION_VERSIONING -include /tmp/dpdk-20.02/arm64-armv8a-
>> linuxapp-gcc/include/rte_config.h -D_GNU_SOURCE -
>> DALLOW_EXPERIMENTAL_API -O3 -W -Wall -Wstrict-prototypes -Wmissing-
>> prototypes -Wmissing-declarations -Wold-style-definition -Wpointer-arith -
>> Wcast-align -Wnested-externs -Wcast-qual -Wformat-nonliteral -Wformat-
>> security -Wundef -Wwrite-strings -Wdeprecated -Wno-missing-field-initializers -
>> Wimplicit-fallthrough=2 -Wno-format-truncation -Wno-address-of-packed-
>> member    -o rte_ethdev.o -c /tmp/dpdk-20.02/lib/librte_ethdev/rte_ethdev.c
>>> gcc -Wp,-MD,./.rte_class_eth.o.d.tmp   -pthread -I/tmp/dpdk-
>> 20.02/lib/librte_eal/linux/eal/include  -march=armv8-a+crc -
>> DRTE_MACHINE_CPUFLAG_NEON -DRTE_MACHINE_CPUFLAG_CRC32  -
>> I/tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include -
>> DRTE_USE_FUNCTION_VERSIONING -include /tmp/dpdk-20.02/arm64-armv8a-
>> linuxapp-gcc/include/rte_config.h -D_GNU_SOURCE -
>> DALLOW_EXPERIMENTAL_API -O3 -W -Wall -Wstrict-prototypes -Wmissing-
>> prototypes -Wmissing-declarations -Wold-style-definition -Wpointer-arith -
>> Wcast-align -Wnested-externs -Wcast-qual -Wformat-nonliteral -Wformat-
>> security -Wundef -Wwrite-strings -Wdeprecated -Wno-missing-field-initializers -
>> Wimplicit-fallthrough=2 -Wno-format-truncation -Wno-address-of-packed-
>> member    -o rte_class_eth.o -c /tmp/dpdk-
>> 20.02/lib/librte_ethdev/rte_class_eth.c
>>> gcc -Wp,-MD,./.rte_flow.o.d.tmp   -pthread -I/tmp/dpdk-
>> 20.02/lib/librte_eal/linux/eal/include  -march=armv8-a+crc -
>> DRTE_MACHINE_CPUFLAG_NEON -DRTE_MACHINE_CPUFLAG_CRC32  -
>> I/tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include -
>> DRTE_USE_FUNCTION_VERSIONING -include /tmp/dpdk-20.02/arm64-armv8a-
>> linuxapp-gcc/include/rte_config.h -D_GNU_SOURCE -
>> DALLOW_EXPERIMENTAL_API -O3 -W -Wall -Wstrict-prototypes -Wmissing-
>> prototypes -Wmissing-declarations -Wold-style-definition -Wpointer-arith -
>> Wcast-align -Wnested-externs -Wcast-qual -Wformat-nonliteral -Wformat-
>> security -Wundef -Wwrite-strings -Wdeprecated -Wno-missing-field-initializers -
>> Wimplicit-fallthrough=2 -Wno-format-truncation -Wno-address-of-packed-
>> member    -o rte_flow.o -c /tmp/dpdk-20.02/lib/librte_ethdev/rte_flow.c
>>> gcc -Wp,-MD,./.rte_tm.o.d.tmp   -pthread -I/tmp/dpdk-
>> 20.02/lib/librte_eal/linux/eal/include  -march=armv8-a+crc -
>> DRTE_MACHINE_CPUFLAG_NEON -DRTE_MACHINE_CPUFLAG_CRC32  -
>> I/tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include -
>> DRTE_USE_FUNCTION_VERSIONING -include /tmp/dpdk-20.02/arm64-armv8a-
>> linuxapp-gcc/include/rte_config.h -D_GNU_SOURCE -
>> DALLOW_EXPERIMENTAL_API -O3 -W -Wall -Wstrict-prototypes -Wmissing-
>> prototypes -Wmissing-declarations -Wold-style-definition -Wpointer-arith -
>> Wcast-align -Wnested-externs -Wcast-qual -Wformat-nonliteral -Wformat-
>> security -Wundef -Wwrite-strings -Wdeprecated -Wno-missing-field-initializers -
>> Wimplicit-fallthrough=2 -Wno-format-truncation -Wno-address-of-packed-
>> member    -o rte_tm.o -c /tmp/dpdk-20.02/lib/librte_ethdev/rte_tm.c
>>>   SYMLINK-FILE include/cmdline.h
>>>   SYMLINK-FILE include/cmdline_parse.h
>>>   SYMLINK-FILE include/cmdline_parse_num.h
>>> gcc -Wp,-MD,./.ethdev_profile.o.d.tmp   -pthread -I/tmp/dpdk-
>> 20.02/lib/librte_eal/linux/eal/include  -march=armv8-a+crc -
>> DRTE_MACHINE_CPUFLAG_NEON -DRTE_MACHINE_CPUFLAG_CRC32  -
>> I/tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include -
>> DRTE_USE_FUNCTION_VERSIONING -include /tmp/dpdk-20.02/arm64-armv8a-
>> linuxapp-gcc/include/rte_config.h -D_GNU_SOURCE -
>> DALLOW_EXPERIMENTAL_API -O3 -W -Wall -Wstrict-prototypes -Wmissing-
>> prototypes -Wmissing-declarations -Wold-style-definition -Wpointer-arith -
>> Wcast-align -Wnested-externs -Wcast-qual -Wformat-nonliteral -Wformat-
>> security -Wundef -Wwrite-strings -Wdeprecated -Wno-missing-field-initializers -
>> Wimplicit-fallthrough=2 -Wno-format-truncation -Wno-address-of-packed-
>> member    -o ethdev_profile.o -c /tmp/dpdk-
>> 20.02/lib/librte_ethdev/ethdev_profile.c
>>> gcc -Wp,-MD,./.rte_mtr.o.d.tmp   -pthread -I/tmp/dpdk-
>> 20.02/lib/librte_eal/linux/eal/include  -march=armv8-a+crc -
>> DRTE_MACHINE_CPUFLAG_NEON -DRTE_MACHINE_CPUFLAG_CRC32  -
>> I/tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include -
>> DRTE_USE_FUNCTION_VERSIONING -include /tmp/dpdk-20.02/arm64-armv8a-
>> linuxapp-gcc/include/rte_config.h -D_GNU_SOURCE -
>> DALLOW_EXPERIMENTAL_API -O3 -W -Wall -Wstrict-prototypes -Wmissing-
>> prototypes -Wmissing-declarations -Wold-style-definition -Wpointer-arith -
>> Wcast-align -Wnested-externs -Wcast-qual -Wformat-nonliteral -Wformat-
>> security -Wundef -Wwrite-strings -Wdeprecated -Wno-missing-field-initializers -
>> Wimplicit-fallthrough=2 -Wno-format-truncation -Wno-address-of-packed-
>> member    -o rte_mtr.o -c /tmp/dpdk-20.02/lib/librte_ethdev/rte_mtr.c
>>>   SYMLINK-FILE include/rte_sched.h
>>>   SYMLINK-FILE include/rte_red.h
>>>   SYMLINK-FILE include/rte_approx.h
>>> In file included from /tmp/dpdk-20.02/lib/librte_ethdev/rte_ethdev.c:43:
>>> /tmp/dpdk-20.02/lib/librte_ethdev/rte_ethdev_driver.h:18:10: fatal
>>> error: rte_ethdev.h: No such file or directory  #include <rte_ethdev.h>
>>>           ^~~~~~~~~~~~~~
>>> compilation terminated.
>>> /tmp/dpdk-20.02/mk/internal/rte.compile-pre.mk:114: recipe for target
>>> 'rte_ethdev.o' failed
>>>
>>>> Also for the failed case, after fail can you please check if
>>>> a)
>>>> '.../arm64-armv8a-linuxapp-gcc/build/lib/librte_ethdev/_preinstall'
>>>> file exists
>>>> b) '.../x86_64-native-linux-gcc/include/rte_ethdev_core.h' exists and
>>>> it linked to correct file '../../lib/librte_ethdev/rte_ethdev_core.h'
>>>>
>>>
>>> a) it doesn't exist
>>> b) all of these exist:
>>> rte_ethdev_core.h -> ../../lib/librte_ethdev/rte_ethdev_core.h
>>> rte_ethdev_driver.h -> ../../lib/librte_ethdev/rte_ethdev_driver.h
>>> rte_ethdev.h -> ../../lib/librte_ethdev/rte_ethdev.h
>>> rte_ethdev_pci.h -> ../../lib/librte_ethdev/rte_ethdev_pci.h
>>> rte_ethdev_vdev.h -> ../../lib/librte_ethdev/rte_ethdev_vdev.h
>>>
>>> I was able to reproduce the failure after 7 successful builds.
>>
>> Thanks for details.
>>
>> This time it seems build complaining about not able to find "rte_ethdev.h", so
>> your patch won't fix this case and build still will fail, looks like we need different
>> fix.
>>
> 
> Right, Thomas already pointed this out - I haven't submitted a new patch version since it didn't make much sense before we hash this all out.
> 
>> And in the build command for 'rte_ethdev.c', as you can see there is '-
>> I/tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include' paramter, and you
>> mentioned above "rte_ethdev.h -> ../../lib/librte_ethdev/rte_ethdev.h" link exist
>> under that filder, so still not clear why compiler can't find the files.
>>
>> There is a dependency of build for the '_preinstall' file, so buildling .c files in that
>> library shouldn't start before all header files are installed and '_preinstall' file is
>> created. This is also not clear how '_preinstall' can be missing.
>>
>> Can you please double check above links are valid, I mean you can access to the
>> files from links?
>>
> 
> The links are pointing to existing files:
> testuser@s17-t33-sut1:/tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include$ ll rte_ethdev.h
> lrwxrwxrwx 1 testuser testuser 36 Jun  5 11:01 rte_ethdev.h -> ../../lib/librte_ethdev/rte_ethdev.h
> testuser@s17-t33-sut1:/tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include$ ll ../../lib/librte_ethdev/rte_ethdev.h
> -rw-rw-r-- 1 testuser testuser 171636 Feb 25 15:46 ../../lib/librte_ethdev/rte_ethdev.h
> 
> This is a complete mystery to me. Maybe there are some implementation details in the arm version of gcc or make?

I don't see the lines as following, for symbolic link, did you strip them or not
gettim them at all:
ln -nsf `/tmp/build/dpdk-20.05/buildtools/relpath.sh .....

> 
>>>
>>>>
>>>>>
>>>>> I'll try Meson build a few times. How can I enable those two config
>>>>> options in
>>>> Meson?
>>>>>
>>>>>> Thanks,
>>>>>> ferruh
>>>>>>
>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> Fix this by replacing the directive with #include "file" which
>>>>>>>>>> searches for files first in the directory containing the
>>>>>>>>>> current file and other directories only after that.
>>>>>>>>>>
>>>>>>>>>> Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
>>>>>>>>>
>>>>>>>>> What about #include <rte_ethdev_trace.h> ?
>>>>>>>>>
>>>>>>>>> Except this,
>>>>>>>>> Acked-by: Thomas Monjalon <thomas@monjalon.net>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I would change the title to be more generic (nothing specific to
>>>>>>>>> gcc or Arm
>>>>>>>> here):
>>>>>>>>> 	ethdev: fix local include
>>>>>>>>>
>>>>>>>>> It needs to be backported, so these lines are required:
>>>>>>>>> 	Fixes: 7f0a669e7b04 ("ethdev: add allocation helper for virtual
>> drivers")
>>>>>>>>> 	Fixes: dcd5c8112bc3 ("ethdev: add PCI driver helpers")
>>>>>>>>> 	Fixes: ffc905f3b856 ("ethdev: separate driver APIs")
>>>>>>>>> 	Fixes: 331c447ad913 ("ethdev: separate internal structures into
>>>>>>>>> own
>>>>>>>> header")
>>>>>>>>> 	Cc: stable@dpdk.org
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>
>>>
>>
> 

Re: [dpdk-dev] [PATCH] ethdev: fix dpdk gcc build on Arm

[Juraj Linkeš]

> -----Original Message-----
> From: Ferruh Yigit <ferruh.yigit@intel.com>
> Sent: Monday, June 8, 2020 2:39 PM
> To: Juraj Linkeš <juraj.linkes@pantheon.tech>; Thomas Monjalon
> <thomas@monjalon.net>
> Cc: arybchenko@solarflare.com; dev@dpdk.org; Honnappa Nagarahalli
> <Honnappa.Nagarahalli@arm.com>
> Subject: Re: [dpdk-dev] [PATCH] ethdev: fix dpdk gcc build on Arm
> 
> On 6/8/2020 1:03 PM, Juraj Linkeš wrote:
> >
> >
> >> -----Original Message-----
> >> From: Ferruh Yigit <ferruh.yigit@intel.com>
> >> Sent: Monday, June 8, 2020 1:39 PM
> >> To: Juraj Linkeš <juraj.linkes@pantheon.tech>; Thomas Monjalon
> >> <thomas@monjalon.net>
> >> Cc: arybchenko@solarflare.com; dev@dpdk.org; Honnappa Nagarahalli
> >> <Honnappa.Nagarahalli@arm.com>
> >> Subject: Re: [dpdk-dev] [PATCH] ethdev: fix dpdk gcc build on Arm
> >>
> >> On 6/8/2020 9:17 AM, Juraj Linkeš wrote:
> >>>
> >>>
> >>>> -----Original Message-----
> >>>> From: Ferruh Yigit <ferruh.yigit@intel.com>
> >>>> Sent: Thursday, June 4, 2020 3:30 PM
> >>>> To: Juraj Linkeš <juraj.linkes@pantheon.tech>; Thomas Monjalon
> >>>> <thomas@monjalon.net>
> >>>> Cc: arybchenko@solarflare.com; dev@dpdk.org; Honnappa Nagarahalli
> >>>> <Honnappa.Nagarahalli@arm.com>
> >>>> Subject: Re: [PATCH] ethdev: fix dpdk gcc build on Arm
> >>>>
> >>>> On 6/4/2020 1:55 PM, Juraj Linkeš wrote:
> >>>>>> -----Original Message-----
> >>>>>> From: Ferruh Yigit <ferruh.yigit@intel.com>
> >>>>>> Sent: Thursday, June 4, 2020 2:23 PM
> >>>>>> To: Juraj Linkeš <juraj.linkes@pantheon.tech>; Thomas Monjalon
> >>>>>> <thomas@monjalon.net>
> >>>>>> Cc: arybchenko@solarflare.com; dev@dpdk.org; Honnappa Nagarahalli
> >>>>>> <Honnappa.Nagarahalli@arm.com>
> >>>>>> Subject: Re: [PATCH] ethdev: fix dpdk gcc build on Arm
> >>>>>>
> >>>>>> On 6/4/2020 11:36 AM, Juraj Linkeš wrote:
> >>>>>>>
> >>>>>>>
> >>>>>>>> -----Original Message-----
> >>>>>>>> From: Ferruh Yigit <ferruh.yigit@intel.com>
> >>>>>>>> Sent: Wednesday, June 3, 2020 1:41 PM
> >>>>>>>> To: Thomas Monjalon <thomas@monjalon.net>; Juraj Linkeš
> >>>>>>>> <juraj.linkes@pantheon.tech>
> >>>>>>>> Cc: arybchenko@solarflare.com; dev@dpdk.org
> >>>>>>>> Subject: Re: [PATCH] ethdev: fix dpdk gcc build on Arm
> >>>>>>>>
> >>>>>>>> On 6/3/2020 11:16 AM, Thomas Monjalon wrote:
> >>>>>>>>> 03/06/2020 11:48, Juraj Linkeš:
> >>>>>>>>>> Directive #include <file> in gcc implementation searches for
> >>>>>>>>>> files in a standard list of system directories, which leads
> >>>>>>>>>> to a sporadici build error on Taishan arm machines:
> >>>>>>>>>> /tmp/openvpp-
> testing/dpdk/lib/librte_ethdev/rte_ethdev.h:4287:10:
> >>>>>>>>>> fatal error: rte_ethdev_core.h:
> >>>>>>>>>> No such file or directory #include <rte_ethdev_core.h>
> >>>>>>>>>
> >>>>>>>>> Would be interesting to know why nobody else hit such error?
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> I can't see why this is happening, in the 'mk/rte.lib.mk' we
> >>>>>>>> have
> >> following:
> >>>>>>>>
> >>>>>>>> "
> >>>>>>>> install: _preinstall build _postinstall
> >>>>>>>> build: _preinstall
> >>>>>>>> "
> >>>>>>>>
> >>>>>>>> Which should cause the library header files installed before
> >>>>>>>> building .c files in that library.
> >>>>>>>> So when compiling 'rte_class_eth.c', the header files should be
> >>>>>>>> already in install folder.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> I can see how/why changing to "" fixes the issue but I am not
> >>>>>>>> sure about this
> >>>>>> fix.
> >>>>>>>> "rte_ethdev.h" is a public header file, that applications will
> >>>>>>>> include it in their applications. In the public library it is
> >>>>>>>> more proper to have other includes from system folder, using format
> <>.
> >>>>>>>> Again, I can't see why it is failing but I believe we should
> >>>>>>>> find another solution for _internal_ build error.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> A very simple solution can be following, but that is also not
> >>>>>>>> good, since it solves the issue by creating a dependency to the
> >>>>>>>> order of the header
> >>>>>> includes:
> >>>>>>>>  diff --git a/lib/librte_ethdev/rte_class_eth.c
> >>>>>>>> b/lib/librte_ethdev/rte_class_eth.c
> >>>>>>>>  index 6338355e25..3030c49020 100644
> >>>>>>>>  --- a/lib/librte_ethdev/rte_class_eth.c
> >>>>>>>>  +++ b/lib/librte_ethdev/rte_class_eth.c
> >>>>>>>>  @@ -10,8 +10,8 @@
> >>>>>>>>   #include <rte_kvargs.h>
> >>>>>>>>   #include <rte_log.h>
> >>>>>>>>
> >>>>>>>>  -#include "rte_ethdev.h"
> >>>>>>>>   #include "rte_ethdev_core.h"
> >>>>>>>>  +#include "rte_ethdev.h"
> >>>>>>>>   #include "rte_ethdev_driver.h"
> >>>>>>>>   #include "ethdev_private.h"
> >>>>>>>>
> >>>>>>>
> >>>>>>> Thomas, Ferruh, what should be the solution? I'm not an expert
> >>>>>>> on this and I
> >>>>>> can't really offer anything better, but I'd like that this gets fixed.
> >>>>>>
> >>>>>> First we need to root cause this before trying to solve it.
> >>>>>> Honnappa also said he can reproduce this but our CI builds can't
> >>>>>> (we are talking about tens of builds daily on various platforms),
> >>>>>> need to understand
> >>>> why.
> >>>>>> Also from Makefile I can't see how this is happening, I am
> >>>>>> feeling uneasy to fix something before figuring out how/why it is failing.
> >>>>>>
> >>>>>> Can you please try to collect more data on when/how this happens,
> >>>>>> initial questions I can think of:
> >>>>>> - Can you reproduce this with meson build?
> >>>>>> - Is it bare DPDK build, or build part of other project (I guess
> >>>>>> I saw fd.io on the
> >>>>>> link)
> >>>>>> - - If this is not bare DPDK build what changes has been done to
> >>>>>> build
> >> system?
> >>>>>> - Do you see this with fresh build (new clone) or rebuild of existing
> clone?
> >>>>>> - Can you confirm you have correct RTE_SDK and RTE_TARGET
> >>>>>> environment variables?
> >>>>>> - Can you please share your build command?
> >>>>>>
> >>>>>
> >>>>> I sent an e-mail to dpdk dev a few days back asking for help where
> >>>>> I outlined
> >>>> what we're doing:
> >>>>> We're not doing anything special, just downloading and extracting
> >>>>> the archive,
> >>>> then setting CONFIG_RTE_LIBRTE_MLX5_PMD and
> >>>> CONFIG_RTE_LIBRTE_I40E_16BYTE_RX_DESC to y in
> config/common_base
> >> and
> >>>> then running make install T=arm64-armv8a-linuxapp-gcc -j. As
> >>>> mentioned in the subject, the build server is a Taishan ARM server.
> >>>>>
> >>>>> We're doing a fresh rebuild everytime. The error doesn't happen
> >>>>> everytime,
> >>>> just sometimes - it seems to be random.
> >>>>>
> >>>>> We don't set RTE_SDK nor RTE_TARGET since
> >>>> https://doc.dpdk.org/guides/linux_gsg/build_dpdk.html#installation-
> >>>> of
> >>>> -dpdk- target-environment-using-make doesn't mention those.
> >>>>
> >>>> Can you please try with "V=1",
> >>>> "make install T=arm64-armv8a-linuxapp-gcc -j V=1", this will
> >>>> generate verbose output.
> >>>>
> >>>> And _before_ lines compiling the 'rte_class_eth.c' you should see
> >>>> something like following lines, can you please check:
> >>>> "
> >>>> SYMLINK-FILE include/rte_ethdev_core.h
> >>>>
> >>>> ln -nsf `.../buildtools/relpath.sh
> >>>> .../lib/librte_ethdev/rte_ethdev_core.h
> >>>> .../x86_64-native-linux-gcc/include`
> >>>> .../x86_64-native-linux-gcc/include
> >>>> "
> >>>>
> >>>
> >>> This is the log with V=1 build (a failed build):
> >>> == Build lib/librte_ethdev
> >>> == Build lib/librte_cmdline
> >>> ar crDs librte_lpm.a rte_lpm.o rte_lpm6.o
> >>>   INSTALL-LIB librte_lpm.a
> >>> ar crDs librte_member.a rte_member.o rte_member_ht.o
> rte_member_vbf.o
> >>>   INSTALL-LIB librte_member.a
> >>> ar crDs librte_cryptodev.a rte_cryptodev.o rte_cryptodev_pmd.o
> >>>   INSTALL-LIB librte_cryptodev.a
> >>> ar crDs librte_bbdev.a rte_bbdev.o
> >>> ar crDs librte_compressdev.a rte_compressdev.o rte_compressdev_pmd.o
> >> rte_comp.o
> >>>   INSTALL-LIB librte_compressdev.a
> >>>   INSTALL-LIB librte_bbdev.a
> >>>   SYMLINK-FILE include/rte_ethdev_pci.h
> >>>   SYMLINK-FILE include/rte_tm.h
> >>>   SYMLINK-FILE include/rte_tm_driver.h
> >>>   SYMLINK-FILE include/rte_eth_ctrl.h
> >>>   SYMLINK-FILE include/rte_flow_driver.h
> >>>   SYMLINK-FILE include/rte_ethdev_vdev.h
> >>>   SYMLINK-FILE include/rte_mtr_driver.h
> >>>   SYMLINK-FILE include/rte_ethdev_core.h
> >>>   SYMLINK-FILE include/rte_flow.h
> >>>   SYMLINK-FILE include/rte_mtr.h
> >>>   SYMLINK-FILE include/rte_ethdev_driver.h
> >>>   SYMLINK-FILE include/rte_ethdev.h
> >>>   SYMLINK-FILE include/rte_dev_info.h
> >>> gcc -Wp,-MD,./.ethdev_private.o.d.tmp   -pthread -I/tmp/dpdk-
> >> 20.02/lib/librte_eal/linux/eal/include  -march=armv8-a+crc -
> >> DRTE_MACHINE_CPUFLAG_NEON -DRTE_MACHINE_CPUFLAG_CRC32  -
> >> I/tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include -
> >> DRTE_USE_FUNCTION_VERSIONING -include /tmp/dpdk-20.02/arm64-
> armv8a-
> >> linuxapp-gcc/include/rte_config.h -D_GNU_SOURCE -
> >> DALLOW_EXPERIMENTAL_API -O3 -W -Wall -Wstrict-prototypes -Wmissing-
> >> prototypes -Wmissing-declarations -Wold-style-definition
> >> -Wpointer-arith - Wcast-align -Wnested-externs -Wcast-qual
> >> -Wformat-nonliteral -Wformat- security -Wundef -Wwrite-strings
> >> -Wdeprecated -Wno-missing-field-initializers -
> >> Wimplicit-fallthrough=2 -Wno-format-truncation -Wno-address-of-packed-
> >> member    -o ethdev_private.o -c /tmp/dpdk-
> >> 20.02/lib/librte_ethdev/ethdev_private.c
> >>> gcc -Wp,-MD,./.rte_ethdev.o.d.tmp   -pthread -I/tmp/dpdk-
> >> 20.02/lib/librte_eal/linux/eal/include  -march=armv8-a+crc -
> >> DRTE_MACHINE_CPUFLAG_NEON -DRTE_MACHINE_CPUFLAG_CRC32  -
> >> I/tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include -
> >> DRTE_USE_FUNCTION_VERSIONING -include /tmp/dpdk-20.02/arm64-
> armv8a-
> >> linuxapp-gcc/include/rte_config.h -D_GNU_SOURCE -
> >> DALLOW_EXPERIMENTAL_API -O3 -W -Wall -Wstrict-prototypes -Wmissing-
> >> prototypes -Wmissing-declarations -Wold-style-definition
> >> -Wpointer-arith - Wcast-align -Wnested-externs -Wcast-qual
> >> -Wformat-nonliteral -Wformat- security -Wundef -Wwrite-strings
> >> -Wdeprecated -Wno-missing-field-initializers -
> >> Wimplicit-fallthrough=2 -Wno-format-truncation -Wno-address-of-packed-
> >> member    -o rte_ethdev.o -c /tmp/dpdk-
> 20.02/lib/librte_ethdev/rte_ethdev.c
> >>> gcc -Wp,-MD,./.rte_class_eth.o.d.tmp   -pthread -I/tmp/dpdk-
> >> 20.02/lib/librte_eal/linux/eal/include  -march=armv8-a+crc -
> >> DRTE_MACHINE_CPUFLAG_NEON -DRTE_MACHINE_CPUFLAG_CRC32  -
> >> I/tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include -
> >> DRTE_USE_FUNCTION_VERSIONING -include /tmp/dpdk-20.02/arm64-
> armv8a-
> >> linuxapp-gcc/include/rte_config.h -D_GNU_SOURCE -
> >> DALLOW_EXPERIMENTAL_API -O3 -W -Wall -Wstrict-prototypes -Wmissing-
> >> prototypes -Wmissing-declarations -Wold-style-definition
> >> -Wpointer-arith - Wcast-align -Wnested-externs -Wcast-qual
> >> -Wformat-nonliteral -Wformat- security -Wundef -Wwrite-strings
> >> -Wdeprecated -Wno-missing-field-initializers -
> >> Wimplicit-fallthrough=2 -Wno-format-truncation -Wno-address-of-packed-
> >> member    -o rte_class_eth.o -c /tmp/dpdk-
> >> 20.02/lib/librte_ethdev/rte_class_eth.c
> >>> gcc -Wp,-MD,./.rte_flow.o.d.tmp   -pthread -I/tmp/dpdk-
> >> 20.02/lib/librte_eal/linux/eal/include  -march=armv8-a+crc -
> >> DRTE_MACHINE_CPUFLAG_NEON -DRTE_MACHINE_CPUFLAG_CRC32  -
> >> I/tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include -
> >> DRTE_USE_FUNCTION_VERSIONING -include /tmp/dpdk-20.02/arm64-
> armv8a-
> >> linuxapp-gcc/include/rte_config.h -D_GNU_SOURCE -
> >> DALLOW_EXPERIMENTAL_API -O3 -W -Wall -Wstrict-prototypes -Wmissing-
> >> prototypes -Wmissing-declarations -Wold-style-definition
> >> -Wpointer-arith - Wcast-align -Wnested-externs -Wcast-qual
> >> -Wformat-nonliteral -Wformat- security -Wundef -Wwrite-strings
> >> -Wdeprecated -Wno-missing-field-initializers -
> >> Wimplicit-fallthrough=2 -Wno-format-truncation -Wno-address-of-packed-
> >> member    -o rte_flow.o -c /tmp/dpdk-20.02/lib/librte_ethdev/rte_flow.c
> >>> gcc -Wp,-MD,./.rte_tm.o.d.tmp   -pthread -I/tmp/dpdk-
> >> 20.02/lib/librte_eal/linux/eal/include  -march=armv8-a+crc -
> >> DRTE_MACHINE_CPUFLAG_NEON -DRTE_MACHINE_CPUFLAG_CRC32  -
> >> I/tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include -
> >> DRTE_USE_FUNCTION_VERSIONING -include /tmp/dpdk-20.02/arm64-
> armv8a-
> >> linuxapp-gcc/include/rte_config.h -D_GNU_SOURCE -
> >> DALLOW_EXPERIMENTAL_API -O3 -W -Wall -Wstrict-prototypes -Wmissing-
> >> prototypes -Wmissing-declarations -Wold-style-definition
> >> -Wpointer-arith - Wcast-align -Wnested-externs -Wcast-qual
> >> -Wformat-nonliteral -Wformat- security -Wundef -Wwrite-strings
> >> -Wdeprecated -Wno-missing-field-initializers -
> >> Wimplicit-fallthrough=2 -Wno-format-truncation -Wno-address-of-packed-
> >> member    -o rte_tm.o -c /tmp/dpdk-20.02/lib/librte_ethdev/rte_tm.c
> >>>   SYMLINK-FILE include/cmdline.h
> >>>   SYMLINK-FILE include/cmdline_parse.h
> >>>   SYMLINK-FILE include/cmdline_parse_num.h
> >>> gcc -Wp,-MD,./.ethdev_profile.o.d.tmp   -pthread -I/tmp/dpdk-
> >> 20.02/lib/librte_eal/linux/eal/include  -march=armv8-a+crc -
> >> DRTE_MACHINE_CPUFLAG_NEON -DRTE_MACHINE_CPUFLAG_CRC32  -
> >> I/tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include -
> >> DRTE_USE_FUNCTION_VERSIONING -include /tmp/dpdk-20.02/arm64-
> armv8a-
> >> linuxapp-gcc/include/rte_config.h -D_GNU_SOURCE -
> >> DALLOW_EXPERIMENTAL_API -O3 -W -Wall -Wstrict-prototypes -Wmissing-
> >> prototypes -Wmissing-declarations -Wold-style-definition
> >> -Wpointer-arith - Wcast-align -Wnested-externs -Wcast-qual
> >> -Wformat-nonliteral -Wformat- security -Wundef -Wwrite-strings
> >> -Wdeprecated -Wno-missing-field-initializers -
> >> Wimplicit-fallthrough=2 -Wno-format-truncation -Wno-address-of-packed-
> >> member    -o ethdev_profile.o -c /tmp/dpdk-
> >> 20.02/lib/librte_ethdev/ethdev_profile.c
> >>> gcc -Wp,-MD,./.rte_mtr.o.d.tmp   -pthread -I/tmp/dpdk-
> >> 20.02/lib/librte_eal/linux/eal/include  -march=armv8-a+crc -
> >> DRTE_MACHINE_CPUFLAG_NEON -DRTE_MACHINE_CPUFLAG_CRC32  -
> >> I/tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include -
> >> DRTE_USE_FUNCTION_VERSIONING -include /tmp/dpdk-20.02/arm64-
> armv8a-
> >> linuxapp-gcc/include/rte_config.h -D_GNU_SOURCE -
> >> DALLOW_EXPERIMENTAL_API -O3 -W -Wall -Wstrict-prototypes -Wmissing-
> >> prototypes -Wmissing-declarations -Wold-style-definition
> >> -Wpointer-arith - Wcast-align -Wnested-externs -Wcast-qual
> >> -Wformat-nonliteral -Wformat- security -Wundef -Wwrite-strings
> >> -Wdeprecated -Wno-missing-field-initializers -
> >> Wimplicit-fallthrough=2 -Wno-format-truncation -Wno-address-of-packed-
> >> member    -o rte_mtr.o -c /tmp/dpdk-20.02/lib/librte_ethdev/rte_mtr.c
> >>>   SYMLINK-FILE include/rte_sched.h
> >>>   SYMLINK-FILE include/rte_red.h
> >>>   SYMLINK-FILE include/rte_approx.h
> >>> In file included from /tmp/dpdk-20.02/lib/librte_ethdev/rte_ethdev.c:43:
> >>> /tmp/dpdk-20.02/lib/librte_ethdev/rte_ethdev_driver.h:18:10: fatal
> >>> error: rte_ethdev.h: No such file or directory  #include <rte_ethdev.h>
> >>>           ^~~~~~~~~~~~~~
> >>> compilation terminated.
> >>> /tmp/dpdk-20.02/mk/internal/rte.compile-pre.mk:114: recipe for
> >>> target 'rte_ethdev.o' failed
> >>>
> >>>> Also for the failed case, after fail can you please check if
> >>>> a)
> >>>> '.../arm64-armv8a-linuxapp-gcc/build/lib/librte_ethdev/_preinstall'
> >>>> file exists
> >>>> b) '.../x86_64-native-linux-gcc/include/rte_ethdev_core.h' exists
> >>>> and it linked to correct file '../../lib/librte_ethdev/rte_ethdev_core.h'
> >>>>
> >>>
> >>> a) it doesn't exist
> >>> b) all of these exist:
> >>> rte_ethdev_core.h -> ../../lib/librte_ethdev/rte_ethdev_core.h
> >>> rte_ethdev_driver.h -> ../../lib/librte_ethdev/rte_ethdev_driver.h
> >>> rte_ethdev.h -> ../../lib/librte_ethdev/rte_ethdev.h
> >>> rte_ethdev_pci.h -> ../../lib/librte_ethdev/rte_ethdev_pci.h
> >>> rte_ethdev_vdev.h -> ../../lib/librte_ethdev/rte_ethdev_vdev.h
> >>>
> >>> I was able to reproduce the failure after 7 successful builds.
> >>
> >> Thanks for details.
> >>
> >> This time it seems build complaining about not able to find
> >> "rte_ethdev.h", so your patch won't fix this case and build still
> >> will fail, looks like we need different fix.
> >>
> >
> > Right, Thomas already pointed this out - I haven't submitted a new patch
> version since it didn't make much sense before we hash this all out.
> >
> >> And in the build command for 'rte_ethdev.c', as you can see there is
> >> '- I/tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include' paramter, and
> >> you mentioned above "rte_ethdev.h ->
> >> ../../lib/librte_ethdev/rte_ethdev.h" link exist under that filder, so still not
> clear why compiler can't find the files.
> >>
> >> There is a dependency of build for the '_preinstall' file, so
> >> buildling .c files in that library shouldn't start before all header
> >> files are installed and '_preinstall' file is created. This is also not clear how
> '_preinstall' can be missing.
> >>
> >> Can you please double check above links are valid, I mean you can
> >> access to the files from links?
> >>
> >
> > The links are pointing to existing files:
> > testuser@s17-t33-sut1:/tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/includ
> > e$ ll rte_ethdev.h lrwxrwxrwx 1 testuser testuser 36 Jun  5 11:01
> > rte_ethdev.h -> ../../lib/librte_ethdev/rte_ethdev.h
> > testuser@s17-t33-sut1:/tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/includ
> > e$ ll ../../lib/librte_ethdev/rte_ethdev.h
> > -rw-rw-r-- 1 testuser testuser 171636 Feb 25 15:46
> > ../../lib/librte_ethdev/rte_ethdev.h
> >
> > This is a complete mystery to me. Maybe there are some implementation
> details in the arm version of gcc or make?
> 
> I don't see the lines as following, for symbolic link, did you strip them or not
> gettim them at all:
> ln -nsf `/tmp/build/dpdk-20.05/buildtools/relpath.sh .....
> 

Those are not in the output. Is that an indication of something going worng?

> >
> >>>
> >>>>
> >>>>>
> >>>>> I'll try Meson build a few times. How can I enable those two
> >>>>> config options in
> >>>> Meson?
> >>>>>
> >>>>>> Thanks,
> >>>>>> ferruh
> >>>>>>
> >>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>> Fix this by replacing the directive with #include "file"
> >>>>>>>>>> which searches for files first in the directory containing
> >>>>>>>>>> the current file and other directories only after that.
> >>>>>>>>>>
> >>>>>>>>>> Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
> >>>>>>>>>
> >>>>>>>>> What about #include <rte_ethdev_trace.h> ?
> >>>>>>>>>
> >>>>>>>>> Except this,
> >>>>>>>>> Acked-by: Thomas Monjalon <thomas@monjalon.net>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> I would change the title to be more generic (nothing specific
> >>>>>>>>> to gcc or Arm
> >>>>>>>> here):
> >>>>>>>>> 	ethdev: fix local include
> >>>>>>>>>
> >>>>>>>>> It needs to be backported, so these lines are required:
> >>>>>>>>> 	Fixes: 7f0a669e7b04 ("ethdev: add allocation helper for
> >>>>>>>>> virtual
> >> drivers")
> >>>>>>>>> 	Fixes: dcd5c8112bc3 ("ethdev: add PCI driver helpers")
> >>>>>>>>> 	Fixes: ffc905f3b856 ("ethdev: separate driver APIs")
> >>>>>>>>> 	Fixes: 331c447ad913 ("ethdev: separate internal structures
> >>>>>>>>> into own
> >>>>>>>> header")
> >>>>>>>>> 	Cc: stable@dpdk.org
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>
> >>>
> >>
> >
> 

Re: [dpdk-dev] [PATCH] ethdev: fix dpdk gcc build on Arm

[Ferruh Yigit]

On 6/8/2020 2:19 PM, Juraj Linkeš wrote:
> 
> 
>> -----Original Message-----
>> From: Ferruh Yigit <ferruh.yigit@intel.com>
>> Sent: Monday, June 8, 2020 2:39 PM
>> To: Juraj Linkeš <juraj.linkes@pantheon.tech>; Thomas Monjalon
>> <thomas@monjalon.net>
>> Cc: arybchenko@solarflare.com; dev@dpdk.org; Honnappa Nagarahalli
>> <Honnappa.Nagarahalli@arm.com>
>> Subject: Re: [dpdk-dev] [PATCH] ethdev: fix dpdk gcc build on Arm
>>
>> On 6/8/2020 1:03 PM, Juraj Linkeš wrote:
>>>
>>>
>>>> -----Original Message-----
>>>> From: Ferruh Yigit <ferruh.yigit@intel.com>
>>>> Sent: Monday, June 8, 2020 1:39 PM
>>>> To: Juraj Linkeš <juraj.linkes@pantheon.tech>; Thomas Monjalon
>>>> <thomas@monjalon.net>
>>>> Cc: arybchenko@solarflare.com; dev@dpdk.org; Honnappa Nagarahalli
>>>> <Honnappa.Nagarahalli@arm.com>
>>>> Subject: Re: [dpdk-dev] [PATCH] ethdev: fix dpdk gcc build on Arm
>>>>
>>>> On 6/8/2020 9:17 AM, Juraj Linkeš wrote:
>>>>>
>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Ferruh Yigit <ferruh.yigit@intel.com>
>>>>>> Sent: Thursday, June 4, 2020 3:30 PM
>>>>>> To: Juraj Linkeš <juraj.linkes@pantheon.tech>; Thomas Monjalon
>>>>>> <thomas@monjalon.net>
>>>>>> Cc: arybchenko@solarflare.com; dev@dpdk.org; Honnappa Nagarahalli
>>>>>> <Honnappa.Nagarahalli@arm.com>
>>>>>> Subject: Re: [PATCH] ethdev: fix dpdk gcc build on Arm
>>>>>>
>>>>>> On 6/4/2020 1:55 PM, Juraj Linkeš wrote:
>>>>>>>> -----Original Message-----
>>>>>>>> From: Ferruh Yigit <ferruh.yigit@intel.com>
>>>>>>>> Sent: Thursday, June 4, 2020 2:23 PM
>>>>>>>> To: Juraj Linkeš <juraj.linkes@pantheon.tech>; Thomas Monjalon
>>>>>>>> <thomas@monjalon.net>
>>>>>>>> Cc: arybchenko@solarflare.com; dev@dpdk.org; Honnappa Nagarahalli
>>>>>>>> <Honnappa.Nagarahalli@arm.com>
>>>>>>>> Subject: Re: [PATCH] ethdev: fix dpdk gcc build on Arm
>>>>>>>>
>>>>>>>> On 6/4/2020 11:36 AM, Juraj Linkeš wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> -----Original Message-----
>>>>>>>>>> From: Ferruh Yigit <ferruh.yigit@intel.com>
>>>>>>>>>> Sent: Wednesday, June 3, 2020 1:41 PM
>>>>>>>>>> To: Thomas Monjalon <thomas@monjalon.net>; Juraj Linkeš
>>>>>>>>>> <juraj.linkes@pantheon.tech>
>>>>>>>>>> Cc: arybchenko@solarflare.com; dev@dpdk.org
>>>>>>>>>> Subject: Re: [PATCH] ethdev: fix dpdk gcc build on Arm
>>>>>>>>>>
>>>>>>>>>> On 6/3/2020 11:16 AM, Thomas Monjalon wrote:
>>>>>>>>>>> 03/06/2020 11:48, Juraj Linkeš:
>>>>>>>>>>>> Directive #include <file> in gcc implementation searches for
>>>>>>>>>>>> files in a standard list of system directories, which leads
>>>>>>>>>>>> to a sporadici build error on Taishan arm machines:
>>>>>>>>>>>> /tmp/openvpp-
>> testing/dpdk/lib/librte_ethdev/rte_ethdev.h:4287:10:
>>>>>>>>>>>> fatal error: rte_ethdev_core.h:
>>>>>>>>>>>> No such file or directory #include <rte_ethdev_core.h>
>>>>>>>>>>>
>>>>>>>>>>> Would be interesting to know why nobody else hit such error?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I can't see why this is happening, in the 'mk/rte.lib.mk' we
>>>>>>>>>> have
>>>> following:
>>>>>>>>>>
>>>>>>>>>> "
>>>>>>>>>> install: _preinstall build _postinstall
>>>>>>>>>> build: _preinstall
>>>>>>>>>> "
>>>>>>>>>>
>>>>>>>>>> Which should cause the library header files installed before
>>>>>>>>>> building .c files in that library.
>>>>>>>>>> So when compiling 'rte_class_eth.c', the header files should be
>>>>>>>>>> already in install folder.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I can see how/why changing to "" fixes the issue but I am not
>>>>>>>>>> sure about this
>>>>>>>> fix.
>>>>>>>>>> "rte_ethdev.h" is a public header file, that applications will
>>>>>>>>>> include it in their applications. In the public library it is
>>>>>>>>>> more proper to have other includes from system folder, using format
>> <>.
>>>>>>>>>> Again, I can't see why it is failing but I believe we should
>>>>>>>>>> find another solution for _internal_ build error.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> A very simple solution can be following, but that is also not
>>>>>>>>>> good, since it solves the issue by creating a dependency to the
>>>>>>>>>> order of the header
>>>>>>>> includes:
>>>>>>>>>>  diff --git a/lib/librte_ethdev/rte_class_eth.c
>>>>>>>>>> b/lib/librte_ethdev/rte_class_eth.c
>>>>>>>>>>  index 6338355e25..3030c49020 100644
>>>>>>>>>>  --- a/lib/librte_ethdev/rte_class_eth.c
>>>>>>>>>>  +++ b/lib/librte_ethdev/rte_class_eth.c
>>>>>>>>>>  @@ -10,8 +10,8 @@
>>>>>>>>>>   #include <rte_kvargs.h>
>>>>>>>>>>   #include <rte_log.h>
>>>>>>>>>>
>>>>>>>>>>  -#include "rte_ethdev.h"
>>>>>>>>>>   #include "rte_ethdev_core.h"
>>>>>>>>>>  +#include "rte_ethdev.h"
>>>>>>>>>>   #include "rte_ethdev_driver.h"
>>>>>>>>>>   #include "ethdev_private.h"
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thomas, Ferruh, what should be the solution? I'm not an expert
>>>>>>>>> on this and I
>>>>>>>> can't really offer anything better, but I'd like that this gets fixed.
>>>>>>>>
>>>>>>>> First we need to root cause this before trying to solve it.
>>>>>>>> Honnappa also said he can reproduce this but our CI builds can't
>>>>>>>> (we are talking about tens of builds daily on various platforms),
>>>>>>>> need to understand
>>>>>> why.
>>>>>>>> Also from Makefile I can't see how this is happening, I am
>>>>>>>> feeling uneasy to fix something before figuring out how/why it is failing.
>>>>>>>>
>>>>>>>> Can you please try to collect more data on when/how this happens,
>>>>>>>> initial questions I can think of:
>>>>>>>> - Can you reproduce this with meson build?
>>>>>>>> - Is it bare DPDK build, or build part of other project (I guess
>>>>>>>> I saw fd.io on the
>>>>>>>> link)
>>>>>>>> - - If this is not bare DPDK build what changes has been done to
>>>>>>>> build
>>>> system?
>>>>>>>> - Do you see this with fresh build (new clone) or rebuild of existing
>> clone?
>>>>>>>> - Can you confirm you have correct RTE_SDK and RTE_TARGET
>>>>>>>> environment variables?
>>>>>>>> - Can you please share your build command?
>>>>>>>>
>>>>>>>
>>>>>>> I sent an e-mail to dpdk dev a few days back asking for help where
>>>>>>> I outlined
>>>>>> what we're doing:
>>>>>>> We're not doing anything special, just downloading and extracting
>>>>>>> the archive,
>>>>>> then setting CONFIG_RTE_LIBRTE_MLX5_PMD and
>>>>>> CONFIG_RTE_LIBRTE_I40E_16BYTE_RX_DESC to y in
>> config/common_base
>>>> and
>>>>>> then running make install T=arm64-armv8a-linuxapp-gcc -j. As
>>>>>> mentioned in the subject, the build server is a Taishan ARM server.
>>>>>>>
>>>>>>> We're doing a fresh rebuild everytime. The error doesn't happen
>>>>>>> everytime,
>>>>>> just sometimes - it seems to be random.
>>>>>>>
>>>>>>> We don't set RTE_SDK nor RTE_TARGET since
>>>>>> https://doc.dpdk.org/guides/linux_gsg/build_dpdk.html#installation-
>>>>>> of
>>>>>> -dpdk- target-environment-using-make doesn't mention those.
>>>>>>
>>>>>> Can you please try with "V=1",
>>>>>> "make install T=arm64-armv8a-linuxapp-gcc -j V=1", this will
>>>>>> generate verbose output.
>>>>>>
>>>>>> And _before_ lines compiling the 'rte_class_eth.c' you should see
>>>>>> something like following lines, can you please check:
>>>>>> "
>>>>>> SYMLINK-FILE include/rte_ethdev_core.h
>>>>>>
>>>>>> ln -nsf `.../buildtools/relpath.sh
>>>>>> .../lib/librte_ethdev/rte_ethdev_core.h
>>>>>> .../x86_64-native-linux-gcc/include`
>>>>>> .../x86_64-native-linux-gcc/include
>>>>>> "
>>>>>>
>>>>>
>>>>> This is the log with V=1 build (a failed build):
>>>>> == Build lib/librte_ethdev
>>>>> == Build lib/librte_cmdline
>>>>> ar crDs librte_lpm.a rte_lpm.o rte_lpm6.o
>>>>>   INSTALL-LIB librte_lpm.a
>>>>> ar crDs librte_member.a rte_member.o rte_member_ht.o
>> rte_member_vbf.o
>>>>>   INSTALL-LIB librte_member.a
>>>>> ar crDs librte_cryptodev.a rte_cryptodev.o rte_cryptodev_pmd.o
>>>>>   INSTALL-LIB librte_cryptodev.a
>>>>> ar crDs librte_bbdev.a rte_bbdev.o
>>>>> ar crDs librte_compressdev.a rte_compressdev.o rte_compressdev_pmd.o
>>>> rte_comp.o
>>>>>   INSTALL-LIB librte_compressdev.a
>>>>>   INSTALL-LIB librte_bbdev.a
>>>>>   SYMLINK-FILE include/rte_ethdev_pci.h
>>>>>   SYMLINK-FILE include/rte_tm.h
>>>>>   SYMLINK-FILE include/rte_tm_driver.h
>>>>>   SYMLINK-FILE include/rte_eth_ctrl.h
>>>>>   SYMLINK-FILE include/rte_flow_driver.h
>>>>>   SYMLINK-FILE include/rte_ethdev_vdev.h
>>>>>   SYMLINK-FILE include/rte_mtr_driver.h
>>>>>   SYMLINK-FILE include/rte_ethdev_core.h
>>>>>   SYMLINK-FILE include/rte_flow.h
>>>>>   SYMLINK-FILE include/rte_mtr.h
>>>>>   SYMLINK-FILE include/rte_ethdev_driver.h
>>>>>   SYMLINK-FILE include/rte_ethdev.h
>>>>>   SYMLINK-FILE include/rte_dev_info.h
>>>>> gcc -Wp,-MD,./.ethdev_private.o.d.tmp   -pthread -I/tmp/dpdk-
>>>> 20.02/lib/librte_eal/linux/eal/include  -march=armv8-a+crc -
>>>> DRTE_MACHINE_CPUFLAG_NEON -DRTE_MACHINE_CPUFLAG_CRC32  -
>>>> I/tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include -
>>>> DRTE_USE_FUNCTION_VERSIONING -include /tmp/dpdk-20.02/arm64-
>> armv8a-
>>>> linuxapp-gcc/include/rte_config.h -D_GNU_SOURCE -
>>>> DALLOW_EXPERIMENTAL_API -O3 -W -Wall -Wstrict-prototypes -Wmissing-
>>>> prototypes -Wmissing-declarations -Wold-style-definition
>>>> -Wpointer-arith - Wcast-align -Wnested-externs -Wcast-qual
>>>> -Wformat-nonliteral -Wformat- security -Wundef -Wwrite-strings
>>>> -Wdeprecated -Wno-missing-field-initializers -
>>>> Wimplicit-fallthrough=2 -Wno-format-truncation -Wno-address-of-packed-
>>>> member    -o ethdev_private.o -c /tmp/dpdk-
>>>> 20.02/lib/librte_ethdev/ethdev_private.c
>>>>> gcc -Wp,-MD,./.rte_ethdev.o.d.tmp   -pthread -I/tmp/dpdk-
>>>> 20.02/lib/librte_eal/linux/eal/include  -march=armv8-a+crc -
>>>> DRTE_MACHINE_CPUFLAG_NEON -DRTE_MACHINE_CPUFLAG_CRC32  -
>>>> I/tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include -
>>>> DRTE_USE_FUNCTION_VERSIONING -include /tmp/dpdk-20.02/arm64-
>> armv8a-
>>>> linuxapp-gcc/include/rte_config.h -D_GNU_SOURCE -
>>>> DALLOW_EXPERIMENTAL_API -O3 -W -Wall -Wstrict-prototypes -Wmissing-
>>>> prototypes -Wmissing-declarations -Wold-style-definition
>>>> -Wpointer-arith - Wcast-align -Wnested-externs -Wcast-qual
>>>> -Wformat-nonliteral -Wformat- security -Wundef -Wwrite-strings
>>>> -Wdeprecated -Wno-missing-field-initializers -
>>>> Wimplicit-fallthrough=2 -Wno-format-truncation -Wno-address-of-packed-
>>>> member    -o rte_ethdev.o -c /tmp/dpdk-
>> 20.02/lib/librte_ethdev/rte_ethdev.c
>>>>> gcc -Wp,-MD,./.rte_class_eth.o.d.tmp   -pthread -I/tmp/dpdk-
>>>> 20.02/lib/librte_eal/linux/eal/include  -march=armv8-a+crc -
>>>> DRTE_MACHINE_CPUFLAG_NEON -DRTE_MACHINE_CPUFLAG_CRC32  -
>>>> I/tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include -
>>>> DRTE_USE_FUNCTION_VERSIONING -include /tmp/dpdk-20.02/arm64-
>> armv8a-
>>>> linuxapp-gcc/include/rte_config.h -D_GNU_SOURCE -
>>>> DALLOW_EXPERIMENTAL_API -O3 -W -Wall -Wstrict-prototypes -Wmissing-
>>>> prototypes -Wmissing-declarations -Wold-style-definition
>>>> -Wpointer-arith - Wcast-align -Wnested-externs -Wcast-qual
>>>> -Wformat-nonliteral -Wformat- security -Wundef -Wwrite-strings
>>>> -Wdeprecated -Wno-missing-field-initializers -
>>>> Wimplicit-fallthrough=2 -Wno-format-truncation -Wno-address-of-packed-
>>>> member    -o rte_class_eth.o -c /tmp/dpdk-
>>>> 20.02/lib/librte_ethdev/rte_class_eth.c
>>>>> gcc -Wp,-MD,./.rte_flow.o.d.tmp   -pthread -I/tmp/dpdk-
>>>> 20.02/lib/librte_eal/linux/eal/include  -march=armv8-a+crc -
>>>> DRTE_MACHINE_CPUFLAG_NEON -DRTE_MACHINE_CPUFLAG_CRC32  -
>>>> I/tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include -
>>>> DRTE_USE_FUNCTION_VERSIONING -include /tmp/dpdk-20.02/arm64-
>> armv8a-
>>>> linuxapp-gcc/include/rte_config.h -D_GNU_SOURCE -
>>>> DALLOW_EXPERIMENTAL_API -O3 -W -Wall -Wstrict-prototypes -Wmissing-
>>>> prototypes -Wmissing-declarations -Wold-style-definition
>>>> -Wpointer-arith - Wcast-align -Wnested-externs -Wcast-qual
>>>> -Wformat-nonliteral -Wformat- security -Wundef -Wwrite-strings
>>>> -Wdeprecated -Wno-missing-field-initializers -
>>>> Wimplicit-fallthrough=2 -Wno-format-truncation -Wno-address-of-packed-
>>>> member    -o rte_flow.o -c /tmp/dpdk-20.02/lib/librte_ethdev/rte_flow.c
>>>>> gcc -Wp,-MD,./.rte_tm.o.d.tmp   -pthread -I/tmp/dpdk-
>>>> 20.02/lib/librte_eal/linux/eal/include  -march=armv8-a+crc -
>>>> DRTE_MACHINE_CPUFLAG_NEON -DRTE_MACHINE_CPUFLAG_CRC32  -
>>>> I/tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include -
>>>> DRTE_USE_FUNCTION_VERSIONING -include /tmp/dpdk-20.02/arm64-
>> armv8a-
>>>> linuxapp-gcc/include/rte_config.h -D_GNU_SOURCE -
>>>> DALLOW_EXPERIMENTAL_API -O3 -W -Wall -Wstrict-prototypes -Wmissing-
>>>> prototypes -Wmissing-declarations -Wold-style-definition
>>>> -Wpointer-arith - Wcast-align -Wnested-externs -Wcast-qual
>>>> -Wformat-nonliteral -Wformat- security -Wundef -Wwrite-strings
>>>> -Wdeprecated -Wno-missing-field-initializers -
>>>> Wimplicit-fallthrough=2 -Wno-format-truncation -Wno-address-of-packed-
>>>> member    -o rte_tm.o -c /tmp/dpdk-20.02/lib/librte_ethdev/rte_tm.c
>>>>>   SYMLINK-FILE include/cmdline.h
>>>>>   SYMLINK-FILE include/cmdline_parse.h
>>>>>   SYMLINK-FILE include/cmdline_parse_num.h
>>>>> gcc -Wp,-MD,./.ethdev_profile.o.d.tmp   -pthread -I/tmp/dpdk-
>>>> 20.02/lib/librte_eal/linux/eal/include  -march=armv8-a+crc -
>>>> DRTE_MACHINE_CPUFLAG_NEON -DRTE_MACHINE_CPUFLAG_CRC32  -
>>>> I/tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include -
>>>> DRTE_USE_FUNCTION_VERSIONING -include /tmp/dpdk-20.02/arm64-
>> armv8a-
>>>> linuxapp-gcc/include/rte_config.h -D_GNU_SOURCE -
>>>> DALLOW_EXPERIMENTAL_API -O3 -W -Wall -Wstrict-prototypes -Wmissing-
>>>> prototypes -Wmissing-declarations -Wold-style-definition
>>>> -Wpointer-arith - Wcast-align -Wnested-externs -Wcast-qual
>>>> -Wformat-nonliteral -Wformat- security -Wundef -Wwrite-strings
>>>> -Wdeprecated -Wno-missing-field-initializers -
>>>> Wimplicit-fallthrough=2 -Wno-format-truncation -Wno-address-of-packed-
>>>> member    -o ethdev_profile.o -c /tmp/dpdk-
>>>> 20.02/lib/librte_ethdev/ethdev_profile.c
>>>>> gcc -Wp,-MD,./.rte_mtr.o.d.tmp   -pthread -I/tmp/dpdk-
>>>> 20.02/lib/librte_eal/linux/eal/include  -march=armv8-a+crc -
>>>> DRTE_MACHINE_CPUFLAG_NEON -DRTE_MACHINE_CPUFLAG_CRC32  -
>>>> I/tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include -
>>>> DRTE_USE_FUNCTION_VERSIONING -include /tmp/dpdk-20.02/arm64-
>> armv8a-
>>>> linuxapp-gcc/include/rte_config.h -D_GNU_SOURCE -
>>>> DALLOW_EXPERIMENTAL_API -O3 -W -Wall -Wstrict-prototypes -Wmissing-
>>>> prototypes -Wmissing-declarations -Wold-style-definition
>>>> -Wpointer-arith - Wcast-align -Wnested-externs -Wcast-qual
>>>> -Wformat-nonliteral -Wformat- security -Wundef -Wwrite-strings
>>>> -Wdeprecated -Wno-missing-field-initializers -
>>>> Wimplicit-fallthrough=2 -Wno-format-truncation -Wno-address-of-packed-
>>>> member    -o rte_mtr.o -c /tmp/dpdk-20.02/lib/librte_ethdev/rte_mtr.c
>>>>>   SYMLINK-FILE include/rte_sched.h
>>>>>   SYMLINK-FILE include/rte_red.h
>>>>>   SYMLINK-FILE include/rte_approx.h
>>>>> In file included from /tmp/dpdk-20.02/lib/librte_ethdev/rte_ethdev.c:43:
>>>>> /tmp/dpdk-20.02/lib/librte_ethdev/rte_ethdev_driver.h:18:10: fatal
>>>>> error: rte_ethdev.h: No such file or directory  #include <rte_ethdev.h>
>>>>>           ^~~~~~~~~~~~~~
>>>>> compilation terminated.
>>>>> /tmp/dpdk-20.02/mk/internal/rte.compile-pre.mk:114: recipe for
>>>>> target 'rte_ethdev.o' failed
>>>>>
>>>>>> Also for the failed case, after fail can you please check if
>>>>>> a)
>>>>>> '.../arm64-armv8a-linuxapp-gcc/build/lib/librte_ethdev/_preinstall'
>>>>>> file exists
>>>>>> b) '.../x86_64-native-linux-gcc/include/rte_ethdev_core.h' exists
>>>>>> and it linked to correct file '../../lib/librte_ethdev/rte_ethdev_core.h'
>>>>>>
>>>>>
>>>>> a) it doesn't exist
>>>>> b) all of these exist:
>>>>> rte_ethdev_core.h -> ../../lib/librte_ethdev/rte_ethdev_core.h
>>>>> rte_ethdev_driver.h -> ../../lib/librte_ethdev/rte_ethdev_driver.h
>>>>> rte_ethdev.h -> ../../lib/librte_ethdev/rte_ethdev.h
>>>>> rte_ethdev_pci.h -> ../../lib/librte_ethdev/rte_ethdev_pci.h
>>>>> rte_ethdev_vdev.h -> ../../lib/librte_ethdev/rte_ethdev_vdev.h
>>>>>
>>>>> I was able to reproduce the failure after 7 successful builds.
>>>>
>>>> Thanks for details.
>>>>
>>>> This time it seems build complaining about not able to find
>>>> "rte_ethdev.h", so your patch won't fix this case and build still
>>>> will fail, looks like we need different fix.
>>>>
>>>
>>> Right, Thomas already pointed this out - I haven't submitted a new patch
>> version since it didn't make much sense before we hash this all out.
>>>
>>>> And in the build command for 'rte_ethdev.c', as you can see there is
>>>> '- I/tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/include' paramter, and
>>>> you mentioned above "rte_ethdev.h ->
>>>> ../../lib/librte_ethdev/rte_ethdev.h" link exist under that filder, so still not
>> clear why compiler can't find the files.
>>>>
>>>> There is a dependency of build for the '_preinstall' file, so
>>>> buildling .c files in that library shouldn't start before all header
>>>> files are installed and '_preinstall' file is created. This is also not clear how
>> '_preinstall' can be missing.
>>>>
>>>> Can you please double check above links are valid, I mean you can
>>>> access to the files from links?
>>>>
>>>
>>> The links are pointing to existing files:
>>> testuser@s17-t33-sut1:/tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/includ
>>> e$ ll rte_ethdev.h lrwxrwxrwx 1 testuser testuser 36 Jun  5 11:01
>>> rte_ethdev.h -> ../../lib/librte_ethdev/rte_ethdev.h
>>> testuser@s17-t33-sut1:/tmp/dpdk-20.02/arm64-armv8a-linuxapp-gcc/includ
>>> e$ ll ../../lib/librte_ethdev/rte_ethdev.h
>>> -rw-rw-r-- 1 testuser testuser 171636 Feb 25 15:46
>>> ../../lib/librte_ethdev/rte_ethdev.h
>>>
>>> This is a complete mystery to me. Maybe there are some implementation
>> details in the arm version of gcc or make?
>>
>> I don't see the lines as following, for symbolic link, did you strip them or not
>> gettim them at all:
>> ln -nsf `/tmp/build/dpdk-20.05/buildtools/relpath.sh .....
>>
> 
> Those are not in the output. Is that an indication of something going worng?

Not sure, just trying to figure out, what I have is something like:
https://pastebin.com/aEveexD6

Just trying to understand why it may be missing for your case.


> 
>>>
>>>>>
>>>>>>
>>>>>>>
>>>>>>> I'll try Meson build a few times. How can I enable those two
>>>>>>> config options in
>>>>>> Meson?
>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> ferruh
>>>>>>>>
>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> Fix this by replacing the directive with #include "file"
>>>>>>>>>>>> which searches for files first in the directory containing
>>>>>>>>>>>> the current file and other directories only after that.
>>>>>>>>>>>>
>>>>>>>>>>>> Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
>>>>>>>>>>>
>>>>>>>>>>> What about #include <rte_ethdev_trace.h> ?
>>>>>>>>>>>
>>>>>>>>>>> Except this,
>>>>>>>>>>> Acked-by: Thomas Monjalon <thomas@monjalon.net>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I would change the title to be more generic (nothing specific
>>>>>>>>>>> to gcc or Arm
>>>>>>>>>> here):
>>>>>>>>>>> 	ethdev: fix local include
>>>>>>>>>>>
>>>>>>>>>>> It needs to be backported, so these lines are required:
>>>>>>>>>>> 	Fixes: 7f0a669e7b04 ("ethdev: add allocation helper for
>>>>>>>>>>> virtual
>>>> drivers")
>>>>>>>>>>> 	Fixes: dcd5c8112bc3 ("ethdev: add PCI driver helpers")
>>>>>>>>>>> 	Fixes: ffc905f3b856 ("ethdev: separate driver APIs")
>>>>>>>>>>> 	Fixes: 331c447ad913 ("ethdev: separate internal structures
>>>>>>>>>>> into own
>>>>>>>>>> header")
>>>>>>>>>>> 	Cc: stable@dpdk.org
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>
>>>>
>>>
>>
> 

Re: [dpdk-dev] [PATCH] ethdev: fix dpdk gcc build on Arm

[Bruce Richardson]

On Thu, Jun 04, 2020 at 12:55:40PM +0000, Juraj Linkeš wrote:
> > -----Original Message-----
> > From: Ferruh Yigit <ferruh.yigit@intel.com>
> > Sent: Thursday, June 4, 2020 2:23 PM
> > To: Juraj Linkeš <juraj.linkes@pantheon.tech>; Thomas Monjalon
> > <thomas@monjalon.net>
> > Cc: arybchenko@solarflare.com; dev@dpdk.org; Honnappa Nagarahalli
> > <Honnappa.Nagarahalli@arm.com>
> > Subject: Re: [PATCH] ethdev: fix dpdk gcc build on Arm
> > 
> > On 6/4/2020 11:36 AM, Juraj Linkeš wrote:
> > >
> > >
> > >> -----Original Message-----
> > >> From: Ferruh Yigit <ferruh.yigit@intel.com>
> > >> Sent: Wednesday, June 3, 2020 1:41 PM
> > >> To: Thomas Monjalon <thomas@monjalon.net>; Juraj Linkeš
> > >> <juraj.linkes@pantheon.tech>
> > >> Cc: arybchenko@solarflare.com; dev@dpdk.org
> > >> Subject: Re: [PATCH] ethdev: fix dpdk gcc build on Arm
> > >>
> > >> On 6/3/2020 11:16 AM, Thomas Monjalon wrote:
> > >>> 03/06/2020 11:48, Juraj Linkeš:
> > >>>> Directive #include <file> in gcc implementation searches for files
> > >>>> in a standard list of system directories, which leads to a
> > >>>> sporadici build error on Taishan arm machines:
> > >>>> /tmp/openvpp-testing/dpdk/lib/librte_ethdev/rte_ethdev.h:4287:10:
> > >>>> fatal error: rte_ethdev_core.h:
> > >>>> No such file or directory #include <rte_ethdev_core.h>
> > >>>
> > >>> Would be interesting to know why nobody else hit such error?
> > >>
> > >>
> > >> I can't see why this is happening, in the 'mk/rte.lib.mk' we have following:
> > >>
> > >> "
> > >> install: _preinstall build _postinstall
> > >> build: _preinstall
> > >> "
> > >>
> > >> Which should cause the library header files installed before building
> > >> .c files in that library.
> > >> So when compiling 'rte_class_eth.c', the header files should be
> > >> already in install folder.
> > >>
> > >>
> > >> I can see how/why changing to "" fixes the issue but I am not sure about this
> > fix.
> > >> "rte_ethdev.h" is a public header file, that applications will
> > >> include it in their applications. In the public library it is more
> > >> proper to have other includes from system folder, using format <>.
> > >> Again, I can't see why it is failing but I believe we should find
> > >> another solution for _internal_ build error.
> > >>
> > >>
> > >> A very simple solution can be following, but that is also not good,
> > >> since it solves the issue by creating a dependency to the order of the header
> > includes:
> > >>  diff --git a/lib/librte_ethdev/rte_class_eth.c
> > >> b/lib/librte_ethdev/rte_class_eth.c
> > >>  index 6338355e25..3030c49020 100644
> > >>  --- a/lib/librte_ethdev/rte_class_eth.c
> > >>  +++ b/lib/librte_ethdev/rte_class_eth.c
> > >>  @@ -10,8 +10,8 @@
> > >>   #include <rte_kvargs.h>
> > >>   #include <rte_log.h>
> > >>
> > >>  -#include "rte_ethdev.h"
> > >>   #include "rte_ethdev_core.h"
> > >>  +#include "rte_ethdev.h"
> > >>   #include "rte_ethdev_driver.h"
> > >>   #include "ethdev_private.h"
> > >>
> > >
> > > Thomas, Ferruh, what should be the solution? I'm not an expert on this and I
> > can't really offer anything better, but I'd like that this gets fixed.
> > 
> > First we need to root cause this before trying to solve it. Honnappa also said he
> > can reproduce this but our CI builds can't (we are talking about tens of builds
> > daily on various platforms), need to understand why.
> > Also from Makefile I can't see how this is happening, I am feeling uneasy to fix
> > something before figuring out how/why it is failing.
> > 
> > Can you please try to collect more data on when/how this happens, initial
> > questions I can think of:
> > - Can you reproduce this with meson build?
> > - Is it bare DPDK build, or build part of other project (I guess I saw fd.io on the
> > link)
> > - - If this is not bare DPDK build what changes has been done to build system?
> > - Do you see this with fresh build (new clone) or rebuild of existing clone?
> > - Can you confirm you have correct RTE_SDK and RTE_TARGET environment
> > variables?
> > - Can you please share your build command?
> > 
> 
> I sent an e-mail to dpdk dev a few days back asking for help where I outlined what we're doing:
> We're not doing anything special, just downloading and extracting the archive, then setting CONFIG_RTE_LIBRTE_MLX5_PMD and CONFIG_RTE_LIBRTE_I40E_16BYTE_RX_DESC to y in config/common_base and then running make install T=arm64-armv8a-linuxapp-gcc -j. As mentioned in the subject, the build server is a Taishan ARM server.
> 
> We're doing a fresh rebuild everytime. The error doesn't happen everytime, just sometimes - it seems to be random.
> 
> We don't set RTE_SDK nor RTE_TARGET since https://doc.dpdk.org/guides/linux_gsg/build_dpdk.html#installation-of-dpdk-target-environment-using-make doesn't mention those.
> 
> I'll try Meson build a few times. How can I enable those two config options in Meson?
> 

The MLX drivers will be enabled automatically if the required dependencies
are found. There is no meson build option to switch to using 16B
descriptors in the i40e driver, since we are really trying to limit the
build-time configurability in meson. However, you should be able to enable
it by setting "-DRTE_LIBRTE_I40E_16BYTE_RX_DESC" in CFLAGS before calling
meson.

Re: [dpdk-dev] [PATCH] ethdev: fix dpdk gcc build on Arm

[Juraj Linkeš]

> -----Original Message-----
> From: Bruce Richardson <bruce.richardson@intel.com>
> Sent: Thursday, June 4, 2020 3:42 PM
> To: Juraj Linkeš <juraj.linkes@pantheon.tech>
> Cc: Ferruh Yigit <ferruh.yigit@intel.com>; Thomas Monjalon
> <thomas@monjalon.net>; arybchenko@solarflare.com; dev@dpdk.org;
> Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>
> Subject: Re: [dpdk-dev] [PATCH] ethdev: fix dpdk gcc build on Arm
> 
> On Thu, Jun 04, 2020 at 12:55:40PM +0000, Juraj Linkeš wrote:
> > > -----Original Message-----
> > > From: Ferruh Yigit <ferruh.yigit@intel.com>
> > > Sent: Thursday, June 4, 2020 2:23 PM
> > > To: Juraj Linkeš <juraj.linkes@pantheon.tech>; Thomas Monjalon
> > > <thomas@monjalon.net>
> > > Cc: arybchenko@solarflare.com; dev@dpdk.org; Honnappa Nagarahalli
> > > <Honnappa.Nagarahalli@arm.com>
> > > Subject: Re: [PATCH] ethdev: fix dpdk gcc build on Arm
> > >
> > > On 6/4/2020 11:36 AM, Juraj Linkeš wrote:
> > > >
> > > >
> > > >> -----Original Message-----
> > > >> From: Ferruh Yigit <ferruh.yigit@intel.com>
> > > >> Sent: Wednesday, June 3, 2020 1:41 PM
> > > >> To: Thomas Monjalon <thomas@monjalon.net>; Juraj Linkeš
> > > >> <juraj.linkes@pantheon.tech>
> > > >> Cc: arybchenko@solarflare.com; dev@dpdk.org
> > > >> Subject: Re: [PATCH] ethdev: fix dpdk gcc build on Arm
> > > >>
> > > >> On 6/3/2020 11:16 AM, Thomas Monjalon wrote:
> > > >>> 03/06/2020 11:48, Juraj Linkeš:
> > > >>>> Directive #include <file> in gcc implementation searches for
> > > >>>> files in a standard list of system directories, which leads to
> > > >>>> a sporadici build error on Taishan arm machines:
> > > >>>> /tmp/openvpp-testing/dpdk/lib/librte_ethdev/rte_ethdev.h:4287:10:
> > > >>>> fatal error: rte_ethdev_core.h:
> > > >>>> No such file or directory #include <rte_ethdev_core.h>
> > > >>>
> > > >>> Would be interesting to know why nobody else hit such error?
> > > >>
> > > >>
> > > >> I can't see why this is happening, in the 'mk/rte.lib.mk' we have following:
> > > >>
> > > >> "
> > > >> install: _preinstall build _postinstall
> > > >> build: _preinstall
> > > >> "
> > > >>
> > > >> Which should cause the library header files installed before
> > > >> building .c files in that library.
> > > >> So when compiling 'rte_class_eth.c', the header files should be
> > > >> already in install folder.
> > > >>
> > > >>
> > > >> I can see how/why changing to "" fixes the issue but I am not
> > > >> sure about this
> > > fix.
> > > >> "rte_ethdev.h" is a public header file, that applications will
> > > >> include it in their applications. In the public library it is
> > > >> more proper to have other includes from system folder, using format <>.
> > > >> Again, I can't see why it is failing but I believe we should find
> > > >> another solution for _internal_ build error.
> > > >>
> > > >>
> > > >> A very simple solution can be following, but that is also not
> > > >> good, since it solves the issue by creating a dependency to the
> > > >> order of the header
> > > includes:
> > > >>  diff --git a/lib/librte_ethdev/rte_class_eth.c
> > > >> b/lib/librte_ethdev/rte_class_eth.c
> > > >>  index 6338355e25..3030c49020 100644
> > > >>  --- a/lib/librte_ethdev/rte_class_eth.c
> > > >>  +++ b/lib/librte_ethdev/rte_class_eth.c
> > > >>  @@ -10,8 +10,8 @@
> > > >>   #include <rte_kvargs.h>
> > > >>   #include <rte_log.h>
> > > >>
> > > >>  -#include "rte_ethdev.h"
> > > >>   #include "rte_ethdev_core.h"
> > > >>  +#include "rte_ethdev.h"
> > > >>   #include "rte_ethdev_driver.h"
> > > >>   #include "ethdev_private.h"
> > > >>
> > > >
> > > > Thomas, Ferruh, what should be the solution? I'm not an expert on
> > > > this and I
> > > can't really offer anything better, but I'd like that this gets fixed.
> > >
> > > First we need to root cause this before trying to solve it. Honnappa
> > > also said he can reproduce this but our CI builds can't (we are
> > > talking about tens of builds daily on various platforms), need to understand
> why.
> > > Also from Makefile I can't see how this is happening, I am feeling
> > > uneasy to fix something before figuring out how/why it is failing.
> > >
> > > Can you please try to collect more data on when/how this happens,
> > > initial questions I can think of:
> > > - Can you reproduce this with meson build?
> > > - Is it bare DPDK build, or build part of other project (I guess I
> > > saw fd.io on the
> > > link)
> > > - - If this is not bare DPDK build what changes has been done to build system?
> > > - Do you see this with fresh build (new clone) or rebuild of existing clone?
> > > - Can you confirm you have correct RTE_SDK and RTE_TARGET
> > > environment variables?
> > > - Can you please share your build command?
> > >
> >
> > I sent an e-mail to dpdk dev a few days back asking for help where I outlined
> what we're doing:
> > We're not doing anything special, just downloading and extracting the archive,
> then setting CONFIG_RTE_LIBRTE_MLX5_PMD and
> CONFIG_RTE_LIBRTE_I40E_16BYTE_RX_DESC to y in config/common_base and
> then running make install T=arm64-armv8a-linuxapp-gcc -j. As mentioned in the
> subject, the build server is a Taishan ARM server.
> >
> > We're doing a fresh rebuild everytime. The error doesn't happen everytime,
> just sometimes - it seems to be random.
> >
> > We don't set RTE_SDK nor RTE_TARGET since
> https://doc.dpdk.org/guides/linux_gsg/build_dpdk.html#installation-of-dpdk-
> target-environment-using-make doesn't mention those.
> >
> > I'll try Meson build a few times. How can I enable those two config options in
> Meson?
> >
> 
> The MLX drivers will be enabled automatically if the required dependencies are
> found. There is no meson build option to switch to using 16B descriptors in the
> i40e driver, since we are really trying to limit the build-time configurability in
> meson. However, you should be able to enable it by setting "-
> DRTE_LIBRTE_I40E_16BYTE_RX_DESC" in CFLAGS before calling meson.

Thanks Bruce, I'll try the passing the cflag. Seems to be working without it and I'm curious to see whether this will have any impact.

Re: [dpdk-dev] [PATCH] ethdev: fix dpdk gcc build on Arm

[Bruce Richardson]

On Fri, Jun 05, 2020 at 12:00:56PM +0000, Juraj Linkeš wrote:
> 
> 
> > -----Original Message-----
> > From: Bruce Richardson <bruce.richardson@intel.com>
> > Sent: Thursday, June 4, 2020 3:42 PM
> > To: Juraj Linkeš <juraj.linkes@pantheon.tech>
> > Cc: Ferruh Yigit <ferruh.yigit@intel.com>; Thomas Monjalon
> > <thomas@monjalon.net>; arybchenko@solarflare.com; dev@dpdk.org;
> > Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>
> > Subject: Re: [dpdk-dev] [PATCH] ethdev: fix dpdk gcc build on Arm
> > 
> > On Thu, Jun 04, 2020 at 12:55:40PM +0000, Juraj Linkeš wrote:
> > > > -----Original Message-----
> > > > From: Ferruh Yigit <ferruh.yigit@intel.com>
> > > > Sent: Thursday, June 4, 2020 2:23 PM
> > > > To: Juraj Linkeš <juraj.linkes@pantheon.tech>; Thomas Monjalon
> > > > <thomas@monjalon.net>
> > > > Cc: arybchenko@solarflare.com; dev@dpdk.org; Honnappa Nagarahalli
> > > > <Honnappa.Nagarahalli@arm.com>
> > > > Subject: Re: [PATCH] ethdev: fix dpdk gcc build on Arm
> > > >
> > > > On 6/4/2020 11:36 AM, Juraj Linkeš wrote:
> > > > >
> > > > >
> > > > >> -----Original Message-----
> > > > >> From: Ferruh Yigit <ferruh.yigit@intel.com>
> > > > >> Sent: Wednesday, June 3, 2020 1:41 PM
> > > > >> To: Thomas Monjalon <thomas@monjalon.net>; Juraj Linkeš
> > > > >> <juraj.linkes@pantheon.tech>
> > > > >> Cc: arybchenko@solarflare.com; dev@dpdk.org
> > > > >> Subject: Re: [PATCH] ethdev: fix dpdk gcc build on Arm
> > > > >>
> > > > >> On 6/3/2020 11:16 AM, Thomas Monjalon wrote:
> > > > >>> 03/06/2020 11:48, Juraj Linkeš:
> > > > >>>> Directive #include <file> in gcc implementation searches for
> > > > >>>> files in a standard list of system directories, which leads to
> > > > >>>> a sporadici build error on Taishan arm machines:
> > > > >>>> /tmp/openvpp-testing/dpdk/lib/librte_ethdev/rte_ethdev.h:4287:10:
> > > > >>>> fatal error: rte_ethdev_core.h:
> > > > >>>> No such file or directory #include <rte_ethdev_core.h>
> > > > >>>
> > > > >>> Would be interesting to know why nobody else hit such error?
> > > > >>
> > > > >>
> > > > >> I can't see why this is happening, in the 'mk/rte.lib.mk' we have following:
> > > > >>
> > > > >> "
> > > > >> install: _preinstall build _postinstall
> > > > >> build: _preinstall
> > > > >> "
> > > > >>
> > > > >> Which should cause the library header files installed before
> > > > >> building .c files in that library.
> > > > >> So when compiling 'rte_class_eth.c', the header files should be
> > > > >> already in install folder.
> > > > >>
> > > > >>
> > > > >> I can see how/why changing to "" fixes the issue but I am not
> > > > >> sure about this
> > > > fix.
> > > > >> "rte_ethdev.h" is a public header file, that applications will
> > > > >> include it in their applications. In the public library it is
> > > > >> more proper to have other includes from system folder, using format <>.
> > > > >> Again, I can't see why it is failing but I believe we should find
> > > > >> another solution for _internal_ build error.
> > > > >>
> > > > >>
> > > > >> A very simple solution can be following, but that is also not
> > > > >> good, since it solves the issue by creating a dependency to the
> > > > >> order of the header
> > > > includes:
> > > > >>  diff --git a/lib/librte_ethdev/rte_class_eth.c
> > > > >> b/lib/librte_ethdev/rte_class_eth.c
> > > > >>  index 6338355e25..3030c49020 100644
> > > > >>  --- a/lib/librte_ethdev/rte_class_eth.c
> > > > >>  +++ b/lib/librte_ethdev/rte_class_eth.c
> > > > >>  @@ -10,8 +10,8 @@
> > > > >>   #include <rte_kvargs.h>
> > > > >>   #include <rte_log.h>
> > > > >>
> > > > >>  -#include "rte_ethdev.h"
> > > > >>   #include "rte_ethdev_core.h"
> > > > >>  +#include "rte_ethdev.h"
> > > > >>   #include "rte_ethdev_driver.h"
> > > > >>   #include "ethdev_private.h"
> > > > >>
> > > > >
> > > > > Thomas, Ferruh, what should be the solution? I'm not an expert on
> > > > > this and I
> > > > can't really offer anything better, but I'd like that this gets fixed.
> > > >
> > > > First we need to root cause this before trying to solve it. Honnappa
> > > > also said he can reproduce this but our CI builds can't (we are
> > > > talking about tens of builds daily on various platforms), need to understand
> > why.
> > > > Also from Makefile I can't see how this is happening, I am feeling
> > > > uneasy to fix something before figuring out how/why it is failing.
> > > >
> > > > Can you please try to collect more data on when/how this happens,
> > > > initial questions I can think of:
> > > > - Can you reproduce this with meson build?
> > > > - Is it bare DPDK build, or build part of other project (I guess I
> > > > saw fd.io on the
> > > > link)
> > > > - - If this is not bare DPDK build what changes has been done to build system?
> > > > - Do you see this with fresh build (new clone) or rebuild of existing clone?
> > > > - Can you confirm you have correct RTE_SDK and RTE_TARGET
> > > > environment variables?
> > > > - Can you please share your build command?
> > > >
> > >
> > > I sent an e-mail to dpdk dev a few days back asking for help where I outlined
> > what we're doing:
> > > We're not doing anything special, just downloading and extracting the archive,
> > then setting CONFIG_RTE_LIBRTE_MLX5_PMD and
> > CONFIG_RTE_LIBRTE_I40E_16BYTE_RX_DESC to y in config/common_base and
> > then running make install T=arm64-armv8a-linuxapp-gcc -j. As mentioned in the
> > subject, the build server is a Taishan ARM server.
> > >
> > > We're doing a fresh rebuild everytime. The error doesn't happen everytime,
> > just sometimes - it seems to be random.
> > >
> > > We don't set RTE_SDK nor RTE_TARGET since
> > https://doc.dpdk.org/guides/linux_gsg/build_dpdk.html#installation-of-dpdk-
> > target-environment-using-make doesn't mention those.
> > >
> > > I'll try Meson build a few times. How can I enable those two config options in
> > Meson?
> > >
> > 
> > The MLX drivers will be enabled automatically if the required dependencies are
> > found. There is no meson build option to switch to using 16B descriptors in the
> > i40e driver, since we are really trying to limit the build-time configurability in
> > meson. However, you should be able to enable it by setting "-
> > DRTE_LIBRTE_I40E_16BYTE_RX_DESC" in CFLAGS before calling meson.
> 
> Thanks Bruce, I'll try the passing the cflag. Seems to be working without it and I'm curious to see whether this will have any impact.

Yes, functionally everything should be the same with or without the 16B
descriptor flag, using the smaller descriptors can just save a small amount
of PCI bandwidth and give a smaller cache footprint. For interests sake,
please let us know if you see any perf difference in your app with or
without the setting.

Thanks,
/Bruce

Re: [dpdk-dev] [PATCH] ethdev: fix dpdk gcc build on Arm

[Honnappa Nagarahalli]

<snip>

> 
> 03/06/2020 11:48, Juraj Linkeš:
> > Directive #include <file> in gcc implementation searches for files in
> > a standard list of system directories, which leads to a sporadici
> > build error on Taishan arm machines:
> > /tmp/openvpp-testing/dpdk/lib/librte_ethdev/rte_ethdev.h:4287:10:
> > fatal error: rte_ethdev_core.h:
> > No such file or directory #include <rte_ethdev_core.h>
> 
> Would be interesting to know why nobody else hit such error?
I have hit this multiple times (after a fresh config) on other Arm machines, subsequent run of the compilation would fix this. Thanks for fixing.

> 
> 
> > Fix this by replacing the directive with #include "file" which
> > searches for files first in the directory containing the current file
> > and other directories only after that.
> >
> > Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
> 
> What about #include <rte_ethdev_trace.h> ?
> 
> Except this,
> Acked-by: Thomas Monjalon <thomas@monjalon.net>
> 
> 
> I would change the title to be more generic (nothing specific to gcc or Arm
> here):
> 	ethdev: fix local include
> 
> It needs to be backported, so these lines are required:
> 	Fixes: 7f0a669e7b04 ("ethdev: add allocation helper for virtual
> drivers")
> 	Fixes: dcd5c8112bc3 ("ethdev: add PCI driver helpers")
> 	Fixes: ffc905f3b856 ("ethdev: separate driver APIs")
> 	Fixes: 331c447ad913 ("ethdev: separate internal structures into own
> header")
> 	Cc: stable@dpdk.org
> 
>