[dpdk-dev] Clang Scan build results

[dpdk-dev] Clang Scan build results

[Wiles, Roger Keith]

Hi Everyone,

I built dpdk with Clang and used the scan-build analyzer to produce a report. The report is about 13M in size so not very nice to send to the list. I decided to place the report on github.com if you want to see the results.

While running scan-build the build would fail, but I forced the build to continue using the scan-build option to get as much of the report as possible. I did not have time yet to understand why the build stopped and I hope to dig into it more later.

Running scan-build is pretty simple, so would be a nice test report if you wanted to add it to dpdk.org site and maintain its output for review. It would be nice to run once in a while to weed out any basic problems. We could run something like PC-Lint or Coverity, but they cost money :-)

Here is the link to my Pktgen-DPDK repos on github:

# git clone git://github.com/Pktgen/dpdk-scan-build-results.git

Let me know if you have any questions or suggestions.

Thanks
++Keith


Keith Wiles, Principal Technologist with CTO office, Wind River mobile 972-213-5533

Re: [dpdk-dev] Clang Scan build results

[Jay Rolette]

*> We could run something like PC-Lint or Coverity, but they cost money :-)*
Pretty sure Coverity is free for open source projects...

Jay

On Wed, Aug 27, 2014 at 10:13 AM, Wiles, Roger Keith <
keith.wiles@windriver.com> wrote:

> Hi Everyone,
>
> I built dpdk with Clang and used the scan-build analyzer to produce a
> report. The report is about 13M in size so not very nice to send to the
> list. I decided to place the report on github.com if you want to see the
> results.
>
> While running scan-build the build would fail, but I forced the build to
> continue using the scan-build option to get as much of the report as
> possible. I did not have time yet to understand why the build stopped and I
> hope to dig into it more later.
>
> Running scan-build is pretty simple, so would be a nice test report if you
> wanted to add it to dpdk.org site and maintain its output for review. It
> would be nice to run once in a while to weed out any basic problems. We
> could run something like PC-Lint or Coverity, but they cost money :-)
>
> Here is the link to my Pktgen-DPDK repos on github:
>
> # git clone git://github.com/Pktgen/dpdk-scan-build-results.git
>
> Let me know if you have any questions or suggestions.
>
> Thanks
> ++Keith
>
>
> Keith Wiles, Principal Technologist with CTO office, Wind River mobile
> 972-213-5533
>
>

Re: [dpdk-dev] Clang Scan build results

[Wiles, Roger Keith]

Nice, we had to buy one and that was not cheap :-) I groped around on the Coverity site and did not find any statement about being free to open source, but I may have just missed it.

I did find that PC-Lint, Coverity, scan-build, … all seem to test different parts of your code and some are better then others. Some have a lot false positives and some report huge number of issues, but it just depends on the type and level of scan you want. One thing I found was you need to run different tools to find different problems as none of them do everything IMO.

On Aug 27, 2014, at 10:24 AM, Jay Rolette <rolette@infiniteio.com<mailto:rolette@infiniteio.com>> wrote:

> We could run something like PC-Lint or Coverity, but they cost money :-)
Pretty sure Coverity is free for open source projects...

Jay

On Wed, Aug 27, 2014 at 10:13 AM, Wiles, Roger Keith <keith.wiles@windriver.com<mailto:keith.wiles@windriver.com>> wrote:
Hi Everyone,

I built dpdk with Clang and used the scan-build analyzer to produce a report. The report is about 13M in size so not very nice to send to the list. I decided to place the report on github.com<http://github.com/> if you want to see the results.

While running scan-build the build would fail, but I forced the build to continue using the scan-build option to get as much of the report as possible. I did not have time yet to understand why the build stopped and I hope to dig into it more later.

Running scan-build is pretty simple, so would be a nice test report if you wanted to add it to dpdk.org<http://dpdk.org/> site and maintain its output for review. It would be nice to run once in a while to weed out any basic problems. We could run something like PC-Lint or Coverity, but they cost money :-)

Here is the link to my Pktgen-DPDK repos on github:

# git clone git://github.com/Pktgen/dpdk-scan-build-results.git<http://github.com/Pktgen/dpdk-scan-build-results.git>

Let me know if you have any questions or suggestions.

Thanks
++Keith


Keith Wiles, Principal Technologist with CTO office, Wind River mobile 972-213-5533<tel:972-213-5533>



Keith Wiles, Principal Technologist with CTO office, Wind River mobile 972-213-5533

Re: [dpdk-dev] Clang Scan build results

[Jay Rolette]

Here's the link you want: https://scan.coverity.com/

Check the FAQ for terms and you'll need to register the product, but I
didn't see anything obvious that should get in the way from being able to
use it for DPDK.

Agree with Coverity not being cheap. I bought it for my last company.
Really liked that Coverity had a much higher signal-to-noise ratio than
other tools at the time. Possibly the gap has closed since then, but you
know it can't be as bad as lint!


On Wed, Aug 27, 2014 at 10:52 AM, Wiles, Roger Keith <
keith.wiles@windriver.com> wrote:

>  Nice, we had to buy one and that was not cheap :-) I groped around on the
> Coverity site and did not find any statement about being free to open
> source, but I may have just missed it.
>
>  I did find that PC-Lint, Coverity, scan-build, … all seem to test
> different parts of your code and some are better then others. Some have a
> lot false positives and some report huge number of issues, but it just
> depends on the type and level of scan you want. One thing I found was you
> need to run different tools to find different problems as none of them do
> everything IMO.
>
>  On Aug 27, 2014, at 10:24 AM, Jay Rolette <rolette@infiniteio.com> wrote:
>
>  *> We could run something like PC-Lint or Coverity, but they cost money
> :-)*
> Pretty sure Coverity is free for open source projects...
>
>  Jay
>
> On Wed, Aug 27, 2014 at 10:13 AM, Wiles, Roger Keith <
> keith.wiles@windriver.com> wrote:
>
>> Hi Everyone,
>>
>> I built dpdk with Clang and used the scan-build analyzer to produce a
>> report. The report is about 13M in size so not very nice to send to the
>> list. I decided to place the report on github.com if you want to see the
>> results.
>>
>> While running scan-build the build would fail, but I forced the build to
>> continue using the scan-build option to get as much of the report as
>> possible. I did not have time yet to understand why the build stopped and I
>> hope to dig into it more later.
>>
>> Running scan-build is pretty simple, so would be a nice test report if
>> you wanted to add it to dpdk.org site and maintain its output for
>> review. It would be nice to run once in a while to weed out any basic
>> problems. We could run something like PC-Lint or Coverity, but they cost
>> money :-)
>>
>> Here is the link to my Pktgen-DPDK repos on github:
>>
>> # git clone git://github.com/Pktgen/dpdk-scan-build-results.git
>>
>> Let me know if you have any questions or suggestions.
>>
>> Thanks
>> ++Keith
>>
>>
>> Keith Wiles, Principal Technologist with CTO office, Wind River mobile
>> 972-213-5533
>>
>>
>
>  *Keith **Wiles*, Principal Technologist with CTO office, *Wind River *mobile
> 972-213-5533
>
>

Re: [dpdk-dev] Clang Scan build results

[Thomas Monjalon]

2014-08-27 15:52, Wiles, Roger Keith:
> I groped around on the Coverity site and did not find any statement about
> being free to open source, but I may have just missed it.

It's here:
	https://scan.coverity.com

Some checks have been done with TrustInSoft Analyzer:
	http://dpdk.org/ml/archives/dev/2014-May/002365.html
Noticeable example:
	http://dpdk.org/browse/dpdk/commit/?id=2612a4b935144accff

I didn't yet check all your clang report but about all the errors in
Intel base drivers, you should know that we prefer not fix them because
we try to keep it close from the Intel imports. It's Intel decision to fix
these base drivers errors and publish them here.

-- 
Thomas

Re: [dpdk-dev] Clang Scan build results

[Wiles, Roger Keith]

Not sure why cite marks are missing again :-( I hope they come back some day. I still must not have Mac Mail configure correctly, but I only really see a couple of options. If someone has a solution let me know.

On Aug 27, 2014, at 11:09 AM, Thomas Monjalon <thomas.monjalon@6wind.com> wrote:

> 2014-08-27 15:52, Wiles, Roger Keith:
>> I groped around on the Coverity site and did not find any statement about
>> being free to open source, but I may have just missed it.
> 
> It's here:
> 	https://scan.coverity.com
> 
> Some checks have been done with TrustInSoft Analyzer:
> 	http://dpdk.org/ml/archives/dev/2014-May/002365.html
> Noticeable example:
> 	http://dpdk.org/browse/dpdk/commit/?id=2612a4b935144accff

If coverity is free then maybe that is one we need to use instead of scan-build, but I still like running multiple tools to see which one does the best and to report other types of issues. Just my opinion.
> 
> I didn't yet check all your clang report but about all the errors in
> Intel base drivers, you should know that we prefer not fix them because
> we try to keep it close from the Intel imports. It's Intel decision to fix
> these base drivers errors and publish them here.

Yes, I noticed the drivers had a few problems and were one of the files I had to force scan-build work.
> 
> -- 
> Thomas

Keith Wiles, Principal Technologist with CTO office, Wind River mobile 972-213-5533

Re: [dpdk-dev] Clang Scan build results

[Vincent JARDIN]

> It's here:
>       https://scan.coverity.com
>
> Some checks have been done with TrustInSoft Analyzer:
>       http://dpdk.org/ml/archives/dev/2014-May/002365.html
> Noticeable example:
>       http://dpdk.org/browse/dpdk/commit/?id=2612a4b935144accff

Registered on coverity,  anyone for the admin? I need emails / names.

Re: [dpdk-dev] Clang Scan build results

[Julien Cretin]

Hi all!

I took a look at the scan-results from Keith to compare with my analysis. My analysis is on test-pmd (called with a specific command-line and with start tx_first at the prompt) and scans exhaustively for all undefined behaviors (overflow, division by zero, invalid pointer dereference or comparison, etc.). The result (on the last revision of master) is that test-pmd is free from these kind of bugs in the perimeter of the analysis. This result comes from the fact that TrustInSoft Analyzer relies on formal methods and is sound: it does not remain silent about an undefined behavior. Of course there are false positives, but in my test-pmd analysis it happens that there is no alarms at all (neither false nor true).

Although the scan-build report contains false-positives (I found some related to my analysis) and is not exhaustive (false-negatives), it scans more code than my analysis (I don't look at drivers since I use a generic driver to exhaustively emulate all network behaviors, and I don't look at app/test which seems to contain also a lot of alarms).

I will soon make my analysis public and I regularly replay it on new versions of the master branch. Anyone interested can already contact me for more information.

Cheers,
Julien

Re: [dpdk-dev] Clang Scan build results

[Matthew Hall]

On Wed, Aug 27, 2014 at 03:13:43PM +0000, Wiles, Roger Keith wrote:
> Hi Everyone,

Hi Keith,

For me the build failed with clang but I made a series of awful patches to get 
it to compile... not sure if the clang failures could be related to your 
scan-build failures. If it will help you I can provide you the patches I made 
to get it to work on clang... they are not ready for the DPDK master branch 
but it's really good to get safe output from scan-build.

> It would be nice to run once in a while to weed out any basic problems. We 
> could run something like PC-Lint or Coverity, but they cost money :-)

Not 100% true... you can run Coverity for free on open source if you are the 
maintainer... given Intel, Wind River, and 6WIND all have some form of 
maintainership authority over DPDK there should be a way to qualify via this 
avenue.

Matthew.

Re: [dpdk-dev] Clang Scan build results

[Wiles, Roger Keith]

Hi Matthew,

On Aug 27, 2014, at 10:00 PM, Matthew Hall <mhall@mhcomputing.net> wrote:

> On Wed, Aug 27, 2014 at 03:13:43PM +0000, Wiles, Roger Keith wrote:
>> Hi Everyone,
> 
> Hi Keith,
> 
> For me the build failed with clang but I made a series of awful patches to get 
> it to compile... not sure if the clang failures could be related to your 
> scan-build failures. If it will help you I can provide you the patches I made 
> to get it to work on clang... they are not ready for the DPDK master branch 
> but it's really good to get safe output from scan-build.

I was able to get DPDK to build using target x86_64-native-linuxapp-clang without errors, but with scan-build it gave me errors that were not in the normal clang build. Was this the target system you were building or something else?
> 
>> It would be nice to run once in a while to weed out any basic problems. We 
>> could run something like PC-Lint or Coverity, but they cost money :-)
> 
> Not 100% true... you can run Coverity for free on open source if you are the 
> maintainer... given Intel, Wind River, and 6WIND all have some form of 
> maintainership authority over DPDK there should be a way to qualify via this 
> avenue.

Yes, it was pointer out to me that Coverity is free to open source via scan.coverity.com site, just have to register and get accepted.
> 
> Matthew.

Keith Wiles, Principal Technologist with CTO office, Wind River mobile 972-213-5533