We have a public Coverity scan triggered by John for the community: https://scan.coverity.com/projects/dpdk-data-plane-development-kit Note there is a tool to help with this task: http://thyrsus.com/gitweb/?p=coverity-submit.git;a=shortlog;h=refs/tags/1.13 I see two issues with this scan: - it is run manually - not all code is scanned currently Note that we should be able to run one scan per day for free: https://scan.coverity.com/faq#frequency With David, we looked at automating the Coverity scan, with the help of Travis automation: https://scan.coverity.com/travis_ci Such automation cannot be configured on the existing Coverity project. I tried to open a new Coverity project connected to our GitHub. I have a very poor confidence in Coverity/Travis/GitHub integration. I will explain below why. 1/ The instructions were wrong. In this command, there are two mistakes: openssl s_client -connect https://scan.coverity.com:443 | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' | sudo tee -a /etc/ssl/certs/ca- For the record, a proper a simpler command is: true | openssl s_client -connect scan.coverity.com:443 | openssl x509 | sudo tee -a /etc/ssl/certs/ca-certificates.crt 2/ The coverity scan is triggered as a job addon. The rest of the job must be cancelled with this tricky patch: -script: ./.ci/${TRAVIS_OS_NAME}-build.sh +script: if [ "${COVERITY_SCAN_BRANCH}" != 1 ] ; then ./.ci/${TRAVIS_OS_NAME}-build.sh ; fi 3/ We need only to prepare the source code once per day. But our .travis.yml has many jobs which must be dropped or ignored. 4/ A big encrypted token must be added in the configuration: # encrypted COVERITY_SCAN_TOKEN - secure: "VgRYG9N5adKkM9/QpPgswn1c+VXS1mFVN0vgdjuC/bDv2x4u...etc..." 5/ The addon is triggered when pushing to a specific branch (adding config for the record): coverity_scan: project: name: "DPDK/dpdk" notification_email: test-report@dpdk.org build_command_prepend: "meson build -Dexamples=all" build_command: "ninja -C build" branch_pattern: coverity_scan 6/ This attempt failed with this log (no more information): $ export PROJECT_NAME=DPDK/dpdk Coverity Scan analysis selected for branch coverity_scan. Coverity Scan API access denied. Check $PROJECT_NAME and $COVERITY_SCAN_TOKEN. So I am giving up with Travis+Coverity. The only benefit of Travis is to have a central build configuration. So when a driver is enabled in Travis, it would be scanned in Coverity. Note: Coverity does a build step to prepare the sources. Now the question: how can we better configure the community Coverity scan? I propose to set it up in our community lab. Comments? Suggestions?
Thomas Monjalon <thomas@monjalon.net> writes: > We have a public Coverity scan triggered by John for the community: > https://scan.coverity.com/projects/dpdk-data-plane-development-kit > Note there is a tool to help with this task: > http://thyrsus.com/gitweb/?p=coverity-submit.git;a=shortlog;h=refs/tags/1.13 > > I see two issues with this scan: > - it is run manually > - not all code is scanned currently > > Note that we should be able to run one scan per day for free: > https://scan.coverity.com/faq#frequency > > With David, we looked at automating the Coverity scan, > with the help of Travis automation: > https://scan.coverity.com/travis_ci > Such automation cannot be configured on the existing Coverity project. Why not? > I tried to open a new Coverity project connected to our GitHub. I don't know that it will work. Either you'll need a separate GitHub, or you'll need to use a special branch. > I have a very poor confidence in Coverity/Travis/GitHub integration. > I will explain below why. Hrrm.. lots of projects use it. And they do just what you prescribe below (skipping jobs/builds when on the coverity branch). > 1/ The instructions were wrong. In this command, there are two mistakes: > openssl s_client -connect https://scan.coverity.com:443 | > sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' | > sudo tee -a /etc/ssl/certs/ca- > For the record, a proper a simpler command is: > true | openssl s_client -connect scan.coverity.com:443 | > openssl x509 | > sudo tee -a /etc/ssl/certs/ca-certificates.crt Okay, that's fixable. > 2/ The coverity scan is triggered as a job addon. > The rest of the job must be cancelled with this tricky patch: > > -script: ./.ci/${TRAVIS_OS_NAME}-build.sh > +script: if [ "${COVERITY_SCAN_BRANCH}" != 1 ] ; then ./.ci/${TRAVIS_OS_NAME}-build.sh ; fi More than that, because we probably also want: if ([[ "${TRAVIS_JOB_NUMBER##*.}" == "1" ]] && [[ "${TRAVIS_BRANCH}" == "coverity_scan" ]]); then ./.ci/${TRAVIS_OS_NAME}-build.sh ; fi That will only do one job (which solves 3/ below) > 3/ We need only to prepare the source code once per day. > But our .travis.yml has many jobs which must be dropped or ignored. > > 4/ A big encrypted token must be added in the configuration: > # encrypted COVERITY_SCAN_TOKEN > - secure: "VgRYG9N5adKkM9/QpPgswn1c+VXS1mFVN0vgdjuC/bDv2x4u...etc..." Why it's a problem? > 5/ The addon is triggered when pushing to a specific branch > (adding config for the record): > coverity_scan: > project: > name: "DPDK/dpdk" > notification_email: test-report@dpdk.org > build_command_prepend: "meson build -Dexamples=all" > build_command: "ninja -C build" > branch_pattern: coverity_scan > > 6/ This attempt failed with this log (no more information): > $ export PROJECT_NAME=DPDK/dpdk > Coverity Scan analysis selected for branch coverity_scan. > Coverity Scan API access denied. Check $PROJECT_NAME and $COVERITY_SCAN_TOKEN. Probably there is an issue with the token + PROJECT_NAME. > > So I am giving up with Travis+Coverity. > The only benefit of Travis is to have a central build configuration. > So when a driver is enabled in Travis, it would be scanned in Coverity. > Note: Coverity does a build step to prepare the sources. I can try to assist with this if you've not completely abandoned the idea. > > Now the question: how can we better configure the community Coverity scan? > I propose to set it up in our community lab. > Comments? Suggestions? Since we do have something working, but it's manual, is there a way to at least make it happen automatically? Maybe some cron job?
11/03/2020 18:34, Aaron Conole: > Thomas Monjalon <thomas@monjalon.net> writes: > > > We have a public Coverity scan triggered by John for the community: > > https://scan.coverity.com/projects/dpdk-data-plane-development-kit > > Note there is a tool to help with this task: > > http://thyrsus.com/gitweb/?p=coverity-submit.git;a=shortlog;h=refs/tags/1.13 > > > > I see two issues with this scan: > > - it is run manually > > - not all code is scanned currently > > > > Note that we should be able to run one scan per day for free: > > https://scan.coverity.com/faq#frequency > > > > With David, we looked at automating the Coverity scan, > > with the help of Travis automation: > > https://scan.coverity.com/travis_ci > > Such automation cannot be configured on the existing Coverity project. > > Why not? Because Coverity does not allow it. Travis integration is possible only if the project was created with GitHub credentials. > > I tried to open a new Coverity project connected to our GitHub. > > I don't know that it will work. Either you'll need a separate GitHub, > or you'll need to use a special branch. > > > I have a very poor confidence in Coverity/Travis/GitHub integration. > > I will explain below why. > > Hrrm.. lots of projects use it. And they do just what you prescribe > below (skipping jobs/builds when on the coverity branch). Which project is using Travis integration of Coverity? How do they automatically update the specific branch without conflict? > > 1/ The instructions were wrong. In this command, there are two mistakes: > > openssl s_client -connect https://scan.coverity.com:443 | > > sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' | > > sudo tee -a /etc/ssl/certs/ca- > > For the record, a proper a simpler command is: > > true | openssl s_client -connect scan.coverity.com:443 | > > openssl x509 | > > sudo tee -a /etc/ssl/certs/ca-certificates.crt > > Okay, that's fixable. > > > 2/ The coverity scan is triggered as a job addon. > > The rest of the job must be cancelled with this tricky patch: > > > > -script: ./.ci/${TRAVIS_OS_NAME}-build.sh > > +script: if [ "${COVERITY_SCAN_BRANCH}" != 1 ] ; then ./.ci/${TRAVIS_OS_NAME}-build.sh ; fi > > More than that, because we probably also want: > > if ([[ "${TRAVIS_JOB_NUMBER##*.}" == "1" ]] && [[ "${TRAVIS_BRANCH}" == "coverity_scan" ]]); then ./.ci/${TRAVIS_OS_NAME}-build.sh ; fi > > That will only do one job (which solves 3/ below) OK good > > 3/ We need only to prepare the source code once per day. > > But our .travis.yml has many jobs which must be dropped or ignored. > > > > 4/ A big encrypted token must be added in the configuration: > > # encrypted COVERITY_SCAN_TOKEN > > - secure: "VgRYG9N5adKkM9/QpPgswn1c+VXS1mFVN0vgdjuC/bDv2x4u...etc..." > > Why it's a problem? It's not a problem. I explained all steps in this email, that's why it's here. > > 5/ The addon is triggered when pushing to a specific branch > > (adding config for the record): > > coverity_scan: > > project: > > name: "DPDK/dpdk" > > notification_email: test-report@dpdk.org > > build_command_prepend: "meson build -Dexamples=all" > > build_command: "ninja -C build" > > branch_pattern: coverity_scan > > > > 6/ This attempt failed with this log (no more information): > > $ export PROJECT_NAME=DPDK/dpdk > > Coverity Scan analysis selected for branch coverity_scan. > > Coverity Scan API access denied. Check $PROJECT_NAME and $COVERITY_SCAN_TOKEN. > > Probably there is an issue with the token + PROJECT_NAME. Probably. How can I debug it? > > So I am giving up with Travis+Coverity. > > The only benefit of Travis is to have a central build configuration. > > So when a driver is enabled in Travis, it would be scanned in Coverity. > > Note: Coverity does a build step to prepare the sources. > > I can try to assist with this if you've not completely abandoned the idea. OK, feel free to ping me for troubleshooting. > > Now the question: how can we better configure the community Coverity scan? > > I propose to set it up in our community lab. > > Comments? Suggestions? > > Since we do have something working, but it's manual, is there a way to > at least make it happen automatically? Maybe some cron job? Yes a cron job, but where? I proposed a server of the community lab.
Thomas Monjalon <thomas@monjalon.net> writes: > 11/03/2020 18:34, Aaron Conole: >> Thomas Monjalon <thomas@monjalon.net> writes: >> >> > We have a public Coverity scan triggered by John for the community: >> > https://scan.coverity.com/projects/dpdk-data-plane-development-kit >> > Note there is a tool to help with this task: >> > http://thyrsus.com/gitweb/?p=coverity-submit.git;a=shortlog;h=refs/tags/1.13 >> > >> > I see two issues with this scan: >> > - it is run manually >> > - not all code is scanned currently >> > >> > Note that we should be able to run one scan per day for free: >> > https://scan.coverity.com/faq#frequency >> > >> > With David, we looked at automating the Coverity scan, >> > with the help of Travis automation: >> > https://scan.coverity.com/travis_ci >> > Such automation cannot be configured on the existing Coverity project. >> >> Why not? > > Because Coverity does not allow it. > Travis integration is possible only if the project was created with GitHub credentials. > > >> > I tried to open a new Coverity project connected to our GitHub. >> >> I don't know that it will work. Either you'll need a separate GitHub, >> or you'll need to use a special branch. >> >> > I have a very poor confidence in Coverity/Travis/GitHub integration. >> > I will explain below why. >> >> Hrrm.. lots of projects use it. And they do just what you prescribe >> below (skipping jobs/builds when on the coverity branch). > > Which project is using Travis integration of Coverity? > How do they automatically update the specific branch without conflict? RabbitMQ-c looked to be doing it: https://github.com/rabbitmq/rabbitmq-c redirects to: https://github.com/alanxz/rabbitmq-c/blob/master/.travis.yml Looks like they haven't done a push in a while, though. :-/ That project also uses coveralls (something I want to integrate the DPDK project with). > >> > 1/ The instructions were wrong. In this command, there are two mistakes: >> > openssl s_client -connect https://scan.coverity.com:443 | >> > sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' | >> > sudo tee -a /etc/ssl/certs/ca- >> > For the record, a proper a simpler command is: >> > true | openssl s_client -connect scan.coverity.com:443 | >> > openssl x509 | >> > sudo tee -a /etc/ssl/certs/ca-certificates.crt >> >> Okay, that's fixable. >> >> > 2/ The coverity scan is triggered as a job addon. >> > The rest of the job must be cancelled with this tricky patch: >> > >> > -script: ./.ci/${TRAVIS_OS_NAME}-build.sh >> > +script: if [ "${COVERITY_SCAN_BRANCH}" != 1 ] ; then ./.ci/${TRAVIS_OS_NAME}-build.sh ; fi >> >> More than that, because we probably also want: >> >> if ([[ "${TRAVIS_JOB_NUMBER##*.}" == "1" ]] && [[ "${TRAVIS_BRANCH}" >> == "coverity_scan" ]]); then ./.ci/${TRAVIS_OS_NAME}-build.sh ; fi >> >> That will only do one job (which solves 3/ below) > > OK good > >> > 3/ We need only to prepare the source code once per day. >> > But our .travis.yml has many jobs which must be dropped or ignored. >> > >> > 4/ A big encrypted token must be added in the configuration: >> > # encrypted COVERITY_SCAN_TOKEN >> > - secure: "VgRYG9N5adKkM9/QpPgswn1c+VXS1mFVN0vgdjuC/bDv2x4u...etc..." >> >> Why it's a problem? > > It's not a problem. I explained all steps in this email, that's why it's here. Ahh, okay. > >> > 5/ The addon is triggered when pushing to a specific branch >> > (adding config for the record): >> > coverity_scan: >> > project: >> > name: "DPDK/dpdk" >> > notification_email: test-report@dpdk.org >> > build_command_prepend: "meson build -Dexamples=all" >> > build_command: "ninja -C build" >> > branch_pattern: coverity_scan >> > >> > 6/ This attempt failed with this log (no more information): >> > $ export PROJECT_NAME=DPDK/dpdk >> > Coverity Scan analysis selected for branch coverity_scan. >> > Coverity Scan API access denied. Check $PROJECT_NAME and $COVERITY_SCAN_TOKEN. >> >> Probably there is an issue with the token + PROJECT_NAME. > > Probably. How can I debug it? Need to go through the steps at (which you probably already did): https://docs.travis-ci.com/user/coverity-scan/ And make sure the account that has access to the travis repo also has access to the coverity scan project. The nice thing on that page is step 8 will have the blocks needed for the security token (that needs to go in env: global: - secure=...) and the project settings. I will ping you offline, though. > >> > So I am giving up with Travis+Coverity. >> > The only benefit of Travis is to have a central build configuration. >> > So when a driver is enabled in Travis, it would be scanned in Coverity. >> > Note: Coverity does a build step to prepare the sources. >> >> I can try to assist with this if you've not completely abandoned the idea. > > OK, feel free to ping me for troubleshooting. Will do. > >> > Now the question: how can we better configure the community Coverity scan? >> > I propose to set it up in our community lab. >> > Comments? Suggestions? >> >> Since we do have something working, but it's manual, is there a way to >> at least make it happen automatically? Maybe some cron job? > > Yes a cron job, but where? I proposed a server of the community lab. If it's just a matter of pushing something (or even a few small steps), we can add it to the robot's sunday "master" rebuild. Just need whatever credentials.