DPDK patches and discussions
 help / color / mirror / Atom feed
* [dpdk-dev] [PATCH 1/3] scripts: support parallel building in validate-abi.sh via -j[N] option
@ 2016-03-10 10:53 Panu Matilainen
  2016-03-10 10:53 ` [dpdk-dev] [PATCH 2/3] scripts: avoid editing defconfig_* files in validate-abi.sh Panu Matilainen
                   ` (2 more replies)
  0 siblings, 3 replies; 11+ messages in thread
From: Panu Matilainen @ 2016-03-10 10:53 UTC (permalink / raw)
  To: dev

Signed-off-by: Panu Matilainen <pmatilai@redhat.com>
---
 doc/guides/contributing/versioning.rst |  4 +++-
 scripts/validate-abi.sh                | 13 ++++++++++---
 2 files changed, 13 insertions(+), 4 deletions(-)

diff --git a/doc/guides/contributing/versioning.rst b/doc/guides/contributing/versioning.rst
index ae10a98..33b03a1 100644
--- a/doc/guides/contributing/versioning.rst
+++ b/doc/guides/contributing/versioning.rst
@@ -469,11 +469,13 @@ utilities which can be installed via a package manager. For example::
 
 The syntax of the ``validate-abi.sh`` utility is::
 
-   ./scripts/validate-abi.sh <REV1> <REV2> <TARGET>
+   ./scripts/validate-abi.sh [-j[N]] <REV1> <REV2> <TARGET>
 
 Where ``REV1`` and ``REV2`` are valid gitrevisions(7)
 https://www.kernel.org/pub/software/scm/git/docs/gitrevisions.html
 on the local repo and target is the usual DPDK compilation target.
+The optional -j[N] switch enables parallel building with at most
+N simultaneous jobs, ie the same as -j option of ``make``.
 
 For example:
 
diff --git a/scripts/validate-abi.sh b/scripts/validate-abi.sh
index c36ad61..f094582 100755
--- a/scripts/validate-abi.sh
+++ b/scripts/validate-abi.sh
@@ -27,13 +27,20 @@
 #   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
 #   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 
+case "$1" in
+    -j*)
+        MAKEJOBS="$1"
+        shift
+        ;;
+esac
+
 TAG1=$1
 TAG2=$2
 TARGET=$3
 ABI_DIR=`mktemp -d -p /tmp ABI.XXXXXX`
 
 usage() {
-	echo "$0 <REV1> <REV2> <TARGET>"
+	echo "$0 [-j[N]] <REV1> <REV2> <TARGET>"
 }
 
 log() {
@@ -183,7 +190,7 @@ log "INFO" "Configuring DPDK $TAG1"
 make config T=$TARGET O=$TARGET > $VERBOSE 2>&1
 
 log "INFO" "Building DPDK $TAG1. This might take a moment"
-make O=$TARGET > $VERBOSE 2>&1
+make $MAKEJOBS O=$TARGET > $VERBOSE 2>&1
 
 if [ $? -ne 0 ]
 then
@@ -214,7 +221,7 @@ log "INFO" "Configuring DPDK $TAG2"
 make config T=$TARGET O=$TARGET > $VERBOSE 2>&1
 
 log "INFO" "Building DPDK $TAG2. This might take a moment"
-make O=$TARGET > $VERBOSE 2>&1
+make $MAKEJOBS O=$TARGET > $VERBOSE 2>&1
 
 if [ $? -ne 0 ]
 then
-- 
2.5.0

^ permalink raw reply	[flat|nested] 11+ messages in thread

* [dpdk-dev] [PATCH 2/3] scripts: avoid editing defconfig_* files in validate-abi.sh
  2016-03-10 10:53 [dpdk-dev] [PATCH 1/3] scripts: support parallel building in validate-abi.sh via -j[N] option Panu Matilainen
@ 2016-03-10 10:53 ` Panu Matilainen
  2016-03-10 12:25   ` Ferruh Yigit
  2016-03-10 10:53 ` [dpdk-dev] [PATCH 3/3] scripts: ignore self-generated directories in validate-abi startup check Panu Matilainen
  2016-03-10 12:52 ` [dpdk-dev] [PATCH 1/3] scripts: support parallel building in validate-abi.sh via -j[N] option Ferruh Yigit
  2 siblings, 1 reply; 11+ messages in thread
From: Panu Matilainen @ 2016-03-10 10:53 UTC (permalink / raw)
  To: dev

The defconfig_* files are templates which are not supposed to be
edited, and doing so tends to leave unwanted cruft behind. Edit
the "working copy" config instead, which is the intended DPDK way.

Signed-off-by: Panu Matilainen <pmatilai@redhat.com>
---
 scripts/validate-abi.sh | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)

diff --git a/scripts/validate-abi.sh b/scripts/validate-abi.sh
index f094582..ea60639 100755
--- a/scripts/validate-abi.sh
+++ b/scripts/validate-abi.sh
@@ -90,11 +90,11 @@ cleanup_and_exit() {
 # Make sure we configure SHARED libraries
 # Also turn off IGB and KNI as those require kernel headers to build
 fixup_config() {
-	sed -i -e"$ a\CONFIG_RTE_BUILD_SHARED_LIB=y" config/defconfig_$TARGET
-	sed -i -e"$ a\CONFIG_RTE_NEXT_ABI=n" config/defconfig_$TARGET
-	sed -i -e"$ a\CONFIG_RTE_EAL_IGB_UIO=n" config/defconfig_$TARGET
-	sed -i -e"$ a\CONFIG_RTE_LIBRTE_KNI=n" config/defconfig_$TARGET
-	sed -i -e"$ a\CONFIG_RTE_KNI_KMOD=n" config/defconfig_$TARGET
+	sed -i -e"$ a\CONFIG_RTE_BUILD_SHARED_LIB=y" $TARGET/.config
+	sed -i -e"$ a\CONFIG_RTE_NEXT_ABI=n" $TARGET/.config
+	sed -i -e"$ a\CONFIG_RTE_EAL_IGB_UIO=n" $TARGET/.config
+	sed -i -e"$ a\CONFIG_RTE_LIBRTE_KNI=n" $TARGET/.config
+	sed -i -e"$ a\CONFIG_RTE_KNI_KMOD=n" $TARGET/.config
 }
 
 ###########################################
@@ -177,8 +177,6 @@ log "INFO" "Checking out version $TAG1 of the dpdk"
 # Move to the old version of the tree
 git checkout $HASH1
 
-fixup_config
-
 # Checking abi compliance relies on using the dwarf information in
 # The shared objects.  Thats only included in the DSO's if we build
 # with -g
@@ -189,6 +187,8 @@ export EXTRA_LDFLAGS="$EXTRA_LDFLAGS -g"
 log "INFO" "Configuring DPDK $TAG1"
 make config T=$TARGET O=$TARGET > $VERBOSE 2>&1
 
+fixup_config
+
 log "INFO" "Building DPDK $TAG1. This might take a moment"
 make $MAKEJOBS O=$TARGET > $VERBOSE 2>&1
 
@@ -214,12 +214,12 @@ git reset --hard
 log "INFO" "Checking out version $TAG2 of the dpdk"
 git checkout $HASH2
 
-fixup_config
-
 # Now configure the build
 log "INFO" "Configuring DPDK $TAG2"
 make config T=$TARGET O=$TARGET > $VERBOSE 2>&1
 
+fixup_config
+
 log "INFO" "Building DPDK $TAG2. This might take a moment"
 make $MAKEJOBS O=$TARGET > $VERBOSE 2>&1
 
-- 
2.5.0

^ permalink raw reply	[flat|nested] 11+ messages in thread

* [dpdk-dev] [PATCH 3/3] scripts: ignore self-generated directories in validate-abi startup check
  2016-03-10 10:53 [dpdk-dev] [PATCH 1/3] scripts: support parallel building in validate-abi.sh via -j[N] option Panu Matilainen
  2016-03-10 10:53 ` [dpdk-dev] [PATCH 2/3] scripts: avoid editing defconfig_* files in validate-abi.sh Panu Matilainen
@ 2016-03-10 10:53 ` Panu Matilainen
  2016-03-10 12:22   ` Ferruh Yigit
  2016-03-10 12:52 ` [dpdk-dev] [PATCH 1/3] scripts: support parallel building in validate-abi.sh via -j[N] option Ferruh Yigit
  2 siblings, 1 reply; 11+ messages in thread
From: Panu Matilainen @ 2016-03-10 10:53 UTC (permalink / raw)
  To: dev

When doing multiple runs of validate-abi.sh, the git status check
will more often than not unnecessarily fail with "Working directory not
clean" error because of the compat_result and compile target directories
from the previous run. Filter out the self-generated directories
when checking.

Signed-off-by: Panu Matilainen <pmatilai@redhat.com>
---
 scripts/validate-abi.sh | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/scripts/validate-abi.sh b/scripts/validate-abi.sh
index ea60639..a21f883 100755
--- a/scripts/validate-abi.sh
+++ b/scripts/validate-abi.sh
@@ -163,8 +163,7 @@ log "INFO" "against DPDK DSOs built from version $TAG2."
 log "INFO" ""
 
 # Check to make sure we have a clean tree
-git status | grep -q clean
-if [ $? -ne 0 ]
+if [ $(git status --porcelain | grep -vE "($TARGET|compat_reports)" | wc -l)  -ne 0 ]
 then
 	log "WARN" "Working directory not clean, aborting"
 	cleanup_and_exit 1
-- 
2.5.0

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [dpdk-dev] [PATCH 3/3] scripts: ignore self-generated directories in validate-abi startup check
  2016-03-10 10:53 ` [dpdk-dev] [PATCH 3/3] scripts: ignore self-generated directories in validate-abi startup check Panu Matilainen
@ 2016-03-10 12:22   ` Ferruh Yigit
  2016-03-10 12:29     ` Panu Matilainen
  0 siblings, 1 reply; 11+ messages in thread
From: Ferruh Yigit @ 2016-03-10 12:22 UTC (permalink / raw)
  To: Panu Matilainen, dev

On 3/10/2016 10:53 AM, Panu Matilainen wrote:
> When doing multiple runs of validate-abi.sh, the git status check
> will more often than not unnecessarily fail with "Working directory not
> clean" error because of the compat_result and compile target directories
> from the previous run. Filter out the self-generated directories
> when checking.
> 
> Signed-off-by: Panu Matilainen <pmatilai@redhat.com>
> ---
>  scripts/validate-abi.sh | 3 +--
>  1 file changed, 1 insertion(+), 2 deletions(-)
> 
> diff --git a/scripts/validate-abi.sh b/scripts/validate-abi.sh
> index ea60639..a21f883 100755
> --- a/scripts/validate-abi.sh
> +++ b/scripts/validate-abi.sh
> @@ -163,8 +163,7 @@ log "INFO" "against DPDK DSOs built from version $TAG2."
>  log "INFO" ""
>  
>  # Check to make sure we have a clean tree
> -git status | grep -q clean
> -if [ $? -ne 0 ]
> +if [ $(git status --porcelain | grep -vE "($TARGET|compat_reports)" | wc -l)  -ne 0 ]
>  then
>  	log "WARN" "Working directory not clean, aborting"
>  	cleanup_and_exit 1
> 
Hi Panu,

This check catches untracked files too, does it makes sense to limit
error only to modified files (local or staged)?

This also prevents specific "compat_reports" folder check.

And of course mentioned change requires "git clean -fd" removed, or
replaced with "make clean"

Thanks,
ferruh

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [dpdk-dev] [PATCH 2/3] scripts: avoid editing defconfig_* files in validate-abi.sh
  2016-03-10 10:53 ` [dpdk-dev] [PATCH 2/3] scripts: avoid editing defconfig_* files in validate-abi.sh Panu Matilainen
@ 2016-03-10 12:25   ` Ferruh Yigit
  2016-03-10 12:36     ` Panu Matilainen
  0 siblings, 1 reply; 11+ messages in thread
From: Ferruh Yigit @ 2016-03-10 12:25 UTC (permalink / raw)
  To: Panu Matilainen, dev

On 3/10/2016 10:53 AM, Panu Matilainen wrote:
> The defconfig_* files are templates which are not supposed to be
> edited, and doing so tends to leave unwanted cruft behind. Edit
> the "working copy" config instead, which is the intended DPDK way.
> 
> Signed-off-by: Panu Matilainen <pmatilai@redhat.com>
> ---
>  scripts/validate-abi.sh | 18 +++++++++---------
>  1 file changed, 9 insertions(+), 9 deletions(-)
> 
> diff --git a/scripts/validate-abi.sh b/scripts/validate-abi.sh
> index f094582..ea60639 100755
> --- a/scripts/validate-abi.sh
> +++ b/scripts/validate-abi.sh
> @@ -90,11 +90,11 @@ cleanup_and_exit() {
>  # Make sure we configure SHARED libraries
>  # Also turn off IGB and KNI as those require kernel headers to build
>  fixup_config() {
> -	sed -i -e"$ a\CONFIG_RTE_BUILD_SHARED_LIB=y" config/defconfig_$TARGET
> -	sed -i -e"$ a\CONFIG_RTE_NEXT_ABI=n" config/defconfig_$TARGET
> -	sed -i -e"$ a\CONFIG_RTE_EAL_IGB_UIO=n" config/defconfig_$TARGET
> -	sed -i -e"$ a\CONFIG_RTE_LIBRTE_KNI=n" config/defconfig_$TARGET
> -	sed -i -e"$ a\CONFIG_RTE_KNI_KMOD=n" config/defconfig_$TARGET
> +	sed -i -e"$ a\CONFIG_RTE_BUILD_SHARED_LIB=y" $TARGET/.config
> +	sed -i -e"$ a\CONFIG_RTE_NEXT_ABI=n" $TARGET/.config
> +	sed -i -e"$ a\CONFIG_RTE_EAL_IGB_UIO=n" $TARGET/.config
> +	sed -i -e"$ a\CONFIG_RTE_LIBRTE_KNI=n" $TARGET/.config
> +	sed -i -e"$ a\CONFIG_RTE_KNI_KMOD=n" $TARGET/.config
>  }
>  
Since working .config copy updated, is there a benefit to save and
restore the original .config file (if exists)?

And is "git reset --hard" still required in the script?

Thanks,
ferruh

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [dpdk-dev] [PATCH 3/3] scripts: ignore self-generated directories in validate-abi startup check
  2016-03-10 12:22   ` Ferruh Yigit
@ 2016-03-10 12:29     ` Panu Matilainen
  2016-03-10 12:34       ` Ferruh Yigit
  0 siblings, 1 reply; 11+ messages in thread
From: Panu Matilainen @ 2016-03-10 12:29 UTC (permalink / raw)
  To: Ferruh Yigit, dev

On 03/10/2016 02:22 PM, Ferruh Yigit wrote:
> On 3/10/2016 10:53 AM, Panu Matilainen wrote:
>> When doing multiple runs of validate-abi.sh, the git status check
>> will more often than not unnecessarily fail with "Working directory not
>> clean" error because of the compat_result and compile target directories
>> from the previous run. Filter out the self-generated directories
>> when checking.
>>
>> Signed-off-by: Panu Matilainen <pmatilai@redhat.com>
>> ---
>>   scripts/validate-abi.sh | 3 +--
>>   1 file changed, 1 insertion(+), 2 deletions(-)
>>
>> diff --git a/scripts/validate-abi.sh b/scripts/validate-abi.sh
>> index ea60639..a21f883 100755
>> --- a/scripts/validate-abi.sh
>> +++ b/scripts/validate-abi.sh
>> @@ -163,8 +163,7 @@ log "INFO" "against DPDK DSOs built from version $TAG2."
>>   log "INFO" ""
>>
>>   # Check to make sure we have a clean tree
>> -git status | grep -q clean
>> -if [ $? -ne 0 ]
>> +if [ $(git status --porcelain | grep -vE "($TARGET|compat_reports)" | wc -l)  -ne 0 ]
>>   then
>>   	log "WARN" "Working directory not clean, aborting"
>>   	cleanup_and_exit 1
>>
> Hi Panu,
>
> This check catches untracked files too, does it makes sense to limit
> error only to modified files (local or staged)?

I did ponder about that, untracked files seem mostly harmless in this 
picture but erred on the side of caution.

>
> This also prevents specific "compat_reports" folder check.
>
> And of course mentioned change requires "git clean -fd" removed, or
> replaced with "make clean"

Sorry, I dont understand you mean by these two comments.

	- Panu -

> Thanks,
> ferruh
>

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [dpdk-dev] [PATCH 3/3] scripts: ignore self-generated directories in validate-abi startup check
  2016-03-10 12:29     ` Panu Matilainen
@ 2016-03-10 12:34       ` Ferruh Yigit
  2016-03-10 12:39         ` Panu Matilainen
  0 siblings, 1 reply; 11+ messages in thread
From: Ferruh Yigit @ 2016-03-10 12:34 UTC (permalink / raw)
  To: Panu Matilainen, dev

On 3/10/2016 12:29 PM, Panu Matilainen wrote:
> On 03/10/2016 02:22 PM, Ferruh Yigit wrote:
>> On 3/10/2016 10:53 AM, Panu Matilainen wrote:
>>> When doing multiple runs of validate-abi.sh, the git status check
>>> will more often than not unnecessarily fail with "Working directory not
>>> clean" error because of the compat_result and compile target directories
>>> from the previous run. Filter out the self-generated directories
>>> when checking.
>>>
>>> Signed-off-by: Panu Matilainen <pmatilai@redhat.com>
>>> ---
>>>   scripts/validate-abi.sh | 3 +--
>>>   1 file changed, 1 insertion(+), 2 deletions(-)
>>>
>>> diff --git a/scripts/validate-abi.sh b/scripts/validate-abi.sh
>>> index ea60639..a21f883 100755
>>> --- a/scripts/validate-abi.sh
>>> +++ b/scripts/validate-abi.sh
>>> @@ -163,8 +163,7 @@ log "INFO" "against DPDK DSOs built from version $TAG2."
>>>   log "INFO" ""
>>>
>>>   # Check to make sure we have a clean tree
>>> -git status | grep -q clean
>>> -if [ $? -ne 0 ]
>>> +if [ $(git status --porcelain | grep -vE "($TARGET|compat_reports)" | wc -l)  -ne 0 ]
>>>   then
>>>   	log "WARN" "Working directory not clean, aborting"
>>>   	cleanup_and_exit 1
>>>
>> Hi Panu,
>>
>> This check catches untracked files too, does it makes sense to limit
>> error only to modified files (local or staged)?
> 
> I did ponder about that, untracked files seem mostly harmless in this 
> picture but erred on the side of caution.
> 
This is something prevents me running script from working tree, and
forces to create a new clone.

>>
>> This also prevents specific "compat_reports" folder check.
>>
>> And of course mentioned change requires "git clean -fd" removed, or
>> replaced with "make clean"
> 
> Sorry, I dont understand you mean by these two comments.
> 
If untracked files accepted by script, "compat_reports" exclusion is no
more required, and "git clean -fd" needs to removed from script.

Regards,
ferruh

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [dpdk-dev] [PATCH 2/3] scripts: avoid editing defconfig_* files in validate-abi.sh
  2016-03-10 12:25   ` Ferruh Yigit
@ 2016-03-10 12:36     ` Panu Matilainen
  0 siblings, 0 replies; 11+ messages in thread
From: Panu Matilainen @ 2016-03-10 12:36 UTC (permalink / raw)
  To: Ferruh Yigit, dev

On 03/10/2016 02:25 PM, Ferruh Yigit wrote:
> On 3/10/2016 10:53 AM, Panu Matilainen wrote:
>> The defconfig_* files are templates which are not supposed to be
>> edited, and doing so tends to leave unwanted cruft behind. Edit
>> the "working copy" config instead, which is the intended DPDK way.
>>
>> Signed-off-by: Panu Matilainen <pmatilai@redhat.com>
>> ---
>>   scripts/validate-abi.sh | 18 +++++++++---------
>>   1 file changed, 9 insertions(+), 9 deletions(-)
>>
>> diff --git a/scripts/validate-abi.sh b/scripts/validate-abi.sh
>> index f094582..ea60639 100755
>> --- a/scripts/validate-abi.sh
>> +++ b/scripts/validate-abi.sh
>> @@ -90,11 +90,11 @@ cleanup_and_exit() {
>>   # Make sure we configure SHARED libraries
>>   # Also turn off IGB and KNI as those require kernel headers to build
>>   fixup_config() {
>> -	sed -i -e"$ a\CONFIG_RTE_BUILD_SHARED_LIB=y" config/defconfig_$TARGET
>> -	sed -i -e"$ a\CONFIG_RTE_NEXT_ABI=n" config/defconfig_$TARGET
>> -	sed -i -e"$ a\CONFIG_RTE_EAL_IGB_UIO=n" config/defconfig_$TARGET
>> -	sed -i -e"$ a\CONFIG_RTE_LIBRTE_KNI=n" config/defconfig_$TARGET
>> -	sed -i -e"$ a\CONFIG_RTE_KNI_KMOD=n" config/defconfig_$TARGET
>> +	sed -i -e"$ a\CONFIG_RTE_BUILD_SHARED_LIB=y" $TARGET/.config
>> +	sed -i -e"$ a\CONFIG_RTE_NEXT_ABI=n" $TARGET/.config
>> +	sed -i -e"$ a\CONFIG_RTE_EAL_IGB_UIO=n" $TARGET/.config
>> +	sed -i -e"$ a\CONFIG_RTE_LIBRTE_KNI=n" $TARGET/.config
>> +	sed -i -e"$ a\CONFIG_RTE_KNI_KMOD=n" $TARGET/.config
>>   }
>>
> Since working .config copy updated, is there a benefit to save and
> restore the original .config file (if exists)?

Maybe, but it already is being modified and not restored, so that 
behavior is not new in this patch.

> And is "git reset --hard" still required in the script?

Perhaps not, strictly speaking. It does ensure there are no other 
artifacts left around though.

	- Panu -

> Thanks,
> ferruh
>

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [dpdk-dev] [PATCH 3/3] scripts: ignore self-generated directories in validate-abi startup check
  2016-03-10 12:34       ` Ferruh Yigit
@ 2016-03-10 12:39         ` Panu Matilainen
  2016-03-10 12:47           ` Ferruh Yigit
  0 siblings, 1 reply; 11+ messages in thread
From: Panu Matilainen @ 2016-03-10 12:39 UTC (permalink / raw)
  To: Ferruh Yigit, dev

On 03/10/2016 02:34 PM, Ferruh Yigit wrote:
> On 3/10/2016 12:29 PM, Panu Matilainen wrote:
>> On 03/10/2016 02:22 PM, Ferruh Yigit wrote:
>>> On 3/10/2016 10:53 AM, Panu Matilainen wrote:
>>>> When doing multiple runs of validate-abi.sh, the git status check
>>>> will more often than not unnecessarily fail with "Working directory not
>>>> clean" error because of the compat_result and compile target directories
>>>> from the previous run. Filter out the self-generated directories
>>>> when checking.
>>>>
>>>> Signed-off-by: Panu Matilainen <pmatilai@redhat.com>
>>>> ---
>>>>    scripts/validate-abi.sh | 3 +--
>>>>    1 file changed, 1 insertion(+), 2 deletions(-)
>>>>
>>>> diff --git a/scripts/validate-abi.sh b/scripts/validate-abi.sh
>>>> index ea60639..a21f883 100755
>>>> --- a/scripts/validate-abi.sh
>>>> +++ b/scripts/validate-abi.sh
>>>> @@ -163,8 +163,7 @@ log "INFO" "against DPDK DSOs built from version $TAG2."
>>>>    log "INFO" ""
>>>>
>>>>    # Check to make sure we have a clean tree
>>>> -git status | grep -q clean
>>>> -if [ $? -ne 0 ]
>>>> +if [ $(git status --porcelain | grep -vE "($TARGET|compat_reports)" | wc -l)  -ne 0 ]
>>>>    then
>>>>    	log "WARN" "Working directory not clean, aborting"
>>>>    	cleanup_and_exit 1
>>>>
>>> Hi Panu,
>>>
>>> This check catches untracked files too, does it makes sense to limit
>>> error only to modified files (local or staged)?
>>
>> I did ponder about that, untracked files seem mostly harmless in this
>> picture but erred on the side of caution.
>>
> This is something prevents me running script from working tree, and
> forces to create a new clone.

Hmm, what untracked files you typically have in your working tree then?

>>>
>>> This also prevents specific "compat_reports" folder check.
>>>
>>> And of course mentioned change requires "git clean -fd" removed, or
>>> replaced with "make clean"
>>
>> Sorry, I dont understand you mean by these two comments.
>>
> If untracked files accepted by script, "compat_reports" exclusion is no
> more required, and "git clean -fd" needs to removed from script.

Ah, sure. Thanks for clarifying.

	- Panu -

>
> Regards,
> ferruh
>

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [dpdk-dev] [PATCH 3/3] scripts: ignore self-generated directories in validate-abi startup check
  2016-03-10 12:39         ` Panu Matilainen
@ 2016-03-10 12:47           ` Ferruh Yigit
  0 siblings, 0 replies; 11+ messages in thread
From: Ferruh Yigit @ 2016-03-10 12:47 UTC (permalink / raw)
  To: Panu Matilainen, dev

On 3/10/2016 12:39 PM, Panu Matilainen wrote:
> On 03/10/2016 02:34 PM, Ferruh Yigit wrote:
>> On 3/10/2016 12:29 PM, Panu Matilainen wrote:
>>> On 03/10/2016 02:22 PM, Ferruh Yigit wrote:
>>>> On 3/10/2016 10:53 AM, Panu Matilainen wrote:
>>>>> When doing multiple runs of validate-abi.sh, the git status check
>>>>> will more often than not unnecessarily fail with "Working directory not
>>>>> clean" error because of the compat_result and compile target directories
>>>>> from the previous run. Filter out the self-generated directories
>>>>> when checking.
>>>>>
>>>>> Signed-off-by: Panu Matilainen <pmatilai@redhat.com>
>>>>> ---
>>>>>    scripts/validate-abi.sh | 3 +--
>>>>>    1 file changed, 1 insertion(+), 2 deletions(-)
>>>>>
>>>>> diff --git a/scripts/validate-abi.sh b/scripts/validate-abi.sh
>>>>> index ea60639..a21f883 100755
>>>>> --- a/scripts/validate-abi.sh
>>>>> +++ b/scripts/validate-abi.sh
>>>>> @@ -163,8 +163,7 @@ log "INFO" "against DPDK DSOs built from version $TAG2."
>>>>>    log "INFO" ""
>>>>>
>>>>>    # Check to make sure we have a clean tree
>>>>> -git status | grep -q clean
>>>>> -if [ $? -ne 0 ]
>>>>> +if [ $(git status --porcelain | grep -vE "($TARGET|compat_reports)" | wc -l)  -ne 0 ]
>>>>>    then
>>>>>    	log "WARN" "Working directory not clean, aborting"
>>>>>    	cleanup_and_exit 1
>>>>>
>>>> Hi Panu,
>>>>
>>>> This check catches untracked files too, does it makes sense to limit
>>>> error only to modified files (local or staged)?
>>>
>>> I did ponder about that, untracked files seem mostly harmless in this
>>> picture but erred on the side of caution.
>>>
>> This is something prevents me running script from working tree, and
>> forces to create a new clone.
> 
> Hmm, what untracked files you typically have in your working tree then?
> 

cscope.out, various sym links, perf.data, and some more J, I want to
keep in working directory.

>>>>
>>>> This also prevents specific "compat_reports" folder check.
>>>>
>>>> And of course mentioned change requires "git clean -fd" removed, or
>>>> replaced with "make clean"
>>>
>>> Sorry, I dont understand you mean by these two comments.
>>>
>> If untracked files accepted by script, "compat_reports" exclusion is no
>> more required, and "git clean -fd" needs to removed from script.
> 
> Ah, sure. Thanks for clarifying.
> 
> 	- Panu -
> 
>>
>> Regards,
>> ferruh
>>
> 

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [dpdk-dev] [PATCH 1/3] scripts: support parallel building in validate-abi.sh via -j[N] option
  2016-03-10 10:53 [dpdk-dev] [PATCH 1/3] scripts: support parallel building in validate-abi.sh via -j[N] option Panu Matilainen
  2016-03-10 10:53 ` [dpdk-dev] [PATCH 2/3] scripts: avoid editing defconfig_* files in validate-abi.sh Panu Matilainen
  2016-03-10 10:53 ` [dpdk-dev] [PATCH 3/3] scripts: ignore self-generated directories in validate-abi startup check Panu Matilainen
@ 2016-03-10 12:52 ` Ferruh Yigit
  2 siblings, 0 replies; 11+ messages in thread
From: Ferruh Yigit @ 2016-03-10 12:52 UTC (permalink / raw)
  To: Panu Matilainen, dev

On 3/10/2016 10:53 AM, Panu Matilainen wrote:
> Signed-off-by: Panu Matilainen <pmatilai@redhat.com>
> ---
>  doc/guides/contributing/versioning.rst |  4 +++-
>  scripts/validate-abi.sh                | 13 ++++++++++---
>  2 files changed, 13 insertions(+), 4 deletions(-)
> 
> diff --git a/doc/guides/contributing/versioning.rst b/doc/guides/contributing/versioning.rst
> index ae10a98..33b03a1 100644
> --- a/doc/guides/contributing/versioning.rst
> +++ b/doc/guides/contributing/versioning.rst
> @@ -469,11 +469,13 @@ utilities which can be installed via a package manager. For example::
>  
>  The syntax of the ``validate-abi.sh`` utility is::
>  
> -   ./scripts/validate-abi.sh <REV1> <REV2> <TARGET>
> +   ./scripts/validate-abi.sh [-j[N]] <REV1> <REV2> <TARGET>
>  
>  Where ``REV1`` and ``REV2`` are valid gitrevisions(7)
>  https://www.kernel.org/pub/software/scm/git/docs/gitrevisions.html
>  on the local repo and target is the usual DPDK compilation target.
> +The optional -j[N] switch enables parallel building with at most
> +N simultaneous jobs, ie the same as -j option of ``make``.
>  
>  For example:
>  
> diff --git a/scripts/validate-abi.sh b/scripts/validate-abi.sh
> index c36ad61..f094582 100755
> --- a/scripts/validate-abi.sh
> +++ b/scripts/validate-abi.sh
> @@ -27,13 +27,20 @@
>  #   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
>  #   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
>  
> +case "$1" in
> +    -j*)
> +        MAKEJOBS="$1"
> +        shift
> +        ;;
> +esac
> +
>  TAG1=$1
>  TAG2=$2
>  TARGET=$3
>  ABI_DIR=`mktemp -d -p /tmp ABI.XXXXXX`
>  
>  usage() {
> -	echo "$0 <REV1> <REV2> <TARGET>"
> +	echo "$0 [-j[N]] <REV1> <REV2> <TARGET>"
>  }
>  
>  log() {
> @@ -183,7 +190,7 @@ log "INFO" "Configuring DPDK $TAG1"
>  make config T=$TARGET O=$TARGET > $VERBOSE 2>&1
>  
>  log "INFO" "Building DPDK $TAG1. This might take a moment"
> -make O=$TARGET > $VERBOSE 2>&1
> +make $MAKEJOBS O=$TARGET > $VERBOSE 2>&1
>  
>  if [ $? -ne 0 ]
>  then
> @@ -214,7 +221,7 @@ log "INFO" "Configuring DPDK $TAG2"
>  make config T=$TARGET O=$TARGET > $VERBOSE 2>&1
>  
>  log "INFO" "Building DPDK $TAG2. This might take a moment"
> -make O=$TARGET > $VERBOSE 2>&1
> +make $MAKEJOBS O=$TARGET > $VERBOSE 2>&1
>  
>  if [ $? -ne 0 ]
>  then
> 

This is something good to have.

I also found following is also working, not sure if need to document in
script:
"MAKEFLAGS="-j32" ./validate-abi.sh X Y Z"

And was just thinking if needs to address "-j N" usage (space after -j),
it is your call. Other than this,

Acked-by: Ferruh Yigit <ferruh.yigit@intel.com>

^ permalink raw reply	[flat|nested] 11+ messages in thread

end of thread, other threads:[~2016-03-10 12:52 UTC | newest]

Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-03-10 10:53 [dpdk-dev] [PATCH 1/3] scripts: support parallel building in validate-abi.sh via -j[N] option Panu Matilainen
2016-03-10 10:53 ` [dpdk-dev] [PATCH 2/3] scripts: avoid editing defconfig_* files in validate-abi.sh Panu Matilainen
2016-03-10 12:25   ` Ferruh Yigit
2016-03-10 12:36     ` Panu Matilainen
2016-03-10 10:53 ` [dpdk-dev] [PATCH 3/3] scripts: ignore self-generated directories in validate-abi startup check Panu Matilainen
2016-03-10 12:22   ` Ferruh Yigit
2016-03-10 12:29     ` Panu Matilainen
2016-03-10 12:34       ` Ferruh Yigit
2016-03-10 12:39         ` Panu Matilainen
2016-03-10 12:47           ` Ferruh Yigit
2016-03-10 12:52 ` [dpdk-dev] [PATCH 1/3] scripts: support parallel building in validate-abi.sh via -j[N] option Ferruh Yigit

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).