[dpdk-dev] [PATCH] scripts: fix symbol overriding in configuration files

[dpdk-dev] [PATCH] scripts: fix symbol overriding in configuration files

[David Marchand]

When redefining the same symbol in configuration (basically after an inclusion),
we need to undefine the previous symbol to avoid "redefined" errors.

Signed-off-by: David Marchand <david.marchand@6wind.com>
---
 scripts/gen-config-h.sh |   12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/scripts/gen-config-h.sh b/scripts/gen-config-h.sh
index efd7667..2fac08c 100755
--- a/scripts/gen-config-h.sh
+++ b/scripts/gen-config-h.sh
@@ -33,11 +33,11 @@
 
 echo "#ifndef __RTE_CONFIG_H"
 echo "#define __RTE_CONFIG_H"
-grep CONFIG_ $1							 \
-| grep -v '^[ \t]*#'							 \
-| sed 's,CONFIG_\(.*\)=y.*$,#define \1 1,'			 \
-| sed 's,CONFIG_\(.*\)=n.*$,#undef \1,'				 \
-| sed 's,CONFIG_\(.*\)=\(.*\)$,#define \1 \2,'			 \
-| sed 's,\# CONFIG_\(.*\) is not set$,#undef \1,'
+grep CONFIG_ $1 |
+grep -v '^[ \t]*#' |
+sed 's,CONFIG_\(.*\)=y.*$,#undef \1\n#define \1 1,' |
+sed 's,CONFIG_\(.*\)=n.*$,#undef \1,' |
+sed 's,CONFIG_\(.*\)=\(.*\)$,#undef \1\n#define \1 \2,' |
+sed 's,\# CONFIG_\(.*\) is not set$,#undef \1,'
 echo "#endif /* __RTE_CONFIG_H */"
 
-- 
1.7.10.4

Re: [dpdk-dev] [PATCH] scripts: fix symbol overriding in configuration files

[Thomas Monjalon]

> When redefining the same symbol in configuration (basically after an inclusion),
> we need to undefine the previous symbol to avoid "redefined" errors.
> 
> Signed-off-by: David Marchand <david.marchand@6wind.com>

Acked-by: Thomas Monjalon <thomas.monjalon@6wind.com>

Applied

Thanks
-- 
Thomas

Re: [dpdk-dev] [PATCH] scripts: fix symbol overriding in configuration files

[Bruce Richardson]

On Thu, Nov 27, 2014 at 10:17:22AM -0800, Thomas Monjalon wrote:
> > When redefining the same symbol in configuration (basically after an inclusion),
> > we need to undefine the previous symbol to avoid "redefined" errors.
> > 
> > Signed-off-by: David Marchand <david.marchand@6wind.com>
> 
> Acked-by: Thomas Monjalon <thomas.monjalon@6wind.com>
> 
> Applied
>
This patch doesn't seem to work on FreeBSD. I'm still investigating how to fix
it but right now compilation with gcc/clang on FreeBSD is broken due to the
config.h file having lines like the below in it :-(

/usr/home/bruce/dpdk.org/x86_64-native-bsdapp-clang/include/rte_config.h:3:21: fatal error: extra tokens at end of #undef directive [-Wextra-tokens]
#undef RTE_EXEC_ENVn#define RTE_EXEC_ENV "bsdapp"

I'll send on a fix as soon as I can.

Regards,
/Bruce

Re: [dpdk-dev] [PATCH] scripts: fix symbol overriding in configuration files

[David Marchand]

Hello Bruce,

On Fri, Nov 28, 2014 at 2:56 PM, Bruce Richardson <
bruce.richardson@intel.com> wrote:

> On Thu, Nov 27, 2014 at 10:17:22AM -0800, Thomas Monjalon wrote:
> > > When redefining the same symbol in configuration (basically after an
> inclusion),
> > > we need to undefine the previous symbol to avoid "redefined" errors.
> > >
> > > Signed-off-by: David Marchand <david.marchand@6wind.com>
> >
> > Acked-by: Thomas Monjalon <thomas.monjalon@6wind.com>
> >
> > Applied
> >
> This patch doesn't seem to work on FreeBSD. I'm still investigating how to
> fix
> it but right now compilation with gcc/clang on FreeBSD is broken due to the
> config.h file having lines like the below in it :-(
>
> /usr/home/bruce/
> dpdk.org/x86_64-native-bsdapp-clang/include/rte_config.h:3:21: fatal
> error: extra tokens at end of #undef directive [-Wextra-tokens]
> #undef RTE_EXEC_ENVn#define RTE_EXEC_ENV "bsdapp"
>

This is probably because of the \n in the sed.

Can you try something like this (I did not like this version of my patch at
first because it is not that readable) ?


diff --git a/scripts/gen-config-h.sh b/scripts/gen-config-h.sh
index 2fac08c..d36efd6 100755
--- a/scripts/gen-config-h.sh
+++ b/scripts/gen-config-h.sh
@@ -35,9 +35,11 @@ echo "#ifndef __RTE_CONFIG_H"
 echo "#define __RTE_CONFIG_H"
 grep CONFIG_ $1 |
 grep -v '^[ \t]*#' |
-sed 's,CONFIG_\(.*\)=y.*$,#undef \1\n#define \1 1,' |
+sed 's,CONFIG_\(.*\)=y.*$,#undef \1\
+#define \1 1,' |
 sed 's,CONFIG_\(.*\)=n.*$,#undef \1,' |
-sed 's,CONFIG_\(.*\)=\(.*\)$,#undef \1\n#define \1 \2,' |
+sed 's,CONFIG_\(.*\)=\(.*\)$,#undef \1\
+#define \1 \2,' |
 sed 's,\# CONFIG_\(.*\) is not set$,#undef \1,'
 echo "#endif /* __RTE_CONFIG_H */"



-- 
David Marchand

Re: [dpdk-dev] [PATCH] scripts: fix symbol overriding in configuration files

[Bruce Richardson]

On Fri, Nov 28, 2014 at 03:06:11PM +0100, David Marchand wrote:
> Hello Bruce,
> 
> On Fri, Nov 28, 2014 at 2:56 PM, Bruce Richardson <
> bruce.richardson@intel.com> wrote:
> 
> > On Thu, Nov 27, 2014 at 10:17:22AM -0800, Thomas Monjalon wrote:
> > > > When redefining the same symbol in configuration (basically after an
> > inclusion),
> > > > we need to undefine the previous symbol to avoid "redefined" errors.
> > > >
> > > > Signed-off-by: David Marchand <david.marchand@6wind.com>
> > >
> > > Acked-by: Thomas Monjalon <thomas.monjalon@6wind.com>
> > >
> > > Applied
> > >
> > This patch doesn't seem to work on FreeBSD. I'm still investigating how to
> > fix
> > it but right now compilation with gcc/clang on FreeBSD is broken due to the
> > config.h file having lines like the below in it :-(
> >
> > /usr/home/bruce/
> > dpdk.org/x86_64-native-bsdapp-clang/include/rte_config.h:3:21: fatal
> > error: extra tokens at end of #undef directive [-Wextra-tokens]
> > #undef RTE_EXEC_ENVn#define RTE_EXEC_ENV "bsdapp"
> >
> 
> This is probably because of the \n in the sed.
> 
> Can you try something like this (I did not like this version of my patch at
> first because it is not that readable) ?
> 
> 
> diff --git a/scripts/gen-config-h.sh b/scripts/gen-config-h.sh
> index 2fac08c..d36efd6 100755
> --- a/scripts/gen-config-h.sh
> +++ b/scripts/gen-config-h.sh
> @@ -35,9 +35,11 @@ echo "#ifndef __RTE_CONFIG_H"
>  echo "#define __RTE_CONFIG_H"
>  grep CONFIG_ $1 |
>  grep -v '^[ \t]*#' |
> -sed 's,CONFIG_\(.*\)=y.*$,#undef \1\n#define \1 1,' |
> +sed 's,CONFIG_\(.*\)=y.*$,#undef \1\
> +#define \1 1,' |
>  sed 's,CONFIG_\(.*\)=n.*$,#undef \1,' |
> -sed 's,CONFIG_\(.*\)=\(.*\)$,#undef \1\n#define \1 \2,' |
> +sed 's,CONFIG_\(.*\)=\(.*\)$,#undef \1\
> +#define \1 \2,' |
>  sed 's,\# CONFIG_\(.*\) is not set$,#undef \1,'
>  echo "#endif /* __RTE_CONFIG_H */"
>
I tried a number of different things to get sed to introduce "\n" characters,
but I missed trying that one. I've sent on an alternative fix now, which uses
tr instead of sed to insert the "\n"'s. It's not a fix I'm terribly happy with,
but having seen the above (but not tried it yet), it actually doesn't seem that
bad in comparison :-)

/Bruce

Re: [dpdk-dev] [PATCH] scripts: fix symbol overriding in configuration files

[Bruce Richardson]

On Fri, Nov 28, 2014 at 03:06:11PM +0100, David Marchand wrote:
> Hello Bruce,
> 
> On Fri, Nov 28, 2014 at 2:56 PM, Bruce Richardson <
> bruce.richardson@intel.com> wrote:
> 
> > On Thu, Nov 27, 2014 at 10:17:22AM -0800, Thomas Monjalon wrote:
> > > > When redefining the same symbol in configuration (basically after an
> > inclusion),
> > > > we need to undefine the previous symbol to avoid "redefined" errors.
> > > >
> > > > Signed-off-by: David Marchand <david.marchand@6wind.com>
> > >
> > > Acked-by: Thomas Monjalon <thomas.monjalon@6wind.com>
> > >
> > > Applied
> > >
> > This patch doesn't seem to work on FreeBSD. I'm still investigating how to
> > fix
> > it but right now compilation with gcc/clang on FreeBSD is broken due to the
> > config.h file having lines like the below in it :-(
> >
> > /usr/home/bruce/
> > dpdk.org/x86_64-native-bsdapp-clang/include/rte_config.h:3:21: fatal
> > error: extra tokens at end of #undef directive [-Wextra-tokens]
> > #undef RTE_EXEC_ENVn#define RTE_EXEC_ENV "bsdapp"
> >
> 
> This is probably because of the \n in the sed.
> 
> Can you try something like this (I did not like this version of my patch at
> first because it is not that readable) ?
> 
> 
> diff --git a/scripts/gen-config-h.sh b/scripts/gen-config-h.sh
> index 2fac08c..d36efd6 100755
> --- a/scripts/gen-config-h.sh
> +++ b/scripts/gen-config-h.sh
> @@ -35,9 +35,11 @@ echo "#ifndef __RTE_CONFIG_H"
>  echo "#define __RTE_CONFIG_H"
>  grep CONFIG_ $1 |
>  grep -v '^[ \t]*#' |
> -sed 's,CONFIG_\(.*\)=y.*$,#undef \1\n#define \1 1,' |
> +sed 's,CONFIG_\(.*\)=y.*$,#undef \1\
> +#define \1 1,' |
>  sed 's,CONFIG_\(.*\)=n.*$,#undef \1,' |
> -sed 's,CONFIG_\(.*\)=\(.*\)$,#undef \1\n#define \1 \2,' |
> +sed 's,CONFIG_\(.*\)=\(.*\)$,#undef \1\
> +#define \1 \2,' |
>  sed 's,\# CONFIG_\(.*\) is not set$,#undef \1,'
>  echo "#endif /* __RTE_CONFIG_H */"
> 
>
This proposed fix also works. I don't mind whether my patch proposal or this
fix gets applied - gen-config-h.sh is not a commonly modified script anyway.

/Bruce

Re: [dpdk-dev] [PATCH] scripts: fix symbol overriding in configuration files

[David Marchand]

On Fri, Nov 28, 2014 at 3:43 PM, Bruce Richardson <
bruce.richardson@intel.com> wrote:

> This proposed fix also works. I don't mind whether my patch proposal or
> this
> fix gets applied - gen-config-h.sh is not a commonly modified script
> anyway.
>

I prefer my ugliness ;-)
But Thomas just proposed me something that could work.
Trying ...


-- 
David Marchand

Re: [dpdk-dev] [PATCH] scripts: fix symbol overriding in configuration files

[Bruce Richardson]

On Fri, Nov 28, 2014 at 03:49:29PM +0100, David Marchand wrote:
> On Fri, Nov 28, 2014 at 3:43 PM, Bruce Richardson <
> bruce.richardson@intel.com> wrote:
> 
> > This proposed fix also works. I don't mind whether my patch proposal or
> > this
> > fix gets applied - gen-config-h.sh is not a commonly modified script
> > anyway.
> >
> 
> I prefer my ugliness ;-)
> But Thomas just proposed me something that could work.
> Trying ...
>
Yes, it's ugly, but it's probably more resilient. I'm looking forward to getting
an option C.

/Bruce

Re: [dpdk-dev] [PATCH] scripts: fix symbol overriding in configuration files

[David Marchand]

On Fri, Nov 28, 2014 at 3:59 PM, Bruce Richardson <
bruce.richardson@intel.com> wrote:

> Yes, it's ugly, but it's probably more resilient. I'm looking forward to
> getting
> an option C.
>

Option C and D are ugly as well (using some bashism like $' ' or using an
intermediate variable with a newline in it).
Our "posix" guy told me that he prefers the initial version of the patch.
I will send a patch with this.

Thomas ?

-- 
David Marchand

[dpdk-dev] [PATCH] scripts: fix newline character in sed expression

[David Marchand]

Use of \n in sed expression is not portable and triggered an invalid
configuration on BSD (at least).
Replace with an explicit newline.

Reported-by: Bruce Richardson <bruce.richardson@intel.com>
Signed-off-by: David Marchand <david.marchand@6wind.com>
---
 scripts/gen-config-h.sh |    6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/scripts/gen-config-h.sh b/scripts/gen-config-h.sh
index 2fac08c..d36efd6 100755
--- a/scripts/gen-config-h.sh
+++ b/scripts/gen-config-h.sh
@@ -35,9 +35,11 @@ echo "#ifndef __RTE_CONFIG_H"
 echo "#define __RTE_CONFIG_H"
 grep CONFIG_ $1 |
 grep -v '^[ \t]*#' |
-sed 's,CONFIG_\(.*\)=y.*$,#undef \1\n#define \1 1,' |
+sed 's,CONFIG_\(.*\)=y.*$,#undef \1\
+#define \1 1,' |
 sed 's,CONFIG_\(.*\)=n.*$,#undef \1,' |
-sed 's,CONFIG_\(.*\)=\(.*\)$,#undef \1\n#define \1 \2,' |
+sed 's,CONFIG_\(.*\)=\(.*\)$,#undef \1\
+#define \1 \2,' |
 sed 's,\# CONFIG_\(.*\) is not set$,#undef \1,'
 echo "#endif /* __RTE_CONFIG_H */"
 
-- 
1.7.10.4

Re: [dpdk-dev] [PATCH] scripts: fix newline character in sed expression

[Bruce Richardson]

On Fri, Nov 28, 2014 at 04:42:44PM +0100, David Marchand wrote:
> Use of \n in sed expression is not portable and triggered an invalid
> configuration on BSD (at least).
> Replace with an explicit newline.
> 
> Reported-by: Bruce Richardson <bruce.richardson@intel.com>
> Signed-off-by: David Marchand <david.marchand@6wind.com>

Acked-by: Bruce Richardson <bruce.richardson@intel.com>

> ---
>  scripts/gen-config-h.sh |    6 ++++--
>  1 file changed, 4 insertions(+), 2 deletions(-)
> 
> diff --git a/scripts/gen-config-h.sh b/scripts/gen-config-h.sh
> index 2fac08c..d36efd6 100755
> --- a/scripts/gen-config-h.sh
> +++ b/scripts/gen-config-h.sh
> @@ -35,9 +35,11 @@ echo "#ifndef __RTE_CONFIG_H"
>  echo "#define __RTE_CONFIG_H"
>  grep CONFIG_ $1 |
>  grep -v '^[ \t]*#' |
> -sed 's,CONFIG_\(.*\)=y.*$,#undef \1\n#define \1 1,' |
> +sed 's,CONFIG_\(.*\)=y.*$,#undef \1\
> +#define \1 1,' |
>  sed 's,CONFIG_\(.*\)=n.*$,#undef \1,' |
> -sed 's,CONFIG_\(.*\)=\(.*\)$,#undef \1\n#define \1 \2,' |
> +sed 's,CONFIG_\(.*\)=\(.*\)$,#undef \1\
> +#define \1 \2,' |
>  sed 's,\# CONFIG_\(.*\) is not set$,#undef \1,'
>  echo "#endif /* __RTE_CONFIG_H */"
>  
> -- 
> 1.7.10.4
> 

Re: [dpdk-dev] [PATCH] scripts: fix newline character in sed expression

[Thomas Monjalon]

> > Use of \n in sed expression is not portable and triggered an invalid
> > configuration on BSD (at least).
> > Replace with an explicit newline.
> > 
> > Reported-by: Bruce Richardson <bruce.richardson@intel.com>
> > Signed-off-by: David Marchand <david.marchand@6wind.com>
> 
> Acked-by: Bruce Richardson <bruce.richardson@intel.com>

Applied

Thanks
-- 
Thomas

[dpdk-dev] [PATCH] scripts: fix merged lines on FreeBSD in config.h

[Bruce Richardson]

Since commit 0a91453d, "Fix symbol overriding in configuration", the
rte_config.h can have two lines generated for a single directive to
enable a feature - one line to undef the feature value, and a second
to enable or set the new value. On FreeBSD, sed inserts an "n" char
instead of the "\n" carriage return, leading to compiler errors.
This patch fixes that by having sed insert a "$" character instead
of attempting a "\n", and then using tr subsequently to turn "$"
characters into real "\n" characters.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 scripts/gen-config-h.sh | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/scripts/gen-config-h.sh b/scripts/gen-config-h.sh
index 2fac08c..647bf4b 100755
--- a/scripts/gen-config-h.sh
+++ b/scripts/gen-config-h.sh
@@ -35,9 +35,10 @@ echo "#ifndef __RTE_CONFIG_H"
 echo "#define __RTE_CONFIG_H"
 grep CONFIG_ $1 |
 grep -v '^[ \t]*#' |
-sed 's,CONFIG_\(.*\)=y.*$,#undef \1\n#define \1 1,' |
+sed 's,CONFIG_\(.*\)=y.*$,#undef \1$#define \1 1,' |
 sed 's,CONFIG_\(.*\)=n.*$,#undef \1,' |
-sed 's,CONFIG_\(.*\)=\(.*\)$,#undef \1\n#define \1 \2,' |
-sed 's,\# CONFIG_\(.*\) is not set$,#undef \1,'
+sed 's,CONFIG_\(.*\)=\(.*\)$,#undef \1$#define \1 \2,' |
+sed 's,\# CONFIG_\(.*\) is not set$,#undef \1,' |
+tr '$' '\n'
 echo "#endif /* __RTE_CONFIG_H */"
 
-- 
2.1.0

Re: [dpdk-dev] [PATCH] scripts: fix symbol overriding in configuration files

[Bruce Richardson]

On Thu, Nov 27, 2014 at 12:29:05PM +0100, David Marchand wrote:
> When redefining the same symbol in configuration (basically after an inclusion),
> we need to undefine the previous symbol to avoid "redefined" errors.
> 
> Signed-off-by: David Marchand <david.marchand@6wind.com>

Though I see this patch is already replied, my comment on the below is that
it may be best as two separate patches, since you are doing two things there
that makes the actual change hard to see. One patch should move the "|" from the
start of the next line to end of the previous, and the second patch should then
add the undef statements. As it is, I had to stare at this for a while to work
out why the grep lines were changing to undefine previously defined values. :-)

/Bruce

> ---
>  scripts/gen-config-h.sh |   12 ++++++------
>  1 file changed, 6 insertions(+), 6 deletions(-)
> 
> diff --git a/scripts/gen-config-h.sh b/scripts/gen-config-h.sh
> index efd7667..2fac08c 100755
> --- a/scripts/gen-config-h.sh
> +++ b/scripts/gen-config-h.sh
> @@ -33,11 +33,11 @@
>  
>  echo "#ifndef __RTE_CONFIG_H"
>  echo "#define __RTE_CONFIG_H"
> -grep CONFIG_ $1							 \
> -| grep -v '^[ \t]*#'							 \
> -| sed 's,CONFIG_\(.*\)=y.*$,#define \1 1,'			 \
> -| sed 's,CONFIG_\(.*\)=n.*$,#undef \1,'				 \
> -| sed 's,CONFIG_\(.*\)=\(.*\)$,#define \1 \2,'			 \
> -| sed 's,\# CONFIG_\(.*\) is not set$,#undef \1,'
> +grep CONFIG_ $1 |
> +grep -v '^[ \t]*#' |
> +sed 's,CONFIG_\(.*\)=y.*$,#undef \1\n#define \1 1,' |
> +sed 's,CONFIG_\(.*\)=n.*$,#undef \1,' |
> +sed 's,CONFIG_\(.*\)=\(.*\)$,#undef \1\n#define \1 \2,' |
> +sed 's,\# CONFIG_\(.*\) is not set$,#undef \1,'
>  echo "#endif /* __RTE_CONFIG_H */"
>  
> -- 
> 1.7.10.4
>