[dpdk-dev] [PATCH] config: default to shared library

[dpdk-dev] [PATCH] config: default to shared library

[Panu Matilainen]

With symbol versioning its vital that developers test their code in
shared library mode, otherwise we'll be playing "add the forgotten
symbol export" from here to eternity. By defaulting to shared we
should catch more of these cases early, but without taking away anybodys
ability to build static.

Signed-off-by: Panu Matilainen <pmatilai@redhat.com>
---
 config/common_bsdapp   | 2 +-
 config/common_linuxapp | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/config/common_bsdapp b/config/common_bsdapp
index 8ff4dc2..76b97be 100644
--- a/config/common_bsdapp
+++ b/config/common_bsdapp
@@ -76,7 +76,7 @@ CONFIG_RTE_FORCE_INTRINSICS=n
 #
 # Compile to share library
 #
-CONFIG_RTE_BUILD_SHARED_LIB=n
+CONFIG_RTE_BUILD_SHARED_LIB=y
 
 #
 # Combine to one single library
diff --git a/config/common_linuxapp b/config/common_linuxapp
index 97f1c9e..1852951 100644
--- a/config/common_linuxapp
+++ b/config/common_linuxapp
@@ -76,7 +76,7 @@ CONFIG_RTE_FORCE_INTRINSICS=n
 #
 # Compile to share library
 #
-CONFIG_RTE_BUILD_SHARED_LIB=n
+CONFIG_RTE_BUILD_SHARED_LIB=y
 
 #
 # Combine to one single library
-- 
2.1.0

Re: [dpdk-dev] [PATCH] config: default to shared library

[Thomas Monjalon]

Hi Panu,

2015-03-04 08:17, Panu Matilainen:
> With symbol versioning its vital that developers test their code in
> shared library mode, otherwise we'll be playing "add the forgotten
> symbol export" from here to eternity.

Yes we must improve the sanity checks.
A lot of options must be tested (or removed) and not only shared libs.
But the error you reported before (missing export of rte_eth_dev_release_port)
cannot be seen even with this patch.
It means we need more tools.
Though, default configuration is not a tool.

> By defaulting to shared we should catch more of these cases early,
> but without taking away anybodys ability to build static.

Shared libraries are convenient for distributions but have a performance
impact. I think that static build must remain the default choice.

Re: [dpdk-dev] [PATCH] config: default to shared library

[Bruce Richardson]

On Wed, Mar 04, 2015 at 10:24:44AM +0100, Thomas Monjalon wrote:
> Hi Panu,
> 
> 2015-03-04 08:17, Panu Matilainen:
> > With symbol versioning its vital that developers test their code in
> > shared library mode, otherwise we'll be playing "add the forgotten
> > symbol export" from here to eternity.
> 
> Yes we must improve the sanity checks.
> A lot of options must be tested (or removed) and not only shared libs.
> But the error you reported before (missing export of rte_eth_dev_release_port)
> cannot be seen even with this patch.
> It means we need more tools.
> Though, default configuration is not a tool.
> 
> > By defaulting to shared we should catch more of these cases early,
> > but without taking away anybodys ability to build static.
> 
> Shared libraries are convenient for distributions but have a performance
> impact. I think that static build must remain the default choice.
> 
+1

Re: [dpdk-dev] [PATCH] config: default to shared library

[Panu Matilainen]

On 03/04/2015 11:24 AM, Thomas Monjalon wrote:
> Hi Panu,
>
> 2015-03-04 08:17, Panu Matilainen:
>> With symbol versioning its vital that developers test their code in
>> shared library mode, otherwise we'll be playing "add the forgotten
>> symbol export" from here to eternity.
>
> Yes we must improve the sanity checks.
> A lot of options must be tested (or removed) and not only shared libs.
> But the error you reported before (missing export of rte_eth_dev_release_port)
> cannot be seen even with this patch.

I know, I didn't say it would have directly caught it. It would've 
likely been found earlier though, if nothing else then in testing of the 
new librte_pmd_null which clearly nobody had tried in shared lib 
configuration.

> It means we need more tools.
> Though, default configuration is not a tool.

Yes, default config is not a tool, its a recommendation of sorts both 
for developers and users. It also tends to be the setup that is rarely 
broken because it happens to get the most testing :)

>
>> By defaulting to shared we should catch more of these cases early,
>> but without taking away anybodys ability to build static.
>
> Shared libraries are convenient for distributions but have a performance
> impact. I think that static build must remain the default choice.

For distros, this is not a matter of *convenience*, its the only 
technically feasible choice.

I didn't want to make the commit message into a shared library sermon, 
but if you look at the OSS landscape overall the common wisdom is that 
shared library benefits outweigh any performance impact by so much that 
static libs are almost nowhere to be found. I can change the text into a 
full-blown rationale why shared libraries should be the default if that 
makes any difference.

	- Panu -

Re: [dpdk-dev] [PATCH] config: default to shared library

[Neil Horman]

On Wed, Mar 04, 2015 at 01:05:07PM +0200, Panu Matilainen wrote:
> On 03/04/2015 11:24 AM, Thomas Monjalon wrote:
> >Hi Panu,
> >
> >2015-03-04 08:17, Panu Matilainen:
> >>With symbol versioning its vital that developers test their code in
> >>shared library mode, otherwise we'll be playing "add the forgotten
> >>symbol export" from here to eternity.
> >
> >Yes we must improve the sanity checks.
> >A lot of options must be tested (or removed) and not only shared libs.
> >But the error you reported before (missing export of rte_eth_dev_release_port)
> >cannot be seen even with this patch.
> 
> I know, I didn't say it would have directly caught it. It would've likely
> been found earlier though, if nothing else then in testing of the new
> librte_pmd_null which clearly nobody had tried in shared lib configuration.
> 
This is accurate.  The default config is a tool, in the sense that it leverages
the implicit testing of any users who are experimenting with the DPDK.  Any
users out there using the DPDK test/example applications would have realized
something was amiss when the testpmd app refused to run with the null or pcap
pmd, since there was a missing symbol.  That "social fuzzing" has value, but it
only works if the defaults are carefully selected.  Currently, building for
shared libraries exposes more existing bugs than static libraries, and so we
should set that as our default so as to catch them.

> >It means we need more tools.
> >Though, default configuration is not a tool.
> 
> Yes, default config is not a tool, its a recommendation of sorts both for
> developers and users. It also tends to be the setup that is rarely broken
> because it happens to get the most testing :)
> 
And it is a tool (see above).

> >
> >>By defaulting to shared we should catch more of these cases early,
> >>but without taking away anybodys ability to build static.
> >
> >Shared libraries are convenient for distributions but have a performance
> >impact. I think that static build must remain the default choice.
> 

If utmost performance is the concern, isn't it reasonable to assume that users
in that demographic will customize their configuration to achieve that?  No one
assumes that something is tuned to be perfect for their needs out of the box if
their needs are extreemely biased to a single quality.  The best course of
action here is to set the default to be adventageous toward catching bugs, and
document the changes needed to bias for performance.

> For distros, this is not a matter of *convenience*, its the only technically
> feasible choice.
> 
> I didn't want to make the commit message into a shared library sermon, but
> if you look at the OSS landscape overall the common wisdom is that shared
> library benefits outweigh any performance impact by so much that static libs
> are almost nowhere to be found. I can change the text into a full-blown
> rationale why shared libraries should be the default if that makes any
> difference.
> 
Embedded applications actually do make extensive use of static linking to try
achieve greater performance, but they tend to be proprietary, and as such are
the exception that proves the rule.  Once an application itself becomes open
source, it biases toward shared libraries, because the minor performance impact
is well worth the increased manageability and security found in DSO's

Acked-by: Neil Horman <nhorman@tuxdriver.com>

> 	- Panu -
> 
> 

Re: [dpdk-dev] [PATCH] config: default to shared library

[Bruce Richardson]

On Wed, Mar 04, 2015 at 06:28:05AM -0500, Neil Horman wrote:
> On Wed, Mar 04, 2015 at 01:05:07PM +0200, Panu Matilainen wrote:
> > On 03/04/2015 11:24 AM, Thomas Monjalon wrote:
> > >Hi Panu,
> > >
> > >2015-03-04 08:17, Panu Matilainen:
> > >>With symbol versioning its vital that developers test their code in
> > >>shared library mode, otherwise we'll be playing "add the forgotten
> > >>symbol export" from here to eternity.
> > >
> > >Yes we must improve the sanity checks.
> > >A lot of options must be tested (or removed) and not only shared libs.
> > >But the error you reported before (missing export of rte_eth_dev_release_port)
> > >cannot be seen even with this patch.
> > 
> > I know, I didn't say it would have directly caught it. It would've likely
> > been found earlier though, if nothing else then in testing of the new
> > librte_pmd_null which clearly nobody had tried in shared lib configuration.
> > 
> This is accurate.  The default config is a tool, in the sense that it leverages
> the implicit testing of any users who are experimenting with the DPDK.  Any
> users out there using the DPDK test/example applications would have realized
> something was amiss when the testpmd app refused to run with the null or pcap
> pmd, since there was a missing symbol.  That "social fuzzing" has value, but it
> only works if the defaults are carefully selected.  Currently, building for
> shared libraries exposes more existing bugs than static libraries, and so we
> should set that as our default so as to catch them.
> 
> > >It means we need more tools.
> > >Though, default configuration is not a tool.
> > 
> > Yes, default config is not a tool, its a recommendation of sorts both for
> > developers and users. It also tends to be the setup that is rarely broken
> > because it happens to get the most testing :)
> > 
> And it is a tool (see above).
> 
> > >
> > >>By defaulting to shared we should catch more of these cases early,
> > >>but without taking away anybodys ability to build static.
> > >
> > >Shared libraries are convenient for distributions but have a performance
> > >impact. I think that static build must remain the default choice.
> > 
> 
> If utmost performance is the concern, isn't it reasonable to assume that users
> in that demographic will customize their configuration to achieve that?  No one
> assumes that something is tuned to be perfect for their needs out of the box if
> their needs are extreemely biased to a single quality.  The best course of
> action here is to set the default to be adventageous toward catching bugs, and
> document the changes needed to bias for performance.
> 
> > For distros, this is not a matter of *convenience*, its the only technically
> > feasible choice.

As I understand it, build for the "default" cpu rather than "native" is the only
feasible choice also, so how about re-introducing a new defconfig file for
"default" (or perhaps better name), where you have lowest-common denominator
instruction-set and building for shared libraries?
Would that work for everyone, or do people feel it would be too confusing to have
more defconfig files available?

/Bruce

> > 
> > I didn't want to make the commit message into a shared library sermon, but
> > if you look at the OSS landscape overall the common wisdom is that shared
> > library benefits outweigh any performance impact by so much that static libs
> > are almost nowhere to be found. I can change the text into a full-blown
> > rationale why shared libraries should be the default if that makes any
> > difference.
> > 
> Embedded applications actually do make extensive use of static linking to try
> achieve greater performance, but they tend to be proprietary, and as such are
> the exception that proves the rule.  Once an application itself becomes open
> source, it biases toward shared libraries, because the minor performance impact
> is well worth the increased manageability and security found in DSO's
> 
> Acked-by: Neil Horman <nhorman@tuxdriver.com>
> 
> > 	- Panu -
> > 
> > 

Re: [dpdk-dev] [PATCH] config: default to shared library

[Panu Matilainen]

On 03/04/2015 03:08 PM, Bruce Richardson wrote:
> On Wed, Mar 04, 2015 at 06:28:05AM -0500, Neil Horman wrote:
>> On Wed, Mar 04, 2015 at 01:05:07PM +0200, Panu Matilainen wrote:
>>> On 03/04/2015 11:24 AM, Thomas Monjalon wrote:
>>>> Hi Panu,
>>>>
>>>> 2015-03-04 08:17, Panu Matilainen:
>>>>> With symbol versioning its vital that developers test their code in
>>>>> shared library mode, otherwise we'll be playing "add the forgotten
>>>>> symbol export" from here to eternity.
>>>>
>>>> Yes we must improve the sanity checks.
>>>> A lot of options must be tested (or removed) and not only shared libs.
>>>> But the error you reported before (missing export of rte_eth_dev_release_port)
>>>> cannot be seen even with this patch.
>>>
>>> I know, I didn't say it would have directly caught it. It would've likely
>>> been found earlier though, if nothing else then in testing of the new
>>> librte_pmd_null which clearly nobody had tried in shared lib configuration.
>>>
>> This is accurate.  The default config is a tool, in the sense that it leverages
>> the implicit testing of any users who are experimenting with the DPDK.  Any
>> users out there using the DPDK test/example applications would have realized
>> something was amiss when the testpmd app refused to run with the null or pcap
>> pmd, since there was a missing symbol.  That "social fuzzing" has value, but it
>> only works if the defaults are carefully selected.  Currently, building for
>> shared libraries exposes more existing bugs than static libraries, and so we
>> should set that as our default so as to catch them.
>>
>>>> It means we need more tools.
>>>> Though, default configuration is not a tool.
>>>
>>> Yes, default config is not a tool, its a recommendation of sorts both for
>>> developers and users. It also tends to be the setup that is rarely broken
>>> because it happens to get the most testing :)
>>>
>> And it is a tool (see above).
>>
>>>>
>>>>> By defaulting to shared we should catch more of these cases early,
>>>>> but without taking away anybodys ability to build static.
>>>>
>>>> Shared libraries are convenient for distributions but have a performance
>>>> impact. I think that static build must remain the default choice.
>>>
>>
>> If utmost performance is the concern, isn't it reasonable to assume that users
>> in that demographic will customize their configuration to achieve that?  No one
>> assumes that something is tuned to be perfect for their needs out of the box if
>> their needs are extreemely biased to a single quality.  The best course of
>> action here is to set the default to be adventageous toward catching bugs, and
>> document the changes needed to bias for performance.
>>
>>> For distros, this is not a matter of *convenience*, its the only technically
>>> feasible choice.
>
> As I understand it, build for the "default" cpu rather than "native" is the only
> feasible choice also, so how about re-introducing a new defconfig file for
> "default" (or perhaps better name), where you have lowest-common denominator
> instruction-set and building for shared libraries?
> Would that work for everyone, or do people feel it would be too confusing to have
> more defconfig files available?

Given the opposition to defaulting to shared, another config file seems 
like a fair compromise to me, whether "default" or something else. As 
for the naming, one possibility would be calling it "shared", implying 
both lowest-common denominator instruction set to be shareable across 
many systems and shared libraries.

	- Panu -

Re: [dpdk-dev] [PATCH] config: default to shared library

[Bruce Richardson]

On Wed, Mar 04, 2015 at 03:24:12PM +0200, Panu Matilainen wrote:
> On 03/04/2015 03:08 PM, Bruce Richardson wrote:
> >On Wed, Mar 04, 2015 at 06:28:05AM -0500, Neil Horman wrote:
> >>On Wed, Mar 04, 2015 at 01:05:07PM +0200, Panu Matilainen wrote:
> >>>On 03/04/2015 11:24 AM, Thomas Monjalon wrote:
> >>>>Hi Panu,
> >>>>
> >>>>2015-03-04 08:17, Panu Matilainen:
> >>>>>With symbol versioning its vital that developers test their code in
> >>>>>shared library mode, otherwise we'll be playing "add the forgotten
> >>>>>symbol export" from here to eternity.
> >>>>
> >>>>Yes we must improve the sanity checks.
> >>>>A lot of options must be tested (or removed) and not only shared libs.
> >>>>But the error you reported before (missing export of rte_eth_dev_release_port)
> >>>>cannot be seen even with this patch.
> >>>
> >>>I know, I didn't say it would have directly caught it. It would've likely
> >>>been found earlier though, if nothing else then in testing of the new
> >>>librte_pmd_null which clearly nobody had tried in shared lib configuration.
> >>>
> >>This is accurate.  The default config is a tool, in the sense that it leverages
> >>the implicit testing of any users who are experimenting with the DPDK.  Any
> >>users out there using the DPDK test/example applications would have realized
> >>something was amiss when the testpmd app refused to run with the null or pcap
> >>pmd, since there was a missing symbol.  That "social fuzzing" has value, but it
> >>only works if the defaults are carefully selected.  Currently, building for
> >>shared libraries exposes more existing bugs than static libraries, and so we
> >>should set that as our default so as to catch them.
> >>
> >>>>It means we need more tools.
> >>>>Though, default configuration is not a tool.
> >>>
> >>>Yes, default config is not a tool, its a recommendation of sorts both for
> >>>developers and users. It also tends to be the setup that is rarely broken
> >>>because it happens to get the most testing :)
> >>>
> >>And it is a tool (see above).
> >>
> >>>>
> >>>>>By defaulting to shared we should catch more of these cases early,
> >>>>>but without taking away anybodys ability to build static.
> >>>>
> >>>>Shared libraries are convenient for distributions but have a performance
> >>>>impact. I think that static build must remain the default choice.
> >>>
> >>
> >>If utmost performance is the concern, isn't it reasonable to assume that users
> >>in that demographic will customize their configuration to achieve that?  No one
> >>assumes that something is tuned to be perfect for their needs out of the box if
> >>their needs are extreemely biased to a single quality.  The best course of
> >>action here is to set the default to be adventageous toward catching bugs, and
> >>document the changes needed to bias for performance.
> >>
> >>>For distros, this is not a matter of *convenience*, its the only technically
> >>>feasible choice.
> >
> >As I understand it, build for the "default" cpu rather than "native" is the only
> >feasible choice also, so how about re-introducing a new defconfig file for
> >"default" (or perhaps better name), where you have lowest-common denominator
> >instruction-set and building for shared libraries?
> >Would that work for everyone, or do people feel it would be too confusing to have
> >more defconfig files available?
> 
> Given the opposition to defaulting to shared, another config file seems like
> a fair compromise to me, whether "default" or something else. As for the
> naming, one possibility would be calling it "shared", implying both
> lowest-common denominator instruction set to be shareable across many
> systems and shared libraries.
> 
> 	- Panu -

The naming scheme for configs is meant to be:
"ARCH-MACHINE-EXECENV-TOOLCHAIN" 
as documented in the Getting Started Guide. "Default" has been used up till now
to refer to the lowest common denominator instruction set supported, which for
x86_64 is a core2 baseline, I believe. "shared" doesn't really fit into this
naming scheme, and there is nothing to allow extra notes to be added to the
name.
Without changing this scheme, I would suggest we rename "default" to "generic",
which I think is a slightly better term for it, and we set the
"x86_64-generic-linuxapp-gcc" target to build shared libs.

/Bruce

Re: [dpdk-dev] [PATCH] config: default to shared library

[Panu Matilainen]

On 03/04/2015 03:31 PM, Bruce Richardson wrote:
> On Wed, Mar 04, 2015 at 03:24:12PM +0200, Panu Matilainen wrote:
>> On 03/04/2015 03:08 PM, Bruce Richardson wrote:
>>> On Wed, Mar 04, 2015 at 06:28:05AM -0500, Neil Horman wrote:
>>>> On Wed, Mar 04, 2015 at 01:05:07PM +0200, Panu Matilainen wrote:
>>>>> On 03/04/2015 11:24 AM, Thomas Monjalon wrote:
>>>>>> Hi Panu,
>>>>>>
>>>>>> 2015-03-04 08:17, Panu Matilainen:
>>>>>>> With symbol versioning its vital that developers test their code in
>>>>>>> shared library mode, otherwise we'll be playing "add the forgotten
>>>>>>> symbol export" from here to eternity.
>>>>>>
>>>>>> Yes we must improve the sanity checks.
>>>>>> A lot of options must be tested (or removed) and not only shared libs.
>>>>>> But the error you reported before (missing export of rte_eth_dev_release_port)
>>>>>> cannot be seen even with this patch.
>>>>>
>>>>> I know, I didn't say it would have directly caught it. It would've likely
>>>>> been found earlier though, if nothing else then in testing of the new
>>>>> librte_pmd_null which clearly nobody had tried in shared lib configuration.
>>>>>
>>>> This is accurate.  The default config is a tool, in the sense that it leverages
>>>> the implicit testing of any users who are experimenting with the DPDK.  Any
>>>> users out there using the DPDK test/example applications would have realized
>>>> something was amiss when the testpmd app refused to run with the null or pcap
>>>> pmd, since there was a missing symbol.  That "social fuzzing" has value, but it
>>>> only works if the defaults are carefully selected.  Currently, building for
>>>> shared libraries exposes more existing bugs than static libraries, and so we
>>>> should set that as our default so as to catch them.
>>>>
>>>>>> It means we need more tools.
>>>>>> Though, default configuration is not a tool.
>>>>>
>>>>> Yes, default config is not a tool, its a recommendation of sorts both for
>>>>> developers and users. It also tends to be the setup that is rarely broken
>>>>> because it happens to get the most testing :)
>>>>>
>>>> And it is a tool (see above).
>>>>
>>>>>>
>>>>>>> By defaulting to shared we should catch more of these cases early,
>>>>>>> but without taking away anybodys ability to build static.
>>>>>>
>>>>>> Shared libraries are convenient for distributions but have a performance
>>>>>> impact. I think that static build must remain the default choice.
>>>>>
>>>>
>>>> If utmost performance is the concern, isn't it reasonable to assume that users
>>>> in that demographic will customize their configuration to achieve that?  No one
>>>> assumes that something is tuned to be perfect for their needs out of the box if
>>>> their needs are extreemely biased to a single quality.  The best course of
>>>> action here is to set the default to be adventageous toward catching bugs, and
>>>> document the changes needed to bias for performance.
>>>>
>>>>> For distros, this is not a matter of *convenience*, its the only technically
>>>>> feasible choice.
>>>
>>> As I understand it, build for the "default" cpu rather than "native" is the only
>>> feasible choice also, so how about re-introducing a new defconfig file for
>>> "default" (or perhaps better name), where you have lowest-common denominator
>>> instruction-set and building for shared libraries?
>>> Would that work for everyone, or do people feel it would be too confusing to have
>>> more defconfig files available?
>>
>> Given the opposition to defaulting to shared, another config file seems like
>> a fair compromise to me, whether "default" or something else. As for the
>> naming, one possibility would be calling it "shared", implying both
>> lowest-common denominator instruction set to be shareable across many
>> systems and shared libraries.
>>
>> 	- Panu -
>
> The naming scheme for configs is meant to be:
> "ARCH-MACHINE-EXECENV-TOOLCHAIN"
> as documented in the Getting Started Guide. "Default" has been used up till now
> to refer to the lowest common denominator instruction set supported, which for
> x86_64 is a core2 baseline, I believe. "shared" doesn't really fit into this
> naming scheme, and there is nothing to allow extra notes to be added to the
> name.

Right, but then there's "ivshmem" that doesn't fit that description 
either AFAICS.

> Without changing this scheme, I would suggest we rename "default" to "generic",
> which I think is a slightly better term for it, and we set the
> "x86_64-generic-linuxapp-gcc" target to build shared libs.

Works for me. It is indeed more descriptive than either "default" or 
"shared" for the purpose.

	- Panu -

Re: [dpdk-dev] [PATCH] config: default to shared library

[Bruce Richardson]

On Wed, Mar 04, 2015 at 03:41:49PM +0200, Panu Matilainen wrote:
> On 03/04/2015 03:31 PM, Bruce Richardson wrote:
> >On Wed, Mar 04, 2015 at 03:24:12PM +0200, Panu Matilainen wrote:
> >>On 03/04/2015 03:08 PM, Bruce Richardson wrote:
> >>>On Wed, Mar 04, 2015 at 06:28:05AM -0500, Neil Horman wrote:
> >>>>On Wed, Mar 04, 2015 at 01:05:07PM +0200, Panu Matilainen wrote:
> >>>>>On 03/04/2015 11:24 AM, Thomas Monjalon wrote:
> >>>>>>Hi Panu,
> >>>>>>
> >>>>>>2015-03-04 08:17, Panu Matilainen:
> >>>>>>>With symbol versioning its vital that developers test their code in
> >>>>>>>shared library mode, otherwise we'll be playing "add the forgotten
> >>>>>>>symbol export" from here to eternity.
> >>>>>>
> >>>>>>Yes we must improve the sanity checks.
> >>>>>>A lot of options must be tested (or removed) and not only shared libs.
> >>>>>>But the error you reported before (missing export of rte_eth_dev_release_port)
> >>>>>>cannot be seen even with this patch.
> >>>>>
> >>>>>I know, I didn't say it would have directly caught it. It would've likely
> >>>>>been found earlier though, if nothing else then in testing of the new
> >>>>>librte_pmd_null which clearly nobody had tried in shared lib configuration.
> >>>>>
> >>>>This is accurate.  The default config is a tool, in the sense that it leverages
> >>>>the implicit testing of any users who are experimenting with the DPDK.  Any
> >>>>users out there using the DPDK test/example applications would have realized
> >>>>something was amiss when the testpmd app refused to run with the null or pcap
> >>>>pmd, since there was a missing symbol.  That "social fuzzing" has value, but it
> >>>>only works if the defaults are carefully selected.  Currently, building for
> >>>>shared libraries exposes more existing bugs than static libraries, and so we
> >>>>should set that as our default so as to catch them.
> >>>>
> >>>>>>It means we need more tools.
> >>>>>>Though, default configuration is not a tool.
> >>>>>
> >>>>>Yes, default config is not a tool, its a recommendation of sorts both for
> >>>>>developers and users. It also tends to be the setup that is rarely broken
> >>>>>because it happens to get the most testing :)
> >>>>>
> >>>>And it is a tool (see above).
> >>>>
> >>>>>>
> >>>>>>>By defaulting to shared we should catch more of these cases early,
> >>>>>>>but without taking away anybodys ability to build static.
> >>>>>>
> >>>>>>Shared libraries are convenient for distributions but have a performance
> >>>>>>impact. I think that static build must remain the default choice.
> >>>>>
> >>>>
> >>>>If utmost performance is the concern, isn't it reasonable to assume that users
> >>>>in that demographic will customize their configuration to achieve that?  No one
> >>>>assumes that something is tuned to be perfect for their needs out of the box if
> >>>>their needs are extreemely biased to a single quality.  The best course of
> >>>>action here is to set the default to be adventageous toward catching bugs, and
> >>>>document the changes needed to bias for performance.
> >>>>
> >>>>>For distros, this is not a matter of *convenience*, its the only technically
> >>>>>feasible choice.
> >>>
> >>>As I understand it, build for the "default" cpu rather than "native" is the only
> >>>feasible choice also, so how about re-introducing a new defconfig file for
> >>>"default" (or perhaps better name), where you have lowest-common denominator
> >>>instruction-set and building for shared libraries?
> >>>Would that work for everyone, or do people feel it would be too confusing to have
> >>>more defconfig files available?
> >>
> >>Given the opposition to defaulting to shared, another config file seems like
> >>a fair compromise to me, whether "default" or something else. As for the
> >>naming, one possibility would be calling it "shared", implying both
> >>lowest-common denominator instruction set to be shareable across many
> >>systems and shared libraries.
> >>
> >>	- Panu -
> >
> >The naming scheme for configs is meant to be:
> >"ARCH-MACHINE-EXECENV-TOOLCHAIN"
> >as documented in the Getting Started Guide. "Default" has been used up till now
> >to refer to the lowest common denominator instruction set supported, which for
> >x86_64 is a core2 baseline, I believe. "shared" doesn't really fit into this
> >naming scheme, and there is nothing to allow extra notes to be added to the
> >name.
> 
> Right, but then there's "ivshmem" that doesn't fit that description either
> AFAICS.

Ah, yes, forgotten about that one! :-)

> 
> >Without changing this scheme, I would suggest we rename "default" to "generic",
> >which I think is a slightly better term for it, and we set the
> >"x86_64-generic-linuxapp-gcc" target to build shared libs.
> 
> Works for me. It is indeed more descriptive than either "default" or
> "shared" for the purpose.
> 
> 	- Panu -
> 

Re: [dpdk-dev] [PATCH] config: default to shared library

[Thomas Monjalon]

2015-03-04 13:49, Bruce Richardson:
> On Wed, Mar 04, 2015 at 03:41:49PM +0200, Panu Matilainen wrote:
> > On 03/04/2015 03:31 PM, Bruce Richardson wrote:
> > >On Wed, Mar 04, 2015 at 03:24:12PM +0200, Panu Matilainen wrote:
> > >>On 03/04/2015 03:08 PM, Bruce Richardson wrote:
> > >>>On Wed, Mar 04, 2015 at 06:28:05AM -0500, Neil Horman wrote:
> > >>>>On Wed, Mar 04, 2015 at 01:05:07PM +0200, Panu Matilainen wrote:
> > >>>>>On 03/04/2015 11:24 AM, Thomas Monjalon wrote:
> > >>>>>>Hi Panu,
> > >>>>>>
> > >>>>>>2015-03-04 08:17, Panu Matilainen:
> > >>>>>>>With symbol versioning its vital that developers test their code in
> > >>>>>>>shared library mode, otherwise we'll be playing "add the forgotten
> > >>>>>>>symbol export" from here to eternity.
> > >>>>>>
> > >>>>>>Yes we must improve the sanity checks.
> > >>>>>>A lot of options must be tested (or removed) and not only shared libs.
> > >>>>>>But the error you reported before (missing export of rte_eth_dev_release_port)
> > >>>>>>cannot be seen even with this patch.
> > >>>>>
> > >>>>>I know, I didn't say it would have directly caught it. It would've likely
> > >>>>>been found earlier though, if nothing else then in testing of the new
> > >>>>>librte_pmd_null which clearly nobody had tried in shared lib configuration.
> > >>>>>
> > >>>>This is accurate.  The default config is a tool, in the sense that it leverages
> > >>>>the implicit testing of any users who are experimenting with the DPDK.  Any
> > >>>>users out there using the DPDK test/example applications would have realized
> > >>>>something was amiss when the testpmd app refused to run with the null or pcap
> > >>>>pmd, since there was a missing symbol.  That "social fuzzing" has value, but it
> > >>>>only works if the defaults are carefully selected.  Currently, building for
> > >>>>shared libraries exposes more existing bugs than static libraries, and so we
> > >>>>should set that as our default so as to catch them.
> > >>>>
> > >>>>>>It means we need more tools.
> > >>>>>>Though, default configuration is not a tool.
> > >>>>>
> > >>>>>Yes, default config is not a tool, its a recommendation of sorts both for
> > >>>>>developers and users. It also tends to be the setup that is rarely broken
> > >>>>>because it happens to get the most testing :)
> > >>>>>
> > >>>>And it is a tool (see above).
> > >>>>
> > >>>>>>
> > >>>>>>>By defaulting to shared we should catch more of these cases early,
> > >>>>>>>but without taking away anybodys ability to build static.
> > >>>>>>
> > >>>>>>Shared libraries are convenient for distributions but have a performance
> > >>>>>>impact. I think that static build must remain the default choice.
> > >>>>>
> > >>>>
> > >>>>If utmost performance is the concern, isn't it reasonable to assume that users
> > >>>>in that demographic will customize their configuration to achieve that?  No one
> > >>>>assumes that something is tuned to be perfect for their needs out of the box if
> > >>>>their needs are extreemely biased to a single quality.  The best course of
> > >>>>action here is to set the default to be adventageous toward catching bugs, and
> > >>>>document the changes needed to bias for performance.
> > >>>>
> > >>>>>For distros, this is not a matter of *convenience*, its the only technically
> > >>>>>feasible choice.
> > >>>
> > >>>As I understand it, build for the "default" cpu rather than "native" is the only
> > >>>feasible choice also, so how about re-introducing a new defconfig file for
> > >>>"default" (or perhaps better name), where you have lowest-common denominator
> > >>>instruction-set and building for shared libraries?
> > >>>Would that work for everyone, or do people feel it would be too confusing to have
> > >>>more defconfig files available?
> > >>
> > >>Given the opposition to defaulting to shared, another config file seems like
> > >>a fair compromise to me, whether "default" or something else. As for the
> > >>naming, one possibility would be calling it "shared", implying both
> > >>lowest-common denominator instruction set to be shareable across many
> > >>systems and shared libraries.
> > >>
> > >>	- Panu -
> > >
> > >The naming scheme for configs is meant to be:
> > >"ARCH-MACHINE-EXECENV-TOOLCHAIN"
> > >as documented in the Getting Started Guide. "Default" has been used up till now
> > >to refer to the lowest common denominator instruction set supported, which for
> > >x86_64 is a core2 baseline, I believe. "shared" doesn't really fit into this
> > >naming scheme, and there is nothing to allow extra notes to be added to the
> > >name.
> > 
> > Right, but then there's "ivshmem" that doesn't fit that description either
> > AFAICS.
> 
> Ah, yes, forgotten about that one! :-)

off-topic:
I think we should remove "#ifdef RTE_LIBRTE_IVSHMEM" in EAL and then remove
defconfig_x86_64-ivshmem-linuxapp-*.

> > >Without changing this scheme, I would suggest we rename "default" to "generic",
> > >which I think is a slightly better term for it, and we set the
> > >"x86_64-generic-linuxapp-gcc" target to build shared libs.
> > 
> > Works for me. It is indeed more descriptive than either "default" or
> > "shared" for the purpose.

+1 for x86_64-generic-linuxapp-gcc

Re: [dpdk-dev] [PATCH] config: default to shared library

[David Marchand]

On Wed, Mar 4, 2015 at 2:49 PM, Bruce Richardson <bruce.richardson@intel.com
> wrote:

> On Wed, Mar 04, 2015 at 03:41:49PM +0200, Panu Matilainen wrote:
> > Right, but then there's "ivshmem" that doesn't fit that description
> either
> > AFAICS.
>
> Ah, yes, forgotten about that one! :-)


Well, this is out of scope, but this config file should not exist in the
first place.
>From my point of view, the ivshmem implementation is just badly hooked into
the eal, and this is the only reason why ivshmem should have a build option
(disabled) by default.

If we could cleanup this, then there would be no exception to the naming
convention.


-- 
David Marchand

Re: [dpdk-dev] [PATCH] config: default to shared library

[Bruce Richardson]

On Wed, Mar 04, 2015 at 02:57:50PM +0100, David Marchand wrote:
> On Wed, Mar 4, 2015 at 2:49 PM, Bruce Richardson <bruce.richardson@intel.com
> > wrote:
> 
> > On Wed, Mar 04, 2015 at 03:41:49PM +0200, Panu Matilainen wrote:
> > > Right, but then there's "ivshmem" that doesn't fit that description
> > either
> > > AFAICS.
> >
> > Ah, yes, forgotten about that one! :-)
> 
> 
> Well, this is out of scope, but this config file should not exist in the
> first place.
> From my point of view, the ivshmem implementation is just badly hooked into
> the eal, and this is the only reason why ivshmem should have a build option
> (disabled) by default.
> 
> If we could cleanup this, then there would be no exception to the naming
> convention.
> 
No objections here! :-)

> 
> -- 
> David Marchand

Re: [dpdk-dev] [PATCH] config: default to shared library

[Thomas Monjalon]

2015-03-04 13:05, Panu Matilainen:
> On 03/04/2015 11:24 AM, Thomas Monjalon wrote:
> > Hi Panu,
> >
> > 2015-03-04 08:17, Panu Matilainen:
> >> With symbol versioning its vital that developers test their code in
> >> shared library mode, otherwise we'll be playing "add the forgotten
> >> symbol export" from here to eternity.
> >
> > Yes we must improve the sanity checks.
> > A lot of options must be tested (or removed) and not only shared libs.
> > But the error you reported before (missing export of rte_eth_dev_release_port)
> > cannot be seen even with this patch.
> 
> I know, I didn't say it would have directly caught it. It would've 
> likely been found earlier though, if nothing else then in testing of the 
> new librte_pmd_null which clearly nobody had tried in shared lib 
> configuration.

Exactly, there was no test in shared lib mode.
Then I recommend having a basic testpmd trial which should be easy with
PMD null.
Any script to ease this test is welcome.

> > It means we need more tools.
> > Though, default configuration is not a tool.
> 
> Yes, default config is not a tool, its a recommendation of sorts both 
> for developers and users. It also tends to be the setup that is rarely 
> broken because it happens to get the most testing :)
> 
> >
> >> By defaulting to shared we should catch more of these cases early,
> >> but without taking away anybodys ability to build static.
> >
> > Shared libraries are convenient for distributions but have a performance
> > impact. I think that static build must remain the default choice.
> 
> For distros, this is not a matter of *convenience*, its the only 
> technically feasible choice.

"only" is a religious word here :) We could imagine an automatic rebuild
of DPDK applications when DPDK is updated. But it's not the topic :)

> I didn't want to make the commit message into a shared library sermon, 
> but if you look at the OSS landscape overall the common wisdom is that 
> shared library benefits outweigh any performance impact by so much that 
> static libs are almost nowhere to be found. I can change the text into a 
> full-blown rationale why shared libraries should be the default if that 
> makes any difference.

No, as Bruce agreed, it's better to keep static as default, because DPDK is
performance-oriented.
I agree we really have a problem with tests and I'd prefer we solve it by
adding some scripts. I'm currently thinking about which scripts to add,
like building different configurations. Then the second step will be to
automate the launch of these scripts to catch wrong submissions earlier.
Maybe that something like patchew (http://qemu.patchew.org) could help for
the second step (thanks David for the tip).