[dpdk-stable] [dpdk-dev] [PATCH] devtools: skip the symbol check when map file under drivers

[dpdk-stable] [dpdk-dev] [PATCH] devtools: skip the symbol check when map file under drivers

[jerinj]

From: Jerin Jacob <jerinj@marvell.com>

Drivers do not have ABI.
Skip the symbol check if map file under drivers directory.

Fixes: 4bec48184e33 ("devtools: add checks for ABI symbol addition")

Cc: stable@dpdk.org
Cc: Neil Horman <nhorman@tuxdriver.com>

Signed-off-by: Jerin Jacob <jerinj@marvell.com>
---
 devtools/check-symbol-change.sh | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/devtools/check-symbol-change.sh b/devtools/check-symbol-change.sh
index c5434f3bb..444beddad 100755
--- a/devtools/check-symbol-change.sh
+++ b/devtools/check-symbol-change.sh
@@ -93,6 +93,14 @@ check_for_rule_violations()
 		if [ "$ar" = "add" ]
 		then
 
+			directory=`echo $mname | cut -d / -f 2`
+			if [ "$directory" = "drivers" ]
+			then
+				# Drivers do not have ABI. Skip further
+				# processing if the map file is under
+				# drivers directory
+				continue
+			fi
 			if [ "$secname" = "unknown" ]
 			then
 				# Just inform the user of this occurrence, but
-- 
2.21.0

Re: [dpdk-stable] [dpdk-dev] [PATCH] devtools: skip the symbol check when map file under drivers

[Neil Horman]

On Wed, May 22, 2019 at 01:26:28AM +0530, jerinj@marvell.com wrote:
> From: Jerin Jacob <jerinj@marvell.com>
> 
> Drivers do not have ABI.
> Skip the symbol check if map file under drivers directory.
> 
> Fixes: 4bec48184e33 ("devtools: add checks for ABI symbol addition")
> 
> Cc: stable@dpdk.org
> Cc: Neil Horman <nhorman@tuxdriver.com>
> 
Sorry, but I'm not ok with this, because many of our DPDK PMDs have functions
that get exported which are meant to be called by applications directly.  The
dpaa2 driver is a good example, the cryptodev scheduler is another.  Take a look
at their version.map files to see what I mean.

Unless we are willing to make drivers opaque objects that are only accessible
from the [eth|crypto|etc]dev apis in the DPDK core, we have the potential for
exported symbols, which means we have an ABI that has to be maintained, or at
least recognized and reviewed for consistency

Nacked-by: Neil Horman <nhorman@tuxdriver.com>


> Signed-off-by: Jerin Jacob <jerinj@marvell.com>
> ---
>  devtools/check-symbol-change.sh | 8 ++++++++
>  1 file changed, 8 insertions(+)
> 
> diff --git a/devtools/check-symbol-change.sh b/devtools/check-symbol-change.sh
> index c5434f3bb..444beddad 100755
> --- a/devtools/check-symbol-change.sh
> +++ b/devtools/check-symbol-change.sh
> @@ -93,6 +93,14 @@ check_for_rule_violations()
>  		if [ "$ar" = "add" ]
>  		then
>  
> +			directory=`echo $mname | cut -d / -f 2`
> +			if [ "$directory" = "drivers" ]
> +			then
> +				# Drivers do not have ABI. Skip further
> +				# processing if the map file is under
> +				# drivers directory
> +				continue
> +			fi
>  			if [ "$secname" = "unknown" ]
>  			then
>  				# Just inform the user of this occurrence, but
> -- 
> 2.21.0
> 
> 

Re: [dpdk-stable] [EXT] Re: [dpdk-dev] [PATCH] devtools: skip the symbol check when map file under drivers

[Jerin Jacob Kollanukkaran]

> -----Original Message-----
> From: Neil Horman <nhorman@tuxdriver.com>
> Sent: Wednesday, May 22, 2019 1:57 AM
> To: Jerin Jacob Kollanukkaran <jerinj@marvell.com>
> Cc: dev@dpdk.org; thomas@monjalon.net; stable@dpdk.org
> Subject: [EXT] Re: [dpdk-dev] [PATCH] devtools: skip the symbol check when
> map file under drivers
> 
> On Wed, May 22, 2019 at 01:26:28AM +0530, jerinj@marvell.com wrote:
> > From: Jerin Jacob <jerinj@marvell.com>
> >
> > Drivers do not have ABI.
> > Skip the symbol check if map file under drivers directory.
> >
> > Fixes: 4bec48184e33 ("devtools: add checks for ABI symbol addition")
> >
> > Cc: stable@dpdk.org
> > Cc: Neil Horman <nhorman@tuxdriver.com>
> >
> Sorry, but I'm not ok with this, because many of our DPDK PMDs have functions
> that get exported which are meant to be called by applications directly.  The

OK. Just to update my knowledge, Should those API needs to go through ABI/API
depreciation process?

Actually, I am concerned about the APIs, which is called between drviers not
the application. For example,
drivers/common/dpaax/rte_common_dpaax_version.map

it is not interface to application rather it is for intra driver case.
I think, I can change my logic to Skip the symbols which NOT starting with rte_.
Agree?

Context:
I am adding a new driver/common/octeontx2 directory and it has some API which
Needs to shared between drivers not to the application. For me, it does not make
sense to go through any ABI process in such case.


> dpaa2 driver is a good example, the cryptodev scheduler is another.  Take a look
> at their version.map files to see what I mean.
> 
> Unless we are willing to make drivers opaque objects that are only accessible
> from the [eth|crypto|etc]dev apis in the DPDK core, we have the potential for
> exported symbols, which means we have an ABI that has to be maintained, or at
> least recognized and reviewed for consistency
> 
> Nacked-by: Neil Horman <nhorman@tuxdriver.com>
> 
> 
> > Signed-off-by: Jerin Jacob <jerinj@marvell.com>
> > ---
> >  devtools/check-symbol-change.sh | 8 ++++++++
> >  1 file changed, 8 insertions(+)
> >
> > diff --git a/devtools/check-symbol-change.sh
> > b/devtools/check-symbol-change.sh index c5434f3bb..444beddad 100755
> > --- a/devtools/check-symbol-change.sh
> > +++ b/devtools/check-symbol-change.sh
> > @@ -93,6 +93,14 @@ check_for_rule_violations()
> >  		if [ "$ar" = "add" ]
> >  		then
> >
> > +			directory=`echo $mname | cut -d / -f 2`
> > +			if [ "$directory" = "drivers" ]
> > +			then
> > +				# Drivers do not have ABI. Skip further
> > +				# processing if the map file is under
> > +				# drivers directory
> > +				continue
> > +			fi
> >  			if [ "$secname" = "unknown" ]
> >  			then
> >  				# Just inform the user of this occurrence, but
> > --
> > 2.21.0
> >
> >

Re: [dpdk-stable] [dpdk-dev] [EXT] Re: [PATCH] devtools: skip the symbol check when map file under drivers

[Bruce Richardson]

On Wed, May 22, 2019 at 03:05:54AM +0000, Jerin Jacob Kollanukkaran wrote:
> > -----Original Message-----
> > From: Neil Horman <nhorman@tuxdriver.com>
> > Sent: Wednesday, May 22, 2019 1:57 AM
> > To: Jerin Jacob Kollanukkaran <jerinj@marvell.com>
> > Cc: dev@dpdk.org; thomas@monjalon.net; stable@dpdk.org
> > Subject: [EXT] Re: [dpdk-dev] [PATCH] devtools: skip the symbol check when
> > map file under drivers
> > 
> > On Wed, May 22, 2019 at 01:26:28AM +0530, jerinj@marvell.com wrote:
> > > From: Jerin Jacob <jerinj@marvell.com>
> > >
> > > Drivers do not have ABI.
> > > Skip the symbol check if map file under drivers directory.
> > >
> > > Fixes: 4bec48184e33 ("devtools: add checks for ABI symbol addition")
> > >
> > > Cc: stable@dpdk.org
> > > Cc: Neil Horman <nhorman@tuxdriver.com>
> > >
> > Sorry, but I'm not ok with this, because many of our DPDK PMDs have functions
> > that get exported which are meant to be called by applications directly.  The
> 
> OK. Just to update my knowledge, Should those API needs to go through ABI/API
> depreciation process?
> 
> Actually, I am concerned about the APIs, which is called between drviers not
> the application. For example,
> drivers/common/dpaax/rte_common_dpaax_version.map
> 
> it is not interface to application rather it is for intra driver case.
> I think, I can change my logic to Skip the symbols which NOT starting with rte_.
> Agree?
> 
> Context:
> I am adding a new driver/common/octeontx2 directory and it has some API which
> Needs to shared between drivers not to the application. For me, it does not make
> sense to go through any ABI process in such case.
> 
> 
Maybe not, but other drivers will have APIs designed for apps to call
directly - some NIC drivers have them, and I suspect that rawdev drivers
will need them a lot. Therefore, it's best to have the drivers directory
scanned by our tooling. 

Re: [dpdk-stable] [EXT] Re: [dpdk-dev] [PATCH] devtools: skip the symbol check when map file under drivers

[Neil Horman]

On Wed, May 22, 2019 at 03:05:54AM +0000, Jerin Jacob Kollanukkaran wrote:
> > -----Original Message-----
> > From: Neil Horman <nhorman@tuxdriver.com>
> > Sent: Wednesday, May 22, 2019 1:57 AM
> > To: Jerin Jacob Kollanukkaran <jerinj@marvell.com>
> > Cc: dev@dpdk.org; thomas@monjalon.net; stable@dpdk.org
> > Subject: [EXT] Re: [dpdk-dev] [PATCH] devtools: skip the symbol check when
> > map file under drivers
> > 
> > On Wed, May 22, 2019 at 01:26:28AM +0530, jerinj@marvell.com wrote:
> > > From: Jerin Jacob <jerinj@marvell.com>
> > >
> > > Drivers do not have ABI.
> > > Skip the symbol check if map file under drivers directory.
> > >
> > > Fixes: 4bec48184e33 ("devtools: add checks for ABI symbol addition")
> > >
> > > Cc: stable@dpdk.org
> > > Cc: Neil Horman <nhorman@tuxdriver.com>
> > >
> > Sorry, but I'm not ok with this, because many of our DPDK PMDs have functions
> > that get exported which are meant to be called by applications directly.  The
> 
> OK. Just to update my knowledge, Should those API needs to go through ABI/API
> depreciation process?
> 
Yes, they definately should, they are API's just as any other in the core DPDK
library.

> Actually, I am concerned about the APIs, which is called between drviers not
> the application. For example,
> drivers/common/dpaax/rte_common_dpaax_version.map
> 
> it is not interface to application rather it is for intra driver case.
> I think, I can change my logic to Skip the symbols which NOT starting with rte_.
> Agree?
> 
No, Thats just one case, and if those calls are between drivers, so be it, but
those still need to be stable, and we have other examples (like the bonding or
dummy driver), which have additional APIs that are explicitly meant to be used
by an application.

> Context:
> I am adding a new driver/common/octeontx2 directory and it has some API which
> Needs to shared between drivers not to the application. For me, it does not make
> sense to go through any ABI process in such case.
> 
Why?  If you create an API thats reachable from another block of code (be it a
driver or an application), you've created a dependency in which that API must
remain stable, lest you risk breaking something.  If an end user writes an
out-of-tree PMD which makes use of an an additional driver API, then you need to
keep it stable or you will break them.

> 
> > dpaa2 driver is a good example, the cryptodev scheduler is another.  Take a look
> > at their version.map files to see what I mean.
> > 
> > Unless we are willing to make drivers opaque objects that are only accessible
> > from the [eth|crypto|etc]dev apis in the DPDK core, we have the potential for
> > exported symbols, which means we have an ABI that has to be maintained, or at
> > least recognized and reviewed for consistency
> > 
> > Nacked-by: Neil Horman <nhorman@tuxdriver.com>
> > 
> > 
> > > Signed-off-by: Jerin Jacob <jerinj@marvell.com>
> > > ---
> > >  devtools/check-symbol-change.sh | 8 ++++++++
> > >  1 file changed, 8 insertions(+)
> > >
> > > diff --git a/devtools/check-symbol-change.sh
> > > b/devtools/check-symbol-change.sh index c5434f3bb..444beddad 100755
> > > --- a/devtools/check-symbol-change.sh
> > > +++ b/devtools/check-symbol-change.sh
> > > @@ -93,6 +93,14 @@ check_for_rule_violations()
> > >  		if [ "$ar" = "add" ]
> > >  		then
> > >
> > > +			directory=`echo $mname | cut -d / -f 2`
> > > +			if [ "$directory" = "drivers" ]
> > > +			then
> > > +				# Drivers do not have ABI. Skip further
> > > +				# processing if the map file is under
> > > +				# drivers directory
> > > +				continue
> > > +			fi
> > >  			if [ "$secname" = "unknown" ]
> > >  			then
> > >  				# Just inform the user of this occurrence, but
> > > --
> > > 2.21.0
> > >
> > >
>