* Re: [dpdk-stable] [dpdk-dev] [PATCH] devtools: skip the symbol check when map file under drivers
@ 2019-05-22 13:12 Jerin Jacob Kollanukkaran
2019-05-22 13:40 ` Neil Horman
0 siblings, 1 reply; 6+ messages in thread
From: Jerin Jacob Kollanukkaran @ 2019-05-22 13:12 UTC (permalink / raw)
To: Neil Horman; +Cc: dev, thomas, stable
> -----Original Message-----
> From: Neil Horman <nhorman@tuxdriver.com>
> Sent: Wednesday, May 22, 2019 6:16 PM
> To: Jerin Jacob Kollanukkaran <jerinj@marvell.com>
> Cc: dev@dpdk.org; thomas@monjalon.net; stable@dpdk.org
> Subject: Re: [EXT] Re: [dpdk-dev] [PATCH] devtools: skip the symbol check
> when map file under drivers
>
> On Wed, May 22, 2019 at 03:05:54AM +0000, Jerin Jacob Kollanukkaran
> wrote:
> > > -----Original Message-----
> > > From: Neil Horman <nhorman@tuxdriver.com>
> > > Sent: Wednesday, May 22, 2019 1:57 AM
> > > To: Jerin Jacob Kollanukkaran <jerinj@marvell.com>
> > > Cc: dev@dpdk.org; thomas@monjalon.net; stable@dpdk.org
> > > Subject: [EXT] Re: [dpdk-dev] [PATCH] devtools: skip the symbol
> > > check when map file under drivers
> > >
> > > On Wed, May 22, 2019 at 01:26:28AM +0530, jerinj@marvell.com wrote:
> > > > From: Jerin Jacob <jerinj@marvell.com>
> > > >
> > > > Drivers do not have ABI.
> > > > Skip the symbol check if map file under drivers directory.
> > > >
> > > > Fixes: 4bec48184e33 ("devtools: add checks for ABI symbol
> > > > addition")
> > > >
> > > > Cc: stable@dpdk.org
> > > > Cc: Neil Horman <nhorman@tuxdriver.com>
> > > >
> > > Sorry, but I'm not ok with this, because many of our DPDK PMDs have
> > > functions that get exported which are meant to be called by
> > > applications directly. The
> >
> > OK. Just to update my knowledge, Should those API needs to go through
> > ABI/API depreciation process?
> >
> Yes, they definately should, they are API's just as any other in the core DPDK
> library.
OK
>
> > Actually, I am concerned about the APIs, which is called between
> > drviers not the application. For example,
> > drivers/common/dpaax/rte_common_dpaax_version.map
> >
> > it is not interface to application rather it is for intra driver case.
> > I think, I can change my logic to Skip the symbols which NOT starting with
> rte_.
> > Agree?
> >
> No, Thats just one case, and if those calls are between drivers, so be it, but
> those still need to be stable, and we have other examples (like the bonding
> or dummy driver), which have additional APIs that are explicitly meant to be
> used by an application.
There is no disagreement on the API that exposed to application.
I am concerned with internal driver APIs. For example, I am getting following warning
ERROR: symbol otx2_mbox_alloc_msg_rsp is added in the DPDK_19.05 section, but is expected to be added in the EXPERIMENTAL section of the version map
This API suppose to be called only a octeontx2 network driver from octeontx2 common driver
i.e application should not expect any stability on intra driver functions or it does not meant to
be used by application.
Thomas,
Any thought on this?
>
> > Context:
> > I am adding a new driver/common/octeontx2 directory and it has some
> > API which Needs to shared between drivers not to the application. For
> > me, it does not make sense to go through any ABI process in such case.
> >
> Why? If you create an API thats reachable from another block of code (be it
> a driver or an application), you've created a dependency in which that API
> must remain stable, lest you risk breaking something. If an end user writes
> an out-of-tree PMD which makes use of an an additional driver API, then you
> need to keep it stable or you will break them.
>
> >
> > > dpaa2 driver is a good example, the cryptodev scheduler is another.
> > > Take a look at their version.map files to see what I mean.
> > >
> > > Unless we are willing to make drivers opaque objects that are only
> > > accessible from the [eth|crypto|etc]dev apis in the DPDK core, we
> > > have the potential for exported symbols, which means we have an ABI
> > > that has to be maintained, or at least recognized and reviewed for
> > > consistency
> > >
> > > Nacked-by: Neil Horman <nhorman@tuxdriver.com>
> > >
> > >
> > > > Signed-off-by: Jerin Jacob <jerinj@marvell.com>
> > > > ---
> > > > devtools/check-symbol-change.sh | 8 ++++++++
> > > > 1 file changed, 8 insertions(+)
> > > >
> > > > diff --git a/devtools/check-symbol-change.sh
> > > > b/devtools/check-symbol-change.sh index c5434f3bb..444beddad
> > > > 100755
> > > > --- a/devtools/check-symbol-change.sh
> > > > +++ b/devtools/check-symbol-change.sh
> > > > @@ -93,6 +93,14 @@ check_for_rule_violations()
> > > > if [ "$ar" = "add" ]
> > > > then
> > > >
> > > > + directory=`echo $mname | cut -d / -f 2`
> > > > + if [ "$directory" = "drivers" ]
> > > > + then
> > > > + # Drivers do not have ABI. Skip further
> > > > + # processing if the map file is under
> > > > + # drivers directory
> > > > + continue
> > > > + fi
> > > > if [ "$secname" = "unknown" ]
> > > > then
> > > > # Just inform the user of this occurrence, but
> > > > --
> > > > 2.21.0
> > > >
> > > >
> >
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [dpdk-stable] [dpdk-dev] [PATCH] devtools: skip the symbol check when map file under drivers
2019-05-22 13:12 [dpdk-stable] [dpdk-dev] [PATCH] devtools: skip the symbol check when map file under drivers Jerin Jacob Kollanukkaran
@ 2019-05-22 13:40 ` Neil Horman
2019-05-22 14:12 ` Thomas Monjalon
0 siblings, 1 reply; 6+ messages in thread
From: Neil Horman @ 2019-05-22 13:40 UTC (permalink / raw)
To: Jerin Jacob Kollanukkaran; +Cc: dev, thomas, stable
On Wed, May 22, 2019 at 01:12:34PM +0000, Jerin Jacob Kollanukkaran wrote:
> > -----Original Message-----
> > From: Neil Horman <nhorman@tuxdriver.com>
> > Sent: Wednesday, May 22, 2019 6:16 PM
> > To: Jerin Jacob Kollanukkaran <jerinj@marvell.com>
> > Cc: dev@dpdk.org; thomas@monjalon.net; stable@dpdk.org
> > Subject: Re: [EXT] Re: [dpdk-dev] [PATCH] devtools: skip the symbol check
> > when map file under drivers
> >
> > On Wed, May 22, 2019 at 03:05:54AM +0000, Jerin Jacob Kollanukkaran
> > wrote:
> > > > -----Original Message-----
> > > > From: Neil Horman <nhorman@tuxdriver.com>
> > > > Sent: Wednesday, May 22, 2019 1:57 AM
> > > > To: Jerin Jacob Kollanukkaran <jerinj@marvell.com>
> > > > Cc: dev@dpdk.org; thomas@monjalon.net; stable@dpdk.org
> > > > Subject: [EXT] Re: [dpdk-dev] [PATCH] devtools: skip the symbol
> > > > check when map file under drivers
> > > >
> > > > On Wed, May 22, 2019 at 01:26:28AM +0530, jerinj@marvell.com wrote:
> > > > > From: Jerin Jacob <jerinj@marvell.com>
> > > > >
> > > > > Drivers do not have ABI.
> > > > > Skip the symbol check if map file under drivers directory.
> > > > >
> > > > > Fixes: 4bec48184e33 ("devtools: add checks for ABI symbol
> > > > > addition")
> > > > >
> > > > > Cc: stable@dpdk.org
> > > > > Cc: Neil Horman <nhorman@tuxdriver.com>
> > > > >
> > > > Sorry, but I'm not ok with this, because many of our DPDK PMDs have
> > > > functions that get exported which are meant to be called by
> > > > applications directly. The
> > >
> > > OK. Just to update my knowledge, Should those API needs to go through
> > > ABI/API depreciation process?
> > >
> > Yes, they definately should, they are API's just as any other in the core DPDK
> > library.
>
> OK
>
>
> >
> > > Actually, I am concerned about the APIs, which is called between
> > > drviers not the application. For example,
> > > drivers/common/dpaax/rte_common_dpaax_version.map
> > >
> > > it is not interface to application rather it is for intra driver case.
> > > I think, I can change my logic to Skip the symbols which NOT starting with
> > rte_.
> > > Agree?
> > >
> > No, Thats just one case, and if those calls are between drivers, so be it, but
> > those still need to be stable, and we have other examples (like the bonding
> > or dummy driver), which have additional APIs that are explicitly meant to be
> > used by an application.
>
> There is no disagreement on the API that exposed to application.
> I am concerned with internal driver APIs. For example, I am getting following warning
>
> ERROR: symbol otx2_mbox_alloc_msg_rsp is added in the DPDK_19.05 section, but is expected to be added in the EXPERIMENTAL section of the version map
>
Thats a warning about the fact that you added an API call in a versioned section
of a library instead of the Experimental section, thats part of our policy. New
APIs need to go through the experimental tag first.
I understand what you are saying about driver only apis, and I mentioned that in
my other email farther down the thread. The problem is that "driver only apis"
are currently just a conceptual thing for us to discuss. They're still,
practially speaking, API's that any downstream user can access and become
dependent on, which we need to manage, either by keeping the API stable, so it
stays usable for all callers, or by developing a way to mark driver only API's
as such. I proposed a method that you might use to do the latter in my other email.
> This API suppose to be called only a octeontx2 network driver from octeontx2 common driver
> i.e application should not expect any stability on intra driver functions or it does not meant to
> be used by application.
>
Ok, but again, your assertion is that its driver to driver only, but in
practicaility, that assertion is irrelevant. Those symbols are still exposed
for general use, and weather or not you say they aren't part of the ABI, the
fact of the matter is, there is no way to tell the difference from a linked
object standpoint. Instead of hobbling the tool to just not scan anything, you
need to find a way to differentiate these symbols, so that you can enforce your
assertion that there are restrictions on where these APIs are called from.
Neil
> Thomas,
> Any thought on this?
>
>
>
> >
> > >
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [dpdk-stable] [dpdk-dev] [PATCH] devtools: skip the symbol check when map file under drivers
2019-05-22 13:40 ` Neil Horman
@ 2019-05-22 14:12 ` Thomas Monjalon
2019-05-22 14:33 ` Neil Horman
0 siblings, 1 reply; 6+ messages in thread
From: Thomas Monjalon @ 2019-05-22 14:12 UTC (permalink / raw)
To: Jerin Jacob Kollanukkaran; +Cc: Neil Horman, dev, stable
22/05/2019 15:40, Neil Horman:
> On Wed, May 22, 2019 at 01:12:34PM +0000, Jerin Jacob Kollanukkaran wrote:
> > From: Neil Horman <nhorman@tuxdriver.com>
> > > On Wed, May 22, 2019 at 03:05:54AM +0000, Jerin Jacob Kollanukkaran
> > > wrote:
> > > > From: Neil Horman <nhorman@tuxdriver.com>
> > > > > On Wed, May 22, 2019 at 01:26:28AM +0530, jerinj@marvell.com wrote:
> > > > > > From: Jerin Jacob <jerinj@marvell.com>
> > > > > >
> > > > > > Drivers do not have ABI.
> > > > > > Skip the symbol check if map file under drivers directory.
> > > > > >
> > > > > > Fixes: 4bec48184e33 ("devtools: add checks for ABI symbol
> > > > > > addition")
> > > > > >
> > > > > > Cc: stable@dpdk.org
> > > > > > Cc: Neil Horman <nhorman@tuxdriver.com>
> > > > > >
> > > > > Sorry, but I'm not ok with this, because many of our DPDK PMDs have
> > > > > functions that get exported which are meant to be called by
> > > > > applications directly. The
> > > >
> > > > OK. Just to update my knowledge, Should those API needs to go through
> > > > ABI/API depreciation process?
> > > >
> > > Yes, they definately should, they are API's just as any other in the core DPDK
> > > library.
> >
> > OK
> >
> >
> > >
> > > > Actually, I am concerned about the APIs, which is called between
> > > > drviers not the application. For example,
> > > > drivers/common/dpaax/rte_common_dpaax_version.map
> > > >
> > > > it is not interface to application rather it is for intra driver case.
> > > > I think, I can change my logic to Skip the symbols which NOT starting with
> > > rte_.
> > > > Agree?
> > > >
> > > No, Thats just one case, and if those calls are between drivers, so be it, but
> > > those still need to be stable, and we have other examples (like the bonding
> > > or dummy driver), which have additional APIs that are explicitly meant to be
> > > used by an application.
> >
> > There is no disagreement on the API that exposed to application.
> > I am concerned with internal driver APIs. For example, I am getting following warning
> >
> > ERROR: symbol otx2_mbox_alloc_msg_rsp is added in the DPDK_19.05 section, but is expected to be added in the EXPERIMENTAL section of the version map
> >
> Thats a warning about the fact that you added an API call in a versioned section
> of a library instead of the Experimental section, thats part of our policy. New
> APIs need to go through the experimental tag first.
>
> I understand what you are saying about driver only apis, and I mentioned that in
> my other email farther down the thread. The problem is that "driver only apis"
> are currently just a conceptual thing for us to discuss. They're still,
> practially speaking, API's that any downstream user can access and become
> dependent on, which we need to manage, either by keeping the API stable, so it
> stays usable for all callers, or by developing a way to mark driver only API's
> as such. I proposed a method that you might use to do the latter in my other email.
>
> > This API suppose to be called only a octeontx2 network driver from octeontx2 common driver
> > i.e application should not expect any stability on intra driver functions or it does not meant to
> > be used by application.
> >
> Ok, but again, your assertion is that its driver to driver only, but in
> practicaility, that assertion is irrelevant. Those symbols are still exposed
> for general use, and weather or not you say they aren't part of the ABI, the
> fact of the matter is, there is no way to tell the difference from a linked
> object standpoint. Instead of hobbling the tool to just not scan anything, you
> need to find a way to differentiate these symbols, so that you can enforce your
> assertion that there are restrictions on where these APIs are called from.
>
> Neil
>
> > Thomas,
> > Any thought on this?
As Neil said, we need to differentiate the internal APIs.
We already have this issue in a number of places like EAL, or ethdev,
and it was poorly addressed with some comments like "@internal".
Practically I don't care about stability of these internal functions,
but I agree that it creates a mess in the tooling and confuse users.
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [dpdk-stable] [dpdk-dev] [PATCH] devtools: skip the symbol check when map file under drivers
2019-05-22 14:12 ` Thomas Monjalon
@ 2019-05-22 14:33 ` Neil Horman
0 siblings, 0 replies; 6+ messages in thread
From: Neil Horman @ 2019-05-22 14:33 UTC (permalink / raw)
To: Thomas Monjalon; +Cc: Jerin Jacob Kollanukkaran, dev, stable
On Wed, May 22, 2019 at 04:12:40PM +0200, Thomas Monjalon wrote:
> 22/05/2019 15:40, Neil Horman:
> > On Wed, May 22, 2019 at 01:12:34PM +0000, Jerin Jacob Kollanukkaran wrote:
> > > From: Neil Horman <nhorman@tuxdriver.com>
> > > > On Wed, May 22, 2019 at 03:05:54AM +0000, Jerin Jacob Kollanukkaran
> > > > wrote:
> > > > > From: Neil Horman <nhorman@tuxdriver.com>
> > > > > > On Wed, May 22, 2019 at 01:26:28AM +0530, jerinj@marvell.com wrote:
> > > > > > > From: Jerin Jacob <jerinj@marvell.com>
> > > > > > >
> > > > > > > Drivers do not have ABI.
> > > > > > > Skip the symbol check if map file under drivers directory.
> > > > > > >
> > > > > > > Fixes: 4bec48184e33 ("devtools: add checks for ABI symbol
> > > > > > > addition")
> > > > > > >
> > > > > > > Cc: stable@dpdk.org
> > > > > > > Cc: Neil Horman <nhorman@tuxdriver.com>
> > > > > > >
> > > > > > Sorry, but I'm not ok with this, because many of our DPDK PMDs have
> > > > > > functions that get exported which are meant to be called by
> > > > > > applications directly. The
> > > > >
> > > > > OK. Just to update my knowledge, Should those API needs to go through
> > > > > ABI/API depreciation process?
> > > > >
> > > > Yes, they definately should, they are API's just as any other in the core DPDK
> > > > library.
> > >
> > > OK
> > >
> > >
> > > >
> > > > > Actually, I am concerned about the APIs, which is called between
> > > > > drviers not the application. For example,
> > > > > drivers/common/dpaax/rte_common_dpaax_version.map
> > > > >
> > > > > it is not interface to application rather it is for intra driver case.
> > > > > I think, I can change my logic to Skip the symbols which NOT starting with
> > > > rte_.
> > > > > Agree?
> > > > >
> > > > No, Thats just one case, and if those calls are between drivers, so be it, but
> > > > those still need to be stable, and we have other examples (like the bonding
> > > > or dummy driver), which have additional APIs that are explicitly meant to be
> > > > used by an application.
> > >
> > > There is no disagreement on the API that exposed to application.
> > > I am concerned with internal driver APIs. For example, I am getting following warning
> > >
> > > ERROR: symbol otx2_mbox_alloc_msg_rsp is added in the DPDK_19.05 section, but is expected to be added in the EXPERIMENTAL section of the version map
> > >
> > Thats a warning about the fact that you added an API call in a versioned section
> > of a library instead of the Experimental section, thats part of our policy. New
> > APIs need to go through the experimental tag first.
> >
> > I understand what you are saying about driver only apis, and I mentioned that in
> > my other email farther down the thread. The problem is that "driver only apis"
> > are currently just a conceptual thing for us to discuss. They're still,
> > practially speaking, API's that any downstream user can access and become
> > dependent on, which we need to manage, either by keeping the API stable, so it
> > stays usable for all callers, or by developing a way to mark driver only API's
> > as such. I proposed a method that you might use to do the latter in my other email.
> >
> > > This API suppose to be called only a octeontx2 network driver from octeontx2 common driver
> > > i.e application should not expect any stability on intra driver functions or it does not meant to
> > > be used by application.
> > >
> > Ok, but again, your assertion is that its driver to driver only, but in
> > practicaility, that assertion is irrelevant. Those symbols are still exposed
> > for general use, and weather or not you say they aren't part of the ABI, the
> > fact of the matter is, there is no way to tell the difference from a linked
> > object standpoint. Instead of hobbling the tool to just not scan anything, you
> > need to find a way to differentiate these symbols, so that you can enforce your
> > assertion that there are restrictions on where these APIs are called from.
> >
> > Neil
> >
> > > Thomas,
> > > Any thought on this?
>
> As Neil said, we need to differentiate the internal APIs.
> We already have this issue in a number of places like EAL, or ethdev,
> and it was poorly addressed with some comments like "@internal".
>
> Practically I don't care about stability of these internal functions,
> but I agree that it creates a mess in the tooling and confuse users.
>
Agreed, If we can mark them in a way that can enforce no outside usage, then
there need not be any ABI compatibility guarantee
Neil
>
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* [dpdk-stable] [dpdk-dev] [PATCH] devtools: skip the symbol check when map file under drivers
@ 2019-05-21 19:56 jerinj
2019-05-21 20:27 ` Neil Horman
0 siblings, 1 reply; 6+ messages in thread
From: jerinj @ 2019-05-21 19:56 UTC (permalink / raw)
To: Neil Horman; +Cc: dev, thomas, Jerin Jacob, stable
From: Jerin Jacob <jerinj@marvell.com>
Drivers do not have ABI.
Skip the symbol check if map file under drivers directory.
Fixes: 4bec48184e33 ("devtools: add checks for ABI symbol addition")
Cc: stable@dpdk.org
Cc: Neil Horman <nhorman@tuxdriver.com>
Signed-off-by: Jerin Jacob <jerinj@marvell.com>
---
devtools/check-symbol-change.sh | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/devtools/check-symbol-change.sh b/devtools/check-symbol-change.sh
index c5434f3bb..444beddad 100755
--- a/devtools/check-symbol-change.sh
+++ b/devtools/check-symbol-change.sh
@@ -93,6 +93,14 @@ check_for_rule_violations()
if [ "$ar" = "add" ]
then
+ directory=`echo $mname | cut -d / -f 2`
+ if [ "$directory" = "drivers" ]
+ then
+ # Drivers do not have ABI. Skip further
+ # processing if the map file is under
+ # drivers directory
+ continue
+ fi
if [ "$secname" = "unknown" ]
then
# Just inform the user of this occurrence, but
--
2.21.0
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [dpdk-stable] [dpdk-dev] [PATCH] devtools: skip the symbol check when map file under drivers
2019-05-21 19:56 jerinj
@ 2019-05-21 20:27 ` Neil Horman
0 siblings, 0 replies; 6+ messages in thread
From: Neil Horman @ 2019-05-21 20:27 UTC (permalink / raw)
To: jerinj; +Cc: dev, thomas, stable
On Wed, May 22, 2019 at 01:26:28AM +0530, jerinj@marvell.com wrote:
> From: Jerin Jacob <jerinj@marvell.com>
>
> Drivers do not have ABI.
> Skip the symbol check if map file under drivers directory.
>
> Fixes: 4bec48184e33 ("devtools: add checks for ABI symbol addition")
>
> Cc: stable@dpdk.org
> Cc: Neil Horman <nhorman@tuxdriver.com>
>
Sorry, but I'm not ok with this, because many of our DPDK PMDs have functions
that get exported which are meant to be called by applications directly. The
dpaa2 driver is a good example, the cryptodev scheduler is another. Take a look
at their version.map files to see what I mean.
Unless we are willing to make drivers opaque objects that are only accessible
from the [eth|crypto|etc]dev apis in the DPDK core, we have the potential for
exported symbols, which means we have an ABI that has to be maintained, or at
least recognized and reviewed for consistency
Nacked-by: Neil Horman <nhorman@tuxdriver.com>
> Signed-off-by: Jerin Jacob <jerinj@marvell.com>
> ---
> devtools/check-symbol-change.sh | 8 ++++++++
> 1 file changed, 8 insertions(+)
>
> diff --git a/devtools/check-symbol-change.sh b/devtools/check-symbol-change.sh
> index c5434f3bb..444beddad 100755
> --- a/devtools/check-symbol-change.sh
> +++ b/devtools/check-symbol-change.sh
> @@ -93,6 +93,14 @@ check_for_rule_violations()
> if [ "$ar" = "add" ]
> then
>
> + directory=`echo $mname | cut -d / -f 2`
> + if [ "$directory" = "drivers" ]
> + then
> + # Drivers do not have ABI. Skip further
> + # processing if the map file is under
> + # drivers directory
> + continue
> + fi
> if [ "$secname" = "unknown" ]
> then
> # Just inform the user of this occurrence, but
> --
> 2.21.0
>
>
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2019-05-22 14:33 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-05-22 13:12 [dpdk-stable] [dpdk-dev] [PATCH] devtools: skip the symbol check when map file under drivers Jerin Jacob Kollanukkaran
2019-05-22 13:40 ` Neil Horman
2019-05-22 14:12 ` Thomas Monjalon
2019-05-22 14:33 ` Neil Horman
-- strict thread matches above, loose matches on Subject: below --
2019-05-21 19:56 jerinj
2019-05-21 20:27 ` Neil Horman
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).