* [dpdk-dev] [PATCH v2] doc: deprication notice to mark tm spec as experimental
2020-05-05 6:47 3% ` Ray Kinsella
@ 2020-05-05 8:07 5% ` Nithin Dabilpuram
1 sibling, 0 replies; 200+ results
From: Nithin Dabilpuram @ 2020-05-05 8:07 UTC (permalink / raw)
To: ferruh.yigit, bruce.richardson, cristian.dumitrescu, thomas,
bluca, jasvinder.singh, arybchenko, ray.kinsella, nhorman,
ktraynor, david.marchand, John McNamara, Marko Kovacevic
Cc: dev, jerinj, kkanas, Nithin Dabilpuram
From: Nithin Dabilpuram <ndabilpuram@marvell.com>
Based on the discussion in mail thread, it is concluded that
all traffic manager API's (rte_tm.h) need to be marked experimental
till few more releases to support further improvements to spec.
https://mails.dpdk.org/archives/dev/2020-April/164970.html
Adding deprication notice for the same in advance.
Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>
---
doc/guides/rel_notes/deprecation.rst | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
index 1339f54..2c76f36 100644
--- a/doc/guides/rel_notes/deprecation.rst
+++ b/doc/guides/rel_notes/deprecation.rst
@@ -118,3 +118,10 @@ Deprecation Notices
Python 2 support will be completely removed in 20.11.
In 20.08, explicit deprecation warnings will be displayed when running
scripts with Python 2.
+
+* traffic manager: All traffic manager API's in ``rte_tm.h`` were mistakenly made
+ abi stable in the v19.11 release. The TM maintainer and other contributor's have
+ agreed to keep the TM API's as experimental in expectation of additional spec
+ improvements. Therefore, all API's in ``rte_tm.h`` will be marked back as
+ experimental in v20.11 DPDK release. For more details, please see `the thread
+ <https://mails.dpdk.org/archives/dev/2020-April/164970.html>`_.
--
2.8.4
^ permalink raw reply [relevance 5%]
* Re: [dpdk-dev] [PATCH v1] abi: document reasons behind the three part versioning
2020-05-04 21:46 4% ` Thomas Monjalon
@ 2020-05-05 8:07 4% ` Ray Kinsella
0 siblings, 0 replies; 200+ results
From: Ray Kinsella @ 2020-05-05 8:07 UTC (permalink / raw)
To: Thomas Monjalon
Cc: dev, ferruh.yigit, Bruce Richardson, John McNamara, Marko Kovacevic
On 04/05/2020 22:46, Thomas Monjalon wrote:
> 23/04/2020 12:12, Ray Kinsella:
>> Clarify the reasons behind the three part version numbering scheme.
>> Documents the fixes made in f26c2b3.
>
> Please add these lines:
> Fixes: f26c2b39b271 ("build: fix soname info for 19.11 compatibility")
> Cc: stable@dpdk.org
>
>
>> Signed-off-by: Ray Kinsella <mdr@ashroe.eu>
>> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
>> ---
>> --- a/doc/guides/contributing/abi_policy.rst
>> +++ b/doc/guides/contributing/abi_policy.rst
>> + lengthening the stability period. Additional implementation detail can be
>> + found in the :ref:`release notes <20_05_abi_changes>`.
>
> The three part version was done in 20.02.
> I think you should update the 20.02 release notes instead of 20.05.
>
I agreed,
However I didn't think that was possible on a quarterly, after the release was formally complete.
Wouldn't that involve either a respin or a .1 ?
Ray K
^ permalink raw reply [relevance 4%]
* [dpdk-dev] [PATCH v11 0/2] support for VFIO-PCI VF token interface
` (2 preceding siblings ...)
2020-04-26 1:55 4% ` [dpdk-dev] [PATCH v10 0/2] support for VFIO-PCI VF token interface Haiyue Wang
@ 2020-05-05 7:46 4% ` Haiyue Wang
3 siblings, 0 replies; 200+ results
From: Haiyue Wang @ 2020-05-05 7:46 UTC (permalink / raw)
To: dev, anatoly.burakov, thomas, jerinj, david.marchand; +Cc: Haiyue Wang
v11: Use the eal parameter to pass the VF token, then not every PCI
device needs to be specified with this token. Also no ABI issue
now.
v10: Use the __rte_internal to mark the internal API changing.
v9: Rewrite the document.
v8: Update the document.
v7: Add the Fixes tag in uuid, the release note and help
document.
v6: Drop the Fixes tag in uuid, since the file has been
moved to another place, not suitable to apply on stable.
And this is not a bug, just some kind of enhancement.
v5: 1. Add the VF token parse error handling.
2. Split into two patches for different logic module.
3. Add more comments into the code for explaining the design.
4. Drop the ABI change workaround, this patch set focuses on code review.
v4: 1. Ignore rte_vfio_setup_device ABI check since it is
for Linux driver use.
v3: Fix the Travis build failed:
(1). rte_uuid.h:97:55: error: unknown type name ‘size_t’
(2). rte_uuid.h:58:2: error: implicit declaration of function ‘memcpy’
v2: Fix the FreeBSD build error.
v1: Update the commit message.
RFC v2:
Based on Vamsi's RFC v1, and Alex's patch for Qemu
[https://lore.kernel.org/lkml/20200204161737.34696b91@w520.home/]:
Use the devarg to pass-down the VF token.
RFC v1: https://patchwork.dpdk.org/patch/66281/ by Vamsi.
Haiyue Wang (2):
eal: add uuid dependent header files explicitly
eal: support for VFIO-PCI VF token
doc/guides/linux_gsg/linux_drivers.rst | 35 ++++++++++++++++++-
doc/guides/linux_gsg/linux_eal_parameters.rst | 4 +++
doc/guides/rel_notes/release_20_05.rst | 6 ++++
lib/librte_eal/common/eal_common_options.c | 2 ++
lib/librte_eal/common/eal_internal_cfg.h | 2 ++
lib/librte_eal/common/eal_options.h | 2 ++
lib/librte_eal/freebsd/eal.c | 5 +++
lib/librte_eal/include/rte_eal.h | 12 +++++++
lib/librte_eal/include/rte_uuid.h | 2 ++
lib/librte_eal/linux/eal.c | 29 +++++++++++++++
lib/librte_eal/linux/eal_vfio.c | 16 ++++++++-
lib/librte_eal/rte_eal_version.map | 1 +
12 files changed, 114 insertions(+), 2 deletions(-)
--
2.26.2
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH v4 1/4] ethdev: add tm support for shaper config in pkt mode
2020-05-01 13:18 3% ` [dpdk-dev] " Jerin Jacob
@ 2020-05-05 8:01 0% ` Ray Kinsella
0 siblings, 0 replies; 200+ results
From: Ray Kinsella @ 2020-05-05 8:01 UTC (permalink / raw)
To: Jerin Jacob, Thomas Monjalon
Cc: Dumitrescu, Cristian, Bruce Richardson, Ferruh Yigit,
Luca Boccassi, Nithin Dabilpuram, Singh, Jasvinder,
Andrew Rybchenko, dev, jerinj, kkanas, Nithin Dabilpuram,
Kinsella, Ray, Neil Horman, Kevin Traynor, David Marchand
On 01/05/2020 14:18, Jerin Jacob wrote:
> On Tue, Apr 28, 2020 at 9:24 PM Thomas Monjalon <thomas@monjalon.net> wrote:
>>
>> 28/04/2020 17:04, Luca Boccassi:
>>> On Tue, 2020-04-28 at 15:45 +0100, Bruce Richardson wrote:
>>>> On Tue, Apr 28, 2020 at 03:06:20PM +0100, Ferruh Yigit wrote:
>>>>> On 4/27/2020 5:59 PM, Jerin Jacob wrote:
>>>>>> On Mon, Apr 27, 2020 at 10:19 PM Ferruh Yigit <ferruh.yigit@intel.com> wrote:
>>>>>>> On 4/27/2020 5:29 PM, Jerin Jacob wrote:
>>>>>>>> On Mon, Apr 27, 2020 at 9:42 PM Ferruh Yigit <ferruh.yigit@intel.com> wrote:
>>>>>>>>> On 4/27/2020 10:19 AM, Dumitrescu, Cristian wrote:
>>>>>>>>>> From: Yigit, Ferruh <ferruh.yigit@intel.com>
>>>>>>>>>>> On 4/24/2020 11:28 AM, Dumitrescu, Cristian wrote:
>>>>>>>>>>>> From: Nithin Dabilpuram <nithind1988@gmail.com>
>>>>>>>>>>>>> This patch also updates tm port/level/node capability structures with
>>>>>>>>>>>>> exiting features of scheduler wfq packet mode, scheduler wfq byte mode
>>>>>>>>>>>>> and private/shared shaper byte mode.
>>>>>>>>>>>>>
>>>>>>>>>>>>> SoftNIC PMD is also updated with new capabilities.
>> [...]
>>>>>>>>>>> Hi Nithin,
>>>>>>>>>>>
>>>>>>>>>>> It looks like patch is causing ABI break, I am getting following warning [1],
>>>>>>>>>>> can you please check?
>>>>>>>>>>>
>>>>>>>>>>> [1]
>>>>>>>>>>> https://pastebin.com/XYNFg14u
>>>>>>>>>>
>>>>>>>>>> Hi Ferruh,
>>>>>>>>>>
>>>>>>>>>> The RTE_TM API is marked as experimental,
>>>>>>>>>> but it looks that this was not correctly marked
>>>>>>>>>> when __rte_experimental ABI checker was introduced.
>>>>>>>>>>
>>>>>>>>>> It is marked as experimental at the top of the rte_tm.h,
>>>>>>>>>> similarly to other APIs introduced around same time,
>>>>>>>>>> but it was not correctly picked up by the ABI check procedure
>>>>>>>>>> when later introduced, so __rte_experimental was not added to every function.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> :(
>>>>>>>>>
>>>>>>>>> Is it time to mature them?
>>>>>>>>>
>>>>>>>>> As you said they are not marked as experimental both in header file (function
>>>>>>>>> declarations) and .map file.
>>>>>>>>>
>>>>>>>>> The problem is, they are not marked as experimental in DPDK_20.0 ABI (v19.11),
>>>>>>>>> so marking them as experimental now will break the ABI. Not sure what to do,
>>>>>>>>> cc'ed a few ABI related names for comment.
>>>>>>>>>
>>>>>>>>> For me, we need to proceed as the experimental tag removed and APIs become
>>>>>>>>> mature starting from v19.11, since this is what happened in practice, and remove
>>>>>>>>> a few existing being experimental references in the doxygen comments.
>>>>>>>>
>>>>>>>> I think, accidentally we can not make a library as NON-experimental.
>>>>>>>> TM never went through experimental to mature transition(see git log
>>>>>>>> lib/librte_ethdev/rte_tm.h)
>>>>>>>> It was a bug to not mark as experimental in each function in the ABI process.
>>>>>>>> Some of the features like packet marking are not even implemented by any HW.
>>>>>>>> I think, we can make API stable only all the features are implemented
>>>>>>>> by one or two HW.
>>
>> Yes this is what was decided one or two years ago I think.
>> But rte_tm API was introduced 3 years ago and is implemented by 6 PMDs.
>
> None of the 6 PMDS covers all the features.
>
>>
>>>>> Existing experimental marks are text only for human parsing.
>>>>>
>>>>> The compiler attribute and build time checks are missing, and the symbol in the
>>>>> binary doesn't have experimental tag. Our scripts and automated checks won't
>>>>> detect it as experimental.
>>>>>
>>>>> My point is just having experimental comment in header file is not enough to
>>>>> qualify the APIs as experimental.
>>>>>
>>>>>> Anyway, we need to break the ABI to make it work on various HW.
>>
>> Yes this is why I was asking in 19.11 to check our API,
>> in order to avoid such situation.
>>
>>
>>>>>> I am not sure what to do?
>>
>> Either manage ABI versioning, or wait 20.11.
>
> ABI change are in structures. So the function versioning does not
> help. So we will wait for 20.11 then :-(
>
yes - I spent some time looking at this also, and came to same conclusion.
Ray K
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [EXT] Re: [PATCH] doc: deprication notice to mark tm spec as experimental
2020-05-05 6:47 3% ` Ray Kinsella
@ 2020-05-05 7:45 0% ` Nithin Dabilpuram
0 siblings, 0 replies; 200+ results
From: Nithin Dabilpuram @ 2020-05-05 7:45 UTC (permalink / raw)
To: Ray Kinsella
Cc: Nithin Dabilpuram, ferruh.yigit, bruce.richardson,
cristian.dumitrescu, thomas, bluca, jasvinder.singh, arybchenko,
ray.kinsella, nhorman, ktraynor, david.marchand, John McNamara,
Marko Kovacevic, dev, jerinj, kkanas
On Tue, May 05, 2020 at 07:47:43AM +0100, Ray Kinsella wrote:
> External Email
>
> ----------------------------------------------------------------------
>
>
> On 01/05/2020 18:16, Nithin Dabilpuram wrote:
> > From: Nithin Dabilpuram <ndabilpuram@marvell.com>
> >
> > Based on the discussion in below mail thread, it is concluded that
> > all traffic manager API's (rte_tm.h) need to be marked experimental
> > till few more releases to support further improvements to spec.
> >
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__mails.dpdk.org_archives_dev_2020-2DApril_164970.html&d=DwIDaQ&c=nKjWec2b6R0mOyPaz7xtfQ&r=FZ_tPCbgFOh18zwRPO9H0yDx8VW38vuapifdDfc8SFQ&m=fMY2AL-_MFowqy_OpcWEwioupC8kxy4GNU6R3LZP_2A&s=WCnFYbXTODzmkAWYDdBurNyAJ8Hxti2qkxXNnuoBic4&e=
> >
> > Adding deprication notice for the same in advance.
> >
> > Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>
> > ---
> > doc/guides/rel_notes/deprecation.rst | 9 +++++++++
> > 1 file changed, 9 insertions(+)
> >
> > diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
> > index 1339f54..8c90ec9 100644
> > --- a/doc/guides/rel_notes/deprecation.rst
> > +++ b/doc/guides/rel_notes/deprecation.rst
> > @@ -118,3 +118,12 @@ Deprecation Notices
> > Python 2 support will be completely removed in 20.11.
> > In 20.08, explicit deprecation warnings will be displayed when running
> > scripts with Python 2.
> > +
> > +* traffic manager: Based on the discussion in the thread
> > + https://urldefense.proofpoint.com/v2/url?u=https-3A__mails.dpdk.org_archives_dev_2020-2DApril_164970.html&d=DwIDaQ&c=nKjWec2b6R0mOyPaz7xtfQ&r=FZ_tPCbgFOh18zwRPO9H0yDx8VW38vuapifdDfc8SFQ&m=fMY2AL-_MFowqy_OpcWEwioupC8kxy4GNU6R3LZP_2A&s=WCnFYbXTODzmkAWYDdBurNyAJ8Hxti2qkxXNnuoBic4&e= it turns out that
> > + all traffic manager API's in ``rte_tm.h`` were mistakenly made stable even
> > + though the MAINTAINER's file and header file text mentions it to be still
> > + experimental. Given that even now, both the maintainer and other contributor's
> > + are inclined towards keeping those API's in experimental state till some more
> > + releases in support of spec improvements, it is decided that all API's in
> > + ``rte_tm.h`` will be marked back as experimental in v20.11 DPDK release.
> >
>
> Slightly wordy - suggest making it a small bit more concise as follows.
>
> * traffic manager: All traffic manager API's in ``rte_tm.h`` were mistakenly made
> abi stable in the v19.11 release. The TM maintainer and other contributor's have
> agreed to keep the TM API's as experimental in expectation of additional spec
> improvements. Therefore, all API's in ``rte_tm.h`` will be marked back as
> experimental in v20.11 DPDK release. For more detail, please see `the thread
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__mails.dpdk.org_archives_dev_2020-2DApril_164970.html&d=DwIDaQ&c=nKjWec2b6R0mOyPaz7xtfQ&r=FZ_tPCbgFOh18zwRPO9H0yDx8VW38vuapifdDfc8SFQ&m=fMY2AL-_MFowqy_OpcWEwioupC8kxy4GNU6R3LZP_2A&s=WCnFYbXTODzmkAWYDdBurNyAJ8Hxti2qkxXNnuoBic4&e= >`_.
Ack, will send v2.
>
> Ray K
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH] doc: deprication notice to mark tm spec as experimental
@ 2020-05-05 6:47 3% ` Ray Kinsella
2020-05-05 7:45 0% ` [dpdk-dev] [EXT] " Nithin Dabilpuram
2020-05-05 8:07 5% ` [dpdk-dev] [PATCH v2] " Nithin Dabilpuram
1 sibling, 1 reply; 200+ results
From: Ray Kinsella @ 2020-05-05 6:47 UTC (permalink / raw)
To: Nithin Dabilpuram, ferruh.yigit, bruce.richardson,
cristian.dumitrescu, thomas, bluca, jasvinder.singh, arybchenko,
ray.kinsella, nhorman, ktraynor, david.marchand, John McNamara,
Marko Kovacevic
Cc: dev, jerinj, kkanas, Nithin Dabilpuram
On 01/05/2020 18:16, Nithin Dabilpuram wrote:
> From: Nithin Dabilpuram <ndabilpuram@marvell.com>
>
> Based on the discussion in below mail thread, it is concluded that
> all traffic manager API's (rte_tm.h) need to be marked experimental
> till few more releases to support further improvements to spec.
>
> https://mails.dpdk.org/archives/dev/2020-April/164970.html
>
> Adding deprication notice for the same in advance.
>
> Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>
> ---
> doc/guides/rel_notes/deprecation.rst | 9 +++++++++
> 1 file changed, 9 insertions(+)
>
> diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
> index 1339f54..8c90ec9 100644
> --- a/doc/guides/rel_notes/deprecation.rst
> +++ b/doc/guides/rel_notes/deprecation.rst
> @@ -118,3 +118,12 @@ Deprecation Notices
> Python 2 support will be completely removed in 20.11.
> In 20.08, explicit deprecation warnings will be displayed when running
> scripts with Python 2.
> +
> +* traffic manager: Based on the discussion in the thread
> + https://mails.dpdk.org/archives/dev/2020-April/164970.html it turns out that
> + all traffic manager API's in ``rte_tm.h`` were mistakenly made stable even
> + though the MAINTAINER's file and header file text mentions it to be still
> + experimental. Given that even now, both the maintainer and other contributor's
> + are inclined towards keeping those API's in experimental state till some more
> + releases in support of spec improvements, it is decided that all API's in
> + ``rte_tm.h`` will be marked back as experimental in v20.11 DPDK release.
>
Slightly wordy - suggest making it a small bit more concise as follows.
* traffic manager: All traffic manager API's in ``rte_tm.h`` were mistakenly made
abi stable in the v19.11 release. The TM maintainer and other contributor's have
agreed to keep the TM API's as experimental in expectation of additional spec
improvements. Therefore, all API's in ``rte_tm.h`` will be marked back as
experimental in v20.11 DPDK release. For more detail, please see `the thread
<https://mails.dpdk.org/archives/dev/2020-April/164970.html>`_.
Ray K
^ permalink raw reply [relevance 3%]
* Re: [dpdk-dev] [PATCH v5] mempool: return 0 if area is too small on populate
2020-05-04 15:47 0% ` Lukasz Wojciechowski
@ 2020-05-04 22:30 0% ` Thomas Monjalon
0 siblings, 0 replies; 200+ results
From: Thomas Monjalon @ 2020-05-04 22:30 UTC (permalink / raw)
To: Olivier Matz
Cc: Andrew Rybchenko, dev, david.marchand, jingjing.wu,
john.mcnamara, marko.kovacevic, mdr, nhorman, xiaoyun.li,
Lukasz Wojciechowski
04/05/2020 17:47, Lukasz Wojciechowski:
> W dniu 04.05.2020 o 14:54, Andrew Rybchenko pisze:
> > On 5/4/20 3:49 PM, Olivier Matz wrote:
> >> Change rte_mempool_populate_iova() and rte_mempool_populate_virt() to
> >> return 0 instead of -EINVAL when there is not enough room to store one
> >> object, as it can be helpful for applications to distinguish this
> >> specific case.
> >>
> >> As this is an ABI change, use symbol versioning to preserve old
> >> behavior for binary applications.
> >>
> >> Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
> >> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> > Acked-by: Andrew Rybchenko <arybchenko@solarflare.com>
> Acked-by: Lukasz Wojciechowski <l.wojciechow@partner.samsung.com>
Applied, thanks
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH v6 1/1] abi: change references to abi 20.0.1 to abi v21
2020-04-30 10:27 31% ` Ray Kinsella
@ 2020-05-04 22:23 6% ` Thomas Monjalon
0 siblings, 0 replies; 200+ results
From: Thomas Monjalon @ 2020-05-04 22:23 UTC (permalink / raw)
To: Ray Kinsella
Cc: dev, david.marchand, Neil Horman, Jingjing Wu, Beilei Xing,
Matan Azrad, Shahaf Shuler, Viacheslav Ovsiienko, Jerin Jacob,
Nithin Dabilpuram, Alfredo Cardigliano, Mahipal Challa,
Cristian Dumitrescu, Ray Kinsella
30/04/2020 12:27, Ray Kinsella:
> Change references to abi 20.0.1 to use abi v21, see
> https://doc.dpdk.org/guides/contributing/abi_policy.html#general-guidelines
>
> "Major ABI versions are declared no more frequently than yearly.
> Compatibility with the major ABI version is mandatory in subsequent
> releases until a new major ABI version is declared."
>
> Combined abi policy and versioning in maintainers, add map files to the
> filter to more closely monitor future abi changes.
>
> Signed-off-by: Ray Kinsella <mdr@ashroe.eu>
Applied with added change for recent igc PMD:
--- a/drivers/net/igc/rte_pmd_igc_version.map
+++ b/drivers/net/igc/rte_pmd_igc_version.map
@@ -1,3 +1,3 @@
-DPDK_20.0.2 {
+DPDK_21 {
local: *;
};
^ permalink raw reply [relevance 6%]
* Re: [dpdk-dev] [PATCH v6 1/1] abi: change references to abi 20.0.1 to abi v21
2020-04-30 10:27 11% ` [dpdk-dev] [PATCH v6 1/1] " Ray Kinsella
2020-04-30 10:27 31% ` Ray Kinsella
@ 2020-05-04 22:05 6% ` Thomas Monjalon
1 sibling, 0 replies; 200+ results
From: Thomas Monjalon @ 2020-05-04 22:05 UTC (permalink / raw)
To: Ray Kinsella
Cc: dev, david.marchand, Ray Kinsella, Neil Horman, Jingjing Wu,
Beilei Xing, Matan Azrad, Shahaf Shuler, Viacheslav Ovsiienko,
Jerin Jacob, Nithin Dabilpuram, Alfredo Cardigliano,
Mahipal Challa, Cristian Dumitrescu
30/04/2020 12:27, Ray Kinsella:
> Sending again, as I missed the cover letter last time.
You don't need a cover letter for a single patch.
The changelog can be added below --- in a patch.
^ permalink raw reply [relevance 6%]
* Re: [dpdk-dev] [PATCH v1] abi: document reasons behind the three part versioning
2020-04-23 10:12 24% [dpdk-dev] [PATCH v1] abi: document reasons behind the three part versioning Ray Kinsella
@ 2020-05-04 21:46 4% ` Thomas Monjalon
2020-05-05 8:07 4% ` Ray Kinsella
0 siblings, 1 reply; 200+ results
From: Thomas Monjalon @ 2020-05-04 21:46 UTC (permalink / raw)
To: Ray Kinsella
Cc: dev, ferruh.yigit, Bruce Richardson, John McNamara, Marko Kovacevic
23/04/2020 12:12, Ray Kinsella:
> Clarify the reasons behind the three part version numbering scheme.
> Documents the fixes made in f26c2b3.
Please add these lines:
Fixes: f26c2b39b271 ("build: fix soname info for 19.11 compatibility")
Cc: stable@dpdk.org
> Signed-off-by: Ray Kinsella <mdr@ashroe.eu>
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> ---
> --- a/doc/guides/contributing/abi_policy.rst
> +++ b/doc/guides/contributing/abi_policy.rst
> + lengthening the stability period. Additional implementation detail can be
> + found in the :ref:`release notes <20_05_abi_changes>`.
The three part version was done in 20.02.
I think you should update the 20.02 release notes instead of 20.05.
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH v4] eal/cpuflags: add x86 based cpu flags
2020-04-25 16:04 4% ` Thomas Monjalon
@ 2020-04-27 9:22 4% ` Kinsella, Ray
2020-04-27 9:27 4% ` Ray Kinsella
1 sibling, 0 replies; 200+ results
From: Kinsella, Ray @ 2020-04-27 9:22 UTC (permalink / raw)
To: Thomas Monjalon, nhorman, Laatz, Kevin
Cc: dev, Richardson, Bruce, Van Haaren, Harry, david.marchand,
Ray Kinsella, Trahe, Fiona
> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: Saturday 25 April 2020 17:04
> To: Kinsella, Ray <ray.kinsella@intel.com>; nhorman@tuxdriver.com;
> Laatz, Kevin <kevin.laatz@intel.com>
> Cc: dev@dpdk.org; Richardson, Bruce <bruce.richardson@intel.com>; Van
> Haaren, Harry <harry.van.haaren@intel.com>; david.marchand@redhat.com
> Subject: Re: [dpdk-dev] [PATCH v4] eal/cpuflags: add x86 based cpu
> flags
>
> 16/04/2020 13:00, Kevin Laatz:
> > This patch adds CPU flags which will enable the detection of ISA
> > features available on more recent x86 based CPUs.
> [...]
> > --- a/devtools/libabigail.abignore
> > +++ b/devtools/libabigail.abignore
> > +; Ignore this enum update as it should not be allocated by the
> > +application [suppress_type]
> > + type_kind = enum
> > + name = rte_cpu_flag_t
> > + changed_enumerators = RTE_CPUFLAG_NUMFLAGS
>
> The justification is not correct.
> The application is allowed to use RTE_CPUFLAG_NUMFLAGS in array
> allocation.
> But no API is returning a CPU flag, so the new flags will remain
> unknown to the application.
>
> However, there is a behaviour change:
> The functions rte_cpu_get_flag_name() and rte_cpu_get_flag_enabled()
> will now accept new values, which were previously considered as an
> error.
> Is it an ABI breakage? I would say no.
We saw something similar with the Cryptodev's rte_crypto_sym_xform_type also.
Libabigail appears to be particularly sensitive to changes to enumerations.
Leaving it to the user to decide if there is a problem.
I am seeing a bit of weirdness though between versions of libabigail.
1.7.1 seems to fine with the change, however 1.2 is reporting an issue.
Kevin - what version are you using?
>
> PS: Who is REALLY maintaining the ABI?
> We really miss someone who carefully check all these things, and take
> care of the doc and tooling.
I would say that I am missing these changes to libabigail.ignore, which would be useful.
Should we consolidate the ABI Policy and ABI Versioning sections of the MAINTAINERS file?
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH v5] mempool: return 0 if area is too small on populate
2020-05-04 12:54 0% ` Andrew Rybchenko
@ 2020-05-04 15:47 0% ` Lukasz Wojciechowski
2020-05-04 22:30 0% ` Thomas Monjalon
0 siblings, 1 reply; 200+ results
From: Lukasz Wojciechowski @ 2020-05-04 15:47 UTC (permalink / raw)
To: Andrew Rybchenko, Olivier Matz, thomas
Cc: david.marchand, dev, jingjing.wu, john.mcnamara, marko.kovacevic,
mdr, nhorman, xiaoyun.li
W dniu 04.05.2020 o 14:54, Andrew Rybchenko pisze:
> On 5/4/20 3:49 PM, Olivier Matz wrote:
>> Change rte_mempool_populate_iova() and rte_mempool_populate_virt() to
>> return 0 instead of -EINVAL when there is not enough room to store one
>> object, as it can be helpful for applications to distinguish this
>> specific case.
>>
>> As this is an ABI change, use symbol versioning to preserve old
>> behavior for binary applications.
>>
>> Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
>> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> Acked-by: Andrew Rybchenko <arybchenko@solarflare.com>
Acked-by: Lukasz Wojciechowski <l.wojciechow@partner.samsung.com>
--
Lukasz Wojciechowski
Principal Software Engineer
Samsung R&D Institute Poland
Samsung Electronics
Office +48 22 377 88 25
l.wojciechow@partner.samsung.com
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH v5] mempool: return 0 if area is too small on populate
2020-05-04 12:49 11% ` [dpdk-dev] [PATCH v5] " Olivier Matz
@ 2020-05-04 12:54 0% ` Andrew Rybchenko
2020-05-04 15:47 0% ` Lukasz Wojciechowski
0 siblings, 1 reply; 200+ results
From: Andrew Rybchenko @ 2020-05-04 12:54 UTC (permalink / raw)
To: Olivier Matz, thomas
Cc: david.marchand, dev, jingjing.wu, john.mcnamara, marko.kovacevic,
mdr, nhorman, xiaoyun.li, Lukasz Wojciechowski
On 5/4/20 3:49 PM, Olivier Matz wrote:
> Change rte_mempool_populate_iova() and rte_mempool_populate_virt() to
> return 0 instead of -EINVAL when there is not enough room to store one
> object, as it can be helpful for applications to distinguish this
> specific case.
>
> As this is an ABI change, use symbol versioning to preserve old
> behavior for binary applications.
>
> Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Andrew Rybchenko <arybchenko@solarflare.com>
^ permalink raw reply [relevance 0%]
* [dpdk-dev] [PATCH v5] mempool: return 0 if area is too small on populate
2020-04-26 16:52 12% ` [dpdk-dev] [PATCH v4] " Thomas Monjalon
@ 2020-05-04 12:49 11% ` Olivier Matz
2020-05-04 12:54 0% ` Andrew Rybchenko
0 siblings, 1 reply; 200+ results
From: Olivier Matz @ 2020-05-04 12:49 UTC (permalink / raw)
To: thomas
Cc: arybchenko, david.marchand, dev, jingjing.wu, john.mcnamara,
marko.kovacevic, mdr, nhorman, olivier.matz, xiaoyun.li,
Lukasz Wojciechowski
Change rte_mempool_populate_iova() and rte_mempool_populate_virt() to
return 0 instead of -EINVAL when there is not enough room to store one
object, as it can be helpful for applications to distinguish this
specific case.
As this is an ABI change, use symbol versioning to preserve old
behavior for binary applications.
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
changes in v5:
- fix version: 20.0.2 -> 21
- fix commit log: populate_iova -> populate_virt
changes in v4:
- move populate_iova trace in v20_0_2 base function
changes in v3:
- rebase
- remove deprecation notice
- notify API change in release notes
- fix ABI version from 20.0.1 to 20.0.2 (should be 21 maybe)
doc/guides/rel_notes/deprecation.rst | 5 --
doc/guides/rel_notes/release_20_05.rst | 4 ++
examples/ntb/ntb_fwd.c | 2 +-
lib/librte_mempool/meson.build | 2 +
lib/librte_mempool/rte_mempool.c | 80 ++++++++++++++++++----
lib/librte_mempool/rte_mempool.h | 14 ++--
lib/librte_mempool/rte_mempool_version.map | 7 ++
7 files changed, 92 insertions(+), 22 deletions(-)
diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
index 1339f54f5..20aa745b7 100644
--- a/doc/guides/rel_notes/deprecation.rst
+++ b/doc/guides/rel_notes/deprecation.rst
@@ -65,11 +65,6 @@ Deprecation Notices
structure would be made internal (or removed if all dependencies are cleared)
in future releases.
-* mempool: starting from v20.05, the API of rte_mempool_populate_iova()
- and rte_mempool_populate_virt() will change to return 0 instead
- of -EINVAL when there is not enough room to store one object. The ABI
- will be preserved until 20.11.
-
* ethdev: the legacy filter API, including
``rte_eth_dev_filter_supported()``, ``rte_eth_dev_filter_ctrl()`` as well
as filter types MACVLAN, ETHERTYPE, FLEXIBLE, SYN, NTUPLE, TUNNEL, FDIR,
diff --git a/doc/guides/rel_notes/release_20_05.rst b/doc/guides/rel_notes/release_20_05.rst
index b124c3f28..ab20a7d02 100644
--- a/doc/guides/rel_notes/release_20_05.rst
+++ b/doc/guides/rel_notes/release_20_05.rst
@@ -241,6 +241,10 @@ API Changes
Also, make sure to start the actual text at the margin.
=========================================================
+* mempool: The API of ``rte_mempool_populate_iova()`` and
+ ``rte_mempool_populate_virt()`` changed to return 0 instead of -EINVAL
+ when there is not enough room to store one object.
+
ABI Changes
-----------
diff --git a/examples/ntb/ntb_fwd.c b/examples/ntb/ntb_fwd.c
index d49189e17..eba8ebf9f 100644
--- a/examples/ntb/ntb_fwd.c
+++ b/examples/ntb/ntb_fwd.c
@@ -1319,7 +1319,7 @@ ntb_mbuf_pool_create(uint16_t mbuf_seg_size, uint32_t nb_mbuf,
mz->len - ntb_info.ntb_hdr_size,
ntb_mempool_mz_free,
(void *)(uintptr_t)mz);
- if (ret < 0) {
+ if (ret <= 0) {
rte_memzone_free(mz);
rte_mempool_free(mp);
return NULL;
diff --git a/lib/librte_mempool/meson.build b/lib/librte_mempool/meson.build
index a6e861cbf..7dbe6b9be 100644
--- a/lib/librte_mempool/meson.build
+++ b/lib/librte_mempool/meson.build
@@ -9,6 +9,8 @@ foreach flag: extra_flags
endif
endforeach
+use_function_versioning = true
+
sources = files('rte_mempool.c', 'rte_mempool_ops.c',
'rte_mempool_ops_default.c', 'mempool_trace_points.c')
headers = files('rte_mempool.h', 'rte_mempool_trace.h',
diff --git a/lib/librte_mempool/rte_mempool.c b/lib/librte_mempool/rte_mempool.c
index 0be8f9f59..0bde995b5 100644
--- a/lib/librte_mempool/rte_mempool.c
+++ b/lib/librte_mempool/rte_mempool.c
@@ -31,6 +31,7 @@
#include <rte_string_fns.h>
#include <rte_spinlock.h>
#include <rte_tailq.h>
+#include <rte_function_versioning.h>
#include "rte_mempool.h"
#include "rte_mempool_trace.h"
@@ -303,12 +304,17 @@ mempool_ops_alloc_once(struct rte_mempool *mp)
return 0;
}
+__vsym int
+rte_mempool_populate_iova_v21(struct rte_mempool *mp, char *vaddr,
+ rte_iova_t iova, size_t len, rte_mempool_memchunk_free_cb_t *free_cb,
+ void *opaque);
+
/* Add objects in the pool, using a physically contiguous memory
* zone. Return the number of objects added, or a negative value
* on error.
*/
-static int
-__rte_mempool_populate_iova(struct rte_mempool *mp, char *vaddr,
+__vsym int
+rte_mempool_populate_iova_v21(struct rte_mempool *mp, char *vaddr,
rte_iova_t iova, size_t len, rte_mempool_memchunk_free_cb_t *free_cb,
void *opaque)
{
@@ -359,6 +365,8 @@ __rte_mempool_populate_iova(struct rte_mempool *mp, char *vaddr,
STAILQ_INSERT_TAIL(&mp->mem_list, memhdr, next);
mp->nb_mem_chunks++;
+
+ rte_mempool_trace_populate_iova(mp, vaddr, iova, len, free_cb, opaque);
return i;
fail:
@@ -366,21 +374,34 @@ __rte_mempool_populate_iova(struct rte_mempool *mp, char *vaddr,
return ret;
}
-int
-rte_mempool_populate_iova(struct rte_mempool *mp, char *vaddr,
+BIND_DEFAULT_SYMBOL(rte_mempool_populate_iova, _v21, 21);
+MAP_STATIC_SYMBOL(
+ int rte_mempool_populate_iova(struct rte_mempool *mp, char *vaddr,
+ rte_iova_t iova, size_t len,
+ rte_mempool_memchunk_free_cb_t *free_cb,
+ void *opaque),
+ rte_mempool_populate_iova_v21);
+
+__vsym int
+rte_mempool_populate_iova_v20(struct rte_mempool *mp, char *vaddr,
+ rte_iova_t iova, size_t len, rte_mempool_memchunk_free_cb_t *free_cb,
+ void *opaque);
+
+__vsym int
+rte_mempool_populate_iova_v20(struct rte_mempool *mp, char *vaddr,
rte_iova_t iova, size_t len, rte_mempool_memchunk_free_cb_t *free_cb,
void *opaque)
{
int ret;
- ret = __rte_mempool_populate_iova(mp, vaddr, iova, len, free_cb,
+ ret = rte_mempool_populate_iova_v21(mp, vaddr, iova, len, free_cb,
opaque);
if (ret == 0)
ret = -EINVAL;
- rte_mempool_trace_populate_iova(mp, vaddr, iova, len, free_cb, opaque);
return ret;
}
+VERSION_SYMBOL(rte_mempool_populate_iova, _v20, 20.0);
static rte_iova_t
get_iova(void *addr)
@@ -395,11 +416,16 @@ get_iova(void *addr)
return ms->iova + RTE_PTR_DIFF(addr, ms->addr);
}
+__vsym int
+rte_mempool_populate_virt_v21(struct rte_mempool *mp, char *addr,
+ size_t len, size_t pg_sz, rte_mempool_memchunk_free_cb_t *free_cb,
+ void *opaque);
+
/* Populate the mempool with a virtual area. Return the number of
* objects added, or a negative value on error.
*/
-int
-rte_mempool_populate_virt(struct rte_mempool *mp, char *addr,
+__vsym int
+rte_mempool_populate_virt_v21(struct rte_mempool *mp, char *addr,
size_t len, size_t pg_sz, rte_mempool_memchunk_free_cb_t *free_cb,
void *opaque)
{
@@ -432,7 +458,7 @@ rte_mempool_populate_virt(struct rte_mempool *mp, char *addr,
break;
}
- ret = __rte_mempool_populate_iova(mp, addr + off, iova,
+ ret = rte_mempool_populate_iova_v21(mp, addr + off, iova,
phys_len, free_cb, opaque);
if (ret == 0)
continue;
@@ -443,9 +469,6 @@ rte_mempool_populate_virt(struct rte_mempool *mp, char *addr,
cnt += ret;
}
- if (cnt == 0)
- return -EINVAL;
-
rte_mempool_trace_populate_virt(mp, addr, len, pg_sz, free_cb, opaque);
return cnt;
@@ -453,6 +476,35 @@ rte_mempool_populate_virt(struct rte_mempool *mp, char *addr,
rte_mempool_free_memchunks(mp);
return ret;
}
+BIND_DEFAULT_SYMBOL(rte_mempool_populate_virt, _v21, 21);
+MAP_STATIC_SYMBOL(
+ int rte_mempool_populate_virt(struct rte_mempool *mp,
+ char *addr, size_t len, size_t pg_sz,
+ rte_mempool_memchunk_free_cb_t *free_cb,
+ void *opaque),
+ rte_mempool_populate_virt_v21);
+
+__vsym int
+rte_mempool_populate_virt_v20(struct rte_mempool *mp, char *addr,
+ size_t len, size_t pg_sz, rte_mempool_memchunk_free_cb_t *free_cb,
+ void *opaque);
+
+__vsym int
+rte_mempool_populate_virt_v20(struct rte_mempool *mp, char *addr,
+ size_t len, size_t pg_sz, rte_mempool_memchunk_free_cb_t *free_cb,
+ void *opaque)
+{
+ int ret;
+
+ ret = rte_mempool_populate_virt_v21(mp, addr, len, pg_sz,
+ free_cb, opaque);
+
+ if (ret == 0)
+ ret = -EINVAL;
+
+ return ret;
+}
+VERSION_SYMBOL(rte_mempool_populate_virt, _v20, 20.0);
/* Get the minimal page size used in a mempool before populating it. */
int
@@ -609,6 +661,8 @@ rte_mempool_populate_default(struct rte_mempool *mp)
mz->len, pg_sz,
rte_mempool_memchunk_mz_free,
(void *)(uintptr_t)mz);
+ if (ret == 0) /* should not happen */
+ ret = -ENOBUFS;
if (ret < 0) {
rte_memzone_free(mz);
goto fail;
@@ -701,6 +755,8 @@ rte_mempool_populate_anon(struct rte_mempool *mp)
ret = rte_mempool_populate_virt(mp, addr, size, getpagesize(),
rte_mempool_memchunk_anon_free, addr);
+ if (ret == 0) /* should not happen */
+ ret = -ENOBUFS;
if (ret < 0) {
rte_errno = -ret;
goto fail;
diff --git a/lib/librte_mempool/rte_mempool.h b/lib/librte_mempool/rte_mempool.h
index 6e0573ea4..652d19f9f 100644
--- a/lib/librte_mempool/rte_mempool.h
+++ b/lib/librte_mempool/rte_mempool.h
@@ -1112,9 +1112,12 @@ rte_mempool_free(struct rte_mempool *mp);
* @param opaque
* An opaque argument passed to free_cb.
* @return
- * The number of objects added on success.
+ * The number of objects added on success (strictly positive).
* On error, the chunk is not added in the memory list of the
- * mempool and a negative errno is returned.
+ * mempool the following code is returned:
+ * (0): not enough room in chunk for one object.
+ * (-ENOSPC): mempool is already populated.
+ * (-ENOMEM): allocation failure.
*/
int rte_mempool_populate_iova(struct rte_mempool *mp, char *vaddr,
rte_iova_t iova, size_t len, rte_mempool_memchunk_free_cb_t *free_cb,
@@ -1139,9 +1142,12 @@ int rte_mempool_populate_iova(struct rte_mempool *mp, char *vaddr,
* @param opaque
* An opaque argument passed to free_cb.
* @return
- * The number of objects added on success.
+ * The number of objects added on success (strictly positive).
* On error, the chunk is not added in the memory list of the
- * mempool and a negative errno is returned.
+ * mempool the following code is returned:
+ * (0): not enough room in chunk for one object.
+ * (-ENOSPC): mempool is already populated.
+ * (-ENOMEM): allocation failure.
*/
int
rte_mempool_populate_virt(struct rte_mempool *mp, char *addr,
diff --git a/lib/librte_mempool/rte_mempool_version.map b/lib/librte_mempool/rte_mempool_version.map
index 695dd6e04..826a0b882 100644
--- a/lib/librte_mempool/rte_mempool_version.map
+++ b/lib/librte_mempool/rte_mempool_version.map
@@ -31,6 +31,13 @@ DPDK_20.0 {
local: *;
};
+DPDK_21 {
+ global:
+
+ rte_mempool_populate_iova;
+ rte_mempool_populate_virt;
+} DPDK_20.0;
+
EXPERIMENTAL {
global:
--
2.25.1
^ permalink raw reply [relevance 11%]
* [dpdk-dev] [PATCH v3 0/8] NXP DPAAx fixes and enhancements
@ 2020-05-04 12:41 3% ` Hemant Agrawal
0 siblings, 0 replies; 200+ results
From: Hemant Agrawal @ 2020-05-04 12:41 UTC (permalink / raw)
To: dev, ferruh.yigit
v3: Limiting the patches to avoid ABI breakage.
Apeksha Gupta (1):
bus/fslmc: fix dereferencing null pointer
Hemant Agrawal (2):
net/dpaa2: add default Rx params in devinfo
net/dpaa2: reduce prints in queue count functions
Jun Yang (1):
net/dpaa2: use cong group id for multiple tcs
Nipun Gupta (3):
net/dpaa2: do not prefetch annotaion for physical mode
drivers: dpaa2 enhance portal alloc failure log
net/dpaa2: support UDP dst port based muxing
Rohit Raj (1):
net/dpaa2: fix 10g port negotiation issue
drivers/bus/fslmc/portal/dpaa2_hw_pvt.h | 6 +--
drivers/bus/fslmc/qbman/qbman_debug.c | 9 ++--
drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c | 8 ++-
drivers/event/dpaa2/dpaa2_eventdev.c | 8 ++-
drivers/mempool/dpaa2/dpaa2_hw_mempool.c | 12 +++--
drivers/net/dpaa/dpaa_ethdev.c | 4 ++
drivers/net/dpaa/dpaa_ethdev.h | 1 +
drivers/net/dpaa2/dpaa2_ethdev.c | 32 ++++++++----
drivers/net/dpaa2/dpaa2_ethdev.h | 2 +
drivers/net/dpaa2/dpaa2_mux.c | 24 ++++++++-
drivers/net/dpaa2/dpaa2_rxtx.c | 56 ++++++++++++++-------
drivers/raw/dpaa2_cmdif/dpaa2_cmdif.c | 8 ++-
drivers/raw/dpaa2_qdma/dpaa2_qdma.c | 12 +++--
13 files changed, 134 insertions(+), 48 deletions(-)
--
2.17.1
^ permalink raw reply [relevance 3%]
* Re: [dpdk-dev] [PATCH] common/octeontx: fix GCC 9.1 ABI break
2020-05-02 16:10 4% [dpdk-dev] [PATCH] common/octeontx: fix GCC 9.1 ABI break pbhagavatula
@ 2020-05-04 9:17 4% ` Harman Kalra
0 siblings, 0 replies; 200+ results
From: Harman Kalra @ 2020-05-04 9:17 UTC (permalink / raw)
To: pbhagavatula; +Cc: jerinj, ferruh.yigit, thomas, dev, stable
On Sat, May 02, 2020 at 09:40:31PM +0530, pbhagavatula@marvell.com wrote:
> From: Pavan Nikhilesh <pbhagavatula@marvell.com>
>
> GCC 9.1 fixes a bug with passing bitfields as pass by value in function
> parameters and generates a warning for the same as below:
>
> drivers/common/octeontx/octeontx_mbox.c:282:1: note: parameter passing
> for argument of type ‘struct mbox_intf_ver’ changed in GCC 9.1
>
> Fix the warning generated by passing bitfield as pass by reference.
>
> Fixes: b4134b2d31cc ("common/octeontx: update mbox to version 1.1.3")
> Cc: stable@dpdk.org
>
> Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
Acked-by: Harman Kalra <hkalra@marvell.com>
> ---
> More info on GCC bug
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88469
> https://gcc.gnu.org/git/?p=gcc.git&a=commit;h=c590597c45948c6e6fa282878198fd226da95998
>
> drivers/common/octeontx/octeontx_mbox.c | 17 +++++++++--------
> 1 file changed, 9 insertions(+), 8 deletions(-)
>
> diff --git a/drivers/common/octeontx/octeontx_mbox.c b/drivers/common/octeontx/octeontx_mbox.c
> index 2fd253107..effe0b267 100644
> --- a/drivers/common/octeontx/octeontx_mbox.c
> +++ b/drivers/common/octeontx/octeontx_mbox.c
> @@ -279,7 +279,7 @@ octeontx_start_domain(void)
> }
>
> static int
> -octeontx_check_mbox_version(struct mbox_intf_ver app_intf_ver,
> +octeontx_check_mbox_version(struct mbox_intf_ver *app_intf_ver,
> struct mbox_intf_ver *intf_ver)
> {
> struct mbox_intf_ver kernel_intf_ver = {0};
> @@ -290,8 +290,9 @@ octeontx_check_mbox_version(struct mbox_intf_ver app_intf_ver,
> hdr.coproc = NO_COPROC;
> hdr.msg = RM_INTERFACE_VERSION;
>
> - result = octeontx_mbox_send(&hdr, &app_intf_ver, sizeof(app_intf_ver),
> - &kernel_intf_ver, sizeof(kernel_intf_ver));
> + result = octeontx_mbox_send(&hdr, app_intf_ver,
> + sizeof(struct mbox_intf_ver),
> + &kernel_intf_ver, sizeof(kernel_intf_ver));
> if (result != sizeof(kernel_intf_ver)) {
> mbox_log_err("Could not send interface version. Err=%d. FuncErr=%d\n",
> result, hdr.res_code);
> @@ -301,9 +302,9 @@ octeontx_check_mbox_version(struct mbox_intf_ver app_intf_ver,
> if (intf_ver)
> *intf_ver = kernel_intf_ver;
>
> - if (app_intf_ver.platform != kernel_intf_ver.platform ||
> - app_intf_ver.major != kernel_intf_ver.major ||
> - app_intf_ver.minor != kernel_intf_ver.minor)
> + if (app_intf_ver->platform != kernel_intf_ver.platform ||
> + app_intf_ver->major != kernel_intf_ver.major ||
> + app_intf_ver->minor != kernel_intf_ver.minor)
> result = -EINVAL;
>
> return result;
> @@ -312,7 +313,7 @@ octeontx_check_mbox_version(struct mbox_intf_ver app_intf_ver,
> int
> octeontx_mbox_init(void)
> {
> - const struct mbox_intf_ver MBOX_INTERFACE_VERSION = {
> + struct mbox_intf_ver MBOX_INTERFACE_VERSION = {
> .platform = 0x01,
> .major = 0x01,
> .minor = 0x03
> @@ -330,7 +331,7 @@ octeontx_mbox_init(void)
> return ret;
> }
>
> - ret = octeontx_check_mbox_version(MBOX_INTERFACE_VERSION,
> + ret = octeontx_check_mbox_version(&MBOX_INTERFACE_VERSION,
> &rm_intf_ver);
> if (ret < 0) {
> mbox_log_err("MBOX version: Kernel(%d.%d.%d) != DPDK(%d.%d.%d)",
> --
> 2.26.2
>
^ permalink raw reply [relevance 4%]
* [dpdk-dev] [PATCH v3 1/4] lib: introduce IF Proxy library
2020-05-04 8:53 2% ` [dpdk-dev] [PATCH v3 " Andrzej Ostruszka
@ 2020-05-04 8:53 1% ` Andrzej Ostruszka
0 siblings, 0 replies; 200+ results
From: Andrzej Ostruszka @ 2020-05-04 8:53 UTC (permalink / raw)
To: dev, Thomas Monjalon
This library allows to designate ports visible to the system (such as
Tun/Tap or KNI) as port representors serving as proxies for other DPDK
ports. When such a proxy is configured this library initially queries
network configuration from the system and later monitors its changes.
The information gathered is passed to the application either via a set
of user registered callbacks or as an event added to the configured
notification queue (or a combination of these two mechanisms). This way
user can use normal network utilities (like those from the iproute2
suite) to configure DPDK ports.
Signed-off-by: Andrzej Ostruszka <aostruszka@marvell.com>
---
MAINTAINERS | 3 +
config/common_base | 5 +
config/common_linux | 1 +
lib/Makefile | 2 +
lib/librte_eal/include/rte_eal_interrupts.h | 2 +
lib/librte_eal/linux/eal_interrupts.c | 14 +-
lib/librte_if_proxy/Makefile | 29 +
lib/librte_if_proxy/if_proxy_common.c | 564 ++++++++++++++++++
lib/librte_if_proxy/if_proxy_priv.h | 97 +++
lib/librte_if_proxy/linux/Makefile | 4 +
lib/librte_if_proxy/linux/if_proxy.c | 563 ++++++++++++++++++
lib/librte_if_proxy/meson.build | 19 +
lib/librte_if_proxy/rte_if_proxy.h | 585 +++++++++++++++++++
lib/librte_if_proxy/rte_if_proxy_version.map | 20 +
lib/meson.build | 2 +-
15 files changed, 1905 insertions(+), 5 deletions(-)
create mode 100644 lib/librte_if_proxy/Makefile
create mode 100644 lib/librte_if_proxy/if_proxy_common.c
create mode 100644 lib/librte_if_proxy/if_proxy_priv.h
create mode 100644 lib/librte_if_proxy/linux/Makefile
create mode 100644 lib/librte_if_proxy/linux/if_proxy.c
create mode 100644 lib/librte_if_proxy/meson.build
create mode 100644 lib/librte_if_proxy/rte_if_proxy.h
create mode 100644 lib/librte_if_proxy/rte_if_proxy_version.map
diff --git a/MAINTAINERS b/MAINTAINERS
index e05c80504..1013745ce 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1472,6 +1472,9 @@ F: examples/bpf/
F: app/test/test_bpf.c
F: doc/guides/prog_guide/bpf_lib.rst
+IF Proxy - EXPERIMENTAL
+M: Andrzej Ostruszka <aostruszka@marvell.com>
+F: lib/librte_if_proxy/
Test Applications
-----------------
diff --git a/config/common_base b/config/common_base
index 14000ba07..95ca8dbf6 100644
--- a/config/common_base
+++ b/config/common_base
@@ -1087,6 +1087,11 @@ CONFIG_RTE_LIBRTE_BPF_ELF=n
#
CONFIG_RTE_LIBRTE_IPSEC=y
+#
+# Compile librte_if_proxy
+#
+CONFIG_RTE_LIBRTE_IF_PROXY=n
+
#
# Compile the test application
#
diff --git a/config/common_linux b/config/common_linux
index 816810671..1244eb0ae 100644
--- a/config/common_linux
+++ b/config/common_linux
@@ -16,6 +16,7 @@ CONFIG_RTE_LIBRTE_VHOST_NUMA=y
CONFIG_RTE_LIBRTE_VHOST_POSTCOPY=n
CONFIG_RTE_LIBRTE_PMD_VHOST=y
CONFIG_RTE_LIBRTE_IFC_PMD=y
+CONFIG_RTE_LIBRTE_IF_PROXY=y
CONFIG_RTE_LIBRTE_PMD_AF_PACKET=y
CONFIG_RTE_LIBRTE_PMD_MEMIF=y
CONFIG_RTE_LIBRTE_PMD_SOFTNIC=y
diff --git a/lib/Makefile b/lib/Makefile
index d0ec3919b..1e7d78183 100644
--- a/lib/Makefile
+++ b/lib/Makefile
@@ -118,6 +118,8 @@ DIRS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += librte_telemetry
DEPDIRS-librte_telemetry := librte_eal librte_metrics librte_ethdev
DIRS-$(CONFIG_RTE_LIBRTE_RCU) += librte_rcu
DEPDIRS-librte_rcu := librte_eal librte_ring
+DIRS-$(CONFIG_RTE_LIBRTE_IF_PROXY) += librte_if_proxy
+DEPDIRS-librte_if_proxy := librte_eal librte_ethdev
ifeq ($(CONFIG_RTE_EXEC_ENV_LINUX),y)
DIRS-$(CONFIG_RTE_LIBRTE_KNI) += librte_kni
diff --git a/lib/librte_eal/include/rte_eal_interrupts.h b/lib/librte_eal/include/rte_eal_interrupts.h
index 773a34a42..296a3853d 100644
--- a/lib/librte_eal/include/rte_eal_interrupts.h
+++ b/lib/librte_eal/include/rte_eal_interrupts.h
@@ -36,6 +36,8 @@ enum rte_intr_handle_type {
RTE_INTR_HANDLE_VDEV, /**< virtual device */
RTE_INTR_HANDLE_DEV_EVENT, /**< device event handle */
RTE_INTR_HANDLE_VFIO_REQ, /**< VFIO request handle */
+ RTE_INTR_HANDLE_NETLINK, /**< netlink notification handle */
+
RTE_INTR_HANDLE_MAX /**< count of elements */
};
diff --git a/lib/librte_eal/linux/eal_interrupts.c b/lib/librte_eal/linux/eal_interrupts.c
index 16e7a7d51..91ddafc59 100644
--- a/lib/librte_eal/linux/eal_interrupts.c
+++ b/lib/librte_eal/linux/eal_interrupts.c
@@ -691,6 +691,9 @@ rte_intr_enable(const struct rte_intr_handle *intr_handle)
break;
/* not used at this moment */
case RTE_INTR_HANDLE_ALARM:
+#if RTE_LIBRTE_IF_PROXY
+ case RTE_INTR_HANDLE_NETLINK:
+#endif
rc = -1;
break;
#ifdef VFIO_PRESENT
@@ -818,6 +821,9 @@ rte_intr_disable(const struct rte_intr_handle *intr_handle)
break;
/* not used at this moment */
case RTE_INTR_HANDLE_ALARM:
+#if RTE_LIBRTE_IF_PROXY
+ case RTE_INTR_HANDLE_NETLINK:
+#endif
rc = -1;
break;
#ifdef VFIO_PRESENT
@@ -915,12 +921,12 @@ eal_intr_process_interrupts(struct epoll_event *events, int nfds)
break;
#endif
#endif
- case RTE_INTR_HANDLE_VDEV:
case RTE_INTR_HANDLE_EXT:
- bytes_read = 0;
- call = true;
- break;
+ case RTE_INTR_HANDLE_VDEV:
case RTE_INTR_HANDLE_DEV_EVENT:
+#if RTE_LIBRTE_IF_PROXY
+ case RTE_INTR_HANDLE_NETLINK:
+#endif
bytes_read = 0;
call = true;
break;
diff --git a/lib/librte_if_proxy/Makefile b/lib/librte_if_proxy/Makefile
new file mode 100644
index 000000000..43cb702a2
--- /dev/null
+++ b/lib/librte_if_proxy/Makefile
@@ -0,0 +1,29 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(C) 2020 Marvell International Ltd.
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+# library name
+LIB = librte_if_proxy.a
+
+CFLAGS += -DALLOW_EXPERIMENTAL_API
+CFLAGS += -O3
+CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR)
+LDLIBS += -lrte_eal -lrte_ethdev
+
+EXPORT_MAP := rte_if_proxy_version.map
+
+LIBABIVER := 1
+
+# all source are stored in SRCS-y
+SRCS-$(CONFIG_RTE_LIBRTE_IF_PROXY) := if_proxy_common.c
+
+SYSDIR := $(patsubst "%app",%,$(CONFIG_RTE_EXEC_ENV))
+include $(SRCDIR)/$(SYSDIR)/Makefile
+
+SRCS-$(CONFIG_RTE_LIBRTE_IF_PROXY) += $(addprefix $(SYSDIR)/,$(SRCS))
+
+# install this header file
+SYMLINK-$(CONFIG_RTE_LIBRTE_IF_PROXY)-include := rte_if_proxy.h
+
+include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/lib/librte_if_proxy/if_proxy_common.c b/lib/librte_if_proxy/if_proxy_common.c
new file mode 100644
index 000000000..6f72511f4
--- /dev/null
+++ b/lib/librte_if_proxy/if_proxy_common.c
@@ -0,0 +1,564 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(C) 2020 Marvell International Ltd.
+ */
+
+#include "if_proxy_priv.h"
+#include <rte_string_fns.h>
+
+/* Definitions of data mentioned in if_proxy_priv.h and local ones. */
+int ifpx_log_type;
+
+/* Table keeping mapping between port and their proxies. */
+static
+uint16_t ifpx_ports[RTE_MAX_ETHPORTS];
+
+rte_spinlock_t ifpx_lock = RTE_SPINLOCK_INITIALIZER;
+
+struct ifpx_proxies_head ifpx_proxies = TAILQ_HEAD_INITIALIZER(ifpx_proxies);
+
+struct ifpx_queue_node {
+ TAILQ_ENTRY(ifpx_queue_node) elem;
+ uint16_t state;
+ struct rte_ring *r;
+};
+static
+TAILQ_HEAD(ifpx_queues_head, ifpx_queue_node) ifpx_queues =
+ TAILQ_HEAD_INITIALIZER(ifpx_queues);
+
+/* All callbacks have similar signature (taking pointer to some event) so we'll
+ * use this f_ptr to typecast and invoke them in a generic way. There is one
+ * exception though - notification about completed initial configuration - and
+ * it is handled separately.
+ */
+union ifpx_cb_ptr {
+ int (*f_ptr)(void *ev); /* type for normal event notification */
+ union rte_ifpx_cb_ptr cb;
+} ifpx_callbacks[RTE_IFPX_NUM_EVENTS];
+
+uint64_t rte_ifpx_events_available(void)
+{
+ if (ifpx_platform.events)
+ return ifpx_platform.events();
+
+ /* If callback is not provided then all events are supported. */
+ return (1ULL << RTE_IFPX_NUM_EVENTS) - 1;
+}
+
+uint16_t rte_ifpx_proxy_create(enum rte_ifpx_proxy_type type)
+{
+ char devargs[16] = { '\0' };
+ int dev_cnt = 0, nlen;
+ uint16_t port_id;
+
+ switch (type) {
+ case RTE_IFPX_DEFAULT:
+ case RTE_IFPX_TAP:
+ nlen = strlcpy(devargs, "net_tap", sizeof(devargs));
+ break;
+ case RTE_IFPX_KNI:
+ nlen = strlcpy(devargs, "net_kni", sizeof(devargs));
+ break;
+ default:
+ IFPX_LOG(ERR, "Unknown proxy type: %d", type);
+ return RTE_MAX_ETHPORTS;
+ }
+
+ RTE_ETH_FOREACH_DEV(port_id) {
+ if (strcmp(rte_eth_devices[port_id].device->driver->name,
+ devargs) == 0)
+ ++dev_cnt;
+ }
+ snprintf(devargs+nlen, sizeof(devargs)-nlen, "%d", dev_cnt);
+
+ return rte_ifpx_proxy_create_by_devarg(devargs);
+}
+
+uint16_t rte_ifpx_proxy_create_by_devarg(const char *devarg)
+{
+ uint16_t port_id = RTE_MAX_ETHPORTS;
+ struct rte_dev_iterator iter;
+
+ if (rte_dev_probe(devarg) < 0) {
+ IFPX_LOG(ERR, "Failed to create proxy port %s\n", devarg);
+ return RTE_MAX_ETHPORTS;
+ }
+
+ if (rte_eth_iterator_init(&iter, devarg) == 0) {
+ port_id = rte_eth_iterator_next(&iter);
+ if (port_id != RTE_MAX_ETHPORTS)
+ rte_eth_iterator_cleanup(&iter);
+ }
+
+ return port_id;
+}
+
+int ifpx_proxy_destroy(struct ifpx_proxy_node *px)
+{
+ unsigned int i;
+ uint16_t proxy_id = px->proxy_id;
+
+ /* This function is expected to be called with a lock held. */
+ RTE_ASSERT(rte_spinlock_trylock(&ifpx_lock) == 0);
+
+ if (px->state & IN_USE) {
+ px->state |= DEL_PENDING;
+ return 0;
+ }
+
+ TAILQ_REMOVE(&ifpx_proxies, px, elem);
+ free(px);
+
+ /* Clear any bindings for this proxy. */
+ for (i = 0; i < RTE_DIM(ifpx_ports); ++i) {
+ if (ifpx_ports[i] == proxy_id)
+ ifpx_ports[i] = RTE_MAX_ETHPORTS;
+ }
+
+ return rte_dev_remove(rte_eth_devices[proxy_id].device);
+}
+
+int rte_ifpx_proxy_destroy(uint16_t proxy_id)
+{
+ struct ifpx_proxy_node *px;
+ int ec;
+
+ rte_spinlock_lock(&ifpx_lock);
+ TAILQ_FOREACH(px, &ifpx_proxies, elem) {
+ if (px->proxy_id == proxy_id)
+ break;
+ }
+ if (!px) {
+ ec = -EINVAL;
+ goto exit;
+ }
+
+ ec = ifpx_proxy_destroy(px);
+exit:
+ rte_spinlock_unlock(&ifpx_lock);
+ return ec;
+}
+
+int rte_ifpx_queue_add(struct rte_ring *r)
+{
+ struct ifpx_queue_node *node;
+ int ec = 0;
+
+ if (!r)
+ return -EINVAL;
+
+ rte_spinlock_lock(&ifpx_lock);
+ TAILQ_FOREACH(node, &ifpx_queues, elem) {
+ if (node->r == r) {
+ ec = -EEXIST;
+ goto exit;
+ }
+ }
+
+ node = malloc(sizeof(*node));
+ if (!node) {
+ ec = -ENOMEM;
+ goto exit;
+ }
+
+ node->r = r;
+ TAILQ_INSERT_TAIL(&ifpx_queues, node, elem);
+exit:
+ rte_spinlock_unlock(&ifpx_lock);
+
+ return ec;
+}
+
+int rte_ifpx_queue_remove(struct rte_ring *r)
+{
+ struct ifpx_queue_node *node, *next;
+ int ec = -EINVAL;
+
+ if (!r)
+ return ec;
+
+ rte_spinlock_lock(&ifpx_lock);
+ for (node = TAILQ_FIRST(&ifpx_queues); node; node = next) {
+ next = TAILQ_NEXT(node, elem);
+ if (node->r != r)
+ continue;
+ TAILQ_REMOVE(&ifpx_queues, node, elem);
+ free(node);
+ ec = 0;
+ break;
+ }
+ rte_spinlock_unlock(&ifpx_lock);
+
+ return ec;
+}
+
+int rte_ifpx_port_bind(uint16_t port_id, uint16_t proxy_id)
+{
+ struct rte_eth_dev_info proxy_eth_info;
+ struct ifpx_proxy_node *px;
+ int ec;
+
+ rte_spinlock_lock(&ifpx_lock);
+
+ if (port_id >= RTE_MAX_ETHPORTS || proxy_id >= RTE_MAX_ETHPORTS ||
+ /* port is a proxy */
+ ifpx_ports[port_id] == port_id) {
+ IFPX_LOG(ERR, "Invalid port_id: %d", port_id);
+ ec = -EINVAL;
+ goto error;
+ }
+
+ /* Do automatic rebinding but issue a warning since this is not
+ * considered to be a valid behaviour.
+ */
+ if (ifpx_ports[port_id] != RTE_MAX_ETHPORTS) {
+ IFPX_LOG(WARNING, "Port already bound: %d -> %d", port_id,
+ ifpx_ports[port_id]);
+ }
+
+ /* Search for existing proxy - if not found add one to the list. */
+ TAILQ_FOREACH(px, &ifpx_proxies, elem) {
+ if (px->proxy_id == proxy_id)
+ break;
+ }
+ if (!px) {
+ ec = rte_eth_dev_info_get(proxy_id, &proxy_eth_info);
+ if (ec < 0 || proxy_eth_info.if_index == 0) {
+ IFPX_LOG(ERR, "Invalid proxy: %d", proxy_id);
+ if (ec >= 0)
+ ec = -EINVAL;
+ goto error;
+ }
+ px = malloc(sizeof(*px));
+ if (!px) {
+ ec = -ENOMEM;
+ goto error;
+ }
+ px->proxy_id = proxy_id;
+ px->info.if_index = proxy_eth_info.if_index;
+ rte_eth_dev_get_mtu(proxy_id, &px->info.mtu);
+ rte_eth_macaddr_get(proxy_id, &px->info.mac);
+ memset(px->info.if_name, 0, sizeof(px->info.if_name));
+ TAILQ_INSERT_TAIL(&ifpx_proxies, px, elem);
+ }
+ ifpx_ports[port_id] = proxy_id;
+ rte_spinlock_unlock(&ifpx_lock);
+
+ /* Add proxy MAC to the port - since port will often just forward
+ * packets from the proxy/system they will be sent with proxy MAC as
+ * src. In order to pass communication in other direction we should be
+ * accepting packets with proxy MAC as dst.
+ */
+ rte_eth_dev_mac_addr_add(port_id, &px->info.mac, 0);
+
+ if (ifpx_platform.get_info)
+ ifpx_platform.get_info(px->info.if_index);
+
+ return 0;
+
+error:
+ rte_spinlock_unlock(&ifpx_lock);
+ return ec;
+}
+
+int rte_ifpx_port_unbind(uint16_t port_id)
+{
+ unsigned int i, cnt;
+ uint16_t proxy_id;
+ struct ifpx_proxy_node *px;
+ int ec = 0;
+
+ rte_spinlock_lock(&ifpx_lock);
+ if (port_id >= RTE_MAX_ETHPORTS ||
+ ifpx_ports[port_id] == RTE_MAX_ETHPORTS ||
+ /* port is a proxy */
+ ifpx_ports[port_id] == port_id) {
+ ec = -EINVAL;
+ goto exit;
+ }
+
+ proxy_id = ifpx_ports[port_id];
+ ifpx_ports[port_id] = RTE_MAX_ETHPORTS;
+
+ for (i = 0, cnt = 0; i < RTE_DIM(ifpx_ports); ++i) {
+ if (ifpx_ports[i] == proxy_id)
+ ++cnt;
+ }
+
+ /* If there is no port bound to this proxy then remove it. */
+ if (cnt == 0) {
+ TAILQ_FOREACH(px, &ifpx_proxies, elem) {
+ if (px->proxy_id == proxy_id)
+ break;
+ }
+ RTE_ASSERT(px);
+ ec = ifpx_proxy_destroy(px);
+ }
+exit:
+ rte_spinlock_unlock(&ifpx_lock);
+ return ec;
+}
+
+int rte_ifpx_callbacks_register(unsigned int len,
+ const struct rte_ifpx_callback cbs[])
+{
+ unsigned int i;
+
+ if (!cbs || len == 0)
+ return -EINVAL;
+
+ rte_spinlock_lock(&ifpx_lock);
+
+ for (i = 0; i < len; ++i) {
+ if (cbs[i].type < 0 || cbs[i].type > RTE_IFPX_LAST_EVENT) {
+ IFPX_LOG(WARNING, "Invalid event type: %d",
+ cbs[i].type);
+ continue;
+ }
+ ifpx_callbacks[i].cb = cbs[i].callback;
+ }
+
+ rte_spinlock_unlock(&ifpx_lock);
+
+ return 0;
+}
+
+void rte_ifpx_callbacks_unregister_all(void)
+{
+ rte_spinlock_lock(&ifpx_lock);
+ memset(&ifpx_callbacks, 0, sizeof(ifpx_callbacks));
+ rte_spinlock_unlock(&ifpx_lock);
+}
+
+int rte_ifpx_callbacks_unregister(enum rte_ifpx_event_type ev)
+{
+ if (ev < 0 || ev > RTE_IFPX_CFG_DONE)
+ return -EINVAL;
+
+ rte_spinlock_lock(&ifpx_lock);
+ ifpx_callbacks[ev].f_ptr = NULL;
+ rte_spinlock_unlock(&ifpx_lock);
+
+ return 0;
+}
+
+uint16_t rte_ifpx_proxy_get(uint16_t port_id)
+{
+ uint16_t p = RTE_MAX_ETHPORTS;
+
+ if (port_id < RTE_MAX_ETHPORTS) {
+ rte_spinlock_lock(&ifpx_lock);
+ p = ifpx_ports[port_id];
+ rte_spinlock_unlock(&ifpx_lock);
+ }
+
+ return p;
+}
+
+unsigned int rte_ifpx_port_get(uint16_t proxy_id,
+ uint16_t *ports, unsigned int num)
+{
+ unsigned int p, cnt = 0;
+
+ rte_spinlock_lock(&ifpx_lock);
+ for (p = 0; p < RTE_DIM(ifpx_ports); ++p) {
+ if (ifpx_ports[p] == proxy_id && ifpx_ports[p] != p) {
+ ++cnt;
+ if (ports && num > 0) {
+ *ports++ = p;
+ --num;
+ }
+ }
+ }
+ rte_spinlock_unlock(&ifpx_lock);
+
+ return cnt;
+}
+
+const struct rte_ifpx_info *rte_ifpx_info_get(uint16_t port_id)
+{
+ struct ifpx_proxy_node *px;
+
+ rte_spinlock_lock(&ifpx_lock);
+
+ if (port_id >= RTE_MAX_ETHPORTS ||
+ ifpx_ports[port_id] == RTE_MAX_ETHPORTS) {
+ rte_spinlock_unlock(&ifpx_lock);
+ return NULL;
+ }
+
+ TAILQ_FOREACH(px, &ifpx_proxies, elem) {
+ if (px->proxy_id == ifpx_ports[port_id])
+ break;
+ }
+ rte_spinlock_unlock(&ifpx_lock);
+ RTE_ASSERT(px && "Internal IF Proxy library error");
+
+ return &px->info;
+}
+
+static
+void queue_event(const struct rte_ifpx_event *ev, struct rte_ring *r)
+{
+ struct rte_ifpx_event *e = malloc(sizeof(*ev));
+
+ if (!e) {
+ IFPX_LOG(ERR, "Failed to allocate event!");
+ return;
+ }
+ RTE_ASSERT(r);
+
+ *e = *ev;
+ rte_ring_sp_enqueue(r, e);
+}
+
+void ifpx_notify_event(struct rte_ifpx_event *ev, struct ifpx_proxy_node *px)
+{
+ struct ifpx_queue_node *q;
+ int done = 0;
+ uint16_t p, proxy_id;
+
+ if (px) {
+ if (px->state & DEL_PENDING)
+ return;
+ proxy_id = px->proxy_id;
+ RTE_ASSERT(proxy_id != RTE_MAX_ETHPORTS);
+ px->state |= IN_USE;
+ } else
+ proxy_id = RTE_MAX_ETHPORTS;
+
+ RTE_ASSERT(ev && ev->type >= 0 && ev->type <= RTE_IFPX_LAST_EVENT);
+ /* This function is expected to be called with a lock held. */
+ RTE_ASSERT(rte_spinlock_trylock(&ifpx_lock) == 0);
+
+ if (ifpx_callbacks[ev->type].f_ptr) {
+ union ifpx_cb_ptr fun = ifpx_callbacks[ev->type];
+
+ /* Below we drop the lock for the time of callback call to allow
+ * for calling of IF Proxy API.
+ */
+ if (px) {
+ for (p = 0; p < RTE_DIM(ifpx_ports); ++p) {
+ if (ifpx_ports[p] != proxy_id ||
+ ifpx_ports[p] == p)
+ continue;
+ ev->data.port_id = p;
+ rte_spinlock_unlock(&ifpx_lock);
+ done = fun.f_ptr(&ev->data) || done;
+ rte_spinlock_lock(&ifpx_lock);
+ }
+ } else {
+ RTE_ASSERT(ev->type == RTE_IFPX_CFG_DONE);
+ rte_spinlock_unlock(&ifpx_lock);
+ done = fun.cb.cfg_done();
+ rte_spinlock_lock(&ifpx_lock);
+ }
+ }
+ if (done)
+ goto exit;
+
+ /* Event not "consumed" yet so try to notify via queues. */
+ TAILQ_FOREACH(q, &ifpx_queues, elem) {
+ if (px) {
+ for (p = 0; p < RTE_DIM(ifpx_ports); ++p) {
+ if (ifpx_ports[p] != proxy_id ||
+ ifpx_ports[p] == p)
+ continue;
+ /* Set the port_id - the remaining params should
+ * be filled before calling this function.
+ */
+ ev->data.port_id = p;
+ queue_event(ev, q->r);
+ }
+ } else
+ queue_event(ev, q->r);
+ }
+exit:
+ if (px)
+ px->state &= ~IN_USE;
+}
+
+void ifpx_cleanup_proxies(void)
+{
+ struct ifpx_proxy_node *px, *next;
+ for (px = TAILQ_FIRST(&ifpx_proxies); px; px = next) {
+ next = TAILQ_NEXT(px, elem);
+ if (px->state & DEL_PENDING)
+ ifpx_proxy_destroy(px);
+ }
+}
+
+int rte_ifpx_listen(void)
+{
+ int ec;
+
+ if (!ifpx_platform.listen)
+ return -ENOTSUP;
+
+ ec = ifpx_platform.listen();
+ if (ec == 0 && ifpx_platform.get_info)
+ ifpx_platform.get_info(0);
+
+ return ec;
+}
+
+int rte_ifpx_close(void)
+{
+ struct ifpx_proxy_node *px;
+ struct ifpx_queue_node *q;
+ unsigned int p;
+ int ec = 0;
+
+ rte_spinlock_lock(&ifpx_lock);
+
+ if (ifpx_platform.close) {
+ ec = ifpx_platform.close();
+ if (ec != 0)
+ IFPX_LOG(ERR, "Platform 'close' calback failed.");
+ }
+
+ /* Remove queues. */
+ while (!TAILQ_EMPTY(&ifpx_queues)) {
+ q = TAILQ_FIRST(&ifpx_queues);
+ TAILQ_REMOVE(&ifpx_queues, q, elem);
+ free(q);
+ }
+
+ /* Clear callbacks. */
+ memset(&ifpx_callbacks, 0, sizeof(ifpx_callbacks));
+
+ /* Unbind ports. */
+ for (p = 0; p < RTE_DIM(ifpx_ports); ++p) {
+ if (ifpx_ports[p] == RTE_MAX_ETHPORTS)
+ continue;
+ /* We don't need to call rte_ifpx_port_unbind() here since we
+ * clear proxies below anyway, just clearing the mapping is
+ * enough (and besides it would deadlock :)).
+ */
+ ifpx_ports[p] = RTE_MAX_ETHPORTS;
+ }
+
+ /* Clear proxies. */
+ while (!TAILQ_EMPTY(&ifpx_proxies)) {
+ px = TAILQ_FIRST(&ifpx_proxies);
+ TAILQ_REMOVE(&ifpx_proxies, px, elem);
+ free(px);
+ }
+
+ rte_spinlock_unlock(&ifpx_lock);
+
+ return ec;
+}
+
+RTE_INIT(if_proxy_init)
+{
+ unsigned int i;
+ for (i = 0; i < RTE_DIM(ifpx_ports); ++i)
+ ifpx_ports[i] = RTE_MAX_ETHPORTS;
+
+ ifpx_log_type = rte_log_register("lib.if_proxy");
+ if (ifpx_log_type >= 0)
+ rte_log_set_level(ifpx_log_type, RTE_LOG_WARNING);
+
+ if (ifpx_platform.init)
+ ifpx_platform.init();
+}
diff --git a/lib/librte_if_proxy/if_proxy_priv.h b/lib/librte_if_proxy/if_proxy_priv.h
new file mode 100644
index 000000000..7691494be
--- /dev/null
+++ b/lib/librte_if_proxy/if_proxy_priv.h
@@ -0,0 +1,97 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(C) 2020 Marvell International Ltd.
+ */
+#ifndef _IF_PROXY_PRIV_H_
+#define _IF_PROXY_PRIV_H_
+
+#include "rte_if_proxy.h"
+#include <rte_spinlock.h>
+
+#define RTE_IFPX_LAST_EVENT RTE_IFPX_CFG_DONE
+#define RTE_IFPX_NUM_EVENTS (RTE_IFPX_LAST_EVENT+1)
+#define RTE_IFPX_EVENT_INVALID RTE_IFPX_NUM_EVENTS
+
+extern int ifpx_log_type;
+#define IFPX_LOG(level, fmt, args...) \
+ rte_log(RTE_LOG_ ## level, ifpx_log_type, "%s(): " fmt "\n", \
+ __func__, ##args)
+
+/* Since this library is really a slow/config path we guard all internal data
+ * with a lock - and only one for all of them should be enough.
+ */
+extern rte_spinlock_t ifpx_lock;
+
+enum ifpx_node_status {
+ IN_USE = 1U << 0,
+ DEL_PENDING = 1U << 1,
+};
+
+/* List of configured proxies */
+struct ifpx_proxy_node {
+ TAILQ_ENTRY(ifpx_proxy_node) elem;
+ uint16_t proxy_id;
+ uint16_t state;
+ struct rte_ifpx_info info;
+};
+extern
+TAILQ_HEAD(ifpx_proxies_head, ifpx_proxy_node) ifpx_proxies;
+
+/* This function should be called by the implementation whenever it notices
+ * change in the network configuration. The arguments are:
+ * - ev : pointer to filled event data structure (all fields are expected to be
+ * filled, with the exception of 'port_id' for all proxy/port related
+ * events: this function clones the event notification for each bound port
+ * and fills 'port_id' appropriately).
+ * - px : proxy node when given event is proxy/port related, otherwise pass NULL
+ */
+void ifpx_notify_event(struct rte_ifpx_event *ev, struct ifpx_proxy_node *px);
+
+/* This function should be called by the implementation whenever it is done with
+ * notification about network configuration change. It is only really needed
+ * for the case of callback based API since from the callback user might attempt
+ * to remove proxies. Only implementation really knows when notification for
+ * given proxy is finished so it is a duty of it to call this function to
+ * cleanup all proxies that has been marked for deletion.
+ */
+void ifpx_cleanup_proxies(void);
+
+/* This is the internal function removing the proxy from the list. It is
+ * related to the notification function above and intended to be used by the
+ * platform implementation for the case of callback based API.
+ * During notification via callback the internal lock is released so that
+ * operation would not deadlock on an attempt to take a lock. However
+ * modification (destruction) is not really performed - instead the
+ * callbacks/proxies are marked as "to be deleted".
+ * Handling of callbacks that are "to be deleted" is done by the
+ * ifpx_notify_event() function itself however it cannot delete the proxies (in
+ * particular the proxy passed as an argument) since they might still be
+ * referred by the calling function. So it is a responsibility of the platform
+ * implementation to check after calling notification function if there are any
+ * proxies to be removed and use ifpx_proxy_destroy() to actually release them.
+ */
+int ifpx_proxy_destroy(struct ifpx_proxy_node *px);
+
+/* Every implementation should provide definition of this structure:
+ * - init : called during library initialization (NULL when not needed)
+ * - events : this should return bitmask of supported events (can be NULL if all
+ * defined events are supported by the implementation)
+ * - listen : this function should start service listening to the network
+ * configuration events/changes,
+ * - close : this function should close the service started by listen()
+ * - get_info : this function should query system for current configuration of
+ * interface with index 'if_index'. After successful initialization of
+ * listening service this function is called with 0 as an argument. In that
+ * case configuration of all ports should be obtained - and when this
+ * procedure completes a RTE_IFPX_CFG_DONE event should be signaled via
+ * ifpx_notify_event().
+ */
+extern
+struct ifpx_platform_callbacks {
+ void (*init)(void);
+ uint64_t (*events)(void);
+ int (*listen)(void);
+ int (*close)(void);
+ void (*get_info)(int if_index);
+} ifpx_platform;
+
+#endif /* _IF_PROXY_PRIV_H_ */
diff --git a/lib/librte_if_proxy/linux/Makefile b/lib/librte_if_proxy/linux/Makefile
new file mode 100644
index 000000000..275b7e1e3
--- /dev/null
+++ b/lib/librte_if_proxy/linux/Makefile
@@ -0,0 +1,4 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(C) 2020 Marvell International Ltd.
+
+SRCS += if_proxy.c
diff --git a/lib/librte_if_proxy/linux/if_proxy.c b/lib/librte_if_proxy/linux/if_proxy.c
new file mode 100644
index 000000000..618631b01
--- /dev/null
+++ b/lib/librte_if_proxy/linux/if_proxy.c
@@ -0,0 +1,563 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(C) 2020 Marvell International Ltd.
+ */
+#include <if_proxy_priv.h>
+#include <rte_interrupts.h>
+#include <rte_string_fns.h>
+
+#include <stdbool.h>
+#include <unistd.h>
+#include <errno.h>
+#include <sys/socket.h>
+#include <linux/rtnetlink.h>
+#include <linux/if.h>
+
+static
+struct rte_intr_handle ifpx_irq = {
+ .type = RTE_INTR_HANDLE_NETLINK,
+ .fd = -1,
+};
+
+static
+unsigned int ifpx_pid;
+
+static
+int request_info(int type, int index)
+{
+ static rte_spinlock_t send_lock = RTE_SPINLOCK_INITIALIZER;
+ struct info_get {
+ struct nlmsghdr h;
+ union {
+ struct ifinfomsg ifm;
+ struct ifaddrmsg ifa;
+ struct rtmsg rtm;
+ struct ndmsg ndm;
+ } __rte_aligned(NLMSG_ALIGNTO);
+ } info_req;
+ int ret;
+
+ memset(&info_req, 0, sizeof(info_req));
+ /* First byte of these messages is family, so just make sure that this
+ * memset is enough to get all families.
+ */
+ RTE_ASSERT(AF_UNSPEC == 0);
+
+ info_req.h.nlmsg_pid = ifpx_pid;
+ info_req.h.nlmsg_type = type;
+ info_req.h.nlmsg_flags = NLM_F_REQUEST | NLM_F_DUMP;
+ info_req.h.nlmsg_len = offsetof(struct info_get, ifm);
+
+ switch (type) {
+ case RTM_GETLINK:
+ info_req.h.nlmsg_len += sizeof(info_req.ifm);
+ info_req.ifm.ifi_index = index;
+ break;
+ case RTM_GETADDR:
+ info_req.h.nlmsg_len += sizeof(info_req.ifa);
+ info_req.ifa.ifa_index = index;
+ break;
+ case RTM_GETROUTE:
+ info_req.h.nlmsg_len += sizeof(info_req.rtm);
+ break;
+ case RTM_GETNEIGH:
+ info_req.h.nlmsg_len += sizeof(info_req.ndm);
+ break;
+ default:
+ IFPX_LOG(WARNING, "Unhandled message type: %d", type);
+ return -EINVAL;
+ }
+ /* Store request type (and if it is global or link specific) in 'seq'.
+ * Later it is used during handling of reply to continue requesting of
+ * information dump from system - if needed.
+ */
+ info_req.h.nlmsg_seq = index << 8 | type;
+
+ IFPX_LOG(DEBUG, "\tRequesting msg %d for: %u", type, index);
+
+ rte_spinlock_lock(&send_lock);
+retry:
+ ret = send(ifpx_irq.fd, &info_req, info_req.h.nlmsg_len, 0);
+ if (ret < 0) {
+ if (errno == EINTR) {
+ IFPX_LOG(DEBUG, "send() interrupted");
+ goto retry;
+ }
+ IFPX_LOG(ERR, "Failed to send netlink msg: %d", errno);
+ rte_errno = errno;
+ }
+ rte_spinlock_unlock(&send_lock);
+
+ return ret;
+}
+
+static
+void handle_link(const struct nlmsghdr *h)
+{
+ const struct ifinfomsg *ifi = NLMSG_DATA(h);
+ int alen = h->nlmsg_len - NLMSG_LENGTH(sizeof(*ifi));
+ const struct rtattr *attrs[IFLA_MAX+1] = { NULL };
+ const struct rtattr *attr;
+ struct ifpx_proxy_node *px;
+ struct rte_ifpx_event ev;
+
+ IFPX_LOG(DEBUG, "\tLink action (%u): %u, 0x%x/0x%x (flags/changed)",
+ ifi->ifi_index, h->nlmsg_type, ifi->ifi_flags,
+ ifi->ifi_change);
+
+ rte_spinlock_lock(&ifpx_lock);
+ TAILQ_FOREACH(px, &ifpx_proxies, elem) {
+ if (px->info.if_index == (unsigned int)ifi->ifi_index)
+ break;
+ }
+
+ /* Drop messages that are not associated with any proxy */
+ if (!px)
+ goto exit;
+ /* When message is a reply to request for specific interface then keep
+ * it only when it contains info for this interface.
+ */
+ if (h->nlmsg_pid == ifpx_pid && h->nlmsg_seq >> 8 &&
+ (h->nlmsg_seq >> 8) != (unsigned int)ifi->ifi_index)
+ goto exit;
+
+ for (attr = IFLA_RTA(ifi); RTA_OK(attr, alen);
+ attr = RTA_NEXT(attr, alen)) {
+ if (attr->rta_type > IFLA_MAX)
+ continue;
+ attrs[attr->rta_type] = attr;
+ }
+
+ if (ifi->ifi_change & IFF_UP) {
+ ev.type = RTE_IFPX_LINK_CHANGE;
+ ev.link_change.is_up = ifi->ifi_flags & IFF_UP;
+ ifpx_notify_event(&ev, px);
+ }
+ if (attrs[IFLA_MTU]) {
+ uint16_t mtu = *(const int *)RTA_DATA(attrs[IFLA_MTU]);
+ if (mtu != px->info.mtu) {
+ px->info.mtu = mtu;
+ ev.type = RTE_IFPX_MTU_CHANGE;
+ ev.mtu_change.mtu = mtu;
+ ifpx_notify_event(&ev, px);
+ }
+ }
+ if (attrs[IFLA_ADDRESS]) {
+ const struct rte_ether_addr *mac =
+ RTA_DATA(attrs[IFLA_ADDRESS]);
+
+ RTE_ASSERT(RTA_PAYLOAD(attrs[IFLA_ADDRESS]) ==
+ RTE_ETHER_ADDR_LEN);
+ if (memcmp(mac, &px->info.mac, RTE_ETHER_ADDR_LEN) != 0) {
+ rte_ether_addr_copy(mac, &px->info.mac);
+ ev.type = RTE_IFPX_MAC_CHANGE;
+ rte_ether_addr_copy(mac, &ev.mac_change.mac);
+ ifpx_notify_event(&ev, px);
+ }
+ }
+ if (h->nlmsg_pid == ifpx_pid) {
+ RTE_ASSERT((h->nlmsg_seq & 0xFF) == RTM_GETLINK);
+ /* If this is reply for specific link request (not initial
+ * global dump) then follow up with address request, otherwise
+ * just store the interface name.
+ */
+ if (h->nlmsg_seq >> 8)
+ request_info(RTM_GETADDR, ifi->ifi_index);
+ else if (!px->info.if_name[0] && attrs[IFLA_IFNAME])
+ strlcpy(px->info.if_name, RTA_DATA(attrs[IFLA_IFNAME]),
+ sizeof(px->info.if_name));
+ }
+
+ ifpx_cleanup_proxies();
+exit:
+ rte_spinlock_unlock(&ifpx_lock);
+}
+
+static
+void handle_addr(const struct nlmsghdr *h, bool needs_del)
+{
+ const struct ifaddrmsg *ifa = NLMSG_DATA(h);
+ int alen = h->nlmsg_len - NLMSG_LENGTH(sizeof(*ifa));
+ const struct rtattr *attrs[IFA_MAX+1] = { NULL };
+ const struct rtattr *attr;
+ struct ifpx_proxy_node *px;
+ struct rte_ifpx_event ev;
+ const uint8_t *ip;
+
+ IFPX_LOG(DEBUG, "\tAddr action (%u): %u, family: %u",
+ ifa->ifa_index, h->nlmsg_type, ifa->ifa_family);
+
+ rte_spinlock_lock(&ifpx_lock);
+ TAILQ_FOREACH(px, &ifpx_proxies, elem) {
+ if (px->info.if_index == ifa->ifa_index)
+ break;
+ }
+
+ /* Drop messages that are not associated with any proxy */
+ if (!px)
+ goto exit;
+ /* When message is a reply to request for specific interface then keep
+ * it only when it contains info for this interface.
+ */
+ if (h->nlmsg_pid == ifpx_pid && h->nlmsg_seq >> 8 &&
+ (h->nlmsg_seq >> 8) != ifa->ifa_index)
+ goto exit;
+
+ for (attr = IFA_RTA(ifa); RTA_OK(attr, alen);
+ attr = RTA_NEXT(attr, alen)) {
+ if (attr->rta_type > IFA_MAX)
+ continue;
+ attrs[attr->rta_type] = attr;
+ }
+
+ if (attrs[IFA_ADDRESS]) {
+ ip = RTA_DATA(attrs[IFA_ADDRESS]);
+ if (ifa->ifa_family == AF_INET) {
+ ev.type = needs_del ? RTE_IFPX_ADDR_DEL
+ : RTE_IFPX_ADDR_ADD;
+ ev.addr_change.ip =
+ RTE_IPV4(ip[0], ip[1], ip[2], ip[3]);
+ } else {
+ ev.type = needs_del ? RTE_IFPX_ADDR6_DEL
+ : RTE_IFPX_ADDR6_ADD;
+ memcpy(ev.addr6_change.ip, ip, 16);
+ }
+ ifpx_notify_event(&ev, px);
+ ifpx_cleanup_proxies();
+ }
+exit:
+ rte_spinlock_unlock(&ifpx_lock);
+}
+
+static
+void handle_route(const struct nlmsghdr *h, bool needs_del)
+{
+ const struct rtmsg *r = NLMSG_DATA(h);
+ int alen = h->nlmsg_len - NLMSG_LENGTH(sizeof(*r));
+ const struct rtattr *attrs[RTA_MAX+1] = { NULL };
+ const struct rtattr *attr;
+ struct rte_ifpx_event ev;
+ struct ifpx_proxy_node *px = NULL;
+ const uint8_t *ip;
+
+ IFPX_LOG(DEBUG, "\tRoute action: %u, family: %u",
+ h->nlmsg_type, r->rtm_family);
+
+ for (attr = RTM_RTA(r); RTA_OK(attr, alen);
+ attr = RTA_NEXT(attr, alen)) {
+ if (attr->rta_type > RTA_MAX)
+ continue;
+ attrs[attr->rta_type] = attr;
+ }
+
+ memset(&ev, 0, sizeof(ev));
+ ev.type = RTE_IFPX_EVENT_INVALID;
+
+ rte_spinlock_lock(&ifpx_lock);
+ if (attrs[RTA_OIF]) {
+ int if_index = *((int32_t *)RTA_DATA(attrs[RTA_OIF]));
+
+ if (if_index > 0) {
+ TAILQ_FOREACH(px, &ifpx_proxies, elem) {
+ if (px->info.if_index == (uint32_t)if_index)
+ break;
+ }
+ }
+ }
+ /* We are only interested in routes related to the proxy interfaces and
+ * we need to have dst - otherwise skip the message.
+ */
+ if (!px || !attrs[RTA_DST])
+ goto exit;
+
+ ip = RTA_DATA(attrs[RTA_DST]);
+ /* This is common to both IPv4/6. */
+ ev.route_change.depth = r->rtm_dst_len;
+ if (r->rtm_family == AF_INET) {
+ ev.type = needs_del ? RTE_IFPX_ROUTE_DEL
+ : RTE_IFPX_ROUTE_ADD;
+ ev.route_change.ip = RTE_IPV4(ip[0], ip[1], ip[2], ip[3]);
+ } else {
+ ev.type = needs_del ? RTE_IFPX_ROUTE6_DEL
+ : RTE_IFPX_ROUTE6_ADD;
+ memcpy(ev.route6_change.ip, ip, 16);
+ }
+ if (attrs[RTA_GATEWAY]) {
+ ip = RTA_DATA(attrs[RTA_GATEWAY]);
+ if (r->rtm_family == AF_INET)
+ ev.route_change.gateway =
+ RTE_IPV4(ip[0], ip[1], ip[2], ip[3]);
+ else
+ memcpy(ev.route6_change.gateway, ip, 16);
+ }
+
+ ifpx_notify_event(&ev, px);
+ /* Let's check for proxies to remove here too - just in case somebody
+ * removed the non-proxy related callback.
+ */
+ ifpx_cleanup_proxies();
+exit:
+ rte_spinlock_unlock(&ifpx_lock);
+}
+
+/* Link, addr and route related messages seem to have this macro defined but not
+ * neighbour one. Define one if it is missing - const qualifiers added just to
+ * silence compiler - for some reason it is not needed in equivalent macros for
+ * other messages and here compiler is complaining about (char*) cast on pointer
+ * to const.
+ */
+#ifndef NDA_RTA
+#define NDA_RTA(r) ((const struct rtattr *)(((const char *)(r)) + \
+ NLMSG_ALIGN(sizeof(struct ndmsg))))
+#endif
+
+static
+void handle_neigh(const struct nlmsghdr *h, bool needs_del)
+{
+ const struct ndmsg *n = NLMSG_DATA(h);
+ int alen = h->nlmsg_len - NLMSG_LENGTH(sizeof(*n));
+ const struct rtattr *attrs[NDA_MAX+1] = { NULL };
+ const struct rtattr *attr;
+ struct ifpx_proxy_node *px;
+ struct rte_ifpx_event ev;
+ const uint8_t *ip;
+
+ IFPX_LOG(DEBUG, "\tNeighbour action: %u, family: %u, state: %u, if: %d",
+ h->nlmsg_type, n->ndm_family, n->ndm_state, n->ndm_ifindex);
+
+ for (attr = NDA_RTA(n); RTA_OK(attr, alen);
+ attr = RTA_NEXT(attr, alen)) {
+ if (attr->rta_type > NDA_MAX)
+ continue;
+ attrs[attr->rta_type] = attr;
+ }
+
+ memset(&ev, 0, sizeof(ev));
+ ev.type = RTE_IFPX_EVENT_INVALID;
+
+ rte_spinlock_lock(&ifpx_lock);
+ TAILQ_FOREACH(px, &ifpx_proxies, elem) {
+ if (px->info.if_index == (unsigned int)n->ndm_ifindex)
+ break;
+ }
+ /* We need only subset of neighbourhood related to proxy interfaces.
+ * lladdr seems to be needed only for adding new entry - modifications
+ * (also reported via RTM_NEWLINK) and deletion include only dst.
+ */
+ if (!px || !attrs[NDA_DST] || (!needs_del && !attrs[NDA_LLADDR]))
+ goto exit;
+
+ ip = RTA_DATA(attrs[NDA_DST]);
+ if (n->ndm_family == AF_INET) {
+ ev.type = needs_del ? RTE_IFPX_NEIGH_DEL
+ : RTE_IFPX_NEIGH_ADD;
+ ev.neigh_change.ip = RTE_IPV4(ip[0], ip[1], ip[2], ip[3]);
+ } else {
+ ev.type = needs_del ? RTE_IFPX_NEIGH6_DEL
+ : RTE_IFPX_NEIGH6_ADD;
+ memcpy(ev.neigh6_change.ip, ip, 16);
+ }
+ if (attrs[NDA_LLADDR])
+ rte_ether_addr_copy(RTA_DATA(attrs[NDA_LLADDR]),
+ &ev.neigh_change.mac);
+
+ ifpx_notify_event(&ev, px);
+ /* Let's check for proxies to remove here too - just in case somebody
+ * removed the non-proxy related callback.
+ */
+ ifpx_cleanup_proxies();
+exit:
+ rte_spinlock_unlock(&ifpx_lock);
+}
+
+static
+void if_proxy_intr_callback(void *arg __rte_unused)
+{
+ struct nlmsghdr *h;
+ struct sockaddr_nl addr;
+ socklen_t addr_len;
+ char buf[8192];
+ ssize_t len;
+
+restart:
+ len = recvfrom(ifpx_irq.fd, buf, sizeof(buf), 0,
+ (struct sockaddr *)&addr, &addr_len);
+ if (len < 0) {
+ if (errno == EINTR) {
+ IFPX_LOG(DEBUG, "recvfrom() interrupted");
+ goto restart;
+ }
+ IFPX_LOG(ERR, "Failed to read netlink msg: %ld (errno %d)",
+ len, errno);
+ return;
+ }
+ if (addr_len != sizeof(addr)) {
+ IFPX_LOG(ERR, "Invalid netlink addr size: %d", addr_len);
+ return;
+ }
+ IFPX_LOG(DEBUG, "Read %lu bytes (buf %lu) from %u/%u", len,
+ sizeof(buf), addr.nl_pid, addr.nl_groups);
+
+ for (h = (struct nlmsghdr *)buf; NLMSG_OK(h, len);
+ h = NLMSG_NEXT(h, len)) {
+ IFPX_LOG(DEBUG, "Recv msg: %u (%u/%u/%u seq/flags/pid)",
+ h->nlmsg_type, h->nlmsg_seq, h->nlmsg_flags,
+ h->nlmsg_pid);
+
+ switch (h->nlmsg_type) {
+ case RTM_NEWLINK:
+ case RTM_DELLINK:
+ handle_link(h);
+ break;
+ case RTM_NEWADDR:
+ case RTM_DELADDR:
+ handle_addr(h, h->nlmsg_type == RTM_DELADDR);
+ break;
+ case RTM_NEWROUTE:
+ case RTM_DELROUTE:
+ handle_route(h, h->nlmsg_type == RTM_DELROUTE);
+ break;
+ case RTM_NEWNEIGH:
+ case RTM_DELNEIGH:
+ handle_neigh(h, h->nlmsg_type == RTM_DELNEIGH);
+ break;
+ }
+
+ /* If this is a reply for global request then follow up with
+ * additional requests and notify about finish.
+ */
+ if (h->nlmsg_pid == ifpx_pid && (h->nlmsg_seq >> 8) == 0 &&
+ h->nlmsg_type == NLMSG_DONE) {
+ if ((h->nlmsg_seq & 0xFF) == RTM_GETLINK)
+ request_info(RTM_GETADDR, 0);
+ else if ((h->nlmsg_seq & 0xFF) == RTM_GETADDR)
+ request_info(RTM_GETROUTE, 0);
+ else if ((h->nlmsg_seq & 0xFF) == RTM_GETROUTE)
+ request_info(RTM_GETNEIGH, 0);
+ else {
+ struct rte_ifpx_event ev = {
+ .type = RTE_IFPX_CFG_DONE
+ };
+
+ RTE_ASSERT((h->nlmsg_seq & 0xFF) ==
+ RTM_GETNEIGH);
+ rte_spinlock_lock(&ifpx_lock);
+ ifpx_notify_event(&ev, NULL);
+ rte_spinlock_unlock(&ifpx_lock);
+ }
+ }
+ }
+ IFPX_LOG(DEBUG, "Finished msg loop: %ld bytes left", len);
+}
+
+static
+int nlink_listen(void)
+{
+ struct sockaddr_nl addr = {
+ .nl_family = AF_NETLINK,
+ .nl_pid = 0,
+ };
+ socklen_t addr_len = sizeof(addr);
+ int ret;
+
+ if (ifpx_irq.fd != -1) {
+ rte_errno = EBUSY;
+ return -1;
+ }
+
+ addr.nl_groups = 1 << (RTNLGRP_LINK-1)
+ | 1 << (RTNLGRP_NEIGH-1)
+ | 1 << (RTNLGRP_IPV4_IFADDR-1)
+ | 1 << (RTNLGRP_IPV6_IFADDR-1)
+ | 1 << (RTNLGRP_IPV4_ROUTE-1)
+ | 1 << (RTNLGRP_IPV6_ROUTE-1);
+
+ ifpx_irq.fd = socket(AF_NETLINK, SOCK_RAW | SOCK_CLOEXEC,
+ NETLINK_ROUTE);
+ if (ifpx_irq.fd == -1) {
+ IFPX_LOG(ERR, "Failed to create netlink socket: %d", errno);
+ goto error;
+ }
+ /* Starting with kernel 4.19 you can request dump for a specific
+ * interface and kernel will filter out and send only relevant info.
+ * Otherwise NLM_F_DUMP will generate info for all interfaces and you
+ * need to filter them yourself.
+ */
+#ifdef NETLINK_DUMP_STRICT_CHK
+ ret = 1; /* use this var also as an input param */
+ ret = setsockopt(ifpx_irq.fd, SOL_SOCKET, NETLINK_DUMP_STRICT_CHK,
+ &ret, sizeof(ret));
+ if (ret < 0) {
+ IFPX_LOG(ERR, "Failed to set socket option: %d", errno);
+ goto error;
+ }
+#endif
+
+ ret = bind(ifpx_irq.fd, (struct sockaddr *)&addr, addr_len);
+ if (ret < 0) {
+ IFPX_LOG(ERR, "Failed to bind socket: %d", errno);
+ goto error;
+ }
+ ret = getsockname(ifpx_irq.fd, (struct sockaddr *)&addr, &addr_len);
+ if (ret < 0) {
+ IFPX_LOG(ERR, "Failed to get socket addr: %d", errno);
+ goto error;
+ } else {
+ ifpx_pid = addr.nl_pid;
+ IFPX_LOG(DEBUG, "Assigned port ID: %u", addr.nl_pid);
+ }
+
+ ret = rte_intr_callback_register(&ifpx_irq, if_proxy_intr_callback,
+ NULL);
+ if (ret == 0)
+ return 0;
+
+error:
+ rte_errno = errno;
+ if (ifpx_irq.fd != -1) {
+ close(ifpx_irq.fd);
+ ifpx_irq.fd = -1;
+ }
+ return -1;
+}
+
+static
+int nlink_close(void)
+{
+ int ec;
+
+ if (ifpx_irq.fd < 0)
+ return -EBADFD;
+
+ /* Drop the lock for the time of unregistering - otherwise we might dead
+ * lock e.g. we take a lock here and try to unregister and wait for the
+ * interrupt lock but it is taken already because notification comes
+ * and executes proxy callback which will try to take a lock.
+ */
+ rte_spinlock_unlock(&ifpx_lock);
+ do
+ ec = rte_intr_callback_unregister(&ifpx_irq,
+ if_proxy_intr_callback, NULL);
+ while (ec == -EAGAIN); /* unlikely but possible - at least I think so */
+ rte_spinlock_lock(&ifpx_lock);
+
+ close(ifpx_irq.fd);
+ ifpx_irq.fd = -1;
+ ifpx_pid = 0;
+
+ return 0;
+}
+
+static
+void nlink_get_info(int if_index)
+{
+ if (ifpx_irq.fd != -1)
+ request_info(RTM_GETLINK, if_index);
+}
+
+struct ifpx_platform_callbacks ifpx_platform = {
+ .init = NULL,
+ .events = NULL,
+ .listen = nlink_listen,
+ .close = nlink_close,
+ .get_info = nlink_get_info,
+};
diff --git a/lib/librte_if_proxy/meson.build b/lib/librte_if_proxy/meson.build
new file mode 100644
index 000000000..f0c1a6e15
--- /dev/null
+++ b/lib/librte_if_proxy/meson.build
@@ -0,0 +1,19 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(C) 2020 Marvell International Ltd.
+
+# Currently only implemented on Linux
+if not is_linux
+ build = false
+ reason = 'only supported on linux'
+endif
+
+version = 1
+allow_experimental_apis = true
+
+deps += ['ethdev']
+sources = files('if_proxy_common.c')
+headers = files('rte_if_proxy.h')
+
+if is_linux
+ sources += files('linux/if_proxy.c')
+endif
diff --git a/lib/librte_if_proxy/rte_if_proxy.h b/lib/librte_if_proxy/rte_if_proxy.h
new file mode 100644
index 000000000..2378b4424
--- /dev/null
+++ b/lib/librte_if_proxy/rte_if_proxy.h
@@ -0,0 +1,585 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(C) 2020 Marvell International Ltd.
+ */
+
+#ifndef _RTE_IF_PROXY_H_
+#define _RTE_IF_PROXY_H_
+
+/**
+ * @file
+ * RTE IF Proxy library
+ *
+ * The IF Proxy library allows for monitoring of system network configuration
+ * and configuration of DPDK ports by using usual system utilities (like the
+ * ones from iproute2 package).
+ *
+ * It is based on the notion of "proxy interface" which actually can be any DPDK
+ * port which is also visible to the system - that is it has non-zero 'if_index'
+ * field in 'rte_eth_dev_info' structure.
+ *
+ * If application doesn't have any such port (or doesn't want to use it for
+ * proxy) it can create one by calling:
+ *
+ * proxy_id = rte_ifpx_create(RTE_IFPX_DEFAULT);
+ *
+ * This function is just a wrapper that constructs valid 'devargs' string based
+ * on the proxy type chosen (currently Tap or KNI) and creates the interface by
+ * calling rte_ifpx_dev_create().
+ *
+ * Once one has DPDK port capable of being proxy one can bind target DPDK port
+ * to it by calling.
+ *
+ * rte_ifpx_port_bind(port_id, proxy_id);
+ *
+ * This binding is a logical one - there is no automatic packet forwarding
+ * between port and it's proxy since the library doesn't know the structure of
+ * application's packet processing. It remains application responsibility to
+ * forward the packets from/to proxy port (by calling the usual DPDK RX/TX burst
+ * API). However when the library notes some change to the proxy interface it
+ * will simply call appropriate callback with 'port_id' of the DPDK port that is
+ * bound to this proxy interface. The binding can be 1 to many - that is many
+ * ports can point to one proxy - in that case registered callbacks will be
+ * called for every bound port.
+ *
+ * The callbacks that are used for notifications are described by the
+ * 'rte_ifpx_callback' structure and they are registered by calling:
+ *
+ * rte_ifpx_callbacks_register(len, cbs);
+ *
+ * where cbs is an array of callback pointers.
+ * @see rte_ifpx_callbacks_register()
+ *
+ * Finally the application should call:
+ *
+ * rte_ifpx_listen();
+ *
+ * which will query system for present network configuration and start listening
+ * to its changes.
+ */
+
+#include <rte_eal.h>
+#include <rte_ethdev.h>
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+/**
+ * Enum naming the type of proxy to create.
+ *
+ * @see rte_ifpx_create()
+ */
+enum rte_ifpx_proxy_type {
+ RTE_IFPX_DEFAULT, /**< Use default proxy type for given arch. */
+ RTE_IFPX_TAP, /**< Use Tap based port for proxy. */
+ RTE_IFPX_KNI /**< Use KNI based port for proxy. */
+};
+
+/**
+ * Create DPDK port that can serve as an interface proxy.
+ *
+ * This function is just a wrapper around rte_ifpx_create_by_devarg() that
+ * constructs its 'devarg' argument based on type of proxy requested.
+ *
+ * @param type
+ * A type of proxy to create.
+ *
+ * @return
+ * DPDK port id on success, RTE_MAX_ETHPORTS otherwise.
+ *
+ * @see enum rte_ifpx_type
+ * @see rte_ifpx_create_by_devarg()
+ */
+__rte_experimental
+uint16_t rte_ifpx_proxy_create(enum rte_ifpx_proxy_type type);
+
+/**
+ * Create DPDK port that can serve as an interface proxy.
+ *
+ * @param devarg
+ * A string passed to rte_dev_probe() to create proxy port.
+ *
+ * @return
+ * DPDK port id on success, RTE_MAX_ETHPORTS otherwise.
+ */
+__rte_experimental
+uint16_t rte_ifpx_proxy_create_by_devarg(const char *devarg);
+
+/**
+ * Remove DPDK proxy port.
+ *
+ * In addition to removing the proxy port the bindings (if any) are cleared.
+ *
+ * @param proxy_id
+ * Port id of the proxy that should be removed.
+ *
+ * @return
+ * 0 on success, negative on error.
+ */
+__rte_experimental
+int rte_ifpx_proxy_destroy(uint16_t proxy_id);
+
+/**
+ * The rte_ifpx_event_type enum lists all possible event types that can be
+ * signaled by this library. To learn what events are supported on your
+ * platform call rte_ifpx_events_available().
+ *
+ * NOTE - in order to keep ABI stable do not reorder these enums freely.
+ */
+enum rte_ifpx_event_type {
+ RTE_IFPX_MAC_CHANGE, /**< @see struct rte_ifpx_mac_change */
+ RTE_IFPX_MTU_CHANGE, /**< @see struct rte_ifpx_mtu_change */
+ RTE_IFPX_LINK_CHANGE, /**< @see struct rte_ifpx_link_change */
+ RTE_IFPX_ADDR_ADD, /**< @see struct rte_ifpx_addr_change */
+ RTE_IFPX_ADDR_DEL, /**< @see struct rte_ifpx_addr_change */
+ RTE_IFPX_ADDR6_ADD, /**< @see struct rte_ifpx_addr6_change */
+ RTE_IFPX_ADDR6_DEL, /**< @see struct rte_ifpx_addr6_change */
+ RTE_IFPX_ROUTE_ADD, /**< @see struct rte_ifpx_route_change */
+ RTE_IFPX_ROUTE_DEL, /**< @see struct rte_ifpx_route_change */
+ RTE_IFPX_ROUTE6_ADD, /**< @see struct rte_ifpx_route6_change */
+ RTE_IFPX_ROUTE6_DEL, /**< @see struct rte_ifpx_route6_change */
+ RTE_IFPX_NEIGH_ADD, /**< @see struct rte_ifpx_neigh_change */
+ RTE_IFPX_NEIGH_DEL, /**< @see struct rte_ifpx_neigh_change */
+ RTE_IFPX_NEIGH6_ADD, /**< @see struct rte_ifpx_neigh6_change */
+ RTE_IFPX_NEIGH6_DEL, /**< @see struct rte_ifpx_neigh6_change */
+ RTE_IFPX_CFG_DONE, /**< This event is a lib specific event - it is
+ * signaled when initial network configuration
+ * query is finished and has no event data.
+ */
+};
+
+/**
+ * Get the bit mask of implemented events/callbacks for this platform.
+ *
+ * @return
+ * Bit mask of events/callbacks implemented: each event type can be tested by
+ * checking bit (1 << ev) where 'ev' is one of the rte_ifpx_event_type enum
+ * values.
+ * @see enum rte_ifpx_event_type
+ */
+__rte_experimental
+uint64_t rte_ifpx_events_available(void);
+
+/**
+ * The rte_ifpx_event defines structure used to pass notification event to
+ * application. Each event type has its own dedicated inner structure - these
+ * structures are also used when using callbacks notifications.
+ */
+struct rte_ifpx_event {
+ enum rte_ifpx_event_type type;
+ union {
+ /** Structure used to pass notification about MAC change of the
+ * proxy interface.
+ * @see RTE_IFPX_MAC_CHANGE
+ */
+ struct rte_ifpx_mac_change {
+ uint16_t port_id;
+ struct rte_ether_addr mac;
+ } mac_change;
+ /** Structure used to pass notification about MTU change.
+ * @see RTE_IFPX_MTU_CHANGE
+ */
+ struct rte_ifpx_mtu_change {
+ uint16_t port_id;
+ uint16_t mtu;
+ } mtu_change;
+ /** Structure used to pass notification about link going
+ * up/down.
+ * @see RTE_IFPX_LINK_CHANGE
+ */
+ struct rte_ifpx_link_change {
+ uint16_t port_id;
+ int is_up;
+ } link_change;
+ /** Structure used to pass notification about IPv4 address being
+ * added/removed. All IPv4 addresses reported by this library
+ * are in host order.
+ * @see RTE_IFPX_ADDR_ADD
+ * @see RTE_IFPX_ADDR_DEL
+ */
+ struct rte_ifpx_addr_change {
+ uint16_t port_id;
+ uint32_t ip;
+ } addr_change;
+ /** Structure used to pass notification about IPv6 address being
+ * added/removed.
+ * @see RTE_IFPX_ADDR6_ADD
+ * @see RTE_IFPX_ADDR6_DEL
+ */
+ struct rte_ifpx_addr6_change {
+ uint16_t port_id;
+ uint8_t ip[16];
+ } addr6_change;
+ /** Structure used to pass notification about IPv4 route being
+ * added/removed.
+ * @see RTE_IFPX_ROUTE_ADD
+ * @see RTE_IFPX_ROUTE_DEL
+ */
+ struct rte_ifpx_route_change {
+ uint16_t port_id;
+ uint8_t depth;
+ uint32_t ip;
+ uint32_t gateway;
+ } route_change;
+ /** Structure used to pass notification about IPv6 route being
+ * added/removed.
+ * @see RTE_IFPX_ROUTE6_ADD
+ * @see RTE_IFPX_ROUTE6_DEL
+ */
+ struct rte_ifpx_route6_change {
+ uint16_t port_id;
+ uint8_t depth;
+ uint8_t ip[16];
+ uint8_t gateway[16];
+ } route6_change;
+ /** Structure used to pass notification about IPv4 neighbour
+ * info changes.
+ * @see RTE_IFPX_NEIGH_ADD
+ * @see RTE_IFPX_NEIGH_DEL
+ */
+ struct rte_ifpx_neigh_change {
+ uint16_t port_id;
+ struct rte_ether_addr mac;
+ uint32_t ip;
+ } neigh_change;
+ /** Structure used to pass notification about IPv6 neighbour
+ * info changes.
+ * @see RTE_IFPX_NEIGH6_ADD
+ * @see RTE_IFPX_NEIGH6_DEL
+ */
+ struct rte_ifpx_neigh6_change {
+ uint16_t port_id;
+ struct rte_ether_addr mac;
+ uint8_t ip[16];
+ } neigh6_change;
+ /* This structure is used internally - to abstract common parts
+ * of proxy/port related events and to be able to refer to this
+ * union without giving it a name.
+ */
+ struct {
+ uint16_t port_id;
+ } data;
+ };
+};
+
+/**
+ * This library can deliver notification about network configuration changes
+ * either by the use of registered callbacks and/or by queueing change events to
+ * configured notification queues. The logic used is:
+ * 1. If there is callback registered for given event type it is called. In
+ * case of many ports to one proxy binding, this callback is called for every
+ * port bound.
+ * 2. If this callback returns non-zero value (for any of ports in case of
+ * many-1 bindings) the handling of an event is considered as complete.
+ * 3. Otherwise the event is added to each configured event queue. The event is
+ * allocated with malloc() so after dequeueing and handling the application
+ * should deallocate it with free().
+ *
+ * This dual notification mechanism is meant to provide some flexibility to
+ * application writer. For example, if you store your data in a single writer/
+ * many readers coherent data structure you could just update this structure
+ * from the callback. If you keep separate copy per lcore/port you could make
+ * some common preparations (if applicable) in the callback, return 0 and use
+ * notification queues to pick up the change and update data structures. Or you
+ * could skip the callbacks altogether and just use notification queues - and
+ * configure them at the level appropriate for your application design (one
+ * global / one per lcore / one per port ...).
+ */
+
+/**
+ * Add notification queue to the list of queues.
+ *
+ * @param r
+ * Ring used for queueing of notification events - application can assume that
+ * there is only one producer.
+ * @return
+ * 0 on success, negative otherwise.
+ */
+int rte_ifpx_queue_add(struct rte_ring *r);
+
+/**
+ * Remove notification queue from the list of queues.
+ *
+ * @param r
+ * Notification ring used for queueing of notification events (previously
+ * added via rte_ifpx_queue_add()).
+ * @return
+ * 0 on success, negative otherwise.
+ */
+int rte_ifpx_queue_remove(struct rte_ring *r);
+
+/**
+ * This union groups the callback types that might be called as a notification
+ * events for changing network configuration. Not every platform might
+ * implement all of them and you can query the availability with
+ * rte_ifpx_events_available() function.
+ * @see rte_ifpx_events_available()
+ * @see rte_ifpx_callbacks_register()
+ */
+union rte_ifpx_cb_ptr {
+ int (*mac_change)(const struct rte_ifpx_mac_change *event);
+ /**< Callback for notification about MAC change of the proxy interface.
+ * This callback (as all other port related callbacks) is called for
+ * each port (with its port_id as a first argument) bound to the proxy
+ * interface for which change has been observed.
+ * @see struct rte_ifpx_mac_change
+ * @return non-zero if event handling is finished
+ */
+ int (*mtu_change)(const struct rte_ifpx_mtu_change *event);
+ /**< Callback for notification about MTU change.
+ * @see struct rte_ifpx_mtu_change
+ * @return non-zero if event handling is finished
+ */
+ int (*link_change)(const struct rte_ifpx_link_change *event);
+ /**< Callback for notification about link going up/down.
+ * @see struct rte_ifpx_link_change
+ * @return non-zero if event handling is finished
+ */
+ int (*addr_add)(const struct rte_ifpx_addr_change *event);
+ /**< Callback for notification about IPv4 address being added.
+ * @see struct rte_ifpx_addr_change
+ * @return non-zero if event handling is finished
+ */
+ int (*addr_del)(const struct rte_ifpx_addr_change *event);
+ /**< Callback for notification about IPv4 address removal.
+ * @see struct rte_ifpx_addr_change
+ * @return non-zero if event handling is finished
+ */
+ int (*addr6_add)(const struct rte_ifpx_addr6_change *event);
+ /**< Callback for notification about IPv6 address being added.
+ * @see struct rte_ifpx_addr6_change
+ */
+ int (*addr6_del)(const struct rte_ifpx_addr6_change *event);
+ /**< Callback for notification about IPv4 address removal.
+ * @see struct rte_ifpx_addr6_change
+ * @return non-zero if event handling is finished
+ */
+ /* Please note that "route" callbacks might be also called when user
+ * adds address to the interface (that is in addition to address related
+ * callbacks).
+ */
+ int (*route_add)(const struct rte_ifpx_route_change *event);
+ /**< Callback for notification about IPv4 route being added.
+ * @see struct rte_ifpx_route_change
+ * @return non-zero if event handling is finished
+ */
+ int (*route_del)(const struct rte_ifpx_route_change *event);
+ /**< Callback for notification about IPv4 route removal.
+ * @see struct rte_ifpx_route_change
+ * @return non-zero if event handling is finished
+ */
+ int (*route6_add)(const struct rte_ifpx_route6_change *event);
+ /**< Callback for notification about IPv6 route being added.
+ * @see struct rte_ifpx_route6_change
+ * @return non-zero if event handling is finished
+ */
+ int (*route6_del)(const struct rte_ifpx_route6_change *event);
+ /**< Callback for notification about IPv6 route removal.
+ * @see struct rte_ifpx_route6_change
+ * @return non-zero if event handling is finished
+ */
+ int (*neigh_add)(const struct rte_ifpx_neigh_change *event);
+ /**< Callback for notification about IPv4 neighbour being added.
+ * @see struct rte_ifpx_neigh_change
+ * @return non-zero if event handling is finished
+ */
+ int (*neigh_del)(const struct rte_ifpx_neigh_change *event);
+ /**< Callback for notification about IPv4 neighbour removal.
+ * @see struct rte_ifpx_neigh_change
+ * @return non-zero if event handling is finished
+ */
+ int (*neigh6_add)(const struct rte_ifpx_neigh6_change *event);
+ /**< Callback for notification about IPv6 neighbour being added.
+ * @see struct rte_ifpx_neigh_change
+ */
+ int (*neigh6_del)(const struct rte_ifpx_neigh6_change *event);
+ /**< Callback for notification about IPv6 neighbour removal.
+ * @see struct rte_ifpx_neigh_change
+ * @return non-zero if event handling is finished
+ */
+ int (*cfg_done)(void);
+ /**< Lib specific callback - called when initial network configuration
+ * query is finished.
+ * @return non-zero if event handling is finished
+ */
+};
+
+/**
+ * This structure is a "tagged union" used to pass the callback for
+ * registration.
+ *
+ * @see union rte_ifpx_cb_ptr
+ * @see rte_ifpx_events_available()
+ * @see rte_ifpx_callbacks_register()
+ */
+struct rte_ifpx_callback {
+ enum rte_ifpx_event_type type;
+ union rte_ifpx_cb_ptr callback;
+};
+
+/**
+ * Register proxy callbacks.
+ *
+ * This function registers callbacks to be called upon appropriate network
+ * event notification.
+ *
+ * @param cbs
+ * Set of callbacks that will be called. The library does not take any
+ * ownership of the pointer passed - the callbacks are stored internally.
+ *
+ * @return
+ * 0 on success, negative otherwise.
+ */
+__rte_experimental
+int rte_ifpx_callbacks_register(unsigned int len,
+ const struct rte_ifpx_callback cbs[]);
+
+/**
+ * Unregister proxy callbacks.
+ *
+ * This function unregisters all callbacks previously registered with
+ * rte_ifpx_callbacks_register().
+ */
+__rte_experimental
+void rte_ifpx_callbacks_unregister_all(void);
+
+/**
+ * Unregister proxy callback.
+ *
+ * This function unregisters one callback previously registered with
+ * rte_ifpx_callbacks_register().
+ *
+ * @param ev
+ * Type of event for which callback should be removed.
+ *
+ * @return
+ * 0 on success, negative otherwise.
+ */
+__rte_experimental
+int rte_ifpx_callbacks_unregister(enum rte_ifpx_event_type ev);
+
+/**
+ * Bind the port to its proxy.
+ *
+ * After calling this function all network configuration of the proxy (and it's
+ * changes) will be passed to given port by calling registered callbacks with
+ * 'port_id' as an argument.
+ *
+ * Note: since both arguments are of the same type in order to not mix them and
+ * ease remembering the order the first one is kept the same for bind/unbind.
+ *
+ * @param port_id
+ * Id of the port to be bound.
+ * @param proxy_id
+ * Id of the proxy the port needs to be bound to.
+ * @return
+ * 0 on success, negative on error.
+ */
+__rte_experimental
+int rte_ifpx_port_bind(uint16_t port_id, uint16_t proxy_id);
+
+/**
+ * Unbind the port from its proxy.
+ *
+ * After calling this function registered callbacks will no longer be called for
+ * this port (but they might be called for other ports in one to many binding
+ * scenario).
+ *
+ * @param port_id
+ * Id of the port to unbind.
+ * @return
+ * 0 on success, negative on error.
+ */
+__rte_experimental
+int rte_ifpx_port_unbind(uint16_t port_id);
+
+/**
+ * Get the system network configuration and start listening to its changes.
+ *
+ * @return
+ * 0 on success, negative otherwise.
+ */
+__rte_experimental
+int rte_ifpx_listen(void);
+
+/**
+ * Remove all bindings/callbacks and stop listening to network configuration.
+ *
+ * @return
+ * 0 on success, negative otherwise.
+ */
+__rte_experimental
+int rte_ifpx_close(void);
+
+/**
+ * Get the id of the proxy the port is bound to.
+ *
+ * @param port_id
+ * Id of the port for which to get proxy.
+ * @return
+ * Port id of the proxy on success, RTE_MAX_ETHPORTS on error.
+ */
+__rte_experimental
+uint16_t rte_ifpx_proxy_get(uint16_t port_id);
+
+/**
+ * Test for port acting as a proxy.
+ *
+ * @param port_id
+ * Id of the port.
+ * @return
+ * 1 if port acts as a proxy, 0 otherwise.
+ */
+static inline
+int rte_ifpx_is_proxy(uint16_t port_id)
+{
+ return rte_ifpx_proxy_get(port_id) == port_id;
+}
+
+/**
+ * Get the ids of the ports bound to the proxy.
+ *
+ * @param proxy_id
+ * Id of the proxy for which to get ports.
+ * @param ports
+ * Array where to store the port ids.
+ * @param num
+ * Size of the 'ports' array.
+ * @return
+ * The number of ports bound to given proxy. Note that bound ports are filled
+ * in 'ports' array up to its size but the return value is always the total
+ * number of ports bound - so you can make call first with NULL/0 to query for
+ * the size of the buffer to create or call it with the buffer you have and
+ * later check if it was large enough.
+ */
+__rte_experimental
+unsigned int rte_ifpx_port_get(uint16_t proxy_id,
+ uint16_t *ports, unsigned int num);
+
+/**
+ * The structure containing some properties of the proxy interface.
+ */
+struct rte_ifpx_info {
+ unsigned int if_index; /* entry valid iff if_index != 0 */
+ uint16_t mtu;
+ struct rte_ether_addr mac;
+ char if_name[RTE_ETH_NAME_MAX_LEN];
+};
+
+/**
+ * Get the properties of the proxy interface. Argument can be either id of the
+ * proxy or an id of a port that is bound to it.
+ *
+ * @param port_id
+ * Id of the port (or proxy) for which to get proxy properties.
+ * @return
+ * Pointer to the proxy information structure.
+ */
+__rte_experimental
+const struct rte_ifpx_info *rte_ifpx_info_get(uint16_t port_id);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* _RTE_IF_PROXY_H_ */
diff --git a/lib/librte_if_proxy/rte_if_proxy_version.map b/lib/librte_if_proxy/rte_if_proxy_version.map
new file mode 100644
index 000000000..6da35d096
--- /dev/null
+++ b/lib/librte_if_proxy/rte_if_proxy_version.map
@@ -0,0 +1,20 @@
+EXPERIMENTAL {
+ global:
+
+ rte_ifpx_callbacks_register;
+ rte_ifpx_callbacks_unregister;
+ rte_ifpx_callbacks_unregister_all;
+ rte_ifpx_close;
+ rte_ifpx_events_available;
+ rte_ifpx_info_get;
+ rte_ifpx_listen;
+ rte_ifpx_port_bind;
+ rte_ifpx_port_get;
+ rte_ifpx_port_unbind;
+ rte_ifpx_proxy_create;
+ rte_ifpx_proxy_create_by_devarg;
+ rte_ifpx_proxy_destroy;
+ rte_ifpx_proxy_get;
+
+ local: *;
+};
diff --git a/lib/meson.build b/lib/meson.build
index 07a65a625..caa54f7b5 100644
--- a/lib/meson.build
+++ b/lib/meson.build
@@ -21,7 +21,7 @@ libraries = [
'acl', 'bbdev', 'bitratestats', 'cfgfile',
'compressdev', 'cryptodev',
'distributor', 'efd', 'eventdev',
- 'gro', 'gso', 'ip_frag', 'jobstats',
+ 'gro', 'gso', 'if_proxy', 'ip_frag', 'jobstats',
'kni', 'latencystats', 'lpm', 'member',
'power', 'pdump', 'rawdev',
'rib', 'reorder', 'sched', 'security', 'stack', 'vhost',
--
2.17.1
^ permalink raw reply [relevance 1%]
* [dpdk-dev] [PATCH v3 0/4] Introduce IF proxy library
@ 2020-05-04 8:53 2% ` Andrzej Ostruszka
2020-05-04 8:53 1% ` [dpdk-dev] [PATCH v3 1/4] lib: introduce IF Proxy library Andrzej Ostruszka
0 siblings, 1 reply; 200+ results
From: Andrzej Ostruszka @ 2020-05-04 8:53 UTC (permalink / raw)
To: dev
All
Please find in this patch set updated version of IF Proxy library. This
version addresses comments received so far with some additional minor
improvements/changes. This version does not change the notification
scheme yet since discussion about general DPDK messaging/notification
scheme has has not started so please expect yet another version when
this will crystallize. I also have received some performance
improvements for the example application from Harman (thank you) but
decided to hold them. This is because in future the example application
might either be merged into the regular l3fwd or be kept separate
- depending on the outcome of the discussion.
Changes in V3
=============
- Changed callback registration scheme to make the ABI more robust
- Added new platform callback to provide mask with events available
- All library data access is guarded with a lock
- When port is unbound and proxy has no more ports then it is
automatically released
Changes in V2
=============
- Cleaned up checkpatch warnings
- Removed dead/unused code and added gateway clearing in l3fwd-ifpx
What is this useful for
=======================
Usually, when an ethernet port is assigned to DPDK it vanishes from the
system and user looses ability to control it via normal configuration
utilities (e.g. those from iproute2 package). Moreover by default DPDK
application is not aware of the network configuration of the system.
To address both of these issues application needs to:
- add some command line interface (or other mechanism) allowing for
control of the port and its configuration
- query the status of network configuration and monitor its changes
The purpose of this library is to help with both of these tasks (as long
as they remain in domain of configuration available to the system). In
other words, if DPDK application has some special needs, that cannot be
addressed by the normal system configuration utilities, then they need
to be solved by the application itself.
The connection between DPDK and system is based on the existence of
ports that are visible to both DPDK and system (like Tap, KNI and
possibly some other drivers). These ports serve as an interface
proxies.
Let's visualize the action of the library by the following example:
Linux | DPDK
==============================================================
|
| +-------+ +-------+
| | Port1 | | Port2 |
"ip link set dev tap1 mtu 1600" | +-------+ +-------+
| | ^ ^ ^
| +------+ | mtu_change | |
`->| Tap1 |---' callback | |
+------+ | |
"ip addr add 198.51.100.14 \ | | |
dev tap2" | | |
| +------+ | |
+->| Tap2 |------------------' |
| +------+ addr_add callback |
"ip route add 198.0.2.0/24 \ | | |
dev tap2" | | route_add callback |
| `---------------------'
So we have two ports Port1 and Port2 that are not visible to the system.
We create two proxy interfaces (here based on Tap driver) and bind the
ports to their proxies. When user issues a command changing MTU for
Tap1 interface the library notes this and calls "mtu_change" callback
for the Port1. Similarly when user adds an IPv4 address to the Tap2
interface "addr_add" callback is called for the Port2 and the same
happens for configuration of routing rule pointing to Tap2. Apart from
callbacks this library can notify about changes via adding events to
notification queues. See below for more inforamtion about that and
a complete list of available callbacks.
Please note that nothing has been mentioned about forwarding of the
packets between system and DPDK. Since the proxies are normal DPDK
ports you can receive/send to them via usual RX/TX burst API. However
since the library is not aware of the structure of packet processing
used by the application it cannot automatically forward the packets - it
is responsibility of the application to include proxy ports into its
packet processing engine.
As mentioned above the intention of the library is to:
- provide information about network configuration that would allow
application to decide what to do with the packets received on DPDK
ports,
- allow for control of the ports via standard configuration utilities
Although the library only helps you to identify proxy for given port
(and vice versa) and calls appropriate callbacks it does open some
interesting possibilities. For example you can use the proxy ports to
forward packets for protocols that you do not wish to handle in DPDK
application to the system protocol stack and just listen to the
configuration changes - so that way you can "offload" handling of those
protocols to the system.
How to use it
=============
Usage of this library is rather simple. You have to:
1. Create proxy (if you don't have port suitable for being proxy or you
have one but do not wish to use it as a proxy).
2. Bind port to proxy.
3. Register callbacks and/or event queues.
4. Start listening to the network configuration.
The only mandatory requirement for DPDK port to be able to act as
a proxy is that it is visible in the system - this is checked during
port to proxy binding by calling rte_eth_dev_info_get() on proxy port
and inspecting 'if_index' field (it has to be non-zero).
One can create such port in the application by calling:
proxy_id = rte_ifpx_create(RTE_IFPX_DEFAULT);
Upon success this returns id of DPDK proxy port created
(RTE_MAX_ETHPORTS on failure). The argument selects type of proxy port
to create (currently Tap/KNI only). This function actually is just
a wrapper around:
uint16_t rte_ifpx_create_by_devarg(const char *devarg);
creating valid 'devarg' string for the chosen type of proxy. If you have
other driver capable of acting as a proxy you can call
rte_ifpx_create_by_devarg() directly passing appropriate argument.
Once you have id of both port and proxy you can bind the two via:
rte_ifpx_port_bind(port_id, proxy_id);
This creates logical binding - as mentioned above there is no automatic
packet forwarding. With this binding whenever user changes the state of
proxy interface in the system (link up/down, change mac/mtu, add/remove
IPv4/IPv6) you get appropriate notification for the bound port.
So far we've mentioned several times that the library calls callbacks.
They are grouped in 'struct rte_ifpx_callbacks' and user provides them
to the library via:
rte_ifpx_callbacks_register(len, cbs);
It is worth mentioning that the context (lcore/thread) in which these
callbacks are called is implementation defined. It might differ between
different platforms, so the application needs to assume that some kind
of inter lcore/thread synchronization/communication is required.
Apart from notification via callbacks this library also supports
notifying about the changes via adding events to the configured
notification queues. The queues are registered via:
int rte_ifpx_queue_add(struct rte_ring *r);
and the actual logic used is: if there is callback registered then it is
called, if it returns non-zero then event is considered completed,
otherwise event is added to each configured notification queue.
That way application can update data structures that are safe to be
modified by single writer from within callback or do the common
preprocessing steps (if any needed) in callback and data that is
replicated can be updated during handling of queued events.
Once we have bindings in place and notification configured, the only
essential part that remains is to get the current network configuration
and start listening to its changes. This is accomplished via a call to:
rte_ifpx_listen();
And basically this is all one needs to understand how to use this
library. Other less essential parts include:
- ability to query what events are available for given platform
- getting mapping between proxy and port
- unbinding the ports from proxy
- destroying proxy port
- closing the listening service
- getting basic information about proxy
Currently available features and implementation
===============================================
The library's API is system independent but it obviously needs some
system dependent parts. We provide exemplary Linux implementation
(based on netlink sockets). Very similar implementation is possible for
FreeBSD (with the usage of PF_ROUTE sockets). Windows implementation
would need to differ much (probably IP Helper library would be of some
help).
Here is the list of currently implemented callbacks:
int (*mac_change)(const struct rte_ifpx_mac_change *event);
int (*mtu_change)(const struct rte_ifpx_mtu_change *event);
int (*link_change)(const struct rte_ifpx_link_change *event);
int (*addr_add)(const struct rte_ifpx_addr_change *event);
int (*addr_del)(const struct rte_ifpx_addr_change *event);
int (*addr6_add)(const struct rte_ifpx_addr6_change *event);
int (*addr6_del)(const struct rte_ifpx_addr6_change *event);
int (*route_add)(const struct rte_ifpx_route_change *event);
int (*route_del)(const struct rte_ifpx_route_change *event);
int (*route6_add)(const struct rte_ifpx_route6_change *event);
int (*route6_del)(const struct rte_ifpx_route6_change *event);
int (*neigh_add)(const struct rte_ifpx_neigh_change *event);
int (*neigh_del)(const struct rte_ifpx_neigh_change *event);
int (*neigh6_add)(const struct rte_ifpx_neigh6_change *event);
int (*neigh6_del)(const struct rte_ifpx_neigh6_change *event);
int (*cfg_done)(void);
They are all rather self-descriptive with the exception of the last one.
When the user calls rte_ifpx_listen() the library first queries the
system for its current configuration. That might require several
request/reply exchanges between DPDK and system and once it is finished
this callback is called to let application know that all info has been
gathered.
It is worth to mention also that while typical case would be a 1-to-1
mapping between port and proxy, the 1-to-many mapping is also supported.
In that case related callbacks will be called for each port bound to
given proxy interface - it is application responsibility to define
semantic of such mapping (e.g. all changes apply to all ports, or link
changes apply to all but other are accepted in "round robin" fashion, or
some other logic).
As mentioned above Linux implementation is based on netlink socket.
This socket is registered as file descriptor in EAL interrupts
(similarly to how EAL alarms are implemented).
With regards
Andrzej Ostruszka
Andrzej Ostruszka (4):
lib: introduce IF Proxy library
if_proxy: add library documentation
if_proxy: add simple functionality test
if_proxy: add example application
MAINTAINERS | 6 +
app/test/Makefile | 5 +
app/test/meson.build | 4 +
app/test/test_if_proxy.c | 707 +++++++++++
config/common_base | 5 +
config/common_linux | 1 +
doc/guides/prog_guide/if_proxy_lib.rst | 142 +++
doc/guides/prog_guide/index.rst | 1 +
examples/Makefile | 1 +
examples/l3fwd-ifpx/Makefile | 60 +
examples/l3fwd-ifpx/l3fwd.c | 1128 ++++++++++++++++++
examples/l3fwd-ifpx/l3fwd.h | 98 ++
examples/l3fwd-ifpx/main.c | 740 ++++++++++++
examples/l3fwd-ifpx/meson.build | 11 +
examples/meson.build | 2 +-
lib/Makefile | 2 +
lib/librte_eal/include/rte_eal_interrupts.h | 2 +
lib/librte_eal/linux/eal_interrupts.c | 14 +-
lib/librte_if_proxy/Makefile | 29 +
lib/librte_if_proxy/if_proxy_common.c | 564 +++++++++
lib/librte_if_proxy/if_proxy_priv.h | 97 ++
lib/librte_if_proxy/linux/Makefile | 4 +
lib/librte_if_proxy/linux/if_proxy.c | 563 +++++++++
lib/librte_if_proxy/meson.build | 19 +
lib/librte_if_proxy/rte_if_proxy.h | 585 +++++++++
lib/librte_if_proxy/rte_if_proxy_version.map | 20 +
lib/meson.build | 2 +-
27 files changed, 4806 insertions(+), 6 deletions(-)
create mode 100644 app/test/test_if_proxy.c
create mode 100644 doc/guides/prog_guide/if_proxy_lib.rst
create mode 100644 examples/l3fwd-ifpx/Makefile
create mode 100644 examples/l3fwd-ifpx/l3fwd.c
create mode 100644 examples/l3fwd-ifpx/l3fwd.h
create mode 100644 examples/l3fwd-ifpx/main.c
create mode 100644 examples/l3fwd-ifpx/meson.build
create mode 100644 lib/librte_if_proxy/Makefile
create mode 100644 lib/librte_if_proxy/if_proxy_common.c
create mode 100644 lib/librte_if_proxy/if_proxy_priv.h
create mode 100644 lib/librte_if_proxy/linux/Makefile
create mode 100644 lib/librte_if_proxy/linux/if_proxy.c
create mode 100644 lib/librte_if_proxy/meson.build
create mode 100644 lib/librte_if_proxy/rte_if_proxy.h
create mode 100644 lib/librte_if_proxy/rte_if_proxy_version.map
--
2.17.1
^ permalink raw reply [relevance 2%]
* Re: [dpdk-dev] [PATCH v3 0/8] eal: cleanup resources on shutdown
2020-04-28 23:58 3% ` [dpdk-dev] [PATCH v3 0/8] " Stephen Hemminger
@ 2020-05-03 17:21 0% ` David Marchand
0 siblings, 0 replies; 200+ results
From: David Marchand @ 2020-05-03 17:21 UTC (permalink / raw)
To: Stephen Hemminger; +Cc: dev
On Wed, Apr 29, 2020 at 1:58 AM Stephen Hemminger
<stephen@networkplumber.org> wrote:
>
> Started using valgrind with DPDK, and there are lots of leftover
> memory and file descriptors. This makes it hard to find application
> leaks versus DPDK leaks.
>
> The DPDK has a function that applications can use to tell it
> to cleanup resources on shutdown (rte_eal_cleanup). But the
> current coverage of that API is spotty. Many internal parts of
> DPDK leave files and allocated memory behind.
>
> This patch set is a first step at getting the sub-parts of
> DPDK to cleanup after themselves. These are the easier ones,
> the harder and more critical ones are in the drivers
> and the memory subsystem.
>
> There are no new exposed API or ABI changes here.
>
> v3
> - fix a couple of minor checkpatch complaints
>
> v2
> - rebase after 20.05 file renames
> - incorporate review comment feedback
> - hold off some of the more involved patches for later
Same segfault as v1.
$ ./devtools/test-null.sh ./build/app/dpdk-testpmd 0x3 --plop
./build/app/dpdk-testpmd: unrecognized option '--plop'
EAL: Detected 8 lcore(s)
EAL: Detected 1 NUMA nodes
Usage: ./build/app/dpdk-testpmd [options]
(snip)
EAL: FATAL: Invalid 'command line' arguments.
EAL: Invalid 'command line' arguments.
EAL: Error - exiting with code: 1
Cause: Cannot init EAL: Invalid argument
./devtools/test-null.sh: line 32: 23134 Broken pipe (
sleep 1 && echo stop )
23135 Segmentation fault (core dumped) | $testpmd -c
$coremask --no-huge -m 20 $libs -w 0:0.0 --vdev net_null1 --vdev
net_null2 $eal_options -- --no-mlockall --total-num-mbufs=2048
$testpmd_options -ia
--
David Marchand
^ permalink raw reply [relevance 0%]
* [dpdk-dev] [PATCH] common/octeontx: fix GCC 9.1 ABI break
@ 2020-05-02 16:10 4% pbhagavatula
2020-05-04 9:17 4% ` Harman Kalra
0 siblings, 1 reply; 200+ results
From: pbhagavatula @ 2020-05-02 16:10 UTC (permalink / raw)
To: jerinj, ferruh.yigit, Harman Kalra; +Cc: thomas, dev, Pavan Nikhilesh, stable
From: Pavan Nikhilesh <pbhagavatula@marvell.com>
GCC 9.1 fixes a bug with passing bitfields as pass by value in function
parameters and generates a warning for the same as below:
drivers/common/octeontx/octeontx_mbox.c:282:1: note: parameter passing
for argument of type ‘struct mbox_intf_ver’ changed in GCC 9.1
Fix the warning generated by passing bitfield as pass by reference.
Fixes: b4134b2d31cc ("common/octeontx: update mbox to version 1.1.3")
Cc: stable@dpdk.org
Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
---
More info on GCC bug
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88469
https://gcc.gnu.org/git/?p=gcc.git&a=commit;h=c590597c45948c6e6fa282878198fd226da95998
drivers/common/octeontx/octeontx_mbox.c | 17 +++++++++--------
1 file changed, 9 insertions(+), 8 deletions(-)
diff --git a/drivers/common/octeontx/octeontx_mbox.c b/drivers/common/octeontx/octeontx_mbox.c
index 2fd253107..effe0b267 100644
--- a/drivers/common/octeontx/octeontx_mbox.c
+++ b/drivers/common/octeontx/octeontx_mbox.c
@@ -279,7 +279,7 @@ octeontx_start_domain(void)
}
static int
-octeontx_check_mbox_version(struct mbox_intf_ver app_intf_ver,
+octeontx_check_mbox_version(struct mbox_intf_ver *app_intf_ver,
struct mbox_intf_ver *intf_ver)
{
struct mbox_intf_ver kernel_intf_ver = {0};
@@ -290,8 +290,9 @@ octeontx_check_mbox_version(struct mbox_intf_ver app_intf_ver,
hdr.coproc = NO_COPROC;
hdr.msg = RM_INTERFACE_VERSION;
- result = octeontx_mbox_send(&hdr, &app_intf_ver, sizeof(app_intf_ver),
- &kernel_intf_ver, sizeof(kernel_intf_ver));
+ result = octeontx_mbox_send(&hdr, app_intf_ver,
+ sizeof(struct mbox_intf_ver),
+ &kernel_intf_ver, sizeof(kernel_intf_ver));
if (result != sizeof(kernel_intf_ver)) {
mbox_log_err("Could not send interface version. Err=%d. FuncErr=%d\n",
result, hdr.res_code);
@@ -301,9 +302,9 @@ octeontx_check_mbox_version(struct mbox_intf_ver app_intf_ver,
if (intf_ver)
*intf_ver = kernel_intf_ver;
- if (app_intf_ver.platform != kernel_intf_ver.platform ||
- app_intf_ver.major != kernel_intf_ver.major ||
- app_intf_ver.minor != kernel_intf_ver.minor)
+ if (app_intf_ver->platform != kernel_intf_ver.platform ||
+ app_intf_ver->major != kernel_intf_ver.major ||
+ app_intf_ver->minor != kernel_intf_ver.minor)
result = -EINVAL;
return result;
@@ -312,7 +313,7 @@ octeontx_check_mbox_version(struct mbox_intf_ver app_intf_ver,
int
octeontx_mbox_init(void)
{
- const struct mbox_intf_ver MBOX_INTERFACE_VERSION = {
+ struct mbox_intf_ver MBOX_INTERFACE_VERSION = {
.platform = 0x01,
.major = 0x01,
.minor = 0x03
@@ -330,7 +331,7 @@ octeontx_mbox_init(void)
return ret;
}
- ret = octeontx_check_mbox_version(MBOX_INTERFACE_VERSION,
+ ret = octeontx_check_mbox_version(&MBOX_INTERFACE_VERSION,
&rm_intf_ver);
if (ret < 0) {
mbox_log_err("MBOX version: Kernel(%d.%d.%d) != DPDK(%d.%d.%d)",
--
2.26.2
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH v3 3/4] stack: check flags on creation
2020-04-27 23:16 3% ` [dpdk-dev] [PATCH v3 3/4] stack: " Stephen Hemminger
@ 2020-05-02 2:36 0% ` Eads, Gage
0 siblings, 0 replies; 200+ results
From: Eads, Gage @ 2020-05-02 2:36 UTC (permalink / raw)
To: Stephen Hemminger, dev; +Cc: Olivier Matz
> -----Original Message-----
> From: Stephen Hemminger <stephen@networkplumber.org>
> Sent: Monday, April 27, 2020 6:16 PM
> To: dev@dpdk.org
> Cc: Stephen Hemminger <stephen@networkplumber.org>; Eads, Gage
> <gage.eads@intel.com>; Olivier Matz <olivier.matz@6wind.com>
> Subject: [PATCH v3 3/4] stack: check flags on creation
>
> All API's should check that they support the flag values passed. If an application
> passes an invalid flag it could cause problems in later ABI.
>
> Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
> ---
Acked-by: Gage Eads <gage.eads@intel.com>
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH v4 1/4] ethdev: add tm support for shaper config in pkt mode
2020-04-28 15:54 5% ` Thomas Monjalon
2020-04-29 8:45 0% ` Dumitrescu, Cristian
@ 2020-05-01 13:18 3% ` Jerin Jacob
2020-05-05 8:01 0% ` Ray Kinsella
1 sibling, 1 reply; 200+ results
From: Jerin Jacob @ 2020-05-01 13:18 UTC (permalink / raw)
To: Thomas Monjalon
Cc: Dumitrescu, Cristian, Bruce Richardson, Ferruh Yigit,
Luca Boccassi, Nithin Dabilpuram, Singh, Jasvinder,
Andrew Rybchenko, dev, jerinj, kkanas, Nithin Dabilpuram,
Kinsella, Ray, Neil Horman, Kevin Traynor, David Marchand
On Tue, Apr 28, 2020 at 9:24 PM Thomas Monjalon <thomas@monjalon.net> wrote:
>
> 28/04/2020 17:04, Luca Boccassi:
> > On Tue, 2020-04-28 at 15:45 +0100, Bruce Richardson wrote:
> > > On Tue, Apr 28, 2020 at 03:06:20PM +0100, Ferruh Yigit wrote:
> > > > On 4/27/2020 5:59 PM, Jerin Jacob wrote:
> > > > > On Mon, Apr 27, 2020 at 10:19 PM Ferruh Yigit <ferruh.yigit@intel.com> wrote:
> > > > > > On 4/27/2020 5:29 PM, Jerin Jacob wrote:
> > > > > > > On Mon, Apr 27, 2020 at 9:42 PM Ferruh Yigit <ferruh.yigit@intel.com> wrote:
> > > > > > > > On 4/27/2020 10:19 AM, Dumitrescu, Cristian wrote:
> > > > > > > > > From: Yigit, Ferruh <ferruh.yigit@intel.com>
> > > > > > > > > > On 4/24/2020 11:28 AM, Dumitrescu, Cristian wrote:
> > > > > > > > > > > From: Nithin Dabilpuram <nithind1988@gmail.com>
> > > > > > > > > > > > This patch also updates tm port/level/node capability structures with
> > > > > > > > > > > > exiting features of scheduler wfq packet mode, scheduler wfq byte mode
> > > > > > > > > > > > and private/shared shaper byte mode.
> > > > > > > > > > > >
> > > > > > > > > > > > SoftNIC PMD is also updated with new capabilities.
> [...]
> > > > > > > > > > Hi Nithin,
> > > > > > > > > >
> > > > > > > > > > It looks like patch is causing ABI break, I am getting following warning [1],
> > > > > > > > > > can you please check?
> > > > > > > > > >
> > > > > > > > > > [1]
> > > > > > > > > > https://pastebin.com/XYNFg14u
> > > > > > > > >
> > > > > > > > > Hi Ferruh,
> > > > > > > > >
> > > > > > > > > The RTE_TM API is marked as experimental,
> > > > > > > > > but it looks that this was not correctly marked
> > > > > > > > > when __rte_experimental ABI checker was introduced.
> > > > > > > > >
> > > > > > > > > It is marked as experimental at the top of the rte_tm.h,
> > > > > > > > > similarly to other APIs introduced around same time,
> > > > > > > > > but it was not correctly picked up by the ABI check procedure
> > > > > > > > > when later introduced, so __rte_experimental was not added to every function.
> > > > > > > > >
> > > > > > > >
> > > > > > > > :(
> > > > > > > >
> > > > > > > > Is it time to mature them?
> > > > > > > >
> > > > > > > > As you said they are not marked as experimental both in header file (function
> > > > > > > > declarations) and .map file.
> > > > > > > >
> > > > > > > > The problem is, they are not marked as experimental in DPDK_20.0 ABI (v19.11),
> > > > > > > > so marking them as experimental now will break the ABI. Not sure what to do,
> > > > > > > > cc'ed a few ABI related names for comment.
> > > > > > > >
> > > > > > > > For me, we need to proceed as the experimental tag removed and APIs become
> > > > > > > > mature starting from v19.11, since this is what happened in practice, and remove
> > > > > > > > a few existing being experimental references in the doxygen comments.
> > > > > > >
> > > > > > > I think, accidentally we can not make a library as NON-experimental.
> > > > > > > TM never went through experimental to mature transition(see git log
> > > > > > > lib/librte_ethdev/rte_tm.h)
> > > > > > > It was a bug to not mark as experimental in each function in the ABI process.
> > > > > > > Some of the features like packet marking are not even implemented by any HW.
> > > > > > > I think, we can make API stable only all the features are implemented
> > > > > > > by one or two HW.
>
> Yes this is what was decided one or two years ago I think.
> But rte_tm API was introduced 3 years ago and is implemented by 6 PMDs.
None of the 6 PMDS covers all the features.
>
> > > > Existing experimental marks are text only for human parsing.
> > > >
> > > > The compiler attribute and build time checks are missing, and the symbol in the
> > > > binary doesn't have experimental tag. Our scripts and automated checks won't
> > > > detect it as experimental.
> > > >
> > > > My point is just having experimental comment in header file is not enough to
> > > > qualify the APIs as experimental.
> > > >
> > > > > Anyway, we need to break the ABI to make it work on various HW.
>
> Yes this is why I was asking in 19.11 to check our API,
> in order to avoid such situation.
>
>
> > > > > I am not sure what to do?
>
> Either manage ABI versioning, or wait 20.11.
ABI change are in structures. So the function versioning does not
help. So we will wait for 20.11 then :-(
^ permalink raw reply [relevance 3%]
* Re: [dpdk-dev] [PATCH v3 0/4] add AESNI-MB rawdev for multi-function processing
2020-04-22 13:44 3% ` Akhil Goyal
2020-04-22 14:21 0% ` Coyle, David
@ 2020-05-01 13:18 0% ` Zhang, Roy Fan
1 sibling, 0 replies; 200+ results
From: Zhang, Roy Fan @ 2020-05-01 13:18 UTC (permalink / raw)
To: Akhil Goyal, Coyle, David, Doherty, Declan, Thomas Monjalon,
Yigit, Ferruh, Trahe, Fiona
Cc: techboard, dev, De Lara Guarch, Pablo, Ryan, Brendan,
Hemant Agrawal, Anoob Joseph, Ruifeng Wang, Liron Himi,
Nagadheeraj Rottela, Srikanth Jampala, Gagandeep Singh, Jay Zhou,
Ravi Kumar, Richardson, Bruce, olivier.matz,
honnappa.nagarahalli, Stephen Hemminger, alexr
Hi Akhil,
> -----Original Message-----
> From: dev <dev-bounces@dpdk.org> On Behalf Of Akhil Goyal
> Sent: Wednesday, April 22, 2020 2:44 PM
> To: Coyle, David <david.coyle@intel.com>; Doherty, Declan
> <declan.doherty@intel.com>; Thomas Monjalon <thomas@monjalon.net>;
> Yigit, Ferruh <ferruh.yigit@intel.com>; Trahe, Fiona <fiona.trahe@intel.com>
> Cc: techboard@dpdk.org; dev@dpdk.org; De Lara Guarch, Pablo
> <pablo.de.lara.guarch@intel.com>; Ryan, Brendan
> <brendan.ryan@intel.com>; Hemant Agrawal <hemant.agrawal@nxp.com>;
> Anoob Joseph <anoobj@marvell.com>; Ruifeng Wang
> <ruifeng.wang@arm.com>; Liron Himi <lironh@marvell.com>; Nagadheeraj
> Rottela <rnagadheeraj@marvell.com>; Srikanth Jampala
> <jsrikanth@marvell.com>; Gagandeep Singh <G.Singh@nxp.com>; Jay Zhou
> <jianjay.zhou@huawei.com>; Ravi Kumar <ravi1.kumar@amd.com>;
> Richardson, Bruce <bruce.richardson@intel.com>; olivier.matz@6wind.com;
> honnappa.nagarahalli@arm.com; Stephen Hemminger
> <stephen@networkplumber.org>; alexr@mellanox.com
> Subject: Re: [dpdk-dev] [PATCH v3 0/4] add AESNI-MB rawdev for multi-
> function processing
...
> Yes, it is preferred, but it should be a union to
> rte_crypto_sym_op/rte_crypto_asym_op.
> Crypto_op->type as RTE_CRYPTO_OP_TYPE_SECURITY and sess_type as
> RTE_CRYPTO_OP_SECURITY_SESSION
> The size of rte_crypto_op will remain as is and there will be no ABI breakage I
> guess.
>
[Fan: with this way the PMD will have to do rte_crypto_op.type check, and then look into rte_security_op field, only when it find the security_op type is crypto_crc, it will process the security_op data. Would that being too many reads and checking for a single op? Can we create a new API for rte_security to process rte_security_ops for Crypto_CRC or future needs?]
...
Regards,
Fan
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [EXT] Re: [PATCH v4 1/4] ethdev: add tm support for shaper config in pkt mode
2020-05-01 10:27 0% ` Ferruh Yigit
@ 2020-05-01 13:16 0% ` Nithin Dabilpuram
0 siblings, 0 replies; 200+ results
From: Nithin Dabilpuram @ 2020-05-01 13:16 UTC (permalink / raw)
To: Ferruh Yigit
Cc: Bruce Richardson, Dumitrescu, Cristian, Thomas Monjalon,
Jerin Jacob, Luca Boccassi, Nithin Dabilpuram, Singh, Jasvinder,
Andrew Rybchenko, dev, jerinj, kkanas, Kinsella, Ray,
Neil Horman, Kevin Traynor, David Marchand
On Fri, May 01, 2020 at 11:27:02AM +0100, Ferruh Yigit wrote:
> External Email
>
> ----------------------------------------------------------------------
> On 4/29/2020 10:03 AM, Bruce Richardson wrote:
> > On Wed, Apr 29, 2020 at 09:45:44AM +0100, Dumitrescu, Cristian wrote:
> >>
> >>
> >>> -----Original Message-----
> >>> From: Thomas Monjalon <thomas@monjalon.net>
> >>> Sent: Tuesday, April 28, 2020 4:54 PM
> >>> To: Jerin Jacob <jerinjacobk@gmail.com>; Dumitrescu, Cristian
> >>> <cristian.dumitrescu@intel.com>
> >>> Cc: Richardson, Bruce <bruce.richardson@intel.com>; Yigit, Ferruh
> >>> <ferruh.yigit@intel.com>; Luca Boccassi <bluca@debian.org>; Nithin
> >>> Dabilpuram <nithind1988@gmail.com>; Singh, Jasvinder
> >>> <jasvinder.singh@intel.com>; Andrew Rybchenko
> >>> <arybchenko@solarflare.com>; dev@dpdk.org; jerinj@marvell.com;
> >>> kkanas@marvell.com; Nithin Dabilpuram <ndabilpuram@marvell.com>;
> >>> Kinsella, Ray <ray.kinsella@intel.com>; Neil Horman
> >>> <nhorman@tuxdriver.com>; Kevin Traynor <ktraynor@redhat.com>; David
> >>> Marchand <david.marchand@redhat.com>
> >>> Subject: Re: [dpdk-dev] [PATCH v4 1/4] ethdev: add tm support for shaper
> >>> config in pkt mode
> >>>
> >>> 28/04/2020 17:04, Luca Boccassi:
> >>>> On Tue, 2020-04-28 at 15:45 +0100, Bruce Richardson wrote:
> >>>>> On Tue, Apr 28, 2020 at 03:06:20PM +0100, Ferruh Yigit wrote:
> >>>>>> On 4/27/2020 5:59 PM, Jerin Jacob wrote:
> >>>>>>> On Mon, Apr 27, 2020 at 10:19 PM Ferruh Yigit
> >>> <ferruh.yigit@intel.com> wrote:
> >>>>>>>> On 4/27/2020 5:29 PM, Jerin Jacob wrote:
> >>>>>>>>> On Mon, Apr 27, 2020 at 9:42 PM Ferruh Yigit
> >>> <ferruh.yigit@intel.com> wrote:
> >>>>>>>>>> On 4/27/2020 10:19 AM, Dumitrescu, Cristian wrote:
> >>>>>>>>>>> From: Yigit, Ferruh <ferruh.yigit@intel.com>
> >>>>>>>>>>>> On 4/24/2020 11:28 AM, Dumitrescu, Cristian wrote:
> >>>>>>>>>>>>> From: Nithin Dabilpuram <nithind1988@gmail.com>
> >>>>>>>>>>>>>> This patch also updates tm port/level/node capability
> >>> structures with
> >>>>>>>>>>>>>> exiting features of scheduler wfq packet mode,
> >>> scheduler wfq byte mode
> >>>>>>>>>>>>>> and private/shared shaper byte mode.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> SoftNIC PMD is also updated with new capabilities.
> >>> [...]
> >>>>>>>>>>>> Hi Nithin,
> >>>>>>>>>>>>
> >>>>>>>>>>>> It looks like patch is causing ABI break, I am getting following
> >>> warning [1],
> >>>>>>>>>>>> can you please check?
> >>>>>>>>>>>>
> >>>>>>>>>>>> [1]
> >>>>>>>>>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__pastebin.com_XYNFg14u&d=DwIDaQ&c=nKjWec2b6R0mOyPaz7xtfQ&r=FZ_tPCbgFOh18zwRPO9H0yDx8VW38vuapifdDfc8SFQ&m=ej5sP3-cEhEoCTZOia-QivXqgljtzBcMLtZGs-5c-Uc&s=B8z_5mQ2xO3C1izjmRe2zBApMrCUcW6KcAN-adglhJQ&e=
> >>>>>>>>>>>
> >>>>>>>>>>> Hi Ferruh,
> >>>>>>>>>>>
> >>>>>>>>>>> The RTE_TM API is marked as experimental,
> >>>>>>>>>>> but it looks that this was not correctly marked
> >>>>>>>>>>> when __rte_experimental ABI checker was introduced.
> >>>>>>>>>>>
> >>>>>>>>>>> It is marked as experimental at the top of the rte_tm.h,
> >>>>>>>>>>> similarly to other APIs introduced around same time,
> >>>>>>>>>>> but it was not correctly picked up by the ABI check procedure
> >>>>>>>>>>> when later introduced, so __rte_experimental was not added
> >>> to every function.
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> :(
> >>>>>>>>>>
> >>>>>>>>>> Is it time to mature them?
> >>>>>>>>>>
> >>>>>>>>>> As you said they are not marked as experimental both in header
> >>> file (function
> >>>>>>>>>> declarations) and .map file.
> >>>>>>>>>>
> >>>>>>>>>> The problem is, they are not marked as experimental in
> >>> DPDK_20.0 ABI (v19.11),
> >>>>>>>>>> so marking them as experimental now will break the ABI. Not
> >>> sure what to do,
> >>>>>>>>>> cc'ed a few ABI related names for comment.
> >>>>>>>>>>
> >>>>>>>>>> For me, we need to proceed as the experimental tag removed
> >>> and APIs become
> >>>>>>>>>> mature starting from v19.11, since this is what happened in
> >>> practice, and remove
> >>>>>>>>>> a few existing being experimental references in the doxygen
> >>> comments.
> >>>>>>>>>
> >>>>>>>>> I think, accidentally we can not make a library as NON-
> >>> experimental.
> >>>>>>>>> TM never went through experimental to mature transition(see git
> >>> log
> >>>>>>>>> lib/librte_ethdev/rte_tm.h)
> >>>>>>>>> It was a bug to not mark as experimental in each function in the
> >>> ABI process.
> >>>>>>>>> Some of the features like packet marking are not even
> >>> implemented by any HW.
> >>>>>>>>> I think, we can make API stable only all the features are
> >>> implemented
> >>>>>>>>> by one or two HW.
> >>>
> >>> Yes this is what was decided one or two years ago I think.
> >>> But rte_tm API was introduced 3 years ago and is implemented by 6 PMDs.
> >>>
> >>>
> >>>
> >>>>>>>> Fair enough, specially if the API is not ready yet.
> >>>>>>>>
> >>>>>>>> But they were part of stable ABI, and marking them as experimental
> >>> now will
> >>>>>>>> break the old applications using these APIs.
> >>>>>>>
> >>>>>>> it is still marked as EXPERIMENTAL everywhere and API is not ready
> >>> yet.
> >>>
> >>> rte_tm is implemented in 6 PMDs.
> >>>
> >>>
> >>>>>> Existing experimental marks are text only for human parsing.
> >>>>>>
> >>>>>> The compiler attribute and build time checks are missing, and the
> >>> symbol in the
> >>>>>> binary doesn't have experimental tag. Our scripts and automated
> >>> checks won't
> >>>>>> detect it as experimental.
> >>>>>>
> >>>>>> My point is just having experimental comment in header file is not
> >>> enough to
> >>>>>> qualify the APIs as experimental.
> >>>>>>
> >>>>>>> Anyway, we need to break the ABI to make it work on various HW.
> >>>
> >>> Yes this is why I was asking in 19.11 to check our API,
> >>> in order to avoid such situation.
> >>>
> >>>
> >>>>>>> I am not sure what to do?
> >>>
> >>> Either manage ABI versioning, or wait 20.11.
> >>>
> >>>
> >>>>>>> IMO, We need to send a patch as Fixes: for the bug of not adding
> >>>>>>> __rte_experimental in each function.
> >>>
> >>> No, this is wrong.
> >>>
> >>
> >> Why exactly is this wrong? This is the gap that caused the current discussion, right?
> >>
> > It's wrong for this release, since we can't change things from stable back
> > to experimental. Any such patch will have to wait for 20.11, as agreed in
> > the discussion.
> >
>
> Deferring the patchet for this release.
>
> Reminder that if the option "to mark rte_tm_* as experimental in v20.11"
> selected, requires deprecation notice before v20.11.
Thanks Ferruh for reminder. I'll send a deprecation notice patch for the same.
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH v4 1/4] ethdev: add tm support for shaper config in pkt mode
2020-04-29 9:03 0% ` Bruce Richardson
@ 2020-05-01 10:27 0% ` Ferruh Yigit
2020-05-01 13:16 0% ` [dpdk-dev] [EXT] " Nithin Dabilpuram
0 siblings, 1 reply; 200+ results
From: Ferruh Yigit @ 2020-05-01 10:27 UTC (permalink / raw)
To: Bruce Richardson, Dumitrescu, Cristian
Cc: Thomas Monjalon, Jerin Jacob, Luca Boccassi, Nithin Dabilpuram,
Singh, Jasvinder, Andrew Rybchenko, dev, jerinj, kkanas,
Nithin Dabilpuram, Kinsella, Ray, Neil Horman, Kevin Traynor,
David Marchand
On 4/29/2020 10:03 AM, Bruce Richardson wrote:
> On Wed, Apr 29, 2020 at 09:45:44AM +0100, Dumitrescu, Cristian wrote:
>>
>>
>>> -----Original Message-----
>>> From: Thomas Monjalon <thomas@monjalon.net>
>>> Sent: Tuesday, April 28, 2020 4:54 PM
>>> To: Jerin Jacob <jerinjacobk@gmail.com>; Dumitrescu, Cristian
>>> <cristian.dumitrescu@intel.com>
>>> Cc: Richardson, Bruce <bruce.richardson@intel.com>; Yigit, Ferruh
>>> <ferruh.yigit@intel.com>; Luca Boccassi <bluca@debian.org>; Nithin
>>> Dabilpuram <nithind1988@gmail.com>; Singh, Jasvinder
>>> <jasvinder.singh@intel.com>; Andrew Rybchenko
>>> <arybchenko@solarflare.com>; dev@dpdk.org; jerinj@marvell.com;
>>> kkanas@marvell.com; Nithin Dabilpuram <ndabilpuram@marvell.com>;
>>> Kinsella, Ray <ray.kinsella@intel.com>; Neil Horman
>>> <nhorman@tuxdriver.com>; Kevin Traynor <ktraynor@redhat.com>; David
>>> Marchand <david.marchand@redhat.com>
>>> Subject: Re: [dpdk-dev] [PATCH v4 1/4] ethdev: add tm support for shaper
>>> config in pkt mode
>>>
>>> 28/04/2020 17:04, Luca Boccassi:
>>>> On Tue, 2020-04-28 at 15:45 +0100, Bruce Richardson wrote:
>>>>> On Tue, Apr 28, 2020 at 03:06:20PM +0100, Ferruh Yigit wrote:
>>>>>> On 4/27/2020 5:59 PM, Jerin Jacob wrote:
>>>>>>> On Mon, Apr 27, 2020 at 10:19 PM Ferruh Yigit
>>> <ferruh.yigit@intel.com> wrote:
>>>>>>>> On 4/27/2020 5:29 PM, Jerin Jacob wrote:
>>>>>>>>> On Mon, Apr 27, 2020 at 9:42 PM Ferruh Yigit
>>> <ferruh.yigit@intel.com> wrote:
>>>>>>>>>> On 4/27/2020 10:19 AM, Dumitrescu, Cristian wrote:
>>>>>>>>>>> From: Yigit, Ferruh <ferruh.yigit@intel.com>
>>>>>>>>>>>> On 4/24/2020 11:28 AM, Dumitrescu, Cristian wrote:
>>>>>>>>>>>>> From: Nithin Dabilpuram <nithind1988@gmail.com>
>>>>>>>>>>>>>> This patch also updates tm port/level/node capability
>>> structures with
>>>>>>>>>>>>>> exiting features of scheduler wfq packet mode,
>>> scheduler wfq byte mode
>>>>>>>>>>>>>> and private/shared shaper byte mode.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> SoftNIC PMD is also updated with new capabilities.
>>> [...]
>>>>>>>>>>>> Hi Nithin,
>>>>>>>>>>>>
>>>>>>>>>>>> It looks like patch is causing ABI break, I am getting following
>>> warning [1],
>>>>>>>>>>>> can you please check?
>>>>>>>>>>>>
>>>>>>>>>>>> [1]
>>>>>>>>>>>> https://pastebin.com/XYNFg14u
>>>>>>>>>>>
>>>>>>>>>>> Hi Ferruh,
>>>>>>>>>>>
>>>>>>>>>>> The RTE_TM API is marked as experimental,
>>>>>>>>>>> but it looks that this was not correctly marked
>>>>>>>>>>> when __rte_experimental ABI checker was introduced.
>>>>>>>>>>>
>>>>>>>>>>> It is marked as experimental at the top of the rte_tm.h,
>>>>>>>>>>> similarly to other APIs introduced around same time,
>>>>>>>>>>> but it was not correctly picked up by the ABI check procedure
>>>>>>>>>>> when later introduced, so __rte_experimental was not added
>>> to every function.
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> :(
>>>>>>>>>>
>>>>>>>>>> Is it time to mature them?
>>>>>>>>>>
>>>>>>>>>> As you said they are not marked as experimental both in header
>>> file (function
>>>>>>>>>> declarations) and .map file.
>>>>>>>>>>
>>>>>>>>>> The problem is, they are not marked as experimental in
>>> DPDK_20.0 ABI (v19.11),
>>>>>>>>>> so marking them as experimental now will break the ABI. Not
>>> sure what to do,
>>>>>>>>>> cc'ed a few ABI related names for comment.
>>>>>>>>>>
>>>>>>>>>> For me, we need to proceed as the experimental tag removed
>>> and APIs become
>>>>>>>>>> mature starting from v19.11, since this is what happened in
>>> practice, and remove
>>>>>>>>>> a few existing being experimental references in the doxygen
>>> comments.
>>>>>>>>>
>>>>>>>>> I think, accidentally we can not make a library as NON-
>>> experimental.
>>>>>>>>> TM never went through experimental to mature transition(see git
>>> log
>>>>>>>>> lib/librte_ethdev/rte_tm.h)
>>>>>>>>> It was a bug to not mark as experimental in each function in the
>>> ABI process.
>>>>>>>>> Some of the features like packet marking are not even
>>> implemented by any HW.
>>>>>>>>> I think, we can make API stable only all the features are
>>> implemented
>>>>>>>>> by one or two HW.
>>>
>>> Yes this is what was decided one or two years ago I think.
>>> But rte_tm API was introduced 3 years ago and is implemented by 6 PMDs.
>>>
>>>
>>>
>>>>>>>> Fair enough, specially if the API is not ready yet.
>>>>>>>>
>>>>>>>> But they were part of stable ABI, and marking them as experimental
>>> now will
>>>>>>>> break the old applications using these APIs.
>>>>>>>
>>>>>>> it is still marked as EXPERIMENTAL everywhere and API is not ready
>>> yet.
>>>
>>> rte_tm is implemented in 6 PMDs.
>>>
>>>
>>>>>> Existing experimental marks are text only for human parsing.
>>>>>>
>>>>>> The compiler attribute and build time checks are missing, and the
>>> symbol in the
>>>>>> binary doesn't have experimental tag. Our scripts and automated
>>> checks won't
>>>>>> detect it as experimental.
>>>>>>
>>>>>> My point is just having experimental comment in header file is not
>>> enough to
>>>>>> qualify the APIs as experimental.
>>>>>>
>>>>>>> Anyway, we need to break the ABI to make it work on various HW.
>>>
>>> Yes this is why I was asking in 19.11 to check our API,
>>> in order to avoid such situation.
>>>
>>>
>>>>>>> I am not sure what to do?
>>>
>>> Either manage ABI versioning, or wait 20.11.
>>>
>>>
>>>>>>> IMO, We need to send a patch as Fixes: for the bug of not adding
>>>>>>> __rte_experimental in each function.
>>>
>>> No, this is wrong.
>>>
>>
>> Why exactly is this wrong? This is the gap that caused the current discussion, right?
>>
> It's wrong for this release, since we can't change things from stable back
> to experimental. Any such patch will have to wait for 20.11, as agreed in
> the discussion.
>
Deferring the patchet for this release.
Reminder that if the option "to mark rte_tm_* as experimental in v20.11"
selected, requires deprecation notice before v20.11.
^ permalink raw reply [relevance 0%]
* [dpdk-dev] [PATCH v4 2/2] cryptodev: version cryptodev info get function
@ 2020-04-30 15:52 10% ` Arek Kusztal
0 siblings, 0 replies; 200+ results
From: Arek Kusztal @ 2020-04-30 15:52 UTC (permalink / raw)
To: dev; +Cc: fiona.trahe, ray.kinsella, akhil.goyal, Arek Kusztal
This patch adds versioned function rte_cryptodev_info_get()
to prevent some issues with ABI policy.
Node v21 works in same way as before, returning driver capabilities
directly to the API caller. These capabilities may include new elements
not part of the v20 ABI.
Node v20 function maintains compatibility with v20 ABI releases
by stripping out elements not supported in v20 ABI. Because
rte_cryptodev_info_get is called by other API functions,
rte_cryptodev_sym_capability_get function is versioned the same way.
Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
---
v2:
- changed version numbers of symbols to 20.0.2
v3:
- added v2/v3 informations
- changed version numbers of symbols to 21
v4:
- fixed clang build by movind binding default symbol
- implemented mailing list comments
- added abidiff tool suppression entries
devtools/libabigail.abignore | 8 ++
lib/librte_cryptodev/rte_cryptodev.c | 147 ++++++++++++++++++++++++-
lib/librte_cryptodev/rte_cryptodev.h | 36 +++++-
lib/librte_cryptodev/rte_cryptodev_version.map | 7 ++
4 files changed, 192 insertions(+), 6 deletions(-)
diff --git a/devtools/libabigail.abignore b/devtools/libabigail.abignore
index 1911890..e479e20 100644
--- a/devtools/libabigail.abignore
+++ b/devtools/libabigail.abignore
@@ -24,3 +24,11 @@
type_kind = enum
name = rte_eth_event_type
changed_enumerators = RTE_ETH_EVENT_MAX
+; Ignore Cryptodev AEAD xform enum and AEAD xform strings change
+; due to addition of Chacha20-Poly1305
+[suppress_type]
+ type_kind = enum
+ name = rte_crypto_aead_algorithm
+ changed_enumerators = RTE_CRYPTO_AEAD_LIST_END
+[suppress_variable]
+ name = rte_crypto_aead_algorithm_strings
diff --git a/lib/librte_cryptodev/rte_cryptodev.c b/lib/librte_cryptodev/rte_cryptodev.c
index a499c47..e4a7b71 100644
--- a/lib/librte_cryptodev/rte_cryptodev.c
+++ b/lib/librte_cryptodev/rte_cryptodev.c
@@ -36,6 +36,8 @@
#include <rte_errno.h>
#include <rte_spinlock.h>
#include <rte_string_fns.h>
+#include <rte_compat.h>
+#include <rte_function_versioning.h>
#include "rte_crypto.h"
#include "rte_cryptodev.h"
@@ -57,6 +59,14 @@ static struct rte_cryptodev_global cryptodev_globals = {
/* spinlock for crypto device callbacks */
static rte_spinlock_t rte_cryptodev_cb_lock = RTE_SPINLOCK_INITIALIZER;
+static const struct rte_cryptodev_capabilities
+ cryptodev_undefined_capabilities[] = {
+ RTE_CRYPTODEV_END_OF_CAPABILITIES_LIST()
+};
+
+static struct rte_cryptodev_capabilities
+ *capability_copy[RTE_CRYPTO_MAX_DEVS];
+static uint8_t is_capability_checked[RTE_CRYPTO_MAX_DEVS];
/**
* The user application callback description.
@@ -281,15 +291,15 @@ rte_crypto_auth_operation_strings[] = {
[RTE_CRYPTO_AUTH_OP_GENERATE] = "generate"
};
-const struct rte_cryptodev_symmetric_capability *
-rte_cryptodev_sym_capability_get(uint8_t dev_id,
+const struct rte_cryptodev_symmetric_capability __vsym *
+rte_cryptodev_sym_capability_get_v20(uint8_t dev_id,
const struct rte_cryptodev_sym_capability_idx *idx)
{
const struct rte_cryptodev_capabilities *capability;
struct rte_cryptodev_info dev_info;
int i = 0;
- rte_cryptodev_info_get(dev_id, &dev_info);
+ rte_cryptodev_info_get_v20(dev_id, &dev_info);
while ((capability = &dev_info.capabilities[i++])->op !=
RTE_CRYPTO_OP_TYPE_UNDEFINED) {
@@ -313,8 +323,47 @@ rte_cryptodev_sym_capability_get(uint8_t dev_id,
}
return NULL;
+}
+VERSION_SYMBOL(rte_cryptodev_sym_capability_get, _v20, 20.0);
+
+const struct rte_cryptodev_symmetric_capability __vsym *
+rte_cryptodev_sym_capability_get_v21(uint8_t dev_id,
+ const struct rte_cryptodev_sym_capability_idx *idx)
+{
+ const struct rte_cryptodev_capabilities *capability;
+ struct rte_cryptodev_info dev_info;
+ int i = 0;
+
+ rte_cryptodev_info_get(dev_id, &dev_info);
+
+ while ((capability = &dev_info.capabilities[i++])->op !=
+ RTE_CRYPTO_OP_TYPE_UNDEFINED) {
+ if (capability->op != RTE_CRYPTO_OP_TYPE_SYMMETRIC)
+ continue;
+ if (capability->sym.xform_type != idx->type)
+ continue;
+
+ if (idx->type == RTE_CRYPTO_SYM_XFORM_AUTH &&
+ capability->sym.auth.algo == idx->algo.auth)
+ return &capability->sym;
+
+ if (idx->type == RTE_CRYPTO_SYM_XFORM_CIPHER &&
+ capability->sym.cipher.algo == idx->algo.cipher)
+ return &capability->sym;
+
+ if (idx->type == RTE_CRYPTO_SYM_XFORM_AEAD &&
+ capability->sym.aead.algo == idx->algo.aead)
+ return &capability->sym;
+ }
+
+ return NULL;
}
+MAP_STATIC_SYMBOL(const struct rte_cryptodev_symmetric_capability *
+ rte_cryptodev_sym_capability_get(uint8_t dev_id,
+ const struct rte_cryptodev_sym_capability_idx *idx),
+ rte_cryptodev_sym_capability_get_v21);
+BIND_DEFAULT_SYMBOL(rte_cryptodev_sym_capability_get, _v21, 21);
static int
param_range_check(uint16_t size, const struct rte_crypto_param_range *range)
@@ -1016,6 +1065,12 @@ rte_cryptodev_close(uint8_t dev_id)
retval = (*dev->dev_ops->dev_close)(dev);
rte_cryptodev_trace_close(dev_id, retval);
+ if (capability_copy[dev_id]) {
+ free(capability_copy[dev_id]);
+ capability_copy[dev_id] = NULL;
+ }
+ is_capability_checked[dev_id] = 0;
+
if (retval < 0)
return retval;
@@ -1129,9 +1184,61 @@ rte_cryptodev_stats_reset(uint8_t dev_id)
(*dev->dev_ops->stats_reset)(dev);
}
+static void
+get_v20_capabilities(uint8_t dev_id, struct rte_cryptodev_info *dev_info)
+{
+ const struct rte_cryptodev_capabilities *capability;
+ uint8_t found_invalid_capa = 0;
+ uint8_t counter = 0;
+
+ for (capability = dev_info->capabilities;
+ capability->op != RTE_CRYPTO_OP_TYPE_UNDEFINED;
+ ++capability, ++counter) {
+ if (capability->op == RTE_CRYPTO_OP_TYPE_SYMMETRIC &&
+ capability->sym.xform_type ==
+ RTE_CRYPTO_SYM_XFORM_AEAD
+ && capability->sym.aead.algo >=
+ RTE_CRYPTO_AEAD_CHACHA20_POLY1305) {
+ found_invalid_capa = 1;
+ counter--;
+ }
+ }
+ is_capability_checked[dev_id] = 1;
+ if (!found_invalid_capa)
+ return;
+ capability_copy[dev_id] = malloc(counter *
+ sizeof(struct rte_cryptodev_capabilities));
+ if (capability_copy[dev_id] == NULL) {
+ /*
+ * error case - no memory to store the trimmed
+ * list, so have to return an empty list
+ */
+ dev_info->capabilities =
+ cryptodev_undefined_capabilities;
+ is_capability_checked[dev_id] = 0;
+ } else {
+ counter = 0;
+ for (capability = dev_info->capabilities;
+ capability->op !=
+ RTE_CRYPTO_OP_TYPE_UNDEFINED;
+ capability++) {
+ if (!(capability->op ==
+ RTE_CRYPTO_OP_TYPE_SYMMETRIC
+ && capability->sym.xform_type ==
+ RTE_CRYPTO_SYM_XFORM_AEAD
+ && capability->sym.aead.algo >=
+ RTE_CRYPTO_AEAD_CHACHA20_POLY1305)) {
+ capability_copy[dev_id][counter++] =
+ *capability;
+ }
+ }
+ dev_info->capabilities =
+ capability_copy[dev_id];
+ }
+}
-void
-rte_cryptodev_info_get(uint8_t dev_id, struct rte_cryptodev_info *dev_info)
+void __vsym
+rte_cryptodev_info_get_v20(uint8_t dev_id, struct rte_cryptodev_info *dev_info)
{
struct rte_cryptodev *dev;
@@ -1147,10 +1254,40 @@ rte_cryptodev_info_get(uint8_t dev_id, struct rte_cryptodev_info *dev_info)
RTE_FUNC_PTR_OR_RET(*dev->dev_ops->dev_infos_get);
(*dev->dev_ops->dev_infos_get)(dev, dev_info);
+ if (capability_copy[dev_id] == NULL) {
+ if (!is_capability_checked[dev_id])
+ get_v20_capabilities(dev_id, dev_info);
+ } else
+ dev_info->capabilities = capability_copy[dev_id];
+
dev_info->driver_name = dev->device->driver->name;
dev_info->device = dev->device;
}
+VERSION_SYMBOL(rte_cryptodev_info_get, _v20, 20.0);
+void __vsym
+rte_cryptodev_info_get_v21(uint8_t dev_id, struct rte_cryptodev_info *dev_info)
+{
+ struct rte_cryptodev *dev;
+
+ if (!rte_cryptodev_pmd_is_valid_dev(dev_id)) {
+ CDEV_LOG_ERR("Invalid dev_id=%d", dev_id);
+ return;
+ }
+
+ dev = &rte_crypto_devices[dev_id];
+
+ memset(dev_info, 0, sizeof(struct rte_cryptodev_info));
+
+ RTE_FUNC_PTR_OR_RET(*dev->dev_ops->dev_infos_get);
+ (*dev->dev_ops->dev_infos_get)(dev, dev_info);
+
+ dev_info->driver_name = dev->device->driver->name;
+ dev_info->device = dev->device;
+}
+MAP_STATIC_SYMBOL(void rte_cryptodev_info_get(uint8_t dev_id,
+ struct rte_cryptodev_info *dev_info), rte_cryptodev_info_get_v21);
+BIND_DEFAULT_SYMBOL(rte_cryptodev_info_get, _v21, 21);
int
rte_cryptodev_callback_register(uint8_t dev_id,
diff --git a/lib/librte_cryptodev/rte_cryptodev.h b/lib/librte_cryptodev/rte_cryptodev.h
index 3dbb5ce..f58cedf 100644
--- a/lib/librte_cryptodev/rte_cryptodev.h
+++ b/lib/librte_cryptodev/rte_cryptodev.h
@@ -23,6 +23,8 @@ extern "C" {
#include "rte_dev.h"
#include <rte_common.h>
#include <rte_config.h>
+#include <rte_compat.h>
+#include <rte_function_versioning.h>
#include "rte_cryptodev_trace_fp.h"
@@ -219,6 +221,14 @@ struct rte_cryptodev_asym_capability_idx {
* - Return NULL if the capability not exist.
*/
const struct rte_cryptodev_symmetric_capability *
+rte_cryptodev_sym_capability_get_v20(uint8_t dev_id,
+ const struct rte_cryptodev_sym_capability_idx *idx);
+
+const struct rte_cryptodev_symmetric_capability *
+rte_cryptodev_sym_capability_get_v21(uint8_t dev_id,
+ const struct rte_cryptodev_sym_capability_idx *idx);
+
+const struct rte_cryptodev_symmetric_capability *
rte_cryptodev_sym_capability_get(uint8_t dev_id,
const struct rte_cryptodev_sym_capability_idx *idx);
@@ -762,9 +772,33 @@ rte_cryptodev_stats_reset(uint8_t dev_id);
* the last valid element has it's op field set to
* RTE_CRYPTO_OP_TYPE_UNDEFINED.
*/
-extern void
+
+void
rte_cryptodev_info_get(uint8_t dev_id, struct rte_cryptodev_info *dev_info);
+/* An extra element RTE_CRYPTO_AEAD_CHACHA20_POLY1305 is added
+ * to enum rte_crypto_aead_algorithm, also changing the value of
+ * RTE_CRYPTO_AEAD_LIST_END. To maintain ABI compatibility with applications
+ * which linked against earlier versions, preventing them, for example, from
+ * picking up the new value and using it to index into an array sized too small
+ * for it, it is necessary to have two versions of rte_cryptodev_info_get()
+ * The latest version just returns directly the capabilities retrieved from
+ * the device. The compatible version inspects the capabilities retrieved
+ * from the device, but only returns them directly if the new value
+ * is not included. If the new value is included, it allocates space
+ * for a copy of the device capabilities, trims the new value from this
+ * and returns this copy. It only needs to do this once per device.
+ * For the corner case of a corner case when the alloc may fail,
+ * an empty capability list is returned, as there is no mechanism to return
+ * an error and adding such a mechanism would itself be an ABI breakage.
+ * The compatible version can be removed after the next major ABI release.
+ */
+
+void
+rte_cryptodev_info_get_v20(uint8_t dev_id, struct rte_cryptodev_info *dev_info);
+
+void
+rte_cryptodev_info_get_v21(uint8_t dev_id, struct rte_cryptodev_info *dev_info);
/**
* Register a callback function for specific device id.
diff --git a/lib/librte_cryptodev/rte_cryptodev_version.map b/lib/librte_cryptodev/rte_cryptodev_version.map
index 87f5cd1..07a2d2f 100644
--- a/lib/librte_cryptodev/rte_cryptodev_version.map
+++ b/lib/librte_cryptodev/rte_cryptodev_version.map
@@ -58,6 +58,13 @@ DPDK_20.0 {
local: *;
};
+DPDK_21 {
+ global:
+ rte_cryptodev_info_get;
+ rte_cryptodev_sym_capability_get;
+} DPDK_20.0;
+
+
EXPERIMENTAL {
global:
--
2.1.0
^ permalink raw reply [relevance 10%]
* [dpdk-dev] DPDK Release Status Meeting 30/04/2020
@ 2020-04-30 13:55 4% Ferruh Yigit
0 siblings, 0 replies; 200+ results
From: Ferruh Yigit @ 2020-04-30 13:55 UTC (permalink / raw)
To: dpdk-dev; +Cc: Thomas Monjalon
Minutes 30 April 2020
---------------------
Agenda:
* Release Dates
* -rc1 status
* Subtrees
* OvS
* Opens
Participants:
* Arm
* Debian/Microsoft
* Intel
* Marvell
* Mellanox
* Red Hat
Release Dates
-------------
* v20.05 dates:
* -rc1 is released on Monday, 27 April
* http://inbox.dpdk.org/dev/2138318.D4D8VRik6i@thomas/
* -rc2 Friday 8 May 2020
* Release: Wednesday 20 May 2020
* v20.08 proposal dates, please comment:
* Proposal/V1: Tuesday, 16 June 2020
* Integration/Merge/RC1: Tuesday, 14 July 2020
* Release: Friday, 7 August 2020
* Please send roadmap for the release
-rc1 status
-----------
* Intel completed majority of tests, no major issue found so far
* Test report will be sent when ready
* Marvel test report also doesn't show any major issue
Subtrees
--------
* main
* Mainly fixes at this moment
* gcc10 fixes
* still not all fixed, should be fixed for -rc2
* eal and ABI patches
* __rte_internal
* working on merging rte_graph and rte_telemetry for -rc2
* bitops is cleanup patch, would be good to have it in -rc1
at this stage it may not make this release
* meson patches from Bruce to fix static linking
May be postponed to next release
* Rawdev for compress+crc multi function, short-term
solution needs to be decided in mail list
* next-net
* Started to merge patches, no big patchset for -rc2
* ethdev patch not accepted because of ABI break, it will need to wait 20.11
* http://inbox.dpdk.org/dev/1923738.gORTcIGjah@thomas/
* ~60 patches in backlog for -rc2
* next-crypto
* no update
* next-eventdev
* Only a few patches for -rc2 to be reviewed
* next-virtio
* Good progress this week, some patches already merged for -rc2
* mlx5 vdpa, packet ring vector path, zero copy optimizations, etc...
* Only a few remaining, PR almost ready
* next-net-intel
* Some fixes for -rc2 has been merged and pulled to next-net
* next-net-mrvl
* ~10 patches to be reviewed for -rc2
OvS
---
* Finished validating stable trees
* OvS 2.11 & 2.12 validated with DPDK 18.11.7
* OvS 2.13 & master branch validated with DPDK 19.11.1
Opens
-----
* Latest coverity scan showed 120 new defects, some of them from old code base,
this may be related to the coverity backend changes
* Need more support to resolve all new issues
DPDK Release Status Meetings
============================
The DPDK Release Status Meeting is intended for DPDK Committers to discuss
the status of the master tree and sub-trees, and for project managers to
track progress or milestone dates.
The meeting occurs on Thursdays at 8:30 UTC. If you wish to attend just
send an email to "John McNamara <john.mcnamara@intel.com>" for the invite.
^ permalink raw reply [relevance 4%]
* [dpdk-dev] [PATCH v6 1/1] abi: change references to abi 20.0.1 to abi v21
` (4 preceding siblings ...)
2020-04-30 10:21 31% ` [dpdk-dev] [PATCH v5] " Ray Kinsella
@ 2020-04-30 10:27 11% ` Ray Kinsella
2020-04-30 10:27 31% ` Ray Kinsella
2020-05-04 22:05 6% ` Thomas Monjalon
5 siblings, 2 replies; 200+ results
From: Ray Kinsella @ 2020-04-30 10:27 UTC (permalink / raw)
To: dev
Cc: david.marchand, Ray Kinsella, Thomas Monjalon, Neil Horman,
Jingjing Wu, Beilei Xing, Matan Azrad, Shahaf Shuler,
Viacheslav Ovsiienko, Jerin Jacob, Nithin Dabilpuram,
Alfredo Cardigliano, Mahipal Challa, Cristian Dumitrescu
Sending again, as I missed the cover letter last time.
v6:
* combined policy and versioning in the maintainers file.
Ray Kinsella (1):
abi: change references to abi 20.0.1 to abi v21
MAINTAINERS | 10 +++++-----
devtools/libabigail.abignore | 5 +++++
drivers/common/iavf/rte_common_iavf_version.map | 2 +-
drivers/common/mlx5/rte_common_mlx5_version.map | 2 +-
.../common/octeontx2/rte_common_octeontx2_version.map | 2 +-
drivers/net/ionic/rte_pmd_ionic_version.map | 2 +-
.../octeontx2_ep/rte_rawdev_octeontx2_ep_version.map | 2 +-
drivers/vdpa/mlx5/rte_pmd_mlx5_vdpa_version.map | 2 +-
lib/librte_meter/rte_meter_version.map | 2 +-
9 files changed, 17 insertions(+), 12 deletions(-)
--
2.23.0
^ permalink raw reply [relevance 11%]
* [dpdk-dev] [PATCH v6 1/1] abi: change references to abi 20.0.1 to abi v21
2020-04-30 10:27 11% ` [dpdk-dev] [PATCH v6 1/1] " Ray Kinsella
@ 2020-04-30 10:27 31% ` Ray Kinsella
2020-05-04 22:23 6% ` Thomas Monjalon
2020-05-04 22:05 6% ` Thomas Monjalon
1 sibling, 1 reply; 200+ results
From: Ray Kinsella @ 2020-04-30 10:27 UTC (permalink / raw)
To: dev
Cc: david.marchand, Ray Kinsella, Thomas Monjalon, Neil Horman,
Jingjing Wu, Beilei Xing, Matan Azrad, Shahaf Shuler,
Viacheslav Ovsiienko, Jerin Jacob, Nithin Dabilpuram,
Alfredo Cardigliano, Mahipal Challa, Cristian Dumitrescu
Change references to abi 20.0.1 to use abi v21, see
https://doc.dpdk.org/guides/contributing/abi_policy.html#general-guidelines
"Major ABI versions are declared no more frequently than yearly.
Compatibility with the major ABI version is mandatory in subsequent
releases until a new major ABI version is declared."
Combined abi policy and versioning in maintainers, add map files to the
filter to more closely monitor future abi changes.
Signed-off-by: Ray Kinsella <mdr@ashroe.eu>
---
MAINTAINERS | 10 +++++-----
devtools/libabigail.abignore | 5 +++++
drivers/common/iavf/rte_common_iavf_version.map | 2 +-
drivers/common/mlx5/rte_common_mlx5_version.map | 2 +-
.../common/octeontx2/rte_common_octeontx2_version.map | 2 +-
drivers/net/ionic/rte_pmd_ionic_version.map | 2 +-
.../octeontx2_ep/rte_rawdev_octeontx2_ep_version.map | 2 +-
drivers/vdpa/mlx5/rte_pmd_mlx5_vdpa_version.map | 2 +-
lib/librte_meter/rte_meter_version.map | 2 +-
9 files changed, 17 insertions(+), 12 deletions(-)
diff --git a/MAINTAINERS b/MAINTAINERS
index e05c80504..9756cb752 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -83,10 +83,6 @@ M: Marko Kovacevic <marko.kovacevic@intel.com>
F: README
F: doc/
-ABI Policy
-M: Ray Kinsella <mdr@ashroe.eu>
-F: doc/guides/contributing/abi_*.rst
-
Developers and Maintainers Tools
M: Thomas Monjalon <thomas@monjalon.net>
F: MAINTAINERS
@@ -140,10 +136,12 @@ M: Michael Santana <maicolgabriel@hotmail.com>
F: .travis.yml
F: .ci/
-ABI versioning
+ABI policy & versioning
+M: Ray Kinsella <mdr@ashroe.eu>
M: Neil Horman <nhorman@tuxdriver.com>
F: lib/librte_eal/include/rte_compat.h
F: lib/librte_eal/include/rte_function_versioning.h
+F: doc/guides/contributing/abi_*.rst
F: doc/guides/rel_notes/deprecation.rst
F: devtools/check-abi.sh
F: devtools/check-abi-version.sh
@@ -155,6 +153,8 @@ F: devtools/update_version_map_abi.py
F: devtools/validate-abi.sh
F: buildtools/check-symbols.sh
F: buildtools/map-list-symbol.sh
+F: drivers/*/*/*.map
+F: lib/*/*.map
Driver information
M: Neil Horman <nhorman@tuxdriver.com>
diff --git a/devtools/libabigail.abignore b/devtools/libabigail.abignore
index 986a52771..b0147fde6 100644
--- a/devtools/libabigail.abignore
+++ b/devtools/libabigail.abignore
@@ -29,3 +29,8 @@
type_kind = enum
name = rte_eth_event_type
changed_enumerators = RTE_ETH_EVENT_MAX
+; Explicit ignore ABI 20.0.1
+[suppress_function]
+ symbol_version = DPDK_20.0.1
+[suppress_variable]
+ symbol_version = DPDK_20.0.1
diff --git a/drivers/common/iavf/rte_common_iavf_version.map b/drivers/common/iavf/rte_common_iavf_version.map
index 2f11d67c0..92ceac108 100644
--- a/drivers/common/iavf/rte_common_iavf_version.map
+++ b/drivers/common/iavf/rte_common_iavf_version.map
@@ -1,4 +1,4 @@
-DPDK_20.0.1 {
+DPDK_21 {
global:
iavf_init_adminq;
diff --git a/drivers/common/mlx5/rte_common_mlx5_version.map b/drivers/common/mlx5/rte_common_mlx5_version.map
index b58a37827..564a9a7fb 100644
--- a/drivers/common/mlx5/rte_common_mlx5_version.map
+++ b/drivers/common/mlx5/rte_common_mlx5_version.map
@@ -1,4 +1,4 @@
-DPDK_20.0.1 {
+DPDK_21 {
global:
mlx5_class_get;
diff --git a/drivers/common/octeontx2/rte_common_octeontx2_version.map b/drivers/common/octeontx2/rte_common_octeontx2_version.map
index 8f2404bd9..01279c339 100644
--- a/drivers/common/octeontx2/rte_common_octeontx2_version.map
+++ b/drivers/common/octeontx2/rte_common_octeontx2_version.map
@@ -34,7 +34,7 @@ DPDK_20.0 {
local: *;
};
-DPDK_20.0.1 {
+DPDK_21 {
global:
otx2_eth_dev_is_sec_capable;
diff --git a/drivers/net/ionic/rte_pmd_ionic_version.map b/drivers/net/ionic/rte_pmd_ionic_version.map
index bc8fd6d4d..acdaf587d 100644
--- a/drivers/net/ionic/rte_pmd_ionic_version.map
+++ b/drivers/net/ionic/rte_pmd_ionic_version.map
@@ -1,4 +1,4 @@
-DPDK_20.0.1 {
+DPDK_21 {
local: *;
};
diff --git a/drivers/raw/octeontx2_ep/rte_rawdev_octeontx2_ep_version.map b/drivers/raw/octeontx2_ep/rte_rawdev_octeontx2_ep_version.map
index bc8fd6d4d..acdaf587d 100644
--- a/drivers/raw/octeontx2_ep/rte_rawdev_octeontx2_ep_version.map
+++ b/drivers/raw/octeontx2_ep/rte_rawdev_octeontx2_ep_version.map
@@ -1,4 +1,4 @@
-DPDK_20.0.1 {
+DPDK_21 {
local: *;
};
diff --git a/drivers/vdpa/mlx5/rte_pmd_mlx5_vdpa_version.map b/drivers/vdpa/mlx5/rte_pmd_mlx5_vdpa_version.map
index 179f7f1ae..4a76d1d52 100644
--- a/drivers/vdpa/mlx5/rte_pmd_mlx5_vdpa_version.map
+++ b/drivers/vdpa/mlx5/rte_pmd_mlx5_vdpa_version.map
@@ -1,3 +1,3 @@
-DPDK_20.0.1 {
+DPDK_21 {
local: *;
};
diff --git a/lib/librte_meter/rte_meter_version.map b/lib/librte_meter/rte_meter_version.map
index fc12cc0bf..2c7dadbca 100644
--- a/lib/librte_meter/rte_meter_version.map
+++ b/lib/librte_meter/rte_meter_version.map
@@ -13,7 +13,7 @@ DPDK_20.0 {
local: *;
};
-DPDK_20.0.1 {
+DPDK_21 {
global:
rte_meter_trtcm_rfc4115_color_aware_check;
--
2.23.0
^ permalink raw reply [relevance 31%]
* [dpdk-dev] [PATCH v5] abi: change references to abi 20.0.1 to abi v21
` (3 preceding siblings ...)
2020-04-27 13:45 11% ` [dpdk-dev] [PATCH v4] " Ray Kinsella
@ 2020-04-30 10:21 31% ` Ray Kinsella
2020-04-30 10:27 11% ` [dpdk-dev] [PATCH v6 1/1] " Ray Kinsella
5 siblings, 0 replies; 200+ results
From: Ray Kinsella @ 2020-04-30 10:21 UTC (permalink / raw)
To: dev
Cc: david.marchand, Ray Kinsella, Thomas Monjalon, Neil Horman,
Jingjing Wu, Beilei Xing, Matan Azrad, Shahaf Shuler,
Viacheslav Ovsiienko, Jerin Jacob, Nithin Dabilpuram,
Alfredo Cardigliano, Mahipal Challa, Cristian Dumitrescu
Change references to abi 20.0.1 to use abi v21, see
https://doc.dpdk.org/guides/contributing/abi_policy.html#general-guidelines
"Major ABI versions are declared no more frequently than yearly.
Compatibility with the major ABI version is mandatory in subsequent
releases until a new major ABI version is declared."
Combined abi policy and versioning in maintainers, add map files to the
filter to more closely monitor future abi changes.
Signed-off-by: Ray Kinsella <mdr@ashroe.eu>
---
MAINTAINERS | 10 +++++-----
devtools/libabigail.abignore | 5 +++++
drivers/common/iavf/rte_common_iavf_version.map | 2 +-
drivers/common/mlx5/rte_common_mlx5_version.map | 2 +-
.../common/octeontx2/rte_common_octeontx2_version.map | 2 +-
drivers/net/ionic/rte_pmd_ionic_version.map | 2 +-
.../octeontx2_ep/rte_rawdev_octeontx2_ep_version.map | 2 +-
drivers/vdpa/mlx5/rte_pmd_mlx5_vdpa_version.map | 2 +-
lib/librte_meter/rte_meter_version.map | 2 +-
9 files changed, 17 insertions(+), 12 deletions(-)
diff --git a/MAINTAINERS b/MAINTAINERS
index e05c80504..9756cb752 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -83,10 +83,6 @@ M: Marko Kovacevic <marko.kovacevic@intel.com>
F: README
F: doc/
-ABI Policy
-M: Ray Kinsella <mdr@ashroe.eu>
-F: doc/guides/contributing/abi_*.rst
-
Developers and Maintainers Tools
M: Thomas Monjalon <thomas@monjalon.net>
F: MAINTAINERS
@@ -140,10 +136,12 @@ M: Michael Santana <maicolgabriel@hotmail.com>
F: .travis.yml
F: .ci/
-ABI versioning
+ABI policy & versioning
+M: Ray Kinsella <mdr@ashroe.eu>
M: Neil Horman <nhorman@tuxdriver.com>
F: lib/librte_eal/include/rte_compat.h
F: lib/librte_eal/include/rte_function_versioning.h
+F: doc/guides/contributing/abi_*.rst
F: doc/guides/rel_notes/deprecation.rst
F: devtools/check-abi.sh
F: devtools/check-abi-version.sh
@@ -155,6 +153,8 @@ F: devtools/update_version_map_abi.py
F: devtools/validate-abi.sh
F: buildtools/check-symbols.sh
F: buildtools/map-list-symbol.sh
+F: drivers/*/*/*.map
+F: lib/*/*.map
Driver information
M: Neil Horman <nhorman@tuxdriver.com>
diff --git a/devtools/libabigail.abignore b/devtools/libabigail.abignore
index 986a52771..b0147fde6 100644
--- a/devtools/libabigail.abignore
+++ b/devtools/libabigail.abignore
@@ -29,3 +29,8 @@
type_kind = enum
name = rte_eth_event_type
changed_enumerators = RTE_ETH_EVENT_MAX
+; Explicit ignore ABI 20.0.1
+[suppress_function]
+ symbol_version = DPDK_20.0.1
+[suppress_variable]
+ symbol_version = DPDK_20.0.1
diff --git a/drivers/common/iavf/rte_common_iavf_version.map b/drivers/common/iavf/rte_common_iavf_version.map
index 2f11d67c0..92ceac108 100644
--- a/drivers/common/iavf/rte_common_iavf_version.map
+++ b/drivers/common/iavf/rte_common_iavf_version.map
@@ -1,4 +1,4 @@
-DPDK_20.0.1 {
+DPDK_21 {
global:
iavf_init_adminq;
diff --git a/drivers/common/mlx5/rte_common_mlx5_version.map b/drivers/common/mlx5/rte_common_mlx5_version.map
index b58a37827..564a9a7fb 100644
--- a/drivers/common/mlx5/rte_common_mlx5_version.map
+++ b/drivers/common/mlx5/rte_common_mlx5_version.map
@@ -1,4 +1,4 @@
-DPDK_20.0.1 {
+DPDK_21 {
global:
mlx5_class_get;
diff --git a/drivers/common/octeontx2/rte_common_octeontx2_version.map b/drivers/common/octeontx2/rte_common_octeontx2_version.map
index 8f2404bd9..01279c339 100644
--- a/drivers/common/octeontx2/rte_common_octeontx2_version.map
+++ b/drivers/common/octeontx2/rte_common_octeontx2_version.map
@@ -34,7 +34,7 @@ DPDK_20.0 {
local: *;
};
-DPDK_20.0.1 {
+DPDK_21 {
global:
otx2_eth_dev_is_sec_capable;
diff --git a/drivers/net/ionic/rte_pmd_ionic_version.map b/drivers/net/ionic/rte_pmd_ionic_version.map
index bc8fd6d4d..acdaf587d 100644
--- a/drivers/net/ionic/rte_pmd_ionic_version.map
+++ b/drivers/net/ionic/rte_pmd_ionic_version.map
@@ -1,4 +1,4 @@
-DPDK_20.0.1 {
+DPDK_21 {
local: *;
};
diff --git a/drivers/raw/octeontx2_ep/rte_rawdev_octeontx2_ep_version.map b/drivers/raw/octeontx2_ep/rte_rawdev_octeontx2_ep_version.map
index bc8fd6d4d..acdaf587d 100644
--- a/drivers/raw/octeontx2_ep/rte_rawdev_octeontx2_ep_version.map
+++ b/drivers/raw/octeontx2_ep/rte_rawdev_octeontx2_ep_version.map
@@ -1,4 +1,4 @@
-DPDK_20.0.1 {
+DPDK_21 {
local: *;
};
diff --git a/drivers/vdpa/mlx5/rte_pmd_mlx5_vdpa_version.map b/drivers/vdpa/mlx5/rte_pmd_mlx5_vdpa_version.map
index 179f7f1ae..4a76d1d52 100644
--- a/drivers/vdpa/mlx5/rte_pmd_mlx5_vdpa_version.map
+++ b/drivers/vdpa/mlx5/rte_pmd_mlx5_vdpa_version.map
@@ -1,3 +1,3 @@
-DPDK_20.0.1 {
+DPDK_21 {
local: *;
};
diff --git a/lib/librte_meter/rte_meter_version.map b/lib/librte_meter/rte_meter_version.map
index fc12cc0bf..2c7dadbca 100644
--- a/lib/librte_meter/rte_meter_version.map
+++ b/lib/librte_meter/rte_meter_version.map
@@ -13,7 +13,7 @@ DPDK_20.0 {
local: *;
};
-DPDK_20.0.1 {
+DPDK_21 {
global:
rte_meter_trtcm_rfc4115_color_aware_check;
--
2.23.0
^ permalink raw reply [relevance 31%]
* Re: [dpdk-dev] [PATCH v5] eal/cpuflags: add x86 based cpu flags
2020-04-29 11:39 3% ` David Marchand
@ 2020-04-30 10:02 0% ` Ray Kinsella
0 siblings, 0 replies; 200+ results
From: Ray Kinsella @ 2020-04-30 10:02 UTC (permalink / raw)
To: David Marchand
Cc: Kevin Laatz, dev, Bruce Richardson, Van Haaren Harry,
Thomas Monjalon, Kinsella, Ray, Neil Horman
So that isn't the issue either.
$ grep RTE_CPUFLAG_NUMFLAGS build-gcc-shared/install/dump/librte_eal.dump
4646: <enumerator name='RTE_CPUFLAG_NUMFLAGS' value='104'/>
$ grep RTE_CPUFLAG_NUMFLAGS /build/dpdk/reference/v20.02/build-gcc-shared/dump/librte_eal.dump
3296: <enumerator name='RTE_CPUFLAG_NUMFLAGS' value='87'/>
1.7-1.fc31 @updates
I finally got libabigail complaining about rte_cpu_flag_t, after doing a complete clear down.
So the suppression _is_ required.
I then spent the following hour trying to identify the gremlin in the system with no luck.
In anycase, added my ack below.
On 29/04/2020 12:39, David Marchand wrote:
> On Tue, Apr 28, 2020 at 6:39 PM Ray Kinsella <mdr@ashroe.eu> wrote:
>>> diff --git a/devtools/libabigail.abignore b/devtools/libabigail.abignore
>>> index a59df8f13..045f436fb 100644
>>> --- a/devtools/libabigail.abignore
>>> +++ b/devtools/libabigail.abignore
>>
>> Kevin - you still have the surpession.
>> I am testing locally with 1.7.1, and it doesn't complain when I disable the supression.
>> Are you seeing something different?
>
> Using current master libabigail, without the rule Kevin included, I
> get the warning:
>
> 1 function with some indirect sub-type change:
>
> [C] 'function int rte_cpu_get_flag_enabled(rte_cpu_flag_t)' at
> rte_cpuflags.c:144:1 has some indirect sub-type changes:
> parameter 1 of type 'enum rte_cpu_flag_t' has sub-type changes:
> type size hasn't changed
> 17 enumerator insertions:
> 'rte_cpu_flag_t::RTE_CPUFLAG_AVX512DQ' value '87'
> 'rte_cpu_flag_t::RTE_CPUFLAG_AVX512IFMA' value '88'
> 'rte_cpu_flag_t::RTE_CPUFLAG_AVX512CD' value '89'
> 'rte_cpu_flag_t::RTE_CPUFLAG_AVX512BW' value '90'
> 'rte_cpu_flag_t::RTE_CPUFLAG_AVX512VL' value '91'
> 'rte_cpu_flag_t::RTE_CPUFLAG_AVX512VBMI' value '92'
> 'rte_cpu_flag_t::RTE_CPUFLAG_AVX512VBMI2' value '93'
> 'rte_cpu_flag_t::RTE_CPUFLAG_GFNI' value '94'
> 'rte_cpu_flag_t::RTE_CPUFLAG_VAES' value '95'
> 'rte_cpu_flag_t::RTE_CPUFLAG_VPCLMULQDQ' value '96'
> 'rte_cpu_flag_t::RTE_CPUFLAG_AVX512VNNI' value '97'
> 'rte_cpu_flag_t::RTE_CPUFLAG_AVX512BITALG' value '98'
> 'rte_cpu_flag_t::RTE_CPUFLAG_AVX512VPOPCNTDQ' value '99'
> 'rte_cpu_flag_t::RTE_CPUFLAG_CLDEMOTE' value '100'
> 'rte_cpu_flag_t::RTE_CPUFLAG_MOVDIRI' value '101'
> 'rte_cpu_flag_t::RTE_CPUFLAG_MOVDIR64B' value '102'
> 'rte_cpu_flag_t::RTE_CPUFLAG_AVX512VP2INTERSECT' value '103'
> 1 enumerator change:
> 'rte_cpu_flag_t::RTE_CPUFLAG_NUMFLAGS' from value '87' to
> '104' at rte_cpuflags.h:12:1
>
>
> Ray, could you check that the reference and new dumps in your env
> contain this enum?
>
> $ grep RTE_CPUFLAG_NUMFLAGS
> $HOME/abi/v20.02/x86_64-native-linux-gcc+shared+debug+ASSERT+RTE_IBVERBS_LINK_DLOPEN/dump/librte_eal.dump
> <enumerator name='RTE_CPUFLAG_NUMFLAGS' value='87'/>
> $ grep RTE_CPUFLAG_NUMFLAGS
> $HOME/builds/x86_64-native-linux-gcc+shared+debug+ASSERT+RTE_IBVERBS_LINK_DLOPEN/install/dump/librte_eal.dump
> <enumerator name='RTE_CPUFLAG_NUMFLAGS' value='104'/>
>
> If you are missing those, you might have built dpdk without debuginfo.
>
Acked-By: Ray Kinsella <mdr@ashroe.eu>
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [RFC] hash: unify crc32 API header for x86 and ARM
2020-04-30 9:14 0% ` Van Haaren, Harry
@ 2020-04-30 9:27 0% ` Pavan Nikhilesh Bhagavatula
0 siblings, 0 replies; 200+ results
From: Pavan Nikhilesh Bhagavatula @ 2020-04-30 9:27 UTC (permalink / raw)
To: Van Haaren, Harry, Jerin Jacob Kollanukkaran, thomas, Wang,
Yipeng1, Gobriel, Sameh, Richardson, Bruce, Ruifeng Wang
Cc: dev
>> -----Original Message-----
>> From: dev <dev-bounces@dpdk.org> On Behalf Of
>pbhagavatula@marvell.com
>> Sent: Wednesday, April 29, 2020 7:05 PM
>> To: jerinj@marvell.com; thomas@monjalon.net; Wang, Yipeng1
>> <yipeng1.wang@intel.com>; Gobriel, Sameh
><sameh.gobriel@intel.com>;
>> Richardson, Bruce <bruce.richardson@intel.com>; Ruifeng Wang
>> <ruifeng.wang@arm.com>
>> Cc: dev@dpdk.org; Pavan Nikhilesh <pbhagavatula@marvell.com>
>> Subject: [dpdk-dev] [RFC] hash: unify crc32 API header for x86 and
>ARM
>>
>> From: Pavan Nikhilesh <pbhagavatula@marvell.com>
>>
>> Merge crc32 hash calculation public API headers for x86 and ARM,
>> split implementations of x86 and ARM into their respective private
>> headers.
>> This reduces the ifdef code clutter while keeping current ABI intact.
>>
>> Although we install `rte_crc_arm64.h` it is not used in any of the lib or
>> drivers layers. All the libs and drivers use `rte_hash_crc.h` which falls
>> back to SW crc32 calculation for ARM platform.
>
><snip lots of stuff, to focus on meson install header change>
>
>> diff --git a/lib/librte_hash/meson.build b/lib/librte_hash/meson.build
>> index 6ab46ae9d..90a180bc8 100644
>> --- a/lib/librte_hash/meson.build
>> +++ b/lib/librte_hash/meson.build
>> @@ -1,8 +1,7 @@
>> # SPDX-License-Identifier: BSD-3-Clause
>> # Copyright(c) 2017 Intel Corporation
>>
>> -headers = files('rte_crc_arm64.h',
>> - 'rte_fbk_hash.h',
>> +headers = files('rte_fbk_hash.h',
>> 'rte_hash_crc.h',
>> 'rte_hash.h',
>> 'rte_jhash.h',
>
>Am I right in that previously an application could #include
><rte_crc_arm64.h> and hence if we no
>longer install that file, this will cause a compilation failure on that
>application? Applications shouldn't
>include arch specific headers... but we shouldn't knowingly remove
>publicly accessible includes either.
>
>Perhaps consider just installing a dummy header file if the code cleanup
>in the rest of the patch is desired?
Sure we could either symlink `rte_hash_crc.h` as `rte_crc_arm64.h` or
Just include rte_hash_crc.h in rte_crc_arm64.h for now and remove
rte_crc_arm64.h later with a deprecation notice?
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [RFC] hash: unify crc32 API header for x86 and ARM
2020-04-29 18:05 2% [dpdk-dev] [RFC] hash: unify crc32 API header for x86 and ARM pbhagavatula
@ 2020-04-30 9:14 0% ` Van Haaren, Harry
2020-04-30 9:27 0% ` Pavan Nikhilesh Bhagavatula
0 siblings, 1 reply; 200+ results
From: Van Haaren, Harry @ 2020-04-30 9:14 UTC (permalink / raw)
To: pbhagavatula, jerinj, thomas, Wang, Yipeng1, Gobriel, Sameh,
Richardson, Bruce, Ruifeng Wang
Cc: dev
> -----Original Message-----
> From: dev <dev-bounces@dpdk.org> On Behalf Of pbhagavatula@marvell.com
> Sent: Wednesday, April 29, 2020 7:05 PM
> To: jerinj@marvell.com; thomas@monjalon.net; Wang, Yipeng1
> <yipeng1.wang@intel.com>; Gobriel, Sameh <sameh.gobriel@intel.com>;
> Richardson, Bruce <bruce.richardson@intel.com>; Ruifeng Wang
> <ruifeng.wang@arm.com>
> Cc: dev@dpdk.org; Pavan Nikhilesh <pbhagavatula@marvell.com>
> Subject: [dpdk-dev] [RFC] hash: unify crc32 API header for x86 and ARM
>
> From: Pavan Nikhilesh <pbhagavatula@marvell.com>
>
> Merge crc32 hash calculation public API headers for x86 and ARM,
> split implementations of x86 and ARM into their respective private
> headers.
> This reduces the ifdef code clutter while keeping current ABI intact.
>
> Although we install `rte_crc_arm64.h` it is not used in any of the lib or
> drivers layers. All the libs and drivers use `rte_hash_crc.h` which falls
> back to SW crc32 calculation for ARM platform.
<snip lots of stuff, to focus on meson install header change>
> diff --git a/lib/librte_hash/meson.build b/lib/librte_hash/meson.build
> index 6ab46ae9d..90a180bc8 100644
> --- a/lib/librte_hash/meson.build
> +++ b/lib/librte_hash/meson.build
> @@ -1,8 +1,7 @@
> # SPDX-License-Identifier: BSD-3-Clause
> # Copyright(c) 2017 Intel Corporation
>
> -headers = files('rte_crc_arm64.h',
> - 'rte_fbk_hash.h',
> +headers = files('rte_fbk_hash.h',
> 'rte_hash_crc.h',
> 'rte_hash.h',
> 'rte_jhash.h',
Am I right in that previously an application could #include <rte_crc_arm64.h> and hence if we no
longer install that file, this will cause a compilation failure on that application? Applications shouldn't
include arch specific headers... but we shouldn't knowingly remove publicly accessible includes either.
Perhaps consider just installing a dummy header file if the code cleanup in the rest of the patch is desired?
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH v2] abi: change references to abi 20.0.1 to abi v21
2020-04-29 12:19 9% ` Dodji Seketeli
@ 2020-04-30 8:23 6% ` Ray Kinsella
0 siblings, 0 replies; 200+ results
From: Ray Kinsella @ 2020-04-30 8:23 UTC (permalink / raw)
To: Dodji Seketeli
Cc: David Marchand, dev, Andrzej Ostruszka, Stephen Hemminger,
Thomas Monjalon, Neil Horman, Jingjing Wu, Wenzhuo Lu,
Matan Azrad, Shahaf Shuler, Viacheslav Ovsiienko, Jerin Jacob,
Nithin Dabilpuram, Alfredo Cardigliano, Mahipal Challa,
Cristian Dumitrescu, Wang, Haiyue
On 29/04/2020 13:19, Dodji Seketeli wrote:
> Hello,
>
> Ray Kinsella <mdr@ashroe.eu> writes:
>
>> ah ok, the particular system I made the change on was Ubuntu 18.04.2.
>> which is libabigail 1.2.0.
>
> Whoah, 1.2 is super old.
I have a huge clunking raid'ed "build" server,
that I am pretty conservative about upgrading on v18.04.2 :-)
> In my opinion, one of the hallmarks of static analysis tools (and
> libabigail is just a static analysis framework) is to be able to
> recognize patterns used by developers, as much as we can.
>
> Because we can't really do that at once, we try to add recognition of
> new patterns (of ABI changes) at every single release. Furthermore,
> there are some change patterns that ought to be recognized and
> categorized as harmless, whereas some others out to be categorized as
> harmful. That categorization is also the result of input coming from
> users as you, fine fellows.
>
> All this to say that with every new version, the number of new supported
> features and bug fixes is potentially big.
>
> To alleviate that, some distributors update libabigail even in their old
> stable distros, because the value of having an up to date version there
> outweighs the potential drawbacks.
Well it falls into the same category of problems of upgrading compilers.
User's typically build their software build reliably on a given distro version.
(or number of versions).
If the maintainer upgrades compilers between distro versions, it introduces new
warnings and errors, and all hell will generally break loose, when the user least expects it.
They typically expect that mayhem when upgrading to new distro versions.
Even tools like GNU indent can cause enormous problems.
I would imagine that most maintainers would be pretty conservative about making
such changes.
>
>> Given we still support v19.11 on Ubuntu 18.04.2.
>
> So maybe that's a discussion worth having with the maintainer of the
> Ubuntu package of Libabigail?
yes - I think it would be an interesting discussion alright.
I imagine the response might be inline with my understanding above.
Let's find out - as we can expect v18.04 to be around for at least another
two years I guess.
Another common way to handle this, is to be really explicit about what
OS distros DPDK formally supports building on, which will then imply
we support a small handful of versions of libabigail.
We then simply say we don't support 18.04 anymore - FD.io VPP are planning
this formal switch at the moment.
>
>> I think it's worthwhile keeping the suppression until v20.11?
>
> [...]
>
> David Marchand <david.marchand@redhat.com> writes:
>
>> In Travis, we currently use libabigail 1.6 (mainly because I did not
>> update to 1.7 when it was released).
>
> Right, that's probably another way to stay up to date independently from
> the underlying distribution.
You typically don't want to encourage this, you end up with some people
on a newer version, some people on an old version and never upgrading.
Then you never end up with a consistent view of what mitigations are actually required.
Solving issues, becomes like a game of whack-a-mole.
>
> I hope this helps,
It does, greatly thanks.
>
> Cheers,
>
^ permalink raw reply [relevance 6%]
* Re: [dpdk-dev] [PATCH v4] eal/cpuflags: add x86 based cpu flags
2020-04-29 11:22 0% ` Neil Horman
@ 2020-04-30 7:59 4% ` Ray Kinsella
0 siblings, 0 replies; 200+ results
From: Ray Kinsella @ 2020-04-30 7:59 UTC (permalink / raw)
To: Neil Horman
Cc: Thomas Monjalon, ray.kinsella, Kevin Laatz, dev,
bruce.richardson, harry.van.haaren, david.marchand, Haiyue Wang,
ktraynor
On 29/04/2020 12:22, Neil Horman wrote:
> On Mon, Apr 27, 2020 at 02:58:07PM +0100, Ray Kinsella wrote:
>>
>>
>> On 27/04/2020 13:31, Thomas Monjalon wrote:
>>> 27/04/2020 11:27, Ray Kinsella:
>>>> On 25/04/2020 17:04, Thomas Monjalon wrote:
>>>>> PS: Who is REALLY maintaining the ABI?
>>>>> We really miss someone who carefully check all these things,
>>>>> and take care of the doc and tooling.
>>>>
>>>> I would say that I am missing these changes to libabigail.ignore, which would be useful.
>>>> Should we consolidate the ABI Policy and ABI Versioning sections of the MAINTAINERS file?
>>>
>>> Yes, I think it does not make sense spliting ABI topic in 2 sections
>>> in MAINTAINERS file.
>>> We need to have a clear ownership covering policy, libs, tooling and doc.
>>> Let's agree to merge all in one section please.
>>>
>>
>> I would suggest merging and listing myself and Neil as maintainers?
>> Unless you are aware of another potential owner?
>>
> I'm ok with this
> Neil
ok I will take care of it in the next rev of the
"[PATCH] abi: change references to abi 20.0.1 to abi v21"
>> Ray K
>>
>>
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH v4] ethdev: support flow aging
2020-04-30 7:36 0% ` Matan Azrad
@ 2020-04-30 7:49 0% ` Tom Barbette
0 siblings, 0 replies; 200+ results
From: Tom Barbette @ 2020-04-30 7:49 UTC (permalink / raw)
To: Matan Azrad, Bill Zhou, Ori Kam, wenzhuo.lu, jingjing.wu,
bernard.iremonger, john.mcnamara, marko.kovacevic,
Thomas Monjalon, ferruh.yigit, arybchenko
Cc: dev
Le 30/04/2020 à 09:36, Matan Azrad a écrit :
>
> Hi Tom
>
> From: Tom Barbette
>> Great news!
>>
>> - I can understand why there is no timeout unit. But that's calling for user
>> nightmare. Eg I could only get from the code (and not from documentation
>> yet? ) of the following mlx5 driver patch that the value should be in tenth of
>> seconds. If I build an application that is supposed to work with "any NIC",
>> what can I do? We'd need a way to query the timeout unit (have it in
>> dev_info probably).
>
> Please see the new age action structure in rte_flow.h
> You can see comments there that timeout units is in seconds....
Oh okay, did not catch that. Maybe mention of the unit in the AGE action
documentation of rte_flow.rst would be helpful.
>
>> - It's not totally clear if the rule is automatically removed or not. is this a
>> helper or an OpenFlow-like notification?
>
> Only notification, the aged-out flow should be destroyed (or other action) by the application according to the application needs...
Makes sense.
>
>> - Find a typo and grammar fix inline.
>> - Recently, Mellanox introduced the ability to create 330K flows/s. Any
>> performance considerations if those flow "expire" at the same rate?
>
> We didn't see performance impact (should be same rate like count action).
Ok great!
Thanks!
>
>>
>> Hope it's helpfull,
>>
>> Tom
>>
>> Le 21/04/2020 à 12:11, Bill Zhou a écrit :
>>> From: Dong Zhou <dongz@mellanox.com>
>>>
>>> One of the reasons to destroy a flow is the fact that no packet
>>> matches the flow for "timeout" time.
>>> For example, when TCP\UDP sessions are suddenly closed.
>>>
>>> Currently, there is not any DPDK mechanism for flow aging and the
>>> applications use their own ways to detect and destroy aged-out flows.
>>>
>>> The flow aging implementation need include:
>>> - A new rte_flow action: RTE_FLOW_ACTION_TYPE_AGE to set the timeout
>> and
>>> the application flow context for each flow.
>>> - A new ethdev event: RTE_ETH_EVENT_FLOW_AGED for the driver to
>> report
>>> that there are new aged-out flows.
>>> - A new rte_flow API: rte_flow_get_aged_flows to get the aged-out flows
>>> contexts from the port.
>>> - Support input flow aging command line in Testpmd.
>>>
>>> The new event type addition in the enum is flagged as an ABI breakage,
>>> so an ignore rule is added for these reasons:
>>> - It is not changing value of existing types (except MAX)
>>> - The new value is not used by existing API if the event is not
>>> registered In general, it is safe adding new ethdev event types at the
>>> end of the enum, because of event callback registration mechanism.
>>>
>>> Signed-off-by: Dong Zhou <dongz@mellanox.com>
>>> ---
>>> v2: Removing "* Added support for flow Aging mechanism base on
>> counter."
>>> this line from doc/guides/rel_notes/release_20_05.rst, this patch does
>>> not include this support.
>>>
>>> v3: Update file libabigail.abignore, add one new suppressed
>>> enumeration type for RTE_ETH_EVENT_MAX.
>>>
>>> v4: Add justification in devtools/libabigail.abignore and in the
>>> commit log about the modification of v3.
>>> ---
>>> app/test-pmd/cmdline_flow.c | 26 ++++++++++
>>> devtools/libabigail.abignore | 6 +++
>>> doc/guides/prog_guide/rte_flow.rst | 22 +++++++++
>>> doc/guides/rel_notes/release_20_05.rst | 11 +++++
>>> lib/librte_ethdev/rte_ethdev.h | 1 +
>>> lib/librte_ethdev/rte_ethdev_version.map | 3 ++
>>> lib/librte_ethdev/rte_flow.c | 18 +++++++
>>> lib/librte_ethdev/rte_flow.h | 62 ++++++++++++++++++++++++
>>> lib/librte_ethdev/rte_flow_driver.h | 6 +++
>>> 9 files changed, 155 insertions(+)
>>>
>>> diff --git a/app/test-pmd/cmdline_flow.c b/app/test-pmd/cmdline_flow.c
>>> index e6ab8ff2f7..45bcff3cf5 100644
>>> --- a/app/test-pmd/cmdline_flow.c
>>> +++ b/app/test-pmd/cmdline_flow.c
>>> @@ -343,6 +343,8 @@ enum index {
>>> ACTION_SET_IPV4_DSCP_VALUE,
>>> ACTION_SET_IPV6_DSCP,
>>> ACTION_SET_IPV6_DSCP_VALUE,
>>> + ACTION_AGE,
>>> + ACTION_AGE_TIMEOUT,
>>> };
>>>
>>> /** Maximum size for pattern in struct rte_flow_item_raw. */ @@
>>> -1145,6 +1147,7 @@ static const enum index next_action[] = {
>>> ACTION_SET_META,
>>> ACTION_SET_IPV4_DSCP,
>>> ACTION_SET_IPV6_DSCP,
>>> + ACTION_AGE,
>>> ZERO,
>>> };
>>>
>>> @@ -1370,6 +1373,13 @@ static const enum index action_set_ipv6_dscp[]
>> = {
>>> ZERO,
>>> };
>>>
>>> +static const enum index action_age[] = {
>>> + ACTION_AGE,
>>> + ACTION_AGE_TIMEOUT,
>>> + ACTION_NEXT,
>>> + ZERO,
>>> +};
>>> +
>>> static int parse_set_raw_encap_decap(struct context *, const struct
>> token *,
>>> const char *, unsigned int,
>>> void *, unsigned int);
>>> @@ -3694,6 +3704,22 @@ static const struct token token_list[] = {
>>> (struct rte_flow_action_set_dscp, dscp)),
>>> .call = parse_vc_conf,
>>> },
>>> + [ACTION_AGE] = {
>>> + .name = "age",
>>> + .help = "set a specific metadata header",
>>> + .next = NEXT(action_age),
>>> + .priv = PRIV_ACTION(AGE,
>>> + sizeof(struct rte_flow_action_age)),
>>> + .call = parse_vc,
>>> + },
>>> + [ACTION_AGE_TIMEOUT] = {
>>> + .name = "timeout",
>>> + .help = "flow age timeout value",
>>> + .args = ARGS(ARGS_ENTRY_BF(struct rte_flow_action_age,
>>> + timeout, 24)),
>>> + .next = NEXT(action_age, NEXT_ENTRY(UNSIGNED)),
>>> + .call = parse_vc_conf,
>>> + },
>>> };
>>>
>>> /** Remove and return last entry from argument stack. */ diff --git
>>> a/devtools/libabigail.abignore b/devtools/libabigail.abignore index
>>> a59df8f135..c047adbd79 100644
>>> --- a/devtools/libabigail.abignore
>>> +++ b/devtools/libabigail.abignore
>>> @@ -11,3 +11,9 @@
>>> type_kind = enum
>>> name = rte_crypto_asym_xform_type
>>> changed_enumerators =
>> RTE_CRYPTO_ASYM_XFORM_TYPE_LIST_END
>>> +; Ignore ethdev event enum update because new event cannot be ;
>>> +received if not registered [suppress_type]
>>> + type_kind = enum
>>> + name = rte_eth_event_type
>>> + changed_enumerators = RTE_ETH_EVENT_MAX
>>> diff --git a/doc/guides/prog_guide/rte_flow.rst
>>> b/doc/guides/prog_guide/rte_flow.rst
>>> index 41c147913c..cf4368e1c4 100644
>>> --- a/doc/guides/prog_guide/rte_flow.rst
>>> +++ b/doc/guides/prog_guide/rte_flow.rst
>>> @@ -2616,6 +2616,28 @@ Otherwise, RTE_FLOW_ERROR_TYPE_ACTION
>> error will be returned.
>>> | ``dscp`` | DSCP in low 6 bits, rest ignore |
>>> +-----------+---------------------------------+
>>>
>>> +Action: ``AGE``
>>> +^^^^^^^^^^^^^^^^^^^^^^^^^
>>> +
>>> +Set ageing timeout configuration to a flow.
>>> +
>>> +Event RTE_ETH_EVENT_FLOW_AGED will be reported if timeout passed
>>> +without any matching on the flow.
>>> +
>>> +.. _table_rte_flow_action_age:
>>> +
>>> +.. table:: AGE
>>> +
>>> + +--------------+---------------------------------+
>>> + | Field | Value |
>>> + +==============+=================================+
>>> + | ``timeout`` | 24 bits timeout value |
>>> + +--------------+---------------------------------+
>>> + | ``reserved`` | 8 bits reserved, must be zero |
>>> + +--------------+---------------------------------+
>>> + | ``context`` | user input flow context |
>>> + +--------------+---------------------------------+
>>> +
>>> Negative types
>>> ~~~~~~~~~~~~~~
>>>
>>> diff --git a/doc/guides/rel_notes/release_20_05.rst
>>> b/doc/guides/rel_notes/release_20_05.rst
>>> index bacd4c65a2..ff0cf9f1d6 100644
>>> --- a/doc/guides/rel_notes/release_20_05.rst
>>> +++ b/doc/guides/rel_notes/release_20_05.rst
>>> @@ -135,6 +135,17 @@ New Features
>>> by making use of the event device capabilities. The event mode currently
>> supports
>>> only inline IPsec protocol offload.
>>>
>>> +* **Added flow Aging Support.**
>>> +
>>> + Added flow Aging support to detect and report aged-out flows,
>> including:
>>> +
>>> + * Added new action: RTE_FLOW_ACTION_TYPE_AGE to set the timeout
>> and the
>>> + application flow context for each flow.
>>> + * Added new event: RTE_ETH_EVENT_FLOW_AGED for the driver to
>> report that
>>> + there are new aged-out flows.
>>> + * Added new API: rte_flow_get_aged_flows to get the aged-out flows
>> contexts
>>> + from the port.
>>> +
>>>
>>> Removed Items
>>> -------------
>>> diff --git a/lib/librte_ethdev/rte_ethdev.h
>>> b/lib/librte_ethdev/rte_ethdev.h index 8d69b88f9e..00cc7b4052 100644
>>> --- a/lib/librte_ethdev/rte_ethdev.h
>>> +++ b/lib/librte_ethdev/rte_ethdev.h
>>> @@ -3018,6 +3018,7 @@ enum rte_eth_event_type {
>>> RTE_ETH_EVENT_NEW, /**< port is probed */
>>> RTE_ETH_EVENT_DESTROY, /**< port is released */
>>> RTE_ETH_EVENT_IPSEC, /**< IPsec offload related event */
>>> + RTE_ETH_EVENT_FLOW_AGED,/**< New aged-out flows is detected
>> */
>>> RTE_ETH_EVENT_MAX /**< max value of this enum */
>>> };
>>>
>>> diff --git a/lib/librte_ethdev/rte_ethdev_version.map
>>> b/lib/librte_ethdev/rte_ethdev_version.map
>>> index 3f32fdecf7..fa4b5816be 100644
>>> --- a/lib/librte_ethdev/rte_ethdev_version.map
>>> +++ b/lib/librte_ethdev/rte_ethdev_version.map
>>> @@ -230,4 +230,7 @@ EXPERIMENTAL {
>>>
>>> # added in 20.02
>>> rte_flow_dev_dump;
>>> +
>>> + # added in 20.05
>>> + rte_flow_get_aged_flows;
>>> };
>>> diff --git a/lib/librte_ethdev/rte_flow.c
>>> b/lib/librte_ethdev/rte_flow.c index a5ac1c7fbd..3699edce49 100644
>>> --- a/lib/librte_ethdev/rte_flow.c
>>> +++ b/lib/librte_ethdev/rte_flow.c
>>> @@ -172,6 +172,7 @@ static const struct rte_flow_desc_data
>> rte_flow_desc_action[] = {
>>> MK_FLOW_ACTION(SET_META, sizeof(struct
>> rte_flow_action_set_meta)),
>>> MK_FLOW_ACTION(SET_IPV4_DSCP, sizeof(struct
>> rte_flow_action_set_dscp)),
>>> MK_FLOW_ACTION(SET_IPV6_DSCP, sizeof(struct
>>> rte_flow_action_set_dscp)),
>>> + MK_FLOW_ACTION(AGE, sizeof(struct rte_flow_action_age)),
>>> };
>>>
>>> int
>>> @@ -1232,3 +1233,20 @@ rte_flow_dev_dump(uint16_t port_id, FILE *file,
>> struct rte_flow_error *error)
>>> RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
>>> NULL, rte_strerror(ENOSYS));
>>> }
>>> +
>>> +int
>>> +rte_flow_get_aged_flows(uint16_t port_id, void **contexts,
>>> + uint32_t nb_contexts, struct rte_flow_error *error) {
>>> + struct rte_eth_dev *dev = &rte_eth_devices[port_id];
>>> + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error);
>>> +
>>> + if (unlikely(!ops))
>>> + return -rte_errno;
>>> + if (likely(!!ops->get_aged_flows))
>>> + return flow_err(port_id, ops->get_aged_flows(dev,
>> contexts,
>>> + nb_contexts, error), error);
>>> + return rte_flow_error_set(error, ENOTSUP,
>>> + RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
>>> + NULL, rte_strerror(ENOTSUP));
>>> +}
>>> diff --git a/lib/librte_ethdev/rte_flow.h
>>> b/lib/librte_ethdev/rte_flow.h index 7f3e08fad3..fab44f6c0b 100644
>>> --- a/lib/librte_ethdev/rte_flow.h
>>> +++ b/lib/librte_ethdev/rte_flow.h
>>> @@ -2081,6 +2081,16 @@ enum rte_flow_action_type {
>>> * See struct rte_flow_action_set_dscp.
>>> */
>>> RTE_FLOW_ACTION_TYPE_SET_IPV6_DSCP,
>>> +
>>> + /**
>>> + * Report as aged flow if timeout passed without any matching on the
>>> + * flow.
>>> + *
>>> + * See struct rte_flow_action_age.
>>> + * See function rte_flow_get_aged_flows
>>> + * see enum RTE_ETH_EVENT_FLOW_AGED
>>> + */
>>> + RTE_FLOW_ACTION_TYPE_AGE,
>>> };
>>>
>>> /**
>>> @@ -2122,6 +2132,25 @@ struct rte_flow_action_queue {
>>> uint16_t index; /**< Queue index to use. */
>>> };
>>>
>>> +/**
>>> + * @warning
>>> + * @b EXPERIMENTAL: this structure may change without prior notice
>>> + *
>>> + * RTE_FLOW_ACTION_TYPE_AGE
>>> + *
>>> + * Report flow as aged-out if timeout passed without any matching
>>> + * on the flow. RTE_ETH_EVENT_FLOW_AGED event is triggered when a
>>> + * port detects new aged-out flows.
>>> + *
>>> + * The flow context and the flow handle will be reported by the
>>> + * rte_flow_get_aged_flows API.
>>> + */
>>> +struct rte_flow_action_age {
>>> + uint32_t timeout:24; /**< Time in seconds. */
>>> + uint32_t reserved:8; /**< Reserved, must be zero. */
>>> + void *context;
>>> + /**< The user flow context, NULL means the rte_flow
>> pointer. */ };
>>>
>>> /**
>>> * @warning
>>> @@ -3254,6 +3283,39 @@ rte_flow_conv(enum rte_flow_conv_op op,
>>> const void *src,
>>> struct rte_flow_error *error);
>>>
>>> +/**
>>> + * Get aged-out flows of a given port.
>>> + *
>>> + * RTE_ETH_EVENT_FLOW_AGED event will be triggered when at least
>> one
>>> +new aged
>>> + * out flow was detected after the last call to rte_flow_get_aged_flows.
>>> + * This function can be called to get the aged flows usynchronously
>>> +from the
>> usynchronously
>>> + * event callback or synchronously regardless the event.
>>> + * This is not safe to call rte_flow_get_aged_flows function with
>>> + other flow
>> It is not safe to
>>> + * functions from multiple threads simultaneously.
>>> + *
>>> + * @param port_id
>>> + * Port identifier of Ethernet device.
>>> + * @param[in, out] contexts
>>> + * The address of an array of pointers to the aged-out flows contexts.
>>> + * @param[in] nb_contexts
>>> + * The length of context array pointers.
>>> + * @param[out] error
>>> + * Perform verbose error reporting if not NULL. Initialized in case of
>>> + * error only.
>>> + *
>>> + * @return
>>> + * if nb_contexts is 0, return the amount of all aged contexts.
>>> + * if nb_contexts is not 0 , return the amount of aged flows reported
>>> + * in the context array, otherwise negative errno value.
>>> + *
>>> + * @see rte_flow_action_age
>>> + * @see RTE_ETH_EVENT_FLOW_AGED
>>> + */
>>> +__rte_experimental
>>> +int
>>> +rte_flow_get_aged_flows(uint16_t port_id, void **contexts,
>>> + uint32_t nb_contexts, struct rte_flow_error *error);
>>> +
>>> #ifdef __cplusplus
>>> }
>>> #endif
>>> diff --git a/lib/librte_ethdev/rte_flow_driver.h
>>> b/lib/librte_ethdev/rte_flow_driver.h
>>> index 51a9a57a0f..881cc469b7 100644
>>> --- a/lib/librte_ethdev/rte_flow_driver.h
>>> +++ b/lib/librte_ethdev/rte_flow_driver.h
>>> @@ -101,6 +101,12 @@ struct rte_flow_ops {
>>> (struct rte_eth_dev *dev,
>>> FILE *file,
>>> struct rte_flow_error *error);
>>> + /** See rte_flow_get_aged_flows() */
>>> + int (*get_aged_flows)
>>> + (struct rte_eth_dev *dev,
>>> + void **context,
>>> + uint32_t nb_contexts,
>>> + struct rte_flow_error *err);
>>> };
>>>
>>> /**
>>>
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH v4] ethdev: support flow aging
2020-04-29 14:50 0% ` Tom Barbette
@ 2020-04-30 7:36 0% ` Matan Azrad
2020-04-30 7:49 0% ` Tom Barbette
0 siblings, 1 reply; 200+ results
From: Matan Azrad @ 2020-04-30 7:36 UTC (permalink / raw)
To: Tom Barbette, Bill Zhou, Ori Kam, wenzhuo.lu, jingjing.wu,
bernard.iremonger, john.mcnamara, marko.kovacevic,
Thomas Monjalon, ferruh.yigit, arybchenko
Cc: dev
Hi Tom
From: Tom Barbette
> Great news!
>
> - I can understand why there is no timeout unit. But that's calling for user
> nightmare. Eg I could only get from the code (and not from documentation
> yet? ) of the following mlx5 driver patch that the value should be in tenth of
> seconds. If I build an application that is supposed to work with "any NIC",
> what can I do? We'd need a way to query the timeout unit (have it in
> dev_info probably).
Please see the new age action structure in rte_flow.h
You can see comments there that timeout units is in seconds....
> - It's not totally clear if the rule is automatically removed or not. is this a
> helper or an OpenFlow-like notification?
Only notification, the aged-out flow should be destroyed (or other action) by the application according to the application needs...
> - Find a typo and grammar fix inline.
> - Recently, Mellanox introduced the ability to create 330K flows/s. Any
> performance considerations if those flow "expire" at the same rate?
We didn't see performance impact (should be same rate like count action).
>
> Hope it's helpfull,
>
> Tom
>
> Le 21/04/2020 à 12:11, Bill Zhou a écrit :
> > From: Dong Zhou <dongz@mellanox.com>
> >
> > One of the reasons to destroy a flow is the fact that no packet
> > matches the flow for "timeout" time.
> > For example, when TCP\UDP sessions are suddenly closed.
> >
> > Currently, there is not any DPDK mechanism for flow aging and the
> > applications use their own ways to detect and destroy aged-out flows.
> >
> > The flow aging implementation need include:
> > - A new rte_flow action: RTE_FLOW_ACTION_TYPE_AGE to set the timeout
> and
> > the application flow context for each flow.
> > - A new ethdev event: RTE_ETH_EVENT_FLOW_AGED for the driver to
> report
> > that there are new aged-out flows.
> > - A new rte_flow API: rte_flow_get_aged_flows to get the aged-out flows
> > contexts from the port.
> > - Support input flow aging command line in Testpmd.
> >
> > The new event type addition in the enum is flagged as an ABI breakage,
> > so an ignore rule is added for these reasons:
> > - It is not changing value of existing types (except MAX)
> > - The new value is not used by existing API if the event is not
> > registered In general, it is safe adding new ethdev event types at the
> > end of the enum, because of event callback registration mechanism.
> >
> > Signed-off-by: Dong Zhou <dongz@mellanox.com>
> > ---
> > v2: Removing "* Added support for flow Aging mechanism base on
> counter."
> > this line from doc/guides/rel_notes/release_20_05.rst, this patch does
> > not include this support.
> >
> > v3: Update file libabigail.abignore, add one new suppressed
> > enumeration type for RTE_ETH_EVENT_MAX.
> >
> > v4: Add justification in devtools/libabigail.abignore and in the
> > commit log about the modification of v3.
> > ---
> > app/test-pmd/cmdline_flow.c | 26 ++++++++++
> > devtools/libabigail.abignore | 6 +++
> > doc/guides/prog_guide/rte_flow.rst | 22 +++++++++
> > doc/guides/rel_notes/release_20_05.rst | 11 +++++
> > lib/librte_ethdev/rte_ethdev.h | 1 +
> > lib/librte_ethdev/rte_ethdev_version.map | 3 ++
> > lib/librte_ethdev/rte_flow.c | 18 +++++++
> > lib/librte_ethdev/rte_flow.h | 62 ++++++++++++++++++++++++
> > lib/librte_ethdev/rte_flow_driver.h | 6 +++
> > 9 files changed, 155 insertions(+)
> >
> > diff --git a/app/test-pmd/cmdline_flow.c b/app/test-pmd/cmdline_flow.c
> > index e6ab8ff2f7..45bcff3cf5 100644
> > --- a/app/test-pmd/cmdline_flow.c
> > +++ b/app/test-pmd/cmdline_flow.c
> > @@ -343,6 +343,8 @@ enum index {
> > ACTION_SET_IPV4_DSCP_VALUE,
> > ACTION_SET_IPV6_DSCP,
> > ACTION_SET_IPV6_DSCP_VALUE,
> > + ACTION_AGE,
> > + ACTION_AGE_TIMEOUT,
> > };
> >
> > /** Maximum size for pattern in struct rte_flow_item_raw. */ @@
> > -1145,6 +1147,7 @@ static const enum index next_action[] = {
> > ACTION_SET_META,
> > ACTION_SET_IPV4_DSCP,
> > ACTION_SET_IPV6_DSCP,
> > + ACTION_AGE,
> > ZERO,
> > };
> >
> > @@ -1370,6 +1373,13 @@ static const enum index action_set_ipv6_dscp[]
> = {
> > ZERO,
> > };
> >
> > +static const enum index action_age[] = {
> > + ACTION_AGE,
> > + ACTION_AGE_TIMEOUT,
> > + ACTION_NEXT,
> > + ZERO,
> > +};
> > +
> > static int parse_set_raw_encap_decap(struct context *, const struct
> token *,
> > const char *, unsigned int,
> > void *, unsigned int);
> > @@ -3694,6 +3704,22 @@ static const struct token token_list[] = {
> > (struct rte_flow_action_set_dscp, dscp)),
> > .call = parse_vc_conf,
> > },
> > + [ACTION_AGE] = {
> > + .name = "age",
> > + .help = "set a specific metadata header",
> > + .next = NEXT(action_age),
> > + .priv = PRIV_ACTION(AGE,
> > + sizeof(struct rte_flow_action_age)),
> > + .call = parse_vc,
> > + },
> > + [ACTION_AGE_TIMEOUT] = {
> > + .name = "timeout",
> > + .help = "flow age timeout value",
> > + .args = ARGS(ARGS_ENTRY_BF(struct rte_flow_action_age,
> > + timeout, 24)),
> > + .next = NEXT(action_age, NEXT_ENTRY(UNSIGNED)),
> > + .call = parse_vc_conf,
> > + },
> > };
> >
> > /** Remove and return last entry from argument stack. */ diff --git
> > a/devtools/libabigail.abignore b/devtools/libabigail.abignore index
> > a59df8f135..c047adbd79 100644
> > --- a/devtools/libabigail.abignore
> > +++ b/devtools/libabigail.abignore
> > @@ -11,3 +11,9 @@
> > type_kind = enum
> > name = rte_crypto_asym_xform_type
> > changed_enumerators =
> RTE_CRYPTO_ASYM_XFORM_TYPE_LIST_END
> > +; Ignore ethdev event enum update because new event cannot be ;
> > +received if not registered [suppress_type]
> > + type_kind = enum
> > + name = rte_eth_event_type
> > + changed_enumerators = RTE_ETH_EVENT_MAX
> > diff --git a/doc/guides/prog_guide/rte_flow.rst
> > b/doc/guides/prog_guide/rte_flow.rst
> > index 41c147913c..cf4368e1c4 100644
> > --- a/doc/guides/prog_guide/rte_flow.rst
> > +++ b/doc/guides/prog_guide/rte_flow.rst
> > @@ -2616,6 +2616,28 @@ Otherwise, RTE_FLOW_ERROR_TYPE_ACTION
> error will be returned.
> > | ``dscp`` | DSCP in low 6 bits, rest ignore |
> > +-----------+---------------------------------+
> >
> > +Action: ``AGE``
> > +^^^^^^^^^^^^^^^^^^^^^^^^^
> > +
> > +Set ageing timeout configuration to a flow.
> > +
> > +Event RTE_ETH_EVENT_FLOW_AGED will be reported if timeout passed
> > +without any matching on the flow.
> > +
> > +.. _table_rte_flow_action_age:
> > +
> > +.. table:: AGE
> > +
> > + +--------------+---------------------------------+
> > + | Field | Value |
> > + +==============+=================================+
> > + | ``timeout`` | 24 bits timeout value |
> > + +--------------+---------------------------------+
> > + | ``reserved`` | 8 bits reserved, must be zero |
> > + +--------------+---------------------------------+
> > + | ``context`` | user input flow context |
> > + +--------------+---------------------------------+
> > +
> > Negative types
> > ~~~~~~~~~~~~~~
> >
> > diff --git a/doc/guides/rel_notes/release_20_05.rst
> > b/doc/guides/rel_notes/release_20_05.rst
> > index bacd4c65a2..ff0cf9f1d6 100644
> > --- a/doc/guides/rel_notes/release_20_05.rst
> > +++ b/doc/guides/rel_notes/release_20_05.rst
> > @@ -135,6 +135,17 @@ New Features
> > by making use of the event device capabilities. The event mode currently
> supports
> > only inline IPsec protocol offload.
> >
> > +* **Added flow Aging Support.**
> > +
> > + Added flow Aging support to detect and report aged-out flows,
> including:
> > +
> > + * Added new action: RTE_FLOW_ACTION_TYPE_AGE to set the timeout
> and the
> > + application flow context for each flow.
> > + * Added new event: RTE_ETH_EVENT_FLOW_AGED for the driver to
> report that
> > + there are new aged-out flows.
> > + * Added new API: rte_flow_get_aged_flows to get the aged-out flows
> contexts
> > + from the port.
> > +
> >
> > Removed Items
> > -------------
> > diff --git a/lib/librte_ethdev/rte_ethdev.h
> > b/lib/librte_ethdev/rte_ethdev.h index 8d69b88f9e..00cc7b4052 100644
> > --- a/lib/librte_ethdev/rte_ethdev.h
> > +++ b/lib/librte_ethdev/rte_ethdev.h
> > @@ -3018,6 +3018,7 @@ enum rte_eth_event_type {
> > RTE_ETH_EVENT_NEW, /**< port is probed */
> > RTE_ETH_EVENT_DESTROY, /**< port is released */
> > RTE_ETH_EVENT_IPSEC, /**< IPsec offload related event */
> > + RTE_ETH_EVENT_FLOW_AGED,/**< New aged-out flows is detected
> */
> > RTE_ETH_EVENT_MAX /**< max value of this enum */
> > };
> >
> > diff --git a/lib/librte_ethdev/rte_ethdev_version.map
> > b/lib/librte_ethdev/rte_ethdev_version.map
> > index 3f32fdecf7..fa4b5816be 100644
> > --- a/lib/librte_ethdev/rte_ethdev_version.map
> > +++ b/lib/librte_ethdev/rte_ethdev_version.map
> > @@ -230,4 +230,7 @@ EXPERIMENTAL {
> >
> > # added in 20.02
> > rte_flow_dev_dump;
> > +
> > + # added in 20.05
> > + rte_flow_get_aged_flows;
> > };
> > diff --git a/lib/librte_ethdev/rte_flow.c
> > b/lib/librte_ethdev/rte_flow.c index a5ac1c7fbd..3699edce49 100644
> > --- a/lib/librte_ethdev/rte_flow.c
> > +++ b/lib/librte_ethdev/rte_flow.c
> > @@ -172,6 +172,7 @@ static const struct rte_flow_desc_data
> rte_flow_desc_action[] = {
> > MK_FLOW_ACTION(SET_META, sizeof(struct
> rte_flow_action_set_meta)),
> > MK_FLOW_ACTION(SET_IPV4_DSCP, sizeof(struct
> rte_flow_action_set_dscp)),
> > MK_FLOW_ACTION(SET_IPV6_DSCP, sizeof(struct
> > rte_flow_action_set_dscp)),
> > + MK_FLOW_ACTION(AGE, sizeof(struct rte_flow_action_age)),
> > };
> >
> > int
> > @@ -1232,3 +1233,20 @@ rte_flow_dev_dump(uint16_t port_id, FILE *file,
> struct rte_flow_error *error)
> > RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
> > NULL, rte_strerror(ENOSYS));
> > }
> > +
> > +int
> > +rte_flow_get_aged_flows(uint16_t port_id, void **contexts,
> > + uint32_t nb_contexts, struct rte_flow_error *error) {
> > + struct rte_eth_dev *dev = &rte_eth_devices[port_id];
> > + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error);
> > +
> > + if (unlikely(!ops))
> > + return -rte_errno;
> > + if (likely(!!ops->get_aged_flows))
> > + return flow_err(port_id, ops->get_aged_flows(dev,
> contexts,
> > + nb_contexts, error), error);
> > + return rte_flow_error_set(error, ENOTSUP,
> > + RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
> > + NULL, rte_strerror(ENOTSUP));
> > +}
> > diff --git a/lib/librte_ethdev/rte_flow.h
> > b/lib/librte_ethdev/rte_flow.h index 7f3e08fad3..fab44f6c0b 100644
> > --- a/lib/librte_ethdev/rte_flow.h
> > +++ b/lib/librte_ethdev/rte_flow.h
> > @@ -2081,6 +2081,16 @@ enum rte_flow_action_type {
> > * See struct rte_flow_action_set_dscp.
> > */
> > RTE_FLOW_ACTION_TYPE_SET_IPV6_DSCP,
> > +
> > + /**
> > + * Report as aged flow if timeout passed without any matching on the
> > + * flow.
> > + *
> > + * See struct rte_flow_action_age.
> > + * See function rte_flow_get_aged_flows
> > + * see enum RTE_ETH_EVENT_FLOW_AGED
> > + */
> > + RTE_FLOW_ACTION_TYPE_AGE,
> > };
> >
> > /**
> > @@ -2122,6 +2132,25 @@ struct rte_flow_action_queue {
> > uint16_t index; /**< Queue index to use. */
> > };
> >
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this structure may change without prior notice
> > + *
> > + * RTE_FLOW_ACTION_TYPE_AGE
> > + *
> > + * Report flow as aged-out if timeout passed without any matching
> > + * on the flow. RTE_ETH_EVENT_FLOW_AGED event is triggered when a
> > + * port detects new aged-out flows.
> > + *
> > + * The flow context and the flow handle will be reported by the
> > + * rte_flow_get_aged_flows API.
> > + */
> > +struct rte_flow_action_age {
> > + uint32_t timeout:24; /**< Time in seconds. */
> > + uint32_t reserved:8; /**< Reserved, must be zero. */
> > + void *context;
> > + /**< The user flow context, NULL means the rte_flow
> pointer. */ };
> >
> > /**
> > * @warning
> > @@ -3254,6 +3283,39 @@ rte_flow_conv(enum rte_flow_conv_op op,
> > const void *src,
> > struct rte_flow_error *error);
> >
> > +/**
> > + * Get aged-out flows of a given port.
> > + *
> > + * RTE_ETH_EVENT_FLOW_AGED event will be triggered when at least
> one
> > +new aged
> > + * out flow was detected after the last call to rte_flow_get_aged_flows.
> > + * This function can be called to get the aged flows usynchronously
> > +from the
> usynchronously
> > + * event callback or synchronously regardless the event.
> > + * This is not safe to call rte_flow_get_aged_flows function with
> > + other flow
> It is not safe to
> > + * functions from multiple threads simultaneously.
> > + *
> > + * @param port_id
> > + * Port identifier of Ethernet device.
> > + * @param[in, out] contexts
> > + * The address of an array of pointers to the aged-out flows contexts.
> > + * @param[in] nb_contexts
> > + * The length of context array pointers.
> > + * @param[out] error
> > + * Perform verbose error reporting if not NULL. Initialized in case of
> > + * error only.
> > + *
> > + * @return
> > + * if nb_contexts is 0, return the amount of all aged contexts.
> > + * if nb_contexts is not 0 , return the amount of aged flows reported
> > + * in the context array, otherwise negative errno value.
> > + *
> > + * @see rte_flow_action_age
> > + * @see RTE_ETH_EVENT_FLOW_AGED
> > + */
> > +__rte_experimental
> > +int
> > +rte_flow_get_aged_flows(uint16_t port_id, void **contexts,
> > + uint32_t nb_contexts, struct rte_flow_error *error);
> > +
> > #ifdef __cplusplus
> > }
> > #endif
> > diff --git a/lib/librte_ethdev/rte_flow_driver.h
> > b/lib/librte_ethdev/rte_flow_driver.h
> > index 51a9a57a0f..881cc469b7 100644
> > --- a/lib/librte_ethdev/rte_flow_driver.h
> > +++ b/lib/librte_ethdev/rte_flow_driver.h
> > @@ -101,6 +101,12 @@ struct rte_flow_ops {
> > (struct rte_eth_dev *dev,
> > FILE *file,
> > struct rte_flow_error *error);
> > + /** See rte_flow_get_aged_flows() */
> > + int (*get_aged_flows)
> > + (struct rte_eth_dev *dev,
> > + void **context,
> > + uint32_t nb_contexts,
> > + struct rte_flow_error *err);
> > };
> >
> > /**
> >
^ permalink raw reply [relevance 0%]
* [dpdk-dev] [PATCH v1 2/2] devtools: updating internal symbols ABI version
2020-04-30 5:46 33% [dpdk-dev] [PATCH v1 1/2] devtools: add internal ABI version check Haiyue Wang
@ 2020-04-30 5:46 4% ` Haiyue Wang
0 siblings, 0 replies; 200+ results
From: Haiyue Wang @ 2020-04-30 5:46 UTC (permalink / raw)
To: dev, thomas, david.marchand, bruce.richardson, anatoly.burakov,
nhorman, mdr
Cc: Haiyue Wang
INTERNAL is new introduced version, update the script that automatically
leaving internal section exactly as it is.
Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
---
devtools/update_version_map_abi.py | 37 +++++++++++++++++++++++++++---
1 file changed, 34 insertions(+), 3 deletions(-)
diff --git a/devtools/update_version_map_abi.py b/devtools/update_version_map_abi.py
index 616412a1c..e2104e61e 100755
--- a/devtools/update_version_map_abi.py
+++ b/devtools/update_version_map_abi.py
@@ -50,7 +50,10 @@ def __parse_map_file(f_in):
stable_lines = set()
# copy experimental section as is
experimental_lines = []
+ # copy internal section as is
+ internal_lines = []
in_experimental = False
+ in_internal = False
has_stable = False
# gather all functions
@@ -63,6 +66,7 @@ def __parse_map_file(f_in):
if match:
# whatever section this was, it's not active any more
in_experimental = False
+ in_internal = False
continue
# if we're in the middle of experimental section, we need to copy
@@ -71,6 +75,12 @@ def __parse_map_file(f_in):
experimental_lines += [line]
continue
+ # if we're in the middle of internal section, we need to copy
+ # the section verbatim, so just add the line
+ if in_internal:
+ internal_lines += [line]
+ continue
+
# skip empty lines
if not line:
continue
@@ -81,7 +91,9 @@ def __parse_map_file(f_in):
cur_section = match.group("version")
# is it experimental?
in_experimental = cur_section == "EXPERIMENTAL"
- if not in_experimental:
+ # is it internal?
+ in_internal = cur_section == "INTERNAL"
+ if not in_experimental and not in_internal:
has_stable = True
continue
@@ -90,7 +102,7 @@ def __parse_map_file(f_in):
if match:
stable_lines.add(match.group("func"))
- return has_stable, stable_lines, experimental_lines
+ return has_stable, stable_lines, experimental_lines, internal_lines
def __generate_stable_abi(f_out, abi_version, lines):
@@ -132,6 +144,20 @@ def __generate_experimental_abi(f_out, lines):
# end section
print("};", file=f_out)
+def __generate_internal_abi(f_out, lines):
+ # start internal section
+ print("INTERNAL {", file=f_out)
+
+ # print all internal lines as they were
+ for line in lines:
+ # don't print empty whitespace
+ if not line:
+ print("", file=f_out)
+ else:
+ print("\t{}".format(line), file=f_out)
+
+ # end section
+ print("};", file=f_out)
def __main():
arg_parser = argparse.ArgumentParser(
@@ -158,7 +184,7 @@ def __main():
sys.exit(1)
with open(parsed.map_file) as f_in:
- has_stable, stable_lines, experimental_lines = __parse_map_file(f_in)
+ has_stable, stable_lines, experimental_lines, internal_lines = __parse_map_file(f_in)
with open(parsed.map_file, 'w') as f_out:
need_newline = has_stable and experimental_lines
@@ -169,6 +195,11 @@ def __main():
print(file=f_out)
if experimental_lines:
__generate_experimental_abi(f_out, experimental_lines)
+ if internal_lines:
+ if has_stable or experimental_lines:
+ # separate sections with a newline
+ print(file=f_out)
+ __generate_internal_abi(f_out, internal_lines)
if __name__ == "__main__":
--
2.26.2
^ permalink raw reply [relevance 4%]
* [dpdk-dev] [PATCH v1 1/2] devtools: add internal ABI version check
@ 2020-04-30 5:46 33% Haiyue Wang
2020-04-30 5:46 4% ` [dpdk-dev] [PATCH v1 2/2] devtools: updating internal symbols ABI version Haiyue Wang
0 siblings, 1 reply; 200+ results
From: Haiyue Wang @ 2020-04-30 5:46 UTC (permalink / raw)
To: dev, thomas, david.marchand, bruce.richardson, anatoly.burakov,
nhorman, mdr
Cc: Haiyue Wang
INTERNAL is new introduced version, update the shell script that checks
whether built libraries are versioned with expected ABI (current ABI,
current ABI + 1, EXPERIMENTAL, or INTERNAL).
Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
---
devtools/check-abi-version.sh | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/devtools/check-abi-version.sh b/devtools/check-abi-version.sh
index 9a3d13546..f0cca42a9 100755
--- a/devtools/check-abi-version.sh
+++ b/devtools/check-abi-version.sh
@@ -4,7 +4,7 @@
# Check whether library symbols have correct
# version (provided ABI number or provided ABI
-# number + 1 or EXPERIMENTAL).
+# number + 1 or EXPERIMENTAL or INTERNAL).
# Args:
# $1: path of the library .so file
# $2: ABI major version number to check
@@ -12,7 +12,7 @@
if [ -z "$1" ]; then
echo "Script checks whether library symbols have"
- echo "correct version (ABI_VER/ABI_VER+1/EXPERIMENTAL)"
+ echo "correct version (ABI_VER/ABI_VER+1/EXPERIMENTAL/INTERNAL)"
echo "Usage:"
echo " $0 SO_FILE_PATH [ABI_VER]"
exit 1
@@ -41,11 +41,11 @@ for SYM in $(echo "${OBJ_DUMP_OUTPUT}" | awk '{print $(NF-1) "-" $NF}')
do
version=$(echo $SYM | cut -d'-' -f 1)
symbol=$(echo $SYM | cut -d'-' -f 2)
- case $version in (*"$ABIVER"*|*"$NEXT_ABIVER"*|"EXPERIMENTAL")
+ case $version in (*"$ABIVER"*|*"$NEXT_ABIVER"*|"EXPERIMENTAL"|"INTERNAL")
;;
(*)
echo "Warning: symbol $symbol ($version) should be annotated " \
- "as ABI version $ABIVER / $NEXT_ABIVER, or EXPERIMENTAL."
+ "as ABI version $ABIVER / $NEXT_ABIVER, EXPERIMENTAL, or INTERNAL."
ret=1
;;
esac
--
2.26.2
^ permalink raw reply [relevance 33%]
* [dpdk-dev] [RFC] hash: unify crc32 API header for x86 and ARM
@ 2020-04-29 18:05 2% pbhagavatula
2020-04-30 9:14 0% ` Van Haaren, Harry
0 siblings, 1 reply; 200+ results
From: pbhagavatula @ 2020-04-29 18:05 UTC (permalink / raw)
To: jerinj, thomas, Yipeng Wang, Sameh Gobriel, Bruce Richardson,
Ruifeng Wang
Cc: dev, Pavan Nikhilesh
From: Pavan Nikhilesh <pbhagavatula@marvell.com>
Merge crc32 hash calculation public API headers for x86 and ARM,
split implementations of x86 and ARM into their respective private
headers.
This reduces the ifdef code clutter while keeping current ABI intact.
Although we install `rte_crc_arm64.h` it is not used in any of the lib or
drivers layers. All the libs and drivers use `rte_hash_crc.h` which falls
back to SW crc32 calculation for ARM platform.
Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
---
Currently, if application incorrectly sets CRC32_ARM64 as crc32 algorithm
through `rte_hash_crc_set_alg()` on x86 or vice-versa we fallback to algorithm
set previously via `rte_hash_crc_set_alg()` instead of setting the best
available.
This behaviour should probably change to setting the best available algorithm
and is up for discussion.
app/test/test_hash.c | 6 +
lib/librte_hash/Makefile | 5 -
lib/librte_hash/crc_arm64.h | 67 +++++++++++
lib/librte_hash/crc_x86.h | 68 +++++++++++
lib/librte_hash/meson.build | 3 +-
lib/librte_hash/rte_crc_arm64.h | 183 ------------------------------
lib/librte_hash/rte_hash_crc.h | 193 +++++++++++++-------------------
7 files changed, 219 insertions(+), 306 deletions(-)
create mode 100644 lib/librte_hash/crc_arm64.h
create mode 100644 lib/librte_hash/crc_x86.h
delete mode 100644 lib/librte_hash/rte_crc_arm64.h
diff --git a/app/test/test_hash.c b/app/test/test_hash.c
index afa3a1a3c..7bd457dac 100644
--- a/app/test/test_hash.c
+++ b/app/test/test_hash.c
@@ -195,7 +195,13 @@ test_crc32_hash_alg_equiv(void)
}
/* Resetting to best available algorithm */
+#if defined RTE_ARCH_X86
rte_hash_crc_set_alg(CRC32_SSE42_x64);
+#elif defined RTE_ARCH_ARM64
+ rte_hash_crc_set_alg(CRC32_ARM64);
+#else
+ rte_hash_crc_set_alg(CRC32_SW);
+#endif
if (i == CRC32_ITERATIONS)
return 0;
diff --git a/lib/librte_hash/Makefile b/lib/librte_hash/Makefile
index ec9f86499..f640afc42 100644
--- a/lib/librte_hash/Makefile
+++ b/lib/librte_hash/Makefile
@@ -19,11 +19,6 @@ SRCS-$(CONFIG_RTE_LIBRTE_HASH) += rte_fbk_hash.c
# install this header file
SYMLINK-$(CONFIG_RTE_LIBRTE_HASH)-include := rte_hash.h
SYMLINK-$(CONFIG_RTE_LIBRTE_HASH)-include += rte_hash_crc.h
-ifeq ($(CONFIG_RTE_ARCH_ARM64),y)
-ifneq ($(findstring RTE_MACHINE_CPUFLAG_CRC32,$(CFLAGS)),)
-SYMLINK-$(CONFIG_RTE_LIBRTE_HASH)-include += rte_crc_arm64.h
-endif
-endif
SYMLINK-$(CONFIG_RTE_LIBRTE_HASH)-include += rte_jhash.h
SYMLINK-$(CONFIG_RTE_LIBRTE_HASH)-include += rte_thash.h
SYMLINK-$(CONFIG_RTE_LIBRTE_HASH)-include += rte_fbk_hash.h
diff --git a/lib/librte_hash/crc_arm64.h b/lib/librte_hash/crc_arm64.h
new file mode 100644
index 000000000..8e75f8297
--- /dev/null
+++ b/lib/librte_hash/crc_arm64.h
@@ -0,0 +1,67 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2015 Cavium, Inc
+ */
+
+#ifndef _CRC_ARM64_H_
+#define _CRC_ARM64_H_
+
+/**
+ * @file
+ *
+ * CRC arm64 Hash
+ */
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include <stdint.h>
+#include <rte_cpuflags.h>
+#include <rte_branch_prediction.h>
+#include <rte_common.h>
+
+static inline uint32_t
+crc32c_arm64_u8(uint8_t data, uint32_t init_val)
+{
+ __asm__ volatile(
+ "crc32cb %w[crc], %w[crc], %w[value]"
+ : [crc] "+r" (init_val)
+ : [value] "r" (data));
+ return init_val;
+}
+
+static inline uint32_t
+crc32c_arm64_u16(uint16_t data, uint32_t init_val)
+{
+ __asm__ volatile(
+ "crc32ch %w[crc], %w[crc], %w[value]"
+ : [crc] "+r" (init_val)
+ : [value] "r" (data));
+ return init_val;
+}
+
+static inline uint32_t
+crc32c_arm64_u32(uint32_t data, uint32_t init_val)
+{
+ __asm__ volatile(
+ "crc32cw %w[crc], %w[crc], %w[value]"
+ : [crc] "+r" (init_val)
+ : [value] "r" (data));
+ return init_val;
+}
+
+static inline uint32_t
+crc32c_arm64_u64(uint64_t data, uint32_t init_val)
+{
+ __asm__ volatile(
+ "crc32cx %w[crc], %w[crc], %x[value]"
+ : [crc] "+r" (init_val)
+ : [value] "r" (data));
+ return init_val;
+}
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* _CRC_ARM64_H_ */
diff --git a/lib/librte_hash/crc_x86.h b/lib/librte_hash/crc_x86.h
new file mode 100644
index 000000000..fb45f2e98
--- /dev/null
+++ b/lib/librte_hash/crc_x86.h
@@ -0,0 +1,68 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2010-2014 Intel Corporation
+ */
+
+#ifndef _CRC_X86_H_
+#define _CRC_X86_H_
+
+#if defined(RTE_ARCH_X86)
+static inline uint32_t
+crc32c_sse42_u8(uint8_t data, uint32_t init_val)
+{
+ __asm__ volatile(
+ "crc32b %[data], %[init_val];"
+ : [init_val] "+r" (init_val)
+ : [data] "rm" (data));
+ return init_val;
+}
+
+static inline uint32_t
+crc32c_sse42_u16(uint16_t data, uint32_t init_val)
+{
+ __asm__ volatile(
+ "crc32w %[data], %[init_val];"
+ : [init_val] "+r" (init_val)
+ : [data] "rm" (data));
+ return init_val;
+}
+
+static inline uint32_t
+crc32c_sse42_u32(uint32_t data, uint32_t init_val)
+{
+ __asm__ volatile(
+ "crc32l %[data], %[init_val];"
+ : [init_val] "+r" (init_val)
+ : [data] "rm" (data));
+ return init_val;
+}
+
+static inline uint32_t
+crc32c_sse42_u64_mimic(uint64_t data, uint32_t init_val)
+{
+ union {
+ uint32_t u32[2];
+ uint64_t u64;
+ } d;
+
+ d.u64 = data;
+ init_val = crc32c_sse42_u32(d.u32[0], init_val);
+ init_val = crc32c_sse42_u32(d.u32[1], init_val);
+ return init_val;
+}
+#endif
+
+#ifdef RTE_ARCH_X86_64
+static inline uint32_t
+crc32c_sse42_u64(uint64_t data, uint32_t init_val)
+{
+ uint64_t val = init_val;
+
+ __asm__ volatile(
+ "crc32q %[data], %[init_val];"
+ : [init_val] "+r" (val)
+ : [data] "rm" (data));
+ return (uint32_t)val;
+}
+#endif
+
+#endif
diff --git a/lib/librte_hash/meson.build b/lib/librte_hash/meson.build
index 6ab46ae9d..90a180bc8 100644
--- a/lib/librte_hash/meson.build
+++ b/lib/librte_hash/meson.build
@@ -1,8 +1,7 @@
# SPDX-License-Identifier: BSD-3-Clause
# Copyright(c) 2017 Intel Corporation
-headers = files('rte_crc_arm64.h',
- 'rte_fbk_hash.h',
+headers = files('rte_fbk_hash.h',
'rte_hash_crc.h',
'rte_hash.h',
'rte_jhash.h',
diff --git a/lib/librte_hash/rte_crc_arm64.h b/lib/librte_hash/rte_crc_arm64.h
deleted file mode 100644
index b4628cfc0..000000000
--- a/lib/librte_hash/rte_crc_arm64.h
+++ /dev/null
@@ -1,183 +0,0 @@
-/* SPDX-License-Identifier: BSD-3-Clause
- * Copyright(c) 2015 Cavium, Inc
- */
-
-#ifndef _RTE_CRC_ARM64_H_
-#define _RTE_CRC_ARM64_H_
-
-/**
- * @file
- *
- * RTE CRC arm64 Hash
- */
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-#include <stdint.h>
-#include <rte_cpuflags.h>
-#include <rte_branch_prediction.h>
-#include <rte_common.h>
-
-static inline uint32_t
-crc32c_arm64_u8(uint8_t data, uint32_t init_val)
-{
- __asm__ volatile(
- "crc32cb %w[crc], %w[crc], %w[value]"
- : [crc] "+r" (init_val)
- : [value] "r" (data));
- return init_val;
-}
-
-static inline uint32_t
-crc32c_arm64_u16(uint16_t data, uint32_t init_val)
-{
- __asm__ volatile(
- "crc32ch %w[crc], %w[crc], %w[value]"
- : [crc] "+r" (init_val)
- : [value] "r" (data));
- return init_val;
-}
-
-static inline uint32_t
-crc32c_arm64_u32(uint32_t data, uint32_t init_val)
-{
- __asm__ volatile(
- "crc32cw %w[crc], %w[crc], %w[value]"
- : [crc] "+r" (init_val)
- : [value] "r" (data));
- return init_val;
-}
-
-static inline uint32_t
-crc32c_arm64_u64(uint64_t data, uint32_t init_val)
-{
- __asm__ volatile(
- "crc32cx %w[crc], %w[crc], %x[value]"
- : [crc] "+r" (init_val)
- : [value] "r" (data));
- return init_val;
-}
-
-/**
- * Allow or disallow use of arm64 SIMD instrinsics for CRC32 hash
- * calculation.
- *
- * @param alg
- * An OR of following flags:
- * - (CRC32_SW) Don't use arm64 crc intrinsics
- * - (CRC32_ARM64) Use ARMv8 CRC intrinsic if available
- *
- */
-static inline void
-rte_hash_crc_set_alg(uint8_t alg)
-{
- switch (alg) {
- case CRC32_ARM64:
- if (!rte_cpu_get_flag_enabled(RTE_CPUFLAG_CRC32))
- alg = CRC32_SW;
- /* fall-through */
- case CRC32_SW:
- crc32_alg = alg;
- /* fall-through */
- default:
- break;
- }
-}
-
-/* Setting the best available algorithm */
-RTE_INIT(rte_hash_crc_init_alg)
-{
- rte_hash_crc_set_alg(CRC32_ARM64);
-}
-
-/**
- * Use single crc32 instruction to perform a hash on a 1 byte value.
- * Fall back to software crc32 implementation in case arm64 crc intrinsics is
- * not supported
- *
- * @param data
- * Data to perform hash on.
- * @param init_val
- * Value to initialise hash generator.
- * @return
- * 32bit calculated hash value.
- */
-static inline uint32_t
-rte_hash_crc_1byte(uint8_t data, uint32_t init_val)
-{
- if (likely(crc32_alg & CRC32_ARM64))
- return crc32c_arm64_u8(data, init_val);
-
- return crc32c_1byte(data, init_val);
-}
-
-/**
- * Use single crc32 instruction to perform a hash on a 2 bytes value.
- * Fall back to software crc32 implementation in case arm64 crc intrinsics is
- * not supported
- *
- * @param data
- * Data to perform hash on.
- * @param init_val
- * Value to initialise hash generator.
- * @return
- * 32bit calculated hash value.
- */
-static inline uint32_t
-rte_hash_crc_2byte(uint16_t data, uint32_t init_val)
-{
- if (likely(crc32_alg & CRC32_ARM64))
- return crc32c_arm64_u16(data, init_val);
-
- return crc32c_2bytes(data, init_val);
-}
-
-/**
- * Use single crc32 instruction to perform a hash on a 4 byte value.
- * Fall back to software crc32 implementation in case arm64 crc intrinsics is
- * not supported
- *
- * @param data
- * Data to perform hash on.
- * @param init_val
- * Value to initialise hash generator.
- * @return
- * 32bit calculated hash value.
- */
-static inline uint32_t
-rte_hash_crc_4byte(uint32_t data, uint32_t init_val)
-{
- if (likely(crc32_alg & CRC32_ARM64))
- return crc32c_arm64_u32(data, init_val);
-
- return crc32c_1word(data, init_val);
-}
-
-/**
- * Use single crc32 instruction to perform a hash on a 8 byte value.
- * Fall back to software crc32 implementation in case arm64 crc intrinsics is
- * not supported
- *
- * @param data
- * Data to perform hash on.
- * @param init_val
- * Value to initialise hash generator.
- * @return
- * 32bit calculated hash value.
- */
-static inline uint32_t
-rte_hash_crc_8byte(uint64_t data, uint32_t init_val)
-{
- if (likely(crc32_alg == CRC32_ARM64))
- return crc32c_arm64_u64(data, init_val);
-
- return crc32c_2words(data, init_val);
-}
-
-#ifdef __cplusplus
-}
-#endif
-
-#endif /* _RTE_CRC_ARM64_H_ */
diff --git a/lib/librte_hash/rte_hash_crc.h b/lib/librte_hash/rte_hash_crc.h
index cf28031b3..292697db1 100644
--- a/lib/librte_hash/rte_hash_crc.h
+++ b/lib/librte_hash/rte_hash_crc.h
@@ -21,6 +21,15 @@ extern "C" {
#include <rte_branch_prediction.h>
#include <rte_common.h>
+typedef uint32_t
+(*crc32_handler_1b)(uint8_t data, uint32_t init_val);
+typedef uint32_t
+(*crc32_handler_2b)(uint16_t data, uint32_t init_val);
+typedef uint32_t
+(*crc32_handler_4b)(uint32_t data, uint32_t init_val);
+typedef uint32_t
+(*crc32_handler_8b)(uint64_t data, uint32_t init_val);
+
/* Lookup tables for software implementation of CRC32C */
static const uint32_t crc32c_tables[8][256] = {{
0x00000000, 0xF26B8303, 0xE13B70F7, 0x1350F3F4, 0xC79A971F, 0x35F1141C, 0x26A1E7E8, 0xD4CA64EB,
@@ -322,7 +331,7 @@ crc32c_2bytes(uint16_t data, uint32_t init_val)
}
static inline uint32_t
-crc32c_1word(uint32_t data, uint32_t init_val)
+crc32c_4bytes(uint32_t data, uint32_t init_val)
{
uint32_t crc, term1, term2;
crc = init_val;
@@ -336,7 +345,7 @@ crc32c_1word(uint32_t data, uint32_t init_val)
}
static inline uint32_t
-crc32c_2words(uint64_t data, uint32_t init_val)
+crc32c_8bytes(uint64_t data, uint32_t init_val)
{
uint32_t crc, term1, term2;
union {
@@ -357,109 +366,94 @@ crc32c_2words(uint64_t data, uint32_t init_val)
return crc;
}
-
-#if defined(RTE_ARCH_X86)
-static inline uint32_t
-crc32c_sse42_u8(uint8_t data, uint32_t init_val)
-{
- __asm__ volatile(
- "crc32b %[data], %[init_val];"
- : [init_val] "+r" (init_val)
- : [data] "rm" (data));
- return init_val;
-}
-
-static inline uint32_t
-crc32c_sse42_u16(uint16_t data, uint32_t init_val)
-{
- __asm__ volatile(
- "crc32w %[data], %[init_val];"
- : [init_val] "+r" (init_val)
- : [data] "rm" (data));
- return init_val;
-}
-
-static inline uint32_t
-crc32c_sse42_u32(uint32_t data, uint32_t init_val)
-{
- __asm__ volatile(
- "crc32l %[data], %[init_val];"
- : [init_val] "+r" (init_val)
- : [data] "rm" (data));
- return init_val;
-}
-
-static inline uint32_t
-crc32c_sse42_u64_mimic(uint64_t data, uint64_t init_val)
-{
- union {
- uint32_t u32[2];
- uint64_t u64;
- } d;
-
- d.u64 = data;
- init_val = crc32c_sse42_u32(d.u32[0], (uint32_t)init_val);
- init_val = crc32c_sse42_u32(d.u32[1], (uint32_t)init_val);
- return (uint32_t)init_val;
-}
-#endif
-
-#ifdef RTE_ARCH_X86_64
-static inline uint32_t
-crc32c_sse42_u64(uint64_t data, uint64_t init_val)
-{
- __asm__ volatile(
- "crc32q %[data], %[init_val];"
- : [init_val] "+r" (init_val)
- : [data] "rm" (data));
- return (uint32_t)init_val;
-}
-#endif
-
#define CRC32_SW (1U << 0)
#define CRC32_SSE42 (1U << 1)
#define CRC32_x64 (1U << 2)
#define CRC32_SSE42_x64 (CRC32_x64|CRC32_SSE42)
#define CRC32_ARM64 (1U << 3)
-static uint8_t crc32_alg = CRC32_SW;
+static crc32_handler_1b crc32_1b = crc32c_1byte;
+static crc32_handler_2b crc32_2b = crc32c_2bytes;
+static crc32_handler_4b crc32_4b = crc32c_4bytes;
+static crc32_handler_8b crc32_8b = crc32c_8bytes;
#if defined(RTE_ARCH_ARM64) && defined(RTE_MACHINE_CPUFLAG_CRC32)
-#include "rte_crc_arm64.h"
-#else
+#include "crc_arm64.h"
+#endif
+
+#if defined(RTE_ARCH_X86)
+#include "crc_x86.h"
+#endif
/**
- * Allow or disallow use of SSE4.2 instrinsics for CRC32 hash
+ * Allow or disallow use of SSE4.2/ARMv8 instrinsics for CRC32 hash
* calculation.
*
* @param alg
* An OR of following flags:
- * - (CRC32_SW) Don't use SSE4.2 intrinsics
+ * - (CRC32_SW) Don't use SSE4.2 intrinsics (default non-[x86/ARMv8])
* - (CRC32_SSE42) Use SSE4.2 intrinsics if available
- * - (CRC32_SSE42_x64) Use 64-bit SSE4.2 intrinsic if available (default)
- *
+ * - (CRC32_SSE42_x64) Use 64-bit SSE4.2 intrinsic if available (default x86)
+ * - (CRC32_ARM64) Use ARMv8 CRC intrinsic if available
*/
static inline void
rte_hash_crc_set_alg(uint8_t alg)
{
-#if defined(RTE_ARCH_X86)
- if (alg == CRC32_SSE42_x64 &&
- !rte_cpu_get_flag_enabled(RTE_CPUFLAG_EM64T))
- alg = CRC32_SSE42;
+ switch (alg) {
+ case CRC32_SSE42_x64:
+#if defined RTE_ARCH_X86
+ crc32_1b = crc32c_sse42_u8;
+ crc32_2b = crc32c_sse42_u16;
+ crc32_4b = crc32c_sse42_u32;
+
+ if (!rte_cpu_get_flag_enabled(RTE_CPUFLAG_EM64T))
+ crc32_8b = crc32c_sse42_u64_mimic;
+ else
+ crc32_8b = crc32c_sse42_u64;
#endif
- crc32_alg = alg;
+ break;
+ case CRC32_SSE42:
+#if defined RTE_ARCH_X86
+ crc32_1b = crc32c_sse42_u8;
+ crc32_2b = crc32c_sse42_u16;
+ crc32_4b = crc32c_sse42_u32;
+ crc32_8b = crc32c_sse42_u64_mimic;
+#endif
+ break;
+ case CRC32_ARM64:
+#if defined RTE_ARCH_ARM64
+ if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_CRC32)) {
+ crc32_1b = crc32c_arm64_u8;
+ crc32_2b = crc32c_arm64_u16;
+ crc32_4b = crc32c_arm64_u32;
+ crc32_8b = crc32c_arm64_u64;
+ }
+#endif
+ break;
+ case CRC32_SW:
+ default:
+ crc32_1b = crc32c_1byte;
+ crc32_2b = crc32c_2bytes;
+ crc32_4b = crc32c_4bytes;
+ crc32_8b = crc32c_8bytes;
+ break;
+ }
}
/* Setting the best available algorithm */
RTE_INIT(rte_hash_crc_init_alg)
{
+#if defined RTE_ARCH_X86
rte_hash_crc_set_alg(CRC32_SSE42_x64);
+#elif defined RTE_ARCH_ARM64
+ rte_hash_crc_set_alg(CRC32_ARM64);
+#else
+ rte_hash_crc_set_alg(CRC32_SW);
+#endif
}
/**
- * Use single crc32 instruction to perform a hash on a byte value.
- * Fall back to software crc32 implementation in case SSE4.2 is
- * not supported
+ * Calculate crc32 hash value of 1bytes.
*
* @param data
* Data to perform hash on.
@@ -471,18 +465,11 @@ RTE_INIT(rte_hash_crc_init_alg)
static inline uint32_t
rte_hash_crc_1byte(uint8_t data, uint32_t init_val)
{
-#if defined RTE_ARCH_X86
- if (likely(crc32_alg & CRC32_SSE42))
- return crc32c_sse42_u8(data, init_val);
-#endif
-
- return crc32c_1byte(data, init_val);
+ return (crc32_1b)(data, init_val);
}
/**
- * Use single crc32 instruction to perform a hash on a 2 bytes value.
- * Fall back to software crc32 implementation in case SSE4.2 is
- * not supported
+ * Calculate crc32 hash value of 2bytes.
*
* @param data
* Data to perform hash on.
@@ -494,18 +481,11 @@ rte_hash_crc_1byte(uint8_t data, uint32_t init_val)
static inline uint32_t
rte_hash_crc_2byte(uint16_t data, uint32_t init_val)
{
-#if defined RTE_ARCH_X86
- if (likely(crc32_alg & CRC32_SSE42))
- return crc32c_sse42_u16(data, init_val);
-#endif
-
- return crc32c_2bytes(data, init_val);
+ return (crc32_2b)(data, init_val);
}
/**
- * Use single crc32 instruction to perform a hash on a 4 byte value.
- * Fall back to software crc32 implementation in case SSE4.2 is
- * not supported
+ * Calculate crc32 hash value of 4bytes.
*
* @param data
* Data to perform hash on.
@@ -517,18 +497,11 @@ rte_hash_crc_2byte(uint16_t data, uint32_t init_val)
static inline uint32_t
rte_hash_crc_4byte(uint32_t data, uint32_t init_val)
{
-#if defined RTE_ARCH_X86
- if (likely(crc32_alg & CRC32_SSE42))
- return crc32c_sse42_u32(data, init_val);
-#endif
-
- return crc32c_1word(data, init_val);
+ return (crc32_4b)(data, init_val);
}
/**
- * Use single crc32 instruction to perform a hash on a 8 byte value.
- * Fall back to software crc32 implementation in case SSE4.2 is
- * not supported
+ * Calculate crc32 hash value of 8bytes.
*
* @param data
* Data to perform hash on.
@@ -540,21 +513,9 @@ rte_hash_crc_4byte(uint32_t data, uint32_t init_val)
static inline uint32_t
rte_hash_crc_8byte(uint64_t data, uint32_t init_val)
{
-#ifdef RTE_ARCH_X86_64
- if (likely(crc32_alg == CRC32_SSE42_x64))
- return crc32c_sse42_u64(data, init_val);
-#endif
-
-#if defined RTE_ARCH_X86
- if (likely(crc32_alg & CRC32_SSE42))
- return crc32c_sse42_u64_mimic(data, init_val);
-#endif
-
- return crc32c_2words(data, init_val);
+ return (crc32_8b)(data, init_val);
}
-#endif
-
/**
* Calculate CRC32 hash on user-supplied byte array.
*
--
2.17.1
^ permalink raw reply [relevance 2%]
* Re: [dpdk-dev] [PATCH v4] ethdev: support flow aging
2020-04-21 10:11 2% ` [dpdk-dev] [PATCH v4] " Bill Zhou
2020-04-21 17:13 0% ` Ferruh Yigit
@ 2020-04-29 14:50 0% ` Tom Barbette
2020-04-30 7:36 0% ` Matan Azrad
1 sibling, 1 reply; 200+ results
From: Tom Barbette @ 2020-04-29 14:50 UTC (permalink / raw)
To: Bill Zhou, orika, matan, wenzhuo.lu, jingjing.wu,
bernard.iremonger, john.mcnamara, marko.kovacevic, thomas,
ferruh.yigit, arybchenko
Cc: dev
Great news!
- I can understand why there is no timeout unit. But that's calling for
user nightmare. Eg I could only get from the code (and not from
documentation yet? ) of the following mlx5 driver patch that the value
should be in tenth of seconds. If I build an application that is
supposed to work with "any NIC", what can I do? We'd need a way to query
the timeout unit (have it in dev_info probably).
- It's not totally clear if the rule is automatically removed or not. is
this a helper or an OpenFlow-like notification?
- Find a typo and grammar fix inline.
- Recently, Mellanox introduced the ability to create 330K flows/s. Any
performance considerations if those flow "expire" at the same rate?
Hope it's helpfull,
Tom
Le 21/04/2020 à 12:11, Bill Zhou a écrit :
> From: Dong Zhou <dongz@mellanox.com>
>
> One of the reasons to destroy a flow is the fact that no packet matches the
> flow for "timeout" time.
> For example, when TCP\UDP sessions are suddenly closed.
>
> Currently, there is not any DPDK mechanism for flow aging and the
> applications use their own ways to detect and destroy aged-out flows.
>
> The flow aging implementation need include:
> - A new rte_flow action: RTE_FLOW_ACTION_TYPE_AGE to set the timeout and
> the application flow context for each flow.
> - A new ethdev event: RTE_ETH_EVENT_FLOW_AGED for the driver to report
> that there are new aged-out flows.
> - A new rte_flow API: rte_flow_get_aged_flows to get the aged-out flows
> contexts from the port.
> - Support input flow aging command line in Testpmd.
>
> The new event type addition in the enum is flagged as an ABI breakage, so
> an ignore rule is added for these reasons:
> - It is not changing value of existing types (except MAX)
> - The new value is not used by existing API if the event is not registered
> In general, it is safe adding new ethdev event types at the end of the
> enum, because of event callback registration mechanism.
>
> Signed-off-by: Dong Zhou <dongz@mellanox.com>
> ---
> v2: Removing "* Added support for flow Aging mechanism base on counter."
> this line from doc/guides/rel_notes/release_20_05.rst, this patch does not
> include this support.
>
> v3: Update file libabigail.abignore, add one new suppressed enumeration
> type for RTE_ETH_EVENT_MAX.
>
> v4: Add justification in devtools/libabigail.abignore and in the commit
> log about the modification of v3.
> ---
> app/test-pmd/cmdline_flow.c | 26 ++++++++++
> devtools/libabigail.abignore | 6 +++
> doc/guides/prog_guide/rte_flow.rst | 22 +++++++++
> doc/guides/rel_notes/release_20_05.rst | 11 +++++
> lib/librte_ethdev/rte_ethdev.h | 1 +
> lib/librte_ethdev/rte_ethdev_version.map | 3 ++
> lib/librte_ethdev/rte_flow.c | 18 +++++++
> lib/librte_ethdev/rte_flow.h | 62 ++++++++++++++++++++++++
> lib/librte_ethdev/rte_flow_driver.h | 6 +++
> 9 files changed, 155 insertions(+)
>
> diff --git a/app/test-pmd/cmdline_flow.c b/app/test-pmd/cmdline_flow.c
> index e6ab8ff2f7..45bcff3cf5 100644
> --- a/app/test-pmd/cmdline_flow.c
> +++ b/app/test-pmd/cmdline_flow.c
> @@ -343,6 +343,8 @@ enum index {
> ACTION_SET_IPV4_DSCP_VALUE,
> ACTION_SET_IPV6_DSCP,
> ACTION_SET_IPV6_DSCP_VALUE,
> + ACTION_AGE,
> + ACTION_AGE_TIMEOUT,
> };
>
> /** Maximum size for pattern in struct rte_flow_item_raw. */
> @@ -1145,6 +1147,7 @@ static const enum index next_action[] = {
> ACTION_SET_META,
> ACTION_SET_IPV4_DSCP,
> ACTION_SET_IPV6_DSCP,
> + ACTION_AGE,
> ZERO,
> };
>
> @@ -1370,6 +1373,13 @@ static const enum index action_set_ipv6_dscp[] = {
> ZERO,
> };
>
> +static const enum index action_age[] = {
> + ACTION_AGE,
> + ACTION_AGE_TIMEOUT,
> + ACTION_NEXT,
> + ZERO,
> +};
> +
> static int parse_set_raw_encap_decap(struct context *, const struct token *,
> const char *, unsigned int,
> void *, unsigned int);
> @@ -3694,6 +3704,22 @@ static const struct token token_list[] = {
> (struct rte_flow_action_set_dscp, dscp)),
> .call = parse_vc_conf,
> },
> + [ACTION_AGE] = {
> + .name = "age",
> + .help = "set a specific metadata header",
> + .next = NEXT(action_age),
> + .priv = PRIV_ACTION(AGE,
> + sizeof(struct rte_flow_action_age)),
> + .call = parse_vc,
> + },
> + [ACTION_AGE_TIMEOUT] = {
> + .name = "timeout",
> + .help = "flow age timeout value",
> + .args = ARGS(ARGS_ENTRY_BF(struct rte_flow_action_age,
> + timeout, 24)),
> + .next = NEXT(action_age, NEXT_ENTRY(UNSIGNED)),
> + .call = parse_vc_conf,
> + },
> };
>
> /** Remove and return last entry from argument stack. */
> diff --git a/devtools/libabigail.abignore b/devtools/libabigail.abignore
> index a59df8f135..c047adbd79 100644
> --- a/devtools/libabigail.abignore
> +++ b/devtools/libabigail.abignore
> @@ -11,3 +11,9 @@
> type_kind = enum
> name = rte_crypto_asym_xform_type
> changed_enumerators = RTE_CRYPTO_ASYM_XFORM_TYPE_LIST_END
> +; Ignore ethdev event enum update because new event cannot be
> +; received if not registered
> +[suppress_type]
> + type_kind = enum
> + name = rte_eth_event_type
> + changed_enumerators = RTE_ETH_EVENT_MAX
> diff --git a/doc/guides/prog_guide/rte_flow.rst b/doc/guides/prog_guide/rte_flow.rst
> index 41c147913c..cf4368e1c4 100644
> --- a/doc/guides/prog_guide/rte_flow.rst
> +++ b/doc/guides/prog_guide/rte_flow.rst
> @@ -2616,6 +2616,28 @@ Otherwise, RTE_FLOW_ERROR_TYPE_ACTION error will be returned.
> | ``dscp`` | DSCP in low 6 bits, rest ignore |
> +-----------+---------------------------------+
>
> +Action: ``AGE``
> +^^^^^^^^^^^^^^^^^^^^^^^^^
> +
> +Set ageing timeout configuration to a flow.
> +
> +Event RTE_ETH_EVENT_FLOW_AGED will be reported if
> +timeout passed without any matching on the flow.
> +
> +.. _table_rte_flow_action_age:
> +
> +.. table:: AGE
> +
> + +--------------+---------------------------------+
> + | Field | Value |
> + +==============+=================================+
> + | ``timeout`` | 24 bits timeout value |
> + +--------------+---------------------------------+
> + | ``reserved`` | 8 bits reserved, must be zero |
> + +--------------+---------------------------------+
> + | ``context`` | user input flow context |
> + +--------------+---------------------------------+
> +
> Negative types
> ~~~~~~~~~~~~~~
>
> diff --git a/doc/guides/rel_notes/release_20_05.rst b/doc/guides/rel_notes/release_20_05.rst
> index bacd4c65a2..ff0cf9f1d6 100644
> --- a/doc/guides/rel_notes/release_20_05.rst
> +++ b/doc/guides/rel_notes/release_20_05.rst
> @@ -135,6 +135,17 @@ New Features
> by making use of the event device capabilities. The event mode currently supports
> only inline IPsec protocol offload.
>
> +* **Added flow Aging Support.**
> +
> + Added flow Aging support to detect and report aged-out flows, including:
> +
> + * Added new action: RTE_FLOW_ACTION_TYPE_AGE to set the timeout and the
> + application flow context for each flow.
> + * Added new event: RTE_ETH_EVENT_FLOW_AGED for the driver to report that
> + there are new aged-out flows.
> + * Added new API: rte_flow_get_aged_flows to get the aged-out flows contexts
> + from the port.
> +
>
> Removed Items
> -------------
> diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h
> index 8d69b88f9e..00cc7b4052 100644
> --- a/lib/librte_ethdev/rte_ethdev.h
> +++ b/lib/librte_ethdev/rte_ethdev.h
> @@ -3018,6 +3018,7 @@ enum rte_eth_event_type {
> RTE_ETH_EVENT_NEW, /**< port is probed */
> RTE_ETH_EVENT_DESTROY, /**< port is released */
> RTE_ETH_EVENT_IPSEC, /**< IPsec offload related event */
> + RTE_ETH_EVENT_FLOW_AGED,/**< New aged-out flows is detected */
> RTE_ETH_EVENT_MAX /**< max value of this enum */
> };
>
> diff --git a/lib/librte_ethdev/rte_ethdev_version.map b/lib/librte_ethdev/rte_ethdev_version.map
> index 3f32fdecf7..fa4b5816be 100644
> --- a/lib/librte_ethdev/rte_ethdev_version.map
> +++ b/lib/librte_ethdev/rte_ethdev_version.map
> @@ -230,4 +230,7 @@ EXPERIMENTAL {
>
> # added in 20.02
> rte_flow_dev_dump;
> +
> + # added in 20.05
> + rte_flow_get_aged_flows;
> };
> diff --git a/lib/librte_ethdev/rte_flow.c b/lib/librte_ethdev/rte_flow.c
> index a5ac1c7fbd..3699edce49 100644
> --- a/lib/librte_ethdev/rte_flow.c
> +++ b/lib/librte_ethdev/rte_flow.c
> @@ -172,6 +172,7 @@ static const struct rte_flow_desc_data rte_flow_desc_action[] = {
> MK_FLOW_ACTION(SET_META, sizeof(struct rte_flow_action_set_meta)),
> MK_FLOW_ACTION(SET_IPV4_DSCP, sizeof(struct rte_flow_action_set_dscp)),
> MK_FLOW_ACTION(SET_IPV6_DSCP, sizeof(struct rte_flow_action_set_dscp)),
> + MK_FLOW_ACTION(AGE, sizeof(struct rte_flow_action_age)),
> };
>
> int
> @@ -1232,3 +1233,20 @@ rte_flow_dev_dump(uint16_t port_id, FILE *file, struct rte_flow_error *error)
> RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
> NULL, rte_strerror(ENOSYS));
> }
> +
> +int
> +rte_flow_get_aged_flows(uint16_t port_id, void **contexts,
> + uint32_t nb_contexts, struct rte_flow_error *error)
> +{
> + struct rte_eth_dev *dev = &rte_eth_devices[port_id];
> + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error);
> +
> + if (unlikely(!ops))
> + return -rte_errno;
> + if (likely(!!ops->get_aged_flows))
> + return flow_err(port_id, ops->get_aged_flows(dev, contexts,
> + nb_contexts, error), error);
> + return rte_flow_error_set(error, ENOTSUP,
> + RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
> + NULL, rte_strerror(ENOTSUP));
> +}
> diff --git a/lib/librte_ethdev/rte_flow.h b/lib/librte_ethdev/rte_flow.h
> index 7f3e08fad3..fab44f6c0b 100644
> --- a/lib/librte_ethdev/rte_flow.h
> +++ b/lib/librte_ethdev/rte_flow.h
> @@ -2081,6 +2081,16 @@ enum rte_flow_action_type {
> * See struct rte_flow_action_set_dscp.
> */
> RTE_FLOW_ACTION_TYPE_SET_IPV6_DSCP,
> +
> + /**
> + * Report as aged flow if timeout passed without any matching on the
> + * flow.
> + *
> + * See struct rte_flow_action_age.
> + * See function rte_flow_get_aged_flows
> + * see enum RTE_ETH_EVENT_FLOW_AGED
> + */
> + RTE_FLOW_ACTION_TYPE_AGE,
> };
>
> /**
> @@ -2122,6 +2132,25 @@ struct rte_flow_action_queue {
> uint16_t index; /**< Queue index to use. */
> };
>
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this structure may change without prior notice
> + *
> + * RTE_FLOW_ACTION_TYPE_AGE
> + *
> + * Report flow as aged-out if timeout passed without any matching
> + * on the flow. RTE_ETH_EVENT_FLOW_AGED event is triggered when a
> + * port detects new aged-out flows.
> + *
> + * The flow context and the flow handle will be reported by the
> + * rte_flow_get_aged_flows API.
> + */
> +struct rte_flow_action_age {
> + uint32_t timeout:24; /**< Time in seconds. */
> + uint32_t reserved:8; /**< Reserved, must be zero. */
> + void *context;
> + /**< The user flow context, NULL means the rte_flow pointer. */
> +};
>
> /**
> * @warning
> @@ -3254,6 +3283,39 @@ rte_flow_conv(enum rte_flow_conv_op op,
> const void *src,
> struct rte_flow_error *error);
>
> +/**
> + * Get aged-out flows of a given port.
> + *
> + * RTE_ETH_EVENT_FLOW_AGED event will be triggered when at least one new aged
> + * out flow was detected after the last call to rte_flow_get_aged_flows.
> + * This function can be called to get the aged flows usynchronously from the
usynchronously
> + * event callback or synchronously regardless the event.
> + * This is not safe to call rte_flow_get_aged_flows function with other flow
It is not safe to
> + * functions from multiple threads simultaneously.
> + *
> + * @param port_id
> + * Port identifier of Ethernet device.
> + * @param[in, out] contexts
> + * The address of an array of pointers to the aged-out flows contexts.
> + * @param[in] nb_contexts
> + * The length of context array pointers.
> + * @param[out] error
> + * Perform verbose error reporting if not NULL. Initialized in case of
> + * error only.
> + *
> + * @return
> + * if nb_contexts is 0, return the amount of all aged contexts.
> + * if nb_contexts is not 0 , return the amount of aged flows reported
> + * in the context array, otherwise negative errno value.
> + *
> + * @see rte_flow_action_age
> + * @see RTE_ETH_EVENT_FLOW_AGED
> + */
> +__rte_experimental
> +int
> +rte_flow_get_aged_flows(uint16_t port_id, void **contexts,
> + uint32_t nb_contexts, struct rte_flow_error *error);
> +
> #ifdef __cplusplus
> }
> #endif
> diff --git a/lib/librte_ethdev/rte_flow_driver.h b/lib/librte_ethdev/rte_flow_driver.h
> index 51a9a57a0f..881cc469b7 100644
> --- a/lib/librte_ethdev/rte_flow_driver.h
> +++ b/lib/librte_ethdev/rte_flow_driver.h
> @@ -101,6 +101,12 @@ struct rte_flow_ops {
> (struct rte_eth_dev *dev,
> FILE *file,
> struct rte_flow_error *error);
> + /** See rte_flow_get_aged_flows() */
> + int (*get_aged_flows)
> + (struct rte_eth_dev *dev,
> + void **context,
> + uint32_t nb_contexts,
> + struct rte_flow_error *err);
> };
>
> /**
>
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH v2] abi: change references to abi 20.0.1 to abi v21
2020-04-24 14:50 6% ` Ray Kinsella
2020-04-24 15:01 6% ` David Marchand
@ 2020-04-29 12:19 9% ` Dodji Seketeli
2020-04-30 8:23 6% ` Ray Kinsella
1 sibling, 1 reply; 200+ results
From: Dodji Seketeli @ 2020-04-29 12:19 UTC (permalink / raw)
To: Ray Kinsella
Cc: David Marchand, dev, Andrzej Ostruszka, Stephen Hemminger,
Thomas Monjalon, Neil Horman, Jingjing Wu, Wenzhuo Lu,
Matan Azrad, Shahaf Shuler, Viacheslav Ovsiienko, Jerin Jacob,
Nithin Dabilpuram, Alfredo Cardigliano, Mahipal Challa,
Cristian Dumitrescu, Wang, Haiyue
Hello,
Ray Kinsella <mdr@ashroe.eu> writes:
> ah ok, the particular system I made the change on was Ubuntu 18.04.2.
> which is libabigail 1.2.0.
Whoah, 1.2 is super old.
In my opinion, one of the hallmarks of static analysis tools (and
libabigail is just a static analysis framework) is to be able to
recognize patterns used by developers, as much as we can.
Because we can't really do that at once, we try to add recognition of
new patterns (of ABI changes) at every single release. Furthermore,
there are some change patterns that ought to be recognized and
categorized as harmless, whereas some others out to be categorized as
harmful. That categorization is also the result of input coming from
users as you, fine fellows.
All this to say that with every new version, the number of new supported
features and bug fixes is potentially big.
To alleviate that, some distributors update libabigail even in their old
stable distros, because the value of having an up to date version there
outweighs the potential drawbacks.
> Given we still support v19.11 on Ubuntu 18.04.2.
So maybe that's a discussion worth having with the maintainer of the
Ubuntu package of Libabigail?
> I think it's worthwhile keeping the suppression until v20.11?
[...]
David Marchand <david.marchand@redhat.com> writes:
> In Travis, we currently use libabigail 1.6 (mainly because I did not
> update to 1.7 when it was released).
Right, that's probably another way to stay up to date independently from
the underlying distribution.
I hope this helps,
Cheers,
--
Dodji
^ permalink raw reply [relevance 9%]
* Re: [dpdk-dev] [PATCH v5] eal/cpuflags: add x86 based cpu flags
2020-04-28 18:11 4% ` Laatz, Kevin
@ 2020-04-29 11:39 3% ` David Marchand
2020-04-30 10:02 0% ` Ray Kinsella
1 sibling, 1 reply; 200+ results
From: David Marchand @ 2020-04-29 11:39 UTC (permalink / raw)
To: Ray Kinsella
Cc: Kevin Laatz, dev, Bruce Richardson, Van Haaren Harry,
Thomas Monjalon, Kinsella, Ray, Neil Horman
On Tue, Apr 28, 2020 at 6:39 PM Ray Kinsella <mdr@ashroe.eu> wrote:
> > diff --git a/devtools/libabigail.abignore b/devtools/libabigail.abignore
> > index a59df8f13..045f436fb 100644
> > --- a/devtools/libabigail.abignore
> > +++ b/devtools/libabigail.abignore
>
> Kevin - you still have the surpession.
> I am testing locally with 1.7.1, and it doesn't complain when I disable the supression.
> Are you seeing something different?
Using current master libabigail, without the rule Kevin included, I
get the warning:
1 function with some indirect sub-type change:
[C] 'function int rte_cpu_get_flag_enabled(rte_cpu_flag_t)' at
rte_cpuflags.c:144:1 has some indirect sub-type changes:
parameter 1 of type 'enum rte_cpu_flag_t' has sub-type changes:
type size hasn't changed
17 enumerator insertions:
'rte_cpu_flag_t::RTE_CPUFLAG_AVX512DQ' value '87'
'rte_cpu_flag_t::RTE_CPUFLAG_AVX512IFMA' value '88'
'rte_cpu_flag_t::RTE_CPUFLAG_AVX512CD' value '89'
'rte_cpu_flag_t::RTE_CPUFLAG_AVX512BW' value '90'
'rte_cpu_flag_t::RTE_CPUFLAG_AVX512VL' value '91'
'rte_cpu_flag_t::RTE_CPUFLAG_AVX512VBMI' value '92'
'rte_cpu_flag_t::RTE_CPUFLAG_AVX512VBMI2' value '93'
'rte_cpu_flag_t::RTE_CPUFLAG_GFNI' value '94'
'rte_cpu_flag_t::RTE_CPUFLAG_VAES' value '95'
'rte_cpu_flag_t::RTE_CPUFLAG_VPCLMULQDQ' value '96'
'rte_cpu_flag_t::RTE_CPUFLAG_AVX512VNNI' value '97'
'rte_cpu_flag_t::RTE_CPUFLAG_AVX512BITALG' value '98'
'rte_cpu_flag_t::RTE_CPUFLAG_AVX512VPOPCNTDQ' value '99'
'rte_cpu_flag_t::RTE_CPUFLAG_CLDEMOTE' value '100'
'rte_cpu_flag_t::RTE_CPUFLAG_MOVDIRI' value '101'
'rte_cpu_flag_t::RTE_CPUFLAG_MOVDIR64B' value '102'
'rte_cpu_flag_t::RTE_CPUFLAG_AVX512VP2INTERSECT' value '103'
1 enumerator change:
'rte_cpu_flag_t::RTE_CPUFLAG_NUMFLAGS' from value '87' to
'104' at rte_cpuflags.h:12:1
Ray, could you check that the reference and new dumps in your env
contain this enum?
$ grep RTE_CPUFLAG_NUMFLAGS
$HOME/abi/v20.02/x86_64-native-linux-gcc+shared+debug+ASSERT+RTE_IBVERBS_LINK_DLOPEN/dump/librte_eal.dump
<enumerator name='RTE_CPUFLAG_NUMFLAGS' value='87'/>
$ grep RTE_CPUFLAG_NUMFLAGS
$HOME/builds/x86_64-native-linux-gcc+shared+debug+ASSERT+RTE_IBVERBS_LINK_DLOPEN/install/dump/librte_eal.dump
<enumerator name='RTE_CPUFLAG_NUMFLAGS' value='104'/>
If you are missing those, you might have built dpdk without debuginfo.
--
David Marchand
^ permalink raw reply [relevance 3%]
* Re: [dpdk-dev] [PATCH v4] eal/cpuflags: add x86 based cpu flags
2020-04-27 13:58 0% ` Ray Kinsella
@ 2020-04-29 11:22 0% ` Neil Horman
2020-04-30 7:59 4% ` Ray Kinsella
0 siblings, 1 reply; 200+ results
From: Neil Horman @ 2020-04-29 11:22 UTC (permalink / raw)
To: Ray Kinsella
Cc: Thomas Monjalon, ray.kinsella, Kevin Laatz, dev,
bruce.richardson, harry.van.haaren, david.marchand, Haiyue Wang,
ktraynor
On Mon, Apr 27, 2020 at 02:58:07PM +0100, Ray Kinsella wrote:
>
>
> On 27/04/2020 13:31, Thomas Monjalon wrote:
> > 27/04/2020 11:27, Ray Kinsella:
> >> On 25/04/2020 17:04, Thomas Monjalon wrote:
> >>> PS: Who is REALLY maintaining the ABI?
> >>> We really miss someone who carefully check all these things,
> >>> and take care of the doc and tooling.
> >>
> >> I would say that I am missing these changes to libabigail.ignore, which would be useful.
> >> Should we consolidate the ABI Policy and ABI Versioning sections of the MAINTAINERS file?
> >
> > Yes, I think it does not make sense spliting ABI topic in 2 sections
> > in MAINTAINERS file.
> > We need to have a clear ownership covering policy, libs, tooling and doc.
> > Let's agree to merge all in one section please.
> >
>
> I would suggest merging and listing myself and Neil as maintainers?
> Unless you are aware of another potential owner?
>
I'm ok with this
Neil
> Ray K
>
>
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH v4 1/4] ethdev: add tm support for shaper config in pkt mode
2020-04-29 8:45 0% ` Dumitrescu, Cristian
@ 2020-04-29 9:03 0% ` Bruce Richardson
2020-05-01 10:27 0% ` Ferruh Yigit
0 siblings, 1 reply; 200+ results
From: Bruce Richardson @ 2020-04-29 9:03 UTC (permalink / raw)
To: Dumitrescu, Cristian
Cc: Thomas Monjalon, Jerin Jacob, Yigit, Ferruh, Luca Boccassi,
Nithin Dabilpuram, Singh, Jasvinder, Andrew Rybchenko, dev,
jerinj, kkanas, Nithin Dabilpuram, Kinsella, Ray, Neil Horman,
Kevin Traynor, David Marchand
On Wed, Apr 29, 2020 at 09:45:44AM +0100, Dumitrescu, Cristian wrote:
>
>
> > -----Original Message-----
> > From: Thomas Monjalon <thomas@monjalon.net>
> > Sent: Tuesday, April 28, 2020 4:54 PM
> > To: Jerin Jacob <jerinjacobk@gmail.com>; Dumitrescu, Cristian
> > <cristian.dumitrescu@intel.com>
> > Cc: Richardson, Bruce <bruce.richardson@intel.com>; Yigit, Ferruh
> > <ferruh.yigit@intel.com>; Luca Boccassi <bluca@debian.org>; Nithin
> > Dabilpuram <nithind1988@gmail.com>; Singh, Jasvinder
> > <jasvinder.singh@intel.com>; Andrew Rybchenko
> > <arybchenko@solarflare.com>; dev@dpdk.org; jerinj@marvell.com;
> > kkanas@marvell.com; Nithin Dabilpuram <ndabilpuram@marvell.com>;
> > Kinsella, Ray <ray.kinsella@intel.com>; Neil Horman
> > <nhorman@tuxdriver.com>; Kevin Traynor <ktraynor@redhat.com>; David
> > Marchand <david.marchand@redhat.com>
> > Subject: Re: [dpdk-dev] [PATCH v4 1/4] ethdev: add tm support for shaper
> > config in pkt mode
> >
> > 28/04/2020 17:04, Luca Boccassi:
> > > On Tue, 2020-04-28 at 15:45 +0100, Bruce Richardson wrote:
> > > > On Tue, Apr 28, 2020 at 03:06:20PM +0100, Ferruh Yigit wrote:
> > > > > On 4/27/2020 5:59 PM, Jerin Jacob wrote:
> > > > > > On Mon, Apr 27, 2020 at 10:19 PM Ferruh Yigit
> > <ferruh.yigit@intel.com> wrote:
> > > > > > > On 4/27/2020 5:29 PM, Jerin Jacob wrote:
> > > > > > > > On Mon, Apr 27, 2020 at 9:42 PM Ferruh Yigit
> > <ferruh.yigit@intel.com> wrote:
> > > > > > > > > On 4/27/2020 10:19 AM, Dumitrescu, Cristian wrote:
> > > > > > > > > > From: Yigit, Ferruh <ferruh.yigit@intel.com>
> > > > > > > > > > > On 4/24/2020 11:28 AM, Dumitrescu, Cristian wrote:
> > > > > > > > > > > > From: Nithin Dabilpuram <nithind1988@gmail.com>
> > > > > > > > > > > > > This patch also updates tm port/level/node capability
> > structures with
> > > > > > > > > > > > > exiting features of scheduler wfq packet mode,
> > scheduler wfq byte mode
> > > > > > > > > > > > > and private/shared shaper byte mode.
> > > > > > > > > > > > >
> > > > > > > > > > > > > SoftNIC PMD is also updated with new capabilities.
> > [...]
> > > > > > > > > > > Hi Nithin,
> > > > > > > > > > >
> > > > > > > > > > > It looks like patch is causing ABI break, I am getting following
> > warning [1],
> > > > > > > > > > > can you please check?
> > > > > > > > > > >
> > > > > > > > > > > [1]
> > > > > > > > > > > https://pastebin.com/XYNFg14u
> > > > > > > > > >
> > > > > > > > > > Hi Ferruh,
> > > > > > > > > >
> > > > > > > > > > The RTE_TM API is marked as experimental,
> > > > > > > > > > but it looks that this was not correctly marked
> > > > > > > > > > when __rte_experimental ABI checker was introduced.
> > > > > > > > > >
> > > > > > > > > > It is marked as experimental at the top of the rte_tm.h,
> > > > > > > > > > similarly to other APIs introduced around same time,
> > > > > > > > > > but it was not correctly picked up by the ABI check procedure
> > > > > > > > > > when later introduced, so __rte_experimental was not added
> > to every function.
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > :(
> > > > > > > > >
> > > > > > > > > Is it time to mature them?
> > > > > > > > >
> > > > > > > > > As you said they are not marked as experimental both in header
> > file (function
> > > > > > > > > declarations) and .map file.
> > > > > > > > >
> > > > > > > > > The problem is, they are not marked as experimental in
> > DPDK_20.0 ABI (v19.11),
> > > > > > > > > so marking them as experimental now will break the ABI. Not
> > sure what to do,
> > > > > > > > > cc'ed a few ABI related names for comment.
> > > > > > > > >
> > > > > > > > > For me, we need to proceed as the experimental tag removed
> > and APIs become
> > > > > > > > > mature starting from v19.11, since this is what happened in
> > practice, and remove
> > > > > > > > > a few existing being experimental references in the doxygen
> > comments.
> > > > > > > >
> > > > > > > > I think, accidentally we can not make a library as NON-
> > experimental.
> > > > > > > > TM never went through experimental to mature transition(see git
> > log
> > > > > > > > lib/librte_ethdev/rte_tm.h)
> > > > > > > > It was a bug to not mark as experimental in each function in the
> > ABI process.
> > > > > > > > Some of the features like packet marking are not even
> > implemented by any HW.
> > > > > > > > I think, we can make API stable only all the features are
> > implemented
> > > > > > > > by one or two HW.
> >
> > Yes this is what was decided one or two years ago I think.
> > But rte_tm API was introduced 3 years ago and is implemented by 6 PMDs.
> >
> >
> >
> > > > > > > Fair enough, specially if the API is not ready yet.
> > > > > > >
> > > > > > > But they were part of stable ABI, and marking them as experimental
> > now will
> > > > > > > break the old applications using these APIs.
> > > > > >
> > > > > > it is still marked as EXPERIMENTAL everywhere and API is not ready
> > yet.
> >
> > rte_tm is implemented in 6 PMDs.
> >
> >
> > > > > Existing experimental marks are text only for human parsing.
> > > > >
> > > > > The compiler attribute and build time checks are missing, and the
> > symbol in the
> > > > > binary doesn't have experimental tag. Our scripts and automated
> > checks won't
> > > > > detect it as experimental.
> > > > >
> > > > > My point is just having experimental comment in header file is not
> > enough to
> > > > > qualify the APIs as experimental.
> > > > >
> > > > > > Anyway, we need to break the ABI to make it work on various HW.
> >
> > Yes this is why I was asking in 19.11 to check our API,
> > in order to avoid such situation.
> >
> >
> > > > > > I am not sure what to do?
> >
> > Either manage ABI versioning, or wait 20.11.
> >
> >
> > > > > > IMO, We need to send a patch as Fixes: for the bug of not adding
> > > > > > __rte_experimental in each function.
> >
> > No, this is wrong.
> >
>
> Why exactly is this wrong? This is the gap that caused the current discussion, right?
>
It's wrong for this release, since we can't change things from stable back
to experimental. Any such patch will have to wait for 20.11, as agreed in
the discussion.
/Bruce
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH v4 1/4] ethdev: add tm support for shaper config in pkt mode
2020-04-28 15:54 5% ` Thomas Monjalon
@ 2020-04-29 8:45 0% ` Dumitrescu, Cristian
2020-04-29 9:03 0% ` Bruce Richardson
2020-05-01 13:18 3% ` [dpdk-dev] " Jerin Jacob
1 sibling, 1 reply; 200+ results
From: Dumitrescu, Cristian @ 2020-04-29 8:45 UTC (permalink / raw)
To: Thomas Monjalon, Jerin Jacob
Cc: Richardson, Bruce, Yigit, Ferruh, Luca Boccassi,
Nithin Dabilpuram, Singh, Jasvinder, Andrew Rybchenko, dev,
jerinj, kkanas, Nithin Dabilpuram, Kinsella, Ray, Neil Horman,
Kevin Traynor, David Marchand
> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: Tuesday, April 28, 2020 4:54 PM
> To: Jerin Jacob <jerinjacobk@gmail.com>; Dumitrescu, Cristian
> <cristian.dumitrescu@intel.com>
> Cc: Richardson, Bruce <bruce.richardson@intel.com>; Yigit, Ferruh
> <ferruh.yigit@intel.com>; Luca Boccassi <bluca@debian.org>; Nithin
> Dabilpuram <nithind1988@gmail.com>; Singh, Jasvinder
> <jasvinder.singh@intel.com>; Andrew Rybchenko
> <arybchenko@solarflare.com>; dev@dpdk.org; jerinj@marvell.com;
> kkanas@marvell.com; Nithin Dabilpuram <ndabilpuram@marvell.com>;
> Kinsella, Ray <ray.kinsella@intel.com>; Neil Horman
> <nhorman@tuxdriver.com>; Kevin Traynor <ktraynor@redhat.com>; David
> Marchand <david.marchand@redhat.com>
> Subject: Re: [dpdk-dev] [PATCH v4 1/4] ethdev: add tm support for shaper
> config in pkt mode
>
> 28/04/2020 17:04, Luca Boccassi:
> > On Tue, 2020-04-28 at 15:45 +0100, Bruce Richardson wrote:
> > > On Tue, Apr 28, 2020 at 03:06:20PM +0100, Ferruh Yigit wrote:
> > > > On 4/27/2020 5:59 PM, Jerin Jacob wrote:
> > > > > On Mon, Apr 27, 2020 at 10:19 PM Ferruh Yigit
> <ferruh.yigit@intel.com> wrote:
> > > > > > On 4/27/2020 5:29 PM, Jerin Jacob wrote:
> > > > > > > On Mon, Apr 27, 2020 at 9:42 PM Ferruh Yigit
> <ferruh.yigit@intel.com> wrote:
> > > > > > > > On 4/27/2020 10:19 AM, Dumitrescu, Cristian wrote:
> > > > > > > > > From: Yigit, Ferruh <ferruh.yigit@intel.com>
> > > > > > > > > > On 4/24/2020 11:28 AM, Dumitrescu, Cristian wrote:
> > > > > > > > > > > From: Nithin Dabilpuram <nithind1988@gmail.com>
> > > > > > > > > > > > This patch also updates tm port/level/node capability
> structures with
> > > > > > > > > > > > exiting features of scheduler wfq packet mode,
> scheduler wfq byte mode
> > > > > > > > > > > > and private/shared shaper byte mode.
> > > > > > > > > > > >
> > > > > > > > > > > > SoftNIC PMD is also updated with new capabilities.
> [...]
> > > > > > > > > > Hi Nithin,
> > > > > > > > > >
> > > > > > > > > > It looks like patch is causing ABI break, I am getting following
> warning [1],
> > > > > > > > > > can you please check?
> > > > > > > > > >
> > > > > > > > > > [1]
> > > > > > > > > > https://pastebin.com/XYNFg14u
> > > > > > > > >
> > > > > > > > > Hi Ferruh,
> > > > > > > > >
> > > > > > > > > The RTE_TM API is marked as experimental,
> > > > > > > > > but it looks that this was not correctly marked
> > > > > > > > > when __rte_experimental ABI checker was introduced.
> > > > > > > > >
> > > > > > > > > It is marked as experimental at the top of the rte_tm.h,
> > > > > > > > > similarly to other APIs introduced around same time,
> > > > > > > > > but it was not correctly picked up by the ABI check procedure
> > > > > > > > > when later introduced, so __rte_experimental was not added
> to every function.
> > > > > > > > >
> > > > > > > >
> > > > > > > > :(
> > > > > > > >
> > > > > > > > Is it time to mature them?
> > > > > > > >
> > > > > > > > As you said they are not marked as experimental both in header
> file (function
> > > > > > > > declarations) and .map file.
> > > > > > > >
> > > > > > > > The problem is, they are not marked as experimental in
> DPDK_20.0 ABI (v19.11),
> > > > > > > > so marking them as experimental now will break the ABI. Not
> sure what to do,
> > > > > > > > cc'ed a few ABI related names for comment.
> > > > > > > >
> > > > > > > > For me, we need to proceed as the experimental tag removed
> and APIs become
> > > > > > > > mature starting from v19.11, since this is what happened in
> practice, and remove
> > > > > > > > a few existing being experimental references in the doxygen
> comments.
> > > > > > >
> > > > > > > I think, accidentally we can not make a library as NON-
> experimental.
> > > > > > > TM never went through experimental to mature transition(see git
> log
> > > > > > > lib/librte_ethdev/rte_tm.h)
> > > > > > > It was a bug to not mark as experimental in each function in the
> ABI process.
> > > > > > > Some of the features like packet marking are not even
> implemented by any HW.
> > > > > > > I think, we can make API stable only all the features are
> implemented
> > > > > > > by one or two HW.
>
> Yes this is what was decided one or two years ago I think.
> But rte_tm API was introduced 3 years ago and is implemented by 6 PMDs.
>
>
>
> > > > > > Fair enough, specially if the API is not ready yet.
> > > > > >
> > > > > > But they were part of stable ABI, and marking them as experimental
> now will
> > > > > > break the old applications using these APIs.
> > > > >
> > > > > it is still marked as EXPERIMENTAL everywhere and API is not ready
> yet.
>
> rte_tm is implemented in 6 PMDs.
>
>
> > > > Existing experimental marks are text only for human parsing.
> > > >
> > > > The compiler attribute and build time checks are missing, and the
> symbol in the
> > > > binary doesn't have experimental tag. Our scripts and automated
> checks won't
> > > > detect it as experimental.
> > > >
> > > > My point is just having experimental comment in header file is not
> enough to
> > > > qualify the APIs as experimental.
> > > >
> > > > > Anyway, we need to break the ABI to make it work on various HW.
>
> Yes this is why I was asking in 19.11 to check our API,
> in order to avoid such situation.
>
>
> > > > > I am not sure what to do?
>
> Either manage ABI versioning, or wait 20.11.
>
>
> > > > > IMO, We need to send a patch as Fixes: for the bug of not adding
> > > > > __rte_experimental in each function.
>
> No, this is wrong.
>
Why exactly is this wrong? This is the gap that caused the current discussion, right?
>
> > > > Yes, this is where we are, both you and Cristian suggest API is not ready
> and
> > > > should be experimental, but they were part of stable ABI, making them
> > > > experimental will break the ABI.
> > > > It looks like there is no good option but we should select one of the bad
> ones.
> > > >
> > > > > Traffic Management API - EXPERIMENTAL
> > > > > M: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
> > > > > T: git://dpdk.org/next/dpdk-next-qos
> > > > > F: lib/librte_ethdev/rte_tm*
> > > > > > > > Ray, Neil, David, Luca, Kevin, what do you think?
> > > While I'm not called any of those names, allow me to give my 2c.
> > >
> > > Since these are marked in binaries as part of the stable ABI, I think we
> > > need to honour that for the next two releases 20.05 and 20.08 [which
> means
> > > that we need to put in versioned functions for any changes, not that we
> > > can't change anything]
> > >
> > > For 20.11, I think these should then have one of two options taken:
> > > * have these "fixed" and ready to be marked as stable, and officially part
> > > of v21 ABI or
> > > * mark them as experimental properly, and look to have them as part of
> the
> > > v22 or subsequent ABI
> > >
> > > Given the comments here, I would tend towards the latter of the above
> two
> > > options, but that's really a decision for the maintainers.
> > >
> > > Remember, this is not the first bug we have encountered where we
> messed up
> > > some ABI versions in the 19.11 release, and, like the previous one with
> the
> > > screwed up version number, I think we need to honour the ABI
> committments
> > > made, especially since in this case it's only for a few more months till
> > > 20.11 development starts.
> > >
> > > /Bruce
> >
> > +1
> >
> > If they are not ready now, they haven't been ready for the past 6
> > months either, so staying not ready for 6 more is the lesser evil.
>
> This API is almost 3 years old (release 17.08).
> That's good to improve it but we must respect the ABI contract that
> we all agreed.
>
>
> Summary:
> 17.08: rte_tm is introduced.
> 17.11: rte_mtr is introduced as experimental, but rte_tm remains stable.
> 18.02: __rte_experimental tag is introduced (including for rte_mtr),
> but rte_tm remains untouched as it is in stable ABI.
> 19.11: stable ABI is frozen until 20.11
> 20.05: rte_tm improvement is blocked because of ABI breakage.
>
>
> It should remind everybody of reviewing the new API and policies,
> and maintaining the existing code appropriately.
>
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH v7 0/6] dpdk: introduce __rte_internal tag
2020-04-29 8:22 3% ` David Marchand
@ 2020-04-29 8:24 0% ` Wang, Haiyue
0 siblings, 0 replies; 200+ results
From: Wang, Haiyue @ 2020-04-29 8:24 UTC (permalink / raw)
To: David Marchand
Cc: dev, Thomas Monjalon, Richardson, Bruce, Yigit, Ferruh,
Neil Horman, Ray Kinsella
> -----Original Message-----
> From: David Marchand <david.marchand@redhat.com>
> Sent: Wednesday, April 29, 2020 16:22
> To: Wang, Haiyue <haiyue.wang@intel.com>
> Cc: dev <dev@dpdk.org>; Thomas Monjalon <thomas@monjalon.net>; Richardson, Bruce
> <bruce.richardson@intel.com>; Yigit, Ferruh <ferruh.yigit@intel.com>; Neil Horman
> <nhorman@tuxdriver.com>; Ray Kinsella <mdr@ashroe.eu>
> Subject: Re: [PATCH v7 0/6] dpdk: introduce __rte_internal tag
>
> On Sat, Apr 25, 2020 at 8:10 PM Wang, Haiyue <haiyue.wang@intel.com> wrote:
> >
> > Hi David,
> >
> > > -----Original Message-----
> > > From: David Marchand <david.marchand@redhat.com>
> > > Sent: Saturday, April 25, 2020 22:39
> > > To: Wang, Haiyue <haiyue.wang@intel.com>
> > > Cc: dev <dev@dpdk.org>; Thomas Monjalon <thomas@monjalon.net>; Richardson, Bruce
> > > <bruce.richardson@intel.com>; Yigit, Ferruh <ferruh.yigit@intel.com>; Neil Horman
> > > <nhorman@tuxdriver.com>; Ray Kinsella <mdr@ashroe.eu>
> > > Subject: Re: [PATCH v7 0/6] dpdk: introduce __rte_internal tag
> > >
> > > On Sat, Apr 25, 2020 at 1:02 PM Haiyue Wang <haiyue.wang@intel.com> wrote:
> > > >
> > > > Move the internal function into INTERNAL session to avoid the ABI
> > > > checking, and it is only used for DPDK drivers or related library.
> > > >
> > > > __rte_internal funA
> > > >
> > > > INTERNAL {
> > > > global:
> > > >
> > > > funA
> > > > };
> > >
> > > Thanks a lot for working on this.
> > > I did some modifications (see my replies on patch 3 and 5) and applied
> > > this series.
> > >
> > > We are just missing the update on the scripts mentioned in a previous mail.
> > > Can you work on this for rc2?
> > >
> >
> > Do you mean ?
>
> Sorry, I was referring to another mail:
> """
> > We are missing updates on devtools/check-abi-version.sh and
> > devtools/update_version_map_abi.py.
> """
>
> Those scripts devtools/check-abi-version.sh and
> devtools/update_version_map_abi.py are not "internal" aware.
> Can you take care of them?
>
Got it, I will try.
>
> > > > > This will apply to common drivers that will be 100% internal.
> > > > > Not sure if this is an issue.
> > > >
> > > > This part should be fine, I want others to be aware of this.
> >
> > > I am not one of the ABI maintainers, but in my opinion it is OK
> > > to have "pure internal" libs with version 0.x.
> >
> > I've tested it with Intel's drivers/common/iavf, it works as expected.
> > a). librte_common_iavf.so.0.200.2
> > b). Skipped experimental library librte_common_iavf.dump.
> >
> > This has been updated by your modification.
> > + if is_stable
> > lib_version = abi_version
> > so_version = stable_so_version
> > + else
> > + lib_version = experimental_abi_version
> > + so_version = experimental_so_version
> > endif
>
> Thanks for testing.
>
>
> --
> David Marchand
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH v7 0/6] dpdk: introduce __rte_internal tag
2020-04-25 18:09 0% ` Wang, Haiyue
@ 2020-04-29 8:22 3% ` David Marchand
2020-04-29 8:24 0% ` Wang, Haiyue
0 siblings, 1 reply; 200+ results
From: David Marchand @ 2020-04-29 8:22 UTC (permalink / raw)
To: Wang, Haiyue
Cc: dev, Thomas Monjalon, Richardson, Bruce, Yigit, Ferruh,
Neil Horman, Ray Kinsella
On Sat, Apr 25, 2020 at 8:10 PM Wang, Haiyue <haiyue.wang@intel.com> wrote:
>
> Hi David,
>
> > -----Original Message-----
> > From: David Marchand <david.marchand@redhat.com>
> > Sent: Saturday, April 25, 2020 22:39
> > To: Wang, Haiyue <haiyue.wang@intel.com>
> > Cc: dev <dev@dpdk.org>; Thomas Monjalon <thomas@monjalon.net>; Richardson, Bruce
> > <bruce.richardson@intel.com>; Yigit, Ferruh <ferruh.yigit@intel.com>; Neil Horman
> > <nhorman@tuxdriver.com>; Ray Kinsella <mdr@ashroe.eu>
> > Subject: Re: [PATCH v7 0/6] dpdk: introduce __rte_internal tag
> >
> > On Sat, Apr 25, 2020 at 1:02 PM Haiyue Wang <haiyue.wang@intel.com> wrote:
> > >
> > > Move the internal function into INTERNAL session to avoid the ABI
> > > checking, and it is only used for DPDK drivers or related library.
> > >
> > > __rte_internal funA
> > >
> > > INTERNAL {
> > > global:
> > >
> > > funA
> > > };
> >
> > Thanks a lot for working on this.
> > I did some modifications (see my replies on patch 3 and 5) and applied
> > this series.
> >
> > We are just missing the update on the scripts mentioned in a previous mail.
> > Can you work on this for rc2?
> >
>
> Do you mean ?
Sorry, I was referring to another mail:
"""
> We are missing updates on devtools/check-abi-version.sh and
> devtools/update_version_map_abi.py.
"""
Those scripts devtools/check-abi-version.sh and
devtools/update_version_map_abi.py are not "internal" aware.
Can you take care of them?
> > > > This will apply to common drivers that will be 100% internal.
> > > > Not sure if this is an issue.
> > >
> > > This part should be fine, I want others to be aware of this.
>
> > I am not one of the ABI maintainers, but in my opinion it is OK
> > to have "pure internal" libs with version 0.x.
>
> I've tested it with Intel's drivers/common/iavf, it works as expected.
> a). librte_common_iavf.so.0.200.2
> b). Skipped experimental library librte_common_iavf.dump.
>
> This has been updated by your modification.
> + if is_stable
> lib_version = abi_version
> so_version = stable_so_version
> + else
> + lib_version = experimental_abi_version
> + so_version = experimental_so_version
> endif
Thanks for testing.
--
David Marchand
^ permalink raw reply [relevance 3%]
* Re: [dpdk-dev] [PATCH v3 2/4] hash: check flags on creation
2020-04-27 23:16 3% ` [dpdk-dev] [PATCH v3 2/4] hash: check flags on creation Stephen Hemminger
@ 2020-04-29 1:08 0% ` Wang, Yipeng1
0 siblings, 0 replies; 200+ results
From: Wang, Yipeng1 @ 2020-04-29 1:08 UTC (permalink / raw)
To: Stephen Hemminger, dev; +Cc: Gobriel, Sameh, Richardson, Bruce
> -----Original Message-----
> From: Stephen Hemminger <stephen@networkplumber.org>
> Sent: Monday, April 27, 2020 4:16 PM
> To: dev@dpdk.org
> Cc: Stephen Hemminger <stephen@networkplumber.org>; Wang, Yipeng1
> <yipeng1.wang@intel.com>; Gobriel, Sameh <sameh.gobriel@intel.com>;
> Richardson, Bruce <bruce.richardson@intel.com>
> Subject: [PATCH v3 2/4] hash: check flags on creation
>
> All API's should check that they support the flag values passed. If an application
> passes an invalid flag it could cause problems in later ABI.
>
> Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
> ---
Acked-by: Yipeng Wang <yipeng1.wang@intel.com>
^ permalink raw reply [relevance 0%]
* [dpdk-dev] [PATCH v3 0/8] eal: cleanup resources on shutdown
2020-04-25 19:34 0% ` David Marchand
2020-04-28 23:14 3% ` [dpdk-dev] [PATCH v2 0/9] eal: " Stephen Hemminger
@ 2020-04-28 23:58 3% ` Stephen Hemminger
2020-05-03 17:21 0% ` David Marchand
2 siblings, 1 reply; 200+ results
From: Stephen Hemminger @ 2020-04-28 23:58 UTC (permalink / raw)
To: dev; +Cc: Stephen Hemminger
Started using valgrind with DPDK, and there are lots of leftover
memory and file descriptors. This makes it hard to find application
leaks versus DPDK leaks.
The DPDK has a function that applications can use to tell it
to cleanup resources on shutdown (rte_eal_cleanup). But the
current coverage of that API is spotty. Many internal parts of
DPDK leave files and allocated memory behind.
This patch set is a first step at getting the sub-parts of
DPDK to cleanup after themselves. These are the easier ones,
the harder and more critical ones are in the drivers
and the memory subsystem.
There are no new exposed API or ABI changes here.
v3
- fix a couple of minor checkpatch complaints
v2
- rebase after 20.05 file renames
- incorporate review comment feedback
- hold off some of the more involved patches for later
Stephen Hemminger (8):
eal: log: close on cleanup
eal: log: free dynamic state on cleanup
eal: alarm: close file on cleanup
eal: cleanup threads
eal: mp: end the multiprocess thread during cleanup
eal: vfio: cleanup the mp sync handle
eal: hotplug: cleanup multiprocess resources
eal: malloc: cleanup mp resources
lib/librte_eal/common/eal_common_log.c | 33 ++++++++++++++++++++++++-
lib/librte_eal/common/eal_common_proc.c | 17 +++++++++----
lib/librte_eal/common/eal_private.h | 20 +++++++++++++++
lib/librte_eal/common/hotplug_mp.c | 5 ++++
lib/librte_eal/common/hotplug_mp.h | 6 +++++
lib/librte_eal/common/malloc_heap.c | 6 +++++
lib/librte_eal/common/malloc_heap.h | 3 +++
lib/librte_eal/common/malloc_mp.c | 12 +++++++++
lib/librte_eal/common/malloc_mp.h | 3 +++
lib/librte_eal/freebsd/eal.c | 1 +
lib/librte_eal/freebsd/eal_alarm.c | 10 ++++++++
lib/librte_eal/linux/eal.c | 20 +++++++++++++++
lib/librte_eal/linux/eal_alarm.c | 11 +++++++++
lib/librte_eal/linux/eal_log.c | 14 +++++++++++
lib/librte_eal/linux/eal_vfio.h | 1 +
lib/librte_eal/linux/eal_vfio_mp_sync.c | 8 ++++++
16 files changed, 164 insertions(+), 6 deletions(-)
--
2.20.1
^ permalink raw reply [relevance 3%]
* [dpdk-dev] [PATCH v4 1/8] eal: replace rte_page_sizes with a set of constants
@ 2020-04-28 23:50 4% ` Dmitry Kozlyuk
0 siblings, 0 replies; 200+ results
From: Dmitry Kozlyuk @ 2020-04-28 23:50 UTC (permalink / raw)
To: dev
Cc: Dmitry Malloy (MESHCHANINOV),
Narcisa Ana Maria Vasile, Fady Bader, Tal Shnaiderman,
Dmitry Kozlyuk, Jerin Jacob, John McNamara, Marko Kovacevic,
Anatoly Burakov
Clang on Windows follows MS ABI where enum values are limited to 2^31-1.
Enum rte_page_sizes has members valued above this limit, which get
wrapped to zero, resulting in compilation error (duplicate values in
enum). Using MS ABI is mandatory for Windows EAL to call Win32 APIs.
Remove rte_page_sizes and replace its values with #define's.
This enumeration is not used in public API, so there's no ABI breakage.
Document API change in release notes.
Suggested-by: Jerin Jacob <jerinjacobk@gmail.com>
Signed-off-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
---
doc/guides/rel_notes/release_20_05.rst | 2 ++
lib/librte_eal/include/rte_memory.h | 23 ++++++++++-------------
2 files changed, 12 insertions(+), 13 deletions(-)
diff --git a/doc/guides/rel_notes/release_20_05.rst b/doc/guides/rel_notes/release_20_05.rst
index b124c3f28..76ba59220 100644
--- a/doc/guides/rel_notes/release_20_05.rst
+++ b/doc/guides/rel_notes/release_20_05.rst
@@ -225,6 +225,8 @@ Removed Items
Also, make sure to start the actual text at the margin.
=========================================================
+* ``enum rte_page_sizes`` is removed, ``RTE_PGSIZE_*`` constants are kept.
+
API Changes
-----------
diff --git a/lib/librte_eal/include/rte_memory.h b/lib/librte_eal/include/rte_memory.h
index 3d8d0bd69..65374d53a 100644
--- a/lib/librte_eal/include/rte_memory.h
+++ b/lib/librte_eal/include/rte_memory.h
@@ -24,19 +24,16 @@ extern "C" {
#include <rte_config.h>
#include <rte_fbarray.h>
-__extension__
-enum rte_page_sizes {
- RTE_PGSIZE_4K = 1ULL << 12,
- RTE_PGSIZE_64K = 1ULL << 16,
- RTE_PGSIZE_256K = 1ULL << 18,
- RTE_PGSIZE_2M = 1ULL << 21,
- RTE_PGSIZE_16M = 1ULL << 24,
- RTE_PGSIZE_256M = 1ULL << 28,
- RTE_PGSIZE_512M = 1ULL << 29,
- RTE_PGSIZE_1G = 1ULL << 30,
- RTE_PGSIZE_4G = 1ULL << 32,
- RTE_PGSIZE_16G = 1ULL << 34,
-};
+#define RTE_PGSIZE_4K (1ULL << 12)
+#define RTE_PGSIZE_64K (1ULL << 16)
+#define RTE_PGSIZE_256K (1ULL << 18)
+#define RTE_PGSIZE_2M (1ULL << 21)
+#define RTE_PGSIZE_16M (1ULL << 24)
+#define RTE_PGSIZE_256M (1ULL << 28)
+#define RTE_PGSIZE_512M (1ULL << 29)
+#define RTE_PGSIZE_1G (1ULL << 30)
+#define RTE_PGSIZE_4G (1ULL << 32)
+#define RTE_PGSIZE_16G (1ULL << 34)
#define SOCKET_ID_ANY -1 /**< Any NUMA socket. */
--
2.25.1
^ permalink raw reply [relevance 4%]
* [dpdk-dev] [PATCH v2 0/9] eal: cleanup resources on shutdown
2020-04-25 19:34 0% ` David Marchand
@ 2020-04-28 23:14 3% ` Stephen Hemminger
2020-04-28 23:58 3% ` [dpdk-dev] [PATCH v3 0/8] " Stephen Hemminger
2 siblings, 0 replies; 200+ results
From: Stephen Hemminger @ 2020-04-28 23:14 UTC (permalink / raw)
To: dev; +Cc: Stephen Hemminger
Started using valgrind with DPDK, and there are lots of leftover
memory and file descriptors. This makes it hard to find application
leaks versus DPDK leaks.
The DPDK has a function that applications can use to tell it
to cleanup resources on shutdown (rte_eal_cleanup). But the
current coverage of that API is spotty. Many internal parts of
DPDK leave files and allocated memory behind.
This patch set is a first step at getting the sub-parts of
DPDK to cleanup after themselves. These are the easier ones,
the harder and more critical ones are in the drivers
and the memory subsystem.
There are no new exposed API or ABI changes here.
v2
- rebase after 20.05 file renames
- incorporate review comment feedback
- hold off some of the more involved patches for later
Stephen Hemminger (8):
eal: log: close on cleanup
eal: log: free dynamic state on cleanup
eal: alarm: close file on cleanup
eal: cleanup threads
eal: mp: end the multiprocess thread during cleanup
eal: vfio: cleanup the mp sync handle
eal: hotplug: cleanup multiprocess resources
eal: malloc: cleanup mp resources
lib/librte_eal/common/eal_common_log.c | 33 ++++++++++++++++++++++++-
lib/librte_eal/common/eal_common_proc.c | 17 +++++++++----
lib/librte_eal/common/eal_private.h | 20 +++++++++++++++
lib/librte_eal/common/hotplug_mp.c | 5 ++++
lib/librte_eal/common/hotplug_mp.h | 6 +++++
lib/librte_eal/common/malloc_heap.c | 6 +++++
lib/librte_eal/common/malloc_heap.h | 3 +++
lib/librte_eal/common/malloc_mp.c | 12 +++++++++
lib/librte_eal/common/malloc_mp.h | 3 +++
lib/librte_eal/freebsd/eal.c | 1 +
lib/librte_eal/freebsd/eal_alarm.c | 10 ++++++++
lib/librte_eal/linux/eal.c | 20 +++++++++++++++
lib/librte_eal/linux/eal_alarm.c | 11 +++++++++
lib/librte_eal/linux/eal_log.c | 14 +++++++++++
lib/librte_eal/linux/eal_vfio.h | 1 +
lib/librte_eal/linux/eal_vfio_mp_sync.c | 8 ++++++
16 files changed, 164 insertions(+), 6 deletions(-)
--
2.20.1
^ permalink raw reply [relevance 3%]
* Re: [dpdk-dev] [PATCH v5] eal/cpuflags: add x86 based cpu flags
2020-04-28 18:11 4% ` Laatz, Kevin
@ 2020-04-28 19:55 0% ` Thomas Monjalon
0 siblings, 0 replies; 200+ results
From: Thomas Monjalon @ 2020-04-28 19:55 UTC (permalink / raw)
To: Ray Kinsella, Laatz, Kevin
Cc: dev, Richardson, Bruce, Van Haaren, Harry, Kinsella, Ray,
nhorman, david.marchand
28/04/2020 20:11, Laatz, Kevin:
> > > --- a/devtools/libabigail.abignore
> > > +++ b/devtools/libabigail.abignore
> >
> > Kevin - you still have the surpession.
> > I am testing locally with 1.7.1, and it doesn't complain when I disable the
> > supression.
> > Are you seeing something different?
> >
>
> Ray,
> I have re-tested and with libabigail 1.6 and it reports the addition of the flags as an ABI break without the abignore suppression.
> With the suppression, it will still report changes to existing flags (e.g. inserting a new flag somewhere in the middle) in the enum as an ABI break, as expected.
>
> The Travis CI is also based on Ubuntu 18.04 LTS, which uses libabigail 1.2-1. Without the suppression the community Travis CI builds fail on this false positive.
I think Travis uses libabigail 1.6:
http://git.dpdk.org/dpdk/tree/.ci/linux-build.sh#n61
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH v5] eal/cpuflags: add x86 based cpu flags
@ 2020-04-28 18:11 4% ` Laatz, Kevin
2020-04-28 19:55 0% ` Thomas Monjalon
2020-04-29 11:39 3% ` David Marchand
1 sibling, 1 reply; 200+ results
From: Laatz, Kevin @ 2020-04-28 18:11 UTC (permalink / raw)
To: Ray Kinsella, dev
Cc: Richardson, Bruce, Van Haaren, Harry, thomas, Kinsella, Ray,
nhorman, david.marchand
<snip>
> > ---
> > devtools/libabigail.abignore | 5 +++++
> > lib/librte_eal/x86/include/rte_cpuflags.h | 19 +++++++++++++++++++
> > lib/librte_eal/x86/rte_cpuflags.c | 18 ++++++++++++++++++
> > 3 files changed, 42 insertions(+)
> >
> > diff --git a/devtools/libabigail.abignore
> > b/devtools/libabigail.abignore index a59df8f13..045f436fb 100644
> > --- a/devtools/libabigail.abignore
> > +++ b/devtools/libabigail.abignore
>
> Kevin - you still have the surpession.
> I am testing locally with 1.7.1, and it doesn't complain when I disable the
> supression.
> Are you seeing something different?
>
Ray,
I have re-tested and with libabigail 1.6 and it reports the addition of the flags as an ABI break without the abignore suppression.
With the suppression, it will still report changes to existing flags (e.g. inserting a new flag somewhere in the middle) in the enum as an ABI break, as expected.
The Travis CI is also based on Ubuntu 18.04 LTS, which uses libabigail 1.2-1. Without the suppression the community Travis CI builds fail on this false positive.
-Kevin
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH v4 1/4] ethdev: add tm support for shaper config in pkt mode
2020-04-28 15:30 4% ` Thomas Monjalon
@ 2020-04-28 17:35 0% ` Dumitrescu, Cristian
0 siblings, 0 replies; 200+ results
From: Dumitrescu, Cristian @ 2020-04-28 17:35 UTC (permalink / raw)
To: Thomas Monjalon
Cc: Yigit, Ferruh, Nithin Dabilpuram, Singh, Jasvinder,
Andrew Rybchenko, dev, jerinj, kkanas, Nithin Dabilpuram,
Kinsella, Ray, Neil Horman, Luca Boccassi, Kevin Traynor,
David Marchand, Richardson, Bruce
> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: Tuesday, April 28, 2020 4:30 PM
> To: Dumitrescu, Cristian <cristian.dumitrescu@intel.com>
> Cc: Yigit, Ferruh <ferruh.yigit@intel.com>; Nithin Dabilpuram
> <nithind1988@gmail.com>; Singh, Jasvinder <jasvinder.singh@intel.com>;
> Andrew Rybchenko <arybchenko@solarflare.com>; dev@dpdk.org;
> jerinj@marvell.com; kkanas@marvell.com; Nithin Dabilpuram
> <ndabilpuram@marvell.com>; Kinsella, Ray <ray.kinsella@intel.com>; Neil
> Horman <nhorman@tuxdriver.com>; Luca Boccassi <bluca@debian.org>;
> Kevin Traynor <ktraynor@redhat.com>; David Marchand
> <david.marchand@redhat.com>; Richardson, Bruce
> <bruce.richardson@intel.com>
> Subject: Re: [PATCH v4 1/4] ethdev: add tm support for shaper config in pkt
> mode
>
> 27/04/2020 18:28, Dumitrescu, Cristian:
> > From: Yigit, Ferruh <ferruh.yigit@intel.com>
> > > On 4/27/2020 10:19 AM, Dumitrescu, Cristian wrote:
> > > > From: Yigit, Ferruh <ferruh.yigit@intel.com>
> > > >> Hi Nithin,
> > > >>
> > > >> It looks like patch is causing ABI break, I am getting following warning
> [1],
> > > >> can you please check?
> > > >>
> > > >> [1]
> > > >> https://pastebin.com/XYNFg14u
> > > >
> > > >
> > > > Hi Ferruh,
> > > >
> > > > The RTE_TM API is marked as experimental, but it looks that this was
> not
> > > correctly marked when __rte_experimental ABI checker was introduced.
> > > >
> > > > It is marked as experimental at the top of the rte_tm.h, similarly to
> other
> > > APIs introduced around same time, but it was not correctly picked up by
> the
> > > ABI check procedure when later introduced, so __rte_experimental was
> not
> > > added to every function.
> > > >
> > >
> > > :(
> > >
> > > Is it time to mature them?
> > >
> > > As you said they are not marked as experimental both in header file
> > > (function
> > > declarations) and .map file.
> > >
> > > The problem is, they are not marked as experimental in DPDK_20.0 ABI
> > > (v19.11),
> > > so marking them as experimental now will break the ABI. Not sure what
> to
> > > do,
> > > cc'ed a few ABI related names for comment.
> > >
> > > For me, we need to proceed as the experimental tag removed and APIs
> > > become
> > > mature starting from v19.11, since this is what happened in practice, and
> > > remove
> > > a few existing being experimental references in the doxygen comments.
> > >
> > > Ray, Neil, David, Luca, Kevin, what do you think?
> >
> > Hi Ferruh,
> >
> > IMO your proposed approach is fixing the wrong problem and is
> > probably not the right way of doing things.
> >
> > This API is correctly marked as experimental in the header
> > file rte_tm.h and in the MAINTAINERS file,
>
> in rte_tm.h:
> * @warning
> * @b EXPERIMENTAL: this API may change without prior notice
>
> in MAINTAINERS:
> Traffic Management API - EXPERIMENTAL
> M: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
> T: git://dpdk.org/next/dpdk-next-qos
> F: lib/librte_ethdev/rte_tm*
>
>
> > therefore it should remain experimental,
>
> in rte_ethdev_version.map:
> before 19.11: DPDK_17.08 {
> since 19.11: DPDK_20.0 {
>
> When adding rte_tm in 17.08:
> http://git.dpdk.org/dpdk/diff/lib/librte_ether/rte_ether_version.map?id=5
> d109deffa
>
> In 17.08, early July 2017, the first EXPERIMENTAL section was declared in EAL:
> http://git.dpdk.org/dpdk/diff/lib/librte_eal/linuxapp/eal/rte_eal_version.m
> ap?id=a3ee360f444
>
> When adding rte_mtr in 17.11, all functions are made experimental:
> http://git.dpdk.org/dpdk/diff/lib/librte_ether/rte_ethdev_version.map?id=
> 6613ffe1
>
> In 18.02, the tag __rte_experimental was introduced:
> http://git.dpdk.org/dpdk/commit/?id=7d540a3e735
> and functions are marked (including rte_mtr but not rte_tm):
> http://git.dpdk.org/dpdk/commit/?id=77b7b81e32e
>
>
Thanks, Thomas, for taking the time to put all the historic events in their proper sequence.
> > as more changes are expected from the people like Nithin and others
> > currently upstreaming drivers for it.
>
> They are doing changes in the API introduced 3 years ago.
>
>
> > For some reason, the __rte_experimental tags were not added to
> > this file when the ABI checker was introduced,
> > and this is the real problem that should be fixed.
>
> The mistake was done 2 years ago.
> As maintainer of rte_tm code, you are expected to notice the issue.
Apologies for not noticing this gap at the time. As Thomas shows above, the __rte_experimental tags were introduced several DPDK releases after the rte_tm API.
> As maintainer of rte_mtr code, you were expected to review the change
> on the mtr functions being marked as experimental.
I did.
> I am sorry that it took 2 years to discover the gap.
>
> We can fix the ABI in the ABI-breakage window: in 20.11.
>
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH v4 1/4] ethdev: add tm support for shaper config in pkt mode
2020-04-28 15:04 0% ` Luca Boccassi
@ 2020-04-28 15:54 5% ` Thomas Monjalon
2020-04-29 8:45 0% ` Dumitrescu, Cristian
2020-05-01 13:18 3% ` [dpdk-dev] " Jerin Jacob
0 siblings, 2 replies; 200+ results
From: Thomas Monjalon @ 2020-04-28 15:54 UTC (permalink / raw)
To: Jerin Jacob, Dumitrescu, Cristian
Cc: Bruce Richardson, Ferruh Yigit, Luca Boccassi, Nithin Dabilpuram,
Singh, Jasvinder, Andrew Rybchenko, dev, jerinj, kkanas,
Nithin Dabilpuram, Kinsella, Ray, Neil Horman, Kevin Traynor,
David Marchand
28/04/2020 17:04, Luca Boccassi:
> On Tue, 2020-04-28 at 15:45 +0100, Bruce Richardson wrote:
> > On Tue, Apr 28, 2020 at 03:06:20PM +0100, Ferruh Yigit wrote:
> > > On 4/27/2020 5:59 PM, Jerin Jacob wrote:
> > > > On Mon, Apr 27, 2020 at 10:19 PM Ferruh Yigit <ferruh.yigit@intel.com> wrote:
> > > > > On 4/27/2020 5:29 PM, Jerin Jacob wrote:
> > > > > > On Mon, Apr 27, 2020 at 9:42 PM Ferruh Yigit <ferruh.yigit@intel.com> wrote:
> > > > > > > On 4/27/2020 10:19 AM, Dumitrescu, Cristian wrote:
> > > > > > > > From: Yigit, Ferruh <ferruh.yigit@intel.com>
> > > > > > > > > On 4/24/2020 11:28 AM, Dumitrescu, Cristian wrote:
> > > > > > > > > > From: Nithin Dabilpuram <nithind1988@gmail.com>
> > > > > > > > > > > This patch also updates tm port/level/node capability structures with
> > > > > > > > > > > exiting features of scheduler wfq packet mode, scheduler wfq byte mode
> > > > > > > > > > > and private/shared shaper byte mode.
> > > > > > > > > > >
> > > > > > > > > > > SoftNIC PMD is also updated with new capabilities.
[...]
> > > > > > > > > Hi Nithin,
> > > > > > > > >
> > > > > > > > > It looks like patch is causing ABI break, I am getting following warning [1],
> > > > > > > > > can you please check?
> > > > > > > > >
> > > > > > > > > [1]
> > > > > > > > > https://pastebin.com/XYNFg14u
> > > > > > > >
> > > > > > > > Hi Ferruh,
> > > > > > > >
> > > > > > > > The RTE_TM API is marked as experimental,
> > > > > > > > but it looks that this was not correctly marked
> > > > > > > > when __rte_experimental ABI checker was introduced.
> > > > > > > >
> > > > > > > > It is marked as experimental at the top of the rte_tm.h,
> > > > > > > > similarly to other APIs introduced around same time,
> > > > > > > > but it was not correctly picked up by the ABI check procedure
> > > > > > > > when later introduced, so __rte_experimental was not added to every function.
> > > > > > > >
> > > > > > >
> > > > > > > :(
> > > > > > >
> > > > > > > Is it time to mature them?
> > > > > > >
> > > > > > > As you said they are not marked as experimental both in header file (function
> > > > > > > declarations) and .map file.
> > > > > > >
> > > > > > > The problem is, they are not marked as experimental in DPDK_20.0 ABI (v19.11),
> > > > > > > so marking them as experimental now will break the ABI. Not sure what to do,
> > > > > > > cc'ed a few ABI related names for comment.
> > > > > > >
> > > > > > > For me, we need to proceed as the experimental tag removed and APIs become
> > > > > > > mature starting from v19.11, since this is what happened in practice, and remove
> > > > > > > a few existing being experimental references in the doxygen comments.
> > > > > >
> > > > > > I think, accidentally we can not make a library as NON-experimental.
> > > > > > TM never went through experimental to mature transition(see git log
> > > > > > lib/librte_ethdev/rte_tm.h)
> > > > > > It was a bug to not mark as experimental in each function in the ABI process.
> > > > > > Some of the features like packet marking are not even implemented by any HW.
> > > > > > I think, we can make API stable only all the features are implemented
> > > > > > by one or two HW.
Yes this is what was decided one or two years ago I think.
But rte_tm API was introduced 3 years ago and is implemented by 6 PMDs.
> > > > > Fair enough, specially if the API is not ready yet.
> > > > >
> > > > > But they were part of stable ABI, and marking them as experimental now will
> > > > > break the old applications using these APIs.
> > > >
> > > > it is still marked as EXPERIMENTAL everywhere and API is not ready yet.
rte_tm is implemented in 6 PMDs.
> > > Existing experimental marks are text only for human parsing.
> > >
> > > The compiler attribute and build time checks are missing, and the symbol in the
> > > binary doesn't have experimental tag. Our scripts and automated checks won't
> > > detect it as experimental.
> > >
> > > My point is just having experimental comment in header file is not enough to
> > > qualify the APIs as experimental.
> > >
> > > > Anyway, we need to break the ABI to make it work on various HW.
Yes this is why I was asking in 19.11 to check our API,
in order to avoid such situation.
> > > > I am not sure what to do?
Either manage ABI versioning, or wait 20.11.
> > > > IMO, We need to send a patch as Fixes: for the bug of not adding
> > > > __rte_experimental in each function.
No, this is wrong.
> > > Yes, this is where we are, both you and Cristian suggest API is not ready and
> > > should be experimental, but they were part of stable ABI, making them
> > > experimental will break the ABI.
> > > It looks like there is no good option but we should select one of the bad ones.
> > >
> > > > Traffic Management API - EXPERIMENTAL
> > > > M: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
> > > > T: git://dpdk.org/next/dpdk-next-qos
> > > > F: lib/librte_ethdev/rte_tm*
> > > > > > > Ray, Neil, David, Luca, Kevin, what do you think?
> > While I'm not called any of those names, allow me to give my 2c.
> >
> > Since these are marked in binaries as part of the stable ABI, I think we
> > need to honour that for the next two releases 20.05 and 20.08 [which means
> > that we need to put in versioned functions for any changes, not that we
> > can't change anything]
> >
> > For 20.11, I think these should then have one of two options taken:
> > * have these "fixed" and ready to be marked as stable, and officially part
> > of v21 ABI or
> > * mark them as experimental properly, and look to have them as part of the
> > v22 or subsequent ABI
> >
> > Given the comments here, I would tend towards the latter of the above two
> > options, but that's really a decision for the maintainers.
> >
> > Remember, this is not the first bug we have encountered where we messed up
> > some ABI versions in the 19.11 release, and, like the previous one with the
> > screwed up version number, I think we need to honour the ABI committments
> > made, especially since in this case it's only for a few more months till
> > 20.11 development starts.
> >
> > /Bruce
>
> +1
>
> If they are not ready now, they haven't been ready for the past 6
> months either, so staying not ready for 6 more is the lesser evil.
This API is almost 3 years old (release 17.08).
That's good to improve it but we must respect the ABI contract that
we all agreed.
Summary:
17.08: rte_tm is introduced.
17.11: rte_mtr is introduced as experimental, but rte_tm remains stable.
18.02: __rte_experimental tag is introduced (including for rte_mtr),
but rte_tm remains untouched as it is in stable ABI.
19.11: stable ABI is frozen until 20.11
20.05: rte_tm improvement is blocked because of ABI breakage.
It should remind everybody of reviewing the new API and policies,
and maintaining the existing code appropriately.
^ permalink raw reply [relevance 5%]
* Re: [dpdk-dev] [PATCH v4 1/4] ethdev: add tm support for shaper config in pkt mode
2020-04-28 14:45 5% ` Bruce Richardson
2020-04-28 15:04 0% ` Luca Boccassi
@ 2020-04-28 15:42 3% ` Ray Kinsella
1 sibling, 0 replies; 200+ results
From: Ray Kinsella @ 2020-04-28 15:42 UTC (permalink / raw)
To: Bruce Richardson, Ferruh Yigit
Cc: Jerin Jacob, Dumitrescu, Cristian, Nithin Dabilpuram, Singh,
Jasvinder, Thomas Monjalon, Andrew Rybchenko, dev, jerinj,
kkanas, Nithin Dabilpuram, Kinsella, Ray, Neil Horman,
Luca Boccassi, Kevin Traynor, David Marchand
On 28/04/2020 15:45, Bruce Richardson wrote:
> On Tue, Apr 28, 2020 at 03:06:20PM +0100, Ferruh Yigit wrote:
>> On 4/27/2020 5:59 PM, Jerin Jacob wrote:
>>> On Mon, Apr 27, 2020 at 10:19 PM Ferruh Yigit <ferruh.yigit@intel.com> wrote:
>>>>
>>>> On 4/27/2020 5:29 PM, Jerin Jacob wrote:
>>>>> On Mon, Apr 27, 2020 at 9:42 PM Ferruh Yigit <ferruh.yigit@intel.com> wrote:
>>>>>>
>>>>>> On 4/27/2020 10:19 AM, Dumitrescu, Cristian wrote:
>>>>>>>
>>>>>>>
>>>>>>>> -----Original Message-----
>>>>>>>> From: Yigit, Ferruh <ferruh.yigit@intel.com>
>>>>>>>> Sent: Saturday, April 25, 2020 9:09 PM
>>>>>>>> To: Dumitrescu, Cristian <cristian.dumitrescu@intel.com>; Nithin Dabilpuram
>>>>>>>> <nithind1988@gmail.com>; Singh, Jasvinder <jasvinder.singh@intel.com>;
>>>>>>>> Thomas Monjalon <thomas@monjalon.net>; Andrew Rybchenko
>>>>>>>> <arybchenko@solarflare.com>
>>>>>>>> Cc: dev@dpdk.org; jerinj@marvell.com; kkanas@marvell.com; Nithin
>>>>>>>> Dabilpuram <ndabilpuram@marvell.com>
>>>>>>>> Subject: Re: [PATCH v4 1/4] ethdev: add tm support for shaper config in pkt
>>>>>>>> mode
>>>>>>>>
>>>>>>>> On 4/24/2020 11:28 AM, Dumitrescu, Cristian wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> -----Original Message-----
>>>>>>>>>> From: Nithin Dabilpuram <nithind1988@gmail.com>
>>>>>>>>>> Sent: Wednesday, April 22, 2020 6:21 PM
>>>>>>>>>> To: Singh, Jasvinder <jasvinder.singh@intel.com>; Dumitrescu, Cristian
>>>>>>>>>> <cristian.dumitrescu@intel.com>; Thomas Monjalon
>>>>>>>>>> <thomas@monjalon.net>; Yigit, Ferruh <ferruh.yigit@intel.com>; Andrew
>>>>>>>>>> Rybchenko <arybchenko@solarflare.com>
>>>>>>>>>> Cc: dev@dpdk.org; jerinj@marvell.com; kkanas@marvell.com; Nithin
>>>>>>>>>> Dabilpuram <ndabilpuram@marvell.com>
>>>>>>>>>> Subject: [PATCH v4 1/4] ethdev: add tm support for shaper config in pkt
>>>>>>>>>> mode
>>>>>>>>>>
>>>>>>>>>> From: Nithin Dabilpuram <ndabilpuram@marvell.com>
>>>>>>>>>>
>>>>>>>>>> Some NIC hardware support shaper to work in packet mode i.e
>>>>>>>>>> shaping or ratelimiting traffic is in packets per second (PPS) as
>>>>>>>>>> opposed to default bytes per second (BPS). Hence this patch
>>>>>>>>>> adds support to configure shared or private shaper in packet mode,
>>>>>>>>>> provide rate in PPS and add related tm capabilities in port/level/node
>>>>>>>>>> capability structures.
>>>>>>>>>>
>>>>>>>>>> This patch also updates tm port/level/node capability structures with
>>>>>>>>>> exiting features of scheduler wfq packet mode, scheduler wfq byte mode
>>>>>>>>>> and private/shared shaper byte mode.
>>>>>>>>>>
>>>>>>>>>> SoftNIC PMD is also updated with new capabilities.
>>>>>>>>>>
>>>>>>>>>> Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>
>>>>>>>>>> ---
>>>>>>>>>> v3..v4:
>>>>>>>>>> - Update text under packet_mode as per Cristian.
>>>>>>>>>> - Update rte_eth_softnic_tm.c based on Jasvinder's comments.
>>>>>>>>>> - Add error enum
>>>>>>>> RTE_TM_ERROR_TYPE_SHAPER_PROFILE_PACKET_MODE
>>>>>>>>>> - Fix shaper_profile_check() with packet mode check
>>>>>>>>>> - Fix typo's
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Acked-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
>>>>>>>>>
>>>>>>>>
>>>>>>>> Hi Nithin,
>>>>>>>>
>>>>>>>> It looks like patch is causing ABI break, I am getting following warning [1],
>>>>>>>> can you please check?
>>>>>>>>
>>>>>>>> [1]
>>>>>>>> https://pastebin.com/XYNFg14u
>>>>>>>
>>>>>>>
>>>>>>> Hi Ferruh,
>>>>>>>
>>>>>>> The RTE_TM API is marked as experimental, but it looks that this was not correctly marked when __rte_experimental ABI checker was introduced.
>>>>>>>
>>>>>>> It is marked as experimental at the top of the rte_tm.h, similarly to other APIs introduced around same time, but it was not correctly picked up by the ABI check procedure when later introduced, so __rte_experimental was not added to every function.
>>>>>>>
>>>>>>
>>>>>> :(
>>>>>>
>>>>>> Is it time to mature them?
>>>>>>
>>>>>> As you said they are not marked as experimental both in header file (function
>>>>>> declarations) and .map file.
>>>>>>
>>>>>> The problem is, they are not marked as experimental in DPDK_20.0 ABI (v19.11),
>>>>>> so marking them as experimental now will break the ABI. Not sure what to do,
>>>>>> cc'ed a few ABI related names for comment.
>>>>>>
>>>>>> For me, we need to proceed as the experimental tag removed and APIs become
>>>>>> mature starting from v19.11, since this is what happened in practice, and remove
>>>>>> a few existing being experimental references in the doxygen comments.
>>>>>
>>>>> I think, accidentally we can not make a library as NON-experimental.
>>>>> TM never went through experimental to mature transition(see git log
>>>>> lib/librte_ethdev/rte_tm.h)
>>>>> It was a bug to not mark as experimental in each function in the ABI process.
>>>>> Some of the features like packet marking are not even implemented by any HW.
>>>>> I think, we can make API stable only all the features are implemented
>>>>> by one or two HW.
>>>>
>>>> Fair enough, specially if the API is not ready yet.
>>>>
>>>> But they were part of stable ABI, and marking them as experimental now will
>>>> break the old applications using these APIs.
>>>
>>> it is still marked as EXPERIMENTAL everywhere and API is not ready yet.
>>
>> Existing experimental marks are text only for human parsing.
>>
>> The compiler attribute and build time checks are missing, and the symbol in the
>> binary doesn't have experimental tag. Our scripts and automated checks won't
>> detect it as experimental.
>>
>> My point is just having experimental comment in header file is not enough to
>> qualify the APIs as experimental.
>>
>>> Anyway, we need to break the ABI to make it work on various HW.
>>> I am not sure what to do?
>>> IMO, We need to send a patch as Fixes: for the bug of not adding
>>> __rte_experimental in each function.
>>
>> Yes, this is where we are, both you and Cristian suggest API is not ready and
>> should be experimental, but they were part of stable ABI, making them
>> experimental will break the ABI.
>> It looks like there is no good option but we should select one of the bad ones.
>>
>>>
>>> Traffic Management API - EXPERIMENTAL
>>> M: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
>>> T: git://dpdk.org/next/dpdk-next-qos
>>> F: lib/librte_ethdev/rte_tm*
>>>>
>>>>>
>>>>>>
>>>>>> Ray, Neil, David, Luca, Kevin, what do you think?
>>>>
> While I'm not called any of those names, allow me to give my 2c.
>
> Since these are marked in binaries as part of the stable ABI, I think we
> need to honour that for the next two releases 20.05 and 20.08 [which means
> that we need to put in versioned functions for any changes, not that we
> can't change anything]
>
> For 20.11, I think these should then have one of two options taken:
> * have these "fixed" and ready to be marked as stable, and officially part
> of v21 ABI or
> * mark them as experimental properly, and look to have them as part of the
> v22 or subsequent ABI
>
> Given the comments here, I would tend towards the latter of the above two
> options, but that's really a decision for the maintainers.
>
> Remember, this is not the first bug we have encountered where we messed up
> some ABI versions in the 19.11 release, and, like the previous one with the
> screwed up version number, I think we need to honour the ABI committments
> made, especially since in this case it's only for a few more months till
> 20.11 development starts.
>
> /Bruce
>
So the rte_tm API has been "EXPERIMENTAL" for quiet a long time, as far back as v17.11.
To the extent it predates the experimental infrastructure the community developed
since then. TM added in 17.08, EXPERIMENTAL appears to have been added in 17.11.
That said, some form of stable TM API should have emerged by this point.
So I am not sure that EXPERIMENTAL status was 100% warranted.
As Bruce points out.
It is simpler to wait for the next ABI breakage window in August.
And mark them EXPERIMENTAL at that point.
Thanks,
Ray K
^ permalink raw reply [relevance 3%]
* Re: [dpdk-dev] [PATCH v4 1/4] ethdev: add tm support for shaper config in pkt mode
2020-04-27 16:28 3% ` Dumitrescu, Cristian
@ 2020-04-28 15:30 4% ` Thomas Monjalon
2020-04-28 17:35 0% ` Dumitrescu, Cristian
0 siblings, 1 reply; 200+ results
From: Thomas Monjalon @ 2020-04-28 15:30 UTC (permalink / raw)
To: Dumitrescu, Cristian
Cc: Yigit, Ferruh, Nithin Dabilpuram, Singh, Jasvinder,
Andrew Rybchenko, dev, jerinj, kkanas, Nithin Dabilpuram,
Kinsella, Ray, Neil Horman, Luca Boccassi, Kevin Traynor,
David Marchand, Richardson, Bruce
27/04/2020 18:28, Dumitrescu, Cristian:
> From: Yigit, Ferruh <ferruh.yigit@intel.com>
> > On 4/27/2020 10:19 AM, Dumitrescu, Cristian wrote:
> > > From: Yigit, Ferruh <ferruh.yigit@intel.com>
> > >> Hi Nithin,
> > >>
> > >> It looks like patch is causing ABI break, I am getting following warning [1],
> > >> can you please check?
> > >>
> > >> [1]
> > >> https://pastebin.com/XYNFg14u
> > >
> > >
> > > Hi Ferruh,
> > >
> > > The RTE_TM API is marked as experimental, but it looks that this was not
> > correctly marked when __rte_experimental ABI checker was introduced.
> > >
> > > It is marked as experimental at the top of the rte_tm.h, similarly to other
> > APIs introduced around same time, but it was not correctly picked up by the
> > ABI check procedure when later introduced, so __rte_experimental was not
> > added to every function.
> > >
> >
> > :(
> >
> > Is it time to mature them?
> >
> > As you said they are not marked as experimental both in header file
> > (function
> > declarations) and .map file.
> >
> > The problem is, they are not marked as experimental in DPDK_20.0 ABI
> > (v19.11),
> > so marking them as experimental now will break the ABI. Not sure what to
> > do,
> > cc'ed a few ABI related names for comment.
> >
> > For me, we need to proceed as the experimental tag removed and APIs
> > become
> > mature starting from v19.11, since this is what happened in practice, and
> > remove
> > a few existing being experimental references in the doxygen comments.
> >
> > Ray, Neil, David, Luca, Kevin, what do you think?
>
> Hi Ferruh,
>
> IMO your proposed approach is fixing the wrong problem and is
> probably not the right way of doing things.
>
> This API is correctly marked as experimental in the header
> file rte_tm.h and in the MAINTAINERS file,
in rte_tm.h:
* @warning
* @b EXPERIMENTAL: this API may change without prior notice
in MAINTAINERS:
Traffic Management API - EXPERIMENTAL
M: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
T: git://dpdk.org/next/dpdk-next-qos
F: lib/librte_ethdev/rte_tm*
> therefore it should remain experimental,
in rte_ethdev_version.map:
before 19.11: DPDK_17.08 {
since 19.11: DPDK_20.0 {
When adding rte_tm in 17.08:
http://git.dpdk.org/dpdk/diff/lib/librte_ether/rte_ether_version.map?id=5d109deffa
In 17.08, early July 2017, the first EXPERIMENTAL section was declared in EAL:
http://git.dpdk.org/dpdk/diff/lib/librte_eal/linuxapp/eal/rte_eal_version.map?id=a3ee360f444
When adding rte_mtr in 17.11, all functions are made experimental:
http://git.dpdk.org/dpdk/diff/lib/librte_ether/rte_ethdev_version.map?id=6613ffe1
In 18.02, the tag __rte_experimental was introduced:
http://git.dpdk.org/dpdk/commit/?id=7d540a3e735
and functions are marked (including rte_mtr but not rte_tm):
http://git.dpdk.org/dpdk/commit/?id=77b7b81e32e
> as more changes are expected from the people like Nithin and others
> currently upstreaming drivers for it.
They are doing changes in the API introduced 3 years ago.
> For some reason, the __rte_experimental tags were not added to
> this file when the ABI checker was introduced,
> and this is the real problem that should be fixed.
The mistake was done 2 years ago.
As maintainer of rte_tm code, you are expected to notice the issue.
As maintainer of rte_mtr code, you were expected to review the change
on the mtr functions being marked as experimental.
I am sorry that it took 2 years to discover the gap.
We can fix the ABI in the ABI-breakage window: in 20.11.
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH v4 1/4] ethdev: add tm support for shaper config in pkt mode
2020-04-28 14:45 5% ` Bruce Richardson
@ 2020-04-28 15:04 0% ` Luca Boccassi
2020-04-28 15:54 5% ` Thomas Monjalon
2020-04-28 15:42 3% ` Ray Kinsella
1 sibling, 1 reply; 200+ results
From: Luca Boccassi @ 2020-04-28 15:04 UTC (permalink / raw)
To: Bruce Richardson, Ferruh Yigit
Cc: Jerin Jacob, Dumitrescu, Cristian, Nithin Dabilpuram, Singh,
Jasvinder, Thomas Monjalon, Andrew Rybchenko, dev, jerinj,
kkanas, Nithin Dabilpuram, Kinsella, Ray, Neil Horman,
Kevin Traynor, David Marchand
On Tue, 2020-04-28 at 15:45 +0100, Bruce Richardson wrote:
> On Tue, Apr 28, 2020 at 03:06:20PM +0100, Ferruh Yigit wrote:
> > On 4/27/2020 5:59 PM, Jerin Jacob wrote:
> > > On Mon, Apr 27, 2020 at 10:19 PM Ferruh Yigit <ferruh.yigit@intel.com> wrote:
> > > > On 4/27/2020 5:29 PM, Jerin Jacob wrote:
> > > > > On Mon, Apr 27, 2020 at 9:42 PM Ferruh Yigit <ferruh.yigit@intel.com> wrote:
> > > > > > On 4/27/2020 10:19 AM, Dumitrescu, Cristian wrote:
> > > > > > >
> > > > > > > > -----Original Message-----
> > > > > > > > From: Yigit, Ferruh <ferruh.yigit@intel.com>
> > > > > > > > Sent: Saturday, April 25, 2020 9:09 PM
> > > > > > > > To: Dumitrescu, Cristian <cristian.dumitrescu@intel.com>; Nithin Dabilpuram
> > > > > > > > <nithind1988@gmail.com>; Singh, Jasvinder <jasvinder.singh@intel.com>;
> > > > > > > > Thomas Monjalon <thomas@monjalon.net>; Andrew Rybchenko
> > > > > > > > <arybchenko@solarflare.com>
> > > > > > > > Cc: dev@dpdk.org; jerinj@marvell.com; kkanas@marvell.com; Nithin
> > > > > > > > Dabilpuram <ndabilpuram@marvell.com>
> > > > > > > > Subject: Re: [PATCH v4 1/4] ethdev: add tm support for shaper config in pkt
> > > > > > > > mode
> > > > > > > >
> > > > > > > > On 4/24/2020 11:28 AM, Dumitrescu, Cristian wrote:
> > > > > > > > >
> > > > > > > > > > -----Original Message-----
> > > > > > > > > > From: Nithin Dabilpuram <nithind1988@gmail.com>
> > > > > > > > > > Sent: Wednesday, April 22, 2020 6:21 PM
> > > > > > > > > > To: Singh, Jasvinder <jasvinder.singh@intel.com>; Dumitrescu, Cristian
> > > > > > > > > > <cristian.dumitrescu@intel.com>; Thomas Monjalon
> > > > > > > > > > <thomas@monjalon.net>; Yigit, Ferruh <ferruh.yigit@intel.com>; Andrew
> > > > > > > > > > Rybchenko <arybchenko@solarflare.com>
> > > > > > > > > > Cc: dev@dpdk.org; jerinj@marvell.com; kkanas@marvell.com; Nithin
> > > > > > > > > > Dabilpuram <ndabilpuram@marvell.com>
> > > > > > > > > > Subject: [PATCH v4 1/4] ethdev: add tm support for shaper config in pkt
> > > > > > > > > > mode
> > > > > > > > > >
> > > > > > > > > > From: Nithin Dabilpuram <ndabilpuram@marvell.com>
> > > > > > > > > >
> > > > > > > > > > Some NIC hardware support shaper to work in packet mode i.e
> > > > > > > > > > shaping or ratelimiting traffic is in packets per second (PPS) as
> > > > > > > > > > opposed to default bytes per second (BPS). Hence this patch
> > > > > > > > > > adds support to configure shared or private shaper in packet mode,
> > > > > > > > > > provide rate in PPS and add related tm capabilities in port/level/node
> > > > > > > > > > capability structures.
> > > > > > > > > >
> > > > > > > > > > This patch also updates tm port/level/node capability structures with
> > > > > > > > > > exiting features of scheduler wfq packet mode, scheduler wfq byte mode
> > > > > > > > > > and private/shared shaper byte mode.
> > > > > > > > > >
> > > > > > > > > > SoftNIC PMD is also updated with new capabilities.
> > > > > > > > > >
> > > > > > > > > > Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>
> > > > > > > > > > ---
> > > > > > > > > > v3..v4:
> > > > > > > > > > - Update text under packet_mode as per Cristian.
> > > > > > > > > > - Update rte_eth_softnic_tm.c based on Jasvinder's comments.
> > > > > > > > > > - Add error enum
> > > > > > > > RTE_TM_ERROR_TYPE_SHAPER_PROFILE_PACKET_MODE
> > > > > > > > > > - Fix shaper_profile_check() with packet mode check
> > > > > > > > > > - Fix typo's
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > Acked-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
> > > > > > > > >
> > > > > > > >
> > > > > > > > Hi Nithin,
> > > > > > > >
> > > > > > > > It looks like patch is causing ABI break, I am getting following warning [1],
> > > > > > > > can you please check?
> > > > > > > >
> > > > > > > > [1]
> > > > > > > > https://pastebin.com/XYNFg14u
> > > > > > >
> > > > > > > Hi Ferruh,
> > > > > > >
> > > > > > > The RTE_TM API is marked as experimental, but it looks that this was not correctly marked when __rte_experimental ABI checker was introduced.
> > > > > > >
> > > > > > > It is marked as experimental at the top of the rte_tm.h, similarly to other APIs introduced around same time, but it was not correctly picked up by the ABI check procedure when later introduced, so __rte_experimental was not added to every function.
> > > > > > >
> > > > > >
> > > > > > :(
> > > > > >
> > > > > > Is it time to mature them?
> > > > > >
> > > > > > As you said they are not marked as experimental both in header file (function
> > > > > > declarations) and .map file.
> > > > > >
> > > > > > The problem is, they are not marked as experimental in DPDK_20.0 ABI (v19.11),
> > > > > > so marking them as experimental now will break the ABI. Not sure what to do,
> > > > > > cc'ed a few ABI related names for comment.
> > > > > >
> > > > > > For me, we need to proceed as the experimental tag removed and APIs become
> > > > > > mature starting from v19.11, since this is what happened in practice, and remove
> > > > > > a few existing being experimental references in the doxygen comments.
> > > > >
> > > > > I think, accidentally we can not make a library as NON-experimental.
> > > > > TM never went through experimental to mature transition(see git log
> > > > > lib/librte_ethdev/rte_tm.h)
> > > > > It was a bug to not mark as experimental in each function in the ABI process.
> > > > > Some of the features like packet marking are not even implemented by any HW.
> > > > > I think, we can make API stable only all the features are implemented
> > > > > by one or two HW.
> > > >
> > > > Fair enough, specially if the API is not ready yet.
> > > >
> > > > But they were part of stable ABI, and marking them as experimental now will
> > > > break the old applications using these APIs.
> > >
> > > it is still marked as EXPERIMENTAL everywhere and API is not ready yet.
> >
> > Existing experimental marks are text only for human parsing.
> >
> > The compiler attribute and build time checks are missing, and the symbol in the
> > binary doesn't have experimental tag. Our scripts and automated checks won't
> > detect it as experimental.
> >
> > My point is just having experimental comment in header file is not enough to
> > qualify the APIs as experimental.
> >
> > > Anyway, we need to break the ABI to make it work on various HW.
> > > I am not sure what to do?
> > > IMO, We need to send a patch as Fixes: for the bug of not adding
> > > __rte_experimental in each function.
> >
> > Yes, this is where we are, both you and Cristian suggest API is not ready and
> > should be experimental, but they were part of stable ABI, making them
> > experimental will break the ABI.
> > It looks like there is no good option but we should select one of the bad ones.
> >
> > > Traffic Management API - EXPERIMENTAL
> > > M: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
> > > T: git://dpdk.org/next/dpdk-next-qos
> > > F: lib/librte_ethdev/rte_tm*
> > > > > > Ray, Neil, David, Luca, Kevin, what do you think?
> While I'm not called any of those names, allow me to give my 2c.
>
> Since these are marked in binaries as part of the stable ABI, I think we
> need to honour that for the next two releases 20.05 and 20.08 [which means
> that we need to put in versioned functions for any changes, not that we
> can't change anything]
>
> For 20.11, I think these should then have one of two options taken:
> * have these "fixed" and ready to be marked as stable, and officially part
> of v21 ABI or
> * mark them as experimental properly, and look to have them as part of the
> v22 or subsequent ABI
>
> Given the comments here, I would tend towards the latter of the above two
> options, but that's really a decision for the maintainers.
>
> Remember, this is not the first bug we have encountered where we messed up
> some ABI versions in the 19.11 release, and, like the previous one with the
> screwed up version number, I think we need to honour the ABI committments
> made, especially since in this case it's only for a few more months till
> 20.11 development starts.
>
> /Bruce
+1
If they are not ready now, they haven't been ready for the past 6
months either, so staying not ready for 6 more is the lesser evil.
--
Kind regards,
Luca Boccassi
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH v4 1/4] ethdev: add tm support for shaper config in pkt mode
2020-04-28 14:06 4% ` [dpdk-dev] " Ferruh Yigit
@ 2020-04-28 14:45 5% ` Bruce Richardson
2020-04-28 15:04 0% ` Luca Boccassi
2020-04-28 15:42 3% ` Ray Kinsella
0 siblings, 2 replies; 200+ results
From: Bruce Richardson @ 2020-04-28 14:45 UTC (permalink / raw)
To: Ferruh Yigit
Cc: Jerin Jacob, Dumitrescu, Cristian, Nithin Dabilpuram, Singh,
Jasvinder, Thomas Monjalon, Andrew Rybchenko, dev, jerinj,
kkanas, Nithin Dabilpuram, Kinsella, Ray, Neil Horman,
Luca Boccassi, Kevin Traynor, David Marchand
On Tue, Apr 28, 2020 at 03:06:20PM +0100, Ferruh Yigit wrote:
> On 4/27/2020 5:59 PM, Jerin Jacob wrote:
> > On Mon, Apr 27, 2020 at 10:19 PM Ferruh Yigit <ferruh.yigit@intel.com> wrote:
> >>
> >> On 4/27/2020 5:29 PM, Jerin Jacob wrote:
> >>> On Mon, Apr 27, 2020 at 9:42 PM Ferruh Yigit <ferruh.yigit@intel.com> wrote:
> >>>>
> >>>> On 4/27/2020 10:19 AM, Dumitrescu, Cristian wrote:
> >>>>>
> >>>>>
> >>>>>> -----Original Message-----
> >>>>>> From: Yigit, Ferruh <ferruh.yigit@intel.com>
> >>>>>> Sent: Saturday, April 25, 2020 9:09 PM
> >>>>>> To: Dumitrescu, Cristian <cristian.dumitrescu@intel.com>; Nithin Dabilpuram
> >>>>>> <nithind1988@gmail.com>; Singh, Jasvinder <jasvinder.singh@intel.com>;
> >>>>>> Thomas Monjalon <thomas@monjalon.net>; Andrew Rybchenko
> >>>>>> <arybchenko@solarflare.com>
> >>>>>> Cc: dev@dpdk.org; jerinj@marvell.com; kkanas@marvell.com; Nithin
> >>>>>> Dabilpuram <ndabilpuram@marvell.com>
> >>>>>> Subject: Re: [PATCH v4 1/4] ethdev: add tm support for shaper config in pkt
> >>>>>> mode
> >>>>>>
> >>>>>> On 4/24/2020 11:28 AM, Dumitrescu, Cristian wrote:
> >>>>>>>
> >>>>>>>
> >>>>>>>> -----Original Message-----
> >>>>>>>> From: Nithin Dabilpuram <nithind1988@gmail.com>
> >>>>>>>> Sent: Wednesday, April 22, 2020 6:21 PM
> >>>>>>>> To: Singh, Jasvinder <jasvinder.singh@intel.com>; Dumitrescu, Cristian
> >>>>>>>> <cristian.dumitrescu@intel.com>; Thomas Monjalon
> >>>>>>>> <thomas@monjalon.net>; Yigit, Ferruh <ferruh.yigit@intel.com>; Andrew
> >>>>>>>> Rybchenko <arybchenko@solarflare.com>
> >>>>>>>> Cc: dev@dpdk.org; jerinj@marvell.com; kkanas@marvell.com; Nithin
> >>>>>>>> Dabilpuram <ndabilpuram@marvell.com>
> >>>>>>>> Subject: [PATCH v4 1/4] ethdev: add tm support for shaper config in pkt
> >>>>>>>> mode
> >>>>>>>>
> >>>>>>>> From: Nithin Dabilpuram <ndabilpuram@marvell.com>
> >>>>>>>>
> >>>>>>>> Some NIC hardware support shaper to work in packet mode i.e
> >>>>>>>> shaping or ratelimiting traffic is in packets per second (PPS) as
> >>>>>>>> opposed to default bytes per second (BPS). Hence this patch
> >>>>>>>> adds support to configure shared or private shaper in packet mode,
> >>>>>>>> provide rate in PPS and add related tm capabilities in port/level/node
> >>>>>>>> capability structures.
> >>>>>>>>
> >>>>>>>> This patch also updates tm port/level/node capability structures with
> >>>>>>>> exiting features of scheduler wfq packet mode, scheduler wfq byte mode
> >>>>>>>> and private/shared shaper byte mode.
> >>>>>>>>
> >>>>>>>> SoftNIC PMD is also updated with new capabilities.
> >>>>>>>>
> >>>>>>>> Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>
> >>>>>>>> ---
> >>>>>>>> v3..v4:
> >>>>>>>> - Update text under packet_mode as per Cristian.
> >>>>>>>> - Update rte_eth_softnic_tm.c based on Jasvinder's comments.
> >>>>>>>> - Add error enum
> >>>>>> RTE_TM_ERROR_TYPE_SHAPER_PROFILE_PACKET_MODE
> >>>>>>>> - Fix shaper_profile_check() with packet mode check
> >>>>>>>> - Fix typo's
> >>>>>>>>
> >>>>>>>
> >>>>>>> Acked-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
> >>>>>>>
> >>>>>>
> >>>>>> Hi Nithin,
> >>>>>>
> >>>>>> It looks like patch is causing ABI break, I am getting following warning [1],
> >>>>>> can you please check?
> >>>>>>
> >>>>>> [1]
> >>>>>> https://pastebin.com/XYNFg14u
> >>>>>
> >>>>>
> >>>>> Hi Ferruh,
> >>>>>
> >>>>> The RTE_TM API is marked as experimental, but it looks that this was not correctly marked when __rte_experimental ABI checker was introduced.
> >>>>>
> >>>>> It is marked as experimental at the top of the rte_tm.h, similarly to other APIs introduced around same time, but it was not correctly picked up by the ABI check procedure when later introduced, so __rte_experimental was not added to every function.
> >>>>>
> >>>>
> >>>> :(
> >>>>
> >>>> Is it time to mature them?
> >>>>
> >>>> As you said they are not marked as experimental both in header file (function
> >>>> declarations) and .map file.
> >>>>
> >>>> The problem is, they are not marked as experimental in DPDK_20.0 ABI (v19.11),
> >>>> so marking them as experimental now will break the ABI. Not sure what to do,
> >>>> cc'ed a few ABI related names for comment.
> >>>>
> >>>> For me, we need to proceed as the experimental tag removed and APIs become
> >>>> mature starting from v19.11, since this is what happened in practice, and remove
> >>>> a few existing being experimental references in the doxygen comments.
> >>>
> >>> I think, accidentally we can not make a library as NON-experimental.
> >>> TM never went through experimental to mature transition(see git log
> >>> lib/librte_ethdev/rte_tm.h)
> >>> It was a bug to not mark as experimental in each function in the ABI process.
> >>> Some of the features like packet marking are not even implemented by any HW.
> >>> I think, we can make API stable only all the features are implemented
> >>> by one or two HW.
> >>
> >> Fair enough, specially if the API is not ready yet.
> >>
> >> But they were part of stable ABI, and marking them as experimental now will
> >> break the old applications using these APIs.
> >
> > it is still marked as EXPERIMENTAL everywhere and API is not ready yet.
>
> Existing experimental marks are text only for human parsing.
>
> The compiler attribute and build time checks are missing, and the symbol in the
> binary doesn't have experimental tag. Our scripts and automated checks won't
> detect it as experimental.
>
> My point is just having experimental comment in header file is not enough to
> qualify the APIs as experimental.
>
> > Anyway, we need to break the ABI to make it work on various HW.
> > I am not sure what to do?
> > IMO, We need to send a patch as Fixes: for the bug of not adding
> > __rte_experimental in each function.
>
> Yes, this is where we are, both you and Cristian suggest API is not ready and
> should be experimental, but they were part of stable ABI, making them
> experimental will break the ABI.
> It looks like there is no good option but we should select one of the bad ones.
>
> >
> > Traffic Management API - EXPERIMENTAL
> > M: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
> > T: git://dpdk.org/next/dpdk-next-qos
> > F: lib/librte_ethdev/rte_tm*
> >>
> >>>
> >>>>
> >>>> Ray, Neil, David, Luca, Kevin, what do you think?
> >>
While I'm not called any of those names, allow me to give my 2c.
Since these are marked in binaries as part of the stable ABI, I think we
need to honour that for the next two releases 20.05 and 20.08 [which means
that we need to put in versioned functions for any changes, not that we
can't change anything]
For 20.11, I think these should then have one of two options taken:
* have these "fixed" and ready to be marked as stable, and officially part
of v21 ABI or
* mark them as experimental properly, and look to have them as part of the
v22 or subsequent ABI
Given the comments here, I would tend towards the latter of the above two
options, but that's really a decision for the maintainers.
Remember, this is not the first bug we have encountered where we messed up
some ABI versions in the 19.11 release, and, like the previous one with the
screwed up version number, I think we need to honour the ABI committments
made, especially since in this case it's only for a few more months till
20.11 development starts.
/Bruce
^ permalink raw reply [relevance 5%]
* Re: [dpdk-dev] [PATCH v4 1/4] ethdev: add tm support for shaper config in pkt mode
2020-04-27 16:59 3% ` Jerin Jacob
2020-04-28 11:51 0% ` [dpdk-dev] [EXT] " Nithin Dabilpuram
@ 2020-04-28 14:06 4% ` Ferruh Yigit
2020-04-28 14:45 5% ` Bruce Richardson
1 sibling, 1 reply; 200+ results
From: Ferruh Yigit @ 2020-04-28 14:06 UTC (permalink / raw)
To: Jerin Jacob
Cc: Dumitrescu, Cristian, Nithin Dabilpuram, Singh, Jasvinder,
Thomas Monjalon, Andrew Rybchenko, dev, jerinj, kkanas,
Nithin Dabilpuram, Kinsella, Ray, Neil Horman, Luca Boccassi,
Kevin Traynor, David Marchand, Bruce Richardson
On 4/27/2020 5:59 PM, Jerin Jacob wrote:
> On Mon, Apr 27, 2020 at 10:19 PM Ferruh Yigit <ferruh.yigit@intel.com> wrote:
>>
>> On 4/27/2020 5:29 PM, Jerin Jacob wrote:
>>> On Mon, Apr 27, 2020 at 9:42 PM Ferruh Yigit <ferruh.yigit@intel.com> wrote:
>>>>
>>>> On 4/27/2020 10:19 AM, Dumitrescu, Cristian wrote:
>>>>>
>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Yigit, Ferruh <ferruh.yigit@intel.com>
>>>>>> Sent: Saturday, April 25, 2020 9:09 PM
>>>>>> To: Dumitrescu, Cristian <cristian.dumitrescu@intel.com>; Nithin Dabilpuram
>>>>>> <nithind1988@gmail.com>; Singh, Jasvinder <jasvinder.singh@intel.com>;
>>>>>> Thomas Monjalon <thomas@monjalon.net>; Andrew Rybchenko
>>>>>> <arybchenko@solarflare.com>
>>>>>> Cc: dev@dpdk.org; jerinj@marvell.com; kkanas@marvell.com; Nithin
>>>>>> Dabilpuram <ndabilpuram@marvell.com>
>>>>>> Subject: Re: [PATCH v4 1/4] ethdev: add tm support for shaper config in pkt
>>>>>> mode
>>>>>>
>>>>>> On 4/24/2020 11:28 AM, Dumitrescu, Cristian wrote:
>>>>>>>
>>>>>>>
>>>>>>>> -----Original Message-----
>>>>>>>> From: Nithin Dabilpuram <nithind1988@gmail.com>
>>>>>>>> Sent: Wednesday, April 22, 2020 6:21 PM
>>>>>>>> To: Singh, Jasvinder <jasvinder.singh@intel.com>; Dumitrescu, Cristian
>>>>>>>> <cristian.dumitrescu@intel.com>; Thomas Monjalon
>>>>>>>> <thomas@monjalon.net>; Yigit, Ferruh <ferruh.yigit@intel.com>; Andrew
>>>>>>>> Rybchenko <arybchenko@solarflare.com>
>>>>>>>> Cc: dev@dpdk.org; jerinj@marvell.com; kkanas@marvell.com; Nithin
>>>>>>>> Dabilpuram <ndabilpuram@marvell.com>
>>>>>>>> Subject: [PATCH v4 1/4] ethdev: add tm support for shaper config in pkt
>>>>>>>> mode
>>>>>>>>
>>>>>>>> From: Nithin Dabilpuram <ndabilpuram@marvell.com>
>>>>>>>>
>>>>>>>> Some NIC hardware support shaper to work in packet mode i.e
>>>>>>>> shaping or ratelimiting traffic is in packets per second (PPS) as
>>>>>>>> opposed to default bytes per second (BPS). Hence this patch
>>>>>>>> adds support to configure shared or private shaper in packet mode,
>>>>>>>> provide rate in PPS and add related tm capabilities in port/level/node
>>>>>>>> capability structures.
>>>>>>>>
>>>>>>>> This patch also updates tm port/level/node capability structures with
>>>>>>>> exiting features of scheduler wfq packet mode, scheduler wfq byte mode
>>>>>>>> and private/shared shaper byte mode.
>>>>>>>>
>>>>>>>> SoftNIC PMD is also updated with new capabilities.
>>>>>>>>
>>>>>>>> Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>
>>>>>>>> ---
>>>>>>>> v3..v4:
>>>>>>>> - Update text under packet_mode as per Cristian.
>>>>>>>> - Update rte_eth_softnic_tm.c based on Jasvinder's comments.
>>>>>>>> - Add error enum
>>>>>> RTE_TM_ERROR_TYPE_SHAPER_PROFILE_PACKET_MODE
>>>>>>>> - Fix shaper_profile_check() with packet mode check
>>>>>>>> - Fix typo's
>>>>>>>>
>>>>>>>
>>>>>>> Acked-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
>>>>>>>
>>>>>>
>>>>>> Hi Nithin,
>>>>>>
>>>>>> It looks like patch is causing ABI break, I am getting following warning [1],
>>>>>> can you please check?
>>>>>>
>>>>>> [1]
>>>>>> https://pastebin.com/XYNFg14u
>>>>>
>>>>>
>>>>> Hi Ferruh,
>>>>>
>>>>> The RTE_TM API is marked as experimental, but it looks that this was not correctly marked when __rte_experimental ABI checker was introduced.
>>>>>
>>>>> It is marked as experimental at the top of the rte_tm.h, similarly to other APIs introduced around same time, but it was not correctly picked up by the ABI check procedure when later introduced, so __rte_experimental was not added to every function.
>>>>>
>>>>
>>>> :(
>>>>
>>>> Is it time to mature them?
>>>>
>>>> As you said they are not marked as experimental both in header file (function
>>>> declarations) and .map file.
>>>>
>>>> The problem is, they are not marked as experimental in DPDK_20.0 ABI (v19.11),
>>>> so marking them as experimental now will break the ABI. Not sure what to do,
>>>> cc'ed a few ABI related names for comment.
>>>>
>>>> For me, we need to proceed as the experimental tag removed and APIs become
>>>> mature starting from v19.11, since this is what happened in practice, and remove
>>>> a few existing being experimental references in the doxygen comments.
>>>
>>> I think, accidentally we can not make a library as NON-experimental.
>>> TM never went through experimental to mature transition(see git log
>>> lib/librte_ethdev/rte_tm.h)
>>> It was a bug to not mark as experimental in each function in the ABI process.
>>> Some of the features like packet marking are not even implemented by any HW.
>>> I think, we can make API stable only all the features are implemented
>>> by one or two HW.
>>
>> Fair enough, specially if the API is not ready yet.
>>
>> But they were part of stable ABI, and marking them as experimental now will
>> break the old applications using these APIs.
>
> it is still marked as EXPERIMENTAL everywhere and API is not ready yet.
Existing experimental marks are text only for human parsing.
The compiler attribute and build time checks are missing, and the symbol in the
binary doesn't have experimental tag. Our scripts and automated checks won't
detect it as experimental.
My point is just having experimental comment in header file is not enough to
qualify the APIs as experimental.
> Anyway, we need to break the ABI to make it work on various HW.
> I am not sure what to do?
> IMO, We need to send a patch as Fixes: for the bug of not adding
> __rte_experimental in each function.
Yes, this is where we are, both you and Cristian suggest API is not ready and
should be experimental, but they were part of stable ABI, making them
experimental will break the ABI.
It looks like there is no good option but we should select one of the bad ones.
>
> Traffic Management API - EXPERIMENTAL
> M: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
> T: git://dpdk.org/next/dpdk-next-qos
> F: lib/librte_ethdev/rte_tm*
>>
>>>
>>>>
>>>> Ray, Neil, David, Luca, Kevin, what do you think?
>>
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [EXT] Re: [PATCH v4 1/4] ethdev: add tm support for shaper config in pkt mode
2020-04-28 11:51 0% ` [dpdk-dev] [EXT] " Nithin Dabilpuram
@ 2020-04-28 13:56 0% ` Ferruh Yigit
0 siblings, 0 replies; 200+ results
From: Ferruh Yigit @ 2020-04-28 13:56 UTC (permalink / raw)
To: Nithin Dabilpuram, Jerin Jacob
Cc: Dumitrescu, Cristian, Nithin Dabilpuram, Singh, Jasvinder,
Thomas Monjalon, Andrew Rybchenko, dev, jerinj, kkanas, Kinsella,
Ray, Neil Horman, Luca Boccassi, Kevin Traynor, David Marchand,
Bruce Richardson
On 4/28/2020 12:51 PM, Nithin Dabilpuram wrote:
> On Mon, Apr 27, 2020 at 10:29:48PM +0530, Jerin Jacob wrote:
>> External Email
>>
>> ----------------------------------------------------------------------
>> On Mon, Apr 27, 2020 at 10:19 PM Ferruh Yigit <ferruh.yigit@intel.com> wrote:
>>>
>>> On 4/27/2020 5:29 PM, Jerin Jacob wrote:
>>>> On Mon, Apr 27, 2020 at 9:42 PM Ferruh Yigit <ferruh.yigit@intel.com> wrote:
>>>>>
>>>>> On 4/27/2020 10:19 AM, Dumitrescu, Cristian wrote:
>>>>>>
>>>>>>
>>>>>>> -----Original Message-----
>>>>>>> From: Yigit, Ferruh <ferruh.yigit@intel.com>
>>>>>>> Sent: Saturday, April 25, 2020 9:09 PM
>>>>>>> To: Dumitrescu, Cristian <cristian.dumitrescu@intel.com>; Nithin Dabilpuram
>>>>>>> <nithind1988@gmail.com>; Singh, Jasvinder <jasvinder.singh@intel.com>;
>>>>>>> Thomas Monjalon <thomas@monjalon.net>; Andrew Rybchenko
>>>>>>> <arybchenko@solarflare.com>
>>>>>>> Cc: dev@dpdk.org; jerinj@marvell.com; kkanas@marvell.com; Nithin
>>>>>>> Dabilpuram <ndabilpuram@marvell.com>
>>>>>>> Subject: Re: [PATCH v4 1/4] ethdev: add tm support for shaper config in pkt
>>>>>>> mode
>>>>>>>
>>>>>>> On 4/24/2020 11:28 AM, Dumitrescu, Cristian wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>> -----Original Message-----
>>>>>>>>> From: Nithin Dabilpuram <nithind1988@gmail.com>
>>>>>>>>> Sent: Wednesday, April 22, 2020 6:21 PM
>>>>>>>>> To: Singh, Jasvinder <jasvinder.singh@intel.com>; Dumitrescu, Cristian
>>>>>>>>> <cristian.dumitrescu@intel.com>; Thomas Monjalon
>>>>>>>>> <thomas@monjalon.net>; Yigit, Ferruh <ferruh.yigit@intel.com>; Andrew
>>>>>>>>> Rybchenko <arybchenko@solarflare.com>
>>>>>>>>> Cc: dev@dpdk.org; jerinj@marvell.com; kkanas@marvell.com; Nithin
>>>>>>>>> Dabilpuram <ndabilpuram@marvell.com>
>>>>>>>>> Subject: [PATCH v4 1/4] ethdev: add tm support for shaper config in pkt
>>>>>>>>> mode
>>>>>>>>>
>>>>>>>>> From: Nithin Dabilpuram <ndabilpuram@marvell.com>
>>>>>>>>>
>>>>>>>>> Some NIC hardware support shaper to work in packet mode i.e
>>>>>>>>> shaping or ratelimiting traffic is in packets per second (PPS) as
>>>>>>>>> opposed to default bytes per second (BPS). Hence this patch
>>>>>>>>> adds support to configure shared or private shaper in packet mode,
>>>>>>>>> provide rate in PPS and add related tm capabilities in port/level/node
>>>>>>>>> capability structures.
>>>>>>>>>
>>>>>>>>> This patch also updates tm port/level/node capability structures with
>>>>>>>>> exiting features of scheduler wfq packet mode, scheduler wfq byte mode
>>>>>>>>> and private/shared shaper byte mode.
>>>>>>>>>
>>>>>>>>> SoftNIC PMD is also updated with new capabilities.
>>>>>>>>>
>>>>>>>>> Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>
>>>>>>>>> ---
>>>>>>>>> v3..v4:
>>>>>>>>> - Update text under packet_mode as per Cristian.
>>>>>>>>> - Update rte_eth_softnic_tm.c based on Jasvinder's comments.
>>>>>>>>> - Add error enum
>>>>>>> RTE_TM_ERROR_TYPE_SHAPER_PROFILE_PACKET_MODE
>>>>>>>>> - Fix shaper_profile_check() with packet mode check
>>>>>>>>> - Fix typo's
>>>>>>>>>
>>>>>>>>
>>>>>>>> Acked-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
>>>>>>>>
>>>>>>>
>>>>>>> Hi Nithin,
>>>>>>>
>>>>>>> It looks like patch is causing ABI break, I am getting following warning [1],
>>>>>>> can you please check?
>>>>>>>
>>>>>>> [1]
>>>>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__pastebin.com_XYNFg14u&d=DwIBaQ&c=nKjWec2b6R0mOyPaz7xtfQ&r=FZ_tPCbgFOh18zwRPO9H0yDx8VW38vuapifdDfc8SFQ&m=xJB0Qb2Q-1bl0hEDeknUjJqrCDc3z-h0F0e7kj8KvvI&s=R6xtRQRRIIzAilc5z52oYjyHNlvvJB_9SUsKBkpPC6g&e=
>>>>>>
>>>>>>
>>>>>> Hi Ferruh,
>>>>>>
>>>>>> The RTE_TM API is marked as experimental, but it looks that this was not correctly marked when __rte_experimental ABI checker was introduced.
>>>>>>
>>>>>> It is marked as experimental at the top of the rte_tm.h, similarly to other APIs introduced around same time, but it was not correctly picked up by the ABI check procedure when later introduced, so __rte_experimental was not added to every function.
>>>>>>
>>>>>
>>>>> :(
>>>>>
>>>>> Is it time to mature them?
>>>>>
>>>>> As you said they are not marked as experimental both in header file (function
>>>>> declarations) and .map file.
>>>>>
>>>>> The problem is, they are not marked as experimental in DPDK_20.0 ABI (v19.11),
>>>>> so marking them as experimental now will break the ABI. Not sure what to do,
>>>>> cc'ed a few ABI related names for comment.
>>>>>
>>>>> For me, we need to proceed as the experimental tag removed and APIs become
>>>>> mature starting from v19.11, since this is what happened in practice, and remove
>>>>> a few existing being experimental references in the doxygen comments.
>>>>
>>>> I think, accidentally we can not make a library as NON-experimental.
>>>> TM never went through experimental to mature transition(see git log
>>>> lib/librte_ethdev/rte_tm.h)
>>>> It was a bug to not mark as experimental in each function in the ABI process.
>>>> Some of the features like packet marking are not even implemented by any HW.
>>>> I think, we can make API stable only all the features are implemented
>>>> by one or two HW.
>>>
>>> Fair enough, specially if the API is not ready yet.
>>>
>>> But they were part of stable ABI, and marking them as experimental now will
>>> break the old applications using these APIs.
>>
>> it is still marked as EXPERIMENTAL everywhere and API is not ready yet.
>> Anyway, we need to break the ABI to make it work on various HW.
>> I am not sure what to do?
>> IMO, We need to send a patch as Fixes: for the bug of not adding
>> __rte_experimental in each function.
>>
>> Traffic Management API - EXPERIMENTAL
>> M: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
>> T: git://dpdk.org/next/dpdk-next-qos
>> F: lib/librte_ethdev/rte_tm*
>
> Ray, Neil, David, Luca, Kevin, Ferruh
>
> Any thoughts on this proposal ?
>
> If it is fine, I can send a "Fixes:" patch to update experimental attribute in rte_tm.h
> for all functions so that 20.05 is having the right marking.
Hi Nithin,
Please send the fix patch, we may continue to discuss in that patch but it would
be good to have it ready.
>
>>>
>>>>
>>>>>
>>>>> Ray, Neil, David, Luca, Kevin, what do you think?
>>>
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [EXT] Re: [PATCH v4 1/4] ethdev: add tm support for shaper config in pkt mode
2020-04-27 16:59 3% ` Jerin Jacob
@ 2020-04-28 11:51 0% ` Nithin Dabilpuram
2020-04-28 13:56 0% ` Ferruh Yigit
2020-04-28 14:06 4% ` [dpdk-dev] " Ferruh Yigit
1 sibling, 1 reply; 200+ results
From: Nithin Dabilpuram @ 2020-04-28 11:51 UTC (permalink / raw)
To: Jerin Jacob
Cc: Ferruh Yigit, Dumitrescu, Cristian, Nithin Dabilpuram, Singh,
Jasvinder, Thomas Monjalon, Andrew Rybchenko, dev, jerinj,
kkanas, Kinsella, Ray, Neil Horman, Luca Boccassi, Kevin Traynor,
David Marchand, Bruce Richardson
On Mon, Apr 27, 2020 at 10:29:48PM +0530, Jerin Jacob wrote:
> External Email
>
> ----------------------------------------------------------------------
> On Mon, Apr 27, 2020 at 10:19 PM Ferruh Yigit <ferruh.yigit@intel.com> wrote:
> >
> > On 4/27/2020 5:29 PM, Jerin Jacob wrote:
> > > On Mon, Apr 27, 2020 at 9:42 PM Ferruh Yigit <ferruh.yigit@intel.com> wrote:
> > >>
> > >> On 4/27/2020 10:19 AM, Dumitrescu, Cristian wrote:
> > >>>
> > >>>
> > >>>> -----Original Message-----
> > >>>> From: Yigit, Ferruh <ferruh.yigit@intel.com>
> > >>>> Sent: Saturday, April 25, 2020 9:09 PM
> > >>>> To: Dumitrescu, Cristian <cristian.dumitrescu@intel.com>; Nithin Dabilpuram
> > >>>> <nithind1988@gmail.com>; Singh, Jasvinder <jasvinder.singh@intel.com>;
> > >>>> Thomas Monjalon <thomas@monjalon.net>; Andrew Rybchenko
> > >>>> <arybchenko@solarflare.com>
> > >>>> Cc: dev@dpdk.org; jerinj@marvell.com; kkanas@marvell.com; Nithin
> > >>>> Dabilpuram <ndabilpuram@marvell.com>
> > >>>> Subject: Re: [PATCH v4 1/4] ethdev: add tm support for shaper config in pkt
> > >>>> mode
> > >>>>
> > >>>> On 4/24/2020 11:28 AM, Dumitrescu, Cristian wrote:
> > >>>>>
> > >>>>>
> > >>>>>> -----Original Message-----
> > >>>>>> From: Nithin Dabilpuram <nithind1988@gmail.com>
> > >>>>>> Sent: Wednesday, April 22, 2020 6:21 PM
> > >>>>>> To: Singh, Jasvinder <jasvinder.singh@intel.com>; Dumitrescu, Cristian
> > >>>>>> <cristian.dumitrescu@intel.com>; Thomas Monjalon
> > >>>>>> <thomas@monjalon.net>; Yigit, Ferruh <ferruh.yigit@intel.com>; Andrew
> > >>>>>> Rybchenko <arybchenko@solarflare.com>
> > >>>>>> Cc: dev@dpdk.org; jerinj@marvell.com; kkanas@marvell.com; Nithin
> > >>>>>> Dabilpuram <ndabilpuram@marvell.com>
> > >>>>>> Subject: [PATCH v4 1/4] ethdev: add tm support for shaper config in pkt
> > >>>>>> mode
> > >>>>>>
> > >>>>>> From: Nithin Dabilpuram <ndabilpuram@marvell.com>
> > >>>>>>
> > >>>>>> Some NIC hardware support shaper to work in packet mode i.e
> > >>>>>> shaping or ratelimiting traffic is in packets per second (PPS) as
> > >>>>>> opposed to default bytes per second (BPS). Hence this patch
> > >>>>>> adds support to configure shared or private shaper in packet mode,
> > >>>>>> provide rate in PPS and add related tm capabilities in port/level/node
> > >>>>>> capability structures.
> > >>>>>>
> > >>>>>> This patch also updates tm port/level/node capability structures with
> > >>>>>> exiting features of scheduler wfq packet mode, scheduler wfq byte mode
> > >>>>>> and private/shared shaper byte mode.
> > >>>>>>
> > >>>>>> SoftNIC PMD is also updated with new capabilities.
> > >>>>>>
> > >>>>>> Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>
> > >>>>>> ---
> > >>>>>> v3..v4:
> > >>>>>> - Update text under packet_mode as per Cristian.
> > >>>>>> - Update rte_eth_softnic_tm.c based on Jasvinder's comments.
> > >>>>>> - Add error enum
> > >>>> RTE_TM_ERROR_TYPE_SHAPER_PROFILE_PACKET_MODE
> > >>>>>> - Fix shaper_profile_check() with packet mode check
> > >>>>>> - Fix typo's
> > >>>>>>
> > >>>>>
> > >>>>> Acked-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
> > >>>>>
> > >>>>
> > >>>> Hi Nithin,
> > >>>>
> > >>>> It looks like patch is causing ABI break, I am getting following warning [1],
> > >>>> can you please check?
> > >>>>
> > >>>> [1]
> > >>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__pastebin.com_XYNFg14u&d=DwIBaQ&c=nKjWec2b6R0mOyPaz7xtfQ&r=FZ_tPCbgFOh18zwRPO9H0yDx8VW38vuapifdDfc8SFQ&m=xJB0Qb2Q-1bl0hEDeknUjJqrCDc3z-h0F0e7kj8KvvI&s=R6xtRQRRIIzAilc5z52oYjyHNlvvJB_9SUsKBkpPC6g&e=
> > >>>
> > >>>
> > >>> Hi Ferruh,
> > >>>
> > >>> The RTE_TM API is marked as experimental, but it looks that this was not correctly marked when __rte_experimental ABI checker was introduced.
> > >>>
> > >>> It is marked as experimental at the top of the rte_tm.h, similarly to other APIs introduced around same time, but it was not correctly picked up by the ABI check procedure when later introduced, so __rte_experimental was not added to every function.
> > >>>
> > >>
> > >> :(
> > >>
> > >> Is it time to mature them?
> > >>
> > >> As you said they are not marked as experimental both in header file (function
> > >> declarations) and .map file.
> > >>
> > >> The problem is, they are not marked as experimental in DPDK_20.0 ABI (v19.11),
> > >> so marking them as experimental now will break the ABI. Not sure what to do,
> > >> cc'ed a few ABI related names for comment.
> > >>
> > >> For me, we need to proceed as the experimental tag removed and APIs become
> > >> mature starting from v19.11, since this is what happened in practice, and remove
> > >> a few existing being experimental references in the doxygen comments.
> > >
> > > I think, accidentally we can not make a library as NON-experimental.
> > > TM never went through experimental to mature transition(see git log
> > > lib/librte_ethdev/rte_tm.h)
> > > It was a bug to not mark as experimental in each function in the ABI process.
> > > Some of the features like packet marking are not even implemented by any HW.
> > > I think, we can make API stable only all the features are implemented
> > > by one or two HW.
> >
> > Fair enough, specially if the API is not ready yet.
> >
> > But they were part of stable ABI, and marking them as experimental now will
> > break the old applications using these APIs.
>
> it is still marked as EXPERIMENTAL everywhere and API is not ready yet.
> Anyway, we need to break the ABI to make it work on various HW.
> I am not sure what to do?
> IMO, We need to send a patch as Fixes: for the bug of not adding
> __rte_experimental in each function.
>
> Traffic Management API - EXPERIMENTAL
> M: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
> T: git://dpdk.org/next/dpdk-next-qos
> F: lib/librte_ethdev/rte_tm*
Ray, Neil, David, Luca, Kevin, Ferruh
Any thoughts on this proposal ?
If it is fine, I can send a "Fixes:" patch to update experimental attribute in rte_tm.h
for all functions so that 20.05 is having the right marking.
> >
> > >
> > >>
> > >> Ray, Neil, David, Luca, Kevin, what do you think?
> >
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH v3 0/4] Enforce checking on flag values in API's
2020-04-27 23:16 3% ` [dpdk-dev] [PATCH v3 0/4] Enforce checking on flag values in API's Stephen Hemminger
` (4 preceding siblings ...)
2020-04-28 10:28 0% ` [dpdk-dev] [PATCH v3 0/4] Enforce checking on flag values in API's Bruce Richardson
@ 2020-04-28 11:04 0% ` Ananyev, Konstantin
5 siblings, 0 replies; 200+ results
From: Ananyev, Konstantin @ 2020-04-28 11:04 UTC (permalink / raw)
To: Stephen Hemminger, dev
> The DPDK API's are lax about checking for undefined flag values.
> This makes it impossible to add new bits to existing API's
> without causing ABI breakage. This means we end up doing unnecessary
> symbol versioning just to work around applications that might
> pass in naughty bits.
>
> This is the DPDK analog of the Linux kernel openat() problem.
> Openat api was added but since kernel did not check flags it
> ended up that another syscall openat2() was necessary before
> the flags could be used.
>
> v3 - define mask based on existing defines for ring and hash
>
> Stephen Hemminger (4):
> ring: future proof flag settings
> hash: check flags on creation
> stack: check flags on creation
> cfgfile: check flags value
>
> lib/librte_cfgfile/rte_cfgfile.c | 4 ++++
> lib/librte_hash/rte_cuckoo_hash.c | 14 ++++++++++++++
> lib/librte_ring/rte_ring.c | 12 ++++++++++++
> lib/librte_stack/rte_stack.c | 5 +++++
> 4 files changed, 35 insertions(+)
>
> --
Series Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
> 2.20.1
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH v3 0/4] Enforce checking on flag values in API's
2020-04-27 23:16 3% ` [dpdk-dev] [PATCH v3 0/4] Enforce checking on flag values in API's Stephen Hemminger
` (3 preceding siblings ...)
2020-04-27 23:16 3% ` [dpdk-dev] [PATCH v3 4/4] cfgfile: check flags value Stephen Hemminger
@ 2020-04-28 10:28 0% ` Bruce Richardson
2020-04-28 11:04 0% ` Ananyev, Konstantin
5 siblings, 0 replies; 200+ results
From: Bruce Richardson @ 2020-04-28 10:28 UTC (permalink / raw)
To: Stephen Hemminger; +Cc: dev
On Mon, Apr 27, 2020 at 04:16:21PM -0700, Stephen Hemminger wrote:
> The DPDK API's are lax about checking for undefined flag values.
> This makes it impossible to add new bits to existing API's
> without causing ABI breakage. This means we end up doing unnecessary
> symbol versioning just to work around applications that might
> pass in naughty bits.
>
> This is the DPDK analog of the Linux kernel openat() problem.
> Openat api was added but since kernel did not check flags it
> ended up that another syscall openat2() was necessary before
> the flags could be used.
>
> v3 - define mask based on existing defines for ring and hash
>
> Stephen Hemminger (4):
> ring: future proof flag settings
> hash: check flags on creation
> stack: check flags on creation
> cfgfile: check flags value
>
> lib/librte_cfgfile/rte_cfgfile.c | 4 ++++
> lib/librte_hash/rte_cuckoo_hash.c | 14 ++++++++++++++
> lib/librte_ring/rte_ring.c | 12 ++++++++++++
> lib/librte_stack/rte_stack.c | 5 +++++
> 4 files changed, 35 insertions(+)
>
I think this is a good idea to do in DPDK
Series-acked-by: Bruce Richardson <bruce.richardson@intel.com>
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH v3 4/4] cfgfile: check flags value
2020-04-27 23:16 3% ` [dpdk-dev] [PATCH v3 4/4] cfgfile: check flags value Stephen Hemminger
@ 2020-04-28 10:24 0% ` Bruce Richardson
0 siblings, 0 replies; 200+ results
From: Bruce Richardson @ 2020-04-28 10:24 UTC (permalink / raw)
To: Stephen Hemminger; +Cc: dev, Cristian Dumitrescu
On Mon, Apr 27, 2020 at 04:16:25PM -0700, Stephen Hemminger wrote:
> All API's should check that they support the flag values
> passed. If an application passes an invalid flag it could
> cause problems in later ABI.
>
> Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
> ---
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
^ permalink raw reply [relevance 0%]
* [dpdk-dev] [Bug 465] app/test compile failed with gcc and clang
@ 2020-04-28 9:15 3% bugzilla
0 siblings, 0 replies; 200+ results
From: bugzilla @ 2020-04-28 9:15 UTC (permalink / raw)
To: dev
https://bugs.dpdk.org/show_bug.cgi?id=465
Bug ID: 465
Summary: app/test compile failed with gcc and clang
Product: DPDK
Version: 20.05
Hardware: x86
OS: Linux
Status: UNCONFIRMED
Severity: normal
Priority: Normal
Component: testpmd
Assignee: dev@dpdk.org
Reporter: qimaix.xiao@intel.com
Target Milestone: ---
---------------------------------Description-----------------------------------
build ./app/test failed on dpdk 20.05
---------------------------------reproduce steps-------------------------------
Steps:
1. cd /root/dpdk
2. export RTE_TARGET=x86_64-native-linuxapp-gcc;export RTE_SDK=`pwd`;make
install T=x86_64-native-linuxapp-gcc -j70
3. sed -i -e 's/#define MAX_TRAFFIC_BURST 2048/#define
MAX_TRAFFIC_BURST 32/' app/test/test_pmd_perf.c
4. sed -i -e 's/lpbk_mode = 1/lpbk_mode = 0/' app/test/test_pmd_perf.c
5. rm -rf ./app/test/test_resource_c.res.o
6. rm -rf ./app/test/test_resource_tar.res.o
7. rm -rf ./app/test/test_pci_sysfs.res.o
8. make -j 70 -C app/test
Output:
...
test_mbuf.c:2670:2: error: ‘rte_mbuf_dyn_dump’ is deprecated: Symbol is not yet
part of stable ABI [-Werror=deprecated-declarations]
rte_mbuf_dyn_dump(stdout);
^~~~~~~~~~~~~~~~~
In file included from test_mbuf.c:35:0:
/root/dpdk/x86_64-native-linuxapp-gcc/include/rte_mbuf_dyn.h:235:6: note:
declared here
void rte_mbuf_dyn_dump(FILE *out);
^~~~~~~~~~~~~~~~~
test_flow_classify.c: At top level:
cc1: error: unrecognized command line option ‘-Wno-address-of-packed-member’
[-Werror]
cc1: all warnings being treated as errors
/root/dpdk/mk/internal/rte.compile-pre.mk:114: recipe for target
'test_flow_classify.o' failed
make: *** [test_flow_classify.o] Error 1
test_stack_perf.c: At top level:
cc1: error: unrecognized command line option ‘-Wno-address-of-packed-member’
[-Werror]
cc1: all warnings being treated as errors
/root/dpdk/mk/internal/rte.compile-pre.mk:114: recipe for target
'test_stack_perf.o' failed
make: *** [test_stack_perf.o] Error 1
test_timer_secondary.c: At top level:
cc1: error: unrecognized command line option ‘-Wno-address-of-packed-member’
[-Werror]
cc1: all warnings being treated as errors
/root/dpdk/mk/internal/rte.compile-pre.mk:114: recipe for target
'test_timer_secondary.o' failed
make: *** [test_timer_secondary.o] Error 1
test_stack.c: At top level:
cc1: error: unrecognized command line option ‘-Wno-address-of-packed-member’
[-Werror]
cc1: all warnings being treated as errors
/root/dpdk/mk/internal/rte.compile-pre.mk:114: recipe for target 'test_stack.o'
failed
make: *** [test_stack.o] Error 1
test_mbuf.c: At top level:
cc1: error: unrecognized command line option ‘-Wno-address-of-packed-member’
[-Werror]
cc1: all warnings being treated as errors
/root/dpdk/mk/internal/rte.compile-pre.mk:114: recipe for target 'test_mbuf.o'
failed
make: *** [test_mbuf.o] Error 1
test_ring_perf.c: At top level:
cc1: error: unrecognized command line option ‘-Wno-address-of-packed-member’
[-Werror]
cc1: all warnings being treated as errors
/root/dpdk/mk/internal/rte.compile-pre.mk:114: recipe for target
'test_ring_perf.o' failed
make: *** [test_ring_perf.o] Error 1
test_ring.c: At top level:
cc1: error: unrecognized command line option ‘-Wno-address-of-packed-member’
[-Werror]
cc1: all warnings being treated as errors
/root/dpdk/mk/internal/rte.compile-pre.mk:114: recipe for target 'test_ring.o'
failed
make: *** [test_ring.o] Error 1
Comments: clang has the same issue with gcc
--
You are receiving this mail because:
You are the assignee for the bug.
^ permalink raw reply [relevance 3%]
* Re: [dpdk-dev] [PATCH v3 1/4] ring: future proof flag settings
2020-04-27 23:16 3% ` [dpdk-dev] [PATCH v3 1/4] ring: future proof flag settings Stephen Hemminger
@ 2020-04-28 4:12 0% ` Honnappa Nagarahalli
0 siblings, 0 replies; 200+ results
From: Honnappa Nagarahalli @ 2020-04-28 4:12 UTC (permalink / raw)
To: Stephen Hemminger, dev; +Cc: Konstantin Ananyev, nd, Honnappa Nagarahalli, nd
<snip>
>
> All API's should check that they support the flag values passed.
> These checks ensure that the extra bits can safely be used without risk of ABI
> breakage.
>
> Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
> ---
> lib/librte_ring/rte_ring.c | 12 ++++++++++++
> 1 file changed, 12 insertions(+)
>
> diff --git a/lib/librte_ring/rte_ring.c b/lib/librte_ring/rte_ring.c index
> ebe5ccf0de68..e16dd45a82de 100644
> --- a/lib/librte_ring/rte_ring.c
> +++ b/lib/librte_ring/rte_ring.c
> @@ -42,6 +42,11 @@ static struct rte_tailq_elem rte_ring_tailq = { };
> EAL_REGISTER_TAILQ(rte_ring_tailq)
>
> +/* mask of all valid flag values to ring_create() */ #define
> +RING_F_MASK (RING_F_SP_ENQ | RING_F_SC_DEQ | RING_F_EXACT_SZ | \
> + RING_F_MP_RTS_ENQ | RING_F_MC_RTS_DEQ | \
> + RING_F_MP_HTS_ENQ | RING_F_MC_HTS_DEQ )
> +
> /* true if x is a power of 2 */
> #define POWEROF2(x) ((((x)-1) & (x)) == 0)
>
> @@ -197,6 +202,13 @@ rte_ring_init(struct rte_ring *r, const char *name,
> unsigned count,
> RTE_BUILD_BUG_ON(offsetof(struct rte_ring_headtail, tail) !=
> offsetof(struct rte_ring_rts_headtail, tail.val.pos));
>
> + /* future proof flags, only allow supported values */
> + if (flags & ~RING_F_MASK) {
> + RTE_LOG(ERR, RING,
> + "Unsupported flags requested %#x\n", flags);
> + return -EINVAL;
> + }
> +
> /* init the ring structure */
> memset(r, 0, sizeof(*r));
> ret = strlcpy(r->name, name, sizeof(r->name));
> --
> 2.20.1
^ permalink raw reply [relevance 0%]
* [dpdk-dev] [PATCH v3 4/4] cfgfile: check flags value
2020-04-27 23:16 3% ` [dpdk-dev] [PATCH v3 0/4] Enforce checking on flag values in API's Stephen Hemminger
` (2 preceding siblings ...)
2020-04-27 23:16 3% ` [dpdk-dev] [PATCH v3 3/4] stack: " Stephen Hemminger
@ 2020-04-27 23:16 3% ` Stephen Hemminger
2020-04-28 10:24 0% ` Bruce Richardson
2020-04-28 10:28 0% ` [dpdk-dev] [PATCH v3 0/4] Enforce checking on flag values in API's Bruce Richardson
2020-04-28 11:04 0% ` Ananyev, Konstantin
5 siblings, 1 reply; 200+ results
From: Stephen Hemminger @ 2020-04-27 23:16 UTC (permalink / raw)
To: dev; +Cc: Stephen Hemminger, Cristian Dumitrescu
All API's should check that they support the flag values
passed. If an application passes an invalid flag it could
cause problems in later ABI.
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
---
lib/librte_cfgfile/rte_cfgfile.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/lib/librte_cfgfile/rte_cfgfile.c b/lib/librte_cfgfile/rte_cfgfile.c
index 9049fd9c2319..714717dd9007 100644
--- a/lib/librte_cfgfile/rte_cfgfile.c
+++ b/lib/librte_cfgfile/rte_cfgfile.c
@@ -272,6 +272,10 @@ rte_cfgfile_create(int flags)
int i;
struct rte_cfgfile *cfg;
+ /* future proof flags usage */
+ if (flags & ~(CFG_FLAG_GLOBAL_SECTION | CFG_FLAG_EMPTY_VALUES))
+ return NULL;
+
cfg = malloc(sizeof(*cfg));
if (cfg == NULL)
--
2.20.1
^ permalink raw reply [relevance 3%]
* [dpdk-dev] [PATCH v3 3/4] stack: check flags on creation
2020-04-27 23:16 3% ` [dpdk-dev] [PATCH v3 0/4] Enforce checking on flag values in API's Stephen Hemminger
2020-04-27 23:16 3% ` [dpdk-dev] [PATCH v3 1/4] ring: future proof flag settings Stephen Hemminger
2020-04-27 23:16 3% ` [dpdk-dev] [PATCH v3 2/4] hash: check flags on creation Stephen Hemminger
@ 2020-04-27 23:16 3% ` Stephen Hemminger
2020-05-02 2:36 0% ` Eads, Gage
2020-04-27 23:16 3% ` [dpdk-dev] [PATCH v3 4/4] cfgfile: check flags value Stephen Hemminger
` (2 subsequent siblings)
5 siblings, 1 reply; 200+ results
From: Stephen Hemminger @ 2020-04-27 23:16 UTC (permalink / raw)
To: dev; +Cc: Stephen Hemminger, Gage Eads, Olivier Matz
All API's should check that they support the flag values
passed. If an application passes an invalid flag it could
cause problems in later ABI.
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
---
lib/librte_stack/rte_stack.c | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/lib/librte_stack/rte_stack.c b/lib/librte_stack/rte_stack.c
index d19824f00439..e58fa545fca4 100644
--- a/lib/librte_stack/rte_stack.c
+++ b/lib/librte_stack/rte_stack.c
@@ -59,6 +59,11 @@ rte_stack_create(const char *name, unsigned int count, int socket_id,
unsigned int sz;
int ret;
+ if (flags & ~(RTE_STACK_F_LF)) {
+ STACK_LOG_ERR("Unsupported stack flags %#x\n", flags);
+ return NULL;
+ }
+
#ifdef RTE_ARCH_64
RTE_BUILD_BUG_ON(sizeof(struct rte_stack_lf_head) != 16);
#else
--
2.20.1
^ permalink raw reply [relevance 3%]
* [dpdk-dev] [PATCH v3 2/4] hash: check flags on creation
2020-04-27 23:16 3% ` [dpdk-dev] [PATCH v3 0/4] Enforce checking on flag values in API's Stephen Hemminger
2020-04-27 23:16 3% ` [dpdk-dev] [PATCH v3 1/4] ring: future proof flag settings Stephen Hemminger
@ 2020-04-27 23:16 3% ` Stephen Hemminger
2020-04-29 1:08 0% ` Wang, Yipeng1
2020-04-27 23:16 3% ` [dpdk-dev] [PATCH v3 3/4] stack: " Stephen Hemminger
` (3 subsequent siblings)
5 siblings, 1 reply; 200+ results
From: Stephen Hemminger @ 2020-04-27 23:16 UTC (permalink / raw)
To: dev; +Cc: Stephen Hemminger, Yipeng Wang, Sameh Gobriel, Bruce Richardson
All API's should check that they support the flag values
passed. If an application passes an invalid flag it could
cause problems in later ABI.
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
---
lib/librte_hash/rte_cuckoo_hash.c | 14 ++++++++++++++
1 file changed, 14 insertions(+)
diff --git a/lib/librte_hash/rte_cuckoo_hash.c b/lib/librte_hash/rte_cuckoo_hash.c
index 38767a8a164f..d54bbb47edcb 100644
--- a/lib/librte_hash/rte_cuckoo_hash.c
+++ b/lib/librte_hash/rte_cuckoo_hash.c
@@ -32,6 +32,14 @@
#include "rte_hash.h"
#include "rte_cuckoo_hash.h"
+/* Mask of all flags supported by this version */
+#define RTE_HASH_EXTRA_FLAGS_MASK (RTE_HASH_EXTRA_FLAGS_TRANS_MEM_SUPPORT | \
+ RTE_HASH_EXTRA_FLAGS_MULTI_WRITER_ADD | \
+ RTE_HASH_EXTRA_FLAGS_RW_CONCURRENCY | \
+ RTE_HASH_EXTRA_FLAGS_EXT_TABLE | \
+ RTE_HASH_EXTRA_FLAGS_NO_FREE_ON_DEL | \
+ RTE_HASH_EXTRA_FLAGS_RW_CONCURRENCY_LF)
+
#define FOR_EACH_BUCKET(CURRENT_BKT, START_BUCKET) \
for (CURRENT_BKT = START_BUCKET; \
CURRENT_BKT != NULL; \
@@ -164,6 +172,12 @@ rte_hash_create(const struct rte_hash_parameters *params)
return NULL;
}
+ if (params->extra_flag & ~RTE_HASH_EXTRA_FLAGS_MASK) {
+ rte_errno = EINVAL;
+ RTE_LOG(ERR, HASH, "rte_hash_create: unsupported extra flags\n");
+ return NULL;
+ }
+
/* Validate correct usage of extra options */
if ((params->extra_flag & RTE_HASH_EXTRA_FLAGS_RW_CONCURRENCY) &&
(params->extra_flag & RTE_HASH_EXTRA_FLAGS_RW_CONCURRENCY_LF)) {
--
2.20.1
^ permalink raw reply [relevance 3%]
* [dpdk-dev] [PATCH v3 1/4] ring: future proof flag settings
2020-04-27 23:16 3% ` [dpdk-dev] [PATCH v3 0/4] Enforce checking on flag values in API's Stephen Hemminger
@ 2020-04-27 23:16 3% ` Stephen Hemminger
2020-04-28 4:12 0% ` Honnappa Nagarahalli
2020-04-27 23:16 3% ` [dpdk-dev] [PATCH v3 2/4] hash: check flags on creation Stephen Hemminger
` (4 subsequent siblings)
5 siblings, 1 reply; 200+ results
From: Stephen Hemminger @ 2020-04-27 23:16 UTC (permalink / raw)
To: dev; +Cc: Stephen Hemminger, Honnappa Nagarahalli, Konstantin Ananyev
All API's should check that they support the flag values passed.
These checks ensure that the extra bits can safely be used
without risk of ABI breakage.
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
---
lib/librte_ring/rte_ring.c | 12 ++++++++++++
1 file changed, 12 insertions(+)
diff --git a/lib/librte_ring/rte_ring.c b/lib/librte_ring/rte_ring.c
index ebe5ccf0de68..e16dd45a82de 100644
--- a/lib/librte_ring/rte_ring.c
+++ b/lib/librte_ring/rte_ring.c
@@ -42,6 +42,11 @@ static struct rte_tailq_elem rte_ring_tailq = {
};
EAL_REGISTER_TAILQ(rte_ring_tailq)
+/* mask of all valid flag values to ring_create() */
+#define RING_F_MASK (RING_F_SP_ENQ | RING_F_SC_DEQ | RING_F_EXACT_SZ | \
+ RING_F_MP_RTS_ENQ | RING_F_MC_RTS_DEQ | \
+ RING_F_MP_HTS_ENQ | RING_F_MC_HTS_DEQ )
+
/* true if x is a power of 2 */
#define POWEROF2(x) ((((x)-1) & (x)) == 0)
@@ -197,6 +202,13 @@ rte_ring_init(struct rte_ring *r, const char *name, unsigned count,
RTE_BUILD_BUG_ON(offsetof(struct rte_ring_headtail, tail) !=
offsetof(struct rte_ring_rts_headtail, tail.val.pos));
+ /* future proof flags, only allow supported values */
+ if (flags & ~RING_F_MASK) {
+ RTE_LOG(ERR, RING,
+ "Unsupported flags requested %#x\n", flags);
+ return -EINVAL;
+ }
+
/* init the ring structure */
memset(r, 0, sizeof(*r));
ret = strlcpy(r->name, name, sizeof(r->name));
--
2.20.1
^ permalink raw reply [relevance 3%]
* [dpdk-dev] [PATCH v3 0/4] Enforce checking on flag values in API's
2020-04-24 17:15 3% ` [dpdk-dev] [PATCH v2 0/4] enforce checking on flag values " Stephen Hemminger
@ 2020-04-27 23:16 3% ` Stephen Hemminger
2020-04-27 23:16 3% ` [dpdk-dev] [PATCH v3 1/4] ring: future proof flag settings Stephen Hemminger
` (5 more replies)
1 sibling, 6 replies; 200+ results
From: Stephen Hemminger @ 2020-04-27 23:16 UTC (permalink / raw)
To: dev; +Cc: Stephen Hemminger
The DPDK API's are lax about checking for undefined flag values.
This makes it impossible to add new bits to existing API's
without causing ABI breakage. This means we end up doing unnecessary
symbol versioning just to work around applications that might
pass in naughty bits.
This is the DPDK analog of the Linux kernel openat() problem.
Openat api was added but since kernel did not check flags it
ended up that another syscall openat2() was necessary before
the flags could be used.
v3 - define mask based on existing defines for ring and hash
Stephen Hemminger (4):
ring: future proof flag settings
hash: check flags on creation
stack: check flags on creation
cfgfile: check flags value
lib/librte_cfgfile/rte_cfgfile.c | 4 ++++
lib/librte_hash/rte_cuckoo_hash.c | 14 ++++++++++++++
lib/librte_ring/rte_ring.c | 12 ++++++++++++
lib/librte_stack/rte_stack.c | 5 +++++
4 files changed, 35 insertions(+)
--
2.20.1
^ permalink raw reply [relevance 3%]
* Re: [dpdk-dev] [PATCH v3] mempool: return 0 if area is too small on populate
2020-04-27 11:44 0% ` [dpdk-dev] [PATCH v3] " Ray Kinsella
@ 2020-04-27 18:02 0% ` Lukasz Wojciechowski
0 siblings, 0 replies; 200+ results
From: Lukasz Wojciechowski @ 2020-04-27 18:02 UTC (permalink / raw)
To: Ray Kinsella, Thomas Monjalon, dev
Cc: david.marchand, Olivier Matz, Neil Horman, John McNamara,
Marko Kovacevic, Xiaoyun Li, Jingjing Wu, Andrew Rybchenko
W dniu 27.04.2020 o 13:44, Ray Kinsella pisze:
>
> On 25/04/2020 23:23, Thomas Monjalon wrote:
>> From: Olivier Matz <olivier.matz@6wind.com>
>>
>> Change rte_mempool_populate_iova() and rte_mempool_populate_iova() to
... and rte_mempool_populate_virt()
>> return 0 instead of -EINVAL when there is not enough room to store one
>> object, as it can be helpful for applications to distinguish this
>> specific case.
>>
>> As this is an ABI change, use symbol versioning to preserve old
>> behavior for binary applications.
>>
>> Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
>> ---
>>
>> changes in v3:
>> - rebase
>> - remove deprecation notice
>> - notify API change in release notes
>> - fix ABI version from 20.0.1 to 20.0.2 (should be 21 maybe)
> yes, should be v21.
>
>> This v3 cannot be merged because of a false positive ABI check:
>>
>> 2 Removed functions:
>> 'function int rte_mempool_populate_iova(rte_mempool*, char*, rte_iova_t, size_t, rte_mempool_memchunk_free_cb_t*, void*)' {rte_mempool_populate_iova@@DPDK_20.0}
>> 'function int rte_mempool_populate_virt(rte_mempool*, char*, size_t, size_t, rte_mempool_memchunk_free_cb_t*, void*)' {rte_mempool_populate_virt@@DPDK_20.0}
> Well it's not really a false positive, as the v20 symbols are now missing.
> See notes on VERSION_SYMBOL.
>
>> ---
>> doc/guides/rel_notes/deprecation.rst | 5 --
>> doc/guides/rel_notes/release_20_05.rst | 4 ++
>> examples/ntb/ntb_fwd.c | 2 +-
>> lib/librte_mempool/meson.build | 2 +
>> lib/librte_mempool/rte_mempool.c | 77 ++++++++++++++++++----
>> lib/librte_mempool/rte_mempool.h | 14 ++--
>> lib/librte_mempool/rte_mempool_version.map | 7 ++
>> 7 files changed, 90 insertions(+), 21 deletions(-)
>>
>> diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
>> index 1339f54f5f..20aa745b77 100644
>> --- a/doc/guides/rel_notes/deprecation.rst
>> +++ b/doc/guides/rel_notes/deprecation.rst
>> @@ -65,11 +65,6 @@ Deprecation Notices
>> structure would be made internal (or removed if all dependencies are cleared)
>> in future releases.
>>
>> -* mempool: starting from v20.05, the API of rte_mempool_populate_iova()
>> - and rte_mempool_populate_virt() will change to return 0 instead
>> - of -EINVAL when there is not enough room to store one object. The ABI
>> - will be preserved until 20.11.
>> -
>> * ethdev: the legacy filter API, including
>> ``rte_eth_dev_filter_supported()``, ``rte_eth_dev_filter_ctrl()`` as well
>> as filter types MACVLAN, ETHERTYPE, FLEXIBLE, SYN, NTUPLE, TUNNEL, FDIR,
>> diff --git a/doc/guides/rel_notes/release_20_05.rst b/doc/guides/rel_notes/release_20_05.rst
>> index b124c3f287..ab20a7d021 100644
>> --- a/doc/guides/rel_notes/release_20_05.rst
>> +++ b/doc/guides/rel_notes/release_20_05.rst
>> @@ -241,6 +241,10 @@ API Changes
>> Also, make sure to start the actual text at the margin.
>> =========================================================
>>
>> +* mempool: The API of ``rte_mempool_populate_iova()`` and
>> + ``rte_mempool_populate_virt()`` changed to return 0 instead of -EINVAL
>> + when there is not enough room to store one object.
>> +
>>
>> ABI Changes
>> -----------
>> diff --git a/examples/ntb/ntb_fwd.c b/examples/ntb/ntb_fwd.c
>> index d49189e175..eba8ebf9fa 100644
>> --- a/examples/ntb/ntb_fwd.c
>> +++ b/examples/ntb/ntb_fwd.c
>> @@ -1319,7 +1319,7 @@ ntb_mbuf_pool_create(uint16_t mbuf_seg_size, uint32_t nb_mbuf,
>> mz->len - ntb_info.ntb_hdr_size,
>> ntb_mempool_mz_free,
>> (void *)(uintptr_t)mz);
>> - if (ret < 0) {
>> + if (ret <= 0) {
>> rte_memzone_free(mz);
>> rte_mempool_free(mp);
>> return NULL;
>> diff --git a/lib/librte_mempool/meson.build b/lib/librte_mempool/meson.build
>> index a6e861cbfc..7dbe6b9bea 100644
>> --- a/lib/librte_mempool/meson.build
>> +++ b/lib/librte_mempool/meson.build
>> @@ -9,6 +9,8 @@ foreach flag: extra_flags
>> endif
>> endforeach
>>
>> +use_function_versioning = true
>> +
>> sources = files('rte_mempool.c', 'rte_mempool_ops.c',
>> 'rte_mempool_ops_default.c', 'mempool_trace_points.c')
>> headers = files('rte_mempool.h', 'rte_mempool_trace.h',
>> diff --git a/lib/librte_mempool/rte_mempool.c b/lib/librte_mempool/rte_mempool.c
>> index 0be8f9f59d..edbdafaafb 100644
>> --- a/lib/librte_mempool/rte_mempool.c
>> +++ b/lib/librte_mempool/rte_mempool.c
>> @@ -31,6 +31,7 @@
>> #include <rte_string_fns.h>
>> #include <rte_spinlock.h>
>> #include <rte_tailq.h>
>> +#include <rte_function_versioning.h>
>>
>> #include "rte_mempool.h"
>> #include "rte_mempool_trace.h"
>> @@ -303,12 +304,17 @@ mempool_ops_alloc_once(struct rte_mempool *mp)
>> return 0;
>> }
>>
>> +__vsym int
>> +rte_mempool_populate_iova_v20_0_2(struct rte_mempool *mp, char *vaddr,
>> + rte_iova_t iova, size_t len, rte_mempool_memchunk_free_cb_t *free_cb,
>> + void *opaque);
>> +
>> /* Add objects in the pool, using a physically contiguous memory
>> * zone. Return the number of objects added, or a negative value
>> * on error.
>> */
>> -static int
>> -__rte_mempool_populate_iova(struct rte_mempool *mp, char *vaddr,
>> +__vsym int
>> +rte_mempool_populate_iova_v20_0_2(struct rte_mempool *mp, char *vaddr,
>> rte_iova_t iova, size_t len, rte_mempool_memchunk_free_cb_t *free_cb,
>> void *opaque)
>> {
>> @@ -366,14 +372,27 @@ __rte_mempool_populate_iova(struct rte_mempool *mp, char *vaddr,
>> return ret;
>> }
>>
>> -int
>> -rte_mempool_populate_iova(struct rte_mempool *mp, char *vaddr,
>> +BIND_DEFAULT_SYMBOL(rte_mempool_populate_iova, _v20_0_2, 20.0.2);
> as discussed v21.
>
>> +MAP_STATIC_SYMBOL(
>> + int rte_mempool_populate_iova(struct rte_mempool *mp, char *vaddr,
>> + rte_iova_t iova, size_t len,
>> + rte_mempool_memchunk_free_cb_t *free_cb,
>> + void *opaque),
>> + rte_mempool_populate_iova_v20_0_2);
>> +
>> +__vsym int
>> +rte_mempool_populate_iova_v20(struct rte_mempool *mp, char *vaddr,
>> + rte_iova_t iova, size_t len, rte_mempool_memchunk_free_cb_t *free_cb,
>> + void *opaque);
>> +
>> +__vsym int
>> +rte_mempool_populate_iova_v20(struct rte_mempool *mp, char *vaddr,
>> rte_iova_t iova, size_t len, rte_mempool_memchunk_free_cb_t *free_cb,
>> void *opaque)
>> {
>> int ret;
>>
>> - ret = __rte_mempool_populate_iova(mp, vaddr, iova, len, free_cb,
>> + ret = rte_mempool_populate_iova_v20_0_2(mp, vaddr, iova, len, free_cb,
>> opaque);
>> if (ret == 0)
>> ret = -EINVAL;
>> @@ -381,6 +400,7 @@ rte_mempool_populate_iova(struct rte_mempool *mp, char *vaddr,
>> rte_mempool_trace_populate_iova(mp, vaddr, iova, len, free_cb, opaque);
>> return ret;
>> }
>> +VERSION_SYMBOL(rte_mempool_populate_iova, _v20_0, 20.0);
> this should be
>
> VERSION_SYMBOL(rte_mempool_populate_iova, _v20, 20.0);
>
>>
>> static rte_iova_t
>> get_iova(void *addr)
>> @@ -395,11 +415,16 @@ get_iova(void *addr)
>> return ms->iova + RTE_PTR_DIFF(addr, ms->addr);
>> }
>>
>> +__vsym int
>> +rte_mempool_populate_virt_v20_0_2(struct rte_mempool *mp, char *addr,
>> + size_t len, size_t pg_sz, rte_mempool_memchunk_free_cb_t *free_cb,
>> + void *opaque);
>> +
>> /* Populate the mempool with a virtual area. Return the number of
>> * objects added, or a negative value on error.
>> */
>> -int
>> -rte_mempool_populate_virt(struct rte_mempool *mp, char *addr,
>> +__vsym int
>> +rte_mempool_populate_virt_v20_0_2(struct rte_mempool *mp, char *addr,
>> size_t len, size_t pg_sz, rte_mempool_memchunk_free_cb_t *free_cb,
>> void *opaque)
>> {
>> @@ -432,7 +457,7 @@ rte_mempool_populate_virt(struct rte_mempool *mp, char *addr,
>> break;
>> }
>>
>> - ret = __rte_mempool_populate_iova(mp, addr + off, iova,
>> + ret = rte_mempool_populate_iova_v20_0_2(mp, addr + off, iova,
>> phys_len, free_cb, opaque);
>> if (ret == 0)
>> continue;
>> @@ -443,9 +468,6 @@ rte_mempool_populate_virt(struct rte_mempool *mp, char *addr,
>> cnt += ret;
>> }
>>
>> - if (cnt == 0)
>> - return -EINVAL;
>> -
>> rte_mempool_trace_populate_virt(mp, addr, len, pg_sz, free_cb, opaque);
>> return cnt;
>>
>> @@ -453,6 +475,35 @@ rte_mempool_populate_virt(struct rte_mempool *mp, char *addr,
>> rte_mempool_free_memchunks(mp);
>> return ret;
>> }
>> +BIND_DEFAULT_SYMBOL(rte_mempool_populate_virt, _v20_0_2, 20.0.2);
> as discussed v21.
>
>> +MAP_STATIC_SYMBOL(
>> + int rte_mempool_populate_virt(struct rte_mempool *mp,
>> + char *addr, size_t len, size_t pg_sz,
>> + rte_mempool_memchunk_free_cb_t *free_cb,
>> + void *opaque),
>> + rte_mempool_populate_virt_v20_0_2);
>> +
>> +__vsym int
>> +rte_mempool_populate_virt_v20(struct rte_mempool *mp, char *addr,
>> + size_t len, size_t pg_sz, rte_mempool_memchunk_free_cb_t *free_cb,
>> + void *opaque);
>> +
>> +__vsym int
>> +rte_mempool_populate_virt_v20(struct rte_mempool *mp, char *addr,
>> + size_t len, size_t pg_sz, rte_mempool_memchunk_free_cb_t *free_cb,
>> + void *opaque)
>> +{
>> + int ret;
>> +
>> + ret = rte_mempool_populate_virt_v20_0_2(mp, addr, len, pg_sz,
>> + free_cb, opaque);
>> +
>> + if (ret == 0)
>> + ret = -EINVAL;
>> +
>> + return ret;
>> +}
>> +VERSION_SYMBOL(rte_mempool_populate_virt, _v20_0, 20.0);
> this should be
> VERSION_SYMBOL(rte_mempool_populate_virt, _v20, 20.0);
>
>
>>
>> /* Get the minimal page size used in a mempool before populating it. */
>> int
>> @@ -609,6 +660,8 @@ rte_mempool_populate_default(struct rte_mempool *mp)
>> mz->len, pg_sz,
>> rte_mempool_memchunk_mz_free,
>> (void *)(uintptr_t)mz);
>> + if (ret == 0) /* should not happen */
>> + ret = -ENOBUFS;
>> if (ret < 0) {
>> rte_memzone_free(mz);
>> goto fail;
>> @@ -701,6 +754,8 @@ rte_mempool_populate_anon(struct rte_mempool *mp)
>>
>> ret = rte_mempool_populate_virt(mp, addr, size, getpagesize(),
>> rte_mempool_memchunk_anon_free, addr);
>> + if (ret == 0) /* should not happen */
>> + ret = -ENOBUFS;
>> if (ret < 0) {
>> rte_errno = -ret;
>> goto fail;
>> diff --git a/lib/librte_mempool/rte_mempool.h b/lib/librte_mempool/rte_mempool.h
>> index 6e0573ea42..652d19f9f1 100644
>> --- a/lib/librte_mempool/rte_mempool.h
>> +++ b/lib/librte_mempool/rte_mempool.h
>> @@ -1112,9 +1112,12 @@ rte_mempool_free(struct rte_mempool *mp);
>> * @param opaque
>> * An opaque argument passed to free_cb.
>> * @return
>> - * The number of objects added on success.
>> + * The number of objects added on success (strictly positive).
>> * On error, the chunk is not added in the memory list of the
>> - * mempool and a negative errno is returned.
>> + * mempool the following code is returned:
>> + * (0): not enough room in chunk for one object.
>> + * (-ENOSPC): mempool is already populated.
>> + * (-ENOMEM): allocation failure.
>> */
>> int rte_mempool_populate_iova(struct rte_mempool *mp, char *vaddr,
>> rte_iova_t iova, size_t len, rte_mempool_memchunk_free_cb_t *free_cb,
>> @@ -1139,9 +1142,12 @@ int rte_mempool_populate_iova(struct rte_mempool *mp, char *vaddr,
>> * @param opaque
>> * An opaque argument passed to free_cb.
>> * @return
>> - * The number of objects added on success.
>> + * The number of objects added on success (strictly positive).
>> * On error, the chunk is not added in the memory list of the
>> - * mempool and a negative errno is returned.
>> + * mempool the following code is returned:
>> + * (0): not enough room in chunk for one object.
>> + * (-ENOSPC): mempool is already populated.
>> + * (-ENOMEM): allocation failure.
>> */
>> int
>> rte_mempool_populate_virt(struct rte_mempool *mp, char *addr,
>> diff --git a/lib/librte_mempool/rte_mempool_version.map b/lib/librte_mempool/rte_mempool_version.map
>> index 695dd6e04f..9e58093665 100644
>> --- a/lib/librte_mempool/rte_mempool_version.map
>> +++ b/lib/librte_mempool/rte_mempool_version.map
>> @@ -31,6 +31,13 @@ DPDK_20.0 {
>> local: *;
>> };
>>
>> +DPDK_20.0.2 {
> as discussed DPDK_21
>
>> + global:
>> +
>> + rte_mempool_populate_iova;
>> + rte_mempool_populate_virt;
>> +} DPDK_20.0;
>> +
>> EXPERIMENTAL {
>> global:
>>
>>
--
Lukasz Wojciechowski
Principal Software Engineer
Samsung R&D Institute Poland
Samsung Electronics
Office +48 22 377 88 25
l.wojciechow@partner.samsung.com
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH v4 1/4] ethdev: add tm support for shaper config in pkt mode
2020-04-27 16:49 3% ` Ferruh Yigit
@ 2020-04-27 16:59 3% ` Jerin Jacob
2020-04-28 11:51 0% ` [dpdk-dev] [EXT] " Nithin Dabilpuram
2020-04-28 14:06 4% ` [dpdk-dev] " Ferruh Yigit
0 siblings, 2 replies; 200+ results
From: Jerin Jacob @ 2020-04-27 16:59 UTC (permalink / raw)
To: Ferruh Yigit
Cc: Dumitrescu, Cristian, Nithin Dabilpuram, Singh, Jasvinder,
Thomas Monjalon, Andrew Rybchenko, dev, jerinj, kkanas,
Nithin Dabilpuram, Kinsella, Ray, Neil Horman, Luca Boccassi,
Kevin Traynor, David Marchand, Bruce Richardson
On Mon, Apr 27, 2020 at 10:19 PM Ferruh Yigit <ferruh.yigit@intel.com> wrote:
>
> On 4/27/2020 5:29 PM, Jerin Jacob wrote:
> > On Mon, Apr 27, 2020 at 9:42 PM Ferruh Yigit <ferruh.yigit@intel.com> wrote:
> >>
> >> On 4/27/2020 10:19 AM, Dumitrescu, Cristian wrote:
> >>>
> >>>
> >>>> -----Original Message-----
> >>>> From: Yigit, Ferruh <ferruh.yigit@intel.com>
> >>>> Sent: Saturday, April 25, 2020 9:09 PM
> >>>> To: Dumitrescu, Cristian <cristian.dumitrescu@intel.com>; Nithin Dabilpuram
> >>>> <nithind1988@gmail.com>; Singh, Jasvinder <jasvinder.singh@intel.com>;
> >>>> Thomas Monjalon <thomas@monjalon.net>; Andrew Rybchenko
> >>>> <arybchenko@solarflare.com>
> >>>> Cc: dev@dpdk.org; jerinj@marvell.com; kkanas@marvell.com; Nithin
> >>>> Dabilpuram <ndabilpuram@marvell.com>
> >>>> Subject: Re: [PATCH v4 1/4] ethdev: add tm support for shaper config in pkt
> >>>> mode
> >>>>
> >>>> On 4/24/2020 11:28 AM, Dumitrescu, Cristian wrote:
> >>>>>
> >>>>>
> >>>>>> -----Original Message-----
> >>>>>> From: Nithin Dabilpuram <nithind1988@gmail.com>
> >>>>>> Sent: Wednesday, April 22, 2020 6:21 PM
> >>>>>> To: Singh, Jasvinder <jasvinder.singh@intel.com>; Dumitrescu, Cristian
> >>>>>> <cristian.dumitrescu@intel.com>; Thomas Monjalon
> >>>>>> <thomas@monjalon.net>; Yigit, Ferruh <ferruh.yigit@intel.com>; Andrew
> >>>>>> Rybchenko <arybchenko@solarflare.com>
> >>>>>> Cc: dev@dpdk.org; jerinj@marvell.com; kkanas@marvell.com; Nithin
> >>>>>> Dabilpuram <ndabilpuram@marvell.com>
> >>>>>> Subject: [PATCH v4 1/4] ethdev: add tm support for shaper config in pkt
> >>>>>> mode
> >>>>>>
> >>>>>> From: Nithin Dabilpuram <ndabilpuram@marvell.com>
> >>>>>>
> >>>>>> Some NIC hardware support shaper to work in packet mode i.e
> >>>>>> shaping or ratelimiting traffic is in packets per second (PPS) as
> >>>>>> opposed to default bytes per second (BPS). Hence this patch
> >>>>>> adds support to configure shared or private shaper in packet mode,
> >>>>>> provide rate in PPS and add related tm capabilities in port/level/node
> >>>>>> capability structures.
> >>>>>>
> >>>>>> This patch also updates tm port/level/node capability structures with
> >>>>>> exiting features of scheduler wfq packet mode, scheduler wfq byte mode
> >>>>>> and private/shared shaper byte mode.
> >>>>>>
> >>>>>> SoftNIC PMD is also updated with new capabilities.
> >>>>>>
> >>>>>> Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>
> >>>>>> ---
> >>>>>> v3..v4:
> >>>>>> - Update text under packet_mode as per Cristian.
> >>>>>> - Update rte_eth_softnic_tm.c based on Jasvinder's comments.
> >>>>>> - Add error enum
> >>>> RTE_TM_ERROR_TYPE_SHAPER_PROFILE_PACKET_MODE
> >>>>>> - Fix shaper_profile_check() with packet mode check
> >>>>>> - Fix typo's
> >>>>>>
> >>>>>
> >>>>> Acked-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
> >>>>>
> >>>>
> >>>> Hi Nithin,
> >>>>
> >>>> It looks like patch is causing ABI break, I am getting following warning [1],
> >>>> can you please check?
> >>>>
> >>>> [1]
> >>>> https://pastebin.com/XYNFg14u
> >>>
> >>>
> >>> Hi Ferruh,
> >>>
> >>> The RTE_TM API is marked as experimental, but it looks that this was not correctly marked when __rte_experimental ABI checker was introduced.
> >>>
> >>> It is marked as experimental at the top of the rte_tm.h, similarly to other APIs introduced around same time, but it was not correctly picked up by the ABI check procedure when later introduced, so __rte_experimental was not added to every function.
> >>>
> >>
> >> :(
> >>
> >> Is it time to mature them?
> >>
> >> As you said they are not marked as experimental both in header file (function
> >> declarations) and .map file.
> >>
> >> The problem is, they are not marked as experimental in DPDK_20.0 ABI (v19.11),
> >> so marking them as experimental now will break the ABI. Not sure what to do,
> >> cc'ed a few ABI related names for comment.
> >>
> >> For me, we need to proceed as the experimental tag removed and APIs become
> >> mature starting from v19.11, since this is what happened in practice, and remove
> >> a few existing being experimental references in the doxygen comments.
> >
> > I think, accidentally we can not make a library as NON-experimental.
> > TM never went through experimental to mature transition(see git log
> > lib/librte_ethdev/rte_tm.h)
> > It was a bug to not mark as experimental in each function in the ABI process.
> > Some of the features like packet marking are not even implemented by any HW.
> > I think, we can make API stable only all the features are implemented
> > by one or two HW.
>
> Fair enough, specially if the API is not ready yet.
>
> But they were part of stable ABI, and marking them as experimental now will
> break the old applications using these APIs.
it is still marked as EXPERIMENTAL everywhere and API is not ready yet.
Anyway, we need to break the ABI to make it work on various HW.
I am not sure what to do?
IMO, We need to send a patch as Fixes: for the bug of not adding
__rte_experimental in each function.
Traffic Management API - EXPERIMENTAL
M: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
T: git://dpdk.org/next/dpdk-next-qos
F: lib/librte_ethdev/rte_tm*
>
> >
> >>
> >> Ray, Neil, David, Luca, Kevin, what do you think?
>
^ permalink raw reply [relevance 3%]
* Re: [dpdk-dev] [PATCH v4 1/4] ethdev: add tm support for shaper config in pkt mode
2020-04-27 16:29 3% ` Jerin Jacob
@ 2020-04-27 16:49 3% ` Ferruh Yigit
2020-04-27 16:59 3% ` Jerin Jacob
0 siblings, 1 reply; 200+ results
From: Ferruh Yigit @ 2020-04-27 16:49 UTC (permalink / raw)
To: Jerin Jacob
Cc: Dumitrescu, Cristian, Nithin Dabilpuram, Singh, Jasvinder,
Thomas Monjalon, Andrew Rybchenko, dev, jerinj, kkanas,
Nithin Dabilpuram, Kinsella, Ray, Neil Horman, Luca Boccassi,
Kevin Traynor, David Marchand, Bruce Richardson
On 4/27/2020 5:29 PM, Jerin Jacob wrote:
> On Mon, Apr 27, 2020 at 9:42 PM Ferruh Yigit <ferruh.yigit@intel.com> wrote:
>>
>> On 4/27/2020 10:19 AM, Dumitrescu, Cristian wrote:
>>>
>>>
>>>> -----Original Message-----
>>>> From: Yigit, Ferruh <ferruh.yigit@intel.com>
>>>> Sent: Saturday, April 25, 2020 9:09 PM
>>>> To: Dumitrescu, Cristian <cristian.dumitrescu@intel.com>; Nithin Dabilpuram
>>>> <nithind1988@gmail.com>; Singh, Jasvinder <jasvinder.singh@intel.com>;
>>>> Thomas Monjalon <thomas@monjalon.net>; Andrew Rybchenko
>>>> <arybchenko@solarflare.com>
>>>> Cc: dev@dpdk.org; jerinj@marvell.com; kkanas@marvell.com; Nithin
>>>> Dabilpuram <ndabilpuram@marvell.com>
>>>> Subject: Re: [PATCH v4 1/4] ethdev: add tm support for shaper config in pkt
>>>> mode
>>>>
>>>> On 4/24/2020 11:28 AM, Dumitrescu, Cristian wrote:
>>>>>
>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Nithin Dabilpuram <nithind1988@gmail.com>
>>>>>> Sent: Wednesday, April 22, 2020 6:21 PM
>>>>>> To: Singh, Jasvinder <jasvinder.singh@intel.com>; Dumitrescu, Cristian
>>>>>> <cristian.dumitrescu@intel.com>; Thomas Monjalon
>>>>>> <thomas@monjalon.net>; Yigit, Ferruh <ferruh.yigit@intel.com>; Andrew
>>>>>> Rybchenko <arybchenko@solarflare.com>
>>>>>> Cc: dev@dpdk.org; jerinj@marvell.com; kkanas@marvell.com; Nithin
>>>>>> Dabilpuram <ndabilpuram@marvell.com>
>>>>>> Subject: [PATCH v4 1/4] ethdev: add tm support for shaper config in pkt
>>>>>> mode
>>>>>>
>>>>>> From: Nithin Dabilpuram <ndabilpuram@marvell.com>
>>>>>>
>>>>>> Some NIC hardware support shaper to work in packet mode i.e
>>>>>> shaping or ratelimiting traffic is in packets per second (PPS) as
>>>>>> opposed to default bytes per second (BPS). Hence this patch
>>>>>> adds support to configure shared or private shaper in packet mode,
>>>>>> provide rate in PPS and add related tm capabilities in port/level/node
>>>>>> capability structures.
>>>>>>
>>>>>> This patch also updates tm port/level/node capability structures with
>>>>>> exiting features of scheduler wfq packet mode, scheduler wfq byte mode
>>>>>> and private/shared shaper byte mode.
>>>>>>
>>>>>> SoftNIC PMD is also updated with new capabilities.
>>>>>>
>>>>>> Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>
>>>>>> ---
>>>>>> v3..v4:
>>>>>> - Update text under packet_mode as per Cristian.
>>>>>> - Update rte_eth_softnic_tm.c based on Jasvinder's comments.
>>>>>> - Add error enum
>>>> RTE_TM_ERROR_TYPE_SHAPER_PROFILE_PACKET_MODE
>>>>>> - Fix shaper_profile_check() with packet mode check
>>>>>> - Fix typo's
>>>>>>
>>>>>
>>>>> Acked-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
>>>>>
>>>>
>>>> Hi Nithin,
>>>>
>>>> It looks like patch is causing ABI break, I am getting following warning [1],
>>>> can you please check?
>>>>
>>>> [1]
>>>> https://pastebin.com/XYNFg14u
>>>
>>>
>>> Hi Ferruh,
>>>
>>> The RTE_TM API is marked as experimental, but it looks that this was not correctly marked when __rte_experimental ABI checker was introduced.
>>>
>>> It is marked as experimental at the top of the rte_tm.h, similarly to other APIs introduced around same time, but it was not correctly picked up by the ABI check procedure when later introduced, so __rte_experimental was not added to every function.
>>>
>>
>> :(
>>
>> Is it time to mature them?
>>
>> As you said they are not marked as experimental both in header file (function
>> declarations) and .map file.
>>
>> The problem is, they are not marked as experimental in DPDK_20.0 ABI (v19.11),
>> so marking them as experimental now will break the ABI. Not sure what to do,
>> cc'ed a few ABI related names for comment.
>>
>> For me, we need to proceed as the experimental tag removed and APIs become
>> mature starting from v19.11, since this is what happened in practice, and remove
>> a few existing being experimental references in the doxygen comments.
>
> I think, accidentally we can not make a library as NON-experimental.
> TM never went through experimental to mature transition(see git log
> lib/librte_ethdev/rte_tm.h)
> It was a bug to not mark as experimental in each function in the ABI process.
> Some of the features like packet marking are not even implemented by any HW.
> I think, we can make API stable only all the features are implemented
> by one or two HW.
Fair enough, specially if the API is not ready yet.
But they were part of stable ABI, and marking them as experimental now will
break the old applications using these APIs.
>
>>
>> Ray, Neil, David, Luca, Kevin, what do you think?
^ permalink raw reply [relevance 3%]
* Re: [dpdk-dev] [PATCH v4 1/4] ethdev: add tm support for shaper config in pkt mode
2020-04-27 16:12 4% ` Ferruh Yigit
2020-04-27 16:28 3% ` Dumitrescu, Cristian
@ 2020-04-27 16:29 3% ` Jerin Jacob
2020-04-27 16:49 3% ` Ferruh Yigit
1 sibling, 1 reply; 200+ results
From: Jerin Jacob @ 2020-04-27 16:29 UTC (permalink / raw)
To: Ferruh Yigit
Cc: Dumitrescu, Cristian, Nithin Dabilpuram, Singh, Jasvinder,
Thomas Monjalon, Andrew Rybchenko, dev, jerinj, kkanas,
Nithin Dabilpuram, Kinsella, Ray, Neil Horman, Luca Boccassi,
Kevin Traynor, David Marchand, Bruce Richardson
On Mon, Apr 27, 2020 at 9:42 PM Ferruh Yigit <ferruh.yigit@intel.com> wrote:
>
> On 4/27/2020 10:19 AM, Dumitrescu, Cristian wrote:
> >
> >
> >> -----Original Message-----
> >> From: Yigit, Ferruh <ferruh.yigit@intel.com>
> >> Sent: Saturday, April 25, 2020 9:09 PM
> >> To: Dumitrescu, Cristian <cristian.dumitrescu@intel.com>; Nithin Dabilpuram
> >> <nithind1988@gmail.com>; Singh, Jasvinder <jasvinder.singh@intel.com>;
> >> Thomas Monjalon <thomas@monjalon.net>; Andrew Rybchenko
> >> <arybchenko@solarflare.com>
> >> Cc: dev@dpdk.org; jerinj@marvell.com; kkanas@marvell.com; Nithin
> >> Dabilpuram <ndabilpuram@marvell.com>
> >> Subject: Re: [PATCH v4 1/4] ethdev: add tm support for shaper config in pkt
> >> mode
> >>
> >> On 4/24/2020 11:28 AM, Dumitrescu, Cristian wrote:
> >>>
> >>>
> >>>> -----Original Message-----
> >>>> From: Nithin Dabilpuram <nithind1988@gmail.com>
> >>>> Sent: Wednesday, April 22, 2020 6:21 PM
> >>>> To: Singh, Jasvinder <jasvinder.singh@intel.com>; Dumitrescu, Cristian
> >>>> <cristian.dumitrescu@intel.com>; Thomas Monjalon
> >>>> <thomas@monjalon.net>; Yigit, Ferruh <ferruh.yigit@intel.com>; Andrew
> >>>> Rybchenko <arybchenko@solarflare.com>
> >>>> Cc: dev@dpdk.org; jerinj@marvell.com; kkanas@marvell.com; Nithin
> >>>> Dabilpuram <ndabilpuram@marvell.com>
> >>>> Subject: [PATCH v4 1/4] ethdev: add tm support for shaper config in pkt
> >>>> mode
> >>>>
> >>>> From: Nithin Dabilpuram <ndabilpuram@marvell.com>
> >>>>
> >>>> Some NIC hardware support shaper to work in packet mode i.e
> >>>> shaping or ratelimiting traffic is in packets per second (PPS) as
> >>>> opposed to default bytes per second (BPS). Hence this patch
> >>>> adds support to configure shared or private shaper in packet mode,
> >>>> provide rate in PPS and add related tm capabilities in port/level/node
> >>>> capability structures.
> >>>>
> >>>> This patch also updates tm port/level/node capability structures with
> >>>> exiting features of scheduler wfq packet mode, scheduler wfq byte mode
> >>>> and private/shared shaper byte mode.
> >>>>
> >>>> SoftNIC PMD is also updated with new capabilities.
> >>>>
> >>>> Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>
> >>>> ---
> >>>> v3..v4:
> >>>> - Update text under packet_mode as per Cristian.
> >>>> - Update rte_eth_softnic_tm.c based on Jasvinder's comments.
> >>>> - Add error enum
> >> RTE_TM_ERROR_TYPE_SHAPER_PROFILE_PACKET_MODE
> >>>> - Fix shaper_profile_check() with packet mode check
> >>>> - Fix typo's
> >>>>
> >>>
> >>> Acked-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
> >>>
> >>
> >> Hi Nithin,
> >>
> >> It looks like patch is causing ABI break, I am getting following warning [1],
> >> can you please check?
> >>
> >> [1]
> >> https://pastebin.com/XYNFg14u
> >
> >
> > Hi Ferruh,
> >
> > The RTE_TM API is marked as experimental, but it looks that this was not correctly marked when __rte_experimental ABI checker was introduced.
> >
> > It is marked as experimental at the top of the rte_tm.h, similarly to other APIs introduced around same time, but it was not correctly picked up by the ABI check procedure when later introduced, so __rte_experimental was not added to every function.
> >
>
> :(
>
> Is it time to mature them?
>
> As you said they are not marked as experimental both in header file (function
> declarations) and .map file.
>
> The problem is, they are not marked as experimental in DPDK_20.0 ABI (v19.11),
> so marking them as experimental now will break the ABI. Not sure what to do,
> cc'ed a few ABI related names for comment.
>
> For me, we need to proceed as the experimental tag removed and APIs become
> mature starting from v19.11, since this is what happened in practice, and remove
> a few existing being experimental references in the doxygen comments.
I think, accidentally we can not make a library as NON-experimental.
TM never went through experimental to mature transition(see git log
lib/librte_ethdev/rte_tm.h)
It was a bug to not mark as experimental in each function in the ABI process.
Some of the features like packet marking are not even implemented by any HW.
I think, we can make API stable only all the features are implemented
by one or two HW.
>
> Ray, Neil, David, Luca, Kevin, what do you think?
^ permalink raw reply [relevance 3%]
* Re: [dpdk-dev] [PATCH v4 1/4] ethdev: add tm support for shaper config in pkt mode
2020-04-27 16:12 4% ` Ferruh Yigit
@ 2020-04-27 16:28 3% ` Dumitrescu, Cristian
2020-04-28 15:30 4% ` Thomas Monjalon
2020-04-27 16:29 3% ` Jerin Jacob
1 sibling, 1 reply; 200+ results
From: Dumitrescu, Cristian @ 2020-04-27 16:28 UTC (permalink / raw)
To: Yigit, Ferruh, Nithin Dabilpuram, Singh, Jasvinder,
Thomas Monjalon, Andrew Rybchenko
Cc: dev, jerinj, kkanas, Nithin Dabilpuram, Kinsella, Ray,
Neil Horman, Luca Boccassi, Kevin Traynor, David Marchand,
Richardson, Bruce
> -----Original Message-----
> From: Yigit, Ferruh <ferruh.yigit@intel.com>
> Sent: Monday, April 27, 2020 5:13 PM
> To: Dumitrescu, Cristian <cristian.dumitrescu@intel.com>; Nithin Dabilpuram
> <nithind1988@gmail.com>; Singh, Jasvinder <jasvinder.singh@intel.com>;
> Thomas Monjalon <thomas@monjalon.net>; Andrew Rybchenko
> <arybchenko@solarflare.com>
> Cc: dev@dpdk.org; jerinj@marvell.com; kkanas@marvell.com; Nithin
> Dabilpuram <ndabilpuram@marvell.com>; Kinsella, Ray
> <ray.kinsella@intel.com>; Neil Horman <nhorman@tuxdriver.com>; Luca
> Boccassi <bluca@debian.org>; Kevin Traynor <ktraynor@redhat.com>; David
> Marchand <david.marchand@redhat.com>; Richardson, Bruce
> <bruce.richardson@intel.com>
> Subject: Re: [PATCH v4 1/4] ethdev: add tm support for shaper config in pkt
> mode
>
> On 4/27/2020 10:19 AM, Dumitrescu, Cristian wrote:
> >
> >
> >> -----Original Message-----
> >> From: Yigit, Ferruh <ferruh.yigit@intel.com>
> >> Sent: Saturday, April 25, 2020 9:09 PM
> >> To: Dumitrescu, Cristian <cristian.dumitrescu@intel.com>; Nithin
> Dabilpuram
> >> <nithind1988@gmail.com>; Singh, Jasvinder <jasvinder.singh@intel.com>;
> >> Thomas Monjalon <thomas@monjalon.net>; Andrew Rybchenko
> >> <arybchenko@solarflare.com>
> >> Cc: dev@dpdk.org; jerinj@marvell.com; kkanas@marvell.com; Nithin
> >> Dabilpuram <ndabilpuram@marvell.com>
> >> Subject: Re: [PATCH v4 1/4] ethdev: add tm support for shaper config in
> pkt
> >> mode
> >>
> >> On 4/24/2020 11:28 AM, Dumitrescu, Cristian wrote:
> >>>
> >>>
> >>>> -----Original Message-----
> >>>> From: Nithin Dabilpuram <nithind1988@gmail.com>
> >>>> Sent: Wednesday, April 22, 2020 6:21 PM
> >>>> To: Singh, Jasvinder <jasvinder.singh@intel.com>; Dumitrescu, Cristian
> >>>> <cristian.dumitrescu@intel.com>; Thomas Monjalon
> >>>> <thomas@monjalon.net>; Yigit, Ferruh <ferruh.yigit@intel.com>;
> Andrew
> >>>> Rybchenko <arybchenko@solarflare.com>
> >>>> Cc: dev@dpdk.org; jerinj@marvell.com; kkanas@marvell.com; Nithin
> >>>> Dabilpuram <ndabilpuram@marvell.com>
> >>>> Subject: [PATCH v4 1/4] ethdev: add tm support for shaper config in pkt
> >>>> mode
> >>>>
> >>>> From: Nithin Dabilpuram <ndabilpuram@marvell.com>
> >>>>
> >>>> Some NIC hardware support shaper to work in packet mode i.e
> >>>> shaping or ratelimiting traffic is in packets per second (PPS) as
> >>>> opposed to default bytes per second (BPS). Hence this patch
> >>>> adds support to configure shared or private shaper in packet mode,
> >>>> provide rate in PPS and add related tm capabilities in port/level/node
> >>>> capability structures.
> >>>>
> >>>> This patch also updates tm port/level/node capability structures with
> >>>> exiting features of scheduler wfq packet mode, scheduler wfq byte
> mode
> >>>> and private/shared shaper byte mode.
> >>>>
> >>>> SoftNIC PMD is also updated with new capabilities.
> >>>>
> >>>> Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>
> >>>> ---
> >>>> v3..v4:
> >>>> - Update text under packet_mode as per Cristian.
> >>>> - Update rte_eth_softnic_tm.c based on Jasvinder's comments.
> >>>> - Add error enum
> >> RTE_TM_ERROR_TYPE_SHAPER_PROFILE_PACKET_MODE
> >>>> - Fix shaper_profile_check() with packet mode check
> >>>> - Fix typo's
> >>>>
> >>>
> >>> Acked-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
> >>>
> >>
> >> Hi Nithin,
> >>
> >> It looks like patch is causing ABI break, I am getting following warning [1],
> >> can you please check?
> >>
> >> [1]
> >> https://pastebin.com/XYNFg14u
> >
> >
> > Hi Ferruh,
> >
> > The RTE_TM API is marked as experimental, but it looks that this was not
> correctly marked when __rte_experimental ABI checker was introduced.
> >
> > It is marked as experimental at the top of the rte_tm.h, similarly to other
> APIs introduced around same time, but it was not correctly picked up by the
> ABI check procedure when later introduced, so __rte_experimental was not
> added to every function.
> >
>
> :(
>
> Is it time to mature them?
>
> As you said they are not marked as experimental both in header file
> (function
> declarations) and .map file.
>
> The problem is, they are not marked as experimental in DPDK_20.0 ABI
> (v19.11),
> so marking them as experimental now will break the ABI. Not sure what to
> do,
> cc'ed a few ABI related names for comment.
>
> For me, we need to proceed as the experimental tag removed and APIs
> become
> mature starting from v19.11, since this is what happened in practice, and
> remove
> a few existing being experimental references in the doxygen comments.
>
> Ray, Neil, David, Luca, Kevin, what do you think?
Hi Ferruh,
IMO your proposed approach is fixing the wrong problem and is probably not the right way of doing things.
This API is correctly marked as experimental in the header file rte_tm.h and in the MAINTAINERS file, therefore it should remain experimental, as more changes are expected from the people like Nithin and others currently upstreaming drivers for it.
For some reason, the __rte_experimental tags were not added to this file when the ABI checker was introduced, and this is the real problem that should be fixed.
Regards,
Cristian
^ permalink raw reply [relevance 3%]
* Re: [dpdk-dev] [PATCH v4 1/4] ethdev: add tm support for shaper config in pkt mode
2020-04-27 9:19 4% ` Dumitrescu, Cristian
@ 2020-04-27 16:12 4% ` Ferruh Yigit
2020-04-27 16:28 3% ` Dumitrescu, Cristian
2020-04-27 16:29 3% ` Jerin Jacob
0 siblings, 2 replies; 200+ results
From: Ferruh Yigit @ 2020-04-27 16:12 UTC (permalink / raw)
To: Dumitrescu, Cristian, Nithin Dabilpuram, Singh, Jasvinder,
Thomas Monjalon, Andrew Rybchenko
Cc: dev, jerinj, kkanas, Nithin Dabilpuram, Kinsella, Ray,
Neil Horman, Luca Boccassi, Kevin Traynor, David Marchand,
Bruce Richardson
On 4/27/2020 10:19 AM, Dumitrescu, Cristian wrote:
>
>
>> -----Original Message-----
>> From: Yigit, Ferruh <ferruh.yigit@intel.com>
>> Sent: Saturday, April 25, 2020 9:09 PM
>> To: Dumitrescu, Cristian <cristian.dumitrescu@intel.com>; Nithin Dabilpuram
>> <nithind1988@gmail.com>; Singh, Jasvinder <jasvinder.singh@intel.com>;
>> Thomas Monjalon <thomas@monjalon.net>; Andrew Rybchenko
>> <arybchenko@solarflare.com>
>> Cc: dev@dpdk.org; jerinj@marvell.com; kkanas@marvell.com; Nithin
>> Dabilpuram <ndabilpuram@marvell.com>
>> Subject: Re: [PATCH v4 1/4] ethdev: add tm support for shaper config in pkt
>> mode
>>
>> On 4/24/2020 11:28 AM, Dumitrescu, Cristian wrote:
>>>
>>>
>>>> -----Original Message-----
>>>> From: Nithin Dabilpuram <nithind1988@gmail.com>
>>>> Sent: Wednesday, April 22, 2020 6:21 PM
>>>> To: Singh, Jasvinder <jasvinder.singh@intel.com>; Dumitrescu, Cristian
>>>> <cristian.dumitrescu@intel.com>; Thomas Monjalon
>>>> <thomas@monjalon.net>; Yigit, Ferruh <ferruh.yigit@intel.com>; Andrew
>>>> Rybchenko <arybchenko@solarflare.com>
>>>> Cc: dev@dpdk.org; jerinj@marvell.com; kkanas@marvell.com; Nithin
>>>> Dabilpuram <ndabilpuram@marvell.com>
>>>> Subject: [PATCH v4 1/4] ethdev: add tm support for shaper config in pkt
>>>> mode
>>>>
>>>> From: Nithin Dabilpuram <ndabilpuram@marvell.com>
>>>>
>>>> Some NIC hardware support shaper to work in packet mode i.e
>>>> shaping or ratelimiting traffic is in packets per second (PPS) as
>>>> opposed to default bytes per second (BPS). Hence this patch
>>>> adds support to configure shared or private shaper in packet mode,
>>>> provide rate in PPS and add related tm capabilities in port/level/node
>>>> capability structures.
>>>>
>>>> This patch also updates tm port/level/node capability structures with
>>>> exiting features of scheduler wfq packet mode, scheduler wfq byte mode
>>>> and private/shared shaper byte mode.
>>>>
>>>> SoftNIC PMD is also updated with new capabilities.
>>>>
>>>> Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>
>>>> ---
>>>> v3..v4:
>>>> - Update text under packet_mode as per Cristian.
>>>> - Update rte_eth_softnic_tm.c based on Jasvinder's comments.
>>>> - Add error enum
>> RTE_TM_ERROR_TYPE_SHAPER_PROFILE_PACKET_MODE
>>>> - Fix shaper_profile_check() with packet mode check
>>>> - Fix typo's
>>>>
>>>
>>> Acked-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
>>>
>>
>> Hi Nithin,
>>
>> It looks like patch is causing ABI break, I am getting following warning [1],
>> can you please check?
>>
>> [1]
>> https://pastebin.com/XYNFg14u
>
>
> Hi Ferruh,
>
> The RTE_TM API is marked as experimental, but it looks that this was not correctly marked when __rte_experimental ABI checker was introduced.
>
> It is marked as experimental at the top of the rte_tm.h, similarly to other APIs introduced around same time, but it was not correctly picked up by the ABI check procedure when later introduced, so __rte_experimental was not added to every function.
>
:(
Is it time to mature them?
As you said they are not marked as experimental both in header file (function
declarations) and .map file.
The problem is, they are not marked as experimental in DPDK_20.0 ABI (v19.11),
so marking them as experimental now will break the ABI. Not sure what to do,
cc'ed a few ABI related names for comment.
For me, we need to proceed as the experimental tag removed and APIs become
mature starting from v19.11, since this is what happened in practice, and remove
a few existing being experimental references in the doxygen comments.
Ray, Neil, David, Luca, Kevin, what do you think?
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH v4] eal/cpuflags: add x86 based cpu flags
2020-04-27 12:31 3% ` Thomas Monjalon
@ 2020-04-27 13:58 0% ` Ray Kinsella
2020-04-29 11:22 0% ` Neil Horman
0 siblings, 1 reply; 200+ results
From: Ray Kinsella @ 2020-04-27 13:58 UTC (permalink / raw)
To: Thomas Monjalon, ray.kinsella, nhorman, Kevin Laatz
Cc: dev, bruce.richardson, harry.van.haaren, david.marchand,
Haiyue Wang, ktraynor
On 27/04/2020 13:31, Thomas Monjalon wrote:
> 27/04/2020 11:27, Ray Kinsella:
>> On 25/04/2020 17:04, Thomas Monjalon wrote:
>>> PS: Who is REALLY maintaining the ABI?
>>> We really miss someone who carefully check all these things,
>>> and take care of the doc and tooling.
>>
>> I would say that I am missing these changes to libabigail.ignore, which would be useful.
>> Should we consolidate the ABI Policy and ABI Versioning sections of the MAINTAINERS file?
>
> Yes, I think it does not make sense spliting ABI topic in 2 sections
> in MAINTAINERS file.
> We need to have a clear ownership covering policy, libs, tooling and doc.
> Let's agree to merge all in one section please.
>
I would suggest merging and listing myself and Neil as maintainers?
Unless you are aware of another potential owner?
Ray K
^ permalink raw reply [relevance 0%]
* [dpdk-dev] [PATCH v4] abi: change references to abi 20.0.1 to abi v21
2020-04-27 13:45 11% ` [dpdk-dev] [PATCH v4] " Ray Kinsella
@ 2020-04-27 13:45 21% ` Ray Kinsella
0 siblings, 0 replies; 200+ results
From: Ray Kinsella @ 2020-04-27 13:45 UTC (permalink / raw)
To: dev
Cc: aostruszka, stephen, david.marchand, Ray Kinsella,
Thomas Monjalon, Neil Horman, Jingjing Wu, Wenzhuo Lu,
Matan Azrad, Shahaf Shuler, Viacheslav Ovsiienko, Jerin Jacob,
Nithin Dabilpuram, Alfredo Cardigliano, Mahipal Challa,
Cristian Dumitrescu
Change references to abi 20.0.1 to use abi v21, see
https://doc.dpdk.org/guides/contributing/abi_policy.html#general-guidelines.
"Major ABI versions are declared no more frequently than yearly.
Compatibility with the major ABI version is mandatory in subsequent
releases until a new major ABI version is declared."
Add myself as the map file maintainer to more closely monitor future abi
changes.
Signed-off-by: Ray Kinsella <mdr@ashroe.eu>
---
MAINTAINERS | 2 ++
devtools/libabigail.abignore | 5 +++++
drivers/common/iavf/rte_common_iavf_version.map | 2 +-
drivers/common/mlx5/rte_common_mlx5_version.map | 2 +-
drivers/common/octeontx2/rte_common_octeontx2_version.map | 2 +-
drivers/net/ionic/rte_pmd_ionic_version.map | 2 +-
drivers/raw/octeontx2_ep/rte_rawdev_octeontx2_ep_version.map | 2 +-
drivers/vdpa/mlx5/rte_pmd_mlx5_vdpa_version.map | 2 +-
lib/librte_meter/rte_meter_version.map | 2 +-
9 files changed, 14 insertions(+), 7 deletions(-)
diff --git a/MAINTAINERS b/MAINTAINERS
index d31a809..7d521f1 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -86,6 +86,8 @@ F: doc/
ABI Policy
M: Ray Kinsella <mdr@ashroe.eu>
F: doc/guides/contributing/abi_*.rst
+F: drivers/*/*/*.map
+F: lib/*/*.map
Developers and Maintainers Tools
M: Thomas Monjalon <thomas@monjalon.net>
diff --git a/devtools/libabigail.abignore b/devtools/libabigail.abignore
index 986a527..b0147fd 100644
--- a/devtools/libabigail.abignore
+++ b/devtools/libabigail.abignore
@@ -29,3 +29,8 @@
type_kind = enum
name = rte_eth_event_type
changed_enumerators = RTE_ETH_EVENT_MAX
+; Explicit ignore ABI 20.0.1
+[suppress_function]
+ symbol_version = DPDK_20.0.1
+[suppress_variable]
+ symbol_version = DPDK_20.0.1
diff --git a/drivers/common/iavf/rte_common_iavf_version.map b/drivers/common/iavf/rte_common_iavf_version.map
index 2f11d67..92ceac1 100644
--- a/drivers/common/iavf/rte_common_iavf_version.map
+++ b/drivers/common/iavf/rte_common_iavf_version.map
@@ -1,4 +1,4 @@
-DPDK_20.0.1 {
+DPDK_21 {
global:
iavf_init_adminq;
diff --git a/drivers/common/mlx5/rte_common_mlx5_version.map b/drivers/common/mlx5/rte_common_mlx5_version.map
index b58a378..564a9a7 100644
--- a/drivers/common/mlx5/rte_common_mlx5_version.map
+++ b/drivers/common/mlx5/rte_common_mlx5_version.map
@@ -1,4 +1,4 @@
-DPDK_20.0.1 {
+DPDK_21 {
global:
mlx5_class_get;
diff --git a/drivers/common/octeontx2/rte_common_octeontx2_version.map b/drivers/common/octeontx2/rte_common_octeontx2_version.map
index 8f2404b..01279c3 100644
--- a/drivers/common/octeontx2/rte_common_octeontx2_version.map
+++ b/drivers/common/octeontx2/rte_common_octeontx2_version.map
@@ -34,7 +34,7 @@ DPDK_20.0 {
local: *;
};
-DPDK_20.0.1 {
+DPDK_21 {
global:
otx2_eth_dev_is_sec_capable;
diff --git a/drivers/net/ionic/rte_pmd_ionic_version.map b/drivers/net/ionic/rte_pmd_ionic_version.map
index bc8fd6d..acdaf58 100644
--- a/drivers/net/ionic/rte_pmd_ionic_version.map
+++ b/drivers/net/ionic/rte_pmd_ionic_version.map
@@ -1,4 +1,4 @@
-DPDK_20.0.1 {
+DPDK_21 {
local: *;
};
diff --git a/drivers/raw/octeontx2_ep/rte_rawdev_octeontx2_ep_version.map b/drivers/raw/octeontx2_ep/rte_rawdev_octeontx2_ep_version.map
index bc8fd6d..acdaf58 100644
--- a/drivers/raw/octeontx2_ep/rte_rawdev_octeontx2_ep_version.map
+++ b/drivers/raw/octeontx2_ep/rte_rawdev_octeontx2_ep_version.map
@@ -1,4 +1,4 @@
-DPDK_20.0.1 {
+DPDK_21 {
local: *;
};
diff --git a/drivers/vdpa/mlx5/rte_pmd_mlx5_vdpa_version.map b/drivers/vdpa/mlx5/rte_pmd_mlx5_vdpa_version.map
index 179f7f1..4a76d1d 100644
--- a/drivers/vdpa/mlx5/rte_pmd_mlx5_vdpa_version.map
+++ b/drivers/vdpa/mlx5/rte_pmd_mlx5_vdpa_version.map
@@ -1,3 +1,3 @@
-DPDK_20.0.1 {
+DPDK_21 {
local: *;
};
diff --git a/lib/librte_meter/rte_meter_version.map b/lib/librte_meter/rte_meter_version.map
index fc12cc0..2c7dadb 100644
--- a/lib/librte_meter/rte_meter_version.map
+++ b/lib/librte_meter/rte_meter_version.map
@@ -13,7 +13,7 @@ DPDK_20.0 {
local: *;
};
-DPDK_20.0.1 {
+DPDK_21 {
global:
rte_meter_trtcm_rfc4115_color_aware_check;
--
2.7.4
^ permalink raw reply [relevance 21%]
* [dpdk-dev] [PATCH v4] abi: change references to abi 20.0.1 to abi v21
` (2 preceding siblings ...)
2020-04-27 9:06 11% ` [dpdk-dev] [PATCH v3] " Ray Kinsella
@ 2020-04-27 13:45 11% ` Ray Kinsella
2020-04-27 13:45 21% ` Ray Kinsella
2020-04-30 10:21 31% ` [dpdk-dev] [PATCH v5] " Ray Kinsella
2020-04-30 10:27 11% ` [dpdk-dev] [PATCH v6 1/1] " Ray Kinsella
5 siblings, 1 reply; 200+ results
From: Ray Kinsella @ 2020-04-27 13:45 UTC (permalink / raw)
To: dev
Cc: aostruszka, stephen, david.marchand, Ray Kinsella,
Thomas Monjalon, Neil Horman, Jingjing Wu, Wenzhuo Lu,
Matan Azrad, Shahaf Shuler, Viacheslav Ovsiienko, Jerin Jacob,
Nithin Dabilpuram, Alfredo Cardigliano, Mahipal Challa,
Cristian Dumitrescu
v4:
* restored suppressions for v20.0.1 incorrectly removed in v3.
Ray Kinsella (1):
abi: change references to abi 20.0.1 to abi v21
MAINTAINERS | 2 ++
devtools/libabigail.abignore | 5 +++++
drivers/common/iavf/rte_common_iavf_version.map | 2 +-
drivers/common/mlx5/rte_common_mlx5_version.map | 2 +-
drivers/common/octeontx2/rte_common_octeontx2_version.map | 2 +-
drivers/net/ionic/rte_pmd_ionic_version.map | 2 +-
drivers/raw/octeontx2_ep/rte_rawdev_octeontx2_ep_version.map | 2 +-
drivers/vdpa/mlx5/rte_pmd_mlx5_vdpa_version.map | 2 +-
lib/librte_meter/rte_meter_version.map | 2 +-
9 files changed, 14 insertions(+), 7 deletions(-)
--
2.7.4
^ permalink raw reply [relevance 11%]
* Re: [dpdk-dev] [PATCH v4] eal/cpuflags: add x86 based cpu flags
2020-04-27 9:27 4% ` Ray Kinsella
2020-04-27 9:31 0% ` Laatz, Kevin
@ 2020-04-27 12:31 3% ` Thomas Monjalon
2020-04-27 13:58 0% ` Ray Kinsella
1 sibling, 1 reply; 200+ results
From: Thomas Monjalon @ 2020-04-27 12:31 UTC (permalink / raw)
To: ray.kinsella, nhorman, Kevin Laatz, Ray Kinsella
Cc: dev, bruce.richardson, harry.van.haaren, david.marchand,
Haiyue Wang, ktraynor
27/04/2020 11:27, Ray Kinsella:
> On 25/04/2020 17:04, Thomas Monjalon wrote:
> > PS: Who is REALLY maintaining the ABI?
> > We really miss someone who carefully check all these things,
> > and take care of the doc and tooling.
>
> I would say that I am missing these changes to libabigail.ignore, which would be useful.
> Should we consolidate the ABI Policy and ABI Versioning sections of the MAINTAINERS file?
Yes, I think it does not make sense spliting ABI topic in 2 sections
in MAINTAINERS file.
We need to have a clear ownership covering policy, libs, tooling and doc.
Let's agree to merge all in one section please.
^ permalink raw reply [relevance 3%]
* Re: [dpdk-dev] [PATCH v3] mempool: return 0 if area is too small on populate
2020-04-25 22:23 12% ` [dpdk-dev] [PATCH v3] " Thomas Monjalon
2020-04-26 16:52 12% ` [dpdk-dev] [PATCH v4] " Thomas Monjalon
@ 2020-04-27 11:44 0% ` Ray Kinsella
2020-04-27 18:02 0% ` Lukasz Wojciechowski
1 sibling, 1 reply; 200+ results
From: Ray Kinsella @ 2020-04-27 11:44 UTC (permalink / raw)
To: Thomas Monjalon, dev
Cc: david.marchand, Olivier Matz, Neil Horman, John McNamara,
Marko Kovacevic, Xiaoyun Li, Jingjing Wu, Andrew Rybchenko
On 25/04/2020 23:23, Thomas Monjalon wrote:
> From: Olivier Matz <olivier.matz@6wind.com>
>
> Change rte_mempool_populate_iova() and rte_mempool_populate_iova() to
> return 0 instead of -EINVAL when there is not enough room to store one
> object, as it can be helpful for applications to distinguish this
> specific case.
>
> As this is an ABI change, use symbol versioning to preserve old
> behavior for binary applications.
>
> Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
> ---
>
> changes in v3:
> - rebase
> - remove deprecation notice
> - notify API change in release notes
> - fix ABI version from 20.0.1 to 20.0.2 (should be 21 maybe)
yes, should be v21.
>
> This v3 cannot be merged because of a false positive ABI check:
>
> 2 Removed functions:
> 'function int rte_mempool_populate_iova(rte_mempool*, char*, rte_iova_t, size_t, rte_mempool_memchunk_free_cb_t*, void*)' {rte_mempool_populate_iova@@DPDK_20.0}
> 'function int rte_mempool_populate_virt(rte_mempool*, char*, size_t, size_t, rte_mempool_memchunk_free_cb_t*, void*)' {rte_mempool_populate_virt@@DPDK_20.0}
Well it's not really a false positive, as the v20 symbols are now missing.
See notes on VERSION_SYMBOL.
>
> ---
> doc/guides/rel_notes/deprecation.rst | 5 --
> doc/guides/rel_notes/release_20_05.rst | 4 ++
> examples/ntb/ntb_fwd.c | 2 +-
> lib/librte_mempool/meson.build | 2 +
> lib/librte_mempool/rte_mempool.c | 77 ++++++++++++++++++----
> lib/librte_mempool/rte_mempool.h | 14 ++--
> lib/librte_mempool/rte_mempool_version.map | 7 ++
> 7 files changed, 90 insertions(+), 21 deletions(-)
>
> diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
> index 1339f54f5f..20aa745b77 100644
> --- a/doc/guides/rel_notes/deprecation.rst
> +++ b/doc/guides/rel_notes/deprecation.rst
> @@ -65,11 +65,6 @@ Deprecation Notices
> structure would be made internal (or removed if all dependencies are cleared)
> in future releases.
>
> -* mempool: starting from v20.05, the API of rte_mempool_populate_iova()
> - and rte_mempool_populate_virt() will change to return 0 instead
> - of -EINVAL when there is not enough room to store one object. The ABI
> - will be preserved until 20.11.
> -
> * ethdev: the legacy filter API, including
> ``rte_eth_dev_filter_supported()``, ``rte_eth_dev_filter_ctrl()`` as well
> as filter types MACVLAN, ETHERTYPE, FLEXIBLE, SYN, NTUPLE, TUNNEL, FDIR,
> diff --git a/doc/guides/rel_notes/release_20_05.rst b/doc/guides/rel_notes/release_20_05.rst
> index b124c3f287..ab20a7d021 100644
> --- a/doc/guides/rel_notes/release_20_05.rst
> +++ b/doc/guides/rel_notes/release_20_05.rst
> @@ -241,6 +241,10 @@ API Changes
> Also, make sure to start the actual text at the margin.
> =========================================================
>
> +* mempool: The API of ``rte_mempool_populate_iova()`` and
> + ``rte_mempool_populate_virt()`` changed to return 0 instead of -EINVAL
> + when there is not enough room to store one object.
> +
>
> ABI Changes
> -----------
> diff --git a/examples/ntb/ntb_fwd.c b/examples/ntb/ntb_fwd.c
> index d49189e175..eba8ebf9fa 100644
> --- a/examples/ntb/ntb_fwd.c
> +++ b/examples/ntb/ntb_fwd.c
> @@ -1319,7 +1319,7 @@ ntb_mbuf_pool_create(uint16_t mbuf_seg_size, uint32_t nb_mbuf,
> mz->len - ntb_info.ntb_hdr_size,
> ntb_mempool_mz_free,
> (void *)(uintptr_t)mz);
> - if (ret < 0) {
> + if (ret <= 0) {
> rte_memzone_free(mz);
> rte_mempool_free(mp);
> return NULL;
> diff --git a/lib/librte_mempool/meson.build b/lib/librte_mempool/meson.build
> index a6e861cbfc..7dbe6b9bea 100644
> --- a/lib/librte_mempool/meson.build
> +++ b/lib/librte_mempool/meson.build
> @@ -9,6 +9,8 @@ foreach flag: extra_flags
> endif
> endforeach
>
> +use_function_versioning = true
> +
> sources = files('rte_mempool.c', 'rte_mempool_ops.c',
> 'rte_mempool_ops_default.c', 'mempool_trace_points.c')
> headers = files('rte_mempool.h', 'rte_mempool_trace.h',
> diff --git a/lib/librte_mempool/rte_mempool.c b/lib/librte_mempool/rte_mempool.c
> index 0be8f9f59d..edbdafaafb 100644
> --- a/lib/librte_mempool/rte_mempool.c
> +++ b/lib/librte_mempool/rte_mempool.c
> @@ -31,6 +31,7 @@
> #include <rte_string_fns.h>
> #include <rte_spinlock.h>
> #include <rte_tailq.h>
> +#include <rte_function_versioning.h>
>
> #include "rte_mempool.h"
> #include "rte_mempool_trace.h"
> @@ -303,12 +304,17 @@ mempool_ops_alloc_once(struct rte_mempool *mp)
> return 0;
> }
>
> +__vsym int
> +rte_mempool_populate_iova_v20_0_2(struct rte_mempool *mp, char *vaddr,
> + rte_iova_t iova, size_t len, rte_mempool_memchunk_free_cb_t *free_cb,
> + void *opaque);
> +
> /* Add objects in the pool, using a physically contiguous memory
> * zone. Return the number of objects added, or a negative value
> * on error.
> */
> -static int
> -__rte_mempool_populate_iova(struct rte_mempool *mp, char *vaddr,
> +__vsym int
> +rte_mempool_populate_iova_v20_0_2(struct rte_mempool *mp, char *vaddr,
> rte_iova_t iova, size_t len, rte_mempool_memchunk_free_cb_t *free_cb,
> void *opaque)
> {
> @@ -366,14 +372,27 @@ __rte_mempool_populate_iova(struct rte_mempool *mp, char *vaddr,
> return ret;
> }
>
> -int
> -rte_mempool_populate_iova(struct rte_mempool *mp, char *vaddr,
> +BIND_DEFAULT_SYMBOL(rte_mempool_populate_iova, _v20_0_2, 20.0.2);
as discussed v21.
> +MAP_STATIC_SYMBOL(
> + int rte_mempool_populate_iova(struct rte_mempool *mp, char *vaddr,
> + rte_iova_t iova, size_t len,
> + rte_mempool_memchunk_free_cb_t *free_cb,
> + void *opaque),
> + rte_mempool_populate_iova_v20_0_2);
> +
> +__vsym int
> +rte_mempool_populate_iova_v20(struct rte_mempool *mp, char *vaddr,
> + rte_iova_t iova, size_t len, rte_mempool_memchunk_free_cb_t *free_cb,
> + void *opaque);
> +
> +__vsym int
> +rte_mempool_populate_iova_v20(struct rte_mempool *mp, char *vaddr,
> rte_iova_t iova, size_t len, rte_mempool_memchunk_free_cb_t *free_cb,
> void *opaque)
> {
> int ret;
>
> - ret = __rte_mempool_populate_iova(mp, vaddr, iova, len, free_cb,
> + ret = rte_mempool_populate_iova_v20_0_2(mp, vaddr, iova, len, free_cb,
> opaque);
> if (ret == 0)
> ret = -EINVAL;
> @@ -381,6 +400,7 @@ rte_mempool_populate_iova(struct rte_mempool *mp, char *vaddr,
> rte_mempool_trace_populate_iova(mp, vaddr, iova, len, free_cb, opaque);
> return ret;
> }
> +VERSION_SYMBOL(rte_mempool_populate_iova, _v20_0, 20.0);
this should be
VERSION_SYMBOL(rte_mempool_populate_iova, _v20, 20.0);
>
> static rte_iova_t
> get_iova(void *addr)
> @@ -395,11 +415,16 @@ get_iova(void *addr)
> return ms->iova + RTE_PTR_DIFF(addr, ms->addr);
> }
>
> +__vsym int
> +rte_mempool_populate_virt_v20_0_2(struct rte_mempool *mp, char *addr,
> + size_t len, size_t pg_sz, rte_mempool_memchunk_free_cb_t *free_cb,
> + void *opaque);
> +
> /* Populate the mempool with a virtual area. Return the number of
> * objects added, or a negative value on error.
> */
> -int
> -rte_mempool_populate_virt(struct rte_mempool *mp, char *addr,
> +__vsym int
> +rte_mempool_populate_virt_v20_0_2(struct rte_mempool *mp, char *addr,
> size_t len, size_t pg_sz, rte_mempool_memchunk_free_cb_t *free_cb,
> void *opaque)
> {
> @@ -432,7 +457,7 @@ rte_mempool_populate_virt(struct rte_mempool *mp, char *addr,
> break;
> }
>
> - ret = __rte_mempool_populate_iova(mp, addr + off, iova,
> + ret = rte_mempool_populate_iova_v20_0_2(mp, addr + off, iova,
> phys_len, free_cb, opaque);
> if (ret == 0)
> continue;
> @@ -443,9 +468,6 @@ rte_mempool_populate_virt(struct rte_mempool *mp, char *addr,
> cnt += ret;
> }
>
> - if (cnt == 0)
> - return -EINVAL;
> -
> rte_mempool_trace_populate_virt(mp, addr, len, pg_sz, free_cb, opaque);
> return cnt;
>
> @@ -453,6 +475,35 @@ rte_mempool_populate_virt(struct rte_mempool *mp, char *addr,
> rte_mempool_free_memchunks(mp);
> return ret;
> }
> +BIND_DEFAULT_SYMBOL(rte_mempool_populate_virt, _v20_0_2, 20.0.2);
as discussed v21.
> +MAP_STATIC_SYMBOL(
> + int rte_mempool_populate_virt(struct rte_mempool *mp,
> + char *addr, size_t len, size_t pg_sz,
> + rte_mempool_memchunk_free_cb_t *free_cb,
> + void *opaque),
> + rte_mempool_populate_virt_v20_0_2);
> +
> +__vsym int
> +rte_mempool_populate_virt_v20(struct rte_mempool *mp, char *addr,
> + size_t len, size_t pg_sz, rte_mempool_memchunk_free_cb_t *free_cb,
> + void *opaque);
> +
> +__vsym int
> +rte_mempool_populate_virt_v20(struct rte_mempool *mp, char *addr,
> + size_t len, size_t pg_sz, rte_mempool_memchunk_free_cb_t *free_cb,
> + void *opaque)
> +{
> + int ret;
> +
> + ret = rte_mempool_populate_virt_v20_0_2(mp, addr, len, pg_sz,
> + free_cb, opaque);
> +
> + if (ret == 0)
> + ret = -EINVAL;
> +
> + return ret;
> +}
> +VERSION_SYMBOL(rte_mempool_populate_virt, _v20_0, 20.0);
this should be
VERSION_SYMBOL(rte_mempool_populate_virt, _v20, 20.0);
>
> /* Get the minimal page size used in a mempool before populating it. */
> int
> @@ -609,6 +660,8 @@ rte_mempool_populate_default(struct rte_mempool *mp)
> mz->len, pg_sz,
> rte_mempool_memchunk_mz_free,
> (void *)(uintptr_t)mz);
> + if (ret == 0) /* should not happen */
> + ret = -ENOBUFS;
> if (ret < 0) {
> rte_memzone_free(mz);
> goto fail;
> @@ -701,6 +754,8 @@ rte_mempool_populate_anon(struct rte_mempool *mp)
>
> ret = rte_mempool_populate_virt(mp, addr, size, getpagesize(),
> rte_mempool_memchunk_anon_free, addr);
> + if (ret == 0) /* should not happen */
> + ret = -ENOBUFS;
> if (ret < 0) {
> rte_errno = -ret;
> goto fail;
> diff --git a/lib/librte_mempool/rte_mempool.h b/lib/librte_mempool/rte_mempool.h
> index 6e0573ea42..652d19f9f1 100644
> --- a/lib/librte_mempool/rte_mempool.h
> +++ b/lib/librte_mempool/rte_mempool.h
> @@ -1112,9 +1112,12 @@ rte_mempool_free(struct rte_mempool *mp);
> * @param opaque
> * An opaque argument passed to free_cb.
> * @return
> - * The number of objects added on success.
> + * The number of objects added on success (strictly positive).
> * On error, the chunk is not added in the memory list of the
> - * mempool and a negative errno is returned.
> + * mempool the following code is returned:
> + * (0): not enough room in chunk for one object.
> + * (-ENOSPC): mempool is already populated.
> + * (-ENOMEM): allocation failure.
> */
> int rte_mempool_populate_iova(struct rte_mempool *mp, char *vaddr,
> rte_iova_t iova, size_t len, rte_mempool_memchunk_free_cb_t *free_cb,
> @@ -1139,9 +1142,12 @@ int rte_mempool_populate_iova(struct rte_mempool *mp, char *vaddr,
> * @param opaque
> * An opaque argument passed to free_cb.
> * @return
> - * The number of objects added on success.
> + * The number of objects added on success (strictly positive).
> * On error, the chunk is not added in the memory list of the
> - * mempool and a negative errno is returned.
> + * mempool the following code is returned:
> + * (0): not enough room in chunk for one object.
> + * (-ENOSPC): mempool is already populated.
> + * (-ENOMEM): allocation failure.
> */
> int
> rte_mempool_populate_virt(struct rte_mempool *mp, char *addr,
> diff --git a/lib/librte_mempool/rte_mempool_version.map b/lib/librte_mempool/rte_mempool_version.map
> index 695dd6e04f..9e58093665 100644
> --- a/lib/librte_mempool/rte_mempool_version.map
> +++ b/lib/librte_mempool/rte_mempool_version.map
> @@ -31,6 +31,13 @@ DPDK_20.0 {
> local: *;
> };
>
> +DPDK_20.0.2 {
as discussed DPDK_21
> + global:
> +
> + rte_mempool_populate_iova;
> + rte_mempool_populate_virt;
> +} DPDK_20.0;
> +
> EXPERIMENTAL {
> global:
>
>
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH v4] eal/cpuflags: add x86 based cpu flags
2020-04-27 9:35 0% ` Ray Kinsella
@ 2020-04-27 10:08 0% ` Laatz, Kevin
0 siblings, 0 replies; 200+ results
From: Laatz, Kevin @ 2020-04-27 10:08 UTC (permalink / raw)
To: Ray Kinsella, Thomas Monjalon, Kinsella, Ray, nhorman
Cc: dev, Richardson, Bruce, Van Haaren, Harry, david.marchand
> On 27/04/2020 10:31, Laatz, Kevin wrote:
> >
> >> (replying this time to the list)
> >>
> >> On 25/04/2020 17:04, Thomas Monjalon wrote:
> >>> 16/04/2020 13:00, Kevin Laatz:
> >>>> This patch adds CPU flags which will enable the detection of ISA
> >>>> features available on more recent x86 based CPUs.
> >>> [...]
> >>>> --- a/devtools/libabigail.abignore
> >>>> +++ b/devtools/libabigail.abignore
> >>>> +; Ignore this enum update as it should not be allocated by the
> >>>> +application [suppress_type]
> >>>> + type_kind = enum
> >>>> + name = rte_cpu_flag_t
> >>>> + changed_enumerators = RTE_CPUFLAG_NUMFLAGS
> >>>
> >>> The justification is not correct.
> >>> The application is allowed to use RTE_CPUFLAG_NUMFLAGS in array
> >> allocation.
> >>> But no API is returning a CPU flag, so the new flags will remain
> >>> unknown to the application.
> >>>
> >>> However, there is a behaviour change:
> >>> The functions rte_cpu_get_flag_name() and
> rte_cpu_get_flag_enabled()
> >>> will now accept new values, which were previously considered as an
> error.
> >>> Is it an ABI breakage? I would say no.
> >>
> >> We saw something similar with the Cryptodev's
> >> rte_crypto_sym_xform_type also.
> >> Libabigail appears to be particularly sensitive to changes to enumerations.
> >> Leaving it to the user to decide if there is a problem.
> >>
> >> I am seeing a bit of weirdness though between versions of libabigail.
> >> 1.7.1 seems to fine with the change, however 1.2 is reporting an issue.
> >>
> >> Kevin - what version are you using?
> >
> > I'm using version 1.6.0
>
> right you are either on Fedora 31 or some Ubuntu v19.xx, right?
At the time of making the patch: Ubuntu 18.04 with a manually upgraded libabigail
Currently Ubuntu 20.04 (beta).
>
> >
> >>
> >>>
> >>> PS: Who is REALLY maintaining the ABI?
> >>> We really miss someone who carefully check all these things, and
> >>> take care of the doc and tooling.
> >>>
> >>>
> >>
> >> I would say that I am missing these changes to libabigail.ignore,
> >> which would be useful.
> >> Should we consolidate the ABI Policy and ABI Versioning sections of
> >> the MAINTAINERS file?
> >
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH v4] eal/cpuflags: add x86 based cpu flags
2020-04-27 9:31 0% ` Laatz, Kevin
@ 2020-04-27 9:35 0% ` Ray Kinsella
2020-04-27 10:08 0% ` Laatz, Kevin
0 siblings, 1 reply; 200+ results
From: Ray Kinsella @ 2020-04-27 9:35 UTC (permalink / raw)
To: Laatz, Kevin, Thomas Monjalon, Kinsella, Ray, nhorman
Cc: dev, Richardson, Bruce, Van Haaren, Harry, david.marchand
On 27/04/2020 10:31, Laatz, Kevin wrote:
>
>> (replying this time to the list)
>>
>> On 25/04/2020 17:04, Thomas Monjalon wrote:
>>> 16/04/2020 13:00, Kevin Laatz:
>>>> This patch adds CPU flags which will enable the detection of ISA
>>>> features available on more recent x86 based CPUs.
>>> [...]
>>>> --- a/devtools/libabigail.abignore
>>>> +++ b/devtools/libabigail.abignore
>>>> +; Ignore this enum update as it should not be allocated by the
>>>> +application [suppress_type]
>>>> + type_kind = enum
>>>> + name = rte_cpu_flag_t
>>>> + changed_enumerators = RTE_CPUFLAG_NUMFLAGS
>>>
>>> The justification is not correct.
>>> The application is allowed to use RTE_CPUFLAG_NUMFLAGS in array
>> allocation.
>>> But no API is returning a CPU flag, so the new flags will remain
>>> unknown to the application.
>>>
>>> However, there is a behaviour change:
>>> The functions rte_cpu_get_flag_name() and rte_cpu_get_flag_enabled()
>>> will now accept new values, which were previously considered as an error.
>>> Is it an ABI breakage? I would say no.
>>
>> We saw something similar with the Cryptodev's rte_crypto_sym_xform_type
>> also.
>> Libabigail appears to be particularly sensitive to changes to enumerations.
>> Leaving it to the user to decide if there is a problem.
>>
>> I am seeing a bit of weirdness though between versions of libabigail.
>> 1.7.1 seems to fine with the change, however 1.2 is reporting an issue.
>>
>> Kevin - what version are you using?
>
> I'm using version 1.6.0
right you are either on Fedora 31 or some Ubuntu v19.xx, right?
>
>>
>>>
>>> PS: Who is REALLY maintaining the ABI?
>>> We really miss someone who carefully check all these things, and take
>>> care of the doc and tooling.
>>>
>>>
>>
>> I would say that I am missing these changes to libabigail.ignore, which would
>> be useful.
>> Should we consolidate the ABI Policy and ABI Versioning sections of the
>> MAINTAINERS file?
>
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH v4] eal/cpuflags: add x86 based cpu flags
2020-04-27 9:27 4% ` Ray Kinsella
@ 2020-04-27 9:31 0% ` Laatz, Kevin
2020-04-27 9:35 0% ` Ray Kinsella
2020-04-27 12:31 3% ` Thomas Monjalon
1 sibling, 1 reply; 200+ results
From: Laatz, Kevin @ 2020-04-27 9:31 UTC (permalink / raw)
To: Ray Kinsella, Thomas Monjalon, Kinsella, Ray, nhorman
Cc: dev, Richardson, Bruce, Van Haaren, Harry, david.marchand
> (replying this time to the list)
>
> On 25/04/2020 17:04, Thomas Monjalon wrote:
> > 16/04/2020 13:00, Kevin Laatz:
> >> This patch adds CPU flags which will enable the detection of ISA
> >> features available on more recent x86 based CPUs.
> > [...]
> >> --- a/devtools/libabigail.abignore
> >> +++ b/devtools/libabigail.abignore
> >> +; Ignore this enum update as it should not be allocated by the
> >> +application [suppress_type]
> >> + type_kind = enum
> >> + name = rte_cpu_flag_t
> >> + changed_enumerators = RTE_CPUFLAG_NUMFLAGS
> >
> > The justification is not correct.
> > The application is allowed to use RTE_CPUFLAG_NUMFLAGS in array
> allocation.
> > But no API is returning a CPU flag, so the new flags will remain
> > unknown to the application.
> >
> > However, there is a behaviour change:
> > The functions rte_cpu_get_flag_name() and rte_cpu_get_flag_enabled()
> > will now accept new values, which were previously considered as an error.
> > Is it an ABI breakage? I would say no.
>
> We saw something similar with the Cryptodev's rte_crypto_sym_xform_type
> also.
> Libabigail appears to be particularly sensitive to changes to enumerations.
> Leaving it to the user to decide if there is a problem.
>
> I am seeing a bit of weirdness though between versions of libabigail.
> 1.7.1 seems to fine with the change, however 1.2 is reporting an issue.
>
> Kevin - what version are you using?
I'm using version 1.6.0
>
> >
> > PS: Who is REALLY maintaining the ABI?
> > We really miss someone who carefully check all these things, and take
> > care of the doc and tooling.
> >
> >
>
> I would say that I am missing these changes to libabigail.ignore, which would
> be useful.
> Should we consolidate the ABI Policy and ABI Versioning sections of the
> MAINTAINERS file?
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH v4] eal/cpuflags: add x86 based cpu flags
2020-04-25 16:04 4% ` Thomas Monjalon
2020-04-27 9:22 4% ` Kinsella, Ray
@ 2020-04-27 9:27 4% ` Ray Kinsella
2020-04-27 9:31 0% ` Laatz, Kevin
2020-04-27 12:31 3% ` Thomas Monjalon
1 sibling, 2 replies; 200+ results
From: Ray Kinsella @ 2020-04-27 9:27 UTC (permalink / raw)
To: Thomas Monjalon, ray.kinsella, nhorman, Kevin Laatz
Cc: dev, bruce.richardson, harry.van.haaren, david.marchand
(replying this time to the list)
On 25/04/2020 17:04, Thomas Monjalon wrote:
> 16/04/2020 13:00, Kevin Laatz:
>> This patch adds CPU flags which will enable the detection of ISA
>> features available on more recent x86 based CPUs.
> [...]
>> --- a/devtools/libabigail.abignore
>> +++ b/devtools/libabigail.abignore
>> +; Ignore this enum update as it should not be allocated by the application
>> +[suppress_type]
>> + type_kind = enum
>> + name = rte_cpu_flag_t
>> + changed_enumerators = RTE_CPUFLAG_NUMFLAGS
>
> The justification is not correct.
> The application is allowed to use RTE_CPUFLAG_NUMFLAGS in array allocation.
> But no API is returning a CPU flag, so the new flags will remain unknown
> to the application.
>
> However, there is a behaviour change:
> The functions rte_cpu_get_flag_name() and rte_cpu_get_flag_enabled()
> will now accept new values, which were previously considered as an error.
> Is it an ABI breakage? I would say no.
We saw something similar with the Cryptodev's rte_crypto_sym_xform_type also.
Libabigail appears to be particularly sensitive to changes to enumerations.
Leaving it to the user to decide if there is a problem.
I am seeing a bit of weirdness though between versions of libabigail.
1.7.1 seems to fine with the change, however 1.2 is reporting an issue.
Kevin - what version are you using?
>
> PS: Who is REALLY maintaining the ABI?
> We really miss someone who carefully check all these things,
> and take care of the doc and tooling.
>
>
I would say that I am missing these changes to libabigail.ignore, which would be useful.
Should we consolidate the ABI Policy and ABI Versioning sections of the MAINTAINERS file?
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH v4 1/4] ethdev: add tm support for shaper config in pkt mode
2020-04-25 20:09 3% ` Ferruh Yigit
@ 2020-04-27 9:19 4% ` Dumitrescu, Cristian
2020-04-27 16:12 4% ` Ferruh Yigit
0 siblings, 1 reply; 200+ results
From: Dumitrescu, Cristian @ 2020-04-27 9:19 UTC (permalink / raw)
To: Yigit, Ferruh, Nithin Dabilpuram, Singh, Jasvinder,
Thomas Monjalon, Andrew Rybchenko
Cc: dev, jerinj, kkanas, Nithin Dabilpuram
> -----Original Message-----
> From: Yigit, Ferruh <ferruh.yigit@intel.com>
> Sent: Saturday, April 25, 2020 9:09 PM
> To: Dumitrescu, Cristian <cristian.dumitrescu@intel.com>; Nithin Dabilpuram
> <nithind1988@gmail.com>; Singh, Jasvinder <jasvinder.singh@intel.com>;
> Thomas Monjalon <thomas@monjalon.net>; Andrew Rybchenko
> <arybchenko@solarflare.com>
> Cc: dev@dpdk.org; jerinj@marvell.com; kkanas@marvell.com; Nithin
> Dabilpuram <ndabilpuram@marvell.com>
> Subject: Re: [PATCH v4 1/4] ethdev: add tm support for shaper config in pkt
> mode
>
> On 4/24/2020 11:28 AM, Dumitrescu, Cristian wrote:
> >
> >
> >> -----Original Message-----
> >> From: Nithin Dabilpuram <nithind1988@gmail.com>
> >> Sent: Wednesday, April 22, 2020 6:21 PM
> >> To: Singh, Jasvinder <jasvinder.singh@intel.com>; Dumitrescu, Cristian
> >> <cristian.dumitrescu@intel.com>; Thomas Monjalon
> >> <thomas@monjalon.net>; Yigit, Ferruh <ferruh.yigit@intel.com>; Andrew
> >> Rybchenko <arybchenko@solarflare.com>
> >> Cc: dev@dpdk.org; jerinj@marvell.com; kkanas@marvell.com; Nithin
> >> Dabilpuram <ndabilpuram@marvell.com>
> >> Subject: [PATCH v4 1/4] ethdev: add tm support for shaper config in pkt
> >> mode
> >>
> >> From: Nithin Dabilpuram <ndabilpuram@marvell.com>
> >>
> >> Some NIC hardware support shaper to work in packet mode i.e
> >> shaping or ratelimiting traffic is in packets per second (PPS) as
> >> opposed to default bytes per second (BPS). Hence this patch
> >> adds support to configure shared or private shaper in packet mode,
> >> provide rate in PPS and add related tm capabilities in port/level/node
> >> capability structures.
> >>
> >> This patch also updates tm port/level/node capability structures with
> >> exiting features of scheduler wfq packet mode, scheduler wfq byte mode
> >> and private/shared shaper byte mode.
> >>
> >> SoftNIC PMD is also updated with new capabilities.
> >>
> >> Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>
> >> ---
> >> v3..v4:
> >> - Update text under packet_mode as per Cristian.
> >> - Update rte_eth_softnic_tm.c based on Jasvinder's comments.
> >> - Add error enum
> RTE_TM_ERROR_TYPE_SHAPER_PROFILE_PACKET_MODE
> >> - Fix shaper_profile_check() with packet mode check
> >> - Fix typo's
> >>
> >
> > Acked-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
> >
>
> Hi Nithin,
>
> It looks like patch is causing ABI break, I am getting following warning [1],
> can you please check?
>
> [1]
> https://pastebin.com/XYNFg14u
Hi Ferruh,
The RTE_TM API is marked as experimental, but it looks that this was not correctly marked when __rte_experimental ABI checker was introduced.
It is marked as experimental at the top of the rte_tm.h, similarly to other APIs introduced around same time, but it was not correctly picked up by the ABI check procedure when later introduced, so __rte_experimental was not added to every function.
Regards,
Cristian
^ permalink raw reply [relevance 4%]
* [dpdk-dev] [PATCH v3] abi: change references to abi 20.0.1 to abi v21
2020-04-27 9:06 11% ` [dpdk-dev] [PATCH v3] " Ray Kinsella
@ 2020-04-27 9:06 16% ` Ray Kinsella
0 siblings, 0 replies; 200+ results
From: Ray Kinsella @ 2020-04-27 9:06 UTC (permalink / raw)
To: dev
Cc: aostruszka, stephen, david.marchand, Ray Kinsella,
Thomas Monjalon, Jingjing Wu, Wenzhuo Lu, Matan Azrad,
Shahaf Shuler, Viacheslav Ovsiienko, Jerin Jacob,
Nithin Dabilpuram, Alfredo Cardigliano, Mahipal Challa,
Cristian Dumitrescu
Change references to abi 20.0.1 to use abi v21, see
https://doc.dpdk.org/guides/contributing/abi_policy.html#general-guidelines.
"Major ABI versions are declared no more frequently than yearly.
Compatibility with the major ABI version is mandatory in subsequent
releases until a new major ABI version is declared."
Add myself as the map file maintainer to more closely monitor future abi
changes.
Signed-off-by: Ray Kinsella <mdr@ashroe.eu>
---
MAINTAINERS | 2 ++
drivers/common/iavf/rte_common_iavf_version.map | 2 +-
drivers/common/mlx5/rte_common_mlx5_version.map | 2 +-
drivers/common/octeontx2/rte_common_octeontx2_version.map | 2 +-
drivers/net/ionic/rte_pmd_ionic_version.map | 2 +-
drivers/raw/octeontx2_ep/rte_rawdev_octeontx2_ep_version.map | 2 +-
drivers/vdpa/mlx5/rte_pmd_mlx5_vdpa_version.map | 2 +-
lib/librte_meter/rte_meter_version.map | 2 +-
8 files changed, 9 insertions(+), 7 deletions(-)
diff --git a/MAINTAINERS b/MAINTAINERS
index d31a809..7d521f1 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -86,6 +86,8 @@ F: doc/
ABI Policy
M: Ray Kinsella <mdr@ashroe.eu>
F: doc/guides/contributing/abi_*.rst
+F: drivers/*/*/*.map
+F: lib/*/*.map
Developers and Maintainers Tools
M: Thomas Monjalon <thomas@monjalon.net>
diff --git a/drivers/common/iavf/rte_common_iavf_version.map b/drivers/common/iavf/rte_common_iavf_version.map
index 2f11d67..92ceac1 100644
--- a/drivers/common/iavf/rte_common_iavf_version.map
+++ b/drivers/common/iavf/rte_common_iavf_version.map
@@ -1,4 +1,4 @@
-DPDK_20.0.1 {
+DPDK_21 {
global:
iavf_init_adminq;
diff --git a/drivers/common/mlx5/rte_common_mlx5_version.map b/drivers/common/mlx5/rte_common_mlx5_version.map
index b58a378..564a9a7 100644
--- a/drivers/common/mlx5/rte_common_mlx5_version.map
+++ b/drivers/common/mlx5/rte_common_mlx5_version.map
@@ -1,4 +1,4 @@
-DPDK_20.0.1 {
+DPDK_21 {
global:
mlx5_class_get;
diff --git a/drivers/common/octeontx2/rte_common_octeontx2_version.map b/drivers/common/octeontx2/rte_common_octeontx2_version.map
index 8f2404b..01279c3 100644
--- a/drivers/common/octeontx2/rte_common_octeontx2_version.map
+++ b/drivers/common/octeontx2/rte_common_octeontx2_version.map
@@ -34,7 +34,7 @@ DPDK_20.0 {
local: *;
};
-DPDK_20.0.1 {
+DPDK_21 {
global:
otx2_eth_dev_is_sec_capable;
diff --git a/drivers/net/ionic/rte_pmd_ionic_version.map b/drivers/net/ionic/rte_pmd_ionic_version.map
index bc8fd6d..acdaf58 100644
--- a/drivers/net/ionic/rte_pmd_ionic_version.map
+++ b/drivers/net/ionic/rte_pmd_ionic_version.map
@@ -1,4 +1,4 @@
-DPDK_20.0.1 {
+DPDK_21 {
local: *;
};
diff --git a/drivers/raw/octeontx2_ep/rte_rawdev_octeontx2_ep_version.map b/drivers/raw/octeontx2_ep/rte_rawdev_octeontx2_ep_version.map
index bc8fd6d..acdaf58 100644
--- a/drivers/raw/octeontx2_ep/rte_rawdev_octeontx2_ep_version.map
+++ b/drivers/raw/octeontx2_ep/rte_rawdev_octeontx2_ep_version.map
@@ -1,4 +1,4 @@
-DPDK_20.0.1 {
+DPDK_21 {
local: *;
};
diff --git a/drivers/vdpa/mlx5/rte_pmd_mlx5_vdpa_version.map b/drivers/vdpa/mlx5/rte_pmd_mlx5_vdpa_version.map
index 179f7f1..4a76d1d 100644
--- a/drivers/vdpa/mlx5/rte_pmd_mlx5_vdpa_version.map
+++ b/drivers/vdpa/mlx5/rte_pmd_mlx5_vdpa_version.map
@@ -1,3 +1,3 @@
-DPDK_20.0.1 {
+DPDK_21 {
local: *;
};
diff --git a/lib/librte_meter/rte_meter_version.map b/lib/librte_meter/rte_meter_version.map
index fc12cc0..2c7dadb 100644
--- a/lib/librte_meter/rte_meter_version.map
+++ b/lib/librte_meter/rte_meter_version.map
@@ -13,7 +13,7 @@ DPDK_20.0 {
local: *;
};
-DPDK_20.0.1 {
+DPDK_21 {
global:
rte_meter_trtcm_rfc4115_color_aware_check;
--
2.7.4
^ permalink raw reply [relevance 16%]
* [dpdk-dev] [PATCH v3] abi: change references to abi 20.0.1 to abi v21
2020-04-23 6:41 21% ` [dpdk-dev] [PATCH v2] " Ray Kinsella
@ 2020-04-27 9:06 11% ` Ray Kinsella
2020-04-27 9:06 16% ` Ray Kinsella
2020-04-27 13:45 11% ` [dpdk-dev] [PATCH v4] " Ray Kinsella
` (2 subsequent siblings)
5 siblings, 1 reply; 200+ results
From: Ray Kinsella @ 2020-04-27 9:06 UTC (permalink / raw)
To: dev
Cc: aostruszka, stephen, david.marchand, Ray Kinsella,
Thomas Monjalon, Jingjing Wu, Wenzhuo Lu, Matan Azrad,
Shahaf Shuler, Viacheslav Ovsiienko, Jerin Jacob,
Nithin Dabilpuram, Alfredo Cardigliano, Mahipal Challa,
Cristian Dumitrescu
v3:
* Removed libabigail supressions in previous version.
* Clarified abi policy for only major abi versions in git commit.
Ray Kinsella (1):
abi: change references to abi 20.0.1 to abi v21
MAINTAINERS | 2 ++
drivers/common/iavf/rte_common_iavf_version.map | 2 +-
drivers/common/mlx5/rte_common_mlx5_version.map | 2 +-
drivers/common/octeontx2/rte_common_octeontx2_version.map | 2 +-
drivers/net/ionic/rte_pmd_ionic_version.map | 2 +-
drivers/raw/octeontx2_ep/rte_rawdev_octeontx2_ep_version.map | 2 +-
drivers/vdpa/mlx5/rte_pmd_mlx5_vdpa_version.map | 2 +-
lib/librte_meter/rte_meter_version.map | 2 +-
8 files changed, 9 insertions(+), 7 deletions(-)
--
2.7.4
^ permalink raw reply [relevance 11%]
* Re: [dpdk-dev] [PATCH v3] lib/timer: relax barrier for status update
2020-04-26 14:20 0% ` Phil Yang
@ 2020-04-26 19:30 0% ` Carrillo, Erik G
0 siblings, 0 replies; 200+ results
From: Carrillo, Erik G @ 2020-04-26 19:30 UTC (permalink / raw)
To: Phil Yang, thomas
Cc: rsanford, dev, david.marchand, Honnappa Nagarahalli, Gavin Hu,
nd, nd, nd
> -----Original Message-----
> From: Phil Yang <Phil.Yang@arm.com>
> Sent: Sunday, April 26, 2020 9:20 AM
> To: Carrillo, Erik G <erik.g.carrillo@intel.com>; thomas@monjalon.net
> Cc: rsanford@akamai.com; dev@dpdk.org; david.marchand@redhat.com;
> Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>; Gavin Hu
> <Gavin.Hu@arm.com>; nd <nd@arm.com>; nd <nd@arm.com>; nd
> <nd@arm.com>
> Subject: RE: [dpdk-dev] [PATCH v3] lib/timer: relax barrier for status update
>
> > -----Original Message-----
> > From: Carrillo, Erik G <erik.g.carrillo@intel.com>
> > Sent: Sunday, April 26, 2020 8:19 PM
> > To: Phil Yang <Phil.Yang@arm.com>; thomas@monjalon.net
> > Cc: rsanford@akamai.com; dev@dpdk.org; david.marchand@redhat.com;
> > Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>; Gavin Hu
> > <Gavin.Hu@arm.com>; nd <nd@arm.com>; nd <nd@arm.com>
> > Subject: RE: [dpdk-dev] [PATCH v3] lib/timer: relax barrier for status
> > update
> >
> >
> >
> > > -----Original Message-----
> > > From: Phil Yang <Phil.Yang@arm.com>
> > > Sent: Sunday, April 26, 2020 2:36 AM
> > > To: thomas@monjalon.net
> > > Cc: Carrillo, Erik G <erik.g.carrillo@intel.com>;
> > > rsanford@akamai.com; dev@dpdk.org; david.marchand@redhat.com;
> > > Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>; Gavin Hu
> > > <Gavin.Hu@arm.com>; nd <nd@arm.com>; nd <nd@arm.com>
> > > Subject: RE: [dpdk-dev] [PATCH v3] lib/timer: relax barrier for
> > > status
> > update
> > >
> > > > -----Original Message-----
> > > > From: Thomas Monjalon <thomas@monjalon.net>
> > > > Sent: Sunday, April 26, 2020 1:18 AM
> > > > To: Phil Yang <Phil.Yang@arm.com>
> > > > Cc: erik.g.carrillo@intel.com; rsanford@akamai.com; dev@dpdk.org;
> > > > david.marchand@redhat.com; Honnappa Nagarahalli
> > > > <Honnappa.Nagarahalli@arm.com>; Gavin Hu <Gavin.Hu@arm.com>;
> nd
> > > > <nd@arm.com>
> > > > Subject: Re: [dpdk-dev] [PATCH v3] lib/timer: relax barrier for
> > > > status update
> > > >
> > > > 24/04/2020 09:24, Phil Yang:
> > > > > Volatile has no ordering semantics. The rte_timer structure
> > > > > defines timer status as a volatile variable and uses the
> > > > > rte_r/wmb barrier to guarantee inter-thread visibility.
> > > > >
> > > > > This patch optimized the volatile operation with c11 atomic
> > > > > operations and one-way barrier to save the performance penalty.
> > > > > According to the timer_perf_autotest benchmarking results, this
> > > > > patch can uplift 10%~16% timer appending performance, 3%~20%
> > > > > timer resetting performance and
> > > > 45%
> > > > > timer callbacks scheduling performance on aarch64 and no loss in
> > > > > performance for x86.
> > > > >
> > > > > Suggested-by: Honnappa Nagarahalli
> > > > > <honnappa.nagarahalli@arm.com>
> > > > > Signed-off-by: Phil Yang <phil.yang@arm.com>
> > > > > Reviewed-by: Gavin Hu <gavin.hu@arm.com>
> > > > > Acked-by: Erik Gabriel Carrillo <erik.g.carrillo@intel.com>
> > > > [...]
> > > > > --- a/lib/librte_timer/rte_timer.h
> > > > > +++ b/lib/librte_timer/rte_timer.h
> > > > > @@ -101,7 +101,7 @@ struct rte_timer
> > > > > - volatile union rte_timer_status status; /**< Status of timer.
> */
> > > > > + union rte_timer_status status; /**< Status of timer. */
> > > >
> > > > Unfortunately, I cannot merge this patch because it breaks the ABI:
> > > >
> > > > [C]'function void rte_timer_init(rte_timer*)' at
> > > > rte_timer.c:214:1 has some indirect sub-type changes:
> > > > parameter 1 of type 'rte_timer*' has sub-type changes:
> > > > in pointed to type 'struct rte_timer' at rte_timer.h:100:1:
> > > > type size hasn't changed
> > > > 1 data member changes (2 filtered):
> > > > type of 'volatile rte_timer_status rte_timer::status' changed:
> > > > entity changed from 'volatile rte_timer_status' to
> > > > 'union rte_timer_status' at rte_timer.h:67:1
> > > > type size hasn't changed
> > > >
> > >
> > > I think we can revert it to the original definition of rte_timer and
> > > keep the union rte_timer_status volatile-qualified.
> > > Because with or without this 'volatile' qualify, it generates the
> > > same code
> > on
> > > aarch64 and x86.
> > > So it seems acceptable to ignore it to make the ABI compatible?
> > >
> > > Thank,
> > > Phil
> >
> > I was wondering about this also. Is the performance improvement on
> > aarch64 still the same in that case?
>
> Yes. it is.
> It got the same performance improvement on aarch64 and no performance
> loss on x86.
>
> I will update it in v4.
Great - thanks, Phil.
^ permalink raw reply [relevance 0%]
* [dpdk-dev] [PATCH v4] mempool: return 0 if area is too small on populate
2020-04-25 22:23 12% ` [dpdk-dev] [PATCH v3] " Thomas Monjalon
@ 2020-04-26 16:52 12% ` Thomas Monjalon
2020-05-04 12:49 11% ` [dpdk-dev] [PATCH v5] " Olivier Matz
2020-04-27 11:44 0% ` [dpdk-dev] [PATCH v3] " Ray Kinsella
1 sibling, 1 reply; 200+ results
From: Thomas Monjalon @ 2020-04-26 16:52 UTC (permalink / raw)
To: dev
Cc: david.marchand, Ray Kinsella, Olivier Matz, Neil Horman,
John McNamara, Marko Kovacevic, Xiaoyun Li, Jingjing Wu,
Andrew Rybchenko
From: Olivier Matz <olivier.matz@6wind.com>
Change rte_mempool_populate_iova() and rte_mempool_populate_iova() to
return 0 instead of -EINVAL when there is not enough room to store one
object, as it can be helpful for applications to distinguish this
specific case.
As this is an ABI change, use symbol versioning to preserve old
behavior for binary applications.
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
We are missing the symbols versioned for the previous ABI:
rte_mempool_populate_iova@@DPDK_20.0
rte_mempool_populate_virt@@DPDK_20.0
changes in v4:
- move populate_iova trace in v20_0_2 base function
changes in v3:
- rebase
- remove deprecation notice
- notify API change in release notes
- fix ABI version from 20.0.1 to 20.0.2 (should be 21 maybe)
---
doc/guides/rel_notes/deprecation.rst | 5 --
doc/guides/rel_notes/release_20_05.rst | 4 ++
examples/ntb/ntb_fwd.c | 2 +-
lib/librte_mempool/meson.build | 2 +
lib/librte_mempool/rte_mempool.c | 80 ++++++++++++++++++----
lib/librte_mempool/rte_mempool.h | 14 ++--
lib/librte_mempool/rte_mempool_version.map | 7 ++
7 files changed, 92 insertions(+), 22 deletions(-)
diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
index 1339f54f5f..20aa745b77 100644
--- a/doc/guides/rel_notes/deprecation.rst
+++ b/doc/guides/rel_notes/deprecation.rst
@@ -65,11 +65,6 @@ Deprecation Notices
structure would be made internal (or removed if all dependencies are cleared)
in future releases.
-* mempool: starting from v20.05, the API of rte_mempool_populate_iova()
- and rte_mempool_populate_virt() will change to return 0 instead
- of -EINVAL when there is not enough room to store one object. The ABI
- will be preserved until 20.11.
-
* ethdev: the legacy filter API, including
``rte_eth_dev_filter_supported()``, ``rte_eth_dev_filter_ctrl()`` as well
as filter types MACVLAN, ETHERTYPE, FLEXIBLE, SYN, NTUPLE, TUNNEL, FDIR,
diff --git a/doc/guides/rel_notes/release_20_05.rst b/doc/guides/rel_notes/release_20_05.rst
index b124c3f287..ab20a7d021 100644
--- a/doc/guides/rel_notes/release_20_05.rst
+++ b/doc/guides/rel_notes/release_20_05.rst
@@ -241,6 +241,10 @@ API Changes
Also, make sure to start the actual text at the margin.
=========================================================
+* mempool: The API of ``rte_mempool_populate_iova()`` and
+ ``rte_mempool_populate_virt()`` changed to return 0 instead of -EINVAL
+ when there is not enough room to store one object.
+
ABI Changes
-----------
diff --git a/examples/ntb/ntb_fwd.c b/examples/ntb/ntb_fwd.c
index d49189e175..eba8ebf9fa 100644
--- a/examples/ntb/ntb_fwd.c
+++ b/examples/ntb/ntb_fwd.c
@@ -1319,7 +1319,7 @@ ntb_mbuf_pool_create(uint16_t mbuf_seg_size, uint32_t nb_mbuf,
mz->len - ntb_info.ntb_hdr_size,
ntb_mempool_mz_free,
(void *)(uintptr_t)mz);
- if (ret < 0) {
+ if (ret <= 0) {
rte_memzone_free(mz);
rte_mempool_free(mp);
return NULL;
diff --git a/lib/librte_mempool/meson.build b/lib/librte_mempool/meson.build
index a6e861cbfc..7dbe6b9bea 100644
--- a/lib/librte_mempool/meson.build
+++ b/lib/librte_mempool/meson.build
@@ -9,6 +9,8 @@ foreach flag: extra_flags
endif
endforeach
+use_function_versioning = true
+
sources = files('rte_mempool.c', 'rte_mempool_ops.c',
'rte_mempool_ops_default.c', 'mempool_trace_points.c')
headers = files('rte_mempool.h', 'rte_mempool_trace.h',
diff --git a/lib/librte_mempool/rte_mempool.c b/lib/librte_mempool/rte_mempool.c
index 0be8f9f59d..0a6119d6ad 100644
--- a/lib/librte_mempool/rte_mempool.c
+++ b/lib/librte_mempool/rte_mempool.c
@@ -31,6 +31,7 @@
#include <rte_string_fns.h>
#include <rte_spinlock.h>
#include <rte_tailq.h>
+#include <rte_function_versioning.h>
#include "rte_mempool.h"
#include "rte_mempool_trace.h"
@@ -303,12 +304,17 @@ mempool_ops_alloc_once(struct rte_mempool *mp)
return 0;
}
+__vsym int
+rte_mempool_populate_iova_v20_0_2(struct rte_mempool *mp, char *vaddr,
+ rte_iova_t iova, size_t len, rte_mempool_memchunk_free_cb_t *free_cb,
+ void *opaque);
+
/* Add objects in the pool, using a physically contiguous memory
* zone. Return the number of objects added, or a negative value
* on error.
*/
-static int
-__rte_mempool_populate_iova(struct rte_mempool *mp, char *vaddr,
+__vsym int
+rte_mempool_populate_iova_v20_0_2(struct rte_mempool *mp, char *vaddr,
rte_iova_t iova, size_t len, rte_mempool_memchunk_free_cb_t *free_cb,
void *opaque)
{
@@ -359,6 +365,8 @@ __rte_mempool_populate_iova(struct rte_mempool *mp, char *vaddr,
STAILQ_INSERT_TAIL(&mp->mem_list, memhdr, next);
mp->nb_mem_chunks++;
+
+ rte_mempool_trace_populate_iova(mp, vaddr, iova, len, free_cb, opaque);
return i;
fail:
@@ -366,21 +374,34 @@ __rte_mempool_populate_iova(struct rte_mempool *mp, char *vaddr,
return ret;
}
-int
-rte_mempool_populate_iova(struct rte_mempool *mp, char *vaddr,
+BIND_DEFAULT_SYMBOL(rte_mempool_populate_iova, _v20_0_2, 20.0.2);
+MAP_STATIC_SYMBOL(
+ int rte_mempool_populate_iova(struct rte_mempool *mp, char *vaddr,
+ rte_iova_t iova, size_t len,
+ rte_mempool_memchunk_free_cb_t *free_cb,
+ void *opaque),
+ rte_mempool_populate_iova_v20_0_2);
+
+__vsym int
+rte_mempool_populate_iova_v20(struct rte_mempool *mp, char *vaddr,
+ rte_iova_t iova, size_t len, rte_mempool_memchunk_free_cb_t *free_cb,
+ void *opaque);
+
+__vsym int
+rte_mempool_populate_iova_v20(struct rte_mempool *mp, char *vaddr,
rte_iova_t iova, size_t len, rte_mempool_memchunk_free_cb_t *free_cb,
void *opaque)
{
int ret;
- ret = __rte_mempool_populate_iova(mp, vaddr, iova, len, free_cb,
+ ret = rte_mempool_populate_iova_v20_0_2(mp, vaddr, iova, len, free_cb,
opaque);
if (ret == 0)
ret = -EINVAL;
- rte_mempool_trace_populate_iova(mp, vaddr, iova, len, free_cb, opaque);
return ret;
}
+VERSION_SYMBOL(rte_mempool_populate_iova, _v20_0, 20.0);
static rte_iova_t
get_iova(void *addr)
@@ -395,11 +416,16 @@ get_iova(void *addr)
return ms->iova + RTE_PTR_DIFF(addr, ms->addr);
}
+__vsym int
+rte_mempool_populate_virt_v20_0_2(struct rte_mempool *mp, char *addr,
+ size_t len, size_t pg_sz, rte_mempool_memchunk_free_cb_t *free_cb,
+ void *opaque);
+
/* Populate the mempool with a virtual area. Return the number of
* objects added, or a negative value on error.
*/
-int
-rte_mempool_populate_virt(struct rte_mempool *mp, char *addr,
+__vsym int
+rte_mempool_populate_virt_v20_0_2(struct rte_mempool *mp, char *addr,
size_t len, size_t pg_sz, rte_mempool_memchunk_free_cb_t *free_cb,
void *opaque)
{
@@ -432,7 +458,7 @@ rte_mempool_populate_virt(struct rte_mempool *mp, char *addr,
break;
}
- ret = __rte_mempool_populate_iova(mp, addr + off, iova,
+ ret = rte_mempool_populate_iova_v20_0_2(mp, addr + off, iova,
phys_len, free_cb, opaque);
if (ret == 0)
continue;
@@ -443,9 +469,6 @@ rte_mempool_populate_virt(struct rte_mempool *mp, char *addr,
cnt += ret;
}
- if (cnt == 0)
- return -EINVAL;
-
rte_mempool_trace_populate_virt(mp, addr, len, pg_sz, free_cb, opaque);
return cnt;
@@ -453,6 +476,35 @@ rte_mempool_populate_virt(struct rte_mempool *mp, char *addr,
rte_mempool_free_memchunks(mp);
return ret;
}
+BIND_DEFAULT_SYMBOL(rte_mempool_populate_virt, _v20_0_2, 20.0.2);
+MAP_STATIC_SYMBOL(
+ int rte_mempool_populate_virt(struct rte_mempool *mp,
+ char *addr, size_t len, size_t pg_sz,
+ rte_mempool_memchunk_free_cb_t *free_cb,
+ void *opaque),
+ rte_mempool_populate_virt_v20_0_2);
+
+__vsym int
+rte_mempool_populate_virt_v20(struct rte_mempool *mp, char *addr,
+ size_t len, size_t pg_sz, rte_mempool_memchunk_free_cb_t *free_cb,
+ void *opaque);
+
+__vsym int
+rte_mempool_populate_virt_v20(struct rte_mempool *mp, char *addr,
+ size_t len, size_t pg_sz, rte_mempool_memchunk_free_cb_t *free_cb,
+ void *opaque)
+{
+ int ret;
+
+ ret = rte_mempool_populate_virt_v20_0_2(mp, addr, len, pg_sz,
+ free_cb, opaque);
+
+ if (ret == 0)
+ ret = -EINVAL;
+
+ return ret;
+}
+VERSION_SYMBOL(rte_mempool_populate_virt, _v20_0, 20.0);
/* Get the minimal page size used in a mempool before populating it. */
int
@@ -609,6 +661,8 @@ rte_mempool_populate_default(struct rte_mempool *mp)
mz->len, pg_sz,
rte_mempool_memchunk_mz_free,
(void *)(uintptr_t)mz);
+ if (ret == 0) /* should not happen */
+ ret = -ENOBUFS;
if (ret < 0) {
rte_memzone_free(mz);
goto fail;
@@ -701,6 +755,8 @@ rte_mempool_populate_anon(struct rte_mempool *mp)
ret = rte_mempool_populate_virt(mp, addr, size, getpagesize(),
rte_mempool_memchunk_anon_free, addr);
+ if (ret == 0) /* should not happen */
+ ret = -ENOBUFS;
if (ret < 0) {
rte_errno = -ret;
goto fail;
diff --git a/lib/librte_mempool/rte_mempool.h b/lib/librte_mempool/rte_mempool.h
index 6e0573ea42..652d19f9f1 100644
--- a/lib/librte_mempool/rte_mempool.h
+++ b/lib/librte_mempool/rte_mempool.h
@@ -1112,9 +1112,12 @@ rte_mempool_free(struct rte_mempool *mp);
* @param opaque
* An opaque argument passed to free_cb.
* @return
- * The number of objects added on success.
+ * The number of objects added on success (strictly positive).
* On error, the chunk is not added in the memory list of the
- * mempool and a negative errno is returned.
+ * mempool the following code is returned:
+ * (0): not enough room in chunk for one object.
+ * (-ENOSPC): mempool is already populated.
+ * (-ENOMEM): allocation failure.
*/
int rte_mempool_populate_iova(struct rte_mempool *mp, char *vaddr,
rte_iova_t iova, size_t len, rte_mempool_memchunk_free_cb_t *free_cb,
@@ -1139,9 +1142,12 @@ int rte_mempool_populate_iova(struct rte_mempool *mp, char *vaddr,
* @param opaque
* An opaque argument passed to free_cb.
* @return
- * The number of objects added on success.
+ * The number of objects added on success (strictly positive).
* On error, the chunk is not added in the memory list of the
- * mempool and a negative errno is returned.
+ * mempool the following code is returned:
+ * (0): not enough room in chunk for one object.
+ * (-ENOSPC): mempool is already populated.
+ * (-ENOMEM): allocation failure.
*/
int
rte_mempool_populate_virt(struct rte_mempool *mp, char *addr,
diff --git a/lib/librte_mempool/rte_mempool_version.map b/lib/librte_mempool/rte_mempool_version.map
index 695dd6e04f..9e58093665 100644
--- a/lib/librte_mempool/rte_mempool_version.map
+++ b/lib/librte_mempool/rte_mempool_version.map
@@ -31,6 +31,13 @@ DPDK_20.0 {
local: *;
};
+DPDK_20.0.2 {
+ global:
+
+ rte_mempool_populate_iova;
+ rte_mempool_populate_virt;
+} DPDK_20.0;
+
EXPERIMENTAL {
global:
--
2.26.0
^ permalink raw reply [relevance 12%]
* [dpdk-dev] [PATCH v4] lib/timer: relax barrier for status update
2020-04-25 17:17 3% ` Thomas Monjalon
@ 2020-04-26 14:45 2% ` Phil Yang
1 sibling, 0 replies; 200+ results
From: Phil Yang @ 2020-04-26 14:45 UTC (permalink / raw)
To: erik.g.carrillo, thomas, dev
Cc: david.marchand, rsanford, Honnappa.Nagarahalli, Gavin.Hu, nd
Volatile has no ordering semantics. The rte_timer structure defines
timer status as a volatile variable and uses the rte_r/wmb barrier
to guarantee inter-thread visibility.
This patch optimized the volatile operation with c11 atomic operations
and one-way barrier to save the performance penalty. According to the
timer_perf_autotest benchmarking results, this patch can uplift 10%~16%
timer appending performance, 3%~20% timer resetting performance and 45%
timer callbacks scheduling performance on aarch64 and no loss in
performance for x86.
Suggested-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
Signed-off-by: Phil Yang <phil.yang@arm.com>
Reviewed-by: Gavin Hu <gavin.hu@arm.com>
Acked-by: Erik Gabriel Carrillo <erik.g.carrillo@intel.com>
---
It is still using built-ins as the wrapper functions for C11 built-ins
are not defined yet
v4:
fix ABI break in rte_timer struct.
v3:
fix typo.
v2:
Changed the memory ordering comment in timer_set_config_state.
lib/librte_timer/rte_timer.c | 87 ++++++++++++++++++++++++++++++--------------
1 file changed, 60 insertions(+), 27 deletions(-)
diff --git a/lib/librte_timer/rte_timer.c b/lib/librte_timer/rte_timer.c
index 269e921..6d19ce4 100644
--- a/lib/librte_timer/rte_timer.c
+++ b/lib/librte_timer/rte_timer.c
@@ -10,7 +10,6 @@
#include <assert.h>
#include <sys/queue.h>
-#include <rte_atomic.h>
#include <rte_common.h>
#include <rte_cycles.h>
#include <rte_eal_memconfig.h>
@@ -218,7 +217,7 @@ rte_timer_init(struct rte_timer *tim)
status.state = RTE_TIMER_STOP;
status.owner = RTE_TIMER_NO_OWNER;
- tim->status.u32 = status.u32;
+ __atomic_store_n(&tim->status.u32, status.u32, __ATOMIC_RELAXED);
}
/*
@@ -239,9 +238,9 @@ timer_set_config_state(struct rte_timer *tim,
/* wait that the timer is in correct status before update,
* and mark it as being configured */
- while (success == 0) {
- prev_status.u32 = tim->status.u32;
+ prev_status.u32 = __atomic_load_n(&tim->status.u32, __ATOMIC_RELAXED);
+ while (success == 0) {
/* timer is running on another core
* or ready to run on local core, exit
*/
@@ -258,9 +257,15 @@ timer_set_config_state(struct rte_timer *tim,
* mark it atomically as being configured */
status.state = RTE_TIMER_CONFIG;
status.owner = (int16_t)lcore_id;
- success = rte_atomic32_cmpset(&tim->status.u32,
- prev_status.u32,
- status.u32);
+ /* CONFIG states are acting as locked states. If the
+ * timer is in CONFIG state, the state cannot be changed
+ * by other threads. So, we should use ACQUIRE here.
+ */
+ success = __atomic_compare_exchange_n(&tim->status.u32,
+ &prev_status.u32,
+ status.u32, 0,
+ __ATOMIC_ACQUIRE,
+ __ATOMIC_RELAXED);
}
ret_prev_status->u32 = prev_status.u32;
@@ -279,20 +284,27 @@ timer_set_running_state(struct rte_timer *tim)
/* wait that the timer is in correct status before update,
* and mark it as running */
- while (success == 0) {
- prev_status.u32 = tim->status.u32;
+ prev_status.u32 = __atomic_load_n(&tim->status.u32, __ATOMIC_RELAXED);
+ while (success == 0) {
/* timer is not pending anymore */
if (prev_status.state != RTE_TIMER_PENDING)
return -1;
- /* here, we know that timer is stopped or pending,
- * mark it atomically as being configured */
+ /* we know that the timer will be pending at this point
+ * mark it atomically as being running
+ */
status.state = RTE_TIMER_RUNNING;
status.owner = (int16_t)lcore_id;
- success = rte_atomic32_cmpset(&tim->status.u32,
- prev_status.u32,
- status.u32);
+ /* RUNNING states are acting as locked states. If the
+ * timer is in RUNNING state, the state cannot be changed
+ * by other threads. So, we should use ACQUIRE here.
+ */
+ success = __atomic_compare_exchange_n(&tim->status.u32,
+ &prev_status.u32,
+ status.u32, 0,
+ __ATOMIC_ACQUIRE,
+ __ATOMIC_RELAXED);
}
return 0;
@@ -520,10 +532,12 @@ __rte_timer_reset(struct rte_timer *tim, uint64_t expire,
/* update state: as we are in CONFIG state, only us can modify
* the state so we don't need to use cmpset() here */
- rte_wmb();
status.state = RTE_TIMER_PENDING;
status.owner = (int16_t)tim_lcore;
- tim->status.u32 = status.u32;
+ /* The "RELEASE" ordering guarantees the memory operations above
+ * the status update are observed before the update by all threads
+ */
+ __atomic_store_n(&tim->status.u32, status.u32, __ATOMIC_RELEASE);
if (tim_lcore != lcore_id || !local_is_locked)
rte_spinlock_unlock(&priv_timer[tim_lcore].list_lock);
@@ -600,10 +614,12 @@ __rte_timer_stop(struct rte_timer *tim, int local_is_locked,
}
/* mark timer as stopped */
- rte_wmb();
status.state = RTE_TIMER_STOP;
status.owner = RTE_TIMER_NO_OWNER;
- tim->status.u32 = status.u32;
+ /* The "RELEASE" ordering guarantees the memory operations above
+ * the status update are observed before the update by all threads
+ */
+ __atomic_store_n(&tim->status.u32, status.u32, __ATOMIC_RELEASE);
return 0;
}
@@ -637,7 +653,8 @@ rte_timer_stop_sync(struct rte_timer *tim)
int
rte_timer_pending(struct rte_timer *tim)
{
- return tim->status.state == RTE_TIMER_PENDING;
+ return __atomic_load_n(&tim->status.state,
+ __ATOMIC_RELAXED) == RTE_TIMER_PENDING;
}
/* must be called periodically, run all timer that expired */
@@ -739,8 +756,12 @@ __rte_timer_manage(struct rte_timer_data *timer_data)
/* remove from done list and mark timer as stopped */
status.state = RTE_TIMER_STOP;
status.owner = RTE_TIMER_NO_OWNER;
- rte_wmb();
- tim->status.u32 = status.u32;
+ /* The "RELEASE" ordering guarantees the memory
+ * operations above the status update are observed
+ * before the update by all threads
+ */
+ __atomic_store_n(&tim->status.u32, status.u32,
+ __ATOMIC_RELEASE);
}
else {
/* keep it in list and mark timer as pending */
@@ -748,8 +769,12 @@ __rte_timer_manage(struct rte_timer_data *timer_data)
status.state = RTE_TIMER_PENDING;
__TIMER_STAT_ADD(priv_timer, pending, 1);
status.owner = (int16_t)lcore_id;
- rte_wmb();
- tim->status.u32 = status.u32;
+ /* The "RELEASE" ordering guarantees the memory
+ * operations above the status update are observed
+ * before the update by all threads
+ */
+ __atomic_store_n(&tim->status.u32, status.u32,
+ __ATOMIC_RELEASE);
__rte_timer_reset(tim, tim->expire + tim->period,
tim->period, lcore_id, tim->f, tim->arg, 1,
timer_data);
@@ -919,8 +944,12 @@ rte_timer_alt_manage(uint32_t timer_data_id,
/* remove from done list and mark timer as stopped */
status.state = RTE_TIMER_STOP;
status.owner = RTE_TIMER_NO_OWNER;
- rte_wmb();
- tim->status.u32 = status.u32;
+ /* The "RELEASE" ordering guarantees the memory
+ * operations above the status update are observed
+ * before the update by all threads
+ */
+ __atomic_store_n(&tim->status.u32, status.u32,
+ __ATOMIC_RELEASE);
} else {
/* keep it in list and mark timer as pending */
rte_spinlock_lock(
@@ -928,8 +957,12 @@ rte_timer_alt_manage(uint32_t timer_data_id,
status.state = RTE_TIMER_PENDING;
__TIMER_STAT_ADD(data->priv_timer, pending, 1);
status.owner = (int16_t)this_lcore;
- rte_wmb();
- tim->status.u32 = status.u32;
+ /* The "RELEASE" ordering guarantees the memory
+ * operations above the status update are observed
+ * before the update by all threads
+ */
+ __atomic_store_n(&tim->status.u32, status.u32,
+ __ATOMIC_RELEASE);
__rte_timer_reset(tim, tim->expire + tim->period,
tim->period, this_lcore, tim->f, tim->arg, 1,
data);
--
2.7.4
^ permalink raw reply [relevance 2%]
* Re: [dpdk-dev] [PATCH v3] lib/timer: relax barrier for status update
2020-04-26 12:18 0% ` Carrillo, Erik G
@ 2020-04-26 14:20 0% ` Phil Yang
2020-04-26 19:30 0% ` Carrillo, Erik G
0 siblings, 1 reply; 200+ results
From: Phil Yang @ 2020-04-26 14:20 UTC (permalink / raw)
To: Carrillo, Erik G, thomas
Cc: rsanford, dev, david.marchand, Honnappa Nagarahalli, Gavin Hu,
nd, nd, nd
> -----Original Message-----
> From: Carrillo, Erik G <erik.g.carrillo@intel.com>
> Sent: Sunday, April 26, 2020 8:19 PM
> To: Phil Yang <Phil.Yang@arm.com>; thomas@monjalon.net
> Cc: rsanford@akamai.com; dev@dpdk.org; david.marchand@redhat.com;
> Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>; Gavin Hu
> <Gavin.Hu@arm.com>; nd <nd@arm.com>; nd <nd@arm.com>
> Subject: RE: [dpdk-dev] [PATCH v3] lib/timer: relax barrier for status update
>
>
>
> > -----Original Message-----
> > From: Phil Yang <Phil.Yang@arm.com>
> > Sent: Sunday, April 26, 2020 2:36 AM
> > To: thomas@monjalon.net
> > Cc: Carrillo, Erik G <erik.g.carrillo@intel.com>; rsanford@akamai.com;
> > dev@dpdk.org; david.marchand@redhat.com; Honnappa Nagarahalli
> > <Honnappa.Nagarahalli@arm.com>; Gavin Hu <Gavin.Hu@arm.com>; nd
> > <nd@arm.com>; nd <nd@arm.com>
> > Subject: RE: [dpdk-dev] [PATCH v3] lib/timer: relax barrier for status
> update
> >
> > > -----Original Message-----
> > > From: Thomas Monjalon <thomas@monjalon.net>
> > > Sent: Sunday, April 26, 2020 1:18 AM
> > > To: Phil Yang <Phil.Yang@arm.com>
> > > Cc: erik.g.carrillo@intel.com; rsanford@akamai.com; dev@dpdk.org;
> > > david.marchand@redhat.com; Honnappa Nagarahalli
> > > <Honnappa.Nagarahalli@arm.com>; Gavin Hu <Gavin.Hu@arm.com>; nd
> > > <nd@arm.com>
> > > Subject: Re: [dpdk-dev] [PATCH v3] lib/timer: relax barrier for status
> > > update
> > >
> > > 24/04/2020 09:24, Phil Yang:
> > > > Volatile has no ordering semantics. The rte_timer structure defines
> > > > timer status as a volatile variable and uses the rte_r/wmb barrier
> > > > to guarantee inter-thread visibility.
> > > >
> > > > This patch optimized the volatile operation with c11 atomic
> > > > operations and one-way barrier to save the performance penalty.
> > > > According to the timer_perf_autotest benchmarking results, this
> > > > patch can uplift 10%~16% timer appending performance, 3%~20% timer
> > > > resetting performance and
> > > 45%
> > > > timer callbacks scheduling performance on aarch64 and no loss in
> > > > performance for x86.
> > > >
> > > > Suggested-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
> > > > Signed-off-by: Phil Yang <phil.yang@arm.com>
> > > > Reviewed-by: Gavin Hu <gavin.hu@arm.com>
> > > > Acked-by: Erik Gabriel Carrillo <erik.g.carrillo@intel.com>
> > > [...]
> > > > --- a/lib/librte_timer/rte_timer.h
> > > > +++ b/lib/librte_timer/rte_timer.h
> > > > @@ -101,7 +101,7 @@ struct rte_timer
> > > > - volatile union rte_timer_status status; /**< Status of timer. */
> > > > + union rte_timer_status status; /**< Status of timer. */
> > >
> > > Unfortunately, I cannot merge this patch because it breaks the ABI:
> > >
> > > [C]'function void rte_timer_init(rte_timer*)' at rte_timer.c:214:1
> > > has some indirect sub-type changes:
> > > parameter 1 of type 'rte_timer*' has sub-type changes:
> > > in pointed to type 'struct rte_timer' at rte_timer.h:100:1:
> > > type size hasn't changed
> > > 1 data member changes (2 filtered):
> > > type of 'volatile rte_timer_status rte_timer::status' changed:
> > > entity changed from 'volatile rte_timer_status' to 'union
> > > rte_timer_status' at rte_timer.h:67:1
> > > type size hasn't changed
> > >
> >
> > I think we can revert it to the original definition of rte_timer and keep the
> > union rte_timer_status volatile-qualified.
> > Because with or without this 'volatile' qualify, it generates the same code
> on
> > aarch64 and x86.
> > So it seems acceptable to ignore it to make the ABI compatible?
> >
> > Thank,
> > Phil
>
> I was wondering about this also. Is the performance improvement on
> aarch64 still the same in that case?
Yes. it is.
It got the same performance improvement on aarch64 and no performance loss on x86.
I will update it in v4.
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH v3] lib/timer: relax barrier for status update
2020-04-26 7:36 3% ` Phil Yang
@ 2020-04-26 12:18 0% ` Carrillo, Erik G
2020-04-26 14:20 0% ` Phil Yang
0 siblings, 1 reply; 200+ results
From: Carrillo, Erik G @ 2020-04-26 12:18 UTC (permalink / raw)
To: Phil Yang, thomas
Cc: rsanford, dev, david.marchand, Honnappa Nagarahalli, Gavin Hu, nd, nd
> -----Original Message-----
> From: Phil Yang <Phil.Yang@arm.com>
> Sent: Sunday, April 26, 2020 2:36 AM
> To: thomas@monjalon.net
> Cc: Carrillo, Erik G <erik.g.carrillo@intel.com>; rsanford@akamai.com;
> dev@dpdk.org; david.marchand@redhat.com; Honnappa Nagarahalli
> <Honnappa.Nagarahalli@arm.com>; Gavin Hu <Gavin.Hu@arm.com>; nd
> <nd@arm.com>; nd <nd@arm.com>
> Subject: RE: [dpdk-dev] [PATCH v3] lib/timer: relax barrier for status update
>
> > -----Original Message-----
> > From: Thomas Monjalon <thomas@monjalon.net>
> > Sent: Sunday, April 26, 2020 1:18 AM
> > To: Phil Yang <Phil.Yang@arm.com>
> > Cc: erik.g.carrillo@intel.com; rsanford@akamai.com; dev@dpdk.org;
> > david.marchand@redhat.com; Honnappa Nagarahalli
> > <Honnappa.Nagarahalli@arm.com>; Gavin Hu <Gavin.Hu@arm.com>; nd
> > <nd@arm.com>
> > Subject: Re: [dpdk-dev] [PATCH v3] lib/timer: relax barrier for status
> > update
> >
> > 24/04/2020 09:24, Phil Yang:
> > > Volatile has no ordering semantics. The rte_timer structure defines
> > > timer status as a volatile variable and uses the rte_r/wmb barrier
> > > to guarantee inter-thread visibility.
> > >
> > > This patch optimized the volatile operation with c11 atomic
> > > operations and one-way barrier to save the performance penalty.
> > > According to the timer_perf_autotest benchmarking results, this
> > > patch can uplift 10%~16% timer appending performance, 3%~20% timer
> > > resetting performance and
> > 45%
> > > timer callbacks scheduling performance on aarch64 and no loss in
> > > performance for x86.
> > >
> > > Suggested-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
> > > Signed-off-by: Phil Yang <phil.yang@arm.com>
> > > Reviewed-by: Gavin Hu <gavin.hu@arm.com>
> > > Acked-by: Erik Gabriel Carrillo <erik.g.carrillo@intel.com>
> > [...]
> > > --- a/lib/librte_timer/rte_timer.h
> > > +++ b/lib/librte_timer/rte_timer.h
> > > @@ -101,7 +101,7 @@ struct rte_timer
> > > - volatile union rte_timer_status status; /**< Status of timer. */
> > > + union rte_timer_status status; /**< Status of timer. */
> >
> > Unfortunately, I cannot merge this patch because it breaks the ABI:
> >
> > [C]'function void rte_timer_init(rte_timer*)' at rte_timer.c:214:1
> > has some indirect sub-type changes:
> > parameter 1 of type 'rte_timer*' has sub-type changes:
> > in pointed to type 'struct rte_timer' at rte_timer.h:100:1:
> > type size hasn't changed
> > 1 data member changes (2 filtered):
> > type of 'volatile rte_timer_status rte_timer::status' changed:
> > entity changed from 'volatile rte_timer_status' to 'union
> > rte_timer_status' at rte_timer.h:67:1
> > type size hasn't changed
> >
>
> I think we can revert it to the original definition of rte_timer and keep the
> union rte_timer_status volatile-qualified.
> Because with or without this 'volatile' qualify, it generates the same code on
> aarch64 and x86.
> So it seems acceptable to ignore it to make the ABI compatible?
>
> Thank,
> Phil
I was wondering about this also. Is the performance improvement on aarch64 still the same in that case?
Thanks,
Erik
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH v3] lib/timer: relax barrier for status update
2020-04-25 17:17 3% ` Thomas Monjalon
@ 2020-04-26 7:36 3% ` Phil Yang
2020-04-26 12:18 0% ` Carrillo, Erik G
0 siblings, 1 reply; 200+ results
From: Phil Yang @ 2020-04-26 7:36 UTC (permalink / raw)
To: thomas
Cc: erik.g.carrillo, rsanford, dev, david.marchand,
Honnappa Nagarahalli, Gavin Hu, nd, nd
> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: Sunday, April 26, 2020 1:18 AM
> To: Phil Yang <Phil.Yang@arm.com>
> Cc: erik.g.carrillo@intel.com; rsanford@akamai.com; dev@dpdk.org;
> david.marchand@redhat.com; Honnappa Nagarahalli
> <Honnappa.Nagarahalli@arm.com>; Gavin Hu <Gavin.Hu@arm.com>; nd
> <nd@arm.com>
> Subject: Re: [dpdk-dev] [PATCH v3] lib/timer: relax barrier for status update
>
> 24/04/2020 09:24, Phil Yang:
> > Volatile has no ordering semantics. The rte_timer structure defines
> > timer status as a volatile variable and uses the rte_r/wmb barrier
> > to guarantee inter-thread visibility.
> >
> > This patch optimized the volatile operation with c11 atomic operations
> > and one-way barrier to save the performance penalty. According to the
> > timer_perf_autotest benchmarking results, this patch can uplift 10%~16%
> > timer appending performance, 3%~20% timer resetting performance and
> 45%
> > timer callbacks scheduling performance on aarch64 and no loss in
> > performance for x86.
> >
> > Suggested-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
> > Signed-off-by: Phil Yang <phil.yang@arm.com>
> > Reviewed-by: Gavin Hu <gavin.hu@arm.com>
> > Acked-by: Erik Gabriel Carrillo <erik.g.carrillo@intel.com>
> [...]
> > --- a/lib/librte_timer/rte_timer.h
> > +++ b/lib/librte_timer/rte_timer.h
> > @@ -101,7 +101,7 @@ struct rte_timer
> > - volatile union rte_timer_status status; /**< Status of timer. */
> > + union rte_timer_status status; /**< Status of timer. */
>
> Unfortunately, I cannot merge this patch because it breaks the ABI:
>
> [C]'function void rte_timer_init(rte_timer*)' at rte_timer.c:214:1 has some
> indirect sub-type changes:
> parameter 1 of type 'rte_timer*' has sub-type changes:
> in pointed to type 'struct rte_timer' at rte_timer.h:100:1:
> type size hasn't changed
> 1 data member changes (2 filtered):
> type of 'volatile rte_timer_status rte_timer::status' changed:
> entity changed from 'volatile rte_timer_status' to 'union
> rte_timer_status' at rte_timer.h:67:1
> type size hasn't changed
>
I think we can revert it to the original definition of rte_timer and keep the union rte_timer_status volatile-qualified.
Because with or without this 'volatile' qualify, it generates the same code on aarch64 and x86.
So it seems acceptable to ignore it to make the ABI compatible?
Thank,
Phil
^ permalink raw reply [relevance 3%]
* [dpdk-dev] [PATCH v10 0/2] support for VFIO-PCI VF token interface
2020-04-22 5:08 4% ` [dpdk-dev] [PATCH v9 " Haiyue Wang
@ 2020-04-26 1:55 4% ` Haiyue Wang
2020-05-05 7:46 4% ` [dpdk-dev] [PATCH v11 " Haiyue Wang
3 siblings, 0 replies; 200+ results
From: Haiyue Wang @ 2020-04-26 1:55 UTC (permalink / raw)
To: dev, anatoly.burakov, thomas, vattunuru, jerinj, alex.williamson,
david.marchand
Cc: Haiyue Wang
v10: Use the __rte_internal to mark the internal API changing.
v9: Rewrite the document.
v8: Update the document.
v7: Add the Fixes tag in uuid, the release note and help
document.
v6: Drop the Fixes tag in uuid, since the file has been
moved to another place, not suitable to apply on stable.
And this is not a bug, just some kind of enhancement.
v5: 1. Add the VF token parse error handling.
2. Split into two patches for different logic module.
3. Add more comments into the code for explaining the design.
4. Drop the ABI change workaround, this patch set focuses on code review.
v4: 1. Ignore rte_vfio_setup_device ABI check since it is
for Linux driver use.
v3: Fix the Travis build failed:
(1). rte_uuid.h:97:55: error: unknown type name ‘size_t’
(2). rte_uuid.h:58:2: error: implicit declaration of function ‘memcpy’
v2: Fix the FreeBSD build error.
v1: Update the commit message.
RFC v2:
Based on Vamsi's RFC v1, and Alex's patch for Qemu
[https://lore.kernel.org/lkml/20200204161737.34696b91@w520.home/]:
Use the devarg to pass-down the VF token.
RFC v1: https://patchwork.dpdk.org/patch/66281/ by Vamsi.
Haiyue Wang (2):
eal: add uuid dependent header files explicitly
eal: support for VFIO-PCI VF token
doc/guides/linux_gsg/linux_drivers.rst | 41 +++++++++++++-
doc/guides/rel_notes/release_20_05.rst | 6 +++
drivers/bus/pci/linux/pci_vfio.c | 74 +++++++++++++++++++++++++-
lib/librte_eal/freebsd/eal.c | 3 +-
lib/librte_eal/include/rte_uuid.h | 2 +
lib/librte_eal/include/rte_vfio.h | 26 ++++++++-
lib/librte_eal/linux/eal_vfio.c | 20 +++++--
lib/librte_eal/rte_eal_version.map | 8 ++-
8 files changed, 171 insertions(+), 9 deletions(-)
--
2.26.2
^ permalink raw reply [relevance 4%]
* [dpdk-dev] [PATCH v3] mempool: return 0 if area is too small on populate
@ 2020-04-25 22:23 12% ` Thomas Monjalon
2020-04-26 16:52 12% ` [dpdk-dev] [PATCH v4] " Thomas Monjalon
2020-04-27 11:44 0% ` [dpdk-dev] [PATCH v3] " Ray Kinsella
0 siblings, 2 replies; 200+ results
From: Thomas Monjalon @ 2020-04-25 22:23 UTC (permalink / raw)
To: dev
Cc: david.marchand, Ray Kinsella, Olivier Matz, Neil Horman,
John McNamara, Marko Kovacevic, Xiaoyun Li, Jingjing Wu,
Andrew Rybchenko
From: Olivier Matz <olivier.matz@6wind.com>
Change rte_mempool_populate_iova() and rte_mempool_populate_iova() to
return 0 instead of -EINVAL when there is not enough room to store one
object, as it can be helpful for applications to distinguish this
specific case.
As this is an ABI change, use symbol versioning to preserve old
behavior for binary applications.
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
---
changes in v3:
- rebase
- remove deprecation notice
- notify API change in release notes
- fix ABI version from 20.0.1 to 20.0.2 (should be 21 maybe)
This v3 cannot be merged because of a false positive ABI check:
2 Removed functions:
'function int rte_mempool_populate_iova(rte_mempool*, char*, rte_iova_t, size_t, rte_mempool_memchunk_free_cb_t*, void*)' {rte_mempool_populate_iova@@DPDK_20.0}
'function int rte_mempool_populate_virt(rte_mempool*, char*, size_t, size_t, rte_mempool_memchunk_free_cb_t*, void*)' {rte_mempool_populate_virt@@DPDK_20.0}
---
doc/guides/rel_notes/deprecation.rst | 5 --
doc/guides/rel_notes/release_20_05.rst | 4 ++
examples/ntb/ntb_fwd.c | 2 +-
lib/librte_mempool/meson.build | 2 +
lib/librte_mempool/rte_mempool.c | 77 ++++++++++++++++++----
lib/librte_mempool/rte_mempool.h | 14 ++--
lib/librte_mempool/rte_mempool_version.map | 7 ++
7 files changed, 90 insertions(+), 21 deletions(-)
diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
index 1339f54f5f..20aa745b77 100644
--- a/doc/guides/rel_notes/deprecation.rst
+++ b/doc/guides/rel_notes/deprecation.rst
@@ -65,11 +65,6 @@ Deprecation Notices
structure would be made internal (or removed if all dependencies are cleared)
in future releases.
-* mempool: starting from v20.05, the API of rte_mempool_populate_iova()
- and rte_mempool_populate_virt() will change to return 0 instead
- of -EINVAL when there is not enough room to store one object. The ABI
- will be preserved until 20.11.
-
* ethdev: the legacy filter API, including
``rte_eth_dev_filter_supported()``, ``rte_eth_dev_filter_ctrl()`` as well
as filter types MACVLAN, ETHERTYPE, FLEXIBLE, SYN, NTUPLE, TUNNEL, FDIR,
diff --git a/doc/guides/rel_notes/release_20_05.rst b/doc/guides/rel_notes/release_20_05.rst
index b124c3f287..ab20a7d021 100644
--- a/doc/guides/rel_notes/release_20_05.rst
+++ b/doc/guides/rel_notes/release_20_05.rst
@@ -241,6 +241,10 @@ API Changes
Also, make sure to start the actual text at the margin.
=========================================================
+* mempool: The API of ``rte_mempool_populate_iova()`` and
+ ``rte_mempool_populate_virt()`` changed to return 0 instead of -EINVAL
+ when there is not enough room to store one object.
+
ABI Changes
-----------
diff --git a/examples/ntb/ntb_fwd.c b/examples/ntb/ntb_fwd.c
index d49189e175..eba8ebf9fa 100644
--- a/examples/ntb/ntb_fwd.c
+++ b/examples/ntb/ntb_fwd.c
@@ -1319,7 +1319,7 @@ ntb_mbuf_pool_create(uint16_t mbuf_seg_size, uint32_t nb_mbuf,
mz->len - ntb_info.ntb_hdr_size,
ntb_mempool_mz_free,
(void *)(uintptr_t)mz);
- if (ret < 0) {
+ if (ret <= 0) {
rte_memzone_free(mz);
rte_mempool_free(mp);
return NULL;
diff --git a/lib/librte_mempool/meson.build b/lib/librte_mempool/meson.build
index a6e861cbfc..7dbe6b9bea 100644
--- a/lib/librte_mempool/meson.build
+++ b/lib/librte_mempool/meson.build
@@ -9,6 +9,8 @@ foreach flag: extra_flags
endif
endforeach
+use_function_versioning = true
+
sources = files('rte_mempool.c', 'rte_mempool_ops.c',
'rte_mempool_ops_default.c', 'mempool_trace_points.c')
headers = files('rte_mempool.h', 'rte_mempool_trace.h',
diff --git a/lib/librte_mempool/rte_mempool.c b/lib/librte_mempool/rte_mempool.c
index 0be8f9f59d..edbdafaafb 100644
--- a/lib/librte_mempool/rte_mempool.c
+++ b/lib/librte_mempool/rte_mempool.c
@@ -31,6 +31,7 @@
#include <rte_string_fns.h>
#include <rte_spinlock.h>
#include <rte_tailq.h>
+#include <rte_function_versioning.h>
#include "rte_mempool.h"
#include "rte_mempool_trace.h"
@@ -303,12 +304,17 @@ mempool_ops_alloc_once(struct rte_mempool *mp)
return 0;
}
+__vsym int
+rte_mempool_populate_iova_v20_0_2(struct rte_mempool *mp, char *vaddr,
+ rte_iova_t iova, size_t len, rte_mempool_memchunk_free_cb_t *free_cb,
+ void *opaque);
+
/* Add objects in the pool, using a physically contiguous memory
* zone. Return the number of objects added, or a negative value
* on error.
*/
-static int
-__rte_mempool_populate_iova(struct rte_mempool *mp, char *vaddr,
+__vsym int
+rte_mempool_populate_iova_v20_0_2(struct rte_mempool *mp, char *vaddr,
rte_iova_t iova, size_t len, rte_mempool_memchunk_free_cb_t *free_cb,
void *opaque)
{
@@ -366,14 +372,27 @@ __rte_mempool_populate_iova(struct rte_mempool *mp, char *vaddr,
return ret;
}
-int
-rte_mempool_populate_iova(struct rte_mempool *mp, char *vaddr,
+BIND_DEFAULT_SYMBOL(rte_mempool_populate_iova, _v20_0_2, 20.0.2);
+MAP_STATIC_SYMBOL(
+ int rte_mempool_populate_iova(struct rte_mempool *mp, char *vaddr,
+ rte_iova_t iova, size_t len,
+ rte_mempool_memchunk_free_cb_t *free_cb,
+ void *opaque),
+ rte_mempool_populate_iova_v20_0_2);
+
+__vsym int
+rte_mempool_populate_iova_v20(struct rte_mempool *mp, char *vaddr,
+ rte_iova_t iova, size_t len, rte_mempool_memchunk_free_cb_t *free_cb,
+ void *opaque);
+
+__vsym int
+rte_mempool_populate_iova_v20(struct rte_mempool *mp, char *vaddr,
rte_iova_t iova, size_t len, rte_mempool_memchunk_free_cb_t *free_cb,
void *opaque)
{
int ret;
- ret = __rte_mempool_populate_iova(mp, vaddr, iova, len, free_cb,
+ ret = rte_mempool_populate_iova_v20_0_2(mp, vaddr, iova, len, free_cb,
opaque);
if (ret == 0)
ret = -EINVAL;
@@ -381,6 +400,7 @@ rte_mempool_populate_iova(struct rte_mempool *mp, char *vaddr,
rte_mempool_trace_populate_iova(mp, vaddr, iova, len, free_cb, opaque);
return ret;
}
+VERSION_SYMBOL(rte_mempool_populate_iova, _v20_0, 20.0);
static rte_iova_t
get_iova(void *addr)
@@ -395,11 +415,16 @@ get_iova(void *addr)
return ms->iova + RTE_PTR_DIFF(addr, ms->addr);
}
+__vsym int
+rte_mempool_populate_virt_v20_0_2(struct rte_mempool *mp, char *addr,
+ size_t len, size_t pg_sz, rte_mempool_memchunk_free_cb_t *free_cb,
+ void *opaque);
+
/* Populate the mempool with a virtual area. Return the number of
* objects added, or a negative value on error.
*/
-int
-rte_mempool_populate_virt(struct rte_mempool *mp, char *addr,
+__vsym int
+rte_mempool_populate_virt_v20_0_2(struct rte_mempool *mp, char *addr,
size_t len, size_t pg_sz, rte_mempool_memchunk_free_cb_t *free_cb,
void *opaque)
{
@@ -432,7 +457,7 @@ rte_mempool_populate_virt(struct rte_mempool *mp, char *addr,
break;
}
- ret = __rte_mempool_populate_iova(mp, addr + off, iova,
+ ret = rte_mempool_populate_iova_v20_0_2(mp, addr + off, iova,
phys_len, free_cb, opaque);
if (ret == 0)
continue;
@@ -443,9 +468,6 @@ rte_mempool_populate_virt(struct rte_mempool *mp, char *addr,
cnt += ret;
}
- if (cnt == 0)
- return -EINVAL;
-
rte_mempool_trace_populate_virt(mp, addr, len, pg_sz, free_cb, opaque);
return cnt;
@@ -453,6 +475,35 @@ rte_mempool_populate_virt(struct rte_mempool *mp, char *addr,
rte_mempool_free_memchunks(mp);
return ret;
}
+BIND_DEFAULT_SYMBOL(rte_mempool_populate_virt, _v20_0_2, 20.0.2);
+MAP_STATIC_SYMBOL(
+ int rte_mempool_populate_virt(struct rte_mempool *mp,
+ char *addr, size_t len, size_t pg_sz,
+ rte_mempool_memchunk_free_cb_t *free_cb,
+ void *opaque),
+ rte_mempool_populate_virt_v20_0_2);
+
+__vsym int
+rte_mempool_populate_virt_v20(struct rte_mempool *mp, char *addr,
+ size_t len, size_t pg_sz, rte_mempool_memchunk_free_cb_t *free_cb,
+ void *opaque);
+
+__vsym int
+rte_mempool_populate_virt_v20(struct rte_mempool *mp, char *addr,
+ size_t len, size_t pg_sz, rte_mempool_memchunk_free_cb_t *free_cb,
+ void *opaque)
+{
+ int ret;
+
+ ret = rte_mempool_populate_virt_v20_0_2(mp, addr, len, pg_sz,
+ free_cb, opaque);
+
+ if (ret == 0)
+ ret = -EINVAL;
+
+ return ret;
+}
+VERSION_SYMBOL(rte_mempool_populate_virt, _v20_0, 20.0);
/* Get the minimal page size used in a mempool before populating it. */
int
@@ -609,6 +660,8 @@ rte_mempool_populate_default(struct rte_mempool *mp)
mz->len, pg_sz,
rte_mempool_memchunk_mz_free,
(void *)(uintptr_t)mz);
+ if (ret == 0) /* should not happen */
+ ret = -ENOBUFS;
if (ret < 0) {
rte_memzone_free(mz);
goto fail;
@@ -701,6 +754,8 @@ rte_mempool_populate_anon(struct rte_mempool *mp)
ret = rte_mempool_populate_virt(mp, addr, size, getpagesize(),
rte_mempool_memchunk_anon_free, addr);
+ if (ret == 0) /* should not happen */
+ ret = -ENOBUFS;
if (ret < 0) {
rte_errno = -ret;
goto fail;
diff --git a/lib/librte_mempool/rte_mempool.h b/lib/librte_mempool/rte_mempool.h
index 6e0573ea42..652d19f9f1 100644
--- a/lib/librte_mempool/rte_mempool.h
+++ b/lib/librte_mempool/rte_mempool.h
@@ -1112,9 +1112,12 @@ rte_mempool_free(struct rte_mempool *mp);
* @param opaque
* An opaque argument passed to free_cb.
* @return
- * The number of objects added on success.
+ * The number of objects added on success (strictly positive).
* On error, the chunk is not added in the memory list of the
- * mempool and a negative errno is returned.
+ * mempool the following code is returned:
+ * (0): not enough room in chunk for one object.
+ * (-ENOSPC): mempool is already populated.
+ * (-ENOMEM): allocation failure.
*/
int rte_mempool_populate_iova(struct rte_mempool *mp, char *vaddr,
rte_iova_t iova, size_t len, rte_mempool_memchunk_free_cb_t *free_cb,
@@ -1139,9 +1142,12 @@ int rte_mempool_populate_iova(struct rte_mempool *mp, char *vaddr,
* @param opaque
* An opaque argument passed to free_cb.
* @return
- * The number of objects added on success.
+ * The number of objects added on success (strictly positive).
* On error, the chunk is not added in the memory list of the
- * mempool and a negative errno is returned.
+ * mempool the following code is returned:
+ * (0): not enough room in chunk for one object.
+ * (-ENOSPC): mempool is already populated.
+ * (-ENOMEM): allocation failure.
*/
int
rte_mempool_populate_virt(struct rte_mempool *mp, char *addr,
diff --git a/lib/librte_mempool/rte_mempool_version.map b/lib/librte_mempool/rte_mempool_version.map
index 695dd6e04f..9e58093665 100644
--- a/lib/librte_mempool/rte_mempool_version.map
+++ b/lib/librte_mempool/rte_mempool_version.map
@@ -31,6 +31,13 @@ DPDK_20.0 {
local: *;
};
+DPDK_20.0.2 {
+ global:
+
+ rte_mempool_populate_iova;
+ rte_mempool_populate_virt;
+} DPDK_20.0;
+
EXPERIMENTAL {
global:
--
2.26.0
^ permalink raw reply [relevance 12%]
* Re: [dpdk-dev] [PATCH v4 1/4] ethdev: add tm support for shaper config in pkt mode
@ 2020-04-25 20:09 3% ` Ferruh Yigit
2020-04-27 9:19 4% ` Dumitrescu, Cristian
0 siblings, 1 reply; 200+ results
From: Ferruh Yigit @ 2020-04-25 20:09 UTC (permalink / raw)
To: Dumitrescu, Cristian, Nithin Dabilpuram, Singh, Jasvinder,
Thomas Monjalon, Andrew Rybchenko
Cc: dev, jerinj, kkanas, Nithin Dabilpuram
On 4/24/2020 11:28 AM, Dumitrescu, Cristian wrote:
>
>
>> -----Original Message-----
>> From: Nithin Dabilpuram <nithind1988@gmail.com>
>> Sent: Wednesday, April 22, 2020 6:21 PM
>> To: Singh, Jasvinder <jasvinder.singh@intel.com>; Dumitrescu, Cristian
>> <cristian.dumitrescu@intel.com>; Thomas Monjalon
>> <thomas@monjalon.net>; Yigit, Ferruh <ferruh.yigit@intel.com>; Andrew
>> Rybchenko <arybchenko@solarflare.com>
>> Cc: dev@dpdk.org; jerinj@marvell.com; kkanas@marvell.com; Nithin
>> Dabilpuram <ndabilpuram@marvell.com>
>> Subject: [PATCH v4 1/4] ethdev: add tm support for shaper config in pkt
>> mode
>>
>> From: Nithin Dabilpuram <ndabilpuram@marvell.com>
>>
>> Some NIC hardware support shaper to work in packet mode i.e
>> shaping or ratelimiting traffic is in packets per second (PPS) as
>> opposed to default bytes per second (BPS). Hence this patch
>> adds support to configure shared or private shaper in packet mode,
>> provide rate in PPS and add related tm capabilities in port/level/node
>> capability structures.
>>
>> This patch also updates tm port/level/node capability structures with
>> exiting features of scheduler wfq packet mode, scheduler wfq byte mode
>> and private/shared shaper byte mode.
>>
>> SoftNIC PMD is also updated with new capabilities.
>>
>> Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>
>> ---
>> v3..v4:
>> - Update text under packet_mode as per Cristian.
>> - Update rte_eth_softnic_tm.c based on Jasvinder's comments.
>> - Add error enum RTE_TM_ERROR_TYPE_SHAPER_PROFILE_PACKET_MODE
>> - Fix shaper_profile_check() with packet mode check
>> - Fix typo's
>>
>
> Acked-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
>
Hi Nithin,
It looks like patch is causing ABI break, I am getting following warning [1],
can you please check?
[1]
https://pastebin.com/XYNFg14u
^ permalink raw reply [relevance 3%]
* Re: [dpdk-dev] [PATCH 00/14] cleanup resources on shutdown
@ 2020-04-25 19:34 0% ` David Marchand
2020-04-28 23:14 3% ` [dpdk-dev] [PATCH v2 0/9] eal: " Stephen Hemminger
2020-04-28 23:58 3% ` [dpdk-dev] [PATCH v3 0/8] " Stephen Hemminger
2 siblings, 0 replies; 200+ results
From: David Marchand @ 2020-04-25 19:34 UTC (permalink / raw)
To: Stephen Hemminger; +Cc: dev
On Sat, Jan 4, 2020 at 2:34 AM Stephen Hemminger
<stephen@networkplumber.org> wrote:
>
> Recently started using valgrind with DPDK, and the results
> are not clean.
>
> The DPDK has a function that applications can use to tell it
> to cleanup resources on shutdown (rte_eal_cleanup). But the
> current coverage of that API is spotty. Many internal parts of
> DPDK leave files and allocated memory behind.
>
> This patch set is a start at getting the sub-parts of
> DPDK to cleanup after themselves. These are the easier ones,
> the harder and more critical ones are in the drivers
> and the memory subsystem.
>
> There are no visible API or ABI changes here.
I was about to push the series (except patch 10), but I hit a crash
when passing an invalid option to test-null.sh.
Reproduced with:
Core was generated by
`/home/dmarchan/builds/x86_64-native-linux-gcc+shared+kmods/app/testpmd
-c 0x3 --log-level='.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x00007fd5231dba64 in pthread_cancel () from /usr/lib64/libpthread.so.0
Missing separate debuginfos, use: dnf debuginfo-install
elfutils-libelf-0.178-7.fc30.x86_64 glibc-2.29-28.fc30.x86_64
jansson-2.12-2.fc30.x86_64 libgcc-9.2.1-1.fc30.x86_64
libpcap-1.9.1-1.fc30.x86_64 numactl-libs-2.0.12-2.fc30.x86_64
zlib-1.2.11-19.fc30.x86_64
(gdb) bt full
#0 0x00007fd5231dba64 in pthread_cancel () from /usr/lib64/libpthread.so.0
No symbol table info available.
#1 0x00007fd52320c586 in rte_eal_cleanup () at
/home/dmarchan/dpdk/lib/librte_eal/linux/eal.c:1339
i = 1
#2 0x00007fd523215f5e in rte_exit (exit_code=exit_code@entry=1,
format=format@entry=0x47ada4 "Cannot init EAL: %s\n") at
/home/dmarchan/dpdk/lib/librte_eal/linux/eal_debug.c:83
ap = {{gp_offset = 24, fp_offset = 48, overflow_arg_area =
0x7ffecdf7aa70, reg_save_area = 0x7ffecdf7a9a0}}
#3 0x000000000043535b in main (argc=21, argv=0x7ffecdf7abc8) at
/home/dmarchan/dpdk/app/test-pmd/testpmd.c:3647
diag = -1
port_id = <optimized out>
count = <optimized out>
ret = <optimized out>
(gdb) f 1
#1 0x00007fd52320c586 in rte_eal_cleanup () at
/home/dmarchan/dpdk/lib/librte_eal/linux/eal.c:1339
1339 pthread_cancel(lcore_config[i].thread_id);
(gdb) p lcore_config[1].thread_id
$1 = 0
rte_eal_cleanup() is called from rte_exit() by testpmd.
But since rte_eal_init() failed at parsing, lcore_config[*].thread_id
are invalid, and we crash on pthread_cancel.
I have no quick idea to fix this, series postponed to rc2.
--
David Marchand
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH v7 0/6] dpdk: introduce __rte_internal tag
2020-04-25 14:39 0% ` [dpdk-dev] [PATCH v7 0/6] dpdk: introduce __rte_internal tag David Marchand
2020-04-25 16:34 0% ` Wang, Haiyue
@ 2020-04-25 18:09 0% ` Wang, Haiyue
2020-04-29 8:22 3% ` David Marchand
1 sibling, 1 reply; 200+ results
From: Wang, Haiyue @ 2020-04-25 18:09 UTC (permalink / raw)
To: David Marchand
Cc: dev, Thomas Monjalon, Richardson, Bruce, Yigit, Ferruh,
Neil Horman, Ray Kinsella
Hi David,
> -----Original Message-----
> From: David Marchand <david.marchand@redhat.com>
> Sent: Saturday, April 25, 2020 22:39
> To: Wang, Haiyue <haiyue.wang@intel.com>
> Cc: dev <dev@dpdk.org>; Thomas Monjalon <thomas@monjalon.net>; Richardson, Bruce
> <bruce.richardson@intel.com>; Yigit, Ferruh <ferruh.yigit@intel.com>; Neil Horman
> <nhorman@tuxdriver.com>; Ray Kinsella <mdr@ashroe.eu>
> Subject: Re: [PATCH v7 0/6] dpdk: introduce __rte_internal tag
>
> On Sat, Apr 25, 2020 at 1:02 PM Haiyue Wang <haiyue.wang@intel.com> wrote:
> >
> > Move the internal function into INTERNAL session to avoid the ABI
> > checking, and it is only used for DPDK drivers or related library.
> >
> > __rte_internal funA
> >
> > INTERNAL {
> > global:
> >
> > funA
> > };
>
> Thanks a lot for working on this.
> I did some modifications (see my replies on patch 3 and 5) and applied
> this series.
>
> We are just missing the update on the scripts mentioned in a previous mail.
> Can you work on this for rc2?
>
Do you mean ?
> > > This will apply to common drivers that will be 100% internal.
> > > Not sure if this is an issue.
> >
> > This part should be fine, I want others to be aware of this.
> I am not one of the ABI maintainers, but in my opinion it is OK
> to have "pure internal" libs with version 0.x.
I've tested it with Intel's drivers/common/iavf, it works as expected.
a). librte_common_iavf.so.0.200.2
b). Skipped experimental library librte_common_iavf.dump.
This has been updated by your modification.
+ if is_stable
lib_version = abi_version
so_version = stable_so_version
+ else
+ lib_version = experimental_abi_version
+ so_version = experimental_so_version
endif
> Thanks again!
>
>
> --
> David Marchand
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH v3] lib/timer: relax barrier for status update
@ 2020-04-25 17:17 3% ` Thomas Monjalon
2020-04-26 7:36 3% ` Phil Yang
2020-04-26 14:45 2% ` [dpdk-dev] [PATCH v4] " Phil Yang
1 sibling, 1 reply; 200+ results
From: Thomas Monjalon @ 2020-04-25 17:17 UTC (permalink / raw)
To: Phil Yang
Cc: erik.g.carrillo, rsanford, dev, david.marchand,
Honnappa.Nagarahalli, Gavin.Hu, nd
24/04/2020 09:24, Phil Yang:
> Volatile has no ordering semantics. The rte_timer structure defines
> timer status as a volatile variable and uses the rte_r/wmb barrier
> to guarantee inter-thread visibility.
>
> This patch optimized the volatile operation with c11 atomic operations
> and one-way barrier to save the performance penalty. According to the
> timer_perf_autotest benchmarking results, this patch can uplift 10%~16%
> timer appending performance, 3%~20% timer resetting performance and 45%
> timer callbacks scheduling performance on aarch64 and no loss in
> performance for x86.
>
> Suggested-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
> Signed-off-by: Phil Yang <phil.yang@arm.com>
> Reviewed-by: Gavin Hu <gavin.hu@arm.com>
> Acked-by: Erik Gabriel Carrillo <erik.g.carrillo@intel.com>
[...]
> --- a/lib/librte_timer/rte_timer.h
> +++ b/lib/librte_timer/rte_timer.h
> @@ -101,7 +101,7 @@ struct rte_timer
> - volatile union rte_timer_status status; /**< Status of timer. */
> + union rte_timer_status status; /**< Status of timer. */
Unfortunately, I cannot merge this patch because it breaks the ABI:
[C]'function void rte_timer_init(rte_timer*)' at rte_timer.c:214:1 has some indirect sub-type changes:
parameter 1 of type 'rte_timer*' has sub-type changes:
in pointed to type 'struct rte_timer' at rte_timer.h:100:1:
type size hasn't changed
1 data member changes (2 filtered):
type of 'volatile rte_timer_status rte_timer::status' changed:
entity changed from 'volatile rte_timer_status' to 'union rte_timer_status' at rte_timer.h:67:1
type size hasn't changed
^ permalink raw reply [relevance 3%]
* Re: [dpdk-dev] [PATCH v7 0/6] dpdk: introduce __rte_internal tag
2020-04-25 14:39 0% ` [dpdk-dev] [PATCH v7 0/6] dpdk: introduce __rte_internal tag David Marchand
@ 2020-04-25 16:34 0% ` Wang, Haiyue
2020-04-25 18:09 0% ` Wang, Haiyue
1 sibling, 0 replies; 200+ results
From: Wang, Haiyue @ 2020-04-25 16:34 UTC (permalink / raw)
To: David Marchand
Cc: dev, Thomas Monjalon, Richardson, Bruce, Yigit, Ferruh,
Neil Horman, Ray Kinsella
> -----Original Message-----
> From: David Marchand <david.marchand@redhat.com>
> Sent: Saturday, April 25, 2020 22:39
> To: Wang, Haiyue <haiyue.wang@intel.com>
> Cc: dev <dev@dpdk.org>; Thomas Monjalon <thomas@monjalon.net>; Richardson, Bruce
> <bruce.richardson@intel.com>; Yigit, Ferruh <ferruh.yigit@intel.com>; Neil Horman
> <nhorman@tuxdriver.com>; Ray Kinsella <mdr@ashroe.eu>
> Subject: Re: [PATCH v7 0/6] dpdk: introduce __rte_internal tag
>
> On Sat, Apr 25, 2020 at 1:02 PM Haiyue Wang <haiyue.wang@intel.com> wrote:
> >
> > Move the internal function into INTERNAL session to avoid the ABI
> > checking, and it is only used for DPDK drivers or related library.
> >
> > __rte_internal funA
> >
> > INTERNAL {
> > global:
> >
> > funA
> > };
>
> Thanks a lot for working on this.
> I did some modifications (see my replies on patch 3 and 5) and applied
> this series.
>
> We are just missing the update on the scripts mentioned in a previous mail.
> Can you work on this for rc2?
>
Sure, it's my pleasure. ;-)
> Thanks again!
>
>
> --
> David Marchand
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH v4] eal/cpuflags: add x86 based cpu flags
@ 2020-04-25 16:04 4% ` Thomas Monjalon
2020-04-27 9:22 4% ` Kinsella, Ray
2020-04-27 9:27 4% ` Ray Kinsella
1 sibling, 2 replies; 200+ results
From: Thomas Monjalon @ 2020-04-25 16:04 UTC (permalink / raw)
To: ray.kinsella, nhorman, Kevin Laatz
Cc: dev, bruce.richardson, harry.van.haaren, david.marchand
16/04/2020 13:00, Kevin Laatz:
> This patch adds CPU flags which will enable the detection of ISA
> features available on more recent x86 based CPUs.
[...]
> --- a/devtools/libabigail.abignore
> +++ b/devtools/libabigail.abignore
> +; Ignore this enum update as it should not be allocated by the application
> +[suppress_type]
> + type_kind = enum
> + name = rte_cpu_flag_t
> + changed_enumerators = RTE_CPUFLAG_NUMFLAGS
The justification is not correct.
The application is allowed to use RTE_CPUFLAG_NUMFLAGS in array allocation.
But no API is returning a CPU flag, so the new flags will remain unknown
to the application.
However, there is a behaviour change:
The functions rte_cpu_get_flag_name() and rte_cpu_get_flag_enabled()
will now accept new values, which were previously considered as an error.
Is it an ABI breakage? I would say no.
PS: Who is REALLY maintaining the ABI?
We really miss someone who carefully check all these things,
and take care of the doc and tooling.
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH v7 0/6] dpdk: introduce __rte_internal tag
2020-04-25 10:56 5% ` [dpdk-dev] [PATCH v7 0/6] dpdk: introduce __rte_internal tag Haiyue Wang
` (4 preceding siblings ...)
2020-04-25 10:56 7% ` [dpdk-dev] [PATCH v7 5/6] devtools: exempt internal ABI checking Haiyue Wang
@ 2020-04-25 14:39 0% ` David Marchand
2020-04-25 16:34 0% ` Wang, Haiyue
2020-04-25 18:09 0% ` Wang, Haiyue
5 siblings, 2 replies; 200+ results
From: David Marchand @ 2020-04-25 14:39 UTC (permalink / raw)
To: Haiyue Wang
Cc: dev, Thomas Monjalon, Bruce Richardson, Yigit, Ferruh,
Neil Horman, Ray Kinsella
On Sat, Apr 25, 2020 at 1:02 PM Haiyue Wang <haiyue.wang@intel.com> wrote:
>
> Move the internal function into INTERNAL session to avoid the ABI
> checking, and it is only used for DPDK drivers or related library.
>
> __rte_internal funA
>
> INTERNAL {
> global:
>
> funA
> };
Thanks a lot for working on this.
I did some modifications (see my replies on patch 3 and 5) and applied
this series.
We are just missing the update on the scripts mentioned in a previous mail.
Can you work on this for rc2?
Thanks again!
--
David Marchand
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH v7 5/6] devtools: exempt internal ABI checking
2020-04-25 10:56 7% ` [dpdk-dev] [PATCH v7 5/6] devtools: exempt internal ABI checking Haiyue Wang
@ 2020-04-25 14:34 4% ` David Marchand
0 siblings, 0 replies; 200+ results
From: David Marchand @ 2020-04-25 14:34 UTC (permalink / raw)
To: Haiyue Wang
Cc: dev, Thomas Monjalon, Bruce Richardson, Yigit, Ferruh,
Neil Horman, Ray Kinsella
On Sat, Apr 25, 2020 at 1:02 PM Haiyue Wang <haiyue.wang@intel.com> wrote:
>
> No need to restrict the ABI on symbols that are only used by core
> libraries.
>
> Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
Rather than add a special case for INTERNAL, we can invert the logic
in this script: identify "stable" sections symbol.
I went with the following patch:
diff --git a/devtools/check-symbol-change.sh b/devtools/check-symbol-change.sh
index ed2178e36e..f329d5fa62 100755
--- a/devtools/check-symbol-change.sh
+++ b/devtools/check-symbol-change.sh
@@ -77,6 +77,10 @@ build_map_changes()
}
+is_stable_section() {
+ [ "$1" != 'EXPERIMENTAL' ] && [ "$1" != 'INTERNAL' ]
+}
+
check_for_rule_violations()
{
local mapdb="$1"
@@ -110,11 +114,11 @@ check_for_rule_violations()
# section directly
if [ -z "$oldsecname" ]
then
- if [ "$secname" = 'EXPERIMENTAL' ]
+ if ! is_stable_section $secname
then
echo -n "INFO: symbol $symname has "
echo -n "been added to the "
- echo -n "EXPERIMENTAL section of the "
+ echo -n "$secname section of the "
echo "version map"
continue
else
@@ -137,7 +141,7 @@ check_for_rule_violations()
# This symbol is moving between two sections (the
# original section is not experimental).
# This can be legit, just warn.
- if [ "$oldsecname" != 'EXPERIMENTAL' ]
+ if is_stable_section $oldsecname
then
echo -n "INFO: symbol $symname is being "
echo -n "moved from $oldsecname to $secname. "
@@ -148,9 +152,9 @@ check_for_rule_violations()
else
if ! grep -q "$mname $symname .* add" "$mapdb" && \
- [ "$secname" != "EXPERIMENTAL" ]
+ is_stable_section $secname
then
- # Just inform users that non-experimenal
+ # Just inform users that stable
# symbols need to go through a deprecation
# process
echo -n "INFO: symbol $symname is being "
--
David Marchand
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH v7 3/6] mk: add internal tag check
2020-04-25 10:56 5% ` [dpdk-dev] [PATCH v7 3/6] mk: add internal tag check Haiyue Wang
@ 2020-04-25 14:34 3% ` David Marchand
0 siblings, 0 replies; 200+ results
From: David Marchand @ 2020-04-25 14:34 UTC (permalink / raw)
To: Haiyue Wang
Cc: dev, Thomas Monjalon, Bruce Richardson, Yigit, Ferruh,
Neil Horman, Ray Kinsella
On Sat, Apr 25, 2020 at 1:02 PM Haiyue Wang <haiyue.wang@intel.com> wrote:
>
> Add checks during build to ensure that all symbols in the INTERNAL
> version map section have __internal tags on their definitions, and
> enable the warnings needed to announce their use.
>
> Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
> ---
> MAINTAINERS | 2 +-
> ...-experimental-syms.sh => check-symbols.sh} | 31 +++++++++++++++++++
> buildtools/meson.build | 2 +-
> mk/internal/rte.compile-pre.mk | 6 ++--
> 4 files changed, 36 insertions(+), 5 deletions(-)
> rename buildtools/{check-experimental-syms.sh => check-symbols.sh} (61%)
Just missing a little update on drivers/meson.build and lib/meson.build.
Squashed with:
diff --git a/buildtools/meson.build b/buildtools/meson.build
index 3e8d31b0c5..d5f8291beb 100644
--- a/buildtools/meson.build
+++ b/buildtools/meson.build
@@ -6,7 +6,7 @@ subdir('pmdinfogen')
pkgconf = find_program('pkg-config', 'pkgconf', required: false)
pmdinfo = find_program('gen-pmdinfo-cfile.sh')
list_dir_globs = find_program('list-dir-globs.py')
-check_experimental_syms = find_program('check-symbols.sh')
+check_symbols = find_program('check-symbols.sh')
ldflags_ibverbs_static = find_program('options-ibverbs-static.sh')
# set up map-to-def script using python, either built-in or external
@@ -20,4 +20,4 @@ map_to_def_cmd = py3 + files('map_to_def.py')
sphinx_wrapper = py3 + files('call-sphinx-build.py')
# stable ABI always starts with "DPDK_"
-is_experimental_cmd = [find_program('grep', 'findstr'), '^DPDK_']
+is_stable_cmd = [find_program('grep', 'findstr'), '^DPDK_']
diff --git a/drivers/meson.build b/drivers/meson.build
index f3dd23dd43..dc293b270b 100644
--- a/drivers/meson.build
+++ b/drivers/meson.build
@@ -131,15 +131,15 @@ foreach class:dpdk_driver_classes
meson.current_source_dir(),
drv_path, lib_name)
- is_experimental = run_command(is_experimental_cmd,
- files(version_map)).returncode()
+ is_stable = run_command(is_stable_cmd,
+ files(version_map)).returncode() == 0
- if is_experimental != 0
- lib_version = experimental_abi_version
- so_version = experimental_so_version
- else
+ if is_stable
lib_version = abi_version
so_version = stable_so_version
+ else
+ lib_version = experimental_abi_version
+ so_version = experimental_so_version
endif
# now build the static driver
@@ -168,14 +168,14 @@ foreach class:dpdk_driver_classes
else
lk_args = ['-Wl,--version-script=' +
version_map]
# on unix systems check the output of the
- # experimental syms script, using it as a
+ # check-symbols.sh script, using it as a
# dependency of the .so build
- lk_deps += custom_target(lib_name + '.exp_chk',
- command: [check_experimental_syms,
+ lk_deps += custom_target(lib_name + '.sym_chk',
+ command: [check_symbols,
version_map, '@INPUT@'],
capture: true,
input: static_lib,
- output: lib_name + '.exp_chk')
+ output: lib_name + '.sym_chk')
endif
shared_lib = shared_library(lib_name,
diff --git a/lib/meson.build b/lib/meson.build
index 8697941ae0..07a65a6256 100644
--- a/lib/meson.build
+++ b/lib/meson.build
@@ -109,15 +109,15 @@ foreach l:libraries
version_map = '@0@/@1@/rte_@2@_version.map'.format(
meson.current_source_dir(),
dir_name, name)
- is_experimental = run_command(is_experimental_cmd,
- files(version_map)).returncode()
+ is_stable = run_command(is_stable_cmd,
+ files(version_map)).returncode() == 0
- if is_experimental != 0
- lib_version = experimental_abi_version
- so_version = experimental_so_version
- else
+ if is_stable
lib_version = abi_version
so_version = stable_so_version
+ else
+ lib_version = experimental_abi_version
+ so_version = experimental_so_version
endif
# first build static lib
@@ -160,14 +160,14 @@ foreach l:libraries
lk_deps = [version_map, def_file]
if not is_windows
# on unix systems check the output of the
- # experimental syms script, using it as a
+ # check-symbols.sh script, using it as a
# dependency of the .so build
- lk_deps += custom_target(name + '.exp_chk',
- command: [check_experimental_syms,
+ lk_deps += custom_target(name + '.sym_chk',
+ command: [check_symbols,
version_map, '@INPUT@'],
capture: true,
input: static_lib,
- output: name + '.exp_chk')
+ output: name + '.sym_chk')
endif
shared_lib = shared_library(libname,
--
David Marchand
^ permalink raw reply [relevance 3%]
* Re: [dpdk-dev] [PATCH v5 1/1] eal: add internal ABI marking support
2020-04-25 14:21 4% ` David Marchand
@ 2020-04-25 14:24 7% ` Thomas Monjalon
0 siblings, 0 replies; 200+ results
From: Thomas Monjalon @ 2020-04-25 14:24 UTC (permalink / raw)
To: Wang, Haiyue, David Marchand
Cc: dev, Richardson, Bruce, Yigit, Ferruh, Neil Horman, Ray Kinsella
25/04/2020 16:21, David Marchand:
> On Sat, Apr 25, 2020 at 8:10 AM Wang, Haiyue <haiyue.wang@intel.com> wrote:
> > - For fully experimental libraries, we have a special so version:
> > https://git.dpdk.org/dpdk/tree/drivers/meson.build#n131
> >
> > This will apply to common drivers that will be 100% internal.
> > Not sure if this is an issue.
>
> This part should be fine, I want others to be aware of this.
I am not one of the ABI maintainers, but in my opinion it is OK
to have "pure internal" libs with version 0.x.
^ permalink raw reply [relevance 7%]
* Re: [dpdk-dev] [PATCH v5 1/1] eal: add internal ABI marking support
2020-04-25 6:10 7% ` Wang, Haiyue
@ 2020-04-25 14:21 4% ` David Marchand
2020-04-25 14:24 7% ` Thomas Monjalon
0 siblings, 1 reply; 200+ results
From: David Marchand @ 2020-04-25 14:21 UTC (permalink / raw)
To: Wang, Haiyue
Cc: dev, Thomas Monjalon, Richardson, Bruce, Yigit, Ferruh,
Neil Horman, Ray Kinsella
On Sat, Apr 25, 2020 at 8:10 AM Wang, Haiyue <haiyue.wang@intel.com> wrote:
>
> Hi David,
>
> Try to fix the issues you mentioned, except below, plan to
> another patch set, I need more time to test these adding.
Thanks for working on this topic.
>
> ====
>
> We are missing updates on devtools/check-abi-version.sh and
> devtools/update_version_map_abi.py.
Those two scripts can be updated later:
- I suspect the first one to be broken already,
- the 2nd one is for 20.11 when we will update all map files.
>
> More importantly on this file:
>
> - drivers/meson.build is not updated to check for internal symbols, see:
> https://git.dpdk.org/dpdk/tree/drivers/meson.build#n166
On this point, your series is almost good to go, I would just get rid
of the "experimental" mentions.
I will reply on the patch.
> - For fully experimental libraries, we have a special so version:
> https://git.dpdk.org/dpdk/tree/drivers/meson.build#n131
>
> This will apply to common drivers that will be 100% internal.
> Not sure if this is an issue.
This part should be fine, I want others to be aware of this.
--
David Marchand
^ permalink raw reply [relevance 4%]
* [dpdk-dev] [PATCH v7 5/6] devtools: exempt internal ABI checking
2020-04-25 10:56 5% ` [dpdk-dev] [PATCH v7 0/6] dpdk: introduce __rte_internal tag Haiyue Wang
` (3 preceding siblings ...)
2020-04-25 10:56 13% ` [dpdk-dev] [PATCH v7 4/6] devtools: ignore internal ABI check Haiyue Wang
@ 2020-04-25 10:56 7% ` Haiyue Wang
2020-04-25 14:34 4% ` David Marchand
2020-04-25 14:39 0% ` [dpdk-dev] [PATCH v7 0/6] dpdk: introduce __rte_internal tag David Marchand
5 siblings, 1 reply; 200+ results
From: Haiyue Wang @ 2020-04-25 10:56 UTC (permalink / raw)
To: dev, thomas, david.marchand, bruce.richardson, ferruh.yigit,
nhorman, mdr
Cc: Haiyue Wang
No need to restrict the ABI on symbols that are only used by core
libraries.
Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
---
devtools/check-symbol-change.sh | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/devtools/check-symbol-change.sh b/devtools/check-symbol-change.sh
index ed2178e36..7b6d5f40f 100755
--- a/devtools/check-symbol-change.sh
+++ b/devtools/check-symbol-change.sh
@@ -91,6 +91,13 @@ check_for_rule_violations()
if [ "$ar" = "add" ]
then
+ if [ "$secname" = "INTERNAL" ]
+ then
+ # these are absolved from any further checking
+ echo "Skipping symbol $symname in INTERNAL"
+ continue
+ fi
+
if [ "$secname" = "unknown" ]
then
# Just inform the user of this occurrence, but
@@ -148,6 +155,7 @@ check_for_rule_violations()
else
if ! grep -q "$mname $symname .* add" "$mapdb" && \
+ [ "$secname" != "INTERNAL" ] && \
[ "$secname" != "EXPERIMENTAL" ]
then
# Just inform users that non-experimenal
--
2.26.2
^ permalink raw reply [relevance 7%]
* [dpdk-dev] [PATCH v7 4/6] devtools: ignore internal ABI check
2020-04-25 10:56 5% ` [dpdk-dev] [PATCH v7 0/6] dpdk: introduce __rte_internal tag Haiyue Wang
` (2 preceding siblings ...)
2020-04-25 10:56 5% ` [dpdk-dev] [PATCH v7 3/6] mk: add internal tag check Haiyue Wang
@ 2020-04-25 10:56 13% ` Haiyue Wang
2020-04-25 10:56 7% ` [dpdk-dev] [PATCH v7 5/6] devtools: exempt internal ABI checking Haiyue Wang
2020-04-25 14:39 0% ` [dpdk-dev] [PATCH v7 0/6] dpdk: introduce __rte_internal tag David Marchand
5 siblings, 0 replies; 200+ results
From: Haiyue Wang @ 2020-04-25 10:56 UTC (permalink / raw)
To: dev, thomas, david.marchand, bruce.richardson, ferruh.yigit,
nhorman, mdr
Cc: Haiyue Wang
Ignore the internal version ABI check, this kind of ABI is used only
by drivers and libraries.
Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
---
devtools/libabigail.abignore | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/devtools/libabigail.abignore b/devtools/libabigail.abignore
index 1911890a7..986a52771 100644
--- a/devtools/libabigail.abignore
+++ b/devtools/libabigail.abignore
@@ -3,6 +3,11 @@
[suppress_variable]
symbol_version = EXPERIMENTAL
+[suppress_function]
+ symbol_version = INTERNAL
+[suppress_variable]
+ symbol_version = INTERNAL
+
; Explicit ignore for driver-only ABI
[suppress_type]
name = rte_cryptodev_ops
--
2.26.2
^ permalink raw reply [relevance 13%]
* [dpdk-dev] [PATCH v7 3/6] mk: add internal tag check
2020-04-25 10:56 5% ` [dpdk-dev] [PATCH v7 0/6] dpdk: introduce __rte_internal tag Haiyue Wang
2020-04-25 10:56 13% ` [dpdk-dev] [PATCH v7 1/6] eal: add internal ABI tag definition Haiyue Wang
2020-04-25 10:56 3% ` [dpdk-dev] [PATCH v7 2/6] build: enable internal API tag Haiyue Wang
@ 2020-04-25 10:56 5% ` Haiyue Wang
2020-04-25 14:34 3% ` David Marchand
2020-04-25 10:56 13% ` [dpdk-dev] [PATCH v7 4/6] devtools: ignore internal ABI check Haiyue Wang
` (2 subsequent siblings)
5 siblings, 1 reply; 200+ results
From: Haiyue Wang @ 2020-04-25 10:56 UTC (permalink / raw)
To: dev, thomas, david.marchand, bruce.richardson, ferruh.yigit,
nhorman, mdr
Cc: Haiyue Wang
Add checks during build to ensure that all symbols in the INTERNAL
version map section have __internal tags on their definitions, and
enable the warnings needed to announce their use.
Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
---
MAINTAINERS | 2 +-
...-experimental-syms.sh => check-symbols.sh} | 31 +++++++++++++++++++
buildtools/meson.build | 2 +-
mk/internal/rte.compile-pre.mk | 6 ++--
4 files changed, 36 insertions(+), 5 deletions(-)
rename buildtools/{check-experimental-syms.sh => check-symbols.sh} (61%)
diff --git a/MAINTAINERS b/MAINTAINERS
index a8d24e332..85298d426 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -153,7 +153,7 @@ F: devtools/libabigail.abignore
F: devtools/update-abi.sh
F: devtools/update_version_map_abi.py
F: devtools/validate-abi.sh
-F: buildtools/check-experimental-syms.sh
+F: buildtools/check-symbols.sh
F: buildtools/map-list-symbol.sh
Driver information
diff --git a/buildtools/check-experimental-syms.sh b/buildtools/check-symbols.sh
similarity index 61%
rename from buildtools/check-experimental-syms.sh
rename to buildtools/check-symbols.sh
index f3603e5ba..3df57c322 100755
--- a/buildtools/check-experimental-syms.sh
+++ b/buildtools/check-symbols.sh
@@ -54,4 +54,35 @@ do
}
done
+for SYM in `$LIST_SYMBOL -S INTERNAL $MAPFILE |cut -d ' ' -f 3`
+do
+ if grep -q "\.text.*[[:space:]]$SYM$" $DUMPFILE &&
+ ! grep -q "\.text\.internal.*[[:space:]]$SYM$" $DUMPFILE
+ then
+ cat >&2 <<- END_OF_MESSAGE
+ $SYM is not flagged as internal
+ but is listed in version map
+ Please add __rte_internal to the definition of $SYM
+ END_OF_MESSAGE
+ ret=1
+ fi
+done
+
+# Filter out symbols suffixed with a . for icc
+for SYM in `awk '{
+ if ($2 != "l" && $4 == ".text.internal" && !($NF ~ /\.$/)) {
+ print $NF
+ }
+}' $DUMPFILE`
+do
+ $LIST_SYMBOL -S INTERNAL -s $SYM -q $MAPFILE || {
+ cat >&2 <<- END_OF_MESSAGE
+ $SYM is flagged as internal
+ but is not listed in version map
+ Please add $SYM to the version map
+ END_OF_MESSAGE
+ ret=1
+ }
+done
+
exit $ret
diff --git a/buildtools/meson.build b/buildtools/meson.build
index 9812917e5..3e8d31b0c 100644
--- a/buildtools/meson.build
+++ b/buildtools/meson.build
@@ -6,7 +6,7 @@ subdir('pmdinfogen')
pkgconf = find_program('pkg-config', 'pkgconf', required: false)
pmdinfo = find_program('gen-pmdinfo-cfile.sh')
list_dir_globs = find_program('list-dir-globs.py')
-check_experimental_syms = find_program('check-experimental-syms.sh')
+check_experimental_syms = find_program('check-symbols.sh')
ldflags_ibverbs_static = find_program('options-ibverbs-static.sh')
# set up map-to-def script using python, either built-in or external
diff --git a/mk/internal/rte.compile-pre.mk b/mk/internal/rte.compile-pre.mk
index 82fe098f7..df05b5576 100644
--- a/mk/internal/rte.compile-pre.mk
+++ b/mk/internal/rte.compile-pre.mk
@@ -56,8 +56,8 @@ C_TO_O = $(CC) -Wp,-MD,$(call obj2dep,$(@)).tmp $(CPPFLAGS) $(CFLAGS) \
C_TO_O_STR = $(subst ','\'',$(C_TO_O)) #'# fix syntax highlight
C_TO_O_DISP = $(if $(V),"$(C_TO_O_STR)"," CC $(@)")
endif
-EXPERIMENTAL_CHECK = $(RTE_SDK)/buildtools/check-experimental-syms.sh
-CHECK_EXPERIMENTAL = $(EXPERIMENTAL_CHECK) $(SRCDIR)/$(EXPORT_MAP) $@
+CHECK_SYMBOLS_SCRIPT = $(RTE_SDK)/buildtools/check-symbols.sh
+CHECK_SYMBOLS = $(CHECK_SYMBOLS_SCRIPT) $(SRCDIR)/$(EXPORT_MAP) $@
PMDINFO_GEN = $(RTE_SDK_BIN)/app/dpdk-pmdinfogen $@ $@.pmd.c
PMDINFO_CC = $(CC) $(CPPFLAGS) $(CFLAGS) $(EXTRA_CFLAGS) -c -o $@.pmd.o $@.pmd.c
@@ -75,7 +75,7 @@ C_TO_O_DO = @set -e; \
echo $(C_TO_O_DISP); \
$(C_TO_O) && \
$(PMDINFO_TO_O) && \
- $(CHECK_EXPERIMENTAL) && \
+ $(CHECK_SYMBOLS) && \
echo $(C_TO_O_CMD) > $(call obj2cmd,$(@)) && \
sed 's,'$@':,dep_'$@' =,' $(call obj2dep,$(@)).tmp > $(call obj2dep,$(@)) && \
rm -f $(call obj2dep,$(@)).tmp
--
2.26.2
^ permalink raw reply [relevance 5%]
* [dpdk-dev] [PATCH v7 2/6] build: enable internal API tag
2020-04-25 10:56 5% ` [dpdk-dev] [PATCH v7 0/6] dpdk: introduce __rte_internal tag Haiyue Wang
2020-04-25 10:56 13% ` [dpdk-dev] [PATCH v7 1/6] eal: add internal ABI tag definition Haiyue Wang
@ 2020-04-25 10:56 3% ` Haiyue Wang
2020-04-25 10:56 5% ` [dpdk-dev] [PATCH v7 3/6] mk: add internal tag check Haiyue Wang
` (3 subsequent siblings)
5 siblings, 0 replies; 200+ results
From: Haiyue Wang @ 2020-04-25 10:56 UTC (permalink / raw)
To: dev, thomas, david.marchand, bruce.richardson, ferruh.yigit,
nhorman, mdr
Cc: Haiyue Wang
Allow the drivers and libraries to use the internal tag for marking
internal ABI symbols.
Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
---
drivers/meson.build | 5 ++++-
lib/meson.build | 5 ++++-
mk/target/generic/rte.vars.mk | 1 +
3 files changed, 9 insertions(+), 2 deletions(-)
diff --git a/drivers/meson.build b/drivers/meson.build
index 4d8f842ab..f3dd23dd4 100644
--- a/drivers/meson.build
+++ b/drivers/meson.build
@@ -20,7 +20,10 @@ dpdk_driver_classes = ['common',
disabled_drivers = run_command(list_dir_globs, get_option('disable_drivers'),
).stdout().split()
-default_cflags = machine_args + ['-DALLOW_EXPERIMENTAL_API']
+default_cflags = machine_args
+default_cflags += ['-DALLOW_EXPERIMENTAL_API']
+default_cflags += ['-DALLOW_INTERNAL_API']
+
if cc.has_argument('-Wno-format-truncation')
default_cflags += '-Wno-format-truncation'
endif
diff --git a/lib/meson.build b/lib/meson.build
index c28b8df83..8697941ae 100644
--- a/lib/meson.build
+++ b/lib/meson.build
@@ -38,7 +38,10 @@ if is_windows
libraries = ['kvargs','eal'] # only supported libraries for windows
endif
-default_cflags = machine_args + ['-DALLOW_EXPERIMENTAL_API']
+default_cflags = machine_args
+default_cflags += ['-DALLOW_EXPERIMENTAL_API']
+default_cflags += ['-DALLOW_INTERNAL_API']
+
if cc.has_argument('-Wno-format-truncation')
default_cflags += '-Wno-format-truncation'
endif
diff --git a/mk/target/generic/rte.vars.mk b/mk/target/generic/rte.vars.mk
index ec2672897..11b0418e5 100644
--- a/mk/target/generic/rte.vars.mk
+++ b/mk/target/generic/rte.vars.mk
@@ -106,6 +106,7 @@ ifeq ($(BUILDING_RTE_SDK),1)
# building sdk
CFLAGS += -include $(RTE_OUTPUT)/include/rte_config.h
CFLAGS += -DALLOW_EXPERIMENTAL_API
+CFLAGS += -DALLOW_INTERNAL_API
else
# if we are building an external application, include SDK's lib and
# includes too
--
2.26.2
^ permalink raw reply [relevance 3%]
* [dpdk-dev] [PATCH v7 1/6] eal: add internal ABI tag definition
2020-04-25 10:56 5% ` [dpdk-dev] [PATCH v7 0/6] dpdk: introduce __rte_internal tag Haiyue Wang
@ 2020-04-25 10:56 13% ` Haiyue Wang
2020-04-25 10:56 3% ` [dpdk-dev] [PATCH v7 2/6] build: enable internal API tag Haiyue Wang
` (4 subsequent siblings)
5 siblings, 0 replies; 200+ results
From: Haiyue Wang @ 2020-04-25 10:56 UTC (permalink / raw)
To: dev, thomas, david.marchand, bruce.richardson, ferruh.yigit,
nhorman, mdr
Cc: Haiyue Wang
Introduce the __rte_internal tag to mark internal ABI function which is
used only by the drivers or other libraries.
Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
---
lib/librte_eal/include/rte_compat.h | 13 +++++++++++++
1 file changed, 13 insertions(+)
diff --git a/lib/librte_eal/include/rte_compat.h b/lib/librte_eal/include/rte_compat.h
index 3eb33784b..4cd8f68d6 100644
--- a/lib/librte_eal/include/rte_compat.h
+++ b/lib/librte_eal/include/rte_compat.h
@@ -19,4 +19,17 @@ __attribute__((section(".text.experimental")))
#endif
+#ifndef ALLOW_INTERNAL_API
+
+#define __rte_internal \
+__attribute__((error("Symbol is not public ABI"), \
+section(".text.internal")))
+
+#else
+
+#define __rte_internal \
+__attribute__((section(".text.internal")))
+
+#endif
+
#endif /* _RTE_COMPAT_H_ */
--
2.26.2
^ permalink raw reply [relevance 13%]
* [dpdk-dev] [PATCH v7 0/6] dpdk: introduce __rte_internal tag
` (3 preceding siblings ...)
2020-04-25 6:04 5% ` [dpdk-dev] [PATCH v6 0/6] dpdk: introduce __rte_internal tag Haiyue Wang
@ 2020-04-25 10:56 5% ` Haiyue Wang
2020-04-25 10:56 13% ` [dpdk-dev] [PATCH v7 1/6] eal: add internal ABI tag definition Haiyue Wang
` (5 more replies)
4 siblings, 6 replies; 200+ results
From: Haiyue Wang @ 2020-04-25 10:56 UTC (permalink / raw)
To: dev, thomas, david.marchand, bruce.richardson, ferruh.yigit,
nhorman, mdr
Cc: Haiyue Wang
Move the internal function into INTERNAL session to avoid the ABI
checking, and it is only used for DPDK drivers or related library.
__rte_internal funA
INTERNAL {
global:
funA
};
v7: Fix the meson build error
v6: split into small patches, and add the missed handling.
v5: add the checkpatch for __rte_internal style
v4: add the ABI check suppression rules
v3: based on Neil's v2 patch https://patchwork.dpdk.org/cover/54771/
Use the ALLOW_INTERNAL_API to mark this new feature.
Haiyue Wang (6):
eal: add internal ABI tag definition
build: enable internal API tag
mk: add internal tag check
devtools: ignore internal ABI check
devtools: exempt internal ABI checking
devtools: enforce internal tag at the beginning
MAINTAINERS | 2 +-
...-experimental-syms.sh => check-symbols.sh} | 31 +++++++++++++++
buildtools/meson.build | 2 +-
devtools/check-symbol-change.sh | 8 ++++
devtools/checkpatches.sh | 39 +++++++++++++++++++
devtools/libabigail.abignore | 5 +++
drivers/meson.build | 5 ++-
lib/librte_eal/include/rte_compat.h | 13 +++++++
lib/meson.build | 5 ++-
mk/internal/rte.compile-pre.mk | 6 +--
mk/target/generic/rte.vars.mk | 1 +
11 files changed, 110 insertions(+), 7 deletions(-)
rename buildtools/{check-experimental-syms.sh => check-symbols.sh} (61%)
--
2.26.2
^ permalink raw reply [relevance 5%]
* Re: [dpdk-dev] [PATCH v2 1/4] ring: future proof flag settings
2020-04-24 19:02 0% ` Stephen Hemminger
@ 2020-04-25 9:20 0% ` Ananyev, Konstantin
0 siblings, 0 replies; 200+ results
From: Ananyev, Konstantin @ 2020-04-25 9:20 UTC (permalink / raw)
To: Stephen Hemminger, Honnappa Nagarahalli; +Cc: dev, nd
> -----Original Message-----
> From: dev <dev-bounces@dpdk.org> On Behalf Of Stephen Hemminger
> Sent: Friday, April 24, 2020 8:02 PM
> To: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>
> Cc: dev@dpdk.org; nd <nd@arm.com>
> Subject: Re: [dpdk-dev] [PATCH v2 1/4] ring: future proof flag settings
>
> On Fri, 24 Apr 2020 18:07:15 +0000
> Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com> wrote:
>
> > <snip>
> >
> > >
> > > All API's should check that they support the flag values passed.
> > > These checks ensure that the extra bits can safely be used without risk of ABI
> > > breakage.
> > >
> > > Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
> > > ---
> > > lib/librte_ring/rte_ring.c | 10 ++++++++++
> > > 1 file changed, 10 insertions(+)
> > >
> > > diff --git a/lib/librte_ring/rte_ring.c b/lib/librte_ring/rte_ring.c index
> > > ebe5ccf0de68..70685121581f 100644
> > > --- a/lib/librte_ring/rte_ring.c
> > > +++ b/lib/librte_ring/rte_ring.c
> > > @@ -42,6 +42,9 @@ static struct rte_tailq_elem rte_ring_tailq = { };
> > > EAL_REGISTER_TAILQ(rte_ring_tailq)
> > >
> > > +/* mask of all valid flag values to ring_create() */
> > > +#define RING_F_MASK 0x007F
> > Is it better to construct this using the actual flag #defines?
>
> sure, but it gets long
+1 to use public defines here.
^ permalink raw reply [relevance 0%]
* [dpdk-dev] [PATCH v6 5/6] devtools: exempt internal ABI checking
2020-04-25 6:04 5% ` [dpdk-dev] [PATCH v6 0/6] dpdk: introduce __rte_internal tag Haiyue Wang
` (2 preceding siblings ...)
2020-04-25 6:04 13% ` [dpdk-dev] [PATCH v6 4/6] devtools: ignore internal ABI check Haiyue Wang
@ 2020-04-25 6:04 7% ` Haiyue Wang
3 siblings, 0 replies; 200+ results
From: Haiyue Wang @ 2020-04-25 6:04 UTC (permalink / raw)
To: dev, thomas, david.marchand, bruce.richardson, ferruh.yigit,
nhorman, mdr
Cc: Haiyue Wang
No need to restrict the ABI on symbols that are only used by core
libraries.
Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
---
devtools/check-symbol-change.sh | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/devtools/check-symbol-change.sh b/devtools/check-symbol-change.sh
index ed2178e36..7b6d5f40f 100755
--- a/devtools/check-symbol-change.sh
+++ b/devtools/check-symbol-change.sh
@@ -91,6 +91,13 @@ check_for_rule_violations()
if [ "$ar" = "add" ]
then
+ if [ "$secname" = "INTERNAL" ]
+ then
+ # these are absolved from any further checking
+ echo "Skipping symbol $symname in INTERNAL"
+ continue
+ fi
+
if [ "$secname" = "unknown" ]
then
# Just inform the user of this occurrence, but
@@ -148,6 +155,7 @@ check_for_rule_violations()
else
if ! grep -q "$mname $symname .* add" "$mapdb" && \
+ [ "$secname" != "INTERNAL" ] && \
[ "$secname" != "EXPERIMENTAL" ]
then
# Just inform users that non-experimenal
--
2.26.2
^ permalink raw reply [relevance 7%]
* [dpdk-dev] [PATCH v6 4/6] devtools: ignore internal ABI check
2020-04-25 6:04 5% ` [dpdk-dev] [PATCH v6 0/6] dpdk: introduce __rte_internal tag Haiyue Wang
2020-04-25 6:04 13% ` [dpdk-dev] [PATCH v6 1/6] eal: add internal ABI tag definition Haiyue Wang
2020-04-25 6:04 3% ` [dpdk-dev] [PATCH v6 2/6] build: enable internal API tag Haiyue Wang
@ 2020-04-25 6:04 13% ` Haiyue Wang
2020-04-25 6:04 7% ` [dpdk-dev] [PATCH v6 5/6] devtools: exempt internal ABI checking Haiyue Wang
3 siblings, 0 replies; 200+ results
From: Haiyue Wang @ 2020-04-25 6:04 UTC (permalink / raw)
To: dev, thomas, david.marchand, bruce.richardson, ferruh.yigit,
nhorman, mdr
Cc: Haiyue Wang
Ignore the internal version ABI check, this kind of ABI is used only
by drivers and libraries.
Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
---
devtools/libabigail.abignore | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/devtools/libabigail.abignore b/devtools/libabigail.abignore
index 1911890a7..986a52771 100644
--- a/devtools/libabigail.abignore
+++ b/devtools/libabigail.abignore
@@ -3,6 +3,11 @@
[suppress_variable]
symbol_version = EXPERIMENTAL
+[suppress_function]
+ symbol_version = INTERNAL
+[suppress_variable]
+ symbol_version = INTERNAL
+
; Explicit ignore for driver-only ABI
[suppress_type]
name = rte_cryptodev_ops
--
2.26.2
^ permalink raw reply [relevance 13%]
* [dpdk-dev] [PATCH v6 2/6] build: enable internal API tag
2020-04-25 6:04 5% ` [dpdk-dev] [PATCH v6 0/6] dpdk: introduce __rte_internal tag Haiyue Wang
2020-04-25 6:04 13% ` [dpdk-dev] [PATCH v6 1/6] eal: add internal ABI tag definition Haiyue Wang
@ 2020-04-25 6:04 3% ` Haiyue Wang
2020-04-25 6:04 13% ` [dpdk-dev] [PATCH v6 4/6] devtools: ignore internal ABI check Haiyue Wang
2020-04-25 6:04 7% ` [dpdk-dev] [PATCH v6 5/6] devtools: exempt internal ABI checking Haiyue Wang
3 siblings, 0 replies; 200+ results
From: Haiyue Wang @ 2020-04-25 6:04 UTC (permalink / raw)
To: dev, thomas, david.marchand, bruce.richardson, ferruh.yigit,
nhorman, mdr
Cc: Haiyue Wang
Allow the drivers and libraries to use the internal tag for marking
internal ABI symbols.
Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
---
drivers/meson.build | 5 ++++-
lib/meson.build | 5 ++++-
mk/target/generic/rte.vars.mk | 1 +
3 files changed, 9 insertions(+), 2 deletions(-)
diff --git a/drivers/meson.build b/drivers/meson.build
index 4d8f842ab..f3dd23dd4 100644
--- a/drivers/meson.build
+++ b/drivers/meson.build
@@ -20,7 +20,10 @@ dpdk_driver_classes = ['common',
disabled_drivers = run_command(list_dir_globs, get_option('disable_drivers'),
).stdout().split()
-default_cflags = machine_args + ['-DALLOW_EXPERIMENTAL_API']
+default_cflags = machine_args
+default_cflags += ['-DALLOW_EXPERIMENTAL_API']
+default_cflags += ['-DALLOW_INTERNAL_API']
+
if cc.has_argument('-Wno-format-truncation')
default_cflags += '-Wno-format-truncation'
endif
diff --git a/lib/meson.build b/lib/meson.build
index c28b8df83..8697941ae 100644
--- a/lib/meson.build
+++ b/lib/meson.build
@@ -38,7 +38,10 @@ if is_windows
libraries = ['kvargs','eal'] # only supported libraries for windows
endif
-default_cflags = machine_args + ['-DALLOW_EXPERIMENTAL_API']
+default_cflags = machine_args
+default_cflags += ['-DALLOW_EXPERIMENTAL_API']
+default_cflags += ['-DALLOW_INTERNAL_API']
+
if cc.has_argument('-Wno-format-truncation')
default_cflags += '-Wno-format-truncation'
endif
diff --git a/mk/target/generic/rte.vars.mk b/mk/target/generic/rte.vars.mk
index ec2672897..11b0418e5 100644
--- a/mk/target/generic/rte.vars.mk
+++ b/mk/target/generic/rte.vars.mk
@@ -106,6 +106,7 @@ ifeq ($(BUILDING_RTE_SDK),1)
# building sdk
CFLAGS += -include $(RTE_OUTPUT)/include/rte_config.h
CFLAGS += -DALLOW_EXPERIMENTAL_API
+CFLAGS += -DALLOW_INTERNAL_API
else
# if we are building an external application, include SDK's lib and
# includes too
--
2.26.2
^ permalink raw reply [relevance 3%]
* [dpdk-dev] [PATCH v6 1/6] eal: add internal ABI tag definition
2020-04-25 6:04 5% ` [dpdk-dev] [PATCH v6 0/6] dpdk: introduce __rte_internal tag Haiyue Wang
@ 2020-04-25 6:04 13% ` Haiyue Wang
2020-04-25 6:04 3% ` [dpdk-dev] [PATCH v6 2/6] build: enable internal API tag Haiyue Wang
` (2 subsequent siblings)
3 siblings, 0 replies; 200+ results
From: Haiyue Wang @ 2020-04-25 6:04 UTC (permalink / raw)
To: dev, thomas, david.marchand, bruce.richardson, ferruh.yigit,
nhorman, mdr
Cc: Haiyue Wang
Introduce the __rte_internal tag to mark internal ABI function which is
used only by the drivers or other libraries.
Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
---
lib/librte_eal/include/rte_compat.h | 13 +++++++++++++
1 file changed, 13 insertions(+)
diff --git a/lib/librte_eal/include/rte_compat.h b/lib/librte_eal/include/rte_compat.h
index 3eb33784b..4cd8f68d6 100644
--- a/lib/librte_eal/include/rte_compat.h
+++ b/lib/librte_eal/include/rte_compat.h
@@ -19,4 +19,17 @@ __attribute__((section(".text.experimental")))
#endif
+#ifndef ALLOW_INTERNAL_API
+
+#define __rte_internal \
+__attribute__((error("Symbol is not public ABI"), \
+section(".text.internal")))
+
+#else
+
+#define __rte_internal \
+__attribute__((section(".text.internal")))
+
+#endif
+
#endif /* _RTE_COMPAT_H_ */
--
2.26.2
^ permalink raw reply [relevance 13%]
* [dpdk-dev] [PATCH v6 0/6] dpdk: introduce __rte_internal tag
` (2 preceding siblings ...)
2020-04-23 3:19 4% ` [dpdk-dev] [PATCH v5 0/1] dpdk: introduce __rte_internal tag Haiyue Wang
@ 2020-04-25 6:04 5% ` Haiyue Wang
2020-04-25 6:04 13% ` [dpdk-dev] [PATCH v6 1/6] eal: add internal ABI tag definition Haiyue Wang
` (3 more replies)
2020-04-25 10:56 5% ` [dpdk-dev] [PATCH v7 0/6] dpdk: introduce __rte_internal tag Haiyue Wang
4 siblings, 4 replies; 200+ results
From: Haiyue Wang @ 2020-04-25 6:04 UTC (permalink / raw)
To: dev, thomas, david.marchand, bruce.richardson, ferruh.yigit,
nhorman, mdr
Cc: Haiyue Wang
Move the internal function into INTERNAL session to avoid the ABI
checking, and it is only used for DPDK drivers or related library.
__rte_internal funA
INTERNAL {
global:
funA
};
v6: split into small patches, and add the missed handling.
v5: add the checkpatch for __rte_internal style
v4: add the ABI check suppression rules
v3: based on Neil's v2 patch https://patchwork.dpdk.org/cover/54771/
Use the ALLOW_INTERNAL_API to mark this new feature.
Haiyue Wang (6):
eal: add internal ABI tag definition
build: enable internal API tag
mk: add internal tag check
devtools: ignore internal ABI check
devtools: exempt internal ABI checking
devtools: enforce internal tag at the beginning
...-experimental-syms.sh => check-symbols.sh} | 31 +++++++++++++++
devtools/check-symbol-change.sh | 8 ++++
devtools/checkpatches.sh | 39 +++++++++++++++++++
devtools/libabigail.abignore | 5 +++
drivers/meson.build | 5 ++-
lib/librte_eal/include/rte_compat.h | 13 +++++++
lib/meson.build | 5 ++-
mk/internal/rte.compile-pre.mk | 6 +--
mk/target/generic/rte.vars.mk | 1 +
9 files changed, 108 insertions(+), 5 deletions(-)
rename buildtools/{check-experimental-syms.sh => check-symbols.sh} (61%)
--
2.26.2
^ permalink raw reply [relevance 5%]
* Re: [dpdk-dev] [PATCH v5 1/1] eal: add internal ABI marking support
2020-04-24 14:52 7% ` David Marchand
@ 2020-04-25 6:10 7% ` Wang, Haiyue
2020-04-25 14:21 4% ` David Marchand
0 siblings, 1 reply; 200+ results
From: Wang, Haiyue @ 2020-04-25 6:10 UTC (permalink / raw)
To: David Marchand
Cc: dev, Thomas Monjalon, Richardson, Bruce, Yigit, Ferruh,
Neil Horman, Ray Kinsella
Hi David,
Try to fix the issues you mentioned, except below, plan to
another patch set, I need more time to test these adding.
====
We are missing updates on devtools/check-abi-version.sh and
devtools/update_version_map_abi.py.
More importantly on this file:
- drivers/meson.build is not updated to check for internal symbols, see:
https://git.dpdk.org/dpdk/tree/drivers/meson.build#n166
- For fully experimental libraries, we have a special so version:
https://git.dpdk.org/dpdk/tree/drivers/meson.build#n131
This will apply to common drivers that will be 100% internal.
Not sure if this is an issue.
BR,
Haiyue
> -----Original Message-----
> From: David Marchand <david.marchand@redhat.com>
> Sent: Friday, April 24, 2020 22:53
> To: Wang, Haiyue <haiyue.wang@intel.com>
> Cc: dev <dev@dpdk.org>; Thomas Monjalon <thomas@monjalon.net>; Richardson, Bruce
> <bruce.richardson@intel.com>; Yigit, Ferruh <ferruh.yigit@intel.com>; Neil Horman
> <nhorman@tuxdriver.com>; Ray Kinsella <mdr@ashroe.eu>
> Subject: Re: [PATCH v5 1/1] eal: add internal ABI marking support
>
> On Thu, Apr 23, 2020 at 5:25 AM Haiyue Wang <haiyue.wang@intel.com> wrote:
> >
> > Introduce __rte_internal tag to mark internal ABI function which is used
> > by the drivers or other libraries.
> >
> > Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
> > ---
> > buildtools/check-internal-syms.sh | 57 +++++++++++++++++++++++++++++
> > devtools/check-symbol-change.sh | 7 ++++
> > devtools/checkpatches.sh | 38 +++++++++++++++++++
> > devtools/libabigail.abignore | 5 +++
> > drivers/meson.build | 2 +-
> > lib/librte_eal/include/rte_compat.h | 13 +++++++
> > lib/meson.build | 2 +-
> > mk/internal/rte.compile-pre.mk | 3 ++
> > mk/target/generic/rte.vars.mk | 1 +
> > 9 files changed, 126 insertions(+), 2 deletions(-)
> > create mode 100755 buildtools/check-internal-syms.sh
>
> We can have a single script (let's say devtools/check-symbols.sh) that
> contains both experimental and internal symbols.
>
>
> > diff --git a/devtools/check-symbol-change.sh b/devtools/check-symbol-change.sh
> > index ed2178e36..978979077 100755
> > --- a/devtools/check-symbol-change.sh
> > +++ b/devtools/check-symbol-change.sh
> > @@ -91,6 +91,13 @@ check_for_rule_violations()
> > if [ "$ar" = "add" ]
> > then
> >
> > + if [ "$secname" = "INTERNAL" ]
> > + then
> > + # these are absolved from any further checking
> > + echo "Skipping symbol $symname in INTERNAL"
> > + continue
> > + fi
> > +
> > if [ "$secname" = "unknown" ]
> > then
> > # Just inform the user of this occurrence, but
>
> This only handles symbol additions to the INTERNAL section.
> Other cases like moving or removing a INTERNAL symbol are not handled.
>
>
> > diff --git a/devtools/checkpatches.sh b/devtools/checkpatches.sh
> > index c30ce64cc..2df8d7f2c 100755
> > --- a/devtools/checkpatches.sh
> > +++ b/devtools/checkpatches.sh
> > @@ -111,6 +111,36 @@ check_experimental_tags() { # <patch>
> > return $res
> > }
> >
> > +check_internal_tags() { # <patch>
> > + res=0
> > +
> > + cat "$1" |awk '
> > + BEGIN {
> > + current_file = "";
> > + ret = 0;
> > + }
> > + /^+++ b\// {
> > + current_file = $2;
> > + }
> > + /^+.*__rte_internal/ {
> > + if (current_file ~ ".c$" ) {
> > + print "Please only put __rte_internal tags in " \
> > + "headers ("current_file")";
> > + ret = 1;
> > + }
> > + if ($1 != "+__rte_internal" || $2 != "") {
> > + print "__rte_internal must appear alone on the line" \
> > + " immediately preceding the return type of a function."
> > + ret = 1;
> > + }
> > + }
> > + END {
> > + exit ret;
> > + }' || res=1
> > +
> > + return $res
> > +}
> > +
> > number=0
> > range='origin/master..'
> > quiet=false
> > @@ -194,6 +224,14 @@ check () { # <patch> <commit> <title>
> > ret=1
> > fi
> >
> > + ! $verbose || printf '\nChecking __rte_internal tags:\n'
> > + report=$(check_internal_tags "$tmpinput")
> > + if [ $? -ne 0 ] ; then
> > + $headline_printed || print_headline "$3"
> > + printf '%s\n' "$report"
> > + ret=1
> > + fi
> > +
> > if [ "$tmpinput" != "$1" ]; then
> > rm -f "$tmpinput"
> > trap - INT
> > diff --git a/devtools/libabigail.abignore b/devtools/libabigail.abignore
> > index cd86d89ca..3e8e2ea74 100644
> > --- a/devtools/libabigail.abignore
> > +++ b/devtools/libabigail.abignore
> > @@ -3,6 +3,11 @@
> > [suppress_variable]
> > symbol_version = EXPERIMENTAL
> >
> > +[suppress_function]
> > + symbol_version = INTERNAL
> > +[suppress_variable]
> > + symbol_version = INTERNAL
> > +
> > ; Explicit ignore for driver-only ABI
> > [suppress_type]
> > name = rte_cryptodev_ops
> > diff --git a/drivers/meson.build b/drivers/meson.build
> > index 4d8f842ab..cac07161f 100644
> > --- a/drivers/meson.build
> > +++ b/drivers/meson.build
> > @@ -20,7 +20,7 @@ dpdk_driver_classes = ['common',
> > disabled_drivers = run_command(list_dir_globs, get_option('disable_drivers'),
> > ).stdout().split()
> >
> > -default_cflags = machine_args + ['-DALLOW_EXPERIMENTAL_API']
> > +default_cflags = machine_args + ['-DALLOW_EXPERIMENTAL_API'] + ['-DALLOW_INTERNAL_API']
>
> Nit:
> default_cflags = machine_args
> default_cflags += ['-DALLOW_EXPERIMENTAL_API']
> default_cflags += ['-DALLOW_INTERNAL_API']
>
> > if cc.has_argument('-Wno-format-truncation')
> > default_cflags += '-Wno-format-truncation'
> > endif
>
> More importantly on this file:
>
> - drivers/meson.build is not updated to check for internal symbols, see:
> https://git.dpdk.org/dpdk/tree/drivers/meson.build#n166
>
>
> - For fully experimental libraries, we have a special so version:
> https://git.dpdk.org/dpdk/tree/drivers/meson.build#n131
>
> This will apply to common drivers that will be 100% internal.
> Not sure if this is an issue.
>
>
> > diff --git a/lib/librte_eal/include/rte_compat.h b/lib/librte_eal/include/rte_compat.h
> > index 3eb33784b..4cd8f68d6 100644
> > --- a/lib/librte_eal/include/rte_compat.h
> > +++ b/lib/librte_eal/include/rte_compat.h
> > @@ -19,4 +19,17 @@ __attribute__((section(".text.experimental")))
> >
> > #endif
> >
> > +#ifndef ALLOW_INTERNAL_API
> > +
> > +#define __rte_internal \
> > +__attribute__((error("Symbol is not public ABI"), \
> > +section(".text.internal")))
> > +
> > +#else
> > +
> > +#define __rte_internal \
> > +__attribute__((section(".text.internal")))
> > +
> > +#endif
> > +
> > #endif /* _RTE_COMPAT_H_ */
> > diff --git a/lib/meson.build b/lib/meson.build
> > index c28b8df83..4d2f90d6a 100644
> > --- a/lib/meson.build
> > +++ b/lib/meson.build
> > @@ -38,7 +38,7 @@ if is_windows
> > libraries = ['kvargs','eal'] # only supported libraries for windows
> > endif
> >
> > -default_cflags = machine_args + ['-DALLOW_EXPERIMENTAL_API']
> > +default_cflags = machine_args + ['-DALLOW_EXPERIMENTAL_API'] + ['-DALLOW_INTERNAL_API']
>
> Same comments as for drivers/meson.build.
>
>
> > diff --git a/mk/internal/rte.compile-pre.mk b/mk/internal/rte.compile-pre.mk
> > index 82fe098f7..0369786a5 100644
> > --- a/mk/internal/rte.compile-pre.mk
> > +++ b/mk/internal/rte.compile-pre.mk
> > @@ -58,6 +58,8 @@ C_TO_O_DISP = $(if $(V),"$(C_TO_O_STR)"," CC $(@)")
> > endif
> > EXPERIMENTAL_CHECK = $(RTE_SDK)/buildtools/check-experimental-syms.sh
> > CHECK_EXPERIMENTAL = $(EXPERIMENTAL_CHECK) $(SRCDIR)/$(EXPORT_MAP) $@
> > +INTERNAL_CHECK = $(RTE_SDK)/buildtools/check-internal-syms.sh
> > +CHECK_INTERNAL = $(INTERNAL_CHECK) $(SRCDIR)/$(EXPORT_MAP) $@
>
> With a single script, we can go with:
> CHECK_SYMBOLS_SCRIPT = $(RTE_SDK)/buildtools/check-symbols.sh
> CHECK_SYMBOLS = $(CHECK_SYMBOLS_SCRIPT) $(SRCDIR)/$(EXPORT_MAP) $@
>
> >
> > PMDINFO_GEN = $(RTE_SDK_BIN)/app/dpdk-pmdinfogen $@ $@.pmd.c
> > PMDINFO_CC = $(CC) $(CPPFLAGS) $(CFLAGS) $(EXTRA_CFLAGS) -c -o $@.pmd.o $@.pmd.c
> > @@ -76,6 +78,7 @@ C_TO_O_DO = @set -e; \
> > $(C_TO_O) && \
> > $(PMDINFO_TO_O) && \
> > $(CHECK_EXPERIMENTAL) && \
> > + $(CHECK_INTERNAL) && \
>
> + $(CHECK_SYMBOLS) && \
>
> > echo $(C_TO_O_CMD) > $(call obj2cmd,$(@)) && \
> > sed 's,'$@':,dep_'$@' =,' $(call obj2dep,$(@)).tmp > $(call obj2dep,$(@)) && \
> > rm -f $(call obj2dep,$(@)).tmp
> > diff --git a/mk/target/generic/rte.vars.mk b/mk/target/generic/rte.vars.mk
> > index ec2672897..11b0418e5 100644
> > --- a/mk/target/generic/rte.vars.mk
> > +++ b/mk/target/generic/rte.vars.mk
> > @@ -106,6 +106,7 @@ ifeq ($(BUILDING_RTE_SDK),1)
> > # building sdk
> > CFLAGS += -include $(RTE_OUTPUT)/include/rte_config.h
> > CFLAGS += -DALLOW_EXPERIMENTAL_API
> > +CFLAGS += -DALLOW_INTERNAL_API
> > else
> > # if we are building an external application, include SDK's lib and
> > # includes too
> > --
> > 2.26.2
> >
>
> We are missing updates on devtools/check-abi-version.sh and
> devtools/update_version_map_abi.py.
>
>
> --
> David Marchand
^ permalink raw reply [relevance 7%]
* Re: [dpdk-dev] [PATCH v2 1/4] ring: future proof flag settings
2020-04-24 18:07 0% ` Honnappa Nagarahalli
@ 2020-04-24 19:02 0% ` Stephen Hemminger
2020-04-25 9:20 0% ` Ananyev, Konstantin
0 siblings, 1 reply; 200+ results
From: Stephen Hemminger @ 2020-04-24 19:02 UTC (permalink / raw)
To: Honnappa Nagarahalli; +Cc: dev, nd
On Fri, 24 Apr 2020 18:07:15 +0000
Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com> wrote:
> <snip>
>
> >
> > All API's should check that they support the flag values passed.
> > These checks ensure that the extra bits can safely be used without risk of ABI
> > breakage.
> >
> > Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
> > ---
> > lib/librte_ring/rte_ring.c | 10 ++++++++++
> > 1 file changed, 10 insertions(+)
> >
> > diff --git a/lib/librte_ring/rte_ring.c b/lib/librte_ring/rte_ring.c index
> > ebe5ccf0de68..70685121581f 100644
> > --- a/lib/librte_ring/rte_ring.c
> > +++ b/lib/librte_ring/rte_ring.c
> > @@ -42,6 +42,9 @@ static struct rte_tailq_elem rte_ring_tailq = { };
> > EAL_REGISTER_TAILQ(rte_ring_tailq)
> >
> > +/* mask of all valid flag values to ring_create() */
> > +#define RING_F_MASK 0x007F
> Is it better to construct this using the actual flag #defines?
sure, but it gets long
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH v2 1/4] ring: future proof flag settings
2020-04-24 17:15 3% ` [dpdk-dev] [PATCH v2 1/4] ring: future proof flag settings Stephen Hemminger
@ 2020-04-24 18:07 0% ` Honnappa Nagarahalli
2020-04-24 19:02 0% ` Stephen Hemminger
0 siblings, 1 reply; 200+ results
From: Honnappa Nagarahalli @ 2020-04-24 18:07 UTC (permalink / raw)
To: Stephen Hemminger, dev; +Cc: nd, Honnappa Nagarahalli, nd
<snip>
>
> All API's should check that they support the flag values passed.
> These checks ensure that the extra bits can safely be used without risk of ABI
> breakage.
>
> Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
> ---
> lib/librte_ring/rte_ring.c | 10 ++++++++++
> 1 file changed, 10 insertions(+)
>
> diff --git a/lib/librte_ring/rte_ring.c b/lib/librte_ring/rte_ring.c index
> ebe5ccf0de68..70685121581f 100644
> --- a/lib/librte_ring/rte_ring.c
> +++ b/lib/librte_ring/rte_ring.c
> @@ -42,6 +42,9 @@ static struct rte_tailq_elem rte_ring_tailq = { };
> EAL_REGISTER_TAILQ(rte_ring_tailq)
>
> +/* mask of all valid flag values to ring_create() */
> +#define RING_F_MASK 0x007F
Is it better to construct this using the actual flag #defines?
> +
> /* true if x is a power of 2 */
> #define POWEROF2(x) ((((x)-1) & (x)) == 0)
>
> @@ -197,6 +200,13 @@ rte_ring_init(struct rte_ring *r, const char *name,
> unsigned count,
> RTE_BUILD_BUG_ON(offsetof(struct rte_ring_headtail, tail) !=
> offsetof(struct rte_ring_rts_headtail, tail.val.pos));
>
> + /* future proof flags, only allow supported values */
> + if (flags & ~RING_F_MASK) {
> + RTE_LOG(ERR, RING,
> + "Unsupported flags requested %#x\n", flags);
> + return -EINVAL;
> + }
> +
> /* init the ring structure */
> memset(r, 0, sizeof(*r));
> ret = strlcpy(r->name, name, sizeof(r->name));
> --
> 2.20.1
^ permalink raw reply [relevance 0%]
* [dpdk-dev] [PATCH v2 4/4] cfgfile: check flags value
2020-04-24 17:15 3% ` [dpdk-dev] [PATCH v2 0/4] enforce checking on flag values " Stephen Hemminger
` (2 preceding siblings ...)
2020-04-24 17:15 3% ` [dpdk-dev] [PATCH v2 3/4] stack: " Stephen Hemminger
@ 2020-04-24 17:15 3% ` Stephen Hemminger
3 siblings, 0 replies; 200+ results
From: Stephen Hemminger @ 2020-04-24 17:15 UTC (permalink / raw)
To: dev; +Cc: Stephen Hemminger
All API's should check that they support the flag values
passed. If an application passes an invalid flag it could
cause problems in later ABI.
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
---
lib/librte_cfgfile/rte_cfgfile.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/lib/librte_cfgfile/rte_cfgfile.c b/lib/librte_cfgfile/rte_cfgfile.c
index 9049fd9c2319..714717dd9007 100644
--- a/lib/librte_cfgfile/rte_cfgfile.c
+++ b/lib/librte_cfgfile/rte_cfgfile.c
@@ -272,6 +272,10 @@ rte_cfgfile_create(int flags)
int i;
struct rte_cfgfile *cfg;
+ /* future proof flags usage */
+ if (flags & ~(CFG_FLAG_GLOBAL_SECTION | CFG_FLAG_EMPTY_VALUES))
+ return NULL;
+
cfg = malloc(sizeof(*cfg));
if (cfg == NULL)
--
2.20.1
^ permalink raw reply [relevance 3%]
* [dpdk-dev] [PATCH v2 3/4] stack: check flags on creation
2020-04-24 17:15 3% ` [dpdk-dev] [PATCH v2 0/4] enforce checking on flag values " Stephen Hemminger
2020-04-24 17:15 3% ` [dpdk-dev] [PATCH v2 1/4] ring: future proof flag settings Stephen Hemminger
2020-04-24 17:15 3% ` [dpdk-dev] [PATCH v2 2/4] hash: check flags on creation Stephen Hemminger
@ 2020-04-24 17:15 3% ` Stephen Hemminger
2020-04-24 17:15 3% ` [dpdk-dev] [PATCH v2 4/4] cfgfile: check flags value Stephen Hemminger
3 siblings, 0 replies; 200+ results
From: Stephen Hemminger @ 2020-04-24 17:15 UTC (permalink / raw)
To: dev; +Cc: Stephen Hemminger
All API's should check that they support the flag values
passed. If an application passes an invalid flag it could
cause problems in later ABI.
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
---
lib/librte_stack/rte_stack.c | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/lib/librte_stack/rte_stack.c b/lib/librte_stack/rte_stack.c
index d19824f00439..e58fa545fca4 100644
--- a/lib/librte_stack/rte_stack.c
+++ b/lib/librte_stack/rte_stack.c
@@ -59,6 +59,11 @@ rte_stack_create(const char *name, unsigned int count, int socket_id,
unsigned int sz;
int ret;
+ if (flags & ~(RTE_STACK_F_LF)) {
+ STACK_LOG_ERR("Unsupported stack flags %#x\n", flags);
+ return NULL;
+ }
+
#ifdef RTE_ARCH_64
RTE_BUILD_BUG_ON(sizeof(struct rte_stack_lf_head) != 16);
#else
--
2.20.1
^ permalink raw reply [relevance 3%]
* [dpdk-dev] [PATCH v2 2/4] hash: check flags on creation
2020-04-24 17:15 3% ` [dpdk-dev] [PATCH v2 0/4] enforce checking on flag values " Stephen Hemminger
2020-04-24 17:15 3% ` [dpdk-dev] [PATCH v2 1/4] ring: future proof flag settings Stephen Hemminger
@ 2020-04-24 17:15 3% ` Stephen Hemminger
2020-04-24 17:15 3% ` [dpdk-dev] [PATCH v2 3/4] stack: " Stephen Hemminger
2020-04-24 17:15 3% ` [dpdk-dev] [PATCH v2 4/4] cfgfile: check flags value Stephen Hemminger
3 siblings, 0 replies; 200+ results
From: Stephen Hemminger @ 2020-04-24 17:15 UTC (permalink / raw)
To: dev; +Cc: Stephen Hemminger
All API's should check that they support the flag values
passed. If an application passes an invalid flag it could
cause problems in later ABI.
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
---
lib/librte_hash/rte_cuckoo_hash.c | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/lib/librte_hash/rte_cuckoo_hash.c b/lib/librte_hash/rte_cuckoo_hash.c
index 6af8ca42e94f..ab9cd91b4955 100644
--- a/lib/librte_hash/rte_cuckoo_hash.c
+++ b/lib/librte_hash/rte_cuckoo_hash.c
@@ -32,6 +32,9 @@
#include "rte_hash.h"
#include "rte_cuckoo_hash.h"
+/* Mask of all flags supported by this version */
+#define RTE_HASH_EXTRA_FLAGS_MASK 0x2f
+
#define FOR_EACH_BUCKET(CURRENT_BKT, START_BUCKET) \
for (CURRENT_BKT = START_BUCKET; \
CURRENT_BKT != NULL; \
@@ -164,6 +167,12 @@ rte_hash_create(const struct rte_hash_parameters *params)
return NULL;
}
+ if (params->extra_flag & ~RTE_HASH_EXTRA_FLAGS_MASK) {
+ rte_errno = EINVAL;
+ RTE_LOG(ERR, HASH, "rte_hash_create: unsupported extra flags\n");
+ return NULL;
+ }
+
/* Validate correct usage of extra options */
if ((params->extra_flag & RTE_HASH_EXTRA_FLAGS_RW_CONCURRENCY) &&
(params->extra_flag & RTE_HASH_EXTRA_FLAGS_RW_CONCURRENCY_LF)) {
--
2.20.1
^ permalink raw reply [relevance 3%]
* [dpdk-dev] [PATCH v2 1/4] ring: future proof flag settings
2020-04-24 17:15 3% ` [dpdk-dev] [PATCH v2 0/4] enforce checking on flag values " Stephen Hemminger
@ 2020-04-24 17:15 3% ` Stephen Hemminger
2020-04-24 18:07 0% ` Honnappa Nagarahalli
2020-04-24 17:15 3% ` [dpdk-dev] [PATCH v2 2/4] hash: check flags on creation Stephen Hemminger
` (2 subsequent siblings)
3 siblings, 1 reply; 200+ results
From: Stephen Hemminger @ 2020-04-24 17:15 UTC (permalink / raw)
To: dev; +Cc: Stephen Hemminger
All API's should check that they support the flag values passed.
These checks ensure that the extra bits can safely be used
without risk of ABI breakage.
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
---
lib/librte_ring/rte_ring.c | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/lib/librte_ring/rte_ring.c b/lib/librte_ring/rte_ring.c
index ebe5ccf0de68..70685121581f 100644
--- a/lib/librte_ring/rte_ring.c
+++ b/lib/librte_ring/rte_ring.c
@@ -42,6 +42,9 @@ static struct rte_tailq_elem rte_ring_tailq = {
};
EAL_REGISTER_TAILQ(rte_ring_tailq)
+/* mask of all valid flag values to ring_create() */
+#define RING_F_MASK 0x007F
+
/* true if x is a power of 2 */
#define POWEROF2(x) ((((x)-1) & (x)) == 0)
@@ -197,6 +200,13 @@ rte_ring_init(struct rte_ring *r, const char *name, unsigned count,
RTE_BUILD_BUG_ON(offsetof(struct rte_ring_headtail, tail) !=
offsetof(struct rte_ring_rts_headtail, tail.val.pos));
+ /* future proof flags, only allow supported values */
+ if (flags & ~RING_F_MASK) {
+ RTE_LOG(ERR, RING,
+ "Unsupported flags requested %#x\n", flags);
+ return -EINVAL;
+ }
+
/* init the ring structure */
memset(r, 0, sizeof(*r));
ret = strlcpy(r->name, name, sizeof(r->name));
--
2.20.1
^ permalink raw reply [relevance 3%]
* [dpdk-dev] [PATCH v2 0/4] enforce checking on flag values in API's
@ 2020-04-24 17:15 3% ` Stephen Hemminger
2020-04-24 17:15 3% ` [dpdk-dev] [PATCH v2 1/4] ring: future proof flag settings Stephen Hemminger
` (3 more replies)
2020-04-27 23:16 3% ` [dpdk-dev] [PATCH v3 0/4] Enforce checking on flag values in API's Stephen Hemminger
1 sibling, 4 replies; 200+ results
From: Stephen Hemminger @ 2020-04-24 17:15 UTC (permalink / raw)
To: dev; +Cc: Stephen Hemminger
Since the DPDK api's are lazy about checking for flag values
it means the flags can never be extended in future releases
without causing ABI breakage. This means we end up doing unnecessary
symbol versioning just to work around applications that might
pass in naughty bits.
This is the DPDK analog of the Linux kernel openat() problem.
Openat api was added but since kernel did not check flags it
ended up that another syscall openat2() was necessary before
the flags could be used.
v2 - rebase and fix checkpatch warnings from RFC
Stephen Hemminger (4):
ring: future proof flag settings
hash: check flags on creation
stack: check flags on creation
cfgfile: check flags value
lib/librte_cfgfile/rte_cfgfile.c | 4 ++++
lib/librte_hash/rte_cuckoo_hash.c | 9 +++++++++
lib/librte_ring/rte_ring.c | 10 ++++++++++
lib/librte_stack/rte_stack.c | 5 +++++
4 files changed, 28 insertions(+)
--
2.20.1
^ permalink raw reply [relevance 3%]
* Re: [dpdk-dev] [PATCH v4 02/18] telemetry: move code to metrics for later reuse
2020-04-24 15:29 3% ` Stephen Hemminger
@ 2020-04-24 15:49 0% ` Bruce Richardson
0 siblings, 0 replies; 200+ results
From: Bruce Richardson @ 2020-04-24 15:49 UTC (permalink / raw)
To: Stephen Hemminger
Cc: Ciara Power, dev, kevin.laatz, reshma.pattan, jerinjacobk,
david.marchand, keith.wiles, mb, thomas
On Fri, Apr 24, 2020 at 08:29:53AM -0700, Stephen Hemminger wrote:
> On Fri, 24 Apr 2020 13:41:43 +0100
> Ciara Power <ciara.power@intel.com> wrote:
>
> > This commit moves some of the telemetry library code to a new file in
> > the metrics library. No modifications are made to the moved code,
> > except what is needed to allow it to compile and run. The additional
> > code in metrics is built only when the Jansson library is present.
> > Telemetry functions as normal, using the functions from the
> > metrics_telemetry file. This move will enable code be reused by the new
> > version of telemetry in a later commit, to support backward
> > compatibility with the existing telemetry usage.
> >
> > Signed-off-by: Ciara Power <ciara.power@intel.com>
>
>
> Minor comments, none of these are show stoppers.
>
>
> > diff --git a/lib/librte_metrics/rte_metrics.c b/lib/librte_metrics/rte_metrics.c
> > index df5e32c59f..9b38d7787c 100644
> > --- a/lib/librte_metrics/rte_metrics.c
> > +++ b/lib/librte_metrics/rte_metrics.c
> > @@ -13,7 +13,6 @@
> > #include <rte_memzone.h>
> > #include <rte_spinlock.h>
> >
> > -#define RTE_METRICS_MAX_METRICS 256
> > #define RTE_METRICS_MEMZONE_NAME "RTE_METRICS"
> >
> > /**
> > diff --git a/lib/librte_metrics/rte_metrics.h b/lib/librte_metrics/rte_metrics.h
> > index 77bffe08e4..466ca98c31 100644
> > --- a/lib/librte_metrics/rte_metrics.h
> > +++ b/lib/librte_metrics/rte_metrics.h
> > @@ -32,6 +32,7 @@ extern "C" {
> >
> > /** Maximum length of metric name (including null-terminator) */
> > #define RTE_METRICS_MAX_NAME_LEN 64
> > +#define RTE_METRICS_MAX_METRICS 256
>
> Exposing max metrics to API/ABI does limit you in the future.
>
I'm not sure moving the definition limits us any more than before. So long
as the structures which are defined based on this definition are internal
and only accessed via API functions, there should be no problems with
changing this and using function versioning for compatibility.
/Bruce
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH v4 02/18] telemetry: move code to metrics for later reuse
@ 2020-04-24 15:29 3% ` Stephen Hemminger
2020-04-24 15:49 0% ` Bruce Richardson
0 siblings, 1 reply; 200+ results
From: Stephen Hemminger @ 2020-04-24 15:29 UTC (permalink / raw)
To: Ciara Power
Cc: dev, kevin.laatz, reshma.pattan, jerinjacobk, david.marchand,
keith.wiles, mb, thomas
On Fri, 24 Apr 2020 13:41:43 +0100
Ciara Power <ciara.power@intel.com> wrote:
> This commit moves some of the telemetry library code to a new file in
> the metrics library. No modifications are made to the moved code,
> except what is needed to allow it to compile and run. The additional
> code in metrics is built only when the Jansson library is present.
> Telemetry functions as normal, using the functions from the
> metrics_telemetry file. This move will enable code be reused by the new
> version of telemetry in a later commit, to support backward
> compatibility with the existing telemetry usage.
>
> Signed-off-by: Ciara Power <ciara.power@intel.com>
Minor comments, none of these are show stoppers.
> diff --git a/lib/librte_metrics/rte_metrics.c b/lib/librte_metrics/rte_metrics.c
> index df5e32c59f..9b38d7787c 100644
> --- a/lib/librte_metrics/rte_metrics.c
> +++ b/lib/librte_metrics/rte_metrics.c
> @@ -13,7 +13,6 @@
> #include <rte_memzone.h>
> #include <rte_spinlock.h>
>
> -#define RTE_METRICS_MAX_METRICS 256
> #define RTE_METRICS_MEMZONE_NAME "RTE_METRICS"
>
> /**
> diff --git a/lib/librte_metrics/rte_metrics.h b/lib/librte_metrics/rte_metrics.h
> index 77bffe08e4..466ca98c31 100644
> --- a/lib/librte_metrics/rte_metrics.h
> +++ b/lib/librte_metrics/rte_metrics.h
> @@ -32,6 +32,7 @@ extern "C" {
>
> /** Maximum length of metric name (including null-terminator) */
> #define RTE_METRICS_MAX_NAME_LEN 64
> +#define RTE_METRICS_MAX_METRICS 256
Exposing max metrics to API/ABI does limit you in the future.
> diff --git a/lib/librte_metrics/rte_metrics_telemetry.c b/lib/librte_metrics/rte_metrics_telemetry.c
> new file mode 100644
> index 0000000000..a6b2616714
> --- /dev/null
> +++ b/lib/librte_metrics/rte_metrics_telemetry.c
> @@ -0,0 +1,719 @@
...
> +static int32_t
> +rte_metrics_tel_is_port_active(int port_id)
portid should be uint16_t like rte_ethdev
return bool rather than int
> +{
> + int ret;
> +
> + ret = rte_eth_find_next(port_id);
> + if (ret == port_id)
> + return 1;
> +
> + METRICS_LOG_ERR("port_id: %d is invalid, not active",
> + port_id);
> +
> + return 0;
> +}
> +
> +static int32_t
> +rte_metrics_tel_reg_port_ethdev_to_metrics(uint16_t port_id)
> +{
> + int ret, num_xstats, ret_val, i;
> + struct rte_eth_xstat *eth_xstats = NULL;
> + struct rte_eth_xstat_name *eth_xstats_names = NULL;
You don't need to initialize all variables, and it is actually
bad practice with modern compilers since it breaks checking
for using values before set.
> +
> + if (!rte_eth_dev_is_valid_port(port_id)) {
> + METRICS_LOG_ERR("port_id: %d is invalid", port_id);
> + return -EINVAL;
> + }
Validity of port_id is already checked by rte_eth_xstats_get
> + num_xstats = rte_eth_xstats_get(port_id, NULL, 0);
> + if (num_xstats < 0) {
> + METRICS_LOG_ERR("rte_eth_xstats_get(%u) failed: %d",
> + port_id, num_xstats);
> + return -EPERM;
> + }
> +
> + eth_xstats = malloc(sizeof(struct rte_eth_xstat) * num_xstats);
> + if (eth_xstats == NULL) {
> + METRICS_LOG_ERR("Failed to malloc memory for xstats");
> + return -ENOMEM;
> + }
> +
> + ret = rte_eth_xstats_get(port_id, eth_xstats, num_xstats);
> + const char *xstats_names[num_xstats];
Please don't put declarations in the middle of the code.
You should malloc the names as well
> + eth_xstats_names = malloc(sizeof(struct rte_eth_xstat_name)
> + * num_xstats);
> + if (ret < 0 || ret > num_xstats) {
> + METRICS_LOG_ERR("rte_eth_xstats_get(%u) len%i failed: %d",
> + port_id, num_xstats, ret);
> + ret_val = -EPERM;
> + goto free_xstats;
> + }
> +
> + if (eth_xstats_names == NULL) {
> + METRICS_LOG_ERR("Failed to malloc memory for xstats_names");
> + ret_val = -ENOMEM;
> + goto free_xstats;
> + }
> +
> + ret = rte_eth_xstats_get_names(port_id, eth_xstats_names, num_xstats);
> + if (ret < 0 || ret > num_xstats) {
> + METRICS_LOG_ERR("rte_eth_xstats_get_names(%u) len%i failed: %d",
> + port_id, num_xstats, ret);
> + ret_val = -EPERM;
> + goto free_xstats;
> + }
> +
> + for (i = 0; i < num_xstats; i++)
> + xstats_names[i] = eth_xstats_names[eth_xstats[i].id].name;
> +
> + ret_val = rte_metrics_reg_names(xstats_names, num_xstats);
> + if (ret_val < 0) {
> + METRICS_LOG_ERR("rte_metrics_reg_names failed - metrics may already be registered");
> + ret_val = -1;
> + goto free_xstats;
> + }
> +
> + goto free_xstats;
> +
> +free_xstats:
> + free(eth_xstats);
> + free(eth_xstats_names);
> + return ret_val;
> +}
> +
> +int32_t
> +rte_metrics_tel_reg_all_ethdev(int *metrics_register_done, int *reg_index_list)
> +{
> + struct driver_index {
> + const void *dev_ops;
> + int reg_index;
> + } drv_idx[RTE_MAX_ETHPORTS] = { {0} };
> + int nb_drv_idx = 0;
> + uint16_t pid;
> + int ret;
> +
> + RTE_ETH_FOREACH_DEV(pid) {
> + int i;
> + /* Different device types have different numbers of stats, so
> + * first check if the stats for this type of device have
> + * already been registered
> + */
> + for (i = 0; i < nb_drv_idx; i++) {
> + if (rte_eth_devices[pid].dev_ops ==
> + drv_idx[i].dev_ops) {
> + reg_index_list[pid] = drv_idx[i].reg_index;
> + break;
> + }
> + }
> + if (i < nb_drv_idx)
> + continue; /* we found a match, go to next port */
> +
> + /* No match, register a new set of xstats for this port */
> + ret = rte_metrics_tel_reg_port_ethdev_to_metrics(pid);
> + if (ret < 0) {
> + METRICS_LOG_ERR("Failed to register ethdev metrics");
> + return -1;
> + }
> + reg_index_list[pid] = ret;
> + drv_idx[nb_drv_idx].dev_ops = rte_eth_devices[pid].dev_ops;
> + drv_idx[nb_drv_idx].reg_index = ret;
> + nb_drv_idx++;
> + }
> +
> + *metrics_register_done = 1;
> + return 0;
> +}
> +
> +static int32_t
> +rte_metrics_tel_update_metrics_ethdev(uint16_t port_id, int reg_start_index)
> +{
> + int ret, num_xstats, i;
> + struct rte_eth_xstat *eth_xstats;
> +
> + if (!rte_eth_dev_is_valid_port(port_id)) {
> + METRICS_LOG_ERR("port_id: %d is invalid", port_id);
> + return -EINVAL;
> + }
> +
> + ret = rte_metrics_tel_is_port_active(port_id);
> + if (ret < 1)
> + return -EINVAL;
> +
> + num_xstats = rte_eth_xstats_get(port_id, NULL, 0);
> + if (num_xstats < 0) {
> + METRICS_LOG_ERR("rte_eth_xstats_get(%u) failed: %d", port_id,
> + num_xstats);
> + return -EPERM;
> + }
The number of metrics on a port should not change (as long as it has
not been hot plugged). So you could optimize by knowing number of
stats from last query.
> + eth_xstats = malloc(sizeof(struct rte_eth_xstat) * num_xstats);
> + if (eth_xstats == NULL) {
> + METRICS_LOG_ERR("Failed to malloc memory for xstats");
> + return -ENOMEM;
> + }
> +
> + ret = rte_eth_xstats_get(port_id, eth_xstats, num_xstats);
> + if (ret < 0 || ret > num_xstats) {
> + free(eth_xstats);
> + METRICS_LOG_ERR("rte_eth_xstats_get(%u) len%i failed: %d",
> + port_id, num_xstats, ret);
> + return -EPERM;
> + }
> +
> + uint64_t xstats_values[num_xstats];
> + for (i = 0; i < num_xstats; i++)
> + xstats_values[i] = eth_xstats[i].value;
> +
> + ret = rte_metrics_update_values(port_id, reg_start_index, xstats_values,
> + num_xstats);
> + if (ret < 0) {
> + METRICS_LOG_ERR("Could not update metrics values");
> + free(eth_xstats);
> + return -EPERM;
> + }
> +
> + free(eth_xstats);
> + return 0;
> +}
> +
> +static int
> +rte_metrics_tel_get_metrics(uint32_t port_id, struct rte_metric_value
> + *metrics, struct rte_metric_name *names, int num_metrics)
Don't break awkwardly in the function declaration.
> +{
> + int ret, num_values;
> +
> + if (num_metrics < 0) {
if you don't want negative, make it unsigned.
> + METRICS_LOG_ERR("Invalid metrics count");
> + return -EINVAL;
> + } else if (num_metrics == 0) {
else after return non needed.
> + METRICS_LOG_ERR("No metrics to display (none have been registered)");
> + return -EPERM;
> + }
> +
> + if (metrics == NULL) {
> + METRICS_LOG_ERR("Metrics must be initialised.");
> + return -EINVAL;
> + }
> +
> + if (names == NULL) {
> + METRICS_LOG_ERR("Names must be initialised.");
> + return -EINVAL;
> + }
> +
> + ret = rte_metrics_get_names(names, num_metrics);
> + if (ret < 0 || ret > num_metrics) {
> + METRICS_LOG_ERR("Cannot get metrics names");
> + return -EPERM;
> + }
> +
> + num_values = rte_metrics_get_values(port_id, NULL, 0);
> + ret = rte_metrics_get_values(port_id, metrics, num_values);
> + if (ret < 0 || ret > num_values) {
> + METRICS_LOG_ERR("Cannot get metrics values");
> + return -EPERM;
> + }
> +
> + return 0;
> +}
^ permalink raw reply [relevance 3%]
* Re: [dpdk-dev] [PATCH] cryptodev: add support for user callback functions
@ 2020-04-24 15:10 3% ` Ananyev, Konstantin
0 siblings, 0 replies; 200+ results
From: Ananyev, Konstantin @ 2020-04-24 15:10 UTC (permalink / raw)
To: Gujjar, Abhinandan S, Doherty, Declan, jerinj, akhil.goyal, dev
Cc: Vangati, Narender, Gujjar, Abhinandan S
> In an eventdev world, multiple workers (with ordered queue) will be
> working on IPsec ESP processing. The ESP header's sequence number is
> unique and has to be sequentially incremented in an orderly manner.
> This rises a need for incrementing sequence number in crypto stage
> especially in event crypto adapter. By adding a user callback to
> cryptodev at enqueue burst, the user callback will get executed
> in the context of event crypto adapter. This helps the application
> to increment the ESP sequence number atomically and orderly manner.
>
> Signed-off-by: Abhinandan Gujjar <abhinandan.gujjar@intel.com>
> ---
> config/common_base | 1 +
> lib/librte_cryptodev/rte_cryptodev.c | 120 ++++++++++++++++++++++++
> lib/librte_cryptodev/rte_cryptodev.h | 125 ++++++++++++++++++++++++-
> lib/librte_cryptodev/rte_cryptodev_version.map | 3 +
> 4 files changed, 248 insertions(+), 1 deletion(-)
>
> diff --git a/config/common_base b/config/common_base
> index 9ec689d..6f93acb 100644
> --- a/config/common_base
> +++ b/config/common_base
> @@ -586,6 +586,7 @@ CONFIG_RTE_LIBRTE_PMD_BBDEV_FPGA_5GNR_FEC=y
> #
> CONFIG_RTE_LIBRTE_CRYPTODEV=y
> CONFIG_RTE_CRYPTO_MAX_DEVS=64
> +CONFIG_RTE_CRYPTODEV_CALLBACKS=y
>
> #
> # Compile PMD for ARMv8 Crypto device
> diff --git a/lib/librte_cryptodev/rte_cryptodev.c b/lib/librte_cryptodev/rte_cryptodev.c
> index 2849b2e..5a4cba9 100644
> --- a/lib/librte_cryptodev/rte_cryptodev.c
> +++ b/lib/librte_cryptodev/rte_cryptodev.c
> @@ -56,6 +56,9 @@
> /* spinlock for crypto device callbacks */
> static rte_spinlock_t rte_cryptodev_cb_lock = RTE_SPINLOCK_INITIALIZER;
>
> +/* spinlock for crypto device enq callbacks */
> +static rte_spinlock_t rte_cryptodev_enq_cb_lock = RTE_SPINLOCK_INITIALIZER;
> +
>
> /**
> * The user application callback description.
> @@ -1256,6 +1259,123 @@ struct rte_cryptodev *
> rte_spinlock_unlock(&rte_cryptodev_cb_lock);
> }
>
> +const struct rte_cryptodev_enq_callback *__rte_experimental
> +rte_cryptodev_add_enq_callback(uint8_t dev_id,
> + uint16_t qp_id,
> + rte_cryptodev_enq_cb_fn cb_fn,
> + void *cb_arg)
> +{
> + struct rte_cryptodev *dev;
> + struct rte_cryptodev_enq_callback *cb, *tail;
> +
> + if (!cb_fn)
> + return NULL;
> +
> + if (!rte_cryptodev_pmd_is_valid_dev(dev_id)) {
> + CDEV_LOG_ERR("Invalid dev_id=%" PRIu8, dev_id);
> + return NULL;
> + }
> +
> + dev = &rte_crypto_devices[dev_id];
> + if (qp_id >= dev->data->nb_queue_pairs) {
> + CDEV_LOG_ERR("Invalid queue_pair_id=%d", qp_id);
> + return NULL;
> + }
> +
> + cb = rte_zmalloc(NULL, sizeof(*cb), 0);
> + if (cb == NULL) {
> + CDEV_LOG_ERR("Failed to allocate memory for callback on "
> + "dev=%d, queue_pair_id=%d", dev_id, qp_id);
> + rte_errno = ENOMEM;
> + return NULL;
> + }
> +
> + cb->fn = cb_fn;
> + cb->arg = cb_arg;
> +
> + rte_spinlock_lock(&rte_cryptodev_enq_cb_lock);
> + if (dev->enq_cbs == NULL) {
> + dev->enq_cbs = rte_zmalloc(NULL, sizeof(cb) *
> + dev->data->nb_queue_pairs, 0);
> + if (dev->enq_cbs == NULL) {
> + CDEV_LOG_ERR("Failed to allocate memory for callbacks");
> + rte_errno = ENOMEM;
> + rte_free(cb);
> + return NULL;
> + }
> + }
> +
> + /* Add the callbacks in fifo order. */
> + tail = dev->enq_cbs[qp_id];
> + if (tail) {
> + while (tail->next)
> + tail = tail->next;
> + tail->next = cb;
> + } else
> + dev->enq_cbs[qp_id] = cb;
> +
> + rte_spinlock_unlock(&rte_cryptodev_enq_cb_lock);
> +
> + return cb;
> +}
> +
> +int __rte_experimental
> +rte_cryptodev_remove_enq_callback(uint8_t dev_id,
> + uint16_t qp_id,
> + const struct rte_cryptodev_enq_callback *cb)
> +{
> + struct rte_cryptodev *dev;
> + struct rte_cryptodev_enq_callback **prev_cb, *curr_cb;
> + uint16_t qp;
> + int free_mem = 1;
> + int ret = -EINVAL;
> +
> + if (!cb)
> + return ret;
> +
> + if (!rte_cryptodev_pmd_is_valid_dev(dev_id)) {
> + CDEV_LOG_ERR("Invalid dev_id=%" PRIu8, dev_id);
> + return ret;
> + }
> +
> + dev = &rte_crypto_devices[dev_id];
> + if (qp_id >= dev->data->nb_queue_pairs) {
> + CDEV_LOG_ERR("Invalid queue_pair_id=%d", qp_id);
> + return ret;
> + }
> +
> + rte_spinlock_lock(&rte_cryptodev_enq_cb_lock);
> + if (dev->enq_cbs == NULL) {
> + rte_spinlock_unlock(&rte_cryptodev_enq_cb_lock);
> + return ret;
> + }
> +
> + prev_cb = &dev->enq_cbs[qp_id];
> + for (; *prev_cb != NULL; prev_cb = &curr_cb->next) {
> + curr_cb = *prev_cb;
> + if (curr_cb == cb) {
> + /* Remove the user cb from the callback list. */
> + *prev_cb = curr_cb->next;
> + ret = 0;
> + break;
> + }
> + }
> +
> + for (qp = 0; qp < dev->data->nb_queue_pairs; qp++)
> + if (dev->enq_cbs[qp] != NULL) {
> + free_mem = 0;
> + break;
> + }
> +
> + if (free_mem) {
> + rte_free(dev->enq_cbs);
> + dev->enq_cbs = NULL;
> + }
> +
> + rte_spinlock_unlock(&rte_cryptodev_enq_cb_lock);
> +
> + return ret;
> +}
Pls don't re-implement pitfall we have with ethdev rx/tx callback:
right now with ethdev approach it is impossible to know when it is safe to free
removed CB and free used by CB resources if any.
Unless you do dev_stop() of course.
So majority of ethdev CB uses have to either leave CB allocated forever
after removal (memory leak), or invent some specific sync methods underneath
that API.
We probably need to introduce some sync mechanism here straightway (RCU or so)
to avoid same issue.
>
> int
> rte_cryptodev_sym_session_init(uint8_t dev_id,
> diff --git a/lib/librte_cryptodev/rte_cryptodev.h b/lib/librte_cryptodev/rte_cryptodev.h
> index f4846d2..2cf466b 100644
> --- a/lib/librte_cryptodev/rte_cryptodev.h
> +++ b/lib/librte_cryptodev/rte_cryptodev.h
> @@ -518,6 +518,32 @@ struct rte_cryptodev_qp_conf {
> };
>
> /**
> + * Function type used for pre processing crypto ops when enqueue burst is
> + * called.
> + *
> + * The callback function is called on enqueue burst immediately
> + * before the crypto ops are put onto the hardware queue for processing.
> + *
> + * @param dev_id The identifier of the device.
> + * @param qp_id The index of the queue pair in which ops are
> + * to be enqueued for processing. The value
> + * must be in the range [0, nb_queue_pairs - 1]
> + * previously supplied to
> + * *rte_cryptodev_configure*.
> + * @param ops The address of an array of *nb_ops* pointers
> + * to *rte_crypto_op* structures which contain
> + * the crypto operations to be processed.
> + * @param nb_ops The number of operations to process.
> + * @param user_param The arbitrary user parameter passed in by the
> + * application when the callback was originally
> + * registered.
> + * @return The number of ops to be enqueued to the
> + * crypto device.
> + */
> +typedef uint16_t (*rte_cryptodev_enq_cb_fn)(uint16_t dev_id, uint16_t qp_id,
> + struct rte_crypto_op **ops, uint16_t nb_ops, void *user_param);
> +
> +/**
> * Typedef for application callback function to be registered by application
> * software for notification of device events
> *
> @@ -800,7 +826,6 @@ struct rte_cryptodev_config {
> enum rte_cryptodev_event_type event,
> rte_cryptodev_cb_fn cb_fn, void *cb_arg);
>
> -
> typedef uint16_t (*dequeue_pkt_burst_t)(void *qp,
> struct rte_crypto_op **ops, uint16_t nb_ops);
> /**< Dequeue processed packets from queue pair of a device. */
> @@ -817,6 +842,17 @@ typedef uint16_t (*enqueue_pkt_burst_t)(void *qp,
> /** Structure to keep track of registered callbacks */
> TAILQ_HEAD(rte_cryptodev_cb_list, rte_cryptodev_callback);
>
> +/**
> + * @internal
> + * Structure used to hold information about the callbacks to be called for a
> + * queue pair on enqueue.
> + */
> +struct rte_cryptodev_enq_callback {
> + struct rte_cryptodev_enq_callback *next;
> + rte_cryptodev_enq_cb_fn fn;
> + void *arg;
> +};
> +
> /** The data structure associated with each crypto device. */
> struct rte_cryptodev {
> dequeue_pkt_burst_t dequeue_burst;
> @@ -839,6 +875,9 @@ struct rte_cryptodev {
> struct rte_cryptodev_cb_list link_intr_cbs;
> /**< User application callback for interrupts if present */
>
> + struct rte_cryptodev_enq_callback **enq_cbs;
> + /**< User application callback for pre enqueue processing */
> +
Why not to put it at the very end of the structure?
Would be safer in terms of ABI breakage problem, etc.
> void *security_ctx;
> /**< Context for security ops */
>
> @@ -966,6 +1005,18 @@ struct rte_cryptodev_data {
> {
> struct rte_cryptodev *dev = &rte_cryptodevs[dev_id];
>
> +#ifdef RTE_CRYPTODEV_CALLBACKS
> + if (unlikely(dev->enq_cbs != NULL && dev->enq_cbs[qp_id] != NULL)) {
> + struct rte_cryptodev_enq_callback *cb =
> + dev->enq_cbs[qp_id];
> +
> + do {
> + nb_ops = cb->fn(dev_id, qp_id, ops, nb_ops,
> + cb->arg);
> + cb = cb->next;
> + } while (cb != NULL);
> + }
> +#endif
> return (*dev->enqueue_burst)(
> dev->data->queue_pairs[qp_id], ops, nb_ops);
> }
> @@ -1296,6 +1347,78 @@ struct rte_cryptodev_asym_session *
> struct rte_cryptodev_sym_session *sess, union rte_crypto_sym_ofs ofs,
> struct rte_crypto_sym_vec *vec);
>
> +
> +/**
> + * Add a user callback for a given crypto device and queue pair which will be
> + * called on crypto ops enqueue.
> + *
> + * This API configures a function to be called for each burst of crypto ops
> + * received on a given crypto device queue pair. The return value is a pointer
> + * that can be used later to remove the callback using
> + * rte_cryptodev_remove_enq_callback().
> + *
> + * Multiple functions are called in the order that they are added.
> + *
> + * @param dev_id The identifier of the device.
> + * @param qp_id The index of the queue pair in which ops are
> + * to be enqueued for processing. The value
> + * must be in the range [0, nb_queue_pairs - 1]
> + * previously supplied to
> + * *rte_cryptodev_configure*.
> + * @param cb_fn The callback function
> + * @param cb_arg A generic pointer parameter which will be passed
> + * to each invocation of the callback function on
> + * this crypto device and queue pair.
> + *
> + * @return
> + * NULL on error.
> + * On success, a pointer value which can later be used to remove the callback.
> + */
> +
> +const struct rte_cryptodev_enq_callback * __rte_experimental
> +rte_cryptodev_add_enq_callback(uint8_t dev_id,
> + uint16_t qp_id,
> + rte_cryptodev_enq_cb_fn cb_fn,
> + void *cb_arg);
> +
> +
> +/**
> + * Remove a user callback function for given crypto device and queue pair.
> + *
> + * This function is used to removed callbacks that were added to a crypto
> + * device queue pair using rte_cryptodev_add_enq_callback().
> + *
> + * Note: the callback is removed from the callback list but it isn't freed
> + * since the it may still be in use. The memory for the callback can be
> + * subsequently freed back by the application by calling rte_free().
> + *
> + * - Immediately - if the crypto device is stopped, or user knows that
> + * no callbacks are in flight e.g. if called from the thread doing enq/deq
> + * on that queue.
> + *
> + * - After a short delay - where the delay is sufficient to allow any
> + * in-flight callbacks to complete.
> + *
> + * @param dev_id The identifier of the device.
> + * @param qp_id The index of the queue pair in which ops are
> + * to be enqueued for processing. The value
> + * must be in the range [0, nb_queue_pairs - 1]
> + * previously supplied to
> + * *rte_cryptodev_configure*.
> + * @param cb Pointer to user supplied callback created via
> + * rte_cryptodev_add_enq_callback().
> + *
> + * @return
> + * - 0: Success. Callback was removed.
> + * - -EINVAL: The dev_id or the qp_id is out of range, or the callback
> + * is NULL or not found for the crypto device queue pair.
> + */
> +
> +int __rte_experimental
> +rte_cryptodev_remove_enq_callback(uint8_t dev_id,
> + uint16_t qp_id,
> + const struct rte_cryptodev_enq_callback *cb);
> +
> #ifdef __cplusplus
> }
> #endif
> diff --git a/lib/librte_cryptodev/rte_cryptodev_version.map b/lib/librte_cryptodev/rte_cryptodev_version.map
> index 6e41b4b..e8d3e77 100644
> --- a/lib/librte_cryptodev/rte_cryptodev_version.map
> +++ b/lib/librte_cryptodev/rte_cryptodev_version.map
> @@ -58,9 +58,11 @@ DPDK_20.0 {
> local: *;
> };
>
> +
> EXPERIMENTAL {
> global:
>
> + rte_cryptodev_add_enq_callback;
> rte_cryptodev_asym_capability_get;
> rte_cryptodev_asym_get_header_session_size;
> rte_cryptodev_asym_get_private_session_size;
> @@ -71,6 +73,7 @@ EXPERIMENTAL {
> rte_cryptodev_asym_session_init;
> rte_cryptodev_asym_xform_capability_check_modlen;
> rte_cryptodev_asym_xform_capability_check_optype;
> + rte_cryptodev_remove_enq_callback;
> rte_cryptodev_sym_cpu_crypto_process;
> rte_cryptodev_sym_get_existing_header_session_size;
> rte_cryptodev_sym_session_get_user_data;
> --
> 1.9.1
^ permalink raw reply [relevance 3%]
* Re: [dpdk-dev] [PATCH v2] abi: change references to abi 20.0.1 to abi v21
2020-04-24 14:50 6% ` Ray Kinsella
@ 2020-04-24 15:01 6% ` David Marchand
2020-04-29 12:19 9% ` Dodji Seketeli
1 sibling, 0 replies; 200+ results
From: David Marchand @ 2020-04-24 15:01 UTC (permalink / raw)
To: Ray Kinsella
Cc: Dodji Seketeli, dev, Andrzej Ostruszka, Stephen Hemminger,
Thomas Monjalon, Neil Horman, Jingjing Wu, Wenzhuo Lu,
Matan Azrad, Shahaf Shuler, Viacheslav Ovsiienko, Jerin Jacob,
Nithin Dabilpuram, Alfredo Cardigliano, Mahipal Challa,
Cristian Dumitrescu, Wang, Haiyue
On Fri, Apr 24, 2020 at 4:50 PM Ray Kinsella <mdr@ashroe.eu> wrote:
>
> ah ok, the particular system I made the change on was Ubuntu 18.04.2.
> which is libabigail 1.2.0.
>
> Given we still support v19.11 on Ubuntu 18.04.2.
> I think it's worthwhile keeping the suppression until v20.11?
1.2 is fairly old and does not seem maintained by Ubuntu (asked for
backport, and got ignored so far).
https://bugs.launchpad.net/ubuntu/+source/libabigail/+bug/1863607
Dodji told me that there were new checks added in newer sersions.
I also fear that supporting an old libabigail version will force us to
add a lot of other rules.
In Travis, we currently use libabigail 1.6 (mainly because I did not
update to 1.7 when it was released).
--
David Marchand
^ permalink raw reply [relevance 6%]
* Re: [dpdk-dev] [PATCH v5 1/1] eal: add internal ABI marking support
2020-04-23 3:19 15% ` [dpdk-dev] [PATCH v5 1/1] eal: add internal ABI marking support Haiyue Wang
@ 2020-04-24 14:52 7% ` David Marchand
2020-04-25 6:10 7% ` Wang, Haiyue
0 siblings, 1 reply; 200+ results
From: David Marchand @ 2020-04-24 14:52 UTC (permalink / raw)
To: Haiyue Wang
Cc: dev, Thomas Monjalon, Bruce Richardson, Yigit, Ferruh,
Neil Horman, Ray Kinsella
On Thu, Apr 23, 2020 at 5:25 AM Haiyue Wang <haiyue.wang@intel.com> wrote:
>
> Introduce __rte_internal tag to mark internal ABI function which is used
> by the drivers or other libraries.
>
> Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
> ---
> buildtools/check-internal-syms.sh | 57 +++++++++++++++++++++++++++++
> devtools/check-symbol-change.sh | 7 ++++
> devtools/checkpatches.sh | 38 +++++++++++++++++++
> devtools/libabigail.abignore | 5 +++
> drivers/meson.build | 2 +-
> lib/librte_eal/include/rte_compat.h | 13 +++++++
> lib/meson.build | 2 +-
> mk/internal/rte.compile-pre.mk | 3 ++
> mk/target/generic/rte.vars.mk | 1 +
> 9 files changed, 126 insertions(+), 2 deletions(-)
> create mode 100755 buildtools/check-internal-syms.sh
We can have a single script (let's say devtools/check-symbols.sh) that
contains both experimental and internal symbols.
> diff --git a/devtools/check-symbol-change.sh b/devtools/check-symbol-change.sh
> index ed2178e36..978979077 100755
> --- a/devtools/check-symbol-change.sh
> +++ b/devtools/check-symbol-change.sh
> @@ -91,6 +91,13 @@ check_for_rule_violations()
> if [ "$ar" = "add" ]
> then
>
> + if [ "$secname" = "INTERNAL" ]
> + then
> + # these are absolved from any further checking
> + echo "Skipping symbol $symname in INTERNAL"
> + continue
> + fi
> +
> if [ "$secname" = "unknown" ]
> then
> # Just inform the user of this occurrence, but
This only handles symbol additions to the INTERNAL section.
Other cases like moving or removing a INTERNAL symbol are not handled.
> diff --git a/devtools/checkpatches.sh b/devtools/checkpatches.sh
> index c30ce64cc..2df8d7f2c 100755
> --- a/devtools/checkpatches.sh
> +++ b/devtools/checkpatches.sh
> @@ -111,6 +111,36 @@ check_experimental_tags() { # <patch>
> return $res
> }
>
> +check_internal_tags() { # <patch>
> + res=0
> +
> + cat "$1" |awk '
> + BEGIN {
> + current_file = "";
> + ret = 0;
> + }
> + /^+++ b\// {
> + current_file = $2;
> + }
> + /^+.*__rte_internal/ {
> + if (current_file ~ ".c$" ) {
> + print "Please only put __rte_internal tags in " \
> + "headers ("current_file")";
> + ret = 1;
> + }
> + if ($1 != "+__rte_internal" || $2 != "") {
> + print "__rte_internal must appear alone on the line" \
> + " immediately preceding the return type of a function."
> + ret = 1;
> + }
> + }
> + END {
> + exit ret;
> + }' || res=1
> +
> + return $res
> +}
> +
> number=0
> range='origin/master..'
> quiet=false
> @@ -194,6 +224,14 @@ check () { # <patch> <commit> <title>
> ret=1
> fi
>
> + ! $verbose || printf '\nChecking __rte_internal tags:\n'
> + report=$(check_internal_tags "$tmpinput")
> + if [ $? -ne 0 ] ; then
> + $headline_printed || print_headline "$3"
> + printf '%s\n' "$report"
> + ret=1
> + fi
> +
> if [ "$tmpinput" != "$1" ]; then
> rm -f "$tmpinput"
> trap - INT
> diff --git a/devtools/libabigail.abignore b/devtools/libabigail.abignore
> index cd86d89ca..3e8e2ea74 100644
> --- a/devtools/libabigail.abignore
> +++ b/devtools/libabigail.abignore
> @@ -3,6 +3,11 @@
> [suppress_variable]
> symbol_version = EXPERIMENTAL
>
> +[suppress_function]
> + symbol_version = INTERNAL
> +[suppress_variable]
> + symbol_version = INTERNAL
> +
> ; Explicit ignore for driver-only ABI
> [suppress_type]
> name = rte_cryptodev_ops
> diff --git a/drivers/meson.build b/drivers/meson.build
> index 4d8f842ab..cac07161f 100644
> --- a/drivers/meson.build
> +++ b/drivers/meson.build
> @@ -20,7 +20,7 @@ dpdk_driver_classes = ['common',
> disabled_drivers = run_command(list_dir_globs, get_option('disable_drivers'),
> ).stdout().split()
>
> -default_cflags = machine_args + ['-DALLOW_EXPERIMENTAL_API']
> +default_cflags = machine_args + ['-DALLOW_EXPERIMENTAL_API'] + ['-DALLOW_INTERNAL_API']
Nit:
default_cflags = machine_args
default_cflags += ['-DALLOW_EXPERIMENTAL_API']
default_cflags += ['-DALLOW_INTERNAL_API']
> if cc.has_argument('-Wno-format-truncation')
> default_cflags += '-Wno-format-truncation'
> endif
More importantly on this file:
- drivers/meson.build is not updated to check for internal symbols, see:
https://git.dpdk.org/dpdk/tree/drivers/meson.build#n166
- For fully experimental libraries, we have a special so version:
https://git.dpdk.org/dpdk/tree/drivers/meson.build#n131
This will apply to common drivers that will be 100% internal.
Not sure if this is an issue.
> diff --git a/lib/librte_eal/include/rte_compat.h b/lib/librte_eal/include/rte_compat.h
> index 3eb33784b..4cd8f68d6 100644
> --- a/lib/librte_eal/include/rte_compat.h
> +++ b/lib/librte_eal/include/rte_compat.h
> @@ -19,4 +19,17 @@ __attribute__((section(".text.experimental")))
>
> #endif
>
> +#ifndef ALLOW_INTERNAL_API
> +
> +#define __rte_internal \
> +__attribute__((error("Symbol is not public ABI"), \
> +section(".text.internal")))
> +
> +#else
> +
> +#define __rte_internal \
> +__attribute__((section(".text.internal")))
> +
> +#endif
> +
> #endif /* _RTE_COMPAT_H_ */
> diff --git a/lib/meson.build b/lib/meson.build
> index c28b8df83..4d2f90d6a 100644
> --- a/lib/meson.build
> +++ b/lib/meson.build
> @@ -38,7 +38,7 @@ if is_windows
> libraries = ['kvargs','eal'] # only supported libraries for windows
> endif
>
> -default_cflags = machine_args + ['-DALLOW_EXPERIMENTAL_API']
> +default_cflags = machine_args + ['-DALLOW_EXPERIMENTAL_API'] + ['-DALLOW_INTERNAL_API']
Same comments as for drivers/meson.build.
> diff --git a/mk/internal/rte.compile-pre.mk b/mk/internal/rte.compile-pre.mk
> index 82fe098f7..0369786a5 100644
> --- a/mk/internal/rte.compile-pre.mk
> +++ b/mk/internal/rte.compile-pre.mk
> @@ -58,6 +58,8 @@ C_TO_O_DISP = $(if $(V),"$(C_TO_O_STR)"," CC $(@)")
> endif
> EXPERIMENTAL_CHECK = $(RTE_SDK)/buildtools/check-experimental-syms.sh
> CHECK_EXPERIMENTAL = $(EXPERIMENTAL_CHECK) $(SRCDIR)/$(EXPORT_MAP) $@
> +INTERNAL_CHECK = $(RTE_SDK)/buildtools/check-internal-syms.sh
> +CHECK_INTERNAL = $(INTERNAL_CHECK) $(SRCDIR)/$(EXPORT_MAP) $@
With a single script, we can go with:
CHECK_SYMBOLS_SCRIPT = $(RTE_SDK)/buildtools/check-symbols.sh
CHECK_SYMBOLS = $(CHECK_SYMBOLS_SCRIPT) $(SRCDIR)/$(EXPORT_MAP) $@
>
> PMDINFO_GEN = $(RTE_SDK_BIN)/app/dpdk-pmdinfogen $@ $@.pmd.c
> PMDINFO_CC = $(CC) $(CPPFLAGS) $(CFLAGS) $(EXTRA_CFLAGS) -c -o $@.pmd.o $@.pmd.c
> @@ -76,6 +78,7 @@ C_TO_O_DO = @set -e; \
> $(C_TO_O) && \
> $(PMDINFO_TO_O) && \
> $(CHECK_EXPERIMENTAL) && \
> + $(CHECK_INTERNAL) && \
+ $(CHECK_SYMBOLS) && \
> echo $(C_TO_O_CMD) > $(call obj2cmd,$(@)) && \
> sed 's,'$@':,dep_'$@' =,' $(call obj2dep,$(@)).tmp > $(call obj2dep,$(@)) && \
> rm -f $(call obj2dep,$(@)).tmp
> diff --git a/mk/target/generic/rte.vars.mk b/mk/target/generic/rte.vars.mk
> index ec2672897..11b0418e5 100644
> --- a/mk/target/generic/rte.vars.mk
> +++ b/mk/target/generic/rte.vars.mk
> @@ -106,6 +106,7 @@ ifeq ($(BUILDING_RTE_SDK),1)
> # building sdk
> CFLAGS += -include $(RTE_OUTPUT)/include/rte_config.h
> CFLAGS += -DALLOW_EXPERIMENTAL_API
> +CFLAGS += -DALLOW_INTERNAL_API
> else
> # if we are building an external application, include SDK's lib and
> # includes too
> --
> 2.26.2
>
We are missing updates on devtools/check-abi-version.sh and
devtools/update_version_map_abi.py.
--
David Marchand
^ permalink raw reply [relevance 7%]
* Re: [dpdk-dev] [PATCH v2] abi: change references to abi 20.0.1 to abi v21
2020-04-24 14:10 6% ` David Marchand
@ 2020-04-24 14:50 6% ` Ray Kinsella
2020-04-24 15:01 6% ` David Marchand
2020-04-29 12:19 9% ` Dodji Seketeli
0 siblings, 2 replies; 200+ results
From: Ray Kinsella @ 2020-04-24 14:50 UTC (permalink / raw)
To: David Marchand, Dodji Seketeli
Cc: dev, Andrzej Ostruszka, Stephen Hemminger, Thomas Monjalon,
Neil Horman, Jingjing Wu, Wenzhuo Lu, Matan Azrad, Shahaf Shuler,
Viacheslav Ovsiienko, Jerin Jacob, Nithin Dabilpuram,
Alfredo Cardigliano, Mahipal Challa, Cristian Dumitrescu, Wang,
Haiyue
ah ok, the particular system I made the change on was Ubuntu 18.04.2.
which is libabigail 1.2.0.
Given we still support v19.11 on Ubuntu 18.04.2.
I think it's worthwhile keeping the suppression until v20.11?
Ray K
On 24/04/2020 15:10, David Marchand wrote:
> On Thu, Apr 23, 2020 at 8:48 AM Ray Kinsella <mdr@ashroe.eu> wrote:
>>
>> Change references to abi 20.0.1 to use abi v21, add myself as the map
>> file maintainer to more closely monitor future abi changes. Add
>> suppressions that were missed on changes to librte_lpm.
>
> About the lpm change below, this might be because you are using an old
> version of libabigail.
> I think the change went in:
> $ git describe --contains 215b7eb4
> libabigail-1.4~8
>
> I use either libabigail 1.7 or the current master (when testing Dodji
> enhancements).
>
>
>> diff --git a/devtools/libabigail.abignore b/devtools/libabigail.abignore
>> index cd86d89ca..20c821cee 100644
>> --- a/devtools/libabigail.abignore
>> +++ b/devtools/libabigail.abignore
>> @@ -2,7 +2,6 @@
>> symbol_version = EXPERIMENTAL
>> [suppress_variable]
>> symbol_version = EXPERIMENTAL
>> -
>> ; Explicit ignore for driver-only ABI
>> [suppress_type]
>> name = rte_cryptodev_ops
>> @@ -18,3 +17,15 @@
>> [suppress_type]
>> type_kind = struct
>> name = rte_event_ring
>> +; Explicit ignore ABI 20.0.1
>> +[suppress_function]
>> + symbol_version = DPDK_20.0.1
>> +[suppress_variable]
>> + symbol_version = DPDK_20.0.1
>> +; Explicit ignore of const lpm6_lookup change
>> +[suppress_function]
>> + name = rte_lpm6_delete
>> + parameter = '1 uint8_t*
>> +[suppress_function]
>> + name = rte_lpm6_is_rule_present
>> + parameter = '1 uint8_t*
>
>
^ permalink raw reply [relevance 6%]
* Re: [dpdk-dev] [PATCH v2] abi: change references to abi 20.0.1 to abi v21
2020-04-23 6:41 21% ` [dpdk-dev] [PATCH v2] " Ray Kinsella
2020-04-24 9:15 6% ` Thomas Monjalon
@ 2020-04-24 14:10 6% ` David Marchand
2020-04-24 14:50 6% ` Ray Kinsella
1 sibling, 1 reply; 200+ results
From: David Marchand @ 2020-04-24 14:10 UTC (permalink / raw)
To: Ray Kinsella, Dodji Seketeli
Cc: dev, Andrzej Ostruszka, Stephen Hemminger, Thomas Monjalon,
Neil Horman, Jingjing Wu, Wenzhuo Lu, Matan Azrad, Shahaf Shuler,
Viacheslav Ovsiienko, Jerin Jacob, Nithin Dabilpuram,
Alfredo Cardigliano, Mahipal Challa, Cristian Dumitrescu, Wang,
Haiyue
On Thu, Apr 23, 2020 at 8:48 AM Ray Kinsella <mdr@ashroe.eu> wrote:
>
> Change references to abi 20.0.1 to use abi v21, add myself as the map
> file maintainer to more closely monitor future abi changes. Add
> suppressions that were missed on changes to librte_lpm.
About the lpm change below, this might be because you are using an old
version of libabigail.
I think the change went in:
$ git describe --contains 215b7eb4
libabigail-1.4~8
I use either libabigail 1.7 or the current master (when testing Dodji
enhancements).
> diff --git a/devtools/libabigail.abignore b/devtools/libabigail.abignore
> index cd86d89ca..20c821cee 100644
> --- a/devtools/libabigail.abignore
> +++ b/devtools/libabigail.abignore
> @@ -2,7 +2,6 @@
> symbol_version = EXPERIMENTAL
> [suppress_variable]
> symbol_version = EXPERIMENTAL
> -
> ; Explicit ignore for driver-only ABI
> [suppress_type]
> name = rte_cryptodev_ops
> @@ -18,3 +17,15 @@
> [suppress_type]
> type_kind = struct
> name = rte_event_ring
> +; Explicit ignore ABI 20.0.1
> +[suppress_function]
> + symbol_version = DPDK_20.0.1
> +[suppress_variable]
> + symbol_version = DPDK_20.0.1
> +; Explicit ignore of const lpm6_lookup change
> +[suppress_function]
> + name = rte_lpm6_delete
> + parameter = '1 uint8_t*
> +[suppress_function]
> + name = rte_lpm6_is_rule_present
> + parameter = '1 uint8_t*
--
David Marchand
^ permalink raw reply [relevance 6%]
* Re: [dpdk-dev] [PATCH v2] abi: change references to abi 20.0.1 to abi v21
2020-04-23 6:41 21% ` [dpdk-dev] [PATCH v2] " Ray Kinsella
@ 2020-04-24 9:15 6% ` Thomas Monjalon
2020-04-24 14:10 6% ` David Marchand
1 sibling, 0 replies; 200+ results
From: Thomas Monjalon @ 2020-04-24 9:15 UTC (permalink / raw)
To: Ray Kinsella
Cc: dev, aostruszka, stephen, Neil Horman, Jingjing Wu, Wenzhuo Lu,
Matan Azrad, Shahaf Shuler, Viacheslav Ovsiienko, Jerin Jacob,
Nithin Dabilpuram, Alfredo Cardigliano, Mahipal Challa,
Cristian Dumitrescu
23/04/2020 08:41, Ray Kinsella:
> Change references to abi 20.0.1 to use abi v21, add myself as the map
> file maintainer to more closely monitor future abi changes. Add
> suppressions that were missed on changes to librte_lpm.
>
> Signed-off-by: Ray Kinsella <mdr@ashroe.eu>
Please could you add a reference to the doc chapter stating this rule,
in the commit log? Thanks
^ permalink raw reply [relevance 6%]
* [dpdk-dev] DPDK Release Status Meeting 23/04/2020
@ 2020-04-23 13:08 4% Ferruh Yigit
0 siblings, 0 replies; 200+ results
From: Ferruh Yigit @ 2020-04-23 13:08 UTC (permalink / raw)
To: dpdk-dev; +Cc: Thomas Monjalon
Minutes 23 April 2020
---------------------
Agenda:
* Release Dates
* Subtrees
Participants:
* Arm
* Debian/Microsoft
* Intel
* Marvell
* Mellanox
* NXP
* Red Hat
Release Dates
-------------
* v20.05 dates:
* Integration/Merge/RC1 pushed to *Friday 24 April 2020*
* Release: Wednesday 20 May 2020
* PRC holiday on 1-5 May, need to take into account for validation effort
Subtrees
--------
* main
* crypto tree pulled, in progress of pulling next-net
* Not able to review rte_graph, planning to merge it in -rc2
* Merged ring patches from Konstantin and RCU patches from Honnappa
* Looking trace series, planning to have them in -rc1
* Some fixes for eal and vfio merged
* Checking fixes and changes related to ABI, from Ray and Neil
* gcc10 support
* Some fixes are missing
* Fixes can go after -rc1
* Need a long term plan for the new compiler support
* CI checks exists for new compilers
* Hash library not reviewed
* Some patches related atomic discussed in techboard,
not much to do for the release
* Windows, some patches was not ready postponed to 20.08
* Telemetry may go in -rc2
* There will be a new version, possibly today, to make it more generic
* Call to review for Windows patches
* next-net
* closed the tree yesterday, ready for -rc1
* next-crypto
* pulled for -rc1
* There are two existing issues
* Cryptodev ABI versioning, Fiona working on
* Not sure if it will be ready for -rc1
* rte_security issue, mainline is broken as of now
* Fix is in the mail list, should be merged quickly
* next-eventdev
* No more patches for -rc1, already pulled to main
* next-virtio
* Pulled for -rc1
* Some fixes for next -rc
* Mellanox vdpa queue stats patch postponed to next release
* Mellanox vdpa virtq patch requires rebase, it may wait next -rc
* Packet ring vectorized path patch under review, should be OK for -rc2
* Good to ask vector implementation from all architectures
* next-net-intel
* Most patches merged, only some fixes for -rc2
* fm10k still has build errors, postponed to -rc2
* next-net-mlx
* Some small fixes for -rc2
* next-net-mrvl
* All patches merged for -rc1
DPDK Release Status Meetings
============================
The DPDK Release Status Meeting is intended for DPDK Committers to discuss
the status of the master tree and sub-trees, and for project managers to
track progress or milestone dates.
The meeting occurs on Thursdays at 8:30 UTC. If you wish to attend just
send an email to "John McNamara <john.mcnamara@intel.com>" for the invite.
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCHv3] Remove validate-abi.sh from tree
2020-04-22 12:18 4% ` Thomas Monjalon
2020-04-22 13:19 4% ` Ray Kinsella
@ 2020-04-23 11:03 7% ` Neil Horman
1 sibling, 0 replies; 200+ results
From: Neil Horman @ 2020-04-23 11:03 UTC (permalink / raw)
To: Thomas Monjalon; +Cc: Ray Kinsella, dev, david.marchand
On Wed, Apr 22, 2020 at 02:18:05PM +0200, Thomas Monjalon wrote:
> 22/04/2020 14:07, Neil Horman:
> > On Wed, Apr 22, 2020 at 12:43:44PM +0100, Ray Kinsella wrote:
> > > On 21/04/2020 22:42, Thomas Monjalon wrote:
> > > > 21/04/2020 20:56, Neil Horman:
> > > >> On Tue, Apr 21, 2020 at 01:46:43PM +0200, Thomas Monjalon wrote:
> > > >>> 21/04/2020 13:12, Neil Horman:
> > > >>>> On Fri, Apr 17, 2020 at 04:42:38PM +0100, Ray Kinsella wrote:
> > > >>>>> On 17/04/2020 13:10, Thomas Monjalon wrote:
> > > >>>>>> 17/04/2020 13:47, Ray Kinsella:
> > > >>>>>>> On 17/04/2020 11:20, Thomas Monjalon wrote:
> > > >>>>>>>> 17/04/2020 12:11, Ray Kinsella:
> > > >>>>>>>>> check-abi.sh appears to be backward step in terms of usability.
> > > >>>>>>>>
> > > >>>>>>>> No, check-abi.sh benefits from a nice integration in build scripts.
> > > >>>>>>>> See below.
> > > >>>>>>>>
> > > >>>>>>>>> With validate-abi.sh I do can do a "validate-abi.sh HEAD~1 HEAD".
> > > >>>>>>>>> And it will do the build, install, dump and comparison for me.
> > > >>>>>>>>> And it picked up my 20.0.2 - > 21.0 changes no problem.
> > > >>>>>>>>>
> > > >>>>>>>>> With check-abi on the other hand, I need to the build and install myself.
> > > >>>>>>>>> check-abi requires dump files, but I see no reference in the documentation to how these are created.
> > > >>>>>>>>> It silently fails when it doesn't find any ...
> > > >>>>>>>>>
> > > >>>>>>>>> Do I run abi-dumper on the so's myself, or how does it work?
> > > >>>>>>>>
> > > >>>>>>>> check-abi.sh is integrated in test-build.sh and test-meson-builds.sh.
> > > >>>>>>>> Probably we should document usage in these scripts.
> > > >>>>>>>
> > > >>>>>>> Looks like I need to set DPDK_ABI_REF_VERSION=master, not obvious.
> > > >>>>>>> Any tips or tricks would be welcome.
> > > >>>>>>
> > > >>>>>> export DPDK_ABI_REF_VERSION=v20.02
> > > >>>>>> or
> > > >>>>>> export DPDK_ABI_REF_VERSION=v19.11
> > > >>>>>>
> > > >>>>>> Depends on which compatibility you want to test...
> > > >>>>>>
> > > >>>>>
> > > >>>>> Few things ...
> > > >>>>>
> > > >>>>> 1. test-meson-build.sh keep barfing complaining about reference paths.
> > > >>>>> ValueError: dst_dir must be absolute, got reference/v19.11/build-gcc-static/usr/local/share/dpdk/examples/bbdev_app
> > > >>>>>
> > > >>>>> Under the hood, ninja install is failing complaining that it needs an absolute path.
> > > >>>>> I fixed this in test_meson_build.sh and will send a patch in a minute.
> > > >>>>> Though it's strange no-one else has seen it?
> > > >>>>>
> > > >>>>> 2. test-meson-build.sh compares the abi for the static builds, which doesn't make any sense.
> > > >>>>>
> > > >>>>> 3. test-meson-build.sh will only take a branch in DPDK_ABI_REF_VERSION that exists locally.
> > > >>>>> In order to get it to compare HEAD against HEAD~1, which you would imagine is a pretty common case.
> > > >>>>> I had a create a branch for HEAD~1, in validate-abi this a pretty simple `validate-abi HEAD~1 HEAD`
> > > >>>>>
> > > >>>> I think this code in test-meson-build.sh should probably be fixed:
> > > >>>>
> > > >>>> if [ ! -d $abirefdir/src ]; then
> > > >>>> git clone --local --no-hardlinks \
> > > >>>> --single-branch \
> > > >>>> -b $DPDK_ABI_REF_VERSION \
> > > >>>> $srcdir $abirefdir/src
> > > >>>> fi
> > > >>>>
> > > >>>> Like you noted, using -b allows us to checkout a tag/branch in the cloned
> > > >>>> repository but requires that it exist locally. We should probably prefix the
> > > >>>> checkout with a git fetch --tags
> > > >>>
> > > >>> I don't understand your concern.
> > > >>> A reference is an older version, so it should be in the git tree.
> > > >>>
> > > >> yes, but not unless you've done a recent pull or fetch. If you set
> > > >> DPDK_ABI_REF_VERSION to a tag/branch that didn't exist as of the last time you
> > > >> updated the tree, it won't be there (which it sounds like what is being
> > > >> encountered here). You can fix that by doing a git pull or git fetch prior to
> > > >> running this script (or internal to the script)
> > > >
> > > > Sorry I still don't understand the case.
> > > > We want to compare the current version C with a reference R which is older.
> > > > If the reference R is not in the tree, it means the version C is not in the tree.
> > > > But C is the current version, so it is in the tree by definition.
> > > >
> > >
> > > So I can just relate my experience ....
> > >
> > > root@silpixa00395806:/build/dpdk# DPDK_ABI_REF_VERSION=HEAD~1 ./devtools/test-meson-builds.sh
> > > ninja -C ./build-gcc-static
> > > ninja: Entering directory `./build-gcc-static'
> > > [1766/2204] Compiling C object 'examples/c590b3c@@dpdk-vm_power_manager@exe/vm_power_manager_channel_monitor.c.o'.
> > > ../examples/vm_power_manager/channel_monitor.c:22:9: note: #pragma message: Jansson dev libs unavailable, not including JSON parsing
> > > #pragma message "Jansson dev libs unavailable, not including JSON parsing"
> > > ^~~~~~~
> > > [2204/2204] Linking target drivers/librte_pmd_softnic.so.20.0.2.
> > > Cloning into 'reference/HEAD~1/src'...
> > > warning: Could not find remote branch HEAD~1 to clone.
> > > fatal: Remote branch HEAD~1 not found in upstream origin
> > > fatal: The remote end hung up unexpectedly
> > >
> > Ah, So its not the problem i was describing, I think the problem you are seeing
> > is that the -b option only operates on branches and tags, not arbitrary git
> > revisions.
> >
> > To fix that, what we probably need to do is alter test-build.sh and
> > test-meson-build.sh such that the git clone operation is preceded by something
> > like this:
> > git tag ABI_CHECK_TAG $DPDK_ABI_REF_VERSION
> >
> > git clone ....
> >
> > git tag -d ABI_CHECK_TAG
> >
> > Doing so will guarantee that the source tree has a tag reference that the git
> > clone operation can use to do a checkout with a -b option on.
>
> I don't see the benefit of such test.
> Can we just document that the reference must be an existing tag?
>
We could, but like Ray said, if you want people to use this, you'll want to make
it easy for them. Many people will want to test the ABI against a well known
tag, but some are going to want to test against an arbitrary version (i.e.
against the merge base of their feature branch and the point they branched
from), which means they will attempt to check against an arbitrary hash value or
non-tag symbolic reference. Theres no reason we can't enable that, the only
reason it doesn't work is because of an ideosyncracy of the git branch command,
and the above fixes that.
Neil
>
>
^ permalink raw reply [relevance 7%]
* Re: [dpdk-dev] [PATCHv3] Remove validate-abi.sh from tree
2020-04-22 12:16 4% ` Thomas Monjalon
@ 2020-04-23 10:57 4% ` Neil Horman
0 siblings, 0 replies; 200+ results
From: Neil Horman @ 2020-04-23 10:57 UTC (permalink / raw)
To: Thomas Monjalon; +Cc: Ray Kinsella, dev, david.marchand
On Wed, Apr 22, 2020 at 02:16:57PM +0200, Thomas Monjalon wrote:
> 22/04/2020 14:01, Neil Horman:
> > On Tue, Apr 21, 2020 at 11:42:42PM +0200, Thomas Monjalon wrote:
> > > 21/04/2020 20:56, Neil Horman:
> > > > On Tue, Apr 21, 2020 at 01:46:43PM +0200, Thomas Monjalon wrote:
> > > > > 21/04/2020 13:12, Neil Horman:
> > > > > > On Fri, Apr 17, 2020 at 04:42:38PM +0100, Ray Kinsella wrote:
> > > > > > > On 17/04/2020 13:10, Thomas Monjalon wrote:
> > > > > > > > 17/04/2020 13:47, Ray Kinsella:
> > > > > > > >> On 17/04/2020 11:20, Thomas Monjalon wrote:
> > > > > > > >>> 17/04/2020 12:11, Ray Kinsella:
> > > > > > > >>>> check-abi.sh appears to be backward step in terms of usability.
> > > > > > > >>>
> > > > > > > >>> No, check-abi.sh benefits from a nice integration in build scripts.
> > > > > > > >>> See below.
> > > > > > > >>>
> > > > > > > >>>> With validate-abi.sh I do can do a "validate-abi.sh HEAD~1 HEAD".
> > > > > > > >>>> And it will do the build, install, dump and comparison for me.
> > > > > > > >>>> And it picked up my 20.0.2 - > 21.0 changes no problem.
> > > > > > > >>>>
> > > > > > > >>>> With check-abi on the other hand, I need to the build and install myself.
> > > > > > > >>>> check-abi requires dump files, but I see no reference in the documentation to how these are created.
> > > > > > > >>>> It silently fails when it doesn't find any ...
> > > > > > > >>>>
> > > > > > > >>>> Do I run abi-dumper on the so's myself, or how does it work?
> > > > > > > >>>
> > > > > > > >>> check-abi.sh is integrated in test-build.sh and test-meson-builds.sh.
> > > > > > > >>> Probably we should document usage in these scripts.
> > > > > > > >>
> > > > > > > >> Looks like I need to set DPDK_ABI_REF_VERSION=master, not obvious.
> > > > > > > >> Any tips or tricks would be welcome.
> > > > > > > >
> > > > > > > > export DPDK_ABI_REF_VERSION=v20.02
> > > > > > > > or
> > > > > > > > export DPDK_ABI_REF_VERSION=v19.11
> > > > > > > >
> > > > > > > > Depends on which compatibility you want to test...
> > > > > > > >
> > > > > > >
> > > > > > > Few things ...
> > > > > > >
> > > > > > > 1. test-meson-build.sh keep barfing complaining about reference paths.
> > > > > > > ValueError: dst_dir must be absolute, got reference/v19.11/build-gcc-static/usr/local/share/dpdk/examples/bbdev_app
> > > > > > >
> > > > > > > Under the hood, ninja install is failing complaining that it needs an absolute path.
> > > > > > > I fixed this in test_meson_build.sh and will send a patch in a minute.
> > > > > > > Though it's strange no-one else has seen it?
> > > > > > >
> > > > > > > 2. test-meson-build.sh compares the abi for the static builds, which doesn't make any sense.
> > > > > > >
> > > > > > > 3. test-meson-build.sh will only take a branch in DPDK_ABI_REF_VERSION that exists locally.
> > > > > > > In order to get it to compare HEAD against HEAD~1, which you would imagine is a pretty common case.
> > > > > > > I had a create a branch for HEAD~1, in validate-abi this a pretty simple `validate-abi HEAD~1 HEAD`
> > > > > > >
> > > > > > I think this code in test-meson-build.sh should probably be fixed:
> > > > > >
> > > > > > if [ ! -d $abirefdir/src ]; then
> > > > > > git clone --local --no-hardlinks \
> > > > > > --single-branch \
> > > > > > -b $DPDK_ABI_REF_VERSION \
> > > > > > $srcdir $abirefdir/src
> > > > > > fi
> > > > > >
> > > > > > Like you noted, using -b allows us to checkout a tag/branch in the cloned
> > > > > > repository but requires that it exist locally. We should probably prefix the
> > > > > > checkout with a git fetch --tags
> > > > >
> > > > > I don't understand your concern.
> > > > > A reference is an older version, so it should be in the git tree.
> > > > >
> > > > yes, but not unless you've done a recent pull or fetch. If you set
> > > > DPDK_ABI_REF_VERSION to a tag/branch that didn't exist as of the last time you
> > > > updated the tree, it won't be there (which it sounds like what is being
> > > > encountered here). You can fix that by doing a git pull or git fetch prior to
> > > > running this script (or internal to the script)
> > >
> > > Sorry I still don't understand the case.
> > > We want to compare the current version C with a reference R which is older.
> > > If the reference R is not in the tree, it means the version C is not in the tree.
> > > But C is the current version, so it is in the tree by definition.
> > >
> >
> >
> > +---+
> > | |
> > +---+ | | DPDK 20.11
> > | |Feature Branch commit +-+-+
> > | | |
> > +-+-+ +---+ +-v-+
> > | | | master HEAD | | master HEAD
> > | +-+-+ +-+-+
> > | | |
> > | | |
> > | +-v-+ +-v-+
> > +---->+ | DPDK 19.11 | | DPDK 19.11
> > +---+ +---+
> > X X X X
> > XXXXXXX XXXXXXXXX XXXXXXX XXXXXXXXX
> > XXX XX XXX XX
> > XXXX XXXX
> >
> > Local GIT copy DPDK.ORG
> >
> >
> >
> > If a user clones from dpdk.org when dpdk 19.11 is tagged in the tree, or any
> > time before dpdk 20.11 is tagged, then creates a feature branch, they may want
> > to compare their abi to the latest stable version. If they check dpdk.org and
> > see that dpdk 20.11 is the latest tag in the tree, they can run check-abi.sh
> > with the reference tag specified as 20.11, which exists in the dpdk.org tree,
> > but may not have been pulled into their local copy yet.
> >
> > I'm not saying this is definately what happened, but it would explain the
> > reported observations.
>
> OK now I understand.
> The user specified a reference which is not in his local tree.
> I vote for considering this case as an user mistake,
> and document it as a limitation of the tool integration in our build
> testing scripts.
>
I'd be fine with that, given that this turned out to not be rays issue.
Neil
>
>
^ permalink raw reply [relevance 4%]
* [dpdk-dev] [PATCH v1] abi: document reasons behind the three part versioning
@ 2020-04-23 10:12 24% Ray Kinsella
2020-05-04 21:46 4% ` Thomas Monjalon
0 siblings, 1 reply; 200+ results
From: Ray Kinsella @ 2020-04-23 10:12 UTC (permalink / raw)
To: dev
Cc: ferruh.yigit, Ray Kinsella, Bruce Richardson, John McNamara,
Marko Kovacevic
Clarify the reasons behind the three part version numbering scheme.
Documents the fixes made in f26c2b3.
Signed-off-by: Ray Kinsella <mdr@ashroe.eu>
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
doc/guides/contributing/abi_policy.rst | 3 ++-
doc/guides/rel_notes/release_20_05.rst | 12 ++++++++++++
2 files changed, 14 insertions(+), 1 deletion(-)
diff --git a/doc/guides/contributing/abi_policy.rst b/doc/guides/contributing/abi_policy.rst
index 05ca959..86e7dd9 100644
--- a/doc/guides/contributing/abi_policy.rst
+++ b/doc/guides/contributing/abi_policy.rst
@@ -39,7 +39,8 @@ General Guidelines
releases, over a number of release cycles. This change begins with
maintaining ABI stability through one year of DPDK releases starting from
DPDK 19.11. This policy will be reviewed in 2020, with intention of
- lengthening the stability period.
+ lengthening the stability period. Additional implementation detail can be
+ found in the :ref:`release notes <20_05_abi_changes>`.
What is an ABI?
~~~~~~~~~~~~~~~
diff --git a/doc/guides/rel_notes/release_20_05.rst b/doc/guides/rel_notes/release_20_05.rst
index 7f2049a..8653f7a 100644
--- a/doc/guides/rel_notes/release_20_05.rst
+++ b/doc/guides/rel_notes/release_20_05.rst
@@ -164,6 +164,7 @@ API Changes
Also, make sure to start the actual text at the margin.
=========================================================
+.. _20_05_abi_changes:
ABI Changes
-----------
@@ -180,6 +181,17 @@ ABI Changes
Also, make sure to start the actual text at the margin.
=========================================================
+* The soname for each stable ABI version should be just the ABI version major
+ number without the minor number. Unfortunately both major and minor were used
+ in the v19.11 release, causing version v20.x releases to be incompatible with
+ ABI v20.0.
+
+ The `commit f26c2b3
+ <https://git.dpdk.org/dpdk/commit/?id=f26c2b39b271cdcd857ba518c5e48c78cb1c30af>`_
+ fixed the issue by switching from 2-part to 3-part ABI version numbers so that
+ we can keep v20.0 as soname and using the final digits to identify the DPDK
+ 20.x releases which are ABI compatible.
+
* No ABI change that would break compatibility with DPDK 20.02 and 19.11.
--
2.7.4
^ permalink raw reply [relevance 24%]
* [dpdk-dev] [PATCH v2] abi: change references to abi 20.0.1 to abi v21
@ 2020-04-23 6:41 21% ` Ray Kinsella
2020-04-24 9:15 6% ` Thomas Monjalon
2020-04-24 14:10 6% ` David Marchand
2020-04-27 9:06 11% ` [dpdk-dev] [PATCH v3] " Ray Kinsella
` (3 subsequent siblings)
5 siblings, 2 replies; 200+ results
From: Ray Kinsella @ 2020-04-23 6:41 UTC (permalink / raw)
To: dev
Cc: aostruszka, stephen, Ray Kinsella, Thomas Monjalon, Neil Horman,
Jingjing Wu, Wenzhuo Lu, Matan Azrad, Shahaf Shuler,
Viacheslav Ovsiienko, Jerin Jacob, Nithin Dabilpuram,
Alfredo Cardigliano, Mahipal Challa, Cristian Dumitrescu
Change references to abi 20.0.1 to use abi v21, add myself as the map
file maintainer to more closely monitor future abi changes. Add
suppressions that were missed on changes to librte_lpm.
Signed-off-by: Ray Kinsella <mdr@ashroe.eu>
---
MAINTAINERS | 2 ++
devtools/libabigail.abignore | 13 ++++++++++++-
drivers/common/iavf/rte_common_iavf_version.map | 2 +-
drivers/common/mlx5/rte_common_mlx5_version.map | 2 +-
.../octeontx2/rte_common_octeontx2_version.map | 2 +-
drivers/net/ionic/rte_pmd_ionic_version.map | 2 +-
.../rte_rawdev_octeontx2_ep_version.map | 2 +-
drivers/vdpa/mlx5/rte_pmd_mlx5_vdpa_version.map | 2 +-
lib/librte_meter/rte_meter_version.map | 2 +-
9 files changed, 21 insertions(+), 8 deletions(-)
diff --git a/MAINTAINERS b/MAINTAINERS
index 7b81e2d1b..c24fd374d 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -86,6 +86,8 @@ F: doc/
ABI Policy
M: Ray Kinsella <mdr@ashroe.eu>
F: doc/guides/contributing/abi_*.rst
+F: drivers/*/*/*.map
+F: lib/*/*.map
Developers and Maintainers Tools
M: Thomas Monjalon <thomas@monjalon.net>
diff --git a/devtools/libabigail.abignore b/devtools/libabigail.abignore
index cd86d89ca..20c821cee 100644
--- a/devtools/libabigail.abignore
+++ b/devtools/libabigail.abignore
@@ -2,7 +2,6 @@
symbol_version = EXPERIMENTAL
[suppress_variable]
symbol_version = EXPERIMENTAL
-
; Explicit ignore for driver-only ABI
[suppress_type]
name = rte_cryptodev_ops
@@ -18,3 +17,15 @@
[suppress_type]
type_kind = struct
name = rte_event_ring
+; Explicit ignore ABI 20.0.1
+[suppress_function]
+ symbol_version = DPDK_20.0.1
+[suppress_variable]
+ symbol_version = DPDK_20.0.1
+; Explicit ignore of const lpm6_lookup change
+[suppress_function]
+ name = rte_lpm6_delete
+ parameter = '1 uint8_t*
+[suppress_function]
+ name = rte_lpm6_is_rule_present
+ parameter = '1 uint8_t*
diff --git a/drivers/common/iavf/rte_common_iavf_version.map b/drivers/common/iavf/rte_common_iavf_version.map
index 2f11d67c0..92ceac108 100644
--- a/drivers/common/iavf/rte_common_iavf_version.map
+++ b/drivers/common/iavf/rte_common_iavf_version.map
@@ -1,4 +1,4 @@
-DPDK_20.0.1 {
+DPDK_21 {
global:
iavf_init_adminq;
diff --git a/drivers/common/mlx5/rte_common_mlx5_version.map b/drivers/common/mlx5/rte_common_mlx5_version.map
index aede2a0a5..fe151fe0d 100644
--- a/drivers/common/mlx5/rte_common_mlx5_version.map
+++ b/drivers/common/mlx5/rte_common_mlx5_version.map
@@ -1,4 +1,4 @@
-DPDK_20.0.1 {
+DPDK_21 {
global:
mlx5_class_get;
diff --git a/drivers/common/octeontx2/rte_common_octeontx2_version.map b/drivers/common/octeontx2/rte_common_octeontx2_version.map
index 8f2404bd9..01279c339 100644
--- a/drivers/common/octeontx2/rte_common_octeontx2_version.map
+++ b/drivers/common/octeontx2/rte_common_octeontx2_version.map
@@ -34,7 +34,7 @@ DPDK_20.0 {
local: *;
};
-DPDK_20.0.1 {
+DPDK_21 {
global:
otx2_eth_dev_is_sec_capable;
diff --git a/drivers/net/ionic/rte_pmd_ionic_version.map b/drivers/net/ionic/rte_pmd_ionic_version.map
index bc8fd6d4d..acdaf587d 100644
--- a/drivers/net/ionic/rte_pmd_ionic_version.map
+++ b/drivers/net/ionic/rte_pmd_ionic_version.map
@@ -1,4 +1,4 @@
-DPDK_20.0.1 {
+DPDK_21 {
local: *;
};
diff --git a/drivers/raw/octeontx2_ep/rte_rawdev_octeontx2_ep_version.map b/drivers/raw/octeontx2_ep/rte_rawdev_octeontx2_ep_version.map
index bc8fd6d4d..acdaf587d 100644
--- a/drivers/raw/octeontx2_ep/rte_rawdev_octeontx2_ep_version.map
+++ b/drivers/raw/octeontx2_ep/rte_rawdev_octeontx2_ep_version.map
@@ -1,4 +1,4 @@
-DPDK_20.0.1 {
+DPDK_21 {
local: *;
};
diff --git a/drivers/vdpa/mlx5/rte_pmd_mlx5_vdpa_version.map b/drivers/vdpa/mlx5/rte_pmd_mlx5_vdpa_version.map
index 179f7f1ae..4a76d1d52 100644
--- a/drivers/vdpa/mlx5/rte_pmd_mlx5_vdpa_version.map
+++ b/drivers/vdpa/mlx5/rte_pmd_mlx5_vdpa_version.map
@@ -1,3 +1,3 @@
-DPDK_20.0.1 {
+DPDK_21 {
local: *;
};
diff --git a/lib/librte_meter/rte_meter_version.map b/lib/librte_meter/rte_meter_version.map
index fc12cc0bf..2c7dadbca 100644
--- a/lib/librte_meter/rte_meter_version.map
+++ b/lib/librte_meter/rte_meter_version.map
@@ -13,7 +13,7 @@ DPDK_20.0 {
local: *;
};
-DPDK_20.0.1 {
+DPDK_21 {
global:
rte_meter_trtcm_rfc4115_color_aware_check;
--
2.17.1
^ permalink raw reply [relevance 21%]
* [dpdk-dev] [PATCH v5 1/1] eal: add internal ABI marking support
2020-04-23 3:19 4% ` [dpdk-dev] [PATCH v5 0/1] dpdk: introduce __rte_internal tag Haiyue Wang
@ 2020-04-23 3:19 15% ` Haiyue Wang
2020-04-24 14:52 7% ` David Marchand
0 siblings, 1 reply; 200+ results
From: Haiyue Wang @ 2020-04-23 3:19 UTC (permalink / raw)
To: dev, thomas, david.marchand, bruce.richardson, ferruh.yigit,
nhorman, mdr
Cc: Haiyue Wang
Introduce __rte_internal tag to mark internal ABI function which is used
by the drivers or other libraries.
Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
---
buildtools/check-internal-syms.sh | 57 +++++++++++++++++++++++++++++
devtools/check-symbol-change.sh | 7 ++++
devtools/checkpatches.sh | 38 +++++++++++++++++++
devtools/libabigail.abignore | 5 +++
drivers/meson.build | 2 +-
lib/librte_eal/include/rte_compat.h | 13 +++++++
lib/meson.build | 2 +-
mk/internal/rte.compile-pre.mk | 3 ++
mk/target/generic/rte.vars.mk | 1 +
9 files changed, 126 insertions(+), 2 deletions(-)
create mode 100755 buildtools/check-internal-syms.sh
diff --git a/buildtools/check-internal-syms.sh b/buildtools/check-internal-syms.sh
new file mode 100755
index 000000000..5043acf87
--- /dev/null
+++ b/buildtools/check-internal-syms.sh
@@ -0,0 +1,57 @@
+#!/bin/sh
+
+# SPDX-License-Identifier: BSD-3-Clause
+
+MAPFILE=$1
+OBJFILE=$2
+
+LIST_SYMBOL=$(dirname $(readlink -f $0))/map-list-symbol.sh
+
+# added check for "make -C test/" usage
+if [ ! -e $MAPFILE ] || [ ! -f $OBJFILE ]
+then
+ exit 0
+fi
+
+if [ -d $MAPFILE ]
+then
+ exit 0
+fi
+
+DUMPFILE=$(mktemp -t dpdk.${0##*/}.XXX.objdump)
+trap 'rm -f "$DUMPFILE"' EXIT
+objdump -t $OBJFILE >$DUMPFILE
+
+ret=0
+for SYM in `$LIST_SYMBOL -S INTERNAL $MAPFILE |cut -d ' ' -f 3`
+do
+ if grep -q "\.text.*[[:space:]]$SYM$" $DUMPFILE &&
+ ! grep -q "\.text\.internal.*[[:space:]]$SYM$" $DUMPFILE
+ then
+ cat >&2 <<- END_OF_MESSAGE
+ $SYM is not flagged as internal
+ but is listed in version map
+ Please add __rte_internal to the definition of $SYM
+ END_OF_MESSAGE
+ ret=1
+ fi
+done
+
+# Filter out symbols suffixed with a . for icc
+for SYM in `awk '{
+ if ($2 != "l" && $4 == ".text.internal" && !($NF ~ /\.$/)) {
+ print $NF
+ }
+}' $DUMPFILE`
+do
+ $LIST_SYMBOL -S INTERNAL -s $SYM -q $MAPFILE || {
+ cat >&2 <<- END_OF_MESSAGE
+ $SYM is flagged as internal
+ but is not listed in version map
+ Please add $SYM to the version map
+ END_OF_MESSAGE
+ ret=1
+ }
+done
+
+exit $ret
diff --git a/devtools/check-symbol-change.sh b/devtools/check-symbol-change.sh
index ed2178e36..978979077 100755
--- a/devtools/check-symbol-change.sh
+++ b/devtools/check-symbol-change.sh
@@ -91,6 +91,13 @@ check_for_rule_violations()
if [ "$ar" = "add" ]
then
+ if [ "$secname" = "INTERNAL" ]
+ then
+ # these are absolved from any further checking
+ echo "Skipping symbol $symname in INTERNAL"
+ continue
+ fi
+
if [ "$secname" = "unknown" ]
then
# Just inform the user of this occurrence, but
diff --git a/devtools/checkpatches.sh b/devtools/checkpatches.sh
index c30ce64cc..2df8d7f2c 100755
--- a/devtools/checkpatches.sh
+++ b/devtools/checkpatches.sh
@@ -111,6 +111,36 @@ check_experimental_tags() { # <patch>
return $res
}
+check_internal_tags() { # <patch>
+ res=0
+
+ cat "$1" |awk '
+ BEGIN {
+ current_file = "";
+ ret = 0;
+ }
+ /^+++ b\// {
+ current_file = $2;
+ }
+ /^+.*__rte_internal/ {
+ if (current_file ~ ".c$" ) {
+ print "Please only put __rte_internal tags in " \
+ "headers ("current_file")";
+ ret = 1;
+ }
+ if ($1 != "+__rte_internal" || $2 != "") {
+ print "__rte_internal must appear alone on the line" \
+ " immediately preceding the return type of a function."
+ ret = 1;
+ }
+ }
+ END {
+ exit ret;
+ }' || res=1
+
+ return $res
+}
+
number=0
range='origin/master..'
quiet=false
@@ -194,6 +224,14 @@ check () { # <patch> <commit> <title>
ret=1
fi
+ ! $verbose || printf '\nChecking __rte_internal tags:\n'
+ report=$(check_internal_tags "$tmpinput")
+ if [ $? -ne 0 ] ; then
+ $headline_printed || print_headline "$3"
+ printf '%s\n' "$report"
+ ret=1
+ fi
+
if [ "$tmpinput" != "$1" ]; then
rm -f "$tmpinput"
trap - INT
diff --git a/devtools/libabigail.abignore b/devtools/libabigail.abignore
index cd86d89ca..3e8e2ea74 100644
--- a/devtools/libabigail.abignore
+++ b/devtools/libabigail.abignore
@@ -3,6 +3,11 @@
[suppress_variable]
symbol_version = EXPERIMENTAL
+[suppress_function]
+ symbol_version = INTERNAL
+[suppress_variable]
+ symbol_version = INTERNAL
+
; Explicit ignore for driver-only ABI
[suppress_type]
name = rte_cryptodev_ops
diff --git a/drivers/meson.build b/drivers/meson.build
index 4d8f842ab..cac07161f 100644
--- a/drivers/meson.build
+++ b/drivers/meson.build
@@ -20,7 +20,7 @@ dpdk_driver_classes = ['common',
disabled_drivers = run_command(list_dir_globs, get_option('disable_drivers'),
).stdout().split()
-default_cflags = machine_args + ['-DALLOW_EXPERIMENTAL_API']
+default_cflags = machine_args + ['-DALLOW_EXPERIMENTAL_API'] + ['-DALLOW_INTERNAL_API']
if cc.has_argument('-Wno-format-truncation')
default_cflags += '-Wno-format-truncation'
endif
diff --git a/lib/librte_eal/include/rte_compat.h b/lib/librte_eal/include/rte_compat.h
index 3eb33784b..4cd8f68d6 100644
--- a/lib/librte_eal/include/rte_compat.h
+++ b/lib/librte_eal/include/rte_compat.h
@@ -19,4 +19,17 @@ __attribute__((section(".text.experimental")))
#endif
+#ifndef ALLOW_INTERNAL_API
+
+#define __rte_internal \
+__attribute__((error("Symbol is not public ABI"), \
+section(".text.internal")))
+
+#else
+
+#define __rte_internal \
+__attribute__((section(".text.internal")))
+
+#endif
+
#endif /* _RTE_COMPAT_H_ */
diff --git a/lib/meson.build b/lib/meson.build
index c28b8df83..4d2f90d6a 100644
--- a/lib/meson.build
+++ b/lib/meson.build
@@ -38,7 +38,7 @@ if is_windows
libraries = ['kvargs','eal'] # only supported libraries for windows
endif
-default_cflags = machine_args + ['-DALLOW_EXPERIMENTAL_API']
+default_cflags = machine_args + ['-DALLOW_EXPERIMENTAL_API'] + ['-DALLOW_INTERNAL_API']
if cc.has_argument('-Wno-format-truncation')
default_cflags += '-Wno-format-truncation'
endif
diff --git a/mk/internal/rte.compile-pre.mk b/mk/internal/rte.compile-pre.mk
index 82fe098f7..0369786a5 100644
--- a/mk/internal/rte.compile-pre.mk
+++ b/mk/internal/rte.compile-pre.mk
@@ -58,6 +58,8 @@ C_TO_O_DISP = $(if $(V),"$(C_TO_O_STR)"," CC $(@)")
endif
EXPERIMENTAL_CHECK = $(RTE_SDK)/buildtools/check-experimental-syms.sh
CHECK_EXPERIMENTAL = $(EXPERIMENTAL_CHECK) $(SRCDIR)/$(EXPORT_MAP) $@
+INTERNAL_CHECK = $(RTE_SDK)/buildtools/check-internal-syms.sh
+CHECK_INTERNAL = $(INTERNAL_CHECK) $(SRCDIR)/$(EXPORT_MAP) $@
PMDINFO_GEN = $(RTE_SDK_BIN)/app/dpdk-pmdinfogen $@ $@.pmd.c
PMDINFO_CC = $(CC) $(CPPFLAGS) $(CFLAGS) $(EXTRA_CFLAGS) -c -o $@.pmd.o $@.pmd.c
@@ -76,6 +78,7 @@ C_TO_O_DO = @set -e; \
$(C_TO_O) && \
$(PMDINFO_TO_O) && \
$(CHECK_EXPERIMENTAL) && \
+ $(CHECK_INTERNAL) && \
echo $(C_TO_O_CMD) > $(call obj2cmd,$(@)) && \
sed 's,'$@':,dep_'$@' =,' $(call obj2dep,$(@)).tmp > $(call obj2dep,$(@)) && \
rm -f $(call obj2dep,$(@)).tmp
diff --git a/mk/target/generic/rte.vars.mk b/mk/target/generic/rte.vars.mk
index ec2672897..11b0418e5 100644
--- a/mk/target/generic/rte.vars.mk
+++ b/mk/target/generic/rte.vars.mk
@@ -106,6 +106,7 @@ ifeq ($(BUILDING_RTE_SDK),1)
# building sdk
CFLAGS += -include $(RTE_OUTPUT)/include/rte_config.h
CFLAGS += -DALLOW_EXPERIMENTAL_API
+CFLAGS += -DALLOW_INTERNAL_API
else
# if we are building an external application, include SDK's lib and
# includes too
--
2.26.2
^ permalink raw reply [relevance 15%]
* [dpdk-dev] [PATCH v5 0/1] dpdk: introduce __rte_internal tag
2020-04-22 13:52 4% ` [dpdk-dev] [PATCH v3 0/1] " Haiyue Wang
2020-04-22 16:37 4% ` [dpdk-dev] [PATCH v4 0/1] dpdk: introduce __rte_internal tag Haiyue Wang
@ 2020-04-23 3:19 4% ` Haiyue Wang
2020-04-23 3:19 15% ` [dpdk-dev] [PATCH v5 1/1] eal: add internal ABI marking support Haiyue Wang
2020-04-25 6:04 5% ` [dpdk-dev] [PATCH v6 0/6] dpdk: introduce __rte_internal tag Haiyue Wang
2020-04-25 10:56 5% ` [dpdk-dev] [PATCH v7 0/6] dpdk: introduce __rte_internal tag Haiyue Wang
4 siblings, 1 reply; 200+ results
From: Haiyue Wang @ 2020-04-23 3:19 UTC (permalink / raw)
To: dev, thomas, david.marchand, bruce.richardson, ferruh.yigit,
nhorman, mdr
Cc: Haiyue Wang
Move the internal function into INTERNAL session to avoid the ABI
checking, and it is only used for DPDK drivers or related library.
__rte_internal funA
INTERNAL {
global:
funA
};
v5: add the checkpatch for __rte_internal style
v4: add the ABI check suppression rules
v3: based on Neil's v2 patch https://patchwork.dpdk.org/cover/54771/
Use the ALLOW_INTERNAL_API to mark this new feature.
Haiyue Wang (1):
eal: add internal ABI marking support
buildtools/check-internal-syms.sh | 57 +++++++++++++++++++++++++++++
devtools/check-symbol-change.sh | 7 ++++
devtools/checkpatches.sh | 38 +++++++++++++++++++
devtools/libabigail.abignore | 5 +++
drivers/meson.build | 2 +-
lib/librte_eal/include/rte_compat.h | 13 +++++++
lib/meson.build | 2 +-
mk/internal/rte.compile-pre.mk | 3 ++
mk/target/generic/rte.vars.mk | 1 +
9 files changed, 126 insertions(+), 2 deletions(-)
create mode 100755 buildtools/check-internal-syms.sh
--
2.26.2
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH v3 10/10] eal/windows: implement basic memory management
@ 2020-04-23 1:00 3% ` Dmitry Kozlyuk
0 siblings, 0 replies; 200+ results
From: Dmitry Kozlyuk @ 2020-04-23 1:00 UTC (permalink / raw)
To: Ranjit Menon
Cc: dev, Dmitry Malloy (MESHCHANINOV),
Narcisa Ana Maria Vasile, Fady Bader, Tal Shnaiderman,
Thomas Monjalon, Harini Ramakrishnan, Omar Cardona, Kadam,
Pallavi, Mcnamara, John, Kovacevic, Marko, Burakov, Anatoly,
Richardson, Bruce
On 2020-04-16 11:34 GMT-0700 Ranjit Menon wrote:
[snip]
> > --- /dev/null
> > +++ b/lib/librte_eal/windows/include/rte_virt2phys.h
[snip]
>
> This file is a duplicate of: <kmods>: windows/virt2phys/virt2phys.h
> This is by design, since it documents the driver interface.
>
> So, to prevent the two files going out-of-sync, should we:
> 1. Make the two filenames the same?
> 2. Add comments to both files referencing each other and the need to
> keep them both in sync?
> 3. Do both (1) and (2)?
>
> This will also be an issue for the upcoming Windows netuio kernel driver
> and I reckon this could be an issue for Linux kernel modules too.
>
> Thoughts?
I guess it won't be much of an issue in practice. When you edit driver
interface you're already considering user-mode and kernel interaction, by
definition of that interface, so you can't forget about the counterpart.
I'd be more worried about ABI, because memory corruption in driver will crash
the system, being hard to debug. Can we leverage abigail for these interfaces?
--
Dmitry Kozlyuk
^ permalink raw reply [relevance 3%]
* [dpdk-dev] [PATCH v7 00/32] DPDK Trace support
@ 2020-04-22 19:03 1% ` jerinj
1 sibling, 0 replies; 200+ results
From: jerinj @ 2020-04-22 19:03 UTC (permalink / raw)
Cc: dev, thomas, bruce.richardson, david.marchand, mattias.ronnblom,
skori, Jerin Jacob
From: Jerin Jacob <jerinj@marvell.com>
v7
~~
1) Remove "meson: add libatomic as a global dependency for i686 clang"
patch from series as it already applied on master.
2) Doc improvements (Thomas)
Based on http://patches.dpdk.org/patch/68907/ comments.
3) Change tracepoint name(Thomas)
from:
rte_trace_<library_name>_[<domain>_]<name>
to:
rte_<library_name>_trace_[<domain>_]<name>
4) To align with item (3), the trace filename changed. Example:
from:
lib/librte_mempool/rte_trace_mempool.h
to:
lib/librte_mempool/rte_mempool_trace.h
5) change name of performance test case from "trace_perf" to
"test_perf_autotest"
to align with other performance test case name.
6) Add UT and performance test case in meson test infra(David)
7) Remove all the instance of if (.._is_enabled() == false) to if (!.._is_enabled())(David)
8) Fix missing alphabetical sorting in header files
in "eal/trace: implement trace operation APIs patch" (David)
9) trace_point_is_invalid() function cleanup(David)
V6
~~
Following changes are based on David's feedback.
1) Changed rte_trace_t to rte_trace_point_t
2) Reworked the header file to have following API name and filename changes
rte_trace.h has following trace control APIs
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
a) rte_trace_is_enabled()
b) rte_trace_mode_set()
c) rte_trace_mode_get()
d) rte_trace_pattern()
e) rte_trace_regexp()
f) rte_trace_save()
g) rte_trace_metadata_dump()
h) rte_trace_dump()
rte_trace_point.h has following tracepoint related APIs
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
a) rte_trace_point_enable()
b) rte_trace_point_disable()
c) rte_trace_point_is_enabled()
d) rte_trace_point_lookup()
e) rte_trace_point_emit_*()
f) RTE_TRACE_POINT_* macros
3) made rte_trace_fp_enabled() as internal.
4) Added common macros to have rte_trace_point_emit_* definition at
single place a.k.a more common code now.
5) git grep -l rte_trace_ctf |xargs sed -i -e
's/rte_trace_ctf_/rte_tracepoint_emit_/g'.
6) Added the reason new section for tracepoint in git commit log
7) Sorted symbols alphabetic order in rte_eal_version.map file.
8) Fix snprint return handling bug in __rte_trace_point_emit_field() function.
9) Doc: added newline for section titles.
10) ctf_field and ctf_count made it as static.
v5
~~
Following API rework based David and Thomas feedback.
1) Rename - "Shell pattern" to "Globbing pattern"
2) Remove the log "level" notion from trace library.
3) Remove rte_trace_global_[level/mode]* API from trace library.
4) EAL command line arg to --trace=regex/globing from --trace-level=regex/globing:level
5) Remove "@b EXPERIMENTAL: this API may change without prior notice" from each functions.
6) Use FP instead of DP for fastpath reference.
7) Updated documentation to reflect the above rework.
8) Updated UT to to reflect the above rework.
8) Updated UT for a FP trace point emit.
9) Updated performance test case for a FP trace point emission.
10) Updated git commit comments to reflect the above rework.
v4:
~~
1) Rebased to master.
2) Adapted to latest EAL directory structure change.
3) Fix possible build issue with out of tree application wherein
it does not define -DALLOW_EXPERIMENTAL_API. Fixed by making
fast path trace functions as NOP as it was getting included
in the inline functions of ethdev,mempool, cryptodev, etc(David)
4) Removed DALLOW_EXPERIMENTAL_API definition from individual driver files.
Now it set as global using http://patches.dpdk.org/patch/67758/ patch (David)
5) Added new meson option(-Denable_trace_dp=true)for enabling the datapath trace point (David, Bruce)
6) Changed the authorship and Rewrote the programmer's guide based on the below feedback(Thomas)
http://patches.dpdk.org/patch/67352/
v3:
~~
1) Fix the following build issues reported by CI
http://mails.dpdk.org/archives/test-report/2020-March/122060.html
a) clang + i686 meson build issue(Fixed in the the patch meson: add libatomic as a global dependency for i686 clang)
b) fixed build issue with FreeBSD with top of tree change.
c) fixed missing experimental API for iavf and ice with meson build in avx512 files.
v2:
~~
Addressed the following review comments from Mattias Rönnblom:
1) Changed
from:
typedef uint64_t* rte_trace_t;
to
typedef uint64_t rte_trace_t;
Initially thought to make the handle as
struct rte_trace {
uint64_t val;
}
but changed to uint64_t for the following reasons
a) It is opaque to the application and it will fix the compile-time type
check as well.
b) The handle has an index that will point to an internal slow-path
structure so no ABI change required in the future.
c) RTE_TRACE_POINT_DEFINE need to expose trace object. So it is better
to keep as uint64_t and avoid one more indirection for no use.
2)
Changed:
from:
enum rte_trace_mode_e {
to:
enum rte_trace_mode {
3) removed [out] "found" param from rte_trace_pattern() and
rte_trace_regexp()
4) Changed rte_trace_from_name to rte_trace_by_name
5) rte_trace_is_dp_enabled() return bool now
6) in __rte_trace_point_register() the argument fn change to register_fn
7) removed !! from rte_trace_is_enabled()
8) Remove uninitialized "rc warning" from rte_trace_pattern() and
rte_trace_regexp()
9) fixup bool return type for trace_entry_compare()
10) fixup calloc casting in trace_mkdir()
11) check fclose() return in trace_meta_save() and trace_mem_save()
12) rte_trace_ctf_* macro cleanup
13) added release notes
14) fix build issues reported by CI
http://mails.dpdk.org/archives/test-report/2020-March/121235.html
This patch set contains
~~~~~~~~~~~~~~~~~~~~~~~~
# The native implementation of common trace format(CTF)[1] based tracer
# Public API to create the trace points.
# Add tracepoints to eal, ethdev, mempool, eventdev and cryptodev
library for tracing support
# A unit test case
# Performance test case to measure the trace overhead. (See eal/trace:
# add trace performance test cases, patch)
# Programmers guide for Trace support(See doc: add trace library guide,
# patch)
# Tested OS:
~~~~~~~~~~~
- Linux
- FreeBSD
# Tested open source CTF trace viewers
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- Babeltrace
- Tracecompass
# Trace overhead comparison with LTTng
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
trace overhead data on x86:[2]
# 236 cycles with LTTng(>100ns)
# 18 cycles(7ns) with Native DPDK CTF emitter.(See eal/trace: add trace
# performance test cases patch)
trace overhead data on arm64:
# 312 cycles to 1100 cycles with LTTng based on the class of arm64 CPU.
# 11 cycles to 13 cycles with Native DPDK CTF emitter based on the
class of arm64 CPU.
18 cycles(on x86) vs 11 cycles(on arm64) is due to rdtsc() overhead in
x86. It seems rdtsc takes around 15cycles in x86.
More details:
~~~~~~~~~~~~~
# The Native DPDK CTF trace support does not have any dependency on
third-party library.
The generated output file is compatible with LTTng as both are using
CTF trace format.
The performance gain comes from:
1) exploit dpdk worker thread usage model to avoid atomics and use per
core variables
2) use hugepage,
3) avoid a lot function pointers in fast-path etc
4) avoid unaligned store for arm64 etc
Features:
~~~~~~~~~
- No specific limit on the events. A string-based event like rte_log
for pattern matching
- Dynamic enable/disable support.
- Instructmention overhead is ~1 cycle. i.e cost of adding the code
wth out using trace feature.
- Timestamp support for all the events using DPDK rte_rtdsc
- No dependency on another library. Clean room native implementation of CTF.
Functional test case:
a) echo "trace_autotest" | sudo ./build/app/test/dpdk-test -c 0x3 --trace=.*
The above command emits the following trace events
<code>
uint8_t i;
rte_trace_lib_eal_generic_void();
rte_trace_lib_eal_generic_u64(0x10000000000000);
rte_trace_lib_eal_generic_u32(0x10000000);
rte_trace_lib_eal_generic_u16(0xffee);
rte_trace_lib_eal_generic_u8(0xc);
rte_trace_lib_eal_generic_i64(-1234);
rte_trace_lib_eal_generic_i32(-1234567);
rte_trace_lib_eal_generic_i16(12);
rte_trace_lib_eal_generic_i8(-3);
rte_trace_lib_eal_generic_string("my string");
rte_trace_lib_eal_generic_function(__func__);
</code>
Install babeltrace package in Linux and point the generated trace file
to babel trace. By default trace file created under
<user>/dpdk-traces/time_stamp/
example:
# babeltrace /root/dpdk-traces/rte-2020-02-15-PM-02-56-51 | more
[13:27:36.138468807] (+?.?????????) lib.eal.generic.void: { cpu_id =0, name = "dpdk-test" }, { }
[13:27:36.138468851] (+0.000000044) lib.eal.generic.u64: { cpu_id = 0, name = "dpdk-test" }, { in = 4503599627370496 }
[13:27:36.138468860] (+0.000000009) lib.eal.generic.u32: { cpu_id = 0, name = "dpdk-test" }, { in = 268435456 }
[13:27:36.138468934] (+0.000000074) lib.eal.generic.u16: { cpu_id = 0, name = "dpdk-test" }, { in = 65518 }
[13:27:36.138468949] (+0.000000015) lib.eal.generic.u8: { cpu_id = 0, name = "dpdk-test" }, { in = 12 }
[13:27:36.138468956] (+0.000000007) lib.eal.generic.i64: { cpu_id = 0, name = "dpdk-test" }, { in = -1234 }
[13:27:36.138468963] (+0.000000007) lib.eal.generic.i32: { cpu_id = 0, name = "dpdk-test" }, { in = -1234567 }
[13:27:36.138469024] (+0.000000061) lib.eal.generic.i16: { cpu_id = 0, name = "dpdk-test" }, { in = 12 }
[13:27:36.138469044] (+0.000000020) lib.eal.generic.i8: { cpu_id = 0, name = "dpdk-test" }, { in = -3 }
[13:27:36.138469051] (+0.000000007) lib.eal.generic.string: { cpu_id = 0, name = "dpdk-test" }, { str = "my string" }
[13:27:36.138469203] (+0.000000152) lib.eal.generic.func: { cpu_id = 0, name = "dpdk-test" }, { func = "test_trace_points" }
# There is a GUI based trace viewer available in Windows, Linux and Mac.
It is called as tracecompass.(https://www.eclipse.org/tracecompass/)
The example screenshot and Histogram of above DPDK trace using
Tracecompass.
https://github.com/jerinjacobk/share/blob/master/dpdk_trace.JPG
File walk through:
~~~~~~~~~~~~~~~~~~
lib/librte_eal/common/include/rte_trace.h - Public API for Trace
provider and Trace control
lib/librte_eal/common/eal_common_trace.c - main trace implementation
lib/librte_eal/common/eal_common_trace_ctf.c - CTF metadata spec
implementation
lib/librte_eal/common/eal_common_trace_utils.c - command line utils
and filesystem operations.
lib/librte_eal/common/eal_common_trace_points.c - trace points for EAL
library
lib/librte_eal/common/include/rte_trace_eal.h - EAL tracepoint public
API.
lib/librte_eal/common/eal_trace.h - Private trace header file.
[1] https://diamon.org/ctf/
[2] The above test is ported to LTTng for finding the LTTng trace
overhead. It available at
https://github.com/jerinjacobk/lttng-overhead
https://github.com/jerinjacobk/lttng-overhead/blob/master/README
Jerin Jacob (22):
eal: introduce API for getting thread name
eal/trace: define the public API for trace support
eal/trace: implement trace register API
eal/trace: implement trace operation APIs
eal/trace: add internal trace init and fini interface
eal/trace: get bootup timestamp for trace
eal/trace: create CTF TDSL metadata in memory
eal/trace: implement trace memory allocation
eal/trace: implement debug dump function
eal/trace: implement trace save
eal/trace: implement registration payload
eal/trace: implement provider payload
eal/trace: hook internal trace APIs to Linux
eal/trace: hook internal trace APIs to FreeBSD
eal/trace: add generic tracepoints
eal/trace: add alarm tracepoints
eal/trace: add memory tracepoints
eal/trace: add memzone tracepoints
eal/trace: add thread tracepoints
eal/trace: add interrupt tracepoints
eal/trace: add trace performance test cases
doc: add trace library guide
Sunil Kumar Kori (10):
eal/trace: handle CTF keyword collision
eal/trace: add trace configuration parameter
eal/trace: add trace dir configuration parameter
eal/trace: add trace bufsize configuration parameter
eal/trace: add trace mode configuration parameter
eal/trace: add unit test cases
ethdev: add tracepoints
eventdev: add tracepoints
cryptodev: add tracepoints
mempool: add tracepoints
MAINTAINERS | 8 +
app/test/Makefile | 4 +-
app/test/meson.build | 5 +
app/test/test_trace.c | 213 ++++++++
app/test/test_trace.h | 15 +
app/test/test_trace_perf.c | 183 +++++++
app/test/test_trace_register.c | 17 +
config/common_base | 1 +
config/meson.build | 1 +
doc/api/doxy-api-index.md | 4 +-
doc/guides/linux_gsg/eal_args.include.rst | 52 ++
doc/guides/prog_guide/build-sdk-meson.rst | 5 +
doc/guides/prog_guide/index.rst | 1 +
doc/guides/prog_guide/trace_lib.rst | 348 ++++++++++++
doc/guides/rel_notes/release_20_05.rst | 9 +
examples/cmdline/Makefile | 1 +
examples/cmdline/meson.build | 1 +
examples/distributor/Makefile | 1 +
examples/distributor/meson.build | 1 +
examples/ethtool/ethtool-app/Makefile | 1 +
examples/eventdev_pipeline/meson.build | 1 +
examples/flow_filtering/Makefile | 1 +
examples/flow_filtering/meson.build | 1 +
examples/helloworld/Makefile | 1 +
examples/helloworld/meson.build | 1 +
examples/ioat/Makefile | 1 +
examples/ioat/meson.build | 1 +
examples/ip_fragmentation/Makefile | 2 +
examples/ip_fragmentation/meson.build | 1 +
examples/ip_reassembly/Makefile | 1 +
examples/ip_reassembly/meson.build | 1 +
examples/ipv4_multicast/Makefile | 1 +
examples/ipv4_multicast/meson.build | 1 +
examples/l2fwd-cat/Makefile | 1 +
examples/l2fwd-cat/meson.build | 1 +
examples/l2fwd-event/Makefile | 1 +
examples/l2fwd-event/meson.build | 1 +
examples/l2fwd-jobstats/Makefile | 1 +
examples/l2fwd-jobstats/meson.build | 1 +
examples/l2fwd-keepalive/Makefile | 1 +
examples/l2fwd-keepalive/ka-agent/Makefile | 1 +
examples/l2fwd-keepalive/meson.build | 1 +
examples/l3fwd-acl/Makefile | 1 +
examples/l3fwd-acl/meson.build | 1 +
examples/l3fwd/Makefile | 1 +
examples/l3fwd/meson.build | 1 +
examples/link_status_interrupt/Makefile | 1 +
examples/link_status_interrupt/meson.build | 1 +
.../client_server_mp/mp_client/Makefile | 1 +
.../client_server_mp/mp_client/meson.build | 1 +
.../client_server_mp/mp_server/meson.build | 1 +
examples/multi_process/hotplug_mp/Makefile | 1 +
examples/multi_process/hotplug_mp/meson.build | 1 +
examples/multi_process/simple_mp/Makefile | 1 +
examples/multi_process/simple_mp/meson.build | 1 +
examples/multi_process/symmetric_mp/Makefile | 1 +
.../multi_process/symmetric_mp/meson.build | 1 +
examples/ntb/Makefile | 1 +
examples/ntb/meson.build | 1 +
examples/packet_ordering/Makefile | 1 +
examples/packet_ordering/meson.build | 1 +
.../performance-thread/l3fwd-thread/Makefile | 1 +
.../l3fwd-thread/meson.build | 1 +
.../performance-thread/pthread_shim/Makefile | 1 +
.../pthread_shim/meson.build | 1 +
examples/ptpclient/Makefile | 1 +
examples/ptpclient/meson.build | 1 +
examples/qos_meter/Makefile | 1 +
examples/qos_meter/meson.build | 1 +
examples/qos_sched/Makefile | 1 +
examples/qos_sched/meson.build | 1 +
examples/server_node_efd/node/Makefile | 1 +
examples/server_node_efd/node/meson.build | 1 +
examples/server_node_efd/server/Makefile | 1 +
examples/server_node_efd/server/meson.build | 1 +
examples/service_cores/Makefile | 1 +
examples/service_cores/meson.build | 1 +
examples/skeleton/Makefile | 1 +
examples/skeleton/meson.build | 1 +
examples/timer/Makefile | 1 +
examples/timer/meson.build | 1 +
examples/vm_power_manager/Makefile | 1 +
examples/vm_power_manager/meson.build | 1 +
examples/vmdq/Makefile | 1 +
examples/vmdq/meson.build | 1 +
examples/vmdq_dcb/Makefile | 1 +
examples/vmdq_dcb/meson.build | 1 +
lib/librte_cryptodev/Makefile | 4 +-
lib/librte_cryptodev/cryptodev_trace_points.c | 70 +++
lib/librte_cryptodev/meson.build | 6 +-
lib/librte_cryptodev/rte_cryptodev.c | 18 +
lib/librte_cryptodev/rte_cryptodev.h | 6 +
lib/librte_cryptodev/rte_cryptodev_trace.h | 148 ++++++
lib/librte_cryptodev/rte_cryptodev_trace_fp.h | 38 ++
.../rte_cryptodev_version.map | 18 +
lib/librte_eal/common/eal_common_memzone.c | 9 +
lib/librte_eal/common/eal_common_options.c | 65 +++
lib/librte_eal/common/eal_common_thread.c | 4 +-
lib/librte_eal/common/eal_common_trace.c | 494 ++++++++++++++++++
lib/librte_eal/common/eal_common_trace_ctf.c | 488 +++++++++++++++++
.../common/eal_common_trace_points.c | 115 ++++
.../common/eal_common_trace_utils.c | 478 +++++++++++++++++
lib/librte_eal/common/eal_options.h | 8 +
lib/librte_eal/common/eal_private.h | 5 +
lib/librte_eal/common/eal_trace.h | 122 +++++
lib/librte_eal/common/meson.build | 4 +
lib/librte_eal/common/rte_malloc.c | 60 ++-
lib/librte_eal/freebsd/Makefile | 4 +
lib/librte_eal/freebsd/eal.c | 10 +
lib/librte_eal/freebsd/eal_alarm.c | 3 +
lib/librte_eal/freebsd/eal_interrupts.c | 54 +-
lib/librte_eal/freebsd/eal_thread.c | 21 +-
lib/librte_eal/include/meson.build | 5 +
lib/librte_eal/include/rte_eal_trace.h | 278 ++++++++++
lib/librte_eal/include/rte_lcore.h | 17 +
lib/librte_eal/include/rte_trace.h | 141 +++++
lib/librte_eal/include/rte_trace_point.h | 306 +++++++++++
.../include/rte_trace_point_provider.h | 131 +++++
.../include/rte_trace_point_register.h | 39 ++
lib/librte_eal/linux/Makefile | 4 +
lib/librte_eal/linux/eal.c | 9 +
lib/librte_eal/linux/eal_alarm.c | 4 +
lib/librte_eal/linux/eal_interrupts.c | 84 +--
lib/librte_eal/linux/eal_thread.c | 27 +-
lib/librte_eal/rte_eal_version.map | 50 ++
lib/librte_ethdev/Makefile | 3 +
lib/librte_ethdev/ethdev_trace_points.c | 43 ++
lib/librte_ethdev/meson.build | 5 +-
lib/librte_ethdev/rte_ethdev.c | 12 +
lib/librte_ethdev/rte_ethdev.h | 5 +
lib/librte_ethdev/rte_ethdev_trace.h | 97 ++++
lib/librte_ethdev/rte_ethdev_trace_fp.h | 44 ++
lib/librte_ethdev/rte_ethdev_version.map | 10 +
lib/librte_eventdev/Makefile | 3 +
lib/librte_eventdev/eventdev_trace_points.c | 173 ++++++
lib/librte_eventdev/meson.build | 3 +
.../rte_event_crypto_adapter.c | 10 +
.../rte_event_eth_rx_adapter.c | 11 +
.../rte_event_eth_tx_adapter.c | 13 +-
.../rte_event_eth_tx_adapter.h | 2 +
lib/librte_eventdev/rte_event_timer_adapter.c | 8 +-
lib/librte_eventdev/rte_event_timer_adapter.h | 8 +
lib/librte_eventdev/rte_eventdev.c | 9 +
lib/librte_eventdev/rte_eventdev.h | 5 +-
lib/librte_eventdev/rte_eventdev_trace.h | 311 +++++++++++
lib/librte_eventdev/rte_eventdev_trace_fp.h | 85 +++
lib/librte_eventdev/rte_eventdev_version.map | 42 ++
lib/librte_mempool/Makefile | 3 +
lib/librte_mempool/mempool_trace_points.c | 108 ++++
lib/librte_mempool/meson.build | 5 +-
lib/librte_mempool/rte_mempool.c | 16 +
lib/librte_mempool/rte_mempool.h | 13 +
lib/librte_mempool/rte_mempool_ops.c | 7 +
lib/librte_mempool/rte_mempool_trace.h | 178 +++++++
lib/librte_mempool/rte_mempool_trace_fp.h | 116 ++++
lib/librte_mempool/rte_mempool_version.map | 26 +
meson_options.txt | 2 +
157 files changed, 5587 insertions(+), 75 deletions(-)
create mode 100644 app/test/test_trace.c
create mode 100644 app/test/test_trace.h
create mode 100644 app/test/test_trace_perf.c
create mode 100644 app/test/test_trace_register.c
create mode 100644 doc/guides/prog_guide/trace_lib.rst
create mode 100644 lib/librte_cryptodev/cryptodev_trace_points.c
create mode 100644 lib/librte_cryptodev/rte_cryptodev_trace.h
create mode 100644 lib/librte_cryptodev/rte_cryptodev_trace_fp.h
create mode 100644 lib/librte_eal/common/eal_common_trace.c
create mode 100644 lib/librte_eal/common/eal_common_trace_ctf.c
create mode 100644 lib/librte_eal/common/eal_common_trace_points.c
create mode 100644 lib/librte_eal/common/eal_common_trace_utils.c
create mode 100644 lib/librte_eal/common/eal_trace.h
create mode 100644 lib/librte_eal/include/rte_eal_trace.h
create mode 100644 lib/librte_eal/include/rte_trace.h
create mode 100644 lib/librte_eal/include/rte_trace_point.h
create mode 100644 lib/librte_eal/include/rte_trace_point_provider.h
create mode 100644 lib/librte_eal/include/rte_trace_point_register.h
create mode 100644 lib/librte_ethdev/ethdev_trace_points.c
create mode 100644 lib/librte_ethdev/rte_ethdev_trace.h
create mode 100644 lib/librte_ethdev/rte_ethdev_trace_fp.h
create mode 100644 lib/librte_eventdev/eventdev_trace_points.c
create mode 100644 lib/librte_eventdev/rte_eventdev_trace.h
create mode 100644 lib/librte_eventdev/rte_eventdev_trace_fp.h
create mode 100644 lib/librte_mempool/mempool_trace_points.c
create mode 100644 lib/librte_mempool/rte_mempool_trace.h
create mode 100644 lib/librte_mempool/rte_mempool_trace_fp.h
--
2.25.1
^ permalink raw reply [relevance 1%]
* Re: [dpdk-dev] [PATCH v3 1/1] eal: add internal ABI mark support
2020-04-22 14:13 4% ` David Marchand
@ 2020-04-22 16:44 4% ` Wang, Haiyue
0 siblings, 0 replies; 200+ results
From: Wang, Haiyue @ 2020-04-22 16:44 UTC (permalink / raw)
To: David Marchand
Cc: dev, Thomas Monjalon, Richardson, Bruce, Yigit, Ferruh, Ray Kinsella
> -----Original Message-----
> From: David Marchand <david.marchand@redhat.com>
> Sent: Wednesday, April 22, 2020 22:14
> To: Wang, Haiyue <haiyue.wang@intel.com>
> Cc: dev <dev@dpdk.org>; Thomas Monjalon <thomas@monjalon.net>; Richardson, Bruce
> <bruce.richardson@intel.com>; Yigit, Ferruh <ferruh.yigit@intel.com>; Ray Kinsella <mdr@ashroe.eu>
> Subject: Re: [PATCH v3 1/1] eal: add internal ABI mark support
>
> On Wed, Apr 22, 2020 at 3:58 PM Haiyue Wang <haiyue.wang@intel.com> wrote:
> >
> > Introduce __rte_internal tag to mark internal ABI function, this kind of
> > function can't be called by external application.
> >
> > Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
> > ---
> > buildtools/check-internal-syms.sh | 57 +++++++++++++++++++++++++++++
> > devtools/check-symbol-change.sh | 7 ++++
> > drivers/meson.build | 2 +-
> > lib/librte_eal/include/rte_compat.h | 13 +++++++
> > lib/meson.build | 2 +-
> > mk/internal/rte.compile-pre.mk | 3 ++
> > mk/target/generic/rte.vars.mk | 1 +
> > 7 files changed, 83 insertions(+), 2 deletions(-)
> > create mode 100755 buildtools/check-internal-syms.sh
>
> I did not enter the details, but I suppose this patch lacks an update
> on the suppression rules so that INTERNAL versioned symbols are
> skipped.
> https://git.dpdk.org/dpdk/tree/devtools/libabigail.abignore
>
>
Added it in v4.
> --
> David Marchand
^ permalink raw reply [relevance 4%]
* [dpdk-dev] [PATCH v4 1/1] eal: add internal ABI mark support
2020-04-22 16:37 4% ` [dpdk-dev] [PATCH v4 0/1] dpdk: introduce __rte_internal tag Haiyue Wang
@ 2020-04-22 16:37 15% ` Haiyue Wang
0 siblings, 0 replies; 200+ results
From: Haiyue Wang @ 2020-04-22 16:37 UTC (permalink / raw)
To: dev, thomas, david.marchand, bruce.richardson, ferruh.yigit,
nhorman, mdr
Cc: Haiyue Wang
Introduce __rte_internal tag to mark internal ABI function, this kind of
function can't be called by external application.
Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
---
buildtools/check-internal-syms.sh | 57 +++++++++++++++++++++++++++++
devtools/check-symbol-change.sh | 7 ++++
devtools/libabigail.abignore | 5 +++
drivers/meson.build | 2 +-
lib/librte_eal/include/rte_compat.h | 13 +++++++
lib/meson.build | 2 +-
mk/internal/rte.compile-pre.mk | 3 ++
mk/target/generic/rte.vars.mk | 1 +
8 files changed, 88 insertions(+), 2 deletions(-)
create mode 100755 buildtools/check-internal-syms.sh
diff --git a/buildtools/check-internal-syms.sh b/buildtools/check-internal-syms.sh
new file mode 100755
index 000000000..5043acf87
--- /dev/null
+++ b/buildtools/check-internal-syms.sh
@@ -0,0 +1,57 @@
+#!/bin/sh
+
+# SPDX-License-Identifier: BSD-3-Clause
+
+MAPFILE=$1
+OBJFILE=$2
+
+LIST_SYMBOL=$(dirname $(readlink -f $0))/map-list-symbol.sh
+
+# added check for "make -C test/" usage
+if [ ! -e $MAPFILE ] || [ ! -f $OBJFILE ]
+then
+ exit 0
+fi
+
+if [ -d $MAPFILE ]
+then
+ exit 0
+fi
+
+DUMPFILE=$(mktemp -t dpdk.${0##*/}.XXX.objdump)
+trap 'rm -f "$DUMPFILE"' EXIT
+objdump -t $OBJFILE >$DUMPFILE
+
+ret=0
+for SYM in `$LIST_SYMBOL -S INTERNAL $MAPFILE |cut -d ' ' -f 3`
+do
+ if grep -q "\.text.*[[:space:]]$SYM$" $DUMPFILE &&
+ ! grep -q "\.text\.internal.*[[:space:]]$SYM$" $DUMPFILE
+ then
+ cat >&2 <<- END_OF_MESSAGE
+ $SYM is not flagged as internal
+ but is listed in version map
+ Please add __rte_internal to the definition of $SYM
+ END_OF_MESSAGE
+ ret=1
+ fi
+done
+
+# Filter out symbols suffixed with a . for icc
+for SYM in `awk '{
+ if ($2 != "l" && $4 == ".text.internal" && !($NF ~ /\.$/)) {
+ print $NF
+ }
+}' $DUMPFILE`
+do
+ $LIST_SYMBOL -S INTERNAL -s $SYM -q $MAPFILE || {
+ cat >&2 <<- END_OF_MESSAGE
+ $SYM is flagged as internal
+ but is not listed in version map
+ Please add $SYM to the version map
+ END_OF_MESSAGE
+ ret=1
+ }
+done
+
+exit $ret
diff --git a/devtools/check-symbol-change.sh b/devtools/check-symbol-change.sh
index ed2178e36..978979077 100755
--- a/devtools/check-symbol-change.sh
+++ b/devtools/check-symbol-change.sh
@@ -91,6 +91,13 @@ check_for_rule_violations()
if [ "$ar" = "add" ]
then
+ if [ "$secname" = "INTERNAL" ]
+ then
+ # these are absolved from any further checking
+ echo "Skipping symbol $symname in INTERNAL"
+ continue
+ fi
+
if [ "$secname" = "unknown" ]
then
# Just inform the user of this occurrence, but
diff --git a/devtools/libabigail.abignore b/devtools/libabigail.abignore
index cd86d89ca..3e8e2ea74 100644
--- a/devtools/libabigail.abignore
+++ b/devtools/libabigail.abignore
@@ -3,6 +3,11 @@
[suppress_variable]
symbol_version = EXPERIMENTAL
+[suppress_function]
+ symbol_version = INTERNAL
+[suppress_variable]
+ symbol_version = INTERNAL
+
; Explicit ignore for driver-only ABI
[suppress_type]
name = rte_cryptodev_ops
diff --git a/drivers/meson.build b/drivers/meson.build
index 4d8f842ab..cac07161f 100644
--- a/drivers/meson.build
+++ b/drivers/meson.build
@@ -20,7 +20,7 @@ dpdk_driver_classes = ['common',
disabled_drivers = run_command(list_dir_globs, get_option('disable_drivers'),
).stdout().split()
-default_cflags = machine_args + ['-DALLOW_EXPERIMENTAL_API']
+default_cflags = machine_args + ['-DALLOW_EXPERIMENTAL_API'] + ['-DALLOW_INTERNAL_API']
if cc.has_argument('-Wno-format-truncation')
default_cflags += '-Wno-format-truncation'
endif
diff --git a/lib/librte_eal/include/rte_compat.h b/lib/librte_eal/include/rte_compat.h
index 3eb33784b..4cd8f68d6 100644
--- a/lib/librte_eal/include/rte_compat.h
+++ b/lib/librte_eal/include/rte_compat.h
@@ -19,4 +19,17 @@ __attribute__((section(".text.experimental")))
#endif
+#ifndef ALLOW_INTERNAL_API
+
+#define __rte_internal \
+__attribute__((error("Symbol is not public ABI"), \
+section(".text.internal")))
+
+#else
+
+#define __rte_internal \
+__attribute__((section(".text.internal")))
+
+#endif
+
#endif /* _RTE_COMPAT_H_ */
diff --git a/lib/meson.build b/lib/meson.build
index 63c17ee75..981c2e397 100644
--- a/lib/meson.build
+++ b/lib/meson.build
@@ -36,7 +36,7 @@ if is_windows
libraries = ['kvargs','eal'] # only supported libraries for windows
endif
-default_cflags = machine_args + ['-DALLOW_EXPERIMENTAL_API']
+default_cflags = machine_args + ['-DALLOW_EXPERIMENTAL_API'] + ['-DALLOW_INTERNAL_API']
if cc.has_argument('-Wno-format-truncation')
default_cflags += '-Wno-format-truncation'
endif
diff --git a/mk/internal/rte.compile-pre.mk b/mk/internal/rte.compile-pre.mk
index 82fe098f7..0369786a5 100644
--- a/mk/internal/rte.compile-pre.mk
+++ b/mk/internal/rte.compile-pre.mk
@@ -58,6 +58,8 @@ C_TO_O_DISP = $(if $(V),"$(C_TO_O_STR)"," CC $(@)")
endif
EXPERIMENTAL_CHECK = $(RTE_SDK)/buildtools/check-experimental-syms.sh
CHECK_EXPERIMENTAL = $(EXPERIMENTAL_CHECK) $(SRCDIR)/$(EXPORT_MAP) $@
+INTERNAL_CHECK = $(RTE_SDK)/buildtools/check-internal-syms.sh
+CHECK_INTERNAL = $(INTERNAL_CHECK) $(SRCDIR)/$(EXPORT_MAP) $@
PMDINFO_GEN = $(RTE_SDK_BIN)/app/dpdk-pmdinfogen $@ $@.pmd.c
PMDINFO_CC = $(CC) $(CPPFLAGS) $(CFLAGS) $(EXTRA_CFLAGS) -c -o $@.pmd.o $@.pmd.c
@@ -76,6 +78,7 @@ C_TO_O_DO = @set -e; \
$(C_TO_O) && \
$(PMDINFO_TO_O) && \
$(CHECK_EXPERIMENTAL) && \
+ $(CHECK_INTERNAL) && \
echo $(C_TO_O_CMD) > $(call obj2cmd,$(@)) && \
sed 's,'$@':,dep_'$@' =,' $(call obj2dep,$(@)).tmp > $(call obj2dep,$(@)) && \
rm -f $(call obj2dep,$(@)).tmp
diff --git a/mk/target/generic/rte.vars.mk b/mk/target/generic/rte.vars.mk
index ec2672897..11b0418e5 100644
--- a/mk/target/generic/rte.vars.mk
+++ b/mk/target/generic/rte.vars.mk
@@ -106,6 +106,7 @@ ifeq ($(BUILDING_RTE_SDK),1)
# building sdk
CFLAGS += -include $(RTE_OUTPUT)/include/rte_config.h
CFLAGS += -DALLOW_EXPERIMENTAL_API
+CFLAGS += -DALLOW_INTERNAL_API
else
# if we are building an external application, include SDK's lib and
# includes too
--
2.26.2
^ permalink raw reply [relevance 15%]
* [dpdk-dev] [PATCH v4 0/1] dpdk: introduce __rte_internal tag
2020-04-22 13:52 4% ` [dpdk-dev] [PATCH v3 0/1] " Haiyue Wang
@ 2020-04-22 16:37 4% ` Haiyue Wang
2020-04-22 16:37 15% ` [dpdk-dev] [PATCH v4 1/1] eal: add internal ABI mark support Haiyue Wang
2020-04-23 3:19 4% ` [dpdk-dev] [PATCH v5 0/1] dpdk: introduce __rte_internal tag Haiyue Wang
` (2 subsequent siblings)
4 siblings, 1 reply; 200+ results
From: Haiyue Wang @ 2020-04-22 16:37 UTC (permalink / raw)
To: dev, thomas, david.marchand, bruce.richardson, ferruh.yigit,
nhorman, mdr
Cc: Haiyue Wang
Move the internal function into INTERNAL session to avoid the ABI
checking, and it is only used for DPDK drivers or related library.
__rte_internal funA
INTERNAL {
global:
funA
};
v4: add the ABI check suppression rules
v3: based on Neil's v2 patch https://patchwork.dpdk.org/cover/54771/
Use the ALLOW_INTERNAL_API to mark this new feature.
Haiyue Wang (1):
eal: add internal ABI mark support
buildtools/check-internal-syms.sh | 57 +++++++++++++++++++++++++++++
devtools/check-symbol-change.sh | 7 ++++
devtools/libabigail.abignore | 5 +++
drivers/meson.build | 2 +-
lib/librte_eal/include/rte_compat.h | 13 +++++++
lib/meson.build | 2 +-
mk/internal/rte.compile-pre.mk | 3 ++
mk/target/generic/rte.vars.mk | 1 +
8 files changed, 88 insertions(+), 2 deletions(-)
create mode 100755 buildtools/check-internal-syms.sh
--
2.26.2
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH v3 0/4] add AESNI-MB rawdev for multi-function processing
2020-04-22 13:44 3% ` Akhil Goyal
@ 2020-04-22 14:21 0% ` Coyle, David
2020-05-01 13:18 0% ` Zhang, Roy Fan
1 sibling, 0 replies; 200+ results
From: Coyle, David @ 2020-04-22 14:21 UTC (permalink / raw)
To: Akhil Goyal, Doherty, Declan, Thomas Monjalon, Yigit, Ferruh,
Trahe, Fiona
Cc: techboard, dev, De Lara Guarch, Pablo, Ryan, Brendan,
Hemant Agrawal, Anoob Joseph, Ruifeng Wang, Liron Himi,
Nagadheeraj Rottela, Srikanth Jampala, Gagandeep Singh, Jay Zhou,
Ravi Kumar, Richardson, Bruce, olivier.matz,
honnappa.nagarahalli, Stephen Hemminger, alexr
Hi Akhil
> -----Original Message-----
> From: Akhil Goyal <akhil.goyal@nxp.com>
> Sent: Wednesday, April 22, 2020 2:44 PM
>
> Hi David,
> > Hi Akhil,
> >
> > > > >
> > > I did not look at your patches completely, but looking at the ops
> > > that you have added For rawdev are pretty much same as that of a crypto
> device.
> > >
> > > I see that there are 2 types of ops that you need
> > > - session create/destroy
> > > - enq/deq
> > >
> > > On the first impression of your patchset, I see that you want to enq
> > > to driver only once for both The operations - CRC and crypto.
> > >
> > > So what is the issue in using the cryptodev_enqueue for processing
> > > in the existing AESNI-MB driver.
> > > For session creation, the cryptodev layer will not give flexibility
> > > to add
> > > CRC+crypto kind of sessions.
> > > But in case of rte_security, you can define your new session xform
> > > based on your requirement.
> > >
> > > And while doing the cryptodev enq/deq, based on the session type,
> > > you can process the packet Specific to your usecase in your aesni-mb
> > > PMD
> > >
> > > Now if you want to add compression also along with crypto, then you
> > > can define another xform which Will be combination of
> > > crypto+compression and the aesni-mb PMD can have another mode
> which
> > > Can make sessions based on the new xform and the enq and deq can be
> > > done using the cryptodev enq/deq.
> > > For all your cases you will be having only one action type -
> > > lookaside protocol and can define different Protocols (that may not be
> standard).
> > >
> > > So to conclude, your AESNI-MB will have 3 types of operations
> > > - plain crypto
> > > - crc+crypto
> > > - compression+crypto
> > >
> > > I believe this is doable or did I miss something very obvious?
> >
> > [DC] Thank you for this feedback
> >
> > I have done this exact same analysis on rte_security and how we could use
> it.
> >
> > The main issue of this approach (and it may be possible to easily
> > overcome) is that ultimately crypto_op's need to be enqueued into
> > cryptodev. This means we can't easily control the CRC (or compression
> > in the future) at the operation level - application developers using
> > this API would create a
> > Crypto+CRC security xform session for a
> > particular flow but may want to turn off the CRC part for some packets
> > in that flow.
> >
> > There are a number of ways this issue could possibly be overcome:
> > 1) the auth offset/length fields in a rte_crypto_op could be
> > overloaded to control the CRC part of the combined operation
> > - this is not the cleanest approach
> > 2) we add a "security" op struct of some type to the union at end of
> > the rte_crypto_op
> > - to avoid any circular dependencies, this would need to be opaque
> > to rte_cryptodev
> > - rte_cryptodev should not be aware of rte_security
> >
> > Number 2 above is probably the cleaner and more preferable approach.
>
> Yes, it is preferred, but it should be a union to
> rte_crypto_sym_op/rte_crypto_asym_op.
> Crypto_op->type as RTE_CRYPTO_OP_TYPE_SECURITY and sess_type as
> RTE_CRYPTO_OP_SECURITY_SESSION The size of rte_crypto_op will remain
> as is and there will be no ABI breakage I guess.
[DC] Yes we would add to this union at the end of rte_crypto_op
__extension__
union {
struct rte_crypto_sym_op sym[0];
/**< Symmetric operation parameters */
struct rte_crypto_asym_op asym[0];
/**< Asymmetric operation parameters */
}; /**< operation specific parameters */
I haven't figured out the finer details yet, but it should be straightforward to add some security element here.
As these are zero length arrays, we won't be affecting the size of rte_crypto_op if we add another zero length array.
We should not include rte_security.h and add something like struct rte_security_op sec[0] here though, as that would
cause a circular dependency between rte_cryptodev and rte_security.
This should be resolvable though
>
> One more thing that can be looked into is the recently added CPU crypto
> process API If that could of any use, we may extend that if need be.
[DC] This is also being targeted at QAT and we would like to maintain the same
Interface for these use-cases for both AESNI-MB and QAT.
So I think the traditional enqueue/dequeue API is what we would initially use as it
means users of this API can easily switch between AESNI-MB and QAT. However, we
may look at the CPU crypto API for AESNI-MB in the future.
>
> >
> > The other approach is that CRC is either on/off at the session level.
> > That limitation would then need to be adhered by application
> > developers, which is something we would ideally like to avoid.
>
> You mean that CRC can be on/off per session as well as per packet?
> I think that can also be handled when you are defining your own security_op
> for per packet.
[DC] I meant that if we didn't take the approach defining a security_op, then
we would have turn on/off CRC at the session level and impose that limit on
the app developers. But yes, by defining a security_op, we can probably turn
it on/off at both session and op level.
>
> >
> > The rawdev multi-function approach did not have these issues which is
> > one of the reasons we have pursued this approach to date.
> >
> > However, we think the rte_security approach is workable.
> > It still requires some deeper analysis but with your support, we think
> > we can overcome the challenges.
> >
> Yes, please let me know where ever my help is required.
[DC] Thank you, appreciate that
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH v3 1/1] eal: add internal ABI mark support
2020-04-22 13:52 11% ` [dpdk-dev] [PATCH v3 1/1] eal: add internal ABI mark support Haiyue Wang
@ 2020-04-22 14:13 4% ` David Marchand
2020-04-22 16:44 4% ` Wang, Haiyue
0 siblings, 1 reply; 200+ results
From: David Marchand @ 2020-04-22 14:13 UTC (permalink / raw)
To: Haiyue Wang
Cc: dev, Thomas Monjalon, Bruce Richardson, Yigit, Ferruh, Ray Kinsella
On Wed, Apr 22, 2020 at 3:58 PM Haiyue Wang <haiyue.wang@intel.com> wrote:
>
> Introduce __rte_internal tag to mark internal ABI function, this kind of
> function can't be called by external application.
>
> Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
> ---
> buildtools/check-internal-syms.sh | 57 +++++++++++++++++++++++++++++
> devtools/check-symbol-change.sh | 7 ++++
> drivers/meson.build | 2 +-
> lib/librte_eal/include/rte_compat.h | 13 +++++++
> lib/meson.build | 2 +-
> mk/internal/rte.compile-pre.mk | 3 ++
> mk/target/generic/rte.vars.mk | 1 +
> 7 files changed, 83 insertions(+), 2 deletions(-)
> create mode 100755 buildtools/check-internal-syms.sh
I did not enter the details, but I suppose this patch lacks an update
on the suppression rules so that INTERNAL versioned symbols are
skipped.
https://git.dpdk.org/dpdk/tree/devtools/libabigail.abignore
--
David Marchand
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH v9 2/2] eal: support for VFIO-PCI VF token
@ 2020-04-22 14:02 3% ` Wang, Haiyue
0 siblings, 0 replies; 200+ results
From: Wang, Haiyue @ 2020-04-22 14:02 UTC (permalink / raw)
To: Thomas Monjalon
Cc: dev, Burakov, Anatoly, vattunuru, jerinj, alex.williamson,
david.marchand
> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: Wednesday, April 22, 2020 16:24
> To: Wang, Haiyue <haiyue.wang@intel.com>
> Cc: dev@dpdk.org; Burakov, Anatoly <anatoly.burakov@intel.com>; vattunuru@marvell.com;
> jerinj@marvell.com; alex.williamson@redhat.com; david.marchand@redhat.com
> Subject: Re: [PATCH v9 2/2] eal: support for VFIO-PCI VF token
>
> 22/04/2020 07:08, Haiyue Wang:
> > --- a/devtools/libabigail.abignore
> > +++ b/devtools/libabigail.abignore
> > +[suppress_function]
> > + name = rte_vfio_setup_device
>
> There is no comment to justify ignoring this symbol.
>
> The current policy is to reject any change in libabigail.abignore
> which could be better handled with (unfinished) rte_internal mark.
>
>
While updating the vfio patch, also doing the ABI things. Found that:
1). [suppress_function]
name = rte_vfio_setup_device
can make CI happy, but not scale.
2). rte_internal is beautiful, but make CI cry : https://patchwork.dpdk.org/patch/69109/
1 Removed function:
[D] 'function int rte_vfio_setup_device(const char*, const char*, int*, vfio_device_info*)' {rte_vfio_setup_device@@DPDK_20.0}
^ permalink raw reply [relevance 3%]
* [dpdk-dev] [PATCH v3 1/1] eal: add internal ABI mark support
2020-04-22 13:52 4% ` [dpdk-dev] [PATCH v3 0/1] " Haiyue Wang
@ 2020-04-22 13:52 11% ` Haiyue Wang
2020-04-22 14:13 4% ` David Marchand
0 siblings, 1 reply; 200+ results
From: Haiyue Wang @ 2020-04-22 13:52 UTC (permalink / raw)
To: dev, thomas, david.marchand, bruce.richardson, ferruh.yigit; +Cc: Haiyue Wang
Introduce __rte_internal tag to mark internal ABI function, this kind of
function can't be called by external application.
Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
---
buildtools/check-internal-syms.sh | 57 +++++++++++++++++++++++++++++
devtools/check-symbol-change.sh | 7 ++++
drivers/meson.build | 2 +-
lib/librte_eal/include/rte_compat.h | 13 +++++++
lib/meson.build | 2 +-
mk/internal/rte.compile-pre.mk | 3 ++
mk/target/generic/rte.vars.mk | 1 +
7 files changed, 83 insertions(+), 2 deletions(-)
create mode 100755 buildtools/check-internal-syms.sh
diff --git a/buildtools/check-internal-syms.sh b/buildtools/check-internal-syms.sh
new file mode 100755
index 000000000..5043acf87
--- /dev/null
+++ b/buildtools/check-internal-syms.sh
@@ -0,0 +1,57 @@
+#!/bin/sh
+
+# SPDX-License-Identifier: BSD-3-Clause
+
+MAPFILE=$1
+OBJFILE=$2
+
+LIST_SYMBOL=$(dirname $(readlink -f $0))/map-list-symbol.sh
+
+# added check for "make -C test/" usage
+if [ ! -e $MAPFILE ] || [ ! -f $OBJFILE ]
+then
+ exit 0
+fi
+
+if [ -d $MAPFILE ]
+then
+ exit 0
+fi
+
+DUMPFILE=$(mktemp -t dpdk.${0##*/}.XXX.objdump)
+trap 'rm -f "$DUMPFILE"' EXIT
+objdump -t $OBJFILE >$DUMPFILE
+
+ret=0
+for SYM in `$LIST_SYMBOL -S INTERNAL $MAPFILE |cut -d ' ' -f 3`
+do
+ if grep -q "\.text.*[[:space:]]$SYM$" $DUMPFILE &&
+ ! grep -q "\.text\.internal.*[[:space:]]$SYM$" $DUMPFILE
+ then
+ cat >&2 <<- END_OF_MESSAGE
+ $SYM is not flagged as internal
+ but is listed in version map
+ Please add __rte_internal to the definition of $SYM
+ END_OF_MESSAGE
+ ret=1
+ fi
+done
+
+# Filter out symbols suffixed with a . for icc
+for SYM in `awk '{
+ if ($2 != "l" && $4 == ".text.internal" && !($NF ~ /\.$/)) {
+ print $NF
+ }
+}' $DUMPFILE`
+do
+ $LIST_SYMBOL -S INTERNAL -s $SYM -q $MAPFILE || {
+ cat >&2 <<- END_OF_MESSAGE
+ $SYM is flagged as internal
+ but is not listed in version map
+ Please add $SYM to the version map
+ END_OF_MESSAGE
+ ret=1
+ }
+done
+
+exit $ret
diff --git a/devtools/check-symbol-change.sh b/devtools/check-symbol-change.sh
index ed2178e36..978979077 100755
--- a/devtools/check-symbol-change.sh
+++ b/devtools/check-symbol-change.sh
@@ -91,6 +91,13 @@ check_for_rule_violations()
if [ "$ar" = "add" ]
then
+ if [ "$secname" = "INTERNAL" ]
+ then
+ # these are absolved from any further checking
+ echo "Skipping symbol $symname in INTERNAL"
+ continue
+ fi
+
if [ "$secname" = "unknown" ]
then
# Just inform the user of this occurrence, but
diff --git a/drivers/meson.build b/drivers/meson.build
index 4d8f842ab..cac07161f 100644
--- a/drivers/meson.build
+++ b/drivers/meson.build
@@ -20,7 +20,7 @@ dpdk_driver_classes = ['common',
disabled_drivers = run_command(list_dir_globs, get_option('disable_drivers'),
).stdout().split()
-default_cflags = machine_args + ['-DALLOW_EXPERIMENTAL_API']
+default_cflags = machine_args + ['-DALLOW_EXPERIMENTAL_API'] + ['-DALLOW_INTERNAL_API']
if cc.has_argument('-Wno-format-truncation')
default_cflags += '-Wno-format-truncation'
endif
diff --git a/lib/librte_eal/include/rte_compat.h b/lib/librte_eal/include/rte_compat.h
index 3eb33784b..4cd8f68d6 100644
--- a/lib/librte_eal/include/rte_compat.h
+++ b/lib/librte_eal/include/rte_compat.h
@@ -19,4 +19,17 @@ __attribute__((section(".text.experimental")))
#endif
+#ifndef ALLOW_INTERNAL_API
+
+#define __rte_internal \
+__attribute__((error("Symbol is not public ABI"), \
+section(".text.internal")))
+
+#else
+
+#define __rte_internal \
+__attribute__((section(".text.internal")))
+
+#endif
+
#endif /* _RTE_COMPAT_H_ */
diff --git a/lib/meson.build b/lib/meson.build
index 63c17ee75..981c2e397 100644
--- a/lib/meson.build
+++ b/lib/meson.build
@@ -36,7 +36,7 @@ if is_windows
libraries = ['kvargs','eal'] # only supported libraries for windows
endif
-default_cflags = machine_args + ['-DALLOW_EXPERIMENTAL_API']
+default_cflags = machine_args + ['-DALLOW_EXPERIMENTAL_API'] + ['-DALLOW_INTERNAL_API']
if cc.has_argument('-Wno-format-truncation')
default_cflags += '-Wno-format-truncation'
endif
diff --git a/mk/internal/rte.compile-pre.mk b/mk/internal/rte.compile-pre.mk
index 82fe098f7..0369786a5 100644
--- a/mk/internal/rte.compile-pre.mk
+++ b/mk/internal/rte.compile-pre.mk
@@ -58,6 +58,8 @@ C_TO_O_DISP = $(if $(V),"$(C_TO_O_STR)"," CC $(@)")
endif
EXPERIMENTAL_CHECK = $(RTE_SDK)/buildtools/check-experimental-syms.sh
CHECK_EXPERIMENTAL = $(EXPERIMENTAL_CHECK) $(SRCDIR)/$(EXPORT_MAP) $@
+INTERNAL_CHECK = $(RTE_SDK)/buildtools/check-internal-syms.sh
+CHECK_INTERNAL = $(INTERNAL_CHECK) $(SRCDIR)/$(EXPORT_MAP) $@
PMDINFO_GEN = $(RTE_SDK_BIN)/app/dpdk-pmdinfogen $@ $@.pmd.c
PMDINFO_CC = $(CC) $(CPPFLAGS) $(CFLAGS) $(EXTRA_CFLAGS) -c -o $@.pmd.o $@.pmd.c
@@ -76,6 +78,7 @@ C_TO_O_DO = @set -e; \
$(C_TO_O) && \
$(PMDINFO_TO_O) && \
$(CHECK_EXPERIMENTAL) && \
+ $(CHECK_INTERNAL) && \
echo $(C_TO_O_CMD) > $(call obj2cmd,$(@)) && \
sed 's,'$@':,dep_'$@' =,' $(call obj2dep,$(@)).tmp > $(call obj2dep,$(@)) && \
rm -f $(call obj2dep,$(@)).tmp
diff --git a/mk/target/generic/rte.vars.mk b/mk/target/generic/rte.vars.mk
index ec2672897..11b0418e5 100644
--- a/mk/target/generic/rte.vars.mk
+++ b/mk/target/generic/rte.vars.mk
@@ -106,6 +106,7 @@ ifeq ($(BUILDING_RTE_SDK),1)
# building sdk
CFLAGS += -include $(RTE_OUTPUT)/include/rte_config.h
CFLAGS += -DALLOW_EXPERIMENTAL_API
+CFLAGS += -DALLOW_INTERNAL_API
else
# if we are building an external application, include SDK's lib and
# includes too
--
2.26.2
^ permalink raw reply [relevance 11%]
* [dpdk-dev] [PATCH v3 0/1] dpdk: introduce __rte_internal tag
@ 2020-04-22 13:52 4% ` Haiyue Wang
2020-04-22 13:52 11% ` [dpdk-dev] [PATCH v3 1/1] eal: add internal ABI mark support Haiyue Wang
2020-04-22 16:37 4% ` [dpdk-dev] [PATCH v4 0/1] dpdk: introduce __rte_internal tag Haiyue Wang
` (3 subsequent siblings)
4 siblings, 1 reply; 200+ results
From: Haiyue Wang @ 2020-04-22 13:52 UTC (permalink / raw)
To: dev, thomas, david.marchand, bruce.richardson, ferruh.yigit; +Cc: Haiyue Wang
Move the internal function into INTERNAL session to avoid the ABI
checking, and it is only used for DPDK drivers or related library.
__rte_internal funA
INTERNAL {
global:
funA
};
v3: based on Neil's v2 patch https://patchwork.dpdk.org/cover/54771/
Use the ALLOW_INTERNAL_API to mark this new feature.
Haiyue Wang (1):
eal: add internal ABI mark support
buildtools/check-internal-syms.sh | 57 +++++++++++++++++++++++++++++
devtools/check-symbol-change.sh | 7 ++++
drivers/meson.build | 2 +-
lib/librte_eal/include/rte_compat.h | 13 +++++++
lib/meson.build | 2 +-
mk/internal/rte.compile-pre.mk | 3 ++
mk/target/generic/rte.vars.mk | 1 +
7 files changed, 83 insertions(+), 2 deletions(-)
create mode 100755 buildtools/check-internal-syms.sh
--
2.26.2
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH v3 0/4] add AESNI-MB rawdev for multi-function processing
@ 2020-04-22 13:44 3% ` Akhil Goyal
2020-04-22 14:21 0% ` Coyle, David
2020-05-01 13:18 0% ` Zhang, Roy Fan
0 siblings, 2 replies; 200+ results
From: Akhil Goyal @ 2020-04-22 13:44 UTC (permalink / raw)
To: Coyle, David, Doherty, Declan, Thomas Monjalon, Yigit, Ferruh,
Trahe, Fiona
Cc: techboard, dev, De Lara Guarch, Pablo, Ryan, Brendan,
Hemant Agrawal, Anoob Joseph, Ruifeng Wang, Liron Himi,
Nagadheeraj Rottela, Srikanth Jampala, Gagandeep Singh, Jay Zhou,
Ravi Kumar, Richardson, Bruce, olivier.matz,
honnappa.nagarahalli, Stephen Hemminger, alexr
Hi David,
> Hi Akhil,
>
> > -----Original Message-----
> > From: Akhil Goyal <akhil.goyal@nxp.com>
> > Sent: Wednesday, April 22, 2020 11:51 AM
> > Hi David,
> > > > >>
> > > > >> I don't agree rte_security addresses the problem of different
> > > > >> device types supporting the same services. The problem being
> > > > >> addressed here is a single device which supports the chaining of
> > > > >> multiple services (sym crypto & error detection)
> > > > >
> > > > > Doing IPsec processing in Rx or Tx of a NIC is not chaining?
> > > > >
> > > > I wouldn't consider an inline crypto offload or full IPsec offload a
> > > > chained operation in the vein being proposed here where completely
> > > > independent services (in the view of DPDK which are currently on
> > > > independent devices and APIs) are linked together.
> > > >
> > > > We did look at using rte_security here but it wasn't considered
> > > > suitable for a chaining of non-crypto operations such as CRC or
> > > > possibly compression in the future, as it would still run into the
> > > > issue of having to use the cryptodev enq/deq API in the lookaside offload
> > case.
> > > >
> > > >
> > I did not look at your patches completely, but looking at the ops that you
> > have added For rawdev are pretty much same as that of a crypto device.
> >
> > I see that there are 2 types of ops that you need
> > - session create/destroy
> > - enq/deq
> >
> > On the first impression of your patchset, I see that you want to enq to driver
> > only once for both The operations - CRC and crypto.
> >
> > So what is the issue in using the cryptodev_enqueue for processing in the
> > existing AESNI-MB driver.
> > For session creation, the cryptodev layer will not give flexibility to add
> > CRC+crypto kind of sessions.
> > But in case of rte_security, you can define your new session xform based on
> > your requirement.
> >
> > And while doing the cryptodev enq/deq, based on the session type, you can
> > process the packet Specific to your usecase in your aesni-mb PMD
> >
> > Now if you want to add compression also along with crypto, then you can
> > define another xform which Will be combination of crypto+compression and
> > the aesni-mb PMD can have another mode which Can make sessions based
> > on the new xform and the enq and deq can be done using the cryptodev
> > enq/deq.
> > For all your cases you will be having only one action type - lookaside protocol
> > and can define different Protocols (that may not be standard).
> >
> > So to conclude, your AESNI-MB will have 3 types of operations
> > - plain crypto
> > - crc+crypto
> > - compression+crypto
> >
> > I believe this is doable or did I miss something very obvious?
>
> [DC] Thank you for this feedback
>
> I have done this exact same analysis on rte_security and how we could use it.
>
> The main issue of this approach (and it may be possible to easily overcome) is
> that ultimately crypto_op's need
> to be enqueued into cryptodev. This means we can't easily control the CRC (or
> compression in the future) at the
> operation level - application developers using this API would create a
> Crypto+CRC security xform session for a
> particular flow but may want to turn off the CRC part for some packets in that
> flow.
>
> There are a number of ways this issue could possibly be overcome:
> 1) the auth offset/length fields in a rte_crypto_op could be overloaded to
> control the CRC part of the combined operation
> - this is not the cleanest approach
> 2) we add a "security" op struct of some type to the union at end of the
> rte_crypto_op
> - to avoid any circular dependencies, this would need to be opaque to
> rte_cryptodev
> - rte_cryptodev should not be aware of rte_security
>
> Number 2 above is probably the cleaner and more preferable approach.
Yes, it is preferred, but it should be a union to rte_crypto_sym_op/rte_crypto_asym_op.
Crypto_op->type as RTE_CRYPTO_OP_TYPE_SECURITY and sess_type as
RTE_CRYPTO_OP_SECURITY_SESSION
The size of rte_crypto_op will remain as is and there will be no ABI breakage I guess.
One more thing that can be looked into is the recently added CPU crypto process API
If that could of any use, we may extend that if need be.
>
> The other approach is that CRC is either on/off at the session level. That
> limitation would then need to be adhered
> by application developers, which is something we would ideally like to avoid.
You mean that CRC can be on/off per session as well as per packet?
I think that can also be handled when you are defining your own security_op for per packet.
>
> The rawdev multi-function approach did not have these issues which is one of
> the reasons we have pursued this
> approach to date.
>
> However, we think the rte_security approach is workable.
> It still requires some deeper analysis but with your support, we think we can
> overcome the challenges.
>
Yes, please let me know where ever my help is required.
^ permalink raw reply [relevance 3%]
* Re: [dpdk-dev] [PATCHv3] Remove validate-abi.sh from tree
2020-04-22 13:19 4% ` Ray Kinsella
@ 2020-04-22 13:30 4% ` Thomas Monjalon
0 siblings, 0 replies; 200+ results
From: Thomas Monjalon @ 2020-04-22 13:30 UTC (permalink / raw)
To: Neil Horman, Ray Kinsella; +Cc: dev, david.marchand
22/04/2020 15:19, Ray Kinsella:
> On 22/04/2020 13:18, Thomas Monjalon wrote:
> > 22/04/2020 14:07, Neil Horman:
> >> On Wed, Apr 22, 2020 at 12:43:44PM +0100, Ray Kinsella wrote:
> >>> On 21/04/2020 22:42, Thomas Monjalon wrote:
> >>>> 21/04/2020 20:56, Neil Horman:
> >>>>> On Tue, Apr 21, 2020 at 01:46:43PM +0200, Thomas Monjalon wrote:
> >>>>>> 21/04/2020 13:12, Neil Horman:
> >>>>>>> On Fri, Apr 17, 2020 at 04:42:38PM +0100, Ray Kinsella wrote:
> >>>>>>>> On 17/04/2020 13:10, Thomas Monjalon wrote:
> >>>>>>>>> 17/04/2020 13:47, Ray Kinsella:
> >>>>>>>>>> On 17/04/2020 11:20, Thomas Monjalon wrote:
> >>>>>>>>>>> 17/04/2020 12:11, Ray Kinsella:
> >>>>>>>>>>>> check-abi.sh appears to be backward step in terms of usability.
> >>>>>>>>>>>
> >>>>>>>>>>> No, check-abi.sh benefits from a nice integration in build scripts.
> >>>>>>>>>>> See below.
> >>>>>>>>>>>
> >>>>>>>>>>>> With validate-abi.sh I do can do a "validate-abi.sh HEAD~1 HEAD".
> >>>>>>>>>>>> And it will do the build, install, dump and comparison for me.
> >>>>>>>>>>>> And it picked up my 20.0.2 - > 21.0 changes no problem.
> >>>>>>>>>>>>
> >>>>>>>>>>>> With check-abi on the other hand, I need to the build and install myself.
> >>>>>>>>>>>> check-abi requires dump files, but I see no reference in the documentation to how these are created.
> >>>>>>>>>>>> It silently fails when it doesn't find any ...
> >>>>>>>>>>>>
> >>>>>>>>>>>> Do I run abi-dumper on the so's myself, or how does it work?
> >>>>>>>>>>>
> >>>>>>>>>>> check-abi.sh is integrated in test-build.sh and test-meson-builds.sh.
> >>>>>>>>>>> Probably we should document usage in these scripts.
> >>>>>>>>>>
> >>>>>>>>>> Looks like I need to set DPDK_ABI_REF_VERSION=master, not obvious.
> >>>>>>>>>> Any tips or tricks would be welcome.
> >>>>>>>>>
> >>>>>>>>> export DPDK_ABI_REF_VERSION=v20.02
> >>>>>>>>> or
> >>>>>>>>> export DPDK_ABI_REF_VERSION=v19.11
> >>>>>>>>>
> >>>>>>>>> Depends on which compatibility you want to test...
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>> Few things ...
> >>>>>>>>
> >>>>>>>> 1. test-meson-build.sh keep barfing complaining about reference paths.
> >>>>>>>> ValueError: dst_dir must be absolute, got reference/v19.11/build-gcc-static/usr/local/share/dpdk/examples/bbdev_app
> >>>>>>>>
> >>>>>>>> Under the hood, ninja install is failing complaining that it needs an absolute path.
> >>>>>>>> I fixed this in test_meson_build.sh and will send a patch in a minute.
> >>>>>>>> Though it's strange no-one else has seen it?
> >>>>>>>>
> >>>>>>>> 2. test-meson-build.sh compares the abi for the static builds, which doesn't make any sense.
> >>>>>>>>
> >>>>>>>> 3. test-meson-build.sh will only take a branch in DPDK_ABI_REF_VERSION that exists locally.
> >>>>>>>> In order to get it to compare HEAD against HEAD~1, which you would imagine is a pretty common case.
> >>>>>>>> I had a create a branch for HEAD~1, in validate-abi this a pretty simple `validate-abi HEAD~1 HEAD`
> >>>>>>>>
> >>>>>>> I think this code in test-meson-build.sh should probably be fixed:
> >>>>>>>
> >>>>>>> if [ ! -d $abirefdir/src ]; then
> >>>>>>> git clone --local --no-hardlinks \
> >>>>>>> --single-branch \
> >>>>>>> -b $DPDK_ABI_REF_VERSION \
> >>>>>>> $srcdir $abirefdir/src
> >>>>>>> fi
> >>>>>>>
> >>>>>>> Like you noted, using -b allows us to checkout a tag/branch in the cloned
> >>>>>>> repository but requires that it exist locally. We should probably prefix the
> >>>>>>> checkout with a git fetch --tags
> >>>>>>
> >>>>>> I don't understand your concern.
> >>>>>> A reference is an older version, so it should be in the git tree.
> >>>>>>
> >>>>> yes, but not unless you've done a recent pull or fetch. If you set
> >>>>> DPDK_ABI_REF_VERSION to a tag/branch that didn't exist as of the last time you
> >>>>> updated the tree, it won't be there (which it sounds like what is being
> >>>>> encountered here). You can fix that by doing a git pull or git fetch prior to
> >>>>> running this script (or internal to the script)
> >>>>
> >>>> Sorry I still don't understand the case.
> >>>> We want to compare the current version C with a reference R which is older.
> >>>> If the reference R is not in the tree, it means the version C is not in the tree.
> >>>> But C is the current version, so it is in the tree by definition.
> >>>>
> >>>
> >>> So I can just relate my experience ....
> >>>
> >>> root@silpixa00395806:/build/dpdk# DPDK_ABI_REF_VERSION=HEAD~1 ./devtools/test-meson-builds.sh
> >>> ninja -C ./build-gcc-static
> >>> ninja: Entering directory `./build-gcc-static'
> >>> [1766/2204] Compiling C object 'examples/c590b3c@@dpdk-vm_power_manager@exe/vm_power_manager_channel_monitor.c.o'.
> >>> ../examples/vm_power_manager/channel_monitor.c:22:9: note: #pragma message: Jansson dev libs unavailable, not including JSON parsing
> >>> #pragma message "Jansson dev libs unavailable, not including JSON parsing"
> >>> ^~~~~~~
> >>> [2204/2204] Linking target drivers/librte_pmd_softnic.so.20.0.2.
> >>> Cloning into 'reference/HEAD~1/src'...
> >>> warning: Could not find remote branch HEAD~1 to clone.
> >>> fatal: Remote branch HEAD~1 not found in upstream origin
> >>> fatal: The remote end hung up unexpectedly
> >>>
> >> Ah, So its not the problem i was describing, I think the problem you are seeing
> >> is that the -b option only operates on branches and tags, not arbitrary git
> >> revisions.
> >>
> >> To fix that, what we probably need to do is alter test-build.sh and
> >> test-meson-build.sh such that the git clone operation is preceded by something
> >> like this:
> >> git tag ABI_CHECK_TAG $DPDK_ABI_REF_VERSION
> >>
> >> git clone ....
> >>
> >> git tag -d ABI_CHECK_TAG
> >>
> >> Doing so will guarantee that the source tree has a tag reference that the git
> >> clone operation can use to do a checkout with a -b option on.
> >
> > I don't see the benefit of such test.
> > Can we just document that the reference must be an existing tag?
> >
> You want people to use this thing right?
Yes, compare their own code with a well known tag.
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCHv3] Remove validate-abi.sh from tree
2020-04-22 12:18 4% ` Thomas Monjalon
@ 2020-04-22 13:19 4% ` Ray Kinsella
2020-04-22 13:30 4% ` Thomas Monjalon
2020-04-23 11:03 7% ` Neil Horman
1 sibling, 1 reply; 200+ results
From: Ray Kinsella @ 2020-04-22 13:19 UTC (permalink / raw)
To: Thomas Monjalon, Neil Horman; +Cc: dev, david.marchand
On 22/04/2020 13:18, Thomas Monjalon wrote:
> 22/04/2020 14:07, Neil Horman:
>> On Wed, Apr 22, 2020 at 12:43:44PM +0100, Ray Kinsella wrote:
>>> On 21/04/2020 22:42, Thomas Monjalon wrote:
>>>> 21/04/2020 20:56, Neil Horman:
>>>>> On Tue, Apr 21, 2020 at 01:46:43PM +0200, Thomas Monjalon wrote:
>>>>>> 21/04/2020 13:12, Neil Horman:
>>>>>>> On Fri, Apr 17, 2020 at 04:42:38PM +0100, Ray Kinsella wrote:
>>>>>>>> On 17/04/2020 13:10, Thomas Monjalon wrote:
>>>>>>>>> 17/04/2020 13:47, Ray Kinsella:
>>>>>>>>>> On 17/04/2020 11:20, Thomas Monjalon wrote:
>>>>>>>>>>> 17/04/2020 12:11, Ray Kinsella:
>>>>>>>>>>>> check-abi.sh appears to be backward step in terms of usability.
>>>>>>>>>>>
>>>>>>>>>>> No, check-abi.sh benefits from a nice integration in build scripts.
>>>>>>>>>>> See below.
>>>>>>>>>>>
>>>>>>>>>>>> With validate-abi.sh I do can do a "validate-abi.sh HEAD~1 HEAD".
>>>>>>>>>>>> And it will do the build, install, dump and comparison for me.
>>>>>>>>>>>> And it picked up my 20.0.2 - > 21.0 changes no problem.
>>>>>>>>>>>>
>>>>>>>>>>>> With check-abi on the other hand, I need to the build and install myself.
>>>>>>>>>>>> check-abi requires dump files, but I see no reference in the documentation to how these are created.
>>>>>>>>>>>> It silently fails when it doesn't find any ...
>>>>>>>>>>>>
>>>>>>>>>>>> Do I run abi-dumper on the so's myself, or how does it work?
>>>>>>>>>>>
>>>>>>>>>>> check-abi.sh is integrated in test-build.sh and test-meson-builds.sh.
>>>>>>>>>>> Probably we should document usage in these scripts.
>>>>>>>>>>
>>>>>>>>>> Looks like I need to set DPDK_ABI_REF_VERSION=master, not obvious.
>>>>>>>>>> Any tips or tricks would be welcome.
>>>>>>>>>
>>>>>>>>> export DPDK_ABI_REF_VERSION=v20.02
>>>>>>>>> or
>>>>>>>>> export DPDK_ABI_REF_VERSION=v19.11
>>>>>>>>>
>>>>>>>>> Depends on which compatibility you want to test...
>>>>>>>>>
>>>>>>>>
>>>>>>>> Few things ...
>>>>>>>>
>>>>>>>> 1. test-meson-build.sh keep barfing complaining about reference paths.
>>>>>>>> ValueError: dst_dir must be absolute, got reference/v19.11/build-gcc-static/usr/local/share/dpdk/examples/bbdev_app
>>>>>>>>
>>>>>>>> Under the hood, ninja install is failing complaining that it needs an absolute path.
>>>>>>>> I fixed this in test_meson_build.sh and will send a patch in a minute.
>>>>>>>> Though it's strange no-one else has seen it?
>>>>>>>>
>>>>>>>> 2. test-meson-build.sh compares the abi for the static builds, which doesn't make any sense.
>>>>>>>>
>>>>>>>> 3. test-meson-build.sh will only take a branch in DPDK_ABI_REF_VERSION that exists locally.
>>>>>>>> In order to get it to compare HEAD against HEAD~1, which you would imagine is a pretty common case.
>>>>>>>> I had a create a branch for HEAD~1, in validate-abi this a pretty simple `validate-abi HEAD~1 HEAD`
>>>>>>>>
>>>>>>> I think this code in test-meson-build.sh should probably be fixed:
>>>>>>>
>>>>>>> if [ ! -d $abirefdir/src ]; then
>>>>>>> git clone --local --no-hardlinks \
>>>>>>> --single-branch \
>>>>>>> -b $DPDK_ABI_REF_VERSION \
>>>>>>> $srcdir $abirefdir/src
>>>>>>> fi
>>>>>>>
>>>>>>> Like you noted, using -b allows us to checkout a tag/branch in the cloned
>>>>>>> repository but requires that it exist locally. We should probably prefix the
>>>>>>> checkout with a git fetch --tags
>>>>>>
>>>>>> I don't understand your concern.
>>>>>> A reference is an older version, so it should be in the git tree.
>>>>>>
>>>>> yes, but not unless you've done a recent pull or fetch. If you set
>>>>> DPDK_ABI_REF_VERSION to a tag/branch that didn't exist as of the last time you
>>>>> updated the tree, it won't be there (which it sounds like what is being
>>>>> encountered here). You can fix that by doing a git pull or git fetch prior to
>>>>> running this script (or internal to the script)
>>>>
>>>> Sorry I still don't understand the case.
>>>> We want to compare the current version C with a reference R which is older.
>>>> If the reference R is not in the tree, it means the version C is not in the tree.
>>>> But C is the current version, so it is in the tree by definition.
>>>>
>>>
>>> So I can just relate my experience ....
>>>
>>> root@silpixa00395806:/build/dpdk# DPDK_ABI_REF_VERSION=HEAD~1 ./devtools/test-meson-builds.sh
>>> ninja -C ./build-gcc-static
>>> ninja: Entering directory `./build-gcc-static'
>>> [1766/2204] Compiling C object 'examples/c590b3c@@dpdk-vm_power_manager@exe/vm_power_manager_channel_monitor.c.o'.
>>> ../examples/vm_power_manager/channel_monitor.c:22:9: note: #pragma message: Jansson dev libs unavailable, not including JSON parsing
>>> #pragma message "Jansson dev libs unavailable, not including JSON parsing"
>>> ^~~~~~~
>>> [2204/2204] Linking target drivers/librte_pmd_softnic.so.20.0.2.
>>> Cloning into 'reference/HEAD~1/src'...
>>> warning: Could not find remote branch HEAD~1 to clone.
>>> fatal: Remote branch HEAD~1 not found in upstream origin
>>> fatal: The remote end hung up unexpectedly
>>>
>> Ah, So its not the problem i was describing, I think the problem you are seeing
>> is that the -b option only operates on branches and tags, not arbitrary git
>> revisions.
>>
>> To fix that, what we probably need to do is alter test-build.sh and
>> test-meson-build.sh such that the git clone operation is preceded by something
>> like this:
>> git tag ABI_CHECK_TAG $DPDK_ABI_REF_VERSION
>>
>> git clone ....
>>
>> git tag -d ABI_CHECK_TAG
>>
>> Doing so will guarantee that the source tree has a tag reference that the git
>> clone operation can use to do a checkout with a -b option on.
>
> I don't see the benefit of such test.
> Can we just document that the reference must be an existing tag?
>
You want people to use this thing right?
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCHv3] Remove validate-abi.sh from tree
2020-04-22 12:07 4% ` Neil Horman
@ 2020-04-22 12:18 4% ` Thomas Monjalon
2020-04-22 13:19 4% ` Ray Kinsella
2020-04-23 11:03 7% ` Neil Horman
0 siblings, 2 replies; 200+ results
From: Thomas Monjalon @ 2020-04-22 12:18 UTC (permalink / raw)
To: Ray Kinsella, Neil Horman; +Cc: dev, david.marchand
22/04/2020 14:07, Neil Horman:
> On Wed, Apr 22, 2020 at 12:43:44PM +0100, Ray Kinsella wrote:
> > On 21/04/2020 22:42, Thomas Monjalon wrote:
> > > 21/04/2020 20:56, Neil Horman:
> > >> On Tue, Apr 21, 2020 at 01:46:43PM +0200, Thomas Monjalon wrote:
> > >>> 21/04/2020 13:12, Neil Horman:
> > >>>> On Fri, Apr 17, 2020 at 04:42:38PM +0100, Ray Kinsella wrote:
> > >>>>> On 17/04/2020 13:10, Thomas Monjalon wrote:
> > >>>>>> 17/04/2020 13:47, Ray Kinsella:
> > >>>>>>> On 17/04/2020 11:20, Thomas Monjalon wrote:
> > >>>>>>>> 17/04/2020 12:11, Ray Kinsella:
> > >>>>>>>>> check-abi.sh appears to be backward step in terms of usability.
> > >>>>>>>>
> > >>>>>>>> No, check-abi.sh benefits from a nice integration in build scripts.
> > >>>>>>>> See below.
> > >>>>>>>>
> > >>>>>>>>> With validate-abi.sh I do can do a "validate-abi.sh HEAD~1 HEAD".
> > >>>>>>>>> And it will do the build, install, dump and comparison for me.
> > >>>>>>>>> And it picked up my 20.0.2 - > 21.0 changes no problem.
> > >>>>>>>>>
> > >>>>>>>>> With check-abi on the other hand, I need to the build and install myself.
> > >>>>>>>>> check-abi requires dump files, but I see no reference in the documentation to how these are created.
> > >>>>>>>>> It silently fails when it doesn't find any ...
> > >>>>>>>>>
> > >>>>>>>>> Do I run abi-dumper on the so's myself, or how does it work?
> > >>>>>>>>
> > >>>>>>>> check-abi.sh is integrated in test-build.sh and test-meson-builds.sh.
> > >>>>>>>> Probably we should document usage in these scripts.
> > >>>>>>>
> > >>>>>>> Looks like I need to set DPDK_ABI_REF_VERSION=master, not obvious.
> > >>>>>>> Any tips or tricks would be welcome.
> > >>>>>>
> > >>>>>> export DPDK_ABI_REF_VERSION=v20.02
> > >>>>>> or
> > >>>>>> export DPDK_ABI_REF_VERSION=v19.11
> > >>>>>>
> > >>>>>> Depends on which compatibility you want to test...
> > >>>>>>
> > >>>>>
> > >>>>> Few things ...
> > >>>>>
> > >>>>> 1. test-meson-build.sh keep barfing complaining about reference paths.
> > >>>>> ValueError: dst_dir must be absolute, got reference/v19.11/build-gcc-static/usr/local/share/dpdk/examples/bbdev_app
> > >>>>>
> > >>>>> Under the hood, ninja install is failing complaining that it needs an absolute path.
> > >>>>> I fixed this in test_meson_build.sh and will send a patch in a minute.
> > >>>>> Though it's strange no-one else has seen it?
> > >>>>>
> > >>>>> 2. test-meson-build.sh compares the abi for the static builds, which doesn't make any sense.
> > >>>>>
> > >>>>> 3. test-meson-build.sh will only take a branch in DPDK_ABI_REF_VERSION that exists locally.
> > >>>>> In order to get it to compare HEAD against HEAD~1, which you would imagine is a pretty common case.
> > >>>>> I had a create a branch for HEAD~1, in validate-abi this a pretty simple `validate-abi HEAD~1 HEAD`
> > >>>>>
> > >>>> I think this code in test-meson-build.sh should probably be fixed:
> > >>>>
> > >>>> if [ ! -d $abirefdir/src ]; then
> > >>>> git clone --local --no-hardlinks \
> > >>>> --single-branch \
> > >>>> -b $DPDK_ABI_REF_VERSION \
> > >>>> $srcdir $abirefdir/src
> > >>>> fi
> > >>>>
> > >>>> Like you noted, using -b allows us to checkout a tag/branch in the cloned
> > >>>> repository but requires that it exist locally. We should probably prefix the
> > >>>> checkout with a git fetch --tags
> > >>>
> > >>> I don't understand your concern.
> > >>> A reference is an older version, so it should be in the git tree.
> > >>>
> > >> yes, but not unless you've done a recent pull or fetch. If you set
> > >> DPDK_ABI_REF_VERSION to a tag/branch that didn't exist as of the last time you
> > >> updated the tree, it won't be there (which it sounds like what is being
> > >> encountered here). You can fix that by doing a git pull or git fetch prior to
> > >> running this script (or internal to the script)
> > >
> > > Sorry I still don't understand the case.
> > > We want to compare the current version C with a reference R which is older.
> > > If the reference R is not in the tree, it means the version C is not in the tree.
> > > But C is the current version, so it is in the tree by definition.
> > >
> >
> > So I can just relate my experience ....
> >
> > root@silpixa00395806:/build/dpdk# DPDK_ABI_REF_VERSION=HEAD~1 ./devtools/test-meson-builds.sh
> > ninja -C ./build-gcc-static
> > ninja: Entering directory `./build-gcc-static'
> > [1766/2204] Compiling C object 'examples/c590b3c@@dpdk-vm_power_manager@exe/vm_power_manager_channel_monitor.c.o'.
> > ../examples/vm_power_manager/channel_monitor.c:22:9: note: #pragma message: Jansson dev libs unavailable, not including JSON parsing
> > #pragma message "Jansson dev libs unavailable, not including JSON parsing"
> > ^~~~~~~
> > [2204/2204] Linking target drivers/librte_pmd_softnic.so.20.0.2.
> > Cloning into 'reference/HEAD~1/src'...
> > warning: Could not find remote branch HEAD~1 to clone.
> > fatal: Remote branch HEAD~1 not found in upstream origin
> > fatal: The remote end hung up unexpectedly
> >
> Ah, So its not the problem i was describing, I think the problem you are seeing
> is that the -b option only operates on branches and tags, not arbitrary git
> revisions.
>
> To fix that, what we probably need to do is alter test-build.sh and
> test-meson-build.sh such that the git clone operation is preceded by something
> like this:
> git tag ABI_CHECK_TAG $DPDK_ABI_REF_VERSION
>
> git clone ....
>
> git tag -d ABI_CHECK_TAG
>
> Doing so will guarantee that the source tree has a tag reference that the git
> clone operation can use to do a checkout with a -b option on.
I don't see the benefit of such test.
Can we just document that the reference must be an existing tag?
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCHv3] Remove validate-abi.sh from tree
2020-04-22 12:01 8% ` Neil Horman
@ 2020-04-22 12:16 4% ` Thomas Monjalon
2020-04-23 10:57 4% ` Neil Horman
0 siblings, 1 reply; 200+ results
From: Thomas Monjalon @ 2020-04-22 12:16 UTC (permalink / raw)
To: Neil Horman; +Cc: Ray Kinsella, dev, david.marchand
22/04/2020 14:01, Neil Horman:
> On Tue, Apr 21, 2020 at 11:42:42PM +0200, Thomas Monjalon wrote:
> > 21/04/2020 20:56, Neil Horman:
> > > On Tue, Apr 21, 2020 at 01:46:43PM +0200, Thomas Monjalon wrote:
> > > > 21/04/2020 13:12, Neil Horman:
> > > > > On Fri, Apr 17, 2020 at 04:42:38PM +0100, Ray Kinsella wrote:
> > > > > > On 17/04/2020 13:10, Thomas Monjalon wrote:
> > > > > > > 17/04/2020 13:47, Ray Kinsella:
> > > > > > >> On 17/04/2020 11:20, Thomas Monjalon wrote:
> > > > > > >>> 17/04/2020 12:11, Ray Kinsella:
> > > > > > >>>> check-abi.sh appears to be backward step in terms of usability.
> > > > > > >>>
> > > > > > >>> No, check-abi.sh benefits from a nice integration in build scripts.
> > > > > > >>> See below.
> > > > > > >>>
> > > > > > >>>> With validate-abi.sh I do can do a "validate-abi.sh HEAD~1 HEAD".
> > > > > > >>>> And it will do the build, install, dump and comparison for me.
> > > > > > >>>> And it picked up my 20.0.2 - > 21.0 changes no problem.
> > > > > > >>>>
> > > > > > >>>> With check-abi on the other hand, I need to the build and install myself.
> > > > > > >>>> check-abi requires dump files, but I see no reference in the documentation to how these are created.
> > > > > > >>>> It silently fails when it doesn't find any ...
> > > > > > >>>>
> > > > > > >>>> Do I run abi-dumper on the so's myself, or how does it work?
> > > > > > >>>
> > > > > > >>> check-abi.sh is integrated in test-build.sh and test-meson-builds.sh.
> > > > > > >>> Probably we should document usage in these scripts.
> > > > > > >>
> > > > > > >> Looks like I need to set DPDK_ABI_REF_VERSION=master, not obvious.
> > > > > > >> Any tips or tricks would be welcome.
> > > > > > >
> > > > > > > export DPDK_ABI_REF_VERSION=v20.02
> > > > > > > or
> > > > > > > export DPDK_ABI_REF_VERSION=v19.11
> > > > > > >
> > > > > > > Depends on which compatibility you want to test...
> > > > > > >
> > > > > >
> > > > > > Few things ...
> > > > > >
> > > > > > 1. test-meson-build.sh keep barfing complaining about reference paths.
> > > > > > ValueError: dst_dir must be absolute, got reference/v19.11/build-gcc-static/usr/local/share/dpdk/examples/bbdev_app
> > > > > >
> > > > > > Under the hood, ninja install is failing complaining that it needs an absolute path.
> > > > > > I fixed this in test_meson_build.sh and will send a patch in a minute.
> > > > > > Though it's strange no-one else has seen it?
> > > > > >
> > > > > > 2. test-meson-build.sh compares the abi for the static builds, which doesn't make any sense.
> > > > > >
> > > > > > 3. test-meson-build.sh will only take a branch in DPDK_ABI_REF_VERSION that exists locally.
> > > > > > In order to get it to compare HEAD against HEAD~1, which you would imagine is a pretty common case.
> > > > > > I had a create a branch for HEAD~1, in validate-abi this a pretty simple `validate-abi HEAD~1 HEAD`
> > > > > >
> > > > > I think this code in test-meson-build.sh should probably be fixed:
> > > > >
> > > > > if [ ! -d $abirefdir/src ]; then
> > > > > git clone --local --no-hardlinks \
> > > > > --single-branch \
> > > > > -b $DPDK_ABI_REF_VERSION \
> > > > > $srcdir $abirefdir/src
> > > > > fi
> > > > >
> > > > > Like you noted, using -b allows us to checkout a tag/branch in the cloned
> > > > > repository but requires that it exist locally. We should probably prefix the
> > > > > checkout with a git fetch --tags
> > > >
> > > > I don't understand your concern.
> > > > A reference is an older version, so it should be in the git tree.
> > > >
> > > yes, but not unless you've done a recent pull or fetch. If you set
> > > DPDK_ABI_REF_VERSION to a tag/branch that didn't exist as of the last time you
> > > updated the tree, it won't be there (which it sounds like what is being
> > > encountered here). You can fix that by doing a git pull or git fetch prior to
> > > running this script (or internal to the script)
> >
> > Sorry I still don't understand the case.
> > We want to compare the current version C with a reference R which is older.
> > If the reference R is not in the tree, it means the version C is not in the tree.
> > But C is the current version, so it is in the tree by definition.
> >
>
>
> +---+
> | |
> +---+ | | DPDK 20.11
> | |Feature Branch commit +-+-+
> | | |
> +-+-+ +---+ +-v-+
> | | | master HEAD | | master HEAD
> | +-+-+ +-+-+
> | | |
> | | |
> | +-v-+ +-v-+
> +---->+ | DPDK 19.11 | | DPDK 19.11
> +---+ +---+
> X X X X
> XXXXXXX XXXXXXXXX XXXXXXX XXXXXXXXX
> XXX XX XXX XX
> XXXX XXXX
>
> Local GIT copy DPDK.ORG
>
>
>
> If a user clones from dpdk.org when dpdk 19.11 is tagged in the tree, or any
> time before dpdk 20.11 is tagged, then creates a feature branch, they may want
> to compare their abi to the latest stable version. If they check dpdk.org and
> see that dpdk 20.11 is the latest tag in the tree, they can run check-abi.sh
> with the reference tag specified as 20.11, which exists in the dpdk.org tree,
> but may not have been pulled into their local copy yet.
>
> I'm not saying this is definately what happened, but it would explain the
> reported observations.
OK now I understand.
The user specified a reference which is not in his local tree.
I vote for considering this case as an user mistake,
and document it as a limitation of the tool integration in our build
testing scripts.
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCHv3] Remove validate-abi.sh from tree
2020-04-22 11:43 4% ` Ray Kinsella
@ 2020-04-22 12:07 4% ` Neil Horman
2020-04-22 12:18 4% ` Thomas Monjalon
0 siblings, 1 reply; 200+ results
From: Neil Horman @ 2020-04-22 12:07 UTC (permalink / raw)
To: Ray Kinsella; +Cc: Thomas Monjalon, dev, david.marchand
On Wed, Apr 22, 2020 at 12:43:44PM +0100, Ray Kinsella wrote:
>
>
> On 21/04/2020 22:42, Thomas Monjalon wrote:
> > 21/04/2020 20:56, Neil Horman:
> >> On Tue, Apr 21, 2020 at 01:46:43PM +0200, Thomas Monjalon wrote:
> >>> 21/04/2020 13:12, Neil Horman:
> >>>> On Fri, Apr 17, 2020 at 04:42:38PM +0100, Ray Kinsella wrote:
> >>>>> On 17/04/2020 13:10, Thomas Monjalon wrote:
> >>>>>> 17/04/2020 13:47, Ray Kinsella:
> >>>>>>> On 17/04/2020 11:20, Thomas Monjalon wrote:
> >>>>>>>> 17/04/2020 12:11, Ray Kinsella:
> >>>>>>>>> check-abi.sh appears to be backward step in terms of usability.
> >>>>>>>>
> >>>>>>>> No, check-abi.sh benefits from a nice integration in build scripts.
> >>>>>>>> See below.
> >>>>>>>>
> >>>>>>>>> With validate-abi.sh I do can do a "validate-abi.sh HEAD~1 HEAD".
> >>>>>>>>> And it will do the build, install, dump and comparison for me.
> >>>>>>>>> And it picked up my 20.0.2 - > 21.0 changes no problem.
> >>>>>>>>>
> >>>>>>>>> With check-abi on the other hand, I need to the build and install myself.
> >>>>>>>>> check-abi requires dump files, but I see no reference in the documentation to how these are created.
> >>>>>>>>> It silently fails when it doesn't find any ...
> >>>>>>>>>
> >>>>>>>>> Do I run abi-dumper on the so's myself, or how does it work?
> >>>>>>>>
> >>>>>>>> check-abi.sh is integrated in test-build.sh and test-meson-builds.sh.
> >>>>>>>> Probably we should document usage in these scripts.
> >>>>>>>
> >>>>>>> Looks like I need to set DPDK_ABI_REF_VERSION=master, not obvious.
> >>>>>>> Any tips or tricks would be welcome.
> >>>>>>
> >>>>>> export DPDK_ABI_REF_VERSION=v20.02
> >>>>>> or
> >>>>>> export DPDK_ABI_REF_VERSION=v19.11
> >>>>>>
> >>>>>> Depends on which compatibility you want to test...
> >>>>>>
> >>>>>
> >>>>> Few things ...
> >>>>>
> >>>>> 1. test-meson-build.sh keep barfing complaining about reference paths.
> >>>>> ValueError: dst_dir must be absolute, got reference/v19.11/build-gcc-static/usr/local/share/dpdk/examples/bbdev_app
> >>>>>
> >>>>> Under the hood, ninja install is failing complaining that it needs an absolute path.
> >>>>> I fixed this in test_meson_build.sh and will send a patch in a minute.
> >>>>> Though it's strange no-one else has seen it?
> >>>>>
> >>>>> 2. test-meson-build.sh compares the abi for the static builds, which doesn't make any sense.
> >>>>>
> >>>>> 3. test-meson-build.sh will only take a branch in DPDK_ABI_REF_VERSION that exists locally.
> >>>>> In order to get it to compare HEAD against HEAD~1, which you would imagine is a pretty common case.
> >>>>> I had a create a branch for HEAD~1, in validate-abi this a pretty simple `validate-abi HEAD~1 HEAD`
> >>>>>
> >>>> I think this code in test-meson-build.sh should probably be fixed:
> >>>>
> >>>> if [ ! -d $abirefdir/src ]; then
> >>>> git clone --local --no-hardlinks \
> >>>> --single-branch \
> >>>> -b $DPDK_ABI_REF_VERSION \
> >>>> $srcdir $abirefdir/src
> >>>> fi
> >>>>
> >>>> Like you noted, using -b allows us to checkout a tag/branch in the cloned
> >>>> repository but requires that it exist locally. We should probably prefix the
> >>>> checkout with a git fetch --tags
> >>>
> >>> I don't understand your concern.
> >>> A reference is an older version, so it should be in the git tree.
> >>>
> >> yes, but not unless you've done a recent pull or fetch. If you set
> >> DPDK_ABI_REF_VERSION to a tag/branch that didn't exist as of the last time you
> >> updated the tree, it won't be there (which it sounds like what is being
> >> encountered here). You can fix that by doing a git pull or git fetch prior to
> >> running this script (or internal to the script)
> >
> > Sorry I still don't understand the case.
> > We want to compare the current version C with a reference R which is older.
> > If the reference R is not in the tree, it means the version C is not in the tree.
> > But C is the current version, so it is in the tree by definition.
> >
>
> So I can just relate my experience ....
>
> root@silpixa00395806:/build/dpdk# DPDK_ABI_REF_VERSION=HEAD~1 ./devtools/test-meson-builds.sh
> ninja -C ./build-gcc-static
> ninja: Entering directory `./build-gcc-static'
> [1766/2204] Compiling C object 'examples/c590b3c@@dpdk-vm_power_manager@exe/vm_power_manager_channel_monitor.c.o'.
> ../examples/vm_power_manager/channel_monitor.c:22:9: note: #pragma message: Jansson dev libs unavailable, not including JSON parsing
> #pragma message "Jansson dev libs unavailable, not including JSON parsing"
> ^~~~~~~
> [2204/2204] Linking target drivers/librte_pmd_softnic.so.20.0.2.
> Cloning into 'reference/HEAD~1/src'...
> warning: Could not find remote branch HEAD~1 to clone.
> fatal: Remote branch HEAD~1 not found in upstream origin
> fatal: The remote end hung up unexpectedly
>
Ah, So its not the problem i was describing, I think the problem you are seeing
is that the -b option only operates on branches and tags, not arbitrary git
revisions.
To fix that, what we probably need to do is alter test-build.sh and
test-meson-build.sh such that the git clone operation is preceded by something
like this:
git tag ABI_CHECK_TAG $DPDK_ABI_REF_VERSION
git clone ....
git tag -d ABI_CHECK_TAG
Doing so will guarantee that the source tree has a tag reference that the git
clone operation can use to do a checkout with a -b option on.
Neil
>
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCHv3] Remove validate-abi.sh from tree
2020-04-21 21:42 4% ` Thomas Monjalon
2020-04-22 11:43 4% ` Ray Kinsella
@ 2020-04-22 12:01 8% ` Neil Horman
2020-04-22 12:16 4% ` Thomas Monjalon
1 sibling, 1 reply; 200+ results
From: Neil Horman @ 2020-04-22 12:01 UTC (permalink / raw)
To: Thomas Monjalon; +Cc: Ray Kinsella, dev, david.marchand
On Tue, Apr 21, 2020 at 11:42:42PM +0200, Thomas Monjalon wrote:
> 21/04/2020 20:56, Neil Horman:
> > On Tue, Apr 21, 2020 at 01:46:43PM +0200, Thomas Monjalon wrote:
> > > 21/04/2020 13:12, Neil Horman:
> > > > On Fri, Apr 17, 2020 at 04:42:38PM +0100, Ray Kinsella wrote:
> > > > > On 17/04/2020 13:10, Thomas Monjalon wrote:
> > > > > > 17/04/2020 13:47, Ray Kinsella:
> > > > > >> On 17/04/2020 11:20, Thomas Monjalon wrote:
> > > > > >>> 17/04/2020 12:11, Ray Kinsella:
> > > > > >>>> check-abi.sh appears to be backward step in terms of usability.
> > > > > >>>
> > > > > >>> No, check-abi.sh benefits from a nice integration in build scripts.
> > > > > >>> See below.
> > > > > >>>
> > > > > >>>> With validate-abi.sh I do can do a "validate-abi.sh HEAD~1 HEAD".
> > > > > >>>> And it will do the build, install, dump and comparison for me.
> > > > > >>>> And it picked up my 20.0.2 - > 21.0 changes no problem.
> > > > > >>>>
> > > > > >>>> With check-abi on the other hand, I need to the build and install myself.
> > > > > >>>> check-abi requires dump files, but I see no reference in the documentation to how these are created.
> > > > > >>>> It silently fails when it doesn't find any ...
> > > > > >>>>
> > > > > >>>> Do I run abi-dumper on the so's myself, or how does it work?
> > > > > >>>
> > > > > >>> check-abi.sh is integrated in test-build.sh and test-meson-builds.sh.
> > > > > >>> Probably we should document usage in these scripts.
> > > > > >>
> > > > > >> Looks like I need to set DPDK_ABI_REF_VERSION=master, not obvious.
> > > > > >> Any tips or tricks would be welcome.
> > > > > >
> > > > > > export DPDK_ABI_REF_VERSION=v20.02
> > > > > > or
> > > > > > export DPDK_ABI_REF_VERSION=v19.11
> > > > > >
> > > > > > Depends on which compatibility you want to test...
> > > > > >
> > > > >
> > > > > Few things ...
> > > > >
> > > > > 1. test-meson-build.sh keep barfing complaining about reference paths.
> > > > > ValueError: dst_dir must be absolute, got reference/v19.11/build-gcc-static/usr/local/share/dpdk/examples/bbdev_app
> > > > >
> > > > > Under the hood, ninja install is failing complaining that it needs an absolute path.
> > > > > I fixed this in test_meson_build.sh and will send a patch in a minute.
> > > > > Though it's strange no-one else has seen it?
> > > > >
> > > > > 2. test-meson-build.sh compares the abi for the static builds, which doesn't make any sense.
> > > > >
> > > > > 3. test-meson-build.sh will only take a branch in DPDK_ABI_REF_VERSION that exists locally.
> > > > > In order to get it to compare HEAD against HEAD~1, which you would imagine is a pretty common case.
> > > > > I had a create a branch for HEAD~1, in validate-abi this a pretty simple `validate-abi HEAD~1 HEAD`
> > > > >
> > > > I think this code in test-meson-build.sh should probably be fixed:
> > > >
> > > > if [ ! -d $abirefdir/src ]; then
> > > > git clone --local --no-hardlinks \
> > > > --single-branch \
> > > > -b $DPDK_ABI_REF_VERSION \
> > > > $srcdir $abirefdir/src
> > > > fi
> > > >
> > > > Like you noted, using -b allows us to checkout a tag/branch in the cloned
> > > > repository but requires that it exist locally. We should probably prefix the
> > > > checkout with a git fetch --tags
> > >
> > > I don't understand your concern.
> > > A reference is an older version, so it should be in the git tree.
> > >
> > yes, but not unless you've done a recent pull or fetch. If you set
> > DPDK_ABI_REF_VERSION to a tag/branch that didn't exist as of the last time you
> > updated the tree, it won't be there (which it sounds like what is being
> > encountered here). You can fix that by doing a git pull or git fetch prior to
> > running this script (or internal to the script)
>
> Sorry I still don't understand the case.
> We want to compare the current version C with a reference R which is older.
> If the reference R is not in the tree, it means the version C is not in the tree.
> But C is the current version, so it is in the tree by definition.
>
+---+
| |
+---+ | | DPDK 20.11
| |Feature Branch commit +-+-+
| | |
+-+-+ +---+ +-v-+
| | | master HEAD | | master HEAD
| +-+-+ +-+-+
| | |
| | |
| +-v-+ +-v-+
+---->+ | DPDK 19.11 | | DPDK 19.11
+---+ +---+
X X X X
XXXXXXX XXXXXXXXX XXXXXXX XXXXXXXXX
XXX XX XXX XX
XXXX XXXX
Local GIT copy DPDK.ORG
If a user clones from dpdk.org when dpdk 19.11 is tagged in the tree, or any
time before dpdk 20.11 is tagged, then creates a feature branch, they may want
to compare their abi to the latest stable version. If they check dpdk.org and
see that dpdk 20.11 is the latest tag in the tree, they can run check-abi.sh
with the reference tag specified as 20.11, which exists in the dpdk.org tree,
but may not have been pulled into their local copy yet.
I'm not saying this is definately what happened, but it would explain the
reported observations.
Neil
^ permalink raw reply [relevance 8%]
* Re: [dpdk-dev] [PATCHv3] Remove validate-abi.sh from tree
2020-04-21 21:42 4% ` Thomas Monjalon
@ 2020-04-22 11:43 4% ` Ray Kinsella
2020-04-22 12:07 4% ` Neil Horman
2020-04-22 12:01 8% ` Neil Horman
1 sibling, 1 reply; 200+ results
From: Ray Kinsella @ 2020-04-22 11:43 UTC (permalink / raw)
To: Thomas Monjalon, Neil Horman; +Cc: dev, david.marchand
On 21/04/2020 22:42, Thomas Monjalon wrote:
> 21/04/2020 20:56, Neil Horman:
>> On Tue, Apr 21, 2020 at 01:46:43PM +0200, Thomas Monjalon wrote:
>>> 21/04/2020 13:12, Neil Horman:
>>>> On Fri, Apr 17, 2020 at 04:42:38PM +0100, Ray Kinsella wrote:
>>>>> On 17/04/2020 13:10, Thomas Monjalon wrote:
>>>>>> 17/04/2020 13:47, Ray Kinsella:
>>>>>>> On 17/04/2020 11:20, Thomas Monjalon wrote:
>>>>>>>> 17/04/2020 12:11, Ray Kinsella:
>>>>>>>>> check-abi.sh appears to be backward step in terms of usability.
>>>>>>>>
>>>>>>>> No, check-abi.sh benefits from a nice integration in build scripts.
>>>>>>>> See below.
>>>>>>>>
>>>>>>>>> With validate-abi.sh I do can do a "validate-abi.sh HEAD~1 HEAD".
>>>>>>>>> And it will do the build, install, dump and comparison for me.
>>>>>>>>> And it picked up my 20.0.2 - > 21.0 changes no problem.
>>>>>>>>>
>>>>>>>>> With check-abi on the other hand, I need to the build and install myself.
>>>>>>>>> check-abi requires dump files, but I see no reference in the documentation to how these are created.
>>>>>>>>> It silently fails when it doesn't find any ...
>>>>>>>>>
>>>>>>>>> Do I run abi-dumper on the so's myself, or how does it work?
>>>>>>>>
>>>>>>>> check-abi.sh is integrated in test-build.sh and test-meson-builds.sh.
>>>>>>>> Probably we should document usage in these scripts.
>>>>>>>
>>>>>>> Looks like I need to set DPDK_ABI_REF_VERSION=master, not obvious.
>>>>>>> Any tips or tricks would be welcome.
>>>>>>
>>>>>> export DPDK_ABI_REF_VERSION=v20.02
>>>>>> or
>>>>>> export DPDK_ABI_REF_VERSION=v19.11
>>>>>>
>>>>>> Depends on which compatibility you want to test...
>>>>>>
>>>>>
>>>>> Few things ...
>>>>>
>>>>> 1. test-meson-build.sh keep barfing complaining about reference paths.
>>>>> ValueError: dst_dir must be absolute, got reference/v19.11/build-gcc-static/usr/local/share/dpdk/examples/bbdev_app
>>>>>
>>>>> Under the hood, ninja install is failing complaining that it needs an absolute path.
>>>>> I fixed this in test_meson_build.sh and will send a patch in a minute.
>>>>> Though it's strange no-one else has seen it?
>>>>>
>>>>> 2. test-meson-build.sh compares the abi for the static builds, which doesn't make any sense.
>>>>>
>>>>> 3. test-meson-build.sh will only take a branch in DPDK_ABI_REF_VERSION that exists locally.
>>>>> In order to get it to compare HEAD against HEAD~1, which you would imagine is a pretty common case.
>>>>> I had a create a branch for HEAD~1, in validate-abi this a pretty simple `validate-abi HEAD~1 HEAD`
>>>>>
>>>> I think this code in test-meson-build.sh should probably be fixed:
>>>>
>>>> if [ ! -d $abirefdir/src ]; then
>>>> git clone --local --no-hardlinks \
>>>> --single-branch \
>>>> -b $DPDK_ABI_REF_VERSION \
>>>> $srcdir $abirefdir/src
>>>> fi
>>>>
>>>> Like you noted, using -b allows us to checkout a tag/branch in the cloned
>>>> repository but requires that it exist locally. We should probably prefix the
>>>> checkout with a git fetch --tags
>>>
>>> I don't understand your concern.
>>> A reference is an older version, so it should be in the git tree.
>>>
>> yes, but not unless you've done a recent pull or fetch. If you set
>> DPDK_ABI_REF_VERSION to a tag/branch that didn't exist as of the last time you
>> updated the tree, it won't be there (which it sounds like what is being
>> encountered here). You can fix that by doing a git pull or git fetch prior to
>> running this script (or internal to the script)
>
> Sorry I still don't understand the case.
> We want to compare the current version C with a reference R which is older.
> If the reference R is not in the tree, it means the version C is not in the tree.
> But C is the current version, so it is in the tree by definition.
>
So I can just relate my experience ....
root@silpixa00395806:/build/dpdk# DPDK_ABI_REF_VERSION=HEAD~1 ./devtools/test-meson-builds.sh
ninja -C ./build-gcc-static
ninja: Entering directory `./build-gcc-static'
[1766/2204] Compiling C object 'examples/c590b3c@@dpdk-vm_power_manager@exe/vm_power_manager_channel_monitor.c.o'.
../examples/vm_power_manager/channel_monitor.c:22:9: note: #pragma message: Jansson dev libs unavailable, not including JSON parsing
#pragma message "Jansson dev libs unavailable, not including JSON parsing"
^~~~~~~
[2204/2204] Linking target drivers/librte_pmd_softnic.so.20.0.2.
Cloning into 'reference/HEAD~1/src'...
warning: Could not find remote branch HEAD~1 to clone.
fatal: Remote branch HEAD~1 not found in upstream origin
fatal: The remote end hung up unexpectedly
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH v3] cryptodev: version cryptodev info get function
@ 2020-04-22 8:36 0% ` Ray Kinsella
0 siblings, 0 replies; 200+ results
From: Ray Kinsella @ 2020-04-22 8:36 UTC (permalink / raw)
To: Trahe, Fiona, Akhil Goyal, Kusztal, ArkadiuszX, dev; +Cc: Thomas Monjalon
On 20/04/2020 18:26, Ray Kinsella wrote:
>
>
> On 20/04/2020 17:54, Trahe, Fiona wrote:
>> Hi Ray, Akhil,
>>
>>
>>> On 20/04/2020 15:22, Akhil Goyal wrote:
>>>>
>>>>
>>>>>
>>>>> This patch adds versioned function rte_cryptodev_info_get()
>>>>> to prevent some issues with ABI policy.
>>>>> Node v21 works in same way as before, returning driver capabilities
>>>>> directly to the API caller. These capabilities may include new elements
>>>>> not part of the v20 ABI.
>>>>> Node v20 function maintains compatibility with v20 ABI releases
>>>>> by stripping out elements not supported in v20 ABI. Because
>>>>> rte_cryptodev_info_get is called by other API functions,
>>>>> rte_cryptodev_sym_capability_get function is versioned the same way.
>>>>>
>>>>> Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
>>>>> ---
>>>>> v2:
>>>>> - changed version numbers of symbols to 20.0.2
>>>>> v3:
>>>>> - added v2/v3 informations
>>>>> - changed version numbers of symbols to 21
>>>>> - fixed checkpatch issues
>>>>>
>>>>> This patch depends on following patches:
>>>>>
>>>>> [1] - "[v3] cryptodev: add chacha20-poly1305 aead algorithm"
>>>>> (https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpatchwor
>>>>> k.dpdk.org%2Fpatch%2F64549%2F&data=02%7C01%7Cakhil.goyal%40nxp.
>>>>> com%7Ce6789fd42a5946c128e508d7e2dffe2f%7C686ea1d3bc2b4c6fa92cd99c
>>>>> 5c301635%7C0%7C0%7C637227323980059545&sdata=50eQJE7WHTME6d
>>>>> qA7Nfk%2B50PVAyJrpKlMw%2BoGtA1%2FTc%3D&reserved=0)
>>>>
>>>> Please include the dependent patches in a single series in your next version.
>>>>>
>>>>> lib/librte_cryptodev/rte_cryptodev.c | 143 ++++++++++++++++++++++++-
>>>>> lib/librte_cryptodev/rte_cryptodev.h | 39 ++++++-
>>>>> lib/librte_cryptodev/rte_cryptodev_version.map | 7 ++
>>>>> 3 files changed, 186 insertions(+), 3 deletions(-)
>>>>>
>>>>> diff --git a/lib/librte_cryptodev/rte_cryptodev.c
>>>>> b/lib/librte_cryptodev/rte_cryptodev.c
>>>>> index 6d1d0e9..b061447 100644
>>>>> --- a/lib/librte_cryptodev/rte_cryptodev.c
>>>>> +++ b/lib/librte_cryptodev/rte_cryptodev.c
>>>>> @@ -41,6 +41,9 @@
>>>>> #include "rte_cryptodev.h"
>>>>> #include "rte_cryptodev_pmd.h"
>>>>>
>>>>> +#include <rte_compat.h>
>>>>> +#include <rte_function_versioning.h>
>>>>> +
>>>>> static uint8_t nb_drivers;
>>>>>
>>>>> static struct rte_cryptodev rte_crypto_devices[RTE_CRYPTO_MAX_DEVS];
>>>>> @@ -56,6 +59,14 @@ static struct rte_cryptodev_global cryptodev_globals = {
>>>>> /* spinlock for crypto device callbacks */
>>>>> static rte_spinlock_t rte_cryptodev_cb_lock = RTE_SPINLOCK_INITIALIZER;
>>>>>
>>>>> +static const struct rte_cryptodev_capabilities
>>>>> + cryptodev_undefined_capabilities[] = {
>>>>> + RTE_CRYPTODEV_END_OF_CAPABILITIES_LIST()
>>>>> +};
>>>>> +
>>>>> +static struct rte_cryptodev_capabilities
>>>>> + *capability_copies[RTE_CRYPTO_MAX_DEVS];
>>>>
>>>> Capabilities_copy is a better name as it is copy of many capabilities.
>> [Fiona] ok
>>
>>
>>>>> const struct rte_cryptodev_symmetric_capability *
>>>>> -rte_cryptodev_sym_capability_get(uint8_t dev_id,
>>>>> +rte_cryptodev_sym_capability_get_v20(uint8_t dev_id,
>>>>
>>>> __vsym Annotation to be used in a declaration of the internal symbol
>>>> to signal that it is being used as an implementation of a particular
>>>> version of symbol.
>> [Fiona] ok
>>
>>
>>>>> + }
>>>>> +
>>>>> + return NULL;
>>>>> +
>>>>
>>>> Extra line
>> [Fiona] ok
>>
>>>>
>>>>> +}
>>>>> +VERSION_SYMBOL(rte_cryptodev_sym_capability_get, _v20, 20.0);
>>>>> +
>>>>> +const struct rte_cryptodev_symmetric_capability *
>>>>
>>>> __vsym annotation
>> [Fiona] ok
>>
>>
>>>>
>>>>> +rte_cryptodev_sym_capability_get_v21(uint8_t dev_id,
>>>>> const struct rte_cryptodev_sym_capability_idx *idx)
>>>>> {
>>>>> const struct rte_cryptodev_capabilities *capability;
>>>>> @@ -313,6 +360,10 @@ rte_cryptodev_sym_capability_get(uint8_t dev_id,
>>>>> return NULL;
>>>>>
>>>>> }
>>>>> +MAP_STATIC_SYMBOL(const struct rte_cryptodev_symmetric_capability *
>>>>> + rte_cryptodev_sym_capability_get(uint8_t dev_id,
>>>>> + const struct rte_cryptodev_sym_capability_idx *idx),
>>>>> + rte_cryptodev_sym_capability_get_v21);
>>>>>
>>>>> static int
>>>>> param_range_check(uint16_t size, const struct rte_crypto_param_range *range)
>>>>> @@ -999,6 +1050,13 @@ rte_cryptodev_close(uint8_t dev_id)
>>>>> RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_close, -ENOTSUP);
>>>>> retval = (*dev->dev_ops->dev_close)(dev);
>>>>>
>>>>> +
>>>>> + if (capability_copies[dev_id]) {
>>>>> + free(capability_copies[dev_id]);
>>>>> + capability_copies[dev_id] = NULL;
>>>>> + }
>>>>> + is_capability_checked[dev_id] = 0;
>>>>> +
>>>>> if (retval < 0)
>>>>> return retval;
>>>>>
>>>>> @@ -1111,9 +1169,61 @@ rte_cryptodev_stats_reset(uint8_t dev_id)
>>>>> (*dev->dev_ops->stats_reset)(dev);
>>>>> }
>>>>>
>>>>> +static void
>>>>> +get_v20_capabilities(uint8_t dev_id, struct rte_cryptodev_info *dev_info)
>>>>> +{
>>>>> + const struct rte_cryptodev_capabilities *capability;
>>>>> + uint8_t found_invalid_capa = 0;
>>>>> + uint8_t counter = 0;
>>>>> +
>>>>> + for (capability = dev_info->capabilities;
>>>>> + capability->op != RTE_CRYPTO_OP_TYPE_UNDEFINED;
>>>>> + ++capability, ++counter) {
>>>>> + if (capability->op == RTE_CRYPTO_OP_TYPE_SYMMETRIC &&
>>>>> + capability->sym.xform_type ==
>>>>> + RTE_CRYPTO_SYM_XFORM_AEAD
>>>>> + && capability->sym.aead.algo >=
>>>>> + RTE_CRYPTO_AEAD_CHACHA20_POLY1305) {
>>>>> + found_invalid_capa = 1;
>>>>> + counter--;
>>>>> + }
>>>>> + }
>>>>> + is_capability_checked[dev_id] = 1;
>>>>> + if (found_invalid_capa) {
>>>>
>>>> Code becomes unreadable due to indentation which can be avoided.
>> [Fiona] ok
>>
>>
>>
>>>>> +void
>>>>> +rte_cryptodev_info_get_v21(uint8_t dev_id, struct rte_cryptodev_info
>>>>> *dev_info);
>>>>> +BIND_DEFAULT_SYMBOL(rte_cryptodev_info_get, _v21, 21);
>>>>
>>>> I am not sure if we need to bind for _v20 also
>>>> BIND_DEFAULT_SYMBOL(rte_cryptodev_info_get, _v20, 20);
>>>
>>> The correct call to VERSION_SYMBOL is already above.
>> [Fiona] ok, so won't do this.
>>
>>
>>>> Ray, can you please suggest if it required or not? And what all we need to check?
>>>
>>> See below.
>>>
>>>>
>>>> The patch is still showing Incompatibilities
>>>> NOTICE: ABI may be incompatible, check reports/logs for details.
>>>> NOTICE: Incompatible list: librte_cryptodev.so
>>>
>>> So I looked through the issues it is complaining about, these are here.
>>> https://travis-ci.com/github/ovsrobot/dpdk/jobs/320526253#L4540
>>>
>>> Basically they all are warnings related to the changes to the enumeration
>>> rte_crypto_aead_algorithm.
>>>
>>> Essentially the new member RTE_CRYPTO_AEAD_CHACHA20_POLY1305.
>>> The change to the end value RTE_CRYPTO_AEAD_LIST_END.
>>> Members of this type "enum rte_crypto_aead_algorithm algo" are demeeded to also have changed,
>>> but they haven't.
>>>
>>> With the additional work to create the v20 version of rte_cryptodev_info_get.
>>> I think all reasonable steps have been been taken here.
>> [Fiona] Do we need to change the tool or somehow mark as a false positive?
>
> Yes, I will take a look at it.
So take a look at devtools/libabigail.abignore.
Suppressions on 'enum rte_crypto_aead_algorithm' & 'rte_crypto_aead_algorithm_strings',
_should_ take care of it.
Please check with test_build.sh & test_meson_build.sh with DPDK_ABI_REF_VERSION=v19.11
>
>>
>>
>>>>> /**
>>>>> * Register a callback function for specific device id.
>>>>> diff --git a/lib/librte_cryptodev/rte_cryptodev_version.map
>>>>> b/lib/librte_cryptodev/rte_cryptodev_version.map
>>>>> index 6e41b4b..512a4a7 100644
>>>>> --- a/lib/librte_cryptodev/rte_cryptodev_version.map
>>>>> +++ b/lib/librte_cryptodev/rte_cryptodev_version.map
>>>>> @@ -58,6 +58,13 @@ DPDK_20.0 {
>>>>> local: *;
>>>>> };
>>>>>
>>>>> +DPDK_21 {
>>>
>>> Should be DPDK_21.0
>> [Fiona] Can you explain why?
>> I thought it could go back to a 2-number system with _v21 ABI.
>> I thought we'd clarified that DPDK_20.0 is only there due to a mistake, that should have been DPDK_20.
>>
>>
>>>>> + global:
>>>>> + rte_cryptodev_info_get;
>>>>> + rte_cryptodev_sym_capability_get;
>>>>> +} DPDK_20.0;
>>>>> +
>>>>> +
>>>>> EXPERIMENTAL {
>>>>> global:
>>>>>
>>>>> --
>>>>> 2.1.0
>>>>
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH] abi: change references to abi 20.0.1 to abi v21
2020-04-22 8:18 9% ` Thomas Monjalon
@ 2020-04-22 8:28 6% ` Ray Kinsella
0 siblings, 0 replies; 200+ results
From: Ray Kinsella @ 2020-04-22 8:28 UTC (permalink / raw)
To: Thomas Monjalon, David Marchand
Cc: dev, Wang, Haiyue, Matan Azrad, Anoob Joseph, Yigit, Ferruh,
Mahipal Challa, Eelco Chaudron, Cristian Dumitrescu, Jingjing Wu,
Wenzhuo Lu, Shahaf Shuler, Viacheslav Ovsiienko,
Jerin Jacob Kollanukkaran, Nithin Dabilpuram,
Alfredo Cardigliano, Neil Horman
On 22/04/2020 09:18, Thomas Monjalon wrote:
> 22/04/2020 10:07, Ray Kinsella:
>> On 20/04/2020 13:20, David Marchand wrote:
>>> On Mon, Apr 20, 2020 at 1:57 PM Ray Kinsella <mdr@ashroe.eu> wrote:
>>>>
>>>> Travis ABI check warnings, can be safely ignored in this case, I think.
>>>>
>>>> https://travis-ci.com/github/ovsrobot/dpdk/builds/161009923
>>>
>>> How about comparing to 19.11 ABI then?
>>>
>> Concluded on separate thread it is best to leave this as v20.02.
>
> It means the ABI tool will be reporting a failure?
> The tool needs to reliably report zero false positive.
> How can we mitigate this issue?
>
>
So I suggest we handle by adding
[suppress_function]
symbol_version = 20.0.1
[suppress_variable]
symbol_version = 20.0.1
to libabigail.abignore.
libabigail will then ignore the removals?
Ray K
^ permalink raw reply [relevance 6%]
* Re: [dpdk-dev] [PATCH] cryptodev: version rte_cryptodev_info_get function
2020-04-21 9:36 0% ` Thomas Monjalon
@ 2020-04-22 8:21 0% ` Ray Kinsella
0 siblings, 0 replies; 200+ results
From: Ray Kinsella @ 2020-04-22 8:21 UTC (permalink / raw)
To: Thomas Monjalon
Cc: Trahe, Fiona, Richardson, Bruce, dev, Kusztal, ArkadiuszX,
Neil Horman, Luca Boccassi, Kevin Traynor, Yigit, Ferruh,
Akhil Goyal, David Marchand
On 21/04/2020 10:36, Thomas Monjalon wrote:
> 21/04/2020 08:01, Ray Kinsella:
>>
>> On 20/04/2020 18:37, Thomas Monjalon wrote:
>>> 20/04/2020 19:31, Ray Kinsella:
>>>>
>>>> Our only commitment is to the stability of the v19.11/v20 ABI, until v21.
>>>>
>>>> That said, once an ABI migrates from EXPERIMENTAL to v21, it _shouldn't_ be changing.
>>>> We don't have a strict commitment to the v21 ABI until v20.11.
>>>>
>>>> However if v21 is changing across quarterlies (outside of additions) ... something else is wrong.
>>>
>>> The only way to check a symbol is not changing in a quarterly release
>>> is to test it. That's what we wanted to enforce:
>>> compare 20.02 ABI in 20.05 release.
>>>
>>> What other process do you suggest?
>>>
>>
>> Well I guess it's understanding the reason why you are doing something.
>> I can see reasons for wanting to test against both v19.11 and v20.02.
>>
>> v19.11 because our strict commitment is to the v20 abi.
>> v20.02 to ensure that v21 symbols are not changing between quarterly releases.
>>
>> On v20, since you tested v20.02 against v19.11 during the v20.02 release cycle.
>> The v20 symbols should not have changed during the v20.02 release cycle.
>>
>> I take your point, that then testing v20.05 against v20.02 would catch both v20 and v21 changes.
>
> OK, so we need a policy or process update to make this conclusion clear to everybody.
>
yes, I replied to David Marchand on this separately.
In the case the process is already in place ... Travis is routinely testing against v20.02.
If we wanted to make this abundantly clear we could add a note/section explaining the process to
guides/contributing/abi_versioning.rst ?
So I think we need two other policy additions.
1. Fiona is clear, that we need a note on the policy document to cover the version numbering error
that was made in DPDK 19.11. Bruce's commit (f26c2b39b271cdcd857ba518c5e48c78cb1c30af) on the subject,
explains it very well ...
2. We also need to merge "doc: alias to experimental tag for stable apis"
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH] abi: change references to abi 20.0.1 to abi v21
2020-04-22 8:07 6% ` Ray Kinsella
2020-04-22 8:11 6% ` David Marchand
@ 2020-04-22 8:18 9% ` Thomas Monjalon
2020-04-22 8:28 6% ` Ray Kinsella
1 sibling, 1 reply; 200+ results
From: Thomas Monjalon @ 2020-04-22 8:18 UTC (permalink / raw)
To: David Marchand, Ray Kinsella
Cc: dev, Wang, Haiyue, Matan Azrad, Anoob Joseph, Yigit, Ferruh,
Mahipal Challa, Eelco Chaudron, Cristian Dumitrescu, Jingjing Wu,
Wenzhuo Lu, Shahaf Shuler, Viacheslav Ovsiienko,
Jerin Jacob Kollanukkaran, Nithin Dabilpuram,
Alfredo Cardigliano, Neil Horman
22/04/2020 10:07, Ray Kinsella:
> On 20/04/2020 13:20, David Marchand wrote:
> > On Mon, Apr 20, 2020 at 1:57 PM Ray Kinsella <mdr@ashroe.eu> wrote:
> >>
> >> Travis ABI check warnings, can be safely ignored in this case, I think.
> >>
> >> https://travis-ci.com/github/ovsrobot/dpdk/builds/161009923
> >
> > How about comparing to 19.11 ABI then?
> >
> Concluded on separate thread it is best to leave this as v20.02.
It means the ABI tool will be reporting a failure?
The tool needs to reliably report zero false positive.
How can we mitigate this issue?
^ permalink raw reply [relevance 9%]
* Re: [dpdk-dev] [PATCH] abi: change references to abi 20.0.1 to abi v21
2020-04-22 8:07 6% ` Ray Kinsella
@ 2020-04-22 8:11 6% ` David Marchand
2020-04-22 8:18 9% ` Thomas Monjalon
1 sibling, 0 replies; 200+ results
From: David Marchand @ 2020-04-22 8:11 UTC (permalink / raw)
To: Ray Kinsella
Cc: dev, Wang, Haiyue, Matan Azrad, Anoob Joseph, Yigit, Ferruh,
Mahipal Challa, Eelco Chaudron, Cristian Dumitrescu,
Thomas Monjalon, Jingjing Wu, Wenzhuo Lu, Shahaf Shuler,
Viacheslav Ovsiienko, Jerin Jacob Kollanukkaran,
Nithin Dabilpuram, Alfredo Cardigliano, Neil Horman
On Wed, Apr 22, 2020 at 10:08 AM Ray Kinsella <mdr@ashroe.eu> wrote:
>
> Concluded on separate thread it is best to leave this as v20.02.
Ok, thanks for confirming.
--
David Marchand
^ permalink raw reply [relevance 6%]
* Re: [dpdk-dev] [PATCH] abi: change references to abi 20.0.1 to abi v21
@ 2020-04-22 8:07 6% ` Ray Kinsella
2020-04-22 8:11 6% ` David Marchand
2020-04-22 8:18 9% ` Thomas Monjalon
0 siblings, 2 replies; 200+ results
From: Ray Kinsella @ 2020-04-22 8:07 UTC (permalink / raw)
To: David Marchand
Cc: dev, Wang, Haiyue, Matan Azrad, Anoob Joseph, Yigit, Ferruh,
Mahipal Challa, Eelco Chaudron, Cristian Dumitrescu,
Thomas Monjalon, Jingjing Wu, Wenzhuo Lu, Shahaf Shuler,
Viacheslav Ovsiienko, Jerin Jacob Kollanukkaran,
Nithin Dabilpuram, Alfredo Cardigliano, Neil Horman
Concluded on separate thread it is best to leave this as v20.02.
Thanks,
Ray K
On 20/04/2020 13:20, David Marchand wrote:
> On Mon, Apr 20, 2020 at 1:57 PM Ray Kinsella <mdr@ashroe.eu> wrote:
>>
>> Travis ABI check warnings, can be safely ignored in this case, I think.
>>
>> https://travis-ci.com/github/ovsrobot/dpdk/builds/161009923
>
> How about comparing to 19.11 ABI then?
>
>
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -36,7 +36,7 @@ script: ./.ci/${TRAVIS_OS_NAME}-build.sh
>
> env:
> global:
> - - REF_GIT_TAG=v20.02
> + - REF_GIT_TAG=v19.11
>
> jobs:
> include:
>
>
^ permalink raw reply [relevance 6%]
* [dpdk-dev] [PATCH v9 2/2] eal: support for VFIO-PCI VF token
2020-04-22 5:08 4% ` [dpdk-dev] [PATCH v9 " Haiyue Wang
@ 2020-04-22 5:08 3% ` Haiyue Wang
0 siblings, 1 reply; 200+ results
From: Haiyue Wang @ 2020-04-22 5:08 UTC (permalink / raw)
To: dev, anatoly.burakov, thomas, vattunuru, jerinj, alex.williamson,
david.marchand
Cc: Haiyue Wang
The kernel module vfio-pci introduces the VF token to enable SR-IOV
support since 5.7.
The VF token can be set by a vfio-pci based PF driver and must be known
by the vfio-pci based VF driver in order to gain access to the device.
Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
Acked-by: Vamsi Attunuru <vattunuru@marvell.com>
Tested-by: Vamsi Attunuru <vattunuru@marvell.com>
---
devtools/libabigail.abignore | 2 +
doc/guides/linux_gsg/linux_drivers.rst | 41 +++++++++++++-
doc/guides/rel_notes/release_20_05.rst | 5 ++
drivers/bus/pci/linux/pci_vfio.c | 74 +++++++++++++++++++++++++-
lib/librte_eal/freebsd/eal.c | 3 +-
lib/librte_eal/include/rte_vfio.h | 24 ++++++++-
lib/librte_eal/linux/eal_vfio.c | 20 +++++--
7 files changed, 161 insertions(+), 8 deletions(-)
diff --git a/devtools/libabigail.abignore b/devtools/libabigail.abignore
index cd86d89ca..01d987a1e 100644
--- a/devtools/libabigail.abignore
+++ b/devtools/libabigail.abignore
@@ -6,6 +6,8 @@
; Explicit ignore for driver-only ABI
[suppress_type]
name = rte_cryptodev_ops
+[suppress_function]
+ name = rte_vfio_setup_device
; Ignore this enum update as it is part of an experimental API
[suppress_type]
type_kind = enum
diff --git a/doc/guides/linux_gsg/linux_drivers.rst b/doc/guides/linux_gsg/linux_drivers.rst
index 238f3e900..b42fd708b 100644
--- a/doc/guides/linux_gsg/linux_drivers.rst
+++ b/doc/guides/linux_gsg/linux_drivers.rst
@@ -72,11 +72,50 @@ Note that in order to use VFIO, your kernel must support it.
VFIO kernel modules have been included in the Linux kernel since version 3.6.0 and are usually present by default,
however please consult your distributions documentation to make sure that is the case.
+The ``vfio-pci`` module since Linux version 5.7 supports the creation of virtual
+functions. After the PF is bound to vfio-pci module, the user can create the VFs
+by sysfs interface, and these VFs are bound to vfio-pci module automatically.
+
+When the PF is bound to vfio-pci, it has initial VF token generated by random. For
+security reason, this token is write only, the user can't read it from the kernel
+directly. For accessing the VF, the user needs to start the PF with token parameter
+to setup a VF token (uuid format), then the VF can be accessed with this new known
+VF token.
+
+Also if the DPDK application running on the PF device exits, the user wants to start
+the PF with another different VF token value, it has no issue if no application like
+DPDK or KVM runs on VFs. Otherwise, the PF will fail to start until all VFs are free
+to use, after that, the user can select a new VF token to start the PF device.
+
+DPDK will use the keyword ``vf_token`` as the device argument to pass the VF token
+value to PF and its related VFs, the PMD should not use it, and this argument will
+be pruned from the device argument list, so the PMD can parse its own valid device
+arguments successfully.
+
+.. code-block:: console
+
+ 1. Generate the VF token by uuid command
+ 14d63f20-8445-11ea-8900-1f9ce7d5650d
+
+ 2. sudo modprobe vfio-pci enable_sriov=1
+
+ 2. ./usertools/dpdk-devbind.py -b vfio-pci 0000:86:00.0
+
+ 3. echo 2 > /sys/bus/pci/devices/0000:86:00.0/sriov_numvfs
+
+ 4. Start the PF:
+ ./x86_64-native-linux-gcc/app/testpmd -l 22-25 -n 4 \
+ -w 86:00.0,vf_token=14d63f20-8445-11ea-8900-1f9ce7d5650d --file-prefix=pf -- -i
+
+ 5. Start the VF:
+ ./x86_64-native-linux-gcc/app/testpmd -l 26-29 -n 4 \
+ -w 86:02.0,vf_token=14d63f20-8445-11ea-8900-1f9ce7d5650d --file-prefix=vf0 -- -i
+
Also, to use VFIO, both kernel and BIOS must support and be configured to use IO virtualization (such as Intel® VT-d).
.. note::
- ``vfio-pci`` module doesn't support the creation of virtual functions.
+ ``vfio-pci`` module doesn't support the creation of virtual functions before Linux version 5.7.
For proper operation of VFIO when running DPDK applications as a non-privileged user, correct permissions should also be set up.
This can be done by using the DPDK setup script (called dpdk-setup.sh and located in the usertools directory).
diff --git a/doc/guides/rel_notes/release_20_05.rst b/doc/guides/rel_notes/release_20_05.rst
index 709372e5e..9460e1eb2 100644
--- a/doc/guides/rel_notes/release_20_05.rst
+++ b/doc/guides/rel_notes/release_20_05.rst
@@ -97,6 +97,11 @@ New Features
by making use of the event device capabilities. The event mode currently supports
only inline IPsec protocol offload.
+* **Added the support for vfio-pci new VF token interface.**
+
+ Since Linux version 5.7, vfio-pci supports a shared VF token (UUID) to represent
+ the trust between SR-IOV PF and the created VFs. Update the method to gain access
+ to the device by appending the VF token.
Removed Items
-------------
diff --git a/drivers/bus/pci/linux/pci_vfio.c b/drivers/bus/pci/linux/pci_vfio.c
index 64cd84a68..efb64e2ba 100644
--- a/drivers/bus/pci/linux/pci_vfio.c
+++ b/drivers/bus/pci/linux/pci_vfio.c
@@ -11,6 +11,7 @@
#include <sys/mman.h>
#include <stdbool.h>
+#include <rte_devargs.h>
#include <rte_log.h>
#include <rte_pci.h>
#include <rte_bus_pci.h>
@@ -644,12 +645,72 @@ pci_vfio_msix_is_mappable(int vfio_dev_fd, int msix_region)
return ret;
}
+static int
+vfio_pci_vf_token_arg(struct rte_devargs *devargs, rte_uuid_t uuid)
+{
+#define VF_TOKEN_ARG "vf_token="
+ char c, *p, *vf_token;
+
+ memset(uuid, 0, sizeof(rte_uuid_t));
+
+ if (devargs == NULL)
+ return 0;
+
+ p = strstr(devargs->args, VF_TOKEN_ARG);
+ if (!p)
+ return 0;
+
+ vf_token = p + strlen(VF_TOKEN_ARG);
+ if (strlen(vf_token) < (RTE_UUID_STRLEN - 1)) {
+ RTE_LOG(ERR, EAL, "The VF token length is too short\n");
+ return -1;
+ }
+
+ c = vf_token[RTE_UUID_STRLEN - 1];
+ if (c != '\0' && c != ',') {
+ RTE_LOG(ERR, EAL,
+ "The VF token ends with a invalid character : %c\n", c);
+ return -1;
+ }
+
+ vf_token[RTE_UUID_STRLEN - 1] = '\0';
+ if (rte_uuid_parse(vf_token, uuid)) {
+ RTE_LOG(ERR, EAL,
+ "The VF token is invalid : %s\n", vf_token);
+ vf_token[RTE_UUID_STRLEN - 1] = c;
+ return -1;
+ }
+
+ RTE_LOG(DEBUG, EAL,
+ "The VF token is found : %s\n", vf_token);
+
+ vf_token[RTE_UUID_STRLEN - 1] = c;
+
+ /* This VF token will be treated as a invalid device argument if the
+ * PMD calls the rte_devargs parse API with its own valid argument list,
+ * so it needs to purge this vfio-pci specific argument.
+ */
+ if (c != '\0') {
+ /* 1. Handle the case : 'vf_token=uuid,arg1=val1' */
+ memmove(p, vf_token + RTE_UUID_STRLEN,
+ strlen(vf_token + RTE_UUID_STRLEN) + 1);
+ } else {
+ /* 2. Handle the case : 'arg1=val1,vf_token=uuid' */
+ if (p != devargs->args)
+ p--;
+
+ *p = '\0';
+ }
+
+ return 0;
+}
static int
pci_vfio_map_resource_primary(struct rte_pci_device *dev)
{
struct vfio_device_info device_info = { .argsz = sizeof(device_info) };
char pci_addr[PATH_MAX] = {0};
+ rte_uuid_t vf_token;
int vfio_dev_fd;
struct rte_pci_addr *loc = &dev->addr;
int i, ret;
@@ -668,8 +729,12 @@ pci_vfio_map_resource_primary(struct rte_pci_device *dev)
snprintf(pci_addr, sizeof(pci_addr), PCI_PRI_FMT,
loc->domain, loc->bus, loc->devid, loc->function);
+ ret = vfio_pci_vf_token_arg(dev->device.devargs, vf_token);
+ if (ret)
+ return ret;
+
ret = rte_vfio_setup_device(rte_pci_get_sysfs_path(), pci_addr,
- &vfio_dev_fd, &device_info);
+ &vfio_dev_fd, &device_info, vf_token);
if (ret)
return ret;
@@ -798,6 +863,7 @@ pci_vfio_map_resource_secondary(struct rte_pci_device *dev)
{
struct vfio_device_info device_info = { .argsz = sizeof(device_info) };
char pci_addr[PATH_MAX] = {0};
+ rte_uuid_t vf_token;
int vfio_dev_fd;
struct rte_pci_addr *loc = &dev->addr;
int i, ret;
@@ -830,8 +896,12 @@ pci_vfio_map_resource_secondary(struct rte_pci_device *dev)
return -1;
}
+ ret = vfio_pci_vf_token_arg(dev->device.devargs, vf_token);
+ if (ret)
+ return ret;
+
ret = rte_vfio_setup_device(rte_pci_get_sysfs_path(), pci_addr,
- &vfio_dev_fd, &device_info);
+ &vfio_dev_fd, &device_info, vf_token);
if (ret)
return ret;
diff --git a/lib/librte_eal/freebsd/eal.c b/lib/librte_eal/freebsd/eal.c
index 80dc9aa78..86d5a5f49 100644
--- a/lib/librte_eal/freebsd/eal.c
+++ b/lib/librte_eal/freebsd/eal.c
@@ -995,7 +995,8 @@ rte_eal_vfio_intr_mode(void)
int rte_vfio_setup_device(__rte_unused const char *sysfs_base,
__rte_unused const char *dev_addr,
__rte_unused int *vfio_dev_fd,
- __rte_unused struct vfio_device_info *device_info)
+ __rte_unused struct vfio_device_info *device_info,
+ __rte_unused rte_uuid_t vf_token)
{
return -1;
}
diff --git a/lib/librte_eal/include/rte_vfio.h b/lib/librte_eal/include/rte_vfio.h
index 20ed8c45a..28d918cde 100644
--- a/lib/librte_eal/include/rte_vfio.h
+++ b/lib/librte_eal/include/rte_vfio.h
@@ -16,6 +16,8 @@ extern "C" {
#include <stdint.h>
+#include <rte_uuid.h>
+
/*
* determine if VFIO is present on the system
*/
@@ -102,13 +104,33 @@ struct vfio_device_info;
* @param device_info
* Device information.
*
+ * @param vf_token
+ * Before linux 5.7, the PF bound to vfio-pci doesn't support SR-IOV to
+ * create VFs for security reason. Now the VF token is introduced to work
+ * as some degree of trust or collaboration between PF and VFs.
+ *
+ * A). as VF device, if the PF is a vfio device and it is bound to the
+ * vfio-pci driver, the user needs to provide a VF token to access the
+ * device, in the form of appending a vf_token to the device name, for
+ * example:
+ * "-w 04:10.0,vf_token=bd8d9d2b-5a5f-4f5a-a211-f591514ba1f3"
+ *
+ * B). as PF device, When presented with a PF which has VFs in use, the
+ * user must also provide the current VF token to prove collaboration with
+ * existing VF users. If VFs are not in use, the VF token provided for the
+ * PF device will act to set the VF token.
+ *
+ * The vf_token can be zero uuid, which will be ignored to pass into the
+ * vfio-pci module.
+ *
* @return
* 0 on success.
* <0 on failure.
* >1 if the device cannot be managed this way.
*/
int rte_vfio_setup_device(const char *sysfs_base, const char *dev_addr,
- int *vfio_dev_fd, struct vfio_device_info *device_info);
+ int *vfio_dev_fd, struct vfio_device_info *device_info,
+ rte_uuid_t vf_token);
/**
* Release a device mapped to a VFIO-managed I/O MMU group.
diff --git a/lib/librte_eal/linux/eal_vfio.c b/lib/librte_eal/linux/eal_vfio.c
index d26e1649a..e8d7cbda5 100644
--- a/lib/librte_eal/linux/eal_vfio.c
+++ b/lib/librte_eal/linux/eal_vfio.c
@@ -702,7 +702,8 @@ rte_vfio_clear_group(int vfio_group_fd)
int
rte_vfio_setup_device(const char *sysfs_base, const char *dev_addr,
- int *vfio_dev_fd, struct vfio_device_info *device_info)
+ int *vfio_dev_fd, struct vfio_device_info *device_info,
+ rte_uuid_t vf_token)
{
struct vfio_group_status group_status = {
.argsz = sizeof(group_status)
@@ -712,6 +713,7 @@ rte_vfio_setup_device(const char *sysfs_base, const char *dev_addr,
int vfio_container_fd;
int vfio_group_fd;
int iommu_group_num;
+ char dev[PATH_MAX];
int i, ret;
/* get group number */
@@ -895,8 +897,19 @@ rte_vfio_setup_device(const char *sysfs_base, const char *dev_addr,
t->type_id, t->name);
}
+ if (!rte_uuid_is_null(vf_token)) {
+ char vf_token_str[RTE_UUID_STRLEN];
+
+ rte_uuid_unparse(vf_token, vf_token_str, sizeof(vf_token_str));
+ snprintf(dev, sizeof(dev),
+ "%s vf_token=%s", dev_addr, vf_token_str);
+ } else {
+ snprintf(dev, sizeof(dev),
+ "%s", dev_addr);
+ }
+
/* get a file descriptor for the device */
- *vfio_dev_fd = ioctl(vfio_group_fd, VFIO_GROUP_GET_DEVICE_FD, dev_addr);
+ *vfio_dev_fd = ioctl(vfio_group_fd, VFIO_GROUP_GET_DEVICE_FD, dev);
if (*vfio_dev_fd < 0) {
/* if we cannot get a device fd, this implies a problem with
* the VFIO group or the container not having IOMMU configured.
@@ -2083,7 +2096,8 @@ int
rte_vfio_setup_device(__rte_unused const char *sysfs_base,
__rte_unused const char *dev_addr,
__rte_unused int *vfio_dev_fd,
- __rte_unused struct vfio_device_info *device_info)
+ __rte_unused struct vfio_device_info *device_info,
+ __rte_unused rte_uuid_t vf_token)
{
return -1;
}
--
2.26.2
^ permalink raw reply [relevance 3%]
* [dpdk-dev] [PATCH v9 0/2] support for VFIO-PCI VF token interface
@ 2020-04-22 5:08 4% ` Haiyue Wang
2020-04-22 5:08 3% ` [dpdk-dev] [PATCH v9 2/2] eal: support for VFIO-PCI VF token Haiyue Wang
2020-04-26 1:55 4% ` [dpdk-dev] [PATCH v10 0/2] support for VFIO-PCI VF token interface Haiyue Wang
2020-05-05 7:46 4% ` [dpdk-dev] [PATCH v11 " Haiyue Wang
3 siblings, 1 reply; 200+ results
From: Haiyue Wang @ 2020-04-22 5:08 UTC (permalink / raw)
To: dev, anatoly.burakov, thomas, vattunuru, jerinj, alex.williamson,
david.marchand
Cc: Haiyue Wang
v9: Rewrite the document.
v8: Update the document.
https://patchwork.dpdk.org/cover/68859/
v7: Add the Fixes tag in uuid, the release note and help
document.
https://patchwork.dpdk.org/cover/68845/
v6: Drop the Fixes tag in uuid, since the file has been
moved to another place, not suitable to apply on stable.
And this is not a bug, just some kind of enhancement.
https://patchwork.dpdk.org/cover/68367/
v5: 1. Add the VF token parse error handling.
2. Split into two patches for different logic module.
3. Add more comments into the code for explaining the design.
4. Drop the ABI change workaround, this patch set focuses on code review.
https://patchwork.dpdk.org/cover/68364/
v4: 1. Ignore rte_vfio_setup_device ABI check since it is
for Linux driver use.
https://patchwork.dpdk.org/patch/68255/
v3: Fix the Travis build failed:
(1). rte_uuid.h:97:55: error: unknown type name ‘size_t’
(2). rte_uuid.h:58:2: error: implicit declaration of function ‘memcpy’
https://patchwork.dpdk.org/patch/68254/
v2: Fix the FreeBSD build error.
https://patchwork.dpdk.org/patch/68240/
v1: Update the commit message.
https://patchwork.dpdk.org/patch/68237/
RFC v2: https://patchwork.dpdk.org/patch/68114/
Based on Vamsi's RFC v1, and Alex's patch for Qemu
[https://lore.kernel.org/lkml/20200204161737.34696b91@w520.home/]:
Use the devarg to pass-down the VF token.
RFC v1: https://patchwork.dpdk.org/patch/66281/ by Vamsi.
Haiyue Wang (2):
eal: add uuid dependent header files explicitly
eal: support for VFIO-PCI VF token
devtools/libabigail.abignore | 2 +
doc/guides/linux_gsg/linux_drivers.rst | 41 +++++++++++++-
doc/guides/rel_notes/release_20_05.rst | 5 ++
drivers/bus/pci/linux/pci_vfio.c | 74 +++++++++++++++++++++++++-
lib/librte_eal/freebsd/eal.c | 3 +-
lib/librte_eal/include/rte_uuid.h | 2 +
lib/librte_eal/include/rte_vfio.h | 24 ++++++++-
lib/librte_eal/linux/eal_vfio.c | 20 +++++--
8 files changed, 163 insertions(+), 8 deletions(-)
--
2.26.2
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCHv3] Remove validate-abi.sh from tree
2020-04-21 18:56 4% ` Neil Horman
@ 2020-04-21 21:42 4% ` Thomas Monjalon
2020-04-22 11:43 4% ` Ray Kinsella
2020-04-22 12:01 8% ` Neil Horman
0 siblings, 2 replies; 200+ results
From: Thomas Monjalon @ 2020-04-21 21:42 UTC (permalink / raw)
To: Neil Horman; +Cc: Ray Kinsella, dev, david.marchand
21/04/2020 20:56, Neil Horman:
> On Tue, Apr 21, 2020 at 01:46:43PM +0200, Thomas Monjalon wrote:
> > 21/04/2020 13:12, Neil Horman:
> > > On Fri, Apr 17, 2020 at 04:42:38PM +0100, Ray Kinsella wrote:
> > > > On 17/04/2020 13:10, Thomas Monjalon wrote:
> > > > > 17/04/2020 13:47, Ray Kinsella:
> > > > >> On 17/04/2020 11:20, Thomas Monjalon wrote:
> > > > >>> 17/04/2020 12:11, Ray Kinsella:
> > > > >>>> check-abi.sh appears to be backward step in terms of usability.
> > > > >>>
> > > > >>> No, check-abi.sh benefits from a nice integration in build scripts.
> > > > >>> See below.
> > > > >>>
> > > > >>>> With validate-abi.sh I do can do a "validate-abi.sh HEAD~1 HEAD".
> > > > >>>> And it will do the build, install, dump and comparison for me.
> > > > >>>> And it picked up my 20.0.2 - > 21.0 changes no problem.
> > > > >>>>
> > > > >>>> With check-abi on the other hand, I need to the build and install myself.
> > > > >>>> check-abi requires dump files, but I see no reference in the documentation to how these are created.
> > > > >>>> It silently fails when it doesn't find any ...
> > > > >>>>
> > > > >>>> Do I run abi-dumper on the so's myself, or how does it work?
> > > > >>>
> > > > >>> check-abi.sh is integrated in test-build.sh and test-meson-builds.sh.
> > > > >>> Probably we should document usage in these scripts.
> > > > >>
> > > > >> Looks like I need to set DPDK_ABI_REF_VERSION=master, not obvious.
> > > > >> Any tips or tricks would be welcome.
> > > > >
> > > > > export DPDK_ABI_REF_VERSION=v20.02
> > > > > or
> > > > > export DPDK_ABI_REF_VERSION=v19.11
> > > > >
> > > > > Depends on which compatibility you want to test...
> > > > >
> > > >
> > > > Few things ...
> > > >
> > > > 1. test-meson-build.sh keep barfing complaining about reference paths.
> > > > ValueError: dst_dir must be absolute, got reference/v19.11/build-gcc-static/usr/local/share/dpdk/examples/bbdev_app
> > > >
> > > > Under the hood, ninja install is failing complaining that it needs an absolute path.
> > > > I fixed this in test_meson_build.sh and will send a patch in a minute.
> > > > Though it's strange no-one else has seen it?
> > > >
> > > > 2. test-meson-build.sh compares the abi for the static builds, which doesn't make any sense.
> > > >
> > > > 3. test-meson-build.sh will only take a branch in DPDK_ABI_REF_VERSION that exists locally.
> > > > In order to get it to compare HEAD against HEAD~1, which you would imagine is a pretty common case.
> > > > I had a create a branch for HEAD~1, in validate-abi this a pretty simple `validate-abi HEAD~1 HEAD`
> > > >
> > > I think this code in test-meson-build.sh should probably be fixed:
> > >
> > > if [ ! -d $abirefdir/src ]; then
> > > git clone --local --no-hardlinks \
> > > --single-branch \
> > > -b $DPDK_ABI_REF_VERSION \
> > > $srcdir $abirefdir/src
> > > fi
> > >
> > > Like you noted, using -b allows us to checkout a tag/branch in the cloned
> > > repository but requires that it exist locally. We should probably prefix the
> > > checkout with a git fetch --tags
> >
> > I don't understand your concern.
> > A reference is an older version, so it should be in the git tree.
> >
> yes, but not unless you've done a recent pull or fetch. If you set
> DPDK_ABI_REF_VERSION to a tag/branch that didn't exist as of the last time you
> updated the tree, it won't be there (which it sounds like what is being
> encountered here). You can fix that by doing a git pull or git fetch prior to
> running this script (or internal to the script)
Sorry I still don't understand the case.
We want to compare the current version C with a reference R which is older.
If the reference R is not in the tree, it means the version C is not in the tree.
But C is the current version, so it is in the tree by definition.
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCHv3] Remove validate-abi.sh from tree
2020-04-21 11:46 4% ` Thomas Monjalon
@ 2020-04-21 18:56 4% ` Neil Horman
2020-04-21 21:42 4% ` Thomas Monjalon
0 siblings, 1 reply; 200+ results
From: Neil Horman @ 2020-04-21 18:56 UTC (permalink / raw)
To: Thomas Monjalon; +Cc: Ray Kinsella, dev, david.marchand
On Tue, Apr 21, 2020 at 01:46:43PM +0200, Thomas Monjalon wrote:
> 21/04/2020 13:12, Neil Horman:
> > On Fri, Apr 17, 2020 at 04:42:38PM +0100, Ray Kinsella wrote:
> > > On 17/04/2020 13:10, Thomas Monjalon wrote:
> > > > 17/04/2020 13:47, Ray Kinsella:
> > > >> On 17/04/2020 11:20, Thomas Monjalon wrote:
> > > >>> 17/04/2020 12:11, Ray Kinsella:
> > > >>>> check-abi.sh appears to be backward step in terms of usability.
> > > >>>
> > > >>> No, check-abi.sh benefits from a nice integration in build scripts.
> > > >>> See below.
> > > >>>
> > > >>>> With validate-abi.sh I do can do a "validate-abi.sh HEAD~1 HEAD".
> > > >>>> And it will do the build, install, dump and comparison for me.
> > > >>>> And it picked up my 20.0.2 - > 21.0 changes no problem.
> > > >>>>
> > > >>>> With check-abi on the other hand, I need to the build and install myself.
> > > >>>> check-abi requires dump files, but I see no reference in the documentation to how these are created.
> > > >>>> It silently fails when it doesn't find any ...
> > > >>>>
> > > >>>> Do I run abi-dumper on the so's myself, or how does it work?
> > > >>>
> > > >>> check-abi.sh is integrated in test-build.sh and test-meson-builds.sh.
> > > >>> Probably we should document usage in these scripts.
> > > >>
> > > >> Looks like I need to set DPDK_ABI_REF_VERSION=master, not obvious.
> > > >> Any tips or tricks would be welcome.
> > > >
> > > > export DPDK_ABI_REF_VERSION=v20.02
> > > > or
> > > > export DPDK_ABI_REF_VERSION=v19.11
> > > >
> > > > Depends on which compatibility you want to test...
> > > >
> > >
> > > Few things ...
> > >
> > > 1. test-meson-build.sh keep barfing complaining about reference paths.
> > > ValueError: dst_dir must be absolute, got reference/v19.11/build-gcc-static/usr/local/share/dpdk/examples/bbdev_app
> > >
> > > Under the hood, ninja install is failing complaining that it needs an absolute path.
> > > I fixed this in test_meson_build.sh and will send a patch in a minute.
> > > Though it's strange no-one else has seen it?
> > >
> > > 2. test-meson-build.sh compares the abi for the static builds, which doesn't make any sense.
> > >
> > > 3. test-meson-build.sh will only take a branch in DPDK_ABI_REF_VERSION that exists locally.
> > > In order to get it to compare HEAD against HEAD~1, which you would imagine is a pretty common case.
> > > I had a create a branch for HEAD~1, in validate-abi this a pretty simple `validate-abi HEAD~1 HEAD`
> > >
> > I think this code in test-meson-build.sh should probably be fixed:
> >
> > if [ ! -d $abirefdir/src ]; then
> > git clone --local --no-hardlinks \
> > --single-branch \
> > -b $DPDK_ABI_REF_VERSION \
> > $srcdir $abirefdir/src
> > fi
> >
> > Like you noted, using -b allows us to checkout a tag/branch in the cloned
> > repository but requires that it exist locally. We should probably prefix the
> > checkout with a git fetch --tags
>
> I don't understand your concern.
> A reference is an older version, so it should be in the git tree.
>
yes, but not unless you've done a recent pull or fetch. If you set
DPDK_ABI_REF_VERSION to a tag/branch that didn't exist as of the last time you
updated the tree, it won't be there (which it sounds like what is being
encountered here). You can fix that by doing a git pull or git fetch prior to
running this script (or internal to the script)
Neil
>
>
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH v8 0/2] support for VFIO-PCI VF token interface
2020-04-21 8:47 0% ` Thomas Monjalon
@ 2020-04-21 17:35 0% ` Wang, Haiyue
0 siblings, 0 replies; 200+ results
From: Wang, Haiyue @ 2020-04-21 17:35 UTC (permalink / raw)
To: Thomas Monjalon, David Marchand
Cc: Neil Horman, dev, Burakov, Anatoly, Vamsi Attunuru,
Jerin Jacob Kollanukkaran, Yigit, Ferruh, Richardson, Bruce,
Kinsella, Ray
> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: Tuesday, April 21, 2020 16:48
> To: David Marchand <david.marchand@redhat.com>; Wang, Haiyue <haiyue.wang@intel.com>
> Cc: Neil Horman <nhorman@tuxdriver.com>; dev <dev@dpdk.org>; Burakov, Anatoly
> <anatoly.burakov@intel.com>; Vamsi Attunuru <vattunuru@marvell.com>; Jerin Jacob Kollanukkaran
> <jerinj@marvell.com>; Yigit, Ferruh <ferruh.yigit@intel.com>; Richardson, Bruce
> <bruce.richardson@intel.com>; Kinsella, Ray <ray.kinsella@intel.com>
> Subject: Re: [PATCH v8 0/2] support for VFIO-PCI VF token interface
>
> 21/04/2020 04:52, Wang, Haiyue:
> > From: Thomas Monjalon <thomas@monjalon.net>
> > > 21/04/2020 03:38, Wang, Haiyue:
> > > > From: Thomas Monjalon <thomas@monjalon.net>
> > > > > 20/04/2020 19:37, Wang, Haiyue:
> > > > > > From: Thomas Monjalon <thomas@monjalon.net>
> > > > > > > 20/04/2020 19:02, Wang, Haiyue:
> > > > > > > > From: David Marchand <david.marchand@redhat.com>
> > > > > > > > > I had a look at the CI, I can see we are still missing bits to handle
> > > > > > > > > the ABI failure on rte_vfio_setup_device.
> > > > > > > >
> > > > > > > > Yes, not handle it now.
> > > > > > > >
> > > > > > > > If 'rte_vfio_setup_device' can be internal, not official DPDK API, then __rte_internal
> > > > > > > > is the best way to handle ABI issue.
> > > > > > >
> > > > > > > Please could you help finishing integration of __rte_internal?
> > > > > >
> > > > > > I thought it should be Neil ? "Yes, I'll get back to this today" ;-)
> > > > > >
> http://inbox.dpdk.org/dev/CAJFAV8ydLkV0afEHqbh6KeA3Box0Yxb3N0MNGtMD4S9bmSgT0A@mail.gmail.com/
> > > > >
> > > > > It did not happen after several months.
> > > > > If you want to unblock your patches, I think it is safer to finish yourself.
> > > > >
> > > >
> > > > Unfortunately, this __rte_internal will still make the ci fail to run when move the function
> > > > to INTERNAL session:
> > > [...]
> > > > +INTERNAL {
> > > > + global:
> > > > +
> > > > + # added in 20.05
> > > > + rte_vfio_setup_device;
> > > > +};
> > >
> > > Why do you need to move the symbol explicitly in .map?
> > > The tool should ignore symbols moving to internal, as an exception until 20.11.
> >
> > If not move the symbol explicitly in .map, another kind of error happened.
> >
> > ./devtools/check-abi.sh old_abi new_abi
> > Functions changes summary: 0 Removed, 1 Changed, 0 Added function
> > Variables changes summary: 0 Removed, 0 Changed, 0 Added variable
> >
> > 1 function with some indirect sub-type change:
> >
> > [C] 'function int rte_vfio_setup_device(const char*, const char*, int*, vfio_device_info*)' at
> eal_vfio.c:704:1 has some indirect sub-type changes:
> > parameter 5 of type 'unsigned char*' was added
> >
> > Error: ABI issue reported for 'abidiff --suppr ./devtools/libabigail.abignore --no-added-syms --
> headers-dir1 old_abi/include --headers-dir2 new_abi/include old_abi/dump/librte_eal.dump
> new_abi/dump/librte_eal.dump'
>
> This is what I said: you need to add a rule to ignore internal symbols.
>
Got the kind reply from Dodji, the maintainer of libabigail, his suggestion
meets what we have done for 'EXPERIMENTAL'. So it seems that have to mark
all needed functions as INTERNAL firstly, which obviously will break the CI
as the public functions are removed....
"
__PROJECT_INTERNAL_USE_ONLY_VERSION__ {
global: funA
};
...
Then, once you have all your internal functions marked in the ELF binary
with the proper ELF version string, you can tell libabigail to suppress
all functions that have that version by writing a suppression
specification file that has this content:
[suppress_function]
symbol_version = __PROJECT_INTERNAL_USE_ONLY_VERSION__
"
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH v4] ethdev: support flow aging
2020-04-21 10:11 2% ` [dpdk-dev] [PATCH v4] " Bill Zhou
@ 2020-04-21 17:13 0% ` Ferruh Yigit
2020-04-29 14:50 0% ` Tom Barbette
1 sibling, 0 replies; 200+ results
From: Ferruh Yigit @ 2020-04-21 17:13 UTC (permalink / raw)
To: Bill Zhou, orika, matan, wenzhuo.lu, jingjing.wu,
bernard.iremonger, john.mcnamara, marko.kovacevic, thomas,
arybchenko
Cc: dev
On 4/21/2020 11:11 AM, Bill Zhou wrote:
> From: Dong Zhou <dongz@mellanox.com>
>
> One of the reasons to destroy a flow is the fact that no packet matches the
> flow for "timeout" time.
> For example, when TCP\UDP sessions are suddenly closed.
>
> Currently, there is not any DPDK mechanism for flow aging and the
> applications use their own ways to detect and destroy aged-out flows.
>
> The flow aging implementation need include:
> - A new rte_flow action: RTE_FLOW_ACTION_TYPE_AGE to set the timeout and
> the application flow context for each flow.
> - A new ethdev event: RTE_ETH_EVENT_FLOW_AGED for the driver to report
> that there are new aged-out flows.
> - A new rte_flow API: rte_flow_get_aged_flows to get the aged-out flows
> contexts from the port.
> - Support input flow aging command line in Testpmd.
>
> The new event type addition in the enum is flagged as an ABI breakage, so
> an ignore rule is added for these reasons:
> - It is not changing value of existing types (except MAX)
> - The new value is not used by existing API if the event is not registered
> In general, it is safe adding new ethdev event types at the end of the
> enum, because of event callback registration mechanism.
>
> Signed-off-by: Dong Zhou <dongz@mellanox.com>
Carrying ack from prev versions:
Acked-by: Ori Kam <orika@mellanox.com>
Acked-by: Andrew Rybchenko <arybchenko@solarflare.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>
Acked-by: Matan Azrad <matan@mellanox.com>
Applied to dpdk-next-net/master, thanks.
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH v2] ethdev: support flow aging
2020-04-21 10:04 0% ` Ferruh Yigit
2020-04-21 10:09 0% ` Thomas Monjalon
@ 2020-04-21 15:59 0% ` Andrew Rybchenko
1 sibling, 0 replies; 200+ results
From: Andrew Rybchenko @ 2020-04-21 15:59 UTC (permalink / raw)
To: Ferruh Yigit, Thomas Monjalon, Bill Zhou
Cc: Ori Kam, Matan Azrad, wenzhuo.lu, jingjing.wu, bernard.iremonger,
john.mcnamara, marko.kovacevic, dev
On 4/21/20 1:04 PM, Ferruh Yigit wrote:
> On 4/20/2020 5:10 PM, Thomas Monjalon wrote:
>> 20/04/2020 16:06, Ferruh Yigit:
>>> On 4/18/2020 10:44 AM, Thomas Monjalon wrote:
>>>> 18/04/2020 07:04, Bill Zhou:
>>>>> From: Ferruh Yigit <ferruh.yigit@intel.com>
>>>>>> On 4/14/2020 9:32 AM, Dong Zhou wrote:
>>>>>>> --- a/lib/librte_ethdev/rte_ethdev.h
>>>>>>> +++ b/lib/librte_ethdev/rte_ethdev.h
>>>>>>> @@ -3015,6 +3015,7 @@ enum rte_eth_event_type {
>>>>>>> RTE_ETH_EVENT_NEW, /**< port is probed */
>>>>>>> RTE_ETH_EVENT_DESTROY, /**< port is released */
>>>>>>> RTE_ETH_EVENT_IPSEC, /**< IPsec offload related event */
>>>>>>> + RTE_ETH_EVENT_FLOW_AGED,/**< New aged-out flows is detected
>>>>>> */
>>>>>>> RTE_ETH_EVENT_MAX /**< max value of this enum */
>>>>>>> };
>>>>>>
>>>>>>
>>>>>> Just recognized that this is failing in ABI check [1], as far as last time for a
>>>>>> similar enum warning a QAT patch has been dropped, should this need to
>>>>>> wait for
>>>>>> 20.11 too?
>>>>>
>>>>> This patch is commonly used for flow aging, there are 2 other patches have
>>>>> implement flow aging in mlx5 driver reply to this patch.
>> [...]
>>>> These MAX values in enums are a pain.
>>>> We can try to think what can be done, waiting 20.11.
>>>> Not sure there is a solution, except hijacking an existing value
>>>> not used in the PMD, waiting the definitive value in 20.11...
>>>
>>> Dropping from the tree as of now, to not cause more merge conflicts, we can add
>>> it later when issue is resolved.
>>
>> Thanks for dropping, that's the right thing to do
>> when a patch is breaking ABI check.
>>
>> After some thoughts, I think it is acceptable to make a v3
>> which ignore this specific enum change. I explain my thought below:
>>
>> An enum can accept a new value at 2 conditions:
>> - added as last value (not changing old values)
>> - new value not used by existing API
>>
>> The value RTE_ETH_EVENT_FLOW_AGED meet the above 2 conditions:
>> - only RTE_ETH_EVENT_MAX is changed, which is consistent
>> - new value sent to the app only if the app registered for it
>>
>
> Same here, as far as I can see it is safe to get this change.
>
> If any DPDK API returns this enum, either as return of the API or as output
> parameter, this still can be problem, because application may use that returned
> value, this was the concern in the QAT sample.
>
> But here application registers an event and DPDK library process callback for
> it, so application callbacks won't be called for anything that application
> doesn't already know about, in that respect this should be safe for old
> applications.
>
> Not sure if we can generalize above two conditions for all enum changes, but we
> can investigate them case by case as we get the warnings.
>
>> So, except if I miss something, I suggest we add this exception:
>> Allow new value in rte_eth_event_type if added just before RTE_ETH_EVENT_MAX.
>> In other words, allow changing the value of RTE_ETH_EVENT_MAX.
>> The file to add such exception is devtools/libabigail.abignore.
>>
>
> OK to exception.
Me too
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH v6 05/33] eal/trace: implement trace operation APIs
2020-04-21 13:40 3% ` Jerin Jacob
@ 2020-04-21 14:09 0% ` David Marchand
0 siblings, 0 replies; 200+ results
From: David Marchand @ 2020-04-21 14:09 UTC (permalink / raw)
To: Jerin Jacob
Cc: Jerin Jacob Kollanukkaran, Sunil Kumar Kori, dev,
Thomas Monjalon, Bruce Richardson, Mattias Rönnblom
On Tue, Apr 21, 2020 at 3:47 PM Jerin Jacob <jerinjacobk@gmail.com> wrote:
> > > @@ -20,6 +22,151 @@ static RTE_DEFINE_PER_LCORE(int, ctf_count);
> > > static struct trace_point_head tp_list = STAILQ_HEAD_INITIALIZER(tp_list);
> > > static struct trace trace;
> > >
> > > +bool
> > > +rte_trace_is_enabled(void)
> > > +{
> > > + return trace.status;
> > > +}
> > > +
> > > +static void
> > > +trace_mode_set(rte_trace_point_t *trace, enum rte_trace_mode mode)
> > > +{
> > > + if (mode == RTE_TRACE_MODE_OVERWRITE)
> > > + __atomic_and_fetch(trace, ~__RTE_TRACE_FIELD_ENABLE_DISCARD,
> > > + __ATOMIC_RELEASE);
> > > + else
> > > + __atomic_or_fetch(trace, __RTE_TRACE_FIELD_ENABLE_DISCARD,
> > > + __ATOMIC_RELEASE);
> > > +}
> > > +
> > > +void
> > > +rte_trace_mode_set(enum rte_trace_mode mode)
> > > +{
> > > + struct trace_point *tp;
> > > +
> > > + if (rte_trace_is_enabled() == false)
> > > + return;
> >
> > rte_trace_is_enabled() returns a boolean, no need to test its value, should be:
> > if (!rte_trace_is_enabled())
>
> I like the ! scheme. I thought, DPDK community like == false scheme.
> I will change it in v7.
Not obvious, but I understand this part as talking about the boolean case:
https://doc.dpdk.org/guides/contributing/coding_style.html#null-pointers
"Do not use ! for tests unless it is a boolean, for example, use:"
> > > +int
> > > +rte_trace_point_enable(rte_trace_point_t *trace)
> > > +{
> > > + if (trace_point_is_invalid(trace))
> > > + return -ERANGE;
> > > +
> > > + __atomic_or_fetch(trace, __RTE_TRACE_FIELD_ENABLE_MASK,
> > > + __ATOMIC_RELEASE);
> > > + return 0;
> > > +}
> > > +
> > > +int
> > > +rte_trace_point_disable(rte_trace_point_t *trace)
> > > +{
> > > + if (trace_point_is_invalid(trace))
> > > + return -ERANGE;
> > > +
> > > + __atomic_and_fetch(trace, ~__RTE_TRACE_FIELD_ENABLE_MASK,
> > > + __ATOMIC_RELEASE);
> > > + return 0;
> > > +}
> >
> > For both rte_trace_point_enable/disable, we only check tracepoint validity.
> > Can't we just return a boolean, do we expect many error codes?
>
> I think, it is better to return "int" so that we may not ABI the
> change in future.
Ok, fair enough.
> > > +
> > > +int
> > > +rte_trace_pattern(const char *pattern, bool enable)
> > > +{
> > > + struct trace_point *tp;
> > > + int rc = 0, found = 0;
> > > +
> > > + STAILQ_FOREACH(tp, &tp_list, next) {
> > > + if (fnmatch(pattern, tp->name, 0) == 0) {
> > > + if (enable)
> > > + rc = rte_trace_point_enable(tp->handle);
> > > + else
> > > + rc = rte_trace_point_disable(tp->handle);
> > > + found = 1;
> > > + }
> > > + if (rc < 0)
> > > + return rc;
> > > + }
> > > +
> > > + return rc | found;
> > > +}
> > > +
> > > +int
> > > +rte_trace_regexp(const char *regex, bool enable)
> > > +{
> > > + struct trace_point *tp;
> > > + int rc = 0, found = 0;
> > > + regex_t r;
> > > +
> > > + if (regcomp(&r, regex, 0) != 0)
> > > + return -EINVAL;
> > > +
> > > + STAILQ_FOREACH(tp, &tp_list, next) {
> > > + if (regexec(&r, tp->name, 0, NULL, 0) == 0) {
> > > + if (enable)
> > > + rc = rte_trace_point_enable(tp->handle);
> > > + else
> > > + rc = rte_trace_point_disable(tp->handle);
> > > + found = 1;
> > > + }
> > > + if (rc < 0)
> > > + return rc;
> > > + }
> > > + regfree(&r);
> > > +
> > > + return rc | found;
> > > +}
> >
> > For both rte_trace_pattern/rte_trace_regexp, tp is a member of tp_list.
> > It means this tracepoint went through proper registration.
> > Hence, checking for a return value from rte_trace_point_(en|dis)able
> > is unneeded.
>
> Yes. I thought of it, But I think, it is safe to check the return code
> as it absorbs any
> future rte_trace_point_enable() changes. It slow-path code, so it OK IMO.
Would be odd to predict what happened if we broke for this loop on one
tracepoint error, but ok to leave as is.
--
David Marchand
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH v6 05/33] eal/trace: implement trace operation APIs
@ 2020-04-21 13:40 3% ` Jerin Jacob
2020-04-21 14:09 0% ` David Marchand
0 siblings, 1 reply; 200+ results
From: Jerin Jacob @ 2020-04-21 13:40 UTC (permalink / raw)
To: David Marchand
Cc: Jerin Jacob Kollanukkaran, Sunil Kumar Kori, dev,
Thomas Monjalon, Bruce Richardson, Mattias Rönnblom
On Tue, Apr 21, 2020 at 6:19 PM David Marchand
<david.marchand@redhat.com> wrote:
>
> On Sun, Apr 19, 2020 at 12:02 PM <jerinj@marvell.com> wrote:
> > diff --git a/lib/librte_eal/common/eal_common_trace.c b/lib/librte_eal/common/eal_common_trace.c
> > index 5c5cbd2a1..1ca702f68 100644
> > --- a/lib/librte_eal/common/eal_common_trace.c
> > +++ b/lib/librte_eal/common/eal_common_trace.c
> > @@ -3,7 +3,9 @@
> > */
> >
> > #include <inttypes.h>
> > +#include <fnmatch.h>
> > #include <sys/queue.h>
> > +#include <regex.h>
>
> Alphabetical sort when possible.
Ack. Wil fix it v7
>
> >
> > #include <rte_common.h>
> > #include <rte_errno.h>
> > @@ -20,6 +22,151 @@ static RTE_DEFINE_PER_LCORE(int, ctf_count);
> > static struct trace_point_head tp_list = STAILQ_HEAD_INITIALIZER(tp_list);
> > static struct trace trace;
> >
> > +bool
> > +rte_trace_is_enabled(void)
> > +{
> > + return trace.status;
> > +}
> > +
> > +static void
> > +trace_mode_set(rte_trace_point_t *trace, enum rte_trace_mode mode)
> > +{
> > + if (mode == RTE_TRACE_MODE_OVERWRITE)
> > + __atomic_and_fetch(trace, ~__RTE_TRACE_FIELD_ENABLE_DISCARD,
> > + __ATOMIC_RELEASE);
> > + else
> > + __atomic_or_fetch(trace, __RTE_TRACE_FIELD_ENABLE_DISCARD,
> > + __ATOMIC_RELEASE);
> > +}
> > +
> > +void
> > +rte_trace_mode_set(enum rte_trace_mode mode)
> > +{
> > + struct trace_point *tp;
> > +
> > + if (rte_trace_is_enabled() == false)
> > + return;
>
> rte_trace_is_enabled() returns a boolean, no need to test its value, should be:
> if (!rte_trace_is_enabled())
I like the ! scheme. I thought, DPDK community like == false scheme.
I will change it in v7.
>
>
> > +
> > + STAILQ_FOREACH(tp, &tp_list, next)
> > + trace_mode_set(tp->handle, mode);
> > +
> > + trace.mode = mode;
> > +}
> > +
> > +enum
> > +rte_trace_mode rte_trace_mode_get(void)
> > +{
> > + return trace.mode;
> > +}
> > +
> > +static bool
> > +trace_point_is_invalid(rte_trace_point_t *t)
> > +{
> > + if (t == NULL)
> > + return false;
>
> Should be "return true"?
>
> Or maybe simply rewrite as:
>
> static bool
> trace_point_is_invalid(rte_trace_point_t *t)
> {
> return (t == NULL) || (trace_id_get(t) >= trace.nb_trace_points);
> }
Ack.
>
> > +
> > + if (trace_id_get(t) >= trace.nb_trace_points)
> > + return true;
> > +
> > + return false;
> > +}
> > +
> > +bool
> > +rte_trace_point_is_enabled(rte_trace_point_t *trace)
> > +{
> > + uint64_t val;
> > +
> > + if (trace_point_is_invalid(trace))
> > + return false;
> > +
> > + val = __atomic_load_n(trace, __ATOMIC_ACQUIRE);
> > + return val & __RTE_TRACE_FIELD_ENABLE_MASK;
>
> We return a boolean, should be
> return (val & __RTE_TRACE_FIELD_ENABLE_MASK) != 0;
Ack.
>
>
> > +}
> > +
> > +int
> > +rte_trace_point_enable(rte_trace_point_t *trace)
> > +{
> > + if (trace_point_is_invalid(trace))
> > + return -ERANGE;
> > +
> > + __atomic_or_fetch(trace, __RTE_TRACE_FIELD_ENABLE_MASK,
> > + __ATOMIC_RELEASE);
> > + return 0;
> > +}
> > +
> > +int
> > +rte_trace_point_disable(rte_trace_point_t *trace)
> > +{
> > + if (trace_point_is_invalid(trace))
> > + return -ERANGE;
> > +
> > + __atomic_and_fetch(trace, ~__RTE_TRACE_FIELD_ENABLE_MASK,
> > + __ATOMIC_RELEASE);
> > + return 0;
> > +}
>
> For both rte_trace_point_enable/disable, we only check tracepoint validity.
> Can't we just return a boolean, do we expect many error codes?
I think, it is better to return "int" so that we may not ABI the
change in future.
>
>
> > +
> > +int
> > +rte_trace_pattern(const char *pattern, bool enable)
> > +{
> > + struct trace_point *tp;
> > + int rc = 0, found = 0;
> > +
> > + STAILQ_FOREACH(tp, &tp_list, next) {
> > + if (fnmatch(pattern, tp->name, 0) == 0) {
> > + if (enable)
> > + rc = rte_trace_point_enable(tp->handle);
> > + else
> > + rc = rte_trace_point_disable(tp->handle);
> > + found = 1;
> > + }
> > + if (rc < 0)
> > + return rc;
> > + }
> > +
> > + return rc | found;
> > +}
> > +
> > +int
> > +rte_trace_regexp(const char *regex, bool enable)
> > +{
> > + struct trace_point *tp;
> > + int rc = 0, found = 0;
> > + regex_t r;
> > +
> > + if (regcomp(&r, regex, 0) != 0)
> > + return -EINVAL;
> > +
> > + STAILQ_FOREACH(tp, &tp_list, next) {
> > + if (regexec(&r, tp->name, 0, NULL, 0) == 0) {
> > + if (enable)
> > + rc = rte_trace_point_enable(tp->handle);
> > + else
> > + rc = rte_trace_point_disable(tp->handle);
> > + found = 1;
> > + }
> > + if (rc < 0)
> > + return rc;
> > + }
> > + regfree(&r);
> > +
> > + return rc | found;
> > +}
>
> For both rte_trace_pattern/rte_trace_regexp, tp is a member of tp_list.
> It means this tracepoint went through proper registration.
> Hence, checking for a return value from rte_trace_point_(en|dis)able
> is unneeded.
Yes. I thought of it, But I think, it is safe to check the return code
as it absorbs any
future rte_trace_point_enable() changes. It slow-path code, so it OK IMO.
>
>
> --
> David Marchand
>
^ permalink raw reply [relevance 3%]
* Re: [dpdk-dev] [PATCHv3] Remove validate-abi.sh from tree
2020-04-21 11:12 4% ` Neil Horman
@ 2020-04-21 11:46 4% ` Thomas Monjalon
2020-04-21 18:56 4% ` Neil Horman
0 siblings, 1 reply; 200+ results
From: Thomas Monjalon @ 2020-04-21 11:46 UTC (permalink / raw)
To: Ray Kinsella, Neil Horman; +Cc: dev, david.marchand
21/04/2020 13:12, Neil Horman:
> On Fri, Apr 17, 2020 at 04:42:38PM +0100, Ray Kinsella wrote:
> > On 17/04/2020 13:10, Thomas Monjalon wrote:
> > > 17/04/2020 13:47, Ray Kinsella:
> > >> On 17/04/2020 11:20, Thomas Monjalon wrote:
> > >>> 17/04/2020 12:11, Ray Kinsella:
> > >>>> check-abi.sh appears to be backward step in terms of usability.
> > >>>
> > >>> No, check-abi.sh benefits from a nice integration in build scripts.
> > >>> See below.
> > >>>
> > >>>> With validate-abi.sh I do can do a "validate-abi.sh HEAD~1 HEAD".
> > >>>> And it will do the build, install, dump and comparison for me.
> > >>>> And it picked up my 20.0.2 - > 21.0 changes no problem.
> > >>>>
> > >>>> With check-abi on the other hand, I need to the build and install myself.
> > >>>> check-abi requires dump files, but I see no reference in the documentation to how these are created.
> > >>>> It silently fails when it doesn't find any ...
> > >>>>
> > >>>> Do I run abi-dumper on the so's myself, or how does it work?
> > >>>
> > >>> check-abi.sh is integrated in test-build.sh and test-meson-builds.sh.
> > >>> Probably we should document usage in these scripts.
> > >>
> > >> Looks like I need to set DPDK_ABI_REF_VERSION=master, not obvious.
> > >> Any tips or tricks would be welcome.
> > >
> > > export DPDK_ABI_REF_VERSION=v20.02
> > > or
> > > export DPDK_ABI_REF_VERSION=v19.11
> > >
> > > Depends on which compatibility you want to test...
> > >
> >
> > Few things ...
> >
> > 1. test-meson-build.sh keep barfing complaining about reference paths.
> > ValueError: dst_dir must be absolute, got reference/v19.11/build-gcc-static/usr/local/share/dpdk/examples/bbdev_app
> >
> > Under the hood, ninja install is failing complaining that it needs an absolute path.
> > I fixed this in test_meson_build.sh and will send a patch in a minute.
> > Though it's strange no-one else has seen it?
> >
> > 2. test-meson-build.sh compares the abi for the static builds, which doesn't make any sense.
> >
> > 3. test-meson-build.sh will only take a branch in DPDK_ABI_REF_VERSION that exists locally.
> > In order to get it to compare HEAD against HEAD~1, which you would imagine is a pretty common case.
> > I had a create a branch for HEAD~1, in validate-abi this a pretty simple `validate-abi HEAD~1 HEAD`
> >
> I think this code in test-meson-build.sh should probably be fixed:
>
> if [ ! -d $abirefdir/src ]; then
> git clone --local --no-hardlinks \
> --single-branch \
> -b $DPDK_ABI_REF_VERSION \
> $srcdir $abirefdir/src
> fi
>
> Like you noted, using -b allows us to checkout a tag/branch in the cloned
> repository but requires that it exist locally. We should probably prefix the
> checkout with a git fetch --tags
I don't understand your concern.
A reference is an older version, so it should be in the git tree.
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH v7 00/10] New sync modes for ring
@ 2020-04-21 11:31 0% ` David Marchand
0 siblings, 0 replies; 200+ results
From: David Marchand @ 2020-04-21 11:31 UTC (permalink / raw)
To: Konstantin Ananyev, Honnappa Nagarahalli
Cc: dev, jielong.zjl, Pavan Nikhilesh, Jerin Jacob Kollanukkaran,
Thomas Monjalon
On Mon, Apr 20, 2020 at 2:28 PM Konstantin Ananyev
<konstantin.ananyev@intel.com> wrote:
> These days more and more customers use(/try to use) DPDK based apps within
> overcommitted systems (multiple acttive threads over same pysical cores):
> VM, container deployments, etc.
> One quite common problem they hit:
> Lock-Holder-Preemption/Lock-Waiter-Preemption with rte_ring.
> LHP is quite a common problem for spin-based sync primitives
> (spin-locks, etc.) on overcommitted systems.
> The situation gets much worse when some sort of
> fair-locking technique is used (ticket-lock, etc.).
> As now not only lock-owner but also lock-waiters scheduling
> order matters a lot (LWP).
> These two problems are well-known for kernel within VMs:
> http://www-archive.xenproject.org/files/xensummitboston08/LHP.pdf
> https://www.cs.hs-rm.de/~kaiser/events/wamos2017/Slides/selcuk.pdf
> The problem with rte_ring is that while head accusion is sort of
> un-fair locking, waiting on tail is very similar to ticket lock schema -
> tail has to be updated in particular order.
> That makes current rte_ring implementation to perform
> really pure on some overcommited scenarios.
> It is probably not possible to completely resolve LHP problem in
> userspace only (without some kernel communication/intervention).
> But removing fairness at tail update helps to avoid LWP and
> can mitigate the situation significantly.
> This patch proposes two new optional ring synchronization modes:
> 1) Head/Tail Sync (HTS) mode
> In that mode enqueue/dequeue operation is fully serialized:
> only one thread at a time is allowed to perform given op.
> As another enhancement provide ability to split enqueue/dequeue
> operation into two phases:
> - enqueue/dequeue start
> - enqueue/dequeue finish
> That allows user to inspect objects in the ring without removing
> them from it (aka MT safe peek).
> 2) Relaxed Tail Sync (RTS)
> The main difference from original MP/MC algorithm is that
> tail value is increased not by every thread that finished enqueue/dequeue,
> but only by the last one.
> That allows threads to avoid spinning on ring tail value,
> leaving actual tail value change to the last thread in the update queue.
>
> Note that these new sync modes are optional.
> For current rte_ring users nothing should change
> (both in terms of API/ABI and performance).
> Existing sync modes MP/MC,SP/SC kept untouched, set up in the same
> way (via flags and _init_), and MP/MC remains as default one.
> The only thing that changed:
> Format of prod/cons now could differ depending on mode selected at _init_.
> So user has to stick with one sync model through whole ring lifetime.
> In other words, user can't create a ring for let say SP mode and then
> in the middle of data-path change his mind and start using MP_RTS mode.
> For existing modes (SP/MP, SC/MC) format remains the same and
> user can still use them interchangeably, though of course it is an
> error prone practice.
>
> Test results on IA (see below) show significant improvements
> for average enqueue/dequeue op times on overcommitted systems.
> For 'classic' DPDK deployments (one thread per core) original MP/MC
> algorithm still shows best numbers, though for 64-bit target
> RTS numbers are not that far away.
> Numbers were produced by new UT test-case: ring_stress_autotest, i.e.:
> echo ring_stress_autotest | ./dpdk-test -n 4 --lcores='...'
>
> X86_64 @ Intel(R) Xeon(R) Platinum 8160 CPU @ 2.10GHz
> DEQ+ENQ average cycles/obj
> MP/MC HTS RTS
> 1thread@1core(--lcores=6-7) 8.00 8.15 8.99
> 2thread@2core(--lcores=6-8) 19.14 19.61 20.35
> 4thread@4core(--lcores=6-10) 29.43 29.79 31.82
> 8thread@8core(--lcores=6-14) 110.59 192.81 119.50
> 16thread@16core(--lcores=6-22) 461.03 813.12 495.59
> 32thread/@32core(--lcores='6-22,55-70') 982.90 1972.38 1160.51
>
> 2thread@1core(--lcores='6,(10-11)@7' 20140.50 23.58 25.14
> 4thread@2core(--lcores='6,(10-11)@7,(20-21)@8' 153680.60 76.88 80.05
> 8thread@2core(--lcores='6,(10-13)@7,(20-23)@8' 280314.32 294.72 318.79
> 16thread@2core(--lcores='6,(10-17)@7,(20-27)@8' 643176.59 1144.02 1175.14
> 32thread@2core(--lcores='6,(10-25)@7,(30-45)@8' 4264238.80 4627.48 4892.68
>
> 8thread@2core(--lcores='6,(10-17)@(7,8))' 321085.98 298.59 307.47
> 16thread@4core(--lcores='6,(20-35)@(7-10))' 1900705.61 575.35 678.29
> 32thread@4core(--lcores='6,(20-51)@(7-10))' 5510445.85 2164.36 2714.12
>
> i686 @ Intel(R) Xeon(R) Platinum 8160 CPU @ 2.10GHz
> DEQ+ENQ average cycles/obj
> MP/MC HTS RTS
> 1thread@1core(--lcores=6-7) 7.85 12.13 11.31
> 2thread@2core(--lcores=6-8) 17.89 24.52 21.86
> 8thread@8core(--lcores=6-14) 32.58 354.20 54.58
> 32thread/@32core(--lcores='6-22,55-70') 813.77 6072.41 2169.91
>
> 2thread@1core(--lcores='6,(10-11)@7' 16095.00 36.06 34.74
> 8thread@2core(--lcores='6,(10-13)@7,(20-23)@8' 1140354.54 346.61 361.57
> 16thread@2core(--lcores='6,(10-17)@7,(20-27)@8' 1920417.86 1314.90 1416.65
>
> 8thread@2core(--lcores='6,(10-17)@(7,8))' 594358.61 332.70 357.74
> 32thread@4core(--lcores='6,(20-51)@(7-10))' 5319896.86 2836.44 3028.87
I fixed a couple of typos and split the doc updates.
Series applied with the patch from Pavan.
Thanks for the work Konstantin, Honnappa.
--
David Marchand
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCHv3] Remove validate-abi.sh from tree
@ 2020-04-21 11:12 4% ` Neil Horman
2020-04-21 11:46 4% ` Thomas Monjalon
0 siblings, 1 reply; 200+ results
From: Neil Horman @ 2020-04-21 11:12 UTC (permalink / raw)
To: Ray Kinsella; +Cc: Thomas Monjalon, dev, david.marchand
On Fri, Apr 17, 2020 at 04:42:38PM +0100, Ray Kinsella wrote:
>
>
> On 17/04/2020 13:10, Thomas Monjalon wrote:
> > 17/04/2020 13:47, Ray Kinsella:
> >> On 17/04/2020 11:20, Thomas Monjalon wrote:
> >>> 17/04/2020 12:11, Ray Kinsella:
> >>>> check-abi.sh appears to be backward step in terms of usability.
> >>>
> >>> No, check-abi.sh benefits from a nice integration in build scripts.
> >>> See below.
> >>>
> >>>> With validate-abi.sh I do can do a "validate-abi.sh HEAD~1 HEAD".
> >>>> And it will do the build, install, dump and comparison for me.
> >>>> And it picked up my 20.0.2 - > 21.0 changes no problem.
> >>>>
> >>>> With check-abi on the other hand, I need to the build and install myself.
> >>>> check-abi requires dump files, but I see no reference in the documentation to how these are created.
> >>>> It silently fails when it doesn't find any ...
> >>>>
> >>>> Do I run abi-dumper on the so's myself, or how does it work?
> >>>
> >>> check-abi.sh is integrated in test-build.sh and test-meson-builds.sh.
> >>> Probably we should document usage in these scripts.
> >>
> >> Looks like I need to set DPDK_ABI_REF_VERSION=master, not obvious.
> >> Any tips or tricks would be welcome.
> >
> > export DPDK_ABI_REF_VERSION=v20.02
> > or
> > export DPDK_ABI_REF_VERSION=v19.11
> >
> > Depends on which compatibility you want to test...
> >
>
> Few things ...
>
> 1. test-meson-build.sh keep barfing complaining about reference paths.
> ValueError: dst_dir must be absolute, got reference/v19.11/build-gcc-static/usr/local/share/dpdk/examples/bbdev_app
>
> Under the hood, ninja install is failing complaining that it needs an absolute path.
> I fixed this in test_meson_build.sh and will send a patch in a minute.
> Though it's strange no-one else has seen it?
>
> 2. test-meson-build.sh compares the abi for the static builds, which doesn't make any sense.
>
> 3. test-meson-build.sh will only take a branch in DPDK_ABI_REF_VERSION that exists locally.
> In order to get it to compare HEAD against HEAD~1, which you would imagine is a pretty common case.
> I had a create a branch for HEAD~1, in validate-abi this a pretty simple `validate-abi HEAD~1 HEAD`
>
I think this code in test-meson-build.sh should probably be fixed:
if [ ! -d $abirefdir/src ]; then
git clone --local --no-hardlinks \
--single-branch \
-b $DPDK_ABI_REF_VERSION \
$srcdir $abirefdir/src
fi
Like you noted, using -b allows us to checkout a tag/branch in the cloned
repository but requires that it exist locally. We should probably prefix the
checkout with a git fetch --tags
Neil
> Thanks,
>
> Ray K
>
>
>
> Ray K
>
>
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH] devtools: remove useless files from ABI reference
2020-04-21 9:26 4% ` David Marchand
@ 2020-04-21 10:33 4% ` Bruce Richardson
0 siblings, 0 replies; 200+ results
From: Bruce Richardson @ 2020-04-21 10:33 UTC (permalink / raw)
To: David Marchand; +Cc: Thomas Monjalon, dev
On Tue, Apr 21, 2020 at 11:26:58AM +0200, David Marchand wrote:
> On Tue, Apr 21, 2020 at 11:22 AM Thomas Monjalon <thomas@monjalon.net> wrote:
> >
> > 21/04/2020 09:49, David Marchand:
> > > On Tue, Apr 21, 2020 at 4:05 AM Thomas Monjalon <thomas@monjalon.net> wrote:
> > > > + rm $abirefdir/$targetdir/usr/local/lib/*.a
> > >
> > > The install directory for libraries is not lib/ in all cases.
> > > This breaks the ABI generation for gcc-shared:
> > >
> > > DESTDIR=/home/dmarchan/abi/v20.02/build-gcc-shared ninja -C
> > > /home/dmarchan/abi/v20.02/build install >/dev/null
> > > rm: cannot remove
> > > '/home/dmarchan/abi/v20.02/build-gcc-shared/usr/local/lib/*.a': No
> > > such file or directory
> > >
> > > $ ls ~/abi/v20.02/build-gcc-shared/usr/local/
> > > bin include lib64 share
> >
> > Is it enough to change lib to lib* ?
>
> I can see only lib/ or lib64/ so ok for me.
>
If it's using lib, then the .a files could be in /lib/x86_64-linux-gnu/,
rather than in lib directly. Therefore I think it's better to use find or
similar to remove the .a files from /usr/local/:
find /usr/local -name 'librte*.a' -exec rm -f {}
^ permalink raw reply [relevance 4%]
* [dpdk-dev] [PATCH v4] ethdev: support flow aging
@ 2020-04-21 10:11 2% ` Bill Zhou
2020-04-21 17:13 0% ` Ferruh Yigit
2020-04-29 14:50 0% ` Tom Barbette
0 siblings, 2 replies; 200+ results
From: Bill Zhou @ 2020-04-21 10:11 UTC (permalink / raw)
To: orika, matan, wenzhuo.lu, jingjing.wu, bernard.iremonger,
john.mcnamara, marko.kovacevic, thomas, ferruh.yigit, arybchenko
Cc: dev, Dong Zhou
From: Dong Zhou <dongz@mellanox.com>
One of the reasons to destroy a flow is the fact that no packet matches the
flow for "timeout" time.
For example, when TCP\UDP sessions are suddenly closed.
Currently, there is not any DPDK mechanism for flow aging and the
applications use their own ways to detect and destroy aged-out flows.
The flow aging implementation need include:
- A new rte_flow action: RTE_FLOW_ACTION_TYPE_AGE to set the timeout and
the application flow context for each flow.
- A new ethdev event: RTE_ETH_EVENT_FLOW_AGED for the driver to report
that there are new aged-out flows.
- A new rte_flow API: rte_flow_get_aged_flows to get the aged-out flows
contexts from the port.
- Support input flow aging command line in Testpmd.
The new event type addition in the enum is flagged as an ABI breakage, so
an ignore rule is added for these reasons:
- It is not changing value of existing types (except MAX)
- The new value is not used by existing API if the event is not registered
In general, it is safe adding new ethdev event types at the end of the
enum, because of event callback registration mechanism.
Signed-off-by: Dong Zhou <dongz@mellanox.com>
---
v2: Removing "* Added support for flow Aging mechanism base on counter."
this line from doc/guides/rel_notes/release_20_05.rst, this patch does not
include this support.
v3: Update file libabigail.abignore, add one new suppressed enumeration
type for RTE_ETH_EVENT_MAX.
v4: Add justification in devtools/libabigail.abignore and in the commit
log about the modification of v3.
---
app/test-pmd/cmdline_flow.c | 26 ++++++++++
devtools/libabigail.abignore | 6 +++
doc/guides/prog_guide/rte_flow.rst | 22 +++++++++
doc/guides/rel_notes/release_20_05.rst | 11 +++++
lib/librte_ethdev/rte_ethdev.h | 1 +
lib/librte_ethdev/rte_ethdev_version.map | 3 ++
lib/librte_ethdev/rte_flow.c | 18 +++++++
lib/librte_ethdev/rte_flow.h | 62 ++++++++++++++++++++++++
lib/librte_ethdev/rte_flow_driver.h | 6 +++
9 files changed, 155 insertions(+)
diff --git a/app/test-pmd/cmdline_flow.c b/app/test-pmd/cmdline_flow.c
index e6ab8ff2f7..45bcff3cf5 100644
--- a/app/test-pmd/cmdline_flow.c
+++ b/app/test-pmd/cmdline_flow.c
@@ -343,6 +343,8 @@ enum index {
ACTION_SET_IPV4_DSCP_VALUE,
ACTION_SET_IPV6_DSCP,
ACTION_SET_IPV6_DSCP_VALUE,
+ ACTION_AGE,
+ ACTION_AGE_TIMEOUT,
};
/** Maximum size for pattern in struct rte_flow_item_raw. */
@@ -1145,6 +1147,7 @@ static const enum index next_action[] = {
ACTION_SET_META,
ACTION_SET_IPV4_DSCP,
ACTION_SET_IPV6_DSCP,
+ ACTION_AGE,
ZERO,
};
@@ -1370,6 +1373,13 @@ static const enum index action_set_ipv6_dscp[] = {
ZERO,
};
+static const enum index action_age[] = {
+ ACTION_AGE,
+ ACTION_AGE_TIMEOUT,
+ ACTION_NEXT,
+ ZERO,
+};
+
static int parse_set_raw_encap_decap(struct context *, const struct token *,
const char *, unsigned int,
void *, unsigned int);
@@ -3694,6 +3704,22 @@ static const struct token token_list[] = {
(struct rte_flow_action_set_dscp, dscp)),
.call = parse_vc_conf,
},
+ [ACTION_AGE] = {
+ .name = "age",
+ .help = "set a specific metadata header",
+ .next = NEXT(action_age),
+ .priv = PRIV_ACTION(AGE,
+ sizeof(struct rte_flow_action_age)),
+ .call = parse_vc,
+ },
+ [ACTION_AGE_TIMEOUT] = {
+ .name = "timeout",
+ .help = "flow age timeout value",
+ .args = ARGS(ARGS_ENTRY_BF(struct rte_flow_action_age,
+ timeout, 24)),
+ .next = NEXT(action_age, NEXT_ENTRY(UNSIGNED)),
+ .call = parse_vc_conf,
+ },
};
/** Remove and return last entry from argument stack. */
diff --git a/devtools/libabigail.abignore b/devtools/libabigail.abignore
index a59df8f135..c047adbd79 100644
--- a/devtools/libabigail.abignore
+++ b/devtools/libabigail.abignore
@@ -11,3 +11,9 @@
type_kind = enum
name = rte_crypto_asym_xform_type
changed_enumerators = RTE_CRYPTO_ASYM_XFORM_TYPE_LIST_END
+; Ignore ethdev event enum update because new event cannot be
+; received if not registered
+[suppress_type]
+ type_kind = enum
+ name = rte_eth_event_type
+ changed_enumerators = RTE_ETH_EVENT_MAX
diff --git a/doc/guides/prog_guide/rte_flow.rst b/doc/guides/prog_guide/rte_flow.rst
index 41c147913c..cf4368e1c4 100644
--- a/doc/guides/prog_guide/rte_flow.rst
+++ b/doc/guides/prog_guide/rte_flow.rst
@@ -2616,6 +2616,28 @@ Otherwise, RTE_FLOW_ERROR_TYPE_ACTION error will be returned.
| ``dscp`` | DSCP in low 6 bits, rest ignore |
+-----------+---------------------------------+
+Action: ``AGE``
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Set ageing timeout configuration to a flow.
+
+Event RTE_ETH_EVENT_FLOW_AGED will be reported if
+timeout passed without any matching on the flow.
+
+.. _table_rte_flow_action_age:
+
+.. table:: AGE
+
+ +--------------+---------------------------------+
+ | Field | Value |
+ +==============+=================================+
+ | ``timeout`` | 24 bits timeout value |
+ +--------------+---------------------------------+
+ | ``reserved`` | 8 bits reserved, must be zero |
+ +--------------+---------------------------------+
+ | ``context`` | user input flow context |
+ +--------------+---------------------------------+
+
Negative types
~~~~~~~~~~~~~~
diff --git a/doc/guides/rel_notes/release_20_05.rst b/doc/guides/rel_notes/release_20_05.rst
index bacd4c65a2..ff0cf9f1d6 100644
--- a/doc/guides/rel_notes/release_20_05.rst
+++ b/doc/guides/rel_notes/release_20_05.rst
@@ -135,6 +135,17 @@ New Features
by making use of the event device capabilities. The event mode currently supports
only inline IPsec protocol offload.
+* **Added flow Aging Support.**
+
+ Added flow Aging support to detect and report aged-out flows, including:
+
+ * Added new action: RTE_FLOW_ACTION_TYPE_AGE to set the timeout and the
+ application flow context for each flow.
+ * Added new event: RTE_ETH_EVENT_FLOW_AGED for the driver to report that
+ there are new aged-out flows.
+ * Added new API: rte_flow_get_aged_flows to get the aged-out flows contexts
+ from the port.
+
Removed Items
-------------
diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h
index 8d69b88f9e..00cc7b4052 100644
--- a/lib/librte_ethdev/rte_ethdev.h
+++ b/lib/librte_ethdev/rte_ethdev.h
@@ -3018,6 +3018,7 @@ enum rte_eth_event_type {
RTE_ETH_EVENT_NEW, /**< port is probed */
RTE_ETH_EVENT_DESTROY, /**< port is released */
RTE_ETH_EVENT_IPSEC, /**< IPsec offload related event */
+ RTE_ETH_EVENT_FLOW_AGED,/**< New aged-out flows is detected */
RTE_ETH_EVENT_MAX /**< max value of this enum */
};
diff --git a/lib/librte_ethdev/rte_ethdev_version.map b/lib/librte_ethdev/rte_ethdev_version.map
index 3f32fdecf7..fa4b5816be 100644
--- a/lib/librte_ethdev/rte_ethdev_version.map
+++ b/lib/librte_ethdev/rte_ethdev_version.map
@@ -230,4 +230,7 @@ EXPERIMENTAL {
# added in 20.02
rte_flow_dev_dump;
+
+ # added in 20.05
+ rte_flow_get_aged_flows;
};
diff --git a/lib/librte_ethdev/rte_flow.c b/lib/librte_ethdev/rte_flow.c
index a5ac1c7fbd..3699edce49 100644
--- a/lib/librte_ethdev/rte_flow.c
+++ b/lib/librte_ethdev/rte_flow.c
@@ -172,6 +172,7 @@ static const struct rte_flow_desc_data rte_flow_desc_action[] = {
MK_FLOW_ACTION(SET_META, sizeof(struct rte_flow_action_set_meta)),
MK_FLOW_ACTION(SET_IPV4_DSCP, sizeof(struct rte_flow_action_set_dscp)),
MK_FLOW_ACTION(SET_IPV6_DSCP, sizeof(struct rte_flow_action_set_dscp)),
+ MK_FLOW_ACTION(AGE, sizeof(struct rte_flow_action_age)),
};
int
@@ -1232,3 +1233,20 @@ rte_flow_dev_dump(uint16_t port_id, FILE *file, struct rte_flow_error *error)
RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
NULL, rte_strerror(ENOSYS));
}
+
+int
+rte_flow_get_aged_flows(uint16_t port_id, void **contexts,
+ uint32_t nb_contexts, struct rte_flow_error *error)
+{
+ struct rte_eth_dev *dev = &rte_eth_devices[port_id];
+ const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error);
+
+ if (unlikely(!ops))
+ return -rte_errno;
+ if (likely(!!ops->get_aged_flows))
+ return flow_err(port_id, ops->get_aged_flows(dev, contexts,
+ nb_contexts, error), error);
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL, rte_strerror(ENOTSUP));
+}
diff --git a/lib/librte_ethdev/rte_flow.h b/lib/librte_ethdev/rte_flow.h
index 7f3e08fad3..fab44f6c0b 100644
--- a/lib/librte_ethdev/rte_flow.h
+++ b/lib/librte_ethdev/rte_flow.h
@@ -2081,6 +2081,16 @@ enum rte_flow_action_type {
* See struct rte_flow_action_set_dscp.
*/
RTE_FLOW_ACTION_TYPE_SET_IPV6_DSCP,
+
+ /**
+ * Report as aged flow if timeout passed without any matching on the
+ * flow.
+ *
+ * See struct rte_flow_action_age.
+ * See function rte_flow_get_aged_flows
+ * see enum RTE_ETH_EVENT_FLOW_AGED
+ */
+ RTE_FLOW_ACTION_TYPE_AGE,
};
/**
@@ -2122,6 +2132,25 @@ struct rte_flow_action_queue {
uint16_t index; /**< Queue index to use. */
};
+/**
+ * @warning
+ * @b EXPERIMENTAL: this structure may change without prior notice
+ *
+ * RTE_FLOW_ACTION_TYPE_AGE
+ *
+ * Report flow as aged-out if timeout passed without any matching
+ * on the flow. RTE_ETH_EVENT_FLOW_AGED event is triggered when a
+ * port detects new aged-out flows.
+ *
+ * The flow context and the flow handle will be reported by the
+ * rte_flow_get_aged_flows API.
+ */
+struct rte_flow_action_age {
+ uint32_t timeout:24; /**< Time in seconds. */
+ uint32_t reserved:8; /**< Reserved, must be zero. */
+ void *context;
+ /**< The user flow context, NULL means the rte_flow pointer. */
+};
/**
* @warning
@@ -3254,6 +3283,39 @@ rte_flow_conv(enum rte_flow_conv_op op,
const void *src,
struct rte_flow_error *error);
+/**
+ * Get aged-out flows of a given port.
+ *
+ * RTE_ETH_EVENT_FLOW_AGED event will be triggered when at least one new aged
+ * out flow was detected after the last call to rte_flow_get_aged_flows.
+ * This function can be called to get the aged flows usynchronously from the
+ * event callback or synchronously regardless the event.
+ * This is not safe to call rte_flow_get_aged_flows function with other flow
+ * functions from multiple threads simultaneously.
+ *
+ * @param port_id
+ * Port identifier of Ethernet device.
+ * @param[in, out] contexts
+ * The address of an array of pointers to the aged-out flows contexts.
+ * @param[in] nb_contexts
+ * The length of context array pointers.
+ * @param[out] error
+ * Perform verbose error reporting if not NULL. Initialized in case of
+ * error only.
+ *
+ * @return
+ * if nb_contexts is 0, return the amount of all aged contexts.
+ * if nb_contexts is not 0 , return the amount of aged flows reported
+ * in the context array, otherwise negative errno value.
+ *
+ * @see rte_flow_action_age
+ * @see RTE_ETH_EVENT_FLOW_AGED
+ */
+__rte_experimental
+int
+rte_flow_get_aged_flows(uint16_t port_id, void **contexts,
+ uint32_t nb_contexts, struct rte_flow_error *error);
+
#ifdef __cplusplus
}
#endif
diff --git a/lib/librte_ethdev/rte_flow_driver.h b/lib/librte_ethdev/rte_flow_driver.h
index 51a9a57a0f..881cc469b7 100644
--- a/lib/librte_ethdev/rte_flow_driver.h
+++ b/lib/librte_ethdev/rte_flow_driver.h
@@ -101,6 +101,12 @@ struct rte_flow_ops {
(struct rte_eth_dev *dev,
FILE *file,
struct rte_flow_error *error);
+ /** See rte_flow_get_aged_flows() */
+ int (*get_aged_flows)
+ (struct rte_eth_dev *dev,
+ void **context,
+ uint32_t nb_contexts,
+ struct rte_flow_error *err);
};
/**
--
2.21.0
^ permalink raw reply [relevance 2%]
* Re: [dpdk-dev] [PATCH v2] ethdev: support flow aging
2020-04-21 10:04 0% ` Ferruh Yigit
@ 2020-04-21 10:09 0% ` Thomas Monjalon
2020-04-21 15:59 0% ` Andrew Rybchenko
1 sibling, 0 replies; 200+ results
From: Thomas Monjalon @ 2020-04-21 10:09 UTC (permalink / raw)
To: Bill Zhou, Ferruh Yigit
Cc: Ori Kam, Matan Azrad, wenzhuo.lu, jingjing.wu, bernard.iremonger,
john.mcnamara, marko.kovacevic, arybchenko, dev
21/04/2020 12:04, Ferruh Yigit:
> On 4/20/2020 5:10 PM, Thomas Monjalon wrote:
> > 20/04/2020 16:06, Ferruh Yigit:
> >> On 4/18/2020 10:44 AM, Thomas Monjalon wrote:
> >>> 18/04/2020 07:04, Bill Zhou:
> >>>> From: Ferruh Yigit <ferruh.yigit@intel.com>
> >>>>> On 4/14/2020 9:32 AM, Dong Zhou wrote:
> >>>>>> --- a/lib/librte_ethdev/rte_ethdev.h
> >>>>>> +++ b/lib/librte_ethdev/rte_ethdev.h
> >>>>>> @@ -3015,6 +3015,7 @@ enum rte_eth_event_type {
> >>>>>> RTE_ETH_EVENT_NEW, /**< port is probed */
> >>>>>> RTE_ETH_EVENT_DESTROY, /**< port is released */
> >>>>>> RTE_ETH_EVENT_IPSEC, /**< IPsec offload related event */
> >>>>>> + RTE_ETH_EVENT_FLOW_AGED,/**< New aged-out flows is detected
> >>>>> */
> >>>>>> RTE_ETH_EVENT_MAX /**< max value of this enum */
> >>>>>> };
> >>>>>
> >>>>>
> >>>>> Just recognized that this is failing in ABI check [1], as far as last time for a
> >>>>> similar enum warning a QAT patch has been dropped, should this need to
> >>>>> wait for
> >>>>> 20.11 too?
> >>>>
> >>>> This patch is commonly used for flow aging, there are 2 other patches have
> >>>> implement flow aging in mlx5 driver reply to this patch.
> > [...]
> >>> These MAX values in enums are a pain.
> >>> We can try to think what can be done, waiting 20.11.
> >>> Not sure there is a solution, except hijacking an existing value
> >>> not used in the PMD, waiting the definitive value in 20.11...
> >>
> >> Dropping from the tree as of now, to not cause more merge conflicts, we can add
> >> it later when issue is resolved.
> >
> > Thanks for dropping, that's the right thing to do
> > when a patch is breaking ABI check.
> >
> > After some thoughts, I think it is acceptable to make a v3
> > which ignore this specific enum change. I explain my thought below:
> >
> > An enum can accept a new value at 2 conditions:
> > - added as last value (not changing old values)
> > - new value not used by existing API
> >
> > The value RTE_ETH_EVENT_FLOW_AGED meet the above 2 conditions:
> > - only RTE_ETH_EVENT_MAX is changed, which is consistent
> > - new value sent to the app only if the app registered for it
> >
>
> Same here, as far as I can see it is safe to get this change.
>
> If any DPDK API returns this enum, either as return of the API or as output
> parameter, this still can be problem, because application may use that returned
> value, this was the concern in the QAT sample.
>
> But here application registers an event and DPDK library process callback for
> it, so application callbacks won't be called for anything that application
> doesn't already know about, in that respect this should be safe for old
> applications.
>
> Not sure if we can generalize above two conditions for all enum changes, but we
> can investigate them case by case as we get the warnings.
>
> > So, except if I miss something, I suggest we add this exception:
> > Allow new value in rte_eth_event_type if added just before RTE_ETH_EVENT_MAX.
> > In other words, allow changing the value of RTE_ETH_EVENT_MAX.
> > The file to add such exception is devtools/libabigail.abignore.
> >
>
> OK to exception.
v3 was sent.
I hope we'll get a v4 with justification for the exception.
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH v2] ethdev: support flow aging
@ 2020-04-21 10:04 0% ` Ferruh Yigit
2020-04-21 10:09 0% ` Thomas Monjalon
2020-04-21 15:59 0% ` Andrew Rybchenko
0 siblings, 2 replies; 200+ results
From: Ferruh Yigit @ 2020-04-21 10:04 UTC (permalink / raw)
To: Thomas Monjalon, Bill Zhou
Cc: Ori Kam, Matan Azrad, wenzhuo.lu, jingjing.wu, bernard.iremonger,
john.mcnamara, marko.kovacevic, arybchenko, dev
On 4/20/2020 5:10 PM, Thomas Monjalon wrote:
> 20/04/2020 16:06, Ferruh Yigit:
>> On 4/18/2020 10:44 AM, Thomas Monjalon wrote:
>>> 18/04/2020 07:04, Bill Zhou:
>>>> From: Ferruh Yigit <ferruh.yigit@intel.com>
>>>>> On 4/14/2020 9:32 AM, Dong Zhou wrote:
>>>>>> --- a/lib/librte_ethdev/rte_ethdev.h
>>>>>> +++ b/lib/librte_ethdev/rte_ethdev.h
>>>>>> @@ -3015,6 +3015,7 @@ enum rte_eth_event_type {
>>>>>> RTE_ETH_EVENT_NEW, /**< port is probed */
>>>>>> RTE_ETH_EVENT_DESTROY, /**< port is released */
>>>>>> RTE_ETH_EVENT_IPSEC, /**< IPsec offload related event */
>>>>>> + RTE_ETH_EVENT_FLOW_AGED,/**< New aged-out flows is detected
>>>>> */
>>>>>> RTE_ETH_EVENT_MAX /**< max value of this enum */
>>>>>> };
>>>>>
>>>>>
>>>>> Just recognized that this is failing in ABI check [1], as far as last time for a
>>>>> similar enum warning a QAT patch has been dropped, should this need to
>>>>> wait for
>>>>> 20.11 too?
>>>>
>>>> This patch is commonly used for flow aging, there are 2 other patches have
>>>> implement flow aging in mlx5 driver reply to this patch.
> [...]
>>> These MAX values in enums are a pain.
>>> We can try to think what can be done, waiting 20.11.
>>> Not sure there is a solution, except hijacking an existing value
>>> not used in the PMD, waiting the definitive value in 20.11...
>>
>> Dropping from the tree as of now, to not cause more merge conflicts, we can add
>> it later when issue is resolved.
>
> Thanks for dropping, that's the right thing to do
> when a patch is breaking ABI check.
>
> After some thoughts, I think it is acceptable to make a v3
> which ignore this specific enum change. I explain my thought below:
>
> An enum can accept a new value at 2 conditions:
> - added as last value (not changing old values)
> - new value not used by existing API
>
> The value RTE_ETH_EVENT_FLOW_AGED meet the above 2 conditions:
> - only RTE_ETH_EVENT_MAX is changed, which is consistent
> - new value sent to the app only if the app registered for it
>
Same here, as far as I can see it is safe to get this change.
If any DPDK API returns this enum, either as return of the API or as output
parameter, this still can be problem, because application may use that returned
value, this was the concern in the QAT sample.
But here application registers an event and DPDK library process callback for
it, so application callbacks won't be called for anything that application
doesn't already know about, in that respect this should be safe for old
applications.
Not sure if we can generalize above two conditions for all enum changes, but we
can investigate them case by case as we get the warnings.
> So, except if I miss something, I suggest we add this exception:
> Allow new value in rte_eth_event_type if added just before RTE_ETH_EVENT_MAX.
> In other words, allow changing the value of RTE_ETH_EVENT_MAX.
> The file to add such exception is devtools/libabigail.abignore.
>
OK to exception.
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] rte_vfio_container_dma_map/unmap functions
@ 2020-04-21 9:50 0% ` Burakov, Anatoly
0 siblings, 0 replies; 200+ results
From: Burakov, Anatoly @ 2020-04-21 9:50 UTC (permalink / raw)
To: Thomas Monjalon; +Cc: dev, Tal Shnaiderman, bruce.richardson, david.marchand
On 20-Apr-20 6:39 PM, Thomas Monjalon wrote:
> 20/04/2020 16:07, Burakov, Anatoly:
>> On 19-Apr-20 2:10 PM, Thomas Monjalon wrote:
>>> 19/04/2020 15:09, Thomas Monjalon:
>>>> 17/04/2020 16:09, Burakov, Anatoly:
>>>>> On 17-Apr-20 3:05 PM, Burakov, Anatoly wrote:
>>>>>> On 22-Mar-20 5:20 PM, Tal Shnaiderman wrote:
>>>>>>> Hi Anatoly,
>>>>>>>
>>>>>>> I’m working on the implementation of bus/pci driver for Windows,
>>>>>>> pci_common.c uses the titled functions however they are relevant only
>>>>>>> for Linux OS.
>>>>>>>
>>>>>>> I’m wondering if the implementation of those functions should be moved
>>>>>>> to a Linux specific area since FreeBSD (and now Windows) are forced to
>>>>>>> implemented those in the current state.
>>>>>
>>>>> Unfortunately, we don't have a generic API for these, but since we
>>>>> export a single API on all platforms, either all platforms have to
>>>>> implement these functions, or none of them do. There's simply no way to
>>>>> avoid implementing stubs for these functions, short of coming up with a
>>>>> generic API that would replace these. Given that this API is heavily
>>>>> Linux specific, i don't see that happening.
>>>>
>>>> Because it is Linux specific, we should not force FreeBSD and Windows
>>>> having stubs. Can we move VFIO calls in Linux-specific files?
>>>>
>>>> I think rte_vfio.h should be moved in lib/librte_eal/linux/include.
>>>
>>> +Cc Bruce and David
>>
>> ...and have a Linux-specific ABI?
>
> Yes, the ABI is different depending on arch and OS.
> That's a fact, and I don't see any problem with it.
>
OK, no objections then :)
--
Thanks,
Anatoly
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH] cryptodev: version rte_cryptodev_info_get function
2020-04-21 6:01 3% ` Ray Kinsella
@ 2020-04-21 9:36 0% ` Thomas Monjalon
2020-04-22 8:21 0% ` Ray Kinsella
0 siblings, 1 reply; 200+ results
From: Thomas Monjalon @ 2020-04-21 9:36 UTC (permalink / raw)
To: Ray Kinsella
Cc: Trahe, Fiona, Richardson, Bruce, dev, Kusztal, ArkadiuszX,
Neil Horman, Luca Boccassi, Kevin Traynor, Yigit, Ferruh,
Akhil Goyal
21/04/2020 08:01, Ray Kinsella:
>
> On 20/04/2020 18:37, Thomas Monjalon wrote:
> > 20/04/2020 19:31, Ray Kinsella:
> >>
> >> Our only commitment is to the stability of the v19.11/v20 ABI, until v21.
> >>
> >> That said, once an ABI migrates from EXPERIMENTAL to v21, it _shouldn't_ be changing.
> >> We don't have a strict commitment to the v21 ABI until v20.11.
> >>
> >> However if v21 is changing across quarterlies (outside of additions) ... something else is wrong.
> >
> > The only way to check a symbol is not changing in a quarterly release
> > is to test it. That's what we wanted to enforce:
> > compare 20.02 ABI in 20.05 release.
> >
> > What other process do you suggest?
> >
>
> Well I guess it's understanding the reason why you are doing something.
> I can see reasons for wanting to test against both v19.11 and v20.02.
>
> v19.11 because our strict commitment is to the v20 abi.
> v20.02 to ensure that v21 symbols are not changing between quarterly releases.
>
> On v20, since you tested v20.02 against v19.11 during the v20.02 release cycle.
> The v20 symbols should not have changed during the v20.02 release cycle.
>
> I take your point, that then testing v20.05 against v20.02 would catch both v20 and v21 changes.
OK, so we need a policy or process update to make this conclusion clear to everybody.
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH] devtools: remove useless files from ABI reference
2020-04-21 9:22 4% ` Thomas Monjalon
@ 2020-04-21 9:26 4% ` David Marchand
2020-04-21 10:33 4% ` Bruce Richardson
0 siblings, 1 reply; 200+ results
From: David Marchand @ 2020-04-21 9:26 UTC (permalink / raw)
To: Thomas Monjalon; +Cc: dev, Bruce Richardson
On Tue, Apr 21, 2020 at 11:22 AM Thomas Monjalon <thomas@monjalon.net> wrote:
>
> 21/04/2020 09:49, David Marchand:
> > On Tue, Apr 21, 2020 at 4:05 AM Thomas Monjalon <thomas@monjalon.net> wrote:
> > > + rm $abirefdir/$targetdir/usr/local/lib/*.a
> >
> > The install directory for libraries is not lib/ in all cases.
> > This breaks the ABI generation for gcc-shared:
> >
> > DESTDIR=/home/dmarchan/abi/v20.02/build-gcc-shared ninja -C
> > /home/dmarchan/abi/v20.02/build install >/dev/null
> > rm: cannot remove
> > '/home/dmarchan/abi/v20.02/build-gcc-shared/usr/local/lib/*.a': No
> > such file or directory
> >
> > $ ls ~/abi/v20.02/build-gcc-shared/usr/local/
> > bin include lib64 share
>
> Is it enough to change lib to lib* ?
I can see only lib/ or lib64/ so ok for me.
--
David Marchand
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH] devtools: remove useless files from ABI reference
2020-04-21 7:49 9% ` David Marchand
@ 2020-04-21 9:22 4% ` Thomas Monjalon
2020-04-21 9:26 4% ` David Marchand
0 siblings, 1 reply; 200+ results
From: Thomas Monjalon @ 2020-04-21 9:22 UTC (permalink / raw)
To: David Marchand; +Cc: dev, Bruce Richardson
21/04/2020 09:49, David Marchand:
> On Tue, Apr 21, 2020 at 4:05 AM Thomas Monjalon <thomas@monjalon.net> wrote:
> > + rm $abirefdir/$targetdir/usr/local/lib/*.a
>
> The install directory for libraries is not lib/ in all cases.
> This breaks the ABI generation for gcc-shared:
>
> DESTDIR=/home/dmarchan/abi/v20.02/build-gcc-shared ninja -C
> /home/dmarchan/abi/v20.02/build install >/dev/null
> rm: cannot remove
> '/home/dmarchan/abi/v20.02/build-gcc-shared/usr/local/lib/*.a': No
> such file or directory
>
> $ ls ~/abi/v20.02/build-gcc-shared/usr/local/
> bin include lib64 share
Is it enough to change lib to lib* ?
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH] devtools: skip ABI check in static builds
2020-04-21 8:02 4% ` David Marchand
@ 2020-04-21 9:20 4% ` Thomas Monjalon
0 siblings, 0 replies; 200+ results
From: Thomas Monjalon @ 2020-04-21 9:20 UTC (permalink / raw)
To: dev; +Cc: Ray Kinsella, David Marchand
21/04/2020 10:02, David Marchand:
> On Tue, Apr 21, 2020 at 3:33 AM Thomas Monjalon <thomas@monjalon.net> wrote:
> >
> > When running make with CONFIG_RTE_BUILD_SHARED_LIB=n,
> > no shared library is built.
> > In this case, no need to run ABI check.
> >
> > With meson, both shared and static libraries are always built.
> >
> > Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
>
> Acked-by: David Marchand <david.marchand@redhat.com>
Reviewed-by: Ray Kinsella <mdr@ashroe.eu>
Applied
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH] devtools: skip ABI check in static builds
2020-04-21 6:11 4% ` Ray Kinsella
@ 2020-04-21 9:15 4% ` Thomas Monjalon
0 siblings, 0 replies; 200+ results
From: Thomas Monjalon @ 2020-04-21 9:15 UTC (permalink / raw)
To: Ray Kinsella; +Cc: dev, david.marchand
21/04/2020 08:11, Ray Kinsella:
>
> On 21/04/2020 02:33, Thomas Monjalon wrote:
> > When running make with CONFIG_RTE_BUILD_SHARED_LIB=n,
> > no shared library is built.
> > In this case, no need to run ABI check.
> >
> > With meson, both shared and static libraries are always built.
>
> You can easily do the same thing for meson, by checking the 3rd parameter to build() in test-meson-build.sh,
No, both library types are always compiled.
The only option is to choose which library to link in built-in apps.
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH v8 0/2] support for VFIO-PCI VF token interface
2020-04-21 2:52 4% ` Wang, Haiyue
@ 2020-04-21 8:47 0% ` Thomas Monjalon
2020-04-21 17:35 0% ` Wang, Haiyue
0 siblings, 1 reply; 200+ results
From: Thomas Monjalon @ 2020-04-21 8:47 UTC (permalink / raw)
To: David Marchand, Wang, Haiyue
Cc: Neil Horman, dev, Burakov, Anatoly, Vamsi Attunuru,
Jerin Jacob Kollanukkaran, Yigit, Ferruh, Richardson, Bruce,
Kinsella, Ray
21/04/2020 04:52, Wang, Haiyue:
> From: Thomas Monjalon <thomas@monjalon.net>
> > 21/04/2020 03:38, Wang, Haiyue:
> > > From: Thomas Monjalon <thomas@monjalon.net>
> > > > 20/04/2020 19:37, Wang, Haiyue:
> > > > > From: Thomas Monjalon <thomas@monjalon.net>
> > > > > > 20/04/2020 19:02, Wang, Haiyue:
> > > > > > > From: David Marchand <david.marchand@redhat.com>
> > > > > > > > I had a look at the CI, I can see we are still missing bits to handle
> > > > > > > > the ABI failure on rte_vfio_setup_device.
> > > > > > >
> > > > > > > Yes, not handle it now.
> > > > > > >
> > > > > > > If 'rte_vfio_setup_device' can be internal, not official DPDK API, then __rte_internal
> > > > > > > is the best way to handle ABI issue.
> > > > > >
> > > > > > Please could you help finishing integration of __rte_internal?
> > > > >
> > > > > I thought it should be Neil ? "Yes, I'll get back to this today" ;-)
> > > > > http://inbox.dpdk.org/dev/CAJFAV8ydLkV0afEHqbh6KeA3Box0Yxb3N0MNGtMD4S9bmSgT0A@mail.gmail.com/
> > > >
> > > > It did not happen after several months.
> > > > If you want to unblock your patches, I think it is safer to finish yourself.
> > > >
> > >
> > > Unfortunately, this __rte_internal will still make the ci fail to run when move the function
> > > to INTERNAL session:
> > [...]
> > > +INTERNAL {
> > > + global:
> > > +
> > > + # added in 20.05
> > > + rte_vfio_setup_device;
> > > +};
> >
> > Why do you need to move the symbol explicitly in .map?
> > The tool should ignore symbols moving to internal, as an exception until 20.11.
>
> If not move the symbol explicitly in .map, another kind of error happened.
>
> ./devtools/check-abi.sh old_abi new_abi
> Functions changes summary: 0 Removed, 1 Changed, 0 Added function
> Variables changes summary: 0 Removed, 0 Changed, 0 Added variable
>
> 1 function with some indirect sub-type change:
>
> [C] 'function int rte_vfio_setup_device(const char*, const char*, int*, vfio_device_info*)' at eal_vfio.c:704:1 has some indirect sub-type changes:
> parameter 5 of type 'unsigned char*' was added
>
> Error: ABI issue reported for 'abidiff --suppr ./devtools/libabigail.abignore --no-added-syms --headers-dir1 old_abi/include --headers-dir2 new_abi/include old_abi/dump/librte_eal.dump new_abi/dump/librte_eal.dump'
This is what I said: you need to add a rule to ignore internal symbols.
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH] devtools: skip ABI check in static builds
2020-04-21 6:11 4% ` Ray Kinsella
@ 2020-04-21 8:02 4% ` David Marchand
2020-04-21 9:20 4% ` Thomas Monjalon
1 sibling, 1 reply; 200+ results
From: David Marchand @ 2020-04-21 8:02 UTC (permalink / raw)
To: Thomas Monjalon; +Cc: dev, Ray Kinsella
On Tue, Apr 21, 2020 at 3:33 AM Thomas Monjalon <thomas@monjalon.net> wrote:
>
> When running make with CONFIG_RTE_BUILD_SHARED_LIB=n,
> no shared library is built.
> In this case, no need to run ABI check.
>
> With meson, both shared and static libraries are always built.
>
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: David Marchand <david.marchand@redhat.com>
--
David Marchand
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH] devtools: remove useless files from ABI reference
2020-04-21 2:04 13% [dpdk-dev] [PATCH] devtools: remove useless files from ABI reference Thomas Monjalon
@ 2020-04-21 7:49 9% ` David Marchand
2020-04-21 9:22 4% ` Thomas Monjalon
0 siblings, 1 reply; 200+ results
From: David Marchand @ 2020-04-21 7:49 UTC (permalink / raw)
To: Thomas Monjalon; +Cc: dev, Bruce Richardson
On Tue, Apr 21, 2020 at 4:05 AM Thomas Monjalon <thomas@monjalon.net> wrote:
>
> When building an ABI reference with meson, some static libraries
> are built and linked in apps. They are useless and take a lot of space.
> Those binaries, and other useless files (examples and doc files)
> in the share/ directory, are removed after being installed.
>
> In order to save time when building the ABI reference,
> the examples (which are not installed anyway) are not compiled.
>
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> ---
> devtools/test-meson-builds.sh | 7 ++++++-
> 1 file changed, 6 insertions(+), 1 deletion(-)
>
> diff --git a/devtools/test-meson-builds.sh b/devtools/test-meson-builds.sh
> index e8df017596..16271e057d 100755
> --- a/devtools/test-meson-builds.sh
> +++ b/devtools/test-meson-builds.sh
> @@ -141,10 +141,15 @@ build () # <directory> <target compiler> <meson options>
> fi
>
> rm -rf $abirefdir/build
> - config $abirefdir/src $abirefdir/build $*
> + config $abirefdir/src $abirefdir/build -Dexamples= $*
> compile $abirefdir/build
> install_target $abirefdir/build $abirefdir/$targetdir
> $srcdir/devtools/gen-abi.sh $abirefdir/$targetdir
> +
> + # save disk space by removing static libs and apps
> + rm $abirefdir/$targetdir/usr/local/lib/*.a
The install directory for libraries is not lib/ in all cases.
This breaks the ABI generation for gcc-shared:
DESTDIR=/home/dmarchan/abi/v20.02/build-gcc-shared ninja -C
/home/dmarchan/abi/v20.02/build install >/dev/null
rm: cannot remove
'/home/dmarchan/abi/v20.02/build-gcc-shared/usr/local/lib/*.a': No
such file or directory
$ ls ~/abi/v20.02/build-gcc-shared/usr/local/
bin include lib64 share
--
David Marchand
^ permalink raw reply [relevance 9%]
* Re: [dpdk-dev] [PATCH] devtools: skip ABI check in static builds
@ 2020-04-21 6:11 4% ` Ray Kinsella
2020-04-21 9:15 4% ` Thomas Monjalon
2020-04-21 8:02 4% ` David Marchand
1 sibling, 1 reply; 200+ results
From: Ray Kinsella @ 2020-04-21 6:11 UTC (permalink / raw)
To: Thomas Monjalon, dev; +Cc: david.marchand
On 21/04/2020 02:33, Thomas Monjalon wrote:
> When running make with CONFIG_RTE_BUILD_SHARED_LIB=n,
> no shared library is built.
> In this case, no need to run ABI check.
>
> With meson, both shared and static libraries are always built.
You can easily do the same thing for meson, by checking the 3rd parameter to build() in test-meson-build.sh,
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> ---
> devtools/test-build.sh | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/devtools/test-build.sh b/devtools/test-build.sh
> index a298115002..6e53f86fc8 100755
> --- a/devtools/test-build.sh
> +++ b/devtools/test-build.sh
> @@ -262,6 +262,7 @@ for conf in $configs ; do
> EXTRA_LDFLAGS="$DPDK_DEP_LDFLAGS" $verbose \
> O=$(readlink -f $dir)/examples
> unset RTE_TARGET
> + grep -q 'SHARED_LIB=n' $dir/.config || # skip ABI check with static libs
> if [ -n "$DPDK_ABI_REF_VERSION" ]; then
> abirefdir=${DPDK_ABI_REF_DIR:-reference}/$DPDK_ABI_REF_VERSION
> if [ ! -d $abirefdir/$conf ]; then
>
Reviewed-by: Ray Kinsella <mdr@ashroe.eu>
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH] cryptodev: version rte_cryptodev_info_get function
@ 2020-04-21 6:01 3% ` Ray Kinsella
2020-04-21 9:36 0% ` Thomas Monjalon
0 siblings, 1 reply; 200+ results
From: Ray Kinsella @ 2020-04-21 6:01 UTC (permalink / raw)
To: Thomas Monjalon, Trahe, Fiona, Richardson, Bruce
Cc: dev, Kusztal, ArkadiuszX, Neil Horman, Luca Boccassi,
Kevin Traynor, Yigit, Ferruh, Akhil Goyal
On 20/04/2020 18:37, Thomas Monjalon wrote:
> 20/04/2020 19:31, Ray Kinsella:
>>
>> Our only commitment is to the stability of the v19.11/v20 ABI, until v21.
>>
>> That said, once an ABI migrates from EXPERIMENTAL to v21, it _shouldn't_ be changing.
>> We don't have a strict commitment to the v21 ABI until v20.11.
>>
>> However if v21 is changing across quarterlies (outside of additions) ... something else is wrong.
>
> The only way to check a symbol is not changing in a quarterly release
> is to test it. That's what we wanted to enforce:
> compare 20.02 ABI in 20.05 release.
>
> What other process do you suggest?
>
Well I guess it's understanding the reason why you are doing something.
I can see reasons for wanting to test against both v19.11 and v20.02.
v19.11 because our strict commitment is to the v20 abi.
v20.02 to ensure that v21 symbols are not changing between quarterly releases.
On v20, since you tested v20.02 against v19.11 during the v20.02 release cycle.
The v20 symbols should not have changed during the v20.02 release cycle.
I take your point, that then testing v20.05 against v20.02 would catch both v20 and v21 changes.
Thanks,
Ray K
^ permalink raw reply [relevance 3%]
* Re: [dpdk-dev] [PATCH v8 0/2] support for VFIO-PCI VF token interface
2020-04-21 2:12 0% ` Thomas Monjalon
@ 2020-04-21 2:52 4% ` Wang, Haiyue
2020-04-21 8:47 0% ` Thomas Monjalon
0 siblings, 1 reply; 200+ results
From: Wang, Haiyue @ 2020-04-21 2:52 UTC (permalink / raw)
To: Thomas Monjalon, David Marchand
Cc: Neil Horman, dev, Burakov, Anatoly, Vamsi Attunuru,
Jerin Jacob Kollanukkaran, Yigit, Ferruh, Richardson, Bruce,
Kinsella, Ray
> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: Tuesday, April 21, 2020 10:13
> To: David Marchand <david.marchand@redhat.com>; Wang, Haiyue <haiyue.wang@intel.com>
> Cc: Neil Horman <nhorman@tuxdriver.com>; dev <dev@dpdk.org>; Burakov, Anatoly
> <anatoly.burakov@intel.com>; Vamsi Attunuru <vattunuru@marvell.com>; Jerin Jacob Kollanukkaran
> <jerinj@marvell.com>; Yigit, Ferruh <ferruh.yigit@intel.com>; Richardson, Bruce
> <bruce.richardson@intel.com>; Kinsella, Ray <ray.kinsella@intel.com>
> Subject: Re: [PATCH v8 0/2] support for VFIO-PCI VF token interface
>
> 21/04/2020 03:38, Wang, Haiyue:
> > From: Thomas Monjalon <thomas@monjalon.net>
> > > 20/04/2020 19:37, Wang, Haiyue:
> > > > From: Thomas Monjalon <thomas@monjalon.net>
> > > > > 20/04/2020 19:02, Wang, Haiyue:
> > > > > > From: David Marchand <david.marchand@redhat.com>
> > > > > > > I had a look at the CI, I can see we are still missing bits to handle
> > > > > > > the ABI failure on rte_vfio_setup_device.
> > > > > >
> > > > > > Yes, not handle it now.
> > > > > >
> > > > > > If 'rte_vfio_setup_device' can be internal, not official DPDK API, then __rte_internal
> > > > > > is the best way to handle ABI issue.
> > > > >
> > > > > Please could you help finishing integration of __rte_internal?
> > > >
> > > > I thought it should be Neil ? "Yes, I'll get back to this today" ;-)
> > > > http://inbox.dpdk.org/dev/CAJFAV8ydLkV0afEHqbh6KeA3Box0Yxb3N0MNGtMD4S9bmSgT0A@mail.gmail.com/
> > >
> > > It did not happen after several months.
> > > If you want to unblock your patches, I think it is safer to finish yourself.
> > >
> >
> > Unfortunately, this __rte_internal will still make the ci fail to run when move the function
> > to INTERNAL session:
> [...]
> > +INTERNAL {
> > + global:
> > +
> > + # added in 20.05
> > + rte_vfio_setup_device;
> > +};
>
> Why do you need to move the symbol explicitly in .map?
> The tool should ignore symbols moving to internal, as an exception until 20.11.
>
If not move the symbol explicitly in .map, another kind of error happened.
./devtools/check-abi.sh old_abi new_abi
Functions changes summary: 0 Removed, 1 Changed, 0 Added function
Variables changes summary: 0 Removed, 0 Changed, 0 Added variable
1 function with some indirect sub-type change:
[C] 'function int rte_vfio_setup_device(const char*, const char*, int*, vfio_device_info*)' at eal_vfio.c:704:1 has some indirect sub-type changes:
parameter 5 of type 'unsigned char*' was added
Error: ABI issue reported for 'abidiff --suppr ./devtools/libabigail.abignore --no-added-syms --headers-dir1 old_abi/include --headers-dir2 new_abi/include old_abi/dump/librte_eal.dump new_abi/dump/librte_eal.dump'
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH v8 0/2] support for VFIO-PCI VF token interface
2020-04-21 1:38 3% ` Wang, Haiyue
@ 2020-04-21 2:12 0% ` Thomas Monjalon
2020-04-21 2:52 4% ` Wang, Haiyue
0 siblings, 1 reply; 200+ results
From: Thomas Monjalon @ 2020-04-21 2:12 UTC (permalink / raw)
To: David Marchand, Wang, Haiyue
Cc: Neil Horman, dev, Burakov, Anatoly, Vamsi Attunuru,
Jerin Jacob Kollanukkaran, Yigit, Ferruh, Richardson, Bruce,
Kinsella, Ray
21/04/2020 03:38, Wang, Haiyue:
> From: Thomas Monjalon <thomas@monjalon.net>
> > 20/04/2020 19:37, Wang, Haiyue:
> > > From: Thomas Monjalon <thomas@monjalon.net>
> > > > 20/04/2020 19:02, Wang, Haiyue:
> > > > > From: David Marchand <david.marchand@redhat.com>
> > > > > > I had a look at the CI, I can see we are still missing bits to handle
> > > > > > the ABI failure on rte_vfio_setup_device.
> > > > >
> > > > > Yes, not handle it now.
> > > > >
> > > > > If 'rte_vfio_setup_device' can be internal, not official DPDK API, then __rte_internal
> > > > > is the best way to handle ABI issue.
> > > >
> > > > Please could you help finishing integration of __rte_internal?
> > >
> > > I thought it should be Neil ? "Yes, I'll get back to this today" ;-)
> > > http://inbox.dpdk.org/dev/CAJFAV8ydLkV0afEHqbh6KeA3Box0Yxb3N0MNGtMD4S9bmSgT0A@mail.gmail.com/
> >
> > It did not happen after several months.
> > If you want to unblock your patches, I think it is safer to finish yourself.
> >
>
> Unfortunately, this __rte_internal will still make the ci fail to run when move the function
> to INTERNAL session:
[...]
> +INTERNAL {
> + global:
> +
> + # added in 20.05
> + rte_vfio_setup_device;
> +};
Why do you need to move the symbol explicitly in .map?
The tool should ignore symbols moving to internal, as an exception until 20.11.
^ permalink raw reply [relevance 0%]
* [dpdk-dev] [PATCH] devtools: remove useless files from ABI reference
@ 2020-04-21 2:04 13% Thomas Monjalon
2020-04-21 7:49 9% ` David Marchand
0 siblings, 1 reply; 200+ results
From: Thomas Monjalon @ 2020-04-21 2:04 UTC (permalink / raw)
To: dev; +Cc: bruce.richardson, david.marchand
When building an ABI reference with meson, some static libraries
are built and linked in apps. They are useless and take a lot of space.
Those binaries, and other useless files (examples and doc files)
in the share/ directory, are removed after being installed.
In order to save time when building the ABI reference,
the examples (which are not installed anyway) are not compiled.
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
devtools/test-meson-builds.sh | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/devtools/test-meson-builds.sh b/devtools/test-meson-builds.sh
index e8df017596..16271e057d 100755
--- a/devtools/test-meson-builds.sh
+++ b/devtools/test-meson-builds.sh
@@ -141,10 +141,15 @@ build () # <directory> <target compiler> <meson options>
fi
rm -rf $abirefdir/build
- config $abirefdir/src $abirefdir/build $*
+ config $abirefdir/src $abirefdir/build -Dexamples= $*
compile $abirefdir/build
install_target $abirefdir/build $abirefdir/$targetdir
$srcdir/devtools/gen-abi.sh $abirefdir/$targetdir
+
+ # save disk space by removing static libs and apps
+ rm $abirefdir/$targetdir/usr/local/lib/*.a
+ rm -rf $abirefdir/$targetdir/usr/local/bin
+ rm -rf $abirefdir/$targetdir/usr/local/share
fi
install_target $builds_dir/$targetdir \
--
2.26.0
^ permalink raw reply [relevance 13%]
* Re: [dpdk-dev] [PATCH v8 0/2] support for VFIO-PCI VF token interface
@ 2020-04-21 1:38 3% ` Wang, Haiyue
2020-04-21 2:12 0% ` Thomas Monjalon
0 siblings, 1 reply; 200+ results
From: Wang, Haiyue @ 2020-04-21 1:38 UTC (permalink / raw)
To: Thomas Monjalon, David Marchand
Cc: Neil Horman, dev, Burakov, Anatoly, Vamsi Attunuru,
Jerin Jacob Kollanukkaran, Yigit, Ferruh, Richardson, Bruce,
Kinsella, Ray
Hi Thomas & David,
> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: Tuesday, April 21, 2020 01:43
> To: Wang, Haiyue <haiyue.wang@intel.com>
> Cc: Neil Horman <nhorman@tuxdriver.com>; David Marchand <david.marchand@redhat.com>; dev
> <dev@dpdk.org>; Burakov, Anatoly <anatoly.burakov@intel.com>; Vamsi Attunuru <vattunuru@marvell.com>;
> Jerin Jacob Kollanukkaran <jerinj@marvell.com>; Alex Williamson <alex.williamson@redhat.com>
> Subject: Re: [PATCH v8 0/2] support for VFIO-PCI VF token interface
>
> 20/04/2020 19:37, Wang, Haiyue:
> > From: Thomas Monjalon <thomas@monjalon.net>
> > > 20/04/2020 19:02, Wang, Haiyue:
> > > > From: David Marchand <david.marchand@redhat.com>
> > > > > I had a look at the CI, I can see we are still missing bits to handle
> > > > > the ABI failure on rte_vfio_setup_device.
> > > >
> > > > Yes, not handle it now.
> > > >
> > > > If 'rte_vfio_setup_device' can be internal, not official DPDK API, then __rte_internal
> > > > is the best way to handle ABI issue.
> > >
> > > Please could you help finishing integration of __rte_internal?
> >
> > I thought it should be Neil ? "Yes, I'll get back to this today" ;-)
> > http://inbox.dpdk.org/dev/CAJFAV8ydLkV0afEHqbh6KeA3Box0Yxb3N0MNGtMD4S9bmSgT0A@mail.gmail.com/
>
> It did not happen after several months.
> If you want to unblock your patches, I think it is safer to finish yourself.
>
Unfortunately, this __rte_internal will still make the ci fail to run when move the function
to INTERNAL session:
--- a/devtools/libabigail.abignore
+++ b/devtools/libabigail.abignore
@@ -3,6 +3,11 @@
[suppress_variable]
symbol_version = EXPERIMENTAL
+[suppress_function]
+ symbol_version = INTERNAL
+[suppress_variable]
+ symbol_version = INTERNAL
+
--- a/lib/librte_eal/rte_eal_version.map
+++ b/lib/librte_eal/rte_eal_version.map
@@ -213,7 +213,6 @@ DPDK_20.0 {
rte_vfio_is_enabled;
rte_vfio_noiommu_is_enabled;
rte_vfio_release_device;
- rte_vfio_setup_device;
rte_vlog;
rte_zmalloc;
rte_zmalloc_socket;
@@ -339,3 +338,10 @@ EXPERIMENTAL {
# added in 20.05
rte_log_can_log;
};
+
+INTERNAL {
+ global:
+
+ # added in 20.05
+ rte_vfio_setup_device;
+};
Functions changes summary: 1 Removed, 0 Changed, 0 Added function
Variables changes summary: 0 Removed, 0 Changed, 0 Added variable
1 Removed function:
[D] 'function int rte_vfio_setup_device(const char*, const char*, int*, vfio_device_info*)' {rte_vfio_setup_device@@DPDK_20.0}
Error: ABI issue reported for 'abidiff --suppr ./devtools/libabigail.abignore --no-added-syms --headers-dir1 old_abi/include --headers-dir2 new_abi/include old_abi/dump/librte_eal.dump new_abi/dump/librte_eal.dump'
^ permalink raw reply [relevance 3%]
Results 6001-6200 of ~18000 next (older) | prev (newer) | reverse | sort options + mbox downloads above
-- links below jump to the message on this page --
2019-06-13 14:23 [dpdk-dev] [PATCH v2 0/10] dpdk: introduce __rte_internal tag Neil Horman
2020-04-22 13:52 4% ` [dpdk-dev] [PATCH v3 0/1] " Haiyue Wang
2020-04-22 13:52 11% ` [dpdk-dev] [PATCH v3 1/1] eal: add internal ABI mark support Haiyue Wang
2020-04-22 14:13 4% ` David Marchand
2020-04-22 16:44 4% ` Wang, Haiyue
2020-04-22 16:37 4% ` [dpdk-dev] [PATCH v4 0/1] dpdk: introduce __rte_internal tag Haiyue Wang
2020-04-22 16:37 15% ` [dpdk-dev] [PATCH v4 1/1] eal: add internal ABI mark support Haiyue Wang
2020-04-23 3:19 4% ` [dpdk-dev] [PATCH v5 0/1] dpdk: introduce __rte_internal tag Haiyue Wang
2020-04-23 3:19 15% ` [dpdk-dev] [PATCH v5 1/1] eal: add internal ABI marking support Haiyue Wang
2020-04-24 14:52 7% ` David Marchand
2020-04-25 6:10 7% ` Wang, Haiyue
2020-04-25 14:21 4% ` David Marchand
2020-04-25 14:24 7% ` Thomas Monjalon
2020-04-25 6:04 5% ` [dpdk-dev] [PATCH v6 0/6] dpdk: introduce __rte_internal tag Haiyue Wang
2020-04-25 6:04 13% ` [dpdk-dev] [PATCH v6 1/6] eal: add internal ABI tag definition Haiyue Wang
2020-04-25 6:04 3% ` [dpdk-dev] [PATCH v6 2/6] build: enable internal API tag Haiyue Wang
2020-04-25 6:04 13% ` [dpdk-dev] [PATCH v6 4/6] devtools: ignore internal ABI check Haiyue Wang
2020-04-25 6:04 7% ` [dpdk-dev] [PATCH v6 5/6] devtools: exempt internal ABI checking Haiyue Wang
2020-04-25 10:56 5% ` [dpdk-dev] [PATCH v7 0/6] dpdk: introduce __rte_internal tag Haiyue Wang
2020-04-25 10:56 13% ` [dpdk-dev] [PATCH v7 1/6] eal: add internal ABI tag definition Haiyue Wang
2020-04-25 10:56 3% ` [dpdk-dev] [PATCH v7 2/6] build: enable internal API tag Haiyue Wang
2020-04-25 10:56 5% ` [dpdk-dev] [PATCH v7 3/6] mk: add internal tag check Haiyue Wang
2020-04-25 14:34 3% ` David Marchand
2020-04-25 10:56 13% ` [dpdk-dev] [PATCH v7 4/6] devtools: ignore internal ABI check Haiyue Wang
2020-04-25 10:56 7% ` [dpdk-dev] [PATCH v7 5/6] devtools: exempt internal ABI checking Haiyue Wang
2020-04-25 14:34 4% ` David Marchand
2020-04-25 14:39 0% ` [dpdk-dev] [PATCH v7 0/6] dpdk: introduce __rte_internal tag David Marchand
2020-04-25 16:34 0% ` Wang, Haiyue
2020-04-25 18:09 0% ` Wang, Haiyue
2020-04-29 8:22 3% ` David Marchand
2020-04-29 8:24 0% ` Wang, Haiyue
2020-01-04 1:33 [dpdk-dev] [PATCH 00/14] cleanup resources on shutdown Stephen Hemminger
2020-04-25 19:34 0% ` David Marchand
2020-04-28 23:14 3% ` [dpdk-dev] [PATCH v2 0/9] eal: " Stephen Hemminger
2020-04-28 23:58 3% ` [dpdk-dev] [PATCH v3 0/8] " Stephen Hemminger
2020-05-03 17:21 0% ` David Marchand
2020-01-17 14:57 [dpdk-dev] [PATCH v2 3/3] [20.05] mempool: return 0 if area is too small on populate Olivier Matz
2020-04-25 22:23 12% ` [dpdk-dev] [PATCH v3] " Thomas Monjalon
2020-04-26 16:52 12% ` [dpdk-dev] [PATCH v4] " Thomas Monjalon
2020-05-04 12:49 11% ` [dpdk-dev] [PATCH v5] " Olivier Matz
2020-05-04 12:54 0% ` Andrew Rybchenko
2020-05-04 15:47 0% ` Lukasz Wojciechowski
2020-05-04 22:30 0% ` Thomas Monjalon
2020-04-27 11:44 0% ` [dpdk-dev] [PATCH v3] " Ray Kinsella
2020-04-27 18:02 0% ` Lukasz Wojciechowski
2020-02-12 23:08 [dpdk-dev] [RFC 0/4] Enforce flag checking in API's Stephen Hemminger
2020-04-24 17:15 3% ` [dpdk-dev] [PATCH v2 0/4] enforce checking on flag values " Stephen Hemminger
2020-04-24 17:15 3% ` [dpdk-dev] [PATCH v2 1/4] ring: future proof flag settings Stephen Hemminger
2020-04-24 18:07 0% ` Honnappa Nagarahalli
2020-04-24 19:02 0% ` Stephen Hemminger
2020-04-25 9:20 0% ` Ananyev, Konstantin
2020-04-24 17:15 3% ` [dpdk-dev] [PATCH v2 2/4] hash: check flags on creation Stephen Hemminger
2020-04-24 17:15 3% ` [dpdk-dev] [PATCH v2 3/4] stack: " Stephen Hemminger
2020-04-24 17:15 3% ` [dpdk-dev] [PATCH v2 4/4] cfgfile: check flags value Stephen Hemminger
2020-04-27 23:16 3% ` [dpdk-dev] [PATCH v3 0/4] Enforce checking on flag values in API's Stephen Hemminger
2020-04-27 23:16 3% ` [dpdk-dev] [PATCH v3 1/4] ring: future proof flag settings Stephen Hemminger
2020-04-28 4:12 0% ` Honnappa Nagarahalli
2020-04-27 23:16 3% ` [dpdk-dev] [PATCH v3 2/4] hash: check flags on creation Stephen Hemminger
2020-04-29 1:08 0% ` Wang, Yipeng1
2020-04-27 23:16 3% ` [dpdk-dev] [PATCH v3 3/4] stack: " Stephen Hemminger
2020-05-02 2:36 0% ` Eads, Gage
2020-04-27 23:16 3% ` [dpdk-dev] [PATCH v3 4/4] cfgfile: check flags value Stephen Hemminger
2020-04-28 10:24 0% ` Bruce Richardson
2020-04-28 10:28 0% ` [dpdk-dev] [PATCH v3 0/4] Enforce checking on flag values in API's Bruce Richardson
2020-04-28 11:04 0% ` Ananyev, Konstantin
2020-03-05 4:33 [dpdk-dev] [RFC v1 1/1] vfio: set vf token and gain vf device access vattunuru
2020-04-20 17:13 ` [dpdk-dev] [PATCH v8 0/2] support for VFIO-PCI VF token interface Thomas Monjalon
2020-04-20 17:37 ` Wang, Haiyue
2020-04-20 17:42 ` Thomas Monjalon
2020-04-21 1:38 3% ` Wang, Haiyue
2020-04-21 2:12 0% ` Thomas Monjalon
2020-04-21 2:52 4% ` Wang, Haiyue
2020-04-21 8:47 0% ` Thomas Monjalon
2020-04-21 17:35 0% ` Wang, Haiyue
2020-04-22 5:08 4% ` [dpdk-dev] [PATCH v9 " Haiyue Wang
2020-04-22 5:08 3% ` [dpdk-dev] [PATCH v9 2/2] eal: support for VFIO-PCI VF token Haiyue Wang
2020-04-22 8:23 ` Thomas Monjalon
2020-04-22 14:02 3% ` Wang, Haiyue
2020-04-26 1:55 4% ` [dpdk-dev] [PATCH v10 0/2] support for VFIO-PCI VF token interface Haiyue Wang
2020-05-05 7:46 4% ` [dpdk-dev] [PATCH v11 " Haiyue Wang
2020-03-06 9:57 [dpdk-dev] [PATCH v2 00/16] NXP DPAAx fixes and enhancements Hemant Agrawal
2020-05-04 12:41 3% ` [dpdk-dev] [PATCH v3 0/8] " Hemant Agrawal
2020-03-06 16:41 [dpdk-dev] [PATCH 0/4] Introduce IF proxy library Andrzej Ostruszka
2020-05-04 8:53 2% ` [dpdk-dev] [PATCH v3 " Andrzej Ostruszka
2020-05-04 8:53 1% ` [dpdk-dev] [PATCH v3 1/4] lib: introduce IF Proxy library Andrzej Ostruszka
2020-03-18 20:41 [dpdk-dev] [PATCH] cryptodev: version rte_cryptodev_info_get function Arek Kusztal
2020-04-20 16:59 ` Trahe, Fiona
2020-04-20 17:31 ` Ray Kinsella
2020-04-20 17:37 ` Thomas Monjalon
2020-04-21 6:01 3% ` Ray Kinsella
2020-04-21 9:36 0% ` Thomas Monjalon
2020-04-22 8:21 0% ` Ray Kinsella
2020-03-19 17:18 [dpdk-dev] [PATCH 00/12] update and simplify telemetry library Ciara Power
2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 00/18] " Ciara Power
2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 02/18] telemetry: move code to metrics for later reuse Ciara Power
2020-04-24 15:29 3% ` Stephen Hemminger
2020-04-24 15:49 0% ` Bruce Richardson
2020-03-22 17:20 [dpdk-dev] rte_vfio_container_dma_map/unmap functions Tal Shnaiderman
2020-04-19 13:10 ` Thomas Monjalon
2020-04-20 14:07 ` Burakov, Anatoly
2020-04-20 17:39 ` Thomas Monjalon
2020-04-21 9:50 0% ` Burakov, Anatoly
2020-03-30 12:15 [dpdk-dev] [PATCH v3] eal/cpuflags: add x86 based cpu flags Kevin Laatz
2020-04-16 11:00 ` [dpdk-dev] [PATCH v4] " Kevin Laatz
2020-04-25 16:04 4% ` Thomas Monjalon
2020-04-27 9:22 4% ` Kinsella, Ray
2020-04-27 9:27 4% ` Ray Kinsella
2020-04-27 9:31 0% ` Laatz, Kevin
2020-04-27 9:35 0% ` Ray Kinsella
2020-04-27 10:08 0% ` Laatz, Kevin
2020-04-27 12:31 3% ` Thomas Monjalon
2020-04-27 13:58 0% ` Ray Kinsella
2020-04-29 11:22 0% ` Neil Horman
2020-04-30 7:59 4% ` Ray Kinsella
2020-04-28 12:40 ` [dpdk-dev] [PATCH v5] " Kevin Laatz
2020-04-28 16:39 ` Ray Kinsella
2020-04-28 18:11 4% ` Laatz, Kevin
2020-04-28 19:55 0% ` Thomas Monjalon
2020-04-29 11:39 3% ` David Marchand
2020-04-30 10:02 0% ` Ray Kinsella
2020-03-30 16:00 [dpdk-dev] [PATCH 1/2] ethdev: add tm cap for private shaper packet mode Nithin Dabilpuram
2020-04-22 17:21 ` [dpdk-dev] [PATCH v4 1/4] ethdev: add tm support for shaper config in pkt mode Nithin Dabilpuram
2020-04-24 10:28 ` Dumitrescu, Cristian
2020-04-25 20:09 3% ` Ferruh Yigit
2020-04-27 9:19 4% ` Dumitrescu, Cristian
2020-04-27 16:12 4% ` Ferruh Yigit
2020-04-27 16:28 3% ` Dumitrescu, Cristian
2020-04-28 15:30 4% ` Thomas Monjalon
2020-04-28 17:35 0% ` Dumitrescu, Cristian
2020-04-27 16:29 3% ` Jerin Jacob
2020-04-27 16:49 3% ` Ferruh Yigit
2020-04-27 16:59 3% ` Jerin Jacob
2020-04-28 11:51 0% ` [dpdk-dev] [EXT] " Nithin Dabilpuram
2020-04-28 13:56 0% ` Ferruh Yigit
2020-04-28 14:06 4% ` [dpdk-dev] " Ferruh Yigit
2020-04-28 14:45 5% ` Bruce Richardson
2020-04-28 15:04 0% ` Luca Boccassi
2020-04-28 15:54 5% ` Thomas Monjalon
2020-04-29 8:45 0% ` Dumitrescu, Cristian
2020-04-29 9:03 0% ` Bruce Richardson
2020-05-01 10:27 0% ` Ferruh Yigit
2020-05-01 13:16 0% ` [dpdk-dev] [EXT] " Nithin Dabilpuram
2020-05-01 13:18 3% ` [dpdk-dev] " Jerin Jacob
2020-05-05 8:01 0% ` Ray Kinsella
2020-04-28 15:42 3% ` Ray Kinsella
2020-04-10 9:46 [dpdk-dev] [PATCH] ethdev: support flow aging BillZhou
2020-04-18 9:44 ` [dpdk-dev] [PATCH v2] " Thomas Monjalon
2020-04-20 14:06 ` Ferruh Yigit
2020-04-20 16:10 ` Thomas Monjalon
2020-04-21 10:04 0% ` Ferruh Yigit
2020-04-21 10:09 0% ` Thomas Monjalon
2020-04-21 15:59 0% ` Andrew Rybchenko
2020-04-10 14:27 [dpdk-dev] [PATCH v3 0/4] add AESNI-MB rawdev for multi-function processing David Coyle
2020-04-14 14:44 ` Thomas Monjalon
2020-04-15 22:19 ` Doherty, Declan
2020-04-15 22:33 ` Thomas Monjalon
2020-04-21 16:46 ` Doherty, Declan
2020-04-21 17:23 ` Coyle, David
2020-04-22 10:51 ` Akhil Goyal
2020-04-22 13:17 ` Coyle, David
2020-04-22 13:44 3% ` Akhil Goyal
2020-04-22 14:21 0% ` Coyle, David
2020-05-01 13:18 0% ` Zhang, Roy Fan
2020-04-10 16:43 [dpdk-dev] [PATCH v2 00/10] eal: Windows basic memory management Dmitry Kozlyuk
2020-04-14 19:44 ` [dpdk-dev] [PATCH v3 00/10] " Dmitry Kozlyuk
2020-04-14 19:44 ` [dpdk-dev] [PATCH v3 10/10] eal/windows: implement " Dmitry Kozlyuk
2020-04-16 18:34 ` Ranjit Menon
2020-04-23 1:00 3% ` Dmitry Kozlyuk
2020-04-28 23:50 ` [dpdk-dev] [PATCH v4 0/8] Windows " Dmitry Kozlyuk
2020-04-28 23:50 4% ` [dpdk-dev] [PATCH v4 1/8] eal: replace rte_page_sizes with a set of constants Dmitry Kozlyuk
2020-04-13 15:00 [dpdk-dev] [PATCH v5 00/33] DPDK Trace support jerinj
2020-04-19 10:01 ` [dpdk-dev] [PATCH v6 " jerinj
2020-04-19 10:01 ` [dpdk-dev] [PATCH v6 05/33] eal/trace: implement trace operation APIs jerinj
2020-04-21 12:49 ` David Marchand
2020-04-21 13:40 3% ` Jerin Jacob
2020-04-21 14:09 0% ` David Marchand
2020-04-22 19:03 1% ` [dpdk-dev] [PATCH v7 00/32] DPDK Trace support jerinj
2020-04-16 14:54 [dpdk-dev] [PATCHv3] Remove validate-abi.sh from tree Neil Horman
2020-04-17 10:20 ` Thomas Monjalon
2020-04-17 11:47 ` Ray Kinsella
2020-04-17 12:10 ` Thomas Monjalon
2020-04-17 15:42 ` Ray Kinsella
2020-04-21 11:12 4% ` Neil Horman
2020-04-21 11:46 4% ` Thomas Monjalon
2020-04-21 18:56 4% ` Neil Horman
2020-04-21 21:42 4% ` Thomas Monjalon
2020-04-22 11:43 4% ` Ray Kinsella
2020-04-22 12:07 4% ` Neil Horman
2020-04-22 12:18 4% ` Thomas Monjalon
2020-04-22 13:19 4% ` Ray Kinsella
2020-04-22 13:30 4% ` Thomas Monjalon
2020-04-23 11:03 7% ` Neil Horman
2020-04-22 12:01 8% ` Neil Horman
2020-04-22 12:16 4% ` Thomas Monjalon
2020-04-23 10:57 4% ` Neil Horman
2020-04-17 14:59 [dpdk-dev] [PATCH v3] cryptodev: version cryptodev info get function Arek Kusztal
2020-04-20 14:22 ` Akhil Goyal
2020-04-20 15:21 ` Ray Kinsella
2020-04-20 16:54 ` Trahe, Fiona
2020-04-20 17:26 ` Ray Kinsella
2020-04-22 8:36 0% ` Ray Kinsella
2020-04-20 8:33 [dpdk-dev] [PATCH] cryptodev: add support for user callback functions Abhinandan Gujjar
2020-04-24 15:10 3% ` Ananyev, Konstantin
2020-04-20 9:34 [dpdk-dev] [PATCH] abi: change references to abi 20.0.1 to abi v21 Ray Kinsella
2020-04-20 11:57 ` Ray Kinsella
2020-04-20 12:20 ` David Marchand
2020-04-22 8:07 6% ` Ray Kinsella
2020-04-22 8:11 6% ` David Marchand
2020-04-22 8:18 9% ` Thomas Monjalon
2020-04-22 8:28 6% ` Ray Kinsella
2020-04-23 6:41 21% ` [dpdk-dev] [PATCH v2] " Ray Kinsella
2020-04-24 9:15 6% ` Thomas Monjalon
2020-04-24 14:10 6% ` David Marchand
2020-04-24 14:50 6% ` Ray Kinsella
2020-04-24 15:01 6% ` David Marchand
2020-04-29 12:19 9% ` Dodji Seketeli
2020-04-30 8:23 6% ` Ray Kinsella
2020-04-27 9:06 11% ` [dpdk-dev] [PATCH v3] " Ray Kinsella
2020-04-27 9:06 16% ` Ray Kinsella
2020-04-27 13:45 11% ` [dpdk-dev] [PATCH v4] " Ray Kinsella
2020-04-27 13:45 21% ` Ray Kinsella
2020-04-30 10:21 31% ` [dpdk-dev] [PATCH v5] " Ray Kinsella
2020-04-30 10:27 11% ` [dpdk-dev] [PATCH v6 1/1] " Ray Kinsella
2020-04-30 10:27 31% ` Ray Kinsella
2020-05-04 22:23 6% ` Thomas Monjalon
2020-05-04 22:05 6% ` Thomas Monjalon
2020-04-20 12:11 [dpdk-dev] [PATCH v6 00/10] New sync modes for ring Konstantin Ananyev
2020-04-20 12:28 ` [dpdk-dev] [PATCH v7 " Konstantin Ananyev
2020-04-21 11:31 0% ` David Marchand
2020-04-20 16:05 [dpdk-dev] [PATCH v2] lib/timer: relax barrier for status update Phil Yang
2020-04-24 7:24 ` [dpdk-dev] [PATCH v3] " Phil Yang
2020-04-25 17:17 3% ` Thomas Monjalon
2020-04-26 7:36 3% ` Phil Yang
2020-04-26 12:18 0% ` Carrillo, Erik G
2020-04-26 14:20 0% ` Phil Yang
2020-04-26 19:30 0% ` Carrillo, Erik G
2020-04-26 14:45 2% ` [dpdk-dev] [PATCH v4] " Phil Yang
2020-04-21 1:33 [dpdk-dev] [PATCH] devtools: skip ABI check in static builds Thomas Monjalon
2020-04-21 6:11 4% ` Ray Kinsella
2020-04-21 9:15 4% ` Thomas Monjalon
2020-04-21 8:02 4% ` David Marchand
2020-04-21 9:20 4% ` Thomas Monjalon
2020-04-21 2:04 13% [dpdk-dev] [PATCH] devtools: remove useless files from ABI reference Thomas Monjalon
2020-04-21 7:49 9% ` David Marchand
2020-04-21 9:22 4% ` Thomas Monjalon
2020-04-21 9:26 4% ` David Marchand
2020-04-21 10:33 4% ` Bruce Richardson
2020-04-21 6:22 [dpdk-dev] [PATCH v3] ethdev: support flow aging Bill Zhou
2020-04-21 10:11 2% ` [dpdk-dev] [PATCH v4] " Bill Zhou
2020-04-21 17:13 0% ` Ferruh Yigit
2020-04-29 14:50 0% ` Tom Barbette
2020-04-30 7:36 0% ` Matan Azrad
2020-04-30 7:49 0% ` Tom Barbette
2020-04-23 10:12 24% [dpdk-dev] [PATCH v1] abi: document reasons behind the three part versioning Ray Kinsella
2020-05-04 21:46 4% ` Thomas Monjalon
2020-05-05 8:07 4% ` Ray Kinsella
2020-04-23 13:08 4% [dpdk-dev] DPDK Release Status Meeting 23/04/2020 Ferruh Yigit
2020-04-28 9:15 3% [dpdk-dev] [Bug 465] app/test compile failed with gcc and clang bugzilla
2020-04-29 18:05 2% [dpdk-dev] [RFC] hash: unify crc32 API header for x86 and ARM pbhagavatula
2020-04-30 9:14 0% ` Van Haaren, Harry
2020-04-30 9:27 0% ` Pavan Nikhilesh Bhagavatula
2020-04-30 5:46 33% [dpdk-dev] [PATCH v1 1/2] devtools: add internal ABI version check Haiyue Wang
2020-04-30 5:46 4% ` [dpdk-dev] [PATCH v1 2/2] devtools: updating internal symbols ABI version Haiyue Wang
2020-04-30 13:55 4% [dpdk-dev] DPDK Release Status Meeting 30/04/2020 Ferruh Yigit
2020-04-30 15:52 [dpdk-dev] [PATCH v4 1/2] cryptodev: add chacha20-poly1305 aead algorithm Arek Kusztal
2020-04-30 15:52 10% ` [dpdk-dev] [PATCH v4 2/2] cryptodev: version cryptodev info get function Arek Kusztal
2020-05-01 17:16 [dpdk-dev] [PATCH] doc: deprication notice to mark tm spec as experimental Nithin Dabilpuram
2020-05-05 6:47 3% ` Ray Kinsella
2020-05-05 7:45 0% ` [dpdk-dev] [EXT] " Nithin Dabilpuram
2020-05-05 8:07 5% ` [dpdk-dev] [PATCH v2] " Nithin Dabilpuram
2020-05-02 16:10 4% [dpdk-dev] [PATCH] common/octeontx: fix GCC 9.1 ABI break pbhagavatula
2020-05-04 9:17 4% ` Harman Kalra
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).