* Re: [dpdk-dev] [PATCH v5 12/14] librte_ethdev: add ESP and AH flow types to RSS
@ 2020-01-15 14:36 0% ` Ferruh Yigit
0 siblings, 0 replies; 200+ results
From: Ferruh Yigit @ 2020-01-15 14:36 UTC (permalink / raw)
To: Iremonger, Bernard, Andrew Rybchenko, dev, Xing, Beilei, Zhang,
Qi Z, Doherty, Declan
Cc: Ananyev, Konstantin, Byrne, Stephen1, Zhang, Helin, Ori Kam,
Thomas Monjalon, Richardson, Bruce, Kinsella, Ray,
David Marchand, Luca Boccassi, Jerin Jacob, Olivier MATZ
On 1/15/2020 2:11 PM, Iremonger, Bernard wrote:
> Hi Ferruh,
>
> <snip>
>
>> Subject: Re: [dpdk-dev] [PATCH v5 12/14] librte_ethdev: add ESP and AH
>> flow types to RSS
>>
>> On 1/15/2020 10:55 AM, Andrew Rybchenko wrote:
>>> On 1/15/20 1:44 PM, Ferruh Yigit wrote:
>>>> On 1/15/2020 9:13 AM, Andrew Rybchenko wrote:
>>>>> On 1/14/20 9:45 PM, Ferruh Yigit wrote:
>>>>>> On 1/14/2020 1:55 PM, Bernard Iremonger wrote:
>>>>>>> Add macros for the following protocols in the DDP esp-ah profile:
>>>>>>> ESP
>>>>>>> AH
>>>>>>>
>>>>>>> Add the following RSS macro for IPsec:
>>>>>>> ETH_RSS_IPSEC
>>>>>>>
>>>>>>> Signed-off-by: Bernard Iremonger <bernard.iremonger@intel.com>
>>>>>> +Ori and other ethdev maintainers.
>>>>>>
>>>>>> Ori, can you please check this patch?
>>>>>>
>>>>>>> ---
>>>>>>> lib/librte_ethdev/rte_ethdev.h | 14 +++++++++++++-
>>>>>>> 1 file changed, 13 insertions(+), 1 deletion(-)
>>>>>>>
>>>>>>> diff --git a/lib/librte_ethdev/rte_ethdev.h
>>>>>>> b/lib/librte_ethdev/rte_ethdev.h index 18a9def..208ec90 100644
>>>>>>> --- a/lib/librte_ethdev/rte_ethdev.h
>>>>>>> +++ b/lib/librte_ethdev/rte_ethdev.h
>>>>>>> @@ -484,7 +484,9 @@ struct rte_eth_rss_conf {
>>>>>>> #define RTE_ETH_FLOW_NVGRE 21 /**< NVGRE protocol
>> based flow */
>>>>>>> #define RTE_ETH_FLOW_VXLAN_GPE 22 /**< VXLAN-GPE
>> protocol based flow */
>>>>>>> #define RTE_ETH_FLOW_GTPU 23 /**< GTPU protocol based
>> flow */
>>>>>>> -#define RTE_ETH_FLOW_MAX 24
>>>>>>> +#define RTE_ETH_FLOW_AH 24 /**< AH protocol based flow
>> */
>>>>>>> +#define RTE_ETH_FLOW_ESP 25 /**< ESP protocol based
>> flow */
>>>>>>> +#define RTE_ETH_FLOW_MAX 26
>>>>> Isn't changing RTE_ETH_FLOW_MAX value breaking ABI?
>>>>> Is v20.11 target release of the patch?
>>>> I can't see how this can cause an ABI break, unless the
>>>> 'RTE_ETH_FLOW_MAX' value used as size of an array in the middle of a
>> struct.
>>>> There is 'struct rte_eth_fdir_flex_conf' but the array is at the end
>>>> there, so it should be OK, unless that struct is not in the middle of another
>> struct.
>>>
>>> rte_eth_fdir_flex_conf -> rte_fdir_conf -> rte_eth_conf (in the
>>> middle)
>>
>> Yes, this looks like an ABI break and this is very annoying not able to even add
>> a new RTE_FLOW type.
>>
>> We need to find a proper way to handle this, at first glance I can see stop
>> using _MAX macros for the array size can work and perhaps we can use
>> another big enough hardcoded value for all similar array size. Any other
>> option?
>>
>> But we can do this on 20.11, we need a solution until that time.
>
> This patch is required to handle RSS for the esp-ah.pkg DDP profile, it does not affect the i40e FD and testpmd changes in this patch set.
> The esp-ah.pkg DDP profile is not released yet.
>
> Given that the merge deadline is today, I propose to drop this patch from the v6 patch set.
+1, if it can be separated, better to do it to not block rest of the work.
>
>>
>>>
>>>> And there are values calculated from 'RTE_ETH_FLOW_MAX', like
>>>> 'RTE_FLOW_MASK_ARRAY_SIZE', same concern applies there, it very hard
>> to follow.
>>>>
>>>> Bernard,
>>>>
>>>> Can you please run the ABI version script to be sure this is not breaking
>> the ABI?
>>>>
>>>>
>>>>>>>
>>>>>>> /*
>>>>>>> * Below macros are defined for RSS offload types, they can be
>>>>>>> used to @@ -511,6 +513,12 @@ struct rte_eth_rss_conf {
>>>>>>> #define ETH_RSS_GENEVE (1ULL << 20)
>>>>>>> #define ETH_RSS_NVGRE (1ULL << 21)
>>>>>>> #define ETH_RSS_GTPU (1ULL << 23)
>>>>>>> +#define ETH_RSS_AH (1ULL << 24)
>>>>>>> +#define ETH_RSS_ESP (1ULL << 25)
>>>>>>> +
>>>>>>> +
>>>>>>> +
>>>>>>> +
>>>>>>>
>>>>>>> /*
>>>>>>> * We use the following macros to combine with above ETH_RSS_*
>>>>>>> for @@ -571,6 +579,10 @@ rte_eth_rss_hf_refine(uint64_t rss_hf)
>>>>>>> ETH_RSS_NONFRAG_IPV4_SCTP | \
>>>>>>> ETH_RSS_NONFRAG_IPV6_SCTP)
>>>>>>>
>>>>>>> +#define ETH_RSS_IPSEC ( \
>>>>>>> +ETH_RSS_AH | \
>>>>>>> +ETH_RSS_ESP)
>>>>>>> +
>>>>>>> #define ETH_RSS_TUNNEL ( \
>>>>>>> ETH_RSS_VXLAN | \
>>>>>>> ETH_RSS_GENEVE | \
>>>>>>>
>>>
> Regards,
>
> Bernard.
>
^ permalink raw reply [relevance 0%]
* [dpdk-dev] [PATCH v6 00/13] net/i40e: ESP support
@ 2020-01-15 15:53 3% ` Bernard Iremonger
2020-01-16 12:44 3% ` [dpdk-dev] [PATCH v7 00/10] " Bernard Iremonger
0 siblings, 1 reply; 200+ results
From: Bernard Iremonger @ 2020-01-15 15:53 UTC (permalink / raw)
To: dev, beilei.xing, qi.z.zhang, declan.doherty
Cc: konstantin.ananyev, stephen1.byrne, helin.zhang, Bernard Iremonger
Changes in V6:
-------------
Rebase to latest master branch
Fixed patch 0001-app-testpmd-parse-flow-command-line-for-ESP.patch
Dropped the following v5 patch due to ABI breakage:
0012-librte_ethdev-add-ESP-and-AH-flow-types-to-RSS.patch
Changes in V5:
-------------
Rebase to latest master branch
Update 0012-librte_ethdev-add-ESP-and-AH-flow-types-to-RSS.patch
Changes in V4:
-------------
Add extra patch for testpmd
Add extra patch for config
Split v3 patch "net/i40e: process ESP flows" into 4 patches
Changes in V3:
-------------
Added i40e_flow_set_filter_spi() function in i40e_flow.c
Set UDP destination port to 4500 for ESP in i40e_ethdev.h
Split flow structures into 4 instead of 2 in i40e_ethdev.h
Dropped extra printf from commandline_flow.c
Changes in V2:
--------------
Moved change in app/test-pmd/config.c to a seperate patch.
Added extra parameter to fill_ip6_head() in i40e_fdir.c
set is_udp to false in i40e_flow_fdir_get_pctype_value() in i40e_flow.c
Bernard Iremonger (13):
app/testpmd: parse flow command line for ESP
app/testpmd: improve debug
app/testpmd: dump Rx and Tx mbuf
net/i40e: improve RSS debug
net/i40e: handle ESP tunnel
net/i40e: support ipsec-ah profile
net/i40e: support ESP in customized code
net/i40e: support ESP flows
net/i40e: support ESP in Flow Director
config: add debug to I40E Flow Director
net/i40e: display Flow Director packet
doc: release note for ESP
doc: update i40e user guide
app/test-pmd/cmdline_flow.c | 34 +++++++-
app/test-pmd/config.c | 2 +-
app/test-pmd/util.c | 1 +
config/common_base | 1 +
doc/guides/nics/i40e.rst | 4 +-
doc/guides/rel_notes/release_20_02.rst | 9 +++
drivers/net/i40e/i40e_ethdev.c | 52 ++++++++++++-
drivers/net/i40e/i40e_ethdev.h | 38 +++++++++
drivers/net/i40e/i40e_fdir.c | 138 ++++++++++++++++++++++++++++++---
drivers/net/i40e/i40e_flow.c | 135 +++++++++++++++++++++++++++++++-
drivers/net/i40e/rte_pmd_i40e.c | 3 +-
11 files changed, 395 insertions(+), 22 deletions(-)
--
2.7.4
^ permalink raw reply [relevance 3%]
* [dpdk-dev] DPDK Release Status Meeting 16/01/2020
@ 2020-01-16 11:13 4% Ferruh Yigit
2020-01-16 12:48 3% ` Ananyev, Konstantin
2020-01-17 16:00 0% ` Honnappa Nagarahalli
0 siblings, 2 replies; 200+ results
From: Ferruh Yigit @ 2020-01-16 11:13 UTC (permalink / raw)
To: dpdk-dev; +Cc: Thomas Monjalon
Minutes 16 January 2020
-----------------------
Agenda:
* Release Dates
* Subtrees
* OvS
Participants:
* Debian/Microsoft
* Intel
* Marvell
* Mellanox
* NXP
* Red Hat
Release Dates
-------------
* v20.02 dates:
* Integration deadline passed, it was on Wednesday 15 January 2020
* RC1 is expected before the weekend, possibly on Friday 17 January
* PRC holidays on 24-20 Jan (inclusive)
* Release: Friday 14 February 2020
* v20.05 proposal:
* Proposal/V1: Friday 6 March 2020
* Integration/Merge/RC1: Friday 10 April 2020
* Release: Friday 1 May || Wed 13 May
* 1-5 May holiday on PRC, we need to do the release before or after that
Please comment on between 1 May or 13 May.
Subtrees
--------
* main
* Will try to do -rc1 before Monday with best effort
* Two series from arm
* "ring library" updates
* May go in in this release
* Use WFE for aarch64
* ABI tooling patches
* Tech-board selected David's approach
* Automated checks will be in CI after David's patch
* We need automated checks to be sure we are not breaking the ABI
otherwise it is not always easy for developer to catch the break
* Waiting new version of Neil's __rte_internal pathcset
* next-net
* Almost all ethdev changes merged, planning to finalize last
remaining 1-2 ones today
* All sub-trees pulled except some mlx and brcm one that are waiting fix
* Some PMD patches may be postponed to -rc2
* next-net-crypto
* Pull request sent
* There is a performance concern on some ipsec-gw patches,
they can go in -rc2 if the issue is solved
* CPU crypto from last release may be breaking ABI, need to confirm
and discussed dummy variable is missing, may be postponed to next release
* Some PMD changes are postponed to -rc2
* next-net-eventdev
* Pull request sent, it is a small set
* next-net-virtio
* All commit pulled to next-net
* virtio vDPA drive will be postponed to next release
* Some patches waiting user change and some are under review
* Can be some more patches for -rc2
* next-net-intel
* Nothing critical, most of the patches have been pulled
* 'fm10k' patches are not reviewed, may go in -rc2 if reviewed
otherwise will be postponed to next release
* LTS
* 17.11.10-rc1 released, please test and report results
* https://mails.dpdk.org/archives/dev/2020-January/154915.html
* Release planned on January / February
* 18.11.6-rc2 released, please test and report results
* https://mails.dpdk.org/archives/dev/2020-January/155132.html
* Only a few patches on top of -rc1
* Intel will plan a validation, it won't be full validation
* Release planned on 31st January
OvS
---
* Using DPDK experimental APIs discussed, it is accepted to use once as
exception, so feature won't be blocked
* DPDK need to figure out how to backport removing experimental tags to the
LTS releases
DPDK Release Status Meetings
============================
The DPDK Release Status Meeting is intended for DPDK Committers to discuss
the status of the master tree and sub-trees, and for project managers to
track progress or milestone dates.
The meeting occurs on Thursdays at 8:30 UTC. If you wish to attend just
send an email to "John McNamara <john.mcnamara@intel.com>" for the invite.
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [dpdk-stable] [PATCH] meter: move RFC4115 trTCM APIs as none experimental
@ 2020-01-16 11:25 5% ` David Marchand
2020-01-16 11:54 0% ` Neil Horman
0 siblings, 1 reply; 200+ results
From: David Marchand @ 2020-01-16 11:25 UTC (permalink / raw)
To: Ray Kinsella, Thomas Monjalon, Neil Horman
Cc: Cristian Dumitrescu, dpdk stable, dev, Eelco Chaudron,
Kevin Traynor, Ian Stokes, Ilya Maximets, Luca Boccassi, Yigit,
Ferruh
On Tue, Dec 17, 2019 at 2:08 PM Eelco Chaudron <echaudro@redhat.com> wrote:
>
> Moved RFC4115 APIs to none experimental as they have been there
> since 19.02. Also, these APIs are the same as the none RFC4115 APIs.
>
> Signed-off-by: Eelco Chaudron <echaudro@redhat.com>
There is a discussion on the OVS ml at the moment to get these symbols
in the stable ABI for 19.11.
I want to understand how this would be done.
- I take this patch in 20.02, these symbols are added in the 20.0.1 ABI.
On the other hand, the 19.11 release maintains the 20.0 ABI.
Does it mean the backport adds these symbols with the 20.0 version in
the 19.11 branch?
Or is 20.0.1 version acceptable / a thing we want?
- These symbol already existed in the 20.0 ABI, versioned as EXPERIMENTAL.
We can go and remove these entries since we are not bound to preserve
the experimental APIs.
But, on the other hand, nothing should prevent us from keeping some
aliases so that the symbols versioned EXPERIMENTAL are still available
to existing users.
--
David Marchand
^ permalink raw reply [relevance 5%]
* Re: [dpdk-dev] [PATCH] add ABI checks
@ 2020-01-16 11:52 4% ` Neil Horman
2020-01-16 14:20 4% ` Thomas Monjalon
0 siblings, 1 reply; 200+ results
From: Neil Horman @ 2020-01-16 11:52 UTC (permalink / raw)
To: Thomas Monjalon
Cc: David Marchand, Richardson, Bruce, dev, Laatz, Kevin, aconole,
Michael Santana, Mcnamara, John, Kovacevic, Marko, Kinsella, Ray
On Wed, Jan 15, 2020 at 01:38:17PM +0100, Thomas Monjalon wrote:
> 15/01/2020 12:33, Neil Horman:
> > On Wed, Jan 15, 2020 at 12:19:30AM +0100, Thomas Monjalon wrote:
> > > 20/12/2019 17:20, Kinsella, Ray:
> > > > From: Richardson, Bruce <bruce.richardson@intel.com>
> > > > > From: David Marchand <david.marchand@redhat.com>
> > > > > > +Checking ABI compatibility
> > > > > > +--------------------------
> > > > > > +
> > > > > > +The first thing is to build reference binaries for the latest
> > > > > release
> > > > > > +your patches are built on top of.
> > > > > > +
> > > > > > +Either you are in a git tree and an easy way to identify this is to
> > > > > run::
> > > > > > +
> > > > > > + git checkout $(git describe --abbrev=0)
> > > > > > +
> > > > > > +Or you use a tarball and you extract the sources in a director of
> > > > > > +your
> > > > > > choice.
> > > > > > +
> > > > > > +Next is building those sources, refer to the previous paragraph.
> > > > > > +You can set ``DPDK_BUILD_TEST_DIR=reference``, so that the builds
> > > > > > +occur in this directory.
> > > > > > +
> > > > > > +Finally, the ABI dump files are generated with the
> > > > > > +``devtools/gen-abi-reference.sh`` script. This script will look for
> > > > > > +builds in the current sub directory ``reference``. But you can set
> > > > > > +the environment variable ``DPDK_ABI_REF_BUILD_DIR`` to a different
> > > > > location.
> > > > > > +
> > > > > > +Once done, you can check your current binaries ABI with this
> > > > > > +reference with the ``devtools/check-abi-reference.sh`` script.
> > > > > >
> > > > >
> > > > > I still very much dislike forcing the user to generate his own
> > > > > reference version to compare the ABI against. These should be archived
> > > > > and the user should just be able to pull them down via git or http or
> > > > > otherwise. Two reasons for this:
> > > > >
> > > > > 1. Less error prone, since there is no chance of the user having an
> > > > > incorrect build for whatever reason.
> > > > >
> > > > > 2. Less effort for the user than asking them to do extra builds. The
> > > > > more steps the user has to follow, the less likely they are to attempt
> > > > > the process.
> > > >
> > > > +1 ... 100% agree with this.
> > > >
> > > > Many people won't know or understand what the reference is,
> > > > or why they to generate it.
> > >
> > > I don't want to generate and save the reference in git for each arch.
> > >
> > Can I ask what your reluctance is? Is it related to not wanting to have to save
> > all this information that is otherwise not used for building purposes?
>
> Yes I prefer keeping only the sources in the repository.
> And these dumps are big.
> And last but not the least, there is no ready-to-use environment to build
> and dump all libs for all archs.
>
> > If so I might suggest saving the dumps in a separate git tree and pulling them
> > in as a git submodule when the check is performed
> >
> > I really like the idea of caching the results so everyone is working from a
> > known ABI baseline.
>
> You don't trust the result of the build made from tagged sources?
>
I trust the result from the tools, sure, its trusting that people will take the
extra time to build a version from a prior tag that I'm less sure of.
Consistent use in my mind is predicated on ease and timeliness of use.
I get not wanting to store large dumps in the source tree, but storage is cheap,
and I don't see the issue with storing the xml dump in a separate git tree to be
referenced through a git submodule that gets pulled in when the check is run.
Neil
> > > We can make reference build more automatic with a command like this:
> > > git clone --branch v19.11 . $DPDK_BUILD_TEST_DIR/abiref-19.11
> > >
> > > Also I don't like mixing build and check steps.
> > > I believe the compilation should be simple and right to the point.
> > >
> > > This approach, from David, does not prevent from saving the dumps later
> > > if we really feel a strong need.
> > >
> > > That's why I suggest going with this patch.
>
>
>
>
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [dpdk-stable] [PATCH] meter: move RFC4115 trTCM APIs as none experimental
2020-01-16 11:25 5% ` [dpdk-dev] [dpdk-stable] " David Marchand
@ 2020-01-16 11:54 0% ` Neil Horman
2020-01-16 12:42 0% ` Ferruh Yigit
0 siblings, 1 reply; 200+ results
From: Neil Horman @ 2020-01-16 11:54 UTC (permalink / raw)
To: David Marchand
Cc: Ray Kinsella, Thomas Monjalon, Cristian Dumitrescu, dpdk stable,
dev, Eelco Chaudron, Kevin Traynor, Ian Stokes, Ilya Maximets,
Luca Boccassi, Yigit, Ferruh
On Thu, Jan 16, 2020 at 12:25:06PM +0100, David Marchand wrote:
> On Tue, Dec 17, 2019 at 2:08 PM Eelco Chaudron <echaudro@redhat.com> wrote:
> >
> > Moved RFC4115 APIs to none experimental as they have been there
> > since 19.02. Also, these APIs are the same as the none RFC4115 APIs.
> >
> > Signed-off-by: Eelco Chaudron <echaudro@redhat.com>
>
> There is a discussion on the OVS ml at the moment to get these symbols
> in the stable ABI for 19.11.
> I want to understand how this would be done.
>
> - I take this patch in 20.02, these symbols are added in the 20.0.1 ABI.
> On the other hand, the 19.11 release maintains the 20.0 ABI.
>
> Does it mean the backport adds these symbols with the 20.0 version in
> the 19.11 branch?
> Or is 20.0.1 version acceptable / a thing we want?
>
>
> - These symbol already existed in the 20.0 ABI, versioned as EXPERIMENTAL.
> We can go and remove these entries since we are not bound to preserve
> the experimental APIs.
> But, on the other hand, nothing should prevent us from keeping some
> aliases so that the symbols versioned EXPERIMENTAL are still available
> to existing users.
>
I would say that choice is up to you. If you want to alias them to be nice to
prior users, thats fine by me. But experimental means experimental, and so users
have to be prepared to rebuild when things change, even if that change is
changing the version from experimental to a concrete version.
Neil
>
> --
> David Marchand
>
>
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [dpdk-stable] [PATCH] meter: move RFC4115 trTCM APIs as none experimental
2020-01-16 11:54 0% ` Neil Horman
@ 2020-01-16 12:42 0% ` Ferruh Yigit
2020-01-16 14:38 0% ` Thomas Monjalon
` (2 more replies)
0 siblings, 3 replies; 200+ results
From: Ferruh Yigit @ 2020-01-16 12:42 UTC (permalink / raw)
To: Neil Horman, David Marchand
Cc: Ray Kinsella, Thomas Monjalon, Cristian Dumitrescu, dpdk stable,
dev, Eelco Chaudron, Kevin Traynor, Ian Stokes, Ilya Maximets,
Luca Boccassi
On 1/16/2020 11:54 AM, Neil Horman wrote:
> On Thu, Jan 16, 2020 at 12:25:06PM +0100, David Marchand wrote:
>> On Tue, Dec 17, 2019 at 2:08 PM Eelco Chaudron <echaudro@redhat.com> wrote:
>>>
>>> Moved RFC4115 APIs to none experimental as they have been there
>>> since 19.02. Also, these APIs are the same as the none RFC4115 APIs.
>>>
>>> Signed-off-by: Eelco Chaudron <echaudro@redhat.com>
>>
>> There is a discussion on the OVS ml at the moment to get these symbols
>> in the stable ABI for 19.11.
>> I want to understand how this would be done.
>>
>> - I take this patch in 20.02, these symbols are added in the 20.0.1 ABI.
>> On the other hand, the 19.11 release maintains the 20.0 ABI.
>>
>> Does it mean the backport adds these symbols with the 20.0 version in
>> the 19.11 branch?
>> Or is 20.0.1 version acceptable / a thing we want?
>>
>>
>> - These symbol already existed in the 20.0 ABI, versioned as EXPERIMENTAL.
>> We can go and remove these entries since we are not bound to preserve
>> the experimental APIs.
>> But, on the other hand, nothing should prevent us from keeping some
>> aliases so that the symbols versioned EXPERIMENTAL are still available
>> to existing users.
>>
> I would say that choice is up to you. If you want to alias them to be nice to
> prior users, thats fine by me. But experimental means experimental, and so users
> have to be prepared to rebuild when things change, even if that change is
> changing the version from experimental to a concrete version.
>
I would prefer to keep the alias and don't break the existing users, specially
for the case experimental API is becoming mature without change.
^ permalink raw reply [relevance 0%]
* [dpdk-dev] [PATCH v7 00/10] net/i40e: ESP support
2020-01-15 15:53 3% ` [dpdk-dev] [PATCH v6 00/13] net/i40e: ESP support Bernard Iremonger
@ 2020-01-16 12:44 3% ` Bernard Iremonger
2020-01-16 17:42 0% ` Ferruh Yigit
0 siblings, 1 reply; 200+ results
From: Bernard Iremonger @ 2020-01-16 12:44 UTC (permalink / raw)
To: dev, beilei.xing, qi.z.zhang, declan.doherty
Cc: konstantin.ananyev, stephen1.byrne, helin.zhang, Bernard Iremonger
Add support for ESP flows to testpmd.
Improve debug information in testpmd and the i40e PMD.
Process ESP flows on the i40e Flow Director.
Changes in V7:
-------------
Dropped the following debug patch from v6 patchset
0003-app-testpmd-dump-Rx-and-Tx-mbuf.patch
Included release notes and doc patches in feature patches.
Changes in V6:
-------------
Rebase to latest master branch
Fixed patch 0001-app-testpmd-parse-flow-command-line-for-ESP.patch
Dropped the following v5 patch due to ABI breakage:
0012-librte_ethdev-add-ESP-and-AH-flow-types-to-RSS.patch
Changes in V5:
-------------
Rebase to latest master branch
Update 0012-librte_ethdev-add-ESP-and-AH-flow-types-to-RSS.patch
Changes in V4:
-------------
Add extra patch for testpmd
Add extra patch for config
Split v3 patch "net/i40e: process ESP flows" into 4 patches
Changes in V3:
-------------
Added i40e_flow_set_filter_spi() function in i40e_flow.c
Set UDP destination port to 4500 for ESP in i40e_ethdev.h
Split flow structures into 4 instead of 2 in i40e_ethdev.h
Dropped extra printf from commandline_flow.c
Changes in V2:
--------------
Moved change in app/test-pmd/config.c to a seperate patch.
Added extra parameter to fill_ip6_head() in i40e_fdir.c
set is_udp to false in i40e_flow_fdir_get_pctype_value() in i40e_flow.c
Bernard Iremonger (10):
app/testpmd: parse flow command line for ESP
app/testpmd: improve debug
net/i40e: improve RSS debug
net/i40e: handle ESP tunnel
net/i40e: support ipsec-ah profile
net/i40e: support ESP in customized code
net/i40e: support ESP flows
net/i40e: support ESP in Flow Director
config: add debug to I40E Flow Director
net/i40e: display Flow Director packet
app/test-pmd/cmdline_flow.c | 34 ++++++-
app/test-pmd/config.c | 2 +-
config/common_base | 1 +
doc/guides/nics/i40e.rst | 4 +-
doc/guides/rel_notes/release_20_02.rst | 10 ++
doc/guides/testpmd_app_ug/testpmd_funcs.rst | 14 +++
drivers/net/i40e/i40e_ethdev.c | 52 ++++++++++-
drivers/net/i40e/i40e_ethdev.h | 38 ++++++++
drivers/net/i40e/i40e_fdir.c | 138 ++++++++++++++++++++++++++--
drivers/net/i40e/i40e_flow.c | 135 ++++++++++++++++++++++++++-
drivers/net/i40e/rte_pmd_i40e.c | 3 +-
11 files changed, 409 insertions(+), 22 deletions(-)
--
2.7.4
^ permalink raw reply [relevance 3%]
* Re: [dpdk-dev] DPDK Release Status Meeting 16/01/2020
2020-01-16 11:13 4% [dpdk-dev] DPDK Release Status Meeting 16/01/2020 Ferruh Yigit
@ 2020-01-16 12:48 3% ` Ananyev, Konstantin
2020-01-16 13:17 4% ` Akhil Goyal
2020-01-17 16:00 0% ` Honnappa Nagarahalli
1 sibling, 1 reply; 200+ results
From: Ananyev, Konstantin @ 2020-01-16 12:48 UTC (permalink / raw)
To: Yigit, Ferruh, dpdk-dev; +Cc: Thomas Monjalon, akhil.goyal
Hi everyone,
>
> * next-net-crypto
> * Pull request sent
> * There is a performance concern on some ipsec-gw patches,
> they can go in -rc2 if the issue is solved
> * CPU crypto from last release may be breaking ABI, need to confirm
AFAIK, there is no ABI breakage.
> and discussed dummy variable is missing, may be postponed to next release
Not sure I understand last sentence, could the author explain
what dummy variables we are talking about.
Konstantin
^ permalink raw reply [relevance 3%]
* Re: [dpdk-dev] DPDK Release Status Meeting 16/01/2020
2020-01-16 12:48 3% ` Ananyev, Konstantin
@ 2020-01-16 13:17 4% ` Akhil Goyal
2020-01-16 13:39 4% ` Ananyev, Konstantin
0 siblings, 1 reply; 200+ results
From: Akhil Goyal @ 2020-01-16 13:17 UTC (permalink / raw)
To: Ananyev, Konstantin, Yigit, Ferruh, dpdk-dev; +Cc: Thomas Monjalon
Hi Konstantin,
> Hi everyone,
>
> >
> > * next-net-crypto
> > * Pull request sent
> > * There is a performance concern on some ipsec-gw patches,
> > they can go in -rc2 if the issue is solved
> > * CPU crypto from last release may be breaking ABI, need to confirm
>
> AFAIK, there is no ABI breakage.
This is the output of validate-abi.sh.
Change Effect
1 Field sym_cpu_process has been added to this type. 1) This field will not be initialized by old clients.
2) Size of the inclusive type has been changed.
NOTE: this field should be accessed only from the new library functions, otherwise it may result in crash or incorrect behavior of applications.
2 Size of this type has been changed from 128 bytes to 136 bytes. The fields or parameters of such data type may be incorrectly initialized or accessed by old client applications.
Apart from that, IPSEC also has breakage, but that is experimental, so not an issue.
>
> > and discussed dummy variable is missing, may be postponed to next release
>
> Not sure I understand last sentence, could the author explain
> what dummy variables we are talking about.
In one of the techboard meeting around 19.11 timeframe, during the discussion for approving methodology for CPU-crypto, it was proposed that in order to avoid delay, a dummy variable can be introduced in cryptodev API/ABI to avoid any ABI breakage in upcoming releases. But this was not done.
>
> Konstantin
>
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] DPDK Release Status Meeting 16/01/2020
2020-01-16 13:17 4% ` Akhil Goyal
@ 2020-01-16 13:39 4% ` Ananyev, Konstantin
2020-01-16 16:42 4% ` Akhil Goyal
0 siblings, 1 reply; 200+ results
From: Ananyev, Konstantin @ 2020-01-16 13:39 UTC (permalink / raw)
To: Akhil Goyal, Yigit, Ferruh, dpdk-dev; +Cc: Thomas Monjalon
Hi Akhil,
> > >
> > > * next-net-crypto
> > > * Pull request sent
> > > * There is a performance concern on some ipsec-gw patches,
> > > they can go in -rc2 if the issue is solved
> > > * CPU crypto from last release may be breaking ABI, need to confirm
> >
> > AFAIK, there is no ABI breakage.
>
> This is the output of validate-abi.sh.
>
> Change Effect
> 1 Field sym_cpu_process has been added to this type. 1) This field will not be initialized by old clients.
> 2) Size of the inclusive type has been changed.
>
> NOTE: this field should be accessed only from the new library
> functions, otherwise it may result in crash or incorrect behavior of applications.
> 2 Size of this type has been changed from 128 bytes to 136 bytes. The fields or parameters of such data type may be incorrectly
> initialized or accessed by old client applications.
This is struct rte_cryptodev_ops, which is AFAIK, not part of public API.
So not sure, why do you consider it as ABI breakage?
>
> Apart from that, IPSEC also has breakage, but that is experimental, so not an issue.
>
> >
> > > and discussed dummy variable is missing, may be postponed to next release
> >
> > Not sure I understand last sentence, could the author explain
> > what dummy variables we are talking about.
>
> In one of the techboard meeting around 19.11 timeframe, during the discussion for approving methodology for CPU-crypto, it was
> proposed that in order to avoid delay, a dummy variable can be introduced in cryptodev API/ABI to avoid any ABI breakage in
> upcoming releases. But this was not done.
Dummy variable for what?
If you are talking about sym_cpu_process - we just added it into rte_cryptodev_ops, instead of
' struct rte_cryptodev' instead.
That way we avoid any ABI breakage without introducing any churn in rte_cryptodev itself , see above.
Konstantin
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH] add ABI checks
2020-01-16 11:52 4% ` Neil Horman
@ 2020-01-16 14:20 4% ` Thomas Monjalon
2020-01-16 18:49 4% ` Neil Horman
0 siblings, 1 reply; 200+ results
From: Thomas Monjalon @ 2020-01-16 14:20 UTC (permalink / raw)
To: Neil Horman
Cc: David Marchand, Richardson, Bruce, dev, Laatz, Kevin, aconole,
Michael Santana, Mcnamara, John, Kovacevic, Marko, Kinsella, Ray
16/01/2020 12:52, Neil Horman:
> On Wed, Jan 15, 2020 at 01:38:17PM +0100, Thomas Monjalon wrote:
> > 15/01/2020 12:33, Neil Horman:
> > > On Wed, Jan 15, 2020 at 12:19:30AM +0100, Thomas Monjalon wrote:
> > > > 20/12/2019 17:20, Kinsella, Ray:
> > > > > From: Richardson, Bruce <bruce.richardson@intel.com>
> > > > > > From: David Marchand <david.marchand@redhat.com>
> > > > > > > +Checking ABI compatibility
> > > > > > > +--------------------------
> > > > > > > +
> > > > > > > +The first thing is to build reference binaries for the latest
> > > > > > release
> > > > > > > +your patches are built on top of.
> > > > > > > +
> > > > > > > +Either you are in a git tree and an easy way to identify this is to
> > > > > > run::
> > > > > > > +
> > > > > > > + git checkout $(git describe --abbrev=0)
> > > > > > > +
> > > > > > > +Or you use a tarball and you extract the sources in a director of
> > > > > > > +your
> > > > > > > choice.
> > > > > > > +
> > > > > > > +Next is building those sources, refer to the previous paragraph.
> > > > > > > +You can set ``DPDK_BUILD_TEST_DIR=reference``, so that the builds
> > > > > > > +occur in this directory.
> > > > > > > +
> > > > > > > +Finally, the ABI dump files are generated with the
> > > > > > > +``devtools/gen-abi-reference.sh`` script. This script will look for
> > > > > > > +builds in the current sub directory ``reference``. But you can set
> > > > > > > +the environment variable ``DPDK_ABI_REF_BUILD_DIR`` to a different
> > > > > > location.
> > > > > > > +
> > > > > > > +Once done, you can check your current binaries ABI with this
> > > > > > > +reference with the ``devtools/check-abi-reference.sh`` script.
> > > > > > >
> > > > > >
> > > > > > I still very much dislike forcing the user to generate his own
> > > > > > reference version to compare the ABI against. These should be archived
> > > > > > and the user should just be able to pull them down via git or http or
> > > > > > otherwise. Two reasons for this:
> > > > > >
> > > > > > 1. Less error prone, since there is no chance of the user having an
> > > > > > incorrect build for whatever reason.
> > > > > >
> > > > > > 2. Less effort for the user than asking them to do extra builds. The
> > > > > > more steps the user has to follow, the less likely they are to attempt
> > > > > > the process.
> > > > >
> > > > > +1 ... 100% agree with this.
> > > > >
> > > > > Many people won't know or understand what the reference is,
> > > > > or why they to generate it.
> > > >
> > > > I don't want to generate and save the reference in git for each arch.
> > > >
> > > Can I ask what your reluctance is? Is it related to not wanting to have to save
> > > all this information that is otherwise not used for building purposes?
> >
> > Yes I prefer keeping only the sources in the repository.
> > And these dumps are big.
> > And last but not the least, there is no ready-to-use environment to build
> > and dump all libs for all archs.
> >
> > > If so I might suggest saving the dumps in a separate git tree and pulling them
> > > in as a git submodule when the check is performed
> > >
> > > I really like the idea of caching the results so everyone is working from a
> > > known ABI baseline.
> >
> > You don't trust the result of the build made from tagged sources?
> >
> I trust the result from the tools, sure, its trusting that people will take the
> extra time to build a version from a prior tag that I'm less sure of.
> Consistent use in my mind is predicated on ease and timeliness of use.
>
> I get not wanting to store large dumps in the source tree, but storage is cheap,
> and I don't see the issue with storing the xml dump in a separate git tree to be
> referenced through a git submodule that gets pulled in when the check is run.
Yes this is an option.
My fear is that this reference database will not be complete
if we don't build it for all libraries/drivers on all archs,
managing setups and dependencies.
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [dpdk-stable] [PATCH] meter: move RFC4115 trTCM APIs as none experimental
2020-01-16 12:42 0% ` Ferruh Yigit
@ 2020-01-16 14:38 0% ` Thomas Monjalon
2020-01-17 8:27 0% ` David Marchand
2020-01-22 13:55 0% ` Ray Kinsella
2020-01-30 16:33 4% ` Ray Kinsella
2 siblings, 1 reply; 200+ results
From: Thomas Monjalon @ 2020-01-16 14:38 UTC (permalink / raw)
To: Neil Horman, David Marchand, Ferruh Yigit
Cc: Ray Kinsella, Cristian Dumitrescu, dpdk stable, dev,
Eelco Chaudron, Kevin Traynor, Ian Stokes, Ilya Maximets,
Luca Boccassi
16/01/2020 13:42, Ferruh Yigit:
> On 1/16/2020 11:54 AM, Neil Horman wrote:
> > On Thu, Jan 16, 2020 at 12:25:06PM +0100, David Marchand wrote:
> >> On Tue, Dec 17, 2019 at 2:08 PM Eelco Chaudron <echaudro@redhat.com> wrote:
> >>>
> >>> Moved RFC4115 APIs to none experimental as they have been there
> >>> since 19.02. Also, these APIs are the same as the none RFC4115 APIs.
> >>>
> >>> Signed-off-by: Eelco Chaudron <echaudro@redhat.com>
> >>
> >> There is a discussion on the OVS ml at the moment to get these symbols
> >> in the stable ABI for 19.11.
> >> I want to understand how this would be done.
> >>
> >> - I take this patch in 20.02, these symbols are added in the 20.0.1 ABI.
> >> On the other hand, the 19.11 release maintains the 20.0 ABI.
> >>
> >> Does it mean the backport adds these symbols with the 20.0 version in
> >> the 19.11 branch?
> >> Or is 20.0.1 version acceptable / a thing we want?
We cannot have the symbol with different versions in different releases,
otherwise the compatibility is broken when upgrading.
So we have no choice, the symbol must have version 20.0.1
in release 19.11.1, as in release 20.02.
> >> - These symbol already existed in the 20.0 ABI, versioned as EXPERIMENTAL.
> >> We can go and remove these entries since we are not bound to preserve
> >> the experimental APIs.
> >> But, on the other hand, nothing should prevent us from keeping some
> >> aliases so that the symbols versioned EXPERIMENTAL are still available
> >> to existing users.
> >>
> > I would say that choice is up to you. If you want to alias them to be nice to
> > prior users, thats fine by me. But experimental means experimental, and so users
> > have to be prepared to rebuild when things change, even if that change is
> > changing the version from experimental to a concrete version.
> >
>
> I would prefer to keep the alias and don't break the existing users, specially
> for the case experimental API is becoming mature without change.
I agree, it's cool to be nice :)
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] DPDK Release Status Meeting 16/01/2020
2020-01-16 13:39 4% ` Ananyev, Konstantin
@ 2020-01-16 16:42 4% ` Akhil Goyal
2020-01-16 17:47 3% ` Ferruh Yigit
0 siblings, 1 reply; 200+ results
From: Akhil Goyal @ 2020-01-16 16:42 UTC (permalink / raw)
To: Ananyev, Konstantin, Yigit, Ferruh, dpdk-dev; +Cc: Thomas Monjalon
Hi Konstantin,
> Hi Akhil,
>
> > > >
> > > > * next-net-crypto
> > > > * Pull request sent
> > > > * There is a performance concern on some ipsec-gw patches,
> > > > they can go in -rc2 if the issue is solved
> > > > * CPU crypto from last release may be breaking ABI, need to confirm
> > >
> > > AFAIK, there is no ABI breakage.
> >
> > This is the output of validate-abi.sh.
> >
> > Change Effect
> > 1 Field sym_cpu_process has been added to this type. 1) This field will
> not be initialized by old clients.
> > 2) Size of the
> inclusive type has been changed.
> >
> > NOTE: this
> field should be accessed only from the new library
> > functions, otherwise it may result in crash or incorrect behavior of applications.
> > 2 Size of this type has been changed from 128 bytes to 136 bytes. The
> fields or parameters of such data type may be incorrectly
> > initialized or accessed by old client applications.
>
> This is struct rte_cryptodev_ops, which is AFAIK, not part of public API.
> So not sure, why do you consider it as ABI breakage?
>
If this is not an issue, than I am fine with it.
> >
> > Apart from that, IPSEC also has breakage, but that is experimental, so not an
> issue.
> >
> > >
> > > > and discussed dummy variable is missing, may be postponed to next
> release
> > >
> > > Not sure I understand last sentence, could the author explain
> > > what dummy variables we are talking about.
> >
> > In one of the techboard meeting around 19.11 timeframe, during the
> discussion for approving methodology for CPU-crypto, it was
> > proposed that in order to avoid delay, a dummy variable can be introduced in
> cryptodev API/ABI to avoid any ABI breakage in
> > upcoming releases. But this was not done.
>
> Dummy variable for what?
> If you are talking about sym_cpu_process - we just added it into
> rte_cryptodev_ops, instead of
> ' struct rte_cryptodev' instead.
> That way we avoid any ABI breakage without introducing any churn in
> rte_cryptodev itself , see above.
Then why was there so much resistance on this approach when there is no ABI breakage.
I thought there was ABI breakage because of this change.
BTW this patchset is a bit late and it came after merge deadline 15 Jan.
Ideally all library related patches should go in RC1.
I would check if I could make it to the RC2.
I have 3 IPSec series to work on before RC2.
-Akhil
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH v7 00/10] net/i40e: ESP support
2020-01-16 12:44 3% ` [dpdk-dev] [PATCH v7 00/10] " Bernard Iremonger
@ 2020-01-16 17:42 0% ` Ferruh Yigit
2020-01-21 16:10 0% ` Iremonger, Bernard
0 siblings, 1 reply; 200+ results
From: Ferruh Yigit @ 2020-01-16 17:42 UTC (permalink / raw)
To: Bernard Iremonger, dev, beilei.xing, qi.z.zhang, declan.doherty
Cc: konstantin.ananyev, stephen1.byrne, helin.zhang
On 1/16/2020 12:44 PM, Bernard Iremonger wrote:
> Add support for ESP flows to testpmd.
> Improve debug information in testpmd and the i40e PMD.
> Process ESP flows on the i40e Flow Director.
>
> Changes in V7:
> -------------
> Dropped the following debug patch from v6 patchset
> 0003-app-testpmd-dump-Rx-and-Tx-mbuf.patch
> Included release notes and doc patches in feature patches.
>
> Changes in V6:
> -------------
> Rebase to latest master branch
> Fixed patch 0001-app-testpmd-parse-flow-command-line-for-ESP.patch
> Dropped the following v5 patch due to ABI breakage:
> 0012-librte_ethdev-add-ESP-and-AH-flow-types-to-RSS.patch
>
> Changes in V5:
> -------------
> Rebase to latest master branch
> Update 0012-librte_ethdev-add-ESP-and-AH-flow-types-to-RSS.patch
>
> Changes in V4:
> -------------
> Add extra patch for testpmd
> Add extra patch for config
> Split v3 patch "net/i40e: process ESP flows" into 4 patches
>
> Changes in V3:
> -------------
> Added i40e_flow_set_filter_spi() function in i40e_flow.c
> Set UDP destination port to 4500 for ESP in i40e_ethdev.h
> Split flow structures into 4 instead of 2 in i40e_ethdev.h
> Dropped extra printf from commandline_flow.c
>
> Changes in V2:
> --------------
> Moved change in app/test-pmd/config.c to a seperate patch.
> Added extra parameter to fill_ip6_head() in i40e_fdir.c
> set is_udp to false in i40e_flow_fdir_get_pctype_value() in i40e_flow.c
> Bernard Iremonger (10):
> app/testpmd: parse flow command line for ESP
> app/testpmd: improve debug
> net/i40e: improve RSS debug
> net/i40e: handle ESP tunnel
> net/i40e: support ipsec-ah profile
> net/i40e: support ESP in customized code
> net/i40e: support ESP flows
> net/i40e: support ESP in Flow Director
> config: add debug to I40E Flow Director
> net/i40e: display Flow Director packet
Series applied to dpdk-next-net/master, thanks.
(Except 9/10 & 10/10, they are dropped while merging.)
Series wasn't rebased on next-net, I had to resolve conflict almost in all
patches, can you please validate the result in next-net?
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] DPDK Release Status Meeting 16/01/2020
2020-01-16 16:42 4% ` Akhil Goyal
@ 2020-01-16 17:47 3% ` Ferruh Yigit
0 siblings, 0 replies; 200+ results
From: Ferruh Yigit @ 2020-01-16 17:47 UTC (permalink / raw)
To: Akhil Goyal, Ananyev, Konstantin, dpdk-dev; +Cc: Thomas Monjalon
On 1/16/2020 4:42 PM, Akhil Goyal wrote:
> Hi Konstantin,
>
>> Hi Akhil,
>>
>>>>>
>>>>> * next-net-crypto
>>>>> * Pull request sent
>>>>> * There is a performance concern on some ipsec-gw patches,
>>>>> they can go in -rc2 if the issue is solved
>>>>> * CPU crypto from last release may be breaking ABI, need to confirm
>>>>
>>>> AFAIK, there is no ABI breakage.
>>>
>>> This is the output of validate-abi.sh.
>>>
>>> Change Effect
>>> 1 Field sym_cpu_process has been added to this type. 1) This field will
>> not be initialized by old clients.
>>> 2) Size of the
>> inclusive type has been changed.
>>>
>>> NOTE: this
>> field should be accessed only from the new library
>>> functions, otherwise it may result in crash or incorrect behavior of applications.
>>> 2 Size of this type has been changed from 128 bytes to 136 bytes. The
>> fields or parameters of such data type may be incorrectly
>>> initialized or accessed by old client applications.
>>
>> This is struct rte_cryptodev_ops, which is AFAIK, not part of public API.
>> So not sure, why do you consider it as ABI breakage?
>>
>
> If this is not an issue, than I am fine with it.
The ABI change between cryptodev and PMDs are allowed, that is contained within
DPDK and not a user interface [1].
[1] Unless some inline functions are directly accessing the dev_ops, as
(unfortunately) done in the ethdev.
>
>>>
>>> Apart from that, IPSEC also has breakage, but that is experimental, so not an
>> issue.
>>>
>>>>
>>>>> and discussed dummy variable is missing, may be postponed to next
>> release
>>>>
>>>> Not sure I understand last sentence, could the author explain
>>>> what dummy variables we are talking about.
>>>
>>> In one of the techboard meeting around 19.11 timeframe, during the
>> discussion for approving methodology for CPU-crypto, it was
>>> proposed that in order to avoid delay, a dummy variable can be introduced in
>> cryptodev API/ABI to avoid any ABI breakage in
>>> upcoming releases. But this was not done.
>>
>> Dummy variable for what?
>> If you are talking about sym_cpu_process - we just added it into
>> rte_cryptodev_ops, instead of
>> ' struct rte_cryptodev' instead.
>> That way we avoid any ABI breakage without introducing any churn in
>> rte_cryptodev itself , see above.
>
> Then why was there so much resistance on this approach when there is no ABI breakage.
> I thought there was ABI breakage because of this change.
>
> BTW this patchset is a bit late and it came after merge deadline 15 Jan.
> Ideally all library related patches should go in RC1.
> I would check if I could make it to the RC2.
> I have 3 IPSec series to work on before RC2.
>
> -Akhil
>
^ permalink raw reply [relevance 3%]
* Re: [dpdk-dev] [PATCH] add ABI checks
2020-01-16 14:20 4% ` Thomas Monjalon
@ 2020-01-16 18:49 4% ` Neil Horman
2020-01-16 20:01 4% ` Thomas Monjalon
0 siblings, 1 reply; 200+ results
From: Neil Horman @ 2020-01-16 18:49 UTC (permalink / raw)
To: Thomas Monjalon
Cc: David Marchand, Richardson, Bruce, dev, Laatz, Kevin, aconole,
Michael Santana, Mcnamara, John, Kovacevic, Marko, Kinsella, Ray
On Thu, Jan 16, 2020 at 03:20:48PM +0100, Thomas Monjalon wrote:
> 16/01/2020 12:52, Neil Horman:
> > On Wed, Jan 15, 2020 at 01:38:17PM +0100, Thomas Monjalon wrote:
> > > 15/01/2020 12:33, Neil Horman:
> > > > On Wed, Jan 15, 2020 at 12:19:30AM +0100, Thomas Monjalon wrote:
> > > > > 20/12/2019 17:20, Kinsella, Ray:
> > > > > > From: Richardson, Bruce <bruce.richardson@intel.com>
> > > > > > > From: David Marchand <david.marchand@redhat.com>
> > > > > > > > +Checking ABI compatibility
> > > > > > > > +--------------------------
> > > > > > > > +
> > > > > > > > +The first thing is to build reference binaries for the latest
> > > > > > > release
> > > > > > > > +your patches are built on top of.
> > > > > > > > +
> > > > > > > > +Either you are in a git tree and an easy way to identify this is to
> > > > > > > run::
> > > > > > > > +
> > > > > > > > + git checkout $(git describe --abbrev=0)
> > > > > > > > +
> > > > > > > > +Or you use a tarball and you extract the sources in a director of
> > > > > > > > +your
> > > > > > > > choice.
> > > > > > > > +
> > > > > > > > +Next is building those sources, refer to the previous paragraph.
> > > > > > > > +You can set ``DPDK_BUILD_TEST_DIR=reference``, so that the builds
> > > > > > > > +occur in this directory.
> > > > > > > > +
> > > > > > > > +Finally, the ABI dump files are generated with the
> > > > > > > > +``devtools/gen-abi-reference.sh`` script. This script will look for
> > > > > > > > +builds in the current sub directory ``reference``. But you can set
> > > > > > > > +the environment variable ``DPDK_ABI_REF_BUILD_DIR`` to a different
> > > > > > > location.
> > > > > > > > +
> > > > > > > > +Once done, you can check your current binaries ABI with this
> > > > > > > > +reference with the ``devtools/check-abi-reference.sh`` script.
> > > > > > > >
> > > > > > >
> > > > > > > I still very much dislike forcing the user to generate his own
> > > > > > > reference version to compare the ABI against. These should be archived
> > > > > > > and the user should just be able to pull them down via git or http or
> > > > > > > otherwise. Two reasons for this:
> > > > > > >
> > > > > > > 1. Less error prone, since there is no chance of the user having an
> > > > > > > incorrect build for whatever reason.
> > > > > > >
> > > > > > > 2. Less effort for the user than asking them to do extra builds. The
> > > > > > > more steps the user has to follow, the less likely they are to attempt
> > > > > > > the process.
> > > > > >
> > > > > > +1 ... 100% agree with this.
> > > > > >
> > > > > > Many people won't know or understand what the reference is,
> > > > > > or why they to generate it.
> > > > >
> > > > > I don't want to generate and save the reference in git for each arch.
> > > > >
> > > > Can I ask what your reluctance is? Is it related to not wanting to have to save
> > > > all this information that is otherwise not used for building purposes?
> > >
> > > Yes I prefer keeping only the sources in the repository.
> > > And these dumps are big.
> > > And last but not the least, there is no ready-to-use environment to build
> > > and dump all libs for all archs.
> > >
> > > > If so I might suggest saving the dumps in a separate git tree and pulling them
> > > > in as a git submodule when the check is performed
> > > >
> > > > I really like the idea of caching the results so everyone is working from a
> > > > known ABI baseline.
> > >
> > > You don't trust the result of the build made from tagged sources?
> > >
> > I trust the result from the tools, sure, its trusting that people will take the
> > extra time to build a version from a prior tag that I'm less sure of.
> > Consistent use in my mind is predicated on ease and timeliness of use.
> >
> > I get not wanting to store large dumps in the source tree, but storage is cheap,
> > and I don't see the issue with storing the xml dump in a separate git tree to be
> > referenced through a git submodule that gets pulled in when the check is run.
>
> Yes this is an option.
> My fear is that this reference database will not be complete
> if we don't build it for all libraries/drivers on all archs,
> managing setups and dependencies.
>
I can understand that, but I would have assumed that we would have done all
config build for all supported arches as part of the CI for a release, from
which we could archive the results. Is that not the case?
Neil
>
>
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH] add ABI checks
2020-01-16 18:49 4% ` Neil Horman
@ 2020-01-16 20:01 4% ` Thomas Monjalon
2020-01-17 19:01 4% ` Neil Horman
0 siblings, 1 reply; 200+ results
From: Thomas Monjalon @ 2020-01-16 20:01 UTC (permalink / raw)
To: Neil Horman
Cc: David Marchand, Richardson, Bruce, dev, Laatz, Kevin, aconole,
Michael Santana, Mcnamara, John, Kovacevic, Marko, Kinsella, Ray
16/01/2020 19:49, Neil Horman:
> On Thu, Jan 16, 2020 at 03:20:48PM +0100, Thomas Monjalon wrote:
> > 16/01/2020 12:52, Neil Horman:
> > > On Wed, Jan 15, 2020 at 01:38:17PM +0100, Thomas Monjalon wrote:
> > > > 15/01/2020 12:33, Neil Horman:
> > > > > On Wed, Jan 15, 2020 at 12:19:30AM +0100, Thomas Monjalon wrote:
> > > > > > 20/12/2019 17:20, Kinsella, Ray:
> > > > > > > From: Richardson, Bruce <bruce.richardson@intel.com>
> > > > > > > > From: David Marchand <david.marchand@redhat.com>
> > > > > > > > > +Checking ABI compatibility
> > > > > > > > > +--------------------------
> > > > > > > > > +
> > > > > > > > > +The first thing is to build reference binaries for the latest
> > > > > > > > release
> > > > > > > > > +your patches are built on top of.
> > > > > > > > > +
> > > > > > > > > +Either you are in a git tree and an easy way to identify this is to
> > > > > > > > run::
> > > > > > > > > +
> > > > > > > > > + git checkout $(git describe --abbrev=0)
> > > > > > > > > +
> > > > > > > > > +Or you use a tarball and you extract the sources in a director of
> > > > > > > > > +your
> > > > > > > > > choice.
> > > > > > > > > +
> > > > > > > > > +Next is building those sources, refer to the previous paragraph.
> > > > > > > > > +You can set ``DPDK_BUILD_TEST_DIR=reference``, so that the builds
> > > > > > > > > +occur in this directory.
> > > > > > > > > +
> > > > > > > > > +Finally, the ABI dump files are generated with the
> > > > > > > > > +``devtools/gen-abi-reference.sh`` script. This script will look for
> > > > > > > > > +builds in the current sub directory ``reference``. But you can set
> > > > > > > > > +the environment variable ``DPDK_ABI_REF_BUILD_DIR`` to a different
> > > > > > > > location.
> > > > > > > > > +
> > > > > > > > > +Once done, you can check your current binaries ABI with this
> > > > > > > > > +reference with the ``devtools/check-abi-reference.sh`` script.
> > > > > > > > >
> > > > > > > >
> > > > > > > > I still very much dislike forcing the user to generate his own
> > > > > > > > reference version to compare the ABI against. These should be archived
> > > > > > > > and the user should just be able to pull them down via git or http or
> > > > > > > > otherwise. Two reasons for this:
> > > > > > > >
> > > > > > > > 1. Less error prone, since there is no chance of the user having an
> > > > > > > > incorrect build for whatever reason.
> > > > > > > >
> > > > > > > > 2. Less effort for the user than asking them to do extra builds. The
> > > > > > > > more steps the user has to follow, the less likely they are to attempt
> > > > > > > > the process.
> > > > > > >
> > > > > > > +1 ... 100% agree with this.
> > > > > > >
> > > > > > > Many people won't know or understand what the reference is,
> > > > > > > or why they to generate it.
> > > > > >
> > > > > > I don't want to generate and save the reference in git for each arch.
> > > > > >
> > > > > Can I ask what your reluctance is? Is it related to not wanting to have to save
> > > > > all this information that is otherwise not used for building purposes?
> > > >
> > > > Yes I prefer keeping only the sources in the repository.
> > > > And these dumps are big.
> > > > And last but not the least, there is no ready-to-use environment to build
> > > > and dump all libs for all archs.
> > > >
> > > > > If so I might suggest saving the dumps in a separate git tree and pulling them
> > > > > in as a git submodule when the check is performed
> > > > >
> > > > > I really like the idea of caching the results so everyone is working from a
> > > > > known ABI baseline.
> > > >
> > > > You don't trust the result of the build made from tagged sources?
> > > >
> > > I trust the result from the tools, sure, its trusting that people will take the
> > > extra time to build a version from a prior tag that I'm less sure of.
> > > Consistent use in my mind is predicated on ease and timeliness of use.
> > >
> > > I get not wanting to store large dumps in the source tree, but storage is cheap,
> > > and I don't see the issue with storing the xml dump in a separate git tree to be
> > > referenced through a git submodule that gets pulled in when the check is run.
> >
> > Yes this is an option.
> > My fear is that this reference database will not be complete
> > if we don't build it for all libraries/drivers on all archs,
> > managing setups and dependencies.
> >
> I can understand that, but I would have assumed that we would have done all
> config build for all supported arches as part of the CI for a release, from
> which we could archive the results. Is that not the case?
No, we don't have a fully complete setup covering all dependencies and archs.
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [dpdk-stable] [PATCH] meter: move RFC4115 trTCM APIs as none experimental
2020-01-16 14:38 0% ` Thomas Monjalon
@ 2020-01-17 8:27 0% ` David Marchand
2020-01-17 10:46 0% ` Luca Boccassi
0 siblings, 1 reply; 200+ results
From: David Marchand @ 2020-01-17 8:27 UTC (permalink / raw)
To: Thomas Monjalon, Luca Boccassi
Cc: Neil Horman, Ferruh Yigit, Ray Kinsella, Cristian Dumitrescu,
dpdk stable, dev, Eelco Chaudron, Kevin Traynor, Ian Stokes,
Ilya Maximets
On Thu, Jan 16, 2020 at 3:38 PM Thomas Monjalon <thomas@monjalon.net> wrote:
>
> 16/01/2020 13:42, Ferruh Yigit:
> > On 1/16/2020 11:54 AM, Neil Horman wrote:
> > > On Thu, Jan 16, 2020 at 12:25:06PM +0100, David Marchand wrote:
> > >> On Tue, Dec 17, 2019 at 2:08 PM Eelco Chaudron <echaudro@redhat.com> wrote:
> > >>>
> > >>> Moved RFC4115 APIs to none experimental as they have been there
> > >>> since 19.02. Also, these APIs are the same as the none RFC4115 APIs.
> > >>>
> > >>> Signed-off-by: Eelco Chaudron <echaudro@redhat.com>
> > >>
> > >> There is a discussion on the OVS ml at the moment to get these symbols
> > >> in the stable ABI for 19.11.
> > >> I want to understand how this would be done.
> > >>
> > >> - I take this patch in 20.02, these symbols are added in the 20.0.1 ABI.
> > >> On the other hand, the 19.11 release maintains the 20.0 ABI.
> > >>
> > >> Does it mean the backport adds these symbols with the 20.0 version in
> > >> the 19.11 branch?
> > >> Or is 20.0.1 version acceptable / a thing we want?
>
> We cannot have the symbol with different versions in different releases,
> otherwise the compatibility is broken when upgrading.
> So we have no choice, the symbol must have version 20.0.1
> in release 19.11.1, as in release 20.02.
Indeed, good point.
> > >> - These symbol already existed in the 20.0 ABI, versioned as EXPERIMENTAL.
> > >> We can go and remove these entries since we are not bound to preserve
> > >> the experimental APIs.
> > >> But, on the other hand, nothing should prevent us from keeping some
> > >> aliases so that the symbols versioned EXPERIMENTAL are still available
> > >> to existing users.
> > >>
> > > I would say that choice is up to you. If you want to alias them to be nice to
> > > prior users, thats fine by me. But experimental means experimental, and so users
> > > have to be prepared to rebuild when things change, even if that change is
> > > changing the version from experimental to a concrete version.
> > >
> >
> > I would prefer to keep the alias and don't break the existing users, specially
> > for the case experimental API is becoming mature without change.
>
> I agree, it's cool to be nice :)
Luca, opinion?
--
David Marchand
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [dpdk-stable] [PATCH] meter: move RFC4115 trTCM APIs as none experimental
2020-01-17 8:27 0% ` David Marchand
@ 2020-01-17 10:46 0% ` Luca Boccassi
0 siblings, 0 replies; 200+ results
From: Luca Boccassi @ 2020-01-17 10:46 UTC (permalink / raw)
To: David Marchand, Thomas Monjalon
Cc: Neil Horman, Ferruh Yigit, Ray Kinsella, Cristian Dumitrescu,
dpdk stable, dev, Eelco Chaudron, Kevin Traynor, Ian Stokes,
Ilya Maximets
On Fri, 2020-01-17 at 09:27 +0100, David Marchand wrote:
> On Thu, Jan 16, 2020 at 3:38 PM Thomas Monjalon <
> thomas@monjalon.net
> > wrote:
> > 16/01/2020 13:42, Ferruh Yigit:
> > > On 1/16/2020 11:54 AM, Neil Horman wrote:
> > > > On Thu, Jan 16, 2020 at 12:25:06PM +0100, David Marchand wrote:
> > > > > On Tue, Dec 17, 2019 at 2:08 PM Eelco Chaudron <
> > > > > echaudro@redhat.com
> > > > > > wrote:
> > > > > > Moved RFC4115 APIs to none experimental as they have been
> > > > > > there
> > > > > > since 19.02. Also, these APIs are the same as the none
> > > > > > RFC4115 APIs.
> > > > > >
> > > > > > Signed-off-by: Eelco Chaudron <
> > > > > > echaudro@redhat.com
> > > > > > >
> > > > >
> > > > > There is a discussion on the OVS ml at the moment to get
> > > > > these symbols
> > > > > in the stable ABI for 19.11.
> > > > > I want to understand how this would be done.
> > > > >
> > > > > - I take this patch in 20.02, these symbols are added in the
> > > > > 20.0.1 ABI.
> > > > > On the other hand, the 19.11 release maintains the 20.0 ABI.
> > > > >
> > > > > Does it mean the backport adds these symbols with the 20.0
> > > > > version in
> > > > > the 19.11 branch?
> > > > > Or is 20.0.1 version acceptable / a thing we want?
> >
> > We cannot have the symbol with different versions in different
> > releases,
> > otherwise the compatibility is broken when upgrading.
> > So we have no choice, the symbol must have version 20.0.1
> > in release 19.11.1, as in release 20.02.
>
> Indeed, good point.
>
>
> > > > > - These symbol already existed in the 20.0 ABI, versioned as
> > > > > EXPERIMENTAL.
> > > > > We can go and remove these entries since we are not bound to
> > > > > preserve
> > > > > the experimental APIs.
> > > > > But, on the other hand, nothing should prevent us from
> > > > > keeping some
> > > > > aliases so that the symbols versioned EXPERIMENTAL are still
> > > > > available
> > > > > to existing users.
> > > > >
> > > >
> > > > I would say that choice is up to you. If you want to alias
> > > > them to be nice to
> > > > prior users, thats fine by me. But experimental means
> > > > experimental, and so users
> > > > have to be prepared to rebuild when things change, even if that
> > > > change is
> > > > changing the version from experimental to a concrete version.
> > > >
> > >
> > > I would prefer to keep the alias and don't break the existing
> > > users, specially
> > > for the case experimental API is becoming mature without change.
> >
> > I agree, it's cool to be nice :)
>
> Luca, opinion?
I'd not only prefer it but also go as far as require it for backporting
to 19.11 - experimental means experimental, but stable means stable :-)
--
Kind regards,
Luca Boccassi
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] DPDK Release Status Meeting 16/01/2020
@ 2020-01-17 11:39 0% Ananyev, Konstantin
0 siblings, 0 replies; 200+ results
From: Ananyev, Konstantin @ 2020-01-17 11:39 UTC (permalink / raw)
To: Yigit, Ferruh, Akhil Goyal, dpdk-dev; +Cc: Thomas Monjalon
Hi lads,
> >>>>> * next-net-crypto
> >>>>> * Pull request sent
> >>>>> * There is a performance concern on some ipsec-gw patches,
> >>>>> they can go in -rc2 if the issue is solved
> >>>>> * CPU crypto from last release may be breaking ABI, need to confirm
> >>>>
> >>>> AFAIK, there is no ABI breakage.
> >>>
> >>> This is the output of validate-abi.sh.
> >>>
> >>> Change Effect
> >>> 1 Field sym_cpu_process has been added to this type. 1) This field will
> >> not be initialized by old clients.
> >>> 2) Size of the
> >> inclusive type has been changed.
> >>>
> >>> NOTE: this
> >> field should be accessed only from the new library
> >>> functions, otherwise it may result in crash or incorrect behavior of applications.
> >>> 2 Size of this type has been changed from 128 bytes to 136 bytes. The
> >> fields or parameters of such data type may be incorrectly
> >>> initialized or accessed by old client applications.
> >>
> >> This is struct rte_cryptodev_ops, which is AFAIK, not part of public API.
> >> So not sure, why do you consider it as ABI breakage?
> >>
> >
> > If this is not an issue, than I am fine with it.
>
> The ABI change between cryptodev and PMDs are allowed, that is contained within
> DPDK and not a user interface [1].
>
> [1] Unless some inline functions are directly accessing the dev_ops, as
> (unfortunately) done in the ethdev.
Thanks Ferruh for confirmation.
For cryptodev we don't have such inline functions, plus the we add
new entry at the bottom of struct rte_cryptodev_ops, so I believe we
are safe here.
>
> >
> >>>
> >>> Apart from that, IPSEC also has breakage, but that is experimental, so not an
> >> issue.
> >>>
> >>>>
> >>>>> and discussed dummy variable is missing, may be postponed to next
> >> release
> >>>>
> >>>> Not sure I understand last sentence, could the author explain
> >>>> what dummy variables we are talking about.
> >>>
> >>> In one of the techboard meeting around 19.11 timeframe, during the
> >> discussion for approving methodology for CPU-crypto, it was
> >>> proposed that in order to avoid delay, a dummy variable can be introduced in
> >> cryptodev API/ABI to avoid any ABI breakage in
> >>> upcoming releases. But this was not done.
> >>
> >> Dummy variable for what?
> >> If you are talking about sym_cpu_process - we just added it into
> >> rte_cryptodev_ops, instead of
> >> ' struct rte_cryptodev' instead.
> >> That way we avoid any ABI breakage without introducing any churn in
> >> rte_cryptodev itself , see above.
> >
> > Then why was there so much resistance on this approach when there is no ABI breakage.
> > I thought there was ABI breakage because of this change.
> >
> > BTW this patchset is a bit late and it came after merge deadline 15 Jan.
> > Ideally all library related patches should go in RC1.
> > I would check if I could make it to the RC2.
> > I have 3 IPSec series to work on before RC2.
> >
^ permalink raw reply [relevance 0%]
* [dpdk-dev] [PATCH v2 0/3] mempool: fix mempool virt populate with small chunks
@ 2020-01-17 14:57 5% ` Olivier Matz
2020-01-17 14:57 3% ` [dpdk-dev] [PATCH v2 1/3] " Olivier Matz
` (3 more replies)
0 siblings, 4 replies; 200+ results
From: Olivier Matz @ 2020-01-17 14:57 UTC (permalink / raw)
To: dev
Cc: Zhang, AlvinX, Burakov, Anatoly, Andrew Rybchenko,
Bruce Richardson, David Marchand, dpdk stable
rte_mempool_populate_virt() sometimes fail, when it calls
rte_mempool_populate_iova() with an area which is too small to store one
object. This should not be an error.
I prepared a v2 which implements an ABI compatibility through symbol
versioning, as suggested [1]. It looks a bit overkill to me, but it
was an interresting exercice.
v2 changes:
- The initial requirement is to fix an issue at mempool creation. As
the fix will probably be backported in 19.11.x, the first patch
in the patchset does not break API/ABI.
- Using symbol versioning helps to preserve ABI, but for the API
the breakage has to be announced 1 release in advance, so there
is a separate patch for this.
- There is a 3rd patch for 20.05 that makes the new API public and
implements ABI versioning (if ok, I'll remove it from the patchset in
next version and send it separately)
- It appears that returning -ENOBUFS instead of -EINVAL is not ideal
because, in theory, mempool_ops_alloc_once() could also return
-ENOBUFS, and it would be forwarded to the caller by
rte_mempool_populate_iova() too, and misinterpreted as "there is not
enough room".
Returning 0 instead of -ENOBUFS was initially suggested by Anatoly,
and it does not suffer from this problem. It is doable if we properly
document that the memory chunk is not added to the mempool when
returning 0. It has an impact on populate_virt(), which has to be
versioned too.
There are some checkpatch warnings, but I'm not sure how if I
should solve them:
- it complains about forward declaration in .c, but without it, it does
not compile due to additional warnings in cflags
- it complains that modified symbols should be marked as experimental
Thanks to David for helping me to test and fix the ABI part of the
patchset.
[1] http://patchwork.dpdk.org/patch/64369/
Olivier Matz (3):
mempool: fix mempool virt populate with small chunks
doc: announce API change for mempool IOVA populate
mempool: return 0 if area is too small on populate
doc/guides/rel_notes/deprecation.rst | 5 ++
examples/ntb/ntb_fwd.c | 2 +-
lib/librte_mempool/meson.build | 1 +
lib/librte_mempool/rte_mempool.c | 88 ++++++++++++++++++++--
lib/librte_mempool/rte_mempool.h | 14 +++-
lib/librte_mempool/rte_mempool_version.map | 7 ++
6 files changed, 105 insertions(+), 12 deletions(-)
--
2.20.1
^ permalink raw reply [relevance 5%]
* [dpdk-dev] [PATCH v2 1/3] mempool: fix mempool virt populate with small chunks
2020-01-17 14:57 5% ` [dpdk-dev] [PATCH v2 0/3] " Olivier Matz
@ 2020-01-17 14:57 3% ` Olivier Matz
2020-01-17 14:57 8% ` [dpdk-dev] [PATCH v2 2/3] doc: announce API change for mempool IOVA populate Olivier Matz
` (2 subsequent siblings)
3 siblings, 0 replies; 200+ results
From: Olivier Matz @ 2020-01-17 14:57 UTC (permalink / raw)
To: dev
Cc: Zhang, AlvinX, Burakov, Anatoly, Andrew Rybchenko,
Bruce Richardson, David Marchand, dpdk stable
To populate a mempool with a virtual area, the mempool code calls
rte_mempool_populate_iova() for each iova-contiguous area. It happens
(rarely) that this area is too small to store one object. In this case,
rte_mempool_populate_iova() returns an error, which is forwarded by
rte_mempool_populate_virt().
This case should not throw an error in
rte_mempool_populate_virt(). Instead, the area that is too small should
just be ignored.
To fix this issue, change the return value of
rte_mempool_populate_iova() to 0 when no object can be populated,
so it can be ignored by the caller. As this would be an API/ABI change,
only do this modification internally for now.
Fixes: 354788b60cfd ("mempool: allow populating with unaligned virtual area")
Cc: stable@dpdk.org
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Tested-by: Anatoly Burakov <anatoly.burakov@intel.com>
Tested-by: Zhang Alvin <alvinx.zhang@intel.com>
---
lib/librte_mempool/rte_mempool.c | 30 +++++++++++++++++++++++++-----
1 file changed, 25 insertions(+), 5 deletions(-)
diff --git a/lib/librte_mempool/rte_mempool.c b/lib/librte_mempool/rte_mempool.c
index f8d453d21..b434d9f17 100644
--- a/lib/librte_mempool/rte_mempool.c
+++ b/lib/librte_mempool/rte_mempool.c
@@ -297,8 +297,8 @@ mempool_ops_alloc_once(struct rte_mempool *mp)
* zone. Return the number of objects added, or a negative value
* on error.
*/
-int
-rte_mempool_populate_iova(struct rte_mempool *mp, char *vaddr,
+static int
+__rte_mempool_populate_iova(struct rte_mempool *mp, char *vaddr,
rte_iova_t iova, size_t len, rte_mempool_memchunk_free_cb_t *free_cb,
void *opaque)
{
@@ -332,7 +332,7 @@ rte_mempool_populate_iova(struct rte_mempool *mp, char *vaddr,
off = RTE_PTR_ALIGN_CEIL(vaddr, RTE_MEMPOOL_ALIGN) - vaddr;
if (off > len) {
- ret = -EINVAL;
+ ret = 0;
goto fail;
}
@@ -343,7 +343,7 @@ rte_mempool_populate_iova(struct rte_mempool *mp, char *vaddr,
/* not enough room to store one object */
if (i == 0) {
- ret = -EINVAL;
+ ret = 0;
goto fail;
}
@@ -356,6 +356,21 @@ rte_mempool_populate_iova(struct rte_mempool *mp, char *vaddr,
return ret;
}
+int
+rte_mempool_populate_iova(struct rte_mempool *mp, char *vaddr,
+ rte_iova_t iova, size_t len, rte_mempool_memchunk_free_cb_t *free_cb,
+ void *opaque)
+{
+ int ret;
+
+ ret = __rte_mempool_populate_iova(mp, vaddr, iova, len, free_cb,
+ opaque);
+ if (ret == 0)
+ ret = -EINVAL;
+
+ return ret;
+}
+
static rte_iova_t
get_iova(void *addr)
{
@@ -406,8 +421,10 @@ rte_mempool_populate_virt(struct rte_mempool *mp, char *addr,
break;
}
- ret = rte_mempool_populate_iova(mp, addr + off, iova,
+ ret = __rte_mempool_populate_iova(mp, addr + off, iova,
phys_len, free_cb, opaque);
+ if (ret == 0)
+ continue;
if (ret < 0)
goto fail;
/* no need to call the free callback for next chunks */
@@ -415,6 +432,9 @@ rte_mempool_populate_virt(struct rte_mempool *mp, char *addr,
cnt += ret;
}
+ if (cnt == 0)
+ return -EINVAL;
+
return cnt;
fail:
--
2.20.1
^ permalink raw reply [relevance 3%]
* [dpdk-dev] [PATCH v2 3/3] [20.05] mempool: return 0 if area is too small on populate
2020-01-17 14:57 5% ` [dpdk-dev] [PATCH v2 0/3] " Olivier Matz
2020-01-17 14:57 3% ` [dpdk-dev] [PATCH v2 1/3] " Olivier Matz
2020-01-17 14:57 8% ` [dpdk-dev] [PATCH v2 2/3] doc: announce API change for mempool IOVA populate Olivier Matz
@ 2020-01-17 14:57 2% ` Olivier Matz
2020-01-17 20:32 0% ` David Marchand
2020-01-20 12:02 0% ` [dpdk-dev] [PATCH v2 0/3] mempool: fix mempool virt populate with small chunks Thomas Monjalon
3 siblings, 1 reply; 200+ results
From: Olivier Matz @ 2020-01-17 14:57 UTC (permalink / raw)
To: dev
Cc: Zhang, AlvinX, Burakov, Anatoly, Andrew Rybchenko,
Bruce Richardson, David Marchand, dpdk stable
Change rte_mempool_populate_iova() and rte_mempool_populate_iova() to
return 0 instead of -EINVAL when there is not enough room to store one
object, as it can be helpful for applications to distinguish this
specific case.
As this is an ABI change, use symbol versioning to preserve old
behavior for binary applications.
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
---
examples/ntb/ntb_fwd.c | 2 +-
lib/librte_mempool/meson.build | 1 +
lib/librte_mempool/rte_mempool.c | 76 ++++++++++++++++++----
lib/librte_mempool/rte_mempool.h | 14 ++--
lib/librte_mempool/rte_mempool_version.map | 7 ++
5 files changed, 84 insertions(+), 16 deletions(-)
diff --git a/examples/ntb/ntb_fwd.c b/examples/ntb/ntb_fwd.c
index c914256dd..99a43688c 100644
--- a/examples/ntb/ntb_fwd.c
+++ b/examples/ntb/ntb_fwd.c
@@ -1313,7 +1313,7 @@ ntb_mbuf_pool_create(uint16_t mbuf_seg_size, uint32_t nb_mbuf,
mz->len - ntb_info.ntb_hdr_size,
ntb_mempool_mz_free,
(void *)(uintptr_t)mz);
- if (ret < 0) {
+ if (ret <= 0) {
rte_memzone_free(mz);
rte_mempool_free(mp);
return NULL;
diff --git a/lib/librte_mempool/meson.build b/lib/librte_mempool/meson.build
index f8710b61b..4fe267cd8 100644
--- a/lib/librte_mempool/meson.build
+++ b/lib/librte_mempool/meson.build
@@ -18,3 +18,4 @@ deps += ['ring']
# memseg walk is not yet part of stable API
allow_experimental_apis = true
+use_function_versioning = true
diff --git a/lib/librte_mempool/rte_mempool.c b/lib/librte_mempool/rte_mempool.c
index b434d9f17..0ebb21eec 100644
--- a/lib/librte_mempool/rte_mempool.c
+++ b/lib/librte_mempool/rte_mempool.c
@@ -31,6 +31,7 @@
#include <rte_string_fns.h>
#include <rte_spinlock.h>
#include <rte_tailq.h>
+#include <rte_function_versioning.h>
#include "rte_mempool.h"
@@ -293,12 +294,17 @@ mempool_ops_alloc_once(struct rte_mempool *mp)
return 0;
}
+__vsym int
+rte_mempool_populate_iova_v20_0_1(struct rte_mempool *mp, char *vaddr,
+ rte_iova_t iova, size_t len, rte_mempool_memchunk_free_cb_t *free_cb,
+ void *opaque);
+
/* Add objects in the pool, using a physically contiguous memory
* zone. Return the number of objects added, or a negative value
* on error.
*/
-static int
-__rte_mempool_populate_iova(struct rte_mempool *mp, char *vaddr,
+__vsym int
+rte_mempool_populate_iova_v20_0_1(struct rte_mempool *mp, char *vaddr,
rte_iova_t iova, size_t len, rte_mempool_memchunk_free_cb_t *free_cb,
void *opaque)
{
@@ -355,21 +361,34 @@ __rte_mempool_populate_iova(struct rte_mempool *mp, char *vaddr,
rte_free(memhdr);
return ret;
}
+BIND_DEFAULT_SYMBOL(rte_mempool_populate_iova, _v20_0_1, 20.0.1);
+MAP_STATIC_SYMBOL(
+ int rte_mempool_populate_iova(struct rte_mempool *mp, char *vaddr,
+ rte_iova_t iova, size_t len,
+ rte_mempool_memchunk_free_cb_t *free_cb,
+ void *opaque),
+ rte_mempool_populate_iova_v20_0_1);
+
+__vsym int
+rte_mempool_populate_iova_v20_0(struct rte_mempool *mp, char *vaddr,
+ rte_iova_t iova, size_t len, rte_mempool_memchunk_free_cb_t *free_cb,
+ void *opaque);
-int
-rte_mempool_populate_iova(struct rte_mempool *mp, char *vaddr,
+__vsym int
+rte_mempool_populate_iova_v20_0(struct rte_mempool *mp, char *vaddr,
rte_iova_t iova, size_t len, rte_mempool_memchunk_free_cb_t *free_cb,
void *opaque)
{
int ret;
- ret = __rte_mempool_populate_iova(mp, vaddr, iova, len, free_cb,
+ ret = rte_mempool_populate_iova_v20_0_1(mp, vaddr, iova, len, free_cb,
opaque);
if (ret == 0)
ret = -EINVAL;
return ret;
}
+VERSION_SYMBOL(rte_mempool_populate_iova, _v20_0, 20.0);
static rte_iova_t
get_iova(void *addr)
@@ -384,11 +403,16 @@ get_iova(void *addr)
return ms->iova + RTE_PTR_DIFF(addr, ms->addr);
}
+__vsym int
+rte_mempool_populate_virt_v20_0_1(struct rte_mempool *mp, char *addr,
+ size_t len, size_t pg_sz, rte_mempool_memchunk_free_cb_t *free_cb,
+ void *opaque);
+
/* Populate the mempool with a virtual area. Return the number of
* objects added, or a negative value on error.
*/
-int
-rte_mempool_populate_virt(struct rte_mempool *mp, char *addr,
+__vsym int
+rte_mempool_populate_virt_v20_0_1(struct rte_mempool *mp, char *addr,
size_t len, size_t pg_sz, rte_mempool_memchunk_free_cb_t *free_cb,
void *opaque)
{
@@ -421,7 +445,7 @@ rte_mempool_populate_virt(struct rte_mempool *mp, char *addr,
break;
}
- ret = __rte_mempool_populate_iova(mp, addr + off, iova,
+ ret = rte_mempool_populate_iova_v20_0_1(mp, addr + off, iova,
phys_len, free_cb, opaque);
if (ret == 0)
continue;
@@ -432,15 +456,41 @@ rte_mempool_populate_virt(struct rte_mempool *mp, char *addr,
cnt += ret;
}
- if (cnt == 0)
- return -EINVAL;
-
return cnt;
fail:
rte_mempool_free_memchunks(mp);
return ret;
}
+BIND_DEFAULT_SYMBOL(rte_mempool_populate_virt, _v20_0_1, 20.0.1);
+MAP_STATIC_SYMBOL(
+ int rte_mempool_populate_virt(struct rte_mempool *mp,
+ char *addr, size_t len, size_t pg_sz,
+ rte_mempool_memchunk_free_cb_t *free_cb,
+ void *opaque),
+ rte_mempool_populate_virt_v20_0_1);
+
+__vsym int
+rte_mempool_populate_virt_v20_0(struct rte_mempool *mp, char *addr,
+ size_t len, size_t pg_sz, rte_mempool_memchunk_free_cb_t *free_cb,
+ void *opaque);
+
+__vsym int
+rte_mempool_populate_virt_v20_0(struct rte_mempool *mp, char *addr,
+ size_t len, size_t pg_sz, rte_mempool_memchunk_free_cb_t *free_cb,
+ void *opaque)
+{
+ int ret;
+
+ ret = rte_mempool_populate_virt_v20_0_1(mp, addr, len, pg_sz,
+ free_cb, opaque);
+
+ if (ret == 0)
+ ret = -EINVAL;
+
+ return ret;
+}
+VERSION_SYMBOL(rte_mempool_populate_virt, _v20_0, 20.0);
/* Get the minimal page size used in a mempool before populating it. */
int
@@ -601,6 +651,8 @@ rte_mempool_populate_default(struct rte_mempool *mp)
mz->len, pg_sz,
rte_mempool_memchunk_mz_free,
(void *)(uintptr_t)mz);
+ if (ret == 0) /* should not happen */
+ ret = -ENOBUFS;
if (ret < 0) {
rte_memzone_free(mz);
goto fail;
@@ -692,6 +744,8 @@ rte_mempool_populate_anon(struct rte_mempool *mp)
ret = rte_mempool_populate_virt(mp, addr, size, getpagesize(),
rte_mempool_memchunk_anon_free, addr);
+ if (ret == 0) /* should not happen */
+ ret = -ENOBUFS;
if (ret < 0) {
rte_errno = -ret;
goto fail;
diff --git a/lib/librte_mempool/rte_mempool.h b/lib/librte_mempool/rte_mempool.h
index 0a1dc6059..5a106a432 100644
--- a/lib/librte_mempool/rte_mempool.h
+++ b/lib/librte_mempool/rte_mempool.h
@@ -1106,9 +1106,12 @@ rte_mempool_free(struct rte_mempool *mp);
* @param opaque
* An opaque argument passed to free_cb.
* @return
- * The number of objects added on success.
+ * The number of objects added on success (strictly positive).
* On error, the chunk is not added in the memory list of the
- * mempool and a negative errno is returned.
+ * mempool the following code is returned:
+ * (0): not enough room in chunk for one object.
+ * (-ENOSPC): mempool is already populated.
+ * (-ENOMEM): allocation failure.
*/
int rte_mempool_populate_iova(struct rte_mempool *mp, char *vaddr,
rte_iova_t iova, size_t len, rte_mempool_memchunk_free_cb_t *free_cb,
@@ -1133,9 +1136,12 @@ int rte_mempool_populate_iova(struct rte_mempool *mp, char *vaddr,
* @param opaque
* An opaque argument passed to free_cb.
* @return
- * The number of objects added on success.
+ * The number of objects added on success (strictly positive).
* On error, the chunk is not added in the memory list of the
- * mempool and a negative errno is returned.
+ * mempool the following code is returned:
+ * (0): not enough room in chunk for one object.
+ * (-ENOSPC): mempool is already populated.
+ * (-ENOMEM): allocation failure.
*/
int
rte_mempool_populate_virt(struct rte_mempool *mp, char *addr,
diff --git a/lib/librte_mempool/rte_mempool_version.map b/lib/librte_mempool/rte_mempool_version.map
index d002dfc46..34d977e47 100644
--- a/lib/librte_mempool/rte_mempool_version.map
+++ b/lib/librte_mempool/rte_mempool_version.map
@@ -35,6 +35,13 @@ DPDK_20.0 {
local: *;
};
+DPDK_20.0.1 {
+ global:
+
+ rte_mempool_populate_iova;
+ rte_mempool_populate_virt;
+} DPDK_20.0;
+
EXPERIMENTAL {
global:
--
2.20.1
^ permalink raw reply [relevance 2%]
* [dpdk-dev] [PATCH v2 2/3] doc: announce API change for mempool IOVA populate
2020-01-17 14:57 5% ` [dpdk-dev] [PATCH v2 0/3] " Olivier Matz
2020-01-17 14:57 3% ` [dpdk-dev] [PATCH v2 1/3] " Olivier Matz
@ 2020-01-17 14:57 8% ` Olivier Matz
2020-01-17 20:32 0% ` David Marchand
2020-01-17 14:57 2% ` [dpdk-dev] [PATCH v2 3/3] [20.05] mempool: return 0 if area is too small on populate Olivier Matz
2020-01-20 12:02 0% ` [dpdk-dev] [PATCH v2 0/3] mempool: fix mempool virt populate with small chunks Thomas Monjalon
3 siblings, 1 reply; 200+ results
From: Olivier Matz @ 2020-01-17 14:57 UTC (permalink / raw)
To: dev
Cc: Zhang, AlvinX, Burakov, Anatoly, Andrew Rybchenko,
Bruce Richardson, David Marchand, dpdk stable
Starting from v20.05, rte_mempool_populate_iova() will return
-ENOBUFS. The ABI will be preserved through symbol versioning until
20.11.
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
---
doc/guides/rel_notes/deprecation.rst | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
index afa94b43e..e11c7f436 100644
--- a/doc/guides/rel_notes/deprecation.rst
+++ b/doc/guides/rel_notes/deprecation.rst
@@ -86,3 +86,8 @@ Deprecation Notices
to set new power environment if power environment was already initialized.
In this case the function will return -1 unless the environment is unset first
(using ``rte_power_unset_env``). Other function usage scenarios will not change.
+
+* mempool: starting from v20.05, the API of rte_mempool_populate_iova()
+ and rte_mempool_populate_virt() will change to return 0 instead
+ of -EINVAL when there is not enough room to store one object. The ABI
+ will be preserved until 20.11.
--
2.20.1
^ permalink raw reply [relevance 8%]
* Re: [dpdk-dev] DPDK Release Status Meeting 16/01/2020
2020-01-16 11:13 4% [dpdk-dev] DPDK Release Status Meeting 16/01/2020 Ferruh Yigit
2020-01-16 12:48 3% ` Ananyev, Konstantin
@ 2020-01-17 16:00 0% ` Honnappa Nagarahalli
1 sibling, 0 replies; 200+ results
From: Honnappa Nagarahalli @ 2020-01-17 16:00 UTC (permalink / raw)
To: Ferruh Yigit, dpdk-dev
Cc: thomas, Akhil.goyal@nxp.com, Ruifeng Wang, nd, Honnappa Nagarahalli, nd
<snip>
>
> Minutes 16 January 2020
> -----------------------
>
> Agenda:
> * Release Dates
> * Subtrees
> * OvS
>
> Participants:
> * Debian/Microsoft
> * Intel
> * Marvell
> * Mellanox
> * NXP
> * Red Hat
>
>
> Release Dates
> -------------
>
> * v20.02 dates:
> * Integration deadline passed, it was on Wednesday 15 January 2020
> * RC1 is expected before the weekend, possibly on Friday 17 January
> * PRC holidays on 24-20 Jan (inclusive)
> * Release: Friday 14 February 2020
>
> * v20.05 proposal:
> * Proposal/V1: Friday 6 March 2020
> * Integration/Merge/RC1: Friday 10 April 2020
> * Release: Friday 1 May || Wed 13 May
> * 1-5 May holiday on PRC, we need to do the release before or after that
> Please comment on between 1 May or 13 May.
>
>
> Subtrees
> --------
>
> * main
> * Will try to do -rc1 before Monday with best effort
> * Two series from arm
> * "ring library" updates
> * May go in in this release
> * Use WFE for aarch64
Arm has one more series for replacing Marvell's crypto library. The Aarch64 crypto library needed API renaming, clean up work and tagging for the release. The RFC series will be converted to a patch. The patch itself is not different from the RFC except of API and public structure renaming. The patch should be on the mailing list on 19th (Sunday).
> * ABI tooling patches
> * Tech-board selected David's approach
> * Automated checks will be in CI after David's patch
> * We need automated checks to be sure we are not breaking the ABI
> otherwise it is not always easy for developer to catch the break
> * Waiting new version of Neil's __rte_internal pathcset
>
> * next-net
> * Almost all ethdev changes merged, planning to finalize last
> remaining 1-2 ones today
> * All sub-trees pulled except some mlx and brcm one that are waiting fix
> * Some PMD patches may be postponed to -rc2
>
> * next-net-crypto
> * Pull request sent
> * There is a performance concern on some ipsec-gw patches,
> they can go in -rc2 if the issue is solved
> * CPU crypto from last release may be breaking ABI, need to confirm
> and discussed dummy variable is missing, may be postponed to next release
> * Some PMD changes are postponed to -rc2
>
> * next-net-eventdev
> * Pull request sent, it is a small set
>
> * next-net-virtio
> * All commit pulled to next-net
> * virtio vDPA drive will be postponed to next release
> * Some patches waiting user change and some are under review
> * Can be some more patches for -rc2
>
> * next-net-intel
> * Nothing critical, most of the patches have been pulled
> * 'fm10k' patches are not reviewed, may go in -rc2 if reviewed
> otherwise will be postponed to next release
>
> * LTS
>
> * 17.11.10-rc1 released, please test and report results
> * https://mails.dpdk.org/archives/dev/2020-January/154915.html
> * Release planned on January / February
>
> * 18.11.6-rc2 released, please test and report results
> * https://mails.dpdk.org/archives/dev/2020-January/155132.html
> * Only a few patches on top of -rc1
> * Intel will plan a validation, it won't be full validation
> * Release planned on 31st January
>
>
> OvS
> ---
>
> * Using DPDK experimental APIs discussed, it is accepted to use once as
> exception, so feature won't be blocked
> * DPDK need to figure out how to backport removing experimental tags to the
> LTS releases
>
>
>
> DPDK Release Status Meetings
> ============================
>
> The DPDK Release Status Meeting is intended for DPDK Committers to discuss
> the status of the master tree and sub-trees, and for project managers to track
> progress or milestone dates.
>
> The meeting occurs on Thursdays at 8:30 UTC. If you wish to attend just send
> an email to "John McNamara <john.mcnamara@intel.com>" for the invite.
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH] add ABI checks
2020-01-16 20:01 4% ` Thomas Monjalon
@ 2020-01-17 19:01 4% ` Neil Horman
2020-01-17 21:26 9% ` David Marchand
0 siblings, 1 reply; 200+ results
From: Neil Horman @ 2020-01-17 19:01 UTC (permalink / raw)
To: Thomas Monjalon
Cc: David Marchand, Richardson, Bruce, dev, Laatz, Kevin, aconole,
Michael Santana, Mcnamara, John, Kovacevic, Marko, Kinsella, Ray
On Thu, Jan 16, 2020 at 09:01:52PM +0100, Thomas Monjalon wrote:
> 16/01/2020 19:49, Neil Horman:
> > On Thu, Jan 16, 2020 at 03:20:48PM +0100, Thomas Monjalon wrote:
> > > 16/01/2020 12:52, Neil Horman:
> > > > On Wed, Jan 15, 2020 at 01:38:17PM +0100, Thomas Monjalon wrote:
> > > > > 15/01/2020 12:33, Neil Horman:
> > > > > > On Wed, Jan 15, 2020 at 12:19:30AM +0100, Thomas Monjalon wrote:
> > > > > > > 20/12/2019 17:20, Kinsella, Ray:
> > > > > > > > From: Richardson, Bruce <bruce.richardson@intel.com>
> > > > > > > > > From: David Marchand <david.marchand@redhat.com>
> > > > > > > > > > +Checking ABI compatibility
> > > > > > > > > > +--------------------------
> > > > > > > > > > +
> > > > > > > > > > +The first thing is to build reference binaries for the latest
> > > > > > > > > release
> > > > > > > > > > +your patches are built on top of.
> > > > > > > > > > +
> > > > > > > > > > +Either you are in a git tree and an easy way to identify this is to
> > > > > > > > > run::
> > > > > > > > > > +
> > > > > > > > > > + git checkout $(git describe --abbrev=0)
> > > > > > > > > > +
> > > > > > > > > > +Or you use a tarball and you extract the sources in a director of
> > > > > > > > > > +your
> > > > > > > > > > choice.
> > > > > > > > > > +
> > > > > > > > > > +Next is building those sources, refer to the previous paragraph.
> > > > > > > > > > +You can set ``DPDK_BUILD_TEST_DIR=reference``, so that the builds
> > > > > > > > > > +occur in this directory.
> > > > > > > > > > +
> > > > > > > > > > +Finally, the ABI dump files are generated with the
> > > > > > > > > > +``devtools/gen-abi-reference.sh`` script. This script will look for
> > > > > > > > > > +builds in the current sub directory ``reference``. But you can set
> > > > > > > > > > +the environment variable ``DPDK_ABI_REF_BUILD_DIR`` to a different
> > > > > > > > > location.
> > > > > > > > > > +
> > > > > > > > > > +Once done, you can check your current binaries ABI with this
> > > > > > > > > > +reference with the ``devtools/check-abi-reference.sh`` script.
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > I still very much dislike forcing the user to generate his own
> > > > > > > > > reference version to compare the ABI against. These should be archived
> > > > > > > > > and the user should just be able to pull them down via git or http or
> > > > > > > > > otherwise. Two reasons for this:
> > > > > > > > >
> > > > > > > > > 1. Less error prone, since there is no chance of the user having an
> > > > > > > > > incorrect build for whatever reason.
> > > > > > > > >
> > > > > > > > > 2. Less effort for the user than asking them to do extra builds. The
> > > > > > > > > more steps the user has to follow, the less likely they are to attempt
> > > > > > > > > the process.
> > > > > > > >
> > > > > > > > +1 ... 100% agree with this.
> > > > > > > >
> > > > > > > > Many people won't know or understand what the reference is,
> > > > > > > > or why they to generate it.
> > > > > > >
> > > > > > > I don't want to generate and save the reference in git for each arch.
> > > > > > >
> > > > > > Can I ask what your reluctance is? Is it related to not wanting to have to save
> > > > > > all this information that is otherwise not used for building purposes?
> > > > >
> > > > > Yes I prefer keeping only the sources in the repository.
> > > > > And these dumps are big.
> > > > > And last but not the least, there is no ready-to-use environment to build
> > > > > and dump all libs for all archs.
> > > > >
> > > > > > If so I might suggest saving the dumps in a separate git tree and pulling them
> > > > > > in as a git submodule when the check is performed
> > > > > >
> > > > > > I really like the idea of caching the results so everyone is working from a
> > > > > > known ABI baseline.
> > > > >
> > > > > You don't trust the result of the build made from tagged sources?
> > > > >
> > > > I trust the result from the tools, sure, its trusting that people will take the
> > > > extra time to build a version from a prior tag that I'm less sure of.
> > > > Consistent use in my mind is predicated on ease and timeliness of use.
> > > >
> > > > I get not wanting to store large dumps in the source tree, but storage is cheap,
> > > > and I don't see the issue with storing the xml dump in a separate git tree to be
> > > > referenced through a git submodule that gets pulled in when the check is run.
> > >
> > > Yes this is an option.
> > > My fear is that this reference database will not be complete
> > > if we don't build it for all libraries/drivers on all archs,
> > > managing setups and dependencies.
> > >
> > I can understand that, but I would have assumed that we would have done all
> > config build for all supported arches as part of the CI for a release, from
> > which we could archive the results. Is that not the case?
>
> No, we don't have a fully complete setup covering all dependencies and archs.
>
Hmmm....I wonder if its worth orchestrating the build system to use a git
submodule storing whatever our CI system can produce and using it as a cache,
and falling back to a local build if the appropriate arch isn't found? That
might offer some incentive to alternate arch maintainers to contribute hardware
or compute time to populating it in pursuit of better build times?
Neil
>
>
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH v9 5/6] lib/hash: use ring with 32b element size to save memory
@ 2020-01-17 20:27 5% ` David Marchand
2020-01-17 20:54 0% ` Honnappa Nagarahalli
0 siblings, 1 reply; 200+ results
From: David Marchand @ 2020-01-17 20:27 UTC (permalink / raw)
To: Honnappa Nagarahalli
Cc: Olivier Matz, Stephen Hemminger, Jerin Jacob Kollanukkaran,
Bruce Richardson, Pavan Nikhilesh, Ananyev, Konstantin, Wang,
Yipeng1, dev, Dharmik Thakkar,
Ruifeng Wang (Arm Technology China),
Gavin Hu, nd, Thomas Monjalon
On Thu, Jan 16, 2020 at 6:25 AM Honnappa Nagarahalli
<honnappa.nagarahalli@arm.com> wrote:
>
> The freelist and external bucket indices are 32b. Using rings
> that use 32b element sizes will save memory.
>
> Signed-off-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
> Reviewed-by: Gavin Hu <gavin.hu@arm.com>
> Reviewed-by: Ola Liljedahl <ola.liljedahl@arm.com>
> ---
> lib/librte_hash/rte_cuckoo_hash.c | 94 ++++++++++++++++---------------
> lib/librte_hash/rte_cuckoo_hash.h | 2 +-
> 2 files changed, 50 insertions(+), 46 deletions(-)
>
[snip]
> diff --git a/lib/librte_hash/rte_cuckoo_hash.h b/lib/librte_hash/rte_cuckoo_hash.h
> index fb19bb27d..345de6bf9 100644
> --- a/lib/librte_hash/rte_cuckoo_hash.h
> +++ b/lib/librte_hash/rte_cuckoo_hash.h
> @@ -124,7 +124,7 @@ const rte_hash_cmp_eq_t cmp_jump_table[NUM_KEY_CMP_CASES] = {
>
> struct lcore_cache {
> unsigned len; /**< Cache len */
> - void *objs[LCORE_CACHE_SIZE]; /**< Cache objects */
> + uint32_t objs[LCORE_CACHE_SIZE]; /**< Cache objects */
This triggers a warning in ABI checks:
1 function with some indirect sub-type change:
[C]'function int32_t rte_hash_add_key(const rte_hash*, void*)' at
rte_cuckoo_hash.c:1118:1 has some indirect sub-type changes:
parameter 1 of type 'const rte_hash*' has sub-type changes:
in pointed to type 'const rte_hash':
in unqualified underlying type 'struct rte_hash' at
rte_cuckoo_hash.h:160:1:
type size hasn't changed
1 data member change:
type of 'lcore_cache* rte_hash::local_free_slots' changed:
in pointed to type 'struct lcore_cache' at rte_cuckoo_hash.h:125:1:
type size changed from 4608 to 2560 (in bits)
1 data member change:
type of 'void* lcore_cache::objs[64]' changed:
array element type 'void*' changed:
entity changed from 'void*' to 'typedef uint32_t'
at stdint-uintn.h:26:1
type size changed from 64 to 32 (in bits)
type name changed from 'void*[64]' to 'uint32_t[64]'
array type size changed from 4096 to 2048
and offset changed from 64 to 32 (in bits) (by -32 bits)
As far as I can see, the local_free_slots field in rte_hash is
supposed to be internal and should just be hidden from users.
lib/librte_hash/rte_cuckoo_hash.c: h->local_free_slots =
rte_zmalloc_socket(NULL,
lib/librte_hash/rte_cuckoo_hash.c: rte_free(h->local_free_slots);
lib/librte_hash/rte_cuckoo_hash.c: cached_cnt +=
h->local_free_slots[i].len;
lib/librte_hash/rte_cuckoo_hash.c:
h->local_free_slots[i].len = 0;
lib/librte_hash/rte_cuckoo_hash.c: cached_free_slots =
&h->local_free_slots[lcore_id];
lib/librte_hash/rte_cuckoo_hash.c: cached_free_slots =
&h->local_free_slots[lcore_id];
lib/librte_hash/rte_cuckoo_hash.c: cached_free_slots =
&h->local_free_slots[lcore_id];
lib/librte_hash/rte_cuckoo_hash.h: struct lcore_cache *local_free_slots;
Not sure how users could make use of this.
But the abi check flags this as a breakage since this type was exported.
I can see three options:
- we stick to our "no abi breakage" policy, this change is postponed
to the next ABI breakage, and at the same time, we hide this type and
inspect the rest of the rte_hash API to avoid new issues in the
future,
- we duplicate structures and API by using function versioning to keep
the exact rte_hash v20.0 ABI and a v20.0.1 ABI with the resized and
cleaned structures,
- we override the ABI freeze here by ruling that this was an internal
structure that users should not access (ugh..)
Seeing how this is an optimisation, my preference goes to the first option.
--
David Marchand
^ permalink raw reply [relevance 5%]
* Re: [dpdk-dev] [PATCH v2 3/3] [20.05] mempool: return 0 if area is too small on populate
2020-01-17 14:57 2% ` [dpdk-dev] [PATCH v2 3/3] [20.05] mempool: return 0 if area is too small on populate Olivier Matz
@ 2020-01-17 20:32 0% ` David Marchand
0 siblings, 0 replies; 200+ results
From: David Marchand @ 2020-01-17 20:32 UTC (permalink / raw)
To: Olivier Matz
Cc: dev, Zhang, AlvinX, Burakov, Anatoly, Andrew Rybchenko,
Bruce Richardson, dpdk stable
On Fri, Jan 17, 2020 at 3:58 PM Olivier Matz <olivier.matz@6wind.com> wrote:
>
> Change rte_mempool_populate_iova() and rte_mempool_populate_iova() to
> return 0 instead of -EINVAL when there is not enough room to store one
> object, as it can be helpful for applications to distinguish this
> specific case.
>
> As this is an ABI change, use symbol versioning to preserve old
> behavior for binary applications.
>
> Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
> ---
> examples/ntb/ntb_fwd.c | 2 +-
> lib/librte_mempool/meson.build | 1 +
> lib/librte_mempool/rte_mempool.c | 76 ++++++++++++++++++----
> lib/librte_mempool/rte_mempool.h | 14 ++--
> lib/librte_mempool/rte_mempool_version.map | 7 ++
> 5 files changed, 84 insertions(+), 16 deletions(-)
>
> diff --git a/examples/ntb/ntb_fwd.c b/examples/ntb/ntb_fwd.c
> index c914256dd..99a43688c 100644
> --- a/examples/ntb/ntb_fwd.c
> +++ b/examples/ntb/ntb_fwd.c
> @@ -1313,7 +1313,7 @@ ntb_mbuf_pool_create(uint16_t mbuf_seg_size, uint32_t nb_mbuf,
> mz->len - ntb_info.ntb_hdr_size,
> ntb_mempool_mz_free,
> (void *)(uintptr_t)mz);
> - if (ret < 0) {
> + if (ret <= 0) {
> rte_memzone_free(mz);
> rte_mempool_free(mp);
> return NULL;
> diff --git a/lib/librte_mempool/meson.build b/lib/librte_mempool/meson.build
> index f8710b61b..4fe267cd8 100644
> --- a/lib/librte_mempool/meson.build
> +++ b/lib/librte_mempool/meson.build
> @@ -18,3 +18,4 @@ deps += ['ring']
>
> # memseg walk is not yet part of stable API
> allow_experimental_apis = true
> +use_function_versioning = true
> diff --git a/lib/librte_mempool/rte_mempool.c b/lib/librte_mempool/rte_mempool.c
> index b434d9f17..0ebb21eec 100644
> --- a/lib/librte_mempool/rte_mempool.c
> +++ b/lib/librte_mempool/rte_mempool.c
> @@ -31,6 +31,7 @@
> #include <rte_string_fns.h>
> #include <rte_spinlock.h>
> #include <rte_tailq.h>
> +#include <rte_function_versioning.h>
>
> #include "rte_mempool.h"
>
> @@ -293,12 +294,17 @@ mempool_ops_alloc_once(struct rte_mempool *mp)
> return 0;
> }
>
> +__vsym int
> +rte_mempool_populate_iova_v20_0_1(struct rte_mempool *mp, char *vaddr,
> + rte_iova_t iova, size_t len, rte_mempool_memchunk_free_cb_t *free_cb,
> + void *opaque);
> +
For 20.05, this should be 20.0.2.
> /* Add objects in the pool, using a physically contiguous memory
> * zone. Return the number of objects added, or a negative value
> * on error.
> */
> -static int
> -__rte_mempool_populate_iova(struct rte_mempool *mp, char *vaddr,
> +__vsym int
> +rte_mempool_populate_iova_v20_0_1(struct rte_mempool *mp, char *vaddr,
> rte_iova_t iova, size_t len, rte_mempool_memchunk_free_cb_t *free_cb,
> void *opaque)
> {
> @@ -355,21 +361,34 @@ __rte_mempool_populate_iova(struct rte_mempool *mp, char *vaddr,
> rte_free(memhdr);
> return ret;
> }
> +BIND_DEFAULT_SYMBOL(rte_mempool_populate_iova, _v20_0_1, 20.0.1);
> +MAP_STATIC_SYMBOL(
> + int rte_mempool_populate_iova(struct rte_mempool *mp, char *vaddr,
> + rte_iova_t iova, size_t len,
> + rte_mempool_memchunk_free_cb_t *free_cb,
> + void *opaque),
> + rte_mempool_populate_iova_v20_0_1);
> +
> +__vsym int
> +rte_mempool_populate_iova_v20_0(struct rte_mempool *mp, char *vaddr,
> + rte_iova_t iova, size_t len, rte_mempool_memchunk_free_cb_t *free_cb,
> + void *opaque);
>
> -int
> -rte_mempool_populate_iova(struct rte_mempool *mp, char *vaddr,
> +__vsym int
> +rte_mempool_populate_iova_v20_0(struct rte_mempool *mp, char *vaddr,
> rte_iova_t iova, size_t len, rte_mempool_memchunk_free_cb_t *free_cb,
> void *opaque)
> {
> int ret;
>
> - ret = __rte_mempool_populate_iova(mp, vaddr, iova, len, free_cb,
> + ret = rte_mempool_populate_iova_v20_0_1(mp, vaddr, iova, len, free_cb,
> opaque);
> if (ret == 0)
> ret = -EINVAL;
>
> return ret;
> }
> +VERSION_SYMBOL(rte_mempool_populate_iova, _v20_0, 20.0);
>
> static rte_iova_t
> get_iova(void *addr)
> @@ -384,11 +403,16 @@ get_iova(void *addr)
> return ms->iova + RTE_PTR_DIFF(addr, ms->addr);
> }
>
> +__vsym int
> +rte_mempool_populate_virt_v20_0_1(struct rte_mempool *mp, char *addr,
> + size_t len, size_t pg_sz, rte_mempool_memchunk_free_cb_t *free_cb,
> + void *opaque);
> +
> /* Populate the mempool with a virtual area. Return the number of
> * objects added, or a negative value on error.
> */
> -int
> -rte_mempool_populate_virt(struct rte_mempool *mp, char *addr,
> +__vsym int
> +rte_mempool_populate_virt_v20_0_1(struct rte_mempool *mp, char *addr,
> size_t len, size_t pg_sz, rte_mempool_memchunk_free_cb_t *free_cb,
> void *opaque)
> {
> @@ -421,7 +445,7 @@ rte_mempool_populate_virt(struct rte_mempool *mp, char *addr,
> break;
> }
>
> - ret = __rte_mempool_populate_iova(mp, addr + off, iova,
> + ret = rte_mempool_populate_iova_v20_0_1(mp, addr + off, iova,
> phys_len, free_cb, opaque);
> if (ret == 0)
> continue;
> @@ -432,15 +456,41 @@ rte_mempool_populate_virt(struct rte_mempool *mp, char *addr,
> cnt += ret;
> }
>
> - if (cnt == 0)
> - return -EINVAL;
> -
> return cnt;
>
> fail:
> rte_mempool_free_memchunks(mp);
> return ret;
> }
> +BIND_DEFAULT_SYMBOL(rte_mempool_populate_virt, _v20_0_1, 20.0.1);
> +MAP_STATIC_SYMBOL(
> + int rte_mempool_populate_virt(struct rte_mempool *mp,
> + char *addr, size_t len, size_t pg_sz,
> + rte_mempool_memchunk_free_cb_t *free_cb,
> + void *opaque),
> + rte_mempool_populate_virt_v20_0_1);
> +
> +__vsym int
> +rte_mempool_populate_virt_v20_0(struct rte_mempool *mp, char *addr,
> + size_t len, size_t pg_sz, rte_mempool_memchunk_free_cb_t *free_cb,
> + void *opaque);
> +
> +__vsym int
> +rte_mempool_populate_virt_v20_0(struct rte_mempool *mp, char *addr,
> + size_t len, size_t pg_sz, rte_mempool_memchunk_free_cb_t *free_cb,
> + void *opaque)
> +{
> + int ret;
> +
> + ret = rte_mempool_populate_virt_v20_0_1(mp, addr, len, pg_sz,
> + free_cb, opaque);
> +
> + if (ret == 0)
> + ret = -EINVAL;
> +
> + return ret;
> +}
> +VERSION_SYMBOL(rte_mempool_populate_virt, _v20_0, 20.0);
>
> /* Get the minimal page size used in a mempool before populating it. */
> int
> @@ -601,6 +651,8 @@ rte_mempool_populate_default(struct rte_mempool *mp)
> mz->len, pg_sz,
> rte_mempool_memchunk_mz_free,
> (void *)(uintptr_t)mz);
> + if (ret == 0) /* should not happen */
> + ret = -ENOBUFS;
> if (ret < 0) {
> rte_memzone_free(mz);
> goto fail;
> @@ -692,6 +744,8 @@ rte_mempool_populate_anon(struct rte_mempool *mp)
>
> ret = rte_mempool_populate_virt(mp, addr, size, getpagesize(),
> rte_mempool_memchunk_anon_free, addr);
> + if (ret == 0) /* should not happen */
> + ret = -ENOBUFS;
> if (ret < 0) {
> rte_errno = -ret;
> goto fail;
> diff --git a/lib/librte_mempool/rte_mempool.h b/lib/librte_mempool/rte_mempool.h
> index 0a1dc6059..5a106a432 100644
> --- a/lib/librte_mempool/rte_mempool.h
> +++ b/lib/librte_mempool/rte_mempool.h
> @@ -1106,9 +1106,12 @@ rte_mempool_free(struct rte_mempool *mp);
> * @param opaque
> * An opaque argument passed to free_cb.
> * @return
> - * The number of objects added on success.
> + * The number of objects added on success (strictly positive).
> * On error, the chunk is not added in the memory list of the
> - * mempool and a negative errno is returned.
> + * mempool the following code is returned:
> + * (0): not enough room in chunk for one object.
> + * (-ENOSPC): mempool is already populated.
> + * (-ENOMEM): allocation failure.
> */
> int rte_mempool_populate_iova(struct rte_mempool *mp, char *vaddr,
> rte_iova_t iova, size_t len, rte_mempool_memchunk_free_cb_t *free_cb,
> @@ -1133,9 +1136,12 @@ int rte_mempool_populate_iova(struct rte_mempool *mp, char *vaddr,
> * @param opaque
> * An opaque argument passed to free_cb.
> * @return
> - * The number of objects added on success.
> + * The number of objects added on success (strictly positive).
> * On error, the chunk is not added in the memory list of the
> - * mempool and a negative errno is returned.
> + * mempool the following code is returned:
> + * (0): not enough room in chunk for one object.
> + * (-ENOSPC): mempool is already populated.
> + * (-ENOMEM): allocation failure.
> */
> int
> rte_mempool_populate_virt(struct rte_mempool *mp, char *addr,
> diff --git a/lib/librte_mempool/rte_mempool_version.map b/lib/librte_mempool/rte_mempool_version.map
> index d002dfc46..34d977e47 100644
> --- a/lib/librte_mempool/rte_mempool_version.map
> +++ b/lib/librte_mempool/rte_mempool_version.map
> @@ -35,6 +35,13 @@ DPDK_20.0 {
> local: *;
> };
>
> +DPDK_20.0.1 {
> + global:
> +
> + rte_mempool_populate_iova;
> + rte_mempool_populate_virt;
> +} DPDK_20.0;
> +
> EXPERIMENTAL {
> global:
>
> --
> 2.20.1
>
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH v2 2/3] doc: announce API change for mempool IOVA populate
2020-01-17 14:57 8% ` [dpdk-dev] [PATCH v2 2/3] doc: announce API change for mempool IOVA populate Olivier Matz
@ 2020-01-17 20:32 0% ` David Marchand
0 siblings, 0 replies; 200+ results
From: David Marchand @ 2020-01-17 20:32 UTC (permalink / raw)
To: Olivier Matz
Cc: dev, Zhang, AlvinX, Burakov, Anatoly, Andrew Rybchenko,
Bruce Richardson, dpdk stable
On Fri, Jan 17, 2020 at 3:58 PM Olivier Matz <olivier.matz@6wind.com> wrote:
>
> Starting from v20.05, rte_mempool_populate_iova() will return
> -ENOBUFS. The ABI will be preserved through symbol versioning until
This deprecation notice describes a 0 return value, not -ENOBUFS.
> 20.11.
>
> Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
> ---
> doc/guides/rel_notes/deprecation.rst | 5 +++++
> 1 file changed, 5 insertions(+)
>
> diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
> index afa94b43e..e11c7f436 100644
> --- a/doc/guides/rel_notes/deprecation.rst
> +++ b/doc/guides/rel_notes/deprecation.rst
> @@ -86,3 +86,8 @@ Deprecation Notices
> to set new power environment if power environment was already initialized.
> In this case the function will return -1 unless the environment is unset first
> (using ``rte_power_unset_env``). Other function usage scenarios will not change.
> +
> +* mempool: starting from v20.05, the API of rte_mempool_populate_iova()
> + and rte_mempool_populate_virt() will change to return 0 instead
> + of -EINVAL when there is not enough room to store one object. The ABI
> + will be preserved until 20.11.
> --
> 2.20.1
>
--
David Marchand
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH v9 5/6] lib/hash: use ring with 32b element size to save memory
2020-01-17 20:27 5% ` David Marchand
@ 2020-01-17 20:54 0% ` Honnappa Nagarahalli
2020-01-17 21:07 3% ` David Marchand
0 siblings, 1 reply; 200+ results
From: Honnappa Nagarahalli @ 2020-01-17 20:54 UTC (permalink / raw)
To: David Marchand
Cc: Olivier Matz, Stephen Hemminger, jerinj, Bruce Richardson,
Pavan Nikhilesh, Ananyev, Konstantin, Wang, Yipeng1, dev,
Dharmik Thakkar, Ruifeng Wang, Gavin Hu, nd, thomas,
Honnappa Nagarahalli, nd
<snip>
>
> On Thu, Jan 16, 2020 at 6:25 AM Honnappa Nagarahalli
> <honnappa.nagarahalli@arm.com> wrote:
> >
> > The freelist and external bucket indices are 32b. Using rings that use
> > 32b element sizes will save memory.
> >
> > Signed-off-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
> > Reviewed-by: Gavin Hu <gavin.hu@arm.com>
> > Reviewed-by: Ola Liljedahl <ola.liljedahl@arm.com>
> > ---
> > lib/librte_hash/rte_cuckoo_hash.c | 94
> > ++++++++++++++++--------------- lib/librte_hash/rte_cuckoo_hash.h |
> > 2 +-
> > 2 files changed, 50 insertions(+), 46 deletions(-)
> >
>
> [snip]
>
> > diff --git a/lib/librte_hash/rte_cuckoo_hash.h
> > b/lib/librte_hash/rte_cuckoo_hash.h
> > index fb19bb27d..345de6bf9 100644
> > --- a/lib/librte_hash/rte_cuckoo_hash.h
> > +++ b/lib/librte_hash/rte_cuckoo_hash.h
> > @@ -124,7 +124,7 @@ const rte_hash_cmp_eq_t
> > cmp_jump_table[NUM_KEY_CMP_CASES] = {
> >
> > struct lcore_cache {
> > unsigned len; /**< Cache len */
> > - void *objs[LCORE_CACHE_SIZE]; /**< Cache objects */
> > + uint32_t objs[LCORE_CACHE_SIZE]; /**< Cache objects */
>
> This triggers a warning in ABI checks:
>
> 1 function with some indirect sub-type change:
>
> [C]'function int32_t rte_hash_add_key(const rte_hash*, void*)' at
> rte_cuckoo_hash.c:1118:1 has some indirect sub-type changes:
> parameter 1 of type 'const rte_hash*' has sub-type changes:
> in pointed to type 'const rte_hash':
> in unqualified underlying type 'struct rte_hash' at
> rte_cuckoo_hash.h:160:1:
> type size hasn't changed
> 1 data member change:
> type of 'lcore_cache* rte_hash::local_free_slots' changed:
> in pointed to type 'struct lcore_cache' at rte_cuckoo_hash.h:125:1:
> type size changed from 4608 to 2560 (in bits)
> 1 data member change:
> type of 'void* lcore_cache::objs[64]' changed:
> array element type 'void*' changed:
> entity changed from 'void*' to 'typedef uint32_t'
> at stdint-uintn.h:26:1
> type size changed from 64 to 32 (in bits)
> type name changed from 'void*[64]' to 'uint32_t[64]'
> array type size changed from 4096 to 2048
> and offset changed from 64 to 32 (in bits) (by -32 bits)
>
> As far as I can see, the local_free_slots field in rte_hash is supposed to be
> internal and should just be hidden from users.
> lib/librte_hash/rte_cuckoo_hash.c: h->local_free_slots =
> rte_zmalloc_socket(NULL,
> lib/librte_hash/rte_cuckoo_hash.c: rte_free(h->local_free_slots);
> lib/librte_hash/rte_cuckoo_hash.c: cached_cnt +=
> h->local_free_slots[i].len;
> lib/librte_hash/rte_cuckoo_hash.c:
> h->local_free_slots[i].len = 0;
> lib/librte_hash/rte_cuckoo_hash.c: cached_free_slots =
> &h->local_free_slots[lcore_id];
> lib/librte_hash/rte_cuckoo_hash.c: cached_free_slots =
> &h->local_free_slots[lcore_id];
> lib/librte_hash/rte_cuckoo_hash.c: cached_free_slots =
> &h->local_free_slots[lcore_id];
> lib/librte_hash/rte_cuckoo_hash.h: struct lcore_cache *local_free_slots;
>
> Not sure how users could make use of this.
> But the abi check flags this as a breakage since this type was exported.
I think this is a false positive.
Users include 'rte_hash.h' file which does not define the structure. It just has the declaration 'struct rte_hash'. The actual structure is defined in 'rte_cuckoo_hash.h'. But this is not included by the user. So, the application does not have visibility into 'struct rte_hash' as defined in 'rte_cuckoo_hash.h'.
The 'rte_create_hash' API returns a pointer to the 'struct rte_hash'. All the APIs are non-inline and just take this pointer as the argument. So, the 'struct rte_hash' as defined in 'rte_cuckoo_hash.h' is not used by the user.
You can take a look at test_hash_readwrite_lf.c and function 'check_bucket'. This function is written as the test case cannot access the 'struct rte_hash' from 'rte_cuckoo_hash.h'.
>
> I can see three options:
> - we stick to our "no abi breakage" policy, this change is postponed to the next
> ABI breakage, and at the same time, we hide this type and inspect the rest of
> the rte_hash API to avoid new issues in the future,
> - we duplicate structures and API by using function versioning to keep the
> exact rte_hash v20.0 ABI and a v20.0.1 ABI with the resized and cleaned
> structures,
> - we override the ABI freeze here by ruling that this was an internal structure
> that users should not access (ugh..)
>
> Seeing how this is an optimisation, my preference goes to the first option.
>
>
> --
> David Marchand
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH v9 5/6] lib/hash: use ring with 32b element size to save memory
2020-01-17 20:54 0% ` Honnappa Nagarahalli
@ 2020-01-17 21:07 3% ` David Marchand
2020-01-17 22:24 3% ` Wang, Yipeng1
0 siblings, 1 reply; 200+ results
From: David Marchand @ 2020-01-17 21:07 UTC (permalink / raw)
To: Honnappa Nagarahalli
Cc: Olivier Matz, Stephen Hemminger, jerinj, Bruce Richardson,
Pavan Nikhilesh, Ananyev, Konstantin, Wang, Yipeng1, dev,
Dharmik Thakkar, Ruifeng Wang, Gavin Hu, nd, thomas
On Fri, Jan 17, 2020 at 9:54 PM Honnappa Nagarahalli
<Honnappa.Nagarahalli@arm.com> wrote:
>
> <snip>
>
> >
> > On Thu, Jan 16, 2020 at 6:25 AM Honnappa Nagarahalli
> > <honnappa.nagarahalli@arm.com> wrote:
> > >
> > > The freelist and external bucket indices are 32b. Using rings that use
> > > 32b element sizes will save memory.
> > >
> > > Signed-off-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
> > > Reviewed-by: Gavin Hu <gavin.hu@arm.com>
> > > Reviewed-by: Ola Liljedahl <ola.liljedahl@arm.com>
> > > ---
> > > lib/librte_hash/rte_cuckoo_hash.c | 94
> > > ++++++++++++++++--------------- lib/librte_hash/rte_cuckoo_hash.h |
> > > 2 +-
> > > 2 files changed, 50 insertions(+), 46 deletions(-)
> > >
> >
> > [snip]
> >
> > > diff --git a/lib/librte_hash/rte_cuckoo_hash.h
> > > b/lib/librte_hash/rte_cuckoo_hash.h
> > > index fb19bb27d..345de6bf9 100644
> > > --- a/lib/librte_hash/rte_cuckoo_hash.h
> > > +++ b/lib/librte_hash/rte_cuckoo_hash.h
> > > @@ -124,7 +124,7 @@ const rte_hash_cmp_eq_t
> > > cmp_jump_table[NUM_KEY_CMP_CASES] = {
> > >
> > > struct lcore_cache {
> > > unsigned len; /**< Cache len */
> > > - void *objs[LCORE_CACHE_SIZE]; /**< Cache objects */
> > > + uint32_t objs[LCORE_CACHE_SIZE]; /**< Cache objects */
> >
> > This triggers a warning in ABI checks:
> >
> > 1 function with some indirect sub-type change:
> >
> > [C]'function int32_t rte_hash_add_key(const rte_hash*, void*)' at
> > rte_cuckoo_hash.c:1118:1 has some indirect sub-type changes:
> > parameter 1 of type 'const rte_hash*' has sub-type changes:
> > in pointed to type 'const rte_hash':
> > in unqualified underlying type 'struct rte_hash' at
> > rte_cuckoo_hash.h:160:1:
> > type size hasn't changed
> > 1 data member change:
> > type of 'lcore_cache* rte_hash::local_free_slots' changed:
> > in pointed to type 'struct lcore_cache' at rte_cuckoo_hash.h:125:1:
> > type size changed from 4608 to 2560 (in bits)
> > 1 data member change:
> > type of 'void* lcore_cache::objs[64]' changed:
> > array element type 'void*' changed:
> > entity changed from 'void*' to 'typedef uint32_t'
> > at stdint-uintn.h:26:1
> > type size changed from 64 to 32 (in bits)
> > type name changed from 'void*[64]' to 'uint32_t[64]'
> > array type size changed from 4096 to 2048
> > and offset changed from 64 to 32 (in bits) (by -32 bits)
> >
> > As far as I can see, the local_free_slots field in rte_hash is supposed to be
> > internal and should just be hidden from users.
> > lib/librte_hash/rte_cuckoo_hash.c: h->local_free_slots =
> > rte_zmalloc_socket(NULL,
> > lib/librte_hash/rte_cuckoo_hash.c: rte_free(h->local_free_slots);
> > lib/librte_hash/rte_cuckoo_hash.c: cached_cnt +=
> > h->local_free_slots[i].len;
> > lib/librte_hash/rte_cuckoo_hash.c:
> > h->local_free_slots[i].len = 0;
> > lib/librte_hash/rte_cuckoo_hash.c: cached_free_slots =
> > &h->local_free_slots[lcore_id];
> > lib/librte_hash/rte_cuckoo_hash.c: cached_free_slots =
> > &h->local_free_slots[lcore_id];
> > lib/librte_hash/rte_cuckoo_hash.c: cached_free_slots =
> > &h->local_free_slots[lcore_id];
> > lib/librte_hash/rte_cuckoo_hash.h: struct lcore_cache *local_free_slots;
> >
> > Not sure how users could make use of this.
> > But the abi check flags this as a breakage since this type was exported.
> I think this is a false positive.
>
> Users include 'rte_hash.h' file which does not define the structure. It just has the declaration 'struct rte_hash'. The actual structure is defined in 'rte_cuckoo_hash.h'. But this is not included by the user. So, the application does not have visibility into 'struct rte_hash' as defined in 'rte_cuckoo_hash.h'.
>
> The 'rte_create_hash' API returns a pointer to the 'struct rte_hash'. All the APIs are non-inline and just take this pointer as the argument. So, the 'struct rte_hash' as defined in 'rte_cuckoo_hash.h' is not used by the user.
Indeed, it seems properly hidden.
Scratching the rest of the mail.
Looked at abidiff, I can see it takes a public headers directory to
filter the ABI changes.
Need to make this work.
--
David Marchand
^ permalink raw reply [relevance 3%]
* Re: [dpdk-dev] [PATCH] add ABI checks
2020-01-17 19:01 4% ` Neil Horman
@ 2020-01-17 21:26 9% ` David Marchand
0 siblings, 0 replies; 200+ results
From: David Marchand @ 2020-01-17 21:26 UTC (permalink / raw)
To: Neil Horman, Thomas Monjalon
Cc: Richardson, Bruce, dev, Laatz, Kevin, aconole, Michael Santana,
Mcnamara, John, Kovacevic, Marko, Kinsella, Ray, Olivier Matz,
Honnappa Nagarahalli
On Fri, Jan 17, 2020 at 8:02 PM Neil Horman <nhorman@tuxdriver.com> wrote:
> Hmmm....I wonder if its worth orchestrating the build system to use a git
> submodule storing whatever our CI system can produce and using it as a cache,
> and falling back to a local build if the appropriate arch isn't found? That
> might offer some incentive to alternate arch maintainers to contribute hardware
> or compute time to populating it in pursuit of better build times?
Thomas, Neil,
I would prefer we consider this in a second step.
Just want to give a little update on this patch.
Building dpdk with debuginfo seems necessary, if we want to catch
internal structures changes.
I suppose Bruce patch can be merged already:
https://patchwork.dpdk.org/patch/63400/
libabigail in Ubuntu 16.04 (Xenial) is a bit old (version 1.0-rc3) and
exhibits a false positive on the linkage name of the default symbol if
we do function versioning: we caught this with Olivier when he asked
for help on his patch for mempool.
This issue disappears with version 1.2 that is in Ubuntu 18.04 (Bionic).
After running the check in my environment (fc30 with libabigail 1.5),
I caught issues with abidiff when comparing a .so and an abi dump.
I reported this upstream, and Dodji fixed this right away.
https://sourceware.org/bugzilla/show_bug.cgi?id=25409
The fix should be part of version 1.7.
This could be ignored if we only do the checks with binaries generated with gcc.
I just got a false positive on an ABI change for the "custom element
size" ring series, but on a type that is not user visible (thanks
Honnappa).
So this needs more work: I hope the --headers-dirX options will help
but it means we must store the public headers with the ABI dumps.
I can prepare a v2 and have the CI only warns about changes without
reporting a failure.
--
David Marchand
^ permalink raw reply [relevance 9%]
* Re: [dpdk-dev] [PATCH v9 5/6] lib/hash: use ring with 32b element size to save memory
2020-01-17 21:07 3% ` David Marchand
@ 2020-01-17 22:24 3% ` Wang, Yipeng1
0 siblings, 0 replies; 200+ results
From: Wang, Yipeng1 @ 2020-01-17 22:24 UTC (permalink / raw)
To: David Marchand, Honnappa Nagarahalli
Cc: Olivier Matz, Stephen Hemminger, jerinj, Richardson, Bruce,
Pavan Nikhilesh, Ananyev, Konstantin, dev, Dharmik Thakkar,
Ruifeng Wang, Gavin Hu, nd, thomas
>-----Original Message-----
>> > Not sure how users could make use of this.
>> > But the abi check flags this as a breakage since this type was exported.
>> I think this is a false positive.
>>
>> Users include 'rte_hash.h' file which does not define the structure. It just has the declaration 'struct rte_hash'. The actual structure is
>defined in 'rte_cuckoo_hash.h'. But this is not included by the user. So, the application does not have visibility into 'struct rte_hash' as
>defined in 'rte_cuckoo_hash.h'.
>>
>> The 'rte_create_hash' API returns a pointer to the 'struct rte_hash'. All the APIs are non-inline and just take this pointer as the
>argument. So, the 'struct rte_hash' as defined in 'rte_cuckoo_hash.h' is not used by the user.
>
>Indeed, it seems properly hidden.
>Scratching the rest of the mail.
>
>Looked at abidiff, I can see it takes a public headers directory to
>filter the ABI changes.
>Need to make this work.
>
>
>--
>David Marchand
[Wang, Yipeng]
Hi, Honnappa, I read the new API defs and this patch to rte_hash looks good to me.
Passed unit tests as well.
And I agree with you that the internals of rte_hash is hidden from users.
As long as the false warning in abi-check script is no concern:
Acked-by: Yipeng Wang <yipeng1.wang@intel.com>
^ permalink raw reply [relevance 3%]
* Re: [dpdk-dev] [dpdk-web] [PATCH] add build system update as nice to have
@ 2020-01-18 0:10 3% ` Honnappa Nagarahalli
0 siblings, 0 replies; 200+ results
From: Honnappa Nagarahalli @ 2020-01-18 0:10 UTC (permalink / raw)
To: Richardson, Bruce, Yigit, Ferruh, web, dev
Cc: thomas, David Marchand, Honnappa Nagarahalli, nd, nd
<snip>
I think this is a good feature to have.
>
> 2 thoughts on this, and adding dev list to discussion.
>
> 1) This would only apply to make builds, I think. Any internal headers should
> not be passed to the "headers = " line in meson which tracks headers for
> installation only (all headers are found at build time in their original source
> locations, not by being copied to a central location, so internal-only headers
> need no action).
> 2) Not having some of these headers precludes the development of other
> out-of-tree drivers without the full DPDK source tree. This is probably not a
> major concern, but is there a use-case where we want to allow people to
> rebuild their own private drivers just using an installed DPDK instance? Based
> on that, I see three categories - regular public headers, headers for driver
> APIs (maybe for a dev package), and purely internal headers.
I cannot think of any advantage of developing out of tree drivers with an installed DPDK instance.
Do we need to do ABI for driver API headers? I hope not.
>
> Regards,
> /Bruce
>
> > -----Original Message-----
> > From: web <web-bounces@dpdk.org> On Behalf Of Ferruh Yigit
> > Sent: Friday, January 17, 2020 1:19 PM
> > To: web@dpdk.org
> > Cc: Thomas Monjalon <thomas@monjalon.net>; David Marchand
> > <david.marchand@redhat.com>
> > Subject: [dpdk-web] [PATCH] add build system update as nice to have
> >
> > Some headers in DPDK needs to be shared between various libraries,
> > these are treated as public header by build system and put into same
> > folder, other libraries can consume the header from this folder.
> > But this cause a side affect that internal headers exposed to the
> > application.
> >
> > A simple sample of this is 'rte_cryptodev_pmd.h', it is provided by
> > 'cryptodev' library and used by crypto PMDs, but this headers
> > shouldn't be used by applications.
> >
> > A solution can be using two different interim folders, one for public
> > headers and other for DPDK wide headers. DPDK components can look both
> > folders but only ones in the public header folder will be installed to
> > system folders.
> >
> > Cc: David Marchand <david.marchand@redhat.com>
> >
> > Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
> > ---
> > content/roadmap/_index.md | 1 +
> > 1 file changed, 1 insertion(+)
> >
> > diff --git a/content/roadmap/_index.md b/content/roadmap/_index.md
> > index 6cb2e68..ea920bb 100644
> > --- a/content/roadmap/_index.md
> > +++ b/content/roadmap/_index.md
> > @@ -49,6 +49,7 @@ This list is obviously neither complete nor guaranteed.
> > - default configuration from files
> > - generic white/blacklisting
> > - libedit integration
> > +- don't expose DPDK wide header to public, like rte_cryptodev_pmd.h
> >
> > ### Cycle model {#cycle}
> > ----
> > --
> > 2.24.1
^ permalink raw reply [relevance 3%]
* Re: [dpdk-dev] [PATCH v10 0/6] lib/ring: APIs to support custom element size
@ 2020-01-19 19:31 3% ` David Marchand
0 siblings, 0 replies; 200+ results
From: David Marchand @ 2020-01-19 19:31 UTC (permalink / raw)
To: Honnappa Nagarahalli
Cc: Olivier Matz, Stephen Hemminger, Jerin Jacob Kollanukkaran,
Bruce Richardson, Pavan Nikhilesh, Ananyev, Konstantin, Wang,
Yipeng1, David Christensen, dev, Dharmik Thakkar,
Ruifeng Wang (Arm Technology China),
Gavin Hu, nd
On Sat, Jan 18, 2020 at 8:33 PM Honnappa Nagarahalli
<honnappa.nagarahalli@arm.com> wrote:
>
> The current rte_ring hard-codes the type of the ring element to 'void *',
> hence the size of the element is hard-coded to 32b/64b. Since the ring
> element type is not an input to rte_ring APIs, it results in couple
> of issues:
>
> 1) If an application requires to store an element which is not 64b, it
> needs to write its own ring APIs similar to rte_event_ring APIs. This
> creates additional burden on the programmers, who end up making
> work-arounds and often waste memory.
> 2) If there are multiple libraries that store elements of the same
> type, currently they would have to write their own rte_ring APIs. This
> results in code duplication.
>
> This patch adds new APIs to support configurable ring element size.
> The APIs support custom element sizes by allowing to define the ring
> element to be a multiple of 32b.
>
> The aim is to achieve same performance as the existing ring
> implementation.
>
> v10
> - Improved comments in test case files (Olivier)
> - Fixed possible memory leaks (Olivier)
> - Changed 'test_ring_with_exact_size' to use unaligned
> addresses (Konstantin)
> - Changed the commit message for eventdev (Jerin)
Thanks for working on this and a big thanks to all reviewers too.
The CI has been switched to Ubuntu 18.04, so that we won't hit the
Travis timeout with clang 7.
There is still some work on the ABI checks, because of the abidiff
report on rte_cuckoo_hash.h I mentioned: passing the public headers to
abidw/abidiff should do the trick.
Series applied.
--
David Marchand
^ permalink raw reply [relevance 3%]
* Re: [dpdk-dev] [PATCH v2] cfgfile: update map file
@ 2020-01-19 23:31 0% ` Thomas Monjalon
0 siblings, 0 replies; 200+ results
From: Thomas Monjalon @ 2020-01-19 23:31 UTC (permalink / raw)
To: lironh; +Cc: dev, stable, Dumitrescu, Cristian, david.marchand
17/12/2019 12:12, Dumitrescu, Cristian:
>
> > -----Original Message-----
> > From: lironh@marvell.com <lironh@marvell.com>
> > Sent: Tuesday, December 17, 2019 11:52 AM
> > To: dev@dpdk.org
> > Cc: Dumitrescu, Cristian <cristian.dumitrescu@intel.com>; stable@dpdk.org;
> > Liron Himi <lironh@marvell.com>
> > Subject: [PATCH v2] cfgfile: update map file
> >
> > From: Liron Himi <lironh@marvell.com>
> >
> > rte_cfgfile_section_num_entries_by_index was missing from the map
> > file.
> > meson build failed when calling this function, due to linking a binary to cfgfile
> > built
> > as a shared library
> >
> > Fixes: 85ff364f3b ("build: align symbols with global ABI version")
> >
> > Signed-off-by: Liron Himi <lironh@marvell.com>
>
> Acked-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
Fixing the "Fixes" line with the real root cause:
Fixes: 3d2e0448ebb5 ("cfgfile: add section number of entries by index")
Cc: stable@dpdk.org
Applied, thanks
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH v2 0/3] mempool: fix mempool virt populate with small chunks
2020-01-17 14:57 5% ` [dpdk-dev] [PATCH v2 0/3] " Olivier Matz
` (2 preceding siblings ...)
2020-01-17 14:57 2% ` [dpdk-dev] [PATCH v2 3/3] [20.05] mempool: return 0 if area is too small on populate Olivier Matz
@ 2020-01-20 12:02 0% ` Thomas Monjalon
3 siblings, 0 replies; 200+ results
From: Thomas Monjalon @ 2020-01-20 12:02 UTC (permalink / raw)
To: Olivier Matz
Cc: dev, Zhang, AlvinX, Burakov, Anatoly, Andrew Rybchenko,
Bruce Richardson, David Marchand, dpdk stable
17/01/2020 15:57, Olivier Matz:
> rte_mempool_populate_virt() sometimes fail, when it calls
> rte_mempool_populate_iova() with an area which is too small to store one
> object. This should not be an error.
>
> I prepared a v2 which implements an ABI compatibility through symbol
> versioning, as suggested [1]. It looks a bit overkill to me, but it
> was an interresting exercice.
[...]
> Olivier Matz (3):
> mempool: fix mempool virt populate with small chunks
> doc: announce API change for mempool IOVA populate
> mempool: return 0 if area is too small on populate
Applied patches 1 & 2 (with commit log fixed)
Patch 3 is for 20.05.
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH v4 4/5] app/testpmd: add mempool with external data buffers
@ 2020-01-20 14:11 3% ` Olivier Matz
0 siblings, 0 replies; 200+ results
From: Olivier Matz @ 2020-01-20 14:11 UTC (permalink / raw)
To: Viacheslav Ovsiienko; +Cc: dev, matan, rasland, orika, shahafs, stephen, thomas
On Thu, Jan 16, 2020 at 01:04:28PM +0000, Viacheslav Ovsiienko wrote:
> The new mbuf pool type is added to testpmd. To engage the
> mbuf pool with externally attached data buffers the parameter
> "--mp-alloc=xbuf" should be specified in testpmd command line.
>
> The objective of this patch is just to test whether mbuf pool
> with externally attached data buffers works OK. The memory for
> data buffers is allocated from DPDK memory, so this is not
> "true" external memory from some physical device (this is
> supposed the most common use case for such kind of mbuf pool).
>
> The user should be aware that not all drivers support the mbuf
> with EXT_ATTACHED_BUF flags set in newly allocated mbuf (many
> PMDs just overwrite ol_flags field and flag value is getting
> lost).
>
> Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
> ---
> app/test-pmd/config.c | 2 ++
> app/test-pmd/flowgen.c | 3 +-
> app/test-pmd/parameters.c | 2 ++
> app/test-pmd/testpmd.c | 81 +++++++++++++++++++++++++++++++++++++++++++++++
> app/test-pmd/testpmd.h | 4 ++-
> app/test-pmd/txonly.c | 3 +-
> 6 files changed, 92 insertions(+), 3 deletions(-)
>
> diff --git a/app/test-pmd/config.c b/app/test-pmd/config.c
> index 9da1ffb..5c6fe18 100644
> --- a/app/test-pmd/config.c
> +++ b/app/test-pmd/config.c
> @@ -2395,6 +2395,8 @@ struct igb_ring_desc_16_bytes {
> return "xmem";
> case MP_ALLOC_XMEM_HUGE:
> return "xmemhuge";
> + case MP_ALLOC_XBUF:
> + return "xbuf";
> default:
> return "invalid";
> }
> diff --git a/app/test-pmd/flowgen.c b/app/test-pmd/flowgen.c
> index 03b72aa..ae50cdc 100644
> --- a/app/test-pmd/flowgen.c
> +++ b/app/test-pmd/flowgen.c
> @@ -199,7 +199,8 @@
> sizeof(*ip_hdr));
> pkt->nb_segs = 1;
> pkt->pkt_len = pkt_size;
> - pkt->ol_flags = ol_flags;
> + pkt->ol_flags &= EXT_ATTACHED_MBUF;
> + pkt->ol_flags |= ol_flags;
> pkt->vlan_tci = vlan_tci;
> pkt->vlan_tci_outer = vlan_tci_outer;
> pkt->l2_len = sizeof(struct rte_ether_hdr);
This shows that we have to be careful when using a mempool with
external memory pinned mbufs. Maybe that's something that should
be mentionned in the release note?
That's not the first time I'm asking myself if ol_flags shouldn't be
splitted in ol_flags and flags. Certainly something to think about for
next ABI breakage release.
Acked-by: Olivier Matz <olivier.matz@6wind.com>
^ permalink raw reply [relevance 3%]
* Re: [dpdk-dev] [PATCH v7 00/10] net/i40e: ESP support
2020-01-16 17:42 0% ` Ferruh Yigit
@ 2020-01-21 16:10 0% ` Iremonger, Bernard
0 siblings, 0 replies; 200+ results
From: Iremonger, Bernard @ 2020-01-21 16:10 UTC (permalink / raw)
To: Yigit, Ferruh, dev, Xing, Beilei, Zhang, Qi Z, Doherty, Declan
Cc: Ananyev, Konstantin, Byrne, Stephen1, Zhang, Helin
Hi Ferruh,
> -----Original Message-----
> From: Yigit, Ferruh <ferruh.yigit@intel.com>
> Sent: Thursday, January 16, 2020 5:43 PM
> To: Iremonger, Bernard <bernard.iremonger@intel.com>; dev@dpdk.org;
> Xing, Beilei <beilei.xing@intel.com>; Zhang, Qi Z <qi.z.zhang@intel.com>;
> Doherty, Declan <declan.doherty@intel.com>
> Cc: Ananyev, Konstantin <konstantin.ananyev@intel.com>; Byrne, Stephen1
> <stephen1.byrne@intel.com>; Zhang, Helin <helin.zhang@intel.com>
> Subject: Re: [dpdk-dev] [PATCH v7 00/10] net/i40e: ESP support
>
> On 1/16/2020 12:44 PM, Bernard Iremonger wrote:
> > Add support for ESP flows to testpmd.
> > Improve debug information in testpmd and the i40e PMD.
> > Process ESP flows on the i40e Flow Director.
> >
> > Changes in V7:
> > -------------
> > Dropped the following debug patch from v6 patchset
> > 0003-app-testpmd-dump-Rx-and-Tx-mbuf.patch
> > Included release notes and doc patches in feature patches.
> >
> > Changes in V6:
> > -------------
> > Rebase to latest master branch
> > Fixed patch 0001-app-testpmd-parse-flow-command-line-for-ESP.patch
> > Dropped the following v5 patch due to ABI breakage:
> > 0012-librte_ethdev-add-ESP-and-AH-flow-types-to-RSS.patch
> >
> > Changes in V5:
> > -------------
> > Rebase to latest master branch
> > Update 0012-librte_ethdev-add-ESP-and-AH-flow-types-to-RSS.patch
> >
> > Changes in V4:
> > -------------
> > Add extra patch for testpmd
> > Add extra patch for config
> > Split v3 patch "net/i40e: process ESP flows" into 4 patches
> >
> > Changes in V3:
> > -------------
> > Added i40e_flow_set_filter_spi() function in i40e_flow.c Set UDP
> > destination port to 4500 for ESP in i40e_ethdev.h Split flow
> > structures into 4 instead of 2 in i40e_ethdev.h Dropped extra printf
> > from commandline_flow.c
> >
> > Changes in V2:
> > --------------
> > Moved change in app/test-pmd/config.c to a seperate patch.
> > Added extra parameter to fill_ip6_head() in i40e_fdir.c set is_udp to
> > false in i40e_flow_fdir_get_pctype_value() in i40e_flow.c Bernard
> > Iremonger (10):
> > app/testpmd: parse flow command line for ESP
> > app/testpmd: improve debug
> > net/i40e: improve RSS debug
> > net/i40e: handle ESP tunnel
> > net/i40e: support ipsec-ah profile
> > net/i40e: support ESP in customized code
> > net/i40e: support ESP flows
> > net/i40e: support ESP in Flow Director
> > config: add debug to I40E Flow Director
> > net/i40e: display Flow Director packet
>
> Series applied to dpdk-next-net/master, thanks.
>
> (Except 9/10 & 10/10, they are dropped while merging.)
>
> Series wasn't rebased on next-net, I had to resolve conflict almost in all
> patches, can you please validate the result in next-net?
I have tested the merged result in DPDK 20.02-rc1.
I found an issue with i40e_fdir.c, I have sent a patch to fix the issue.
Regards,
Bernard.
^ permalink raw reply [relevance 0%]
* [dpdk-dev] [PATCH] net/octeontx2: extend RSS supported offload types
@ 2020-01-22 3:46 3% kirankumark
2020-01-22 8:05 0% ` Jerin Jacob
2020-01-24 3:46 3% ` [dpdk-dev] [PATCH v2] " kirankumark
0 siblings, 2 replies; 200+ results
From: kirankumark @ 2020-01-22 3:46 UTC (permalink / raw)
To: Jerin Jacob, Nithin Dabilpuram, Vamsi Attunuru, Kiran Kumar K; +Cc: dev
From: Kiran Kumar K <kirankumark@marvell.com>
Extend RSS offload types for octeontx2. Add support to select
L3 SRC, L3 DST, L4 SRC and L4 DST for RSS calculation.
Add support to select L3 SRC or DST only, L4 SRC or DST only for RSS
calculation.
With this requirement there will be following combinations,
IPV[4,6]_SRC_ONLY, IPV[4,6]_DST_ONLY, [TCP,UDP,SCTP]_SRC_ONLY,
[TCP,UDP,SCTP]_DST_ONLY. So, instead of creating a bit for each
combination, we are using upper 4 bits (31:28) in the flow_key_cfg
to represent the SRC, DST selection. 31 => L3_SRC, 30 => L3_DST,
29 => L4_SRC, 28 => L4_DST. These won't be part of flow_cfg, so that
we don't need to change the existing ABI.
Signed-off-by: Kiran Kumar K <kirankumark@marvell.com>
---
drivers/common/octeontx2/otx2_mbox.h | 6 +++++-
| 12 ++++++++++++
2 files changed, 17 insertions(+), 1 deletion(-)
diff --git a/drivers/common/octeontx2/otx2_mbox.h b/drivers/common/octeontx2/otx2_mbox.h
index e0e4e2f63..4972b8a6e 100644
--- a/drivers/common/octeontx2/otx2_mbox.h
+++ b/drivers/common/octeontx2/otx2_mbox.h
@@ -918,7 +918,11 @@ struct nix_rss_flowkey_cfg {
#define FLOW_KEY_TYPE_INNR_UDP BIT(15)
#define FLOW_KEY_TYPE_INNR_SCTP BIT(16)
#define FLOW_KEY_TYPE_INNR_ETH_DMAC BIT(17)
- uint8_t group; /* RSS context or group */
+#define FLOW_KEY_TYPE_L4_DST BIT(28)
+#define FLOW_KEY_TYPE_L4_SRC BIT(29)
+#define FLOW_KEY_TYPE_L3_DST BIT(30)
+#define FLOW_KEY_TYPE_L3_SRC BIT(31)
+ uint8_t __otx2_io group; /* RSS context or group */
};
struct nix_rss_flowkey_cfg_rsp {
--git a/drivers/net/octeontx2/otx2_rss.c b/drivers/net/octeontx2/otx2_rss.c
index bc7b64387..7a8c8f3de 100644
--- a/drivers/net/octeontx2/otx2_rss.c
+++ b/drivers/net/octeontx2/otx2_rss.c
@@ -210,6 +210,18 @@ otx2_rss_ethdev_to_nix(struct otx2_eth_dev *dev, uint64_t ethdev_rss,
dev->rss_info.nix_rss = ethdev_rss;
+ if (ethdev_rss & ETH_RSS_L3_SRC_ONLY)
+ flowkey_cfg |= FLOW_KEY_TYPE_L3_SRC;
+
+ if (ethdev_rss & ETH_RSS_L3_DST_ONLY)
+ flowkey_cfg |= FLOW_KEY_TYPE_L3_DST;
+
+ if (ethdev_rss & ETH_RSS_L4_SRC_ONLY)
+ flowkey_cfg |= FLOW_KEY_TYPE_L4_SRC;
+
+ if (ethdev_rss & ETH_RSS_L4_DST_ONLY)
+ flowkey_cfg |= FLOW_KEY_TYPE_L4_DST;
+
if (ethdev_rss & RSS_IPV4_ENABLE)
flowkey_cfg |= flow_key_type[rss_level][RSS_IPV4_INDEX];
--
2.17.1
^ permalink raw reply [relevance 3%]
* Re: [dpdk-dev] [PATCH] net/octeontx2: extend RSS supported offload types
2020-01-22 3:46 3% [dpdk-dev] [PATCH] net/octeontx2: extend RSS supported offload types kirankumark
@ 2020-01-22 8:05 0% ` Jerin Jacob
2020-01-23 5:30 0% ` [dpdk-dev] [EXT] " Kiran Kumar Kokkilagadda
2020-01-24 3:46 3% ` [dpdk-dev] [PATCH v2] " kirankumark
1 sibling, 1 reply; 200+ results
From: Jerin Jacob @ 2020-01-22 8:05 UTC (permalink / raw)
To: Kiran Kumar K; +Cc: Jerin Jacob, Nithin Dabilpuram, Vamsi Attunuru, dpdk-dev
On Wed, Jan 22, 2020 at 9:16 AM <kirankumark@marvell.com> wrote:
>
> From: Kiran Kumar K <kirankumark@marvell.com>
>
> Extend RSS offload types for octeontx2. Add support to select
> L3 SRC, L3 DST, L4 SRC and L4 DST for RSS calculation.
>
> Add support to select L3 SRC or DST only, L4 SRC or DST only for RSS
> calculation.
>
> With this requirement there will be following combinations,
> IPV[4,6]_SRC_ONLY, IPV[4,6]_DST_ONLY, [TCP,UDP,SCTP]_SRC_ONLY,
> [TCP,UDP,SCTP]_DST_ONLY. So, instead of creating a bit for each
> combination, we are using upper 4 bits (31:28) in the flow_key_cfg
> to represent the SRC, DST selection. 31 => L3_SRC, 30 => L3_DST,
> 29 => L4_SRC, 28 => L4_DST. These won't be part of flow_cfg, so that
> we don't need to change the existing ABI.
>
> Signed-off-by: Kiran Kumar K <kirankumark@marvell.com>
> ---
>
> struct nix_rss_flowkey_cfg_rsp {
> diff --git a/drivers/net/octeontx2/otx2_rss.c b/drivers/net/octeontx2/otx2_rss.c
> index bc7b64387..7a8c8f3de 100644
> --- a/drivers/net/octeontx2/otx2_rss.c
> +++ b/drivers/net/octeontx2/otx2_rss.c
> @@ -210,6 +210,18 @@ otx2_rss_ethdev_to_nix(struct otx2_eth_dev *dev, uint64_t ethdev_rss,
>
> dev->rss_info.nix_rss = ethdev_rss;
>
Should n't we update the struct
rte_eth_dev_info::flow_type_rss_offloads to show this capability to
the driver. Please check other drivers.
> + if (ethdev_rss & ETH_RSS_L3_SRC_ONLY)
> + flowkey_cfg |= FLOW_KEY_TYPE_L3_SRC;
> +
> + if (ethdev_rss & ETH_RSS_L3_DST_ONLY)
> + flowkey_cfg |= FLOW_KEY_TYPE_L3_DST;
> +
> + if (ethdev_rss & ETH_RSS_L4_SRC_ONLY)
> + flowkey_cfg |= FLOW_KEY_TYPE_L4_SRC;
> +
> + if (ethdev_rss & ETH_RSS_L4_DST_ONLY)
> + flowkey_cfg |= FLOW_KEY_TYPE_L4_DST;
> +
> if (ethdev_rss & RSS_IPV4_ENABLE)
> flowkey_cfg |= flow_key_type[rss_level][RSS_IPV4_INDEX];
>
> --
> 2.17.1
>
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [dpdk-stable] [PATCH] meter: move RFC4115 trTCM APIs as none experimental
2020-01-16 12:42 0% ` Ferruh Yigit
2020-01-16 14:38 0% ` Thomas Monjalon
@ 2020-01-22 13:55 0% ` Ray Kinsella
2020-01-30 16:33 4% ` Ray Kinsella
2 siblings, 0 replies; 200+ results
From: Ray Kinsella @ 2020-01-22 13:55 UTC (permalink / raw)
To: Ferruh Yigit, Neil Horman, David Marchand
Cc: Thomas Monjalon, Cristian Dumitrescu, dpdk stable, dev,
Eelco Chaudron, Kevin Traynor, Ian Stokes, Ilya Maximets,
Luca Boccassi
On 16/01/2020 12:42, Ferruh Yigit wrote:
> On 1/16/2020 11:54 AM, Neil Horman wrote:
>> On Thu, Jan 16, 2020 at 12:25:06PM +0100, David Marchand wrote:
>>> On Tue, Dec 17, 2019 at 2:08 PM Eelco Chaudron <echaudro@redhat.com> wrote:
>>>>
>>>> Moved RFC4115 APIs to none experimental as they have been there
>>>> since 19.02. Also, these APIs are the same as the none RFC4115 APIs.
>>>>
>>>> Signed-off-by: Eelco Chaudron <echaudro@redhat.com>
>>>
>>> There is a discussion on the OVS ml at the moment to get these symbols
>>> in the stable ABI for 19.11.
>>> I want to understand how this would be done.
>>>
>>> - I take this patch in 20.02, these symbols are added in the 20.0.1 ABI.
>>> On the other hand, the 19.11 release maintains the 20.0 ABI.
>>>
>>> Does it mean the backport adds these symbols with the 20.0 version in
>>> the 19.11 branch?
>>> Or is 20.0.1 version acceptable / a thing we want?
>>>
>>>
>>> - These symbol already existed in the 20.0 ABI, versioned as EXPERIMENTAL.
>>> We can go and remove these entries since we are not bound to preserve
>>> the experimental APIs.
>>> But, on the other hand, nothing should prevent us from keeping some
>>> aliases so that the symbols versioned EXPERIMENTAL are still available
>>> to existing users.
>>>
>> I would say that choice is up to you. If you want to alias them to be nice to
>> prior users, thats fine by me. But experimental means experimental, and so users
>> have to be prepared to rebuild when things change, even if that change is
>> changing the version from experimental to a concrete version.
>>
>
> I would prefer to keep the alias and don't break the existing users, specially
> for the case experimental API is becoming mature without change.
>
+1 keeping an alias until 20.11/v21 is a good idea, particularly if the API is "baked"
and has not changed in some time - as is the case here.
Ray K
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [EXT] Re: [PATCH] net/octeontx2: extend RSS supported offload types
2020-01-22 8:05 0% ` Jerin Jacob
@ 2020-01-23 5:30 0% ` Kiran Kumar Kokkilagadda
0 siblings, 0 replies; 200+ results
From: Kiran Kumar Kokkilagadda @ 2020-01-23 5:30 UTC (permalink / raw)
To: Jerin Jacob
Cc: Jerin Jacob Kollanukkaran, Nithin Kumar Dabilpuram,
Vamsi Krishna Attunuru, dpdk-dev
> -----Original Message-----
> From: Jerin Jacob <jerinjacobk@gmail.com>
> Sent: Wednesday, January 22, 2020 1:35 PM
> To: Kiran Kumar Kokkilagadda <kirankumark@marvell.com>
> Cc: Jerin Jacob Kollanukkaran <jerinj@marvell.com>; Nithin Kumar Dabilpuram
> <ndabilpuram@marvell.com>; Vamsi Krishna Attunuru
> <vattunuru@marvell.com>; dpdk-dev <dev@dpdk.org>
> Subject: [EXT] Re: [dpdk-dev] [PATCH] net/octeontx2: extend RSS supported
> offload types
>
> External Email
>
> ----------------------------------------------------------------------
> On Wed, Jan 22, 2020 at 9:16 AM <kirankumark@marvell.com> wrote:
> >
> > From: Kiran Kumar K <kirankumark@marvell.com>
> >
> > Extend RSS offload types for octeontx2. Add support to select
> > L3 SRC, L3 DST, L4 SRC and L4 DST for RSS calculation.
> >
> > Add support to select L3 SRC or DST only, L4 SRC or DST only for RSS
> > calculation.
> >
> > With this requirement there will be following combinations,
> > IPV[4,6]_SRC_ONLY, IPV[4,6]_DST_ONLY, [TCP,UDP,SCTP]_SRC_ONLY,
> > [TCP,UDP,SCTP]_DST_ONLY. So, instead of creating a bit for each
> > combination, we are using upper 4 bits (31:28) in the flow_key_cfg to
> > represent the SRC, DST selection. 31 => L3_SRC, 30 => L3_DST,
> > 29 => L4_SRC, 28 => L4_DST. These won't be part of flow_cfg, so that
> > we don't need to change the existing ABI.
> >
> > Signed-off-by: Kiran Kumar K <kirankumark@marvell.com>
> > ---
> >
> > struct nix_rss_flowkey_cfg_rsp {
> > diff --git a/drivers/net/octeontx2/otx2_rss.c
> > b/drivers/net/octeontx2/otx2_rss.c
> > index bc7b64387..7a8c8f3de 100644
> > --- a/drivers/net/octeontx2/otx2_rss.c
> > +++ b/drivers/net/octeontx2/otx2_rss.c
> > @@ -210,6 +210,18 @@ otx2_rss_ethdev_to_nix(struct otx2_eth_dev *dev,
> > uint64_t ethdev_rss,
> >
> > dev->rss_info.nix_rss = ethdev_rss;
> >
>
> Should n't we update the struct
> rte_eth_dev_info::flow_type_rss_offloads to show this capability to the driver.
> Please check other drivers.
Verified with testpmd. I need to update the supported RSS offloads. Will send V2.
>
>
>
> > + if (ethdev_rss & ETH_RSS_L3_SRC_ONLY)
> > + flowkey_cfg |= FLOW_KEY_TYPE_L3_SRC;
> > +
> > + if (ethdev_rss & ETH_RSS_L3_DST_ONLY)
> > + flowkey_cfg |= FLOW_KEY_TYPE_L3_DST;
> > +
> > + if (ethdev_rss & ETH_RSS_L4_SRC_ONLY)
> > + flowkey_cfg |= FLOW_KEY_TYPE_L4_SRC;
> > +
> > + if (ethdev_rss & ETH_RSS_L4_DST_ONLY)
> > + flowkey_cfg |= FLOW_KEY_TYPE_L4_DST;
> > +
> > if (ethdev_rss & RSS_IPV4_ENABLE)
> > flowkey_cfg |=
> > flow_key_type[rss_level][RSS_IPV4_INDEX];
> >
> > --
> > 2.17.1
> >
^ permalink raw reply [relevance 0%]
* [dpdk-dev] [PATCH v2] net/octeontx2: extend RSS supported offload types
2020-01-22 3:46 3% [dpdk-dev] [PATCH] net/octeontx2: extend RSS supported offload types kirankumark
2020-01-22 8:05 0% ` Jerin Jacob
@ 2020-01-24 3:46 3% ` kirankumark
2020-01-28 11:14 0% ` Jerin Jacob
1 sibling, 1 reply; 200+ results
From: kirankumark @ 2020-01-24 3:46 UTC (permalink / raw)
To: Jerin Jacob, Nithin Dabilpuram, Vamsi Attunuru, Kiran Kumar K; +Cc: dev
From: Kiran Kumar K <kirankumark@marvell.com>
Extend RSS offload types for octeontx2. Add support to select
L3 SRC, L3 DST, L4 SRC and L4 DST for RSS calculation.
Add support to select L3 SRC or DST only, L4 SRC or DST only for RSS
calculation.
With this requirement there will be following combinations,
IPV[4,6]_SRC_ONLY, IPV[4,6]_DST_ONLY, [TCP,UDP,SCTP]_SRC_ONLY,
[TCP,UDP,SCTP]_DST_ONLY. So, instead of creating a bit for each
combination, we are using upper 4 bits (31:28) in the flow_key_cfg
to represent the SRC, DST selection. 31 => L3_SRC, 30 => L3_DST,
29 => L4_SRC, 28 => L4_DST. These won't be part of flow_cfg, so that
we don't need to change the existing ABI.
Signed-off-by: Kiran Kumar K <kirankumark@marvell.com>
---
V2 Changes:
* Updated supported offloads
drivers/common/octeontx2/otx2_mbox.h | 6 +++++-
drivers/net/octeontx2/otx2_ethdev.h | 6 +++++-
| 12 ++++++++++++
3 files changed, 22 insertions(+), 2 deletions(-)
diff --git a/drivers/common/octeontx2/otx2_mbox.h b/drivers/common/octeontx2/otx2_mbox.h
index e0e4e2f63..4972b8a6e 100644
--- a/drivers/common/octeontx2/otx2_mbox.h
+++ b/drivers/common/octeontx2/otx2_mbox.h
@@ -918,7 +918,11 @@ struct nix_rss_flowkey_cfg {
#define FLOW_KEY_TYPE_INNR_UDP BIT(15)
#define FLOW_KEY_TYPE_INNR_SCTP BIT(16)
#define FLOW_KEY_TYPE_INNR_ETH_DMAC BIT(17)
- uint8_t group; /* RSS context or group */
+#define FLOW_KEY_TYPE_L4_DST BIT(28)
+#define FLOW_KEY_TYPE_L4_SRC BIT(29)
+#define FLOW_KEY_TYPE_L3_DST BIT(30)
+#define FLOW_KEY_TYPE_L3_SRC BIT(31)
+ uint8_t __otx2_io group; /* RSS context or group */
};
struct nix_rss_flowkey_cfg_rsp {
diff --git a/drivers/net/octeontx2/otx2_ethdev.h b/drivers/net/octeontx2/otx2_ethdev.h
index 3f3fdecc4..c075b8d1a 100644
--- a/drivers/net/octeontx2/otx2_ethdev.h
+++ b/drivers/net/octeontx2/otx2_ethdev.h
@@ -113,9 +113,13 @@
#define CQ_TIMER_THRESH_DEFAULT 0xAULL /* ~1usec i.e (0xA * 100nsec) */
#define CQ_TIMER_THRESH_MAX 255
+#define NIX_RSS_L3_L4_SRC_DST (ETH_RSS_L3_SRC_ONLY | ETH_RSS_L3_DST_ONLY \
+ | ETH_RSS_L4_SRC_ONLY | ETH_RSS_L4_DST_ONLY)
+
#define NIX_RSS_OFFLOAD (ETH_RSS_PORT | ETH_RSS_IP | ETH_RSS_UDP |\
ETH_RSS_TCP | ETH_RSS_SCTP | \
- ETH_RSS_TUNNEL | ETH_RSS_L2_PAYLOAD)
+ ETH_RSS_TUNNEL | ETH_RSS_L2_PAYLOAD | \
+ NIX_RSS_L3_L4_SRC_DST)
#define NIX_TX_OFFLOAD_CAPA ( \
DEV_TX_OFFLOAD_MBUF_FAST_FREE | \
--git a/drivers/net/octeontx2/otx2_rss.c b/drivers/net/octeontx2/otx2_rss.c
index bc7b64387..7a8c8f3de 100644
--- a/drivers/net/octeontx2/otx2_rss.c
+++ b/drivers/net/octeontx2/otx2_rss.c
@@ -210,6 +210,18 @@ otx2_rss_ethdev_to_nix(struct otx2_eth_dev *dev, uint64_t ethdev_rss,
dev->rss_info.nix_rss = ethdev_rss;
+ if (ethdev_rss & ETH_RSS_L3_SRC_ONLY)
+ flowkey_cfg |= FLOW_KEY_TYPE_L3_SRC;
+
+ if (ethdev_rss & ETH_RSS_L3_DST_ONLY)
+ flowkey_cfg |= FLOW_KEY_TYPE_L3_DST;
+
+ if (ethdev_rss & ETH_RSS_L4_SRC_ONLY)
+ flowkey_cfg |= FLOW_KEY_TYPE_L4_SRC;
+
+ if (ethdev_rss & ETH_RSS_L4_DST_ONLY)
+ flowkey_cfg |= FLOW_KEY_TYPE_L4_DST;
+
if (ethdev_rss & RSS_IPV4_ENABLE)
flowkey_cfg |= flow_key_type[rss_level][RSS_IPV4_INDEX];
--
2.17.1
^ permalink raw reply [relevance 3%]
* [dpdk-dev] [PATCH 1/5] lpm: make ipv6 address immutable
@ 2020-01-26 1:09 3% ` Stephen Hemminger
0 siblings, 0 replies; 200+ results
From: Stephen Hemminger @ 2020-01-26 1:09 UTC (permalink / raw)
To: dev; +Cc: Stephen Hemminger
Both the table setup and lookup do no modify their arguments.
Therefore the parameter should be constant.
This is not actually an API breakage since programs can be
recompiled without change. This is not an ABI breakage because
old programs will still run.
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
---
lib/librte_lpm/rte_lpm6.c | 8 ++++----
lib/librte_lpm/rte_lpm6.h | 6 +++---
2 files changed, 7 insertions(+), 7 deletions(-)
diff --git a/lib/librte_lpm/rte_lpm6.c b/lib/librte_lpm/rte_lpm6.c
index c46e557e2389..5ea818abd6e9 100644
--- a/lib/librte_lpm/rte_lpm6.c
+++ b/lib/librte_lpm/rte_lpm6.c
@@ -854,8 +854,8 @@ simulate_add(struct rte_lpm6 *lpm, const uint8_t *masked_ip, uint8_t depth)
* Add a route
*/
int
-rte_lpm6_add(struct rte_lpm6 *lpm, uint8_t *ip, uint8_t depth,
- uint32_t next_hop)
+rte_lpm6_add(struct rte_lpm6 *lpm, const uint8_t *ip, uint8_t depth,
+ uint32_t next_hop)
{
struct rte_lpm6_tbl_entry *tbl;
struct rte_lpm6_tbl_entry *tbl_next = NULL;
@@ -913,7 +913,7 @@ rte_lpm6_add(struct rte_lpm6 *lpm, uint8_t *ip, uint8_t depth,
*/
static inline int
lookup_step(const struct rte_lpm6 *lpm, const struct rte_lpm6_tbl_entry *tbl,
- const struct rte_lpm6_tbl_entry **tbl_next, uint8_t *ip,
+ const struct rte_lpm6_tbl_entry **tbl_next, const uint8_t *ip,
uint8_t first_byte, uint32_t *next_hop)
{
uint32_t tbl8_index, tbl_entry;
@@ -943,7 +943,7 @@ lookup_step(const struct rte_lpm6 *lpm, const struct rte_lpm6_tbl_entry *tbl,
* Looks up an IP
*/
int
-rte_lpm6_lookup(const struct rte_lpm6 *lpm, uint8_t *ip,
+rte_lpm6_lookup(const struct rte_lpm6 *lpm, const uint8_t *ip,
uint32_t *next_hop)
{
const struct rte_lpm6_tbl_entry *tbl;
diff --git a/lib/librte_lpm/rte_lpm6.h b/lib/librte_lpm/rte_lpm6.h
index 37dfb20249a8..042991f8cdb7 100644
--- a/lib/librte_lpm/rte_lpm6.h
+++ b/lib/librte_lpm/rte_lpm6.h
@@ -94,8 +94,8 @@ rte_lpm6_free(struct rte_lpm6 *lpm);
* 0 on success, negative value otherwise
*/
int
-rte_lpm6_add(struct rte_lpm6 *lpm, uint8_t *ip, uint8_t depth,
- uint32_t next_hop);
+rte_lpm6_add(struct rte_lpm6 *lpm, const uint8_t *ip, uint8_t depth,
+ uint32_t next_hop);
/**
* Check if a rule is present in the LPM table,
@@ -171,7 +171,7 @@ rte_lpm6_delete_all(struct rte_lpm6 *lpm);
* -EINVAL for incorrect arguments, -ENOENT on lookup miss, 0 on lookup hit
*/
int
-rte_lpm6_lookup(const struct rte_lpm6 *lpm, uint8_t *ip, uint32_t *next_hop);
+rte_lpm6_lookup(const struct rte_lpm6 *lpm, const uint8_t *ip, uint32_t *next_hop);
/**
* Lookup multiple IP addresses in an LPM table.
--
2.20.1
^ permalink raw reply [relevance 3%]
* Re: [dpdk-dev] [RFC v2 0/7] RFC: Support MACSEC offload in the RTE_SECURITY infrastructure.
@ 2020-01-27 11:25 0% ` Akhil Goyal
0 siblings, 0 replies; 200+ results
From: Akhil Goyal @ 2020-01-27 11:25 UTC (permalink / raw)
To: Pavel Belous, dev
Cc: Ferruh Yigit, John McNamara, Declan Doherty, Konstantin Ananyev,
Thomas Monjalon, Igor Russkikh, Fenilkumar Patel,
Hitesh K Maisheri
Hi Pavel,
> From: Pavel Belous <Pavel.Belous@aquantia.com>
>
> This RFC suggest possible API to implement generic MACSEC HW
> offload in DPDK infrastructure.
>
> Right now two PMDs implementing MACSEC hw offload via private
> API: ixgbe (Intel) and atlantic (Aquantia).
>
> During that private API discussion it was decided to go further
> with well defined public API, based most probably on rte_security
> infrastructure.
>
> Here is that previous discussion:
>
> http://inbox.dpdk.org/dev/20190416101145.nVecHKp3w14Ptd_hne-DqHhKyzbre88PwNI-OAowXJM@z/
>
> Declaring macsec API via rte_security gives a good data-centric view on
> parameters
> and operations macsec supports. Old, pure functional API (basically ixbe only API)
> presented function calls with big argument lists which is hard to extend and
> analyse.
>
> However, I'd like to note rte_security has to be used via explicitly created
> mempools - this hardens abit the usage.
> It also may be hard to extend the structures in the ABI compatible way.
>
> One of the problems with MACSEC is that internally implementation and
> hardware
> support could be either very simple, doing only endpoint encryption with a single
> TX SC (Secure Connection), or quite complex, capable to do flexible filtering
> and SC matching based on mac, vlan, ethertype and other.
>
> Different macsec hardware supports some custom features and from our
> experience
> users would like to configure these as well. Therefore there will probably be
> needed a number of PMD specific macsec operators support.
>
> Examples include: custom in-the-clear tag (matched by vlan id or mask),
> configurable internal logic to allow both secure and unsecure traffic,
> bypass filters on specific ethertypes.
> To support such extensions, suggest use rte_security_macsec_op enum with
> vendor specific operation codes.
>
> In context of rte_security, MACSEC operations should normally be based on
> security session create and update calls.
>
> Session create is used to setup overall session. Thats equivalent of old
> `macsec enable` operation.
>
> Session update is used to update security connections and associations.
> Here xform->op contains the required operation: rx/tx session/association
> add/update/removal.
>
The patches look good from rte_security perspective. You can send the formal
Patches for 20.05 window.
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH v2] net/octeontx2: extend RSS supported offload types
2020-01-24 3:46 3% ` [dpdk-dev] [PATCH v2] " kirankumark
@ 2020-01-28 11:14 0% ` Jerin Jacob
0 siblings, 0 replies; 200+ results
From: Jerin Jacob @ 2020-01-28 11:14 UTC (permalink / raw)
To: Kiran Kumar K; +Cc: Jerin Jacob, Nithin Dabilpuram, Vamsi Attunuru, dpdk-dev
On Fri, Jan 24, 2020 at 9:16 AM <kirankumark@marvell.com> wrote:
>
> From: Kiran Kumar K <kirankumark@marvell.com>
>
> Extend RSS offload types for octeontx2. Add support to select
> L3 SRC, L3 DST, L4 SRC and L4 DST for RSS calculation.
>
> Add support to select L3 SRC or DST only, L4 SRC or DST only for RSS
> calculation.
>
> With this requirement there will be following combinations,
> IPV[4,6]_SRC_ONLY, IPV[4,6]_DST_ONLY, [TCP,UDP,SCTP]_SRC_ONLY,
> [TCP,UDP,SCTP]_DST_ONLY. So, instead of creating a bit for each
> combination, we are using upper 4 bits (31:28) in the flow_key_cfg
> to represent the SRC, DST selection. 31 => L3_SRC, 30 => L3_DST,
> 29 => L4_SRC, 28 => L4_DST. These won't be part of flow_cfg, so that
> we don't need to change the existing ABI.
>
> Signed-off-by: Kiran Kumar K <kirankumark@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>
Applied to dpdk-next-net-mrvl/master. Thanks
> ---
> V2 Changes:
> * Updated supported offloads
>
> drivers/common/octeontx2/otx2_mbox.h | 6 +++++-
> drivers/net/octeontx2/otx2_ethdev.h | 6 +++++-
> drivers/net/octeontx2/otx2_rss.c | 12 ++++++++++++
> 3 files changed, 22 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/common/octeontx2/otx2_mbox.h b/drivers/common/octeontx2/otx2_mbox.h
> index e0e4e2f63..4972b8a6e 100644
> --- a/drivers/common/octeontx2/otx2_mbox.h
> +++ b/drivers/common/octeontx2/otx2_mbox.h
> @@ -918,7 +918,11 @@ struct nix_rss_flowkey_cfg {
> #define FLOW_KEY_TYPE_INNR_UDP BIT(15)
> #define FLOW_KEY_TYPE_INNR_SCTP BIT(16)
> #define FLOW_KEY_TYPE_INNR_ETH_DMAC BIT(17)
> - uint8_t group; /* RSS context or group */
> +#define FLOW_KEY_TYPE_L4_DST BIT(28)
> +#define FLOW_KEY_TYPE_L4_SRC BIT(29)
> +#define FLOW_KEY_TYPE_L3_DST BIT(30)
> +#define FLOW_KEY_TYPE_L3_SRC BIT(31)
> + uint8_t __otx2_io group; /* RSS context or group */
> };
>
> struct nix_rss_flowkey_cfg_rsp {
> diff --git a/drivers/net/octeontx2/otx2_ethdev.h b/drivers/net/octeontx2/otx2_ethdev.h
> index 3f3fdecc4..c075b8d1a 100644
> --- a/drivers/net/octeontx2/otx2_ethdev.h
> +++ b/drivers/net/octeontx2/otx2_ethdev.h
> @@ -113,9 +113,13 @@
> #define CQ_TIMER_THRESH_DEFAULT 0xAULL /* ~1usec i.e (0xA * 100nsec) */
> #define CQ_TIMER_THRESH_MAX 255
>
> +#define NIX_RSS_L3_L4_SRC_DST (ETH_RSS_L3_SRC_ONLY | ETH_RSS_L3_DST_ONLY \
> + | ETH_RSS_L4_SRC_ONLY | ETH_RSS_L4_DST_ONLY)
> +
> #define NIX_RSS_OFFLOAD (ETH_RSS_PORT | ETH_RSS_IP | ETH_RSS_UDP |\
> ETH_RSS_TCP | ETH_RSS_SCTP | \
> - ETH_RSS_TUNNEL | ETH_RSS_L2_PAYLOAD)
> + ETH_RSS_TUNNEL | ETH_RSS_L2_PAYLOAD | \
> + NIX_RSS_L3_L4_SRC_DST)
>
> #define NIX_TX_OFFLOAD_CAPA ( \
> DEV_TX_OFFLOAD_MBUF_FAST_FREE | \
> diff --git a/drivers/net/octeontx2/otx2_rss.c b/drivers/net/octeontx2/otx2_rss.c
> index bc7b64387..7a8c8f3de 100644
> --- a/drivers/net/octeontx2/otx2_rss.c
> +++ b/drivers/net/octeontx2/otx2_rss.c
> @@ -210,6 +210,18 @@ otx2_rss_ethdev_to_nix(struct otx2_eth_dev *dev, uint64_t ethdev_rss,
>
> dev->rss_info.nix_rss = ethdev_rss;
>
> + if (ethdev_rss & ETH_RSS_L3_SRC_ONLY)
> + flowkey_cfg |= FLOW_KEY_TYPE_L3_SRC;
> +
> + if (ethdev_rss & ETH_RSS_L3_DST_ONLY)
> + flowkey_cfg |= FLOW_KEY_TYPE_L3_DST;
> +
> + if (ethdev_rss & ETH_RSS_L4_SRC_ONLY)
> + flowkey_cfg |= FLOW_KEY_TYPE_L4_SRC;
> +
> + if (ethdev_rss & ETH_RSS_L4_DST_ONLY)
> + flowkey_cfg |= FLOW_KEY_TYPE_L4_DST;
> +
> if (ethdev_rss & RSS_IPV4_ENABLE)
> flowkey_cfg |= flow_key_type[rss_level][RSS_IPV4_INDEX];
>
> --
> 2.17.1
>
^ permalink raw reply [relevance 0%]
* [dpdk-dev] [RFC] meter: fix ABI break due to experimental tag removal
@ 2020-01-29 12:29 3% Ferruh Yigit
2020-01-29 14:46 4% ` Bruce Richardson
` (3 more replies)
0 siblings, 4 replies; 200+ results
From: Ferruh Yigit @ 2020-01-29 12:29 UTC (permalink / raw)
To: Neil Horman, Cristian Dumitrescu, Eelco Chaudron
Cc: dev, Thomas Monjalon, Luca Boccassi, David Marchand,
Bruce Richardson, Ian Stokes
Duplicated the existing symbol and versioned one as experimental and
other as stable.
Created VERSION_SYMBOL_EXPERIMENTAL helper macro.
Updated the 'check-experimental-syms.sh' buildtool, which was
complaining that the symbol is in EXPERIMENTAL tag in .map file but it
is not in the .experimental section (__rte_experimental tag is missing).
Updated tool in a way it won't complain if the symbol in the
EXPERIMENTAL tag duplicated in some other block in .map file (versioned)
Updated meson build system to allow the versioning,
'use_function_versioning = true', not sure why it was disabled by
default.
Fixes: 30512af820fe ("meter: remove experimental flag from RFC4115 trTCM API")
Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
---
Cc: Neil Horman <nhorman@tuxdriver.com>
Cc: Thomas Monjalon <thomas@monjalon.net>
Cc: Luca Boccassi <bluca@debian.org>
Cc: David Marchand <david.marchand@redhat.com>
Cc: Bruce Richardson <bruce.richardson@intel.com>
Cc: Ian Stokes <ian.stokes@intel.com>
Cc: Eelco Chaudron <echaudro@redhat.com>
I didn't like the idea of duplicating the symbol but not able to find to
alias it without duplicating, if there is a way please share.
Also not tested with old binaries, only verified the symbols in new
binaries.
---
buildtools/check-experimental-syms.sh | 3 +-
.../common/include/rte_function_versioning.h | 4 ++
lib/librte_meter/rte_meter.c | 59 ++++++++++++++++++-
lib/librte_meter/rte_meter_version.map | 8 +++
lib/meson.build | 2 +-
5 files changed, 71 insertions(+), 5 deletions(-)
diff --git a/buildtools/check-experimental-syms.sh b/buildtools/check-experimental-syms.sh
index f3603e5ba..35c4bf006 100755
--- a/buildtools/check-experimental-syms.sh
+++ b/buildtools/check-experimental-syms.sh
@@ -26,7 +26,8 @@ ret=0
for SYM in `$LIST_SYMBOL -S EXPERIMENTAL $MAPFILE |cut -d ' ' -f 3`
do
if grep -q "\.text.*[[:space:]]$SYM$" $DUMPFILE &&
- ! grep -q "\.text\.experimental.*[[:space:]]$SYM$" $DUMPFILE
+ ! grep -q "\.text\.experimental.*[[:space:]]$SYM$" $DUMPFILE &&
+ $LIST_SYMBOL -s $SYM $MAPFILE | grep -q EXPERIMENTAL
then
cat >&2 <<- END_OF_MESSAGE
$SYM is not flagged as experimental
diff --git a/lib/librte_eal/common/include/rte_function_versioning.h b/lib/librte_eal/common/include/rte_function_versioning.h
index c924351d5..ab102b06e 100644
--- a/lib/librte_eal/common/include/rte_function_versioning.h
+++ b/lib/librte_eal/common/include/rte_function_versioning.h
@@ -46,6 +46,9 @@
*/
#define VERSION_SYMBOL(b, e, n) __asm__(".symver " RTE_STR(b) RTE_STR(e) ", " RTE_STR(b) "@DPDK_" RTE_STR(n))
+
+#define VERSION_SYMBOL_EXPERIMENTAL(b, e) __asm__(".symver " RTE_STR(b) RTE_STR(e) ", " RTE_STR(b) "@EXPERIMENTAL")
+
/*
* BIND_DEFAULT_SYMBOL
* Creates a symbol version entry instructing the linker to bind references to
@@ -79,6 +82,7 @@
* No symbol versioning in use
*/
#define VERSION_SYMBOL(b, e, n)
+#define VERSION_SYMBOL_EXPERIMENTAL(b, e)
#define __vsym
#define BIND_DEFAULT_SYMBOL(b, e, n)
#define MAP_STATIC_SYMBOL(f, p) f __attribute__((alias(RTE_STR(p))))
diff --git a/lib/librte_meter/rte_meter.c b/lib/librte_meter/rte_meter.c
index da01429a8..5244537fa 100644
--- a/lib/librte_meter/rte_meter.c
+++ b/lib/librte_meter/rte_meter.c
@@ -9,6 +9,7 @@
#include <rte_common.h>
#include <rte_log.h>
#include <rte_cycles.h>
+#include <rte_function_versioning.h>
#include "rte_meter.h"
@@ -119,8 +120,8 @@ rte_meter_trtcm_config(struct rte_meter_trtcm *m,
return 0;
}
-int
-rte_meter_trtcm_rfc4115_profile_config(
+static int
+rte_meter_trtcm_rfc4115_profile_config_(
struct rte_meter_trtcm_rfc4115_profile *p,
struct rte_meter_trtcm_rfc4115_params *params)
{
@@ -145,7 +146,35 @@ rte_meter_trtcm_rfc4115_profile_config(
}
int
-rte_meter_trtcm_rfc4115_config(
+rte_meter_trtcm_rfc4115_profile_config_s(
+ struct rte_meter_trtcm_rfc4115_profile *p,
+ struct rte_meter_trtcm_rfc4115_params *params);
+int
+rte_meter_trtcm_rfc4115_profile_config_s(
+ struct rte_meter_trtcm_rfc4115_profile *p,
+ struct rte_meter_trtcm_rfc4115_params *params)
+{
+ return rte_meter_trtcm_rfc4115_profile_config_(p, params);
+}
+BIND_DEFAULT_SYMBOL(rte_meter_trtcm_rfc4115_profile_config, _s, 20.0.1);
+MAP_STATIC_SYMBOL(int rte_meter_trtcm_rfc4115_profile_config(struct rte_meter_trtcm_rfc4115_profile *p,
+ struct rte_meter_trtcm_rfc4115_params *params), rte_meter_trtcm_rfc4115_profile_config_s);
+
+int
+rte_meter_trtcm_rfc4115_profile_config_e(
+ struct rte_meter_trtcm_rfc4115_profile *p,
+ struct rte_meter_trtcm_rfc4115_params *params);
+int
+rte_meter_trtcm_rfc4115_profile_config_e(
+ struct rte_meter_trtcm_rfc4115_profile *p,
+ struct rte_meter_trtcm_rfc4115_params *params)
+{
+ return rte_meter_trtcm_rfc4115_profile_config_(p, params);
+}
+VERSION_SYMBOL_EXPERIMENTAL(rte_meter_trtcm_rfc4115_profile_config, _e);
+
+static int
+rte_meter_trtcm_rfc4115_config_(
struct rte_meter_trtcm_rfc4115 *m,
struct rte_meter_trtcm_rfc4115_profile *p)
{
@@ -160,3 +189,27 @@ rte_meter_trtcm_rfc4115_config(
return 0;
}
+
+int
+rte_meter_trtcm_rfc4115_config_s(struct rte_meter_trtcm_rfc4115 *m,
+ struct rte_meter_trtcm_rfc4115_profile *p);
+int
+rte_meter_trtcm_rfc4115_config_s(struct rte_meter_trtcm_rfc4115 *m,
+ struct rte_meter_trtcm_rfc4115_profile *p)
+{
+ return rte_meter_trtcm_rfc4115_config_(m, p);
+}
+BIND_DEFAULT_SYMBOL(rte_meter_trtcm_rfc4115_config, _s, 20.0.1);
+MAP_STATIC_SYMBOL(int rte_meter_trtcm_rfc4115_config(struct rte_meter_trtcm_rfc4115 *m,
+ struct rte_meter_trtcm_rfc4115_profile *p), rte_meter_trtcm_rfc4115_config_s);
+
+int
+rte_meter_trtcm_rfc4115_config_e(struct rte_meter_trtcm_rfc4115 *m,
+ struct rte_meter_trtcm_rfc4115_profile *p);
+int
+rte_meter_trtcm_rfc4115_config_e(struct rte_meter_trtcm_rfc4115 *m,
+ struct rte_meter_trtcm_rfc4115_profile *p)
+{
+ return rte_meter_trtcm_rfc4115_config_(m, p);
+}
+VERSION_SYMBOL_EXPERIMENTAL(rte_meter_trtcm_rfc4115_config, _e);
diff --git a/lib/librte_meter/rte_meter_version.map b/lib/librte_meter/rte_meter_version.map
index fc12cc0bf..b4b043174 100644
--- a/lib/librte_meter/rte_meter_version.map
+++ b/lib/librte_meter/rte_meter_version.map
@@ -20,4 +20,12 @@ DPDK_20.0.1 {
rte_meter_trtcm_rfc4115_color_blind_check;
rte_meter_trtcm_rfc4115_config;
rte_meter_trtcm_rfc4115_profile_config;
+
} DPDK_20.0;
+
+EXPERIMENTAL {
+ global:
+
+ rte_meter_trtcm_rfc4115_config;
+ rte_meter_trtcm_rfc4115_profile_config;
+};
diff --git a/lib/meson.build b/lib/meson.build
index 0af3efab2..553964831 100644
--- a/lib/meson.build
+++ b/lib/meson.build
@@ -48,7 +48,7 @@ foreach l:libraries
reason = '<unknown reason>' # set if build == false to explain why
name = l
allow_experimental_apis = false
- use_function_versioning = false
+ use_function_versioning = true
sources = []
headers = []
includes = []
--
2.24.1
^ permalink raw reply [relevance 3%]
* Re: [dpdk-dev] [RFC] meter: fix ABI break due to experimental tag removal
2020-01-29 12:29 3% [dpdk-dev] [RFC] meter: fix ABI break due to experimental tag removal Ferruh Yigit
@ 2020-01-29 14:46 4% ` Bruce Richardson
2020-01-29 14:57 4% ` Andrzej Ostruszka
` (2 subsequent siblings)
3 siblings, 0 replies; 200+ results
From: Bruce Richardson @ 2020-01-29 14:46 UTC (permalink / raw)
To: Ferruh Yigit
Cc: Neil Horman, Cristian Dumitrescu, Eelco Chaudron, dev,
Thomas Monjalon, Luca Boccassi, David Marchand, Ian Stokes
On Wed, Jan 29, 2020 at 12:29:53PM +0000, Ferruh Yigit wrote:
> Duplicated the existing symbol and versioned one as experimental and
> other as stable.
>
> Created VERSION_SYMBOL_EXPERIMENTAL helper macro.
>
> Updated the 'check-experimental-syms.sh' buildtool, which was
> complaining that the symbol is in EXPERIMENTAL tag in .map file but it
> is not in the .experimental section (__rte_experimental tag is missing).
> Updated tool in a way it won't complain if the symbol in the
> EXPERIMENTAL tag duplicated in some other block in .map file (versioned)
>
> Updated meson build system to allow the versioning,
> 'use_function_versioning = true', not sure why it was disabled by
> default.
>
Because when enabled everything in the library must be built twice - once
for static lib and differently for a dynamic lib. Therefore unless a
library actually needs versioned symbols, we only build everything once to
save on build time.
/Bruce
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [RFC] meter: fix ABI break due to experimental tag removal
2020-01-29 12:29 3% [dpdk-dev] [RFC] meter: fix ABI break due to experimental tag removal Ferruh Yigit
2020-01-29 14:46 4% ` Bruce Richardson
@ 2020-01-29 14:57 4% ` Andrzej Ostruszka
2020-01-29 16:25 4% ` Ferruh Yigit
2020-01-29 16:43 3% ` [dpdk-dev] [RFC v2] " Ferruh Yigit
2020-02-02 18:53 7% ` [dpdk-dev] [RFC] " Neil Horman
3 siblings, 1 reply; 200+ results
From: Andrzej Ostruszka @ 2020-01-29 14:57 UTC (permalink / raw)
To: dev
On 1/29/20 1:29 PM, Ferruh Yigit wrote:
[...]
> Updated meson build system to allow the versioning,
> 'use_function_versioning = true', not sure why it was disabled by
> default.
AFAIR this is to save build time with meson. By default static build is
made and from the objects from the static build shared library is
constructed. This works unless function versioning is used - because
for the static build "attribute alias" is used and for shared build
.symver is used. So 'use_function_versioning' by default is false and
only libraries that actually use them have it set to true (and perform
separate shared build - see lib/meson.build).
I've now noticed that LPM, Distributor and Timer libraries no longer use
function versioning but have it still set in their meson.build.
With regards
Andrzej Ostruszka
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [RFC] meter: fix ABI break due to experimental tag removal
2020-01-29 14:57 4% ` Andrzej Ostruszka
@ 2020-01-29 16:25 4% ` Ferruh Yigit
2020-01-29 16:30 4% ` Andrzej Ostruszka
0 siblings, 1 reply; 200+ results
From: Ferruh Yigit @ 2020-01-29 16:25 UTC (permalink / raw)
To: Andrzej Ostruszka, dev
On 1/29/2020 2:57 PM, Andrzej Ostruszka wrote:
> On 1/29/20 1:29 PM, Ferruh Yigit wrote:
> [...]
>> Updated meson build system to allow the versioning,
>> 'use_function_versioning = true', not sure why it was disabled by
>> default.
>
> AFAIR this is to save build time with meson. By default static build is
> made and from the objects from the static build shared library is
> constructed. This works unless function versioning is used - because
> for the static build "attribute alias" is used and for shared build
> .symver is used. So 'use_function_versioning' by default is false and
> only libraries that actually use them have it set to true (and perform
> separate shared build - see lib/meson.build).
Hi Andrzej,
Thanks for clarification, I will reduce it to the library.
>
> I've now noticed that LPM, Distributor and Timer libraries no longer use
> function versioning but have it still set in their meson.build.
Right, I can remove them.
>
> With regards
> Andrzej Ostruszka
>
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [RFC] meter: fix ABI break due to experimental tag removal
2020-01-29 16:25 4% ` Ferruh Yigit
@ 2020-01-29 16:30 4% ` Andrzej Ostruszka
2020-01-29 16:40 4% ` Ferruh Yigit
0 siblings, 1 reply; 200+ results
From: Andrzej Ostruszka @ 2020-01-29 16:30 UTC (permalink / raw)
To: Ferruh Yigit, dev
On 1/29/20 5:25 PM, Ferruh Yigit wrote:
> On 1/29/2020 2:57 PM, Andrzej Ostruszka wrote:
[...]
>> I've now noticed that LPM, Distributor and Timer libraries no longer use
>> function versioning but have it still set in their meson.build.
>
> Right, I can remove them.
Bruce was kind enough to give me a nudge for that :).
If you decide to do it in one patch let me know so that I can drop my
patch in patchwork.
With regards
Andrzej Ostruszka
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [RFC] meter: fix ABI break due to experimental tag removal
2020-01-29 16:30 4% ` Andrzej Ostruszka
@ 2020-01-29 16:40 4% ` Ferruh Yigit
0 siblings, 0 replies; 200+ results
From: Ferruh Yigit @ 2020-01-29 16:40 UTC (permalink / raw)
To: Andrzej Ostruszka, dev
On 1/29/2020 4:30 PM, Andrzej Ostruszka wrote:
> On 1/29/20 5:25 PM, Ferruh Yigit wrote:
>> On 1/29/2020 2:57 PM, Andrzej Ostruszka wrote:
> [...]
>>> I've now noticed that LPM, Distributor and Timer libraries no longer use
>>> function versioning but have it still set in their meson.build.
>>
>> Right, I can remove them.
>
> Bruce was kind enough to give me a nudge for that :).
> If you decide to do it in one patch let me know so that I can drop my
> patch in patchwork.
Separate patch makes more sense, thanks for the patch.
^ permalink raw reply [relevance 4%]
* [dpdk-dev] [RFC v2] meter: fix ABI break due to experimental tag removal
2020-01-29 12:29 3% [dpdk-dev] [RFC] meter: fix ABI break due to experimental tag removal Ferruh Yigit
2020-01-29 14:46 4% ` Bruce Richardson
2020-01-29 14:57 4% ` Andrzej Ostruszka
@ 2020-01-29 16:43 3% ` Ferruh Yigit
2020-01-29 17:49 4% ` Andrzej Ostruszka
2020-01-30 12:33 7% ` Thomas Monjalon
2020-02-02 18:53 7% ` [dpdk-dev] [RFC] " Neil Horman
3 siblings, 2 replies; 200+ results
From: Ferruh Yigit @ 2020-01-29 16:43 UTC (permalink / raw)
To: Neil Horman, Cristian Dumitrescu, Eelco Chaudron
Cc: dev, Thomas Monjalon, Luca Boccassi, David Marchand,
Bruce Richardson, Ian Stokes, Andrzej Ostruszka
Duplicated the existing symbol and versioned one as experimental and
other as stable.
Created VERSION_SYMBOL_EXPERIMENTAL helper macro.
Updated the 'check-experimental-syms.sh' buildtool, which was
complaining that the symbol is in EXPERIMENTAL tag in .map file but it
is not in the .experimental section (__rte_experimental tag is missing).
Updated tool in a way it won't complain if the symbol in the
EXPERIMENTAL tag duplicated in some other block in .map file (versioned)
Enabled function versioning for meson build for the library.
Fixes: 30512af820fe ("meter: remove experimental flag from RFC4115 trTCM API")
Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
---
Cc: Neil Horman <nhorman@tuxdriver.com>
Cc: Thomas Monjalon <thomas@monjalon.net>
Cc: Luca Boccassi <bluca@debian.org>
Cc: David Marchand <david.marchand@redhat.com>
Cc: Bruce Richardson <bruce.richardson@intel.com>
Cc: Ian Stokes <ian.stokes@intel.com>
Cc: Eelco Chaudron <echaudro@redhat.com>
Cc: Andrzej Ostruszka <amo@semihalf.com>
v2:
* allow versioning only for meter library, instead of enabling it
globally
I didn't like the idea of duplicating the symbol but not able to find to
alias it without duplicating, if there is a way please share.
Also not tested with old binaries, only verified the symbols in new
binaries.
---
buildtools/check-experimental-syms.sh | 3 +-
.../common/include/rte_function_versioning.h | 4 ++
lib/librte_meter/meson.build | 1 +
lib/librte_meter/rte_meter.c | 59 ++++++++++++++++++-
lib/librte_meter/rte_meter_version.map | 8 +++
5 files changed, 71 insertions(+), 4 deletions(-)
diff --git a/buildtools/check-experimental-syms.sh b/buildtools/check-experimental-syms.sh
index f3603e5ba..35c4bf006 100755
--- a/buildtools/check-experimental-syms.sh
+++ b/buildtools/check-experimental-syms.sh
@@ -26,7 +26,8 @@ ret=0
for SYM in `$LIST_SYMBOL -S EXPERIMENTAL $MAPFILE |cut -d ' ' -f 3`
do
if grep -q "\.text.*[[:space:]]$SYM$" $DUMPFILE &&
- ! grep -q "\.text\.experimental.*[[:space:]]$SYM$" $DUMPFILE
+ ! grep -q "\.text\.experimental.*[[:space:]]$SYM$" $DUMPFILE &&
+ $LIST_SYMBOL -s $SYM $MAPFILE | grep -q EXPERIMENTAL
then
cat >&2 <<- END_OF_MESSAGE
$SYM is not flagged as experimental
diff --git a/lib/librte_eal/common/include/rte_function_versioning.h b/lib/librte_eal/common/include/rte_function_versioning.h
index c924351d5..ab102b06e 100644
--- a/lib/librte_eal/common/include/rte_function_versioning.h
+++ b/lib/librte_eal/common/include/rte_function_versioning.h
@@ -46,6 +46,9 @@
*/
#define VERSION_SYMBOL(b, e, n) __asm__(".symver " RTE_STR(b) RTE_STR(e) ", " RTE_STR(b) "@DPDK_" RTE_STR(n))
+
+#define VERSION_SYMBOL_EXPERIMENTAL(b, e) __asm__(".symver " RTE_STR(b) RTE_STR(e) ", " RTE_STR(b) "@EXPERIMENTAL")
+
/*
* BIND_DEFAULT_SYMBOL
* Creates a symbol version entry instructing the linker to bind references to
@@ -79,6 +82,7 @@
* No symbol versioning in use
*/
#define VERSION_SYMBOL(b, e, n)
+#define VERSION_SYMBOL_EXPERIMENTAL(b, e)
#define __vsym
#define BIND_DEFAULT_SYMBOL(b, e, n)
#define MAP_STATIC_SYMBOL(f, p) f __attribute__((alias(RTE_STR(p))))
diff --git a/lib/librte_meter/meson.build b/lib/librte_meter/meson.build
index 646fd4d43..fce036843 100644
--- a/lib/librte_meter/meson.build
+++ b/lib/librte_meter/meson.build
@@ -3,3 +3,4 @@
sources = files('rte_meter.c')
headers = files('rte_meter.h')
+use_function_versioning = true
diff --git a/lib/librte_meter/rte_meter.c b/lib/librte_meter/rte_meter.c
index da01429a8..5244537fa 100644
--- a/lib/librte_meter/rte_meter.c
+++ b/lib/librte_meter/rte_meter.c
@@ -9,6 +9,7 @@
#include <rte_common.h>
#include <rte_log.h>
#include <rte_cycles.h>
+#include <rte_function_versioning.h>
#include "rte_meter.h"
@@ -119,8 +120,8 @@ rte_meter_trtcm_config(struct rte_meter_trtcm *m,
return 0;
}
-int
-rte_meter_trtcm_rfc4115_profile_config(
+static int
+rte_meter_trtcm_rfc4115_profile_config_(
struct rte_meter_trtcm_rfc4115_profile *p,
struct rte_meter_trtcm_rfc4115_params *params)
{
@@ -145,7 +146,35 @@ rte_meter_trtcm_rfc4115_profile_config(
}
int
-rte_meter_trtcm_rfc4115_config(
+rte_meter_trtcm_rfc4115_profile_config_s(
+ struct rte_meter_trtcm_rfc4115_profile *p,
+ struct rte_meter_trtcm_rfc4115_params *params);
+int
+rte_meter_trtcm_rfc4115_profile_config_s(
+ struct rte_meter_trtcm_rfc4115_profile *p,
+ struct rte_meter_trtcm_rfc4115_params *params)
+{
+ return rte_meter_trtcm_rfc4115_profile_config_(p, params);
+}
+BIND_DEFAULT_SYMBOL(rte_meter_trtcm_rfc4115_profile_config, _s, 20.0.1);
+MAP_STATIC_SYMBOL(int rte_meter_trtcm_rfc4115_profile_config(struct rte_meter_trtcm_rfc4115_profile *p,
+ struct rte_meter_trtcm_rfc4115_params *params), rte_meter_trtcm_rfc4115_profile_config_s);
+
+int
+rte_meter_trtcm_rfc4115_profile_config_e(
+ struct rte_meter_trtcm_rfc4115_profile *p,
+ struct rte_meter_trtcm_rfc4115_params *params);
+int
+rte_meter_trtcm_rfc4115_profile_config_e(
+ struct rte_meter_trtcm_rfc4115_profile *p,
+ struct rte_meter_trtcm_rfc4115_params *params)
+{
+ return rte_meter_trtcm_rfc4115_profile_config_(p, params);
+}
+VERSION_SYMBOL_EXPERIMENTAL(rte_meter_trtcm_rfc4115_profile_config, _e);
+
+static int
+rte_meter_trtcm_rfc4115_config_(
struct rte_meter_trtcm_rfc4115 *m,
struct rte_meter_trtcm_rfc4115_profile *p)
{
@@ -160,3 +189,27 @@ rte_meter_trtcm_rfc4115_config(
return 0;
}
+
+int
+rte_meter_trtcm_rfc4115_config_s(struct rte_meter_trtcm_rfc4115 *m,
+ struct rte_meter_trtcm_rfc4115_profile *p);
+int
+rte_meter_trtcm_rfc4115_config_s(struct rte_meter_trtcm_rfc4115 *m,
+ struct rte_meter_trtcm_rfc4115_profile *p)
+{
+ return rte_meter_trtcm_rfc4115_config_(m, p);
+}
+BIND_DEFAULT_SYMBOL(rte_meter_trtcm_rfc4115_config, _s, 20.0.1);
+MAP_STATIC_SYMBOL(int rte_meter_trtcm_rfc4115_config(struct rte_meter_trtcm_rfc4115 *m,
+ struct rte_meter_trtcm_rfc4115_profile *p), rte_meter_trtcm_rfc4115_config_s);
+
+int
+rte_meter_trtcm_rfc4115_config_e(struct rte_meter_trtcm_rfc4115 *m,
+ struct rte_meter_trtcm_rfc4115_profile *p);
+int
+rte_meter_trtcm_rfc4115_config_e(struct rte_meter_trtcm_rfc4115 *m,
+ struct rte_meter_trtcm_rfc4115_profile *p)
+{
+ return rte_meter_trtcm_rfc4115_config_(m, p);
+}
+VERSION_SYMBOL_EXPERIMENTAL(rte_meter_trtcm_rfc4115_config, _e);
diff --git a/lib/librte_meter/rte_meter_version.map b/lib/librte_meter/rte_meter_version.map
index fc12cc0bf..b4b043174 100644
--- a/lib/librte_meter/rte_meter_version.map
+++ b/lib/librte_meter/rte_meter_version.map
@@ -20,4 +20,12 @@ DPDK_20.0.1 {
rte_meter_trtcm_rfc4115_color_blind_check;
rte_meter_trtcm_rfc4115_config;
rte_meter_trtcm_rfc4115_profile_config;
+
} DPDK_20.0;
+
+EXPERIMENTAL {
+ global:
+
+ rte_meter_trtcm_rfc4115_config;
+ rte_meter_trtcm_rfc4115_profile_config;
+};
--
2.24.1
^ permalink raw reply [relevance 3%]
* [dpdk-dev] [PATCH v2 0/4] add ABI checks
@ 2020-01-29 17:26 9% ` David Marchand
2020-01-29 17:26 2% ` [dpdk-dev] [PATCH v2 3/4] build: test meson installation David Marchand
` (2 more replies)
2020-01-30 16:00 9% ` [dpdk-dev] [PATCH v3 " David Marchand
2020-02-02 21:08 9% ` [dpdk-dev] [PATCH v4 0/3] " David Marchand
3 siblings, 3 replies; 200+ results
From: David Marchand @ 2020-01-29 17:26 UTC (permalink / raw)
To: dev
Cc: thomas, bruce.richardson, kevin.laatz, aconole, nhorman,
akhil.goyal, anoobj
Here is the current state of the ABI checks.
libabigail has some issues when mixing dump and so files compiled with
clang [1], so for now, all checks are done on dumps only.
libabigail 1.0-rc3 in Xenial reported issues that disappear with the
version 1.2 in Bionic.
To avoid getting warnings on internal types like [2], the checks now make
use of the public headers part of a dpdk installation (patch 2 and 3 to
prepare for this).
Some internal rte_hash headers were installed by meson, so patch 1 fixes
this.
The most important point, abidiff complains on the rc1 cryptodev changes:
- Chacha20-Poly1305 AEAD support,
- ECPM and ECDSA support
A suppression rule has been put on the internal type rte_cryptodev_ops.
But other changes are an ABI breakage afaiu. I put suppression rules on
them so that we can run the checks, but some action must be taken for
20.02 if my analysis is confirmed.
Special thanks to Dodji the libabigail maintainer for helping on this
topic.
1: https://sourceware.org/bugzilla/show_bug.cgi?id=25409
2: http://inbox.dpdk.org/dev/CAJFAV8yFKoDZROX9Mkyp7pDMvXw3e7mHwxjfrcjD5ZoFB2tZ8w@mail.gmail.com/
--
David Marchand
David Marchand (4):
hash: fix meson headers packaging
build: split build helper
build: test meson installation
add ABI checks
.ci/linux-build.sh | 29 ++++++++-
.travis.yml | 20 +++++-
MAINTAINERS | 2 +
devtools/check-abi.sh | 59 +++++++++++++++++
devtools/dpdk.abignore | 20 ++++++
devtools/gen-abi.sh | 26 ++++++++
devtools/test-build.sh | 45 ++++++++++++-
devtools/test-meson-builds.sh | 98 ++++++++++++++++++++++-------
devtools/test-null.sh | 1 +
doc/guides/contributing/patches.rst | 13 ++++
lib/librte_hash/meson.build | 5 +-
11 files changed, 286 insertions(+), 32 deletions(-)
create mode 100755 devtools/check-abi.sh
create mode 100644 devtools/dpdk.abignore
create mode 100755 devtools/gen-abi.sh
--
2.23.0
^ permalink raw reply [relevance 9%]
* [dpdk-dev] [PATCH v2 3/4] build: test meson installation
2020-01-29 17:26 9% ` [dpdk-dev] [PATCH v2 0/4] " David Marchand
@ 2020-01-29 17:26 2% ` David Marchand
2020-01-29 17:26 28% ` [dpdk-dev] [PATCH v2 4/4] add ABI checks David Marchand
2020-01-30 10:57 4% ` [dpdk-dev] [PATCH v2 0/4] " Luca Boccassi
2 siblings, 0 replies; 200+ results
From: David Marchand @ 2020-01-29 17:26 UTC (permalink / raw)
To: dev
Cc: thomas, bruce.richardson, kevin.laatz, aconole, nhorman,
akhil.goyal, anoobj, Michael Santana, Ferruh Yigit,
Andrew Rybchenko
Let's test installing with meson as part of Travis and the
devtools/test-meson-builds.sh script.
The resulting headers and binaries make more sense to run checks
against, since they should be the same as what the final users get.
In this patch, test-null.sh is now called on an installed testpmd.
For this to work LD_LIBRARY_PATH must be set appropriately, but it can
be hard to debug, so a call to ldd has been added.
The (future) ABI compatibility checks will also need to filter its report
against installed public headers.
Signed-off-by: David Marchand <david.marchand@redhat.com>
---
.ci/linux-build.sh | 6 +++++-
devtools/test-meson-builds.sh | 27 ++++++++++++++++-----------
devtools/test-null.sh | 1 +
3 files changed, 22 insertions(+), 12 deletions(-)
diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh
index ccc3a7ccd..e61aa2b0a 100755
--- a/.ci/linux-build.sh
+++ b/.ci/linux-build.sh
@@ -29,11 +29,15 @@ if [ "$BUILD_32BIT" = "1" ]; then
fi
OPTS="$OPTS --default-library=$DEF_LIB"
+OPTS="$OPTS --prefix=/usr -Dlibdir=lib"
meson build --werror -Dexamples=all $OPTS
ninja -C build
+DESTDIR=$(pwd)/install ninja -C build install
if [ "$AARCH64" != "1" ]; then
- devtools/test-null.sh
+ export LD_LIBRARY_PATH=$(pwd)/install/usr/lib
+ devtools/test-null.sh install/usr/bin/dpdk-testpmd
+ unset LD_LIBRARY_PATH
fi
if [ "$RUN_TESTS" = "1" ]; then
diff --git a/devtools/test-meson-builds.sh b/devtools/test-meson-builds.sh
index aed175889..254588ae6 100755
--- a/devtools/test-meson-builds.sh
+++ b/devtools/test-meson-builds.sh
@@ -63,7 +63,7 @@ config () # <dir> <builddir> <meson options>
shift
builddir=$1
shift
- options="--werror -Dexamples=all"
+ options="--werror -Dexamples=all --prefix=/usr -Dlibdir=lib"
for option in $DPDK_MESON_OPTIONS ; do
options="$options -D$option"
done
@@ -74,22 +74,29 @@ config () # <dir> <builddir> <meson options>
fi
}
-compile () # <builddir>
+compile () # <builddir> <installdir>
{
builddir=$1
shift
+ export DESTDIR=$1
+ shift
+ rm -rf $DESTDIR
if [ -n "$TEST_MESON_BUILD_VERY_VERBOSE" ] ; then
# for full output from ninja use "-v"
echo "$ninja_cmd -v -C $builddir"
$ninja_cmd -v -C $builddir
+ $ninja_cmd -v -C $builddir install
elif [ -n "$TEST_MESON_BUILD_VERBOSE" ] ; then
# for keeping the history of short cmds, pipe through cat
echo "$ninja_cmd -C $builddir | cat"
$ninja_cmd -C $builddir | cat
+ $ninja_cmd -C $builddir install | cat
else
echo "$ninja_cmd -C $builddir"
$ninja_cmd -C $builddir
+ $ninja_cmd -C $builddir install
fi
+ unset DESTDIR
}
build () # <directory> <target compiler> <meson options>
@@ -102,7 +109,8 @@ build () # <directory> <target compiler> <meson options>
command -v ${CC##* } >/dev/null 2>&1 || return 0
load_env $targetcc || return 0
config $srcdir $builds_dir/$targetdir $*
- compile $builds_dir/$targetdir
+ compile $builds_dir/$targetdir \
+ $(readlink -f $builds_dir/$targetdir/install)
}
if [ "$1" = "-vv" ] ; then
@@ -134,7 +142,7 @@ ok=$(cc -march=$default_machine -E - < /dev/null > /dev/null 2>&1 || echo false)
if [ "$ok" = "false" ] ; then
default_machine='corei7'
fi
-build build-x86-default cc -Dlibdir=lib -Dmachine=$default_machine $use_shared
+build build-x86-default cc -Dmachine=$default_machine $use_shared
c=aarch64-linux-gnu-gcc
# generic armv8a with clang as host compiler
@@ -150,12 +158,9 @@ for f in $srcdir/config/arm/arm64_[bdo]*gcc ; do
unset CC
done
-# Test installation of the x86-default target, to be used for checking
-# the sample apps build using the pkg-config file for cflags and libs
-build_path=$(readlink -f $builds_dir/build-x86-default)
-export DESTDIR=$build_path/install-root
-$ninja_cmd -C $build_path install
-
+# Use the x86-default target, to check the sample apps build using the
+# pkg-config file for cflags and libs
+export DESTDIR=$(readlink -f $builds_dir/build-x86-default/install)
load_env cc
pc_file=$(find $DESTDIR -name libdpdk.pc)
export PKG_CONFIG_PATH=$(dirname $pc_file):$PKG_CONFIG_PATH
@@ -165,6 +170,6 @@ if pkg-config --define-prefix libdpdk >/dev/null 2>&1; then
export PKGCONF="pkg-config --define-prefix"
for example in cmdline helloworld l2fwd l3fwd skeleton timer; do
echo "## Building $example"
- $MAKE -C $DESTDIR/usr/local/share/dpdk/examples/$example clean all
+ $MAKE -C $DESTDIR/usr/share/dpdk/examples/$example clean all
done
fi
diff --git a/devtools/test-null.sh b/devtools/test-null.sh
index 548de8113..8121ec9e3 100755
--- a/devtools/test-null.sh
+++ b/devtools/test-null.sh
@@ -21,6 +21,7 @@ fi
if ldd $testpmd | grep -q librte_ ; then
export LD_LIBRARY_PATH=$build/drivers:$build/lib:$LD_LIBRARY_PATH
+ ldd $testpmd
libs='-d librte_mempool_ring.so -d librte_pmd_null.so'
else
libs=
--
2.23.0
^ permalink raw reply [relevance 2%]
* [dpdk-dev] [PATCH v2 4/4] add ABI checks
2020-01-29 17:26 9% ` [dpdk-dev] [PATCH v2 0/4] " David Marchand
2020-01-29 17:26 2% ` [dpdk-dev] [PATCH v2 3/4] build: test meson installation David Marchand
@ 2020-01-29 17:26 28% ` David Marchand
2020-01-29 17:42 7% ` Thomas Monjalon
2020-01-30 10:57 4% ` [dpdk-dev] [PATCH v2 0/4] " Luca Boccassi
2 siblings, 1 reply; 200+ results
From: David Marchand @ 2020-01-29 17:26 UTC (permalink / raw)
To: dev
Cc: thomas, bruce.richardson, kevin.laatz, aconole, nhorman,
akhil.goyal, anoobj, Michael Santana, John McNamara,
Marko Kovacevic
For normal developers, those checks are disabled.
Enabling them requires a configuration that will trigger the ABI dumps
generation as part of the existing devtools/test-build.sh and
devtools/test-meson-builds.sh scripts.
Those checks are enabled in the CI for the default meson options on x86
and aarch64 so that proposed patches are validated via our CI robot.
A cache of the ABI is stored in travis jobs to avoid rebuilding too
often.
Checks can be only informational by setting ABI_CHECKS_WARN_ONLY when
breaking the ABI in a future release.
Explicit suppression rules have been added on internal structures
exposed to crypto and security drivers as the current ABI policy does
not apply to them.
This could be improved in the future by carefully splitting the headers
content with application and driver "users" in mind.
We currently have issues reported for librte_crypto recent changes for
which suppression rules have been added too.
Mellanox glue libraries are explicitly skipped as they are not part of
the application ABI.
Signed-off-by: David Marchand <david.marchand@redhat.com>
---
Changelog since v1:
- reworked the scripts so that the build test scripts clone and build the
reference automatically. A developer only needs to set one variable to
enable the checks,
- meson builds are done with debug so that abidiff can inspect the
structures,
- abidiff checks only public types by looking at installed headers,
- abidiff has some issue when comparing a dump with a .so built with clang
so all diff are now done with dump files only,
- suppression rules have been added to waive warnings on exposed internal
types,
- an abi breakage has been reported on changes in cryptodev.
For now, suppression rules have been put in place to let the CI run,
---
.ci/linux-build.sh | 23 +++++++++++
.travis.yml | 20 +++++++++-
MAINTAINERS | 2 +
devtools/check-abi.sh | 59 +++++++++++++++++++++++++++++
devtools/dpdk.abignore | 20 ++++++++++
devtools/gen-abi.sh | 26 +++++++++++++
devtools/test-build.sh | 45 ++++++++++++++++++++--
devtools/test-meson-builds.sh | 35 ++++++++++++++++-
doc/guides/contributing/patches.rst | 13 +++++++
9 files changed, 236 insertions(+), 7 deletions(-)
create mode 100755 devtools/check-abi.sh
create mode 100644 devtools/dpdk.abignore
create mode 100755 devtools/gen-abi.sh
diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh
index e61aa2b0a..95bd869c3 100755
--- a/.ci/linux-build.sh
+++ b/.ci/linux-build.sh
@@ -30,6 +30,7 @@ fi
OPTS="$OPTS --default-library=$DEF_LIB"
OPTS="$OPTS --prefix=/usr -Dlibdir=lib"
+OPTS="$OPTS --buildtype=debugoptimized"
meson build --werror -Dexamples=all $OPTS
ninja -C build
DESTDIR=$(pwd)/install ninja -C build install
@@ -40,6 +41,28 @@ if [ "$AARCH64" != "1" ]; then
unset LD_LIBRARY_PATH
fi
+if [ "$ABI_CHECKS" = "1" ]; then
+ REF_GIT_REPO=${REF_GIT_REPO:-https://dpdk.org/git/dpdk}
+ REF_GIT_TAG=${REF_GIT_TAG:-v19.11}
+
+ if [ "$(cat reference/VERSION 2>/dev/null)" != "$REF_GIT_TAG" ]; then
+ rm -rf reference
+ fi
+
+ if [ ! -d reference ]; then
+ refsrcdir=$(readlink -f $(pwd)/../dpdk-$REF_GIT_TAG)
+ git clone --single-branch -b $REF_GIT_TAG $REF_GIT_REPO $refsrcdir
+ meson --werror $OPTS $refsrcdir $refsrcdir/build
+ ninja -C $refsrcdir/build
+ DESTDIR=$(pwd)/reference ninja -C $refsrcdir/build install
+ devtools/gen-abi.sh reference
+ echo $REF_GIT_TAG > reference/VERSION
+ fi
+
+ devtools/gen-abi.sh install
+ devtools/check-abi.sh reference install ${ABI_CHECKS_WARN_ONLY:-}
+fi
+
if [ "$RUN_TESTS" = "1" ]; then
sudo meson test -C build --suite fast-tests -t 3
fi
diff --git a/.travis.yml b/.travis.yml
index 8162f1c05..22539d823 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -1,5 +1,8 @@
language: c
-cache: ccache
+cache:
+ ccache: true
+ directories:
+ - reference
compiler:
- gcc
- clang
@@ -21,7 +24,7 @@ aarch64_packages: &aarch64_packages
extra_packages: &extra_packages
- *required_packages
- - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+ - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4, abigail-tools]
build_32b_packages: &build_32b_packages
- *required_packages
@@ -151,5 +154,18 @@ matrix:
packages:
- *required_packages
- *doc_packages
+ - env: DEF_LIB="shared" EXTRA_PACKAGES=1 ABI_CHECKS=1
+ compiler: gcc
+ addons:
+ apt:
+ packages:
+ - *extra_packages
+ - env: DEF_LIB="shared" EXTRA_PACKAGES=1 ABI_CHECKS=1
+ arch: arm64
+ compiler: gcc
+ addons:
+ apt:
+ packages:
+ - *extra_packages
script: ./.ci/${TRAVIS_OS_NAME}-build.sh
diff --git a/MAINTAINERS b/MAINTAINERS
index 94bccae6d..6dae4ee63 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -144,8 +144,10 @@ M: Neil Horman <nhorman@tuxdriver.com>
F: lib/librte_eal/common/include/rte_compat.h
F: lib/librte_eal/common/include/rte_function_versioning.h
F: doc/guides/rel_notes/deprecation.rst
+F: devtools/check-abi.sh
F: devtools/check-abi-version.sh
F: devtools/check-symbol-change.sh
+F: devtools/gen-abi.sh
F: devtools/update-abi.sh
F: devtools/update_version_map_abi.py
F: devtools/validate-abi.sh
diff --git a/devtools/check-abi.sh b/devtools/check-abi.sh
new file mode 100755
index 000000000..5872499ec
--- /dev/null
+++ b/devtools/check-abi.sh
@@ -0,0 +1,59 @@
+#!/bin/sh -e
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright (c) 2019 Red Hat, Inc.
+
+if [ $# != 2 ] && [ $# != 3 ]; then
+ echo "Usage: $0 refdir newdir [warnonly]"
+ exit 1
+fi
+
+refdir=$1
+newdir=$2
+warnonly=${3:-}
+ABIDIFF_OPTIONS="--suppr $(dirname $0)/dpdk.abignore --no-added-syms"
+
+if [ ! -d $refdir ]; then
+ echo "Error: reference directory '$refdir' does not exist."
+ exit 1
+fi
+incdir=$(find $refdir -type d -a -name include)
+if [ -z "$incdir" ] || [ ! -e "$incdir" ]; then
+ echo "WARNING: could not identify a include directory for $refdir, expect false positives..."
+else
+ ABIDIFF_OPTIONS="$ABIDIFF_OPTIONS --headers-dir1 $incdir"
+fi
+
+if [ ! -d $newdir ]; then
+ echo "Error: directory to check '$newdir' does not exist."
+ exit 1
+fi
+incdir2=$(find $newdir -type d -a -name include)
+if [ -z "$incdir2" ] || [ ! -e "$incdir2" ]; then
+ echo "WARNING: could not identify a include directory for $newdir, expect false positives..."
+else
+ ABIDIFF_OPTIONS="$ABIDIFF_OPTIONS --headers-dir2 $incdir2"
+fi
+
+error=
+for dump in $(find $refdir -name "*.dump"); do
+ name=$(basename $dump)
+ # skip glue drivers, example librte_pmd_mlx5_glue.dump
+ # We can't rely on a suppression rule for now:
+ # https://sourceware.org/bugzilla/show_bug.cgi?id=25480
+ if [ "$name" != "${name%%_glue.dump}" ]; then
+ echo "Skipping ${dump}..."
+ continue
+ fi
+ dump2=$(find $newdir -name $name)
+ if [ -z "$dump2" ] || [ ! -e "$dump2" ]; then
+ echo "Error: can't find $name in $newdir"
+ error=1
+ continue
+ fi
+ if ! abidiff $ABIDIFF_OPTIONS $dump $dump2; then
+ echo "Error: ABI issue reported for 'abidiff $ABIDIFF_OPTIONS $dump $dump2'"
+ error=1
+ fi
+done
+
+[ -z "$error" ] || [ -n "$warnonly" ]
diff --git a/devtools/dpdk.abignore b/devtools/dpdk.abignore
new file mode 100644
index 000000000..0c01eebea
--- /dev/null
+++ b/devtools/dpdk.abignore
@@ -0,0 +1,20 @@
+[suppress_function]
+ symbol_version = EXPERIMENTAL
+[suppress_variable]
+ symbol_version = EXPERIMENTAL
+
+; Explicit ignore for driver-only ABI
+[suppress_type]
+ name = rte_cryptodev_ops
+
+; FIXME
+[suppress_type]
+ type_kind = enum
+ name = rte_crypto_aead_algorithm
+ changed_enumerators = RTE_CRYPTO_AEAD_LIST_END
+[suppress_type]
+ type_kind = enum
+ name = rte_crypto_asym_xform_type
+ changed_enumerators = RTE_CRYPTO_ASYM_XFORM_TYPE_LIST_END
+[suppress_variable]
+ name = rte_crypto_aead_algorithm_strings
diff --git a/devtools/gen-abi.sh b/devtools/gen-abi.sh
new file mode 100755
index 000000000..c44b0e228
--- /dev/null
+++ b/devtools/gen-abi.sh
@@ -0,0 +1,26 @@
+#!/bin/sh -e
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright (c) 2019 Red Hat, Inc.
+
+if [ $# != 1 ]; then
+ echo "Usage: $0 installdir"
+ exit 1
+fi
+
+installdir=$1
+if [ ! -d $installdir ]; then
+ echo "Error: install directory '$installdir' does not exist."
+ exit 1
+fi
+
+dumpdir=$installdir/dump
+rm -rf $dumpdir
+mkdir -p $dumpdir
+for f in $(find $installdir -name "*.so.*"); do
+ if test -L $f; then
+ continue
+ fi
+
+ libname=$(basename $f)
+ abidw --out-file $dumpdir/${libname%.so*}.dump $f
+done
diff --git a/devtools/test-build.sh b/devtools/test-build.sh
index 52305fbb8..fba30bac9 100755
--- a/devtools/test-build.sh
+++ b/devtools/test-build.sh
@@ -30,7 +30,8 @@ default_path=$PATH
# - LIBSSO_SNOW3G_PATH
# - LIBSSO_KASUMI_PATH
# - LIBSSO_ZUC_PATH
-. $(dirname $(readlink -f $0))/load-devel-config
+devtools_dir=$(dirname $(readlink -f $0))
+. $devtools_dir/load-devel-config
print_usage () {
echo "usage: $(basename $0) [-h] [-jX] [-s] [config1 [config2] ...]]"
@@ -64,6 +65,7 @@ print_help () {
[ -z $MAKE ] && echo "Cannot find make or gmake" && exit 1
J=$DPDK_MAKE_JOBS
+refsrcdir=$(mktemp -d -t dpdk-${DPDK_ABI_REF_VERSION:-}.XXX)
builds_dir=${DPDK_BUILD_TEST_DIR:-.}
short=false
unset verbose
@@ -91,13 +93,14 @@ on_exit ()
[ "$DPDK_NOTIFY" != notify-send ] || \
notify-send -u low --icon=dialog-error 'DPDK build' 'failed'
fi
+ rm -rf $refsrcdir
}
# catch manual interrupt to ignore notification
trap "signal=INT ; trap - INT ; kill -INT $$" INT
# notify result on exit
trap on_exit EXIT
-cd $(dirname $(readlink -f $0))/..
+cd $devtools_dir/..
reset_env ()
{
@@ -233,7 +236,7 @@ for conf in $configs ; do
# reload config with DPDK_TARGET set
DPDK_TARGET=$target
reset_env
- . $(dirname $(readlink -f $0))/load-devel-config
+ . $devtools_dir/load-devel-config
options=$(echo $conf | sed 's,[^~+]*,,')
dir=$builds_dir/$conf
@@ -253,6 +256,42 @@ for conf in $configs ; do
EXTRA_LDFLAGS="$DPDK_DEP_LDFLAGS" $verbose \
O=$(readlink -f $dir)/examples
unset RTE_TARGET
+ if [ -n "$DPDK_ABI_REF_VERSION" ]; then
+ DPDK_ABI_REF_DIR=${DPDK_ABI_REF_DIR:-reference}
+ abirefdir=$DPDK_ABI_REF_DIR/$DPDK_ABI_REF_VERSION/$conf
+ if [ ! -d $abirefdir ]; then
+ # clone current sources
+ if [ ! -d $refsrcdir/.git ]; then
+ git clone --local --no-hardlinks \
+ --single-branch \
+ -b $DPDK_ABI_REF_VERSION \
+ $(pwd) $refsrcdir
+ fi
+
+ cd $refsrcdir
+
+ rm -rf build
+ config build $target $options
+
+ echo -n "================== Build $conf "
+ echo "($DPDK_ABI_REF_VERSION)"
+ ${MAKE} -j$J EXTRA_CFLAGS="$maxerr $DPDK_DEP_CFLAGS" \
+ EXTRA_LDFLAGS="$DPDK_DEP_LDFLAGS" $verbose \
+ O=build
+ ! $short || break
+ export RTE_TARGET=$target
+ ${MAKE} install O=build DESTDIR=$abirefdir \
+ prefix=
+ $devtools_dir/gen-abi.sh $abirefdir
+
+ # back to current workdir
+ cd $devtools_dir/..
+ fi
+
+ echo "================== Check ABI $conf"
+ $devtools_dir/gen-abi.sh $dir/install
+ $devtools_dir/check-abi.sh $abirefdir $dir/install
+ fi
echo "################## $conf done."
unset dir
done
diff --git a/devtools/test-meson-builds.sh b/devtools/test-meson-builds.sh
index 254588ae6..1b410c784 100755
--- a/devtools/test-meson-builds.sh
+++ b/devtools/test-meson-builds.sh
@@ -16,8 +16,15 @@ srcdir=$(dirname $(readlink -f $0))/..
MESON=${MESON:-meson}
use_shared="--default-library=shared"
+refsrcdir=$(mktemp -d -t dpdk-${DPDK_ABI_REF_VERSION:-}.XXX)
builds_dir=${DPDK_BUILD_TEST_DIR:-.}
+on_exit ()
+{
+ rm -rf $refsrcdir
+}
+trap on_exit EXIT
+
if command -v gmake >/dev/null 2>&1 ; then
MAKE=gmake
else
@@ -63,7 +70,9 @@ config () # <dir> <builddir> <meson options>
shift
builddir=$1
shift
- options="--werror -Dexamples=all --prefix=/usr -Dlibdir=lib"
+ options=
+ options="$options --werror --buildtype=debugoptimized -Dexamples=all"
+ options="$options --prefix=/usr -Dlibdir=lib"
for option in $DPDK_MESON_OPTIONS ; do
options="$options -D$option"
done
@@ -96,7 +105,6 @@ compile () # <builddir> <installdir>
$ninja_cmd -C $builddir
$ninja_cmd -C $builddir install
fi
- unset DESTDIR
}
build () # <directory> <target compiler> <meson options>
@@ -111,6 +119,29 @@ build () # <directory> <target compiler> <meson options>
config $srcdir $builds_dir/$targetdir $*
compile $builds_dir/$targetdir \
$(readlink -f $builds_dir/$targetdir/install)
+ if [ -n "$DPDK_ABI_REF_VERSION" ]; then
+ DPDK_ABI_REF_DIR=${DPDK_ABI_REF_DIR:-reference}
+ abirefdir=$DPDK_ABI_REF_DIR/$DPDK_ABI_REF_VERSION/$targetdir
+ if [ ! -d $abirefdir ]; then
+ # clone current sources
+ if [ ! -d $refsrcdir/.git ]; then
+ git clone --local --no-hardlinks \
+ --single-branch \
+ -b $DPDK_ABI_REF_VERSION \
+ $srcdir $refsrcdir
+ fi
+
+ rm -rf $refsrcdir/build
+ config $refsrcdir $refsrcdir/build $*
+ compile $refsrcdir/build $abirefdir
+ $srcdir/devtools/gen-abi.sh $abirefdir
+ fi
+
+ $srcdir/devtools/gen-abi.sh \
+ $(readlink -f $builds_dir/$targetdir/install)
+ $srcdir/devtools/check-abi.sh $abirefdir \
+ $(readlink -f $builds_dir/$targetdir/install)
+ fi
}
if [ "$1" = "-vv" ] ; then
diff --git a/doc/guides/contributing/patches.rst b/doc/guides/contributing/patches.rst
index 0686450e4..2e16741ca 100644
--- a/doc/guides/contributing/patches.rst
+++ b/doc/guides/contributing/patches.rst
@@ -513,6 +513,19 @@ in a single subfolder called "__builds" created in the current directory.
Setting ``DPDK_BUILD_TEST_DIR`` to an absolute directory path e.g. ``/tmp`` is also supported.
+Checking ABI compatibility
+--------------------------
+
+By default, ABI compatibility checks are disabled.
+
+To enable them, a reference version must be selected via the environment
+variable ``DPDK_ABI_REF_VERSION``.
+
+The ``devtools/test-build.sh`` and ``devtools/test-meson-builds.sh`` scripts
+then build this reference version in a temporary directory and store the
+results in the ``DPDK_ABI_REF_DIR`` directory.
+
+
Sending Patches
---------------
--
2.23.0
^ permalink raw reply [relevance 28%]
* Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
2020-01-29 17:26 28% ` [dpdk-dev] [PATCH v2 4/4] add ABI checks David Marchand
@ 2020-01-29 17:42 7% ` Thomas Monjalon
2020-01-29 18:10 8% ` Anoob Joseph
0 siblings, 1 reply; 200+ results
From: Thomas Monjalon @ 2020-01-29 17:42 UTC (permalink / raw)
To: akhil.goyal, anoobj
Cc: dev, David Marchand, bruce.richardson, nhorman, John McNamara
Anoob, Akhil,
Please we need to revert or fix the ABI breakages in cryptodev very soon.
The FIXME section below must be empty.
Thanks
29/01/2020 18:26, David Marchand:
> We currently have issues reported for librte_crypto recent changes for
> which suppression rules have been added too.
[..]
> --- /dev/null
> +++ b/devtools/dpdk.abignore
> +; FIXME
> +[suppress_type]
> + type_kind = enum
> + name = rte_crypto_aead_algorithm
> + changed_enumerators = RTE_CRYPTO_AEAD_LIST_END
> +[suppress_type]
> + type_kind = enum
> + name = rte_crypto_asym_xform_type
> + changed_enumerators = RTE_CRYPTO_ASYM_XFORM_TYPE_LIST_END
> +[suppress_variable]
> + name = rte_crypto_aead_algorithm_strings
^ permalink raw reply [relevance 7%]
* Re: [dpdk-dev] [PATCH 2/2] eal: add eal_parse_optionlist to parse user input
@ 2020-01-29 17:44 4% ` David Marchand
2020-01-29 18:07 0% ` Ferruh Yigit
0 siblings, 1 reply; 200+ results
From: David Marchand @ 2020-01-29 17:44 UTC (permalink / raw)
To: Ferruh Yigit, Thomas Monjalon
Cc: Hariprasad Govindharajan, dev, Stephen Hemminger
On Tue, Jan 28, 2020 at 6:35 PM Ferruh Yigit <ferruh.yigit@intel.com> wrote:
> On 1/27/2020 10:30 AM, Hariprasad Govindharajan wrote:
> > In current version, there is a function which parses
> > the corelist based on user value. A new generic
> > function eal_parse_optionlist is added which will
> > parse corelist as well as similar user input so
> > that we can use it as a public API too.
> >
> > Signed-off-by: Hariprasad Govindharajan <hariprasad.govindharajan@intel.com>
>
> Hi David,
>
> Overall this patchset is to add '--portlist' command to testpmd and remove
> existing 64 port limitation.
>
> And in this patch re-uses the exiting parser function in eal and converts it to
> API, question is if eal is good place to have this API, what do you think about it?
Exporting string parsers from the EAL has little value.
Ok we avoid code duplication (and I can see other places in the tree
where it might be used), but in the end we will have to maintain this
API in the ABI when it enters the stable ABI.
I am for avoiding this.
--
David Marchand
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [RFC v2] meter: fix ABI break due to experimental tag removal
2020-01-29 16:43 3% ` [dpdk-dev] [RFC v2] " Ferruh Yigit
@ 2020-01-29 17:49 4% ` Andrzej Ostruszka
2020-01-30 12:33 7% ` Thomas Monjalon
1 sibling, 0 replies; 200+ results
From: Andrzej Ostruszka @ 2020-01-29 17:49 UTC (permalink / raw)
To: Ferruh Yigit, Neil Horman, Cristian Dumitrescu, Eelco Chaudron
Cc: dev, Thomas Monjalon, Luca Boccassi, David Marchand,
Bruce Richardson, Ian Stokes
On 1/29/20 5:43 PM, Ferruh Yigit wrote:
[...]
> diff --git a/lib/librte_meter/rte_meter.c b/lib/librte_meter/rte_meter.c
> index da01429a8..5244537fa 100644
> --- a/lib/librte_meter/rte_meter.c
> +++ b/lib/librte_meter/rte_meter.c
> @@ -9,6 +9,7 @@
> #include <rte_common.h>
> #include <rte_log.h>
> #include <rte_cycles.h>
> +#include <rte_function_versioning.h>
>
> #include "rte_meter.h"
>
> @@ -119,8 +120,8 @@ rte_meter_trtcm_config(struct rte_meter_trtcm *m,
> return 0;
> }
>
> -int
> -rte_meter_trtcm_rfc4115_profile_config(
> +static int
> +rte_meter_trtcm_rfc4115_profile_config_(
> struct rte_meter_trtcm_rfc4115_profile *p,
> struct rte_meter_trtcm_rfc4115_params *params)
> {
> @@ -145,7 +146,35 @@ rte_meter_trtcm_rfc4115_profile_config(
> }
>
> int
> -rte_meter_trtcm_rfc4115_config(
> +rte_meter_trtcm_rfc4115_profile_config_s(
> + struct rte_meter_trtcm_rfc4115_profile *p,
> + struct rte_meter_trtcm_rfc4115_params *params);
> +int
> +rte_meter_trtcm_rfc4115_profile_config_s(
> + struct rte_meter_trtcm_rfc4115_profile *p,
> + struct rte_meter_trtcm_rfc4115_params *params)
> +{
> + return rte_meter_trtcm_rfc4115_profile_config_(p, params);
> +}
> +BIND_DEFAULT_SYMBOL(rte_meter_trtcm_rfc4115_profile_config, _s, 20.0.1);
You might want to mark these version symbols with __vsym macro. Without
this shared lib build with LTO might remove them (due to long standing
gcc bug - it does not have proper way to mark .symver in internal
representation and so does not recognize that function is used).
This comment is global - for all symbols mentioned in BIND_/VERSION_ macros.
Out of curiosity - why do you need separate declaration just before
definition?
With regards
Andrzej Ostruszka
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH 2/2] eal: add eal_parse_optionlist to parse user input
2020-01-29 17:44 4% ` David Marchand
@ 2020-01-29 18:07 0% ` Ferruh Yigit
2020-01-29 19:19 0% ` Stephen Hemminger
0 siblings, 1 reply; 200+ results
From: Ferruh Yigit @ 2020-01-29 18:07 UTC (permalink / raw)
To: David Marchand, Thomas Monjalon
Cc: Hariprasad Govindharajan, dev, Stephen Hemminger
On 1/29/2020 5:44 PM, David Marchand wrote:
> On Tue, Jan 28, 2020 at 6:35 PM Ferruh Yigit <ferruh.yigit@intel.com> wrote:
>> On 1/27/2020 10:30 AM, Hariprasad Govindharajan wrote:
>>> In current version, there is a function which parses
>>> the corelist based on user value. A new generic
>>> function eal_parse_optionlist is added which will
>>> parse corelist as well as similar user input so
>>> that we can use it as a public API too.
>>>
>>> Signed-off-by: Hariprasad Govindharajan <hariprasad.govindharajan@intel.com>
>>
>> Hi David,
>>
>> Overall this patchset is to add '--portlist' command to testpmd and remove
>> existing 64 port limitation.
>>
>> And in this patch re-uses the exiting parser function in eal and converts it to
>> API, question is if eal is good place to have this API, what do you think about it?
>
> Exporting string parsers from the EAL has little value.
> Ok we avoid code duplication (and I can see other places in the tree
> where it might be used), but in the end we will have to maintain this
> API in the ABI when it enters the stable ABI.
>
> I am for avoiding this.
>
>
The same function can be used in some sample applications too (which are using
port mask), so instead of duplicating it multiple times, it would be nice to
have this function somewhere that applications can use.
Does it makes sense to have a rte_util.c (under in eal or as a separate library)
to have this kind of application helper functions?
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
2020-01-29 17:42 7% ` Thomas Monjalon
@ 2020-01-29 18:10 8% ` Anoob Joseph
2020-01-29 20:03 7% ` David Marchand
2020-01-30 13:06 8% ` Trahe, Fiona
0 siblings, 2 replies; 200+ results
From: Anoob Joseph @ 2020-01-29 18:10 UTC (permalink / raw)
To: Thomas Monjalon, akhil.goyal
Cc: dev, David Marchand, bruce.richardson, nhorman, John McNamara
Hi Thomas,
The asymmetric crypto library is experimental. Changes to experimental code paths is allowed, right?
Also, I was wondering why changing the LIST_END would cause breakage. Before we introduced the ABI checks and ABI freeze policy, it was always allowed to add enums to the end. I'm just trying to understand the real impact of this case.
If we don't allow the LIST_END to be modified, then it means no feature can be implemented in between. And the best way to overcome that would be to just remove the LIST_END or set LIST_END to a very high value.
Thanks,
Anoob
> -----Original Message-----
> From: dev <dev-bounces@dpdk.org> On Behalf Of Thomas Monjalon
> Sent: Wednesday, January 29, 2020 11:13 PM
> To: akhil.goyal@nxp.com; Anoob Joseph <anoobj@marvell.com>
> Cc: dev@dpdk.org; David Marchand <david.marchand@redhat.com>;
> bruce.richardson@intel.com; nhorman@tuxdriver.com; John McNamara
> <john.mcnamara@intel.com>
> Subject: Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
>
> Anoob, Akhil,
>
> Please we need to revert or fix the ABI breakages in cryptodev very soon.
> The FIXME section below must be empty.
>
> Thanks
>
> 29/01/2020 18:26, David Marchand:
> > We currently have issues reported for librte_crypto recent changes for
> > which suppression rules have been added too.
> [..]
> > --- /dev/null
> > +++ b/devtools/dpdk.abignore
> > +; FIXME
> > +[suppress_type]
> > + type_kind = enum
> > + name = rte_crypto_aead_algorithm
> > + changed_enumerators = RTE_CRYPTO_AEAD_LIST_END
> > +[suppress_type]
> > + type_kind = enum
> > + name = rte_crypto_asym_xform_type
> > + changed_enumerators = RTE_CRYPTO_ASYM_XFORM_TYPE_LIST_END
> > +[suppress_variable]
> > + name = rte_crypto_aead_algorithm_strings
>
>
>
^ permalink raw reply [relevance 8%]
* Re: [dpdk-dev] [PATCH 2/2] eal: add eal_parse_optionlist to parse user input
2020-01-29 18:07 0% ` Ferruh Yigit
@ 2020-01-29 19:19 0% ` Stephen Hemminger
2020-01-31 11:25 0% ` Govindharajan, Hariprasad
2020-01-31 14:42 0% ` Govindharajan, Hariprasad
0 siblings, 2 replies; 200+ results
From: Stephen Hemminger @ 2020-01-29 19:19 UTC (permalink / raw)
To: Ferruh Yigit
Cc: David Marchand, Thomas Monjalon, Hariprasad Govindharajan, dev
On Wed, 29 Jan 2020 18:07:05 +0000
Ferruh Yigit <ferruh.yigit@intel.com> wrote:
> On 1/29/2020 5:44 PM, David Marchand wrote:
> > On Tue, Jan 28, 2020 at 6:35 PM Ferruh Yigit <ferruh.yigit@intel.com> wrote:
> >> On 1/27/2020 10:30 AM, Hariprasad Govindharajan wrote:
> >>> In current version, there is a function which parses
> >>> the corelist based on user value. A new generic
> >>> function eal_parse_optionlist is added which will
> >>> parse corelist as well as similar user input so
> >>> that we can use it as a public API too.
> >>>
> >>> Signed-off-by: Hariprasad Govindharajan <hariprasad.govindharajan@intel.com>
> >>
> >> Hi David,
> >>
> >> Overall this patchset is to add '--portlist' command to testpmd and remove
> >> existing 64 port limitation.
> >>
> >> And in this patch re-uses the exiting parser function in eal and converts it to
> >> API, question is if eal is good place to have this API, what do you think about it?
> >
> > Exporting string parsers from the EAL has little value.
> > Ok we avoid code duplication (and I can see other places in the tree
> > where it might be used), but in the end we will have to maintain this
> > API in the ABI when it enters the stable ABI.
> >
> > I am for avoiding this.
> >
> >
>
> The same function can be used in some sample applications too (which are using
> port mask), so instead of duplicating it multiple times, it would be nice to
> have this function somewhere that applications can use.
>
> Does it makes sense to have a rte_util.c (under in eal or as a separate library)
> to have this kind of application helper functions?
It makes sense to have a rte library that handles arbitrary size bitvector
and has string handling functions. Kind of like what kernel has for
the cpuset parsing. This could be used for cpus in EAL and port-masks
or other arrays in applications.
But just doing copy/paste of existing code without thinking about how
API should work is a bad idea.
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
2020-01-29 18:10 8% ` Anoob Joseph
@ 2020-01-29 20:03 7% ` David Marchand
2020-01-29 20:13 4% ` Akhil Goyal
2020-01-30 13:06 8% ` Trahe, Fiona
1 sibling, 1 reply; 200+ results
From: David Marchand @ 2020-01-29 20:03 UTC (permalink / raw)
To: Anoob Joseph
Cc: Thomas Monjalon, akhil.goyal, dev, bruce.richardson, nhorman,
John McNamara, dodji
On Wed, Jan 29, 2020 at 7:10 PM Anoob Joseph <anoobj@marvell.com> wrote:
> The asymmetric crypto library is experimental. Changes to experimental code paths is allowed, right?
The asymmetric crypto enum is referenced by a function part of the stable ABI.
It is possible to waive this enum, if we are sure no use out of the
experimental asym crypto APIs is possible.
The rest of the changes touch stable symbols.
Adding the abidiff report:
[C]'function void rte_cryptodev_info_get(uint8_t,
rte_cryptodev_info*)' at rte_cryptodev.c:1115:1 has some indirect
sub-type changes:
parameter 2 of type 'rte_cryptodev_info*' has sub-type changes:
in pointed to type 'struct rte_cryptodev_info' at rte_cryptodev.h:468:1:
type size hasn't changed
1 data member change:
type of 'const rte_cryptodev_capabilities*
rte_cryptodev_info::capabilities' changed:
in pointed to type 'const rte_cryptodev_capabilities':
in unqualified underlying type 'struct
rte_cryptodev_capabilities' at rte_cryptodev.h:176:1:
type size hasn't changed
1 data member change:
type of '__anonymous_union__ ' changed:
type size hasn't changed
1 data member change:
type of 'rte_cryptodev_asymmetric_capability
__anonymous_union__::asym' changed:
type size hasn't changed
1 data member change:
type of
'rte_cryptodev_asymmetric_xform_capability
rte_cryptodev_asymmetric_capability::xform_capa' changed:
type size hasn't changed
1 data member change:
type of 'rte_crypto_asym_xform_type
rte_cryptodev_asymmetric_xform_capability::xform_type' changed:
type size hasn't changed
2 enumerator insertions:
'rte_crypto_asym_xform_type::RTE_CRYPTO_ASYM_XFORM_ECDSA' value '7'
'rte_crypto_asym_xform_type::RTE_CRYPTO_ASYM_XFORM_ECPM' value '8'
1 enumerator change:
'rte_crypto_asym_xform_type::RTE_CRYPTO_ASYM_XFORM_TYPE_LIST_END' from
value '7' to '9' at rte_crypto_asym.h:60:1
[C]'function int
rte_cryptodev_get_aead_algo_enum(rte_crypto_aead_algorithm*, const
char*)' at rte_cryptodev.c:239:1 has some indirect sub-type changes:
parameter 1 of type 'rte_crypto_aead_algorithm*' has sub-type changes:
in pointed to type 'enum rte_crypto_aead_algorithm' at
rte_crypto_sym.h:346:1:
type size hasn't changed
1 enumerator insertion:
'rte_crypto_aead_algorithm::RTE_CRYPTO_AEAD_CHACHA20_POLY1305'
value '3'
1 enumerator change:
'rte_crypto_aead_algorithm::RTE_CRYPTO_AEAD_LIST_END' from
value '3' to '4' at rte_crypto_sym.h:346:1
[C]'const char* rte_crypto_aead_algorithm_strings[1]' was changed at
rte_crypto_sym.h:358:1:
size of symbol (in bytes) changed from 24 to 32
--
David Marchand
^ permalink raw reply [relevance 7%]
* Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
2020-01-29 20:03 7% ` David Marchand
@ 2020-01-29 20:13 4% ` Akhil Goyal
2020-01-30 16:09 8% ` Ferruh Yigit
0 siblings, 1 reply; 200+ results
From: Akhil Goyal @ 2020-01-29 20:13 UTC (permalink / raw)
To: David Marchand, Anoob Joseph, Trahe, Fiona, Kusztal, ArkadiuszX
Cc: Thomas Monjalon, dev, bruce.richardson, nhorman, John McNamara, dodji
>
> On Wed, Jan 29, 2020 at 7:10 PM Anoob Joseph <anoobj@marvell.com> wrote:
> > The asymmetric crypto library is experimental. Changes to experimental code
> paths is allowed, right?
>
> The asymmetric crypto enum is referenced by a function part of the stable ABI.
> It is possible to waive this enum, if we are sure no use out of the
> experimental asym crypto APIs is possible.
>
> The rest of the changes touch stable symbols.
>
> Adding the abidiff report:
>
> [C]'function void rte_cryptodev_info_get(uint8_t,
> rte_cryptodev_info*)' at rte_cryptodev.c:1115:1 has some indirect
> sub-type changes:
> parameter 2 of type 'rte_cryptodev_info*' has sub-type changes:
> in pointed to type 'struct rte_cryptodev_info' at rte_cryptodev.h:468:1:
> type size hasn't changed
> 1 data member change:
> type of 'const rte_cryptodev_capabilities*
> rte_cryptodev_info::capabilities' changed:
> in pointed to type 'const rte_cryptodev_capabilities':
> in unqualified underlying type 'struct
> rte_cryptodev_capabilities' at rte_cryptodev.h:176:1:
> type size hasn't changed
> 1 data member change:
> type of '__anonymous_union__ ' changed:
> type size hasn't changed
> 1 data member change:
> type of 'rte_cryptodev_asymmetric_capability
> __anonymous_union__::asym' changed:
> type size hasn't changed
> 1 data member change:
> type of
> 'rte_cryptodev_asymmetric_xform_capability
> rte_cryptodev_asymmetric_capability::xform_capa' changed:
> type size hasn't changed
> 1 data member change:
> type of 'rte_crypto_asym_xform_type
> rte_cryptodev_asymmetric_xform_capability::xform_type' changed:
> type size hasn't changed
> 2 enumerator insertions:
>
> 'rte_crypto_asym_xform_type::RTE_CRYPTO_ASYM_XFORM_ECDSA' value '7'
>
> 'rte_crypto_asym_xform_type::RTE_CRYPTO_ASYM_XFORM_ECPM' value '8'
> 1 enumerator change:
>
> 'rte_crypto_asym_xform_type::RTE_CRYPTO_ASYM_XFORM_TYPE_LIST_END'
> from
> value '7' to '9' at rte_crypto_asym.h:60:1
>
I believe these enums will be used only in case of ASYM case which is experimental.
>
> [C]'function int
> rte_cryptodev_get_aead_algo_enum(rte_crypto_aead_algorithm*, const
> char*)' at rte_cryptodev.c:239:1 has some indirect sub-type changes:
> parameter 1 of type 'rte_crypto_aead_algorithm*' has sub-type changes:
> in pointed to type 'enum rte_crypto_aead_algorithm' at
> rte_crypto_sym.h:346:1:
> type size hasn't changed
> 1 enumerator insertion:
> 'rte_crypto_aead_algorithm::RTE_CRYPTO_AEAD_CHACHA20_POLY1305'
> value '3'
> 1 enumerator change:
> 'rte_crypto_aead_algorithm::RTE_CRYPTO_AEAD_LIST_END' from
> value '3' to '4' at rte_crypto_sym.h:346:1
>
>
> [C]'const char* rte_crypto_aead_algorithm_strings[1]' was changed at
> rte_crypto_sym.h:358:1:
> size of symbol (in bytes) changed from 24 to 32
>
>
+Fiona and Arek
We may need to revert the chacha-poly patches.
> --
> David Marchand
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH v2 0/4] add ABI checks
2020-01-29 17:26 9% ` [dpdk-dev] [PATCH v2 0/4] " David Marchand
2020-01-29 17:26 2% ` [dpdk-dev] [PATCH v2 3/4] build: test meson installation David Marchand
2020-01-29 17:26 28% ` [dpdk-dev] [PATCH v2 4/4] add ABI checks David Marchand
@ 2020-01-30 10:57 4% ` Luca Boccassi
2 siblings, 0 replies; 200+ results
From: Luca Boccassi @ 2020-01-30 10:57 UTC (permalink / raw)
To: David Marchand, dev
On Wed, 2020-01-29 at 18:26 +0100, David Marchand wrote:
> Here is the current state of the ABI checks.
>
> libabigail has some issues when mixing dump and so files compiled
> with
> clang [1], so for now, all checks are done on dumps only.
> libabigail 1.0-rc3 in Xenial reported issues that disappear with the
> version 1.2 in Bionic.
>
> To avoid getting warnings on internal types like [2], the checks now
> make
> use of the public headers part of a dpdk installation (patch 2 and 3
> to
> prepare for this).
>
> Some internal rte_hash headers were installed by meson, so patch 1
> fixes
> this.
>
> The most important point, abidiff complains on the rc1 cryptodev
> changes:
> - Chacha20-Poly1305 AEAD support,
> - ECPM and ECDSA support
>
> A suppression rule has been put on the internal type
> rte_cryptodev_ops.
> But other changes are an ABI breakage afaiu. I put suppression rules
> on
> them so that we can run the checks, but some action must be taken for
> 20.02 if my analysis is confirmed.
>
> Special thanks to Dodji the libabigail maintainer for helping on this
> topic.
>
> 1:
> https://sourceware.org/bugzilla/show_bug.cgi?id=25409
>
> 2:
> http://inbox.dpdk.org/dev/CAJFAV8yFKoDZROX9Mkyp7pDMvXw3e7mHwxjfrcjD5ZoFB2tZ8w@mail.gmail.com/
>
>
> --
> David Marchand
>
> David Marchand (4):
> hash: fix meson headers packaging
> build: split build helper
> build: test meson installation
> add ABI checks
>
> .ci/linux-build.sh | 29 ++++++++-
> .travis.yml | 20 +++++-
> MAINTAINERS | 2 +
> devtools/check-abi.sh | 59 +++++++++++++++++
> devtools/dpdk.abignore | 20 ++++++
> devtools/gen-abi.sh | 26 ++++++++
> devtools/test-build.sh | 45 ++++++++++++-
> devtools/test-meson-builds.sh | 98 ++++++++++++++++++++++-----
> --
> devtools/test-null.sh | 1 +
> doc/guides/contributing/patches.rst | 13 ++++
> lib/librte_hash/meson.build | 5 +-
> 11 files changed, 286 insertions(+), 32 deletions(-)
> create mode 100755 devtools/check-abi.sh
> create mode 100644 devtools/dpdk.abignore
> create mode 100755 devtools/gen-abi.sh
>
Series-acked-by: Luca Boccassi <bluca@debian.org>
--
Kind regards,
Luca Boccassi
^ permalink raw reply [relevance 4%]
* [dpdk-dev] DPDK Release Status Meeting 30/01/2020
@ 2020-01-30 12:16 4% Ferruh Yigit
0 siblings, 0 replies; 200+ results
From: Ferruh Yigit @ 2020-01-30 12:16 UTC (permalink / raw)
To: dpdk-dev; +Cc: Thomas Monjalon
Minutes 30 January 2020
-----------------------
Agenda:
* Release Dates
* -rc1 status
* Subtrees
* OvS
* Opens
Participants:
* Debian/Microsoft
* Intel
* Mellanox
* NXP
* Red Hat
Release Dates
-------------
* v20.02 dates:
* -rc2 pushed to *Wednesday 5 February 2020*
* -rc3 Monday 10 February 2020
* Release pushed to *Wednesday 19 February 2020*
* Release pushed because of extended PRC holidays
* v20.05 dates:
* Proposal/V1: Friday 6 March 2020
* Integration/Merge/RC1: Friday 10 April 2020
* Release: Friday 1 May 2020
-rc1 Status
-----------
* No new major/critical issue reported
Subtrees
--------
* main
* Windows patches waiting for new version, if they are ready they can be
merged to -rc2
* Will merge more patches this afternoon and tomorrow
* ABI tooling will be ready for -rc2
* meter library breaks the ABI and needs fixing before release
* next-net
* Nothing interesting, mostly fixes for -rc2
* There is a big mlx patchset in the queue, mlx tree will be ready
today/tomorrow
* mlx vdpa patchset is waiting for review from Maxime
* next-net-crypto
* armv8 crypto patch merged
* Vilademir send a new version for SAD patchset which reduces the performance
drop
* SAD patch discussed in techboard, functionality has priority in sample
applications, so patchset can proceed with existing performance drop.
* CPU crypto will be reviewed if there is enough time, -rc2 being pushed may
provide required time for review and if it is ready it will be considered
for -rc2
* There is an ABI break in crypto tree, waiting input from Fiona, will try to
solve it, if not will revert the commit causing it
* next-net-eventdev
* no update
* next-net-virtio
* Will get fixes for -rc2
* virtio vdpa postponed to next release
* LTS
* 17.11.10-rc1 released, please test and report results
* https://mails.dpdk.org/archives/dev/2020-January/154915.html
* Got test results from Intel, Red Hat & Mellanox
* QAT regression reported by Intel testing, need help from developers
* mlx reported regressions, needs more details
* Will wait two more week for above issues clarified
* Rough release date is mid February
* 18.11.6-rc2 released, please test and report results
* https://mails.dpdk.org/archives/dev/2020-January/155132.html
* It has been confirmed regressions reported in -rc1 fixed in -rc2
* OvS completed testing with -rc2, no issue found
* Release expected sometime this week
OvS
---
* Validated LTS releases, they are good to go
* 2.13 release expected on mid February
Opens
-----
* Coverity run last week, there are outstanding issues, please check:
https://scan.coverity.com/projects/dpdk-data-plane-development-kit?tab=overview
* John sent email to individuals whose patch causing a Coverity issue
But everyone is welcome to fix the issues listed
* For information, icc compiler support reduced by technical board, warning
disabling will be done in global level and having warnings will be allowed
DPDK Release Status Meetings
============================
The DPDK Release Status Meeting is intended for DPDK Committers to discuss
the status of the master tree and sub-trees, and for project managers to
track progress or milestone dates.
The meeting occurs on Thursdays at 8:30 UTC. If you wish to attend just
send an email to "John McNamara <john.mcnamara@intel.com>" for the invite.
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [RFC v2] meter: fix ABI break due to experimental tag removal
2020-01-29 16:43 3% ` [dpdk-dev] [RFC v2] " Ferruh Yigit
2020-01-29 17:49 4% ` Andrzej Ostruszka
@ 2020-01-30 12:33 7% ` Thomas Monjalon
2020-01-30 12:57 4% ` Luca Boccassi
2020-01-31 9:25 4% ` Ferruh Yigit
1 sibling, 2 replies; 200+ results
From: Thomas Monjalon @ 2020-01-30 12:33 UTC (permalink / raw)
To: Ferruh Yigit
Cc: Neil Horman, Cristian Dumitrescu, Eelco Chaudron, dev,
Luca Boccassi, David Marchand, Bruce Richardson, Ian Stokes,
Andrzej Ostruszka
Hi,
I disagree with the need of this patch.
The symbol was experimental, meaning we can change it.
Removing experimental tag is not an ABI break.
29/01/2020 17:43, Ferruh Yigit:
> Duplicated the existing symbol and versioned one as experimental and
> other as stable.
[..]
> --- a/lib/librte_meter/rte_meter_version.map
> +++ b/lib/librte_meter/rte_meter_version.map
> @@ -20,4 +20,12 @@ DPDK_20.0.1 {
> rte_meter_trtcm_rfc4115_color_blind_check;
> rte_meter_trtcm_rfc4115_config;
> rte_meter_trtcm_rfc4115_profile_config;
> +
> } DPDK_20.0;
> +
> +EXPERIMENTAL {
> + global:
> +
> + rte_meter_trtcm_rfc4115_config;
> + rte_meter_trtcm_rfc4115_profile_config;
> +};
^ permalink raw reply [relevance 7%]
* Re: [dpdk-dev] [RFC v2] meter: fix ABI break due to experimental tag removal
2020-01-30 12:33 7% ` Thomas Monjalon
@ 2020-01-30 12:57 4% ` Luca Boccassi
2020-01-30 14:17 8% ` Thomas Monjalon
2020-01-31 9:25 4% ` Ferruh Yigit
1 sibling, 1 reply; 200+ results
From: Luca Boccassi @ 2020-01-30 12:57 UTC (permalink / raw)
To: Thomas Monjalon, Ferruh Yigit
Cc: Neil Horman, Cristian Dumitrescu, Eelco Chaudron, dev,
David Marchand, Bruce Richardson, Ian Stokes, Andrzej Ostruszka
On Thu, 2020-01-30 at 13:33 +0100, Thomas Monjalon wrote:
> Hi,
>
> I disagree with the need of this patch.
> The symbol was experimental, meaning we can change it.
> Removing experimental tag is not an ABI break.
Hi,
This symbol change was requested for backport in 19.11.x, and
experimental or not I'm not too keen on backward incompatible changes
to the public interface in an _LTS point release_. The compromise was
to see if we could support both symbols version, which makes the change
backward compatible.
If you prefer not to have this patch in mainline I'm also fine in
taking it just for the LTS. I agree with you that it is not required
for mainline releases (although nicer for me if it's a backport rather
than a new change).
> 29/01/2020 17:43, Ferruh Yigit:
> > Duplicated the existing symbol and versioned one as experimental
> > and
> > other as stable.
>
> [..]
> > --- a/lib/librte_meter/rte_meter_version.map
> > +++ b/lib/librte_meter/rte_meter_version.map
> > @@ -20,4 +20,12 @@ DPDK_20.0.1 {
> > rte_meter_trtcm_rfc4115_color_blind_check;
> > rte_meter_trtcm_rfc4115_config;
> > rte_meter_trtcm_rfc4115_profile_config;
> > +
> > } DPDK_20.0;
> > +
> > +EXPERIMENTAL {
> > + global:
> > +
> > + rte_meter_trtcm_rfc4115_config;
> > + rte_meter_trtcm_rfc4115_profile_config;
> > +};
>
>
>
>
>
--
Kind regards,
Luca Boccassi
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
2020-01-29 18:10 8% ` Anoob Joseph
2020-01-29 20:03 7% ` David Marchand
@ 2020-01-30 13:06 8% ` Trahe, Fiona
2020-01-30 15:59 4% ` Thomas Monjalon
1 sibling, 1 reply; 200+ results
From: Trahe, Fiona @ 2020-01-30 13:06 UTC (permalink / raw)
To: Anoob Joseph, Thomas Monjalon, akhil.goyal
Cc: dev, David Marchand, Richardson, Bruce, nhorman, Mcnamara, John,
Trahe, Fiona, Kusztal, ArkadiuszX, Yigit, Ferruh
We were unaware the LIST_END change could constitute an ABI breakage, but can see how it affects the array size when picked up.
We're exploring options.
I agree with Anoob's point that if we don't allow the LIST_END to be modified, then it means no feature can be implemented without ABI breakage.
Anyone object to removing those LIST_END elements - or have a better suggestion? Would have to be in 20.11 I suppose.
> -----Original Message-----
> From: dev <dev-bounces@dpdk.org> On Behalf Of Anoob Joseph
> Sent: Wednesday, January 29, 2020 6:10 PM
> To: Thomas Monjalon <thomas@monjalon.net>; akhil.goyal@nxp.com
> Cc: dev@dpdk.org; David Marchand <david.marchand@redhat.com>; Richardson, Bruce
> <bruce.richardson@intel.com>; nhorman@tuxdriver.com; Mcnamara, John
> <john.mcnamara@intel.com>
> Subject: Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
>
> Hi Thomas,
>
> The asymmetric crypto library is experimental. Changes to experimental code paths is allowed, right?
>
> Also, I was wondering why changing the LIST_END would cause breakage. Before we introduced the ABI
> checks and ABI freeze policy, it was always allowed to add enums to the end. I'm just trying to
> understand the real impact of this case.
>
> If we don't allow the LIST_END to be modified, then it means no feature can be implemented in
> between. And the best way to overcome that would be to just remove the LIST_END or set LIST_END to
> a very high value.
>
> Thanks,
> Anoob
>
> > -----Original Message-----
> > From: dev <dev-bounces@dpdk.org> On Behalf Of Thomas Monjalon
> > Sent: Wednesday, January 29, 2020 11:13 PM
> > To: akhil.goyal@nxp.com; Anoob Joseph <anoobj@marvell.com>
> > Cc: dev@dpdk.org; David Marchand <david.marchand@redhat.com>;
> > bruce.richardson@intel.com; nhorman@tuxdriver.com; John McNamara
> > <john.mcnamara@intel.com>
> > Subject: Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
> >
> > Anoob, Akhil,
> >
> > Please we need to revert or fix the ABI breakages in cryptodev very soon.
> > The FIXME section below must be empty.
> >
> > Thanks
> >
> > 29/01/2020 18:26, David Marchand:
> > > We currently have issues reported for librte_crypto recent changes for
> > > which suppression rules have been added too.
> > [..]
> > > --- /dev/null
> > > +++ b/devtools/dpdk.abignore
> > > +; FIXME
> > > +[suppress_type]
> > > + type_kind = enum
> > > + name = rte_crypto_aead_algorithm
> > > + changed_enumerators = RTE_CRYPTO_AEAD_LIST_END
> > > +[suppress_type]
> > > + type_kind = enum
> > > + name = rte_crypto_asym_xform_type
> > > + changed_enumerators = RTE_CRYPTO_ASYM_XFORM_TYPE_LIST_END
> > > +[suppress_variable]
> > > + name = rte_crypto_aead_algorithm_strings
> >
> >
> >
^ permalink raw reply [relevance 8%]
* Re: [dpdk-dev] [RFC v2] meter: fix ABI break due to experimental tag removal
2020-01-30 12:57 4% ` Luca Boccassi
@ 2020-01-30 14:17 8% ` Thomas Monjalon
2020-01-30 14:21 4% ` Luca Boccassi
0 siblings, 1 reply; 200+ results
From: Thomas Monjalon @ 2020-01-30 14:17 UTC (permalink / raw)
To: Ferruh Yigit, Luca Boccassi
Cc: Neil Horman, Cristian Dumitrescu, Eelco Chaudron, dev,
David Marchand, Bruce Richardson, Ian Stokes, Andrzej Ostruszka
30/01/2020 13:57, Luca Boccassi:
> On Thu, 2020-01-30 at 13:33 +0100, Thomas Monjalon wrote:
> > Hi,
> >
> > I disagree with the need of this patch.
> > The symbol was experimental, meaning we can change it.
> > Removing experimental tag is not an ABI break.
>
> Hi,
>
> This symbol change was requested for backport in 19.11.x, and
> experimental or not I'm not too keen on backward incompatible changes
> to the public interface in an _LTS point release_. The compromise was
> to see if we could support both symbols version, which makes the change
> backward compatible.
>
> If you prefer not to have this patch in mainline I'm also fine in
> taking it just for the LTS. I agree with you that it is not required
> for mainline releases (although nicer for me if it's a backport rather
> than a new change).
I would like to avoid opening the door for maintaining the experimental ABI
in the mainline. Please take it directly in the LTS.
The next question is to know whether we really want to have such patch in LTS.
Anyway, 19.11.0 has this symbol as experimental.
How adding a non-experimental version of the function in 19.11.1 will change
the ABI status of the whole 19.11 branch?
> > 29/01/2020 17:43, Ferruh Yigit:
> > > Duplicated the existing symbol and versioned one as experimental
> > > and
> > > other as stable.
> >
> > [..]
> > > --- a/lib/librte_meter/rte_meter_version.map
> > > +++ b/lib/librte_meter/rte_meter_version.map
> > > @@ -20,4 +20,12 @@ DPDK_20.0.1 {
> > > rte_meter_trtcm_rfc4115_color_blind_check;
> > > rte_meter_trtcm_rfc4115_config;
> > > rte_meter_trtcm_rfc4115_profile_config;
> > > +
> > > } DPDK_20.0;
> > > +
> > > +EXPERIMENTAL {
> > > + global:
> > > +
> > > + rte_meter_trtcm_rfc4115_config;
> > > + rte_meter_trtcm_rfc4115_profile_config;
> > > +};
^ permalink raw reply [relevance 8%]
* [dpdk-dev] [PATCH] doc: deprecate using MAX values as array size
@ 2020-01-30 14:20 16% Ferruh Yigit
0 siblings, 0 replies; 200+ results
From: Ferruh Yigit @ 2020-01-30 14:20 UTC (permalink / raw)
To: Neil Horman, John McNamara, Marko Kovacevic; +Cc: dev
Adding the deprecation notice as reminder for next ABI breakage release
(20.11).
This one time breakage is required to be able to extend enum/define
without breaking ABI.
Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
---
doc/guides/rel_notes/deprecation.rst | 14 ++++++++++++++
1 file changed, 14 insertions(+)
diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
index dfcca87ab..99d81564a 100644
--- a/doc/guides/rel_notes/deprecation.rst
+++ b/doc/guides/rel_notes/deprecation.rst
@@ -38,6 +38,20 @@ Deprecation Notices
remove it from the externally visible ABI and allow it to be updated in the
future.
+* lib: will fix extending some enum/define breaking the ABI. There are multiple
+ samples in DPDK that enum/define terminated with a ``.*MAX.*`` value which is
+ used by iterators, and arrays holding these values are sized with this
+ ``.*MAX.*`` value. So extending this enum/define increases the ``.*MAX.*``
+ value which increases the size of the array and depending on how/where the
+ array is used this may break the ABI.
+ ``RTE_ETH_FLOW_MAX`` is one sample of the mentioned case, adding a new flow
+ type will break the ABI because of ``flex_mask[RTE_ETH_FLOW_MAX]`` array
+ usage in following public struct hierarchy:
+ ``rte_eth_fdir_flex_conf -> rte_fdir_conf -> rte_eth_conf (in the middle)``.
+ Need to identify this kind of usages and fix in 20.11, otherwise this blocks
+ us extending existing enum/define.
+ One solution can be using a fixed size array instead of ``.*MAX.*`` value.
+
* dpaa2: removal of ``rte_dpaa2_memsegs`` structure which has been replaced
by a pa-va search library. This structure was earlier being used for holding
memory segments used by dpaa2 driver for faster pa->va translation. This
--
2.24.1
^ permalink raw reply [relevance 16%]
* Re: [dpdk-dev] [RFC v2] meter: fix ABI break due to experimental tag removal
2020-01-30 14:17 8% ` Thomas Monjalon
@ 2020-01-30 14:21 4% ` Luca Boccassi
2020-01-30 15:55 4% ` Thomas Monjalon
0 siblings, 1 reply; 200+ results
From: Luca Boccassi @ 2020-01-30 14:21 UTC (permalink / raw)
To: Thomas Monjalon, Ferruh Yigit
Cc: Neil Horman, Cristian Dumitrescu, Eelco Chaudron, dev,
David Marchand, Bruce Richardson, Ian Stokes, Andrzej Ostruszka
On Thu, 2020-01-30 at 15:17 +0100, Thomas Monjalon wrote:
> 30/01/2020 13:57, Luca Boccassi:
> > On Thu, 2020-01-30 at 13:33 +0100, Thomas Monjalon wrote:
> > > Hi,
> > >
> > > I disagree with the need of this patch.
> > > The symbol was experimental, meaning we can change it.
> > > Removing experimental tag is not an ABI break.
> >
> > Hi,
> >
> > This symbol change was requested for backport in 19.11.x, and
> > experimental or not I'm not too keen on backward incompatible
> > changes
> > to the public interface in an _LTS point release_. The compromise
> > was
> > to see if we could support both symbols version, which makes the
> > change
> > backward compatible.
> >
> > If you prefer not to have this patch in mainline I'm also fine in
> > taking it just for the LTS. I agree with you that it is not
> > required
> > for mainline releases (although nicer for me if it's a backport
> > rather
> > than a new change).
>
> I would like to avoid opening the door for maintaining the
> experimental ABI
> in the mainline. Please take it directly in the LTS.
>
> The next question is to know whether we really want to have such
> patch in LTS.
> Anyway, 19.11.0 has this symbol as experimental.
> How adding a non-experimental version of the function in 19.11.1 will
> change
> the ABI status of the whole 19.11 branch?
The problem is not adding the new symbol, but removing the experimental
one. Changing the version of the symbol was requested by OVS for
inclusion in 19.11.
--
Kind regards,
Luca Boccassi
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [RFC v2] meter: fix ABI break due to experimental tag removal
2020-01-30 14:21 4% ` Luca Boccassi
@ 2020-01-30 15:55 4% ` Thomas Monjalon
2020-01-30 16:04 4% ` Luca Boccassi
0 siblings, 1 reply; 200+ results
From: Thomas Monjalon @ 2020-01-30 15:55 UTC (permalink / raw)
To: Luca Boccassi
Cc: Ferruh Yigit, Neil Horman, Cristian Dumitrescu, Eelco Chaudron,
dev, David Marchand, Bruce Richardson, Ian Stokes,
Andrzej Ostruszka
30/01/2020 15:21, Luca Boccassi:
> On Thu, 2020-01-30 at 15:17 +0100, Thomas Monjalon wrote:
> > 30/01/2020 13:57, Luca Boccassi:
> > > On Thu, 2020-01-30 at 13:33 +0100, Thomas Monjalon wrote:
> > > > Hi,
> > > >
> > > > I disagree with the need of this patch.
> > > > The symbol was experimental, meaning we can change it.
> > > > Removing experimental tag is not an ABI break.
> > >
> > > Hi,
> > >
> > > This symbol change was requested for backport in 19.11.x, and
> > > experimental or not I'm not too keen on backward incompatible
> > > changes
> > > to the public interface in an _LTS point release_. The compromise
> > > was
> > > to see if we could support both symbols version, which makes the
> > > change
> > > backward compatible.
> > >
> > > If you prefer not to have this patch in mainline I'm also fine in
> > > taking it just for the LTS. I agree with you that it is not
> > > required
> > > for mainline releases (although nicer for me if it's a backport
> > > rather
> > > than a new change).
> >
> > I would like to avoid opening the door for maintaining the
> > experimental ABI
> > in the mainline. Please take it directly in the LTS.
> >
> > The next question is to know whether we really want to have such
> > patch in LTS.
> > Anyway, 19.11.0 has this symbol as experimental.
> > How adding a non-experimental version of the function in 19.11.1 will
> > change
> > the ABI status of the whole 19.11 branch?
>
> The problem is not adding the new symbol, but removing the experimental
> one. Changing the version of the symbol was requested by OVS for
> inclusion in 19.11.
Yes, sorry, this is what I meant.
Given 19.11.0 already has the symbol as experimental,
and that applications like OVS had to accept it as experimental,
why removing experimental tag in 19.11.1?
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
2020-01-30 13:06 8% ` Trahe, Fiona
@ 2020-01-30 15:59 4% ` Thomas Monjalon
2020-01-30 16:42 7% ` Ferruh Yigit
2020-01-30 23:49 4% ` Ananyev, Konstantin
0 siblings, 2 replies; 200+ results
From: Thomas Monjalon @ 2020-01-30 15:59 UTC (permalink / raw)
To: Anoob Joseph, akhil.goyal, Trahe, Fiona
Cc: dev, David Marchand, Richardson, Bruce, nhorman, Mcnamara, John,
Trahe, Fiona, Kusztal, ArkadiuszX, Yigit, Ferruh
30/01/2020 14:06, Trahe, Fiona:
> We were unaware the LIST_END change could constitute an ABI breakage, but can see how it affects the array size when picked up.
> We're exploring options.
>
> I agree with Anoob's point that if we don't allow the LIST_END to be modified, then it means no feature can be implemented without ABI breakage.
> Anyone object to removing those LIST_END elements - or have a better suggestion? Would have to be in 20.11 I suppose.
Yes, having max value right after the last value is ridiculous,
it prevents adding any value.
In 20.11, we should remove all these *_END and *_MAX from API enums
and replace them with a separate #define with reasonnable maximums.
^ permalink raw reply [relevance 4%]
* [dpdk-dev] [PATCH v3 0/4] add ABI checks
2020-01-29 17:26 9% ` [dpdk-dev] [PATCH v2 0/4] " David Marchand
@ 2020-01-30 16:00 9% ` David Marchand
2020-01-30 16:00 3% ` [dpdk-dev] [PATCH v3 3/4] build: test meson installation David Marchand
2020-01-30 16:00 28% ` [dpdk-dev] [PATCH v3 4/4] add ABI checks David Marchand
2020-02-02 21:08 9% ` [dpdk-dev] [PATCH v4 0/3] " David Marchand
3 siblings, 2 replies; 200+ results
From: David Marchand @ 2020-01-30 16:00 UTC (permalink / raw)
To: dev
Cc: thomas, bruce.richardson, kevin.laatz, aconole, nhorman,
akhil.goyal, anoobj, bluca, fiona.trahe, ferruh.yigit
Here is the current state of the ABI checks.
libabigail has some issues when mixing dump and so files compiled with
clang [1], so for now, all checks are done on dumps only.
libabigail 1.0-rc3 in Xenial reported issues that disappear with the
version 1.2 in Bionic.
To avoid getting warnings on internal types like [2], the checks now make
use of the public headers part of a dpdk installation (patch 2 and 3 to
prepare for this).
Some internal rte_hash headers were installed by meson, so patch 1 fixes
this.
The most important point, abidiff complains on the rc1 cryptodev changes:
- Chacha20-Poly1305 AEAD support,
- ECPM and ECDSA support
A suppression rule has been put on the internal type rte_cryptodev_ops.
But other changes are an ABI breakage afaiu. I put suppression rules on
them so that we can run the checks, but some action must be taken for
20.02 if my analysis is confirmed.
Special thanks to Dodji the libabigail maintainer for helping on this
topic.
1: https://sourceware.org/bugzilla/show_bug.cgi?id=25409
2: http://inbox.dpdk.org/dev/CAJFAV8yFKoDZROX9Mkyp7pDMvXw3e7mHwxjfrcjD5ZoFB2tZ8w@mail.gmail.com/
--
David Marchand
Changelog since v2:
- incorporated offlist Thomas comments wrt to existing build envs (do
not change --prefix, force builds with debug enabled)
David Marchand (4):
hash: fix meson headers packaging
build: split build helper
build: test meson installation
add ABI checks
.ci/linux-build.sh | 29 ++++++++-
.travis.yml | 20 +++++-
MAINTAINERS | 2 +
devtools/check-abi.sh | 59 +++++++++++++++++
devtools/dpdk.abignore | 21 +++++++
devtools/gen-abi.sh | 26 ++++++++
devtools/test-build.sh | 51 +++++++++++++--
devtools/test-meson-builds.sh | 98 ++++++++++++++++++++++-------
doc/guides/contributing/patches.rst | 15 +++++
lib/librte_hash/meson.build | 5 +-
10 files changed, 292 insertions(+), 34 deletions(-)
create mode 100755 devtools/check-abi.sh
create mode 100644 devtools/dpdk.abignore
create mode 100755 devtools/gen-abi.sh
--
2.23.0
^ permalink raw reply [relevance 9%]
* [dpdk-dev] [PATCH v3 3/4] build: test meson installation
2020-01-30 16:00 9% ` [dpdk-dev] [PATCH v3 " David Marchand
@ 2020-01-30 16:00 3% ` David Marchand
2020-01-30 22:17 3% ` Thomas Monjalon
2020-01-30 16:00 28% ` [dpdk-dev] [PATCH v3 4/4] add ABI checks David Marchand
1 sibling, 1 reply; 200+ results
From: David Marchand @ 2020-01-30 16:00 UTC (permalink / raw)
To: dev
Cc: thomas, bruce.richardson, kevin.laatz, aconole, nhorman,
akhil.goyal, anoobj, bluca, fiona.trahe, ferruh.yigit,
Michael Santana
Let's test installing with meson as part of Travis and the
devtools/test-meson-builds.sh script.
The resulting headers and binaries make more sense to run checks
against, since they should be the same as what the final users get.
test-null.sh is now called on an installed testpmd.
The (future) ABI compatibility checks will also need to filter its report
against installed public headers.
Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Luca Boccassi <bluca@debian.org>
---
Changelog since v2:
- removed forced --prefix= parameter in test-meson-builds.sh to avoid
breakage of existing environments,
- simplified install with no special verbose mode,
- removed ldd call in test-null.sh since the main user of this script
finds this change too much for his eyes,
---
.ci/linux-build.sh | 6 +++++-
devtools/test-meson-builds.sh | 21 +++++++++++----------
2 files changed, 16 insertions(+), 11 deletions(-)
diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh
index ccc3a7ccd..e61aa2b0a 100755
--- a/.ci/linux-build.sh
+++ b/.ci/linux-build.sh
@@ -29,11 +29,15 @@ if [ "$BUILD_32BIT" = "1" ]; then
fi
OPTS="$OPTS --default-library=$DEF_LIB"
+OPTS="$OPTS --prefix=/usr -Dlibdir=lib"
meson build --werror -Dexamples=all $OPTS
ninja -C build
+DESTDIR=$(pwd)/install ninja -C build install
if [ "$AARCH64" != "1" ]; then
- devtools/test-null.sh
+ export LD_LIBRARY_PATH=$(pwd)/install/usr/lib
+ devtools/test-null.sh install/usr/bin/dpdk-testpmd
+ unset LD_LIBRARY_PATH
fi
if [ "$RUN_TESTS" = "1" ]; then
diff --git a/devtools/test-meson-builds.sh b/devtools/test-meson-builds.sh
index fb6c404e5..747fb4a1a 100755
--- a/devtools/test-meson-builds.sh
+++ b/devtools/test-meson-builds.sh
@@ -66,7 +66,7 @@ config () # <dir> <builddir> <meson options>
if [ -f "$builddir/build.ninja" ] ; then
return
fi
- options="--werror -Dexamples=all"
+ options="--werror -Dexamples=all -Dlibdir=lib"
for option in $DPDK_MESON_OPTIONS ; do
options="$options -D$option"
done
@@ -75,7 +75,7 @@ config () # <dir> <builddir> <meson options>
$MESON $options $dir $builddir
}
-compile () # <builddir>
+compile () # <builddir> <installdir>
{
builddir=$1
if [ -n "$TEST_MESON_BUILD_VERY_VERBOSE" ] ; then
@@ -90,6 +90,9 @@ compile () # <builddir>
echo "$ninja_cmd -C $builddir"
$ninja_cmd -C $builddir
fi
+ rm -rf $2
+ echo "DESTDIR=$2 $ninja_cmd -C $builddir install"
+ DESTDIR=$2 $ninja_cmd -C $builddir install
}
build () # <directory> <target compiler> <meson options>
@@ -102,7 +105,8 @@ build () # <directory> <target compiler> <meson options>
command -v ${CC##* } >/dev/null 2>&1 || return 0
load_env $targetcc || return 0
config $srcdir $builds_dir/$targetdir $*
- compile $builds_dir/$targetdir
+ compile $builds_dir/$targetdir \
+ $(readlink -f $builds_dir/$targetdir/install)
}
if [ "$1" = "-vv" ] ; then
@@ -134,7 +138,7 @@ ok=$(cc -march=$default_machine -E - < /dev/null > /dev/null 2>&1 || echo false)
if [ "$ok" = "false" ] ; then
default_machine='corei7'
fi
-build build-x86-default cc -Dlibdir=lib -Dmachine=$default_machine $use_shared
+build build-x86-default cc -Dmachine=$default_machine $use_shared
c=aarch64-linux-gnu-gcc
# generic armv8a with clang as host compiler
@@ -150,12 +154,9 @@ for f in $srcdir/config/arm/arm64_[bdo]*gcc ; do
unset CC
done
-# Test installation of the x86-default target, to be used for checking
-# the sample apps build using the pkg-config file for cflags and libs
-build_path=$(readlink -f $builds_dir/build-x86-default)
-export DESTDIR=$build_path/install-root
-$ninja_cmd -C $build_path install
-
+# Use the x86-default target, to check the sample apps build using the
+# pkg-config file for cflags and libs
+export DESTDIR=$(readlink -f $builds_dir/build-x86-default/install)
load_env cc
pc_file=$(find $DESTDIR -name libdpdk.pc)
export PKG_CONFIG_PATH=$(dirname $pc_file):$PKG_CONFIG_PATH
--
2.23.0
^ permalink raw reply [relevance 3%]
* [dpdk-dev] [PATCH v3 4/4] add ABI checks
2020-01-30 16:00 9% ` [dpdk-dev] [PATCH v3 " David Marchand
2020-01-30 16:00 3% ` [dpdk-dev] [PATCH v3 3/4] build: test meson installation David Marchand
@ 2020-01-30 16:00 28% ` David Marchand
2020-01-30 22:32 4% ` Thomas Monjalon
2020-01-30 22:44 4% ` Thomas Monjalon
1 sibling, 2 replies; 200+ results
From: David Marchand @ 2020-01-30 16:00 UTC (permalink / raw)
To: dev
Cc: thomas, bruce.richardson, kevin.laatz, aconole, nhorman,
akhil.goyal, anoobj, bluca, fiona.trahe, ferruh.yigit,
Michael Santana, John McNamara, Marko Kovacevic
For normal developers, those checks are disabled.
Enabling them requires a configuration that will trigger the ABI dumps
generation as part of the existing devtools/test-build.sh and
devtools/test-meson-builds.sh scripts.
Those checks are enabled in the CI for the default meson options on x86
and aarch64 so that proposed patches are validated via our CI robot.
A cache of the ABI is stored in travis jobs to avoid rebuilding too
often.
Checks can be only informational by setting ABI_CHECKS_WARN_ONLY when
breaking the ABI in a future release.
Explicit suppression rules have been added on internal structures
exposed to crypto drivers as the current ABI policy does not apply to
them.
This could be improved in the future by carefully splitting the headers
content with application and driver "users" in mind.
We currently have issues reported for librte_crypto recent changes for
which suppression rules have been added too.
Mellanox glue libraries are explicitly skipped as they are not part of
the application ABI.
Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Luca Boccassi <bluca@debian.org>
---
Changelog since v2:
- forced -g / buildtype=debugoptimised in the test scripts so that
we can check ABI in existing environments,
- little update on the documentation,
Changelog since v1:
- reworked the scripts so that the build test scripts clone and build the
reference automatically. A developer only needs to set one variable to
enable the checks,
- meson builds are done with debug so that abidiff can inspect the
structures,
- abidiff checks only public types by looking at installed headers,
- abidiff has some issue when comparing a dump with a .so built with clang
so all diff are now done with dump files only,
- suppression rules have been added to waive warnings on exposed internal
types,
- an abi breakage has been reported on changes in cryptodev.
For now, suppression rules have been put in place to let the CI run,
---
.ci/linux-build.sh | 23 +++++++++++
.travis.yml | 20 +++++++++-
MAINTAINERS | 2 +
devtools/check-abi.sh | 59 +++++++++++++++++++++++++++++
devtools/dpdk.abignore | 21 ++++++++++
devtools/gen-abi.sh | 26 +++++++++++++
devtools/test-build.sh | 51 ++++++++++++++++++++++---
devtools/test-meson-builds.sh | 37 +++++++++++++++++-
doc/guides/contributing/patches.rst | 15 ++++++++
9 files changed, 246 insertions(+), 8 deletions(-)
create mode 100755 devtools/check-abi.sh
create mode 100644 devtools/dpdk.abignore
create mode 100755 devtools/gen-abi.sh
diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh
index e61aa2b0a..95bd869c3 100755
--- a/.ci/linux-build.sh
+++ b/.ci/linux-build.sh
@@ -30,6 +30,7 @@ fi
OPTS="$OPTS --default-library=$DEF_LIB"
OPTS="$OPTS --prefix=/usr -Dlibdir=lib"
+OPTS="$OPTS --buildtype=debugoptimized"
meson build --werror -Dexamples=all $OPTS
ninja -C build
DESTDIR=$(pwd)/install ninja -C build install
@@ -40,6 +41,28 @@ if [ "$AARCH64" != "1" ]; then
unset LD_LIBRARY_PATH
fi
+if [ "$ABI_CHECKS" = "1" ]; then
+ REF_GIT_REPO=${REF_GIT_REPO:-https://dpdk.org/git/dpdk}
+ REF_GIT_TAG=${REF_GIT_TAG:-v19.11}
+
+ if [ "$(cat reference/VERSION 2>/dev/null)" != "$REF_GIT_TAG" ]; then
+ rm -rf reference
+ fi
+
+ if [ ! -d reference ]; then
+ refsrcdir=$(readlink -f $(pwd)/../dpdk-$REF_GIT_TAG)
+ git clone --single-branch -b $REF_GIT_TAG $REF_GIT_REPO $refsrcdir
+ meson --werror $OPTS $refsrcdir $refsrcdir/build
+ ninja -C $refsrcdir/build
+ DESTDIR=$(pwd)/reference ninja -C $refsrcdir/build install
+ devtools/gen-abi.sh reference
+ echo $REF_GIT_TAG > reference/VERSION
+ fi
+
+ devtools/gen-abi.sh install
+ devtools/check-abi.sh reference install ${ABI_CHECKS_WARN_ONLY:-}
+fi
+
if [ "$RUN_TESTS" = "1" ]; then
sudo meson test -C build --suite fast-tests -t 3
fi
diff --git a/.travis.yml b/.travis.yml
index 8162f1c05..22539d823 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -1,5 +1,8 @@
language: c
-cache: ccache
+cache:
+ ccache: true
+ directories:
+ - reference
compiler:
- gcc
- clang
@@ -21,7 +24,7 @@ aarch64_packages: &aarch64_packages
extra_packages: &extra_packages
- *required_packages
- - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+ - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4, abigail-tools]
build_32b_packages: &build_32b_packages
- *required_packages
@@ -151,5 +154,18 @@ matrix:
packages:
- *required_packages
- *doc_packages
+ - env: DEF_LIB="shared" EXTRA_PACKAGES=1 ABI_CHECKS=1
+ compiler: gcc
+ addons:
+ apt:
+ packages:
+ - *extra_packages
+ - env: DEF_LIB="shared" EXTRA_PACKAGES=1 ABI_CHECKS=1
+ arch: arm64
+ compiler: gcc
+ addons:
+ apt:
+ packages:
+ - *extra_packages
script: ./.ci/${TRAVIS_OS_NAME}-build.sh
diff --git a/MAINTAINERS b/MAINTAINERS
index 94bccae6d..6dae4ee63 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -144,8 +144,10 @@ M: Neil Horman <nhorman@tuxdriver.com>
F: lib/librte_eal/common/include/rte_compat.h
F: lib/librte_eal/common/include/rte_function_versioning.h
F: doc/guides/rel_notes/deprecation.rst
+F: devtools/check-abi.sh
F: devtools/check-abi-version.sh
F: devtools/check-symbol-change.sh
+F: devtools/gen-abi.sh
F: devtools/update-abi.sh
F: devtools/update_version_map_abi.py
F: devtools/validate-abi.sh
diff --git a/devtools/check-abi.sh b/devtools/check-abi.sh
new file mode 100755
index 000000000..5872499ec
--- /dev/null
+++ b/devtools/check-abi.sh
@@ -0,0 +1,59 @@
+#!/bin/sh -e
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright (c) 2019 Red Hat, Inc.
+
+if [ $# != 2 ] && [ $# != 3 ]; then
+ echo "Usage: $0 refdir newdir [warnonly]"
+ exit 1
+fi
+
+refdir=$1
+newdir=$2
+warnonly=${3:-}
+ABIDIFF_OPTIONS="--suppr $(dirname $0)/dpdk.abignore --no-added-syms"
+
+if [ ! -d $refdir ]; then
+ echo "Error: reference directory '$refdir' does not exist."
+ exit 1
+fi
+incdir=$(find $refdir -type d -a -name include)
+if [ -z "$incdir" ] || [ ! -e "$incdir" ]; then
+ echo "WARNING: could not identify a include directory for $refdir, expect false positives..."
+else
+ ABIDIFF_OPTIONS="$ABIDIFF_OPTIONS --headers-dir1 $incdir"
+fi
+
+if [ ! -d $newdir ]; then
+ echo "Error: directory to check '$newdir' does not exist."
+ exit 1
+fi
+incdir2=$(find $newdir -type d -a -name include)
+if [ -z "$incdir2" ] || [ ! -e "$incdir2" ]; then
+ echo "WARNING: could not identify a include directory for $newdir, expect false positives..."
+else
+ ABIDIFF_OPTIONS="$ABIDIFF_OPTIONS --headers-dir2 $incdir2"
+fi
+
+error=
+for dump in $(find $refdir -name "*.dump"); do
+ name=$(basename $dump)
+ # skip glue drivers, example librte_pmd_mlx5_glue.dump
+ # We can't rely on a suppression rule for now:
+ # https://sourceware.org/bugzilla/show_bug.cgi?id=25480
+ if [ "$name" != "${name%%_glue.dump}" ]; then
+ echo "Skipping ${dump}..."
+ continue
+ fi
+ dump2=$(find $newdir -name $name)
+ if [ -z "$dump2" ] || [ ! -e "$dump2" ]; then
+ echo "Error: can't find $name in $newdir"
+ error=1
+ continue
+ fi
+ if ! abidiff $ABIDIFF_OPTIONS $dump $dump2; then
+ echo "Error: ABI issue reported for 'abidiff $ABIDIFF_OPTIONS $dump $dump2'"
+ error=1
+ fi
+done
+
+[ -z "$error" ] || [ -n "$warnonly" ]
diff --git a/devtools/dpdk.abignore b/devtools/dpdk.abignore
new file mode 100644
index 000000000..f2903612c
--- /dev/null
+++ b/devtools/dpdk.abignore
@@ -0,0 +1,21 @@
+[suppress_function]
+ symbol_version = EXPERIMENTAL
+[suppress_variable]
+ symbol_version = EXPERIMENTAL
+
+; Explicit ignore for driver-only ABI
+[suppress_type]
+ name = rte_cryptodev_ops
+; Ignore this enum update as it is part of an experimental API
+[suppress_type]
+ type_kind = enum
+ name = rte_crypto_asym_xform_type
+ changed_enumerators = RTE_CRYPTO_ASYM_XFORM_TYPE_LIST_END
+
+; FIXME
+[suppress_type]
+ type_kind = enum
+ name = rte_crypto_aead_algorithm
+ changed_enumerators = RTE_CRYPTO_AEAD_LIST_END
+[suppress_variable]
+ name = rte_crypto_aead_algorithm_strings
diff --git a/devtools/gen-abi.sh b/devtools/gen-abi.sh
new file mode 100755
index 000000000..c44b0e228
--- /dev/null
+++ b/devtools/gen-abi.sh
@@ -0,0 +1,26 @@
+#!/bin/sh -e
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright (c) 2019 Red Hat, Inc.
+
+if [ $# != 1 ]; then
+ echo "Usage: $0 installdir"
+ exit 1
+fi
+
+installdir=$1
+if [ ! -d $installdir ]; then
+ echo "Error: install directory '$installdir' does not exist."
+ exit 1
+fi
+
+dumpdir=$installdir/dump
+rm -rf $dumpdir
+mkdir -p $dumpdir
+for f in $(find $installdir -name "*.so.*"); do
+ if test -L $f; then
+ continue
+ fi
+
+ libname=$(basename $f)
+ abidw --out-file $dumpdir/${libname%.so*}.dump $f
+done
diff --git a/devtools/test-build.sh b/devtools/test-build.sh
index 52305fbb8..a97e1280e 100755
--- a/devtools/test-build.sh
+++ b/devtools/test-build.sh
@@ -30,7 +30,8 @@ default_path=$PATH
# - LIBSSO_SNOW3G_PATH
# - LIBSSO_KASUMI_PATH
# - LIBSSO_ZUC_PATH
-. $(dirname $(readlink -f $0))/load-devel-config
+devtools_dir=$(dirname $(readlink -f $0))
+. $devtools_dir/load-devel-config
print_usage () {
echo "usage: $(basename $0) [-h] [-jX] [-s] [config1 [config2] ...]]"
@@ -64,10 +65,12 @@ print_help () {
[ -z $MAKE ] && echo "Cannot find make or gmake" && exit 1
J=$DPDK_MAKE_JOBS
+refsrcdir=$(mktemp -d -t dpdk-${DPDK_ABI_REF_VERSION:-}.XXX)
builds_dir=${DPDK_BUILD_TEST_DIR:-.}
short=false
unset verbose
-maxerr=-Wfatal-errors
+# for ABI checks, we need debuginfo
+test_cflags="-Wfatal-errors -g"
while getopts hj:sv ARG ; do
case $ARG in
j ) J=$OPTARG ;;
@@ -91,13 +94,14 @@ on_exit ()
[ "$DPDK_NOTIFY" != notify-send ] || \
notify-send -u low --icon=dialog-error 'DPDK build' 'failed'
fi
+ rm -rf $refsrcdir
}
# catch manual interrupt to ignore notification
trap "signal=INT ; trap - INT ; kill -INT $$" INT
# notify result on exit
trap on_exit EXIT
-cd $(dirname $(readlink -f $0))/..
+cd $devtools_dir/..
reset_env ()
{
@@ -233,14 +237,14 @@ for conf in $configs ; do
# reload config with DPDK_TARGET set
DPDK_TARGET=$target
reset_env
- . $(dirname $(readlink -f $0))/load-devel-config
+ . $devtools_dir/load-devel-config
options=$(echo $conf | sed 's,[^~+]*,,')
dir=$builds_dir/$conf
config $dir $target $options
echo "================== Build $conf"
- ${MAKE} -j$J EXTRA_CFLAGS="$maxerr $DPDK_DEP_CFLAGS" \
+ ${MAKE} -j$J EXTRA_CFLAGS="$test_cflags $DPDK_DEP_CFLAGS" \
EXTRA_LDFLAGS="$DPDK_DEP_LDFLAGS" $verbose O=$dir
! $short || break
export RTE_TARGET=$target
@@ -253,6 +257,43 @@ for conf in $configs ; do
EXTRA_LDFLAGS="$DPDK_DEP_LDFLAGS" $verbose \
O=$(readlink -f $dir)/examples
unset RTE_TARGET
+ if [ -n "$DPDK_ABI_REF_VERSION" ]; then
+ DPDK_ABI_REF_DIR=${DPDK_ABI_REF_DIR:-reference}
+ abirefdir=$DPDK_ABI_REF_DIR/$DPDK_ABI_REF_VERSION/$conf
+ if [ ! -d $abirefdir ]; then
+ # clone current sources
+ if [ ! -d $refsrcdir/.git ]; then
+ git clone --local --no-hardlinks \
+ --single-branch \
+ -b $DPDK_ABI_REF_VERSION \
+ $(pwd) $refsrcdir
+ fi
+
+ cd $refsrcdir
+
+ rm -rf build
+ config build $target $options
+
+ echo -n "================== Build $conf "
+ echo "($DPDK_ABI_REF_VERSION)"
+ ${MAKE} -j$J \
+ EXTRA_CFLAGS="$test_cflags $DPDK_DEP_CFLAGS" \
+ EXTRA_LDFLAGS="$DPDK_DEP_LDFLAGS" $verbose \
+ O=build
+ ! $short || break
+ export RTE_TARGET=$target
+ ${MAKE} install O=build DESTDIR=$abirefdir \
+ prefix=
+ $devtools_dir/gen-abi.sh $abirefdir
+
+ # back to current workdir
+ cd $devtools_dir/..
+ fi
+
+ echo "================== Check ABI $conf"
+ $devtools_dir/gen-abi.sh $dir/install
+ $devtools_dir/check-abi.sh $abirefdir $dir/install
+ fi
echo "################## $conf done."
unset dir
done
diff --git a/devtools/test-meson-builds.sh b/devtools/test-meson-builds.sh
index 747fb4a1a..23e93770d 100755
--- a/devtools/test-meson-builds.sh
+++ b/devtools/test-meson-builds.sh
@@ -16,8 +16,15 @@ srcdir=$(dirname $(readlink -f $0))/..
MESON=${MESON:-meson}
use_shared="--default-library=shared"
+refsrcdir=$(mktemp -d -t dpdk-${DPDK_ABI_REF_VERSION:-}.XXX)
builds_dir=${DPDK_BUILD_TEST_DIR:-.}
+on_exit ()
+{
+ rm -rf $refsrcdir
+}
+trap on_exit EXIT
+
if command -v gmake >/dev/null 2>&1 ; then
MAKE=gmake
else
@@ -64,9 +71,14 @@ config () # <dir> <builddir> <meson options>
builddir=$1
shift
if [ -f "$builddir/build.ninja" ] ; then
+ # for existing environments, force debugoptimized so that ABI
+ # checks can run
+ $MESON configure --buildtype=debugoptimized $builddir
return
fi
- options="--werror -Dexamples=all -Dlibdir=lib"
+ options=
+ options="$options --werror -Dexamples=all -Dlibdir=lib"
+ options="$options --buildtype=debugoptimized"
for option in $DPDK_MESON_OPTIONS ; do
options="$options -D$option"
done
@@ -107,6 +119,29 @@ build () # <directory> <target compiler> <meson options>
config $srcdir $builds_dir/$targetdir $*
compile $builds_dir/$targetdir \
$(readlink -f $builds_dir/$targetdir/install)
+ if [ -n "$DPDK_ABI_REF_VERSION" ]; then
+ DPDK_ABI_REF_DIR=${DPDK_ABI_REF_DIR:-reference}
+ abirefdir=$DPDK_ABI_REF_DIR/$DPDK_ABI_REF_VERSION/$targetdir
+ if [ ! -d $abirefdir ]; then
+ # clone current sources
+ if [ ! -d $refsrcdir/.git ]; then
+ git clone --local --no-hardlinks \
+ --single-branch \
+ -b $DPDK_ABI_REF_VERSION \
+ $srcdir $refsrcdir
+ fi
+
+ rm -rf $refsrcdir/build
+ config $refsrcdir $refsrcdir/build $*
+ compile $refsrcdir/build $abirefdir
+ $srcdir/devtools/gen-abi.sh $abirefdir
+ fi
+
+ $srcdir/devtools/gen-abi.sh \
+ $(readlink -f $builds_dir/$targetdir/install)
+ $srcdir/devtools/check-abi.sh $abirefdir \
+ $(readlink -f $builds_dir/$targetdir/install)
+ fi
}
if [ "$1" = "-vv" ] ; then
diff --git a/doc/guides/contributing/patches.rst b/doc/guides/contributing/patches.rst
index 0686450e4..59442824a 100644
--- a/doc/guides/contributing/patches.rst
+++ b/doc/guides/contributing/patches.rst
@@ -513,6 +513,21 @@ in a single subfolder called "__builds" created in the current directory.
Setting ``DPDK_BUILD_TEST_DIR`` to an absolute directory path e.g. ``/tmp`` is also supported.
+Checking ABI compatibility
+--------------------------
+
+By default, ABI compatibility checks are disabled.
+
+To enable them, a reference version must be selected via the environment
+variable ``DPDK_ABI_REF_VERSION``.
+
+The ``devtools/test-build.sh`` and ``devtools/test-meson-builds.sh`` scripts
+then build this reference version in a temporary directory and store the
+results in a subfolder of the current working directory.
+The environment variable ``DPDK_ABI_REF_DIR`` can be set so that the results go
+to a different location.
+
+
Sending Patches
---------------
--
2.23.0
^ permalink raw reply [relevance 28%]
* Re: [dpdk-dev] [RFC v2] meter: fix ABI break due to experimental tag removal
2020-01-30 15:55 4% ` Thomas Monjalon
@ 2020-01-30 16:04 4% ` Luca Boccassi
2020-01-30 16:15 4% ` Eelco Chaudron
0 siblings, 1 reply; 200+ results
From: Luca Boccassi @ 2020-01-30 16:04 UTC (permalink / raw)
To: Thomas Monjalon
Cc: Ferruh Yigit, Neil Horman, Cristian Dumitrescu, Eelco Chaudron,
dev, David Marchand, Bruce Richardson, Ian Stokes,
Andrzej Ostruszka
On Thu, 2020-01-30 at 16:55 +0100, Thomas Monjalon wrote:
> 30/01/2020 15:21, Luca Boccassi:
> > On Thu, 2020-01-30 at 15:17 +0100, Thomas Monjalon wrote:
> > > 30/01/2020 13:57, Luca Boccassi:
> > > > On Thu, 2020-01-30 at 13:33 +0100, Thomas Monjalon wrote:
> > > > > Hi,
> > > > >
> > > > > I disagree with the need of this patch.
> > > > > The symbol was experimental, meaning we can change it.
> > > > > Removing experimental tag is not an ABI break.
> > > >
> > > > Hi,
> > > >
> > > > This symbol change was requested for backport in 19.11.x, and
> > > > experimental or not I'm not too keen on backward incompatible
> > > > changes
> > > > to the public interface in an _LTS point release_. The
> > > > compromise
> > > > was
> > > > to see if we could support both symbols version, which makes
> > > > the
> > > > change
> > > > backward compatible.
> > > >
> > > > If you prefer not to have this patch in mainline I'm also fine
> > > > in
> > > > taking it just for the LTS. I agree with you that it is not
> > > > required
> > > > for mainline releases (although nicer for me if it's a backport
> > > > rather
> > > > than a new change).
> > >
> > > I would like to avoid opening the door for maintaining the
> > > experimental ABI
> > > in the mainline. Please take it directly in the LTS.
> > >
> > > The next question is to know whether we really want to have such
> > > patch in LTS.
> > > Anyway, 19.11.0 has this symbol as experimental.
> > > How adding a non-experimental version of the function in 19.11.1
> > > will
> > > change
> > > the ABI status of the whole 19.11 branch?
> >
> > The problem is not adding the new symbol, but removing the
> > experimental
> > one. Changing the version of the symbol was requested by OVS for
> > inclusion in 19.11.
>
> Yes, sorry, this is what I meant.
> Given 19.11.0 already has the symbol as experimental,
> and that applications like OVS had to accept it as experimental,
> why removing experimental tag in 19.11.1?
I think it was mentioned that it was preferred not to suppress the
compiler warning to avoid any accidental use in the future, but the OVS
maintainer(s) should answer as I might remember wrongly.
--
Kind regards,
Luca Boccassi
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
2020-01-29 20:13 4% ` Akhil Goyal
@ 2020-01-30 16:09 8% ` Ferruh Yigit
2020-01-30 20:18 4% ` Thomas Monjalon
0 siblings, 1 reply; 200+ results
From: Ferruh Yigit @ 2020-01-30 16:09 UTC (permalink / raw)
To: Akhil Goyal, David Marchand, Anoob Joseph, Trahe, Fiona, Kusztal,
ArkadiuszX
Cc: Thomas Monjalon, dev, bruce.richardson, nhorman, John McNamara,
dodji, Andrew Rybchenko
On 1/29/2020 8:13 PM, Akhil Goyal wrote:
>
>
>>
>> On Wed, Jan 29, 2020 at 7:10 PM Anoob Joseph <anoobj@marvell.com> wrote:
>>> The asymmetric crypto library is experimental. Changes to experimental code
>> paths is allowed, right?
>>
>> The asymmetric crypto enum is referenced by a function part of the stable ABI.
>> It is possible to waive this enum, if we are sure no use out of the
>> experimental asym crypto APIs is possible.
>>
>> The rest of the changes touch stable symbols.
>>
>> Adding the abidiff report:
>>
>> [C]'function void rte_cryptodev_info_get(uint8_t,
>> rte_cryptodev_info*)' at rte_cryptodev.c:1115:1 has some indirect
>> sub-type changes:
>> parameter 2 of type 'rte_cryptodev_info*' has sub-type changes:
>> in pointed to type 'struct rte_cryptodev_info' at rte_cryptodev.h:468:1:
>> type size hasn't changed
>> 1 data member change:
>> type of 'const rte_cryptodev_capabilities*
>> rte_cryptodev_info::capabilities' changed:
>> in pointed to type 'const rte_cryptodev_capabilities':
>> in unqualified underlying type 'struct
>> rte_cryptodev_capabilities' at rte_cryptodev.h:176:1:
>> type size hasn't changed
>> 1 data member change:
>> type of '__anonymous_union__ ' changed:
>> type size hasn't changed
>> 1 data member change:
>> type of 'rte_cryptodev_asymmetric_capability
>> __anonymous_union__::asym' changed:
>> type size hasn't changed
>> 1 data member change:
>> type of
>> 'rte_cryptodev_asymmetric_xform_capability
>> rte_cryptodev_asymmetric_capability::xform_capa' changed:
>> type size hasn't changed
>> 1 data member change:
>> type of 'rte_crypto_asym_xform_type
>> rte_cryptodev_asymmetric_xform_capability::xform_type' changed:
>> type size hasn't changed
>> 2 enumerator insertions:
>>
>> 'rte_crypto_asym_xform_type::RTE_CRYPTO_ASYM_XFORM_ECDSA' value '7'
>>
>> 'rte_crypto_asym_xform_type::RTE_CRYPTO_ASYM_XFORM_ECPM' value '8'
>> 1 enumerator change:
>>
>> 'rte_crypto_asym_xform_type::RTE_CRYPTO_ASYM_XFORM_TYPE_LIST_END'
>> from
>> value '7' to '9' at rte_crypto_asym.h:60:1
>>
>
> I believe these enums will be used only in case of ASYM case which is experimental.
Independent from being experiment and not, this shouldn't be a problem, I think
this is a false positive.
The ABI break can happen when a struct has been shared between the application
and the library (DPDK) and the layout of that memory know differently by
application and the library.
Here in all cases, there is no layout/size change.
As to the value changes of the enums, since application compiled with old DPDK,
it will know only up to '6', 7 and more means invalid to the application. So it
won't send these values also it should ignore these values from library. Only
consequence is old application won't able to use new features those new enums
provide but that is expected/normal.
>
>>
>> [C]'function int
>> rte_cryptodev_get_aead_algo_enum(rte_crypto_aead_algorithm*, const
>> char*)' at rte_cryptodev.c:239:1 has some indirect sub-type changes:
>> parameter 1 of type 'rte_crypto_aead_algorithm*' has sub-type changes:
>> in pointed to type 'enum rte_crypto_aead_algorithm' at
>> rte_crypto_sym.h:346:1:
>> type size hasn't changed
>> 1 enumerator insertion:
>> 'rte_crypto_aead_algorithm::RTE_CRYPTO_AEAD_CHACHA20_POLY1305'
>> value '3'
>> 1 enumerator change:
>> 'rte_crypto_aead_algorithm::RTE_CRYPTO_AEAD_LIST_END' from
>> value '3' to '4' at rte_crypto_sym.h:346:1
Same as above, no layout change.
>>
>>
>> [C]'const char* rte_crypto_aead_algorithm_strings[1]' was changed at
>> rte_crypto_sym.h:358:1:
>> size of symbol (in bytes) changed from 24 to 32
>>
The shared memory size changes, but this is global variable in the library, and
the values application can request 'RTE_CRYPTO_AEAD_AES_CCM' &
'RTE_CRYPTO_AEAD_AES_GCM' is already there, so there is no backward
compatibility issue here.
>>
> +Fiona and Arek
>
> We may need to revert the chacha-poly patches.
>
I don't see any ABI break in this case, can someone explain if I am missing
anything here?
^ permalink raw reply [relevance 8%]
* Re: [dpdk-dev] [PATCH v2] build: remove unused function versioning
@ 2020-01-30 16:11 3% ` David Marchand
2020-02-05 20:44 0% ` David Marchand
0 siblings, 1 reply; 200+ results
From: David Marchand @ 2020-01-30 16:11 UTC (permalink / raw)
To: Andrzej Ostruszka
Cc: dev, David Hunt, Bruce Richardson, Vladimir Medvedkin,
Robert Sanford, Erik Gabriel Carrillo, Marcin Baran, dpdk stable
On Thu, Jan 30, 2020 at 10:25 AM Andrzej Ostruszka
<aostruszka@marvell.com> wrote:
>
> Timer, LPM and Distributor libraries no longer use function versioning
> and therefore do not need separate build for static and shared version
> of libraries.
>
> This patch removes use_function_versioning from their meson build files
> and corresponding include from the sources.
>
> Signed-off-by: Andrzej Ostruszka <aostruszka@marvell.com>
> Acked-by: Bruce Richardson <bruce.richardson@intel.com>
>
> Fixes: f2fb215843a9 ("timer: remove deprecated code")
> Fixes: 6e5b51676176 ("distributor: remove deprecated code")
> Fixes: c381a8d554b7 ("lpm: remove deprecated code")
> Cc: marcinx.baran@intel.com
>
> ---
> V2:
> - removed also includes (updating only meson files causes build
> failures), and updated commit msg accordingly
Thanks, I had the exact same patch in store (caught when working with
Olivier on some ABI stuff for mempool).
Cc: stable@dpdk.org
Acked-by: David Marchand <david.marchand@redhat.com>
--
David Marchand
^ permalink raw reply [relevance 3%]
* Re: [dpdk-dev] [RFC v2] meter: fix ABI break due to experimental tag removal
2020-01-30 16:04 4% ` Luca Boccassi
@ 2020-01-30 16:15 4% ` Eelco Chaudron
2020-01-30 20:20 4% ` Thomas Monjalon
0 siblings, 1 reply; 200+ results
From: Eelco Chaudron @ 2020-01-30 16:15 UTC (permalink / raw)
To: Luca Boccassi
Cc: Thomas Monjalon, Ferruh Yigit, Neil Horman, Cristian Dumitrescu,
dev, David Marchand, Bruce Richardson, Ian Stokes,
Andrzej Ostruszka
On 30 Jan 2020, at 17:04, Luca Boccassi wrote:
> On Thu, 2020-01-30 at 16:55 +0100, Thomas Monjalon wrote:
>> 30/01/2020 15:21, Luca Boccassi:
>>> On Thu, 2020-01-30 at 15:17 +0100, Thomas Monjalon wrote:
>>>> 30/01/2020 13:57, Luca Boccassi:
>>>>> On Thu, 2020-01-30 at 13:33 +0100, Thomas Monjalon wrote:
>>>>>> Hi,
>>>>>>
>>>>>> I disagree with the need of this patch.
>>>>>> The symbol was experimental, meaning we can change it.
>>>>>> Removing experimental tag is not an ABI break.
>>>>>
>>>>> Hi,
>>>>>
>>>>> This symbol change was requested for backport in 19.11.x, and
>>>>> experimental or not I'm not too keen on backward incompatible
>>>>> changes
>>>>> to the public interface in an _LTS point release_. The
>>>>> compromise
>>>>> was
>>>>> to see if we could support both symbols version, which makes
>>>>> the
>>>>> change
>>>>> backward compatible.
>>>>>
>>>>> If you prefer not to have this patch in mainline I'm also fine
>>>>> in
>>>>> taking it just for the LTS. I agree with you that it is not
>>>>> required
>>>>> for mainline releases (although nicer for me if it's a backport
>>>>> rather
>>>>> than a new change).
>>>>
>>>> I would like to avoid opening the door for maintaining the
>>>> experimental ABI
>>>> in the mainline. Please take it directly in the LTS.
>>>>
>>>> The next question is to know whether we really want to have such
>>>> patch in LTS.
>>>> Anyway, 19.11.0 has this symbol as experimental.
>>>> How adding a non-experimental version of the function in 19.11.1
>>>> will
>>>> change
>>>> the ABI status of the whole 19.11 branch?
>>>
>>> The problem is not adding the new symbol, but removing the
>>> experimental
>>> one. Changing the version of the symbol was requested by OVS for
>>> inclusion in 19.11.
>>
>> Yes, sorry, this is what I meant.
>> Given 19.11.0 already has the symbol as experimental,
>> and that applications like OVS had to accept it as experimental,
>> why removing experimental tag in 19.11.1?
>
> I think it was mentioned that it was preferred not to suppress the
> compiler warning to avoid any accidental use in the future, but the
> OVS
> maintainer(s) should answer as I might remember wrongly.
Yes this is the reason, OVS compiles with -Werror so we would like to
avoid the warnings. You can not disable them per include, it’s global
for all of DPDK.
//Eelco
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [dpdk-stable] [PATCH] meter: move RFC4115 trTCM APIs as none experimental
2020-01-16 12:42 0% ` Ferruh Yigit
2020-01-16 14:38 0% ` Thomas Monjalon
2020-01-22 13:55 0% ` Ray Kinsella
@ 2020-01-30 16:33 4% ` Ray Kinsella
2 siblings, 0 replies; 200+ results
From: Ray Kinsella @ 2020-01-30 16:33 UTC (permalink / raw)
To: Ferruh Yigit, Neil Horman, David Marchand
Cc: Thomas Monjalon, Cristian Dumitrescu, dpdk stable, dev,
Eelco Chaudron, Kevin Traynor, Ian Stokes, Ilya Maximets,
Luca Boccassi
On 16/01/2020 12:42, Ferruh Yigit wrote:
> On 1/16/2020 11:54 AM, Neil Horman wrote:
>> On Thu, Jan 16, 2020 at 12:25:06PM +0100, David Marchand wrote:
>>> On Tue, Dec 17, 2019 at 2:08 PM Eelco Chaudron <echaudro@redhat.com> wrote:
>>>>
>>>> Moved RFC4115 APIs to none experimental as they have been there
>>>> since 19.02. Also, these APIs are the same as the none RFC4115 APIs.
>>>>
>>>> Signed-off-by: Eelco Chaudron <echaudro@redhat.com>
>>>
>>> There is a discussion on the OVS ml at the moment to get these symbols
>>> in the stable ABI for 19.11.
>>> I want to understand how this would be done.
>>>
>>> - I take this patch in 20.02, these symbols are added in the 20.0.1 ABI.
>>> On the other hand, the 19.11 release maintains the 20.0 ABI.
>>>
>>> Does it mean the backport adds these symbols with the 20.0 version in
>>> the 19.11 branch?
>>> Or is 20.0.1 version acceptable / a thing we want?
>>>
>>>
>>> - These symbol already existed in the 20.0 ABI, versioned as EXPERIMENTAL.
>>> We can go and remove these entries since we are not bound to preserve
>>> the experimental APIs.
>>> But, on the other hand, nothing should prevent us from keeping some
>>> aliases so that the symbols versioned EXPERIMENTAL are still available
>>> to existing users.
>>>
>> I would say that choice is up to you. If you want to alias them to be nice to
>> prior users, thats fine by me. But experimental means experimental, and so users
>> have to be prepared to rebuild when things change, even if that change is
>> changing the version from experimental to a concrete version.
>>
>
> I would prefer to keep the alias and don't break the existing users, specially
> for the case experimental API is becoming mature without change.
>
Apologies for dragging up an older thread.
I had a thought later that we should add something formal to the ABI Policy.
Such that if we have an API that has been part of DPDK as experimental for a long time,
to the point it has a number of users - out of kindness, the maintainer may ask a
contributor to add an alias until the next major ABI version is declared?
So as not to break consuming projects - make sense?
Thanks,
Ray K
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
2020-01-30 15:59 4% ` Thomas Monjalon
@ 2020-01-30 16:42 7% ` Ferruh Yigit
2020-01-30 23:49 4% ` Ananyev, Konstantin
1 sibling, 0 replies; 200+ results
From: Ferruh Yigit @ 2020-01-30 16:42 UTC (permalink / raw)
To: Thomas Monjalon, Anoob Joseph, akhil.goyal, Trahe, Fiona
Cc: dev, David Marchand, Richardson, Bruce, nhorman, Mcnamara, John,
Kusztal, ArkadiuszX
On 1/30/2020 3:59 PM, Thomas Monjalon wrote:
> 30/01/2020 14:06, Trahe, Fiona:
>> We were unaware the LIST_END change could constitute an ABI breakage, but can see how it affects the array size when picked up.
>> We're exploring options.
>>
>> I agree with Anoob's point that if we don't allow the LIST_END to be modified, then it means no feature can be implemented without ABI breakage.
>> Anyone object to removing those LIST_END elements - or have a better suggestion? Would have to be in 20.11 I suppose.
>
> Yes, having max value right after the last value is ridiculous,
> it prevents adding any value.
> In 20.11, we should remove all these *_END and *_MAX from API enums
> and replace them with a separate #define with reasonnable maximums.
>
>
I disagree, that kind of usage is common and lets loops iterate on the valid
elements, and it is not a source of ABI break on its own.
Indeed other way around, not having MAX, is problematic, if we don't have the
MAX value to compare and decide if a provided value is valid or not, and when
new version of the library introduces the new values, how old application can
detect unsupported new values?
As far as I can see the problem occurs when that *_END and *_MAX used to define
the size of array in the public struct. This usage prevents adding new values
and I already send a deprecation notice for it:
https://patches.dpdk.org/patch/65359/
^ permalink raw reply [relevance 7%]
* [dpdk-dev] [PATCH] doc: deprecation notice to move igb_uio to a new repo
@ 2020-01-30 16:58 4% jerinj
2020-01-30 17:30 0% ` Ferruh Yigit
0 siblings, 1 reply; 200+ results
From: jerinj @ 2020-01-30 16:58 UTC (permalink / raw)
To: dev, Neil Horman, John McNamara, Marko Kovacevic
Cc: thomas, techboard, ferruh.yigit, Jerin Jacob
From: Jerin Jacob <jerinj@marvell.com>
Based on the tech board meeting held on 2019-11-06,
It's been decided to move igb_uio kernel module to a new repository
hosted by dpdk.org in v20.11 release.
http://mails.dpdk.org/archives/dev/2019-November/151763.html
Adding the deprecation notice for the same in advance.
Signed-off-by: Jerin Jacob <jerinj@marvell.com>
---
doc/guides/rel_notes/deprecation.rst | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
index dfcca87ab..2b9d0113f 100644
--- a/doc/guides/rel_notes/deprecation.rst
+++ b/doc/guides/rel_notes/deprecation.rst
@@ -38,6 +38,12 @@ Deprecation Notices
remove it from the externally visible ABI and allow it to be updated in the
future.
+* igb_uio: In the view of reducing the kernel dependency from the main tree,
+ as a first step, The technical board has been decided to move ``igb_uio``
+ kernel module to a new repository hosted by dpdk.org in v20.11 release.
+ Minutes of Technical Board Meeting of `2019-11-06
+ <http://mails.dpdk.org/archives/dev/2019-November/151763.html>`_.
+
* dpaa2: removal of ``rte_dpaa2_memsegs`` structure which has been replaced
by a pa-va search library. This structure was earlier being used for holding
memory segments used by dpaa2 driver for faster pa->va translation. This
--
2.24.1
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH] doc: deprecation notice to move igb_uio to a new repo
2020-01-30 16:58 4% [dpdk-dev] [PATCH] doc: deprecation notice to move igb_uio to a new repo jerinj
@ 2020-01-30 17:30 0% ` Ferruh Yigit
0 siblings, 0 replies; 200+ results
From: Ferruh Yigit @ 2020-01-30 17:30 UTC (permalink / raw)
To: jerinj, dev, Neil Horman, John McNamara, Marko Kovacevic
Cc: thomas, techboard
On 1/30/2020 4:58 PM, jerinj@marvell.com wrote:
> From: Jerin Jacob <jerinj@marvell.com>
>
> Based on the tech board meeting held on 2019-11-06,
> It's been decided to move igb_uio kernel module to a new repository
> hosted by dpdk.org in v20.11 release.
>
> http://mails.dpdk.org/archives/dev/2019-November/151763.html
>
> Adding the deprecation notice for the same in advance.
>
> Signed-off-by: Jerin Jacob <jerinj@marvell.com>
> ---
> doc/guides/rel_notes/deprecation.rst | 6 ++++++
> 1 file changed, 6 insertions(+)
>
> diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
> index dfcca87ab..2b9d0113f 100644
> --- a/doc/guides/rel_notes/deprecation.rst
> +++ b/doc/guides/rel_notes/deprecation.rst
> @@ -38,6 +38,12 @@ Deprecation Notices
> remove it from the externally visible ABI and allow it to be updated in the
> future.
>
> +* igb_uio: In the view of reducing the kernel dependency from the main tree,
> + as a first step, The technical board has been decided to move ``igb_uio``
> + kernel module to a new repository hosted by dpdk.org in v20.11 release.
> + Minutes of Technical Board Meeting of `2019-11-06
> + <http://mails.dpdk.org/archives/dev/2019-November/151763.html>`_.
> +
> * dpaa2: removal of ``rte_dpaa2_memsegs`` structure which has been replaced
> by a pa-va search library. This structure was earlier being used for holding
> memory segments used by dpaa2 driver for faster pa->va translation. This
>
Acked-by: Ferruh Yigit <ferruh.yigit@intel.com>
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
2020-01-30 16:09 8% ` Ferruh Yigit
@ 2020-01-30 20:18 4% ` Thomas Monjalon
2020-01-31 9:03 7% ` Ferruh Yigit
2020-01-31 14:16 4% ` Trahe, Fiona
0 siblings, 2 replies; 200+ results
From: Thomas Monjalon @ 2020-01-30 20:18 UTC (permalink / raw)
To: Akhil Goyal, David Marchand, Anoob Joseph, Trahe, Fiona, Kusztal,
ArkadiuszX, Ferruh Yigit
Cc: dev, bruce.richardson, nhorman, John McNamara, dodji, Andrew Rybchenko
30/01/2020 17:09, Ferruh Yigit:
> On 1/29/2020 8:13 PM, Akhil Goyal wrote:
> >
> >
> >>
> >> On Wed, Jan 29, 2020 at 7:10 PM Anoob Joseph <anoobj@marvell.com> wrote:
> >>> The asymmetric crypto library is experimental. Changes to experimental code
> >> paths is allowed, right?
> >>
> >> The asymmetric crypto enum is referenced by a function part of the stable ABI.
> >> It is possible to waive this enum, if we are sure no use out of the
> >> experimental asym crypto APIs is possible.
> >>
> >> The rest of the changes touch stable symbols.
> >>
> >> Adding the abidiff report:
> >>
> >> [C]'function void rte_cryptodev_info_get(uint8_t,
> >> rte_cryptodev_info*)' at rte_cryptodev.c:1115:1 has some indirect
> >> sub-type changes:
> >> parameter 2 of type 'rte_cryptodev_info*' has sub-type changes:
> >> in pointed to type 'struct rte_cryptodev_info' at rte_cryptodev.h:468:1:
> >> type size hasn't changed
> >> 1 data member change:
> >> type of 'const rte_cryptodev_capabilities*
> >> rte_cryptodev_info::capabilities' changed:
> >> in pointed to type 'const rte_cryptodev_capabilities':
> >> in unqualified underlying type 'struct
> >> rte_cryptodev_capabilities' at rte_cryptodev.h:176:1:
> >> type size hasn't changed
> >> 1 data member change:
> >> type of '__anonymous_union__ ' changed:
> >> type size hasn't changed
> >> 1 data member change:
> >> type of 'rte_cryptodev_asymmetric_capability
> >> __anonymous_union__::asym' changed:
> >> type size hasn't changed
> >> 1 data member change:
> >> type of
> >> 'rte_cryptodev_asymmetric_xform_capability
> >> rte_cryptodev_asymmetric_capability::xform_capa' changed:
> >> type size hasn't changed
> >> 1 data member change:
> >> type of 'rte_crypto_asym_xform_type
> >> rte_cryptodev_asymmetric_xform_capability::xform_type' changed:
> >> type size hasn't changed
> >> 2 enumerator insertions:
> >>
> >> 'rte_crypto_asym_xform_type::RTE_CRYPTO_ASYM_XFORM_ECDSA' value '7'
> >>
> >> 'rte_crypto_asym_xform_type::RTE_CRYPTO_ASYM_XFORM_ECPM' value '8'
> >> 1 enumerator change:
> >>
> >> 'rte_crypto_asym_xform_type::RTE_CRYPTO_ASYM_XFORM_TYPE_LIST_END'
> >> from
> >> value '7' to '9' at rte_crypto_asym.h:60:1
> >>
> >
> > I believe these enums will be used only in case of ASYM case which is experimental.
>
> Independent from being experiment and not, this shouldn't be a problem, I think
> this is a false positive.
>
> The ABI break can happen when a struct has been shared between the application
> and the library (DPDK) and the layout of that memory know differently by
> application and the library.
>
> Here in all cases, there is no layout/size change.
>
> As to the value changes of the enums, since application compiled with old DPDK,
> it will know only up to '6', 7 and more means invalid to the application. So it
> won't send these values also it should ignore these values from library. Only
> consequence is old application won't able to use new features those new enums
> provide but that is expected/normal.
If library give higher value than expected by the application,
if the application uses this value as array index,
there can be an access out of bounds.
> >> [C]'function int
> >> rte_cryptodev_get_aead_algo_enum(rte_crypto_aead_algorithm*, const
> >> char*)' at rte_cryptodev.c:239:1 has some indirect sub-type changes:
> >> parameter 1 of type 'rte_crypto_aead_algorithm*' has sub-type changes:
> >> in pointed to type 'enum rte_crypto_aead_algorithm' at
> >> rte_crypto_sym.h:346:1:
> >> type size hasn't changed
> >> 1 enumerator insertion:
> >> 'rte_crypto_aead_algorithm::RTE_CRYPTO_AEAD_CHACHA20_POLY1305'
> >> value '3'
> >> 1 enumerator change:
> >> 'rte_crypto_aead_algorithm::RTE_CRYPTO_AEAD_LIST_END' from
> >> value '3' to '4' at rte_crypto_sym.h:346:1
>
> Same as above, no layout change.
>
> >>
> >>
> >> [C]'const char* rte_crypto_aead_algorithm_strings[1]' was changed at
> >> rte_crypto_sym.h:358:1:
> >> size of symbol (in bytes) changed from 24 to 32
> >>
>
> The shared memory size changes, but this is global variable in the library, and
> the values application can request 'RTE_CRYPTO_AEAD_AES_CCM' &
> 'RTE_CRYPTO_AEAD_AES_GCM' is already there, so there is no backward
> compatibility issue here.
For this one, I don't know what is the breakage.
> > +Fiona and Arek
> >
> > We may need to revert the chacha-poly patches.
> >
>
> I don't see any ABI break in this case, can someone explain if I am missing
> anything here?
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [RFC v2] meter: fix ABI break due to experimental tag removal
2020-01-30 16:15 4% ` Eelco Chaudron
@ 2020-01-30 20:20 4% ` Thomas Monjalon
2020-02-03 11:10 4% ` Eelco Chaudron
0 siblings, 1 reply; 200+ results
From: Thomas Monjalon @ 2020-01-30 20:20 UTC (permalink / raw)
To: Luca Boccassi, Eelco Chaudron
Cc: Ferruh Yigit, Neil Horman, Cristian Dumitrescu, dev,
David Marchand, Bruce Richardson, Ian Stokes, Andrzej Ostruszka
30/01/2020 17:15, Eelco Chaudron:
> On 30 Jan 2020, at 17:04, Luca Boccassi wrote:
> > On Thu, 2020-01-30 at 16:55 +0100, Thomas Monjalon wrote:
> >> 30/01/2020 15:21, Luca Boccassi:
> >>> On Thu, 2020-01-30 at 15:17 +0100, Thomas Monjalon wrote:
> >>>> 30/01/2020 13:57, Luca Boccassi:
> >>>>> On Thu, 2020-01-30 at 13:33 +0100, Thomas Monjalon wrote:
> >>>>>> Hi,
> >>>>>>
> >>>>>> I disagree with the need of this patch.
> >>>>>> The symbol was experimental, meaning we can change it.
> >>>>>> Removing experimental tag is not an ABI break.
> >>>>>
> >>>>> Hi,
> >>>>>
> >>>>> This symbol change was requested for backport in 19.11.x, and
> >>>>> experimental or not I'm not too keen on backward incompatible
> >>>>> changes
> >>>>> to the public interface in an _LTS point release_. The
> >>>>> compromise
> >>>>> was
> >>>>> to see if we could support both symbols version, which makes
> >>>>> the
> >>>>> change
> >>>>> backward compatible.
> >>>>>
> >>>>> If you prefer not to have this patch in mainline I'm also fine
> >>>>> in
> >>>>> taking it just for the LTS. I agree with you that it is not
> >>>>> required
> >>>>> for mainline releases (although nicer for me if it's a backport
> >>>>> rather
> >>>>> than a new change).
> >>>>
> >>>> I would like to avoid opening the door for maintaining the
> >>>> experimental ABI
> >>>> in the mainline. Please take it directly in the LTS.
> >>>>
> >>>> The next question is to know whether we really want to have such
> >>>> patch in LTS.
> >>>> Anyway, 19.11.0 has this symbol as experimental.
> >>>> How adding a non-experimental version of the function in 19.11.1
> >>>> will
> >>>> change
> >>>> the ABI status of the whole 19.11 branch?
> >>>
> >>> The problem is not adding the new symbol, but removing the
> >>> experimental
> >>> one. Changing the version of the symbol was requested by OVS for
> >>> inclusion in 19.11.
> >>
> >> Yes, sorry, this is what I meant.
> >> Given 19.11.0 already has the symbol as experimental,
> >> and that applications like OVS had to accept it as experimental,
> >> why removing experimental tag in 19.11.1?
> >
> > I think it was mentioned that it was preferred not to suppress the
> > compiler warning to avoid any accidental use in the future, but the
> > OVS
> > maintainer(s) should answer as I might remember wrongly.
>
> Yes this is the reason, OVS compiles with -Werror so we would like to
> avoid the warnings. You can not disable them per include, it’s global
> for all of DPDK.
Yes but anyway OVS must accept the experimental function as the next release
will use it with DPDK 19.11.0.
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH v3 3/4] build: test meson installation
2020-01-30 16:00 3% ` [dpdk-dev] [PATCH v3 3/4] build: test meson installation David Marchand
@ 2020-01-30 22:17 3% ` Thomas Monjalon
0 siblings, 0 replies; 200+ results
From: Thomas Monjalon @ 2020-01-30 22:17 UTC (permalink / raw)
To: David Marchand
Cc: dev, bruce.richardson, kevin.laatz, aconole, nhorman,
akhil.goyal, anoobj, bluca, fiona.trahe, ferruh.yigit,
Michael Santana
30/01/2020 17:00, David Marchand:
> Let's test installing with meson as part of Travis and the
> devtools/test-meson-builds.sh script.
For test-meson-builds.sh, I would prefer avoiding install if not needed.
Can we install only if compiling examples or checking ABI?
> -compile () # <builddir>
> +compile () # <builddir> <installdir>
> {
> builddir=$1
> if [ -n "$TEST_MESON_BUILD_VERY_VERBOSE" ] ; then
> @@ -90,6 +90,9 @@ compile () # <builddir>
> echo "$ninja_cmd -C $builddir"
> $ninja_cmd -C $builddir
> fi
> + rm -rf $2
> + echo "DESTDIR=$2 $ninja_cmd -C $builddir install"
> + DESTDIR=$2 $ninja_cmd -C $builddir install
> }
>
> build () # <directory> <target compiler> <meson options>
> @@ -102,7 +105,8 @@ build () # <directory> <target compiler> <meson options>
> command -v ${CC##* } >/dev/null 2>&1 || return 0
> load_env $targetcc || return 0
> config $srcdir $builds_dir/$targetdir $*
> - compile $builds_dir/$targetdir
> + compile $builds_dir/$targetdir \
> + $(readlink -f $builds_dir/$targetdir/install)
> }
^ permalink raw reply [relevance 3%]
* Re: [dpdk-dev] [PATCH v3 4/4] add ABI checks
2020-01-30 16:00 28% ` [dpdk-dev] [PATCH v3 4/4] add ABI checks David Marchand
@ 2020-01-30 22:32 4% ` Thomas Monjalon
2020-02-01 15:29 4% ` David Marchand
2020-01-30 22:44 4% ` Thomas Monjalon
1 sibling, 1 reply; 200+ results
From: Thomas Monjalon @ 2020-01-30 22:32 UTC (permalink / raw)
To: David Marchand
Cc: dev, bruce.richardson, kevin.laatz, aconole, nhorman,
akhil.goyal, anoobj, bluca, fiona.trahe, ferruh.yigit,
Michael Santana, John McNamara, Marko Kovacevic
30/01/2020 17:00, David Marchand:
> Enabling them requires a configuration that will trigger the ABI dumps
> generation as part of the existing devtools/test-build.sh and
> devtools/test-meson-builds.sh scripts.
[...]
> --- a/devtools/test-meson-builds.sh
> +++ b/devtools/test-meson-builds.sh
> if [ -f "$builddir/build.ninja" ] ; then
> + # for existing environments, force debugoptimized so that ABI
> + # checks can run
> + $MESON configure --buildtype=debugoptimized $builddir
This is forcing meson to re-run each time, even if the buildtype is
already "debugoptimized".
Please query meson configuration to avoid useless re-run:
$MESON configure $builddir | awk '$1=="buildtype" {print $2}'
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH v3 4/4] add ABI checks
2020-01-30 16:00 28% ` [dpdk-dev] [PATCH v3 4/4] add ABI checks David Marchand
2020-01-30 22:32 4% ` Thomas Monjalon
@ 2020-01-30 22:44 4% ` Thomas Monjalon
1 sibling, 0 replies; 200+ results
From: Thomas Monjalon @ 2020-01-30 22:44 UTC (permalink / raw)
To: David Marchand
Cc: dev, bruce.richardson, kevin.laatz, aconole, nhorman,
akhil.goyal, anoobj, bluca, fiona.trahe, ferruh.yigit,
Michael Santana, John McNamara, Marko Kovacevic
30/01/2020 17:00, David Marchand:
> --- a/devtools/test-meson-builds.sh
> +++ b/devtools/test-meson-builds.sh
> +refsrcdir=$(mktemp -d -t dpdk-${DPDK_ABI_REF_VERSION:-}.XXX)
Instead of a temporary source directory,
could it be inside $DPDK_ABI_REF_DIR/$DPDK_ABI_REF_VERSION ?
I feel it would more "hackable" for debugging of the process.
[..]
> + if [ -n "$DPDK_ABI_REF_VERSION" ]; then
> + DPDK_ABI_REF_DIR=${DPDK_ABI_REF_DIR:-reference}
> + abirefdir=$DPDK_ABI_REF_DIR/$DPDK_ABI_REF_VERSION/$targetdir
> + if [ ! -d $abirefdir ]; then
> + # clone current sources
> + if [ ! -d $refsrcdir/.git ]; then
> + git clone --local --no-hardlinks \
> + --single-branch \
> + -b $DPDK_ABI_REF_VERSION \
> + $srcdir $refsrcdir
> + fi
> +
> + rm -rf $refsrcdir/build
> + config $refsrcdir $refsrcdir/build $*
> + compile $refsrcdir/build $abirefdir
> + $srcdir/devtools/gen-abi.sh $abirefdir
> + fi
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
2020-01-30 15:59 4% ` Thomas Monjalon
2020-01-30 16:42 7% ` Ferruh Yigit
@ 2020-01-30 23:49 4% ` Ananyev, Konstantin
2020-01-31 9:07 4% ` Ferruh Yigit
1 sibling, 1 reply; 200+ results
From: Ananyev, Konstantin @ 2020-01-30 23:49 UTC (permalink / raw)
To: Thomas Monjalon, Anoob Joseph, akhil.goyal, Trahe, Fiona
Cc: dev, David Marchand, Richardson, Bruce, nhorman, Mcnamara, John,
Trahe, Fiona, Kusztal, ArkadiuszX, Yigit, Ferruh
> -----Original Message-----
> From: dev <dev-bounces@dpdk.org> On Behalf Of Thomas Monjalon
> Sent: Thursday, January 30, 2020 4:00 PM
> To: Anoob Joseph <anoobj@marvell.com>; akhil.goyal@nxp.com; Trahe, Fiona <fiona.trahe@intel.com>
> Cc: dev@dpdk.org; David Marchand <david.marchand@redhat.com>; Richardson, Bruce <bruce.richardson@intel.com>;
> nhorman@tuxdriver.com; Mcnamara, John <john.mcnamara@intel.com>; Trahe, Fiona <fiona.trahe@intel.com>; Kusztal, ArkadiuszX
> <arkadiuszx.kusztal@intel.com>; Yigit, Ferruh <ferruh.yigit@intel.com>
> Subject: Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
>
> 30/01/2020 14:06, Trahe, Fiona:
> > We were unaware the LIST_END change could constitute an ABI breakage, but can see how it affects the array size when picked up.
> > We're exploring options.
> >
> > I agree with Anoob's point that if we don't allow the LIST_END to be modified, then it means no feature can be implemented without ABI
> breakage.
> > Anyone object to removing those LIST_END elements - or have a better suggestion? Would have to be in 20.11 I suppose.
>
> Yes, having max value right after the last value is ridiculous,
> it prevents adding any value.
> In 20.11, we should remove all these *_END and *_MAX from API enums
> and replace them with a separate #define with reasonnable maximums.
>
I think we'd better avoid public structs that have array of _MAX elems in them.
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
2020-01-30 20:18 4% ` Thomas Monjalon
@ 2020-01-31 9:03 7% ` Ferruh Yigit
2020-01-31 10:26 7% ` Ananyev, Konstantin
2020-01-31 14:16 4% ` Trahe, Fiona
1 sibling, 1 reply; 200+ results
From: Ferruh Yigit @ 2020-01-31 9:03 UTC (permalink / raw)
To: Thomas Monjalon, Akhil Goyal, David Marchand, Anoob Joseph,
Trahe, Fiona, Kusztal, ArkadiuszX
Cc: dev, bruce.richardson, nhorman, John McNamara, dodji, Andrew Rybchenko
On 1/30/2020 8:18 PM, Thomas Monjalon wrote:
> 30/01/2020 17:09, Ferruh Yigit:
>> On 1/29/2020 8:13 PM, Akhil Goyal wrote:
>>>
>>>
>>>>
>>>> On Wed, Jan 29, 2020 at 7:10 PM Anoob Joseph <anoobj@marvell.com> wrote:
>>>>> The asymmetric crypto library is experimental. Changes to experimental code
>>>> paths is allowed, right?
>>>>
>>>> The asymmetric crypto enum is referenced by a function part of the stable ABI.
>>>> It is possible to waive this enum, if we are sure no use out of the
>>>> experimental asym crypto APIs is possible.
>>>>
>>>> The rest of the changes touch stable symbols.
>>>>
>>>> Adding the abidiff report:
>>>>
>>>> [C]'function void rte_cryptodev_info_get(uint8_t,
>>>> rte_cryptodev_info*)' at rte_cryptodev.c:1115:1 has some indirect
>>>> sub-type changes:
>>>> parameter 2 of type 'rte_cryptodev_info*' has sub-type changes:
>>>> in pointed to type 'struct rte_cryptodev_info' at rte_cryptodev.h:468:1:
>>>> type size hasn't changed
>>>> 1 data member change:
>>>> type of 'const rte_cryptodev_capabilities*
>>>> rte_cryptodev_info::capabilities' changed:
>>>> in pointed to type 'const rte_cryptodev_capabilities':
>>>> in unqualified underlying type 'struct
>>>> rte_cryptodev_capabilities' at rte_cryptodev.h:176:1:
>>>> type size hasn't changed
>>>> 1 data member change:
>>>> type of '__anonymous_union__ ' changed:
>>>> type size hasn't changed
>>>> 1 data member change:
>>>> type of 'rte_cryptodev_asymmetric_capability
>>>> __anonymous_union__::asym' changed:
>>>> type size hasn't changed
>>>> 1 data member change:
>>>> type of
>>>> 'rte_cryptodev_asymmetric_xform_capability
>>>> rte_cryptodev_asymmetric_capability::xform_capa' changed:
>>>> type size hasn't changed
>>>> 1 data member change:
>>>> type of 'rte_crypto_asym_xform_type
>>>> rte_cryptodev_asymmetric_xform_capability::xform_type' changed:
>>>> type size hasn't changed
>>>> 2 enumerator insertions:
>>>>
>>>> 'rte_crypto_asym_xform_type::RTE_CRYPTO_ASYM_XFORM_ECDSA' value '7'
>>>>
>>>> 'rte_crypto_asym_xform_type::RTE_CRYPTO_ASYM_XFORM_ECPM' value '8'
>>>> 1 enumerator change:
>>>>
>>>> 'rte_crypto_asym_xform_type::RTE_CRYPTO_ASYM_XFORM_TYPE_LIST_END'
>>>> from
>>>> value '7' to '9' at rte_crypto_asym.h:60:1
>>>>
>>>
>>> I believe these enums will be used only in case of ASYM case which is experimental.
>>
>> Independent from being experiment and not, this shouldn't be a problem, I think
>> this is a false positive.
>>
>> The ABI break can happen when a struct has been shared between the application
>> and the library (DPDK) and the layout of that memory know differently by
>> application and the library.
>>
>> Here in all cases, there is no layout/size change.
>>
>> As to the value changes of the enums, since application compiled with old DPDK,
>> it will know only up to '6', 7 and more means invalid to the application. So it
>> won't send these values also it should ignore these values from library. Only
>> consequence is old application won't able to use new features those new enums
>> provide but that is expected/normal.
>
> If library give higher value than expected by the application,
> if the application uses this value as array index,
> there can be an access out of bounds.
First this concern is not an ABI break concern, but application should ignore
any value bigger than the MAX value it knows.
Otherwise this would mean we can't add any new enum or define to the project,
which is wrong I believe.
>
>
>>>> [C]'function int
>>>> rte_cryptodev_get_aead_algo_enum(rte_crypto_aead_algorithm*, const
>>>> char*)' at rte_cryptodev.c:239:1 has some indirect sub-type changes:
>>>> parameter 1 of type 'rte_crypto_aead_algorithm*' has sub-type changes:
>>>> in pointed to type 'enum rte_crypto_aead_algorithm' at
>>>> rte_crypto_sym.h:346:1:
>>>> type size hasn't changed
>>>> 1 enumerator insertion:
>>>> 'rte_crypto_aead_algorithm::RTE_CRYPTO_AEAD_CHACHA20_POLY1305'
>>>> value '3'
>>>> 1 enumerator change:
>>>> 'rte_crypto_aead_algorithm::RTE_CRYPTO_AEAD_LIST_END' from
>>>> value '3' to '4' at rte_crypto_sym.h:346:1
>>
>> Same as above, no layout change.
>>
>>>>
>>>>
>>>> [C]'const char* rte_crypto_aead_algorithm_strings[1]' was changed at
>>>> rte_crypto_sym.h:358:1:
>>>> size of symbol (in bytes) changed from 24 to 32
>>>>
>>
>> The shared memory size changes, but this is global variable in the library, and
>> the values application can request 'RTE_CRYPTO_AEAD_AES_CCM' &
>> 'RTE_CRYPTO_AEAD_AES_GCM' is already there, so there is no backward
>> compatibility issue here.
>
> For this one, I don't know what is the breakage.
>
>
>>> +Fiona and Arek
>>>
>>> We may need to revert the chacha-poly patches.
>>>
>>
>> I don't see any ABI break in this case, can someone explain if I am missing
>> anything here?
>
>
>
>
>
^ permalink raw reply [relevance 7%]
* Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
2020-01-30 23:49 4% ` Ananyev, Konstantin
@ 2020-01-31 9:07 4% ` Ferruh Yigit
2020-01-31 9:37 4% ` Ananyev, Konstantin
0 siblings, 1 reply; 200+ results
From: Ferruh Yigit @ 2020-01-31 9:07 UTC (permalink / raw)
To: Ananyev, Konstantin, Thomas Monjalon, Anoob Joseph, akhil.goyal,
Trahe, Fiona
Cc: dev, David Marchand, Richardson, Bruce, nhorman, Mcnamara, John,
Kusztal, ArkadiuszX
On 1/30/2020 11:49 PM, Ananyev, Konstantin wrote:
>
>
>> -----Original Message-----
>> From: dev <dev-bounces@dpdk.org> On Behalf Of Thomas Monjalon
>> Sent: Thursday, January 30, 2020 4:00 PM
>> To: Anoob Joseph <anoobj@marvell.com>; akhil.goyal@nxp.com; Trahe, Fiona <fiona.trahe@intel.com>
>> Cc: dev@dpdk.org; David Marchand <david.marchand@redhat.com>; Richardson, Bruce <bruce.richardson@intel.com>;
>> nhorman@tuxdriver.com; Mcnamara, John <john.mcnamara@intel.com>; Trahe, Fiona <fiona.trahe@intel.com>; Kusztal, ArkadiuszX
>> <arkadiuszx.kusztal@intel.com>; Yigit, Ferruh <ferruh.yigit@intel.com>
>> Subject: Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
>>
>> 30/01/2020 14:06, Trahe, Fiona:
>>> We were unaware the LIST_END change could constitute an ABI breakage, but can see how it affects the array size when picked up.
>>> We're exploring options.
>>>
>>> I agree with Anoob's point that if we don't allow the LIST_END to be modified, then it means no feature can be implemented without ABI
>> breakage.
>>> Anyone object to removing those LIST_END elements - or have a better suggestion? Would have to be in 20.11 I suppose.
>>
>> Yes, having max value right after the last value is ridiculous,
>> it prevents adding any value.
>> In 20.11, we should remove all these *_END and *_MAX from API enums
>> and replace them with a separate #define with reasonnable maximums.
>>
>
> I think we'd better avoid public structs that have array of _MAX elems in them.
>
That should fix, but we need to wait for 20.11 for the change, and what should
be the new array size?
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [RFC v2] meter: fix ABI break due to experimental tag removal
2020-01-30 12:33 7% ` Thomas Monjalon
2020-01-30 12:57 4% ` Luca Boccassi
@ 2020-01-31 9:25 4% ` Ferruh Yigit
1 sibling, 0 replies; 200+ results
From: Ferruh Yigit @ 2020-01-31 9:25 UTC (permalink / raw)
To: Thomas Monjalon
Cc: Neil Horman, Cristian Dumitrescu, Eelco Chaudron, dev,
Luca Boccassi, David Marchand, Bruce Richardson, Ian Stokes,
Andrzej Ostruszka
On 1/30/2020 12:33 PM, Thomas Monjalon wrote:
> Hi,
>
> I disagree with the need of this patch.
> The symbol was experimental, meaning we can change it.
> Removing experimental tag is not an ABI break.
In theory what you said is correct, this is experimental API and anything can
happen to it, this is the contract.
And when we need to change an experimental API, the users will be affected,
there is no escape from it.
But for this case limitation is more mechanical I believe,
because API is not changing at all, and it is becoming mature.
So we are breaking the user application because the experimental API they are
using becoming mature without any change.
This looks annoying from the user perspective, and if we can prevent this break,
I am for preventing it.
>
>
> 29/01/2020 17:43, Ferruh Yigit:
>> Duplicated the existing symbol and versioned one as experimental and
>> other as stable.
> [..]
>> --- a/lib/librte_meter/rte_meter_version.map
>> +++ b/lib/librte_meter/rte_meter_version.map
>> @@ -20,4 +20,12 @@ DPDK_20.0.1 {
>> rte_meter_trtcm_rfc4115_color_blind_check;
>> rte_meter_trtcm_rfc4115_config;
>> rte_meter_trtcm_rfc4115_profile_config;
>> +
>> } DPDK_20.0;
>> +
>> +EXPERIMENTAL {
>> + global:
>> +
>> + rte_meter_trtcm_rfc4115_config;
>> + rte_meter_trtcm_rfc4115_profile_config;
>> +};
>
>
>
>
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
2020-01-31 9:07 4% ` Ferruh Yigit
@ 2020-01-31 9:37 4% ` Ananyev, Konstantin
0 siblings, 0 replies; 200+ results
From: Ananyev, Konstantin @ 2020-01-31 9:37 UTC (permalink / raw)
To: Yigit, Ferruh, Thomas Monjalon, Anoob Joseph, akhil.goyal, Trahe, Fiona
Cc: dev, David Marchand, Richardson, Bruce, nhorman, Mcnamara, John,
Kusztal, ArkadiuszX
> -----Original Message-----
> From: Yigit, Ferruh <ferruh.yigit@intel.com>
> Sent: Friday, January 31, 2020 9:07 AM
> To: Ananyev, Konstantin <konstantin.ananyev@intel.com>; Thomas Monjalon <thomas@monjalon.net>; Anoob Joseph
> <anoobj@marvell.com>; akhil.goyal@nxp.com; Trahe, Fiona <fiona.trahe@intel.com>
> Cc: dev@dpdk.org; David Marchand <david.marchand@redhat.com>; Richardson, Bruce <bruce.richardson@intel.com>;
> nhorman@tuxdriver.com; Mcnamara, John <john.mcnamara@intel.com>; Kusztal, ArkadiuszX <arkadiuszx.kusztal@intel.com>
> Subject: Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
>
> On 1/30/2020 11:49 PM, Ananyev, Konstantin wrote:
> >
> >
> >> -----Original Message-----
> >> From: dev <dev-bounces@dpdk.org> On Behalf Of Thomas Monjalon
> >> Sent: Thursday, January 30, 2020 4:00 PM
> >> To: Anoob Joseph <anoobj@marvell.com>; akhil.goyal@nxp.com; Trahe, Fiona <fiona.trahe@intel.com>
> >> Cc: dev@dpdk.org; David Marchand <david.marchand@redhat.com>; Richardson, Bruce <bruce.richardson@intel.com>;
> >> nhorman@tuxdriver.com; Mcnamara, John <john.mcnamara@intel.com>; Trahe, Fiona <fiona.trahe@intel.com>; Kusztal, ArkadiuszX
> >> <arkadiuszx.kusztal@intel.com>; Yigit, Ferruh <ferruh.yigit@intel.com>
> >> Subject: Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
> >>
> >> 30/01/2020 14:06, Trahe, Fiona:
> >>> We were unaware the LIST_END change could constitute an ABI breakage, but can see how it affects the array size when picked up.
> >>> We're exploring options.
> >>>
> >>> I agree with Anoob's point that if we don't allow the LIST_END to be modified, then it means no feature can be implemented without
> ABI
> >> breakage.
> >>> Anyone object to removing those LIST_END elements - or have a better suggestion? Would have to be in 20.11 I suppose.
> >>
> >> Yes, having max value right after the last value is ridiculous,
> >> it prevents adding any value.
> >> In 20.11, we should remove all these *_END and *_MAX from API enums
> >> and replace them with a separate #define with reasonnable maximums.
> >>
> >
> > I think we'd better avoid public structs that have array of _MAX elems in them.
> >
>
> That should fix, but we need to wait for 20.11 for the change, and what should
> be the new array size?
Make it dynamic whenever possible?
Make Input/output args to provide both pointer and size,
or use some predefined value for terminating element (NULL, -1, etc.)?
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
2020-01-31 9:03 7% ` Ferruh Yigit
@ 2020-01-31 10:26 7% ` Ananyev, Konstantin
0 siblings, 0 replies; 200+ results
From: Ananyev, Konstantin @ 2020-01-31 10:26 UTC (permalink / raw)
To: Yigit, Ferruh, Thomas Monjalon, Akhil Goyal, David Marchand,
Anoob Joseph, Trahe, Fiona, Kusztal, ArkadiuszX
Cc: dev, Richardson, Bruce, nhorman, Mcnamara, John, dodji, Andrew Rybchenko
>
> On 1/30/2020 8:18 PM, Thomas Monjalon wrote:
> > 30/01/2020 17:09, Ferruh Yigit:
> >> On 1/29/2020 8:13 PM, Akhil Goyal wrote:
> >>>
> >>>
> >>>>
> >>>> On Wed, Jan 29, 2020 at 7:10 PM Anoob Joseph <anoobj@marvell.com> wrote:
> >>>>> The asymmetric crypto library is experimental. Changes to experimental code
> >>>> paths is allowed, right?
> >>>>
> >>>> The asymmetric crypto enum is referenced by a function part of the stable ABI.
> >>>> It is possible to waive this enum, if we are sure no use out of the
> >>>> experimental asym crypto APIs is possible.
> >>>>
> >>>> The rest of the changes touch stable symbols.
> >>>>
> >>>> Adding the abidiff report:
> >>>>
> >>>> [C]'function void rte_cryptodev_info_get(uint8_t,
> >>>> rte_cryptodev_info*)' at rte_cryptodev.c:1115:1 has some indirect
> >>>> sub-type changes:
> >>>> parameter 2 of type 'rte_cryptodev_info*' has sub-type changes:
> >>>> in pointed to type 'struct rte_cryptodev_info' at rte_cryptodev.h:468:1:
> >>>> type size hasn't changed
> >>>> 1 data member change:
> >>>> type of 'const rte_cryptodev_capabilities*
> >>>> rte_cryptodev_info::capabilities' changed:
> >>>> in pointed to type 'const rte_cryptodev_capabilities':
> >>>> in unqualified underlying type 'struct
> >>>> rte_cryptodev_capabilities' at rte_cryptodev.h:176:1:
> >>>> type size hasn't changed
> >>>> 1 data member change:
> >>>> type of '__anonymous_union__ ' changed:
> >>>> type size hasn't changed
> >>>> 1 data member change:
> >>>> type of 'rte_cryptodev_asymmetric_capability
> >>>> __anonymous_union__::asym' changed:
> >>>> type size hasn't changed
> >>>> 1 data member change:
> >>>> type of
> >>>> 'rte_cryptodev_asymmetric_xform_capability
> >>>> rte_cryptodev_asymmetric_capability::xform_capa' changed:
> >>>> type size hasn't changed
> >>>> 1 data member change:
> >>>> type of 'rte_crypto_asym_xform_type
> >>>> rte_cryptodev_asymmetric_xform_capability::xform_type' changed:
> >>>> type size hasn't changed
> >>>> 2 enumerator insertions:
> >>>>
> >>>> 'rte_crypto_asym_xform_type::RTE_CRYPTO_ASYM_XFORM_ECDSA' value '7'
> >>>>
> >>>> 'rte_crypto_asym_xform_type::RTE_CRYPTO_ASYM_XFORM_ECPM' value '8'
> >>>> 1 enumerator change:
> >>>>
> >>>> 'rte_crypto_asym_xform_type::RTE_CRYPTO_ASYM_XFORM_TYPE_LIST_END'
> >>>> from
> >>>> value '7' to '9' at rte_crypto_asym.h:60:1
> >>>>
> >>>
> >>> I believe these enums will be used only in case of ASYM case which is experimental.
> >>
> >> Independent from being experiment and not, this shouldn't be a problem, I think
> >> this is a false positive.
> >>
> >> The ABI break can happen when a struct has been shared between the application
> >> and the library (DPDK) and the layout of that memory know differently by
> >> application and the library.
> >>
> >> Here in all cases, there is no layout/size change.
> >>
> >> As to the value changes of the enums, since application compiled with old DPDK,
> >> it will know only up to '6', 7 and more means invalid to the application. So it
> >> won't send these values also it should ignore these values from library. Only
> >> consequence is old application won't able to use new features those new enums
> >> provide but that is expected/normal.
> >
> > If library give higher value than expected by the application,
> > if the application uses this value as array index,
> > there can be an access out of bounds.
>
> First this concern is not an ABI break concern, but application should ignore
> any value bigger than the MAX value it knows.
> Otherwise this would mean we can't add any new enum or define to the project,
> which is wrong I believe.
>
> >
> >
> >>>> [C]'function int
> >>>> rte_cryptodev_get_aead_algo_enum(rte_crypto_aead_algorithm*, const
> >>>> char*)' at rte_cryptodev.c:239:1 has some indirect sub-type changes:
> >>>> parameter 1 of type 'rte_crypto_aead_algorithm*' has sub-type changes:
> >>>> in pointed to type 'enum rte_crypto_aead_algorithm' at
> >>>> rte_crypto_sym.h:346:1:
> >>>> type size hasn't changed
> >>>> 1 enumerator insertion:
> >>>> 'rte_crypto_aead_algorithm::RTE_CRYPTO_AEAD_CHACHA20_POLY1305'
> >>>> value '3'
> >>>> 1 enumerator change:
> >>>> 'rte_crypto_aead_algorithm::RTE_CRYPTO_AEAD_LIST_END' from
> >>>> value '3' to '4' at rte_crypto_sym.h:346:1
> >>
> >> Same as above, no layout change.
> >>
> >>>>
> >>>>
> >>>> [C]'const char* rte_crypto_aead_algorithm_strings[1]' was changed at
> >>>> rte_crypto_sym.h:358:1:
> >>>> size of symbol (in bytes) changed from 24 to 32
> >>>>
> >>
> >> The shared memory size changes, but this is global variable in the library, and
> >> the values application can request 'RTE_CRYPTO_AEAD_AES_CCM' &
> >> 'RTE_CRYPTO_AEAD_AES_GCM' is already there, so there is no backward
> >> compatibility issue here.
> >
> > For this one, I don't know what is the breakage.
Reading through this report, I am also don't see why it is considered as ABI breakage.
Yes, size of rte_crypto_aead_algorithm_strings[] has changed, but this array is not public one.
Also I don't see any place where we use RTE_CRYPTO_AEAD_LIST_END to define array size
in our public API.
At first glance it looks like false positive to me.
Do I miss something obvious here?
Konstantin
> >
> >
> >>> +Fiona and Arek
> >>>
> >>> We may need to revert the chacha-poly patches.
> >>>
> >>
> >> I don't see any ABI break in this case, can someone explain if I am missing
> >> anything here?
> >
> >
> >
> >
> >
^ permalink raw reply [relevance 7%]
* Re: [dpdk-dev] [PATCH 2/2] eal: add eal_parse_optionlist to parse user input
2020-01-29 19:19 0% ` Stephen Hemminger
@ 2020-01-31 11:25 0% ` Govindharajan, Hariprasad
2020-01-31 14:42 0% ` Govindharajan, Hariprasad
1 sibling, 0 replies; 200+ results
From: Govindharajan, Hariprasad @ 2020-01-31 11:25 UTC (permalink / raw)
To: Stephen Hemminger, Yigit, Ferruh; +Cc: David Marchand, Thomas Monjalon, dev
Hi,
I am planning to move the existing parser function to the testpmd file instead of keeping it in the eal and will revert the eal back.
Also, I am planning to create an util file in eal with this parser and do a RFC.
@Stephen Hemminger, We already investigated the existing function and then decided to re-use it as seen in the patch. For creating an API, is there any other specific requirements
that should be addressed? Please clarify us.
Thanks
G Hariprasad
-----Original Message-----
From: Stephen Hemminger <stephen@networkplumber.org>
Sent: Wednesday, January 29, 2020 7:20 PM
To: Yigit, Ferruh <ferruh.yigit@intel.com>
Cc: David Marchand <david.marchand@redhat.com>; Thomas Monjalon <thomas@monjalon.net>; Govindharajan, Hariprasad <hariprasad.govindharajan@intel.com>; dev <dev@dpdk.org>
Subject: Re: [dpdk-dev] [PATCH 2/2] eal: add eal_parse_optionlist to parse user input
On Wed, 29 Jan 2020 18:07:05 +0000
Ferruh Yigit <ferruh.yigit@intel.com> wrote:
> On 1/29/2020 5:44 PM, David Marchand wrote:
> > On Tue, Jan 28, 2020 at 6:35 PM Ferruh Yigit <ferruh.yigit@intel.com> wrote:
> >> On 1/27/2020 10:30 AM, Hariprasad Govindharajan wrote:
> >>> In current version, there is a function which parses the corelist
> >>> based on user value. A new generic function eal_parse_optionlist
> >>> is added which will parse corelist as well as similar user input
> >>> so that we can use it as a public API too.
> >>>
> >>> Signed-off-by: Hariprasad Govindharajan
> >>> <hariprasad.govindharajan@intel.com>
> >>
> >> Hi David,
> >>
> >> Overall this patchset is to add '--portlist' command to testpmd and
> >> remove existing 64 port limitation.
> >>
> >> And in this patch re-uses the exiting parser function in eal and
> >> converts it to API, question is if eal is good place to have this API, what do you think about it?
> >
> > Exporting string parsers from the EAL has little value.
> > Ok we avoid code duplication (and I can see other places in the tree
> > where it might be used), but in the end we will have to maintain
> > this API in the ABI when it enters the stable ABI.
> >
> > I am for avoiding this.
> >
> >
>
> The same function can be used in some sample applications too (which
> are using port mask), so instead of duplicating it multiple times, it
> would be nice to have this function somewhere that applications can use.
>
> Does it makes sense to have a rte_util.c (under in eal or as a
> separate library) to have this kind of application helper functions?
It makes sense to have a rte library that handles arbitrary size bitvector and has string handling functions. Kind of like what kernel has for the cpuset parsing. This could be used for cpus in EAL and port-masks or other arrays in applications.
But just doing copy/paste of existing code without thinking about how API should work is a bad idea.
^ permalink raw reply [relevance 0%]
* [dpdk-dev] Minutes of Technical Board Meeting, 2020-01-29
@ 2020-01-31 12:36 4% Olivier Matz
0 siblings, 0 replies; 200+ results
From: Olivier Matz @ 2020-01-31 12:36 UTC (permalink / raw)
To: dev
Minutes of Technical Board Meeting, 2020-01-29
Members Attending
-----------------
- Bruce
- Ferruh
- Hemant
- Honnappa
- Jerin
- Kevin
- Konstantin
- Olivier (chair)
- Stephen
- Thomas
NOTE: The technical board meetings every second Wednesday on IRC
channel #dpdk-board, at 3pm UTC. Meetings are public and DPDK community
members are welcome to attend.
NOTE: Next meeting will be on Wednesday 2020-02-12 @3pm UTC, and will be
chaired by Stephen.
1/ Integration of SAD capability in ipsec-secgw
With v5, performance drop is less than 2% on x86, so it can be
integrated if the performance is confirmed on other archs.
2/ Guidelines for examples and apps distributed in DPDK repository
- In dpdk/examples directory, the applications should demonstrate
the API usage in priority, and the performance (ideally both).
- If both goals cannot be reconciled in the example application, a
performance application can be created in dpdk/app (for main
features).
3/ ICC support in DPDK
- It was already agreed that compilers supported in DPDK are GCC and
CLANG: http://mails.dpdk.org/archives/dev/2019-June/135847.html
- However patches for ICC support are accepted.
- An ICC compilation warning should not be fatal.
(note: this is already the default for meson builds).
- Some ICC compilation warnings will be disabled at a global level
4/ Name of repository for kernel modules
On 2019-11-06, we decided to move igb_uio in a separate repository
before 20.11:
http://mails.dpdk.org/archives/dev/2019-November/151763.html
A repository will also be needed to host netuio for Windows.
The name of this repository will be *dpdk-kmods*.
AR for Jerin: write the deprecation notice for removal of igb_uio.
already done: http://inbox.dpdk.org/dev/20200130165853.693270-1-jerinj@marvell.com
5/ Reorganize EAL to ease windows port integration
Thomas will propose a patch to simplify the layout of
dpdk/lib/librte_eal.
6/ ABI topics
In 20.02-rc1, some possible ABI breaks were found in meter, crypto
and security libraries: to be checked and fixed before the release.
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
2020-01-30 20:18 4% ` Thomas Monjalon
2020-01-31 9:03 7% ` Ferruh Yigit
@ 2020-01-31 14:16 4% ` Trahe, Fiona
2020-02-02 13:05 9% ` Thomas Monjalon
1 sibling, 1 reply; 200+ results
From: Trahe, Fiona @ 2020-01-31 14:16 UTC (permalink / raw)
To: Thomas Monjalon, Akhil Goyal, David Marchand, Anoob Joseph,
Kusztal, ArkadiuszX, Yigit, Ferruh
Cc: dev, Richardson, Bruce, nhorman, Mcnamara, John, dodji,
Andrew Rybchenko, Trahe, Fiona
> > > I believe these enums will be used only in case of ASYM case which is experimental.
> >
> > Independent from being experiment and not, this shouldn't be a problem, I think
> > this is a false positive.
> >
> > The ABI break can happen when a struct has been shared between the application
> > and the library (DPDK) and the layout of that memory know differently by
> > application and the library.
> >
> > Here in all cases, there is no layout/size change.
> >
> > As to the value changes of the enums, since application compiled with old DPDK,
> > it will know only up to '6', 7 and more means invalid to the application. So it
> > won't send these values also it should ignore these values from library. Only
> > consequence is old application won't able to use new features those new enums
> > provide but that is expected/normal.
>
> If library give higher value than expected by the application,
> if the application uses this value as array index,
> there can be an access out of bounds.
[Fiona] All asymmetric APIs are experimental so above shouldn't be a problem.
But for the same issue with sym crypto below, I believe Ferruh's explanation makes
sense and I don't see how there can be an API breakage.
So if an application hasn't compiled against the new lib it will be still using the old value
which will be within bounds. If it's picking up the higher new value from the lib it must
have been compiled against the lib so shouldn't have problems.
There are also no structs on the API which contain arrays using this for sizing, so I don't see an
opportunity for an appl to have a mismatch in memory addresses.
> > >> [C]'function int
> > >> rte_cryptodev_get_aead_algo_enum(rte_crypto_aead_algorithm*, const
> > >> char*)' at rte_cryptodev.c:239:1 has some indirect sub-type changes:
> > >> parameter 1 of type 'rte_crypto_aead_algorithm*' has sub-type changes:
> > >> in pointed to type 'enum rte_crypto_aead_algorithm' at
> > >> rte_crypto_sym.h:346:1:
> > >> type size hasn't changed
> > >> 1 enumerator insertion:
> > >> 'rte_crypto_aead_algorithm::RTE_CRYPTO_AEAD_CHACHA20_POLY1305'
> > >> value '3'
> > >> 1 enumerator change:
> > >> 'rte_crypto_aead_algorithm::RTE_CRYPTO_AEAD_LIST_END' from
> > >> value '3' to '4' at rte_crypto_sym.h:346:1
> >
> > Same as above, no layout change.
> >
> > >>
> > >>
> > >> [C]'const char* rte_crypto_aead_algorithm_strings[1]' was changed at
> > >> rte_crypto_sym.h:358:1:
> > >> size of symbol (in bytes) changed from 24 to 32
> > >>
> >
> > The shared memory size changes, but this is global variable in the library, and
> > the values application can request 'RTE_CRYPTO_AEAD_AES_CCM' &
> > 'RTE_CRYPTO_AEAD_AES_GCM' is already there, so there is no backward
> > compatibility issue here.
>
> For this one, I don't know what is the breakage.
>
>
> > > +Fiona and Arek
> > >
> > > We may need to revert the chacha-poly patches.
> > >
> >
> > I don't see any ABI break in this case, can someone explain if I am missing
> > anything here?
>
>
>
>
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH 2/2] eal: add eal_parse_optionlist to parse user input
2020-01-29 19:19 0% ` Stephen Hemminger
2020-01-31 11:25 0% ` Govindharajan, Hariprasad
@ 2020-01-31 14:42 0% ` Govindharajan, Hariprasad
1 sibling, 0 replies; 200+ results
From: Govindharajan, Hariprasad @ 2020-01-31 14:42 UTC (permalink / raw)
To: Stephen Hemminger, Yigit, Ferruh; +Cc: David Marchand, Thomas Monjalon, dev
> -----Original Message-----
> From: Stephen Hemminger <stephen@networkplumber.org>
> Sent: Wednesday, January 29, 2020 7:20 PM
> To: Yigit, Ferruh <ferruh.yigit@intel.com>
> Cc: David Marchand <david.marchand@redhat.com>; Thomas Monjalon
> <thomas@monjalon.net>; Govindharajan, Hariprasad
> <hariprasad.govindharajan@intel.com>; dev <dev@dpdk.org>
> Subject: Re: [dpdk-dev] [PATCH 2/2] eal: add eal_parse_optionlist to parse
> user input
>
> On Wed, 29 Jan 2020 18:07:05 +0000
> Ferruh Yigit <ferruh.yigit@intel.com> wrote:
>
> > On 1/29/2020 5:44 PM, David Marchand wrote:
> > > On Tue, Jan 28, 2020 at 6:35 PM Ferruh Yigit <ferruh.yigit@intel.com>
> wrote:
> > >> On 1/27/2020 10:30 AM, Hariprasad Govindharajan wrote:
> > >>> In current version, there is a function which parses the corelist
> > >>> based on user value. A new generic function eal_parse_optionlist
> > >>> is added which will parse corelist as well as similar user input
> > >>> so that we can use it as a public API too.
> > >>>
> > >>> Signed-off-by: Hariprasad Govindharajan
> > >>> <hariprasad.govindharajan@intel.com>
> > >>
> > >> Hi David,
> > >>
> > >> Overall this patchset is to add '--portlist' command to testpmd and
> > >> remove existing 64 port limitation.
> > >>
> > >> And in this patch re-uses the exiting parser function in eal and
> > >> converts it to API, question is if eal is good place to have this API, what
> do you think about it?
> > >
> > > Exporting string parsers from the EAL has little value.
> > > Ok we avoid code duplication (and I can see other places in the tree
> > > where it might be used), but in the end we will have to maintain
> > > this API in the ABI when it enters the stable ABI.
> > >
> > > I am for avoiding this.
> > >
> > >
> >
> > The same function can be used in some sample applications too (which
> > are using port mask), so instead of duplicating it multiple times, it
> > would be nice to have this function somewhere that applications can use.
> >
> > Does it makes sense to have a rte_util.c (under in eal or as a
> > separate library) to have this kind of application helper functions?
>
> It makes sense to have a rte library that handles arbitrary size bitvector and
> has string handling functions. Kind of like what kernel has for the cpuset
> parsing. This could be used for cpus in EAL and port-masks or other arrays in
> applications.
>
> But just doing copy/paste of existing code without thinking about how API
> should work is a bad idea.
[Govindharajan, Hariprasad]
Hi,
PLEASE IGNORE MY PREVIOUS EMAIL.
I am planning to move the existing parser function to the testpmd file instead of keeping it in the eal and will revert the eal back.
Also, I am planning to create an util file in eal with this parser and do a RFC.
@Stephen Hemminger, We already investigated the existing function and then decided to re-use it as seen in the patch. For creating an API, is there any other specific requirements that should be addressed? Please clarify us.
Thanks
G Hariprasad
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH v3 4/4] add ABI checks
2020-01-30 22:32 4% ` Thomas Monjalon
@ 2020-02-01 15:29 4% ` David Marchand
0 siblings, 0 replies; 200+ results
From: David Marchand @ 2020-02-01 15:29 UTC (permalink / raw)
To: Thomas Monjalon
Cc: dev, Bruce Richardson, Kevin Laatz, Aaron Conole, Neil Horman,
Akhil Goyal, Anoob Joseph, Luca Boccassi, Trahe, Fiona, Yigit,
Ferruh, Michael Santana, John McNamara, Marko Kovacevic
On Thu, Jan 30, 2020 at 11:33 PM Thomas Monjalon <thomas@monjalon.net> wrote:
>
> 30/01/2020 17:00, David Marchand:
> > Enabling them requires a configuration that will trigger the ABI dumps
> > generation as part of the existing devtools/test-build.sh and
> > devtools/test-meson-builds.sh scripts.
> [...]
> > --- a/devtools/test-meson-builds.sh
> > +++ b/devtools/test-meson-builds.sh
> > if [ -f "$builddir/build.ninja" ] ; then
> > + # for existing environments, force debugoptimized so that ABI
> > + # checks can run
> > + $MESON configure --buildtype=debugoptimized $builddir
>
> This is forcing meson to re-run each time, even if the buildtype is
> already "debugoptimized".
> Please query meson configuration to avoid useless re-run:
>
> $MESON configure $builddir | awk '$1=="buildtype" {print $2}'
>
>
Good catch, will fix.
--
David Marchand
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
2020-01-31 14:16 4% ` Trahe, Fiona
@ 2020-02-02 13:05 9% ` Thomas Monjalon
2020-02-02 14:41 4% ` Ananyev, Konstantin
0 siblings, 1 reply; 200+ results
From: Thomas Monjalon @ 2020-02-02 13:05 UTC (permalink / raw)
To: Akhil Goyal, Yigit, Ferruh, Trahe, Fiona
Cc: David Marchand, Anoob Joseph, Kusztal, ArkadiuszX, dev,
Richardson, Bruce, nhorman, Mcnamara, John, dodji,
Andrew Rybchenko, Trahe, Fiona, aconole
31/01/2020 15:16, Trahe, Fiona:
> On 1/30/2020 8:18 PM, Thomas Monjalon wrote:
> > 30/01/2020 17:09, Ferruh Yigit:
> > > On 1/29/2020 8:13 PM, Akhil Goyal wrote:
> > > >
> > > > I believe these enums will be used only in case of ASYM case which is experimental.
> > >
> > > Independent from being experiment and not, this shouldn't be a problem, I think
> > > this is a false positive.
> > >
> > > The ABI break can happen when a struct has been shared between the application
> > > and the library (DPDK) and the layout of that memory know differently by
> > > application and the library.
> > >
> > > Here in all cases, there is no layout/size change.
> > >
> > > As to the value changes of the enums, since application compiled with old DPDK,
> > > it will know only up to '6', 7 and more means invalid to the application. So it
> > > won't send these values also it should ignore these values from library. Only
> > > consequence is old application won't able to use new features those new enums
> > > provide but that is expected/normal.
> >
> > If library give higher value than expected by the application,
> > if the application uses this value as array index,
> > there can be an access out of bounds.
>
> [Fiona] All asymmetric APIs are experimental so above shouldn't be a problem.
> But for the same issue with sym crypto below, I believe Ferruh's explanation makes
> sense and I don't see how there can be an API breakage.
> So if an application hasn't compiled against the new lib it will be still using the old value
> which will be within bounds. If it's picking up the higher new value from the lib it must
> have been compiled against the lib so shouldn't have problems.
You say there is no ABI issue because the application will be re-compiled
for the updated library. Indeed, compilation fixes compatibility issues.
But this is not relevant for ABI compatibility.
ABI compatibility means we can upgrade the library without recompiling
the application and it must work.
You think it is a false positive because you assume the application
"picks" the new value. I think you miss the case where the new value
is returned by a function in the upgraded library.
> There are also no structs on the API which contain arrays using this
> for sizing, so I don't see an opportunity for an appl to have a
> mismatch in memory addresses.
Let me demonstrate where the API may "use" the new value
RTE_CRYPTO_AEAD_CHACHA20_POLY1305 and how it impacts the application.
Once upon a time a DPDK application counting the number of devices
supporting each AEAD algo (in order to find the best supported algo).
It is done in an array indexed by algo id:
int aead_dev_count[RTE_CRYPTO_AEAD_LIST_END];
The application is compiled with DPDK 19.11,
where RTE_CRYPTO_AEAD_LIST_END = 3.
So the size of the application array aead_dev_count is 3.
This binary is run with DPDK 20.02,
where RTE_CRYPTO_AEAD_CHACHA20_POLY1305 = 3.
When calling rte_cryptodev_info_get() on a device QAT_GEN3,
rte_cryptodev_info.capabilities.sym.aead.algo is set to
RTE_CRYPTO_AEAD_CHACHA20_POLY1305 (= 3).
The application uses this value:
++ aead_dev_count[info.capabilities.sym.aead.algo];
The application is crashing because of out of bound access.
^ permalink raw reply [relevance 9%]
* Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
2020-02-02 13:05 9% ` Thomas Monjalon
@ 2020-02-02 14:41 4% ` Ananyev, Konstantin
2020-02-03 9:30 7% ` Ferruh Yigit
0 siblings, 1 reply; 200+ results
From: Ananyev, Konstantin @ 2020-02-02 14:41 UTC (permalink / raw)
To: Thomas Monjalon, Akhil Goyal, Yigit, Ferruh, Trahe, Fiona
Cc: David Marchand, Anoob Joseph, Kusztal, ArkadiuszX, dev,
Richardson, Bruce, nhorman, Mcnamara, John, dodji,
Andrew Rybchenko, Trahe, Fiona, aconole
> 31/01/2020 15:16, Trahe, Fiona:
> > On 1/30/2020 8:18 PM, Thomas Monjalon wrote:
> > > 30/01/2020 17:09, Ferruh Yigit:
> > > > On 1/29/2020 8:13 PM, Akhil Goyal wrote:
> > > > >
> > > > > I believe these enums will be used only in case of ASYM case which is experimental.
> > > >
> > > > Independent from being experiment and not, this shouldn't be a problem, I think
> > > > this is a false positive.
> > > >
> > > > The ABI break can happen when a struct has been shared between the application
> > > > and the library (DPDK) and the layout of that memory know differently by
> > > > application and the library.
> > > >
> > > > Here in all cases, there is no layout/size change.
> > > >
> > > > As to the value changes of the enums, since application compiled with old DPDK,
> > > > it will know only up to '6', 7 and more means invalid to the application. So it
> > > > won't send these values also it should ignore these values from library. Only
> > > > consequence is old application won't able to use new features those new enums
> > > > provide but that is expected/normal.
> > >
> > > If library give higher value than expected by the application,
> > > if the application uses this value as array index,
> > > there can be an access out of bounds.
> >
> > [Fiona] All asymmetric APIs are experimental so above shouldn't be a problem.
> > But for the same issue with sym crypto below, I believe Ferruh's explanation makes
> > sense and I don't see how there can be an API breakage.
> > So if an application hasn't compiled against the new lib it will be still using the old value
> > which will be within bounds. If it's picking up the higher new value from the lib it must
> > have been compiled against the lib so shouldn't have problems.
>
> You say there is no ABI issue because the application will be re-compiled
> for the updated library. Indeed, compilation fixes compatibility issues.
> But this is not relevant for ABI compatibility.
> ABI compatibility means we can upgrade the library without recompiling
> the application and it must work.
> You think it is a false positive because you assume the application
> "picks" the new value. I think you miss the case where the new value
> is returned by a function in the upgraded library.
>
> > There are also no structs on the API which contain arrays using this
> > for sizing, so I don't see an opportunity for an appl to have a
> > mismatch in memory addresses.
>
> Let me demonstrate where the API may "use" the new value
> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 and how it impacts the application.
>
> Once upon a time a DPDK application counting the number of devices
> supporting each AEAD algo (in order to find the best supported algo).
> It is done in an array indexed by algo id:
> int aead_dev_count[RTE_CRYPTO_AEAD_LIST_END];
> The application is compiled with DPDK 19.11,
> where RTE_CRYPTO_AEAD_LIST_END = 3.
> So the size of the application array aead_dev_count is 3.
> This binary is run with DPDK 20.02,
> where RTE_CRYPTO_AEAD_CHACHA20_POLY1305 = 3.
> When calling rte_cryptodev_info_get() on a device QAT_GEN3,
> rte_cryptodev_info.capabilities.sym.aead.algo is set to
> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 (= 3).
> The application uses this value:
> ++ aead_dev_count[info.capabilities.sym.aead.algo];
> The application is crashing because of out of bound access.
I'd say this is an example of bad written app.
It probably should check that returned by library value doesn't
exceed its internal array size.
Konstantin
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [RFC] meter: fix ABI break due to experimental tag removal
2020-01-29 12:29 3% [dpdk-dev] [RFC] meter: fix ABI break due to experimental tag removal Ferruh Yigit
` (2 preceding siblings ...)
2020-01-29 16:43 3% ` [dpdk-dev] [RFC v2] " Ferruh Yigit
@ 2020-02-02 18:53 7% ` Neil Horman
2020-02-03 12:53 4% ` Ferruh Yigit
3 siblings, 1 reply; 200+ results
From: Neil Horman @ 2020-02-02 18:53 UTC (permalink / raw)
To: Ferruh Yigit
Cc: Cristian Dumitrescu, Eelco Chaudron, dev, Thomas Monjalon,
Luca Boccassi, David Marchand, Bruce Richardson, Ian Stokes
On Wed, Jan 29, 2020 at 12:29:53PM +0000, Ferruh Yigit wrote:
> Duplicated the existing symbol and versioned one as experimental and
> other as stable.
>
> Created VERSION_SYMBOL_EXPERIMENTAL helper macro.
>
> Updated the 'check-experimental-syms.sh' buildtool, which was
> complaining that the symbol is in EXPERIMENTAL tag in .map file but it
> is not in the .experimental section (__rte_experimental tag is missing).
> Updated tool in a way it won't complain if the symbol in the
> EXPERIMENTAL tag duplicated in some other block in .map file (versioned)
>
> Updated meson build system to allow the versioning,
> 'use_function_versioning = true', not sure why it was disabled by
> default.
>
> Fixes: 30512af820fe ("meter: remove experimental flag from RFC4115 trTCM API")
>
I'm not sure I understand the purpose here.
I think I understand what you are trying to do. I believe that you are
trying to move the experimental symbols in librte_meter to be part of
the official ABI, while still maintaining backward compatibility with
applications built against the experimental symbol versions, correct?
I can understand the desire, and I'm not exactly opposed to providing a
mechanism for that, but it seems somewhat complex, and to be honest,
part of the drawback to using experimental symbols as an application
developer is understanding that you may need to rebuild when those
symbols change (even if the change is the symbol removal and replacement
with an identical one that has a versioned tag).
I think the right answer for people encountering this condition is to
just rebuild. Otherwise, we are creating an environment in which we are
implicitly communicating to users that we are maintaining a modicum of
stability in experimental symobls.
A few more nits in line
> Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
> ---
> Cc: Neil Horman <nhorman@tuxdriver.com>
> Cc: Thomas Monjalon <thomas@monjalon.net>
> Cc: Luca Boccassi <bluca@debian.org>
> Cc: David Marchand <david.marchand@redhat.com>
> Cc: Bruce Richardson <bruce.richardson@intel.com>
> Cc: Ian Stokes <ian.stokes@intel.com>
> Cc: Eelco Chaudron <echaudro@redhat.com>
>
> I didn't like the idea of duplicating the symbol but not able to find to
> alias it without duplicating, if there is a way please share.
>
> Also not tested with old binaries, only verified the symbols in new
> binaries.
> ---
> buildtools/check-experimental-syms.sh | 3 +-
> .../common/include/rte_function_versioning.h | 4 ++
> lib/librte_meter/rte_meter.c | 59 ++++++++++++++++++-
> lib/librte_meter/rte_meter_version.map | 8 +++
> lib/meson.build | 2 +-
> 5 files changed, 71 insertions(+), 5 deletions(-)
>
> diff --git a/buildtools/check-experimental-syms.sh b/buildtools/check-experimental-syms.sh
> index f3603e5ba..35c4bf006 100755
> --- a/buildtools/check-experimental-syms.sh
> +++ b/buildtools/check-experimental-syms.sh
> @@ -26,7 +26,8 @@ ret=0
> for SYM in `$LIST_SYMBOL -S EXPERIMENTAL $MAPFILE |cut -d ' ' -f 3`
> do
> if grep -q "\.text.*[[:space:]]$SYM$" $DUMPFILE &&
> - ! grep -q "\.text\.experimental.*[[:space:]]$SYM$" $DUMPFILE
> + ! grep -q "\.text\.experimental.*[[:space:]]$SYM$" $DUMPFILE &&
> + $LIST_SYMBOL -s $SYM $MAPFILE | grep -q EXPERIMENTAL
> then
> cat >&2 <<- END_OF_MESSAGE
> $SYM is not flagged as experimental
> diff --git a/lib/librte_eal/common/include/rte_function_versioning.h b/lib/librte_eal/common/include/rte_function_versioning.h
> index c924351d5..ab102b06e 100644
> --- a/lib/librte_eal/common/include/rte_function_versioning.h
> +++ b/lib/librte_eal/common/include/rte_function_versioning.h
> @@ -46,6 +46,9 @@
> */
> #define VERSION_SYMBOL(b, e, n) __asm__(".symver " RTE_STR(b) RTE_STR(e) ", " RTE_STR(b) "@DPDK_" RTE_STR(n))
>
> +
> +#define VERSION_SYMBOL_EXPERIMENTAL(b, e) __asm__(".symver " RTE_STR(b) RTE_STR(e) ", " RTE_STR(b) "@EXPERIMENTAL")
> +
I don't know that you want to make a explicit clone of this macro.
instead what you might want to do is something like:
#define __VERSION_SYMBOL(b, e, n, t) __asm__(".symver " RTE_STR(b) RTE_STR(e) ", " RTE_STR(b) "@" RTE_STR(t) "_" RTE_STR(n))
#define VERSION_SYMBOL(b, e, n) __VERSION_SYMBOL(b, e, n, DPDK)
Thats not exactly right, but you get the idea, then you can emulate
VERSION_SYMBOL_EXPERIMENTAL with __VERSION_SYMBOL(b, e, n, EXPERIMENTAL)
> /*
> * BIND_DEFAULT_SYMBOL
> * Creates a symbol version entry instructing the linker to bind references to
> @@ -79,6 +82,7 @@
> * No symbol versioning in use
> */
> #define VERSION_SYMBOL(b, e, n)
> +#define VERSION_SYMBOL_EXPERIMENTAL(b, e)
> #define __vsym
> #define BIND_DEFAULT_SYMBOL(b, e, n)
> #define MAP_STATIC_SYMBOL(f, p) f __attribute__((alias(RTE_STR(p))))
> diff --git a/lib/librte_meter/rte_meter.c b/lib/librte_meter/rte_meter.c
> index da01429a8..5244537fa 100644
> --- a/lib/librte_meter/rte_meter.c
> +++ b/lib/librte_meter/rte_meter.c
> @@ -9,6 +9,7 @@
> #include <rte_common.h>
> #include <rte_log.h>
> #include <rte_cycles.h>
> +#include <rte_function_versioning.h>
>
> #include "rte_meter.h"
>
> @@ -119,8 +120,8 @@ rte_meter_trtcm_config(struct rte_meter_trtcm *m,
> return 0;
> }
>
> -int
> -rte_meter_trtcm_rfc4115_profile_config(
> +static int
> +rte_meter_trtcm_rfc4115_profile_config_(
> struct rte_meter_trtcm_rfc4115_profile *p,
> struct rte_meter_trtcm_rfc4115_params *params)
> {
> @@ -145,7 +146,35 @@ rte_meter_trtcm_rfc4115_profile_config(
> }
>
> int
> -rte_meter_trtcm_rfc4115_config(
> +rte_meter_trtcm_rfc4115_profile_config_s(
> + struct rte_meter_trtcm_rfc4115_profile *p,
> + struct rte_meter_trtcm_rfc4115_params *params);
> +int
> +rte_meter_trtcm_rfc4115_profile_config_s(
> + struct rte_meter_trtcm_rfc4115_profile *p,
> + struct rte_meter_trtcm_rfc4115_params *params)
> +{
> + return rte_meter_trtcm_rfc4115_profile_config_(p, params);
> +}
> +BIND_DEFAULT_SYMBOL(rte_meter_trtcm_rfc4115_profile_config, _s, 20.0.1);
> +MAP_STATIC_SYMBOL(int rte_meter_trtcm_rfc4115_profile_config(struct rte_meter_trtcm_rfc4115_profile *p,
> + struct rte_meter_trtcm_rfc4115_params *params), rte_meter_trtcm_rfc4115_profile_config_s);
> +
> +int
> +rte_meter_trtcm_rfc4115_profile_config_e(
> + struct rte_meter_trtcm_rfc4115_profile *p,
> + struct rte_meter_trtcm_rfc4115_params *params);
> +int
> +rte_meter_trtcm_rfc4115_profile_config_e(
> + struct rte_meter_trtcm_rfc4115_profile *p,
> + struct rte_meter_trtcm_rfc4115_params *params)
> +{
> + return rte_meter_trtcm_rfc4115_profile_config_(p, params);
> +}
> +VERSION_SYMBOL_EXPERIMENTAL(rte_meter_trtcm_rfc4115_profile_config, _e);
> +
> +static int
> +rte_meter_trtcm_rfc4115_config_(
> struct rte_meter_trtcm_rfc4115 *m,
> struct rte_meter_trtcm_rfc4115_profile *p)
> {
> @@ -160,3 +189,27 @@ rte_meter_trtcm_rfc4115_config(
>
> return 0;
> }
> +
> +int
> +rte_meter_trtcm_rfc4115_config_s(struct rte_meter_trtcm_rfc4115 *m,
> + struct rte_meter_trtcm_rfc4115_profile *p);
> +int
> +rte_meter_trtcm_rfc4115_config_s(struct rte_meter_trtcm_rfc4115 *m,
> + struct rte_meter_trtcm_rfc4115_profile *p)
> +{
> + return rte_meter_trtcm_rfc4115_config_(m, p);
> +}
> +BIND_DEFAULT_SYMBOL(rte_meter_trtcm_rfc4115_config, _s, 20.0.1);
> +MAP_STATIC_SYMBOL(int rte_meter_trtcm_rfc4115_config(struct rte_meter_trtcm_rfc4115 *m,
> + struct rte_meter_trtcm_rfc4115_profile *p), rte_meter_trtcm_rfc4115_config_s);
> +
> +int
> +rte_meter_trtcm_rfc4115_config_e(struct rte_meter_trtcm_rfc4115 *m,
> + struct rte_meter_trtcm_rfc4115_profile *p);
> +int
> +rte_meter_trtcm_rfc4115_config_e(struct rte_meter_trtcm_rfc4115 *m,
> + struct rte_meter_trtcm_rfc4115_profile *p)
> +{
> + return rte_meter_trtcm_rfc4115_config_(m, p);
> +}
> +VERSION_SYMBOL_EXPERIMENTAL(rte_meter_trtcm_rfc4115_config, _e);
> diff --git a/lib/librte_meter/rte_meter_version.map b/lib/librte_meter/rte_meter_version.map
> index fc12cc0bf..b4b043174 100644
> --- a/lib/librte_meter/rte_meter_version.map
> +++ b/lib/librte_meter/rte_meter_version.map
> @@ -20,4 +20,12 @@ DPDK_20.0.1 {
> rte_meter_trtcm_rfc4115_color_blind_check;
> rte_meter_trtcm_rfc4115_config;
> rte_meter_trtcm_rfc4115_profile_config;
> +
> } DPDK_20.0;
> +
> +EXPERIMENTAL {
> + global:
> +
> + rte_meter_trtcm_rfc4115_config;
> + rte_meter_trtcm_rfc4115_profile_config;
> +};
> diff --git a/lib/meson.build b/lib/meson.build
> index 0af3efab2..553964831 100644
> --- a/lib/meson.build
> +++ b/lib/meson.build
> @@ -48,7 +48,7 @@ foreach l:libraries
> reason = '<unknown reason>' # set if build == false to explain why
> name = l
> allow_experimental_apis = false
> - use_function_versioning = false
> + use_function_versioning = true
> sources = []
> headers = []
> includes = []
> --
> 2.24.1
>
>
^ permalink raw reply [relevance 7%]
* [dpdk-dev] [PATCH v4 0/3] add ABI checks
` (2 preceding siblings ...)
2020-01-30 16:00 9% ` [dpdk-dev] [PATCH v3 " David Marchand
@ 2020-02-02 21:08 9% ` David Marchand
2020-02-02 21:08 28% ` [dpdk-dev] [PATCH v4 3/3] " David Marchand
2020-02-05 14:13 4% ` [dpdk-dev] [PATCH v4 0/3] " Thomas Monjalon
3 siblings, 2 replies; 200+ results
From: David Marchand @ 2020-02-02 21:08 UTC (permalink / raw)
To: dev
Cc: thomas, bruce.richardson, kevin.laatz, aconole, nhorman,
akhil.goyal, anoobj, bluca, fiona.trahe, ferruh.yigit
Here is the current state of the ABI checks.
libabigail has some issues when mixing dump and so files compiled with
clang [1], so for now, all checks are done on dumps only.
libabigail 1.0-rc3 in Xenial reported issues that disappear with the
version 1.2 in Bionic.
To avoid getting warnings on internal types like [2], the checks now make
use of the public headers part of a dpdk installation (patch 2 and 3 to
prepare for this).
Some internal rte_hash headers were installed by meson, so patch 1 fixes
this.
The most important point, abidiff complains on the rc1 cryptodev changes:
- Chacha20-Poly1305 AEAD support,
- ECPM and ECDSA support
A suppression rule has been put on the internal type rte_cryptodev_ops.
But other changes are an ABI breakage afaiu. I put suppression rules on
them so that we can run the checks, but some action must be taken for
20.02 if my analysis is confirmed.
Special thanks to Dodji the libabigail maintainer for helping on this
topic.
1: https://sourceware.org/bugzilla/show_bug.cgi?id=25409
2: http://inbox.dpdk.org/dev/CAJFAV8yFKoDZROX9Mkyp7pDMvXw3e7mHwxjfrcjD5ZoFB2tZ8w@mail.gmail.com/
--
David Marchand
Changelog since v3:
- removed patch 3 based on Thomas feedback,
- switched to fixed sources location for building ABI references,
Changelog since v2:
- incorporated offlist Thomas comments wrt to existing build envs (do
not change --prefix, force builds with debug enabled)
David Marchand (3):
hash: fix meson headers packaging
build: split build helper
add ABI checks
.ci/linux-build.sh | 24 ++++++++
.travis.yml | 20 +++++-
MAINTAINERS | 3 +
devtools/check-abi.sh | 59 ++++++++++++++++++
devtools/dpdk.abignore | 21 +++++++
devtools/gen-abi.sh | 26 ++++++++
devtools/test-build.sh | 52 ++++++++++++++--
devtools/test-meson-builds.sh | 96 ++++++++++++++++++++++++-----
doc/guides/contributing/patches.rst | 15 +++++
lib/librte_hash/meson.build | 5 +-
10 files changed, 293 insertions(+), 28 deletions(-)
create mode 100755 devtools/check-abi.sh
create mode 100644 devtools/dpdk.abignore
create mode 100755 devtools/gen-abi.sh
--
2.23.0
^ permalink raw reply [relevance 9%]
* [dpdk-dev] [PATCH v4 3/3] add ABI checks
2020-02-02 21:08 9% ` [dpdk-dev] [PATCH v4 0/3] " David Marchand
@ 2020-02-02 21:08 28% ` David Marchand
2020-02-05 14:13 4% ` [dpdk-dev] [PATCH v4 0/3] " Thomas Monjalon
1 sibling, 0 replies; 200+ results
From: David Marchand @ 2020-02-02 21:08 UTC (permalink / raw)
To: dev
Cc: thomas, bruce.richardson, kevin.laatz, aconole, nhorman,
akhil.goyal, anoobj, bluca, fiona.trahe, ferruh.yigit,
Michael Santana, John McNamara, Marko Kovacevic
For normal developers, those checks are disabled.
Enabling them requires a configuration that will trigger the ABI dumps
generation as part of the existing devtools/test-build.sh and
devtools/test-meson-builds.sh scripts.
Those checks are enabled in the CI for the default meson options on x86
and aarch64 so that proposed patches are validated via our CI robot.
A cache of the ABI is stored in travis jobs to avoid rebuilding too
often.
Checks can be informational only, by setting ABI_CHECKS_WARN_ONLY when
breaking the ABI in a future release.
Explicit suppression rules have been added on internal structures
exposed to crypto drivers as the current ABI policy does not apply to
them.
This could be improved in the future by carefully splitting the headers
content with application and driver "users" in mind.
We currently have issues reported for librte_crypto recent changes for
which suppression rules have been added too.
Mellanox glue libraries are explicitly skipped as they are not part of
the application ABI.
Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Luca Boccassi <bluca@debian.org>
---
Changelog since v3:
- rebased following patch 3 removal,
- added dpdk.abignore to MAINTAINERS,
- avoided meson reconfiguration when unneeded,
- switched to fixed sources location for building ABI references,
Changelog since v2:
- forced -g / buildtype=debugoptimised in the test scripts so that
we can check ABI in existing environments,
- little update on the documentation,
Changelog since v1:
- reworked the scripts so that the build test scripts clone and build the
reference automatically. A developer only needs to set one variable to
enable the checks,
- meson builds are done with debug so that abidiff can inspect the
structures,
- abidiff checks only public types by looking at installed headers,
- abidiff has some issue when comparing a dump with a .so built with clang
so all diff are now done with dump files only,
- suppression rules have been added to waive warnings on exposed internal
types,
- an abi breakage has been reported on changes in cryptodev.
For now, suppression rules have been put in place to let the CI run,
---
.ci/linux-build.sh | 24 ++++++++++++
.travis.yml | 20 +++++++++-
MAINTAINERS | 3 ++
devtools/check-abi.sh | 59 +++++++++++++++++++++++++++++
devtools/dpdk.abignore | 21 ++++++++++
devtools/gen-abi.sh | 26 +++++++++++++
devtools/test-build.sh | 52 ++++++++++++++++++++++---
devtools/test-meson-builds.sh | 50 ++++++++++++++++++++++--
doc/guides/contributing/patches.rst | 15 ++++++++
9 files changed, 260 insertions(+), 10 deletions(-)
create mode 100755 devtools/check-abi.sh
create mode 100644 devtools/dpdk.abignore
create mode 100755 devtools/gen-abi.sh
diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh
index ccc3a7ccd..c7c3840fc 100755
--- a/.ci/linux-build.sh
+++ b/.ci/linux-build.sh
@@ -29,6 +29,7 @@ if [ "$BUILD_32BIT" = "1" ]; then
fi
OPTS="$OPTS --default-library=$DEF_LIB"
+OPTS="$OPTS --buildtype=debugoptimized"
meson build --werror -Dexamples=all $OPTS
ninja -C build
@@ -36,6 +37,29 @@ if [ "$AARCH64" != "1" ]; then
devtools/test-null.sh
fi
+if [ "$ABI_CHECKS" = "1" ]; then
+ REF_GIT_REPO=${REF_GIT_REPO:-https://dpdk.org/git/dpdk}
+ REF_GIT_TAG=${REF_GIT_TAG:-v19.11}
+
+ if [ "$(cat reference/VERSION 2>/dev/null)" != "$REF_GIT_TAG" ]; then
+ rm -rf reference
+ fi
+
+ if [ ! -d reference ]; then
+ refsrcdir=$(readlink -f $(pwd)/../dpdk-$REF_GIT_TAG)
+ git clone --single-branch -b $REF_GIT_TAG $REF_GIT_REPO $refsrcdir
+ meson --werror $OPTS $refsrcdir $refsrcdir/build
+ ninja -C $refsrcdir/build
+ DESTDIR=$(pwd)/reference ninja -C $refsrcdir/build install
+ devtools/gen-abi.sh reference
+ echo $REF_GIT_TAG > reference/VERSION
+ fi
+
+ DESTDIR=$(pwd)/install ninja -C build install
+ devtools/gen-abi.sh install
+ devtools/check-abi.sh reference install ${ABI_CHECKS_WARN_ONLY:-}
+fi
+
if [ "$RUN_TESTS" = "1" ]; then
sudo meson test -C build --suite fast-tests -t 3
fi
diff --git a/.travis.yml b/.travis.yml
index 8162f1c05..22539d823 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -1,5 +1,8 @@
language: c
-cache: ccache
+cache:
+ ccache: true
+ directories:
+ - reference
compiler:
- gcc
- clang
@@ -21,7 +24,7 @@ aarch64_packages: &aarch64_packages
extra_packages: &extra_packages
- *required_packages
- - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+ - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4, abigail-tools]
build_32b_packages: &build_32b_packages
- *required_packages
@@ -151,5 +154,18 @@ matrix:
packages:
- *required_packages
- *doc_packages
+ - env: DEF_LIB="shared" EXTRA_PACKAGES=1 ABI_CHECKS=1
+ compiler: gcc
+ addons:
+ apt:
+ packages:
+ - *extra_packages
+ - env: DEF_LIB="shared" EXTRA_PACKAGES=1 ABI_CHECKS=1
+ arch: arm64
+ compiler: gcc
+ addons:
+ apt:
+ packages:
+ - *extra_packages
script: ./.ci/${TRAVIS_OS_NAME}-build.sh
diff --git a/MAINTAINERS b/MAINTAINERS
index 94bccae6d..db4b8c5b6 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -144,8 +144,11 @@ M: Neil Horman <nhorman@tuxdriver.com>
F: lib/librte_eal/common/include/rte_compat.h
F: lib/librte_eal/common/include/rte_function_versioning.h
F: doc/guides/rel_notes/deprecation.rst
+F: devtools/check-abi.sh
F: devtools/check-abi-version.sh
F: devtools/check-symbol-change.sh
+F: devtools/dpdk.abignore
+F: devtools/gen-abi.sh
F: devtools/update-abi.sh
F: devtools/update_version_map_abi.py
F: devtools/validate-abi.sh
diff --git a/devtools/check-abi.sh b/devtools/check-abi.sh
new file mode 100755
index 000000000..5872499ec
--- /dev/null
+++ b/devtools/check-abi.sh
@@ -0,0 +1,59 @@
+#!/bin/sh -e
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright (c) 2019 Red Hat, Inc.
+
+if [ $# != 2 ] && [ $# != 3 ]; then
+ echo "Usage: $0 refdir newdir [warnonly]"
+ exit 1
+fi
+
+refdir=$1
+newdir=$2
+warnonly=${3:-}
+ABIDIFF_OPTIONS="--suppr $(dirname $0)/dpdk.abignore --no-added-syms"
+
+if [ ! -d $refdir ]; then
+ echo "Error: reference directory '$refdir' does not exist."
+ exit 1
+fi
+incdir=$(find $refdir -type d -a -name include)
+if [ -z "$incdir" ] || [ ! -e "$incdir" ]; then
+ echo "WARNING: could not identify a include directory for $refdir, expect false positives..."
+else
+ ABIDIFF_OPTIONS="$ABIDIFF_OPTIONS --headers-dir1 $incdir"
+fi
+
+if [ ! -d $newdir ]; then
+ echo "Error: directory to check '$newdir' does not exist."
+ exit 1
+fi
+incdir2=$(find $newdir -type d -a -name include)
+if [ -z "$incdir2" ] || [ ! -e "$incdir2" ]; then
+ echo "WARNING: could not identify a include directory for $newdir, expect false positives..."
+else
+ ABIDIFF_OPTIONS="$ABIDIFF_OPTIONS --headers-dir2 $incdir2"
+fi
+
+error=
+for dump in $(find $refdir -name "*.dump"); do
+ name=$(basename $dump)
+ # skip glue drivers, example librte_pmd_mlx5_glue.dump
+ # We can't rely on a suppression rule for now:
+ # https://sourceware.org/bugzilla/show_bug.cgi?id=25480
+ if [ "$name" != "${name%%_glue.dump}" ]; then
+ echo "Skipping ${dump}..."
+ continue
+ fi
+ dump2=$(find $newdir -name $name)
+ if [ -z "$dump2" ] || [ ! -e "$dump2" ]; then
+ echo "Error: can't find $name in $newdir"
+ error=1
+ continue
+ fi
+ if ! abidiff $ABIDIFF_OPTIONS $dump $dump2; then
+ echo "Error: ABI issue reported for 'abidiff $ABIDIFF_OPTIONS $dump $dump2'"
+ error=1
+ fi
+done
+
+[ -z "$error" ] || [ -n "$warnonly" ]
diff --git a/devtools/dpdk.abignore b/devtools/dpdk.abignore
new file mode 100644
index 000000000..f2903612c
--- /dev/null
+++ b/devtools/dpdk.abignore
@@ -0,0 +1,21 @@
+[suppress_function]
+ symbol_version = EXPERIMENTAL
+[suppress_variable]
+ symbol_version = EXPERIMENTAL
+
+; Explicit ignore for driver-only ABI
+[suppress_type]
+ name = rte_cryptodev_ops
+; Ignore this enum update as it is part of an experimental API
+[suppress_type]
+ type_kind = enum
+ name = rte_crypto_asym_xform_type
+ changed_enumerators = RTE_CRYPTO_ASYM_XFORM_TYPE_LIST_END
+
+; FIXME
+[suppress_type]
+ type_kind = enum
+ name = rte_crypto_aead_algorithm
+ changed_enumerators = RTE_CRYPTO_AEAD_LIST_END
+[suppress_variable]
+ name = rte_crypto_aead_algorithm_strings
diff --git a/devtools/gen-abi.sh b/devtools/gen-abi.sh
new file mode 100755
index 000000000..c44b0e228
--- /dev/null
+++ b/devtools/gen-abi.sh
@@ -0,0 +1,26 @@
+#!/bin/sh -e
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright (c) 2019 Red Hat, Inc.
+
+if [ $# != 1 ]; then
+ echo "Usage: $0 installdir"
+ exit 1
+fi
+
+installdir=$1
+if [ ! -d $installdir ]; then
+ echo "Error: install directory '$installdir' does not exist."
+ exit 1
+fi
+
+dumpdir=$installdir/dump
+rm -rf $dumpdir
+mkdir -p $dumpdir
+for f in $(find $installdir -name "*.so.*"); do
+ if test -L $f; then
+ continue
+ fi
+
+ libname=$(basename $f)
+ abidw --out-file $dumpdir/${libname%.so*}.dump $f
+done
diff --git a/devtools/test-build.sh b/devtools/test-build.sh
index 52305fbb8..2f55e9147 100755
--- a/devtools/test-build.sh
+++ b/devtools/test-build.sh
@@ -6,6 +6,8 @@ default_path=$PATH
# Load config options:
# - ARMV8_CRYPTO_LIB_PATH
+# - DPDK_ABI_REF_DIR
+# - DPDK_ABI_REF_VERSION
# - DPDK_BUILD_TEST_CONFIGS (defconfig1+option1+option2 defconfig2)
# - DPDK_BUILD_TEST_DIR
# - DPDK_DEP_ARCHIVE
@@ -30,7 +32,8 @@ default_path=$PATH
# - LIBSSO_SNOW3G_PATH
# - LIBSSO_KASUMI_PATH
# - LIBSSO_ZUC_PATH
-. $(dirname $(readlink -f $0))/load-devel-config
+devtools_dir=$(dirname $(readlink -f $0))
+. $devtools_dir/load-devel-config
print_usage () {
echo "usage: $(basename $0) [-h] [-jX] [-s] [config1 [config2] ...]]"
@@ -67,7 +70,8 @@ J=$DPDK_MAKE_JOBS
builds_dir=${DPDK_BUILD_TEST_DIR:-.}
short=false
unset verbose
-maxerr=-Wfatal-errors
+# for ABI checks, we need debuginfo
+test_cflags="-Wfatal-errors -g"
while getopts hj:sv ARG ; do
case $ARG in
j ) J=$OPTARG ;;
@@ -97,7 +101,7 @@ trap "signal=INT ; trap - INT ; kill -INT $$" INT
# notify result on exit
trap on_exit EXIT
-cd $(dirname $(readlink -f $0))/..
+cd $devtools_dir/..
reset_env ()
{
@@ -233,14 +237,14 @@ for conf in $configs ; do
# reload config with DPDK_TARGET set
DPDK_TARGET=$target
reset_env
- . $(dirname $(readlink -f $0))/load-devel-config
+ . $devtools_dir/load-devel-config
options=$(echo $conf | sed 's,[^~+]*,,')
dir=$builds_dir/$conf
config $dir $target $options
echo "================== Build $conf"
- ${MAKE} -j$J EXTRA_CFLAGS="$maxerr $DPDK_DEP_CFLAGS" \
+ ${MAKE} -j$J EXTRA_CFLAGS="$test_cflags $DPDK_DEP_CFLAGS" \
EXTRA_LDFLAGS="$DPDK_DEP_LDFLAGS" $verbose O=$dir
! $short || break
export RTE_TARGET=$target
@@ -253,6 +257,44 @@ for conf in $configs ; do
EXTRA_LDFLAGS="$DPDK_DEP_LDFLAGS" $verbose \
O=$(readlink -f $dir)/examples
unset RTE_TARGET
+ if [ -n "$DPDK_ABI_REF_VERSION" ]; then
+ abirefdir=${DPDK_ABI_REF_DIR:-reference}/$DPDK_ABI_REF_VERSION
+ if [ ! -d $abirefdir/$conf ]; then
+ # clone current sources
+ if [ ! -d $abirefdir/src ]; then
+ git clone --local --no-hardlinks \
+ --single-branch \
+ -b $DPDK_ABI_REF_VERSION \
+ $(pwd) $abirefdir/src
+ fi
+
+ cd $abirefdir/src
+
+ rm -rf $abirefdir/build
+ config $abirefdir/build $target $options
+
+ echo -n "================== Build $conf "
+ echo "($DPDK_ABI_REF_VERSION)"
+ ${MAKE} -j$J \
+ EXTRA_CFLAGS="$test_cflags $DPDK_DEP_CFLAGS" \
+ EXTRA_LDFLAGS="$DPDK_DEP_LDFLAGS" $verbose \
+ O=$abirefdir/build
+ ! $short || break
+ export RTE_TARGET=$target
+ ${MAKE} install O=$abirefdir/build \
+ DESTDIR=$abirefdir/$conf \
+ prefix=
+ unset RTE_TARGET
+ $devtools_dir/gen-abi.sh $abirefdir/$conf
+
+ # back to current workdir
+ cd $devtools_dir/..
+ fi
+
+ echo "================== Check ABI $conf"
+ $devtools_dir/gen-abi.sh $dir/install
+ $devtools_dir/check-abi.sh $abirefdir/$conf $dir/install
+ fi
echo "################## $conf done."
unset dir
done
diff --git a/devtools/test-meson-builds.sh b/devtools/test-meson-builds.sh
index fb6c404e5..c2d33b3b9 100755
--- a/devtools/test-meson-builds.sh
+++ b/devtools/test-meson-builds.sh
@@ -64,9 +64,18 @@ config () # <dir> <builddir> <meson options>
builddir=$1
shift
if [ -f "$builddir/build.ninja" ] ; then
+ # for existing environments, switch to debugoptimized if unset
+ # so that ABI checks can run
+ if ! $MESON configure $builddir |
+ awk '$1=="buildtype" {print $2}' |
+ grep -qw debugoptimized; then
+ $MESON configure --buildtype=debugoptimized $builddir
+ fi
return
fi
- options="--werror -Dexamples=all"
+ options=
+ options="$options --werror -Dexamples=all"
+ options="$options --buildtype=debugoptimized"
for option in $DPDK_MESON_OPTIONS ; do
options="$options -D$option"
done
@@ -92,6 +101,13 @@ compile () # <builddir>
fi
}
+install_target () # <builddir> <installdir>
+{
+ rm -rf $2
+ echo "DESTDIR=$2 $ninja_cmd -C $1 install"
+ DESTDIR=$2 $ninja_cmd -C $1 install
+}
+
build () # <directory> <target compiler> <meson options>
{
targetdir=$1
@@ -103,6 +119,31 @@ build () # <directory> <target compiler> <meson options>
load_env $targetcc || return 0
config $srcdir $builds_dir/$targetdir $*
compile $builds_dir/$targetdir
+ if [ -n "$DPDK_ABI_REF_VERSION" ]; then
+ abirefdir=${DPDK_ABI_REF_DIR:-reference}/$DPDK_ABI_REF_VERSION
+ if [ ! -d $abirefdir/$targetdir ]; then
+ # clone current sources
+ if [ ! -d $abirefdir/src ]; then
+ git clone --local --no-hardlinks \
+ --single-branch \
+ -b $DPDK_ABI_REF_VERSION \
+ $srcdir $abirefdir/src
+ fi
+
+ rm -rf $abirefdir/build
+ config $abirefdir/src $abirefdir/build $*
+ compile $abirefdir/build
+ install_target $abirefdir/build $abirefdir/$targetdir
+ $srcdir/devtools/gen-abi.sh $abirefdir/$targetdir
+ fi
+
+ install_target $builds_dir/$targetdir \
+ $(readlink -f $builds_dir/$targetdir/install)
+ $srcdir/devtools/gen-abi.sh \
+ $(readlink -f $builds_dir/$targetdir/install)
+ $srcdir/devtools/check-abi.sh $abirefdir/$targetdir \
+ $(readlink -f $builds_dir/$targetdir/install)
+ fi
}
if [ "$1" = "-vv" ] ; then
@@ -153,8 +194,11 @@ done
# Test installation of the x86-default target, to be used for checking
# the sample apps build using the pkg-config file for cflags and libs
build_path=$(readlink -f $builds_dir/build-x86-default)
-export DESTDIR=$build_path/install-root
-$ninja_cmd -C $build_path install
+export DESTDIR=$build_path/install
+# No need to reinstall if ABI checks are enabled
+if [ -z "$DPDK_ABI_REF_VERSION" ]; then
+ install_target $build_path $DESTDIR
+fi
load_env cc
pc_file=$(find $DESTDIR -name libdpdk.pc)
diff --git a/doc/guides/contributing/patches.rst b/doc/guides/contributing/patches.rst
index 0686450e4..59442824a 100644
--- a/doc/guides/contributing/patches.rst
+++ b/doc/guides/contributing/patches.rst
@@ -513,6 +513,21 @@ in a single subfolder called "__builds" created in the current directory.
Setting ``DPDK_BUILD_TEST_DIR`` to an absolute directory path e.g. ``/tmp`` is also supported.
+Checking ABI compatibility
+--------------------------
+
+By default, ABI compatibility checks are disabled.
+
+To enable them, a reference version must be selected via the environment
+variable ``DPDK_ABI_REF_VERSION``.
+
+The ``devtools/test-build.sh`` and ``devtools/test-meson-builds.sh`` scripts
+then build this reference version in a temporary directory and store the
+results in a subfolder of the current working directory.
+The environment variable ``DPDK_ABI_REF_DIR`` can be set so that the results go
+to a different location.
+
+
Sending Patches
---------------
--
2.23.0
^ permalink raw reply [relevance 28%]
* Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
2020-02-02 14:41 4% ` Ananyev, Konstantin
@ 2020-02-03 9:30 7% ` Ferruh Yigit
2020-02-03 11:50 9% ` Neil Horman
2020-02-03 17:09 9% ` Thomas Monjalon
0 siblings, 2 replies; 200+ results
From: Ferruh Yigit @ 2020-02-03 9:30 UTC (permalink / raw)
To: Ananyev, Konstantin, Thomas Monjalon, Akhil Goyal, Trahe, Fiona
Cc: David Marchand, Anoob Joseph, Kusztal, ArkadiuszX, dev,
Richardson, Bruce, nhorman, Mcnamara, John, dodji,
Andrew Rybchenko, aconole
On 2/2/2020 2:41 PM, Ananyev, Konstantin wrote:
>
>> 31/01/2020 15:16, Trahe, Fiona:
>>> On 1/30/2020 8:18 PM, Thomas Monjalon wrote:
>>>> 30/01/2020 17:09, Ferruh Yigit:
>>>>> On 1/29/2020 8:13 PM, Akhil Goyal wrote:
>>>>>>
>>>>>> I believe these enums will be used only in case of ASYM case which is experimental.
>>>>>
>>>>> Independent from being experiment and not, this shouldn't be a problem, I think
>>>>> this is a false positive.
>>>>>
>>>>> The ABI break can happen when a struct has been shared between the application
>>>>> and the library (DPDK) and the layout of that memory know differently by
>>>>> application and the library.
>>>>>
>>>>> Here in all cases, there is no layout/size change.
>>>>>
>>>>> As to the value changes of the enums, since application compiled with old DPDK,
>>>>> it will know only up to '6', 7 and more means invalid to the application. So it
>>>>> won't send these values also it should ignore these values from library. Only
>>>>> consequence is old application won't able to use new features those new enums
>>>>> provide but that is expected/normal.
>>>>
>>>> If library give higher value than expected by the application,
>>>> if the application uses this value as array index,
>>>> there can be an access out of bounds.
>>>
>>> [Fiona] All asymmetric APIs are experimental so above shouldn't be a problem.
>>> But for the same issue with sym crypto below, I believe Ferruh's explanation makes
>>> sense and I don't see how there can be an API breakage.
>>> So if an application hasn't compiled against the new lib it will be still using the old value
>>> which will be within bounds. If it's picking up the higher new value from the lib it must
>>> have been compiled against the lib so shouldn't have problems.
>>
>> You say there is no ABI issue because the application will be re-compiled
>> for the updated library. Indeed, compilation fixes compatibility issues.
>> But this is not relevant for ABI compatibility.
>> ABI compatibility means we can upgrade the library without recompiling
>> the application and it must work.
>> You think it is a false positive because you assume the application
>> "picks" the new value. I think you miss the case where the new value
>> is returned by a function in the upgraded library.
>>
>>> There are also no structs on the API which contain arrays using this
>>> for sizing, so I don't see an opportunity for an appl to have a
>>> mismatch in memory addresses.
>>
>> Let me demonstrate where the API may "use" the new value
>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 and how it impacts the application.
>>
>> Once upon a time a DPDK application counting the number of devices
>> supporting each AEAD algo (in order to find the best supported algo).
>> It is done in an array indexed by algo id:
>> int aead_dev_count[RTE_CRYPTO_AEAD_LIST_END];
>> The application is compiled with DPDK 19.11,
>> where RTE_CRYPTO_AEAD_LIST_END = 3.
>> So the size of the application array aead_dev_count is 3.
>> This binary is run with DPDK 20.02,
>> where RTE_CRYPTO_AEAD_CHACHA20_POLY1305 = 3.
>> When calling rte_cryptodev_info_get() on a device QAT_GEN3,
>> rte_cryptodev_info.capabilities.sym.aead.algo is set to
>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 (= 3).
>> The application uses this value:
>> ++ aead_dev_count[info.capabilities.sym.aead.algo];
>> The application is crashing because of out of bound access.
>
> I'd say this is an example of bad written app.
> It probably should check that returned by library value doesn't
> exceed its internal array size.
+1
Application should ignore values >= MAX.
Do you suggest we don't extend any enum or define between ABI breakage releases
to be sure bad written applications not affected?
^ permalink raw reply [relevance 7%]
* Re: [dpdk-dev] [RFC v2] meter: fix ABI break due to experimental tag removal
2020-01-30 20:20 4% ` Thomas Monjalon
@ 2020-02-03 11:10 4% ` Eelco Chaudron
0 siblings, 0 replies; 200+ results
From: Eelco Chaudron @ 2020-02-03 11:10 UTC (permalink / raw)
To: Thomas Monjalon
Cc: Luca Boccassi, Ferruh Yigit, Neil Horman, Cristian Dumitrescu,
dev, David Marchand, Bruce Richardson, Ian Stokes,
Andrzej Ostruszka
On 30 Jan 2020, at 21:20, Thomas Monjalon wrote:
<SNIP>
>>>>
>>>> Yes, sorry, this is what I meant.
>>>> Given 19.11.0 already has the symbol as experimental,
>>>> and that applications like OVS had to accept it as experimental,
>>>> why removing experimental tag in 19.11.1?
>>>
>>> I think it was mentioned that it was preferred not to suppress the
>>> compiler warning to avoid any accidental use in the future, but the
>>> OVS
>>> maintainer(s) should answer as I might remember wrongly.
>>
>> Yes this is the reason, OVS compiles with -Werror so we would like to
>> avoid the warnings. You can not disable them per include, it’s
>> global
>> for all of DPDK.
>
> Yes but anyway OVS must accept the experimental function as the next
> release
> will use it with DPDK 19.11.0.
Yes, we do for now, but we would like to remove it ASAP as it opens up
the code base for the experimental APIs to slip in…
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
2020-02-03 9:30 7% ` Ferruh Yigit
@ 2020-02-03 11:50 9% ` Neil Horman
2020-02-03 13:09 7% ` Ferruh Yigit
2020-02-03 17:09 9% ` Thomas Monjalon
1 sibling, 1 reply; 200+ results
From: Neil Horman @ 2020-02-03 11:50 UTC (permalink / raw)
To: Ferruh Yigit
Cc: Ananyev, Konstantin, Thomas Monjalon, Akhil Goyal, Trahe, Fiona,
David Marchand, Anoob Joseph, Kusztal, ArkadiuszX, dev,
Richardson, Bruce, Mcnamara, John, dodji, Andrew Rybchenko,
aconole
On Mon, Feb 03, 2020 at 09:30:06AM +0000, Ferruh Yigit wrote:
> On 2/2/2020 2:41 PM, Ananyev, Konstantin wrote:
> >
> >> 31/01/2020 15:16, Trahe, Fiona:
> >>> On 1/30/2020 8:18 PM, Thomas Monjalon wrote:
> >>>> 30/01/2020 17:09, Ferruh Yigit:
> >>>>> On 1/29/2020 8:13 PM, Akhil Goyal wrote:
> >>>>>>
> >>>>>> I believe these enums will be used only in case of ASYM case which is experimental.
> >>>>>
> >>>>> Independent from being experiment and not, this shouldn't be a problem, I think
> >>>>> this is a false positive.
> >>>>>
> >>>>> The ABI break can happen when a struct has been shared between the application
> >>>>> and the library (DPDK) and the layout of that memory know differently by
> >>>>> application and the library.
> >>>>>
> >>>>> Here in all cases, there is no layout/size change.
> >>>>>
> >>>>> As to the value changes of the enums, since application compiled with old DPDK,
> >>>>> it will know only up to '6', 7 and more means invalid to the application. So it
> >>>>> won't send these values also it should ignore these values from library. Only
> >>>>> consequence is old application won't able to use new features those new enums
> >>>>> provide but that is expected/normal.
> >>>>
> >>>> If library give higher value than expected by the application,
> >>>> if the application uses this value as array index,
> >>>> there can be an access out of bounds.
> >>>
> >>> [Fiona] All asymmetric APIs are experimental so above shouldn't be a problem.
> >>> But for the same issue with sym crypto below, I believe Ferruh's explanation makes
> >>> sense and I don't see how there can be an API breakage.
> >>> So if an application hasn't compiled against the new lib it will be still using the old value
> >>> which will be within bounds. If it's picking up the higher new value from the lib it must
> >>> have been compiled against the lib so shouldn't have problems.
> >>
> >> You say there is no ABI issue because the application will be re-compiled
> >> for the updated library. Indeed, compilation fixes compatibility issues.
> >> But this is not relevant for ABI compatibility.
> >> ABI compatibility means we can upgrade the library without recompiling
> >> the application and it must work.
> >> You think it is a false positive because you assume the application
> >> "picks" the new value. I think you miss the case where the new value
> >> is returned by a function in the upgraded library.
> >>
> >>> There are also no structs on the API which contain arrays using this
> >>> for sizing, so I don't see an opportunity for an appl to have a
> >>> mismatch in memory addresses.
> >>
> >> Let me demonstrate where the API may "use" the new value
> >> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 and how it impacts the application.
> >>
> >> Once upon a time a DPDK application counting the number of devices
> >> supporting each AEAD algo (in order to find the best supported algo).
> >> It is done in an array indexed by algo id:
> >> int aead_dev_count[RTE_CRYPTO_AEAD_LIST_END];
> >> The application is compiled with DPDK 19.11,
> >> where RTE_CRYPTO_AEAD_LIST_END = 3.
> >> So the size of the application array aead_dev_count is 3.
> >> This binary is run with DPDK 20.02,
> >> where RTE_CRYPTO_AEAD_CHACHA20_POLY1305 = 3.
> >> When calling rte_cryptodev_info_get() on a device QAT_GEN3,
> >> rte_cryptodev_info.capabilities.sym.aead.algo is set to
> >> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 (= 3).
> >> The application uses this value:
> >> ++ aead_dev_count[info.capabilities.sym.aead.algo];
> >> The application is crashing because of out of bound access.
> >
> > I'd say this is an example of bad written app.
> > It probably should check that returned by library value doesn't
> > exceed its internal array size.
>
> +1
>
> Application should ignore values >= MAX.
>
The example is still somewhat valid in it general principle though. While
extending an ennumeration may be flagged by libabigail as an ABI breakage, its
not necessecarily a false positive. By extending the ennumeration, all the
previous entries in an array defined by said ennumeration remain constant in
their offsets, so you can 'get away with such a change' in terms of preserving
backwards compatibility in the above example, but you cannot, for example,
shuffle the values in the ennumeration, as doing so would cause a functional
breakage (i.e. requesting an instance of RTE_CRYPTO_AEAD_CHACHA20_POLY1305 might
instead give you an instance of RTE_CRYPTO_AEAD_AES_GCM.
These sorts of changes are the type that we could collectively waive in terms of
ABI checking, as they should be ok, but the errors from libabigail should be
taken as an indicator that this API could be rewritten (for example by removing
the abi entirely, and adding an API call that returns an array of instance name
and ids), so that changes of the above sort arent required.
> Do you suggest we don't extend any enum or define between ABI breakage releases
> to be sure bad written applications not affected?
>
As noted above, we could waive such corner cases, and probably be fine, but the
error from the ABI check still serves a valid purpose in that its an indicator
that your library API is ABI sensitive to code changes that re-architecture may
address
Neil
^ permalink raw reply [relevance 9%]
* Re: [dpdk-dev] [RFC] meter: fix ABI break due to experimental tag removal
2020-02-02 18:53 7% ` [dpdk-dev] [RFC] " Neil Horman
@ 2020-02-03 12:53 4% ` Ferruh Yigit
2020-02-04 12:02 9% ` Neil Horman
0 siblings, 1 reply; 200+ results
From: Ferruh Yigit @ 2020-02-03 12:53 UTC (permalink / raw)
To: Neil Horman
Cc: Cristian Dumitrescu, Eelco Chaudron, dev, Thomas Monjalon,
Luca Boccassi, David Marchand, Bruce Richardson, Ian Stokes
On 2/2/2020 6:53 PM, Neil Horman wrote:
> On Wed, Jan 29, 2020 at 12:29:53PM +0000, Ferruh Yigit wrote:
>> Duplicated the existing symbol and versioned one as experimental and
>> other as stable.
>>
>> Created VERSION_SYMBOL_EXPERIMENTAL helper macro.
>>
>> Updated the 'check-experimental-syms.sh' buildtool, which was
>> complaining that the symbol is in EXPERIMENTAL tag in .map file but it
>> is not in the .experimental section (__rte_experimental tag is missing).
>> Updated tool in a way it won't complain if the symbol in the
>> EXPERIMENTAL tag duplicated in some other block in .map file (versioned)
>>
>> Updated meson build system to allow the versioning,
>> 'use_function_versioning = true', not sure why it was disabled by
>> default.
>>
>> Fixes: 30512af820fe ("meter: remove experimental flag from RFC4115 trTCM API")
>>
> I'm not sure I understand the purpose here.
>
> I think I understand what you are trying to do. I believe that you are
> trying to move the experimental symbols in librte_meter to be part of
> the official ABI, while still maintaining backward compatibility with
> applications built against the experimental symbol versions, correct?
Yes, exactly.
>
> I can understand the desire, and I'm not exactly opposed to providing a
> mechanism for that, but it seems somewhat complex, and to be honest,
> part of the drawback to using experimental symbols as an application
> developer is understanding that you may need to rebuild when those
> symbols change (even if the change is the symbol removal and replacement
> with an identical one that has a versioned tag).
>
> I think the right answer for people encountering this condition is to
> just rebuild. Otherwise, we are creating an environment in which we are
> implicitly communicating to users that we are maintaining a modicum of
> stability in experimental symobls.
This will mean, even the the API was perfect it will force its users to
recompile (and re-package, re-deploy etc..) at least once, this is feeling like
we are still breaking user applications easily.
And this may create a stronger motivation by applications not to use
experimental APIs, I can't decide if this is a good thing or bad thing.
The issue is amplified in the LTS,
If we remove experimental tag in LTS, will be breaking the apps using that
experimental API, just to mature the API.
If we keep it as experimental, the API will be mature in main repo, but the LTS
has to keep exact same API as experimental up to two years.
But if we can do the versioning in the master, LTS can backport it and can have
mature version of that API in LTS without breaking the existing users.
>
> A few more nits in line
>
>> Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
>> ---
>> Cc: Neil Horman <nhorman@tuxdriver.com>
>> Cc: Thomas Monjalon <thomas@monjalon.net>
>> Cc: Luca Boccassi <bluca@debian.org>
>> Cc: David Marchand <david.marchand@redhat.com>
>> Cc: Bruce Richardson <bruce.richardson@intel.com>
>> Cc: Ian Stokes <ian.stokes@intel.com>
>> Cc: Eelco Chaudron <echaudro@redhat.com>
>>
>> I didn't like the idea of duplicating the symbol but not able to find to
>> alias it without duplicating, if there is a way please share.
>>
>> Also not tested with old binaries, only verified the symbols in new
>> binaries.
>> ---
>> buildtools/check-experimental-syms.sh | 3 +-
>> .../common/include/rte_function_versioning.h | 4 ++
>> lib/librte_meter/rte_meter.c | 59 ++++++++++++++++++-
>> lib/librte_meter/rte_meter_version.map | 8 +++
>> lib/meson.build | 2 +-
>> 5 files changed, 71 insertions(+), 5 deletions(-)
>>
>> diff --git a/buildtools/check-experimental-syms.sh b/buildtools/check-experimental-syms.sh
>> index f3603e5ba..35c4bf006 100755
>> --- a/buildtools/check-experimental-syms.sh
>> +++ b/buildtools/check-experimental-syms.sh
>> @@ -26,7 +26,8 @@ ret=0
>> for SYM in `$LIST_SYMBOL -S EXPERIMENTAL $MAPFILE |cut -d ' ' -f 3`
>> do
>> if grep -q "\.text.*[[:space:]]$SYM$" $DUMPFILE &&
>> - ! grep -q "\.text\.experimental.*[[:space:]]$SYM$" $DUMPFILE
>> + ! grep -q "\.text\.experimental.*[[:space:]]$SYM$" $DUMPFILE &&
>> + $LIST_SYMBOL -s $SYM $MAPFILE | grep -q EXPERIMENTAL
>> then
>> cat >&2 <<- END_OF_MESSAGE
>> $SYM is not flagged as experimental
>> diff --git a/lib/librte_eal/common/include/rte_function_versioning.h b/lib/librte_eal/common/include/rte_function_versioning.h
>> index c924351d5..ab102b06e 100644
>> --- a/lib/librte_eal/common/include/rte_function_versioning.h
>> +++ b/lib/librte_eal/common/include/rte_function_versioning.h
>> @@ -46,6 +46,9 @@
>> */
>> #define VERSION_SYMBOL(b, e, n) __asm__(".symver " RTE_STR(b) RTE_STR(e) ", " RTE_STR(b) "@DPDK_" RTE_STR(n))
>>
>> +
>> +#define VERSION_SYMBOL_EXPERIMENTAL(b, e) __asm__(".symver " RTE_STR(b) RTE_STR(e) ", " RTE_STR(b) "@EXPERIMENTAL")
>> +
> I don't know that you want to make a explicit clone of this macro.
> instead what you might want to do is something like:
>
> #define __VERSION_SYMBOL(b, e, n, t) __asm__(".symver " RTE_STR(b) RTE_STR(e) ", " RTE_STR(b) "@" RTE_STR(t) "_" RTE_STR(n))
>
> #define VERSION_SYMBOL(b, e, n) __VERSION_SYMBOL(b, e, n, DPDK)
>
> Thats not exactly right, but you get the idea, then you can emulate
> VERSION_SYMBOL_EXPERIMENTAL with __VERSION_SYMBOL(b, e, n, EXPERIMENTAL)
+1
>
>
>> /*
>> * BIND_DEFAULT_SYMBOL
>> * Creates a symbol version entry instructing the linker to bind references to
>> @@ -79,6 +82,7 @@
>> * No symbol versioning in use
>> */
>> #define VERSION_SYMBOL(b, e, n)
>> +#define VERSION_SYMBOL_EXPERIMENTAL(b, e)
>> #define __vsym
>> #define BIND_DEFAULT_SYMBOL(b, e, n)
>> #define MAP_STATIC_SYMBOL(f, p) f __attribute__((alias(RTE_STR(p))))
>> diff --git a/lib/librte_meter/rte_meter.c b/lib/librte_meter/rte_meter.c
>> index da01429a8..5244537fa 100644
>> --- a/lib/librte_meter/rte_meter.c
>> +++ b/lib/librte_meter/rte_meter.c
>> @@ -9,6 +9,7 @@
>> #include <rte_common.h>
>> #include <rte_log.h>
>> #include <rte_cycles.h>
>> +#include <rte_function_versioning.h>
>>
>> #include "rte_meter.h"
>>
>> @@ -119,8 +120,8 @@ rte_meter_trtcm_config(struct rte_meter_trtcm *m,
>> return 0;
>> }
>>
>> -int
>> -rte_meter_trtcm_rfc4115_profile_config(
>> +static int
>> +rte_meter_trtcm_rfc4115_profile_config_(
>> struct rte_meter_trtcm_rfc4115_profile *p,
>> struct rte_meter_trtcm_rfc4115_params *params)
>> {
>> @@ -145,7 +146,35 @@ rte_meter_trtcm_rfc4115_profile_config(
>> }
>>
>> int
>> -rte_meter_trtcm_rfc4115_config(
>> +rte_meter_trtcm_rfc4115_profile_config_s(
>> + struct rte_meter_trtcm_rfc4115_profile *p,
>> + struct rte_meter_trtcm_rfc4115_params *params);
>> +int
>> +rte_meter_trtcm_rfc4115_profile_config_s(
>> + struct rte_meter_trtcm_rfc4115_profile *p,
>> + struct rte_meter_trtcm_rfc4115_params *params)
>> +{
>> + return rte_meter_trtcm_rfc4115_profile_config_(p, params);
>> +}
>> +BIND_DEFAULT_SYMBOL(rte_meter_trtcm_rfc4115_profile_config, _s, 20.0.1);
>> +MAP_STATIC_SYMBOL(int rte_meter_trtcm_rfc4115_profile_config(struct rte_meter_trtcm_rfc4115_profile *p,
>> + struct rte_meter_trtcm_rfc4115_params *params), rte_meter_trtcm_rfc4115_profile_config_s);
>> +
>> +int
>> +rte_meter_trtcm_rfc4115_profile_config_e(
>> + struct rte_meter_trtcm_rfc4115_profile *p,
>> + struct rte_meter_trtcm_rfc4115_params *params);
>> +int
>> +rte_meter_trtcm_rfc4115_profile_config_e(
>> + struct rte_meter_trtcm_rfc4115_profile *p,
>> + struct rte_meter_trtcm_rfc4115_params *params)
>> +{
>> + return rte_meter_trtcm_rfc4115_profile_config_(p, params);
>> +}
>> +VERSION_SYMBOL_EXPERIMENTAL(rte_meter_trtcm_rfc4115_profile_config, _e);
>> +
>> +static int
>> +rte_meter_trtcm_rfc4115_config_(
>> struct rte_meter_trtcm_rfc4115 *m,
>> struct rte_meter_trtcm_rfc4115_profile *p)
>> {
>> @@ -160,3 +189,27 @@ rte_meter_trtcm_rfc4115_config(
>>
>> return 0;
>> }
>> +
>> +int
>> +rte_meter_trtcm_rfc4115_config_s(struct rte_meter_trtcm_rfc4115 *m,
>> + struct rte_meter_trtcm_rfc4115_profile *p);
>> +int
>> +rte_meter_trtcm_rfc4115_config_s(struct rte_meter_trtcm_rfc4115 *m,
>> + struct rte_meter_trtcm_rfc4115_profile *p)
>> +{
>> + return rte_meter_trtcm_rfc4115_config_(m, p);
>> +}
>> +BIND_DEFAULT_SYMBOL(rte_meter_trtcm_rfc4115_config, _s, 20.0.1);
>> +MAP_STATIC_SYMBOL(int rte_meter_trtcm_rfc4115_config(struct rte_meter_trtcm_rfc4115 *m,
>> + struct rte_meter_trtcm_rfc4115_profile *p), rte_meter_trtcm_rfc4115_config_s);
>> +
>> +int
>> +rte_meter_trtcm_rfc4115_config_e(struct rte_meter_trtcm_rfc4115 *m,
>> + struct rte_meter_trtcm_rfc4115_profile *p);
>> +int
>> +rte_meter_trtcm_rfc4115_config_e(struct rte_meter_trtcm_rfc4115 *m,
>> + struct rte_meter_trtcm_rfc4115_profile *p)
>> +{
>> + return rte_meter_trtcm_rfc4115_config_(m, p);
>> +}
>> +VERSION_SYMBOL_EXPERIMENTAL(rte_meter_trtcm_rfc4115_config, _e);
>> diff --git a/lib/librte_meter/rte_meter_version.map b/lib/librte_meter/rte_meter_version.map
>> index fc12cc0bf..b4b043174 100644
>> --- a/lib/librte_meter/rte_meter_version.map
>> +++ b/lib/librte_meter/rte_meter_version.map
>> @@ -20,4 +20,12 @@ DPDK_20.0.1 {
>> rte_meter_trtcm_rfc4115_color_blind_check;
>> rte_meter_trtcm_rfc4115_config;
>> rte_meter_trtcm_rfc4115_profile_config;
>> +
>> } DPDK_20.0;
>> +
>> +EXPERIMENTAL {
>> + global:
>> +
>> + rte_meter_trtcm_rfc4115_config;
>> + rte_meter_trtcm_rfc4115_profile_config;
>> +};
>> diff --git a/lib/meson.build b/lib/meson.build
>> index 0af3efab2..553964831 100644
>> --- a/lib/meson.build
>> +++ b/lib/meson.build
>> @@ -48,7 +48,7 @@ foreach l:libraries
>> reason = '<unknown reason>' # set if build == false to explain why
>> name = l
>> allow_experimental_apis = false
>> - use_function_versioning = false
>> + use_function_versioning = true
>> sources = []
>> headers = []
>> includes = []
>> --
>> 2.24.1
>>
>>
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
2020-02-03 11:50 9% ` Neil Horman
@ 2020-02-03 13:09 7% ` Ferruh Yigit
2020-02-03 14:00 7% ` Dodji Seketeli
2020-02-03 15:08 7% ` Trahe, Fiona
0 siblings, 2 replies; 200+ results
From: Ferruh Yigit @ 2020-02-03 13:09 UTC (permalink / raw)
To: Neil Horman
Cc: Ananyev, Konstantin, Thomas Monjalon, Akhil Goyal, Trahe, Fiona,
David Marchand, Anoob Joseph, Kusztal, ArkadiuszX, dev,
Richardson, Bruce, Mcnamara, John, dodji, Andrew Rybchenko,
aconole
On 2/3/2020 11:50 AM, Neil Horman wrote:
> On Mon, Feb 03, 2020 at 09:30:06AM +0000, Ferruh Yigit wrote:
>> On 2/2/2020 2:41 PM, Ananyev, Konstantin wrote:
>>>
>>>> 31/01/2020 15:16, Trahe, Fiona:
>>>>> On 1/30/2020 8:18 PM, Thomas Monjalon wrote:
>>>>>> 30/01/2020 17:09, Ferruh Yigit:
>>>>>>> On 1/29/2020 8:13 PM, Akhil Goyal wrote:
>>>>>>>>
>>>>>>>> I believe these enums will be used only in case of ASYM case which is experimental.
>>>>>>>
>>>>>>> Independent from being experiment and not, this shouldn't be a problem, I think
>>>>>>> this is a false positive.
>>>>>>>
>>>>>>> The ABI break can happen when a struct has been shared between the application
>>>>>>> and the library (DPDK) and the layout of that memory know differently by
>>>>>>> application and the library.
>>>>>>>
>>>>>>> Here in all cases, there is no layout/size change.
>>>>>>>
>>>>>>> As to the value changes of the enums, since application compiled with old DPDK,
>>>>>>> it will know only up to '6', 7 and more means invalid to the application. So it
>>>>>>> won't send these values also it should ignore these values from library. Only
>>>>>>> consequence is old application won't able to use new features those new enums
>>>>>>> provide but that is expected/normal.
>>>>>>
>>>>>> If library give higher value than expected by the application,
>>>>>> if the application uses this value as array index,
>>>>>> there can be an access out of bounds.
>>>>>
>>>>> [Fiona] All asymmetric APIs are experimental so above shouldn't be a problem.
>>>>> But for the same issue with sym crypto below, I believe Ferruh's explanation makes
>>>>> sense and I don't see how there can be an API breakage.
>>>>> So if an application hasn't compiled against the new lib it will be still using the old value
>>>>> which will be within bounds. If it's picking up the higher new value from the lib it must
>>>>> have been compiled against the lib so shouldn't have problems.
>>>>
>>>> You say there is no ABI issue because the application will be re-compiled
>>>> for the updated library. Indeed, compilation fixes compatibility issues.
>>>> But this is not relevant for ABI compatibility.
>>>> ABI compatibility means we can upgrade the library without recompiling
>>>> the application and it must work.
>>>> You think it is a false positive because you assume the application
>>>> "picks" the new value. I think you miss the case where the new value
>>>> is returned by a function in the upgraded library.
>>>>
>>>>> There are also no structs on the API which contain arrays using this
>>>>> for sizing, so I don't see an opportunity for an appl to have a
>>>>> mismatch in memory addresses.
>>>>
>>>> Let me demonstrate where the API may "use" the new value
>>>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 and how it impacts the application.
>>>>
>>>> Once upon a time a DPDK application counting the number of devices
>>>> supporting each AEAD algo (in order to find the best supported algo).
>>>> It is done in an array indexed by algo id:
>>>> int aead_dev_count[RTE_CRYPTO_AEAD_LIST_END];
>>>> The application is compiled with DPDK 19.11,
>>>> where RTE_CRYPTO_AEAD_LIST_END = 3.
>>>> So the size of the application array aead_dev_count is 3.
>>>> This binary is run with DPDK 20.02,
>>>> where RTE_CRYPTO_AEAD_CHACHA20_POLY1305 = 3.
>>>> When calling rte_cryptodev_info_get() on a device QAT_GEN3,
>>>> rte_cryptodev_info.capabilities.sym.aead.algo is set to
>>>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 (= 3).
>>>> The application uses this value:
>>>> ++ aead_dev_count[info.capabilities.sym.aead.algo];
>>>> The application is crashing because of out of bound access.
>>>
>>> I'd say this is an example of bad written app.
>>> It probably should check that returned by library value doesn't
>>> exceed its internal array size.
>>
>> +1
>>
>> Application should ignore values >= MAX.
>>
> The example is still somewhat valid in it general principle though. While
> extending an ennumeration may be flagged by libabigail as an ABI breakage, its
> not necessecarily a false positive. By extending the ennumeration, all the
> previous entries in an array defined by said ennumeration remain constant in
> their offsets, so you can 'get away with such a change' in terms of preserving
> backwards compatibility in the above example, but you cannot, for example,
> shuffle the values in the ennumeration, as doing so would cause a functional
> breakage (i.e. requesting an instance of RTE_CRYPTO_AEAD_CHACHA20_POLY1305 might
> instead give you an instance of RTE_CRYPTO_AEAD_AES_GCM.
+1 the change/shuffle of the existing values are problematic, but we don't have
it in this case.
>
> These sorts of changes are the type that we could collectively waive in terms of
> ABI checking, as they should be ok, but the errors from libabigail should be
> taken as an indicator that this API could be rewritten (for example by removing
> the abi entirely, and adding an API call that returns an array of instance name
> and ids), so that changes of the above sort arent required.
We can spend more time on it, but I can't see for now how to escape returning
enumaration as indication of type, and this looks legitimate sage as long as
other side verifies the received value is valid in the type range.
>
>
>> Do you suggest we don't extend any enum or define between ABI breakage releases
>> to be sure bad written applications not affected?
>>
> As noted above, we could waive such corner cases, and probably be fine, but the
> error from the ABI check still serves a valid purpose in that its an indicator
> that your library API is ABI sensitive to code changes that re-architecture may
> address
>
The concern is when there are cases we can waive, we can't directly rely on the
tool and automate it. These indicators good for improving the code, but not good
to use it as build time checker.
Is there any way to reduce the failure only to definite ABI breakages?
^ permalink raw reply [relevance 7%]
* Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
2020-02-03 14:00 7% ` Dodji Seketeli
@ 2020-02-03 14:46 4% ` Ferruh Yigit
0 siblings, 0 replies; 200+ results
From: Ferruh Yigit @ 2020-02-03 14:46 UTC (permalink / raw)
To: Dodji Seketeli
Cc: Neil Horman, Ananyev, Konstantin, Thomas Monjalon, Akhil Goyal,
Trahe, Fiona, David Marchand, Anoob Joseph, Kusztal, ArkadiuszX,
dev, Richardson, Bruce, Mcnamara, John, Andrew Rybchenko,
aconole
On 2/3/2020 2:00 PM, Dodji Seketeli wrote:
> Hello,
>
> Ferruh Yigit <ferruh.yigit@intel.com> a écrit:
>
> [...]
>
>> +1 the change/shuffle of the existing values are problematic, but we don't have
>> it in this case.
>
> Right.
>
> [...]
>
>> The concern is when there are cases we can waive, we can't directly rely on the
>> tool and automate it. These indicators good for improving the code, but not good
>> to use it as build time checker.
>
> Well, it depends. The tooling as it is today have the capability to
> automatically "waive" some classes of A{B,P}I change reports that you
> guys (the developers) deem harmless, in the context of your project.
>
> For instance, in the precise case of interest here, one could define a
> "suppression specification" to teach the ABI verifier that, for the enum
> rte_crypto_asym_xform_type, the only enumerator which numerical value is
> allowed to change is the one named RTE_CRYPTO_ASYM_XFORM_TYPE_LIST_END.
>
> The content of the suppression specification file would look like:
>
> [suppress_type]
> # So, in practise, this rule is to allow adding enumerators
> # only to the of the the rte_crypto_asym_xform_type enum,
> # right before the RTE_CRYPTO_ASYM_XFORM_TYPE_LIST_END
> # enumerator which is meant to always be the last enumerator.
> type_kind = enum
> name = rte_crypto_asym_xform_type
> changed_enumerators = RTE_CRYPTO_ASYM_XFORM_TYPE_LIST_END
>
> This way, you can hopefully reduce the surface of the changes you want
> to see reported, tailored in a way that is specific to your project.
> This should hopefully bring the system closer to a state that would
> allow you guys to having something that is automated enough to have it
> be triggered at build time.
Thanks, at least this provides a way to silence the warnings not an issue for us
as we hit them.
Is there a more global, don't warn on new enums kind of option?
Although I assume this is not possible since _END or _MAX enum value will be
changing and tool can't know its usage and will report the change.
>
> And if there is some sensibly needed tweaking that the libabigail
> tooling doesn't allow you guys to do right away, I'd be happy to hear
> about it and try to add the functionality to the framework for you guys.
>
>> Is there any way to reduce the failure only to definite ABI breakages?
>
> I hope my comment above somewhat answers this question of yours. If it
> does not, please tell me.
>
> Cheers,
>
^ permalink raw reply [relevance 4%]
* [dpdk-dev] Minutes of Technical Board Meeting, 2020-01-15
@ 2020-02-03 14:53 5% Maxime Coquelin
0 siblings, 0 replies; 200+ results
From: Maxime Coquelin @ 2020-02-03 14:53 UTC (permalink / raw)
To: DPDK Techboard, dev
Minutes of Technical Board Meeting, 2020-01-15
Members Attending
-----------------
- Ferruh
- Kevin
- Bruce
- Konstantin
- Thomas
- Hemant
- Maxime (Chair)
- Olivier
- Stephen
- Jerin
- Honnappa
NOTE: The technical board meetings every second Wednesday on IRC
channel #dpdk-board, at 3pm UTC. Meetings are public and DPDK community
members are welcome to attend.
NOTE: Next meeting will be on Wednesday 2020-01-29 @3pm UTC, and will be
chaired by Olivier.
1) Estimation of the rework needed for the new vDPA devices class
- Matan already posted the rework, including moving IFC driver
- Main driver impacted will be Mellanox
- LTSes maintainers are fine with the rework
- Thomas applied the rework to master branch, so that Matan can post
the Mellanox vDPA series
2) Action request for Thomas to add approved e-mail IDs to the security
pre-release mailing list
- Done by Thomas
3) UNH Lab decisions
- Technical Board proposed to have member companies to assign people to
add more tests in UNH Lab.
- The Governing Board approved the proposal for one quarter.
- Top contributing companies will be asked to assign one contributor
for 20% FTE.
- Thomas volunteered to manage this trial
4) Kubernetes integration status
- Proposal to write a blog post about DPDK status in Kubernetes and
plans
- Maxime volunteered to write this blog post
- Draft review target on 2020-02-12 Techboard meeting
5) ABI check tooling
- Question about where/whether to store ABI dumps
- Techboard agrees it should not be stored in main DPDK git repo
- Techboard voted to have the ABI check being done by the CI with
scripts provided in DPDK git repo, vs. ABI check done at build time
- No decision yet by the Techboard on whether the ABI dumps for the
releases should be stored somewhere, or generated each time
^ permalink raw reply [relevance 5%]
* Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
2020-02-03 13:09 7% ` Ferruh Yigit
2020-02-03 14:00 7% ` Dodji Seketeli
@ 2020-02-03 15:08 7% ` Trahe, Fiona
1 sibling, 0 replies; 200+ results
From: Trahe, Fiona @ 2020-02-03 15:08 UTC (permalink / raw)
To: Yigit, Ferruh, Neil Horman
Cc: Ananyev, Konstantin, Thomas Monjalon, Akhil Goyal,
David Marchand, Anoob Joseph, Kusztal, ArkadiuszX, dev,
Richardson, Bruce, Mcnamara, John, dodji, Andrew Rybchenko,
aconole, Trahe, Fiona
> >
> > These sorts of changes are the type that we could collectively waive in terms of
> > ABI checking, as they should be ok, but the errors from libabigail should be
> > taken as an indicator that this API could be rewritten (for example by removing
> > the abi entirely, and adding an API call that returns an array of instance name
> > and ids), so that changes of the above sort arent required.
>
> We can spend more time on it, but I can't see for now how to escape returning
> enumaration as indication of type, and this looks legitimate sage as long as
> other side verifies the received value is valid in the type range.
[Fiona] Regarding re-work to make the original code more robust to ABI breakage
One option would be to remove LIST_END from the enum, but keep the enum and allow appending values to it.
Instead of LIST_END have a static var keeping track of the MAX_NUM_AEAD_ALGOS
and an API call rte_cryptodev_get_max_aead_algos() forcing an application to dynamically size any array to accommodate any new values.
The API is safer I think - but there are other pitfalls with this approach - the MAX can more easily get out-of-sync with the enum.
And the application still needs to safely handle values it doesn't recognise.
Anyone think this is a better way?
I still think the best solution is to suppress changes to the LIST_END element.
^ permalink raw reply [relevance 7%]
* Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
2020-02-03 13:09 7% ` Ferruh Yigit
@ 2020-02-03 14:00 7% ` Dodji Seketeli
2020-02-03 14:46 4% ` Ferruh Yigit
2020-02-03 15:08 7% ` Trahe, Fiona
1 sibling, 1 reply; 200+ results
From: Dodji Seketeli @ 2020-02-03 14:00 UTC (permalink / raw)
To: Ferruh Yigit
Cc: Neil Horman, Ananyev, Konstantin, Thomas Monjalon, Akhil Goyal,
Trahe, Fiona, David Marchand, Anoob Joseph, Kusztal, ArkadiuszX,
dev, Richardson, Bruce, Mcnamara, John, Andrew Rybchenko,
aconole
Hello,
Ferruh Yigit <ferruh.yigit@intel.com> a écrit:
[...]
> +1 the change/shuffle of the existing values are problematic, but we don't have
> it in this case.
Right.
[...]
> The concern is when there are cases we can waive, we can't directly rely on the
> tool and automate it. These indicators good for improving the code, but not good
> to use it as build time checker.
Well, it depends. The tooling as it is today have the capability to
automatically "waive" some classes of A{B,P}I change reports that you
guys (the developers) deem harmless, in the context of your project.
For instance, in the precise case of interest here, one could define a
"suppression specification" to teach the ABI verifier that, for the enum
rte_crypto_asym_xform_type, the only enumerator which numerical value is
allowed to change is the one named RTE_CRYPTO_ASYM_XFORM_TYPE_LIST_END.
The content of the suppression specification file would look like:
[suppress_type]
# So, in practise, this rule is to allow adding enumerators
# only to the of the the rte_crypto_asym_xform_type enum,
# right before the RTE_CRYPTO_ASYM_XFORM_TYPE_LIST_END
# enumerator which is meant to always be the last enumerator.
type_kind = enum
name = rte_crypto_asym_xform_type
changed_enumerators = RTE_CRYPTO_ASYM_XFORM_TYPE_LIST_END
This way, you can hopefully reduce the surface of the changes you want
to see reported, tailored in a way that is specific to your project.
This should hopefully bring the system closer to a state that would
allow you guys to having something that is automated enough to have it
be triggered at build time.
And if there is some sensibly needed tweaking that the libabigail
tooling doesn't allow you guys to do right away, I'd be happy to hear
about it and try to add the functionality to the framework for you guys.
> Is there any way to reduce the failure only to definite ABI breakages?
I hope my comment above somewhat answers this question of yours. If it
does not, please tell me.
Cheers,
--
Dodji
^ permalink raw reply [relevance 7%]
* Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
2020-02-03 9:30 7% ` Ferruh Yigit
2020-02-03 11:50 9% ` Neil Horman
@ 2020-02-03 17:09 9% ` Thomas Monjalon
2020-02-03 17:34 8% ` Thomas Monjalon
2020-02-03 17:40 8% ` Ferruh Yigit
1 sibling, 2 replies; 200+ results
From: Thomas Monjalon @ 2020-02-03 17:09 UTC (permalink / raw)
To: Ananyev, Konstantin, Akhil Goyal, Trahe, Fiona, Ferruh Yigit
Cc: David Marchand, Anoob Joseph, Kusztal, ArkadiuszX, dev,
Richardson, Bruce, nhorman, Mcnamara, John, dodji,
Andrew Rybchenko, aconole, bluca, ktraynor
03/02/2020 10:30, Ferruh Yigit:
> On 2/2/2020 2:41 PM, Ananyev, Konstantin wrote:
> > 02/02/2020 14:05, Thomas Monjalon:
> >> 31/01/2020 15:16, Trahe, Fiona:
> >>> On 1/30/2020 8:18 PM, Thomas Monjalon wrote:
> >>>> 30/01/2020 17:09, Ferruh Yigit:
> >>>>> On 1/29/2020 8:13 PM, Akhil Goyal wrote:
> >>>>>>
> >>>>>> I believe these enums will be used only in case of ASYM case which is experimental.
> >>>>>
> >>>>> Independent from being experiment and not, this shouldn't be a problem, I think
> >>>>> this is a false positive.
> >>>>>
> >>>>> The ABI break can happen when a struct has been shared between the application
> >>>>> and the library (DPDK) and the layout of that memory know differently by
> >>>>> application and the library.
> >>>>>
> >>>>> Here in all cases, there is no layout/size change.
> >>>>>
> >>>>> As to the value changes of the enums, since application compiled with old DPDK,
> >>>>> it will know only up to '6', 7 and more means invalid to the application. So it
> >>>>> won't send these values also it should ignore these values from library. Only
> >>>>> consequence is old application won't able to use new features those new enums
> >>>>> provide but that is expected/normal.
> >>>>
> >>>> If library give higher value than expected by the application,
> >>>> if the application uses this value as array index,
> >>>> there can be an access out of bounds.
> >>>
> >>> [Fiona] All asymmetric APIs are experimental so above shouldn't be a problem.
> >>> But for the same issue with sym crypto below, I believe Ferruh's explanation makes
> >>> sense and I don't see how there can be an API breakage.
> >>> So if an application hasn't compiled against the new lib it will be still using the old value
> >>> which will be within bounds. If it's picking up the higher new value from the lib it must
> >>> have been compiled against the lib so shouldn't have problems.
> >>
> >> You say there is no ABI issue because the application will be re-compiled
> >> for the updated library. Indeed, compilation fixes compatibility issues.
> >> But this is not relevant for ABI compatibility.
> >> ABI compatibility means we can upgrade the library without recompiling
> >> the application and it must work.
> >> You think it is a false positive because you assume the application
> >> "picks" the new value. I think you miss the case where the new value
> >> is returned by a function in the upgraded library.
> >>
> >>> There are also no structs on the API which contain arrays using this
> >>> for sizing, so I don't see an opportunity for an appl to have a
> >>> mismatch in memory addresses.
> >>
> >> Let me demonstrate where the API may "use" the new value
> >> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 and how it impacts the application.
> >>
> >> Once upon a time a DPDK application counting the number of devices
> >> supporting each AEAD algo (in order to find the best supported algo).
> >> It is done in an array indexed by algo id:
> >> int aead_dev_count[RTE_CRYPTO_AEAD_LIST_END];
> >> The application is compiled with DPDK 19.11,
> >> where RTE_CRYPTO_AEAD_LIST_END = 3.
> >> So the size of the application array aead_dev_count is 3.
> >> This binary is run with DPDK 20.02,
> >> where RTE_CRYPTO_AEAD_CHACHA20_POLY1305 = 3.
> >> When calling rte_cryptodev_info_get() on a device QAT_GEN3,
> >> rte_cryptodev_info.capabilities.sym.aead.algo is set to
> >> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 (= 3).
> >> The application uses this value:
> >> ++ aead_dev_count[info.capabilities.sym.aead.algo];
> >> The application is crashing because of out of bound access.
> >
> > I'd say this is an example of bad written app.
> > It probably should check that returned by library value doesn't
> > exceed its internal array size.
>
> +1
>
> Application should ignore values >= MAX.
Of course, blaming the API user is a lot easier than looking at the API.
Here the API has RTE_CRYPTO_AEAD_LIST_END which can be understood
as the max value for the application.
Value ranges are part of the ABI compatibility contract.
It seems you expect the application developer to be aware that
DPDK could return a higher value, so the application should
check every enum values after calling an API. CRAZY.
When we decide to announce an ABI compatibility and do some marketing,
everyone is OK. But when we need to really make our ABI compatible,
I see little or no effort. DISAPPOINTING.
> Do you suggest we don't extend any enum or define between ABI breakage releases
> to be sure bad written applications not affected?
I suggest we must consider not breaking any assumption made on the API.
Here we are breaking the enum range because nothing mentions _LIST_END
is not really the absolute end of the enum.
The solution is to make the change below in 20.02 + backport in 19.11.1:
- _LIST_END
+ _LIST_END, /* an ABI-compatible version may increase this value */
+ _LIST_MAX = _LIST_END + 42 /* room for ABI-compatible additions */
};
Then *_LIST_END values could be ignored by libabigail with such a change.
If such a patch is not done by tomorrow, I will have to revert
Chacha-Poly commits before 20.02-rc2, because
1/ LIST_END, without any comment, means "size of range"
2/ we do not blame users for undocumented ABI changes
3/ we respect the ABI compatibility contract
^ permalink raw reply [relevance 9%]
* Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
2020-02-03 17:09 9% ` Thomas Monjalon
@ 2020-02-03 17:34 8% ` Thomas Monjalon
2020-02-03 18:55 7% ` Ray Kinsella
2020-02-03 17:40 8% ` Ferruh Yigit
1 sibling, 1 reply; 200+ results
From: Thomas Monjalon @ 2020-02-03 17:34 UTC (permalink / raw)
To: David Marchand, nhorman, bluca, ktraynor, Ray Kinsella
Cc: Ananyev, Konstantin, Akhil Goyal, Trahe, Fiona, Ferruh Yigit,
dev, Anoob Joseph, Kusztal, ArkadiuszX, dev, Richardson, Bruce,
Mcnamara, John, dodji, Andrew Rybchenko, aconole
03/02/2020 18:09, Thomas Monjalon:
> 03/02/2020 10:30, Ferruh Yigit:
> > On 2/2/2020 2:41 PM, Ananyev, Konstantin wrote:
> > > 02/02/2020 14:05, Thomas Monjalon:
> > >> 31/01/2020 15:16, Trahe, Fiona:
> > >>> On 1/30/2020 8:18 PM, Thomas Monjalon wrote:
> > >>>> If library give higher value than expected by the application,
> > >>>> if the application uses this value as array index,
> > >>>> there can be an access out of bounds.
> > >>>
> > >>> [Fiona] All asymmetric APIs are experimental so above shouldn't be a problem.
> > >>> But for the same issue with sym crypto below, I believe Ferruh's explanation makes
> > >>> sense and I don't see how there can be an API breakage.
> > >>> So if an application hasn't compiled against the new lib it will be still using the old value
> > >>> which will be within bounds. If it's picking up the higher new value from the lib it must
> > >>> have been compiled against the lib so shouldn't have problems.
> > >>
> > >> You say there is no ABI issue because the application will be re-compiled
> > >> for the updated library. Indeed, compilation fixes compatibility issues.
> > >> But this is not relevant for ABI compatibility.
> > >> ABI compatibility means we can upgrade the library without recompiling
> > >> the application and it must work.
> > >> You think it is a false positive because you assume the application
> > >> "picks" the new value. I think you miss the case where the new value
> > >> is returned by a function in the upgraded library.
> > >>
> > >>> There are also no structs on the API which contain arrays using this
> > >>> for sizing, so I don't see an opportunity for an appl to have a
> > >>> mismatch in memory addresses.
> > >>
> > >> Let me demonstrate where the API may "use" the new value
> > >> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 and how it impacts the application.
> > >>
> > >> Once upon a time a DPDK application counting the number of devices
> > >> supporting each AEAD algo (in order to find the best supported algo).
> > >> It is done in an array indexed by algo id:
> > >> int aead_dev_count[RTE_CRYPTO_AEAD_LIST_END];
> > >> The application is compiled with DPDK 19.11,
> > >> where RTE_CRYPTO_AEAD_LIST_END = 3.
> > >> So the size of the application array aead_dev_count is 3.
> > >> This binary is run with DPDK 20.02,
> > >> where RTE_CRYPTO_AEAD_CHACHA20_POLY1305 = 3.
> > >> When calling rte_cryptodev_info_get() on a device QAT_GEN3,
> > >> rte_cryptodev_info.capabilities.sym.aead.algo is set to
> > >> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 (= 3).
> > >> The application uses this value:
> > >> ++ aead_dev_count[info.capabilities.sym.aead.algo];
> > >> The application is crashing because of out of bound access.
> > >
> > > I'd say this is an example of bad written app.
> > > It probably should check that returned by library value doesn't
> > > exceed its internal array size.
> >
> > +1
> >
> > Application should ignore values >= MAX.
>
> Of course, blaming the API user is a lot easier than looking at the API.
> Here the API has RTE_CRYPTO_AEAD_LIST_END which can be understood
> as the max value for the application.
> Value ranges are part of the ABI compatibility contract.
> It seems you expect the application developer to be aware that
> DPDK could return a higher value, so the application should
> check every enum values after calling an API. CRAZY.
>
> When we decide to announce an ABI compatibility and do some marketing,
> everyone is OK. But when we need to really make our ABI compatible,
> I see little or no effort. DISAPPOINTING.
>
> > Do you suggest we don't extend any enum or define between ABI breakage releases
> > to be sure bad written applications not affected?
>
> I suggest we must consider not breaking any assumption made on the API.
> Here we are breaking the enum range because nothing mentions _LIST_END
> is not really the absolute end of the enum.
> The solution is to make the change below in 20.02 + backport in 19.11.1:
Thinking twice, merging such change before 20.11 is breaking the
ABI assumption based on the API 19.11.0.
I ask the release maintainers (Luca, Kevin, David and me) and
the ABI maintainers (Neil and Ray) to vote for a or b solution:
a) add comment and LIST_MAX as below in 20.02 + 19.11.1
b) wait 20.11 and revert Chacha-Poly from 20.02
> - _LIST_END
> + _LIST_END, /* an ABI-compatible version may increase this value */
> + _LIST_MAX = _LIST_END + 42 /* room for ABI-compatible additions */
> };
>
> Then *_LIST_END values could be ignored by libabigail with such a change.
>
> If such a patch is not done by tomorrow, I will have to revert
> Chacha-Poly commits before 20.02-rc2, because
>
> 1/ LIST_END, without any comment, means "size of range"
> 2/ we do not blame users for undocumented ABI changes
> 3/ we respect the ABI compatibility contract
^ permalink raw reply [relevance 8%]
* Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
2020-02-03 17:09 9% ` Thomas Monjalon
2020-02-03 17:34 8% ` Thomas Monjalon
@ 2020-02-03 17:40 8% ` Ferruh Yigit
2020-02-03 18:40 9% ` Thomas Monjalon
2020-02-04 10:16 9% ` Akhil Goyal
1 sibling, 2 replies; 200+ results
From: Ferruh Yigit @ 2020-02-03 17:40 UTC (permalink / raw)
To: Thomas Monjalon, Ananyev, Konstantin, Akhil Goyal, Trahe, Fiona
Cc: David Marchand, Anoob Joseph, Kusztal, ArkadiuszX, dev,
Richardson, Bruce, nhorman, Mcnamara, John, dodji,
Andrew Rybchenko, aconole, bluca, ktraynor
On 2/3/2020 5:09 PM, Thomas Monjalon wrote:
> 03/02/2020 10:30, Ferruh Yigit:
>> On 2/2/2020 2:41 PM, Ananyev, Konstantin wrote:
>>> 02/02/2020 14:05, Thomas Monjalon:
>>>> 31/01/2020 15:16, Trahe, Fiona:
>>>>> On 1/30/2020 8:18 PM, Thomas Monjalon wrote:
>>>>>> 30/01/2020 17:09, Ferruh Yigit:
>>>>>>> On 1/29/2020 8:13 PM, Akhil Goyal wrote:
>>>>>>>>
>>>>>>>> I believe these enums will be used only in case of ASYM case which is experimental.
>>>>>>>
>>>>>>> Independent from being experiment and not, this shouldn't be a problem, I think
>>>>>>> this is a false positive.
>>>>>>>
>>>>>>> The ABI break can happen when a struct has been shared between the application
>>>>>>> and the library (DPDK) and the layout of that memory know differently by
>>>>>>> application and the library.
>>>>>>>
>>>>>>> Here in all cases, there is no layout/size change.
>>>>>>>
>>>>>>> As to the value changes of the enums, since application compiled with old DPDK,
>>>>>>> it will know only up to '6', 7 and more means invalid to the application. So it
>>>>>>> won't send these values also it should ignore these values from library. Only
>>>>>>> consequence is old application won't able to use new features those new enums
>>>>>>> provide but that is expected/normal.
>>>>>>
>>>>>> If library give higher value than expected by the application,
>>>>>> if the application uses this value as array index,
>>>>>> there can be an access out of bounds.
>>>>>
>>>>> [Fiona] All asymmetric APIs are experimental so above shouldn't be a problem.
>>>>> But for the same issue with sym crypto below, I believe Ferruh's explanation makes
>>>>> sense and I don't see how there can be an API breakage.
>>>>> So if an application hasn't compiled against the new lib it will be still using the old value
>>>>> which will be within bounds. If it's picking up the higher new value from the lib it must
>>>>> have been compiled against the lib so shouldn't have problems.
>>>>
>>>> You say there is no ABI issue because the application will be re-compiled
>>>> for the updated library. Indeed, compilation fixes compatibility issues.
>>>> But this is not relevant for ABI compatibility.
>>>> ABI compatibility means we can upgrade the library without recompiling
>>>> the application and it must work.
>>>> You think it is a false positive because you assume the application
>>>> "picks" the new value. I think you miss the case where the new value
>>>> is returned by a function in the upgraded library.
>>>>
>>>>> There are also no structs on the API which contain arrays using this
>>>>> for sizing, so I don't see an opportunity for an appl to have a
>>>>> mismatch in memory addresses.
>>>>
>>>> Let me demonstrate where the API may "use" the new value
>>>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 and how it impacts the application.
>>>>
>>>> Once upon a time a DPDK application counting the number of devices
>>>> supporting each AEAD algo (in order to find the best supported algo).
>>>> It is done in an array indexed by algo id:
>>>> int aead_dev_count[RTE_CRYPTO_AEAD_LIST_END];
>>>> The application is compiled with DPDK 19.11,
>>>> where RTE_CRYPTO_AEAD_LIST_END = 3.
>>>> So the size of the application array aead_dev_count is 3.
>>>> This binary is run with DPDK 20.02,
>>>> where RTE_CRYPTO_AEAD_CHACHA20_POLY1305 = 3.
>>>> When calling rte_cryptodev_info_get() on a device QAT_GEN3,
>>>> rte_cryptodev_info.capabilities.sym.aead.algo is set to
>>>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 (= 3).
>>>> The application uses this value:
>>>> ++ aead_dev_count[info.capabilities.sym.aead.algo];
>>>> The application is crashing because of out of bound access.
>>>
>>> I'd say this is an example of bad written app.
>>> It probably should check that returned by library value doesn't
>>> exceed its internal array size.
>>
>> +1
>>
>> Application should ignore values >= MAX.
>
> Of course, blaming the API user is a lot easier than looking at the API.
> Here the API has RTE_CRYPTO_AEAD_LIST_END which can be understood
> as the max value for the application.
> Value ranges are part of the ABI compatibility contract.
> It seems you expect the application developer to be aware that
> DPDK could return a higher value, so the application should
> check every enum values after calling an API. CRAZY.
>
> When we decide to announce an ABI compatibility and do some marketing,
> everyone is OK. But when we need to really make our ABI compatible,
> I see little or no effort. DISAPPOINTING.
This is not to blame the user or to do less work, this is more sane approach
that library provides the _END/_MAX value and application uses it as valid range
check.
>
>> Do you suggest we don't extend any enum or define between ABI breakage releases
>> to be sure bad written applications not affected?
>
> I suggest we must consider not breaking any assumption made on the API.
> Here we are breaking the enum range because nothing mentions _LIST_END
> is not really the absolute end of the enum.
> The solution is to make the change below in 20.02 + backport in 19.11.1:
>
> - _LIST_END
> + _LIST_END, /* an ABI-compatible version may increase this value */
> + _LIST_MAX = _LIST_END + 42 /* room for ABI-compatible additions */
> };
>
What is the point of "_LIST_MAX" here?
Application should know the "_LIST_END" of when it has been compiled for the
valid range check. Next time it is compiled "_LIST_END" may be different value
but same logic applies.
When "_LIST_END" is missing, application can't protect itself, in that case
library should send only the values application knows when it is compiled, this
means either we can't extend our enum/defines until next ABI breakage, or we
need to do ABI versioning to the functions that returns an enum each time enum
value extended.
I believe it is saner to provide _END/_MAX values to the application to use. And
if required comment them to clarify the expected usage.
But in above suggestion application can't use or rely on "_LIST_MAX", it doesn't
mean anything to application.
> Then *_LIST_END values could be ignored by libabigail with such a change.
>
> If such a patch is not done by tomorrow, I will have to revert
> Chacha-Poly commits before 20.02-rc2, because
>
> 1/ LIST_END, without any comment, means "size of range"
> 2/ we do not blame users for undocumented ABI changes
> 3/ we respect the ABI compatibility contract
>
>
^ permalink raw reply [relevance 8%]
* Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
2020-02-03 17:40 8% ` Ferruh Yigit
@ 2020-02-03 18:40 9% ` Thomas Monjalon
2020-02-04 9:19 4% ` Ferruh Yigit
2020-02-04 10:16 9% ` Akhil Goyal
1 sibling, 1 reply; 200+ results
From: Thomas Monjalon @ 2020-02-03 18:40 UTC (permalink / raw)
To: Ferruh Yigit
Cc: Ananyev, Konstantin, Akhil Goyal, Trahe, Fiona, David Marchand,
Anoob Joseph, Kusztal, ArkadiuszX, dev, Richardson, Bruce,
nhorman, Mcnamara, John, dodji, Andrew Rybchenko, aconole, bluca,
ktraynor
03/02/2020 18:40, Ferruh Yigit:
> On 2/3/2020 5:09 PM, Thomas Monjalon wrote:
> > 03/02/2020 10:30, Ferruh Yigit:
> >> On 2/2/2020 2:41 PM, Ananyev, Konstantin wrote:
> >>> 02/02/2020 14:05, Thomas Monjalon:
> >>>> 31/01/2020 15:16, Trahe, Fiona:
> >>>>> On 1/30/2020 8:18 PM, Thomas Monjalon wrote:
> >>>>>> 30/01/2020 17:09, Ferruh Yigit:
> >>>>>>> On 1/29/2020 8:13 PM, Akhil Goyal wrote:
> >>>>>>>>
> >>>>>>>> I believe these enums will be used only in case of ASYM case which is experimental.
> >>>>>>>
> >>>>>>> Independent from being experiment and not, this shouldn't be a problem, I think
> >>>>>>> this is a false positive.
> >>>>>>>
> >>>>>>> The ABI break can happen when a struct has been shared between the application
> >>>>>>> and the library (DPDK) and the layout of that memory know differently by
> >>>>>>> application and the library.
> >>>>>>>
> >>>>>>> Here in all cases, there is no layout/size change.
> >>>>>>>
> >>>>>>> As to the value changes of the enums, since application compiled with old DPDK,
> >>>>>>> it will know only up to '6', 7 and more means invalid to the application. So it
> >>>>>>> won't send these values also it should ignore these values from library. Only
> >>>>>>> consequence is old application won't able to use new features those new enums
> >>>>>>> provide but that is expected/normal.
> >>>>>>
> >>>>>> If library give higher value than expected by the application,
> >>>>>> if the application uses this value as array index,
> >>>>>> there can be an access out of bounds.
> >>>>>
> >>>>> [Fiona] All asymmetric APIs are experimental so above shouldn't be a problem.
> >>>>> But for the same issue with sym crypto below, I believe Ferruh's explanation makes
> >>>>> sense and I don't see how there can be an API breakage.
> >>>>> So if an application hasn't compiled against the new lib it will be still using the old value
> >>>>> which will be within bounds. If it's picking up the higher new value from the lib it must
> >>>>> have been compiled against the lib so shouldn't have problems.
> >>>>
> >>>> You say there is no ABI issue because the application will be re-compiled
> >>>> for the updated library. Indeed, compilation fixes compatibility issues.
> >>>> But this is not relevant for ABI compatibility.
> >>>> ABI compatibility means we can upgrade the library without recompiling
> >>>> the application and it must work.
> >>>> You think it is a false positive because you assume the application
> >>>> "picks" the new value. I think you miss the case where the new value
> >>>> is returned by a function in the upgraded library.
> >>>>
> >>>>> There are also no structs on the API which contain arrays using this
> >>>>> for sizing, so I don't see an opportunity for an appl to have a
> >>>>> mismatch in memory addresses.
> >>>>
> >>>> Let me demonstrate where the API may "use" the new value
> >>>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 and how it impacts the application.
> >>>>
> >>>> Once upon a time a DPDK application counting the number of devices
> >>>> supporting each AEAD algo (in order to find the best supported algo).
> >>>> It is done in an array indexed by algo id:
> >>>> int aead_dev_count[RTE_CRYPTO_AEAD_LIST_END];
> >>>> The application is compiled with DPDK 19.11,
> >>>> where RTE_CRYPTO_AEAD_LIST_END = 3.
> >>>> So the size of the application array aead_dev_count is 3.
> >>>> This binary is run with DPDK 20.02,
> >>>> where RTE_CRYPTO_AEAD_CHACHA20_POLY1305 = 3.
> >>>> When calling rte_cryptodev_info_get() on a device QAT_GEN3,
> >>>> rte_cryptodev_info.capabilities.sym.aead.algo is set to
> >>>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 (= 3).
> >>>> The application uses this value:
> >>>> ++ aead_dev_count[info.capabilities.sym.aead.algo];
> >>>> The application is crashing because of out of bound access.
> >>>
> >>> I'd say this is an example of bad written app.
> >>> It probably should check that returned by library value doesn't
> >>> exceed its internal array size.
> >>
> >> +1
> >>
> >> Application should ignore values >= MAX.
> >
> > Of course, blaming the API user is a lot easier than looking at the API.
> > Here the API has RTE_CRYPTO_AEAD_LIST_END which can be understood
> > as the max value for the application.
> > Value ranges are part of the ABI compatibility contract.
> > It seems you expect the application developer to be aware that
> > DPDK could return a higher value, so the application should
> > check every enum values after calling an API. CRAZY.
> >
> > When we decide to announce an ABI compatibility and do some marketing,
> > everyone is OK. But when we need to really make our ABI compatible,
> > I see little or no effort. DISAPPOINTING.
>
> This is not to blame the user or to do less work, this is more sane approach
> that library provides the _END/_MAX value and application uses it as valid range
> check.
>
> >> Do you suggest we don't extend any enum or define between ABI breakage releases
> >> to be sure bad written applications not affected?
> >
> > I suggest we must consider not breaking any assumption made on the API.
> > Here we are breaking the enum range because nothing mentions _LIST_END
> > is not really the absolute end of the enum.
> > The solution is to make the change below in 20.02 + backport in 19.11.1:
> >
> > - _LIST_END
> > + _LIST_END, /* an ABI-compatible version may increase this value */
> > + _LIST_MAX = _LIST_END + 42 /* room for ABI-compatible additions */
> > };
> >
>
> What is the point of "_LIST_MAX" here?
_LIST_MAX is range of value that DPDK can return in the ABI contract.
So the appplication can rely on the range 0.._LIST_MAX.
> Application should know the "_LIST_END" of when it has been compiled for the
> valid range check. Next time it is compiled "_LIST_END" may be different value
> but same logic applies.
No, ABI compatibility contract means you can compile your application
with DPDK 19.11.0 and run it with DPDK 20.02.
So _LIST_END comes from 19.11 and does not include ChachaPoly.
> When "_LIST_END" is missing, application can't protect itself, in that case
> library should send only the values application knows when it is compiled, this
> means either we can't extend our enum/defines until next ABI breakage, or we
> need to do ABI versioning to the functions that returns an enum each time enum
> value extended.
If we define _LIST_MAX as a bigger value than current _LIST_END,
we have some room to add values in between.
If (as of now) we don't have _LIST_MAX room, then yes we must version
the functions returning the enum.
In this case, the proper solution is to implement
rte_cryptodev_info_get_v1911() so it filters out
RTE_CRYPTO_AEAD_CHACHA20_POLY1305 capability.
With this solution, an application compiled with DPDK 19.11 will keep
seeing the same range as before, while a 20.02 application could
see and use ChachaPoly.
This is another proposal that I was expecting from the crypto team,
instead of claiming there is no issue (and wasting precious time).
> I believe it is saner to provide _END/_MAX values to the application to use. And
> if required comment them to clarify the expected usage.
>
> But in above suggestion application can't use or rely on "_LIST_MAX", it doesn't
> mean anything to application.
I don't understand what you mean. I think you misunderstood what is ABI compat.
> > Then *_LIST_END values could be ignored by libabigail with such a change.
> >
> > If such a patch is not done by tomorrow, I will have to revert
> > Chacha-Poly commits before 20.02-rc2, because
> >
> > 1/ LIST_END, without any comment, means "size of range"
> > 2/ we do not blame users for undocumented ABI changes
> > 3/ we respect the ABI compatibility contract
^ permalink raw reply [relevance 9%]
* Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
2020-02-03 17:34 8% ` Thomas Monjalon
@ 2020-02-03 18:55 7% ` Ray Kinsella
2020-02-03 21:07 7% ` Thomas Monjalon
2020-02-04 9:51 4% ` David Marchand
0 siblings, 2 replies; 200+ results
From: Ray Kinsella @ 2020-02-03 18:55 UTC (permalink / raw)
To: Thomas Monjalon, David Marchand, nhorman, bluca, ktraynor
Cc: Ananyev, Konstantin, Akhil Goyal, Trahe, Fiona, Ferruh Yigit,
dev, Anoob Joseph, Kusztal, ArkadiuszX, Richardson, Bruce,
Mcnamara, John, dodji, Andrew Rybchenko, aconole
On 03/02/2020 17:34, Thomas Monjalon wrote:
> 03/02/2020 18:09, Thomas Monjalon:
>> 03/02/2020 10:30, Ferruh Yigit:
>>> On 2/2/2020 2:41 PM, Ananyev, Konstantin wrote:
>>>> 02/02/2020 14:05, Thomas Monjalon:
>>>>> 31/01/2020 15:16, Trahe, Fiona:
>>>>>> On 1/30/2020 8:18 PM, Thomas Monjalon wrote:
>>>>>>> If library give higher value than expected by the application,
>>>>>>> if the application uses this value as array index,
>>>>>>> there can be an access out of bounds.
>>>>>>
>>>>>> [Fiona] All asymmetric APIs are experimental so above shouldn't be a problem.
>>>>>> But for the same issue with sym crypto below, I believe Ferruh's explanation makes
>>>>>> sense and I don't see how there can be an API breakage.
>>>>>> So if an application hasn't compiled against the new lib it will be still using the old value
>>>>>> which will be within bounds. If it's picking up the higher new value from the lib it must
>>>>>> have been compiled against the lib so shouldn't have problems.
>>>>>
>>>>> You say there is no ABI issue because the application will be re-compiled
>>>>> for the updated library. Indeed, compilation fixes compatibility issues.
>>>>> But this is not relevant for ABI compatibility.
>>>>> ABI compatibility means we can upgrade the library without recompiling
>>>>> the application and it must work.
>>>>> You think it is a false positive because you assume the application
>>>>> "picks" the new value. I think you miss the case where the new value
>>>>> is returned by a function in the upgraded library.
>>>>>
>>>>>> There are also no structs on the API which contain arrays using this
>>>>>> for sizing, so I don't see an opportunity for an appl to have a
>>>>>> mismatch in memory addresses.
>>>>>
>>>>> Let me demonstrate where the API may "use" the new value
>>>>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 and how it impacts the application.
>>>>>
>>>>> Once upon a time a DPDK application counting the number of devices
>>>>> supporting each AEAD algo (in order to find the best supported algo).
>>>>> It is done in an array indexed by algo id:
>>>>> int aead_dev_count[RTE_CRYPTO_AEAD_LIST_END];
>>>>> The application is compiled with DPDK 19.11,
>>>>> where RTE_CRYPTO_AEAD_LIST_END = 3.
>>>>> So the size of the application array aead_dev_count is 3.
>>>>> This binary is run with DPDK 20.02,
>>>>> where RTE_CRYPTO_AEAD_CHACHA20_POLY1305 = 3.
>>>>> When calling rte_cryptodev_info_get() on a device QAT_GEN3,
>>>>> rte_cryptodev_info.capabilities.sym.aead.algo is set to
>>>>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 (= 3).
>>>>> The application uses this value:
>>>>> ++ aead_dev_count[info.capabilities.sym.aead.algo];
>>>>> The application is crashing because of out of bound access.
>>>>
>>>> I'd say this is an example of bad written app.
>>>> It probably should check that returned by library value doesn't
>>>> exceed its internal array size.
>>>
>>> +1
>>>
>>> Application should ignore values >= MAX.
>>
>> Of course, blaming the API user is a lot easier than looking at the API.
>> Here the API has RTE_CRYPTO_AEAD_LIST_END which can be understood
>> as the max value for the application.
>> Value ranges are part of the ABI compatibility contract.
>> It seems you expect the application developer to be aware that
>> DPDK could return a higher value, so the application should
>> check every enum values after calling an API. CRAZY.
>>
>> When we decide to announce an ABI compatibility and do some marketing,
>> everyone is OK. But when we need to really make our ABI compatible,
>> I see little or no effort. DISAPPOINTING.
>>
>>> Do you suggest we don't extend any enum or define between ABI breakage releases
>>> to be sure bad written applications not affected?
>>
>> I suggest we must consider not breaking any assumption made on the API.
>> Here we are breaking the enum range because nothing mentions _LIST_END
>> is not really the absolute end of the enum.
>> The solution is to make the change below in 20.02 + backport in 19.11.1:
>
> Thinking twice, merging such change before 20.11 is breaking the
> ABI assumption based on the API 19.11.0.
> I ask the release maintainers (Luca, Kevin, David and me) and
> the ABI maintainers (Neil and Ray) to vote for a or b solution:
> a) add comment and LIST_MAX as below in 20.02 + 19.11.1
That would still be an ABI breakage though right.
> b) wait 20.11 and revert Chacha-Poly from 20.02
Thanks for analysis above Fiona, Ferruh and all.
That is a nasty one alright - there is no "good" answer here.
I agree with Ferruh's sentiments overall, we should rethink this API for 20.11.
Could do without an enumeration?
There a c) though right.
We could work around the issue by api versioning rte_cryptodev_info_get() and friends.
So they only support/acknowledge the existence of Chacha-Poly for applications build against > 20.02.
It would be painful I know.
It would also mean that Chacha-Poly would only be available to those building against >= 20.02.
>
>
>> - _LIST_END
>> + _LIST_END, /* an ABI-compatible version may increase this value */
>> + _LIST_MAX = _LIST_END + 42 /* room for ABI-compatible additions */
>> };
>>
>> Then *_LIST_END values could be ignored by libabigail with such a change.
>>
>> If such a patch is not done by tomorrow, I will have to revert
>> Chacha-Poly commits before 20.02-rc2, because
>>
>> 1/ LIST_END, without any comment, means "size of range"
>> 2/ we do not blame users for undocumented ABI changes
>> 3/ we respect the ABI compatibility contract
^ permalink raw reply [relevance 7%]
* Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
2020-02-03 18:55 7% ` Ray Kinsella
@ 2020-02-03 21:07 7% ` Thomas Monjalon
2020-02-04 9:46 8% ` Ferruh Yigit
2020-02-04 10:24 9% ` Thomas Monjalon
2020-02-04 9:51 4% ` David Marchand
1 sibling, 2 replies; 200+ results
From: Thomas Monjalon @ 2020-02-03 21:07 UTC (permalink / raw)
To: David Marchand, nhorman, bluca, ktraynor, Ray Kinsella
Cc: Ananyev, Konstantin, Akhil Goyal, Trahe, Fiona, Ferruh Yigit,
dev, Anoob Joseph, Kusztal, ArkadiuszX, Richardson, Bruce,
Mcnamara, John, dodji, Andrew Rybchenko, aconole
03/02/2020 19:55, Ray Kinsella:
> On 03/02/2020 17:34, Thomas Monjalon wrote:
> > 03/02/2020 18:09, Thomas Monjalon:
> >> 03/02/2020 10:30, Ferruh Yigit:
> >>> On 2/2/2020 2:41 PM, Ananyev, Konstantin wrote:
> >>>> 02/02/2020 14:05, Thomas Monjalon:
> >>>>> 31/01/2020 15:16, Trahe, Fiona:
> >>>>>> On 1/30/2020 8:18 PM, Thomas Monjalon wrote:
> >>>>>>> If library give higher value than expected by the application,
> >>>>>>> if the application uses this value as array index,
> >>>>>>> there can be an access out of bounds.
> >>>>>>
> >>>>>> [Fiona] All asymmetric APIs are experimental so above shouldn't be a problem.
> >>>>>> But for the same issue with sym crypto below, I believe Ferruh's explanation makes
> >>>>>> sense and I don't see how there can be an API breakage.
> >>>>>> So if an application hasn't compiled against the new lib it will be still using the old value
> >>>>>> which will be within bounds. If it's picking up the higher new value from the lib it must
> >>>>>> have been compiled against the lib so shouldn't have problems.
> >>>>>
> >>>>> You say there is no ABI issue because the application will be re-compiled
> >>>>> for the updated library. Indeed, compilation fixes compatibility issues.
> >>>>> But this is not relevant for ABI compatibility.
> >>>>> ABI compatibility means we can upgrade the library without recompiling
> >>>>> the application and it must work.
> >>>>> You think it is a false positive because you assume the application
> >>>>> "picks" the new value. I think you miss the case where the new value
> >>>>> is returned by a function in the upgraded library.
> >>>>>
> >>>>>> There are also no structs on the API which contain arrays using this
> >>>>>> for sizing, so I don't see an opportunity for an appl to have a
> >>>>>> mismatch in memory addresses.
> >>>>>
> >>>>> Let me demonstrate where the API may "use" the new value
> >>>>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 and how it impacts the application.
> >>>>>
> >>>>> Once upon a time a DPDK application counting the number of devices
> >>>>> supporting each AEAD algo (in order to find the best supported algo).
> >>>>> It is done in an array indexed by algo id:
> >>>>> int aead_dev_count[RTE_CRYPTO_AEAD_LIST_END];
> >>>>> The application is compiled with DPDK 19.11,
> >>>>> where RTE_CRYPTO_AEAD_LIST_END = 3.
> >>>>> So the size of the application array aead_dev_count is 3.
> >>>>> This binary is run with DPDK 20.02,
> >>>>> where RTE_CRYPTO_AEAD_CHACHA20_POLY1305 = 3.
> >>>>> When calling rte_cryptodev_info_get() on a device QAT_GEN3,
> >>>>> rte_cryptodev_info.capabilities.sym.aead.algo is set to
> >>>>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 (= 3).
> >>>>> The application uses this value:
> >>>>> ++ aead_dev_count[info.capabilities.sym.aead.algo];
> >>>>> The application is crashing because of out of bound access.
> >>>>
> >>>> I'd say this is an example of bad written app.
> >>>> It probably should check that returned by library value doesn't
> >>>> exceed its internal array size.
> >>>
> >>> +1
> >>>
> >>> Application should ignore values >= MAX.
> >>
> >> Of course, blaming the API user is a lot easier than looking at the API.
> >> Here the API has RTE_CRYPTO_AEAD_LIST_END which can be understood
> >> as the max value for the application.
> >> Value ranges are part of the ABI compatibility contract.
> >> It seems you expect the application developer to be aware that
> >> DPDK could return a higher value, so the application should
> >> check every enum values after calling an API. CRAZY.
> >>
> >> When we decide to announce an ABI compatibility and do some marketing,
> >> everyone is OK. But when we need to really make our ABI compatible,
> >> I see little or no effort. DISAPPOINTING.
> >>
> >>> Do you suggest we don't extend any enum or define between ABI breakage releases
> >>> to be sure bad written applications not affected?
> >>
> >> I suggest we must consider not breaking any assumption made on the API.
> >> Here we are breaking the enum range because nothing mentions _LIST_END
> >> is not really the absolute end of the enum.
> >> The solution is to make the change below in 20.02 + backport in 19.11.1:
> >
> > Thinking twice, merging such change before 20.11 is breaking the
> > ABI assumption based on the API 19.11.0.
> > I ask the release maintainers (Luca, Kevin, David and me) and
> > the ABI maintainers (Neil and Ray) to vote for a or b solution:
> > a) add comment and LIST_MAX as below in 20.02 + 19.11.1
>
> That would still be an ABI breakage though right.
>
> > b) wait 20.11 and revert Chacha-Poly from 20.02
>
> Thanks for analysis above Fiona, Ferruh and all.
>
> That is a nasty one alright - there is no "good" answer here.
> I agree with Ferruh's sentiments overall, we should rethink this API for 20.11.
> Could do without an enumeration?
>
> There a c) though right.
> We could work around the issue by api versioning rte_cryptodev_info_get() and friends.
> So they only support/acknowledge the existence of Chacha-Poly for applications build against > 20.02.
I agree there is a c) as I proposed in another email:
http://mails.dpdk.org/archives/dev/2020-February/156919.html
"
In this case, the proper solution is to implement
rte_cryptodev_info_get_v1911() so it filters out
RTE_CRYPTO_AEAD_CHACHA20_POLY1305 capability.
With this solution, an application compiled with DPDK 19.11 will keep
seeing the same range as before, while a 20.02 application could
see and use ChachaPoly.
"
> It would be painful I know.
Not so painful in my opinion.
Just need to call rte_cryptodev_info_get() from
rte_cryptodev_info_get_v1911() and filter the value
in the 19.11 range: [0..AES_GCM].
> It would also mean that Chacha-Poly would only be available to
> those building against >= 20.02.
Yes exactly.
The addition of comments and LIST_MAX like below are still valid
to avoid versioning after 20.11.
> >> - _LIST_END
> >> + _LIST_END, /* an ABI-compatible version may increase this value */
> >> + _LIST_MAX = _LIST_END + 42 /* room for ABI-compatible additions */
> >> };
> >>
> >> Then *_LIST_END values could be ignored by libabigail with such a change.
In order to avoid ABI check complaining, the best is to completely
remove LIST_END in DPDK 20.11.
> >> If such a patch is not done by tomorrow, I will have to revert
> >> Chacha-Poly commits before 20.02-rc2, because
> >>
> >> 1/ LIST_END, without any comment, means "size of range"
> >> 2/ we do not blame users for undocumented ABI changes
> >> 3/ we respect the ABI compatibility contract
^ permalink raw reply [relevance 7%]
* Re: [dpdk-dev] [PATCH v7] eal: add manual probing option
@ 2020-02-03 22:21 3% ` Thomas Monjalon
2020-02-04 10:03 0% ` Gaetan Rivet
0 siblings, 1 reply; 200+ results
From: Thomas Monjalon @ 2020-02-03 22:21 UTC (permalink / raw)
To: Gaetan Rivet, Pavan Nikhilesh Bhagavatula
Cc: dev, David Marchand, Vamsi Krishna Attunuru, Jerin Jacob Kollanukkaran
03/02/2020 06:16, Pavan Nikhilesh Bhagavatula:
> @David Marchand @thomas@monjalon.net
>
> Ping?
>
> Are there any more changes required for this patch? It's been in queue since last October.
Sorry we have not decided whether it is a good idea or not.
All changes related to probing are very sensitive,
and we know a big refactoring would be better than stacking
more and more options and corner cases.
As we are busy with ABI stability stuff, we did not allocate
enough time to properly think about this feature.
Please accept our apologies, and let's consider it as
a high priority for 20.05 cycle.
^ permalink raw reply [relevance 3%]
* Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
2020-02-03 18:40 9% ` Thomas Monjalon
@ 2020-02-04 9:19 4% ` Ferruh Yigit
2020-02-04 9:45 4% ` Thomas Monjalon
0 siblings, 1 reply; 200+ results
From: Ferruh Yigit @ 2020-02-04 9:19 UTC (permalink / raw)
To: Thomas Monjalon
Cc: Ananyev, Konstantin, Akhil Goyal, Trahe, Fiona, David Marchand,
Anoob Joseph, Kusztal, ArkadiuszX, dev, Richardson, Bruce,
nhorman, Mcnamara, John, dodji, Andrew Rybchenko, aconole, bluca,
ktraynor
On 2/3/2020 6:40 PM, Thomas Monjalon wrote:
> 03/02/2020 18:40, Ferruh Yigit:
>> On 2/3/2020 5:09 PM, Thomas Monjalon wrote:
>>> 03/02/2020 10:30, Ferruh Yigit:
>>>> On 2/2/2020 2:41 PM, Ananyev, Konstantin wrote:
>>>>> 02/02/2020 14:05, Thomas Monjalon:
>>>>>> 31/01/2020 15:16, Trahe, Fiona:
>>>>>>> On 1/30/2020 8:18 PM, Thomas Monjalon wrote:
>>>>>>>> 30/01/2020 17:09, Ferruh Yigit:
>>>>>>>>> On 1/29/2020 8:13 PM, Akhil Goyal wrote:
>>>>>>>>>>
>>>>>>>>>> I believe these enums will be used only in case of ASYM case which is experimental.
>>>>>>>>>
>>>>>>>>> Independent from being experiment and not, this shouldn't be a problem, I think
>>>>>>>>> this is a false positive.
>>>>>>>>>
>>>>>>>>> The ABI break can happen when a struct has been shared between the application
>>>>>>>>> and the library (DPDK) and the layout of that memory know differently by
>>>>>>>>> application and the library.
>>>>>>>>>
>>>>>>>>> Here in all cases, there is no layout/size change.
>>>>>>>>>
>>>>>>>>> As to the value changes of the enums, since application compiled with old DPDK,
>>>>>>>>> it will know only up to '6', 7 and more means invalid to the application. So it
>>>>>>>>> won't send these values also it should ignore these values from library. Only
>>>>>>>>> consequence is old application won't able to use new features those new enums
>>>>>>>>> provide but that is expected/normal.
>>>>>>>>
>>>>>>>> If library give higher value than expected by the application,
>>>>>>>> if the application uses this value as array index,
>>>>>>>> there can be an access out of bounds.
>>>>>>>
>>>>>>> [Fiona] All asymmetric APIs are experimental so above shouldn't be a problem.
>>>>>>> But for the same issue with sym crypto below, I believe Ferruh's explanation makes
>>>>>>> sense and I don't see how there can be an API breakage.
>>>>>>> So if an application hasn't compiled against the new lib it will be still using the old value
>>>>>>> which will be within bounds. If it's picking up the higher new value from the lib it must
>>>>>>> have been compiled against the lib so shouldn't have problems.
>>>>>>
>>>>>> You say there is no ABI issue because the application will be re-compiled
>>>>>> for the updated library. Indeed, compilation fixes compatibility issues.
>>>>>> But this is not relevant for ABI compatibility.
>>>>>> ABI compatibility means we can upgrade the library without recompiling
>>>>>> the application and it must work.
>>>>>> You think it is a false positive because you assume the application
>>>>>> "picks" the new value. I think you miss the case where the new value
>>>>>> is returned by a function in the upgraded library.
>>>>>>
>>>>>>> There are also no structs on the API which contain arrays using this
>>>>>>> for sizing, so I don't see an opportunity for an appl to have a
>>>>>>> mismatch in memory addresses.
>>>>>>
>>>>>> Let me demonstrate where the API may "use" the new value
>>>>>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 and how it impacts the application.
>>>>>>
>>>>>> Once upon a time a DPDK application counting the number of devices
>>>>>> supporting each AEAD algo (in order to find the best supported algo).
>>>>>> It is done in an array indexed by algo id:
>>>>>> int aead_dev_count[RTE_CRYPTO_AEAD_LIST_END];
>>>>>> The application is compiled with DPDK 19.11,
>>>>>> where RTE_CRYPTO_AEAD_LIST_END = 3.
>>>>>> So the size of the application array aead_dev_count is 3.
>>>>>> This binary is run with DPDK 20.02,
>>>>>> where RTE_CRYPTO_AEAD_CHACHA20_POLY1305 = 3.
>>>>>> When calling rte_cryptodev_info_get() on a device QAT_GEN3,
>>>>>> rte_cryptodev_info.capabilities.sym.aead.algo is set to
>>>>>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 (= 3).
>>>>>> The application uses this value:
>>>>>> ++ aead_dev_count[info.capabilities.sym.aead.algo];
>>>>>> The application is crashing because of out of bound access.
>>>>>
>>>>> I'd say this is an example of bad written app.
>>>>> It probably should check that returned by library value doesn't
>>>>> exceed its internal array size.
>>>>
>>>> +1
>>>>
>>>> Application should ignore values >= MAX.
>>>
>>> Of course, blaming the API user is a lot easier than looking at the API.
>>> Here the API has RTE_CRYPTO_AEAD_LIST_END which can be understood
>>> as the max value for the application.
>>> Value ranges are part of the ABI compatibility contract.
>>> It seems you expect the application developer to be aware that
>>> DPDK could return a higher value, so the application should
>>> check every enum values after calling an API. CRAZY.
>>>
>>> When we decide to announce an ABI compatibility and do some marketing,
>>> everyone is OK. But when we need to really make our ABI compatible,
>>> I see little or no effort. DISAPPOINTING.
>>
>> This is not to blame the user or to do less work, this is more sane approach
>> that library provides the _END/_MAX value and application uses it as valid range
>> check.
>>
>>>> Do you suggest we don't extend any enum or define between ABI breakage releases
>>>> to be sure bad written applications not affected?
>>>
>>> I suggest we must consider not breaking any assumption made on the API.
>>> Here we are breaking the enum range because nothing mentions _LIST_END
>>> is not really the absolute end of the enum.
>>> The solution is to make the change below in 20.02 + backport in 19.11.1:
>>>
>>> - _LIST_END
>>> + _LIST_END, /* an ABI-compatible version may increase this value */
>>> + _LIST_MAX = _LIST_END + 42 /* room for ABI-compatible additions */
>>> };
>>>
>>
>> What is the point of "_LIST_MAX" here?
>
> _LIST_MAX is range of value that DPDK can return in the ABI contract.
> So the appplication can rely on the range 0.._LIST_MAX.
>
>> Application should know the "_LIST_END" of when it has been compiled for the
>> valid range check. Next time it is compiled "_LIST_END" may be different value
>> but same logic applies.
>
> No, ABI compatibility contract means you can compile your application
> with DPDK 19.11.0 and run it with DPDK 20.02.
> So _LIST_END comes from 19.11 and does not include ChachaPoly.
That is what I mean, let me try to give a sample.
DPDK19.11 returns, A=1, B=2, END=3
Application compiled with DPDK19.11, it will process A, B and ignore anything ">= 3"
DPDK20.02 returns A=1, B=2, C=3, D=4, END=5
Old application will still only will know/use A, B and can ignore when library
sends C=3, D=4 etc...
In above, if you add another limit as you suggested, like MAX=10 and ask
application to use it,
Application compiled with DPDK19.11 will be OK since library only sends A,B and
application uses them.
But with DPDK20.02 application may have problem, since library will be sending
C=3, which is valid according to the check " <= MAX (10)", how application will
know to ignore it.
So application should use _END to know the valid ones according it, if so what
is the point of having _MAX.
>
>> When "_LIST_END" is missing, application can't protect itself, in that case
>> library should send only the values application knows when it is compiled, this
>> means either we can't extend our enum/defines until next ABI breakage, or we
>> need to do ABI versioning to the functions that returns an enum each time enum
>> value extended.
>
> If we define _LIST_MAX as a bigger value than current _LIST_END,
> we have some room to add values in between.
>
> If (as of now) we don't have _LIST_MAX room, then yes we must version
> the functions returning the enum.
> In this case, the proper solution is to implement
> rte_cryptodev_info_get_v1911() so it filters out
> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 capability.
> With this solution, an application compiled with DPDK 19.11 will keep
> seeing the same range as before, while a 20.02 application could
> see and use ChachaPoly.
> This is another proposal that I was expecting from the crypto team,
> instead of claiming there is no issue (and wasting precious time).
>
>
>> I believe it is saner to provide _END/_MAX values to the application to use. And
>> if required comment them to clarify the expected usage.
>>
>> But in above suggestion application can't use or rely on "_LIST_MAX", it doesn't
>> mean anything to application.
>
> I don't understand what you mean. I think you misunderstood what is ABI compat.
>
>
>>> Then *_LIST_END values could be ignored by libabigail with such a change.
>>>
>>> If such a patch is not done by tomorrow, I will have to revert
>>> Chacha-Poly commits before 20.02-rc2, because
>>>
>>> 1/ LIST_END, without any comment, means "size of range"
>>> 2/ we do not blame users for undocumented ABI changes
>>> 3/ we respect the ABI compatibility contract
>
>
>
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
2020-02-04 9:19 4% ` Ferruh Yigit
@ 2020-02-04 9:45 4% ` Thomas Monjalon
2020-02-04 9:56 4% ` Ferruh Yigit
0 siblings, 1 reply; 200+ results
From: Thomas Monjalon @ 2020-02-04 9:45 UTC (permalink / raw)
To: Ferruh Yigit
Cc: dev, Ananyev, Konstantin, Akhil Goyal, Trahe, Fiona,
David Marchand, Anoob Joseph, Kusztal, ArkadiuszX, dev,
Richardson, Bruce, nhorman, Mcnamara, John, dodji,
Andrew Rybchenko, aconole, bluca, ktraynor
04/02/2020 10:19, Ferruh Yigit:
> On 2/3/2020 6:40 PM, Thomas Monjalon wrote:
> > 03/02/2020 18:40, Ferruh Yigit:
> >> On 2/3/2020 5:09 PM, Thomas Monjalon wrote:
> >>> 03/02/2020 10:30, Ferruh Yigit:
> >>>> On 2/2/2020 2:41 PM, Ananyev, Konstantin wrote:
> >>>>> 02/02/2020 14:05, Thomas Monjalon:
> >>>>>> 31/01/2020 15:16, Trahe, Fiona:
> >>>>>>> On 1/30/2020 8:18 PM, Thomas Monjalon wrote:
> >>>>>>>> 30/01/2020 17:09, Ferruh Yigit:
> >>>>>>>>> On 1/29/2020 8:13 PM, Akhil Goyal wrote:
> >>>>>>>>>>
> >>>>>>>>>> I believe these enums will be used only in case of ASYM case which is experimental.
> >>>>>>>>>
> >>>>>>>>> Independent from being experiment and not, this shouldn't be a problem, I think
> >>>>>>>>> this is a false positive.
> >>>>>>>>>
> >>>>>>>>> The ABI break can happen when a struct has been shared between the application
> >>>>>>>>> and the library (DPDK) and the layout of that memory know differently by
> >>>>>>>>> application and the library.
> >>>>>>>>>
> >>>>>>>>> Here in all cases, there is no layout/size change.
> >>>>>>>>>
> >>>>>>>>> As to the value changes of the enums, since application compiled with old DPDK,
> >>>>>>>>> it will know only up to '6', 7 and more means invalid to the application. So it
> >>>>>>>>> won't send these values also it should ignore these values from library. Only
> >>>>>>>>> consequence is old application won't able to use new features those new enums
> >>>>>>>>> provide but that is expected/normal.
> >>>>>>>>
> >>>>>>>> If library give higher value than expected by the application,
> >>>>>>>> if the application uses this value as array index,
> >>>>>>>> there can be an access out of bounds.
> >>>>>>>
> >>>>>>> [Fiona] All asymmetric APIs are experimental so above shouldn't be a problem.
> >>>>>>> But for the same issue with sym crypto below, I believe Ferruh's explanation makes
> >>>>>>> sense and I don't see how there can be an API breakage.
> >>>>>>> So if an application hasn't compiled against the new lib it will be still using the old value
> >>>>>>> which will be within bounds. If it's picking up the higher new value from the lib it must
> >>>>>>> have been compiled against the lib so shouldn't have problems.
> >>>>>>
> >>>>>> You say there is no ABI issue because the application will be re-compiled
> >>>>>> for the updated library. Indeed, compilation fixes compatibility issues.
> >>>>>> But this is not relevant for ABI compatibility.
> >>>>>> ABI compatibility means we can upgrade the library without recompiling
> >>>>>> the application and it must work.
> >>>>>> You think it is a false positive because you assume the application
> >>>>>> "picks" the new value. I think you miss the case where the new value
> >>>>>> is returned by a function in the upgraded library.
> >>>>>>
> >>>>>>> There are also no structs on the API which contain arrays using this
> >>>>>>> for sizing, so I don't see an opportunity for an appl to have a
> >>>>>>> mismatch in memory addresses.
> >>>>>>
> >>>>>> Let me demonstrate where the API may "use" the new value
> >>>>>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 and how it impacts the application.
> >>>>>>
> >>>>>> Once upon a time a DPDK application counting the number of devices
> >>>>>> supporting each AEAD algo (in order to find the best supported algo).
> >>>>>> It is done in an array indexed by algo id:
> >>>>>> int aead_dev_count[RTE_CRYPTO_AEAD_LIST_END];
> >>>>>> The application is compiled with DPDK 19.11,
> >>>>>> where RTE_CRYPTO_AEAD_LIST_END = 3.
> >>>>>> So the size of the application array aead_dev_count is 3.
> >>>>>> This binary is run with DPDK 20.02,
> >>>>>> where RTE_CRYPTO_AEAD_CHACHA20_POLY1305 = 3.
> >>>>>> When calling rte_cryptodev_info_get() on a device QAT_GEN3,
> >>>>>> rte_cryptodev_info.capabilities.sym.aead.algo is set to
> >>>>>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 (= 3).
> >>>>>> The application uses this value:
> >>>>>> ++ aead_dev_count[info.capabilities.sym.aead.algo];
> >>>>>> The application is crashing because of out of bound access.
> >>>>>
> >>>>> I'd say this is an example of bad written app.
> >>>>> It probably should check that returned by library value doesn't
> >>>>> exceed its internal array size.
> >>>>
> >>>> +1
> >>>>
> >>>> Application should ignore values >= MAX.
> >>>
> >>> Of course, blaming the API user is a lot easier than looking at the API.
> >>> Here the API has RTE_CRYPTO_AEAD_LIST_END which can be understood
> >>> as the max value for the application.
> >>> Value ranges are part of the ABI compatibility contract.
> >>> It seems you expect the application developer to be aware that
> >>> DPDK could return a higher value, so the application should
> >>> check every enum values after calling an API. CRAZY.
> >>>
> >>> When we decide to announce an ABI compatibility and do some marketing,
> >>> everyone is OK. But when we need to really make our ABI compatible,
> >>> I see little or no effort. DISAPPOINTING.
> >>
> >> This is not to blame the user or to do less work, this is more sane approach
> >> that library provides the _END/_MAX value and application uses it as valid range
> >> check.
> >>
> >>>> Do you suggest we don't extend any enum or define between ABI breakage releases
> >>>> to be sure bad written applications not affected?
> >>>
> >>> I suggest we must consider not breaking any assumption made on the API.
> >>> Here we are breaking the enum range because nothing mentions _LIST_END
> >>> is not really the absolute end of the enum.
> >>> The solution is to make the change below in 20.02 + backport in 19.11.1:
> >>>
> >>> - _LIST_END
> >>> + _LIST_END, /* an ABI-compatible version may increase this value */
> >>> + _LIST_MAX = _LIST_END + 42 /* room for ABI-compatible additions */
> >>> };
> >>>
> >>
> >> What is the point of "_LIST_MAX" here?
> >
> > _LIST_MAX is range of value that DPDK can return in the ABI contract.
> > So the appplication can rely on the range 0.._LIST_MAX.
> >
> >> Application should know the "_LIST_END" of when it has been compiled for the
> >> valid range check. Next time it is compiled "_LIST_END" may be different value
> >> but same logic applies.
> >
> > No, ABI compatibility contract means you can compile your application
> > with DPDK 19.11.0 and run it with DPDK 20.02.
> > So _LIST_END comes from 19.11 and does not include ChachaPoly.
>
> That is what I mean, let me try to give a sample.
>
> DPDK19.11 returns, A=1, B=2, END=3
>
> Application compiled with DPDK19.11, it will process A, B and ignore anything ">= 3"
No, the application will not ignore anything ">=3" as I explained above,
and you blamed the application for it.
Nothing in the API says the application must filter value higher than 3,
because as of now, values higher than 3 are PMD bug.
> DPDK20.02 returns A=1, B=2, C=3, D=4, END=5
>
> Old application will still only will know/use A, B and can ignore when library
> sends C=3, D=4 etc...
>
>
> In above, if you add another limit as you suggested, like MAX=10 and ask
> application to use it,
>
> Application compiled with DPDK19.11 will be OK since library only sends A,B and
> application uses them.
>
> But with DPDK20.02 application may have problem, since library will be sending
> C=3, which is valid according to the check " <= MAX (10)", how application will
> know to ignore it.
Why application should ignore value C=3 with DPDK 20.02?
> So application should use _END to know the valid ones according it, if so what
> is the point of having _MAX.
>
>
> >> When "_LIST_END" is missing, application can't protect itself, in that case
> >> library should send only the values application knows when it is compiled, this
> >> means either we can't extend our enum/defines until next ABI breakage, or we
> >> need to do ABI versioning to the functions that returns an enum each time enum
> >> value extended.
> >
> > If we define _LIST_MAX as a bigger value than current _LIST_END,
> > we have some room to add values in between.
> >
> > If (as of now) we don't have _LIST_MAX room, then yes we must version
> > the functions returning the enum.
> > In this case, the proper solution is to implement
> > rte_cryptodev_info_get_v1911() so it filters out
> > RTE_CRYPTO_AEAD_CHACHA20_POLY1305 capability.
> > With this solution, an application compiled with DPDK 19.11 will keep
> > seeing the same range as before, while a 20.02 application could
> > see and use ChachaPoly.
> > This is another proposal that I was expecting from the crypto team,
> > instead of claiming there is no issue (and wasting precious time).
> >
> >
> >> I believe it is saner to provide _END/_MAX values to the application to use. And
> >> if required comment them to clarify the expected usage.
> >>
> >> But in above suggestion application can't use or rely on "_LIST_MAX", it doesn't
> >> mean anything to application.
> >
> > I don't understand what you mean. I think you misunderstood what is ABI compat.
> >
> >
> >>> Then *_LIST_END values could be ignored by libabigail with such a change.
> >>>
> >>> If such a patch is not done by tomorrow, I will have to revert
> >>> Chacha-Poly commits before 20.02-rc2, because
> >>>
> >>> 1/ LIST_END, without any comment, means "size of range"
> >>> 2/ we do not blame users for undocumented ABI changes
> >>> 3/ we respect the ABI compatibility contract
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
2020-02-03 18:55 7% ` Ray Kinsella
2020-02-03 21:07 7% ` Thomas Monjalon
@ 2020-02-04 9:51 4% ` David Marchand
2020-02-04 10:10 4% ` Trahe, Fiona
2020-02-05 11:10 4% ` Ray Kinsella
1 sibling, 2 replies; 200+ results
From: David Marchand @ 2020-02-04 9:51 UTC (permalink / raw)
To: Ray Kinsella
Cc: Thomas Monjalon, Neil Horman, Luca Boccassi, Kevin Traynor,
Ananyev, Konstantin, Akhil Goyal, Trahe, Fiona, Ferruh Yigit,
dev, Anoob Joseph, Kusztal, ArkadiuszX, Richardson, Bruce,
Mcnamara, John, dodji, Andrew Rybchenko, Aaron Conole
On Mon, Feb 3, 2020 at 7:56 PM Ray Kinsella <mdr@ashroe.eu> wrote:
> On 03/02/2020 17:34, Thomas Monjalon wrote:
> > 03/02/2020 18:09, Thomas Monjalon:
> >> 03/02/2020 10:30, Ferruh Yigit:
> >>> On 2/2/2020 2:41 PM, Ananyev, Konstantin wrote:
> >>>> 02/02/2020 14:05, Thomas Monjalon:
> >>>>> 31/01/2020 15:16, Trahe, Fiona:
> >>>>>> On 1/30/2020 8:18 PM, Thomas Monjalon wrote:
> >>>>>>> If library give higher value than expected by the application,
> >>>>>>> if the application uses this value as array index,
> >>>>>>> there can be an access out of bounds.
> >>>>>>
> >>>>>> [Fiona] All asymmetric APIs are experimental so above shouldn't be a problem.
> >>>>>> But for the same issue with sym crypto below, I believe Ferruh's explanation makes
> >>>>>> sense and I don't see how there can be an API breakage.
> >>>>>> So if an application hasn't compiled against the new lib it will be still using the old value
> >>>>>> which will be within bounds. If it's picking up the higher new value from the lib it must
> >>>>>> have been compiled against the lib so shouldn't have problems.
> >>>>>
> >>>>> You say there is no ABI issue because the application will be re-compiled
> >>>>> for the updated library. Indeed, compilation fixes compatibility issues.
> >>>>> But this is not relevant for ABI compatibility.
> >>>>> ABI compatibility means we can upgrade the library without recompiling
> >>>>> the application and it must work.
> >>>>> You think it is a false positive because you assume the application
> >>>>> "picks" the new value. I think you miss the case where the new value
> >>>>> is returned by a function in the upgraded library.
> >>>>>
> >>>>>> There are also no structs on the API which contain arrays using this
> >>>>>> for sizing, so I don't see an opportunity for an appl to have a
> >>>>>> mismatch in memory addresses.
> >>>>>
> >>>>> Let me demonstrate where the API may "use" the new value
> >>>>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 and how it impacts the application.
> >>>>>
> >>>>> Once upon a time a DPDK application counting the number of devices
> >>>>> supporting each AEAD algo (in order to find the best supported algo).
> >>>>> It is done in an array indexed by algo id:
> >>>>> int aead_dev_count[RTE_CRYPTO_AEAD_LIST_END];
> >>>>> The application is compiled with DPDK 19.11,
> >>>>> where RTE_CRYPTO_AEAD_LIST_END = 3.
> >>>>> So the size of the application array aead_dev_count is 3.
> >>>>> This binary is run with DPDK 20.02,
> >>>>> where RTE_CRYPTO_AEAD_CHACHA20_POLY1305 = 3.
> >>>>> When calling rte_cryptodev_info_get() on a device QAT_GEN3,
> >>>>> rte_cryptodev_info.capabilities.sym.aead.algo is set to
> >>>>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 (= 3).
> >>>>> The application uses this value:
> >>>>> ++ aead_dev_count[info.capabilities.sym.aead.algo];
> >>>>> The application is crashing because of out of bound access.
> >>>>
> >>>> I'd say this is an example of bad written app.
> >>>> It probably should check that returned by library value doesn't
> >>>> exceed its internal array size.
> >>>
> >>> +1
> >>>
> >>> Application should ignore values >= MAX.
> >>
> >> Of course, blaming the API user is a lot easier than looking at the API.
> >> Here the API has RTE_CRYPTO_AEAD_LIST_END which can be understood
> >> as the max value for the application.
> >> Value ranges are part of the ABI compatibility contract.
> >> It seems you expect the application developer to be aware that
> >> DPDK could return a higher value, so the application should
> >> check every enum values after calling an API. CRAZY.
> >>
> >> When we decide to announce an ABI compatibility and do some marketing,
> >> everyone is OK. But when we need to really make our ABI compatible,
> >> I see little or no effort. DISAPPOINTING.
> >>
> >>> Do you suggest we don't extend any enum or define between ABI breakage releases
> >>> to be sure bad written applications not affected?
> >>
> >> I suggest we must consider not breaking any assumption made on the API.
> >> Here we are breaking the enum range because nothing mentions _LIST_END
> >> is not really the absolute end of the enum.
> >> The solution is to make the change below in 20.02 + backport in 19.11.1:
> >
> > Thinking twice, merging such change before 20.11 is breaking the
> > ABI assumption based on the API 19.11.0.
> > I ask the release maintainers (Luca, Kevin, David and me) and
> > the ABI maintainers (Neil and Ray) to vote for a or b solution:
> > a) add comment and LIST_MAX as below in 20.02 + 19.11.1
>
> That would still be an ABI breakage though right.
Yes.
>
> > b) wait 20.11 and revert Chacha-Poly from 20.02
>
> Thanks for analysis above Fiona, Ferruh and all.
>
> That is a nasty one alright - there is no "good" answer here.
> I agree with Ferruh's sentiments overall, we should rethink this API for 20.11.
> Could do without an enumeration?
>
> There a c) though right.
> We could work around the issue by api versioning rte_cryptodev_info_get() and friends.
It has a lot of friends, but it sounds like the right approach.
Is someone looking into this?
> So they only support/acknowledge the existence of Chacha-Poly for applications build against > 20.02.
>
> It would be painful I know.
> It would also mean that Chacha-Poly would only be available to those building against >= 20.02.
Yes.
--
David Marchand
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH v7] eal: add manual probing option
2020-02-03 22:21 3% ` Thomas Monjalon
@ 2020-02-04 10:03 0% ` Gaetan Rivet
2020-02-04 11:07 0% ` Thomas Monjalon
0 siblings, 1 reply; 200+ results
From: Gaetan Rivet @ 2020-02-04 10:03 UTC (permalink / raw)
To: Thomas Monjalon, Pavan Nikhilesh Bhagavatula
Cc: dev, David Marchand, Vamsi Krishna Attunuru, Jerin Jacob Kollanukkaran
On 03/02/2020 23:21, Thomas Monjalon wrote:
> 03/02/2020 06:16, Pavan Nikhilesh Bhagavatula:
>> @David Marchand @thomas@monjalon.net
>>
>> Ping?
>>
>> Are there any more changes required for this patch? It's been in queue since last October.
>
> Sorry we have not decided whether it is a good idea or not.
>
> All changes related to probing are very sensitive,
> and we know a big refactoring would be better than stacking
> more and more options and corner cases.
>
> As we are busy with ABI stability stuff, we did not allocate
> enough time to properly think about this feature.
> Please accept our apologies, and let's consider it as
> a high priority for 20.05 cycle.
>
>
>
Hello Thomas,
This is unfortunate. I pushed Pavan to accept an alternative implementation of this functionality that was less obtrusive, to make the integration smoother. I took care to alleviate those risks from the common path.
The big refactoring is needed yes, but considering the current path I'm not seeing it happen in 20.05. If that means taking this patch as-is in 20.05 for Marvell users, I'm not sure much is gained from waiting 3 months, except minimal risk avoidance.
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
2020-02-04 9:56 4% ` Ferruh Yigit
@ 2020-02-04 10:08 4% ` Bruce Richardson
2020-02-04 10:17 4% ` Kevin Traynor
1 sibling, 0 replies; 200+ results
From: Bruce Richardson @ 2020-02-04 10:08 UTC (permalink / raw)
To: Ferruh Yigit
Cc: Thomas Monjalon, dev, Ananyev, Konstantin, Akhil Goyal, Trahe,
Fiona, David Marchand, Anoob Joseph, Kusztal, ArkadiuszX,
nhorman, Mcnamara, John, dodji, Andrew Rybchenko, aconole, bluca,
ktraynor
On Tue, Feb 04, 2020 at 09:56:31AM +0000, Ferruh Yigit wrote:
> On 2/4/2020 9:45 AM, Thomas Monjalon wrote:
> > 04/02/2020 10:19, Ferruh Yigit:
> >> On 2/3/2020 6:40 PM, Thomas Monjalon wrote:
> >>> 03/02/2020 18:40, Ferruh Yigit:
> >>>> On 2/3/2020 5:09 PM, Thomas Monjalon wrote:
> >>>>> 03/02/2020 10:30, Ferruh Yigit:
> >>>>>> On 2/2/2020 2:41 PM, Ananyev, Konstantin wrote:
> >>>>>>> 02/02/2020 14:05, Thomas Monjalon:
> >>>>>>>> 31/01/2020 15:16, Trahe, Fiona:
> >>>>>>>>> On 1/30/2020 8:18 PM, Thomas Monjalon wrote:
> >>>>>>>>>> 30/01/2020 17:09, Ferruh Yigit:
> >>>>>>>>>>> On 1/29/2020 8:13 PM, Akhil Goyal wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>> I believe these enums will be used only in case of ASYM case which is experimental.
> >>>>>>>>>>>
> >>>>>>>>>>> Independent from being experiment and not, this shouldn't be a problem, I think
> >>>>>>>>>>> this is a false positive.
> >>>>>>>>>>>
> >>>>>>>>>>> The ABI break can happen when a struct has been shared between the application
> >>>>>>>>>>> and the library (DPDK) and the layout of that memory know differently by
> >>>>>>>>>>> application and the library.
> >>>>>>>>>>>
> >>>>>>>>>>> Here in all cases, there is no layout/size change.
> >>>>>>>>>>>
> >>>>>>>>>>> As to the value changes of the enums, since application compiled with old DPDK,
> >>>>>>>>>>> it will know only up to '6', 7 and more means invalid to the application. So it
> >>>>>>>>>>> won't send these values also it should ignore these values from library. Only
> >>>>>>>>>>> consequence is old application won't able to use new features those new enums
> >>>>>>>>>>> provide but that is expected/normal.
> >>>>>>>>>>
> >>>>>>>>>> If library give higher value than expected by the application,
> >>>>>>>>>> if the application uses this value as array index,
> >>>>>>>>>> there can be an access out of bounds.
> >>>>>>>>>
> >>>>>>>>> [Fiona] All asymmetric APIs are experimental so above shouldn't be a problem.
> >>>>>>>>> But for the same issue with sym crypto below, I believe Ferruh's explanation makes
> >>>>>>>>> sense and I don't see how there can be an API breakage.
> >>>>>>>>> So if an application hasn't compiled against the new lib it will be still using the old value
> >>>>>>>>> which will be within bounds. If it's picking up the higher new value from the lib it must
> >>>>>>>>> have been compiled against the lib so shouldn't have problems.
> >>>>>>>>
> >>>>>>>> You say there is no ABI issue because the application will be re-compiled
> >>>>>>>> for the updated library. Indeed, compilation fixes compatibility issues.
> >>>>>>>> But this is not relevant for ABI compatibility.
> >>>>>>>> ABI compatibility means we can upgrade the library without recompiling
> >>>>>>>> the application and it must work.
> >>>>>>>> You think it is a false positive because you assume the application
> >>>>>>>> "picks" the new value. I think you miss the case where the new value
> >>>>>>>> is returned by a function in the upgraded library.
> >>>>>>>>
> >>>>>>>>> There are also no structs on the API which contain arrays using this
> >>>>>>>>> for sizing, so I don't see an opportunity for an appl to have a
> >>>>>>>>> mismatch in memory addresses.
> >>>>>>>>
> >>>>>>>> Let me demonstrate where the API may "use" the new value
> >>>>>>>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 and how it impacts the application.
> >>>>>>>>
> >>>>>>>> Once upon a time a DPDK application counting the number of devices
> >>>>>>>> supporting each AEAD algo (in order to find the best supported algo).
> >>>>>>>> It is done in an array indexed by algo id:
> >>>>>>>> int aead_dev_count[RTE_CRYPTO_AEAD_LIST_END];
> >>>>>>>> The application is compiled with DPDK 19.11,
> >>>>>>>> where RTE_CRYPTO_AEAD_LIST_END = 3.
> >>>>>>>> So the size of the application array aead_dev_count is 3.
> >>>>>>>> This binary is run with DPDK 20.02,
> >>>>>>>> where RTE_CRYPTO_AEAD_CHACHA20_POLY1305 = 3.
> >>>>>>>> When calling rte_cryptodev_info_get() on a device QAT_GEN3,
> >>>>>>>> rte_cryptodev_info.capabilities.sym.aead.algo is set to
> >>>>>>>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 (= 3).
> >>>>>>>> The application uses this value:
> >>>>>>>> ++ aead_dev_count[info.capabilities.sym.aead.algo];
> >>>>>>>> The application is crashing because of out of bound access.
> >>>>>>>
> >>>>>>> I'd say this is an example of bad written app.
> >>>>>>> It probably should check that returned by library value doesn't
> >>>>>>> exceed its internal array size.
> >>>>>>
> >>>>>> +1
> >>>>>>
> >>>>>> Application should ignore values >= MAX.
> >>>>>
> >>>>> Of course, blaming the API user is a lot easier than looking at the API.
> >>>>> Here the API has RTE_CRYPTO_AEAD_LIST_END which can be understood
> >>>>> as the max value for the application.
> >>>>> Value ranges are part of the ABI compatibility contract.
> >>>>> It seems you expect the application developer to be aware that
> >>>>> DPDK could return a higher value, so the application should
> >>>>> check every enum values after calling an API. CRAZY.
> >>>>>
> >>>>> When we decide to announce an ABI compatibility and do some marketing,
> >>>>> everyone is OK. But when we need to really make our ABI compatible,
> >>>>> I see little or no effort. DISAPPOINTING.
> >>>>
> >>>> This is not to blame the user or to do less work, this is more sane approach
> >>>> that library provides the _END/_MAX value and application uses it as valid range
> >>>> check.
> >>>>
> >>>>>> Do you suggest we don't extend any enum or define between ABI breakage releases
> >>>>>> to be sure bad written applications not affected?
> >>>>>
> >>>>> I suggest we must consider not breaking any assumption made on the API.
> >>>>> Here we are breaking the enum range because nothing mentions _LIST_END
> >>>>> is not really the absolute end of the enum.
> >>>>> The solution is to make the change below in 20.02 + backport in 19.11.1:
> >>>>>
> >>>>> - _LIST_END
> >>>>> + _LIST_END, /* an ABI-compatible version may increase this value */
> >>>>> + _LIST_MAX = _LIST_END + 42 /* room for ABI-compatible additions */
> >>>>> };
> >>>>>
> >>>>
> >>>> What is the point of "_LIST_MAX" here?
> >>>
> >>> _LIST_MAX is range of value that DPDK can return in the ABI contract.
> >>> So the appplication can rely on the range 0.._LIST_MAX.
> >>>
> >>>> Application should know the "_LIST_END" of when it has been compiled for the
> >>>> valid range check. Next time it is compiled "_LIST_END" may be different value
> >>>> but same logic applies.
> >>>
> >>> No, ABI compatibility contract means you can compile your application
> >>> with DPDK 19.11.0 and run it with DPDK 20.02.
> >>> So _LIST_END comes from 19.11 and does not include ChachaPoly.
> >>
> >> That is what I mean, let me try to give a sample.
> >>
> >> DPDK19.11 returns, A=1, B=2, END=3
> >>
> >> Application compiled with DPDK19.11, it will process A, B and ignore anything ">= 3"
> >
> > No, the application will not ignore anything ">=3" as I explained above,
> > and you blamed the application for it.
> > Nothing in the API says the application must filter value higher than 3,
> > because as of now, values higher than 3 are PMD bug.
>
> When application compiled, that is the END value, anything bigger than this
> value is not valid, if any application use the return value directly I think it
> is doing something wrong.
> But yes there may be applications relying on library will always send in the
> range. We never communicated this. But we can add comments to clarify this.
>
I don't think we should do so, as for any function returning an enum by
definition it should never return an out-of-range value. I strongly agree
with the suggestion of versioning the functions so that the ranges seen by
apps are clamped to the expected 19.11 compatible values.
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
2020-02-04 9:51 4% ` David Marchand
@ 2020-02-04 10:10 4% ` Trahe, Fiona
2020-02-04 10:38 4% ` Thomas Monjalon
2020-02-05 11:10 4% ` Ray Kinsella
1 sibling, 1 reply; 200+ results
From: Trahe, Fiona @ 2020-02-04 10:10 UTC (permalink / raw)
To: David Marchand, Ray Kinsella
Cc: Thomas Monjalon, Neil Horman, Luca Boccassi, Kevin Traynor,
Ananyev, Konstantin, Akhil Goyal, Yigit, Ferruh, dev,
Anoob Joseph, Kusztal, ArkadiuszX, Richardson, Bruce, Mcnamara,
John, dodji, Andrew Rybchenko, Aaron Conole, Trahe, Fiona
And not used for sizing > >
> > There a c) though right.
> > We could work around the issue by api versioning rte_cryptodev_info_get() and friends.
>
> It has a lot of friends, but it sounds like the right approach.
> Is someone looking into this?
[Fiona] Yes. But not clear yet if can be done by tomorrow.
But even if feasible, that only works around the current issue.
There is a bigger issue to be decided here -
Should we be removing LIST_END/MAX values from all enums in 20.11?
Or defining through API comment that they should only be used as a range boundary and
NOT to size an array. And so having a fixed value is not part of the API contract.
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
2020-02-03 17:40 8% ` Ferruh Yigit
2020-02-03 18:40 9% ` Thomas Monjalon
@ 2020-02-04 10:16 9% ` Akhil Goyal
2020-02-04 10:28 8% ` Thomas Monjalon
2020-02-04 21:59 8% ` [dpdk-dev] " Neil Horman
1 sibling, 2 replies; 200+ results
From: Akhil Goyal @ 2020-02-04 10:16 UTC (permalink / raw)
To: Ferruh Yigit, Thomas Monjalon, Ananyev, Konstantin, Trahe, Fiona
Cc: David Marchand, Anoob Joseph, Kusztal, ArkadiuszX, dev,
Richardson, Bruce, nhorman, Mcnamara, John, dodji,
Andrew Rybchenko, aconole, bluca, ktraynor
Hi,
> On 2/3/2020 5:09 PM, Thomas Monjalon wrote:
> > 03/02/2020 10:30, Ferruh Yigit:
> >> On 2/2/2020 2:41 PM, Ananyev, Konstantin wrote:
> >>> 02/02/2020 14:05, Thomas Monjalon:
> >>>> 31/01/2020 15:16, Trahe, Fiona:
> >>>>> On 1/30/2020 8:18 PM, Thomas Monjalon wrote:
> >>>>>> 30/01/2020 17:09, Ferruh Yigit:
> >>>>>>> On 1/29/2020 8:13 PM, Akhil Goyal wrote:
> >>>>>>>>
> >>>>>>>> I believe these enums will be used only in case of ASYM case which is
> experimental.
> >>>>>>>
> >>>>>>> Independent from being experiment and not, this shouldn't be a
> problem, I think
> >>>>>>> this is a false positive.
> >>>>>>>
> >>>>>>> The ABI break can happen when a struct has been shared between the
> application
> >>>>>>> and the library (DPDK) and the layout of that memory know differently
> by
> >>>>>>> application and the library.
> >>>>>>>
> >>>>>>> Here in all cases, there is no layout/size change.
> >>>>>>>
> >>>>>>> As to the value changes of the enums, since application compiled with
> old DPDK,
> >>>>>>> it will know only up to '6', 7 and more means invalid to the application.
> So it
> >>>>>>> won't send these values also it should ignore these values from library.
> Only
> >>>>>>> consequence is old application won't able to use new features those
> new enums
> >>>>>>> provide but that is expected/normal.
> >>>>>>
> >>>>>> If library give higher value than expected by the application,
> >>>>>> if the application uses this value as array index,
> >>>>>> there can be an access out of bounds.
> >>>>>
> >>>>> [Fiona] All asymmetric APIs are experimental so above shouldn't be a
> problem.
> >>>>> But for the same issue with sym crypto below, I believe Ferruh's
> explanation makes
> >>>>> sense and I don't see how there can be an API breakage.
> >>>>> So if an application hasn't compiled against the new lib it will be still using
> the old value
> >>>>> which will be within bounds. If it's picking up the higher new value from
> the lib it must
> >>>>> have been compiled against the lib so shouldn't have problems.
> >>>>
> >>>> You say there is no ABI issue because the application will be re-compiled
> >>>> for the updated library. Indeed, compilation fixes compatibility issues.
> >>>> But this is not relevant for ABI compatibility.
> >>>> ABI compatibility means we can upgrade the library without recompiling
> >>>> the application and it must work.
> >>>> You think it is a false positive because you assume the application
> >>>> "picks" the new value. I think you miss the case where the new value
> >>>> is returned by a function in the upgraded library.
> >>>>
> >>>>> There are also no structs on the API which contain arrays using this
> >>>>> for sizing, so I don't see an opportunity for an appl to have a
> >>>>> mismatch in memory addresses.
> >>>>
> >>>> Let me demonstrate where the API may "use" the new value
> >>>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 and how it impacts the
> application.
> >>>>
> >>>> Once upon a time a DPDK application counting the number of devices
> >>>> supporting each AEAD algo (in order to find the best supported algo).
> >>>> It is done in an array indexed by algo id:
> >>>> int aead_dev_count[RTE_CRYPTO_AEAD_LIST_END];
> >>>> The application is compiled with DPDK 19.11,
> >>>> where RTE_CRYPTO_AEAD_LIST_END = 3.
> >>>> So the size of the application array aead_dev_count is 3.
> >>>> This binary is run with DPDK 20.02,
> >>>> where RTE_CRYPTO_AEAD_CHACHA20_POLY1305 = 3.
> >>>> When calling rte_cryptodev_info_get() on a device QAT_GEN3,
> >>>> rte_cryptodev_info.capabilities.sym.aead.algo is set to
> >>>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 (= 3).
> >>>> The application uses this value:
> >>>> ++ aead_dev_count[info.capabilities.sym.aead.algo];
> >>>> The application is crashing because of out of bound access.
> >>>
> >>> I'd say this is an example of bad written app.
> >>> It probably should check that returned by library value doesn't
> >>> exceed its internal array size.
> >>
> >> +1
> >>
> >> Application should ignore values >= MAX.
> >
> > Of course, blaming the API user is a lot easier than looking at the API.
> > Here the API has RTE_CRYPTO_AEAD_LIST_END which can be understood
> > as the max value for the application.
> > Value ranges are part of the ABI compatibility contract.
> > It seems you expect the application developer to be aware that
> > DPDK could return a higher value, so the application should
> > check every enum values after calling an API. CRAZY.
> >
> > When we decide to announce an ABI compatibility and do some marketing,
> > everyone is OK. But when we need to really make our ABI compatible,
> > I see little or no effort. DISAPPOINTING.
>
> This is not to blame the user or to do less work, this is more sane approach
> that library provides the _END/_MAX value and application uses it as valid range
> check.
>
> >
> >> Do you suggest we don't extend any enum or define between ABI breakage
> releases
> >> to be sure bad written applications not affected?
> >
> > I suggest we must consider not breaking any assumption made on the API.
> > Here we are breaking the enum range because nothing mentions _LIST_END
> > is not really the absolute end of the enum.
> > The solution is to make the change below in 20.02 + backport in 19.11.1:
> >
> > - _LIST_END
> > + _LIST_END, /* an ABI-compatible version may increase this value */
> > + _LIST_MAX = _LIST_END + 42 /* room for ABI-compatible additions */
> > };
> >
>
> What is the point of "_LIST_MAX" here?
>
> Application should know the "_LIST_END" of when it has been compiled for the
> valid range check. Next time it is compiled "_LIST_END" may be different value
> but same logic applies.
>
> When "_LIST_END" is missing, application can't protect itself, in that case
> library should send only the values application knows when it is compiled, this
> means either we can't extend our enum/defines until next ABI breakage, or we
> need to do ABI versioning to the functions that returns an enum each time enum
> value extended.
>
> I believe it is saner to provide _END/_MAX values to the application to use. And
> if required comment them to clarify the expected usage.
>
> But in above suggestion application can't use or rely on "_LIST_MAX", it doesn't
> mean anything to application.
>
Can we have something like
enum rte_crypto_aead_algorithm {
RTE_CRYPTO_AEAD_AES_CCM = 1,
/**< AES algorithm in CCM mode. */
RTE_CRYPTO_AEAD_AES_GCM,
/**< AES algorithm in GCM mode. */
RTE_CRYPTO_AEAD_LIST_END,
/**< List end for 19.11 ABI compatibility */
RTE_CRYPTO_AEAD_CHACHA20_POLY1305,
/**< Chacha20 cipher with poly1305 authenticator */
RTE_CRYPTO_AEAD_LIST_END_2011
/**< List end for 20.11 ABI compatibility */
};
And in 20.11 release we alter the RTE_CRYPTO_AEAD_LIST_END to the end and remove RTE_CRYPTO_AEAD_LIST_END_2011
I believe it will be ok for any application which need to use the chacha poly assume that this algo is
Experimental and will move to formal list in 20.11. This can be documented in the documentation.
I believe there is no way to add a new enum as experimental so far. This way we can formalize this
requirement as well.
I believe this way effect of ABI breakage will be nullified.
-Akhil
> > Then *_LIST_END values could be ignored by libabigail with such a change.
> >
> > If such a patch is not done by tomorrow, I will have to revert
> > Chacha-Poly commits before 20.02-rc2, because
> >
> > 1/ LIST_END, without any comment, means "size of range"
> > 2/ we do not blame users for undocumented ABI changes
> > 3/ we respect the ABI compatibility contract
> >
> >
^ permalink raw reply [relevance 9%]
* Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
2020-02-04 9:56 4% ` Ferruh Yigit
2020-02-04 10:08 4% ` Bruce Richardson
@ 2020-02-04 10:17 4% ` Kevin Traynor
1 sibling, 0 replies; 200+ results
From: Kevin Traynor @ 2020-02-04 10:17 UTC (permalink / raw)
To: Ferruh Yigit, Thomas Monjalon
Cc: dev, Ananyev, Konstantin, Akhil Goyal, Trahe, Fiona,
David Marchand, Anoob Joseph, Kusztal, ArkadiuszX, Richardson,
Bruce, nhorman, Mcnamara, John, dodji, Andrew Rybchenko, aconole,
bluca
On 04/02/2020 09:56, Ferruh Yigit wrote:
> On 2/4/2020 9:45 AM, Thomas Monjalon wrote:
>> 04/02/2020 10:19, Ferruh Yigit:
>>> On 2/3/2020 6:40 PM, Thomas Monjalon wrote:
>>>> 03/02/2020 18:40, Ferruh Yigit:
>>>>> On 2/3/2020 5:09 PM, Thomas Monjalon wrote:
>>>>>> 03/02/2020 10:30, Ferruh Yigit:
>>>>>>> On 2/2/2020 2:41 PM, Ananyev, Konstantin wrote:
>>>>>>>> 02/02/2020 14:05, Thomas Monjalon:
>>>>>>>>> 31/01/2020 15:16, Trahe, Fiona:
>>>>>>>>>> On 1/30/2020 8:18 PM, Thomas Monjalon wrote:
>>>>>>>>>>> 30/01/2020 17:09, Ferruh Yigit:
>>>>>>>>>>>> On 1/29/2020 8:13 PM, Akhil Goyal wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> I believe these enums will be used only in case of ASYM case which is experimental.
>>>>>>>>>>>>
>>>>>>>>>>>> Independent from being experiment and not, this shouldn't be a problem, I think
>>>>>>>>>>>> this is a false positive.
>>>>>>>>>>>>
>>>>>>>>>>>> The ABI break can happen when a struct has been shared between the application
>>>>>>>>>>>> and the library (DPDK) and the layout of that memory know differently by
>>>>>>>>>>>> application and the library.
>>>>>>>>>>>>
>>>>>>>>>>>> Here in all cases, there is no layout/size change.
>>>>>>>>>>>>
>>>>>>>>>>>> As to the value changes of the enums, since application compiled with old DPDK,
>>>>>>>>>>>> it will know only up to '6', 7 and more means invalid to the application. So it
>>>>>>>>>>>> won't send these values also it should ignore these values from library. Only
>>>>>>>>>>>> consequence is old application won't able to use new features those new enums
>>>>>>>>>>>> provide but that is expected/normal.
>>>>>>>>>>>
>>>>>>>>>>> If library give higher value than expected by the application,
>>>>>>>>>>> if the application uses this value as array index,
>>>>>>>>>>> there can be an access out of bounds.
>>>>>>>>>>
>>>>>>>>>> [Fiona] All asymmetric APIs are experimental so above shouldn't be a problem.
>>>>>>>>>> But for the same issue with sym crypto below, I believe Ferruh's explanation makes
>>>>>>>>>> sense and I don't see how there can be an API breakage.
>>>>>>>>>> So if an application hasn't compiled against the new lib it will be still using the old value
>>>>>>>>>> which will be within bounds. If it's picking up the higher new value from the lib it must
>>>>>>>>>> have been compiled against the lib so shouldn't have problems.
>>>>>>>>>
>>>>>>>>> You say there is no ABI issue because the application will be re-compiled
>>>>>>>>> for the updated library. Indeed, compilation fixes compatibility issues.
>>>>>>>>> But this is not relevant for ABI compatibility.
>>>>>>>>> ABI compatibility means we can upgrade the library without recompiling
>>>>>>>>> the application and it must work.
>>>>>>>>> You think it is a false positive because you assume the application
>>>>>>>>> "picks" the new value. I think you miss the case where the new value
>>>>>>>>> is returned by a function in the upgraded library.
>>>>>>>>>
>>>>>>>>>> There are also no structs on the API which contain arrays using this
>>>>>>>>>> for sizing, so I don't see an opportunity for an appl to have a
>>>>>>>>>> mismatch in memory addresses.
>>>>>>>>>
>>>>>>>>> Let me demonstrate where the API may "use" the new value
>>>>>>>>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 and how it impacts the application.
>>>>>>>>>
>>>>>>>>> Once upon a time a DPDK application counting the number of devices
>>>>>>>>> supporting each AEAD algo (in order to find the best supported algo).
>>>>>>>>> It is done in an array indexed by algo id:
>>>>>>>>> int aead_dev_count[RTE_CRYPTO_AEAD_LIST_END];
>>>>>>>>> The application is compiled with DPDK 19.11,
>>>>>>>>> where RTE_CRYPTO_AEAD_LIST_END = 3.
>>>>>>>>> So the size of the application array aead_dev_count is 3.
>>>>>>>>> This binary is run with DPDK 20.02,
>>>>>>>>> where RTE_CRYPTO_AEAD_CHACHA20_POLY1305 = 3.
>>>>>>>>> When calling rte_cryptodev_info_get() on a device QAT_GEN3,
>>>>>>>>> rte_cryptodev_info.capabilities.sym.aead.algo is set to
>>>>>>>>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 (= 3).
>>>>>>>>> The application uses this value:
>>>>>>>>> ++ aead_dev_count[info.capabilities.sym.aead.algo];
>>>>>>>>> The application is crashing because of out of bound access.
>>>>>>>>
>>>>>>>> I'd say this is an example of bad written app.
>>>>>>>> It probably should check that returned by library value doesn't
>>>>>>>> exceed its internal array size.
>>>>>>>
>>>>>>> +1
>>>>>>>
>>>>>>> Application should ignore values >= MAX.
>>>>>>
>>>>>> Of course, blaming the API user is a lot easier than looking at the API.
>>>>>> Here the API has RTE_CRYPTO_AEAD_LIST_END which can be understood
>>>>>> as the max value for the application.
>>>>>> Value ranges are part of the ABI compatibility contract.
>>>>>> It seems you expect the application developer to be aware that
>>>>>> DPDK could return a higher value, so the application should
>>>>>> check every enum values after calling an API. CRAZY.
>>>>>>
>>>>>> When we decide to announce an ABI compatibility and do some marketing,
>>>>>> everyone is OK. But when we need to really make our ABI compatible,
>>>>>> I see little or no effort. DISAPPOINTING.
>>>>>
>>>>> This is not to blame the user or to do less work, this is more sane approach
>>>>> that library provides the _END/_MAX value and application uses it as valid range
>>>>> check.
>>>>>
>>>>>>> Do you suggest we don't extend any enum or define between ABI breakage releases
>>>>>>> to be sure bad written applications not affected?
>>>>>>
>>>>>> I suggest we must consider not breaking any assumption made on the API.
>>>>>> Here we are breaking the enum range because nothing mentions _LIST_END
>>>>>> is not really the absolute end of the enum.
>>>>>> The solution is to make the change below in 20.02 + backport in 19.11.1:
>>>>>>
>>>>>> - _LIST_END
>>>>>> + _LIST_END, /* an ABI-compatible version may increase this value */
>>>>>> + _LIST_MAX = _LIST_END + 42 /* room for ABI-compatible additions */
>>>>>> };
>>>>>>
>>>>>
>>>>> What is the point of "_LIST_MAX" here?
>>>>
>>>> _LIST_MAX is range of value that DPDK can return in the ABI contract.
>>>> So the appplication can rely on the range 0.._LIST_MAX.
>>>>
>>>>> Application should know the "_LIST_END" of when it has been compiled for the
>>>>> valid range check. Next time it is compiled "_LIST_END" may be different value
>>>>> but same logic applies.
>>>>
>>>> No, ABI compatibility contract means you can compile your application
>>>> with DPDK 19.11.0 and run it with DPDK 20.02.
>>>> So _LIST_END comes from 19.11 and does not include ChachaPoly.
>>>
>>> That is what I mean, let me try to give a sample.
>>>
>>> DPDK19.11 returns, A=1, B=2, END=3
>>>
>>> Application compiled with DPDK19.11, it will process A, B and ignore anything ">= 3"
>>
>> No, the application will not ignore anything ">=3" as I explained above,
>> and you blamed the application for it.
>> Nothing in the API says the application must filter value higher than 3,
>> because as of now, values higher than 3 are PMD bug.
>
> When application compiled, that is the END value, anything bigger than this
> value is not valid, if any application use the return value directly I think it
> is doing something wrong.
I don't think we can make an assumption that the application will/should
range check *and* silently ignore what it considers out of bounds values.
An application may not range check, but if it does, it may ignore these
new values (we got lucky), or it could print errors or even decide to
abort because it considers that DPDK is now returning values higher than
the (compiled) max so something must be corrupt.
Versioning sounds the best solution to me too, but I'm not sure how
difficult the mechanics are in this case.
> But yes there may be applications relying on library will always send in the
> range. We never communicated this. But we can add comments to clarify this.
>
>>
>>
>>> DPDK20.02 returns A=1, B=2, C=3, D=4, END=5
>>>
>>> Old application will still only will know/use A, B and can ignore when library
>>> sends C=3, D=4 etc...
>>>
>>>
>>> In above, if you add another limit as you suggested, like MAX=10 and ask
>>> application to use it,
>>>
>>> Application compiled with DPDK19.11 will be OK since library only sends A,B and
>>> application uses them.
>>>
>>> But with DPDK20.02 application may have problem, since library will be sending
>>> C=3, which is valid according to the check " <= MAX (10)", how application will
>>> know to ignore it.
>>
>> Why application should ignore value C=3 with DPDK 20.02?
>
> This is the application compiled with DPDK19.11, and running with DPDK20.02.
>
> So for the application this is the value >= MAX and something it doesn't know
> what to do.
>
>>
>>
>>> So application should use _END to know the valid ones according it, if so what
>>> is the point of having _MAX.
>>>
>>>
>>>>> When "_LIST_END" is missing, application can't protect itself, in that case
>>>>> library should send only the values application knows when it is compiled, this
>>>>> means either we can't extend our enum/defines until next ABI breakage, or we
>>>>> need to do ABI versioning to the functions that returns an enum each time enum
>>>>> value extended.
>>>>
>>>> If we define _LIST_MAX as a bigger value than current _LIST_END,
>>>> we have some room to add values in between.
>>>>
>>>> If (as of now) we don't have _LIST_MAX room, then yes we must version
>>>> the functions returning the enum.
>>>> In this case, the proper solution is to implement
>>>> rte_cryptodev_info_get_v1911() so it filters out
>>>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 capability.
>>>> With this solution, an application compiled with DPDK 19.11 will keep
>>>> seeing the same range as before, while a 20.02 application could
>>>> see and use ChachaPoly.
>>>> This is another proposal that I was expecting from the crypto team,
>>>> instead of claiming there is no issue (and wasting precious time).
>>>>
>>>>
>>>>> I believe it is saner to provide _END/_MAX values to the application to use. And
>>>>> if required comment them to clarify the expected usage.
>>>>>
>>>>> But in above suggestion application can't use or rely on "_LIST_MAX", it doesn't
>>>>> mean anything to application.
>>>>
>>>> I don't understand what you mean. I think you misunderstood what is ABI compat.
>>>>
>>>>
>>>>>> Then *_LIST_END values could be ignored by libabigail with such a change.
>>>>>>
>>>>>> If such a patch is not done by tomorrow, I will have to revert
>>>>>> Chacha-Poly commits before 20.02-rc2, because
>>>>>>
>>>>>> 1/ LIST_END, without any comment, means "size of range"
>>>>>> 2/ we do not blame users for undocumented ABI changes
>>>>>> 3/ we respect the ABI compatibility contract
>>
>>
>>
>
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
2020-02-03 21:07 7% ` Thomas Monjalon
2020-02-04 9:46 8% ` Ferruh Yigit
@ 2020-02-04 10:24 9% ` Thomas Monjalon
2020-02-04 12:44 4% ` Trahe, Fiona
` (2 more replies)
1 sibling, 3 replies; 200+ results
From: Thomas Monjalon @ 2020-02-04 10:24 UTC (permalink / raw)
To: David Marchand, nhorman, bluca, ktraynor, Ray Kinsella, dev,
Akhil Goyal, Trahe, Fiona, Ferruh Yigit
Cc: Ananyev, Konstantin, dev, Anoob Joseph, Kusztal, ArkadiuszX,
Richardson, Bruce, Mcnamara, John, dodji, Andrew Rybchenko,
aconole
RED FLAG
I don't see a lot of reactions, so I summarize the issue.
We need action TODAY!
API makes think that rte_cryptodev_info_get() cannot return
a value >= 3 (RTE_CRYPTO_AEAD_LIST_END in 19.11).
Current 20.02 returns 3 (RTE_CRYPTO_AEAD_CHACHA20_POLY1305).
The ABI compatibility contract is broken currently.
There are 3 possible outcomes:
a) Change the API comments and backport to 19.11.1
The details are discussed between Ferruh and me.
Either put responsibility on API user (with explicit comment),
or expose ABI extension allowance with a new API max value.
In both cases, this is breaking the implicit contract of 19.11.0.
This option can be chosen only if release and ABI maintainers
vote for it.
b) Revert Chacha-Poly from 20.02-rc2.
c) Add versioned function rte_cryptodev_info_get_v1911()
which calls rte_cryptodev_info_get() and filters out
RTE_CRYPTO_AEAD_CHACHA20_POLY1305 capability.
So Chacha-Poly capability would be seen and usable only
if compiling with DPDK 20.02.
I hope it is clear what are the actions for everybody:
- ABI and release maintainers must say yes or no to the proposal (a)
- In the meantime, crypto team must send a patch for the proposal (c)
- If (a) and (c) are not possible at the end of today, I will take (b)
Note: do not say it is too short for (c), as it was possible to work
on such solution since the issue was exposed on last Wednesday.
03/02/2020 22:07, Thomas Monjalon:
> 03/02/2020 19:55, Ray Kinsella:
> > On 03/02/2020 17:34, Thomas Monjalon wrote:
> > > 03/02/2020 18:09, Thomas Monjalon:
> > >> 03/02/2020 10:30, Ferruh Yigit:
> > >>> On 2/2/2020 2:41 PM, Ananyev, Konstantin wrote:
> > >>>> 02/02/2020 14:05, Thomas Monjalon:
> > >>>>> 31/01/2020 15:16, Trahe, Fiona:
> > >>>>>> On 1/30/2020 8:18 PM, Thomas Monjalon wrote:
> > >>>>>>> If library give higher value than expected by the application,
> > >>>>>>> if the application uses this value as array index,
> > >>>>>>> there can be an access out of bounds.
> > >>>>>>
> > >>>>>> [Fiona] All asymmetric APIs are experimental so above shouldn't be a problem.
> > >>>>>> But for the same issue with sym crypto below, I believe Ferruh's explanation makes
> > >>>>>> sense and I don't see how there can be an API breakage.
> > >>>>>> So if an application hasn't compiled against the new lib it will be still using the old value
> > >>>>>> which will be within bounds. If it's picking up the higher new value from the lib it must
> > >>>>>> have been compiled against the lib so shouldn't have problems.
> > >>>>>
> > >>>>> You say there is no ABI issue because the application will be re-compiled
> > >>>>> for the updated library. Indeed, compilation fixes compatibility issues.
> > >>>>> But this is not relevant for ABI compatibility.
> > >>>>> ABI compatibility means we can upgrade the library without recompiling
> > >>>>> the application and it must work.
> > >>>>> You think it is a false positive because you assume the application
> > >>>>> "picks" the new value. I think you miss the case where the new value
> > >>>>> is returned by a function in the upgraded library.
> > >>>>>
> > >>>>>> There are also no structs on the API which contain arrays using this
> > >>>>>> for sizing, so I don't see an opportunity for an appl to have a
> > >>>>>> mismatch in memory addresses.
> > >>>>>
> > >>>>> Let me demonstrate where the API may "use" the new value
> > >>>>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 and how it impacts the application.
> > >>>>>
> > >>>>> Once upon a time a DPDK application counting the number of devices
> > >>>>> supporting each AEAD algo (in order to find the best supported algo).
> > >>>>> It is done in an array indexed by algo id:
> > >>>>> int aead_dev_count[RTE_CRYPTO_AEAD_LIST_END];
> > >>>>> The application is compiled with DPDK 19.11,
> > >>>>> where RTE_CRYPTO_AEAD_LIST_END = 3.
> > >>>>> So the size of the application array aead_dev_count is 3.
> > >>>>> This binary is run with DPDK 20.02,
> > >>>>> where RTE_CRYPTO_AEAD_CHACHA20_POLY1305 = 3.
> > >>>>> When calling rte_cryptodev_info_get() on a device QAT_GEN3,
> > >>>>> rte_cryptodev_info.capabilities.sym.aead.algo is set to
> > >>>>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 (= 3).
> > >>>>> The application uses this value:
> > >>>>> ++ aead_dev_count[info.capabilities.sym.aead.algo];
> > >>>>> The application is crashing because of out of bound access.
> > >>>>
> > >>>> I'd say this is an example of bad written app.
> > >>>> It probably should check that returned by library value doesn't
> > >>>> exceed its internal array size.
> > >>>
> > >>> +1
> > >>>
> > >>> Application should ignore values >= MAX.
> > >>
> > >> Of course, blaming the API user is a lot easier than looking at the API.
> > >> Here the API has RTE_CRYPTO_AEAD_LIST_END which can be understood
> > >> as the max value for the application.
> > >> Value ranges are part of the ABI compatibility contract.
> > >> It seems you expect the application developer to be aware that
> > >> DPDK could return a higher value, so the application should
> > >> check every enum values after calling an API. CRAZY.
> > >>
> > >> When we decide to announce an ABI compatibility and do some marketing,
> > >> everyone is OK. But when we need to really make our ABI compatible,
> > >> I see little or no effort. DISAPPOINTING.
> > >>
> > >>> Do you suggest we don't extend any enum or define between ABI breakage releases
> > >>> to be sure bad written applications not affected?
> > >>
> > >> I suggest we must consider not breaking any assumption made on the API.
> > >> Here we are breaking the enum range because nothing mentions _LIST_END
> > >> is not really the absolute end of the enum.
> > >> The solution is to make the change below in 20.02 + backport in 19.11.1:
> > >
> > > Thinking twice, merging such change before 20.11 is breaking the
> > > ABI assumption based on the API 19.11.0.
> > > I ask the release maintainers (Luca, Kevin, David and me) and
> > > the ABI maintainers (Neil and Ray) to vote for a or b solution:
> > > a) add comment and LIST_MAX as below in 20.02 + 19.11.1
> >
> > That would still be an ABI breakage though right.
> >
> > > b) wait 20.11 and revert Chacha-Poly from 20.02
> >
> > Thanks for analysis above Fiona, Ferruh and all.
> >
> > That is a nasty one alright - there is no "good" answer here.
> > I agree with Ferruh's sentiments overall, we should rethink this API for 20.11.
> > Could do without an enumeration?
> >
> > There a c) though right.
> > We could work around the issue by api versioning rte_cryptodev_info_get() and friends.
> > So they only support/acknowledge the existence of Chacha-Poly for applications build against > 20.02.
>
> I agree there is a c) as I proposed in another email:
> http://mails.dpdk.org/archives/dev/2020-February/156919.html
> "
> In this case, the proper solution is to implement
> rte_cryptodev_info_get_v1911() so it filters out
> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 capability.
> With this solution, an application compiled with DPDK 19.11 will keep
> seeing the same range as before, while a 20.02 application could
> see and use ChachaPoly.
> "
>
> > It would be painful I know.
>
> Not so painful in my opinion.
> Just need to call rte_cryptodev_info_get() from
> rte_cryptodev_info_get_v1911() and filter the value
> in the 19.11 range: [0..AES_GCM].
>
> > It would also mean that Chacha-Poly would only be available to
> > those building against >= 20.02.
>
> Yes exactly.
>
> The addition of comments and LIST_MAX like below are still valid
> to avoid versioning after 20.11.
>
> > >> - _LIST_END
> > >> + _LIST_END, /* an ABI-compatible version may increase this value */
> > >> + _LIST_MAX = _LIST_END + 42 /* room for ABI-compatible additions */
> > >> };
> > >>
> > >> Then *_LIST_END values could be ignored by libabigail with such a change.
>
> In order to avoid ABI check complaining, the best is to completely
> remove LIST_END in DPDK 20.11.
>
>
> > >> If such a patch is not done by tomorrow, I will have to revert
> > >> Chacha-Poly commits before 20.02-rc2, because
> > >>
> > >> 1/ LIST_END, without any comment, means "size of range"
> > >> 2/ we do not blame users for undocumented ABI changes
> > >> 3/ we respect the ABI compatibility contract
^ permalink raw reply [relevance 9%]
* Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
2020-02-03 21:07 7% ` Thomas Monjalon
@ 2020-02-04 9:46 8% ` Ferruh Yigit
2020-02-04 10:24 9% ` Thomas Monjalon
1 sibling, 0 replies; 200+ results
From: Ferruh Yigit @ 2020-02-04 9:46 UTC (permalink / raw)
To: Thomas Monjalon, David Marchand, nhorman, bluca, ktraynor, Ray Kinsella
Cc: Ananyev, Konstantin, Akhil Goyal, Trahe, Fiona, dev,
Anoob Joseph, Kusztal, ArkadiuszX, Richardson, Bruce, Mcnamara,
John, dodji, Andrew Rybchenko, aconole
On 2/3/2020 9:07 PM, Thomas Monjalon wrote:
> 03/02/2020 19:55, Ray Kinsella:
>> On 03/02/2020 17:34, Thomas Monjalon wrote:
>>> 03/02/2020 18:09, Thomas Monjalon:
>>>> 03/02/2020 10:30, Ferruh Yigit:
>>>>> On 2/2/2020 2:41 PM, Ananyev, Konstantin wrote:
>>>>>> 02/02/2020 14:05, Thomas Monjalon:
>>>>>>> 31/01/2020 15:16, Trahe, Fiona:
>>>>>>>> On 1/30/2020 8:18 PM, Thomas Monjalon wrote:
>>>>>>>>> If library give higher value than expected by the application,
>>>>>>>>> if the application uses this value as array index,
>>>>>>>>> there can be an access out of bounds.
>>>>>>>>
>>>>>>>> [Fiona] All asymmetric APIs are experimental so above shouldn't be a problem.
>>>>>>>> But for the same issue with sym crypto below, I believe Ferruh's explanation makes
>>>>>>>> sense and I don't see how there can be an API breakage.
>>>>>>>> So if an application hasn't compiled against the new lib it will be still using the old value
>>>>>>>> which will be within bounds. If it's picking up the higher new value from the lib it must
>>>>>>>> have been compiled against the lib so shouldn't have problems.
>>>>>>>
>>>>>>> You say there is no ABI issue because the application will be re-compiled
>>>>>>> for the updated library. Indeed, compilation fixes compatibility issues.
>>>>>>> But this is not relevant for ABI compatibility.
>>>>>>> ABI compatibility means we can upgrade the library without recompiling
>>>>>>> the application and it must work.
>>>>>>> You think it is a false positive because you assume the application
>>>>>>> "picks" the new value. I think you miss the case where the new value
>>>>>>> is returned by a function in the upgraded library.
>>>>>>>
>>>>>>>> There are also no structs on the API which contain arrays using this
>>>>>>>> for sizing, so I don't see an opportunity for an appl to have a
>>>>>>>> mismatch in memory addresses.
>>>>>>>
>>>>>>> Let me demonstrate where the API may "use" the new value
>>>>>>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 and how it impacts the application.
>>>>>>>
>>>>>>> Once upon a time a DPDK application counting the number of devices
>>>>>>> supporting each AEAD algo (in order to find the best supported algo).
>>>>>>> It is done in an array indexed by algo id:
>>>>>>> int aead_dev_count[RTE_CRYPTO_AEAD_LIST_END];
>>>>>>> The application is compiled with DPDK 19.11,
>>>>>>> where RTE_CRYPTO_AEAD_LIST_END = 3.
>>>>>>> So the size of the application array aead_dev_count is 3.
>>>>>>> This binary is run with DPDK 20.02,
>>>>>>> where RTE_CRYPTO_AEAD_CHACHA20_POLY1305 = 3.
>>>>>>> When calling rte_cryptodev_info_get() on a device QAT_GEN3,
>>>>>>> rte_cryptodev_info.capabilities.sym.aead.algo is set to
>>>>>>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 (= 3).
>>>>>>> The application uses this value:
>>>>>>> ++ aead_dev_count[info.capabilities.sym.aead.algo];
>>>>>>> The application is crashing because of out of bound access.
>>>>>>
>>>>>> I'd say this is an example of bad written app.
>>>>>> It probably should check that returned by library value doesn't
>>>>>> exceed its internal array size.
>>>>>
>>>>> +1
>>>>>
>>>>> Application should ignore values >= MAX.
>>>>
>>>> Of course, blaming the API user is a lot easier than looking at the API.
>>>> Here the API has RTE_CRYPTO_AEAD_LIST_END which can be understood
>>>> as the max value for the application.
>>>> Value ranges are part of the ABI compatibility contract.
>>>> It seems you expect the application developer to be aware that
>>>> DPDK could return a higher value, so the application should
>>>> check every enum values after calling an API. CRAZY.
>>>>
>>>> When we decide to announce an ABI compatibility and do some marketing,
>>>> everyone is OK. But when we need to really make our ABI compatible,
>>>> I see little or no effort. DISAPPOINTING.
>>>>
>>>>> Do you suggest we don't extend any enum or define between ABI breakage releases
>>>>> to be sure bad written applications not affected?
>>>>
>>>> I suggest we must consider not breaking any assumption made on the API.
>>>> Here we are breaking the enum range because nothing mentions _LIST_END
>>>> is not really the absolute end of the enum.
>>>> The solution is to make the change below in 20.02 + backport in 19.11.1:
>>>
>>> Thinking twice, merging such change before 20.11 is breaking the
>>> ABI assumption based on the API 19.11.0.
>>> I ask the release maintainers (Luca, Kevin, David and me) and
>>> the ABI maintainers (Neil and Ray) to vote for a or b solution:
>>> a) add comment and LIST_MAX as below in 20.02 + 19.11.1
>>
>> That would still be an ABI breakage though right.
>>
>>> b) wait 20.11 and revert Chacha-Poly from 20.02
>>
>> Thanks for analysis above Fiona, Ferruh and all.
>>
>> That is a nasty one alright - there is no "good" answer here.
>> I agree with Ferruh's sentiments overall, we should rethink this API for 20.11.
>> Could do without an enumeration?
>>
>> There a c) though right.
>> We could work around the issue by api versioning rte_cryptodev_info_get() and friends.
>> So they only support/acknowledge the existence of Chacha-Poly for applications build against > 20.02.
>
> I agree there is a c) as I proposed in another email:
> http://mails.dpdk.org/archives/dev/2020-February/156919.html
> "
> In this case, the proper solution is to implement
> rte_cryptodev_info_get_v1911() so it filters out
> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 capability.
> With this solution, an application compiled with DPDK 19.11 will keep
> seeing the same range as before, while a 20.02 application could
> see and use ChachaPoly.
> "
>
>> It would be painful I know.
>
> Not so painful in my opinion.
> Just need to call rte_cryptodev_info_get() from
> rte_cryptodev_info_get_v1911() and filter the value
> in the 19.11 range: [0..AES_GCM].
>
>> It would also mean that Chacha-Poly would only be available to
>> those building against >= 20.02.
>
> Yes exactly.
>
> The addition of comments and LIST_MAX like below are still valid
> to avoid versioning after 20.11.
>
>>>> - _LIST_END
>>>> + _LIST_END, /* an ABI-compatible version may increase this value */
>>>> + _LIST_MAX = _LIST_END + 42 /* room for ABI-compatible additions */
>>>> };
>>>>
>>>> Then *_LIST_END values could be ignored by libabigail with such a change.
>
> In order to avoid ABI check complaining, the best is to completely
> remove LIST_END in DPDK 20.11.
We can remove LIST_END only if we go with option (c).
Two different approach,
- Provide the LIST_END and expect application protect itself against new values
can be coming in newer version of the library
- Do ABI versioning to prevent application receive new values at all, (c).
We can select one, but I believe the selection shouldn't be based on just
silencing the ABI check tool.
>
>
>>>> If such a patch is not done by tomorrow, I will have to revert
>>>> Chacha-Poly commits before 20.02-rc2, because
>>>>
>>>> 1/ LIST_END, without any comment, means "size of range"
>>>> 2/ we do not blame users for undocumented ABI changes
>>>> 3/ we respect the ABI compatibility contract
>
>
>
^ permalink raw reply [relevance 8%]
* Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
2020-02-04 10:16 9% ` Akhil Goyal
@ 2020-02-04 10:28 8% ` Thomas Monjalon
2020-02-04 10:32 4% ` Akhil Goyal
2020-02-04 21:59 8% ` [dpdk-dev] " Neil Horman
1 sibling, 1 reply; 200+ results
From: Thomas Monjalon @ 2020-02-04 10:28 UTC (permalink / raw)
To: Akhil Goyal
Cc: Ferruh Yigit, Ananyev, Konstantin, Trahe, Fiona, dev,
David Marchand, Anoob Joseph, Kusztal, ArkadiuszX, dev,
Richardson, Bruce, nhorman, Mcnamara, John, dodji,
Andrew Rybchenko, aconole, bluca, ktraynor
04/02/2020 11:16, Akhil Goyal:
> Hi,
> > On 2/3/2020 5:09 PM, Thomas Monjalon wrote:
> > > 03/02/2020 10:30, Ferruh Yigit:
> > >> On 2/2/2020 2:41 PM, Ananyev, Konstantin wrote:
> > >>> 02/02/2020 14:05, Thomas Monjalon:
> > >>>> 31/01/2020 15:16, Trahe, Fiona:
> > >>>>> On 1/30/2020 8:18 PM, Thomas Monjalon wrote:
> > >>>>>> 30/01/2020 17:09, Ferruh Yigit:
> > >>>>>>> On 1/29/2020 8:13 PM, Akhil Goyal wrote:
> > >>>>>>>>
> > >>>>>>>> I believe these enums will be used only in case of ASYM case which is
> > experimental.
> > >>>>>>>
> > >>>>>>> Independent from being experiment and not, this shouldn't be a
> > problem, I think
> > >>>>>>> this is a false positive.
> > >>>>>>>
> > >>>>>>> The ABI break can happen when a struct has been shared between the
> > application
> > >>>>>>> and the library (DPDK) and the layout of that memory know differently
> > by
> > >>>>>>> application and the library.
> > >>>>>>>
> > >>>>>>> Here in all cases, there is no layout/size change.
> > >>>>>>>
> > >>>>>>> As to the value changes of the enums, since application compiled with
> > old DPDK,
> > >>>>>>> it will know only up to '6', 7 and more means invalid to the application.
> > So it
> > >>>>>>> won't send these values also it should ignore these values from library.
> > Only
> > >>>>>>> consequence is old application won't able to use new features those
> > new enums
> > >>>>>>> provide but that is expected/normal.
> > >>>>>>
> > >>>>>> If library give higher value than expected by the application,
> > >>>>>> if the application uses this value as array index,
> > >>>>>> there can be an access out of bounds.
> > >>>>>
> > >>>>> [Fiona] All asymmetric APIs are experimental so above shouldn't be a
> > problem.
> > >>>>> But for the same issue with sym crypto below, I believe Ferruh's
> > explanation makes
> > >>>>> sense and I don't see how there can be an API breakage.
> > >>>>> So if an application hasn't compiled against the new lib it will be still using
> > the old value
> > >>>>> which will be within bounds. If it's picking up the higher new value from
> > the lib it must
> > >>>>> have been compiled against the lib so shouldn't have problems.
> > >>>>
> > >>>> You say there is no ABI issue because the application will be re-compiled
> > >>>> for the updated library. Indeed, compilation fixes compatibility issues.
> > >>>> But this is not relevant for ABI compatibility.
> > >>>> ABI compatibility means we can upgrade the library without recompiling
> > >>>> the application and it must work.
> > >>>> You think it is a false positive because you assume the application
> > >>>> "picks" the new value. I think you miss the case where the new value
> > >>>> is returned by a function in the upgraded library.
> > >>>>
> > >>>>> There are also no structs on the API which contain arrays using this
> > >>>>> for sizing, so I don't see an opportunity for an appl to have a
> > >>>>> mismatch in memory addresses.
> > >>>>
> > >>>> Let me demonstrate where the API may "use" the new value
> > >>>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 and how it impacts the
> > application.
> > >>>>
> > >>>> Once upon a time a DPDK application counting the number of devices
> > >>>> supporting each AEAD algo (in order to find the best supported algo).
> > >>>> It is done in an array indexed by algo id:
> > >>>> int aead_dev_count[RTE_CRYPTO_AEAD_LIST_END];
> > >>>> The application is compiled with DPDK 19.11,
> > >>>> where RTE_CRYPTO_AEAD_LIST_END = 3.
> > >>>> So the size of the application array aead_dev_count is 3.
> > >>>> This binary is run with DPDK 20.02,
> > >>>> where RTE_CRYPTO_AEAD_CHACHA20_POLY1305 = 3.
> > >>>> When calling rte_cryptodev_info_get() on a device QAT_GEN3,
> > >>>> rte_cryptodev_info.capabilities.sym.aead.algo is set to
> > >>>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 (= 3).
> > >>>> The application uses this value:
> > >>>> ++ aead_dev_count[info.capabilities.sym.aead.algo];
> > >>>> The application is crashing because of out of bound access.
> > >>>
> > >>> I'd say this is an example of bad written app.
> > >>> It probably should check that returned by library value doesn't
> > >>> exceed its internal array size.
> > >>
> > >> +1
> > >>
> > >> Application should ignore values >= MAX.
> > >
> > > Of course, blaming the API user is a lot easier than looking at the API.
> > > Here the API has RTE_CRYPTO_AEAD_LIST_END which can be understood
> > > as the max value for the application.
> > > Value ranges are part of the ABI compatibility contract.
> > > It seems you expect the application developer to be aware that
> > > DPDK could return a higher value, so the application should
> > > check every enum values after calling an API. CRAZY.
> > >
> > > When we decide to announce an ABI compatibility and do some marketing,
> > > everyone is OK. But when we need to really make our ABI compatible,
> > > I see little or no effort. DISAPPOINTING.
> >
> > This is not to blame the user or to do less work, this is more sane approach
> > that library provides the _END/_MAX value and application uses it as valid range
> > check.
> >
> > >
> > >> Do you suggest we don't extend any enum or define between ABI breakage
> > releases
> > >> to be sure bad written applications not affected?
> > >
> > > I suggest we must consider not breaking any assumption made on the API.
> > > Here we are breaking the enum range because nothing mentions _LIST_END
> > > is not really the absolute end of the enum.
> > > The solution is to make the change below in 20.02 + backport in 19.11.1:
> > >
> > > - _LIST_END
> > > + _LIST_END, /* an ABI-compatible version may increase this value */
> > > + _LIST_MAX = _LIST_END + 42 /* room for ABI-compatible additions */
> > > };
> > >
> >
> > What is the point of "_LIST_MAX" here?
> >
> > Application should know the "_LIST_END" of when it has been compiled for the
> > valid range check. Next time it is compiled "_LIST_END" may be different value
> > but same logic applies.
> >
> > When "_LIST_END" is missing, application can't protect itself, in that case
> > library should send only the values application knows when it is compiled, this
> > means either we can't extend our enum/defines until next ABI breakage, or we
> > need to do ABI versioning to the functions that returns an enum each time enum
> > value extended.
> >
> > I believe it is saner to provide _END/_MAX values to the application to use. And
> > if required comment them to clarify the expected usage.
> >
> > But in above suggestion application can't use or rely on "_LIST_MAX", it doesn't
> > mean anything to application.
> >
>
> Can we have something like
> enum rte_crypto_aead_algorithm {
> RTE_CRYPTO_AEAD_AES_CCM = 1,
> /**< AES algorithm in CCM mode. */
> RTE_CRYPTO_AEAD_AES_GCM,
> /**< AES algorithm in GCM mode. */
> RTE_CRYPTO_AEAD_LIST_END,
> /**< List end for 19.11 ABI compatibility */
> RTE_CRYPTO_AEAD_CHACHA20_POLY1305,
> /**< Chacha20 cipher with poly1305 authenticator */
> RTE_CRYPTO_AEAD_LIST_END_2011
> /**< List end for 20.11 ABI compatibility */
> };
>
> And in 20.11 release we alter the RTE_CRYPTO_AEAD_LIST_END to the end and remove RTE_CRYPTO_AEAD_LIST_END_2011
>
> I believe it will be ok for any application which need to use the chacha poly assume that this algo is
> Experimental and will move to formal list in 20.11. This can be documented in the documentation.
> I believe there is no way to add a new enum as experimental so far. This way we can formalize this
> requirement as well.
>
> I believe this way effect of ABI breakage will be nullified.
This is a possibility in the (a) proposal.
But it breaks API (and ABI) because a high value is returned
while not expected by the application.
I guess ABI and release maintainers will vote no to such breakage.
Note: I vote no.
> > > Then *_LIST_END values could be ignored by libabigail with such a change.
> > >
> > > If such a patch is not done by tomorrow, I will have to revert
> > > Chacha-Poly commits before 20.02-rc2, because
> > >
> > > 1/ LIST_END, without any comment, means "size of range"
> > > 2/ we do not blame users for undocumented ABI changes
> > > 3/ we respect the ABI compatibility contract
^ permalink raw reply [relevance 8%]
* Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
2020-02-04 10:28 8% ` Thomas Monjalon
@ 2020-02-04 10:32 4% ` Akhil Goyal
2020-02-04 11:35 4% ` Bruce Richardson
2020-02-04 22:10 7% ` Neil Horman
0 siblings, 2 replies; 200+ results
From: Akhil Goyal @ 2020-02-04 10:32 UTC (permalink / raw)
To: Thomas Monjalon
Cc: Ferruh Yigit, Ananyev, Konstantin, Trahe, Fiona, dev,
David Marchand, Anoob Joseph, Kusztal, ArkadiuszX, dev,
Richardson, Bruce, nhorman, Mcnamara, John, dodji,
Andrew Rybchenko, aconole, bluca, ktraynor
>
> 04/02/2020 11:16, Akhil Goyal:
> > Hi,
> > > On 2/3/2020 5:09 PM, Thomas Monjalon wrote:
> > > > 03/02/2020 10:30, Ferruh Yigit:
> > > >> On 2/2/2020 2:41 PM, Ananyev, Konstantin wrote:
> > > >>> 02/02/2020 14:05, Thomas Monjalon:
> > > >>>> 31/01/2020 15:16, Trahe, Fiona:
> > > >>>>> On 1/30/2020 8:18 PM, Thomas Monjalon wrote:
> > > >>>>>> 30/01/2020 17:09, Ferruh Yigit:
> > > >>>>>>> On 1/29/2020 8:13 PM, Akhil Goyal wrote:
> > > >>>>>>>>
> > > >>>>>>>> I believe these enums will be used only in case of ASYM case which
> is
> > > experimental.
> > > >>>>>>>
> > > >>>>>>> Independent from being experiment and not, this shouldn't be a
> > > problem, I think
> > > >>>>>>> this is a false positive.
> > > >>>>>>>
> > > >>>>>>> The ABI break can happen when a struct has been shared between
> the
> > > application
> > > >>>>>>> and the library (DPDK) and the layout of that memory know
> differently
> > > by
> > > >>>>>>> application and the library.
> > > >>>>>>>
> > > >>>>>>> Here in all cases, there is no layout/size change.
> > > >>>>>>>
> > > >>>>>>> As to the value changes of the enums, since application compiled
> with
> > > old DPDK,
> > > >>>>>>> it will know only up to '6', 7 and more means invalid to the
> application.
> > > So it
> > > >>>>>>> won't send these values also it should ignore these values from
> library.
> > > Only
> > > >>>>>>> consequence is old application won't able to use new features
> those
> > > new enums
> > > >>>>>>> provide but that is expected/normal.
> > > >>>>>>
> > > >>>>>> If library give higher value than expected by the application,
> > > >>>>>> if the application uses this value as array index,
> > > >>>>>> there can be an access out of bounds.
> > > >>>>>
> > > >>>>> [Fiona] All asymmetric APIs are experimental so above shouldn't be a
> > > problem.
> > > >>>>> But for the same issue with sym crypto below, I believe Ferruh's
> > > explanation makes
> > > >>>>> sense and I don't see how there can be an API breakage.
> > > >>>>> So if an application hasn't compiled against the new lib it will be still
> using
> > > the old value
> > > >>>>> which will be within bounds. If it's picking up the higher new value
> from
> > > the lib it must
> > > >>>>> have been compiled against the lib so shouldn't have problems.
> > > >>>>
> > > >>>> You say there is no ABI issue because the application will be re-
> compiled
> > > >>>> for the updated library. Indeed, compilation fixes compatibility issues.
> > > >>>> But this is not relevant for ABI compatibility.
> > > >>>> ABI compatibility means we can upgrade the library without
> recompiling
> > > >>>> the application and it must work.
> > > >>>> You think it is a false positive because you assume the application
> > > >>>> "picks" the new value. I think you miss the case where the new value
> > > >>>> is returned by a function in the upgraded library.
> > > >>>>
> > > >>>>> There are also no structs on the API which contain arrays using this
> > > >>>>> for sizing, so I don't see an opportunity for an appl to have a
> > > >>>>> mismatch in memory addresses.
> > > >>>>
> > > >>>> Let me demonstrate where the API may "use" the new value
> > > >>>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 and how it impacts the
> > > application.
> > > >>>>
> > > >>>> Once upon a time a DPDK application counting the number of devices
> > > >>>> supporting each AEAD algo (in order to find the best supported algo).
> > > >>>> It is done in an array indexed by algo id:
> > > >>>> int aead_dev_count[RTE_CRYPTO_AEAD_LIST_END];
> > > >>>> The application is compiled with DPDK 19.11,
> > > >>>> where RTE_CRYPTO_AEAD_LIST_END = 3.
> > > >>>> So the size of the application array aead_dev_count is 3.
> > > >>>> This binary is run with DPDK 20.02,
> > > >>>> where RTE_CRYPTO_AEAD_CHACHA20_POLY1305 = 3.
> > > >>>> When calling rte_cryptodev_info_get() on a device QAT_GEN3,
> > > >>>> rte_cryptodev_info.capabilities.sym.aead.algo is set to
> > > >>>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 (= 3).
> > > >>>> The application uses this value:
> > > >>>> ++ aead_dev_count[info.capabilities.sym.aead.algo];
> > > >>>> The application is crashing because of out of bound access.
> > > >>>
> > > >>> I'd say this is an example of bad written app.
> > > >>> It probably should check that returned by library value doesn't
> > > >>> exceed its internal array size.
> > > >>
> > > >> +1
> > > >>
> > > >> Application should ignore values >= MAX.
> > > >
> > > > Of course, blaming the API user is a lot easier than looking at the API.
> > > > Here the API has RTE_CRYPTO_AEAD_LIST_END which can be understood
> > > > as the max value for the application.
> > > > Value ranges are part of the ABI compatibility contract.
> > > > It seems you expect the application developer to be aware that
> > > > DPDK could return a higher value, so the application should
> > > > check every enum values after calling an API. CRAZY.
> > > >
> > > > When we decide to announce an ABI compatibility and do some marketing,
> > > > everyone is OK. But when we need to really make our ABI compatible,
> > > > I see little or no effort. DISAPPOINTING.
> > >
> > > This is not to blame the user or to do less work, this is more sane approach
> > > that library provides the _END/_MAX value and application uses it as valid
> range
> > > check.
> > >
> > > >
> > > >> Do you suggest we don't extend any enum or define between ABI
> breakage
> > > releases
> > > >> to be sure bad written applications not affected?
> > > >
> > > > I suggest we must consider not breaking any assumption made on the API.
> > > > Here we are breaking the enum range because nothing mentions
> _LIST_END
> > > > is not really the absolute end of the enum.
> > > > The solution is to make the change below in 20.02 + backport in 19.11.1:
> > > >
> > > > - _LIST_END
> > > > + _LIST_END, /* an ABI-compatible version may increase this value */
> > > > + _LIST_MAX = _LIST_END + 42 /* room for ABI-compatible additions */
> > > > };
> > > >
> > >
> > > What is the point of "_LIST_MAX" here?
> > >
> > > Application should know the "_LIST_END" of when it has been compiled for
> the
> > > valid range check. Next time it is compiled "_LIST_END" may be different
> value
> > > but same logic applies.
> > >
> > > When "_LIST_END" is missing, application can't protect itself, in that case
> > > library should send only the values application knows when it is compiled,
> this
> > > means either we can't extend our enum/defines until next ABI breakage, or
> we
> > > need to do ABI versioning to the functions that returns an enum each time
> enum
> > > value extended.
> > >
> > > I believe it is saner to provide _END/_MAX values to the application to use.
> And
> > > if required comment them to clarify the expected usage.
> > >
> > > But in above suggestion application can't use or rely on "_LIST_MAX", it
> doesn't
> > > mean anything to application.
> > >
> >
> > Can we have something like
> > enum rte_crypto_aead_algorithm {
> > RTE_CRYPTO_AEAD_AES_CCM = 1,
> > /**< AES algorithm in CCM mode. */
> > RTE_CRYPTO_AEAD_AES_GCM,
> > /**< AES algorithm in GCM mode. */
> > RTE_CRYPTO_AEAD_LIST_END,
> > /**< List end for 19.11 ABI compatibility */
> > RTE_CRYPTO_AEAD_CHACHA20_POLY1305,
> > /**< Chacha20 cipher with poly1305 authenticator */
> > RTE_CRYPTO_AEAD_LIST_END_2011
> > /**< List end for 20.11 ABI compatibility */
> > };
> >
> > And in 20.11 release we alter the RTE_CRYPTO_AEAD_LIST_END to the end
> and remove RTE_CRYPTO_AEAD_LIST_END_2011
> >
> > I believe it will be ok for any application which need to use the chacha poly
> assume that this algo is
> > Experimental and will move to formal list in 20.11. This can be documented in
> the documentation.
> > I believe there is no way to add a new enum as experimental so far. This way
> we can formalize this
> > requirement as well.
> >
> > I believe this way effect of ABI breakage will be nullified.
>
> This is a possibility in the (a) proposal.
> But it breaks API (and ABI) because a high value is returned
> while not expected by the application.
>
> I guess ABI and release maintainers will vote no to such breakage.
> Note: I vote no.
>
If that is the case, I would say we should go with b).
Versioned APIs does not look good and adds more confusion.
>
> > > > Then *_LIST_END values could be ignored by libabigail with such a change.
> > > >
> > > > If such a patch is not done by tomorrow, I will have to revert
> > > > Chacha-Poly commits before 20.02-rc2, because
> > > >
> > > > 1/ LIST_END, without any comment, means "size of range"
> > > > 2/ we do not blame users for undocumented ABI changes
> > > > 3/ we respect the ABI compatibility contract
>
>
>
>
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
2020-02-04 9:45 4% ` Thomas Monjalon
@ 2020-02-04 9:56 4% ` Ferruh Yigit
2020-02-04 10:08 4% ` Bruce Richardson
2020-02-04 10:17 4% ` Kevin Traynor
0 siblings, 2 replies; 200+ results
From: Ferruh Yigit @ 2020-02-04 9:56 UTC (permalink / raw)
To: Thomas Monjalon
Cc: dev, Ananyev, Konstantin, Akhil Goyal, Trahe, Fiona,
David Marchand, Anoob Joseph, Kusztal, ArkadiuszX, Richardson,
Bruce, nhorman, Mcnamara, John, dodji, Andrew Rybchenko, aconole,
bluca, ktraynor
On 2/4/2020 9:45 AM, Thomas Monjalon wrote:
> 04/02/2020 10:19, Ferruh Yigit:
>> On 2/3/2020 6:40 PM, Thomas Monjalon wrote:
>>> 03/02/2020 18:40, Ferruh Yigit:
>>>> On 2/3/2020 5:09 PM, Thomas Monjalon wrote:
>>>>> 03/02/2020 10:30, Ferruh Yigit:
>>>>>> On 2/2/2020 2:41 PM, Ananyev, Konstantin wrote:
>>>>>>> 02/02/2020 14:05, Thomas Monjalon:
>>>>>>>> 31/01/2020 15:16, Trahe, Fiona:
>>>>>>>>> On 1/30/2020 8:18 PM, Thomas Monjalon wrote:
>>>>>>>>>> 30/01/2020 17:09, Ferruh Yigit:
>>>>>>>>>>> On 1/29/2020 8:13 PM, Akhil Goyal wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> I believe these enums will be used only in case of ASYM case which is experimental.
>>>>>>>>>>>
>>>>>>>>>>> Independent from being experiment and not, this shouldn't be a problem, I think
>>>>>>>>>>> this is a false positive.
>>>>>>>>>>>
>>>>>>>>>>> The ABI break can happen when a struct has been shared between the application
>>>>>>>>>>> and the library (DPDK) and the layout of that memory know differently by
>>>>>>>>>>> application and the library.
>>>>>>>>>>>
>>>>>>>>>>> Here in all cases, there is no layout/size change.
>>>>>>>>>>>
>>>>>>>>>>> As to the value changes of the enums, since application compiled with old DPDK,
>>>>>>>>>>> it will know only up to '6', 7 and more means invalid to the application. So it
>>>>>>>>>>> won't send these values also it should ignore these values from library. Only
>>>>>>>>>>> consequence is old application won't able to use new features those new enums
>>>>>>>>>>> provide but that is expected/normal.
>>>>>>>>>>
>>>>>>>>>> If library give higher value than expected by the application,
>>>>>>>>>> if the application uses this value as array index,
>>>>>>>>>> there can be an access out of bounds.
>>>>>>>>>
>>>>>>>>> [Fiona] All asymmetric APIs are experimental so above shouldn't be a problem.
>>>>>>>>> But for the same issue with sym crypto below, I believe Ferruh's explanation makes
>>>>>>>>> sense and I don't see how there can be an API breakage.
>>>>>>>>> So if an application hasn't compiled against the new lib it will be still using the old value
>>>>>>>>> which will be within bounds. If it's picking up the higher new value from the lib it must
>>>>>>>>> have been compiled against the lib so shouldn't have problems.
>>>>>>>>
>>>>>>>> You say there is no ABI issue because the application will be re-compiled
>>>>>>>> for the updated library. Indeed, compilation fixes compatibility issues.
>>>>>>>> But this is not relevant for ABI compatibility.
>>>>>>>> ABI compatibility means we can upgrade the library without recompiling
>>>>>>>> the application and it must work.
>>>>>>>> You think it is a false positive because you assume the application
>>>>>>>> "picks" the new value. I think you miss the case where the new value
>>>>>>>> is returned by a function in the upgraded library.
>>>>>>>>
>>>>>>>>> There are also no structs on the API which contain arrays using this
>>>>>>>>> for sizing, so I don't see an opportunity for an appl to have a
>>>>>>>>> mismatch in memory addresses.
>>>>>>>>
>>>>>>>> Let me demonstrate where the API may "use" the new value
>>>>>>>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 and how it impacts the application.
>>>>>>>>
>>>>>>>> Once upon a time a DPDK application counting the number of devices
>>>>>>>> supporting each AEAD algo (in order to find the best supported algo).
>>>>>>>> It is done in an array indexed by algo id:
>>>>>>>> int aead_dev_count[RTE_CRYPTO_AEAD_LIST_END];
>>>>>>>> The application is compiled with DPDK 19.11,
>>>>>>>> where RTE_CRYPTO_AEAD_LIST_END = 3.
>>>>>>>> So the size of the application array aead_dev_count is 3.
>>>>>>>> This binary is run with DPDK 20.02,
>>>>>>>> where RTE_CRYPTO_AEAD_CHACHA20_POLY1305 = 3.
>>>>>>>> When calling rte_cryptodev_info_get() on a device QAT_GEN3,
>>>>>>>> rte_cryptodev_info.capabilities.sym.aead.algo is set to
>>>>>>>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 (= 3).
>>>>>>>> The application uses this value:
>>>>>>>> ++ aead_dev_count[info.capabilities.sym.aead.algo];
>>>>>>>> The application is crashing because of out of bound access.
>>>>>>>
>>>>>>> I'd say this is an example of bad written app.
>>>>>>> It probably should check that returned by library value doesn't
>>>>>>> exceed its internal array size.
>>>>>>
>>>>>> +1
>>>>>>
>>>>>> Application should ignore values >= MAX.
>>>>>
>>>>> Of course, blaming the API user is a lot easier than looking at the API.
>>>>> Here the API has RTE_CRYPTO_AEAD_LIST_END which can be understood
>>>>> as the max value for the application.
>>>>> Value ranges are part of the ABI compatibility contract.
>>>>> It seems you expect the application developer to be aware that
>>>>> DPDK could return a higher value, so the application should
>>>>> check every enum values after calling an API. CRAZY.
>>>>>
>>>>> When we decide to announce an ABI compatibility and do some marketing,
>>>>> everyone is OK. But when we need to really make our ABI compatible,
>>>>> I see little or no effort. DISAPPOINTING.
>>>>
>>>> This is not to blame the user or to do less work, this is more sane approach
>>>> that library provides the _END/_MAX value and application uses it as valid range
>>>> check.
>>>>
>>>>>> Do you suggest we don't extend any enum or define between ABI breakage releases
>>>>>> to be sure bad written applications not affected?
>>>>>
>>>>> I suggest we must consider not breaking any assumption made on the API.
>>>>> Here we are breaking the enum range because nothing mentions _LIST_END
>>>>> is not really the absolute end of the enum.
>>>>> The solution is to make the change below in 20.02 + backport in 19.11.1:
>>>>>
>>>>> - _LIST_END
>>>>> + _LIST_END, /* an ABI-compatible version may increase this value */
>>>>> + _LIST_MAX = _LIST_END + 42 /* room for ABI-compatible additions */
>>>>> };
>>>>>
>>>>
>>>> What is the point of "_LIST_MAX" here?
>>>
>>> _LIST_MAX is range of value that DPDK can return in the ABI contract.
>>> So the appplication can rely on the range 0.._LIST_MAX.
>>>
>>>> Application should know the "_LIST_END" of when it has been compiled for the
>>>> valid range check. Next time it is compiled "_LIST_END" may be different value
>>>> but same logic applies.
>>>
>>> No, ABI compatibility contract means you can compile your application
>>> with DPDK 19.11.0 and run it with DPDK 20.02.
>>> So _LIST_END comes from 19.11 and does not include ChachaPoly.
>>
>> That is what I mean, let me try to give a sample.
>>
>> DPDK19.11 returns, A=1, B=2, END=3
>>
>> Application compiled with DPDK19.11, it will process A, B and ignore anything ">= 3"
>
> No, the application will not ignore anything ">=3" as I explained above,
> and you blamed the application for it.
> Nothing in the API says the application must filter value higher than 3,
> because as of now, values higher than 3 are PMD bug.
When application compiled, that is the END value, anything bigger than this
value is not valid, if any application use the return value directly I think it
is doing something wrong.
But yes there may be applications relying on library will always send in the
range. We never communicated this. But we can add comments to clarify this.
>
>
>> DPDK20.02 returns A=1, B=2, C=3, D=4, END=5
>>
>> Old application will still only will know/use A, B and can ignore when library
>> sends C=3, D=4 etc...
>>
>>
>> In above, if you add another limit as you suggested, like MAX=10 and ask
>> application to use it,
>>
>> Application compiled with DPDK19.11 will be OK since library only sends A,B and
>> application uses them.
>>
>> But with DPDK20.02 application may have problem, since library will be sending
>> C=3, which is valid according to the check " <= MAX (10)", how application will
>> know to ignore it.
>
> Why application should ignore value C=3 with DPDK 20.02?
This is the application compiled with DPDK19.11, and running with DPDK20.02.
So for the application this is the value >= MAX and something it doesn't know
what to do.
>
>
>> So application should use _END to know the valid ones according it, if so what
>> is the point of having _MAX.
>>
>>
>>>> When "_LIST_END" is missing, application can't protect itself, in that case
>>>> library should send only the values application knows when it is compiled, this
>>>> means either we can't extend our enum/defines until next ABI breakage, or we
>>>> need to do ABI versioning to the functions that returns an enum each time enum
>>>> value extended.
>>>
>>> If we define _LIST_MAX as a bigger value than current _LIST_END,
>>> we have some room to add values in between.
>>>
>>> If (as of now) we don't have _LIST_MAX room, then yes we must version
>>> the functions returning the enum.
>>> In this case, the proper solution is to implement
>>> rte_cryptodev_info_get_v1911() so it filters out
>>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 capability.
>>> With this solution, an application compiled with DPDK 19.11 will keep
>>> seeing the same range as before, while a 20.02 application could
>>> see and use ChachaPoly.
>>> This is another proposal that I was expecting from the crypto team,
>>> instead of claiming there is no issue (and wasting precious time).
>>>
>>>
>>>> I believe it is saner to provide _END/_MAX values to the application to use. And
>>>> if required comment them to clarify the expected usage.
>>>>
>>>> But in above suggestion application can't use or rely on "_LIST_MAX", it doesn't
>>>> mean anything to application.
>>>
>>> I don't understand what you mean. I think you misunderstood what is ABI compat.
>>>
>>>
>>>>> Then *_LIST_END values could be ignored by libabigail with such a change.
>>>>>
>>>>> If such a patch is not done by tomorrow, I will have to revert
>>>>> Chacha-Poly commits before 20.02-rc2, because
>>>>>
>>>>> 1/ LIST_END, without any comment, means "size of range"
>>>>> 2/ we do not blame users for undocumented ABI changes
>>>>> 3/ we respect the ABI compatibility contract
>
>
>
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
2020-02-04 10:10 4% ` Trahe, Fiona
@ 2020-02-04 10:38 4% ` Thomas Monjalon
0 siblings, 0 replies; 200+ results
From: Thomas Monjalon @ 2020-02-04 10:38 UTC (permalink / raw)
To: Trahe, Fiona
Cc: David Marchand, Ray Kinsella, dev, Neil Horman, Luca Boccassi,
Kevin Traynor, Ananyev, Konstantin, Akhil Goyal, Yigit, Ferruh,
dev, Anoob Joseph, Kusztal, ArkadiuszX, Richardson, Bruce,
Mcnamara, John, dodji, Andrew Rybchenko, Aaron Conole
04/02/2020 11:10, Trahe, Fiona:
> And not used for sizing > >
> > > There a c) though right.
> > > We could work around the issue by api versioning rte_cryptodev_info_get() and friends.
> >
> > It has a lot of friends, but it sounds like the right approach.
> > Is someone looking into this?
> [Fiona] Yes. But not clear yet if can be done by tomorrow.
Should be done by today now.
> But even if feasible, that only works around the current issue.
> There is a bigger issue to be decided here -
> Should we be removing LIST_END/MAX values from all enums in 20.11?
> Or defining through API comment that they should only be used as a range boundary and
> NOT to size an array. And so having a fixed value is not part of the API contract.
Please let's discuss 20.11 API later. It is not so urgent.
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH v7] eal: add manual probing option
2020-02-04 10:03 0% ` Gaetan Rivet
@ 2020-02-04 11:07 0% ` Thomas Monjalon
2020-02-04 12:43 0% ` Gaetan Rivet
0 siblings, 1 reply; 200+ results
From: Thomas Monjalon @ 2020-02-04 11:07 UTC (permalink / raw)
To: Pavan Nikhilesh Bhagavatula, Gaetan Rivet
Cc: dev, dev, David Marchand, Vamsi Krishna Attunuru,
Jerin Jacob Kollanukkaran
04/02/2020 11:03, Gaetan Rivet:
> On 03/02/2020 23:21, Thomas Monjalon wrote:
> > 03/02/2020 06:16, Pavan Nikhilesh Bhagavatula:
> >> @David Marchand @thomas@monjalon.net
> >>
> >> Ping?
> >>
> >> Are there any more changes required for this patch? It's been in queue since last October.
> >
> > Sorry we have not decided whether it is a good idea or not.
> >
> > All changes related to probing are very sensitive,
> > and we know a big refactoring would be better than stacking
> > more and more options and corner cases.
> >
> > As we are busy with ABI stability stuff, we did not allocate
> > enough time to properly think about this feature.
> > Please accept our apologies, and let's consider it as
> > a high priority for 20.05 cycle.
> >
>
> Hello Thomas,
>
> This is unfortunate. I pushed Pavan to accept an alternative implementation of this functionality that was less obtrusive, to make the integration smoother. I took care to alleviate those risks from the common path.
>
> The big refactoring is needed yes, but considering the current path I'm not seeing it happen in 20.05. If that means taking this patch as-is in 20.05 for Marvell users, I'm not sure much is gained from waiting 3 months, except minimal risk avoidance.
Yes, life is full of bad decisions and consequences.
I still think there is a risk in adding new user expectations,
and maintaining some code to workaround unknown issues.
The real question here is to know why this patch?
Is it to workaround a broken driver?
Or to workaround a broken design in EAL and bus drivers?
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
2020-02-04 10:32 4% ` Akhil Goyal
@ 2020-02-04 11:35 4% ` Bruce Richardson
2020-02-04 22:10 7% ` Neil Horman
1 sibling, 0 replies; 200+ results
From: Bruce Richardson @ 2020-02-04 11:35 UTC (permalink / raw)
To: Akhil Goyal
Cc: Thomas Monjalon, Ferruh Yigit, Ananyev, Konstantin, Trahe, Fiona,
dev, David Marchand, Anoob Joseph, Kusztal, ArkadiuszX, nhorman,
Mcnamara, John, dodji, Andrew Rybchenko, aconole, bluca,
ktraynor
On Tue, Feb 04, 2020 at 10:32:01AM +0000, Akhil Goyal wrote:
>
> >
> > 04/02/2020 11:16, Akhil Goyal:
> > > Hi,
> > > > On 2/3/2020 5:09 PM, Thomas Monjalon wrote:
> > > > > 03/02/2020 10:30, Ferruh Yigit:
> > > > >> On 2/2/2020 2:41 PM, Ananyev, Konstantin wrote:
> > > > >>> 02/02/2020 14:05, Thomas Monjalon:
> > > > >>>> 31/01/2020 15:16, Trahe, Fiona:
> > > > >>>>> On 1/30/2020 8:18 PM, Thomas Monjalon wrote:
> > > > >>>>>> 30/01/2020 17:09, Ferruh Yigit:
> > > > >>>>>>> On 1/29/2020 8:13 PM, Akhil Goyal wrote:
> > > > >>>>>>>>
> > > > >>>>>>>> I believe these enums will be used only in case of ASYM case which
> > is
> > > > experimental.
> > > > >>>>>>>
> > > > >>>>>>> Independent from being experiment and not, this shouldn't be a
> > > > problem, I think
> > > > >>>>>>> this is a false positive.
> > > > >>>>>>>
> > > > >>>>>>> The ABI break can happen when a struct has been shared between
> > the
> > > > application
> > > > >>>>>>> and the library (DPDK) and the layout of that memory know
> > differently
> > > > by
> > > > >>>>>>> application and the library.
> > > > >>>>>>>
> > > > >>>>>>> Here in all cases, there is no layout/size change.
> > > > >>>>>>>
> > > > >>>>>>> As to the value changes of the enums, since application compiled
> > with
> > > > old DPDK,
> > > > >>>>>>> it will know only up to '6', 7 and more means invalid to the
> > application.
> > > > So it
> > > > >>>>>>> won't send these values also it should ignore these values from
> > library.
> > > > Only
> > > > >>>>>>> consequence is old application won't able to use new features
> > those
> > > > new enums
> > > > >>>>>>> provide but that is expected/normal.
> > > > >>>>>>
> > > > >>>>>> If library give higher value than expected by the application,
> > > > >>>>>> if the application uses this value as array index,
> > > > >>>>>> there can be an access out of bounds.
> > > > >>>>>
> > > > >>>>> [Fiona] All asymmetric APIs are experimental so above shouldn't be a
> > > > problem.
> > > > >>>>> But for the same issue with sym crypto below, I believe Ferruh's
> > > > explanation makes
> > > > >>>>> sense and I don't see how there can be an API breakage.
> > > > >>>>> So if an application hasn't compiled against the new lib it will be still
> > using
> > > > the old value
> > > > >>>>> which will be within bounds. If it's picking up the higher new value
> > from
> > > > the lib it must
> > > > >>>>> have been compiled against the lib so shouldn't have problems.
> > > > >>>>
> > > > >>>> You say there is no ABI issue because the application will be re-
> > compiled
> > > > >>>> for the updated library. Indeed, compilation fixes compatibility issues.
> > > > >>>> But this is not relevant for ABI compatibility.
> > > > >>>> ABI compatibility means we can upgrade the library without
> > recompiling
> > > > >>>> the application and it must work.
> > > > >>>> You think it is a false positive because you assume the application
> > > > >>>> "picks" the new value. I think you miss the case where the new value
> > > > >>>> is returned by a function in the upgraded library.
> > > > >>>>
> > > > >>>>> There are also no structs on the API which contain arrays using this
> > > > >>>>> for sizing, so I don't see an opportunity for an appl to have a
> > > > >>>>> mismatch in memory addresses.
> > > > >>>>
> > > > >>>> Let me demonstrate where the API may "use" the new value
> > > > >>>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 and how it impacts the
> > > > application.
> > > > >>>>
> > > > >>>> Once upon a time a DPDK application counting the number of devices
> > > > >>>> supporting each AEAD algo (in order to find the best supported algo).
> > > > >>>> It is done in an array indexed by algo id:
> > > > >>>> int aead_dev_count[RTE_CRYPTO_AEAD_LIST_END];
> > > > >>>> The application is compiled with DPDK 19.11,
> > > > >>>> where RTE_CRYPTO_AEAD_LIST_END = 3.
> > > > >>>> So the size of the application array aead_dev_count is 3.
> > > > >>>> This binary is run with DPDK 20.02,
> > > > >>>> where RTE_CRYPTO_AEAD_CHACHA20_POLY1305 = 3.
> > > > >>>> When calling rte_cryptodev_info_get() on a device QAT_GEN3,
> > > > >>>> rte_cryptodev_info.capabilities.sym.aead.algo is set to
> > > > >>>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 (= 3).
> > > > >>>> The application uses this value:
> > > > >>>> ++ aead_dev_count[info.capabilities.sym.aead.algo];
> > > > >>>> The application is crashing because of out of bound access.
> > > > >>>
> > > > >>> I'd say this is an example of bad written app.
> > > > >>> It probably should check that returned by library value doesn't
> > > > >>> exceed its internal array size.
> > > > >>
> > > > >> +1
> > > > >>
> > > > >> Application should ignore values >= MAX.
> > > > >
> > > > > Of course, blaming the API user is a lot easier than looking at the API.
> > > > > Here the API has RTE_CRYPTO_AEAD_LIST_END which can be understood
> > > > > as the max value for the application.
> > > > > Value ranges are part of the ABI compatibility contract.
> > > > > It seems you expect the application developer to be aware that
> > > > > DPDK could return a higher value, so the application should
> > > > > check every enum values after calling an API. CRAZY.
> > > > >
> > > > > When we decide to announce an ABI compatibility and do some marketing,
> > > > > everyone is OK. But when we need to really make our ABI compatible,
> > > > > I see little or no effort. DISAPPOINTING.
> > > >
> > > > This is not to blame the user or to do less work, this is more sane approach
> > > > that library provides the _END/_MAX value and application uses it as valid
> > range
> > > > check.
> > > >
> > > > >
> > > > >> Do you suggest we don't extend any enum or define between ABI
> > breakage
> > > > releases
> > > > >> to be sure bad written applications not affected?
> > > > >
> > > > > I suggest we must consider not breaking any assumption made on the API.
> > > > > Here we are breaking the enum range because nothing mentions
> > _LIST_END
> > > > > is not really the absolute end of the enum.
> > > > > The solution is to make the change below in 20.02 + backport in 19.11.1:
> > > > >
> > > > > - _LIST_END
> > > > > + _LIST_END, /* an ABI-compatible version may increase this value */
> > > > > + _LIST_MAX = _LIST_END + 42 /* room for ABI-compatible additions */
> > > > > };
> > > > >
> > > >
> > > > What is the point of "_LIST_MAX" here?
> > > >
> > > > Application should know the "_LIST_END" of when it has been compiled for
> > the
> > > > valid range check. Next time it is compiled "_LIST_END" may be different
> > value
> > > > but same logic applies.
> > > >
> > > > When "_LIST_END" is missing, application can't protect itself, in that case
> > > > library should send only the values application knows when it is compiled,
> > this
> > > > means either we can't extend our enum/defines until next ABI breakage, or
> > we
> > > > need to do ABI versioning to the functions that returns an enum each time
> > enum
> > > > value extended.
> > > >
> > > > I believe it is saner to provide _END/_MAX values to the application to use.
> > And
> > > > if required comment them to clarify the expected usage.
> > > >
> > > > But in above suggestion application can't use or rely on "_LIST_MAX", it
> > doesn't
> > > > mean anything to application.
> > > >
> > >
> > > Can we have something like
> > > enum rte_crypto_aead_algorithm {
> > > RTE_CRYPTO_AEAD_AES_CCM = 1,
> > > /**< AES algorithm in CCM mode. */
> > > RTE_CRYPTO_AEAD_AES_GCM,
> > > /**< AES algorithm in GCM mode. */
> > > RTE_CRYPTO_AEAD_LIST_END,
> > > /**< List end for 19.11 ABI compatibility */
> > > RTE_CRYPTO_AEAD_CHACHA20_POLY1305,
> > > /**< Chacha20 cipher with poly1305 authenticator */
> > > RTE_CRYPTO_AEAD_LIST_END_2011
> > > /**< List end for 20.11 ABI compatibility */
> > > };
> > >
> > > And in 20.11 release we alter the RTE_CRYPTO_AEAD_LIST_END to the end
> > and remove RTE_CRYPTO_AEAD_LIST_END_2011
> > >
> > > I believe it will be ok for any application which need to use the chacha poly
> > assume that this algo is
> > > Experimental and will move to formal list in 20.11. This can be documented in
> > the documentation.
> > > I believe there is no way to add a new enum as experimental so far. This way
> > we can formalize this
> > > requirement as well.
> > >
> > > I believe this way effect of ABI breakage will be nullified.
> >
> > This is a possibility in the (a) proposal.
> > But it breaks API (and ABI) because a high value is returned
> > while not expected by the application.
> >
> > I guess ABI and release maintainers will vote no to such breakage.
> > Note: I vote no.
> >
>
> If that is the case, I would say we should go with b).
>
> Versioned APIs does not look good and adds more confusion.
>
How does it add confusion, it's the standard and recommended way to fix
things like this? To maintain stable ABIs in the medium and long term we
need to get used to using versioning and not be afraid of it. Developers
will soon get used to the added bit of complexity it involves.
Regards,
/Bruce
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [RFC] meter: fix ABI break due to experimental tag removal
2020-02-03 12:53 4% ` Ferruh Yigit
@ 2020-02-04 12:02 9% ` Neil Horman
2020-02-05 10:04 4% ` Luca Boccassi
0 siblings, 1 reply; 200+ results
From: Neil Horman @ 2020-02-04 12:02 UTC (permalink / raw)
To: Ferruh Yigit
Cc: Cristian Dumitrescu, Eelco Chaudron, dev, Thomas Monjalon,
Luca Boccassi, David Marchand, Bruce Richardson, Ian Stokes
On Mon, Feb 03, 2020 at 12:53:28PM +0000, Ferruh Yigit wrote:
> On 2/2/2020 6:53 PM, Neil Horman wrote:
> > On Wed, Jan 29, 2020 at 12:29:53PM +0000, Ferruh Yigit wrote:
> >> Duplicated the existing symbol and versioned one as experimental and
> >> other as stable.
> >>
> >> Created VERSION_SYMBOL_EXPERIMENTAL helper macro.
> >>
> >> Updated the 'check-experimental-syms.sh' buildtool, which was
> >> complaining that the symbol is in EXPERIMENTAL tag in .map file but it
> >> is not in the .experimental section (__rte_experimental tag is missing).
> >> Updated tool in a way it won't complain if the symbol in the
> >> EXPERIMENTAL tag duplicated in some other block in .map file (versioned)
> >>
> >> Updated meson build system to allow the versioning,
> >> 'use_function_versioning = true', not sure why it was disabled by
> >> default.
> >>
> >> Fixes: 30512af820fe ("meter: remove experimental flag from RFC4115 trTCM API")
> >>
> > I'm not sure I understand the purpose here.
> >
> > I think I understand what you are trying to do. I believe that you are
> > trying to move the experimental symbols in librte_meter to be part of
> > the official ABI, while still maintaining backward compatibility with
> > applications built against the experimental symbol versions, correct?
>
> Yes, exactly.
>
Ok, so we're on the same page, good.
> >
> > I can understand the desire, and I'm not exactly opposed to providing a
> > mechanism for that, but it seems somewhat complex, and to be honest,
> > part of the drawback to using experimental symbols as an application
> > developer is understanding that you may need to rebuild when those
> > symbols change (even if the change is the symbol removal and replacement
> > with an identical one that has a versioned tag).
> >
> > I think the right answer for people encountering this condition is to
> > just rebuild. Otherwise, we are creating an environment in which we are
> > implicitly communicating to users that we are maintaining a modicum of
> > stability in experimental symobls.
>
> This will mean, even the the API was perfect it will force its users to
> recompile (and re-package, re-deploy etc..) at least once, this is feeling like
Yes, thats correct.
> we are still breaking user applications easily.
we reserve that right, based on the fact that we marked these symbols as
experimental. Thats the deal we provide. Stable (non experimental) API calls
are guaranteed not to change for the ABI lifetime, and its our responsibility to
maintain that. However, experimental API's are marked as such because we can
change them (even if that change is to remove it and replace it with an
identical stable version), and application developers assume that risk by using
them. I understand the removal of an experimental symbol, replacing it with a
stable identical version is a trivial case of that change, and if we can ease
the burden, thats fine, but I don't think we need to go out of our way, or incur
any additional burden to ease that transition. We created the experimental
symbol marking to ease the burden of this sort of development on us, we should
be willing to use it.
> And this may create a stronger motivation by applications not to use
> experimental APIs, I can't decide if this is a good thing or bad thing.
>
Yeah, for stable, long term support applications, that makes sense, they
shouldn't be using experimental API's when they need the guarantee of support,
or they should be understanding of their responsibility to rebuild, when we
change the API's in both complex and trivial cases like this.
>
> The issue is amplified in the LTS,
> If we remove experimental tag in LTS, will be breaking the apps using that
> experimental API, just to mature the API.
You shouldn't be making this sort of transition in LTS. Just leave the
experimental symbol as is there, and commit the transition for the next LTS
release.
> If we keep it as experimental, the API will be mature in main repo, but the LTS
> has to keep exact same API as experimental up to two years.
Yes, thats the meaning of LTS. Things stay stable for its lifetime. And if the
only thing you are doing is replacing the experimental ABI version with a stable
version (leaving the details of the ABI unchanged), then its a moot point. The
only difference will be that the LTS release will have symbols marked as
experimental.
>
> But if we can do the versioning in the master, LTS can backport it and can have
> mature version of that API in LTS without breaking the existing users.
>
But why bother? The only thing you've changed is the version tagging. Its ok
to leave that alone in LTS, you just cant change it.
Thats part of the burden of an LTS release, it will have some drift from the
upstream head, because you have to keep things stable. So you stabilize the
upstream ABI version for this API and just never backport it to the current LTS
release.
> >
> > A few more nits in line
> >
> >> Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
> >> ---
> >> Cc: Neil Horman <nhorman@tuxdriver.com>
> >> Cc: Thomas Monjalon <thomas@monjalon.net>
> >> Cc: Luca Boccassi <bluca@debian.org>
> >> Cc: David Marchand <david.marchand@redhat.com>
> >> Cc: Bruce Richardson <bruce.richardson@intel.com>
> >> Cc: Ian Stokes <ian.stokes@intel.com>
> >> Cc: Eelco Chaudron <echaudro@redhat.com>
> >>
> >> I didn't like the idea of duplicating the symbol but not able to find to
> >> alias it without duplicating, if there is a way please share.
> >>
> >> Also not tested with old binaries, only verified the symbols in new
> >> binaries.
> >> ---
> >> buildtools/check-experimental-syms.sh | 3 +-
> >> .../common/include/rte_function_versioning.h | 4 ++
> >> lib/librte_meter/rte_meter.c | 59 ++++++++++++++++++-
> >> lib/librte_meter/rte_meter_version.map | 8 +++
> >> lib/meson.build | 2 +-
> >> 5 files changed, 71 insertions(+), 5 deletions(-)
> >>
> >> diff --git a/buildtools/check-experimental-syms.sh b/buildtools/check-experimental-syms.sh
> >> index f3603e5ba..35c4bf006 100755
> >> --- a/buildtools/check-experimental-syms.sh
> >> +++ b/buildtools/check-experimental-syms.sh
> >> @@ -26,7 +26,8 @@ ret=0
> >> for SYM in `$LIST_SYMBOL -S EXPERIMENTAL $MAPFILE |cut -d ' ' -f 3`
> >> do
> >> if grep -q "\.text.*[[:space:]]$SYM$" $DUMPFILE &&
> >> - ! grep -q "\.text\.experimental.*[[:space:]]$SYM$" $DUMPFILE
> >> + ! grep -q "\.text\.experimental.*[[:space:]]$SYM$" $DUMPFILE &&
> >> + $LIST_SYMBOL -s $SYM $MAPFILE | grep -q EXPERIMENTAL
> >> then
> >> cat >&2 <<- END_OF_MESSAGE
> >> $SYM is not flagged as experimental
> >> diff --git a/lib/librte_eal/common/include/rte_function_versioning.h b/lib/librte_eal/common/include/rte_function_versioning.h
> >> index c924351d5..ab102b06e 100644
> >> --- a/lib/librte_eal/common/include/rte_function_versioning.h
> >> +++ b/lib/librte_eal/common/include/rte_function_versioning.h
> >> @@ -46,6 +46,9 @@
> >> */
> >> #define VERSION_SYMBOL(b, e, n) __asm__(".symver " RTE_STR(b) RTE_STR(e) ", " RTE_STR(b) "@DPDK_" RTE_STR(n))
> >>
> >> +
> >> +#define VERSION_SYMBOL_EXPERIMENTAL(b, e) __asm__(".symver " RTE_STR(b) RTE_STR(e) ", " RTE_STR(b) "@EXPERIMENTAL")
> >> +
> > I don't know that you want to make a explicit clone of this macro.
> > instead what you might want to do is something like:
> >
> > #define __VERSION_SYMBOL(b, e, n, t) __asm__(".symver " RTE_STR(b) RTE_STR(e) ", " RTE_STR(b) "@" RTE_STR(t) "_" RTE_STR(n))
> >
> > #define VERSION_SYMBOL(b, e, n) __VERSION_SYMBOL(b, e, n, DPDK)
> >
> > Thats not exactly right, but you get the idea, then you can emulate
> > VERSION_SYMBOL_EXPERIMENTAL with __VERSION_SYMBOL(b, e, n, EXPERIMENTAL)
>
> +1
>
> >
> >
> >> /*
> >> * BIND_DEFAULT_SYMBOL
> >> * Creates a symbol version entry instructing the linker to bind references to
> >> @@ -79,6 +82,7 @@
> >> * No symbol versioning in use
> >> */
> >> #define VERSION_SYMBOL(b, e, n)
> >> +#define VERSION_SYMBOL_EXPERIMENTAL(b, e)
> >> #define __vsym
> >> #define BIND_DEFAULT_SYMBOL(b, e, n)
> >> #define MAP_STATIC_SYMBOL(f, p) f __attribute__((alias(RTE_STR(p))))
> >> diff --git a/lib/librte_meter/rte_meter.c b/lib/librte_meter/rte_meter.c
> >> index da01429a8..5244537fa 100644
> >> --- a/lib/librte_meter/rte_meter.c
> >> +++ b/lib/librte_meter/rte_meter.c
> >> @@ -9,6 +9,7 @@
> >> #include <rte_common.h>
> >> #include <rte_log.h>
> >> #include <rte_cycles.h>
> >> +#include <rte_function_versioning.h>
> >>
> >> #include "rte_meter.h"
> >>
> >> @@ -119,8 +120,8 @@ rte_meter_trtcm_config(struct rte_meter_trtcm *m,
> >> return 0;
> >> }
> >>
> >> -int
> >> -rte_meter_trtcm_rfc4115_profile_config(
> >> +static int
> >> +rte_meter_trtcm_rfc4115_profile_config_(
> >> struct rte_meter_trtcm_rfc4115_profile *p,
> >> struct rte_meter_trtcm_rfc4115_params *params)
> >> {
> >> @@ -145,7 +146,35 @@ rte_meter_trtcm_rfc4115_profile_config(
> >> }
> >>
> >> int
> >> -rte_meter_trtcm_rfc4115_config(
> >> +rte_meter_trtcm_rfc4115_profile_config_s(
> >> + struct rte_meter_trtcm_rfc4115_profile *p,
> >> + struct rte_meter_trtcm_rfc4115_params *params);
> >> +int
> >> +rte_meter_trtcm_rfc4115_profile_config_s(
> >> + struct rte_meter_trtcm_rfc4115_profile *p,
> >> + struct rte_meter_trtcm_rfc4115_params *params)
> >> +{
> >> + return rte_meter_trtcm_rfc4115_profile_config_(p, params);
> >> +}
> >> +BIND_DEFAULT_SYMBOL(rte_meter_trtcm_rfc4115_profile_config, _s, 20.0.1);
> >> +MAP_STATIC_SYMBOL(int rte_meter_trtcm_rfc4115_profile_config(struct rte_meter_trtcm_rfc4115_profile *p,
> >> + struct rte_meter_trtcm_rfc4115_params *params), rte_meter_trtcm_rfc4115_profile_config_s);
> >> +
> >> +int
> >> +rte_meter_trtcm_rfc4115_profile_config_e(
> >> + struct rte_meter_trtcm_rfc4115_profile *p,
> >> + struct rte_meter_trtcm_rfc4115_params *params);
> >> +int
> >> +rte_meter_trtcm_rfc4115_profile_config_e(
> >> + struct rte_meter_trtcm_rfc4115_profile *p,
> >> + struct rte_meter_trtcm_rfc4115_params *params)
> >> +{
> >> + return rte_meter_trtcm_rfc4115_profile_config_(p, params);
> >> +}
> >> +VERSION_SYMBOL_EXPERIMENTAL(rte_meter_trtcm_rfc4115_profile_config, _e);
> >> +
> >> +static int
> >> +rte_meter_trtcm_rfc4115_config_(
> >> struct rte_meter_trtcm_rfc4115 *m,
> >> struct rte_meter_trtcm_rfc4115_profile *p)
> >> {
> >> @@ -160,3 +189,27 @@ rte_meter_trtcm_rfc4115_config(
> >>
> >> return 0;
> >> }
> >> +
> >> +int
> >> +rte_meter_trtcm_rfc4115_config_s(struct rte_meter_trtcm_rfc4115 *m,
> >> + struct rte_meter_trtcm_rfc4115_profile *p);
> >> +int
> >> +rte_meter_trtcm_rfc4115_config_s(struct rte_meter_trtcm_rfc4115 *m,
> >> + struct rte_meter_trtcm_rfc4115_profile *p)
> >> +{
> >> + return rte_meter_trtcm_rfc4115_config_(m, p);
> >> +}
> >> +BIND_DEFAULT_SYMBOL(rte_meter_trtcm_rfc4115_config, _s, 20.0.1);
> >> +MAP_STATIC_SYMBOL(int rte_meter_trtcm_rfc4115_config(struct rte_meter_trtcm_rfc4115 *m,
> >> + struct rte_meter_trtcm_rfc4115_profile *p), rte_meter_trtcm_rfc4115_config_s);
> >> +
> >> +int
> >> +rte_meter_trtcm_rfc4115_config_e(struct rte_meter_trtcm_rfc4115 *m,
> >> + struct rte_meter_trtcm_rfc4115_profile *p);
> >> +int
> >> +rte_meter_trtcm_rfc4115_config_e(struct rte_meter_trtcm_rfc4115 *m,
> >> + struct rte_meter_trtcm_rfc4115_profile *p)
> >> +{
> >> + return rte_meter_trtcm_rfc4115_config_(m, p);
> >> +}
> >> +VERSION_SYMBOL_EXPERIMENTAL(rte_meter_trtcm_rfc4115_config, _e);
> >> diff --git a/lib/librte_meter/rte_meter_version.map b/lib/librte_meter/rte_meter_version.map
> >> index fc12cc0bf..b4b043174 100644
> >> --- a/lib/librte_meter/rte_meter_version.map
> >> +++ b/lib/librte_meter/rte_meter_version.map
> >> @@ -20,4 +20,12 @@ DPDK_20.0.1 {
> >> rte_meter_trtcm_rfc4115_color_blind_check;
> >> rte_meter_trtcm_rfc4115_config;
> >> rte_meter_trtcm_rfc4115_profile_config;
> >> +
> >> } DPDK_20.0;
> >> +
> >> +EXPERIMENTAL {
> >> + global:
> >> +
> >> + rte_meter_trtcm_rfc4115_config;
> >> + rte_meter_trtcm_rfc4115_profile_config;
> >> +};
> >> diff --git a/lib/meson.build b/lib/meson.build
> >> index 0af3efab2..553964831 100644
> >> --- a/lib/meson.build
> >> +++ b/lib/meson.build
> >> @@ -48,7 +48,7 @@ foreach l:libraries
> >> reason = '<unknown reason>' # set if build == false to explain why
> >> name = l
> >> allow_experimental_apis = false
> >> - use_function_versioning = false
> >> + use_function_versioning = true
> >> sources = []
> >> headers = []
> >> includes = []
> >> --
> >> 2.24.1
> >>
> >>
>
>
^ permalink raw reply [relevance 9%]
* Re: [dpdk-dev] [PATCH v7] eal: add manual probing option
2020-02-04 11:07 0% ` Thomas Monjalon
@ 2020-02-04 12:43 0% ` Gaetan Rivet
2020-02-04 15:06 0% ` Thomas Monjalon
0 siblings, 1 reply; 200+ results
From: Gaetan Rivet @ 2020-02-04 12:43 UTC (permalink / raw)
To: Thomas Monjalon, Pavan Nikhilesh Bhagavatula, David Marchand
Cc: dev, Vamsi Krishna Attunuru, Jerin Jacob Kollanukkaran
On 04/02/2020 12:07, Thomas Monjalon wrote:
> 04/02/2020 11:03, Gaetan Rivet:
>> On 03/02/2020 23:21, Thomas Monjalon wrote:
>>> 03/02/2020 06:16, Pavan Nikhilesh Bhagavatula:
>>>> @David Marchand @thomas@monjalon.net
>>>>
>>>> Ping?
>>>>
>>>> Are there any more changes required for this patch? It's been in queue since last October.
>>>
>>> Sorry we have not decided whether it is a good idea or not.
>>>
>>> All changes related to probing are very sensitive,
>>> and we know a big refactoring would be better than stacking
>>> more and more options and corner cases.
>>>
>>> As we are busy with ABI stability stuff, we did not allocate
>>> enough time to properly think about this feature.
>>> Please accept our apologies, and let's consider it as
>>> a high priority for 20.05 cycle.
>>>
>>
>> Hello Thomas,
>>
>> This is unfortunate. I pushed Pavan to accept an alternative implementation of this functionality that was less obtrusive, to make the integration smoother. I took care to alleviate those risks from the common path.
>>
>> The big refactoring is needed yes, but considering the current path I'm not seeing it happen in 20.05. If that means taking this patch as-is in 20.05 for Marvell users, I'm not sure much is gained from waiting 3 months, except minimal risk avoidance.
>
>
> Yes, life is full of bad decisions and consequences.
Ah, yes, but I stand by my initial opinion, the first implementation [1] was riskier and less useful.
>
> I still think there is a risk in adding new user expectations,
> and maintaining some code to workaround unknown issues.
>
> The real question here is to know why this patch?
> Is it to workaround a broken driver?
> Or to workaround a broken design in EAL and bus drivers?
>
>
Two birds - one stone here: OVS needed a way to disable automatic probing cleanly (current workaround seen in multiple deployment is to add a dummy whitelisted device, which will be ignored by the PCI bus --> it sets the bus in whitelist mode but avoid probing anything), and as a bonus this option allows using devices that depends on other devices being probed already (LAG, representors, failsafe, etc).
I'm not sure having a dependent-probe by default is good, and that would be a big change.
If we are doing the genesis of this patch, the initial motivation should be asked for more details from Marvell people and David for the OVS side.
[1]: First proposal:
http://mails.dpdk.org/archives/dev/2019-September/144166.html
My arguments:
http://mails.dpdk.org/archives/dev/2019-September/144564.html
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
2020-02-04 10:24 9% ` Thomas Monjalon
@ 2020-02-04 12:44 4% ` Trahe, Fiona
2020-02-04 15:52 8% ` Trahe, Fiona
2020-02-04 12:57 4% ` Kevin Traynor
2020-02-04 14:44 4% ` Aaron Conole
2 siblings, 1 reply; 200+ results
From: Trahe, Fiona @ 2020-02-04 12:44 UTC (permalink / raw)
To: Thomas Monjalon, David Marchand, nhorman, bluca, ktraynor,
Ray Kinsella, dev, Akhil Goyal, Yigit, Ferruh
Cc: Ananyev, Konstantin, dev, Anoob Joseph, Kusztal, ArkadiuszX,
Richardson, Bruce, Mcnamara, John, dodji, Andrew Rybchenko,
aconole, Trahe, Fiona
> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: Tuesday, February 4, 2020 10:24 AM
> To: David Marchand <david.marchand@redhat.com>; nhorman@tuxdriver.com; bluca@debian.org;
> ktraynor@redhat.com; Ray Kinsella <mdr@ashroe.eu>; dev@dpdk.org; Akhil Goyal
> <akhil.goyal@nxp.com>; Trahe, Fiona <fiona.trahe@intel.com>; Yigit, Ferruh <ferruh.yigit@intel.com>
> Cc: Ananyev, Konstantin <konstantin.ananyev@intel.com>; dev@dpdk.org; Anoob Joseph
> <anoobj@marvell.com>; Kusztal, ArkadiuszX <arkadiuszx.kusztal@intel.com>; Richardson, Bruce
> <bruce.richardson@intel.com>; Mcnamara, John <john.mcnamara@intel.com>; dodji@seketeli.net;
> Andrew Rybchenko <arybchenko@solarflare.com>; aconole@redhat.com
> Subject: Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
>
> RED FLAG
>
> I don't see a lot of reactions, so I summarize the issue.
> We need action TODAY!
>
> API makes think that rte_cryptodev_info_get() cannot return
> a value >= 3 (RTE_CRYPTO_AEAD_LIST_END in 19.11).
> Current 20.02 returns 3 (RTE_CRYPTO_AEAD_CHACHA20_POLY1305).
> The ABI compatibility contract is broken currently.
>
> There are 3 possible outcomes:
>
> a) Change the API comments and backport to 19.11.1
> The details are discussed between Ferruh and me.
> Either put responsibility on API user (with explicit comment),
> or expose ABI extension allowance with a new API max value.
> In both cases, this is breaking the implicit contract of 19.11.0.
> This option can be chosen only if release and ABI maintainers
> vote for it.
>
> b) Revert Chacha-Poly from 20.02-rc2.
>
> c) Add versioned function rte_cryptodev_info_get_v1911()
> which calls rte_cryptodev_info_get() and filters out
> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 capability.
> So Chacha-Poly capability would be seen and usable only
> if compiling with DPDK 20.02.
>
> I hope it is clear what are the actions for everybody:
> - ABI and release maintainers must say yes or no to the proposal (a)
> - In the meantime, crypto team must send a patch for the proposal (c)
> - If (a) and (c) are not possible at the end of today, I will take (b)
>
> Note: do not say it is too short for (c), as it was possible to work
> on such solution since the issue was exposed on last Wednesday.
>
[Fiona] Thanks for your understanding and help with our scheduling Thomas.
We are working on a patch, when it is ready we will send it.
If it's not ready by end of your today, of course, go ahead with (b) and
we will work towards 20.05.
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
2020-02-04 10:24 9% ` Thomas Monjalon
2020-02-04 12:44 4% ` Trahe, Fiona
@ 2020-02-04 12:57 4% ` Kevin Traynor
2020-02-04 14:44 4% ` Aaron Conole
2 siblings, 0 replies; 200+ results
From: Kevin Traynor @ 2020-02-04 12:57 UTC (permalink / raw)
To: Thomas Monjalon, David Marchand, nhorman, bluca, Ray Kinsella,
dev, Akhil Goyal, Trahe, Fiona, Ferruh Yigit
Cc: Ananyev, Konstantin, Anoob Joseph, Kusztal, ArkadiuszX,
Richardson, Bruce, Mcnamara, John, dodji, Andrew Rybchenko,
aconole
On 04/02/2020 10:24, Thomas Monjalon wrote:
> RED FLAG
>
> I don't see a lot of reactions, so I summarize the issue.
> We need action TODAY!
>
> API makes think that rte_cryptodev_info_get() cannot return
> a value >= 3 (RTE_CRYPTO_AEAD_LIST_END in 19.11).
> Current 20.02 returns 3 (RTE_CRYPTO_AEAD_CHACHA20_POLY1305).
> The ABI compatibility contract is broken currently.
>
> There are 3 possible outcomes:
>
> a) Change the API comments and backport to 19.11.1
> The details are discussed between Ferruh and me.
> Either put responsibility on API user (with explicit comment),
> or expose ABI extension allowance with a new API max value.
> In both cases, this is breaking the implicit contract of 19.11.0.
> This option can be chosen only if release and ABI maintainers
> vote for it.
>
> b) Revert Chacha-Poly from 20.02-rc2.
>
> c) Add versioned function rte_cryptodev_info_get_v1911()
> which calls rte_cryptodev_info_get() and filters out
> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 capability.
> So Chacha-Poly capability would be seen and usable only
> if compiling with DPDK 20.02.
>
Maybe a separate version of rte_cryptodev_get_aead_algo_enum() also
needed to handle chacha string differently.
> I hope it is clear what are the actions for everybody:
> - ABI and release maintainers must say yes or no to the proposal (a)
My 2c for a) is No.
> - In the meantime, crypto team must send a patch for the proposal (c)
> - If (a) and (c) are not possible at the end of today, I will take (b)
>
> Note: do not say it is too short for (c), as it was possible to work
> on such solution since the issue was exposed on last Wednesday.
>
Could it be reverted today if necessary and re-added later in the
release cycle? It seems like something modular that should not
invalidate earlier testing.
>
> 03/02/2020 22:07, Thomas Monjalon:
>> 03/02/2020 19:55, Ray Kinsella:
>>> On 03/02/2020 17:34, Thomas Monjalon wrote:
>>>> 03/02/2020 18:09, Thomas Monjalon:
>>>>> 03/02/2020 10:30, Ferruh Yigit:
>>>>>> On 2/2/2020 2:41 PM, Ananyev, Konstantin wrote:
>>>>>>> 02/02/2020 14:05, Thomas Monjalon:
>>>>>>>> 31/01/2020 15:16, Trahe, Fiona:
>>>>>>>>> On 1/30/2020 8:18 PM, Thomas Monjalon wrote:
>>>>>>>>>> If library give higher value than expected by the application,
>>>>>>>>>> if the application uses this value as array index,
>>>>>>>>>> there can be an access out of bounds.
>>>>>>>>>
>>>>>>>>> [Fiona] All asymmetric APIs are experimental so above shouldn't be a problem.
>>>>>>>>> But for the same issue with sym crypto below, I believe Ferruh's explanation makes
>>>>>>>>> sense and I don't see how there can be an API breakage.
>>>>>>>>> So if an application hasn't compiled against the new lib it will be still using the old value
>>>>>>>>> which will be within bounds. If it's picking up the higher new value from the lib it must
>>>>>>>>> have been compiled against the lib so shouldn't have problems.
>>>>>>>>
>>>>>>>> You say there is no ABI issue because the application will be re-compiled
>>>>>>>> for the updated library. Indeed, compilation fixes compatibility issues.
>>>>>>>> But this is not relevant for ABI compatibility.
>>>>>>>> ABI compatibility means we can upgrade the library without recompiling
>>>>>>>> the application and it must work.
>>>>>>>> You think it is a false positive because you assume the application
>>>>>>>> "picks" the new value. I think you miss the case where the new value
>>>>>>>> is returned by a function in the upgraded library.
>>>>>>>>
>>>>>>>>> There are also no structs on the API which contain arrays using this
>>>>>>>>> for sizing, so I don't see an opportunity for an appl to have a
>>>>>>>>> mismatch in memory addresses.
>>>>>>>>
>>>>>>>> Let me demonstrate where the API may "use" the new value
>>>>>>>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 and how it impacts the application.
>>>>>>>>
>>>>>>>> Once upon a time a DPDK application counting the number of devices
>>>>>>>> supporting each AEAD algo (in order to find the best supported algo).
>>>>>>>> It is done in an array indexed by algo id:
>>>>>>>> int aead_dev_count[RTE_CRYPTO_AEAD_LIST_END];
>>>>>>>> The application is compiled with DPDK 19.11,
>>>>>>>> where RTE_CRYPTO_AEAD_LIST_END = 3.
>>>>>>>> So the size of the application array aead_dev_count is 3.
>>>>>>>> This binary is run with DPDK 20.02,
>>>>>>>> where RTE_CRYPTO_AEAD_CHACHA20_POLY1305 = 3.
>>>>>>>> When calling rte_cryptodev_info_get() on a device QAT_GEN3,
>>>>>>>> rte_cryptodev_info.capabilities.sym.aead.algo is set to
>>>>>>>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 (= 3).
>>>>>>>> The application uses this value:
>>>>>>>> ++ aead_dev_count[info.capabilities.sym.aead.algo];
>>>>>>>> The application is crashing because of out of bound access.
>>>>>>>
>>>>>>> I'd say this is an example of bad written app.
>>>>>>> It probably should check that returned by library value doesn't
>>>>>>> exceed its internal array size.
>>>>>>
>>>>>> +1
>>>>>>
>>>>>> Application should ignore values >= MAX.
>>>>>
>>>>> Of course, blaming the API user is a lot easier than looking at the API.
>>>>> Here the API has RTE_CRYPTO_AEAD_LIST_END which can be understood
>>>>> as the max value for the application.
>>>>> Value ranges are part of the ABI compatibility contract.
>>>>> It seems you expect the application developer to be aware that
>>>>> DPDK could return a higher value, so the application should
>>>>> check every enum values after calling an API. CRAZY.
>>>>>
>>>>> When we decide to announce an ABI compatibility and do some marketing,
>>>>> everyone is OK. But when we need to really make our ABI compatible,
>>>>> I see little or no effort. DISAPPOINTING.
>>>>>
>>>>>> Do you suggest we don't extend any enum or define between ABI breakage releases
>>>>>> to be sure bad written applications not affected?
>>>>>
>>>>> I suggest we must consider not breaking any assumption made on the API.
>>>>> Here we are breaking the enum range because nothing mentions _LIST_END
>>>>> is not really the absolute end of the enum.
>>>>> The solution is to make the change below in 20.02 + backport in 19.11.1:
>>>>
>>>> Thinking twice, merging such change before 20.11 is breaking the
>>>> ABI assumption based on the API 19.11.0.
>>>> I ask the release maintainers (Luca, Kevin, David and me) and
>>>> the ABI maintainers (Neil and Ray) to vote for a or b solution:
>>>> a) add comment and LIST_MAX as below in 20.02 + 19.11.1
>>>
>>> That would still be an ABI breakage though right.
>>>
>>>> b) wait 20.11 and revert Chacha-Poly from 20.02
>>>
>>> Thanks for analysis above Fiona, Ferruh and all.
>>>
>>> That is a nasty one alright - there is no "good" answer here.
>>> I agree with Ferruh's sentiments overall, we should rethink this API for 20.11.
>>> Could do without an enumeration?
>>>
>>> There a c) though right.
>>> We could work around the issue by api versioning rte_cryptodev_info_get() and friends.
>>> So they only support/acknowledge the existence of Chacha-Poly for applications build against > 20.02.
>>
>> I agree there is a c) as I proposed in another email:
>> http://mails.dpdk.org/archives/dev/2020-February/156919.html
>> "
>> In this case, the proper solution is to implement
>> rte_cryptodev_info_get_v1911() so it filters out
>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 capability.
>> With this solution, an application compiled with DPDK 19.11 will keep
>> seeing the same range as before, while a 20.02 application could
>> see and use ChachaPoly.
>> "
>>
>>> It would be painful I know.
>>
>> Not so painful in my opinion.
>> Just need to call rte_cryptodev_info_get() from
>> rte_cryptodev_info_get_v1911() and filter the value
>> in the 19.11 range: [0..AES_GCM].
>>
>>> It would also mean that Chacha-Poly would only be available to
>>> those building against >= 20.02.
>>
>> Yes exactly.
>>
>> The addition of comments and LIST_MAX like below are still valid
>> to avoid versioning after 20.11.
>>
>>>>> - _LIST_END
>>>>> + _LIST_END, /* an ABI-compatible version may increase this value */
>>>>> + _LIST_MAX = _LIST_END + 42 /* room for ABI-compatible additions */
>>>>> };
>>>>>
>>>>> Then *_LIST_END values could be ignored by libabigail with such a change.
>>
>> In order to avoid ABI check complaining, the best is to completely
>> remove LIST_END in DPDK 20.11.
>>
>>
>>>>> If such a patch is not done by tomorrow, I will have to revert
>>>>> Chacha-Poly commits before 20.02-rc2, because
>>>>>
>>>>> 1/ LIST_END, without any comment, means "size of range"
>>>>> 2/ we do not blame users for undocumented ABI changes
>>>>> 3/ we respect the ABI compatibility contract
>
>
>
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
2020-02-04 10:24 9% ` Thomas Monjalon
2020-02-04 12:44 4% ` Trahe, Fiona
2020-02-04 12:57 4% ` Kevin Traynor
@ 2020-02-04 14:44 4% ` Aaron Conole
2020-02-04 19:49 7% ` Neil Horman
2 siblings, 1 reply; 200+ results
From: Aaron Conole @ 2020-02-04 14:44 UTC (permalink / raw)
To: Thomas Monjalon
Cc: David Marchand, nhorman, bluca, ktraynor, Ray Kinsella, dev,
Akhil Goyal, Trahe, Fiona, Ferruh Yigit, Ananyev, Konstantin,
Anoob Joseph, Kusztal, ArkadiuszX, Richardson, Bruce, Mcnamara,
John, dodji, Andrew Rybchenko
Thomas Monjalon <thomas@monjalon.net> writes:
> RED FLAG
>
> I don't see a lot of reactions, so I summarize the issue.
> We need action TODAY!
>
> API makes think that rte_cryptodev_info_get() cannot return
> a value >= 3 (RTE_CRYPTO_AEAD_LIST_END in 19.11).
> Current 20.02 returns 3 (RTE_CRYPTO_AEAD_CHACHA20_POLY1305).
> The ABI compatibility contract is broken currently.
>
> There are 3 possible outcomes:
>
> a) Change the API comments and backport to 19.11.1
> The details are discussed between Ferruh and me.
> Either put responsibility on API user (with explicit comment),
> or expose ABI extension allowance with a new API max value.
> In both cases, this is breaking the implicit contract of 19.11.0.
> This option can be chosen only if release and ABI maintainers
> vote for it.
>
> b) Revert Chacha-Poly from 20.02-rc2.
>
> c) Add versioned function rte_cryptodev_info_get_v1911()
> which calls rte_cryptodev_info_get() and filters out
> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 capability.
> So Chacha-Poly capability would be seen and usable only
> if compiling with DPDK 20.02.
>
> I hope it is clear what are the actions for everybody:
> - ABI and release maintainers must say yes or no to the proposal (a)
> - In the meantime, crypto team must send a patch for the proposal (c)
> - If (a) and (c) are not possible at the end of today, I will take (b)
>
> Note: do not say it is too short for (c), as it was possible to work
> on such solution since the issue was exposed on last Wednesday.
While I'm not a maintainer, if I my opinion counts for anything, I'd
choose option c or b. Absolutely NACK to a.
>
> 03/02/2020 22:07, Thomas Monjalon:
>> 03/02/2020 19:55, Ray Kinsella:
>> > On 03/02/2020 17:34, Thomas Monjalon wrote:
>> > > 03/02/2020 18:09, Thomas Monjalon:
>> > >> 03/02/2020 10:30, Ferruh Yigit:
>> > >>> On 2/2/2020 2:41 PM, Ananyev, Konstantin wrote:
>> > >>>> 02/02/2020 14:05, Thomas Monjalon:
>> > >>>>> 31/01/2020 15:16, Trahe, Fiona:
>> > >>>>>> On 1/30/2020 8:18 PM, Thomas Monjalon wrote:
>> > >>>>>>> If library give higher value than expected by the application,
>> > >>>>>>> if the application uses this value as array index,
>> > >>>>>>> there can be an access out of bounds.
>> > >>>>>>
>> > >>>>>> [Fiona] All asymmetric APIs are experimental so above shouldn't be a problem.
>> > >>>>>> But for the same issue with sym crypto below, I believe Ferruh's explanation makes
>> > >>>>>> sense and I don't see how there can be an API breakage.
>> > >>>>>> So if an application hasn't compiled against the new lib it
>> > >>>>>> will be still using the old value
>> > >>>>>> which will be within bounds. If it's picking up the higher
>> > >>>>>> new value from the lib it must
>> > >>>>>> have been compiled against the lib so shouldn't have problems.
>> > >>>>>
>> > >>>>> You say there is no ABI issue because the application will be re-compiled
>> > >>>>> for the updated library. Indeed, compilation fixes compatibility issues.
>> > >>>>> But this is not relevant for ABI compatibility.
>> > >>>>> ABI compatibility means we can upgrade the library without recompiling
>> > >>>>> the application and it must work.
>> > >>>>> You think it is a false positive because you assume the application
>> > >>>>> "picks" the new value. I think you miss the case where the new value
>> > >>>>> is returned by a function in the upgraded library.
>> > >>>>>
>> > >>>>>> There are also no structs on the API which contain arrays using this
>> > >>>>>> for sizing, so I don't see an opportunity for an appl to have a
>> > >>>>>> mismatch in memory addresses.
>> > >>>>>
>> > >>>>> Let me demonstrate where the API may "use" the new value
>> > >>>>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 and how it impacts the application.
>> > >>>>>
>> > >>>>> Once upon a time a DPDK application counting the number of devices
>> > >>>>> supporting each AEAD algo (in order to find the best supported algo).
>> > >>>>> It is done in an array indexed by algo id:
>> > >>>>> int aead_dev_count[RTE_CRYPTO_AEAD_LIST_END];
>> > >>>>> The application is compiled with DPDK 19.11,
>> > >>>>> where RTE_CRYPTO_AEAD_LIST_END = 3.
>> > >>>>> So the size of the application array aead_dev_count is 3.
>> > >>>>> This binary is run with DPDK 20.02,
>> > >>>>> where RTE_CRYPTO_AEAD_CHACHA20_POLY1305 = 3.
>> > >>>>> When calling rte_cryptodev_info_get() on a device QAT_GEN3,
>> > >>>>> rte_cryptodev_info.capabilities.sym.aead.algo is set to
>> > >>>>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 (= 3).
>> > >>>>> The application uses this value:
>> > >>>>> ++ aead_dev_count[info.capabilities.sym.aead.algo];
>> > >>>>> The application is crashing because of out of bound access.
>> > >>>>
>> > >>>> I'd say this is an example of bad written app.
>> > >>>> It probably should check that returned by library value doesn't
>> > >>>> exceed its internal array size.
>> > >>>
>> > >>> +1
>> > >>>
>> > >>> Application should ignore values >= MAX.
>> > >>
>> > >> Of course, blaming the API user is a lot easier than looking at the API.
>> > >> Here the API has RTE_CRYPTO_AEAD_LIST_END which can be understood
>> > >> as the max value for the application.
>> > >> Value ranges are part of the ABI compatibility contract.
>> > >> It seems you expect the application developer to be aware that
>> > >> DPDK could return a higher value, so the application should
>> > >> check every enum values after calling an API. CRAZY.
>> > >>
>> > >> When we decide to announce an ABI compatibility and do some marketing,
>> > >> everyone is OK. But when we need to really make our ABI compatible,
>> > >> I see little or no effort. DISAPPOINTING.
>> > >>
>> > >>> Do you suggest we don't extend any enum or define between ABI breakage releases
>> > >>> to be sure bad written applications not affected?
>> > >>
>> > >> I suggest we must consider not breaking any assumption made on the API.
>> > >> Here we are breaking the enum range because nothing mentions _LIST_END
>> > >> is not really the absolute end of the enum.
>> > >> The solution is to make the change below in 20.02 + backport in 19.11.1:
>> > >
>> > > Thinking twice, merging such change before 20.11 is breaking the
>> > > ABI assumption based on the API 19.11.0.
>> > > I ask the release maintainers (Luca, Kevin, David and me) and
>> > > the ABI maintainers (Neil and Ray) to vote for a or b solution:
>> > > a) add comment and LIST_MAX as below in 20.02 + 19.11.1
>> >
>> > That would still be an ABI breakage though right.
>> >
>> > > b) wait 20.11 and revert Chacha-Poly from 20.02
>> >
>> > Thanks for analysis above Fiona, Ferruh and all.
>> >
>> > That is a nasty one alright - there is no "good" answer here.
>> > I agree with Ferruh's sentiments overall, we should rethink this API for 20.11.
>> > Could do without an enumeration?
>> >
>> > There a c) though right.
>> > We could work around the issue by api versioning rte_cryptodev_info_get() and friends.
>> > So they only support/acknowledge the existence of Chacha-Poly for
>> > applications build against > 20.02.
>>
>> I agree there is a c) as I proposed in another email:
>> http://mails.dpdk.org/archives/dev/2020-February/156919.html
>> "
>> In this case, the proper solution is to implement
>> rte_cryptodev_info_get_v1911() so it filters out
>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 capability.
>> With this solution, an application compiled with DPDK 19.11 will keep
>> seeing the same range as before, while a 20.02 application could
>> see and use ChachaPoly.
>> "
>>
>> > It would be painful I know.
>>
>> Not so painful in my opinion.
>> Just need to call rte_cryptodev_info_get() from
>> rte_cryptodev_info_get_v1911() and filter the value
>> in the 19.11 range: [0..AES_GCM].
>>
>> > It would also mean that Chacha-Poly would only be available to
>> > those building against >= 20.02.
>>
>> Yes exactly.
>>
>> The addition of comments and LIST_MAX like below are still valid
>> to avoid versioning after 20.11.
>>
>> > >> - _LIST_END
>> > >> + _LIST_END, /* an ABI-compatible version may increase this value */
>> > >> + _LIST_MAX = _LIST_END + 42 /* room for ABI-compatible additions */
>> > >> };
>> > >>
>> > >> Then *_LIST_END values could be ignored by libabigail with such a change.
>>
>> In order to avoid ABI check complaining, the best is to completely
>> remove LIST_END in DPDK 20.11.
>>
>>
>> > >> If such a patch is not done by tomorrow, I will have to revert
>> > >> Chacha-Poly commits before 20.02-rc2, because
>> > >>
>> > >> 1/ LIST_END, without any comment, means "size of range"
>> > >> 2/ we do not blame users for undocumented ABI changes
>> > >> 3/ we respect the ABI compatibility contract
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH v7] eal: add manual probing option
2020-02-04 12:43 0% ` Gaetan Rivet
@ 2020-02-04 15:06 0% ` Thomas Monjalon
2020-02-04 16:02 0% ` Gaetan Rivet
0 siblings, 1 reply; 200+ results
From: Thomas Monjalon @ 2020-02-04 15:06 UTC (permalink / raw)
To: Pavan Nikhilesh Bhagavatula, Gaetan Rivet, Vamsi Krishna Attunuru
Cc: David Marchand, dev, Jerin Jacob Kollanukkaran
04/02/2020 13:43, Gaetan Rivet:
> On 04/02/2020 12:07, Thomas Monjalon wrote:
> > 04/02/2020 11:03, Gaetan Rivet:
> >> On 03/02/2020 23:21, Thomas Monjalon wrote:
> >>> 03/02/2020 06:16, Pavan Nikhilesh Bhagavatula:
> >>>> @David Marchand @thomas@monjalon.net
> >>>>
> >>>> Ping?
> >>>>
> >>>> Are there any more changes required for this patch? It's been in queue since last October.
> >>>
> >>> Sorry we have not decided whether it is a good idea or not.
> >>>
> >>> All changes related to probing are very sensitive,
> >>> and we know a big refactoring would be better than stacking
> >>> more and more options and corner cases.
> >>>
> >>> As we are busy with ABI stability stuff, we did not allocate
> >>> enough time to properly think about this feature.
> >>> Please accept our apologies, and let's consider it as
> >>> a high priority for 20.05 cycle.
> >>>
> >>
> >> Hello Thomas,
> >>
> >> This is unfortunate. I pushed Pavan to accept an alternative implementation of this functionality that was less obtrusive, to make the integration smoother. I took care to alleviate those risks from the common path.
> >>
> >> The big refactoring is needed yes, but considering the current path I'm not seeing it happen in 20.05. If that means taking this patch as-is in 20.05 for Marvell users, I'm not sure much is gained from waiting 3 months, except minimal risk avoidance.
> >
> >
> > Yes, life is full of bad decisions and consequences.
>
>
> Ah, yes, but I stand by my initial opinion, the first implementation [1] was riskier and less useful.
>
> >
> > I still think there is a risk in adding new user expectations,
> > and maintaining some code to workaround unknown issues.
> >
> > The real question here is to know why this patch?
> > Is it to workaround a broken driver?
> > Or to workaround a broken design in EAL and bus drivers?
>
> Two birds - one stone here: OVS needed a way to disable automatic probing cleanly (current workaround seen in multiple deployment is to add a dummy whitelisted device, which will be ignored by the PCI bus --> it sets the bus in whitelist mode but avoid probing anything), and as a bonus this option allows using devices that depends on other devices being probed already (LAG, representors, failsafe, etc).
>
> I'm not sure having a dependent-probe by default is good, and that would be a big change.
>
> If we are doing the genesis of this patch, the initial motivation should be asked for more details from Marvell people and David for the OVS side.
>
> [1]: First proposal:
> http://mails.dpdk.org/archives/dev/2019-September/144166.html
> My arguments:
> http://mails.dpdk.org/archives/dev/2019-September/144564.html
OK so there are two needs:
1/ Better control whitelist/blacklist mode.
We already know that a rework is needed here.
Unfortunately neither you or me had time to work on it,
and others who were interested disappeared.
2/ Associate ports with equivalent properties in applications.
This must be done in applications.
Tweaking the probe order is a hack.
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
2020-02-04 12:44 4% ` Trahe, Fiona
@ 2020-02-04 15:52 8% ` Trahe, Fiona
2020-02-04 15:59 7% ` Thomas Monjalon
0 siblings, 1 reply; 200+ results
From: Trahe, Fiona @ 2020-02-04 15:52 UTC (permalink / raw)
To: Thomas Monjalon, David Marchand, nhorman, bluca, ktraynor,
Ray Kinsella, dev, Akhil Goyal, Yigit, Ferruh
Cc: Ananyev, Konstantin, dev, Anoob Joseph, Kusztal, ArkadiuszX,
Richardson, Bruce, Mcnamara, John, dodji, Andrew Rybchenko,
aconole, Trahe, Fiona
> We are working on a patch, when it is ready we will send it.
> If it's not ready by end of your today, of course, go ahead with (b) and
> we will work towards 20.05.
We will not be sending a patch today.
The patch we're working on will provide two versions of rte_cryptodev_info_get(),
both call the same PMD function from the dev_ops info_get fn ptr.
The default version operates s as normal, the 19.11 version searches
through the list returned by the PMD, looking for sym.aead.algo = ChaChaPoly, it needs to strip it from the list.
As PMDs just pass a ptr to their capabilities list ( it isn't a linked list, but an array
with an end marker = RTE_CRYPTODEV_END_OF_CAPABILITIES_LIST) if the API
layer detects Chacha it must allocate some space and store a local copy of the
trimmed list. This must be stored only once per device.
This versioning will apply to any PMD which wants to take advantage of the new API between now and 20.11.
Note, I expect the ABI checker tools will still complain of ABI breakage as the LIST_END value will still change.
We are also reviewing all other cryptodev APIs in case there is any other API which needs versioning.
Anyone see any problem with this approach?
^ permalink raw reply [relevance 8%]
* Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
2020-02-04 15:52 8% ` Trahe, Fiona
@ 2020-02-04 15:59 7% ` Thomas Monjalon
2020-02-04 17:46 4% ` Trahe, Fiona
0 siblings, 1 reply; 200+ results
From: Thomas Monjalon @ 2020-02-04 15:59 UTC (permalink / raw)
To: Trahe, Fiona
Cc: David Marchand, nhorman, bluca, ktraynor, Ray Kinsella, dev,
Akhil Goyal, Yigit, Ferruh, Ananyev, Konstantin, dev,
Anoob Joseph, Kusztal, ArkadiuszX, Richardson, Bruce, Mcnamara,
John, dodji, Andrew Rybchenko, aconole
04/02/2020 16:52, Trahe, Fiona:
>
> > We are working on a patch, when it is ready we will send it.
> > If it's not ready by end of your today, of course, go ahead with (b) and
> > we will work towards 20.05.
>
> We will not be sending a patch today.
> The patch we're working on will provide two versions of rte_cryptodev_info_get(),
> both call the same PMD function from the dev_ops info_get fn ptr.
> The default version operates s as normal, the 19.11 version searches
> through the list returned by the PMD, looking for sym.aead.algo = ChaChaPoly, it needs to strip it from the list.
> As PMDs just pass a ptr to their capabilities list ( it isn't a linked list, but an array
> with an end marker = RTE_CRYPTODEV_END_OF_CAPABILITIES_LIST) if the API
> layer detects Chacha it must allocate some space and store a local copy of the
> trimmed list. This must be stored only once per device.
I don't understand what you have to store.
Can't you just set the algo to 0 if it is ChaCha?
> This versioning will apply to any PMD which wants to take advantage of the new API between now and 20.11.
>
> Note, I expect the ABI checker tools will still complain of ABI breakage as the LIST_END value will still change.
Right, you need to update the ignore list for the tool.
> We are also reviewing all other cryptodev APIs in case there is any other API which needs versioning.
>
> Anyone see any problem with this approach?
The other issue is with all other functions accepting this enum as input.
We should continue returning an error if getting Chacha as input with
19.11 version of these functions.
But I would tend to consider this small ABI breakage can be ignored
as it is in the error path.
^ permalink raw reply [relevance 7%]
* Re: [dpdk-dev] [PATCH v7] eal: add manual probing option
2020-02-04 15:06 0% ` Thomas Monjalon
@ 2020-02-04 16:02 0% ` Gaetan Rivet
2020-02-10 14:51 0% ` Jerin Jacob
0 siblings, 1 reply; 200+ results
From: Gaetan Rivet @ 2020-02-04 16:02 UTC (permalink / raw)
To: Thomas Monjalon, Pavan Nikhilesh Bhagavatula, Vamsi Krishna Attunuru
Cc: David Marchand, dev, Jerin Jacob Kollanukkaran
On 04/02/2020 16:06, Thomas Monjalon wrote:
> 04/02/2020 13:43, Gaetan Rivet:
>> On 04/02/2020 12:07, Thomas Monjalon wrote:
>>> 04/02/2020 11:03, Gaetan Rivet:
>>>> On 03/02/2020 23:21, Thomas Monjalon wrote:
>>>>> 03/02/2020 06:16, Pavan Nikhilesh Bhagavatula:
>>>>>> @David Marchand @thomas@monjalon.net
>>>>>>
>>>>>> Ping?
>>>>>>
>>>>>> Are there any more changes required for this patch? It's been in queue since last October.
>>>>>
>>>>> Sorry we have not decided whether it is a good idea or not.
>>>>>
>>>>> All changes related to probing are very sensitive,
>>>>> and we know a big refactoring would be better than stacking
>>>>> more and more options and corner cases.
>>>>>
>>>>> As we are busy with ABI stability stuff, we did not allocate
>>>>> enough time to properly think about this feature.
>>>>> Please accept our apologies, and let's consider it as
>>>>> a high priority for 20.05 cycle.
>>>>>
>>>>
>>>> Hello Thomas,
>>>>
>>>> This is unfortunate. I pushed Pavan to accept an alternative implementation of this functionality that was less obtrusive, to make the integration smoother. I took care to alleviate those risks from the common path.
>>>>
>>>> The big refactoring is needed yes, but considering the current path I'm not seeing it happen in 20.05. If that means taking this patch as-is in 20.05 for Marvell users, I'm not sure much is gained from waiting 3 months, except minimal risk avoidance.
>>>
>>>
>>> Yes, life is full of bad decisions and consequences.
>>
>>
>> Ah, yes, but I stand by my initial opinion, the first implementation [1] was riskier and less useful.
>>
>>>
>>> I still think there is a risk in adding new user expectations,
>>> and maintaining some code to workaround unknown issues.
>>>
>>> The real question here is to know why this patch?
>>> Is it to workaround a broken driver?
>>> Or to workaround a broken design in EAL and bus drivers?
>>
>> Two birds - one stone here: OVS needed a way to disable automatic probing cleanly (current workaround seen in multiple deployment is to add a dummy whitelisted device, which will be ignored by the PCI bus --> it sets the bus in whitelist mode but avoid probing anything), and as a bonus this option allows using devices that depends on other devices being probed already (LAG, representors, failsafe, etc).
>>
>> I'm not sure having a dependent-probe by default is good, and that would be a big change.
>>
>> If we are doing the genesis of this patch, the initial motivation should be asked for more details from Marvell people and David for the OVS side.
>>
>> [1]: First proposal:
>> http://mails.dpdk.org/archives/dev/2019-September/144166.html
>> My arguments:
>> http://mails.dpdk.org/archives/dev/2019-September/144564.html
>
>
> OK so there are two needs:
>
> 1/ Better control whitelist/blacklist mode.
> We already know that a rework is needed here.
> Unfortunately neither you or me had time to work on it,
> and others who were interested disappeared.
>
> 2/ Associate ports with equivalent properties in applications.
> This must be done in applications.
> Tweaking the probe order is a hack.
>
>
An application that want to tightly control the port init order, currently (by doing exactly like me here, hotpluging one by one the ports), would still need the bigger hack that consist in inserting a whitelist PCI devargs with a dummy address, depending on a undocumented PCI bus feature consisting in ignoring matching errors but keeping probing policy from failed devargs processing.
Instead, with this patch this app can do
rte_eal_manual_probe_set(1);
rte_eal_init();
to have the same behavior and be able to hotplug ports as it sees fit.
You are worried about creating user expectations about this behavior (being forced to replicate in some way the functionality during the rewrite, as I understand it?), but then you are currently forcing users to expect this workaround to exist in the PCI bus, blocking devs from touching it as it will thus break current app configurations. I've seen systemd unit file using this -w dummy flag, as well as the programmatic equivalent. Which is better, to have to rework it cutting short these configs, or to propose beforehand a deprecation path?.
This rework won't happen in 20.05, nor in the medium term unless you decide to drive this change. This workaround serves three needs (PCI normalization, port congruence and port dependency) in a low-risk implementation.
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
2020-02-04 15:59 7% ` Thomas Monjalon
@ 2020-02-04 17:46 4% ` Trahe, Fiona
0 siblings, 0 replies; 200+ results
From: Trahe, Fiona @ 2020-02-04 17:46 UTC (permalink / raw)
To: Thomas Monjalon
Cc: David Marchand, nhorman, bluca, ktraynor, Ray Kinsella, dev,
Akhil Goyal, Yigit, Ferruh, Ananyev, Konstantin, dev,
Anoob Joseph, Kusztal, ArkadiuszX, Richardson, Bruce, Mcnamara,
John, dodji, Andrew Rybchenko, aconole, Trahe, Fiona
> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: Tuesday, February 4, 2020 4:00 PM
> To: Trahe, Fiona <fiona.trahe@intel.com>
> Cc: David Marchand <david.marchand@redhat.com>; nhorman@tuxdriver.com; bluca@debian.org;
> ktraynor@redhat.com; Ray Kinsella <mdr@ashroe.eu>; dev@dpdk.org; Akhil Goyal
> <akhil.goyal@nxp.com>; Yigit, Ferruh <ferruh.yigit@intel.com>; Ananyev, Konstantin
> <konstantin.ananyev@intel.com>; dev@dpdk.org; Anoob Joseph <anoobj@marvell.com>; Kusztal,
> ArkadiuszX <arkadiuszx.kusztal@intel.com>; Richardson, Bruce <bruce.richardson@intel.com>;
> Mcnamara, John <john.mcnamara@intel.com>; dodji@seketeli.net; Andrew Rybchenko
> <arybchenko@solarflare.com>; aconole@redhat.com
> Subject: Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
>
> 04/02/2020 16:52, Trahe, Fiona:
> >
> > > We are working on a patch, when it is ready we will send it.
> > > If it's not ready by end of your today, of course, go ahead with (b) and
> > > we will work towards 20.05.
> >
> > We will not be sending a patch today.
> > The patch we're working on will provide two versions of rte_cryptodev_info_get(),
> > both call the same PMD function from the dev_ops info_get fn ptr.
> > The default version operates s as normal, the 19.11 version searches
> > through the list returned by the PMD, looking for sym.aead.algo = ChaChaPoly, it needs to strip it from
> the list.
> > As PMDs just pass a ptr to their capabilities list ( it isn't a linked list, but an array
> > with an end marker = RTE_CRYPTODEV_END_OF_CAPABILITIES_LIST) if the API
> > layer detects Chacha it must allocate some space and store a local copy of the
> > trimmed list. This must be stored only once per device.
>
> I don't understand what you have to store.
> Can't you just set the algo to 0 if it is ChaCha?
[Fiona] it returns a pointer to data in the PMD domain, which the API couldn't and shouldn't overwrite, e.g.
static const struct rte_cryptodev_capabilities qat_gen3_sym_capabilities[]
>
> > This versioning will apply to any PMD which wants to take advantage of the new API between now and
> 20.11.
> >
> > Note, I expect the ABI checker tools will still complain of ABI breakage as the LIST_END value will still
> change.
>
> Right, you need to update the ignore list for the tool.
>
> > We are also reviewing all other cryptodev APIs in case there is any other API which needs versioning.
> >
> > Anyone see any problem with this approach?
>
> The other issue is with all other functions accepting this enum as input.
> We should continue returning an error if getting Chacha as input with
> 19.11 version of these functions.
> But I would tend to consider this small ABI breakage can be ignored
> as it is in the error path.
[Fiona] The QAT PMD tests for and handles this error. I expect other PMDs do too.
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
2020-02-04 14:44 4% ` Aaron Conole
@ 2020-02-04 19:49 7% ` Neil Horman
0 siblings, 0 replies; 200+ results
From: Neil Horman @ 2020-02-04 19:49 UTC (permalink / raw)
To: Aaron Conole
Cc: Thomas Monjalon, David Marchand, bluca, ktraynor, Ray Kinsella,
dev, Akhil Goyal, Trahe, Fiona, Ferruh Yigit, Ananyev,
Konstantin, Anoob Joseph, Kusztal, ArkadiuszX, Richardson, Bruce,
Mcnamara, John, dodji, Andrew Rybchenko
On Tue, Feb 04, 2020 at 09:44:53AM -0500, Aaron Conole wrote:
> Thomas Monjalon <thomas@monjalon.net> writes:
>
> > RED FLAG
> >
> > I don't see a lot of reactions, so I summarize the issue.
> > We need action TODAY!
> >
> > API makes think that rte_cryptodev_info_get() cannot return
> > a value >= 3 (RTE_CRYPTO_AEAD_LIST_END in 19.11).
> > Current 20.02 returns 3 (RTE_CRYPTO_AEAD_CHACHA20_POLY1305).
> > The ABI compatibility contract is broken currently.
> >
> > There are 3 possible outcomes:
> >
> > a) Change the API comments and backport to 19.11.1
> > The details are discussed between Ferruh and me.
> > Either put responsibility on API user (with explicit comment),
> > or expose ABI extension allowance with a new API max value.
> > In both cases, this is breaking the implicit contract of 19.11.0.
> > This option can be chosen only if release and ABI maintainers
> > vote for it.
> >
> > b) Revert Chacha-Poly from 20.02-rc2.
> >
> > c) Add versioned function rte_cryptodev_info_get_v1911()
> > which calls rte_cryptodev_info_get() and filters out
> > RTE_CRYPTO_AEAD_CHACHA20_POLY1305 capability.
> > So Chacha-Poly capability would be seen and usable only
> > if compiling with DPDK 20.02.
> >
> > I hope it is clear what are the actions for everybody:
> > - ABI and release maintainers must say yes or no to the proposal (a)
> > - In the meantime, crypto team must send a patch for the proposal (c)
> > - If (a) and (c) are not possible at the end of today, I will take (b)
> >
> > Note: do not say it is too short for (c), as it was possible to work
> > on such solution since the issue was exposed on last Wednesday.
>
> While I'm not a maintainer, if I my opinion counts for anything, I'd
> choose option c or b. Absolutely NACK to a.
>
Agreed, options c and b are reasonable, a isn't. ABI commitments are ours, not
users.
Neil
> >
> > 03/02/2020 22:07, Thomas Monjalon:
> >> 03/02/2020 19:55, Ray Kinsella:
> >> > On 03/02/2020 17:34, Thomas Monjalon wrote:
> >> > > 03/02/2020 18:09, Thomas Monjalon:
> >> > >> 03/02/2020 10:30, Ferruh Yigit:
> >> > >>> On 2/2/2020 2:41 PM, Ananyev, Konstantin wrote:
> >> > >>>> 02/02/2020 14:05, Thomas Monjalon:
> >> > >>>>> 31/01/2020 15:16, Trahe, Fiona:
> >> > >>>>>> On 1/30/2020 8:18 PM, Thomas Monjalon wrote:
> >> > >>>>>>> If library give higher value than expected by the application,
> >> > >>>>>>> if the application uses this value as array index,
> >> > >>>>>>> there can be an access out of bounds.
> >> > >>>>>>
> >> > >>>>>> [Fiona] All asymmetric APIs are experimental so above shouldn't be a problem.
> >> > >>>>>> But for the same issue with sym crypto below, I believe Ferruh's explanation makes
> >> > >>>>>> sense and I don't see how there can be an API breakage.
> >> > >>>>>> So if an application hasn't compiled against the new lib it
> >> > >>>>>> will be still using the old value
> >> > >>>>>> which will be within bounds. If it's picking up the higher
> >> > >>>>>> new value from the lib it must
> >> > >>>>>> have been compiled against the lib so shouldn't have problems.
> >> > >>>>>
> >> > >>>>> You say there is no ABI issue because the application will be re-compiled
> >> > >>>>> for the updated library. Indeed, compilation fixes compatibility issues.
> >> > >>>>> But this is not relevant for ABI compatibility.
> >> > >>>>> ABI compatibility means we can upgrade the library without recompiling
> >> > >>>>> the application and it must work.
> >> > >>>>> You think it is a false positive because you assume the application
> >> > >>>>> "picks" the new value. I think you miss the case where the new value
> >> > >>>>> is returned by a function in the upgraded library.
> >> > >>>>>
> >> > >>>>>> There are also no structs on the API which contain arrays using this
> >> > >>>>>> for sizing, so I don't see an opportunity for an appl to have a
> >> > >>>>>> mismatch in memory addresses.
> >> > >>>>>
> >> > >>>>> Let me demonstrate where the API may "use" the new value
> >> > >>>>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 and how it impacts the application.
> >> > >>>>>
> >> > >>>>> Once upon a time a DPDK application counting the number of devices
> >> > >>>>> supporting each AEAD algo (in order to find the best supported algo).
> >> > >>>>> It is done in an array indexed by algo id:
> >> > >>>>> int aead_dev_count[RTE_CRYPTO_AEAD_LIST_END];
> >> > >>>>> The application is compiled with DPDK 19.11,
> >> > >>>>> where RTE_CRYPTO_AEAD_LIST_END = 3.
> >> > >>>>> So the size of the application array aead_dev_count is 3.
> >> > >>>>> This binary is run with DPDK 20.02,
> >> > >>>>> where RTE_CRYPTO_AEAD_CHACHA20_POLY1305 = 3.
> >> > >>>>> When calling rte_cryptodev_info_get() on a device QAT_GEN3,
> >> > >>>>> rte_cryptodev_info.capabilities.sym.aead.algo is set to
> >> > >>>>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 (= 3).
> >> > >>>>> The application uses this value:
> >> > >>>>> ++ aead_dev_count[info.capabilities.sym.aead.algo];
> >> > >>>>> The application is crashing because of out of bound access.
> >> > >>>>
> >> > >>>> I'd say this is an example of bad written app.
> >> > >>>> It probably should check that returned by library value doesn't
> >> > >>>> exceed its internal array size.
> >> > >>>
> >> > >>> +1
> >> > >>>
> >> > >>> Application should ignore values >= MAX.
> >> > >>
> >> > >> Of course, blaming the API user is a lot easier than looking at the API.
> >> > >> Here the API has RTE_CRYPTO_AEAD_LIST_END which can be understood
> >> > >> as the max value for the application.
> >> > >> Value ranges are part of the ABI compatibility contract.
> >> > >> It seems you expect the application developer to be aware that
> >> > >> DPDK could return a higher value, so the application should
> >> > >> check every enum values after calling an API. CRAZY.
> >> > >>
> >> > >> When we decide to announce an ABI compatibility and do some marketing,
> >> > >> everyone is OK. But when we need to really make our ABI compatible,
> >> > >> I see little or no effort. DISAPPOINTING.
> >> > >>
> >> > >>> Do you suggest we don't extend any enum or define between ABI breakage releases
> >> > >>> to be sure bad written applications not affected?
> >> > >>
> >> > >> I suggest we must consider not breaking any assumption made on the API.
> >> > >> Here we are breaking the enum range because nothing mentions _LIST_END
> >> > >> is not really the absolute end of the enum.
> >> > >> The solution is to make the change below in 20.02 + backport in 19.11.1:
> >> > >
> >> > > Thinking twice, merging such change before 20.11 is breaking the
> >> > > ABI assumption based on the API 19.11.0.
> >> > > I ask the release maintainers (Luca, Kevin, David and me) and
> >> > > the ABI maintainers (Neil and Ray) to vote for a or b solution:
> >> > > a) add comment and LIST_MAX as below in 20.02 + 19.11.1
> >> >
> >> > That would still be an ABI breakage though right.
> >> >
> >> > > b) wait 20.11 and revert Chacha-Poly from 20.02
> >> >
> >> > Thanks for analysis above Fiona, Ferruh and all.
> >> >
> >> > That is a nasty one alright - there is no "good" answer here.
> >> > I agree with Ferruh's sentiments overall, we should rethink this API for 20.11.
> >> > Could do without an enumeration?
> >> >
> >> > There a c) though right.
> >> > We could work around the issue by api versioning rte_cryptodev_info_get() and friends.
> >> > So they only support/acknowledge the existence of Chacha-Poly for
> >> > applications build against > 20.02.
> >>
> >> I agree there is a c) as I proposed in another email:
> >> http://mails.dpdk.org/archives/dev/2020-February/156919.html
> >> "
> >> In this case, the proper solution is to implement
> >> rte_cryptodev_info_get_v1911() so it filters out
> >> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 capability.
> >> With this solution, an application compiled with DPDK 19.11 will keep
> >> seeing the same range as before, while a 20.02 application could
> >> see and use ChachaPoly.
> >> "
> >>
> >> > It would be painful I know.
> >>
> >> Not so painful in my opinion.
> >> Just need to call rte_cryptodev_info_get() from
> >> rte_cryptodev_info_get_v1911() and filter the value
> >> in the 19.11 range: [0..AES_GCM].
> >>
> >> > It would also mean that Chacha-Poly would only be available to
> >> > those building against >= 20.02.
> >>
> >> Yes exactly.
> >>
> >> The addition of comments and LIST_MAX like below are still valid
> >> to avoid versioning after 20.11.
> >>
> >> > >> - _LIST_END
> >> > >> + _LIST_END, /* an ABI-compatible version may increase this value */
> >> > >> + _LIST_MAX = _LIST_END + 42 /* room for ABI-compatible additions */
> >> > >> };
> >> > >>
> >> > >> Then *_LIST_END values could be ignored by libabigail with such a change.
> >>
> >> In order to avoid ABI check complaining, the best is to completely
> >> remove LIST_END in DPDK 20.11.
> >>
> >>
> >> > >> If such a patch is not done by tomorrow, I will have to revert
> >> > >> Chacha-Poly commits before 20.02-rc2, because
> >> > >>
> >> > >> 1/ LIST_END, without any comment, means "size of range"
> >> > >> 2/ we do not blame users for undocumented ABI changes
> >> > >> 3/ we respect the ABI compatibility contract
>
>
^ permalink raw reply [relevance 7%]
* Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
2020-02-04 10:16 9% ` Akhil Goyal
2020-02-04 10:28 8% ` Thomas Monjalon
@ 2020-02-04 21:59 8% ` Neil Horman
1 sibling, 0 replies; 200+ results
From: Neil Horman @ 2020-02-04 21:59 UTC (permalink / raw)
To: Akhil Goyal
Cc: Ferruh Yigit, Thomas Monjalon, Ananyev, Konstantin, Trahe, Fiona,
David Marchand, Anoob Joseph, Kusztal, ArkadiuszX, dev,
Richardson, Bruce, Mcnamara, John, dodji, Andrew Rybchenko,
aconole, bluca, ktraynor
On Tue, Feb 04, 2020 at 10:16:56AM +0000, Akhil Goyal wrote:
> Hi,
> > On 2/3/2020 5:09 PM, Thomas Monjalon wrote:
> > > 03/02/2020 10:30, Ferruh Yigit:
> > >> On 2/2/2020 2:41 PM, Ananyev, Konstantin wrote:
> > >>> 02/02/2020 14:05, Thomas Monjalon:
> > >>>> 31/01/2020 15:16, Trahe, Fiona:
> > >>>>> On 1/30/2020 8:18 PM, Thomas Monjalon wrote:
> > >>>>>> 30/01/2020 17:09, Ferruh Yigit:
> > >>>>>>> On 1/29/2020 8:13 PM, Akhil Goyal wrote:
> > >>>>>>>>
> > >>>>>>>> I believe these enums will be used only in case of ASYM case which is
> > experimental.
> > >>>>>>>
> > >>>>>>> Independent from being experiment and not, this shouldn't be a
> > problem, I think
> > >>>>>>> this is a false positive.
> > >>>>>>>
> > >>>>>>> The ABI break can happen when a struct has been shared between the
> > application
> > >>>>>>> and the library (DPDK) and the layout of that memory know differently
> > by
> > >>>>>>> application and the library.
> > >>>>>>>
> > >>>>>>> Here in all cases, there is no layout/size change.
> > >>>>>>>
> > >>>>>>> As to the value changes of the enums, since application compiled with
> > old DPDK,
> > >>>>>>> it will know only up to '6', 7 and more means invalid to the application.
> > So it
> > >>>>>>> won't send these values also it should ignore these values from library.
> > Only
> > >>>>>>> consequence is old application won't able to use new features those
> > new enums
> > >>>>>>> provide but that is expected/normal.
> > >>>>>>
> > >>>>>> If library give higher value than expected by the application,
> > >>>>>> if the application uses this value as array index,
> > >>>>>> there can be an access out of bounds.
> > >>>>>
> > >>>>> [Fiona] All asymmetric APIs are experimental so above shouldn't be a
> > problem.
> > >>>>> But for the same issue with sym crypto below, I believe Ferruh's
> > explanation makes
> > >>>>> sense and I don't see how there can be an API breakage.
> > >>>>> So if an application hasn't compiled against the new lib it will be still using
> > the old value
> > >>>>> which will be within bounds. If it's picking up the higher new value from
> > the lib it must
> > >>>>> have been compiled against the lib so shouldn't have problems.
> > >>>>
> > >>>> You say there is no ABI issue because the application will be re-compiled
> > >>>> for the updated library. Indeed, compilation fixes compatibility issues.
> > >>>> But this is not relevant for ABI compatibility.
> > >>>> ABI compatibility means we can upgrade the library without recompiling
> > >>>> the application and it must work.
> > >>>> You think it is a false positive because you assume the application
> > >>>> "picks" the new value. I think you miss the case where the new value
> > >>>> is returned by a function in the upgraded library.
> > >>>>
> > >>>>> There are also no structs on the API which contain arrays using this
> > >>>>> for sizing, so I don't see an opportunity for an appl to have a
> > >>>>> mismatch in memory addresses.
> > >>>>
> > >>>> Let me demonstrate where the API may "use" the new value
> > >>>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 and how it impacts the
> > application.
> > >>>>
> > >>>> Once upon a time a DPDK application counting the number of devices
> > >>>> supporting each AEAD algo (in order to find the best supported algo).
> > >>>> It is done in an array indexed by algo id:
> > >>>> int aead_dev_count[RTE_CRYPTO_AEAD_LIST_END];
> > >>>> The application is compiled with DPDK 19.11,
> > >>>> where RTE_CRYPTO_AEAD_LIST_END = 3.
> > >>>> So the size of the application array aead_dev_count is 3.
> > >>>> This binary is run with DPDK 20.02,
> > >>>> where RTE_CRYPTO_AEAD_CHACHA20_POLY1305 = 3.
> > >>>> When calling rte_cryptodev_info_get() on a device QAT_GEN3,
> > >>>> rte_cryptodev_info.capabilities.sym.aead.algo is set to
> > >>>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 (= 3).
> > >>>> The application uses this value:
> > >>>> ++ aead_dev_count[info.capabilities.sym.aead.algo];
> > >>>> The application is crashing because of out of bound access.
> > >>>
> > >>> I'd say this is an example of bad written app.
> > >>> It probably should check that returned by library value doesn't
> > >>> exceed its internal array size.
> > >>
> > >> +1
> > >>
> > >> Application should ignore values >= MAX.
> > >
> > > Of course, blaming the API user is a lot easier than looking at the API.
> > > Here the API has RTE_CRYPTO_AEAD_LIST_END which can be understood
> > > as the max value for the application.
> > > Value ranges are part of the ABI compatibility contract.
> > > It seems you expect the application developer to be aware that
> > > DPDK could return a higher value, so the application should
> > > check every enum values after calling an API. CRAZY.
> > >
> > > When we decide to announce an ABI compatibility and do some marketing,
> > > everyone is OK. But when we need to really make our ABI compatible,
> > > I see little or no effort. DISAPPOINTING.
> >
> > This is not to blame the user or to do less work, this is more sane approach
> > that library provides the _END/_MAX value and application uses it as valid range
> > check.
> >
> > >
> > >> Do you suggest we don't extend any enum or define between ABI breakage
> > releases
> > >> to be sure bad written applications not affected?
> > >
> > > I suggest we must consider not breaking any assumption made on the API.
> > > Here we are breaking the enum range because nothing mentions _LIST_END
> > > is not really the absolute end of the enum.
> > > The solution is to make the change below in 20.02 + backport in 19.11.1:
> > >
> > > - _LIST_END
> > > + _LIST_END, /* an ABI-compatible version may increase this value */
> > > + _LIST_MAX = _LIST_END + 42 /* room for ABI-compatible additions */
> > > };
> > >
> >
> > What is the point of "_LIST_MAX" here?
> >
> > Application should know the "_LIST_END" of when it has been compiled for the
> > valid range check. Next time it is compiled "_LIST_END" may be different value
> > but same logic applies.
> >
> > When "_LIST_END" is missing, application can't protect itself, in that case
> > library should send only the values application knows when it is compiled, this
> > means either we can't extend our enum/defines until next ABI breakage, or we
> > need to do ABI versioning to the functions that returns an enum each time enum
> > value extended.
> >
> > I believe it is saner to provide _END/_MAX values to the application to use. And
> > if required comment them to clarify the expected usage.
> >
> > But in above suggestion application can't use or rely on "_LIST_MAX", it doesn't
> > mean anything to application.
> >
>
> Can we have something like
> enum rte_crypto_aead_algorithm {
> RTE_CRYPTO_AEAD_AES_CCM = 1,
> /**< AES algorithm in CCM mode. */
> RTE_CRYPTO_AEAD_AES_GCM,
> /**< AES algorithm in GCM mode. */
> RTE_CRYPTO_AEAD_LIST_END,
> /**< List end for 19.11 ABI compatibility */
> RTE_CRYPTO_AEAD_CHACHA20_POLY1305,
> /**< Chacha20 cipher with poly1305 authenticator */
> RTE_CRYPTO_AEAD_LIST_END_2011
> /**< List end for 20.11 ABI compatibility */
> };
>
> And in 20.11 release we alter the RTE_CRYPTO_AEAD_LIST_END to the end and remove RTE_CRYPTO_AEAD_LIST_END_2011
>
> I believe it will be ok for any application which need to use the chacha poly assume that this algo is
> Experimental and will move to formal list in 20.11. This can be documented in the documentation.
> I believe there is no way to add a new enum as experimental so far. This way we can formalize this
> requirement as well.
>
> I believe this way effect of ABI breakage will be nullified.
>
Thats not really helpful though, in that libabigail will then complain that
you've aliased an old ennumeration name in the ABI to a new name.
A better solution would be do one of the following:
a) add an API call - something like rte_crypto_get_max_alg(), which returns at
run time the maximum number of algorithms available, so that the application is
forced to learn that number at run time, rather than at compile time
b) Modify the API such that you pass in an algorithm name rather than an index
value defined by an ennumeration. You may also add an API call that dumps all
the available strings back to the user.
a) would be nice, but it would still require an ABI change, which is less than
optimal
b) is nice because it gives you flexibility in how you search for algs - if you
want to add a new alg, you just update your internal tables with a new name
string, and applications get get its information by querying based on that
string, rather than an index.
b is also nice because it can just superscede the existing implementation (i.e.
the current implementation remains unchanged, but only supports the current,
existing algs), if you want to get the new algs, you use the new API calls
(which can also find the already existing algs), and at some point in the future
we can just deprecate the old API.
Neil
>
> -Akhil
>
> > > Then *_LIST_END values could be ignored by libabigail with such a change.
> > >
> > > If such a patch is not done by tomorrow, I will have to revert
> > > Chacha-Poly commits before 20.02-rc2, because
> > >
> > > 1/ LIST_END, without any comment, means "size of range"
> > > 2/ we do not blame users for undocumented ABI changes
> > > 3/ we respect the ABI compatibility contract
> > >
> > >
>
^ permalink raw reply [relevance 8%]
* Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
2020-02-04 10:32 4% ` Akhil Goyal
2020-02-04 11:35 4% ` Bruce Richardson
@ 2020-02-04 22:10 7% ` Neil Horman
2020-02-05 6:16 7% ` [dpdk-dev] [EXT] " Anoob Joseph
1 sibling, 1 reply; 200+ results
From: Neil Horman @ 2020-02-04 22:10 UTC (permalink / raw)
To: Akhil Goyal
Cc: Thomas Monjalon, Ferruh Yigit, Ananyev, Konstantin, Trahe, Fiona,
dev, David Marchand, Anoob Joseph, Kusztal, ArkadiuszX,
Richardson, Bruce, Mcnamara, John, dodji, Andrew Rybchenko,
aconole, bluca, ktraynor
On Tue, Feb 04, 2020 at 10:32:01AM +0000, Akhil Goyal wrote:
>
> >
> > 04/02/2020 11:16, Akhil Goyal:
> > > Hi,
> > > > On 2/3/2020 5:09 PM, Thomas Monjalon wrote:
> > > > > 03/02/2020 10:30, Ferruh Yigit:
> > > > >> On 2/2/2020 2:41 PM, Ananyev, Konstantin wrote:
> > > > >>> 02/02/2020 14:05, Thomas Monjalon:
> > > > >>>> 31/01/2020 15:16, Trahe, Fiona:
> > > > >>>>> On 1/30/2020 8:18 PM, Thomas Monjalon wrote:
> > > > >>>>>> 30/01/2020 17:09, Ferruh Yigit:
> > > > >>>>>>> On 1/29/2020 8:13 PM, Akhil Goyal wrote:
> > > > >>>>>>>>
> > > > >>>>>>>> I believe these enums will be used only in case of ASYM case which
> > is
> > > > experimental.
> > > > >>>>>>>
> > > > >>>>>>> Independent from being experiment and not, this shouldn't be a
> > > > problem, I think
> > > > >>>>>>> this is a false positive.
> > > > >>>>>>>
> > > > >>>>>>> The ABI break can happen when a struct has been shared between
> > the
> > > > application
> > > > >>>>>>> and the library (DPDK) and the layout of that memory know
> > differently
> > > > by
> > > > >>>>>>> application and the library.
> > > > >>>>>>>
> > > > >>>>>>> Here in all cases, there is no layout/size change.
> > > > >>>>>>>
> > > > >>>>>>> As to the value changes of the enums, since application compiled
> > with
> > > > old DPDK,
> > > > >>>>>>> it will know only up to '6', 7 and more means invalid to the
> > application.
> > > > So it
> > > > >>>>>>> won't send these values also it should ignore these values from
> > library.
> > > > Only
> > > > >>>>>>> consequence is old application won't able to use new features
> > those
> > > > new enums
> > > > >>>>>>> provide but that is expected/normal.
> > > > >>>>>>
> > > > >>>>>> If library give higher value than expected by the application,
> > > > >>>>>> if the application uses this value as array index,
> > > > >>>>>> there can be an access out of bounds.
> > > > >>>>>
> > > > >>>>> [Fiona] All asymmetric APIs are experimental so above shouldn't be a
> > > > problem.
> > > > >>>>> But for the same issue with sym crypto below, I believe Ferruh's
> > > > explanation makes
> > > > >>>>> sense and I don't see how there can be an API breakage.
> > > > >>>>> So if an application hasn't compiled against the new lib it will be still
> > using
> > > > the old value
> > > > >>>>> which will be within bounds. If it's picking up the higher new value
> > from
> > > > the lib it must
> > > > >>>>> have been compiled against the lib so shouldn't have problems.
> > > > >>>>
> > > > >>>> You say there is no ABI issue because the application will be re-
> > compiled
> > > > >>>> for the updated library. Indeed, compilation fixes compatibility issues.
> > > > >>>> But this is not relevant for ABI compatibility.
> > > > >>>> ABI compatibility means we can upgrade the library without
> > recompiling
> > > > >>>> the application and it must work.
> > > > >>>> You think it is a false positive because you assume the application
> > > > >>>> "picks" the new value. I think you miss the case where the new value
> > > > >>>> is returned by a function in the upgraded library.
> > > > >>>>
> > > > >>>>> There are also no structs on the API which contain arrays using this
> > > > >>>>> for sizing, so I don't see an opportunity for an appl to have a
> > > > >>>>> mismatch in memory addresses.
> > > > >>>>
> > > > >>>> Let me demonstrate where the API may "use" the new value
> > > > >>>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 and how it impacts the
> > > > application.
> > > > >>>>
> > > > >>>> Once upon a time a DPDK application counting the number of devices
> > > > >>>> supporting each AEAD algo (in order to find the best supported algo).
> > > > >>>> It is done in an array indexed by algo id:
> > > > >>>> int aead_dev_count[RTE_CRYPTO_AEAD_LIST_END];
> > > > >>>> The application is compiled with DPDK 19.11,
> > > > >>>> where RTE_CRYPTO_AEAD_LIST_END = 3.
> > > > >>>> So the size of the application array aead_dev_count is 3.
> > > > >>>> This binary is run with DPDK 20.02,
> > > > >>>> where RTE_CRYPTO_AEAD_CHACHA20_POLY1305 = 3.
> > > > >>>> When calling rte_cryptodev_info_get() on a device QAT_GEN3,
> > > > >>>> rte_cryptodev_info.capabilities.sym.aead.algo is set to
> > > > >>>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 (= 3).
> > > > >>>> The application uses this value:
> > > > >>>> ++ aead_dev_count[info.capabilities.sym.aead.algo];
> > > > >>>> The application is crashing because of out of bound access.
> > > > >>>
> > > > >>> I'd say this is an example of bad written app.
> > > > >>> It probably should check that returned by library value doesn't
> > > > >>> exceed its internal array size.
> > > > >>
> > > > >> +1
> > > > >>
> > > > >> Application should ignore values >= MAX.
> > > > >
> > > > > Of course, blaming the API user is a lot easier than looking at the API.
> > > > > Here the API has RTE_CRYPTO_AEAD_LIST_END which can be understood
> > > > > as the max value for the application.
> > > > > Value ranges are part of the ABI compatibility contract.
> > > > > It seems you expect the application developer to be aware that
> > > > > DPDK could return a higher value, so the application should
> > > > > check every enum values after calling an API. CRAZY.
> > > > >
> > > > > When we decide to announce an ABI compatibility and do some marketing,
> > > > > everyone is OK. But when we need to really make our ABI compatible,
> > > > > I see little or no effort. DISAPPOINTING.
> > > >
> > > > This is not to blame the user or to do less work, this is more sane approach
> > > > that library provides the _END/_MAX value and application uses it as valid
> > range
> > > > check.
> > > >
> > > > >
> > > > >> Do you suggest we don't extend any enum or define between ABI
> > breakage
> > > > releases
> > > > >> to be sure bad written applications not affected?
> > > > >
> > > > > I suggest we must consider not breaking any assumption made on the API.
> > > > > Here we are breaking the enum range because nothing mentions
> > _LIST_END
> > > > > is not really the absolute end of the enum.
> > > > > The solution is to make the change below in 20.02 + backport in 19.11.1:
> > > > >
> > > > > - _LIST_END
> > > > > + _LIST_END, /* an ABI-compatible version may increase this value */
> > > > > + _LIST_MAX = _LIST_END + 42 /* room for ABI-compatible additions */
> > > > > };
> > > > >
> > > >
> > > > What is the point of "_LIST_MAX" here?
> > > >
> > > > Application should know the "_LIST_END" of when it has been compiled for
> > the
> > > > valid range check. Next time it is compiled "_LIST_END" may be different
> > value
> > > > but same logic applies.
> > > >
> > > > When "_LIST_END" is missing, application can't protect itself, in that case
> > > > library should send only the values application knows when it is compiled,
> > this
> > > > means either we can't extend our enum/defines until next ABI breakage, or
> > we
> > > > need to do ABI versioning to the functions that returns an enum each time
> > enum
> > > > value extended.
> > > >
> > > > I believe it is saner to provide _END/_MAX values to the application to use.
> > And
> > > > if required comment them to clarify the expected usage.
> > > >
> > > > But in above suggestion application can't use or rely on "_LIST_MAX", it
> > doesn't
> > > > mean anything to application.
> > > >
> > >
> > > Can we have something like
> > > enum rte_crypto_aead_algorithm {
> > > RTE_CRYPTO_AEAD_AES_CCM = 1,
> > > /**< AES algorithm in CCM mode. */
> > > RTE_CRYPTO_AEAD_AES_GCM,
> > > /**< AES algorithm in GCM mode. */
> > > RTE_CRYPTO_AEAD_LIST_END,
> > > /**< List end for 19.11 ABI compatibility */
> > > RTE_CRYPTO_AEAD_CHACHA20_POLY1305,
> > > /**< Chacha20 cipher with poly1305 authenticator */
> > > RTE_CRYPTO_AEAD_LIST_END_2011
> > > /**< List end for 20.11 ABI compatibility */
> > > };
> > >
> > > And in 20.11 release we alter the RTE_CRYPTO_AEAD_LIST_END to the end
> > and remove RTE_CRYPTO_AEAD_LIST_END_2011
> > >
> > > I believe it will be ok for any application which need to use the chacha poly
> > assume that this algo is
> > > Experimental and will move to formal list in 20.11. This can be documented in
> > the documentation.
> > > I believe there is no way to add a new enum as experimental so far. This way
> > we can formalize this
> > > requirement as well.
> > >
> > > I believe this way effect of ABI breakage will be nullified.
> >
> > This is a possibility in the (a) proposal.
> > But it breaks API (and ABI) because a high value is returned
> > while not expected by the application.
> >
> > I guess ABI and release maintainers will vote no to such breakage.
> > Note: I vote no.
> >
>
> If that is the case, I would say we should go with b).
>
> Versioned APIs does not look good and adds more confusion.
>
What makes you say that?
Versioned APIs are the way you maintain backward compatibility.
If a library doesn't use versioned API's, then they either:
1) break frequently, causing application headaches
2) have APIS that are so mature, strictly defined, and small, they never change anyway
3) go to the trouble of creating compat libs for as far back as they need to
support
DPDK doesn't yet have a mature, stable API, so we have to do (1) or (2). (1)
has already been declared a bad idea, because application developers and distros
have declared a desire for backwards compatibility. We could go with (3)
instead of ABI versioning, but between compat libs and versioning, the latter is
the much less difficult way to handle that.
Neil
^ permalink raw reply [relevance 7%]
* [dpdk-dev] [RFC PATCH 0/7] vfio/pci: SR-IOV support
@ 2020-02-04 23:05 3% Alex Williamson
2020-02-11 11:18 0% ` Jerin Jacob
0 siblings, 1 reply; 200+ results
From: Alex Williamson @ 2020-02-04 23:05 UTC (permalink / raw)
To: kvm
Cc: linux-pci, linux-kernel, dev, mtosatti, thomas, bluca,
jerinjacobk, bruce.richardson, cohuck
There seems to be an ongoing desire to use userspace, vfio-based
drivers for both SR-IOV PF and VF devices. The fundamental issue
with this concept is that the VF is not fully independent of the PF
driver. Minimally the PF driver might be able to deny service to the
VF, VF data paths might be dependent on the state of the PF device,
or the PF my have some degree of ability to inspect or manipulate the
VF data. It therefore would seem irresponsible to unleash VFs onto
the system, managed by a user owned PF.
We address this in a few ways in this series. First, we can use a bus
notifier and the driver_override facility to make sure VFs are bound
to the vfio-pci driver by default. This should eliminate the chance
that a VF is accidentally bound and used by host drivers. We don't
however remove the ability for a host admin to change this override.
The next issue we need to address is how we let userspace drivers
opt-in to this participation with the PF driver. We do not want an
admin to be able to unwittingly assign one of these VFs to a tenant
that isn't working in collaboration with the PF driver. We could use
IOMMU grouping, but this seems to push too far towards tightly coupled
PF and VF drivers. This series introduces a "VF token", implemented
as a UUID, as a shared secret between PF and VF drivers. The token
needs to be set by the PF driver and used as part of the device
matching by the VF driver. Provisions in the code also account for
restarting the PF driver with active VF drivers, requiring the PF to
use the current token to re-gain access to the PF.
The above solutions introduce a bit of a modification to the VFIO ABI
and an additional ABI extension. The modification is that the
VFIO_GROUP_GET_DEVICE_FD ioctl is specified to require a char string
from the user providing the device name. For this solution, we extend
the syntax to allow the device name followed by key/value pairs. In
this case we add "vf_token=3e7e882e-1daf-417f-ad8d-882eea5ee337", for
example. These options are expected to be space separated. Matching
these key/value pairs is entirely left to the vfio bus driver (ex.
vfio-pci) and the internal ops structure is extended to allow this
optional support. This extension should be fully backwards compatible
to existing userspace, such code will simply fail to open these newly
exposed devices, as intended.
I've been debating whether instead of the above we should allow the
user to get the device fd as normal, but restrict the interfaces until
the user authenticates, but I'm afraid this would be a less backwards
compatible solution. It would be just as unclear to the user why a
device read/write/mmap/ioctl failed as it might be to why getting the
device fd could fail. However in the latter case, I believe we do a
better job of restricting how far userspace code might go before they
ultimately fail. I'd welcome discussion in the space, and or course
the extension of the GET_DEVICE_FD string.
Finally, the user needs to be able to set a VF token. I add a
VFIO_DEVICE_FEATURE ioctl for this that's meant to be reusable for
getting, setting, and probing arbitrary features of a device.
I'll reply to this cover letter with a very basic example of a QEMU
update to support this interface, though I haven't found a device yet
that behaves well with the PF running in one VM with the VF in
another, or really even just a PF running in a VM with SR-IOV enabled.
I know these devices exist though, and I suspect QEMU will not be the
primary user of this support for now, but this behavior reaffirms my
concerns to prevent mis-use.
Please comment. In particular, does this approach meet the DPDK needs
for userspace PF and VF drivers, with the hopefully minor hurdle of
sharing a token between drivers. The token is of course left to
userspace how to manage, and might be static (and not very secret) for
a given set of drivers. Thanks,
Alex
---
Alex Williamson (7):
vfio: Include optional device match in vfio_device_ops callbacks
vfio/pci: Implement match ops
vfio/pci: Introduce VF token
vfio: Introduce VFIO_DEVICE_FEATURE ioctl and first user
vfio/pci: Add sriov_configure support
vfio/pci: Remove dev_fmt definition
vfio/pci: Cleanup .probe() exit paths
drivers/vfio/pci/vfio_pci.c | 315 ++++++++++++++++++++++++++++++++---
drivers/vfio/pci/vfio_pci_private.h | 10 +
drivers/vfio/vfio.c | 19 ++
include/linux/vfio.h | 3
include/uapi/linux/vfio.h | 37 ++++
5 files changed, 356 insertions(+), 28 deletions(-)
^ permalink raw reply [relevance 3%]
* Re: [dpdk-dev] [EXT] Re: [PATCH v2 4/4] add ABI checks
2020-02-04 22:10 7% ` Neil Horman
@ 2020-02-05 6:16 7% ` Anoob Joseph
2020-02-05 14:33 4% ` Trahe, Fiona
0 siblings, 1 reply; 200+ results
From: Anoob Joseph @ 2020-02-05 6:16 UTC (permalink / raw)
To: Neil Horman, Akhil Goyal, Trahe, Fiona
Cc: Thomas Monjalon, Ferruh Yigit, Ananyev, Konstantin, Trahe, Fiona,
dev, David Marchand, Kusztal, ArkadiuszX, Richardson, Bruce,
Mcnamara, John, dodji, Andrew Rybchenko, aconole, bluca,
ktraynor
Hi Akhil, Neil, Fiona
Sorry for the late response. I want to propose a new change in line with what you folks had proposed.
May be we can treat the new features EXPERIMENTAL until a new stable release.
enum rte_crypto_aead_algorithm {
RTE_CRYPTO_AEAD_AES_CCM = 1,
/**< AES algorithm in CCM mode. */
RTE_CRYPTO_AEAD_AES_GCM,
/**< AES algorithm in GCM mode. */
RTE_CRYPTO_AEAD_LIST_END,
/**< List end for stable */
/** EXPERIMENTAL */
RTE_CRYPTO_AEAD_CHACHA20_POLY1305,
/**< Chacha20 cipher with poly1305 authenticator */
RTE_CRYPTO_AEAD_LIST_END_EXPERIMENTAL
/**< List end */
};
And then introduce an experimental API,
const struct rte_cryptodev_capabilities *
rte_cryptodev_get_exp_capabilites(uint8_t dev_id);
The PMD owner is expected to add new capabilities (only new ones) to this one until the new feature is deemed stable (ie, in one of the next stable releases). We don't expect users to change their API/ABI. For applications using EXPERIMENTAL is allowed to use the above capabilities to get the EXPERIMENTAL features.
This does involve moving around code in PMD when one feature is added, but that's the risk PMD owner is taking by upstreaming as EXPERIMENTAL and not in stable release.
Thanks,
Anoob
> -----Original Message-----
> From: Neil Horman <nhorman@tuxdriver.com>
> Sent: Wednesday, February 5, 2020 3:40 AM
> To: Akhil Goyal <akhil.goyal@nxp.com>
> Cc: Thomas Monjalon <thomas@monjalon.net>; Ferruh Yigit
> <ferruh.yigit@intel.com>; Ananyev, Konstantin
> <konstantin.ananyev@intel.com>; Trahe, Fiona <fiona.trahe@intel.com>;
> dev@dpdk.org; David Marchand <david.marchand@redhat.com>; Anoob Joseph
> <anoobj@marvell.com>; Kusztal, ArkadiuszX <arkadiuszx.kusztal@intel.com>;
> Richardson, Bruce <bruce.richardson@intel.com>; Mcnamara, John
> <john.mcnamara@intel.com>; dodji@seketeli.net; Andrew Rybchenko
> <arybchenko@solarflare.com>; aconole@redhat.com; bluca@debian.org;
> ktraynor@redhat.com
> Subject: [EXT] Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
>
> External Email
>
> ----------------------------------------------------------------------
> On Tue, Feb 04, 2020 at 10:32:01AM +0000, Akhil Goyal wrote:
> >
> > >
> > > 04/02/2020 11:16, Akhil Goyal:
> > > > Hi,
> > > > > On 2/3/2020 5:09 PM, Thomas Monjalon wrote:
> > > > > > 03/02/2020 10:30, Ferruh Yigit:
> > > > > >> On 2/2/2020 2:41 PM, Ananyev, Konstantin wrote:
> > > > > >>> 02/02/2020 14:05, Thomas Monjalon:
> > > > > >>>> 31/01/2020 15:16, Trahe, Fiona:
> > > > > >>>>> On 1/30/2020 8:18 PM, Thomas Monjalon wrote:
> > > > > >>>>>> 30/01/2020 17:09, Ferruh Yigit:
> > > > > >>>>>>> On 1/29/2020 8:13 PM, Akhil Goyal wrote:
> > > > > >>>>>>>>
> > > > > >>>>>>>> I believe these enums will be used only in case of ASYM
> > > > > >>>>>>>> case which
> > > is
> > > > > experimental.
> > > > > >>>>>>>
> > > > > >>>>>>> Independent from being experiment and not, this
> > > > > >>>>>>> shouldn't be a
> > > > > problem, I think
> > > > > >>>>>>> this is a false positive.
> > > > > >>>>>>>
> > > > > >>>>>>> The ABI break can happen when a struct has been shared
> > > > > >>>>>>> between
> > > the
> > > > > application
> > > > > >>>>>>> and the library (DPDK) and the layout of that memory
> > > > > >>>>>>> know
> > > differently
> > > > > by
> > > > > >>>>>>> application and the library.
> > > > > >>>>>>>
> > > > > >>>>>>> Here in all cases, there is no layout/size change.
> > > > > >>>>>>>
> > > > > >>>>>>> As to the value changes of the enums, since application
> > > > > >>>>>>> compiled
> > > with
> > > > > old DPDK,
> > > > > >>>>>>> it will know only up to '6', 7 and more means invalid to
> > > > > >>>>>>> the
> > > application.
> > > > > So it
> > > > > >>>>>>> won't send these values also it should ignore these
> > > > > >>>>>>> values from
> > > library.
> > > > > Only
> > > > > >>>>>>> consequence is old application won't able to use new
> > > > > >>>>>>> features
> > > those
> > > > > new enums
> > > > > >>>>>>> provide but that is expected/normal.
> > > > > >>>>>>
> > > > > >>>>>> If library give higher value than expected by the
> > > > > >>>>>> application, if the application uses this value as array
> > > > > >>>>>> index, there can be an access out of bounds.
> > > > > >>>>>
> > > > > >>>>> [Fiona] All asymmetric APIs are experimental so above
> > > > > >>>>> shouldn't be a
> > > > > problem.
> > > > > >>>>> But for the same issue with sym crypto below, I believe
> > > > > >>>>> Ferruh's
> > > > > explanation makes
> > > > > >>>>> sense and I don't see how there can be an API breakage.
> > > > > >>>>> So if an application hasn't compiled against the new lib
> > > > > >>>>> it will be still
> > > using
> > > > > the old value
> > > > > >>>>> which will be within bounds. If it's picking up the higher
> > > > > >>>>> new value
> > > from
> > > > > the lib it must
> > > > > >>>>> have been compiled against the lib so shouldn't have problems.
> > > > > >>>>
> > > > > >>>> You say there is no ABI issue because the application will
> > > > > >>>> be re-
> > > compiled
> > > > > >>>> for the updated library. Indeed, compilation fixes compatibility
> issues.
> > > > > >>>> But this is not relevant for ABI compatibility.
> > > > > >>>> ABI compatibility means we can upgrade the library without
> > > recompiling
> > > > > >>>> the application and it must work.
> > > > > >>>> You think it is a false positive because you assume the
> > > > > >>>> application "picks" the new value. I think you miss the
> > > > > >>>> case where the new value is returned by a function in the upgraded
> library.
> > > > > >>>>
> > > > > >>>>> There are also no structs on the API which contain arrays
> > > > > >>>>> using this for sizing, so I don't see an opportunity for
> > > > > >>>>> an appl to have a mismatch in memory addresses.
> > > > > >>>>
> > > > > >>>> Let me demonstrate where the API may "use" the new value
> > > > > >>>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 and how it impacts the
> > > > > application.
> > > > > >>>>
> > > > > >>>> Once upon a time a DPDK application counting the number of
> > > > > >>>> devices supporting each AEAD algo (in order to find the best
> supported algo).
> > > > > >>>> It is done in an array indexed by algo id:
> > > > > >>>> int aead_dev_count[RTE_CRYPTO_AEAD_LIST_END];
> > > > > >>>> The application is compiled with DPDK 19.11, where
> > > > > >>>> RTE_CRYPTO_AEAD_LIST_END = 3.
> > > > > >>>> So the size of the application array aead_dev_count is 3.
> > > > > >>>> This binary is run with DPDK 20.02, where
> > > > > >>>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 = 3.
> > > > > >>>> When calling rte_cryptodev_info_get() on a device QAT_GEN3,
> > > > > >>>> rte_cryptodev_info.capabilities.sym.aead.algo is set to
> > > > > >>>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 (= 3).
> > > > > >>>> The application uses this value:
> > > > > >>>> ++ aead_dev_count[info.capabilities.sym.aead.algo];
> > > > > >>>> The application is crashing because of out of bound access.
> > > > > >>>
> > > > > >>> I'd say this is an example of bad written app.
> > > > > >>> It probably should check that returned by library value
> > > > > >>> doesn't exceed its internal array size.
> > > > > >>
> > > > > >> +1
> > > > > >>
> > > > > >> Application should ignore values >= MAX.
> > > > > >
> > > > > > Of course, blaming the API user is a lot easier than looking at the API.
> > > > > > Here the API has RTE_CRYPTO_AEAD_LIST_END which can be
> > > > > > understood as the max value for the application.
> > > > > > Value ranges are part of the ABI compatibility contract.
> > > > > > It seems you expect the application developer to be aware that
> > > > > > DPDK could return a higher value, so the application should
> > > > > > check every enum values after calling an API. CRAZY.
> > > > > >
> > > > > > When we decide to announce an ABI compatibility and do some
> > > > > > marketing, everyone is OK. But when we need to really make our
> > > > > > ABI compatible, I see little or no effort. DISAPPOINTING.
> > > > >
> > > > > This is not to blame the user or to do less work, this is more
> > > > > sane approach that library provides the _END/_MAX value and
> > > > > application uses it as valid
> > > range
> > > > > check.
> > > > >
> > > > > >
> > > > > >> Do you suggest we don't extend any enum or define between ABI
> > > breakage
> > > > > releases
> > > > > >> to be sure bad written applications not affected?
> > > > > >
> > > > > > I suggest we must consider not breaking any assumption made on the
> API.
> > > > > > Here we are breaking the enum range because nothing mentions
> > > _LIST_END
> > > > > > is not really the absolute end of the enum.
> > > > > > The solution is to make the change below in 20.02 + backport in
> 19.11.1:
> > > > > >
> > > > > > - _LIST_END
> > > > > > + _LIST_END, /* an ABI-compatible version may increase this
> > > > > > + value */ _LIST_MAX = _LIST_END + 42 /* room for
> > > > > > + ABI-compatible additions */
> > > > > > };
> > > > > >
> > > > >
> > > > > What is the point of "_LIST_MAX" here?
> > > > >
> > > > > Application should know the "_LIST_END" of when it has been
> > > > > compiled for
> > > the
> > > > > valid range check. Next time it is compiled "_LIST_END" may be
> > > > > different
> > > value
> > > > > but same logic applies.
> > > > >
> > > > > When "_LIST_END" is missing, application can't protect itself,
> > > > > in that case library should send only the values application
> > > > > knows when it is compiled,
> > > this
> > > > > means either we can't extend our enum/defines until next ABI
> > > > > breakage, or
> > > we
> > > > > need to do ABI versioning to the functions that returns an enum
> > > > > each time
> > > enum
> > > > > value extended.
> > > > >
> > > > > I believe it is saner to provide _END/_MAX values to the application to
> use.
> > > And
> > > > > if required comment them to clarify the expected usage.
> > > > >
> > > > > But in above suggestion application can't use or rely on
> > > > > "_LIST_MAX", it
> > > doesn't
> > > > > mean anything to application.
> > > > >
> > > >
> > > > Can we have something like
> > > > enum rte_crypto_aead_algorithm {
> > > > RTE_CRYPTO_AEAD_AES_CCM = 1,
> > > > /**< AES algorithm in CCM mode. */
> > > > RTE_CRYPTO_AEAD_AES_GCM,
> > > > /**< AES algorithm in GCM mode. */
> > > > RTE_CRYPTO_AEAD_LIST_END,
> > > > /**< List end for 19.11 ABI compatibility */
> > > > RTE_CRYPTO_AEAD_CHACHA20_POLY1305,
> > > > /**< Chacha20 cipher with poly1305 authenticator */
> > > > RTE_CRYPTO_AEAD_LIST_END_2011
> > > > /**< List end for 20.11 ABI compatibility */ };
> > > >
> > > > And in 20.11 release we alter the RTE_CRYPTO_AEAD_LIST_END to the
> > > > end
> > > and remove RTE_CRYPTO_AEAD_LIST_END_2011
> > > >
> > > > I believe it will be ok for any application which need to use the
> > > > chacha poly
> > > assume that this algo is
> > > > Experimental and will move to formal list in 20.11. This can be
> > > > documented in
> > > the documentation.
> > > > I believe there is no way to add a new enum as experimental so
> > > > far. This way
> > > we can formalize this
> > > > requirement as well.
> > > >
> > > > I believe this way effect of ABI breakage will be nullified.
> > >
> > > This is a possibility in the (a) proposal.
> > > But it breaks API (and ABI) because a high value is returned while
> > > not expected by the application.
> > >
> > > I guess ABI and release maintainers will vote no to such breakage.
> > > Note: I vote no.
> > >
> >
> > If that is the case, I would say we should go with b).
> >
> > Versioned APIs does not look good and adds more confusion.
> >
> What makes you say that?
>
> Versioned APIs are the way you maintain backward compatibility.
>
> If a library doesn't use versioned API's, then they either:
>
> 1) break frequently, causing application headaches
> 2) have APIS that are so mature, strictly defined, and small, they never change
> anyway
> 3) go to the trouble of creating compat libs for as far back as they need to
> support
>
> DPDK doesn't yet have a mature, stable API, so we have to do (1) or (2). (1) has
> already been declared a bad idea, because application developers and distros
> have declared a desire for backwards compatibility. We could go with (3)
> instead of ABI versioning, but between compat libs and versioning, the latter is
> the much less difficult way to handle that.
>
> Neil
^ permalink raw reply [relevance 7%]
* Re: [dpdk-dev] [PATCH 00/14] cleanup resources on shutdown
@ 2020-02-05 9:32 0% ` David Marchand
2020-02-05 12:07 0% ` Stephen Hemminger
0 siblings, 1 reply; 200+ results
From: David Marchand @ 2020-02-05 9:32 UTC (permalink / raw)
To: Stephen Hemminger; +Cc: dev, Luca Boccassi, Bruce Richardson
Hello Stephen,
On Sat, Jan 4, 2020 at 2:34 AM Stephen Hemminger
<stephen@networkplumber.org> wrote:
>
> Recently started using valgrind with DPDK, and the results
> are not clean.
>
> The DPDK has a function that applications can use to tell it
> to cleanup resources on shutdown (rte_eal_cleanup). But the
> current coverage of that API is spotty. Many internal parts of
> DPDK leave files and allocated memory behind.
>
> This patch set is a start at getting the sub-parts of
> DPDK to cleanup after themselves. These are the easier ones,
> the harder and more critical ones are in the drivers
> and the memory subsystem.
>
> There are no visible API or ABI changes here.
Could you share what you did to run a dpdk application with valgrind?
I tried with testpmd and a 3.15 valgrind (fc30), but I get an init
failure on the cpu flags.
$ LD_LIBRARY_PATH=/home/dmarchan/builds/build-gcc-shared/install/usr/local/lib64
valgrind /home/dmarchan/builds/build-gcc-shared/install/usr/local/bin/dpdk-testpmd
-c 3 --no-huge -m 20 -d librte_mempool_ring.so -d librte_pmd_null.so
-w 0:0.0 --vdev net_null1 --vdev net_null2 -- --no-mlockall
--total-num-mbufs=2048 -ia
==10258== Memcheck, a memory error detector
==10258== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==10258== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright info
==10258== Command:
/home/dmarchan/builds/build-gcc-shared/install/usr/local/bin/dpdk-testpmd
-c 3 --no-huge -m 20 -d librte_mempool_ring.so -d librte_pmd_null.so
-w 0:0.0 --vdev net_null1 --vdev net_null2 -- --no-mlockall
--total-num-mbufs=2048 -ia
==10258==
ERROR: This system does not support "RDSEED".
Please check that RTE_MACHINE is set correctly.
EAL: FATAL: unsupported cpu type.
EAL: unsupported cpu type.
EAL: Error - exiting with code: 1
Cause: Cannot init EAL: Operation not supported
==10258==
==10258== HEAP SUMMARY:
==10258== in use at exit: 1,388 bytes in 49 blocks
==10258== total heap usage: 97 allocs, 48 frees, 89,426 bytes allocated
==10258==
==10258== LEAK SUMMARY:
==10258== definitely lost: 0 bytes in 0 blocks
==10258== indirectly lost: 0 bytes in 0 blocks
==10258== possibly lost: 0 bytes in 0 blocks
==10258== still reachable: 1,388 bytes in 49 blocks
==10258== suppressed: 0 bytes in 0 blocks
==10258== Rerun with --leak-check=full to see details of leaked memory
==10258==
==10258== For lists of detected and suppressed errors, rerun with: -s
==10258== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
Thanks.
--
David Marchand
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [RFC] meter: fix ABI break due to experimental tag removal
2020-02-04 12:02 9% ` Neil Horman
@ 2020-02-05 10:04 4% ` Luca Boccassi
2020-02-05 11:32 7% ` Neil Horman
0 siblings, 1 reply; 200+ results
From: Luca Boccassi @ 2020-02-05 10:04 UTC (permalink / raw)
To: Neil Horman, Ferruh Yigit
Cc: Cristian Dumitrescu, Eelco Chaudron, dev, Thomas Monjalon,
David Marchand, Bruce Richardson, Ian Stokes
On Tue, 2020-02-04 at 07:02 -0500, Neil Horman wrote:
> > But if we can do the versioning in the master, LTS can backport it
> > and can have
> > mature version of that API in LTS without breaking the existing
> > users.
> >
>
> But why bother? The only thing you've changed is the version
> tagging. Its ok
> to leave that alone in LTS, you just cant change it.
>
> Thats part of the burden of an LTS release, it will have some drift
> from the
> upstream head, because you have to keep things stable. So you
> stabilize the
> upstream ABI version for this API and just never backport it to the
> current LTS
> release.
Hi,
A customer (OVS) explicitly and specifically requested backporting the
symbol change to 19.11, as they don't want to enable experimental APIs
in their releases. I replied that it would only be acceptable with
aliasing to keep compatibility, and Ferruh very kindly did the work to
implement that.
--
Kind regards,
Luca Boccassi
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
2020-02-04 9:51 4% ` David Marchand
2020-02-04 10:10 4% ` Trahe, Fiona
@ 2020-02-05 11:10 4% ` Ray Kinsella
1 sibling, 0 replies; 200+ results
From: Ray Kinsella @ 2020-02-05 11:10 UTC (permalink / raw)
To: David Marchand
Cc: Thomas Monjalon, Neil Horman, Luca Boccassi, Kevin Traynor,
Ananyev, Konstantin, Akhil Goyal, Trahe, Fiona, Ferruh Yigit,
dev, Anoob Joseph, Kusztal, ArkadiuszX, Richardson, Bruce,
Mcnamara, John, dodji, Andrew Rybchenko, Aaron Conole
On 04/02/2020 09:51, David Marchand wrote:
> On Mon, Feb 3, 2020 at 7:56 PM Ray Kinsella <mdr@ashroe.eu> wrote:
>> On 03/02/2020 17:34, Thomas Monjalon wrote:
>>> 03/02/2020 18:09, Thomas Monjalon:
>>>> 03/02/2020 10:30, Ferruh Yigit:
>>>>> On 2/2/2020 2:41 PM, Ananyev, Konstantin wrote:
>>>>>> 02/02/2020 14:05, Thomas Monjalon:
>>>>>>> 31/01/2020 15:16, Trahe, Fiona:
>>>>>>>> On 1/30/2020 8:18 PM, Thomas Monjalon wrote:
>>>>>>>>> If library give higher value than expected by the application,
>>>>>>>>> if the application uses this value as array index,
>>>>>>>>> there can be an access out of bounds.
>>>>>>>>
>>>>>>>> [Fiona] All asymmetric APIs are experimental so above shouldn't be a problem.
>>>>>>>> But for the same issue with sym crypto below, I believe Ferruh's explanation makes
>>>>>>>> sense and I don't see how there can be an API breakage.
>>>>>>>> So if an application hasn't compiled against the new lib it will be still using the old value
>>>>>>>> which will be within bounds. If it's picking up the higher new value from the lib it must
>>>>>>>> have been compiled against the lib so shouldn't have problems.
>>>>>>>
>>>>>>> You say there is no ABI issue because the application will be re-compiled
>>>>>>> for the updated library. Indeed, compilation fixes compatibility issues.
>>>>>>> But this is not relevant for ABI compatibility.
>>>>>>> ABI compatibility means we can upgrade the library without recompiling
>>>>>>> the application and it must work.
>>>>>>> You think it is a false positive because you assume the application
>>>>>>> "picks" the new value. I think you miss the case where the new value
>>>>>>> is returned by a function in the upgraded library.
>>>>>>>
>>>>>>>> There are also no structs on the API which contain arrays using this
>>>>>>>> for sizing, so I don't see an opportunity for an appl to have a
>>>>>>>> mismatch in memory addresses.
>>>>>>>
>>>>>>> Let me demonstrate where the API may "use" the new value
>>>>>>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 and how it impacts the application.
>>>>>>>
>>>>>>> Once upon a time a DPDK application counting the number of devices
>>>>>>> supporting each AEAD algo (in order to find the best supported algo).
>>>>>>> It is done in an array indexed by algo id:
>>>>>>> int aead_dev_count[RTE_CRYPTO_AEAD_LIST_END];
>>>>>>> The application is compiled with DPDK 19.11,
>>>>>>> where RTE_CRYPTO_AEAD_LIST_END = 3.
>>>>>>> So the size of the application array aead_dev_count is 3.
>>>>>>> This binary is run with DPDK 20.02,
>>>>>>> where RTE_CRYPTO_AEAD_CHACHA20_POLY1305 = 3.
>>>>>>> When calling rte_cryptodev_info_get() on a device QAT_GEN3,
>>>>>>> rte_cryptodev_info.capabilities.sym.aead.algo is set to
>>>>>>> RTE_CRYPTO_AEAD_CHACHA20_POLY1305 (= 3).
>>>>>>> The application uses this value:
>>>>>>> ++ aead_dev_count[info.capabilities.sym.aead.algo];
>>>>>>> The application is crashing because of out of bound access.
>>>>>>
>>>>>> I'd say this is an example of bad written app.
>>>>>> It probably should check that returned by library value doesn't
>>>>>> exceed its internal array size.
>>>>>
>>>>> +1
>>>>>
>>>>> Application should ignore values >= MAX.
>>>>
>>>> Of course, blaming the API user is a lot easier than looking at the API.
>>>> Here the API has RTE_CRYPTO_AEAD_LIST_END which can be understood
>>>> as the max value for the application.
>>>> Value ranges are part of the ABI compatibility contract.
>>>> It seems you expect the application developer to be aware that
>>>> DPDK could return a higher value, so the application should
>>>> check every enum values after calling an API. CRAZY.
>>>>
>>>> When we decide to announce an ABI compatibility and do some marketing,
>>>> everyone is OK. But when we need to really make our ABI compatible,
>>>> I see little or no effort. DISAPPOINTING.
>>>>
>>>>> Do you suggest we don't extend any enum or define between ABI breakage releases
>>>>> to be sure bad written applications not affected?
>>>>
>>>> I suggest we must consider not breaking any assumption made on the API.
>>>> Here we are breaking the enum range because nothing mentions _LIST_END
>>>> is not really the absolute end of the enum.
>>>> The solution is to make the change below in 20.02 + backport in 19.11.1:
>>>
>>> Thinking twice, merging such change before 20.11 is breaking the
>>> ABI assumption based on the API 19.11.0.
>>> I ask the release maintainers (Luca, Kevin, David and me) and
>>> the ABI maintainers (Neil and Ray) to vote for a or b solution:
>>> a) add comment and LIST_MAX as below in 20.02 + 19.11.1
>>
>> That would still be an ABI breakage though right.
>
> Yes.
>
>
>>
>>> b) wait 20.11 and revert Chacha-Poly from 20.02
>>
>> Thanks for analysis above Fiona, Ferruh and all.
>>
>> That is a nasty one alright - there is no "good" answer here.
>> I agree with Ferruh's sentiments overall, we should rethink this API for 20.11.
>> Could do without an enumeration?
>>
>> There a c) though right.
>> We could work around the issue by api versioning rte_cryptodev_info_get() and friends.
>
> It has a lot of friends, but it sounds like the right approach.
+1
> Is someone looking into this?
Looks to be in hand now.
>
>
>> So they only support/acknowledge the existence of Chacha-Poly for applications build against > 20.02.
>>
>> It would be painful I know.
>> It would also mean that Chacha-Poly would only be available to those building against >= 20.02.
>
> Yes.
>
>
> --
> David Marchand
>
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [RFC] meter: fix ABI break due to experimental tag removal
2020-02-05 10:04 4% ` Luca Boccassi
@ 2020-02-05 11:32 7% ` Neil Horman
0 siblings, 0 replies; 200+ results
From: Neil Horman @ 2020-02-05 11:32 UTC (permalink / raw)
To: Luca Boccassi
Cc: Ferruh Yigit, Cristian Dumitrescu, Eelco Chaudron, dev,
Thomas Monjalon, David Marchand, Bruce Richardson, Ian Stokes
On Wed, Feb 05, 2020 at 11:04:29AM +0100, Luca Boccassi wrote:
> On Tue, 2020-02-04 at 07:02 -0500, Neil Horman wrote:
> > > But if we can do the versioning in the master, LTS can backport it
> > > and can have
> > > mature version of that API in LTS without breaking the existing
> > > users.
> > >
> >
> > But why bother? The only thing you've changed is the version
> > tagging. Its ok
> > to leave that alone in LTS, you just cant change it.
> >
> > Thats part of the burden of an LTS release, it will have some drift
> > from the
> > upstream head, because you have to keep things stable. So you
> > stabilize the
> > upstream ABI version for this API and just never backport it to the
> > current LTS
> > release.
>
> Hi,
>
> A customer (OVS) explicitly and specifically requested backporting the
> symbol change to 19.11, as they don't want to enable experimental APIs
> in their releases. I replied that it would only be acceptable with
> aliasing to keep compatibility, and Ferruh very kindly did the work to
> implement that.
>
but, thats part of the agreement, no? You can't always have new features and
stability at the same time.
I get that this is an odd corner case, because strictly speaking you could waive
the ABI change that libabigail is reporting, and most application users (like
OVS) could get away with it, because their application does all the right things
to make it ok, but I don't think you can make a decsion like this for all users
based on the request of a single user.
It seems like the right thing is for OVS to augment their build time
configuration to allow developers to select the ability to use experimental apis
at compile time, and then the decision is placed in the hands of end users.
Neil
> --
> Kind regards,
> Luca Boccassi
>
^ permalink raw reply [relevance 7%]
* Re: [dpdk-dev] [PATCH 00/14] cleanup resources on shutdown
2020-02-05 9:32 0% ` David Marchand
@ 2020-02-05 12:07 0% ` Stephen Hemminger
2020-02-05 12:32 0% ` David Marchand
0 siblings, 1 reply; 200+ results
From: Stephen Hemminger @ 2020-02-05 12:07 UTC (permalink / raw)
To: David Marchand; +Cc: dev, Luca Boccassi, Bruce Richardson
On Wed, 5 Feb 2020 10:32:49 +0100
David Marchand <david.marchand@redhat.com> wrote:
> Hello Stephen,
>
> On Sat, Jan 4, 2020 at 2:34 AM Stephen Hemminger
> <stephen@networkplumber.org> wrote:
> >
> > Recently started using valgrind with DPDK, and the results
> > are not clean.
> >
> > The DPDK has a function that applications can use to tell it
> > to cleanup resources on shutdown (rte_eal_cleanup). But the
> > current coverage of that API is spotty. Many internal parts of
> > DPDK leave files and allocated memory behind.
> >
> > This patch set is a start at getting the sub-parts of
> > DPDK to cleanup after themselves. These are the easier ones,
> > the harder and more critical ones are in the drivers
> > and the memory subsystem.
> >
> > There are no visible API or ABI changes here.
>
> Could you share what you did to run a dpdk application with valgrind?
>
> I tried with testpmd and a 3.15 valgrind (fc30), but I get an init
> failure on the cpu flags.
>
> $ LD_LIBRARY_PATH=/home/dmarchan/builds/build-gcc-shared/install/usr/local/lib64
> valgrind /home/dmarchan/builds/build-gcc-shared/install/usr/local/bin/dpdk-testpmd
> -c 3 --no-huge -m 20 -d librte_mempool_ring.so -d librte_pmd_null.so
> -w 0:0.0 --vdev net_null1 --vdev net_null2 -- --no-mlockall
> --total-num-mbufs=2048 -ia
> ==10258== Memcheck, a memory error detector
> ==10258== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
> ==10258== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright info
> ==10258== Command:
> /home/dmarchan/builds/build-gcc-shared/install/usr/local/bin/dpdk-testpmd
> -c 3 --no-huge -m 20 -d librte_mempool_ring.so -d librte_pmd_null.so
> -w 0:0.0 --vdev net_null1 --vdev net_null2 -- --no-mlockall
> --total-num-mbufs=2048 -ia
> ==10258==
> ERROR: This system does not support "RDSEED".
> Please check that RTE_MACHINE is set correctly.
> EAL: FATAL: unsupported cpu type.
> EAL: unsupported cpu type.
> EAL: Error - exiting with code: 1
> Cause: Cannot init EAL: Operation not supported
> ==10258==
> ==10258== HEAP SUMMARY:
> ==10258== in use at exit: 1,388 bytes in 49 blocks
> ==10258== total heap usage: 97 allocs, 48 frees, 89,426 bytes allocated
> ==10258==
> ==10258== LEAK SUMMARY:
> ==10258== definitely lost: 0 bytes in 0 blocks
> ==10258== indirectly lost: 0 bytes in 0 blocks
> ==10258== possibly lost: 0 bytes in 0 blocks
> ==10258== still reachable: 1,388 bytes in 49 blocks
> ==10258== suppressed: 0 bytes in 0 blocks
> ==10258== Rerun with --leak-check=full to see details of leaked memory
> ==10258==
> ==10258== For lists of detected and suppressed errors, rerun with: -s
> ==10258== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
>
I am testing with valgrind on ARM.
It should be possible on x86 but you need to dial down the RTE_MACHINE
choice to something valgrind understands.
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH 00/14] cleanup resources on shutdown
2020-02-05 12:07 0% ` Stephen Hemminger
@ 2020-02-05 12:32 0% ` David Marchand
0 siblings, 0 replies; 200+ results
From: David Marchand @ 2020-02-05 12:32 UTC (permalink / raw)
To: Stephen Hemminger; +Cc: dev, Luca Boccassi, Bruce Richardson
On Wed, Feb 5, 2020 at 1:07 PM Stephen Hemminger
<stephen@networkplumber.org> wrote:
>
> On Wed, 5 Feb 2020 10:32:49 +0100
> David Marchand <david.marchand@redhat.com> wrote:
>
> > Hello Stephen,
> >
> > On Sat, Jan 4, 2020 at 2:34 AM Stephen Hemminger
> > <stephen@networkplumber.org> wrote:
> > >
> > > Recently started using valgrind with DPDK, and the results
> > > are not clean.
> > >
> > > The DPDK has a function that applications can use to tell it
> > > to cleanup resources on shutdown (rte_eal_cleanup). But the
> > > current coverage of that API is spotty. Many internal parts of
> > > DPDK leave files and allocated memory behind.
> > >
> > > This patch set is a start at getting the sub-parts of
> > > DPDK to cleanup after themselves. These are the easier ones,
> > > the harder and more critical ones are in the drivers
> > > and the memory subsystem.
> > >
> > > There are no visible API or ABI changes here.
> >
> > Could you share what you did to run a dpdk application with valgrind?
> >
> > I tried with testpmd and a 3.15 valgrind (fc30), but I get an init
> > failure on the cpu flags.
> >
> > $ LD_LIBRARY_PATH=/home/dmarchan/builds/build-gcc-shared/install/usr/local/lib64
> > valgrind /home/dmarchan/builds/build-gcc-shared/install/usr/local/bin/dpdk-testpmd
> > -c 3 --no-huge -m 20 -d librte_mempool_ring.so -d librte_pmd_null.so
> > -w 0:0.0 --vdev net_null1 --vdev net_null2 -- --no-mlockall
> > --total-num-mbufs=2048 -ia
> > ==10258== Memcheck, a memory error detector
> > ==10258== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
> > ==10258== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright info
> > ==10258== Command:
> > /home/dmarchan/builds/build-gcc-shared/install/usr/local/bin/dpdk-testpmd
> > -c 3 --no-huge -m 20 -d librte_mempool_ring.so -d librte_pmd_null.so
> > -w 0:0.0 --vdev net_null1 --vdev net_null2 -- --no-mlockall
> > --total-num-mbufs=2048 -ia
> > ==10258==
> > ERROR: This system does not support "RDSEED".
> > Please check that RTE_MACHINE is set correctly.
> > EAL: FATAL: unsupported cpu type.
> > EAL: unsupported cpu type.
> > EAL: Error - exiting with code: 1
> > Cause: Cannot init EAL: Operation not supported
> > ==10258==
> > ==10258== HEAP SUMMARY:
> > ==10258== in use at exit: 1,388 bytes in 49 blocks
> > ==10258== total heap usage: 97 allocs, 48 frees, 89,426 bytes allocated
> > ==10258==
> > ==10258== LEAK SUMMARY:
> > ==10258== definitely lost: 0 bytes in 0 blocks
> > ==10258== indirectly lost: 0 bytes in 0 blocks
> > ==10258== possibly lost: 0 bytes in 0 blocks
> > ==10258== still reachable: 1,388 bytes in 49 blocks
> > ==10258== suppressed: 0 bytes in 0 blocks
> > ==10258== Rerun with --leak-check=full to see details of leaked memory
> > ==10258==
> > ==10258== For lists of detected and suppressed errors, rerun with: -s
> > ==10258== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
> >
>
> I am testing with valgrind on ARM.
> It should be possible on x86 but you need to dial down the RTE_MACHINE
> choice to something valgrind understands.
>
Ok, so no black magic in valgrind :-)
Yeah I managed to run with the x86-default target we have in
test-meson-builds.sh.
Thanks.
--
David Marchand
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH v4 0/3] add ABI checks
2020-02-02 21:08 9% ` [dpdk-dev] [PATCH v4 0/3] " David Marchand
2020-02-02 21:08 28% ` [dpdk-dev] [PATCH v4 3/3] " David Marchand
@ 2020-02-05 14:13 4% ` Thomas Monjalon
1 sibling, 0 replies; 200+ results
From: Thomas Monjalon @ 2020-02-05 14:13 UTC (permalink / raw)
To: David Marchand
Cc: dev, bruce.richardson, kevin.laatz, aconole, nhorman,
akhil.goyal, anoobj, bluca, fiona.trahe, ferruh.yigit,
Arek Kusztal
02/02/2020 22:08, David Marchand:
> Here is the current state of the ABI checks.
>
> libabigail has some issues when mixing dump and so files compiled with
> clang [1], so for now, all checks are done on dumps only.
> libabigail 1.0-rc3 in Xenial reported issues that disappear with the
> version 1.2 in Bionic.
>
> To avoid getting warnings on internal types like [2], the checks now make
> use of the public headers part of a dpdk installation (patch 2 and 3 to
> prepare for this).
>
> Some internal rte_hash headers were installed by meson, so patch 1 fixes
> this.
>
> The most important point, abidiff complains on the rc1 cryptodev changes:
> - Chacha20-Poly1305 AEAD support,
> - ECPM and ECDSA support
>
> A suppression rule has been put on the internal type rte_cryptodev_ops.
> But other changes are an ABI breakage afaiu. I put suppression rules on
> them so that we can run the checks, but some action must be taken for
> 20.02 if my analysis is confirmed.
The suppression rules for Chacha-Poly are removed,
and Chacha-Poly commits are reverted while merging this series.
> Special thanks to Dodji the libabigail maintainer for helping on this
> topic.
>
> 1: https://sourceware.org/bugzilla/show_bug.cgi?id=25409
> 2: http://inbox.dpdk.org/dev/CAJFAV8yFKoDZROX9Mkyp7pDMvXw3e7mHwxjfrcjD5ZoFB2tZ8w@mail.gmail.com/
Applied, thanks
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH v3] cryptodev: add chacha20-poly1305 aead algorithm
@ 2020-02-05 14:28 5% ` Thomas Monjalon
2020-02-05 14:41 0% ` Trahe, Fiona
0 siblings, 1 reply; 200+ results
From: Thomas Monjalon @ 2020-02-05 14:28 UTC (permalink / raw)
To: Anoob Joseph, Arek Kusztal, fiona.trahe, Akhil Goyal
Cc: dev, Narayana Prasad Raju Athreya, Tejasree Kondoj,
Jerin Jacob Kollanukkaran, bluca, ktraynor, david.marchand,
nhorman, ray.kinsella, ferruh.yigit, bruce.richardson
15/01/2020 16:53, Akhil Goyal:
> > > This patch adds Chacha20-Poly1305 AEAD algorithm to Cryptodev.
> > >
> > > Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
> > > Acked-by: Fiona Trahe <fiona.trahe@intel.com>
> >
> > Acked-by: Anoob Joseph <anoobj@marvell.com>
> >
> Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
>
> Applied to dpdk-next-crypto
As discussed in other email threads,
this patch was causing an ABI breakage and had to be reverted:
http://git.dpdk.org/dpdk/commit/?id=f5862ae99
Sorry, this feature will miss 20.02 release I'm afraid.
2 actions should be taken:
- try to re-intoduce the feature with function versioning
- improve ABI stability in DPDK 20.11 for future features
We are all (hardly) learning how to maintain ABI compatibility,
as this is our first real issue since the new ABI policy.
Thanks for understanding.
^ permalink raw reply [relevance 5%]
* Re: [dpdk-dev] [EXT] Re: [PATCH v2 4/4] add ABI checks
2020-02-05 6:16 7% ` [dpdk-dev] [EXT] " Anoob Joseph
@ 2020-02-05 14:33 4% ` Trahe, Fiona
0 siblings, 0 replies; 200+ results
From: Trahe, Fiona @ 2020-02-05 14:33 UTC (permalink / raw)
To: Anoob Joseph, Neil Horman, Akhil Goyal
Cc: Thomas Monjalon, Yigit, Ferruh, Ananyev, Konstantin, dev,
David Marchand, Kusztal, ArkadiuszX, Richardson, Bruce, Mcnamara,
John, dodji, Andrew Rybchenko, aconole, bluca, ktraynor, Trahe,
Fiona
Hi Anoob,
> -----Original Message-----
> From: Anoob Joseph <anoobj@marvell.com>
> Sent: Wednesday, February 5, 2020 6:16 AM
> To: Neil Horman <nhorman@tuxdriver.com>; Akhil Goyal <akhil.goyal@nxp.com>; Trahe, Fiona
> <fiona.trahe@intel.com>
> Cc: Thomas Monjalon <thomas@monjalon.net>; Yigit, Ferruh <ferruh.yigit@intel.com>; Ananyev,
> Konstantin <konstantin.ananyev@intel.com>; Trahe, Fiona <fiona.trahe@intel.com>; dev@dpdk.org;
> David Marchand <david.marchand@redhat.com>; Kusztal, ArkadiuszX <arkadiuszx.kusztal@intel.com>;
> Richardson, Bruce <bruce.richardson@intel.com>; Mcnamara, John <john.mcnamara@intel.com>;
> dodji@seketeli.net; Andrew Rybchenko <arybchenko@solarflare.com>; aconole@redhat.com;
> bluca@debian.org; ktraynor@redhat.com
> Subject: RE: [EXT] Re: [dpdk-dev] [PATCH v2 4/4] add ABI checks
>
> Hi Akhil, Neil, Fiona
>
> Sorry for the late response. I want to propose a new change in line with what you folks had proposed.
>
> May be we can treat the new features EXPERIMENTAL until a new stable release.
>
> enum rte_crypto_aead_algorithm {
> RTE_CRYPTO_AEAD_AES_CCM = 1,
> /**< AES algorithm in CCM mode. */
> RTE_CRYPTO_AEAD_AES_GCM,
> /**< AES algorithm in GCM mode. */
> RTE_CRYPTO_AEAD_LIST_END,
> /**< List end for stable */
> /** EXPERIMENTAL */
> RTE_CRYPTO_AEAD_CHACHA20_POLY1305,
> /**< Chacha20 cipher with poly1305 authenticator */
> RTE_CRYPTO_AEAD_LIST_END_EXPERIMENTAL
> /**< List end */
> };
>
> And then introduce an experimental API,
>
> const struct rte_cryptodev_capabilities *
> rte_cryptodev_get_exp_capabilites(uint8_t dev_id);
>
> The PMD owner is expected to add new capabilities (only new ones) to this one until the new feature is
> deemed stable (ie, in one of the next stable releases). We don't expect users to change their API/ABI.
> For applications using EXPERIMENTAL is allowed to use the above capabilities to get the EXPERIMENTAL
> features.
>
> This does involve moving around code in PMD when one feature is added, but that's the risk PMD
> owner is taking by upstreaming as EXPERIMENTAL and not in stable release.
>
> Thanks,
> Anoob
[Fiona] Thanks for the suggestion Anoob.
I like the enum part of the idea - but not the new temporary API as the applications need to be aware of it and would have to change again when it's removed.
I explored an alternative way of using the current experimental infrastructure, i.e.:
enum rte_crypto_aead_algorithm {
RTE_CRYPTO_AEAD_AES_CCM = 1,
/**< AES algorithm in CCM mode. */
RTE_CRYPTO_AEAD_AES_GCM,
/**< AES algorithm in GCM mode. */
#ifdef ALLOW_EXPERIMENTAL_API
RTE_CRYPTO_AEAD_CHACHA20_POLY1305,
/**< Chacha20 cipher with poly1305 authenticator */
#endif
RTE_CRYPTO_AEAD_LIST_END,
/**< List end */
};
No new rte_cryptodev_get_exp_capabilities() needed.
Any PMD that implements the experimental API must do the same:
#ifdef ALLOW_EXPERIMENTAL_API
<PMD code processing new enum>
#endif
Same with test code.
Any 19.11 production code that wants to run against shared objects from
20.02 can be expected to build DPDK with ALLOW_EXPERIMENTAL_API disabled,
so will not pick up the new feature.
However, it appears the flag is not globally consistent, i.e. most PMDs have it set, even if the application doesn't set it. So this probably wouldn't work.
We're testing the approach outlined yesterday and believe it satisfactorily resolves the issue, so will stick with that.
^ permalink raw reply [relevance 4%]
* [dpdk-dev] Travis warning about job configs
@ 2020-02-05 14:34 3% David Marchand
0 siblings, 0 replies; 200+ results
From: David Marchand @ 2020-02-05 14:34 UTC (permalink / raw)
To: Aaron Conole, Michael Santana; +Cc: dev
While monitoring the newly pushed ABI check in Travis, I noticed that
Travis has now a feature to validate the configuration we pass.
You can notice a new "Build config validation" box in
https://travis-ci.com/DPDK/dpdk/builds/147554312/config
Not sure those warnings are a real issue, but it could become one in the future.
--
David Marchand
^ permalink raw reply [relevance 3%]
* Re: [dpdk-dev] [PATCH v3] cryptodev: add chacha20-poly1305 aead algorithm
2020-02-05 14:28 5% ` Thomas Monjalon
@ 2020-02-05 14:41 0% ` Trahe, Fiona
2020-02-05 15:21 0% ` Thomas Monjalon
0 siblings, 1 reply; 200+ results
From: Trahe, Fiona @ 2020-02-05 14:41 UTC (permalink / raw)
To: Thomas Monjalon, Anoob Joseph, Kusztal, ArkadiuszX, Akhil Goyal
Cc: dev, Narayana Prasad Raju Athreya, Tejasree Kondoj,
Jerin Jacob Kollanukkaran, bluca, ktraynor, david.marchand,
nhorman, Kinsella, Ray, Yigit, Ferruh, Richardson, Bruce, Trahe,
Fiona
Hi Thomas,
> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: Wednesday, February 5, 2020 2:29 PM
> To: Anoob Joseph <anoobj@marvell.com>; Kusztal, ArkadiuszX <arkadiuszx.kusztal@intel.com>; Trahe,
> Fiona <fiona.trahe@intel.com>; Akhil Goyal <akhil.goyal@nxp.com>
> Cc: dev@dpdk.org; Narayana Prasad Raju Athreya <pathreya@marvell.com>; Tejasree Kondoj
> <ktejasree@marvell.com>; Jerin Jacob Kollanukkaran <jerinj@marvell.com>; bluca@debian.org;
> ktraynor@redhat.com; david.marchand@redhat.com; nhorman@tuxdriver.com; Kinsella, Ray
> <ray.kinsella@intel.com>; Yigit, Ferruh <ferruh.yigit@intel.com>; Richardson, Bruce
> <bruce.richardson@intel.com>
> Subject: Re: [dpdk-dev] [PATCH v3] cryptodev: add chacha20-poly1305 aead algorithm
>
> 15/01/2020 16:53, Akhil Goyal:
> > > > This patch adds Chacha20-Poly1305 AEAD algorithm to Cryptodev.
> > > >
> > > > Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
> > > > Acked-by: Fiona Trahe <fiona.trahe@intel.com>
> > >
> > > Acked-by: Anoob Joseph <anoobj@marvell.com>
> > >
> > Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
> >
> > Applied to dpdk-next-crypto
>
> As discussed in other email threads,
> this patch was causing an ABI breakage and had to be reverted:
> http://git.dpdk.org/dpdk/commit/?id=f5862ae99
> Sorry, this feature will miss 20.02 release I'm afraid.
>
> 2 actions should be taken:
> - try to re-intoduce the feature with function versioning
> - improve ABI stability in DPDK 20.11 for future features
[Fiona] Agreed, it's underway.
The QAT ChaChaPoly patchset including the unit tests will also need to be reverted.
https://patches.dpdk.org/patch/64729/
https://patches.dpdk.org/patch/64730/
^ permalink raw reply [relevance 0%]
* [dpdk-dev] [PATCH] doc: alias to experimental tag for stable apis
@ 2020-02-05 15:17 17% Ray Kinsella
2020-02-05 15:43 17% ` [dpdk-dev] [PATCH v3] " Ray Kinsella
0 siblings, 1 reply; 200+ results
From: Ray Kinsella @ 2020-02-05 15:17 UTC (permalink / raw)
To: dev; +Cc: john.mcnamara, nhorman, ferruh.yigit, marko.kovacevic, Ray Kinsella
When a maintainer is promoting an API to become part of the next major ABI
version by removing the experimental tag, possibly a few releases in advance of
the declaration of the next ABI version. The maintainer may choose to offer an
alias to the experimental tag, as removing the tag before the declaration of the
next major ABI version, would cause an ABI breakage for applications using the
API.
Signed-off-by: Ray Kinsella <mdr@ashroe.eu>
---
doc/guides/contributing/abi_policy.rst | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/doc/guides/contributing/abi_policy.rst b/doc/guides/contributing/abi_policy.rst
index 05ca959..9a4a102 100644
--- a/doc/guides/contributing/abi_policy.rst
+++ b/doc/guides/contributing/abi_policy.rst
@@ -159,6 +159,11 @@ The requirements for changing the ABI are:
``experimental``, as described in the section on :ref:`Experimental APIs
and Libraries <experimental_apis>`.
+ - In situations where an ``experimental`` API has been stable for some time.
+ When promoting the API to become part of the next ABI version, the
+ maintainer may choose to provide an alias to the ``experimental`` tag, so
+ as not to break consuming applications.
+
#. If a newly proposed API functionally replaces an existing one, when the new
API becomes non-experimental, then the old one is marked with
``__rte_deprecated``.
@@ -317,6 +322,11 @@ not required. Though, an API should remain in experimental state for at least
one release. Thereafter, the normal process of posting patch for review to
mailing list can be followed.
+After the experimental tag has been formally removed, a tree/sub-tree maintainer
+may choose to offer an alias to the experimental tag so as not to break
+applications using the API. This alias can then be dropped at the declaration of
+next major ABI version.
+
Libraries
~~~~~~~~~
--
2.7.4
^ permalink raw reply [relevance 17%]
* Re: [dpdk-dev] [PATCH v3] cryptodev: add chacha20-poly1305 aead algorithm
2020-02-05 14:41 0% ` Trahe, Fiona
@ 2020-02-05 15:21 0% ` Thomas Monjalon
0 siblings, 0 replies; 200+ results
From: Thomas Monjalon @ 2020-02-05 15:21 UTC (permalink / raw)
To: Trahe, Fiona
Cc: Anoob Joseph, Kusztal, ArkadiuszX, Akhil Goyal, dev,
Narayana Prasad Raju Athreya, Tejasree Kondoj,
Jerin Jacob Kollanukkaran, bluca, ktraynor, david.marchand,
nhorman, Kinsella, Ray, Yigit, Ferruh, Richardson, Bruce
05/02/2020 15:41, Trahe, Fiona:
> From: Thomas Monjalon <thomas@monjalon.net>
> > 15/01/2020 16:53, Akhil Goyal:
> > > > > This patch adds Chacha20-Poly1305 AEAD algorithm to Cryptodev.
> > > > >
> > > > > Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
> > > > > Acked-by: Fiona Trahe <fiona.trahe@intel.com>
> > > >
> > > > Acked-by: Anoob Joseph <anoobj@marvell.com>
> > > >
> > > Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
> > >
> > > Applied to dpdk-next-crypto
> >
> > As discussed in other email threads,
> > this patch was causing an ABI breakage and had to be reverted:
> > http://git.dpdk.org/dpdk/commit/?id=f5862ae99
> > Sorry, this feature will miss 20.02 release I'm afraid.
> >
> > 2 actions should be taken:
> > - try to re-intoduce the feature with function versioning
> > - improve ABI stability in DPDK 20.11 for future features
>
> [Fiona] Agreed, it's underway.
Any estimation of time?
I expect a lot of discussions on the solution,
especially because it is a primer.
> The QAT ChaChaPoly patchset including the unit tests will also need to be reverted.
> https://patches.dpdk.org/patch/64729/
> https://patches.dpdk.org/patch/64730/
Yes, this is what I did in one commit.
^ permalink raw reply [relevance 0%]
* [dpdk-dev] [PATCH v2] doc: alias to experimental tag for stable apis
@ 2020-02-05 15:28 17% Ray Kinsella
0 siblings, 0 replies; 200+ results
From: Ray Kinsella @ 2020-02-05 15:28 UTC (permalink / raw)
To: dev; +Cc: john.mcnamara, nhorman, ferruh.yigit, marko.kovacevic, Ray Kinsella
When a maintainer is promoting an API to become part of the next major ABI
version by removing the experimental tag, possibly a few releases in
advance of the declaration of the next ABI version. The maintainer may
choose to offer an alias to the experimental tag, as removing the tag
before the declaration of the next major ABI version, would cause an ABI
breakage for applications using the API.
Signed-off-by: Ray Kinsella <mdr@ashroe.eu>
---
doc/guides/contributing/abi_policy.rst | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/doc/guides/contributing/abi_policy.rst b/doc/guides/contributing/abi_policy.rst
index 05ca959..9a4a102 100644
--- a/doc/guides/contributing/abi_policy.rst
+++ b/doc/guides/contributing/abi_policy.rst
@@ -159,6 +159,11 @@ The requirements for changing the ABI are:
``experimental``, as described in the section on :ref:`Experimental APIs
and Libraries <experimental_apis>`.
+ - In situations where an ``experimental`` API has been stable for some time.
+ When promoting the API to become part of the next ABI version, the
+ maintainer may choose to provide an alias to the ``experimental`` tag, so
+ as not to break consuming applications.
+
#. If a newly proposed API functionally replaces an existing one, when the new
API becomes non-experimental, then the old one is marked with
``__rte_deprecated``.
@@ -317,6 +322,11 @@ not required. Though, an API should remain in experimental state for at least
one release. Thereafter, the normal process of posting patch for review to
mailing list can be followed.
+After the experimental tag has been formally removed, a tree/sub-tree maintainer
+may choose to offer an alias to the experimental tag so as not to break
+applications using the API. This alias can then be dropped at the declaration of
+next major ABI version.
+
Libraries
~~~~~~~~~
--
2.7.4
^ permalink raw reply [relevance 17%]
* [dpdk-dev] [PATCH v3] doc: alias to experimental tag for stable apis
2020-02-05 15:17 17% [dpdk-dev] [PATCH] doc: alias to experimental tag for stable apis Ray Kinsella
@ 2020-02-05 15:43 17% ` Ray Kinsella
0 siblings, 0 replies; 200+ results
From: Ray Kinsella @ 2020-02-05 15:43 UTC (permalink / raw)
To: dev; +Cc: john.mcnamara, nhorman, ferruh.yigit, marko.kovacevic, Ray Kinsella
When a maintainer is promoting an API to become part of the next major ABI
version by removing the experimental tag, possibly a few releases in
advance of the declaration of the next ABI version. The maintainer may
choose to offer an alias to the experimental tag, as removing the tag
before the declaration of the next major ABI version, would cause an ABI
breakage for applications using the API.
Signed-off-by: Ray Kinsella <mdr@ashroe.eu>
---
doc/guides/contributing/abi_policy.rst | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/doc/guides/contributing/abi_policy.rst b/doc/guides/contributing/abi_policy.rst
index 05ca959..9a4a102 100644
--- a/doc/guides/contributing/abi_policy.rst
+++ b/doc/guides/contributing/abi_policy.rst
@@ -159,6 +159,11 @@ The requirements for changing the ABI are:
``experimental``, as described in the section on :ref:`Experimental APIs
and Libraries <experimental_apis>`.
+ - In situations where an ``experimental`` API has been stable for some time.
+ When promoting the API to become part of the next ABI version, the
+ maintainer may choose to provide an alias to the ``experimental`` tag, so
+ as not to break consuming applications.
+
#. If a newly proposed API functionally replaces an existing one, when the new
API becomes non-experimental, then the old one is marked with
``__rte_deprecated``.
@@ -317,6 +322,11 @@ not required. Though, an API should remain in experimental state for at least
one release. Thereafter, the normal process of posting patch for review to
mailing list can be followed.
+After the experimental tag has been formally removed, a tree/sub-tree maintainer
+may choose to offer an alias to the experimental tag so as not to break
+applications using the API. This alias can then be dropped at the declaration of
+next major ABI version.
+
Libraries
~~~~~~~~~
--
2.7.4
^ permalink raw reply [relevance 17%]
* Re: [dpdk-dev] [PATCH] buildtools: allow pedantic empty pmdinfo
@ 2020-02-05 17:29 3% ` Thomas Monjalon
0 siblings, 0 replies; 200+ results
From: Thomas Monjalon @ 2020-02-05 17:29 UTC (permalink / raw)
To: Bruce Richardson
Cc: dev, ferruh.yigit, matan, maxime.coquelin, david.marchand
05/02/2020 18:14, Bruce Richardson:
> On Wed, Feb 05, 2020 at 06:07:23PM +0100, Thomas Monjalon wrote:
> > If a driver has no info generated by pmdinfogen,
> > and if this driver is compiled as pedantic,
> > then an failure would occur:
> > drivers/rte_common_mlx5.pmd.c:1: error:
> > ISO C forbids an empty translation unit [-Werror=pedantic]
> > Such error is triggered with the new mlx5 common directory.
> >
> > In order to allow an "empty driver info" compiled in pedantic mode,
> > the script generating .pmd.c file is modified to add a static string
> > unconditionnaly.
> > The minimal generated code is:
> > static __attribute__((unused)) const char *generator =
> > "/path/to/dpdk/buildtools/gen-pmdinfo-cfile.sh";
> >
> > Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
>
> LGTM
>
> Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Applied, thanks, it will allow pulling next-net
without breaking debug compilation for the new ABI tooling.
^ permalink raw reply [relevance 3%]
* Re: [dpdk-dev] [PATCH v2] build: remove unused function versioning
2020-01-30 16:11 3% ` David Marchand
@ 2020-02-05 20:44 0% ` David Marchand
0 siblings, 0 replies; 200+ results
From: David Marchand @ 2020-02-05 20:44 UTC (permalink / raw)
To: Andrzej Ostruszka
Cc: dev, David Hunt, Bruce Richardson, Vladimir Medvedkin,
Robert Sanford, Erik Gabriel Carrillo, Marcin Baran, dpdk stable
On Thu, Jan 30, 2020 at 5:11 PM David Marchand
<david.marchand@redhat.com> wrote:
>
> On Thu, Jan 30, 2020 at 10:25 AM Andrzej Ostruszka
> <aostruszka@marvell.com> wrote:
> >
> > Timer, LPM and Distributor libraries no longer use function versioning
> > and therefore do not need separate build for static and shared version
> > of libraries.
> >
> > This patch removes use_function_versioning from their meson build files
> > and corresponding include from the sources.
> >
> > Signed-off-by: Andrzej Ostruszka <aostruszka@marvell.com>
> > Acked-by: Bruce Richardson <bruce.richardson@intel.com>
> >
> > Fixes: f2fb215843a9 ("timer: remove deprecated code")
> > Fixes: 6e5b51676176 ("distributor: remove deprecated code")
> > Fixes: c381a8d554b7 ("lpm: remove deprecated code")
> > Cc: marcinx.baran@intel.com
> >
> > ---
> > V2:
> > - removed also includes (updating only meson files causes build
> > failures), and updated commit msg accordingly
>
> Thanks, I had the exact same patch in store (caught when working with
> Olivier on some ABI stuff for mempool).
>
> Cc: stable@dpdk.org
>
> Acked-by: David Marchand <david.marchand@redhat.com>
Applied, thanks.
--
David Marchand
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH v2 1/2] crypto/ccp: sha3 support enabling in ccp
@ 2020-02-05 22:22 3% ` Thomas Monjalon
2020-02-06 9:34 0% ` David Marchand
0 siblings, 1 reply; 200+ results
From: Thomas Monjalon @ 2020-02-05 22:22 UTC (permalink / raw)
To: Kumar, Ravi1, Sardar, Shamsher singh; +Cc: dev, Akhil Goyal, david.marchand
05/02/2020 13:24, Akhil Goyal:
> > For series,
> > Acked-by: Ravi Kumar <ravi1.kumar@amd.com>
> >
> > >
> > >From: Sardar Shamsher Singh <Shamshersingh.Sardar@amd.com>
> > >
> > >sha3 support enabled in AMD-CCP crypto controller
> > >
> > >Signed-off-by: Sardar Shamsher Singh <Shamshersingh.Sardar@amd.com>
> > >---
> Change patch title and description as below
> cryptodev: fix missing SHA3 algo strings
>
> SHA3 support was added earlier but algo strings were
> missing. This patch add the missing strings.
>
> Fixes: 1df800f89518 ("crypto/ccp: support SHA3 family")
> Cc: stable@dpdk.org
>
> Signed-off-by: Sardar Shamsher Singh <Shamshersingh.Sardar@amd.com>
> Acked-by: Ravi Kumar <ravi1.kumar@amd.com>
> Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
>
> Applied to dpdk-next-crypto
Sorry I must drop this patch because it triggers an ABI warning:
[C]'const char* rte_crypto_auth_algorithm_strings[]' was changed at rte_crypto_sym.h:320:1:
size of symbol changed from 168 to 232
I don't understand how the size of this array can impact the application,
but I prefer to stay on the safe side for now.
Please let's discuss this patch in 20.02-rc3 timeframe.
^ permalink raw reply [relevance 3%]
* Re: [dpdk-dev] [PATCH v2 1/2] crypto/ccp: sha3 support enabling in ccp
2020-02-05 22:22 3% ` Thomas Monjalon
@ 2020-02-06 9:34 0% ` David Marchand
0 siblings, 0 replies; 200+ results
From: David Marchand @ 2020-02-06 9:34 UTC (permalink / raw)
To: Thomas Monjalon
Cc: Kumar, Ravi1, Sardar, Shamsher singh, dev, Akhil Goyal, dodji
On Wed, Feb 5, 2020 at 11:22 PM Thomas Monjalon <thomas@monjalon.net> wrote:
> 05/02/2020 13:24, Akhil Goyal:
> > > For series,
> > > Acked-by: Ravi Kumar <ravi1.kumar@amd.com>
> > >
> > > >
> > > >From: Sardar Shamsher Singh <Shamshersingh.Sardar@amd.com>
> > > >
> > > >sha3 support enabled in AMD-CCP crypto controller
> > > >
> > > >Signed-off-by: Sardar Shamsher Singh <Shamshersingh.Sardar@amd.com>
> > > >---
> > Change patch title and description as below
> > cryptodev: fix missing SHA3 algo strings
> >
> > SHA3 support was added earlier but algo strings were
> > missing. This patch add the missing strings.
> >
> > Fixes: 1df800f89518 ("crypto/ccp: support SHA3 family")
> > Cc: stable@dpdk.org
> >
> > Signed-off-by: Sardar Shamsher Singh <Shamshersingh.Sardar@amd.com>
> > Acked-by: Ravi Kumar <ravi1.kumar@amd.com>
> > Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
> >
> > Applied to dpdk-next-crypto
>
> Sorry I must drop this patch because it triggers an ABI warning:
> [C]'const char* rte_crypto_auth_algorithm_strings[]' was changed at rte_crypto_sym.h:320:1:
> size of symbol changed from 168 to 232
This is still not clear to me, but here is how I understand the issue.
An exposed array (and its size) ends up in both the shared library and
the final binary data section.
[dmarchan@wsfd-netdev66 dpdk]$ readelf -sW
~/builds/build-gcc-shared/app/dpdk-test-crypto-perf |grep
rte_crypto_auth_algorithm_strings
86: 00000000004141a0 168 OBJECT GLOBAL DEFAULT 24
rte_crypto_auth_algorithm_strings@DPDK_20.0 (4)
308: 00000000004141a0 168 OBJECT GLOBAL DEFAULT 24
rte_crypto_auth_algorithm_strings@@DPDK_20.0
[dmarchan@wsfd-netdev66 dpdk]$ readelf -sW
~/builds/build-gcc-shared/lib/librte_cryptodev.so |grep
rte_crypto_auth_algorithm_strings
57: 000000000000b220 168 OBJECT GLOBAL DEFAULT 23
rte_crypto_auth_algorithm_strings@@DPDK_20.0
158: 000000000000b220 168 OBJECT GLOBAL DEFAULT 23
rte_crypto_auth_algorithm_strings
At runtime, the linker chooses to rewire all access to the final
binary data section, not the shared library local representation.
Now, if we update the array size, the shared library code is built
with the assumption of the increased size.
But at runtime with an "old" binary, the shared library code runs with
a shorter array, with potential out of bound access.
Interesting article:
https://developers.redhat.com/blog/2019/05/06/how-c-array-sizes-become-part-of-the-binary-interface-of-a-library/
--
David Marchand
^ permalink raw reply [relevance 0%]
* [dpdk-dev] DPDK Release Status Meeting 6/02/2020
@ 2020-02-06 10:32 4% Ferruh Yigit
0 siblings, 0 replies; 200+ results
From: Ferruh Yigit @ 2020-02-06 10:32 UTC (permalink / raw)
To: dpdk-dev; +Cc: Thomas Monjalon
Minutes 6 February 2020
-----------------------
Agenda:
* Release Dates
* Subtrees
* OvS
Participants:
* Intel
* Marvell
* Mellanox
* NXP
* Red Hat
Release Dates
-------------
* v20.02 dates:
* -rc2 pushed to *Thursday 6 February 2020*
* -rc3 pushed to *Friday 14 February 2020*
* Release pushed to *Friday 21 February 2020*
* Release pushed because of extended PRC holidays
* v20.05 dates:
* Proposal/V1: Friday 6 March 2020
* Integration/Merge/RC1: Friday 10 April 2020
* Release: Friday 1 May 2020
Subtrees
--------
* main
* Pulled next-net & next-net-crypto
* Dropped two patches from crypto because of ABI breakages
* ccp sha3 & chacha20-poly
* ABI check patches merged
* In travis checks enabled by default
* next-net
* pulled, nothing in backlog for -rc3, there can be fixes
* next-net-crypto
* ipsecgw eventmode can be postponed, Akhil will check
* CPU crypto patches merged
* next-net-eventdev
* There can be a few fixes for -rc3
* next-net-virtio
* Three small fixes for -rc3
* next-net-mlx
* pulled
* LTS
* 17.11.10-rc1 released, please test and report results
* https://mails.dpdk.org/archives/dev/2020-January/154915.html
* Got test results from Intel, Red Hat & Mellanox
* QAT regression reported by Intel testing, need help from developers
* mlx reported regressions, needs more details
* Will wait two more week for above issues clarified
* Rough release date is mid February
* 18.11.6 released
* https://mails.dpdk.org/archives/dev/2020-January/156760.html
OvS
---
* Release on track
* Backporting removing experimental tag under discussion in DPDK
* https://mails.dpdk.org/archives/dev/2020-January/155419.html
DPDK Release Status Meetings
============================
The DPDK Release Status Meeting is intended for DPDK Committers to discuss
the status of the master tree and sub-trees, and for project managers to
track progress or milestone dates.
The meeting occurs on Thursdays at 8:30 UTC. If you wish to attend just
send an email to "John McNamara <john.mcnamara@intel.com>" for the invite.
^ permalink raw reply [relevance 4%]
* [dpdk-dev] [PATCH v1] common/iavf: update the ABI version
@ 2020-02-06 12:04 7% Haiyue Wang
2020-02-11 2:08 4% ` Ye Xiaolong
0 siblings, 1 reply; 200+ results
From: Haiyue Wang @ 2020-02-06 12:04 UTC (permalink / raw)
To: dev, xiaolong.ye; +Cc: thomas, Haiyue Wang
The new symbols should be in ABI version 20.0.1.
Fixes: 89214fe915b8 ("net/iavf/base: move to drivers common directory")
Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
---
drivers/common/iavf/rte_common_iavf_version.map | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/common/iavf/rte_common_iavf_version.map b/drivers/common/iavf/rte_common_iavf_version.map
index 1a0839262..2f11d67c0 100644
--- a/drivers/common/iavf/rte_common_iavf_version.map
+++ b/drivers/common/iavf/rte_common_iavf_version.map
@@ -1,4 +1,4 @@
-DPDK_20.0 {
+DPDK_20.0.1 {
global:
iavf_init_adminq;
--
2.17.1
^ permalink raw reply [relevance 7%]
* Re: [dpdk-dev] [PATCH v7 1/2] eal: add API to check if its interrupt context
@ 2020-02-06 15:35 3% ` David Marchand
0 siblings, 0 replies; 200+ results
From: David Marchand @ 2020-02-06 15:35 UTC (permalink / raw)
To: Sunil Kumar Kori; +Cc: Bruce Richardson, dev, Harman Kalra
On Tue, Jan 14, 2020 at 10:05 AM Sunil Kumar Kori <skori@marvell.com> wrote:
>
> From: Harman Kalra <hkalra@marvell.com>
>
> Added an API to check if current execution is in interrupt
> context. This will be helpful to handle nested interrupt cases.
>
> Signed-off-by: Harman Kalra <hkalra@marvell.com>
> Signed-off-by: Sunil Kumar Kori <skori@marvell.com>
> Reviewed-by: Jerin Jacob <jerinj@marvell.com>
> ---
> v7:
> - No changes.
> v6:
> - No changes.
> v5:
> - Fix shared library compilation error
> v4:
> - No changes.
> v3:
> - API Comment is updated as per man page.
> - Scope updated within the library/driver only.
> - Remove experimental tag
> v2:
> - Rebased patch on 19.11-rc4
>
> lib/librte_eal/common/include/rte_eal_interrupts.h | 11 +++++++++++
> lib/librte_eal/freebsd/eal/eal_interrupts.c | 5 +++++
> lib/librte_eal/linux/eal/eal_interrupts.c | 5 +++++
> lib/librte_eal/rte_eal_version.map | 1 +
> 4 files changed, 22 insertions(+)
>
> diff --git a/lib/librte_eal/common/include/rte_eal_interrupts.h b/lib/librte_eal/common/include/rte_eal_interrupts.h
> index b370c0d26..19d1d45ab 100644
> --- a/lib/librte_eal/common/include/rte_eal_interrupts.h
> +++ b/lib/librte_eal/common/include/rte_eal_interrupts.h
> @@ -220,4 +220,15 @@ rte_intr_allow_others(struct rte_intr_handle *intr_handle);
> int
> rte_intr_cap_multiple(struct rte_intr_handle *intr_handle);
>
> +/**
> + * @internal
> + * Check if currently executing in interrupt context
> + *
> + * @return
> + * - non zero in case of interrupt context
> + * - zero in case of process context
> + */
> +int
> +rte_thread_is_intr(void);
> +
> #endif /* _RTE_EAL_INTERRUPTS_H_ */
> diff --git a/lib/librte_eal/freebsd/eal/eal_interrupts.c b/lib/librte_eal/freebsd/eal/eal_interrupts.c
> index f6831b790..ce2a27b4a 100644
> --- a/lib/librte_eal/freebsd/eal/eal_interrupts.c
> +++ b/lib/librte_eal/freebsd/eal/eal_interrupts.c
> @@ -671,3 +671,8 @@ rte_intr_free_epoll_fd(struct rte_intr_handle *intr_handle)
> {
> RTE_SET_USED(intr_handle);
> }
> +
> +int rte_thread_is_intr(void)
> +{
> + return pthread_equal(intr_thread, pthread_self());
> +}
> diff --git a/lib/librte_eal/linux/eal/eal_interrupts.c b/lib/librte_eal/linux/eal/eal_interrupts.c
> index 14ebb108c..cb8e10709 100644
> --- a/lib/librte_eal/linux/eal/eal_interrupts.c
> +++ b/lib/librte_eal/linux/eal/eal_interrupts.c
> @@ -1488,3 +1488,8 @@ rte_intr_cap_multiple(struct rte_intr_handle *intr_handle)
>
> return 0;
> }
> +
> +int rte_thread_is_intr(void)
> +{
> + return pthread_equal(intr_thread, pthread_self());
> +}
> diff --git a/lib/librte_eal/rte_eal_version.map b/lib/librte_eal/rte_eal_version.map
> index e38d02530..06ed2e9dc 100644
> --- a/lib/librte_eal/rte_eal_version.map
> +++ b/lib/librte_eal/rte_eal_version.map
> @@ -91,6 +91,7 @@ DPDK_20.0 {
> rte_intr_free_epoll_fd;
> rte_intr_rx_ctl;
> rte_intr_tls_epfd;
> + rte_thread_is_intr;
> rte_keepalive_create;
> rte_keepalive_dispatch_pings;
> rte_keepalive_mark_alive;
> --
> 2.17.1
We can't put this symbol in the stable ABI and just flag it as
internal in the function description.
Users won't notice this warning.
I moved this as an experimental API and enabled the use of
experimental api in octeontx2 common driver.
We can revisit once we have the rte_internal tag Neil had proposed.
Series applied, thanks.
--
David Marchand
^ permalink raw reply [relevance 3%]
* [dpdk-dev] [dpdk-announce] release candidate 20.02-rc2
@ 2020-02-06 22:33 4% Thomas Monjalon
0 siblings, 0 replies; 200+ results
From: Thomas Monjalon @ 2020-02-06 22:33 UTC (permalink / raw)
To: announce
A new DPDK release candidate is ready for testing:
https://git.dpdk.org/dpdk/tag/?id=v20.02-rc2
226 patches were integrated.
The release notes so far:
http://doc.dpdk.org/guides/rel_notes/release_20_02.html
Highlights of 20.02-rc2:
- ABI check tooling
- Mellanox vDPA driver
- Marvell OCTEON TX2 inline IPsec
- synchronous CPU crypto API
- CPU crypto based on new libraries (intel-ipsec-mb and AArch64cryptolib)
- reverted Chacha20-Poly1305 algorithm (because of ABI issue)
- event mode in l3fwd example
The -rc3 should include only some bug fixes, simple cleanups, doc
and tooling. We have one week to complete this milestone.
Then one more week (allowing -rc4) should be needed before the release.
The schedule is shifting a bit to let more time to Chinese contributors.
Thank you everyone
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH v2] service: don't walk out of bounds when checking services
@ 2020-02-07 14:27 3% ` Aaron Conole
0 siblings, 0 replies; 200+ results
From: Aaron Conole @ 2020-02-07 14:27 UTC (permalink / raw)
To: Kevin Traynor
Cc: Harry van Haaren, nikhil.rao, David Marchand, dev,
Bruce Richardson, Pavan Nikhilesh, Gage Eads, Thomas Monjalon,
dpdk stable
Kevin Traynor <ktraynor@redhat.com> writes:
> On 20/12/2019 14:43, David Marchand wrote:
>> On Wed, Dec 4, 2019 at 9:34 AM David Marchand <david.marchand@redhat.com> wrote:
>>>
>>> On Wed, Dec 4, 2019 at 9:33 AM David Marchand <david.marchand@redhat.com> wrote:
>>>>
>>>> On Tue, Dec 3, 2019 at 10:15 PM Aaron Conole <aconole@redhat.com> wrote:
>>>>>
>>>>> The service_valid call is used without properly bounds checking the
>>>>> input parameter. Almost all instances of the service_valid call are
>>>>> inside a for() loop that prevents excessive walks, but some of the
>>>>> public APIs don't bounds check and will pass invalid arguments.
>>>>>
>>>>> Prevent this by using SERVICE_GET_OR_ERR_RET where it makes sense,
>>>>> and adding a bounds check to one service_valid() use.
>>>>>
>>>>> Fixes: 8d39d3e237c2 ("service: fix race in service on app lcore function")
>>>>> Fixes: e9139a32f6e8 ("service: add function to run on app lcore")
>>>>> Fixes: e30dd31847d2 ("service: add mechanism for quiescing")
>>> Cc: stable@dpdk.org
>>>
>
> With the commit below, this patch will apply cleanly on 18.11.
>
> Seems ok to me to add below commit, wdyt?
If I'm reading it correctly, the move is for an internal data structure
in librte_eal, so I think it shouldn't be an ABI breakage.
Looks safe to me as well.
> commit e484ccddbe1b41886fef1e445ef2fdfa55086198
> Author: Nikhil Rao <nikhil.rao@intel.com>
> Date: Mon Sep 16 15:31:02 2019 +0530
>
> service: avoid false sharing on core state
>
>
>>>>> Signed-off-by: Aaron Conole <aconole@redhat.com>
>>>>
>>>> Reviewed-by: David Marchand <david.marchand@redhat.com>
>>
>> Applied, thanks.
>>
>>
>> --
>> David Marchand
>>
^ permalink raw reply [relevance 3%]
* [dpdk-dev] [PATCH] common/octeontx2: update version map
@ 2020-02-10 10:27 3% Anoob Joseph
2020-02-11 14:04 0% ` David Marchand
0 siblings, 1 reply; 200+ results
From: Anoob Joseph @ 2020-02-10 10:27 UTC (permalink / raw)
To: Akhil Goyal, Jerin Jacob, Thomas Monjalon
Cc: Anoob Joseph, Narayana Prasad, dev
Move symbols added in 20.02 release to DPDK_20.0.1 ABI.
Fixes: d06551535a09 ("common/octeontx2: add security capability routine")
Fixes: 3fe4d07d1678 ("crypto/octeontx2: enable CPT to share QP with ethdev")
Fixes: f44e71637755 ("net/octeontx2: add security session operations")
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
---
.../common/octeontx2/rte_common_octeontx2_version.map | 17 +++++++++++------
1 file changed, 11 insertions(+), 6 deletions(-)
diff --git a/drivers/common/octeontx2/rte_common_octeontx2_version.map b/drivers/common/octeontx2/rte_common_octeontx2_version.map
index 19a7b19..a25fc69 100644
--- a/drivers/common/octeontx2/rte_common_octeontx2_version.map
+++ b/drivers/common/octeontx2/rte_common_octeontx2_version.map
@@ -5,7 +5,6 @@ DPDK_20.0 {
otx2_dev_fini;
otx2_dev_priv_init;
otx2_disable_irqs;
- otx2_eth_dev_is_sec_capable;
otx2_intra_dev_get_cfg;
otx2_logtype_base;
otx2_logtype_dpi;
@@ -28,11 +27,6 @@ DPDK_20.0 {
otx2_npa_pf_func_get;
otx2_npa_set_defaults;
otx2_register_irq;
- otx2_sec_idev_cfg_init;
- otx2_sec_idev_tx_cpt_qp_add;
- otx2_sec_idev_tx_cpt_qp_remove;
- otx2_sec_idev_tx_cpt_qp_get;
- otx2_sec_idev_tx_cpt_qp_put;
otx2_sso_pf_func_get;
otx2_sso_pf_func_set;
otx2_unregister_irq;
@@ -40,6 +34,17 @@ DPDK_20.0 {
local: *;
};
+DPDK_20.0.1 {
+ global:
+
+ otx2_eth_dev_is_sec_capable;
+ otx2_sec_idev_cfg_init;
+ otx2_sec_idev_tx_cpt_qp_add;
+ otx2_sec_idev_tx_cpt_qp_remove;
+ otx2_sec_idev_tx_cpt_qp_get;
+ otx2_sec_idev_tx_cpt_qp_put;
+};
+
EXPERIMENTAL {
global:
--
2.7.4
^ permalink raw reply [relevance 3%]
* Re: [dpdk-dev] [PATCH v7] eal: add manual probing option
2020-02-04 16:02 0% ` Gaetan Rivet
@ 2020-02-10 14:51 0% ` Jerin Jacob
2020-02-10 15:27 0% ` Thomas Monjalon
0 siblings, 1 reply; 200+ results
From: Jerin Jacob @ 2020-02-10 14:51 UTC (permalink / raw)
To: Gaetan Rivet
Cc: Thomas Monjalon, Pavan Nikhilesh Bhagavatula,
Vamsi Krishna Attunuru, David Marchand, dpdk-dev,
Jerin Jacob Kollanukkaran, Richardson, Bruce
On Tue, Feb 4, 2020 at 9:32 PM Gaetan Rivet <grive@u256.net> wrote:
>
> On 04/02/2020 16:06, Thomas Monjalon wrote:
> > 04/02/2020 13:43, Gaetan Rivet:
> >> On 04/02/2020 12:07, Thomas Monjalon wrote:
> >>> 04/02/2020 11:03, Gaetan Rivet:
> >>>> On 03/02/2020 23:21, Thomas Monjalon wrote:
> >>>>> 03/02/2020 06:16, Pavan Nikhilesh Bhagavatula:
> >>>>>> @David Marchand @thomas@monjalon.net
> >>>>>>
> >>>>>> Ping?
> >>>>>>
> >>>>>> Are there any more changes required for this patch? It's been in queue since last October.
> >>>>>
> >>>>> Sorry we have not decided whether it is a good idea or not.
> >>>>>
> >>>>> All changes related to probing are very sensitive,
> >>>>> and we know a big refactoring would be better than stacking
> >>>>> more and more options and corner cases.
> >>>>>
> >>>>> As we are busy with ABI stability stuff, we did not allocate
> >>>>> enough time to properly think about this feature.
> >>>>> Please accept our apologies, and let's consider it as
> >>>>> a high priority for 20.05 cycle.
> >>>>>
> >>>>
> >>>> Hello Thomas,
> >>>>
> >>>> This is unfortunate. I pushed Pavan to accept an alternative implementation of this functionality that was less obtrusive, to make the integration smoother. I took care to alleviate those risks from the common path.
> >>>>
> >>>> The big refactoring is needed yes, but considering the current path I'm not seeing it happen in 20.05. If that means taking this patch as-is in 20.05 for Marvell users, I'm not sure much is gained from waiting 3 months, except minimal risk avoidance.
> >>>
> >>>
> >>> Yes, life is full of bad decisions and consequences.
> >>
> >>
> >> Ah, yes, but I stand by my initial opinion, the first implementation [1] was riskier and less useful.
> >>
> >>>
> >>> I still think there is a risk in adding new user expectations,
> >>> and maintaining some code to workaround unknown issues.
> >>>
> >>> The real question here is to know why this patch?
> >>> Is it to workaround a broken driver?
> >>> Or to workaround a broken design in EAL and bus drivers?
> >>
> >> Two birds - one stone here: OVS needed a way to disable automatic probing cleanly (current workaround seen in multiple deployment is to add a dummy whitelisted device, which will be ignored by the PCI bus --> it sets the bus in whitelist mode but avoid probing anything), and as a bonus this option allows using devices that depends on other devices being probed already (LAG, representors, failsafe, etc).
> >>
> >> I'm not sure having a dependent-probe by default is good, and that would be a big change.
> >>
> >> If we are doing the genesis of this patch, the initial motivation should be asked for more details from Marvell people and David for the OVS side.
> >>
> >> [1]: First proposal:
> >> http://mails.dpdk.org/archives/dev/2019-September/144166.html
> >> My arguments:
> >> http://mails.dpdk.org/archives/dev/2019-September/144564.html
> >
> >
> > OK so there are two needs:
> >
> > 1/ Better control whitelist/blacklist mode.
> > We already know that a rework is needed here.
> > Unfortunately neither you or me had time to work on it,
> > and others who were interested disappeared.
> >
> > 2/ Associate ports with equivalent properties in applications.
> > This must be done in applications.
> > Tweaking the probe order is a hack.
> >
> >
>
> An application that want to tightly control the port init order, currently (by doing exactly like me here, hotpluging one by one the ports), would still need the bigger hack that consist in inserting a whitelist PCI devargs with a dummy address, depending on a undocumented PCI bus feature consisting in ignoring matching errors but keeping probing policy from failed devargs processing.
>
> Instead, with this patch this app can do
>
> rte_eal_manual_probe_set(1);
> rte_eal_init();
>
> to have the same behavior and be able to hotplug ports as it sees fit.
>
> You are worried about creating user expectations about this behavior (being forced to replicate in some way the functionality during the rewrite, as I understand it?), but then you are currently forcing users to expect this workaround to exist in the PCI bus, blocking devs from touching it as it will thus break current app configurations. I've seen systemd unit file using this -w dummy flag, as well as the programmatic equivalent. Which is better, to have to rework it cutting short these configs, or to propose beforehand a deprecation path?.
>
> This rework won't happen in 20.05, nor in the medium term unless you decide to drive this change. This workaround serves three needs (PCI normalization, port congruence and port dependency) in a low-risk implementation.
Thomas,
What would be the resolution of this? What is your recommendation to
fix the issue as you have the concern of this patch?
Issue:
1) When l2fwd does the forwarding for simplicity and performance
reason it just xor the port to find the destination port to forward.
2) If the adjacent ports are not symmetrical(example: one is 40G and
other 10G) then forwarding will drop the packets.
So, either
a) We need to control the probing order
b) Or Application need
1) To track the symmetrical ports and maintain the forwarding table OR
2) Have the command-line option to specify destination port like l3fwd.
We can fix it in the application, but do we need to complicate l2fwd?
I am fine with that, if that is consensus.
Thoughts? If you think, there is a rework needed in eal then could you
enumerate the items for the rework.
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH v7] eal: add manual probing option
2020-02-10 14:51 0% ` Jerin Jacob
@ 2020-02-10 15:27 0% ` Thomas Monjalon
2020-02-10 16:33 0% ` Jerin Jacob
0 siblings, 1 reply; 200+ results
From: Thomas Monjalon @ 2020-02-10 15:27 UTC (permalink / raw)
To: Jerin Jacob, Jerin Jacob Kollanukkaran
Cc: Gaetan Rivet, Pavan Nikhilesh Bhagavatula,
Vamsi Krishna Attunuru, David Marchand, dpdk-dev, Richardson,
Bruce
10/02/2020 15:51, Jerin Jacob:
> On Tue, Feb 4, 2020 at 9:32 PM Gaetan Rivet <grive@u256.net> wrote:
> >
> > On 04/02/2020 16:06, Thomas Monjalon wrote:
> > > 04/02/2020 13:43, Gaetan Rivet:
> > >> On 04/02/2020 12:07, Thomas Monjalon wrote:
> > >>> 04/02/2020 11:03, Gaetan Rivet:
> > >>>> On 03/02/2020 23:21, Thomas Monjalon wrote:
> > >>>>> 03/02/2020 06:16, Pavan Nikhilesh Bhagavatula:
> > >>>>>> @David Marchand @thomas@monjalon.net
> > >>>>>>
> > >>>>>> Ping?
> > >>>>>>
> > >>>>>> Are there any more changes required for this patch? It's been in queue since last October.
> > >>>>>
> > >>>>> Sorry we have not decided whether it is a good idea or not.
> > >>>>>
> > >>>>> All changes related to probing are very sensitive,
> > >>>>> and we know a big refactoring would be better than stacking
> > >>>>> more and more options and corner cases.
> > >>>>>
> > >>>>> As we are busy with ABI stability stuff, we did not allocate
> > >>>>> enough time to properly think about this feature.
> > >>>>> Please accept our apologies, and let's consider it as
> > >>>>> a high priority for 20.05 cycle.
> > >>>>>
> > >>>>
> > >>>> Hello Thomas,
> > >>>>
> > >>>> This is unfortunate. I pushed Pavan to accept an alternative implementation of this functionality that was less obtrusive, to make the integration smoother. I took care to alleviate those risks from the common path.
> > >>>>
> > >>>> The big refactoring is needed yes, but considering the current path I'm not seeing it happen in 20.05. If that means taking this patch as-is in 20.05 for Marvell users, I'm not sure much is gained from waiting 3 months, except minimal risk avoidance.
> > >>>
> > >>>
> > >>> Yes, life is full of bad decisions and consequences.
> > >>
> > >>
> > >> Ah, yes, but I stand by my initial opinion, the first implementation [1] was riskier and less useful.
> > >>
> > >>>
> > >>> I still think there is a risk in adding new user expectations,
> > >>> and maintaining some code to workaround unknown issues.
> > >>>
> > >>> The real question here is to know why this patch?
> > >>> Is it to workaround a broken driver?
> > >>> Or to workaround a broken design in EAL and bus drivers?
> > >>
> > >> Two birds - one stone here: OVS needed a way to disable automatic probing cleanly (current workaround seen in multiple deployment is to add a dummy whitelisted device, which will be ignored by the PCI bus --> it sets the bus in whitelist mode but avoid probing anything), and as a bonus this option allows using devices that depends on other devices being probed already (LAG, representors, failsafe, etc).
> > >>
> > >> I'm not sure having a dependent-probe by default is good, and that would be a big change.
> > >>
> > >> If we are doing the genesis of this patch, the initial motivation should be asked for more details from Marvell people and David for the OVS side.
> > >>
> > >> [1]: First proposal:
> > >> http://mails.dpdk.org/archives/dev/2019-September/144166.html
> > >> My arguments:
> > >> http://mails.dpdk.org/archives/dev/2019-September/144564.html
> > >
> > >
> > > OK so there are two needs:
> > >
> > > 1/ Better control whitelist/blacklist mode.
> > > We already know that a rework is needed here.
> > > Unfortunately neither you or me had time to work on it,
> > > and others who were interested disappeared.
> > >
> > > 2/ Associate ports with equivalent properties in applications.
> > > This must be done in applications.
> > > Tweaking the probe order is a hack.
> > >
> > >
> >
> > An application that want to tightly control the port init order, currently (by doing exactly like me here, hotpluging one by one the ports), would still need the bigger hack that consist in inserting a whitelist PCI devargs with a dummy address, depending on a undocumented PCI bus feature consisting in ignoring matching errors but keeping probing policy from failed devargs processing.
> >
> > Instead, with this patch this app can do
> >
> > rte_eal_manual_probe_set(1);
> > rte_eal_init();
> >
> > to have the same behavior and be able to hotplug ports as it sees fit.
> >
> > You are worried about creating user expectations about this behavior (being forced to replicate in some way the functionality during the rewrite, as I understand it?), but then you are currently forcing users to expect this workaround to exist in the PCI bus, blocking devs from touching it as it will thus break current app configurations. I've seen systemd unit file using this -w dummy flag, as well as the programmatic equivalent. Which is better, to have to rework it cutting short these configs, or to propose beforehand a deprecation path?.
> >
> > This rework won't happen in 20.05, nor in the medium term unless you decide to drive this change. This workaround serves three needs (PCI normalization, port congruence and port dependency) in a low-risk implementation.
>
> Thomas,
>
> What would be the resolution of this? What is your recommendation to
> fix the issue as you have the concern of this patch?
>
> Issue:
> 1) When l2fwd does the forwarding for simplicity and performance
> reason it just xor the port to find the destination port to forward.
> 2) If the adjacent ports are not symmetrical(example: one is 40G and
> other 10G) then forwarding will drop the packets.
>
> So, either
> a) We need to control the probing order
>
> b) Or Application need
> 1) To track the symmetrical ports and maintain the forwarding table OR
> 2) Have the command-line option to specify destination port like l3fwd.
>
> We can fix it in the application, but do we need to complicate l2fwd?
> I am fine with that, if that is consensus.
You are describing an application issue,
that's why I believe it should be fixed in the application.
Should we have a command line option to configure the forwarding rules
in the application (2)? I think yes.
Should we implement an application logic to automatically create
the best forwarding rules (1)? It would be nice, but anyway,
I think we need manual config (2) as a fallback.
> Thoughts? If you think, there is a rework needed in eal then could you
> enumerate the items for the rework.
Sorry I don't have time to describe dive into EAL probing and
enumerate the items to rework.
The most important issues I remind are:
- white/blacklist policy is a mess and should be done in a higher layer
- devargs syntax should allow generic matching (thanks to class awareness and generic syntax)
Starting from these 2 items, we could imagine a generic path to
disable automatic probing, but I think the l2fwd logic should not
rely on probing order anyway.
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH v7] eal: add manual probing option
2020-02-10 15:27 0% ` Thomas Monjalon
@ 2020-02-10 16:33 0% ` Jerin Jacob
0 siblings, 0 replies; 200+ results
From: Jerin Jacob @ 2020-02-10 16:33 UTC (permalink / raw)
To: Thomas Monjalon
Cc: Jerin Jacob Kollanukkaran, Gaetan Rivet,
Pavan Nikhilesh Bhagavatula, Vamsi Krishna Attunuru,
David Marchand, dpdk-dev, Richardson, Bruce
On Mon, Feb 10, 2020 at 8:57 PM Thomas Monjalon <thomas@monjalon.net> wrote:
>
> 10/02/2020 15:51, Jerin Jacob:
> > On Tue, Feb 4, 2020 at 9:32 PM Gaetan Rivet <grive@u256.net> wrote:
> > >
> > > On 04/02/2020 16:06, Thomas Monjalon wrote:
> > > > 04/02/2020 13:43, Gaetan Rivet:
> > > >> On 04/02/2020 12:07, Thomas Monjalon wrote:
> > > >>> 04/02/2020 11:03, Gaetan Rivet:
> > > >>>> On 03/02/2020 23:21, Thomas Monjalon wrote:
> > > >>>>> 03/02/2020 06:16, Pavan Nikhilesh Bhagavatula:
> > > >>>>>> @David Marchand @thomas@monjalon.net
> > > >>>>>>
> > > >>>>>> Ping?
> > > >>>>>>
> > > >>>>>> Are there any more changes required for this patch? It's been in queue since last October.
> > > >>>>>
> > > >>>>> Sorry we have not decided whether it is a good idea or not.
> > > >>>>>
> > > >>>>> All changes related to probing are very sensitive,
> > > >>>>> and we know a big refactoring would be better than stacking
> > > >>>>> more and more options and corner cases.
> > > >>>>>
> > > >>>>> As we are busy with ABI stability stuff, we did not allocate
> > > >>>>> enough time to properly think about this feature.
> > > >>>>> Please accept our apologies, and let's consider it as
> > > >>>>> a high priority for 20.05 cycle.
> > > >>>>>
> > > >>>>
> > > >>>> Hello Thomas,
> > > >>>>
> > > >>>> This is unfortunate. I pushed Pavan to accept an alternative implementation of this functionality that was less obtrusive, to make the integration smoother. I took care to alleviate those risks from the common path.
> > > >>>>
> > > >>>> The big refactoring is needed yes, but considering the current path I'm not seeing it happen in 20.05. If that means taking this patch as-is in 20.05 for Marvell users, I'm not sure much is gained from waiting 3 months, except minimal risk avoidance.
> > > >>>
> > > >>>
> > > >>> Yes, life is full of bad decisions and consequences.
> > > >>
> > > >>
> > > >> Ah, yes, but I stand by my initial opinion, the first implementation [1] was riskier and less useful.
> > > >>
> > > >>>
> > > >>> I still think there is a risk in adding new user expectations,
> > > >>> and maintaining some code to workaround unknown issues.
> > > >>>
> > > >>> The real question here is to know why this patch?
> > > >>> Is it to workaround a broken driver?
> > > >>> Or to workaround a broken design in EAL and bus drivers?
> > > >>
> > > >> Two birds - one stone here: OVS needed a way to disable automatic probing cleanly (current workaround seen in multiple deployment is to add a dummy whitelisted device, which will be ignored by the PCI bus --> it sets the bus in whitelist mode but avoid probing anything), and as a bonus this option allows using devices that depends on other devices being probed already (LAG, representors, failsafe, etc).
> > > >>
> > > >> I'm not sure having a dependent-probe by default is good, and that would be a big change.
> > > >>
> > > >> If we are doing the genesis of this patch, the initial motivation should be asked for more details from Marvell people and David for the OVS side.
> > > >>
> > > >> [1]: First proposal:
> > > >> http://mails.dpdk.org/archives/dev/2019-September/144166.html
> > > >> My arguments:
> > > >> http://mails.dpdk.org/archives/dev/2019-September/144564.html
> > > >
> > > >
> > > > OK so there are two needs:
> > > >
> > > > 1/ Better control whitelist/blacklist mode.
> > > > We already know that a rework is needed here.
> > > > Unfortunately neither you or me had time to work on it,
> > > > and others who were interested disappeared.
> > > >
> > > > 2/ Associate ports with equivalent properties in applications.
> > > > This must be done in applications.
> > > > Tweaking the probe order is a hack.
> > > >
> > > >
> > >
> > > An application that want to tightly control the port init order, currently (by doing exactly like me here, hotpluging one by one the ports), would still need the bigger hack that consist in inserting a whitelist PCI devargs with a dummy address, depending on a undocumented PCI bus feature consisting in ignoring matching errors but keeping probing policy from failed devargs processing.
> > >
> > > Instead, with this patch this app can do
> > >
> > > rte_eal_manual_probe_set(1);
> > > rte_eal_init();
> > >
> > > to have the same behavior and be able to hotplug ports as it sees fit.
> > >
> > > You are worried about creating user expectations about this behavior (being forced to replicate in some way the functionality during the rewrite, as I understand it?), but then you are currently forcing users to expect this workaround to exist in the PCI bus, blocking devs from touching it as it will thus break current app configurations. I've seen systemd unit file using this -w dummy flag, as well as the programmatic equivalent. Which is better, to have to rework it cutting short these configs, or to propose beforehand a deprecation path?.
> > >
> > > This rework won't happen in 20.05, nor in the medium term unless you decide to drive this change. This workaround serves three needs (PCI normalization, port congruence and port dependency) in a low-risk implementation.
> >
> > Thomas,
> >
> > What would be the resolution of this? What is your recommendation to
> > fix the issue as you have the concern of this patch?
> >
> > Issue:
> > 1) When l2fwd does the forwarding for simplicity and performance
> > reason it just xor the port to find the destination port to forward.
> > 2) If the adjacent ports are not symmetrical(example: one is 40G and
> > other 10G) then forwarding will drop the packets.
> >
> > So, either
> > a) We need to control the probing order
> >
> > b) Or Application need
> > 1) To track the symmetrical ports and maintain the forwarding table OR
> > 2) Have the command-line option to specify destination port like l3fwd.
> >
> > We can fix it in the application, but do we need to complicate l2fwd?
> > I am fine with that, if that is consensus.
>
> You are describing an application issue,
> that's why I believe it should be fixed in the application.
Thanks for the quick reply and I agree.
>
> Should we have a command line option to configure the forwarding rules
> in the application (2)? I think yes.
> Should we implement an application logic to automatically create
> the best forwarding rules (1)? It would be nice, but anyway,
> I think we need manual config (2) as a fallback.
>
>
> > Thoughts? If you think, there is a rework needed in eal then could you
> > enumerate the items for the rework.
>
> Sorry I don't have time to describe dive into EAL probing and
> enumerate the items to rework.
> The most important issues I remind are:
> - white/blacklist policy is a mess and should be done in a higher layer
> - devargs syntax should allow generic matching (thanks to class awareness and generic syntax)
>
> Starting from these 2 items, we could imagine a generic path to
> disable automatic probing, but I think the l2fwd logic should not
> rely on probing order anyway.
+ Bruce as l2fwd maintainer.
Since in any case, l2fwd needs to be updated, We will focus on l2fwd
change for v20.05 and leaving the fate of this patch to EAL
maintainers.
Let us know, Are we are OK with below change in l2fwd as
- Introduce an array-based port lookup table instead of hardcoding to
xor based lookup.
- if no argument specified fill dest port as xor of source
- If argument is specified override the lookup table with a
user-specified destination port.
>
>
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH v1] common/iavf: update the ABI version
2020-02-06 12:04 7% [dpdk-dev] [PATCH v1] common/iavf: update the ABI version Haiyue Wang
@ 2020-02-11 2:08 4% ` Ye Xiaolong
0 siblings, 0 replies; 200+ results
From: Ye Xiaolong @ 2020-02-11 2:08 UTC (permalink / raw)
To: Haiyue Wang; +Cc: dev, thomas
On 02/06, Haiyue Wang wrote:
>The new symbols should be in ABI version 20.0.1.
>
>Fixes: 89214fe915b8 ("net/iavf/base: move to drivers common directory")
>
>Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
>---
> drivers/common/iavf/rte_common_iavf_version.map | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
>diff --git a/drivers/common/iavf/rte_common_iavf_version.map b/drivers/common/iavf/rte_common_iavf_version.map
>index 1a0839262..2f11d67c0 100644
>--- a/drivers/common/iavf/rte_common_iavf_version.map
>+++ b/drivers/common/iavf/rte_common_iavf_version.map
>@@ -1,4 +1,4 @@
>-DPDK_20.0 {
>+DPDK_20.0.1 {
> global:
>
> iavf_init_adminq;
>--
>2.17.1
>
Acked-by: Xiaolong Ye <xiaolong.ye@intel.com>
Applied to dpdk-next-net-intel, Thanks.
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [RFC PATCH 0/7] vfio/pci: SR-IOV support
2020-02-04 23:05 3% [dpdk-dev] [RFC PATCH 0/7] vfio/pci: SR-IOV support Alex Williamson
@ 2020-02-11 11:18 0% ` Jerin Jacob
0 siblings, 0 replies; 200+ results
From: Jerin Jacob @ 2020-02-11 11:18 UTC (permalink / raw)
To: Alex Williamson
Cc: kvm, linux-pci, linux-kernel, dpdk-dev, mtosatti,
Thomas Monjalon, Luca Boccassi, Richardson, Bruce, cohuck,
Vamsi Attunuru
On Wed, Feb 5, 2020 at 4:35 AM Alex Williamson
<alex.williamson@redhat.com> wrote:
>
> There seems to be an ongoing desire to use userspace, vfio-based
> drivers for both SR-IOV PF and VF devices. The fundamental issue
> with this concept is that the VF is not fully independent of the PF
> driver. Minimally the PF driver might be able to deny service to the
> VF, VF data paths might be dependent on the state of the PF device,
> or the PF my have some degree of ability to inspect or manipulate the
> VF data. It therefore would seem irresponsible to unleash VFs onto
> the system, managed by a user owned PF.
>
> We address this in a few ways in this series. First, we can use a bus
> notifier and the driver_override facility to make sure VFs are bound
> to the vfio-pci driver by default. This should eliminate the chance
> that a VF is accidentally bound and used by host drivers. We don't
> however remove the ability for a host admin to change this override.
>
> The next issue we need to address is how we let userspace drivers
> opt-in to this participation with the PF driver. We do not want an
> admin to be able to unwittingly assign one of these VFs to a tenant
> that isn't working in collaboration with the PF driver. We could use
> IOMMU grouping, but this seems to push too far towards tightly coupled
> PF and VF drivers. This series introduces a "VF token", implemented
> as a UUID, as a shared secret between PF and VF drivers. The token
> needs to be set by the PF driver and used as part of the device
> matching by the VF driver. Provisions in the code also account for
> restarting the PF driver with active VF drivers, requiring the PF to
> use the current token to re-gain access to the PF.
Thanks Alex for the series. DPDK realizes this use-case through, an out of
tree igb_uio module, for non VFIO devices. Supporting this use case, with
VFIO, will be a great enhancement for DPDK as we are planning to
get rid of out of tree modules any focus only on userspace aspects.
From the DPDK perspective, we have following use-cases
1) VF representer or OVS/vSwitch use cases where
DPDK PF acts as an HW switch to steer traffic to VF
using the rte_flow library backed by HW CAMs.
2) Unlike, other PCI class of devices, Network class of PCIe devices
would have additional
capability on the PF devices such as promiscuous mode support etc
leverage that in DPDK
PF and VF use cases.
That would boil down to the use of the following topology.
a) PF bound to DPDK/VFIO and VF bound to Linux
b) PF bound to DPDK/VFIO and VF bound to DPDK/VFIO
Tested the use case (a) and it works this patch. Tested use case(b), it
works with patch provided both PF and VF under the same application.
Regarding the use case where PF bound to DPDK/VFIO and
VF bound to DPDK/VFIO are _two different_ processes then sharing the UUID
will be a little tricky thing in terms of usage. But if that is the
purpose of bringing
UUID to the equation then it fine.
Overall this series looks good to me. We can test the next non-RFC
series and give
Tested-by by after testing with DPDK.
>
> The above solutions introduce a bit of a modification to the VFIO ABI
> and an additional ABI extension. The modification is that the
> VFIO_GROUP_GET_DEVICE_FD ioctl is specified to require a char string
> from the user providing the device name. For this solution, we extend
> the syntax to allow the device name followed by key/value pairs. In
> this case we add "vf_token=3e7e882e-1daf-417f-ad8d-882eea5ee337", for
> example. These options are expected to be space separated. Matching
> these key/value pairs is entirely left to the vfio bus driver (ex.
> vfio-pci) and the internal ops structure is extended to allow this
> optional support. This extension should be fully backwards compatible
> to existing userspace, such code will simply fail to open these newly
> exposed devices, as intended.
>
> I've been debating whether instead of the above we should allow the
> user to get the device fd as normal, but restrict the interfaces until
> the user authenticates, but I'm afraid this would be a less backwards
> compatible solution. It would be just as unclear to the user why a
> device read/write/mmap/ioctl failed as it might be to why getting the
> device fd could fail. However in the latter case, I believe we do a
> better job of restricting how far userspace code might go before they
> ultimately fail. I'd welcome discussion in the space, and or course
> the extension of the GET_DEVICE_FD string.
>
> Finally, the user needs to be able to set a VF token. I add a
> VFIO_DEVICE_FEATURE ioctl for this that's meant to be reusable for
> getting, setting, and probing arbitrary features of a device.
>
> I'll reply to this cover letter with a very basic example of a QEMU
> update to support this interface, though I haven't found a device yet
> that behaves well with the PF running in one VM with the VF in
> another, or really even just a PF running in a VM with SR-IOV enabled.
> I know these devices exist though, and I suspect QEMU will not be the
> primary user of this support for now, but this behavior reaffirms my
> concerns to prevent mis-use.
>
> Please comment. In particular, does this approach meet the DPDK needs
> for userspace PF and VF drivers, with the hopefully minor hurdle of
> sharing a token between drivers. The token is of course left to
> userspace how to manage, and might be static (and not very secret) for
> a given set of drivers. Thanks,
>
> Alex
>
> ---
>
> Alex Williamson (7):
> vfio: Include optional device match in vfio_device_ops callbacks
> vfio/pci: Implement match ops
> vfio/pci: Introduce VF token
> vfio: Introduce VFIO_DEVICE_FEATURE ioctl and first user
> vfio/pci: Add sriov_configure support
> vfio/pci: Remove dev_fmt definition
> vfio/pci: Cleanup .probe() exit paths
>
>
> drivers/vfio/pci/vfio_pci.c | 315 ++++++++++++++++++++++++++++++++---
> drivers/vfio/pci/vfio_pci_private.h | 10 +
> drivers/vfio/vfio.c | 19 ++
> include/linux/vfio.h | 3
> include/uapi/linux/vfio.h | 37 ++++
> 5 files changed, 356 insertions(+), 28 deletions(-)
>
^ permalink raw reply [relevance 0%]
* [dpdk-dev] [PATCH 1/1] raw/octeontx2_ep: update the ABI version
@ 2020-02-11 11:27 7% Mahipal Challa
2020-02-11 14:05 4% ` David Marchand
0 siblings, 1 reply; 200+ results
From: Mahipal Challa @ 2020-02-11 11:27 UTC (permalink / raw)
To: thomas, dev; +Cc: jerinj, pathreya, snilla, venkatn
Changed the ABI version to 20.0.1.
Fixes: 56d46d13f736 ("raw/octeontx2_ep: add build infra and device probe")
Signed-off-by: Mahipal Challa <mchalla@marvell.com>
---
drivers/raw/octeontx2_ep/rte_rawdev_octeontx2_ep_version.map | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/raw/octeontx2_ep/rte_rawdev_octeontx2_ep_version.map b/drivers/raw/octeontx2_ep/rte_rawdev_octeontx2_ep_version.map
index ff357af..bc8fd6d 100644
--- a/drivers/raw/octeontx2_ep/rte_rawdev_octeontx2_ep_version.map
+++ b/drivers/raw/octeontx2_ep/rte_rawdev_octeontx2_ep_version.map
@@ -1,4 +1,4 @@
-DPDK_20.02 {
+DPDK_20.0.1 {
local: *;
};
--
1.8.3.1
^ permalink raw reply [relevance 7%]
* Re: [dpdk-dev] [PATCH] common/octeontx2: update version map
2020-02-10 10:27 3% [dpdk-dev] [PATCH] common/octeontx2: update version map Anoob Joseph
@ 2020-02-11 14:04 0% ` David Marchand
0 siblings, 0 replies; 200+ results
From: David Marchand @ 2020-02-11 14:04 UTC (permalink / raw)
To: Anoob Joseph
Cc: Akhil Goyal, Jerin Jacob, Thomas Monjalon, Narayana Prasad, dev
On Mon, Feb 10, 2020 at 11:28 AM Anoob Joseph <anoobj@marvell.com> wrote:
>
> Move symbols added in 20.02 release to DPDK_20.0.1 ABI.
>
> Fixes: d06551535a09 ("common/octeontx2: add security capability routine")
> Fixes: 3fe4d07d1678 ("crypto/octeontx2: enable CPT to share QP with ethdev")
> Fixes: f44e71637755 ("net/octeontx2: add security session operations")
>
> Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Small fix below.
Acked-by: David Marchand <david.marchand@redhat.com>
> ---
> .../common/octeontx2/rte_common_octeontx2_version.map | 17 +++++++++++------
> 1 file changed, 11 insertions(+), 6 deletions(-)
>
> diff --git a/drivers/common/octeontx2/rte_common_octeontx2_version.map b/drivers/common/octeontx2/rte_common_octeontx2_version.map
> index 19a7b19..a25fc69 100644
> --- a/drivers/common/octeontx2/rte_common_octeontx2_version.map
> +++ b/drivers/common/octeontx2/rte_common_octeontx2_version.map
> @@ -5,7 +5,6 @@ DPDK_20.0 {
> otx2_dev_fini;
> otx2_dev_priv_init;
> otx2_disable_irqs;
> - otx2_eth_dev_is_sec_capable;
> otx2_intra_dev_get_cfg;
> otx2_logtype_base;
> otx2_logtype_dpi;
> @@ -28,11 +27,6 @@ DPDK_20.0 {
> otx2_npa_pf_func_get;
> otx2_npa_set_defaults;
> otx2_register_irq;
> - otx2_sec_idev_cfg_init;
> - otx2_sec_idev_tx_cpt_qp_add;
> - otx2_sec_idev_tx_cpt_qp_remove;
> - otx2_sec_idev_tx_cpt_qp_get;
> - otx2_sec_idev_tx_cpt_qp_put;
> otx2_sso_pf_func_get;
> otx2_sso_pf_func_set;
> otx2_unregister_irq;
> @@ -40,6 +34,17 @@ DPDK_20.0 {
> local: *;
> };
>
> +DPDK_20.0.1 {
> + global:
> +
> + otx2_eth_dev_is_sec_capable;
> + otx2_sec_idev_cfg_init;
> + otx2_sec_idev_tx_cpt_qp_add;
> + otx2_sec_idev_tx_cpt_qp_remove;
> + otx2_sec_idev_tx_cpt_qp_get;
> + otx2_sec_idev_tx_cpt_qp_put;
> +};
> +
DPDK_20.0.1 must inherit from DPDK_20.0 node.
I fixed it.
Applied, thanks.
--
David Marchand
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH 1/1] raw/octeontx2_ep: update the ABI version
2020-02-11 11:27 7% [dpdk-dev] [PATCH 1/1] raw/octeontx2_ep: update the ABI version Mahipal Challa
@ 2020-02-11 14:05 4% ` David Marchand
0 siblings, 0 replies; 200+ results
From: David Marchand @ 2020-02-11 14:05 UTC (permalink / raw)
To: Mahipal Challa
Cc: Thomas Monjalon, dev, Jerin Jacob Kollanukkaran, pathreya,
snilla, venkatn
On Tue, Feb 11, 2020 at 12:29 PM Mahipal Challa <mchalla@marvell.com> wrote:
>
> Changed the ABI version to 20.0.1.
>
> Fixes: 56d46d13f736 ("raw/octeontx2_ep: add build infra and device probe")
>
> Signed-off-by: Mahipal Challa <mchalla@marvell.com>
> ---
> drivers/raw/octeontx2_ep/rte_rawdev_octeontx2_ep_version.map | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/raw/octeontx2_ep/rte_rawdev_octeontx2_ep_version.map b/drivers/raw/octeontx2_ep/rte_rawdev_octeontx2_ep_version.map
> index ff357af..bc8fd6d 100644
> --- a/drivers/raw/octeontx2_ep/rte_rawdev_octeontx2_ep_version.map
> +++ b/drivers/raw/octeontx2_ep/rte_rawdev_octeontx2_ep_version.map
> @@ -1,4 +1,4 @@
> -DPDK_20.02 {
> +DPDK_20.0.1 {
>
> local: *;
> };
> --
> 1.8.3.1
>
Acked-by: David Marchand <david.marchand@redhat.com>
Applied, thanks.
--
David Marchand
^ permalink raw reply [relevance 4%]
* [dpdk-dev] [PATCH 2/2] vdpa/mlx5: fix ABI version
2020-02-11 19:48 7% [dpdk-dev] [PATCH 1/2] common/mlx5: fix " Matan Azrad
@ 2020-02-11 19:48 7% ` Matan Azrad
2020-02-11 20:31 4% ` Slava Ovsiienko
2020-02-11 20:31 4% ` [dpdk-dev] [PATCH 1/2] common/mlx5: " Slava Ovsiienko
2020-02-12 8:53 4% ` David Marchand
2 siblings, 1 reply; 200+ results
From: Matan Azrad @ 2020-02-11 19:48 UTC (permalink / raw)
To: dev; +Cc: Viacheslav Ovsiienko, Thomas Monjalon, David Marchand
Changed the ABI version to 20.0.1.
Fixes: 95276abaaf0a ("vdpa/mlx5: introduce Mellanox vDPA driver")
Signed-off-by: Matan Azrad <matan@mellanox.com>
---
drivers/vdpa/mlx5/rte_pmd_mlx5_vdpa_version.map | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/vdpa/mlx5/rte_pmd_mlx5_vdpa_version.map b/drivers/vdpa/mlx5/rte_pmd_mlx5_vdpa_version.map
index 143836e..179f7f1 100644
--- a/drivers/vdpa/mlx5/rte_pmd_mlx5_vdpa_version.map
+++ b/drivers/vdpa/mlx5/rte_pmd_mlx5_vdpa_version.map
@@ -1,3 +1,3 @@
-DPDK_20.02 {
+DPDK_20.0.1 {
local: *;
};
--
1.8.3.1
^ permalink raw reply [relevance 7%]
* [dpdk-dev] [PATCH 1/2] common/mlx5: fix ABI version
@ 2020-02-11 19:48 7% Matan Azrad
2020-02-11 19:48 7% ` [dpdk-dev] [PATCH 2/2] vdpa/mlx5: " Matan Azrad
` (2 more replies)
0 siblings, 3 replies; 200+ results
From: Matan Azrad @ 2020-02-11 19:48 UTC (permalink / raw)
To: dev; +Cc: Viacheslav Ovsiienko, Thomas Monjalon, David Marchand
Changed the ABI version to 20.0.1.
Fixes: 7b4f1e6bd367 ("common/mlx5: introduce common library")
Signed-off-by: Matan Azrad <matan@mellanox.com>
---
drivers/common/mlx5/rte_common_mlx5_version.map | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/common/mlx5/rte_common_mlx5_version.map b/drivers/common/mlx5/rte_common_mlx5_version.map
index b1a2014..aede2a0 100644
--- a/drivers/common/mlx5/rte_common_mlx5_version.map
+++ b/drivers/common/mlx5/rte_common_mlx5_version.map
@@ -1,4 +1,4 @@
-DPDK_20.02 {
+DPDK_20.0.1 {
global:
mlx5_class_get;
--
1.8.3.1
^ permalink raw reply [relevance 7%]
* Re: [dpdk-dev] [PATCH 1/2] common/mlx5: fix ABI version
2020-02-11 19:48 7% [dpdk-dev] [PATCH 1/2] common/mlx5: fix " Matan Azrad
2020-02-11 19:48 7% ` [dpdk-dev] [PATCH 2/2] vdpa/mlx5: " Matan Azrad
@ 2020-02-11 20:31 4% ` Slava Ovsiienko
2020-02-12 8:53 4% ` David Marchand
2 siblings, 0 replies; 200+ results
From: Slava Ovsiienko @ 2020-02-11 20:31 UTC (permalink / raw)
To: Matan Azrad, dev; +Cc: Thomas Monjalon, David Marchand
> -----Original Message-----
> From: Matan Azrad <matan@mellanox.com>
> Sent: Tuesday, February 11, 2020 21:49
> To: dev@dpdk.org
> Cc: Slava Ovsiienko <viacheslavo@mellanox.com>; Thomas Monjalon
> <thomas@monjalon.net>; David Marchand <david.marchand@redhat.com>
> Subject: [PATCH 1/2] common/mlx5: fix ABI version
>
> Changed the ABI version to 20.0.1.
>
> Fixes: 7b4f1e6bd367 ("common/mlx5: introduce common library")
>
> Signed-off-by: Matan Azrad <matan@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
> ---
> drivers/common/mlx5/rte_common_mlx5_version.map | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/common/mlx5/rte_common_mlx5_version.map
> b/drivers/common/mlx5/rte_common_mlx5_version.map
> index b1a2014..aede2a0 100644
> --- a/drivers/common/mlx5/rte_common_mlx5_version.map
> +++ b/drivers/common/mlx5/rte_common_mlx5_version.map
> @@ -1,4 +1,4 @@
> -DPDK_20.02 {
> +DPDK_20.0.1 {
> global:
>
> mlx5_class_get;
> --
> 1.8.3.1
^ permalink raw reply [relevance 4%]
* Re: [dpdk-dev] [PATCH 2/2] vdpa/mlx5: fix ABI version
2020-02-11 19:48 7% ` [dpdk-dev] [PATCH 2/2] vdpa/mlx5: " Matan Azrad
@ 2020-02-11 20:31 4% ` Slava Ovsiienko
0 siblings, 0 replies; 200+ results
From: Slava Ovsiienko @ 2020-02-11 20:31 UTC (permalink / raw)
To: Matan Azrad, dev; +Cc: Thomas Monjalon, David Marchand
> -----Original Message-----
> From: Matan Azrad <matan@mellanox.com>
> Sent: Tuesday, February 11, 2020 21:49
> To: dev@dpdk.org
> Cc: Slava Ovsiienko <viacheslavo@mellanox.com>; Thomas Monjalon
> <thomas@monjalon.net>; David Marchand <david.marchand@redhat.com>
> Subject: [PATCH 2/2] vdpa/mlx5: fix ABI version
>
> Changed the ABI version to 20.0.1.
>
> Fixes: 95276abaaf0a ("vdpa/mlx5: introduce Mellanox vDPA driver")
>
> Signed-off-by: Matan Azrad <matan@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
> ---
> drivers/vdpa/mlx5/rte_pmd_mlx5_vdpa_version.map | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/vdpa/mlx5/rte_pmd_mlx5_vdpa_version.map
> b/drivers/vdpa/mlx5/rte_pmd_mlx5_vdpa_version.map
> index 143836e..179f7f1 100644
> --- a/drivers/vdpa/mlx5/rte_pmd_mlx5_vdpa_version.map
> +++ b/drivers/vdpa/mlx5/rte_pmd_mlx5_vdpa_version.map
> @@ -1,3 +1,3 @@
> -DPDK_20.02 {
> +DPDK_20.0.1 {
> local: *;
> };
> --
> 1.8.3.1
^ permalink raw reply [relevance 4%]
* [dpdk-dev] [PATCH 0/7] vfio/pci: SR-IOV support
@ 2020-02-11 23:05 2% Alex Williamson
0 siblings, 0 replies; 200+ results
From: Alex Williamson @ 2020-02-11 23:05 UTC (permalink / raw)
To: kvm
Cc: linux-pci, linux-kernel, dev, mtosatti, thomas, bluca,
jerinjacobk, bruce.richardson, cohuck
Given the mostly positive feedback from the RFC[1], here's a new
non-RFC revision. Changes since RFC:
- vfio_device_ops.match semantics refined
- Use helpers for struct pci_dev.physfn to avoid breakage without
CONFIG_PCI_IOV
- Relax to allow SR-IOV configuration changes while PF is opened.
There are potentially interesting use cases here, including
perhaps QEMU emulating an SR-IOV capability and calling out
to a privileged entity to manipulate sriov_numvfs and corral
the resulting devices.
- Retest vfio_device_feature.argsz to include uuid length.
- Add Connie's R-b on 6/7
I still wish we had a solution to make it less opaque to the user
why a VFIO_GROUP_GET_DEVICE_FD() has failed if a VF token is
required, but this is still the best I've been able to come up with.
If there are objections or better ideas, please raise them now.
The synopsis of this series is that we have an ongoing desire to drive
PCIe SR-IOV PFs from userspace with VFIO. There's an immediate need
for this with DPDK drivers and potentially interesting future use
cases in virtualization. We've been reluctant to add this support
previously due to the dependency and trust relationship between the
VF device and PF driver. Minimally the PF driver can induce a denial
of service to the VF, but depending on the specific implementation,
the PF driver might also be responsible for moving data between VFs
or have direct access to the state of the VF, including data or state
otherwise private to the VF or VF driver.
To help resolve these concerns, we introduce a VF token into the VFIO
PCI ABI, which acts as a shared secret key between drivers. The
userspace PF driver is required to set the VF token to a known value
and userspace VF drivers are required to provide the token to access
the VF device. If a PF driver is restarted with VF drivers in use, it
must also provide the current token in order to prevent a rogue
untrusted PF driver from replacing a known driver. The degree to
which this new token is considered secret is left to the userspace
drivers, the kernel intentionally provides no means to retrieve the
current token.
Note that the above token is only required for this new model where
both the PF and VF devices are usable through vfio-pci. Existing
models of VFIO drivers where the PF is used without SR-IOV enabled
or the VF is bound to a userspace driver with an in-kernel, host PF
driver are unaffected.
The latter configuration above also highlights a new inverted scenario
that is now possible, a userspace PF driver with in-kernel VF drivers.
I believe this is a scenario that should be allowed, but should not be
enabled by default. This series includes code to set a default
driver_override for VFs sourced from a vfio-pci user owned PF, such
that the VFs are also bound to vfio-pci. This model is compatible
with tools like driverctl and allows the system administrator to
decide if other bindings should be enabled. The VF token interface
above exists only between vfio-pci PF and VF drivers, once a VF is
bound to another driver, the administrator has effectively pronounced
the device as trusted. The vfio-pci driver will note alternate
binding in dmesg for logging and debugging purposes.
Please review, comment, and test. The example QEMU implementation
provided with the RFC[2] is still current for this version. Thanks,
Alex
[1] https://lore.kernel.org/lkml/158085337582.9445.17682266437583505502.stgit@gimli.home/
[2] https://lore.kernel.org/lkml/20200204161737.34696b91@w520.home/
---
Alex Williamson (7):
vfio: Include optional device match in vfio_device_ops callbacks
vfio/pci: Implement match ops
vfio/pci: Introduce VF token
vfio: Introduce VFIO_DEVICE_FEATURE ioctl and first user
vfio/pci: Add sriov_configure support
vfio/pci: Remove dev_fmt definition
vfio/pci: Cleanup .probe() exit paths
drivers/vfio/pci/vfio_pci.c | 312 ++++++++++++++++++++++++++++++++---
drivers/vfio/pci/vfio_pci_private.h | 10 +
drivers/vfio/vfio.c | 20 ++
include/linux/vfio.h | 4
include/uapi/linux/vfio.h | 37 ++++
5 files changed, 355 insertions(+), 28 deletions(-)
^ permalink raw reply [relevance 2%]
* Re: [dpdk-dev] [PATCH 1/2] common/mlx5: fix ABI version
2020-02-11 19:48 7% [dpdk-dev] [PATCH 1/2] common/mlx5: fix " Matan Azrad
2020-02-11 19:48 7% ` [dpdk-dev] [PATCH 2/2] vdpa/mlx5: " Matan Azrad
2020-02-11 20:31 4% ` [dpdk-dev] [PATCH 1/2] common/mlx5: " Slava Ovsiienko
@ 2020-02-12 8:53 4% ` David Marchand
2 siblings, 0 replies; 200+ results
From: David Marchand @ 2020-02-12 8:53 UTC (permalink / raw)
To: Matan Azrad; +Cc: dev, Viacheslav Ovsiienko, Thomas Monjalon
On Tue, Feb 11, 2020 at 8:48 PM Matan Azrad <matan@mellanox.com> wrote:
>
> Changed the ABI version to 20.0.1.
>
> Fixes: 7b4f1e6bd367 ("common/mlx5: introduce common library")
>
> Signed-off-by: Matan Azrad <matan@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
Series applied, thanks.
--
David Marchand
^ permalink raw reply [relevance 4%]
* [dpdk-dev] [RFC 0/4] Enforce flag checking in API's
@ 2020-02-12 23:08 3% Stephen Hemminger
2020-02-12 23:08 3% ` [dpdk-dev] [RFC 1/4] ring: future proof flag settings Stephen Hemminger
` (3 more replies)
0 siblings, 4 replies; 200+ results
From: Stephen Hemminger @ 2020-02-12 23:08 UTC (permalink / raw)
To: dev; +Cc: Stephen Hemminger
Since the DPDK api's are lazy about checking for flag values
it means the flags can never be extended in future releases
without causing ABI breakage. This means we end up doing unnecessary
symbol versioning just to work around applications that might
pass in naughty bits.
This is the DPDK analog of the Linux kernel openat() problem.
Openat api was added but since kernel did not check flags it
ended up that another syscall openat2() was necessary before
the flags could be used.
Stephen Hemminger (4):
ring: future proof flag settings
hash: check flags on creation
stack: check flags on creation
cfgfile: check flags value
lib/librte_cfgfile/rte_cfgfile.c | 4 ++++
lib/librte_hash/rte_cuckoo_hash.c | 9 +++++++++
lib/librte_ring/rte_ring.c | 7 +++++++
lib/librte_stack/rte_stack.c | 5 +++++
4 files changed, 25 insertions(+)
--
2.20.1
^ permalink raw reply [relevance 3%]
* [dpdk-dev] [RFC 1/4] ring: future proof flag settings
2020-02-12 23:08 3% [dpdk-dev] [RFC 0/4] Enforce flag checking in API's Stephen Hemminger
@ 2020-02-12 23:08 3% ` Stephen Hemminger
2020-02-13 11:12 0% ` Ananyev, Konstantin
2020-02-12 23:08 3% ` [dpdk-dev] [RFC 2/4] hash: check flags on creation Stephen Hemminger
` (2 subsequent siblings)
3 siblings, 1 reply; 200+ results
From: Stephen Hemminger @ 2020-02-12 23:08 UTC (permalink / raw)
To: dev; +Cc: Stephen Hemminger
All API's should check that they support the flag values
passed. If an application passes an invalid flag it could
cause problems in later ABI.
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
---
lib/librte_ring/rte_ring.c | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/lib/librte_ring/rte_ring.c b/lib/librte_ring/rte_ring.c
index 77e5de099b81..6cd8831649ea 100644
--- a/lib/librte_ring/rte_ring.c
+++ b/lib/librte_ring/rte_ring.c
@@ -100,6 +100,13 @@ rte_ring_init(struct rte_ring *r, const char *name, unsigned count,
RTE_BUILD_BUG_ON((offsetof(struct rte_ring, prod) &
RTE_CACHE_LINE_MASK) != 0);
+ /* future proof flags, only allow supported values */
+ if (flags & ~(RING_F_SP_ENQ | RING_F_SC_DEQ)) {
+ RTE_LOG(ERR, RING,
+ "Unsupported flags requested %d\n", flags);
+ return -EINVAL;
+ }
+
/* init the ring structure */
memset(r, 0, sizeof(*r));
ret = strlcpy(r->name, name, sizeof(r->name));
--
2.20.1
^ permalink raw reply [relevance 3%]
* [dpdk-dev] [RFC 2/4] hash: check flags on creation
2020-02-12 23:08 3% [dpdk-dev] [RFC 0/4] Enforce flag checking in API's Stephen Hemminger
2020-02-12 23:08 3% ` [dpdk-dev] [RFC 1/4] ring: future proof flag settings Stephen Hemminger
@ 2020-02-12 23:08 3% ` Stephen Hemminger
2020-02-12 23:08 3% ` [dpdk-dev] [RFC 3/4] stack: " Stephen Hemminger
2020-02-12 23:08 3% ` [dpdk-dev] [RFC 4/4] cfgfile: check flags value Stephen Hemminger
3 siblings, 0 replies; 200+ results
From: Stephen Hemminger @ 2020-02-12 23:08 UTC (permalink / raw)
To: dev; +Cc: Stephen Hemminger
All API's should check that they support the flag values
passed. If an application passes an invalid flag it could
cause problems in later ABI.
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
---
lib/librte_hash/rte_cuckoo_hash.c | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/lib/librte_hash/rte_cuckoo_hash.c b/lib/librte_hash/rte_cuckoo_hash.c
index 6af8ca42e94f..ab9cd91b4955 100644
--- a/lib/librte_hash/rte_cuckoo_hash.c
+++ b/lib/librte_hash/rte_cuckoo_hash.c
@@ -32,6 +32,9 @@
#include "rte_hash.h"
#include "rte_cuckoo_hash.h"
+/* Mask of all flags supported by this version */
+#define RTE_HASH_EXTRA_FLAGS_MASK 0x2f
+
#define FOR_EACH_BUCKET(CURRENT_BKT, START_BUCKET) \
for (CURRENT_BKT = START_BUCKET; \
CURRENT_BKT != NULL; \
@@ -164,6 +167,12 @@ rte_hash_create(const struct rte_hash_parameters *params)
return NULL;
}
+ if (params->extra_flag & ~RTE_HASH_EXTRA_FLAGS_MASK) {
+ rte_errno = EINVAL;
+ RTE_LOG(ERR, HASH, "rte_hash_create: unsupported extra flags\n");
+ return NULL;
+ }
+
/* Validate correct usage of extra options */
if ((params->extra_flag & RTE_HASH_EXTRA_FLAGS_RW_CONCURRENCY) &&
(params->extra_flag & RTE_HASH_EXTRA_FLAGS_RW_CONCURRENCY_LF)) {
--
2.20.1
^ permalink raw reply [relevance 3%]
* [dpdk-dev] [RFC 3/4] stack: check flags on creation
2020-02-12 23:08 3% [dpdk-dev] [RFC 0/4] Enforce flag checking in API's Stephen Hemminger
2020-02-12 23:08 3% ` [dpdk-dev] [RFC 1/4] ring: future proof flag settings Stephen Hemminger
2020-02-12 23:08 3% ` [dpdk-dev] [RFC 2/4] hash: check flags on creation Stephen Hemminger
@ 2020-02-12 23:08 3% ` Stephen Hemminger
2020-02-12 23:08 3% ` [dpdk-dev] [RFC 4/4] cfgfile: check flags value Stephen Hemminger
3 siblings, 0 replies; 200+ results
From: Stephen Hemminger @ 2020-02-12 23:08 UTC (permalink / raw)
To: dev; +Cc: Stephen Hemminger
All API's should check that they support the flag values
passed. If an application passes an invalid flag it could
cause problems in later ABI.
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
---
lib/librte_stack/rte_stack.c | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/lib/librte_stack/rte_stack.c b/lib/librte_stack/rte_stack.c
index d19824f00439..4b09ff30d6bd 100644
--- a/lib/librte_stack/rte_stack.c
+++ b/lib/librte_stack/rte_stack.c
@@ -59,6 +59,11 @@ rte_stack_create(const char *name, unsigned int count, int socket_id,
unsigned int sz;
int ret;
+ if (flags & ~(RTE_STACK_F_LF)) {
+ STACK_LOG_ERR("Unsupported stack flags %#x\n", flags);
+ return NULL;
+ }
+
#ifdef RTE_ARCH_64
RTE_BUILD_BUG_ON(sizeof(struct rte_stack_lf_head) != 16);
#else
--
2.20.1
^ permalink raw reply [relevance 3%]
* [dpdk-dev] [RFC 4/4] cfgfile: check flags value
2020-02-12 23:08 3% [dpdk-dev] [RFC 0/4] Enforce flag checking in API's Stephen Hemminger
` (2 preceding siblings ...)
2020-02-12 23:08 3% ` [dpdk-dev] [RFC 3/4] stack: " Stephen Hemminger
@ 2020-02-12 23:08 3% ` Stephen Hemminger
3 siblings, 0 replies; 200+ results
From: Stephen Hemminger @ 2020-02-12 23:08 UTC (permalink / raw)
To: dev; +Cc: Stephen Hemminger
All API's should check that they support the flag values
passed. If an application passes an invalid flag it could
cause problems in later ABI.
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
---
lib/librte_cfgfile/rte_cfgfile.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/lib/librte_cfgfile/rte_cfgfile.c b/lib/librte_cfgfile/rte_cfgfile.c
index 9049fd9c2319..cec82c82ee20 100644
--- a/lib/librte_cfgfile/rte_cfgfile.c
+++ b/lib/librte_cfgfile/rte_cfgfile.c
@@ -272,6 +272,10 @@ rte_cfgfile_create(int flags)
int i;
struct rte_cfgfile *cfg;
+ /* future proof flags usage */
+ if (flags & ~(CFG_FLAG_GLOBAL_SECTION | CFG_FLAG_EMPTY_VALUES))
+ return NULL;
+
cfg = malloc(sizeof(*cfg));
if (cfg == NULL)
--
2.20.1
^ permalink raw reply [relevance 3%]
* Re: [dpdk-dev] [RFC 1/4] ring: future proof flag settings
2020-02-12 23:08 3% ` [dpdk-dev] [RFC 1/4] ring: future proof flag settings Stephen Hemminger
@ 2020-02-13 11:12 0% ` Ananyev, Konstantin
0 siblings, 0 replies; 200+ results
From: Ananyev, Konstantin @ 2020-02-13 11:12 UTC (permalink / raw)
To: Stephen Hemminger, dev
> -----Original Message-----
> From: dev <dev-bounces@dpdk.org> On Behalf Of Stephen Hemminger
> Sent: Wednesday, February 12, 2020 11:08 PM
> To: dev@dpdk.org
> Cc: Stephen Hemminger <stephen@networkplumber.org>
> Subject: [dpdk-dev] [RFC 1/4] ring: future proof flag settings
>
> All API's should check that they support the flag values
> passed. If an application passes an invalid flag it could
> cause problems in later ABI.
>
> Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
> ---
> lib/librte_ring/rte_ring.c | 7 +++++++
> 1 file changed, 7 insertions(+)
>
> diff --git a/lib/librte_ring/rte_ring.c b/lib/librte_ring/rte_ring.c
> index 77e5de099b81..6cd8831649ea 100644
> --- a/lib/librte_ring/rte_ring.c
> +++ b/lib/librte_ring/rte_ring.c
> @@ -100,6 +100,13 @@ rte_ring_init(struct rte_ring *r, const char *name, unsigned count,
> RTE_BUILD_BUG_ON((offsetof(struct rte_ring, prod) &
> RTE_CACHE_LINE_MASK) != 0);
>
> + /* future proof flags, only allow supported values */
> + if (flags & ~(RING_F_SP_ENQ | RING_F_SC_DEQ)) {
> + RTE_LOG(ERR, RING,
> + "Unsupported flags requested %d\n", flags);
> + return -EINVAL;
> + }
> +
Good idea, I think, but seems you forgot RING_F_EXACT_SZ.
> /* init the ring structure */
> memset(r, 0, sizeof(*r));
> ret = strlcpy(r->name, name, sizeof(r->name));
> --
> 2.20.1
^ permalink raw reply [relevance 0%]
Results 7201-7400 of ~18000 next (newer) | prev (older) | reverse | sort options + mbox downloads above
-- links below jump to the message on this page --
2019-09-06 19:05 [dpdk-dev] [PATCH v2 0/6] lib/ring: templates to support custom element size Honnappa Nagarahalli
2020-01-16 5:25 ` [dpdk-dev] [PATCH v9 0/6] lib/ring: APIs " Honnappa Nagarahalli
2020-01-16 5:25 ` [dpdk-dev] [PATCH v9 5/6] lib/hash: use ring with 32b element size to save memory Honnappa Nagarahalli
2020-01-17 20:27 5% ` David Marchand
2020-01-17 20:54 0% ` Honnappa Nagarahalli
2020-01-17 21:07 3% ` David Marchand
2020-01-17 22:24 3% ` Wang, Yipeng1
2020-01-18 19:32 ` [dpdk-dev] [PATCH v10 0/6] lib/ring: APIs to support custom element size Honnappa Nagarahalli
2020-01-19 19:31 3% ` David Marchand
2019-10-25 15:46 [dpdk-dev] [PATCH v5] eal: add manual probing option Gaetan Rivet
2020-01-23 9:58 ` [dpdk-dev] [PATCH v7] " Gaetan Rivet
2020-02-03 5:16 ` Pavan Nikhilesh Bhagavatula
2020-02-03 22:21 3% ` Thomas Monjalon
2020-02-04 10:03 0% ` Gaetan Rivet
2020-02-04 11:07 0% ` Thomas Monjalon
2020-02-04 12:43 0% ` Gaetan Rivet
2020-02-04 15:06 0% ` Thomas Monjalon
2020-02-04 16:02 0% ` Gaetan Rivet
2020-02-10 14:51 0% ` Jerin Jacob
2020-02-10 15:27 0% ` Thomas Monjalon
2020-02-10 16:33 0% ` Jerin Jacob
2019-10-25 17:53 [dpdk-dev] [RFC v2 0/7] RFC: Support MACSEC offload in the RTE_SECURITY infrastructure Pavel Belous
2020-01-27 11:25 0% ` Akhil Goyal
2019-11-18 9:50 [dpdk-dev] [RFC v20.20] mbuf: introduce pktmbuf pool with pinned external buffers Shahaf Shuler
2020-01-16 13:04 ` [dpdk-dev] [PATCH v4 0/5] mbuf: detach mbuf with pinned external buffer Viacheslav Ovsiienko
2020-01-16 13:04 ` [dpdk-dev] [PATCH v4 4/5] app/testpmd: add mempool with external data buffers Viacheslav Ovsiienko
2020-01-20 14:11 3% ` Olivier Matz
2019-11-26 14:56 [dpdk-dev] [PATCH] service: don't walk out of bounds when checking services Aaron Conole
2019-12-03 21:15 ` [dpdk-dev] [PATCH v2] " Aaron Conole
2019-12-04 8:33 ` David Marchand
2019-12-04 8:34 ` David Marchand
2019-12-20 14:43 ` David Marchand
2020-02-07 12:04 ` Kevin Traynor
2020-02-07 14:27 3% ` Aaron Conole
2019-12-17 10:43 [dpdk-dev] [PATCH] lib/cfgfile: update map file lironh
2019-12-17 11:52 ` [dpdk-dev] [PATCH v2] cfgfile: " lironh
2019-12-17 11:12 ` Dumitrescu, Cristian
2020-01-19 23:31 0% ` Thomas Monjalon
2019-12-17 13:07 [dpdk-dev] [PATCH] meter: move RFC4115 trTCM APIs as none experimental Eelco Chaudron
2020-01-16 11:25 5% ` [dpdk-dev] [dpdk-stable] " David Marchand
2020-01-16 11:54 0% ` Neil Horman
2020-01-16 12:42 0% ` Ferruh Yigit
2020-01-16 14:38 0% ` Thomas Monjalon
2020-01-17 8:27 0% ` David Marchand
2020-01-17 10:46 0% ` Luca Boccassi
2020-01-22 13:55 0% ` Ray Kinsella
2020-01-30 16:33 4% ` Ray Kinsella
2019-12-20 6:56 [dpdk-dev] [PATCH v6 2/2] common/octeontx2: add polling based response mbox message Sunil Kumar Kori
2020-01-14 9:04 ` [dpdk-dev] [PATCH v7 1/2] eal: add API to check if its interrupt context Sunil Kumar Kori
2020-02-06 15:35 3% ` David Marchand
2019-12-20 15:20 [dpdk-dev] [PATCH] add ABI checks David Marchand
2020-01-14 23:19 ` Thomas Monjalon
2020-01-15 11:33 ` Neil Horman
2020-01-15 12:38 ` Thomas Monjalon
2020-01-16 11:52 4% ` Neil Horman
2020-01-16 14:20 4% ` Thomas Monjalon
2020-01-16 18:49 4% ` Neil Horman
2020-01-16 20:01 4% ` Thomas Monjalon
2020-01-17 19:01 4% ` Neil Horman
2020-01-17 21:26 9% ` David Marchand
2020-01-29 17:26 9% ` [dpdk-dev] [PATCH v2 0/4] " David Marchand
2020-01-29 17:26 2% ` [dpdk-dev] [PATCH v2 3/4] build: test meson installation David Marchand
2020-01-29 17:26 28% ` [dpdk-dev] [PATCH v2 4/4] add ABI checks David Marchand
2020-01-29 17:42 7% ` Thomas Monjalon
2020-01-29 18:10 8% ` Anoob Joseph
2020-01-29 20:03 7% ` David Marchand
2020-01-29 20:13 4% ` Akhil Goyal
2020-01-30 16:09 8% ` Ferruh Yigit
2020-01-30 20:18 4% ` Thomas Monjalon
2020-01-31 9:03 7% ` Ferruh Yigit
2020-01-31 10:26 7% ` Ananyev, Konstantin
2020-01-31 14:16 4% ` Trahe, Fiona
2020-02-02 13:05 9% ` Thomas Monjalon
2020-02-02 14:41 4% ` Ananyev, Konstantin
2020-02-03 9:30 7% ` Ferruh Yigit
2020-02-03 11:50 9% ` Neil Horman
2020-02-03 13:09 7% ` Ferruh Yigit
2020-02-03 14:00 7% ` Dodji Seketeli
2020-02-03 14:46 4% ` Ferruh Yigit
2020-02-03 15:08 7% ` Trahe, Fiona
2020-02-03 17:09 9% ` Thomas Monjalon
2020-02-03 17:34 8% ` Thomas Monjalon
2020-02-03 18:55 7% ` Ray Kinsella
2020-02-03 21:07 7% ` Thomas Monjalon
2020-02-04 9:46 8% ` Ferruh Yigit
2020-02-04 10:24 9% ` Thomas Monjalon
2020-02-04 12:44 4% ` Trahe, Fiona
2020-02-04 15:52 8% ` Trahe, Fiona
2020-02-04 15:59 7% ` Thomas Monjalon
2020-02-04 17:46 4% ` Trahe, Fiona
2020-02-04 12:57 4% ` Kevin Traynor
2020-02-04 14:44 4% ` Aaron Conole
2020-02-04 19:49 7% ` Neil Horman
2020-02-04 9:51 4% ` David Marchand
2020-02-04 10:10 4% ` Trahe, Fiona
2020-02-04 10:38 4% ` Thomas Monjalon
2020-02-05 11:10 4% ` Ray Kinsella
2020-02-03 17:40 8% ` Ferruh Yigit
2020-02-03 18:40 9% ` Thomas Monjalon
2020-02-04 9:19 4% ` Ferruh Yigit
2020-02-04 9:45 4% ` Thomas Monjalon
2020-02-04 9:56 4% ` Ferruh Yigit
2020-02-04 10:08 4% ` Bruce Richardson
2020-02-04 10:17 4% ` Kevin Traynor
2020-02-04 10:16 9% ` Akhil Goyal
2020-02-04 10:28 8% ` Thomas Monjalon
2020-02-04 10:32 4% ` Akhil Goyal
2020-02-04 11:35 4% ` Bruce Richardson
2020-02-04 22:10 7% ` Neil Horman
2020-02-05 6:16 7% ` [dpdk-dev] [EXT] " Anoob Joseph
2020-02-05 14:33 4% ` Trahe, Fiona
2020-02-04 21:59 8% ` [dpdk-dev] " Neil Horman
2020-01-30 13:06 8% ` Trahe, Fiona
2020-01-30 15:59 4% ` Thomas Monjalon
2020-01-30 16:42 7% ` Ferruh Yigit
2020-01-30 23:49 4% ` Ananyev, Konstantin
2020-01-31 9:07 4% ` Ferruh Yigit
2020-01-31 9:37 4% ` Ananyev, Konstantin
2020-01-30 10:57 4% ` [dpdk-dev] [PATCH v2 0/4] " Luca Boccassi
2020-01-30 16:00 9% ` [dpdk-dev] [PATCH v3 " David Marchand
2020-01-30 16:00 3% ` [dpdk-dev] [PATCH v3 3/4] build: test meson installation David Marchand
2020-01-30 22:17 3% ` Thomas Monjalon
2020-01-30 16:00 28% ` [dpdk-dev] [PATCH v3 4/4] add ABI checks David Marchand
2020-01-30 22:32 4% ` Thomas Monjalon
2020-02-01 15:29 4% ` David Marchand
2020-01-30 22:44 4% ` Thomas Monjalon
2020-02-02 21:08 9% ` [dpdk-dev] [PATCH v4 0/3] " David Marchand
2020-02-02 21:08 28% ` [dpdk-dev] [PATCH v4 3/3] " David Marchand
2020-02-05 14:13 4% ` [dpdk-dev] [PATCH v4 0/3] " Thomas Monjalon
2020-01-04 1:33 [dpdk-dev] [PATCH 00/14] cleanup resources on shutdown Stephen Hemminger
2020-02-05 9:32 0% ` David Marchand
2020-02-05 12:07 0% ` Stephen Hemminger
2020-02-05 12:32 0% ` David Marchand
2020-01-09 13:27 [dpdk-dev] [PATCH] mempool: fix mempool virt populate with small chunks Olivier Matz
2020-01-17 14:57 5% ` [dpdk-dev] [PATCH v2 0/3] " Olivier Matz
2020-01-17 14:57 3% ` [dpdk-dev] [PATCH v2 1/3] " Olivier Matz
2020-01-17 14:57 8% ` [dpdk-dev] [PATCH v2 2/3] doc: announce API change for mempool IOVA populate Olivier Matz
2020-01-17 20:32 0% ` David Marchand
2020-01-17 14:57 2% ` [dpdk-dev] [PATCH v2 3/3] [20.05] mempool: return 0 if area is too small on populate Olivier Matz
2020-01-17 20:32 0% ` David Marchand
2020-01-20 12:02 0% ` [dpdk-dev] [PATCH v2 0/3] mempool: fix mempool virt populate with small chunks Thomas Monjalon
2020-01-10 15:20 [dpdk-dev] [PATCH v4 00/14] net/i40e: ESP support Bernard Iremonger
2020-01-14 13:55 ` [dpdk-dev] [PATCH v5 12/14] librte_ethdev: add ESP and AH flow types to RSS Bernard Iremonger
2020-01-14 18:45 ` Ferruh Yigit
2020-01-15 9:13 ` Andrew Rybchenko
2020-01-15 10:44 ` Ferruh Yigit
2020-01-15 10:55 ` Andrew Rybchenko
2020-01-15 12:28 ` Ferruh Yigit
2020-01-15 14:11 ` Iremonger, Bernard
2020-01-15 14:36 0% ` Ferruh Yigit
2020-01-13 10:43 [dpdk-dev] [PATCH v3] cryptodev: add chacha20-poly1305 aead algorithm Arek Kusztal
2020-01-14 13:18 ` Anoob Joseph
2020-01-15 15:53 ` Akhil Goyal
2020-02-05 14:28 5% ` Thomas Monjalon
2020-02-05 14:41 0% ` Trahe, Fiona
2020-02-05 15:21 0% ` Thomas Monjalon
2020-01-14 13:55 [dpdk-dev] [PATCH v5 01/14] app/testpmd: parse flow command line for ESP Bernard Iremonger
2020-01-15 15:53 3% ` [dpdk-dev] [PATCH v6 00/13] net/i40e: ESP support Bernard Iremonger
2020-01-16 12:44 3% ` [dpdk-dev] [PATCH v7 00/10] " Bernard Iremonger
2020-01-16 17:42 0% ` Ferruh Yigit
2020-01-21 16:10 0% ` Iremonger, Bernard
2020-01-16 11:13 4% [dpdk-dev] DPDK Release Status Meeting 16/01/2020 Ferruh Yigit
2020-01-16 12:48 3% ` Ananyev, Konstantin
2020-01-16 13:17 4% ` Akhil Goyal
2020-01-16 13:39 4% ` Ananyev, Konstantin
2020-01-16 16:42 4% ` Akhil Goyal
2020-01-16 17:47 3% ` Ferruh Yigit
2020-01-17 16:00 0% ` Honnappa Nagarahalli
2020-01-17 11:39 0% Ananyev, Konstantin
[not found] <20200117131913.72621-1-ferruh.yigit@intel.com>
2020-01-17 13:29 ` [dpdk-dev] [dpdk-web] [PATCH] add build system update as nice to have Richardson, Bruce
2020-01-18 0:10 3% ` Honnappa Nagarahalli
2020-01-22 3:46 3% [dpdk-dev] [PATCH] net/octeontx2: extend RSS supported offload types kirankumark
2020-01-22 8:05 0% ` Jerin Jacob
2020-01-23 5:30 0% ` [dpdk-dev] [EXT] " Kiran Kumar Kokkilagadda
2020-01-24 3:46 3% ` [dpdk-dev] [PATCH v2] " kirankumark
2020-01-28 11:14 0% ` Jerin Jacob
2020-01-22 10:52 [dpdk-dev] [PATCH v2 1/2] crypto/ccp: sha3 support enabling in ccp ssardar
2020-01-22 11:11 ` Kumar, Ravi1
2020-02-05 12:24 ` Akhil Goyal
2020-02-05 22:22 3% ` Thomas Monjalon
2020-02-06 9:34 0% ` David Marchand
2020-01-26 1:09 [dpdk-dev] [PATCH 0/5] l3fwd and lpm cleanups Stephen Hemminger
2020-01-26 1:09 3% ` [dpdk-dev] [PATCH 1/5] lpm: make ipv6 address immutable Stephen Hemminger
2020-01-27 10:30 [dpdk-dev] [PATCH 0/2] app/testpmd: add portlist option to the testpmd Hariprasad Govindharajan
2020-01-27 10:30 ` [dpdk-dev] [PATCH 2/2] eal: add eal_parse_optionlist to parse user input Hariprasad Govindharajan
2020-01-28 17:35 ` Ferruh Yigit
2020-01-29 17:44 4% ` David Marchand
2020-01-29 18:07 0% ` Ferruh Yigit
2020-01-29 19:19 0% ` Stephen Hemminger
2020-01-31 11:25 0% ` Govindharajan, Hariprasad
2020-01-31 14:42 0% ` Govindharajan, Hariprasad
2020-01-29 12:29 3% [dpdk-dev] [RFC] meter: fix ABI break due to experimental tag removal Ferruh Yigit
2020-01-29 14:46 4% ` Bruce Richardson
2020-01-29 14:57 4% ` Andrzej Ostruszka
2020-01-29 16:25 4% ` Ferruh Yigit
2020-01-29 16:30 4% ` Andrzej Ostruszka
2020-01-29 16:40 4% ` Ferruh Yigit
2020-01-29 16:43 3% ` [dpdk-dev] [RFC v2] " Ferruh Yigit
2020-01-29 17:49 4% ` Andrzej Ostruszka
2020-01-30 12:33 7% ` Thomas Monjalon
2020-01-30 12:57 4% ` Luca Boccassi
2020-01-30 14:17 8% ` Thomas Monjalon
2020-01-30 14:21 4% ` Luca Boccassi
2020-01-30 15:55 4% ` Thomas Monjalon
2020-01-30 16:04 4% ` Luca Boccassi
2020-01-30 16:15 4% ` Eelco Chaudron
2020-01-30 20:20 4% ` Thomas Monjalon
2020-02-03 11:10 4% ` Eelco Chaudron
2020-01-31 9:25 4% ` Ferruh Yigit
2020-02-02 18:53 7% ` [dpdk-dev] [RFC] " Neil Horman
2020-02-03 12:53 4% ` Ferruh Yigit
2020-02-04 12:02 9% ` Neil Horman
2020-02-05 10:04 4% ` Luca Boccassi
2020-02-05 11:32 7% ` Neil Horman
2020-01-29 16:23 [dpdk-dev] [PATCH] build: remove function versioning from meson files Andrzej Ostruszka
2020-01-30 9:25 ` [dpdk-dev] [PATCH v2] build: remove unused function versioning Andrzej Ostruszka
2020-01-30 16:11 3% ` David Marchand
2020-02-05 20:44 0% ` David Marchand
2020-01-30 12:16 4% [dpdk-dev] DPDK Release Status Meeting 30/01/2020 Ferruh Yigit
2020-01-30 14:20 16% [dpdk-dev] [PATCH] doc: deprecate using MAX values as array size Ferruh Yigit
2020-01-30 16:58 4% [dpdk-dev] [PATCH] doc: deprecation notice to move igb_uio to a new repo jerinj
2020-01-30 17:30 0% ` Ferruh Yigit
2020-01-31 12:36 4% [dpdk-dev] Minutes of Technical Board Meeting, 2020-01-29 Olivier Matz
2020-02-03 14:53 5% [dpdk-dev] Minutes of Technical Board Meeting, 2020-01-15 Maxime Coquelin
2020-02-04 23:05 3% [dpdk-dev] [RFC PATCH 0/7] vfio/pci: SR-IOV support Alex Williamson
2020-02-11 11:18 0% ` Jerin Jacob
2020-02-05 14:34 3% [dpdk-dev] Travis warning about job configs David Marchand
2020-02-05 15:17 17% [dpdk-dev] [PATCH] doc: alias to experimental tag for stable apis Ray Kinsella
2020-02-05 15:43 17% ` [dpdk-dev] [PATCH v3] " Ray Kinsella
2020-02-05 15:28 17% [dpdk-dev] [PATCH v2] " Ray Kinsella
2020-02-05 17:07 [dpdk-dev] [PATCH] buildtools: allow pedantic empty pmdinfo Thomas Monjalon
2020-02-05 17:14 ` Bruce Richardson
2020-02-05 17:29 3% ` Thomas Monjalon
2020-02-06 10:32 4% [dpdk-dev] DPDK Release Status Meeting 6/02/2020 Ferruh Yigit
2020-02-06 12:04 7% [dpdk-dev] [PATCH v1] common/iavf: update the ABI version Haiyue Wang
2020-02-11 2:08 4% ` Ye Xiaolong
2020-02-06 22:33 4% [dpdk-dev] [dpdk-announce] release candidate 20.02-rc2 Thomas Monjalon
2020-02-10 10:27 3% [dpdk-dev] [PATCH] common/octeontx2: update version map Anoob Joseph
2020-02-11 14:04 0% ` David Marchand
2020-02-11 11:27 7% [dpdk-dev] [PATCH 1/1] raw/octeontx2_ep: update the ABI version Mahipal Challa
2020-02-11 14:05 4% ` David Marchand
2020-02-11 19:48 7% [dpdk-dev] [PATCH 1/2] common/mlx5: fix " Matan Azrad
2020-02-11 19:48 7% ` [dpdk-dev] [PATCH 2/2] vdpa/mlx5: " Matan Azrad
2020-02-11 20:31 4% ` Slava Ovsiienko
2020-02-11 20:31 4% ` [dpdk-dev] [PATCH 1/2] common/mlx5: " Slava Ovsiienko
2020-02-12 8:53 4% ` David Marchand
2020-02-11 23:05 2% [dpdk-dev] [PATCH 0/7] vfio/pci: SR-IOV support Alex Williamson
2020-02-12 23:08 3% [dpdk-dev] [RFC 0/4] Enforce flag checking in API's Stephen Hemminger
2020-02-12 23:08 3% ` [dpdk-dev] [RFC 1/4] ring: future proof flag settings Stephen Hemminger
2020-02-13 11:12 0% ` Ananyev, Konstantin
2020-02-12 23:08 3% ` [dpdk-dev] [RFC 2/4] hash: check flags on creation Stephen Hemminger
2020-02-12 23:08 3% ` [dpdk-dev] [RFC 3/4] stack: " Stephen Hemminger
2020-02-12 23:08 3% ` [dpdk-dev] [RFC 4/4] cfgfile: check flags value Stephen Hemminger
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).