Re: [dpdk-dev] [PATCH v4 1/4] ethdev: add tm support for shaper config in pkt mode

[Thomas Monjalon <thomas@monjalon.net>]

28/04/2020 17:04, Luca Boccassi:
> On Tue, 2020-04-28 at 15:45 +0100, Bruce Richardson wrote:
> > On Tue, Apr 28, 2020 at 03:06:20PM +0100, Ferruh Yigit wrote:
> > > On 4/27/2020 5:59 PM, Jerin Jacob wrote:
> > > > On Mon, Apr 27, 2020 at 10:19 PM Ferruh Yigit <ferruh.yigit@intel.com> wrote:
> > > > > On 4/27/2020 5:29 PM, Jerin Jacob wrote:
> > > > > > On Mon, Apr 27, 2020 at 9:42 PM Ferruh Yigit <ferruh.yigit@intel.com> wrote:
> > > > > > > On 4/27/2020 10:19 AM, Dumitrescu, Cristian wrote:
> > > > > > > > From: Yigit, Ferruh <ferruh.yigit@intel.com>
> > > > > > > > > On 4/24/2020 11:28 AM, Dumitrescu, Cristian wrote:
> > > > > > > > > > From: Nithin Dabilpuram <nithind1988@gmail.com>
> > > > > > > > > > > This patch also updates tm port/level/node capability structures with
> > > > > > > > > > > exiting features of scheduler wfq packet mode, scheduler wfq byte mode
> > > > > > > > > > > and private/shared shaper byte mode.
> > > > > > > > > > > 
> > > > > > > > > > > SoftNIC PMD is also updated with new capabilities.
[...]
> > > > > > > > > Hi Nithin,
> > > > > > > > > 
> > > > > > > > > It looks like patch is causing ABI break, I am getting following warning [1],
> > > > > > > > > can you please check?
> > > > > > > > > 
> > > > > > > > > [1]
> > > > > > > > > https://pastebin.com/XYNFg14u
> > > > > > > > 
> > > > > > > > Hi Ferruh,
> > > > > > > > 
> > > > > > > > The RTE_TM API is marked as experimental,
> > > > > > > > but it looks that this was not correctly marked
> > > > > > > > when __rte_experimental ABI checker was introduced.
> > > > > > > > 
> > > > > > > > It is marked as experimental at the top of the rte_tm.h,
> > > > > > > > similarly to other APIs introduced around same time,
> > > > > > > > but it was not correctly picked up by the ABI check procedure
> > > > > > > > when later introduced, so __rte_experimental was not added to every function.
> > > > > > > > 
> > > > > > > 
> > > > > > > :(
> > > > > > > 
> > > > > > > Is it time to mature them?
> > > > > > > 
> > > > > > > As you said they are not marked as experimental both in header file (function
> > > > > > > declarations) and .map file.
> > > > > > > 
> > > > > > > The problem is, they are not marked as experimental in DPDK_20.0 ABI (v19.11),
> > > > > > > so marking them as experimental now will break the ABI. Not sure what to do,
> > > > > > > cc'ed a few ABI related names for comment.
> > > > > > > 
> > > > > > > For me, we need to proceed as the experimental tag removed and APIs become
> > > > > > > mature starting from v19.11, since this is what happened in practice, and remove
> > > > > > > a few existing being experimental references in the doxygen comments.
> > > > > > 
> > > > > > I think, accidentally we can not make a library as NON-experimental.
> > > > > > TM never went through experimental to mature transition(see git log
> > > > > > lib/librte_ethdev/rte_tm.h)
> > > > > > It was a bug to not mark as experimental in each function in the ABI process.
> > > > > > Some of the features like packet marking are not even implemented by any HW.
> > > > > > I think, we can make API stable only all the features are implemented
> > > > > > by one or two HW.

Yes this is what was decided one or two years ago I think.
But rte_tm API was introduced 3 years ago and is implemented by 6 PMDs.



> > > > > Fair enough, specially if the API is not ready yet.
> > > > > 
> > > > > But they were part of stable ABI, and marking them as experimental now will
> > > > > break the old applications using these APIs.
> > > > 
> > > > it is still marked as EXPERIMENTAL everywhere and API is not ready yet.

rte_tm is implemented in 6 PMDs.


> > > Existing experimental marks are text only for human parsing.
> > > 
> > > The compiler attribute and build time checks are missing, and the symbol in the
> > > binary doesn't have experimental tag. Our scripts and automated checks won't
> > > detect it as experimental.
> > > 
> > > My point is just having experimental comment in header file is not enough to
> > > qualify the APIs as experimental.
> > > 
> > > > Anyway, we need to break the ABI to make it work on various HW.

Yes this is why I was asking in 19.11 to check our API,
in order to avoid such situation.


> > > > I am not sure what to do?

Either manage ABI versioning, or wait 20.11.


> > > > IMO, We need to send a patch as Fixes: for the bug of not adding
> > > > __rte_experimental in each function.

No, this is wrong.


> > > Yes, this is where we are, both you and Cristian suggest API is not ready and
> > > should be experimental, but they were part of stable ABI, making them
> > > experimental will break the ABI.
> > > It looks like there is no good option but we should select one of the bad ones.
> > > 
> > > > Traffic Management API - EXPERIMENTAL
> > > > M: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
> > > > T: git://dpdk.org/next/dpdk-next-qos
> > > > F: lib/librte_ethdev/rte_tm*
> > > > > > > Ray, Neil, David, Luca, Kevin, what do you think?
> > While I'm not called any of those names, allow me to give my 2c.
> > 
> > Since these are marked in binaries as part of the stable ABI, I think we
> > need to honour that for the next two releases 20.05 and 20.08 [which means
> > that we need to put in versioned functions for any changes, not that we
> > can't change anything]
> > 
> > For 20.11, I think these should then have one of two options taken:
> > * have these "fixed" and ready to be marked as stable, and officially part
> >   of v21 ABI or
> > * mark them as experimental properly, and look to have them as part of the
> >   v22 or subsequent ABI
> > 
> > Given the comments here, I would tend towards the latter of the above two
> > options, but that's really a decision for the maintainers.
> > 
> > Remember, this is not the first bug we have encountered where we messed up
> > some ABI versions in the 19.11 release, and, like the previous one with the
> > screwed up version number, I think we need to honour the ABI committments
> > made, especially since in this case it's only for a few more months till
> > 20.11 development starts.
> > 
> > /Bruce
> 
> +1
> 
> If they are not ready now, they haven't been ready for the past 6
> months either, so staying not ready for 6 more is the lesser evil.

This API is almost 3 years old (release 17.08).
That's good to improve it but we must respect the ABI contract that
we all agreed.


Summary:
17.08: rte_tm is introduced.
17.11: rte_mtr is introduced as experimental, but rte_tm remains stable.
18.02: __rte_experimental tag is introduced (including for rte_mtr),
but rte_tm remains untouched as it is in stable ABI.
19.11: stable ABI is frozen until 20.11
20.05: rte_tm improvement is blocked because of ABI breakage.


It should remind everybody of reviewing the new API and policies,
and maintaining the existing code appropriately.