[dpdk-dev] The mbuf API needs some cleaning up

[dpdk-dev] The mbuf API needs some cleaning up

[Morten Brørup]

The MBUF library exposes some macros and constants without the RTE_ prefix. I propose cleaning up these, so better names get into the coming LTS release.

The worst is:
#define MBUF_INVALID_PORT UINT16_MAX

I say it's the worst because when we were looking for the official "invalid" port value for our application, we didn't find this one. (Probably because its documentation is wrong.)

MBUF_INVALID_PORT is defined in rte_mbuf_core.h without any description, and in rte_mbuf.h, where it is injected between the rte_pktmbuf_reset() function and its description, so the API documentation shows the function's description for the constant, and no description for the function.

I propose keeping it at a sensible location in rte_mbuf_core.h only, adding a description, and renaming it to:
#define RTE_PORT_INVALID UINT16_MAX

For backwards compatibility, we could add:
/* this old name is deprecated */
#define MBUF_INVALID_PORT RTE_PORT_INVALID

I also wonder why there are no compiler warnings about the double definition?


There are also the data buffer location constants:
#define EXT_ATTACHED_MBUF    (1ULL << 61)
and
#define IND_ATTACHED_MBUF    (1ULL << 62)

There are already macros (with good names) for reading these, so simply adding the RTE_ prefix to these two constants suffices.


And all the packet offload flags, such as:
#define PKT_RX_VLAN          (1ULL << 0)

They are supposed to be used by applications, so I guess we should keep them unchanged for ABI stability reasons.


And the local macro:
#define MBUF_RAW_ALLOC_CHECK(m) do { \

This might as well be an internal inline function:
/* internal */
static inline void
__rte_mbuf_raw_alloc_check(const struct rte_mbuf *m)

Or we could keep it a macro and move it next to
__rte_mbuf_sanity_check(), keeping it clear that it is only relevant when RTE_LIBRTE_MBUF_DEBUG is set. But rename it to lower case, similar to the __rte_mbuf_sanity_check() macro.


Med venlig hilsen / kind regards
- Morten Brørup

Re: [dpdk-dev] The mbuf API needs some cleaning up

[Olivier Matz]

Hi Morten,

Thanks for the feedback.

On Mon, Jul 13, 2020 at 11:57:38AM +0200, Morten Brørup wrote:

> The MBUF library exposes some macros and constants without the RTE_ prefix. I
> propose cleaning up these, so better names get into the coming LTS release.

Yes, Thomas talked about it some time ago and he even drafted a patch to
fix it. We can target 20.11 for the changes, but I think we'll have to
keep a compat API until 21.11.

> The worst is:
> #define MBUF_INVALID_PORT UINT16_MAX
> 
> I say it's the worst because when we were looking for the official "invalid"
> port value for our application, we didn't find this one. (Probably because its
> documentation is wrong.)
>
> MBUF_INVALID_PORT is defined in rte_mbuf_core.h without any description, and
> in rte_mbuf.h, where it is injected between the rte_pktmbuf_reset() function
> and its description, so the API documentation shows the function's description
> for the constant, and no description for the function.

The one in rte_mbuf_core.h should be kept, with a documentation.

> I propose keeping it at a sensible location in rte_mbuf_core.h only, adding a description, and renaming it to:
> #define RTE_PORT_INVALID UINT16_MAX

I suggest RTE_MBUF_PORT_INVALID

> For backwards compatibility, we could add:
> /* this old name is deprecated */
> #define MBUF_INVALID_PORT RTE_PORT_INVALID
> 
> I also wonder why there are no compiler warnings about the double definition?

If the value is the same, the compiler won't complain.

> There are also the data buffer location constants:
> #define EXT_ATTACHED_MBUF    (1ULL << 61)
> and
> #define IND_ATTACHED_MBUF    (1ULL << 62)
> 
>
> There are already macros (with good names) for reading these, so
> simply adding the RTE_ prefix to these two constants suffices.

Some applications use it, we also need a compat here.

> And all the packet offload flags, such as:
> #define PKT_RX_VLAN          (1ULL << 0)
> 
>
> They are supposed to be used by applications, so I guess we should
> keep them unchanged for ABI stability reasons.

I propose RTE_MBUF_F_<name> for the mbuf flags.

> And the local macro:
> #define MBUF_RAW_ALLOC_CHECK(m) do { \
> 
> This might as well be an internal inline function:
> /* internal */
> static inline void
> __rte_mbuf_raw_alloc_check(const struct rte_mbuf *m)
> 

agree, I don't think a macro is mandatory here


Thanks,
Olivier


> Or we could keep it a macro and move it next to
> __rte_mbuf_sanity_check(), keeping it clear that it is only relevant when
> RTE_LIBRTE_MBUF_DEBUG is set. But rename it to lower case, similar to the
> __rte_mbuf_sanity_check() macro.
>
> 
> Med venlig hilsen / kind regards
> - Morten Brørup
> 

Re: [dpdk-dev] The mbuf API needs some cleaning up

[Morten Brørup]

> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Olivier Matz
> Sent: Friday, July 31, 2020 5:25 PM
> 
> Hi Morten,
> 
> Thanks for the feedback.
> 
> On Mon, Jul 13, 2020 at 11:57:38AM +0200, Morten Brørup wrote:
> 
> > The MBUF library exposes some macros and constants without the RTE_
> prefix. I
> > propose cleaning up these, so better names get into the coming LTS
> release.
> 
> Yes, Thomas talked about it some time ago and he even drafted a patch
> to
> fix it. We can target 20.11 for the changes, but I think we'll have to
> keep a compat API until 21.11.
> 

Great, then I will back off. No need for multiple patches fixing the same things. :-)

And I agree with all your feedback... although I do consider the mbuf port_id so much at the core of DPDK that I suggested RTE_PORT_INVALID over RTE_MBUF_PORT_INVALID, but I don't feel strongly about it. Whatever you and Thomas prefer is probably fine.

> > The worst is:
> > #define MBUF_INVALID_PORT UINT16_MAX
> >
> > I say it's the worst because when we were looking for the official
> "invalid"
> > port value for our application, we didn't find this one. (Probably
> because its
> > documentation is wrong.)
> >
> > MBUF_INVALID_PORT is defined in rte_mbuf_core.h without any
> description, and
> > in rte_mbuf.h, where it is injected between the rte_pktmbuf_reset()
> function
> > and its description, so the API documentation shows the function's
> description
> > for the constant, and no description for the function.
> 
> The one in rte_mbuf_core.h should be kept, with a documentation.
> 
> > I propose keeping it at a sensible location in rte_mbuf_core.h only,
> adding a description, and renaming it to:
> > #define RTE_PORT_INVALID UINT16_MAX
> 
> I suggest RTE_MBUF_PORT_INVALID
> 
> > For backwards compatibility, we could add:
> > /* this old name is deprecated */
> > #define MBUF_INVALID_PORT RTE_PORT_INVALID
> >
> > I also wonder why there are no compiler warnings about the double
> definition?
> 
> If the value is the same, the compiler won't complain.
> 
> > There are also the data buffer location constants:
> > #define EXT_ATTACHED_MBUF    (1ULL << 61)
> > and
> > #define IND_ATTACHED_MBUF    (1ULL << 62)
> >
> >
> > There are already macros (with good names) for reading these, so
> > simply adding the RTE_ prefix to these two constants suffices.
> 
> Some applications use it, we also need a compat here.
> 
> > And all the packet offload flags, such as:
> > #define PKT_RX_VLAN          (1ULL << 0)
> >
> >
> > They are supposed to be used by applications, so I guess we should
> > keep them unchanged for ABI stability reasons.
> 
> I propose RTE_MBUF_F_<name> for the mbuf flags.
> 
> > And the local macro:
> > #define MBUF_RAW_ALLOC_CHECK(m) do { \
> >
> > This might as well be an internal inline function:
> > /* internal */
> > static inline void
> > __rte_mbuf_raw_alloc_check(const struct rte_mbuf *m)
> >
> 
> agree, I don't think a macro is mandatory here
> 
> 
> Thanks,
> Olivier
> 
> 
> > Or we could keep it a macro and move it next to
> > __rte_mbuf_sanity_check(), keeping it clear that it is only relevant
> when
> > RTE_LIBRTE_MBUF_DEBUG is set. But rename it to lower case, similar to
> the
> > __rte_mbuf_sanity_check() macro.
> >
> >
> > Med venlig hilsen / kind regards
> > - Morten Brørup
> >

Re: [dpdk-dev] The mbuf API needs some cleaning up

[Thomas Monjalon]

03/08/2020 10:42, Morten Brørup:
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Olivier Matz
> > Sent: Friday, July 31, 2020 5:25 PM
> > 
> > Hi Morten,
> > 
> > Thanks for the feedback.
> > 
> > On Mon, Jul 13, 2020 at 11:57:38AM +0200, Morten Brørup wrote:
> > 
> > > The MBUF library exposes some macros and constants without the RTE_
> > prefix. I
> > > propose cleaning up these, so better names get into the coming LTS
> > release.
> > 
> > Yes, Thomas talked about it some time ago and he even drafted a patch
> > to
> > fix it. We can target 20.11 for the changes, but I think we'll have to
> > keep a compat API until 21.11.
> > 
> 
> Great, then I will back off. No need for multiple patches fixing the same things. :-)

Morten, you are very welcome to work on cleaning mbuf API.
I think we must focus on introducing the new names,
while keeping old names as alias for one year.

The other things to do is to provide a devtools script
to help converting applications to the new names.