[PATCH] doc: update TAP device features

[PATCH] doc: update TAP device features

[Stephen Hemminger]

The TAP device does have per-queue stats and handles multi-process.

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
---
 doc/guides/nics/features/tap.ini | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/doc/guides/nics/features/tap.ini b/doc/guides/nics/features/tap.ini
index f26355e57f..f2ea5cd833 100644
--- a/doc/guides/nics/features/tap.ini
+++ b/doc/guides/nics/features/tap.ini
@@ -14,10 +14,12 @@ Basic stats          = Y
 L3 checksum offload  = Y
 L4 checksum offload  = Y
 MTU update           = Y
+Multiprocess aware   = Y
 Multicast MAC filter = Y
 Unicast MAC filter   = Y
 Packet type parsing  = Y
 Flow control         = Y
+Stats per queue      = Y
 Linux                = Y
 ARMv7                = Y
 ARMv8                = Y
-- 
2.45.2

Re: [PATCH] doc: update TAP device features

[Ferruh Yigit]

On 9/4/2024 4:42 PM, Stephen Hemminger wrote:
> The TAP device does have per-queue stats and handles multi-process.
> 
> Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
> ---
>  doc/guides/nics/features/tap.ini | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/doc/guides/nics/features/tap.ini b/doc/guides/nics/features/tap.ini
> index f26355e57f..f2ea5cd833 100644
> --- a/doc/guides/nics/features/tap.ini
> +++ b/doc/guides/nics/features/tap.ini
> @@ -14,10 +14,12 @@ Basic stats          = Y
>  L3 checksum offload  = Y
>  L4 checksum offload  = Y
>  MTU update           = Y
> +Multiprocess aware   = Y
>

ack

>  Multicast MAC filter = Y
>  Unicast MAC filter   = Y
>  Packet type parsing  = Y
>  Flow control         = Y
> +Stats per queue      = Y
>

This feature name is misleading,
it is for 'rte_eth_dev_set_[rt]x_queue_stats_mapping()' API, which is
indeed for covering limitation for some drivers.
Tap does support getting stats per queue, but doesn't support above
documented feature.

>  Linux                = Y
>  ARMv7                = Y
>  ARMv8                = Y

Re: [PATCH] doc: update TAP device features

[Stephen Hemminger]

On Fri, 4 Oct 2024 02:48:21 +0100
Ferruh Yigit <ferruh.yigit@amd.com> wrote:

> On 9/4/2024 4:42 PM, Stephen Hemminger wrote:
> > The TAP device does have per-queue stats and handles multi-process.
> > 
> > Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
> > ---
> >  doc/guides/nics/features/tap.ini | 2 ++
> >  1 file changed, 2 insertions(+)
> > 
> > diff --git a/doc/guides/nics/features/tap.ini b/doc/guides/nics/features/tap.ini
> > index f26355e57f..f2ea5cd833 100644
> > --- a/doc/guides/nics/features/tap.ini
> > +++ b/doc/guides/nics/features/tap.ini
> > @@ -14,10 +14,12 @@ Basic stats          = Y
> >  L3 checksum offload  = Y
> >  L4 checksum offload  = Y
> >  MTU update           = Y
> > +Multiprocess aware   = Y
> >  
> 
> ack
> 
> >  Multicast MAC filter = Y
> >  Unicast MAC filter   = Y
> >  Packet type parsing  = Y
> >  Flow control         = Y
> > +Stats per queue      = Y
> >  
> 
> This feature name is misleading,
> it is for 'rte_eth_dev_set_[rt]x_queue_stats_mapping()' API, which is
> indeed for covering limitation for some drivers.
> Tap does support getting stats per queue, but doesn't support above
> documented feature.

The stats queue mapping was a feature that was hinted at being removed.
It only exists because of HW limitations on Intel ixgbe NIC and SW
limitations from RTE_ETHDEV_QUEUE_STAT_CNTRS.

Perhaps there should be a generic SW emulation for this the mapping?

Re: [PATCH] doc: update TAP device features

[Ferruh Yigit]

On 10/4/2024 3:26 AM, Stephen Hemminger wrote:
> On Fri, 4 Oct 2024 02:48:21 +0100
> Ferruh Yigit <ferruh.yigit@amd.com> wrote:
> 
>> On 9/4/2024 4:42 PM, Stephen Hemminger wrote:
>>> The TAP device does have per-queue stats and handles multi-process.
>>>
>>> Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
>>> ---
>>>  doc/guides/nics/features/tap.ini | 2 ++
>>>  1 file changed, 2 insertions(+)
>>>
>>> diff --git a/doc/guides/nics/features/tap.ini b/doc/guides/nics/features/tap.ini
>>> index f26355e57f..f2ea5cd833 100644
>>> --- a/doc/guides/nics/features/tap.ini
>>> +++ b/doc/guides/nics/features/tap.ini
>>> @@ -14,10 +14,12 @@ Basic stats          = Y
>>>  L3 checksum offload  = Y
>>>  L4 checksum offload  = Y
>>>  MTU update           = Y
>>> +Multiprocess aware   = Y
>>>  
>>
>> ack
>>
>>>  Multicast MAC filter = Y
>>>  Unicast MAC filter   = Y
>>>  Packet type parsing  = Y
>>>  Flow control         = Y
>>> +Stats per queue      = Y
>>>  
>>
>> This feature name is misleading,
>> it is for 'rte_eth_dev_set_[rt]x_queue_stats_mapping()' API, which is
>> indeed for covering limitation for some drivers.
>> Tap does support getting stats per queue, but doesn't support above
>> documented feature.
> 
> The stats queue mapping was a feature that was hinted at being removed.
> It only exists because of HW limitations on Intel ixgbe NIC and SW
> limitations from RTE_ETHDEV_QUEUE_STAT_CNTRS.
> 


We have a plan to remove 'RTE_ETHDEV_QUEUE_STAT_CNTRS', by moving queue
stats to xstats.

But ixgbe limitation is there.

> Perhaps there should be a generic SW emulation for this the mapping?
>

Ack, cc'ed Bruce.
But I am not sure ROI of the effort at this stage.

Re: [PATCH] doc: update TAP device features

[Bruce Richardson]

On Fri, Oct 04, 2024 at 05:09:21AM +0100, Ferruh Yigit wrote:
> On 10/4/2024 3:26 AM, Stephen Hemminger wrote:
> > On Fri, 4 Oct 2024 02:48:21 +0100
> > Ferruh Yigit <ferruh.yigit@amd.com> wrote:
> > 
> >> On 9/4/2024 4:42 PM, Stephen Hemminger wrote:
> >>> The TAP device does have per-queue stats and handles multi-process.
> >>>
> >>> Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
> >>> ---
> >>>  doc/guides/nics/features/tap.ini | 2 ++
> >>>  1 file changed, 2 insertions(+)
> >>>
> >>> diff --git a/doc/guides/nics/features/tap.ini b/doc/guides/nics/features/tap.ini
> >>> index f26355e57f..f2ea5cd833 100644
> >>> --- a/doc/guides/nics/features/tap.ini
> >>> +++ b/doc/guides/nics/features/tap.ini
> >>> @@ -14,10 +14,12 @@ Basic stats          = Y
> >>>  L3 checksum offload  = Y
> >>>  L4 checksum offload  = Y
> >>>  MTU update           = Y
> >>> +Multiprocess aware   = Y
> >>>  
> >>
> >> ack
> >>
> >>>  Multicast MAC filter = Y
> >>>  Unicast MAC filter   = Y
> >>>  Packet type parsing  = Y
> >>>  Flow control         = Y
> >>> +Stats per queue      = Y
> >>>  
> >>
> >> This feature name is misleading,
> >> it is for 'rte_eth_dev_set_[rt]x_queue_stats_mapping()' API, which is
> >> indeed for covering limitation for some drivers.
> >> Tap does support getting stats per queue, but doesn't support above
> >> documented feature.
> > 
> > The stats queue mapping was a feature that was hinted at being removed.
> > It only exists because of HW limitations on Intel ixgbe NIC and SW
> > limitations from RTE_ETHDEV_QUEUE_STAT_CNTRS.
> > 
> 
> 
> We have a plan to remove 'RTE_ETHDEV_QUEUE_STAT_CNTRS', by moving queue
> stats to xstats.
> 
> But ixgbe limitation is there.
> 
> > Perhaps there should be a generic SW emulation for this the mapping?
> >
> 
> Ack, cc'ed Bruce.
> But I am not sure ROI of the effort at this stage.

Not sure what the specific ask for me is here. :-) Overall, I think moving
queue stats to xstats is the best way to go.

/Bruce

Re: [PATCH] doc: update TAP device features

[Stephen Hemminger]

On Fri, 4 Oct 2024 08:54:43 +0100
Bruce Richardson <bruce.richardson@intel.com> wrote:

> On Fri, Oct 04, 2024 at 05:09:21AM +0100, Ferruh Yigit wrote:
> > On 10/4/2024 3:26 AM, Stephen Hemminger wrote:  
> > > On Fri, 4 Oct 2024 02:48:21 +0100
> > > Ferruh Yigit <ferruh.yigit@amd.com> wrote:
> > >   
> > >> On 9/4/2024 4:42 PM, Stephen Hemminger wrote:  
> > >>> The TAP device does have per-queue stats and handles multi-process.
> > >>>
> > >>> Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
> > >>> ---
> > >>>  doc/guides/nics/features/tap.ini | 2 ++
> > >>>  1 file changed, 2 insertions(+)
> > >>>
> > >>> diff --git a/doc/guides/nics/features/tap.ini b/doc/guides/nics/features/tap.ini
> > >>> index f26355e57f..f2ea5cd833 100644
> > >>> --- a/doc/guides/nics/features/tap.ini
> > >>> +++ b/doc/guides/nics/features/tap.ini
> > >>> @@ -14,10 +14,12 @@ Basic stats          = Y
> > >>>  L3 checksum offload  = Y
> > >>>  L4 checksum offload  = Y
> > >>>  MTU update           = Y
> > >>> +Multiprocess aware   = Y
> > >>>    
> > >>
> > >> ack
> > >>  
> > >>>  Multicast MAC filter = Y
> > >>>  Unicast MAC filter   = Y
> > >>>  Packet type parsing  = Y
> > >>>  Flow control         = Y
> > >>> +Stats per queue      = Y
> > >>>    
> > >>
> > >> This feature name is misleading,
> > >> it is for 'rte_eth_dev_set_[rt]x_queue_stats_mapping()' API, which is
> > >> indeed for covering limitation for some drivers.
> > >> Tap does support getting stats per queue, but doesn't support above
> > >> documented feature.  
> > > 
> > > The stats queue mapping was a feature that was hinted at being removed.
> > > It only exists because of HW limitations on Intel ixgbe NIC and SW
> > > limitations from RTE_ETHDEV_QUEUE_STAT_CNTRS.
> > >   
> > 
> > 
> > We have a plan to remove 'RTE_ETHDEV_QUEUE_STAT_CNTRS', by moving queue
> > stats to xstats.
> > 
> > But ixgbe limitation is there.
> >   
> > > Perhaps there should be a generic SW emulation for this the mapping?
> > >  
> > 
> > Ack, cc'ed Bruce.
> > But I am not sure ROI of the effort at this stage.  
> 
> Not sure what the specific ask for me is here. :-) Overall, I think moving
> queue stats to xstats is the best way to go.
> 
> /Bruce

The problem is that xstats are free form, so it is not possible for application
to use them in device independent manner.

Re: [PATCH] doc: update TAP device features

[Ferruh Yigit]

On 10/4/2024 8:54 AM, Bruce Richardson wrote:
> On Fri, Oct 04, 2024 at 05:09:21AM +0100, Ferruh Yigit wrote:
>> On 10/4/2024 3:26 AM, Stephen Hemminger wrote:
>>> On Fri, 4 Oct 2024 02:48:21 +0100
>>> Ferruh Yigit <ferruh.yigit@amd.com> wrote:
>>>
>>>> On 9/4/2024 4:42 PM, Stephen Hemminger wrote:
>>>>> The TAP device does have per-queue stats and handles multi-process.
>>>>>
>>>>> Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
>>>>> ---
>>>>>  doc/guides/nics/features/tap.ini | 2 ++
>>>>>  1 file changed, 2 insertions(+)
>>>>>
>>>>> diff --git a/doc/guides/nics/features/tap.ini b/doc/guides/nics/features/tap.ini
>>>>> index f26355e57f..f2ea5cd833 100644
>>>>> --- a/doc/guides/nics/features/tap.ini
>>>>> +++ b/doc/guides/nics/features/tap.ini
>>>>> @@ -14,10 +14,12 @@ Basic stats          = Y
>>>>>  L3 checksum offload  = Y
>>>>>  L4 checksum offload  = Y
>>>>>  MTU update           = Y
>>>>> +Multiprocess aware   = Y
>>>>>  
>>>>
>>>> ack
>>>>
>>>>>  Multicast MAC filter = Y
>>>>>  Unicast MAC filter   = Y
>>>>>  Packet type parsing  = Y
>>>>>  Flow control         = Y
>>>>> +Stats per queue      = Y
>>>>>  
>>>>
>>>> This feature name is misleading,
>>>> it is for 'rte_eth_dev_set_[rt]x_queue_stats_mapping()' API, which is
>>>> indeed for covering limitation for some drivers.
>>>> Tap does support getting stats per queue, but doesn't support above
>>>> documented feature.
>>>
>>> The stats queue mapping was a feature that was hinted at being removed.
>>> It only exists because of HW limitations on Intel ixgbe NIC and SW
>>> limitations from RTE_ETHDEV_QUEUE_STAT_CNTRS.
>>>
>>
>>
>> We have a plan to remove 'RTE_ETHDEV_QUEUE_STAT_CNTRS', by moving queue
>> stats to xstats.
>>
>> But ixgbe limitation is there.
>>
>>> Perhaps there should be a generic SW emulation for this the mapping?
>>>
>>
>> Ack, cc'ed Bruce.
>> But I am not sure ROI of the effort at this stage.
> 
> Not sure what the specific ask for me is here. :-) Overall, I think moving
> queue stats to xstats is the best way to go.
> 

cc'ed because of "generic SW emulation" comment.

I was thinking if this mapping can be done transparent to the user by
driver mapping queue <-> stats_register before reading stats, but @Bruce
let me know this won't work because the stats tracking only happens
after the mapping.

@Stephen, do you have something specific in your mind for SW emulation
for mapping?

Per queue stats

[Stephen Hemminger]

...
> >>> The stats queue mapping was a feature that was hinted at being removed.
> >>> It only exists because of HW limitations on Intel ixgbe NIC and SW
> >>> limitations from RTE_ETHDEV_QUEUE_STAT_CNTRS.
> >>>  
> >>
> >>
> >> We have a plan to remove 'RTE_ETHDEV_QUEUE_STAT_CNTRS', by moving queue
> >> stats to xstats.
> >>
> >> But ixgbe limitation is there.
> >>  
> >>> Perhaps there should be a generic SW emulation for this the mapping?
> >>>  
> >>
> >> Ack, cc'ed Bruce.
> >> But I am not sure ROI of the effort at this stage.  
> > 
> > Not sure what the specific ask for me is here. :-) Overall, I think moving
> > queue stats to xstats is the best way to go.
> >   
> 
> cc'ed because of "generic SW emulation" comment.
> 
> I was thinking if this mapping can be done transparent to the user by
> driver mapping queue <-> stats_register before reading stats, but @Bruce
> let me know this won't work because the stats tracking only happens
> after the mapping.
> 
> @Stephen, do you have something specific in your mind for SW emulation
> for mapping?

Probably not worth doing mapping tables for SW drivers.

My preference would be something like:
	1. Introduce new API's rte_ethdev_get_queue_stats() and rte_ethdev_reset_queue_stats().
           with callbacks in ethdev ops.
           if driver does not support this, then do a shim in ethdev that uses old stats fields.
        2. Mark the stats mapping API's as deprecated and always return error.
           Making special case for ixgbe at this point really is not worth maintaining.
        3. After all drivers converted over, drop the fields in ethdev stats for queues.
           And fix the examples, etc. Yes this would be a big API change.

Would take a year to get done (ie 25.11).

Re: [PATCH] doc: update TAP device features

[Stephen Hemminger]

On Fri, 4 Oct 2024 02:48:21 +0100
Ferruh Yigit <ferruh.yigit@amd.com> wrote:

> > +Stats per queue      = Y
> >  
> 
> This feature name is misleading,
> it is for 'rte_eth_dev_set_[rt]x_queue_stats_mapping()' API, which is
> indeed for covering limitation for some drivers.
> Tap does support getting stats per queue, but doesn't support above
> documented feature.

Just looked and the current state of the feature matrix is not correct
for this part of the table. Although the feature intro says it is
stats per queue mapping, that is not what other drivers are doing.

Only these drivers have queue_stats_mapping_set:
  cnxk, enic, igc, ixgbe, txgbe, virtio

These drivers claim to have 'Stats per queue' but do not have stats mapping:
  af_xdp, ark, atlantic, avp, bnxt, cxgbe, failsafe, fm10, hinic, hns3,
  ionic, mlx4, mlx5, netvsc, nfb, nfp, ngbe, qede, thunderx, vmxnet3

So the overwhelming practice is that the features.rst description is wrong!