[dpdk-dev] Rx/Tx offloads checks behaviour in 18.05

[dpdk-dev] Rx/Tx offloads checks behaviour in 18.05

[Andrew Rybchenko]

Hi all,

I think that Rx/Tx offloads checks behaviour is inconsistent in next-net 
as of today.

Consistency checks are removed from PMDs and substituted with error logs 
in ethdev.
Basically application which is not switched to new offload API has no 
way to find out if,
for example, Rx scatter is supported. Rx scatter offload was introduced 
in 17.11 to
substitute corresponding flag in device Rx mode.

Not updated application could try to enable Rx scatter on device 
configure and
get failure if it is not supported.  Yes it is not fine-grained and it 
could be numerous
reasons behind the configure failure. With 18.05 configure will pass and 
moreover
hardware may be configured to do Rx scatter despite of no real support 
in PMD.
Consequences could be really different from simply dropping scattered 
packet or
delivery of truncated packets to spoiling of memory etc.

Similar could happen with multi-segment packet on Tx. Application configures
Tx queue without NOMULTISEG flag, TxQ setup passes (with error log that
multi-segment is not supported, but it is just an error log) and 
application generates
multi-segment packets which are simply truncated (if the first segment 
length is used
as packet length on transmit) or garbage is sent (if total packet length 
is used, i.e.
possible disclosure of security sensitive information since it could be 
data from
neighbour packet).

I think we have the following options:

A. Rollback corresponding changes which remove checks from PMDs
     (at least some PMDs will be not affected).

B. Fail configure if unsupported offload is requested (all PMDs must be 
converted
     in the release, so reporting of supported offloads must be correct) AND
     add check that offloads requested on Tx queue level (derived from 
txq_flags)
     are supported at least somewhere (i.e. tx_offload_capa)

C. Describe the behaviour changes in release notes to try to make it at 
least
     create for DPDK users. I don't like the option at all.

Any other ideas?

I would vote for B since it is a step forward and even if it makes some apps
to fail I think it is better than consequences of missing checks.
I'll make a patch for option B and test it meanwhile.

Andrew

Re: [dpdk-dev] Rx/Tx offloads checks behaviour in 18.05

[Ferruh Yigit]

On 5/11/2018 8:07 AM, Andrew Rybchenko wrote:
> Hi all,
> 
> I think that Rx/Tx offloads checks behaviour is inconsistent in next-net 
> as of today.
> 
> Consistency checks are removed from PMDs and substituted with error logs 
> in ethdev.

Yes.

> Basically application which is not switched to new offload API has no 
> way to find out if,
> for example, Rx scatter is supported. Rx scatter offload was introduced 
> in 17.11 to
> substitute corresponding flag in device Rx mode.
> 
> Not updated application could try to enable Rx scatter on device 
> configure and
> get failure if it is not supported.  Yes it is not fine-grained and it 
> could be numerous
> reasons behind the configure failure. With 18.05 configure will pass and 
> moreover
> hardware may be configured to do Rx scatter despite of no real support 
> in PMD.
> Consequences could be really different from simply dropping scattered 
> packet or
> delivery of truncated packets to spoiling of memory etc.
> 
> Similar could happen with multi-segment packet on Tx. Application configures
> Tx queue without NOMULTISEG flag, TxQ setup passes (with error log that
> multi-segment is not supported, but it is just an error log) and 
> application generates
> multi-segment packets which are simply truncated (if the first segment 
> length is used
> as packet length on transmit) or garbage is sent (if total packet length 
> is used, i.e.
> possible disclosure of security sensitive information since it could be 
> data from
> neighbour packet).

How common these error cases do you think?

> 
> I think we have the following options:
> 
> A. Rollback corresponding changes which remove checks from PMDs
>      (at least some PMDs will be not affected).
> 
> B. Fail configure if unsupported offload is requested (all PMDs must be 
> converted
>      in the release, so reporting of supported offloads must be correct) AND
>      add check that offloads requested on Tx queue level (derived from 
> txq_flags)
>      are supported at least somewhere (i.e. tx_offload_capa)

Issue is not PMDs, they should support new offload API. Concern is breaking
application which is out of our control.

With current approach some old application may request invalid offload and PMD
won't return an error to app, agreed this is a concern.
But adding error returns will break same applications, in a better more obvious
way, and has possibility to break more applications, ones really not concerned
about offload may be hit as well.

> 
> C. Describe the behaviour changes in release notes to try to make it at 
> least
>      create for DPDK users. I don't like the option at all.
> 
> Any other ideas?
> 
> I would vote for B since it is a step forward and even if it makes some apps
> to fail I think it is better than consequences of missing checks.
> I'll make a patch for option B and test it meanwhile.
> 
> Andrew
> 

Re: [dpdk-dev] Rx/Tx offloads checks behaviour in 18.05

[Andrew Rybchenko]

On 05/11/2018 05:22 PM, Ferruh Yigit wrote:
> On 5/11/2018 8:07 AM, Andrew Rybchenko wrote:
>> Hi all,
>>
>> I think that Rx/Tx offloads checks behaviour is inconsistent in next-net
>> as of today.
>>
>> Consistency checks are removed from PMDs and substituted with error logs
>> in ethdev.
> Yes.
>
>> Basically application which is not switched to new offload API has no
>> way to find out if,
>> for example, Rx scatter is supported. Rx scatter offload was introduced
>> in 17.11 to
>> substitute corresponding flag in device Rx mode.
>>
>> Not updated application could try to enable Rx scatter on device
>> configure and
>> get failure if it is not supported.  Yes it is not fine-grained and it
>> could be numerous
>> reasons behind the configure failure. With 18.05 configure will pass and
>> moreover
>> hardware may be configured to do Rx scatter despite of no real support
>> in PMD.
>> Consequences could be really different from simply dropping scattered
>> packet or
>> delivery of truncated packets to spoiling of memory etc.
>>
>> Similar could happen with multi-segment packet on Tx. Application configures
>> Tx queue without NOMULTISEG flag, TxQ setup passes (with error log that
>> multi-segment is not supported, but it is just an error log) and
>> application generates
>> multi-segment packets which are simply truncated (if the first segment
>> length is used
>> as packet length on transmit) or garbage is sent (if total packet length
>> is used, i.e.
>> possible disclosure of security sensitive information since it could be
>> data from
>> neighbour packet).
> How common these error cases do you think?

I don't know. My fear is that consequences are really bad and it is a
regression since checks from PMDs are removed.

>> I think we have the following options:
>>
>> A. Rollback corresponding changes which remove checks from PMDs
>>       (at least some PMDs will be not affected).
>>
>> B. Fail configure if unsupported offload is requested (all PMDs must be
>> converted
>>       in the release, so reporting of supported offloads must be correct) AND
>>       add check that offloads requested on Tx queue level (derived from
>> txq_flags)
>>       are supported at least somewhere (i.e. tx_offload_capa)
> Issue is not PMDs, they should support new offload API. Concern is breaking
> application which is out of our control.
>
> With current approach some old application may request invalid offload and PMD
> won't return an error to app, agreed this is a concern.
> But adding error returns will break same applications, in a better more obvious
> way, and has possibility to break more applications, ones really not concerned
> about offload may be hit as well.

It depends on which PMD is used. Yes, it was no checks in ethdev before.
If PMD does not support multi-segment Tx, some checksum or VLAN
insertion offload, but application requests it and rely on it, it will 
result in
invalid packets sent to network.

I realize that some applications may simply use empty txq_flags, but do
not use any offloads in fact. If so, such PMDs will fail to setup TxQ if
checks are made fatal, return error and underlying PMD does not
  support these offloads.

At least it is safer behaviour than transmitting garbage.
Yes, not easy decision.

I will publish my patches which passed our tests.

>> C. Describe the behaviour changes in release notes to try to make it at
>> least
>>       create for DPDK users. I don't like the option at all.
>>
>> Any other ideas?
>>
>> I would vote for B since it is a step forward and even if it makes some apps
>> to fail I think it is better than consequences of missing checks.
>> I'll make a patch for option B and test it meanwhile.
>>
>> Andrew
>>

Re: [dpdk-dev] Rx/Tx offloads checks behaviour in 18.05

[Shahaf Shuler]

Friday, May 11, 2018 7:09 PM, Andrew Rybchenko:
On 05/11/2018 05:22 PM, Ferruh Yigit wrote:

I think we have the following options:



A. Rollback corresponding changes which remove checks from PMDs

     (at least some PMDs will be not affected).



B. Fail configure if unsupported offload is requested (all PMDs must be

converted

     in the release, so reporting of supported offloads must be correct) AND

     add check that offloads requested on Tx queue level (derived from

txq_flags)

     are supported at least somewhere (i.e. tx_offload_capa)



        Issue is not PMDs, they should support new offload API. Concern is breaking

         application which is out of our control.



        With current approach some old application may request invalid offload and PMD

        won't return an error to app, agreed this is a concern.

        But adding error returns will break same applications, in a better more obvious

        way, and has possibility to break more applications, ones really not concerned

        about offload may be hit as well.

It depends on which PMD is used. Yes, it was no checks in ethdev before.
If PMD does not support multi-segment Tx, some checksum or VLAN
insertion offload, but application requests it and rely on it, it will result in
invalid packets sent to network.

I realize that some applications may simply use empty txq_flags, but do
not use any offloads in fact. If so, such PMDs will fail to setup TxQ if
checks are made fatal, return error and underlying PMD does not
 support these offloads.

At least it is safer behaviour than transmitting garbage.
Yes, not easy decision.

I will publish my patches which passed our tests.

I agree with Andrew here. Even though there is a concern about the existing application we cannot use only logging.
It is better to enforce the right behavior rather than having wrong configuration silently accepted by the PMD.

Re: [dpdk-dev] Rx/Tx offloads checks behaviour in 18.05

[Ferruh Yigit]

On 5/13/2018 6:30 AM, Shahaf Shuler wrote:
> Friday, May 11, 2018 7:09 PM, Andrew Rybchenko:
> 
> On 05/11/2018 05:22 PM, Ferruh Yigit wrote:
> 
>         I think we have the following options:
> 
>          
> 
>         A. Rollback corresponding changes which remove checks from PMDs
> 
>              (at least some PMDs will be not affected).
> 
>          
> 
>         B. Fail configure if unsupported offload is requested (all PMDs must be 
> 
>         converted
> 
>              in the release, so reporting of supported offloads must be correct) AND
> 
>              add check that offloads requested on Tx queue level (derived from 
> 
>         txq_flags)
> 
>              are supported at least somewhere (i.e. tx_offload_capa)
> 
>      
> 
>             Issue is not PMDs, they should support new offload API. Concern is breaking
> 
>             application which is out of our control.
> 
>             With current approach some old application may request invalid offload and PMD
> 
>             won't return an error to app, agreed this is a concern.
> 
>             But adding error returns will break same applications, in a better more obvious
> 
>             way, and has possibility to break more applications, ones really not concerned
> 
>             about offload may be hit as well.
> 
> 
> It depends on which PMD is used. Yes, it was no checks in ethdev before.
> If PMD does not support multi-segment Tx, some checksum or VLAN
> insertion offload, but application requests it and rely on it, it will result in
> invalid packets sent to network.
> 
> I realize that some applications may simply use empty txq_flags, but do
> not use any offloads in fact. If so, such PMDs will fail to setup TxQ if
> checks are made fatal, return error and underlying PMD does not
>  support these offloads.
> 
> At least it is safer behaviour than transmitting garbage.
> Yes, not easy decision.
> 
> I will publish my patches which passed our tests.
> 
>  
> 
> I agree with Andrew here. Even though there is a concern about the existing
> application we cannot use only logging.
> 
> It is better to enforce the right behavior rather than having wrong
> configuration silently accepted by the PMD.

What do you think send an error for the application that switch to new
offloading API and only print error log for the old ones. I believe we can
detect this via ETH_TXQ_FLAGS_IGNORE and ignore_offload_bitfield.

And we will already force old applications to switch to new API next release,
next release we can introduce return error without condition.
Does it make sense?

Re: [dpdk-dev] Rx/Tx offloads checks behaviour in 18.05

[Shahaf Shuler]

Monday, May 14, 2018 1:00 PM, Ferruh Yigit:
> >
> > It depends on which PMD is used. Yes, it was no checks in ethdev before.
> > If PMD does not support multi-segment Tx, some checksum or VLAN
> > insertion offload, but application requests it and rely on it, it will
> > result in invalid packets sent to network.
> >
> > I realize that some applications may simply use empty txq_flags, but
> > do not use any offloads in fact. If so, such PMDs will fail to setup
> > TxQ if checks are made fatal, return error and underlying PMD does not
> >  support these offloads.
> >
> > At least it is safer behaviour than transmitting garbage.
> > Yes, not easy decision.
> >
> > I will publish my patches which passed our tests.
> >
> >
> >
> > I agree with Andrew here. Even though there is a concern about the
> > existing application we cannot use only logging.
> >
> > It is better to enforce the right behavior rather than having wrong
> > configuration silently accepted by the PMD.
> 
> What do you think send an error for the application that switch to new
> offloading API and only print error log for the old ones. I believe we can
> detect this via ETH_TXQ_FLAGS_IGNORE and ignore_offload_bitfield.
> 
> And we will already force old applications to switch to new API next release,
> next release we can introduce return error without condition.
> Does it make sense?

The issue is currently none of the PMDs verify the offloads v.s. the caps as ethdev layer does it. 
We cannot have offload to be set if it is not supported, even under the old API. allowing such breach on ethdev means we need to enforce it back again on each PMD. 

Re: [dpdk-dev] Rx/Tx offloads checks behaviour in 18.05

[Ferruh Yigit]

On 5/14/2018 11:19 AM, Shahaf Shuler wrote:
> Monday, May 14, 2018 1:00 PM, Ferruh Yigit:
>>>
>>> It depends on which PMD is used. Yes, it was no checks in ethdev before.
>>> If PMD does not support multi-segment Tx, some checksum or VLAN
>>> insertion offload, but application requests it and rely on it, it will
>>> result in invalid packets sent to network.
>>>
>>> I realize that some applications may simply use empty txq_flags, but
>>> do not use any offloads in fact. If so, such PMDs will fail to setup
>>> TxQ if checks are made fatal, return error and underlying PMD does not
>>>  support these offloads.
>>>
>>> At least it is safer behaviour than transmitting garbage.
>>> Yes, not easy decision.
>>>
>>> I will publish my patches which passed our tests.
>>>
>>>
>>>
>>> I agree with Andrew here. Even though there is a concern about the
>>> existing application we cannot use only logging.
>>>
>>> It is better to enforce the right behavior rather than having wrong
>>> configuration silently accepted by the PMD.
>>
>> What do you think send an error for the application that switch to new
>> offloading API and only print error log for the old ones. I believe we can
>> detect this via ETH_TXQ_FLAGS_IGNORE and ignore_offload_bitfield.
>>
>> And we will already force old applications to switch to new API next release,
>> next release we can introduce return error without condition.
>> Does it make sense?
> 
> The issue is currently none of the PMDs verify the offloads v.s. the caps as ethdev layer does it. 
> We cannot have offload to be set if it is not supported, even under the old API. allowing such breach on ethdev means we need to enforce it back again on each PMD. 

Hi Shahaf, Andrew,

Thank you for sharing your concern I agree this is a valid one and thanks for
the solution provided.

Ethdev configure() and queue_setup() are two common functions used for almost
all (if not all) DPDK networking applications, independent from they are
interested in offloading or not.
I am not comfortable on possibility to break them when we already scheduled a
break next release withing the process.

With Wei's latest updates within comment it mentions return error will be added
in next release, this is good from documenting intention point of view.


With taking the pressure to close the v18.05 I am for getting offloading patch
as it is and reject Andrew's update.
And do new offloading API related work, including adding these returns, early in
next release (18.08)

Thanks,
ferruh

Re: [dpdk-dev] Rx/Tx offloads checks behaviour in 18.05

[Thomas Monjalon]

14/05/2018 16:10, Ferruh Yigit:
> On 5/14/2018 11:19 AM, Shahaf Shuler wrote:
> > Monday, May 14, 2018 1:00 PM, Ferruh Yigit:
> >>>
> >>> It depends on which PMD is used. Yes, it was no checks in ethdev before.
> >>> If PMD does not support multi-segment Tx, some checksum or VLAN
> >>> insertion offload, but application requests it and rely on it, it will
> >>> result in invalid packets sent to network.
> >>>
> >>> I realize that some applications may simply use empty txq_flags, but
> >>> do not use any offloads in fact. If so, such PMDs will fail to setup
> >>> TxQ if checks are made fatal, return error and underlying PMD does not
> >>>  support these offloads.
> >>>
> >>> At least it is safer behaviour than transmitting garbage.
> >>> Yes, not easy decision.
> >>>
> >>> I will publish my patches which passed our tests.
> >>>
> >>>
> >>>
> >>> I agree with Andrew here. Even though there is a concern about the
> >>> existing application we cannot use only logging.
> >>>
> >>> It is better to enforce the right behavior rather than having wrong
> >>> configuration silently accepted by the PMD.
> >>
> >> What do you think send an error for the application that switch to new
> >> offloading API and only print error log for the old ones. I believe we can
> >> detect this via ETH_TXQ_FLAGS_IGNORE and ignore_offload_bitfield.
> >>
> >> And we will already force old applications to switch to new API next release,
> >> next release we can introduce return error without condition.
> >> Does it make sense?
> > 
> > The issue is currently none of the PMDs verify the offloads v.s. the caps as ethdev layer does it. 
> > We cannot have offload to be set if it is not supported, even under the old API. allowing such breach on ethdev means we need to enforce it back again on each PMD. 
> 
> Hi Shahaf, Andrew,
> 
> Thank you for sharing your concern I agree this is a valid one and thanks for
> the solution provided.
> 
> Ethdev configure() and queue_setup() are two common functions used for almost
> all (if not all) DPDK networking applications, independent from they are
> interested in offloading or not.
> I am not comfortable on possibility to break them when we already scheduled a
> break next release withing the process.
> 
> With Wei's latest updates within comment it mentions return error will be added
> in next release, this is good from documenting intention point of view.
> 
> 
> With taking the pressure to close the v18.05 I am for getting offloading patch
> as it is and reject Andrew's update.
> And do new offloading API related work, including adding these returns, early in
> next release (18.08)

It is a really difficult decision, but I agree with Ferruh.
We discussed it and we cannot take more risk in 18.05, even if it is
a bad choice from API perspective.

Thank you Andrew, and let's fix it in early days of 18.08.