[dpdk-dev] question regarding rx checksum offload flags

[dpdk-dev] question regarding rx checksum offload flags

[Lance Richardson]

I was looking for some clarification regarding how rx checksum
flags should be set for tunnel packets having both inner and outer
IP/L4 headers.

Based on comments in rte_mbuf_core.h, it seems to me. that the
inner (encapsulated) IP header checksum status should determine
which of these goes into ol_flags:
    PKT_RX_IP_CKSUM_UNKNOWN
    PKT_RX_IP_CKSUM_BAD
    PKT_RX_IP_CKSUM_GOOD
    PKT_RX_IP_CKSUM_NONE

Similarly, the L4 checksum status should determine which of these
goes into ol_flags:
   PKT_RX_L4_CKSUM_UNKNOWN
   PKT_RX_L4_CKSUM_BAD
   PKT_RX_L4_CKSUM_GOOD
   PKT_RX_L4_CKSUM_NONE

The IP header checksum status for the outer IP header should determine
whether this flag is set in ol_flags:
    PKT_RX_EIP_CKSUM_BAD

And for UDP-based tunnel encapsulations, the outer L4 checksum status
should determine which of these goes into ol_flags:
    PKT_RX_OUTER_L4_CKSUM_UNKNOWN
    PKT_RX_OUTER_L4_CKSUM_BAD
    PKT_RX_OUTER_L4_CKSUM_GOOD
    PKT_RX_OUTER_L4_CKSUM_INVALID

Finally, the checksum status of inner headers should have no influence
on PKT_RX_EIP_CKSUM_BAD or PKT_RX_OUTER_L4_CKSUM_*, and
likewise the checksum status of outer headers should have no influence
on PKT_RX_L4_CKSUM_* or PKT_RX_IP_CKSUM_*.

Is this correct? Apologies for such a basic question, but I'm having trouble
correlating the above with implementations.

Thanks and regards,
    Lance

Re: [dpdk-dev] question regarding rx checksum offload flags

[Lance Richardson]

On Mon, Aug 24, 2020 at 4:11 PM Lance Richardson
<lance.richardson@broadcom.com> wrote:
<snip>
> The IP header checksum status for the outer IP header should determine
> whether this flag is set in ol_flags:
>     PKT_RX_EIP_CKSUM_BAD

A simpler question might be simply "how is PKT_RX_EIP_CKSUM_BAD"
intended to be used?
It seems to be intended to be set if the outer IP checksum is not
correct.. do I have that right?

Thanks,
   Lance

Re: [dpdk-dev] question regarding rx checksum offload flags

[Lance Richardson]

On Tue, Aug 25, 2020 at 10:39 AM Lance Richardson
<lance.richardson@broadcom.com> wrote:
>
> On Mon, Aug 24, 2020 at 4:11 PM Lance Richardson
> <lance.richardson@broadcom.com> wrote:
> <snip>
> > The IP header checksum status for the outer IP header should determine
> > whether this flag is set in ol_flags:
> >     PKT_RX_EIP_CKSUM_BAD
>
> A simpler question might be simply "how is PKT_RX_EIP_CKSUM_BAD"
> intended to be used?
> It seems to be intended to be set if the outer IP checksum is not
> correct.. do I have that right?
>
> Thanks,
>    Lance

+Olivier

Re: [dpdk-dev] question regarding rx checksum offload flags

[Olivier Matz]

Hi Lance,

On Mon, Aug 24, 2020 at 04:11:45PM -0400, Lance Richardson wrote:
> I was looking for some clarification regarding how rx checksum
> flags should be set for tunnel packets having both inner and outer
> IP/L4 headers.
> 
> Based on comments in rte_mbuf_core.h, it seems to me. that the
> inner (encapsulated) IP header checksum status should determine
> which of these goes into ol_flags:
>     PKT_RX_IP_CKSUM_UNKNOWN
>     PKT_RX_IP_CKSUM_BAD
>     PKT_RX_IP_CKSUM_GOOD
>     PKT_RX_IP_CKSUM_NONE
> 
> Similarly, the L4 checksum status should determine which of these
> goes into ol_flags:
>    PKT_RX_L4_CKSUM_UNKNOWN
>    PKT_RX_L4_CKSUM_BAD
>    PKT_RX_L4_CKSUM_GOOD
>    PKT_RX_L4_CKSUM_NONE
> 
> The IP header checksum status for the outer IP header should determine
> whether this flag is set in ol_flags:
>     PKT_RX_EIP_CKSUM_BAD
> 
> And for UDP-based tunnel encapsulations, the outer L4 checksum status
> should determine which of these goes into ol_flags:
>     PKT_RX_OUTER_L4_CKSUM_UNKNOWN
>     PKT_RX_OUTER_L4_CKSUM_BAD
>     PKT_RX_OUTER_L4_CKSUM_GOOD
>     PKT_RX_OUTER_L4_CKSUM_INVALID
> 
> Finally, the checksum status of inner headers should have no influence
> on PKT_RX_EIP_CKSUM_BAD or PKT_RX_OUTER_L4_CKSUM_*, and
> likewise the checksum status of outer headers should have no influence
> on PKT_RX_L4_CKSUM_* or PKT_RX_IP_CKSUM_*.
> 
> Is this correct? Apologies for such a basic question, but I'm having trouble
> correlating the above with implementations.
> 
> Thanks and regards,
>     Lance

The PKT_RX_EIP_CKSUM_BAD flag was added by these commits:

https://git.dpdk.org/dpdk/commit/?id=c22265f6fd4cdcac9ee1b4970e4af8459d267516
https://git.dpdk.org/dpdk/commit/?id=d909af8f72ca3f8ab4fe1942abfb4f53e15ff8bc

First, to be honnest, I don't think this API is the right one. From a software
stack point of view, it would have been more logical to have PKT_RX_INNER_*
flags instead of outer.

That said, your understanding looks correct to me. I think this is the
expected behavior when the DEV_RX_OFFLOAD_OUTER* capability is enabled.

If the capability is not set, only the PKT_RX_IP_CKSUM* and
PKT_RX_L4_CKSUM* flags may be set, and they reference the first layer.

Regards,
Olivier

Re: [dpdk-dev] question regarding rx checksum offload flags

[Lance Richardson]

On Thu, Sep 17, 2020 at 10:05 AM Olivier Matz <olivier.matz@6wind.com> wrote:
>
> Hi Lance,
>
> On Mon, Aug 24, 2020 at 04:11:45PM -0400, Lance Richardson wrote:
> > I was looking for some clarification regarding how rx checksum
> > flags should be set for tunnel packets having both inner and outer
> > IP/L4 headers.
> >
> > Based on comments in rte_mbuf_core.h, it seems to me. that the
> > inner (encapsulated) IP header checksum status should determine
> > which of these goes into ol_flags:
> >     PKT_RX_IP_CKSUM_UNKNOWN
> >     PKT_RX_IP_CKSUM_BAD
> >     PKT_RX_IP_CKSUM_GOOD
> >     PKT_RX_IP_CKSUM_NONE
> >
> > Similarly, the L4 checksum status should determine which of these
> > goes into ol_flags:
> >    PKT_RX_L4_CKSUM_UNKNOWN
> >    PKT_RX_L4_CKSUM_BAD
> >    PKT_RX_L4_CKSUM_GOOD
> >    PKT_RX_L4_CKSUM_NONE
> >
> > The IP header checksum status for the outer IP header should determine
> > whether this flag is set in ol_flags:
> >     PKT_RX_EIP_CKSUM_BAD
> >
> > And for UDP-based tunnel encapsulations, the outer L4 checksum status
> > should determine which of these goes into ol_flags:
> >     PKT_RX_OUTER_L4_CKSUM_UNKNOWN
> >     PKT_RX_OUTER_L4_CKSUM_BAD
> >     PKT_RX_OUTER_L4_CKSUM_GOOD
> >     PKT_RX_OUTER_L4_CKSUM_INVALID
> >
> > Finally, the checksum status of inner headers should have no influence
> > on PKT_RX_EIP_CKSUM_BAD or PKT_RX_OUTER_L4_CKSUM_*, and
> > likewise the checksum status of outer headers should have no influence
> > on PKT_RX_L4_CKSUM_* or PKT_RX_IP_CKSUM_*.
> >
> > Is this correct? Apologies for such a basic question, but I'm having trouble
> > correlating the above with implementations.
> >
> > Thanks and regards,
> >     Lance
>
> The PKT_RX_EIP_CKSUM_BAD flag was added by these commits:
>
> https://git.dpdk.org/dpdk/commit/?id=c22265f6fd4cdcac9ee1b4970e4af8459d267516
> https://git.dpdk.org/dpdk/commit/?id=d909af8f72ca3f8ab4fe1942abfb4f53e15ff8bc
>
> First, to be honnest, I don't think this API is the right one. From a software
> stack point of view, it would have been more logical to have PKT_RX_INNER_*
> flags instead of outer.
>
> That said, your understanding looks correct to me. I think this is the
> expected behavior when the DEV_RX_OFFLOAD_OUTER* capability is enabled.
>
> If the capability is not set, only the PKT_RX_IP_CKSUM* and
> PKT_RX_L4_CKSUM* flags may be set, and they reference the first layer.
>
> Regards,
> Olivier

Hi Olivier,

I've been thinking about how to address this for the bnxt PMD which always
reports PKT_RX_OUTER_L4_CKSUM_* and PKT_RX_EIP_CKSUM_BAD
regardless of whether DEV_RX_OFFLOAD_OUTER_UDP_CKSUM or
DEV_RX_OFFLOAD_OUTER_IPV4_CKSUM are enabled.

One option would be to modify the PMD to respect the outer UDP and
outer IPv4 checksum offload configuration. Since the mapping of hardware
checksum status to mbuf checksum status is table-based, this would add
very little overhead to the packet handling path, but it would add some
(a larger table would be required, and the index would need to include
the tunnel/non-tunnel status of the packet).

Another option would be to modify the PMD to force the outer UDP and
outer IPv4 checksum offloads to always be enabled (there is at least
one existing PMD that currently does this).

The second option should be the least disruptive to existing users, and
would require the least effort to implement, but will it be an acceptable
approach going forward? (If not, it seems the first option would be the
right one to choose.)

Regards,

    Lance

-- 
This electronic communication and the information and any files transmitted 
with it, or attached to it, are confidential and are intended solely for 
the use of the individual or entity to whom it is addressed and may contain 
information that is confidential, legally privileged, protected by privacy 
laws, or otherwise restricted from disclosure to anyone else. If you are 
not the intended recipient or the person responsible for delivering the 
e-mail to the intended recipient, you are hereby notified that any use, 
copying, distributing, dissemination, forwarding, printing, or copying of 
this e-mail is strictly prohibited. If you received this e-mail in error, 
please return the e-mail to the sender, delete it from your computer, and 
destroy any printed copy of it.

Re: [dpdk-dev] question regarding rx checksum offload flags

[Lance Richardson]

On Mon, Dec 14, 2020 at 12:41 PM Lance Richardson
<lance.richardson@broadcom.com> wrote:
>
> On Thu, Sep 17, 2020 at 10:05 AM Olivier Matz <olivier.matz@6wind.com> wrote:
> >
> > Hi Lance,
> >
> > On Mon, Aug 24, 2020 at 04:11:45PM -0400, Lance Richardson wrote:
> > > I was looking for some clarification regarding how rx checksum
> > > flags should be set for tunnel packets having both inner and outer
> > > IP/L4 headers.
> > >
> > > Based on comments in rte_mbuf_core.h, it seems to me. that the
> > > inner (encapsulated) IP header checksum status should determine
> > > which of these goes into ol_flags:
> > >     PKT_RX_IP_CKSUM_UNKNOWN
> > >     PKT_RX_IP_CKSUM_BAD
> > >     PKT_RX_IP_CKSUM_GOOD
> > >     PKT_RX_IP_CKSUM_NONE
> > >
> > > Similarly, the L4 checksum status should determine which of these
> > > goes into ol_flags:
> > >    PKT_RX_L4_CKSUM_UNKNOWN
> > >    PKT_RX_L4_CKSUM_BAD
> > >    PKT_RX_L4_CKSUM_GOOD
> > >    PKT_RX_L4_CKSUM_NONE
> > >
> > > The IP header checksum status for the outer IP header should determine
> > > whether this flag is set in ol_flags:
> > >     PKT_RX_EIP_CKSUM_BAD
> > >
> > > And for UDP-based tunnel encapsulations, the outer L4 checksum status
> > > should determine which of these goes into ol_flags:
> > >     PKT_RX_OUTER_L4_CKSUM_UNKNOWN
> > >     PKT_RX_OUTER_L4_CKSUM_BAD
> > >     PKT_RX_OUTER_L4_CKSUM_GOOD
> > >     PKT_RX_OUTER_L4_CKSUM_INVALID
> > >
> > > Finally, the checksum status of inner headers should have no influence
> > > on PKT_RX_EIP_CKSUM_BAD or PKT_RX_OUTER_L4_CKSUM_*, and
> > > likewise the checksum status of outer headers should have no influence
> > > on PKT_RX_L4_CKSUM_* or PKT_RX_IP_CKSUM_*.
> > >
> > > Is this correct? Apologies for such a basic question, but I'm having trouble
> > > correlating the above with implementations.
> > >
> > > Thanks and regards,
> > >     Lance
> >
> > The PKT_RX_EIP_CKSUM_BAD flag was added by these commits:
> >
> > https://git.dpdk.org/dpdk/commit/?id=c22265f6fd4cdcac9ee1b4970e4af8459d267516
> > https://git.dpdk.org/dpdk/commit/?id=d909af8f72ca3f8ab4fe1942abfb4f53e15ff8bc
> >
> > First, to be honnest, I don't think this API is the right one. From a software
> > stack point of view, it would have been more logical to have PKT_RX_INNER_*
> > flags instead of outer.
> >
> > That said, your understanding looks correct to me. I think this is the
> > expected behavior when the DEV_RX_OFFLOAD_OUTER* capability is enabled.
> >
> > If the capability is not set, only the PKT_RX_IP_CKSUM* and
> > PKT_RX_L4_CKSUM* flags may be set, and they reference the first layer.
> >
> > Regards,
> > Olivier
>
> Hi Olivier,
>
> I've been thinking about how to address this for the bnxt PMD which always
> reports PKT_RX_OUTER_L4_CKSUM_* and PKT_RX_EIP_CKSUM_BAD
> regardless of whether DEV_RX_OFFLOAD_OUTER_UDP_CKSUM or
> DEV_RX_OFFLOAD_OUTER_IPV4_CKSUM are enabled.
>
> One option would be to modify the PMD to respect the outer UDP and
> outer IPv4 checksum offload configuration. Since the mapping of hardware
> checksum status to mbuf checksum status is table-based, this would add
> very little overhead to the packet handling path, but it would add some
> (a larger table would be required, and the index would need to include
> the tunnel/non-tunnel status of the packet).
>
> Another option would be to modify the PMD to force the outer UDP and
> outer IPv4 checksum offloads to always be enabled (there is at least
> one existing PMD that currently does this).
>
> The second option should be the least disruptive to existing users, and
> would require the least effort to implement, but will it be an acceptable
> approach going forward? (If not, it seems the first option would be the
> right one to choose.)
>
> Regards,
>
>     Lance
>

Apologies for the corporate-enforced email footer, picking back up on
 a personal email account.

I'm not sure whether this is an rte_mbuf API question or a PMD
implementation question, likewise not sure whether it should  be
directed to Olivier or Ferruh or someone else. Any advice is
appreciated.

Thanks,
    Lance

Re: [dpdk-dev] question regarding rx checksum offload flags

[Olivier Matz]

Hi Lance,

On Mon, Dec 14, 2020 at 12:41:34PM -0500, Lance Richardson wrote:
> On Thu, Sep 17, 2020 at 10:05 AM Olivier Matz <olivier.matz@6wind.com> wrote:
> >
> > Hi Lance,
> >
> > On Mon, Aug 24, 2020 at 04:11:45PM -0400, Lance Richardson wrote:
> > > I was looking for some clarification regarding how rx checksum
> > > flags should be set for tunnel packets having both inner and outer
> > > IP/L4 headers.
> > >
> > > Based on comments in rte_mbuf_core.h, it seems to me. that the
> > > inner (encapsulated) IP header checksum status should determine
> > > which of these goes into ol_flags:
> > >     PKT_RX_IP_CKSUM_UNKNOWN
> > >     PKT_RX_IP_CKSUM_BAD
> > >     PKT_RX_IP_CKSUM_GOOD
> > >     PKT_RX_IP_CKSUM_NONE
> > >
> > > Similarly, the L4 checksum status should determine which of these
> > > goes into ol_flags:
> > >    PKT_RX_L4_CKSUM_UNKNOWN
> > >    PKT_RX_L4_CKSUM_BAD
> > >    PKT_RX_L4_CKSUM_GOOD
> > >    PKT_RX_L4_CKSUM_NONE
> > >
> > > The IP header checksum status for the outer IP header should determine
> > > whether this flag is set in ol_flags:
> > >     PKT_RX_EIP_CKSUM_BAD
> > >
> > > And for UDP-based tunnel encapsulations, the outer L4 checksum status
> > > should determine which of these goes into ol_flags:
> > >     PKT_RX_OUTER_L4_CKSUM_UNKNOWN
> > >     PKT_RX_OUTER_L4_CKSUM_BAD
> > >     PKT_RX_OUTER_L4_CKSUM_GOOD
> > >     PKT_RX_OUTER_L4_CKSUM_INVALID
> > >
> > > Finally, the checksum status of inner headers should have no influence
> > > on PKT_RX_EIP_CKSUM_BAD or PKT_RX_OUTER_L4_CKSUM_*, and
> > > likewise the checksum status of outer headers should have no influence
> > > on PKT_RX_L4_CKSUM_* or PKT_RX_IP_CKSUM_*.
> > >
> > > Is this correct? Apologies for such a basic question, but I'm having trouble
> > > correlating the above with implementations.
> > >
> > > Thanks and regards,
> > >     Lance
> >
> > The PKT_RX_EIP_CKSUM_BAD flag was added by these commits:
> >
> > https://git.dpdk.org/dpdk/commit/?id=c22265f6fd4cdcac9ee1b4970e4af8459d267516
> > https://git.dpdk.org/dpdk/commit/?id=d909af8f72ca3f8ab4fe1942abfb4f53e15ff8bc
> >
> > First, to be honnest, I don't think this API is the right one. From a software
> > stack point of view, it would have been more logical to have PKT_RX_INNER_*
> > flags instead of outer.
> >
> > That said, your understanding looks correct to me. I think this is the
> > expected behavior when the DEV_RX_OFFLOAD_OUTER* capability is enabled.
> >
> > If the capability is not set, only the PKT_RX_IP_CKSUM* and
> > PKT_RX_L4_CKSUM* flags may be set, and they reference the first layer.
> >
> > Regards,
> > Olivier
> 
> Hi Olivier,
> 
> I've been thinking about how to address this for the bnxt PMD which always
> reports PKT_RX_OUTER_L4_CKSUM_* and PKT_RX_EIP_CKSUM_BAD
> regardless of whether DEV_RX_OFFLOAD_OUTER_UDP_CKSUM or
> DEV_RX_OFFLOAD_OUTER_IPV4_CKSUM are enabled.
> 
> One option would be to modify the PMD to respect the outer UDP and
> outer IPv4 checksum offload configuration. Since the mapping of hardware
> checksum status to mbuf checksum status is table-based, this would add
> very little overhead to the packet handling path, but it would add some
> (a larger table would be required, and the index would need to include
> the tunnel/non-tunnel status of the packet).
> 
> Another option would be to modify the PMD to force the outer UDP and
> outer IPv4 checksum offloads to always be enabled (there is at least
> one existing PMD that currently does this).
> 
> The second option should be the least disruptive to existing users, and
> would require the least effort to implement, but will it be an acceptable
> approach going forward? (If not, it seems the first option would be the
> right one to choose.)

I think the first option is the right one. It matches the behavior of
other drivers.

Regards,
Olivier