[dpdk-dev] vmxnet3 TX TCP/UDP checksum not getting computed with L2_len > 14

[dpdk-dev] vmxnet3 TX TCP/UDP checksum not getting computed with L2_len > 14

[Padam Jeet Singh]

Hello,

Issue observed when using vmxnet3 based interface on packet with following structure is sent:

Ethernet + PPPoE + PPP (22 bytes) as the Layer 2 header, 
IPv4 (20) 
UDP
DNS Payload

The tx offload value in this case is 0x0f0000000000000 (PKT_TX_IPV4  | PKT_TX_IP_CKSUM | PKT_TX_UDP_CKSUM)

The checksum of the packet seen by the receiver shows incorrect checksum and it’s value is the pseudo checksum value that was set at the time of the TX. However the IP header checksum is correct.

The same issue is not seen when the L2 header is a just the Ethernet (14 bytes).

Also, with the same setup on the same hardware if we switch the driver from vmxnet3 to e1000e, all checksums are computed correctly.

Is this a DPDK vmxnet3 driver bug or that of underlying esxi? The ESXi version is 6.0.0 (Build 3620759).

Thanks,
Padam

Re: [dpdk-dev] vmxnet3 TX TCP/UDP checksum not getting computed with L2_len > 14

[Stephen Hemminger]

On Sun, 17 Jun 2018 14:55:06 +0530
Padam Jeet Singh <padam.singh@inventum.net> wrote:

> Hello,
> 
> Issue observed when using vmxnet3 based interface on packet with following structure is sent:
> 
> Ethernet + PPPoE + PPP (22 bytes) as the Layer 2 header, 
> IPv4 (20) 
> UDP
> DNS Payload
> 
> The tx offload value in this case is 0x0f0000000000000 (PKT_TX_IPV4  | PKT_TX_IP_CKSUM | PKT_TX_UDP_CKSUM)
> 
> The checksum of the packet seen by the receiver shows incorrect checksum and it’s value is the pseudo checksum value that was set at the time of the TX. However the IP header checksum is correct.
> 
> The same issue is not seen when the L2 header is a just the Ethernet (14 bytes).
> 
> Also, with the same setup on the same hardware if we switch the driver from vmxnet3 to e1000e, all checksums are computed correctly.
> 
> Is this a DPDK vmxnet3 driver bug or that of underlying esxi? The ESXi version is 6.0.0 (Build 3620759).
> 
> Thanks,
> Padam

I don't think VMWare supports IP checksum offload. Since IP checksum is trivial and in cache,
the IP header checksum offload is usually not a speed up anyway. Linux for example, never does
IP header checksum offload.

Re: [dpdk-dev] vmxnet3 TX TCP/UDP checksum not getting computed with L2_len > 14

[Padam Jeet Singh]

> On 17-Jun-2018, at 10:16 PM, Stephen Hemminger <stephen@networkplumber.org> wrote:
> 
> On Sun, 17 Jun 2018 14:55:06 +0530
> Padam Jeet Singh <padam.singh@inventum.net> wrote:
> 
>> Hello,
>> 
>> Issue observed when using vmxnet3 based interface on packet with following structure is sent:
>> 
>> Ethernet + PPPoE + PPP (22 bytes) as the Layer 2 header, 
>> IPv4 (20) 
>> UDP
>> DNS Payload
>> 
>> The tx offload value in this case is 0x0f0000000000000 (PKT_TX_IPV4  | PKT_TX_IP_CKSUM | PKT_TX_UDP_CKSUM)
>> 
>> The checksum of the packet seen by the receiver shows incorrect checksum and it’s value is the pseudo checksum value that was set at the time of the TX. However the IP header checksum is correct.
>> 
>> The same issue is not seen when the L2 header is a just the Ethernet (14 bytes).
>> 
>> Also, with the same setup on the same hardware if we switch the driver from vmxnet3 to e1000e, all checksums are computed correctly.
>> 
>> Is this a DPDK vmxnet3 driver bug or that of underlying esxi? The ESXi version is 6.0.0 (Build 3620759).
>> 
>> Thanks,
>> Padam
> 
> I don't think VMWare supports IP checksum offload. Since IP checksum is trivial and in cache,
> the IP header checksum offload is usually not a speed up anyway. Linux for example, never does
> IP header checksum offload.

It’s not the IP checksum - it’s the TCP/UDP checksum (L4) that’s not getting computed. In fact the IP checksum is coming correctly because we are handing it in software. The driver when queried returns TX offload flags with the IP header checksum bit off, but L4 checksum (TCP & UDP) are available. The UDP checksum gets computed correctly if and only if the l2_len is 14. In case of other encapsulations, e.g. pppoe (l2_len=22), this checksum computation is being skipped and the packet is simply sent out with the pseudo checksum value of the IP header. 

Re: [dpdk-dev] vmxnet3 TX TCP/UDP checksum not getting computed with L2_len > 14

[Yong Wang]

-----Original Message-----
From: Padam Jeet Singh <padam.singh@inventum.net>
Date: Sunday, June 17, 2018 at 12:00 PM
To: Stephen Hemminger <stephen@networkplumber.org>
Cc: "dev@dpdk.org" <dev@dpdk.org>, Yong Wang <yongwang@vmware.com>
Subject: Re: [dpdk-dev] vmxnet3 TX TCP/UDP checksum not getting computed with L2_len > 14


    >> On 17-Jun-2018, at 10:16 PM, Stephen Hemminger <stephen@networkplumber.org> wrote:
    >> 
    >> On Sun, 17 Jun 2018 14:55:06 +0530
    >> Padam Jeet Singh <padam.singh@inventum.net> wrote:
    >> 
    >>> Hello,
    >>> 
    >>> Issue observed when using vmxnet3 based interface on packet with following structure is sent:
    >>> 
    >>> Ethernet + PPPoE + PPP (22 bytes) as the Layer 2 header, 
    >>> IPv4 (20) 
    >>> UDP
    >>> DNS Payload
    >>> 
    >>> The tx offload value in this case is 0x0f0000000000000 (PKT_TX_IPV4  | PKT_TX_IP_CKSUM | PKT_TX_UDP_CKSUM)
    >>> 
    >>> The checksum of the packet seen by the receiver shows incorrect checksum and it’s value is the pseudo checksum value that was set at the time of the TX. However the IP header checksum is correct.
    >>> 
    >>> The same issue is not seen when the L2 header is a just the Ethernet (14 bytes).
    >>> 
    >>> Also, with the same setup on the same hardware if we switch the driver from vmxnet3 to e1000e, all checksums are computed correctly.
    >>> 
    >>> Is this a DPDK vmxnet3 driver bug or that of underlying esxi? The ESXi version is 6.0.0 (Build 3620759).
    >>> 
    >>> Thanks,
    >>> Padam
    >> 
    >> I don't think VMWare supports IP checksum offload. Since IP checksum is trivial and in cache,
    >> the IP header checksum offload is usually not a speed up anyway. Linux for example, never does
    >> IP header checksum offload.
    
    >  It’s not the IP checksum - it’s the TCP/UDP checksum (L4) that’s not getting computed. In fact the IP checksum is coming correctly because we are handing it in software. The driver when queried returns TX offload flags with the IP header checksum bit off, but L4 checksum (TCP & UDP) are available. The UDP checksum gets computed correctly if and only if the l2_len is 14. In case of other encapsulations, e.g. pppoe (l2_len=22), this checksum computation is being skipped and the packet is simply sent out with the pseudo checksum value of the IP header. 

Vmxnet3 backend does not support Ethernet + PPPoE + PPP as layer 2 header if l4 checksum if requested.