From: Franck Baudin <franck.baudin@qosmos.com>
To: "Xie, Huawei" <huawei.xie@intel.com>,
"Gray, Mark D" <mark.d.gray@intel.com>,
Thomas Monjalon <thomas.monjalon@6wind.com>
Cc: "dev@dpdk.org" <dev@dpdk.org>,
"dpdk-ovs@lists.01.org" <dpdk-ovs@lists.01.org>
Subject: Re: [dpdk-dev] Wrong TCP checksum of packets sent by Linux guest (virtIO/vhost)
Date: Thu, 04 Sep 2014 11:24:16 +0200 [thread overview]
Message-ID: <54082FC0.1060205@qosmos.com> (raw)
In-Reply-To: <C37D651A908B024F974696C65296B57B0F2859D3@SHSMSX101.ccr.corp.intel.com>
Hi Huawei,
On 09/04/14 04:54, Xie, Huawei wrote:
> Hi Franck:
> I checked your original thread.
> root at linux-native:~# tcpdump -vnei eth0
> tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
> 17:27:09.262926 00:1b:21:b9:9b:2c > 52:54:00:51:51:12, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 47743, offset 0, flags [DF], proto TCP (6), length 60)
> 1.1.1.3.34272 > 1.1.1.2.5555: Flags [S], cksum 0x0435 (incorrect -> 0xb2dd), seq 1963818356, win 14600, options [mss 1460,sackOK,TS val 49530635 ecr 0,nop,wscale 7], length 0
> 17:27:09.263220 52:54:00:51:51:12 > 00:1b:21:b9:9b:2c, ethertype IPv4 (0x0800), length 60: (tos 0x0, ttl 64, id 3367, offset 0, flags [DF], proto TCP (6), length 40)
> 1.1.1.2.5555 > 1.1.1.3.34272: Flags [R.], cksum 0x1db4 (correct), seq 0, ack 1963818357, win 0, length 0
>
> The packet from the guest, received on the native machine, has "correct" checksum 0x1db4. The TCP connection is refused is due to this packet has R flag. Could you check if it is the firewall that blocks the
> connection?
>
> Besides, I did ssh tcp test between two guest VMs, the ssh connection could be established correctly with tx checksum off. I didn't use OVDK, but set the arp table and route table manually.
> The packet flow is
> Guest A(virtio)<->vhost example->Guest B(virtio)
I re-made a test between two identical VMs, UDP and ICMP are fine. Some
observations:
on both client & server:
ethtool -k eth1 | grep ": on"
generic-receive-offload: on
highdma: on [fixed]
rx-vlan-filter: on [fixed]
ethtool -k eth1
Features for eth1:
rx-checksumming: off [fixed]
tx-checksumming: off
tx-checksum-ipv4: off [fixed]
tx-checksum-unneeded: off [fixed]
tx-checksum-ip-generic: off [fixed]
tx-checksum-ipv6: off [fixed]
tx-checksum-fcoe-crc: off [fixed]
tx-checksum-sctp: off [fixed]
scatter-gather: off
tx-scatter-gather: off [fixed]
tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: off
tx-tcp-segmentation: off [fixed]
tx-tcp-ecn-segmentation: off [fixed]
tx-tcp6-segmentation: off [fixed]
udp-fragmentation-offload: off [fixed]
generic-segmentation-offload: off [requested on]
generic-receive-offload: on
large-receive-offload: off [fixed]
rx-vlan-offload: off [fixed]
tx-vlan-offload: off [fixed]
ntuple-filters: off [fixed]
receive-hashing: off [fixed]
highdma: on [fixed]
rx-vlan-filter: on [fixed]
vlan-challenged: off [fixed]
tx-lockless: off [fixed]
netns-local: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off [fixed]
fcoe-mtu: off [fixed]
tx-nocache-copy: off
loopback: off [fixed]
On the client side:
11:00:23.296592 IP (tos 0x0, ttl 64, id 25239, offset 0, flags [DF],
proto TCP
(6), length 60)
1.1.1.2.47192 > 1.1.1.1.dsf: Flags [S], cksum 0x0433 (incorrect ->
0xe3bd), seq 3885781956, win 14600, options [mss 1460,sackOK,TS val
4294914387
ecr 0,nop,wscale 5], length 0
0x0000: 4500 003c 6297 4000 4006 d420 0101 0102 E..<b.@.@.......
0x0010: 0101 0101 b858 022b e79c 53c4 0000 0000 .....X.+..S.....
0x0020: a002 3908 0433 0000 0204 05b4 0402 080a ..9..3..........
0x0030: ffff 3153 0000 0000 0103 0305 ..1S........
11:00:23.296881 IP (tos 0x0, ttl 64, id 6543, offset 0, flags [DF],
proto TCP
(6), length 40)
1.1.1.1.dsf > 1.1.1.2.47192: Flags [R.], cksum 0xb5e6 (correct), seq 0,
ack 3885781957, win 0, length 0
0x0000: 4500 0028 198f 4000 4006 1d3d 0101 0101 E..(..@.@..=....
0x0010: 0101 0102 022b b858 0000 0000 e79c 53c5 .....+.X......S.
0x0020: 5014 0000 b5e6 0000 P.......
On the server side:
11:00:23.631991 IP (tos 0x0, ttl 64, id 25239, offset 0, flags [DF],
proto TCP
(6), length 60)
1.1.1.2.47192 > 1.1.1.1.dsf: Flags [S], cksum 0xe3bd (correct), seq
3885781956, win 14600, options [mss 1460,sackOK,TS val 4294914387 ecr
0,nop,wscale 5], length 0
0x0000: 4500 003c 6297 4000 4006 d420 0101 0102 E..<b.@.@.......
0x0010: 0101 0101 b858 022b e79c 53c4 0000 0000 .....X.+..S.....
0x0020: a002 3908 e3bd 0000 0204 05b4 0402 080a ..9.............
0x0030: ffff 3153 0000 0000 0103 0305 ..1S........
11:00:23.632034 IP (tos 0x0, ttl 64, id 6543, offset 0, flags [DF],
proto TCP
(6), length 40)
1.1.1.1.dsf > 1.1.1.2.47192: Flags [R.], cksum 0xb5e6 (correct), seq 0,
ack 3885781957, win 0, length 0
0x0000: 4500 0028 198f 4000 4006 1d3d 0101 0101 E..(..@.@..=....
0x0010: 0101 0102 022b b858 0000 0000 e79c 53c5 .....+.X......S.
0x0020: 5014 0000 b5e6 0000 P.......
On the client, ethtool announce that TX checksum offload is disabled,
but tcpdump show it's not. On the server side, all packets looks fine
from a checksum perspective but the SYN is answered by a RST. Please
note that the same images behave correctly with a regular openvswitch,
as long as TX checksuming offloading is disabled.
If anyone has an equivalent setup, functioning properly, thanks for
sharing its configurations parameters. if not, I might be able to wait
for vhost backend PMD driver: I will then with testpmd instead of
dpdk-ovs, to see where is the problem (maybe in my configuration...).
When so you plan to release the vhost backend PMD driver?
Thanks!
Franck
next prev parent reply other threads:[~2014-09-04 9:19 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-09-02 13:20 Franck BAUDIN
2014-09-02 13:29 ` Thomas Monjalon
2014-09-02 14:37 ` Franck Baudin
2014-09-03 10:00 ` Gray, Mark D
2014-09-03 11:13 ` Xie, Huawei
2014-09-03 14:12 ` Franck Baudin
2014-09-03 16:07 ` Gray, Mark D
2014-09-04 2:54 ` Xie, Huawei
2014-09-04 9:24 ` Franck Baudin [this message]
2014-09-04 10:35 ` Xie, Huawei
2014-09-03 16:15 ` Gray, Mark D
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=54082FC0.1060205@qosmos.com \
--to=franck.baudin@qosmos.com \
--cc=dev@dpdk.org \
--cc=dpdk-ovs@lists.01.org \
--cc=huawei.xie@intel.com \
--cc=mark.d.gray@intel.com \
--cc=thomas.monjalon@6wind.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).