From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mc26.lon.server.colt.net (mc26.lon.server.colt.net [212.74.77.106]) by dpdk.org (Postfix) with SMTP id CFE45B381 for ; Thu, 4 Sep 2014 11:19:36 +0200 (CEST) Received: from mc26.lon.server.colt.net (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E9A53C2E4D for ; Thu, 4 Sep 2014 09:24:16 +0000 (UTC) Received: from mx3.qosmos.com (unknown [195.68.92.43]) by mc26.lon.server.colt.net (Postfix) with ESMTP id C9DDDC2DC0 for ; Thu, 4 Sep 2014 09:24:16 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.04,464,1406584800"; d="scan'208";a="1217209" Received: from unknown (HELO cercis.foret) ([10.10.2.42]) by mx3.qosmos.com with ESMTP; 04 Sep 2014 11:24:16 +0200 Message-ID: <54082FC0.1060205@qosmos.com> Date: Thu, 04 Sep 2014 11:24:16 +0200 From: Franck Baudin User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.7.0 MIME-Version: 1.0 To: "Xie, Huawei" , "Gray, Mark D" , Thomas Monjalon References: <1686757.iSdNo6aMGt@xps13> <738D45BC1F695740A983F43CFE1B7EA92D72308F@IRSMSX102.ger.corp.intel.com> <540721E8.2010307@qosmos.com> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-TM-AS-MML: No X-TM-AS-Product-Ver: IMSVA-8.2.0.1679-7.5.0.1018-20928.006 X-TM-AS-Result: No--13.386-5.0-31-10 X-imss-scan-details: No--13.386-5.0-31-10 X-TM-AS-User-Approved-Sender: No X-TMASE-Version: IMSVA-8.2.0.1679-7.5.1018-20928.006 X-TMASE-Result: 10--13.386000-5.000000 X-TMASE-MatchedRID: L+num7NHV/Itjw5zGtj91Ca1MaKuob8PC/ExpXrHizxwG72s9z/Zf9nD 0nQ0Nnz4uI3WBEQRv97skDgSeVf5wSCJQ0hMWhDv96LM5HQIdGo7r2Gtb9iBYRBMzcCTqrAdVrK 5UPh956XBe21rbtQyhNaiBOAn6SeM9g9xMD4cPT1GWpKGpvfmnp5yoMTQ4h/1/k12Sb2MaHlWFs MQfbcNE4Yk3QiNWOE1n8wrgJJ0KIvN8cEGV9agBZGzIhDiMWXrAZn/4A9db2TIQofYImpSX75BE qXwSs2UYAd0iBq6BIYAAlsVlaGFCz13WcdbGR6Qz4XVyhFFZHI3GofkGVB3jivIhzVcH4iEEWXA mFVxJU6r+lby8CJY5Zj4oSfJftZ+S22k34pVV33Y5KPiokD1BtSqEluSYtV7fp8/V+6fG8KjxYy RBa/qJQPTK4qtAgwIPcCXjNqUmkUgBwKKRHe+r63VfRC6oJeDZenVe9Wq7vvvyl7FxaSIY7KVPN 12MHfzEwZXlqp7MNg= Cc: "dev@dpdk.org" , "dpdk-ovs@lists.01.org" Subject: Re: [dpdk-dev] Wrong TCP checksum of packets sent by Linux guest (virtIO/vhost) X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Sep 2014 09:19:37 -0000 Hi Huawei, On 09/04/14 04:54, Xie, Huawei wrote: > Hi Franck: > I checked your original thread. > root at linux-native:~# tcpdump -vnei eth0 > tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes > 17:27:09.262926 00:1b:21:b9:9b:2c > 52:54:00:51:51:12, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 47743, offset 0, flags [DF], proto TCP (6), length 60) > 1.1.1.3.34272 > 1.1.1.2.5555: Flags [S], cksum 0x0435 (incorrect -> 0xb2dd), seq 1963818356, win 14600, options [mss 1460,sackOK,TS val 49530635 ecr 0,nop,wscale 7], length 0 > 17:27:09.263220 52:54:00:51:51:12 > 00:1b:21:b9:9b:2c, ethertype IPv4 (0x0800), length 60: (tos 0x0, ttl 64, id 3367, offset 0, flags [DF], proto TCP (6), length 40) > 1.1.1.2.5555 > 1.1.1.3.34272: Flags [R.], cksum 0x1db4 (correct), seq 0, ack 1963818357, win 0, length 0 > > The packet from the guest, received on the native machine, has "correct" checksum 0x1db4. The TCP connection is refused is due to this packet has R flag. Could you check if it is the firewall that blocks the > connection? > > Besides, I did ssh tcp test between two guest VMs, the ssh connection could be established correctly with tx checksum off. I didn't use OVDK, but set the arp table and route table manually. > The packet flow is > Guest A(virtio)<->vhost example->Guest B(virtio) I re-made a test between two identical VMs, UDP and ICMP are fine. Some observations: on both client & server: ethtool -k eth1 | grep ": on" generic-receive-offload: on highdma: on [fixed] rx-vlan-filter: on [fixed] ethtool -k eth1 Features for eth1: rx-checksumming: off [fixed] tx-checksumming: off tx-checksum-ipv4: off [fixed] tx-checksum-unneeded: off [fixed] tx-checksum-ip-generic: off [fixed] tx-checksum-ipv6: off [fixed] tx-checksum-fcoe-crc: off [fixed] tx-checksum-sctp: off [fixed] scatter-gather: off tx-scatter-gather: off [fixed] tx-scatter-gather-fraglist: off [fixed] tcp-segmentation-offload: off tx-tcp-segmentation: off [fixed] tx-tcp-ecn-segmentation: off [fixed] tx-tcp6-segmentation: off [fixed] udp-fragmentation-offload: off [fixed] generic-segmentation-offload: off [requested on] generic-receive-offload: on large-receive-offload: off [fixed] rx-vlan-offload: off [fixed] tx-vlan-offload: off [fixed] ntuple-filters: off [fixed] receive-hashing: off [fixed] highdma: on [fixed] rx-vlan-filter: on [fixed] vlan-challenged: off [fixed] tx-lockless: off [fixed] netns-local: off [fixed] tx-gso-robust: off [fixed] tx-fcoe-segmentation: off [fixed] fcoe-mtu: off [fixed] tx-nocache-copy: off loopback: off [fixed] On the client side: 11:00:23.296592 IP (tos 0x0, ttl 64, id 25239, offset 0, flags [DF], proto TCP (6), length 60) 1.1.1.2.47192 > 1.1.1.1.dsf: Flags [S], cksum 0x0433 (incorrect -> 0xe3bd), seq 3885781956, win 14600, options [mss 1460,sackOK,TS val 4294914387 ecr 0,nop,wscale 5], length 0 0x0000: 4500 003c 6297 4000 4006 d420 0101 0102 E.. 1.1.1.2.47192: Flags [R.], cksum 0xb5e6 (correct), seq 0, ack 3885781957, win 0, length 0 0x0000: 4500 0028 198f 4000 4006 1d3d 0101 0101 E..(..@.@..=.... 0x0010: 0101 0102 022b b858 0000 0000 e79c 53c5 .....+.X......S. 0x0020: 5014 0000 b5e6 0000 P....... On the server side: 11:00:23.631991 IP (tos 0x0, ttl 64, id 25239, offset 0, flags [DF], proto TCP (6), length 60) 1.1.1.2.47192 > 1.1.1.1.dsf: Flags [S], cksum 0xe3bd (correct), seq 3885781956, win 14600, options [mss 1460,sackOK,TS val 4294914387 ecr 0,nop,wscale 5], length 0 0x0000: 4500 003c 6297 4000 4006 d420 0101 0102 E.. 1.1.1.2.47192: Flags [R.], cksum 0xb5e6 (correct), seq 0, ack 3885781957, win 0, length 0 0x0000: 4500 0028 198f 4000 4006 1d3d 0101 0101 E..(..@.@..=.... 0x0010: 0101 0102 022b b858 0000 0000 e79c 53c5 .....+.X......S. 0x0020: 5014 0000 b5e6 0000 P....... On the client, ethtool announce that TX checksum offload is disabled, but tcpdump show it's not. On the server side, all packets looks fine from a checksum perspective but the SYN is answered by a RST. Please note that the same images behave correctly with a regular openvswitch, as long as TX checksuming offloading is disabled. If anyone has an equivalent setup, functioning properly, thanks for sharing its configurations parameters. if not, I might be able to wait for vhost backend PMD driver: I will then with testpmd instead of dpdk-ovs, to see where is the problem (maybe in my configuration...). When so you plan to release the vhost backend PMD driver? Thanks! Franck