From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by dpdk.org (Postfix) with ESMTP id 17964DE3 for ; Mon, 8 Oct 2018 06:49:29 +0200 (CEST) X-Amp-Result: UNSCANNABLE X-Amp-File-Uploaded: False Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by orsmga106.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 07 Oct 2018 21:49:28 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.54,355,1534834800"; d="scan'208";a="86520457" Received: from btwcube1.sh.intel.com (HELO debian) ([10.67.104.158]) by FMSMGA003.fm.intel.com with ESMTP; 07 Oct 2018 21:49:27 -0700 Date: Mon, 8 Oct 2018 12:48:12 +0800 From: Tiwei Bie To: Eric Tremblay Cc: "users@dpdk.org" , "zhihong.wang@intel.com" , "maxime.coquelin@redhat.com" Message-ID: <20181008044812.GA17177@debian> References: <20180918052951.GA1658@debian> <20180919050815.GA20551@debian> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) Subject: Re: [dpdk-users] Virtio-user exceptional path performance X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Oct 2018 04:49:30 -0000 On Mon, Sep 24, 2018 at 03:25:20PM +0000, Eric Tremblay wrote: > It does not work even if I do the following: >=20 > testpmd> set fwd csum > Set csum packet forwarding mode > testpmd> csum set tcp hw 0 > Parse tunnel is off > IP checksum offload is sw > UDP checksum offload is sw > TCP checksum offload is hw > SCTP checksum offload is sw > Outer-Ip checksum offload is sw > testpmd> csum set tcp hw 1 > Parse tunnel is off > IP checksum offload is sw > UDP checksum offload is sw > TCP checksum offload is hw > SCTP checksum offload is sw > Outer-Ip checksum offload is sw > testpmd> >=20 > I also tried using a NIC but it doesn't work either. Packets sent through= the virtio-user tap interface arrive on the other server with an incorrect= checksum. Is the checksum offloading broken for this type of virtual devic= e? The Rx and Tx checksum offloading works in virtio-user. Below are the steps I used on my machine: # Launch the testpmd with two virtio-user ports $ ./x86_64-native-linuxapp-gcc/app/testpmd \ -l 0,1 \ --file-prefix=3Dvhost-kernel \ --vdev=3Dvirtio_user0,path=3D/dev/vhost-net,queue_size=3D1024,iface=3Dvirt= io0 \ --vdev=3Dvirtio_user1,path=3D/dev/vhost-net,queue_size=3D1024,iface=3Dvirt= io1 \ --no-pci \ -- \ -i \ --disable-hw-vlan \ --enable-lro \ --enable-rx-cksum \ --eth-peer=3D0,00:01:02:03:04:01 \ --eth-peer=3D1,00:01:02:03:04:02 # Configure and start testpmd testpmd> set fwd csum testpmd> csum set tcp hw 0 testpmd> csum set tcp hw 1 testpmd> start # Configure the first tap interface $ ifconfig virtio0 hw ether 00:01:02:03:04:01 $ ifconfig virtio0 1.1.1.1 up # Create a network namespace $ ip netns add ns1 # Configure the second tap interface $ ip link set virtio1 netns ns1 $ ip netns exec ns1 ifconfig virtio1 hw ether 00:01:02:03:04:02 $ ip netns exec ns1 ifconfig virtio1 1.1.1.2 up # Start the iperf server $ ip netns exec ns1 iperf -s -i 1 # Start the iperf client $ iperf -c 1.1.1.2 -i 1 -t 60 >=20 > Eric=20 >=20 > -----Original Message----- > From: Tiwei Bie =20 > Sent: September 19, 2018 1:08 AM > To: Eric Tremblay > Cc: users@dpdk.org; zhihong.wang@intel.com; maxime.coquelin@redhat.com > Subject: Re: [dpdk-users] Virtio-user exceptional path performance >=20 > On Tue, Sep 18, 2018 at 02:45:24PM +0000, Eric Tremblay wrote: > > Thanks for the reply. > >=20 > > With the patch, the tap interface is created in the kernel even when sp= ecifying the --enable-lro and --enable-rx-cksum options. However, the behav= iour is the same as without these options. That is, ethtool shows LRO and r= x-checksum as off (and [fixed]) and TCP traffic gets dropped unless tx-chec= ksum is turned off. The difference is that testpmd now says that LRO and RX= checksum are on (even though they don't seem to be). >=20 > Are you using csum fwd in testpmd? You also need to enable the hw csum of= fload. You can do it via: >=20 > testpmd> csum set tcp hw 0 > testpmd> csum set tcp hw 1 >=20 > And make sure you get below output: >=20 > TCP checksum offload is hw >=20 > >=20 > > Eric > >=20 > > -----Original Message----- > > From: Tiwei Bie > > Sent: September 18, 2018 1:30 AM > > To: Eric Tremblay ; users@dpdk.org > > Cc: zhihong.wang@intel.com; maxime.coquelin@redhat.com > > Subject: Re: [dpdk-users] Virtio-user exceptional path performance > >=20 > > On Tue, Sep 04, 2018 at 05:54:20PM +0000, Eric Tremblay wrote: > > > Hello, > > >=20 > > > I am testing the performance capabilities of the virtio-user exceptio= nal path for use in container networking scenarios. I am trying to reproduc= e the results obtained in Test Case 1 described in the paper "VIRTIO-USER: = A New Versatile Channel for Kernel-Bypass Networks". That is, I am creatin= g two virtio-user vdevs in testpmd, putting each one in separate network na= mespaces and running iperf3 between the two. However, I am unable to obtain= the near 25Gbps throughput shown in the paper. Instead, I obtained a mere = 3Gbps. Has anyone else been able to reproduce these results? My understandi= ng is that this type of virtual device takes advantage of both checksum off= loading and LRO in order to get such high throughput. However, for some rea= son, I am unable to use RX checksum offloading nor LRO in my setup. If I sp= ecify the --enable-lro and --enable-rx-cksum options in testpmd (as shown h= ere https://doc.dpdk.org/guides-17.11/howto/virtio_user_as_exceptional_path= =2Ehtml), the tap device is never created in the kernel. If I do not specif= y these options, the tap is created correctly but the rx-checksum and LRO a= re off and cannot be turned on using ethtool (they are marked as [fixed]). > >=20 > > Somehow below fix isn't backported to 17.11 while the commit it fixed w= as backported. > >=20 > > https://git.dpdk.org/dpdk/commit/?id=3Dbce7e9050f9b > > commit bce7e9050f9b ("net/virtio-user: fix start with kernel vhost") > >=20 > > Please apply above fix locally and see if it works. > >=20 > > Thanks > >=20 > > >=20 > > > Does anyone have any idea what may be preventing me from taking advan= tage of checksum offloading and LRO? Also, I am I right in assuming that th= is is the reason why the performance is so poor or could there be another p= roblem with my setup? I am using DPDK 17.11 on CentOS 7.5. Any help would b= e appreciated. > > >=20 > > > Regards, > > >=20 > > > Eric > > >=20