DPDK patches and discussions
 help / color / mirror / Atom feed
* [dpdk-dev] [PATCH 00/12] net/virtio: add offload support
@ 2016-07-21  8:08 Olivier Matz
  2016-07-21  8:08 ` [dpdk-dev] [PATCH 01/12] virtio: move device initialization in a function Olivier Matz
                   ` (13 more replies)
  0 siblings, 14 replies; 97+ messages in thread
From: Olivier Matz @ 2016-07-21  8:08 UTC (permalink / raw)
  To: dev, yuanhan.liu, konstantin.ananyev
  Cc: sugesh.chandran, bruce.richardson, jianfeng.tan, helin.zhang,
	adrien.mazarguil

This patchset, targetted for 16.11, introduces the support of rx and tx
offload in virtio pmd.  To achieve this, some new mbuf flags must be
introduced, as discussed in [1].

It applies on top of:
- 16.07-rc3
- software packet type [2]
- testpmd enhancements [3]
- virtio packet corruption fix [4]

The new mbuf checksum flags are backward compatible for current
applications that assume that unknown_csum = good_cum (since there
was only a bad_csum flag). But it the patchset is integrated, we
should consider updating the PMDs to match the new API for 16.11.

[1] http://dpdk.org/ml/archives/dev/2016-May/039920.html
[2] http://dpdk.org/ml/archives/dev/2016-July/043333.html
[3] http://dpdk.org/ml/archives/dev/2016-July/043826.html
[4] http://dpdk.org/ml/archives/dev/2016-July/044266.html

Olivier Matz (12):
  virtio: move device initialization in a function
  virtio: setup and start cq in configure callback
  virtio: reinitialize the device in configure callback
  mbuf: add function to calculate a checksum
  mbuf: add new Rx checksum mbuf flags
  app/testpmd: fix checksum stats in csum engine
  mbuf: new flag for LRO
  app/testpmd: display lro segment size
  virtio: add Rx checksum offload support
  virtio: add Tx checksum offload support
  virtio: add Lro support
  virtio: add Tso support

 app/test-pmd/csumonly.c                |   8 +-
 doc/guides/rel_notes/release_16_11.rst |  16 ++
 drivers/net/virtio/virtio_ethdev.c     | 184 ++++++++++++++---------
 drivers/net/virtio/virtio_ethdev.h     |  14 +-
 drivers/net/virtio/virtio_pci.h        |   4 +-
 drivers/net/virtio/virtio_rxtx.c       | 267 ++++++++++++++++++++++++++++++---
 drivers/net/virtio/virtqueue.h         |   1 +
 lib/librte_mbuf/rte_mbuf.c             |  73 ++++++++-
 lib/librte_mbuf/rte_mbuf.h             |  71 ++++++++-
 lib/librte_mbuf/rte_mbuf_version.map   |   1 +
 10 files changed, 530 insertions(+), 109 deletions(-)

Test plan
=========

Platform description
--------------------

  guest (dpdk)
  +----------------+
  |                |
  |                |
  |         port0  +-----<---+
  |       ixgbe /  |         |
  |       directio |         |
  |                |         |
  |    port1       |         ^ flow1
  +----------------+         | (flow2 is the reverse)
         |                   |
         | virtio            |
         v                   |
  +----------------+         |
  |     tap0   /   |         |
  |1.1.1.1   /     |         |
  |ns-tap  /       |         |
  |      /         |         |
  |    /   ixgbe2  +------>--+
  |  /    1.1.1.2  |
  |/      ns-ixgbe |
  +----------------+
  host (linux, vhost-net)


flow1:
  host -(ixgbe)-> guest -(virtio)-> host
  1.1.1.2 -> 1.1.1.1

flow2:
  host -(virtio)-> guest -(ixgbe)-> host
  1.1.1.2 -> 1.1.1.1

Host configuration
------------------

Start qemu with:

- a ne2k management interface to avoi any conflict with dpdk
- 2 ixgbe interfaces given to with vm through vfio
- a virtio net device, connected to a tap interface through vhost-net

  /usr/bin/qemu-system-x86_64 -k fr -daemonize --enable-kvm -m 1G -cpu host \
    -smp 3 -serial telnet::40564,server,nowait -serial null \
    -qmp tcp::44340,server,nowait -monitor telnet::49229,server,nowait \
    -device ne2k_pci,mac=de:ad:de:01:02:03,netdev=user.0,addr=03 \
    -netdev user,id=user.0,hostfwd=tcp::34965-:22 \
    -device vfio-pci,host=0000:04:00.0 -device vfio-pci,host=0000:04:00.1 \
    -netdev type=tap,id=vhostnet0,script=no,vhost=on,queues=8 \
    -device virtio-net-pci,netdev=vhostnet0,ioeventfd=on,mq=on,vectors=17 \
    -hda "/path/to/ubuntu-14.04-template.qcow2" \
    -snapshot -vga none -display none

Move the tap interface in a netns, and configure it:

  ip netns add ns-tap
  ip netns exec ns-tap ip l set lo up
  ip link set tap0 netns ns-tap
  ip netns exec ns-tap ip l set tap0 down
  ip netns exec ns-tap ip l set addr 02:00:00:00:00:01 dev tap0
  ip netns exec ns-tap ip l set tap0 up
  ip netns exec ns-tap ip a a 1.1.1.1/24 dev tap0
  ip netns exec ns-tap arp -s 1.1.1.2 02:00:00:00:00:00
  ip netns exec ns-tap ip a

Move the ixgbe interface in a netns, and configure it:

  IXGBE=ixgbe2
  ip netns add ns-ixgbe
  ip netns exec ns-ixgbe ip l set lo up
  ip link set ${IXGBE} netns ns-ixgbe
  ip netns exec ns-ixgbe ip l set ${IXGBE} down
  ip netns exec ns-ixgbe ip l set addr 02:00:00:00:00:00 dev ${IXGBE}
  ip netns exec ns-ixgbe ip l set ${IXGBE} up
  ip netns exec ns-ixgbe ip a a 1.1.1.2/24 dev ${IXGBE}
  ip netns exec ns-ixgbe arp -s 1.1.1.1 02:00:00:00:00:01
  ip netns exec ns-ixgbe ip a

Guest configuration
-------------------

List of pci devices:

  00:02.0 Ethernet controller [0200]: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection [8086:10fb] (rev 01)
  00:03.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL-8029(AS) [10ec:8029]
  00:04.0 Ethernet controller [0200]: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection [8086:10fb] (rev 01)
  00:05.0 Ethernet controller [0200]: Red Hat, Inc Virtio network device [1af4:1000]

Compile dpdk:

  cd dpdk.org
  make config T=x86_64-native-linuxapp-gcc
  make -j4

Prepare environment:

  mkdir -p /mnt/huge
  mount -t hugetlbfs nodev /mnt/huge
  echo 256 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages
  modprobe uio_pci_generic
  python tools/dpdk_nic_bind.py -b uio_pci_generic 0000:00:02.0
  python tools/dpdk_nic_bind.py -b uio_pci_generic 0000:00:05.0

Run test
========

The test uses iperf to validate connectivity between the 2 netns of the
host and trough the guest.

Iperf is run with:

  # flow1: host -(ixgbe)-> guest -(virtio)-> host
  ip netns exec ns-tap iperf -s
  ip netns exec ns-ixgbe iperf -c 1.1.1.1 -t 10

  # flow2: host -(virtio)-> guest -(ixgbe)-> host
  ip netns exec ns-ixgbe iperf -s
  ip netns exec ns-tap iperf -c 1.1.1.2 -t 10

The guest runs testpmd with csum forward engine, its configuration
depends on the test case.

test1: large packets (lro/tso)
------------------------------

Configuration of testpmd:

  ./build/app/testpmd -l 0,1 --log-level 8 -- --total-num-mbufs=16384 \
    -i --port-topology=chained --disable-hw-vlan-filter \
    --disable-hw-vlan-strip --enable-rx-cksum --enable-lro \
    --crc-strip --txqflags=0

  set fwd csum
  tso set 1440 0
  csum set ip hw 0
  csum set tcp hw 0
  tso set 1440 1
  #csum set ip hw 1 # not supported by virtio
  csum set tcp hw 1
  start

Iperf log:

  root@ubuntu1404:~# ip netns exec ns-ixgbe iperf -c 1.1.1.1 -t 10
  ------------------------------------------------------------
  Client connecting to 1.1.1.1, TCP port 5001
  TCP window size: 85.0 KByte (default)
  ------------------------------------------------------------
  [  3] local 1.1.1.2 port 54460 connected with 1.1.1.1 port 5001
  [ ID] Interval       Transfer     Bandwidth
  [  3]  0.0-10.0 sec  6.14 GBytes  5.27 Gbits/sec
  root@ubuntu1404:~# ip netns exec ns-tap iperf -c 1.1.1.2 -t 10
  ------------------------------------------------------------
  Client connecting to 1.1.1.2, TCP port 5001
  TCP window size: 85.0 KByte (default)
  ------------------------------------------------------------
  [  3] local 1.1.1.1 port 58312 connected with 1.1.1.2 port 5001
  [ ID] Interval       Transfer     Bandwidth
  [  3]  0.0-10.0 sec  6.70 GBytes  5.76 Gbits/sec

Example of what we see with "set verbose 1" in testpmd:

  -- flow1: ixgbe2 -> port0 (ixgbe) -> testpmd -> port1 (virtio) <-> tap0
  port=0, mbuf=0x7f968ad9fdc0, pkt_len=24682, nb_segs=13:
  rx: l2_len=14 ethertype=800 l3_len=20 l4_proto=6 l4_len=32 flags=PKT_RX_L4_CKSUM_UNKNOWN PKT_RX_IP_CKSUM_UNKNOWN
  tx: m->l2_len=14 m->l3_len=20 m->l4_len=32
  tx: m->tso_segsz=1440
  tx: flags=PKT_TX_IP_CKSUM PKT_TX_L4_NO_CKSUM PKT_TX_TCP_SEG PKT_TX_IPV4

  -- flow2: tap0 -> port1 (virtio)-> testpmd -> port0 (ixgbe) -> ixgbe2
  port=1, mbuf=0x7f968acc9f40, pkt_len=42058, nb_segs=21:
  rx: l2_len=14 ethertype=800 l3_len=20 l4_proto=6 l4_len=32 flags=PKT_RX_L4_CKSUM_NONE PKT_RX_IP_CKSUM_UNKNOWN PKT_RX_LRO
  rx: m->lro_segsz=1440
  tx: m->l2_len=14 m->l3_len=20 m->l4_len=32
  tx: m->tso_segsz=1440
  tx: flags=PKT_TX_IP_CKSUM PKT_TX_L4_NO_CKSUM PKT_TX_TCP_SEG PKT_TX_IPV4

test2: hardware checksum only
-----------------------------

Configuration of testpmd:

  ./build/app/testpmd -l 0,1 --log-level 8 -- --total-num-mbufs=16384 \
    -i --port-topology=chained --disable-hw-vlan-filter \
    --disable-hw-vlan-strip --enable-rx-cksum --crc-strip --txqflags=0

  set fwd csum
  csum set ip hw 0
  csum set tcp hw 0
  csum set tcp hw 1
  start

Iperf log:

  root@ubuntu1404:~# ip netns exec ns-ixgbe iperf -c 1.1.1.1 -t 10
  ------------------------------------------------------------
  Client connecting to 1.1.1.1, TCP port 5001
  TCP window size: 85.0 KByte (default)
  ------------------------------------------------------------
  [  3] local 1.1.1.2 port 54462 connected with 1.1.1.1 port 5001
  [ ID] Interval       Transfer     Bandwidth
  [  3]  0.0-10.0 sec  4.49 GBytes  3.86 Gbits/sec
  root@ubuntu1404:~# ip netns exec ns-tap iperf -c 1.1.1.2 -t 10
  ------------------------------------------------------------
  Client connecting to 1.1.1.2, TCP port 5001
  TCP window size: 85.0 KByte (default)
  ------------------------------------------------------------
  [  3] local 1.1.1.1 port 58314 connected with 1.1.1.2 port 5001
  [ ID] Interval       Transfer     Bandwidth
  [  3]  0.0-10.0 sec  6.66 GBytes  5.72 Gbits/sec

Example of what we see with "set verbose 1" in testpmd:

  -- flow1: ixgbe2 -> port0 (ixgbe) -> testpmd -> port1 (virtio) <-> tap0
  port=0, mbuf=0x7f0adca89b40, pkt_len=1514, nb_segs=1:
  rx: l2_len=14 ethertype=800 l3_len=20 l4_proto=6 l4_len=32 flags=PKT_RX_L4_CKSUM_UNKNOWN PKT_RX_IP_CKSUM_UNKNOWN
  tx: m->l2_len=14 m->l3_len=20 m->l4_len=32
  tx: flags=PKT_TX_TCP_CKSUM PKT_TX_IPV4

  -- flow2: tap0 -> port1 (virtio)-> testpmd -> port0 (ixgbe) -> ixgbe2
  port=1, mbuf=0x7f0adcb98d80, pkt_len=1514, nb_segs=1:
  rx: l2_len=14 ethertype=800 l3_len=20 l4_proto=6 l4_len=32 flags=PKT_RX_L4_CKSUM_NONE PKT_RX_IP_CKSUM_UNKNOWN
  tx: m->l2_len=14 m->l3_len=20 m->l4_len=32
  tx: flags=PKT_TX_IP_CKSUM PKT_TX_TCP_CKSUM PKT_TX_IPV4

test3: no offload
-----------------

Configuration of testpmd:

  ./build/app/testpmd -l 0,1 --log-level 8 -- --total-num-mbufs=16384 \
    -i --port-topology=chained --disable-hw-vlan-filter --disable-hw-vlan-strip

  set fwd csum
  start

Iperf log:

  root@ubuntu1404:~# ip netns exec ns-ixgbe iperf -c 1.1.1.1 -t 10
  ------------------------------------------------------------
  Client connecting to 1.1.1.1, TCP port 5001
  TCP window size: 85.0 KByte (default)
  ------------------------------------------------------------
  [  3] local 1.1.1.2 port 54466 connected with 1.1.1.1 port 5001
  [ ID] Interval       Transfer     Bandwidth
  [  3]  0.0-10.0 sec  4.29 GBytes  3.68 Gbits/sec
  root@ubuntu1404:~# ip netns exec ns-tap iperf -c 1.1.1.2 -t 10
  ------------------------------------------------------------
  Client connecting to 1.1.1.2, TCP port 5001
  TCP window size: 85.0 KByte (default)
  ------------------------------------------------------------
  [  3] local 1.1.1.1 port 58316 connected with 1.1.1.2 port 5001
  [ ID] Interval       Transfer     Bandwidth
  [  3]  0.0-10.0 sec  6.66 GBytes  5.72 Gbits/sec

Example of what we see with "set verbose 1" in testpmd:

  -- flow1: ixgbe2 -> port0 (ixgbe) -> testpmd -> port1 (virtio) <-> tap0
  port=0, mbuf=0x7faf38b3e700, pkt_len=1514, nb_segs=1:
  rx: l2_len=14 ethertype=800 l3_len=20 l4_proto=6 l4_len=32 flags=PKT_RX_L4_CKSUM_UNKNOWN PKT_RX_IP_CKSUM_UNKNOWN
  tx: flags=PKT_TX_L4_NO_CKSUM PKT_TX_IPV4

  -- flow2: tap0 -> port1 (virtio)-> testpmd -> port0 (ixgbe) -> ixgbe2
  port=1, mbuf=0x7faf38b71500, pkt_len=1514, nb_segs=1:
  rx: l2_len=14 ethertype=800 l3_len=20 l4_proto=6 l4_len=32 flags=PKT_RX_L4_CKSUM_UNKNOWN PKT_RX_IP_CKSUM_UNKNOWN
  tx: flags=PKT_TX_L4_NO_CKSUM PKT_TX_IPV4


-- 
2.8.1

^ permalink raw reply	[flat|nested] 97+ messages in thread

end of thread, other threads:[~2016-11-08 14:58 UTC | newest]

Thread overview: 97+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-07-21  8:08 [dpdk-dev] [PATCH 00/12] net/virtio: add offload support Olivier Matz
2016-07-21  8:08 ` [dpdk-dev] [PATCH 01/12] virtio: move device initialization in a function Olivier Matz
2016-07-21  8:08 ` [dpdk-dev] [PATCH 02/12] virtio: setup and start cq in configure callback Olivier Matz
2016-07-21 21:15   ` Stephen Hemminger
2016-07-22  7:54     ` Olivier Matz
2016-07-21  8:08 ` [dpdk-dev] [PATCH 03/12] virtio: reinitialize the device " Olivier Matz
2016-07-21  8:08 ` [dpdk-dev] [PATCH 04/12] mbuf: add function to calculate a checksum Olivier Matz
2016-07-21 10:51   ` Ananyev, Konstantin
2016-07-21 16:26     ` Don Provan
2016-07-21 16:46       ` Olivier Matz
2016-07-22  8:24     ` Olivier Matz
2016-08-29 14:52       ` Olivier Matz
2016-07-21  8:08 ` [dpdk-dev] [PATCH 05/12] mbuf: add new Rx checksum mbuf flags Olivier Matz
2016-07-21 21:22   ` Stephen Hemminger
2016-07-22  8:03     ` Olivier Matz
2016-07-21  8:08 ` [dpdk-dev] [PATCH 06/12] app/testpmd: fix checksum stats in csum engine Olivier Matz
2016-07-21  8:08 ` [dpdk-dev] [PATCH 07/12] mbuf: new flag for LRO Olivier Matz
2016-07-21  8:08 ` [dpdk-dev] [PATCH 08/12] app/testpmd: display lro segment size Olivier Matz
2016-07-21  8:08 ` [dpdk-dev] [PATCH 09/12] virtio: add Rx checksum offload support Olivier Matz
2016-07-27  9:52   ` Wang, Xiao W
2016-07-21  8:08 ` [dpdk-dev] [PATCH 10/12] virtio: add Tx " Olivier Matz
2016-07-21  8:08 ` [dpdk-dev] [PATCH 11/12] virtio: add Lro support Olivier Matz
2016-07-21  8:08 ` [dpdk-dev] [PATCH 12/12] virtio: add Tso support Olivier Matz
2016-10-03  9:00 ` [dpdk-dev] [PATCH v2 00/12] net/virtio: add offload support Olivier Matz
2016-10-03  9:00   ` [dpdk-dev] [PATCH v2 01/12] virtio: move device initialization in a function Olivier Matz
2016-10-11 12:30     ` Maxime Coquelin
2016-10-03  9:00   ` [dpdk-dev] [PATCH v2 02/12] virtio: setup and start cq in configure callback Olivier Matz
2016-10-11 12:47     ` Maxime Coquelin
2016-10-03  9:00   ` [dpdk-dev] [PATCH v2 03/12] virtio: reinitialize the device " Olivier Matz
2016-10-11 13:13     ` Maxime Coquelin
2016-10-12 14:41     ` Yuanhan Liu
2016-10-12 16:01       ` Olivier MATZ
2016-10-13  7:54         ` Yuanhan Liu
2016-10-13 13:57           ` Olivier MATZ
2016-10-03  9:00   ` [dpdk-dev] [PATCH v2 04/12] net: add function to calculate a checksum in a mbuf Olivier Matz
2016-10-11 13:25     ` Maxime Coquelin
2016-10-11 13:33       ` Olivier MATZ
2016-10-03  9:00   ` [dpdk-dev] [PATCH v2 05/12] mbuf: add new Rx checksum mbuf flags Olivier Matz
2016-10-11 13:43     ` Maxime Coquelin
2016-10-03  9:00   ` [dpdk-dev] [PATCH v2 06/12] app/testpmd: fix checksum stats in csum engine Olivier Matz
2016-10-11 13:46     ` Maxime Coquelin
2016-10-03  9:00   ` [dpdk-dev] [PATCH v2 07/12] mbuf: new flag for LRO Olivier Matz
2016-10-11 13:48     ` Maxime Coquelin
2016-10-03  9:00   ` [dpdk-dev] [PATCH v2 08/12] app/testpmd: display lro segment size Olivier Matz
2016-10-11 13:49     ` Maxime Coquelin
2016-10-03  9:00   ` [dpdk-dev] [PATCH v2 09/12] virtio: add Rx checksum offload support Olivier Matz
2016-10-03 12:51     ` Maxime Coquelin
2016-10-05 11:56       ` Olivier Matz
2016-10-05 13:27         ` Maxime Coquelin
2016-10-05 13:30           ` Olivier Matz
2016-10-12 13:02           ` Yuanhan Liu
2016-10-12 15:55             ` Olivier MATZ
2016-10-11 14:04     ` Maxime Coquelin
2016-10-11 14:29       ` Olivier MATZ
2016-10-11 14:36         ` Maxime Coquelin
2016-10-11 14:49           ` Olivier MATZ
2016-10-03  9:00   ` [dpdk-dev] [PATCH v2 10/12] virtio: add Tx " Olivier Matz
2016-10-07  7:25     ` Maxime Coquelin
2016-10-07 16:36       ` Olivier Matz
2016-10-13  8:38     ` Yuanhan Liu
2016-10-13 13:58       ` Olivier MATZ
2016-10-03  9:00   ` [dpdk-dev] [PATCH v2 11/12] virtio: add Lro support Olivier Matz
2016-10-11 14:21     ` Maxime Coquelin
2016-10-03  9:00   ` [dpdk-dev] [PATCH v2 12/12] virtio: add Tso support Olivier Matz
2016-10-13  8:18     ` Yuanhan Liu
2016-10-13 14:02       ` Olivier MATZ
2016-10-13 14:16         ` Yuanhan Liu
2016-10-13 14:52           ` Olivier MATZ
2016-10-13 15:01             ` Yuanhan Liu
2016-10-13 15:15               ` Olivier MATZ
2016-10-13 15:29                 ` Yuanhan Liu
2016-10-13 15:45                   ` Olivier Matz
2016-10-13 16:01                     ` Yuanhan Liu
2016-10-13 15:04             ` Yuanhan Liu
2016-10-13 23:33       ` Stephen Hemminger
2016-10-18 14:07         ` Olivier Matz
2016-10-11 11:35   ` [dpdk-dev] [PATCH v2 00/12] net/virtio: add offload support Yuanhan Liu
2016-10-11 12:14     ` Olivier MATZ
2016-10-11 15:37       ` Yuanhan Liu
2016-10-13 14:15 ` [dpdk-dev] [PATCH v3 " Olivier Matz
2016-10-13 14:16   ` [dpdk-dev] [PATCH v3 01/12] net/virtio: move device initialization in a function Olivier Matz
2016-10-13 14:16   ` [dpdk-dev] [PATCH v3 02/12] net/virtio: setup and start cq in configure callback Olivier Matz
2016-11-02  1:38     ` Yao, Lei A
2016-11-08 14:58       ` Olivier Matz
2016-10-13 14:16   ` [dpdk-dev] [PATCH v3 03/12] net/virtio: reinitialize the device " Olivier Matz
2016-10-13 14:16   ` [dpdk-dev] [PATCH v3 04/12] net: add function to calculate a checksum in a mbuf Olivier Matz
2016-10-13 14:16   ` [dpdk-dev] [PATCH v3 05/12] mbuf: add new Rx checksum mbuf flags Olivier Matz
2016-10-13 14:16   ` [dpdk-dev] [PATCH v3 06/12] app/testpmd: adapt checksum stats in csum engine Olivier Matz
2016-10-13 14:16   ` [dpdk-dev] [PATCH v3 07/12] mbuf: new flag for LRO Olivier Matz
2016-10-13 14:16   ` [dpdk-dev] [PATCH v3 08/12] app/testpmd: display lro segment size Olivier Matz
2016-10-13 14:16   ` [dpdk-dev] [PATCH v3 09/12] net/virtio: add Rx checksum offload support Olivier Matz
2016-10-13 14:16   ` [dpdk-dev] [PATCH v3 10/12] net/virtio: add Tx " Olivier Matz
2016-10-13 14:16   ` [dpdk-dev] [PATCH v3 11/12] net/virtio: add Lro support Olivier Matz
2016-10-13 14:16   ` [dpdk-dev] [PATCH v3 12/12] net/virtio: add Tso support Olivier Matz
2016-10-13 16:05     ` Yuanhan Liu
2016-10-13 18:50       ` Thomas Monjalon
2016-10-13 19:58         ` Olivier Matz

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).