Re: [dpdk-dev] [PATCH v2 00/17] ixgbe/mbuf: add TSO support

["Liu, Jijiang" <jijiang.liu@intel.com>]

Hi Olivier,

Do you plan to integrate this patch set with latest mbuf changes into DPDK1.8?

If yes, I can develop TSO feature of tunneling packet based on this patch set and the Mirek's i40e TSO patch set in the next release.

Thanks
Jijiang Liu

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Olivier Matz
> Sent: Monday, May 19, 2014 9:56 PM
> To: dev@dpdk.org
> Subject: [dpdk-dev] [PATCH v2 00/17] ixgbe/mbuf: add TSO support
> 
> This series add TSO support in ixgbe DPDK driver. This is the second version of the
> series. The first version (RFC) was posted 10 days ago and discussed on the list
> [1]. The list of changes is provided at the end of this cover letter.
> 
> Also, as discussed previously on the list [2], one problem is that there is not
> enough room in rte_mbuf today to store the required information to implement
> this feature:
>   - a new ol_flag
>   - the MSS
>   - the L4 header len
> 
> A solution would be to increase the size of the mbuf to 2 cache lines but it could
> have a bad impact on performance. This series proposes some rework to
> drastically reduce the size of the rte_mbuf structures before implementing the
> TSO, avoiding to change the mbuf size to 128 bytes.
> 
> After the rework of mbuf structures, the size of rte_mbuf structure is reduced by
> 9 bytes. The implementation of TSO requires to double the size of ol_flags (16 to
> 32 bits) and to double the size of offload information in order to add the mss and
> the l4 header length (32 to 64 bits). At the end of the whole series,
> sizeof(rte_mbuf) is still 64 bytes and 4 bytes are available for future use.
> 
> This rework causes a lot of modifications in the mbuf structure, implying some
> changes in the applications that directly use the mbuf structure fields instead of
> using the API functions (sometimes there is no function). In my opinion, it's the
> proper moment for this evolution as the 1.7.0 window is open.
> 
> About TSO, the new fields in mbuf try to be generic enough to apply to other
> hardware in the future. To delegate the TCP segmentation to the hardware, the
> user has to:
> 
>   - set the PKT_TX_TCP_SEG flag in mbuf->ol_flags (this flag implies
>     PKT_TX_IP_CKSUM and PKT_TX_TCP_CKSUM)
>   - fill the mbuf->hw_offload information: l2_len, l3_len, l4_len, mss
>   - calculate the pseudo header checksum and set it in the TCP header,
>     as required when doing hardware TCP checksum offload
>   - set the IP checksum to 0
> 
> Compilation of DPDK and examples is tested for the following
> targets: x86_64-*-linuxapp-gcc, i686-*-linuxapp-gcc, x86_64-*-bsdapp-gcc
> 
> The mbuf rework series is validated with autotests:
> 
>   cd dpdk.org/
>   make install T=x86_64-default-linuxapp-gcc
>   cd x86_64-default-linuxapp-gcc/
>   modprobe uio
>   insmod kmod/igb_uio.ko
>   python ../tools/igb_uio_bind.py -b igb_uio 0000:02:00.0
>   echo 0 > /proc/sys/kernel/randomize_va_space
>   echo 1000 > /sys/devices/system/node/node0/hugepages/hugepages-
> 2048kB/nr_hugepages
>   echo 1000 > /sys/devices/system/node/node1/hugepages/hugepages-
> 2048kB/nr_hugepages
>   mount -t hugetlbfs none /mnt/huge
>   make test
> 
> TSO is validated with IPv4 and IPv6 with testpmd (see the commit log of patch
> "ixgbe: support TCP segmentation offload" for details).
> 
> The performance non-regression has been tested with 6WINDGate fast path, and
> with test-pmd [3].
> 
> [1] http://dpdk.org/ml/archives/dev/2014-May/002322.html
> [2] http://dpdk.org/ml/archives/dev/2013-October/thread.html#572
> [3] http://dpdk.org/ml/archives/dev/2014-May/002516.html
> 
> 
> Changes included in v2:
> - rebase on head, resolve conflicts in 3 test-pmd engines
> - add more comments in rte_mbuf.h about how to use TSO
> - ixgbe: advertise the TSO feature
> - ixgbe: small optimization, move some tso code in the if (tx_ol_req)
> - split "mbuf: replace data pointer by an offset" (cosmetics vs functional)
> - split "ixgbe/mbuf: add TSO support" (ixgbe vs generic changes)
> - modifications of external PMDs (memnic, virtio, vmxnet3)
> - I did not add the new checksum flags PKT_RX_L4_CKSUM_GOOD and
>   PKT_RX_IP_CKSUM_GOOD as proposed on the list. The patch is ready but
>   I don't think it should be included in this series.
> 
> Olivier Matz (17):
>  dpdk             igb/ixgbe: fix IP checksum calculation
>  dpdk             mbuf: rename RTE_MBUF_SCATTER_GATHER into
>  dpdk             mbuf: remove rte_ctrlmbuf
>  dpdk             mbuf: remove the rte_pktmbuf structure
>  dpdk             mbuf: merge physaddr and buf_len in a bitfield
>  dpdk             mbuf: cosmetic changes in rte_mbuf structure
>  dpdk             mbuf: replace data pointer by an offset
>  dpdk             mbuf: add functions to get the name of an ol_flag
>  dpdk             mbuf: change ol_flags to 32 bits
>  dpdk             mbuf: rename vlan_macip_len in hw_offload and increase
>  dpdk             testpmd: modify source address to validate checksum
>  dpdk             mbuf: generic support of TCP segmentation offload
>  dpdk             ixgbe: support TCP segmentation offload
>  virtio-net-pmd   pmd: adapt to new rte_mbuf structure
>  vmxnet3-usermap  pmd: remove support of old dpdk versions  vmxnet3-usermap
> pmd: adapt to new rte_mbuf structure
>  memnic           pmd: adapt to new rte_mbuf structure
> 
> Diffstat:
>  dpdk.org/app/test-pmd/cmdline.c                                        |   60 -
>  dpdk.org/app/test-pmd/config.c                                         |   18
>  dpdk.org/app/test-pmd/csumonly.c                                       |   56
>  dpdk.org/app/test-pmd/flowgen.c                                        |   26
>  dpdk.org/app/test-pmd/icmpecho.c                                       |    6
>  dpdk.org/app/test-pmd/ieee1588fwd.c                                    |    6
>  dpdk.org/app/test-pmd/macfwd-retry.c                                   |    4
>  dpdk.org/app/test-pmd/macfwd.c                                         |   16
>  dpdk.org/app/test-pmd/macswap.c                                        |   16
>  dpdk.org/app/test-pmd/rxonly.c                                         |   53
>  dpdk.org/app/test-pmd/testpmd.c                                        |   14
>  dpdk.org/app/test-pmd/testpmd.h                                        |   15
>  dpdk.org/app/test-pmd/txonly.c                                         |   59 -
>  dpdk.org/app/test/commands.c                                           |    2
>  dpdk.org/app/test/test_mbuf.c                                          |  106 -
>  dpdk.org/app/test/test_sched.c                                         |    4
>  dpdk.org/config/defconfig_i686-default-linuxapp-gcc                    |    2
>  dpdk.org/config/defconfig_i686-default-linuxapp-icc                    |    2
>  dpdk.org/config/defconfig_x86_64-default-bsdapp-gcc                    |    2
>  dpdk.org/config/defconfig_x86_64-default-linuxapp-gcc                  |    2
>  dpdk.org/config/defconfig_x86_64-default-linuxapp-icc                  |    2
>  dpdk.org/doc/doxy-api.conf                                             |    2
>  dpdk.org/examples/dpdk_qat/crypto.c                                    |   22
>  dpdk.org/examples/dpdk_qat/main.c                                      |    2
>  dpdk.org/examples/exception_path/main.c                                |   13
>  dpdk.org/examples/ip_reassembly/ipv4_rsmbl.h                           |   30
>  dpdk.org/examples/ip_reassembly/main.c                                 |   10
>  dpdk.org/examples/ipv4_frag/Makefile                                   |    4
>  dpdk.org/examples/ipv4_frag/main.c                                     |    4
>  dpdk.org/examples/ipv4_frag/rte_ipv4_frag.h                            |   42
>  dpdk.org/examples/ipv4_multicast/Makefile                              |    4
>  dpdk.org/examples/ipv4_multicast/main.c                                |   16
>  dpdk.org/examples/l3fwd-power/main.c                                   |    2
>  dpdk.org/examples/l3fwd-vf/main.c                                      |    2
>  dpdk.org/examples/l3fwd/main.c                                         |   10
>  dpdk.org/examples/load_balancer/runtime.c                              |    2
>  dpdk.org/examples/multi_process/client_server_mp/mp_client/client.c    |    2
>  dpdk.org/examples/quota_watermark/qw/main.c                            |    4
>  dpdk.org/examples/vhost/main.c                                         |   43
>  dpdk.org/examples/vhost_xen/main.c                                     |   24
>  dpdk.org/lib/librte_eal/bsdapp/eal/include/exec-env/rte_kni_common.h   |    2
>  dpdk.org/lib/librte_eal/linuxapp/eal/include/exec-env/rte_kni_common.h |    2
>  dpdk.org/lib/librte_mbuf/rte_mbuf.c                                    |  105 -
>  dpdk.org/lib/librte_mbuf/rte_mbuf.h                                    |  580 ++++------
>  dpdk.org/lib/librte_pmd_e1000/em_rxtx.c                                |  163 +-
>  dpdk.org/lib/librte_pmd_e1000/igb_rxtx.c                               |  212 +--
>  dpdk.org/lib/librte_pmd_ixgbe/ixgbe_ethdev.c                           |    3
>  dpdk.org/lib/librte_pmd_ixgbe/ixgbe_rxtx.c                             |  419 ++++---
>  dpdk.org/lib/librte_pmd_ixgbe/ixgbe_rxtx.h                             |   10
>  dpdk.org/lib/librte_pmd_pcap/rte_eth_pcap.c                            |   14
>  dpdk.org/lib/librte_pmd_virtio/virtio_rxtx.c                           |   20
>  dpdk.org/lib/librte_pmd_virtio/virtqueue.h                             |   11
>  dpdk.org/lib/librte_pmd_vmxnet3/vmxnet3_rxtx.c                         |   37
>  dpdk.org/lib/librte_pmd_xenvirt/rte_eth_xenvirt.c                      |   14
>  dpdk.org/lib/librte_pmd_xenvirt/virtqueue.h                            |    4
>  dpdk.org/lib/librte_sched/rte_sched.c                                  |   14
>  dpdk.org/lib/librte_sched/rte_sched.h                                  |   10
>  memnic/pmd/pmd_memnic.c                                                |   14
>  vmxnet3-usermap/pmd/vmxnet3.c                                          |  119 --
>  virtio-net-pmd/virtio_user.c                                           |   44
>  60 files changed, 1217 insertions(+), 1289 deletions(-)
> 
> 
> --
> 1.9.2