Re: [dpdk-dev] [PATCH v5 0/3] Support TCP/IPv4 GRO in DPDK

[Jiayu Hu <jiayu.hu@intel.com>]

On Mon, Jun 19, 2017 at 09:39:11AM +0800, Tan, Jianfeng wrote:
> Hi Jiayu,
> 
> You need to update the document:
> - Release note file: release_17_08.rst.
> - A howto doc is welcomed.

Thanks. I will update them in next patch.

> 
> 
> On 6/18/2017 3:21 PM, Jiayu Hu wrote:
> > Generic Receive Offload (GRO) is a widely used SW-based offloading
> > technique to reduce per-packet processing overhead. It gains performance
> > by reassembling small packets into large ones. Therefore, we propose to
> > support GRO in DPDK.
> > 
> > To enable more flexibility to applications, DPDK GRO is implemented as
> > a user library. Applications explicitly use the GRO library to merge
> > small packets into large ones. DPDK GRO provides two reassembly modes.
> > One is called lightweigth mode, the other is called heavyweight mode.
> > If applications want merge packets in a simple way, they can use
> > lightweigth mode. If applications need more fine-grained controls,
> > they can choose heavyweigth mode.
> 
> So what's the real difference between the two modes? Might be an example is
> good way to clarify.

The heavyweight mode merges packets in a burst-mode. Applications just need
to give N packets to the heavyweight mode API, rte_gro_reassemble_burst.
After rte_gro_reassemble_burst returns, packets are merged together. For
applications, to use the heavyweight mode is very simple and they don't
need to allocate any GRO tables before. The lightweight mode enables more
flexibility to applications. Applications need to create a GRO table before
invoking the lightweight mode API, rte_gro_reassemble, to merge packets.
Besides, rte_gro_reassemble just processes one packet at a time. No matter
if the packet is merged successfully or not, it's stored in the GRO table.
When applications want these processed packets, they need to manually flush
them from the GRO table. You can see more detaileds in the next patch
'add Generic Receive Offload API framework'.

> 
> > 
> > This patchset is to support TCP/IPv4 GRO in DPDK. The first patch is to
> > provide a GRO API framework. The second patch is to support TCP/IPv4 GRO.
> > The last patch demonstrates how to use GRO library in app/testpmd.
> 
> In which mode?

Testpmd just demonstrate the usage of the lightweight mode.

> 
> > 
> > We perform two iperf tests (with DPDK GRO and without DPDK GRO) to see
> > the performance gains from DPDK GRO. Specifically, the experiment
> > environment is:
> > a. Two 10Gbps physical ports (p0 and p1) on one host are linked together;
> > b. p0 is in networking namespace ns1, whose IP is 1.1.2.3. Iperf client
> > runs on p0, which sends TCP/IPv4 packets. The OS in VM is ubuntu 14.04;
> > c. testpmd runs on p1. Besides, testpmd has a vdev which connects to a
> > VM via vhost-user and virtio-net. The VM runs iperf server, whose IP
> > is 1.1.2.4;
> > d. p0 turns on TSO; VM turns off kernel GRO; testpmd runs in iofwd mode.
> > iperf client and server use the following commands:
> > 	- client: ip netns exec ns1 iperf -c 1.1.2.4 -i2 -t 60 -f g -m
> > 	- server: iperf -s -f g
> > Two test cases are:
> > a. w/o DPDK GRO: run testpmd without GRO
> > b. w DPDK GRO: testpmd enables GRO for p1
> > Result:
> > With GRO, the throughput improvement is around 40%.
> 
> Do you try running several pairs of iperf-s and iperf-c tests (on 40Gb
> NICs)? It can not only prove the performance, but also the functionality
> correctness.

Besides the one pair scenario, I just tried two pairs of iperf-s and iperf-c.
Thanks for your advices, amd I will do more testes in next patch.


Thanks,
Jiayu

> 
> Thanks,
> Jianfeng
> 
> > 
> > Change log
> > ==========
> > v5:
> > - fix some bugs
> > - fix coding style issues
> > v4:
> > - implement DPDK GRO as an application-used library
> > - introduce lightweight and heavyweight working modes to enable
> > 	fine-grained controls to applications
> > - replace cuckoo hash tables with simpler table structure
> > v3:
> > - fix compilation issues.
> > v2:
> > - provide generic reassembly function;
> > - implement GRO as a device ability:
> > add APIs for devices to support GRO;
> > add APIs for applications to enable/disable GRO;
> > - update testpmd example.
> > 
> > Jiayu Hu (3):
> >    lib: add Generic Receive Offload API framework
> >    lib/gro: add TCP/IPv4 GRO support
> >    app/testpmd: enable TCP/IPv4 GRO
> > 
> >   app/test-pmd/cmdline.c       |  45 ++++
> >   app/test-pmd/config.c        |  29 +++
> >   app/test-pmd/iofwd.c         |   6 +
> >   app/test-pmd/testpmd.c       |   3 +
> >   app/test-pmd/testpmd.h       |  11 +
> >   config/common_base           |   5 +
> >   lib/Makefile                 |   1 +
> >   lib/librte_gro/Makefile      |  51 +++++
> >   lib/librte_gro/rte_gro.c     | 248 ++++++++++++++++++++
> >   lib/librte_gro/rte_gro.h     | 217 ++++++++++++++++++
> >   lib/librte_gro/rte_gro_tcp.c | 527 +++++++++++++++++++++++++++++++++++++++++++
> >   lib/librte_gro/rte_gro_tcp.h | 210 +++++++++++++++++
> >   mk/rte.app.mk                |   1 +
> >   13 files changed, 1354 insertions(+)
> >   create mode 100644 lib/librte_gro/Makefile
> >   create mode 100644 lib/librte_gro/rte_gro.c
> >   create mode 100644 lib/librte_gro/rte_gro.h
> >   create mode 100644 lib/librte_gro/rte_gro_tcp.c
> >   create mode 100644 lib/librte_gro/rte_gro_tcp.h
> >