[dpdk-dev] [RFC PATCH 00/14] Extend the mbuf structure

[Bruce Richardson <bruce.richardson@intel.com>]

This patch set expands and enhances the mbuf data structure. This set includes patches previously
submitted by Olivier to rework the mbuf, but takes the rework further than proposed there.

NOTE: This is still a work in progress! Feedback at this stage is still welcome though.

Outline of changes are:
* adds a second cache line to the structure
* moves some fields to that cache line to make space
* eliminates the control mbuf structure again to save space
* compresses the data pointer down to an offset to save space
* adds in new fields for:
  - packet type
  - sequence number
  - inner_l3, l4 lengths 
  - max segment size
  - user data pointer
  - second vlan tag
* the flags field is expanded out to 64-bit, giving 48 new flags. The hash field is now called filters 
  and is also expanded.
* some other structures are flattened out or otherwise reworked, e.g. vlan_macip field is gone.
* the field order has changed as fields are ordered more by function where they are used, e.g. 
  rearm vs RX vs TX.
  
Following this patch set, the mbuf is larger, but still has free space in it for future expansion. The
first cache line has 8 bytes free, and the second has 32 bytes free.

Current status: This patch set compiles on 64-bit gcc and clang, and handles throughput traffic using
testpmd, in the fast path, and the full-features path cases, including handling jumbo frames. No
performance regressions are seen in either of those cases, however further testing is needed, and is 
ongoing. Also, 32-bit support is not yet done.

Bruce Richardson (10):
  mbuf: rename in_port to just port
  mbuf: reorder fields by time-of-use
  ixgbe: rework vector pmd following mbuf changes
  mbuf: split mbuf across two cache lines.
  Fix performance regression due to moved pool ptr
  mbuf: set next pointer to NULL on mbuf free.
  ixgbe: make mbuf_initializer queue variable global
  ixgbe: Make vector stores unaligned
  mbuf: cleanup + added in additional mbuf fields.
  ixgbe: Allow vector RX of scattered packets

Olivier Matz (4):
  mbuf: rename RTE_MBUF_SCATTER_GATHER into RTE_MBUF_REFCNT
  mbuf: remove rte_ctrlmbuf
  mbuf: remove the rte_pktmbuf structure
  mbuf: replace data pointer by an offset

 app/test-pmd/cmdline.c                             |   2 -
 app/test-pmd/csumonly.c                            |   6 +-
 app/test-pmd/flowgen.c                             |  18 +-
 app/test-pmd/icmpecho.c                            |   4 +-
 app/test-pmd/ieee1588fwd.c                         |   6 +-
 app/test-pmd/macfwd-retry.c                        |   2 +-
 app/test-pmd/macfwd.c                              |   8 +-
 app/test-pmd/macswap.c                             |   8 +-
 app/test-pmd/rxonly.c                              |  13 +-
 app/test-pmd/testpmd.c                             |  11 +-
 app/test-pmd/testpmd.h                             |   2 +-
 app/test-pmd/txonly.c                              |  45 +--
 app/test/commands.c                                |   2 -
 app/test/packet_burst_generator.c                  |  47 +--
 app/test/test_distributor.c                        |  18 +-
 app/test/test_distributor_perf.c                   |   4 +-
 app/test/test_mbuf.c                               | 102 +----
 app/test/test_sched.c                              |   4 +-
 app/test/test_table_acl.c                          |   7 +-
 app/test/test_table_pipeline.c                     |  10 +-
 config/common_bsdapp                               |   2 +-
 config/common_linuxapp                             |   2 +-
 doc/doxy-api.conf                                  |   2 +-
 examples/dpdk_qat/crypto.c                         |  22 +-
 examples/dpdk_qat/main.c                           |   2 +-
 examples/exception_path/main.c                     |  11 +-
 examples/ip_pipeline/main.c                        |   2 +-
 examples/ip_reassembly/main.c                      |   8 +-
 examples/ipv4_multicast/Makefile                   |   4 +-
 examples/ipv4_multicast/main.c                     |  16 +-
 examples/l3fwd-acl/main.c                          |   2 +-
 examples/l3fwd-power/main.c                        |   2 +-
 examples/l3fwd-vf/main.c                           |   2 +-
 examples/l3fwd/main.c                              |  10 +-
 examples/load_balancer/runtime.c                   |   2 +-
 .../client_server_mp/mp_client/client.c            |   2 +-
 examples/quota_watermark/qw/main.c                 |   4 +-
 examples/vhost/main.c                              |  79 ++--
 examples/vhost_xen/main.c                          |  22 +-
 lib/librte_distributor/rte_distributor.c           |   2 +-
 lib/librte_ip_frag/ip_frag_common.h                |  13 +-
 lib/librte_ip_frag/rte_ipv4_fragmentation.c        |  40 +-
 lib/librte_ip_frag/rte_ipv4_reassembly.c           |   6 +-
 lib/librte_ip_frag/rte_ipv6_fragmentation.c        |  38 +-
 lib/librte_ip_frag/rte_ipv6_reassembly.c           |   5 +-
 lib/librte_mbuf/rte_mbuf.c                         |  91 ++---
 lib/librte_mbuf/rte_mbuf.h                         | 424 ++++++++-------------
 lib/librte_pmd_bond/rte_eth_bond_pmd.c             |   4 +-
 lib/librte_pmd_e1000/e1000_ethdev.h                |  21 +
 lib/librte_pmd_e1000/em_rxtx.c                     |  73 ++--
 lib/librte_pmd_e1000/igb_rxtx.c                    |  83 ++--
 lib/librte_pmd_i40e/i40e_rxtx.c                    | 109 +++---
 lib/librte_pmd_ixgbe/ixgbe_ethdev.h                |  21 +
 lib/librte_pmd_ixgbe/ixgbe_rxtx.c                  | 151 ++++----
 lib/librte_pmd_ixgbe/ixgbe_rxtx.h                  |  25 +-
 lib/librte_pmd_ixgbe/ixgbe_rxtx_vec.c              | 395 ++++++++++---------
 lib/librte_pmd_pcap/rte_eth_pcap.c                 |  14 +-
 lib/librte_pmd_virtio/virtio_rxtx.c                |  22 +-
 lib/librte_pmd_virtio/virtqueue.h                  |   3 +-
 lib/librte_pmd_vmxnet3/vmxnet3_rxtx.c              |  29 +-
 lib/librte_pmd_xenvirt/rte_eth_xenvirt.c           |  14 +-
 lib/librte_pmd_xenvirt/virtqueue.h                 |   4 +-
 lib/librte_sched/rte_sched.c                       |  14 +-
 lib/librte_sched/rte_sched.h                       |  21 +-
 64 files changed, 1021 insertions(+), 1116 deletions(-)

-- 
1.9.3