From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5228DA04BA; Wed, 7 Oct 2020 17:09:04 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 21B6E1BDBD; Wed, 7 Oct 2020 17:07:11 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 945441BCD5 for ; Wed, 7 Oct 2020 17:07:01 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from viacheslavo@nvidia.com) with SMTP; 7 Oct 2020 18:06:56 +0300 Received: from nvidia.com (pegasus12.mtr.labs.mlnx [10.210.17.40]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 097F6uPD010965; Wed, 7 Oct 2020 18:06:56 +0300 From: Viacheslav Ovsiienko To: dev@dpdk.org Cc: thomasm@monjalon.net, stephen@networkplumber.org, ferruh.yigit@intel.com, olivier.matz@6wind.com, jerinjacobk@gmail.com, maxime.coquelin@redhat.com, david.marchand@redhat.com, arybchenko@solarflare.com Date: Wed, 7 Oct 2020 15:06:46 +0000 Message-Id: <1602083215-22921-1-git-send-email-viacheslavo@nvidia.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: Subject: [dpdk-dev] [PATCH v2 0/9] ethdev: introduce Rx buffer split X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The DPDK datapath in the transmit direction is very flexible. An application can build the multi-segment packet and manages almost all data aspects - the memory pools where segments are allocated from, the segment lengths, the memory attributes like external buffers, registered for DMA, etc. In the receiving direction, the datapath is much less flexible, an application can only specify the memory pool to configure the receiving queue and nothing more. In order to extend receiving datapath capabilities it is proposed to add the way to provide extended information how to split the packets being received. The following structure is introduced to specify the Rx packet segment: struct rte_eth_rxseg { struct rte_mempool *mp; /* memory pools to allocate segment from */ uint16_t length; /* segment maximal data length */ uint16_t offset; /* data offset from beginning of mbuf data buffer */ uint32_t reserved; /* reserved field */ }; The new routine rte_eth_rx_queue_setup_ex() is introduced to setup the given Rx queue using the new extended Rx packet segment description: int rte_eth_rx_queue_setup_ex(uint16_t port_id, uint16_t rx_queue_id, uint16_t nb_rx_desc, unsigned int socket_id, const struct rte_eth_rxconf *rx_conf, const struct rte_eth_rxseg *rx_seg, uint16_t n_seg) This routine presents the two new parameters: rx_seg - pointer the array of segment descriptions, each element describes the memory pool, maximal data length, initial data offset from the beginning of data buffer in mbuf n_seg - number of elements in the array The new offload flag DEV_RX_OFFLOAD_BUFFER_SPLIT in device capabilities is introduced to present the way for PMD to report to application about supporting Rx packet split to configurable segments. Prior invoking the rte_eth_rx_queue_setup_ex() routine application should check DEV_RX_OFFLOAD_BUFFER_SPLIT flag. If the Rx queue is configured with new routine the packets being received will be split into multiple segments pushed to the mbufs with specified attributes. The PMD will allocate the first mbuf from the pool specified in the first segment descriptor and puts the data staring at specified offset in the allocated mbuf data buffer. If packet length exceeds the specified segment length the next mbuf will be allocated according to the next segment descriptor (if any) and data will be put in its data buffer at specified offset and not exceeding specified length. If there is no next descriptor the next mbuf will be allocated and filled in the same way (from the same pool and with the same buffer offset/length) as the current one. For example, let's suppose we configured the Rx queue with the following segments: seg0 - pool0, len0=14B, off0=RTE_PKTMBUF_HEADROOM seg1 - pool1, len1=20B, off1=0B seg2 - pool2, len2=20B, off2=0B seg3 - pool3, len3=512B, off3=0B The packet 46 bytes long will look like the following: seg0 - 14B long @ RTE_PKTMBUF_HEADROOM in mbuf from pool0 seg1 - 20B long @ 0 in mbuf from pool1 seg2 - 12B long @ 0 in mbuf from pool2 The packet 1500 bytes long will look like the following: seg0 - 14B @ RTE_PKTMBUF_HEADROOM in mbuf from pool0 seg1 - 20B @ 0 in mbuf from pool1 seg2 - 20B @ 0 in mbuf from pool2 seg3 - 512B @ 0 in mbuf from pool3 seg4 - 512B @ 0 in mbuf from pool3 seg5 - 422B @ 0 in mbuf from pool3 The offload DEV_RX_OFFLOAD_SCATTER must be present and configured to support new buffer split feature (if n_seg is greater than one). The new approach would allow splitting the ingress packets into multiple parts pushed to the memory with different attributes. For example, the packet headers can be pushed to the embedded data buffers within mbufs and the application data into the external buffers attached to mbufs allocated from the different memory pools. The memory attributes for the split parts may differ either - for example the application data may be pushed into the external memory located on the dedicated physical device, say GPU or NVMe. This would improve the DPDK receiving datapath flexibility with preserving compatibility with existing API. Also, the proposed segment description might be used to specify Rx packet split for some other features. For example, provide the way to specify the extra memory pool for the Header Split feature of some Intel PMD. Signed-off-by: Viacheslav Ovsiienko --- [RFC]: http://patches.dpdk.org/patch/75582/ Related deprecation note (revoked): http://patches.dpdk.org/patch/75205/ v1: http://patches.dpdk.org/patch/79594/ v2: add feature supoport to mlx5 PMD Viacheslav Ovsiienko (9): ethdev: introduce Rx buffer split app/testpmd: add multiple pools per core creation app/testpmd: add buffer split offload configuration app/testpmd: add rxpkts commands and parameters app/testpmd: add extended Rx queue setup net/mlx5: add extended Rx queue setup routine net/mlx5: configure Rx queue to support split net/mlx5: register multiple pool for Rx queue net/mlx5: update Rx datapath to support split app/test-pmd/bpf_cmd.c | 4 +- app/test-pmd/cmdline.c | 96 +++++++++++--- app/test-pmd/config.c | 63 ++++++++- app/test-pmd/parameters.c | 39 +++++- app/test-pmd/testpmd.c | 108 +++++++++++----- app/test-pmd/testpmd.h | 41 +++++- doc/guides/nics/features.rst | 15 +++ doc/guides/rel_notes/release_20_11.rst | 6 + doc/guides/testpmd_app_ug/run_app.rst | 16 ++- doc/guides/testpmd_app_ug/testpmd_funcs.rst | 21 ++- drivers/net/mlx5/linux/mlx5_os.c | 2 + drivers/net/mlx5/mlx5.h | 3 + drivers/net/mlx5/mlx5_mr.c | 3 + drivers/net/mlx5/mlx5_rxq.c | 194 +++++++++++++++++++++++----- drivers/net/mlx5/mlx5_rxtx.c | 3 +- drivers/net/mlx5/mlx5_rxtx.h | 10 +- drivers/net/mlx5/mlx5_trigger.c | 20 +-- lib/librte_ethdev/rte_ethdev.c | 174 +++++++++++++++++++++++++ lib/librte_ethdev/rte_ethdev.h | 16 +++ lib/librte_ethdev/rte_ethdev_driver.h | 10 ++ 20 files changed, 733 insertions(+), 111 deletions(-) -- 1.8.3.1