From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id D6B25A0352; Thu, 16 Jan 2020 14:04:50 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 00E541D52C; Thu, 16 Jan 2020 14:04:45 +0100 (CET) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id E2EF21D425 for ; Thu, 16 Jan 2020 14:04:41 +0100 (CET) Received: from Internal Mail-Server by MTLPINE1 (envelope-from viacheslavo@mellanox.com) with ESMTPS (AES256-SHA encrypted); 16 Jan 2020 15:04:38 +0200 Received: from pegasus11.mtr.labs.mlnx (pegasus11.mtr.labs.mlnx [10.210.16.104]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 00GD4cK4008238; Thu, 16 Jan 2020 15:04:38 +0200 Received: from pegasus11.mtr.labs.mlnx (localhost [127.0.0.1]) by pegasus11.mtr.labs.mlnx (8.14.7/8.14.7) with ESMTP id 00GD4cLm000841; Thu, 16 Jan 2020 13:04:38 GMT Received: (from viacheslavo@localhost) by pegasus11.mtr.labs.mlnx (8.14.7/8.14.7/Submit) id 00GD4bbK000840; Thu, 16 Jan 2020 13:04:37 GMT X-Authentication-Warning: pegasus11.mtr.labs.mlnx: viacheslavo set sender to viacheslavo@mellanox.com using -f From: Viacheslav Ovsiienko To: dev@dpdk.org Cc: matan@mellanox.com, rasland@mellanox.com, orika@mellanox.com, shahafs@mellanox.com, olivier.matz@6wind.com, stephen@networkplumber.org, thomas@mellanox.net Date: Thu, 16 Jan 2020 13:04:24 +0000 Message-Id: <1579179869-32508-1-git-send-email-viacheslavo@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <20191118094938.192850-1-shahafs@mellanox.com> References: <20191118094938.192850-1-shahafs@mellanox.com> Subject: [dpdk-dev] [PATCH v4 0/5] mbuf: detach mbuf with pinned external buffer X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Today's pktmbuf pool contains only mbufs with no external buffers. This means data buffer for the mbuf should be placed right after the mbuf structure (+ the private data when enabled). On some cases, the application would want to have the buffers allocated from a different device in the platform. This is in order to do zero copy for the packet directly to the device memory. Examples for such devices can be GPU or storage device. For such cases the native pktmbuf pool does not fit since each mbuf would need to point to external buffer. To support above, the pktmbuf pool will be populated with mbuf pointing to the device buffers using the mbuf external buffer feature. The PMD will populate its receive queues with those buffer, so that every packet received will be scattered directly to the device memory. on the other direction, embedding the buffer pointer to the transmit queues of the NIC, will make the DMA to fetch device memory using peer to peer communication. Such mbuf with external buffer should be handled with care when mbuf is freed. Mainly the external buffer should not be detached, so that it can be reused for the next packet receive. This patch introduce a new flag on the rte_pktmbuf_pool_private structure to specify this mempool is for mbuf with pinned external buffer. Upon detach this flag is validated and buffer is not detached. A new mempool create wrapper is also introduced to help application to create and populate such mempool. Signed-off-by: Shahaf Shuler Signed-off-by: Viacheslav Ovsiienko --- RFC: http://patches.dpdk.org/patch/63077 v1: - http://patches.dpdk.org/cover/64424 v2: - fix rte_experimantal issue on comment addressing - rte_mbuf_has_pinned_extbuf return type is uint32_t - fix Power9 compilation issue v3: - http://patches.dpdk.org/cover/64424/ - fix "#include leftover v4: - introduce rte_pktmbuf_priv_flags - support cloning pinned mbufs as for regular mbufs with external buffers - address the minor comments Viacheslav Ovsiienko (5): mbuf: introduce routine to get private mbuf pool flags mbuf: detach mbuf with pinned external buffer mbuf: create packet pool with external memory buffers app/testpmd: add mempool with external data buffers net/mlx5: allow use allocated mbuf with external buffer app/test-pmd/config.c | 2 + app/test-pmd/flowgen.c | 3 +- app/test-pmd/parameters.c | 2 + app/test-pmd/testpmd.c | 81 +++++++++++++ app/test-pmd/testpmd.h | 4 +- app/test-pmd/txonly.c | 3 +- drivers/net/mlx5/mlx5_rxq.c | 7 +- drivers/net/mlx5/mlx5_rxtx.c | 2 +- drivers/net/mlx5/mlx5_rxtx.h | 2 +- drivers/net/mlx5/mlx5_rxtx_vec.h | 14 +-- drivers/net/mlx5/mlx5_rxtx_vec_altivec.h | 5 +- drivers/net/mlx5/mlx5_rxtx_vec_neon.h | 29 ++--- drivers/net/mlx5/mlx5_rxtx_vec_sse.h | 2 +- lib/librte_mbuf/rte_mbuf.c | 178 +++++++++++++++++++++++++++- lib/librte_mbuf/rte_mbuf.h | 196 +++++++++++++++++++++++++++++-- lib/librte_mbuf/rte_mbuf_version.map | 1 + 16 files changed, 487 insertions(+), 44 deletions(-) -- 1.8.3.1