From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by dpdk.org (Postfix) with ESMTP id AEEC8B4D7 for ; Wed, 18 Feb 2015 12:03:17 +0100 (CET) Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga103.jf.intel.com with ESMTP; 18 Feb 2015 02:57:47 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.09,512,1418112000"; d="scan'208";a="653742454" Received: from irvmail001.ir.intel.com ([163.33.26.43]) by orsmga001.jf.intel.com with ESMTP; 18 Feb 2015 03:03:06 -0800 Received: from sivswdev02.ir.intel.com (sivswdev02.ir.intel.com [10.237.217.46]) by irvmail001.ir.intel.com (8.14.3/8.13.6/MailSET/Hub) with ESMTP id t1IB340S022803 for ; Wed, 18 Feb 2015 11:03:04 GMT Received: from sivswdev02.ir.intel.com (localhost [127.0.0.1]) by sivswdev02.ir.intel.com with ESMTP id t1IB34O8004233 for ; Wed, 18 Feb 2015 11:03:04 GMT Received: (from smonroy@localhost) by sivswdev02.ir.intel.com with id t1IB34ou004229 for dev@dpdk.org; Wed, 18 Feb 2015 11:03:04 GMT From: Sergio Gonzalez Monroy To: dev@dpdk.org Date: Wed, 18 Feb 2015 11:03:02 +0000 Message-Id: <1424257383-4177-2-git-send-email-sergio.gonzalez.monroy@intel.com> X-Mailer: git-send-email 1.8.5.4 In-Reply-To: <1424257383-4177-1-git-send-email-sergio.gonzalez.monroy@intel.com> References: <1424102913-18944-1-git-send-email-sergio.gonzalez.monroy@intel.com> <1424257383-4177-1-git-send-email-sergio.gonzalez.monroy@intel.com> Subject: [dpdk-dev] [PATCH v2 1/2] mbuf: Introduce IND_ATTACHED_MBUF flag X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Feb 2015 11:03:20 -0000 Currently for mbufs with refcnt, we cannot free mbufs with external memory buffers (ie. vhost zero copy), as they are recognized as indirect attached mbufs and therefore we free the direct mbuf it points to, resulting in an error in the case of external memory buffers. We solve the issue by introducing the IND_ATTACHED_MBUF flag, which indicates that the mbuf is an indirect attached mbuf pointing to another mbuf. When we free an mbuf, we only free the direct mbuf if the flag is set. Freeing an mbuf with external buffer is the same as freeing a non attached mbuf. The flag is set during attach and clear on detach. So in the case of vhost zero copy where we have mbufs with external buffers, by default we just free the mbuf and it is up to the user to deal with the external buffer. This patch would allow the removal of the RTE_MBUF_REFCNT config option, setting refcnt for all mbufs permanently. The patch also modifies the vhost example as it was using the RTE_MBUF_INDERECT macro to detect if it was an mbuf with external buffer. Signed-off-by: Sergio Gonzalez Monroy Acked-by: Olivier Matz --- v2: - Add missing parenthesis to RTE_MBUF_INDIRECT macro examples/vhost/main.c | 6 ++++-- lib/librte_mbuf/rte_mbuf.h | 15 +++++++++------ 2 files changed, 13 insertions(+), 8 deletions(-) diff --git a/examples/vhost/main.c b/examples/vhost/main.c index 3a35359..5e341d6 100644 --- a/examples/vhost/main.c +++ b/examples/vhost/main.c @@ -139,6 +139,8 @@ /* Number of descriptors per cacheline. */ #define DESC_PER_CACHELINE (RTE_CACHE_LINE_SIZE / sizeof(struct vring_desc)) +#define MBUF_EXT_MEM(mb) (RTE_MBUF_FROM_BADDR((mb)->buf_addr) != (mb)) + /* mask of enabled ports */ static uint32_t enabled_port_mask = 0; @@ -1567,7 +1569,7 @@ txmbuf_clean_zcp(struct virtio_net *dev, struct vpool *vpool) for (index = 0; index < mbuf_count; index++) { mbuf = __rte_mbuf_raw_alloc(vpool->pool); - if (likely(RTE_MBUF_INDIRECT(mbuf))) + if (likely(MBUF_EXT_MEM(mbuf))) pktmbuf_detach_zcp(mbuf); rte_ring_sp_enqueue(vpool->ring, mbuf); @@ -1630,7 +1632,7 @@ static void mbuf_destroy_zcp(struct vpool *vpool) for (index = 0; index < mbuf_count; index++) { mbuf = __rte_mbuf_raw_alloc(vpool->pool); if (likely(mbuf != NULL)) { - if (likely(RTE_MBUF_INDIRECT(mbuf))) + if (likely(MBUF_EXT_MEM(mbuf))) pktmbuf_detach_zcp(mbuf); rte_ring_sp_enqueue(vpool->ring, (void *)mbuf); } diff --git a/lib/librte_mbuf/rte_mbuf.h b/lib/librte_mbuf/rte_mbuf.h index 16059c6..1e5aa1f 100644 --- a/lib/librte_mbuf/rte_mbuf.h +++ b/lib/librte_mbuf/rte_mbuf.h @@ -162,6 +162,8 @@ extern "C" { /** Tell the NIC it's an outer IPv6 packet for tunneling packet */ #define PKT_TX_OUTER_IPV6 (1ULL << 60) +#define IND_ATTACHED_MBUF (1ULL << 62) /**< Indirect attached mbuf */ + /* Use final bit of flags to indicate a control mbuf */ #define CTRL_MBUF_FLAG (1ULL << 63) /**< Mbuf contains control data */ @@ -305,13 +307,12 @@ struct rte_mbuf { /** * Returns TRUE if given mbuf is indirect, or FALSE otherwise. */ -#define RTE_MBUF_INDIRECT(mb) (RTE_MBUF_FROM_BADDR((mb)->buf_addr) != (mb)) +#define RTE_MBUF_INDIRECT(mb) ((mb)->ol_flags & IND_ATTACHED_MBUF) /** * Returns TRUE if given mbuf is direct, or FALSE otherwise. */ -#define RTE_MBUF_DIRECT(mb) (RTE_MBUF_FROM_BADDR((mb)->buf_addr) == (mb)) - +#define RTE_MBUF_DIRECT(mb) (!RTE_MBUF_INDIRECT(mb)) /** * Private data in case of pktmbuf pool. @@ -713,7 +714,7 @@ static inline void rte_pktmbuf_attach(struct rte_mbuf *mi, struct rte_mbuf *md) mi->next = NULL; mi->pkt_len = mi->data_len; mi->nb_segs = 1; - mi->ol_flags = md->ol_flags; + mi->ol_flags = md->ol_flags | IND_ATTACHED_MBUF; mi->packet_type = md->packet_type; __rte_mbuf_sanity_check(mi, 1); @@ -744,6 +745,8 @@ static inline void rte_pktmbuf_detach(struct rte_mbuf *m) RTE_PKTMBUF_HEADROOM : m->buf_len; m->data_len = 0; + + m->ol_flags = 0; } #endif /* RTE_MBUF_REFCNT */ @@ -757,7 +760,6 @@ __rte_pktmbuf_prefree_seg(struct rte_mbuf *m) #ifdef RTE_MBUF_REFCNT if (likely (rte_mbuf_refcnt_read(m) == 1) || likely (rte_mbuf_refcnt_update(m, -1) == 0)) { - struct rte_mbuf *md = RTE_MBUF_FROM_BADDR(m->buf_addr); rte_mbuf_refcnt_set(m, 0); @@ -765,7 +767,8 @@ __rte_pktmbuf_prefree_seg(struct rte_mbuf *m) * - detach mbuf * - free attached mbuf segment */ - if (unlikely (md != m)) { + if (RTE_MBUF_INDIRECT(m)) { + struct rte_mbuf *md = RTE_MBUF_FROM_BADDR(m->buf_addr); rte_pktmbuf_detach(m); if (rte_mbuf_refcnt_update(md, -1) == 0) __rte_mbuf_raw_free(md); -- 1.9.3