From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by dpdk.org (Postfix) with ESMTP id B7A25B603 for ; Mon, 16 Feb 2015 17:12:22 +0100 (CET) Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga103.fm.intel.com with ESMTP; 16 Feb 2015 08:04:36 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.09,588,1418112000"; d="scan'208";a="528234274" Received: from irvmail001.ir.intel.com ([163.33.26.43]) by orsmga003.jf.intel.com with ESMTP; 16 Feb 2015 08:03:31 -0800 Received: from sivswdev02.ir.intel.com (sivswdev02.ir.intel.com [10.237.217.46]) by irvmail001.ir.intel.com (8.14.3/8.13.6/MailSET/Hub) with ESMTP id t1GG8XjV003704 for ; Mon, 16 Feb 2015 16:08:34 GMT Received: from sivswdev02.ir.intel.com (localhost [127.0.0.1]) by sivswdev02.ir.intel.com with ESMTP id t1GG8XME018989 for ; Mon, 16 Feb 2015 16:08:33 GMT Received: (from smonroy@localhost) by sivswdev02.ir.intel.com with id t1GG8XEx018985 for dev@dpdk.org; Mon, 16 Feb 2015 16:08:33 GMT From: Sergio Gonzalez Monroy To: dev@dpdk.org Date: Mon, 16 Feb 2015 16:08:32 +0000 Message-Id: <1424102913-18944-2-git-send-email-sergio.gonzalez.monroy@intel.com> X-Mailer: git-send-email 1.8.5.4 In-Reply-To: <1424102913-18944-1-git-send-email-sergio.gonzalez.monroy@intel.com> References: <1424102913-18944-1-git-send-email-sergio.gonzalez.monroy@intel.com> Subject: [dpdk-dev] [PATCH 1/2] mbuf: Introduce IND_ATTACHED_MBUF flag X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Feb 2015 16:12:24 -0000 Currently for mbufs with refcnt, we cannot free mbufs with external memory buffers (ie. vhost zero copy), as they are recognized as indirect attached mbufs and therefore we free the direct mbuf it points to, resulting in an error in the case of external memory buffers. We solve the issue by introducing the IND_ATTACHED_MBUF flag, which indicates that the mbuf is an indirect attached mbuf pointing to another mbuf. When we free an mbuf, we only free the direct mbuf if the flag is set. Freeing an mbuf with external buffer is the same as freeing a non attached mbuf. The flag is set during attach and clear on detach. So in the case of vhost zero copy where we have mbufs with external buffers, by default we just free the mbuf and it is up to the user to deal with the external buffer. This patch would allow the removal of the RTE_MBUF_REFCNT config option, setting refcnt for all mbufs permanently. The patch also modifies the vhost example as it was using the RTE_MBUF_INDERECT macro to detect if it was an mbuf with external buffer. Signed-off-by: Sergio Gonzalez Monroy --- examples/vhost/main.c | 6 ++++-- lib/librte_mbuf/rte_mbuf.h | 15 +++++++++------ 2 files changed, 13 insertions(+), 8 deletions(-) diff --git a/examples/vhost/main.c b/examples/vhost/main.c index 3a35359..5e341d6 100644 --- a/examples/vhost/main.c +++ b/examples/vhost/main.c @@ -139,6 +139,8 @@ /* Number of descriptors per cacheline. */ #define DESC_PER_CACHELINE (RTE_CACHE_LINE_SIZE / sizeof(struct vring_desc)) +#define MBUF_EXT_MEM(mb) (RTE_MBUF_FROM_BADDR((mb)->buf_addr) != (mb)) + /* mask of enabled ports */ static uint32_t enabled_port_mask = 0; @@ -1567,7 +1569,7 @@ txmbuf_clean_zcp(struct virtio_net *dev, struct vpool *vpool) for (index = 0; index < mbuf_count; index++) { mbuf = __rte_mbuf_raw_alloc(vpool->pool); - if (likely(RTE_MBUF_INDIRECT(mbuf))) + if (likely(MBUF_EXT_MEM(mbuf))) pktmbuf_detach_zcp(mbuf); rte_ring_sp_enqueue(vpool->ring, mbuf); @@ -1630,7 +1632,7 @@ static void mbuf_destroy_zcp(struct vpool *vpool) for (index = 0; index < mbuf_count; index++) { mbuf = __rte_mbuf_raw_alloc(vpool->pool); if (likely(mbuf != NULL)) { - if (likely(RTE_MBUF_INDIRECT(mbuf))) + if (likely(MBUF_EXT_MEM(mbuf))) pktmbuf_detach_zcp(mbuf); rte_ring_sp_enqueue(vpool->ring, (void *)mbuf); } diff --git a/lib/librte_mbuf/rte_mbuf.h b/lib/librte_mbuf/rte_mbuf.h index 16059c6..12e7545 100644 --- a/lib/librte_mbuf/rte_mbuf.h +++ b/lib/librte_mbuf/rte_mbuf.h @@ -162,6 +162,8 @@ extern "C" { /** Tell the NIC it's an outer IPv6 packet for tunneling packet */ #define PKT_TX_OUTER_IPV6 (1ULL << 60) +#define IND_ATTACHED_MBUF (1ULL << 62) /**< Indirect attached mbuf */ + /* Use final bit of flags to indicate a control mbuf */ #define CTRL_MBUF_FLAG (1ULL << 63) /**< Mbuf contains control data */ @@ -305,13 +307,12 @@ struct rte_mbuf { /** * Returns TRUE if given mbuf is indirect, or FALSE otherwise. */ -#define RTE_MBUF_INDIRECT(mb) (RTE_MBUF_FROM_BADDR((mb)->buf_addr) != (mb)) +#define RTE_MBUF_INDIRECT(mb) (mb->ol_flags & IND_ATTACHED_MBUF) /** * Returns TRUE if given mbuf is direct, or FALSE otherwise. */ -#define RTE_MBUF_DIRECT(mb) (RTE_MBUF_FROM_BADDR((mb)->buf_addr) == (mb)) - +#define RTE_MBUF_DIRECT(mb) (!RTE_MBUF_INDIRECT(mb)) /** * Private data in case of pktmbuf pool. @@ -713,7 +714,7 @@ static inline void rte_pktmbuf_attach(struct rte_mbuf *mi, struct rte_mbuf *md) mi->next = NULL; mi->pkt_len = mi->data_len; mi->nb_segs = 1; - mi->ol_flags = md->ol_flags; + mi->ol_flags = md->ol_flags | IND_ATTACHED_MBUF; mi->packet_type = md->packet_type; __rte_mbuf_sanity_check(mi, 1); @@ -744,6 +745,8 @@ static inline void rte_pktmbuf_detach(struct rte_mbuf *m) RTE_PKTMBUF_HEADROOM : m->buf_len; m->data_len = 0; + + m->ol_flags = 0; } #endif /* RTE_MBUF_REFCNT */ @@ -757,7 +760,6 @@ __rte_pktmbuf_prefree_seg(struct rte_mbuf *m) #ifdef RTE_MBUF_REFCNT if (likely (rte_mbuf_refcnt_read(m) == 1) || likely (rte_mbuf_refcnt_update(m, -1) == 0)) { - struct rte_mbuf *md = RTE_MBUF_FROM_BADDR(m->buf_addr); rte_mbuf_refcnt_set(m, 0); @@ -765,7 +767,8 @@ __rte_pktmbuf_prefree_seg(struct rte_mbuf *m) * - detach mbuf * - free attached mbuf segment */ - if (unlikely (md != m)) { + if (RTE_MBUF_INDIRECT(m)) { + struct rte_mbuf *md = RTE_MBUF_FROM_BADDR(m->buf_addr); rte_pktmbuf_detach(m); if (rte_mbuf_refcnt_update(md, -1) == 0) __rte_mbuf_raw_free(md); -- 1.9.3