From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <s.vyazmitinov@brain4net.com>
Received: from mail-lf0-f46.google.com (mail-lf0-f46.google.com
[209.85.215.46]) by dpdk.org (Postfix) with ESMTP id A2F6C5A29
for <dev@dpdk.org>; Wed, 11 Jan 2017 04:24:35 +0100 (CET)
Received: by mail-lf0-f46.google.com with SMTP id v186so74150972lfa.1
for <dev@dpdk.org>; Tue, 10 Jan 2017 19:24:35 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=brain4net.com; s=google;
h=from:to:cc:subject:date:message-id;
bh=4uDDDg4PHN8TDakzyYWcXL5KLSMy6BzbLcfc6BHt0Cg=;
b=Tenma90G09IDGPqrPLclVJKQ6PQDRzOS66b8Rl7DH2fx1GI0PruNhOmCEXWjuPwrE/
KB/WWHBYddHfeWzXJoF8p0PWfxcTj0TxKdxRt95uVMS8KpfQVP2gLBu96n1YxYuTmkIC
fCm9f24ThcZg4oo00kW291zA4LhGE5ZYqlm08=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20161025;
h=x-gm-message-state:from:to:cc:subject:date:message-id;
bh=4uDDDg4PHN8TDakzyYWcXL5KLSMy6BzbLcfc6BHt0Cg=;
b=KAED9pQAQM1ZquUYDmR7bnNizoB2M2wZbQvJ+5RTFfQ1xARqtX0+3Al3lB3BbCRUVE
RLu77Owi0ERi5EEiqumRsrL1eO3TpBOpK/egF2DryBZhMF8XZuGRR6bUarDpfLubc9tc
a+Yd5AXQN/3Fkayb6ZT+QtUQw6duhOqe2TKyFglgZzNmzvqlKuOeQlmMXt9SrNlQdRRD
xNhlTVv41R1Ii3DzEFq7UuYDP2iBSQdgSSEgwFpAI/prw7I2AIXA0LvsEpi57mRQCNr8
sonyZJxR/ZsUh0lhOdZ6eQ40FXRCTJB0ybia9rdzikq5IfArzBiRz1/3QZzRlEgAcZ/S
fcEQ==
X-Gm-Message-State: AIkVDXLa24mGghz/hjk27Zyl54HTa4yaWO04gxJGaar+74z5XWW7QWziRYME2HbyBdAY6Fu4
X-Received: by 10.46.20.73 with SMTP id 9mr2378965lju.10.1484105075205;
Tue, 10 Jan 2017 19:24:35 -0800 (PST)
Received: from localhost.localdomain ([91.201.72.212])
by smtp.gmail.com with ESMTPSA id l67sm4513lfb.15.2017.01.10.19.24.34
(version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128);
Tue, 10 Jan 2017 19:24:34 -0800 (PST)
From: Sergey Vyazmitinov <s.vyazmitinov@brain4net.com>
To: olivier.matz@6wind.com
Cc: ferruh.yigit@intel.com, dev@dpdk.org,
Sergey Vyazmitinov <s.vyazmitinov@brain4net.com>
Date: Fri, 30 Dec 2016 04:50:16 +0700
Message-Id: <1483048216-2936-1-git-send-email-s.vyazmitinov@brain4net.com>
X-Mailer: git-send-email 2.7.4
Subject: [dpdk-dev] [PATCH] kni: use bulk functions to allocate and free
mbufs
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <http://dpdk.org/ml/options/dev>,
<mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://dpdk.org/ml/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <http://dpdk.org/ml/listinfo/dev>,
<mailto:dev-request@dpdk.org?subject=subscribe>
X-List-Received-Date: Wed, 11 Jan 2017 03:24:35 -0000
Optimized kni_allocate_mbufs and kni_free_mbufs by using mbuf bulk
functions. This can improve performance more than two times.
Signed-off-by: Sergey Vyazmitinov <s.vyazmitinov@brain4net.com>
---
lib/librte_kni/rte_kni.c | 44 +++++++++++++++++-----------------------
lib/librte_kni/rte_kni_fifo.h | 18 ++++++++++++++++
lib/librte_mbuf/rte_mbuf.h | 32 +++++++++++++++++++++++++++++
lib/librte_mempool/rte_mempool.h | 6 ++++++
4 files changed, 75 insertions(+), 25 deletions(-)
diff --git a/lib/librte_kni/rte_kni.c b/lib/librte_kni/rte_kni.c
index a80cefd..cb4cfa6 100644
--- a/lib/librte_kni/rte_kni.c
+++ b/lib/librte_kni/rte_kni.c
@@ -590,22 +590,21 @@ rte_kni_rx_burst(struct rte_kni *kni, struct rte_mbuf **mbufs, unsigned num)
static void
kni_free_mbufs(struct rte_kni *kni)
{
- int i, ret;
+ unsigned freeing;
struct rte_mbuf *pkts[MAX_MBUF_BURST_NUM];
- ret = kni_fifo_get(kni->free_q, (void **)pkts, MAX_MBUF_BURST_NUM);
- if (likely(ret > 0)) {
- for (i = 0; i < ret; i++)
- rte_pktmbuf_free(pkts[i]);
+ freeing = kni_fifo_get(kni->free_q, (void **)pkts, MAX_MBUF_BURST_NUM);
+ if (likely(freeing > 0)) {
+ rte_pktmbuf_free_bulk(kni->pktmbuf_pool, pkts, freeing);
}
}
static void
kni_allocate_mbufs(struct rte_kni *kni)
{
- int i, ret;
- struct rte_mbuf *pkts[MAX_MBUF_BURST_NUM];
- void *phys[MAX_MBUF_BURST_NUM];
+ unsigned count, allocated, put;
+ struct rte_mbuf *pkts[KNI_FIFO_COUNT_MAX];
+ void *phys[KNI_FIFO_COUNT_MAX];
RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pool) !=
offsetof(struct rte_kni_mbuf, pool));
@@ -628,28 +627,23 @@ kni_allocate_mbufs(struct rte_kni *kni)
return;
}
- for (i = 0; i < MAX_MBUF_BURST_NUM; i++) {
- pkts[i] = rte_pktmbuf_alloc(kni->pktmbuf_pool);
- if (unlikely(pkts[i] == NULL)) {
- /* Out of memory */
- RTE_LOG(ERR, KNI, "Out of memory\n");
- break;
- }
- phys[i] = va2pa(pkts[i]);
- }
+ /* Calculate alloc queue free space */
+ count = kni_fifo_free_count(kni->alloc_q);
- /* No pkt mbuf alocated */
- if (i <= 0)
- return;
+ /* Get buffers from mempool */
+ allocated = rte_pktmbuf_alloc_bulk(kni->pktmbuf_pool, pkts, count);
+ for (unsigned i = 0; i < allocated; i++)
+ phys[i] = va2pa(pkts[i]);
- ret = kni_fifo_put(kni->alloc_q, phys, i);
+ /* Put buffers into alloc queue */
+ put = kni_fifo_put(kni->alloc_q, (void **)phys, allocated);
/* Check if any mbufs not put into alloc_q, and then free them */
- if (ret >= 0 && ret < i && ret < MAX_MBUF_BURST_NUM) {
- int j;
-
- for (j = ret; j < i; j++)
+ if (unlikely(put < allocated)) {
+ for (unsigned j = put; j < allocated; j++) {
+ RTE_LOG(ERR, KNI, "Free allocated buffer\n");
rte_pktmbuf_free(pkts[j]);
+ }
}
}
diff --git a/lib/librte_kni/rte_kni_fifo.h b/lib/librte_kni/rte_kni_fifo.h
index 8cb8587..361ddb0 100644
--- a/lib/librte_kni/rte_kni_fifo.h
+++ b/lib/librte_kni/rte_kni_fifo.h
@@ -91,3 +91,21 @@ kni_fifo_get(struct rte_kni_fifo *fifo, void **data, unsigned num)
fifo->read = new_read;
return i;
}
+
+/**
+ * Get the num of elements in the fifo
+ */
+static inline unsigned
+kni_fifo_count(struct rte_kni_fifo *fifo)
+{
+ return (fifo->len + fifo->write - fifo->read) & (fifo->len - 1);
+}
+
+/**
+ * Get the num of available elements in the fifo
+ */
+static inline unsigned
+kni_fifo_free_count(struct rte_kni_fifo *fifo)
+{
+ return (fifo->read - fifo->write - 1) & (fifo->len - 1);
+}
diff --git a/lib/librte_mbuf/rte_mbuf.h b/lib/librte_mbuf/rte_mbuf.h
index 4476d75..707c300 100644
--- a/lib/librte_mbuf/rte_mbuf.h
+++ b/lib/librte_mbuf/rte_mbuf.h
@@ -1261,6 +1261,38 @@ static inline void rte_pktmbuf_free(struct rte_mbuf *m)
}
/**
+ * Free n packets mbuf back into its original mempool.
+ *
+ * Free each mbuf, and all its segments in case of chained buffers. Each
+ * segment is added back into its original mempool.
+ *
+ * @param mp
+ * The packets mempool.
+ * @param mbufs
+ * The packets mbufs array to be freed.
+ * @param n
+ * Number of packets.
+ */
+static inline void rte_pktmbuf_free_bulk(struct rte_mempool *mp,
+ struct rte_mbuf **mbufs, unsigned n)
+{
+ struct rte_mbuf *mbuf, *m_next;
+ unsigned i;
+ for (i = 0; i < n; ++i) {
+ mbuf = mbufs[i];
+ __rte_mbuf_sanity_check(mbuf, 1);
+
+ mbuf = mbuf->next;
+ while (mbuf != NULL) {
+ m_next = mbuf->next;
+ rte_pktmbuf_free_seg(mbuf);
+ mbuf = m_next;
+ }
+ }
+ rte_mempool_put_bulk(mp, (void * const *)mbufs, n);
+}
+
+/**
* Creates a "clone" of the given packet mbuf.
*
* Walks through all segments of the given packet mbuf, and for each of them:
diff --git a/lib/librte_mempool/rte_mempool.h b/lib/librte_mempool/rte_mempool.h
index d315d42..e612a0a 100644
--- a/lib/librte_mempool/rte_mempool.h
+++ b/lib/librte_mempool/rte_mempool.h
@@ -1497,6 +1497,12 @@ rte_mempool_get(struct rte_mempool *mp, void **obj_p)
return rte_mempool_get_bulk(mp, obj_p, 1);
}
+static inline int __attribute__((always_inline))
+rte_mempool_get_n(struct rte_mempool *mp, void **obj_p, int n)
+{
+ return rte_mempool_get_bulk(mp, obj_p, n);
+}
+
/**
* Return the number of entries in the mempool.
*
--
2.7.4