* [dpdk-dev] [PATCH v2 0/2] net/i40e: improve free mbuf for Tx
@ 2021-06-30 2:04 Feifei Wang
2021-06-30 2:04 ` [dpdk-dev] [PATCH v2 1/2] net/i40e: improve performance for scalar Tx Feifei Wang
2021-06-30 2:04 ` [dpdk-dev] [PATCH v2 2/2] net/i40e: improve performance for vector Tx Feifei Wang
0 siblings, 2 replies; 5+ messages in thread
From: Feifei Wang @ 2021-06-30 2:04 UTC (permalink / raw)
Cc: dev, nd, Feifei Wang
For i40e Tx path, use bulk free of the buffers when mbuf fast free
mode is enabled. This can efficiently improve the performance.
v2:
1. fix bug when tx_rs_thres > RTE_I40E_TX_MAX_FREE_BUF_SZ (Beilei)
Feifei Wang (2):
net/i40e: improve performance for scalar Tx
net/i40e: improve performance for vector Tx
drivers/net/i40e/i40e_rxtx.c | 26 +++++++++++++++++++++----
drivers/net/i40e/i40e_rxtx_vec_common.h | 11 +++++++++++
2 files changed, 33 insertions(+), 4 deletions(-)
--
2.25.1
^ permalink raw reply [flat|nested] 5+ messages in thread
* [dpdk-dev] [PATCH v2 1/2] net/i40e: improve performance for scalar Tx
2021-06-30 2:04 [dpdk-dev] [PATCH v2 0/2] net/i40e: improve free mbuf for Tx Feifei Wang
@ 2021-06-30 2:04 ` Feifei Wang
2021-06-30 3:42 ` Xing, Beilei
2021-06-30 2:04 ` [dpdk-dev] [PATCH v2 2/2] net/i40e: improve performance for vector Tx Feifei Wang
1 sibling, 1 reply; 5+ messages in thread
From: Feifei Wang @ 2021-06-30 2:04 UTC (permalink / raw)
To: Beilei Xing; +Cc: dev, nd, Feifei Wang, Ruifeng Wang
For i40e scalar Tx path, if implement FAST_FREE_MBUF mode, it means
per-queue all mbufs come from the same mempool and have refcnt = 1.
Thus we can use bulk free of the buffers when mbuf fast free mode is
enabled.
Following are the test results with this patch:
MRR L3FWD Test:
two ports & bi-directional flows & one core
RX API: i40e_recv_pkts_bulk_alloc
TX API: i40e_xmit_pkts_simple
ring_descs_size = 1024;
Ring_I40E_TX_MAX_FREE_SZ = 64;
tx_rs_thresh = I40E_DEFAULT_TX_RSBIT_THRESH = 32;
tx_free_thresh = I40E_DEFAULT_TX_FREE_THRESH = 32;
For scalar path in arm platform with default 'tx_rs_thresh':
In n1sdp, performance is improved by 7.9%;
In thunderx2, performance is improved by 7.6%.
For scalar path in x86 platform with default 'tx_rs_thresh':
performance is improved by 4.7%.
Suggested-by: Ruifeng Wang <ruifeng.wang@arm.com>
Signed-off-by: Feifei Wang <feifei.wang2@arm.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
---
drivers/net/i40e/i40e_rxtx.c | 26 ++++++++++++++++++++++----
1 file changed, 22 insertions(+), 4 deletions(-)
diff --git a/drivers/net/i40e/i40e_rxtx.c b/drivers/net/i40e/i40e_rxtx.c
index 6c58decece..8c72391cde 100644
--- a/drivers/net/i40e/i40e_rxtx.c
+++ b/drivers/net/i40e/i40e_rxtx.c
@@ -1294,7 +1294,11 @@ static __rte_always_inline int
i40e_tx_free_bufs(struct i40e_tx_queue *txq)
{
struct i40e_tx_entry *txep;
- uint16_t i;
+ int n = txq->tx_rs_thresh;
+ uint16_t i = 0, j = 0;
+ struct rte_mbuf *free[RTE_I40E_TX_MAX_FREE_BUF_SZ];
+ const int32_t k = RTE_ALIGN_FLOOR(n, RTE_I40E_TX_MAX_FREE_BUF_SZ);
+ const int32_t m = n % RTE_I40E_TX_MAX_FREE_BUF_SZ;
if ((txq->tx_ring[txq->tx_next_dd].cmd_type_offset_bsz &
rte_cpu_to_le_64(I40E_TXD_QW1_DTYPE_MASK)) !=
@@ -1307,9 +1311,23 @@ i40e_tx_free_bufs(struct i40e_tx_queue *txq)
rte_prefetch0((txep + i)->mbuf);
if (txq->offloads & DEV_TX_OFFLOAD_MBUF_FAST_FREE) {
- for (i = 0; i < txq->tx_rs_thresh; ++i, ++txep) {
- rte_mempool_put(txep->mbuf->pool, txep->mbuf);
- txep->mbuf = NULL;
+ if (k) {
+ for (j = 0; j != k; j += RTE_I40E_TX_MAX_FREE_BUF_SZ) {
+ for (i = 0; i < RTE_I40E_TX_MAX_FREE_BUF_SZ; ++i, ++txep) {
+ free[i] = txep->mbuf;
+ txep->mbuf = NULL;
+ }
+ rte_mempool_put_bulk(free[0]->pool, (void **)free,
+ RTE_I40E_TX_MAX_FREE_BUF_SZ);
+ }
+ }
+
+ if (m) {
+ for (i = 0; i < m; ++i, ++txep) {
+ free[i] = txep->mbuf;
+ txep->mbuf = NULL;
+ }
+ rte_mempool_put_bulk(free[0]->pool, (void **)free, m);
}
} else {
for (i = 0; i < txq->tx_rs_thresh; ++i, ++txep) {
--
2.25.1
^ permalink raw reply [flat|nested] 5+ messages in thread
* [dpdk-dev] [PATCH v2 2/2] net/i40e: improve performance for vector Tx
2021-06-30 2:04 [dpdk-dev] [PATCH v2 0/2] net/i40e: improve free mbuf for Tx Feifei Wang
2021-06-30 2:04 ` [dpdk-dev] [PATCH v2 1/2] net/i40e: improve performance for scalar Tx Feifei Wang
@ 2021-06-30 2:04 ` Feifei Wang
1 sibling, 0 replies; 5+ messages in thread
From: Feifei Wang @ 2021-06-30 2:04 UTC (permalink / raw)
To: Beilei Xing; +Cc: dev, nd, Feifei Wang, Ruifeng Wang
For i40e vector Tx path, if tx_offload is set as FAST_FREE_MBUF mode,
no mbuf fast free operations are executed. To fix this, add mbuf fast
free mode for vector Tx path.
Furthermore, for i40e vector Tx path, if implement FAST_FREE_MBUF mode,
it means per-queue all mbufs come from the same mempool and have
refcnt = 1. Thus we can use bulk free of the buffers when mbuf fast free
mode is enabled.
For vector path in arm platform:
In n1sdp, performance is improved by 18.4%;
In thunderx2, performance is improved by 23%.
For vector path in x86 platform:
No performance changes.
Suggested-by: Ruifeng Wang <ruifeng.wang@arm.com>
Signed-off-by: Feifei Wang <feifei.wang2@arm.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
---
drivers/net/i40e/i40e_rxtx_vec_common.h | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/drivers/net/i40e/i40e_rxtx_vec_common.h b/drivers/net/i40e/i40e_rxtx_vec_common.h
index 16fcf0aec6..f52ed98d62 100644
--- a/drivers/net/i40e/i40e_rxtx_vec_common.h
+++ b/drivers/net/i40e/i40e_rxtx_vec_common.h
@@ -99,6 +99,16 @@ i40e_tx_free_bufs(struct i40e_tx_queue *txq)
* tx_next_dd - (tx_rs_thresh-1)
*/
txep = &txq->sw_ring[txq->tx_next_dd - (n - 1)];
+
+ if (txq->offloads & DEV_TX_OFFLOAD_MBUF_FAST_FREE) {
+ for (i = 0; i < n; i++) {
+ free[i] = txep[i].mbuf;
+ txep[i].mbuf = NULL;
+ }
+ rte_mempool_put_bulk(free[0]->pool, (void **)free, n);
+ goto done;
+ }
+
m = rte_pktmbuf_prefree_seg(txep[0].mbuf);
if (likely(m != NULL)) {
free[0] = m;
@@ -126,6 +136,7 @@ i40e_tx_free_bufs(struct i40e_tx_queue *txq)
}
}
+done:
/* buffers were freed, update counters */
txq->nb_tx_free = (uint16_t)(txq->nb_tx_free + txq->tx_rs_thresh);
txq->tx_next_dd = (uint16_t)(txq->tx_next_dd + txq->tx_rs_thresh);
--
2.25.1
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [dpdk-dev] [PATCH v2 1/2] net/i40e: improve performance for scalar Tx
2021-06-30 2:04 ` [dpdk-dev] [PATCH v2 1/2] net/i40e: improve performance for scalar Tx Feifei Wang
@ 2021-06-30 3:42 ` Xing, Beilei
2021-06-30 5:30 ` [dpdk-dev] 回复: " Feifei Wang
0 siblings, 1 reply; 5+ messages in thread
From: Xing, Beilei @ 2021-06-30 3:42 UTC (permalink / raw)
To: Feifei Wang; +Cc: dev, nd, Ruifeng Wang
> -----Original Message-----
> From: Feifei Wang <feifei.wang2@arm.com>
> Sent: Wednesday, June 30, 2021 10:04 AM
> To: Xing, Beilei <beilei.xing@intel.com>
> Cc: dev@dpdk.org; nd@arm.com; Feifei Wang <feifei.wang2@arm.com>;
> Ruifeng Wang <ruifeng.wang@arm.com>
> Subject: [PATCH v2 1/2] net/i40e: improve performance for scalar Tx
>
> For i40e scalar Tx path, if implement FAST_FREE_MBUF mode, it means per-
> queue all mbufs come from the same mempool and have refcnt = 1.
>
> Thus we can use bulk free of the buffers when mbuf fast free mode is
> enabled.
>
> Following are the test results with this patch:
>
> MRR L3FWD Test:
> two ports & bi-directional flows & one core RX API:
> i40e_recv_pkts_bulk_alloc TX API: i40e_xmit_pkts_simple ring_descs_size =
> 1024; Ring_I40E_TX_MAX_FREE_SZ = 64; tx_rs_thresh =
> I40E_DEFAULT_TX_RSBIT_THRESH = 32; tx_free_thresh =
> I40E_DEFAULT_TX_FREE_THRESH = 32;
>
> For scalar path in arm platform with default 'tx_rs_thresh':
> In n1sdp, performance is improved by 7.9%; In thunderx2, performance is
> improved by 7.6%.
>
> For scalar path in x86 platform with default 'tx_rs_thresh':
> performance is improved by 4.7%.
>
> Suggested-by: Ruifeng Wang <ruifeng.wang@arm.com>
> Signed-off-by: Feifei Wang <feifei.wang2@arm.com>
> Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
> ---
> drivers/net/i40e/i40e_rxtx.c | 26 ++++++++++++++++++++++----
> 1 file changed, 22 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/net/i40e/i40e_rxtx.c b/drivers/net/i40e/i40e_rxtx.c index
> 6c58decece..8c72391cde 100644
> --- a/drivers/net/i40e/i40e_rxtx.c
> +++ b/drivers/net/i40e/i40e_rxtx.c
> @@ -1294,7 +1294,11 @@ static __rte_always_inline int
> i40e_tx_free_bufs(struct i40e_tx_queue *txq) {
> struct i40e_tx_entry *txep;
> - uint16_t i;
> + int n = txq->tx_rs_thresh;
Thanks for the patch, just little comment, can we use 'tx_rs_thresh' to replace 'n' to make it more readable?
> + uint16_t i = 0, j = 0;
> + struct rte_mbuf *free[RTE_I40E_TX_MAX_FREE_BUF_SZ];
> + const int32_t k = RTE_ALIGN_FLOOR(n,
> RTE_I40E_TX_MAX_FREE_BUF_SZ);
> + const int32_t m = n % RTE_I40E_TX_MAX_FREE_BUF_SZ;
>
> if ((txq->tx_ring[txq->tx_next_dd].cmd_type_offset_bsz &
> rte_cpu_to_le_64(I40E_TXD_QW1_DTYPE_MASK)) !=
> @@ -1307,9 +1311,23 @@ i40e_tx_free_bufs(struct i40e_tx_queue *txq)
> rte_prefetch0((txep + i)->mbuf);
>
> if (txq->offloads & DEV_TX_OFFLOAD_MBUF_FAST_FREE) {
> - for (i = 0; i < txq->tx_rs_thresh; ++i, ++txep) {
> - rte_mempool_put(txep->mbuf->pool, txep->mbuf);
> - txep->mbuf = NULL;
> + if (k) {
> + for (j = 0; j != k; j += RTE_I40E_TX_MAX_FREE_BUF_SZ)
> {
> + for (i = 0; i < RTE_I40E_TX_MAX_FREE_BUF_SZ;
> ++i, ++txep) {
> + free[i] = txep->mbuf;
> + txep->mbuf = NULL;
> + }
> + rte_mempool_put_bulk(free[0]->pool, (void
> **)free,
> +
> RTE_I40E_TX_MAX_FREE_BUF_SZ);
> + }
> + }
> +
> + if (m) {
> + for (i = 0; i < m; ++i, ++txep) {
> + free[i] = txep->mbuf;
> + txep->mbuf = NULL;
> + }
> + rte_mempool_put_bulk(free[0]->pool, (void **)free,
> m);
> }
> } else {
> for (i = 0; i < txq->tx_rs_thresh; ++i, ++txep) {
> --
> 2.25.1
^ permalink raw reply [flat|nested] 5+ messages in thread
* [dpdk-dev] 回复: [PATCH v2 1/2] net/i40e: improve performance for scalar Tx
2021-06-30 3:42 ` Xing, Beilei
@ 2021-06-30 5:30 ` Feifei Wang
0 siblings, 0 replies; 5+ messages in thread
From: Feifei Wang @ 2021-06-30 5:30 UTC (permalink / raw)
To: Xing, Beilei; +Cc: dev, nd, Ruifeng Wang, nd
> -----邮件原件-----
> 发件人: Xing, Beilei <beilei.xing@intel.com>
> 发送时间: 2021年6月30日 11:43
> 收件人: Feifei Wang <Feifei.Wang2@arm.com>
> 抄送: dev@dpdk.org; nd <nd@arm.com>; Ruifeng Wang
> <Ruifeng.Wang@arm.com>
> 主题: RE: [PATCH v2 1/2] net/i40e: improve performance for scalar Tx
>
>
>
> > -----Original Message-----
> > From: Feifei Wang <feifei.wang2@arm.com>
> > Sent: Wednesday, June 30, 2021 10:04 AM
> > To: Xing, Beilei <beilei.xing@intel.com>
> > Cc: dev@dpdk.org; nd@arm.com; Feifei Wang <feifei.wang2@arm.com>;
> > Ruifeng Wang <ruifeng.wang@arm.com>
> > Subject: [PATCH v2 1/2] net/i40e: improve performance for scalar Tx
> >
> > For i40e scalar Tx path, if implement FAST_FREE_MBUF mode, it means
> > per- queue all mbufs come from the same mempool and have refcnt = 1.
> >
> > Thus we can use bulk free of the buffers when mbuf fast free mode is
> > enabled.
> >
> > Following are the test results with this patch:
> >
> > MRR L3FWD Test:
> > two ports & bi-directional flows & one core RX API:
> > i40e_recv_pkts_bulk_alloc TX API: i40e_xmit_pkts_simple
> > ring_descs_size = 1024; Ring_I40E_TX_MAX_FREE_SZ = 64; tx_rs_thresh =
> > I40E_DEFAULT_TX_RSBIT_THRESH = 32; tx_free_thresh =
> > I40E_DEFAULT_TX_FREE_THRESH = 32;
> >
> > For scalar path in arm platform with default 'tx_rs_thresh':
> > In n1sdp, performance is improved by 7.9%; In thunderx2, performance
> > is improved by 7.6%.
> >
> > For scalar path in x86 platform with default 'tx_rs_thresh':
> > performance is improved by 4.7%.
> >
> > Suggested-by: Ruifeng Wang <ruifeng.wang@arm.com>
> > Signed-off-by: Feifei Wang <feifei.wang2@arm.com>
> > Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
> > ---
> > drivers/net/i40e/i40e_rxtx.c | 26 ++++++++++++++++++++++----
> > 1 file changed, 22 insertions(+), 4 deletions(-)
> >
> > diff --git a/drivers/net/i40e/i40e_rxtx.c
> > b/drivers/net/i40e/i40e_rxtx.c index 6c58decece..8c72391cde 100644
> > --- a/drivers/net/i40e/i40e_rxtx.c
> > +++ b/drivers/net/i40e/i40e_rxtx.c
> > @@ -1294,7 +1294,11 @@ static __rte_always_inline int
> > i40e_tx_free_bufs(struct i40e_tx_queue *txq) {
> > struct i40e_tx_entry *txep;
> > - uint16_t i;
> > + int n = txq->tx_rs_thresh;
>
> Thanks for the patch, just little comment, can we use 'tx_rs_thresh' to
> replace 'n' to make it more readable?
Good comments for this, I will update it, thanks.
>
> > + uint16_t i = 0, j = 0;
> > + struct rte_mbuf *free[RTE_I40E_TX_MAX_FREE_BUF_SZ];
> > + const int32_t k = RTE_ALIGN_FLOOR(n,
> > RTE_I40E_TX_MAX_FREE_BUF_SZ);
> > + const int32_t m = n % RTE_I40E_TX_MAX_FREE_BUF_SZ;
> >
> > if ((txq->tx_ring[txq->tx_next_dd].cmd_type_offset_bsz &
> >
> rte_cpu_to_le_64(I40E_TXD_QW1_DTYPE_MASK)) != @@ -1307,9
> +1311,23
> > @@ i40e_tx_free_bufs(struct i40e_tx_queue *txq)
> > rte_prefetch0((txep + i)->mbuf);
> >
> > if (txq->offloads & DEV_TX_OFFLOAD_MBUF_FAST_FREE) {
> > - for (i = 0; i < txq->tx_rs_thresh; ++i, ++txep) {
> > - rte_mempool_put(txep->mbuf->pool, txep->mbuf);
> > - txep->mbuf = NULL;
> > + if (k) {
> > + for (j = 0; j != k; j +=
> RTE_I40E_TX_MAX_FREE_BUF_SZ)
> > {
> > + for (i = 0; i <
> RTE_I40E_TX_MAX_FREE_BUF_SZ;
> > ++i, ++txep) {
> > + free[i] = txep->mbuf;
> > + txep->mbuf = NULL;
> > + }
> > + rte_mempool_put_bulk(free[0]->pool, (void
> > **)free,
> > +
> > RTE_I40E_TX_MAX_FREE_BUF_SZ);
> > + }
> > + }
> > +
> > + if (m) {
> > + for (i = 0; i < m; ++i, ++txep) {
> > + free[i] = txep->mbuf;
> > + txep->mbuf = NULL;
> > + }
> > + rte_mempool_put_bulk(free[0]->pool, (void **)free,
> > m);
> > }
> > } else {
> > for (i = 0; i < txq->tx_rs_thresh; ++i, ++txep) {
> > --
> > 2.25.1
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2021-06-30 5:30 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-06-30 2:04 [dpdk-dev] [PATCH v2 0/2] net/i40e: improve free mbuf for Tx Feifei Wang
2021-06-30 2:04 ` [dpdk-dev] [PATCH v2 1/2] net/i40e: improve performance for scalar Tx Feifei Wang
2021-06-30 3:42 ` Xing, Beilei
2021-06-30 5:30 ` [dpdk-dev] 回复: " Feifei Wang
2021-06-30 2:04 ` [dpdk-dev] [PATCH v2 2/2] net/i40e: improve performance for vector Tx Feifei Wang
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).