[PATCH] kni: fix possible alloc_q starvation when mbufs are exhausted

[PATCH] kni: fix possible alloc_q starvation when mbufs are exhausted

[Yangchao Zhou]

In some scenarios, mbufs returned by rte_kni_rx_burst are not freed
immediately. So kni_allocate_mbufs may be failed, but we don't know.

Even worse, when alloc_q is completely exhausted, kni_net_tx in
rte_kni.ko will drop all tx packets. kni_allocate_mbufs is never
called again, even if the mbufs are eventually freed.

In this patch, we always try to allocate mbufs for alloc_q.

Don't worry about alloc_q being allocated too many mbufs, in fact,
the old logic will gradually fill up alloc_q.
Also, the cost of more calls to kni_allocate_mbufs should be acceptable.

Fixes: 3e12a98fe397 ("kni: optimize Rx burst")
Cc: Hemant@freescale.com
Cc: stable@dpdk.org

Signed-off-by: Yangchao Zhou <zhouyates@gmail.com>
---
 lib/kni/rte_kni.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/lib/kni/rte_kni.c b/lib/kni/rte_kni.c
index 8ab6c47153..265c5262f7 100644
--- a/lib/kni/rte_kni.c
+++ b/lib/kni/rte_kni.c
@@ -634,9 +634,9 @@ rte_kni_rx_burst(struct rte_kni *kni, struct rte_mbuf **mbufs, unsigned int num)
 {
 	unsigned int ret = kni_fifo_get(kni->tx_q, (void **)mbufs, num);
 
-	/* If buffers removed, allocate mbufs and then put them into alloc_q */
-	if (ret)
-		kni_allocate_mbufs(kni);
+	/* Always try to allocate mbufs for alloc_q to avoid starvation when
+	 * kni->pktmbuf_pool is exhausted. */
+	kni_allocate_mbufs(kni);
 
 	return ret;
 }
-- 
2.25.1

[PATCH v2] kni: fix possible alloc_q starvation when mbufs are exhausted

[Yangchao Zhou]

In some scenarios, mbufs returned by rte_kni_rx_burst are not freed
immediately. So kni_allocate_mbufs may be failed, but we don't know.

Even worse, when alloc_q is completely exhausted, kni_net_tx in
rte_kni.ko will drop all tx packets. kni_allocate_mbufs is never
called again, even if the mbufs are eventually freed.

In this patch, we always try to allocate mbufs for alloc_q.

Don't worry about alloc_q being allocated too many mbufs, in fact,
the old logic will gradually fill up alloc_q.
Also, the cost of more calls to kni_allocate_mbufs should be acceptable.

Fixes: 3e12a98fe397 ("kni: optimize Rx burst")
Cc: Hemant@freescale.com
Cc: stable@dpdk.org

Signed-off-by: Yangchao Zhou <zhouyates@gmail.com>
---
 lib/kni/rte_kni.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/lib/kni/rte_kni.c b/lib/kni/rte_kni.c
index 8ab6c47153..ea9a507de3 100644
--- a/lib/kni/rte_kni.c
+++ b/lib/kni/rte_kni.c
@@ -634,9 +634,11 @@ rte_kni_rx_burst(struct rte_kni *kni, struct rte_mbuf **mbufs, unsigned int num)
 {
 	unsigned int ret = kni_fifo_get(kni->tx_q, (void **)mbufs, num);
 
-	/* If buffers removed, allocate mbufs and then put them into alloc_q */
-	if (ret)
-		kni_allocate_mbufs(kni);
+	/*
+	 * Always try to allocate mbufs for alloc_q to avoid starvation when
+	 * kni->pktmbuf_pool is exhausted.
+	 */
+	kni_allocate_mbufs(kni);
 
 	return ret;
 }
-- 
2.25.1

Re: [PATCH v2] kni: fix possible alloc_q starvation when mbufs are exhausted

[Stephen Hemminger]

On Wed,  9 Nov 2022 14:04:34 +0800
Yangchao Zhou <zhouyates@gmail.com> wrote:

> In some scenarios, mbufs returned by rte_kni_rx_burst are not freed
> immediately. So kni_allocate_mbufs may be failed, but we don't know.
> 
> Even worse, when alloc_q is completely exhausted, kni_net_tx in
> rte_kni.ko will drop all tx packets. kni_allocate_mbufs is never
> called again, even if the mbufs are eventually freed.
> 
> In this patch, we always try to allocate mbufs for alloc_q.
> 
> Don't worry about alloc_q being allocated too many mbufs, in fact,
> the old logic will gradually fill up alloc_q.
> Also, the cost of more calls to kni_allocate_mbufs should be acceptable.
> 
> Fixes: 3e12a98fe397 ("kni: optimize Rx burst")
> Cc: Hemant@freescale.com
> Cc: stable@dpdk.org
> 
> Signed-off-by: Yangchao Zhou <zhouyates@gmail.com>

Since fifo_get returning 0 (no buffers) is very common would this
change impact performance.

If the problem is pool draining might be better to make the pool
bigger.

Re: [PATCH v2] kni: fix possible alloc_q starvation when mbufs are exhausted

[Matt]

[-- Attachment #1: Type: text/plain, Size: 1844 bytes --]

On Thu, Nov 10, 2022 at 12:39 AM Stephen Hemminger <
stephen@networkplumber.org> wrote:

> On Wed,  9 Nov 2022 14:04:34 +0800
> Yangchao Zhou <zhouyates@gmail.com> wrote:
>
> > In some scenarios, mbufs returned by rte_kni_rx_burst are not freed
> > immediately. So kni_allocate_mbufs may be failed, but we don't know.
> >
> > Even worse, when alloc_q is completely exhausted, kni_net_tx in
> > rte_kni.ko will drop all tx packets. kni_allocate_mbufs is never
> > called again, even if the mbufs are eventually freed.
> >
> > In this patch, we always try to allocate mbufs for alloc_q.
> >
> > Don't worry about alloc_q being allocated too many mbufs, in fact,
> > the old logic will gradually fill up alloc_q.
> > Also, the cost of more calls to kni_allocate_mbufs should be acceptable.
> >
> > Fixes: 3e12a98fe397 ("kni: optimize Rx burst")
> > Cc: Hemant@freescale.com
> > Cc: stable@dpdk.org
> >
> > Signed-off-by: Yangchao Zhou <zhouyates@gmail.com>
>
> Since fifo_get returning 0 (no buffers) is very common would this
> change impact performance.
>
It does add a little cost, but there is no extra mbuf allocation
and deallocation.

>
> If the problem is pool draining might be better to make the pool
> bigger.
>
Yes, using a larger pool can avoid this problem. But this may lead to
resource wastage and full resource calculation is a challenge for developers
as it involves to mempool caching mechanism, IP fragment cache,
ARP cache, NIC txq, other transit queue, etc.

The mbuf allocation failure may also occur on many NIC drivers,
but if the mbuf allocation fails, the mbuf is not taken out so that
it can be recovered after a retry later.
KNI currently does not have such a takedown and recovery mechanism.
It is also possible to consider implementing something similar to
the NIC driver, but with more changes and other overheads.

[-- Attachment #2: Type: text/html, Size: 2795 bytes --]

Re: [PATCH v2] kni: fix possible alloc_q starvation when mbufs are exhausted

[Ferruh Yigit]

On 11/11/2022 9:12 AM, Matt wrote:
> 
> 
> On Thu, Nov 10, 2022 at 12:39 AM Stephen Hemminger
> <stephen@networkplumber.org <mailto:stephen@networkplumber.org>> wrote:
> 
>     On Wed,  9 Nov 2022 14:04:34 +0800
>     Yangchao Zhou <zhouyates@gmail.com <mailto:zhouyates@gmail.com>> wrote:
> 
>     > In some scenarios, mbufs returned by rte_kni_rx_burst are not freed
>     > immediately. So kni_allocate_mbufs may be failed, but we don't know.
>     >
>     > Even worse, when alloc_q is completely exhausted, kni_net_tx in
>     > rte_kni.ko will drop all tx packets. kni_allocate_mbufs is never
>     > called again, even if the mbufs are eventually freed.
>     >
>     > In this patch, we always try to allocate mbufs for alloc_q.
>     >
>     > Don't worry about alloc_q being allocated too many mbufs, in fact,
>     > the old logic will gradually fill up alloc_q.
>     > Also, the cost of more calls to kni_allocate_mbufs should be
>     acceptable.
>     >
>     > Fixes: 3e12a98fe397 ("kni: optimize Rx burst")
>     > Cc: Hemant@freescale.com <mailto:Hemant@freescale.com>
>     > Cc: stable@dpdk.org <mailto:stable@dpdk.org>
>     >
>     > Signed-off-by: Yangchao Zhou <zhouyates@gmail.com
>     <mailto:zhouyates@gmail.com>>
> 
>     Since fifo_get returning 0 (no buffers) is very common would this
>     change impact performance.
> 
> It does add a little cost, but there is no extra mbuf allocation
> and deallocation.
> 

It is common that 'rte_kni_rx_burst()' called in continuous loop, I
expect to try to fill alloc_q in each call impacts performance, even no
mbuf allocated at all.

> 
>     If the problem is pool draining might be better to make the pool
>     bigger.
> 
> Yes, using a larger pool can avoid this problem. But this may lead to
> resource wastage and full resource calculation is a challenge for developers
> as it involves to mempool caching mechanism, IP fragment cache,
> ARP cache, NIC txq, other transit queue, etc.
> 
> The mbuf allocation failure may also occur on many NIC drivers,
> but if the mbuf allocation fails, the mbuf is not taken out so that
> it can be recovered after a retry later.
> KNI currently does not have such a takedown and recovery mechanism.
> It is also possible to consider implementing something similar to
> the NIC driver, but with more changes and other overheads.

I agree this can cause starvation and recovery mechanism is missing,

Following may be an option, but still concerned about performance impact
and not sure about upstreaming it. Other solution can be to use larger
pool as Stephen suggested.

rte_kni_rx_burst()
	ret = kni_fifo_get()

	if (ret & kni_fifo_count(alloc_q) == 0)
		kni_allocate_mbufs()
	

[PATCH v3] kni: fix possible alloc_q starvation when mbufs are exhausted

[Yangchao Zhou]

In some scenarios, mbufs returned by rte_kni_rx_burst are not freed
immediately. So kni_allocate_mbufs may be failed, but we don't know.

Even worse, when alloc_q is completely exhausted, kni_net_tx in
rte_kni.ko will drop all tx packets. kni_allocate_mbufs is never
called again, even if the mbufs are eventually freed.

In this patch, we try to allocate mbufs for alloc_q when it is empty.

According to historical experience, the performance bottleneck of KNI
is offen the usleep_range of kni thread in rte_kni.ko.
The check of kni_fifo_count is trivial and the cost should be acceptable.

Fixes: 3e12a98fe397 ("kni: optimize Rx burst")
Cc: stable@dpdk.org

Signed-off-by: Yangchao Zhou <zhouyates@gmail.com>
---
 lib/kni/rte_kni.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/lib/kni/rte_kni.c b/lib/kni/rte_kni.c
index 8ab6c47153..bfa6a001ff 100644
--- a/lib/kni/rte_kni.c
+++ b/lib/kni/rte_kni.c
@@ -634,8 +634,8 @@ rte_kni_rx_burst(struct rte_kni *kni, struct rte_mbuf **mbufs, unsigned int num)
 {
 	unsigned int ret = kni_fifo_get(kni->tx_q, (void **)mbufs, num);
 
-	/* If buffers removed, allocate mbufs and then put them into alloc_q */
-	if (ret)
+	/* If buffers removed or alloc_q is empty, allocate mbufs and then put them into alloc_q */
+	if (ret || (kni_fifo_count(kni->alloc_q) == 0))
 		kni_allocate_mbufs(kni);
 
 	return ret;
-- 
2.25.1

Re: [PATCH v3] kni: fix possible alloc_q starvation when mbufs are exhausted

[Ferruh Yigit]

On 12/30/2022 4:23 AM, Yangchao Zhou wrote:
> In some scenarios, mbufs returned by rte_kni_rx_burst are not freed
> immediately. So kni_allocate_mbufs may be failed, but we don't know.
> 
> Even worse, when alloc_q is completely exhausted, kni_net_tx in
> rte_kni.ko will drop all tx packets. kni_allocate_mbufs is never
> called again, even if the mbufs are eventually freed.
> 
> In this patch, we try to allocate mbufs for alloc_q when it is empty.
> 
> According to historical experience, the performance bottleneck of KNI
> is offen the usleep_range of kni thread in rte_kni.ko.
> The check of kni_fifo_count is trivial and the cost should be acceptable.
> 

Hi Yangchao,

Are you observing any performance impact with this change in you use case?


> Fixes: 3e12a98fe397 ("kni: optimize Rx burst")
> Cc: stable@dpdk.org
> 
> Signed-off-by: Yangchao Zhou <zhouyates@gmail.com>
> ---
>  lib/kni/rte_kni.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/lib/kni/rte_kni.c b/lib/kni/rte_kni.c
> index 8ab6c47153..bfa6a001ff 100644
> --- a/lib/kni/rte_kni.c
> +++ b/lib/kni/rte_kni.c
> @@ -634,8 +634,8 @@ rte_kni_rx_burst(struct rte_kni *kni, struct rte_mbuf **mbufs, unsigned int num)
>  {
>  	unsigned int ret = kni_fifo_get(kni->tx_q, (void **)mbufs, num);
>  
> -	/* If buffers removed, allocate mbufs and then put them into alloc_q */
> -	if (ret)
> +	/* If buffers removed or alloc_q is empty, allocate mbufs and then put them into alloc_q */
> +	if (ret || (kni_fifo_count(kni->alloc_q) == 0))
>  		kni_allocate_mbufs(kni);
>  
>  	return ret;

Re: [PATCH v3] kni: fix possible alloc_q starvation when mbufs are exhausted

[Matt]

[-- Attachment #1: Type: text/plain, Size: 2095 bytes --]

Hi Ferruh,

In my case, the traffic is not large, so I can't see the impact.
I also tested under high load(>2Mpps with 2 DPDK cores and 2 kernel threads)
and found no significant difference in performance either.
I think the reason should be that it will not
run to 'kni_fifo_count(kni->alloc_q) == 0' under high load.

On Tue, Jan 3, 2023 at 8:47 PM Ferruh Yigit <ferruh.yigit@amd.com> wrote:

> On 12/30/2022 4:23 AM, Yangchao Zhou wrote:
> > In some scenarios, mbufs returned by rte_kni_rx_burst are not freed
> > immediately. So kni_allocate_mbufs may be failed, but we don't know.
> >
> > Even worse, when alloc_q is completely exhausted, kni_net_tx in
> > rte_kni.ko will drop all tx packets. kni_allocate_mbufs is never
> > called again, even if the mbufs are eventually freed.
> >
> > In this patch, we try to allocate mbufs for alloc_q when it is empty.
> >
> > According to historical experience, the performance bottleneck of KNI
> > is offen the usleep_range of kni thread in rte_kni.ko.
> > The check of kni_fifo_count is trivial and the cost should be acceptable.
> >
>
> Hi Yangchao,
>
> Are you observing any performance impact with this change in you use case?
>
>
> > Fixes: 3e12a98fe397 ("kni: optimize Rx burst")
> > Cc: stable@dpdk.org
> >
> > Signed-off-by: Yangchao Zhou <zhouyates@gmail.com>
> > ---
> >  lib/kni/rte_kni.c | 4 ++--
> >  1 file changed, 2 insertions(+), 2 deletions(-)
> >
> > diff --git a/lib/kni/rte_kni.c b/lib/kni/rte_kni.c
> > index 8ab6c47153..bfa6a001ff 100644
> > --- a/lib/kni/rte_kni.c
> > +++ b/lib/kni/rte_kni.c
> > @@ -634,8 +634,8 @@ rte_kni_rx_burst(struct rte_kni *kni, struct
> rte_mbuf **mbufs, unsigned int num)
> >  {
> >       unsigned int ret = kni_fifo_get(kni->tx_q, (void **)mbufs, num);
> >
> > -     /* If buffers removed, allocate mbufs and then put them into
> alloc_q */
> > -     if (ret)
> > +     /* If buffers removed or alloc_q is empty, allocate mbufs and then
> put them into alloc_q */
> > +     if (ret || (kni_fifo_count(kni->alloc_q) == 0))
> >               kni_allocate_mbufs(kni);
> >
> >       return ret;
>
>

[-- Attachment #2: Type: text/html, Size: 2851 bytes --]

Re: [PATCH v3] kni: fix possible alloc_q starvation when mbufs are exhausted

[Ferruh Yigit]

On 1/4/2023 11:57 AM, Matt wrote:
> Hi Ferruh,
> 
> In my case, the traffic is not large, so I can't see the impact.
> I also tested under high load(>2Mpps with 2 DPDK cores and 2 kernel threads)
> and found no significant difference in performance either.
> I think the reason should be that it will not
> run to 'kni_fifo_count(kni->alloc_q) == 0' under high load.
> 

I agree, additional check most likely hit on the low bandwidth,
thanks for checking for performance impact.

> On Tue, Jan 3, 2023 at 8:47 PM Ferruh Yigit <ferruh.yigit@amd.com
> <mailto:ferruh.yigit@amd.com>> wrote:
> 
>     On 12/30/2022 4:23 AM, Yangchao Zhou wrote:
>     > In some scenarios, mbufs returned by rte_kni_rx_burst are not freed
>     > immediately. So kni_allocate_mbufs may be failed, but we don't know.
>     >
>     > Even worse, when alloc_q is completely exhausted, kni_net_tx in
>     > rte_kni.ko will drop all tx packets. kni_allocate_mbufs is never
>     > called again, even if the mbufs are eventually freed.
>     >
>     > In this patch, we try to allocate mbufs for alloc_q when it is empty.
>     >
>     > According to historical experience, the performance bottleneck of KNI
>     > is offen the usleep_range of kni thread in rte_kni.ko.
>     > The check of kni_fifo_count is trivial and the cost should be
>     acceptable.
>     >
> 
>     Hi Yangchao,
> 
>     Are you observing any performance impact with this change in you use
>     case?
> 
> 
>     > Fixes: 3e12a98fe397 ("kni: optimize Rx burst")
>     > Cc: stable@dpdk.org <mailto:stable@dpdk.org>
>     >
>     > Signed-off-by: Yangchao Zhou <zhouyates@gmail.com
>     <mailto:zhouyates@gmail.com>>

Acked-by: Ferruh Yigit <ferruh.yigit@amd.com>

Re: [PATCH v3] kni: fix possible alloc_q starvation when mbufs are exhausted

[Thomas Monjalon]

04/01/2023 15:34, Ferruh Yigit:
> On 1/4/2023 11:57 AM, Matt wrote:
> > Hi Ferruh,
> > 
> > In my case, the traffic is not large, so I can't see the impact.
> > I also tested under high load(>2Mpps with 2 DPDK cores and 2 kernel threads)
> > and found no significant difference in performance either.
> > I think the reason should be that it will not
> > run to 'kni_fifo_count(kni->alloc_q) == 0' under high load.
> > 
> 
> I agree, additional check most likely hit on the low bandwidth,
> thanks for checking for performance impact.
> 
> > On Tue, Jan 3, 2023 at 8:47 PM Ferruh Yigit <ferruh.yigit@amd.com
> > <mailto:ferruh.yigit@amd.com>> wrote:
> > 
> >     On 12/30/2022 4:23 AM, Yangchao Zhou wrote:
> >     > In some scenarios, mbufs returned by rte_kni_rx_burst are not freed
> >     > immediately. So kni_allocate_mbufs may be failed, but we don't know.
> >     >
> >     > Even worse, when alloc_q is completely exhausted, kni_net_tx in
> >     > rte_kni.ko will drop all tx packets. kni_allocate_mbufs is never
> >     > called again, even if the mbufs are eventually freed.
> >     >
> >     > In this patch, we try to allocate mbufs for alloc_q when it is empty.
> >     >
> >     > According to historical experience, the performance bottleneck of KNI
> >     > is offen the usleep_range of kni thread in rte_kni.ko.
> >     > The check of kni_fifo_count is trivial and the cost should be
> >     acceptable.
> >     >
> > 
> >     Hi Yangchao,
> > 
> >     Are you observing any performance impact with this change in you use
> >     case?
> > 
> > 
> >     > Fixes: 3e12a98fe397 ("kni: optimize Rx burst")
> >     > Cc: stable@dpdk.org <mailto:stable@dpdk.org>
> >     >
> >     > Signed-off-by: Yangchao Zhou <zhouyates@gmail.com
> >     <mailto:zhouyates@gmail.com>>
> 
> Acked-by: Ferruh Yigit <ferruh.yigit@amd.com>

Applied, thanks.