From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 24350A0542; Fri, 11 Nov 2022 10:12:34 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id BA1E242D1D; Fri, 11 Nov 2022 10:12:33 +0100 (CET) Received: from mail-wm1-f54.google.com (mail-wm1-f54.google.com [209.85.128.54]) by mails.dpdk.org (Postfix) with ESMTP id 9603442D0C; Fri, 11 Nov 2022 10:12:32 +0100 (CET) Received: by mail-wm1-f54.google.com with SMTP id fn7-20020a05600c688700b003b4fb113b86so2776921wmb.0; Fri, 11 Nov 2022 01:12:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=FmqsaBN83ZplKSPih6g/0wQCmF0xWb92h6mda4tD4UM=; b=mqbzD8BSDOnVkCkPiF/xcKlu+MoqDGxvf9zyj8qq721Fc7NwR0QohDtUo1tUvo6SAo Ttv+lLSL6MgLx3axJKBaW4s97c/6ZEpHA+zu0cFSCS2BRhdixxtvxvEXNcyC+QQd3Sbt SOXDvLArCgdeHoTPcVuKsAtUsnS/8g7B0JSky/EE3WnuPevtpEe6RsgoPOKFZGb/8FUy UgzeHpB3L9ieiQSQGWy6l39frq9aF7FfRAloceLnpT6X6e8fJg9/4aDtuWKdMtR0cKpK kF4KTYkmHa8IRRpb8d10/cf3zwYfFlTB5PE4IW44Sr3ngwpT9QOKKRQRjBTDrMWXe4pm Q4+A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=FmqsaBN83ZplKSPih6g/0wQCmF0xWb92h6mda4tD4UM=; b=xsnh04nYb0UIgoxmmhbZhlxHI2zHhcm8Q7futNQa9MPlaxCsDxkivHXXV0QYPVsh1m OgRS+7AJGm9jCxucYgOCzNKXA5GmEyNop+IEPse9D2p80lM+NRb3LQAQ2+sTI1/nlHut VP8YWT0p2Fxt99I0H05Oj6g8zhMiwXeYu/wgEcLH8JijCQvliqDdO8NgG5AZ/27c86lZ 5pNUfcepvZOXfxIsnNyaujfYwx6eEe6th5538KgTjM4ePjg7oiOVu5p+EuLp3yfbclyi b70KqV/G68Fv4wzy62lQnzEbP+OWfKYnwd02SYe0oK2AUbWukvjhpQjRVWe71apJAdr3 2QQg== X-Gm-Message-State: ANoB5pnAVydKi9ioNghw1U5/D64D6mElxqIttVpQAgVLEBgK5Ie91E9j 98tTarVCFNj56ApW7Wuxk/L/sFidc3Fpm9RnFK8= X-Google-Smtp-Source: AA0mqf6NzgxAQGABqRHrrHH1VALKQM4fgT+LggAYqx9in2sgMtSNn9D03QcjeOEFUaUBbgnLAFOwnte73gEItVBoC9o= X-Received: by 2002:a7b:c85a:0:b0:3cf:781a:4310 with SMTP id c26-20020a7bc85a000000b003cf781a4310mr596453wml.150.1668157952239; Fri, 11 Nov 2022 01:12:32 -0800 (PST) MIME-Version: 1.0 References: <20221109051339.1998037-1-zhouyates@gmail.com> <20221109060434.2012064-1-zhouyates@gmail.com> <20221109083909.6536bce8@hermes.local> In-Reply-To: <20221109083909.6536bce8@hermes.local> From: Matt Date: Fri, 11 Nov 2022 17:12:20 +0800 Message-ID: Subject: Re: [PATCH v2] kni: fix possible alloc_q starvation when mbufs are exhausted To: Stephen Hemminger Cc: dev@dpdk.org, Hemant@freescale.com, stable@dpdk.org Content-Type: multipart/alternative; boundary="0000000000002c2e7e05ed2e4ac0" X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org --0000000000002c2e7e05ed2e4ac0 Content-Type: text/plain; charset="UTF-8" On Thu, Nov 10, 2022 at 12:39 AM Stephen Hemminger < stephen@networkplumber.org> wrote: > On Wed, 9 Nov 2022 14:04:34 +0800 > Yangchao Zhou wrote: > > > In some scenarios, mbufs returned by rte_kni_rx_burst are not freed > > immediately. So kni_allocate_mbufs may be failed, but we don't know. > > > > Even worse, when alloc_q is completely exhausted, kni_net_tx in > > rte_kni.ko will drop all tx packets. kni_allocate_mbufs is never > > called again, even if the mbufs are eventually freed. > > > > In this patch, we always try to allocate mbufs for alloc_q. > > > > Don't worry about alloc_q being allocated too many mbufs, in fact, > > the old logic will gradually fill up alloc_q. > > Also, the cost of more calls to kni_allocate_mbufs should be acceptable. > > > > Fixes: 3e12a98fe397 ("kni: optimize Rx burst") > > Cc: Hemant@freescale.com > > Cc: stable@dpdk.org > > > > Signed-off-by: Yangchao Zhou > > Since fifo_get returning 0 (no buffers) is very common would this > change impact performance. > It does add a little cost, but there is no extra mbuf allocation and deallocation. > > If the problem is pool draining might be better to make the pool > bigger. > Yes, using a larger pool can avoid this problem. But this may lead to resource wastage and full resource calculation is a challenge for developers as it involves to mempool caching mechanism, IP fragment cache, ARP cache, NIC txq, other transit queue, etc. The mbuf allocation failure may also occur on many NIC drivers, but if the mbuf allocation fails, the mbuf is not taken out so that it can be recovered after a retry later. KNI currently does not have such a takedown and recovery mechanism. It is also possible to consider implementing something similar to the NIC driver, but with more changes and other overheads. --0000000000002c2e7e05ed2e4ac0 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Thu, Nov 10, 2022 at 12:39 AM Step= hen Hemminger <stephen@net= workplumber.org> wrote:
On Wed,=C2=A0 9 Nov 2022 14:04:34 +0800
Yangchao Zhou <= zhouyates@gmail.com> wrote:

> In some scenarios, mbufs returned by rte_kni_rx_burst are not freed > immediately. So kni_allocate_mbufs may be failed, but we don't kno= w.
>
> Even worse, when alloc_q is completely exhausted, kni_net_tx in
> rte_kni.ko will drop all tx packets. kni_allocate_mbufs is never
> called again, even if the mbufs are eventually freed.
>
> In this patch, we always try to allocate mbufs for alloc_q.
>
> Don't worry about alloc_q being allocated too many mbufs, in fact,=
> the old logic will gradually fill up alloc_q.
> Also, the cost of more calls to kni_allocate_mbufs should be acceptabl= e.
>
> Fixes: 3e12a98fe397 ("kni: optimize Rx burst")
> Cc: Hemant@f= reescale.com
> Cc: stable@dpdk.o= rg
>
> Signed-off-by: Yangchao Zhou <zhouyates@gmail.com>

Since fifo_get returning 0 (no buffers) is very common would this
change impact performance.
It does add a little cost, = but there is no extra mbuf allocation
and deallocation.
=

If the problem is pool draining might be better to make the pool
bigger.
Yes, using a larger pool can avoid this proble= m. But this may lead to
resource wastage and full resource calcul= ation is a challenge for developers
as it involves to mempool cac= hing mechanism, IP fragment cache,
ARP cache, NIC txq, other tran= sit queue, etc.

The mbuf allocation failure may also occur on many N= IC drivers,
but if the mbuf allocation fails, the mbuf is not tak= en out so that
it can be recovered after a retry later.
KNI cu= rrently does not have such a takedown and recovery mechanism.
It = is also possible to consider implementing something similar to
th= e NIC driver, but with more changes and other overheads.
--0000000000002c2e7e05ed2e4ac0--