From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1E7FFA04A3; Fri, 17 Dec 2021 17:24:45 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id B4FA44013F; Fri, 17 Dec 2021 17:24:44 +0100 (CET) Received: from mail-pg1-f172.google.com (mail-pg1-f172.google.com [209.85.215.172]) by mails.dpdk.org (Postfix) with ESMTP id 4840140040 for ; Fri, 17 Dec 2021 17:24:43 +0100 (CET) Received: by mail-pg1-f172.google.com with SMTP id r138so2568138pgr.13 for ; Fri, 17 Dec 2021 08:24:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20210112.gappssmtp.com; s=20210112; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=2AeBySaXpcHyt8+/+t62G0J7C1a6G0GCUFDZj8j1PNU=; b=X2krD700rUImXLijPvm9+7cxCcVDWWPnfPd8cwp94LXvxSRUYan7cQKKLJPV4KC9t0 OLFO3xAcg6hc89G20lAB4dI3V/3qhYw05e1nkcMweWbvPdxxtV9ibUiuS/gFCdZ5fm2i P/L1Pab1C7gRNAoziffd+vd8adE73K7nm2tTqcUaQtESkdKw/002WU6DA0yv5OaeEMMi s22+Ie/JldGORcDE7raznOlwxt2gx7XMQrG0kZZFoCCEp5reMh6uHf/i0baNhvyLH/bw pg0s3r27tZDQ+i6tx36a+SJhKl+R/4s4GoM++A/L+lb8a+SWO3y6zEJoAEd9J0fNcjTD 0X9g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2AeBySaXpcHyt8+/+t62G0J7C1a6G0GCUFDZj8j1PNU=; b=ScyDuUdTgqyU+XVhEy9/t8VGACUZxKD47C4wSQCfs4RdKn4A6GFIvXb9/vyOwz5Lbe 7E3d+Qyu/cZ87nMGLXrhpOPCfLy/xluizuOq0+iAV4Pf4+YBDim+exM6eEztubW0XYmn Nbx+Ipr2qeBotlF0Pq6m10jWxZKMYEfozauY0Zdfk6fulems+QfXNrwq+tjImHpSgyqz NRfD0RKLIU0hDaRJOlFaBmNzJnW4dKBGTH1IW0Wcs2/MKSYRZ3LeuxBvCgeXNZmGNWqy V2efuo6WWWOocwlyejmgRZFwjbuRlu7pHz7rVoarHuQSfKfXzVS/LZFbP1C/BhwHKdMi B81w== X-Gm-Message-State: AOAM532BXJziUdxlXblNh1Gq2a+m0YwdxvxGD2+0A9P3DyYv0m0O9uGn 6yj+T2EHNmcMa4vPto/rSVZa2A== X-Google-Smtp-Source: ABdhPJyfhiqexykjdQ3Cx0hECqlqMrn/1pLQkBubJkplOjxqLKsfj0f+23uvSDLPcEzBNuIP3YAMbg== X-Received: by 2002:a62:1997:0:b0:4ba:5cb2:7c34 with SMTP id 145-20020a621997000000b004ba5cb27c34mr3975710pfz.8.1639758282350; Fri, 17 Dec 2021 08:24:42 -0800 (PST) Received: from hermes.local (204-195-112-199.wavecable.com. [204.195.112.199]) by smtp.gmail.com with ESMTPSA id mq14sm13284690pjb.54.2021.12.17.08.24.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 17 Dec 2021 08:24:41 -0800 (PST) Date: Fri, 17 Dec 2021 08:24:39 -0800 From: Stephen Hemminger To: Tudor Cornea Cc: ferruh.yigit@intel.com, thomas@monjalon.net, dev@dpdk.org Subject: Re: [PATCH] kernel/kni: retry the xmit in case ring is full Message-ID: <20211217082439.13d36631@hermes.local> In-Reply-To: <1639753232-115930-1-git-send-email-tudor.cornea@gmail.com> References: <1639753232-115930-1-git-send-email-tudor.cornea@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Fri, 17 Dec 2021 17:00:32 +0200 Tudor Cornea wrote: > This patch attempts to avoid dropping packets that are to be > transmitted, in case there is no space in the KNI ring. > > We have a use case in which we leverage the Linux TCP / IP stack for > control plane, and some protocols might be sensitive to packet drops. > > This might mean that the sender (Kernel) might be moving at a faster pace > than the receiver end (DPDK application), or it might have some brief > moments of bursty traffic patterns. > > Requeuing the packets could add a kind of backpressure until a transmit > window is available to us. > > The burden of retransmitting is shifted to the caller of ndo_start_xmit, > which in our case is the configured queuing discipline. This way, the > user should be able to influence the behavior w.r.t dropping packets, > by picking the desired queuing discipline. > > Although it should technically be a good approach, from what > I have tested, stopping the queue prior to returning NETDEV_TX_BUSY seems > to add some extra overhead, and degrade the control-plane performance > a bit. > > Signed-off-by: Tudor Cornea NAK Doing this risks having a CPU lockup if userspace does not keep up or the DPDK application gets stuck. There are better ways to solve the TCP stack queue overrun issue: 1. Use a better queueing discipline on the kni device. The Linux default of pfifo_fast has bufferbloat issues. Use fq_codel, fq, codel or pie? 2. KNI should implement BQL so that TCP stack can see lock backpressure about possible queue depth. As a simple workaround increase the KNI ring size. It won't solve the whole problem but i tcan help