From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <dev-bounces@dpdk.org> Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A42F345C88; Tue, 5 Nov 2024 16:55:19 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4948B4027E; Tue, 5 Nov 2024 16:55:19 +0100 (CET) Received: from mail-pl1-f181.google.com (mail-pl1-f181.google.com [209.85.214.181]) by mails.dpdk.org (Postfix) with ESMTP id 06A4940151 for <dev@dpdk.org>; Tue, 5 Nov 2024 16:55:17 +0100 (CET) Received: by mail-pl1-f181.google.com with SMTP id d9443c01a7336-21116b187c4so39549725ad.3 for <dev@dpdk.org>; Tue, 05 Nov 2024 07:55:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20230601.gappssmtp.com; s=20230601; t=1730822117; x=1731426917; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:from:to:cc:subject:date :message-id:reply-to; bh=4jT74CTkQTNTteUOz0uxUtlE3cdsW/e+E42VA/sZiiM=; b=Rvw2AnR/VQljGdeCbQcAZX06clFH5n0EKUYEyj50dpF7WZ8OaDvqdla1nPcONxt32N +KTZiGYYZZLlPHGQ14rIsalFQ2uBPLdMPwQ4SqkFApX+u98EL9uGZ5O6S5ktQ3C34ZYH FPxinwDoCE3LGgvVaffXksJmGrEo3cpb3Rcokdhjn/P3FD9O76FwnI8uA1ZapvDQae1h gxsZJSEe1YMvSJT8xQWPWlhBGxKDaDAGV2p038wYUiLS37Nviku875mFdlY9GQfH8TKV uhNesIoyLGKRrXRIgss8+4nLSNmwfqLQ0l5os9C8WPh5F+q+betPW1JemmTTOTGzwH4G Pq8Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730822117; x=1731426917; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4jT74CTkQTNTteUOz0uxUtlE3cdsW/e+E42VA/sZiiM=; b=Qmv68uJ69Z2oCwkfQg985/uTORsfKS4nOAGIZnDL8CVwGYnW6K23k4YBRPe9bNOcyM G8qUitBUOC7SoJ50w5Rd4R+L7Fi1kF+nHpHEfaMH0/5w6wGbbK7SV4D5vn6OjK6DkRG4 twwR3RUa/TJruI6cTPTZqkQQzWe3NLJfuFIE/9E1PvAgOeRNIFudkg2fjclQncKH1HTf EBp2+JAanWBTiRCOrPwh01aR82ridc4cHlA0/SEJdSqszlUCCnnp04BIPUgqgwDhdQ7W 2wIntWlWRZyCDF9McJovPJXfCpiOSaOMo9DtS8NAZKuVkWIrNZJHG7bqww+3Heyk8BBi pJ/A== X-Forwarded-Encrypted: i=1; AJvYcCWsuCh8WhJVVWT9WmP6e8fycTEGwZXek4rl7qYCh5xqiet624+nYRmVqSCf3Kn7q/svFFw=@dpdk.org X-Gm-Message-State: AOJu0YxR8eqqV+W1MuhngZYGwIYrCllgjEgTRxnl6jodzmI7lR8RJkFe 6i6IL5fPY5qp2/6AmoB53+buwE7hTCvxgvOxRaXv/jCUSrRwwaiFQECPLyapi/Y= X-Google-Smtp-Source: AGHT+IEyP65fQz9eOZWJz6l4WIsKcCvOARozhapfbPzkqXpLlVjw/Az3bEKH1Xz0RpMCNgyD4TlQcw== X-Received: by 2002:a17:902:dacf:b0:20c:7be3:2832 with SMTP id d9443c01a7336-2111af6c545mr242399705ad.31.1730822116843; Tue, 05 Nov 2024 07:55:16 -0800 (PST) Received: from hermes.local (204-195-96-226.wavecable.com. [204.195.96.226]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-211057d41c2sm80393165ad.254.2024.11.05.07.55.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 Nov 2024 07:55:16 -0800 (PST) Date: Tue, 5 Nov 2024 07:55:14 -0800 From: Stephen Hemminger <stephen@networkplumber.org> To: Morten =?UTF-8?B?QnLDuHJ1cA==?= <mb@smartsharesystems.com> Cc: =?UTF-8?B?THVrw6HFoSDFoGnFoW1pxaE=?= <sismis@cesnet.cz>, <anatoly.burakov@intel.com>, <ian.stokes@intel.com>, <dev@dpdk.org>, <bruce.richardson@intel.com> Subject: Re: [PATCH] net: increase the maximum of RX/TX descriptors Message-ID: <20241105075514.45ebea5c@hermes.local> In-Reply-To: <98CBD80474FA8B44BF855DF32C47DC35E9F876@smartserver.smartshare.dk> References: <20241029124832.224112-1-sismis@cesnet.cz> <98CBD80474FA8B44BF855DF32C47DC35E9F845@smartserver.smartshare.dk> <b62a69ae-59dc-43c5-8e94-36c21d83e37f@cesnet.cz> <20241030082020.2fe8eadb@hermes.local> <75463f4f-4139-4a53-9e63-05fe4cccb74f@cesnet.cz> <20241030090643.66af553f@hermes.local> <98CBD80474FA8B44BF855DF32C47DC35E9F876@smartserver.smartshare.dk> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions <dev.dpdk.org> List-Unsubscribe: <https://mails.dpdk.org/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://mails.dpdk.org/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <https://mails.dpdk.org/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> Errors-To: dev-bounces@dpdk.org On Tue, 5 Nov 2024 09:49:39 +0100 Morten Br=C3=B8rup <mb@smartsharesystems.com> wrote: > >=20 > > I suspect AF_PACKET provides an intermediate step which can buffer more > > or spread out the work. =20 >=20 > Agree. It's a Linux scheduling issue. >=20 > With DPDK polling, there is no interrupt in the kernel scheduler. > If the CPU core running the DPDK polling thread is running some other thr= ead when the packets arrive on the hardware, the DPDK polling thread is NOT= scheduled immediately, but has to wait for the kernel scheduler to switch = to this thread instead of the other thread. > Quite a lot of time can pass before this happens - the kernel scheduler d= oes not know that the DPDK polling thread has urgent work pending. > And the number of RX descriptors needs to be big enough to absorb all pac= kets arriving during the scheduling delay. > It is not well described how to *guarantee* that nothing but the DPDK pol= ling thread runs on a dedicated CPU core. That why any non-trivial DPDK application needs to run on isolated cpu's.