From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1FE7046B0A for ; Sat, 5 Jul 2025 21:08:40 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 163EF4028B; Sat, 5 Jul 2025 21:08:40 +0200 (CEST) Received: from mail-qk1-f170.google.com (mail-qk1-f170.google.com [209.85.222.170]) by mails.dpdk.org (Postfix) with ESMTP id A14544025F for ; Sat, 5 Jul 2025 21:08:38 +0200 (CEST) Received: by mail-qk1-f170.google.com with SMTP id af79cd13be357-7d0a0bcd3f3so220881385a.1 for ; Sat, 05 Jul 2025 12:08:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20230601.gappssmtp.com; s=20230601; t=1751742518; x=1752347318; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:from:to:cc:subject:date :message-id:reply-to; bh=0XaOj6J3kfMZvyraMwAL9e9KHq01891wkqWj1UJ9Ifw=; b=dlnozby+AzkTRiuPhf53DfR7UDgVn3Afd8eK0cqNiawUriuYOPlbzeyUclBrPwT/xm T6IIdjV2bnhv3dtMPueCFDH45wX8/nK390WBKdPmFD8M6zsQF+6pLUGEjErzkge9JEXL eezSbw59CwJylXqwLCDe8cneuIH70i4Yd0ttr2ONCPuBi4JOZPjyHE9h+1SzGGgd6OUY NWfZP/9SB5J4PQw9P2cHygx1Pv2Tx0qEubtcKzE/HasavQodE7TRQ7Zg6r7ec+w33q81 SByzjIWybrcu76dnDJtNbc+odQK83e0l48pPd9y56AMFwu6r0FUe0F3/L4jQIID8hO+F Gp/g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1751742518; x=1752347318; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=0XaOj6J3kfMZvyraMwAL9e9KHq01891wkqWj1UJ9Ifw=; b=RipfVB58cpn648qmzzNwIL8Iez/UqWB7hgou3gepPhP0n0H94uT4H+3oau02Zvmxeu KHJrlts0SAwQxELErKAcO8Q+P+7KOf0uPAkeHA16bCU/gbOOpqASAuUiJypqvAhtfRpg 5iuRA3FsrV/o9yXjOh268YfPxtUsdQ7tMgRdzos4u79Gp7xqDZl0jwjmnC5mRtzo9De0 yFJL/ac/1ca4AdspBC5M3+zVKiVRkGbRRVs7ZxZ36idytsgBMxHQ/MzQ6i4Pg/VD0tEC qhf2sblP4pbPSCfRdggO6UD61Zuf6atzOF3P00umi8kFYDwJDbLNwlxWyetGl2wsgPj2 YRPw== X-Forwarded-Encrypted: i=1; AJvYcCUFS6T5A2nM03GZUyQubHo60alUIhBNcdnC79LwPdAW24Ny59ErjQfub7l7WGRYJ09KAXZaaw==@dpdk.org X-Gm-Message-State: AOJu0YxjHUVKwzOzN6WNba7D++pEhkQcnfGIqyDrpb101uUTV3m15N+7 Q1fBq8gFa+//lTCLmogmVX/DiU10/MohkgA54pdJe3FyEV5XMfZK9zeZ4s2/n01NxRUAxhiaEMW pe7Ri X-Gm-Gg: ASbGncv4H/OYnb4roivHten+bDHeRYn6UO59XTOn+ei33M6uTVe+BZn82xDNuxanCi6 SHhlQzjOZPyl29bJofVhj283Z5W6ROZdUceLMZkYrrZ/zuZ4bMvY8vJr9bjx8oxziU17pgDILYZ IHxRtw1H23ymdhfnebvIoUbhewMksVOloDDSWgk/JyM1X/LuP9P/KZyF0ugFl14ya5GMjU7sSWh WNRE6IcQVv0tNSebhYybqCOsr45rY7VF8OmZrVq76PvCzSC9Gj9oFV2no5Q/EiJPztWLSGVan+H ex4MEH1RP818+/7CPYwRI4Jq64I/9Tw2Wn5SKt3/ZIWuG5mfs9vaJLHYmUQv3ukfT0jvUDMimmg Utp0xpZxfAlhVGRqiMJIywI1Hl/EWyiLmTNYt9g5lcnPPPEjdqA== X-Google-Smtp-Source: AGHT+IFl95Z5sSVpUpqEMLNioJuYNnU8deyupCQ2oAyiPpavVrQHsiocP0hCMf9t6cYrdfrzzS2HGg== X-Received: by 2002:a05:620a:4396:b0:7ce:ea9d:5967 with SMTP id af79cd13be357-7d5dc6b8cfdmr1118791085a.15.1751742517921; Sat, 05 Jul 2025 12:08:37 -0700 (PDT) Received: from hermes.local (204-195-96-226.wavecable.com. [204.195.96.226]) by smtp.gmail.com with ESMTPSA id af79cd13be357-7d5dbdb3865sm351133985a.29.2025.07.05.12.08.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 05 Jul 2025 12:08:37 -0700 (PDT) Date: Sat, 5 Jul 2025 12:08:34 -0700 From: Stephen Hemminger To: "Lombardo, Ed" Cc: Ivan Malov , users Subject: Re: dpdk Tx falling short Message-ID: <20250705120834.78849e56@hermes.local> In-Reply-To: References: <20250704074957.5848175a@hermes.local> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org On Sat, 5 Jul 2025 17:36:08 +0000 "Lombardo, Ed" wrote: > Hi Stephen, > I saw your response to more mempools and cache behavior. > > I have a goal to support 2x100G next, and if I can't get 10G with DPDK then something is seriously wrong. > > Should I build the dpdk static libraries with LTO? > > Thanks, > Ed Are you doing anything in the fast path that is an obvious cache miss. at 10Gbit/sec and size of 84 bytes = 67.2ns CPU's haven't got that much faster 3G cpu that is 201 cycles. Single cache miss is 32ns, so two cache misses means per-packet budget is gone. Obvious cache misses. - passing packets to worker with ring - using spinlocks (cost 16ns) - fetching TSC - syscalls? Also, never ever use floating point. Kernel related and older but worth looking at: https://people.netfilter.org/hawk/presentations/LCA2015/net_stack_challenges_100G_LCA2015.pdf