From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 04F1F45B97; Tue, 22 Oct 2024 03:12:56 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C52BB402BC; Tue, 22 Oct 2024 03:12:55 +0200 (CEST) Received: from mail-pf1-f173.google.com (mail-pf1-f173.google.com [209.85.210.173]) by mails.dpdk.org (Postfix) with ESMTP id 75FAB40295 for ; Tue, 22 Oct 2024 03:12:54 +0200 (CEST) Received: by mail-pf1-f173.google.com with SMTP id d2e1a72fcca58-71e4244fdc6so3552177b3a.0 for ; Mon, 21 Oct 2024 18:12:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20230601.gappssmtp.com; s=20230601; t=1729559573; x=1730164373; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:from:to:cc:subject:date :message-id:reply-to; bh=xVMlppsuxyg1/MFCUWKUrBRn7jHjbzar3e+CMgV7uZM=; b=sYF6qBzMs9smoJcXgTlMaL9LHA/uaUWEKaanOeGnuWB9IFznDL9rvcU7C3ArLY5xF6 IkHMbJIM9+24qv71zg4cJ+mYyqi21ZJhwr7xCvVfAgdz6rJMTWH5+KqOj7XQSez6miJf QFu5Pb/2M4rdgMQUVuk61BiG9oKopuxRlwSNkU5EoGci684sU6cImEW1aL/as/cyW/j7 bNOX8NHWrM8TqWSPq5MmgemRLXGvkf/j9bB2BclZhw4SqdP+w52bv2cYLw4a5FCsgEZw slG8vA+vFv4v+cSKEnnRjh+E40D+0IoTg2s0exmmpTdRFxf+OxjD1D9wKF5bVgK0ygrF nl9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729559573; x=1730164373; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xVMlppsuxyg1/MFCUWKUrBRn7jHjbzar3e+CMgV7uZM=; b=VEq5vulyl/n6/ty7ziqbzLebHL02AyiqDRyiTNDer2k9cWwtl0+3hmFwG5xedMR6PJ lpSgH1SABcXggyicl1mTzYH0yryGR23AkzVbOhBS4jpIJ4WrK9atk1y4R+u8uXU8pUsh msyuWNzst1urQGgB0gwGgor/hQmZZpPhZcJ1nSliRwbNyTtJIhkZ0/X+8j/tUnxsppYR ohRvrY7GAQNAyLBGjxYP3pzQySnKNBxFKtHIPToc1o+LnIIPRx2Qo0qoLA6QLJgIH6Yl IqBBjFGDeXUltfoigyfDEzQRaCQL28w0P/vzbgm9DiXS1oBFtq1PP4TfbLQTWVZvB5QU Cr0g== X-Gm-Message-State: AOJu0Yz69YNKS27sT91jNolAmhIsds0VGQs7tNSTSWwCbYCBv0EVtCL4 C/BVaZQXbluhgAEfEXVpS8SShoyY0feFZsbRq0zZqgOJ6u545xVwPbyscm3Jl38= X-Google-Smtp-Source: AGHT+IErkPRc7vTpIHJQGZdwiSHHmLCI+oBrsjhvXT1J5gDoPTo/1/Pli56xxP1if6hlQkDtthLy3A== X-Received: by 2002:a05:6a00:10c4:b0:71e:f4:dbc with SMTP id d2e1a72fcca58-71ea323a1d7mr14387590b3a.25.1729559573438; Mon, 21 Oct 2024 18:12:53 -0700 (PDT) Received: from hermes.local (204-195-96-226.wavecable.com. [204.195.96.226]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-7eaeab1e1f2sm3772455a12.24.2024.10.21.18.12.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Oct 2024 18:12:53 -0700 (PDT) Date: Mon, 21 Oct 2024 18:12:51 -0700 From: Stephen Hemminger To: Wathsala Vithanage Cc: dev@dpdk.org, nd@arm.com Subject: Re: [RFC v3 0/2] An API for Stashing Packets into CPU caches Message-ID: <20241021181251.6f9f69b6@hermes.local> In-Reply-To: <20241021015246.304431-1-wathsala.vithanage@arm.com> References: <20240715221141.16153-1-wathsala.vithanage@arm.com> <20241021015246.304431-1-wathsala.vithanage@arm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Mon, 21 Oct 2024 01:52:44 +0000 Wathsala Vithanage wrote: > DPDK applications benefit from Direct Cache Access (DCA) features like > Intel DDIO and Arm's write-allocate-to-SLC. However, those features do > not allow fine-grained control of direct cache access, such as stashing > packets into upper-level caches (L2 caches) of a processor or the shared > cache of a chiplet. PCIe TLP Processing Hints (TPH) addresses this need > in a vendor-agnostic manner. TPH capability has existed since > PCI Express Base Specification revision 3.0; today, numerous Network > Interface Cards and interconnects from different vendors support TPH > capability. TPH comprises a steering tag (ST) and a processing hint > (PH). ST specifies the cache level of a CPU at which the data should be > written to (or DCAed into), while PH is a hint provided by the PCIe > requester to the completer on an upcoming traffic pattern. Some NIC > vendors bundle TPH capability with fine-grained control over the type of > objects that can be stashed into CPU caches, such as > > - Rx/Tx queue descriptors > - Packet-headers > - Packet-payloads > - Data from a given offset from the start of a packet > > Note that stashable object types are outside the scope of PCIe standard; > therefore, vendors could support any combination of the above items as > they see fit. > > To enable TPH and fine-grained packet stashing, this API extends the > ethdev library, PCI library, and the PCI driver. In this design, the > application via the ethdev stashing API provides hints to the PMD to > indicate the underlying hardware at which processor and cache level it > prefers a packet to end up. Once the PMD receives a CPU and a > cache-level combination, it must extract the matching ST from the TPH > ACPI _DSM of the PCIe root port to which the NIC is connected. To > facilitate the extraction of STs, the PCI library and the PCI driver > APIs are extended. There is a fundamental conflict with the increasing growth of "nerd knobs" like this in the DPDK. Users already have problems understanding DPDK and adding more complexity does not help. So any new feature like this should be: 1. Just work right without any configuration. It can't suck by default. 2. The API's should be used in the drivers and core, not exposed up to the application. Most of the hot data structures are in the drivers now. 3. Fit into existing API models. Like rte_prefetch(). Is the goal of DPDK enabling high speed applications, or enabling vendor benchmarks?