From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B77CE45E3A; Thu, 5 Dec 2024 16:40:40 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5F6D340648; Thu, 5 Dec 2024 16:40:40 +0100 (CET) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id D4B69402E3 for ; Thu, 5 Dec 2024 16:40:37 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1733413237; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jrKnooVKZnfmS5V6vFCpmMQ6Lr/6ZIqIdyhR4Z2rhuM=; b=jCb51rN1t0LV2/Xo0fFJVJx3JVt0P0vnNZAjdlQKrTyape28zIoXLKjm+unAj2PFrzPS5C gMUMbNZgQTlnY6Js9CuvGAauQWIblverF+ruel60OP2F57PvBRmeuH2E8il3FA3uakAJFS LZ5Gao6PlOpE0lQw1NzHxECw8vFwdfc= Received: from mail-lf1-f70.google.com (mail-lf1-f70.google.com [209.85.167.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-296-796y5JqwPJSEfXxWpMfgnw-1; Thu, 05 Dec 2024 10:40:35 -0500 X-MC-Unique: 796y5JqwPJSEfXxWpMfgnw-1 X-Mimecast-MFC-AGG-ID: 796y5JqwPJSEfXxWpMfgnw Received: by mail-lf1-f70.google.com with SMTP id 2adb3069b0e04-53e21e51f9dso715897e87.1 for ; Thu, 05 Dec 2024 07:40:35 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733413234; x=1734018034; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jrKnooVKZnfmS5V6vFCpmMQ6Lr/6ZIqIdyhR4Z2rhuM=; b=TzDm7+I27AOtfvPlSsyA0WNi7LgQIT5hLI2X0UnXU+u1R3LZO/NYEo5t9245aEfB7u Hl+nVpKwkIfSDMz/+3ceI+vJJC6cF7SyPpQtpuguu5wC6OGS/P4jZTDfeCv9fsyn6jO5 QTLp4MNMk1YibMpJYXw+VGEYiqaHV3FJxg0KcAhuqfvG50x2L1h1LJLSvUgaVox5opXh TH1HjEIDWJCrCCO3bdbVjEZKx9ux/pasI40t2glHtAWZwJeAUUi8++yRG+6A7BMuaJxY myOGcIqSXYNYuLpCvvwEoNrmCSe8d9vO9oFG1fD20a2JPNw6P8i9cqqZeo8r3qh7pLS7 lVgA== X-Forwarded-Encrypted: i=1; AJvYcCX9E17UaxAqW7NRwfkUbEWy5fUVG4xbxxUJAmouNragS8Fuqhz5Whzj0MY8jDmi2+rBop0=@dpdk.org X-Gm-Message-State: AOJu0YzyYAK3WD4WKLPOZbzW9Mbz3KHrMB5jf0Il6Vd1KSWGy1b8R55+ Xhb+BfqOM0IKPT8ZUU5G9bMlmL5rz6BqX4dxkj20M3scNAXLEcB4wkosUvv1es7f9OjUXyWZbXI WqE05WlhkPu0XpBiPIAEMLBMUdyYY2h9wEYSpya9X+x/FweoKnlz4aaTSigdLmiZyyyGQXXidBF kwC+lR87CESyBsn1kAqCykca3TJeJ/ X-Gm-Gg: ASbGnctae6xshNB5+/9ehAZ3yHHyfEpQMHjtmpf3W4ByaJoKUmNCnmv82f4qZc2yoS8 URnzLK39s+pmhLNiYh5TQ1opDZKcTIMKXDg== X-Received: by 2002:a05:6512:2392:b0:53e:1c3e:666 with SMTP id 2adb3069b0e04-53e21700c2amr1226477e87.10.1733413234184; Thu, 05 Dec 2024 07:40:34 -0800 (PST) X-Google-Smtp-Source: AGHT+IFMstnGJOwbGg+iP4IXq30dv4+H+VBmrN5ocgfJF81u7Q0GdS8xh8wByTWgBjn6SVzToL7qaL+OTcIBpi8hjJQ= X-Received: by 2002:a05:6512:2392:b0:53e:1c3e:666 with SMTP id 2adb3069b0e04-53e21700c2amr1226462e87.10.1733413233769; Thu, 05 Dec 2024 07:40:33 -0800 (PST) MIME-Version: 1.0 References: <20240715221141.16153-1-wathsala.vithanage@arm.com> <20241021015246.304431-1-wathsala.vithanage@arm.com> <20241021015246.304431-3-wathsala.vithanage@arm.com> <20241203131322.41f3afb3@hermes.local> In-Reply-To: <20241203131322.41f3afb3@hermes.local> From: David Marchand Date: Thu, 5 Dec 2024 16:40:22 +0100 Message-ID: Subject: Re: [RFC v3 2/2] ethdev: introduce the cache stashing hints API To: Stephen Hemminger Cc: Wathsala Vithanage , Thomas Monjalon , Ferruh Yigit , Andrew Rybchenko , dev@dpdk.org, nd@arm.com, Honnappa Nagarahalli , Dhruv Tripathi X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: KMwkJGk6CI-3t3IhbaWhg75xzYcVY7thZHFEQ9M1_wc_1733413234 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Tue, Dec 3, 2024 at 10:13=E2=80=AFPM Stephen Hemminger wrote: > > On Mon, 21 Oct 2024 01:52:46 +0000 > Wathsala Vithanage wrote: > > > Extend the ethdev library to enable the stashing of different data > > objects, such as the ones listed below, into CPU caches directly > > from the NIC. > > > > - Rx/Tx queue descriptors > > - Rx packets > > - Packet headers > > - packet payloads > > - Data of a packet at an offset from the start of the packet > > > > The APIs are designed in a hardware/vendor agnostic manner such that > > supporting PMDs could use any capabilities available in the underlying > > hardware for fine-grained stashing of data objects into a CPU cache > > (e.g., Steering Tags int PCIe TLP Processing Hints). > > > > The API provides an interface to query the availability of stashing > > capabilities, i.e., platform/NIC support, stashable object types, etc, > > via the rte_eth_dev_stashing_capabilities_get interface. > > > > The function pair rte_eth_dev_stashing_rx_config_set and > > rte_eth_dev_stashing_tx_config_set sets the stashing hint (the CPU, > > cache level, and data object types) on the Rx and Tx queues. > > > > PMDs that support stashing must register their implementations with the > > following eth_dev_ops callbacks, which are invoked by the ethdev > > functions listed above. > > > > - stashing_capabilities_get > > - stashing_rx_hints_set > > - stashing_tx_hints_set > > > > Signed-off-by: Wathsala Vithanage > > Reviewed-by: Honnappa Nagarahalli > > Reviewed-by: Dhruv Tripathi > > > > --- > > lib/ethdev/ethdev_driver.h | 66 +++++++++++++++ > > lib/ethdev/rte_ethdev.c | 120 +++++++++++++++++++++++++++ > > lib/ethdev/rte_ethdev.h | 161 +++++++++++++++++++++++++++++++++++++ > > lib/ethdev/version.map | 4 + > > 4 files changed, 351 insertions(+) > > > > diff --git a/lib/ethdev/ethdev_driver.h b/lib/ethdev/ethdev_driver.h > > index 1fd4562b40..7caaea54a8 100644 > > --- a/lib/ethdev/ethdev_driver.h > > +++ b/lib/ethdev/ethdev_driver.h > > @@ -1367,6 +1367,68 @@ enum rte_eth_dev_operation { > > typedef uint64_t (*eth_get_restore_flags_t)(struct rte_eth_dev *dev, > > enum rte_eth_dev_operation op= ); > > > > +/** > > + * @internal > > + * Set cache stashing hints in Rx queue. > > + * > > + * @param dev > > + * Port (ethdev) handle. > > + * @param queue_id > > + * Rx queue. > > + * @param config > > + * Stashing hints configuration for the queue. > > + * > > + * @return > > + * -ENOTSUP if the device or the platform does not support cache sta= shing. > > + * -ENOSYS if the underlying PMD hasn't implemented cache stashing = feature. > > + * -EINVAL on invalid arguments. > > + * 0 on success. > > + */ > > +typedef int (*eth_stashing_rx_hints_set_t)(struct rte_eth_dev *dev, ui= nt16_t queue_id, > > + struct rte_eth_stashing_config= *config); > > + > > +/** > > + * @internal > > + * Set cache stashing hints in Tx queue. > > + * > > + * @param dev > > + * Port (ethdev) handle. > > + * @param queue_id > > + * Tx queue. > > + * @param config > > + * Stashing hints configuration for the queue. > > + * > > + * @return > > + * -ENOTSUP if the device or the platform does not support cache sta= shing. > > + * -ENOSYS if the underlying PMD hasn't implemented cache stashing = feature. > > + * -EINVAL on invalid arguments. > > + * 0 on success. > > + */ > > +typedef int (*eth_stashing_tx_hints_set_t)(struct rte_eth_dev *dev, ui= nt16_t queue_id, > > + struct rte_eth_stashing_config= *config); > > + > > +/** > > + * @internal > > + * Get cache stashing object types supported in the ethernet device. > > + * The return value indicates availability of stashing hints support > > + * in the hardware and the PMD. > > + * > > + * @param dev > > + * Port (ethdev) handle. > > + * @param objects > > + * PMD sets supported bits on return. > > + * > > + * @return > > + * -ENOTSUP if the device or the platform does not support cache sta= shing. > > + * -ENOSYS if the underlying PMD hasn't implemented cache stashing = feature. > > + * -EINVAL on NULL values for types or hints parameters. > > + * On return, types and hints parameters will have bits set for supp= orted > > + * object types and hints. > > + * 0 on success. > > + */ > > +typedef int (*eth_stashing_capabilities_get_t)(struct rte_eth_dev *dev= , > > + uint16_t *objects); > > + > > /** > > * @internal A structure containing the functions exported by an Ether= net driver. > > */ > > @@ -1393,6 +1455,10 @@ struct eth_dev_ops { > > eth_mac_addr_remove_t mac_addr_remove; /**< Remove MAC addre= ss */ > > eth_mac_addr_add_t mac_addr_add; /**< Add a MAC address = */ > > eth_mac_addr_set_t mac_addr_set; /**< Set a MAC address = */ > > + eth_stashing_rx_hints_set_t stashing_rx_hints_set; /**< Set Rx = cache stashing*/ > > + eth_stashing_tx_hints_set_t stashing_tx_hints_set; /**< Set Tx = cache stashing*/ > > + /** Get supported stashing hints*/ > > + eth_stashing_capabilities_get_t stashing_capabilities_get; > > /** Set list of multicast addresses */ > > eth_set_mc_addr_list_t set_mc_addr_list; > > mtu_set_t mtu_set; /**< Set MTU */ > > Since eth_dev_ops is visible in application binary, it is part of the ABI= . > Therefore it can not be changed until 25.11 release. The layout of eth_dev_ops is not exposed to applications as it is in a private header. Could you clarify where you see a breakage for an application? I see an ABI breakage for out of tree drivers though. This could be avoided by moving those added ops at the end of the struct? --=20 David Marchand