From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 857A244191; Wed, 18 Sep 2024 11:22:47 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4F3FC42EF1; Wed, 18 Sep 2024 11:22:47 +0200 (CEST) Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2080.outbound.protection.outlook.com [40.107.237.80]) by mails.dpdk.org (Postfix) with ESMTP id 9F7E442E57 for ; Wed, 18 Sep 2024 11:22:45 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=ivLVNO68WcoxmsS1XiDh46St9JwD+er+B7dDUjBU6Uw3zlRauHDAsz/qs+sw7HxEoLbRCqyRfkIlkwCZATKUqTfOI4lY9ZWLArNDZ8UiZWZdjqzZeTYxd3LrVhngXd7kybNnGdQyrzXXFxFqB1MgTq3MtkOWfsWcRvaf5N1el/9pxb2mfHw/IG+paOIIlAi7dbUUFla4wFo0MDDhDHzPkebnV/uwbQ/XAEafI0Ll9UR1zP+RwRGUdQQ9qAlyPWhRB4MoBrA4ntudkuiTn+asOtcagMHAo+tJDTFBKA0mELHMgzOZcfwaXcZBJj/zkUSOtcrgtzjh58HCwSGzSI3fVA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=RmISnjFETsz8V3iEGFLvubDUBVKUlHAqUnOC8rVOL04=; b=D8J1EqnCS7+wMNidOk/eM78cynUQQ2+CdDWtIqkJ5N4y7R+iyRNL/AaBrnb7O5ENnpusZ1mLzod6QYJo17Ta5ylJSecLT2wOBltujSSvtPk3Gxmy3kqeyjU0yZXa1XG08LyQH8BKt7yrkSP8J6BfnbqfmCFTGfDrxIFYcUIfVeQJrMGR7az3Ug39MQJdMtA8XRyU4dIIG82kmm0lo4d99hw17M/rs+xu+K8A8xjjbTZbf/yQlw8/4IP0R/cfc0oSTgg0mEXM6Jm1DQZpPTXC+2QoasepQ/Kk+MN8aYK8i+1lV5PI9z/Qu8m2QGLPDLDMoyGpl3a2sz6VpvhTZzH3Zw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=monjalon.net smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=RmISnjFETsz8V3iEGFLvubDUBVKUlHAqUnOC8rVOL04=; b=li4pVakSIKQ6ygVPePhmJWk/06lIJQACiwF1TKfQO7ErL+B9sHyFnI5ojMUoTc5nubKD3J+YSsT+DoG0QpAy3vy35aJHytkJpVw2tYc2jEJcHrOwrQ/ML6u8+uFe987p7OcbDQIfrYlaHFXtdTAgjD5Oe2VH4aPPa8Q2j+btkEDtfIrUvmAWzZRcLRX+ED5Q5OYPG5y+ThhVinfV5+3I+EvyiPhlR9wq5stoP1p66SKkIwLZ046CsEaRQl5xsTHXwzvHvbuDFMFOTEeWE+mgl5PZutd7bBThfPW13gZuAO2Fl37z0tPhu/nQh3bSf/87doUxzUAM0AKhbu9NqVr4bQ== Received: from BN9PR03CA0067.namprd03.prod.outlook.com (2603:10b6:408:fc::12) by DM4PR12MB6568.namprd12.prod.outlook.com (2603:10b6:8:8f::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7962.24; Wed, 18 Sep 2024 09:22:41 +0000 Received: from BN1PEPF0000468E.namprd05.prod.outlook.com (2603:10b6:408:fc:cafe::e6) by BN9PR03CA0067.outlook.office365.com (2603:10b6:408:fc::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7939.24 via Frontend Transport; Wed, 18 Sep 2024 09:22:41 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by BN1PEPF0000468E.mail.protection.outlook.com (10.167.243.139) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7918.13 via Frontend Transport; Wed, 18 Sep 2024 09:22:41 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 18 Sep 2024 02:22:27 -0700 Received: from nvidia.com (10.126.230.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 18 Sep 2024 02:22:25 -0700 From: Dariusz Sosnowski To: Thomas Monjalon , Ferruh Yigit , Andrew Rybchenko CC: Subject: [RFC 0/4] ethdev: rework config restore Date: Wed, 18 Sep 2024 11:21:57 +0200 Message-ID: <20240918092201.33772-1-dsosnowski@nvidia.com> X-Mailer: git-send-email 2.39.5 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [10.126.230.35] X-ClientProxiedBy: rnnvmail202.nvidia.com (10.129.68.7) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN1PEPF0000468E:EE_|DM4PR12MB6568:EE_ X-MS-Office365-Filtering-Correlation-Id: 9087c94a-082b-4e23-a69c-08dcd7c37241 X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|376014|1800799024|36860700013|82310400026; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?lVYYXXTsj/7ueSlFkJkq8R3MVfi2pUPpD31YjAgvxvRnSyQ71l+XQUYFm0xX?= =?us-ascii?Q?7zARguWAOnjVWSJXjOf5JqfoigkiFa+Sa6QxBFIfGz+YYiiZYxxiLwHNagKj?= =?us-ascii?Q?5Cjn1mvKM8ep9yGasYaizIhA44Cs03MIc9LSgW9XxUeVuHQD0pgPrvFVH/20?= =?us-ascii?Q?egkHVQnfIwv/igrMpvqhUQuRJ21UZgzDl8hEX/AeKJEOpCMg105v5bwH1nJu?= =?us-ascii?Q?yT/C91PPUg/8snFmzlUu7bXUQEN8ZPJIFJTbM8JiLnnn67f/aL7YALhInuPu?= =?us-ascii?Q?8kKa9QBbRxpNsdo6BnsD4CN/VxkbnM44G6OFyhBgCtAawfUjWV7Fwr6ni9YY?= =?us-ascii?Q?qwnYiz0VP55xViuacGUtmZ4ENEP+f+5jCI3Vg5j9Qh6AjXlng8LatCq0YZRm?= =?us-ascii?Q?I/PknvGNrZBGPDx8xKpaMGw3+EtZXxQ7yej3HXhWsfwxeJ0+bfNblPwZDScf?= =?us-ascii?Q?Yhfsxzti0uA1Vprzpe672WuGC2V664S6fedXfIZw4WKjIGFe5kPH4Q5GzGYB?= =?us-ascii?Q?N5Pz8Jowup1IJFT7oFIDOZQ8bAmYanw46oV1Hc0U3FdFOyMSV7tVYjoLtD0/?= =?us-ascii?Q?lvlRZg/ZCIqWdF6E3qeu9eDaF3mGksSH100VtHVtOLFfuKSHj17MaZ9RaE4f?= =?us-ascii?Q?dlEqrMKysRA8WxShbE8VlQ7ZJ32lV7IXgSPonIWRoAZghbDFSXBz3ZLNWwl7?= =?us-ascii?Q?ZpN4fas45dDTA0YAJ0xMHaKjv1cNFB+RrRNznhxuoX+XSv8XmWPFDBcJO5cp?= =?us-ascii?Q?h+SIsq0pFskejHNld21OFmZiMJYRkQl9Ol7CX44oKFf8XcEXXoaGADcNYh5C?= =?us-ascii?Q?/PxuolfnSifRNLSbgOR46PijFtssHxJ6olAeu4PwpETqXDynegLlt36Z1uqN?= =?us-ascii?Q?IsDTEV8KTDQlm33tIh8UAgBXkzUX0Lc+ROmiZdRENlAJ2rpQCnWVOGolQxL3?= =?us-ascii?Q?LwdZuEGM0P5QLvuwogVnIIAKFNi4uZTX4rC52lw5PWE3DIbT4VcfH56duvP7?= =?us-ascii?Q?QfiHvX8pZ+NLOaKXH9T6dvG5QTxnKpUHGfew7znJk2bVNCXU4HICYo7/7LGd?= =?us-ascii?Q?MCu4cqhjE4wPLCgh8syfhFIUwnclv/jaeGBhaWq9QLq9wYyxwZmf6eW1WrUU?= =?us-ascii?Q?BN/cQpfpfjqrm43qV5OIfcN1FaJP/1sSD6voV2OT3VZZ5Bd54t3Vv8ePqOqi?= =?us-ascii?Q?QTb1g0B+Urvoiw01URqQMVY7jyNZ/0I+4kZ5f1anoK6kwI9lE86coX0XSxdf?= =?us-ascii?Q?shvPR6YV3PlifCOb34kzspOr/aYFSWME4d5B5b07KJewZ5H2yRaZp6+9Wf3j?= =?us-ascii?Q?l3yPRhk78gEzDs+4voUKdAuuwBtvS/Z1Irrj+m0XWkAHTIG0AapA5/VpMvm/?= =?us-ascii?Q?wKgy/Pgc3s07Z1os8YoSI469BxR0nodhH0xNG7Xiz79ZABr2LeSZvGuU4xLj?= =?us-ascii?Q?EdEBS0hT9YpnIqDGH4H8sNcvmHmj8nUJ?= X-Forefront-Antispam-Report: CIP:216.228.117.161; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge2.nvidia.com; CAT:NONE; SFS:(13230040)(376014)(1800799024)(36860700013)(82310400026); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Sep 2024 09:22:41.3236 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 9087c94a-082b-4e23-a69c-08dcd7c37241 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.161]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN1PEPF0000468E.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB6568 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Hi all, We have been working on optimizing the latency of calls to rte_eth_dev_start(), on ports spawned by mlx5 PMD. Most of the work requires changes in the implementation of .dev_start() PMD callback, but I also wanted to start a discussion regarding configuration restore. rte_eth_dev_start() does a few things on top of calling .dev_start() callback: - Before calling it: - eth_dev_mac_restore() - if device supports RTE_ETH_DEV_NOLIVE_MAC_ADDR; - After calling it: - eth_dev_mac_restore() - if device does not support RTE_ETH_DEV_NOLIVE_MAC_ADDR; - restore promiscuous config - restore all multicast config eth_dev_mac_restore() iterates over all known MAC addresses - stored in rte_eth_dev_data.mac_addrs array - and calls .mac_addr_set() and .mac_addr_add() callbacks to apply these MAC addresses. Promiscuous config restore checks if promiscuous mode is enabled or not, and calls .promiscuous_enable() or .promiscuous_disable() callback. All multicast config restore checks if all multicast mode is enabled or not, and calls .allmulticast_enable() or .allmulticast_disable() callback. Callbacks are called directly in all of these cases, to bypass the checks for applying the same configuration, which exist in relevant APIs. Checks are bypassed to force drivers to reapply the configuration. Let's consider what happens in the following sequence of API calls. 1. rte_eth_dev_configure() 2. rte_eth_tx_queue_setup() 3. rte_eth_rx_queue_setup() 4. rte_eth_promiscuous_enable() - Call dev->dev_ops->promiscuous_enable() - Stores promiscuous state in dev->data->promiscuous 5. rte_eth_allmulticast_enable() - Call dev->dev_ops->allmulticast_enable() - Stores allmulticast state in dev->data->allmulticast 6. rte_eth_dev_start() - Call dev->dev_ops->dev_start() - Call dev->dev_ops->mac_addr_set() - apply default MAC address - Call dev->dev_ops->promiscuous_enable() - Call dev->dev_ops->allmulticast_enable() Even though all configuration is available in dev->data after step 5, library forces reapplying this configuration in step 6. In mlx5 PMD case all relevant callbacks require communication with the kernel driver, to configure the device (mlx5 PMD must create/destroy new kernel flow rules and/or change netdev config). mlx5 PMD handles applying all configuration in .dev_start(), so the following forced callbacks force additional communication with the kernel. The same configuration is applied multiple times. As an optimization, mlx5 PMD could check if a given configuration was applied, but this would duplicate the functionality of the library (for example rte_eth_promiscuous_enable() does not call the driver if dev->data->promiscuous is set). Question: Since all of the configuration is available before .dev_start() callback is called, why ethdev library does not expect .dev_start() to take this configuration into account? In other words, why library has to reapply the configuration? I could not find any particular reason why configuration restore exists as part of the process (it was in the initial DPDK commit). The patches included in this RFC, propose a mechanism which would help with managing which drivers rely on forceful configuration restore. Drivers could advertise if forceful configuration restore is needed through `RTE_ETH_DEV_*_FORCE_RESTORE` device flag. If this flag is set, then the driver in question requires ethdev to forcefully restore configuration. This way, if we would conclude that it makes sense for .dev_start() to handle all starting configuration aspects, we could track which drivers still rely on configuration restore. Dariusz Sosnowski (4): ethdev: rework config restore ethdev: omit promiscuous config restore if not required ethdev: omit all multicast config restore if not required ethdev: omit MAC address restore if not required lib/ethdev/rte_ethdev.c | 39 ++++++++++++++++++++++++++++++++++----- lib/ethdev/rte_ethdev.h | 18 ++++++++++++++++++ 2 files changed, 52 insertions(+), 5 deletions(-) -- 2.39.5