From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0721EA0C41; Thu, 4 Nov 2021 15:02:36 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 6491242732; Thu, 4 Nov 2021 15:02:29 +0100 (CET) Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2062.outbound.protection.outlook.com [40.107.237.62]) by mails.dpdk.org (Postfix) with ESMTP id 8D27A42732 for ; Thu, 4 Nov 2021 15:02:27 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=fIRmhqScE8Dx+W03oVHSWDA4EweNqekaipMTcdSIO/NVF721AcjmOrFL/pXRB3pkRaF1HdDroUBEUFVtDwnwYcnNTJj/+fMoXMrmL2AQuq6E547yCl0SqfTPWxic5TIus8dgllAPIjE5vhARHQpMw11Ig+0hkqTYXOs5jUp8DlpUhA6Vo11uwHOq2ArHJ6XwUqOYud5h6ZsxUf6NMh7SUW7/PiZoK7flF9G7bwu88xh+1N+uQUDOmaOj+N8imliWN57syjHT7ZbkTs8sKJhD6sozjAJwn0vsenBd11KtKV5/2q7wRAK+/OhsnOF1n9BS1Q1Ho2Nf0+lNZjHiaiWiRg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=TdsbF58IENxXCu7UM6ucgLX3E3Ee21gpPjcz01HahrM=; b=MSwptMAFdpJXHvnbs2QIOhS9lp6OMirIJYE1xlDV5Vl3w2H6wfaeCo05TXAjP7DcsKEc8Jy0EJIiJE/ysBoq7GeHs/iImbuXr+6wiTAXW+GMgx3RCY+HVH5ejTKswyxoMpyB4Rn85jWNZz0Q6iEK0YT21Jqkgm/m97hjzyzlb8+r1L/ZObrwxhdb0mXw6uUvyGUv3KnFRQzkxyErlxI6Y5T/VHUnjf//Jaav87RxftmykZIidVGttDdbpKtNxZan84KQ9fnaNE8yfShB8sexl6eFPOwtjbStVm5jRrTT7QNu50q3qWXU5Vy9YPDU0v2e1QK6WncbvrBXKDPy/gKKJw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=monjalon.net smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=TdsbF58IENxXCu7UM6ucgLX3E3Ee21gpPjcz01HahrM=; b=g3uDgdU5n3prLOnKB2aLv3Da3/WCiWinGl05xnw+urB/hj+S0OInfl5TtSD04uFt/GLZtiW7eGbmJOvhFQOCjcYqVBfXMQ+b+T5kJJTKp4cm344ZuSOu/NFoen9ht/wCwrlSmQSxW7/lPxxHJ5l+oMyRm1y+xvvsxtQcof6uOSLP0seUmx5OQ+5s4967XIldbDy4926SFAT8L7M5bMWuWriLXunoqvRVvZw65DhvelDnpobHjkZu8ffe2d9aiIVv9ZtOuVeeGOiKZqbt3IBRIxTxLYNLjlBjuxH50P+sQ7SeE0gvMFplymlLg2/5lgT1n10HM65tnd7ol+D4NyQXAA== Received: from BN8PR04CA0006.namprd04.prod.outlook.com (2603:10b6:408:70::19) by DM5PR12MB1705.namprd12.prod.outlook.com (2603:10b6:3:10c::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4649.17; Thu, 4 Nov 2021 14:02:25 +0000 Received: from BN8NAM11FT032.eop-nam11.prod.protection.outlook.com (2603:10b6:408:70:cafe::1c) by BN8PR04CA0006.outlook.office365.com (2603:10b6:408:70::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4669.11 via Frontend Transport; Thu, 4 Nov 2021 14:02:25 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; monjalon.net; dkim=none (message not signed) header.d=none;monjalon.net; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by BN8NAM11FT032.mail.protection.outlook.com (10.13.177.88) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4669.10 via Frontend Transport; Thu, 4 Nov 2021 14:02:24 +0000 Received: from nvidia.com (172.20.187.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Thu, 4 Nov 2021 14:02:20 +0000 From: Bing Zhao To: , CC: , , , Date: Thu, 4 Nov 2021 16:01:54 +0200 Message-ID: <20211104140154.51122-3-bingz@nvidia.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20211104140154.51122-1-bingz@nvidia.com> References: <20211104112644.17278-1-bingz@nvidia.com> <20211104140154.51122-1-bingz@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL107.nvidia.com (172.20.187.13) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 4f8e6b37-1f14-4c7b-250a-08d99f9bbac2 X-MS-TrafficTypeDiagnostic: DM5PR12MB1705: X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:1360; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: VnuQhhKR4AzVRHrgLrKuRKM/kIFRUYVTMHdyxLBHSaXC8QkHqAPovuywKMPu51BlGxVJTnh1/0gsT+Pd/0mikh1TCIa72gAWFIEnDLC90C/LmuHSTe6HcKN6jecXpnuWDI+wcynACzqeybi/q001ME6FQqwiQrk18R5f/rE3jIN4NHWb/3qPXYSlza3DYnIPhR62pvgqGKoTjD47oX+7I3ZeOBk0MBw0rPECOgj8Z1nb8xK/Io2waz7vR+JC8Uq3tPYr+MHinedy2kPMMhRbZ2+DeSusCwWr3QW3smpaFAfPTQ8v5+7HJE2laDwiOYHWxZqUxZlIWd037OZpOccGD1T+fzIiynppdOGPNnQHWSrhNXStgDRJZb5B+zizs+zd+subkGkjjI5ygSTbyJN/qtFdBiJAEH46Z/BK8QbuBGoHk12x95Mkk6+cTi+umWQtAlIUwWuXrH+69XRUmUXH8eLsfG1MVPTRmm8rHPqV07pyclrmj5bktCUPtNGwCS2spT1+m6vvx0wd/MKUyRO2TwVWJVG561AFKuo40gM2t4Ooeos0aCM7B0vkiwkMDTKS6hbGr3FUMS9XsR2cGZBrvRcCOC5hzN6zqcvLkY+rTeJbT+I8mv7iNwN15Ikeuq6nFmujs7XjWuJySPn2hKfaQwZNf9qFI14b6EtmMgFHKcbwU8TQUUCeGoIQp8Vdv8JOR830bkUikHDUttOlG5wRhg== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(36840700001)(46966006)(16526019)(36860700001)(6666004)(186003)(2906002)(6636002)(2616005)(36756003)(508600001)(86362001)(70206006)(70586007)(5660300002)(426003)(336012)(6286002)(8676002)(47076005)(7636003)(4326008)(107886003)(316002)(1076003)(26005)(55016002)(7696005)(356005)(82310400003)(54906003)(8936002)(83380400001)(110136005); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Nov 2021 14:02:24.8210 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 4f8e6b37-1f14-4c7b-250a-08d99f9bbac2 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT032.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR12MB1705 Subject: [dpdk-dev] [PATCH v2 2/2] net/mlx5: check delay drop settings in kernel driver X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The delay drop is the common feature managed on per device basis and the kernel driver is responsible one for the initialization and rearming. By default, the timeout value is set to activate the delay drop when the driver is loaded. A private flag "dropless_rq" is used to control the rearming. Only when it is on, the rearming will be handled once received a timeout event. Or else, the delay drop will be deactivated after the first timeout occurs and all the Rx queues won't have this feature. The PMD is trying to query this flag and warn the application when some queues are created with delay drop but the flag is off. The documents are also updated in this commit. Signed-off-by: Bing Zhao --- doc/guides/nics/mlx5.rst | 26 +++++ doc/guides/rel_notes/release_21_11.rst | 1 + drivers/net/mlx5/linux/mlx5_ethdev_os.c | 114 ++++++++++++++++++++++ drivers/net/mlx5/mlx5.h | 1 + drivers/net/mlx5/mlx5_trigger.c | 18 ++++ drivers/net/mlx5/windows/mlx5_ethdev_os.c | 17 ++++ 6 files changed, 177 insertions(+) diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst index bb92520dff..2874a34cb6 100644 --- a/doc/guides/nics/mlx5.rst +++ b/doc/guides/nics/mlx5.rst @@ -1123,6 +1123,27 @@ Driver options By default, the PMD will set this value to 1. +- ``delay_drop_en`` parameter [int] + + Bitmask value for the Rx queue delay drop attribute. Bit 0 is used for standard + Rx queue and bit 1 is used for hairpin Rx queue. + By default, the delay drop will be enabled for all hairpin Rx queues (if any) + and disabled for all standard Rx queues. It will be ignored if the NIC does + not support the attribute. + A timeout value is set in the driver to control the waiting time before dropping + a packet when there is no WQE available on a delay drop Rx queue. Once the timer + is expired, the delay drop will be deactivated for all queues. To re-activeate it, + a rearming is needed and now it is part of the kernel driver. + + To enable / disable the delay drop rearming, the private flag ``dropless_rq`` can + be set and queried via ethtool: + + - ethtool --set-priv-flags dropless_rq on (/ off) + - ethtool --show-priv-flags + + The configuration flag is global per PF and can only be set on the PF, once it is on, + all the VFs', SFs' and representors' Rx queues will share the timer and rearming. + .. _mlx5_firmware_config: Firmware configuration @@ -1797,6 +1818,11 @@ Supported hardware offloads | | | | | rdma-core 35 | | | | | | ConnectX-6 Dx | +-----------------------+-----------------+-----------------+ + | Rxq Delay drop | | DPDK 21.11 | | DPDK 21.11 | + | | | OFED 5.5 | | OFED 5.5 | + | | | N/A | | N/A | + | | | ConnectX-5 | | ConnectX-5 | + +-----------------------+-----------------+-----------------+ .. table:: Minimal SW/HW versions for shared action offload :name: sact diff --git a/doc/guides/rel_notes/release_21_11.rst b/doc/guides/rel_notes/release_21_11.rst index 13d8330873..76d18aeb6b 100644 --- a/doc/guides/rel_notes/release_21_11.rst +++ b/doc/guides/rel_notes/release_21_11.rst @@ -191,6 +191,7 @@ New Features * Added implicit mempool registration to avoid data path hiccups (opt-out). * Added NIC offloads for the PMD on Windows (TSO, VLAN strip, CRC keep). * Added socket direct mode bonding support. + * Added delay drop support for Rx queue. * **Updated Solarflare network PMD.** diff --git a/drivers/net/mlx5/linux/mlx5_ethdev_os.c b/drivers/net/mlx5/linux/mlx5_ethdev_os.c index 9d0e491d0c..1dd2d74c77 100644 --- a/drivers/net/mlx5/linux/mlx5_ethdev_os.c +++ b/drivers/net/mlx5/linux/mlx5_ethdev_os.c @@ -1630,3 +1630,117 @@ mlx5_get_mac(struct rte_eth_dev *dev, uint8_t (*mac)[RTE_ETHER_ADDR_LEN]) memcpy(mac, request.ifr_hwaddr.sa_data, RTE_ETHER_ADDR_LEN); return 0; } + +/* + * Query dropless_rq private flag value provided by ETHTOOL. + * + * @param dev + * Pointer to Ethernet device. + * + * @return + * - 0 on success, flag is not set. + * - 1 on success, flag is set. + * - negative errno value otherwise and rte_errno is set. + */ +int mlx5_get_flag_dropless_rq(struct rte_eth_dev *dev) +{ + struct { + struct ethtool_sset_info hdr; + uint32_t buf[1]; + } sset_info; + struct ethtool_drvinfo drvinfo; + struct ifreq ifr; + struct ethtool_gstrings *strings = NULL; + struct ethtool_value flags; + const int32_t flag_len = sizeof(flags.data) * CHAR_BIT; + int32_t str_sz; + int32_t len; + int32_t i; + int ret; + + sset_info.hdr.cmd = ETHTOOL_GSSET_INFO; + sset_info.hdr.reserved = 0; + sset_info.hdr.sset_mask = 1ULL << ETH_SS_PRIV_FLAGS; + ifr.ifr_data = (caddr_t)&sset_info; + ret = mlx5_ifreq(dev, SIOCETHTOOL, &ifr); + if (!ret) { + const uint32_t *sset_lengths = sset_info.hdr.data; + + len = sset_info.hdr.sset_mask ? sset_lengths[0] : 0; + } else if (ret == -EOPNOTSUPP) { + drvinfo.cmd = ETHTOOL_GDRVINFO; + ifr.ifr_data = (caddr_t)&drvinfo; + ret = mlx5_ifreq(dev, SIOCETHTOOL, &ifr); + if (ret) { + DRV_LOG(WARNING, "port %u cannot get the driver info", + dev->data->port_id); + goto exit; + } + len = *(uint32_t *)((char *)&drvinfo + + offsetof(struct ethtool_drvinfo, n_priv_flags)); + } else { + DRV_LOG(WARNING, "port %u cannot get the sset info", + dev->data->port_id); + goto exit; + } + if (!len) { + DRV_LOG(WARNING, "port %u does not have private flag", + dev->data->port_id); + rte_errno = EOPNOTSUPP; + ret = -rte_errno; + goto exit; + } else if (len > flag_len) { + DRV_LOG(WARNING, "port %u maximal private flags number is %d", + dev->data->port_id, flag_len); + len = flag_len; + } + str_sz = ETH_GSTRING_LEN * len; + strings = (struct ethtool_gstrings *) + mlx5_malloc(0, str_sz + sizeof(struct ethtool_gstrings), 0, + SOCKET_ID_ANY); + if (!strings) { + DRV_LOG(WARNING, "port %u unable to allocate memory for" + " private flags", dev->data->port_id); + rte_errno = ENOMEM; + ret = -rte_errno; + goto exit; + } + strings->cmd = ETHTOOL_GSTRINGS; + strings->string_set = ETH_SS_PRIV_FLAGS; + strings->len = len; + ifr.ifr_data = (caddr_t)strings; + ret = mlx5_ifreq(dev, SIOCETHTOOL, &ifr); + if (ret) { + DRV_LOG(WARNING, "port %u unable to get private flags strings", + dev->data->port_id); + goto exit; + } + for (i = 0; i < len; i++) { + strings->data[(i + 1) * ETH_GSTRING_LEN - 1] = 0; + if (!strcmp((const char *)strings->data + i * ETH_GSTRING_LEN, + "dropless_rq")) + break; + } + if (i == len) { + DRV_LOG(WARNING, "port %u does not support dropless_rq", + dev->data->port_id); + rte_errno = EOPNOTSUPP; + ret = -rte_errno; + goto exit; + } + flags.cmd = ETHTOOL_GPFLAGS; + ifr.ifr_data = (caddr_t)&flags; + ret = mlx5_ifreq(dev, SIOCETHTOOL, &ifr); + if (ret) { + DRV_LOG(WARNING, "port %u unable to get private flags status", + dev->data->port_id); + goto exit; + } + if (!(flags.data & (1U << i))) + ret = 0; + else + ret = 1; +exit: + mlx5_free(strings); + return ret; +} diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 8d32d55c9a..e0f40ce31a 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -1599,6 +1599,7 @@ int mlx5_os_read_dev_stat(struct mlx5_priv *priv, int mlx5_os_read_dev_counters(struct rte_eth_dev *dev, uint64_t *stats); int mlx5_os_get_stats_n(struct rte_eth_dev *dev); void mlx5_os_stats_init(struct rte_eth_dev *dev); +int mlx5_get_flag_dropless_rq(struct rte_eth_dev *dev); /* mlx5_mac.c */ diff --git a/drivers/net/mlx5/mlx5_trigger.c b/drivers/net/mlx5/mlx5_trigger.c index ebeeae279e..34fcd2b441 100644 --- a/drivers/net/mlx5/mlx5_trigger.c +++ b/drivers/net/mlx5/mlx5_trigger.c @@ -1126,6 +1126,24 @@ mlx5_dev_start(struct rte_eth_dev *dev) dev->data->port_id, strerror(rte_errno)); goto error; } + if (priv->config.std_delay_drop || priv->config.hp_delay_drop) { + if (!priv->config.vf && !priv->config.sf && + !priv->representor) { + ret = mlx5_get_flag_dropless_rq(dev); + if (ret < 0) + DRV_LOG(WARNING, + "port %u cannot query dropless flag", + dev->data->port_id); + else if (!ret) + DRV_LOG(WARNING, + "port %u dropless_rq OFF, no rearming", + dev->data->port_id); + } else { + DRV_LOG(DEBUG, + "port %u doesn't support dropless_rq flag", + dev->data->port_id); + } + } ret = mlx5_rxq_start(dev); if (ret) { DRV_LOG(ERR, "port %u Rx queue allocation failed: %s", diff --git a/drivers/net/mlx5/windows/mlx5_ethdev_os.c b/drivers/net/mlx5/windows/mlx5_ethdev_os.c index fddc7a6b12..359f73df7c 100644 --- a/drivers/net/mlx5/windows/mlx5_ethdev_os.c +++ b/drivers/net/mlx5/windows/mlx5_ethdev_os.c @@ -389,3 +389,20 @@ mlx5_is_removed(struct rte_eth_dev *dev) return 1; return 0; } + +/* + * Query dropless_rq private flag value provided by ETHTOOL. + * + * @param dev + * Pointer to Ethernet device. + * + * @return + * - 0 on success, flag is not set. + * - 1 on success, flag is set. + * - negative errno value otherwise and rte_errno is set. + */ +int mlx5_get_flag_dropless_rq(struct rte_eth_dev *dev) +{ + RTE_SET_USED(dev); + return -ENOTSUP; +} -- 2.27.0