From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124])
	by inbox.dpdk.org (Postfix) with ESMTP id F0470A054D;
	Tue,  7 Jun 2022 15:00:04 +0200 (CEST)
Received: from [217.70.189.124] (localhost [127.0.0.1])
	by mails.dpdk.org (Postfix) with ESMTP id CB5C74021D;
	Tue,  7 Jun 2022 15:00:04 +0200 (CEST)
Received: from NAM02-BN1-obe.outbound.protection.outlook.com
 (mail-bn1nam07on2059.outbound.protection.outlook.com [40.107.212.59])
 by mails.dpdk.org (Postfix) with ESMTP id CB5D040156
 for <dev@dpdk.org>; Tue,  7 Jun 2022 15:00:03 +0200 (CEST)
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=fFtLSJ3ptiy5lEjjFNOUyHOwMpd6NQQ3OXRBPKfuO27/OWILjgFrnI8KmhL2u9vTDukKD79KOAUjHzugwMpukE67vrV3d0sJc5Ex16UEGF712liruZqSu0EZdQ4vHMGQ3XtJNwhWKXozceZRdZuwEqHUlYVDVbLDV6dQ1gQ7chHg/xvVTHqcQOMEGZItO3jCmuRHGBiFuaprZAMNBdGVgiaIxHH0RIrbdMFKGASnBm7j/eCDnuaWndU1q5jGgjKgiUN6KJI4mMeiiRioSRcZuKV4D9SCdaCEjpqvT8EFzY7PihdBDUH1P2mTn1Z8NEzSEB/ar4/aFIz7CK+eV8ubKw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; 
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=O2n3jpebL6GKOodTetw6H7tvMn3EgpXdhS0boLwOHa0=;
 b=BGxp1dQLnPwDH6RuvGuZ6h5hXQ22fv3sLMjbeEZAnODt0sw1uhkgBr4D1B6N+rZ5jKzEp8WjCX9G0y8+dpO+sq3KXXEdZkNj8zjKy0yiOXe2SDMcRzUQX0QWn2fSe6g3iQrg1EEr9ZRbKCOkGfemgq0as1HmLHtq9Cs3ZP8LHkVNScLivMwjL7CoJ+8ud4rSQ7KmWKMQg9gLna9CGhQl8kHGwx6I14RhcX48dtwcSQJBL2wY4sZiD7qzehsDMCqsh/nrLL8GUHcddyaDjE8ceRHN8E/QRhLwLO8KIiZDtdSxC18dtx9AjNN64TpDQmroska09TcbtHayHx/OYNT8Tw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 12.22.5.235) smtp.rcpttodomain=smartsharesystems.com
 smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none
 header.from=nvidia.com; dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com;
 s=selector2;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=O2n3jpebL6GKOodTetw6H7tvMn3EgpXdhS0boLwOHa0=;
 b=QUk+r2kWwMQAwbDea05hMuE0jkQcgqIe/5iU0J3S0PDRy49UpfIN5LgvNMwAlFiyjEawcGN4t5xYdPVAIIpA9RwMyf4HMpsUc52a65qstX6BOl+wNHyU6bxPW7yr9JLUGNr4eDlXo/7rvZTPBEXMpGBBhUGa6JPRSenUkVPjzBRfyb6vzantQc5j76fjathXVXIbJ75eGvwu5bBv/+WFXSA9tBR89CGC0Nj2qaEHCmLkrVBi1UAJLTiKzaPc3B3pq4D/O9Kk8+DCNS6tq+uiXl/uNwer78JuCeURkns+gMk7g/eA3/XWyVXVWs1RFPX34SNkp9CTUsxuNOP+kZAcUQ==
Received: from MWHPR14CA0006.namprd14.prod.outlook.com (2603:10b6:300:ae::16)
 by CY4PR12MB1784.namprd12.prod.outlook.com (2603:10b6:903:11e::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5314.13; Tue, 7 Jun
 2022 13:00:02 +0000
Received: from CO1NAM11FT024.eop-nam11.prod.protection.outlook.com
 (2603:10b6:300:ae:cafe::ba) by MWHPR14CA0006.outlook.office365.com
 (2603:10b6:300:ae::16) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5314.12 via Frontend
 Transport; Tue, 7 Jun 2022 13:00:02 +0000
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.235)
 smtp.mailfrom=nvidia.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=nvidia.com;
Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates
 12.22.5.235 as permitted sender) receiver=protection.outlook.com;
 client-ip=12.22.5.235; helo=mail.nvidia.com; pr=C
Received: from mail.nvidia.com (12.22.5.235) by
 CO1NAM11FT024.mail.protection.outlook.com (10.13.174.162) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id
 15.20.5314.12 via Frontend Transport; Tue, 7 Jun 2022 13:00:00 +0000
Received: from rnnvmail201.nvidia.com (10.129.68.8) by DRHQMAIL107.nvidia.com
 (10.27.9.16) with Microsoft SMTP Server (TLS) id 15.0.1497.32;
 Tue, 7 Jun 2022 13:00:00 +0000
Received: from nvidia.com (10.126.231.35) by rnnvmail201.nvidia.com
 (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Tue, 7 Jun 2022
 05:59:57 -0700
From: Spike Du <spiked@nvidia.com>
To: <matan@nvidia.com>, <viacheslavo@nvidia.com>, <orika@nvidia.com>,
 <thomas@monjalon.net>
CC: <andrew.rybchenko@oktetlabs.ru>, <stephen@networkplumber.org>,
 <mb@smartsharesystems.com>, <dev@dpdk.org>, <rasland@nvidia.com>
Subject: [PATCH v5 0/7] introduce per-queue available descriptor threshold and
 host shaper
Date: Tue, 7 Jun 2022 15:59:35 +0300
Message-ID: <20220607125942.241379-1-spiked@nvidia.com>
X-Mailer: git-send-email 2.27.0
In-Reply-To: <20220603124821.1148119-1-spiked@nvidia.com>
References: <20220603124821.1148119-1-spiked@nvidia.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-Originating-IP: [10.126.231.35]
X-ClientProxiedBy: rnnvmail202.nvidia.com (10.129.68.7) To
 rnnvmail201.nvidia.com (10.129.68.8)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: f8272b94-0445-461d-960e-08da4885a1ee
X-MS-TrafficTypeDiagnostic: CY4PR12MB1784:EE_
X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr
X-Microsoft-Antispam-PRVS: <CY4PR12MB1784565EBF56CEFEAE3A3C74A8A59@CY4PR12MB1784.namprd12.prod.outlook.com>
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: FjSA4cbqG1vidtUfUS9LY5BrzI5VM6RP6LgpqP02qVz7Ke5cfp4ez5AkXGCIOAFNKNo8DCgDnX90g9uunExFtMGn8zZOygPuQWwmq5679OZL1LnG456WHMnZxgEOfRc0Hp19L6HsuG5nBlC0TVBNoaZQI2qbLvf9GutY2MMDI3f3d1rQJFnIW0KqUf4FA2w6JWdP2YT7Sas6sZkC1ECJmZj00yPiakYRLYEAF01JfvtqM7IsjQlyrw+47I3MkKmA6kHqOLXgBrw/ysUU3hqaU3/QHj2Eog3orWwGVg+5O6QrXGJGbLd7om/mWbTCawDqn84zWEkFWfQMbLmyKiXvOGqg2nJPLIT+D5P/Jwwekrzvvf6LrdQgZJHzWlPCWkk4p4vgw9wShcbRqHcXOo/2gvNJHgXdaTvG0RLyinOtITTY/g7Gu19oK76edS48DGj8t18HLd8yWvO2R0ltGeum+0FtzoXPxnNP5jm4t/st5URfqNTlLwN0FCQmum3g9t+zuaG/dRCIRCRpPR0bMxqLzJyu5LRLURa6yPKqxHKUhBBDU6NpdqC/jkqVPvUC+OAUJp5tDnoezwpumV8Q6tqaWCIN5UIFAeW6cnX9U9ihmA/W0vN8c4XstSaPlYjHZhUjOXLZFRzQ16O2dljCysVLYNjeBVXq2MXVLkf7zbNF3vzuK0asUj4S1YErq1ht9+mbmAdsv49jwUEIRg/5DcZIDrKMOnNF4b8adE2i+GiW2LqBNVebFbFhHmb9vcpWok3I
X-Forefront-Antispam-Report: CIP:12.22.5.235; CTRY:US; LANG:en; SCL:1; SRV:;
 IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:InfoNoRecords; CAT:NONE;
 SFS:(13230001)(4636009)(40470700004)(36840700001)(46966006)(508600001)(40460700003)(5660300002)(70586007)(4326008)(8676002)(70206006)(2906002)(36860700001)(82310400005)(81166007)(356005)(36756003)(107886003)(55016003)(8936002)(6666004)(2616005)(1076003)(186003)(16526019)(336012)(47076005)(26005)(426003)(6286002)(110136005)(83380400001)(7696005)(54906003)(86362001)(316002)(131093003)(36900700001);
 DIR:OUT; SFP:1101; 
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Jun 2022 13:00:00.9714 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: f8272b94-0445-461d-960e-08da4885a1ee
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[12.22.5.235];
 Helo=[mail.nvidia.com]
X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT024.eop-nam11.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR12MB1784
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org

available descriptor threshold(ADT for short) is per RX queue attribute, when RX queue available descriptors for HW is below ADT, HW sends an event to application.
Host shaper can configure shaper rate and avail_thresh-triggered for a host port.
The shaper limits the rate of traffic from host port to embedded ARM rx port on Nvidia BlueField 2 NIC.
If avail_thresh-triggered is enabled, a 100Mbps shaper is enabled automatically when one of the host port's Rx queues receives available descriptor threshold event.

These two features can combine to control traffic from host port to wire port for BlueField 2 NIC.
The traffic flows from host to embedded ARM, then to the physical port.
The work flow is on the ARM system, configure available descriptor threshold to RX queue and enable avail_thresh-triggered flag in host shaper, after receiving available descriptor threshold event, delay a while until RX queue is empty , then disable the shaper. We recycle this work flow to reduce RX queue drops on ARM system.

Add new libethdev API to set available descriptor threshold, add rte event RTE_ETH_EVENT_RX_AVAIL_THRESH to handle available descriptor threshold event. For host shaper, because it doesn't align to existing DPDK framework and is specific to Nvidia NIC, use PMD private API.

For integration with testpmd, put the private cmdline function and available descriptor threshold event handler in mlx5 PMD directory by adding a new file mlx5_testpmd.c. Follow David Marchand's driver specific commands framework to add mlx5 specific commands.


Spike Du (7):
  net/mlx5: add LWM support for Rxq
  common/mlx5: share interrupt management
  ethdev: introduce Rx queue based available descriptor threshold
  net/mlx5: add LWM event handling support
  net/mlx5: support Rx queue based available descriptor threshold
  net/mlx5: add private API to config host port shaper
  app/testpmd: add Host Shaper command

 app/test-pmd/cmdline.c                       |  68 +++++++
 app/test-pmd/config.c                        |  21 ++
 app/test-pmd/testpmd.c                       |  24 +++
 app/test-pmd/testpmd.h                       |   2 +
 doc/guides/nics/mlx5.rst                     |  93 +++++++++
 doc/guides/rel_notes/release_22_07.rst       |   2 +
 drivers/common/mlx5/linux/meson.build        |  13 ++
 drivers/common/mlx5/linux/mlx5_common_os.c   | 131 ++++++++++++
 drivers/common/mlx5/linux/mlx5_common_os.h   |  11 +
 drivers/common/mlx5/mlx5_prm.h               |  26 +++
 drivers/common/mlx5/version.map              |   2 +
 drivers/common/mlx5/windows/mlx5_common_os.h |  24 +++
 drivers/net/mlx5/linux/mlx5_ethdev_os.c      |  71 -------
 drivers/net/mlx5/linux/mlx5_os.c             | 132 +++---------
 drivers/net/mlx5/linux/mlx5_socket.c         |  53 +----
 drivers/net/mlx5/meson.build                 |   4 +
 drivers/net/mlx5/mlx5.c                      |  68 +++++++
 drivers/net/mlx5/mlx5.h                      |  12 +-
 drivers/net/mlx5/mlx5_devx.c                 |  60 +++++-
 drivers/net/mlx5/mlx5_devx.h                 |   1 +
 drivers/net/mlx5/mlx5_rx.c                   | 289 +++++++++++++++++++++++++++
 drivers/net/mlx5/mlx5_rx.h                   |  13 ++
 drivers/net/mlx5/mlx5_testpmd.c              | 201 +++++++++++++++++++
 drivers/net/mlx5/mlx5_testpmd.h              |  26 +++
 drivers/net/mlx5/mlx5_txpp.c                 |  28 +--
 drivers/net/mlx5/rte_pmd_mlx5.h              |  30 +++
 drivers/net/mlx5/version.map                 |   2 +
 drivers/net/mlx5/windows/mlx5_ethdev_os.c    |  22 --
 drivers/vdpa/mlx5/mlx5_vdpa_virtq.c          |  48 +----
 lib/ethdev/ethdev_driver.h                   |  22 ++
 lib/ethdev/rte_ethdev.c                      |  52 +++++
 lib/ethdev/rte_ethdev.h                      |  73 +++++++
 lib/ethdev/version.map                       |   2 +
 33 files changed, 1318 insertions(+), 308 deletions(-)
 create mode 100644 drivers/net/mlx5/mlx5_testpmd.c
 create mode 100644 drivers/net/mlx5/mlx5_testpmd.h

-- 
1.8.3.1