From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0C102A034C; Sun, 19 Jun 2022 10:14:13 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9F13040150; Sun, 19 Jun 2022 10:14:12 +0200 (CEST) Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2072.outbound.protection.outlook.com [40.107.220.72]) by mails.dpdk.org (Postfix) with ESMTP id C303B400EF for ; Sun, 19 Jun 2022 10:14:10 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=QpH4T/M2SRLL16YalYGnBqEtQWTYvZFzdtsELzo9gkjZltFqGbeHzLVVJjXjW9iDOLLnH1+SdO94vtopCtb2W2AS5ZcDadO9NkCdv6HZYcEdsMK3rfAqMRY788xdGBCbKl3vnnRkTjYGOzYJxSnennlrAYE1f/HFXUawHzNG2D6DvthHidcDlPur8DwSsT/LIhTdo3dEZEuLURkrMfvPvOaheL0FwoEsgI3PeJ0v1WHtntCC1hHBFP1fQgkApnkJ5eLk269RpiFCd4j8z9RnI6CAKI79d/VbQJw3SyX6e4aFBg7bCLK471681y45p5qx2ONQFCG9I4GjvlGY1jDhYg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=93bqOnmrpni7Y+E8cZiaZ7UBKcOB050gwxmMdPc1oHo=; b=hscpbIALguYthDZrjoSvZ7I6OjNdahHJamX0Raw4T9w7EarNK/IGInTVpUb9DgyEH+HITE5VKleWDdsZHuuTaS23SbspURNboesJjANxY5YJq79OaV4VRpyQQhT3Ok/gMdxQ0nXwXnuHm7MkYxcl4tUJBUCursy1cZzi3NJA3UKnHO/ywCL0FKsGKhmRpKvae50rCaLoFTxDDNR8savgbGcGPl26sF5LM6HHsP4C3NX2ZmA5ZcN2yAIpqTkyJohIsKYmWNymCjqSCn+5y3LbV9veKZuUX3Cu/TX55I2iH4kgcbByEdwmlmnovvWpaE7ifr8A1wGJ5LKLf+/ix/iRcA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=93bqOnmrpni7Y+E8cZiaZ7UBKcOB050gwxmMdPc1oHo=; b=KDdw+RKt58CwMZydAITUtiIT1T4oCCOYtTo9UDVMJ6EGJlwPCRZI3XyCE7X8TmHoMRmtUpHdUrd/jvRqJOHTBoWjpWTrbh3xEzD+kBk6AKzTuUgFvxtshtFirzjrWoLROJNRKczV3BWkR1LsY226/8S50S3KNhE0ekiVtV4ljbt1d+qrxitadTHT3x8Es171IsJLL4/6f9Z/eLvc4ICifFZcsXjjBEdnhlE7OTaHTfcpSNFIeNfw3g1ez73RRDkmP+zQ241qvQ9D3WR/kOOElQZZArbdi01tixaaS8emnRhgR+Evzan4EKg8iAtS9Npo7i7oeMFv82Caqf9qXM0rZg== Received: from BYAPR12MB3078.namprd12.prod.outlook.com (2603:10b6:a03:a8::31) by MN2PR12MB3584.namprd12.prod.outlook.com (2603:10b6:208:c9::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5353.19; Sun, 19 Jun 2022 08:14:04 +0000 Received: from BYAPR12MB3078.namprd12.prod.outlook.com ([fe80::bd2a:f22f:f435:7b36]) by BYAPR12MB3078.namprd12.prod.outlook.com ([fe80::bd2a:f22f:f435:7b36%5]) with mapi id 15.20.5353.020; Sun, 19 Jun 2022 08:14:03 +0000 From: Raslan Darawsheh To: Spike Du , Matan Azrad , Slava Ovsiienko , Ori Kam , "NBU-Contact-Thomas Monjalon (EXTERNAL)" CC: "andrew.rybchenko@oktetlabs.ru" , "stephen@networkplumber.org" , "mb@smartsharesystems.com" , "dev@dpdk.org" Subject: RE: [PATCH v10 0/6] introduce per-queue available descriptor threshold and host shaper Thread-Topic: [PATCH v10 0/6] introduce per-queue available descriptor threshold and host shaper Thread-Index: AQHYgV0Ed6QvaTFq9UG4uiRCLzCFi61WZjzQ Date: Sun, 19 Jun 2022 08:14:03 +0000 Message-ID: References: <20220615144341.399152-2-spiked@nvidia.com> <20220616084154.635508-1-spiked@nvidia.com> In-Reply-To: <20220616084154.635508-1-spiked@nvidia.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 472bf066-20eb-4083-7312-08da51cbac4e x-ms-traffictypediagnostic: MN2PR12MB3584:EE_ x-ld-processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr x-microsoft-antispam-prvs: x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: qTzh8hMEdpmVatyczKpSJD+MpkSuZp0Tih8j/HuqOk5vtTb0yzwQoqLWIUZPVIKWPWpyORSugl1ue0nxCo9G7KMqTm68Pl9waEuh425xFMpi1oF2UieuQOplPduJGGZUcora0Abw+1RTyqYVM4PwCBqlrdUjSqpoxRSgn0gVlHetUIcozHRaVac83MjyZ83YuwIJrTOCrsz3F5K00tei7vo/s5nqb6WOSFn7I/ZGbgND5QvNEDoby0PiquBHFDUddzbtWq72nMgieAcl/lSbAGPa5FXlthZ9hTeX+FsgJYpe8z0o6IWXIM6xvO1osZc8QjTkPlFomrWti0ydokW1vjL2vFWjYNrEcXX0qvE8QUDNO6Tkfx5tZ42loGpqgJ1WFXHyl4dob/dPB3x0kbGQGNu5rvIQ3CGQSvyODzAxwqNH3909RTUtTugHLmnBU/S1Byb2Le1rYFCv5zMigyxbIyF8Oe42d61QuUCSR7ZRxMnMMlFPjFkFvabVvxptsbcYSnyjUn4s2ZlvYFKGKk97u0wsChlVwBbZKqj66eCwVbZ/Gx7Qvv1EYInLLgvOx1Cu4A2jqYr5VjGCIW06FnfxHeImLSHKIYDFPXtx8qDS1afu+YDqZNIDC95gNe1iQitN81IebEKYnbMrm8MGSEpMbHmySGiAbUib6/SxyCHIEISgsPOnGW3t/kYegjVaxVLvoSx2q2DufiY4/oeztfi1kQN0F9ntOxSVReZlD1nPHEmK9bfdjFMSc3muwZOMrwwL x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BYAPR12MB3078.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230016)(4636009)(366004)(54906003)(52536014)(55016003)(110136005)(38070700005)(316002)(26005)(9686003)(38100700002)(8936002)(5660300002)(83380400001)(53546011)(6506007)(8676002)(66446008)(7696005)(66946007)(64756008)(66556008)(498600001)(66476007)(33656002)(86362001)(76116006)(4326008)(122000001)(2906002)(186003)(71200400001)(131093003); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?yRXpTywrfumOJne5HjL7YDnR7+sNpbuFii8ttY8Vw3GpG8RjKEHn2B+1pabd?= =?us-ascii?Q?ea753YBB9vxElu8GkclqDqWvZvHNb+u1DpXzagbToCR5q3gFVL3eNoclyLzg?= =?us-ascii?Q?cwaLKWXK8QbRRrgm/baJ1C3k7rcf+ppjDi357kUfGZ+UoYIv8nHaWGFXa1dp?= =?us-ascii?Q?LkvYJgz8mbNX+oOsDiUsakJbTqBRJJj6QtaBDRix7U77ygNJ2qbEUoOaJkEu?= =?us-ascii?Q?0QlWe/SVzLfa6Fwoh1/hwQZMXAl1Lb53piqRuwn80k/rGrzIAQ23Wijku1vO?= =?us-ascii?Q?1Oo8VYcPwoxdC7tc8vXhmIytvX1H21UnnE4Nhw15jn13Rh4ie4m1IwMhMWCX?= =?us-ascii?Q?hX004J6moYbXyK4AcC3klkQeyn6JGOoy1zm5tQTMalBqawoWoHZOlif1YL/Z?= =?us-ascii?Q?u0WS7vlFm3yOz+VhPxxfOl2gABYt9g6ATilRTnTpyjxaC5vPriJNb4Erbzuj?= =?us-ascii?Q?eMb9ABXTD7loimrQTqjJmfZ3qAVW/TtNSuT55Nqv1hobYWcFIchzar1APA5C?= =?us-ascii?Q?1jUO8SHJb7WmAmsBwA6pGkDaCu61MmKpkV3uHKi/WQS6T6a5v2oNdxMMiC+4?= =?us-ascii?Q?PcpV/AJzVzgmUNW8L6in8ssQRDOB6teNYHIIHyNr3R1DzEyD7NG+jZ5UQ/ja?= =?us-ascii?Q?ozGRopZTuclkbXmbHLQNyFlKGBXyhu1SNPQzEBjkO1RgQyOrv6HfzprR4c0x?= =?us-ascii?Q?hf0OtswDt7dfBx7hrPvrnPFiXjBY9nSMXpeGNyeY8Ognr3ziqdwMdlzgVffC?= =?us-ascii?Q?st2e6eVqt88OwXI3nWUh8TBPd6Vp8iyvs+yxrCnxch7GfQaCdpqYdDwe4Cn8?= =?us-ascii?Q?RaR1c82i78E5vXPx+9zX4JdtTAg1b4sr/mRzhUZuOTG8bRQpPjKjxqlS08o4?= =?us-ascii?Q?2RiZxdw4J8YjENQPwkHQLEdP4W4wDmN6CSy3tfz5uGDYs8idjR8GQsRzOC6E?= =?us-ascii?Q?lht1pjMe6W3ZwZlF5Fz2XQ9lzjGVq+s4U+9OjLRkfsVTaAu8W2w+OABQeKi6?= =?us-ascii?Q?i3MtkfAa+ToHaJ4MvGCw1rXc98d6wrJVXN5LY3r6ADLeEgRoyh5devwSokKI?= =?us-ascii?Q?YINtDkJC/+zPx3lNQ+9q5mVVw7Nqbl0wyvnmBwW6i98fYxPMsfQ17akhNYhY?= =?us-ascii?Q?1LrtZ51M8ErZpaVhY//icmTn6nzmgTLINJIxJfnib3ZaaJWtRYqgLNlT7wNd?= =?us-ascii?Q?lDC19v6YoZZag5JQfNsRqkINCad2BW+qq0544GCdmVmuIHXYrDdFWcWsgo6Z?= =?us-ascii?Q?jv9pRZW8N30WLLqijqxvmQv5VUixCDO0xn7C4U/LSDOcUSSF0Gd4bqmNXLmB?= =?us-ascii?Q?jr9cJBkNyNKYRBq/XZipQ08P1i0qUnySN2ldsVHtUzKeh/0+KIt3zHqOcslN?= =?us-ascii?Q?2LknD3TlZhX+hMjJufp0ClE+homtmhkVpcyXsAKrYTl/cDR37fAb7bdidEqA?= =?us-ascii?Q?UTiwpa177HHmcQxJ9OJT9wlAaXEQN99KlNP55gGyLUo6/K3Br1SUTze2xopJ?= =?us-ascii?Q?TiEFhMRYKBK1J87BEvCZ124MYcfnPicrsJ1CB56kIztyEaIXmAtzlqFT1Bu3?= =?us-ascii?Q?FzrP1f4PRlN7KeFQSyRtahuzcCUpBfk/VxhJIphJVCOgY1eWFRLzDip9+doD?= =?us-ascii?Q?FhlTE8UIl+9rjc8zbjRduxy7YQ42S9aqMfCFla4xUT8filFUmBNL8e/hf3dx?= =?us-ascii?Q?1GPh9vedWwF6XtArpWjwNKYtcsKH9SsAAigtNoyVQ+Uh5bfiCwAlnF9eh/ZG?= =?us-ascii?Q?cCMGEQScdQ=3D=3D?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: BYAPR12MB3078.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 472bf066-20eb-4083-7312-08da51cbac4e X-MS-Exchange-CrossTenant-originalarrivaltime: 19 Jun 2022 08:14:03.6754 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: cTGzdjFS8xWsj5HqxlEUhMCqO6jsxIj0H+9At5gowKK3rhhY0gOupOvxxcADsQJwh9IquXsPTQP99AC6/hPJ9A== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB3584 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Hi, > -----Original Message----- > From: Spike Du > Sent: Thursday, June 16, 2022 11:42 AM > To: Matan Azrad ; Slava Ovsiienko > ; Ori Kam ; NBU-Contact- > Thomas Monjalon (EXTERNAL) > Cc: andrew.rybchenko@oktetlabs.ru; stephen@networkplumber.org; > mb@smartsharesystems.com; dev@dpdk.org; Raslan Darawsheh > > Subject: [PATCH v10 0/6] introduce per-queue available descriptor thresho= ld > and host shaper >=20 > available descriptor threshold(ADT for short) is per RX queue attribute, = when > RX queue available descriptors for HW is below ADT, HW sends an event to > application. > Host shaper can configure shaper rate and avail_thresh-triggered for a ho= st > port. > The shaper limits the rate of traffic from host port to embedded ARM rx p= ort > on Nvidia BlueField 2 NIC. > If avail_thresh-triggered is enabled, a 100Mbps shaper is enabled > automatically when one of the host port's Rx queues receives available > descriptor threshold event. >=20 > These two features can combine to control traffic from host port to wire = port > for BlueField 2 NIC. > The traffic flows from host to embedded ARM, then to the physical port. > The work flow is on the ARM system, configure available descriptor thresh= old > to RX queue and enable avail_thresh-triggered flag in host shaper, after > receiving available descriptor threshold event, delay a while until RX qu= eue is > empty , then disable the shaper. We recycle this work flow to reduce RX > queue drops on ARM system. >=20 > Add new libethdev API to set available descriptor threshold, add rte even= t > RTE_ETH_EVENT_RX_AVAIL_THRESH to handle available descriptor threshold > event. For host shaper, because it doesn't align to existing DPDK framewo= rk > and is specific to Nvidia NIC, use PMD private API. >=20 > For integration with testpmd, put the private cmdline function and availa= ble > descriptor threshold event handler in mlx5 PMD directory by adding a new > file mlx5_testpmd.c. Follow David Marchand's driver specific commands > framework to add mlx5 specific commands. >=20 > Spike Du (6): > net/mlx5: add LWM support for Rxq > common/mlx5: share interrupt management > net/mlx5: add LWM event handling support > net/mlx5: support Rx queue based available descriptor threshold > net/mlx5: add private API to config host port shaper > app/testpmd: add Host Shaper command >=20 > app/test-pmd/testpmd.c | 7 + > doc/guides/nics/mlx5.rst | 93 +++++++++ > doc/guides/rel_notes/release_22_07.rst | 2 + > drivers/common/mlx5/linux/meson.build | 13 ++ > drivers/common/mlx5/linux/mlx5_common_os.c | 131 ++++++++++++ > drivers/common/mlx5/linux/mlx5_common_os.h | 11 + > drivers/common/mlx5/mlx5_prm.h | 26 +++ > drivers/common/mlx5/version.map | 2 + > drivers/common/mlx5/windows/mlx5_common_os.h | 24 +++ > drivers/net/mlx5/linux/mlx5_ethdev_os.c | 71 ------- > drivers/net/mlx5/linux/mlx5_os.c | 132 +++--------- > drivers/net/mlx5/linux/mlx5_socket.c | 53 +---- > drivers/net/mlx5/meson.build | 3 + > drivers/net/mlx5/mlx5.c | 68 +++++++ > drivers/net/mlx5/mlx5.h | 12 +- > drivers/net/mlx5/mlx5_devx.c | 60 +++++- > drivers/net/mlx5/mlx5_devx.h | 1 + > drivers/net/mlx5/mlx5_rx.c | 288 > +++++++++++++++++++++++++++ > drivers/net/mlx5/mlx5_rx.h | 13 ++ > drivers/net/mlx5/mlx5_testpmd.c | 205 +++++++++++++++++++ > drivers/net/mlx5/mlx5_testpmd.h | 26 +++ > drivers/net/mlx5/mlx5_txpp.c | 28 +-- > drivers/net/mlx5/rte_pmd_mlx5.h | 30 +++ > drivers/net/mlx5/version.map | 2 + > drivers/net/mlx5/windows/mlx5_ethdev_os.c | 22 -- > drivers/vdpa/mlx5/mlx5_vdpa_virtq.c | 48 +---- > 26 files changed, 1063 insertions(+), 308 deletions(-) create mode 1006= 44 > drivers/net/mlx5/mlx5_testpmd.c create mode 100644 > drivers/net/mlx5/mlx5_testpmd.h >=20 > -- > 1.8.3.1 Series applied to next-net-mlx, Kindest regards, Raslan Darawsheh