From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 179D3A09E4 for ; Fri, 29 Jan 2021 19:04:24 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id ECB2340683; Fri, 29 Jan 2021 19:04:23 +0100 (CET) Received: from hqnvemgate24.nvidia.com (hqnvemgate24.nvidia.com [216.228.121.143]) by mails.dpdk.org (Postfix) with ESMTP id 509674067D for ; Fri, 29 Jan 2021 19:04:20 +0100 (CET) Received: from hqpgpgate101.nvidia.com (Not Verified[216.228.121.13]) by hqnvemgate24.nvidia.com (using TLS: TLSv1.2, DES-CBC3-SHA) id ; Fri, 29 Jan 2021 10:04:19 -0800 Received: from HQMAIL107.nvidia.com ([172.20.161.6]) by hqpgpgate101.nvidia.com (PGP Universal service); Fri, 29 Jan 2021 10:04:19 -0800 X-PGP-Universal: processed; by hqpgpgate101.nvidia.com on Fri, 29 Jan 2021 10:04:19 -0800 Received: from HKMAIL104.nvidia.com (10.18.16.13) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Fri, 29 Jan 2021 18:04:18 +0000 Received: from HKMAIL103.nvidia.com (10.18.16.12) by HKMAIL104.nvidia.com (10.18.16.13) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Fri, 29 Jan 2021 18:04:17 +0000 Received: from NAM04-BN8-obe.outbound.protection.outlook.com (104.47.74.45) by HKMAIL103.nvidia.com (10.18.16.12) with Microsoft SMTP Server (TLS) id 15.0.1473.3 via Frontend Transport; Fri, 29 Jan 2021 18:04:16 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=OX9iMvHiZ7/wlthfl4FAR3StrcL7ALUh9EQbecsnbo5X7tX8R46g6yX0jup8BV539SR+lrgHw9V0whrR+JR5QR0JpRiApRXxUAvKSBucL1gkGcoKqU32cVToUCnu2IDhY1PoMn0sfcAomvUhIXrdzc8p4Atc0gCPKAd7+KQCNK45WnQi+qLRlVM99Ga0840GF/zlihmtB3UgSkvqmwNY1o1Y7p0pqtfgqgLuO06qieFklOTCywOmJ0X4ecpxJUfh6toDVd3SWsqTAmMFQElQUtA0MJ5kW/n2Gj389H2GYdYqX2piQz+IWOgbSzob04h2CI5qxMZ14fPl9j+MuX7ILw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3fmxb+9Ejt8rQnuK7LpTLKwK6ty+rMfh9WsI8pJl0fY=; b=KaobLByo4rYSuqUgE2MlTc41QjnonZPvtvcwbXbPyxyefQLrwbhX7AZD7t/U7s6Hr04TUfj8h4jnrq5IJkdzTnVG0wmxVskz5DfHGuI2DTs2bKCb5PrKvi+Tq+IDKEWotKZ5lKJGLf+4Od0eOnnl5R51kMKwQNBWKbDpzs7bS3P6qJVBTD5nXl5+z9kcV3FgXOBqZwVqmu5y3rlPkif8BfvI50caLQ1ji/Mh1ovKJzbSmbS44XblMOmgBoQ8JRSozuSXtX1kRHJF8R8fHgearT901wi3ML2G/M+X6vKky3l1Y8n2jK6/tytnzpNtmZwegJ3f+sjuQTMaXjrHenvtNw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none Received: from DM6PR12MB3753.namprd12.prod.outlook.com (2603:10b6:5:1c7::18) by DM6PR12MB3323.namprd12.prod.outlook.com (2603:10b6:5:11f::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3805.17; Fri, 29 Jan 2021 18:04:14 +0000 Received: from DM6PR12MB3753.namprd12.prod.outlook.com ([fe80::e4a9:f9a1:d873:d07a]) by DM6PR12MB3753.namprd12.prod.outlook.com ([fe80::e4a9:f9a1:d873:d07a%5]) with mapi id 15.20.3784.019; Fri, 29 Jan 2021 18:04:14 +0000 From: Slava Ovsiienko To: NBU-Contact-Thomas Monjalon , Carsten Andrich CC: "users@dpdk.org" Thread-Topic: [dpdk-users] MLX5: Using packet send scheduling / packet pacing Thread-Index: AQHW9l7zo2h+lQj/QUOv/XqXasceLqo+5JuQ Date: Fri, 29 Jan 2021 18:04:13 +0000 Message-ID: References: <43ddc3a8-a48f-0930-a2da-9e2fcdde1b98@tu-ilmenau.de> <1768623.GMc5ikO6cA@thomas> In-Reply-To: <1768623.GMc5ikO6cA@thomas> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: monjalon.net; dkim=none (message not signed) header.d=none;monjalon.net; dmarc=none action=none header.from=nvidia.com; x-originating-ip: [188.163.75.192] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 727c8ac0-26e3-4ffe-868f-08d8c4804974 x-ms-traffictypediagnostic: DM6PR12MB3323: x-ld-processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:9508; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: tFfC8rtF7LxWwxkoyHaqEZiGtMs1LReF+VbOvS8K9gXmefnG+/7Fu8Wyu69vF8e8MRzco0fDX0RtX0Waa6I4Ib0aEoT4k7iTqDUkGOPI41NgQLlXJPZ1r9yDHZDvzmSWc7QUAVjeszo5kNoGkvBG5fr430w4A6u4arOxf5WxcO1lHHYAlmQWhAwUBUL45eDMFN8FaWV8vL/HIOyJ8kKC2LbBJL1coaYldnSVwpwf+Lnu513W+l8vj36dYS/OWmUfJwFqk4R8g7SFuBPZRNlPC86wF7u0i2D0j2ERE9kRwSmbUPnalU3T9XZSFX+NUfQ8IZ2R4xwKfaXLaJxKFLWsZnE0Lf7tCOffkLjt+TExn4SePU8lIi/TwSeXa76TwtlsLHtXIz1R2mDZ25wdZiSd/P27iaiEDqV6mYX4q3a3qhFS++xyDSkMUTyw7flbrdzzpgvk5IT/SLxaPN4DncHsejq8RH/sGhbBlm7vmZUjzWKdHdSQsPkMAnQbRyYcP0V8D8K3XOsQ/YFeakZG+wJlBCWUKKh3IA1fglssUE8K7g3TR6B+MocFCXIKKRkQRgc7YVy35p9C6G2SdQthVu+8qgwf7BCfLWChvZGhrUgaoJM= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM6PR12MB3753.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(136003)(39860400002)(376002)(396003)(346002)(366004)(4326008)(52536014)(33656002)(186003)(26005)(86362001)(6506007)(2906002)(478600001)(71200400001)(76116006)(66446008)(8936002)(83380400001)(53546011)(55236004)(5660300002)(316002)(55016002)(66946007)(66476007)(110136005)(66556008)(7696005)(9686003)(8676002)(64756008)(966005); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: =?us-ascii?Q?9Er7oRX38RQiLmzh9znhgNQaKmqIlyaIPoJnxiJA0cPvH71SbF53uXjoEqUr?= =?us-ascii?Q?m8Ru0kMeeAI15h4cueCoLOy5CeyW5xoZugjUWXd+ZQ3ZoJf1/Z5LJD+REk1B?= =?us-ascii?Q?70utR97p1GNvHF345APdP3egJPjMH2oTOHFjpJGBidH+TXUUsOWbSJ8cWUbG?= =?us-ascii?Q?F4gdLKEuuHSwBqoPsk19kyFmWD51/D1hFWQ+iKkgawrjMxDJh7EJQedZzgqD?= =?us-ascii?Q?08zi61IGF/a1oRARt0Zlb5JcLENjRfXl+cL34+J8UHeYf0b3jJ8ayNCkMShl?= =?us-ascii?Q?zyJLW4z6kFFAc2+SUEPYSnsjUptj1yDJ7JZwvo+MMRB9X/scft7AQmT+BweW?= =?us-ascii?Q?o+vLr80Nnu9qfRFm7qd4MtcmdkDB4HsarPpHNDxIWoFjZRqiR184BlkFPSCh?= =?us-ascii?Q?8KiP5Ki6VIRwEKaqSYKNZvnk2ty4BQbe3iWGoCobWW77DnRaFyAxCI83lpQe?= =?us-ascii?Q?N2dctkfipdVLBQPA2r8bc+1+M/0geCQz4uHapOJByVzC/R+unNAujIqEtS+Z?= =?us-ascii?Q?hVmXsrjPwesGA4O6rXfbGdkXF8Mucq7X9hqtz+gTKwfpyo+KxcYOUG0kdWlW?= =?us-ascii?Q?K4VsEszL9DK69tQ1M7Tnb9PSB1zhmKZrqldLVQDAQt0Dg6GN/c27IbtmRyKq?= =?us-ascii?Q?7/zt3J/oaW+uIGQSSqDGzFqNUr3+NMUZEv1eICVoWUX/U5Y25cYb757Az0iV?= =?us-ascii?Q?Vbx7fA7EkAavLLlw28Xr1/q7d1mH6ALYt2IrTH+4u3844kuanshXda6EUFfx?= =?us-ascii?Q?B63wien6npqN1cCm2ghDIvycZTs7cydtbyMmiMfj8wyX8AxMCDShE+4cbVUM?= =?us-ascii?Q?601771XIVqAgr3NeGk3EKjuaCfY/fsjQTVGpGpKWLytRthQ3w4I8miXo9Gmf?= =?us-ascii?Q?QlWi8OEP4RQ8fXNgffjJTGE66+pkDlHXO+vJKIFD94pDifr24G4Jyb37hexp?= =?us-ascii?Q?wLcY7DKISwnagMxS7PTatFnUmG4pKjqCJjJlaizc7ipvuKiRtNw4AI+kBiIn?= =?us-ascii?Q?Cl8VS6pqJzBNzlP7lIK3WDoofIYVCmWIAA0k0qrCoNyB7tnVdrYb/fQKZ0ka?= =?us-ascii?Q?4VU3rnHK?= x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB3753.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 727c8ac0-26e3-4ffe-868f-08d8c4804974 X-MS-Exchange-CrossTenant-originalarrivaltime: 29 Jan 2021 18:04:13.8997 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 5XxYIxs7UfaVX+vcRr2fXfKh3xKkxyKqX4+nloNpxmqpIFMVUKgwtFqYHb1XQ8LnWPz2tLiPVY4bh3Lhd0Ocdw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB3323 X-OriginatorOrg: Nvidia.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nvidia.com; s=n1; t=1611943459; bh=3fmxb+9Ejt8rQnuK7LpTLKwK6ty+rMfh9WsI8pJl0fY=; h=X-PGP-Universal:ARC-Seal:ARC-Message-Signature: ARC-Authentication-Results:From:To:CC:Subject:Thread-Topic: Thread-Index:Date:Message-ID:References:In-Reply-To: Accept-Language:Content-Language:X-MS-Has-Attach: X-MS-TNEF-Correlator:authentication-results:x-originating-ip: x-ms-publictraffictype:x-ms-office365-filtering-correlation-id: x-ms-traffictypediagnostic:x-ld-processed: x-microsoft-antispam-prvs:x-ms-oob-tlc-oobclassifiers: x-ms-exchange-senderadcheck:x-microsoft-antispam: x-microsoft-antispam-message-info:x-forefront-antispam-report: x-ms-exchange-antispam-messagedata:x-ms-exchange-transport-forked: Content-Type:Content-Transfer-Encoding:MIME-Version: X-MS-Exchange-CrossTenant-AuthAs: X-MS-Exchange-CrossTenant-AuthSource: X-MS-Exchange-CrossTenant-Network-Message-Id: X-MS-Exchange-CrossTenant-originalarrivaltime: X-MS-Exchange-CrossTenant-fromentityheader: X-MS-Exchange-CrossTenant-id:X-MS-Exchange-CrossTenant-mailboxtype: X-MS-Exchange-CrossTenant-userprincipalname: X-MS-Exchange-Transport-CrossTenantHeadersStamped:X-OriginatorOrg; b=kNevZ8Dnd7vp1AfyA8KLLSOEh+rEbrOcmQrWYQSNyL6kf80aKgeIcSDuCZ9CJk0T0 oUyGk3RHs1/cLKzIoqiucr69SDYGQkV2WrHp2VfVIRaLg4sulJYqpA+7RNBoT25qX9 EQcTYc73xICJIpnrWlM6VqtYv15yFlgCQr0tErli2iNOKrmgKW1Y8UzfXgbXlXPNk5 /ysc9hUt/BFeyg0NubepBcNthpEH7tW23kNqQH+ZjpOkfjjGUbGcXyJueWamMMcO9U vxKu8lRFVhFU9NuzYPJZ0hnKvQz2u5v00WduLRvABWAmonT5wV6j/n0TdFpXd7StMW FI3J+DfZnWGDA== Subject: Re: [dpdk-users] MLX5: Using packet send scheduling / packet pacing X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org Sender: "users" Hi, Carsten To use packet pacing capability: - ConnectX-6DX or above is required - the NV settings should be configured (set timestamp to real-time UTC) With best regards, Slava > -----Original Message----- > From: Thomas Monjalon > Sent: Friday, January 29, 2021 18:50 > To: Carsten Andrich > Cc: users@dpdk.org; Slava Ovsiienko > Subject: Re: [dpdk-users] MLX5: Using packet send scheduling / packet pac= ing >=20 > 29/01/2021 17:30, Carsten Andrich: > > Hello everyone, > > > > I'm trying to use packet send scheduling [1] with DPDK 20.11 and the > > MLX5 PMD (NIC: ConnectX-5 MCX516A-CDAT). This patch contains some > > additional information on this feature also know as packet pacing [2]. > > > > According to MLX5's docs, packet pacing requires the "tx_pp" parameter > > [3, CTRL+F: "tx_pp"]. However, when firing up testpmd with that > > parameter, it fails as follows: > > > > > # dpdk-testpmd -a 81:00.0,tx_pp=3D500 -- -i ... > > > EAL: Probe PCI driver: mlx5_pci (15b3:1019) device: 0000:81:00.0 > > > (socket 0) > > > mlx5_pci: WQE rate mode is required for packet pacing > > > mlx5_pci: probe of PCI device 0000:81:00.0 aborted after > > > encountering an error: No such device > > > common_mlx5: Failed to load driver =3D mlx5_pci. > > > > > > EAL: Requested device 0000:81:00.0 cannot be used > > The error message originates here [4] and is caused by what to me > > appears to be a value read from the NIC [5]. Unfortunately, that > > leaves me clueless on how to activate the required "WQE rate mode". > > According to the output of ibv_devinfo, my NIC does support packet paci= ng: > > > > > # ibv_devinfo -v 81:00.0 > > > ... > > > packet_pacing_caps: > > > qp_rate_limit_min: 1kbps > > > qp_rate_limit_max: 100000000kbps > > > supported_qp: > > > SUPPORT_RAW_PACKET > > I'd be grateful for any information on how to get packet pacing up and > > running. Am I just missing another required option (which is not given > > in the docs) or does my NIC lack packet pacing support? > > > > Thank you very much in advance. > > > > Best regards, > > Carsten > > > > [1] > > > https://doc.dpdk.org/api/rte__ethdev_8h.html#a990d8351447a710628cbb24 > a > > 28d3252d [2] https://patches.dpdk.org/patch/73742/ > > [3] https://doc.dpdk.org/guides/nics/mlx5.html#run-time-configuration > > [4] > > http://code.dpdk.org/dpdk/v20.11/source/drivers/net/mlx5/linux/mlx5_os > > .c#L1278 > > [5] > > > http://code.dpdk.org/dpdk/v20.11/source/drivers/common/mlx5/mlx5_devx > _ > > cmds.c#L748 > > > > >=20 >=20 >=20 >=20