From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1A1B141CB3; Thu, 16 Feb 2023 18:58:23 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id F185042D65; Thu, 16 Feb 2023 18:58:22 +0100 (CET) Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2043.outbound.protection.outlook.com [40.107.237.43]) by mails.dpdk.org (Postfix) with ESMTP id 5A28842D1A for ; Thu, 16 Feb 2023 18:58:21 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jAj//731veHCKfBOFWi+HU8p1u84m6ToSs5Y57NBjg1jMK4wG14ce5Bn04RpgHFGd5o/l9TihGvslpCjRSU/6VL4A5owWCT/GTTMnInLeWEJazEnbbRiJ86EBdppjQzs22YCftdGhYtXn3C1Sfc51i8VYhxkaW5evbjajev9F06SwmUQsDVGSP5sbVoNpq/hwLqeMIYQ6cFcZVyS20346mNrU0skF+IWWDgInjtCOdeDrWe3qBreL3Oe0G3IxybQ6jMVop6w8fiTdYwgfQAa9qG/mnt5PRTQW7XKESfkJ/29bhtwQ2gCGZaKCHazNkXJM9uJ53PkoEDoHCc5OS80UQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=NojbW8qzbj9vzbNiQtaAZtBR5lvnshBf5/HPEgEq/w4=; b=n/k0MeyagErthsCepAjTfmw6Eu7FDBvU5nuVpsfjL5MpGuDGo1TZmtRTF/uKw0rASTu0RbXsWoPOyP/h9dMwkx46+5560UmhoZvddvxuBLpZZpbtgiFASYXFmkXdGAMQ5dFtlpOpce0AwcbR9WSyJVUErByaab1RqEOkiGoJT1Q7LHpsDUqZ6yAbIlRjdiHQoUYRJwdld7a0lD/o8UnTJKK7y3u4sdlknjDQXj+3i79JnkYOFsxF/SNfuuyx6NVOYceAX8WpWrYPZol4ei9GUcv/6AUXU0A5YXPIEgrXpPT++ozNSSeKNAse0P7g1MmwF9wd0r2hxj50/rtHkJ6o0g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=NojbW8qzbj9vzbNiQtaAZtBR5lvnshBf5/HPEgEq/w4=; b=q+BlCHf3nnKJLSaXD4kvxV0rGdQycuMylNq0sPxkQARldXlyTmRk85rzRpQ0t2VfbgvikiX9lmOV7TlHDH4zWWfyWkjncD3zfRKbbgQl0eDUZsue7wDmdJQzUwmRCJp9lgogRdNzCP+XgEqXQGRgr4A95oqlll8FSsX09L1wCoo= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=amd.com; Received: from CH2PR12MB4294.namprd12.prod.outlook.com (2603:10b6:610:a9::11) by DM4PR12MB5294.namprd12.prod.outlook.com (2603:10b6:5:39e::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6086.26; Thu, 16 Feb 2023 17:58:19 +0000 Received: from CH2PR12MB4294.namprd12.prod.outlook.com ([fe80::3614:22ed:ed5:5b48]) by CH2PR12MB4294.namprd12.prod.outlook.com ([fe80::3614:22ed:ed5:5b48%7]) with mapi id 15.20.6111.013; Thu, 16 Feb 2023 17:58:19 +0000 Message-ID: Date: Thu, 16 Feb 2023 17:58:11 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.7.2 Content-Language: en-US To: Jiawei Wang , viacheslavo@nvidia.com, orika@nvidia.com, thomas@monjalon.net, andrew.rybchenko@oktetlabs.ru, Aman Singh , Yuying Zhang Cc: dev@dpdk.org, rasland@nvidia.com References: <20230203050717.46914-1-jiaweiw@nvidia.com> <20230214154836.9681-1-jiaweiw@nvidia.com> <20230214154836.9681-2-jiaweiw@nvidia.com> From: Ferruh Yigit Subject: Re: [PATCH v5 1/2] ethdev: introduce the Tx map API for aggregated ports In-Reply-To: <20230214154836.9681-2-jiaweiw@nvidia.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-ClientProxiedBy: LO2P265CA0507.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:13b::14) To CH2PR12MB4294.namprd12.prod.outlook.com (2603:10b6:610:a9::11) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR12MB4294:EE_|DM4PR12MB5294:EE_ X-MS-Office365-Filtering-Correlation-Id: dd68e41e-2fb1-4efd-9d87-08db104762b1 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: VrX57z9Ky5VDOqjYXLaVz90suDGG8UsMMY+oSJQs22lURdl5Yyjg22VOOmNICjxqwBo9FHyTgdWCWWQmbwinZD5ahfC3AK1BYsp4eBeKVBWWUKdPB1zjFLbSYI6mRr+TdyQ1X+MDyr7kBWpZjN2PIpMeWB5Ko6hKC/Sj6uP4LUEOpRFg0VfrQQLQ4aPfywJ87GgCNmAjekUwgUZCDrFV0ckgmNrROWKxvUKbWryhS02T9GZJv1Ia38nQioGIgxPI9BMZ54lTzvcmVWcA2pmtwcYn6ausZ1BiiraG71JqXw8wbGAYuJ7hrpiACJkf8D93qxVjokihsBmdwSiymt/CiFVfghSmop+FNsy3jlpvxAsyNoRJ/tKWFSDacLTjbd5etR1MWc/9pAb/e7nR6DAGXQWM56+iJkyfV191vug4NCsJOoHn+Djc+TNc+k53YhTxJxbHbAv2jQbY9ml6yeFyhjxqt9+PZFcu8amjKWrx6UfZMhBkSwzMipCE5+Z0s9CF2ZpK7ZGEfDX87ZARrfpvUCciH06Wb06EEa5rCxsVpVCDci50Tvv1GVXTjm9BMZMyuScXgVQSfmHiFrRR1EiUOBL77A4wN0mjNQaSGjLQ4hTux6EqoKvklA09dg18zkVSFdBFczhrPJMI02ti0AK/Raooe2ad2mZyy/AzksNb+Ry9YQP8VgpbLluUvKkgbme8NuDWzQU5Sjo11j78M4LH9zRfEMdAyvuO8KRXTtY/z6c= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR12MB4294.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230025)(4636009)(376002)(366004)(136003)(346002)(39860400002)(396003)(451199018)(83380400001)(110136005)(478600001)(2616005)(36756003)(6486002)(6666004)(53546011)(26005)(6506007)(6512007)(186003)(44832011)(38100700002)(41300700001)(5660300002)(86362001)(8936002)(31696002)(31686004)(316002)(66476007)(8676002)(4326008)(66946007)(66556008)(2906002)(43740500002)(45980500001); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?ZnVXVGkrV0ZjNktUV2UrRlNzdWJKR3Q2S0x1eHZ5aDhacXgxem5zb1hNdlBi?= =?utf-8?B?TXBrNURFVnc4Z1kwYUxUcXl0bUNPQTJ5UnBqL1dWWllhWm1TTzd6azhBdUFs?= =?utf-8?B?NngzMUJWM0pSa3BtV0xKMjA5OFQzVk5FQUZ6WnpoRmUyL1lCM3loaVZvbE9Z?= =?utf-8?B?ZXptelBscDZKVnBZMFVRWDhKb2lGU2FXZFVnRnRhZzNFTGRjUU5zSU1jTUxS?= =?utf-8?B?OUV5YlpzQkNRenhUV2Y4WGlRMnJEMnVyaWxzVGZWaWt5ZXdUeTJFWXgxbVFS?= =?utf-8?B?WmsvTUtlclgrWlpFVjIrRjVEZmtCSHVEUWFzY01yMmV3Z0IvY1l0VGVYQndY?= =?utf-8?B?ZUFxZGlzMHliSk5TejVrWFM0ZTdaWkloM3RiUnlGU2UzQ3M3bm5CN0xLTm9x?= =?utf-8?B?eTNRU3Z5RFloSDM0S0tTTy8wV01YdHowNy80NnRseUdzamp1WXFzblp1U2hl?= =?utf-8?B?bU1jdURDaThmVTNZQjZWUHV1dWxxNmdENERGeFU5NENkNGluRDUxR3Q3UHps?= =?utf-8?B?ZzBFZUNwRUNjSjF2Z1ZBSUEzcTNBZmlvRnZUVmVJbU1nOUJFN2U3bFJiakFU?= =?utf-8?B?QUR2N3hMV3UrU21mb2FYR0hqSGY3SG13M0VsNEVrTU9mQmZ0clhmbmRJa3Yz?= =?utf-8?B?ekx3UnprZnlmNFhNL2lDOFM1MWd4ZUMydElHYzlLb05JK3JZVElOWmxqV0Rz?= =?utf-8?B?RG1BNFF5cWZBMkxoV3dhK3VlNmJvcitMaWtLVDVZeGVLRWVvMjArOG1oaGJE?= =?utf-8?B?a0NndUUyMGE3eVlQUlAyRzNIWEdRZE1vdXc1TTJpdzVHR21mMU5teDg5d3Vk?= =?utf-8?B?YkJ0MEZRcXZwdXA0enlKT0hLQyt6aTU1Sk11SVA3eHdveE1aS01vRjh0TkJm?= =?utf-8?B?ZXFNVFR4N0xhQnJ4UE9zN1FPZVlUVjhmeExoZWdyREV6WEVURHNybG8zSmtr?= =?utf-8?B?cExjUkplcGdEVHhYODBGcFQvQXhDaldNbmxHc0dxNWVheU5IYWZWOHovbFg4?= =?utf-8?B?SWk0YkVYQjNPdlNVSzh2b20rVkpvS1ZyVDAyZE1kQmhoTUpVbWozRnJBUDFv?= =?utf-8?B?emMxS29HUURyd3h1RTdJVjJUTllObjY3YU9pNkFPKzFjWHU1QUVBSU5BT0lN?= =?utf-8?B?NS9nU3IydVIwNVFEUGZjVWFxY1pidWlPVWRHdjN1dTRFT2tHNUZDdXlpczFD?= =?utf-8?B?Ni84eHd4NCs1djdBVHJPbE5vZUVrbkpxMVZDcFA1Wk1Bd2xWQW9jSHlKT0lW?= =?utf-8?B?ZlJQSlJ3Y3hVaGIvdUxQcHZRU29kQ1V2OGM0bzFxQTI4MVFTUXhKUjlXNkRz?= =?utf-8?B?SlF3ZlhHYVRrNVdlNXlsWFdrWjhZQm5VUStGQVFDUWxLQ3lMdk9zaEo4Nzkx?= =?utf-8?B?amw0Q2VxSE9FSTBDZEJvWUFTZ3NnRWFjRTJOajlVT2hBTk1oNXVWbTNBejVa?= =?utf-8?B?WVBYTmZKMmdEcFJDUHFoOWgxTHNoUGVmMWxuZGVtZmNNcmlyb0pJeWYrZW8v?= =?utf-8?B?TWpGMEhoNVRLaFpqdlJxcTNhQzk1ZndFVVh5RFlOUXdCL0dKbkF6ejF0TVdH?= =?utf-8?B?SHg1OGxqTzlad2tuVGUvZHRsdWZGTTVXcnM3OExYV295TXlCSlBOT2tSZzBn?= =?utf-8?B?c09qQ1d5NEljQTR3UDFZUmpaVTh6eUJTem44SlVwOEV0UFl5c255MzdxN1FG?= =?utf-8?B?UDUzSHN6TjNjSlJyakNDQ0ZRVk53bkh4SVEwM2Y5dVpmd0pLZkV6RTZpNU1T?= =?utf-8?B?M1lqV29IdnpMWVhvUEExcHRvazNFVGZiNkF6bEVRL1YweEpPOGJERWVpODBr?= =?utf-8?B?ZzNsSXZHc3ptTkNqUTFSQUhhNGNrQTg2QXVqSEpIR0ZnTGJHOHBIY1BRV3B2?= =?utf-8?B?V0Q5b3g2c1FZY3NPRjBEaHJveVNPSVU2eUl1R0VoVll5L0NDQjQ3Ukhub1oz?= =?utf-8?B?ZGVQakljK3IrSTlnSTRKTnljWjFvTDBuSk5JNlVHNjVjQVd5aXB6VkdXdmhh?= =?utf-8?B?cm1EM2U1LzcwTldhemFEYUVrRy9id0JRd0NZdnBlejFtQUM0a04wZGh0ZWdY?= =?utf-8?B?QWRqZnZ4SVJKT0syTE9WTWpMaDBiSitBMnJSUjlCTXhuZUZFc2YzcnRLaEdt?= =?utf-8?Q?5d80C6URXTe13S734SJskuI7I?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: dd68e41e-2fb1-4efd-9d87-08db104762b1 X-MS-Exchange-CrossTenant-AuthSource: CH2PR12MB4294.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Feb 2023 17:58:19.0086 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: fBMl8klRfOdM1yflMsVWOFsW1OJzgCSKTji9BvLH5hY4+PFoDarZA5czVk15/Z3R X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB5294 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 2/14/2023 3:48 PM, Jiawei Wang wrote: > When multiple ports are aggregated into a single DPDK port, > (example: Linux bonding, DPDK bonding, failsafe, etc.), > we want to know which port use for Tx via a queue. > > This patch introduces the new ethdev API > rte_eth_dev_map_aggr_tx_affinity(), it's used to map a Tx queue > with an aggregated port of the DPDK port (specified with port_id), > The affinity is the number of the aggregated port. > Value 0 means no affinity and traffic could be routed to any > aggregated port, this is the default current behavior. > > The maximum number of affinity is given by rte_eth_dev_count_aggr_ports(). > > Add the trace point for ethdev rte_eth_dev_count_aggr_ports() > and rte_eth_dev_map_aggr_tx_affinity() functions. > > Add the testpmd command line: > testpmd> port config (port_id) txq (queue_id) affinity (value) > > For example, there're two physical ports connected to > a single DPDK port (port id 0), and affinity 1 stood for > the first physical port and affinity 2 stood for the second > physical port. > Use the below commands to config tx phy affinity for per Tx Queue: > port config 0 txq 0 affinity 1 > port config 0 txq 1 affinity 1 > port config 0 txq 2 affinity 2 > port config 0 txq 3 affinity 2 > > These commands config the Tx Queue index 0 and Tx Queue index 1 with > phy affinity 1, uses Tx Queue 0 or Tx Queue 1 send packets, > these packets will be sent from the first physical port, and similar > with the second physical port if sending packets with Tx Queue 2 > or Tx Queue 3. > > Signed-off-by: Jiawei Wang <...> > diff --git a/lib/ethdev/rte_ethdev.c b/lib/ethdev/rte_ethdev.c > index dc0a4eb12c..1d5b3a16b2 100644 > --- a/lib/ethdev/rte_ethdev.c > +++ b/lib/ethdev/rte_ethdev.c > @@ -6915,6 +6915,55 @@ rte_eth_buffer_split_get_supported_hdr_ptypes(uint16_t port_id, uint32_t *ptypes > return j; > } > > +int rte_eth_dev_count_aggr_ports(uint16_t port_id) > +{ > + struct rte_eth_dev *dev; > + int ret; > + > + RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV); > + dev = &rte_eth_devices[port_id]; > + > + if (*dev->dev_ops->count_aggr_ports == NULL) > + return -ENOTSUP; What do you think to return a default value when dev_ops is not defined, assuming device is not a bounded device. Not sure which one is better for application, return a default value or error. > + ret = eth_err(port_id, (*dev->dev_ops->count_aggr_ports)(port_id)); > + > + rte_eth_trace_count_aggr_ports(port_id, ret); > + > + return ret; > +} > + > +int rte_eth_dev_map_aggr_tx_affinity(uint16_t port_id, uint16_t tx_queue_id, > + uint8_t affinity) > +{ > + struct rte_eth_dev *dev; > + int ret; > + > + RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV); > + dev = &rte_eth_devices[port_id]; > + > + if (tx_queue_id >= dev->data->nb_tx_queues) { > + RTE_ETHDEV_LOG(ERR, "Invalid Tx queue_id=%u\n", tx_queue_id); > + return -EINVAL; > + } > + Although documentation says this API should be called before configure, if user misses it I guess above can crash, is there a way to add runtime check, like checking 'dev->data->dev_configured'? > + if (*dev->dev_ops->map_aggr_tx_affinity == NULL) > + return -ENOTSUP; > + > + if (dev->data->dev_started) { > + RTE_ETHDEV_LOG(ERR, > + "Port %u must be stopped to allow configuration\n", > + port_id); > + return -EBUSY; > + } > + > + ret = eth_err(port_id, (*dev->dev_ops->map_aggr_tx_affinity)(port_id, > + tx_queue_id, affinity)); > + Should API check if port_id is a bonding port before it continue with mapping? > + rte_eth_trace_map_aggr_tx_affinity(port_id, tx_queue_id, affinity, ret); > + > + return ret; > +} > + > RTE_LOG_REGISTER_DEFAULT(rte_eth_dev_logtype, INFO); > > RTE_INIT(ethdev_init_telemetry) > diff --git a/lib/ethdev/rte_ethdev.h b/lib/ethdev/rte_ethdev.h > index c129ca1eaf..07b8250eb8 100644 > --- a/lib/ethdev/rte_ethdev.h > +++ b/lib/ethdev/rte_ethdev.h > @@ -2589,6 +2589,52 @@ int rte_eth_hairpin_bind(uint16_t tx_port, uint16_t rx_port); > __rte_experimental > int rte_eth_hairpin_unbind(uint16_t tx_port, uint16_t rx_port); > > +/** > + * @warning > + * @b EXPERIMENTAL: this API may change without prior notice. > + * > + * Get the number of aggregated ports. > + * > + * @param port_id > + * The port identifier of the Ethernet device. > + * @return > + * - (>=0) the number of aggregated port if success. > + * - (-ENOTSUP) if not supported. > + */ > +__rte_experimental > +int rte_eth_dev_count_aggr_ports(uint16_t port_id); Can you please give more details in the function description, in the context of this patch it is clear, but someone sees it first time can be confused what is "aggregated ports" is. What is expected value for regular pysical port, that doesn't have any sub-devices, 0 or 1? Can you please document? > + > +/** > + * @warning > + * @b EXPERIMENTAL: this API may change without prior notice. > + * > + * Map a Tx queue with an aggregated port of the DPDK port (specified with port_id). > + * When multiple ports are aggregated into a single one, > + * it allows to choose which port to use for Tx via a queue. > + * > + * The application should use rte_eth_dev_map_aggr_tx_affinity() > + * after rte_eth_dev_configure(), rte_eth_tx_queue_setup(), and > + * before rte_eth_dev_start(). > + * > + * @param port_id > + * The identifier of the port used in rte_eth_tx_burst(). > + * @param tx_queue_id > + * The index of the transmit queue used in rte_eth_tx_burst(). > + * The value must be in the range [0, nb_tx_queue - 1] previously supplied > + * to rte_eth_dev_configure(). > + * @param affinity > + * The number of the aggregated port. > + * Value 0 means no affinity and traffic could be routed to any aggregated port. > + * The first aggregated port is number 1 and so on. > + * The maximum number is given by rte_eth_dev_count_aggr_ports(). > + * > + * @return > + * Zero if successful. Non-zero otherwise. > + */ > +__rte_experimental > +int rte_eth_dev_map_aggr_tx_affinity(uint16_t port_id, uint16_t tx_queue_id, > + uint8_t affinity); > + > /** > * Return the NUMA socket to which an Ethernet device is connected > * > diff --git a/lib/ethdev/version.map b/lib/ethdev/version.map > index dbc2bffe64..685aa71e51 100644 > --- a/lib/ethdev/version.map > +++ b/lib/ethdev/version.map > @@ -300,6 +300,8 @@ EXPERIMENTAL { > rte_mtr_meter_profile_get; > > # added in 23.03 > + rte_eth_dev_count_aggr_ports; > + rte_eth_dev_map_aggr_tx_affinity; > rte_flow_async_create_by_index; > }; >