From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7034341DE6; Mon, 6 Mar 2023 13:49:37 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 505EE40E50; Mon, 6 Mar 2023 13:49:37 +0100 (CET) Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2052.outbound.protection.outlook.com [40.107.244.52]) by mails.dpdk.org (Postfix) with ESMTP id 216A840A8A; Mon, 6 Mar 2023 13:49:35 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ICQIPi9+GzT+ecpnOi+IkM+pJqw7T3d8Y069LUvT/LF1oCyiiMAtnVRNA6jd95Oj5LJPDdjMiPE5SjSrm/WvcwiYVslaziob7L45ykol5Lgz9SlWSYsn+7d6zDp0oro6MfTNP8864aTjORhArxXjaW5J+cB8RTBBGM+2FIydhd6J+68FesR6/hb2DmEdwciwNwI4izlfqZ3NNQqInqPZgfLameVX3Ex0LgdCHhcknRgnXmP/x+LifnxbGGEyg5PrZomWfla4xgACEUcCOP9h1V2of1Q1JF3WOCucqc6i0HLwh4X2Up3IfXoBwX1Qml9ui0DBlmVfbisdiApRUSwN5g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=p50uSw0JS7ShBpVPP353y6NWDJkiHFHa+k14j0eJXHg=; b=bZbko9ekWy2px4arh6fRPzomNKklUxswnBTj7SgmH5Rc5LVxFm32vWoFFaz0WMBkGXdLRAIWvgiE90Hwj4ZK4QeCGDORZCtkq7OOSJncq6ZPM+UGAt1C2ZM1J0DaaJ0qD/o7LCUuRbbSB7ABOP6fFZXu6Pfnm3rJt6hgfI7SQK6qQA9MPtvwG1U0xJBorRrKqQS2OZ/MFWFtplz/PHLhT2pAkoSPUMCpMf6el9bT6cJL3iOaTkpcCY7MJZDDRwwGMAMkK8eOVKQUm2yzorxjAFhMF7diKsVLjVjAHqyCQXtIor0Ga3qiRF+7Yetpn7QK0QkVwAxeybHawF/ixPwViQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=p50uSw0JS7ShBpVPP353y6NWDJkiHFHa+k14j0eJXHg=; b=aDYR/brEXlUPoy910cb4GtDW3nfNRC2Fr9JpzNOS7eo1aQ1HtH0sJ/mPhw9Dgi+UwH9B5FREK8t7/2LOSDqBIxXup8Zb92vbO1tuZVrknhpPhXbWX4nYhQAyRzk+DjXi3Ls1vg8GA8767phvZOh8zMUrkr77sL0bcs2RKvnKElw= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=amd.com; Received: from CH2PR12MB4294.namprd12.prod.outlook.com (2603:10b6:610:a9::11) by SA0PR12MB4512.namprd12.prod.outlook.com (2603:10b6:806:71::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6156.28; Mon, 6 Mar 2023 12:49:32 +0000 Received: from CH2PR12MB4294.namprd12.prod.outlook.com ([fe80::dd5a:8a5c:f493:9640]) by CH2PR12MB4294.namprd12.prod.outlook.com ([fe80::dd5a:8a5c:f493:9640%5]) with mapi id 15.20.6156.028; Mon, 6 Mar 2023 12:49:31 +0000 Message-ID: Date: Mon, 6 Mar 2023 12:49:24 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.8.0 Content-Language: en-US To: =?UTF-8?Q?Morten_Br=c3=b8rup?= , Feifei Wang , Thomas Monjalon , Andrew Rybchenko , "techboard@dpdk.org" Cc: dev@dpdk.org, konstantin.v.ananyev@yandex.ru, nd@arm.com, Honnappa Nagarahalli , Ruifeng Wang References: <20220420081650.2043183-1-feifei.wang2@arm.com> <20230104073043.1120168-1-feifei.wang2@arm.com> <20230104073043.1120168-2-feifei.wang2@arm.com> <98CBD80474FA8B44BF855DF32C47DC35D8761D@smartserver.smartshare.dk> From: Ferruh Yigit Subject: Re: [PATCH v3 1/3] ethdev: enable direct rearm with separate API In-Reply-To: <98CBD80474FA8B44BF855DF32C47DC35D8761D@smartserver.smartshare.dk> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-ClientProxiedBy: LO0P265CA0011.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:355::19) To CH2PR12MB4294.namprd12.prod.outlook.com (2603:10b6:610:a9::11) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR12MB4294:EE_|SA0PR12MB4512:EE_ X-MS-Office365-Filtering-Correlation-Id: 5bfb4808-16c6-4f8b-5781-08db1e413af9 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: hArFgBDD+VIAaL8tsToSTywsyVktQ2QSsoHnX0kYEyPjEKIua2DU44ncY9A6vvRmIVGTRbGJmOoTKX02onkvZkNcblJz+S7QznolKyCC+7xEdIdUZXqpmIlVX42XAIdKIxlwGXxHXMc5BvgLXVVd34SQezCF6i4wQRlYi3P/s2nFvUowabRYPSu+9ZcqRL3g5wSf1Y0QVk5AmK1G0oJ9RuUkuVLjIJUgupjdsq5+A4wX3SRCl5pCy6pKhR7e2/29OU1KDp9Y+nUHpGJek7dsztEnEVffQVqIS+6APTwZcbiL33MTEKAcweG4cGm7AY6X20xevBetwgCmslCKSOXS+zkhiEFtEY+YRy3M+M38vaB+zd6zWCq9CtjUCnvAE+W6AMqPw0RPJ2fn7lJnERb68wre+fs78XWW9Otif6niR3EYEkQmsL3bX7MJJz4KUrkuoBWGw7I0+/DppiePznGD2RvsqmMzhEDWUhJemrddxT7mqGjwYOXO2Qyoe3Vv+TNMiWLw/IwTM53fJgKPSBqMuslfNmHi/lcdzD9/p6QAYhPKtSIaxjG/kl+Z2pVvjw+ita53dWke1RzIp2yCi9tfjYr5rOnc057TvWL7QaBQRO7GIRqW2pHcYXYCpNcsbIN0B4teVznp2EUN8f3oV30V0qtxwe3X8LoQ3FIf6wY68CPhTMhXidqmNuA/qagmA7PWLHcHWQZ6mXhS7AvIrZ7TwiMh8nZObO3xnKgbDWIYepA= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR12MB4294.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230025)(4636009)(376002)(136003)(396003)(346002)(366004)(39860400002)(451199018)(31686004)(36756003)(4326008)(66946007)(41300700001)(8676002)(66476007)(8936002)(66556008)(44832011)(5660300002)(7416002)(2906002)(86362001)(31696002)(38100700002)(6666004)(6486002)(478600001)(110136005)(54906003)(316002)(66574015)(83380400001)(6512007)(6506007)(2616005)(26005)(53546011)(186003)(45980500001)(43740500002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?T0ZXRHJ0SDZFSFpOSFM1b0lLSTJNQjdyNysxeW81ekVuNG1oTnErcjB6ajYw?= =?utf-8?B?OEJTTmZwZUROZVBwMUF3T3ZyU3hCUnFvMlE0a1BrZmJoRXFuVUszK21UdWhS?= =?utf-8?B?Z0VXWXQrekZ2SnJHbjBtN1NFSFNoVzE5alRVaWJSVmtHMko1OWZZTG5qQ2VY?= =?utf-8?B?d1R2WlZuV1lnRFRHcWlmbEtaVVdRRzQ3emZYNDdvcndKbTM5MCtYWlQrSlc0?= =?utf-8?B?TkY0N1FZbllnRFlSbVJ4dVRzVjlrd0NhRU15bGNpMWttUWo0NHVsNHFxVXo1?= =?utf-8?B?dFdoWC82RklDWUhoN0VjODY5ajJFNUpDT1Z4eHlNcVNmWERhMHkvSXRJQlVK?= =?utf-8?B?ZWZOd0VTRkhrRDF4b1dzazFNT0ZkYk9sUUpxMUt5dHJZUnpYS2VTcnBjWHdT?= =?utf-8?B?a0dTYktRcE5OMXJETTF0eWQ2N3pGak1uaFl3dE84anVDL2dWQnVTM3dManF6?= =?utf-8?B?dDRJY0EzcTh6eHpETWhtQjJsdVMyVERvdDhKZ2U2ZWVoR0QxdFFHejF3Ri9o?= =?utf-8?B?YWs5MkpOd1k2Mmt3bXZjMVFCU1FlQWxCSFZyb0VRSjkzN3dReTVCdFlzZEc5?= =?utf-8?B?U3dkQWZpZ3IzWm9QUkV0TUxYR3dDTlRqWlExTmQrRHQvNXpORmJEQi9ML09x?= =?utf-8?B?U0pRZnUzbDYwVFVVM042UHF1dC93NldzVlRPdTl1OWNack9LN3FybGFhYVlp?= =?utf-8?B?emEwTEszTG1Pdm16N2hrNEdPTEFudGwxUGpTb3h1RDN0SDU3ZEJmWlBBZUlq?= =?utf-8?B?Rm9aTEZaVURoN3VJbVRyS0h6Y1hWTEU4NWhEUDYxWmJNeVZBUGxmSjlqNlNz?= =?utf-8?B?b0tPelRRY2w1c2dMMUdVMkxBRGY1YVhyK2xnckFmbE5wNUhUS2Q1U0hSUkUv?= =?utf-8?B?YnBiV3gzb1UyaGZRNlpRbCszZmtkUzg3WFkrbGVMWUwycEtCdS8rOElMYlVJ?= =?utf-8?B?R0RmbzdEYzh4bEhtY1BONXd1VXhseE5kbGR0eHNSSC9FTS9FdDhrS1pINkNR?= =?utf-8?B?RVRoemU3T015U3JZU251VVpSSW1mL1ZRbjhEanVaN3k5ODhTQjltUk1PcU50?= =?utf-8?B?aGZua01tQ21xcmlRb0JndG9PYWxkTE5CRmIvUVBwOWp0cGQrRmJ4TUhYcURD?= =?utf-8?B?YVF1RjA0VkM0Q1l1ZU94SnZkZVpMc3g0UEFOWTlqZm4rWHNIQi9abHBjeVNz?= =?utf-8?B?eXJLODFIaUZMSkx1bDhUZTRzOXZDWEFBYUg0MTROUFVDcytLVGQvemhuYkV6?= =?utf-8?B?TGF0SmlkNEl0ZFoyZmM3RW5WdlhaS2tISEVoT2Y4M3RaOFVkTWZZK0g4K0xH?= =?utf-8?B?OE5kTXJRQnhNU1lTMDFwTHdNdUMwby9VdlJ3ekNSeis4Y0VmZzBFV0grSmdL?= =?utf-8?B?cXBKallLK3o1YlNpc3ZYRXRBakZPRkx5RnVtaU4vemtWY0lhTlp1cnA1dzA3?= =?utf-8?B?WHJyTGkyWnZQd3d6ckN6N1MrOUZqa1VIR3JBeldCUWtueUIrNUN6WEM2aS9p?= =?utf-8?B?WEdNQTVnMHhXTGZnTlhQT21hNEhscmc3Yk1Gdk9hR2IvUFEzMUJaM21lTk1q?= =?utf-8?B?dmZkUkkyUWx1ZkRFK1dabDB6TE5YZStZU0JKOGZ4L1R4L0pqZnpaTU15THl6?= =?utf-8?B?V2d4WHRuOUpPdFE3d0lnUmJrTDlzaUNYd3c1R0FaK3BIVnl6YmdTamNEb0dr?= =?utf-8?B?Q3lHQm5EVUVSQkhTOEdDbVVhSzJVaUt1bTRuNUdqcWJBcUxFdVVzMHhWR2Zo?= =?utf-8?B?eXZoTXQwd3FLa2NCT3l1Vk5ReDBOVlNQbmtNdDdIUTlIR2JUMEgxRmlFK1dI?= =?utf-8?B?OXRSNlJacnlEcUpEeS9hS0ZidThOZWlEbGlQWnZVbGZaRUgwVjBBZjd1Rkho?= =?utf-8?B?cEVFTjYyRDcrSVZFNnM2aWJQZnNiams2N0FJWnQxbDhvYUdnSER0Ujlxa1FP?= =?utf-8?B?cFV5cE4yR0NDeEhWdnZuQWF3bTd2dkQzT2ZlWkc3cks5dVdvOXhZRTgzZGZP?= =?utf-8?B?UjJpTy9jT2R5cFRIeEFxNUZnd2NtOU1ZVDZ6L2dJQ3lLRFlJd0gwbXZDbkwy?= =?utf-8?B?emxsYUFsR0IyeEp2QWMrcHZsdmpSQWVBNElsN3FaeUxLNFZoL295NWlJMDVW?= =?utf-8?Q?xj7N/8uWcfrj9Mj3HJit14czH?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 5bfb4808-16c6-4f8b-5781-08db1e413af9 X-MS-Exchange-CrossTenant-AuthSource: CH2PR12MB4294.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Mar 2023 12:49:31.6839 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: n62IuUYRiK7tCxJ8o7uzHyILWOXaXEoJmO05OXZ2U8DrI4yIbQho7C0S3nPbm2Xg X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR12MB4512 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 1/4/2023 8:21 AM, Morten Brørup wrote: >> From: Feifei Wang [mailto:feifei.wang2@arm.com] >> Sent: Wednesday, 4 January 2023 08.31 >> >> Add 'tx_fill_sw_ring' and 'rx_flush_descriptor' API into direct rearm >> mode for separate Rx and Tx Operation. And this can support different >> multiple sources in direct rearm mode. For examples, Rx driver is >> ixgbe, >> and Tx driver is i40e. >> >> Suggested-by: Honnappa Nagarahalli >> Suggested-by: Ruifeng Wang >> Signed-off-by: Feifei Wang >> Reviewed-by: Ruifeng Wang >> Reviewed-by: Honnappa Nagarahalli >> --- > > This feature looks very promising for performance. I am pleased to see progress on it. > Hi Morten, Yes it brings some performance, but not to generic use case, only to specific and constraint use case. And changes are relatively invasive comparing the usecase it supports, like it adds new two inline datapath functions and a new dev_ops. I am worried the unnecessary complexity and possible regressions in the fundamental and simple parts of the project, with a good intention to gain a few percentage performance in a specific usecase, can hurt the project. I can see this is compared to MBUF_FAST_FREE feature, but MBUF_FAST_FREE is just an offload benefiting from existing offload infrastructure, which requires very small update and logically change in application and simple to implement in the drivers. So, they are not same from complexity perspective. Briefly, I am not comfortable with this change, I would like to see an explicit approval and code review from techboard to proceed. > Please confirm that the fast path functions are still thread safe, i.e. one EAL thread may be calling rte_eth_rx_burst() while another EAL thread is calling rte_eth_tx_burst(). > > A few more comments below. > >> lib/ethdev/ethdev_driver.h | 10 ++ >> lib/ethdev/ethdev_private.c | 2 + >> lib/ethdev/rte_ethdev.c | 52 +++++++++++ >> lib/ethdev/rte_ethdev.h | 174 +++++++++++++++++++++++++++++++++++ >> lib/ethdev/rte_ethdev_core.h | 11 +++ >> lib/ethdev/version.map | 6 ++ >> 6 files changed, 255 insertions(+) >> >> diff --git a/lib/ethdev/ethdev_driver.h b/lib/ethdev/ethdev_driver.h >> index 6a550cfc83..bc539ec862 100644 >> --- a/lib/ethdev/ethdev_driver.h >> +++ b/lib/ethdev/ethdev_driver.h >> @@ -59,6 +59,10 @@ struct rte_eth_dev { >> eth_rx_descriptor_status_t rx_descriptor_status; >> /** Check the status of a Tx descriptor */ >> eth_tx_descriptor_status_t tx_descriptor_status; >> + /** Fill Rx sw-ring with Tx buffers in direct rearm mode */ >> + eth_tx_fill_sw_ring_t tx_fill_sw_ring; > > What is "Rx sw-ring"? Please confirm that this is not an Intel PMD specific term and/or implementation detail, e.g. by providing a conceptual implementation for a non-Intel PMD, e.g. mlx5. > > Please note: I do not request the ability to rearm between drivers from different vendors, I only request that the public ethdev API uses generic terms and concepts, so any NIC vendor can implement the direct-rearm functions in their PMDs. > >> + /** Flush Rx descriptor in direct rearm mode */ >> + eth_rx_flush_descriptor_t rx_flush_descriptor; > > descriptor -> descriptors. There are more than one. Both in comment and function name. > > [...] > >> diff --git a/lib/ethdev/rte_ethdev.h b/lib/ethdev/rte_ethdev.h >> index c129ca1eaf..381c3d535f 100644 >> --- a/lib/ethdev/rte_ethdev.h >> +++ b/lib/ethdev/rte_ethdev.h >> @@ -1818,6 +1818,17 @@ struct rte_eth_txq_info { >> uint8_t queue_state; /**< one of RTE_ETH_QUEUE_STATE_*. */ >> } __rte_cache_min_aligned; >> >> +/** >> + * @internal >> + * Structure used to hold pointers to internal ethdev Rx rearm data. >> + * The main purpose is to load Rx queue rearm data in direct rearm >> mode. >> + */ >> +struct rte_eth_rxq_rearm_data { >> + void *rx_sw_ring; >> + uint16_t *rearm_start; >> + uint16_t *rearm_nb; >> +} __rte_cache_min_aligned; > > Please add descriptions to the fields in this structure. >