From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id CD532437B3; Thu, 28 Dec 2023 14:54:05 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 98E78402EA; Thu, 28 Dec 2023 14:54:05 +0100 (CET) Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2083.outbound.protection.outlook.com [40.107.243.83]) by mails.dpdk.org (Postfix) with ESMTP id 8C4DB402BA for ; Thu, 28 Dec 2023 14:54:00 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=M2Xi5TqOUDwoYNixGd2oogoMpMfHeUR4UxZScXJ7UE8RM6nAG3EaAn5v9QQv0OHdKY4/pg3LSLj31hex8mrMmROyx2eHNPNgBlCKibthunvqILojPxMxq8Ox39eAeLBS5qkbUzHPozdCv+d8i78+N/Nyn5U9oCxQCtEKRtrlvv6epQIyde5h4JERFXYeeQPdOEN1pTS22pCnuV64jATisXtPrSlEcHA5WK13Ogu1tWtFdSszzTjnh6AijWxdIBCn/cOuvKXby91vVkTN/93Humir17jZOUP6ww5RG1UkeJ8xO/j7MEd0CmwYnkjcaN5J2Wt5WN2xKbBn5VepuBpCXA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=vS2vvF8lwz6Vj9RL2U8neGoVFiiIJKRDMyWVbtA57E8=; b=I7mp08DvOYgdCc60PqrTd2hj442JS6vhTfrxWVCCGKYviH5gytk9sfbqZl0jgIzGqOxEj2Bpkl9pgD+XXA2XbgRICEIaqQIH09ELYZpzsR5nrdORnJmoHLId4DMyIefBARAwu6GyoRaAEBcqvNpNzAMKgniWL09aUdMB0MN9O4GHTcHvPVTV57lBGe/kx6MfVPMyoy0Nf5EuuOXEvTI1lQDD5Odq0vH2dswyn+oWQFRBSEzIFH4sG1jek7ZWxhCwk31r7fH+XXHuP84AouNPH7pvWtEOxM+ziCeWGmavlNEc7MsOAa58UlWFGWeecvkMqQwJgH+xq1dKwikNZUDV0Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=vS2vvF8lwz6Vj9RL2U8neGoVFiiIJKRDMyWVbtA57E8=; b=sVHzEBBT3PS87UQpwpc+BF+NDw+667PJhk2yMex4/+eaj7suz8NxbD3/Fpsjy6Nvsq6pUXnJeGNc0dkfbtMEafEzZ4k201tDtavY/a5Ya7plP9na0wX1ARfcMg7516wVZ8lKCgZgHEv4jvSiJPT7wVORnFLok99yPXY0nfTfO7/5T3OaZ1K5uIwYSOqtFoY52F1PLo9Fitsig0j8V2qywdBnwiGY0dtqOP2NGaldkSll59CxA2DR9CmNYLth3vvNusQT8UgEMSvKDSeJHE/+oC7qUclmHFrmmzd4BVxTjLba5DB3uAp9QYhfEdatV0xjYyTRa4h97yFJaiFBsaCPxQ== Received: from IA1PR12MB8311.namprd12.prod.outlook.com (2603:10b6:208:3fa::12) by DM4PR12MB6541.namprd12.prod.outlook.com (2603:10b6:8:88::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7135.21; Thu, 28 Dec 2023 13:53:58 +0000 Received: from IA1PR12MB8311.namprd12.prod.outlook.com ([fe80::2d14:dd8e:f91d:3175]) by IA1PR12MB8311.namprd12.prod.outlook.com ([fe80::2d14:dd8e:f91d:3175%3]) with mapi id 15.20.7135.017; Thu, 28 Dec 2023 13:53:57 +0000 From: Dariusz Sosnowski To: Stephen Hemminger CC: "NBU-Contact-Thomas Monjalon (EXTERNAL)" , Ferruh Yigit , Andrew Rybchenko , Ori Kam , "dev@dpdk.org" Subject: RE: [RFC] ethdev: fast path async flow API Thread-Topic: [RFC] ethdev: fast path async flow API Thread-Index: AQHaOLOJMBiGV3bi0EOK9su1qim887C9ZmKAgAAQCVA= Date: Thu, 28 Dec 2023 13:53:57 +0000 Message-ID: References: <20231227105709.1951231-1-dsosnowski@nvidia.com> <20231227094117.678ce828@hermes.local> In-Reply-To: <20231227094117.678ce828@hermes.local> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: IA1PR12MB8311:EE_|DM4PR12MB6541:EE_ x-ms-office365-filtering-correlation-id: 9c083899-b519-463f-786f-08dc07ac7043 x-ld-processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 7O0m/vBxMX56uaGt1ySddyVLKBp4ifti2ILbUFdJ8SIV63szwT78lnb+OSlO7LDa8F9kpL7sawfx3Efg+nbh151yvtry5dEToQrYizg3aklUPBTpkXzuEjIyN4awdWvTQIedzVheJurfJE+faDBFO1YkDpz45cwHPQeYLui6vaCsq7Wgb6UhOwfAFzeCUCun+83ClZnPem4cWcYLNfaKWwp7ycmXtK89yEgKrkcL198mWW1rpEiTXZ2EMuckfcAqIUx946nkHpNmaBVm8c47IUHNXFQysFE2KxYEeJHuH3C2TIiglw9Ncc23EH8Rbfd3rt/XfgOLnWtAogWHdsynjiLN2b8x5/nzacdLyaQNdhF3HtbKM9+k3v1YSsMFtdjmNHS70c7Ay9PKBQhhaYgDagh+BMrV+SycKjtrhxN0y47G1CUSSjQ7JO9DlzV6mV03gpzDSMKn61+io1W5tM7gug027K4yOIJcfX4bR800KFiuOvcLGW/3Id4Q/HwSA5pNZRKbc1U0VTZsFvRSHKHb3ylPxQ7GUQSHaUNiMs1D3RcODe7k99/GNorU8DkGoVCLI5JruBYh5aPTW8Y9Qtv3JMv+u5wTCeDTMf/dqYlmc4uRbDoOMjSlAn3lTpYN4iCe x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:IA1PR12MB8311.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(396003)(376002)(136003)(39860400002)(366004)(346002)(230922051799003)(186009)(451199024)(64100799003)(1800799012)(66446008)(66476007)(76116006)(66556008)(66946007)(6916009)(64756008)(4326008)(52536014)(54906003)(316002)(8936002)(26005)(83380400001)(478600001)(6506007)(9686003)(55236004)(71200400001)(7696005)(8676002)(5660300002)(2906002)(41300700001)(33656002)(38070700009)(122000001)(38100700002)(86362001)(55016003); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?CIGTwBEI/ze0FqHqoxxcjxYG6Domy1ICkrb7E7dgo6CRaGqxQZB1Lf2UyIKM?= =?us-ascii?Q?VVZzvG5JTFWMhRxH7lvCOXQB+FCYq7XTwLuI564kgWDOWtlec/Z5itKjbTXs?= =?us-ascii?Q?nyo1KnKBf23V2aL5LE2da5bNBoMybyhFHIkNQtqal0tHZ7GucNCTCkJ0Yl/t?= =?us-ascii?Q?49/tXTlym+cvGmFBMUYIwU2AOLs3aBI9sqybytNPp0uZOxcRmiDyRWNGHHJb?= =?us-ascii?Q?qbHPLOPRslQVeg9G/Ap4KCJmroC8lJo1EcrqZqH5fwCTXvnYYVnjsApgQCL/?= =?us-ascii?Q?toEbLfJ7+Rn4oBcHApJuA9nfwCOxRKza3L3OhnPiqJN1+TC0ks+NbB9wCmGg?= =?us-ascii?Q?7YBGX+SSIK71/+G9tyVAxlkFXt+PlPtVmTsAhGmlkm64Nypoi1NRPg099Ti+?= =?us-ascii?Q?6ML27BXyE0fBH+uWQ+x22tKO4B9tKIjdV/4BXY2ypA/N/bb5R9VAQib8tXZZ?= =?us-ascii?Q?2QAL0ScITO8kqXXQbG0Oilq3e98XHRdM8NyXBMWuc6kpWvT9uVsbuyGmrlxZ?= =?us-ascii?Q?B6m5TcrgsvreFAP/Rb/O/xcmWP5g+5YcTf9yRZXbZBHtw23SOryM+CWwHZiN?= =?us-ascii?Q?y+loxw6pi8dwEIEd8uyXfVs/JKUu8RdkJuwAkuvbDJQpY2XA7kqXXfxWBZbn?= =?us-ascii?Q?5Eb7rXLg5Kio9szv4o4lOx4lSWq2sg+f7TK+IViZZ/dx4+GmHyNqDhn2kS65?= =?us-ascii?Q?5OjJGpr6wbOKz9yNztmxyySA9M+UQOcvMuHJvR4OUAEmNslSxzt9j45kGg6W?= =?us-ascii?Q?ZV1oqMxCdlDQGG/Ang/Hm6VNYoXDYAXYpQYr+HR3GfiOZWAemsZoM4ftieLD?= =?us-ascii?Q?7L/c0xqAm8IScwortdB07EJmYFOeJ+mccIlQoE34IWHrM7+AMC0C/qItmRcT?= =?us-ascii?Q?RGbQlWe9Ad5SlBW6qFvELMz5mTFb8r+fhXlYMcBRnBfIJZKVSPWALkT5u+bc?= =?us-ascii?Q?eY/XH//vKQ9O0rKnlOpB5VGjpUkop4wgOOcsnL0+pkahW0M7xzg4+MNtojrj?= =?us-ascii?Q?Kx2lxKUvNyqvwQcWsHPc1QETQMKumQjuOaodhp9Mi0+/NcQIZaLtNUBX6k6N?= =?us-ascii?Q?W2s5mbodf1WBkDDpq2VFPDEeQFFaay+1lYzN39IvBluD60h3k6dw4EkqcxNi?= =?us-ascii?Q?j44EW+VKnj0SVzU+eHD7q/XYFr07RjJXwmOe1PmsHg9DY4NYdnEAzDnU1FjY?= =?us-ascii?Q?kwa/LzR16y45WLB9V3//VzBwtiEyCKHeuVuiUfg86NYsmuUHs0eydpkDAra8?= =?us-ascii?Q?BwLzkZSr09mdGGltyem/+7os8zlUYcDpI29LMpbJ2MJuJkay+Q7qtOV3HfBd?= =?us-ascii?Q?U7b2GhGvVNCW3x915WCNSAuK/r9fY6bLn/MKiGUjySMLFrJFve/00kVac/eh?= =?us-ascii?Q?Gr+kznDmeMT6GqTQXxk6Aq55MFjte61rIeiD021ohUldhD+JuGAwjpLWFzDJ?= =?us-ascii?Q?uJgQvoBDlLxLVe+NKuw9nTMk+ATiTJ7LhU4zbF2WYzvHaOfc6WPQb0gcZgHI?= =?us-ascii?Q?3aJ5dR2tWuUB2/EyRnEKGxOUhIhsfpEI7PrqvM4qdgBaESKV1W8LPO4oSTdN?= =?us-ascii?Q?CVnpAI/3SOB8r40ya1JQF8V3m3XiEePgOv49qz2X?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: IA1PR12MB8311.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9c083899-b519-463f-786f-08dc07ac7043 X-MS-Exchange-CrossTenant-originalarrivaltime: 28 Dec 2023 13:53:57.8453 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: F0z3V5sNz1264HCjboLtxOm9HQksKEEGhhZ1u/6li0EJDrBD4BPFuzVPExaTiHUytHz/40l8vyi17hnhlZdBbA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB6541 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Hi Stephen, > > +/** > > + * @internal > > + * > > + * Fast path async flow functions and related data are held in a flat = array, > one entry per ethdev. > > + * It is assumed that each entry is read-only and cache aligned. > > + */ > > +struct rte_flow_fp_ops { > > + void *ctx; > > + rte_flow_async_create_t async_create; > > + rte_flow_async_create_by_index_t async_create_by_index; > > + rte_flow_async_actions_update_t async_actions_update; > > + rte_flow_async_destroy_t async_destroy; > > + rte_flow_push_t push; > > + rte_flow_pull_t pull; > > + rte_flow_async_action_handle_create_t async_action_handle_create; > > + rte_flow_async_action_handle_destroy_t async_action_handle_destro= y; > > + rte_flow_async_action_handle_update_t async_action_handle_update; > > + rte_flow_async_action_handle_query_t async_action_handle_query; > > + rte_flow_async_action_handle_query_update_t > async_action_handle_query_update; > > + rte_flow_async_action_list_handle_create_t > async_action_list_handle_create; > > + rte_flow_async_action_list_handle_destroy_t > async_action_list_handle_destroy; > > + rte_flow_async_action_list_handle_query_update_t > async_action_list_handle_query_update; > > +} __rte_cache_aligned; >=20 > If you go ahead with this then all usage should be const pointer. > Still think it is bad idea and creates even more technical debt. Could you please elaborate a bit more on why do you think it is a bad idea = and why it creates technical debt?=20 > > **Future considerations** > > > > A case can be made about converting some of the existing stable API=20 > > functions to fast path versions in future LTS releases. > > I don't have any hard data on how such changes would affect=20 > > performance of these APIs, but I assume that the improvement would be n= oticeable. >=20 > The problem is that inline functions create future ABI problems. Agreed, this is why such a change can only be introduced when a new major A= BI version is released. However, even though inlining could reduce function call overhead, I'm not = sure if such a change is needed for synchronous flow API. I mentioned it here as a thing to consider. > Usually, there are other ways to get the same performance benefit. > Often batching updates to hardware will do the trick. This is true, but we still have some library-level overhead. To elaborate more, the current state of flow API is as follows: - With sync flow API: - Applications cannot batch flow operations. - With async flow APIs: - Applications can enqueue multiple flow operations and PMDs can issue ba= tches to HW. - But there is still one function call per enqueued flow operation. The overall API overhead in datapath may be nonnegligible if we conside= r that applications may enqueue a flow rule creation operation for every pa= cket received in SW. This proposal specifically aims at reducing API overhead for async flow API= in a case mentioned above. As a side note, we plan to push changes to mlx5 PMD in 24.03 which will red= uce PMD overhead in such scenarios. This proposal's goal is to reduce overhead of async flow API at library lev= el. Best regards, Dariusz Sosnowski