From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124])
	by inbox.dpdk.org (Postfix) with ESMTP id CD532437B3;
	Thu, 28 Dec 2023 14:54:05 +0100 (CET)
Received: from mails.dpdk.org (localhost [127.0.0.1])
	by mails.dpdk.org (Postfix) with ESMTP id 98E78402EA;
	Thu, 28 Dec 2023 14:54:05 +0100 (CET)
Received: from NAM12-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam12on2083.outbound.protection.outlook.com [40.107.243.83])
 by mails.dpdk.org (Postfix) with ESMTP id 8C4DB402BA
 for <dev@dpdk.org>; Thu, 28 Dec 2023 14:54:00 +0100 (CET)
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=M2Xi5TqOUDwoYNixGd2oogoMpMfHeUR4UxZScXJ7UE8RM6nAG3EaAn5v9QQv0OHdKY4/pg3LSLj31hex8mrMmROyx2eHNPNgBlCKibthunvqILojPxMxq8Ox39eAeLBS5qkbUzHPozdCv+d8i78+N/Nyn5U9oCxQCtEKRtrlvv6epQIyde5h4JERFXYeeQPdOEN1pTS22pCnuV64jATisXtPrSlEcHA5WK13Ogu1tWtFdSszzTjnh6AijWxdIBCn/cOuvKXby91vVkTN/93Humir17jZOUP6ww5RG1UkeJ8xO/j7MEd0CmwYnkjcaN5J2Wt5WN2xKbBn5VepuBpCXA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; 
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=vS2vvF8lwz6Vj9RL2U8neGoVFiiIJKRDMyWVbtA57E8=;
 b=I7mp08DvOYgdCc60PqrTd2hj442JS6vhTfrxWVCCGKYviH5gytk9sfbqZl0jgIzGqOxEj2Bpkl9pgD+XXA2XbgRICEIaqQIH09ELYZpzsR5nrdORnJmoHLId4DMyIefBARAwu6GyoRaAEBcqvNpNzAMKgniWL09aUdMB0MN9O4GHTcHvPVTV57lBGe/kx6MfVPMyoy0Nf5EuuOXEvTI1lQDD5Odq0vH2dswyn+oWQFRBSEzIFH4sG1jek7ZWxhCwk31r7fH+XXHuP84AouNPH7pvWtEOxM+ziCeWGmavlNEc7MsOAa58UlWFGWeecvkMqQwJgH+xq1dKwikNZUDV0Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com;
 dkim=pass header.d=nvidia.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com;
 s=selector2;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=vS2vvF8lwz6Vj9RL2U8neGoVFiiIJKRDMyWVbtA57E8=;
 b=sVHzEBBT3PS87UQpwpc+BF+NDw+667PJhk2yMex4/+eaj7suz8NxbD3/Fpsjy6Nvsq6pUXnJeGNc0dkfbtMEafEzZ4k201tDtavY/a5Ya7plP9na0wX1ARfcMg7516wVZ8lKCgZgHEv4jvSiJPT7wVORnFLok99yPXY0nfTfO7/5T3OaZ1K5uIwYSOqtFoY52F1PLo9Fitsig0j8V2qywdBnwiGY0dtqOP2NGaldkSll59CxA2DR9CmNYLth3vvNusQT8UgEMSvKDSeJHE/+oC7qUclmHFrmmzd4BVxTjLba5DB3uAp9QYhfEdatV0xjYyTRa4h97yFJaiFBsaCPxQ==
Received: from IA1PR12MB8311.namprd12.prod.outlook.com (2603:10b6:208:3fa::12)
 by DM4PR12MB6541.namprd12.prod.outlook.com (2603:10b6:8:88::11) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7135.21; Thu, 28 Dec
 2023 13:53:58 +0000
Received: from IA1PR12MB8311.namprd12.prod.outlook.com
 ([fe80::2d14:dd8e:f91d:3175]) by IA1PR12MB8311.namprd12.prod.outlook.com
 ([fe80::2d14:dd8e:f91d:3175%3]) with mapi id 15.20.7135.017; Thu, 28 Dec 2023
 13:53:57 +0000
From: Dariusz Sosnowski <dsosnowski@nvidia.com>
To: Stephen Hemminger <stephen@networkplumber.org>
CC: "NBU-Contact-Thomas Monjalon (EXTERNAL)" <thomas@monjalon.net>, Ferruh
 Yigit <ferruh.yigit@amd.com>, Andrew Rybchenko
 <andrew.rybchenko@oktetlabs.ru>, Ori Kam <orika@nvidia.com>, "dev@dpdk.org"
 <dev@dpdk.org>
Subject: RE: [RFC] ethdev: fast path async flow API
Thread-Topic: [RFC] ethdev: fast path async flow API
Thread-Index: AQHaOLOJMBiGV3bi0EOK9su1qim887C9ZmKAgAAQCVA=
Date: Thu, 28 Dec 2023 13:53:57 +0000
Message-ID: <IA1PR12MB831187A9641D5C6BD9FE6EB6A49EA@IA1PR12MB8311.namprd12.prod.outlook.com>
References: <20231227105709.1951231-1-dsosnowski@nvidia.com>
 <20231227094117.678ce828@hermes.local>
In-Reply-To: <20231227094117.678ce828@hermes.local>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=nvidia.com;
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: IA1PR12MB8311:EE_|DM4PR12MB6541:EE_
x-ms-office365-filtering-correlation-id: 9c083899-b519-463f-786f-08dc07ac7043
x-ld-processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 7O0m/vBxMX56uaGt1ySddyVLKBp4ifti2ILbUFdJ8SIV63szwT78lnb+OSlO7LDa8F9kpL7sawfx3Efg+nbh151yvtry5dEToQrYizg3aklUPBTpkXzuEjIyN4awdWvTQIedzVheJurfJE+faDBFO1YkDpz45cwHPQeYLui6vaCsq7Wgb6UhOwfAFzeCUCun+83ClZnPem4cWcYLNfaKWwp7ycmXtK89yEgKrkcL198mWW1rpEiTXZ2EMuckfcAqIUx946nkHpNmaBVm8c47IUHNXFQysFE2KxYEeJHuH3C2TIiglw9Ncc23EH8Rbfd3rt/XfgOLnWtAogWHdsynjiLN2b8x5/nzacdLyaQNdhF3HtbKM9+k3v1YSsMFtdjmNHS70c7Ay9PKBQhhaYgDagh+BMrV+SycKjtrhxN0y47G1CUSSjQ7JO9DlzV6mV03gpzDSMKn61+io1W5tM7gug027K4yOIJcfX4bR800KFiuOvcLGW/3Id4Q/HwSA5pNZRKbc1U0VTZsFvRSHKHb3ylPxQ7GUQSHaUNiMs1D3RcODe7k99/GNorU8DkGoVCLI5JruBYh5aPTW8Y9Qtv3JMv+u5wTCeDTMf/dqYlmc4uRbDoOMjSlAn3lTpYN4iCe
x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:;
 IPV:NLI; SFV:NSPM; H:IA1PR12MB8311.namprd12.prod.outlook.com; PTR:; CAT:NONE;
 SFS:(13230031)(396003)(376002)(136003)(39860400002)(366004)(346002)(230922051799003)(186009)(451199024)(64100799003)(1800799012)(66446008)(66476007)(76116006)(66556008)(66946007)(6916009)(64756008)(4326008)(52536014)(54906003)(316002)(8936002)(26005)(83380400001)(478600001)(6506007)(9686003)(55236004)(71200400001)(7696005)(8676002)(5660300002)(2906002)(41300700001)(33656002)(38070700009)(122000001)(38100700002)(86362001)(55016003);
 DIR:OUT; SFP:1101; 
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?CIGTwBEI/ze0FqHqoxxcjxYG6Domy1ICkrb7E7dgo6CRaGqxQZB1Lf2UyIKM?=
 =?us-ascii?Q?VVZzvG5JTFWMhRxH7lvCOXQB+FCYq7XTwLuI564kgWDOWtlec/Z5itKjbTXs?=
 =?us-ascii?Q?nyo1KnKBf23V2aL5LE2da5bNBoMybyhFHIkNQtqal0tHZ7GucNCTCkJ0Yl/t?=
 =?us-ascii?Q?49/tXTlym+cvGmFBMUYIwU2AOLs3aBI9sqybytNPp0uZOxcRmiDyRWNGHHJb?=
 =?us-ascii?Q?qbHPLOPRslQVeg9G/Ap4KCJmroC8lJo1EcrqZqH5fwCTXvnYYVnjsApgQCL/?=
 =?us-ascii?Q?toEbLfJ7+Rn4oBcHApJuA9nfwCOxRKza3L3OhnPiqJN1+TC0ks+NbB9wCmGg?=
 =?us-ascii?Q?7YBGX+SSIK71/+G9tyVAxlkFXt+PlPtVmTsAhGmlkm64Nypoi1NRPg099Ti+?=
 =?us-ascii?Q?6ML27BXyE0fBH+uWQ+x22tKO4B9tKIjdV/4BXY2ypA/N/bb5R9VAQib8tXZZ?=
 =?us-ascii?Q?2QAL0ScITO8kqXXQbG0Oilq3e98XHRdM8NyXBMWuc6kpWvT9uVsbuyGmrlxZ?=
 =?us-ascii?Q?B6m5TcrgsvreFAP/Rb/O/xcmWP5g+5YcTf9yRZXbZBHtw23SOryM+CWwHZiN?=
 =?us-ascii?Q?y+loxw6pi8dwEIEd8uyXfVs/JKUu8RdkJuwAkuvbDJQpY2XA7kqXXfxWBZbn?=
 =?us-ascii?Q?5Eb7rXLg5Kio9szv4o4lOx4lSWq2sg+f7TK+IViZZ/dx4+GmHyNqDhn2kS65?=
 =?us-ascii?Q?5OjJGpr6wbOKz9yNztmxyySA9M+UQOcvMuHJvR4OUAEmNslSxzt9j45kGg6W?=
 =?us-ascii?Q?ZV1oqMxCdlDQGG/Ang/Hm6VNYoXDYAXYpQYr+HR3GfiOZWAemsZoM4ftieLD?=
 =?us-ascii?Q?7L/c0xqAm8IScwortdB07EJmYFOeJ+mccIlQoE34IWHrM7+AMC0C/qItmRcT?=
 =?us-ascii?Q?RGbQlWe9Ad5SlBW6qFvELMz5mTFb8r+fhXlYMcBRnBfIJZKVSPWALkT5u+bc?=
 =?us-ascii?Q?eY/XH//vKQ9O0rKnlOpB5VGjpUkop4wgOOcsnL0+pkahW0M7xzg4+MNtojrj?=
 =?us-ascii?Q?Kx2lxKUvNyqvwQcWsHPc1QETQMKumQjuOaodhp9Mi0+/NcQIZaLtNUBX6k6N?=
 =?us-ascii?Q?W2s5mbodf1WBkDDpq2VFPDEeQFFaay+1lYzN39IvBluD60h3k6dw4EkqcxNi?=
 =?us-ascii?Q?j44EW+VKnj0SVzU+eHD7q/XYFr07RjJXwmOe1PmsHg9DY4NYdnEAzDnU1FjY?=
 =?us-ascii?Q?kwa/LzR16y45WLB9V3//VzBwtiEyCKHeuVuiUfg86NYsmuUHs0eydpkDAra8?=
 =?us-ascii?Q?BwLzkZSr09mdGGltyem/+7os8zlUYcDpI29LMpbJ2MJuJkay+Q7qtOV3HfBd?=
 =?us-ascii?Q?U7b2GhGvVNCW3x915WCNSAuK/r9fY6bLn/MKiGUjySMLFrJFve/00kVac/eh?=
 =?us-ascii?Q?Gr+kznDmeMT6GqTQXxk6Aq55MFjte61rIeiD021ohUldhD+JuGAwjpLWFzDJ?=
 =?us-ascii?Q?uJgQvoBDlLxLVe+NKuw9nTMk+ATiTJ7LhU4zbF2WYzvHaOfc6WPQb0gcZgHI?=
 =?us-ascii?Q?3aJ5dR2tWuUB2/EyRnEKGxOUhIhsfpEI7PrqvM4qdgBaESKV1W8LPO4oSTdN?=
 =?us-ascii?Q?CVnpAI/3SOB8r40ya1JQF8V3m3XiEePgOv49qz2X?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: IA1PR12MB8311.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 9c083899-b519-463f-786f-08dc07ac7043
X-MS-Exchange-CrossTenant-originalarrivaltime: 28 Dec 2023 13:53:57.8453 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: F0z3V5sNz1264HCjboLtxOm9HQksKEEGhhZ1u/6li0EJDrBD4BPFuzVPExaTiHUytHz/40l8vyi17hnhlZdBbA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB6541
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org

Hi Stephen,

> > +/**
> > + * @internal
> > + *
> > + * Fast path async flow functions and related data are held in a flat =
array,
> one entry per ethdev.
> > + * It is assumed that each entry is read-only and cache aligned.
> > + */
> > +struct rte_flow_fp_ops {
> > +     void *ctx;
> > +     rte_flow_async_create_t async_create;
> > +     rte_flow_async_create_by_index_t async_create_by_index;
> > +     rte_flow_async_actions_update_t async_actions_update;
> > +     rte_flow_async_destroy_t async_destroy;
> > +     rte_flow_push_t push;
> > +     rte_flow_pull_t pull;
> > +     rte_flow_async_action_handle_create_t async_action_handle_create;
> > +     rte_flow_async_action_handle_destroy_t async_action_handle_destro=
y;
> > +     rte_flow_async_action_handle_update_t async_action_handle_update;
> > +     rte_flow_async_action_handle_query_t async_action_handle_query;
> > +     rte_flow_async_action_handle_query_update_t
> async_action_handle_query_update;
> > +     rte_flow_async_action_list_handle_create_t
> async_action_list_handle_create;
> > +     rte_flow_async_action_list_handle_destroy_t
> async_action_list_handle_destroy;
> > +     rte_flow_async_action_list_handle_query_update_t
> async_action_list_handle_query_update;
> > +} __rte_cache_aligned;
>=20
> If you go ahead with this then all usage should be const pointer.
> Still think it is bad idea and creates even more technical debt.
Could you please elaborate a bit more on why do you think it is a bad idea =
and why it creates technical debt?=20

> > **Future considerations**
> >
> > A case can be made about converting some of the existing stable API=20
> > functions to fast path versions in future LTS releases.
> > I don't have any hard data on how such changes would affect=20
> > performance of these APIs, but I assume that the improvement would be n=
oticeable.
>=20
> The problem is that inline functions create future ABI problems.
Agreed, this is why such a change can only be introduced when a new major A=
BI version is released.
However, even though inlining could reduce function call overhead, I'm not =
sure if such a change is needed for synchronous flow API.
I mentioned it here as a thing to consider.

> Usually, there are other ways to get the same performance benefit.
> Often batching updates to hardware will do the trick.
This is true, but we still have some library-level overhead.
To elaborate more, the current state of flow API is as follows:
- With sync flow API:
  - Applications cannot batch flow operations.
- With async flow APIs:
  - Applications can enqueue multiple flow operations and PMDs can issue ba=
tches to HW.
  - But there is still one function call per enqueued flow operation.
    The overall API overhead in datapath may be nonnegligible if we conside=
r that applications may enqueue a flow rule creation operation for every pa=
cket received in SW.
This proposal specifically aims at reducing API overhead for async flow API=
 in a case mentioned above.

As a side note, we plan to push changes to mlx5 PMD in 24.03 which will red=
uce PMD overhead in such scenarios.
This proposal's goal is to reduce overhead of async flow API at library lev=
el.

Best regards,
Dariusz Sosnowski