From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 89884A034C; Wed, 21 Dec 2022 14:12:58 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 70D0440A7F; Wed, 21 Dec 2022 14:12:58 +0100 (CET) Received: from mail-ua1-f50.google.com (mail-ua1-f50.google.com [209.85.222.50]) by mails.dpdk.org (Postfix) with ESMTP id D284540A7A for ; Wed, 21 Dec 2022 14:12:56 +0100 (CET) Received: by mail-ua1-f50.google.com with SMTP id c26so3582277uak.5 for ; Wed, 21 Dec 2022 05:12:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=svMAf4003PH8KX/jrfRyQvo1tlfrHF7LN7VxmpbFXjk=; b=Ym4DsdK/rvxGS5nrtMJZIKIkb18GZCNvbogQtBZtaIIR/kzvxjUQfXI2KBo3rrOrRv KQ0IxVE6bVeBBwgEyvTRUJuY7YlzOIa6G4n53z8U8JnuqjaHyBUqJyXpp+k/b4JYGt0+ LWe9UNuVV4vwcrePyQ+5BimBMMOhgch3Cs8pkW45LIFGTm5anPa+s/Z732j30oHQTCk/ fFQKtsgFTGwXuuVh0NoZyjNcn9/jjQrD0iONAPEqg9MEMFDhp5XsdCeafAycarZPzEQz JBoCGrx7EOh7qtAIDl/oD8qrKsmOIfeKSXm4ALWR6nt9/11hx6Z+D9Xf3M+QfLpJJwRi FxOw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=svMAf4003PH8KX/jrfRyQvo1tlfrHF7LN7VxmpbFXjk=; b=oMkQoYMLEn5qUATHPDCKB3Gg1kWNoNnZz1w06oOdJmEXToQa1DopW7YK9CfUETAS0K 8fQuTU9AD7aiWp4Nbi0dnn/mjIscKAWf0ND5ArK09Uu60uDGyHJQxqhypHNaCkvUOROe Kv1eMLjAxW0L1RTMvf517+mT1I9AvLY5zanslflU/634iu9AP/DCOJUSUJOlnyPyjn9B mnCQNsnBouDw3ZAhtr0IwWxPnYR45lcwb90bgdjNP612VY47LsDIsyDXOinFdGV9K3In XQRIJ3lIkDPOV4xpEyTgM4N1nB1y05NsqE+J/5Gy8YnFEdvXJ+kq9gBwgvSPNWkMTQ9L 90VQ== X-Gm-Message-State: AFqh2kra7kwtwjtbWs5kd9w2AClN08uXrw/Cc2DvsvO5xLUOHqpmOzkA g9DiC6a1vzq0G2k4DpUpqLpTZLTBB2yqKq9w6vg= X-Google-Smtp-Source: AMrXdXtbu8hv7Nbxf9TETme+fQ3rV/hp6xg88ODOpF0NS6v5PLOdD+IQWqDoTP4B+yLurJFaQamZlKk88/3PQdVH97o= X-Received: by 2002:ab0:3b42:0:b0:42f:a69a:3b50 with SMTP id o2-20020ab03b42000000b0042fa69a3b50mr127551uaw.107.1671628376099; Wed, 21 Dec 2022 05:12:56 -0800 (PST) MIME-Version: 1.0 References: <20221205215416.7ac53a55@hermes.local> <20221221090017.3715030-1-rongweil@nvidia.com> <20221221090017.3715030-3-rongweil@nvidia.com> In-Reply-To: From: Jerin Jacob Date: Wed, 21 Dec 2022 18:42:29 +0530 Message-ID: Subject: Re: [RFC v3 2/2] ethdev: add API to set process to active or standby To: Rongwei Liu Cc: Matan Azrad , Slava Ovsiienko , Ori Kam , "NBU-Contact-Thomas Monjalon (EXTERNAL)" , Ferruh Yigit , Andrew Rybchenko , "dev@dpdk.org" , Raslan Darawsheh Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Wed, Dec 21, 2022 at 6:20 PM Rongwei Liu wrote: > > HI Jerin: > > BR > Rongwei > > > -----Original Message----- > > From: Jerin Jacob > > Sent: Wednesday, December 21, 2022 20:45 > > To: Rongwei Liu > > Cc: Matan Azrad ; Slava Ovsiienko > > ; Ori Kam ; NBU-Contact- > > Thomas Monjalon (EXTERNAL) ; Ferruh Yigit > > ; Andrew Rybchenko > > ; dev@dpdk.org; Raslan Darawsheh > > > > Subject: Re: [RFC v3 2/2] ethdev: add API to set process to active or s= tandby > > > > External email: Use caution opening links or attachments > > > > > > On Wed, Dec 21, 2022 at 5:35 PM Rongwei Liu wrote= : > > > > > > Hi Jerin: > > > > > > BR > > > Rongwei > > > > > > > -----Original Message----- > > > > From: Jerin Jacob > > > > Sent: Wednesday, December 21, 2022 19:00 > > > > To: Rongwei Liu > > > > Cc: Matan Azrad ; Slava Ovsiienko > > > > ; Ori Kam ; NBU-Contact- > > > > Thomas Monjalon (EXTERNAL) ; Ferruh Yigit > > > > ; Andrew Rybchenko > > > > ; dev@dpdk.org; Raslan Darawsheh > > > > > > > > Subject: Re: [RFC v3 2/2] ethdev: add API to set process to active > > > > or standby > > > > > > > > External email: Use caution opening links or attachments > > > > > > > > > > > > On Wed, Dec 21, 2022 at 3:02 PM Rongwei Liu > > wrote: > > > > > > > > > > HI Jerin: > > > > > > > > > > > > > Hi Rongwei > > > > > > > > > BR > > > > > Rongwei > > > > > > > > > > > -----Original Message----- > > > > > > From: Jerin Jacob > > > > > > Sent: Wednesday, December 21, 2022 17:13 > > > > > > To: Rongwei Liu > > > > > > Cc: Matan Azrad ; Slava Ovsiienko > > > > > > ; Ori Kam ; > > > > > > NBU-Contact- Thomas Monjalon (EXTERNAL) ; > > > > > > Ferruh Yigit ; Andrew Rybchenko > > > > > > ; dev@dpdk.org; Raslan Darawsheh > > > > > > > > > > > > Subject: Re: [RFC v3 2/2] ethdev: add API to set process to > > > > > > active or standby > > > > > > > > > > > > External email: Use caution opening links or attachments > > > > > > > > > > > > > > > > > > On Wed, Dec 21, 2022 at 2:31 PM Rongwei Liu > > > > > > > > > > wrote: > > > > > > > > > > > > > > Users may want to change the DPDK process to different > > > > > > > versions > > > > > > > > > > > > Different version of DPDK? If there is any ABI change how to su= pport > > this? > > > > > > > > > > > There is a new member which was introduced into rte_eth_dev_info > > > > > but it > > > > shouldn=E2=80=99t be ABI breaking since using reserved fields. > > > > > > > > That is just for rte_eth_dev_info. What about the ABI change in > > > > different ethdev structure and rte_flow structures across different= DPDK > > ABI versions. > > > > > > > Besides this, there is no other ABI changes dependency. > > > > > > Assume there is a DPDK process A running with version v21.11 and plan > > > to upgrade to version v22.11. Let' call v22.11 as process B. > > > > OK. That's a relief. I understand the use case now. > > > > Why not simply use standard DPDK multiprocess model then. > > Primary process act as server for slow path API. Secondary process can = come > > and go(aka can be updated at runtime) and use as client to update rules= via > > primary-secondray communication mechanism. > > > Just image if process A and process B have ABI breakage like different rt= e_flow_item_*** and rte_flow_action_*** size and members. > How can we quickly accommodate primary/secondary to be ABI compatible acr= oss different versions? > It will be very huge effort and difficult to implement, at least in my op= inion. > What do you think? Yes. it difficult what ever approach we take, On other hand, ethdev subsystem has other components like rte_tm and other offload etc, We can not simply have rte_eth_process_set_active() and things magical works across different DPDK versions. Example, if rte_flow rule has meter action will depend on another HW piece in NIC card for doing the metering but set by flow API. IMO, Customer can simply use standard multiprocess model if version are compatible without any special intelligence in application. Otherwise they can reload and start the application again or have special intelligence in application to cater the specific area of API they need to leverage based on server and client DPDK versions. > > > > > > > > Now, process A has been running for long time and has lot of rules > > configured. It' "active" role per this API definition. > > > Process B starts and it should call this API and set itself to > > > "standby" role and user can program the flow rules as they want and > > different NIC vendors may have different recommendations. Nvidia sugges= ts > > only program process B with group 0' rules now. > > > > > > The user should sync all desired configurations from process A to > > > process B, and process A starts to yield traffic like "delete all gro= up 0 rules > > for Nvidia' NICs" or quit. > > > After that process B calls this API and set itself to "active" role, = now the hot- > > upgrade finishes. > > > > > > > > > > such as hot upgrade. > > > > > > > There is a strong requirement to simplify the logic and > > > > > > > shorten the traffic downtime as much as possible. > > > > > > > > > > > > > > This update introduces new rte_eth process role definitions: > > > > > > > active or standby. > > > > > > > > > > > > > > The active role means rules are programmed to HW immediately, > > > > > > > and no > > > > > > > > > > > > Why it has to be specific only to rte_flow rule? If it spedieic > > > > > > to rte_flow, why it is in rte_eth_process_ name space? > > > > > For now, this design focuses on the flow rule offloading and > > > > > traffic > > > > redirection. > > > > > When switching process version, it' important to make sure which > > > > application receives and handles the traffic. > > > > > > > > Changing the DPDK version runtime is just beyond rte_flow driver. > > > > > > It' not about changing DPDK version but upgrading DPDK from one PMD > > version to another one. > > > Does the preceding example answer your question? > > > > > > > > > The changing should be effective across all probing eth devices, > > > > > that' why it > > > > was put under rte_eth_process_ (for all rte_eth_dev) name space. > > > > > > > > > > > > Also, if we are moving the standby, What about the rule whose > > > > > > ABI is changed between versions? > > > > > > > > > > Like the comments mentioned: " Before role transition, all the > > > > > rules set by > > > > the active process should be flushed first. " > > > > > > > > What happens to rte_flow flow handles for existing ones which is > > > > created with version X? > > > > Also What if new version Y has ABI change in rte_flow_pattern and > > > > rte_flow_action structure? > > > > > > > > For me, If DPDK version change is needed, simply reload the > > > > application. This API will soon bloat, and it will be a mess if to > > > > start handling Different DPDK version which is not ABI compatible a= t all. > > > > > > > Yes, you are right. Reloading the application is the easiest way but > > > it may have a long time Window that traffic is lost. No traffic arriv= es at > > process A or process B. > > > We are trying to simplify the reloading logic and minimize the traffi= c down > > time as much as possible. > > > The approach may differentiate hugely between different NIC vendors, > > > so I think it should be better if DPDK can provide an abstract API. > > > > > > If process A and process B are ABI different, it doesn't matter. > > > 1. Call this API with process A means older ABI. > > > 2. Call this API with process B means newer ABI. > > > It' have process concept and working scope. > > > > > > > > > > > > > > > > > > > > > > behavior changed. This is the default state. > > > > > > > The standby role means rules are queued in the HW. If no > > > > > > > active roles alive or back to active, the rules are effective > > immediately. > > > > > > > > > > > > > > Signed-off-by: Rongwei Liu