From: Rongwei Liu <rongweil@nvidia.com>
To: Jerin Jacob <jerinjacobk@gmail.com>
Cc: Matan Azrad <matan@nvidia.com>,
Slava Ovsiienko <viacheslavo@nvidia.com>,
Ori Kam <orika@nvidia.com>,
"NBU-Contact-Thomas Monjalon (EXTERNAL)" <thomas@monjalon.net>,
Ferruh Yigit <ferruh.yigit@amd.com>,
Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>,
"dev@dpdk.org" <dev@dpdk.org>,
Raslan Darawsheh <rasland@nvidia.com>
Subject: RE: [RFC v3 2/2] ethdev: add API to set process to active or standby
Date: Wed, 21 Dec 2022 12:50:40 +0000 [thread overview]
Message-ID: <BN9PR12MB5273C8827885EDFFE2C3FC35ABEB9@BN9PR12MB5273.namprd12.prod.outlook.com> (raw)
In-Reply-To: <CALBAE1MZgi0Yd2S2KUBJjGZiYu4h8RfbFL8JKRA2ZPCoJsYeEg@mail.gmail.com>
HI Jerin:
BR
Rongwei
> -----Original Message-----
> From: Jerin Jacob <jerinjacobk@gmail.com>
> Sent: Wednesday, December 21, 2022 20:45
> To: Rongwei Liu <rongweil@nvidia.com>
> Cc: Matan Azrad <matan@nvidia.com>; Slava Ovsiienko
> <viacheslavo@nvidia.com>; Ori Kam <orika@nvidia.com>; NBU-Contact-
> Thomas Monjalon (EXTERNAL) <thomas@monjalon.net>; Ferruh Yigit
> <ferruh.yigit@amd.com>; Andrew Rybchenko
> <andrew.rybchenko@oktetlabs.ru>; dev@dpdk.org; Raslan Darawsheh
> <rasland@nvidia.com>
> Subject: Re: [RFC v3 2/2] ethdev: add API to set process to active or standby
>
> External email: Use caution opening links or attachments
>
>
> On Wed, Dec 21, 2022 at 5:35 PM Rongwei Liu <rongweil@nvidia.com> wrote:
> >
> > Hi Jerin:
> >
> > BR
> > Rongwei
> >
> > > -----Original Message-----
> > > From: Jerin Jacob <jerinjacobk@gmail.com>
> > > Sent: Wednesday, December 21, 2022 19:00
> > > To: Rongwei Liu <rongweil@nvidia.com>
> > > Cc: Matan Azrad <matan@nvidia.com>; Slava Ovsiienko
> > > <viacheslavo@nvidia.com>; Ori Kam <orika@nvidia.com>; NBU-Contact-
> > > Thomas Monjalon (EXTERNAL) <thomas@monjalon.net>; Ferruh Yigit
> > > <ferruh.yigit@amd.com>; Andrew Rybchenko
> > > <andrew.rybchenko@oktetlabs.ru>; dev@dpdk.org; Raslan Darawsheh
> > > <rasland@nvidia.com>
> > > Subject: Re: [RFC v3 2/2] ethdev: add API to set process to active
> > > or standby
> > >
> > > External email: Use caution opening links or attachments
> > >
> > >
> > > On Wed, Dec 21, 2022 at 3:02 PM Rongwei Liu <rongweil@nvidia.com>
> wrote:
> > > >
> > > > HI Jerin:
> > > >
> > >
> > > Hi Rongwei
> > >
> > > > BR
> > > > Rongwei
> > > >
> > > > > -----Original Message-----
> > > > > From: Jerin Jacob <jerinjacobk@gmail.com>
> > > > > Sent: Wednesday, December 21, 2022 17:13
> > > > > To: Rongwei Liu <rongweil@nvidia.com>
> > > > > Cc: Matan Azrad <matan@nvidia.com>; Slava Ovsiienko
> > > > > <viacheslavo@nvidia.com>; Ori Kam <orika@nvidia.com>;
> > > > > NBU-Contact- Thomas Monjalon (EXTERNAL) <thomas@monjalon.net>;
> > > > > Ferruh Yigit <ferruh.yigit@amd.com>; Andrew Rybchenko
> > > > > <andrew.rybchenko@oktetlabs.ru>; dev@dpdk.org; Raslan Darawsheh
> > > > > <rasland@nvidia.com>
> > > > > Subject: Re: [RFC v3 2/2] ethdev: add API to set process to
> > > > > active or standby
> > > > >
> > > > > External email: Use caution opening links or attachments
> > > > >
> > > > >
> > > > > On Wed, Dec 21, 2022 at 2:31 PM Rongwei Liu
> > > > > <rongweil@nvidia.com>
> > > wrote:
> > > > > >
> > > > > > Users may want to change the DPDK process to different
> > > > > > versions
> > > > >
> > > > > Different version of DPDK? If there is any ABI change how to support
> this?
> > > > >
> > > > There is a new member which was introduced into rte_eth_dev_info
> > > > but it
> > > shouldn’t be ABI breaking since using reserved fields.
> > >
> > > That is just for rte_eth_dev_info. What about the ABI change in
> > > different ethdev structure and rte_flow structures across different DPDK
> ABI versions.
> > >
> > Besides this, there is no other ABI changes dependency.
> >
> > Assume there is a DPDK process A running with version v21.11 and plan
> > to upgrade to version v22.11. Let' call v22.11 as process B.
>
> OK. That's a relief. I understand the use case now.
>
> Why not simply use standard DPDK multiprocess model then.
> Primary process act as server for slow path API. Secondary process can come
> and go(aka can be updated at runtime) and use as client to update rules via
> primary-secondray communication mechanism.
>
Just image if process A and process B have ABI breakage like different rte_flow_item_*** and rte_flow_action_*** size and members.
How can we quickly accommodate primary/secondary to be ABI compatible across different versions?
It will be very huge effort and difficult to implement, at least in my opinion.
What do you think?
>
> >
> > Now, process A has been running for long time and has lot of rules
> configured. It' "active" role per this API definition.
> > Process B starts and it should call this API and set itself to
> > "standby" role and user can program the flow rules as they want and
> different NIC vendors may have different recommendations. Nvidia suggests
> only program process B with group 0' rules now.
> >
> > The user should sync all desired configurations from process A to
> > process B, and process A starts to yield traffic like "delete all group 0 rules
> for Nvidia' NICs" or quit.
> > After that process B calls this API and set itself to "active" role, now the hot-
> upgrade finishes.
> >
> > > > > > such as hot upgrade.
> > > > > > There is a strong requirement to simplify the logic and
> > > > > > shorten the traffic downtime as much as possible.
> > > > > >
> > > > > > This update introduces new rte_eth process role definitions:
> > > > > > active or standby.
> > > > > >
> > > > > > The active role means rules are programmed to HW immediately,
> > > > > > and no
> > > > >
> > > > > Why it has to be specific only to rte_flow rule? If it spedieic
> > > > > to rte_flow, why it is in rte_eth_process_ name space?
> > > > For now, this design focuses on the flow rule offloading and
> > > > traffic
> > > redirection.
> > > > When switching process version, it' important to make sure which
> > > application receives and handles the traffic.
> > >
> > > Changing the DPDK version runtime is just beyond rte_flow driver.
> >
> > It' not about changing DPDK version but upgrading DPDK from one PMD
> version to another one.
> > Does the preceding example answer your question?
> > >
> > > > The changing should be effective across all probing eth devices,
> > > > that' why it
> > > was put under rte_eth_process_ (for all rte_eth_dev) name space.
> > > > >
> > > > > Also, if we are moving the standby, What about the rule whose
> > > > > ABI is changed between versions?
> > > >
> > > > Like the comments mentioned: " Before role transition, all the
> > > > rules set by
> > > the active process should be flushed first. "
> > >
> > > What happens to rte_flow flow handles for existing ones which is
> > > created with version X?
> > > Also What if new version Y has ABI change in rte_flow_pattern and
> > > rte_flow_action structure?
> > >
> > > For me, If DPDK version change is needed, simply reload the
> > > application. This API will soon bloat, and it will be a mess if to
> > > start handling Different DPDK version which is not ABI compatible at all.
> > >
> > Yes, you are right. Reloading the application is the easiest way but
> > it may have a long time Window that traffic is lost. No traffic arrives at
> process A or process B.
> > We are trying to simplify the reloading logic and minimize the traffic down
> time as much as possible.
> > The approach may differentiate hugely between different NIC vendors,
> > so I think it should be better if DPDK can provide an abstract API.
> >
> > If process A and process B are ABI different, it doesn't matter.
> > 1. Call this API with process A means older ABI.
> > 2. Call this API with process B means newer ABI.
> > It' have process concept and working scope.
> >
> > >
> > >
> > >
> > > > > > behavior changed. This is the default state.
> > > > > > The standby role means rules are queued in the HW. If no
> > > > > > active roles alive or back to active, the rules are effective
> immediately.
> > > > > >
> > > > > > Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
next prev parent reply other threads:[~2022-12-21 12:50 UTC|newest]
Thread overview: 54+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-12-01 8:20 [RFC 0/2] add API to set process to primary or secondary Rongwei Liu
2022-12-01 8:20 ` [RFC 1/2] ethdev: add group description Rongwei Liu
2022-12-01 8:20 ` [RFC 2/2] ethdev: add API to set process to primary or secondary Rongwei Liu
2022-12-01 15:10 ` Stephen Hemminger
2022-12-02 3:27 ` Rongwei Liu
2022-12-05 16:08 ` Stephen Hemminger
2022-12-06 3:47 ` Rongwei Liu
2022-12-06 5:54 ` Stephen Hemminger
2022-12-21 9:00 ` [RFC v3 0/2] add API to set process to active or standby Rongwei Liu
2022-12-21 9:00 ` [RFC v3 1/2] ethdev: add group description Rongwei Liu
2023-01-18 15:44 ` [PATCH v4 0/3] add API for live migration Rongwei Liu
2023-01-18 15:44 ` [PATCH v4 1/3] ethdev: add flow rule group description Rongwei Liu
2023-01-31 11:53 ` Ori Kam
2023-02-06 12:15 ` Rongwei Liu
2023-02-07 2:57 ` [PATCH v5] " Rongwei Liu
2023-02-08 20:28 ` Ferruh Yigit
2023-02-09 2:06 ` Rongwei Liu
2023-02-09 7:32 ` [PATCH v6] " Rongwei Liu
2023-02-09 8:01 ` Ori Kam
2023-02-09 11:26 ` Ferruh Yigit
2023-01-18 15:44 ` [PATCH v4 2/3] ethdev: add standby state for live migration Rongwei Liu
2023-01-31 13:50 ` Ori Kam
2023-01-31 18:14 ` Jerin Jacob
2023-01-31 22:55 ` Thomas Monjalon
2023-02-01 7:32 ` Andrew Rybchenko
2023-02-01 8:31 ` Thomas Monjalon
2023-02-01 8:40 ` Jerin Jacob
2023-02-01 8:46 ` Thomas Monjalon
2023-02-02 10:23 ` Rongwei Liu
2023-02-01 7:52 ` Andrew Rybchenko
2023-02-01 8:27 ` Thomas Monjalon
2023-02-01 8:40 ` Andrew Rybchenko
2023-01-18 15:44 ` [PATCH v4 3/3] ethdev: add standby flags " Rongwei Liu
2023-01-23 13:20 ` Jerin Jacob
2023-01-30 2:47 ` Rongwei Liu
2023-01-30 17:10 ` Jerin Jacob
2023-01-31 2:53 ` Rongwei Liu
2023-01-31 8:45 ` Jerin Jacob
2023-01-31 9:01 ` Rongwei Liu
2023-01-31 14:37 ` Jerin Jacob
2023-01-31 14:45 ` Ori Kam
2023-01-31 17:50 ` Thomas Monjalon
2023-01-31 18:10 ` Jerin Jacob
2022-12-21 9:00 ` [RFC v3 2/2] ethdev: add API to set process to active or standby Rongwei Liu
2022-12-21 9:12 ` Jerin Jacob
2022-12-21 9:32 ` Rongwei Liu
2022-12-21 10:59 ` Jerin Jacob
2022-12-21 12:05 ` Rongwei Liu
2022-12-21 12:44 ` Jerin Jacob
2022-12-21 12:50 ` Rongwei Liu [this message]
2022-12-21 13:12 ` Jerin Jacob
2022-12-21 14:33 ` Rongwei Liu
2022-12-26 16:44 ` Ori Kam
2023-01-15 22:46 ` Thomas Monjalon
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=BN9PR12MB5273C8827885EDFFE2C3FC35ABEB9@BN9PR12MB5273.namprd12.prod.outlook.com \
--to=rongweil@nvidia.com \
--cc=andrew.rybchenko@oktetlabs.ru \
--cc=dev@dpdk.org \
--cc=ferruh.yigit@amd.com \
--cc=jerinjacobk@gmail.com \
--cc=matan@nvidia.com \
--cc=orika@nvidia.com \
--cc=rasland@nvidia.com \
--cc=thomas@monjalon.net \
--cc=viacheslavo@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).