From: Jerin Jacob <jerinjacobk@gmail.com>
To: Thomas Monjalon <thomas@monjalon.net>
Cc: Rongwei Liu <rongweil@nvidia.com>, Ori Kam <orika@nvidia.com>,
dev@dpdk.org, Matan Azrad <matan@nvidia.com>,
Slava Ovsiienko <viacheslavo@nvidia.com>,
"stephen@networkplumber.org" <stephen@networkplumber.org>,
Raslan Darawsheh <rasland@nvidia.com>,
Ferruh Yigit <ferruh.yigit@amd.com>,
Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Subject: Re: [PATCH v4 3/3] ethdev: add standby flags for live migration
Date: Tue, 31 Jan 2023 23:40:27 +0530 [thread overview]
Message-ID: <CALBAE1MoQx4LwNJc4qXxNepVcLqL308kkvc34+V2xVd5LA8vUg@mail.gmail.com> (raw)
In-Reply-To: <8213257.NyiUUSuA9g@thomas>
On Tue, Jan 31, 2023 at 11:20 PM Thomas Monjalon <thomas@monjalon.net> wrote:
>
> 31/01/2023 15:45, Ori Kam:
> > From: Jerin Jacob <jerinjacobk@gmail.com>
> > > On Tue, Jan 31, 2023 at 2:31 PM Rongwei Liu <rongweil@nvidia.com> wrote:
> > > > From: Jerin Jacob <jerinjacobk@gmail.com>
> > > > > On Tue, Jan 31, 2023 at 8:23 AM Rongwei Liu <rongweil@nvidia.com>
> > > > > > From: Jerin Jacob <jerinjacobk@gmail.com>
> > > > > > > On Mon, Jan 30, 2023 at 8:17 AM Rongwei Liu
> > > > > > > > From: Jerin Jacob <jerinjacobk@gmail.com>
> > > > > > > > > On Wed, Jan 18, 2023 at 9:15 PM Rongwei Liu
> > > > > > > > > > +/**@{@name Process role flags
> > > > > > > > > > + * used when migrating from an application to another one.
> > > > > > > > > > + * @see rte_eth_process_set_active */
> > > > > > > > > > +/**
> > > > > > > > > > + * When set on a standby process, ingress flow rules will be
> > > > > > > > > > +effective
> > > > > > > > > > + * in active and standby processes, so the ingress traffic
> > > > > > > > > > +may be duplicated.
> > > > > > > > > > + */
> > > > > > > > > > +#define RTE_ETH_PROCESS_FLAG_STANDBY_DUP_FLOW_INGRESS RTE_BIT32(0)
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > How to duplicate if action has statefull items for example,
> > > > > > > > > rte_flow_action_security::security_session -> it store the live
> > > > > > > > > pointer rte_flow_action_meter::mtr_id; -> MTR object ID created
> > > > > > > > > with
> > > > > > > > > rte_mtr_create()
> > > > > > > > I agree with you, not all actions can be supported in the
> > > > > > > > active/standby model.
> > > > > > >
> > > > > > > IMO, Where ever rules are not standalone (like QUEUE, RSS) etc, It
> > > > > > > will be architecturally is not possible to migrate with pointers.
> > > > > > > That's where I have concern generalizing this feature for this ethdev.
> > > > > > >
> > > > > > Not sure I understand your concern correctly. What' the pointer concept here?
> > > > >
> > > > > I meant, Any HW resource driver deals with "pointers" or "fixed ID"
> > > > > can not get the same value
> > > > > for the new application. That's where I believe this whole concepts works
> > > > > for very standalone rte_flow patterns and actions.
> > > > >
> > > > > > Queue RSS actions can be migrated per my local test. Active/Standby
> > > > > application have its fully own rxq/txq.
> > > > >
> > > > > Yes. It because it is standalone.
> > > > >
> > > > > > They are totally separated processes and like two members in pipeline.
> > > > > > 2nd member can't be feed if 1st member alive and handle the traffic.
> > > > > >
> [...]
> > > > > > > my view, it should be generic utils functions to track the flow and
> > > > > > > installing the rules using rte_flow APIs and keep the scope only for
> > > > > > > rte_flow.
> > > > > >
> > > > > > For rules part, totally agree with you. Issue is there maybe millions
> > > > > > of flow rules in field and each rule may take different steps
> > > > > > to re-install per vendor' implementations.
> > > > >
> > > > > I understand the desire for millon flow migrations. Which makes sense.
> > > > > IMO, It may be just easy to make this feature just for rte_flow name space.
> > > > > Just have APIs to export() existing rules for the given port
> > > > > and import() the rules
> > > > > exported rather than going to ethdev space and call it as "live migration".
> > > > >
> > > > Do you mean the API naming should be "rte_flow_process_set_role()"
> > > > instead of "rte_eth_process_set_role()" ?
> > > > Also move to rte_flow.c/.h files? Are we good to keep the PMD callback
> > > > in eth_dev layer?
> > >
> > > Yes. something with rte_flow_ prefix and not sure _set_role() kind of
> > > scheme.
> >
> > I think that the process of upgrade relates to the entire port and not only the rte_flow,
> > I don't mind that this flag will be part of rte_flow, but it looks like this information is in higher level.
>
> I agree, application migration is a high-level concept.
> For now we see that we can take advantage of it for some flow rules.
> It could help more use cases.
>
> I also agree that it is not a full solution.
> Migration is complex, that's sure we cannot solve it in few weeks,
> and we'll need to add more functions and helpers to make it easy to use
> in more cases.
Makes sense.
>
>
> > > > Simple export()/import() may not work. Image some flow rules are
> > > exclusive and can't be issued from both applications.
> > > > We need to stop old application. I am afraid this will introduce big time
> > > window which traffic stops.
> > >
> > > Yes, I think the sequence is
> > > rte_flow_rules_export() on app 1
> > > stop the app 1
> > > rte_flow_rules_import() of app 1 by app2.
> > >
> > I don't think export is the best solution, since maybe the second application doesn't want
> > all rules.
> > From my understanding the idea is to set priority between two process so when
> > one application closes the traffic is going to be received by the second application.
> > We have also the option that the second process will get duplicated traffic with the
> > First application.
> >
> > > > Application won't like this behavior.
> > > > With this callback, each PMD can specify each rule, queue it or use lower
> > > priority if exclusive. Or return error.
> > > >
> > > > > > This serial wants to propose a unified interface for upper layer
> > > application'
> > > > > easy use.
> > > > > > >
> > > > > > > That's just my view. I leave to ethdev maintainers for the rest of
> > > > > > > the review and decision on this series.
>
> That's a first step which allows to declare the migration intent.
> We should try to build on top of it and keep it as experimental
> as long as needed to achieve a good migration support.
>
> I am for going in this direction (accept the patch) for now.
> If we discover in the next months that there is a better direction,
> we can change.
Please have a driver support and test application to exercise this API
when merging this patch.
next prev parent reply other threads:[~2023-01-31 18:10 UTC|newest]
Thread overview: 54+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-12-01 8:20 [RFC 0/2] add API to set process to primary or secondary Rongwei Liu
2022-12-01 8:20 ` [RFC 1/2] ethdev: add group description Rongwei Liu
2022-12-01 8:20 ` [RFC 2/2] ethdev: add API to set process to primary or secondary Rongwei Liu
2022-12-01 15:10 ` Stephen Hemminger
2022-12-02 3:27 ` Rongwei Liu
2022-12-05 16:08 ` Stephen Hemminger
2022-12-06 3:47 ` Rongwei Liu
2022-12-06 5:54 ` Stephen Hemminger
2022-12-21 9:00 ` [RFC v3 0/2] add API to set process to active or standby Rongwei Liu
2022-12-21 9:00 ` [RFC v3 1/2] ethdev: add group description Rongwei Liu
2023-01-18 15:44 ` [PATCH v4 0/3] add API for live migration Rongwei Liu
2023-01-18 15:44 ` [PATCH v4 1/3] ethdev: add flow rule group description Rongwei Liu
2023-01-31 11:53 ` Ori Kam
2023-02-06 12:15 ` Rongwei Liu
2023-02-07 2:57 ` [PATCH v5] " Rongwei Liu
2023-02-08 20:28 ` Ferruh Yigit
2023-02-09 2:06 ` Rongwei Liu
2023-02-09 7:32 ` [PATCH v6] " Rongwei Liu
2023-02-09 8:01 ` Ori Kam
2023-02-09 11:26 ` Ferruh Yigit
2023-01-18 15:44 ` [PATCH v4 2/3] ethdev: add standby state for live migration Rongwei Liu
2023-01-31 13:50 ` Ori Kam
2023-01-31 18:14 ` Jerin Jacob
2023-01-31 22:55 ` Thomas Monjalon
2023-02-01 7:32 ` Andrew Rybchenko
2023-02-01 8:31 ` Thomas Monjalon
2023-02-01 8:40 ` Jerin Jacob
2023-02-01 8:46 ` Thomas Monjalon
2023-02-02 10:23 ` Rongwei Liu
2023-02-01 7:52 ` Andrew Rybchenko
2023-02-01 8:27 ` Thomas Monjalon
2023-02-01 8:40 ` Andrew Rybchenko
2023-01-18 15:44 ` [PATCH v4 3/3] ethdev: add standby flags " Rongwei Liu
2023-01-23 13:20 ` Jerin Jacob
2023-01-30 2:47 ` Rongwei Liu
2023-01-30 17:10 ` Jerin Jacob
2023-01-31 2:53 ` Rongwei Liu
2023-01-31 8:45 ` Jerin Jacob
2023-01-31 9:01 ` Rongwei Liu
2023-01-31 14:37 ` Jerin Jacob
2023-01-31 14:45 ` Ori Kam
2023-01-31 17:50 ` Thomas Monjalon
2023-01-31 18:10 ` Jerin Jacob [this message]
2022-12-21 9:00 ` [RFC v3 2/2] ethdev: add API to set process to active or standby Rongwei Liu
2022-12-21 9:12 ` Jerin Jacob
2022-12-21 9:32 ` Rongwei Liu
2022-12-21 10:59 ` Jerin Jacob
2022-12-21 12:05 ` Rongwei Liu
2022-12-21 12:44 ` Jerin Jacob
2022-12-21 12:50 ` Rongwei Liu
2022-12-21 13:12 ` Jerin Jacob
2022-12-21 14:33 ` Rongwei Liu
2022-12-26 16:44 ` Ori Kam
2023-01-15 22:46 ` Thomas Monjalon
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CALBAE1MoQx4LwNJc4qXxNepVcLqL308kkvc34+V2xVd5LA8vUg@mail.gmail.com \
--to=jerinjacobk@gmail.com \
--cc=andrew.rybchenko@oktetlabs.ru \
--cc=dev@dpdk.org \
--cc=ferruh.yigit@amd.com \
--cc=matan@nvidia.com \
--cc=orika@nvidia.com \
--cc=rasland@nvidia.com \
--cc=rongweil@nvidia.com \
--cc=stephen@networkplumber.org \
--cc=thomas@monjalon.net \
--cc=viacheslavo@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).