DPDK patches and discussions
 help / color / mirror / Atom feed
From: Ferruh Yigit <ferruh.yigit@intel.com>
To: Dmitry Kozlyuk <dkozlyuk@nvidia.com>,
	"dev@dpdk.org" <dev@dpdk.org>,
	Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>,
	Ori Kam <orika@nvidia.com>,
	 Raslan Darawsheh <rasland@nvidia.com>
Cc: NBU-Contact-Thomas Monjalon <thomas@monjalon.net>,
	Qi Zhang <qi.z.zhang@intel.com>,
	"jerinj@marvell.com" <jerinj@marvell.com>,
	"Maxime Coquelin" <maxime.coquelin@redhat.com>
Subject: Re: [dpdk-dev] [PATCH 2/5] ethdev: add capability to keep shared objects on restart
Date: Fri, 15 Oct 2021 12:46:34 +0100	[thread overview]
Message-ID: <0f16daab-6040-2973-03f3-8bc4e954c84e@intel.com> (raw)
In-Reply-To: <CH0PR12MB50915AC616ADAC8585592A6EB9B89@CH0PR12MB5091.namprd12.prod.outlook.com>

On 10/14/2021 10:45 PM, Dmitry Kozlyuk wrote:
>> -----Original Message-----
>> From: Ferruh Yigit <ferruh.yigit@intel.com>
>> Sent: 14 октября 2021 г. 16:47
>> To: Dmitry Kozlyuk <dkozlyuk@nvidia.com>; dev@dpdk.org; Andrew Rybchenko
>> <andrew.rybchenko@oktetlabs.ru>; Ori Kam <orika@nvidia.com>; Raslan
>> Darawsheh <rasland@nvidia.com>
>> Cc: NBU-Contact-Thomas Monjalon <thomas@monjalon.net>; Qi Zhang
>> <qi.z.zhang@intel.com>; jerinj@marvell.com; Maxime Coquelin
>> <maxime.coquelin@redhat.com>
>> Subject: Re: [PATCH 2/5] ethdev: add capability to keep shared objects on
>> restart
>>
>> External email: Use caution opening links or attachments
>>
>>
>> On 10/13/2021 9:32 AM, Dmitry Kozlyuk wrote:
>>> This thread continues discussions on previous versions to keep
>>> everything in the thread with final patches:
>>>
>>> [1]:
>>> http://inbox.dpdk.org/dev/d5673b58-5aa6-ca35-5b60-d938e56cfee1@oktetla
>>> bs.ru/
>>> [2]:
>>> http://inbox.dpdk.org/dev/DM8PR12MB5400997CCEC9169AC5AE0C89D6EA9@DM8PR
>>> 12MB5400.namprd12.prod.outlook.com/
>>>
>>> Please see below.
>>>
>>>> -----Original Message-----
>>>> From: Dmitry Kozlyuk <dkozlyuk@nvidia.com>
>>>> Sent: 5 октября 2021 г. 3:52
>>>> To: dev@dpdk.org
>>>> Cc: Dmitry Kozlyuk <dkozlyuk@nvidia.com>; Ori Kam <orika@nvidia.com>;
>>>> NBU- Contact-Thomas Monjalon <thomas@monjalon.net>; Ferruh Yigit
>>>> <ferruh.yigit@intel.com>; Andrew Rybchenko
>>>> <andrew.rybchenko@oktetlabs.ru>
>>>> Subject: [PATCH 2/5] ethdev: add capability to keep shared objects on
>>>> restart
>>>>
>>>> From: Dmitry Kozlyuk <dkozlyuk@nvidia.com>
>>>>
>>>> rte_flow_action_handle_create() did not mention what happens with an
>>>> indirect action when a device is stopped, possibly reconfigured, and
>>>> started again. It is natural for some indirect actions to be
>>>> persistent, like counters and meters; keeping others just saves
>>>> application time and complexity. However, not all PMDs can support it.
>>>> It is proposed to add a device capability to indicate if indirect
>>>> actions are kept across the above sequence or implicitly destroyed.
>>>>
>>>> In the future, indirect actions may not be the only type of objects
>>>> shared between flow rules. The capability bit intends to cover all
>>>> possible types of such objects, hence its name.
>>>>
>>>> It may happen that in the future a PMD acquires support for a type of
>>>> shared objects that it cannot keep across a restart. It is
>>>> undesirable to stop advertising the capability so that applications
>>>> that don't use objects of the problematic type can still take advantage
>> of it.
>>>> This is why PMDs are allowed to keep only a subset of shared objects
>>>> provided that the vendor mandatorily documents it.
>>>>
>>>> If the device is being reconfigured in a way that is incompatible
>>>> with an existing shared objects, PMD is required to report an error.
>>>> This is mandatory, because flow API does not supply users with
>>>> capabilities, so this is the only way for a user to learn that
>>>> configuration is invalid. For example, if queue count changes and RSS
>>>> indirect action specifies queues that are going away, the user must
>>>> update the action before removing the queues or remove the action and
>>>> all flow rules that were using it.
>>>>
>>>> Signed-off-by: Dmitry Kozlyuk <dkozlyuk@nvidia.com>
>>>> ---
>>>> [...]
>>>
>>> Current pain point is that capability bits may be insufficient and a
>>> programmatic way is desired to check which types of objects can be
>>> kept across restart, instead of documenting the limitations.
>>>
>>> I support one of previous Ori's suggestions and want to clarify it [1]:
>>>
>>> Ori: "Another way is to assume that if the action was created before
>> port start it will be kept after port stop."
>>> Andrew: "It does not sound like a solution. May be I simply don't know
>>> target usecase."
>>>
>>> What Ori suggests (offline discussion summary): Suppose an application
>> wants to check whether a shared object (indirect action) or a flow rule of
>> a particular kind. It calls rte_flow_action_handle_create() or
>> rte_flow_create() before rte_eth_dev_start(). If it succeeds, 1) it means
>> objects of this type can be kept across restart, 2) it's a normal object
>> created that will work after the port is started. This is logical, because
>> if the PMD can keep some kind of objects when the port is stopped, it is
>> likely to be able to create them when the port is not started. It is
>> subject to discussion if "object kind" means only "type" or "type +
>> transfer bit" combination; for mlx5 PMD it doesn't matter. One minor
>> drawback is that applications can only do the test when the port is
>> stopped, but it seems likely that the test really needs to be done at
>> startup anyway.
>>>
>>> If this is acceptable:
>>> 1. Capability bits are not needed anymore.
>>> 2. ethdev patches can be accepted in RC1, present behavior is undefined
>> anyway.
>>> 3. PMD patches will need update that can be done by RC2.
>>>
>>
>> Hi Dmitry,
>>
>> Are you planning to update drivers yourself on -rc2?
>> Or do you mean PMD maintainers should update themselves, if so do they
>> know about it?
>>
>> If the ethdev layer is updated in a way to impact the drivers, it should
>> be either:
>> - all drivers updated with a change
>> or
>> - give PMDs time to implement it on their own time, meanwhile they can
>> report their support status by a flag
>>
>> We had multiple sample of second case in the past but it is harder for
>> this case.
>>
>> For this case what about having three states:
>> - FLOW_RULE_KEEP
>> - FLOW_RULE_DESTROY
>> - FLOW_RULE_UNKNOWN
>>
>> And set 'FLOW_RULE_UNKNOWN' for all drivers, to simulate current status,
>> until driver is updated.
> 
> Hi Ferruh,
> 
> Indirect actions are only implemented by mlx5 PMD,
> the patches will be in RC2.
> If we don't use the flag as per the latest suggestion,
> nothing needs to be done for other PMDs.
> Flag can as well be kept with the following semantics:
> 0 => indirect actions are flushed on device stop
> 1 => at least some indirect actions are kept,
>       application should check types it's interested in
> 

My concerns is related to the 'flow rules', not indirect actions,
the patch mentions capability is for both of them.

> Introducing UNKNOWN state seems wrong to me.
> What should an application do when it is reported?
> Now there's just no way to learn how the PMD behaves,
> but if it provides a response, it can't be "I don't know what I do".
> 

I agree 'unknown' state is not ideal, but my intentions is prevent
drivers that not implemented this new feature report wrong capability.

Without capability, application already doesn't know how underlying
PMD behaves, so this is by default 'unknown' state.
I suggest keeping that state until driver explicitly updates its state
to the correct value.

But having below list is good, if you will update all drivers than
no need to have the 'unknown' state, but updating drivers may require
driver maintainers ack which can take some time.

Can you please clarify what is you plan according PMDs, will you update
them all, or will you only update mlx5 in -rc2?
And what is the exact plan for the -rc2 that you mention?

> Here's what I understood from the code, assuming there are no bugs
> Like allowing to stop the port and keep dangling flow handles:
> 
> bnxt        flush
> bonding     depends
> cnxk        can't figure out
> cxgbe       keep
> dpaa2       keep
> e1000       keep
> enic        flush
> failsafe    depends
> hinic       flush
> hns3        keep
> i40e        keep
> iavf        keep
> ice         keep
> igc         keep
> ipn3ke      keep
> ixgbe       keep
> mlx4        keep
> mlx5        flush
> mvpp2       keep
> octeontx2   can't figure out
> qede        keep
> sfc         flush
> softnic     flush
> tap         keep
> txgbe       keep
> 
> Currently one flag would be sufficient to describe PMD behavior:
> they either keep or flush the flow rules.
> If there are indeed no exceptions, which maintainers should confirm,
> I can add flag reporting myself.
> 


  parent reply	other threads:[~2021-10-15 11:46 UTC|newest]

Thread overview: 96+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-10-05  0:52 [dpdk-dev] [PATCH 0/5] Flow entites behavior on port restart dkozlyuk
2021-10-05  0:52 ` [dpdk-dev] [PATCH 1/5] ethdev: add capability to keep flow rules on restart dkozlyuk
2021-10-06  6:15   ` Ori Kam
2021-10-06  6:55     ` Somnath Kotur
2021-10-06 17:15   ` Ajit Khaparde
2021-10-05  0:52 ` [dpdk-dev] [PATCH 2/5] ethdev: add capability to keep shared objects " dkozlyuk
2021-10-06  6:16   ` Ori Kam
2021-10-13  8:32   ` Dmitry Kozlyuk
2021-10-14 13:46     ` Ferruh Yigit
2021-10-14 21:45       ` Dmitry Kozlyuk
2021-10-14 21:48         ` Dmitry Kozlyuk
2021-10-15 11:46         ` Ferruh Yigit [this message]
2021-10-15 12:35           ` Dmitry Kozlyuk
2021-10-15 16:26             ` Ferruh Yigit
2021-10-16 20:32               ` Dmitry Kozlyuk
2021-10-18  8:42                 ` Ferruh Yigit
2021-10-18 11:13                   ` Dmitry Kozlyuk
2021-10-18 11:59                     ` Ferruh Yigit
2021-10-14 14:14     ` Dmitry Kozlyuk
2021-10-15  8:26       ` Andrew Rybchenko
2021-10-15  9:04         ` Dmitry Kozlyuk
2021-10-15  9:36           ` Andrew Rybchenko
2021-10-05  0:52 ` [dpdk-dev] [PATCH 3/5] net/mlx5: discover max flow priority using DevX dkozlyuk
2021-10-05  0:52 ` [dpdk-dev] [PATCH 4/5] net/mlx5: create drop queue " dkozlyuk
2021-10-05  0:52 ` [dpdk-dev] [PATCH 5/5] net/mlx5: preserve indirect actions on restart dkozlyuk
2021-10-15 16:18 ` [dpdk-dev] [PATCH v2 0/5] Flow entites behavior on port restart Dmitry Kozlyuk
2021-10-15 16:18   ` [dpdk-dev] [PATCH v2 1/5] ethdev: add capability to keep flow rules on restart Dmitry Kozlyuk
2021-10-18  8:56     ` Andrew Rybchenko
2021-10-19 12:38       ` Dmitry Kozlyuk
2021-10-18 13:06     ` Zhang, Qi Z
2021-10-18 22:51       ` Dmitry Kozlyuk
2021-10-19  1:00         ` Zhang, Qi Z
2021-10-15 16:18   ` [dpdk-dev] [PATCH v2 2/5] ethdev: add capability to keep shared objects " Dmitry Kozlyuk
2021-10-17  8:10     ` Ori Kam
2021-10-17  9:14       ` Dmitry Kozlyuk
2021-10-17  9:45         ` Ori Kam
2021-10-15 16:18   ` [dpdk-dev] [PATCH v2 3/5] net/mlx5: discover max flow priority using DevX Dmitry Kozlyuk
2021-10-15 16:18   ` [dpdk-dev] [PATCH v2 4/5] net/mlx5: create drop queue " Dmitry Kozlyuk
2021-10-15 16:18   ` [dpdk-dev] [PATCH v2 5/5] net/mlx5: preserve indirect actions on restart Dmitry Kozlyuk
2021-10-19 12:37   ` [dpdk-dev] [PATCH v3 0/6] Flow entites behavior on port restart Dmitry Kozlyuk
2021-10-19 12:37     ` [dpdk-dev] [PATCH v3 1/6] ethdev: add capability to keep flow rules on restart Dmitry Kozlyuk
2021-10-19 15:22       ` Ori Kam
2021-10-19 16:38       ` Ferruh Yigit
2021-10-19 17:13         ` Dmitry Kozlyuk
2021-10-20 10:39       ` Andrew Rybchenko
2021-10-20 11:40         ` Dmitry Kozlyuk
2021-10-20 13:40           ` Ori Kam
2021-10-19 12:37     ` [dpdk-dev] [PATCH v3 2/6] ethdev: add capability to keep shared objects " Dmitry Kozlyuk
2021-10-19 15:22       ` Ori Kam
2021-10-19 12:37     ` [dpdk-dev] [PATCH v3 3/6] net: advertise no support for keeping flow rules Dmitry Kozlyuk
2021-10-20 10:08       ` Andrew Rybchenko
2021-10-20 22:20         ` Dmitry Kozlyuk
2021-10-19 12:37     ` [dpdk-dev] [PATCH v3 4/6] net/mlx5: discover max flow priority using DevX Dmitry Kozlyuk
2021-10-19 12:37     ` [dpdk-dev] [PATCH v3 5/6] net/mlx5: create drop queue " Dmitry Kozlyuk
2021-10-19 12:37     ` [dpdk-dev] [PATCH v3 6/6] net/mlx5: preserve indirect actions on restart Dmitry Kozlyuk
2021-10-20 10:12     ` [dpdk-dev] [PATCH v3 0/6] Flow entites behavior on port restart Andrew Rybchenko
2021-10-20 13:21       ` Dmitry Kozlyuk
2021-10-21  6:34     ` [dpdk-dev] [PATCH v4 " Dmitry Kozlyuk
2021-10-21  6:34       ` [dpdk-dev] [PATCH v4 1/6] ethdev: add capability to keep flow rules on restart Dmitry Kozlyuk
2021-10-21  7:36         ` Ori Kam
2021-10-28 18:33         ` Ajit Khaparde
2021-11-01 15:02         ` Andrew Rybchenko
2021-11-01 15:56           ` Dmitry Kozlyuk
2021-10-21  6:34       ` [dpdk-dev] [PATCH v4 2/6] ethdev: add capability to keep shared objects " Dmitry Kozlyuk
2021-10-21  7:37         ` Ori Kam
2021-10-21 18:28         ` Ajit Khaparde
2021-11-01 15:04         ` Andrew Rybchenko
2021-10-21  6:35       ` [dpdk-dev] [PATCH v4 3/6] net: advertise no support for keeping flow rules Dmitry Kozlyuk
2021-10-21 18:26         ` Ajit Khaparde
2021-10-22  1:38           ` Somnath Kotur
2021-10-27  7:11         ` Hyong Youb Kim (hyonkim)
2021-11-01 15:06         ` Andrew Rybchenko
2021-11-01 16:59           ` Ferruh Yigit
2021-10-21  6:35       ` [dpdk-dev] [PATCH v4 4/6] net/mlx5: discover max flow priority using DevX Dmitry Kozlyuk
2021-10-21  6:35       ` [dpdk-dev] [PATCH v4 5/6] net/mlx5: create drop queue " Dmitry Kozlyuk
2021-10-21  6:35       ` [dpdk-dev] [PATCH v4 6/6] net/mlx5: preserve indirect actions on restart Dmitry Kozlyuk
2021-10-26 11:46       ` [dpdk-dev] [PATCH v4 0/6] Flow entites behavior on port restart Ferruh Yigit
2021-11-01 13:43         ` Ferruh Yigit
2021-11-02 13:49       ` Ferruh Yigit
2021-11-02 13:54       ` [dpdk-dev] [PATCH v5 " Dmitry Kozlyuk
2021-11-02 13:54         ` [dpdk-dev] [PATCH v5 1/6] ethdev: add capability to keep flow rules on restart Dmitry Kozlyuk
2021-11-02 13:54         ` [dpdk-dev] [PATCH v5 2/6] ethdev: add capability to keep shared objects " Dmitry Kozlyuk
2021-11-02 13:54         ` [dpdk-dev] [PATCH v5 3/6] net: advertise no support for keeping flow rules Dmitry Kozlyuk
2021-11-02 13:54         ` [dpdk-dev] [PATCH v5 4/6] net/mlx5: discover max flow priority using DevX Dmitry Kozlyuk
2021-11-02 13:54         ` [dpdk-dev] [PATCH v5 5/6] net/mlx5: create drop queue " Dmitry Kozlyuk
2021-11-02 13:54         ` [dpdk-dev] [PATCH v5 6/6] net/mlx5: preserve indirect actions on restart Dmitry Kozlyuk
2021-11-02 14:23         ` [dpdk-dev] [PATCH v5 0/6] Flow entites behavior on port restart Ferruh Yigit
2021-11-02 17:02           ` Dmitry Kozlyuk
2021-11-02 17:01         ` [dpdk-dev] [PATCH v6 " Dmitry Kozlyuk
2021-11-02 17:01           ` [dpdk-dev] [PATCH v6 1/6] ethdev: add capability to keep flow rules on restart Dmitry Kozlyuk
2021-11-02 17:01           ` [dpdk-dev] [PATCH v6 2/6] ethdev: add capability to keep shared objects " Dmitry Kozlyuk
2021-11-02 17:01           ` [dpdk-dev] [PATCH v6 3/6] net: advertise no support for keeping flow rules Dmitry Kozlyuk
2021-11-02 17:01           ` [dpdk-dev] [PATCH v6 4/6] net/mlx5: discover max flow priority using DevX Dmitry Kozlyuk
2021-11-02 17:01           ` [dpdk-dev] [PATCH v6 5/6] net/mlx5: create drop queue " Dmitry Kozlyuk
2021-11-02 17:01           ` [dpdk-dev] [PATCH v6 6/6] net/mlx5: preserve indirect actions on restart Dmitry Kozlyuk
2021-11-02 18:02           ` [dpdk-dev] [PATCH v6 0/6] Flow entites behavior on port restart Ferruh Yigit

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=0f16daab-6040-2973-03f3-8bc4e954c84e@intel.com \
    --to=ferruh.yigit@intel.com \
    --cc=andrew.rybchenko@oktetlabs.ru \
    --cc=dev@dpdk.org \
    --cc=dkozlyuk@nvidia.com \
    --cc=jerinj@marvell.com \
    --cc=maxime.coquelin@redhat.com \
    --cc=orika@nvidia.com \
    --cc=qi.z.zhang@intel.com \
    --cc=rasland@nvidia.com \
    --cc=thomas@monjalon.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).