From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E439AA0C43; Mon, 18 Oct 2021 15:07:45 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 72FFF410E4; Mon, 18 Oct 2021 15:07:45 +0200 (CEST) Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by mails.dpdk.org (Postfix) with ESMTP id 7D72640DF5 for ; Mon, 18 Oct 2021 15:07:43 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10140"; a="228193364" X-IronPort-AV: E=Sophos;i="5.85,382,1624345200"; d="scan'208";a="228193364" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Oct 2021 06:06:03 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,382,1624345200"; d="scan'208";a="593805281" Received: from fmsmsx604.amr.corp.intel.com ([10.18.126.84]) by orsmga004.jf.intel.com with ESMTP; 18 Oct 2021 06:06:03 -0700 Received: from shsmsx603.ccr.corp.intel.com (10.109.6.143) by fmsmsx604.amr.corp.intel.com (10.18.126.84) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2242.12; Mon, 18 Oct 2021 06:06:02 -0700 Received: from shsmsx601.ccr.corp.intel.com (10.109.6.141) by SHSMSX603.ccr.corp.intel.com (10.109.6.143) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2242.12; Mon, 18 Oct 2021 21:06:01 +0800 Received: from shsmsx601.ccr.corp.intel.com ([10.109.6.141]) by SHSMSX601.ccr.corp.intel.com ([10.109.6.141]) with mapi id 15.01.2242.012; Mon, 18 Oct 2021 21:06:00 +0800 From: "Zhang, Qi Z" To: Dmitry Kozlyuk , "dev@dpdk.org" CC: Ori Kam , Thomas Monjalon , "Yigit, Ferruh" , Andrew Rybchenko Thread-Topic: [dpdk-dev] [PATCH v2 1/5] ethdev: add capability to keep flow rules on restart Thread-Index: AQHXweBjxFZmFHxTbU2xsDnRzDRAjqvYt1uw Date: Mon, 18 Oct 2021 13:06:00 +0000 Message-ID: References: <20211005005216.2427489-1-dkozlyuk@nvidia.com> <20211015161822.3099818-1-dkozlyuk@nvidia.com> <20211015161822.3099818-2-dkozlyuk@nvidia.com> In-Reply-To: <20211015161822.3099818-2-dkozlyuk@nvidia.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-reaction: no-action dlp-version: 11.6.200.16 dlp-product: dlpe-windows x-originating-ip: [10.239.127.36] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH v2 1/5] ethdev: add capability to keep flow rules on restart X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > -----Original Message----- > From: dev On Behalf Of Dmitry Kozlyuk > Sent: Saturday, October 16, 2021 12:18 AM > To: dev@dpdk.org > Cc: Ori Kam ; Thomas Monjalon > ; Yigit, Ferruh ; Andrew > Rybchenko > Subject: [dpdk-dev] [PATCH v2 1/5] ethdev: add capability to keep flow ru= les > on restart >=20 > Currently, it is not specified what happens to the flow rules when the de= vice is > stopped, possibly reconfigured, then started. > If flow rules were kept, it could be convenient for application developer= s, > because they wouldn't need to save and restore them. > However, due to the number of flows and possible creation rate it is > impractical to save all flow rules in DPDK layer. This means that flow ru= les > persistence really depends on whether PMD and HW can implement it > efficiently. It can also be limited by the rule item and action types, an= d its > attributes transfer bit, which together comprise the rule kind. >=20 > Add a device capability bit for PMDs that can keep at least some of the f= low > rules across restart. Without this capability behavior is still unspecifi= ed, which > is now explicitly stated. > Declare that the application can test for persitence of flow rules of a p= articular > kind by attempting to create a rule of that kind when the device is stopp= ed > and checking for the specific error. > This is logical because if the PMD can to create the flow rule when the d= evice > is not started and use it after the start happens, it is natural that it = can move > its internal flow rule object to the same state when the device is stoppe= d and > restore the state when the device is started. >=20 > If the device is being reconfigured in a way that is incompatible with ex= isting > flow rules, PMD is required to report an error. > This is mandatory, because flow API does not supply users with capabiliti= es, so > this is the only way for a user to learn that configuration is invalid. What if a PMD does not flush rules during start /stop cycle, but just want = to simply flush rules during dev_config?=20 Is it reasonable to take above as an typical implementation to avoid all th= e complexity for handling the conflicts? 1. queues are destroyed and re-created with a different number which may im= pact "to queue" action. 2. hash key may be overwritten which impact RSS result. 3. offload flags changes may impact data path selection which cause mark ac= tion does not work. .... > example, if queue count changes and the action of a flow rule specifies q= ueues > that are going away, the user must update or remove the flow rule before > removing the queues. >=20 > Signed-off-by: Dmitry Kozlyuk > --- > doc/guides/prog_guide/rte_flow.rst | 27 +++++++++++++++++++++++++++ > lib/ethdev/rte_ethdev.h | 7 +++++++ > lib/ethdev/rte_flow.h | 1 + > 3 files changed, 35 insertions(+) >=20 > diff --git a/doc/guides/prog_guide/rte_flow.rst > b/doc/guides/prog_guide/rte_flow.rst > index 2b42d5ec8c..b0ced4209b 100644 > --- a/doc/guides/prog_guide/rte_flow.rst > +++ b/doc/guides/prog_guide/rte_flow.rst > @@ -87,6 +87,33 @@ To avoid resource leaks on the PMD side, handles must > be explicitly destroyed by the application before releasing associated > resources such as queues and ports. >=20 > +By default it is unspecified if the flow rules persist after the device = stop. > +If ``RTE_ETH_DEV_CAPA_FLOW_RULE_KEEP`` is not advertised, then rules > +must be explicitly flushed before stopping the device if the > +application needs to ensure they are removed. > +If it is advertised, this means the PMD can keep at least some rules > +across the device stop and start with possible reconfiguration in betwee= n. > +However, it may be only supported for some kinds of rules. > +The kind is a combination of the following rule properties: > + > +- the sequence of item types; > +- the sequence of action types; > +- the value of the transfer attribute. > + > +To test if a particular kind of rules is kept, the application must try > +to create a valid rule of that kind when the device is stopped (after > +it has been configured or started previously). > +If it succeeds, all rules of the same kind are kept at the device stop. > +If it fails with an error of type ``RTE_FLOW_ERROR_TYPE_STATE``, rules > +of this kind are flushed when the device is stopped. > +Rules of a kept kind that are created when the device is stopped, > +including the rules created for the test, will be kept after the device = is started. > +Some configuration changes may be incompatible with existing rules. > +In this case ``rte_eth_dev_configure()``, > +``rte_eth_rx/tx_queue_setup()``, and/or ``rte_eth_dev_start()`` will > +fail with a log message from the PMD that should be similar to the one > +that would be emitted by ``rte_flow_create()`` if an attempt was made to > create the offending rule with the new configuration. > + > The following sections cover: >=20 > - **Attributes** (represented by ``struct rte_flow_attr``): properties o= f a diff > --git a/lib/ethdev/rte_ethdev.h b/lib/ethdev/rte_ethdev.h index > 6d80514ba7..a0b388bb25 100644 > --- a/lib/ethdev/rte_ethdev.h > +++ b/lib/ethdev/rte_ethdev.h > @@ -90,6 +90,11 @@ > * - flow director filtering mode (but not filtering rules) > * - NIC queue statistics mappings > * > + * The following configuration may be retained or not > + * depending on the device capabilities: > + * > + * - flow rules > + * > * Any other configuration will not be stored and will need to be re-ent= ered > * before a call to rte_eth_dev_start(). > * > @@ -1445,6 +1450,8 @@ struct rte_eth_conf { #define > RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP 0x00000001 > /** Device supports Tx queue setup after device started. */ #define > RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP 0x00000002 > +/** Device supports keeping flow rules across restart. */ #define > +RTE_ETH_DEV_CAPA_FLOW_RULE_KEEP 0x00000004 > /**@}*/ >=20 > /* > diff --git a/lib/ethdev/rte_flow.h b/lib/ethdev/rte_flow.h index > a89945061a..aa0182d021 100644 > --- a/lib/ethdev/rte_flow.h > +++ b/lib/ethdev/rte_flow.h > @@ -3344,6 +3344,7 @@ enum rte_flow_error_type { > RTE_FLOW_ERROR_TYPE_ACTION_NUM, /**< Number of actions. */ > RTE_FLOW_ERROR_TYPE_ACTION_CONF, /**< Action configuration. */ > RTE_FLOW_ERROR_TYPE_ACTION, /**< Specific action. */ > + RTE_FLOW_ERROR_TYPE_STATE, /**< Current device state. */ > }; >=20 > /** > -- > 2.25.1