* [dpdk-dev] [PATCH] add flow shared action API @ 2020-07-02 12:05 Andrey Vesnovaty 2020-07-03 15:02 ` Jerin Jacob ` (6 more replies) 0 siblings, 7 replies; 106+ messages in thread From: Andrey Vesnovaty @ 2020-07-02 12:05 UTC (permalink / raw) To: Thomas Monjalon, Ferruh Yigit, Andrew Rybchenko, Ori Kam Cc: dev, Andrey Vesnovaty From: Andrey Vesnovaty <andrey.vesnovaty@gmail.com> This commit introduces extension of DPDK flow action API enabling sharing of single rte_flow_action in multiple flows. The API intended for PMDs where multiple HW offloaded flows can reuse the same HW essence/object representing flow action and modification of such an essence/object effects all the rules using it. Motivation and example === Adding or removing one or more queues to RSS used by multiple flow rules imposes per rule toll for current DPDK flow API; the scenario requires for each flow sharing cloned RSS action: - call `rte_flow_destroy()` - call `rte_flow_create()` with modified RSS action API for sharing action and its in-place update benefits: - reduce the overhead of multiple RSS flow rules reconfiguration . - optimize resource utilization by sharing action across of of multiple flows Change description === Shared action === In order to represent flow action shared by multiple flows new action type RTE_FLOW_ACTION_TYPE_SHARED introduced (see `enum rte_flow_action_type`). Actually the introduced API decouples action from any specific flow and enables sharing of single action by its handle for multiple flows. Shared action create/use/destroy === Shared action may be reused by some or none flow rules at any given moment, IOW shared action reside outside of the context of any flow. Shared action represent HW resources/objects used for action offloading implementation. For allocation/release of all HW resources and all related initializations/cleanups in PMD space required for shared action implementation added new API rte_flow_shared_action_create()/rte_flow_shared_action_destroy(). In addition to the above all preparations needed to maintain shared access to the action resources, configuration and state should be done in rte_flow_shared_action_create(). In order to share some flow action reuse the handle of type `struct rte_flow_shared_action` returned by rte_flow_shared_action_create() as a `conf` field of `struct rte_flow_action` (see "example" section). If some shared action not used by any flow rule all resources allocated by the shared action can be released by rte_flow_shared_action_destroy() (see "example" section). The shared action handle passed as argument to destroy API should not be used i.e. result of the usage is undefined. Shared action re-configuration === Shared action behavior defined by its configuration & and can be updated via rte_flow_shared_action_update() (see "example" section). The shared action update operation modifies HW related resources/objects allocated by the action. The number of operations performed by the update operation should not be dependent on number of flows sharing the related action. On return of shared action updated API action behavior should be according to updated configuration for all flows sharing the action. Shared action query === Provide separate API to query shared action sate (see rte_flow_shared_action_update()). Taking a counter as an example: query returns value aggregating all counter increments across all flow rules sharing the counter. PMD support === The support of introduced API is pure PMD specific design and responsibility for each action type (see struct rte_flow_ops). testpmd === In order to utilize introduced API testpmd cli may implement following extension create/update/destroy/query shared action accordingly flow shared_action create {port_id} [index] {action} flow shared_action update {port_id} {index} {action} flow shared_action destroy {port_id} {index} flow shared_action query {port_id} {index} testpmd example === configure rss to queues 1 & 2 testpmd> flow shared_action create 0 100 rss 1 2 create flow rule utilizing shared action testpmd> flow create 0 ingress \ pattern eth dst is 0c:42:a1:15:fd:ac / ipv6 / tcp / end \ actions shared 100 end / end add 2 more queues testpmd> flow shared_action modify 0 100 rss 1 2 3 4 example === struct rte_flow_action actions[2]; struct rte_flow_action action; /* skipped: initialize action */ struct rte_flow_shared_action *handle = rte_flow_shared_action_create( port_id, &action, &error); actions[0].type = RTE_FLOW_ACTION_TYPE_SHARED; actions[0].conf = handle; actions[1].type = RTE_FLOW_ACTION_TYPE_END; /* skipped: init attr0 & pattern0 args */ struct rte_flow *flow0 = rte_flow_create(port_id, &attr0, pattern0, actions, error); /* create more rules reusing shared action */ struct rte_flow *flow1 = rte_flow_create(port_id, &attr1, pattern1, actions, error); /* skipped: for flows 2 till N */ struct rte_flow *flowN = rte_flow_create(port_id, &attrN, patternN, actions, error); /* update shared action */ struct rte_flow_action updated_action; /* * skipped: initialize updated_action according to desired action * configuration change */ rte_flow_shared_action_update(port_id, handle, updated_action.conf, error); /* * from now on all flows 1 till N will act according to configuration of * updated_action */ /* skipped: destroy all flows 1 till N */ rte_flow_shared_action_destroy(port_id, handle, error); Signed-off-by: Andrey Vesnovaty <andreyv@mellanox.com> Signed-off-by: Andrey Vesnovaty <andrey.vesnovaty@gmail.com> Signed-off-by: Andrey Vesnovaty <andreyv@mellanox.com> --- This patch based on RFC: https://patches.dpdk.org/patch/71820/ --- lib/librte_ethdev/rte_ethdev_version.map | 6 + lib/librte_ethdev/rte_flow.c | 81 +++++++++++++ lib/librte_ethdev/rte_flow.h | 148 ++++++++++++++++++++++- lib/librte_ethdev/rte_flow_driver.h | 22 ++++ 4 files changed, 256 insertions(+), 1 deletion(-) diff --git a/lib/librte_ethdev/rte_ethdev_version.map b/lib/librte_ethdev/rte_ethdev_version.map index 3f32fdecf..e291c2bd9 100644 --- a/lib/librte_ethdev/rte_ethdev_version.map +++ b/lib/librte_ethdev/rte_ethdev_version.map @@ -230,4 +230,10 @@ EXPERIMENTAL { # added in 20.02 rte_flow_dev_dump; + + # added in 20.08 + rte_flow_shared_action_create; + rte_flow_shared_action_destoy; + rte_flow_shared_action_update; + rte_flow_shared_action_query; }; diff --git a/lib/librte_ethdev/rte_flow.c b/lib/librte_ethdev/rte_flow.c index 885a7ff9a..7728057c3 100644 --- a/lib/librte_ethdev/rte_flow.c +++ b/lib/librte_ethdev/rte_flow.c @@ -1231,3 +1231,84 @@ rte_flow_dev_dump(uint16_t port_id, FILE *file, struct rte_flow_error *error) RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL, rte_strerror(ENOSYS)); } + +struct rte_flow_shared_action * +rte_flow_shared_action_create(uint16_t port_id, + const struct rte_flow_action *action, + struct rte_flow_error *error) +{ + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; + struct rte_flow_shared_action *shared_action; + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error); + + if (unlikely(!ops)) + return NULL; + if (likely(!!ops->shared_action_create)) { + shared_action = ops->shared_action_create(dev, action, error); + if (shared_action == NULL) + flow_err(port_id, -rte_errno, error); + return shared_action; + } + rte_flow_error_set(error, ENOSYS, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, rte_strerror(ENOSYS)); + return NULL; +} + +int +rte_flow_shared_action_destoy(uint16_t port_id, + struct rte_flow_shared_action *action, + struct rte_flow_error *error) +{ + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error); + + if (unlikely(!ops)) + return -rte_errno; + if (likely(!!ops->shared_action_destroy)) + return flow_err(port_id, + ops->shared_action_destroy(dev, action, error), + error); + return rte_flow_error_set(error, ENOSYS, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, rte_strerror(ENOSYS)); +} + +int +rte_flow_shared_action_update(uint16_t port_id, + struct rte_flow_shared_action *action, + const void *action_conf, + struct rte_flow_error *error) +{ + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error); + + if (unlikely(!ops)) + return -rte_errno; + if (likely(!!ops->shared_action_update)) + return flow_err(port_id, ops->shared_action_update(dev, action, + action_conf, error), + error); + return rte_flow_error_set(error, ENOSYS, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, rte_strerror(ENOSYS)); +} + +int +rte_flow_shared_action_query(uint16_t port_id, + const struct rte_flow_shared_action *action, + void *data, + struct rte_flow_error *error) +{ + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error); + + if (unlikely(!ops)) + return -rte_errno; + if (likely(!!ops->shared_action_query)) + return flow_err(port_id, ops->shared_action_query(dev, action, + data, error), + error); + return rte_flow_error_set(error, ENOSYS, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, rte_strerror(ENOSYS)); +} diff --git a/lib/librte_ethdev/rte_flow.h b/lib/librte_ethdev/rte_flow.h index 5625dc491..98140ebb1 100644 --- a/lib/librte_ethdev/rte_flow.h +++ b/lib/librte_ethdev/rte_flow.h @@ -1643,7 +1643,8 @@ enum rte_flow_action_type { /** * Enables counters for this flow rule. * - * These counters can be retrieved and reset through rte_flow_query(), + * These counters can be retrieved and reset through rte_flow_query() or + * rte_flow_shared_action_query() if the action provided via handle, * see struct rte_flow_query_count. * * See struct rte_flow_action_count. @@ -2051,6 +2052,14 @@ enum rte_flow_action_type { * See struct rte_flow_action_set_dscp. */ RTE_FLOW_ACTION_TYPE_SET_IPV6_DSCP, + + /** + * Describes action shared a cross multiple flow rules. + * + * Enables multiple rules reference the same action by handle (see + * struct rte_flow_shared_action). + */ + RTE_FLOW_ACTION_TYPE_SHARED, }; /** @@ -2593,6 +2602,20 @@ struct rte_flow_action_set_dscp { uint8_t dscp; }; + +/** + * RTE_FLOW_ACTION_TYPE_SHARED + * + * Opaque type returned after successfully creating a shared action. + * + * This handle can be used to manage and query the related action: + * - share it a cross multiple flow rules + * - update action configuration + * - query action data + * - destroy action + */ +struct rte_flow_shared_action; + /* Mbuf dynamic field offset for metadata. */ extern int rte_flow_dynf_metadata_offs; @@ -3224,6 +3247,129 @@ rte_flow_conv(enum rte_flow_conv_op op, const void *src, struct rte_flow_error *error); +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Create shared action for reuse in multiple flow rules. + * + * @param[in] port_id + * The port identifier of the Ethernet device. + * @param[in] action + * Action configuration for shared action creation. + * @param[out] error + * Perform verbose error reporting if not NULL. PMDs initialize this + * structure in case of error only. + * @return + * A valid handle in case of success, NULL otherwise and rte_errno is set + * to one of the error codes defined: + * - (ENOSYS) if underlying device does not support this functionality. + * - (EIO) if underlying device is removed. + * - (EINVAL) if *action* invalid. + * - (ENOTSUP) if *action* valid but unsupported. + */ +__rte_experimental +struct rte_flow_shared_action * +rte_flow_shared_action_create(uint16_t port_id, + const struct rte_flow_action *action, + struct rte_flow_error *error); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Destroys the shared action by handle. + * + * @param[in] port_id + * The port identifier of the Ethernet device. + * @param[in] action + * Handle for the shared action to be destroyed. + * @param[out] error + * Perform verbose error reporting if not NULL. PMDs initialize this + * structure in case of error only. + * @return + * - (0) if success. + * - (-ENOSYS) if underlying device does not support this functionality. + * - (-EIO) if underlying device is removed. + * - (-ENOENT) if action pointed by *action* handle was not found. + * - (-ETOOMANYREFS) if action pointed by *action* handle still used by one or + * more rules + * rte_errno is also set. + */ +__rte_experimental +int +rte_flow_shared_action_destoy(uint16_t port_id, + struct rte_flow_shared_action *action, + struct rte_flow_error *error); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Updates inplace the shared action configuration pointed by *action* handle + * with the configuration provided as *action_conf* argument. + * The update of the shared action configuration effects all flow rules reusing + * the action via handle. + * + * @param[in] port_id + * The port identifier of the Ethernet device. + * @param[in] action + * Handle for the shared action to be updated. + * @param[in] action_conf + * Action specification used to modify the action pointed by handle. + * action_conf should be of same type with the action pointed by the *action* + * handle argument, otherwise function behavior undefined. + * @param[out] error + * Perform verbose error reporting if not NULL. PMDs initialize this + * structure in case of error only. + * @return + * - (0) if success. + * - (-ENOSYS) if underlying device does not support this functionality. + * - (-EIO) if underlying device is removed. + * - (-EINVAL) if *action_conf* invalid. + * - (-ENOTSUP) if *action_conf* valid but unsupported. + * - (-ENOENT) if action pointed by *ctx* was not found. + * rte_errno is also set. + */ +__rte_experimental +int +rte_flow_shared_action_update(uint16_t port_id, + struct rte_flow_shared_action *action, + const void *action_conf, + struct rte_flow_error *error); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Query the shared action by handle. + * + * This function allows retrieving action-specific data such as counters. + * Data is gathered by special action which may be present/referenced in + * more than one flow rule definition. + * + * \see RTE_FLOW_ACTION_TYPE_COUNT + * + * @param port_id + * Port identifier of Ethernet device. + * @param[in] action + * Handle for the shared action to query. + * @param[in, out] data + * Pointer to storage for the associated query data type. + * @param[out] error + * Perform verbose error reporting if not NULL. PMDs initialize this + * structure in case of error only. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +__rte_experimental +int +rte_flow_shared_action_query(uint16_t port_id, + const struct rte_flow_shared_action *action, + void *data, + struct rte_flow_error *error); + #ifdef __cplusplus } #endif diff --git a/lib/librte_ethdev/rte_flow_driver.h b/lib/librte_ethdev/rte_flow_driver.h index 51a9a57a0..c103d159e 100644 --- a/lib/librte_ethdev/rte_flow_driver.h +++ b/lib/librte_ethdev/rte_flow_driver.h @@ -101,6 +101,28 @@ struct rte_flow_ops { (struct rte_eth_dev *dev, FILE *file, struct rte_flow_error *error); + /** See rte_flow_shared_action_create() */ + struct rte_flow_shared_action *(*shared_action_create) + (struct rte_eth_dev *dev, + const struct rte_flow_action *action, + struct rte_flow_error *error); + /** See rte_flow_shared_action_destroy() */ + int (*shared_action_destroy) + (struct rte_eth_dev *dev, + struct rte_flow_shared_action *shared_action, + struct rte_flow_error *error); + /** See rte_flow_shared_action_update() */ + int (*shared_action_update) + (struct rte_eth_dev *dev, + struct rte_flow_shared_action *shared_action, + const void *action_conf, + struct rte_flow_error *error); + /** See rte_flow_shared_action_query() */ + int (*shared_action_query) + (struct rte_eth_dev *dev, + const struct rte_flow_shared_action *shared_action, + void *data, + struct rte_flow_error *error); }; /** -- 2.26.2 ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [dpdk-dev] [PATCH] add flow shared action API 2020-07-02 12:05 [dpdk-dev] [PATCH] add flow shared action API Andrey Vesnovaty @ 2020-07-03 15:02 ` Jerin Jacob 2020-07-03 15:21 ` Thomas Monjalon 2020-07-04 10:10 ` Andrey Vesnovaty 2020-07-08 21:39 ` [dpdk-dev] [PATCH v2 0/6] add flow shared action API + PMD Andrey Vesnovaty ` (5 subsequent siblings) 6 siblings, 2 replies; 106+ messages in thread From: Jerin Jacob @ 2020-07-03 15:02 UTC (permalink / raw) To: Andrey Vesnovaty Cc: Thomas Monjalon, Ferruh Yigit, Andrew Rybchenko, Ori Kam, dpdk-dev, Andrey Vesnovaty On Fri, Jul 3, 2020 at 8:07 PM Andrey Vesnovaty <andreyv@mellanox.com> wrote: > > From: Andrey Vesnovaty <andrey.vesnovaty@gmail.com> > > This commit introduces extension of DPDK flow action API enabling > sharing of single rte_flow_action in multiple flows. The API intended for > PMDs where multiple HW offloaded flows can reuse the same HW > essence/object representing flow action and modification of such an > essence/object effects all the rules using it. > > Motivation and example > === > Adding or removing one or more queues to RSS used by multiple flow rules > imposes per rule toll for current DPDK flow API; the scenario requires > for each flow sharing cloned RSS action: > - call `rte_flow_destroy()` > - call `rte_flow_create()` with modified RSS action > > API for sharing action and its in-place update benefits: > - reduce the overhead of multiple RSS flow rules reconfiguration . > - optimize resource utilization by sharing action across of of multiple > flows > > Change description > === > > Shared action > === > In order to represent flow action shared by multiple flows new action > type RTE_FLOW_ACTION_TYPE_SHARED introduced (see `enum > rte_flow_action_type`). > Actually the introduced API decouples action from any specific flow and > enables sharing of single action by its handle for multiple flows. > > Shared action create/use/destroy > === > Shared action may be reused by some or none flow rules at any given > moment, IOW shared action reside outside of the context of any flow. > Shared action represent HW resources/objects used for action offloading > implementation. For allocation/release of all HW resources and all > related initializations/cleanups in PMD space required for shared action > implementation added new API > rte_flow_shared_action_create()/rte_flow_shared_action_destroy(). > In addition to the above all preparations needed to maintain shared > access to the action resources, configuration and state should be done in > rte_flow_shared_action_create(). > > In order to share some flow action reuse the handle of type > `struct rte_flow_shared_action` returned by > rte_flow_shared_action_create() as a `conf` field of > `struct rte_flow_action` (see "example" section). > > If some shared action not used by any flow rule all resources allocated > by the shared action can be released by rte_flow_shared_action_destroy() > (see "example" section). The shared action handle passed as argument to > destroy API should not be used i.e. result of the usage is undefined. > > Shared action re-configuration > === > Shared action behavior defined by its configuration & and can be updated > via rte_flow_shared_action_update() (see "example" section). The shared > action update operation modifies HW related resources/objects allocated > by the action. The number of operations performed by the update operation > should not be dependent on number of flows sharing the related action. > On return of shared action updated API action behavior should be > according to updated configuration for all flows sharing the action. > > Shared action query > === > Provide separate API to query shared action sate (see > rte_flow_shared_action_update()). Taking a counter as an example: query > returns value aggregating all counter increments across all flow rules > sharing the counter. > > PMD support > === > The support of introduced API is pure PMD specific design and > responsibility for each action type (see struct rte_flow_ops). > > testpmd > === > In order to utilize introduced API testpmd cli may implement following > extension > create/update/destroy/query shared action accordingly > > flow shared_action create {port_id} [index] {action} > flow shared_action update {port_id} {index} {action} > flow shared_action destroy {port_id} {index} > flow shared_action query {port_id} {index} > > testpmd example > === > > configure rss to queues 1 & 2 > > testpmd> flow shared_action create 0 100 rss 1 2 > > create flow rule utilizing shared action > > testpmd> flow create 0 ingress \ > pattern eth dst is 0c:42:a1:15:fd:ac / ipv6 / tcp / end \ > actions shared 100 end / end > > add 2 more queues > > testpmd> flow shared_action modify 0 100 rss 1 2 3 4 > > example > === > > struct rte_flow_action actions[2]; > struct rte_flow_action action; > /* skipped: initialize action */ > struct rte_flow_shared_action *handle = rte_flow_shared_action_create( > port_id, &action, &error); > actions[0].type = RTE_FLOW_ACTION_TYPE_SHARED; > actions[0].conf = handle; > actions[1].type = RTE_FLOW_ACTION_TYPE_END; > /* skipped: init attr0 & pattern0 args */ > struct rte_flow *flow0 = rte_flow_create(port_id, &attr0, pattern0, > actions, error); > /* create more rules reusing shared action */ > struct rte_flow *flow1 = rte_flow_create(port_id, &attr1, pattern1, > actions, error); > /* skipped: for flows 2 till N */ > struct rte_flow *flowN = rte_flow_create(port_id, &attrN, patternN, > actions, error); > /* update shared action */ > struct rte_flow_action updated_action; > /* > * skipped: initialize updated_action according to desired action > * configuration change > */ > rte_flow_shared_action_update(port_id, handle, updated_action.conf, > error); > /* > * from now on all flows 1 till N will act according to configuration of > * updated_action > */ > /* skipped: destroy all flows 1 till N */ > rte_flow_shared_action_destroy(port_id, handle, error); > > Signed-off-by: Andrey Vesnovaty <andreyv@mellanox.com> > Signed-off-by: Andrey Vesnovaty <andrey.vesnovaty@gmail.com> > Signed-off-by: Andrey Vesnovaty <andreyv@mellanox.com> Duplicate Signoffs. # Request to CC all the people who are already giving comments to this patch. # The Following comment is not addressed in this patch. http://mails.dpdk.org/archives/dev/2020-July/172408.html > --- > This patch based on RFC: https://patches.dpdk.org/patch/71820/ > > --- > lib/librte_ethdev/rte_ethdev_version.map | 6 + > lib/librte_ethdev/rte_flow.c | 81 +++++++++++++ > lib/librte_ethdev/rte_flow.h | 148 ++++++++++++++++++++++- > lib/librte_ethdev/rte_flow_driver.h | 22 ++++ > 4 files changed, 256 insertions(+), 1 deletion(-) > > diff --git a/lib/librte_ethdev/rte_ethdev_version.map b/lib/librte_ethdev/rte_ethdev_version.map > index 3f32fdecf..e291c2bd9 100644 > --- a/lib/librte_ethdev/rte_ethdev_version.map > +++ b/lib/librte_ethdev/rte_ethdev_version.map > @@ -230,4 +230,10 @@ EXPERIMENTAL { > > # added in 20.02 > rte_flow_dev_dump; > + > + # added in 20.08 > + rte_flow_shared_action_create; > + rte_flow_shared_action_destoy; > + rte_flow_shared_action_update; > + rte_flow_shared_action_query; > }; > diff --git a/lib/librte_ethdev/rte_flow.c b/lib/librte_ethdev/rte_flow.c > index 885a7ff9a..7728057c3 100644 > --- a/lib/librte_ethdev/rte_flow.c > +++ b/lib/librte_ethdev/rte_flow.c > @@ -1231,3 +1231,84 @@ rte_flow_dev_dump(uint16_t port_id, FILE *file, struct rte_flow_error *error) > RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > NULL, rte_strerror(ENOSYS)); > } > + > +struct rte_flow_shared_action * > +rte_flow_shared_action_create(uint16_t port_id, > + const struct rte_flow_action *action, > + struct rte_flow_error *error) > +{ > + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; > + struct rte_flow_shared_action *shared_action; > + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error); > + > + if (unlikely(!ops)) > + return NULL; > + if (likely(!!ops->shared_action_create)) { > + shared_action = ops->shared_action_create(dev, action, error); > + if (shared_action == NULL) > + flow_err(port_id, -rte_errno, error); > + return shared_action; > + } > + rte_flow_error_set(error, ENOSYS, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > + NULL, rte_strerror(ENOSYS)); > + return NULL; > +} > + > +int > +rte_flow_shared_action_destoy(uint16_t port_id, > + struct rte_flow_shared_action *action, > + struct rte_flow_error *error) > +{ > + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; > + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error); > + > + if (unlikely(!ops)) > + return -rte_errno; > + if (likely(!!ops->shared_action_destroy)) > + return flow_err(port_id, > + ops->shared_action_destroy(dev, action, error), > + error); > + return rte_flow_error_set(error, ENOSYS, > + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > + NULL, rte_strerror(ENOSYS)); > +} > + > +int > +rte_flow_shared_action_update(uint16_t port_id, > + struct rte_flow_shared_action *action, > + const void *action_conf, > + struct rte_flow_error *error) > +{ > + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; > + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error); > + > + if (unlikely(!ops)) > + return -rte_errno; > + if (likely(!!ops->shared_action_update)) > + return flow_err(port_id, ops->shared_action_update(dev, action, > + action_conf, error), > + error); > + return rte_flow_error_set(error, ENOSYS, > + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > + NULL, rte_strerror(ENOSYS)); > +} > + > +int > +rte_flow_shared_action_query(uint16_t port_id, > + const struct rte_flow_shared_action *action, > + void *data, > + struct rte_flow_error *error) > +{ > + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; > + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error); > + > + if (unlikely(!ops)) > + return -rte_errno; > + if (likely(!!ops->shared_action_query)) > + return flow_err(port_id, ops->shared_action_query(dev, action, > + data, error), > + error); > + return rte_flow_error_set(error, ENOSYS, > + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > + NULL, rte_strerror(ENOSYS)); > +} > diff --git a/lib/librte_ethdev/rte_flow.h b/lib/librte_ethdev/rte_flow.h > index 5625dc491..98140ebb1 100644 > --- a/lib/librte_ethdev/rte_flow.h > +++ b/lib/librte_ethdev/rte_flow.h > @@ -1643,7 +1643,8 @@ enum rte_flow_action_type { > /** > * Enables counters for this flow rule. > * > - * These counters can be retrieved and reset through rte_flow_query(), > + * These counters can be retrieved and reset through rte_flow_query() or > + * rte_flow_shared_action_query() if the action provided via handle, > * see struct rte_flow_query_count. > * > * See struct rte_flow_action_count. > @@ -2051,6 +2052,14 @@ enum rte_flow_action_type { > * See struct rte_flow_action_set_dscp. > */ > RTE_FLOW_ACTION_TYPE_SET_IPV6_DSCP, > + > + /** > + * Describes action shared a cross multiple flow rules. > + * > + * Enables multiple rules reference the same action by handle (see > + * struct rte_flow_shared_action). > + */ > + RTE_FLOW_ACTION_TYPE_SHARED, > }; > > /** > @@ -2593,6 +2602,20 @@ struct rte_flow_action_set_dscp { > uint8_t dscp; > }; > > + > +/** > + * RTE_FLOW_ACTION_TYPE_SHARED > + * > + * Opaque type returned after successfully creating a shared action. > + * > + * This handle can be used to manage and query the related action: > + * - share it a cross multiple flow rules > + * - update action configuration > + * - query action data > + * - destroy action > + */ > +struct rte_flow_shared_action; > + > /* Mbuf dynamic field offset for metadata. */ > extern int rte_flow_dynf_metadata_offs; > > @@ -3224,6 +3247,129 @@ rte_flow_conv(enum rte_flow_conv_op op, > const void *src, > struct rte_flow_error *error); > > +/** > + * @warning > + * @b EXPERIMENTAL: this API may change without prior notice. > + * > + * Create shared action for reuse in multiple flow rules. > + * > + * @param[in] port_id > + * The port identifier of the Ethernet device. > + * @param[in] action > + * Action configuration for shared action creation. > + * @param[out] error > + * Perform verbose error reporting if not NULL. PMDs initialize this > + * structure in case of error only. > + * @return > + * A valid handle in case of success, NULL otherwise and rte_errno is set > + * to one of the error codes defined: > + * - (ENOSYS) if underlying device does not support this functionality. > + * - (EIO) if underlying device is removed. > + * - (EINVAL) if *action* invalid. > + * - (ENOTSUP) if *action* valid but unsupported. > + */ > +__rte_experimental > +struct rte_flow_shared_action * > +rte_flow_shared_action_create(uint16_t port_id, > + const struct rte_flow_action *action, > + struct rte_flow_error *error); > + > +/** > + * @warning > + * @b EXPERIMENTAL: this API may change without prior notice. > + * > + * Destroys the shared action by handle. > + * > + * @param[in] port_id > + * The port identifier of the Ethernet device. > + * @param[in] action > + * Handle for the shared action to be destroyed. > + * @param[out] error > + * Perform verbose error reporting if not NULL. PMDs initialize this > + * structure in case of error only. > + * @return > + * - (0) if success. > + * - (-ENOSYS) if underlying device does not support this functionality. > + * - (-EIO) if underlying device is removed. > + * - (-ENOENT) if action pointed by *action* handle was not found. > + * - (-ETOOMANYREFS) if action pointed by *action* handle still used by one or > + * more rules > + * rte_errno is also set. > + */ > +__rte_experimental > +int > +rte_flow_shared_action_destoy(uint16_t port_id, > + struct rte_flow_shared_action *action, > + struct rte_flow_error *error); > + > +/** > + * @warning > + * @b EXPERIMENTAL: this API may change without prior notice. > + * > + * Updates inplace the shared action configuration pointed by *action* handle > + * with the configuration provided as *action_conf* argument. > + * The update of the shared action configuration effects all flow rules reusing > + * the action via handle. > + * > + * @param[in] port_id > + * The port identifier of the Ethernet device. > + * @param[in] action > + * Handle for the shared action to be updated. > + * @param[in] action_conf > + * Action specification used to modify the action pointed by handle. > + * action_conf should be of same type with the action pointed by the *action* > + * handle argument, otherwise function behavior undefined. > + * @param[out] error > + * Perform verbose error reporting if not NULL. PMDs initialize this > + * structure in case of error only. > + * @return > + * - (0) if success. > + * - (-ENOSYS) if underlying device does not support this functionality. > + * - (-EIO) if underlying device is removed. > + * - (-EINVAL) if *action_conf* invalid. > + * - (-ENOTSUP) if *action_conf* valid but unsupported. > + * - (-ENOENT) if action pointed by *ctx* was not found. > + * rte_errno is also set. > + */ > +__rte_experimental > +int > +rte_flow_shared_action_update(uint16_t port_id, > + struct rte_flow_shared_action *action, > + const void *action_conf, > + struct rte_flow_error *error); > + > +/** > + * @warning > + * @b EXPERIMENTAL: this API may change without prior notice. > + * > + * Query the shared action by handle. > + * > + * This function allows retrieving action-specific data such as counters. > + * Data is gathered by special action which may be present/referenced in > + * more than one flow rule definition. > + * > + * \see RTE_FLOW_ACTION_TYPE_COUNT > + * > + * @param port_id > + * Port identifier of Ethernet device. > + * @param[in] action > + * Handle for the shared action to query. > + * @param[in, out] data > + * Pointer to storage for the associated query data type. > + * @param[out] error > + * Perform verbose error reporting if not NULL. PMDs initialize this > + * structure in case of error only. > + * > + * @return > + * 0 on success, a negative errno value otherwise and rte_errno is set. > + */ > +__rte_experimental > +int > +rte_flow_shared_action_query(uint16_t port_id, > + const struct rte_flow_shared_action *action, > + void *data, > + struct rte_flow_error *error); > + > #ifdef __cplusplus > } > #endif > diff --git a/lib/librte_ethdev/rte_flow_driver.h b/lib/librte_ethdev/rte_flow_driver.h > index 51a9a57a0..c103d159e 100644 > --- a/lib/librte_ethdev/rte_flow_driver.h > +++ b/lib/librte_ethdev/rte_flow_driver.h > @@ -101,6 +101,28 @@ struct rte_flow_ops { > (struct rte_eth_dev *dev, > FILE *file, > struct rte_flow_error *error); > + /** See rte_flow_shared_action_create() */ > + struct rte_flow_shared_action *(*shared_action_create) > + (struct rte_eth_dev *dev, > + const struct rte_flow_action *action, > + struct rte_flow_error *error); > + /** See rte_flow_shared_action_destroy() */ > + int (*shared_action_destroy) > + (struct rte_eth_dev *dev, > + struct rte_flow_shared_action *shared_action, > + struct rte_flow_error *error); > + /** See rte_flow_shared_action_update() */ > + int (*shared_action_update) > + (struct rte_eth_dev *dev, > + struct rte_flow_shared_action *shared_action, > + const void *action_conf, > + struct rte_flow_error *error); > + /** See rte_flow_shared_action_query() */ > + int (*shared_action_query) > + (struct rte_eth_dev *dev, > + const struct rte_flow_shared_action *shared_action, > + void *data, > + struct rte_flow_error *error); > }; > > /** > -- > 2.26.2 > ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [dpdk-dev] [PATCH] add flow shared action API 2020-07-03 15:02 ` Jerin Jacob @ 2020-07-03 15:21 ` Thomas Monjalon 2020-07-04 9:54 ` Andrey Vesnovaty 2020-07-04 10:10 ` Andrey Vesnovaty 1 sibling, 1 reply; 106+ messages in thread From: Thomas Monjalon @ 2020-07-03 15:21 UTC (permalink / raw) To: Andrey Vesnovaty, Jerin Jacob Cc: Ferruh Yigit, Andrew Rybchenko, Ori Kam, dpdk-dev, Andrey Vesnovaty, asafp, shys 03/07/2020 17:02, Jerin Jacob: > On Fri, Jul 3, 2020 at 8:07 PM Andrey Vesnovaty <andreyv@mellanox.com> wrote: > > Signed-off-by: Andrey Vesnovaty <andreyv@mellanox.com> > > Signed-off-by: Andrey Vesnovaty <andrey.vesnovaty@gmail.com> > > Signed-off-by: Andrey Vesnovaty <andreyv@mellanox.com> > > Duplicate Signoffs. > > # Request to CC all the people who are already giving comments to this patch. Yes you are absolutely right Jerin. It is said in the contribution docs, but we must repeat it again and again, especially for newcomers like here. The CC list is very important! Please pay attention to Cc the relevant maintainers AND the persons who already showed some interest. It may be a blocker for merge, thanks for sharing with co-workers. ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [dpdk-dev] [PATCH] add flow shared action API 2020-07-03 15:21 ` Thomas Monjalon @ 2020-07-04 9:54 ` Andrey Vesnovaty 0 siblings, 0 replies; 106+ messages in thread From: Andrey Vesnovaty @ 2020-07-04 9:54 UTC (permalink / raw) To: Thomas Monjalon Cc: Andrey Vesnovaty, Jerin Jacob, Ferruh Yigit, Andrew Rybchenko, Ori Kam, dpdk-dev, asafp, shys Hi, Jerin & Thomas. On Fri, Jul 3, 2020 at 6:21 PM Thomas Monjalon <thomas@monjalon.net> wrote: > 03/07/2020 17:02, Jerin Jacob: > > On Fri, Jul 3, 2020 at 8:07 PM Andrey Vesnovaty <andreyv@mellanox.com> > wrote: > > > Signed-off-by: Andrey Vesnovaty <andreyv@mellanox.com> > > > Signed-off-by: Andrey Vesnovaty <andrey.vesnovaty@gmail.com> > > > Signed-off-by: Andrey Vesnovaty <andreyv@mellanox.com> > > > > Duplicate Signoffs. > > > > # Request to CC all the people who are already giving comments to this > patch. > > Yes you are absolutely right Jerin. > It is said in the contribution docs, but we must repeat it again > and again, especially for newcomers like here. > > The CC list is very important! > Please pay attention to Cc the relevant maintainers > AND the persons who already showed some interest. > Just to clarify my next steps: - remove irrelevant signoffs. - all those who already showed interest should be added to CC field - send PATCH v2 in reply to this email. Thanks & sorry for the mess. > > It may be a blocker for merge, thanks for sharing with co-workers. > > > ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [dpdk-dev] [PATCH] add flow shared action API 2020-07-03 15:02 ` Jerin Jacob 2020-07-03 15:21 ` Thomas Monjalon @ 2020-07-04 10:10 ` Andrey Vesnovaty 2020-07-04 12:33 ` Jerin Jacob 1 sibling, 1 reply; 106+ messages in thread From: Andrey Vesnovaty @ 2020-07-04 10:10 UTC (permalink / raw) To: Jerin Jacob Cc: Andrey Vesnovaty, Thomas Monjalon, Ferruh Yigit, Andrew Rybchenko, Ori Kam, dpdk-dev Thanks, Andrey Vesnovaty (+972)*526775512* | *Skype:* andrey775512 On Fri, Jul 3, 2020 at 6:02 PM Jerin Jacob <jerinjacobk@gmail.com> wrote: > On Fri, Jul 3, 2020 at 8:07 PM Andrey Vesnovaty <andreyv@mellanox.com> > wrote: > > > > From: Andrey Vesnovaty <andrey.vesnovaty@gmail.com> > > > > This commit introduces extension of DPDK flow action API enabling > > sharing of single rte_flow_action in multiple flows. The API intended for > > PMDs where multiple HW offloaded flows can reuse the same HW > > essence/object representing flow action and modification of such an > > essence/object effects all the rules using it. > > > > Motivation and example > > === > > Adding or removing one or more queues to RSS used by multiple flow rules > > imposes per rule toll for current DPDK flow API; the scenario requires > > for each flow sharing cloned RSS action: > > - call `rte_flow_destroy()` > > - call `rte_flow_create()` with modified RSS action > > > > API for sharing action and its in-place update benefits: > > - reduce the overhead of multiple RSS flow rules reconfiguration . > > - optimize resource utilization by sharing action across of of multiple > > flows > > > > Change description > > === > > > > Shared action > > === > > In order to represent flow action shared by multiple flows new action > > type RTE_FLOW_ACTION_TYPE_SHARED introduced (see `enum > > rte_flow_action_type`). > > Actually the introduced API decouples action from any specific flow and > > enables sharing of single action by its handle for multiple flows. > > > > Shared action create/use/destroy > > === > > Shared action may be reused by some or none flow rules at any given > > moment, IOW shared action reside outside of the context of any flow. > > Shared action represent HW resources/objects used for action offloading > > implementation. For allocation/release of all HW resources and all > > related initializations/cleanups in PMD space required for shared action > > implementation added new API > > rte_flow_shared_action_create()/rte_flow_shared_action_destroy(). > > In addition to the above all preparations needed to maintain shared > > access to the action resources, configuration and state should be done in > > rte_flow_shared_action_create(). > > > > In order to share some flow action reuse the handle of type > > `struct rte_flow_shared_action` returned by > > rte_flow_shared_action_create() as a `conf` field of > > `struct rte_flow_action` (see "example" section). > > > > If some shared action not used by any flow rule all resources allocated > > by the shared action can be released by rte_flow_shared_action_destroy() > > (see "example" section). The shared action handle passed as argument to > > destroy API should not be used i.e. result of the usage is undefined. > > > > Shared action re-configuration > > === > > Shared action behavior defined by its configuration & and can be updated > > via rte_flow_shared_action_update() (see "example" section). The shared > > action update operation modifies HW related resources/objects allocated > > by the action. The number of operations performed by the update operation > > should not be dependent on number of flows sharing the related action. > > On return of shared action updated API action behavior should be > > according to updated configuration for all flows sharing the action. > > > > Shared action query > > === > > Provide separate API to query shared action sate (see > > rte_flow_shared_action_update()). Taking a counter as an example: query > > returns value aggregating all counter increments across all flow rules > > sharing the counter. > > > > PMD support > > === > > The support of introduced API is pure PMD specific design and > > responsibility for each action type (see struct rte_flow_ops). > > > > testpmd > > === > > In order to utilize introduced API testpmd cli may implement following > > extension > > create/update/destroy/query shared action accordingly > > > > flow shared_action create {port_id} [index] {action} > > flow shared_action update {port_id} {index} {action} > > flow shared_action destroy {port_id} {index} > > flow shared_action query {port_id} {index} > > > > testpmd example > > === > > > > configure rss to queues 1 & 2 > > > > testpmd> flow shared_action create 0 100 rss 1 2 > > > > create flow rule utilizing shared action > > > > testpmd> flow create 0 ingress \ > > pattern eth dst is 0c:42:a1:15:fd:ac / ipv6 / tcp / end \ > > actions shared 100 end / end > > > > add 2 more queues > > > > testpmd> flow shared_action modify 0 100 rss 1 2 3 4 > > > > example > > === > > > > struct rte_flow_action actions[2]; > > struct rte_flow_action action; > > /* skipped: initialize action */ > > struct rte_flow_shared_action *handle = rte_flow_shared_action_create( > > port_id, &action, &error); > > actions[0].type = RTE_FLOW_ACTION_TYPE_SHARED; > > actions[0].conf = handle; > > actions[1].type = RTE_FLOW_ACTION_TYPE_END; > > /* skipped: init attr0 & pattern0 args */ > > struct rte_flow *flow0 = rte_flow_create(port_id, &attr0, pattern0, > > actions, error); > > /* create more rules reusing shared action */ > > struct rte_flow *flow1 = rte_flow_create(port_id, &attr1, pattern1, > > actions, error); > > /* skipped: for flows 2 till N */ > > struct rte_flow *flowN = rte_flow_create(port_id, &attrN, patternN, > > actions, error); > > /* update shared action */ > > struct rte_flow_action updated_action; > > /* > > * skipped: initialize updated_action according to desired action > > * configuration change > > */ > > rte_flow_shared_action_update(port_id, handle, updated_action.conf, > > error); > > /* > > * from now on all flows 1 till N will act according to configuration of > > * updated_action > > */ > > /* skipped: destroy all flows 1 till N */ > > rte_flow_shared_action_destroy(port_id, handle, error); > > > > Signed-off-by: Andrey Vesnovaty <andreyv@mellanox.com> > > Signed-off-by: Andrey Vesnovaty <andrey.vesnovaty@gmail.com> > > Signed-off-by: Andrey Vesnovaty <andreyv@mellanox.com> > > Duplicate Signoffs. > > # Request to CC all the people who are already giving comments to this > patch. > # The Following comment is not addressed in this patch. > http://mails.dpdk.org/archives/dev/2020-July/172408.html I need to mention the locking issue once again. If there is a need to maintain "shared session" in the generic rte_flow layer all calls to flow_create() with shared action & all delete need to take sharedsession management locks at least for verification. Lock partitioning is also bit problematic since one flow may have more than one shared action. > > > > > --- > > This patch based on RFC: https://patches.dpdk.org/patch/71820/ > > > > --- > > lib/librte_ethdev/rte_ethdev_version.map | 6 + > > lib/librte_ethdev/rte_flow.c | 81 +++++++++++++ > > lib/librte_ethdev/rte_flow.h | 148 ++++++++++++++++++++++- > > lib/librte_ethdev/rte_flow_driver.h | 22 ++++ > > 4 files changed, 256 insertions(+), 1 deletion(-) > > > > diff --git a/lib/librte_ethdev/rte_ethdev_version.map > b/lib/librte_ethdev/rte_ethdev_version.map > > index 3f32fdecf..e291c2bd9 100644 > > --- a/lib/librte_ethdev/rte_ethdev_version.map > > +++ b/lib/librte_ethdev/rte_ethdev_version.map > > @@ -230,4 +230,10 @@ EXPERIMENTAL { > > > > # added in 20.02 > > rte_flow_dev_dump; > > + > > + # added in 20.08 > > + rte_flow_shared_action_create; > > + rte_flow_shared_action_destoy; > > + rte_flow_shared_action_update; > > + rte_flow_shared_action_query; > > }; > > diff --git a/lib/librte_ethdev/rte_flow.c b/lib/librte_ethdev/rte_flow.c > > index 885a7ff9a..7728057c3 100644 > > --- a/lib/librte_ethdev/rte_flow.c > > +++ b/lib/librte_ethdev/rte_flow.c > > @@ -1231,3 +1231,84 @@ rte_flow_dev_dump(uint16_t port_id, FILE *file, > struct rte_flow_error *error) > > RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > > NULL, rte_strerror(ENOSYS)); > > } > > + > > +struct rte_flow_shared_action * > > +rte_flow_shared_action_create(uint16_t port_id, > > + const struct rte_flow_action *action, > > + struct rte_flow_error *error) > > +{ > > + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; > > + struct rte_flow_shared_action *shared_action; > > + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, > error); > > + > > + if (unlikely(!ops)) > > + return NULL; > > + if (likely(!!ops->shared_action_create)) { > > + shared_action = ops->shared_action_create(dev, action, > error); > > + if (shared_action == NULL) > > + flow_err(port_id, -rte_errno, error); > > + return shared_action; > > + } > > + rte_flow_error_set(error, ENOSYS, > RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > > + NULL, rte_strerror(ENOSYS)); > > + return NULL; > > +} > > + > > +int > > +rte_flow_shared_action_destoy(uint16_t port_id, > > + struct rte_flow_shared_action *action, > > + struct rte_flow_error *error) > > +{ > > + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; > > + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, > error); > > + > > + if (unlikely(!ops)) > > + return -rte_errno; > > + if (likely(!!ops->shared_action_destroy)) > > + return flow_err(port_id, > > + ops->shared_action_destroy(dev, action, > error), > > + error); > > + return rte_flow_error_set(error, ENOSYS, > > + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > > + NULL, rte_strerror(ENOSYS)); > > +} > > + > > +int > > +rte_flow_shared_action_update(uint16_t port_id, > > + struct rte_flow_shared_action *action, > > + const void *action_conf, > > + struct rte_flow_error *error) > > +{ > > + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; > > + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, > error); > > + > > + if (unlikely(!ops)) > > + return -rte_errno; > > + if (likely(!!ops->shared_action_update)) > > + return flow_err(port_id, ops->shared_action_update(dev, > action, > > + action_conf, error), > > + error); > > + return rte_flow_error_set(error, ENOSYS, > > + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > > + NULL, rte_strerror(ENOSYS)); > > +} > > + > > +int > > +rte_flow_shared_action_query(uint16_t port_id, > > + const struct rte_flow_shared_action *action, > > + void *data, > > + struct rte_flow_error *error) > > +{ > > + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; > > + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, > error); > > + > > + if (unlikely(!ops)) > > + return -rte_errno; > > + if (likely(!!ops->shared_action_query)) > > + return flow_err(port_id, ops->shared_action_query(dev, > action, > > + data, error), > > + error); > > + return rte_flow_error_set(error, ENOSYS, > > + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > > + NULL, rte_strerror(ENOSYS)); > > +} > > diff --git a/lib/librte_ethdev/rte_flow.h b/lib/librte_ethdev/rte_flow.h > > index 5625dc491..98140ebb1 100644 > > --- a/lib/librte_ethdev/rte_flow.h > > +++ b/lib/librte_ethdev/rte_flow.h > > @@ -1643,7 +1643,8 @@ enum rte_flow_action_type { > > /** > > * Enables counters for this flow rule. > > * > > - * These counters can be retrieved and reset through > rte_flow_query(), > > + * These counters can be retrieved and reset through > rte_flow_query() or > > + * rte_flow_shared_action_query() if the action provided via > handle, > > * see struct rte_flow_query_count. > > * > > * See struct rte_flow_action_count. > > @@ -2051,6 +2052,14 @@ enum rte_flow_action_type { > > * See struct rte_flow_action_set_dscp. > > */ > > RTE_FLOW_ACTION_TYPE_SET_IPV6_DSCP, > > + > > + /** > > + * Describes action shared a cross multiple flow rules. > > + * > > + * Enables multiple rules reference the same action by handle > (see > > + * struct rte_flow_shared_action). > > + */ > > + RTE_FLOW_ACTION_TYPE_SHARED, > > }; > > > > /** > > @@ -2593,6 +2602,20 @@ struct rte_flow_action_set_dscp { > > uint8_t dscp; > > }; > > > > + > > +/** > > + * RTE_FLOW_ACTION_TYPE_SHARED > > + * > > + * Opaque type returned after successfully creating a shared action. > > + * > > + * This handle can be used to manage and query the related action: > > + * - share it a cross multiple flow rules > > + * - update action configuration > > + * - query action data > > + * - destroy action > > + */ > > +struct rte_flow_shared_action; > > + > > /* Mbuf dynamic field offset for metadata. */ > > extern int rte_flow_dynf_metadata_offs; > > > > @@ -3224,6 +3247,129 @@ rte_flow_conv(enum rte_flow_conv_op op, > > const void *src, > > struct rte_flow_error *error); > > > > +/** > > + * @warning > > + * @b EXPERIMENTAL: this API may change without prior notice. > > + * > > + * Create shared action for reuse in multiple flow rules. > > + * > > + * @param[in] port_id > > + * The port identifier of the Ethernet device. > > + * @param[in] action > > + * Action configuration for shared action creation. > > + * @param[out] error > > + * Perform verbose error reporting if not NULL. PMDs initialize this > > + * structure in case of error only. > > + * @return > > + * A valid handle in case of success, NULL otherwise and rte_errno is > set > > + * to one of the error codes defined: > > + * - (ENOSYS) if underlying device does not support this > functionality. > > + * - (EIO) if underlying device is removed. > > + * - (EINVAL) if *action* invalid. > > + * - (ENOTSUP) if *action* valid but unsupported. > > + */ > > +__rte_experimental > > +struct rte_flow_shared_action * > > +rte_flow_shared_action_create(uint16_t port_id, > > + const struct rte_flow_action *action, > > + struct rte_flow_error *error); > > + > > +/** > > + * @warning > > + * @b EXPERIMENTAL: this API may change without prior notice. > > + * > > + * Destroys the shared action by handle. > > + * > > + * @param[in] port_id > > + * The port identifier of the Ethernet device. > > + * @param[in] action > > + * Handle for the shared action to be destroyed. > > + * @param[out] error > > + * Perform verbose error reporting if not NULL. PMDs initialize this > > + * structure in case of error only. > > + * @return > > + * - (0) if success. > > + * - (-ENOSYS) if underlying device does not support this > functionality. > > + * - (-EIO) if underlying device is removed. > > + * - (-ENOENT) if action pointed by *action* handle was not found. > > + * - (-ETOOMANYREFS) if action pointed by *action* handle still used > by one or > > + * more rules > > + * rte_errno is also set. > > + */ > > +__rte_experimental > > +int > > +rte_flow_shared_action_destoy(uint16_t port_id, > > + struct rte_flow_shared_action *action, > > + struct rte_flow_error *error); > > + > > +/** > > + * @warning > > + * @b EXPERIMENTAL: this API may change without prior notice. > > + * > > + * Updates inplace the shared action configuration pointed by *action* > handle > > + * with the configuration provided as *action_conf* argument. > > + * The update of the shared action configuration effects all flow rules > reusing > > + * the action via handle. > > + * > > + * @param[in] port_id > > + * The port identifier of the Ethernet device. > > + * @param[in] action > > + * Handle for the shared action to be updated. > > + * @param[in] action_conf > > + * Action specification used to modify the action pointed by handle. > > + * action_conf should be of same type with the action pointed by the > *action* > > + * handle argument, otherwise function behavior undefined. > > + * @param[out] error > > + * Perform verbose error reporting if not NULL. PMDs initialize this > > + * structure in case of error only. > > + * @return > > + * - (0) if success. > > + * - (-ENOSYS) if underlying device does not support this > functionality. > > + * - (-EIO) if underlying device is removed. > > + * - (-EINVAL) if *action_conf* invalid. > > + * - (-ENOTSUP) if *action_conf* valid but unsupported. > > + * - (-ENOENT) if action pointed by *ctx* was not found. > > + * rte_errno is also set. > > + */ > > +__rte_experimental > > +int > > +rte_flow_shared_action_update(uint16_t port_id, > > + struct rte_flow_shared_action *action, > > + const void *action_conf, > > + struct rte_flow_error *error); > > + > > +/** > > + * @warning > > + * @b EXPERIMENTAL: this API may change without prior notice. > > + * > > + * Query the shared action by handle. > > + * > > + * This function allows retrieving action-specific data such as > counters. > > + * Data is gathered by special action which may be present/referenced in > > + * more than one flow rule definition. > > + * > > + * \see RTE_FLOW_ACTION_TYPE_COUNT > > + * > > + * @param port_id > > + * Port identifier of Ethernet device. > > + * @param[in] action > > + * Handle for the shared action to query. > > + * @param[in, out] data > > + * Pointer to storage for the associated query data type. > > + * @param[out] error > > + * Perform verbose error reporting if not NULL. PMDs initialize this > > + * structure in case of error only. > > + * > > + * @return > > + * 0 on success, a negative errno value otherwise and rte_errno is > set. > > + */ > > +__rte_experimental > > +int > > +rte_flow_shared_action_query(uint16_t port_id, > > + const struct rte_flow_shared_action *action, > > + void *data, > > + struct rte_flow_error *error); > > + > > #ifdef __cplusplus > > } > > #endif > > diff --git a/lib/librte_ethdev/rte_flow_driver.h > b/lib/librte_ethdev/rte_flow_driver.h > > index 51a9a57a0..c103d159e 100644 > > --- a/lib/librte_ethdev/rte_flow_driver.h > > +++ b/lib/librte_ethdev/rte_flow_driver.h > > @@ -101,6 +101,28 @@ struct rte_flow_ops { > > (struct rte_eth_dev *dev, > > FILE *file, > > struct rte_flow_error *error); > > + /** See rte_flow_shared_action_create() */ > > + struct rte_flow_shared_action *(*shared_action_create) > > + (struct rte_eth_dev *dev, > > + const struct rte_flow_action *action, > > + struct rte_flow_error *error); > > + /** See rte_flow_shared_action_destroy() */ > > + int (*shared_action_destroy) > > + (struct rte_eth_dev *dev, > > + struct rte_flow_shared_action *shared_action, > > + struct rte_flow_error *error); > > + /** See rte_flow_shared_action_update() */ > > + int (*shared_action_update) > > + (struct rte_eth_dev *dev, > > + struct rte_flow_shared_action *shared_action, > > + const void *action_conf, > > + struct rte_flow_error *error); > > + /** See rte_flow_shared_action_query() */ > > + int (*shared_action_query) > > + (struct rte_eth_dev *dev, > > + const struct rte_flow_shared_action *shared_action, > > + void *data, > > + struct rte_flow_error *error); > > }; > > > > /** > > -- > > 2.26.2 > > > ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [dpdk-dev] [PATCH] add flow shared action API 2020-07-04 10:10 ` Andrey Vesnovaty @ 2020-07-04 12:33 ` Jerin Jacob 2020-07-05 10:26 ` Ori Kam 0 siblings, 1 reply; 106+ messages in thread From: Jerin Jacob @ 2020-07-04 12:33 UTC (permalink / raw) To: Andrey Vesnovaty Cc: Andrey Vesnovaty, Thomas Monjalon, Ferruh Yigit, Andrew Rybchenko, Ori Kam, dpdk-dev On Sat, Jul 4, 2020 at 3:40 PM Andrey Vesnovaty <andrey.vesnovaty@gmail.com> wrote: > > > Thanks, > > Andrey Vesnovaty > (+972)526775512 | Skype: andrey775512 > > > On Fri, Jul 3, 2020 at 6:02 PM Jerin Jacob <jerinjacobk@gmail.com> wrote: >> >> On Fri, Jul 3, 2020 at 8:07 PM Andrey Vesnovaty <andreyv@mellanox.com> wrote: >> > >> > From: Andrey Vesnovaty <andrey.vesnovaty@gmail.com> >> > >> > This commit introduces extension of DPDK flow action API enabling >> > sharing of single rte_flow_action in multiple flows. The API intended for >> > PMDs where multiple HW offloaded flows can reuse the same HW >> > essence/object representing flow action and modification of such an >> > essence/object effects all the rules using it. >> > >> > Motivation and example >> > === >> > Adding or removing one or more queues to RSS used by multiple flow rules >> > imposes per rule toll for current DPDK flow API; the scenario requires >> > for each flow sharing cloned RSS action: >> > - call `rte_flow_destroy()` >> > - call `rte_flow_create()` with modified RSS action >> > >> > API for sharing action and its in-place update benefits: >> > - reduce the overhead of multiple RSS flow rules reconfiguration . >> > - optimize resource utilization by sharing action across of of multiple >> > flows >> > >> > Change description >> > === >> > >> > Shared action >> > === >> > In order to represent flow action shared by multiple flows new action >> > type RTE_FLOW_ACTION_TYPE_SHARED introduced (see `enum >> > rte_flow_action_type`). >> > Actually the introduced API decouples action from any specific flow and >> > enables sharing of single action by its handle for multiple flows. >> > >> > Shared action create/use/destroy >> > === >> > Shared action may be reused by some or none flow rules at any given >> > moment, IOW shared action reside outside of the context of any flow. >> > Shared action represent HW resources/objects used for action offloading >> > implementation. For allocation/release of all HW resources and all >> > related initializations/cleanups in PMD space required for shared action >> > implementation added new API >> > rte_flow_shared_action_create()/rte_flow_shared_action_destroy(). >> > In addition to the above all preparations needed to maintain shared >> > access to the action resources, configuration and state should be done in >> > rte_flow_shared_action_create(). >> > >> > In order to share some flow action reuse the handle of type >> > `struct rte_flow_shared_action` returned by >> > rte_flow_shared_action_create() as a `conf` field of >> > `struct rte_flow_action` (see "example" section). >> > >> > If some shared action not used by any flow rule all resources allocated >> > by the shared action can be released by rte_flow_shared_action_destroy() >> > (see "example" section). The shared action handle passed as argument to >> > destroy API should not be used i.e. result of the usage is undefined. >> > >> > Shared action re-configuration >> > === >> > Shared action behavior defined by its configuration & and can be updated >> > via rte_flow_shared_action_update() (see "example" section). The shared >> > action update operation modifies HW related resources/objects allocated >> > by the action. The number of operations performed by the update operation >> > should not be dependent on number of flows sharing the related action. >> > On return of shared action updated API action behavior should be >> > according to updated configuration for all flows sharing the action. >> > >> > Shared action query >> > === >> > Provide separate API to query shared action sate (see >> > rte_flow_shared_action_update()). Taking a counter as an example: query >> > returns value aggregating all counter increments across all flow rules >> > sharing the counter. >> > >> > PMD support >> > === >> > The support of introduced API is pure PMD specific design and >> > responsibility for each action type (see struct rte_flow_ops). >> > >> > testpmd >> > === >> > In order to utilize introduced API testpmd cli may implement following >> > extension >> > create/update/destroy/query shared action accordingly >> > >> > flow shared_action create {port_id} [index] {action} >> > flow shared_action update {port_id} {index} {action} >> > flow shared_action destroy {port_id} {index} >> > flow shared_action query {port_id} {index} >> > >> > testpmd example >> > === >> > >> > configure rss to queues 1 & 2 >> > >> > testpmd> flow shared_action create 0 100 rss 1 2 >> > >> > create flow rule utilizing shared action >> > >> > testpmd> flow create 0 ingress \ >> > pattern eth dst is 0c:42:a1:15:fd:ac / ipv6 / tcp / end \ >> > actions shared 100 end / end >> > >> > add 2 more queues >> > >> > testpmd> flow shared_action modify 0 100 rss 1 2 3 4 >> > >> > example >> > === >> > >> > struct rte_flow_action actions[2]; >> > struct rte_flow_action action; >> > /* skipped: initialize action */ >> > struct rte_flow_shared_action *handle = rte_flow_shared_action_create( >> > port_id, &action, &error); >> > actions[0].type = RTE_FLOW_ACTION_TYPE_SHARED; >> > actions[0].conf = handle; >> > actions[1].type = RTE_FLOW_ACTION_TYPE_END; >> > /* skipped: init attr0 & pattern0 args */ >> > struct rte_flow *flow0 = rte_flow_create(port_id, &attr0, pattern0, >> > actions, error); >> > /* create more rules reusing shared action */ >> > struct rte_flow *flow1 = rte_flow_create(port_id, &attr1, pattern1, >> > actions, error); >> > /* skipped: for flows 2 till N */ >> > struct rte_flow *flowN = rte_flow_create(port_id, &attrN, patternN, >> > actions, error); >> > /* update shared action */ >> > struct rte_flow_action updated_action; >> > /* >> > * skipped: initialize updated_action according to desired action >> > * configuration change >> > */ >> > rte_flow_shared_action_update(port_id, handle, updated_action.conf, >> > error); >> > /* >> > * from now on all flows 1 till N will act according to configuration of >> > * updated_action >> > */ >> > /* skipped: destroy all flows 1 till N */ >> > rte_flow_shared_action_destroy(port_id, handle, error); >> > >> > Signed-off-by: Andrey Vesnovaty <andreyv@mellanox.com> >> > Signed-off-by: Andrey Vesnovaty <andrey.vesnovaty@gmail.com> >> > Signed-off-by: Andrey Vesnovaty <andreyv@mellanox.com> >> >> Duplicate Signoffs. >> >> # Request to CC all the people who are already giving comments to this patch. >> # The Following comment is not addressed in this patch. >> http://mails.dpdk.org/archives/dev/2020-July/172408.html > > > I need to mention the locking issue once again. > If there is a need to maintain "shared session" in the generic rte_flow layer all > calls to flow_create() with shared action & all delete need to take sharedsession > management locks at least for verification. Lock partitioning is also bit problematic > since one flow may have more than one shared action. Then, I think better approach would be to introduce rte_flow_action_update() public API which can either take "const struct rte_flow_action []" OR shared context ID, to cater to both cases or something on similar lines. This would allow HW's without have the shared context ID to use the action update. ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [dpdk-dev] [PATCH] add flow shared action API 2020-07-04 12:33 ` Jerin Jacob @ 2020-07-05 10:26 ` Ori Kam 2020-07-06 9:00 ` Jerin Jacob 0 siblings, 1 reply; 106+ messages in thread From: Ori Kam @ 2020-07-05 10:26 UTC (permalink / raw) To: Jerin Jacob, Andrey Vesnovaty Cc: Andrey Vesnovaty, Thomas Monjalon, Ferruh Yigit, Andrew Rybchenko, dpdk-dev Hi Jerin, PSB, Thanks, Ori > -----Original Message----- > From: Jerin Jacob <jerinjacobk@gmail.com> > Sent: Saturday, July 4, 2020 3:33 PM > dpdk-dev <dev@dpdk.org> > Subject: Re: [dpdk-dev] [PATCH] add flow shared action API > > On Sat, Jul 4, 2020 at 3:40 PM Andrey Vesnovaty > <andrey.vesnovaty@gmail.com> wrote: > > > > > > Thanks, > > > > Andrey Vesnovaty > > (+972)526775512 | Skype: andrey775512 > > [..Nip ..] > > I need to mention the locking issue once again. > > If there is a need to maintain "shared session" in the generic rte_flow layer > all > > calls to flow_create() with shared action & all delete need to take > sharedsession > > management locks at least for verification. Lock partitioning is also bit > problematic > > since one flow may have more than one shared action. > > Then, I think better approach would be to introduce > rte_flow_action_update() public > API which can either take "const struct rte_flow_action []" OR shared > context ID, to cater to > both cases or something on similar lines. This would allow HW's > without have the shared context ID > to use the action update. Can you please explain your idea? As I can see if we use the flow_action array it may result in bugs. For example, the application created two flows with the same RSS (not using the context) Then he wants to change one flow to use different RSS, but the result will that both flows will be changed. Also this will enforce the PMD to keep track on all flows which will have memory penalty for some PMDs. ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [dpdk-dev] [PATCH] add flow shared action API 2020-07-05 10:26 ` Ori Kam @ 2020-07-06 9:00 ` Jerin Jacob 2020-07-06 12:28 ` Ori Kam 0 siblings, 1 reply; 106+ messages in thread From: Jerin Jacob @ 2020-07-06 9:00 UTC (permalink / raw) To: Ori Kam Cc: Andrey Vesnovaty, Andrey Vesnovaty, Thomas Monjalon, Ferruh Yigit, Andrew Rybchenko, dpdk-dev On Sun, Jul 5, 2020 at 3:56 PM Ori Kam <orika@mellanox.com> wrote: > > Hi Jerin, > PSB, > > Thanks, > Ori > > > -----Original Message----- > > From: Jerin Jacob <jerinjacobk@gmail.com> > > Sent: Saturday, July 4, 2020 3:33 PM > > dpdk-dev <dev@dpdk.org> > > Subject: Re: [dpdk-dev] [PATCH] add flow shared action API > > > > On Sat, Jul 4, 2020 at 3:40 PM Andrey Vesnovaty > > <andrey.vesnovaty@gmail.com> wrote: > > > > > > > > > Thanks, > > > > > > Andrey Vesnovaty > > > (+972)526775512 | Skype: andrey775512 > > > > > > [..Nip ..] > > > > I need to mention the locking issue once again. > > > If there is a need to maintain "shared session" in the generic rte_flow layer > > all > > > calls to flow_create() with shared action & all delete need to take > > sharedsession > > > management locks at least for verification. Lock partitioning is also bit > > problematic > > > since one flow may have more than one shared action. > > > > Then, I think better approach would be to introduce > > rte_flow_action_update() public > > API which can either take "const struct rte_flow_action []" OR shared > > context ID, to cater to > > both cases or something on similar lines. This would allow HW's > > without have the shared context ID > > to use the action update. > > Can you please explain your idea? I see two types of HW schemes supporting action updates without going through call `rte_flow_destroy()` and call `rte_flow_create()` - The shared HW action context feature - The HW has "pattern" and "action" mapped to different HW objects and action can be updated any time. Other than above-mentioned RSS use case, another use case would be to a) create rte_flow and set the action as DROP (Kind of reserving the HW object) b) Update the action only when the rest of the requirements ready. Any API schematic that supports both notions of HW is fine with me. > As I can see if we use the flow_action array it may result in bugs. > For example, the application created two flows with the same RSS (not using the context) > Then he wants to change one flow to use different RSS, but the result will that both flows > will be changed. Sorry. I don't quite follow this. > Also this will enforce the PMD to keep track on all flows which will have memory penalty for > some PMDs. ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [dpdk-dev] [PATCH] add flow shared action API 2020-07-06 9:00 ` Jerin Jacob @ 2020-07-06 12:28 ` Ori Kam 2020-07-06 13:32 ` Andrey Vesnovaty 0 siblings, 1 reply; 106+ messages in thread From: Ori Kam @ 2020-07-06 12:28 UTC (permalink / raw) To: Jerin Jacob Cc: Andrey Vesnovaty, Andrey Vesnovaty, Thomas Monjalon, Ferruh Yigit, Andrew Rybchenko, dpdk-dev Hi Jerin, > -----Original Message----- > From: Jerin Jacob <jerinjacobk@gmail.com> > Sent: Monday, July 6, 2020 12:00 PM > Subject: Re: [dpdk-dev] [PATCH] add flow shared action API > > On Sun, Jul 5, 2020 at 3:56 PM Ori Kam <orika@mellanox.com> wrote: > > > > Hi Jerin, > > PSB, > > > > Thanks, > > Ori > > > > > -----Original Message----- > > > From: Jerin Jacob <jerinjacobk@gmail.com> > > > Sent: Saturday, July 4, 2020 3:33 PM > > > dpdk-dev <dev@dpdk.org> > > > Subject: Re: [dpdk-dev] [PATCH] add flow shared action API > > > > > > On Sat, Jul 4, 2020 at 3:40 PM Andrey Vesnovaty > > > <andrey.vesnovaty@gmail.com> wrote: > > > > > > > > > > > > Thanks, > > > > > > > > Andrey Vesnovaty > > > > (+972)526775512 | Skype: andrey775512 > > > > > > > > > > [..Nip ..] > > > > > > I need to mention the locking issue once again. > > > > If there is a need to maintain "shared session" in the generic rte_flow > layer > > > all > > > > calls to flow_create() with shared action & all delete need to take > > > sharedsession > > > > management locks at least for verification. Lock partitioning is also bit > > > problematic > > > > since one flow may have more than one shared action. > > > > > > Then, I think better approach would be to introduce > > > rte_flow_action_update() public > > > API which can either take "const struct rte_flow_action []" OR shared > > > context ID, to cater to > > > both cases or something on similar lines. This would allow HW's > > > without have the shared context ID > > > to use the action update. > > > > Can you please explain your idea? > > I see two types of HW schemes supporting action updates without going > through call `rte_flow_destroy()` and call `rte_flow_create()` > - The shared HW action context feature > - The HW has "pattern" and "action" mapped to different HW objects and > action can be updated any time. > Other than above-mentioned RSS use case, another use case would be to > a) create rte_flow and set the action as DROP (Kind of reserving the HW object) > b) Update the action only when the rest of the requirements ready. > > Any API schematic that supports both notions of HW is fine with me. > I have an idea if the API will be changed to something like this, Rte_flow_shared_action_update(uint16_port port, rte_shared_ctx *ctx, rte_flow_action *action, error) This will enable the application to send a different action than the original one to be switched. Assuming the PMD supports this. Does it answer your concerns? > > > As I can see if we use the flow_action array it may result in bugs. > > For example, the application created two flows with the same RSS (not using > the context) > > Then he wants to change one flow to use different RSS, but the result will that > both flows > > will be changed. > > Sorry. I don't quite follow this. > I was trying to show that there must be some context. But I don’t think this is relevant to your current ideas. > > Also this will enforce the PMD to keep track on all flows which will have > memory penalty for > > some PMDs. Best, Ori ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [dpdk-dev] [PATCH] add flow shared action API 2020-07-06 12:28 ` Ori Kam @ 2020-07-06 13:32 ` Andrey Vesnovaty 2020-07-07 2:30 ` Jerin Jacob 0 siblings, 1 reply; 106+ messages in thread From: Andrey Vesnovaty @ 2020-07-06 13:32 UTC (permalink / raw) To: Ori Kam, Jerin Jacob Cc: Andrey Vesnovaty, Thomas Monjalon, Ferruh Yigit, Andrew Rybchenko, dpdk-dev Hi, Jerin. Please see below Ori's suggestion below to implement your rte_flow_action_update() idea with some API changes of rte_flow_shared_action_xxx API changes. On Mon, Jul 6, 2020 at 3:28 PM Ori Kam <orika@mellanox.com> wrote: > Hi Jerin, > > > -----Original Message----- > > From: Jerin Jacob <jerinjacobk@gmail.com> > > Sent: Monday, July 6, 2020 12:00 PM > > Subject: Re: [dpdk-dev] [PATCH] add flow shared action API > > > > On Sun, Jul 5, 2020 at 3:56 PM Ori Kam <orika@mellanox.com> wrote: > > > > > > Hi Jerin, > > > PSB, > > > > > > Thanks, > > > Ori > > > > > > > -----Original Message----- > > > > From: Jerin Jacob <jerinjacobk@gmail.com> > > > > Sent: Saturday, July 4, 2020 3:33 PM > > > > dpdk-dev <dev@dpdk.org> > > > > Subject: Re: [dpdk-dev] [PATCH] add flow shared action API > > > > > > > > On Sat, Jul 4, 2020 at 3:40 PM Andrey Vesnovaty > > > > <andrey.vesnovaty@gmail.com> wrote: > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > Andrey Vesnovaty > > > > > (+972)526775512 | Skype: andrey775512 > > > > > > > > > > > > > > [..Nip ..] > > > > > > > > I need to mention the locking issue once again. > > > > > If there is a need to maintain "shared session" in the generic > rte_flow > > layer > > > > all > > > > > calls to flow_create() with shared action & all delete need to take > > > > sharedsession > > > > > management locks at least for verification. Lock partitioning is > also bit > > > > problematic > > > > > since one flow may have more than one shared action. > > > > > > > > Then, I think better approach would be to introduce > > > > rte_flow_action_update() public > > > > API which can either take "const struct rte_flow_action []" OR shared > > > > context ID, to cater to > > > > both cases or something on similar lines. This would allow HW's > > > > without have the shared context ID > > > > to use the action update. > > > > > > Can you please explain your idea? > > > > I see two types of HW schemes supporting action updates without going > > through call `rte_flow_destroy()` and call `rte_flow_create()` > > - The shared HW action context feature > > - The HW has "pattern" and "action" mapped to different HW objects and > > action can be updated any time. > > Other than above-mentioned RSS use case, another use case would be to > > a) create rte_flow and set the action as DROP (Kind of reserving the HW > object) > > b) Update the action only when the rest of the requirements ready. > > > > Any API schematic that supports both notions of HW is fine with me. > > > I have an idea if the API will be changed to something like this, > Rte_flow_shared_action_update(uint16_port port, rte_shared_ctx *ctx, > rte_flow_action *action, error) > This will enable the application to send a different action than the > original one to be switched. > Assuming the PMD supports this. > Does it answer your concerns? > This allows both: 1. Update action configuration 2. Replace action by some other action For 2 pure software implementation may carate shred action (that can be shared with one flow only, depends on PMD) and later on rte_flow_shared_action_update may replace this action with some other action by handle returned from rte_flow_shared_action_create Doesign between 1 and 2 is per PMD. > > > > > As I can see if we use the flow_action array it may result in bugs. > > > For example, the application created two flows with the same RSS (not > using > > the context) > > > Then he wants to change one flow to use different RSS, but the result > will that > > both flows > > > will be changed. > > > > Sorry. I don't quite follow this. > > > I was trying to show that there must be some context. But I don’t think > this is relevant to > your current ideas. > > > > Also this will enforce the PMD to keep track on all flows which will > have > > memory penalty for > > > some PMDs. > > Best, > Ori > Thanks, Andrey ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [dpdk-dev] [PATCH] add flow shared action API 2020-07-06 13:32 ` Andrey Vesnovaty @ 2020-07-07 2:30 ` Jerin Jacob 2020-07-07 6:21 ` Ori Kam 0 siblings, 1 reply; 106+ messages in thread From: Jerin Jacob @ 2020-07-07 2:30 UTC (permalink / raw) To: Andrey Vesnovaty Cc: Ori Kam, Andrey Vesnovaty, Thomas Monjalon, Ferruh Yigit, Andrew Rybchenko, dpdk-dev On Mon, Jul 6, 2020 at 7:02 PM Andrey Vesnovaty <andrey.vesnovaty@gmail.com> wrote: > > Hi, Jerin. Hi Ori and Andrey, > > Please see below Ori's suggestion below to implement your rte_flow_action_update() idea > with some API changes of rte_flow_shared_action_xxx API changes. > > On Mon, Jul 6, 2020 at 3:28 PM Ori Kam <orika@mellanox.com> wrote: >> >> Hi Jerin, >> >> > -----Original Message----- >> > From: Jerin Jacob <jerinjacobk@gmail.com> >> > Sent: Monday, July 6, 2020 12:00 PM >> > Subject: Re: [dpdk-dev] [PATCH] add flow shared action API >> > >> > On Sun, Jul 5, 2020 at 3:56 PM Ori Kam <orika@mellanox.com> wrote: >> > > >> > > Hi Jerin, >> > > PSB, >> > > >> > > Thanks, >> > > Ori >> > > >> > > > -----Original Message----- >> > > > From: Jerin Jacob <jerinjacobk@gmail.com> >> > > > Sent: Saturday, July 4, 2020 3:33 PM >> > > > dpdk-dev <dev@dpdk.org> >> > > > Subject: Re: [dpdk-dev] [PATCH] add flow shared action API >> > > > >> > > > On Sat, Jul 4, 2020 at 3:40 PM Andrey Vesnovaty >> > > > <andrey.vesnovaty@gmail.com> wrote: >> > > > > >> > > > > >> > > > > Thanks, >> > > > > >> > > > > Andrey Vesnovaty >> > > > > (+972)526775512 | Skype: andrey775512 >> > > > > >> > > >> > > >> > > [..Nip ..] >> > > >> > > > > I need to mention the locking issue once again. >> > > > > If there is a need to maintain "shared session" in the generic rte_flow >> > layer >> > > > all >> > > > > calls to flow_create() with shared action & all delete need to take >> > > > sharedsession >> > > > > management locks at least for verification. Lock partitioning is also bit >> > > > problematic >> > > > > since one flow may have more than one shared action. >> > > > >> > > > Then, I think better approach would be to introduce >> > > > rte_flow_action_update() public >> > > > API which can either take "const struct rte_flow_action []" OR shared >> > > > context ID, to cater to >> > > > both cases or something on similar lines. This would allow HW's >> > > > without have the shared context ID >> > > > to use the action update. >> > > >> > > Can you please explain your idea? >> > >> > I see two types of HW schemes supporting action updates without going >> > through call `rte_flow_destroy()` and call `rte_flow_create()` >> > - The shared HW action context feature >> > - The HW has "pattern" and "action" mapped to different HW objects and >> > action can be updated any time. >> > Other than above-mentioned RSS use case, another use case would be to >> > a) create rte_flow and set the action as DROP (Kind of reserving the HW object) >> > b) Update the action only when the rest of the requirements ready. >> > >> > Any API schematic that supports both notions of HW is fine with me. >> > >> I have an idea if the API will be changed to something like this, >> Rte_flow_shared_action_update(uint16_port port, rte_shared_ctx *ctx, rte_flow_action *action, error) >> This will enable the application to send a different action than the original one to be switched. >> Assuming the PMD supports this. >> Does it answer your concerns? > > > This allows both: > 1. Update action configuration > 2. Replace action by some other action > For 2 pure software implementation may carate shred action (that can be shared > with one flow only, depends on PMD) and later on rte_flow_shared_action_update may replace this > action with some other action by handle returned from rte_flow_shared_action_create > Doesign between 1 and 2 is per PMD. struct rte_flow * object holds the driver representation of the pattern + action. So in order to update the action, we would need struct rte_flow * in API. I think, simple API change would be to accommodate "rte_shared_ctx *ctx, rte_flow_action *action" modes without introducing the emulation for one or other mode, will be. enum rte_flow_action_update_type { RTE_FLOW_ACTION_UPDATE_TYPE_SHARED_ACTION, RTE_FLOW_ACTION_UPDATE_TYPE_ACTION, }; struct rte_flow_action_update_type_param { enum rte_flow_action_update_type type; union { struct rte_flow_action_update_type_shared_action_param { rte_shared_ctx *ctx; } shared_action; struct rte_flow_action_update_type_shared_action_param { rte_flow *flow, rte_flow_action *action; } action; } } rte_flow_action_update(uint16_port port, struct rte_flow_action_update_type_param *param, error) > >> >> > >> > > As I can see if we use the flow_action array it may result in bugs. >> > > For example, the application created two flows with the same RSS (not using >> > the context) >> > > Then he wants to change one flow to use different RSS, but the result will that >> > both flows >> > > will be changed. >> > >> > Sorry. I don't quite follow this. >> > >> I was trying to show that there must be some context. But I don’t think this is relevant to >> your current ideas. >> >> > > Also this will enforce the PMD to keep track on all flows which will have >> > memory penalty for >> > > some PMDs. >> >> Best, >> Ori > > > Thanks, > Andrey ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [dpdk-dev] [PATCH] add flow shared action API 2020-07-07 2:30 ` Jerin Jacob @ 2020-07-07 6:21 ` Ori Kam 2020-07-07 15:21 ` Ferruh Yigit 2020-07-07 19:38 ` Jerin Jacob 0 siblings, 2 replies; 106+ messages in thread From: Ori Kam @ 2020-07-07 6:21 UTC (permalink / raw) To: Jerin Jacob, Andrey Vesnovaty Cc: Andrey Vesnovaty, Thomas Monjalon, Ferruh Yigit, Andrew Rybchenko, dpdk-dev Hi Jerin, Thanks you for your quick reply. > -----Original Message----- > From: Jerin Jacob <jerinjacobk@gmail.com> > Subject: Re: [dpdk-dev] [PATCH] add flow shared action API > > On Mon, Jul 6, 2020 at 7:02 PM Andrey Vesnovaty > <andrey.vesnovaty@gmail.com> wrote: > > > > Hi, Jerin. > > Hi Ori and Andrey, > > > > > > Please see below Ori's suggestion below to implement your > rte_flow_action_update() idea > > with some API changes of rte_flow_shared_action_xxx API changes. > > > > On Mon, Jul 6, 2020 at 3:28 PM Ori Kam <orika@mellanox.com> wrote: > >> > >> Hi Jerin, > >> > >> > -----Original Message----- > >> > From: Jerin Jacob <jerinjacobk@gmail.com> > >> > Sent: Monday, July 6, 2020 12:00 PM > >> > Subject: Re: [dpdk-dev] [PATCH] add flow shared action API > >> > > >> > On Sun, Jul 5, 2020 at 3:56 PM Ori Kam <orika@mellanox.com> wrote: > >> > > > >> > > Hi Jerin, > >> > > PSB, > >> > > > >> > > Thanks, > >> > > Ori > >> > > > >> > > > -----Original Message----- > >> > > > From: Jerin Jacob <jerinjacobk@gmail.com> > >> > > > Sent: Saturday, July 4, 2020 3:33 PM > >> > > > dpdk-dev <dev@dpdk.org> > >> > > > Subject: Re: [dpdk-dev] [PATCH] add flow shared action API > >> > > > > >> > > > On Sat, Jul 4, 2020 at 3:40 PM Andrey Vesnovaty > >> > > > <andrey.vesnovaty@gmail.com> wrote: > >> > > > > > >> > > > > > >> > > > > Thanks, > >> > > > > > >> > > > > Andrey Vesnovaty > >> > > > > (+972)526775512 | Skype: andrey775512 > >> > > > > > >> > > > >> > > > >> > > [..Nip ..] > >> > > > >> > > > > I need to mention the locking issue once again. > >> > > > > If there is a need to maintain "shared session" in the generic > rte_flow > >> > layer > >> > > > all > >> > > > > calls to flow_create() with shared action & all delete need to take > >> > > > sharedsession > >> > > > > management locks at least for verification. Lock partitioning is also > bit > >> > > > problematic > >> > > > > since one flow may have more than one shared action. > >> > > > > >> > > > Then, I think better approach would be to introduce > >> > > > rte_flow_action_update() public > >> > > > API which can either take "const struct rte_flow_action []" OR shared > >> > > > context ID, to cater to > >> > > > both cases or something on similar lines. This would allow HW's > >> > > > without have the shared context ID > >> > > > to use the action update. > >> > > > >> > > Can you please explain your idea? > >> > > >> > I see two types of HW schemes supporting action updates without going > >> > through call `rte_flow_destroy()` and call `rte_flow_create()` > >> > - The shared HW action context feature > >> > - The HW has "pattern" and "action" mapped to different HW objects and > >> > action can be updated any time. > >> > Other than above-mentioned RSS use case, another use case would be to > >> > a) create rte_flow and set the action as DROP (Kind of reserving the HW > object) > >> > b) Update the action only when the rest of the requirements ready. > >> > > >> > Any API schematic that supports both notions of HW is fine with me. > >> > > >> I have an idea if the API will be changed to something like this, > >> Rte_flow_shared_action_update(uint16_port port, rte_shared_ctx *ctx, > rte_flow_action *action, error) > >> This will enable the application to send a different action than the original > one to be switched. > >> Assuming the PMD supports this. > >> Does it answer your concerns? > > > > > > This allows both: > > 1. Update action configuration > > 2. Replace action by some other action > > For 2 pure software implementation may carate shred action (that can be > shared > > with one flow only, depends on PMD) and later on > rte_flow_shared_action_update may replace this > > action with some other action by handle returned from > rte_flow_shared_action_create > > Doesign between 1 and 2 is per PMD. > > struct rte_flow * object holds the driver representation of the > pattern + action. > So in order to update the action, we would need struct rte_flow * in API. > Why is that? The idea is to change the action, the action itself is connected to flows. The PMD can save in the shared_ctx all flows that are connected to this action. > I think, simple API change would be to accommodate "rte_shared_ctx > *ctx, rte_flow_action *action" modes > without introducing the emulation for one or other mode, will be. > > enum rte_flow_action_update_type { > RTE_FLOW_ACTION_UPDATE_TYPE_SHARED_ACTION, > RTE_FLOW_ACTION_UPDATE_TYPE_ACTION, > }; > > struct rte_flow_action_update_type_param { > enum rte_flow_action_update_type type; > union { > struct rte_flow_action_update_type_shared_action_param { > rte_shared_ctx *ctx; > } shared_action; > struct rte_flow_action_update_type_shared_action_param { > rte_flow *flow, > rte_flow_action *action; > } action; > } > } > Thank you for the idea but I fall to see how your suggested API is simpler than the one suggested by me? In my suggestion the PMD simply needs to check if the new action and change the context and to that action, or just change parameters in the action, if it is the same action. Let's go with the original patch API modified to support like you requested also changing the action, based on my comments. > rte_flow_action_update(uint16_port port, struct > rte_flow_action_update_type_param *param, error) > > > > >> > >> > > >> > > As I can see if we use the flow_action array it may result in bugs. > >> > > For example, the application created two flows with the same RSS (not > using > >> > the context) > >> > > Then he wants to change one flow to use different RSS, but the result will > that > >> > both flows > >> > > will be changed. > >> > > >> > Sorry. I don't quite follow this. > >> > > >> I was trying to show that there must be some context. But I don’t think this is > relevant to > >> your current ideas. > >> > >> > > Also this will enforce the PMD to keep track on all flows which will have > >> > memory penalty for > >> > > some PMDs. > >> > >> Best, > >> Ori > > > > > > Thanks, > > Andrey Best, Ori ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [dpdk-dev] [PATCH] add flow shared action API 2020-07-07 6:21 ` Ori Kam @ 2020-07-07 15:21 ` Ferruh Yigit 2020-07-07 17:24 ` Ori Kam 2020-07-07 19:38 ` Jerin Jacob 1 sibling, 1 reply; 106+ messages in thread From: Ferruh Yigit @ 2020-07-07 15:21 UTC (permalink / raw) To: Ori Kam, Jerin Jacob, Andrey Vesnovaty Cc: Andrey Vesnovaty, Thomas Monjalon, Andrew Rybchenko, dpdk-dev On 7/7/2020 7:21 AM, Ori Kam wrote: > Hi Jerin, > Thanks you for your quick reply. > >> -----Original Message----- >> From: Jerin Jacob <jerinjacobk@gmail.com> >> Subject: Re: [dpdk-dev] [PATCH] add flow shared action API >> >> On Mon, Jul 6, 2020 at 7:02 PM Andrey Vesnovaty >> <andrey.vesnovaty@gmail.com> wrote: >>> >>> Hi, Jerin. >> >> Hi Ori and Andrey, >> >> >>> >>> Please see below Ori's suggestion below to implement your >> rte_flow_action_update() idea >>> with some API changes of rte_flow_shared_action_xxx API changes. >>> >>> On Mon, Jul 6, 2020 at 3:28 PM Ori Kam <orika@mellanox.com> wrote: >>>> >>>> Hi Jerin, >>>> >>>>> -----Original Message----- >>>>> From: Jerin Jacob <jerinjacobk@gmail.com> >>>>> Sent: Monday, July 6, 2020 12:00 PM >>>>> Subject: Re: [dpdk-dev] [PATCH] add flow shared action API >>>>> >>>>> On Sun, Jul 5, 2020 at 3:56 PM Ori Kam <orika@mellanox.com> wrote: >>>>>> >>>>>> Hi Jerin, >>>>>> PSB, >>>>>> >>>>>> Thanks, >>>>>> Ori >>>>>> >>>>>>> -----Original Message----- >>>>>>> From: Jerin Jacob <jerinjacobk@gmail.com> >>>>>>> Sent: Saturday, July 4, 2020 3:33 PM >>>>>>> dpdk-dev <dev@dpdk.org> >>>>>>> Subject: Re: [dpdk-dev] [PATCH] add flow shared action API >>>>>>> >>>>>>> On Sat, Jul 4, 2020 at 3:40 PM Andrey Vesnovaty >>>>>>> <andrey.vesnovaty@gmail.com> wrote: >>>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Andrey Vesnovaty >>>>>>>> (+972)526775512 | Skype: andrey775512 >>>>>>>> >>>>>> >>>>>> >>>>>> [..Nip ..] >>>>>> >>>>>>>> I need to mention the locking issue once again. >>>>>>>> If there is a need to maintain "shared session" in the generic >> rte_flow >>>>> layer >>>>>>> all >>>>>>>> calls to flow_create() with shared action & all delete need to take >>>>>>> sharedsession >>>>>>>> management locks at least for verification. Lock partitioning is also >> bit >>>>>>> problematic >>>>>>>> since one flow may have more than one shared action. >>>>>>> >>>>>>> Then, I think better approach would be to introduce >>>>>>> rte_flow_action_update() public >>>>>>> API which can either take "const struct rte_flow_action []" OR shared >>>>>>> context ID, to cater to >>>>>>> both cases or something on similar lines. This would allow HW's >>>>>>> without have the shared context ID >>>>>>> to use the action update. >>>>>> >>>>>> Can you please explain your idea? >>>>> >>>>> I see two types of HW schemes supporting action updates without going >>>>> through call `rte_flow_destroy()` and call `rte_flow_create()` >>>>> - The shared HW action context feature >>>>> - The HW has "pattern" and "action" mapped to different HW objects and >>>>> action can be updated any time. >>>>> Other than above-mentioned RSS use case, another use case would be to >>>>> a) create rte_flow and set the action as DROP (Kind of reserving the HW >> object) >>>>> b) Update the action only when the rest of the requirements ready. >>>>> >>>>> Any API schematic that supports both notions of HW is fine with me. >>>>> >>>> I have an idea if the API will be changed to something like this, >>>> Rte_flow_shared_action_update(uint16_port port, rte_shared_ctx *ctx, >> rte_flow_action *action, error) >>>> This will enable the application to send a different action than the original >> one to be switched. >>>> Assuming the PMD supports this. >>>> Does it answer your concerns? >>> >>> >>> This allows both: >>> 1. Update action configuration >>> 2. Replace action by some other action >>> For 2 pure software implementation may carate shred action (that can be >> shared >>> with one flow only, depends on PMD) and later on >> rte_flow_shared_action_update may replace this >>> action with some other action by handle returned from >> rte_flow_shared_action_create >>> Doesign between 1 and 2 is per PMD. >> >> struct rte_flow * object holds the driver representation of the >> pattern + action. >> So in order to update the action, we would need struct rte_flow * in API. >> > Why is that? The idea is to change the action, the action itself is connected to flows. > The PMD can save in the shared_ctx all flows that are connected to this action. > >> I think, simple API change would be to accommodate "rte_shared_ctx >> *ctx, rte_flow_action *action" modes >> without introducing the emulation for one or other mode, will be. >> >> enum rte_flow_action_update_type { >> RTE_FLOW_ACTION_UPDATE_TYPE_SHARED_ACTION, >> RTE_FLOW_ACTION_UPDATE_TYPE_ACTION, >> }; >> >> struct rte_flow_action_update_type_param { >> enum rte_flow_action_update_type type; >> union { >> struct rte_flow_action_update_type_shared_action_param { >> rte_shared_ctx *ctx; >> } shared_action; >> struct rte_flow_action_update_type_shared_action_param { >> rte_flow *flow, >> rte_flow_action *action; >> } action; >> } >> } >> > Thank you for the idea but I fall to see how your suggested API is simpler than the one suggested by me? > In my suggestion the PMD simply needs to check if the new action and change the > context and to that action, or just change parameters in the action, if it is the same action. > > Let's go with the original patch API modified to support like you requested also changing the action, > based on my comments. > >> rte_flow_action_update(uint16_port port, struct >> rte_flow_action_update_type_param *param, error) >> >>> >>>> >>>>> >>>>>> As I can see if we use the flow_action array it may result in bugs. >>>>>> For example, the application created two flows with the same RSS (not >> using >>>>> the context) >>>>>> Then he wants to change one flow to use different RSS, but the result will >> that >>>>> both flows >>>>>> will be changed. >>>>> >>>>> Sorry. I don't quite follow this. >>>>> >>>> I was trying to show that there must be some context. But I don’t think this is >> relevant to >>>> your current ideas. >>>> >>>>>> Also this will enforce the PMD to keep track on all flows which will have >>>>> memory penalty for >>>>>> some PMDs. Hi Ori, Andrey, This is a set of new APIs and we are very close to the -rc1, so we have only a few days to close the feature to merge them for this release. Also accompanying PMD and testpmd implementation with the proposed API changes looks missing. We can either postpone the patchset to next release to give time for more PMD owners to participate, which can give better API for long term. Or try to to squeeze into this release taking into account that the APIs will be experimental. What do you think, what is you schedule for the feature, do you have room to postpone it? If not, first existing discussions needs to resolved, and it is good to have the PMD and testpmd implementations, do you think can this be done for next few days? Thanks, ferruh ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [dpdk-dev] [PATCH] add flow shared action API 2020-07-07 15:21 ` Ferruh Yigit @ 2020-07-07 17:24 ` Ori Kam 2020-07-07 17:52 ` Ferruh Yigit 0 siblings, 1 reply; 106+ messages in thread From: Ori Kam @ 2020-07-07 17:24 UTC (permalink / raw) To: Ferruh Yigit, Jerin Jacob, Andrey Vesnovaty Cc: Andrey Vesnovaty, Thomas Monjalon, Andrew Rybchenko, dpdk-dev Hi Ferruh, > -----Original Message----- > From: Ferruh Yigit <ferruh.yigit@intel.com> > > On 7/7/2020 7:21 AM, Ori Kam wrote: > > Hi Jerin, > > Thanks you for your quick reply. > > > >> -----Original Message----- > >> From: Jerin Jacob <jerinjacobk@gmail.com> > >> Subject: Re: [dpdk-dev] [PATCH] add flow shared action API > >> > >> On Mon, Jul 6, 2020 at 7:02 PM Andrey Vesnovaty > >> <andrey.vesnovaty@gmail.com> wrote: > >>> > >>> Hi, Jerin. > >> > >> Hi Ori and Andrey, > >> > >> > >>> > >>> Please see below Ori's suggestion below to implement your > >> rte_flow_action_update() idea > >>> with some API changes of rte_flow_shared_action_xxx API changes. > >>> > >>> On Mon, Jul 6, 2020 at 3:28 PM Ori Kam <orika@mellanox.com> wrote: > >>>> > >>>> Hi Jerin, > >>>> > >>>>> -----Original Message----- > >>>>> From: Jerin Jacob <jerinjacobk@gmail.com> > >>>>> Sent: Monday, July 6, 2020 12:00 PM > >>>>> Subject: Re: [dpdk-dev] [PATCH] add flow shared action API > >>>>> > >>>>> On Sun, Jul 5, 2020 at 3:56 PM Ori Kam <orika@mellanox.com> wrote: > >>>>>> > >>>>>> Hi Jerin, > >>>>>> PSB, > >>>>>> > >>>>>> Thanks, > >>>>>> Ori > >>>>>> > >>>>>>> -----Original Message----- > >>>>>>> From: Jerin Jacob <jerinjacobk@gmail.com> > >>>>>>> Sent: Saturday, July 4, 2020 3:33 PM > >>>>>>> dpdk-dev <dev@dpdk.org> > >>>>>>> Subject: Re: [dpdk-dev] [PATCH] add flow shared action API > >>>>>>> > >>>>>>> On Sat, Jul 4, 2020 at 3:40 PM Andrey Vesnovaty > >>>>>>> <andrey.vesnovaty@gmail.com> wrote: > >>>>>>>> > >>>>>>>> > >>>>>>>> Thanks, > >>>>>>>> > >>>>>>>> Andrey Vesnovaty > >>>>>>>> (+972)526775512 | Skype: andrey775512 > >>>>>>>> > >>>>>> > >>>>>> > >>>>>> [..Nip ..] > >>>>>> > >>>>>>>> I need to mention the locking issue once again. > >>>>>>>> If there is a need to maintain "shared session" in the generic > >> rte_flow > >>>>> layer > >>>>>>> all > >>>>>>>> calls to flow_create() with shared action & all delete need to take > >>>>>>> sharedsession > >>>>>>>> management locks at least for verification. Lock partitioning is also > >> bit > >>>>>>> problematic > >>>>>>>> since one flow may have more than one shared action. > >>>>>>> > >>>>>>> Then, I think better approach would be to introduce > >>>>>>> rte_flow_action_update() public > >>>>>>> API which can either take "const struct rte_flow_action []" OR shared > >>>>>>> context ID, to cater to > >>>>>>> both cases or something on similar lines. This would allow HW's > >>>>>>> without have the shared context ID > >>>>>>> to use the action update. > >>>>>> > >>>>>> Can you please explain your idea? > >>>>> > >>>>> I see two types of HW schemes supporting action updates without going > >>>>> through call `rte_flow_destroy()` and call `rte_flow_create()` > >>>>> - The shared HW action context feature > >>>>> - The HW has "pattern" and "action" mapped to different HW objects > and > >>>>> action can be updated any time. > >>>>> Other than above-mentioned RSS use case, another use case would be to > >>>>> a) create rte_flow and set the action as DROP (Kind of reserving the HW > >> object) > >>>>> b) Update the action only when the rest of the requirements ready. > >>>>> > >>>>> Any API schematic that supports both notions of HW is fine with me. > >>>>> > >>>> I have an idea if the API will be changed to something like this, > >>>> Rte_flow_shared_action_update(uint16_port port, rte_shared_ctx *ctx, > >> rte_flow_action *action, error) > >>>> This will enable the application to send a different action than the original > >> one to be switched. > >>>> Assuming the PMD supports this. > >>>> Does it answer your concerns? > >>> > >>> > >>> This allows both: > >>> 1. Update action configuration > >>> 2. Replace action by some other action > >>> For 2 pure software implementation may carate shred action (that can be > >> shared > >>> with one flow only, depends on PMD) and later on > >> rte_flow_shared_action_update may replace this > >>> action with some other action by handle returned from > >> rte_flow_shared_action_create > >>> Doesign between 1 and 2 is per PMD. > >> > >> struct rte_flow * object holds the driver representation of the > >> pattern + action. > >> So in order to update the action, we would need struct rte_flow * in API. > >> > > Why is that? The idea is to change the action, the action itself is connected to > flows. > > The PMD can save in the shared_ctx all flows that are connected to this > action. > > > >> I think, simple API change would be to accommodate "rte_shared_ctx > >> *ctx, rte_flow_action *action" modes > >> without introducing the emulation for one or other mode, will be. > >> > >> enum rte_flow_action_update_type { > >> RTE_FLOW_ACTION_UPDATE_TYPE_SHARED_ACTION, > >> RTE_FLOW_ACTION_UPDATE_TYPE_ACTION, > >> }; > >> > >> struct rte_flow_action_update_type_param { > >> enum rte_flow_action_update_type type; > >> union { > >> struct rte_flow_action_update_type_shared_action_param { > >> rte_shared_ctx *ctx; > >> } shared_action; > >> struct rte_flow_action_update_type_shared_action_param { > >> rte_flow *flow, > >> rte_flow_action *action; > >> } action; > >> } > >> } > >> > > Thank you for the idea but I fall to see how your suggested API is simpler than > the one suggested by me? > > In my suggestion the PMD simply needs to check if the new action and > change the > > context and to that action, or just change parameters in the action, if it is the > same action. > > > > Let's go with the original patch API modified to support like you requested > also changing the action, > > based on my comments. > > > >> rte_flow_action_update(uint16_port port, struct > >> rte_flow_action_update_type_param *param, error) > >> > >>> > >>>> > >>>>> > >>>>>> As I can see if we use the flow_action array it may result in bugs. > >>>>>> For example, the application created two flows with the same RSS (not > >> using > >>>>> the context) > >>>>>> Then he wants to change one flow to use different RSS, but the result > will > >> that > >>>>> both flows > >>>>>> will be changed. > >>>>> > >>>>> Sorry. I don't quite follow this. > >>>>> > >>>> I was trying to show that there must be some context. But I don’t think this > is > >> relevant to > >>>> your current ideas. > >>>> > >>>>>> Also this will enforce the PMD to keep track on all flows which will have > >>>>> memory penalty for > >>>>>> some PMDs. > > Hi Ori, Andrey, > > This is a set of new APIs and we are very close to the -rc1, so we have only a > few days to close the feature to merge them for this release. > > Also accompanying PMD and testpmd implementation with the proposed API > changes > looks missing. > > We can either postpone the patchset to next release to give time for more > PMD > owners to participate, which can give better API for long term. > Or try to to squeeze into this release taking into account that the APIs will be > experimental. > > What do you think, what is you schedule for the feature, do you have room to > postpone it? Not so much it is an important API for Mellanox. > If not, first existing discussions needs to resolved, and it is good to have the > PMD and testpmd implementations, do you think can this be done for next few > days? > I think that this is the correct API to implement, I fully agree that this API is experimental just like any other new API, and might change based on comments and use cases. I know that Mellanox is committed to this feature and that Andrey is working around the clock to complete the missing parts, and should have a version by tomorrow (July 8th ) evening. (with update to flow filtering sample app, testpmd will not be ready by RC1, but it will be for RC2) We would like very much to push it in this version. Thanks, Ori > > Thanks, > ferruh ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [dpdk-dev] [PATCH] add flow shared action API 2020-07-07 17:24 ` Ori Kam @ 2020-07-07 17:52 ` Ferruh Yigit 0 siblings, 0 replies; 106+ messages in thread From: Ferruh Yigit @ 2020-07-07 17:52 UTC (permalink / raw) To: Ori Kam, Jerin Jacob, Andrey Vesnovaty Cc: Andrey Vesnovaty, Thomas Monjalon, Andrew Rybchenko, dpdk-dev On 7/7/2020 6:24 PM, Ori Kam wrote: > Hi Ferruh, > >> -----Original Message----- >> From: Ferruh Yigit <ferruh.yigit@intel.com> >> >> On 7/7/2020 7:21 AM, Ori Kam wrote: >>> Hi Jerin, >>> Thanks you for your quick reply. >>> >>>> -----Original Message----- >>>> From: Jerin Jacob <jerinjacobk@gmail.com> >>>> Subject: Re: [dpdk-dev] [PATCH] add flow shared action API >>>> >>>> On Mon, Jul 6, 2020 at 7:02 PM Andrey Vesnovaty >>>> <andrey.vesnovaty@gmail.com> wrote: >>>>> >>>>> Hi, Jerin. >>>> >>>> Hi Ori and Andrey, >>>> >>>> >>>>> >>>>> Please see below Ori's suggestion below to implement your >>>> rte_flow_action_update() idea >>>>> with some API changes of rte_flow_shared_action_xxx API changes. >>>>> >>>>> On Mon, Jul 6, 2020 at 3:28 PM Ori Kam <orika@mellanox.com> wrote: >>>>>> >>>>>> Hi Jerin, >>>>>> >>>>>>> -----Original Message----- >>>>>>> From: Jerin Jacob <jerinjacobk@gmail.com> >>>>>>> Sent: Monday, July 6, 2020 12:00 PM >>>>>>> Subject: Re: [dpdk-dev] [PATCH] add flow shared action API >>>>>>> >>>>>>> On Sun, Jul 5, 2020 at 3:56 PM Ori Kam <orika@mellanox.com> wrote: >>>>>>>> >>>>>>>> Hi Jerin, >>>>>>>> PSB, >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Ori >>>>>>>> >>>>>>>>> -----Original Message----- >>>>>>>>> From: Jerin Jacob <jerinjacobk@gmail.com> >>>>>>>>> Sent: Saturday, July 4, 2020 3:33 PM >>>>>>>>> dpdk-dev <dev@dpdk.org> >>>>>>>>> Subject: Re: [dpdk-dev] [PATCH] add flow shared action API >>>>>>>>> >>>>>>>>> On Sat, Jul 4, 2020 at 3:40 PM Andrey Vesnovaty >>>>>>>>> <andrey.vesnovaty@gmail.com> wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> >>>>>>>>>> Andrey Vesnovaty >>>>>>>>>> (+972)526775512 | Skype: andrey775512 >>>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> [..Nip ..] >>>>>>>> >>>>>>>>>> I need to mention the locking issue once again. >>>>>>>>>> If there is a need to maintain "shared session" in the generic >>>> rte_flow >>>>>>> layer >>>>>>>>> all >>>>>>>>>> calls to flow_create() with shared action & all delete need to take >>>>>>>>> sharedsession >>>>>>>>>> management locks at least for verification. Lock partitioning is also >>>> bit >>>>>>>>> problematic >>>>>>>>>> since one flow may have more than one shared action. >>>>>>>>> >>>>>>>>> Then, I think better approach would be to introduce >>>>>>>>> rte_flow_action_update() public >>>>>>>>> API which can either take "const struct rte_flow_action []" OR shared >>>>>>>>> context ID, to cater to >>>>>>>>> both cases or something on similar lines. This would allow HW's >>>>>>>>> without have the shared context ID >>>>>>>>> to use the action update. >>>>>>>> >>>>>>>> Can you please explain your idea? >>>>>>> >>>>>>> I see two types of HW schemes supporting action updates without going >>>>>>> through call `rte_flow_destroy()` and call `rte_flow_create()` >>>>>>> - The shared HW action context feature >>>>>>> - The HW has "pattern" and "action" mapped to different HW objects >> and >>>>>>> action can be updated any time. >>>>>>> Other than above-mentioned RSS use case, another use case would be to >>>>>>> a) create rte_flow and set the action as DROP (Kind of reserving the HW >>>> object) >>>>>>> b) Update the action only when the rest of the requirements ready. >>>>>>> >>>>>>> Any API schematic that supports both notions of HW is fine with me. >>>>>>> >>>>>> I have an idea if the API will be changed to something like this, >>>>>> Rte_flow_shared_action_update(uint16_port port, rte_shared_ctx *ctx, >>>> rte_flow_action *action, error) >>>>>> This will enable the application to send a different action than the original >>>> one to be switched. >>>>>> Assuming the PMD supports this. >>>>>> Does it answer your concerns? >>>>> >>>>> >>>>> This allows both: >>>>> 1. Update action configuration >>>>> 2. Replace action by some other action >>>>> For 2 pure software implementation may carate shred action (that can be >>>> shared >>>>> with one flow only, depends on PMD) and later on >>>> rte_flow_shared_action_update may replace this >>>>> action with some other action by handle returned from >>>> rte_flow_shared_action_create >>>>> Doesign between 1 and 2 is per PMD. >>>> >>>> struct rte_flow * object holds the driver representation of the >>>> pattern + action. >>>> So in order to update the action, we would need struct rte_flow * in API. >>>> >>> Why is that? The idea is to change the action, the action itself is connected to >> flows. >>> The PMD can save in the shared_ctx all flows that are connected to this >> action. >>> >>>> I think, simple API change would be to accommodate "rte_shared_ctx >>>> *ctx, rte_flow_action *action" modes >>>> without introducing the emulation for one or other mode, will be. >>>> >>>> enum rte_flow_action_update_type { >>>> RTE_FLOW_ACTION_UPDATE_TYPE_SHARED_ACTION, >>>> RTE_FLOW_ACTION_UPDATE_TYPE_ACTION, >>>> }; >>>> >>>> struct rte_flow_action_update_type_param { >>>> enum rte_flow_action_update_type type; >>>> union { >>>> struct rte_flow_action_update_type_shared_action_param { >>>> rte_shared_ctx *ctx; >>>> } shared_action; >>>> struct rte_flow_action_update_type_shared_action_param { >>>> rte_flow *flow, >>>> rte_flow_action *action; >>>> } action; >>>> } >>>> } >>>> >>> Thank you for the idea but I fall to see how your suggested API is simpler than >> the one suggested by me? >>> In my suggestion the PMD simply needs to check if the new action and >> change the >>> context and to that action, or just change parameters in the action, if it is the >> same action. >>> >>> Let's go with the original patch API modified to support like you requested >> also changing the action, >>> based on my comments. >>> >>>> rte_flow_action_update(uint16_port port, struct >>>> rte_flow_action_update_type_param *param, error) >>>> >>>>> >>>>>> >>>>>>> >>>>>>>> As I can see if we use the flow_action array it may result in bugs. >>>>>>>> For example, the application created two flows with the same RSS (not >>>> using >>>>>>> the context) >>>>>>>> Then he wants to change one flow to use different RSS, but the result >> will >>>> that >>>>>>> both flows >>>>>>>> will be changed. >>>>>>> >>>>>>> Sorry. I don't quite follow this. >>>>>>> >>>>>> I was trying to show that there must be some context. But I don’t think this >> is >>>> relevant to >>>>>> your current ideas. >>>>>> >>>>>>>> Also this will enforce the PMD to keep track on all flows which will have >>>>>>> memory penalty for >>>>>>>> some PMDs. >> >> Hi Ori, Andrey, >> >> This is a set of new APIs and we are very close to the -rc1, so we have only a >> few days to close the feature to merge them for this release. >> >> Also accompanying PMD and testpmd implementation with the proposed API >> changes >> looks missing. >> >> We can either postpone the patchset to next release to give time for more >> PMD >> owners to participate, which can give better API for long term. >> Or try to to squeeze into this release taking into account that the APIs will be >> experimental. >> >> What do you think, what is you schedule for the feature, do you have room to >> postpone it? > Not so much it is an important API for Mellanox. Got it. > >> If not, first existing discussions needs to resolved, and it is good to have the >> PMD and testpmd implementations, do you think can this be done for next few >> days? >> > I think that this is the correct API to implement, I fully agree that this API is experimental > just like any other new API, and might change based on comments and use cases. > I know that Mellanox is committed to this feature and that Andrey is working around the clock > to complete the missing parts, and should have a version by tomorrow (July 8th ) evening. OK > (with update to flow filtering sample app, testpmd will not be ready by RC1, but it will be for RC2) > We would like very much to push it in this version. OK, please conclude the existing discussion before finalizing. ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [dpdk-dev] [PATCH] add flow shared action API 2020-07-07 6:21 ` Ori Kam 2020-07-07 15:21 ` Ferruh Yigit @ 2020-07-07 19:38 ` Jerin Jacob 2020-07-07 21:03 ` Ori Kam 1 sibling, 1 reply; 106+ messages in thread From: Jerin Jacob @ 2020-07-07 19:38 UTC (permalink / raw) To: Ori Kam Cc: Andrey Vesnovaty, Andrey Vesnovaty, Thomas Monjalon, Ferruh Yigit, Andrew Rybchenko, dpdk-dev On Tue, Jul 7, 2020 at 11:51 AM Ori Kam <orika@mellanox.com> wrote: > > Hi Jerin, > Thanks you for your quick reply. > > > -----Original Message----- > > From: Jerin Jacob <jerinjacobk@gmail.com> > > Subject: Re: [dpdk-dev] [PATCH] add flow shared action API > > > > On Mon, Jul 6, 2020 at 7:02 PM Andrey Vesnovaty > > <andrey.vesnovaty@gmail.com> wrote: > > > > > > Hi, Jerin. > > > > Hi Ori and Andrey, > > > > > > > > > > Please see below Ori's suggestion below to implement your > > rte_flow_action_update() idea > > > with some API changes of rte_flow_shared_action_xxx API changes. > > > > > > On Mon, Jul 6, 2020 at 3:28 PM Ori Kam <orika@mellanox.com> wrote: > > >> > > >> Hi Jerin, > > >> > > >> > -----Original Message----- > > >> > From: Jerin Jacob <jerinjacobk@gmail.com> > > >> > Sent: Monday, July 6, 2020 12:00 PM > > >> > Subject: Re: [dpdk-dev] [PATCH] add flow shared action API > > >> > > > >> > On Sun, Jul 5, 2020 at 3:56 PM Ori Kam <orika@mellanox.com> wrote: > > >> > > > > >> > > Hi Jerin, > > >> > > PSB, > > >> > > > > >> > > Thanks, > > >> > > Ori > > >> > > > > >> > > > -----Original Message----- > > >> > > > From: Jerin Jacob <jerinjacobk@gmail.com> > > >> > > > Sent: Saturday, July 4, 2020 3:33 PM > > >> > > > dpdk-dev <dev@dpdk.org> > > >> > > > Subject: Re: [dpdk-dev] [PATCH] add flow shared action API > > >> > > > > > >> > > > On Sat, Jul 4, 2020 at 3:40 PM Andrey Vesnovaty > > >> > > > <andrey.vesnovaty@gmail.com> wrote: > > >> > > > > > > >> > > > > > > >> > > > > Thanks, > > >> > > > > > > >> > > > > Andrey Vesnovaty > > >> > > > > (+972)526775512 | Skype: andrey775512 > > >> > > > > > > >> > > > > >> > > > > >> > > [..Nip ..] > > >> > > > > >> > > > > I need to mention the locking issue once again. > > >> > > > > If there is a need to maintain "shared session" in the generic > > rte_flow > > >> > layer > > >> > > > all > > >> > > > > calls to flow_create() with shared action & all delete need to take > > >> > > > sharedsession > > >> > > > > management locks at least for verification. Lock partitioning is also > > bit > > >> > > > problematic > > >> > > > > since one flow may have more than one shared action. > > >> > > > > > >> > > > Then, I think better approach would be to introduce > > >> > > > rte_flow_action_update() public > > >> > > > API which can either take "const struct rte_flow_action []" OR shared > > >> > > > context ID, to cater to > > >> > > > both cases or something on similar lines. This would allow HW's > > >> > > > without have the shared context ID > > >> > > > to use the action update. > > >> > > > > >> > > Can you please explain your idea? > > >> > > > >> > I see two types of HW schemes supporting action updates without going > > >> > through call `rte_flow_destroy()` and call `rte_flow_create()` > > >> > - The shared HW action context feature > > >> > - The HW has "pattern" and "action" mapped to different HW objects and > > >> > action can be updated any time. > > >> > Other than above-mentioned RSS use case, another use case would be to > > >> > a) create rte_flow and set the action as DROP (Kind of reserving the HW > > object) > > >> > b) Update the action only when the rest of the requirements ready. > > >> > > > >> > Any API schematic that supports both notions of HW is fine with me. > > >> > > > >> I have an idea if the API will be changed to something like this, > > >> Rte_flow_shared_action_update(uint16_port port, rte_shared_ctx *ctx, > > rte_flow_action *action, error) > > >> This will enable the application to send a different action than the original > > one to be switched. > > >> Assuming the PMD supports this. > > >> Does it answer your concerns? > > > > > > > > > This allows both: > > > 1. Update action configuration > > > 2. Replace action by some other action > > > For 2 pure software implementation may carate shred action (that can be > > shared > > > with one flow only, depends on PMD) and later on > > rte_flow_shared_action_update may replace this > > > action with some other action by handle returned from > > rte_flow_shared_action_create > > > Doesign between 1 and 2 is per PMD. > > > > struct rte_flow * object holds the driver representation of the > > pattern + action. > > So in order to update the action, we would need struct rte_flow * in API. > > > Why is that? The idea is to change the action, the action itself is connected to flows. > The PMD can save in the shared_ctx all flows that are connected to this action. > > > I think, simple API change would be to accommodate "rte_shared_ctx > > *ctx, rte_flow_action *action" modes > > without introducing the emulation for one or other mode, will be. > > > > enum rte_flow_action_update_type { > > RTE_FLOW_ACTION_UPDATE_TYPE_SHARED_ACTION, > > RTE_FLOW_ACTION_UPDATE_TYPE_ACTION, > > }; > > > > struct rte_flow_action_update_type_param { > > enum rte_flow_action_update_type type; > > union { > > struct rte_flow_action_update_type_shared_action_param { > > rte_shared_ctx *ctx; > > } shared_action; > > struct rte_flow_action_update_type_shared_action_param { > > rte_flow *flow, > > rte_flow_action *action; > > } action; > > } > > } > > > Thank you for the idea but I fall to see how your suggested API is simpler than the one suggested by me? My thought process with the below-proposed API[1] is that It is dictates "_shared_action_" in API name as well as arguments. I just thought of expressing it as either-or case hence I thought [2] is better. i.e The PMD does not support shared_action, not even need to create one to use rte_flow_action_update() to avoid the confusion. Thoughts? [1] rte_flow_shared_action_update(uint16_port port, rte_shared_ctx *ctx, rte_flow_action *action, error) [2] rte_flow_action_update(uint16_port port, struct rte_flow_action_update_type_param *param, error) > In my suggestion the PMD simply needs to check if the new action and change the > context and to that action, or just change parameters in the action, if it is the same action. > > Let's go with the original patch API modified to support like you requested also changing the action, > based on my comments. > > > rte_flow_action_update(uint16_port port, struct > > rte_flow_action_update_type_param *param, error) > > > > > > > >> > > >> > > > >> > > As I can see if we use the flow_action array it may result in bugs. > > >> > > For example, the application created two flows with the same RSS (not > > using > > >> > the context) > > >> > > Then he wants to change one flow to use different RSS, but the result will > > that > > >> > both flows > > >> > > will be changed. > > >> > > > >> > Sorry. I don't quite follow this. > > >> > > > >> I was trying to show that there must be some context. But I don’t think this is > > relevant to > > >> your current ideas. > > >> > > >> > > Also this will enforce the PMD to keep track on all flows which will have > > >> > memory penalty for > > >> > > some PMDs. > > >> > > >> Best, > > >> Ori > > > > > > > > > Thanks, > > > Andrey > Best, > Ori ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [dpdk-dev] [PATCH] add flow shared action API 2020-07-07 19:38 ` Jerin Jacob @ 2020-07-07 21:03 ` Ori Kam 2020-07-08 9:25 ` Jerin Jacob 0 siblings, 1 reply; 106+ messages in thread From: Ori Kam @ 2020-07-07 21:03 UTC (permalink / raw) To: Jerin Jacob Cc: Andrey Vesnovaty, Andrey Vesnovaty, Thomas Monjalon, Ferruh Yigit, Andrew Rybchenko, dpdk-dev Hi Jerin, > -----Original Message----- > From: Jerin Jacob <jerinjacobk@gmail.com> > > On Tue, Jul 7, 2020 at 11:51 AM Ori Kam <orika@mellanox.com> wrote: > > > > Hi Jerin, > > Thanks you for your quick reply. > > > > > -----Original Message----- > > > From: Jerin Jacob <jerinjacobk@gmail.com> > > > Subject: Re: [dpdk-dev] [PATCH] add flow shared action API > > > > > > On Mon, Jul 6, 2020 at 7:02 PM Andrey Vesnovaty > > > <andrey.vesnovaty@gmail.com> wrote: > > > > > > > > Hi, Jerin. > > > > > > Hi Ori and Andrey, > > > > > > > > > > > > > > Please see below Ori's suggestion below to implement your > > > rte_flow_action_update() idea > > > > with some API changes of rte_flow_shared_action_xxx API changes. > > > > > > > > On Mon, Jul 6, 2020 at 3:28 PM Ori Kam <orika@mellanox.com> wrote: > > > >> > > > >> Hi Jerin, > > > >> > > > >> > -----Original Message----- > > > >> > From: Jerin Jacob <jerinjacobk@gmail.com> > > > >> > Sent: Monday, July 6, 2020 12:00 PM > > > >> > Subject: Re: [dpdk-dev] [PATCH] add flow shared action API > > > >> > > > > >> > On Sun, Jul 5, 2020 at 3:56 PM Ori Kam <orika@mellanox.com> wrote: > > > >> > > > > > >> > > Hi Jerin, > > > >> > > PSB, > > > >> > > > > > >> > > Thanks, > > > >> > > Ori > > > >> > > > > > >> > > > -----Original Message----- > > > >> > > > From: Jerin Jacob <jerinjacobk@gmail.com> > > > >> > > > Sent: Saturday, July 4, 2020 3:33 PM > > > >> > > > dpdk-dev <dev@dpdk.org> > > > >> > > > Subject: Re: [dpdk-dev] [PATCH] add flow shared action API > > > >> > > > > > > >> > > > On Sat, Jul 4, 2020 at 3:40 PM Andrey Vesnovaty > > > >> > > > <andrey.vesnovaty@gmail.com> wrote: > > > >> > > > > > > > >> > > > > > > > >> > > > > Thanks, > > > >> > > > > > > > >> > > > > Andrey Vesnovaty > > > >> > > > > (+972)526775512 | Skype: andrey775512 > > > >> > > > > > > > >> > > > > > >> > > > > > >> > > [..Nip ..] > > > >> > > > > > >> > > > > I need to mention the locking issue once again. > > > >> > > > > If there is a need to maintain "shared session" in the generic > > > rte_flow > > > >> > layer > > > >> > > > all > > > >> > > > > calls to flow_create() with shared action & all delete need to take > > > >> > > > sharedsession > > > >> > > > > management locks at least for verification. Lock partitioning is > also > > > bit > > > >> > > > problematic > > > >> > > > > since one flow may have more than one shared action. > > > >> > > > > > > >> > > > Then, I think better approach would be to introduce > > > >> > > > rte_flow_action_update() public > > > >> > > > API which can either take "const struct rte_flow_action []" OR > shared > > > >> > > > context ID, to cater to > > > >> > > > both cases or something on similar lines. This would allow HW's > > > >> > > > without have the shared context ID > > > >> > > > to use the action update. > > > >> > > > > > >> > > Can you please explain your idea? > > > >> > > > > >> > I see two types of HW schemes supporting action updates without > going > > > >> > through call `rte_flow_destroy()` and call `rte_flow_create()` > > > >> > - The shared HW action context feature > > > >> > - The HW has "pattern" and "action" mapped to different HW objects > and > > > >> > action can be updated any time. > > > >> > Other than above-mentioned RSS use case, another use case would be > to > > > >> > a) create rte_flow and set the action as DROP (Kind of reserving the > HW > > > object) > > > >> > b) Update the action only when the rest of the requirements ready. > > > >> > > > > >> > Any API schematic that supports both notions of HW is fine with me. > > > >> > > > > >> I have an idea if the API will be changed to something like this, > > > >> Rte_flow_shared_action_update(uint16_port port, rte_shared_ctx *ctx, > > > rte_flow_action *action, error) > > > >> This will enable the application to send a different action than the > original > > > one to be switched. > > > >> Assuming the PMD supports this. > > > >> Does it answer your concerns? > > > > > > > > > > > > This allows both: > > > > 1. Update action configuration > > > > 2. Replace action by some other action > > > > For 2 pure software implementation may carate shred action (that can be > > > shared > > > > with one flow only, depends on PMD) and later on > > > rte_flow_shared_action_update may replace this > > > > action with some other action by handle returned from > > > rte_flow_shared_action_create > > > > Doesign between 1 and 2 is per PMD. > > > > > > struct rte_flow * object holds the driver representation of the > > > pattern + action. > > > So in order to update the action, we would need struct rte_flow * in API. > > > > > Why is that? The idea is to change the action, the action itself is connected to > flows. > > The PMD can save in the shared_ctx all flows that are connected to this > action. > > > > > I think, simple API change would be to accommodate "rte_shared_ctx > > > *ctx, rte_flow_action *action" modes > > > without introducing the emulation for one or other mode, will be. > > > > > > enum rte_flow_action_update_type { > > > RTE_FLOW_ACTION_UPDATE_TYPE_SHARED_ACTION, > > > RTE_FLOW_ACTION_UPDATE_TYPE_ACTION, > > > }; > > > > > > struct rte_flow_action_update_type_param { > > > enum rte_flow_action_update_type type; > > > union { > > > struct rte_flow_action_update_type_shared_action_param { > > > rte_shared_ctx *ctx; > > > } shared_action; > > > struct rte_flow_action_update_type_shared_action_param { > > > rte_flow *flow, > > > rte_flow_action *action; > > > } action; > > > } > > > } > > > > > Thank you for the idea but I fall to see how your suggested API is simpler than > the one suggested by me? > > My thought process with the below-proposed API[1] is that It is > dictates "_shared_action_" in API name as > well as arguments. I just thought of expressing it as either-or case > hence I thought [2] is better. i.e The PMD does not support > shared_action, not even need to create one to use > rte_flow_action_update() to avoid the confusion. Thoughts? > > [1] > rte_flow_shared_action_update(uint16_port port, rte_shared_ctx *ctx, > rte_flow_action *action, error) > > [2] > rte_flow_action_update(uint16_port port, struct > rte_flow_action_update_type_param *param, error) > Let me see if I understand you correctly, your suggestion is to allow the application to change one action in one flow, but instead of creating the context the application will just supply the rte_flow and the new actions do I understand correctly? If so this it is a nice idea, but there are some issues with it, 1. The PMD must save the flow which will result in memory consumption. 2. Assume that two flows are using the same RSS action for example, so the PMD reuse the RSS object he created for the first flow also for the second. Now changing this RSS flow may result in also changing the second flow. (this can be solved by always creating new action) 3. It doesn't handle the main use case that the application wants to change number of flows at the same time, which is the idea of this feature. I also think that all PMD that support option 2 can support option 1 since they can save in the ctx a list of flows and simply apply them again. So by definition if PMD supports [2] it also support [1] while the other way is not correct since it forces the PMD to save flows which like I said waste memory. I suggest that we will go with option [1], and if needed in the future we will update the code. using option [2] will result in dead code since at least for the current time no PMD will implement this API. I can suggest one more thing maybe to change the name from shared_ctx to just ctx which implicitly mean it can be shared but not a must. What do you think? (but again I think by definition if a PMD can implement number [2] it can also implement it to number of flows using API [2]. > > In my suggestion the PMD simply needs to check if the new action and > change the > > context and to that action, or just change parameters in the action, if it is the > same action. > > > > Let's go with the original patch API modified to support like you requested > also changing the action, > > based on my comments. > > > > > rte_flow_action_update(uint16_port port, struct > > > rte_flow_action_update_type_param *param, error) > > > > > > > [..nip..] Best, Ori ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [dpdk-dev] [PATCH] add flow shared action API 2020-07-07 21:03 ` Ori Kam @ 2020-07-08 9:25 ` Jerin Jacob 2020-07-08 9:47 ` Ori Kam 0 siblings, 1 reply; 106+ messages in thread From: Jerin Jacob @ 2020-07-08 9:25 UTC (permalink / raw) To: Ori Kam Cc: Andrey Vesnovaty, Andrey Vesnovaty, Thomas Monjalon, Ferruh Yigit, Andrew Rybchenko, dpdk-dev On Wed, Jul 8, 2020 at 2:33 AM Ori Kam <orika@mellanox.com> wrote: > > Hi Jerin, Hi Ori, > > > -----Original Message----- > > From: Jerin Jacob <jerinjacobk@gmail.com> > > > > On Tue, Jul 7, 2020 at 11:51 AM Ori Kam <orika@mellanox.com> wrote: > > > > > > Hi Jerin, > > > Thanks you for your quick reply. > > > > > > > -----Original Message----- > > > > From: Jerin Jacob <jerinjacobk@gmail.com> > > > > Subject: Re: [dpdk-dev] [PATCH] add flow shared action API > > > > > > > > On Mon, Jul 6, 2020 at 7:02 PM Andrey Vesnovaty > > > > <andrey.vesnovaty@gmail.com> wrote: > > > > > > > > > > Hi, Jerin. > > > > > > > > Hi Ori and Andrey, > > > > > > > > > > > > > > > > > > Please see below Ori's suggestion below to implement your > > > > rte_flow_action_update() idea > > > > > with some API changes of rte_flow_shared_action_xxx API changes. > > > > > > > > > > On Mon, Jul 6, 2020 at 3:28 PM Ori Kam <orika@mellanox.com> wrote: > > > > >> > > > > >> Hi Jerin, > > > > >> > > > > >> > -----Original Message----- > > > > >> > From: Jerin Jacob <jerinjacobk@gmail.com> > > > > >> > Sent: Monday, July 6, 2020 12:00 PM > > > > >> > Subject: Re: [dpdk-dev] [PATCH] add flow shared action API > > > > >> > > > > > >> > On Sun, Jul 5, 2020 at 3:56 PM Ori Kam <orika@mellanox.com> wrote: > > > > >> > > > > > > >> > > Hi Jerin, > > > > >> > > PSB, > > > > >> > > > > > > >> > > Thanks, > > > > >> > > Ori > > > > >> > > > > > > >> > > > -----Original Message----- > > > > >> > > > From: Jerin Jacob <jerinjacobk@gmail.com> > > > > >> > > > Sent: Saturday, July 4, 2020 3:33 PM > > > > >> > > > dpdk-dev <dev@dpdk.org> > > > > >> > > > Subject: Re: [dpdk-dev] [PATCH] add flow shared action API > > > > >> > > > > > > > >> > > > On Sat, Jul 4, 2020 at 3:40 PM Andrey Vesnovaty > > > > >> > > > <andrey.vesnovaty@gmail.com> wrote: > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > Thanks, > > > > >> > > > > > > > > >> > > > > Andrey Vesnovaty > > > > >> > > > > (+972)526775512 | Skype: andrey775512 > > > > >> > > > > > > > > >> > > > > > > >> > > > > > > >> > > [..Nip ..] > > > > >> > > > > > > >> > > > > I need to mention the locking issue once again. > > > > >> > > > > If there is a need to maintain "shared session" in the generic > > > > rte_flow > > > > >> > layer > > > > >> > > > all > > > > >> > > > > calls to flow_create() with shared action & all delete need to take > > > > >> > > > sharedsession > > > > >> > > > > management locks at least for verification. Lock partitioning is > > also > > > > bit > > > > >> > > > problematic > > > > >> > > > > since one flow may have more than one shared action. > > > > >> > > > > > > > >> > > > Then, I think better approach would be to introduce > > > > >> > > > rte_flow_action_update() public > > > > >> > > > API which can either take "const struct rte_flow_action []" OR > > shared > > > > >> > > > context ID, to cater to > > > > >> > > > both cases or something on similar lines. This would allow HW's > > > > >> > > > without have the shared context ID > > > > >> > > > to use the action update. > > > > >> > > > > > > >> > > Can you please explain your idea? > > > > >> > > > > > >> > I see two types of HW schemes supporting action updates without > > going > > > > >> > through call `rte_flow_destroy()` and call `rte_flow_create()` > > > > >> > - The shared HW action context feature > > > > >> > - The HW has "pattern" and "action" mapped to different HW objects > > and > > > > >> > action can be updated any time. > > > > >> > Other than above-mentioned RSS use case, another use case would be > > to > > > > >> > a) create rte_flow and set the action as DROP (Kind of reserving the > > HW > > > > object) > > > > >> > b) Update the action only when the rest of the requirements ready. > > > > >> > > > > > >> > Any API schematic that supports both notions of HW is fine with me. > > > > >> > > > > > >> I have an idea if the API will be changed to something like this, > > > > >> Rte_flow_shared_action_update(uint16_port port, rte_shared_ctx *ctx, > > > > rte_flow_action *action, error) > > > > >> This will enable the application to send a different action than the > > original > > > > one to be switched. > > > > >> Assuming the PMD supports this. > > > > >> Does it answer your concerns? > > > > > > > > > > > > > > > This allows both: > > > > > 1. Update action configuration > > > > > 2. Replace action by some other action > > > > > For 2 pure software implementation may carate shred action (that can be > > > > shared > > > > > with one flow only, depends on PMD) and later on > > > > rte_flow_shared_action_update may replace this > > > > > action with some other action by handle returned from > > > > rte_flow_shared_action_create > > > > > Doesign between 1 and 2 is per PMD. > > > > > > > > struct rte_flow * object holds the driver representation of the > > > > pattern + action. > > > > So in order to update the action, we would need struct rte_flow * in API. > > > > > > > Why is that? The idea is to change the action, the action itself is connected to > > flows. > > > The PMD can save in the shared_ctx all flows that are connected to this > > action. > > > > > > > I think, simple API change would be to accommodate "rte_shared_ctx > > > > *ctx, rte_flow_action *action" modes > > > > without introducing the emulation for one or other mode, will be. > > > > > > > > enum rte_flow_action_update_type { > > > > RTE_FLOW_ACTION_UPDATE_TYPE_SHARED_ACTION, > > > > RTE_FLOW_ACTION_UPDATE_TYPE_ACTION, > > > > }; > > > > > > > > struct rte_flow_action_update_type_param { > > > > enum rte_flow_action_update_type type; > > > > union { > > > > struct rte_flow_action_update_type_shared_action_param { > > > > rte_shared_ctx *ctx; > > > > } shared_action; > > > > struct rte_flow_action_update_type_shared_action_param { > > > > rte_flow *flow, > > > > rte_flow_action *action; > > > > } action; > > > > } > > > > } > > > > > > > Thank you for the idea but I fall to see how your suggested API is simpler than > > the one suggested by me? > > > > My thought process with the below-proposed API[1] is that It is > > dictates "_shared_action_" in API name as > > well as arguments. I just thought of expressing it as either-or case > > hence I thought [2] is better. i.e The PMD does not support > > shared_action, not even need to create one to use > > rte_flow_action_update() to avoid the confusion. Thoughts? > > > > [1] > > rte_flow_shared_action_update(uint16_port port, rte_shared_ctx *ctx, > > rte_flow_action *action, error) > > > > [2] > > rte_flow_action_update(uint16_port port, struct > > rte_flow_action_update_type_param *param, error) > > > Let me see if I understand you correctly, your suggestion is to allow > the application to change one action in one flow, but instead of creating > the context the application will just supply the rte_flow and the new actions > do I understand correctly? Yes. > > If so this it is a nice idea, but there are some issues with it, > 1. The PMD must save the flow which will result in memory consumption. struct rte_flow * driver handle any way store that information to as it would be needed for other rte_flow related APIs. > 2. Assume that two flows are using the same RSS action for example, so the PMD > reuse the RSS object he created for the first flow also for the second. Now changing > this RSS flow may result in also changing the second flow. (this can be solved by always > creating new action) It is not resuing the action, it more of updating the action. So the above said issue won't happen. It is removing the need for call `rte_flow_destroy()` and call `rte_flow_create()` if only action needs to update for THE given flow. That's it. > 3. It doesn't handle the main use case that the application wants to change number of > flows at the same time, which is the idea of this feature. We discussed this in detail and tried approach for the common code to make everything as shared action. Andrey quickly realizes that it is difficult without HW support. > > I also think that all PMD that support option 2 can support option 1 since > they can save in the ctx a list of flows and simply apply them again. So by > definition if PMD supports [2] it also support [1] while the other > way is not correct since it forces the PMD to save flows which like I said waste memory. If we use "rte_flow_shared_action_update(uint16_port port, rte_shared_ctx *ctx, rte_flow_action *action, error)", What would be ctx value for the HW does not support a shared context? That's is the only reason for my proposal. I understand, your concern about supporting two modes in PMD, I don't think, PMD needs to support RTE_FLOW_ACTION_UPDATE_TYPE_ACTION if RTE_FLOW_ACTION_UPDATE_TYPE_SHARED_ACTION supported. > > I suggest that we will go with option [1], and if needed in the future we will update the code. > using option [2] will result in dead code since at least for the current time no PMD will implement this > API. We are planning to update our PMD to support this once API is finalized. > > I can suggest one more thing maybe to change the name from shared_ctx to just ctx > which implicitly mean it can be shared but not a must. What do you think? (but again > I think by definition if a PMD can implement number [2] it can also implement it to number > of flows using API [2]. Just void *type is fine too, but we need an argument for type to cast it in application and/or driver. enum rte_flow_action_update_type { RTE_FLOW_ACTION_UPDATE_TYPE_SHARED_ACTION, RTE_FLOW_ACTION_UPDATE_TYPE_ACTION, }; > > > > In my suggestion the PMD simply needs to check if the new action and > > change the > > > context and to that action, or just change parameters in the action, if it is the > > same action. > > > > > > Let's go with the original patch API modified to support like you requested > > also changing the action, > > > based on my comments. > > > > > > > rte_flow_action_update(uint16_port port, struct > > > > rte_flow_action_update_type_param *param, error) > > > > > > > > > > [..nip..] > > Best, > Ori ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [dpdk-dev] [PATCH] add flow shared action API 2020-07-08 9:25 ` Jerin Jacob @ 2020-07-08 9:47 ` Ori Kam 2020-07-08 11:00 ` Jerin Jacob 0 siblings, 1 reply; 106+ messages in thread From: Ori Kam @ 2020-07-08 9:47 UTC (permalink / raw) To: Jerin Jacob Cc: Andrey Vesnovaty, Andrey Vesnovaty, Thomas Monjalon, Ferruh Yigit, Andrew Rybchenko, dpdk-dev Hi Jerin > -----Original Message----- > From: Jerin Jacob <jerinjacobk@gmail.com> > > On Wed, Jul 8, 2020 at 2:33 AM Ori Kam <orika@mellanox.com> wrote: > > > > Hi Jerin, > > Hi Ori, > > > > > > -----Original Message----- [..nip ..] > > > > > I think, simple API change would be to accommodate "rte_shared_ctx > > > > > *ctx, rte_flow_action *action" modes > > > > > without introducing the emulation for one or other mode, will be. > > > > > > > > > > enum rte_flow_action_update_type { > > > > > RTE_FLOW_ACTION_UPDATE_TYPE_SHARED_ACTION, > > > > > RTE_FLOW_ACTION_UPDATE_TYPE_ACTION, > > > > > }; > > > > > > > > > > struct rte_flow_action_update_type_param { > > > > > enum rte_flow_action_update_type type; > > > > > union { > > > > > struct rte_flow_action_update_type_shared_action_param > { > > > > > rte_shared_ctx *ctx; > > > > > } shared_action; > > > > > struct > rte_flow_action_update_type_shared_action_param { > > > > > rte_flow *flow, > > > > > rte_flow_action *action; > > > > > } action; > > > > > } > > > > > } > > > > > > > > > Thank you for the idea but I fall to see how your suggested API is simpler > than > > > the one suggested by me? > > > > > > My thought process with the below-proposed API[1] is that It is > > > dictates "_shared_action_" in API name as > > > well as arguments. I just thought of expressing it as either-or case > > > hence I thought [2] is better. i.e The PMD does not support > > > shared_action, not even need to create one to use > > > rte_flow_action_update() to avoid the confusion. Thoughts? > > > > > > [1] > > > rte_flow_shared_action_update(uint16_port port, rte_shared_ctx *ctx, > > > rte_flow_action *action, error) > > > > > > [2] > > > rte_flow_action_update(uint16_port port, struct > > > rte_flow_action_update_type_param *param, error) > > > > > Let me see if I understand you correctly, your suggestion is to allow > > the application to change one action in one flow, but instead of creating > > the context the application will just supply the rte_flow and the new actions > > do I understand correctly? > > Yes. > > > > > If so this it is a nice idea, but there are some issues with it, > > 1. The PMD must save the flow which will result in memory consumption. > > struct rte_flow * driver handle any way store that information to as > it would be needed > for other rte_flow related APIs. > The driver for example in Mellanox case save only the pointer to the flow so it can destroy it on request. It can't remove it and add it again since it is missing critical info. To save the info will cost memory. I assume this goes to other drivers. There is no real need to save anything but the flow pointer. > > > 2. Assume that two flows are using the same RSS action for example, so the > PMD > > reuse the RSS object he created for the first flow also for the second. Now > changing > > this RSS flow may result in also changing the second flow. (this can be solved > by always > > creating new action) > > It is not resuing the action, it more of updating the action. So the > above said issue won't happen. > It is removing the need for call `rte_flow_destroy()` and call > `rte_flow_create()` if only action > needs to update for THE given flow. That's it. > Again this means that the driver must save all flows so it will waste memory. also this doesn’t save any time, since the application can just do it the same way as the PMD there is no value to do it inside the PMD. > > > 3. It doesn't handle the main use case that the application wants to change > number of > > flows at the same time, which is the idea of this feature. > > We discussed this in detail and tried approach for the common code to > make everything > as shared action. Andrey quickly realizes that it is difficult without > HW support. Like everything in RTE flow this is all about HW support 😊 If the HW doesn’t support it don't do it. > > > > > I also think that all PMD that support option 2 can support option 1 since > > they can save in the ctx a list of flows and simply apply them again. So by > > definition if PMD supports [2] it also support [1] while the other > > way is not correct since it forces the PMD to save flows which like I said > waste memory. > > If we use "rte_flow_shared_action_update(uint16_port port, > rte_shared_ctx *ctx, rte_flow_action *action, error)", > What would be ctx value for the HW does not support a shared context? > That's is the only reason for > my proposal. I understand, your concern about supporting two modes in > PMD, I don't think, > PMD needs to support RTE_FLOW_ACTION_UPDATE_TYPE_ACTION if > RTE_FLOW_ACTION_UPDATE_TYPE_SHARED_ACTION supported. > This is the beauty of it ctx is opaque so the PMD can have what ever it wants to have. Just like an rte_flow that each PMD have different fields and usage. > > > > I suggest that we will go with option [1], and if needed in the future we will > update the code. > > using option [2] will result in dead code since at least for the current time no > PMD will implement this > > API. > > We are planning to update our PMD to support this once API is finalized. > Great very happy to hear that. This means that we should push this feature even faster 😊 > > > > I can suggest one more thing maybe to change the name from shared_ctx to > just ctx > > which implicitly mean it can be shared but not a must. What do you think? > (but again > > I think by definition if a PMD can implement number [2] it can also > implement it to number > > of flows using API [2]. > > Just void *type is fine too, but we need an argument for type to cast > it in application and/or driver. > Like said above this is opaque so the PMD knows what to expect. > enum rte_flow_action_update_type { > RTE_FLOW_ACTION_UPDATE_TYPE_SHARED_ACTION, > RTE_FLOW_ACTION_UPDATE_TYPE_ACTION, > }; > > > > > > > In my suggestion the PMD simply needs to check if the new action and > > > change the > > > > context and to that action, or just change parameters in the action, if it is > the > > > same action. > > > > > > > > Let's go with the original patch API modified to support like you requested > > > also changing the action, > > > > based on my comments. > > > > > > > > > rte_flow_action_update(uint16_port port, struct > > > > > rte_flow_action_update_type_param *param, error) > > > > > > > > > > > > > [..nip..] > > > > Best, > > Ori ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [dpdk-dev] [PATCH] add flow shared action API 2020-07-08 9:47 ` Ori Kam @ 2020-07-08 11:00 ` Jerin Jacob 2020-07-08 11:50 ` Thomas Monjalon 2020-07-08 12:18 ` Ori Kam 0 siblings, 2 replies; 106+ messages in thread From: Jerin Jacob @ 2020-07-08 11:00 UTC (permalink / raw) To: Ori Kam Cc: Andrey Vesnovaty, Andrey Vesnovaty, Thomas Monjalon, Ferruh Yigit, Andrew Rybchenko, dpdk-dev On Wed, Jul 8, 2020 at 3:17 PM Ori Kam <orika@mellanox.com> wrote: > > Hi Jerin > > > -----Original Message----- > > From: Jerin Jacob <jerinjacobk@gmail.com> > > > > On Wed, Jul 8, 2020 at 2:33 AM Ori Kam <orika@mellanox.com> wrote: > > > > > > Hi Jerin, > > > > Hi Ori, > > > > > > > > > -----Original Message----- > [..nip ..] > > > > > > > I think, simple API change would be to accommodate "rte_shared_ctx > > > > > > *ctx, rte_flow_action *action" modes > > > > > > without introducing the emulation for one or other mode, will be. > > > > > > > > > > > > enum rte_flow_action_update_type { > > > > > > RTE_FLOW_ACTION_UPDATE_TYPE_SHARED_ACTION, > > > > > > RTE_FLOW_ACTION_UPDATE_TYPE_ACTION, > > > > > > }; > > > > > > > > > > > > struct rte_flow_action_update_type_param { > > > > > > enum rte_flow_action_update_type type; > > > > > > union { > > > > > > struct rte_flow_action_update_type_shared_action_param > > { > > > > > > rte_shared_ctx *ctx; > > > > > > } shared_action; > > > > > > struct > > rte_flow_action_update_type_shared_action_param { > > > > > > rte_flow *flow, > > > > > > rte_flow_action *action; > > > > > > } action; > > > > > > } > > > > > > } > > > > > > > > > > > Thank you for the idea but I fall to see how your suggested API is simpler > > than > > > > the one suggested by me? > > > > > > > > My thought process with the below-proposed API[1] is that It is > > > > dictates "_shared_action_" in API name as > > > > well as arguments. I just thought of expressing it as either-or case > > > > hence I thought [2] is better. i.e The PMD does not support > > > > shared_action, not even need to create one to use > > > > rte_flow_action_update() to avoid the confusion. Thoughts? > > > > > > > > [1] > > > > rte_flow_shared_action_update(uint16_port port, rte_shared_ctx *ctx, > > > > rte_flow_action *action, error) > > > > > > > > [2] > > > > rte_flow_action_update(uint16_port port, struct > > > > rte_flow_action_update_type_param *param, error) > > > > > > > Let me see if I understand you correctly, your suggestion is to allow > > > the application to change one action in one flow, but instead of creating > > > the context the application will just supply the rte_flow and the new actions > > > do I understand correctly? > > > > Yes. > > > > > > > > If so this it is a nice idea, but there are some issues with it, > > > 1. The PMD must save the flow which will result in memory consumption. > > > > struct rte_flow * driver handle any way store that information to as > > it would be needed > > for other rte_flow related APIs. > > > The driver for example in Mellanox case save only the pointer to the flow so it can > destroy it on request. It can't remove it and add it again since it is missing critical > info. To save the info will cost memory. I assume this goes to other drivers. > There is no real need to save anything but the flow pointer. If driver _need_ this feature, then the driver needs to store some info in the flow object, In our case, it will be the index of the MCAM action, etc. Yes. it up to driver the how they want to support it. > > > > > > 2. Assume that two flows are using the same RSS action for example, so the > > PMD > > > reuse the RSS object he created for the first flow also for the second. Now > > changing > > > this RSS flow may result in also changing the second flow. (this can be solved > > by always > > > creating new action) > > > > It is not resuing the action, it more of updating the action. So the > > above said issue won't happen. > > It is removing the need for call `rte_flow_destroy()` and call > > `rte_flow_create()` if only action > > needs to update for THE given flow. That's it. > > > Again this means that the driver must save all flows so it will waste memory. > also this doesn’t save any time, since the application can just do it the same way > as the PMD there is no value to do it inside the PMD. Why driver need to save all the flow, We need to save all the flow ONLY when we need to emulate the shared context. We need to save only additional info like, MCAM index where the action was allocated or such, so that it can be replaced. > > > > > > 3. It doesn't handle the main use case that the application wants to change > > number of > > > flows at the same time, which is the idea of this feature. > > > > We discussed this in detail and tried approach for the common code to > > make everything > > as shared action. Andrey quickly realizes that it is difficult without > > HW support. > Like everything in RTE flow this is all about HW support > If the HW doesn’t support it don't do it. > > > > > > > > > I also think that all PMD that support option 2 can support option 1 since > > > they can save in the ctx a list of flows and simply apply them again. So by > > > definition if PMD supports [2] it also support [1] while the other > > > way is not correct since it forces the PMD to save flows which like I said > > waste memory. > > > > If we use "rte_flow_shared_action_update(uint16_port port, > > rte_shared_ctx *ctx, rte_flow_action *action, error)", > > What would be ctx value for the HW does not support a shared context? > > That's is the only reason for > > my proposal. I understand, your concern about supporting two modes in > > PMD, I don't think, > > PMD needs to support RTE_FLOW_ACTION_UPDATE_TYPE_ACTION if > > RTE_FLOW_ACTION_UPDATE_TYPE_SHARED_ACTION supported. > > > This is the beauty of it ctx is opaque so the PMD can have what ever it wants to have. Just like > an rte_flow that each PMD have different fields and usage. > > > > > > > I suggest that we will go with option [1], and if needed in the future we will > > update the code. > > > using option [2] will result in dead code since at least for the current time no > > PMD will implement this > > > API. > > > > We are planning to update our PMD to support this once API is finalized. > > > Great very happy to hear that. > This means that we should push this feature even faster > > > > > > > I can suggest one more thing maybe to change the name from shared_ctx to > > just ctx > > > which implicitly mean it can be shared but not a must. What do you think? > > (but again > > > I think by definition if a PMD can implement number [2] it can also > > implement it to number > > > of flows using API [2]. > > > > Just void *type is fine too, but we need an argument for type to cast > > it in application and/or driver. > > > Like said above this is opaque so the PMD knows what to expect. Driver side it is OKAY but how the application can update it? The PMD does not need or does not have a shared object. Otherway to ask is, Could you have share the API call sequence using "rte_flow_shared_action_update(uint16_port port, rte_shared_ctx *ctx, rte_flow_action *action, error)" to enable support for the following category of HW as I mentioned earlier. - The HW has "pattern" and "action" mapped to different HW objects and action can be updated any time without destroying and create.(a,k,a Does not have shared HW object) > > > enum rte_flow_action_update_type { > > RTE_FLOW_ACTION_UPDATE_TYPE_SHARED_ACTION, > > RTE_FLOW_ACTION_UPDATE_TYPE_ACTION, > > }; > > > > > > > > > > > In my suggestion the PMD simply needs to check if the new action and > > > > change the > > > > > context and to that action, or just change parameters in the action, if it is > > the > > > > same action. > > > > > > > > > > Let's go with the original patch API modified to support like you requested > > > > also changing the action, > > > > > based on my comments. > > > > > > > > > > > rte_flow_action_update(uint16_port port, struct > > > > > > rte_flow_action_update_type_param *param, error) > > > > > > > > > > > > > > > > [..nip..] > > > > > > Best, > > > Ori ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [dpdk-dev] [PATCH] add flow shared action API 2020-07-08 11:00 ` Jerin Jacob @ 2020-07-08 11:50 ` Thomas Monjalon 2020-07-08 12:18 ` Ori Kam 1 sibling, 0 replies; 106+ messages in thread From: Thomas Monjalon @ 2020-07-08 11:50 UTC (permalink / raw) To: Jerin Jacob Cc: Ori Kam, Andrey Vesnovaty, Andrey Vesnovaty, Ferruh Yigit, Andrew Rybchenko, dpdk-dev 08/07/2020 13:00, Jerin Jacob: > Driver side it is OKAY but how the application can update it? The PMD does not > need or does not have a shared object. > > Otherway to ask is, Could you have share the API call sequence using > "rte_flow_shared_action_update(uint16_port port, rte_shared_ctx *ctx, > rte_flow_action *action, error)" > > to enable support for the following category of HW as I mentioned earlier. > - The HW has "pattern" and "action" mapped to different HW objects and > action can be updated any time without destroying and create.(a,k,a > Does not have shared HW object) Yes an example of API usage may help. We should get one today. ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [dpdk-dev] [PATCH] add flow shared action API 2020-07-08 11:00 ` Jerin Jacob 2020-07-08 11:50 ` Thomas Monjalon @ 2020-07-08 12:18 ` Ori Kam [not found] ` <20200708204015.24429-2-andreyv@mellanox.com> [not found] ` <20200708204015.24429-3-andreyv@mellanox.com> 1 sibling, 2 replies; 106+ messages in thread From: Ori Kam @ 2020-07-08 12:18 UTC (permalink / raw) To: Jerin Jacob Cc: Andrey Vesnovaty, Andrey Vesnovaty, Thomas Monjalon, Ferruh Yigit, Andrew Rybchenko, dpdk-dev Hi Jerin and Andrey, > -----Original Message----- > From: Jerin Jacob <jerinjacobk@gmail.com> > Subject: Re: [dpdk-dev] [PATCH] add flow shared action API > > On Wed, Jul 8, 2020 at 3:17 PM Ori Kam <orika@mellanox.com> wrote: > > > > Hi Jerin > > > > > -----Original Message----- > > > From: Jerin Jacob <jerinjacobk@gmail.com> > > > > > > On Wed, Jul 8, 2020 at 2:33 AM Ori Kam <orika@mellanox.com> wrote: > > > > > > > > Hi Jerin, > > > > > > Hi Ori, > > > > > > > > > > > > -----Original Message----- > > [..nip ..] > > > > > > > > > I think, simple API change would be to accommodate > "rte_shared_ctx > > > > > > > *ctx, rte_flow_action *action" modes > > > > > > > without introducing the emulation for one or other mode, will be. > > > > > > > > > > > > > > enum rte_flow_action_update_type { > > > > > > > RTE_FLOW_ACTION_UPDATE_TYPE_SHARED_ACTION, > > > > > > > RTE_FLOW_ACTION_UPDATE_TYPE_ACTION, > > > > > > > }; > > > > > > > > > > > > > > struct rte_flow_action_update_type_param { > > > > > > > enum rte_flow_action_update_type type; > > > > > > > union { > > > > > > > struct > rte_flow_action_update_type_shared_action_param > > > { > > > > > > > rte_shared_ctx *ctx; > > > > > > > } shared_action; > > > > > > > struct > > > rte_flow_action_update_type_shared_action_param { > > > > > > > rte_flow *flow, > > > > > > > rte_flow_action *action; > > > > > > > } action; > > > > > > > } > > > > > > > } > > > > > > > > > > > > > Thank you for the idea but I fall to see how your suggested API is > simpler > > > than > > > > > the one suggested by me? > > > > > > > > > > My thought process with the below-proposed API[1] is that It is > > > > > dictates "_shared_action_" in API name as > > > > > well as arguments. I just thought of expressing it as either-or case > > > > > hence I thought [2] is better. i.e The PMD does not support > > > > > shared_action, not even need to create one to use > > > > > rte_flow_action_update() to avoid the confusion. Thoughts? > > > > > > > > > > [1] > > > > > rte_flow_shared_action_update(uint16_port port, rte_shared_ctx *ctx, > > > > > rte_flow_action *action, error) > > > > > > > > > > [2] > > > > > rte_flow_action_update(uint16_port port, struct > > > > > rte_flow_action_update_type_param *param, error) > > > > > > > > > Let me see if I understand you correctly, your suggestion is to allow > > > > the application to change one action in one flow, but instead of creating > > > > the context the application will just supply the rte_flow and the new > actions > > > > do I understand correctly? > > > > > > Yes. > > > > > > > > > > > If so this it is a nice idea, but there are some issues with it, > > > > 1. The PMD must save the flow which will result in memory consumption. > > > > > > struct rte_flow * driver handle any way store that information to as > > > it would be needed > > > for other rte_flow related APIs. > > > > > The driver for example in Mellanox case save only the pointer to the flow so > it can > > destroy it on request. It can't remove it and add it again since it is missing > critical > > info. To save the info will cost memory. I assume this goes to other drivers. > > There is no real need to save anything but the flow pointer. > > If driver _need_ this feature, then the driver needs to store some > info in the flow object, > In our case, it will be the index of the MCAM action, etc. > Yes. it up to driver the how they want to support it. > Or to save the flow in the context, but just like you said it is PMD dependent. > > > > > > > > > > 2. Assume that two flows are using the same RSS action for example, so > the > > > PMD > > > > reuse the RSS object he created for the first flow also for the second. Now > > > changing > > > > this RSS flow may result in also changing the second flow. (this can be > solved > > > by always > > > > creating new action) > > > > > > It is not resuing the action, it more of updating the action. So the > > > above said issue won't happen. > > > It is removing the need for call `rte_flow_destroy()` and call > > > `rte_flow_create()` if only action > > > needs to update for THE given flow. That's it. > > > > > Again this means that the driver must save all flows so it will waste memory. > > also this doesn’t save any time, since the application can just do it the same > way > > as the PMD there is no value to do it inside the PMD. > > Why driver need to save all the flow, We need to save all the flow ONLY when > we need to emulate the shared context. We need to save only additional > info like, > MCAM index where the action was allocated or such, so that it can be > replaced. That’s depends on PMD and HW. > > > > > > > > > > > 3. It doesn't handle the main use case that the application wants to > change > > > number of > > > > flows at the same time, which is the idea of this feature. > > > > > > We discussed this in detail and tried approach for the common code to > > > make everything > > > as shared action. Andrey quickly realizes that it is difficult without > > > HW support. > > Like everything in RTE flow this is all about HW support > > If the HW doesn’t support it don't do it. > > > > > > > > > > > > > I also think that all PMD that support option 2 can support option 1 since > > > > they can save in the ctx a list of flows and simply apply them again. So by > > > > definition if PMD supports [2] it also support [1] while the other > > > > way is not correct since it forces the PMD to save flows which like I said > > > waste memory. > > > > > > If we use "rte_flow_shared_action_update(uint16_port port, > > > rte_shared_ctx *ctx, rte_flow_action *action, error)", > > > What would be ctx value for the HW does not support a shared context? > > > That's is the only reason for > > > my proposal. I understand, your concern about supporting two modes in > > > PMD, I don't think, > > > PMD needs to support RTE_FLOW_ACTION_UPDATE_TYPE_ACTION if > > > RTE_FLOW_ACTION_UPDATE_TYPE_SHARED_ACTION supported. > > > > > This is the beauty of it ctx is opaque so the PMD can have what ever it wants > to have. Just like > > an rte_flow that each PMD have different fields and usage. > > > > > > > > > > I suggest that we will go with option [1], and if needed in the future we > will > > > update the code. > > > > using option [2] will result in dead code since at least for the current time > no > > > PMD will implement this > > > > API. > > > > > > We are planning to update our PMD to support this once API is finalized. > > > > > Great very happy to hear that. > > This means that we should push this feature even faster > > > > > > > > > > I can suggest one more thing maybe to change the name from shared_ctx > to > > > just ctx > > > > which implicitly mean it can be shared but not a must. What do you think? > > > (but again > > > > I think by definition if a PMD can implement number [2] it can also > > > implement it to number > > > > of flows using API [2]. > > > > > > Just void *type is fine too, but we need an argument for type to cast > > > it in application and/or driver. > > > > > Like said above this is opaque so the PMD knows what to expect. > > Driver side it is OKAY but how the application can update it? The PMD does not > need or does not have a shared object. > The PMD is getting the shared object that the PMD created so the PMD has all information. It is the PMD that control what is in the ctx. The application just saves this pointer and gives it to the MD on modify and destroy or the creation of flow. From application point of view he asks the DPDK to change the ctx that it has to do a different action, the application doesn’t know or care what is inside the the ctx. > Otherway to ask is, Could you have share the API call sequence using > "rte_flow_shared_action_update(uint16_port port, rte_shared_ctx *ctx, > rte_flow_action *action, error)" > I hope that by the end of today Andrey will post such a code. > to enable support for the following category of HW as I mentioned earlier. > - The HW has "pattern" and "action" mapped to different HW objects and > action can be updated any time without destroying and create.(a,k,a > Does not have shared HW object) I'm not sure I understand this comment. Let's advance in the following way. Andrey please send V2 that changes the API to use rte_flow_action and a sample code and some PMD code so we can see how it should work. > > > > > > > enum rte_flow_action_update_type { > > > RTE_FLOW_ACTION_UPDATE_TYPE_SHARED_ACTION, > > > RTE_FLOW_ACTION_UPDATE_TYPE_ACTION, > > > }; > > > > > > > > > > > > > > > In my suggestion the PMD simply needs to check if the new action and > > > > > change the > > > > > > context and to that action, or just change parameters in the action, if > it is > > > the > > > > > same action. > > > > > > > > > > > > Let's go with the original patch API modified to support like you > requested > > > > > also changing the action, > > > > > > based on my comments. > > > > > > > > > > > > > rte_flow_action_update(uint16_port port, struct > > > > > > > rte_flow_action_update_type_param *param, error) > > > > > > > > > > > > > > > > > > > [..nip..] > > > > > > > > Best, > > > > Ori ^ permalink raw reply [flat|nested] 106+ messages in thread
[parent not found: <20200708204015.24429-2-andreyv@mellanox.com>]
* Re: [dpdk-dev] [PATCH v2 1/6] ethdev: add flow shared action API [not found] ` <20200708204015.24429-2-andreyv@mellanox.com> @ 2020-07-13 8:04 ` Kinsella, Ray 2020-07-13 10:16 ` Andrew Rybchenko 2020-07-15 8:54 ` Andrew Rybchenko 0 siblings, 2 replies; 106+ messages in thread From: Kinsella, Ray @ 2020-07-13 8:04 UTC (permalink / raw) To: Andrey Vesnovaty, Neil Horman, Thomas Monjalon, Ferruh Yigit, Andrew Rybchenko, Ori Kam Cc: dev, jerinjacobk, stephen, bruce.richardson, viacheslavo, andrey.vesnovaty On 08/07/2020 21:40, Andrey Vesnovaty wrote: > From: Andrey Vesnovaty <andrey.vesnovaty@gmail.com> > > This commit introduces extension of DPDK flow action API enabling > sharing of single rte_flow_action in multiple flows. The API intended for > PMDs where multiple HW offloaded flows can reuse the same HW > essence/object representing flow action and modification of such an > essence/object effects all the rules using it. > > Motivation and example > === > Adding or removing one or more queues to RSS used by multiple flow rules > imposes per rule toll for current DPDK flow API; the scenario requires > for each flow sharing cloned RSS action: > - call `rte_flow_destroy()` > - call `rte_flow_create()` with modified RSS action > > API for sharing action and its in-place update benefits: > - reduce the overhead of multiple RSS flow rules reconfiguration > - optimize resource utilization by sharing action across of multiple > flows > > Change description > === > > Shared action > === > In order to represent flow action shared by multiple flows new action > type RTE_FLOW_ACTION_TYPE_SHARED is introduced (see `enum > rte_flow_action_type`). > Actually the introduced API decouples action from any specific flow and > enables sharing of single action by its handle across multiple flows. > > Shared action create/use/destroy > === > Shared action may be reused by some or none flow rules at any given > moment, i.e. shared action reside outside of the context of any flow. > Shared action represent HW resources/objects used for action offloading > implementation. > API for shared action create (see `rte_flow_shared_action_create()`): > - should allocate HW resources and make related initializations required > for shared action implementation. > - make necessary preparations to maintain shared access to > the action resources, configuration and state. > API for shared action destroy (see `rte_flow_shared_action_destroy()`) > should release HW resources and make related cleanups required for shared > action implementation. > > In order to share some flow action reuse the handle of type > `struct rte_flow_shared_action` returned by > rte_flow_shared_action_create() as a `conf` field of > `struct rte_flow_action` (see "example" section). > > If some shared action not used by any flow rule all resources allocated > by the shared action can be released by rte_flow_shared_action_destroy() > (see "example" section). The shared action handle passed as argument to > destroy API should not be used any further i.e. result of the usage is > undefined. > > Shared action re-configuration > === > Shared action behavior defined by its configuration can be updated via > rte_flow_shared_action_update() (see "example" section). The shared > action update operation modifies HW related resources/objects allocated > on the action creation. The number of operations performed by the update > operation should not be dependent on number of flows sharing the related > action. On return of shared action update API action behavior should be > according to updated configuration for all flows sharing the action. > > Shared action query > === > Provide separate API to query shared action sate (see > rte_flow_shared_action_update()). Taking a counter as an example: query > returns value aggregating all counter increments across all flow rules > sharing the counter. > > PMD support > === > The support of introduced API is pure PMD specific design and > responsibility for each action type (see struct rte_flow_ops). > > testpmd > === > In order to utilize introduced API testpmd cli may implement following > extension > create/update/destroy/query shared action accordingly > > flow shared_action create {port_id} [index] {action} > flow shared_action update {port_id} {index} {action} > flow shared_action destroy {port_id} {index} > flow shared_action query {port_id} {index} > > testpmd example > === > > configure rss to queues 1 & 2 > > testpmd> flow shared_action create 0 100 rss 1 2 > > create flow rule utilizing shared action > > testpmd> flow create 0 ingress \ > pattern eth dst is 0c:42:a1:15:fd:ac / ipv6 / tcp / end \ > actions shared 100 end / end > > add 2 more queues > > testpmd> flow shared_action modify 0 100 rss 1 2 3 4 > > example > === > > struct rte_flow_action actions[2]; > struct rte_flow_action action; > /* skipped: initialize action */ > struct rte_flow_shared_action *handle = rte_flow_shared_action_create( > port_id, &action, &error); > actions[0].type = RTE_FLOW_ACTION_TYPE_SHARED; > actions[0].conf = handle; > actions[1].type = RTE_FLOW_ACTION_TYPE_END; > /* skipped: init attr0 & pattern0 args */ > struct rte_flow *flow0 = rte_flow_create(port_id, &attr0, pattern0, > actions, error); > /* create more rules reusing shared action */ > struct rte_flow *flow1 = rte_flow_create(port_id, &attr1, pattern1, > actions, error); > /* skipped: for flows 2 till N */ > struct rte_flow *flowN = rte_flow_create(port_id, &attrN, patternN, > actions, error); > /* update shared action */ > struct rte_flow_action updated_action; > /* > * skipped: initialize updated_action according to desired action > * configuration change > */ > rte_flow_shared_action_update(port_id, handle, &updated_action, error); > /* > * from now on all flows 1 till N will act according to configuration of > * updated_action > */ > /* skipped: destroy all flows 1 till N */ > rte_flow_shared_action_destroy(port_id, handle, error); > > Signed-off-by: Andrey Vesnovaty <andreyv@mellanox.com> > --- > lib/librte_ethdev/rte_ethdev_version.map | 6 + > lib/librte_ethdev/rte_flow.c | 81 +++++++++++++ > lib/librte_ethdev/rte_flow.h | 148 ++++++++++++++++++++++- > lib/librte_ethdev/rte_flow_driver.h | 22 ++++ > 4 files changed, 256 insertions(+), 1 deletion(-) > > diff --git a/lib/librte_ethdev/rte_ethdev_version.map b/lib/librte_ethdev/rte_ethdev_version.map > index 7155056045..119d84976a 100644 > --- a/lib/librte_ethdev/rte_ethdev_version.map > +++ b/lib/librte_ethdev/rte_ethdev_version.map > @@ -241,4 +241,10 @@ EXPERIMENTAL { > __rte_ethdev_trace_rx_burst; > __rte_ethdev_trace_tx_burst; > rte_flow_get_aged_flows; > + > + # added in 20.08 > + rte_flow_shared_action_create; > + rte_flow_shared_action_destroy; > + rte_flow_shared_action_update; > + rte_flow_shared_action_query; > }; > diff --git a/lib/librte_ethdev/rte_flow.c b/lib/librte_ethdev/rte_flow.c > index 1685be5f73..0ac4d31a13 100644 > --- a/lib/librte_ethdev/rte_flow.c > +++ b/lib/librte_ethdev/rte_flow.c > @@ -1250,3 +1250,84 @@ rte_flow_get_aged_flows(uint16_t port_id, void **contexts, > RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > NULL, rte_strerror(ENOTSUP)); > } > + > +struct rte_flow_shared_action * > +rte_flow_shared_action_create(uint16_t port_id, > + const struct rte_flow_action *action, > + struct rte_flow_error *error) > +{ > + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; > + struct rte_flow_shared_action *shared_action; > + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error); > + > + if (unlikely(!ops)) > + return NULL; > + if (likely(!!ops->shared_action_create)) { > + shared_action = ops->shared_action_create(dev, action, error); > + if (shared_action == NULL) > + flow_err(port_id, -rte_errno, error); > + return shared_action; > + } > + rte_flow_error_set(error, ENOSYS, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > + NULL, rte_strerror(ENOSYS)); > + return NULL; > +} > + > +int > +rte_flow_shared_action_destroy(uint16_t port_id, > + struct rte_flow_shared_action *action, > + struct rte_flow_error *error) > +{ > + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; > + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error); > + > + if (unlikely(!ops)) > + return -rte_errno; > + if (likely(!!ops->shared_action_destroy)) > + return flow_err(port_id, > + ops->shared_action_destroy(dev, action, error), > + error); > + return rte_flow_error_set(error, ENOSYS, > + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > + NULL, rte_strerror(ENOSYS)); > +} > + > +int > +rte_flow_shared_action_update(uint16_t port_id, > + struct rte_flow_shared_action *action, > + const struct rte_flow_action *update, > + struct rte_flow_error *error) > +{ > + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; > + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error); > + > + if (unlikely(!ops)) > + return -rte_errno; > + if (likely(!!ops->shared_action_update)) > + return flow_err(port_id, ops->shared_action_update(dev, action, > + update, error), > + error); > + return rte_flow_error_set(error, ENOSYS, > + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > + NULL, rte_strerror(ENOSYS)); > +} > + > +int > +rte_flow_shared_action_query(uint16_t port_id, > + const struct rte_flow_shared_action *action, > + void *data, > + struct rte_flow_error *error) > +{ > + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; > + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error); > + > + if (unlikely(!ops)) > + return -rte_errno; > + if (likely(!!ops->shared_action_query)) > + return flow_err(port_id, ops->shared_action_query(dev, action, > + data, error), > + error); > + return rte_flow_error_set(error, ENOSYS, > + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > + NULL, rte_strerror(ENOSYS)); > +} > diff --git a/lib/librte_ethdev/rte_flow.h b/lib/librte_ethdev/rte_flow.h > index b0e4199192..257456b14a 100644 > --- a/lib/librte_ethdev/rte_flow.h > +++ b/lib/librte_ethdev/rte_flow.h > @@ -1681,7 +1681,8 @@ enum rte_flow_action_type { > /** > * Enables counters for this flow rule. > * > - * These counters can be retrieved and reset through rte_flow_query(), > + * These counters can be retrieved and reset through rte_flow_query() or > + * rte_flow_shared_action_query() if the action provided via handle, > * see struct rte_flow_query_count. > * > * See struct rte_flow_action_count. > @@ -2099,6 +2100,14 @@ enum rte_flow_action_type { > * see enum RTE_ETH_EVENT_FLOW_AGED > */ > RTE_FLOW_ACTION_TYPE_AGE, > + > + /** > + * Describes action shared a cross multiple flow rules. > + * > + * Enables multiple rules reference the same action by handle (see > + * struct rte_flow_shared_action). > + */ > + RTE_FLOW_ACTION_TYPE_SHARED, > }; > > /** > @@ -2660,6 +2669,20 @@ struct rte_flow_action_set_dscp { > uint8_t dscp; > }; > > + > +/** > + * RTE_FLOW_ACTION_TYPE_SHARED > + * > + * Opaque type returned after successfully creating a shared action. > + * > + * This handle can be used to manage and query the related action: > + * - share it a cross multiple flow rules > + * - update action configuration > + * - query action data > + * - destroy action > + */ > +struct rte_flow_shared_action; > + > /* Mbuf dynamic field offset for metadata. */ > extern int32_t rte_flow_dynf_metadata_offs; > > @@ -3324,6 +3347,129 @@ int > rte_flow_get_aged_flows(uint16_t port_id, void **contexts, > uint32_t nb_contexts, struct rte_flow_error *error); > > +/** > + * @warning > + * @b EXPERIMENTAL: this API may change without prior notice. > + * > + * Create shared action for reuse in multiple flow rules. > + * > + * @param[in] port_id > + * The port identifier of the Ethernet device. > + * @param[in] action > + * Action configuration for shared action creation. > + * @param[out] error > + * Perform verbose error reporting if not NULL. PMDs initialize this > + * structure in case of error only. > + * @return > + * A valid handle in case of success, NULL otherwise and rte_errno is set > + * to one of the error codes defined: > + * - (ENOSYS) if underlying device does not support this functionality. > + * - (EIO) if underlying device is removed. > + * - (EINVAL) if *action* invalid. > + * - (ENOTSUP) if *action* valid but unsupported. > + */ > +__rte_experimental > +struct rte_flow_shared_action * > +rte_flow_shared_action_create(uint16_t port_id, > + const struct rte_flow_action *action, > + struct rte_flow_error *error); > + > +/** > + * @warning > + * @b EXPERIMENTAL: this API may change without prior notice. > + * > + * Destroys the shared action by handle. > + * > + * @param[in] port_id > + * The port identifier of the Ethernet device. > + * @param[in] action > + * Handle for the shared action to be destroyed. > + * @param[out] error > + * Perform verbose error reporting if not NULL. PMDs initialize this > + * structure in case of error only. > + * @return > + * - (0) if success. > + * - (-ENOSYS) if underlying device does not support this functionality. > + * - (-EIO) if underlying device is removed. > + * - (-ENOENT) if action pointed by *action* handle was not found. > + * - (-ETOOMANYREFS) if action pointed by *action* handle still used by one or > + * more rules > + * rte_errno is also set. > + */ > +__rte_experimental > +int > +rte_flow_shared_action_destroy(uint16_t port_id, > + struct rte_flow_shared_action *action, > + struct rte_flow_error *error); > + > +/** > + * @warning > + * @b EXPERIMENTAL: this API may change without prior notice. > + * > + * Updates inplace the shared action configuration pointed by *action* handle > + * with the configuration provided as *update* argument. > + * The update of the shared action configuration effects all flow rules reusing > + * the action via handle. > + * > + * @param[in] port_id > + * The port identifier of the Ethernet device. > + * @param[in] action > + * Handle for the shared action to be updated. > + * @param[in] update > + * Action specification used to modify the action pointed by handle. > + * *update* should be of same type with the action pointed by the *action* > + * handle argument, otherwise considered as invalid. > + * @param[out] error > + * Perform verbose error reporting if not NULL. PMDs initialize this > + * structure in case of error only. > + * @return > + * - (0) if success. > + * - (-ENOSYS) if underlying device does not support this functionality. > + * - (-EIO) if underlying device is removed. > + * - (-EINVAL) if *update* invalid. > + * - (-ENOTSUP) if *update* valid but unsupported. > + * - (-ENOENT) if action pointed by *ctx* was not found. > + * rte_errno is also set. > + */ > +__rte_experimental > +int > +rte_flow_shared_action_update(uint16_t port_id, > + struct rte_flow_shared_action *action, > + const struct rte_flow_action *update, > + struct rte_flow_error *error); > + > +/** > + * @warning > + * @b EXPERIMENTAL: this API may change without prior notice. > + * > + * Query the shared action by handle. > + * > + * This function allows retrieving action-specific data such as counters. > + * Data is gathered by special action which may be present/referenced in > + * more than one flow rule definition. > + * > + * \see RTE_FLOW_ACTION_TYPE_COUNT > + * > + * @param port_id > + * Port identifier of Ethernet device. > + * @param[in] action > + * Handle for the shared action to query. > + * @param[in, out] data > + * Pointer to storage for the associated query data type. > + * @param[out] error > + * Perform verbose error reporting if not NULL. PMDs initialize this > + * structure in case of error only. > + * > + * @return > + * 0 on success, a negative errno value otherwise and rte_errno is set. > + */ > +__rte_experimental > +int > +rte_flow_shared_action_query(uint16_t port_id, > + const struct rte_flow_shared_action *action, > + void *data, > + struct rte_flow_error *error); > + > #ifdef __cplusplus > } > #endif > diff --git a/lib/librte_ethdev/rte_flow_driver.h b/lib/librte_ethdev/rte_flow_driver.h > index 881cc469b7..a2cae1b53c 100644 > --- a/lib/librte_ethdev/rte_flow_driver.h > +++ b/lib/librte_ethdev/rte_flow_driver.h > @@ -107,6 +107,28 @@ struct rte_flow_ops { > void **context, > uint32_t nb_contexts, > struct rte_flow_error *err); > + /** See rte_flow_shared_action_create() */ > + struct rte_flow_shared_action *(*shared_action_create) > + (struct rte_eth_dev *dev, > + const struct rte_flow_action *action, > + struct rte_flow_error *error); > + /** See rte_flow_shared_action_destroy() */ > + int (*shared_action_destroy) > + (struct rte_eth_dev *dev, > + struct rte_flow_shared_action *shared_action, > + struct rte_flow_error *error); > + /** See rte_flow_shared_action_update() */ > + int (*shared_action_update) > + (struct rte_eth_dev *dev, > + struct rte_flow_shared_action *shared_action, > + const struct rte_flow_action *update, > + struct rte_flow_error *error); > + /** See rte_flow_shared_action_query() */ > + int (*shared_action_query) > + (struct rte_eth_dev *dev, > + const struct rte_flow_shared_action *shared_action, > + void *data, > + struct rte_flow_error *error); > }; > > /** > The modification of "struct rte_flow_ops", looks fine. Acked-by: Ray Kinsella <mdr@ashroe.eu> ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [dpdk-dev] [PATCH v2 1/6] ethdev: add flow shared action API 2020-07-13 8:04 ` [dpdk-dev] [PATCH v2 1/6] ethdev: " Kinsella, Ray @ 2020-07-13 10:16 ` Andrew Rybchenko 2020-07-15 8:54 ` Andrew Rybchenko 1 sibling, 0 replies; 106+ messages in thread From: Andrew Rybchenko @ 2020-07-13 10:16 UTC (permalink / raw) To: Andrey Vesnovaty, Neil Horman, Thomas Monjalon, Ferruh Yigit, Andrew Rybchenko, Ori Kam Cc: Kinsella, Ray, dev, jerinjacobk, stephen, bruce.richardson, viacheslavo, andrey.vesnovaty On 7/13/20 11:04 AM, Kinsella, Ray wrote: > > > On 08/07/2020 21:40, Andrey Vesnovaty wrote: >> From: Andrey Vesnovaty <andrey.vesnovaty@gmail.com> >> >> This commit introduces extension of DPDK flow action API enabling >> sharing of single rte_flow_action in multiple flows. The API intended for >> PMDs where multiple HW offloaded flows can reuse the same HW >> essence/object representing flow action and modification of such an >> essence/object effects all the rules using it. >> >> Motivation and example >> === >> Adding or removing one or more queues to RSS used by multiple flow rules >> imposes per rule toll for current DPDK flow API; the scenario requires >> for each flow sharing cloned RSS action: >> - call `rte_flow_destroy()` >> - call `rte_flow_create()` with modified RSS action >> >> API for sharing action and its in-place update benefits: >> - reduce the overhead of multiple RSS flow rules reconfiguration It *allows* to reduce the overhead of multiple RSS flow rules reconfiguration. I.e. it is not mandatory. PMD may do it in SW, if HW does not support it. I see no problem to allow it. Even if it is done in PMD, it is already an optimization since applications have unified interface and internally it should be cheaper to do it. I'd consider to implement generic SW helper API for PMDs which cannot have shared actions in HW, but would like to simulate it in SW. It would allow to avoid the fallback in applications. >> - optimize resource utilization by sharing action across of multiple >> flows >> >> Change description >> === >> >> Shared action >> === >> In order to represent flow action shared by multiple flows new action >> type RTE_FLOW_ACTION_TYPE_SHARED is introduced (see `enum >> rte_flow_action_type`). >> Actually the introduced API decouples action from any specific flow and >> enables sharing of single action by its handle across multiple flows. >> >> Shared action create/use/destroy >> === >> Shared action may be reused by some or none flow rules at any given >> moment, i.e. shared action reside outside of the context of any flow. >> Shared action represent HW resources/objects used for action offloading >> implementation. >> API for shared action create (see `rte_flow_shared_action_create()`): >> - should allocate HW resources and make related initializations required >> for shared action implementation. >> - make necessary preparations to maintain shared access to >> the action resources, configuration and state. >> API for shared action destroy (see `rte_flow_shared_action_destroy()`) >> should release HW resources and make related cleanups required for shared >> action implementation. >> >> In order to share some flow action reuse the handle of type >> `struct rte_flow_shared_action` returned by >> rte_flow_shared_action_create() as a `conf` field of >> `struct rte_flow_action` (see "example" section). >> >> If some shared action not used by any flow rule all resources allocated >> by the shared action can be released by rte_flow_shared_action_destroy() >> (see "example" section). The shared action handle passed as argument to >> destroy API should not be used any further i.e. result of the usage is >> undefined. May be it makes sense to implement housekeeping in ethdev layer? I.e. guarantee consequency using reference counters etc. Will applications benefit from it? >> >> Shared action re-configuration >> === >> Shared action behavior defined by its configuration can be updated via >> rte_flow_shared_action_update() (see "example" section). The shared >> action update operation modifies HW related resources/objects allocated >> on the action creation. The number of operations performed by the update >> operation should not be dependent on number of flows sharing the related >> action. On return of shared action update API action behavior should be >> according to updated configuration for all flows sharing the action. If shared action is simulated in SW, number of operations to do reconfiguration will depend on a number of flow rules using it. I think it is not a problem. So, *should not* used above is OK. Consider: "should not be dependent on" -> "should not depend on" >> >> Shared action query >> === >> Provide separate API to query shared action sate (see sate -> state >> rte_flow_shared_action_update()). Taking a counter as an example: query >> returns value aggregating all counter increments across all flow rules >> sharing the counter. >> >> PMD support >> === >> The support of introduced API is pure PMD specific design and >> responsibility for each action type (see struct rte_flow_ops). >> >> testpmd >> === >> In order to utilize introduced API testpmd cli may implement following >> extension >> create/update/destroy/query shared action accordingly >> >> flow shared_action create {port_id} [index] {action} >> flow shared_action update {port_id} {index} {action} >> flow shared_action destroy {port_id} {index} >> flow shared_action query {port_id} {index} >> >> testpmd example >> === >> >> configure rss to queues 1 & 2 >> >> testpmd> flow shared_action create 0 100 rss 1 2 >> >> create flow rule utilizing shared action >> >> testpmd> flow create 0 ingress \ >> pattern eth dst is 0c:42:a1:15:fd:ac / ipv6 / tcp / end \ >> actions shared 100 end / end >> >> add 2 more queues >> >> testpmd> flow shared_action modify 0 100 rss 1 2 3 4 >> >> example >> === >> >> struct rte_flow_action actions[2]; >> struct rte_flow_action action; >> /* skipped: initialize action */ >> struct rte_flow_shared_action *handle = rte_flow_shared_action_create( >> port_id, &action, &error); It should be possible to have many actions in shared action. I.e. similar to below, it should be an array terminated by RTE_FLOW_ACTION_TYPE_END. It is unclear from the example above. >> actions[0].type = RTE_FLOW_ACTION_TYPE_SHARED; >> actions[0].conf = handle; >> actions[1].type = RTE_FLOW_ACTION_TYPE_END; >> /* skipped: init attr0 & pattern0 args */ >> struct rte_flow *flow0 = rte_flow_create(port_id, &attr0, pattern0, >> actions, error); >> /* create more rules reusing shared action */ >> struct rte_flow *flow1 = rte_flow_create(port_id, &attr1, pattern1, >> actions, error); >> /* skipped: for flows 2 till N */ >> struct rte_flow *flowN = rte_flow_create(port_id, &attrN, patternN, >> actions, error); >> /* update shared action */ >> struct rte_flow_action updated_action; >> /* >> * skipped: initialize updated_action according to desired action >> * configuration change >> */ >> rte_flow_shared_action_update(port_id, handle, &updated_action, error); >> /* >> * from now on all flows 1 till N will act according to configuration of >> * updated_action >> */ >> /* skipped: destroy all flows 1 till N */ >> rte_flow_shared_action_destroy(port_id, handle, error); >> >> Signed-off-by: Andrey Vesnovaty <andreyv@mellanox.com> >> --- >> lib/librte_ethdev/rte_ethdev_version.map | 6 + >> lib/librte_ethdev/rte_flow.c | 81 +++++++++++++ >> lib/librte_ethdev/rte_flow.h | 148 ++++++++++++++++++++++- >> lib/librte_ethdev/rte_flow_driver.h | 22 ++++ >> 4 files changed, 256 insertions(+), 1 deletion(-) >> >> diff --git a/lib/librte_ethdev/rte_ethdev_version.map b/lib/librte_ethdev/rte_ethdev_version.map >> index 7155056045..119d84976a 100644 >> --- a/lib/librte_ethdev/rte_ethdev_version.map >> +++ b/lib/librte_ethdev/rte_ethdev_version.map >> @@ -241,4 +241,10 @@ EXPERIMENTAL { >> __rte_ethdev_trace_rx_burst; >> __rte_ethdev_trace_tx_burst; >> rte_flow_get_aged_flows; >> + >> + # added in 20.08 >> + rte_flow_shared_action_create; >> + rte_flow_shared_action_destroy; >> + rte_flow_shared_action_update; >> + rte_flow_shared_action_query; >> }; >> diff --git a/lib/librte_ethdev/rte_flow.c b/lib/librte_ethdev/rte_flow.c >> index 1685be5f73..0ac4d31a13 100644 >> --- a/lib/librte_ethdev/rte_flow.c >> +++ b/lib/librte_ethdev/rte_flow.c >> @@ -1250,3 +1250,84 @@ rte_flow_get_aged_flows(uint16_t port_id, void **contexts, >> RTE_FLOW_ERROR_TYPE_UNSPECIFIED, >> NULL, rte_strerror(ENOTSUP)); >> } >> + >> +struct rte_flow_shared_action * >> +rte_flow_shared_action_create(uint16_t port_id, >> + const struct rte_flow_action *action, >> + struct rte_flow_error *error) >> +{ >> + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; >> + struct rte_flow_shared_action *shared_action; >> + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error); >> + >> + if (unlikely(!ops)) >> + return NULL; >> + if (likely(!!ops->shared_action_create)) { >> + shared_action = ops->shared_action_create(dev, action, error); >> + if (shared_action == NULL) >> + flow_err(port_id, -rte_errno, error); >> + return shared_action; >> + } >> + rte_flow_error_set(error, ENOSYS, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, >> + NULL, rte_strerror(ENOSYS)); >> + return NULL; >> +} >> + >> +int >> +rte_flow_shared_action_destroy(uint16_t port_id, >> + struct rte_flow_shared_action *action, >> + struct rte_flow_error *error) >> +{ >> + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; >> + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error); >> + >> + if (unlikely(!ops)) >> + return -rte_errno; >> + if (likely(!!ops->shared_action_destroy)) >> + return flow_err(port_id, >> + ops->shared_action_destroy(dev, action, error), >> + error); >> + return rte_flow_error_set(error, ENOSYS, >> + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, >> + NULL, rte_strerror(ENOSYS)); >> +} >> + >> +int >> +rte_flow_shared_action_update(uint16_t port_id, >> + struct rte_flow_shared_action *action, >> + const struct rte_flow_action *update, >> + struct rte_flow_error *error) >> +{ >> + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; >> + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error); >> + >> + if (unlikely(!ops)) >> + return -rte_errno; >> + if (likely(!!ops->shared_action_update)) >> + return flow_err(port_id, ops->shared_action_update(dev, action, >> + update, error), >> + error); >> + return rte_flow_error_set(error, ENOSYS, >> + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, >> + NULL, rte_strerror(ENOSYS)); >> +} >> + >> +int >> +rte_flow_shared_action_query(uint16_t port_id, >> + const struct rte_flow_shared_action *action, >> + void *data, >> + struct rte_flow_error *error) >> +{ >> + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; >> + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error); >> + >> + if (unlikely(!ops)) >> + return -rte_errno; >> + if (likely(!!ops->shared_action_query)) >> + return flow_err(port_id, ops->shared_action_query(dev, action, >> + data, error), >> + error); >> + return rte_flow_error_set(error, ENOSYS, >> + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, >> + NULL, rte_strerror(ENOSYS)); >> +} >> diff --git a/lib/librte_ethdev/rte_flow.h b/lib/librte_ethdev/rte_flow.h >> index b0e4199192..257456b14a 100644 >> --- a/lib/librte_ethdev/rte_flow.h >> +++ b/lib/librte_ethdev/rte_flow.h >> @@ -1681,7 +1681,8 @@ enum rte_flow_action_type { >> /** >> * Enables counters for this flow rule. >> * >> - * These counters can be retrieved and reset through rte_flow_query(), >> + * These counters can be retrieved and reset through rte_flow_query() or >> + * rte_flow_shared_action_query() if the action provided via handle, >> * see struct rte_flow_query_count. >> * >> * See struct rte_flow_action_count. >> @@ -2099,6 +2100,14 @@ enum rte_flow_action_type { >> * see enum RTE_ETH_EVENT_FLOW_AGED >> */ >> RTE_FLOW_ACTION_TYPE_AGE, >> + >> + /** >> + * Describes action shared a cross multiple flow rules. Both options are possible, but I'd propose to stick to: Describes -> Describe Or even: Use actions shared across multiple flow rules. >> + * >> + * Enables multiple rules reference the same action by handle (see Enables -> Allow >> + * struct rte_flow_shared_action). >> + */ >> + RTE_FLOW_ACTION_TYPE_SHARED, >> }; >> >> /** >> @@ -2660,6 +2669,20 @@ struct rte_flow_action_set_dscp { >> uint8_t dscp; >> }; >> >> + >> +/** >> + * RTE_FLOW_ACTION_TYPE_SHARED >> + * >> + * Opaque type returned after successfully creating a shared action. >> + * >> + * This handle can be used to manage and query the related action: >> + * - share it a cross multiple flow rules >> + * - update action configuration >> + * - query action data >> + * - destroy action >> + */ >> +struct rte_flow_shared_action; >> + >> /* Mbuf dynamic field offset for metadata. */ >> extern int32_t rte_flow_dynf_metadata_offs; >> >> @@ -3324,6 +3347,129 @@ int >> rte_flow_get_aged_flows(uint16_t port_id, void **contexts, >> uint32_t nb_contexts, struct rte_flow_error *error); >> >> +/** >> + * @warning >> + * @b EXPERIMENTAL: this API may change without prior notice. >> + * >> + * Create shared action for reuse in multiple flow rules. >> + * >> + * @param[in] port_id >> + * The port identifier of the Ethernet device. >> + * @param[in] action >> + * Action configuration for shared action creation. >> + * @param[out] error >> + * Perform verbose error reporting if not NULL. PMDs initialize this >> + * structure in case of error only. >> + * @return >> + * A valid handle in case of success, NULL otherwise and rte_errno is set >> + * to one of the error codes defined: >> + * - (ENOSYS) if underlying device does not support this functionality. >> + * - (EIO) if underlying device is removed. >> + * - (EINVAL) if *action* invalid. >> + * - (ENOTSUP) if *action* valid but unsupported. >> + */ >> +__rte_experimental >> +struct rte_flow_shared_action * >> +rte_flow_shared_action_create(uint16_t port_id, >> + const struct rte_flow_action *action, >> + struct rte_flow_error *error); >> + >> +/** >> + * @warning >> + * @b EXPERIMENTAL: this API may change without prior notice. >> + * >> + * Destroys the shared action by handle. Destroys -> Destroy >> + * >> + * @param[in] port_id >> + * The port identifier of the Ethernet device. >> + * @param[in] action >> + * Handle for the shared action to be destroyed. >> + * @param[out] error >> + * Perform verbose error reporting if not NULL. PMDs initialize this >> + * structure in case of error only. >> + * @return >> + * - (0) if success. >> + * - (-ENOSYS) if underlying device does not support this functionality. >> + * - (-EIO) if underlying device is removed. >> + * - (-ENOENT) if action pointed by *action* handle was not found. >> + * - (-ETOOMANYREFS) if action pointed by *action* handle still used by one or >> + * more rules >> + * rte_errno is also set. >> + */ >> +__rte_experimental >> +int >> +rte_flow_shared_action_destroy(uint16_t port_id, >> + struct rte_flow_shared_action *action, >> + struct rte_flow_error *error); >> + >> +/** >> + * @warning >> + * @b EXPERIMENTAL: this API may change without prior notice. >> + * >> + * Updates inplace the shared action configuration pointed by *action* handle Updates -> Update inplace -> in-place >> + * with the configuration provided as *update* argument. >> + * The update of the shared action configuration effects all flow rules reusing May be: reusing -> using >> + * the action via handle. The interesting question what to do, if some rule cannot have a new action (i.e. new action is invalid for a rule). May be it is better to highlight it. >> + * >> + * @param[in] port_id >> + * The port identifier of the Ethernet device. >> + * @param[in] action >> + * Handle for the shared action to be updated. >> + * @param[in] update >> + * Action specification used to modify the action pointed by handle. >> + * *update* should be of same type with the action pointed by the *action* >> + * handle argument, otherwise considered as invalid. Why? If it is a generic rule, it should be checked on a generic ethdev layer, but I would not impose the limitation. If PMD may change it, why generic interface specification should restrict it. >> + * @param[out] error >> + * Perform verbose error reporting if not NULL. PMDs initialize this >> + * structure in case of error only. >> + * @return >> + * - (0) if success. >> + * - (-ENOSYS) if underlying device does not support this functionality. >> + * - (-EIO) if underlying device is removed. >> + * - (-EINVAL) if *update* invalid. >> + * - (-ENOTSUP) if *update* valid but unsupported. >> + * - (-ENOENT) if action pointed by *ctx* was not found. >> + * rte_errno is also set. >> + */ >> +__rte_experimental >> +int >> +rte_flow_shared_action_update(uint16_t port_id, >> + struct rte_flow_shared_action *action, >> + const struct rte_flow_action *update, >> + struct rte_flow_error *error); >> + >> +/** >> + * @warning >> + * @b EXPERIMENTAL: this API may change without prior notice. >> + * >> + * Query the shared action by handle. >> + * >> + * This function allows retrieving action-specific data such as counters. "This function allows retrieving" -> Retrieve or Get or Query >> + * Data is gathered by special action which may be present/referenced in >> + * more than one flow rule definition. >> + * >> + * \see RTE_FLOW_ACTION_TYPE_COUNT >> + * >> + * @param port_id >> + * Port identifier of Ethernet device. >> + * @param[in] action >> + * Handle for the shared action to query. >> + * @param[in, out] data >> + * Pointer to storage for the associated query data type. >> + * @param[out] error >> + * Perform verbose error reporting if not NULL. PMDs initialize this >> + * structure in case of error only. >> + * >> + * @return >> + * 0 on success, a negative errno value otherwise and rte_errno is set. >> + */ >> +__rte_experimental >> +int >> +rte_flow_shared_action_query(uint16_t port_id, >> + const struct rte_flow_shared_action *action, >> + void *data, >> + struct rte_flow_error *error); >> + >> #ifdef __cplusplus >> } >> #endif >> diff --git a/lib/librte_ethdev/rte_flow_driver.h b/lib/librte_ethdev/rte_flow_driver.h >> index 881cc469b7..a2cae1b53c 100644 >> --- a/lib/librte_ethdev/rte_flow_driver.h >> +++ b/lib/librte_ethdev/rte_flow_driver.h >> @@ -107,6 +107,28 @@ struct rte_flow_ops { >> void **context, >> uint32_t nb_contexts, >> struct rte_flow_error *err); >> + /** See rte_flow_shared_action_create() */ >> + struct rte_flow_shared_action *(*shared_action_create) >> + (struct rte_eth_dev *dev, >> + const struct rte_flow_action *action, >> + struct rte_flow_error *error); >> + /** See rte_flow_shared_action_destroy() */ >> + int (*shared_action_destroy) >> + (struct rte_eth_dev *dev, >> + struct rte_flow_shared_action *shared_action, >> + struct rte_flow_error *error); >> + /** See rte_flow_shared_action_update() */ >> + int (*shared_action_update) >> + (struct rte_eth_dev *dev, >> + struct rte_flow_shared_action *shared_action, >> + const struct rte_flow_action *update, >> + struct rte_flow_error *error); >> + /** See rte_flow_shared_action_query() */ >> + int (*shared_action_query) >> + (struct rte_eth_dev *dev, >> + const struct rte_flow_shared_action *shared_action, >> + void *data, >> + struct rte_flow_error *error); >> }; >> >> /** >> > > The modification of "struct rte_flow_ops", looks fine. > > Acked-by: Ray Kinsella <mdr@ashroe.eu> > ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [dpdk-dev] [PATCH v2 1/6] ethdev: add flow shared action API 2020-07-13 8:04 ` [dpdk-dev] [PATCH v2 1/6] ethdev: " Kinsella, Ray 2020-07-13 10:16 ` Andrew Rybchenko @ 2020-07-15 8:54 ` Andrew Rybchenko 2020-07-15 9:00 ` Andrew Rybchenko 2020-09-15 11:30 ` Andrey Vesnovaty 1 sibling, 2 replies; 106+ messages in thread From: Andrew Rybchenko @ 2020-07-15 8:54 UTC (permalink / raw) To: Andrey Vesnovaty, Neil Horman, Thomas Monjalon, Ferruh Yigit, Andrew Rybchenko, Ori Kam Cc: Kinsella, Ray, dev, jerinjacobk, stephen, bruce.richardson, viacheslavo, andrey.vesnovaty On 7/13/20 11:04 AM, Kinsella, Ray wrote: > > > On 08/07/2020 21:40, Andrey Vesnovaty wrote: >> From: Andrey Vesnovaty <andrey.vesnovaty@gmail.com> >> >> This commit introduces extension of DPDK flow action API enabling >> sharing of single rte_flow_action in multiple flows. The API intended for >> PMDs where multiple HW offloaded flows can reuse the same HW >> essence/object representing flow action and modification of such an >> essence/object effects all the rules using it. >> >> Motivation and example >> === >> Adding or removing one or more queues to RSS used by multiple flow rules >> imposes per rule toll for current DPDK flow API; the scenario requires >> for each flow sharing cloned RSS action: >> - call `rte_flow_destroy()` >> - call `rte_flow_create()` with modified RSS action >> >> API for sharing action and its in-place update benefits: >> - reduce the overhead of multiple RSS flow rules reconfiguration It *allows* to reduce the overhead of multiple RSS flow rules reconfiguration. I.e. it is not mandatory. PMD may do it in SW, if HW does not support it. I see no problem to allow it. Even if it is done in PMD, it is already an optimization since applications have unified interface and internally it should be cheaper to do it. I'd consider to implement generic SW helper API for PMDs which cannot have shared actions in HW, but would like to simulate it in SW. It would allow to avoid the fallback in applications. >> - optimize resource utilization by sharing action across of multiple >> flows >> >> Change description >> === >> >> Shared action >> === >> In order to represent flow action shared by multiple flows new action >> type RTE_FLOW_ACTION_TYPE_SHARED is introduced (see `enum >> rte_flow_action_type`). >> Actually the introduced API decouples action from any specific flow and >> enables sharing of single action by its handle across multiple flows. >> >> Shared action create/use/destroy >> === >> Shared action may be reused by some or none flow rules at any given >> moment, i.e. shared action reside outside of the context of any flow. >> Shared action represent HW resources/objects used for action offloading >> implementation. >> API for shared action create (see `rte_flow_shared_action_create()`): >> - should allocate HW resources and make related initializations required >> for shared action implementation. >> - make necessary preparations to maintain shared access to >> the action resources, configuration and state. >> API for shared action destroy (see `rte_flow_shared_action_destroy()`) >> should release HW resources and make related cleanups required for shared >> action implementation. >> >> In order to share some flow action reuse the handle of type >> `struct rte_flow_shared_action` returned by >> rte_flow_shared_action_create() as a `conf` field of >> `struct rte_flow_action` (see "example" section). >> >> If some shared action not used by any flow rule all resources allocated >> by the shared action can be released by rte_flow_shared_action_destroy() >> (see "example" section). The shared action handle passed as argument to >> destroy API should not be used any further i.e. result of the usage is >> undefined. May be it makes sense to implement housekeeping in ethdev layer? I.e. guarantee consequency using reference counters etc. Will applications benefit from it? >> >> Shared action re-configuration >> === >> Shared action behavior defined by its configuration can be updated via >> rte_flow_shared_action_update() (see "example" section). The shared >> action update operation modifies HW related resources/objects allocated >> on the action creation. The number of operations performed by the update >> operation should not be dependent on number of flows sharing the related >> action. On return of shared action update API action behavior should be >> according to updated configuration for all flows sharing the action. If shared action is simulated in SW, number of operations to do reconfiguration will depend on a number of flow rules using it. I think it is not a problem. So, *should not* used above is OK. Consider: "should not be dependent on" -> "should not depend on" >> >> Shared action query >> === >> Provide separate API to query shared action sate (see sate -> state >> rte_flow_shared_action_update()). Taking a counter as an example: query >> returns value aggregating all counter increments across all flow rules >> sharing the counter. >> >> PMD support >> === >> The support of introduced API is pure PMD specific design and >> responsibility for each action type (see struct rte_flow_ops). >> >> testpmd >> === >> In order to utilize introduced API testpmd cli may implement following >> extension >> create/update/destroy/query shared action accordingly >> >> flow shared_action create {port_id} [index] {action} >> flow shared_action update {port_id} {index} {action} >> flow shared_action destroy {port_id} {index} >> flow shared_action query {port_id} {index} >> >> testpmd example >> === >> >> configure rss to queues 1 & 2 >> >> testpmd> flow shared_action create 0 100 rss 1 2 >> >> create flow rule utilizing shared action >> >> testpmd> flow create 0 ingress \ >> pattern eth dst is 0c:42:a1:15:fd:ac / ipv6 / tcp / end \ >> actions shared 100 end / end >> >> add 2 more queues >> >> testpmd> flow shared_action modify 0 100 rss 1 2 3 4 >> >> example >> === >> >> struct rte_flow_action actions[2]; >> struct rte_flow_action action; >> /* skipped: initialize action */ >> struct rte_flow_shared_action *handle = rte_flow_shared_action_create( >> port_id, &action, &error); It should be possible to have many actions in shared action. I.e. similar to below, it should be an array terminated by RTE_FLOW_ACTION_TYPE_END. It is unclear from the example above. >> actions[0].type = RTE_FLOW_ACTION_TYPE_SHARED; >> actions[0].conf = handle; >> actions[1].type = RTE_FLOW_ACTION_TYPE_END; >> /* skipped: init attr0 & pattern0 args */ >> struct rte_flow *flow0 = rte_flow_create(port_id, &attr0, pattern0, >> actions, error); >> /* create more rules reusing shared action */ >> struct rte_flow *flow1 = rte_flow_create(port_id, &attr1, pattern1, >> actions, error); >> /* skipped: for flows 2 till N */ >> struct rte_flow *flowN = rte_flow_create(port_id, &attrN, patternN, >> actions, error); >> /* update shared action */ >> struct rte_flow_action updated_action; >> /* >> * skipped: initialize updated_action according to desired action >> * configuration change >> */ >> rte_flow_shared_action_update(port_id, handle, &updated_action, error); >> /* >> * from now on all flows 1 till N will act according to configuration of >> * updated_action >> */ >> /* skipped: destroy all flows 1 till N */ >> rte_flow_shared_action_destroy(port_id, handle, error); >> >> Signed-off-by: Andrey Vesnovaty <andreyv@mellanox.com> >> --- >> lib/librte_ethdev/rte_ethdev_version.map | 6 + >> lib/librte_ethdev/rte_flow.c | 81 +++++++++++++ >> lib/librte_ethdev/rte_flow.h | 148 ++++++++++++++++++++++- >> lib/librte_ethdev/rte_flow_driver.h | 22 ++++ >> 4 files changed, 256 insertions(+), 1 deletion(-) >> >> diff --git a/lib/librte_ethdev/rte_ethdev_version.map >> b/lib/librte_ethdev/rte_ethdev_version.map >> index 7155056045..119d84976a 100644 >> --- a/lib/librte_ethdev/rte_ethdev_version.map >> +++ b/lib/librte_ethdev/rte_ethdev_version.map >> @@ -241,4 +241,10 @@ EXPERIMENTAL { >> __rte_ethdev_trace_rx_burst; >> __rte_ethdev_trace_tx_burst; >> rte_flow_get_aged_flows; >> + >> + # added in 20.08 >> + rte_flow_shared_action_create; >> + rte_flow_shared_action_destroy; >> + rte_flow_shared_action_update; >> + rte_flow_shared_action_query; >> }; >> diff --git a/lib/librte_ethdev/rte_flow.c b/lib/librte_ethdev/rte_flow.c >> index 1685be5f73..0ac4d31a13 100644 >> --- a/lib/librte_ethdev/rte_flow.c >> +++ b/lib/librte_ethdev/rte_flow.c >> @@ -1250,3 +1250,84 @@ rte_flow_get_aged_flows(uint16_t port_id, void >> **contexts, >> RTE_FLOW_ERROR_TYPE_UNSPECIFIED, >> NULL, rte_strerror(ENOTSUP)); >> } >> + >> +struct rte_flow_shared_action * >> +rte_flow_shared_action_create(uint16_t port_id, >> + const struct rte_flow_action *action, >> + struct rte_flow_error *error) >> +{ >> + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; >> + struct rte_flow_shared_action *shared_action; >> + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error); >> + >> + if (unlikely(!ops)) >> + return NULL; >> + if (likely(!!ops->shared_action_create)) { >> + shared_action = ops->shared_action_create(dev, action, error); >> + if (shared_action == NULL) >> + flow_err(port_id, -rte_errno, error); >> + return shared_action; >> + } >> + rte_flow_error_set(error, ENOSYS, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, >> + NULL, rte_strerror(ENOSYS)); >> + return NULL; >> +} >> + >> +int >> +rte_flow_shared_action_destroy(uint16_t port_id, >> + struct rte_flow_shared_action *action, >> + struct rte_flow_error *error) >> +{ >> + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; >> + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error); >> + >> + if (unlikely(!ops)) >> + return -rte_errno; >> + if (likely(!!ops->shared_action_destroy)) >> + return flow_err(port_id, >> + ops->shared_action_destroy(dev, action, error), >> + error); >> + return rte_flow_error_set(error, ENOSYS, >> + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, >> + NULL, rte_strerror(ENOSYS)); >> +} >> + >> +int >> +rte_flow_shared_action_update(uint16_t port_id, >> + struct rte_flow_shared_action *action, >> + const struct rte_flow_action *update, >> + struct rte_flow_error *error) >> +{ >> + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; >> + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error); >> + >> + if (unlikely(!ops)) >> + return -rte_errno; >> + if (likely(!!ops->shared_action_update)) >> + return flow_err(port_id, ops->shared_action_update(dev, action, >> + update, error), >> + error); >> + return rte_flow_error_set(error, ENOSYS, >> + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, >> + NULL, rte_strerror(ENOSYS)); >> +} >> + >> +int >> +rte_flow_shared_action_query(uint16_t port_id, >> + const struct rte_flow_shared_action *action, >> + void *data, >> + struct rte_flow_error *error) >> +{ >> + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; >> + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error); >> + >> + if (unlikely(!ops)) >> + return -rte_errno; >> + if (likely(!!ops->shared_action_query)) >> + return flow_err(port_id, ops->shared_action_query(dev, action, >> + data, error), >> + error); >> + return rte_flow_error_set(error, ENOSYS, >> + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, >> + NULL, rte_strerror(ENOSYS)); >> +} >> diff --git a/lib/librte_ethdev/rte_flow.h b/lib/librte_ethdev/rte_flow.h >> index b0e4199192..257456b14a 100644 >> --- a/lib/librte_ethdev/rte_flow.h >> +++ b/lib/librte_ethdev/rte_flow.h >> @@ -1681,7 +1681,8 @@ enum rte_flow_action_type { >> /** >> * Enables counters for this flow rule. >> * >> - * These counters can be retrieved and reset through rte_flow_query(), >> + * These counters can be retrieved and reset through rte_flow_query() or >> + * rte_flow_shared_action_query() if the action provided via handle, >> * see struct rte_flow_query_count. >> * >> * See struct rte_flow_action_count. >> @@ -2099,6 +2100,14 @@ enum rte_flow_action_type { >> * see enum RTE_ETH_EVENT_FLOW_AGED >> */ >> RTE_FLOW_ACTION_TYPE_AGE, >> + >> + /** >> + * Describes action shared a cross multiple flow rules. Both options are possible, but I'd propose to stick to: Describes -> Describe Or even: Use actions shared across multiple flow rules. >> + * >> + * Enables multiple rules reference the same action by handle (see Enables -> Allow >> + * struct rte_flow_shared_action). >> + */ >> + RTE_FLOW_ACTION_TYPE_SHARED, >> }; >> /** >> @@ -2660,6 +2669,20 @@ struct rte_flow_action_set_dscp { >> uint8_t dscp; >> }; >> + >> +/** >> + * RTE_FLOW_ACTION_TYPE_SHARED >> + * >> + * Opaque type returned after successfully creating a shared action. >> + * >> + * This handle can be used to manage and query the related action: >> + * - share it a cross multiple flow rules >> + * - update action configuration >> + * - query action data >> + * - destroy action >> + */ >> +struct rte_flow_shared_action; >> + >> /* Mbuf dynamic field offset for metadata. */ >> extern int32_t rte_flow_dynf_metadata_offs; >> @@ -3324,6 +3347,129 @@ int >> rte_flow_get_aged_flows(uint16_t port_id, void **contexts, >> uint32_t nb_contexts, struct rte_flow_error *error); >> +/** >> + * @warning >> + * @b EXPERIMENTAL: this API may change without prior notice. >> + * >> + * Create shared action for reuse in multiple flow rules. >> + * >> + * @param[in] port_id >> + * The port identifier of the Ethernet device. >> + * @param[in] action >> + * Action configuration for shared action creation. >> + * @param[out] error >> + * Perform verbose error reporting if not NULL. PMDs initialize this >> + * structure in case of error only. >> + * @return >> + * A valid handle in case of success, NULL otherwise and rte_errno >> is set >> + * to one of the error codes defined: >> + * - (ENOSYS) if underlying device does not support this functionality. >> + * - (EIO) if underlying device is removed. >> + * - (EINVAL) if *action* invalid. >> + * - (ENOTSUP) if *action* valid but unsupported. >> + */ >> +__rte_experimental >> +struct rte_flow_shared_action * >> +rte_flow_shared_action_create(uint16_t port_id, >> + const struct rte_flow_action *action, >> + struct rte_flow_error *error); >> + >> +/** >> + * @warning >> + * @b EXPERIMENTAL: this API may change without prior notice. >> + * >> + * Destroys the shared action by handle. Destroys -> Destroy >> + * >> + * @param[in] port_id >> + * The port identifier of the Ethernet device. >> + * @param[in] action >> + * Handle for the shared action to be destroyed. >> + * @param[out] error >> + * Perform verbose error reporting if not NULL. PMDs initialize this >> + * structure in case of error only. >> + * @return >> + * - (0) if success. >> + * - (-ENOSYS) if underlying device does not support this functionality. >> + * - (-EIO) if underlying device is removed. >> + * - (-ENOENT) if action pointed by *action* handle was not found. >> + * - (-ETOOMANYREFS) if action pointed by *action* handle still used >> by one or >> + * more rules >> + * rte_errno is also set. >> + */ >> +__rte_experimental >> +int >> +rte_flow_shared_action_destroy(uint16_t port_id, >> + struct rte_flow_shared_action *action, >> + struct rte_flow_error *error); >> + >> +/** >> + * @warning >> + * @b EXPERIMENTAL: this API may change without prior notice. >> + * >> + * Updates inplace the shared action configuration pointed by >> *action* handle Updates -> Update inplace -> in-place >> + * with the configuration provided as *update* argument. >> + * The update of the shared action configuration effects all flow >> rules reusing May be: reusing -> using >> + * the action via handle. The interesting question what to do, if some rule cannot have a new action (i.e. new action is invalid for a rule). May be it is better to highlight it. >> + * >> + * @param[in] port_id >> + * The port identifier of the Ethernet device. >> + * @param[in] action >> + * Handle for the shared action to be updated. >> + * @param[in] update >> + * Action specification used to modify the action pointed by handle. >> + * *update* should be of same type with the action pointed by the >> *action* >> + * handle argument, otherwise considered as invalid. Why? If it is a generic rule, it should be checked on a generic ethdev layer, but I would not impose the limitation. If PMD may change it, why generic interface specification should restrict it. >> + * @param[out] error >> + * Perform verbose error reporting if not NULL. PMDs initialize this >> + * structure in case of error only. >> + * @return >> + * - (0) if success. >> + * - (-ENOSYS) if underlying device does not support this functionality. >> + * - (-EIO) if underlying device is removed. >> + * - (-EINVAL) if *update* invalid. >> + * - (-ENOTSUP) if *update* valid but unsupported. >> + * - (-ENOENT) if action pointed by *ctx* was not found. >> + * rte_errno is also set. >> + */ >> +__rte_experimental >> +int >> +rte_flow_shared_action_update(uint16_t port_id, >> + struct rte_flow_shared_action *action, >> + const struct rte_flow_action *update, >> + struct rte_flow_error *error); >> + >> +/** >> + * @warning >> + * @b EXPERIMENTAL: this API may change without prior notice. >> + * >> + * Query the shared action by handle. >> + * >> + * This function allows retrieving action-specific data such as >> counters. "This function allows retrieving" -> Retrieve or Get or Query >> + * Data is gathered by special action which may be present/referenced in >> + * more than one flow rule definition. >> + * >> + * \see RTE_FLOW_ACTION_TYPE_COUNT >> + * >> + * @param port_id >> + * Port identifier of Ethernet device. >> + * @param[in] action >> + * Handle for the shared action to query. >> + * @param[in, out] data >> + * Pointer to storage for the associated query data type. >> + * @param[out] error >> + * Perform verbose error reporting if not NULL. PMDs initialize this >> + * structure in case of error only. >> + * >> + * @return >> + * 0 on success, a negative errno value otherwise and rte_errno is set. >> + */ >> +__rte_experimental >> +int >> +rte_flow_shared_action_query(uint16_t port_id, >> + const struct rte_flow_shared_action *action, >> + void *data, >> + struct rte_flow_error *error); >> + >> #ifdef __cplusplus >> } >> #endif >> diff --git a/lib/librte_ethdev/rte_flow_driver.h >> b/lib/librte_ethdev/rte_flow_driver.h >> index 881cc469b7..a2cae1b53c 100644 >> --- a/lib/librte_ethdev/rte_flow_driver.h >> +++ b/lib/librte_ethdev/rte_flow_driver.h >> @@ -107,6 +107,28 @@ struct rte_flow_ops { >> void **context, >> uint32_t nb_contexts, >> struct rte_flow_error *err); >> + /** See rte_flow_shared_action_create() */ >> + struct rte_flow_shared_action *(*shared_action_create) >> + (struct rte_eth_dev *dev, >> + const struct rte_flow_action *action, >> + struct rte_flow_error *error); >> + /** See rte_flow_shared_action_destroy() */ >> + int (*shared_action_destroy) >> + (struct rte_eth_dev *dev, >> + struct rte_flow_shared_action *shared_action, >> + struct rte_flow_error *error); >> + /** See rte_flow_shared_action_update() */ >> + int (*shared_action_update) >> + (struct rte_eth_dev *dev, >> + struct rte_flow_shared_action *shared_action, >> + const struct rte_flow_action *update, >> + struct rte_flow_error *error); >> + /** See rte_flow_shared_action_query() */ >> + int (*shared_action_query) >> + (struct rte_eth_dev *dev, >> + const struct rte_flow_shared_action *shared_action, >> + void *data, >> + struct rte_flow_error *error); >> }; >> /** >> > > The modification of "struct rte_flow_ops", looks fine. > Acked-by: Ray Kinsella <mdr@ashroe.eu> > ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [dpdk-dev] [PATCH v2 1/6] ethdev: add flow shared action API 2020-07-15 8:54 ` Andrew Rybchenko @ 2020-07-15 9:00 ` Andrew Rybchenko 2020-09-15 11:30 ` Andrey Vesnovaty 1 sibling, 0 replies; 106+ messages in thread From: Andrew Rybchenko @ 2020-07-15 9:00 UTC (permalink / raw) To: Andrey Vesnovaty, Neil Horman, Thomas Monjalon, Ferruh Yigit, Ori Kam Cc: Kinsella, Ray, dev, jerinjacobk, stephen, bruce.richardson, viacheslavo, andrey.vesnovaty I'm sorry for a duplicate of my message from July, 13. The problem is that I've not found my reply in patchwork and mail thread. Now I found out that I'm replying to Ray's mail and the mail is not in the thread since has invalid message headers. Andrew. On 7/15/20 11:54 AM, Andrew Rybchenko wrote: > On 7/13/20 11:04 AM, Kinsella, Ray wrote: >> >> >> On 08/07/2020 21:40, Andrey Vesnovaty wrote: >>> From: Andrey Vesnovaty <andrey.vesnovaty@gmail.com> >>> >>> This commit introduces extension of DPDK flow action API enabling >>> sharing of single rte_flow_action in multiple flows. The API intended for >>> PMDs where multiple HW offloaded flows can reuse the same HW >>> essence/object representing flow action and modification of such an >>> essence/object effects all the rules using it. >>> >>> Motivation and example >>> === >>> Adding or removing one or more queues to RSS used by multiple flow rules >>> imposes per rule toll for current DPDK flow API; the scenario requires >>> for each flow sharing cloned RSS action: >>> - call `rte_flow_destroy()` >>> - call `rte_flow_create()` with modified RSS action >>> >>> API for sharing action and its in-place update benefits: >>> - reduce the overhead of multiple RSS flow rules reconfiguration > > It *allows* to reduce the overhead of multiple RSS flow rules > reconfiguration. I.e. it is not mandatory. PMD may do it in SW, > if HW does not support it. I see no problem to allow it. > Even if it is done in PMD, it is already an optimization since > applications have unified interface and internally it should > be cheaper to do it. > I'd consider to implement generic SW helper API for PMDs which > cannot have shared actions in HW, but would like to simulate it > in SW. It would allow to avoid the fallback in applications. > >>> - optimize resource utilization by sharing action across of multiple >>> flows >>> >>> Change description >>> === >>> >>> Shared action >>> === >>> In order to represent flow action shared by multiple flows new action >>> type RTE_FLOW_ACTION_TYPE_SHARED is introduced (see `enum >>> rte_flow_action_type`). >>> Actually the introduced API decouples action from any specific flow and >>> enables sharing of single action by its handle across multiple flows. >>> >>> Shared action create/use/destroy >>> === >>> Shared action may be reused by some or none flow rules at any given >>> moment, i.e. shared action reside outside of the context of any flow. >>> Shared action represent HW resources/objects used for action offloading >>> implementation. >>> API for shared action create (see `rte_flow_shared_action_create()`): >>> - should allocate HW resources and make related initializations required >>> for shared action implementation. >>> - make necessary preparations to maintain shared access to >>> the action resources, configuration and state. >>> API for shared action destroy (see `rte_flow_shared_action_destroy()`) >>> should release HW resources and make related cleanups required for shared >>> action implementation. >>> >>> In order to share some flow action reuse the handle of type >>> `struct rte_flow_shared_action` returned by >>> rte_flow_shared_action_create() as a `conf` field of >>> `struct rte_flow_action` (see "example" section). >>> >>> If some shared action not used by any flow rule all resources allocated >>> by the shared action can be released by rte_flow_shared_action_destroy() >>> (see "example" section). The shared action handle passed as argument to >>> destroy API should not be used any further i.e. result of the usage is >>> undefined. > > May be it makes sense to implement housekeeping in ethdev > layer? I.e. guarantee consequency using reference counters etc. > Will applications benefit from it? > >>> >>> Shared action re-configuration >>> === >>> Shared action behavior defined by its configuration can be updated via >>> rte_flow_shared_action_update() (see "example" section). The shared >>> action update operation modifies HW related resources/objects allocated >>> on the action creation. The number of operations performed by the update >>> operation should not be dependent on number of flows sharing the related >>> action. On return of shared action update API action behavior should be >>> according to updated configuration for all flows sharing the action. > > If shared action is simulated in SW, number of operations to do > reconfiguration will depend on a number of flow rules using it. > I think it is not a problem. So, *should not* used above is OK. > > Consider: > "should not be dependent on" -> "should not depend on" > >>> >>> Shared action query >>> === >>> Provide separate API to query shared action sate (see > > sate -> state > >>> rte_flow_shared_action_update()). Taking a counter as an example: query >>> returns value aggregating all counter increments across all flow rules >>> sharing the counter. >>> >>> PMD support >>> === >>> The support of introduced API is pure PMD specific design and >>> responsibility for each action type (see struct rte_flow_ops). >>> >>> testpmd >>> === >>> In order to utilize introduced API testpmd cli may implement following >>> extension >>> create/update/destroy/query shared action accordingly >>> >>> flow shared_action create {port_id} [index] {action} >>> flow shared_action update {port_id} {index} {action} >>> flow shared_action destroy {port_id} {index} >>> flow shared_action query {port_id} {index} >>> >>> testpmd example >>> === >>> >>> configure rss to queues 1 & 2 >>> >>> testpmd> flow shared_action create 0 100 rss 1 2 >>> >>> create flow rule utilizing shared action >>> >>> testpmd> flow create 0 ingress \ >>> pattern eth dst is 0c:42:a1:15:fd:ac / ipv6 / tcp / end \ >>> actions shared 100 end / end >>> >>> add 2 more queues >>> >>> testpmd> flow shared_action modify 0 100 rss 1 2 3 4 >>> >>> example >>> === >>> >>> struct rte_flow_action actions[2]; >>> struct rte_flow_action action; >>> /* skipped: initialize action */ >>> struct rte_flow_shared_action *handle = rte_flow_shared_action_create( >>> port_id, &action, &error); > > It should be possible to have many actions in shared action. > I.e. similar to below, it should be an array terminated by > RTE_FLOW_ACTION_TYPE_END. It is unclear from the example > above. > >>> actions[0].type = RTE_FLOW_ACTION_TYPE_SHARED; >>> actions[0].conf = handle; >>> actions[1].type = RTE_FLOW_ACTION_TYPE_END; >>> /* skipped: init attr0 & pattern0 args */ >>> struct rte_flow *flow0 = rte_flow_create(port_id, &attr0, pattern0, >>> actions, error); >>> /* create more rules reusing shared action */ >>> struct rte_flow *flow1 = rte_flow_create(port_id, &attr1, pattern1, >>> actions, error); >>> /* skipped: for flows 2 till N */ >>> struct rte_flow *flowN = rte_flow_create(port_id, &attrN, patternN, >>> actions, error); >>> /* update shared action */ >>> struct rte_flow_action updated_action; >>> /* >>> * skipped: initialize updated_action according to desired action >>> * configuration change >>> */ >>> rte_flow_shared_action_update(port_id, handle, &updated_action, error); >>> /* >>> * from now on all flows 1 till N will act according to configuration of >>> * updated_action >>> */ >>> /* skipped: destroy all flows 1 till N */ >>> rte_flow_shared_action_destroy(port_id, handle, error); >>> >>> Signed-off-by: Andrey Vesnovaty <andreyv@mellanox.com> >>> --- >>> lib/librte_ethdev/rte_ethdev_version.map | 6 + >>> lib/librte_ethdev/rte_flow.c | 81 +++++++++++++ >>> lib/librte_ethdev/rte_flow.h | 148 ++++++++++++++++++++++- >>> lib/librte_ethdev/rte_flow_driver.h | 22 ++++ >>> 4 files changed, 256 insertions(+), 1 deletion(-) >>> >>> diff --git a/lib/librte_ethdev/rte_ethdev_version.map >>> b/lib/librte_ethdev/rte_ethdev_version.map >>> index 7155056045..119d84976a 100644 >>> --- a/lib/librte_ethdev/rte_ethdev_version.map >>> +++ b/lib/librte_ethdev/rte_ethdev_version.map >>> @@ -241,4 +241,10 @@ EXPERIMENTAL { >>> __rte_ethdev_trace_rx_burst; >>> __rte_ethdev_trace_tx_burst; >>> rte_flow_get_aged_flows; >>> + >>> + # added in 20.08 >>> + rte_flow_shared_action_create; >>> + rte_flow_shared_action_destroy; >>> + rte_flow_shared_action_update; >>> + rte_flow_shared_action_query; >>> }; >>> diff --git a/lib/librte_ethdev/rte_flow.c b/lib/librte_ethdev/rte_flow.c >>> index 1685be5f73..0ac4d31a13 100644 >>> --- a/lib/librte_ethdev/rte_flow.c >>> +++ b/lib/librte_ethdev/rte_flow.c >>> @@ -1250,3 +1250,84 @@ rte_flow_get_aged_flows(uint16_t port_id, void >>> **contexts, >>> RTE_FLOW_ERROR_TYPE_UNSPECIFIED, >>> NULL, rte_strerror(ENOTSUP)); >>> } >>> + >>> +struct rte_flow_shared_action * >>> +rte_flow_shared_action_create(uint16_t port_id, >>> + const struct rte_flow_action *action, >>> + struct rte_flow_error *error) >>> +{ >>> + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; >>> + struct rte_flow_shared_action *shared_action; >>> + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error); >>> + >>> + if (unlikely(!ops)) >>> + return NULL; >>> + if (likely(!!ops->shared_action_create)) { >>> + shared_action = ops->shared_action_create(dev, action, error); >>> + if (shared_action == NULL) >>> + flow_err(port_id, -rte_errno, error); >>> + return shared_action; >>> + } >>> + rte_flow_error_set(error, ENOSYS, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, >>> + NULL, rte_strerror(ENOSYS)); >>> + return NULL; >>> +} >>> + >>> +int >>> +rte_flow_shared_action_destroy(uint16_t port_id, >>> + struct rte_flow_shared_action *action, >>> + struct rte_flow_error *error) >>> +{ >>> + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; >>> + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error); >>> + >>> + if (unlikely(!ops)) >>> + return -rte_errno; >>> + if (likely(!!ops->shared_action_destroy)) >>> + return flow_err(port_id, >>> + ops->shared_action_destroy(dev, action, error), >>> + error); >>> + return rte_flow_error_set(error, ENOSYS, >>> + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, >>> + NULL, rte_strerror(ENOSYS)); >>> +} >>> + >>> +int >>> +rte_flow_shared_action_update(uint16_t port_id, >>> + struct rte_flow_shared_action *action, >>> + const struct rte_flow_action *update, >>> + struct rte_flow_error *error) >>> +{ >>> + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; >>> + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error); >>> + >>> + if (unlikely(!ops)) >>> + return -rte_errno; >>> + if (likely(!!ops->shared_action_update)) >>> + return flow_err(port_id, ops->shared_action_update(dev, action, >>> + update, error), >>> + error); >>> + return rte_flow_error_set(error, ENOSYS, >>> + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, >>> + NULL, rte_strerror(ENOSYS)); >>> +} >>> + >>> +int >>> +rte_flow_shared_action_query(uint16_t port_id, >>> + const struct rte_flow_shared_action *action, >>> + void *data, >>> + struct rte_flow_error *error) >>> +{ >>> + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; >>> + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error); >>> + >>> + if (unlikely(!ops)) >>> + return -rte_errno; >>> + if (likely(!!ops->shared_action_query)) >>> + return flow_err(port_id, ops->shared_action_query(dev, action, >>> + data, error), >>> + error); >>> + return rte_flow_error_set(error, ENOSYS, >>> + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, >>> + NULL, rte_strerror(ENOSYS)); >>> +} >>> diff --git a/lib/librte_ethdev/rte_flow.h b/lib/librte_ethdev/rte_flow.h >>> index b0e4199192..257456b14a 100644 >>> --- a/lib/librte_ethdev/rte_flow.h >>> +++ b/lib/librte_ethdev/rte_flow.h >>> @@ -1681,7 +1681,8 @@ enum rte_flow_action_type { >>> /** >>> * Enables counters for this flow rule. >>> * >>> - * These counters can be retrieved and reset through rte_flow_query(), >>> + * These counters can be retrieved and reset through rte_flow_query() or >>> + * rte_flow_shared_action_query() if the action provided via handle, >>> * see struct rte_flow_query_count. >>> * >>> * See struct rte_flow_action_count. >>> @@ -2099,6 +2100,14 @@ enum rte_flow_action_type { >>> * see enum RTE_ETH_EVENT_FLOW_AGED >>> */ >>> RTE_FLOW_ACTION_TYPE_AGE, >>> + >>> + /** >>> + * Describes action shared a cross multiple flow rules. > > Both options are possible, but I'd propose to stick to: > Describes -> Describe > > Or even: > Use actions shared across multiple flow rules. > >>> + * >>> + * Enables multiple rules reference the same action by handle (see > > Enables -> Allow > >>> + * struct rte_flow_shared_action). >>> + */ >>> + RTE_FLOW_ACTION_TYPE_SHARED, >>> }; >>> /** >>> @@ -2660,6 +2669,20 @@ struct rte_flow_action_set_dscp { >>> uint8_t dscp; >>> }; >>> + >>> +/** >>> + * RTE_FLOW_ACTION_TYPE_SHARED >>> + * >>> + * Opaque type returned after successfully creating a shared action. >>> + * >>> + * This handle can be used to manage and query the related action: >>> + * - share it a cross multiple flow rules >>> + * - update action configuration >>> + * - query action data >>> + * - destroy action >>> + */ >>> +struct rte_flow_shared_action; >>> + >>> /* Mbuf dynamic field offset for metadata. */ >>> extern int32_t rte_flow_dynf_metadata_offs; >>> @@ -3324,6 +3347,129 @@ int >>> rte_flow_get_aged_flows(uint16_t port_id, void **contexts, >>> uint32_t nb_contexts, struct rte_flow_error *error); >>> +/** >>> + * @warning >>> + * @b EXPERIMENTAL: this API may change without prior notice. >>> + * >>> + * Create shared action for reuse in multiple flow rules. >>> + * >>> + * @param[in] port_id >>> + * The port identifier of the Ethernet device. >>> + * @param[in] action >>> + * Action configuration for shared action creation. >>> + * @param[out] error >>> + * Perform verbose error reporting if not NULL. PMDs initialize this >>> + * structure in case of error only. >>> + * @return >>> + * A valid handle in case of success, NULL otherwise and rte_errno >>> is set >>> + * to one of the error codes defined: >>> + * - (ENOSYS) if underlying device does not support this functionality. >>> + * - (EIO) if underlying device is removed. >>> + * - (EINVAL) if *action* invalid. >>> + * - (ENOTSUP) if *action* valid but unsupported. >>> + */ >>> +__rte_experimental >>> +struct rte_flow_shared_action * >>> +rte_flow_shared_action_create(uint16_t port_id, >>> + const struct rte_flow_action *action, >>> + struct rte_flow_error *error); >>> + >>> +/** >>> + * @warning >>> + * @b EXPERIMENTAL: this API may change without prior notice. >>> + * >>> + * Destroys the shared action by handle. > > Destroys -> Destroy > >>> + * >>> + * @param[in] port_id >>> + * The port identifier of the Ethernet device. >>> + * @param[in] action >>> + * Handle for the shared action to be destroyed. >>> + * @param[out] error >>> + * Perform verbose error reporting if not NULL. PMDs initialize this >>> + * structure in case of error only. >>> + * @return >>> + * - (0) if success. >>> + * - (-ENOSYS) if underlying device does not support this functionality. >>> + * - (-EIO) if underlying device is removed. >>> + * - (-ENOENT) if action pointed by *action* handle was not found. >>> + * - (-ETOOMANYREFS) if action pointed by *action* handle still used >>> by one or >>> + * more rules >>> + * rte_errno is also set. >>> + */ >>> +__rte_experimental >>> +int >>> +rte_flow_shared_action_destroy(uint16_t port_id, >>> + struct rte_flow_shared_action *action, >>> + struct rte_flow_error *error); >>> + >>> +/** >>> + * @warning >>> + * @b EXPERIMENTAL: this API may change without prior notice. >>> + * >>> + * Updates inplace the shared action configuration pointed by >>> *action* handle > > Updates -> Update > inplace -> in-place > >>> + * with the configuration provided as *update* argument. >>> + * The update of the shared action configuration effects all flow >>> rules reusing > > May be: reusing -> using > >>> + * the action via handle. > > The interesting question what to do, if some rule cannot have a > new action (i.e. new action is invalid for a rule). > May be it is better to highlight it. > >>> + * >>> + * @param[in] port_id >>> + * The port identifier of the Ethernet device. >>> + * @param[in] action >>> + * Handle for the shared action to be updated. >>> + * @param[in] update >>> + * Action specification used to modify the action pointed by handle. >>> + * *update* should be of same type with the action pointed by the >>> *action* >>> + * handle argument, otherwise considered as invalid. > > Why? If it is a generic rule, it should be checked on a generic > ethdev layer, but I would not impose the limitation. If PMD > may change it, why generic interface specification should > restrict it. > >>> + * @param[out] error >>> + * Perform verbose error reporting if not NULL. PMDs initialize this >>> + * structure in case of error only. >>> + * @return >>> + * - (0) if success. >>> + * - (-ENOSYS) if underlying device does not support this functionality. >>> + * - (-EIO) if underlying device is removed. >>> + * - (-EINVAL) if *update* invalid. >>> + * - (-ENOTSUP) if *update* valid but unsupported. >>> + * - (-ENOENT) if action pointed by *ctx* was not found. >>> + * rte_errno is also set. >>> + */ >>> +__rte_experimental >>> +int >>> +rte_flow_shared_action_update(uint16_t port_id, >>> + struct rte_flow_shared_action *action, >>> + const struct rte_flow_action *update, >>> + struct rte_flow_error *error); >>> + >>> +/** >>> + * @warning >>> + * @b EXPERIMENTAL: this API may change without prior notice. >>> + * >>> + * Query the shared action by handle. >>> + * >>> + * This function allows retrieving action-specific data such as >>> counters. > > "This function allows retrieving" -> Retrieve or Get or Query > >>> + * Data is gathered by special action which may be present/referenced in >>> + * more than one flow rule definition. >>> + * >>> + * \see RTE_FLOW_ACTION_TYPE_COUNT >>> + * >>> + * @param port_id >>> + * Port identifier of Ethernet device. >>> + * @param[in] action >>> + * Handle for the shared action to query. >>> + * @param[in, out] data >>> + * Pointer to storage for the associated query data type. >>> + * @param[out] error >>> + * Perform verbose error reporting if not NULL. PMDs initialize this >>> + * structure in case of error only. >>> + * >>> + * @return >>> + * 0 on success, a negative errno value otherwise and rte_errno is set. >>> + */ >>> +__rte_experimental >>> +int >>> +rte_flow_shared_action_query(uint16_t port_id, >>> + const struct rte_flow_shared_action *action, >>> + void *data, >>> + struct rte_flow_error *error); >>> + >>> #ifdef __cplusplus >>> } >>> #endif >>> diff --git a/lib/librte_ethdev/rte_flow_driver.h >>> b/lib/librte_ethdev/rte_flow_driver.h >>> index 881cc469b7..a2cae1b53c 100644 >>> --- a/lib/librte_ethdev/rte_flow_driver.h >>> +++ b/lib/librte_ethdev/rte_flow_driver.h >>> @@ -107,6 +107,28 @@ struct rte_flow_ops { >>> void **context, >>> uint32_t nb_contexts, >>> struct rte_flow_error *err); >>> + /** See rte_flow_shared_action_create() */ >>> + struct rte_flow_shared_action *(*shared_action_create) >>> + (struct rte_eth_dev *dev, >>> + const struct rte_flow_action *action, >>> + struct rte_flow_error *error); >>> + /** See rte_flow_shared_action_destroy() */ >>> + int (*shared_action_destroy) >>> + (struct rte_eth_dev *dev, >>> + struct rte_flow_shared_action *shared_action, >>> + struct rte_flow_error *error); >>> + /** See rte_flow_shared_action_update() */ >>> + int (*shared_action_update) >>> + (struct rte_eth_dev *dev, >>> + struct rte_flow_shared_action *shared_action, >>> + const struct rte_flow_action *update, >>> + struct rte_flow_error *error); >>> + /** See rte_flow_shared_action_query() */ >>> + int (*shared_action_query) >>> + (struct rte_eth_dev *dev, >>> + const struct rte_flow_shared_action *shared_action, >>> + void *data, >>> + struct rte_flow_error *error); >>> }; >>> /** >>> >> >> The modification of "struct rte_flow_ops", looks fine. >> Acked-by: Ray Kinsella <mdr@ashroe.eu> >> > ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [dpdk-dev] [PATCH v2 1/6] ethdev: add flow shared action API 2020-07-15 8:54 ` Andrew Rybchenko 2020-07-15 9:00 ` Andrew Rybchenko @ 2020-09-15 11:30 ` Andrey Vesnovaty 1 sibling, 0 replies; 106+ messages in thread From: Andrey Vesnovaty @ 2020-09-15 11:30 UTC (permalink / raw) To: Andrew Rybchenko, Neil Horman, NBU-Contact-Thomas Monjalon, Ferruh Yigit, jerinj, Ori Kam, Slava Ovsiienko Cc: Kinsella, Ray, dev, jerinjacobk, stephen, bruce.richardson Hi, Andrew, I'm back to this patch set to make it into 20.11. Most of the items you raised regarding SW implementation already discussed with Jerin [1] and answered by Ori [2] and me [3]. Please follow my & Ori answer. In general, the idea of SW implementation for shared action looks nice from application perspective but the changes required in ethdev/rte_flow generic layer about to impact existing rte_flow APIs. Please follow my [3] & Ori's [2] answers & update on your suggestion to the Proposed API. I'll take your suggestions for rephrasing in upcoming patches version (V3). Thanks, Andrey --- [1] https://www.mail-archive.com/dev@dpdk.org/msg173461.html [2] https://www.mail-archive.com/dev@dpdk.org/msg173478.html [3] https://www.mail-archive.com/dev@dpdk.org/msg172921.html > -----Original Message----- > From: Andrew Rybchenko <arybchenko@solarflare.com> > Sent: Wednesday, July 15, 2020 11:54 AM > To: Andrey Vesnovaty <andreyv@mellanox.com>; Neil Horman > <nhorman@tuxdriver.com>; Thomas Monjalon <thomas@monjalon.net>; > Ferruh Yigit <ferruh.yigit@intel.com>; Andrew Rybchenko > <arybchenko@solarflare.com>; Ori Kam <orika@mellanox.com> > Cc: Kinsella, Ray <mdr@ashroe.eu>; dev@dpdk.org; jerinjacobk@gmail.com; > stephen@networkplumber.org; bruce.richardson@intel.com; Slava Ovsiienko > <viacheslavo@mellanox.com>; andrey.vesnovaty@gmail.com > Subject: Re: [dpdk-dev] [PATCH v2 1/6] ethdev: add flow shared action API > Importance: High > > On 7/13/20 11:04 AM, Kinsella, Ray wrote: > > > > > > On 08/07/2020 21:40, Andrey Vesnovaty wrote: > >> From: Andrey Vesnovaty <andrey.vesnovaty@gmail.com> > >> > >> This commit introduces extension of DPDK flow action API enabling > >> sharing of single rte_flow_action in multiple flows. The API intended for > >> PMDs where multiple HW offloaded flows can reuse the same HW > >> essence/object representing flow action and modification of such an > >> essence/object effects all the rules using it. > >> > >> Motivation and example > >> === > >> Adding or removing one or more queues to RSS used by multiple flow rules > >> imposes per rule toll for current DPDK flow API; the scenario requires > >> for each flow sharing cloned RSS action: > >> - call `rte_flow_destroy()` > >> - call `rte_flow_create()` with modified RSS action > >> > >> API for sharing action and its in-place update benefits: > >> - reduce the overhead of multiple RSS flow rules reconfiguration > > It *allows* to reduce the overhead of multiple RSS flow rules > reconfiguration. I.e. it is not mandatory. PMD may do it in SW, > if HW does not support it. I see no problem to allow it. > Even if it is done in PMD, it is already an optimization since > applications have unified interface and internally it should > be cheaper to do it. > I'd consider to implement generic SW helper API for PMDs which > cannot have shared actions in HW, but would like to simulate it > in SW. It would allow to avoid the fallback in applications. > > >> - optimize resource utilization by sharing action across of multiple > >> flows > >> > >> Change description > >> === > >> > >> Shared action > >> === > >> In order to represent flow action shared by multiple flows new action > >> type RTE_FLOW_ACTION_TYPE_SHARED is introduced (see `enum > >> rte_flow_action_type`). > >> Actually the introduced API decouples action from any specific flow and > >> enables sharing of single action by its handle across multiple flows. > >> > >> Shared action create/use/destroy > >> === > >> Shared action may be reused by some or none flow rules at any given > >> moment, i.e. shared action reside outside of the context of any flow. > >> Shared action represent HW resources/objects used for action offloading > >> implementation. > >> API for shared action create (see `rte_flow_shared_action_create()`): > >> - should allocate HW resources and make related initializations required > >> for shared action implementation. > >> - make necessary preparations to maintain shared access to > >> the action resources, configuration and state. > >> API for shared action destroy (see `rte_flow_shared_action_destroy()`) > >> should release HW resources and make related cleanups required for shared > >> action implementation. > >> > >> In order to share some flow action reuse the handle of type > >> `struct rte_flow_shared_action` returned by > >> rte_flow_shared_action_create() as a `conf` field of > >> `struct rte_flow_action` (see "example" section). > >> > >> If some shared action not used by any flow rule all resources allocated > >> by the shared action can be released by rte_flow_shared_action_destroy() > >> (see "example" section). The shared action handle passed as argument to > >> destroy API should not be used any further i.e. result of the usage is > >> undefined. > > May be it makes sense to implement housekeeping in ethdev > layer? I.e. guarantee consequency using reference counters etc. > Will applications benefit from it? > > >> > >> Shared action re-configuration > >> === > >> Shared action behavior defined by its configuration can be updated via > >> rte_flow_shared_action_update() (see "example" section). The shared > >> action update operation modifies HW related resources/objects allocated > >> on the action creation. The number of operations performed by the update > >> operation should not be dependent on number of flows sharing the related > >> action. On return of shared action update API action behavior should be > >> according to updated configuration for all flows sharing the action. > > If shared action is simulated in SW, number of operations to do > reconfiguration will depend on a number of flow rules using it. > I think it is not a problem. So, *should not* used above is OK. > > Consider: > "should not be dependent on" -> "should not depend on" > > >> > >> Shared action query > >> === > >> Provide separate API to query shared action sate (see > > sate -> state > > >> rte_flow_shared_action_update()). Taking a counter as an example: query > >> returns value aggregating all counter increments across all flow rules > >> sharing the counter. > >> > >> PMD support > >> === > >> The support of introduced API is pure PMD specific design and > >> responsibility for each action type (see struct rte_flow_ops). > >> > >> testpmd > >> === > >> In order to utilize introduced API testpmd cli may implement following > >> extension > >> create/update/destroy/query shared action accordingly > >> > >> flow shared_action create {port_id} [index] {action} > >> flow shared_action update {port_id} {index} {action} > >> flow shared_action destroy {port_id} {index} > >> flow shared_action query {port_id} {index} > >> > >> testpmd example > >> === > >> > >> configure rss to queues 1 & 2 > >> > >> testpmd> flow shared_action create 0 100 rss 1 2 > >> > >> create flow rule utilizing shared action > >> > >> testpmd> flow create 0 ingress \ > >> pattern eth dst is 0c:42:a1:15:fd:ac / ipv6 / tcp / end \ > >> actions shared 100 end / end > >> > >> add 2 more queues > >> > >> testpmd> flow shared_action modify 0 100 rss 1 2 3 4 > >> > >> example > >> === > >> > >> struct rte_flow_action actions[2]; > >> struct rte_flow_action action; > >> /* skipped: initialize action */ > >> struct rte_flow_shared_action *handle = rte_flow_shared_action_create( > >> port_id, &action, &error); > > It should be possible to have many actions in shared action. > I.e. similar to below, it should be an array terminated by > RTE_FLOW_ACTION_TYPE_END. It is unclear from the example > above. > > >> actions[0].type = RTE_FLOW_ACTION_TYPE_SHARED; > >> actions[0].conf = handle; > >> actions[1].type = RTE_FLOW_ACTION_TYPE_END; > >> /* skipped: init attr0 & pattern0 args */ > >> struct rte_flow *flow0 = rte_flow_create(port_id, &attr0, pattern0, > >> actions, error); > >> /* create more rules reusing shared action */ > >> struct rte_flow *flow1 = rte_flow_create(port_id, &attr1, pattern1, > >> actions, error); > >> /* skipped: for flows 2 till N */ > >> struct rte_flow *flowN = rte_flow_create(port_id, &attrN, patternN, > >> actions, error); > >> /* update shared action */ > >> struct rte_flow_action updated_action; > >> /* > >> * skipped: initialize updated_action according to desired action > >> * configuration change > >> */ > >> rte_flow_shared_action_update(port_id, handle, &updated_action, error); > >> /* > >> * from now on all flows 1 till N will act according to configuration of > >> * updated_action > >> */ > >> /* skipped: destroy all flows 1 till N */ > >> rte_flow_shared_action_destroy(port_id, handle, error); > >> > >> Signed-off-by: Andrey Vesnovaty <andreyv@mellanox.com> > >> --- > >> lib/librte_ethdev/rte_ethdev_version.map | 6 + > >> lib/librte_ethdev/rte_flow.c | 81 +++++++++++++ > >> lib/librte_ethdev/rte_flow.h | 148 ++++++++++++++++++++++- > >> lib/librte_ethdev/rte_flow_driver.h | 22 ++++ > >> 4 files changed, 256 insertions(+), 1 deletion(-) > >> > >> diff --git a/lib/librte_ethdev/rte_ethdev_version.map > >> b/lib/librte_ethdev/rte_ethdev_version.map > >> index 7155056045..119d84976a 100644 > >> --- a/lib/librte_ethdev/rte_ethdev_version.map > >> +++ b/lib/librte_ethdev/rte_ethdev_version.map > >> @@ -241,4 +241,10 @@ EXPERIMENTAL { > >> __rte_ethdev_trace_rx_burst; > >> __rte_ethdev_trace_tx_burst; > >> rte_flow_get_aged_flows; > >> + > >> + # added in 20.08 > >> + rte_flow_shared_action_create; > >> + rte_flow_shared_action_destroy; > >> + rte_flow_shared_action_update; > >> + rte_flow_shared_action_query; > >> }; > >> diff --git a/lib/librte_ethdev/rte_flow.c b/lib/librte_ethdev/rte_flow.c > >> index 1685be5f73..0ac4d31a13 100644 > >> --- a/lib/librte_ethdev/rte_flow.c > >> +++ b/lib/librte_ethdev/rte_flow.c > >> @@ -1250,3 +1250,84 @@ rte_flow_get_aged_flows(uint16_t port_id, void > >> **contexts, > >> RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > >> NULL, rte_strerror(ENOTSUP)); > >> } > >> + > >> +struct rte_flow_shared_action * > >> +rte_flow_shared_action_create(uint16_t port_id, > >> + const struct rte_flow_action *action, > >> + struct rte_flow_error *error) > >> +{ > >> + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; > >> + struct rte_flow_shared_action *shared_action; > >> + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error); > >> + > >> + if (unlikely(!ops)) > >> + return NULL; > >> + if (likely(!!ops->shared_action_create)) { > >> + shared_action = ops->shared_action_create(dev, action, error); > >> + if (shared_action == NULL) > >> + flow_err(port_id, -rte_errno, error); > >> + return shared_action; > >> + } > >> + rte_flow_error_set(error, ENOSYS, > RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > >> + NULL, rte_strerror(ENOSYS)); > >> + return NULL; > >> +} > >> + > >> +int > >> +rte_flow_shared_action_destroy(uint16_t port_id, > >> + struct rte_flow_shared_action *action, > >> + struct rte_flow_error *error) > >> +{ > >> + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; > >> + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error); > >> + > >> + if (unlikely(!ops)) > >> + return -rte_errno; > >> + if (likely(!!ops->shared_action_destroy)) > >> + return flow_err(port_id, > >> + ops->shared_action_destroy(dev, action, error), > >> + error); > >> + return rte_flow_error_set(error, ENOSYS, > >> + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > >> + NULL, rte_strerror(ENOSYS)); > >> +} > >> + > >> +int > >> +rte_flow_shared_action_update(uint16_t port_id, > >> + struct rte_flow_shared_action *action, > >> + const struct rte_flow_action *update, > >> + struct rte_flow_error *error) > >> +{ > >> + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; > >> + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error); > >> + > >> + if (unlikely(!ops)) > >> + return -rte_errno; > >> + if (likely(!!ops->shared_action_update)) > >> + return flow_err(port_id, ops->shared_action_update(dev, action, > >> + update, error), > >> + error); > >> + return rte_flow_error_set(error, ENOSYS, > >> + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > >> + NULL, rte_strerror(ENOSYS)); > >> +} > >> + > >> +int > >> +rte_flow_shared_action_query(uint16_t port_id, > >> + const struct rte_flow_shared_action *action, > >> + void *data, > >> + struct rte_flow_error *error) > >> +{ > >> + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; > >> + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error); > >> + > >> + if (unlikely(!ops)) > >> + return -rte_errno; > >> + if (likely(!!ops->shared_action_query)) > >> + return flow_err(port_id, ops->shared_action_query(dev, action, > >> + data, error), > >> + error); > >> + return rte_flow_error_set(error, ENOSYS, > >> + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > >> + NULL, rte_strerror(ENOSYS)); > >> +} > >> diff --git a/lib/librte_ethdev/rte_flow.h b/lib/librte_ethdev/rte_flow.h > >> index b0e4199192..257456b14a 100644 > >> --- a/lib/librte_ethdev/rte_flow.h > >> +++ b/lib/librte_ethdev/rte_flow.h > >> @@ -1681,7 +1681,8 @@ enum rte_flow_action_type { > >> /** > >> * Enables counters for this flow rule. > >> * > >> - * These counters can be retrieved and reset through rte_flow_query(), > >> + * These counters can be retrieved and reset through rte_flow_query() or > >> + * rte_flow_shared_action_query() if the action provided via handle, > >> * see struct rte_flow_query_count. > >> * > >> * See struct rte_flow_action_count. > >> @@ -2099,6 +2100,14 @@ enum rte_flow_action_type { > >> * see enum RTE_ETH_EVENT_FLOW_AGED > >> */ > >> RTE_FLOW_ACTION_TYPE_AGE, > >> + > >> + /** > >> + * Describes action shared a cross multiple flow rules. > > Both options are possible, but I'd propose to stick to: > Describes -> Describe > > Or even: > Use actions shared across multiple flow rules. > > >> + * > >> + * Enables multiple rules reference the same action by handle (see > > Enables -> Allow > > >> + * struct rte_flow_shared_action). > >> + */ > >> + RTE_FLOW_ACTION_TYPE_SHARED, > >> }; > >> /** > >> @@ -2660,6 +2669,20 @@ struct rte_flow_action_set_dscp { > >> uint8_t dscp; > >> }; > >> + > >> +/** > >> + * RTE_FLOW_ACTION_TYPE_SHARED > >> + * > >> + * Opaque type returned after successfully creating a shared action. > >> + * > >> + * This handle can be used to manage and query the related action: > >> + * - share it a cross multiple flow rules > >> + * - update action configuration > >> + * - query action data > >> + * - destroy action > >> + */ > >> +struct rte_flow_shared_action; > >> + > >> /* Mbuf dynamic field offset for metadata. */ > >> extern int32_t rte_flow_dynf_metadata_offs; > >> @@ -3324,6 +3347,129 @@ int > >> rte_flow_get_aged_flows(uint16_t port_id, void **contexts, > >> uint32_t nb_contexts, struct rte_flow_error *error); > >> +/** > >> + * @warning > >> + * @b EXPERIMENTAL: this API may change without prior notice. > >> + * > >> + * Create shared action for reuse in multiple flow rules. > >> + * > >> + * @param[in] port_id > >> + * The port identifier of the Ethernet device. > >> + * @param[in] action > >> + * Action configuration for shared action creation. > >> + * @param[out] error > >> + * Perform verbose error reporting if not NULL. PMDs initialize this > >> + * structure in case of error only. > >> + * @return > >> + * A valid handle in case of success, NULL otherwise and rte_errno > >> is set > >> + * to one of the error codes defined: > >> + * - (ENOSYS) if underlying device does not support this functionality. > >> + * - (EIO) if underlying device is removed. > >> + * - (EINVAL) if *action* invalid. > >> + * - (ENOTSUP) if *action* valid but unsupported. > >> + */ > >> +__rte_experimental > >> +struct rte_flow_shared_action * > >> +rte_flow_shared_action_create(uint16_t port_id, > >> + const struct rte_flow_action *action, > >> + struct rte_flow_error *error); > >> + > >> +/** > >> + * @warning > >> + * @b EXPERIMENTAL: this API may change without prior notice. > >> + * > >> + * Destroys the shared action by handle. > > Destroys -> Destroy > > >> + * > >> + * @param[in] port_id > >> + * The port identifier of the Ethernet device. > >> + * @param[in] action > >> + * Handle for the shared action to be destroyed. > >> + * @param[out] error > >> + * Perform verbose error reporting if not NULL. PMDs initialize this > >> + * structure in case of error only. > >> + * @return > >> + * - (0) if success. > >> + * - (-ENOSYS) if underlying device does not support this functionality. > >> + * - (-EIO) if underlying device is removed. > >> + * - (-ENOENT) if action pointed by *action* handle was not found. > >> + * - (-ETOOMANYREFS) if action pointed by *action* handle still used > >> by one or > >> + * more rules > >> + * rte_errno is also set. > >> + */ > >> +__rte_experimental > >> +int > >> +rte_flow_shared_action_destroy(uint16_t port_id, > >> + struct rte_flow_shared_action *action, > >> + struct rte_flow_error *error); > >> + > >> +/** > >> + * @warning > >> + * @b EXPERIMENTAL: this API may change without prior notice. > >> + * > >> + * Updates inplace the shared action configuration pointed by > >> *action* handle > > Updates -> Update > inplace -> in-place > > >> + * with the configuration provided as *update* argument. > >> + * The update of the shared action configuration effects all flow > >> rules reusing > > May be: reusing -> using > > >> + * the action via handle. > > The interesting question what to do, if some rule cannot have a > new action (i.e. new action is invalid for a rule). > May be it is better to highlight it. > > >> + * > >> + * @param[in] port_id > >> + * The port identifier of the Ethernet device. > >> + * @param[in] action > >> + * Handle for the shared action to be updated. > >> + * @param[in] update > >> + * Action specification used to modify the action pointed by handle. > >> + * *update* should be of same type with the action pointed by the > >> *action* > >> + * handle argument, otherwise considered as invalid. > > Why? If it is a generic rule, it should be checked on a generic > ethdev layer, but I would not impose the limitation. If PMD > may change it, why generic interface specification should > restrict it. > > >> + * @param[out] error > >> + * Perform verbose error reporting if not NULL. PMDs initialize this > >> + * structure in case of error only. > >> + * @return > >> + * - (0) if success. > >> + * - (-ENOSYS) if underlying device does not support this functionality. > >> + * - (-EIO) if underlying device is removed. > >> + * - (-EINVAL) if *update* invalid. > >> + * - (-ENOTSUP) if *update* valid but unsupported. > >> + * - (-ENOENT) if action pointed by *ctx* was not found. > >> + * rte_errno is also set. > >> + */ > >> +__rte_experimental > >> +int > >> +rte_flow_shared_action_update(uint16_t port_id, > >> + struct rte_flow_shared_action *action, > >> + const struct rte_flow_action *update, > >> + struct rte_flow_error *error); > >> + > >> +/** > >> + * @warning > >> + * @b EXPERIMENTAL: this API may change without prior notice. > >> + * > >> + * Query the shared action by handle. > >> + * > >> + * This function allows retrieving action-specific data such as > >> counters. > > "This function allows retrieving" -> Retrieve or Get or Query > > >> + * Data is gathered by special action which may be present/referenced in > >> + * more than one flow rule definition. > >> + * > >> + * \see RTE_FLOW_ACTION_TYPE_COUNT > >> + * > >> + * @param port_id > >> + * Port identifier of Ethernet device. > >> + * @param[in] action > >> + * Handle for the shared action to query. > >> + * @param[in, out] data > >> + * Pointer to storage for the associated query data type. > >> + * @param[out] error > >> + * Perform verbose error reporting if not NULL. PMDs initialize this > >> + * structure in case of error only. > >> + * > >> + * @return > >> + * 0 on success, a negative errno value otherwise and rte_errno is set. > >> + */ > >> +__rte_experimental > >> +int > >> +rte_flow_shared_action_query(uint16_t port_id, > >> + const struct rte_flow_shared_action *action, > >> + void *data, > >> + struct rte_flow_error *error); > >> + > >> #ifdef __cplusplus > >> } > >> #endif > >> diff --git a/lib/librte_ethdev/rte_flow_driver.h > >> b/lib/librte_ethdev/rte_flow_driver.h > >> index 881cc469b7..a2cae1b53c 100644 > >> --- a/lib/librte_ethdev/rte_flow_driver.h > >> +++ b/lib/librte_ethdev/rte_flow_driver.h > >> @@ -107,6 +107,28 @@ struct rte_flow_ops { > >> void **context, > >> uint32_t nb_contexts, > >> struct rte_flow_error *err); > >> + /** See rte_flow_shared_action_create() */ > >> + struct rte_flow_shared_action *(*shared_action_create) > >> + (struct rte_eth_dev *dev, > >> + const struct rte_flow_action *action, > >> + struct rte_flow_error *error); > >> + /** See rte_flow_shared_action_destroy() */ > >> + int (*shared_action_destroy) > >> + (struct rte_eth_dev *dev, > >> + struct rte_flow_shared_action *shared_action, > >> + struct rte_flow_error *error); > >> + /** See rte_flow_shared_action_update() */ > >> + int (*shared_action_update) > >> + (struct rte_eth_dev *dev, > >> + struct rte_flow_shared_action *shared_action, > >> + const struct rte_flow_action *update, > >> + struct rte_flow_error *error); > >> + /** See rte_flow_shared_action_query() */ > >> + int (*shared_action_query) > >> + (struct rte_eth_dev *dev, > >> + const struct rte_flow_shared_action *shared_action, > >> + void *data, > >> + struct rte_flow_error *error); > >> }; > >> /** > >> > > > > The modification of "struct rte_flow_ops", looks fine. > > Acked-by: Ray Kinsella <mdr@ashroe.eu> > > > ^ permalink raw reply [flat|nested] 106+ messages in thread
[parent not found: <20200708204015.24429-3-andreyv@mellanox.com>]
* Re: [dpdk-dev] [PATCH v2 2/6] common/mlx5: modify advanced Rx object via DevX [not found] ` <20200708204015.24429-3-andreyv@mellanox.com> @ 2020-07-13 8:06 ` Kinsella, Ray 0 siblings, 0 replies; 106+ messages in thread From: Kinsella, Ray @ 2020-07-13 8:06 UTC (permalink / raw) To: Andrey Vesnovaty, Matan Azrad, Shahaf Shuler, Viacheslav Ovsiienko, Neil Horman Cc: dev, jerinjacobk, thomas, ferruh.yigit, stephen, bruce.richardson, orika, andrey.vesnovaty On 08/07/2020 21:40, Andrey Vesnovaty wrote: > Implement mlx5_devx_cmd_modify_tir() to modify TIR object using DevX > API. > Add related structs in mlx5_prm.h. > > Signed-off-by: Andrey Vesnovaty <andreyv@mellanox.com> > --- > drivers/common/mlx5/mlx5_devx_cmds.c | 84 +++++++++++++++++++ > drivers/common/mlx5/mlx5_devx_cmds.h | 10 +++ > drivers/common/mlx5/mlx5_prm.h | 29 +++++++ > .../common/mlx5/rte_common_mlx5_version.map | 1 + > 4 files changed, 124 insertions(+) > > diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c > index 2179a83983..2a7098ec6d 100644 > --- a/drivers/common/mlx5/mlx5_devx_cmds.c > +++ b/drivers/common/mlx5/mlx5_devx_cmds.c > @@ -825,6 +825,90 @@ mlx5_devx_cmd_create_tir(void *ctx, > return tir; > } > > +/** > + * Modify TIR using DevX API. > + * > + * @param[in] tir > + * Pointer to TIR DevX object structure. > + * @param [in] modify_tir_attr > + * Pointer to TIR modification attributes structure. > + * > + * @return > + * 0 on success, a negative errno value otherwise and rte_errno is set. > + */ > +int > +mlx5_devx_cmd_modify_tir(struct mlx5_devx_obj *tir, > + struct mlx5_devx_modify_tir_attr *modify_tir_attr) > +{ > + struct mlx5_devx_tir_attr *tir_attr = &modify_tir_attr->tir; > + uint32_t in[MLX5_ST_SZ_DW(modify_tir_in)] = {0}; > + uint32_t out[MLX5_ST_SZ_DW(modify_tir_out)] = {0}; > + void *tir_ctx; > + int ret; > + > + MLX5_SET(modify_tir_in, in, opcode, MLX5_CMD_OP_MODIFY_TIR); > + MLX5_SET(modify_tir_in, in, tirn, modify_tir_attr->tirn); > + MLX5_SET64(modify_tir_in, in, modify_bitmask, > + modify_tir_attr->modify_bitmask); > + > + tir_ctx = MLX5_ADDR_OF(modify_rq_in, in, ctx); > + if (modify_tir_attr->modify_bitmask & > + MLX5_MODIFY_TIR_IN_MODIFY_BITMASK_LRO) { > + MLX5_SET(tirc, tir_ctx, lro_timeout_period_usecs, > + tir_attr->lro_timeout_period_usecs); > + MLX5_SET(tirc, tir_ctx, lro_enable_mask, > + tir_attr->lro_enable_mask); > + MLX5_SET(tirc, tir_ctx, lro_max_msg_sz, > + tir_attr->lro_max_msg_sz); > + } > + if (modify_tir_attr->modify_bitmask & > + MLX5_MODIFY_TIR_IN_MODIFY_BITMASK_INDIRECT_TABLE) > + MLX5_SET(tirc, tir_ctx, indirect_table, > + tir_attr->indirect_table); > + if (modify_tir_attr->modify_bitmask & > + MLX5_MODIFY_TIR_IN_MODIFY_BITMASK_HASH) { > + int i; > + void *outer, *inner; > + MLX5_SET(tirc, tir_ctx, rx_hash_symmetric, > + tir_attr->rx_hash_symmetric); > + MLX5_SET(tirc, tir_ctx, rx_hash_fn, tir_attr->rx_hash_fn); > + for (i = 0; i < 10; i++) { > + MLX5_SET(tirc, tir_ctx, rx_hash_toeplitz_key[i], > + tir_attr->rx_hash_toeplitz_key[i]); > + } > + outer = MLX5_ADDR_OF(tirc, tir_ctx, > + rx_hash_field_selector_outer); > + MLX5_SET(rx_hash_field_select, outer, l3_prot_type, > + tir_attr->rx_hash_field_selector_outer.l3_prot_type); > + MLX5_SET(rx_hash_field_select, outer, l4_prot_type, > + tir_attr->rx_hash_field_selector_outer.l4_prot_type); > + MLX5_SET > + (rx_hash_field_select, outer, selected_fields, > + tir_attr->rx_hash_field_selector_outer.selected_fields); > + inner = MLX5_ADDR_OF(tirc, tir_ctx, > + rx_hash_field_selector_inner); > + MLX5_SET(rx_hash_field_select, inner, l3_prot_type, > + tir_attr->rx_hash_field_selector_inner.l3_prot_type); > + MLX5_SET(rx_hash_field_select, inner, l4_prot_type, > + tir_attr->rx_hash_field_selector_inner.l4_prot_type); > + MLX5_SET > + (rx_hash_field_select, inner, selected_fields, > + tir_attr->rx_hash_field_selector_inner.selected_fields); > + } > + if (modify_tir_attr->modify_bitmask & > + MLX5_MODIFY_TIR_IN_MODIFY_BITMASK_SELF_LB_EN) { > + MLX5_SET(tirc, tir_ctx, self_lb_block, tir_attr->self_lb_block); > + } > + ret = mlx5_glue->devx_obj_modify(tir->obj, in, sizeof(in), > + out, sizeof(out)); > + if (ret) { > + DRV_LOG(ERR, "Failed to modify TIR using DevX"); > + rte_errno = errno; > + return -errno; > + } > + return ret; > +} > + > /** > * Create RQT using DevX API. > * > diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h > index 25704efc1f..9b07b39e66 100644 > --- a/drivers/common/mlx5/mlx5_devx_cmds.h > +++ b/drivers/common/mlx5/mlx5_devx_cmds.h > @@ -178,6 +178,13 @@ struct mlx5_devx_tir_attr { > struct mlx5_rx_hash_field_select rx_hash_field_selector_inner; > }; > > +/* TIR attributes structure, used by TIR modify */ > +struct mlx5_devx_modify_tir_attr { > + uint32_t tirn:24; > + uint64_t modify_bitmask; > + struct mlx5_devx_tir_attr tir; > +}; > + > /* RQT attributes structure, used by RQT operations. */ > struct mlx5_devx_rqt_attr { > uint8_t rq_type; > @@ -372,6 +379,9 @@ int mlx5_devx_cmd_modify_qp_state(struct mlx5_devx_obj *qp, > __rte_internal > int mlx5_devx_cmd_modify_rqt(struct mlx5_devx_obj *rqt, > struct mlx5_devx_rqt_attr *rqt_attr); > +__rte_internal > +int mlx5_devx_cmd_modify_tir(struct mlx5_devx_obj *tir, > + struct mlx5_devx_modify_tir_attr *tir_attr); > > /** > * Create virtio queue counters object DevX API. > diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h > index c63795fc84..029767ea34 100644 > --- a/drivers/common/mlx5/mlx5_prm.h > +++ b/drivers/common/mlx5/mlx5_prm.h > @@ -746,6 +746,7 @@ enum { > MLX5_CMD_OP_QUERY_NIC_VPORT_CONTEXT = 0x754, > MLX5_CMD_OP_ALLOC_TRANSPORT_DOMAIN = 0x816, > MLX5_CMD_OP_CREATE_TIR = 0x900, > + MLX5_CMD_OP_MODIFY_TIR = 0x901, > MLX5_CMD_OP_CREATE_SQ = 0X904, > MLX5_CMD_OP_MODIFY_SQ = 0X905, > MLX5_CMD_OP_CREATE_RQ = 0x908, > @@ -1753,6 +1754,34 @@ struct mlx5_ifc_create_tir_in_bits { > struct mlx5_ifc_tirc_bits ctx; > }; > > +enum { > + MLX5_MODIFY_TIR_IN_MODIFY_BITMASK_LRO = 1ULL << 0, > + MLX5_MODIFY_TIR_IN_MODIFY_BITMASK_INDIRECT_TABLE = 1ULL << 1, > + MLX5_MODIFY_TIR_IN_MODIFY_BITMASK_HASH = 1ULL << 2, > + /* bit 3 - tunneled_offload_en modify not supported */ > + MLX5_MODIFY_TIR_IN_MODIFY_BITMASK_SELF_LB_EN = 1ULL << 4, > +}; > + > +struct mlx5_ifc_modify_tir_out_bits { > + u8 status[0x8]; > + u8 reserved_at_8[0x18]; > + u8 syndrome[0x20]; > + u8 reserved_at_40[0x40]; > +}; > + > +struct mlx5_ifc_modify_tir_in_bits { > + u8 opcode[0x10]; > + u8 uid[0x10]; > + u8 reserved_at_20[0x10]; > + u8 op_mod[0x10]; > + u8 reserved_at_40[0x8]; > + u8 tirn[0x18]; > + u8 reserved_at_60[0x20]; > + u8 modify_bitmask[0x40]; > + u8 reserved_at_c0[0x40]; > + struct mlx5_ifc_tirc_bits ctx; > +}; > + > enum { > MLX5_INLINE_Q_TYPE_RQ = 0x0, > MLX5_INLINE_Q_TYPE_VIRTQ = 0x1, > diff --git a/drivers/common/mlx5/rte_common_mlx5_version.map b/drivers/common/mlx5/rte_common_mlx5_version.map > index ae57ebdba5..0bfedab32f 100644 > --- a/drivers/common/mlx5/rte_common_mlx5_version.map > +++ b/drivers/common/mlx5/rte_common_mlx5_version.map > @@ -29,6 +29,7 @@ INTERNAL { > mlx5_devx_cmd_modify_rq; > mlx5_devx_cmd_modify_rqt; > mlx5_devx_cmd_modify_sq; > + mlx5_devx_cmd_modify_tir; > mlx5_devx_cmd_modify_virtq; > mlx5_devx_cmd_qp_query_tis_td; > mlx5_devx_cmd_query_hca_attr; Acked-by: Ray Kinsella <mdr@ashroe.eu> ^ permalink raw reply [flat|nested] 106+ messages in thread
* [dpdk-dev] [PATCH v2 0/6] add flow shared action API + PMD 2020-07-02 12:05 [dpdk-dev] [PATCH] add flow shared action API Andrey Vesnovaty 2020-07-03 15:02 ` Jerin Jacob @ 2020-07-08 21:39 ` Andrey Vesnovaty 2020-07-08 21:39 ` [dpdk-dev] [PATCH v2 1/6] ethdev: add flow shared action API Andrey Vesnovaty ` (6 more replies) 2020-10-06 20:08 ` [dpdk-dev] [PATCH v4 0/2] RTE flow shared action Andrey Vesnovaty ` (4 subsequent siblings) 6 siblings, 7 replies; 106+ messages in thread From: Andrey Vesnovaty @ 2020-07-08 21:39 UTC (permalink / raw) To: dev Cc: jer, jerinjacobk, thomas, ferruh.yigit, stephen, bruce.richardson, orika, viacheslavo, andrey.vesnovaty Hi Jerin and Ori. V2 changes: - First version of PMD implementation for shared action API. - Simple example application demonstaration shared action update. - Changes to shred action API according to discussion of V1. @Jerin Jacob: please take a look at provided example, hopefully it will help to converge our API discussion and reach consensus on it. Thanks, Andrey Andrey Vesnovaty (6): ethdev: add flow shared action API common/mlx5: modify advanced Rx object via DevX net/mlx5: modify hash Rx queue objects net/mlx5: shared action PMD net/mlx5: driver support for shared action examples/flow_filtering: utilize shared RSS action doc/guides/sample_app_ug/flow_filtering.rst | 62 +- drivers/common/mlx5/mlx5_devx_cmds.c | 84 +++ drivers/common/mlx5/mlx5_devx_cmds.h | 10 + drivers/common/mlx5/mlx5_prm.h | 29 + .../common/mlx5/rte_common_mlx5_version.map | 1 + drivers/net/mlx5/mlx5.c | 1 + drivers/net/mlx5/mlx5.h | 2 + drivers/net/mlx5/mlx5_defs.h | 3 + drivers/net/mlx5/mlx5_flow.c | 492 ++++++++++++- drivers/net/mlx5/mlx5_flow.h | 83 +++ drivers/net/mlx5/mlx5_flow_dv.c | 671 +++++++++++++++++- drivers/net/mlx5/mlx5_rxq.c | 300 ++++++-- drivers/net/mlx5/mlx5_rxtx.h | 4 + examples/flow_filtering/flow_blocks.c | 30 +- examples/flow_filtering/main.c | 41 +- lib/librte_ethdev/rte_ethdev_version.map | 6 + lib/librte_ethdev/rte_flow.c | 81 +++ lib/librte_ethdev/rte_flow.h | 148 +++- lib/librte_ethdev/rte_flow_driver.h | 22 + 19 files changed, 1924 insertions(+), 146 deletions(-) -- 2.26.2 ^ permalink raw reply [flat|nested] 106+ messages in thread
* [dpdk-dev] [PATCH v2 1/6] ethdev: add flow shared action API 2020-07-08 21:39 ` [dpdk-dev] [PATCH v2 0/6] add flow shared action API + PMD Andrey Vesnovaty @ 2020-07-08 21:39 ` Andrey Vesnovaty 2020-09-12 2:18 ` Ajit Khaparde 2020-07-08 21:39 ` [dpdk-dev] [PATCH v2 2/6] common/mlx5: modify advanced Rx object via DevX Andrey Vesnovaty ` (5 subsequent siblings) 6 siblings, 1 reply; 106+ messages in thread From: Andrey Vesnovaty @ 2020-07-08 21:39 UTC (permalink / raw) To: dev Cc: jer, jerinjacobk, thomas, ferruh.yigit, stephen, bruce.richardson, orika, viacheslavo, andrey.vesnovaty, Ray Kinsella, Neil Horman, Andrew Rybchenko From: Andrey Vesnovaty <andrey.vesnovaty@gmail.com> This commit introduces extension of DPDK flow action API enabling sharing of single rte_flow_action in multiple flows. The API intended for PMDs where multiple HW offloaded flows can reuse the same HW essence/object representing flow action and modification of such an essence/object effects all the rules using it. Motivation and example === Adding or removing one or more queues to RSS used by multiple flow rules imposes per rule toll for current DPDK flow API; the scenario requires for each flow sharing cloned RSS action: - call `rte_flow_destroy()` - call `rte_flow_create()` with modified RSS action API for sharing action and its in-place update benefits: - reduce the overhead of multiple RSS flow rules reconfiguration - optimize resource utilization by sharing action across of multiple flows Change description === Shared action === In order to represent flow action shared by multiple flows new action type RTE_FLOW_ACTION_TYPE_SHARED is introduced (see `enum rte_flow_action_type`). Actually the introduced API decouples action from any specific flow and enables sharing of single action by its handle across multiple flows. Shared action create/use/destroy === Shared action may be reused by some or none flow rules at any given moment, i.e. shared action reside outside of the context of any flow. Shared action represent HW resources/objects used for action offloading implementation. API for shared action create (see `rte_flow_shared_action_create()`): - should allocate HW resources and make related initializations required for shared action implementation. - make necessary preparations to maintain shared access to the action resources, configuration and state. API for shared action destroy (see `rte_flow_shared_action_destroy()`) should release HW resources and make related cleanups required for shared action implementation. In order to share some flow action reuse the handle of type `struct rte_flow_shared_action` returned by rte_flow_shared_action_create() as a `conf` field of `struct rte_flow_action` (see "example" section). If some shared action not used by any flow rule all resources allocated by the shared action can be released by rte_flow_shared_action_destroy() (see "example" section). The shared action handle passed as argument to destroy API should not be used any further i.e. result of the usage is undefined. Shared action re-configuration === Shared action behavior defined by its configuration can be updated via rte_flow_shared_action_update() (see "example" section). The shared action update operation modifies HW related resources/objects allocated on the action creation. The number of operations performed by the update operation should not be dependent on number of flows sharing the related action. On return of shared action update API action behavior should be according to updated configuration for all flows sharing the action. Shared action query === Provide separate API to query shared action sate (see rte_flow_shared_action_update()). Taking a counter as an example: query returns value aggregating all counter increments across all flow rules sharing the counter. PMD support === The support of introduced API is pure PMD specific design and responsibility for each action type (see struct rte_flow_ops). testpmd === In order to utilize introduced API testpmd cli may implement following extension create/update/destroy/query shared action accordingly flow shared_action create {port_id} [index] {action} flow shared_action update {port_id} {index} {action} flow shared_action destroy {port_id} {index} flow shared_action query {port_id} {index} testpmd example === configure rss to queues 1 & 2 testpmd> flow shared_action create 0 100 rss 1 2 create flow rule utilizing shared action testpmd> flow create 0 ingress \ pattern eth dst is 0c:42:a1:15:fd:ac / ipv6 / tcp / end \ actions shared 100 end / end add 2 more queues testpmd> flow shared_action modify 0 100 rss 1 2 3 4 example === struct rte_flow_action actions[2]; struct rte_flow_action action; /* skipped: initialize action */ struct rte_flow_shared_action *handle = rte_flow_shared_action_create( port_id, &action, &error); actions[0].type = RTE_FLOW_ACTION_TYPE_SHARED; actions[0].conf = handle; actions[1].type = RTE_FLOW_ACTION_TYPE_END; /* skipped: init attr0 & pattern0 args */ struct rte_flow *flow0 = rte_flow_create(port_id, &attr0, pattern0, actions, error); /* create more rules reusing shared action */ struct rte_flow *flow1 = rte_flow_create(port_id, &attr1, pattern1, actions, error); /* skipped: for flows 2 till N */ struct rte_flow *flowN = rte_flow_create(port_id, &attrN, patternN, actions, error); /* update shared action */ struct rte_flow_action updated_action; /* * skipped: initialize updated_action according to desired action * configuration change */ rte_flow_shared_action_update(port_id, handle, &updated_action, error); /* * from now on all flows 1 till N will act according to configuration of * updated_action */ /* skipped: destroy all flows 1 till N */ rte_flow_shared_action_destroy(port_id, handle, error); Signed-off-by: Andrey Vesnovaty <andreyv@mellanox.com> --- lib/librte_ethdev/rte_ethdev_version.map | 6 + lib/librte_ethdev/rte_flow.c | 81 +++++++++++++ lib/librte_ethdev/rte_flow.h | 148 ++++++++++++++++++++++- lib/librte_ethdev/rte_flow_driver.h | 22 ++++ 4 files changed, 256 insertions(+), 1 deletion(-) diff --git a/lib/librte_ethdev/rte_ethdev_version.map b/lib/librte_ethdev/rte_ethdev_version.map index 7155056045..119d84976a 100644 --- a/lib/librte_ethdev/rte_ethdev_version.map +++ b/lib/librte_ethdev/rte_ethdev_version.map @@ -241,4 +241,10 @@ EXPERIMENTAL { __rte_ethdev_trace_rx_burst; __rte_ethdev_trace_tx_burst; rte_flow_get_aged_flows; + + # added in 20.08 + rte_flow_shared_action_create; + rte_flow_shared_action_destroy; + rte_flow_shared_action_update; + rte_flow_shared_action_query; }; diff --git a/lib/librte_ethdev/rte_flow.c b/lib/librte_ethdev/rte_flow.c index 1685be5f73..0ac4d31a13 100644 --- a/lib/librte_ethdev/rte_flow.c +++ b/lib/librte_ethdev/rte_flow.c @@ -1250,3 +1250,84 @@ rte_flow_get_aged_flows(uint16_t port_id, void **contexts, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL, rte_strerror(ENOTSUP)); } + +struct rte_flow_shared_action * +rte_flow_shared_action_create(uint16_t port_id, + const struct rte_flow_action *action, + struct rte_flow_error *error) +{ + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; + struct rte_flow_shared_action *shared_action; + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error); + + if (unlikely(!ops)) + return NULL; + if (likely(!!ops->shared_action_create)) { + shared_action = ops->shared_action_create(dev, action, error); + if (shared_action == NULL) + flow_err(port_id, -rte_errno, error); + return shared_action; + } + rte_flow_error_set(error, ENOSYS, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, rte_strerror(ENOSYS)); + return NULL; +} + +int +rte_flow_shared_action_destroy(uint16_t port_id, + struct rte_flow_shared_action *action, + struct rte_flow_error *error) +{ + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error); + + if (unlikely(!ops)) + return -rte_errno; + if (likely(!!ops->shared_action_destroy)) + return flow_err(port_id, + ops->shared_action_destroy(dev, action, error), + error); + return rte_flow_error_set(error, ENOSYS, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, rte_strerror(ENOSYS)); +} + +int +rte_flow_shared_action_update(uint16_t port_id, + struct rte_flow_shared_action *action, + const struct rte_flow_action *update, + struct rte_flow_error *error) +{ + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error); + + if (unlikely(!ops)) + return -rte_errno; + if (likely(!!ops->shared_action_update)) + return flow_err(port_id, ops->shared_action_update(dev, action, + update, error), + error); + return rte_flow_error_set(error, ENOSYS, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, rte_strerror(ENOSYS)); +} + +int +rte_flow_shared_action_query(uint16_t port_id, + const struct rte_flow_shared_action *action, + void *data, + struct rte_flow_error *error) +{ + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error); + + if (unlikely(!ops)) + return -rte_errno; + if (likely(!!ops->shared_action_query)) + return flow_err(port_id, ops->shared_action_query(dev, action, + data, error), + error); + return rte_flow_error_set(error, ENOSYS, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, rte_strerror(ENOSYS)); +} diff --git a/lib/librte_ethdev/rte_flow.h b/lib/librte_ethdev/rte_flow.h index b0e4199192..257456b14a 100644 --- a/lib/librte_ethdev/rte_flow.h +++ b/lib/librte_ethdev/rte_flow.h @@ -1681,7 +1681,8 @@ enum rte_flow_action_type { /** * Enables counters for this flow rule. * - * These counters can be retrieved and reset through rte_flow_query(), + * These counters can be retrieved and reset through rte_flow_query() or + * rte_flow_shared_action_query() if the action provided via handle, * see struct rte_flow_query_count. * * See struct rte_flow_action_count. @@ -2099,6 +2100,14 @@ enum rte_flow_action_type { * see enum RTE_ETH_EVENT_FLOW_AGED */ RTE_FLOW_ACTION_TYPE_AGE, + + /** + * Describes action shared a cross multiple flow rules. + * + * Enables multiple rules reference the same action by handle (see + * struct rte_flow_shared_action). + */ + RTE_FLOW_ACTION_TYPE_SHARED, }; /** @@ -2660,6 +2669,20 @@ struct rte_flow_action_set_dscp { uint8_t dscp; }; + +/** + * RTE_FLOW_ACTION_TYPE_SHARED + * + * Opaque type returned after successfully creating a shared action. + * + * This handle can be used to manage and query the related action: + * - share it a cross multiple flow rules + * - update action configuration + * - query action data + * - destroy action + */ +struct rte_flow_shared_action; + /* Mbuf dynamic field offset for metadata. */ extern int32_t rte_flow_dynf_metadata_offs; @@ -3324,6 +3347,129 @@ int rte_flow_get_aged_flows(uint16_t port_id, void **contexts, uint32_t nb_contexts, struct rte_flow_error *error); +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Create shared action for reuse in multiple flow rules. + * + * @param[in] port_id + * The port identifier of the Ethernet device. + * @param[in] action + * Action configuration for shared action creation. + * @param[out] error + * Perform verbose error reporting if not NULL. PMDs initialize this + * structure in case of error only. + * @return + * A valid handle in case of success, NULL otherwise and rte_errno is set + * to one of the error codes defined: + * - (ENOSYS) if underlying device does not support this functionality. + * - (EIO) if underlying device is removed. + * - (EINVAL) if *action* invalid. + * - (ENOTSUP) if *action* valid but unsupported. + */ +__rte_experimental +struct rte_flow_shared_action * +rte_flow_shared_action_create(uint16_t port_id, + const struct rte_flow_action *action, + struct rte_flow_error *error); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Destroys the shared action by handle. + * + * @param[in] port_id + * The port identifier of the Ethernet device. + * @param[in] action + * Handle for the shared action to be destroyed. + * @param[out] error + * Perform verbose error reporting if not NULL. PMDs initialize this + * structure in case of error only. + * @return + * - (0) if success. + * - (-ENOSYS) if underlying device does not support this functionality. + * - (-EIO) if underlying device is removed. + * - (-ENOENT) if action pointed by *action* handle was not found. + * - (-ETOOMANYREFS) if action pointed by *action* handle still used by one or + * more rules + * rte_errno is also set. + */ +__rte_experimental +int +rte_flow_shared_action_destroy(uint16_t port_id, + struct rte_flow_shared_action *action, + struct rte_flow_error *error); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Updates inplace the shared action configuration pointed by *action* handle + * with the configuration provided as *update* argument. + * The update of the shared action configuration effects all flow rules reusing + * the action via handle. + * + * @param[in] port_id + * The port identifier of the Ethernet device. + * @param[in] action + * Handle for the shared action to be updated. + * @param[in] update + * Action specification used to modify the action pointed by handle. + * *update* should be of same type with the action pointed by the *action* + * handle argument, otherwise considered as invalid. + * @param[out] error + * Perform verbose error reporting if not NULL. PMDs initialize this + * structure in case of error only. + * @return + * - (0) if success. + * - (-ENOSYS) if underlying device does not support this functionality. + * - (-EIO) if underlying device is removed. + * - (-EINVAL) if *update* invalid. + * - (-ENOTSUP) if *update* valid but unsupported. + * - (-ENOENT) if action pointed by *ctx* was not found. + * rte_errno is also set. + */ +__rte_experimental +int +rte_flow_shared_action_update(uint16_t port_id, + struct rte_flow_shared_action *action, + const struct rte_flow_action *update, + struct rte_flow_error *error); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Query the shared action by handle. + * + * This function allows retrieving action-specific data such as counters. + * Data is gathered by special action which may be present/referenced in + * more than one flow rule definition. + * + * \see RTE_FLOW_ACTION_TYPE_COUNT + * + * @param port_id + * Port identifier of Ethernet device. + * @param[in] action + * Handle for the shared action to query. + * @param[in, out] data + * Pointer to storage for the associated query data type. + * @param[out] error + * Perform verbose error reporting if not NULL. PMDs initialize this + * structure in case of error only. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +__rte_experimental +int +rte_flow_shared_action_query(uint16_t port_id, + const struct rte_flow_shared_action *action, + void *data, + struct rte_flow_error *error); + #ifdef __cplusplus } #endif diff --git a/lib/librte_ethdev/rte_flow_driver.h b/lib/librte_ethdev/rte_flow_driver.h index 881cc469b7..a2cae1b53c 100644 --- a/lib/librte_ethdev/rte_flow_driver.h +++ b/lib/librte_ethdev/rte_flow_driver.h @@ -107,6 +107,28 @@ struct rte_flow_ops { void **context, uint32_t nb_contexts, struct rte_flow_error *err); + /** See rte_flow_shared_action_create() */ + struct rte_flow_shared_action *(*shared_action_create) + (struct rte_eth_dev *dev, + const struct rte_flow_action *action, + struct rte_flow_error *error); + /** See rte_flow_shared_action_destroy() */ + int (*shared_action_destroy) + (struct rte_eth_dev *dev, + struct rte_flow_shared_action *shared_action, + struct rte_flow_error *error); + /** See rte_flow_shared_action_update() */ + int (*shared_action_update) + (struct rte_eth_dev *dev, + struct rte_flow_shared_action *shared_action, + const struct rte_flow_action *update, + struct rte_flow_error *error); + /** See rte_flow_shared_action_query() */ + int (*shared_action_query) + (struct rte_eth_dev *dev, + const struct rte_flow_shared_action *shared_action, + void *data, + struct rte_flow_error *error); }; /** -- 2.26.2 ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [dpdk-dev] [PATCH v2 1/6] ethdev: add flow shared action API 2020-07-08 21:39 ` [dpdk-dev] [PATCH v2 1/6] ethdev: add flow shared action API Andrey Vesnovaty @ 2020-09-12 2:18 ` Ajit Khaparde 2020-09-15 11:50 ` Andrey Vesnovaty 0 siblings, 1 reply; 106+ messages in thread From: Ajit Khaparde @ 2020-09-12 2:18 UTC (permalink / raw) To: Andrey Vesnovaty Cc: dpdk-dev, jer, Jerin Jacob, Thomas Monjalon, Ferruh Yigit, Stephen Hemminger, Bruce Richardson, Ori Kam, Viacheslav Ovsiienko, andrey.vesnovaty, Ray Kinsella, Neil Horman, Andrew Rybchenko On Wed, Jul 8, 2020 at 2:40 PM Andrey Vesnovaty <andreyv@mellanox.com> wrote: > From: Andrey Vesnovaty <andrey.vesnovaty@gmail.com> > > This commit introduces extension of DPDK flow action API enabling > sharing of single rte_flow_action in multiple flows. The API intended for > PMDs where multiple HW offloaded flows can reuse the same HW > essence/object representing flow action and modification of such an > essence/object effects all the rules using it. > > Motivation and example > === > Adding or removing one or more queues to RSS used by multiple flow rules > imposes per rule toll for current DPDK flow API; the scenario requires > for each flow sharing cloned RSS action: > - call `rte_flow_destroy()` > - call `rte_flow_create()` with modified RSS action > > API for sharing action and its in-place update benefits: > - reduce the overhead of multiple RSS flow rules reconfiguration > - optimize resource utilization by sharing action across of multiple > flows > > Change description > === > > Shared action > === > In order to represent flow action shared by multiple flows new action > type RTE_FLOW_ACTION_TYPE_SHARED is introduced (see `enum > rte_flow_action_type`). > Actually the introduced API decouples action from any specific flow and > enables sharing of single action by its handle across multiple flows. > > Shared action create/use/destroy > === > Shared action may be reused by some or none flow rules at any given > moment, i.e. shared action reside outside of the context of any flow. > Shared action represent HW resources/objects used for action offloading > implementation. > API for shared action create (see `rte_flow_shared_action_create()`): > - should allocate HW resources and make related initializations required > for shared action implementation. > - make necessary preparations to maintain shared access to > the action resources, configuration and state. > API for shared action destroy (see `rte_flow_shared_action_destroy()`) > should release HW resources and make related cleanups required for shared > action implementation. > > In order to share some flow action reuse the handle of type > `struct rte_flow_shared_action` returned by > rte_flow_shared_action_create() as a `conf` field of > `struct rte_flow_action` (see "example" section). > > If some shared action not used by any flow rule all resources allocated > by the shared action can be released by rte_flow_shared_action_destroy() > (see "example" section). The shared action handle passed as argument to > destroy API should not be used any further i.e. result of the usage is > undefined. > > Shared action re-configuration > === > Shared action behavior defined by its configuration can be updated via > rte_flow_shared_action_update() (see "example" section). The shared > action update operation modifies HW related resources/objects allocated > on the action creation. The number of operations performed by the update > operation should not be dependent on number of flows sharing the related > action. On return of shared action update API action behavior should be > according to updated configuration for all flows sharing the action. > > Shared action query > === > Provide separate API to query shared action sate (see > rte_flow_shared_action_update()). Taking a counter as an example: query > returns value aggregating all counter increments across all flow rules > sharing the counter. > > PMD support > === > The support of introduced API is pure PMD specific design and > responsibility for each action type (see struct rte_flow_ops). > > testpmd > === > In order to utilize introduced API testpmd cli may implement following > extension > create/update/destroy/query shared action accordingly > > flow shared_action create {port_id} [index] {action} > flow shared_action update {port_id} {index} {action} > flow shared_action destroy {port_id} {index} > flow shared_action query {port_id} {index} > > testpmd example > === > > configure rss to queues 1 & 2 > > testpmd> flow shared_action create 0 100 rss 1 2 > > create flow rule utilizing shared action > > testpmd> flow create 0 ingress \ > pattern eth dst is 0c:42:a1:15:fd:ac / ipv6 / tcp / end \ > actions shared 100 end / end > > add 2 more queues > > testpmd> flow shared_action modify 0 100 rss 1 2 3 4 > > example > === > > struct rte_flow_action actions[2]; > struct rte_flow_action action; > /* skipped: initialize action */ > struct rte_flow_shared_action *handle = rte_flow_shared_action_create( > port_id, &action, &error); > actions[0].type = RTE_FLOW_ACTION_TYPE_SHARED; > actions[0].conf = handle; > actions[1].type = RTE_FLOW_ACTION_TYPE_END; > /* skipped: init attr0 & pattern0 args */ > struct rte_flow *flow0 = rte_flow_create(port_id, &attr0, pattern0, > actions, error); > /* create more rules reusing shared action */ > struct rte_flow *flow1 = rte_flow_create(port_id, &attr1, pattern1, > actions, error); > /* skipped: for flows 2 till N */ > struct rte_flow *flowN = rte_flow_create(port_id, &attrN, patternN, > actions, error); > /* update shared action */ > struct rte_flow_action updated_action; > /* > * skipped: initialize updated_action according to desired action > * configuration change > */ > rte_flow_shared_action_update(port_id, handle, &updated_action, error); > /* > * from now on all flows 1 till N will act according to configuration of > * updated_action > */ > /* skipped: destroy all flows 1 till N */ > rte_flow_shared_action_destroy(port_id, handle, error); > > Signed-off-by: Andrey Vesnovaty <andreyv@mellanox.com> > --- > lib/librte_ethdev/rte_ethdev_version.map | 6 + > lib/librte_ethdev/rte_flow.c | 81 +++++++++++++ > lib/librte_ethdev/rte_flow.h | 148 ++++++++++++++++++++++- > lib/librte_ethdev/rte_flow_driver.h | 22 ++++ > 4 files changed, 256 insertions(+), 1 deletion(-) > > diff --git a/lib/librte_ethdev/rte_ethdev_version.map > b/lib/librte_ethdev/rte_ethdev_version.map > index 7155056045..119d84976a 100644 > --- a/lib/librte_ethdev/rte_ethdev_version.map > +++ b/lib/librte_ethdev/rte_ethdev_version.map > @@ -241,4 +241,10 @@ EXPERIMENTAL { > __rte_ethdev_trace_rx_burst; > __rte_ethdev_trace_tx_burst; > rte_flow_get_aged_flows; > + > + # added in 20.08 > + rte_flow_shared_action_create; > + rte_flow_shared_action_destroy; > + rte_flow_shared_action_update; > + rte_flow_shared_action_query; > }; > diff --git a/lib/librte_ethdev/rte_flow.c b/lib/librte_ethdev/rte_flow.c > index 1685be5f73..0ac4d31a13 100644 > --- a/lib/librte_ethdev/rte_flow.c > +++ b/lib/librte_ethdev/rte_flow.c > @@ -1250,3 +1250,84 @@ rte_flow_get_aged_flows(uint16_t port_id, void > **contexts, > RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > NULL, rte_strerror(ENOTSUP)); > } > + > +struct rte_flow_shared_action * > +rte_flow_shared_action_create(uint16_t port_id, > + const struct rte_flow_action *action, > It will be good to have an attributes argument here. Some hardware devices may have ingress and egress resource pools. So a 'direction' attribute can help share the resource effectively. > + struct rte_flow_error *error) > +{ > + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; > + struct rte_flow_shared_action *shared_action; > + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error); > + > + if (unlikely(!ops)) > + return NULL; > + if (likely(!!ops->shared_action_create)) { > + shared_action = ops->shared_action_create(dev, action, > error); > + if (shared_action == NULL) > + flow_err(port_id, -rte_errno, error); > + return shared_action; > + } > + rte_flow_error_set(error, ENOSYS, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > + NULL, rte_strerror(ENOSYS)); > + return NULL; > +} > + > +int > +rte_flow_shared_action_destroy(uint16_t port_id, > + struct rte_flow_shared_action *action, > + struct rte_flow_error *error) > +{ > + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; > + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error); > + > + if (unlikely(!ops)) > + return -rte_errno; > + if (likely(!!ops->shared_action_destroy)) > + return flow_err(port_id, > + ops->shared_action_destroy(dev, action, > error), > + error); > + return rte_flow_error_set(error, ENOSYS, > + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > + NULL, rte_strerror(ENOSYS)); > +} > + > +int > +rte_flow_shared_action_update(uint16_t port_id, > + struct rte_flow_shared_action *action, > + const struct rte_flow_action *update, > + struct rte_flow_error *error) > +{ > + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; > + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error); > + > + if (unlikely(!ops)) > + return -rte_errno; > + if (likely(!!ops->shared_action_update)) > + return flow_err(port_id, ops->shared_action_update(dev, > action, > + update, error), > + error); > + return rte_flow_error_set(error, ENOSYS, > + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > + NULL, rte_strerror(ENOSYS)); > +} > + > +int > +rte_flow_shared_action_query(uint16_t port_id, > + const struct rte_flow_shared_action *action, > + void *data, > + struct rte_flow_error *error) > +{ > + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; > + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error); > + > + if (unlikely(!ops)) > + return -rte_errno; > + if (likely(!!ops->shared_action_query)) > + return flow_err(port_id, ops->shared_action_query(dev, > action, > + data, error), > + error); > + return rte_flow_error_set(error, ENOSYS, > + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > + NULL, rte_strerror(ENOSYS)); > +} > diff --git a/lib/librte_ethdev/rte_flow.h b/lib/librte_ethdev/rte_flow.h > index b0e4199192..257456b14a 100644 > --- a/lib/librte_ethdev/rte_flow.h > +++ b/lib/librte_ethdev/rte_flow.h > @@ -1681,7 +1681,8 @@ enum rte_flow_action_type { > /** > * Enables counters for this flow rule. > * > - * These counters can be retrieved and reset through > rte_flow_query(), > + * These counters can be retrieved and reset through > rte_flow_query() or > + * rte_flow_shared_action_query() if the action provided via > handle, > * see struct rte_flow_query_count. > * > * See struct rte_flow_action_count. > @@ -2099,6 +2100,14 @@ enum rte_flow_action_type { > * see enum RTE_ETH_EVENT_FLOW_AGED > */ > RTE_FLOW_ACTION_TYPE_AGE, > + > + /** > + * Describes action shared a cross multiple flow rules. > + * > + * Enables multiple rules reference the same action by handle (see > + * struct rte_flow_shared_action). > + */ > + RTE_FLOW_ACTION_TYPE_SHARED, > }; > > /** > @@ -2660,6 +2669,20 @@ struct rte_flow_action_set_dscp { > uint8_t dscp; > }; > > + > +/** > + * RTE_FLOW_ACTION_TYPE_SHARED > + * > + * Opaque type returned after successfully creating a shared action. > + * > + * This handle can be used to manage and query the related action: > + * - share it a cross multiple flow rules > + * - update action configuration > + * - query action data > + * - destroy action > + */ > +struct rte_flow_shared_action; > + > /* Mbuf dynamic field offset for metadata. */ > extern int32_t rte_flow_dynf_metadata_offs; > > @@ -3324,6 +3347,129 @@ int > rte_flow_get_aged_flows(uint16_t port_id, void **contexts, > uint32_t nb_contexts, struct rte_flow_error > *error); > > +/** > + * @warning > + * @b EXPERIMENTAL: this API may change without prior notice. > + * > + * Create shared action for reuse in multiple flow rules. > + * > + * @param[in] port_id > + * The port identifier of the Ethernet device. > + * @param[in] action > + * Action configuration for shared action creation. > + * @param[out] error > + * Perform verbose error reporting if not NULL. PMDs initialize this > + * structure in case of error only. > + * @return > + * A valid handle in case of success, NULL otherwise and rte_errno is > set > + * to one of the error codes defined: > + * - (ENOSYS) if underlying device does not support this functionality. > + * - (EIO) if underlying device is removed. > + * - (EINVAL) if *action* invalid. > + * - (ENOTSUP) if *action* valid but unsupported. > + */ > +__rte_experimental > +struct rte_flow_shared_action * > +rte_flow_shared_action_create(uint16_t port_id, > + const struct rte_flow_action *action, > + struct rte_flow_error *error); > + > +/** > + * @warning > + * @b EXPERIMENTAL: this API may change without prior notice. > + * > + * Destroys the shared action by handle. > + * > + * @param[in] port_id > + * The port identifier of the Ethernet device. > + * @param[in] action > + * Handle for the shared action to be destroyed. > + * @param[out] error > + * Perform verbose error reporting if not NULL. PMDs initialize this > + * structure in case of error only. > + * @return > + * - (0) if success. > + * - (-ENOSYS) if underlying device does not support this functionality. > + * - (-EIO) if underlying device is removed. > + * - (-ENOENT) if action pointed by *action* handle was not found. > + * - (-ETOOMANYREFS) if action pointed by *action* handle still used by > one or > + * more rules > + * rte_errno is also set. > + */ > +__rte_experimental > +int > +rte_flow_shared_action_destroy(uint16_t port_id, > + struct rte_flow_shared_action *action, > + struct rte_flow_error *error); > + > +/** > + * @warning > + * @b EXPERIMENTAL: this API may change without prior notice. > + * > + * Updates inplace the shared action configuration pointed by *action* > handle > + * with the configuration provided as *update* argument. > + * The update of the shared action configuration effects all flow rules > reusing > + * the action via handle. > + * > + * @param[in] port_id > + * The port identifier of the Ethernet device. > + * @param[in] action > + * Handle for the shared action to be updated. > + * @param[in] update > + * Action specification used to modify the action pointed by handle. > + * *update* should be of same type with the action pointed by the > *action* > + * handle argument, otherwise considered as invalid. > + * @param[out] error > + * Perform verbose error reporting if not NULL. PMDs initialize this > + * structure in case of error only. > + * @return > + * - (0) if success. > + * - (-ENOSYS) if underlying device does not support this functionality. > + * - (-EIO) if underlying device is removed. > + * - (-EINVAL) if *update* invalid. > + * - (-ENOTSUP) if *update* valid but unsupported. > + * - (-ENOENT) if action pointed by *ctx* was not found. > + * rte_errno is also set. > + */ > +__rte_experimental > +int > +rte_flow_shared_action_update(uint16_t port_id, > + struct rte_flow_shared_action *action, > + const struct rte_flow_action *update, > + struct rte_flow_error *error); > + > +/** > + * @warning > + * @b EXPERIMENTAL: this API may change without prior notice. > + * > + * Query the shared action by handle. > + * > + * This function allows retrieving action-specific data such as counters. > + * Data is gathered by special action which may be present/referenced in > + * more than one flow rule definition. > + * > + * \see RTE_FLOW_ACTION_TYPE_COUNT > + * > + * @param port_id > + * Port identifier of Ethernet device. > + * @param[in] action > + * Handle for the shared action to query. > + * @param[in, out] data > + * Pointer to storage for the associated query data type. > + * @param[out] error > + * Perform verbose error reporting if not NULL. PMDs initialize this > + * structure in case of error only. > + * > + * @return > + * 0 on success, a negative errno value otherwise and rte_errno is set. > + */ > +__rte_experimental > +int > +rte_flow_shared_action_query(uint16_t port_id, > + const struct rte_flow_shared_action *action, > + void *data, > + struct rte_flow_error *error); > + > #ifdef __cplusplus > } > #endif > diff --git a/lib/librte_ethdev/rte_flow_driver.h > b/lib/librte_ethdev/rte_flow_driver.h > index 881cc469b7..a2cae1b53c 100644 > --- a/lib/librte_ethdev/rte_flow_driver.h > +++ b/lib/librte_ethdev/rte_flow_driver.h > @@ -107,6 +107,28 @@ struct rte_flow_ops { > void **context, > uint32_t nb_contexts, > struct rte_flow_error *err); > + /** See rte_flow_shared_action_create() */ > + struct rte_flow_shared_action *(*shared_action_create) > + (struct rte_eth_dev *dev, > + const struct rte_flow_action *action, > + struct rte_flow_error *error); > + /** See rte_flow_shared_action_destroy() */ > + int (*shared_action_destroy) > + (struct rte_eth_dev *dev, > + struct rte_flow_shared_action *shared_action, > + struct rte_flow_error *error); > + /** See rte_flow_shared_action_update() */ > + int (*shared_action_update) > + (struct rte_eth_dev *dev, > + struct rte_flow_shared_action *shared_action, > + const struct rte_flow_action *update, > + struct rte_flow_error *error); > + /** See rte_flow_shared_action_query() */ > + int (*shared_action_query) > + (struct rte_eth_dev *dev, > + const struct rte_flow_shared_action *shared_action, > + void *data, > + struct rte_flow_error *error); > }; > > /** > -- > 2.26.2 > > ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [dpdk-dev] [PATCH v2 1/6] ethdev: add flow shared action API 2020-09-12 2:18 ` Ajit Khaparde @ 2020-09-15 11:50 ` Andrey Vesnovaty 2020-09-15 15:49 ` Ajit Khaparde 0 siblings, 1 reply; 106+ messages in thread From: Andrey Vesnovaty @ 2020-09-15 11:50 UTC (permalink / raw) To: Ajit Khaparde, NBU-Contact-Thomas Monjalon, Ori Kam, Slava Ovsiienko, jerinj, Andrew Rybchenko Cc: dpdk-dev, jer, Jerin Jacob, NBU-Contact-Thomas Monjalon, Ferruh Yigit, Stephen Hemminger, Bruce Richardson, Ori Kam, Viacheslav Ovsiienko, andrey.vesnovaty, Ray Kinsella, Neil Horman, Thomas Monjalon Hi Ajit, PSB. Thanks, Andrey > -----Original Message----- > From: Ajit Khaparde <ajit.khaparde@broadcom.com> > Sent: Saturday, September 12, 2020 5:18 AM > To: Andrey Vesnovaty <andreyv@mellanox.com> > Cc: dpdk-dev <dev@dpdk.org>; jer@marvell.com; Jerin Jacob > <jerinjacobk@gmail.com>; NBU-Contact-Thomas Monjalon > <thomas@monjalon.net>; Ferruh Yigit <ferruh.yigit@intel.com>; Stephen > Hemminger <stephen@networkplumber.org>; Bruce Richardson > <bruce.richardson@intel.com>; Ori Kam <orika@mellanox.com>; Viacheslav > Ovsiienko <viacheslavo@mellanox.com>; andrey.vesnovaty@gmail.com; Ray > Kinsella <mdr@ashroe.eu>; Neil Horman <nhorman@tuxdriver.com>; Andrew > Rybchenko <arybchenko@solarflare.com> > Subject: Re: [dpdk-dev] [PATCH v2 1/6] ethdev: add flow shared action API > > > > On Wed, Jul 8, 2020 at 2:40 PM Andrey Vesnovaty <andreyv@mellanox.com > <mailto:andreyv@mellanox.com> > wrote: > > > From: Andrey Vesnovaty <andrey.vesnovaty@gmail.com > <mailto:andrey.vesnovaty@gmail.com> > > > This commit introduces extension of DPDK flow action API enabling > sharing of single rte_flow_action in multiple flows. The API intended for > PMDs where multiple HW offloaded flows can reuse the same HW > essence/object representing flow action and modification of such an > essence/object effects all the rules using it. > > Motivation and example > === > Adding or removing one or more queues to RSS used by multiple flow > rules > imposes per rule toll for current DPDK flow API; the scenario requires > for each flow sharing cloned RSS action: > - call `rte_flow_destroy()` > - call `rte_flow_create()` with modified RSS action > > API for sharing action and its in-place update benefits: > - reduce the overhead of multiple RSS flow rules reconfiguration > - optimize resource utilization by sharing action across of multiple > flows > > Change description > === > > Shared action > === > In order to represent flow action shared by multiple flows new action > type RTE_FLOW_ACTION_TYPE_SHARED is introduced (see `enum > rte_flow_action_type`). > Actually the introduced API decouples action from any specific flow and > enables sharing of single action by its handle across multiple flows. > > Shared action create/use/destroy > === > Shared action may be reused by some or none flow rules at any given > moment, i.e. shared action reside outside of the context of any flow. > Shared action represent HW resources/objects used for action > offloading > implementation. > API for shared action create (see `rte_flow_shared_action_create()`): > - should allocate HW resources and make related initializations required > for shared action implementation. > - make necessary preparations to maintain shared access to > the action resources, configuration and state. > API for shared action destroy (see `rte_flow_shared_action_destroy()`) > should release HW resources and make related cleanups required for > shared > action implementation. > > In order to share some flow action reuse the handle of type > `struct rte_flow_shared_action` returned by > rte_flow_shared_action_create() as a `conf` field of > `struct rte_flow_action` (see "example" section). > > If some shared action not used by any flow rule all resources allocated > by the shared action can be released by > rte_flow_shared_action_destroy() > (see "example" section). The shared action handle passed as argument > to > destroy API should not be used any further i.e. result of the usage is > undefined. > > Shared action re-configuration > === > Shared action behavior defined by its configuration can be updated via > rte_flow_shared_action_update() (see "example" section). The shared > action update operation modifies HW related resources/objects > allocated > on the action creation. The number of operations performed by the > update > operation should not be dependent on number of flows sharing the > related > action. On return of shared action update API action behavior should be > according to updated configuration for all flows sharing the action. > > Shared action query > === > Provide separate API to query shared action sate (see > rte_flow_shared_action_update()). Taking a counter as an example: > query > returns value aggregating all counter increments across all flow rules > sharing the counter. > > PMD support > === > The support of introduced API is pure PMD specific design and > responsibility for each action type (see struct rte_flow_ops). > > testpmd > === > In order to utilize introduced API testpmd cli may implement following > extension > create/update/destroy/query shared action accordingly > > flow shared_action create {port_id} [index] {action} > flow shared_action update {port_id} {index} {action} > flow shared_action destroy {port_id} {index} > flow shared_action query {port_id} {index} > > testpmd example > === > > configure rss to queues 1 & 2 > > testpmd> flow shared_action create 0 100 rss 1 2 > > create flow rule utilizing shared action > > testpmd> flow create 0 ingress \ > pattern eth dst is 0c:42:a1:15:fd:ac / ipv6 / tcp / end \ > actions shared 100 end / end > > add 2 more queues > > testpmd> flow shared_action modify 0 100 rss 1 2 3 4 > > example > === > > struct rte_flow_action actions[2]; > struct rte_flow_action action; > /* skipped: initialize action */ > struct rte_flow_shared_action *handle = > rte_flow_shared_action_create( > port_id, &action, &error); > actions[0].type = RTE_FLOW_ACTION_TYPE_SHARED; > actions[0].conf = handle; > actions[1].type = RTE_FLOW_ACTION_TYPE_END; > /* skipped: init attr0 & pattern0 args */ > struct rte_flow *flow0 = rte_flow_create(port_id, &attr0, pattern0, > actions, error); > /* create more rules reusing shared action */ > struct rte_flow *flow1 = rte_flow_create(port_id, &attr1, pattern1, > actions, error); > /* skipped: for flows 2 till N */ > struct rte_flow *flowN = rte_flow_create(port_id, &attrN, patternN, > actions, error); > /* update shared action */ > struct rte_flow_action updated_action; > /* > * skipped: initialize updated_action according to desired action > * configuration change > */ > rte_flow_shared_action_update(port_id, handle, &updated_action, > error); > /* > * from now on all flows 1 till N will act according to configuration of > * updated_action > */ > /* skipped: destroy all flows 1 till N */ > rte_flow_shared_action_destroy(port_id, handle, error); > > Signed-off-by: Andrey Vesnovaty <andreyv@mellanox.com > <mailto:andreyv@mellanox.com> > > --- > lib/librte_ethdev/rte_ethdev_version.map | 6 + > lib/librte_ethdev/rte_flow.c | 81 +++++++++++++ > lib/librte_ethdev/rte_flow.h | 148 ++++++++++++++++++++++- > lib/librte_ethdev/rte_flow_driver.h | 22 ++++ > 4 files changed, 256 insertions(+), 1 deletion(-) > > diff --git a/lib/librte_ethdev/rte_ethdev_version.map > b/lib/librte_ethdev/rte_ethdev_version.map > index 7155056045..119d84976a 100644 > --- a/lib/librte_ethdev/rte_ethdev_version.map > +++ b/lib/librte_ethdev/rte_ethdev_version.map > @@ -241,4 +241,10 @@ EXPERIMENTAL { > __rte_ethdev_trace_rx_burst; > __rte_ethdev_trace_tx_burst; > rte_flow_get_aged_flows; > + > + # added in 20.08 > + rte_flow_shared_action_create; > + rte_flow_shared_action_destroy; > + rte_flow_shared_action_update; > + rte_flow_shared_action_query; > }; > diff --git a/lib/librte_ethdev/rte_flow.c b/lib/librte_ethdev/rte_flow.c > index 1685be5f73..0ac4d31a13 100644 > --- a/lib/librte_ethdev/rte_flow.c > +++ b/lib/librte_ethdev/rte_flow.c > @@ -1250,3 +1250,84 @@ rte_flow_get_aged_flows(uint16_t port_id, > void **contexts, > RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > NULL, rte_strerror(ENOTSUP)); > } > + > +struct rte_flow_shared_action * > +rte_flow_shared_action_create(uint16_t port_id, > + const struct rte_flow_action *action, > > > It will be good to have an attributes argument here. > Some hardware devices may have ingress and egress resource pools. > So a 'direction' attribute can help share the resource effectively. I understand the idea of HW ingress/egress resource separation. Unfortunately on shared action creation it's not defined if it will be used in ingress or egress flow or both. Is the suggestion is to restrict usage of shared action to single direction? > > > > + struct rte_flow_error *error) > +{ > + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; > + struct rte_flow_shared_action *shared_action; > + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error); > + > + if (unlikely(!ops)) > + return NULL; > + if (likely(!!ops->shared_action_create)) { > + shared_action = ops->shared_action_create(dev, action, > error); > + if (shared_action == NULL) > + flow_err(port_id, -rte_errno, error); > + return shared_action; > + } > + rte_flow_error_set(error, ENOSYS, > RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > + NULL, rte_strerror(ENOSYS)); > + return NULL; > +} > + > +int > +rte_flow_shared_action_destroy(uint16_t port_id, > + struct rte_flow_shared_action *action, > + struct rte_flow_error *error) > +{ > + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; > + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error); > + > + if (unlikely(!ops)) > + return -rte_errno; > + if (likely(!!ops->shared_action_destroy)) > + return flow_err(port_id, > + ops->shared_action_destroy(dev, action, error), > + error); > + return rte_flow_error_set(error, ENOSYS, > + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > + NULL, rte_strerror(ENOSYS)); > +} > + > +int > +rte_flow_shared_action_update(uint16_t port_id, > + struct rte_flow_shared_action *action, > + const struct rte_flow_action *update, > + struct rte_flow_error *error) > +{ > + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; > + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error); > + > + if (unlikely(!ops)) > + return -rte_errno; > + if (likely(!!ops->shared_action_update)) > + return flow_err(port_id, ops->shared_action_update(dev, > action, > + update, error), > + error); > + return rte_flow_error_set(error, ENOSYS, > + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > + NULL, rte_strerror(ENOSYS)); > +} > + > +int > +rte_flow_shared_action_query(uint16_t port_id, > + const struct rte_flow_shared_action *action, > + void *data, > + struct rte_flow_error *error) > +{ > + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; > + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error); > + > + if (unlikely(!ops)) > + return -rte_errno; > + if (likely(!!ops->shared_action_query)) > + return flow_err(port_id, ops->shared_action_query(dev, > action, > + data, error), > + error); > + return rte_flow_error_set(error, ENOSYS, > + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > + NULL, rte_strerror(ENOSYS)); > +} > diff --git a/lib/librte_ethdev/rte_flow.h b/lib/librte_ethdev/rte_flow.h > index b0e4199192..257456b14a 100644 > --- a/lib/librte_ethdev/rte_flow.h > +++ b/lib/librte_ethdev/rte_flow.h > @@ -1681,7 +1681,8 @@ enum rte_flow_action_type { > /** > * Enables counters for this flow rule. > * > - * These counters can be retrieved and reset through > rte_flow_query(), > + * These counters can be retrieved and reset through > rte_flow_query() or > + * rte_flow_shared_action_query() if the action provided via > handle, > * see struct rte_flow_query_count. > * > * See struct rte_flow_action_count. > @@ -2099,6 +2100,14 @@ enum rte_flow_action_type { > * see enum RTE_ETH_EVENT_FLOW_AGED > */ > RTE_FLOW_ACTION_TYPE_AGE, > + > + /** > + * Describes action shared a cross multiple flow rules. > + * > + * Enables multiple rules reference the same action by handle (see > + * struct rte_flow_shared_action). > + */ > + RTE_FLOW_ACTION_TYPE_SHARED, > }; > > /** > @@ -2660,6 +2669,20 @@ struct rte_flow_action_set_dscp { > uint8_t dscp; > }; > > + > +/** > + * RTE_FLOW_ACTION_TYPE_SHARED > + * > + * Opaque type returned after successfully creating a shared action. > + * > + * This handle can be used to manage and query the related action: > + * - share it a cross multiple flow rules > + * - update action configuration > + * - query action data > + * - destroy action > + */ > +struct rte_flow_shared_action; > + > /* Mbuf dynamic field offset for metadata. */ > extern int32_t rte_flow_dynf_metadata_offs; > > @@ -3324,6 +3347,129 @@ int > rte_flow_get_aged_flows(uint16_t port_id, void **contexts, > uint32_t nb_contexts, struct rte_flow_error *error); > > +/** > + * @warning > + * @b EXPERIMENTAL: this API may change without prior notice. > + * > + * Create shared action for reuse in multiple flow rules. > + * > + * @param[in] port_id > + * The port identifier of the Ethernet device. > + * @param[in] action > + * Action configuration for shared action creation. > + * @param[out] error > + * Perform verbose error reporting if not NULL. PMDs initialize this > + * structure in case of error only. > + * @return > + * A valid handle in case of success, NULL otherwise and rte_errno is > set > + * to one of the error codes defined: > + * - (ENOSYS) if underlying device does not support this functionality. > + * - (EIO) if underlying device is removed. > + * - (EINVAL) if *action* invalid. > + * - (ENOTSUP) if *action* valid but unsupported. > + */ > +__rte_experimental > +struct rte_flow_shared_action * > +rte_flow_shared_action_create(uint16_t port_id, > + const struct rte_flow_action *action, > + struct rte_flow_error *error); > + > +/** > + * @warning > + * @b EXPERIMENTAL: this API may change without prior notice. > + * > + * Destroys the shared action by handle. > + * > + * @param[in] port_id > + * The port identifier of the Ethernet device. > + * @param[in] action > + * Handle for the shared action to be destroyed. > + * @param[out] error > + * Perform verbose error reporting if not NULL. PMDs initialize this > + * structure in case of error only. > + * @return > + * - (0) if success. > + * - (-ENOSYS) if underlying device does not support this functionality. > + * - (-EIO) if underlying device is removed. > + * - (-ENOENT) if action pointed by *action* handle was not found. > + * - (-ETOOMANYREFS) if action pointed by *action* handle still used > by one or > + * more rules > + * rte_errno is also set. > + */ > +__rte_experimental > +int > +rte_flow_shared_action_destroy(uint16_t port_id, > + struct rte_flow_shared_action *action, > + struct rte_flow_error *error); > + > +/** > + * @warning > + * @b EXPERIMENTAL: this API may change without prior notice. > + * > + * Updates inplace the shared action configuration pointed by *action* > handle > + * with the configuration provided as *update* argument. > + * The update of the shared action configuration effects all flow rules > reusing > + * the action via handle. > + * > + * @param[in] port_id > + * The port identifier of the Ethernet device. > + * @param[in] action > + * Handle for the shared action to be updated. > + * @param[in] update > + * Action specification used to modify the action pointed by handle. > + * *update* should be of same type with the action pointed by the > *action* > + * handle argument, otherwise considered as invalid. > + * @param[out] error > + * Perform verbose error reporting if not NULL. PMDs initialize this > + * structure in case of error only. > + * @return > + * - (0) if success. > + * - (-ENOSYS) if underlying device does not support this functionality. > + * - (-EIO) if underlying device is removed. > + * - (-EINVAL) if *update* invalid. > + * - (-ENOTSUP) if *update* valid but unsupported. > + * - (-ENOENT) if action pointed by *ctx* was not found. > + * rte_errno is also set. > + */ > +__rte_experimental > +int > +rte_flow_shared_action_update(uint16_t port_id, > + struct rte_flow_shared_action *action, > + const struct rte_flow_action *update, > + struct rte_flow_error *error); > + > +/** > + * @warning > + * @b EXPERIMENTAL: this API may change without prior notice. > + * > + * Query the shared action by handle. > + * > + * This function allows retrieving action-specific data such as counters. > + * Data is gathered by special action which may be present/referenced > in > + * more than one flow rule definition. > + * > + * \see RTE_FLOW_ACTION_TYPE_COUNT > + * > + * @param port_id > + * Port identifier of Ethernet device. > + * @param[in] action > + * Handle for the shared action to query. > + * @param[in, out] data > + * Pointer to storage for the associated query data type. > + * @param[out] error > + * Perform verbose error reporting if not NULL. PMDs initialize this > + * structure in case of error only. > + * > + * @return > + * 0 on success, a negative errno value otherwise and rte_errno is set. > + */ > +__rte_experimental > +int > +rte_flow_shared_action_query(uint16_t port_id, > + const struct rte_flow_shared_action *action, > + void *data, > + struct rte_flow_error *error); > + > #ifdef __cplusplus > } > #endif > diff --git a/lib/librte_ethdev/rte_flow_driver.h > b/lib/librte_ethdev/rte_flow_driver.h > index 881cc469b7..a2cae1b53c 100644 > --- a/lib/librte_ethdev/rte_flow_driver.h > +++ b/lib/librte_ethdev/rte_flow_driver.h > @@ -107,6 +107,28 @@ struct rte_flow_ops { > void **context, > uint32_t nb_contexts, > struct rte_flow_error *err); > + /** See rte_flow_shared_action_create() */ > + struct rte_flow_shared_action *(*shared_action_create) > + (struct rte_eth_dev *dev, > + const struct rte_flow_action *action, > + struct rte_flow_error *error); > + /** See rte_flow_shared_action_destroy() */ > + int (*shared_action_destroy) > + (struct rte_eth_dev *dev, > + struct rte_flow_shared_action *shared_action, > + struct rte_flow_error *error); > + /** See rte_flow_shared_action_update() */ > + int (*shared_action_update) > + (struct rte_eth_dev *dev, > + struct rte_flow_shared_action *shared_action, > + const struct rte_flow_action *update, > + struct rte_flow_error *error); > + /** See rte_flow_shared_action_query() */ > + int (*shared_action_query) > + (struct rte_eth_dev *dev, > + const struct rte_flow_shared_action *shared_action, > + void *data, > + struct rte_flow_error *error); > }; > > /** > -- > 2.26.2 > > ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [dpdk-dev] [PATCH v2 1/6] ethdev: add flow shared action API 2020-09-15 11:50 ` Andrey Vesnovaty @ 2020-09-15 15:49 ` Ajit Khaparde 2020-09-16 15:52 ` Andrey Vesnovaty 0 siblings, 1 reply; 106+ messages in thread From: Ajit Khaparde @ 2020-09-15 15:49 UTC (permalink / raw) To: Andrey Vesnovaty Cc: NBU-Contact-Thomas Monjalon, Ori Kam, Slava Ovsiienko, jerinj, Andrew Rybchenko, dpdk-dev, jer, Jerin Jacob, Ferruh Yigit, Stephen Hemminger, Bruce Richardson, Ori Kam, Viacheslav Ovsiienko, andrey.vesnovaty, Ray Kinsella, Neil Horman, Thomas Monjalon, Samik Gupta On Tue, Sep 15, 2020 at 4:51 AM Andrey Vesnovaty <andreyv@nvidia.com> wrote: > Hi Ajit, PSB. > > Thanks, > Andrey > > > -----Original Message----- > > From: Ajit Khaparde <ajit.khaparde@broadcom.com> > > Sent: Saturday, September 12, 2020 5:18 AM > > To: Andrey Vesnovaty <andreyv@mellanox.com> > > Cc: dpdk-dev <dev@dpdk.org>; jer@marvell.com; Jerin Jacob > > <jerinjacobk@gmail.com>; NBU-Contact-Thomas Monjalon > > <thomas@monjalon.net>; Ferruh Yigit <ferruh.yigit@intel.com>; Stephen > > Hemminger <stephen@networkplumber.org>; Bruce Richardson > > <bruce.richardson@intel.com>; Ori Kam <orika@mellanox.com>; Viacheslav > > Ovsiienko <viacheslavo@mellanox.com>; andrey.vesnovaty@gmail.com; Ray > > Kinsella <mdr@ashroe.eu>; Neil Horman <nhorman@tuxdriver.com>; Andrew > > Rybchenko <arybchenko@solarflare.com> > > Subject: Re: [dpdk-dev] [PATCH v2 1/6] ethdev: add flow shared action API > > > > > > > > On Wed, Jul 8, 2020 at 2:40 PM Andrey Vesnovaty <andreyv@mellanox.com > > <mailto:andreyv@mellanox.com> > wrote: > > > > > > From: Andrey Vesnovaty <andrey.vesnovaty@gmail.com > > <mailto:andrey.vesnovaty@gmail.com> > > > > > This commit introduces extension of DPDK flow action API enabling > > sharing of single rte_flow_action in multiple flows. The API > intended for > > PMDs where multiple HW offloaded flows can reuse the same HW > > essence/object representing flow action and modification of such an > > essence/object effects all the rules using it. > > > > Motivation and example > > === > > Adding or removing one or more queues to RSS used by multiple flow > > rules > > imposes per rule toll for current DPDK flow API; the scenario > requires > > for each flow sharing cloned RSS action: > > - call `rte_flow_destroy()` > > - call `rte_flow_create()` with modified RSS action > > > > API for sharing action and its in-place update benefits: > > - reduce the overhead of multiple RSS flow rules reconfiguration > > - optimize resource utilization by sharing action across of > multiple > > flows > > > > Change description > > === > > > > Shared action > > === > > In order to represent flow action shared by multiple flows new > action > > type RTE_FLOW_ACTION_TYPE_SHARED is introduced (see `enum > > rte_flow_action_type`). > > Actually the introduced API decouples action from any specific > flow and > > enables sharing of single action by its handle across multiple > flows. > > > > Shared action create/use/destroy > > === > > Shared action may be reused by some or none flow rules at any given > > moment, i.e. shared action reside outside of the context of any > flow. > > Shared action represent HW resources/objects used for action > > offloading > > implementation. > > API for shared action create (see > `rte_flow_shared_action_create()`): > > - should allocate HW resources and make related initializations > required > > for shared action implementation. > > - make necessary preparations to maintain shared access to > > the action resources, configuration and state. > > API for shared action destroy (see > `rte_flow_shared_action_destroy()`) > > should release HW resources and make related cleanups required for > > shared > > action implementation. > > > > In order to share some flow action reuse the handle of type > > `struct rte_flow_shared_action` returned by > > rte_flow_shared_action_create() as a `conf` field of > > `struct rte_flow_action` (see "example" section). > > > > If some shared action not used by any flow rule all resources > allocated > > by the shared action can be released by > > rte_flow_shared_action_destroy() > > (see "example" section). The shared action handle passed as > argument > > to > > destroy API should not be used any further i.e. result of the > usage is > > undefined. > > > > Shared action re-configuration > > === > > Shared action behavior defined by its configuration can be updated > via > > rte_flow_shared_action_update() (see "example" section). The shared > > action update operation modifies HW related resources/objects > > allocated > > on the action creation. The number of operations performed by the > > update > > operation should not be dependent on number of flows sharing the > > related > > action. On return of shared action update API action behavior > should be > > according to updated configuration for all flows sharing the > action. > > > > Shared action query > > === > > Provide separate API to query shared action sate (see > > rte_flow_shared_action_update()). Taking a counter as an example: > > query > > returns value aggregating all counter increments across all flow > rules > > sharing the counter. > > > > PMD support > > === > > The support of introduced API is pure PMD specific design and > > responsibility for each action type (see struct rte_flow_ops). > > > > testpmd > > === > > In order to utilize introduced API testpmd cli may implement > following > > extension > > create/update/destroy/query shared action accordingly > > > > flow shared_action create {port_id} [index] {action} > > flow shared_action update {port_id} {index} {action} > > flow shared_action destroy {port_id} {index} > > flow shared_action query {port_id} {index} > > > > testpmd example > > === > > > > configure rss to queues 1 & 2 > > > > testpmd> flow shared_action create 0 100 rss 1 2 > > > > create flow rule utilizing shared action > > > > testpmd> flow create 0 ingress \ > > pattern eth dst is 0c:42:a1:15:fd:ac / ipv6 / tcp / end \ > > actions shared 100 end / end > > > > add 2 more queues > > > > testpmd> flow shared_action modify 0 100 rss 1 2 3 4 > > > > example > > === > > > > struct rte_flow_action actions[2]; > > struct rte_flow_action action; > > /* skipped: initialize action */ > > struct rte_flow_shared_action *handle = > > rte_flow_shared_action_create( > > port_id, &action, &error); > > actions[0].type = RTE_FLOW_ACTION_TYPE_SHARED; > > actions[0].conf = handle; > > actions[1].type = RTE_FLOW_ACTION_TYPE_END; > > /* skipped: init attr0 & pattern0 args */ > > struct rte_flow *flow0 = rte_flow_create(port_id, &attr0, pattern0, > > actions, error); > > /* create more rules reusing shared action */ > > struct rte_flow *flow1 = rte_flow_create(port_id, &attr1, pattern1, > > actions, error); > > /* skipped: for flows 2 till N */ > > struct rte_flow *flowN = rte_flow_create(port_id, &attrN, patternN, > > actions, error); > > /* update shared action */ > > struct rte_flow_action updated_action; > > /* > > * skipped: initialize updated_action according to desired action > > * configuration change > > */ > > rte_flow_shared_action_update(port_id, handle, &updated_action, > > error); > > /* > > * from now on all flows 1 till N will act according to > configuration of > > * updated_action > > */ > > /* skipped: destroy all flows 1 till N */ > > rte_flow_shared_action_destroy(port_id, handle, error); > > > > Signed-off-by: Andrey Vesnovaty <andreyv@mellanox.com > > <mailto:andreyv@mellanox.com> > > > --- > > lib/librte_ethdev/rte_ethdev_version.map | 6 + > > lib/librte_ethdev/rte_flow.c | 81 +++++++++++++ > > lib/librte_ethdev/rte_flow.h | 148 > ++++++++++++++++++++++- > > lib/librte_ethdev/rte_flow_driver.h | 22 ++++ > > 4 files changed, 256 insertions(+), 1 deletion(-) > > > > diff --git a/lib/librte_ethdev/rte_ethdev_version.map > > b/lib/librte_ethdev/rte_ethdev_version.map > > index 7155056045..119d84976a 100644 > > --- a/lib/librte_ethdev/rte_ethdev_version.map > > +++ b/lib/librte_ethdev/rte_ethdev_version.map > > @@ -241,4 +241,10 @@ EXPERIMENTAL { > > __rte_ethdev_trace_rx_burst; > > __rte_ethdev_trace_tx_burst; > > rte_flow_get_aged_flows; > > + > > + # added in 20.08 > > + rte_flow_shared_action_create; > > + rte_flow_shared_action_destroy; > > + rte_flow_shared_action_update; > > + rte_flow_shared_action_query; > > }; > > diff --git a/lib/librte_ethdev/rte_flow.c > b/lib/librte_ethdev/rte_flow.c > > index 1685be5f73..0ac4d31a13 100644 > > --- a/lib/librte_ethdev/rte_flow.c > > +++ b/lib/librte_ethdev/rte_flow.c > > @@ -1250,3 +1250,84 @@ rte_flow_get_aged_flows(uint16_t port_id, > > void **contexts, > > RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > > NULL, rte_strerror(ENOTSUP)); > > } > > + > > +struct rte_flow_shared_action * > > +rte_flow_shared_action_create(uint16_t port_id, > > + const struct rte_flow_action *action, > > > > > > It will be good to have an attributes argument here. > > Some hardware devices may have ingress and egress resource pools. > > So a 'direction' attribute can help share the resource effectively. > > I understand the idea of HW ingress/egress resource separation. > Unfortunately on shared action creation it's not defined if it will be used > in ingress or egress flow or both. > Is the suggestion is to restrict usage of shared action to single > direction? > In a way, it is more to add flexibility to this proposal. Specifying the direction explicitly is better than PMD parsing the actions and determining if it is for ingress or egress. If hardware does not have specific pools for either ingress or egress, the PMD can ignore the attribute. So it will be upto the PMD and the underlying hardware to use the direction attribute - if necessary. > > > > > > > > + struct rte_flow_error *error) > > +{ > > + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; > > + struct rte_flow_shared_action *shared_action; > > + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, > error); > > + > > + if (unlikely(!ops)) > > + return NULL; > > + if (likely(!!ops->shared_action_create)) { > > + shared_action = ops->shared_action_create(dev, > action, > > error); > > + if (shared_action == NULL) > > + flow_err(port_id, -rte_errno, error); > > + return shared_action; > > + } > > + rte_flow_error_set(error, ENOSYS, > > RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > > + NULL, rte_strerror(ENOSYS)); > > + return NULL; > > +} > > + > > +int > > +rte_flow_shared_action_destroy(uint16_t port_id, > > + struct rte_flow_shared_action > *action, > > + struct rte_flow_error *error) > > +{ > > + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; > > + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, > error); > > + > > + if (unlikely(!ops)) > > + return -rte_errno; > > + if (likely(!!ops->shared_action_destroy)) > > + return flow_err(port_id, > > + ops->shared_action_destroy(dev, > action, error), > > + error); > > + return rte_flow_error_set(error, ENOSYS, > > + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > > + NULL, rte_strerror(ENOSYS)); > > +} > > + > > +int > > +rte_flow_shared_action_update(uint16_t port_id, > > + struct rte_flow_shared_action > *action, > > + const struct rte_flow_action *update, > > + struct rte_flow_error *error) > > +{ > > + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; > > + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, > error); > > + > > + if (unlikely(!ops)) > > + return -rte_errno; > > + if (likely(!!ops->shared_action_update)) > > + return flow_err(port_id, > ops->shared_action_update(dev, > > action, > > + update, error), > > + error); > > + return rte_flow_error_set(error, ENOSYS, > > + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > > + NULL, rte_strerror(ENOSYS)); > > +} > > + > > +int > > +rte_flow_shared_action_query(uint16_t port_id, > > + const struct rte_flow_shared_action > *action, > > + void *data, > > + struct rte_flow_error *error) > > +{ > > + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; > > + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, > error); > > + > > + if (unlikely(!ops)) > > + return -rte_errno; > > + if (likely(!!ops->shared_action_query)) > > + return flow_err(port_id, > ops->shared_action_query(dev, > > action, > > + data, error), > > + error); > > + return rte_flow_error_set(error, ENOSYS, > > + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > > + NULL, rte_strerror(ENOSYS)); > > +} > > diff --git a/lib/librte_ethdev/rte_flow.h > b/lib/librte_ethdev/rte_flow.h > > index b0e4199192..257456b14a 100644 > > --- a/lib/librte_ethdev/rte_flow.h > > +++ b/lib/librte_ethdev/rte_flow.h > > @@ -1681,7 +1681,8 @@ enum rte_flow_action_type { > > /** > > * Enables counters for this flow rule. > > * > > - * These counters can be retrieved and reset through > > rte_flow_query(), > > + * These counters can be retrieved and reset through > > rte_flow_query() or > > + * rte_flow_shared_action_query() if the action provided > via > > handle, > > * see struct rte_flow_query_count. > > * > > * See struct rte_flow_action_count. > > @@ -2099,6 +2100,14 @@ enum rte_flow_action_type { > > * see enum RTE_ETH_EVENT_FLOW_AGED > > */ > > RTE_FLOW_ACTION_TYPE_AGE, > > + > > + /** > > + * Describes action shared a cross multiple flow rules. > > + * > > + * Enables multiple rules reference the same action by > handle (see > > + * struct rte_flow_shared_action). > > + */ > > + RTE_FLOW_ACTION_TYPE_SHARED, > > }; > > > > /** > > @@ -2660,6 +2669,20 @@ struct rte_flow_action_set_dscp { > > uint8_t dscp; > > }; > > > > + > > +/** > > + * RTE_FLOW_ACTION_TYPE_SHARED > > + * > > + * Opaque type returned after successfully creating a shared > action. > > + * > > + * This handle can be used to manage and query the related action: > > + * - share it a cross multiple flow rules > > + * - update action configuration > > + * - query action data > > + * - destroy action > > + */ > > +struct rte_flow_shared_action; > > + > > /* Mbuf dynamic field offset for metadata. */ > > extern int32_t rte_flow_dynf_metadata_offs; > > > > @@ -3324,6 +3347,129 @@ int > > rte_flow_get_aged_flows(uint16_t port_id, void **contexts, > > uint32_t nb_contexts, struct > rte_flow_error *error); > > > > +/** > > + * @warning > > + * @b EXPERIMENTAL: this API may change without prior notice. > > + * > > + * Create shared action for reuse in multiple flow rules. > > + * > > + * @param[in] port_id > > + * The port identifier of the Ethernet device. > > + * @param[in] action > > + * Action configuration for shared action creation. > > + * @param[out] error > > + * Perform verbose error reporting if not NULL. PMDs initialize > this > > + * structure in case of error only. > > + * @return > > + * A valid handle in case of success, NULL otherwise and > rte_errno is > > set > > + * to one of the error codes defined: > > + * - (ENOSYS) if underlying device does not support this > functionality. > > + * - (EIO) if underlying device is removed. > > + * - (EINVAL) if *action* invalid. > > + * - (ENOTSUP) if *action* valid but unsupported. > > + */ > > +__rte_experimental > > +struct rte_flow_shared_action * > > +rte_flow_shared_action_create(uint16_t port_id, > > + const struct rte_flow_action *action, > > + struct rte_flow_error *error); > > + > > +/** > > + * @warning > > + * @b EXPERIMENTAL: this API may change without prior notice. > > + * > > + * Destroys the shared action by handle. > > + * > > + * @param[in] port_id > > + * The port identifier of the Ethernet device. > > + * @param[in] action > > + * Handle for the shared action to be destroyed. > > + * @param[out] error > > + * Perform verbose error reporting if not NULL. PMDs initialize > this > > + * structure in case of error only. > > + * @return > > + * - (0) if success. > > + * - (-ENOSYS) if underlying device does not support this > functionality. > > + * - (-EIO) if underlying device is removed. > > + * - (-ENOENT) if action pointed by *action* handle was not > found. > > + * - (-ETOOMANYREFS) if action pointed by *action* handle still > used > > by one or > > + * more rules > > + * rte_errno is also set. > > + */ > > +__rte_experimental > > +int > > +rte_flow_shared_action_destroy(uint16_t port_id, > > + struct rte_flow_shared_action > *action, > > + struct rte_flow_error *error); > > + > > +/** > > + * @warning > > + * @b EXPERIMENTAL: this API may change without prior notice. > > + * > > + * Updates inplace the shared action configuration pointed by > *action* > > handle > > + * with the configuration provided as *update* argument. > > + * The update of the shared action configuration effects all flow > rules > > reusing > > + * the action via handle. > > + * > > + * @param[in] port_id > > + * The port identifier of the Ethernet device. > > + * @param[in] action > > + * Handle for the shared action to be updated. > > + * @param[in] update > > + * Action specification used to modify the action pointed by > handle. > > + * *update* should be of same type with the action pointed by > the > > *action* > > + * handle argument, otherwise considered as invalid. > > + * @param[out] error > > + * Perform verbose error reporting if not NULL. PMDs initialize > this > > + * structure in case of error only. > > + * @return > > + * - (0) if success. > > + * - (-ENOSYS) if underlying device does not support this > functionality. > > + * - (-EIO) if underlying device is removed. > > + * - (-EINVAL) if *update* invalid. > > + * - (-ENOTSUP) if *update* valid but unsupported. > > + * - (-ENOENT) if action pointed by *ctx* was not found. > > + * rte_errno is also set. > > + */ > > +__rte_experimental > > +int > > +rte_flow_shared_action_update(uint16_t port_id, > > + struct rte_flow_shared_action > *action, > > + const struct rte_flow_action *update, > > + struct rte_flow_error *error); > > + > > +/** > > + * @warning > > + * @b EXPERIMENTAL: this API may change without prior notice. > > + * > > + * Query the shared action by handle. > > + * > > + * This function allows retrieving action-specific data such as > counters. > > + * Data is gathered by special action which may be > present/referenced > > in > > + * more than one flow rule definition. > > + * > > + * \see RTE_FLOW_ACTION_TYPE_COUNT > > + * > > + * @param port_id > > + * Port identifier of Ethernet device. > > + * @param[in] action > > + * Handle for the shared action to query. > > + * @param[in, out] data > > + * Pointer to storage for the associated query data type. > > + * @param[out] error > > + * Perform verbose error reporting if not NULL. PMDs initialize > this > > + * structure in case of error only. > > + * > > + * @return > > + * 0 on success, a negative errno value otherwise and rte_errno > is set. > > + */ > > +__rte_experimental > > +int > > +rte_flow_shared_action_query(uint16_t port_id, > > + const struct rte_flow_shared_action > *action, > > + void *data, > > + struct rte_flow_error *error); > > + > > #ifdef __cplusplus > > } > > #endif > > diff --git a/lib/librte_ethdev/rte_flow_driver.h > > b/lib/librte_ethdev/rte_flow_driver.h > > index 881cc469b7..a2cae1b53c 100644 > > --- a/lib/librte_ethdev/rte_flow_driver.h > > +++ b/lib/librte_ethdev/rte_flow_driver.h > > @@ -107,6 +107,28 @@ struct rte_flow_ops { > > void **context, > > uint32_t nb_contexts, > > struct rte_flow_error *err); > > + /** See rte_flow_shared_action_create() */ > > + struct rte_flow_shared_action *(*shared_action_create) > > + (struct rte_eth_dev *dev, > > + const struct rte_flow_action *action, > > + struct rte_flow_error *error); > > + /** See rte_flow_shared_action_destroy() */ > > + int (*shared_action_destroy) > > + (struct rte_eth_dev *dev, > > + struct rte_flow_shared_action *shared_action, > > + struct rte_flow_error *error); > > + /** See rte_flow_shared_action_update() */ > > + int (*shared_action_update) > > + (struct rte_eth_dev *dev, > > + struct rte_flow_shared_action *shared_action, > > + const struct rte_flow_action *update, > > + struct rte_flow_error *error); > > + /** See rte_flow_shared_action_query() */ > > + int (*shared_action_query) > > + (struct rte_eth_dev *dev, > > + const struct rte_flow_shared_action > *shared_action, > > + void *data, > > + struct rte_flow_error *error); > > }; > > > > /** > > -- > > 2.26.2 > > > > > > ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [dpdk-dev] [PATCH v2 1/6] ethdev: add flow shared action API 2020-09-15 15:49 ` Ajit Khaparde @ 2020-09-16 15:52 ` Andrey Vesnovaty 2020-09-16 19:20 ` Ajit Khaparde 0 siblings, 1 reply; 106+ messages in thread From: Andrey Vesnovaty @ 2020-09-16 15:52 UTC (permalink / raw) To: Ajit Khaparde Cc: NBU-Contact-Thomas Monjalon, Ori Kam, Slava Ovsiienko, jerinj, Andrew Rybchenko, dpdk-dev, jer, Jerin Jacob, Ferruh Yigit, Stephen Hemminger, Bruce Richardson, Ori Kam, Viacheslav Ovsiienko, andrey.vesnovaty, Ray Kinsella, Neil Horman, Thomas Monjalon, Samik Gupta Hi Ajit For shared action configuration I have following suggestion: struct rte_flow_shared_action_conf { uint32_t no_ingress: 1; uint32_t no_egress: 1; }; /*...*/ rte_flow_shared_action_create(..., const struct rte_flow_shared_action_conf *conf, ...); What do you think? Thanks, Andrey > -----Original Message----- > From: Ajit Khaparde <ajit.khaparde@broadcom.com> > Sent: Tuesday, September 15, 2020 6:49 PM > To: Andrey Vesnovaty <andreyv@nvidia.com> > Cc: NBU-Contact-Thomas Monjalon <thomas@monjalon.net>; Ori Kam > <orika@nvidia.com>; Slava Ovsiienko <viacheslavo@nvidia.com>; > jerinj@marvell.com; Andrew Rybchenko <arybchenko@solarflare.com>; dpdk- > dev <dev@dpdk.org>; jer@marvell.com; Jerin Jacob <jerinjacobk@gmail.com>; > Ferruh Yigit <ferruh.yigit@intel.com>; Stephen Hemminger > <stephen@networkplumber.org>; Bruce Richardson > <bruce.richardson@intel.com>; Ori Kam <orika@mellanox.com>; Viacheslav > Ovsiienko <viacheslavo@mellanox.com>; andrey.vesnovaty@gmail.com; Ray > Kinsella <mdr@ashroe.eu>; Neil Horman <nhorman@tuxdriver.com>; Thomas > Monjalon <tmonjalon@nvidia.com>; Samik Gupta > <samik.gupta@broadcom.com> > Subject: Re: [dpdk-dev] [PATCH v2 1/6] ethdev: add flow shared action API > > > > On Tue, Sep 15, 2020 at 4:51 AM Andrey Vesnovaty <andreyv@nvidia.com > <mailto:andreyv@nvidia.com> > wrote: > > > Hi Ajit, PSB. > > Thanks, > Andrey > > > -----Original Message----- > > From: Ajit Khaparde <ajit.khaparde@broadcom.com > <mailto:ajit.khaparde@broadcom.com> > > > Sent: Saturday, September 12, 2020 5:18 AM > > To: Andrey Vesnovaty <andreyv@mellanox.com > <mailto:andreyv@mellanox.com> > > > Cc: dpdk-dev <dev@dpdk.org <mailto:dev@dpdk.org> >; > jer@marvell.com <mailto:jer@marvell.com> ; Jerin Jacob > > <jerinjacobk@gmail.com <mailto:jerinjacobk@gmail.com> >; NBU- > Contact-Thomas Monjalon > > <thomas@monjalon.net <mailto:thomas@monjalon.net> >; Ferruh > Yigit <ferruh.yigit@intel.com <mailto:ferruh.yigit@intel.com> >; Stephen > > Hemminger <stephen@networkplumber.org > <mailto:stephen@networkplumber.org> >; Bruce Richardson > > <bruce.richardson@intel.com <mailto:bruce.richardson@intel.com> > >; Ori Kam <orika@mellanox.com <mailto:orika@mellanox.com> >; Viacheslav > > Ovsiienko <viacheslavo@mellanox.com > <mailto:viacheslavo@mellanox.com> >; andrey.vesnovaty@gmail.com > <mailto:andrey.vesnovaty@gmail.com> ; Ray > > Kinsella <mdr@ashroe.eu <mailto:mdr@ashroe.eu> >; Neil Horman > <nhorman@tuxdriver.com <mailto:nhorman@tuxdriver.com> >; Andrew > > Rybchenko <arybchenko@solarflare.com > <mailto:arybchenko@solarflare.com> > > > Subject: Re: [dpdk-dev] [PATCH v2 1/6] ethdev: add flow shared action > API > > > > > > > > On Wed, Jul 8, 2020 at 2:40 PM Andrey Vesnovaty > <andreyv@mellanox.com <mailto:andreyv@mellanox.com> > > <mailto:andreyv@mellanox.com <mailto:andreyv@mellanox.com> > > > wrote: > > > > > > From: Andrey Vesnovaty <andrey.vesnovaty@gmail.com > <mailto:andrey.vesnovaty@gmail.com> > > <mailto:andrey.vesnovaty@gmail.com > <mailto:andrey.vesnovaty@gmail.com> > > > > > > This commit introduces extension of DPDK flow action API enabling > > sharing of single rte_flow_action in multiple flows. The API > intended for > > PMDs where multiple HW offloaded flows can reuse the same HW > > essence/object representing flow action and modification of such > an > > essence/object effects all the rules using it. > > > > Motivation and example > > === > > Adding or removing one or more queues to RSS used by multiple > flow > > rules > > imposes per rule toll for current DPDK flow API; the scenario > requires > > for each flow sharing cloned RSS action: > > - call `rte_flow_destroy()` > > - call `rte_flow_create()` with modified RSS action > > > > API for sharing action and its in-place update benefits: > > - reduce the overhead of multiple RSS flow rules reconfiguration > > - optimize resource utilization by sharing action across of multiple > > flows > > > > Change description > > === > > > > Shared action > > === > > In order to represent flow action shared by multiple flows new > action > > type RTE_FLOW_ACTION_TYPE_SHARED is introduced (see `enum > > rte_flow_action_type`). > > Actually the introduced API decouples action from any specific flow > and > > enables sharing of single action by its handle across multiple flows. > > > > Shared action create/use/destroy > > === > > Shared action may be reused by some or none flow rules at any > given > > moment, i.e. shared action reside outside of the context of any > flow. > > Shared action represent HW resources/objects used for action > > offloading > > implementation. > > API for shared action create (see > `rte_flow_shared_action_create()`): > > - should allocate HW resources and make related initializations > required > > for shared action implementation. > > - make necessary preparations to maintain shared access to > > the action resources, configuration and state. > > API for shared action destroy (see > `rte_flow_shared_action_destroy()`) > > should release HW resources and make related cleanups required > for > > shared > > action implementation. > > > > In order to share some flow action reuse the handle of type > > `struct rte_flow_shared_action` returned by > > rte_flow_shared_action_create() as a `conf` field of > > `struct rte_flow_action` (see "example" section). > > > > If some shared action not used by any flow rule all resources > allocated > > by the shared action can be released by > > rte_flow_shared_action_destroy() > > (see "example" section). The shared action handle passed as > argument > > to > > destroy API should not be used any further i.e. result of the usage is > > undefined. > > > > Shared action re-configuration > > === > > Shared action behavior defined by its configuration can be updated > via > > rte_flow_shared_action_update() (see "example" section). The > shared > > action update operation modifies HW related resources/objects > > allocated > > on the action creation. The number of operations performed by > the > > update > > operation should not be dependent on number of flows sharing the > > related > > action. On return of shared action update API action behavior > should be > > according to updated configuration for all flows sharing the action. > > > > Shared action query > > === > > Provide separate API to query shared action sate (see > > rte_flow_shared_action_update()). Taking a counter as an > example: > > query > > returns value aggregating all counter increments across all flow > rules > > sharing the counter. > > > > PMD support > > === > > The support of introduced API is pure PMD specific design and > > responsibility for each action type (see struct rte_flow_ops). > > > > testpmd > > === > > In order to utilize introduced API testpmd cli may implement > following > > extension > > create/update/destroy/query shared action accordingly > > > > flow shared_action create {port_id} [index] {action} > > flow shared_action update {port_id} {index} {action} > > flow shared_action destroy {port_id} {index} > > flow shared_action query {port_id} {index} > > > > testpmd example > > === > > > > configure rss to queues 1 & 2 > > > > testpmd> flow shared_action create 0 100 rss 1 2 > > > > create flow rule utilizing shared action > > > > testpmd> flow create 0 ingress \ > > pattern eth dst is 0c:42:a1:15:fd:ac / ipv6 / tcp / end \ > > actions shared 100 end / end > > > > add 2 more queues > > > > testpmd> flow shared_action modify 0 100 rss 1 2 3 4 > > > > example > > === > > > > struct rte_flow_action actions[2]; > > struct rte_flow_action action; > > /* skipped: initialize action */ > > struct rte_flow_shared_action *handle = > > rte_flow_shared_action_create( > > port_id, &action, &error); > > actions[0].type = RTE_FLOW_ACTION_TYPE_SHARED; > > actions[0].conf = handle; > > actions[1].type = RTE_FLOW_ACTION_TYPE_END; > > /* skipped: init attr0 & pattern0 args */ > > struct rte_flow *flow0 = rte_flow_create(port_id, &attr0, > pattern0, > > actions, error); > > /* create more rules reusing shared action */ > > struct rte_flow *flow1 = rte_flow_create(port_id, &attr1, > pattern1, > > actions, error); > > /* skipped: for flows 2 till N */ > > struct rte_flow *flowN = rte_flow_create(port_id, &attrN, > patternN, > > actions, error); > > /* update shared action */ > > struct rte_flow_action updated_action; > > /* > > * skipped: initialize updated_action according to desired action > > * configuration change > > */ > > rte_flow_shared_action_update(port_id, handle, > &updated_action, > > error); > > /* > > * from now on all flows 1 till N will act according to configuration > of > > * updated_action > > */ > > /* skipped: destroy all flows 1 till N */ > > rte_flow_shared_action_destroy(port_id, handle, error); > > > > Signed-off-by: Andrey Vesnovaty <andreyv@mellanox.com > <mailto:andreyv@mellanox.com> > > <mailto:andreyv@mellanox.com <mailto:andreyv@mellanox.com> > > > > > --- > > lib/librte_ethdev/rte_ethdev_version.map | 6 + > > lib/librte_ethdev/rte_flow.c | 81 +++++++++++++ > > lib/librte_ethdev/rte_flow.h | 148 > ++++++++++++++++++++++- > > lib/librte_ethdev/rte_flow_driver.h | 22 ++++ > > 4 files changed, 256 insertions(+), 1 deletion(-) > > > > diff --git a/lib/librte_ethdev/rte_ethdev_version.map > > b/lib/librte_ethdev/rte_ethdev_version.map > > index 7155056045..119d84976a 100644 > > --- a/lib/librte_ethdev/rte_ethdev_version.map > > +++ b/lib/librte_ethdev/rte_ethdev_version.map > > @@ -241,4 +241,10 @@ EXPERIMENTAL { > > __rte_ethdev_trace_rx_burst; > > __rte_ethdev_trace_tx_burst; > > rte_flow_get_aged_flows; > > + > > + # added in 20.08 > > + rte_flow_shared_action_create; > > + rte_flow_shared_action_destroy; > > + rte_flow_shared_action_update; > > + rte_flow_shared_action_query; > > }; > > diff --git a/lib/librte_ethdev/rte_flow.c > b/lib/librte_ethdev/rte_flow.c > > index 1685be5f73..0ac4d31a13 100644 > > --- a/lib/librte_ethdev/rte_flow.c > > +++ b/lib/librte_ethdev/rte_flow.c > > @@ -1250,3 +1250,84 @@ rte_flow_get_aged_flows(uint16_t > port_id, > > void **contexts, > > RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > > NULL, rte_strerror(ENOTSUP)); > > } > > + > > +struct rte_flow_shared_action * > > +rte_flow_shared_action_create(uint16_t port_id, > > + const struct rte_flow_action *action, > > > > > > It will be good to have an attributes argument here. > > Some hardware devices may have ingress and egress resource pools. > > So a 'direction' attribute can help share the resource effectively. > > I understand the idea of HW ingress/egress resource separation. > Unfortunately on shared action creation it's not defined if it will be used > in ingress or egress flow or both. > Is the suggestion is to restrict usage of shared action to single direction? > > > In a way, it is more to add flexibility to this proposal. > Specifying the direction explicitly is better than PMD parsing the actions > and determining if it is for ingress or egress. > > If hardware does not have specific pools for either ingress or egress, > the PMD can ignore the attribute. > So it will be upto the PMD and the underlying hardware to use the > direction attribute - if necessary. > > > > > > > > > > > > + struct rte_flow_error *error) > > +{ > > + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; > > + struct rte_flow_shared_action *shared_action; > > + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, > error); > > + > > + if (unlikely(!ops)) > > + return NULL; > > + if (likely(!!ops->shared_action_create)) { > > + shared_action = ops->shared_action_create(dev, action, > > error); > > + if (shared_action == NULL) > > + flow_err(port_id, -rte_errno, error); > > + return shared_action; > > + } > > + rte_flow_error_set(error, ENOSYS, > > RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > > + NULL, rte_strerror(ENOSYS)); > > + return NULL; > > +} > > + > > +int > > +rte_flow_shared_action_destroy(uint16_t port_id, > > + struct rte_flow_shared_action *action, > > + struct rte_flow_error *error) > > +{ > > + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; > > + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, > error); > > + > > + if (unlikely(!ops)) > > + return -rte_errno; > > + if (likely(!!ops->shared_action_destroy)) > > + return flow_err(port_id, > > + ops->shared_action_destroy(dev, action, error), > > + error); > > + return rte_flow_error_set(error, ENOSYS, > > + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > > + NULL, rte_strerror(ENOSYS)); > > +} > > + > > +int > > +rte_flow_shared_action_update(uint16_t port_id, > > + struct rte_flow_shared_action *action, > > + const struct rte_flow_action *update, > > + struct rte_flow_error *error) > > +{ > > + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; > > + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, > error); > > + > > + if (unlikely(!ops)) > > + return -rte_errno; > > + if (likely(!!ops->shared_action_update)) > > + return flow_err(port_id, ops->shared_action_update(dev, > > action, > > + update, error), > > + error); > > + return rte_flow_error_set(error, ENOSYS, > > + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > > + NULL, rte_strerror(ENOSYS)); > > +} > > + > > +int > > +rte_flow_shared_action_query(uint16_t port_id, > > + const struct rte_flow_shared_action *action, > > + void *data, > > + struct rte_flow_error *error) > > +{ > > + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; > > + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, > error); > > + > > + if (unlikely(!ops)) > > + return -rte_errno; > > + if (likely(!!ops->shared_action_query)) > > + return flow_err(port_id, ops->shared_action_query(dev, > > action, > > + data, error), > > + error); > > + return rte_flow_error_set(error, ENOSYS, > > + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > > + NULL, rte_strerror(ENOSYS)); > > +} > > diff --git a/lib/librte_ethdev/rte_flow.h > b/lib/librte_ethdev/rte_flow.h > > index b0e4199192..257456b14a 100644 > > --- a/lib/librte_ethdev/rte_flow.h > > +++ b/lib/librte_ethdev/rte_flow.h > > @@ -1681,7 +1681,8 @@ enum rte_flow_action_type { > > /** > > * Enables counters for this flow rule. > > * > > - * These counters can be retrieved and reset through > > rte_flow_query(), > > + * These counters can be retrieved and reset through > > rte_flow_query() or > > + * rte_flow_shared_action_query() if the action provided via > > handle, > > * see struct rte_flow_query_count. > > * > > * See struct rte_flow_action_count. > > @@ -2099,6 +2100,14 @@ enum rte_flow_action_type { > > * see enum RTE_ETH_EVENT_FLOW_AGED > > */ > > RTE_FLOW_ACTION_TYPE_AGE, > > + > > + /** > > + * Describes action shared a cross multiple flow rules. > > + * > > + * Enables multiple rules reference the same action by handle > (see > > + * struct rte_flow_shared_action). > > + */ > > + RTE_FLOW_ACTION_TYPE_SHARED, > > }; > > > > /** > > @@ -2660,6 +2669,20 @@ struct rte_flow_action_set_dscp { > > uint8_t dscp; > > }; > > > > + > > +/** > > + * RTE_FLOW_ACTION_TYPE_SHARED > > + * > > + * Opaque type returned after successfully creating a shared > action. > > + * > > + * This handle can be used to manage and query the related > action: > > + * - share it a cross multiple flow rules > > + * - update action configuration > > + * - query action data > > + * - destroy action > > + */ > > +struct rte_flow_shared_action; > > + > > /* Mbuf dynamic field offset for metadata. */ > > extern int32_t rte_flow_dynf_metadata_offs; > > > > @@ -3324,6 +3347,129 @@ int > > rte_flow_get_aged_flows(uint16_t port_id, void **contexts, > > uint32_t nb_contexts, struct rte_flow_error *error); > > > > +/** > > + * @warning > > + * @b EXPERIMENTAL: this API may change without prior notice. > > + * > > + * Create shared action for reuse in multiple flow rules. > > + * > > + * @param[in] port_id > > + * The port identifier of the Ethernet device. > > + * @param[in] action > > + * Action configuration for shared action creation. > > + * @param[out] error > > + * Perform verbose error reporting if not NULL. PMDs initialize > this > > + * structure in case of error only. > > + * @return > > + * A valid handle in case of success, NULL otherwise and > rte_errno is > > set > > + * to one of the error codes defined: > > + * - (ENOSYS) if underlying device does not support this > functionality. > > + * - (EIO) if underlying device is removed. > > + * - (EINVAL) if *action* invalid. > > + * - (ENOTSUP) if *action* valid but unsupported. > > + */ > > +__rte_experimental > > +struct rte_flow_shared_action * > > +rte_flow_shared_action_create(uint16_t port_id, > > + const struct rte_flow_action *action, > > + struct rte_flow_error *error); > > + > > +/** > > + * @warning > > + * @b EXPERIMENTAL: this API may change without prior notice. > > + * > > + * Destroys the shared action by handle. > > + * > > + * @param[in] port_id > > + * The port identifier of the Ethernet device. > > + * @param[in] action > > + * Handle for the shared action to be destroyed. > > + * @param[out] error > > + * Perform verbose error reporting if not NULL. PMDs initialize > this > > + * structure in case of error only. > > + * @return > > + * - (0) if success. > > + * - (-ENOSYS) if underlying device does not support this > functionality. > > + * - (-EIO) if underlying device is removed. > > + * - (-ENOENT) if action pointed by *action* handle was not > found. > > + * - (-ETOOMANYREFS) if action pointed by *action* handle still > used > > by one or > > + * more rules > > + * rte_errno is also set. > > + */ > > +__rte_experimental > > +int > > +rte_flow_shared_action_destroy(uint16_t port_id, > > + struct rte_flow_shared_action *action, > > + struct rte_flow_error *error); > > + > > +/** > > + * @warning > > + * @b EXPERIMENTAL: this API may change without prior notice. > > + * > > + * Updates inplace the shared action configuration pointed by > *action* > > handle > > + * with the configuration provided as *update* argument. > > + * The update of the shared action configuration effects all flow > rules > > reusing > > + * the action via handle. > > + * > > + * @param[in] port_id > > + * The port identifier of the Ethernet device. > > + * @param[in] action > > + * Handle for the shared action to be updated. > > + * @param[in] update > > + * Action specification used to modify the action pointed by > handle. > > + * *update* should be of same type with the action pointed by > the > > *action* > > + * handle argument, otherwise considered as invalid. > > + * @param[out] error > > + * Perform verbose error reporting if not NULL. PMDs initialize > this > > + * structure in case of error only. > > + * @return > > + * - (0) if success. > > + * - (-ENOSYS) if underlying device does not support this > functionality. > > + * - (-EIO) if underlying device is removed. > > + * - (-EINVAL) if *update* invalid. > > + * - (-ENOTSUP) if *update* valid but unsupported. > > + * - (-ENOENT) if action pointed by *ctx* was not found. > > + * rte_errno is also set. > > + */ > > +__rte_experimental > > +int > > +rte_flow_shared_action_update(uint16_t port_id, > > + struct rte_flow_shared_action *action, > > + const struct rte_flow_action *update, > > + struct rte_flow_error *error); > > + > > +/** > > + * @warning > > + * @b EXPERIMENTAL: this API may change without prior notice. > > + * > > + * Query the shared action by handle. > > + * > > + * This function allows retrieving action-specific data such as > counters. > > + * Data is gathered by special action which may be > present/referenced > > in > > + * more than one flow rule definition. > > + * > > + * \see RTE_FLOW_ACTION_TYPE_COUNT > > + * > > + * @param port_id > > + * Port identifier of Ethernet device. > > + * @param[in] action > > + * Handle for the shared action to query. > > + * @param[in, out] data > > + * Pointer to storage for the associated query data type. > > + * @param[out] error > > + * Perform verbose error reporting if not NULL. PMDs initialize > this > > + * structure in case of error only. > > + * > > + * @return > > + * 0 on success, a negative errno value otherwise and rte_errno > is set. > > + */ > > +__rte_experimental > > +int > > +rte_flow_shared_action_query(uint16_t port_id, > > + const struct rte_flow_shared_action *action, > > + void *data, > > + struct rte_flow_error *error); > > + > > #ifdef __cplusplus > > } > > #endif > > diff --git a/lib/librte_ethdev/rte_flow_driver.h > > b/lib/librte_ethdev/rte_flow_driver.h > > index 881cc469b7..a2cae1b53c 100644 > > --- a/lib/librte_ethdev/rte_flow_driver.h > > +++ b/lib/librte_ethdev/rte_flow_driver.h > > @@ -107,6 +107,28 @@ struct rte_flow_ops { > > void **context, > > uint32_t nb_contexts, > > struct rte_flow_error *err); > > + /** See rte_flow_shared_action_create() */ > > + struct rte_flow_shared_action *(*shared_action_create) > > + (struct rte_eth_dev *dev, > > + const struct rte_flow_action *action, > > + struct rte_flow_error *error); > > + /** See rte_flow_shared_action_destroy() */ > > + int (*shared_action_destroy) > > + (struct rte_eth_dev *dev, > > + struct rte_flow_shared_action *shared_action, > > + struct rte_flow_error *error); > > + /** See rte_flow_shared_action_update() */ > > + int (*shared_action_update) > > + (struct rte_eth_dev *dev, > > + struct rte_flow_shared_action *shared_action, > > + const struct rte_flow_action *update, > > + struct rte_flow_error *error); > > + /** See rte_flow_shared_action_query() */ > > + int (*shared_action_query) > > + (struct rte_eth_dev *dev, > > + const struct rte_flow_shared_action *shared_action, > > + void *data, > > + struct rte_flow_error *error); > > }; > > > > /** > > -- > > 2.26.2 > > > > > > ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [dpdk-dev] [PATCH v2 1/6] ethdev: add flow shared action API 2020-09-16 15:52 ` Andrey Vesnovaty @ 2020-09-16 19:20 ` Ajit Khaparde 2020-09-17 15:33 ` Andrew Rybchenko 0 siblings, 1 reply; 106+ messages in thread From: Ajit Khaparde @ 2020-09-16 19:20 UTC (permalink / raw) To: Andrey Vesnovaty Cc: NBU-Contact-Thomas Monjalon, Ori Kam, Slava Ovsiienko, jerinj, Andrew Rybchenko, dpdk-dev, jer, Jerin Jacob, Ferruh Yigit, Stephen Hemminger, Bruce Richardson, Ori Kam, Viacheslav Ovsiienko, andrey.vesnovaty, Ray Kinsella, Neil Horman, Thomas Monjalon, Samik Gupta On Wed, Sep 16, 2020 at 8:52 AM Andrey Vesnovaty <andreyv@nvidia.com> wrote: > > Hi Ajit > > For shared action configuration I have following suggestion: > > struct rte_flow_shared_action_conf { > uint32_t no_ingress: 1; > uint32_t no_egress: 1; > }; > /*...*/ > rte_flow_shared_action_create(..., const struct rte_flow_shared_action_conf *conf, ...); > > What do you think? Andrey, I think this is good. Application can specify the direction and PMD can decide whether if it needs to honor it or ignore it. Please send the updated version of the patch. Thanks Ajit > > Thanks, > Andrey > > > -----Original Message----- > > From: Ajit Khaparde <ajit.khaparde@broadcom.com> > > Sent: Tuesday, September 15, 2020 6:49 PM > > To: Andrey Vesnovaty <andreyv@nvidia.com> > > Cc: NBU-Contact-Thomas Monjalon <thomas@monjalon.net>; Ori Kam > > <orika@nvidia.com>; Slava Ovsiienko <viacheslavo@nvidia.com>; > > jerinj@marvell.com; Andrew Rybchenko <arybchenko@solarflare.com>; dpdk- > > dev <dev@dpdk.org>; jer@marvell.com; Jerin Jacob <jerinjacobk@gmail.com>; > > Ferruh Yigit <ferruh.yigit@intel.com>; Stephen Hemminger > > <stephen@networkplumber.org>; Bruce Richardson > > <bruce.richardson@intel.com>; Ori Kam <orika@mellanox.com>; Viacheslav > > Ovsiienko <viacheslavo@mellanox.com>; andrey.vesnovaty@gmail.com; Ray > > Kinsella <mdr@ashroe.eu>; Neil Horman <nhorman@tuxdriver.com>; Thomas > > Monjalon <tmonjalon@nvidia.com>; Samik Gupta > > <samik.gupta@broadcom.com> > > Subject: Re: [dpdk-dev] [PATCH v2 1/6] ethdev: add flow shared action API > > > > > > > > On Tue, Sep 15, 2020 at 4:51 AM Andrey Vesnovaty <andreyv@nvidia.com > > <mailto:andreyv@nvidia.com> > wrote: > > > > > > Hi Ajit, PSB. > > > > Thanks, > > Andrey > > > > > -----Original Message----- > > > From: Ajit Khaparde <ajit.khaparde@broadcom.com > > <mailto:ajit.khaparde@broadcom.com> > > > > Sent: Saturday, September 12, 2020 5:18 AM > > > To: Andrey Vesnovaty <andreyv@mellanox.com > > <mailto:andreyv@mellanox.com> > > > > Cc: dpdk-dev <dev@dpdk.org <mailto:dev@dpdk.org> >; > > jer@marvell.com <mailto:jer@marvell.com> ; Jerin Jacob > > > <jerinjacobk@gmail.com <mailto:jerinjacobk@gmail.com> >; NBU- > > Contact-Thomas Monjalon > > > <thomas@monjalon.net <mailto:thomas@monjalon.net> >; Ferruh > > Yigit <ferruh.yigit@intel.com <mailto:ferruh.yigit@intel.com> >; Stephen > > > Hemminger <stephen@networkplumber.org > > <mailto:stephen@networkplumber.org> >; Bruce Richardson > > > <bruce.richardson@intel.com <mailto:bruce.richardson@intel.com> > > >; Ori Kam <orika@mellanox.com <mailto:orika@mellanox.com> >; Viacheslav > > > Ovsiienko <viacheslavo@mellanox.com > > <mailto:viacheslavo@mellanox.com> >; andrey.vesnovaty@gmail.com > > <mailto:andrey.vesnovaty@gmail.com> ; Ray > > > Kinsella <mdr@ashroe.eu <mailto:mdr@ashroe.eu> >; Neil Horman > > <nhorman@tuxdriver.com <mailto:nhorman@tuxdriver.com> >; Andrew > > > Rybchenko <arybchenko@solarflare.com > > <mailto:arybchenko@solarflare.com> > > > > Subject: Re: [dpdk-dev] [PATCH v2 1/6] ethdev: add flow shared action > > API > > > > > > > > > > > > On Wed, Jul 8, 2020 at 2:40 PM Andrey Vesnovaty > > <andreyv@mellanox.com <mailto:andreyv@mellanox.com> > > > <mailto:andreyv@mellanox.com <mailto:andreyv@mellanox.com> > > > > wrote: > > > > > > > > > From: Andrey Vesnovaty <andrey.vesnovaty@gmail.com > > <mailto:andrey.vesnovaty@gmail.com> > > > <mailto:andrey.vesnovaty@gmail.com > > <mailto:andrey.vesnovaty@gmail.com> > > > > > > > > This commit introduces extension of DPDK flow action API enabling > > > sharing of single rte_flow_action in multiple flows. The API > > intended for > > > PMDs where multiple HW offloaded flows can reuse the same HW > > > essence/object representing flow action and modification of such > > an > > > essence/object effects all the rules using it. > > > > > > Motivation and example > > > === > > > Adding or removing one or more queues to RSS used by multiple > > flow > > > rules > > > imposes per rule toll for current DPDK flow API; the scenario > > requires > > > for each flow sharing cloned RSS action: > > > - call `rte_flow_destroy()` > > > - call `rte_flow_create()` with modified RSS action > > > > > > API for sharing action and its in-place update benefits: > > > - reduce the overhead of multiple RSS flow rules reconfiguration > > > - optimize resource utilization by sharing action across of multiple > > > flows > > > > > > Change description > > > === > > > > > > Shared action > > > === > > > In order to represent flow action shared by multiple flows new > > action > > > type RTE_FLOW_ACTION_TYPE_SHARED is introduced (see `enum > > > rte_flow_action_type`). > > > Actually the introduced API decouples action from any specific flow > > and > > > enables sharing of single action by its handle across multiple flows. > > > > > > Shared action create/use/destroy > > > === > > > Shared action may be reused by some or none flow rules at any > > given > > > moment, i.e. shared action reside outside of the context of any > > flow. > > > Shared action represent HW resources/objects used for action > > > offloading > > > implementation. > > > API for shared action create (see > > `rte_flow_shared_action_create()`): > > > - should allocate HW resources and make related initializations > > required > > > for shared action implementation. > > > - make necessary preparations to maintain shared access to > > > the action resources, configuration and state. > > > API for shared action destroy (see > > `rte_flow_shared_action_destroy()`) > > > should release HW resources and make related cleanups required > > for > > > shared > > > action implementation. > > > > > > In order to share some flow action reuse the handle of type > > > `struct rte_flow_shared_action` returned by > > > rte_flow_shared_action_create() as a `conf` field of > > > `struct rte_flow_action` (see "example" section). > > > > > > If some shared action not used by any flow rule all resources > > allocated > > > by the shared action can be released by > > > rte_flow_shared_action_destroy() > > > (see "example" section). The shared action handle passed as > > argument > > > to > > > destroy API should not be used any further i.e. result of the usage is > > > undefined. > > > > > > Shared action re-configuration > > > === > > > Shared action behavior defined by its configuration can be updated > > via > > > rte_flow_shared_action_update() (see "example" section). The > > shared > > > action update operation modifies HW related resources/objects > > > allocated > > > on the action creation. The number of operations performed by > > the > > > update > > > operation should not be dependent on number of flows sharing the > > > related > > > action. On return of shared action update API action behavior > > should be > > > according to updated configuration for all flows sharing the action. > > > > > > Shared action query > > > === > > > Provide separate API to query shared action sate (see > > > rte_flow_shared_action_update()). Taking a counter as an > > example: > > > query > > > returns value aggregating all counter increments across all flow > > rules > > > sharing the counter. > > > > > > PMD support > > > === > > > The support of introduced API is pure PMD specific design and > > > responsibility for each action type (see struct rte_flow_ops). > > > > > > testpmd > > > === > > > In order to utilize introduced API testpmd cli may implement > > following > > > extension > > > create/update/destroy/query shared action accordingly > > > > > > flow shared_action create {port_id} [index] {action} > > > flow shared_action update {port_id} {index} {action} > > > flow shared_action destroy {port_id} {index} > > > flow shared_action query {port_id} {index} > > > > > > testpmd example > > > === > > > > > > configure rss to queues 1 & 2 > > > > > > testpmd> flow shared_action create 0 100 rss 1 2 > > > > > > create flow rule utilizing shared action > > > > > > testpmd> flow create 0 ingress \ > > > pattern eth dst is 0c:42:a1:15:fd:ac / ipv6 / tcp / end \ > > > actions shared 100 end / end > > > > > > add 2 more queues > > > > > > testpmd> flow shared_action modify 0 100 rss 1 2 3 4 > > > > > > example > > > === > > > > > > struct rte_flow_action actions[2]; > > > struct rte_flow_action action; > > > /* skipped: initialize action */ > > > struct rte_flow_shared_action *handle = > > > rte_flow_shared_action_create( > > > port_id, &action, &error); > > > actions[0].type = RTE_FLOW_ACTION_TYPE_SHARED; > > > actions[0].conf = handle; > > > actions[1].type = RTE_FLOW_ACTION_TYPE_END; > > > /* skipped: init attr0 & pattern0 args */ > > > struct rte_flow *flow0 = rte_flow_create(port_id, &attr0, > > pattern0, > > > actions, error); > > > /* create more rules reusing shared action */ > > > struct rte_flow *flow1 = rte_flow_create(port_id, &attr1, > > pattern1, > > > actions, error); > > > /* skipped: for flows 2 till N */ > > > struct rte_flow *flowN = rte_flow_create(port_id, &attrN, > > patternN, > > > actions, error); > > > /* update shared action */ > > > struct rte_flow_action updated_action; > > > /* > > > * skipped: initialize updated_action according to desired action > > > * configuration change > > > */ > > > rte_flow_shared_action_update(port_id, handle, > > &updated_action, > > > error); > > > /* > > > * from now on all flows 1 till N will act according to configuration > > of > > > * updated_action > > > */ > > > /* skipped: destroy all flows 1 till N */ > > > rte_flow_shared_action_destroy(port_id, handle, error); > > > > > > Signed-off-by: Andrey Vesnovaty <andreyv@mellanox.com > > <mailto:andreyv@mellanox.com> > > > <mailto:andreyv@mellanox.com <mailto:andreyv@mellanox.com> > > > > > > > --- > > > lib/librte_ethdev/rte_ethdev_version.map | 6 + > > > lib/librte_ethdev/rte_flow.c | 81 +++++++++++++ > > > lib/librte_ethdev/rte_flow.h | 148 > > ++++++++++++++++++++++- > > > lib/librte_ethdev/rte_flow_driver.h | 22 ++++ > > > 4 files changed, 256 insertions(+), 1 deletion(-) > > > > > > diff --git a/lib/librte_ethdev/rte_ethdev_version.map > > > b/lib/librte_ethdev/rte_ethdev_version.map > > > index 7155056045..119d84976a 100644 > > > --- a/lib/librte_ethdev/rte_ethdev_version.map > > > +++ b/lib/librte_ethdev/rte_ethdev_version.map > > > @@ -241,4 +241,10 @@ EXPERIMENTAL { > > > __rte_ethdev_trace_rx_burst; > > > __rte_ethdev_trace_tx_burst; > > > rte_flow_get_aged_flows; > > > + > > > + # added in 20.08 > > > + rte_flow_shared_action_create; > > > + rte_flow_shared_action_destroy; > > > + rte_flow_shared_action_update; > > > + rte_flow_shared_action_query; > > > }; > > > diff --git a/lib/librte_ethdev/rte_flow.c > > b/lib/librte_ethdev/rte_flow.c > > > index 1685be5f73..0ac4d31a13 100644 > > > --- a/lib/librte_ethdev/rte_flow.c > > > +++ b/lib/librte_ethdev/rte_flow.c > > > @@ -1250,3 +1250,84 @@ rte_flow_get_aged_flows(uint16_t > > port_id, > > > void **contexts, > > > RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > > > NULL, rte_strerror(ENOTSUP)); > > > } > > > + > > > +struct rte_flow_shared_action * > > > +rte_flow_shared_action_create(uint16_t port_id, > > > + const struct rte_flow_action *action, > > > > > > > > > It will be good to have an attributes argument here. > > > Some hardware devices may have ingress and egress resource pools. > > > So a 'direction' attribute can help share the resource effectively. > > > > I understand the idea of HW ingress/egress resource separation. > > Unfortunately on shared action creation it's not defined if it will be used > > in ingress or egress flow or both. > > Is the suggestion is to restrict usage of shared action to single direction? > > > > > > In a way, it is more to add flexibility to this proposal. > > Specifying the direction explicitly is better than PMD parsing the actions > > and determining if it is for ingress or egress. > > > > If hardware does not have specific pools for either ingress or egress, > > the PMD can ignore the attribute. > > So it will be upto the PMD and the underlying hardware to use the > > direction attribute - if necessary. > > > > > > > > > > > > > > > > > > > > + struct rte_flow_error *error) > > > +{ > > > + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; > > > + struct rte_flow_shared_action *shared_action; > > > + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, > > error); > > > + > > > + if (unlikely(!ops)) > > > + return NULL; > > > + if (likely(!!ops->shared_action_create)) { > > > + shared_action = ops->shared_action_create(dev, action, > > > error); > > > + if (shared_action == NULL) > > > + flow_err(port_id, -rte_errno, error); > > > + return shared_action; > > > + } > > > + rte_flow_error_set(error, ENOSYS, > > > RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > > > + NULL, rte_strerror(ENOSYS)); > > > + return NULL; > > > +} > > > + > > > +int > > > +rte_flow_shared_action_destroy(uint16_t port_id, > > > + struct rte_flow_shared_action *action, > > > + struct rte_flow_error *error) > > > +{ > > > + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; > > > + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, > > error); > > > + > > > + if (unlikely(!ops)) > > > + return -rte_errno; > > > + if (likely(!!ops->shared_action_destroy)) > > > + return flow_err(port_id, > > > + ops->shared_action_destroy(dev, action, error), > > > + error); > > > + return rte_flow_error_set(error, ENOSYS, > > > + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > > > + NULL, rte_strerror(ENOSYS)); > > > +} > > > + > > > +int > > > +rte_flow_shared_action_update(uint16_t port_id, > > > + struct rte_flow_shared_action *action, > > > + const struct rte_flow_action *update, > > > + struct rte_flow_error *error) > > > +{ > > > + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; > > > + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, > > error); > > > + > > > + if (unlikely(!ops)) > > > + return -rte_errno; > > > + if (likely(!!ops->shared_action_update)) > > > + return flow_err(port_id, ops->shared_action_update(dev, > > > action, > > > + update, error), > > > + error); > > > + return rte_flow_error_set(error, ENOSYS, > > > + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > > > + NULL, rte_strerror(ENOSYS)); > > > +} > > > + > > > +int > > > +rte_flow_shared_action_query(uint16_t port_id, > > > + const struct rte_flow_shared_action *action, > > > + void *data, > > > + struct rte_flow_error *error) > > > +{ > > > + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; > > > + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, > > error); > > > + > > > + if (unlikely(!ops)) > > > + return -rte_errno; > > > + if (likely(!!ops->shared_action_query)) > > > + return flow_err(port_id, ops->shared_action_query(dev, > > > action, > > > + data, error), > > > + error); > > > + return rte_flow_error_set(error, ENOSYS, > > > + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > > > + NULL, rte_strerror(ENOSYS)); > > > +} > > > diff --git a/lib/librte_ethdev/rte_flow.h > > b/lib/librte_ethdev/rte_flow.h > > > index b0e4199192..257456b14a 100644 > > > --- a/lib/librte_ethdev/rte_flow.h > > > +++ b/lib/librte_ethdev/rte_flow.h > > > @@ -1681,7 +1681,8 @@ enum rte_flow_action_type { > > > /** > > > * Enables counters for this flow rule. > > > * > > > - * These counters can be retrieved and reset through > > > rte_flow_query(), > > > + * These counters can be retrieved and reset through > > > rte_flow_query() or > > > + * rte_flow_shared_action_query() if the action provided via > > > handle, > > > * see struct rte_flow_query_count. > > > * > > > * See struct rte_flow_action_count. > > > @@ -2099,6 +2100,14 @@ enum rte_flow_action_type { > > > * see enum RTE_ETH_EVENT_FLOW_AGED > > > */ > > > RTE_FLOW_ACTION_TYPE_AGE, > > > + > > > + /** > > > + * Describes action shared a cross multiple flow rules. > > > + * > > > + * Enables multiple rules reference the same action by handle > > (see > > > + * struct rte_flow_shared_action). > > > + */ > > > + RTE_FLOW_ACTION_TYPE_SHARED, > > > }; > > > > > > /** > > > @@ -2660,6 +2669,20 @@ struct rte_flow_action_set_dscp { > > > uint8_t dscp; > > > }; > > > > > > + > > > +/** > > > + * RTE_FLOW_ACTION_TYPE_SHARED > > > + * > > > + * Opaque type returned after successfully creating a shared > > action. > > > + * > > > + * This handle can be used to manage and query the related > > action: > > > + * - share it a cross multiple flow rules > > > + * - update action configuration > > > + * - query action data > > > + * - destroy action > > > + */ > > > +struct rte_flow_shared_action; > > > + > > > /* Mbuf dynamic field offset for metadata. */ > > > extern int32_t rte_flow_dynf_metadata_offs; > > > > > > @@ -3324,6 +3347,129 @@ int > > > rte_flow_get_aged_flows(uint16_t port_id, void **contexts, > > > uint32_t nb_contexts, struct rte_flow_error *error); > > > > > > +/** > > > + * @warning > > > + * @b EXPERIMENTAL: this API may change without prior notice. > > > + * > > > + * Create shared action for reuse in multiple flow rules. > > > + * > > > + * @param[in] port_id > > > + * The port identifier of the Ethernet device. > > > + * @param[in] action > > > + * Action configuration for shared action creation. > > > + * @param[out] error > > > + * Perform verbose error reporting if not NULL. PMDs initialize > > this > > > + * structure in case of error only. > > > + * @return > > > + * A valid handle in case of success, NULL otherwise and > > rte_errno is > > > set > > > + * to one of the error codes defined: > > > + * - (ENOSYS) if underlying device does not support this > > functionality. > > > + * - (EIO) if underlying device is removed. > > > + * - (EINVAL) if *action* invalid. > > > + * - (ENOTSUP) if *action* valid but unsupported. > > > + */ > > > +__rte_experimental > > > +struct rte_flow_shared_action * > > > +rte_flow_shared_action_create(uint16_t port_id, > > > + const struct rte_flow_action *action, > > > + struct rte_flow_error *error); > > > + > > > +/** > > > + * @warning > > > + * @b EXPERIMENTAL: this API may change without prior notice. > > > + * > > > + * Destroys the shared action by handle. > > > + * > > > + * @param[in] port_id > > > + * The port identifier of the Ethernet device. > > > + * @param[in] action > > > + * Handle for the shared action to be destroyed. > > > + * @param[out] error > > > + * Perform verbose error reporting if not NULL. PMDs initialize > > this > > > + * structure in case of error only. > > > + * @return > > > + * - (0) if success. > > > + * - (-ENOSYS) if underlying device does not support this > > functionality. > > > + * - (-EIO) if underlying device is removed. > > > + * - (-ENOENT) if action pointed by *action* handle was not > > found. > > > + * - (-ETOOMANYREFS) if action pointed by *action* handle still > > used > > > by one or > > > + * more rules > > > + * rte_errno is also set. > > > + */ > > > +__rte_experimental > > > +int > > > +rte_flow_shared_action_destroy(uint16_t port_id, > > > + struct rte_flow_shared_action *action, > > > + struct rte_flow_error *error); > > > + > > > +/** > > > + * @warning > > > + * @b EXPERIMENTAL: this API may change without prior notice. > > > + * > > > + * Updates inplace the shared action configuration pointed by > > *action* > > > handle > > > + * with the configuration provided as *update* argument. > > > + * The update of the shared action configuration effects all flow > > rules > > > reusing > > > + * the action via handle. > > > + * > > > + * @param[in] port_id > > > + * The port identifier of the Ethernet device. > > > + * @param[in] action > > > + * Handle for the shared action to be updated. > > > + * @param[in] update > > > + * Action specification used to modify the action pointed by > > handle. > > > + * *update* should be of same type with the action pointed by > > the > > > *action* > > > + * handle argument, otherwise considered as invalid. > > > + * @param[out] error > > > + * Perform verbose error reporting if not NULL. PMDs initialize > > this > > > + * structure in case of error only. > > > + * @return > > > + * - (0) if success. > > > + * - (-ENOSYS) if underlying device does not support this > > functionality. > > > + * - (-EIO) if underlying device is removed. > > > + * - (-EINVAL) if *update* invalid. > > > + * - (-ENOTSUP) if *update* valid but unsupported. > > > + * - (-ENOENT) if action pointed by *ctx* was not found. > > > + * rte_errno is also set. > > > + */ > > > +__rte_experimental > > > +int > > > +rte_flow_shared_action_update(uint16_t port_id, > > > + struct rte_flow_shared_action *action, > > > + const struct rte_flow_action *update, > > > + struct rte_flow_error *error); > > > + > > > +/** > > > + * @warning > > > + * @b EXPERIMENTAL: this API may change without prior notice. > > > + * > > > + * Query the shared action by handle. > > > + * > > > + * This function allows retrieving action-specific data such as > > counters. > > > + * Data is gathered by special action which may be > > present/referenced > > > in > > > + * more than one flow rule definition. > > > + * > > > + * \see RTE_FLOW_ACTION_TYPE_COUNT > > > + * > > > + * @param port_id > > > + * Port identifier of Ethernet device. > > > + * @param[in] action > > > + * Handle for the shared action to query. > > > + * @param[in, out] data > > > + * Pointer to storage for the associated query data type. > > > + * @param[out] error > > > + * Perform verbose error reporting if not NULL. PMDs initialize > > this > > > + * structure in case of error only. > > > + * > > > + * @return > > > + * 0 on success, a negative errno value otherwise and rte_errno > > is set. > > > + */ > > > +__rte_experimental > > > +int > > > +rte_flow_shared_action_query(uint16_t port_id, > > > + const struct rte_flow_shared_action *action, > > > + void *data, > > > + struct rte_flow_error *error); > > > + > > > #ifdef __cplusplus > > > } > > > #endif > > > diff --git a/lib/librte_ethdev/rte_flow_driver.h > > > b/lib/librte_ethdev/rte_flow_driver.h > > > index 881cc469b7..a2cae1b53c 100644 > > > --- a/lib/librte_ethdev/rte_flow_driver.h > > > +++ b/lib/librte_ethdev/rte_flow_driver.h > > > @@ -107,6 +107,28 @@ struct rte_flow_ops { > > > void **context, > > > uint32_t nb_contexts, > > > struct rte_flow_error *err); > > > + /** See rte_flow_shared_action_create() */ > > > + struct rte_flow_shared_action *(*shared_action_create) > > > + (struct rte_eth_dev *dev, > > > + const struct rte_flow_action *action, > > > + struct rte_flow_error *error); > > > + /** See rte_flow_shared_action_destroy() */ > > > + int (*shared_action_destroy) > > > + (struct rte_eth_dev *dev, > > > + struct rte_flow_shared_action *shared_action, > > > + struct rte_flow_error *error); > > > + /** See rte_flow_shared_action_update() */ > > > + int (*shared_action_update) > > > + (struct rte_eth_dev *dev, > > > + struct rte_flow_shared_action *shared_action, > > > + const struct rte_flow_action *update, > > > + struct rte_flow_error *error); > > > + /** See rte_flow_shared_action_query() */ > > > + int (*shared_action_query) > > > + (struct rte_eth_dev *dev, > > > + const struct rte_flow_shared_action *shared_action, > > > + void *data, > > > + struct rte_flow_error *error); > > > }; > > > > > > /** > > > -- > > > 2.26.2 > > > > > > > > > > > ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [dpdk-dev] [PATCH v2 1/6] ethdev: add flow shared action API 2020-09-16 19:20 ` Ajit Khaparde @ 2020-09-17 15:33 ` Andrew Rybchenko 2020-09-17 16:02 ` Ori Kam 0 siblings, 1 reply; 106+ messages in thread From: Andrew Rybchenko @ 2020-09-17 15:33 UTC (permalink / raw) To: Ajit Khaparde, Andrey Vesnovaty Cc: NBU-Contact-Thomas Monjalon, Ori Kam, Slava Ovsiienko, jerinj, Andrew Rybchenko, dpdk-dev, jer, Jerin Jacob, Ferruh Yigit, Stephen Hemminger, Bruce Richardson, Ori Kam, Viacheslav Ovsiienko, andrey.vesnovaty, Ray Kinsella, Neil Horman, Thomas Monjalon, Samik Gupta On 9/16/20 10:20 PM, Ajit Khaparde wrote: > On Wed, Sep 16, 2020 at 8:52 AM Andrey Vesnovaty <andreyv@nvidia.com> wrote: >> >> Hi Ajit >> >> For shared action configuration I have following suggestion: >> >> struct rte_flow_shared_action_conf { >> uint32_t no_ingress: 1; >> uint32_t no_egress: 1; >> }; >> /*...*/ >> rte_flow_shared_action_create(..., const struct rte_flow_shared_action_conf *conf, ...); >> >> What do you think? > Andrey, I think this is good. > Application can specify the direction and PMD can decide whether if > it needs to honor it or ignore it. > Please send the updated version of the patch. Personally I dislike negative flags, offloads, fields etc. Don't we have a policy to avoid it. At least we have it for offloads. I see no string reasons here to use negative instead of positive here. ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [dpdk-dev] [PATCH v2 1/6] ethdev: add flow shared action API 2020-09-17 15:33 ` Andrew Rybchenko @ 2020-09-17 16:02 ` Ori Kam 2020-09-24 19:25 ` Ajit Khaparde 0 siblings, 1 reply; 106+ messages in thread From: Ori Kam @ 2020-09-17 16:02 UTC (permalink / raw) To: Andrew Rybchenko, Ajit Khaparde, Andrey Vesnovaty Cc: NBU-Contact-Thomas Monjalon, Slava Ovsiienko, jerinj, dpdk-dev, jer, Jerin Jacob, Ferruh Yigit, Stephen Hemminger, Bruce Richardson, Ori Kam, Viacheslav Ovsiienko, andrey.vesnovaty, Ray Kinsella, Neil Horman, Thomas Monjalon, Samik Gupta > -----Original Message----- > From: Andrew Rybchenko <arybchenko@solarflare.com> > Sent: Thursday, September 17, 2020 6:34 PM > <samik.gupta@broadcom.com> > Subject: Re: [dpdk-dev] [PATCH v2 1/6] ethdev: add flow shared action API > > On 9/16/20 10:20 PM, Ajit Khaparde wrote: > > On Wed, Sep 16, 2020 at 8:52 AM Andrey Vesnovaty <andreyv@nvidia.com> > wrote: > >> > >> Hi Ajit > >> > >> For shared action configuration I have following suggestion: > >> > >> struct rte_flow_shared_action_conf { > >> uint32_t no_ingress: 1; > >> uint32_t no_egress: 1; > >> }; > >> /*...*/ > >> rte_flow_shared_action_create(..., const struct > rte_flow_shared_action_conf *conf, ...); > >> > >> What do you think? > > Andrey, I think this is good. > > Application can specify the direction and PMD can decide whether if > > it needs to honor it or ignore it. > > Please send the updated version of the patch. > > Personally I dislike negative flags, offloads, fields etc. > Don't we have a policy to avoid it. At least we have it for > offloads. I see no string reasons here to use negative > instead of positive here. Agree I think it is better to use positive values and the same names as the attribute in the flow. ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [dpdk-dev] [PATCH v2 1/6] ethdev: add flow shared action API 2020-09-17 16:02 ` Ori Kam @ 2020-09-24 19:25 ` Ajit Khaparde 2020-09-26 11:09 ` Andrey Vesnovaty 0 siblings, 1 reply; 106+ messages in thread From: Ajit Khaparde @ 2020-09-24 19:25 UTC (permalink / raw) To: Ori Kam Cc: Andrew Rybchenko, Andrey Vesnovaty, NBU-Contact-Thomas Monjalon, Slava Ovsiienko, jerinj, dpdk-dev, jer, Jerin Jacob, Ferruh Yigit, Stephen Hemminger, Bruce Richardson, Ori Kam, Viacheslav Ovsiienko, andrey.vesnovaty, Ray Kinsella, Neil Horman, Thomas Monjalon, Samik Gupta On Thu, Sep 17, 2020 at 9:03 AM Ori Kam <orika@nvidia.com> wrote: > > > > > -----Original Message----- > > From: Andrew Rybchenko <arybchenko@solarflare.com> > > Sent: Thursday, September 17, 2020 6:34 PM > > <samik.gupta@broadcom.com> > > Subject: Re: [dpdk-dev] [PATCH v2 1/6] ethdev: add flow shared action API > > > > On 9/16/20 10:20 PM, Ajit Khaparde wrote: > > > On Wed, Sep 16, 2020 at 8:52 AM Andrey Vesnovaty <andreyv@nvidia.com> > > wrote: > > >> > > >> Hi Ajit > > >> > > >> For shared action configuration I have following suggestion: > > >> > > >> struct rte_flow_shared_action_conf { > > >> uint32_t no_ingress: 1; > > >> uint32_t no_egress: 1; > > >> }; > > >> /*...*/ > > >> rte_flow_shared_action_create(..., const struct > > rte_flow_shared_action_conf *conf, ...); > > >> > > >> What do you think? > > > Andrey, I think this is good. > > > Application can specify the direction and PMD can decide whether if > > > it needs to honor it or ignore it. > > > Please send the updated version of the patch. > > > > Personally I dislike negative flags, offloads, fields etc. > > Don't we have a policy to avoid it. At least we have it for > > offloads. I see no string reasons here to use negative > > instead of positive here. > > Agree I think it is better to use positive values and the same names as the > attribute in the flow. Has a new version of the patch been submitted? Thanks > ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [dpdk-dev] [PATCH v2 1/6] ethdev: add flow shared action API 2020-09-24 19:25 ` Ajit Khaparde @ 2020-09-26 11:09 ` Andrey Vesnovaty 2020-10-03 22:06 ` [dpdk-dev] [PATCH v3 00/10] RTE flow shared action Andrey Vesnovaty 0 siblings, 1 reply; 106+ messages in thread From: Andrey Vesnovaty @ 2020-09-26 11:09 UTC (permalink / raw) To: Andrew Rybchenko, Ajit Khaparde, Ori Kam, Andrew Rybchenko Cc: NBU-Contact-Thomas Monjalon, Slava Ovsiienko, jerinj, dpdk-dev, jer, Jerin Jacob, Ferruh Yigit, Stephen Hemminger, Bruce Richardson, Ori Kam, Viacheslav Ovsiienko, andrey.vesnovaty, Ray Kinsella, Neil Horman, Thomas Monjalon, Samik Gupta Hi Ajit, Andrew & Ori. PSB 😉 > -----Original Message----- > From: Ajit Khaparde <ajit.khaparde@broadcom.com> > Sent: Thursday, September 24, 2020 10:25 PM > To: Ori Kam <orika@nvidia.com> > Cc: Andrew Rybchenko <arybchenko@solarflare.com>; Andrey Vesnovaty > <andreyv@nvidia.com>; NBU-Contact-Thomas Monjalon > <thomas@monjalon.net>; Slava Ovsiienko <viacheslavo@nvidia.com>; > jerinj@marvell.com; dpdk-dev <dev@dpdk.org>; jer@marvell.com; Jerin Jacob > <jerinjacobk@gmail.com>; Ferruh Yigit <ferruh.yigit@intel.com>; Stephen > Hemminger <stephen@networkplumber.org>; Bruce Richardson > <bruce.richardson@intel.com>; Ori Kam <orika@mellanox.com>; Viacheslav > Ovsiienko <viacheslavo@mellanox.com>; andrey.vesnovaty@gmail.com; Ray > Kinsella <mdr@ashroe.eu>; Neil Horman <nhorman@tuxdriver.com>; Thomas > Monjalon <tmonjalon@nvidia.com>; Samik Gupta > <samik.gupta@broadcom.com> > Subject: Re: [dpdk-dev] [PATCH v2 1/6] ethdev: add flow shared action API > > On Thu, Sep 17, 2020 at 9:03 AM Ori Kam <orika@nvidia.com> wrote: > > > > > > > > > -----Original Message----- > > > From: Andrew Rybchenko <arybchenko@solarflare.com> > > > Sent: Thursday, September 17, 2020 6:34 PM > > > <samik.gupta@broadcom.com> > > > Subject: Re: [dpdk-dev] [PATCH v2 1/6] ethdev: add flow shared action API > > > > > > On 9/16/20 10:20 PM, Ajit Khaparde wrote: > > > > On Wed, Sep 16, 2020 at 8:52 AM Andrey Vesnovaty > <andreyv@nvidia.com> > > > wrote: > > > >> > > > >> Hi Ajit > > > >> > > > >> For shared action configuration I have following suggestion: > > > >> > > > >> struct rte_flow_shared_action_conf { > > > >> uint32_t no_ingress: 1; > > > >> uint32_t no_egress: 1; > > > >> }; > > > >> /*...*/ > > > >> rte_flow_shared_action_create(..., const struct > > > rte_flow_shared_action_conf *conf, ...); > > > >> > > > >> What do you think? > > > > Andrey, I think this is good. @Ajit Khaparde great to know. > > > > Application can specify the direction and PMD can decide whether if > > > > it needs to honor it or ignore it. > > > > Please send the updated version of the patch. > > > I'm on it. See my answer to your last email. Thanks. > > > Personally I dislike negative flags, offloads, fields etc. > > > Don't we have a policy to avoid it. At least we have it for > > > offloads. I see no string reasons here to use negative > > > instead of positive here. @Andrew Rybchenko & @Ori Kam Got your remark regarding "negative flags". Will replace with: struct rte_flow_shared_action_conf { uint32_t ingress:1; /**< Action valid for rules applied to ingress traffic. */ uint32_t egress:1; /**< Action valid for rules applied to egress traffic. */ }; > > > > Agree I think it is better to use positive values and the same names as the > > attribute in the flow. > Has a new version of the patch been submitted? Thanks > Thanks lot for your comments & remarks. Hopefully new version of the patch will be published next week. The new patch about to have testpmd for shared action to demonstrate usage. > > ^ permalink raw reply [flat|nested] 106+ messages in thread
* [dpdk-dev] [PATCH v3 00/10] RTE flow shared action 2020-09-26 11:09 ` Andrey Vesnovaty @ 2020-10-03 22:06 ` Andrey Vesnovaty 2020-10-03 22:06 ` [dpdk-dev] [PATCH v3 01/10] ethdev: add flow shared action API Andrey Vesnovaty ` (10 more replies) 0 siblings, 11 replies; 106+ messages in thread From: Andrey Vesnovaty @ 2020-10-03 22:06 UTC (permalink / raw) To: dev Cc: jer, jerinjacobk, thomas, ferruh.yigit, stephen, bruce.richardson, orika, viacheslavo, andrey.vesnovaty, mdr, nhorman, ajit.khaparde, samik.gupta This patchset introduces shared action for RTE flow. V3 changes: - implement testpmd for shared action - fix flow_filtering example application - add conf arg to shared action create API Notes: PMD implementation should be considered as draft Andrey Vesnovaty (10): ethdev: add flow shared action API ethdev: add conf arg to shared action icreate API common/mlx5: modify advanced Rx object via DevX net/mlx5: modify hash Rx queue objects net/mlx5: shared action PMD net/mlx5: shared action PMD create conf arg net/mlx5: driver support for shared action net/mlx5: shared action create conf drv support examples/flow_filtering: utilize shared RSS action app/testpmd: support shared action app/test-pmd/cmdline_flow.c | 262 ++++++- app/test-pmd/config.c | 217 ++++++ app/test-pmd/testpmd.h | 19 + doc/guides/sample_app_ug/flow_filtering.rst | 62 +- drivers/common/mlx5/mlx5_devx_cmds.c | 84 +++ drivers/common/mlx5/mlx5_devx_cmds.h | 10 + drivers/common/mlx5/mlx5_prm.h | 29 + .../common/mlx5/rte_common_mlx5_version.map | 1 + drivers/net/mlx5/mlx5.c | 1 + drivers/net/mlx5/mlx5.h | 6 + drivers/net/mlx5/mlx5_defs.h | 3 + drivers/net/mlx5/mlx5_devx.c | 178 ++++- drivers/net/mlx5/mlx5_flow.c | 497 ++++++++++++- drivers/net/mlx5/mlx5_flow.h | 86 +++ drivers/net/mlx5/mlx5_flow_dv.c | 684 +++++++++++++++++- drivers/net/mlx5/mlx5_rxq.c | 103 +++ drivers/net/mlx5/mlx5_rxtx.h | 5 +- examples/flow_filtering/flow_blocks.c | 81 ++- examples/flow_filtering/main.c | 13 +- lib/librte_ethdev/rte_ethdev_version.map | 4 + lib/librte_ethdev/rte_flow.c | 84 +++ lib/librte_ethdev/rte_flow.h | 161 ++++- lib/librte_ethdev/rte_flow_driver.h | 23 + 23 files changed, 2489 insertions(+), 124 deletions(-) -- 2.26.2 ^ permalink raw reply [flat|nested] 106+ messages in thread
* [dpdk-dev] [PATCH v3 01/10] ethdev: add flow shared action API 2020-10-03 22:06 ` [dpdk-dev] [PATCH v3 00/10] RTE flow shared action Andrey Vesnovaty @ 2020-10-03 22:06 ` Andrey Vesnovaty 2020-10-04 11:10 ` Ori Kam 2020-10-04 17:00 ` Stephen Hemminger 2020-10-03 22:06 ` [dpdk-dev] [PATCH v3 02/10] ethdev: add conf arg to shared action icreate API Andrey Vesnovaty ` (9 subsequent siblings) 10 siblings, 2 replies; 106+ messages in thread From: Andrey Vesnovaty @ 2020-10-03 22:06 UTC (permalink / raw) To: dev Cc: jer, jerinjacobk, thomas, ferruh.yigit, stephen, bruce.richardson, orika, viacheslavo, andrey.vesnovaty, mdr, nhorman, ajit.khaparde, samik.gupta, Andrey Vesnovaty, Andrew Rybchenko, Ori Kam This commit introduces extension of DPDK flow action API enabling sharing of single rte_flow_action in multiple flows. The API intended for PMDs where multiple HW offloaded flows can reuse the same HW essence/object representing flow action and modification of such an essence/object effects all the rules using it. Motivation and example === Adding or removing one or more queues to RSS used by multiple flow rules imposes per rule toll for current DPDK flow API; the scenario requires for each flow sharing cloned RSS action: - call `rte_flow_destroy()` - call `rte_flow_create()` with modified RSS action API for sharing action and its in-place update benefits: - reduce the overhead of multiple RSS flow rules reconfiguration - optimize resource utilization by sharing action across of multiple flows Change description === Shared action === In order to represent flow action shared by multiple flows new action type RTE_FLOW_ACTION_TYPE_SHARED is introduced (see `enum rte_flow_action_type`). Actually the introduced API decouples action from any specific flow and enables sharing of single action by its handle across multiple flows. Shared action create/use/destroy === Shared action may be reused by some or none flow rules at any given moment, i.e. shared action reside outside of the context of any flow. Shared action represent HW resources/objects used for action offloading implementation. API for shared action create (see `rte_flow_shared_action_create()`): - should allocate HW resources and make related initializations required for shared action implementation. - make necessary preparations to maintain shared access to the action resources, configuration and state. API for shared action destroy (see `rte_flow_shared_action_destroy()`) should release HW resources and make related cleanups required for shared action implementation. In order to share some flow action reuse the handle of type `struct rte_flow_shared_action` returned by rte_flow_shared_action_create() as a `conf` field of `struct rte_flow_action` (see "example" section). If some shared action not used by any flow rule all resources allocated by the shared action can be released by rte_flow_shared_action_destroy() (see "example" section). The shared action handle passed as argument to destroy API should not be used any further i.e. result of the usage is undefined. Shared action re-configuration === Shared action behavior defined by its configuration can be updated via rte_flow_shared_action_update() (see "example" section). The shared action update operation modifies HW related resources/objects allocated on the action creation. The number of operations performed by the update operation should not be dependent on number of flows sharing the related action. On return of shared action update API action behavior should be according to updated configuration for all flows sharing the action. Shared action query === Provide separate API to query shared action sate (see rte_flow_shared_action_update()). Taking a counter as an example: query returns value aggregating all counter increments across all flow rules sharing the counter. PMD support === The support of introduced API is pure PMD specific design and responsibility for each action type (see struct rte_flow_ops). testpmd === In order to utilize introduced API testpmd cli may implement following extension create/update/destroy/query shared action accordingly flow shared_action (port) create {action_id (id)} (action) / end flow shared_action (port) update (id) (action) / end flow shared_action (port) destroy action_id (id) {action_id (id) [...]} flow shared_action (port) query (id) testpmd example === configure rss to queues 1 & 2 > flow shared_action 0 create action_id 100 rss queues 1 2 end / end create flow rule utilizing shared action > flow create 0 ingress \ pattern eth dst is 0c:42:a1:15:fd:ac / ipv6 / tcp / end \ actions shared 100 / end add 2 more queues > flow shared_action 0 modify 100 rss queues 1 2 3 4 end / end example === struct rte_flow_action actions[2]; struct rte_flow_action action; /* skipped: initialize action */ struct rte_flow_shared_action *handle = rte_flow_shared_action_create( port_id, &action, &error); actions[0].type = RTE_FLOW_ACTION_TYPE_SHARED; actions[0].conf = handle; actions[1].type = RTE_FLOW_ACTION_TYPE_END; /* skipped: init attr0 & pattern0 args */ struct rte_flow *flow0 = rte_flow_create(port_id, &attr0, pattern0, actions, error); /* create more rules reusing shared action */ struct rte_flow *flow1 = rte_flow_create(port_id, &attr1, pattern1, actions, error); /* skipped: for flows 2 till N */ struct rte_flow *flowN = rte_flow_create(port_id, &attrN, patternN, actions, error); /* update shared action */ struct rte_flow_action updated_action; /* * skipped: initialize updated_action according to desired action * configuration change */ rte_flow_shared_action_update(port_id, handle, &updated_action, error); /* * from now on all flows 1 till N will act according to configuration of * updated_action */ /* skipped: destroy all flows 1 till N */ rte_flow_shared_action_destroy(port_id, handle, error); Signed-off-by: Andrey Vesnovaty <andreyv@mellanox.com> Signed-off-by: Andrey Vesnovaty <andreyv@nvidia.com> --- lib/librte_ethdev/rte_ethdev_version.map | 4 + lib/librte_ethdev/rte_flow.c | 82 +++++++++++++ lib/librte_ethdev/rte_flow.h | 148 ++++++++++++++++++++++- lib/librte_ethdev/rte_flow_driver.h | 22 ++++ 4 files changed, 255 insertions(+), 1 deletion(-) diff --git a/lib/librte_ethdev/rte_ethdev_version.map b/lib/librte_ethdev/rte_ethdev_version.map index fc47f6472e..168bf7fc12 100644 --- a/lib/librte_ethdev/rte_ethdev_version.map +++ b/lib/librte_ethdev/rte_ethdev_version.map @@ -225,6 +225,10 @@ EXPERIMENTAL { rte_tm_shared_wred_context_delete; rte_tm_wred_profile_add; rte_tm_wred_profile_delete; + rte_flow_shared_action_create; + rte_flow_shared_action_destroy; + rte_flow_shared_action_update; + rte_flow_shared_action_query; }; INTERNAL { diff --git a/lib/librte_ethdev/rte_flow.c b/lib/librte_ethdev/rte_flow.c index f8fdd68fe9..ba3f01f7c7 100644 --- a/lib/librte_ethdev/rte_flow.c +++ b/lib/librte_ethdev/rte_flow.c @@ -174,6 +174,7 @@ static const struct rte_flow_desc_data rte_flow_desc_action[] = { MK_FLOW_ACTION(SET_IPV4_DSCP, sizeof(struct rte_flow_action_set_dscp)), MK_FLOW_ACTION(SET_IPV6_DSCP, sizeof(struct rte_flow_action_set_dscp)), MK_FLOW_ACTION(AGE, sizeof(struct rte_flow_action_age)), + MK_FLOW_ACTION(SHARED, 0), }; int @@ -1251,3 +1252,84 @@ rte_flow_get_aged_flows(uint16_t port_id, void **contexts, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL, rte_strerror(ENOTSUP)); } + +struct rte_flow_shared_action * +rte_flow_shared_action_create(uint16_t port_id, + const struct rte_flow_action *action, + struct rte_flow_error *error) +{ + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; + struct rte_flow_shared_action *shared_action; + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error); + + if (unlikely(!ops)) + return NULL; + if (likely(!!ops->shared_action_create)) { + shared_action = ops->shared_action_create(dev, action, error); + if (shared_action == NULL) + flow_err(port_id, -rte_errno, error); + return shared_action; + } + rte_flow_error_set(error, ENOSYS, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, rte_strerror(ENOSYS)); + return NULL; +} + +int +rte_flow_shared_action_destroy(uint16_t port_id, + struct rte_flow_shared_action *action, + struct rte_flow_error *error) +{ + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error); + + if (unlikely(!ops)) + return -rte_errno; + if (likely(!!ops->shared_action_destroy)) + return flow_err(port_id, + ops->shared_action_destroy(dev, action, error), + error); + return rte_flow_error_set(error, ENOSYS, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, rte_strerror(ENOSYS)); +} + +int +rte_flow_shared_action_update(uint16_t port_id, + struct rte_flow_shared_action *action, + const struct rte_flow_action *update, + struct rte_flow_error *error) +{ + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error); + + if (unlikely(!ops)) + return -rte_errno; + if (likely(!!ops->shared_action_update)) + return flow_err(port_id, ops->shared_action_update(dev, action, + update, error), + error); + return rte_flow_error_set(error, ENOSYS, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, rte_strerror(ENOSYS)); +} + +int +rte_flow_shared_action_query(uint16_t port_id, + const struct rte_flow_shared_action *action, + void *data, + struct rte_flow_error *error) +{ + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error); + + if (unlikely(!ops)) + return -rte_errno; + if (likely(!!ops->shared_action_query)) + return flow_err(port_id, ops->shared_action_query(dev, action, + data, error), + error); + return rte_flow_error_set(error, ENOSYS, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, rte_strerror(ENOSYS)); +} diff --git a/lib/librte_ethdev/rte_flow.h b/lib/librte_ethdev/rte_flow.h index da8bfa5489..02391316cb 100644 --- a/lib/librte_ethdev/rte_flow.h +++ b/lib/librte_ethdev/rte_flow.h @@ -1714,7 +1714,8 @@ enum rte_flow_action_type { /** * Enables counters for this flow rule. * - * These counters can be retrieved and reset through rte_flow_query(), + * These counters can be retrieved and reset through rte_flow_query() or + * rte_flow_shared_action_query() if the action provided via handle, * see struct rte_flow_query_count. * * See struct rte_flow_action_count. @@ -2132,6 +2133,14 @@ enum rte_flow_action_type { * see enum RTE_ETH_EVENT_FLOW_AGED */ RTE_FLOW_ACTION_TYPE_AGE, + + /** + * Describes action shared a cross multiple flow rules. + * + * Enables multiple rules reference the same action by handle (see + * struct rte_flow_shared_action). + */ + RTE_FLOW_ACTION_TYPE_SHARED, }; /** @@ -2693,6 +2702,20 @@ struct rte_flow_action_set_dscp { uint8_t dscp; }; + +/** + * RTE_FLOW_ACTION_TYPE_SHARED + * + * Opaque type returned after successfully creating a shared action. + * + * This handle can be used to manage and query the related action: + * - share it a cross multiple flow rules + * - update action configuration + * - query action data + * - destroy action + */ +struct rte_flow_shared_action; + /* Mbuf dynamic field offset for metadata. */ extern int32_t rte_flow_dynf_metadata_offs; @@ -3357,6 +3380,129 @@ int rte_flow_get_aged_flows(uint16_t port_id, void **contexts, uint32_t nb_contexts, struct rte_flow_error *error); +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Create shared action for reuse in multiple flow rules. + * + * @param[in] port_id + * The port identifier of the Ethernet device. + * @param[in] action + * Action configuration for shared action creation. + * @param[out] error + * Perform verbose error reporting if not NULL. PMDs initialize this + * structure in case of error only. + * @return + * A valid handle in case of success, NULL otherwise and rte_errno is set + * to one of the error codes defined: + * - (ENOSYS) if underlying device does not support this functionality. + * - (EIO) if underlying device is removed. + * - (EINVAL) if *action* invalid. + * - (ENOTSUP) if *action* valid but unsupported. + */ +__rte_experimental +struct rte_flow_shared_action * +rte_flow_shared_action_create(uint16_t port_id, + const struct rte_flow_action *action, + struct rte_flow_error *error); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Destroys the shared action by handle. + * + * @param[in] port_id + * The port identifier of the Ethernet device. + * @param[in] action + * Handle for the shared action to be destroyed. + * @param[out] error + * Perform verbose error reporting if not NULL. PMDs initialize this + * structure in case of error only. + * @return + * - (0) if success. + * - (-ENOSYS) if underlying device does not support this functionality. + * - (-EIO) if underlying device is removed. + * - (-ENOENT) if action pointed by *action* handle was not found. + * - (-ETOOMANYREFS) if action pointed by *action* handle still used by one or + * more rules + * rte_errno is also set. + */ +__rte_experimental +int +rte_flow_shared_action_destroy(uint16_t port_id, + struct rte_flow_shared_action *action, + struct rte_flow_error *error); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Updates inplace the shared action configuration pointed by *action* handle + * with the configuration provided as *update* argument. + * The update of the shared action configuration effects all flow rules reusing + * the action via handle. + * + * @param[in] port_id + * The port identifier of the Ethernet device. + * @param[in] action + * Handle for the shared action to be updated. + * @param[in] update + * Action specification used to modify the action pointed by handle. + * *update* should be of same type with the action pointed by the *action* + * handle argument, otherwise considered as invalid. + * @param[out] error + * Perform verbose error reporting if not NULL. PMDs initialize this + * structure in case of error only. + * @return + * - (0) if success. + * - (-ENOSYS) if underlying device does not support this functionality. + * - (-EIO) if underlying device is removed. + * - (-EINVAL) if *update* invalid. + * - (-ENOTSUP) if *update* valid but unsupported. + * - (-ENOENT) if action pointed by *ctx* was not found. + * rte_errno is also set. + */ +__rte_experimental +int +rte_flow_shared_action_update(uint16_t port_id, + struct rte_flow_shared_action *action, + const struct rte_flow_action *update, + struct rte_flow_error *error); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Query the shared action by handle. + * + * This function allows retrieving action-specific data such as counters. + * Data is gathered by special action which may be present/referenced in + * more than one flow rule definition. + * + * \see RTE_FLOW_ACTION_TYPE_COUNT + * + * @param port_id + * Port identifier of Ethernet device. + * @param[in] action + * Handle for the shared action to query. + * @param[in, out] data + * Pointer to storage for the associated query data type. + * @param[out] error + * Perform verbose error reporting if not NULL. PMDs initialize this + * structure in case of error only. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +__rte_experimental +int +rte_flow_shared_action_query(uint16_t port_id, + const struct rte_flow_shared_action *action, + void *data, + struct rte_flow_error *error); + #ifdef __cplusplus } #endif diff --git a/lib/librte_ethdev/rte_flow_driver.h b/lib/librte_ethdev/rte_flow_driver.h index 3ee871d3eb..72bfc3b7a7 100644 --- a/lib/librte_ethdev/rte_flow_driver.h +++ b/lib/librte_ethdev/rte_flow_driver.h @@ -108,6 +108,28 @@ struct rte_flow_ops { void **context, uint32_t nb_contexts, struct rte_flow_error *err); + /** See rte_flow_shared_action_create() */ + struct rte_flow_shared_action *(*shared_action_create) + (struct rte_eth_dev *dev, + const struct rte_flow_action *action, + struct rte_flow_error *error); + /** See rte_flow_shared_action_destroy() */ + int (*shared_action_destroy) + (struct rte_eth_dev *dev, + struct rte_flow_shared_action *shared_action, + struct rte_flow_error *error); + /** See rte_flow_shared_action_update() */ + int (*shared_action_update) + (struct rte_eth_dev *dev, + struct rte_flow_shared_action *shared_action, + const struct rte_flow_action *update, + struct rte_flow_error *error); + /** See rte_flow_shared_action_query() */ + int (*shared_action_query) + (struct rte_eth_dev *dev, + const struct rte_flow_shared_action *shared_action, + void *data, + struct rte_flow_error *error); }; /** -- 2.26.2 ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [dpdk-dev] [PATCH v3 01/10] ethdev: add flow shared action API 2020-10-03 22:06 ` [dpdk-dev] [PATCH v3 01/10] ethdev: add flow shared action API Andrey Vesnovaty @ 2020-10-04 11:10 ` Ori Kam 2020-10-06 10:22 ` Andrey Vesnovaty 2020-10-04 17:00 ` Stephen Hemminger 1 sibling, 1 reply; 106+ messages in thread From: Ori Kam @ 2020-10-04 11:10 UTC (permalink / raw) To: Andrey Vesnovaty, dev Cc: jer, jerinjacobk, NBU-Contact-Thomas Monjalon, ferruh.yigit, stephen, bruce.richardson, Slava Ovsiienko, andrey.vesnovaty, mdr, nhorman, ajit.khaparde, samik.gupta, Andrey Vesnovaty, Andrew Rybchenko, Ori Kam Hi Andrey, You are missing the doc update (rte_flow) PSB for more comments. Best, Ori > -----Original Message----- > From: Andrey Vesnovaty <andreyv@nvidia.com> > Sent: Sunday, October 4, 2020 1:06 AM > Subject: [PATCH v3 01/10] ethdev: add flow shared action API > > This commit introduces extension of DPDK flow action API enabling > sharing of single rte_flow_action in multiple flows. The API intended for > PMDs where multiple HW offloaded flows can reuse the same HW > essence/object representing flow action and modification of such an > essence/object effects all the rules using it. > > Motivation and example > === > Adding or removing one or more queues to RSS used by multiple flow rules > imposes per rule toll for current DPDK flow API; the scenario requires > for each flow sharing cloned RSS action: > - call `rte_flow_destroy()` > - call `rte_flow_create()` with modified RSS action > > API for sharing action and its in-place update benefits: > - reduce the overhead of multiple RSS flow rules reconfiguration > - optimize resource utilization by sharing action across of multiple > flows > > Change description > === > > Shared action > === > In order to represent flow action shared by multiple flows new action > type RTE_FLOW_ACTION_TYPE_SHARED is introduced (see `enum > rte_flow_action_type`). > Actually the introduced API decouples action from any specific flow and > enables sharing of single action by its handle across multiple flows. > > Shared action create/use/destroy > === > Shared action may be reused by some or none flow rules at any given > moment, i.e. shared action reside outside of the context of any flow. > Shared action represent HW resources/objects used for action offloading > implementation. > API for shared action create (see `rte_flow_shared_action_create()`): > - should allocate HW resources and make related initializations required > for shared action implementation. > - make necessary preparations to maintain shared access to > the action resources, configuration and state. > API for shared action destroy (see `rte_flow_shared_action_destroy()`) > should release HW resources and make related cleanups required for shared > action implementation. > > In order to share some flow action reuse the handle of type > `struct rte_flow_shared_action` returned by > rte_flow_shared_action_create() as a `conf` field of > `struct rte_flow_action` (see "example" section). > > If some shared action not used by any flow rule all resources allocated > by the shared action can be released by rte_flow_shared_action_destroy() > (see "example" section). The shared action handle passed as argument to > destroy API should not be used any further i.e. result of the usage is > undefined. > > Shared action re-configuration > === > Shared action behavior defined by its configuration can be updated via > rte_flow_shared_action_update() (see "example" section). The shared > action update operation modifies HW related resources/objects allocated > on the action creation. The number of operations performed by the update > operation should not be dependent on number of flows sharing the related > action. On return of shared action update API action behavior should be > according to updated configuration for all flows sharing the action. > > Shared action query > === > Provide separate API to query shared action sate (see I think sate --> state, also it is not clear if we are sharing both configuration and state or only configuration. For example what will happen if we create counter and attached it to number of flow? Does it count for all flow so counter = flow1 + flow2? Or it is only for configuration which means that one each flow gets different counter? For counter there is a shared bit in the action so this can be used to determent if the state should be shared, but counter doesn't have configuration to share. What happens with aging where we have state and configuration? I'm O.K with starting just with configuration, but it should be clear. > rte_flow_shared_action_update()). Taking a counter as an example: query > returns value aggregating all counter increments across all flow rules > sharing the counter. > > PMD support > === > The support of introduced API is pure PMD specific design and > responsibility for each action type (see struct rte_flow_ops). > > testpmd > === > In order to utilize introduced API testpmd cli may implement following > extension > create/update/destroy/query shared action accordingly > > flow shared_action (port) create {action_id (id)} (action) / end > flow shared_action (port) update (id) (action) / end > flow shared_action (port) destroy action_id (id) {action_id (id) [...]} > flow shared_action (port) query (id) > > testpmd example > === > > configure rss to queues 1 & 2 > > > flow shared_action 0 create action_id 100 rss queues 1 2 end / end > > create flow rule utilizing shared action > > > flow create 0 ingress \ > pattern eth dst is 0c:42:a1:15:fd:ac / ipv6 / tcp / end \ > actions shared 100 / end > > add 2 more queues > > > flow shared_action 0 modify 100 rss queues 1 2 3 4 end / end > > example > === > > struct rte_flow_action actions[2]; > struct rte_flow_action action; > /* skipped: initialize action */ > struct rte_flow_shared_action *handle = rte_flow_shared_action_create( > port_id, &action, &error); > actions[0].type = RTE_FLOW_ACTION_TYPE_SHARED; > actions[0].conf = handle; > actions[1].type = RTE_FLOW_ACTION_TYPE_END; > /* skipped: init attr0 & pattern0 args */ > struct rte_flow *flow0 = rte_flow_create(port_id, &attr0, pattern0, > actions, error); > /* create more rules reusing shared action */ > struct rte_flow *flow1 = rte_flow_create(port_id, &attr1, pattern1, > actions, error); > /* skipped: for flows 2 till N */ > struct rte_flow *flowN = rte_flow_create(port_id, &attrN, patternN, > actions, error); > /* update shared action */ > struct rte_flow_action updated_action; > /* > * skipped: initialize updated_action according to desired action > * configuration change > */ > rte_flow_shared_action_update(port_id, handle, &updated_action, error); > /* > * from now on all flows 1 till N will act according to configuration of > * updated_action > */ > /* skipped: destroy all flows 1 till N */ > rte_flow_shared_action_destroy(port_id, handle, error); > > Signed-off-by: Andrey Vesnovaty <andreyv@mellanox.com> > Signed-off-by: Andrey Vesnovaty <andreyv@nvidia.com> > --- > lib/librte_ethdev/rte_ethdev_version.map | 4 + > lib/librte_ethdev/rte_flow.c | 82 +++++++++++++ > lib/librte_ethdev/rte_flow.h | 148 ++++++++++++++++++++++- > lib/librte_ethdev/rte_flow_driver.h | 22 ++++ > 4 files changed, 255 insertions(+), 1 deletion(-) > > diff --git a/lib/librte_ethdev/rte_ethdev_version.map > b/lib/librte_ethdev/rte_ethdev_version.map > index fc47f6472e..168bf7fc12 100644 > --- a/lib/librte_ethdev/rte_ethdev_version.map > +++ b/lib/librte_ethdev/rte_ethdev_version.map > @@ -225,6 +225,10 @@ EXPERIMENTAL { > rte_tm_shared_wred_context_delete; > rte_tm_wred_profile_add; > rte_tm_wred_profile_delete; > + rte_flow_shared_action_create; > + rte_flow_shared_action_destroy; > + rte_flow_shared_action_update; > + rte_flow_shared_action_query; > }; > > INTERNAL { > diff --git a/lib/librte_ethdev/rte_flow.c b/lib/librte_ethdev/rte_flow.c > index f8fdd68fe9..ba3f01f7c7 100644 > --- a/lib/librte_ethdev/rte_flow.c > +++ b/lib/librte_ethdev/rte_flow.c > @@ -174,6 +174,7 @@ static const struct rte_flow_desc_data > rte_flow_desc_action[] = { > MK_FLOW_ACTION(SET_IPV4_DSCP, sizeof(struct > rte_flow_action_set_dscp)), > MK_FLOW_ACTION(SET_IPV6_DSCP, sizeof(struct > rte_flow_action_set_dscp)), > MK_FLOW_ACTION(AGE, sizeof(struct rte_flow_action_age)), > + MK_FLOW_ACTION(SHARED, 0), > }; > > int > @@ -1251,3 +1252,84 @@ rte_flow_get_aged_flows(uint16_t port_id, void > **contexts, > RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > NULL, rte_strerror(ENOTSUP)); > } > + > +struct rte_flow_shared_action * > +rte_flow_shared_action_create(uint16_t port_id, > + const struct rte_flow_action *action, > + struct rte_flow_error *error) You are missing the property for direction that we agreed in previous version. > +{ > + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; > + struct rte_flow_shared_action *shared_action; > + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error); > + > + if (unlikely(!ops)) > + return NULL; > + if (likely(!!ops->shared_action_create)) { > + shared_action = ops->shared_action_create(dev, action, error); > + if (shared_action == NULL) > + flow_err(port_id, -rte_errno, error); > + return shared_action; > + } > + rte_flow_error_set(error, ENOSYS, > RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > + NULL, rte_strerror(ENOSYS)); > + return NULL; > +} > + > +int > +rte_flow_shared_action_destroy(uint16_t port_id, > + struct rte_flow_shared_action *action, > + struct rte_flow_error *error) > +{ > + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; > + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error); > + > + if (unlikely(!ops)) > + return -rte_errno; > + if (likely(!!ops->shared_action_destroy)) > + return flow_err(port_id, > + ops->shared_action_destroy(dev, action, > error), > + error); > + return rte_flow_error_set(error, ENOSYS, > + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > + NULL, rte_strerror(ENOSYS)); > +} > + > +int > +rte_flow_shared_action_update(uint16_t port_id, > + struct rte_flow_shared_action *action, > + const struct rte_flow_action *update, > + struct rte_flow_error *error) > +{ > + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; > + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error); > + > + if (unlikely(!ops)) > + return -rte_errno; > + if (likely(!!ops->shared_action_update)) > + return flow_err(port_id, ops->shared_action_update(dev, > action, > + update, error), > + error); > + return rte_flow_error_set(error, ENOSYS, > + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > + NULL, rte_strerror(ENOSYS)); > +} > + > +int > +rte_flow_shared_action_query(uint16_t port_id, > + const struct rte_flow_shared_action *action, > + void *data, > + struct rte_flow_error *error) > +{ > + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; > + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error); > + > + if (unlikely(!ops)) > + return -rte_errno; > + if (likely(!!ops->shared_action_query)) > + return flow_err(port_id, ops->shared_action_query(dev, action, > + data, error), > + error); > + return rte_flow_error_set(error, ENOSYS, > + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > + NULL, rte_strerror(ENOSYS)); > +} > diff --git a/lib/librte_ethdev/rte_flow.h b/lib/librte_ethdev/rte_flow.h > index da8bfa5489..02391316cb 100644 > --- a/lib/librte_ethdev/rte_flow.h > +++ b/lib/librte_ethdev/rte_flow.h > @@ -1714,7 +1714,8 @@ enum rte_flow_action_type { > /** > * Enables counters for this flow rule. > * > - * These counters can be retrieved and reset through rte_flow_query(), > + * These counters can be retrieved and reset through rte_flow_query() > or > + * rte_flow_shared_action_query() if the action provided via handle, > * see struct rte_flow_query_count. > * > * See struct rte_flow_action_count. > @@ -2132,6 +2133,14 @@ enum rte_flow_action_type { > * see enum RTE_ETH_EVENT_FLOW_AGED > */ > RTE_FLOW_ACTION_TYPE_AGE, > + > + /** > + * Describes action shared a cross multiple flow rules. > + * > + * Enables multiple rules reference the same action by handle (see > + * struct rte_flow_shared_action). > + */ > + RTE_FLOW_ACTION_TYPE_SHARED, > }; > > /** > @@ -2693,6 +2702,20 @@ struct rte_flow_action_set_dscp { > uint8_t dscp; > }; > > + > +/** > + * RTE_FLOW_ACTION_TYPE_SHARED > + * > + * Opaque type returned after successfully creating a shared action. > + * > + * This handle can be used to manage and query the related action: > + * - share it a cross multiple flow rules > + * - update action configuration > + * - query action data > + * - destroy action > + */ > +struct rte_flow_shared_action; > + > /* Mbuf dynamic field offset for metadata. */ > extern int32_t rte_flow_dynf_metadata_offs; > > @@ -3357,6 +3380,129 @@ int > rte_flow_get_aged_flows(uint16_t port_id, void **contexts, > uint32_t nb_contexts, struct rte_flow_error *error); > > +/** > + * @warning > + * @b EXPERIMENTAL: this API may change without prior notice. > + * > + * Create shared action for reuse in multiple flow rules. > + * > + * @param[in] port_id > + * The port identifier of the Ethernet device. > + * @param[in] action > + * Action configuration for shared action creation. > + * @param[out] error > + * Perform verbose error reporting if not NULL. PMDs initialize this > + * structure in case of error only. > + * @return > + * A valid handle in case of success, NULL otherwise and rte_errno is set > + * to one of the error codes defined: > + * - (ENOSYS) if underlying device does not support this functionality. > + * - (EIO) if underlying device is removed. > + * - (EINVAL) if *action* invalid. > + * - (ENOTSUP) if *action* valid but unsupported. > + */ > +__rte_experimental > +struct rte_flow_shared_action * > +rte_flow_shared_action_create(uint16_t port_id, > + const struct rte_flow_action *action, > + struct rte_flow_error *error); > + > +/** > + * @warning > + * @b EXPERIMENTAL: this API may change without prior notice. > + * > + * Destroys the shared action by handle. > + * > + * @param[in] port_id > + * The port identifier of the Ethernet device. > + * @param[in] action > + * Handle for the shared action to be destroyed. > + * @param[out] error > + * Perform verbose error reporting if not NULL. PMDs initialize this > + * structure in case of error only. > + * @return > + * - (0) if success. > + * - (-ENOSYS) if underlying device does not support this functionality. > + * - (-EIO) if underlying device is removed. > + * - (-ENOENT) if action pointed by *action* handle was not found. > + * - (-ETOOMANYREFS) if action pointed by *action* handle still used by one > or > + * more rules > + * rte_errno is also set. > + */ > +__rte_experimental > +int > +rte_flow_shared_action_destroy(uint16_t port_id, > + struct rte_flow_shared_action *action, > + struct rte_flow_error *error); > + > +/** > + * @warning > + * @b EXPERIMENTAL: this API may change without prior notice. > + * > + * Updates inplace the shared action configuration pointed by *action* handle > + * with the configuration provided as *update* argument. > + * The update of the shared action configuration effects all flow rules reusing > + * the action via handle. > + * > + * @param[in] port_id > + * The port identifier of the Ethernet device. > + * @param[in] action > + * Handle for the shared action to be updated. > + * @param[in] update > + * Action specification used to modify the action pointed by handle. > + * *update* should be of same type with the action pointed by the *action* > + * handle argument, otherwise considered as invalid. > + * @param[out] error > + * Perform verbose error reporting if not NULL. PMDs initialize this > + * structure in case of error only. > + * @return > + * - (0) if success. > + * - (-ENOSYS) if underlying device does not support this functionality. > + * - (-EIO) if underlying device is removed. > + * - (-EINVAL) if *update* invalid. > + * - (-ENOTSUP) if *update* valid but unsupported. > + * - (-ENOENT) if action pointed by *ctx* was not found. > + * rte_errno is also set. > + */ > +__rte_experimental > +int > +rte_flow_shared_action_update(uint16_t port_id, > + struct rte_flow_shared_action *action, > + const struct rte_flow_action *update, > + struct rte_flow_error *error); > + > +/** > + * @warning > + * @b EXPERIMENTAL: this API may change without prior notice. > + * > + * Query the shared action by handle. > + * > + * This function allows retrieving action-specific data such as counters. > + * Data is gathered by special action which may be present/referenced in > + * more than one flow rule definition. > + * > + * \see RTE_FLOW_ACTION_TYPE_COUNT > + * > + * @param port_id > + * Port identifier of Ethernet device. > + * @param[in] action > + * Handle for the shared action to query. > + * @param[in, out] data > + * Pointer to storage for the associated query data type. > + * @param[out] error > + * Perform verbose error reporting if not NULL. PMDs initialize this > + * structure in case of error only. > + * > + * @return > + * 0 on success, a negative errno value otherwise and rte_errno is set. > + */ > +__rte_experimental > +int > +rte_flow_shared_action_query(uint16_t port_id, > + const struct rte_flow_shared_action *action, > + void *data, > + struct rte_flow_error *error); > + > #ifdef __cplusplus > } > #endif > diff --git a/lib/librte_ethdev/rte_flow_driver.h > b/lib/librte_ethdev/rte_flow_driver.h > index 3ee871d3eb..72bfc3b7a7 100644 > --- a/lib/librte_ethdev/rte_flow_driver.h > +++ b/lib/librte_ethdev/rte_flow_driver.h > @@ -108,6 +108,28 @@ struct rte_flow_ops { > void **context, > uint32_t nb_contexts, > struct rte_flow_error *err); > + /** See rte_flow_shared_action_create() */ > + struct rte_flow_shared_action *(*shared_action_create) > + (struct rte_eth_dev *dev, > + const struct rte_flow_action *action, > + struct rte_flow_error *error); > + /** See rte_flow_shared_action_destroy() */ > + int (*shared_action_destroy) > + (struct rte_eth_dev *dev, > + struct rte_flow_shared_action *shared_action, > + struct rte_flow_error *error); > + /** See rte_flow_shared_action_update() */ > + int (*shared_action_update) > + (struct rte_eth_dev *dev, > + struct rte_flow_shared_action *shared_action, > + const struct rte_flow_action *update, > + struct rte_flow_error *error); > + /** See rte_flow_shared_action_query() */ > + int (*shared_action_query) > + (struct rte_eth_dev *dev, > + const struct rte_flow_shared_action *shared_action, > + void *data, > + struct rte_flow_error *error); > }; > > /** > -- > 2.26.2 ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [dpdk-dev] [PATCH v3 01/10] ethdev: add flow shared action API 2020-10-04 11:10 ` Ori Kam @ 2020-10-06 10:22 ` Andrey Vesnovaty 0 siblings, 0 replies; 106+ messages in thread From: Andrey Vesnovaty @ 2020-10-06 10:22 UTC (permalink / raw) To: Ori Kam, dev Cc: jer, jerinjacobk, NBU-Contact-Thomas Monjalon, ferruh.yigit, stephen, bruce.richardson, Slava Ovsiienko, andrey.vesnovaty, mdr, nhorman, ajit.khaparde, samik.gupta, Andrey Vesnovaty, Andrew Rybchenko, Ori Kam Hi Ori, PSB Thanks, Andrey > -----Original Message----- > From: Ori Kam <orika@nvidia.com> > Sent: Sunday, October 4, 2020 2:10 PM > To: Andrey Vesnovaty <andreyv@nvidia.com>; dev@dpdk.org > Cc: jer@marvell.com; jerinjacobk@gmail.com; NBU-Contact-Thomas Monjalon > <thomas@monjalon.net>; ferruh.yigit@intel.com; > stephen@networkplumber.org; bruce.richardson@intel.com; Slava Ovsiienko > <viacheslavo@nvidia.com>; andrey.vesnovaty@gmail.com; mdr@ashroe.eu; > nhorman@tuxdriver.com; ajit.khaparde@broadcom.com; > samik.gupta@broadcom.com; Andrey Vesnovaty <andreyv@mellanox.com>; > Andrew Rybchenko <arybchenko@solarflare.com>; Ori Kam > <orika@mellanox.com> > Subject: RE: [PATCH v3 01/10] ethdev: add flow shared action API > > Hi Andrey, > > You are missing the doc update (rte_flow) Got it. Will update in release_notes_20.11.rst & testpmd. > PSB for more comments. > > Best, > Ori > > > -----Original Message----- > > From: Andrey Vesnovaty <andreyv@nvidia.com> > > Sent: Sunday, October 4, 2020 1:06 AM > > Subject: [PATCH v3 01/10] ethdev: add flow shared action API > > > > This commit introduces extension of DPDK flow action API enabling > > sharing of single rte_flow_action in multiple flows. The API intended for > > PMDs where multiple HW offloaded flows can reuse the same HW > > essence/object representing flow action and modification of such an > > essence/object effects all the rules using it. > > > > Motivation and example > > === > > Adding or removing one or more queues to RSS used by multiple flow rules > > imposes per rule toll for current DPDK flow API; the scenario requires > > for each flow sharing cloned RSS action: > > - call `rte_flow_destroy()` > > - call `rte_flow_create()` with modified RSS action > > > > API for sharing action and its in-place update benefits: > > - reduce the overhead of multiple RSS flow rules reconfiguration > > - optimize resource utilization by sharing action across of multiple > > flows > > > > Change description > > === > > > > Shared action > > === > > In order to represent flow action shared by multiple flows new action > > type RTE_FLOW_ACTION_TYPE_SHARED is introduced (see `enum > > rte_flow_action_type`). > > Actually the introduced API decouples action from any specific flow and > > enables sharing of single action by its handle across multiple flows. > > > > Shared action create/use/destroy > > === > > Shared action may be reused by some or none flow rules at any given > > moment, i.e. shared action reside outside of the context of any flow. > > Shared action represent HW resources/objects used for action offloading > > implementation. > > API for shared action create (see `rte_flow_shared_action_create()`): > > - should allocate HW resources and make related initializations required > > for shared action implementation. > > - make necessary preparations to maintain shared access to > > the action resources, configuration and state. > > API for shared action destroy (see `rte_flow_shared_action_destroy()`) > > should release HW resources and make related cleanups required for shared > > action implementation. > > > > In order to share some flow action reuse the handle of type > > `struct rte_flow_shared_action` returned by > > rte_flow_shared_action_create() as a `conf` field of > > `struct rte_flow_action` (see "example" section). > > > > If some shared action not used by any flow rule all resources allocated > > by the shared action can be released by rte_flow_shared_action_destroy() > > (see "example" section). The shared action handle passed as argument to > > destroy API should not be used any further i.e. result of the usage is > > undefined. > > > > Shared action re-configuration > > === > > Shared action behavior defined by its configuration can be updated via > > rte_flow_shared_action_update() (see "example" section). The shared > > action update operation modifies HW related resources/objects allocated > > on the action creation. The number of operations performed by the update > > operation should not be dependent on number of flows sharing the related > > action. On return of shared action update API action behavior should be > > according to updated configuration for all flows sharing the action. > > > > Shared action query > > === > > Provide separate API to query shared action sate (see > > I think sate --> state, also it is not clear if we are sharing both configuration and > state > or only configuration. For example what will happen if we create counter and > attached it > to number of flow? Does it count for all flow so counter = flow1 + flow2? > Or it is only for configuration which means that one each flow gets different > counter? > For counter there is a shared bit in the action so this can be used to determent if > the state > should be shared, but counter doesn't have configuration to share. > What happens with aging where we have state and configuration? > > I'm O.K with starting just with configuration, but it should be clear. > Shared action state considered shared till it's defined differently. So when shared action queried it's considered query for state. Configuration of shred action controlled by rte_flow_sahred_action_create() and rte_flow_shared_action_update() APIs and no supposed to change by other means. Do you think there is a need to query shared action configuration? > > rte_flow_shared_action_update()). Taking a counter as an example: query > > returns value aggregating all counter increments across all flow rules > > sharing the counter. > > > > PMD support > > === > > The support of introduced API is pure PMD specific design and > > responsibility for each action type (see struct rte_flow_ops). > > > > testpmd > > === > > In order to utilize introduced API testpmd cli may implement following > > extension > > create/update/destroy/query shared action accordingly > > > > flow shared_action (port) create {action_id (id)} (action) / end > > flow shared_action (port) update (id) (action) / end > > flow shared_action (port) destroy action_id (id) {action_id (id) [...]} > > flow shared_action (port) query (id) > > > > testpmd example > > === > > > > configure rss to queues 1 & 2 > > > > > flow shared_action 0 create action_id 100 rss queues 1 2 end / end > > > > create flow rule utilizing shared action > > > > > flow create 0 ingress \ > > pattern eth dst is 0c:42:a1:15:fd:ac / ipv6 / tcp / end \ > > actions shared 100 / end > > > > add 2 more queues > > > > > flow shared_action 0 modify 100 rss queues 1 2 3 4 end / end > > > > example > > === > > > > struct rte_flow_action actions[2]; > > struct rte_flow_action action; > > /* skipped: initialize action */ > > struct rte_flow_shared_action *handle = rte_flow_shared_action_create( > > port_id, &action, &error); > > actions[0].type = RTE_FLOW_ACTION_TYPE_SHARED; > > actions[0].conf = handle; > > actions[1].type = RTE_FLOW_ACTION_TYPE_END; > > /* skipped: init attr0 & pattern0 args */ > > struct rte_flow *flow0 = rte_flow_create(port_id, &attr0, pattern0, > > actions, error); > > /* create more rules reusing shared action */ > > struct rte_flow *flow1 = rte_flow_create(port_id, &attr1, pattern1, > > actions, error); > > /* skipped: for flows 2 till N */ > > struct rte_flow *flowN = rte_flow_create(port_id, &attrN, patternN, > > actions, error); > > /* update shared action */ > > struct rte_flow_action updated_action; > > /* > > * skipped: initialize updated_action according to desired action > > * configuration change > > */ > > rte_flow_shared_action_update(port_id, handle, &updated_action, error); > > /* > > * from now on all flows 1 till N will act according to configuration of > > * updated_action > > */ > > /* skipped: destroy all flows 1 till N */ > > rte_flow_shared_action_destroy(port_id, handle, error); > > > > Signed-off-by: Andrey Vesnovaty <andreyv@mellanox.com> > > Signed-off-by: Andrey Vesnovaty <andreyv@nvidia.com> > > --- > > lib/librte_ethdev/rte_ethdev_version.map | 4 + > > lib/librte_ethdev/rte_flow.c | 82 +++++++++++++ > > lib/librte_ethdev/rte_flow.h | 148 ++++++++++++++++++++++- > > lib/librte_ethdev/rte_flow_driver.h | 22 ++++ > > 4 files changed, 255 insertions(+), 1 deletion(-) > > > > diff --git a/lib/librte_ethdev/rte_ethdev_version.map > > b/lib/librte_ethdev/rte_ethdev_version.map > > index fc47f6472e..168bf7fc12 100644 > > --- a/lib/librte_ethdev/rte_ethdev_version.map > > +++ b/lib/librte_ethdev/rte_ethdev_version.map > > @@ -225,6 +225,10 @@ EXPERIMENTAL { > > rte_tm_shared_wred_context_delete; > > rte_tm_wred_profile_add; > > rte_tm_wred_profile_delete; > > + rte_flow_shared_action_create; > > + rte_flow_shared_action_destroy; > > + rte_flow_shared_action_update; > > + rte_flow_shared_action_query; > > }; > > > > INTERNAL { > > diff --git a/lib/librte_ethdev/rte_flow.c b/lib/librte_ethdev/rte_flow.c > > index f8fdd68fe9..ba3f01f7c7 100644 > > --- a/lib/librte_ethdev/rte_flow.c > > +++ b/lib/librte_ethdev/rte_flow.c > > @@ -174,6 +174,7 @@ static const struct rte_flow_desc_data > > rte_flow_desc_action[] = { > > MK_FLOW_ACTION(SET_IPV4_DSCP, sizeof(struct > > rte_flow_action_set_dscp)), > > MK_FLOW_ACTION(SET_IPV6_DSCP, sizeof(struct > > rte_flow_action_set_dscp)), > > MK_FLOW_ACTION(AGE, sizeof(struct rte_flow_action_age)), > > + MK_FLOW_ACTION(SHARED, 0), > > }; > > > > int > > @@ -1251,3 +1252,84 @@ rte_flow_get_aged_flows(uint16_t port_id, void > > **contexts, > > RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > > NULL, rte_strerror(ENOTSUP)); > > } > > + > > +struct rte_flow_shared_action * > > +rte_flow_shared_action_create(uint16_t port_id, > > + const struct rte_flow_action *action, > > + struct rte_flow_error *error) > > You are missing the property for direction that we agreed in previous version. > > > +{ > > + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; > > + struct rte_flow_shared_action *shared_action; > > + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error); > > + > > + if (unlikely(!ops)) > > + return NULL; > > + if (likely(!!ops->shared_action_create)) { > > + shared_action = ops->shared_action_create(dev, action, error); > > + if (shared_action == NULL) > > + flow_err(port_id, -rte_errno, error); > > + return shared_action; > > + } > > + rte_flow_error_set(error, ENOSYS, > > RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > > + NULL, rte_strerror(ENOSYS)); > > + return NULL; > > +} > > + > > +int > > +rte_flow_shared_action_destroy(uint16_t port_id, > > + struct rte_flow_shared_action *action, > > + struct rte_flow_error *error) > > +{ > > + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; > > + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error); > > + > > + if (unlikely(!ops)) > > + return -rte_errno; > > + if (likely(!!ops->shared_action_destroy)) > > + return flow_err(port_id, > > + ops->shared_action_destroy(dev, action, > > error), > > + error); > > + return rte_flow_error_set(error, ENOSYS, > > + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > > + NULL, rte_strerror(ENOSYS)); > > +} > > + > > +int > > +rte_flow_shared_action_update(uint16_t port_id, > > + struct rte_flow_shared_action *action, > > + const struct rte_flow_action *update, > > + struct rte_flow_error *error) > > +{ > > + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; > > + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error); > > + > > + if (unlikely(!ops)) > > + return -rte_errno; > > + if (likely(!!ops->shared_action_update)) > > + return flow_err(port_id, ops->shared_action_update(dev, > > action, > > + update, error), > > + error); > > + return rte_flow_error_set(error, ENOSYS, > > + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > > + NULL, rte_strerror(ENOSYS)); > > +} > > + > > +int > > +rte_flow_shared_action_query(uint16_t port_id, > > + const struct rte_flow_shared_action *action, > > + void *data, > > + struct rte_flow_error *error) > > +{ > > + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; > > + const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error); > > + > > + if (unlikely(!ops)) > > + return -rte_errno; > > + if (likely(!!ops->shared_action_query)) > > + return flow_err(port_id, ops->shared_action_query(dev, action, > > + data, error), > > + error); > > + return rte_flow_error_set(error, ENOSYS, > > + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > > + NULL, rte_strerror(ENOSYS)); > > +} > > diff --git a/lib/librte_ethdev/rte_flow.h b/lib/librte_ethdev/rte_flow.h > > index da8bfa5489..02391316cb 100644 > > --- a/lib/librte_ethdev/rte_flow.h > > +++ b/lib/librte_ethdev/rte_flow.h > > @@ -1714,7 +1714,8 @@ enum rte_flow_action_type { > > /** > > * Enables counters for this flow rule. > > * > > - * These counters can be retrieved and reset through rte_flow_query(), > > + * These counters can be retrieved and reset through rte_flow_query() > > or > > + * rte_flow_shared_action_query() if the action provided via handle, > > * see struct rte_flow_query_count. > > * > > * See struct rte_flow_action_count. > > @@ -2132,6 +2133,14 @@ enum rte_flow_action_type { > > * see enum RTE_ETH_EVENT_FLOW_AGED > > */ > > RTE_FLOW_ACTION_TYPE_AGE, > > + > > + /** > > + * Describes action shared a cross multiple flow rules. > > + * > > + * Enables multiple rules reference the same action by handle (see > > + * struct rte_flow_shared_action). > > + */ > > + RTE_FLOW_ACTION_TYPE_SHARED, > > }; > > > > /** > > @@ -2693,6 +2702,20 @@ struct rte_flow_action_set_dscp { > > uint8_t dscp; > > }; > > > > + > > +/** > > + * RTE_FLOW_ACTION_TYPE_SHARED > > + * > > + * Opaque type returned after successfully creating a shared action. > > + * > > + * This handle can be used to manage and query the related action: > > + * - share it a cross multiple flow rules > > + * - update action configuration > > + * - query action data > > + * - destroy action > > + */ > > +struct rte_flow_shared_action; > > + > > /* Mbuf dynamic field offset for metadata. */ > > extern int32_t rte_flow_dynf_metadata_offs; > > > > @@ -3357,6 +3380,129 @@ int > > rte_flow_get_aged_flows(uint16_t port_id, void **contexts, > > uint32_t nb_contexts, struct rte_flow_error *error); > > > > +/** > > + * @warning > > + * @b EXPERIMENTAL: this API may change without prior notice. > > + * > > + * Create shared action for reuse in multiple flow rules. > > + * > > + * @param[in] port_id > > + * The port identifier of the Ethernet device. > > + * @param[in] action > > + * Action configuration for shared action creation. > > + * @param[out] error > > + * Perform verbose error reporting if not NULL. PMDs initialize this > > + * structure in case of error only. > > + * @return > > + * A valid handle in case of success, NULL otherwise and rte_errno is set > > + * to one of the error codes defined: > > + * - (ENOSYS) if underlying device does not support this functionality. > > + * - (EIO) if underlying device is removed. > > + * - (EINVAL) if *action* invalid. > > + * - (ENOTSUP) if *action* valid but unsupported. > > + */ > > +__rte_experimental > > +struct rte_flow_shared_action * > > +rte_flow_shared_action_create(uint16_t port_id, > > + const struct rte_flow_action *action, > > + struct rte_flow_error *error); > > + > > +/** > > + * @warning > > + * @b EXPERIMENTAL: this API may change without prior notice. > > + * > > + * Destroys the shared action by handle. > > + * > > + * @param[in] port_id > > + * The port identifier of the Ethernet device. > > + * @param[in] action > > + * Handle for the shared action to be destroyed. > > + * @param[out] error > > + * Perform verbose error reporting if not NULL. PMDs initialize this > > + * structure in case of error only. > > + * @return > > + * - (0) if success. > > + * - (-ENOSYS) if underlying device does not support this functionality. > > + * - (-EIO) if underlying device is removed. > > + * - (-ENOENT) if action pointed by *action* handle was not found. > > + * - (-ETOOMANYREFS) if action pointed by *action* handle still used by one > > or > > + * more rules > > + * rte_errno is also set. > > + */ > > +__rte_experimental > > +int > > +rte_flow_shared_action_destroy(uint16_t port_id, > > + struct rte_flow_shared_action *action, > > + struct rte_flow_error *error); > > + > > +/** > > + * @warning > > + * @b EXPERIMENTAL: this API may change without prior notice. > > + * > > + * Updates inplace the shared action configuration pointed by *action* > handle > > + * with the configuration provided as *update* argument. > > + * The update of the shared action configuration effects all flow rules reusing > > + * the action via handle. > > + * > > + * @param[in] port_id > > + * The port identifier of the Ethernet device. > > + * @param[in] action > > + * Handle for the shared action to be updated. > > + * @param[in] update > > + * Action specification used to modify the action pointed by handle. > > + * *update* should be of same type with the action pointed by the *action* > > + * handle argument, otherwise considered as invalid. > > + * @param[out] error > > + * Perform verbose error reporting if not NULL. PMDs initialize this > > + * structure in case of error only. > > + * @return > > + * - (0) if success. > > + * - (-ENOSYS) if underlying device does not support this functionality. > > + * - (-EIO) if underlying device is removed. > > + * - (-EINVAL) if *update* invalid. > > + * - (-ENOTSUP) if *update* valid but unsupported. > > + * - (-ENOENT) if action pointed by *ctx* was not found. > > + * rte_errno is also set. > > + */ > > +__rte_experimental > > +int > > +rte_flow_shared_action_update(uint16_t port_id, > > + struct rte_flow_shared_action *action, > > + const struct rte_flow_action *update, > > + struct rte_flow_error *error); > > + > > +/** > > + * @warning > > + * @b EXPERIMENTAL: this API may change without prior notice. > > + * > > + * Query the shared action by handle. > > + * > > + * This function allows retrieving action-specific data such as counters. > > + * Data is gathered by special action which may be present/referenced in > > + * more than one flow rule definition. > > + * > > + * \see RTE_FLOW_ACTION_TYPE_COUNT > > + * > > + * @param port_id > > + * Port identifier of Ethernet device. > > + * @param[in] action > > + * Handle for the shared action to query. > > + * @param[in, out] data > > + * Pointer to storage for the associated query data type. > > + * @param[out] error > > + * Perform verbose error reporting if not NULL. PMDs initialize this > > + * structure in case of error only. > > + * > > + * @return > > + * 0 on success, a negative errno value otherwise and rte_errno is set. > > + */ > > +__rte_experimental > > +int > > +rte_flow_shared_action_query(uint16_t port_id, > > + const struct rte_flow_shared_action *action, > > + void *data, > > + struct rte_flow_error *error); > > + > > #ifdef __cplusplus > > } > > #endif > > diff --git a/lib/librte_ethdev/rte_flow_driver.h > > b/lib/librte_ethdev/rte_flow_driver.h > > index 3ee871d3eb..72bfc3b7a7 100644 > > --- a/lib/librte_ethdev/rte_flow_driver.h > > +++ b/lib/librte_ethdev/rte_flow_driver.h > > @@ -108,6 +108,28 @@ struct rte_flow_ops { > > void **context, > > uint32_t nb_contexts, > > struct rte_flow_error *err); > > + /** See rte_flow_shared_action_create() */ > > + struct rte_flow_shared_action *(*shared_action_create) > > + (struct rte_eth_dev *dev, > > + const struct rte_flow_action *action, > > + struct rte_flow_error *error); > > + /** See rte_flow_shared_action_destroy() */ > > + int (*shared_action_destroy) > > + (struct rte_eth_dev *dev, > > + struct rte_flow_shared_action *shared_action, > > + struct rte_flow_error *error); > > + /** See rte_flow_shared_action_update() */ > > + int (*shared_action_update) > > + (struct rte_eth_dev *dev, > > + struct rte_flow_shared_action *shared_action, > > + const struct rte_flow_action *update, > > + struct rte_flow_error *error); > > + /** See rte_flow_shared_action_query() */ > > + int (*shared_action_query) > > + (struct rte_eth_dev *dev, > > + const struct rte_flow_shared_action *shared_action, > > + void *data, > > + struct rte_flow_error *error); > > }; > > > > /** > > -- > > 2.26.2 ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [dpdk-dev] [PATCH v3 01/10] ethdev: add flow shared action API 2020-10-03 22:06 ` [dpdk-dev] [PATCH v3 01/10] ethdev: add flow shared action API Andrey Vesnovaty 2020-10-04 11:10 ` Ori Kam @ 2020-10-04 17:00 ` Stephen Hemminger 2020-10-04 17:01 ` Stephen Hemminger 1 sibling, 1 reply; 106+ messages in thread From: Stephen Hemminger @ 2020-10-04 17:00 UTC (permalink / raw) To: Andrey Vesnovaty Cc: dev, jer, jerinjacobk, thomas, ferruh.yigit, bruce.richardson, orika, viacheslavo, andrey.vesnovaty, mdr, nhorman, ajit.khaparde, samik.gupta, Andrey Vesnovaty, Andrew Rybchenko, Ori Kam On Sun, 4 Oct 2020 01:06:10 +0300 Andrey Vesnovaty <andreyv@nvidia.com> wrote: > This commit introduces extension of DPDK flow action API enabling > sharing of single rte_flow_action in multiple flows. The API intended for > PMDs where multiple HW offloaded flows can reuse the same HW > essence/object representing flow action and modification of such an > essence/object effects all the rules using it. > > Motivation and example > === > Adding or removing one or more queues to RSS used by multiple flow rules > imposes per rule toll for current DPDK flow API; the scenario requires > for each flow sharing cloned RSS action: > - call `rte_flow_destroy()` > - call `rte_flow_create()` with modified RSS action > > API for sharing action and its in-place update benefits: > - reduce the overhead of multiple RSS flow rules reconfiguration > - optimize resource utilization by sharing action across of multiple > flows > > Change description > === > > Shared action > === > In order to represent flow action shared by multiple flows new action > type RTE_FLOW_ACTION_TYPE_SHARED is introduced (see `enum > rte_flow_action_type`). > Actually the introduced API decouples action from any specific flow and > enables sharing of single action by its handle across multiple flows. > > Shared action create/use/destroy > === > Shared action may be reused by some or none flow rules at any given > moment, i.e. shared action reside outside of the context of any flow. > Shared action represent HW resources/objects used for action offloading > implementation. > API for shared action create (see `rte_flow_shared_action_create()`): > - should allocate HW resources and make related initializations required > for shared action implementation. > - make necessary preparations to maintain shared access to > the action resources, configuration and state. > API for shared action destroy (see `rte_flow_shared_action_destroy()`) > should release HW resources and make related cleanups required for shared > action implementation. > > In order to share some flow action reuse the handle of type > `struct rte_flow_shared_action` returned by > rte_flow_shared_action_create() as a `conf` field of > `struct rte_flow_action` (see "example" section). > > If some shared action not used by any flow rule all resources allocated > by the shared action can be released by rte_flow_shared_action_destroy() > (see "example" section). The shared action handle passed as argument to > destroy API should not be used any further i.e. result of the usage is > undefined. > > Shared action re-configuration > === > Shared action behavior defined by its configuration can be updated via > rte_flow_shared_action_update() (see "example" section). The shared > action update operation modifies HW related resources/objects allocated > on the action creation. The number of operations performed by the update > operation should not be dependent on number of flows sharing the related > action. On return of shared action update API action behavior should be > according to updated configuration for all flows sharing the action. > > Shared action query > === > Provide separate API to query shared action sate (see > rte_flow_shared_action_update()). Taking a counter as an example: query > returns value aggregating all counter increments across all flow rules > sharing the counter. > > PMD support > === > The support of introduced API is pure PMD specific design and > responsibility for each action type (see struct rte_flow_ops). > > testpmd > === > In order to utilize introduced API testpmd cli may implement following > extension > create/update/destroy/query shared action accordingly > > flow shared_action (port) create {action_id (id)} (action) / end > flow shared_action (port) update (id) (action) / end > flow shared_action (port) destroy action_id (id) {action_id (id) [...]} > flow shared_action (port) query (id) > > testpmd example > === > > configure rss to queues 1 & 2 > > > flow shared_action 0 create action_id 100 rss queues 1 2 end / end > > create flow rule utilizing shared action > > > flow create 0 ingress \ > pattern eth dst is 0c:42:a1:15:fd:ac / ipv6 / tcp / end \ > actions shared 100 / end > > add 2 more queues > > > flow shared_action 0 modify 100 rss queues 1 2 3 4 end / end > > example > === > > struct rte_flow_action actions[2]; > struct rte_flow_action action; > /* skipped: initialize action */ > struct rte_flow_shared_action *handle = rte_flow_shared_action_create( > port_id, &action, &error); > actions[0].type = RTE_FLOW_ACTION_TYPE_SHARED; > actions[0].conf = handle; > actions[1].type = RTE_FLOW_ACTION_TYPE_END; > /* skipped: init attr0 & pattern0 args */ > struct rte_flow *flow0 = rte_flow_create(port_id, &attr0, pattern0, > actions, error); > /* create more rules reusing shared action */ > struct rte_flow *flow1 = rte_flow_create(port_id, &attr1, pattern1, > actions, error); > /* skipped: for flows 2 till N */ > struct rte_flow *flowN = rte_flow_create(port_id, &attrN, patternN, > actions, error); > /* update shared action */ > struct rte_flow_action updated_action; > /* > * skipped: initialize updated_action according to desired action > * configuration change > */ > rte_flow_shared_action_update(port_id, handle, &updated_action, error); > /* > * from now on all flows 1 till N will act according to configuration of > * updated_action > */ > /* skipped: destroy all flows 1 till N */ > rte_flow_shared_action_destroy(port_id, handle, error); > > Signed-off-by: Andrey Vesnovaty <andreyv@mellanox.com> > Signed-off-by: Andrey Vesnovaty <andreyv@nvidia.com> Before this can be accepted, you need to add a test. Ideally part of the test infrastructure, but a a minimum some way to demonstrate usage with testpmd. ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [dpdk-dev] [PATCH v3 01/10] ethdev: add flow shared action API 2020-10-04 17:00 ` Stephen Hemminger @ 2020-10-04 17:01 ` Stephen Hemminger 0 siblings, 0 replies; 106+ messages in thread From: Stephen Hemminger @ 2020-10-04 17:01 UTC (permalink / raw) To: Andrey Vesnovaty Cc: dev, jer, jerinjacobk, thomas, ferruh.yigit, bruce.richardson, orika, viacheslavo, andrey.vesnovaty, mdr, nhorman, ajit.khaparde, samik.gupta, Andrey Vesnovaty, Andrew Rybchenko, Ori Kam On Sun, 4 Oct 2020 10:00:00 -0700 Stephen Hemminger <stephen@networkplumber.org> wrote: > Before this can be accepted, you need to add a test. > Ideally part of the test infrastructure, but a a minimum some way to demonstrate > usage with testpmd. Nevermind, did not read far enough ahead in list to see the tests. ^ permalink raw reply [flat|nested] 106+ messages in thread
* [dpdk-dev] [PATCH v3 02/10] ethdev: add conf arg to shared action icreate API 2020-10-03 22:06 ` [dpdk-dev] [PATCH v3 00/10] RTE flow shared action Andrey Vesnovaty 2020-10-03 22:06 ` [dpdk-dev] [PATCH v3 01/10] ethdev: add flow shared action API Andrey Vesnovaty @ 2020-10-03 22:06 ` Andrey Vesnovaty 2020-10-04 11:11 ` Ori Kam 2020-10-03 22:06 ` [dpdk-dev] [PATCH v3 03/10] common/mlx5: modify advanced Rx object via DevX Andrey Vesnovaty ` (8 subsequent siblings) 10 siblings, 1 reply; 106+ messages in thread From: Andrey Vesnovaty @ 2020-10-03 22:06 UTC (permalink / raw) To: dev Cc: jer, jerinjacobk, thomas, ferruh.yigit, stephen, bruce.richardson, orika, viacheslavo, andrey.vesnovaty, mdr, nhorman, ajit.khaparde, samik.gupta, Ori Kam, Andrew Rybchenko Add configuration argument to shared action create interface. Currently there is only ingress & egress fields but more fields can be added later. Shared action configuration & implementation are PMD specific. Signed-off-by: Andrey Vesnovaty <andreyv@nvidia.com> --- lib/librte_ethdev/rte_flow.c | 4 +++- lib/librte_ethdev/rte_flow.h | 17 +++++++++++++++-- lib/librte_ethdev/rte_flow_driver.h | 5 +++-- 3 files changed, 21 insertions(+), 5 deletions(-) diff --git a/lib/librte_ethdev/rte_flow.c b/lib/librte_ethdev/rte_flow.c index ba3f01f7c7..9afa8905df 100644 --- a/lib/librte_ethdev/rte_flow.c +++ b/lib/librte_ethdev/rte_flow.c @@ -1255,6 +1255,7 @@ rte_flow_get_aged_flows(uint16_t port_id, void **contexts, struct rte_flow_shared_action * rte_flow_shared_action_create(uint16_t port_id, + const struct rte_flow_shared_action_conf *conf, const struct rte_flow_action *action, struct rte_flow_error *error) { @@ -1265,7 +1266,8 @@ rte_flow_shared_action_create(uint16_t port_id, if (unlikely(!ops)) return NULL; if (likely(!!ops->shared_action_create)) { - shared_action = ops->shared_action_create(dev, action, error); + shared_action = ops->shared_action_create(dev, conf, action, + error); if (shared_action == NULL) flow_err(port_id, -rte_errno, error); return shared_action; diff --git a/lib/librte_ethdev/rte_flow.h b/lib/librte_ethdev/rte_flow.h index 02391316cb..8a2db4f6da 100644 --- a/lib/librte_ethdev/rte_flow.h +++ b/lib/librte_ethdev/rte_flow.h @@ -3380,6 +3380,16 @@ int rte_flow_get_aged_flows(uint16_t port_id, void **contexts, uint32_t nb_contexts, struct rte_flow_error *error); +/** + * Specify shared action configuration + */ +struct rte_flow_shared_action_conf { + uint32_t ingress:1; + /**< Action valid for rules applied to ingress traffic. */ + uint32_t egress:1; + /**< Action valid for rules applied to egress traffic. */ +}; + /** * @warning * @b EXPERIMENTAL: this API may change without prior notice. @@ -3388,6 +3398,8 @@ rte_flow_get_aged_flows(uint16_t port_id, void **contexts, * * @param[in] port_id * The port identifier of the Ethernet device. + * @param[in] conf + * Shared action configuration. * @param[in] action * Action configuration for shared action creation. * @param[out] error @@ -3404,6 +3416,7 @@ rte_flow_get_aged_flows(uint16_t port_id, void **contexts, __rte_experimental struct rte_flow_shared_action * rte_flow_shared_action_create(uint16_t port_id, + const struct rte_flow_shared_action_conf *conf, const struct rte_flow_action *action, struct rte_flow_error *error); @@ -3432,8 +3445,8 @@ rte_flow_shared_action_create(uint16_t port_id, __rte_experimental int rte_flow_shared_action_destroy(uint16_t port_id, - struct rte_flow_shared_action *action, - struct rte_flow_error *error); + struct rte_flow_shared_action *action, + struct rte_flow_error *error); /** * @warning diff --git a/lib/librte_ethdev/rte_flow_driver.h b/lib/librte_ethdev/rte_flow_driver.h index 72bfc3b7a7..adaace47ea 100644 --- a/lib/librte_ethdev/rte_flow_driver.h +++ b/lib/librte_ethdev/rte_flow_driver.h @@ -111,8 +111,9 @@ struct rte_flow_ops { /** See rte_flow_shared_action_create() */ struct rte_flow_shared_action *(*shared_action_create) (struct rte_eth_dev *dev, - const struct rte_flow_action *action, - struct rte_flow_error *error); + const struct rte_flow_shared_action_conf *conf, + const struct rte_flow_action *action, + struct rte_flow_error *error); /** See rte_flow_shared_action_destroy() */ int (*shared_action_destroy) (struct rte_eth_dev *dev, -- 2.26.2 ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [dpdk-dev] [PATCH v3 02/10] ethdev: add conf arg to shared action icreate API 2020-10-03 22:06 ` [dpdk-dev] [PATCH v3 02/10] ethdev: add conf arg to shared action icreate API Andrey Vesnovaty @ 2020-10-04 11:11 ` Ori Kam 2020-10-06 10:28 ` Andrey Vesnovaty 0 siblings, 1 reply; 106+ messages in thread From: Ori Kam @ 2020-10-04 11:11 UTC (permalink / raw) To: Andrey Vesnovaty, dev Cc: jer, jerinjacobk, NBU-Contact-Thomas Monjalon, ferruh.yigit, stephen, bruce.richardson, Slava Ovsiienko, andrey.vesnovaty, mdr, nhorman, ajit.khaparde, samik.gupta, Ori Kam, Andrew Rybchenko Hi Andrey Why is this patch not part of the previous one? Best, Ori > -----Original Message----- > From: Andrey Vesnovaty <andreyv@nvidia.com> > Sent: Sunday, October 4, 2020 1:06 AM > To: dev@dpdk.org > Cc: jer@marvell.com; jerinjacobk@gmail.com; NBU-Contact-Thomas Monjalon > <thomas@monjalon.net>; ferruh.yigit@intel.com; > stephen@networkplumber.org; bruce.richardson@intel.com; Ori Kam > <orika@nvidia.com>; Slava Ovsiienko <viacheslavo@nvidia.com>; > andrey.vesnovaty@gmail.com; mdr@ashroe.eu; nhorman@tuxdriver.com; > ajit.khaparde@broadcom.com; samik.gupta@broadcom.com; Ori Kam > <orika@mellanox.com>; Andrew Rybchenko <arybchenko@solarflare.com> > Subject: [PATCH v3 02/10] ethdev: add conf arg to shared action icreate API > > Add configuration argument to shared action create interface. > Currently there is only ingress & egress fields but more fields can be > added later. Shared action configuration & implementation are PMD > specific. > > Signed-off-by: Andrey Vesnovaty <andreyv@nvidia.com> > --- > lib/librte_ethdev/rte_flow.c | 4 +++- > lib/librte_ethdev/rte_flow.h | 17 +++++++++++++++-- > lib/librte_ethdev/rte_flow_driver.h | 5 +++-- > 3 files changed, 21 insertions(+), 5 deletions(-) > > diff --git a/lib/librte_ethdev/rte_flow.c b/lib/librte_ethdev/rte_flow.c > index ba3f01f7c7..9afa8905df 100644 > --- a/lib/librte_ethdev/rte_flow.c > +++ b/lib/librte_ethdev/rte_flow.c > @@ -1255,6 +1255,7 @@ rte_flow_get_aged_flows(uint16_t port_id, void > **contexts, > > struct rte_flow_shared_action * > rte_flow_shared_action_create(uint16_t port_id, > + const struct rte_flow_shared_action_conf *conf, > const struct rte_flow_action *action, > struct rte_flow_error *error) > { > @@ -1265,7 +1266,8 @@ rte_flow_shared_action_create(uint16_t port_id, > if (unlikely(!ops)) > return NULL; > if (likely(!!ops->shared_action_create)) { > - shared_action = ops->shared_action_create(dev, action, error); > + shared_action = ops->shared_action_create(dev, conf, action, > + error); > if (shared_action == NULL) > flow_err(port_id, -rte_errno, error); > return shared_action; > diff --git a/lib/librte_ethdev/rte_flow.h b/lib/librte_ethdev/rte_flow.h > index 02391316cb..8a2db4f6da 100644 > --- a/lib/librte_ethdev/rte_flow.h > +++ b/lib/librte_ethdev/rte_flow.h > @@ -3380,6 +3380,16 @@ int > rte_flow_get_aged_flows(uint16_t port_id, void **contexts, > uint32_t nb_contexts, struct rte_flow_error *error); > > +/** > + * Specify shared action configuration > + */ > +struct rte_flow_shared_action_conf { > + uint32_t ingress:1; > + /**< Action valid for rules applied to ingress traffic. */ > + uint32_t egress:1; > + /**< Action valid for rules applied to egress traffic. */ > +}; > + > /** > * @warning > * @b EXPERIMENTAL: this API may change without prior notice. > @@ -3388,6 +3398,8 @@ rte_flow_get_aged_flows(uint16_t port_id, void > **contexts, > * > * @param[in] port_id > * The port identifier of the Ethernet device. > + * @param[in] conf > + * Shared action configuration. > * @param[in] action > * Action configuration for shared action creation. > * @param[out] error > @@ -3404,6 +3416,7 @@ rte_flow_get_aged_flows(uint16_t port_id, void > **contexts, > __rte_experimental > struct rte_flow_shared_action * > rte_flow_shared_action_create(uint16_t port_id, > + const struct rte_flow_shared_action_conf *conf, > const struct rte_flow_action *action, > struct rte_flow_error *error); > > @@ -3432,8 +3445,8 @@ rte_flow_shared_action_create(uint16_t port_id, > __rte_experimental > int > rte_flow_shared_action_destroy(uint16_t port_id, > - struct rte_flow_shared_action *action, > - struct rte_flow_error *error); > + struct rte_flow_shared_action *action, > + struct rte_flow_error *error); > > /** > * @warning > diff --git a/lib/librte_ethdev/rte_flow_driver.h > b/lib/librte_ethdev/rte_flow_driver.h > index 72bfc3b7a7..adaace47ea 100644 > --- a/lib/librte_ethdev/rte_flow_driver.h > +++ b/lib/librte_ethdev/rte_flow_driver.h > @@ -111,8 +111,9 @@ struct rte_flow_ops { > /** See rte_flow_shared_action_create() */ > struct rte_flow_shared_action *(*shared_action_create) > (struct rte_eth_dev *dev, > - const struct rte_flow_action *action, > - struct rte_flow_error *error); > + const struct rte_flow_shared_action_conf *conf, > + const struct rte_flow_action *action, > + struct rte_flow_error *error); > /** See rte_flow_shared_action_destroy() */ > int (*shared_action_destroy) > (struct rte_eth_dev *dev, > -- > 2.26.2 ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [dpdk-dev] [PATCH v3 02/10] ethdev: add conf arg to shared action icreate API 2020-10-04 11:11 ` Ori Kam @ 2020-10-06 10:28 ` Andrey Vesnovaty 0 siblings, 0 replies; 106+ messages in thread From: Andrey Vesnovaty @ 2020-10-06 10:28 UTC (permalink / raw) To: Ori Kam, dev Cc: jer, jerinjacobk, NBU-Contact-Thomas Monjalon, ferruh.yigit, stephen, bruce.richardson, Slava Ovsiienko, andrey.vesnovaty, mdr, nhorman, ajit.khaparde, samik.gupta, Ori Kam, Andrew Rybchenko PSB > -----Original Message----- > From: Ori Kam <orika@nvidia.com> > Sent: Sunday, October 4, 2020 2:11 PM > To: Andrey Vesnovaty <andreyv@nvidia.com>; dev@dpdk.org > Cc: jer@marvell.com; jerinjacobk@gmail.com; NBU-Contact-Thomas Monjalon > <thomas@monjalon.net>; ferruh.yigit@intel.com; > stephen@networkplumber.org; bruce.richardson@intel.com; Slava Ovsiienko > <viacheslavo@nvidia.com>; andrey.vesnovaty@gmail.com; mdr@ashroe.eu; > nhorman@tuxdriver.com; ajit.khaparde@broadcom.com; > samik.gupta@broadcom.com; Ori Kam <orika@mellanox.com>; Andrew > Rybchenko <arybchenko@solarflare.com> > Subject: RE: [PATCH v3 02/10] ethdev: add conf arg to shared action icreate API > > Hi Andrey > > Why is this patch not part of the previous one? Got it. Will be squashed. > > Best, > Ori > > -----Original Message----- > > From: Andrey Vesnovaty <andreyv@nvidia.com> > > Sent: Sunday, October 4, 2020 1:06 AM > > To: dev@dpdk.org > > Cc: jer@marvell.com; jerinjacobk@gmail.com; NBU-Contact-Thomas > Monjalon > > <thomas@monjalon.net>; ferruh.yigit@intel.com; > > stephen@networkplumber.org; bruce.richardson@intel.com; Ori Kam > > <orika@nvidia.com>; Slava Ovsiienko <viacheslavo@nvidia.com>; > > andrey.vesnovaty@gmail.com; mdr@ashroe.eu; nhorman@tuxdriver.com; > > ajit.khaparde@broadcom.com; samik.gupta@broadcom.com; Ori Kam > > <orika@mellanox.com>; Andrew Rybchenko <arybchenko@solarflare.com> > > Subject: [PATCH v3 02/10] ethdev: add conf arg to shared action icreate API > > > > Add configuration argument to shared action create interface. > > Currently there is only ingress & egress fields but more fields can be > > added later. Shared action configuration & implementation are PMD > > specific. > > > > Signed-off-by: Andrey Vesnovaty <andreyv@nvidia.com> > > --- > > lib/librte_ethdev/rte_flow.c | 4 +++- > > lib/librte_ethdev/rte_flow.h | 17 +++++++++++++++-- > > lib/librte_ethdev/rte_flow_driver.h | 5 +++-- > > 3 files changed, 21 insertions(+), 5 deletions(-) > > > > diff --git a/lib/librte_ethdev/rte_flow.c b/lib/librte_ethdev/rte_flow.c > > index ba3f01f7c7..9afa8905df 100644 > > --- a/lib/librte_ethdev/rte_flow.c > > +++ b/lib/librte_ethdev/rte_flow.c > > @@ -1255,6 +1255,7 @@ rte_flow_get_aged_flows(uint16_t port_id, void > > **contexts, > > > > struct rte_flow_shared_action * > > rte_flow_shared_action_create(uint16_t port_id, > > + const struct rte_flow_shared_action_conf *conf, > > const struct rte_flow_action *action, > > struct rte_flow_error *error) > > { > > @@ -1265,7 +1266,8 @@ rte_flow_shared_action_create(uint16_t port_id, > > if (unlikely(!ops)) > > return NULL; > > if (likely(!!ops->shared_action_create)) { > > - shared_action = ops->shared_action_create(dev, action, error); > > + shared_action = ops->shared_action_create(dev, conf, action, > > + error); > > if (shared_action == NULL) > > flow_err(port_id, -rte_errno, error); > > return shared_action; > > diff --git a/lib/librte_ethdev/rte_flow.h b/lib/librte_ethdev/rte_flow.h > > index 02391316cb..8a2db4f6da 100644 > > --- a/lib/librte_ethdev/rte_flow.h > > +++ b/lib/librte_ethdev/rte_flow.h > > @@ -3380,6 +3380,16 @@ int > > rte_flow_get_aged_flows(uint16_t port_id, void **contexts, > > uint32_t nb_contexts, struct rte_flow_error *error); > > > > +/** > > + * Specify shared action configuration > > + */ > > +struct rte_flow_shared_action_conf { > > + uint32_t ingress:1; > > + /**< Action valid for rules applied to ingress traffic. */ > > + uint32_t egress:1; > > + /**< Action valid for rules applied to egress traffic. */ > > +}; > > + > > /** > > * @warning > > * @b EXPERIMENTAL: this API may change without prior notice. > > @@ -3388,6 +3398,8 @@ rte_flow_get_aged_flows(uint16_t port_id, void > > **contexts, > > * > > * @param[in] port_id > > * The port identifier of the Ethernet device. > > + * @param[in] conf > > + * Shared action configuration. > > * @param[in] action > > * Action configuration for shared action creation. > > * @param[out] error > > @@ -3404,6 +3416,7 @@ rte_flow_get_aged_flows(uint16_t port_id, void > > **contexts, > > __rte_experimental > > struct rte_flow_shared_action * > > rte_flow_shared_action_create(uint16_t port_id, > > + const struct rte_flow_shared_action_conf *conf, > > const struct rte_flow_action *action, > > struct rte_flow_error *error); > > > > @@ -3432,8 +3445,8 @@ rte_flow_shared_action_create(uint16_t port_id, > > __rte_experimental > > int > > rte_flow_shared_action_destroy(uint16_t port_id, > > - struct rte_flow_shared_action *action, > > - struct rte_flow_error *error); > > + struct rte_flow_shared_action *action, > > + struct rte_flow_error *error); > > > > /** > > * @warning > > diff --git a/lib/librte_ethdev/rte_flow_driver.h > > b/lib/librte_ethdev/rte_flow_driver.h > > index 72bfc3b7a7..adaace47ea 100644 > > --- a/lib/librte_ethdev/rte_flow_driver.h > > +++ b/lib/librte_ethdev/rte_flow_driver.h > > @@ -111,8 +111,9 @@ struct rte_flow_ops { > > /** See rte_flow_shared_action_create() */ > > struct rte_flow_shared_action *(*shared_action_create) > > (struct rte_eth_dev *dev, > > - const struct rte_flow_action *action, > > - struct rte_flow_error *error); > > + const struct rte_flow_shared_action_conf *conf, > > + const struct rte_flow_action *action, > > + struct rte_flow_error *error); > > /** See rte_flow_shared_action_destroy() */ > > int (*shared_action_destroy) > > (struct rte_eth_dev *dev, > > -- > > 2.26.2 ^ permalink raw reply [flat|nested] 106+ messages in thread
* [dpdk-dev] [PATCH v3 03/10] common/mlx5: modify advanced Rx object via DevX 2020-10-03 22:06 ` [dpdk-dev] [PATCH v3 00/10] RTE flow shared action Andrey Vesnovaty 2020-10-03 22:06 ` [dpdk-dev] [PATCH v3 01/10] ethdev: add flow shared action API Andrey Vesnovaty 2020-10-03 22:06 ` [dpdk-dev] [PATCH v3 02/10] ethdev: add conf arg to shared action icreate API Andrey Vesnovaty @ 2020-10-03 22:06 ` Andrey Vesnovaty 2020-10-03 22:06 ` [dpdk-dev] [PATCH v3 04/10] net/mlx5: modify hash Rx queue objects Andrey Vesnovaty ` (7 subsequent siblings) 10 siblings, 0 replies; 106+ messages in thread From: Andrey Vesnovaty @ 2020-10-03 22:06 UTC (permalink / raw) To: dev Cc: jer, jerinjacobk, thomas, ferruh.yigit, stephen, bruce.richardson, orika, viacheslavo, andrey.vesnovaty, mdr, nhorman, ajit.khaparde, samik.gupta, Andrey Vesnovaty, Matan Azrad, Shahaf Shuler, Viacheslav Ovsiienko From: Andrey Vesnovaty <andreyv@mellanox.com> Implement mlx5_devx_cmd_modify_tir() to modify TIR object using DevX API. Add related structs in mlx5_prm.h. Signed-off-by: Andrey Vesnovaty <andreyv@mellanox.com> --- drivers/common/mlx5/mlx5_devx_cmds.c | 84 +++++++++++++++++++ drivers/common/mlx5/mlx5_devx_cmds.h | 10 +++ drivers/common/mlx5/mlx5_prm.h | 29 +++++++ .../common/mlx5/rte_common_mlx5_version.map | 1 + 4 files changed, 124 insertions(+) diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c index 7c81ae15a9..2b109c4f65 100644 --- a/drivers/common/mlx5/mlx5_devx_cmds.c +++ b/drivers/common/mlx5/mlx5_devx_cmds.c @@ -1080,6 +1080,90 @@ mlx5_devx_cmd_create_tir(void *ctx, return tir; } +/** + * Modify TIR using DevX API. + * + * @param[in] tir + * Pointer to TIR DevX object structure. + * @param [in] modify_tir_attr + * Pointer to TIR modification attributes structure. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +int +mlx5_devx_cmd_modify_tir(struct mlx5_devx_obj *tir, + struct mlx5_devx_modify_tir_attr *modify_tir_attr) +{ + struct mlx5_devx_tir_attr *tir_attr = &modify_tir_attr->tir; + uint32_t in[MLX5_ST_SZ_DW(modify_tir_in)] = {0}; + uint32_t out[MLX5_ST_SZ_DW(modify_tir_out)] = {0}; + void *tir_ctx; + int ret; + + MLX5_SET(modify_tir_in, in, opcode, MLX5_CMD_OP_MODIFY_TIR); + MLX5_SET(modify_tir_in, in, tirn, modify_tir_attr->tirn); + MLX5_SET64(modify_tir_in, in, modify_bitmask, + modify_tir_attr->modify_bitmask); + + tir_ctx = MLX5_ADDR_OF(modify_rq_in, in, ctx); + if (modify_tir_attr->modify_bitmask & + MLX5_MODIFY_TIR_IN_MODIFY_BITMASK_LRO) { + MLX5_SET(tirc, tir_ctx, lro_timeout_period_usecs, + tir_attr->lro_timeout_period_usecs); + MLX5_SET(tirc, tir_ctx, lro_enable_mask, + tir_attr->lro_enable_mask); + MLX5_SET(tirc, tir_ctx, lro_max_msg_sz, + tir_attr->lro_max_msg_sz); + } + if (modify_tir_attr->modify_bitmask & + MLX5_MODIFY_TIR_IN_MODIFY_BITMASK_INDIRECT_TABLE) + MLX5_SET(tirc, tir_ctx, indirect_table, + tir_attr->indirect_table); + if (modify_tir_attr->modify_bitmask & + MLX5_MODIFY_TIR_IN_MODIFY_BITMASK_HASH) { + int i; + void *outer, *inner; + MLX5_SET(tirc, tir_ctx, rx_hash_symmetric, + tir_attr->rx_hash_symmetric); + MLX5_SET(tirc, tir_ctx, rx_hash_fn, tir_attr->rx_hash_fn); + for (i = 0; i < 10; i++) { + MLX5_SET(tirc, tir_ctx, rx_hash_toeplitz_key[i], + tir_attr->rx_hash_toeplitz_key[i]); + } + outer = MLX5_ADDR_OF(tirc, tir_ctx, + rx_hash_field_selector_outer); + MLX5_SET(rx_hash_field_select, outer, l3_prot_type, + tir_attr->rx_hash_field_selector_outer.l3_prot_type); + MLX5_SET(rx_hash_field_select, outer, l4_prot_type, + tir_attr->rx_hash_field_selector_outer.l4_prot_type); + MLX5_SET + (rx_hash_field_select, outer, selected_fields, + tir_attr->rx_hash_field_selector_outer.selected_fields); + inner = MLX5_ADDR_OF(tirc, tir_ctx, + rx_hash_field_selector_inner); + MLX5_SET(rx_hash_field_select, inner, l3_prot_type, + tir_attr->rx_hash_field_selector_inner.l3_prot_type); + MLX5_SET(rx_hash_field_select, inner, l4_prot_type, + tir_attr->rx_hash_field_selector_inner.l4_prot_type); + MLX5_SET + (rx_hash_field_select, inner, selected_fields, + tir_attr->rx_hash_field_selector_inner.selected_fields); + } + if (modify_tir_attr->modify_bitmask & + MLX5_MODIFY_TIR_IN_MODIFY_BITMASK_SELF_LB_EN) { + MLX5_SET(tirc, tir_ctx, self_lb_block, tir_attr->self_lb_block); + } + ret = mlx5_glue->devx_obj_modify(tir->obj, in, sizeof(in), + out, sizeof(out)); + if (ret) { + DRV_LOG(ERR, "Failed to modify TIR using DevX"); + rte_errno = errno; + return -errno; + } + return ret; +} + /** * Create RQT using DevX API. * diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h index 1c84cea851..ba6cb6ed51 100644 --- a/drivers/common/mlx5/mlx5_devx_cmds.h +++ b/drivers/common/mlx5/mlx5_devx_cmds.h @@ -190,6 +190,13 @@ struct mlx5_devx_tir_attr { struct mlx5_rx_hash_field_select rx_hash_field_selector_inner; }; +/* TIR attributes structure, used by TIR modify */ +struct mlx5_devx_modify_tir_attr { + uint32_t tirn:24; + uint64_t modify_bitmask; + struct mlx5_devx_tir_attr tir; +}; + /* RQT attributes structure, used by RQT operations. */ struct mlx5_devx_rqt_attr { uint8_t rq_type; @@ -434,6 +441,9 @@ __rte_internal int mlx5_devx_cmd_modify_rqt(struct mlx5_devx_obj *rqt, struct mlx5_devx_rqt_attr *rqt_attr); __rte_internal +int mlx5_devx_cmd_modify_tir(struct mlx5_devx_obj *tir, + struct mlx5_devx_modify_tir_attr *tir_attr); +__rte_internal int mlx5_devx_cmd_query_parse_samples(struct mlx5_devx_obj *flex_obj, uint32_t ids[], uint32_t num); diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h index 563e7c86c9..2068f6cc77 100644 --- a/drivers/common/mlx5/mlx5_prm.h +++ b/drivers/common/mlx5/mlx5_prm.h @@ -830,6 +830,7 @@ enum { MLX5_CMD_OP_ACCESS_REGISTER = 0x805, MLX5_CMD_OP_ALLOC_TRANSPORT_DOMAIN = 0x816, MLX5_CMD_OP_CREATE_TIR = 0x900, + MLX5_CMD_OP_MODIFY_TIR = 0x901, MLX5_CMD_OP_CREATE_SQ = 0X904, MLX5_CMD_OP_MODIFY_SQ = 0X905, MLX5_CMD_OP_CREATE_RQ = 0x908, @@ -1858,6 +1859,34 @@ struct mlx5_ifc_create_tir_in_bits { struct mlx5_ifc_tirc_bits ctx; }; +enum { + MLX5_MODIFY_TIR_IN_MODIFY_BITMASK_LRO = 1ULL << 0, + MLX5_MODIFY_TIR_IN_MODIFY_BITMASK_INDIRECT_TABLE = 1ULL << 1, + MLX5_MODIFY_TIR_IN_MODIFY_BITMASK_HASH = 1ULL << 2, + /* bit 3 - tunneled_offload_en modify not supported */ + MLX5_MODIFY_TIR_IN_MODIFY_BITMASK_SELF_LB_EN = 1ULL << 4, +}; + +struct mlx5_ifc_modify_tir_out_bits { + u8 status[0x8]; + u8 reserved_at_8[0x18]; + u8 syndrome[0x20]; + u8 reserved_at_40[0x40]; +}; + +struct mlx5_ifc_modify_tir_in_bits { + u8 opcode[0x10]; + u8 uid[0x10]; + u8 reserved_at_20[0x10]; + u8 op_mod[0x10]; + u8 reserved_at_40[0x8]; + u8 tirn[0x18]; + u8 reserved_at_60[0x20]; + u8 modify_bitmask[0x40]; + u8 reserved_at_c0[0x40]; + struct mlx5_ifc_tirc_bits ctx; +}; + enum { MLX5_INLINE_Q_TYPE_RQ = 0x0, MLX5_INLINE_Q_TYPE_VIRTQ = 0x1, diff --git a/drivers/common/mlx5/rte_common_mlx5_version.map b/drivers/common/mlx5/rte_common_mlx5_version.map index c4d57c08a7..884001ca7d 100644 --- a/drivers/common/mlx5/rte_common_mlx5_version.map +++ b/drivers/common/mlx5/rte_common_mlx5_version.map @@ -30,6 +30,7 @@ INTERNAL { mlx5_devx_cmd_modify_rq; mlx5_devx_cmd_modify_rqt; mlx5_devx_cmd_modify_sq; + mlx5_devx_cmd_modify_tir; mlx5_devx_cmd_modify_virtq; mlx5_devx_cmd_qp_query_tis_td; mlx5_devx_cmd_query_hca_attr; -- 2.26.2 ^ permalink raw reply [flat|nested] 106+ messages in thread
* [dpdk-dev] [PATCH v3 04/10] net/mlx5: modify hash Rx queue objects 2020-10-03 22:06 ` [dpdk-dev] [PATCH v3 00/10] RTE flow shared action Andrey Vesnovaty ` (2 preceding siblings ...) 2020-10-03 22:06 ` [dpdk-dev] [PATCH v3 03/10] common/mlx5: modify advanced Rx object via DevX Andrey Vesnovaty @ 2020-10-03 22:06 ` Andrey Vesnovaty 2020-10-03 22:06 ` [dpdk-dev] [PATCH v3 05/10] net/mlx5: shared action PMD Andrey Vesnovaty ` (6 subsequent siblings) 10 siblings, 0 replies; 106+ messages in thread From: Andrey Vesnovaty @ 2020-10-03 22:06 UTC (permalink / raw) To: dev Cc: jer, jerinjacobk, thomas, ferruh.yigit, stephen, bruce.richardson, orika, viacheslavo, andrey.vesnovaty, mdr, nhorman, ajit.khaparde, samik.gupta, Andrey Vesnovaty, Matan Azrad, Shahaf Shuler, Viacheslav Ovsiienko From: Andrey Vesnovaty <andreyv@mellanox.com> Implement mlx5_hrxq_modify() to modify hash RX queue object. This commit relays on capability to modify TIR object via DevX. Signed-off-by: Andrey Vesnovaty <andreyv@mellanox.com> --- drivers/net/mlx5/mlx5.h | 4 + drivers/net/mlx5/mlx5_devx.c | 178 +++++++++++++++++++++++++++-------- drivers/net/mlx5/mlx5_rxq.c | 103 ++++++++++++++++++++ drivers/net/mlx5/mlx5_rxtx.h | 5 +- 4 files changed, 251 insertions(+), 39 deletions(-) diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 865e72d318..210004bc81 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -747,6 +747,10 @@ struct mlx5_obj_ops { void (*ind_table_destroy)(struct mlx5_ind_table_obj *ind_tbl); int (*hrxq_new)(struct rte_eth_dev *dev, struct mlx5_hrxq *hrxq, int tunnel __rte_unused); + int (*hrxq_modify)(struct rte_eth_dev *dev, struct mlx5_hrxq *hrxq, + const uint8_t *rss_key, + uint64_t hash_fields, + const struct mlx5_ind_table_obj *ind_tbl); void (*hrxq_destroy)(struct mlx5_hrxq *hrxq); int (*drop_action_create)(struct rte_eth_dev *dev); void (*drop_action_destroy)(struct rte_eth_dev *dev); diff --git a/drivers/net/mlx5/mlx5_devx.c b/drivers/net/mlx5/mlx5_devx.c index 3e81fcc252..676f7c6fb3 100644 --- a/drivers/net/mlx5/mlx5_devx.c +++ b/drivers/net/mlx5/mlx5_devx.c @@ -675,33 +675,39 @@ mlx5_devx_ind_table_destroy(struct mlx5_ind_table_obj *ind_tbl) } /** - * Create an Rx Hash queue. + * Set TIR attribute struct with relevant input values. * - * @param dev + * @param[in] dev * Pointer to Ethernet device. - * @param hrxq - * Pointer to Rx Hash queue. - * @param tunnel + * @param[in] rss_key + * RSS key for the Rx hash queue. + * @param[in] hash_fields + * Verbs protocol hash field to make the RSS on. + * @param[in] ind_tbl + * Indirection table for TIR. + * @param[in] queues + * Queues entering in hash queue. In case of empty hash_fields only the + * first queue index will be taken for the indirection table. + * @param[in] queues_n + * Number of queues. + * @param[in] tunnel * Tunnel type. + * @param[out] tir_attr + * Parameters structure for TIR creation/modification. * * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. + * The Verbs/DevX object initialised index, 0 otherwise and rte_errno is set. */ -static int -mlx5_devx_hrxq_new(struct rte_eth_dev *dev, struct mlx5_hrxq *hrxq, - int tunnel __rte_unused) +static void +mlx5_devx_tir_attr_set(struct rte_eth_dev *dev, const uint8_t *rss_key, + uint64_t hash_fields, + const struct mlx5_ind_table_obj *ind_tbl, + int tunnel, enum mlx5_rxq_obj_type rxq_obj_type, + struct mlx5_devx_tir_attr *tir_attr) { struct mlx5_priv *priv = dev->data->dev_private; - struct mlx5_ind_table_obj *ind_tbl = hrxq->ind_table; - struct mlx5_rxq_data *rxq_data = (*priv->rxqs)[ind_tbl->queues[0]]; - struct mlx5_rxq_ctrl *rxq_ctrl = - container_of(rxq_data, struct mlx5_rxq_ctrl, rxq); - struct mlx5_devx_tir_attr tir_attr; - const uint8_t *rss_key = hrxq->rss_key; - uint64_t hash_fields = hrxq->hash_fields; bool lro = true; uint32_t i; - int err; /* Enable TIR LRO only if all the queues were configured for. */ for (i = 0; i < ind_tbl->queues_n; ++i) { @@ -710,26 +716,24 @@ mlx5_devx_hrxq_new(struct rte_eth_dev *dev, struct mlx5_hrxq *hrxq, break; } } - memset(&tir_attr, 0, sizeof(tir_attr)); - tir_attr.disp_type = MLX5_TIRC_DISP_TYPE_INDIRECT; - tir_attr.rx_hash_fn = MLX5_RX_HASH_FN_TOEPLITZ; - tir_attr.tunneled_offload_en = !!tunnel; + memset(tir_attr, 0, sizeof(*tir_attr)); + tir_attr->disp_type = MLX5_TIRC_DISP_TYPE_INDIRECT; + tir_attr->rx_hash_fn = MLX5_RX_HASH_FN_TOEPLITZ; + tir_attr->tunneled_offload_en = !!tunnel; /* If needed, translate hash_fields bitmap to PRM format. */ if (hash_fields) { - struct mlx5_rx_hash_field_select *rx_hash_field_select = NULL; + struct mlx5_rx_hash_field_select *rx_hash_field_select = #ifdef HAVE_IBV_DEVICE_TUNNEL_SUPPORT - rx_hash_field_select = hash_fields & IBV_RX_HASH_INNER ? - &tir_attr.rx_hash_field_selector_inner : - &tir_attr.rx_hash_field_selector_outer; -#else - rx_hash_field_select = &tir_attr.rx_hash_field_selector_outer; + hash_fields & IBV_RX_HASH_INNER ? + &tir_attr->rx_hash_field_selector_inner : #endif + &tir_attr->rx_hash_field_selector_outer; /* 1 bit: 0: IPv4, 1: IPv6. */ rx_hash_field_select->l3_prot_type = !!(hash_fields & MLX5_IPV6_IBV_RX_HASH); /* 1 bit: 0: TCP, 1: UDP. */ rx_hash_field_select->l4_prot_type = - !!(hash_fields & MLX5_UDP_IBV_RX_HASH); + !!(hash_fields & MLX5_UDP_IBV_RX_HASH); /* Bitmask which sets which fields to use in RX Hash. */ rx_hash_field_select->selected_fields = ((!!(hash_fields & MLX5_L3_SRC_IBV_RX_HASH)) << @@ -741,20 +745,53 @@ mlx5_devx_hrxq_new(struct rte_eth_dev *dev, struct mlx5_hrxq *hrxq, (!!(hash_fields & MLX5_L4_DST_IBV_RX_HASH)) << MLX5_RX_HASH_FIELD_SELECT_SELECTED_FIELDS_L4_DPORT; } - if (rxq_ctrl->type == MLX5_RXQ_TYPE_HAIRPIN) - tir_attr.transport_domain = priv->sh->td->id; + if (rxq_obj_type == MLX5_RXQ_OBJ_TYPE_DEVX_HAIRPIN) + tir_attr->transport_domain = priv->sh->td->id; else - tir_attr.transport_domain = priv->sh->tdn; - memcpy(tir_attr.rx_hash_toeplitz_key, rss_key, MLX5_RSS_HASH_KEY_LEN); - tir_attr.indirect_table = ind_tbl->rqt->id; + tir_attr->transport_domain = priv->sh->tdn; + memcpy(tir_attr->rx_hash_toeplitz_key, rss_key, MLX5_RSS_HASH_KEY_LEN); + tir_attr->indirect_table = ind_tbl->rqt->id; if (dev->data->dev_conf.lpbk_mode) - tir_attr.self_lb_block = MLX5_TIRC_SELF_LB_BLOCK_BLOCK_UNICAST; + tir_attr->self_lb_block = + MLX5_TIRC_SELF_LB_BLOCK_BLOCK_UNICAST; if (lro) { - tir_attr.lro_timeout_period_usecs = priv->config.lro.timeout; - tir_attr.lro_max_msg_sz = priv->max_lro_msg_size; - tir_attr.lro_enable_mask = MLX5_TIRC_LRO_ENABLE_MASK_IPV4_LRO | - MLX5_TIRC_LRO_ENABLE_MASK_IPV6_LRO; + tir_attr->lro_timeout_period_usecs = priv->config.lro.timeout; + tir_attr->lro_max_msg_sz = priv->max_lro_msg_size; + tir_attr->lro_enable_mask = + MLX5_TIRC_LRO_ENABLE_MASK_IPV4_LRO | + MLX5_TIRC_LRO_ENABLE_MASK_IPV6_LRO; } +} + +/** + * Create an Rx Hash queue. + * + * @param dev + * Pointer to Ethernet device. + * @param hrxq + * Pointer to Rx Hash queue. + * @param tunnel + * Tunnel type. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +mlx5_devx_hrxq_new(struct rte_eth_dev *dev, struct mlx5_hrxq *hrxq, + int tunnel __rte_unused) +{ + struct mlx5_priv *priv = dev->data->dev_private; + struct mlx5_ind_table_obj *ind_tbl = hrxq->ind_table; + struct mlx5_rxq_data *rxq_data = (*priv->rxqs)[ind_tbl->queues[0]]; + struct mlx5_rxq_ctrl *rxq_ctrl = + container_of(rxq_data, struct mlx5_rxq_ctrl, rxq); + struct mlx5_devx_tir_attr tir_attr = {0}; + const uint8_t *rss_key = hrxq->rss_key; + uint64_t hash_fields = hrxq->hash_fields; + int err; + + mlx5_devx_tir_attr_set(dev, rss_key, hash_fields, ind_tbl, tunnel, + rxq_ctrl->type, &tir_attr); hrxq->tir = mlx5_devx_cmd_create_tir(priv->sh->ctx, &tir_attr); if (!hrxq->tir) { DRV_LOG(ERR, "Port %u cannot create DevX TIR.", @@ -791,6 +828,70 @@ mlx5_devx_tir_destroy(struct mlx5_hrxq *hrxq) claim_zero(mlx5_devx_cmd_destroy(hrxq->tir)); } +/** + * Modify an Rx Hash queue configuration. + * + * @param dev + * Pointer to Ethernet device. + * @param hrxq + * Hash Rx queue to modify. + * @param rss_key + * RSS key for the Rx hash queue. + * @param hash_fields + * Verbs protocol hash field to make the RSS on. + * @param queues + * Queues entering in hash queue. In case of empty hash_fields only the + * first queue index will be taken for the indirection table. + * @param queues_n + * Number of queues. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +mlx5_devx_hrxq_modify(struct rte_eth_dev *dev, struct mlx5_hrxq *hrxq, + const uint8_t *rss_key, + uint64_t hash_fields, + const struct mlx5_ind_table_obj *ind_tbl) +{ + struct mlx5_priv *priv = dev->data->dev_private; + struct mlx5_rxq_data *rxq_data = (*priv->rxqs)[ind_tbl->queues[0]]; + struct mlx5_rxq_ctrl *rxq_ctrl = + container_of(rxq_data, struct mlx5_rxq_ctrl, rxq); + enum mlx5_rxq_obj_type rxq_obj_type = rxq_ctrl->obj->type; + struct mlx5_devx_modify_tir_attr modify_tir = {0}; + + /* validations */ + if (rxq_obj_type == MLX5_RXQ_OBJ_TYPE_IBV) { + /* shared action supported by devx interface only */ + rte_errno = EINVAL; + return -rte_errno; + } + /* + * untested for modification fields: + * - rx_hash_symmetric not set in hrxq_new(), + * - rx_hash_fn set hard-coded in hrxq_new(), + * - lro_xxx not set after rxq setup + */ + if (ind_tbl != hrxq->ind_table) + modify_tir.modify_bitmask |= + MLX5_MODIFY_TIR_IN_MODIFY_BITMASK_INDIRECT_TABLE; + if (hash_fields != hrxq->hash_fields || + memcmp(hrxq->rss_key, rss_key, MLX5_RSS_HASH_KEY_LEN)) + modify_tir.modify_bitmask |= + MLX5_MODIFY_TIR_IN_MODIFY_BITMASK_HASH; + mlx5_devx_tir_attr_set(dev, rss_key, hash_fields, ind_tbl, + 0, /* N/A - tunnel modification unsupported */ + rxq_obj_type, &modify_tir.tir); + if (mlx5_devx_cmd_modify_tir(hrxq->tir, &modify_tir)) { + DRV_LOG(ERR, "port %u cannot modify DevX TIR", + dev->data->port_id); + rte_errno = errno; + return -rte_errno; + } + return 0; +} + /** * Create a DevX drop action for Rx Hash queue. * @@ -833,6 +934,7 @@ struct mlx5_obj_ops devx_obj_ops = { .ind_table_destroy = mlx5_devx_ind_table_destroy, .hrxq_new = mlx5_devx_hrxq_new, .hrxq_destroy = mlx5_devx_tir_destroy, + .hrxq_modify = mlx5_devx_hrxq_modify, .drop_action_create = mlx5_devx_drop_action_create, .drop_action_destroy = mlx5_devx_drop_action_destroy, }; diff --git a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c index 487f9973bb..2618f72be0 100644 --- a/drivers/net/mlx5/mlx5_rxq.c +++ b/drivers/net/mlx5/mlx5_rxq.c @@ -1706,6 +1706,29 @@ mlx5_rxq_get_type(struct rte_eth_dev *dev, uint16_t idx) return MLX5_RXQ_TYPE_UNDEFINED; } +/** + * Match queues listed in arguments to queues contained in indirection table + * object. + * + * @param ind_tbl + * Pointer to indirection table to match. + * @param queues + * Queues to match to ques in indirection table. + * @param queues_n + * Number of queues in the array. + * + * @return + * 1 if all queues in indirection table match 0 othrwise. + */ +static int +mlx5_ind_table_obj_match_queues(const struct mlx5_ind_table_obj *ind_tbl, + const uint16_t *queues, uint32_t queues_n) +{ + return (ind_tbl->queues_n == queues_n) && + (!memcmp(ind_tbl->queues, queues, + ind_tbl->queues_n * sizeof(ind_tbl->queues[0]))); +} + /** * Get an indirection table. * @@ -1902,6 +1925,86 @@ mlx5_hrxq_get(struct rte_eth_dev *dev, return 0; } +/** + * Modify an Rx Hash queue configuration. + * + * @param dev + * Pointer to Ethernet device. + * @param hrxq + * Index to Hash Rx queue to modify. + * @param rss_key + * RSS key for the Rx hash queue. + * @param rss_key_len + * RSS key length. + * @param hash_fields + * Verbs protocol hash field to make the RSS on. + * @param queues + * Queues entering in hash queue. In case of empty hash_fields only the + * first queue index will be taken for the indirection table. + * @param queues_n + * Number of queues. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +int +mlx5_hrxq_modify(struct rte_eth_dev *dev, uint32_t hrxq_idx, + const uint8_t *rss_key, uint32_t rss_key_len, + uint64_t hash_fields, + const uint16_t *queues, uint32_t queues_n) +{ + int err; + struct mlx5_ind_table_obj *ind_tbl = NULL; + struct mlx5_priv *priv = dev->data->dev_private; + struct mlx5_hrxq *hrxq = + mlx5_ipool_get(priv->sh->ipool[MLX5_IPOOL_HRXQ], hrxq_idx); + int ret; + + if (!hrxq) { + rte_errno = EINVAL; + return -rte_errno; + } + /* validations */ + if (hrxq->rss_key_len != rss_key_len) { + /* rss_key_len is fixed size 40 byte & not supposed to change */ + rte_errno = EINVAL; + return -rte_errno; + } + + queues_n = hash_fields ? queues_n : 1; + if (mlx5_ind_table_obj_match_queues(hrxq->ind_table, + queues, queues_n)) { + ind_tbl = hrxq->ind_table; + } else { + ind_tbl = mlx5_ind_table_obj_get(dev, queues, queues_n); + if (!ind_tbl) + ind_tbl = mlx5_ind_table_obj_new(dev, queues, queues_n); + } + if (!ind_tbl) { + rte_errno = ENOMEM; + return -rte_errno; + } + ret = priv->obj_ops.hrxq_modify(dev, hrxq, rss_key, hash_fields, + ind_tbl); + if (ret) { + rte_errno = errno; + goto error; + } + if (ind_tbl != hrxq->ind_table) { + mlx5_ind_table_obj_release(dev, hrxq->ind_table); + hrxq->ind_table = ind_tbl; + } + hrxq->hash_fields = hash_fields; + memcpy(hrxq->rss_key, rss_key, rss_key_len); + return 0; +error: + err = rte_errno; + if (ind_tbl != hrxq->ind_table) + mlx5_ind_table_obj_release(dev, ind_tbl); + rte_errno = err; + return -rte_errno; +} + /** * Release the hash Rx queue. * diff --git a/drivers/net/mlx5/mlx5_rxtx.h b/drivers/net/mlx5/mlx5_rxtx.h index a8e6837ba1..2c28c3a521 100644 --- a/drivers/net/mlx5/mlx5_rxtx.h +++ b/drivers/net/mlx5/mlx5_rxtx.h @@ -387,7 +387,10 @@ void mlx5_drop_action_destroy(struct rte_eth_dev *dev); uint64_t mlx5_get_rx_port_offloads(void); uint64_t mlx5_get_rx_queue_offloads(struct rte_eth_dev *dev); void mlx5_rxq_timestamp_set(struct rte_eth_dev *dev); - +int mlx5_hrxq_modify(struct rte_eth_dev *dev, uint32_t hxrq_idx, + const uint8_t *rss_key, uint32_t rss_key_len, + uint64_t hash_fields, + const uint16_t *queues, uint32_t queues_n); /* mlx5_txq.c */ -- 2.26.2 ^ permalink raw reply [flat|nested] 106+ messages in thread
* [dpdk-dev] [PATCH v3 05/10] net/mlx5: shared action PMD 2020-10-03 22:06 ` [dpdk-dev] [PATCH v3 00/10] RTE flow shared action Andrey Vesnovaty ` (3 preceding siblings ...) 2020-10-03 22:06 ` [dpdk-dev] [PATCH v3 04/10] net/mlx5: modify hash Rx queue objects Andrey Vesnovaty @ 2020-10-03 22:06 ` Andrey Vesnovaty 2020-10-03 22:06 ` [dpdk-dev] [PATCH v3 06/10] net/mlx5: shared action PMD create conf arg Andrey Vesnovaty ` (5 subsequent siblings) 10 siblings, 0 replies; 106+ messages in thread From: Andrey Vesnovaty @ 2020-10-03 22:06 UTC (permalink / raw) To: dev Cc: jer, jerinjacobk, thomas, ferruh.yigit, stephen, bruce.richardson, orika, viacheslavo, andrey.vesnovaty, mdr, nhorman, ajit.khaparde, samik.gupta, Andrey Vesnovaty, Matan Azrad, Shahaf Shuler, Viacheslav Ovsiienko From: Andrey Vesnovaty <andreyv@mellanox.com> Implement rte_flow shared action API for mlx5 PMD. Handle shared action on flow create/destroy. Signed-off-by: Andrey Vesnovaty <andreyv@mellanox.com> --- drivers/net/mlx5/mlx5.c | 1 + drivers/net/mlx5/mlx5.h | 2 + drivers/net/mlx5/mlx5_defs.h | 3 + drivers/net/mlx5/mlx5_flow.c | 492 ++++++++++++++++++++++++++++++++--- drivers/net/mlx5/mlx5_flow.h | 83 ++++++ 5 files changed, 549 insertions(+), 32 deletions(-) diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index 4a807fb4fd..ec0e6b9bfc 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -1374,6 +1374,7 @@ mlx5_dev_close(struct rte_eth_dev *dev) * then this will return directly without any action. */ mlx5_flow_list_flush(dev, &priv->flows, true); + mlx5_shared_action_flush(dev); mlx5_flow_meter_flush(dev, NULL); /* Free the intermediate buffers for flow creation. */ mlx5_flow_free_intermediate(dev); diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 210004bc81..bbe16d4bd4 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -834,6 +834,8 @@ struct mlx5_priv { uint8_t fdb_def_rule; /* Whether fdb jump to table 1 is configured. */ struct mlx5_mp_id mp_id; /* ID of a multi-process process */ LIST_HEAD(fdir, mlx5_fdir_flow) fdir_flows; /* fdir flows. */ + LIST_HEAD(shared_action, rte_flow_shared_action) shared_actions; + /* shared actions */ }; #define PORT_ID(priv) ((priv)->dev_data->port_id) diff --git a/drivers/net/mlx5/mlx5_defs.h b/drivers/net/mlx5/mlx5_defs.h index 90f1839054..a20d6d1e34 100644 --- a/drivers/net/mlx5/mlx5_defs.h +++ b/drivers/net/mlx5/mlx5_defs.h @@ -194,6 +194,9 @@ #define MLX5_HAIRPIN_QUEUE_STRIDE 6 #define MLX5_HAIRPIN_JUMBO_LOG_SIZE (14 + 2) +/* Maximum number of shared actions supported by rte_flow */ +#define MLX5_MAX_SHARED_ACTIONS 1 + /* Definition of static_assert found in /usr/include/assert.h */ #ifndef HAVE_STATIC_ASSERT #define static_assert _Static_assert diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index 55cdc37f00..5757f146b4 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -220,6 +220,25 @@ static const struct rte_flow_expand_node mlx5_support_expansion[] = { }, }; +static struct rte_flow_shared_action * +mlx5_shared_action_create(struct rte_eth_dev *dev, + const struct rte_flow_action *action, + struct rte_flow_error *error); +static int mlx5_shared_action_destroy + (struct rte_eth_dev *dev, + struct rte_flow_shared_action *shared_action, + struct rte_flow_error *error); +static int mlx5_shared_action_update + (struct rte_eth_dev *dev, + struct rte_flow_shared_action *shared_action, + const struct rte_flow_action *action, + struct rte_flow_error *error); +static int mlx5_shared_action_query + (struct rte_eth_dev *dev, + const struct rte_flow_shared_action *action, + void *data, + struct rte_flow_error *error); + static const struct rte_flow_ops mlx5_flow_ops = { .validate = mlx5_flow_validate, .create = mlx5_flow_create, @@ -229,6 +248,10 @@ static const struct rte_flow_ops mlx5_flow_ops = { .query = mlx5_flow_query, .dev_dump = mlx5_flow_dev_dump, .get_aged_flows = mlx5_flow_get_aged_flows, + .shared_action_create = mlx5_shared_action_create, + .shared_action_destroy = mlx5_shared_action_destroy, + .shared_action_update = mlx5_shared_action_update, + .shared_action_query = mlx5_shared_action_query, }; /* Convert FDIR request to Generic flow. */ @@ -995,16 +1018,10 @@ mlx5_flow_validate_action_queue(const struct rte_flow_action *action, /* * Validate the rss action. * - * @param[in] action - * Pointer to the queue action. - * @param[in] action_flags - * Bit-fields that holds the actions detected until now. * @param[in] dev * Pointer to the Ethernet device structure. - * @param[in] attr - * Attributes of flow that includes this action. - * @param[in] item_flags - * Items that were detected. + * @param[in] action + * Pointer to the queue action. * @param[out] error * Pointer to error structure. * @@ -1012,23 +1029,14 @@ mlx5_flow_validate_action_queue(const struct rte_flow_action *action, * 0 on success, a negative errno value otherwise and rte_errno is set. */ int -mlx5_flow_validate_action_rss(const struct rte_flow_action *action, - uint64_t action_flags, - struct rte_eth_dev *dev, - const struct rte_flow_attr *attr, - uint64_t item_flags, - struct rte_flow_error *error) +mlx5_validate_action_rss(struct rte_eth_dev *dev, + const struct rte_flow_action *action, + struct rte_flow_error *error) { struct mlx5_priv *priv = dev->data->dev_private; const struct rte_flow_action_rss *rss = action->conf; - int tunnel = !!(item_flags & MLX5_FLOW_LAYER_TUNNEL); unsigned int i; - if (action_flags & MLX5_FLOW_FATE_ACTIONS) - return rte_flow_error_set(error, EINVAL, - RTE_FLOW_ERROR_TYPE_ACTION, NULL, - "can't have 2 fate actions" - " in same flow"); if (rss->func != RTE_ETH_HASH_FUNCTION_DEFAULT && rss->func != RTE_ETH_HASH_FUNCTION_TOEPLITZ) return rte_flow_error_set(error, ENOTSUP, @@ -1074,15 +1082,17 @@ mlx5_flow_validate_action_rss(const struct rte_flow_action *action, if ((rss->types & (ETH_RSS_L3_SRC_ONLY | ETH_RSS_L3_DST_ONLY)) && !(rss->types & ETH_RSS_IP)) return rte_flow_error_set(error, EINVAL, - RTE_FLOW_ERROR_TYPE_ACTION_CONF, NULL, - "L3 partial RSS requested but L3 RSS" - " type not specified"); + RTE_FLOW_ERROR_TYPE_ACTION_CONF, + NULL, + "L3 partial RSS requested but L3 " + "RSS type not specified"); if ((rss->types & (ETH_RSS_L4_SRC_ONLY | ETH_RSS_L4_DST_ONLY)) && !(rss->types & (ETH_RSS_UDP | ETH_RSS_TCP))) return rte_flow_error_set(error, EINVAL, - RTE_FLOW_ERROR_TYPE_ACTION_CONF, NULL, - "L4 partial RSS requested but L4 RSS" - " type not specified"); + RTE_FLOW_ERROR_TYPE_ACTION_CONF, + NULL, + "L4 partial RSS requested but L4 " + "RSS type not specified"); if (!priv->rxqs_n) return rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ACTION_CONF, @@ -1099,17 +1109,62 @@ mlx5_flow_validate_action_rss(const struct rte_flow_action *action, &rss->queue[i], "queue index out of range"); if (!(*priv->rxqs)[rss->queue[i]]) return rte_flow_error_set - (error, EINVAL, RTE_FLOW_ERROR_TYPE_ACTION_CONF, + (error, EINVAL, + RTE_FLOW_ERROR_TYPE_ACTION_CONF, &rss->queue[i], "queue is not configured"); } + return 0; +} + +/* + * Validate the rss action. + * + * @param[in] action + * Pointer to the queue action. + * @param[in] action_flags + * Bit-fields that holds the actions detected until now. + * @param[in] dev + * Pointer to the Ethernet device structure. + * @param[in] attr + * Attributes of flow that includes this action. + * @param[in] item_flags + * Items that were detected. + * @param[out] error + * Pointer to error structure. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +int +mlx5_flow_validate_action_rss(const struct rte_flow_action *action, + uint64_t action_flags, + struct rte_eth_dev *dev, + const struct rte_flow_attr *attr, + uint64_t item_flags, + struct rte_flow_error *error) +{ + const struct rte_flow_action_rss *rss = action->conf; + int tunnel = !!(item_flags & MLX5_FLOW_LAYER_TUNNEL); + int ret; + + if (action_flags & MLX5_FLOW_FATE_ACTIONS) + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ACTION, NULL, + "can't have 2 fate actions" + " in same flow"); + ret = mlx5_validate_action_rss(dev, action, error); + if (ret) + return ret; if (attr->egress) return rte_flow_error_set(error, ENOTSUP, - RTE_FLOW_ERROR_TYPE_ATTR_EGRESS, NULL, + RTE_FLOW_ERROR_TYPE_ATTR_EGRESS, + NULL, "rss action not supported for " "egress"); if (rss->level > 1 && !tunnel) return rte_flow_error_set(error, EINVAL, - RTE_FLOW_ERROR_TYPE_ACTION_CONF, NULL, + RTE_FLOW_ERROR_TYPE_ACTION_CONF, + NULL, "inner RSS is not supported for " "non-tunnel flows"); if ((item_flags & MLX5_FLOW_LAYER_ECPRI) && @@ -2733,6 +2788,131 @@ flow_get_rss_action(const struct rte_flow_action actions[]) return NULL; } +/* maps shared action to translated non shared in some actions array */ +struct mlx5_translated_shared_action { + struct rte_flow_shared_action *action; /**< Shared action */ + int index; /**< Index in related array of rte_flow_action */ +}; + +/** + * Translates actions of type RTE_FLOW_ACTION_TYPE_SHARED to related + * non shared action if translation possible. + * This functionality used to run same execution path for both shared & non + * shared actions on flow create. All necessary preparations for shared + * action handling should be preformed on *shared* actions list returned by + * from this call. + * + * @param[in] actions + * List of actions to translate. + * @param[out] shared + * List to store translated shared actions. + * @param[in, out] shared_n + * Size of *shared* array. On return should be updated with number of shared + * actions retrieved from the *actions* list. + * @param[out] translated_actions + * List of actions where all shared actions were translated to non shared + * if possible. NULL if no translation took place. + * @param[out] error + * Pointer to the error structure. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +flow_shared_actions_translate(const struct rte_flow_action actions[], + struct mlx5_translated_shared_action *shared, + int *shared_n, + struct rte_flow_action **translated_actions, + struct rte_flow_error *error) +{ + struct rte_flow_action *translated = NULL; + int n; + int copied_n = 0; + struct mlx5_translated_shared_action *shared_end = NULL; + + for (n = 0; actions[n].type != RTE_FLOW_ACTION_TYPE_END; n++) { + if (actions[n].type != RTE_FLOW_ACTION_TYPE_SHARED) + continue; + if (copied_n == *shared_n) { + return rte_flow_error_set + (error, EINVAL, RTE_FLOW_ERROR_TYPE_ACTION_NUM, + NULL, "too many shared actions"); + } + rte_memcpy(&shared[copied_n].action, &actions[n].conf, + sizeof(actions[n].conf)); + shared[copied_n].index = n; + copied_n++; + } + n++; + *shared_n = copied_n; + if (!copied_n) + return 0; + translated = rte_calloc(__func__, n, sizeof(struct rte_flow_action), 0); + rte_memcpy(translated, actions, n * sizeof(struct rte_flow_action)); + for (shared_end = shared + copied_n; shared < shared_end; shared++) { + const struct rte_flow_shared_action *shared_action; + + shared_action = shared->action; + switch (shared_action->type) { + case MLX5_FLOW_ACTION_SHARED_RSS: + translated[shared->index].type = + RTE_FLOW_ACTION_TYPE_RSS; + translated[shared->index].conf = + &shared_action->rss.origin; + break; + default: + rte_free(translated); + return rte_flow_error_set + (error, EINVAL, RTE_FLOW_ERROR_TYPE_ACTION, + NULL, "invalid shared action type"); + } + } + *translated_actions = translated; + return 0; +} + +/** + * Get Shared RSS action from the action list. + * + * @param[in] shared + * Pointer to the list of actions. + * @param[in] shared_n + * Actions list length. + * + * @return + * Pointer to the MLX5 RSS action if exist, else return NULL. + */ +static struct mlx5_shared_action_rss * +flow_get_shared_rss_action(struct mlx5_translated_shared_action *shared, + int shared_n) +{ + struct mlx5_translated_shared_action *shared_end; + + for (shared_end = shared + shared_n; shared < shared_end; shared++) { + struct rte_flow_shared_action *shared_action; + + shared_action = shared->action; + switch (shared_action->type) { + case MLX5_FLOW_ACTION_SHARED_RSS: + rte_atomic32_inc(&shared_action->refcnt); + return &shared_action->rss; + default: + break; + } + } + return NULL; +} + +struct rte_flow_shared_action * +mlx5_flow_get_shared_rss(struct rte_flow *flow) +{ + if (flow->shared_rss) + return container_of(flow->shared_rss, + struct rte_flow_shared_action, rss); + else + return NULL; +} + static unsigned int find_graph_root(const struct rte_flow_item pattern[], uint32_t rss_level) { @@ -4331,13 +4511,16 @@ static uint32_t flow_list_create(struct rte_eth_dev *dev, uint32_t *list, const struct rte_flow_attr *attr, const struct rte_flow_item items[], - const struct rte_flow_action actions[], + const struct rte_flow_action original_actions[], bool external, struct rte_flow_error *error) { struct mlx5_priv *priv = dev->data->dev_private; struct rte_flow *flow = NULL; struct mlx5_flow *dev_flow; const struct rte_flow_action_rss *rss; + struct mlx5_translated_shared_action + shared_actions[MLX5_MAX_SHARED_ACTIONS]; + int shared_actions_n = MLX5_MAX_SHARED_ACTIONS; union { struct rte_flow_expand_rss buf; uint8_t buffer[2048]; @@ -4357,14 +4540,23 @@ flow_list_create(struct rte_eth_dev *dev, uint32_t *list, struct rte_flow_expand_rss *buf = &expand_buffer.buf; struct mlx5_flow_rss_desc *rss_desc = &((struct mlx5_flow_rss_desc *) priv->rss_desc)[!!priv->flow_idx]; - const struct rte_flow_action *p_actions_rx = actions; + const struct rte_flow_action *p_actions_rx; uint32_t i; uint32_t idx = 0; int hairpin_flow; uint32_t hairpin_id = 0; struct rte_flow_attr attr_tx = { .priority = 0 }; - int ret; + const struct rte_flow_action *actions; + struct rte_flow_action *translated_actions = NULL; + int ret = flow_shared_actions_translate(original_actions, + shared_actions, + &shared_actions_n, + &translated_actions, error); + if (ret < 0) + return 0; + actions = (translated_actions) ? translated_actions : original_actions; + p_actions_rx = actions; hairpin_flow = flow_check_hairpin_split(dev, attr, actions); ret = flow_drv_validate(dev, attr, items, p_actions_rx, external, hairpin_flow, error); @@ -4416,6 +4608,8 @@ flow_list_create(struct rte_eth_dev *dev, uint32_t *list, buf->entries = 1; buf->entry[0].pattern = (void *)(uintptr_t)items; } + flow->shared_rss = flow_get_shared_rss_action(shared_actions, + shared_actions_n); /* * Record the start index when there is a nested call. All sub-flows * need to be translated before another calling. @@ -4487,6 +4681,7 @@ flow_list_create(struct rte_eth_dev *dev, uint32_t *list, ILIST_INSERT(priv->sh->ipool[MLX5_IPOOL_RTE_FLOW], list, idx, flow, next); flow_rxq_flags_set(dev, flow); + rte_free(translated_actions); /* Nested flow creation index recovery. */ priv->flow_idx = priv->flow_nested_idx; if (priv->flow_nested_idx) @@ -4501,6 +4696,7 @@ flow_list_create(struct rte_eth_dev *dev, uint32_t *list, rte_errno = ret; /* Restore rte_errno. */ error_before_flow: ret = rte_errno; + rte_free(translated_actions); if (hairpin_id) mlx5_flow_id_release(priv->sh->flow_id_pool, hairpin_id); @@ -6312,3 +6508,235 @@ mlx5_flow_get_aged_flows(struct rte_eth_dev *dev, void **contexts, dev->data->port_id); return -ENOTSUP; } + +/** + * Retrieve driver ops struct. + * + * @param[in] dev + * Pointer to the dev structure. + * @param[in] error_message + * Error message to set if driver ops struct not found. + * @param[out] error + * Perform verbose error reporting if not NULL. Initialized in case of + * error only. + * + * @return + * Pointer to driver ops on success, otherwise NULL and rte_errno is set. + */ +static const struct mlx5_flow_driver_ops * +flow_drv_dv_ops_get(struct rte_eth_dev *dev, + const char *error_message, + struct rte_flow_error *error) +{ + struct rte_flow_attr attr = { .transfer = 0 }; + + if (flow_get_drv_type(dev, &attr) != MLX5_FLOW_TYPE_DV) { + rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION, + NULL, error_message); + DRV_LOG(ERR, "port %u %s.", dev->data->port_id, error_message); + return NULL; + } + + return flow_get_drv_ops(MLX5_FLOW_TYPE_DV); +} + +/* Wrapper for driver action_validate op callback */ +static int +flow_drv_action_validate(struct rte_eth_dev *dev, + const struct rte_flow_action *action, + struct rte_flow_error *error) +{ + const struct mlx5_flow_driver_ops *fops = flow_drv_dv_ops_get(dev, + "action registration unsupported", error); + return (fops) ? fops->action_validate(dev, action, error) : -rte_errno; +} + +/* Wrapper for driver action_create op callback */ +static struct rte_flow_shared_action * +flow_drv_action_create(struct rte_eth_dev *dev, + const struct rte_flow_action *action, + struct rte_flow_error *error) +{ + const struct mlx5_flow_driver_ops *fops = flow_drv_dv_ops_get(dev, + "action registration unsupported", error); + return (fops) ? fops->action_create(dev, action, error) : NULL; +} + +/** + * Destroys the shared action by handle. + * + * @param dev + * Pointer to Ethernet device structure. + * @param[in] action + * Handle for the shared action to be destroyed. + * @param[out] error + * Perform verbose error reporting if not NULL. PMDs initialize this + * structure in case of error only. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + * + * @note: wrapper for driver action_create op callback. + */ +static int +mlx5_shared_action_destroy(struct rte_eth_dev *dev, + struct rte_flow_shared_action *action, + struct rte_flow_error *error) +{ + const struct mlx5_flow_driver_ops *fops = flow_drv_dv_ops_get(dev, + "action registration unsupported", error); + return (fops) ? fops->action_destroy(dev, action, error) : -rte_errno; +} + +/* Wrapper for driver action_destroy op callback */ +static int +flow_drv_action_update(struct rte_eth_dev *dev, + struct rte_flow_shared_action *action, + const void *action_conf, + struct rte_flow_error *error) +{ + const struct mlx5_flow_driver_ops *fops = flow_drv_dv_ops_get(dev, + "action registration unsupported", error); + return (fops) ? fops->action_update(dev, action, + action_conf, error) + : -rte_errno; +} + +/** + * Create shared action for reuse in multiple flow rules. + * + * @param dev + * Pointer to Ethernet device structure. + * @param[in] action + * Action configuration for shared action creation. + * @param[out] error + * Perform verbose error reporting if not NULL. PMDs initialize this + * structure in case of error only. + * @return + * A valid handle in case of success, NULL otherwise and rte_errno is set. + */ +static struct rte_flow_shared_action * +mlx5_shared_action_create(struct rte_eth_dev *dev, + const struct rte_flow_action *action, + struct rte_flow_error *error) +{ + if (flow_drv_action_validate(dev, action, error)) + return NULL; + return flow_drv_action_create(dev, action, error); +} + +/** + * Updates inplace the shared action configuration pointed by *action* handle + * with the configuration provided as *update* argument. + * The update of the shared action configuration effects all flow rules reusing + * the action via handle. + * + * @param dev + * Pointer to Ethernet device structure. + * @param[in] action + * Handle for the shared action to be updated. + * @param[in] update + * Action specification used to modify the action pointed by handle. + * *update* should be of same type with the action pointed by the *action* + * handle argument, otherwise considered as invalid. + * @param[out] error + * Perform verbose error reporting if not NULL. PMDs initialize this + * structure in case of error only. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +mlx5_shared_action_update(struct rte_eth_dev *dev, + struct rte_flow_shared_action *shared_action, + const struct rte_flow_action *action, + struct rte_flow_error *error) +{ + int ret; + + switch (shared_action->type) { + case MLX5_FLOW_ACTION_SHARED_RSS: + if (action->type != RTE_FLOW_ACTION_TYPE_RSS) { + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ACTION, + NULL, + "update action type invalid"); + } + ret = flow_drv_action_validate(dev, action, error); + if (ret) + return ret; + return flow_drv_action_update(dev, shared_action, action->conf, + error); + default: + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION, + NULL, + "action type not supported"); + } +} + +/** + * Query the shared action by handle. + * + * This function allows retrieving action-specific data such as counters. + * Data is gathered by special action which may be present/referenced in + * more than one flow rule definition. + * + * \see RTE_FLOW_ACTION_TYPE_COUNT + * + * @param dev + * Pointer to Ethernet device structure. + * @param[in] action + * Handle for the shared action to query. + * @param[in, out] data + * Pointer to storage for the associated query data type. + * @param[out] error + * Perform verbose error reporting if not NULL. PMDs initialize this + * structure in case of error only. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +mlx5_shared_action_query(struct rte_eth_dev *dev, + const struct rte_flow_shared_action *action, + void *data, + struct rte_flow_error *error) +{ + (void)dev; + switch (action->type) { + case MLX5_FLOW_ACTION_SHARED_RSS: + *((int32_t *)data) = rte_atomic32_read(&action->refcnt); + return 0; + default: + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION, + NULL, + "action type not supported"); + } +} + +/** + * Destroy all shared actions. + * + * @param dev + * Pointer to Ethernet device. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +int +mlx5_shared_action_flush(struct rte_eth_dev *dev) +{ + struct rte_flow_error error; + struct mlx5_priv *priv = dev->data->dev_private; + struct rte_flow_shared_action *action; + int ret = 0; + + while (!LIST_EMPTY(&priv->shared_actions)) { + action = LIST_FIRST(&priv->shared_actions); + ret = mlx5_shared_action_destroy(dev, action, &error); + } + return ret; +} diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h index 92301e494c..e6ada24943 100644 --- a/drivers/net/mlx5/mlx5_flow.h +++ b/drivers/net/mlx5/mlx5_flow.h @@ -196,6 +196,7 @@ enum mlx5_feature_name { #define MLX5_FLOW_ACTION_SET_IPV6_DSCP (1ull << 33) #define MLX5_FLOW_ACTION_AGE (1ull << 34) #define MLX5_FLOW_ACTION_DEFAULT_MISS (1ull << 35) +#define MLX5_FLOW_ACTION_SHARED_RSS (1ull << 36) #define MLX5_FLOW_FATE_ACTIONS \ (MLX5_FLOW_ACTION_DROP | MLX5_FLOW_ACTION_QUEUE | \ @@ -830,6 +831,7 @@ struct mlx5_fdir_flow { /* Flow structure. */ struct rte_flow { ILIST_ENTRY(uint32_t)next; /**< Index to the next flow structure. */ + struct mlx5_shared_action_rss *shared_rss; /** < Shred RSS action. */ uint32_t dev_handles; /**< Device flow handles that are part of the flow. */ uint32_t drv_type:2; /**< Driver type. */ @@ -843,6 +845,62 @@ struct rte_flow { uint16_t meter; /**< Holds flow meter id. */ } __rte_packed; +/* + * Define list of valid combinations of RX Hash fields + * (see enum ibv_rx_hash_fields). + */ +#define MLX5_RSS_HASH_IPV4 (IBV_RX_HASH_SRC_IPV4 | IBV_RX_HASH_DST_IPV4) +#define MLX5_RSS_HASH_IPV4_TCP \ + (MLX5_RSS_HASH_IPV4 | \ + IBV_RX_HASH_SRC_PORT_TCP | IBV_RX_HASH_SRC_PORT_TCP) +#define MLX5_RSS_HASH_IPV4_UDP \ + (MLX5_RSS_HASH_IPV4 | \ + IBV_RX_HASH_SRC_PORT_UDP | IBV_RX_HASH_SRC_PORT_UDP) +#define MLX5_RSS_HASH_IPV6 (IBV_RX_HASH_SRC_IPV6 | IBV_RX_HASH_DST_IPV6) +#define MLX5_RSS_HASH_IPV6_TCP \ + (MLX5_RSS_HASH_IPV6 | \ + IBV_RX_HASH_SRC_PORT_TCP | IBV_RX_HASH_SRC_PORT_TCP) +#define MLX5_RSS_HASH_IPV6_UDP \ + (MLX5_RSS_HASH_IPV6 | \ + IBV_RX_HASH_SRC_PORT_UDP | IBV_RX_HASH_SRC_PORT_UDP) +#define MLX5_RSS_HASH_NONE 0ULL + +/* array of valid combinations of RX Hash fields for RSS */ +static const uint64_t mlx5_rss_hash_fields[] = { + MLX5_RSS_HASH_IPV4, + MLX5_RSS_HASH_IPV4_TCP, + MLX5_RSS_HASH_IPV4_UDP, + MLX5_RSS_HASH_IPV6, + MLX5_RSS_HASH_IPV6_TCP, + MLX5_RSS_HASH_IPV6_UDP, + MLX5_RSS_HASH_NONE, +}; + +#define MLX5_RSS_HASH_FIELDS_LEN RTE_DIM(mlx5_rss_hash_fields) + +/* Shared RSS action structure */ +struct mlx5_shared_action_rss { + struct rte_flow_action_rss origin; /**< Original rte RSS action. */ + uint8_t key[MLX5_RSS_HASH_KEY_LEN]; /**< RSS hash key. */ + uint16_t *queue; /**< Queue indices to use. */ + uint32_t hrxq[MLX5_RSS_HASH_FIELDS_LEN]; + /**< Hash RX queue indexes mapped to mlx5_rss_hash_fields */ + uint32_t hrxq_tunnel[MLX5_RSS_HASH_FIELDS_LEN]; + /**< Hash RX queue indexes for tunneled RSS */ +}; + +struct rte_flow_shared_action { + LIST_ENTRY(rte_flow_shared_action) next; + /**< Pointer to the next element. */ + rte_atomic32_t refcnt; + uint64_t type; + /**< Shared action type (see MLX5_FLOW_ACTION_SHARED_*). */ + union { + struct mlx5_shared_action_rss rss; + /**< Shared RSS action. */ + }; +}; + typedef int (*mlx5_flow_validate_t)(struct rte_eth_dev *dev, const struct rte_flow_attr *attr, const struct rte_flow_item items[], @@ -897,6 +955,22 @@ typedef int (*mlx5_flow_get_aged_flows_t) void **context, uint32_t nb_contexts, struct rte_flow_error *error); +typedef int (*mlx5_flow_action_validate_t)(struct rte_eth_dev *dev, + const struct rte_flow_action *action, + struct rte_flow_error *error); +typedef struct rte_flow_shared_action *(*mlx5_flow_action_create_t) + (struct rte_eth_dev *dev, + const struct rte_flow_action *action, + struct rte_flow_error *error); +typedef int (*mlx5_flow_action_destroy_t) + (struct rte_eth_dev *dev, + struct rte_flow_shared_action *action, + struct rte_flow_error *error); +typedef int (*mlx5_flow_action_update_t) + (struct rte_eth_dev *dev, + struct rte_flow_shared_action *action, + const void *action_conf, + struct rte_flow_error *error); struct mlx5_flow_driver_ops { mlx5_flow_validate_t validate; mlx5_flow_prepare_t prepare; @@ -913,6 +987,10 @@ struct mlx5_flow_driver_ops { mlx5_flow_counter_free_t counter_free; mlx5_flow_counter_query_t counter_query; mlx5_flow_get_aged_flows_t get_aged_flows; + mlx5_flow_action_validate_t action_validate; + mlx5_flow_action_create_t action_create; + mlx5_flow_action_destroy_t action_destroy; + mlx5_flow_action_update_t action_update; }; /* mlx5_flow.c */ @@ -938,6 +1016,9 @@ int mlx5_flow_get_reg_id(struct rte_eth_dev *dev, const struct rte_flow_action *mlx5_flow_find_action (const struct rte_flow_action *actions, enum rte_flow_action_type action); +int mlx5_validate_action_rss(struct rte_eth_dev *dev, + const struct rte_flow_action *action, + struct rte_flow_error *error); int mlx5_flow_validate_action_count(struct rte_eth_dev *dev, const struct rte_flow_attr *attr, struct rte_flow_error *error); @@ -1056,4 +1137,6 @@ int mlx5_flow_destroy_policer_rules(struct rte_eth_dev *dev, const struct rte_flow_attr *attr); int mlx5_flow_meter_flush(struct rte_eth_dev *dev, struct rte_mtr_error *error); +struct rte_flow_shared_action *mlx5_flow_get_shared_rss(struct rte_flow *flow); +int mlx5_shared_action_flush(struct rte_eth_dev *dev); #endif /* RTE_PMD_MLX5_FLOW_H_ */ -- 2.26.2 ^ permalink raw reply [flat|nested] 106+ messages in thread
* [dpdk-dev] [PATCH v3 06/10] net/mlx5: shared action PMD create conf arg 2020-10-03 22:06 ` [dpdk-dev] [PATCH v3 00/10] RTE flow shared action Andrey Vesnovaty ` (4 preceding siblings ...) 2020-10-03 22:06 ` [dpdk-dev] [PATCH v3 05/10] net/mlx5: shared action PMD Andrey Vesnovaty @ 2020-10-03 22:06 ` Andrey Vesnovaty 2020-10-03 22:06 ` [dpdk-dev] [PATCH v3 07/10] net/mlx5: driver support for shared action Andrey Vesnovaty ` (4 subsequent siblings) 10 siblings, 0 replies; 106+ messages in thread From: Andrey Vesnovaty @ 2020-10-03 22:06 UTC (permalink / raw) To: dev Cc: jer, jerinjacobk, thomas, ferruh.yigit, stephen, bruce.richardson, orika, viacheslavo, andrey.vesnovaty, mdr, nhorman, ajit.khaparde, samik.gupta, Matan Azrad, Shahaf Shuler, Viacheslav Ovsiienko Add support for configuration argument in shared action create API. Configuration structure forwarded to verbs/devx layer. Signed-off-by: Andrey Vesnovaty <andreyv@nvidia.com> --- drivers/net/mlx5/mlx5_flow.c | 23 ++++++++++++++--------- drivers/net/mlx5/mlx5_flow.h | 9 ++++++--- 2 files changed, 20 insertions(+), 12 deletions(-) diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index 5757f146b4..5d1d504b21 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -222,6 +222,7 @@ static const struct rte_flow_expand_node mlx5_support_expansion[] = { static struct rte_flow_shared_action * mlx5_shared_action_create(struct rte_eth_dev *dev, + const struct rte_flow_shared_action_conf *conf, const struct rte_flow_action *action, struct rte_flow_error *error); static int mlx5_shared_action_destroy @@ -6544,23 +6545,26 @@ flow_drv_dv_ops_get(struct rte_eth_dev *dev, /* Wrapper for driver action_validate op callback */ static int flow_drv_action_validate(struct rte_eth_dev *dev, + const struct rte_flow_shared_action_conf *conf, const struct rte_flow_action *action, struct rte_flow_error *error) { const struct mlx5_flow_driver_ops *fops = flow_drv_dv_ops_get(dev, "action registration unsupported", error); - return (fops) ? fops->action_validate(dev, action, error) : -rte_errno; + return (fops) ? fops->action_validate(dev, conf, action, error) + : -rte_errno; } /* Wrapper for driver action_create op callback */ static struct rte_flow_shared_action * flow_drv_action_create(struct rte_eth_dev *dev, - const struct rte_flow_action *action, - struct rte_flow_error *error) + const struct rte_flow_shared_action_conf *conf, + const struct rte_flow_action *action, + struct rte_flow_error *error) { const struct mlx5_flow_driver_ops *fops = flow_drv_dv_ops_get(dev, "action registration unsupported", error); - return (fops) ? fops->action_create(dev, action, error) : NULL; + return (fops) ? fops->action_create(dev, conf, action, error) : NULL; } /** @@ -6618,12 +6622,13 @@ flow_drv_action_update(struct rte_eth_dev *dev, */ static struct rte_flow_shared_action * mlx5_shared_action_create(struct rte_eth_dev *dev, - const struct rte_flow_action *action, - struct rte_flow_error *error) + const struct rte_flow_shared_action_conf *conf, + const struct rte_flow_action *action, + struct rte_flow_error *error) { - if (flow_drv_action_validate(dev, action, error)) + if (flow_drv_action_validate(dev, conf, action, error)) return NULL; - return flow_drv_action_create(dev, action, error); + return flow_drv_action_create(dev, conf, action, error); } /** @@ -6663,7 +6668,7 @@ mlx5_shared_action_update(struct rte_eth_dev *dev, NULL, "update action type invalid"); } - ret = flow_drv_action_validate(dev, action, error); + ret = flow_drv_action_validate(dev, NULL, action, error); if (ret) return ret; return flow_drv_action_update(dev, shared_action, action->conf, diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h index e6ada24943..c7be77b961 100644 --- a/drivers/net/mlx5/mlx5_flow.h +++ b/drivers/net/mlx5/mlx5_flow.h @@ -955,11 +955,14 @@ typedef int (*mlx5_flow_get_aged_flows_t) void **context, uint32_t nb_contexts, struct rte_flow_error *error); -typedef int (*mlx5_flow_action_validate_t)(struct rte_eth_dev *dev, - const struct rte_flow_action *action, - struct rte_flow_error *error); +typedef int (*mlx5_flow_action_validate_t) + (struct rte_eth_dev *dev, + const struct rte_flow_shared_action_conf *conf, + const struct rte_flow_action *action, + struct rte_flow_error *error); typedef struct rte_flow_shared_action *(*mlx5_flow_action_create_t) (struct rte_eth_dev *dev, + const struct rte_flow_shared_action_conf *conf, const struct rte_flow_action *action, struct rte_flow_error *error); typedef int (*mlx5_flow_action_destroy_t) -- 2.26.2 ^ permalink raw reply [flat|nested] 106+ messages in thread
* [dpdk-dev] [PATCH v3 07/10] net/mlx5: driver support for shared action 2020-10-03 22:06 ` [dpdk-dev] [PATCH v3 00/10] RTE flow shared action Andrey Vesnovaty ` (5 preceding siblings ...) 2020-10-03 22:06 ` [dpdk-dev] [PATCH v3 06/10] net/mlx5: shared action PMD create conf arg Andrey Vesnovaty @ 2020-10-03 22:06 ` Andrey Vesnovaty 2020-10-03 22:06 ` [dpdk-dev] [PATCH v3 08/10] net/mlx5: shared action create conf drv support Andrey Vesnovaty ` (3 subsequent siblings) 10 siblings, 0 replies; 106+ messages in thread From: Andrey Vesnovaty @ 2020-10-03 22:06 UTC (permalink / raw) To: dev Cc: jer, jerinjacobk, thomas, ferruh.yigit, stephen, bruce.richardson, orika, viacheslavo, andrey.vesnovaty, mdr, nhorman, ajit.khaparde, samik.gupta, Andrey Vesnovaty, Matan Azrad, Shahaf Shuler, Viacheslav Ovsiienko From: Andrey Vesnovaty <andreyv@mellanox.com> Implement shared action create/destroy/update/query. Implement RSS shared action and handle shared RSS on flow apply and release. Note: currently implemented for sharede RSS action only Signed-off-by: Andrey Vesnovaty <andreyv@mellanox.com> --- drivers/net/mlx5/mlx5_flow_dv.c | 671 ++++++++++++++++++++++++++++++-- 1 file changed, 648 insertions(+), 23 deletions(-) diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c index 56529c854b..13effc0af0 100644 --- a/drivers/net/mlx5/mlx5_flow_dv.c +++ b/drivers/net/mlx5/mlx5_flow_dv.c @@ -8878,6 +8878,157 @@ __flow_dv_translate(struct rte_eth_dev *dev, return 0; } +/** + * Set hash RX queue by hash fields (see enum ibv_rx_hash_fields) + * and tunnel. + * + * @param[in, out] action + * Shred RSS action holding hash RX queue objects. + * @param[in] hash_fields + * Defines combination of packet fields to participate in RX hash. + * @param[in] tunnel + * Tunnel type + * @param[in] hrxq_idx + * Hash RX queue index to set. + * + * @return + * 0 on success, otherwise negative errno value. + */ +static int +__flow_dv_action_rss_hrxq_set(struct mlx5_shared_action_rss *action, + const uint64_t hash_fields, + const int tunnel, + uint32_t hrxq_idx) +{ + uint32_t *hrxqs = (tunnel) ? action->hrxq : action->hrxq_tunnel; + + switch (hash_fields & ~IBV_RX_HASH_INNER) { + case MLX5_RSS_HASH_IPV4: + hrxqs[0] = hrxq_idx; + return 0; + case MLX5_RSS_HASH_IPV4_TCP: + hrxqs[1] = hrxq_idx; + return 0; + case MLX5_RSS_HASH_IPV4_UDP: + hrxqs[2] = hrxq_idx; + return 0; + case MLX5_RSS_HASH_IPV6: + hrxqs[3] = hrxq_idx; + return 0; + case MLX5_RSS_HASH_IPV6_TCP: + hrxqs[4] = hrxq_idx; + return 0; + case MLX5_RSS_HASH_IPV6_UDP: + hrxqs[5] = hrxq_idx; + return 0; + case MLX5_RSS_HASH_NONE: + hrxqs[6] = hrxq_idx; + return 0; + default: + return -1; + } +} + +/** + * Look up for hash RX queue by hash fields (see enum ibv_rx_hash_fields) + * and tunnel. + * + * @param[in] action + * Shred RSS action holding hash RX queue objects. + * @param[in] hash_fields + * Defines combination of packet fields to participate in RX hash. + * @param[in] tunnel + * Tunnel type + * + * @return + * Valid hash RX queue index, otherwise 0. + */ +static uint32_t +__flow_dv_action_rss_hrxq_lookup(const struct mlx5_shared_action_rss *action, + const uint64_t hash_fields, + const int tunnel) +{ + const uint32_t *hrxqs = (tunnel) ? action->hrxq : action->hrxq_tunnel; + + switch (hash_fields & ~IBV_RX_HASH_INNER) { + case MLX5_RSS_HASH_IPV4: + return hrxqs[0]; + case MLX5_RSS_HASH_IPV4_TCP: + return hrxqs[1]; + case MLX5_RSS_HASH_IPV4_UDP: + return hrxqs[2]; + case MLX5_RSS_HASH_IPV6: + return hrxqs[3]; + case MLX5_RSS_HASH_IPV6_TCP: + return hrxqs[4]; + case MLX5_RSS_HASH_IPV6_UDP: + return hrxqs[5]; + case MLX5_RSS_HASH_NONE: + return hrxqs[6]; + default: + return 0; + } +} + +/** + * Retrieves hash RX queue suitable for the *flow*. + * If shared action configured for *flow* suitable hash RX queue will be + * retrieved from attached shared action. + * + * @param[in] flow + * Shred RSS action holding hash RX queue objects. + * @param[in] dev_flow + * Pointer to the sub flow. + * @param[out] hrxq + * Pointer to retrieved hash RX queue object. + * + * @return + * Valid hash RX queue index, otherwise 0 and rte_errno is set. + */ +static uint32_t +__flow_dv_rss_get_hrxq(struct rte_eth_dev *dev, struct rte_flow *flow, + struct mlx5_flow *dev_flow, + struct mlx5_hrxq **hrxq) +{ + struct mlx5_priv *priv = dev->data->dev_private; + uint32_t hrxq_idx; + + if (flow->shared_rss) { + hrxq_idx = __flow_dv_action_rss_hrxq_lookup + (flow->shared_rss, dev_flow->hash_fields, + !!(dev_flow->handle->layers & + MLX5_FLOW_LAYER_TUNNEL)); + if (hrxq_idx) { + *hrxq = mlx5_ipool_get(priv->sh->ipool[MLX5_IPOOL_HRXQ], + hrxq_idx); + rte_atomic32_inc(&(*hrxq)->refcnt); + } + } else { + struct mlx5_flow_rss_desc *rss_desc = + &((struct mlx5_flow_rss_desc *)priv->rss_desc) + [!!priv->flow_nested_idx]; + + MLX5_ASSERT(rss_desc->queue_num); + hrxq_idx = mlx5_hrxq_get(dev, rss_desc->key, + MLX5_RSS_HASH_KEY_LEN, + dev_flow->hash_fields, + rss_desc->queue, rss_desc->queue_num); + if (!hrxq_idx) { + hrxq_idx = mlx5_hrxq_new(dev, + rss_desc->key, + MLX5_RSS_HASH_KEY_LEN, + dev_flow->hash_fields, + rss_desc->queue, + rss_desc->queue_num, + !!(dev_flow->handle->layers & + MLX5_FLOW_LAYER_TUNNEL)); + } + *hrxq = mlx5_ipool_get(priv->sh->ipool[MLX5_IPOOL_HRXQ], + hrxq_idx); + } + return hrxq_idx; +} + /** * Apply the flow to the NIC, lock free, * (mutex should be acquired by caller). @@ -8936,30 +9087,10 @@ __flow_dv_apply(struct rte_eth_dev *dev, struct rte_flow *flow, dv->actions[n++] = drop_hrxq->action; } } else if (dh->fate_action == MLX5_FLOW_FATE_QUEUE) { - struct mlx5_hrxq *hrxq; - uint32_t hrxq_idx; - struct mlx5_flow_rss_desc *rss_desc = - &((struct mlx5_flow_rss_desc *)priv->rss_desc) - [!!priv->flow_nested_idx]; + struct mlx5_hrxq *hrxq = NULL; + uint32_t hrxq_idx = __flow_dv_rss_get_hrxq + (dev, flow, dev_flow, &hrxq); - MLX5_ASSERT(rss_desc->queue_num); - hrxq_idx = mlx5_hrxq_get(dev, rss_desc->key, - MLX5_RSS_HASH_KEY_LEN, - dev_flow->hash_fields, - rss_desc->queue, - rss_desc->queue_num); - if (!hrxq_idx) { - hrxq_idx = mlx5_hrxq_new - (dev, rss_desc->key, - MLX5_RSS_HASH_KEY_LEN, - dev_flow->hash_fields, - rss_desc->queue, - rss_desc->queue_num, - !!(dh->layers & - MLX5_FLOW_LAYER_TUNNEL)); - } - hrxq = mlx5_ipool_get(priv->sh->ipool[MLX5_IPOOL_HRXQ], - hrxq_idx); if (!hrxq) { rte_flow_error_set (error, rte_errno, @@ -9378,12 +9509,16 @@ __flow_dv_remove(struct rte_eth_dev *dev, struct rte_flow *flow) static void __flow_dv_destroy(struct rte_eth_dev *dev, struct rte_flow *flow) { + struct rte_flow_shared_action *shared; struct mlx5_flow_handle *dev_handle; struct mlx5_priv *priv = dev->data->dev_private; if (!flow) return; __flow_dv_remove(dev, flow); + shared = mlx5_flow_get_shared_rss(flow); + if (shared) + rte_atomic32_dec(&shared->refcnt); if (flow->counter) { flow_dv_counter_release(dev, flow->counter); flow->counter = 0; @@ -9423,6 +9558,411 @@ __flow_dv_destroy(struct rte_eth_dev *dev, struct rte_flow *flow) } } +/** + * Release array of hash RX queue objects. + * Helper function. + * + * @param[in] dev + * Pointer to the Ethernet device structure. + * @param[in, out] hrxqs + * Array of hash RX queue objects. + * + * @return + * Total number of references to hash RX queue objects in *hrxqs* array + * after this operation. + */ +static int +__flow_dv_hrxqs_release(struct rte_eth_dev *dev, + uint32_t (*hrxqs)[MLX5_RSS_HASH_FIELDS_LEN]) +{ + size_t i; + int remaining = 0, ret = 0, ret_tunnel = 0; + + for (i = 0; i < RTE_DIM(*hrxqs); i++) { + ret = mlx5_hrxq_release(dev, (*hrxqs)[i]); + if (!ret) + (*hrxqs)[i] = 0; + remaining += ret + ret_tunnel; + } + return remaining; +} + +/** + * Release all hash RX queue objects representing shared RSS action. + * + * @param[in] dev + * Pointer to the Ethernet device structure. + * @param[in, out] action + * Shared RSS action to remove hash RX queue objects from. + * + * @return + * Total number of references to hash RX queue objects stored in *action* + * after this operation. + * Expected to be 0 if no external references held. + */ +static int +__flow_dv_action_rss_hrxqs_release(struct rte_eth_dev *dev, + struct mlx5_shared_action_rss *action) +{ + return __flow_dv_hrxqs_release(dev, &action->hrxq) + + __flow_dv_hrxqs_release(dev, &action->hrxq_tunnel); +} + +/** + * Setup shared RSS action. + * Prepare set of hash RX queue objects sufficient to handle all valid + * hash_fields combinations (see enum ibv_rx_hash_fields). + * + * @param[in] dev + * Pointer to the Ethernet device structure. + * @param[in, out] action + * Partially initialized shared RSS action. + * @param[out] error + * Perform verbose error reporting if not NULL. Initialized in case of + * error only. + * + * @return + * 0 on success, otherwise negative errno value. + */ +static int +__flow_dv_action_rss_setup(struct rte_eth_dev *dev, + struct mlx5_shared_action_rss *action, + struct rte_flow_error *error) +{ + size_t i; + int err; + + for (i = 0; i < MLX5_RSS_HASH_FIELDS_LEN; i++) { + uint32_t hrxq_idx; + uint64_t hash_fields = mlx5_rss_hash_fields[i]; + int tunnel; + + for (tunnel = 0; tunnel < 2; tunnel++) { + hrxq_idx = mlx5_hrxq_new(dev, action->origin.key, + MLX5_RSS_HASH_KEY_LEN, + hash_fields, + action->origin.queue, + action->origin.queue_num, + tunnel); + if (!hrxq_idx) { + rte_flow_error_set + (error, rte_errno, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL, + "cannot get hash queue"); + goto error_hrxq_new; + } + err = __flow_dv_action_rss_hrxq_set + (action, hash_fields, tunnel, hrxq_idx); + MLX5_ASSERT(!err); + } + } + return 0; +error_hrxq_new: + err = rte_errno; + __flow_dv_action_rss_hrxqs_release(dev, action); + rte_errno = err; + return -rte_errno; +} + +/** + * Create shared RSS action. + * + * @param[in] dev + * Pointer to the Ethernet device structure. + * @param[in] rss + * RSS action specification used to create shared action. + * @param[out] error + * Perform verbose error reporting if not NULL. Initialized in case of + * error only. + * + * @return + * A valid shared action handle in case of success, NULL otherwise and + * rte_errno is set. + */ +static struct rte_flow_shared_action * +__flow_dv_action_rss_create(struct rte_eth_dev *dev, + const struct rte_flow_action_rss *rss, + struct rte_flow_error *error) +{ + struct rte_flow_shared_action *shared_action = NULL; + void *queue = NULL; + uint32_t queue_size; + struct mlx5_shared_action_rss *shared_rss; + struct rte_flow_action_rss *origin; + const uint8_t *rss_key; + + queue_size = RTE_ALIGN_CEIL(rss->queue_num * sizeof(uint16_t), + sizeof(void *)); + queue = rte_calloc(__func__, 1, queue_size, 0); + shared_action = rte_calloc(__func__, 1, sizeof(*shared_action), 0); + if (!shared_action || !queue) { + rte_flow_error_set(error, ENOMEM, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL, + "cannot allocate resource memory"); + goto error_rss_init; + } + shared_rss = &shared_action->rss; + shared_rss->queue = queue; + origin = &shared_rss->origin; + origin->func = rss->func; + origin->level = rss->level; + /* RSS type 0 indicates default RSS type (ETH_RSS_IP). */ + origin->types = !rss->types ? ETH_RSS_IP : rss->types; + /* NULL RSS key indicates default RSS key. */ + rss_key = !rss->key ? rss_hash_default_key : rss->key; + rte_memcpy(shared_rss->key, rss_key, MLX5_RSS_HASH_KEY_LEN); + origin->key = &shared_rss->key[0]; + origin->key_len = MLX5_RSS_HASH_KEY_LEN; + rte_memcpy(shared_rss->queue, rss->queue, queue_size); + origin->queue = shared_rss->queue; + origin->queue_num = rss->queue_num; + if (__flow_dv_action_rss_setup(dev, shared_rss, error)) + goto error_rss_init; + shared_action->type = MLX5_FLOW_ACTION_SHARED_RSS; + return shared_action; +error_rss_init: + rte_free(shared_action); + rte_free(queue); + return NULL; +} + +/** + * Destroy the shared RSS action. + * Release related hash RX queue objects. + * + * @param[in] dev + * Pointer to the Ethernet device structure. + * @param[in] shared_rss + * The shared RSS action object to be removed. + * @param[out] error + * Perform verbose error reporting if not NULL. Initialized in case of + * error only. + * + * @return + * 0 on success, otherwise negative errno value. + */ +static int +__flow_dv_action_rss_release(struct rte_eth_dev *dev, + struct mlx5_shared_action_rss *shared_rss, + struct rte_flow_error *error) +{ + struct rte_flow_shared_action *shared_action = NULL; + int remaining = __flow_dv_action_rss_hrxqs_release(dev, shared_rss); + + if (remaining) { + return rte_flow_error_set(error, ETOOMANYREFS, + RTE_FLOW_ERROR_TYPE_ACTION, + NULL, + "shared rss hrxq has references"); + } + shared_action = container_of(shared_rss, + struct rte_flow_shared_action, rss); + if (!rte_atomic32_dec_and_test(&shared_action->refcnt)) { + return rte_flow_error_set(error, ETOOMANYREFS, + RTE_FLOW_ERROR_TYPE_ACTION, + NULL, + "shared rss has references"); + } + rte_free(shared_rss->queue); + return 0; +} + +/** + * Create shared action, lock free, + * (mutex should be acquired by caller). + * Dispatcher for action type specific call. + * + * @param[in] dev + * Pointer to the Ethernet device structure. + * @param[in] action + * Action specification used to create shared action. + * @param[out] error + * Perform verbose error reporting if not NULL. Initialized in case of + * error only. + * + * @return + * A valid shared action handle in case of success, NULL otherwise and + * rte_errno is set. + */ +static struct rte_flow_shared_action * +__flow_dv_action_create(struct rte_eth_dev *dev, + const struct rte_flow_action *action, + struct rte_flow_error *error) +{ + struct rte_flow_shared_action *shared_action = NULL; + struct mlx5_priv *priv = dev->data->dev_private; + + switch (action->type) { + case RTE_FLOW_ACTION_TYPE_RSS: + shared_action = __flow_dv_action_rss_create(dev, action->conf, + error); + break; + default: + rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ACTION, + NULL, "action type not supported"); + break; + } + if (shared_action) { + rte_atomic32_inc(&shared_action->refcnt); + LIST_INSERT_HEAD(&priv->shared_actions, shared_action, next); + } + return shared_action; +} + +/** + * Destroy the shared action. + * Release action related resources on the NIC and the memory. + * Lock free, (mutex should be acquired by caller). + * Dispatcher for action type specific call. + * + * @param[in] dev + * Pointer to the Ethernet device structure. + * @param[in] action + * The shared action object to be removed. + * @param[out] error + * Perform verbose error reporting if not NULL. Initialized in case of + * error only. + * + * @return + * 0 on success, otherwise negative errno value. + */ +static int +__flow_dv_action_destroy(struct rte_eth_dev *dev, + struct rte_flow_shared_action *action, + struct rte_flow_error *error) +{ + int ret; + + switch (action->type) { + case MLX5_FLOW_ACTION_SHARED_RSS: + ret = __flow_dv_action_rss_release(dev, &action->rss, error); + break; + default: + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION, + NULL, + "action type not supported"); + } + if (ret) + return ret; + LIST_REMOVE(action, next); + rte_free(action); + return 0; +} + +/** + * Updates in place shared RSS action configuration. + * + * @param[in] dev + * Pointer to the Ethernet device structure. + * @param[in] shared_rss + * The shared RSS action object to be updated. + * @param[in] action_conf + * RSS action specification used to modify *shared_rss*. + * @param[out] error + * Perform verbose error reporting if not NULL. Initialized in case of + * error only. + * + * @return + * 0 on success, otherwise negative errno value. + * @note: currently only support update of RSS queues. + */ +static int +__flow_dv_action_rss_update(struct rte_eth_dev *dev, + struct mlx5_shared_action_rss *shared_rss, + const struct rte_flow_action_rss *action_conf, + struct rte_flow_error *error) +{ + size_t i; + int ret; + void *queue = NULL; + uint32_t queue_size; + const uint8_t *rss_key; + uint32_t rss_key_len; + + queue_size = RTE_ALIGN_CEIL(action_conf->queue_num * sizeof(uint16_t), + sizeof(void *)); + queue = rte_calloc(__func__, 1, queue_size, 0); + if (!queue) { + return rte_flow_error_set(error, ENOMEM, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, + "cannot allocate resource memory"); + } + if (action_conf->key) { + rss_key = action_conf->key; + rss_key_len = action_conf->key_len; + } else { + rss_key = rss_hash_default_key; + rss_key_len = MLX5_RSS_HASH_KEY_LEN; + } + for (i = 0; i < MLX5_RSS_HASH_FIELDS_LEN; i++) { + uint32_t hrxq_idx; + uint64_t hash_fields = mlx5_rss_hash_fields[i]; + int tunnel; + + for (tunnel = 0; tunnel < 2; tunnel++) { + hrxq_idx = __flow_dv_action_rss_hrxq_lookup + (shared_rss, hash_fields, tunnel); + MLX5_ASSERT(hrxq_idx); + ret = mlx5_hrxq_modify + (dev, hrxq_idx, + rss_key, rss_key_len, + hash_fields, + action_conf->queue, action_conf->queue_num); + if (ret) { + rte_free(queue); + return rte_flow_error_set + (error, rte_errno, + RTE_FLOW_ERROR_TYPE_ACTION, NULL, + "cannot update hash queue"); + } + } + } + rte_free(shared_rss->queue); + shared_rss->queue = queue; + rte_memcpy(shared_rss->queue, action_conf->queue, queue_size); + shared_rss->origin.queue = shared_rss->queue; + shared_rss->origin.queue_num = action_conf->queue_num; + return 0; +} + +/** + * Updates in place shared action configuration, lock free, + * (mutex should be acquired by caller). + * + * @param[in] dev + * Pointer to the Ethernet device structure. + * @param[in] action + * The shared action object to be updated. + * @param[in] action_conf + * Action specification used to modify *action*. + * *action_conf* should be of type correlating with type of the *action*, + * otherwise considered as invalid. + * @param[out] error + * Perform verbose error reporting if not NULL. Initialized in case of + * error only. + * + * @return + * 0 on success, otherwise negative errno value. + */ +static int +__flow_dv_action_update(struct rte_eth_dev *dev, + struct rte_flow_shared_action *action, + const void *action_conf, + struct rte_flow_error *error) +{ + switch (action->type) { + case MLX5_FLOW_ACTION_SHARED_RSS: + return __flow_dv_action_rss_update(dev, &action->rss, + action_conf, error); + default: + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION, + NULL, + "action type not supported"); + } +} /** * Query a dv flow rule for its statistics via devx. * @@ -10101,6 +10641,87 @@ flow_dv_counter_free(struct rte_eth_dev *dev, uint32_t cnt) flow_dv_shared_unlock(dev); } +/** + * Validate shared action. + * Dispatcher for action type specific validation. + * + * @param[in] dev + * Pointer to the Ethernet device structure. + * @param[in] action + * The shared action object to validate. + * @param[out] error + * Perform verbose error reporting if not NULL. Initialized in case of + * error only. + * + * @return + * 0 on success, otherwise negative errno value. + */ +static int +flow_dv_action_validate(struct rte_eth_dev *dev, + const struct rte_flow_action *action, + struct rte_flow_error *error) +{ + switch (action->type) { + case RTE_FLOW_ACTION_TYPE_RSS: + return mlx5_validate_action_rss(dev, action, error); + default: + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION, + NULL, + "action type not supported"); + } +} + +/* + * Mutex-protected thunk to lock-free __flow_dv_action_create(). + */ +static struct rte_flow_shared_action * +flow_dv_action_create(struct rte_eth_dev *dev, + const struct rte_flow_action *action, + struct rte_flow_error *error) +{ + struct rte_flow_shared_action *shared_action = NULL; + + flow_dv_shared_lock(dev); + shared_action = __flow_dv_action_create(dev, action, error); + flow_dv_shared_unlock(dev); + return shared_action; +} + +/* + * Mutex-protected thunk to lock-free __flow_dv_action_destroy(). + */ +static int +flow_dv_action_destroy(struct rte_eth_dev *dev, + struct rte_flow_shared_action *action, + struct rte_flow_error *error) +{ + int ret; + + flow_dv_shared_lock(dev); + ret = __flow_dv_action_destroy(dev, action, error); + flow_dv_shared_unlock(dev); + return ret; +} + +/* + * Mutex-protected thunk to lock-free __flow_dv_action_update(). + */ +static int +flow_dv_action_update(struct rte_eth_dev *dev, + struct rte_flow_shared_action *action, + const void *action_conf, + struct rte_flow_error *error) +{ + int ret; + + flow_dv_shared_lock(dev); + ret = __flow_dv_action_update(dev, action, action_conf, + error); + flow_dv_shared_unlock(dev); + return ret; +} + const struct mlx5_flow_driver_ops mlx5_flow_dv_drv_ops = { .validate = flow_dv_validate, .prepare = flow_dv_prepare, @@ -10117,6 +10738,10 @@ const struct mlx5_flow_driver_ops mlx5_flow_dv_drv_ops = { .counter_free = flow_dv_counter_free, .counter_query = flow_dv_counter_query, .get_aged_flows = flow_get_aged_flows, + .action_validate = flow_dv_action_validate, + .action_create = flow_dv_action_create, + .action_destroy = flow_dv_action_destroy, + .action_update = flow_dv_action_update, }; #endif /* HAVE_IBV_FLOW_DV_SUPPORT */ -- 2.26.2 ^ permalink raw reply [flat|nested] 106+ messages in thread
* [dpdk-dev] [PATCH v3 08/10] net/mlx5: shared action create conf drv support 2020-10-03 22:06 ` [dpdk-dev] [PATCH v3 00/10] RTE flow shared action Andrey Vesnovaty ` (6 preceding siblings ...) 2020-10-03 22:06 ` [dpdk-dev] [PATCH v3 07/10] net/mlx5: driver support for shared action Andrey Vesnovaty @ 2020-10-03 22:06 ` Andrey Vesnovaty 2020-10-03 22:06 ` [dpdk-dev] [PATCH v3 09/10] examples/flow_filtering: utilize shared RSS action Andrey Vesnovaty ` (2 subsequent siblings) 10 siblings, 0 replies; 106+ messages in thread From: Andrey Vesnovaty @ 2020-10-03 22:06 UTC (permalink / raw) To: dev Cc: jer, jerinjacobk, thomas, ferruh.yigit, stephen, bruce.richardson, orika, viacheslavo, andrey.vesnovaty, mdr, nhorman, ajit.khaparde, samik.gupta, Matan Azrad, Shahaf Shuler, Viacheslav Ovsiienko Add support for configuration argument in shared action validate/create API. Currently the inly shared action implemented is RSS. For shared RSS action configuration argument ignpred but it can be utilized by other types of shared actions. Signed-off-by: Andrey Vesnovaty <andreyv@nvidia.com> --- drivers/net/mlx5/mlx5_flow_dv.c | 21 +++++++++++++++++---- 1 file changed, 17 insertions(+), 4 deletions(-) diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c index 13effc0af0..2c39b061bb 100644 --- a/drivers/net/mlx5/mlx5_flow_dv.c +++ b/drivers/net/mlx5/mlx5_flow_dv.c @@ -9669,6 +9669,8 @@ __flow_dv_action_rss_setup(struct rte_eth_dev *dev, * * @param[in] dev * Pointer to the Ethernet device structure. + * @param[in] conf + * Shared action configuration. * @param[in] rss * RSS action specification used to create shared action. * @param[out] error @@ -9681,8 +9683,9 @@ __flow_dv_action_rss_setup(struct rte_eth_dev *dev, */ static struct rte_flow_shared_action * __flow_dv_action_rss_create(struct rte_eth_dev *dev, - const struct rte_flow_action_rss *rss, - struct rte_flow_error *error) + const struct rte_flow_shared_action_conf *conf, + const struct rte_flow_action_rss *rss, + struct rte_flow_error *error) { struct rte_flow_shared_action *shared_action = NULL; void *queue = NULL; @@ -9691,6 +9694,7 @@ __flow_dv_action_rss_create(struct rte_eth_dev *dev, struct rte_flow_action_rss *origin; const uint8_t *rss_key; + (void)conf; queue_size = RTE_ALIGN_CEIL(rss->queue_num * sizeof(uint16_t), sizeof(void *)); queue = rte_calloc(__func__, 1, queue_size, 0); @@ -9774,6 +9778,8 @@ __flow_dv_action_rss_release(struct rte_eth_dev *dev, * * @param[in] dev * Pointer to the Ethernet device structure. + * @param[in] conf + * Shared action configuration. * @param[in] action * Action specification used to create shared action. * @param[out] error @@ -9786,6 +9792,7 @@ __flow_dv_action_rss_release(struct rte_eth_dev *dev, */ static struct rte_flow_shared_action * __flow_dv_action_create(struct rte_eth_dev *dev, + const struct rte_flow_shared_action_conf *conf, const struct rte_flow_action *action, struct rte_flow_error *error) { @@ -9794,7 +9801,8 @@ __flow_dv_action_create(struct rte_eth_dev *dev, switch (action->type) { case RTE_FLOW_ACTION_TYPE_RSS: - shared_action = __flow_dv_action_rss_create(dev, action->conf, + shared_action = __flow_dv_action_rss_create(dev, conf, + action->conf, error); break; default: @@ -10647,6 +10655,8 @@ flow_dv_counter_free(struct rte_eth_dev *dev, uint32_t cnt) * * @param[in] dev * Pointer to the Ethernet device structure. + * @param[in] conf + * Shared action configuration. * @param[in] action * The shared action object to validate. * @param[out] error @@ -10658,9 +10668,11 @@ flow_dv_counter_free(struct rte_eth_dev *dev, uint32_t cnt) */ static int flow_dv_action_validate(struct rte_eth_dev *dev, + const struct rte_flow_shared_action_conf *conf, const struct rte_flow_action *action, struct rte_flow_error *error) { + (void)conf; switch (action->type) { case RTE_FLOW_ACTION_TYPE_RSS: return mlx5_validate_action_rss(dev, action, error); @@ -10677,13 +10689,14 @@ flow_dv_action_validate(struct rte_eth_dev *dev, */ static struct rte_flow_shared_action * flow_dv_action_create(struct rte_eth_dev *dev, + const struct rte_flow_shared_action_conf *conf, const struct rte_flow_action *action, struct rte_flow_error *error) { struct rte_flow_shared_action *shared_action = NULL; flow_dv_shared_lock(dev); - shared_action = __flow_dv_action_create(dev, action, error); + shared_action = __flow_dv_action_create(dev, conf, action, error); flow_dv_shared_unlock(dev); return shared_action; } -- 2.26.2 ^ permalink raw reply [flat|nested] 106+ messages in thread
* [dpdk-dev] [PATCH v3 09/10] examples/flow_filtering: utilize shared RSS action 2020-10-03 22:06 ` [dpdk-dev] [PATCH v3 00/10] RTE flow shared action Andrey Vesnovaty ` (7 preceding siblings ...) 2020-10-03 22:06 ` [dpdk-dev] [PATCH v3 08/10] net/mlx5: shared action create conf drv support Andrey Vesnovaty @ 2020-10-03 22:06 ` Andrey Vesnovaty 2020-10-04 11:21 ` Ori Kam 2020-10-03 22:06 ` [dpdk-dev] [PATCH v3 10/10] app/testpmd: support shared action Andrey Vesnovaty 2020-10-04 11:14 ` [dpdk-dev] [PATCH v3 00/10] RTE flow " Ori Kam 10 siblings, 1 reply; 106+ messages in thread From: Andrey Vesnovaty @ 2020-10-03 22:06 UTC (permalink / raw) To: dev Cc: jer, jerinjacobk, thomas, ferruh.yigit, stephen, bruce.richardson, orika, viacheslavo, andrey.vesnovaty, mdr, nhorman, ajit.khaparde, samik.gupta, Andrey Vesnovaty, Marko Kovacevic, Ori Kam, Radu Nicolau, Akhil Goyal, Tomasz Kantecki, Sunil Kumar Kori, Pavan Nikhilesh, John McNamara From: Andrey Vesnovaty <andreyv@mellanox.com> This commit give very first shared RSS action usage example. Queue action used by the flow replaced by shared RSS action having single queue. On each RX burst queue switched 0 <-> 1 utilizing rte_flow_shared_action_update() API. User supposed to observe consistent queue switches on each packet burst. Signed-off-by: Andrey Vesnovaty <andreyv@mellanox.com> Signed-off-by: Andrey Vesnovaty <andreyv@nvidia.com> --- doc/guides/sample_app_ug/flow_filtering.rst | 62 +++++++++++++--- examples/flow_filtering/flow_blocks.c | 81 +++++++++++++++++---- examples/flow_filtering/main.c | 13 +++- 3 files changed, 128 insertions(+), 28 deletions(-) diff --git a/doc/guides/sample_app_ug/flow_filtering.rst b/doc/guides/sample_app_ug/flow_filtering.rst index 5e5a6cd8a0..cfe9334717 100644 --- a/doc/guides/sample_app_ug/flow_filtering.rst +++ b/doc/guides/sample_app_ug/flow_filtering.rst @@ -106,7 +106,7 @@ following code: .. code-block:: c /* create flow for send packet with */ - flow = generate_ipv4_flow(port_id, selected_queue, + flow = generate_ipv4_flow(port_id, shared_action, SRC_IP, EMPTY_MASK, DEST_IP, FULL_MASK, &error); if (!flow) { @@ -242,7 +242,7 @@ The Ethernet port is configured with default settings using the rxq_conf = dev_info.default_rxconf; rxq_conf.offloads = port_conf.rxmode.offloads; -For this example we are configuring number of rx and tx queues that are connected +For this example we are configuring 2 rx and 2 tx queues that are connected to a single port. .. code-block:: c @@ -270,13 +270,22 @@ to a single port. } } +Before we create the flow we create shared action in order to send it as +actions argument when creating a flow. The action is single queue RSS action +similar to action queue with the only difference that shared RSS action +provides update capability after action creation. + +.. code-block:: c + + shared_action = rte_flow_shared_action_create(port_id, &action, &error); + In the next step we create and apply the flow rule. which is to send packets with destination ip equals to 192.168.1.1 to queue number 1. The detail explanation of the ``generate_ipv4_flow()`` appears later in this document: .. code-block:: c - flow = generate_ipv4_flow(port_id, selected_queue, + flow = generate_ipv4_flow(port_id, shared_action, SRC_IP, EMPTY_MASK, DEST_IP, FULL_MASK, &error); @@ -339,6 +348,21 @@ looks like the following: printf("\n"); rte_pktmbuf_free(m); } + if (rss_queue[0] == 0) { + printf(">>> switching queue 0 -> 1\n"); + rss_queue[0] = 1; + } else { + printf(">>> switching queue 1 -> 0\n"); + rss_queue[0] = 0; + } + ret = rte_flow_shared_action_update + (port_id, shared_action, &action, + &error); + if (ret) + rte_exit(EXIT_FAILURE, + ":: error: RSS action update " + "failed: %s\n", + rte_strerror(-ret)); } } } @@ -348,6 +372,8 @@ looks like the following: rte_eth_dev_close(port_id); } +On each loop eteration Rx queue switched using +``rte_flow_shared_action_update()`` API. The main work of the application is reading the packets from all queues and printing for each packet the destination queue: @@ -365,6 +391,21 @@ queues and printing for each packet the destination queue: printf(" - queue=0x%x", (unsigned int)i); printf("\n"); rte_pktmbuf_free(m); + if (rss_queue[0] == 0) { + printf(">>> switching queue 0 -> 1\n"); + rss_queue[0] = 1; + } else { + printf(">>> switching queue 1 -> 0\n"); + rss_queue[0] = 0; + } + ret = rte_flow_shared_action_update + (port_id, shared_action, &action, + &error); + if (ret) + rte_exit(EXIT_FAILURE, + ":: error: RSS action update " + "failed: %s\n", + rte_strerror(-ret)); } } } @@ -378,13 +419,15 @@ The forwarding loop can be interrupted and the application closed using The generate_ipv4_flow function ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + The generate_ipv4_flow function is responsible for creating the flow rule. This function is located in the ``flow_blocks.c`` file. .. code-block:: c static struct rte_flow * - generate_ipv4_flow(uint8_t port_id, uint16_t rx_q, + generate_ipv4_flow(uint8_t port_id, + cstructrte_flow_shared_action *shared_action, uint32_t src_ip, uint32_t src_mask, uint32_t dest_ip, uint32_t dest_mask, struct rte_flow_error *error) @@ -393,7 +436,6 @@ This function is located in the ``flow_blocks.c`` file. struct rte_flow_item pattern[MAX_PATTERN_NUM]; struct rte_flow_action action[MAX_ACTION_NUM]; struct rte_flow *flow = NULL; - struct rte_flow_action_queue queue = { .index = rx_q }; struct rte_flow_item_ipv4 ip_spec; struct rte_flow_item_ipv4 ip_mask; @@ -411,8 +453,8 @@ This function is located in the ``flow_blocks.c`` file. * create the action sequence. * one action only, move packet to queue */ - action[0].type = RTE_FLOW_ACTION_TYPE_QUEUE; - action[0].conf = &queue; + action[0].type = RTE_FLOW_ACTION_TYPE_SHARED; + action[0].conf = shared_action; action[1].type = RTE_FLOW_ACTION_TYPE_END; /* @@ -468,12 +510,12 @@ The following part create the flow attributes, in our case ingress. attr.ingress = 1; The third part defines the action to be taken when a packet matches -the rule. In this case send the packet to queue. +the rule. In this case send the packet to single RSS queue. .. code-block:: c - action[0].type = RTE_FLOW_ACTION_TYPE_QUEUE; - action[0].conf = &queue; + action[0].type = RTE_FLOW_ACTION_TYPE_SHARED; + action[0].conf = shared_action; action[1].type = RTE_FLOW_ACTION_TYPE_END; The fourth part is responsible for creating the pattern and is built from diff --git a/examples/flow_filtering/flow_blocks.c b/examples/flow_filtering/flow_blocks.c index 575d792810..ed17ae073e 100644 --- a/examples/flow_filtering/flow_blocks.c +++ b/examples/flow_filtering/flow_blocks.c @@ -5,12 +5,30 @@ #define MAX_PATTERN_NUM 3 #define MAX_ACTION_NUM 2 -struct rte_flow * -generate_ipv4_flow(uint16_t port_id, uint16_t rx_q, - uint32_t src_ip, uint32_t src_mask, - uint32_t dest_ip, uint32_t dest_mask, - struct rte_flow_error *error); +struct rte_flow_shared_action *shared_action; +uint16_t rss_queue[1] = {0}; + +struct rte_flow_action_rss rss_action = { + .func = RTE_ETH_HASH_FUNCTION_DEFAULT, + .level = 0, + .types = 0, + .key_len = 0, + .key = NULL, + .queue = rss_queue, + .queue_num = 1, +}; +struct rte_flow_action flow_action = { + .type = RTE_FLOW_ACTION_TYPE_RSS, + .conf = &rss_action, +}; + +struct rte_flow * +generate_ipv4_flow(uint16_t port_id, + uint32_t src_ip, uint32_t src_mask, + uint32_t dest_ip, uint32_t dest_mask, + struct rte_flow_error *error); +int update_rss_action(uint16_t port_id, struct rte_flow_error *error); /** * create a flow rule that sends packets with matching src and dest ip @@ -18,8 +36,6 @@ generate_ipv4_flow(uint16_t port_id, uint16_t rx_q, * * @param port_id * The selected port. - * @param rx_q - * The selected target queue. * @param src_ip * The src ip value to match the input packet. * @param src_mask @@ -35,16 +51,15 @@ generate_ipv4_flow(uint16_t port_id, uint16_t rx_q, * A flow if the rule could be created else return NULL. */ struct rte_flow * -generate_ipv4_flow(uint16_t port_id, uint16_t rx_q, - uint32_t src_ip, uint32_t src_mask, - uint32_t dest_ip, uint32_t dest_mask, - struct rte_flow_error *error) +generate_ipv4_flow(uint16_t port_id, + uint32_t src_ip, uint32_t src_mask, + uint32_t dest_ip, uint32_t dest_mask, + struct rte_flow_error *error) { struct rte_flow_attr attr; struct rte_flow_item pattern[MAX_PATTERN_NUM]; struct rte_flow_action action[MAX_ACTION_NUM]; struct rte_flow *flow = NULL; - struct rte_flow_action_queue queue = { .index = rx_q }; struct rte_flow_item_ipv4 ip_spec; struct rte_flow_item_ipv4 ip_mask; int res; @@ -61,10 +76,19 @@ generate_ipv4_flow(uint16_t port_id, uint16_t rx_q, /* * create the action sequence. - * one action only, move packet to queue + * one action only, move packet to shared RSS queue */ - action[0].type = RTE_FLOW_ACTION_TYPE_QUEUE; - action[0].conf = &queue; + shared_action = rte_flow_shared_action_create(port_id, NULL, + &flow_action, error); + if (shared_action) { + action[0].conf = shared_action; + action[0].type = RTE_FLOW_ACTION_TYPE_SHARED; + } else { + if (rte_errno != ENOSYS) + rte_exit(EXIT_FAILURE, ":: action creation failure\n"); + printf("PMD doesn't support shared actions.\n"); + action[0] = flow_action; + } action[1].type = RTE_FLOW_ACTION_TYPE_END; /* @@ -98,3 +122,30 @@ generate_ipv4_flow(uint16_t port_id, uint16_t rx_q, return flow; } + +/** + * updates shared RSS action if flow created with shared action + * + * @param port_id + * The selected port. + * @param[out] error + * Perform verbose error reporting if not NULL. + * + * @return + * 0 on sccess + */ +int +update_rss_action(uint16_t port_id, struct rte_flow_error *error) +{ + if (!shared_action) + return 0; + if (rss_queue[0] == 0) { + printf(">>> switching queue 0 -> 1\n"); + rss_queue[0] = 1; + } else { + printf(">>> switching queue 1 -> 0\n"); + rss_queue[0] = 0; + } + return rte_flow_shared_action_update(port_id, shared_action, + &flow_action, error); +} diff --git a/examples/flow_filtering/main.c b/examples/flow_filtering/main.c index cc9e7e7808..9ef7f099d9 100644 --- a/examples/flow_filtering/main.c +++ b/examples/flow_filtering/main.c @@ -32,8 +32,7 @@ static volatile bool force_quit; static uint16_t port_id; -static uint16_t nr_queues = 5; -static uint8_t selected_queue = 1; +static uint16_t nr_queues = 2; struct rte_mempool *mbuf_pool; struct rte_flow *flow; @@ -61,6 +60,7 @@ main_loop(void) uint16_t nb_rx; uint16_t i; uint16_t j; + int ret; while (!force_quit) { for (i = 0; i < nr_queues; i++) { @@ -82,6 +82,12 @@ main_loop(void) rte_pktmbuf_free(m); } + ret = update_rss_action(port_id, &error); + if (ret) + rte_exit(EXIT_FAILURE, + ":: error: RSS action update " + "failed: %s\n", + rte_strerror(-ret)); } } } @@ -243,8 +249,9 @@ main(int argc, char **argv) init_port(); + /* create flow for send packet with */ - flow = generate_ipv4_flow(port_id, selected_queue, + flow = generate_ipv4_flow(port_id, SRC_IP, EMPTY_MASK, DEST_IP, FULL_MASK, &error); if (!flow) { -- 2.26.2 ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [dpdk-dev] [PATCH v3 09/10] examples/flow_filtering: utilize shared RSS action 2020-10-03 22:06 ` [dpdk-dev] [PATCH v3 09/10] examples/flow_filtering: utilize shared RSS action Andrey Vesnovaty @ 2020-10-04 11:21 ` Ori Kam 2020-10-06 10:34 ` Andrey Vesnovaty 0 siblings, 1 reply; 106+ messages in thread From: Ori Kam @ 2020-10-04 11:21 UTC (permalink / raw) To: Andrey Vesnovaty, dev Cc: jer, jerinjacobk, NBU-Contact-Thomas Monjalon, ferruh.yigit, stephen, bruce.richardson, Slava Ovsiienko, andrey.vesnovaty, mdr, nhorman, ajit.khaparde, samik.gupta, Andrey Vesnovaty, Marko Kovacevic, Ori Kam, Radu Nicolau, Akhil Goyal, Tomasz Kantecki, Sunil Kumar Kori, Pavan Nikhilesh, John McNamara Hi Andrey, I don't think this change should be part of the series. After you update this example will only work if the PMD support shared action. Not all PMD supports it, and in any case this example is to show basic rules and RSS your code changes the app that the RSS action can't be checked. Best, Ori > -----Original Message----- > From: Andrey Vesnovaty <andreyv@nvidia.com> > Sent: Sunday, October 4, 2020 1:06 AM > Subject: [PATCH v3 09/10] examples/flow_filtering: utilize shared RSS action > > From: Andrey Vesnovaty <andreyv@mellanox.com> > > This commit give very first shared RSS action usage example. > Queue action used by the flow replaced by shared RSS action > having single queue. On each RX burst queue switched 0 <-> 1 > utilizing rte_flow_shared_action_update() API. User supposed > to observe consistent queue switches on each packet burst. > > Signed-off-by: Andrey Vesnovaty <andreyv@mellanox.com> > Signed-off-by: Andrey Vesnovaty <andreyv@nvidia.com> > --- > doc/guides/sample_app_ug/flow_filtering.rst | 62 +++++++++++++--- > examples/flow_filtering/flow_blocks.c | 81 +++++++++++++++++---- > examples/flow_filtering/main.c | 13 +++- > 3 files changed, 128 insertions(+), 28 deletions(-) > > diff --git a/doc/guides/sample_app_ug/flow_filtering.rst > b/doc/guides/sample_app_ug/flow_filtering.rst > index 5e5a6cd8a0..cfe9334717 100644 > --- a/doc/guides/sample_app_ug/flow_filtering.rst > +++ b/doc/guides/sample_app_ug/flow_filtering.rst > @@ -106,7 +106,7 @@ following code: > .. code-block:: c > > /* create flow for send packet with */ > - flow = generate_ipv4_flow(port_id, selected_queue, > + flow = generate_ipv4_flow(port_id, shared_action, > SRC_IP, EMPTY_MASK, > DEST_IP, FULL_MASK, &error); > if (!flow) { > @@ -242,7 +242,7 @@ The Ethernet port is configured with default settings > using the > rxq_conf = dev_info.default_rxconf; > rxq_conf.offloads = port_conf.rxmode.offloads; > > -For this example we are configuring number of rx and tx queues that are > connected > +For this example we are configuring 2 rx and 2 tx queues that are connected > to a single port. > > .. code-block:: c > @@ -270,13 +270,22 @@ to a single port. > } > } > > +Before we create the flow we create shared action in order to send it as > +actions argument when creating a flow. The action is single queue RSS action > +similar to action queue with the only difference that shared RSS action > +provides update capability after action creation. > + > +.. code-block:: c > + > + shared_action = rte_flow_shared_action_create(port_id, &action, &error); > + > In the next step we create and apply the flow rule. which is to send packets > with destination ip equals to 192.168.1.1 to queue number 1. The detail > explanation of the ``generate_ipv4_flow()`` appears later in this document: > > .. code-block:: c > > - flow = generate_ipv4_flow(port_id, selected_queue, > + flow = generate_ipv4_flow(port_id, shared_action, > SRC_IP, EMPTY_MASK, > DEST_IP, FULL_MASK, &error); > > @@ -339,6 +348,21 @@ looks like the following: > printf("\n"); > rte_pktmbuf_free(m); > } > + if (rss_queue[0] == 0) { > + printf(">>> switching queue 0 -> 1\n"); > + rss_queue[0] = 1; > + } else { > + printf(">>> switching queue 1 -> 0\n"); > + rss_queue[0] = 0; > + } > + ret = rte_flow_shared_action_update > + (port_id, shared_action, &action, > + &error); > + if (ret) > + rte_exit(EXIT_FAILURE, > + ":: error: RSS action update " > + "failed: %s\n", > + rte_strerror(-ret)); > } > } > } > @@ -348,6 +372,8 @@ looks like the following: > rte_eth_dev_close(port_id); > } > > +On each loop eteration Rx queue switched using > +``rte_flow_shared_action_update()`` API. > The main work of the application is reading the packets from all > queues and printing for each packet the destination queue: > > @@ -365,6 +391,21 @@ queues and printing for each packet the destination > queue: > printf(" - queue=0x%x", (unsigned int)i); > printf("\n"); > rte_pktmbuf_free(m); > + if (rss_queue[0] == 0) { > + printf(">>> switching queue 0 -> 1\n"); > + rss_queue[0] = 1; > + } else { > + printf(">>> switching queue 1 -> 0\n"); > + rss_queue[0] = 0; > + } > + ret = rte_flow_shared_action_update > + (port_id, shared_action, &action, > + &error); > + if (ret) > + rte_exit(EXIT_FAILURE, > + ":: error: RSS action update " > + "failed: %s\n", > + rte_strerror(-ret)); > } > } > } > @@ -378,13 +419,15 @@ The forwarding loop can be interrupted and the > application closed using > The generate_ipv4_flow function > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > + > The generate_ipv4_flow function is responsible for creating the flow rule. > This function is located in the ``flow_blocks.c`` file. > > .. code-block:: c > > static struct rte_flow * > - generate_ipv4_flow(uint8_t port_id, uint16_t rx_q, > + generate_ipv4_flow(uint8_t port_id, > + cstructrte_flow_shared_action *shared_action, > uint32_t src_ip, uint32_t src_mask, > uint32_t dest_ip, uint32_t dest_mask, > struct rte_flow_error *error) > @@ -393,7 +436,6 @@ This function is located in the ``flow_blocks.c`` file. > struct rte_flow_item pattern[MAX_PATTERN_NUM]; > struct rte_flow_action action[MAX_ACTION_NUM]; > struct rte_flow *flow = NULL; > - struct rte_flow_action_queue queue = { .index = rx_q }; > struct rte_flow_item_ipv4 ip_spec; > struct rte_flow_item_ipv4 ip_mask; > > @@ -411,8 +453,8 @@ This function is located in the ``flow_blocks.c`` file. > * create the action sequence. > * one action only, move packet to queue > */ > - action[0].type = RTE_FLOW_ACTION_TYPE_QUEUE; > - action[0].conf = &queue; > + action[0].type = RTE_FLOW_ACTION_TYPE_SHARED; > + action[0].conf = shared_action; > action[1].type = RTE_FLOW_ACTION_TYPE_END; > > /* > @@ -468,12 +510,12 @@ The following part create the flow attributes, in our > case ingress. > attr.ingress = 1; > > The third part defines the action to be taken when a packet matches > -the rule. In this case send the packet to queue. > +the rule. In this case send the packet to single RSS queue. > > .. code-block:: c > > - action[0].type = RTE_FLOW_ACTION_TYPE_QUEUE; > - action[0].conf = &queue; > + action[0].type = RTE_FLOW_ACTION_TYPE_SHARED; > + action[0].conf = shared_action; > action[1].type = RTE_FLOW_ACTION_TYPE_END; > > The fourth part is responsible for creating the pattern and is built from > diff --git a/examples/flow_filtering/flow_blocks.c > b/examples/flow_filtering/flow_blocks.c > index 575d792810..ed17ae073e 100644 > --- a/examples/flow_filtering/flow_blocks.c > +++ b/examples/flow_filtering/flow_blocks.c > @@ -5,12 +5,30 @@ > #define MAX_PATTERN_NUM 3 > #define MAX_ACTION_NUM 2 > > -struct rte_flow * > -generate_ipv4_flow(uint16_t port_id, uint16_t rx_q, > - uint32_t src_ip, uint32_t src_mask, > - uint32_t dest_ip, uint32_t dest_mask, > - struct rte_flow_error *error); > +struct rte_flow_shared_action *shared_action; > +uint16_t rss_queue[1] = {0}; > + > +struct rte_flow_action_rss rss_action = { > + .func = RTE_ETH_HASH_FUNCTION_DEFAULT, > + .level = 0, > + .types = 0, > + .key_len = 0, > + .key = NULL, > + .queue = rss_queue, > + .queue_num = 1, > +}; > > +struct rte_flow_action flow_action = { > + .type = RTE_FLOW_ACTION_TYPE_RSS, > + .conf = &rss_action, > +}; > + > +struct rte_flow * > +generate_ipv4_flow(uint16_t port_id, > + uint32_t src_ip, uint32_t src_mask, > + uint32_t dest_ip, uint32_t dest_mask, > + struct rte_flow_error *error); > +int update_rss_action(uint16_t port_id, struct rte_flow_error *error); > > /** > * create a flow rule that sends packets with matching src and dest ip > @@ -18,8 +36,6 @@ generate_ipv4_flow(uint16_t port_id, uint16_t rx_q, > * > * @param port_id > * The selected port. > - * @param rx_q > - * The selected target queue. > * @param src_ip > * The src ip value to match the input packet. > * @param src_mask > @@ -35,16 +51,15 @@ generate_ipv4_flow(uint16_t port_id, uint16_t rx_q, > * A flow if the rule could be created else return NULL. > */ > struct rte_flow * > -generate_ipv4_flow(uint16_t port_id, uint16_t rx_q, > - uint32_t src_ip, uint32_t src_mask, > - uint32_t dest_ip, uint32_t dest_mask, > - struct rte_flow_error *error) > +generate_ipv4_flow(uint16_t port_id, > + uint32_t src_ip, uint32_t src_mask, > + uint32_t dest_ip, uint32_t dest_mask, > + struct rte_flow_error *error) > { > struct rte_flow_attr attr; > struct rte_flow_item pattern[MAX_PATTERN_NUM]; > struct rte_flow_action action[MAX_ACTION_NUM]; > struct rte_flow *flow = NULL; > - struct rte_flow_action_queue queue = { .index = rx_q }; > struct rte_flow_item_ipv4 ip_spec; > struct rte_flow_item_ipv4 ip_mask; > int res; > @@ -61,10 +76,19 @@ generate_ipv4_flow(uint16_t port_id, uint16_t rx_q, > > /* > * create the action sequence. > - * one action only, move packet to queue > + * one action only, move packet to shared RSS queue > */ > - action[0].type = RTE_FLOW_ACTION_TYPE_QUEUE; > - action[0].conf = &queue; > + shared_action = rte_flow_shared_action_create(port_id, NULL, > + &flow_action, error); > + if (shared_action) { > + action[0].conf = shared_action; > + action[0].type = RTE_FLOW_ACTION_TYPE_SHARED; > + } else { > + if (rte_errno != ENOSYS) > + rte_exit(EXIT_FAILURE, ":: action creation failure\n"); > + printf("PMD doesn't support shared actions.\n"); > + action[0] = flow_action; > + } > action[1].type = RTE_FLOW_ACTION_TYPE_END; > > /* > @@ -98,3 +122,30 @@ generate_ipv4_flow(uint16_t port_id, uint16_t rx_q, > > return flow; > } > + > +/** > + * updates shared RSS action if flow created with shared action > + * > + * @param port_id > + * The selected port. > + * @param[out] error > + * Perform verbose error reporting if not NULL. > + * > + * @return > + * 0 on sccess > + */ > +int > +update_rss_action(uint16_t port_id, struct rte_flow_error *error) > +{ > + if (!shared_action) > + return 0; > + if (rss_queue[0] == 0) { > + printf(">>> switching queue 0 -> 1\n"); > + rss_queue[0] = 1; > + } else { > + printf(">>> switching queue 1 -> 0\n"); > + rss_queue[0] = 0; > + } > + return rte_flow_shared_action_update(port_id, shared_action, > + &flow_action, error); > +} > diff --git a/examples/flow_filtering/main.c b/examples/flow_filtering/main.c > index cc9e7e7808..9ef7f099d9 100644 > --- a/examples/flow_filtering/main.c > +++ b/examples/flow_filtering/main.c > @@ -32,8 +32,7 @@ > static volatile bool force_quit; > > static uint16_t port_id; > -static uint16_t nr_queues = 5; > -static uint8_t selected_queue = 1; > +static uint16_t nr_queues = 2; > struct rte_mempool *mbuf_pool; > struct rte_flow *flow; > > @@ -61,6 +60,7 @@ main_loop(void) > uint16_t nb_rx; > uint16_t i; > uint16_t j; > + int ret; > > while (!force_quit) { > for (i = 0; i < nr_queues; i++) { > @@ -82,6 +82,12 @@ main_loop(void) > > rte_pktmbuf_free(m); > } > + ret = update_rss_action(port_id, &error); > + if (ret) > + rte_exit(EXIT_FAILURE, > + ":: error: RSS action update " > + "failed: %s\n", > + rte_strerror(-ret)); > } > } > } > @@ -243,8 +249,9 @@ main(int argc, char **argv) > > init_port(); > > + > /* create flow for send packet with */ > - flow = generate_ipv4_flow(port_id, selected_queue, > + flow = generate_ipv4_flow(port_id, > SRC_IP, EMPTY_MASK, > DEST_IP, FULL_MASK, &error); > if (!flow) { > -- > 2.26.2 ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [dpdk-dev] [PATCH v3 09/10] examples/flow_filtering: utilize shared RSS action 2020-10-04 11:21 ` Ori Kam @ 2020-10-06 10:34 ` Andrey Vesnovaty 0 siblings, 0 replies; 106+ messages in thread From: Andrey Vesnovaty @ 2020-10-06 10:34 UTC (permalink / raw) To: Ori Kam, dev Cc: jer, jerinjacobk, NBU-Contact-Thomas Monjalon, ferruh.yigit, stephen, bruce.richardson, Slava Ovsiienko, andrey.vesnovaty, mdr, nhorman, ajit.khaparde, samik.gupta, Andrey Vesnovaty, Marko Kovacevic, Ori Kam, Radu Nicolau, Akhil Goyal, Tomasz Kantecki, Sunil Kumar Kori, Pavan Nikhilesh, John McNamara Hi Ori, PSB Thanks, Andrey > -----Original Message----- > From: Ori Kam <orika@nvidia.com> > Sent: Sunday, October 4, 2020 2:22 PM > To: Andrey Vesnovaty <andreyv@nvidia.com>; dev@dpdk.org > Cc: jer@marvell.com; jerinjacobk@gmail.com; NBU-Contact-Thomas Monjalon > <thomas@monjalon.net>; ferruh.yigit@intel.com; > stephen@networkplumber.org; bruce.richardson@intel.com; Slava Ovsiienko > <viacheslavo@nvidia.com>; andrey.vesnovaty@gmail.com; mdr@ashroe.eu; > nhorman@tuxdriver.com; ajit.khaparde@broadcom.com; > samik.gupta@broadcom.com; Andrey Vesnovaty <andreyv@mellanox.com>; > Marko Kovacevic <marko.kovacevic@intel.com>; Ori Kam > <orika@mellanox.com>; Radu Nicolau <radu.nicolau@intel.com>; Akhil Goyal > <akhil.goyal@nxp.com>; Tomasz Kantecki <tomasz.kantecki@intel.com>; Sunil > Kumar Kori <skori@marvell.com>; Pavan Nikhilesh > <pbhagavatula@marvell.com>; John McNamara <john.mcnamara@intel.com> > Subject: RE: [PATCH v3 09/10] examples/flow_filtering: utilize shared RSS action > > Hi Andrey, > > I don't think this change should be part of the series. Now there is PMD support for shared action & having some example isn't critical For this patch series. Removing it. > After you update this example will only work if the PMD support shared action. > Not all PMD supports it, and in any case this example is to show basic rules and > RSS > your code changes the app that the RSS action can't be checked. There is a fallback for PMDs without shared action support but I that agree, it would be better to provide separate example for shared RSS. > > Best, > Ori > > > > -----Original Message----- > > From: Andrey Vesnovaty <andreyv@nvidia.com> > > Sent: Sunday, October 4, 2020 1:06 AM > > Subject: [PATCH v3 09/10] examples/flow_filtering: utilize shared RSS action > > > > From: Andrey Vesnovaty <andreyv@mellanox.com> > > > > This commit give very first shared RSS action usage example. > > Queue action used by the flow replaced by shared RSS action > > having single queue. On each RX burst queue switched 0 <-> 1 > > utilizing rte_flow_shared_action_update() API. User supposed > > to observe consistent queue switches on each packet burst. > > > > Signed-off-by: Andrey Vesnovaty <andreyv@mellanox.com> > > Signed-off-by: Andrey Vesnovaty <andreyv@nvidia.com> > > --- > > doc/guides/sample_app_ug/flow_filtering.rst | 62 +++++++++++++--- > > examples/flow_filtering/flow_blocks.c | 81 +++++++++++++++++---- > > examples/flow_filtering/main.c | 13 +++- > > 3 files changed, 128 insertions(+), 28 deletions(-) > > > > diff --git a/doc/guides/sample_app_ug/flow_filtering.rst > > b/doc/guides/sample_app_ug/flow_filtering.rst > > index 5e5a6cd8a0..cfe9334717 100644 > > --- a/doc/guides/sample_app_ug/flow_filtering.rst > > +++ b/doc/guides/sample_app_ug/flow_filtering.rst > > @@ -106,7 +106,7 @@ following code: > > .. code-block:: c > > > > /* create flow for send packet with */ > > - flow = generate_ipv4_flow(port_id, selected_queue, > > + flow = generate_ipv4_flow(port_id, shared_action, > > SRC_IP, EMPTY_MASK, > > DEST_IP, FULL_MASK, &error); > > if (!flow) { > > @@ -242,7 +242,7 @@ The Ethernet port is configured with default settings > > using the > > rxq_conf = dev_info.default_rxconf; > > rxq_conf.offloads = port_conf.rxmode.offloads; > > > > -For this example we are configuring number of rx and tx queues that are > > connected > > +For this example we are configuring 2 rx and 2 tx queues that are connected > > to a single port. > > > > .. code-block:: c > > @@ -270,13 +270,22 @@ to a single port. > > } > > } > > > > +Before we create the flow we create shared action in order to send it as > > +actions argument when creating a flow. The action is single queue RSS action > > +similar to action queue with the only difference that shared RSS action > > +provides update capability after action creation. > > + > > +.. code-block:: c > > + > > + shared_action = rte_flow_shared_action_create(port_id, &action, &error); > > + > > In the next step we create and apply the flow rule. which is to send packets > > with destination ip equals to 192.168.1.1 to queue number 1. The detail > > explanation of the ``generate_ipv4_flow()`` appears later in this document: > > > > .. code-block:: c > > > > - flow = generate_ipv4_flow(port_id, selected_queue, > > + flow = generate_ipv4_flow(port_id, shared_action, > > SRC_IP, EMPTY_MASK, > > DEST_IP, FULL_MASK, &error); > > > > @@ -339,6 +348,21 @@ looks like the following: > > printf("\n"); > > rte_pktmbuf_free(m); > > } > > + if (rss_queue[0] == 0) { > > + printf(">>> switching queue 0 -> 1\n"); > > + rss_queue[0] = 1; > > + } else { > > + printf(">>> switching queue 1 -> 0\n"); > > + rss_queue[0] = 0; > > + } > > + ret = rte_flow_shared_action_update > > + (port_id, shared_action, &action, > > + &error); > > + if (ret) > > + rte_exit(EXIT_FAILURE, > > + ":: error: RSS action update " > > + "failed: %s\n", > > + rte_strerror(-ret)); > > } > > } > > } > > @@ -348,6 +372,8 @@ looks like the following: > > rte_eth_dev_close(port_id); > > } > > > > +On each loop eteration Rx queue switched using > > +``rte_flow_shared_action_update()`` API. > > The main work of the application is reading the packets from all > > queues and printing for each packet the destination queue: > > > > @@ -365,6 +391,21 @@ queues and printing for each packet the destination > > queue: > > printf(" - queue=0x%x", (unsigned int)i); > > printf("\n"); > > rte_pktmbuf_free(m); > > + if (rss_queue[0] == 0) { > > + printf(">>> switching queue 0 -> 1\n"); > > + rss_queue[0] = 1; > > + } else { > > + printf(">>> switching queue 1 -> 0\n"); > > + rss_queue[0] = 0; > > + } > > + ret = rte_flow_shared_action_update > > + (port_id, shared_action, &action, > > + &error); > > + if (ret) > > + rte_exit(EXIT_FAILURE, > > + ":: error: RSS action update " > > + "failed: %s\n", > > + rte_strerror(-ret)); > > } > > } > > } > > @@ -378,13 +419,15 @@ The forwarding loop can be interrupted and the > > application closed using > > The generate_ipv4_flow function > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > > > + > > The generate_ipv4_flow function is responsible for creating the flow rule. > > This function is located in the ``flow_blocks.c`` file. > > > > .. code-block:: c > > > > static struct rte_flow * > > - generate_ipv4_flow(uint8_t port_id, uint16_t rx_q, > > + generate_ipv4_flow(uint8_t port_id, > > + cstructrte_flow_shared_action *shared_action, > > uint32_t src_ip, uint32_t src_mask, > > uint32_t dest_ip, uint32_t dest_mask, > > struct rte_flow_error *error) > > @@ -393,7 +436,6 @@ This function is located in the ``flow_blocks.c`` file. > > struct rte_flow_item pattern[MAX_PATTERN_NUM]; > > struct rte_flow_action action[MAX_ACTION_NUM]; > > struct rte_flow *flow = NULL; > > - struct rte_flow_action_queue queue = { .index = rx_q }; > > struct rte_flow_item_ipv4 ip_spec; > > struct rte_flow_item_ipv4 ip_mask; > > > > @@ -411,8 +453,8 @@ This function is located in the ``flow_blocks.c`` file. > > * create the action sequence. > > * one action only, move packet to queue > > */ > > - action[0].type = RTE_FLOW_ACTION_TYPE_QUEUE; > > - action[0].conf = &queue; > > + action[0].type = RTE_FLOW_ACTION_TYPE_SHARED; > > + action[0].conf = shared_action; > > action[1].type = RTE_FLOW_ACTION_TYPE_END; > > > > /* > > @@ -468,12 +510,12 @@ The following part create the flow attributes, in our > > case ingress. > > attr.ingress = 1; > > > > The third part defines the action to be taken when a packet matches > > -the rule. In this case send the packet to queue. > > +the rule. In this case send the packet to single RSS queue. > > > > .. code-block:: c > > > > - action[0].type = RTE_FLOW_ACTION_TYPE_QUEUE; > > - action[0].conf = &queue; > > + action[0].type = RTE_FLOW_ACTION_TYPE_SHARED; > > + action[0].conf = shared_action; > > action[1].type = RTE_FLOW_ACTION_TYPE_END; > > > > The fourth part is responsible for creating the pattern and is built from > > diff --git a/examples/flow_filtering/flow_blocks.c > > b/examples/flow_filtering/flow_blocks.c > > index 575d792810..ed17ae073e 100644 > > --- a/examples/flow_filtering/flow_blocks.c > > +++ b/examples/flow_filtering/flow_blocks.c > > @@ -5,12 +5,30 @@ > > #define MAX_PATTERN_NUM 3 > > #define MAX_ACTION_NUM 2 > > > > -struct rte_flow * > > -generate_ipv4_flow(uint16_t port_id, uint16_t rx_q, > > - uint32_t src_ip, uint32_t src_mask, > > - uint32_t dest_ip, uint32_t dest_mask, > > - struct rte_flow_error *error); > > +struct rte_flow_shared_action *shared_action; > > +uint16_t rss_queue[1] = {0}; > > + > > +struct rte_flow_action_rss rss_action = { > > + .func = RTE_ETH_HASH_FUNCTION_DEFAULT, > > + .level = 0, > > + .types = 0, > > + .key_len = 0, > > + .key = NULL, > > + .queue = rss_queue, > > + .queue_num = 1, > > +}; > > > > +struct rte_flow_action flow_action = { > > + .type = RTE_FLOW_ACTION_TYPE_RSS, > > + .conf = &rss_action, > > +}; > > + > > +struct rte_flow * > > +generate_ipv4_flow(uint16_t port_id, > > + uint32_t src_ip, uint32_t src_mask, > > + uint32_t dest_ip, uint32_t dest_mask, > > + struct rte_flow_error *error); > > +int update_rss_action(uint16_t port_id, struct rte_flow_error *error); > > > > /** > > * create a flow rule that sends packets with matching src and dest ip > > @@ -18,8 +36,6 @@ generate_ipv4_flow(uint16_t port_id, uint16_t rx_q, > > * > > * @param port_id > > * The selected port. > > - * @param rx_q > > - * The selected target queue. > > * @param src_ip > > * The src ip value to match the input packet. > > * @param src_mask > > @@ -35,16 +51,15 @@ generate_ipv4_flow(uint16_t port_id, uint16_t rx_q, > > * A flow if the rule could be created else return NULL. > > */ > > struct rte_flow * > > -generate_ipv4_flow(uint16_t port_id, uint16_t rx_q, > > - uint32_t src_ip, uint32_t src_mask, > > - uint32_t dest_ip, uint32_t dest_mask, > > - struct rte_flow_error *error) > > +generate_ipv4_flow(uint16_t port_id, > > + uint32_t src_ip, uint32_t src_mask, > > + uint32_t dest_ip, uint32_t dest_mask, > > + struct rte_flow_error *error) > > { > > struct rte_flow_attr attr; > > struct rte_flow_item pattern[MAX_PATTERN_NUM]; > > struct rte_flow_action action[MAX_ACTION_NUM]; > > struct rte_flow *flow = NULL; > > - struct rte_flow_action_queue queue = { .index = rx_q }; > > struct rte_flow_item_ipv4 ip_spec; > > struct rte_flow_item_ipv4 ip_mask; > > int res; > > @@ -61,10 +76,19 @@ generate_ipv4_flow(uint16_t port_id, uint16_t rx_q, > > > > /* > > * create the action sequence. > > - * one action only, move packet to queue > > + * one action only, move packet to shared RSS queue > > */ > > - action[0].type = RTE_FLOW_ACTION_TYPE_QUEUE; > > - action[0].conf = &queue; > > + shared_action = rte_flow_shared_action_create(port_id, NULL, > > + &flow_action, error); > > + if (shared_action) { > > + action[0].conf = shared_action; > > + action[0].type = RTE_FLOW_ACTION_TYPE_SHARED; > > + } else { > > + if (rte_errno != ENOSYS) > > + rte_exit(EXIT_FAILURE, ":: action creation failure\n"); > > + printf("PMD doesn't support shared actions.\n"); > > + action[0] = flow_action; > > + } > > action[1].type = RTE_FLOW_ACTION_TYPE_END; > > > > /* > > @@ -98,3 +122,30 @@ generate_ipv4_flow(uint16_t port_id, uint16_t rx_q, > > > > return flow; > > } > > + > > +/** > > + * updates shared RSS action if flow created with shared action > > + * > > + * @param port_id > > + * The selected port. > > + * @param[out] error > > + * Perform verbose error reporting if not NULL. > > + * > > + * @return > > + * 0 on sccess > > + */ > > +int > > +update_rss_action(uint16_t port_id, struct rte_flow_error *error) > > +{ > > + if (!shared_action) > > + return 0; > > + if (rss_queue[0] == 0) { > > + printf(">>> switching queue 0 -> 1\n"); > > + rss_queue[0] = 1; > > + } else { > > + printf(">>> switching queue 1 -> 0\n"); > > + rss_queue[0] = 0; > > + } > > + return rte_flow_shared_action_update(port_id, shared_action, > > + &flow_action, error); > > +} > > diff --git a/examples/flow_filtering/main.c b/examples/flow_filtering/main.c > > index cc9e7e7808..9ef7f099d9 100644 > > --- a/examples/flow_filtering/main.c > > +++ b/examples/flow_filtering/main.c > > @@ -32,8 +32,7 @@ > > static volatile bool force_quit; > > > > static uint16_t port_id; > > -static uint16_t nr_queues = 5; > > -static uint8_t selected_queue = 1; > > +static uint16_t nr_queues = 2; > > struct rte_mempool *mbuf_pool; > > struct rte_flow *flow; > > > > @@ -61,6 +60,7 @@ main_loop(void) > > uint16_t nb_rx; > > uint16_t i; > > uint16_t j; > > + int ret; > > > > while (!force_quit) { > > for (i = 0; i < nr_queues; i++) { > > @@ -82,6 +82,12 @@ main_loop(void) > > > > rte_pktmbuf_free(m); > > } > > + ret = update_rss_action(port_id, &error); > > + if (ret) > > + rte_exit(EXIT_FAILURE, > > + ":: error: RSS action update " > > + "failed: %s\n", > > + rte_strerror(-ret)); > > } > > } > > } > > @@ -243,8 +249,9 @@ main(int argc, char **argv) > > > > init_port(); > > > > + > > /* create flow for send packet with */ > > - flow = generate_ipv4_flow(port_id, selected_queue, > > + flow = generate_ipv4_flow(port_id, > > SRC_IP, EMPTY_MASK, > > DEST_IP, FULL_MASK, &error); > > if (!flow) { > > -- > > 2.26.2 ^ permalink raw reply [flat|nested] 106+ messages in thread
* [dpdk-dev] [PATCH v3 10/10] app/testpmd: support shared action 2020-10-03 22:06 ` [dpdk-dev] [PATCH v3 00/10] RTE flow shared action Andrey Vesnovaty ` (8 preceding siblings ...) 2020-10-03 22:06 ` [dpdk-dev] [PATCH v3 09/10] examples/flow_filtering: utilize shared RSS action Andrey Vesnovaty @ 2020-10-03 22:06 ` Andrey Vesnovaty 2020-10-04 11:28 ` Ori Kam 2020-10-04 11:14 ` [dpdk-dev] [PATCH v3 00/10] RTE flow " Ori Kam 10 siblings, 1 reply; 106+ messages in thread From: Andrey Vesnovaty @ 2020-10-03 22:06 UTC (permalink / raw) To: dev Cc: jer, jerinjacobk, thomas, ferruh.yigit, stephen, bruce.richardson, orika, viacheslavo, andrey.vesnovaty, mdr, nhorman, ajit.khaparde, samik.gupta, Ori Kam, Wenzhuo Lu, Beilei Xing, Bernard Iremonger This patch adds shared action support to testpmd CLI. All shared actions created via testpmd CLI assigned ID for further reference in other CLI commands. Shared action ID supplied as CLI argument or assigned by testpmd is similar to flow ID & limited to scope of testpdm CLI. Create shared action syntax: flow shared_action (port) create {action_id (shared_action_id)} \ (action) / end Create shared action examples: flow shared_action 0 create action_id 100 \ rss queues 1 2 end / end This creates shared rss action with id 100 on port 0. flow shared_action 0 create action_id \ rss queues 0 1 end / end This creates shared rss action with id assigned by tetspmd on port 0. Update shared action syntax: flow shared_action (port) update (shared_action_id) (action) / end Update shared action example: flow shared_action 0 update 100 rss queues 0 3 end / end This updates shared rss action having id 100 on port 0 with rss to queues 0 3 (in create example rss queues were 1 & 2). Destroy shared action syntax: flow shared_action (port) destroy action_id (shared_action_id) \ { action_id (shared_action_id) [...]} Update shared action example: flow shared_action 0 destroy action_id 100 action_id 101 This destroys shared actions having id 100 & 101 Query shared action syntax: flow shared_action (port) query (shared_action_id) Query shared action example: flow shared_action 0 query 100 This queries shared actions having id 100 Use shared action as flow action syntax: flow create (port) ... / end actions {action / [...]} \ shared (action_id) / {action / [...]} end Use shared action as flow action example: flow create 0 ingress pattern ... / end \ actions shared 100 / end This creates flow rule where rss action is shared rss action having id 100. All shared action CLIs report status of the command. Shared action query CLI output depends on action type. Signed-off-by: Andrey Vesnovaty <andreyv@nvidia.com> --- app/test-pmd/cmdline_flow.c | 262 +++++++++++++++++++++++++++++++++++- app/test-pmd/config.c | 217 +++++++++++++++++++++++++++++ app/test-pmd/testpmd.h | 19 +++ 3 files changed, 497 insertions(+), 1 deletion(-) diff --git a/app/test-pmd/cmdline_flow.c b/app/test-pmd/cmdline_flow.c index 6263d307ed..ce338d370d 100644 --- a/app/test-pmd/cmdline_flow.c +++ b/app/test-pmd/cmdline_flow.c @@ -49,6 +49,7 @@ enum index { PORT_ID, GROUP_ID, PRIORITY_LEVEL, + SHARED_ACTION_ID, /* Top-level command. */ SET, @@ -60,6 +61,7 @@ enum index { /* Top-level command. */ FLOW, /* Sub-level commands. */ + SHARED_ACTION, VALIDATE, CREATE, DESTROY, @@ -89,6 +91,18 @@ enum index { EGRESS, TRANSFER, + /* Shared action arguments */ + SHARED_ACTION_CREATE, + SHARED_ACTION_UPDATE, + SHARED_ACTION_DESTROY, + SHARED_ACTION_QUERY, + + /* Shared action create arguments */ + SHARED_ACTION_CREATE_ID, + + /* Shared action destroy arguments */ + SHARED_ACTION_DESTROY_ID, + /* Validate/create pattern. */ PATTERN, ITEM_PARAM_IS, @@ -358,6 +372,8 @@ enum index { ACTION_SET_IPV6_DSCP_VALUE, ACTION_AGE, ACTION_AGE_TIMEOUT, + ACTION_SHARED, + SHARED_ACTION_ID2PTR, }; /** Maximum size for pattern in struct rte_flow_item_raw. */ @@ -651,6 +667,13 @@ struct buffer { enum index command; /**< Flow command. */ portid_t port; /**< Affected port ID. */ union { + struct { + uint32_t *action_id; + uint32_t action_id_n; + } sa_destroy; /**< Shared action destroy arguments. */ + struct { + uint32_t action_id; + } sa; /* Shared action query arguments */ struct { struct rte_flow_attr attr; struct rte_flow_item *pattern; @@ -707,6 +730,19 @@ struct parse_action_priv { .size = s, \ }) +static const enum index next_sa_create_attr[] = { + SHARED_ACTION_CREATE_ID, + ACTION_RSS, + ZERO, +}; + +static const enum index next_sa_subcmd[] = { + SHARED_ACTION_CREATE, + SHARED_ACTION_UPDATE, + SHARED_ACTION_DESTROY, + SHARED_ACTION_QUERY, +}; + static const enum index next_vc_attr[] = { GROUP, PRIORITY, @@ -741,6 +777,12 @@ static const enum index next_aged_attr[] = { ZERO, }; +static const enum index next_sa_destroy_attr[] = { + SHARED_ACTION_DESTROY_ID, + END, + ZERO, +}; + static const enum index item_param[] = { ITEM_PARAM_IS, ITEM_PARAM_SPEC, @@ -1189,6 +1231,7 @@ static const enum index next_action[] = { ACTION_SET_IPV4_DSCP, ACTION_SET_IPV6_DSCP, ACTION_AGE, + ACTION_SHARED, ZERO, }; @@ -1546,6 +1589,15 @@ static int parse_ipv6_addr(struct context *, const struct token *, static int parse_port(struct context *, const struct token *, const char *, unsigned int, void *, unsigned int); +static int parse_sa(struct context *, const struct token *, + const char *, unsigned int, + void *, unsigned int); +static int parse_sa_destroy(struct context *ctx, const struct token *token, + const char *str, unsigned int len, + void *buf, unsigned int size); +static int parse_sa_id2ptr(struct context *ctx, const struct token *token, + const char *str, unsigned int len, void *buf, + unsigned int size); static int comp_none(struct context *, const struct token *, unsigned int, char *, unsigned int); static int comp_boolean(struct context *, const struct token *, @@ -1684,13 +1736,21 @@ static const struct token token_list[] = { .call = parse_int, .comp = comp_none, }, + [SHARED_ACTION_ID] = { + .name = "{shared_action_id}", + .type = "SHARED_ACTION_ID", + .help = "shared action id", + .call = parse_int, + .comp = comp_none, + }, /* Top-level command. */ [FLOW] = { .name = "flow", .type = "{command} {port_id} [{arg} [...]]", .help = "manage ingress/egress flow rules", .next = NEXT(NEXT_ENTRY - (VALIDATE, + (SHARED_ACTION, + VALIDATE, CREATE, DESTROY, FLUSH, @@ -1701,7 +1761,45 @@ static const struct token token_list[] = { ISOLATE)), .call = parse_init, }, + /* Top-level command. */ + [SHARED_ACTION] = { + .name = "shared_action", + .type = "{command} {port_id} [{arg} [...]]", + .help = "manage shared actions", + .next = NEXT(next_sa_subcmd, NEXT_ENTRY(PORT_ID)), + .args = ARGS(ARGS_ENTRY(struct buffer, port)), + .call = parse_sa, + }, /* Sub-level commands. */ + [SHARED_ACTION_CREATE] = { + .name = "create", + .help = "create shared action", + .next = NEXT(next_sa_create_attr), + .args = ARGS(ARGS_ENTRY(struct buffer, args.vc.attr.group)), + .call = parse_sa, + }, + [SHARED_ACTION_UPDATE] = { + .name = "update", + .help = "update shared action", + .next = NEXT(NEXT_ENTRY(ACTION_RSS), + NEXT_ENTRY(SHARED_ACTION_ID)), + .args = ARGS(ARGS_ENTRY(struct buffer, args.vc.attr.group)), + .call = parse_sa, + }, + [SHARED_ACTION_DESTROY] = { + .name = "destroy", + .help = "destroy shared action", + .next = NEXT(NEXT_ENTRY(SHARED_ACTION_DESTROY_ID)), + .args = ARGS(ARGS_ENTRY(struct buffer, port)), + .call = parse_sa_destroy, + }, + [SHARED_ACTION_QUERY] = { + .name = "query", + .help = "query shared action", + .next = NEXT(NEXT_ENTRY(END), NEXT_ENTRY(SHARED_ACTION_ID)), + .args = ARGS(ARGS_ENTRY(struct buffer, args.sa.action_id)), + .call = parse_sa, + }, [VALIDATE] = { .name = "validate", .help = "check whether a flow rule can be created", @@ -3841,6 +3939,40 @@ static const struct token token_list[] = { .next = NEXT(action_age, NEXT_ENTRY(UNSIGNED)), .call = parse_vc_conf, }, + /* Shared action destroy arguments. */ + [SHARED_ACTION_DESTROY_ID] = { + .name = "action_id", + .help = "specify a shared action id to destroy", + .next = NEXT(next_sa_destroy_attr, + NEXT_ENTRY(SHARED_ACTION_ID)), + .args = ARGS(ARGS_ENTRY_PTR(struct buffer, + args.sa_destroy.action_id)), + .call = parse_sa_destroy, + }, + /* Shared action create arguments. */ + [SHARED_ACTION_CREATE_ID] = { + .name = "action_id", + .help = "specify a shared action id to create", + .next = NEXT(NEXT_ENTRY(ACTION_RSS), + NEXT_ENTRY(SHARED_ACTION_ID)), + .args = ARGS(ARGS_ENTRY(struct buffer, args.vc.attr.group)), + }, + [ACTION_SHARED] = { + .name = "shared", + .help = "apply shared action by id", + .priv = PRIV_ACTION(SHARED, 0), + .next = NEXT(NEXT_ENTRY(SHARED_ACTION_ID2PTR)), + .args = ARGS(ARGS_ENTRY_ARB(0, sizeof(uint32_t))), + .call = parse_vc, + }, + [SHARED_ACTION_ID2PTR] = { + .name = "{action_id}", + .type = "SHARED_ACTION_ID", + .help = "shared action id", + .next = NEXT(NEXT_ENTRY(ACTION_NEXT)), + .call = parse_sa_id2ptr, + .comp = comp_none, + }, }; /** Remove and return last entry from argument stack. */ @@ -4025,6 +4157,91 @@ parse_init(struct context *ctx, const struct token *token, return len; } +/** Parse tokens for shared action commands. */ +static int +parse_sa(struct context *ctx, const struct token *token, + const char *str, unsigned int len, + void *buf, unsigned int size) +{ + struct buffer *out = buf; + + /* Token name must match. */ + if (parse_default(ctx, token, str, len, NULL, 0) < 0) + return -1; + /* Nothing else to do if there is no buffer. */ + if (!out) + return len; + if (!out->command) { + if (ctx->curr != SHARED_ACTION) + return -1; + if (sizeof(*out) > size) + return -1; + out->command = ctx->curr; + ctx->objdata = 0; + ctx->object = out; + ctx->objmask = NULL; + out->args.vc.data = (uint8_t *)out + size; + return len; + } + switch (ctx->curr) { + case SHARED_ACTION_CREATE: + case SHARED_ACTION_UPDATE: + out->args.vc.actions = + (void *)RTE_ALIGN_CEIL((uintptr_t)(out + 1), + sizeof(double)); + out->args.vc.attr.group = UINT32_MAX; + /* fallthrough */ + case SHARED_ACTION_QUERY: + out->command = ctx->curr; + ctx->objdata = 0; + ctx->object = out; + ctx->objmask = NULL; + return len; + default: + return -1; + } +} + + +/** Parse tokens for shared action destroy command. */ +static int +parse_sa_destroy(struct context *ctx, const struct token *token, + const char *str, unsigned int len, + void *buf, unsigned int size) +{ + struct buffer *out = buf; + uint32_t *action_id; + + /* Token name must match. */ + if (parse_default(ctx, token, str, len, NULL, 0) < 0) + return -1; + /* Nothing else to do if there is no buffer. */ + if (!out) + return len; + if (!out->command || out->command == SHARED_ACTION) { + if (ctx->curr != SHARED_ACTION_DESTROY) + return -1; + if (sizeof(*out) > size) + return -1; + out->command = ctx->curr; + ctx->objdata = 0; + ctx->object = out; + ctx->objmask = NULL; + out->args.sa_destroy.action_id = + (void *)RTE_ALIGN_CEIL((uintptr_t)(out + 1), + sizeof(double)); + return len; + } + action_id = out->args.sa_destroy.action_id + + out->args.sa_destroy.action_id_n++; + if ((uint8_t *)action_id > (uint8_t *)out + size) + return -1; + ctx->objdata = 0; + ctx->object = action_id; + ctx->objmask = NULL; + return len; +} + /** Parse tokens for validate/create commands. */ static int parse_vc(struct context *ctx, const struct token *token, @@ -6092,6 +6309,32 @@ parse_port(struct context *ctx, const struct token *token, return ret; } +static int +parse_sa_id2ptr(struct context *ctx, const struct token *token, + const char *str, unsigned int len, + void *buf, unsigned int size) +{ + struct rte_flow_action *action = ctx->object; + uint32_t id; + int ret; + + (void)buf; + (void)size; + ctx->objdata = 0; + ctx->object = &id; + ctx->objmask = NULL; + ret = parse_int(ctx, token, str, len, ctx->object, sizeof(id)); + ctx->object = action; + if (ret != (int)len) + return ret; + /* set shared action */ + if (action) { + action->conf = port_shared_action_get_by_id(ctx->port, id); + ret = (action->conf) ? ret : -1; + } + return ret; +} + /** Parse set command, initialize output buffer for subsequent tokens. */ static int parse_set_raw_encap_decap(struct context *ctx, const struct token *token, @@ -6541,6 +6784,23 @@ static void cmd_flow_parsed(const struct buffer *in) { switch (in->command) { + case SHARED_ACTION_CREATE: + port_shared_action_create(in->port, + in->args.vc.attr.group, + in->args.vc.actions); + break; + case SHARED_ACTION_DESTROY: + port_shared_action_destroy(in->port, + in->args.sa_destroy.action_id_n, + in->args.sa_destroy.action_id); + break; + case SHARED_ACTION_UPDATE: + port_shared_action_update(in->port, in->args.vc.attr.group, + in->args.vc.actions); + break; + case SHARED_ACTION_QUERY: + port_shared_action_query(in->port, in->args.sa.action_id); + break; case VALIDATE: port_flow_validate(in->port, &in->args.vc.attr, in->args.vc.pattern, in->args.vc.actions); diff --git a/app/test-pmd/config.c b/app/test-pmd/config.c index dabe7c119a..1c76a01050 100644 --- a/app/test-pmd/config.c +++ b/app/test-pmd/config.c @@ -1461,6 +1461,223 @@ rss_config_display(struct rte_flow_action_rss *rss_conf) } } +static struct port_shared_action * +action_get_by_id(portid_t port_id, uint32_t id) +{ + struct rte_port *port; + struct port_shared_action **ppsa; + struct port_shared_action *psa = NULL; + + if (port_id_is_invalid(port_id, ENABLED_WARN) || + port_id == (portid_t)RTE_PORT_ALL) + return NULL; + port = &ports[port_id]; + ppsa = &port->actions_list; + while (*ppsa) { + if ((*ppsa)->id == id) { + psa = *ppsa; + break; + } + ppsa = &(*ppsa)->next; + } + if (!psa) + printf("Failed to find shared action #%u on port %u\n", + id, port_id); + return psa; +} + +static int +action_alloc(portid_t port_id, uint32_t id, + struct port_shared_action **action) +{ + struct rte_port *port; + struct port_shared_action **ppsa; + struct port_shared_action *psa = NULL; + + *action = NULL; + if (port_id_is_invalid(port_id, ENABLED_WARN) || + port_id == (portid_t)RTE_PORT_ALL) + return -EINVAL; + port = &ports[port_id]; + if (id == UINT32_MAX) { + /* taking first available ID */ + if (port->actions_list) { + if (port->actions_list->id == UINT32_MAX - 1) { + printf("Highest shared action ID is already" + " assigned, delete it first"); + return -ENOMEM; + } + id = port->actions_list->id + 1; + } else { + id = 0; + } + } + psa = calloc(1, sizeof(*psa)); + if (!psa) { + printf("Allocation of port %u shared action failed\n", + port_id); + return -ENOMEM; + } + ppsa = &port->actions_list; + while (*ppsa && (*ppsa)->id > id) + ppsa = &(*ppsa)->next; + if (*ppsa && (*ppsa)->id == id) { + printf("Shared action ID %u is already assigned," + " delete it first", id); + free(psa); + return -EINVAL; + } + psa->next = *ppsa; + psa->id = id; + *ppsa = psa; + *action = psa; + return 0; +} + +/** Create shared action */ +int +port_shared_action_create(portid_t port_id, uint32_t id, + const struct rte_flow_action *action) +{ + struct port_shared_action *psa; + int ret; + struct rte_flow_error error; + + ret = action_alloc(port_id, id, &psa); + if (ret) + return ret; + /* Poisoning to make sure PMDs update it in case of error. */ + memset(&error, 0x22, sizeof(error)); + psa->action = rte_flow_shared_action_create(port_id, NULL, action, + &error); + if (!psa->action) { + uint32_t destroy_id = psa->id; + port_shared_action_destroy(port_id, 1, &destroy_id); + return port_flow_complain(&error); + } + psa->type = action->type; + printf("Shared action #%u created\n", psa->id); + return 0; +} + +/** Destroy shared action */ +int +port_shared_action_destroy(portid_t port_id, + uint32_t n, + const uint32_t *actions) +{ + struct rte_port *port; + struct port_shared_action **tmp; + uint32_t c = 0; + int ret = 0; + + if (port_id_is_invalid(port_id, ENABLED_WARN) || + port_id == (portid_t)RTE_PORT_ALL) + return -EINVAL; + port = &ports[port_id]; + tmp = &port->actions_list; + while (*tmp) { + uint32_t i; + + for (i = 0; i != n; ++i) { + struct rte_flow_error error; + struct port_shared_action *psa = *tmp; + + if (actions[i] != psa->id) + continue; + /* + * Poisoning to make sure PMDs update it in case + * of error. + */ + memset(&error, 0x33, sizeof(error)); + + if (psa->action && rte_flow_shared_action_destroy( + port_id, psa->action, &error)) { + ret = port_flow_complain(&error); + continue; + } + *tmp = psa->next; + free(psa); + printf("Shared action #%u destroyed\n", psa->id); + break; + } + if (i == n) + tmp = &(*tmp)->next; + ++c; + } + return ret; +} + + +/** Get shared action by port + id */ +struct rte_flow_shared_action * +port_shared_action_get_by_id(portid_t port_id, uint32_t id) +{ + + struct port_shared_action *psa = action_get_by_id(port_id, id); + + return (psa) ? psa->action : NULL; +} + +/** Update shared action */ +int +port_shared_action_update(portid_t port_id, uint32_t id, + const struct rte_flow_action *action) +{ + struct rte_flow_error error; + struct rte_flow_shared_action *shared_action; + + shared_action = port_shared_action_get_by_id(port_id, id); + if (!shared_action) + return -EINVAL; + if (rte_flow_shared_action_update(port_id, shared_action, action, + &error)) { + return port_flow_complain(&error); + } + printf("Shared action #%u updated\n", id); + return 0; +} + +int +port_shared_action_query(portid_t port_id, uint32_t id) +{ + struct rte_flow_error error; + struct port_shared_action *psa; + uint64_t default_data; + void *data = NULL; + int ret = 0; + + psa = action_get_by_id(port_id, id); + if (!psa) + return -EINVAL; + switch (psa->type) { + case RTE_FLOW_ACTION_TYPE_RSS: + data = &default_data; + break; + default: + printf("Shared action %u (type: %d) on port %u doesn't support" + " query\n", id, psa->type, port_id); + return -1; + } + if (rte_flow_shared_action_query(port_id, psa->action, data, &error)) + ret = port_flow_complain(&error); + switch (psa->type) { + case RTE_FLOW_ACTION_TYPE_RSS: + if (!ret) + printf("Shared RSS action:\n\trefs:%u\n", + *((uint32_t *)data)); + data = NULL; + break; + default: + printf("Shared action %u (type: %d) on port %u doesn't support" + " query\n", id, psa->type, port_id); + ret = -1; + } + if (data) + free(data); + return ret; +} + /** Validate flow rule. */ int port_flow_validate(portid_t port_id, diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h index f139fe7a0a..d995314612 100644 --- a/app/test-pmd/testpmd.h +++ b/app/test-pmd/testpmd.h @@ -142,6 +142,14 @@ struct port_flow { uint8_t data[]; /**< Storage for flow rule description */ }; +/* Descriptor for shared action */ +struct port_shared_action { + struct port_shared_action *next; /**< Next flow in list. */ + uint32_t id; /**< Shared action ID. */ + enum rte_flow_action_type type; /**< Action type. */ + struct rte_flow_shared_action *action; /**< Shared action handle. */ +}; + /** * The data structure associated with each port. */ @@ -172,6 +180,8 @@ struct rte_port { uint32_t mc_addr_nb; /**< nb. of addr. in mc_addr_pool */ uint8_t slave_flag; /**< bonding slave port */ struct port_flow *flow_list; /**< Associated flows. */ + struct port_shared_action *actions_list; + /**< Associated shared actions. */ const struct rte_eth_rxtx_callback *rx_dump_cb[RTE_MAX_QUEUES_PER_PORT+1]; const struct rte_eth_rxtx_callback *tx_dump_cb[RTE_MAX_QUEUES_PER_PORT+1]; /**< metadata value to insert in Tx packets. */ @@ -746,6 +756,14 @@ void port_reg_bit_field_set(portid_t port_id, uint32_t reg_off, uint8_t bit1_pos, uint8_t bit2_pos, uint32_t value); void port_reg_display(portid_t port_id, uint32_t reg_off); void port_reg_set(portid_t port_id, uint32_t reg_off, uint32_t value); +int port_shared_action_create(portid_t port_id, uint32_t id, + const struct rte_flow_action *action); +int port_shared_action_destroy(portid_t port_id, + uint32_t n, const uint32_t *action); +struct rte_flow_shared_action *port_shared_action_get_by_id(portid_t port_id, + uint32_t id); +int port_shared_action_update(portid_t port_id, uint32_t id, + const struct rte_flow_action *action); int port_flow_validate(portid_t port_id, const struct rte_flow_attr *attr, const struct rte_flow_item *pattern, @@ -754,6 +772,7 @@ int port_flow_create(portid_t port_id, const struct rte_flow_attr *attr, const struct rte_flow_item *pattern, const struct rte_flow_action *actions); +int port_shared_action_query(portid_t port_id, uint32_t id); void update_age_action_context(const struct rte_flow_action *actions, struct port_flow *pf); int port_flow_destroy(portid_t port_id, uint32_t n, const uint32_t *rule); -- 2.26.2 ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [dpdk-dev] [PATCH v3 10/10] app/testpmd: support shared action 2020-10-03 22:06 ` [dpdk-dev] [PATCH v3 10/10] app/testpmd: support shared action Andrey Vesnovaty @ 2020-10-04 11:28 ` Ori Kam 2020-10-04 12:04 ` Ori Kam 0 siblings, 1 reply; 106+ messages in thread From: Ori Kam @ 2020-10-04 11:28 UTC (permalink / raw) To: Andrey Vesnovaty, dev Cc: jer, jerinjacobk, NBU-Contact-Thomas Monjalon, ferruh.yigit, stephen, bruce.richardson, Slava Ovsiienko, andrey.vesnovaty, mdr, nhorman, ajit.khaparde, samik.gupta, Ori Kam, Wenzhuo Lu, Beilei Xing, Bernard Iremonger Hi Andrey > -----Original Message----- > From: Andrey Vesnovaty <andreyv@nvidia.com> > Sent: Sunday, October 4, 2020 1:06 AM > Subject: [PATCH v3 10/10] app/testpmd: support shared action > > This patch adds shared action support to testpmd CLI. > > All shared actions created via testpmd CLI assigned ID for further > reference in other CLI commands. Shared action ID supplied as CLI > argument or assigned by testpmd is similar to flow ID & limited to > scope of testpdm CLI. > > Create shared action syntax: > flow shared_action (port) create {action_id (shared_action_id)} \ > (action) / end > > Create shared action examples: > flow shared_action 0 create action_id 100 \ > rss queues 1 2 end / end > This creates shared rss action with id 100 on port 0. > > flow shared_action 0 create action_id \ > rss queues 0 1 end / end > This creates shared rss action with id assigned by tetspmd > on port 0. > > Update shared action syntax: > flow shared_action (port) update (shared_action_id) (action) / end > > Update shared action example: > flow shared_action 0 update 100 rss queues 0 3 end / end > This updates shared rss action having id 100 on port 0 > with rss to queues 0 3 (in create example rss queues were > 1 & 2). > > Destroy shared action syntax: > flow shared_action (port) destroy action_id (shared_action_id) \ > { action_id (shared_action_id) [...]} > > Update shared action example: > flow shared_action 0 destroy action_id 100 action_id 101 > This destroys shared actions having id 100 & 101 > > Query shared action syntax: > flow shared_action (port) query (shared_action_id) > > Query shared action example: > flow shared_action 0 query 100 > This queries shared actions having id 100 > > Use shared action as flow action syntax: > flow create (port) ... / end actions {action / [...]} \ > shared (action_id) / {action / [...]} end > > Use shared action as flow action example: > flow create 0 ingress pattern ... / end \ > actions shared 100 / end > This creates flow rule where rss action is shared rss action > having id 100. > > All shared action CLIs report status of the command. > Shared action query CLI output depends on action type. > > Signed-off-by: Andrey Vesnovaty <andreyv@nvidia.com> > --- Acked-by: Ori Kam <orika@nvidia.com> Thanks, Ori ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [dpdk-dev] [PATCH v3 10/10] app/testpmd: support shared action 2020-10-04 11:28 ` Ori Kam @ 2020-10-04 12:04 ` Ori Kam 2020-10-06 10:36 ` Andrey Vesnovaty 0 siblings, 1 reply; 106+ messages in thread From: Ori Kam @ 2020-10-04 12:04 UTC (permalink / raw) To: Ori Kam, Andrey Vesnovaty, dev Cc: jer, jerinjacobk, NBU-Contact-Thomas Monjalon, ferruh.yigit, stephen, bruce.richardson, Slava Ovsiienko, andrey.vesnovaty, mdr, nhorman, ajit.khaparde, samik.gupta, Ori Kam, Wenzhuo Lu, Beilei Xing, Bernard Iremonger Hi Andrey, I think you are missing documentation: Do/guides/testpmd_app_ug/testpmd_funcs.rst Best, Ori > -----Original Message----- > From: Ori Kam <orika@nvidia.com> > Sent: Sunday, October 4, 2020 2:28 PM .xing@intel.com>; Bernard > Iremonger <bernard.iremonger@intel.com> > Subject: RE: [PATCH v3 10/10] app/testpmd: support shared action > > Hi Andrey > > > -----Original Message----- > > From: Andrey Vesnovaty <andreyv@nvidia.com> > > Sent: Sunday, October 4, 2020 1:06 AM > > Subject: [PATCH v3 10/10] app/testpmd: support shared action > > > > This patch adds shared action support to testpmd CLI. > > > > All shared actions created via testpmd CLI assigned ID for further > > reference in other CLI commands. Shared action ID supplied as CLI > > argument or assigned by testpmd is similar to flow ID & limited to > > scope of testpdm CLI. > > > > Create shared action syntax: > > flow shared_action (port) create {action_id (shared_action_id)} \ > > (action) / end > > > > Create shared action examples: > > flow shared_action 0 create action_id 100 \ > > rss queues 1 2 end / end > > This creates shared rss action with id 100 on port 0. > > > > flow shared_action 0 create action_id \ > > rss queues 0 1 end / end > > This creates shared rss action with id assigned by tetspmd > > on port 0. > > > > Update shared action syntax: > > flow shared_action (port) update (shared_action_id) (action) / end > > > > Update shared action example: > > flow shared_action 0 update 100 rss queues 0 3 end / end > > This updates shared rss action having id 100 on port 0 > > with rss to queues 0 3 (in create example rss queues were > > 1 & 2). > > > > Destroy shared action syntax: > > flow shared_action (port) destroy action_id (shared_action_id) \ > > { action_id (shared_action_id) [...]} > > > > Update shared action example: > > flow shared_action 0 destroy action_id 100 action_id 101 > > This destroys shared actions having id 100 & 101 > > > > Query shared action syntax: > > flow shared_action (port) query (shared_action_id) > > > > Query shared action example: > > flow shared_action 0 query 100 > > This queries shared actions having id 100 > > > > Use shared action as flow action syntax: > > flow create (port) ... / end actions {action / [...]} \ > > shared (action_id) / {action / [...]} end > > > > Use shared action as flow action example: > > flow create 0 ingress pattern ... / end \ > > actions shared 100 / end > > This creates flow rule where rss action is shared rss action > > having id 100. > > > > All shared action CLIs report status of the command. > > Shared action query CLI output depends on action type. > > > > Signed-off-by: Andrey Vesnovaty <andreyv@nvidia.com> > > --- > > Acked-by: Ori Kam <orika@nvidia.com> > Thanks, > Ori ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [dpdk-dev] [PATCH v3 10/10] app/testpmd: support shared action 2020-10-04 12:04 ` Ori Kam @ 2020-10-06 10:36 ` Andrey Vesnovaty 0 siblings, 0 replies; 106+ messages in thread From: Andrey Vesnovaty @ 2020-10-06 10:36 UTC (permalink / raw) To: Ori Kam, dev Cc: jer, jerinjacobk, NBU-Contact-Thomas Monjalon, ferruh.yigit, stephen, bruce.richardson, Slava Ovsiienko, andrey.vesnovaty, mdr, nhorman, ajit.khaparde, samik.gupta, Ori Kam, Wenzhuo Lu, Beilei Xing, Bernard Iremonger Hi Ori. testpmd documentation will be added in v4. Thanks, Andrey > -----Original Message----- > From: Ori Kam <orika@nvidia.com> > Sent: Sunday, October 4, 2020 3:05 PM > To: Ori Kam <orika@nvidia.com>; Andrey Vesnovaty <andreyv@nvidia.com>; > dev@dpdk.org > Cc: jer@marvell.com; jerinjacobk@gmail.com; NBU-Contact-Thomas Monjalon > <thomas@monjalon.net>; ferruh.yigit@intel.com; > stephen@networkplumber.org; bruce.richardson@intel.com; Slava Ovsiienko > <viacheslavo@nvidia.com>; andrey.vesnovaty@gmail.com; mdr@ashroe.eu; > nhorman@tuxdriver.com; ajit.khaparde@broadcom.com; > samik.gupta@broadcom.com; Ori Kam <orika@mellanox.com>; Wenzhuo Lu > <wenzhuo.lu@intel.com>; Beilei Xing <beilei.xing@intel.com>; Bernard > Iremonger <bernard.iremonger@intel.com> > Subject: RE: [PATCH v3 10/10] app/testpmd: support shared action > > Hi Andrey, > > I think you are missing documentation: > Do/guides/testpmd_app_ug/testpmd_funcs.rst > Best, > Ori > > > -----Original Message----- > > From: Ori Kam <orika@nvidia.com> > > Sent: Sunday, October 4, 2020 2:28 PM > .xing@intel.com>; Bernard > > Iremonger <bernard.iremonger@intel.com> > > Subject: RE: [PATCH v3 10/10] app/testpmd: support shared action > > > > Hi Andrey > > > > > -----Original Message----- > > > From: Andrey Vesnovaty <andreyv@nvidia.com> > > > Sent: Sunday, October 4, 2020 1:06 AM > > > Subject: [PATCH v3 10/10] app/testpmd: support shared action > > > > > > This patch adds shared action support to testpmd CLI. > > > > > > All shared actions created via testpmd CLI assigned ID for further > > > reference in other CLI commands. Shared action ID supplied as CLI > > > argument or assigned by testpmd is similar to flow ID & limited to > > > scope of testpdm CLI. > > > > > > Create shared action syntax: > > > flow shared_action (port) create {action_id (shared_action_id)} \ > > > (action) / end > > > > > > Create shared action examples: > > > flow shared_action 0 create action_id 100 \ > > > rss queues 1 2 end / end > > > This creates shared rss action with id 100 on port 0. > > > > > > flow shared_action 0 create action_id \ > > > rss queues 0 1 end / end > > > This creates shared rss action with id assigned by tetspmd > > > on port 0. > > > > > > Update shared action syntax: > > > flow shared_action (port) update (shared_action_id) (action) / end > > > > > > Update shared action example: > > > flow shared_action 0 update 100 rss queues 0 3 end / end > > > This updates shared rss action having id 100 on port 0 > > > with rss to queues 0 3 (in create example rss queues were > > > 1 & 2). > > > > > > Destroy shared action syntax: > > > flow shared_action (port) destroy action_id (shared_action_id) \ > > > { action_id (shared_action_id) [...]} > > > > > > Update shared action example: > > > flow shared_action 0 destroy action_id 100 action_id 101 > > > This destroys shared actions having id 100 & 101 > > > > > > Query shared action syntax: > > > flow shared_action (port) query (shared_action_id) > > > > > > Query shared action example: > > > flow shared_action 0 query 100 > > > This queries shared actions having id 100 > > > > > > Use shared action as flow action syntax: > > > flow create (port) ... / end actions {action / [...]} \ > > > shared (action_id) / {action / [...]} end > > > > > > Use shared action as flow action example: > > > flow create 0 ingress pattern ... / end \ > > > actions shared 100 / end > > > This creates flow rule where rss action is shared rss action > > > having id 100. > > > > > > All shared action CLIs report status of the command. > > > Shared action query CLI output depends on action type. > > > > > > Signed-off-by: Andrey Vesnovaty <andreyv@nvidia.com> > > > --- > > > > Acked-by: Ori Kam <orika@nvidia.com> > > Thanks, > > Ori ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [dpdk-dev] [PATCH v3 00/10] RTE flow shared action 2020-10-03 22:06 ` [dpdk-dev] [PATCH v3 00/10] RTE flow shared action Andrey Vesnovaty ` (9 preceding siblings ...) 2020-10-03 22:06 ` [dpdk-dev] [PATCH v3 10/10] app/testpmd: support shared action Andrey Vesnovaty @ 2020-10-04 11:14 ` Ori Kam 2020-10-06 10:28 ` Andrey Vesnovaty 10 siblings, 1 reply; 106+ messages in thread From: Ori Kam @ 2020-10-04 11:14 UTC (permalink / raw) To: Andrey Vesnovaty, dev Cc: jer, jerinjacobk, NBU-Contact-Thomas Monjalon, ferruh.yigit, stephen, bruce.richardson, Slava Ovsiienko, andrey.vesnovaty, mdr, nhorman, ajit.khaparde, samik.gupta Hi Andrey, I suggest that you split this patchset to two different ones. One for the Mellanox PMD and one RTE just in each patch set reference the second patch. This why it is possible to merge only the RTE level, if other PMD will use it. Best, Ori > -----Original Message----- > From: dev <dev-bounces@dpdk.org> On Behalf Of Andrey Vesnovaty > Sent: Sunday, October 4, 2020 1:06 AM > To: dev@dpdk.org > Cc: jer@marvell.com; jerinjacobk@gmail.com; NBU-Contact-Thomas Monjalon > <thomas@monjalon.net>; ferruh.yigit@intel.com; > stephen@networkplumber.org; bruce.richardson@intel.com; Ori Kam > <orika@nvidia.com>; Slava Ovsiienko <viacheslavo@nvidia.com>; > andrey.vesnovaty@gmail.com; mdr@ashroe.eu; nhorman@tuxdriver.com; > ajit.khaparde@broadcom.com; samik.gupta@broadcom.com > Subject: [dpdk-dev] [PATCH v3 00/10] RTE flow shared action > > This patchset introduces shared action for RTE flow. > > V3 changes: > - implement testpmd for shared action > - fix flow_filtering example application > - add conf arg to shared action create API > > Notes: PMD implementation should be considered as draft > > Andrey Vesnovaty (10): > ethdev: add flow shared action API > ethdev: add conf arg to shared action icreate API > common/mlx5: modify advanced Rx object via DevX > net/mlx5: modify hash Rx queue objects > net/mlx5: shared action PMD > net/mlx5: shared action PMD create conf arg > net/mlx5: driver support for shared action > net/mlx5: shared action create conf drv support > examples/flow_filtering: utilize shared RSS action > app/testpmd: support shared action > > app/test-pmd/cmdline_flow.c | 262 ++++++- > app/test-pmd/config.c | 217 ++++++ > app/test-pmd/testpmd.h | 19 + > doc/guides/sample_app_ug/flow_filtering.rst | 62 +- > drivers/common/mlx5/mlx5_devx_cmds.c | 84 +++ > drivers/common/mlx5/mlx5_devx_cmds.h | 10 + > drivers/common/mlx5/mlx5_prm.h | 29 + > .../common/mlx5/rte_common_mlx5_version.map | 1 + > drivers/net/mlx5/mlx5.c | 1 + > drivers/net/mlx5/mlx5.h | 6 + > drivers/net/mlx5/mlx5_defs.h | 3 + > drivers/net/mlx5/mlx5_devx.c | 178 ++++- > drivers/net/mlx5/mlx5_flow.c | 497 ++++++++++++- > drivers/net/mlx5/mlx5_flow.h | 86 +++ > drivers/net/mlx5/mlx5_flow_dv.c | 684 +++++++++++++++++- > drivers/net/mlx5/mlx5_rxq.c | 103 +++ > drivers/net/mlx5/mlx5_rxtx.h | 5 +- > examples/flow_filtering/flow_blocks.c | 81 ++- > examples/flow_filtering/main.c | 13 +- > lib/librte_ethdev/rte_ethdev_version.map | 4 + > lib/librte_ethdev/rte_flow.c | 84 +++ > lib/librte_ethdev/rte_flow.h | 161 ++++- > lib/librte_ethdev/rte_flow_driver.h | 23 + > 23 files changed, 2489 insertions(+), 124 deletions(-) > > -- > 2.26.2 ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [dpdk-dev] [PATCH v3 00/10] RTE flow shared action 2020-10-04 11:14 ` [dpdk-dev] [PATCH v3 00/10] RTE flow " Ori Kam @ 2020-10-06 10:28 ` Andrey Vesnovaty 0 siblings, 0 replies; 106+ messages in thread From: Andrey Vesnovaty @ 2020-10-06 10:28 UTC (permalink / raw) To: Ori Kam, dev Cc: jer, jerinjacobk, NBU-Contact-Thomas Monjalon, ferruh.yigit, stephen, bruce.richardson, Slava Ovsiienko, andrey.vesnovaty, mdr, nhorman, ajit.khaparde, samik.gupta PSB > -----Original Message----- > From: Ori Kam <orika@nvidia.com> > Sent: Sunday, October 4, 2020 2:14 PM > To: Andrey Vesnovaty <andreyv@nvidia.com>; dev@dpdk.org > Cc: jer@marvell.com; jerinjacobk@gmail.com; NBU-Contact-Thomas Monjalon > <thomas@monjalon.net>; ferruh.yigit@intel.com; > stephen@networkplumber.org; bruce.richardson@intel.com; Slava Ovsiienko > <viacheslavo@nvidia.com>; andrey.vesnovaty@gmail.com; mdr@ashroe.eu; > nhorman@tuxdriver.com; ajit.khaparde@broadcom.com; > samik.gupta@broadcom.com > Subject: RE: [dpdk-dev] [PATCH v3 00/10] RTE flow shared action > > Hi Andrey, > > I suggest that you split this patchset to two different ones. > One for the Mellanox PMD > and one RTE just in each patch set reference the second patch. > This why it is possible to merge only the RTE level, if other PMD will use it. Got it. About to publish RTE level v4 part in next couple of hours. Mellanox PMD part will be published tomorrow. > > Best, > Ori > > > -----Original Message----- > > From: dev <dev-bounces@dpdk.org> On Behalf Of Andrey Vesnovaty > > Sent: Sunday, October 4, 2020 1:06 AM > > To: dev@dpdk.org > > Cc: jer@marvell.com; jerinjacobk@gmail.com; NBU-Contact-Thomas > Monjalon > > <thomas@monjalon.net>; ferruh.yigit@intel.com; > > stephen@networkplumber.org; bruce.richardson@intel.com; Ori Kam > > <orika@nvidia.com>; Slava Ovsiienko <viacheslavo@nvidia.com>; > > andrey.vesnovaty@gmail.com; mdr@ashroe.eu; nhorman@tuxdriver.com; > > ajit.khaparde@broadcom.com; samik.gupta@broadcom.com > > Subject: [dpdk-dev] [PATCH v3 00/10] RTE flow shared action > > > > This patchset introduces shared action for RTE flow. > > > > V3 changes: > > - implement testpmd for shared action > > - fix flow_filtering example application > > - add conf arg to shared action create API > > > > Notes: PMD implementation should be considered as draft > > > > Andrey Vesnovaty (10): > > ethdev: add flow shared action API > > ethdev: add conf arg to shared action icreate API > > common/mlx5: modify advanced Rx object via DevX > > net/mlx5: modify hash Rx queue objects > > net/mlx5: shared action PMD > > net/mlx5: shared action PMD create conf arg > > net/mlx5: driver support for shared action > > net/mlx5: shared action create conf drv support > > examples/flow_filtering: utilize shared RSS action > > app/testpmd: support shared action > > > > app/test-pmd/cmdline_flow.c | 262 ++++++- > > app/test-pmd/config.c | 217 ++++++ > > app/test-pmd/testpmd.h | 19 + > > doc/guides/sample_app_ug/flow_filtering.rst | 62 +- > > drivers/common/mlx5/mlx5_devx_cmds.c | 84 +++ > > drivers/common/mlx5/mlx5_devx_cmds.h | 10 + > > drivers/common/mlx5/mlx5_prm.h | 29 + > > .../common/mlx5/rte_common_mlx5_version.map | 1 + > > drivers/net/mlx5/mlx5.c | 1 + > > drivers/net/mlx5/mlx5.h | 6 + > > drivers/net/mlx5/mlx5_defs.h | 3 + > > drivers/net/mlx5/mlx5_devx.c | 178 ++++- > > drivers/net/mlx5/mlx5_flow.c | 497 ++++++++++++- > > drivers/net/mlx5/mlx5_flow.h | 86 +++ > > drivers/net/mlx5/mlx5_flow_dv.c | 684 +++++++++++++++++- > > drivers/net/mlx5/mlx5_rxq.c | 103 +++ > > drivers/net/mlx5/mlx5_rxtx.h | 5 +- > > examples/flow_filtering/flow_blocks.c | 81 ++- > > examples/flow_filtering/main.c | 13 +- > > lib/librte_ethdev/rte_ethdev_version.map | 4 + > > lib/librte_ethdev/rte_flow.c | 84 +++ > > lib/librte_ethdev/rte_flow.h | 161 ++++- > > lib/librte_ethdev/rte_flow_driver.h | 23 + > > 23 files changed, 2489 insertions(+), 124 deletions(-) > > > > -- > > 2.26.2 ^ permalink raw reply [flat|nested] 106+ messages in thread
* [dpdk-dev] [PATCH v2 2/6] common/mlx5: modify advanced Rx object via DevX 2020-07-08 21:39 ` [dpdk-dev] [PATCH v2 0/6] add flow shared action API + PMD Andrey Vesnovaty 2020-07-08 21:39 ` [dpdk-dev] [PATCH v2 1/6] ethdev: add flow shared action API Andrey Vesnovaty @ 2020-07-08 21:39 ` Andrey Vesnovaty 2020-07-08 21:39 ` [dpdk-dev] [PATCH v2 3/6] net/mlx5: modify hash Rx queue objects Andrey Vesnovaty ` (4 subsequent siblings) 6 siblings, 0 replies; 106+ messages in thread From: Andrey Vesnovaty @ 2020-07-08 21:39 UTC (permalink / raw) To: dev Cc: jer, jerinjacobk, thomas, ferruh.yigit, stephen, bruce.richardson, orika, viacheslavo, andrey.vesnovaty, Matan Azrad, Shahaf Shuler, Ray Kinsella, Neil Horman Implement mlx5_devx_cmd_modify_tir() to modify TIR object using DevX API. Add related structs in mlx5_prm.h. Signed-off-by: Andrey Vesnovaty <andreyv@mellanox.com> --- drivers/common/mlx5/mlx5_devx_cmds.c | 84 +++++++++++++++++++ drivers/common/mlx5/mlx5_devx_cmds.h | 10 +++ drivers/common/mlx5/mlx5_prm.h | 29 +++++++ .../common/mlx5/rte_common_mlx5_version.map | 1 + 4 files changed, 124 insertions(+) diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c index 2179a83983..2a7098ec6d 100644 --- a/drivers/common/mlx5/mlx5_devx_cmds.c +++ b/drivers/common/mlx5/mlx5_devx_cmds.c @@ -825,6 +825,90 @@ mlx5_devx_cmd_create_tir(void *ctx, return tir; } +/** + * Modify TIR using DevX API. + * + * @param[in] tir + * Pointer to TIR DevX object structure. + * @param [in] modify_tir_attr + * Pointer to TIR modification attributes structure. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +int +mlx5_devx_cmd_modify_tir(struct mlx5_devx_obj *tir, + struct mlx5_devx_modify_tir_attr *modify_tir_attr) +{ + struct mlx5_devx_tir_attr *tir_attr = &modify_tir_attr->tir; + uint32_t in[MLX5_ST_SZ_DW(modify_tir_in)] = {0}; + uint32_t out[MLX5_ST_SZ_DW(modify_tir_out)] = {0}; + void *tir_ctx; + int ret; + + MLX5_SET(modify_tir_in, in, opcode, MLX5_CMD_OP_MODIFY_TIR); + MLX5_SET(modify_tir_in, in, tirn, modify_tir_attr->tirn); + MLX5_SET64(modify_tir_in, in, modify_bitmask, + modify_tir_attr->modify_bitmask); + + tir_ctx = MLX5_ADDR_OF(modify_rq_in, in, ctx); + if (modify_tir_attr->modify_bitmask & + MLX5_MODIFY_TIR_IN_MODIFY_BITMASK_LRO) { + MLX5_SET(tirc, tir_ctx, lro_timeout_period_usecs, + tir_attr->lro_timeout_period_usecs); + MLX5_SET(tirc, tir_ctx, lro_enable_mask, + tir_attr->lro_enable_mask); + MLX5_SET(tirc, tir_ctx, lro_max_msg_sz, + tir_attr->lro_max_msg_sz); + } + if (modify_tir_attr->modify_bitmask & + MLX5_MODIFY_TIR_IN_MODIFY_BITMASK_INDIRECT_TABLE) + MLX5_SET(tirc, tir_ctx, indirect_table, + tir_attr->indirect_table); + if (modify_tir_attr->modify_bitmask & + MLX5_MODIFY_TIR_IN_MODIFY_BITMASK_HASH) { + int i; + void *outer, *inner; + MLX5_SET(tirc, tir_ctx, rx_hash_symmetric, + tir_attr->rx_hash_symmetric); + MLX5_SET(tirc, tir_ctx, rx_hash_fn, tir_attr->rx_hash_fn); + for (i = 0; i < 10; i++) { + MLX5_SET(tirc, tir_ctx, rx_hash_toeplitz_key[i], + tir_attr->rx_hash_toeplitz_key[i]); + } + outer = MLX5_ADDR_OF(tirc, tir_ctx, + rx_hash_field_selector_outer); + MLX5_SET(rx_hash_field_select, outer, l3_prot_type, + tir_attr->rx_hash_field_selector_outer.l3_prot_type); + MLX5_SET(rx_hash_field_select, outer, l4_prot_type, + tir_attr->rx_hash_field_selector_outer.l4_prot_type); + MLX5_SET + (rx_hash_field_select, outer, selected_fields, + tir_attr->rx_hash_field_selector_outer.selected_fields); + inner = MLX5_ADDR_OF(tirc, tir_ctx, + rx_hash_field_selector_inner); + MLX5_SET(rx_hash_field_select, inner, l3_prot_type, + tir_attr->rx_hash_field_selector_inner.l3_prot_type); + MLX5_SET(rx_hash_field_select, inner, l4_prot_type, + tir_attr->rx_hash_field_selector_inner.l4_prot_type); + MLX5_SET + (rx_hash_field_select, inner, selected_fields, + tir_attr->rx_hash_field_selector_inner.selected_fields); + } + if (modify_tir_attr->modify_bitmask & + MLX5_MODIFY_TIR_IN_MODIFY_BITMASK_SELF_LB_EN) { + MLX5_SET(tirc, tir_ctx, self_lb_block, tir_attr->self_lb_block); + } + ret = mlx5_glue->devx_obj_modify(tir->obj, in, sizeof(in), + out, sizeof(out)); + if (ret) { + DRV_LOG(ERR, "Failed to modify TIR using DevX"); + rte_errno = errno; + return -errno; + } + return ret; +} + /** * Create RQT using DevX API. * diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h index 25704efc1f..9b07b39e66 100644 --- a/drivers/common/mlx5/mlx5_devx_cmds.h +++ b/drivers/common/mlx5/mlx5_devx_cmds.h @@ -178,6 +178,13 @@ struct mlx5_devx_tir_attr { struct mlx5_rx_hash_field_select rx_hash_field_selector_inner; }; +/* TIR attributes structure, used by TIR modify */ +struct mlx5_devx_modify_tir_attr { + uint32_t tirn:24; + uint64_t modify_bitmask; + struct mlx5_devx_tir_attr tir; +}; + /* RQT attributes structure, used by RQT operations. */ struct mlx5_devx_rqt_attr { uint8_t rq_type; @@ -372,6 +379,9 @@ int mlx5_devx_cmd_modify_qp_state(struct mlx5_devx_obj *qp, __rte_internal int mlx5_devx_cmd_modify_rqt(struct mlx5_devx_obj *rqt, struct mlx5_devx_rqt_attr *rqt_attr); +__rte_internal +int mlx5_devx_cmd_modify_tir(struct mlx5_devx_obj *tir, + struct mlx5_devx_modify_tir_attr *tir_attr); /** * Create virtio queue counters object DevX API. diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h index c63795fc84..029767ea34 100644 --- a/drivers/common/mlx5/mlx5_prm.h +++ b/drivers/common/mlx5/mlx5_prm.h @@ -746,6 +746,7 @@ enum { MLX5_CMD_OP_QUERY_NIC_VPORT_CONTEXT = 0x754, MLX5_CMD_OP_ALLOC_TRANSPORT_DOMAIN = 0x816, MLX5_CMD_OP_CREATE_TIR = 0x900, + MLX5_CMD_OP_MODIFY_TIR = 0x901, MLX5_CMD_OP_CREATE_SQ = 0X904, MLX5_CMD_OP_MODIFY_SQ = 0X905, MLX5_CMD_OP_CREATE_RQ = 0x908, @@ -1753,6 +1754,34 @@ struct mlx5_ifc_create_tir_in_bits { struct mlx5_ifc_tirc_bits ctx; }; +enum { + MLX5_MODIFY_TIR_IN_MODIFY_BITMASK_LRO = 1ULL << 0, + MLX5_MODIFY_TIR_IN_MODIFY_BITMASK_INDIRECT_TABLE = 1ULL << 1, + MLX5_MODIFY_TIR_IN_MODIFY_BITMASK_HASH = 1ULL << 2, + /* bit 3 - tunneled_offload_en modify not supported */ + MLX5_MODIFY_TIR_IN_MODIFY_BITMASK_SELF_LB_EN = 1ULL << 4, +}; + +struct mlx5_ifc_modify_tir_out_bits { + u8 status[0x8]; + u8 reserved_at_8[0x18]; + u8 syndrome[0x20]; + u8 reserved_at_40[0x40]; +}; + +struct mlx5_ifc_modify_tir_in_bits { + u8 opcode[0x10]; + u8 uid[0x10]; + u8 reserved_at_20[0x10]; + u8 op_mod[0x10]; + u8 reserved_at_40[0x8]; + u8 tirn[0x18]; + u8 reserved_at_60[0x20]; + u8 modify_bitmask[0x40]; + u8 reserved_at_c0[0x40]; + struct mlx5_ifc_tirc_bits ctx; +}; + enum { MLX5_INLINE_Q_TYPE_RQ = 0x0, MLX5_INLINE_Q_TYPE_VIRTQ = 0x1, diff --git a/drivers/common/mlx5/rte_common_mlx5_version.map b/drivers/common/mlx5/rte_common_mlx5_version.map index ae57ebdba5..0bfedab32f 100644 --- a/drivers/common/mlx5/rte_common_mlx5_version.map +++ b/drivers/common/mlx5/rte_common_mlx5_version.map @@ -29,6 +29,7 @@ INTERNAL { mlx5_devx_cmd_modify_rq; mlx5_devx_cmd_modify_rqt; mlx5_devx_cmd_modify_sq; + mlx5_devx_cmd_modify_tir; mlx5_devx_cmd_modify_virtq; mlx5_devx_cmd_qp_query_tis_td; mlx5_devx_cmd_query_hca_attr; -- 2.26.2 ^ permalink raw reply [flat|nested] 106+ messages in thread
* [dpdk-dev] [PATCH v2 3/6] net/mlx5: modify hash Rx queue objects 2020-07-08 21:39 ` [dpdk-dev] [PATCH v2 0/6] add flow shared action API + PMD Andrey Vesnovaty 2020-07-08 21:39 ` [dpdk-dev] [PATCH v2 1/6] ethdev: add flow shared action API Andrey Vesnovaty 2020-07-08 21:39 ` [dpdk-dev] [PATCH v2 2/6] common/mlx5: modify advanced Rx object via DevX Andrey Vesnovaty @ 2020-07-08 21:39 ` Andrey Vesnovaty 2020-07-08 21:39 ` [dpdk-dev] [PATCH v2 4/6] net/mlx5: shared action PMD Andrey Vesnovaty ` (3 subsequent siblings) 6 siblings, 0 replies; 106+ messages in thread From: Andrey Vesnovaty @ 2020-07-08 21:39 UTC (permalink / raw) To: dev Cc: jer, jerinjacobk, thomas, ferruh.yigit, stephen, bruce.richardson, orika, viacheslavo, andrey.vesnovaty, Matan Azrad, Shahaf Shuler Implement mlx5_hrxq_modify() to modify hash RX queue object. This commit relays on capability to modify TIR object via DevX. Signed-off-by: Andrey Vesnovaty <andreyv@mellanox.com> --- drivers/net/mlx5/mlx5_rxq.c | 300 ++++++++++++++++++++++++++++------- drivers/net/mlx5/mlx5_rxtx.h | 4 + 2 files changed, 243 insertions(+), 61 deletions(-) diff --git a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c index b436f06107..80c402c4b7 100644 --- a/drivers/net/mlx5/mlx5_rxq.c +++ b/drivers/net/mlx5/mlx5_rxq.c @@ -2274,6 +2274,29 @@ mlx5_ind_table_obj_new(struct rte_eth_dev *dev, const uint16_t *queues, return NULL; } +/** + * Match queues listed in arguments to queues contained in indirection table + * object. + * + * @param ind_tbl + * Pointer to indirection table to match. + * @param queues + * Queues to match to ques in indirection table. + * @param queues_n + * Number of queues in the array. + * + * @return + * 1 if all queues in indirection table match 0 othrwise. + */ +static int +mlx5_ind_table_obj_match_queues(const struct mlx5_ind_table_obj *ind_tbl, + const uint16_t *queues, uint32_t queues_n) +{ + return (ind_tbl->queues_n == queues_n) && + (!memcmp(ind_tbl->queues, queues, + ind_tbl->queues_n * sizeof(ind_tbl->queues[0]))); +} + /** * Get an indirection table. * @@ -2370,6 +2393,102 @@ mlx5_ind_table_obj_verify(struct rte_eth_dev *dev) return ret; } +/* + * Set TIR attribute struct with relevant input values. + * + * @param[in] dev + * Pointer to Ethernet device. + * @param[in] rss_key + * RSS key for the Rx hash queue. + * @param[in] rss_key_len + * RSS key length. + * @param[in] hash_fields + * Verbs protocol hash field to make the RSS on. + * @param[in] queues + * Queues entering in hash queue. In case of empty hash_fields only the + * first queue index will be taken for the indirection table. + * @param[in] queues_n + * Number of queues. + * @param[in] tunnel + * Tunnel type. + * @param[out] tir_attr + * Parameters structure for TIR creation/modification. + * + * @return + * The Verbs/DevX object initialised index, 0 otherwise and rte_errno is set. + */ +static void +mlx5_devx_tir_attr_set(struct rte_eth_dev *dev, + const uint8_t *rss_key, uint32_t rss_key_len, + uint64_t hash_fields, + const uint16_t *queues, uint32_t queues_n, + int tunnel, + enum mlx5_rxq_obj_type rxq_obj_type, int ind_tbl_id, + struct mlx5_devx_tir_attr *tir_attr) +{ + struct mlx5_priv *priv = dev->data->dev_private; + uint32_t i; + uint32_t lro = 1; + + /* Enable TIR LRO only if all the queues were configured for. */ + for (i = 0; i < queues_n; ++i) { + if (!(*priv->rxqs)[queues[i]]->lro) { + lro = 0; + break; + } + } + memset(tir_attr, 0, sizeof(*tir_attr)); + tir_attr->disp_type = MLX5_TIRC_DISP_TYPE_INDIRECT; + tir_attr->rx_hash_fn = MLX5_RX_HASH_FN_TOEPLITZ; + tir_attr->tunneled_offload_en = !!tunnel; + /* If needed, translate hash_fields bitmap to PRM format. */ + if (hash_fields) { +#ifdef HAVE_IBV_DEVICE_TUNNEL_SUPPORT + struct mlx5_rx_hash_field_select *rx_hash_field_select = + hash_fields & IBV_RX_HASH_INNER ? + &tir_attr->rx_hash_field_selector_inner : + &tir_attr->rx_hash_field_selector_outer; +#else + struct mlx5_rx_hash_field_select *rx_hash_field_select = + &tir_attr->rx_hash_field_selector_outer; +#endif + + /* 1 bit: 0: IPv4, 1: IPv6. */ + rx_hash_field_select->l3_prot_type = + !!(hash_fields & MLX5_IPV6_IBV_RX_HASH); + /* 1 bit: 0: TCP, 1: UDP. */ + rx_hash_field_select->l4_prot_type = + !!(hash_fields & MLX5_UDP_IBV_RX_HASH); + /* Bitmask which sets which fields to use in RX Hash. */ + rx_hash_field_select->selected_fields = + ((!!(hash_fields & MLX5_L3_SRC_IBV_RX_HASH)) << + MLX5_RX_HASH_FIELD_SELECT_SELECTED_FIELDS_SRC_IP) | + (!!(hash_fields & MLX5_L3_DST_IBV_RX_HASH)) << + MLX5_RX_HASH_FIELD_SELECT_SELECTED_FIELDS_DST_IP | + (!!(hash_fields & MLX5_L4_SRC_IBV_RX_HASH)) << + MLX5_RX_HASH_FIELD_SELECT_SELECTED_FIELDS_L4_SPORT | + (!!(hash_fields & MLX5_L4_DST_IBV_RX_HASH)) << + MLX5_RX_HASH_FIELD_SELECT_SELECTED_FIELDS_L4_DPORT; + } + if (rxq_obj_type == MLX5_RXQ_OBJ_TYPE_DEVX_HAIRPIN) + tir_attr->transport_domain = priv->sh->td->id; + else + tir_attr->transport_domain = priv->sh->tdn; + memcpy(tir_attr->rx_hash_toeplitz_key, rss_key, rss_key_len); + tir_attr->indirect_table = ind_tbl_id; + if (dev->data->dev_conf.lpbk_mode) + tir_attr->self_lb_block = + MLX5_TIRC_SELF_LB_BLOCK_BLOCK_UNICAST; + if (lro) { + tir_attr->lro_timeout_period_usecs = + priv->config.lro.timeout; + tir_attr->lro_max_msg_sz = priv->max_lro_msg_size; + tir_attr->lro_enable_mask = + MLX5_TIRC_LRO_ENABLE_MASK_IPV4_LRO | + MLX5_TIRC_LRO_ENABLE_MASK_IPV6_LRO; + } +} + /** * Create an Rx Hash queue. * @@ -2493,67 +2612,11 @@ mlx5_hrxq_new(struct rte_eth_dev *dev, } } else { /* ind_tbl->type == MLX5_IND_TBL_TYPE_DEVX */ struct mlx5_devx_tir_attr tir_attr; - uint32_t i; - uint32_t lro = 1; - - /* Enable TIR LRO only if all the queues were configured for. */ - for (i = 0; i < queues_n; ++i) { - if (!(*priv->rxqs)[queues[i]]->lro) { - lro = 0; - break; - } - } - memset(&tir_attr, 0, sizeof(tir_attr)); - tir_attr.disp_type = MLX5_TIRC_DISP_TYPE_INDIRECT; - tir_attr.rx_hash_fn = MLX5_RX_HASH_FN_TOEPLITZ; - tir_attr.tunneled_offload_en = !!tunnel; - /* If needed, translate hash_fields bitmap to PRM format. */ - if (hash_fields) { -#ifdef HAVE_IBV_DEVICE_TUNNEL_SUPPORT - struct mlx5_rx_hash_field_select *rx_hash_field_select = - hash_fields & IBV_RX_HASH_INNER ? - &tir_attr.rx_hash_field_selector_inner : - &tir_attr.rx_hash_field_selector_outer; -#else - struct mlx5_rx_hash_field_select *rx_hash_field_select = - &tir_attr.rx_hash_field_selector_outer; -#endif - - /* 1 bit: 0: IPv4, 1: IPv6. */ - rx_hash_field_select->l3_prot_type = - !!(hash_fields & MLX5_IPV6_IBV_RX_HASH); - /* 1 bit: 0: TCP, 1: UDP. */ - rx_hash_field_select->l4_prot_type = - !!(hash_fields & MLX5_UDP_IBV_RX_HASH); - /* Bitmask which sets which fields to use in RX Hash. */ - rx_hash_field_select->selected_fields = - ((!!(hash_fields & MLX5_L3_SRC_IBV_RX_HASH)) << - MLX5_RX_HASH_FIELD_SELECT_SELECTED_FIELDS_SRC_IP) | - (!!(hash_fields & MLX5_L3_DST_IBV_RX_HASH)) << - MLX5_RX_HASH_FIELD_SELECT_SELECTED_FIELDS_DST_IP | - (!!(hash_fields & MLX5_L4_SRC_IBV_RX_HASH)) << - MLX5_RX_HASH_FIELD_SELECT_SELECTED_FIELDS_L4_SPORT | - (!!(hash_fields & MLX5_L4_DST_IBV_RX_HASH)) << - MLX5_RX_HASH_FIELD_SELECT_SELECTED_FIELDS_L4_DPORT; - } - if (rxq_ctrl->obj->type == MLX5_RXQ_OBJ_TYPE_DEVX_HAIRPIN) - tir_attr.transport_domain = priv->sh->td->id; - else - tir_attr.transport_domain = priv->sh->tdn; - memcpy(tir_attr.rx_hash_toeplitz_key, rss_key, - MLX5_RSS_HASH_KEY_LEN); - tir_attr.indirect_table = ind_tbl->rqt->id; - if (dev->data->dev_conf.lpbk_mode) - tir_attr.self_lb_block = - MLX5_TIRC_SELF_LB_BLOCK_BLOCK_UNICAST; - if (lro) { - tir_attr.lro_timeout_period_usecs = - priv->config.lro.timeout; - tir_attr.lro_max_msg_sz = priv->max_lro_msg_size; - tir_attr.lro_enable_mask = - MLX5_TIRC_LRO_ENABLE_MASK_IPV4_LRO | - MLX5_TIRC_LRO_ENABLE_MASK_IPV6_LRO; - } + mlx5_devx_tir_attr_set + (dev, rss_key, rss_key_len, hash_fields, + queues, queues_n, tunnel, + rxq_ctrl->obj->type, ind_tbl->rqt->id, + &tir_attr); tir = mlx5_devx_cmd_create_tir(priv->sh->ctx, &tir_attr); if (!tir) { DRV_LOG(ERR, "port %u cannot create DevX TIR", @@ -2616,6 +2679,7 @@ mlx5_hrxq_new(struct rte_eth_dev *dev, * Queues entering in hash queue. In case of empty hash_fields only the * first queue index will be taken for the indirection table. * @param queues_n + * * Number of queues. * * @return @@ -2655,6 +2719,120 @@ mlx5_hrxq_get(struct rte_eth_dev *dev, return 0; } +/** + * Modify an Rx Hash queue configuration. + * + * @param dev + * Pointer to Ethernet device. + * @param hrxq + * Index to Hash Rx queue to modify. + * @param rss_key + * RSS key for the Rx hash queue. + * @param rss_key_len + * RSS key length. + * @param hash_fields + * Verbs protocol hash field to make the RSS on. + * @param queues + * Queues entering in hash queue. In case of empty hash_fields only the + * first queue index will be taken for the indirection table. + * @param queues_n + * Number of queues. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +int +mlx5_hrxq_modify(struct rte_eth_dev *dev, uint32_t hrxq_idx, + const uint8_t *rss_key, uint32_t rss_key_len, + uint64_t hash_fields, + const uint16_t *queues, uint32_t queues_n) +{ + int err; + struct mlx5_priv *priv = dev->data->dev_private; + struct mlx5_rxq_data *rxq_data = (*priv->rxqs)[queues[0]]; + struct mlx5_rxq_ctrl *rxq_ctrl = + container_of(rxq_data, struct mlx5_rxq_ctrl, rxq); + struct mlx5_devx_modify_tir_attr modify_tir = {0}; + struct mlx5_ind_table_obj *ind_tbl = NULL; + enum mlx5_ind_tbl_type rxq_obj_type = + rxq_ctrl->obj->type == MLX5_RXQ_OBJ_TYPE_IBV ? + MLX5_IND_TBL_TYPE_IBV : MLX5_IND_TBL_TYPE_DEVX; + struct mlx5_hrxq *hrxq = + mlx5_ipool_get(priv->sh->ipool[MLX5_IPOOL_HRXQ], hrxq_idx); + + if (!hrxq) { + rte_errno = EINVAL; + return -rte_errno; + } + /* validations */ + if (hrxq->ind_table->type != MLX5_IND_TBL_TYPE_DEVX || + rxq_obj_type != MLX5_IND_TBL_TYPE_DEVX) { + /* shared action supported by devx interface only */ + rte_errno = EINVAL; + return -rte_errno; + } + if (hrxq->rss_key_len != rss_key_len) { + /* rss_key_len is fixed size 40 byte & not supposed to change */ + rte_errno = EINVAL; + return -rte_errno; + } + + queues_n = hash_fields ? queues_n : 1; + if (mlx5_ind_table_obj_match_queues(hrxq->ind_table, + queues, queues_n)) { + ind_tbl = hrxq->ind_table; + } else { + ind_tbl = mlx5_ind_table_obj_get(dev, queues, queues_n); + if (!ind_tbl) + ind_tbl = mlx5_ind_table_obj_new(dev, queues, queues_n, + rxq_obj_type); + } + if (!ind_tbl) { + rte_errno = ENOMEM; + return -rte_errno; + } + + /* + * untested for modification fields: + * - rx_hash_symmetric not set in hrxq_new(), + * - rx_hash_fn set hard-coded in hrxq_new(), + * - lro_xxx not set after rxq setup + */ + modify_tir.modify_bitmask |= + (MLX5_MODIFY_TIR_IN_MODIFY_BITMASK_INDIRECT_TABLE & + !!(ind_tbl != hrxq->ind_table)); + modify_tir.modify_bitmask |= + (MLX5_MODIFY_TIR_IN_MODIFY_BITMASK_HASH & + !!(hash_fields != hrxq->hash_fields || + hrxq->rss_key_len != rss_key_len || + memcmp(hrxq->rss_key, rss_key, rss_key_len))); + + mlx5_devx_tir_attr_set(dev, rss_key, rss_key_len, hash_fields, + queues, queues_n, + 0, /* N/A - tunnel modification unsupported */ + rxq_obj_type, ind_tbl->rqt->id, + &modify_tir.tir); + if (mlx5_devx_cmd_modify_tir(hrxq->tir, &modify_tir)) { + DRV_LOG(ERR, "port %u cannot modify DevX TIR", + dev->data->port_id); + rte_errno = errno; + goto error; + } + if (ind_tbl != hrxq->ind_table) { + mlx5_ind_table_obj_release(dev, hrxq->ind_table); + hrxq->ind_table = ind_tbl; + } + hrxq->hash_fields = hash_fields; + memcpy(hrxq->rss_key, rss_key, rss_key_len); + return 0; +error: + err = rte_errno; + if (ind_tbl != hrxq->ind_table) + mlx5_ind_table_obj_release(dev, ind_tbl); + rte_errno = err; + return -rte_errno; +} + /** * Release the hash Rx queue. * diff --git a/drivers/net/mlx5/mlx5_rxtx.h b/drivers/net/mlx5/mlx5_rxtx.h index 26621ff193..5cff28196c 100644 --- a/drivers/net/mlx5/mlx5_rxtx.h +++ b/drivers/net/mlx5/mlx5_rxtx.h @@ -424,6 +424,10 @@ struct mlx5_hrxq *mlx5_hrxq_drop_new(struct rte_eth_dev *dev); void mlx5_hrxq_drop_release(struct rte_eth_dev *dev); uint64_t mlx5_get_rx_port_offloads(void); uint64_t mlx5_get_rx_queue_offloads(struct rte_eth_dev *dev); +int mlx5_hrxq_modify(struct rte_eth_dev *dev, uint32_t hxrq_idx, + const uint8_t *rss_key, uint32_t rss_key_len, + uint64_t hash_fields, + const uint16_t *queues, uint32_t queues_n); /* mlx5_txq.c */ -- 2.26.2 ^ permalink raw reply [flat|nested] 106+ messages in thread
* [dpdk-dev] [PATCH v2 4/6] net/mlx5: shared action PMD 2020-07-08 21:39 ` [dpdk-dev] [PATCH v2 0/6] add flow shared action API + PMD Andrey Vesnovaty ` (2 preceding siblings ...) 2020-07-08 21:39 ` [dpdk-dev] [PATCH v2 3/6] net/mlx5: modify hash Rx queue objects Andrey Vesnovaty @ 2020-07-08 21:39 ` Andrey Vesnovaty 2020-07-08 21:39 ` [dpdk-dev] [PATCH v2 5/6] net/mlx5: driver support for shared action Andrey Vesnovaty ` (2 subsequent siblings) 6 siblings, 0 replies; 106+ messages in thread From: Andrey Vesnovaty @ 2020-07-08 21:39 UTC (permalink / raw) To: dev Cc: jer, jerinjacobk, thomas, ferruh.yigit, stephen, bruce.richardson, orika, viacheslavo, andrey.vesnovaty, Matan Azrad, Shahaf Shuler Implement rte_flow shared action API for mlx5 PMD. Handle shared action on flow create/destroy. Signed-off-by: Andrey Vesnovaty <andreyv@mellanox.com> --- drivers/net/mlx5/mlx5.c | 1 + drivers/net/mlx5/mlx5.h | 2 + drivers/net/mlx5/mlx5_defs.h | 3 + drivers/net/mlx5/mlx5_flow.c | 492 ++++++++++++++++++++++++++++++++--- drivers/net/mlx5/mlx5_flow.h | 83 ++++++ 5 files changed, 549 insertions(+), 32 deletions(-) diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index 86b7671b4d..ea6af3775b 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -1147,6 +1147,7 @@ mlx5_dev_close(struct rte_eth_dev *dev) * then this will return directly without any action. */ mlx5_flow_list_flush(dev, &priv->flows, true); + mlx5_shared_action_flush(dev); mlx5_flow_meter_flush(dev, NULL); /* Free the intermediate buffers for flow creation. */ mlx5_flow_free_intermediate(dev); diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 46e66eb1c6..4add24bd79 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -680,6 +680,8 @@ struct mlx5_priv { uint8_t fdb_def_rule; /* Whether fdb jump to table 1 is configured. */ struct mlx5_mp_id mp_id; /* ID of a multi-process process */ LIST_HEAD(fdir, mlx5_fdir_flow) fdir_flows; /* fdir flows. */ + LIST_HEAD(shared_action, rte_flow_shared_action) shared_actions; + /* shared actions */ }; #define PORT_ID(priv) ((priv)->dev_data->port_id) diff --git a/drivers/net/mlx5/mlx5_defs.h b/drivers/net/mlx5/mlx5_defs.h index 260f584298..dc56c77b59 100644 --- a/drivers/net/mlx5/mlx5_defs.h +++ b/drivers/net/mlx5/mlx5_defs.h @@ -180,6 +180,9 @@ #define MLX5_HAIRPIN_QUEUE_STRIDE 6 #define MLX5_HAIRPIN_JUMBO_LOG_SIZE (14 + 2) +/* Maximum number of shared actions supported by rte_flow */ +#define MLX5_MAX_SHARED_ACTIONS 1 + /* Definition of static_assert found in /usr/include/assert.h */ #ifndef HAVE_STATIC_ASSERT #define static_assert _Static_assert diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index 4a031a523b..5947b4b1ff 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -231,6 +231,25 @@ static const struct rte_flow_expand_node mlx5_support_expansion[] = { }, }; +static struct rte_flow_shared_action * +mlx5_shared_action_create(struct rte_eth_dev *dev, + const struct rte_flow_action *action, + struct rte_flow_error *error); +static int mlx5_shared_action_destroy + (struct rte_eth_dev *dev, + struct rte_flow_shared_action *shared_action, + struct rte_flow_error *error); +static int mlx5_shared_action_update + (struct rte_eth_dev *dev, + struct rte_flow_shared_action *shared_action, + const struct rte_flow_action *action, + struct rte_flow_error *error); +static int mlx5_shared_action_query + (struct rte_eth_dev *dev, + const struct rte_flow_shared_action *action, + void *data, + struct rte_flow_error *error); + static const struct rte_flow_ops mlx5_flow_ops = { .validate = mlx5_flow_validate, .create = mlx5_flow_create, @@ -240,6 +259,10 @@ static const struct rte_flow_ops mlx5_flow_ops = { .query = mlx5_flow_query, .dev_dump = mlx5_flow_dev_dump, .get_aged_flows = mlx5_flow_get_aged_flows, + .shared_action_create = mlx5_shared_action_create, + .shared_action_destroy = mlx5_shared_action_destroy, + .shared_action_update = mlx5_shared_action_update, + .shared_action_query = mlx5_shared_action_query, }; /* Convert FDIR request to Generic flow. */ @@ -1117,16 +1140,10 @@ mlx5_flow_validate_action_queue(const struct rte_flow_action *action, /* * Validate the rss action. * - * @param[in] action - * Pointer to the queue action. - * @param[in] action_flags - * Bit-fields that holds the actions detected until now. * @param[in] dev * Pointer to the Ethernet device structure. - * @param[in] attr - * Attributes of flow that includes this action. - * @param[in] item_flags - * Items that were detected. + * @param[in] action + * Pointer to the queue action. * @param[out] error * Pointer to error structure. * @@ -1134,23 +1151,14 @@ mlx5_flow_validate_action_queue(const struct rte_flow_action *action, * 0 on success, a negative errno value otherwise and rte_errno is set. */ int -mlx5_flow_validate_action_rss(const struct rte_flow_action *action, - uint64_t action_flags, - struct rte_eth_dev *dev, - const struct rte_flow_attr *attr, - uint64_t item_flags, - struct rte_flow_error *error) +mlx5_validate_action_rss(struct rte_eth_dev *dev, + const struct rte_flow_action *action, + struct rte_flow_error *error) { struct mlx5_priv *priv = dev->data->dev_private; const struct rte_flow_action_rss *rss = action->conf; - int tunnel = !!(item_flags & MLX5_FLOW_LAYER_TUNNEL); unsigned int i; - if (action_flags & MLX5_FLOW_FATE_ACTIONS) - return rte_flow_error_set(error, EINVAL, - RTE_FLOW_ERROR_TYPE_ACTION, NULL, - "can't have 2 fate actions" - " in same flow"); if (rss->func != RTE_ETH_HASH_FUNCTION_DEFAULT && rss->func != RTE_ETH_HASH_FUNCTION_TOEPLITZ) return rte_flow_error_set(error, ENOTSUP, @@ -1196,15 +1204,17 @@ mlx5_flow_validate_action_rss(const struct rte_flow_action *action, if ((rss->types & (ETH_RSS_L3_SRC_ONLY | ETH_RSS_L3_DST_ONLY)) && !(rss->types & ETH_RSS_IP)) return rte_flow_error_set(error, EINVAL, - RTE_FLOW_ERROR_TYPE_ACTION_CONF, NULL, - "L3 partial RSS requested but L3 RSS" - " type not specified"); + RTE_FLOW_ERROR_TYPE_ACTION_CONF, + NULL, + "L3 partial RSS requested but L3 " + "RSS type not specified"); if ((rss->types & (ETH_RSS_L4_SRC_ONLY | ETH_RSS_L4_DST_ONLY)) && !(rss->types & (ETH_RSS_UDP | ETH_RSS_TCP))) return rte_flow_error_set(error, EINVAL, - RTE_FLOW_ERROR_TYPE_ACTION_CONF, NULL, - "L4 partial RSS requested but L4 RSS" - " type not specified"); + RTE_FLOW_ERROR_TYPE_ACTION_CONF, + NULL, + "L4 partial RSS requested but L4 " + "RSS type not specified"); if (!priv->rxqs_n) return rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ACTION_CONF, @@ -1221,17 +1231,62 @@ mlx5_flow_validate_action_rss(const struct rte_flow_action *action, &rss->queue[i], "queue index out of range"); if (!(*priv->rxqs)[rss->queue[i]]) return rte_flow_error_set - (error, EINVAL, RTE_FLOW_ERROR_TYPE_ACTION_CONF, + (error, EINVAL, + RTE_FLOW_ERROR_TYPE_ACTION_CONF, &rss->queue[i], "queue is not configured"); } + return 0; +} + +/* + * Validate the rss action. + * + * @param[in] action + * Pointer to the queue action. + * @param[in] action_flags + * Bit-fields that holds the actions detected until now. + * @param[in] dev + * Pointer to the Ethernet device structure. + * @param[in] attr + * Attributes of flow that includes this action. + * @param[in] item_flags + * Items that were detected. + * @param[out] error + * Pointer to error structure. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +int +mlx5_flow_validate_action_rss(const struct rte_flow_action *action, + uint64_t action_flags, + struct rte_eth_dev *dev, + const struct rte_flow_attr *attr, + uint64_t item_flags, + struct rte_flow_error *error) +{ + const struct rte_flow_action_rss *rss = action->conf; + int tunnel = !!(item_flags & MLX5_FLOW_LAYER_TUNNEL); + int ret; + + if (action_flags & MLX5_FLOW_FATE_ACTIONS) + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ACTION, NULL, + "can't have 2 fate actions" + " in same flow"); + ret = mlx5_validate_action_rss(dev, action, error); + if (ret) + return ret; if (attr->egress) return rte_flow_error_set(error, ENOTSUP, - RTE_FLOW_ERROR_TYPE_ATTR_EGRESS, NULL, + RTE_FLOW_ERROR_TYPE_ATTR_EGRESS, + NULL, "rss action not supported for " "egress"); if (rss->level > 1 && !tunnel) return rte_flow_error_set(error, EINVAL, - RTE_FLOW_ERROR_TYPE_ACTION_CONF, NULL, + RTE_FLOW_ERROR_TYPE_ACTION_CONF, + NULL, "inner RSS is not supported for " "non-tunnel flows"); return 0; @@ -2739,6 +2794,131 @@ flow_get_rss_action(const struct rte_flow_action actions[]) return NULL; } +/* maps shared action to translated non shared in some actions array */ +struct mlx5_translated_shared_action { + struct rte_flow_shared_action *action; /**< Shared action */ + int index; /**< Index in related array of rte_flow_action */ +}; + +/** + * Translates actions of type RTE_FLOW_ACTION_TYPE_SHARED to related + * non shared action if translation possible. + * This functionality used to run same execution path for both shared & non + * shared actions on flow create. All necessary preparations for shared + * action handling should be preformed on *shared* actions list returned by + * from this call. + * + * @param[in] actions + * List of actions to translate. + * @param[out] shared + * List to store translated shared actions. + * @param[in, out] shared_n + * Size of *shared* array. On return should be updated with number of shared + * actions retrieved from the *actions* list. + * @param[out] translated_actions + * List of actions where all shared actions were translated to non shared + * if possible. NULL if no translation took place. + * @param[out] error + * Pointer to the error structure. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +flow_shared_actions_translate(const struct rte_flow_action actions[], + struct mlx5_translated_shared_action *shared, + int *shared_n, + struct rte_flow_action **translated_actions, + struct rte_flow_error *error) +{ + struct rte_flow_action *translated = NULL; + int n; + int copied_n = 0; + struct mlx5_translated_shared_action *shared_end = NULL; + + for (n = 0; actions[n].type != RTE_FLOW_ACTION_TYPE_END; n++) { + if (actions[n].type != RTE_FLOW_ACTION_TYPE_SHARED) + continue; + if (copied_n == *shared_n) { + return rte_flow_error_set + (error, EINVAL, RTE_FLOW_ERROR_TYPE_ACTION_NUM, + NULL, "too many shared actions"); + } + rte_memcpy(&shared[copied_n].action, &actions[n].conf, + sizeof(actions[n].conf)); + shared[copied_n].index = n; + copied_n++; + } + n++; + *shared_n = copied_n; + if (!copied_n) + return 0; + translated = rte_calloc(__func__, n, sizeof(struct rte_flow_action), 0); + rte_memcpy(translated, actions, n * sizeof(struct rte_flow_action)); + for (shared_end = shared + copied_n; shared < shared_end; shared++) { + const struct rte_flow_shared_action *shared_action; + + shared_action = shared->action; + switch (shared_action->type) { + case MLX5_FLOW_ACTION_SHARED_RSS: + translated[shared->index].type = + RTE_FLOW_ACTION_TYPE_RSS; + translated[shared->index].conf = + &shared_action->rss.origin; + break; + default: + rte_free(translated); + return rte_flow_error_set + (error, EINVAL, RTE_FLOW_ERROR_TYPE_ACTION, + NULL, "invalid shared action type"); + } + } + *translated_actions = translated; + return 0; +} + +/** + * Get Shared RSS action from the action list. + * + * @param[in] shared + * Pointer to the list of actions. + * @param[in] shared_n + * Actions list length. + * + * @return + * Pointer to the MLX5 RSS action if exist, else return NULL. + */ +static struct mlx5_shared_action_rss * +flow_get_shared_rss_action(struct mlx5_translated_shared_action *shared, + int shared_n) +{ + struct mlx5_translated_shared_action *shared_end; + + for (shared_end = shared + shared_n; shared < shared_end; shared++) { + struct rte_flow_shared_action *shared_action; + + shared_action = shared->action; + switch (shared_action->type) { + case MLX5_FLOW_ACTION_SHARED_RSS: + rte_atomic32_inc(&shared_action->refcnt); + return &shared_action->rss; + default: + break; + } + } + return NULL; +} + +struct rte_flow_shared_action * +mlx5_flow_get_shared_rss(struct rte_flow *flow) +{ + if (flow->shared_rss) + return container_of(flow->shared_rss, + struct rte_flow_shared_action, rss); + else + return NULL; +} + static unsigned int find_graph_root(const struct rte_flow_item pattern[], uint32_t rss_level) { @@ -4328,13 +4508,16 @@ static uint32_t flow_list_create(struct rte_eth_dev *dev, uint32_t *list, const struct rte_flow_attr *attr, const struct rte_flow_item items[], - const struct rte_flow_action actions[], + const struct rte_flow_action original_actions[], bool external, struct rte_flow_error *error) { struct mlx5_priv *priv = dev->data->dev_private; struct rte_flow *flow = NULL; struct mlx5_flow *dev_flow; const struct rte_flow_action_rss *rss; + struct mlx5_translated_shared_action + shared_actions[MLX5_MAX_SHARED_ACTIONS]; + int shared_actions_n = MLX5_MAX_SHARED_ACTIONS; union { struct rte_flow_expand_rss buf; uint8_t buffer[2048]; @@ -4354,14 +4537,23 @@ flow_list_create(struct rte_eth_dev *dev, uint32_t *list, struct rte_flow_expand_rss *buf = &expand_buffer.buf; struct mlx5_flow_rss_desc *rss_desc = &((struct mlx5_flow_rss_desc *) priv->rss_desc)[!!priv->flow_idx]; - const struct rte_flow_action *p_actions_rx = actions; + const struct rte_flow_action *p_actions_rx; uint32_t i; uint32_t idx = 0; int hairpin_flow; uint32_t hairpin_id = 0; struct rte_flow_attr attr_tx = { .priority = 0 }; - int ret; + const struct rte_flow_action *actions; + struct rte_flow_action *translated_actions = NULL; + int ret = flow_shared_actions_translate(original_actions, + shared_actions, + &shared_actions_n, + &translated_actions, error); + if (ret < 0) + return 0; + actions = (translated_actions) ? translated_actions : original_actions; + p_actions_rx = actions; hairpin_flow = flow_check_hairpin_split(dev, attr, actions); ret = flow_drv_validate(dev, attr, items, p_actions_rx, external, hairpin_flow, error); @@ -4413,6 +4605,8 @@ flow_list_create(struct rte_eth_dev *dev, uint32_t *list, buf->entries = 1; buf->entry[0].pattern = (void *)(uintptr_t)items; } + flow->shared_rss = flow_get_shared_rss_action(shared_actions, + shared_actions_n); /* * Record the start index when there is a nested call. All sub-flows * need to be translated before another calling. @@ -4484,6 +4678,7 @@ flow_list_create(struct rte_eth_dev *dev, uint32_t *list, ILIST_INSERT(priv->sh->ipool[MLX5_IPOOL_RTE_FLOW], list, idx, flow, next); flow_rxq_flags_set(dev, flow); + rte_free(translated_actions); /* Nested flow creation index recovery. */ priv->flow_idx = priv->flow_nested_idx; if (priv->flow_nested_idx) @@ -4498,6 +4693,7 @@ flow_list_create(struct rte_eth_dev *dev, uint32_t *list, rte_errno = ret; /* Restore rte_errno. */ error_before_flow: ret = rte_errno; + rte_free(translated_actions); if (hairpin_id) mlx5_flow_id_release(priv->sh->flow_id_pool, hairpin_id); @@ -6296,3 +6492,235 @@ mlx5_flow_get_aged_flows(struct rte_eth_dev *dev, void **contexts, dev->data->port_id); return -ENOTSUP; } + +/** + * Retrieve driver ops struct. + * + * @param[in] dev + * Pointer to the dev structure. + * @param[in] error_message + * Error message to set if driver ops struct not found. + * @param[out] error + * Perform verbose error reporting if not NULL. Initialized in case of + * error only. + * + * @return + * Pointer to driver ops on success, otherwise NULL and rte_errno is set. + */ +static const struct mlx5_flow_driver_ops * +flow_drv_dv_ops_get(struct rte_eth_dev *dev, + const char *error_message, + struct rte_flow_error *error) +{ + struct rte_flow_attr attr = { .transfer = 0 }; + + if (flow_get_drv_type(dev, &attr) != MLX5_FLOW_TYPE_DV) { + rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION, + NULL, error_message); + DRV_LOG(ERR, "port %u %s.", dev->data->port_id, error_message); + return NULL; + } + + return flow_get_drv_ops(MLX5_FLOW_TYPE_DV); +} + +/* Wrapper for driver action_validate op callback */ +static int +flow_drv_action_validate(struct rte_eth_dev *dev, + const struct rte_flow_action *action, + struct rte_flow_error *error) +{ + const struct mlx5_flow_driver_ops *fops = flow_drv_dv_ops_get(dev, + "action registration unsupported", error); + return (fops) ? fops->action_validate(dev, action, error) : -rte_errno; +} + +/* Wrapper for driver action_create op callback */ +static struct rte_flow_shared_action * +flow_drv_action_create(struct rte_eth_dev *dev, + const struct rte_flow_action *action, + struct rte_flow_error *error) +{ + const struct mlx5_flow_driver_ops *fops = flow_drv_dv_ops_get(dev, + "action registration unsupported", error); + return (fops) ? fops->action_create(dev, action, error) : NULL; +} + +/** + * Destroys the shared action by handle. + * + * @param dev + * Pointer to Ethernet device structure. + * @param[in] action + * Handle for the shared action to be destroyed. + * @param[out] error + * Perform verbose error reporting if not NULL. PMDs initialize this + * structure in case of error only. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + * + * @note: wrapper for driver action_create op callback. + */ +static int +mlx5_shared_action_destroy(struct rte_eth_dev *dev, + struct rte_flow_shared_action *action, + struct rte_flow_error *error) +{ + const struct mlx5_flow_driver_ops *fops = flow_drv_dv_ops_get(dev, + "action registration unsupported", error); + return (fops) ? fops->action_destroy(dev, action, error) : -rte_errno; +} + +/* Wrapper for driver action_destroy op callback */ +static int +flow_drv_action_update(struct rte_eth_dev *dev, + struct rte_flow_shared_action *action, + const void *action_conf, + struct rte_flow_error *error) +{ + const struct mlx5_flow_driver_ops *fops = flow_drv_dv_ops_get(dev, + "action registration unsupported", error); + return (fops) ? fops->action_update(dev, action, + action_conf, error) + : -rte_errno; +} + +/** + * Create shared action for reuse in multiple flow rules. + * + * @param dev + * Pointer to Ethernet device structure. + * @param[in] action + * Action configuration for shared action creation. + * @param[out] error + * Perform verbose error reporting if not NULL. PMDs initialize this + * structure in case of error only. + * @return + * A valid handle in case of success, NULL otherwise and rte_errno is set. + */ +static struct rte_flow_shared_action * +mlx5_shared_action_create(struct rte_eth_dev *dev, + const struct rte_flow_action *action, + struct rte_flow_error *error) +{ + if (flow_drv_action_validate(dev, action, error)) + return NULL; + return flow_drv_action_create(dev, action, error); +} + +/** + * Updates inplace the shared action configuration pointed by *action* handle + * with the configuration provided as *update* argument. + * The update of the shared action configuration effects all flow rules reusing + * the action via handle. + * + * @param dev + * Pointer to Ethernet device structure. + * @param[in] action + * Handle for the shared action to be updated. + * @param[in] update + * Action specification used to modify the action pointed by handle. + * *update* should be of same type with the action pointed by the *action* + * handle argument, otherwise considered as invalid. + * @param[out] error + * Perform verbose error reporting if not NULL. PMDs initialize this + * structure in case of error only. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +mlx5_shared_action_update(struct rte_eth_dev *dev, + struct rte_flow_shared_action *shared_action, + const struct rte_flow_action *action, + struct rte_flow_error *error) +{ + int ret; + + switch (shared_action->type) { + case MLX5_FLOW_ACTION_SHARED_RSS: + if (action->type != RTE_FLOW_ACTION_TYPE_RSS) { + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ACTION, + NULL, + "update action type invalid"); + } + ret = flow_drv_action_validate(dev, action, error); + if (ret) + return ret; + return flow_drv_action_update(dev, shared_action, action->conf, + error); + default: + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION, + NULL, + "action type not supported"); + } +} + +/** + * Query the shared action by handle. + * + * This function allows retrieving action-specific data such as counters. + * Data is gathered by special action which may be present/referenced in + * more than one flow rule definition. + * + * \see RTE_FLOW_ACTION_TYPE_COUNT + * + * @param dev + * Pointer to Ethernet device structure. + * @param[in] action + * Handle for the shared action to query. + * @param[in, out] data + * Pointer to storage for the associated query data type. + * @param[out] error + * Perform verbose error reporting if not NULL. PMDs initialize this + * structure in case of error only. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +mlx5_shared_action_query(struct rte_eth_dev *dev, + const struct rte_flow_shared_action *action, + void *data, + struct rte_flow_error *error) +{ + (void)dev; + switch (action->type) { + case MLX5_FLOW_ACTION_SHARED_RSS: + *((int32_t *)data) = rte_atomic32_read(&action->refcnt); + return 0; + default: + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION, + NULL, + "action type not supported"); + } +} + +/** + * Destroy all shared actions. + * + * @param dev + * Pointer to Ethernet device. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +int +mlx5_shared_action_flush(struct rte_eth_dev *dev) +{ + struct rte_flow_error error; + struct mlx5_priv *priv = dev->data->dev_private; + struct rte_flow_shared_action *action; + int ret = 0; + + while (!LIST_EMPTY(&priv->shared_actions)) { + action = LIST_FIRST(&priv->shared_actions); + ret = mlx5_shared_action_destroy(dev, action, &error); + } + return ret; +} diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h index 50ec741157..fec47c4372 100644 --- a/drivers/net/mlx5/mlx5_flow.h +++ b/drivers/net/mlx5/mlx5_flow.h @@ -202,6 +202,7 @@ enum mlx5_feature_name { #define MLX5_FLOW_ACTION_SET_IPV6_DSCP (1ull << 33) #define MLX5_FLOW_ACTION_AGE (1ull << 34) #define MLX5_FLOW_ACTION_DEFAULT_MISS (1ull << 35) +#define MLX5_FLOW_ACTION_SHARED_RSS (1ull << 36) #define MLX5_FLOW_FATE_ACTIONS \ (MLX5_FLOW_ACTION_DROP | MLX5_FLOW_ACTION_QUEUE | \ @@ -821,6 +822,7 @@ struct mlx5_fdir_flow { /* Flow structure. */ struct rte_flow { ILIST_ENTRY(uint32_t)next; /**< Index to the next flow structure. */ + struct mlx5_shared_action_rss *shared_rss; /** < Shred RSS action. */ uint32_t dev_handles; /**< Device flow handles that are part of the flow. */ uint32_t drv_type:2; /**< Driver type. */ @@ -834,6 +836,62 @@ struct rte_flow { uint16_t meter; /**< Holds flow meter id. */ } __rte_packed; +/* + * Define list of valid combinations of RX Hash fields + * (see enum ibv_rx_hash_fields). + */ +#define MLX5_RSS_HASH_IPV4 (IBV_RX_HASH_SRC_IPV4 | IBV_RX_HASH_DST_IPV4) +#define MLX5_RSS_HASH_IPV4_TCP \ + (MLX5_RSS_HASH_IPV4 | \ + IBV_RX_HASH_SRC_PORT_TCP | IBV_RX_HASH_SRC_PORT_TCP) +#define MLX5_RSS_HASH_IPV4_UDP \ + (MLX5_RSS_HASH_IPV4 | \ + IBV_RX_HASH_SRC_PORT_UDP | IBV_RX_HASH_SRC_PORT_UDP) +#define MLX5_RSS_HASH_IPV6 (IBV_RX_HASH_SRC_IPV6 | IBV_RX_HASH_DST_IPV6) +#define MLX5_RSS_HASH_IPV6_TCP \ + (MLX5_RSS_HASH_IPV6 | \ + IBV_RX_HASH_SRC_PORT_TCP | IBV_RX_HASH_SRC_PORT_TCP) +#define MLX5_RSS_HASH_IPV6_UDP \ + (MLX5_RSS_HASH_IPV6 | \ + IBV_RX_HASH_SRC_PORT_UDP | IBV_RX_HASH_SRC_PORT_UDP) +#define MLX5_RSS_HASH_NONE 0ULL + +/* array of valid combinations of RX Hash fields for RSS */ +static const uint64_t mlx5_rss_hash_fields[] = { + MLX5_RSS_HASH_IPV4, + MLX5_RSS_HASH_IPV4_TCP, + MLX5_RSS_HASH_IPV4_UDP, + MLX5_RSS_HASH_IPV6, + MLX5_RSS_HASH_IPV6_TCP, + MLX5_RSS_HASH_IPV6_UDP, + MLX5_RSS_HASH_NONE, +}; + +#define MLX5_RSS_HASH_FIELDS_LEN RTE_DIM(mlx5_rss_hash_fields) + +/* Shared RSS action structure */ +struct mlx5_shared_action_rss { + struct rte_flow_action_rss origin; /**< Original rte RSS action. */ + uint8_t key[MLX5_RSS_HASH_KEY_LEN]; /**< RSS hash key. */ + uint16_t *queue; /**< Queue indices to use. */ + uint32_t hrxq[MLX5_RSS_HASH_FIELDS_LEN]; + /**< Hash RX queue indexes mapped to mlx5_rss_hash_fields */ + uint32_t hrxq_tunnel[MLX5_RSS_HASH_FIELDS_LEN]; + /**< Hash RX queue indexes for tunneled RSS */ +}; + +struct rte_flow_shared_action { + LIST_ENTRY(rte_flow_shared_action) next; + /**< Pointer to the next element. */ + rte_atomic32_t refcnt; + uint64_t type; + /**< Shared action type (see MLX5_FLOW_ACTION_SHARED_*). */ + union { + struct mlx5_shared_action_rss rss; + /**< Shared RSS action. */ + }; +}; + typedef int (*mlx5_flow_validate_t)(struct rte_eth_dev *dev, const struct rte_flow_attr *attr, const struct rte_flow_item items[], @@ -888,6 +946,22 @@ typedef int (*mlx5_flow_get_aged_flows_t) void **context, uint32_t nb_contexts, struct rte_flow_error *error); +typedef int (*mlx5_flow_action_validate_t)(struct rte_eth_dev *dev, + const struct rte_flow_action *action, + struct rte_flow_error *error); +typedef struct rte_flow_shared_action *(*mlx5_flow_action_create_t) + (struct rte_eth_dev *dev, + const struct rte_flow_action *action, + struct rte_flow_error *error); +typedef int (*mlx5_flow_action_destroy_t) + (struct rte_eth_dev *dev, + struct rte_flow_shared_action *action, + struct rte_flow_error *error); +typedef int (*mlx5_flow_action_update_t) + (struct rte_eth_dev *dev, + struct rte_flow_shared_action *action, + const void *action_conf, + struct rte_flow_error *error); struct mlx5_flow_driver_ops { mlx5_flow_validate_t validate; mlx5_flow_prepare_t prepare; @@ -904,6 +978,10 @@ struct mlx5_flow_driver_ops { mlx5_flow_counter_free_t counter_free; mlx5_flow_counter_query_t counter_query; mlx5_flow_get_aged_flows_t get_aged_flows; + mlx5_flow_action_validate_t action_validate; + mlx5_flow_action_create_t action_create; + mlx5_flow_action_destroy_t action_destroy; + mlx5_flow_action_update_t action_update; }; /* mlx5_flow.c */ @@ -928,6 +1006,9 @@ int mlx5_flow_get_reg_id(struct rte_eth_dev *dev, const struct rte_flow_action *mlx5_flow_find_action (const struct rte_flow_action *actions, enum rte_flow_action_type action); +int mlx5_validate_action_rss(struct rte_eth_dev *dev, + const struct rte_flow_action *action, + struct rte_flow_error *error); int mlx5_flow_validate_action_count(struct rte_eth_dev *dev, const struct rte_flow_attr *attr, struct rte_flow_error *error); @@ -1040,4 +1121,6 @@ int mlx5_flow_destroy_policer_rules(struct rte_eth_dev *dev, const struct rte_flow_attr *attr); int mlx5_flow_meter_flush(struct rte_eth_dev *dev, struct rte_mtr_error *error); +struct rte_flow_shared_action *mlx5_flow_get_shared_rss(struct rte_flow *flow); +int mlx5_shared_action_flush(struct rte_eth_dev *dev); #endif /* RTE_PMD_MLX5_FLOW_H_ */ -- 2.26.2 ^ permalink raw reply [flat|nested] 106+ messages in thread
* [dpdk-dev] [PATCH v2 5/6] net/mlx5: driver support for shared action 2020-07-08 21:39 ` [dpdk-dev] [PATCH v2 0/6] add flow shared action API + PMD Andrey Vesnovaty ` (3 preceding siblings ...) 2020-07-08 21:39 ` [dpdk-dev] [PATCH v2 4/6] net/mlx5: shared action PMD Andrey Vesnovaty @ 2020-07-08 21:39 ` Andrey Vesnovaty 2020-07-08 21:39 ` [dpdk-dev] [PATCH v2 6/6] examples/flow_filtering: utilize shared RSS action Andrey Vesnovaty 2020-07-09 4:39 ` [dpdk-dev] [PATCH v2 0/6] add flow shared action API + PMD Jerin Jacob 6 siblings, 0 replies; 106+ messages in thread From: Andrey Vesnovaty @ 2020-07-08 21:39 UTC (permalink / raw) To: dev Cc: jer, jerinjacobk, thomas, ferruh.yigit, stephen, bruce.richardson, orika, viacheslavo, andrey.vesnovaty, Matan Azrad, Shahaf Shuler Implement shared action create/destroy/update/query. Implement RSS shared action and handle shared RSS on flow apply and release. Note: currently implemented for sharede RSS action only Signed-off-by: Andrey Vesnovaty <andreyv@mellanox.com> --- drivers/net/mlx5/mlx5_flow_dv.c | 671 ++++++++++++++++++++++++++++++-- 1 file changed, 647 insertions(+), 24 deletions(-) diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c index d1eb65b01b..93fbfbddab 100644 --- a/drivers/net/mlx5/mlx5_flow_dv.c +++ b/drivers/net/mlx5/mlx5_flow_dv.c @@ -8570,6 +8570,156 @@ __flow_dv_translate(struct rte_eth_dev *dev, return 0; } +/** + * Set hash RX queue by hash fields (see enum ibv_rx_hash_fields) + * and tunnel. + * + * @param[in, out] action + * Shred RSS action holding hash RX queue objects. + * @param[in] hash_fields + * Defines combination of packet fields to participate in RX hash. + * @param[in] tunnel + * Tunnel type + * @param[in] hrxq_idx + * Hash RX queue index to set. + * + * @return + * 0 on success, otherwise negative errno value. + */ +static int +__flow_dv_action_rss_hrxq_set(struct mlx5_shared_action_rss *action, + const uint64_t hash_fields, + const int tunnel, + uint32_t hrxq_idx) +{ + uint32_t *hrxqs = (tunnel) ? action->hrxq : action->hrxq_tunnel; + + switch (hash_fields & ~IBV_RX_HASH_INNER) { + case MLX5_RSS_HASH_IPV4: + hrxqs[0] = hrxq_idx; + return 0; + case MLX5_RSS_HASH_IPV4_TCP: + hrxqs[1] = hrxq_idx; + return 0; + case MLX5_RSS_HASH_IPV4_UDP: + hrxqs[2] = hrxq_idx; + return 0; + case MLX5_RSS_HASH_IPV6: + hrxqs[3] = hrxq_idx; + return 0; + case MLX5_RSS_HASH_IPV6_TCP: + hrxqs[4] = hrxq_idx; + return 0; + case MLX5_RSS_HASH_IPV6_UDP: + hrxqs[5] = hrxq_idx; + return 0; + case MLX5_RSS_HASH_NONE: + hrxqs[6] = hrxq_idx; + return 0; + default: + return -1; + } +} + +/** + * Look up for hash RX queue by hash fields (see enum ibv_rx_hash_fields) + * and tunnel. + * + * @param[in] action + * Shred RSS action holding hash RX queue objects. + * @param[in] hash_fields + * Defines combination of packet fields to participate in RX hash. + * @param[in] tunnel + * Tunnel type + * + * @return + * Valid hash RX queue index, otherwise 0. + */ +static uint32_t +__flow_dv_action_rss_hrxq_lookup(const struct mlx5_shared_action_rss *action, + const uint64_t hash_fields, + const int tunnel) +{ + const uint32_t *hrxqs = (tunnel) ? action->hrxq : action->hrxq_tunnel; + + switch (hash_fields & ~IBV_RX_HASH_INNER) { + case MLX5_RSS_HASH_IPV4: + return hrxqs[0]; + case MLX5_RSS_HASH_IPV4_TCP: + return hrxqs[1]; + case MLX5_RSS_HASH_IPV4_UDP: + return hrxqs[2]; + case MLX5_RSS_HASH_IPV6: + return hrxqs[3]; + case MLX5_RSS_HASH_IPV6_TCP: + return hrxqs[4]; + case MLX5_RSS_HASH_IPV6_UDP: + return hrxqs[5]; + case MLX5_RSS_HASH_NONE: + return hrxqs[6]; + default: + return 0; + } +} + +/** + * Retrieves hash RX queue suitable for the *flow*. + * If shared action configured for *flow* suitable hash RX queue will be + * retrieved from attached shared action. + * + * @param[in] flow + * Shred RSS action holding hash RX queue objects. + * @param[in] dev_flow + * Pointer to the sub flow. + * @param[out] hrxq + * Pointer to retrieved hash RX queue object. + * + * @return + * Valid hash RX queue index, otherwise 0 and rte_errno is set. + */ +static uint32_t +__flow_dv_rss_get_hrxq(struct rte_eth_dev *dev, struct rte_flow *flow, + struct mlx5_flow *dev_flow, + struct mlx5_hrxq **hrxq) +{ + struct mlx5_priv *priv = dev->data->dev_private; + uint32_t hrxq_idx; + struct mlx5_flow_rss_desc *rss_desc = NULL; + + if (flow->shared_rss) { + hrxq_idx = __flow_dv_action_rss_hrxq_lookup + (flow->shared_rss, dev_flow->hash_fields, + !!(dev_flow->handle->layers & + MLX5_FLOW_LAYER_TUNNEL)); + if (hrxq_idx) { + *hrxq = mlx5_ipool_get(priv->sh->ipool[MLX5_IPOOL_HRXQ], + hrxq_idx); + rte_atomic32_inc(&(*hrxq)->refcnt); + } + } else { + rss_desc = &((struct mlx5_flow_rss_desc *)priv->rss_desc) + [!!priv->flow_nested_idx]; + MLX5_ASSERT(rss_desc->queue_num); + hrxq_idx = mlx5_hrxq_get(dev, rss_desc->key, + MLX5_RSS_HASH_KEY_LEN, + dev_flow->hash_fields, + rss_desc->queue, rss_desc->queue_num); + if (!hrxq_idx) { + hrxq_idx = mlx5_hrxq_new(dev, + rss_desc->key, + MLX5_RSS_HASH_KEY_LEN, + dev_flow->hash_fields, + rss_desc->queue, + rss_desc->queue_num, + !!(dev_flow->handle->layers & + MLX5_FLOW_LAYER_TUNNEL)); + } + *hrxq = mlx5_ipool_get(priv->sh->ipool[MLX5_IPOOL_HRXQ], + hrxq_idx); + } + return hrxq_idx; +} + /** * Apply the flow to the NIC, lock free, * (mutex should be acquired by caller). @@ -8628,30 +8778,10 @@ __flow_dv_apply(struct rte_eth_dev *dev, struct rte_flow *flow, dv->actions[n++] = drop_hrxq->action; } } else if (dh->fate_action == MLX5_FLOW_FATE_QUEUE) { - struct mlx5_hrxq *hrxq; - uint32_t hrxq_idx; - struct mlx5_flow_rss_desc *rss_desc = - &((struct mlx5_flow_rss_desc *)priv->rss_desc) - [!!priv->flow_nested_idx]; - - MLX5_ASSERT(rss_desc->queue_num); - hrxq_idx = mlx5_hrxq_get(dev, rss_desc->key, - MLX5_RSS_HASH_KEY_LEN, - dev_flow->hash_fields, - rss_desc->queue, - rss_desc->queue_num); - if (!hrxq_idx) { - hrxq_idx = mlx5_hrxq_new - (dev, rss_desc->key, - MLX5_RSS_HASH_KEY_LEN, - dev_flow->hash_fields, - rss_desc->queue, - rss_desc->queue_num, - !!(dh->layers & - MLX5_FLOW_LAYER_TUNNEL)); - } - hrxq = mlx5_ipool_get(priv->sh->ipool[MLX5_IPOOL_HRXQ], - hrxq_idx); + struct mlx5_hrxq *hrxq = NULL; + uint32_t hrxq_idx = __flow_dv_rss_get_hrxq + (dev, flow, dev_flow, + &hrxq); if (!hrxq) { rte_flow_error_set (error, rte_errno, @@ -9067,12 +9197,16 @@ __flow_dv_remove(struct rte_eth_dev *dev, struct rte_flow *flow) static void __flow_dv_destroy(struct rte_eth_dev *dev, struct rte_flow *flow) { + struct rte_flow_shared_action *shared; struct mlx5_flow_handle *dev_handle; struct mlx5_priv *priv = dev->data->dev_private; if (!flow) return; __flow_dv_remove(dev, flow); + shared = mlx5_flow_get_shared_rss(flow); + if (shared) + rte_atomic32_dec(&shared->refcnt); if (flow->counter) { flow_dv_counter_release(dev, flow->counter); flow->counter = 0; @@ -9112,6 +9246,410 @@ __flow_dv_destroy(struct rte_eth_dev *dev, struct rte_flow *flow) } } +/** + * Release array of hash RX queue objects. + * Helper function. + * + * @param[in] dev + * Pointer to the Ethernet device structure. + * @param[in, out] hrxqs + * Array of hash RX queue objects. + * + * @return + * Total number of references to hash RX queue objects in *hrxqs* array + * after this operation. + */ +static int +__flow_dv_hrxqs_release(struct rte_eth_dev *dev, + uint32_t (*hrxqs)[MLX5_RSS_HASH_FIELDS_LEN]) +{ + size_t i; + int remaining = 0, ret = 0, ret_tunnel = 0; + + for (i = 0; i < RTE_DIM(*hrxqs); i++) { + ret = mlx5_hrxq_release(dev, (*hrxqs)[i]); + if (!ret) + (*hrxqs)[i] = 0; + remaining += ret + ret_tunnel; + } + return remaining; +} + +/** + * Release all hash RX queue objects representing shared RSS action. + * + * @param[in] dev + * Pointer to the Ethernet device structure. + * @param[in, out] action + * Shared RSS action to remove hash RX queue objects from. + * + * @return + * Total number of references to hash RX queue objects stored in *action* + * after this operation. + * Expected to be 0 if no external references held. + */ +static int +__flow_dv_action_rss_hrxqs_release(struct rte_eth_dev *dev, + struct mlx5_shared_action_rss *action) +{ + return __flow_dv_hrxqs_release(dev, &action->hrxq) + + __flow_dv_hrxqs_release(dev, &action->hrxq_tunnel); +} + +/** + * Setup shared RSS action. + * Prepare set of hash RX queue objects sufficient to handle all valid + * hash_fields combinations (see enum ibv_rx_hash_fields). + * + * @param[in] dev + * Pointer to the Ethernet device structure. + * @param[in, out] action + * Partially initialized shared RSS action. + * @param[out] error + * Perform verbose error reporting if not NULL. Initialized in case of + * error only. + * + * @return + * 0 on success, otherwise negative errno value. + */ +static int +__flow_dv_action_rss_setup(struct rte_eth_dev *dev, + struct mlx5_shared_action_rss *action, + struct rte_flow_error *error) +{ + size_t i; + int err; + + for (i = 0; i < MLX5_RSS_HASH_FIELDS_LEN; i++) { + uint32_t hrxq_idx; + uint64_t hash_fields = mlx5_rss_hash_fields[i]; + int tunnel; + + for (tunnel = 0; tunnel < 2; tunnel++) { + hrxq_idx = mlx5_hrxq_new(dev, action->origin.key, + MLX5_RSS_HASH_KEY_LEN, + hash_fields, + action->origin.queue, + action->origin.queue_num, + tunnel); + if (!hrxq_idx) { + rte_flow_error_set + (error, rte_errno, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL, + "cannot get hash queue"); + goto error_hrxq_new; + } + err = __flow_dv_action_rss_hrxq_set + (action, hash_fields, tunnel, hrxq_idx); + MLX5_ASSERT(!err); + } + } + return 0; +error_hrxq_new: + err = rte_errno; + __flow_dv_action_rss_hrxqs_release(dev, action); + rte_errno = err; + return -rte_errno; +} + +/** + * Create shared RSS action. + * + * @param[in] dev + * Pointer to the Ethernet device structure. + * @param[in] rss + * RSS action specification used to create shared action. + * @param[out] error + * Perform verbose error reporting if not NULL. Initialized in case of + * error only. + * + * @return + * A valid shared action handle in case of success, NULL otherwise and + * rte_errno is set. + */ +static struct rte_flow_shared_action * +__flow_dv_action_rss_create(struct rte_eth_dev *dev, + const struct rte_flow_action_rss *rss, + struct rte_flow_error *error) +{ + struct rte_flow_shared_action *shared_action = NULL; + void *queue = NULL; + uint32_t queue_size; + struct mlx5_shared_action_rss *shared_rss; + struct rte_flow_action_rss *origin; + const uint8_t *rss_key; + + queue_size = RTE_ALIGN_CEIL(rss->queue_num * sizeof(uint16_t), + sizeof(void *)); + queue = rte_calloc(__func__, 1, queue_size, 0); + shared_action = rte_calloc(__func__, 1, sizeof(*shared_action), 0); + if (!shared_action || !queue) { + rte_flow_error_set(error, ENOMEM, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL, + "cannot allocate resource memory"); + goto error_rss_init; + } + shared_rss = &shared_action->rss; + shared_rss->queue = queue; + origin = &shared_rss->origin; + origin->func = rss->func; + origin->level = rss->level; + /* RSS type 0 indicates default RSS type (ETH_RSS_IP). */ + origin->types = !rss->types ? ETH_RSS_IP : rss->types; + /* NULL RSS key indicates default RSS key. */ + rss_key = !rss->key ? rss_hash_default_key : rss->key; + rte_memcpy(shared_rss->key, rss_key, MLX5_RSS_HASH_KEY_LEN); + origin->key = &shared_rss->key[0]; + origin->key_len = MLX5_RSS_HASH_KEY_LEN; + rte_memcpy(shared_rss->queue, rss->queue, queue_size); + origin->queue = shared_rss->queue; + origin->queue_num = rss->queue_num; + if (__flow_dv_action_rss_setup(dev, shared_rss, error)) + goto error_rss_init; + return shared_action; +error_rss_init: + rte_free(shared_action); + rte_free(queue); + return NULL; +} + +/** + * Destroy the shared RSS action. + * Release related hash RX queue objects. + * + * @param[in] dev + * Pointer to the Ethernet device structure. + * @param[in] shared_rss + * The shared RSS action object to be removed. + * @param[out] error + * Perform verbose error reporting if not NULL. Initialized in case of + * error only. + * + * @return + * 0 on success, otherwise negative errno value. + */ +static int +__flow_dv_action_rss_release(struct rte_eth_dev *dev, + struct mlx5_shared_action_rss *shared_rss, + struct rte_flow_error *error) +{ + struct rte_flow_shared_action *shared_action = NULL; + int remaining = __flow_dv_action_rss_hrxqs_release(dev, shared_rss); + + if (remaining) { + return rte_flow_error_set(error, ETOOMANYREFS, + RTE_FLOW_ERROR_TYPE_ACTION, + NULL, + "shared rss hrxq has references"); + } + shared_action = container_of(shared_rss, + struct rte_flow_shared_action, rss); + if (!rte_atomic32_dec_and_test(&shared_action->refcnt)) { + return rte_flow_error_set(error, ETOOMANYREFS, + RTE_FLOW_ERROR_TYPE_ACTION, + NULL, + "shared rss has references"); + } + rte_free(shared_rss->queue); + return 0; +} + +/** + * Create shared action, lock free, + * (mutex should be acquired by caller). + * Dispatcher for action type specific call. + * + * @param[in] dev + * Pointer to the Ethernet device structure. + * @param[in] action + * Action specification used to create shared action. + * @param[out] error + * Perform verbose error reporting if not NULL. Initialized in case of + * error only. + * + * @return + * A valid shared action handle in case of success, NULL otherwise and + * rte_errno is set. + */ +static struct rte_flow_shared_action * +__flow_dv_action_create(struct rte_eth_dev *dev, + const struct rte_flow_action *action, + struct rte_flow_error *error) +{ + struct rte_flow_shared_action *shared_action = NULL; + struct mlx5_priv *priv = dev->data->dev_private; + + switch (action->type) { + case RTE_FLOW_ACTION_TYPE_RSS: + shared_action = __flow_dv_action_rss_create(dev, action->conf, + error); + break; + default: + rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ACTION, + NULL, "action type not supported"); + break; + } + if (shared_action) { + rte_atomic32_inc(&shared_action->refcnt); + LIST_INSERT_HEAD(&priv->shared_actions, shared_action, next); + } + return shared_action; +} + +/** + * Destroy the shared action. + * Release action related resources on the NIC and the memory. + * Lock free, (mutex should be acquired by caller). + * Dispatcher for action type specific call. + * + * @param[in] dev + * Pointer to the Ethernet device structure. + * @param[in] action + * The shared action object to be removed. + * @param[out] error + * Perform verbose error reporting if not NULL. Initialized in case of + * error only. + * + * @return + * 0 on success, otherwise negative errno value. + */ +static int +__flow_dv_action_destroy(struct rte_eth_dev *dev, + struct rte_flow_shared_action *action, + struct rte_flow_error *error) +{ + int ret; + + switch (action->type) { + case MLX5_FLOW_ACTION_SHARED_RSS: + ret = __flow_dv_action_rss_release(dev, &action->rss, error); + break; + default: + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION, + NULL, + "action type not supported"); + } + if (ret) + return ret; + LIST_REMOVE(action, next); + rte_free(action); + return 0; +} + +/** + * Updates in place shared RSS action configuration. + * + * @param[in] dev + * Pointer to the Ethernet device structure. + * @param[in] shared_rss + * The shared RSS action object to be updated. + * @param[in] action_conf + * RSS action specification used to modify *shared_rss*. + * @param[out] error + * Perform verbose error reporting if not NULL. Initialized in case of + * error only. + * + * @return + * 0 on success, otherwise negative errno value. + * @note: currently only support update of RSS queues. + */ +static int +__flow_dv_action_rss_update(struct rte_eth_dev *dev, + struct mlx5_shared_action_rss *shared_rss, + const struct rte_flow_action_rss *action_conf, + struct rte_flow_error *error) +{ + size_t i; + int ret; + void *queue = NULL; + uint32_t queue_size; + const uint8_t *rss_key; + uint32_t rss_key_len; + + queue_size = RTE_ALIGN_CEIL(action_conf->queue_num * sizeof(uint16_t), + sizeof(void *)); + queue = rte_calloc(__func__, 1, queue_size, 0); + if (!queue) { + return rte_flow_error_set(error, ENOMEM, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, + "cannot allocate resource memory"); + } + if (action_conf->key) { + rss_key = action_conf->key; + rss_key_len = action_conf->key_len; + } else { + rss_key = rss_hash_default_key; + rss_key_len = MLX5_RSS_HASH_KEY_LEN; + } + for (i = 0; i < MLX5_RSS_HASH_FIELDS_LEN; i++) { + uint32_t hrxq_idx; + uint64_t hash_fields = mlx5_rss_hash_fields[i]; + int tunnel; + + for (tunnel = 0; tunnel < 2; tunnel++) { + hrxq_idx = __flow_dv_action_rss_hrxq_lookup + (shared_rss, hash_fields, tunnel); + MLX5_ASSERT(hrxq_idx); + ret = mlx5_hrxq_modify + (dev, hrxq_idx, + rss_key, rss_key_len, + hash_fields, + action_conf->queue, action_conf->queue_num); + if (ret) { + rte_free(queue); + return rte_flow_error_set + (error, rte_errno, + RTE_FLOW_ERROR_TYPE_ACTION, NULL, + "cannot update hash queue"); + } + } + } + rte_free(shared_rss->queue); + shared_rss->queue = queue; + rte_memcpy(shared_rss->queue, action_conf->queue, queue_size); + shared_rss->origin.queue = shared_rss->queue; + shared_rss->origin.queue_num = action_conf->queue_num; + return 0; +} + +/** + * Updates in place shared action configuration, lock free, + * (mutex should be acquired by caller). + * + * @param[in] dev + * Pointer to the Ethernet device structure. + * @param[in] action + * The shared action object to be updated. + * @param[in] action_conf + * Action specification used to modify *action*. + * *action_conf* should be of type correlating with type of the *action*, + * otherwise considered as invalid. + * @param[out] error + * Perform verbose error reporting if not NULL. Initialized in case of + * error only. + * + * @return + * 0 on success, otherwise negative errno value. + */ +static int +__flow_dv_action_update(struct rte_eth_dev *dev, + struct rte_flow_shared_action *action, + const void *action_conf, + struct rte_flow_error *error) +{ + switch (action->type) { + case MLX5_FLOW_ACTION_SHARED_RSS: + return __flow_dv_action_rss_update(dev, &action->rss, + action_conf, error); + default: + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION, + NULL, + "action type not supported"); + } +} /** * Query a dv flow rule for its statistics via devx. * @@ -9792,6 +10330,87 @@ flow_dv_counter_free(struct rte_eth_dev *dev, uint32_t cnt) flow_dv_shared_unlock(dev); } +/** + * Validate shared action. + * Dispatcher for action type specific validation. + * + * @param[in] dev + * Pointer to the Ethernet device structure. + * @param[in] action + * The shared action object to validate. + * @param[out] error + * Perform verbose error reporting if not NULL. Initialized in case of + * error only. + * + * @return + * 0 on success, otherwise negative errno value. + */ +static int +flow_dv_action_validate(struct rte_eth_dev *dev, + const struct rte_flow_action *action, + struct rte_flow_error *error) +{ + switch (action->type) { + case RTE_FLOW_ACTION_TYPE_RSS: + return mlx5_validate_action_rss(dev, action, error); + default: + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION, + NULL, + "action type not supported"); + } +} + +/* + * Mutex-protected thunk to lock-free __flow_dv_action_create(). + */ +static struct rte_flow_shared_action * +flow_dv_action_create(struct rte_eth_dev *dev, + const struct rte_flow_action *action, + struct rte_flow_error *error) +{ + struct rte_flow_shared_action *shared_action = NULL; + + flow_dv_shared_lock(dev); + shared_action = __flow_dv_action_create(dev, action, error); + flow_dv_shared_unlock(dev); + return shared_action; +} + +/* + * Mutex-protected thunk to lock-free __flow_dv_action_destroy(). + */ +static int +flow_dv_action_destroy(struct rte_eth_dev *dev, + struct rte_flow_shared_action *action, + struct rte_flow_error *error) +{ + int ret; + + flow_dv_shared_lock(dev); + ret = __flow_dv_action_destroy(dev, action, error); + flow_dv_shared_unlock(dev); + return ret; +} + +/* + * Mutex-protected thunk to lock-free __flow_dv_action_update(). + */ +static int +flow_dv_action_update(struct rte_eth_dev *dev, + struct rte_flow_shared_action *action, + const void *action_conf, + struct rte_flow_error *error) +{ + int ret; + + flow_dv_shared_lock(dev); + ret = __flow_dv_action_update(dev, action, action_conf, + error); + flow_dv_shared_unlock(dev); + return ret; +} + const struct mlx5_flow_driver_ops mlx5_flow_dv_drv_ops = { .validate = flow_dv_validate, .prepare = flow_dv_prepare, @@ -9808,6 +10427,10 @@ const struct mlx5_flow_driver_ops mlx5_flow_dv_drv_ops = { .counter_free = flow_dv_counter_free, .counter_query = flow_dv_counter_query, .get_aged_flows = flow_get_aged_flows, + .action_validate = flow_dv_action_validate, + .action_create = flow_dv_action_create, + .action_destroy = flow_dv_action_destroy, + .action_update = flow_dv_action_update, }; #endif /* HAVE_IBV_FLOW_DV_SUPPORT */ -- 2.26.2 ^ permalink raw reply [flat|nested] 106+ messages in thread
* [dpdk-dev] [PATCH v2 6/6] examples/flow_filtering: utilize shared RSS action 2020-07-08 21:39 ` [dpdk-dev] [PATCH v2 0/6] add flow shared action API + PMD Andrey Vesnovaty ` (4 preceding siblings ...) 2020-07-08 21:39 ` [dpdk-dev] [PATCH v2 5/6] net/mlx5: driver support for shared action Andrey Vesnovaty @ 2020-07-08 21:39 ` Andrey Vesnovaty 2020-07-09 4:44 ` Jerin Jacob 2020-07-09 4:39 ` [dpdk-dev] [PATCH v2 0/6] add flow shared action API + PMD Jerin Jacob 6 siblings, 1 reply; 106+ messages in thread From: Andrey Vesnovaty @ 2020-07-08 21:39 UTC (permalink / raw) To: dev Cc: jer, jerinjacobk, thomas, ferruh.yigit, stephen, bruce.richardson, orika, viacheslavo, andrey.vesnovaty, Marko Kovacevic, Radu Nicolau, Akhil Goyal, Tomasz Kantecki, Sunil Kumar Kori, Pavan Nikhilesh, John McNamara This commit gives very first shared RSS action usage example and demonstrates shared action capability for in-place update. First application creates shared action during initialization phase. Later on the flow object created by application uses previously created shared RSS action with 1 queue configured instead of queue action in original application. On each RX queue burst shared RSS action reconfigured via rte_flow_shared_action_update() API to switch queue 0 to 1 & 1 to 0. User supposed to observe consistent queue switches on each packet burst. Signed-off-by: Andrey Vesnovaty <andreyv@mellanox.com> --- doc/guides/sample_app_ug/flow_filtering.rst | 62 +++++++++++++++++---- examples/flow_filtering/flow_blocks.c | 30 +++++----- examples/flow_filtering/main.c | 41 +++++++++++++- 3 files changed, 105 insertions(+), 28 deletions(-) diff --git a/doc/guides/sample_app_ug/flow_filtering.rst b/doc/guides/sample_app_ug/flow_filtering.rst index 5e5a6cd8a0..cfe9334717 100644 --- a/doc/guides/sample_app_ug/flow_filtering.rst +++ b/doc/guides/sample_app_ug/flow_filtering.rst @@ -106,7 +106,7 @@ following code: .. code-block:: c /* create flow for send packet with */ - flow = generate_ipv4_flow(port_id, selected_queue, + flow = generate_ipv4_flow(port_id, shared_action, SRC_IP, EMPTY_MASK, DEST_IP, FULL_MASK, &error); if (!flow) { @@ -242,7 +242,7 @@ The Ethernet port is configured with default settings using the rxq_conf = dev_info.default_rxconf; rxq_conf.offloads = port_conf.rxmode.offloads; -For this example we are configuring number of rx and tx queues that are connected +For this example we are configuring 2 rx and 2 tx queues that are connected to a single port. .. code-block:: c @@ -270,13 +270,22 @@ to a single port. } } +Before we create the flow we create shared action in order to send it as +actions argument when creating a flow. The action is single queue RSS action +similar to action queue with the only difference that shared RSS action +provides update capability after action creation. + +.. code-block:: c + + shared_action = rte_flow_shared_action_create(port_id, &action, &error); + In the next step we create and apply the flow rule. which is to send packets with destination ip equals to 192.168.1.1 to queue number 1. The detail explanation of the ``generate_ipv4_flow()`` appears later in this document: .. code-block:: c - flow = generate_ipv4_flow(port_id, selected_queue, + flow = generate_ipv4_flow(port_id, shared_action, SRC_IP, EMPTY_MASK, DEST_IP, FULL_MASK, &error); @@ -339,6 +348,21 @@ looks like the following: printf("\n"); rte_pktmbuf_free(m); } + if (rss_queue[0] == 0) { + printf(">>> switching queue 0 -> 1\n"); + rss_queue[0] = 1; + } else { + printf(">>> switching queue 1 -> 0\n"); + rss_queue[0] = 0; + } + ret = rte_flow_shared_action_update + (port_id, shared_action, &action, + &error); + if (ret) + rte_exit(EXIT_FAILURE, + ":: error: RSS action update " + "failed: %s\n", + rte_strerror(-ret)); } } } @@ -348,6 +372,8 @@ looks like the following: rte_eth_dev_close(port_id); } +On each loop eteration Rx queue switched using +``rte_flow_shared_action_update()`` API. The main work of the application is reading the packets from all queues and printing for each packet the destination queue: @@ -365,6 +391,21 @@ queues and printing for each packet the destination queue: printf(" - queue=0x%x", (unsigned int)i); printf("\n"); rte_pktmbuf_free(m); + if (rss_queue[0] == 0) { + printf(">>> switching queue 0 -> 1\n"); + rss_queue[0] = 1; + } else { + printf(">>> switching queue 1 -> 0\n"); + rss_queue[0] = 0; + } + ret = rte_flow_shared_action_update + (port_id, shared_action, &action, + &error); + if (ret) + rte_exit(EXIT_FAILURE, + ":: error: RSS action update " + "failed: %s\n", + rte_strerror(-ret)); } } } @@ -378,13 +419,15 @@ The forwarding loop can be interrupted and the application closed using The generate_ipv4_flow function ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + The generate_ipv4_flow function is responsible for creating the flow rule. This function is located in the ``flow_blocks.c`` file. .. code-block:: c static struct rte_flow * - generate_ipv4_flow(uint8_t port_id, uint16_t rx_q, + generate_ipv4_flow(uint8_t port_id, + cstructrte_flow_shared_action *shared_action, uint32_t src_ip, uint32_t src_mask, uint32_t dest_ip, uint32_t dest_mask, struct rte_flow_error *error) @@ -393,7 +436,6 @@ This function is located in the ``flow_blocks.c`` file. struct rte_flow_item pattern[MAX_PATTERN_NUM]; struct rte_flow_action action[MAX_ACTION_NUM]; struct rte_flow *flow = NULL; - struct rte_flow_action_queue queue = { .index = rx_q }; struct rte_flow_item_ipv4 ip_spec; struct rte_flow_item_ipv4 ip_mask; @@ -411,8 +453,8 @@ This function is located in the ``flow_blocks.c`` file. * create the action sequence. * one action only, move packet to queue */ - action[0].type = RTE_FLOW_ACTION_TYPE_QUEUE; - action[0].conf = &queue; + action[0].type = RTE_FLOW_ACTION_TYPE_SHARED; + action[0].conf = shared_action; action[1].type = RTE_FLOW_ACTION_TYPE_END; /* @@ -468,12 +510,12 @@ The following part create the flow attributes, in our case ingress. attr.ingress = 1; The third part defines the action to be taken when a packet matches -the rule. In this case send the packet to queue. +the rule. In this case send the packet to single RSS queue. .. code-block:: c - action[0].type = RTE_FLOW_ACTION_TYPE_QUEUE; - action[0].conf = &queue; + action[0].type = RTE_FLOW_ACTION_TYPE_SHARED; + action[0].conf = shared_action; action[1].type = RTE_FLOW_ACTION_TYPE_END; The fourth part is responsible for creating the pattern and is built from diff --git a/examples/flow_filtering/flow_blocks.c b/examples/flow_filtering/flow_blocks.c index 575d792810..99bfed3172 100644 --- a/examples/flow_filtering/flow_blocks.c +++ b/examples/flow_filtering/flow_blocks.c @@ -6,11 +6,11 @@ #define MAX_ACTION_NUM 2 struct rte_flow * -generate_ipv4_flow(uint16_t port_id, uint16_t rx_q, - uint32_t src_ip, uint32_t src_mask, - uint32_t dest_ip, uint32_t dest_mask, - struct rte_flow_error *error); - +generate_ipv4_flow(uint16_t port_id, + struct rte_flow_shared_action *shared_action, + uint32_t src_ip, uint32_t src_mask, + uint32_t dest_ip, uint32_t dest_mask, + struct rte_flow_error *error); /** * create a flow rule that sends packets with matching src and dest ip @@ -18,8 +18,8 @@ generate_ipv4_flow(uint16_t port_id, uint16_t rx_q, * * @param port_id * The selected port. - * @param rx_q - * The selected target queue. + * @param shared_action + * The shared RSS action with single queue * @param src_ip * The src ip value to match the input packet. * @param src_mask @@ -35,16 +35,16 @@ generate_ipv4_flow(uint16_t port_id, uint16_t rx_q, * A flow if the rule could be created else return NULL. */ struct rte_flow * -generate_ipv4_flow(uint16_t port_id, uint16_t rx_q, - uint32_t src_ip, uint32_t src_mask, - uint32_t dest_ip, uint32_t dest_mask, - struct rte_flow_error *error) +generate_ipv4_flow(uint16_t port_id, + struct rte_flow_shared_action *shared_action, + uint32_t src_ip, uint32_t src_mask, + uint32_t dest_ip, uint32_t dest_mask, + struct rte_flow_error *error) { struct rte_flow_attr attr; struct rte_flow_item pattern[MAX_PATTERN_NUM]; struct rte_flow_action action[MAX_ACTION_NUM]; struct rte_flow *flow = NULL; - struct rte_flow_action_queue queue = { .index = rx_q }; struct rte_flow_item_ipv4 ip_spec; struct rte_flow_item_ipv4 ip_mask; int res; @@ -61,10 +61,10 @@ generate_ipv4_flow(uint16_t port_id, uint16_t rx_q, /* * create the action sequence. - * one action only, move packet to queue + * one action only, move packet to shared RSS queue */ - action[0].type = RTE_FLOW_ACTION_TYPE_QUEUE; - action[0].conf = &queue; + action[0].type = RTE_FLOW_ACTION_TYPE_SHARED; + action[0].conf = shared_action; action[1].type = RTE_FLOW_ACTION_TYPE_END; /* diff --git a/examples/flow_filtering/main.c b/examples/flow_filtering/main.c index cc9e7e7808..d6b18d95fc 100644 --- a/examples/flow_filtering/main.c +++ b/examples/flow_filtering/main.c @@ -32,8 +32,7 @@ static volatile bool force_quit; static uint16_t port_id; -static uint16_t nr_queues = 5; -static uint8_t selected_queue = 1; +static uint16_t nr_queues = 2; struct rte_mempool *mbuf_pool; struct rte_flow *flow; @@ -42,6 +41,24 @@ struct rte_flow *flow; #define FULL_MASK 0xffffffff /* full mask */ #define EMPTY_MASK 0x0 /* empty mask */ +struct rte_flow_shared_action *shared_action; +uint16_t rss_queue[1] = {0}; + +struct rte_flow_action_rss action_rss = { + .func = RTE_ETH_HASH_FUNCTION_DEFAULT, + .level = 0, + .types = 0, + .key_len = 0, + .key = NULL, + .queue = rss_queue, + .queue_num = 1, +}; + +struct rte_flow_action action = { + .type = RTE_FLOW_ACTION_TYPE_RSS, + .conf = &action_rss, +}; + #include "flow_blocks.c" static inline void @@ -61,6 +78,7 @@ main_loop(void) uint16_t nb_rx; uint16_t i; uint16_t j; + int ret; while (!force_quit) { for (i = 0; i < nr_queues; i++) { @@ -82,6 +100,21 @@ main_loop(void) rte_pktmbuf_free(m); } + if (rss_queue[0] == 0) { + printf(">>> switching queue 0 -> 1\n"); + rss_queue[0] = 1; + } else { + printf(">>> switching queue 1 -> 0\n"); + rss_queue[0] = 0; + } + ret = rte_flow_shared_action_update + (port_id, shared_action, &action, + &error); + if (ret) + rte_exit(EXIT_FAILURE, + ":: error: RSS action update " + "failed: %s\n", + rte_strerror(-ret)); } } } @@ -243,8 +276,10 @@ main(int argc, char **argv) init_port(); + shared_action = rte_flow_shared_action_create(port_id, &action, &error); + /* create flow for send packet with */ - flow = generate_ipv4_flow(port_id, selected_queue, + flow = generate_ipv4_flow(port_id, shared_action, SRC_IP, EMPTY_MASK, DEST_IP, FULL_MASK, &error); if (!flow) { -- 2.26.2 ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [dpdk-dev] [PATCH v2 6/6] examples/flow_filtering: utilize shared RSS action 2020-07-08 21:39 ` [dpdk-dev] [PATCH v2 6/6] examples/flow_filtering: utilize shared RSS action Andrey Vesnovaty @ 2020-07-09 4:44 ` Jerin Jacob 2020-07-09 6:08 ` Ori Kam 0 siblings, 1 reply; 106+ messages in thread From: Jerin Jacob @ 2020-07-09 4:44 UTC (permalink / raw) To: Andrey Vesnovaty Cc: dpdk-dev, jer, Thomas Monjalon, Ferruh Yigit, Stephen Hemminger, Richardson, Bruce, Ori Kam, Slava Ovsiienko, Andrey Vesnovaty, Marko Kovacevic, Radu Nicolau, Akhil Goyal, Tomasz Kantecki, Sunil Kumar Kori, Pavan Nikhilesh, John McNamara On Thu, Jul 9, 2020 at 3:09 AM Andrey Vesnovaty <andreyv@mellanox.com> wrote: > > This commit gives very first shared RSS action usage example and > demonstrates shared action capability for in-place update. > > First application creates shared action during initialization phase. What if PMD does not support a shared context, Then this application fails to run? if so, have a mode or probe the capability(if capability present then switches to new mode else fall back to old mode) before changing the application behavior. > Later on the flow object created by application uses previously created > shared RSS action with 1 queue configured instead of queue action in > original application. > > On each RX queue burst shared RSS action reconfigured via > rte_flow_shared_action_update() API to switch queue 0 to 1 & 1 to 0. > User supposed to observe consistent queue switches on each packet burst. > > Signed-off-by: Andrey Vesnovaty <andreyv@mellanox.com> > --- > doc/guides/sample_app_ug/flow_filtering.rst | 62 +++++++++++++++++---- > examples/flow_filtering/flow_blocks.c | 30 +++++----- > examples/flow_filtering/main.c | 41 +++++++++++++- > 3 files changed, 105 insertions(+), 28 deletions(-) > > diff --git a/doc/guides/sample_app_ug/flow_filtering.rst b/doc/guides/sample_app_ug/flow_filtering.rst > index 5e5a6cd8a0..cfe9334717 100644 > --- a/doc/guides/sample_app_ug/flow_filtering.rst > +++ b/doc/guides/sample_app_ug/flow_filtering.rst > @@ -106,7 +106,7 @@ following code: > .. code-block:: c > > /* create flow for send packet with */ > - flow = generate_ipv4_flow(port_id, selected_queue, > + flow = generate_ipv4_flow(port_id, shared_action, > SRC_IP, EMPTY_MASK, > DEST_IP, FULL_MASK, &error); > if (!flow) { > @@ -242,7 +242,7 @@ The Ethernet port is configured with default settings using the > rxq_conf = dev_info.default_rxconf; > rxq_conf.offloads = port_conf.rxmode.offloads; > > -For this example we are configuring number of rx and tx queues that are connected > +For this example we are configuring 2 rx and 2 tx queues that are connected > to a single port. > > .. code-block:: c > @@ -270,13 +270,22 @@ to a single port. > } > } > > +Before we create the flow we create shared action in order to send it as > +actions argument when creating a flow. The action is single queue RSS action > +similar to action queue with the only difference that shared RSS action > +provides update capability after action creation. > + > +.. code-block:: c > + > + shared_action = rte_flow_shared_action_create(port_id, &action, &error); > + > In the next step we create and apply the flow rule. which is to send packets > with destination ip equals to 192.168.1.1 to queue number 1. The detail > explanation of the ``generate_ipv4_flow()`` appears later in this document: > > .. code-block:: c > > - flow = generate_ipv4_flow(port_id, selected_queue, > + flow = generate_ipv4_flow(port_id, shared_action, > SRC_IP, EMPTY_MASK, > DEST_IP, FULL_MASK, &error); > > @@ -339,6 +348,21 @@ looks like the following: > printf("\n"); > rte_pktmbuf_free(m); > } > + if (rss_queue[0] == 0) { > + printf(">>> switching queue 0 -> 1\n"); > + rss_queue[0] = 1; > + } else { > + printf(">>> switching queue 1 -> 0\n"); > + rss_queue[0] = 0; > + } > + ret = rte_flow_shared_action_update > + (port_id, shared_action, &action, > + &error); > + if (ret) > + rte_exit(EXIT_FAILURE, > + ":: error: RSS action update " > + "failed: %s\n", > + rte_strerror(-ret)); > } > } > } > @@ -348,6 +372,8 @@ looks like the following: > rte_eth_dev_close(port_id); > } > > +On each loop eteration Rx queue switched using > +``rte_flow_shared_action_update()`` API. > The main work of the application is reading the packets from all > queues and printing for each packet the destination queue: > > @@ -365,6 +391,21 @@ queues and printing for each packet the destination queue: > printf(" - queue=0x%x", (unsigned int)i); > printf("\n"); > rte_pktmbuf_free(m); > + if (rss_queue[0] == 0) { > + printf(">>> switching queue 0 -> 1\n"); > + rss_queue[0] = 1; > + } else { > + printf(">>> switching queue 1 -> 0\n"); > + rss_queue[0] = 0; > + } > + ret = rte_flow_shared_action_update > + (port_id, shared_action, &action, > + &error); > + if (ret) > + rte_exit(EXIT_FAILURE, > + ":: error: RSS action update " > + "failed: %s\n", > + rte_strerror(-ret)); > } > } > } > @@ -378,13 +419,15 @@ The forwarding loop can be interrupted and the application closed using > The generate_ipv4_flow function > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > + > The generate_ipv4_flow function is responsible for creating the flow rule. > This function is located in the ``flow_blocks.c`` file. > > .. code-block:: c > > static struct rte_flow * > - generate_ipv4_flow(uint8_t port_id, uint16_t rx_q, > + generate_ipv4_flow(uint8_t port_id, > + cstructrte_flow_shared_action *shared_action, > uint32_t src_ip, uint32_t src_mask, > uint32_t dest_ip, uint32_t dest_mask, > struct rte_flow_error *error) > @@ -393,7 +436,6 @@ This function is located in the ``flow_blocks.c`` file. > struct rte_flow_item pattern[MAX_PATTERN_NUM]; > struct rte_flow_action action[MAX_ACTION_NUM]; > struct rte_flow *flow = NULL; > - struct rte_flow_action_queue queue = { .index = rx_q }; > struct rte_flow_item_ipv4 ip_spec; > struct rte_flow_item_ipv4 ip_mask; > > @@ -411,8 +453,8 @@ This function is located in the ``flow_blocks.c`` file. > * create the action sequence. > * one action only, move packet to queue > */ > - action[0].type = RTE_FLOW_ACTION_TYPE_QUEUE; > - action[0].conf = &queue; > + action[0].type = RTE_FLOW_ACTION_TYPE_SHARED; > + action[0].conf = shared_action; > action[1].type = RTE_FLOW_ACTION_TYPE_END; > > /* > @@ -468,12 +510,12 @@ The following part create the flow attributes, in our case ingress. > attr.ingress = 1; > > The third part defines the action to be taken when a packet matches > -the rule. In this case send the packet to queue. > +the rule. In this case send the packet to single RSS queue. > > .. code-block:: c > > - action[0].type = RTE_FLOW_ACTION_TYPE_QUEUE; > - action[0].conf = &queue; > + action[0].type = RTE_FLOW_ACTION_TYPE_SHARED; > + action[0].conf = shared_action; > action[1].type = RTE_FLOW_ACTION_TYPE_END; > > The fourth part is responsible for creating the pattern and is built from > diff --git a/examples/flow_filtering/flow_blocks.c b/examples/flow_filtering/flow_blocks.c > index 575d792810..99bfed3172 100644 > --- a/examples/flow_filtering/flow_blocks.c > +++ b/examples/flow_filtering/flow_blocks.c > @@ -6,11 +6,11 @@ > #define MAX_ACTION_NUM 2 > > struct rte_flow * > -generate_ipv4_flow(uint16_t port_id, uint16_t rx_q, > - uint32_t src_ip, uint32_t src_mask, > - uint32_t dest_ip, uint32_t dest_mask, > - struct rte_flow_error *error); > - > +generate_ipv4_flow(uint16_t port_id, > + struct rte_flow_shared_action *shared_action, > + uint32_t src_ip, uint32_t src_mask, > + uint32_t dest_ip, uint32_t dest_mask, > + struct rte_flow_error *error); > > /** > * create a flow rule that sends packets with matching src and dest ip > @@ -18,8 +18,8 @@ generate_ipv4_flow(uint16_t port_id, uint16_t rx_q, > * > * @param port_id > * The selected port. > - * @param rx_q > - * The selected target queue. > + * @param shared_action > + * The shared RSS action with single queue > * @param src_ip > * The src ip value to match the input packet. > * @param src_mask > @@ -35,16 +35,16 @@ generate_ipv4_flow(uint16_t port_id, uint16_t rx_q, > * A flow if the rule could be created else return NULL. > */ > struct rte_flow * > -generate_ipv4_flow(uint16_t port_id, uint16_t rx_q, > - uint32_t src_ip, uint32_t src_mask, > - uint32_t dest_ip, uint32_t dest_mask, > - struct rte_flow_error *error) > +generate_ipv4_flow(uint16_t port_id, > + struct rte_flow_shared_action *shared_action, > + uint32_t src_ip, uint32_t src_mask, > + uint32_t dest_ip, uint32_t dest_mask, > + struct rte_flow_error *error) > { > struct rte_flow_attr attr; > struct rte_flow_item pattern[MAX_PATTERN_NUM]; > struct rte_flow_action action[MAX_ACTION_NUM]; > struct rte_flow *flow = NULL; > - struct rte_flow_action_queue queue = { .index = rx_q }; > struct rte_flow_item_ipv4 ip_spec; > struct rte_flow_item_ipv4 ip_mask; > int res; > @@ -61,10 +61,10 @@ generate_ipv4_flow(uint16_t port_id, uint16_t rx_q, > > /* > * create the action sequence. > - * one action only, move packet to queue > + * one action only, move packet to shared RSS queue > */ > - action[0].type = RTE_FLOW_ACTION_TYPE_QUEUE; > - action[0].conf = &queue; > + action[0].type = RTE_FLOW_ACTION_TYPE_SHARED; > + action[0].conf = shared_action; > action[1].type = RTE_FLOW_ACTION_TYPE_END; > > /* > diff --git a/examples/flow_filtering/main.c b/examples/flow_filtering/main.c > index cc9e7e7808..d6b18d95fc 100644 > --- a/examples/flow_filtering/main.c > +++ b/examples/flow_filtering/main.c > @@ -32,8 +32,7 @@ > static volatile bool force_quit; > > static uint16_t port_id; > -static uint16_t nr_queues = 5; > -static uint8_t selected_queue = 1; > +static uint16_t nr_queues = 2; > struct rte_mempool *mbuf_pool; > struct rte_flow *flow; > > @@ -42,6 +41,24 @@ struct rte_flow *flow; > #define FULL_MASK 0xffffffff /* full mask */ > #define EMPTY_MASK 0x0 /* empty mask */ > > +struct rte_flow_shared_action *shared_action; > +uint16_t rss_queue[1] = {0}; > + > +struct rte_flow_action_rss action_rss = { > + .func = RTE_ETH_HASH_FUNCTION_DEFAULT, > + .level = 0, > + .types = 0, > + .key_len = 0, > + .key = NULL, > + .queue = rss_queue, > + .queue_num = 1, > +}; > + > +struct rte_flow_action action = { > + .type = RTE_FLOW_ACTION_TYPE_RSS, > + .conf = &action_rss, > +}; > + > #include "flow_blocks.c" > > static inline void > @@ -61,6 +78,7 @@ main_loop(void) > uint16_t nb_rx; > uint16_t i; > uint16_t j; > + int ret; > > while (!force_quit) { > for (i = 0; i < nr_queues; i++) { > @@ -82,6 +100,21 @@ main_loop(void) > > rte_pktmbuf_free(m); > } > + if (rss_queue[0] == 0) { > + printf(">>> switching queue 0 -> 1\n"); > + rss_queue[0] = 1; > + } else { > + printf(">>> switching queue 1 -> 0\n"); > + rss_queue[0] = 0; > + } > + ret = rte_flow_shared_action_update > + (port_id, shared_action, &action, > + &error); > + if (ret) > + rte_exit(EXIT_FAILURE, > + ":: error: RSS action update " > + "failed: %s\n", > + rte_strerror(-ret)); > } > } > } > @@ -243,8 +276,10 @@ main(int argc, char **argv) > > init_port(); > > + shared_action = rte_flow_shared_action_create(port_id, &action, &error); > + > /* create flow for send packet with */ > - flow = generate_ipv4_flow(port_id, selected_queue, > + flow = generate_ipv4_flow(port_id, shared_action, > SRC_IP, EMPTY_MASK, > DEST_IP, FULL_MASK, &error); > if (!flow) { > -- > 2.26.2 > ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [dpdk-dev] [PATCH v2 6/6] examples/flow_filtering: utilize shared RSS action 2020-07-09 4:44 ` Jerin Jacob @ 2020-07-09 6:08 ` Ori Kam 2020-07-09 12:25 ` Andrey Vesnovaty 0 siblings, 1 reply; 106+ messages in thread From: Ori Kam @ 2020-07-09 6:08 UTC (permalink / raw) To: Jerin Jacob, Andrey Vesnovaty Cc: dpdk-dev, jer, Thomas Monjalon, Ferruh Yigit, Stephen Hemminger, Richardson, Bruce, Slava Ovsiienko, Andrey Vesnovaty, Marko Kovacevic, Radu Nicolau, Akhil Goyal, Tomasz Kantecki, Sunil Kumar Kori, Pavan Nikhilesh, John McNamara Hi Andrey, > -----Original Message----- > From: Jerin Jacob <jerinjacobk@gmail.com> > Sent: Thursday, July 9, 2020 7:44 AM > To: Andrey Vesnovaty <andreyv@mellanox.com> > Cc: dpdk-dev <dev@dpdk.org>; jer@marvell.com; Thomas Monjalon > <thomas@monjalon.net>; Ferruh Yigit <ferruh.yigit@intel.com>; Stephen > Hemminger <stephen@networkplumber.org>; Richardson, Bruce > <bruce.richardson@intel.com>; Ori Kam <orika@mellanox.com>; Slava > Ovsiienko <viacheslavo@mellanox.com>; Andrey Vesnovaty > <andrey.vesnovaty@gmail.com>; Marko Kovacevic > <marko.kovacevic@intel.com>; Radu Nicolau <radu.nicolau@intel.com>; Akhil > Goyal <akhil.goyal@nxp.com>; Tomasz Kantecki <tomasz.kantecki@intel.com>; > Sunil Kumar Kori <skori@marvell.com>; Pavan Nikhilesh > <pbhagavatula@marvell.com>; John McNamara <john.mcnamara@intel.com> > Subject: Re: [PATCH v2 6/6] examples/flow_filtering: utilize shared RSS action > > On Thu, Jul 9, 2020 at 3:09 AM Andrey Vesnovaty <andreyv@mellanox.com> > wrote: > > > > This commit gives very first shared RSS action usage example and > > demonstrates shared action capability for in-place update. > > > > First application creates shared action during initialization phase. > > What if PMD does not support a shared context, Then this application > fails to run? > if so, have a mode or probe the capability(if capability present then > switches to new mode else fall back to old mode) before changing the > application behavior. > +1 the new action should be added as a new case and should run only if it is the shared action is supported. > > > Later on the flow object created by application uses previously created > > shared RSS action with 1 queue configured instead of queue action in > > original application. > > > > On each RX queue burst shared RSS action reconfigured via > > rte_flow_shared_action_update() API to switch queue 0 to 1 & 1 to 0. > > User supposed to observe consistent queue switches on each packet burst. > > > > Signed-off-by: Andrey Vesnovaty <andreyv@mellanox.com> > > --- > > doc/guides/sample_app_ug/flow_filtering.rst | 62 +++++++++++++++++---- > > examples/flow_filtering/flow_blocks.c | 30 +++++----- > > examples/flow_filtering/main.c | 41 +++++++++++++- > > 3 files changed, 105 insertions(+), 28 deletions(-) > > > > diff --git a/doc/guides/sample_app_ug/flow_filtering.rst > b/doc/guides/sample_app_ug/flow_filtering.rst > > index 5e5a6cd8a0..cfe9334717 100644 > > --- a/doc/guides/sample_app_ug/flow_filtering.rst > > +++ b/doc/guides/sample_app_ug/flow_filtering.rst > > @@ -106,7 +106,7 @@ following code: > > .. code-block:: c > > > > /* create flow for send packet with */ > > - flow = generate_ipv4_flow(port_id, selected_queue, > > + flow = generate_ipv4_flow(port_id, shared_action, > > SRC_IP, EMPTY_MASK, > > DEST_IP, FULL_MASK, &error); > > if (!flow) { > > @@ -242,7 +242,7 @@ The Ethernet port is configured with default settings > using the > > rxq_conf = dev_info.default_rxconf; > > rxq_conf.offloads = port_conf.rxmode.offloads; > > > > -For this example we are configuring number of rx and tx queues that are > connected > > +For this example we are configuring 2 rx and 2 tx queues that are connected > > to a single port. > > > > .. code-block:: c > > @@ -270,13 +270,22 @@ to a single port. > > } > > } > > > > +Before we create the flow we create shared action in order to send it as > > +actions argument when creating a flow. The action is single queue RSS > action > > +similar to action queue with the only difference that shared RSS action > > +provides update capability after action creation. > > + > > +.. code-block:: c > > + > > + shared_action = rte_flow_shared_action_create(port_id, &action, &error); > > + > > In the next step we create and apply the flow rule. which is to send packets > > with destination ip equals to 192.168.1.1 to queue number 1. The detail > > explanation of the ``generate_ipv4_flow()`` appears later in this document: > > > > .. code-block:: c > > > > - flow = generate_ipv4_flow(port_id, selected_queue, > > + flow = generate_ipv4_flow(port_id, shared_action, > > SRC_IP, EMPTY_MASK, > > DEST_IP, FULL_MASK, &error); > > > > @@ -339,6 +348,21 @@ looks like the following: > > printf("\n"); > > rte_pktmbuf_free(m); > > } > > + if (rss_queue[0] == 0) { > > + printf(">>> switching queue 0 -> 1\n"); > > + rss_queue[0] = 1; > > + } else { > > + printf(">>> switching queue 1 -> 0\n"); > > + rss_queue[0] = 0; > > + } > > + ret = rte_flow_shared_action_update > > + (port_id, shared_action, &action, > > + &error); > > + if (ret) > > + rte_exit(EXIT_FAILURE, > > + ":: error: RSS action update " > > + "failed: %s\n", > > + rte_strerror(-ret)); > > } > > } > > } > > @@ -348,6 +372,8 @@ looks like the following: > > rte_eth_dev_close(port_id); > > } > > > > +On each loop eteration Rx queue switched using > > +``rte_flow_shared_action_update()`` API. > > The main work of the application is reading the packets from all > > queues and printing for each packet the destination queue: > > > > @@ -365,6 +391,21 @@ queues and printing for each packet the destination > queue: > > printf(" - queue=0x%x", (unsigned int)i); > > printf("\n"); > > rte_pktmbuf_free(m); > > + if (rss_queue[0] == 0) { > > + printf(">>> switching queue 0 -> 1\n"); > > + rss_queue[0] = 1; > > + } else { > > + printf(">>> switching queue 1 -> 0\n"); > > + rss_queue[0] = 0; > > + } > > + ret = rte_flow_shared_action_update > > + (port_id, shared_action, &action, > > + &error); > > + if (ret) > > + rte_exit(EXIT_FAILURE, > > + ":: error: RSS action update " > > + "failed: %s\n", > > + rte_strerror(-ret)); > > } > > } > > } > > @@ -378,13 +419,15 @@ The forwarding loop can be interrupted and the > application closed using > > The generate_ipv4_flow function > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > > > + > > The generate_ipv4_flow function is responsible for creating the flow rule. > > This function is located in the ``flow_blocks.c`` file. > > > > .. code-block:: c > > > > static struct rte_flow * > > - generate_ipv4_flow(uint8_t port_id, uint16_t rx_q, > > + generate_ipv4_flow(uint8_t port_id, > > + cstructrte_flow_shared_action *shared_action, > > uint32_t src_ip, uint32_t src_mask, > > uint32_t dest_ip, uint32_t dest_mask, > > struct rte_flow_error *error) > > @@ -393,7 +436,6 @@ This function is located in the ``flow_blocks.c`` file. > > struct rte_flow_item pattern[MAX_PATTERN_NUM]; > > struct rte_flow_action action[MAX_ACTION_NUM]; > > struct rte_flow *flow = NULL; > > - struct rte_flow_action_queue queue = { .index = rx_q }; > > struct rte_flow_item_ipv4 ip_spec; > > struct rte_flow_item_ipv4 ip_mask; > > > > @@ -411,8 +453,8 @@ This function is located in the ``flow_blocks.c`` file. > > * create the action sequence. > > * one action only, move packet to queue > > */ > > - action[0].type = RTE_FLOW_ACTION_TYPE_QUEUE; > > - action[0].conf = &queue; > > + action[0].type = RTE_FLOW_ACTION_TYPE_SHARED; > > + action[0].conf = shared_action; > > action[1].type = RTE_FLOW_ACTION_TYPE_END; > > > > /* > > @@ -468,12 +510,12 @@ The following part create the flow attributes, in > our case ingress. > > attr.ingress = 1; > > > > The third part defines the action to be taken when a packet matches > > -the rule. In this case send the packet to queue. > > +the rule. In this case send the packet to single RSS queue. > > > > .. code-block:: c > > > > - action[0].type = RTE_FLOW_ACTION_TYPE_QUEUE; > > - action[0].conf = &queue; > > + action[0].type = RTE_FLOW_ACTION_TYPE_SHARED; > > + action[0].conf = shared_action; > > action[1].type = RTE_FLOW_ACTION_TYPE_END; > > > > The fourth part is responsible for creating the pattern and is built from > > diff --git a/examples/flow_filtering/flow_blocks.c > b/examples/flow_filtering/flow_blocks.c > > index 575d792810..99bfed3172 100644 > > --- a/examples/flow_filtering/flow_blocks.c > > +++ b/examples/flow_filtering/flow_blocks.c > > @@ -6,11 +6,11 @@ > > #define MAX_ACTION_NUM 2 > > > > struct rte_flow * > > -generate_ipv4_flow(uint16_t port_id, uint16_t rx_q, > > - uint32_t src_ip, uint32_t src_mask, > > - uint32_t dest_ip, uint32_t dest_mask, > > - struct rte_flow_error *error); > > - > > +generate_ipv4_flow(uint16_t port_id, > > + struct rte_flow_shared_action *shared_action, > > + uint32_t src_ip, uint32_t src_mask, > > + uint32_t dest_ip, uint32_t dest_mask, > > + struct rte_flow_error *error); > > > > /** > > * create a flow rule that sends packets with matching src and dest ip > > @@ -18,8 +18,8 @@ generate_ipv4_flow(uint16_t port_id, uint16_t rx_q, > > * > > * @param port_id > > * The selected port. > > - * @param rx_q > > - * The selected target queue. > > + * @param shared_action > > + * The shared RSS action with single queue > > * @param src_ip > > * The src ip value to match the input packet. > > * @param src_mask > > @@ -35,16 +35,16 @@ generate_ipv4_flow(uint16_t port_id, uint16_t rx_q, > > * A flow if the rule could be created else return NULL. > > */ > > struct rte_flow * > > -generate_ipv4_flow(uint16_t port_id, uint16_t rx_q, > > - uint32_t src_ip, uint32_t src_mask, > > - uint32_t dest_ip, uint32_t dest_mask, > > - struct rte_flow_error *error) > > +generate_ipv4_flow(uint16_t port_id, > > + struct rte_flow_shared_action *shared_action, > > + uint32_t src_ip, uint32_t src_mask, > > + uint32_t dest_ip, uint32_t dest_mask, > > + struct rte_flow_error *error) > > { > > struct rte_flow_attr attr; > > struct rte_flow_item pattern[MAX_PATTERN_NUM]; > > struct rte_flow_action action[MAX_ACTION_NUM]; > > struct rte_flow *flow = NULL; > > - struct rte_flow_action_queue queue = { .index = rx_q }; > > struct rte_flow_item_ipv4 ip_spec; > > struct rte_flow_item_ipv4 ip_mask; > > int res; > > @@ -61,10 +61,10 @@ generate_ipv4_flow(uint16_t port_id, uint16_t rx_q, > > > > /* > > * create the action sequence. > > - * one action only, move packet to queue > > + * one action only, move packet to shared RSS queue > > */ > > - action[0].type = RTE_FLOW_ACTION_TYPE_QUEUE; > > - action[0].conf = &queue; > > + action[0].type = RTE_FLOW_ACTION_TYPE_SHARED; > > + action[0].conf = shared_action; > > action[1].type = RTE_FLOW_ACTION_TYPE_END; > > > > /* > > diff --git a/examples/flow_filtering/main.c b/examples/flow_filtering/main.c > > index cc9e7e7808..d6b18d95fc 100644 > > --- a/examples/flow_filtering/main.c > > +++ b/examples/flow_filtering/main.c > > @@ -32,8 +32,7 @@ > > static volatile bool force_quit; > > > > static uint16_t port_id; > > -static uint16_t nr_queues = 5; > > -static uint8_t selected_queue = 1; > > +static uint16_t nr_queues = 2; > > struct rte_mempool *mbuf_pool; > > struct rte_flow *flow; > > > > @@ -42,6 +41,24 @@ struct rte_flow *flow; > > #define FULL_MASK 0xffffffff /* full mask */ > > #define EMPTY_MASK 0x0 /* empty mask */ > > > > +struct rte_flow_shared_action *shared_action; > > +uint16_t rss_queue[1] = {0}; > > + > > +struct rte_flow_action_rss action_rss = { > > + .func = RTE_ETH_HASH_FUNCTION_DEFAULT, > > + .level = 0, > > + .types = 0, > > + .key_len = 0, > > + .key = NULL, > > + .queue = rss_queue, > > + .queue_num = 1, > > +}; > > + > > +struct rte_flow_action action = { > > + .type = RTE_FLOW_ACTION_TYPE_RSS, > > + .conf = &action_rss, > > +}; > > + > > #include "flow_blocks.c" > > > > static inline void > > @@ -61,6 +78,7 @@ main_loop(void) > > uint16_t nb_rx; > > uint16_t i; > > uint16_t j; > > + int ret; > > > > while (!force_quit) { > > for (i = 0; i < nr_queues; i++) { > > @@ -82,6 +100,21 @@ main_loop(void) > > > > rte_pktmbuf_free(m); > > } > > + if (rss_queue[0] == 0) { > > + printf(">>> switching queue 0 -> 1\n"); > > + rss_queue[0] = 1; > > + } else { > > + printf(">>> switching queue 1 -> 0\n"); > > + rss_queue[0] = 0; > > + } > > + ret = rte_flow_shared_action_update > > + (port_id, shared_action, &action, > > + &error); > > + if (ret) > > + rte_exit(EXIT_FAILURE, > > + ":: error: RSS action update " > > + "failed: %s\n", > > + rte_strerror(-ret)); > > } > > } > > } > > @@ -243,8 +276,10 @@ main(int argc, char **argv) > > > > init_port(); > > > > + shared_action = rte_flow_shared_action_create(port_id, &action, > &error); > > + > > /* create flow for send packet with */ > > - flow = generate_ipv4_flow(port_id, selected_queue, > > + flow = generate_ipv4_flow(port_id, shared_action, > > SRC_IP, EMPTY_MASK, > > DEST_IP, FULL_MASK, &error); > > if (!flow) { > > -- > > 2.26.2 > > ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [dpdk-dev] [PATCH v2 6/6] examples/flow_filtering: utilize shared RSS action 2020-07-09 6:08 ` Ori Kam @ 2020-07-09 12:25 ` Andrey Vesnovaty 2020-07-09 12:39 ` Thomas Monjalon 0 siblings, 1 reply; 106+ messages in thread From: Andrey Vesnovaty @ 2020-07-09 12:25 UTC (permalink / raw) To: Ori Kam, jerinj, Ferruh Yigit Cc: dpdk-dev, jer, Thomas Monjalon, Ferruh Yigit, Stephen Hemminger, Richardson, Bruce, Slava Ovsiienko, Andrey Vesnovaty, Marko Kovacevic, Radu Nicolau, Akhil Goyal, Tomasz Kantecki, Sunil Kumar Kori, Pavan Nikhilesh, John McNamara Hi, Jerin and Ferruh First of all it's decided to postpone this effort to 20.11. @jerinj@marvell.com I sincerely believe we can work out great API for - shared action, context - action update / modification / replacement after 20.08 without time pressure & stress. I'm fully committed and will continue to work on this PATCH. I'll resend improved version of this patch once 20.08 is out & we will continue our discussion from here to make it even better. Many thanks to all participants of this discussion, Andrey -----Original Message----- From: Ori Kam <orika@mellanox.com> Sent: Thursday, July 9, 2020 9:09 AM To: Jerin Jacob <jerinjacobk@gmail.com>; Andrey Vesnovaty <andreyv@mellanox.com> Cc: dpdk-dev <dev@dpdk.org>; jer@marvell.com; Thomas Monjalon <thomas@monjalon.net>; Ferruh Yigit <ferruh.yigit@intel.com>; Stephen Hemminger <stephen@networkplumber.org>; Richardson, Bruce <bruce.richardson@intel.com>; Slava Ovsiienko <viacheslavo@mellanox.com>; Andrey Vesnovaty <andrey.vesnovaty@gmail.com>; Marko Kovacevic <marko.kovacevic@intel.com>; Radu Nicolau <radu.nicolau@intel.com>; Akhil Goyal <akhil.goyal@nxp.com>; Tomasz Kantecki <tomasz.kantecki@intel.com>; Sunil Kumar Kori <skori@marvell.com>; Pavan Nikhilesh <pbhagavatula@marvell.com>; John McNamara <john.mcnamara@intel.com> Subject: RE: [PATCH v2 6/6] examples/flow_filtering: utilize shared RSS action Hi Andrey, > -----Original Message----- > From: Jerin Jacob <jerinjacobk@gmail.com> > Sent: Thursday, July 9, 2020 7:44 AM > To: Andrey Vesnovaty <andreyv@mellanox.com> > Cc: dpdk-dev <dev@dpdk.org>; jer@marvell.com; Thomas Monjalon > <thomas@monjalon.net>; Ferruh Yigit <ferruh.yigit@intel.com>; Stephen > Hemminger <stephen@networkplumber.org>; Richardson, Bruce > <bruce.richardson@intel.com>; Ori Kam <orika@mellanox.com>; Slava > Ovsiienko <viacheslavo@mellanox.com>; Andrey Vesnovaty > <andrey.vesnovaty@gmail.com>; Marko Kovacevic > <marko.kovacevic@intel.com>; Radu Nicolau <radu.nicolau@intel.com>; Akhil > Goyal <akhil.goyal@nxp.com>; Tomasz Kantecki <tomasz.kantecki@intel.com>; > Sunil Kumar Kori <skori@marvell.com>; Pavan Nikhilesh > <pbhagavatula@marvell.com>; John McNamara <john.mcnamara@intel.com> > Subject: Re: [PATCH v2 6/6] examples/flow_filtering: utilize shared RSS action > > On Thu, Jul 9, 2020 at 3:09 AM Andrey Vesnovaty <andreyv@mellanox.com> > wrote: > > > > This commit gives very first shared RSS action usage example and > > demonstrates shared action capability for in-place update. > > > > First application creates shared action during initialization phase. > > What if PMD does not support a shared context, Then this application > fails to run? > if so, have a mode or probe the capability(if capability present then > switches to new mode else fall back to old mode) before changing the > application behavior. > +1 the new action should be added as a new case and should run only if it is the shared action is supported. > > > Later on the flow object created by application uses previously created > > shared RSS action with 1 queue configured instead of queue action in > > original application. > > > > On each RX queue burst shared RSS action reconfigured via > > rte_flow_shared_action_update() API to switch queue 0 to 1 & 1 to 0. > > User supposed to observe consistent queue switches on each packet burst. > > > > Signed-off-by: Andrey Vesnovaty <andreyv@mellanox.com> > > --- > > doc/guides/sample_app_ug/flow_filtering.rst | 62 +++++++++++++++++---- > > examples/flow_filtering/flow_blocks.c | 30 +++++----- > > examples/flow_filtering/main.c | 41 +++++++++++++- > > 3 files changed, 105 insertions(+), 28 deletions(-) > > > > diff --git a/doc/guides/sample_app_ug/flow_filtering.rst > b/doc/guides/sample_app_ug/flow_filtering.rst > > index 5e5a6cd8a0..cfe9334717 100644 > > --- a/doc/guides/sample_app_ug/flow_filtering.rst > > +++ b/doc/guides/sample_app_ug/flow_filtering.rst > > @@ -106,7 +106,7 @@ following code: > > .. code-block:: c > > > > /* create flow for send packet with */ > > - flow = generate_ipv4_flow(port_id, selected_queue, > > + flow = generate_ipv4_flow(port_id, shared_action, > > SRC_IP, EMPTY_MASK, > > DEST_IP, FULL_MASK, &error); > > if (!flow) { > > @@ -242,7 +242,7 @@ The Ethernet port is configured with default settings > using the > > rxq_conf = dev_info.default_rxconf; > > rxq_conf.offloads = port_conf.rxmode.offloads; > > > > -For this example we are configuring number of rx and tx queues that are > connected > > +For this example we are configuring 2 rx and 2 tx queues that are connected > > to a single port. > > > > .. code-block:: c > > @@ -270,13 +270,22 @@ to a single port. > > } > > } > > > > +Before we create the flow we create shared action in order to send it as > > +actions argument when creating a flow. The action is single queue RSS > action > > +similar to action queue with the only difference that shared RSS action > > +provides update capability after action creation. > > + > > +.. code-block:: c > > + > > + shared_action = rte_flow_shared_action_create(port_id, &action, &error); > > + > > In the next step we create and apply the flow rule. which is to send packets > > with destination ip equals to 192.168.1.1 to queue number 1. The detail > > explanation of the ``generate_ipv4_flow()`` appears later in this document: > > > > .. code-block:: c > > > > - flow = generate_ipv4_flow(port_id, selected_queue, > > + flow = generate_ipv4_flow(port_id, shared_action, > > SRC_IP, EMPTY_MASK, > > DEST_IP, FULL_MASK, &error); > > > > @@ -339,6 +348,21 @@ looks like the following: > > printf("\n"); > > rte_pktmbuf_free(m); > > } > > + if (rss_queue[0] == 0) { > > + printf(">>> switching queue 0 -> 1\n"); > > + rss_queue[0] = 1; > > + } else { > > + printf(">>> switching queue 1 -> 0\n"); > > + rss_queue[0] = 0; > > + } > > + ret = rte_flow_shared_action_update > > + (port_id, shared_action, &action, > > + &error); > > + if (ret) > > + rte_exit(EXIT_FAILURE, > > + ":: error: RSS action update " > > + "failed: %s\n", > > + rte_strerror(-ret)); > > } > > } > > } > > @@ -348,6 +372,8 @@ looks like the following: > > rte_eth_dev_close(port_id); > > } > > > > +On each loop eteration Rx queue switched using > > +``rte_flow_shared_action_update()`` API. > > The main work of the application is reading the packets from all > > queues and printing for each packet the destination queue: > > > > @@ -365,6 +391,21 @@ queues and printing for each packet the destination > queue: > > printf(" - queue=0x%x", (unsigned int)i); > > printf("\n"); > > rte_pktmbuf_free(m); > > + if (rss_queue[0] == 0) { > > + printf(">>> switching queue 0 -> 1\n"); > > + rss_queue[0] = 1; > > + } else { > > + printf(">>> switching queue 1 -> 0\n"); > > + rss_queue[0] = 0; > > + } > > + ret = rte_flow_shared_action_update > > + (port_id, shared_action, &action, > > + &error); > > + if (ret) > > + rte_exit(EXIT_FAILURE, > > + ":: error: RSS action update " > > + "failed: %s\n", > > + rte_strerror(-ret)); > > } > > } > > } > > @@ -378,13 +419,15 @@ The forwarding loop can be interrupted and the > application closed using > > The generate_ipv4_flow function > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > > > + > > The generate_ipv4_flow function is responsible for creating the flow rule. > > This function is located in the ``flow_blocks.c`` file. > > > > .. code-block:: c > > > > static struct rte_flow * > > - generate_ipv4_flow(uint8_t port_id, uint16_t rx_q, > > + generate_ipv4_flow(uint8_t port_id, > > + cstructrte_flow_shared_action *shared_action, > > uint32_t src_ip, uint32_t src_mask, > > uint32_t dest_ip, uint32_t dest_mask, > > struct rte_flow_error *error) > > @@ -393,7 +436,6 @@ This function is located in the ``flow_blocks.c`` file. > > struct rte_flow_item pattern[MAX_PATTERN_NUM]; > > struct rte_flow_action action[MAX_ACTION_NUM]; > > struct rte_flow *flow = NULL; > > - struct rte_flow_action_queue queue = { .index = rx_q }; > > struct rte_flow_item_ipv4 ip_spec; > > struct rte_flow_item_ipv4 ip_mask; > > > > @@ -411,8 +453,8 @@ This function is located in the ``flow_blocks.c`` file. > > * create the action sequence. > > * one action only, move packet to queue > > */ > > - action[0].type = RTE_FLOW_ACTION_TYPE_QUEUE; > > - action[0].conf = &queue; > > + action[0].type = RTE_FLOW_ACTION_TYPE_SHARED; > > + action[0].conf = shared_action; > > action[1].type = RTE_FLOW_ACTION_TYPE_END; > > > > /* > > @@ -468,12 +510,12 @@ The following part create the flow attributes, in > our case ingress. > > attr.ingress = 1; > > > > The third part defines the action to be taken when a packet matches > > -the rule. In this case send the packet to queue. > > +the rule. In this case send the packet to single RSS queue. > > > > .. code-block:: c > > > > - action[0].type = RTE_FLOW_ACTION_TYPE_QUEUE; > > - action[0].conf = &queue; > > + action[0].type = RTE_FLOW_ACTION_TYPE_SHARED; > > + action[0].conf = shared_action; > > action[1].type = RTE_FLOW_ACTION_TYPE_END; > > > > The fourth part is responsible for creating the pattern and is built from > > diff --git a/examples/flow_filtering/flow_blocks.c > b/examples/flow_filtering/flow_blocks.c > > index 575d792810..99bfed3172 100644 > > --- a/examples/flow_filtering/flow_blocks.c > > +++ b/examples/flow_filtering/flow_blocks.c > > @@ -6,11 +6,11 @@ > > #define MAX_ACTION_NUM 2 > > > > struct rte_flow * > > -generate_ipv4_flow(uint16_t port_id, uint16_t rx_q, > > - uint32_t src_ip, uint32_t src_mask, > > - uint32_t dest_ip, uint32_t dest_mask, > > - struct rte_flow_error *error); > > - > > +generate_ipv4_flow(uint16_t port_id, > > + struct rte_flow_shared_action *shared_action, > > + uint32_t src_ip, uint32_t src_mask, > > + uint32_t dest_ip, uint32_t dest_mask, > > + struct rte_flow_error *error); > > > > /** > > * create a flow rule that sends packets with matching src and dest ip > > @@ -18,8 +18,8 @@ generate_ipv4_flow(uint16_t port_id, uint16_t rx_q, > > * > > * @param port_id > > * The selected port. > > - * @param rx_q > > - * The selected target queue. > > + * @param shared_action > > + * The shared RSS action with single queue > > * @param src_ip > > * The src ip value to match the input packet. > > * @param src_mask > > @@ -35,16 +35,16 @@ generate_ipv4_flow(uint16_t port_id, uint16_t rx_q, > > * A flow if the rule could be created else return NULL. > > */ > > struct rte_flow * > > -generate_ipv4_flow(uint16_t port_id, uint16_t rx_q, > > - uint32_t src_ip, uint32_t src_mask, > > - uint32_t dest_ip, uint32_t dest_mask, > > - struct rte_flow_error *error) > > +generate_ipv4_flow(uint16_t port_id, > > + struct rte_flow_shared_action *shared_action, > > + uint32_t src_ip, uint32_t src_mask, > > + uint32_t dest_ip, uint32_t dest_mask, > > + struct rte_flow_error *error) > > { > > struct rte_flow_attr attr; > > struct rte_flow_item pattern[MAX_PATTERN_NUM]; > > struct rte_flow_action action[MAX_ACTION_NUM]; > > struct rte_flow *flow = NULL; > > - struct rte_flow_action_queue queue = { .index = rx_q }; > > struct rte_flow_item_ipv4 ip_spec; > > struct rte_flow_item_ipv4 ip_mask; > > int res; > > @@ -61,10 +61,10 @@ generate_ipv4_flow(uint16_t port_id, uint16_t rx_q, > > > > /* > > * create the action sequence. > > - * one action only, move packet to queue > > + * one action only, move packet to shared RSS queue > > */ > > - action[0].type = RTE_FLOW_ACTION_TYPE_QUEUE; > > - action[0].conf = &queue; > > + action[0].type = RTE_FLOW_ACTION_TYPE_SHARED; > > + action[0].conf = shared_action; > > action[1].type = RTE_FLOW_ACTION_TYPE_END; > > > > /* > > diff --git a/examples/flow_filtering/main.c b/examples/flow_filtering/main.c > > index cc9e7e7808..d6b18d95fc 100644 > > --- a/examples/flow_filtering/main.c > > +++ b/examples/flow_filtering/main.c > > @@ -32,8 +32,7 @@ > > static volatile bool force_quit; > > > > static uint16_t port_id; > > -static uint16_t nr_queues = 5; > > -static uint8_t selected_queue = 1; > > +static uint16_t nr_queues = 2; > > struct rte_mempool *mbuf_pool; > > struct rte_flow *flow; > > > > @@ -42,6 +41,24 @@ struct rte_flow *flow; > > #define FULL_MASK 0xffffffff /* full mask */ > > #define EMPTY_MASK 0x0 /* empty mask */ > > > > +struct rte_flow_shared_action *shared_action; > > +uint16_t rss_queue[1] = {0}; > > + > > +struct rte_flow_action_rss action_rss = { > > + .func = RTE_ETH_HASH_FUNCTION_DEFAULT, > > + .level = 0, > > + .types = 0, > > + .key_len = 0, > > + .key = NULL, > > + .queue = rss_queue, > > + .queue_num = 1, > > +}; > > + > > +struct rte_flow_action action = { > > + .type = RTE_FLOW_ACTION_TYPE_RSS, > > + .conf = &action_rss, > > +}; > > + > > #include "flow_blocks.c" > > > > static inline void > > @@ -61,6 +78,7 @@ main_loop(void) > > uint16_t nb_rx; > > uint16_t i; > > uint16_t j; > > + int ret; > > > > while (!force_quit) { > > for (i = 0; i < nr_queues; i++) { > > @@ -82,6 +100,21 @@ main_loop(void) > > > > rte_pktmbuf_free(m); > > } > > + if (rss_queue[0] == 0) { > > + printf(">>> switching queue 0 -> 1\n"); > > + rss_queue[0] = 1; > > + } else { > > + printf(">>> switching queue 1 -> 0\n"); > > + rss_queue[0] = 0; > > + } > > + ret = rte_flow_shared_action_update > > + (port_id, shared_action, &action, > > + &error); > > + if (ret) > > + rte_exit(EXIT_FAILURE, > > + ":: error: RSS action update " > > + "failed: %s\n", > > + rte_strerror(-ret)); > > } > > } > > } > > @@ -243,8 +276,10 @@ main(int argc, char **argv) > > > > init_port(); > > > > + shared_action = rte_flow_shared_action_create(port_id, &action, > &error); > > + > > /* create flow for send packet with */ > > - flow = generate_ipv4_flow(port_id, selected_queue, > > + flow = generate_ipv4_flow(port_id, shared_action, > > SRC_IP, EMPTY_MASK, > > DEST_IP, FULL_MASK, &error); > > if (!flow) { > > -- > > 2.26.2 > > ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [dpdk-dev] [PATCH v2 6/6] examples/flow_filtering: utilize shared RSS action 2020-07-09 12:25 ` Andrey Vesnovaty @ 2020-07-09 12:39 ` Thomas Monjalon 0 siblings, 0 replies; 106+ messages in thread From: Thomas Monjalon @ 2020-07-09 12:39 UTC (permalink / raw) To: Andrey Vesnovaty Cc: Ori Kam, jerinj, Ferruh Yigit, dev, jer, Ferruh Yigit, Stephen Hemminger, Richardson, Bruce, Slava Ovsiienko, Andrey Vesnovaty, Marko Kovacevic, Radu Nicolau, Akhil Goyal, Tomasz Kantecki, Sunil Kumar Kori, Pavan Nikhilesh, John McNamara 09/07/2020 14:25, Andrey Vesnovaty: > Hi, Jerin and Ferruh > > First of all it's decided to postpone this effort to 20.11. > @jerinj@marvell.com I sincerely believe we can work out great API for > - shared action, context > - action update / modification / replacement > after 20.08 without time pressure & stress. > > I'm fully committed and will continue to work on this PATCH. > I'll resend improved version of this patch once 20.08 is out & we will > continue our discussion from here to make it even better. I don't think you should wait 20.08 (one month) to progress. Ideally, it should be ready to be merged in the first day of 20.11 cycle. Please add the feature in the release notes (even in 20.08 one in the meantime). ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [dpdk-dev] [PATCH v2 0/6] add flow shared action API + PMD 2020-07-08 21:39 ` [dpdk-dev] [PATCH v2 0/6] add flow shared action API + PMD Andrey Vesnovaty ` (5 preceding siblings ...) 2020-07-08 21:39 ` [dpdk-dev] [PATCH v2 6/6] examples/flow_filtering: utilize shared RSS action Andrey Vesnovaty @ 2020-07-09 4:39 ` Jerin Jacob 6 siblings, 0 replies; 106+ messages in thread From: Jerin Jacob @ 2020-07-09 4:39 UTC (permalink / raw) To: Andrey Vesnovaty Cc: dpdk-dev, jer, Thomas Monjalon, Ferruh Yigit, Stephen Hemminger, Richardson, Bruce, Ori Kam, Slava Ovsiienko, Andrey Vesnovaty On Thu, Jul 9, 2020 at 3:09 AM Andrey Vesnovaty <andreyv@mellanox.com> wrote: > > Hi Jerin and Ori. > > V2 changes: > - First version of PMD implementation for shared action API. > - Simple example application demonstaration shared action update. > - Changes to shred action API according to discussion of V1. > > @Jerin Jacob: please take a look at provided example, hopefully it > will help to converge our API discussion and reach consensus on it. I am sorry, Based on my understanding it is not addressing API sequence query[1] [1] --- http://mails.dpdk.org/archives/dev/2020-July/173997.html ie. Otherway to ask is, Could you have share the API call sequence using "rte_flow_shared_action_update(uint16_port port, rte_shared_ctx *ctx, rte_flow_action *action, error)" to enable support for the following category of HW as I mentioned earlier. - The HW has "pattern" and "action" mapped to different HW objects and action can be updated any time without destroying and create.(a,k,a Does not have shared HW object) --- > > Thanks, > Andrey > > Andrey Vesnovaty (6): > ethdev: add flow shared action API > common/mlx5: modify advanced Rx object via DevX > net/mlx5: modify hash Rx queue objects > net/mlx5: shared action PMD > net/mlx5: driver support for shared action > examples/flow_filtering: utilize shared RSS action > > doc/guides/sample_app_ug/flow_filtering.rst | 62 +- > drivers/common/mlx5/mlx5_devx_cmds.c | 84 +++ > drivers/common/mlx5/mlx5_devx_cmds.h | 10 + > drivers/common/mlx5/mlx5_prm.h | 29 + > .../common/mlx5/rte_common_mlx5_version.map | 1 + > drivers/net/mlx5/mlx5.c | 1 + > drivers/net/mlx5/mlx5.h | 2 + > drivers/net/mlx5/mlx5_defs.h | 3 + > drivers/net/mlx5/mlx5_flow.c | 492 ++++++++++++- > drivers/net/mlx5/mlx5_flow.h | 83 +++ > drivers/net/mlx5/mlx5_flow_dv.c | 671 +++++++++++++++++- > drivers/net/mlx5/mlx5_rxq.c | 300 ++++++-- > drivers/net/mlx5/mlx5_rxtx.h | 4 + > examples/flow_filtering/flow_blocks.c | 30 +- > examples/flow_filtering/main.c | 41 +- > lib/librte_ethdev/rte_ethdev_version.map | 6 + > lib/librte_ethdev/rte_flow.c | 81 +++ > lib/librte_ethdev/rte_flow.h | 148 +++- > lib/librte_ethdev/rte_flow_driver.h | 22 + > 19 files changed, 1924 insertions(+), 146 deletions(-) > > -- > 2.26.2 > ^ permalink raw reply [flat|nested] 106+ messages in thread
* [dpdk-dev] [PATCH v4 0/2] RTE flow shared action 2020-07-02 12:05 [dpdk-dev] [PATCH] add flow shared action API Andrey Vesnovaty 2020-07-03 15:02 ` Jerin Jacob 2020-07-08 21:39 ` [dpdk-dev] [PATCH v2 0/6] add flow shared action API + PMD Andrey Vesnovaty @ 2020-10-06 20:08 ` Andrey Vesnovaty 2020-10-06 20:08 ` [dpdk-dev] [PATCH v4 1/2] ethdev: add flow shared action API Andrey Vesnovaty 2020-10-06 20:08 ` [dpdk-dev] [PATCH v4 2/2] app/testpmd: support shared action Andrey Vesnovaty 2020-10-07 12:56 ` [dpdk-dev] [PATCH v5 0/2] RTE flow " Andrey Vesnovaty ` (3 subsequent siblings) 6 siblings, 2 replies; 106+ messages in thread From: Andrey Vesnovaty @ 2020-10-06 20:08 UTC (permalink / raw) To: dev Cc: jer, jerinjacobk, thomas, ferruh.yigit, stephen, bruce.richardson, orika, viacheslavo, andrey.vesnovaty, mdr, nhorman, ajit.khaparde, samik.gupta This patchset introduces shared action API for RTE flow. V4 changes: * patchset split to: 1. shared action API (this patchset) 2. mlx5 PMD implementation for shared action API * testpmd for shared action CLI documented * release notes for 20.11 updated with shared action feature * misc spelling fixes and rephrase in commit messages * flow_filtering example removed in favor of testpmd Andrey Vesnovaty (2): ethdev: add flow shared action API app/testpmd: support shared action app/test-pmd/cmdline_flow.c | 262 +++++++++++++++++++- app/test-pmd/config.c | 217 ++++++++++++++++ app/test-pmd/testpmd.h | 19 ++ doc/guides/rel_notes/release_20_11.rst | 9 + doc/guides/testpmd_app_ug/testpmd_funcs.rst | 111 +++++++++ lib/librte_ethdev/rte_ethdev_version.map | 4 + lib/librte_ethdev/rte_flow.c | 84 +++++++ lib/librte_ethdev/rte_flow.h | 161 +++++++++++- lib/librte_ethdev/rte_flow_driver.h | 23 ++ 9 files changed, 888 insertions(+), 2 deletions(-) -- 2.26.2 ^ permalink raw reply [flat|nested] 106+ messages in thread
* [dpdk-dev] [PATCH v4 1/2] ethdev: add flow shared action API 2020-10-06 20:08 ` [dpdk-dev] [PATCH v4 0/2] RTE flow shared action Andrey Vesnovaty @ 2020-10-06 20:08 ` Andrey Vesnovaty 2020-10-07 6:27 ` Ori Kam 2020-10-06 20:08 ` [dpdk-dev] [PATCH v4 2/2] app/testpmd: support shared action Andrey Vesnovaty 1 sibling, 1 reply; 106+ messages in thread From: Andrey Vesnovaty @ 2020-10-06 20:08 UTC (permalink / raw) To: dev Cc: jer, jerinjacobk, thomas, ferruh.yigit, stephen, bruce.richardson, orika, viacheslavo, andrey.vesnovaty, mdr, nhorman, ajit.khaparde, samik.gupta, John McNamara, Marko Kovacevic, Andrew Rybchenko, Ori Kam This commit introduces extension of DPDK flow action API enabling sharing of single rte_flow_action in multiple flows. The API intended for PMDs, where multiple HW offloaded flows can reuse the same HW essence/object representing flow action and modification of such an essence/object affects all the rules using it. Motivation and example === Adding or removing one or more queues to RSS used by multiple flow rules imposes per rule toll for current DPDK flow API; the scenario requires for each flow sharing cloned RSS action: - call `rte_flow_destroy()` - call `rte_flow_create()` with modified RSS action API for sharing action and its in-place update benefits: - reduce the overhead of multiple RSS flow rules reconfiguration - optimize resource utilization by sharing action across multiple flows Change description === Shared action === In order to represent flow action shared by multiple flows new action type RTE_FLOW_ACTION_TYPE_SHARED is introduced (see `enum rte_flow_action_type`). Actually the introduced API decouples action from any specific flow and enables sharing of single action by its handle across multiple flows. Shared action create/use/destroy === Shared action may be reused by some or none flow rules at any given moment, i.e. shared action reside outside of the context of any flow. Shared action represent HW resources/objects used for action offloading implementation. API for shared action create (see `rte_flow_shared_action_create()`): - should allocate HW resources and make related initializations required for shared action implementation. - make necessary preparations to maintain shared access to the action resources, configuration and state. API for shared action destroy (see `rte_flow_shared_action_destroy()`) should release HW resources and make related cleanups required for shared action implementation. In order to share some flow action reuse the handle of type `struct rte_flow_shared_action` returned by rte_flow_shared_action_create() as a `conf` field of `struct rte_flow_action` (see "example" section). If some shared action not used by any flow rule all resources allocated by the shared action can be released by rte_flow_shared_action_destroy() (see "example" section). The shared action handle passed as argument to destroy API should not be used any further i.e. result of the usage is undefined. Shared action re-configuration === Shared action behavior defined by its configuration can be updated via rte_flow_shared_action_update() (see "example" section). The shared action update operation modifies HW related resources/objects allocated on the action creation. The number of operations performed by the update operation should not depend on the number of flows sharing the related action. On return of shared action update API action behavior should be according to updated configuration for all flows sharing the action. Shared action query === Provide separate API to query shared action state (see rte_flow_shared_action_update()). Taking a counter as an example: query returns value aggregating all counter increments across all flow rules sharing the counter. This API doesn't query shared action configuration since it is controlled by rte_flow_shared_action_create() and rte_flow_shared_action_update() APIs and no supposed to change by other means. PMD support === The support of introduced API is pure PMD specific design and responsibility for each action type (see struct rte_flow_ops). testpmd === In order to utilize introduced API testpmd cli may implement following extension create/update/destroy/query shared action accordingly flow shared_action (port) create {action_id (id)} (action) / end flow shared_action (port) update (id) (action) / end flow shared_action (port) destroy action_id (id) {action_id (id) [...]} flow shared_action (port) query (id) testpmd example === configure rss to queues 1 & 2 > flow shared_action 0 create action_id 100 rss queues 1 2 end / end create flow rule utilizing shared action > flow create 0 ingress \ pattern eth dst is 0c:42:a1:15:fd:ac / ipv6 / tcp / end \ actions shared 100 / end add 2 more queues > flow shared_action 0 modify 100 rss queues 1 2 3 4 end / end example === struct rte_flow_action actions[2]; struct rte_flow_action action; /* skipped: initialize action */ struct rte_flow_shared_action *handle = rte_flow_shared_action_create( port_id, &action, &error); actions[0].type = RTE_FLOW_ACTION_TYPE_SHARED; actions[0].conf = handle; actions[1].type = RTE_FLOW_ACTION_TYPE_END; /* skipped: init attr0 & pattern0 args */ struct rte_flow *flow0 = rte_flow_create(port_id, &attr0, pattern0, actions, error); /* create more rules reusing shared action */ struct rte_flow *flow1 = rte_flow_create(port_id, &attr1, pattern1, actions, error); /* skipped: for flows 2 ti