From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id D6BD7A04BA; Wed, 7 Oct 2020 22:11:05 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 1C9072BC7; Wed, 7 Oct 2020 22:11:04 +0200 (CEST) Received: from hqnvemgate25.nvidia.com (hqnvemgate25.nvidia.com [216.228.121.64]) by dpdk.org (Postfix) with ESMTP id 77CEA2B8C for ; Wed, 7 Oct 2020 22:11:02 +0200 (CEST) Received: from hqmail.nvidia.com (Not Verified[216.228.121.13]) by hqnvemgate25.nvidia.com (using TLS: TLSv1.2, AES256-SHA) id ; Wed, 07 Oct 2020 13:10:04 -0700 Received: from HQMAIL111.nvidia.com (172.20.187.18) by HQMAIL111.nvidia.com (172.20.187.18) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Wed, 7 Oct 2020 20:10:58 +0000 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (104.47.58.169) by HQMAIL111.nvidia.com (172.20.187.18) with Microsoft SMTP Server (TLS) id 15.0.1473.3 via Frontend Transport; Wed, 7 Oct 2020 20:10:58 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=lk1RMapvdetCyGEgJ9l3sZ8WWpN0baSrETWSpKBNulK0J9y5fSA/s2vZ2ZLVGSaeaXH5SzXM3q6jYereCC6i8HP6nlkpMm99m+VTwJqXgU6xtOYo4g2CgVAfOQ1kXPpzchFQ1O13jLoDGeSa5lYkQXEIYtVX6qykRYMPAn1xaUn/+/N4J8uexRjcoh/NjR5z9pc1dte27xJQz50rOLowqQ1fBI5w7hKiwvCxJOKKF1P6pbqBXxayQjmIFjPtGHbin1U5q5Cvpe+axq9CepN/JAb1WFR6jz28PRtA1CHGnho1BZve0NMQdf1WBM35Ry8BpGNJxvWn8CjoCFk9LDv9Gw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=OIxFeVnjVaPv8teRBVWZeoIUOZXLU4nPka/d6iMwerA=; b=aaZ2DanJZuHW8BMvrNiNJEmMu/yeDXOv2wbFKyOuJ1r6v//n0ziexXbalZr4OBrR/PpcSZF/G0kpHpaSVGQVwUc1KpPsiA2Mh9XlvCZM38wK7+PSck17khv/WbvNHY0lwIzeF51Q9gT8yVG8eQ6g7Na75ZaVBwsIboAK0NLrfZULQ29rh/yE0E6AOMgkJme6IZUjyzCfLKLruUCqoUNIi+Hinar2DmwAOx0r7Lo7LW0rNQSrYv1TWC+nSPpYCq/leEZPTmuhHQNSZn4UFT7OqviUNNz3Z+uH+OQ6Qte8qFDqkJt4H7Lqt8OoY8oI7UQQF8cD01Y7SlBQVVF3s4HmxQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none Received: from MW2PR12MB2492.namprd12.prod.outlook.com (2603:10b6:907:8::19) by MWHPR1201MB2510.namprd12.prod.outlook.com (2603:10b6:300:e8::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3455.21; Wed, 7 Oct 2020 20:10:56 +0000 Received: from MW2PR12MB2492.namprd12.prod.outlook.com ([fe80::39f3:2b4b:6018:4b88]) by MW2PR12MB2492.namprd12.prod.outlook.com ([fe80::39f3:2b4b:6018:4b88%7]) with mapi id 15.20.3455.022; Wed, 7 Oct 2020 20:10:56 +0000 From: Matan Azrad To: Suanming Mou , Ori Kam , NBU-Contact-Thomas Monjalon , Ferruh Yigit , Andrew Rybchenko CC: "dev@dpdk.org" Thread-Topic: [dpdk-dev] [PATCH v3 2/2] ethdev: make rte_flow API thread safe Thread-Index: AQHWnLTOtyuVNrvAgES3xPb6PAGqDamMjPNQ Date: Wed, 7 Oct 2020 20:10:56 +0000 Message-ID: References: <1601194817-208834-1-git-send-email-suanmingm@nvidia.com> <1602080249-36533-1-git-send-email-suanmingm@nvidia.com> <1602080249-36533-3-git-send-email-suanmingm@nvidia.com> In-Reply-To: <1602080249-36533-3-git-send-email-suanmingm@nvidia.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: nvidia.com; dkim=none (message not signed) header.d=none;nvidia.com; dmarc=none action=none header.from=nvidia.com; x-originating-ip: [77.125.45.185] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 2aff3c48-3b3a-405c-182f-08d86afd19da x-ms-traffictypediagnostic: MWHPR1201MB2510: x-ld-processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:4502; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: z9wAUGWWemaiv5H+qufU81ZWQiFX8QeO9uaEOqtAhbc3cgsOl727hNJ+XU0Drwm0f8ndIPnDJ/nmEd4gBilfStvJhJ2eaiwq0WUzhhZG1FmpKnkX5gdvuqnCLHgMXWNOnGxqNUMzOXmbE6yVl41JVPtk9R98suWLhozb/TzWwqqhWe0Ie88RR4jz3BwsAToByDYP6wnfpVp82SvuSYeEFo7JX1xB1IQ0GI+TjUviz+dB5YlTWmiOq3bRxpMgSLzdRnKzodIDqBTx2GqSXW8ebmK4PrrFXlcGjfywgqlechIlLSgzd9FuNngE057XM4mKlKyZuHFL9HXME//GwRj1cQ== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MW2PR12MB2492.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(39860400002)(396003)(346002)(136003)(376002)(366004)(83380400001)(30864003)(9686003)(2906002)(316002)(478600001)(110136005)(55016002)(186003)(52536014)(7696005)(6506007)(71200400001)(5660300002)(86362001)(8936002)(26005)(66946007)(66476007)(4326008)(64756008)(66556008)(76116006)(33656002)(8676002)(66446008); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: bWZqyuyX0Bh8Bllox10QF6cuUE/D2Czmy567I1DAmuiNzSXjdha6SAyoCh4YB+Ibuif5u6WKNuV55zV1tqewuJXaIBqzvOFWlGT2pJ1tkBqFfYBvkrNdAG/OUVXuJlXH6qWkeW9VLKuXrPExMaV6Jjzymca/WzSoFmwiSMsPtBLL0+KsIMg1CkBYW0TFB6Pd7yGx2N/GqMI9WrAkrNCjIFNEVktjtHsGVgilhr0KmKxUsEUvZLwVXXnmi5ZDeai6hoFoZCTo90Y5ysNh3yR5C5J8Cd8xoRSIVLGtDvPA297ha23g1mK9B9/DLtkEDLkNttSfijHaFFx7hNXOkOJINB8J7KJdCVH85ElZuAlsK4QXvwhcphyNKrrES+GMXM4ypjKaff/cae8qo8jJ3Lx06NTvr7zNLtmsASmeQJtZ631v21n/Yl5QAmVYDkquG+1jHh61Lm10o30wRkJJ/O61zQB9OUCnsd5gqJKimwNAlVWdX6nVtjAvkI0cK8V4KPzvM9lUH5m4ntJ23iBxgbGJGfSymCjy2elUHmAqKUmWj4kpH9WcM9haOFFHNgn6l4GjGDyLy2EIFnJjT59vwUYGvYtO513WteSc4S9Qy9G9dy15sfQM+5LmtCerZNBSOiepPEbDAGJzmIAzUzNN8MeDBA== Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MW2PR12MB2492.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2aff3c48-3b3a-405c-182f-08d86afd19da X-MS-Exchange-CrossTenant-originalarrivaltime: 07 Oct 2020 20:10:56.3969 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: n/AdlxeQlDHPsGUo1L8Kgeb9yJayD6en+l38kc8Wfua6l9QtmGElhEzmTm+NqyR7Lx2d32kOGbCs7K7hxIzkFQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR1201MB2510 X-OriginatorOrg: Nvidia.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nvidia.com; s=n1; t=1602101405; bh=OIxFeVnjVaPv8teRBVWZeoIUOZXLU4nPka/d6iMwerA=; h=ARC-Seal:ARC-Message-Signature:ARC-Authentication-Results:From:To: CC:Subject:Thread-Topic:Thread-Index:Date:Message-ID:References: In-Reply-To:Accept-Language:Content-Language:X-MS-Has-Attach: X-MS-TNEF-Correlator:authentication-results:x-originating-ip: x-ms-publictraffictype:x-ms-office365-filtering-correlation-id: x-ms-traffictypediagnostic:x-ld-processed: x-ms-exchange-transport-forked:x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers:x-ms-exchange-senderadcheck: x-microsoft-antispam:x-microsoft-antispam-message-info: x-forefront-antispam-report:x-ms-exchange-antispam-messagedata: Content-Type:Content-Transfer-Encoding:MIME-Version: X-MS-Exchange-CrossTenant-AuthAs: X-MS-Exchange-CrossTenant-AuthSource: X-MS-Exchange-CrossTenant-Network-Message-Id: X-MS-Exchange-CrossTenant-originalarrivaltime: X-MS-Exchange-CrossTenant-fromentityheader: X-MS-Exchange-CrossTenant-id:X-MS-Exchange-CrossTenant-mailboxtype: X-MS-Exchange-CrossTenant-userprincipalname: X-MS-Exchange-Transport-CrossTenantHeadersStamped:X-OriginatorOrg; b=VyN3S1f7RzSBZv3sbEhGNuChMLswzp1fGvCTzB/2udtovpqiXKSYBkgov0XHWIZQu XQhtIiixBBTTpLpUlB/dQxmElwKeQ8dpzTFfS9jH11AKBnczUiYApr5oDE178WZT12 zriqpum/MulKSDSn49WwzuL8hllI4EMzVcxmS+Gx+66t+zLN0PSq6JX7oUdkPE9jrz v+R56e6/NACcDKvkOVQIrYUw5t77dIhZEjJg2amgSdBCbNiQMDL6y/KteddWROZRhh YEm2k6/Aun76TPh7EV/pTqUBta8VQwPX//e9PZ0YwxABV4IU3G96bmEs9coLtDiWAh dccxl11gZq7Gg== Subject: Re: [dpdk-dev] [PATCH v3 2/2] ethdev: make rte_flow API thread safe X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Suanming Mou > Currently, the rte_flow functions are not defined as thread safe. > DPDK applications either call the functions in single thread or add locks > around the functions for the critical section. >=20 Better to write here "or protect any concurrent calling for the rte_flow op= erations using a lock." > For PMDs support the flow operations thread safe natively, the redundant > protection in application hurts the performance of the rte_flow operation > functions. >=20 > And the restriction of thread safe not guaranteed for the rte_flow functi= ons not =3D> is not > also limits the applications' expectation. >=20 > This feature is going to change the rte_flow functions to be thread safe.= As > different PMDs have different flow operations, some may support thread > safe already and others may not. For PMDs don't support flow thread safe > operation, a new lock is defined in ethdev in order to protects thread un= safe > PMDs from rte_flow level. >=20 > A new RTE_ETH_DEV_FLOW_OPS_THREAD_SAFE device flag is added to > determine whether the PMD supports thread safe flow operation or not. > For PMDs support thread safe flow operations, set the > RTE_ETH_DEV_FLOW_OPS_THREAD_SAFE flag, rte_flow level functions will > skip the thread safe helper lock for these PMDs. Again the rte_flow level > thread safe lock only works when PMD operation functions are not thread > safe. >=20 > For the PMDs which don't want the default mutex lock, just set the flag i= n > the PMD, and add the prefer type of lock in the PMD. Then the default > mutex lock is easily replaced by the PMD level lock. >=20 > The change has no effect on the current DPDK applications. No change is > required for the current DPDK applications. For the standard posix > pthread_mutex, if no lock contention with the added rte_flow level mutex, > the mutex only does the atomic increasing in > pthread_mutex_lock() and decreasing in > pthread_mutex_unlock(). No futex() syscall will be involved. >=20 > Signed-off-by: Suanming Mou > --- >=20 > v3: > - update flow_lock/unlock -> fts_enter/exit >=20 > v2: > - Update commit info and description doc. > - Add inline for the flow lock and unlock functions. > - Remove the PMD sample part flag configuration. >=20 > --- >=20 > doc/guides/prog_guide/rte_flow.rst | 9 ++-- > lib/librte_ethdev/rte_ethdev.c | 2 + > lib/librte_ethdev/rte_ethdev.h | 2 + > lib/librte_ethdev/rte_ethdev_core.h | 4 ++ > lib/librte_ethdev/rte_flow.c | 84 ++++++++++++++++++++++++++++--- > ------ > 5 files changed, 78 insertions(+), 23 deletions(-) >=20 > diff --git a/doc/guides/prog_guide/rte_flow.rst > b/doc/guides/prog_guide/rte_flow.rst > index 119b128..ae2ddb3 100644 > --- a/doc/guides/prog_guide/rte_flow.rst > +++ b/doc/guides/prog_guide/rte_flow.rst > @@ -3046,10 +3046,6 @@ Caveats > - API operations are synchronous and blocking (``EAGAIN`` cannot be > returned). >=20 > -- There is no provision for re-entrancy/multi-thread safety, although no= thing > - should prevent different devices from being configured at the same > - time. PMDs may protect their control path functions accordingly. > - > - Stopping the data path (TX/RX) should not be necessary when managing > flow > rules. If this cannot be achieved naturally or with workarounds (such = as > temporarily replacing the burst function pointers), an appropriate err= or @@ > -3101,6 +3097,11 @@ This interface additionally defines the following hel= per > function: > - ``rte_flow_ops_get()``: get generic flow operations structure from a > port. >=20 > +If PMD interfaces do not support re-entrancy/multi-thread safety, > +rte_flow level functions will do it by mutex. The application can test Good to mention mutex per port. > +the dev_flags with RTE_ETH_DEV_FLOW_OPS_THREAD_SAFE in struct > +rte_eth_dev_data to know if the rte_flow thread-safe works under > rte_flow level or PMD level. > + If think it will be helpful to add here a note saying that it is unsafe to = call other control commands In parallel to rte_flow operations.=20 > More will be added over time. >=20 > Device compatibility > diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethde= v.c > index 0f56541..60677fe 100644 > --- a/lib/librte_ethdev/rte_ethdev.c > +++ b/lib/librte_ethdev/rte_ethdev.c > @@ -500,6 +500,7 @@ struct rte_eth_dev * > strlcpy(eth_dev->data->name, name, sizeof(eth_dev->data- > >name)); > eth_dev->data->port_id =3D port_id; > eth_dev->data->mtu =3D RTE_ETHER_MTU; > + pthread_mutex_init(ð_dev->data->fts_mutex, NULL); >=20 > unlock: > rte_spinlock_unlock(&rte_eth_dev_shared_data->ownership_lock); > @@ -564,6 +565,7 @@ struct rte_eth_dev * > rte_free(eth_dev->data->mac_addrs); > rte_free(eth_dev->data->hash_mac_addrs); > rte_free(eth_dev->data->dev_private); > + pthread_mutex_destroy(ð_dev->data->fts_mutex); > memset(eth_dev->data, 0, sizeof(struct > rte_eth_dev_data)); > } >=20 > diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethde= v.h > index d2bf74f..03612fd 100644 > --- a/lib/librte_ethdev/rte_ethdev.h > +++ b/lib/librte_ethdev/rte_ethdev.h > @@ -1664,6 +1664,8 @@ struct rte_eth_dev_owner { #define > RTE_ETH_DEV_REPRESENTOR 0x0010 > /** Device does not support MAC change after started */ #define > RTE_ETH_DEV_NOLIVE_MAC_ADDR 0x0020 > +/** Device PMD supports thread safety flow operation */ #define > +RTE_ETH_DEV_FLOW_OPS_THREAD_SAFE 0x0040 >=20 > /** > * Iterates over valid ethdev ports owned by a specific owner. > diff --git a/lib/librte_ethdev/rte_ethdev_core.h > b/lib/librte_ethdev/rte_ethdev_core.h > index fd3bf92..89df65a 100644 > --- a/lib/librte_ethdev/rte_ethdev_core.h > +++ b/lib/librte_ethdev/rte_ethdev_core.h > @@ -5,6 +5,9 @@ > #ifndef _RTE_ETHDEV_CORE_H_ > #define _RTE_ETHDEV_CORE_H_ >=20 > +#include > +#include > + > /** > * @file > * > @@ -180,6 +183,7 @@ struct rte_eth_dev_data { > * Valid if RTE_ETH_DEV_REPRESENTOR in dev_flags. > */ >=20 > + pthread_mutex_t fts_mutex; /**< rte flow ops thread safety mutex. > */ > uint64_t reserved_64s[4]; /**< Reserved for future fields */ > void *reserved_ptrs[4]; /**< Reserved for future fields */ > } __rte_cache_aligned; > diff --git a/lib/librte_ethdev/rte_flow.c b/lib/librte_ethdev/rte_flow.c = index > f8fdd68..6823458 100644 > --- a/lib/librte_ethdev/rte_flow.c > +++ b/lib/librte_ethdev/rte_flow.c > @@ -207,6 +207,20 @@ struct rte_flow_desc_data { > return -rte_errno; > } >=20 > +static inline void > +fts_enter(struct rte_eth_dev *dev) > +{ > + if (!(dev->data->dev_flags & > RTE_ETH_DEV_FLOW_OPS_THREAD_SAFE)) > + pthread_mutex_lock(&dev->data->fts_mutex); > +} > + > +static inline void > +fts_exit(struct rte_eth_dev *dev) > +{ > + if (!(dev->data->dev_flags & > RTE_ETH_DEV_FLOW_OPS_THREAD_SAFE)) > + pthread_mutex_unlock(&dev->data->fts_mutex); > +} > + > static int > flow_err(uint16_t port_id, int ret, struct rte_flow_error *error) { @@ = - > 346,12 +360,16 @@ struct rte_flow_desc_data { { > const struct rte_flow_ops *ops =3D rte_flow_ops_get(port_id, error); > struct rte_eth_dev *dev =3D &rte_eth_devices[port_id]; > + int ret; >=20 > if (unlikely(!ops)) > return -rte_errno; > - if (likely(!!ops->validate)) > - return flow_err(port_id, ops->validate(dev, attr, pattern, > - actions, error), error); > + if (likely(!!ops->validate)) { > + fts_enter(dev); > + ret =3D ops->validate(dev, attr, pattern, actions, error); > + fts_exit(dev); > + return flow_err(port_id, ret, error); > + } > return rte_flow_error_set(error, ENOSYS, > RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > NULL, rte_strerror(ENOSYS)); > @@ -372,7 +390,9 @@ struct rte_flow * > if (unlikely(!ops)) > return NULL; > if (likely(!!ops->create)) { > + fts_enter(dev); > flow =3D ops->create(dev, attr, pattern, actions, error); > + fts_exit(dev); > if (flow =3D=3D NULL) > flow_err(port_id, -rte_errno, error); > return flow; > @@ -390,12 +410,16 @@ struct rte_flow * > { > struct rte_eth_dev *dev =3D &rte_eth_devices[port_id]; > const struct rte_flow_ops *ops =3D rte_flow_ops_get(port_id, error); > + int ret; >=20 > if (unlikely(!ops)) > return -rte_errno; > - if (likely(!!ops->destroy)) > - return flow_err(port_id, ops->destroy(dev, flow, error), > - error); > + if (likely(!!ops->destroy)) { > + fts_enter(dev); > + ret =3D ops->destroy(dev, flow, error); > + fts_exit(dev); > + return flow_err(port_id, ret, error); > + } > return rte_flow_error_set(error, ENOSYS, > RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > NULL, rte_strerror(ENOSYS)); > @@ -408,11 +432,16 @@ struct rte_flow * > { > struct rte_eth_dev *dev =3D &rte_eth_devices[port_id]; > const struct rte_flow_ops *ops =3D rte_flow_ops_get(port_id, error); > + int ret; >=20 > if (unlikely(!ops)) > return -rte_errno; > - if (likely(!!ops->flush)) > - return flow_err(port_id, ops->flush(dev, error), error); > + if (likely(!!ops->flush)) { > + fts_enter(dev); > + ret =3D ops->flush(dev, error); > + fts_exit(dev); > + return flow_err(port_id, ret, error); > + } > return rte_flow_error_set(error, ENOSYS, > RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > NULL, rte_strerror(ENOSYS)); > @@ -428,12 +457,16 @@ struct rte_flow * > { > struct rte_eth_dev *dev =3D &rte_eth_devices[port_id]; > const struct rte_flow_ops *ops =3D rte_flow_ops_get(port_id, error); > + int ret; >=20 > if (!ops) > return -rte_errno; > - if (likely(!!ops->query)) > - return flow_err(port_id, ops->query(dev, flow, action, data, > - error), error); > + if (likely(!!ops->query)) { > + fts_enter(dev); > + ret =3D ops->query(dev, flow, action, data, error); > + fts_exit(dev); > + return flow_err(port_id, ret, error); > + } > return rte_flow_error_set(error, ENOSYS, > RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > NULL, rte_strerror(ENOSYS)); > @@ -447,11 +480,16 @@ struct rte_flow * > { > struct rte_eth_dev *dev =3D &rte_eth_devices[port_id]; > const struct rte_flow_ops *ops =3D rte_flow_ops_get(port_id, error); > + int ret; >=20 > if (!ops) > return -rte_errno; > - if (likely(!!ops->isolate)) > - return flow_err(port_id, ops->isolate(dev, set, error), error); > + if (likely(!!ops->isolate)) { > + fts_enter(dev); > + ret =3D ops->isolate(dev, set, error); > + fts_exit(dev); > + return flow_err(port_id, ret, error); > + } > return rte_flow_error_set(error, ENOSYS, > RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > NULL, rte_strerror(ENOSYS)); > @@ -1224,12 +1262,16 @@ enum rte_flow_conv_item_spec_type { { > struct rte_eth_dev *dev =3D &rte_eth_devices[port_id]; > const struct rte_flow_ops *ops =3D rte_flow_ops_get(port_id, error); > + int ret; >=20 > if (unlikely(!ops)) > return -rte_errno; > - if (likely(!!ops->dev_dump)) > - return flow_err(port_id, ops->dev_dump(dev, file, error), > - error); > + if (likely(!!ops->dev_dump)) { > + fts_enter(dev); > + ret =3D ops->dev_dump(dev, file, error); > + fts_exit(dev); > + return flow_err(port_id, ret, error); > + } > return rte_flow_error_set(error, ENOSYS, > RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > NULL, rte_strerror(ENOSYS)); > @@ -1241,12 +1283,16 @@ enum rte_flow_conv_item_spec_type { { > struct rte_eth_dev *dev =3D &rte_eth_devices[port_id]; > const struct rte_flow_ops *ops =3D rte_flow_ops_get(port_id, error); > + int ret; >=20 > if (unlikely(!ops)) > return -rte_errno; > - if (likely(!!ops->get_aged_flows)) > - return flow_err(port_id, ops->get_aged_flows(dev, > contexts, > - nb_contexts, error), error); > + if (likely(!!ops->get_aged_flows)) { > + fts_enter(dev); > + ret =3D ops->get_aged_flows(dev, contexts, nb_contexts, > error); > + fts_exit(dev); > + return flow_err(port_id, ret, error); > + } > return rte_flow_error_set(error, ENOTSUP, > RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > NULL, rte_strerror(ENOTSUP)); > -- > 1.8.3.1 Besides the above (not must) suggestion, looks good, Acked-by: Matan Azrad