From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 07B35A04B5; Tue, 27 Oct 2020 16:43:07 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id D51B372DF; Tue, 27 Oct 2020 16:43:05 +0100 (CET) Received: from hqnvemgate26.nvidia.com (hqnvemgate26.nvidia.com [216.228.121.65]) by dpdk.org (Postfix) with ESMTP id 123F16CA9 for ; Tue, 27 Oct 2020 16:43:02 +0100 (CET) Received: from hqmail.nvidia.com (Not Verified[216.228.121.13]) by hqnvemgate26.nvidia.com (using TLS: TLSv1.2, AES256-SHA) id ; Tue, 27 Oct 2020 08:42:41 -0700 Received: from HQMAIL107.nvidia.com (172.20.187.13) by HQMAIL105.nvidia.com (172.20.187.12) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Tue, 27 Oct 2020 15:43:01 +0000 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (104.47.55.174) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1473.3 via Frontend Transport; Tue, 27 Oct 2020 15:43:01 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=fxnPJaCiAZWJS9h0MpAZ5b5r2WRmuYn1W/h74BOwqCHTj95p0Ta/VpfEq76EJWyyjEsFzi4RhpR8CKzxiDY0iNQLR9OUsUwmbB8X/NxxsjSCaRHw23tX9rhcR3ya/+vbCEPpC3iMdYaGyoMueExzSz0/OusOCbEQGvaY2tLXcjUtN9tOe3TBhiXaNsDkDu/n4lalPxWZEMQeuaOzYaj4YMzP7eUdMrivQBpuv/kqE4iiZEhBSfEoyJKNCrXxX1YNO8FVnaCO0Yxd7TGJ19d6bdoyoTdq11oUyqF+ueXkd7tRRTlhvNXJGE+II6fYNK4iAL2D590GBD4kNaJkr1Ou7w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=AXddqx5mXTixB0qFbXE3p2FGbRLngW2fVwP7ma0cvn4=; b=nhZNCgmQnC2MCVLiQjfaNyD1GGHqTiH4VomdSzsj7ybWmfZcv1/uYz5wOmm/ifluP56Y7RHnYCB6Gd+QQyT9sr0OKOyr1AytbaXaJ6HdVaw85JWQAX09koH9AKbRZXaRpcG32Ud7PDg+zaRSx50GGMy3a8CRYq8OtZel8W0N2PNRc17iXnC+Gd3aFtKgAuCx2lw3fn0cJrJ8bxmm5MW/ZYAMWenRu1hdx2fLELmyf6U1OW8hHb0NMWPBYZSPr953KV1lGwD7pSSW+ZTE8mKkCkM/xGemjvahlV2TTB5zGbB9aMw9AUwNuSps+ekcoiksPAEC58HcgcNFMR3JnMwCKA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none Received: from MWHPR12MB1501.namprd12.prod.outlook.com (2603:10b6:301:f::18) by MW2PR12MB2410.namprd12.prod.outlook.com (2603:10b6:907:f::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3477.25; Tue, 27 Oct 2020 15:43:00 +0000 Received: from MWHPR12MB1501.namprd12.prod.outlook.com ([fe80::3ce6:9a73:3c99:2f35]) by MWHPR12MB1501.namprd12.prod.outlook.com ([fe80::3ce6:9a73:3c99:2f35%12]) with mapi id 15.20.3477.028; Tue, 27 Oct 2020 15:43:00 +0000 From: Slava Ovsiienko To: Bing Zhao , Matan Azrad , Ori Kam CC: "dev@dpdk.org" , Raslan Darawsheh Thread-Topic: [PATCH v2 2/2] net/mlx5: add flow sync API Thread-Index: AQHWrHAmjuEpaCpjMECOLLR+sMSXHqmrlqWQ Date: Tue, 27 Oct 2020 15:42:59 +0000 Message-ID: References: <1602255678-108560-1-git-send-email-bingz@nvidia.com> <1603810014-349985-1-git-send-email-bingz@nvidia.com> <1603810014-349985-2-git-send-email-bingz@nvidia.com> In-Reply-To: <1603810014-349985-2-git-send-email-bingz@nvidia.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: nvidia.com; dkim=none (message not signed) header.d=none;nvidia.com; dmarc=none action=none header.from=nvidia.com; x-originating-ip: [95.164.10.10] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 57f66c5a-c866-46c9-7199-08d87a8efbbc x-ms-traffictypediagnostic: MW2PR12MB2410: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:7691; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: B9EARx1g5EAnjXU/WLJIVYoZGm8tK6ZPqLPDACTGXvPlPsM+rkHdutOhKJ7eEKwtw7sascCKzML1xoAOFJkpAIZr5rFM8EqKq6hIb2CE+VqhK1I+Xmuuq90IpwMAdhcF8D8D+W46NxwjBFkt4eYs+n+SHgCB9KiMHS9Am3BZi+zSEVGiwAo53zNGWZDe3n0YKDqkn9JzM8Lw9kiE+Qe4uhR+SdUtCQNJn3y6xNfYGmdMRIcfKmY0i83ncxWNgVtdKt37Y1k1DAwhxIJWqmg3gqG1Zxtc1BZIyrgWM/uICYzQuzU8uZ+zH7q0ZmaE8FSHDamgqJ2ho2h6zlIoa8xIHA== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MWHPR12MB1501.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(39860400002)(376002)(136003)(366004)(396003)(346002)(4326008)(53546011)(107886003)(2906002)(76116006)(5660300002)(64756008)(33656002)(86362001)(83380400001)(66946007)(55016002)(316002)(54906003)(6636002)(9686003)(71200400001)(52536014)(66446008)(186003)(66476007)(8676002)(478600001)(6506007)(7696005)(8936002)(110136005)(66556008)(26005); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: UuxuAFm8NZSh+n8fc7/rMU8r85VRO7hpbrijoocWtnXdBt5wX1vqIc4UQQrAzJRaScoar9bl+YAPU55wBeHXQPZObX9md4DTOeDtFZ2PsPwhwgLH0iD7zGQuZf718vxfAarOIaVPcJraS9Xm+Rn/TnTE25jlYsw9Lz4Vk03o6zrcatFjsB0dz2M6L/dGwt9Gv07m7JwzOUhcZIl0m/qAX11znmOo8gBMiYNvthY+Y5mOFXmNSekh/pRxq/sDvbSgxgga4EURfVIKlQcTyCDJggwAh2IEpm1gTldJWe3h1JWSh7n42MRSRXB/IGxaLkUS2lxcHJWviJ/eI3ZogsbQK+lXv6lV42U1w6hlWzC3pFizFnXugXBOuF7R2VUYH9YhFHLA3he0yNGnEbu39B3BwX+zNilwnci0gT/vUAHa6nV89oCEZ52IV1KteRigwBnwp4PZemIxylupn/COdGXTBP017jdU+mGVwOtOiscKK5fO34HZYbk3LzpHqJVjwCBhQaSyE0fMam0m4cnFsb6hBE1Con1+e+IDMmPNZ2/lvyNfgnJ1C5GeVGye6fxBmWTf4PfUQQSxxA0x2uV8JVhV2o8cLL3RhgLMS7MtNIu5wPd04YQ8yJE6hfHmYrXEsywauY01G8Fu3Gnzi95Y+l+DFg== Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MWHPR12MB1501.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 57f66c5a-c866-46c9-7199-08d87a8efbbc X-MS-Exchange-CrossTenant-originalarrivaltime: 27 Oct 2020 15:42:59.8341 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: c/Wi3I0ALfYQ5L331RutqcZCmR4NON4oABPhJdPI0sg3BxP7xJhYeEgTwg/g1qZlbDSB27NiMLLh8qLyzvr98Q== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW2PR12MB2410 X-OriginatorOrg: Nvidia.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nvidia.com; s=n1; t=1603813361; bh=AXddqx5mXTixB0qFbXE3p2FGbRLngW2fVwP7ma0cvn4=; h=ARC-Seal:ARC-Message-Signature:ARC-Authentication-Results:From:To: CC:Subject:Thread-Topic:Thread-Index:Date:Message-ID:References: In-Reply-To:Accept-Language:Content-Language:X-MS-Has-Attach: X-MS-TNEF-Correlator:authentication-results:x-originating-ip: x-ms-publictraffictype:x-ms-office365-filtering-correlation-id: x-ms-traffictypediagnostic:x-ms-exchange-transport-forked: x-microsoft-antispam-prvs:x-ms-oob-tlc-oobclassifiers: x-ms-exchange-senderadcheck:x-microsoft-antispam: x-microsoft-antispam-message-info:x-forefront-antispam-report: x-ms-exchange-antispam-messagedata:Content-Type: Content-Transfer-Encoding:MIME-Version: X-MS-Exchange-CrossTenant-AuthAs: X-MS-Exchange-CrossTenant-AuthSource: X-MS-Exchange-CrossTenant-Network-Message-Id: X-MS-Exchange-CrossTenant-originalarrivaltime: X-MS-Exchange-CrossTenant-fromentityheader: X-MS-Exchange-CrossTenant-id:X-MS-Exchange-CrossTenant-mailboxtype: X-MS-Exchange-CrossTenant-userprincipalname: X-MS-Exchange-Transport-CrossTenantHeadersStamped:X-OriginatorOrg; b=SGdrjCSeBnTfaP5er8QmgJhVcd4bS0ozITJuYI7K8n46VsRPahqSJmL3zuj0VNlii ofwIj8NHEQ+8ICrqPcoY9ulH09MWLjbQOM7XrcsGKeNfbX6uKgRLDc3eeHRh71nWRS DNRWdFq5RkH1UziCZsuZsts6tKoSYUt0pVi1ZDvhJnLFeFla7/midZmML2WtCe828h NUfUk1S3C0kWrEGIZwTLwEezMXLFw5owvrnfIvuM377h0J3oL3Z0RG7CQ6Wgui61y8 naz78RIPne/VqYI/Wj+wOx8UTC7TT5GO1AX+zKD7doq+6irmuwPJ5Mtw/vWzZ7V++6 hnHzVJxGSf51w== Subject: Re: [dpdk-dev] [PATCH v2 2/2] net/mlx5: add flow sync API X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi, Bing Release notes / mlx5 features documentation update? Beside this: Acked-by: Viacheslav Ovsiienko > -----Original Message----- > From: Bing Zhao > Sent: Tuesday, October 27, 2020 16:47 > To: Slava Ovsiienko ; Matan Azrad > ; Ori Kam > Cc: dev@dpdk.org; Raslan Darawsheh > Subject: [PATCH v2 2/2] net/mlx5: add flow sync API >=20 > When creating a flow, the rule itself might not take effort immediately o= nce > the function call returns with success. It would take some time to let th= e > steering synchronize with the hardware. >=20 > If the application wants the packet to be sent to hit the flow after it i= s created, > this flow sync API can be used to clear the steering HW cache to enforce = next > packet hits the latest rules. >=20 > For TX, usually the NIC TX domain and/or the FDB domain should be > synchronized depends in which domain the flow is created. >=20 > The application could also try to synchronize the NIC RX and/or the FDB > domain for the ingress packets. >=20 > Signed-off-by: Bing Zhao > Acked-by: Ori Kam > --- > drivers/net/mlx5/mlx5_flow.c | 24 ++++++++++++++++++++++++ > drivers/net/mlx5/mlx5_flow.h | 5 +++++ > drivers/net/mlx5/mlx5_flow_dv.c | 27 +++++++++++++++++++++++++++ > drivers/net/mlx5/mlx5_flow_verbs.c | 12 ++++++++++++ > drivers/net/mlx5/rte_pmd_mlx5.h | 25 +++++++++++++++++++++++++ > drivers/net/mlx5/version.map | 2 ++ > 6 files changed, 95 insertions(+) >=20 > diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c = index > 02e19e8..8a1d8da 100644 > --- a/drivers/net/mlx5/mlx5_flow.c > +++ b/drivers/net/mlx5/mlx5_flow.c > @@ -32,6 +32,7 @@ > #include "mlx5_flow_os.h" > #include "mlx5_rxtx.h" > #include "mlx5_common_os.h" > +#include "rte_pmd_mlx5.h" >=20 > static struct mlx5_flow_tunnel * > mlx5_find_tunnel_id(struct rte_eth_dev *dev, uint32_t id); @@ -3043,6 > +3044,14 @@ struct mlx5_flow_tunnel_info { > RTE_FLOW_ERROR_TYPE_UNSPECIFIED, > NULL, NULL); } >=20 > +static int > +flow_null_sync_domain(struct rte_eth_dev *dev __rte_unused, > + uint32_t domains __rte_unused, > + uint32_t flags __rte_unused) > +{ > + return 0; > +} > + > /* Void driver to protect from null pointer reference. */ const struct > mlx5_flow_driver_ops mlx5_flow_null_drv_ops =3D { > .validate =3D flow_null_validate, > @@ -3052,6 +3061,7 @@ struct mlx5_flow_tunnel_info { > .remove =3D flow_null_remove, > .destroy =3D flow_null_destroy, > .query =3D flow_null_query, > + .sync_domain =3D flow_null_sync_domain, > }; >=20 > /** > @@ -8169,3 +8179,17 @@ int mlx5_alloc_tunnel_hub(struct > mlx5_dev_ctx_shared *sh) > mlx5_free(thub); > return err; > } > + > +int rte_pmd_mlx5_sync_flow(uint16_t port_id, uint32_t domains) { > + struct rte_eth_dev *dev =3D &rte_eth_devices[port_id]; > + const struct mlx5_flow_driver_ops *fops; > + int ret; > + struct rte_flow_attr attr =3D { .transfer =3D 0 }; > + > + fops =3D flow_get_drv_ops(flow_get_drv_type(dev, &attr)); > + ret =3D fops->sync_domain(dev, domains, > MLX5DV_DR_DOMAIN_SYNC_FLAGS_HW); > + if (ret > 0) > + ret =3D -ret; > + return ret; > +} > diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h > index 8b5a93f..a22ae21 100644 > --- a/drivers/net/mlx5/mlx5_flow.h > +++ b/drivers/net/mlx5/mlx5_flow.h > @@ -1176,6 +1176,10 @@ typedef int (*mlx5_flow_action_update_t) > struct rte_flow_shared_action *action, > const void *action_conf, > struct rte_flow_error *error); > +typedef int (*mlx5_flow_sync_domain_t) > + (struct rte_eth_dev *dev, > + uint32_t domains, > + uint32_t flags); > struct mlx5_flow_driver_ops { > mlx5_flow_validate_t validate; > mlx5_flow_prepare_t prepare; > @@ -1196,6 +1200,7 @@ struct mlx5_flow_driver_ops { > mlx5_flow_action_create_t action_create; > mlx5_flow_action_destroy_t action_destroy; > mlx5_flow_action_update_t action_update; > + mlx5_flow_sync_domain_t sync_domain; > }; >=20 > /* mlx5_flow.c */ > diff --git a/drivers/net/mlx5/mlx5_flow_dv.c > b/drivers/net/mlx5/mlx5_flow_dv.c index dafe07f..945eae6 100644 > --- a/drivers/net/mlx5/mlx5_flow_dv.c > +++ b/drivers/net/mlx5/mlx5_flow_dv.c > @@ -33,6 +33,7 @@ > #include "mlx5_flow.h" > #include "mlx5_flow_os.h" > #include "mlx5_rxtx.h" > +#include "rte_pmd_mlx5.h" >=20 > #ifdef HAVE_IBV_FLOW_DV_SUPPORT >=20 > @@ -12310,6 +12311,31 @@ struct field_modify_info modify_tcp[] =3D { > return ret; > } >=20 > +static int > +flow_dv_sync_domain(struct rte_eth_dev *dev, uint32_t domains, uint32_t > +flags) { > + struct mlx5_priv *priv =3D dev->data->dev_private; > + int ret =3D 0; > + > + if ((domains & MLX5_DOMAIN_BIT_NIC_RX) && priv->sh->rx_domain > !=3D NULL) { > + ret =3D mlx5_glue->dr_sync_domain(priv->sh->rx_domain, > + flags); > + if (ret !=3D 0) > + return ret; > + } > + if ((domains & MLX5_DOMAIN_BIT_NIC_TX) && priv->sh->tx_domain > !=3D NULL) { > + ret =3D mlx5_glue->dr_sync_domain(priv->sh->tx_domain, flags); > + if (ret !=3D 0) > + return ret; > + } > + if ((domains & MLX5_DOMAIN_BIT_FDB) && priv->sh->fdb_domain !=3D > NULL) { > + ret =3D mlx5_glue->dr_sync_domain(priv->sh->fdb_domain, > flags); > + if (ret !=3D 0) > + return ret; > + } > + return 0; > +} > + > const struct mlx5_flow_driver_ops mlx5_flow_dv_drv_ops =3D { > .validate =3D flow_dv_validate, > .prepare =3D flow_dv_prepare, > @@ -12330,6 +12356,7 @@ struct field_modify_info modify_tcp[] =3D { > .action_create =3D flow_dv_action_create, > .action_destroy =3D flow_dv_action_destroy, > .action_update =3D flow_dv_action_update, > + .sync_domain =3D flow_dv_sync_domain, > }; >=20 > #endif /* HAVE_IBV_FLOW_DV_SUPPORT */ > diff --git a/drivers/net/mlx5/mlx5_flow_verbs.c > b/drivers/net/mlx5/mlx5_flow_verbs.c > index 6bcc009..d04c37f 100644 > --- a/drivers/net/mlx5/mlx5_flow_verbs.c > +++ b/drivers/net/mlx5/mlx5_flow_verbs.c > @@ -2078,6 +2078,17 @@ > return ret; > } >=20 > +static int > +flow_verbs_sync_domain(struct rte_eth_dev *dev, uint32_t domains, > + uint32_t flags) > +{ > + RTE_SET_USED(dev); > + RTE_SET_USED(domains); > + RTE_SET_USED(flags); > + > + return 0; > +} > + > const struct mlx5_flow_driver_ops mlx5_flow_verbs_drv_ops =3D { > .validate =3D flow_verbs_validate, > .prepare =3D flow_verbs_prepare, > @@ -2086,4 +2097,5 @@ > .remove =3D flow_verbs_remove, > .destroy =3D flow_verbs_destroy, > .query =3D flow_verbs_query, > + .sync_domain =3D flow_verbs_sync_domain, > }; > diff --git a/drivers/net/mlx5/rte_pmd_mlx5.h > b/drivers/net/mlx5/rte_pmd_mlx5.h index 8c69228..e531e52 100644 > --- a/drivers/net/mlx5/rte_pmd_mlx5.h > +++ b/drivers/net/mlx5/rte_pmd_mlx5.h > @@ -32,4 +32,29 @@ > __rte_experimental > int rte_pmd_mlx5_get_dyn_flag_names(char *names[], unsigned int n); >=20 > +#define MLX5_DOMAIN_BIT_NIC_RX (1 << 0) /**< NIC RX domain bit mask. > */ > +#define MLX5_DOMAIN_BIT_NIC_TX (1 << 1) /**< NIC TX domain bit mask. > */ > +#define MLX5_DOMAIN_BIT_FDB (1 << 2) /**< FDB (TX + RX) domain bit > mask. */ > + > +/** > + * Synchronize the flows to make them take effort on hardware. > + * It only supports DR flows now. For DV and Verbs flows, there is no > +need to > + * call this function, and a success will return directly in case of Ver= bs. > + * > + * @param[in] port_id > + * The port identifier of the Ethernet device. > + * @param[in] domains > + * Refer to "/usr/include/infiniband/mlx5dv.h". > + * Bitmask of domains in which the synchronization will be done. > + * MLX5_DOMAIN_BIT* macros are used to specify the domains. > + * An ADD or OR operation could be used to synchronize flows in more t= han > + * one domain per call. > + * > + * @return > + * - (0) if successful. > + * - Negative value if an error. > + */ > +__rte_experimental > +int rte_pmd_mlx5_sync_flow(uint16_t port_id, uint32_t domains); > + > #endif > diff --git a/drivers/net/mlx5/version.map b/drivers/net/mlx5/version.map > index bc1d3d0..82a32b5 100644 > --- a/drivers/net/mlx5/version.map > +++ b/drivers/net/mlx5/version.map > @@ -7,4 +7,6 @@ EXPERIMENTAL { >=20 > # added in 20.02 > rte_pmd_mlx5_get_dyn_flag_names; > + # added in 20.11 > + rte_pmd_mlx5_sync_flow; > }; > -- > 1.8.3.1