From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id B1F41A058F; Wed, 25 Mar 2020 15:25:25 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 89A272C15; Wed, 25 Mar 2020 15:25:25 +0100 (CET) Received: from EUR04-HE1-obe.outbound.protection.outlook.com (mail-eopbgr70041.outbound.protection.outlook.com [40.107.7.41]) by dpdk.org (Postfix) with ESMTP id 1A4322BCE for ; Wed, 25 Mar 2020 15:25:24 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=QKCKP6eQjjcG0bKw0H1aHX9Z98cnU11Q1qLVyHHbEV9qWu4nfOflBwDlT6M6+It6Uk2diaAJa1hE298Zdq+exhE+mvEjG9Kz5Aefggl5Bg0MYCyeTQXQqb76252xeabUfRqjagVLFxZUH7blpgsBBrAoWPSCiZrmgtUMgl87nU6bv2yPF96pFSMYwreZaSkaAE6eIFgZojJkDqaO4UHG3O0bPlVSgFeLeHzV8gKMR2Vg16gxJbb9GFd0S5JjGGS27m0PSO2v6g0unaaIqtnhbDLtp3dU36nY/bj7MQgVc+NmvRoLQsJ3OoFblnBztdyFVJM2MTgsB4MBVyuNLXU4mw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=oc/vGYcPjJsJ/och1nEpINugE3O+pmjcZMnFrWEQh1g=; b=Rd1qrwi1zz2J+/bKV3PiOnRXg4M7dIjIgCiiLl8pd16ATni4K32q1+bvqHnuPv3ED5N9NG8bzCJWq3se4vvDuVoH/OZMB0zXd2wSerqMBiarbJoVHgh3zej7BTR0zHAnE3erjFew+Ux5sUovo8YfkQbozfTKX9ySoT7L+GLHSD/rmjSnRXXTMSrXfjFsGc1fAlxwcJrFJNQ9XWZ/Ri7RfILFrcaPR3poZhLcpAmmbwYeZtRPQ/3GQJp2BS/Lk7K7lRlD6c7MB22hLO7LK+91RP2ltQ5/732bvdnHBpAXKAThRh2QT6mudPZGgUByGl7libesaE+D+ozb47xF/ONArg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=mellanox.com; dmarc=pass action=none header.from=mellanox.com; dkim=pass header.d=mellanox.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=oc/vGYcPjJsJ/och1nEpINugE3O+pmjcZMnFrWEQh1g=; b=fUVYxoUtUJ06yns3V+S55p2ViUuKmdZSg7bfjetCC3Xs7YKJKJ7AouyobZkBPWHdx0xhC38jkXPZ/iWnj8YSfmTlP/WGYVU0h19ncnKF+hHTJlF/N0+XD608ZbyvIj+XuywmiZm0tXbRpjEyFiD4+XL+ExBFKP1cP72rr5Xzh1w= Received: from AM0PR05MB6707.eurprd05.prod.outlook.com (10.186.174.145) by AM0PR05MB6420.eurprd05.prod.outlook.com (20.179.34.18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2856.18; Wed, 25 Mar 2020 14:25:23 +0000 Received: from AM0PR05MB6707.eurprd05.prod.outlook.com ([fe80::adbd:45ca:54:bba9]) by AM0PR05MB6707.eurprd05.prod.outlook.com ([fe80::adbd:45ca:54:bba9%7]) with mapi id 15.20.2835.023; Wed, 25 Mar 2020 14:25:23 +0000 From: Raslan Darawsheh To: Shiri Kuzin , "dev@dpdk.org" CC: Matan Azrad , Slava Ovsiienko Thread-Topic: [PATCH] net/mlx5: creating relaxed ordering memory regions Thread-Index: AQHWAdDyPO3wtc/yNkqGbQtXOIhcqahZXqag Date: Wed, 25 Mar 2020 14:25:23 +0000 Message-ID: References: <1585049979-12990-1-git-send-email-shirik@mellanox.com> In-Reply-To: <1585049979-12990-1-git-send-email-shirik@mellanox.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=rasland@mellanox.com; x-originating-ip: [188.161.229.130] x-ms-publictraffictype: Email x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: a4429062-91db-41a9-e602-08d7d0c85ad8 x-ms-traffictypediagnostic: AM0PR05MB6420:|AM0PR05MB6420: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:185; x-forefront-prvs: 0353563E2B x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(4636009)(366004)(39860400002)(136003)(346002)(396003)(376002)(186003)(107886003)(5660300002)(8676002)(52536014)(9686003)(86362001)(2906002)(55016002)(81166006)(81156014)(71200400001)(26005)(66446008)(76116006)(8936002)(66946007)(7696005)(4326008)(33656002)(110136005)(54906003)(66476007)(66556008)(64756008)(316002)(53546011)(6506007)(478600001); DIR:OUT; SFP:1101; SCL:1; SRVR:AM0PR05MB6420; H:AM0PR05MB6707.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 1VHZiXVI9m/JmusojUZ5YFT5xj4/VuEyHFJHr45vNoG318QTx+Dq6FRQRJV1bFZ2Gb2VF3hZhfpkTp2YVINWQdFxC7M5rxxY/36fFgBpEV4RKD+3Hivzxjn8gPl0BK1BzkIyKkyH3OzSkxMIzYpxfY1NHfNozAjWCc+TWfWt8fvYgtG3nOoToTNsknCQkgEDJ03zpW9PyASB7Gts2QddCEtRyc2gMUQPvOeZBflMCOiRdryL1WtOO8G1k6ANmkFx/+JESDu5nZr250ultmkJyxM1UMbCnjCKu/l9/EHyI0e9NW5fAGzs+UJQph733I53xmqp/J9o59ZOs/KmwhgE919mvrPhcpkRASPm36mEPEoeL6gZEFbplUz42o7hk5xKNUt7g8fcNsVdiHJay6I5WvtlPUmjxX0PCqgTHf2oPC/t52flZEwYaszdPHvqlIZJ x-ms-exchange-antispam-messagedata: B6csc8pU4PcpAURZuDbVKNbPm1FSRmu21YDTaL20Kh4Qhoo1T3gaC3x5L7L6l3w1e+nTn6z6Ry8cgMRstSYup0+VvyPas364lNJOqhuA5p/rLq5n+0UtC9+5ZdpTQvLQN/hnYHwly8qPA191RANRwg== Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: a4429062-91db-41a9-e602-08d7d0c85ad8 X-MS-Exchange-CrossTenant-originalarrivaltime: 25 Mar 2020 14:25:23.1728 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: NUuoCnXV5sAln1NJyljsC96aMWCwT3DZTBbgSYM2qEkLn4SmCmoga+1eHDpH6t+4RC5tuxqafSPWKCyxkhVofQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR05MB6420 Subject: Re: [dpdk-dev] [PATCH] net/mlx5: creating relaxed ordering memory regions X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi, > -----Original Message----- > From: Shiri Kuzin > Sent: Tuesday, March 24, 2020 1:40 PM > To: dev@dpdk.org > Cc: Matan Azrad ; Raslan Darawsheh > ; Slava Ovsiienko > Subject: [PATCH] net/mlx5: creating relaxed ordering memory regions >=20 > In the current state, when preforming read/write > transactions we must wait for a completion in order > to run the next transaction, and all transactions are > performed by order. >=20 > Relaxed Ordering is a PCI optimization which by enabling it > we allow the system to perform read/writes in a different > order without having to wait for completion and improve > the performance in that matter. >=20 > This commit introduces the creation of relaxed ordering > memory regions in mlx5. > As relaxed ordering is an optimization, drivers that > do not support it can simply ignore it and therefore > it is enabled by default. >=20 > Signed-off-by: Shiri Kuzin > Acked-by: Matan Azrad > --- > doc/guides/rel_notes/release_20_05.rst | 2 +- > drivers/common/mlx5/mlx5_devx_cmds.c | 4 ++++ > drivers/common/mlx5/mlx5_devx_cmds.h | 1 + > drivers/common/mlx5/mlx5_glue.h | 4 ++++ > drivers/common/mlx5/mlx5_prm.h | 4 +++- > drivers/net/mlx5/mlx5_flow_dv.c | 1 + > drivers/net/mlx5/mlx5_mr.c | 6 ++++-- > drivers/vdpa/mlx5/mlx5_vdpa_lm.c | 1 + > drivers/vdpa/mlx5/mlx5_vdpa_mem.c | 1 + > 9 files changed, 20 insertions(+), 4 deletions(-) >=20 > diff --git a/doc/guides/rel_notes/release_20_05.rst > b/doc/guides/rel_notes/release_20_05.rst > index 000bbf5..c960fd2 100644 > --- a/doc/guides/rel_notes/release_20_05.rst > +++ b/doc/guides/rel_notes/release_20_05.rst > @@ -61,7 +61,7 @@ New Features > Updated Mellanox mlx5 driver with new features and improvements, > including: >=20 > * Added support for matching on IPv4 Time To Live and IPv6 Hop Limit. > - > + * Added support for creating Relaxed Ordering Memory Regions. >=20 > Removed Items > ------------- > diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c > b/drivers/common/mlx5/mlx5_devx_cmds.c > index d960bc9..1157a44 100644 > --- a/drivers/common/mlx5/mlx5_devx_cmds.c > +++ b/drivers/common/mlx5/mlx5_devx_cmds.c > @@ -196,6 +196,10 @@ struct mlx5_devx_obj * > MLX5_SET(mkc, mkc, pd, attr->pd); > MLX5_SET(mkc, mkc, mkey_7_0, attr->umem_id & 0xFF); > MLX5_SET(mkc, mkc, translations_octword_size, translation_size); > + if (attr->relaxed_ordering =3D=3D 1) { > + MLX5_SET(mkc, mkc, relaxed_ordering_write, 0x1); > + MLX5_SET(mkc, mkc, relaxed_ordering_read, 0x1); > + } > MLX5_SET64(mkc, mkc, start_addr, attr->addr); > MLX5_SET64(mkc, mkc, len, attr->size); > mkey->obj =3D mlx5_glue->devx_obj_create(ctx, in, in_size_dw * 4, > out, > diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h > b/drivers/common/mlx5/mlx5_devx_cmds.h > index 6912dc6..20bb294 100644 > --- a/drivers/common/mlx5/mlx5_devx_cmds.h > +++ b/drivers/common/mlx5/mlx5_devx_cmds.h > @@ -33,6 +33,7 @@ struct mlx5_devx_mkey_attr { > uint32_t pd; > uint32_t log_entity_size; > uint32_t pg_access:1; > + uint32_t relaxed_ordering:1; > struct mlx5_klm *klm_array; > int klm_num; > }; > diff --git a/drivers/common/mlx5/mlx5_glue.h > b/drivers/common/mlx5/mlx5_glue.h > index 6238b43..cd1136f 100644 > --- a/drivers/common/mlx5/mlx5_glue.h > +++ b/drivers/common/mlx5/mlx5_glue.h > @@ -98,6 +98,10 @@ > uint64_t comp_mask; }; > #endif >=20 > +#ifndef IBV_ACCESS_RELAXED_ORDERING > +#define IBV_ACCESS_RELAXED_ORDERING 0 > +#endif > + > /* LIB_GLUE_VERSION must be updated every time this structure is > modified. */ > struct mlx5_glue { > const char *version; > diff --git a/drivers/common/mlx5/mlx5_prm.h > b/drivers/common/mlx5/mlx5_prm.h > index eee3a4b..00fd7c1 100644 > --- a/drivers/common/mlx5/mlx5_prm.h > +++ b/drivers/common/mlx5/mlx5_prm.h > @@ -882,7 +882,9 @@ struct mlx5_ifc_mkc_bits { >=20 > u8 translations_octword_size[0x20]; >=20 > - u8 reserved_at_1c0[0x1b]; > + u8 reserved_at_1c0[0x19]; > + u8 relaxed_ordering_read[0x1]; > + u8 reserved_at_1da[0x1]; > u8 log_page_size[0x5]; >=20 > u8 reserved_at_1e0[0x20]; > diff --git a/drivers/net/mlx5/mlx5_flow_dv.c > b/drivers/net/mlx5/mlx5_flow_dv.c > index 2090631..809833b 100644 > --- a/drivers/net/mlx5/mlx5_flow_dv.c > +++ b/drivers/net/mlx5/mlx5_flow_dv.c > @@ -3962,6 +3962,7 @@ struct field_modify_info modify_tcp[] =3D { > mkey_attr.pg_access =3D 0; > mkey_attr.klm_array =3D NULL; > mkey_attr.klm_num =3D 0; > + mkey_attr.relaxed_ordering =3D 1; > mem_mng->dm =3D mlx5_devx_cmd_mkey_create(sh->ctx, > &mkey_attr); > if (!mem_mng->dm) { > mlx5_glue->devx_umem_dereg(mem_mng->umem); > diff --git a/drivers/net/mlx5/mlx5_mr.c b/drivers/net/mlx5/mlx5_mr.c > index cb97c87..fd1245d 100644 > --- a/drivers/net/mlx5/mlx5_mr.c > +++ b/drivers/net/mlx5/mlx5_mr.c > @@ -768,7 +768,8 @@ struct mr_update_mp_data { > * through mlx5_alloc_verbs_buf(). > */ > mr->ibv_mr =3D mlx5_glue->reg_mr(sh->pd, (void *)data.start, len, > - IBV_ACCESS_LOCAL_WRITE); > + IBV_ACCESS_LOCAL_WRITE | > + IBV_ACCESS_RELAXED_ORDERING); > if (mr->ibv_mr =3D=3D NULL) { > DEBUG("port %u fail to create a verbs MR for address (%p)", > dev->data->port_id, (void *)addr); > @@ -1217,7 +1218,8 @@ struct mr_update_mp_data { > if (mr =3D=3D NULL) > return NULL; > mr->ibv_mr =3D mlx5_glue->reg_mr(priv->sh->pd, (void *)addr, len, > - IBV_ACCESS_LOCAL_WRITE); > + IBV_ACCESS_LOCAL_WRITE | > + IBV_ACCESS_RELAXED_ORDERING); > if (mr->ibv_mr =3D=3D NULL) { > DRV_LOG(WARNING, > "port %u fail to create a verbs MR for address (%p)", > diff --git a/drivers/vdpa/mlx5/mlx5_vdpa_lm.c > b/drivers/vdpa/mlx5/mlx5_vdpa_lm.c > index 3358704..4457760 100644 > --- a/drivers/vdpa/mlx5/mlx5_vdpa_lm.c > +++ b/drivers/vdpa/mlx5/mlx5_vdpa_lm.c > @@ -39,6 +39,7 @@ > .pg_access =3D 1, > .klm_array =3D NULL, > .klm_num =3D 0, > + .relaxed_ordering =3D 0, > }; > struct mlx5_devx_virtq_attr attr =3D { > .type =3D > MLX5_VIRTQ_MODIFY_TYPE_DIRTY_BITMAP_PARAMS, > diff --git a/drivers/vdpa/mlx5/mlx5_vdpa_mem.c > b/drivers/vdpa/mlx5/mlx5_vdpa_mem.c > index 398ca35..da31b47 100644 > --- a/drivers/vdpa/mlx5/mlx5_vdpa_mem.c > +++ b/drivers/vdpa/mlx5/mlx5_vdpa_mem.c > @@ -263,6 +263,7 @@ > mkey_attr.pg_access =3D 1; > mkey_attr.klm_array =3D NULL; > mkey_attr.klm_num =3D 0; > + mkey_attr.relaxed_ordering =3D 0; > entry->mkey =3D mlx5_devx_cmd_mkey_create(priv->ctx, > &mkey_attr); > if (!entry->mkey) { > DRV_LOG(ERR, "Failed to create direct Mkey."); > -- > 1.8.3.1 Patch applied to next-net-mlx, Kindest regards, Raslan Darawsheh