From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id C75FBA0613 for ; Thu, 26 Sep 2019 11:33:36 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 0B92F1BF5A; Thu, 26 Sep 2019 11:33:16 +0200 (CEST) Received: from EUR03-DB5-obe.outbound.protection.outlook.com (mail-eopbgr40076.outbound.protection.outlook.com [40.107.4.76]) by dpdk.org (Postfix) with ESMTP id 103761BF5A for ; Thu, 26 Sep 2019 11:33:15 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=UXN+I43+RBBDoXpmZp22SMPcZEZPtlXBkE1h0i657L9THHvA8ok0WrUTN9mtyhT7ab6A/Q9Jwu7v/g7vqxiOZ7yGMW9E97LEOqv6qQwaIOWDpCIK9+87P+fvZj+S9X59KKQ4ciL2+XqlftbYkjJJ6c0lkBlVzolhqZZAxibWOxiicmLgqXApbI2sAZKLS+KHJstEgf3YqXU7XOf4IXYy/vyIYJ7uX4ywZR6gdaNNLNUOb46V+OwNn6hClujMlpBgj1cjeh3QMGOaDfdFj82PJXy5WCn8UcQAzAA6ux9pEUgkZzWyUYqNjaVCrJoPCoCTRdff+14Pf0I/Wvb8LsuopA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=iei2J623cm9Cs0sfRlPM2a1NVXhUkK8yKaogL+auHt0=; b=nQgan+XavdEskbVvMcw50tbT5Yjl4SBWleiAlHwq01roOvyx4/afBQYmPZTuZnvl6tgPjHNfBFwLi7hpAs45IL7jjAzx5MDOIZXTWE2ZUtAvbWGhYvm1bXybX3wykSUgNs/kHg3FUdWEw5Z18v2mLpR/kSqawgq5e3scPx0ElaK223+v5UoemQk9oVp0lNnJPIQPXOW+HG9gVaRTOeIRlW99o0vXsWgNZLwa7ip3R71nZDp1jVWIMs4VkhHPmm3eUG9fUA16ijqq1/76gjeGxDq4pIQdk/ek0oxVZhqNFKPSz+kYGecqtMlcnQWaId7K0huLuQORPb7guxBM7TO6OQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=mellanox.com; dmarc=pass action=none header.from=mellanox.com; dkim=pass header.d=mellanox.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=iei2J623cm9Cs0sfRlPM2a1NVXhUkK8yKaogL+auHt0=; b=bGcSMDjU08ZOSB3f3Ql8irLjIOeVTeSmMjA5pvzojKENlQj8KtqShEryusxUyyLHPaAIBXCwMW2eK4G2KckTCn/55VdjN1sNyqEtmGixGeBKGiuSOh7/lhNPtCKdLABpSEy9FdLLYjp2tX96UYUVbuvORSQLvZf3oLD2rRpg8ww= Received: from AM4PR05MB3265.eurprd05.prod.outlook.com (10.171.188.154) by AM4PR05MB3281.eurprd05.prod.outlook.com (10.171.188.158) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2305.15; Thu, 26 Sep 2019 09:33:14 +0000 Received: from AM4PR05MB3265.eurprd05.prod.outlook.com ([fe80::da9:65ba:1323:a39b]) by AM4PR05MB3265.eurprd05.prod.outlook.com ([fe80::da9:65ba:1323:a39b%7]) with mapi id 15.20.2305.017; Thu, 26 Sep 2019 09:33:14 +0000 From: Slava Ovsiienko To: Ori Kam , Matan Azrad , Shahaf Shuler CC: "dev@dpdk.org" , Ori Kam , "jingjing.wu@intel.com" , "stephen@networkplumber.org" Thread-Topic: [PATCH 07/13] net/mlx5: add hairpin binding function Thread-Index: AQHVdDQAEDf+Wu33/02wdRYq1ZpzVac9smaA Date: Thu, 26 Sep 2019 09:33:14 +0000 Message-ID: References: <1569479349-36962-1-git-send-email-orika@mellanox.com> <1569479349-36962-8-git-send-email-orika@mellanox.com> In-Reply-To: <1569479349-36962-8-git-send-email-orika@mellanox.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=viacheslavo@mellanox.com; x-originating-ip: [95.67.35.250] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: d20d2997-d99f-4035-5d8b-08d742648ded x-ms-office365-filtering-ht: Tenant x-ms-traffictypediagnostic: AM4PR05MB3281:|AM4PR05MB3281: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:2803; x-forefront-prvs: 0172F0EF77 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(4636009)(39860400002)(136003)(396003)(346002)(376002)(366004)(13464003)(199004)(189003)(71190400001)(71200400001)(33656002)(316002)(6246003)(7696005)(6436002)(76176011)(6506007)(53546011)(99286004)(110136005)(8936002)(8676002)(3846002)(9686003)(5024004)(476003)(6116002)(256004)(486006)(102836004)(229853002)(2906002)(54906003)(26005)(81156014)(81166006)(11346002)(66446008)(66946007)(66476007)(66556008)(64756008)(76116006)(6636002)(55016002)(186003)(74316002)(66066001)(5660300002)(25786009)(52536014)(478600001)(446003)(305945005)(7736002)(4326008)(14454004)(86362001); DIR:OUT; SFP:1101; SCL:1; SRVR:AM4PR05MB3281; H:AM4PR05MB3265.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: vM17y0EFpTV83A3PP2FRXXXKtbNQaLqNBr2ydyR85WOrKvaeJBlGSDYVW7gFfgQ15kDpwbTEWcDRrNZTjoAKuLV+m8/0SkXETPDTsDvdkFCRuPIS8ArPqm+lffUgm26ezh/C4whY/xTScl6FUaoFFiVbIQuQAnN7Jp21lj2hmeKafyeVf+yIdyeT8V0/VpjxLJ/d8yGFyfmTLzJehCnUnL8XCo56DL1U39+pKKKqeHlo0A2lIdvJfA6imGF3SUEZsNsFUH5lO/ss6RE1An9XqUhk2CUFInP95KYg/xUtzEgoNgU376wr0aZQrGlh/GYUtonb2KIGWlAz3X8//oobpNTHX64vnwNWsMalgCvXCOAO/px1fKzwQ8/tcvg4Po9+ZNVF5rapoZpF61UEXLwbREvr8GE+ooYRknoLrrgecCY= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: d20d2997-d99f-4035-5d8b-08d742648ded X-MS-Exchange-CrossTenant-originalarrivaltime: 26 Sep 2019 09:33:14.0661 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: GeW4bihGsQ/5O9muA59ojs3MaTHT7yl8EbwDctkxeTHQ81VmyeUNEIvlHixRBBj2Qpm4s3j0H5I37kGa93981jFj+9sxgxiU42tStGn80Cw= X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM4PR05MB3281 Subject: Re: [dpdk-dev] [PATCH 07/13] net/mlx5: add hairpin binding function X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > -----Original Message----- > From: Ori Kam > Sent: Thursday, September 26, 2019 9:29 > To: Matan Azrad ; Shahaf Shuler > ; Slava Ovsiienko > Cc: dev@dpdk.org; Ori Kam ; jingjing.wu@intel.com; > stephen@networkplumber.org > Subject: [PATCH 07/13] net/mlx5: add hairpin binding function >=20 > When starting the port, in addition to creating the queues we need to bin= d > the hairpin queues. >=20 > Signed-off-by: Ori Kam Acked-by: Viacheslav Ovsiienko > --- > drivers/net/mlx5/mlx5.h | 1 + > drivers/net/mlx5/mlx5_devx_cmds.c | 1 + > drivers/net/mlx5/mlx5_prm.h | 6 +++ > drivers/net/mlx5/mlx5_trigger.c | 97 > +++++++++++++++++++++++++++++++++++++++ > 4 files changed, 105 insertions(+) >=20 > diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index > 506920e..41eb35a 100644 > --- a/drivers/net/mlx5/mlx5.h > +++ b/drivers/net/mlx5/mlx5.h > @@ -188,6 +188,7 @@ struct mlx5_hca_attr { > uint32_t log_max_hairpin_queues:5; > uint32_t log_max_hairpin_wq_data_sz:5; > uint32_t log_max_hairpin_num_packets:5; > + uint32_t vhca_id:16; > }; >=20 > /* Flow list . */ > diff --git a/drivers/net/mlx5/mlx5_devx_cmds.c > b/drivers/net/mlx5/mlx5_devx_cmds.c > index 917bbf9..0243733 100644 > --- a/drivers/net/mlx5/mlx5_devx_cmds.c > +++ b/drivers/net/mlx5/mlx5_devx_cmds.c > @@ -334,6 +334,7 @@ struct mlx5_devx_obj * >=20 > log_max_hairpin_wq_data_sz); > attr->log_max_hairpin_num_packets =3D MLX5_GET > (cmd_hca_cap, hcattr, log_min_hairpin_wq_data_sz); > + attr->vhca_id =3D MLX5_GET(cmd_hca_cap, hcattr, vhca_id); > attr->eth_net_offloads =3D MLX5_GET(cmd_hca_cap, hcattr, > eth_net_offloads); > attr->eth_virt =3D MLX5_GET(cmd_hca_cap, hcattr, eth_virt); diff --git > a/drivers/net/mlx5/mlx5_prm.h b/drivers/net/mlx5/mlx5_prm.h index > faa7996..d4084db 100644 > --- a/drivers/net/mlx5/mlx5_prm.h > +++ b/drivers/net/mlx5/mlx5_prm.h > @@ -1611,6 +1611,12 @@ struct mlx5_ifc_create_rqt_in_bits { #pragma > GCC diagnostic error "-Wpedantic" > #endif >=20 > +enum { > + MLX5_SQC_STATE_RST =3D 0x0, > + MLX5_SQC_STATE_RDY =3D 0x1, > + MLX5_SQC_STATE_ERR =3D 0x3, > +}; > + > struct mlx5_ifc_sqc_bits { > u8 rlky[0x1]; > u8 cd_master[0x1]; > diff --git a/drivers/net/mlx5/mlx5_trigger.c > b/drivers/net/mlx5/mlx5_trigger.c index 3ec86c4..a4fcdb3 100644 > --- a/drivers/net/mlx5/mlx5_trigger.c > +++ b/drivers/net/mlx5/mlx5_trigger.c > @@ -162,6 +162,96 @@ > } >=20 > /** > + * Binds Tx queues to Rx queues for hairpin. > + * > + * Binds Tx queues to the target Rx queues. > + * > + * @param dev > + * Pointer to Ethernet device structure. > + * > + * @return > + * 0 on success, a negative errno value otherwise and rte_errno is set= . > + */ > +static int > +mlx5_hairpin_bind(struct rte_eth_dev *dev) { > + struct mlx5_priv *priv =3D dev->data->dev_private; > + struct mlx5_devx_modify_sq_attr sq_attr =3D { 0 }; > + struct mlx5_devx_modify_rq_attr rq_attr =3D { 0 }; > + struct mlx5_txq_ctrl *txq_ctrl; > + struct mlx5_rxq_ctrl *rxq_ctrl; > + struct mlx5_devx_obj *sq; > + struct mlx5_devx_obj *rq; > + unsigned int i; > + int ret =3D 0; > + > + for (i =3D 0; i !=3D priv->txqs_n; ++i) { > + txq_ctrl =3D mlx5_txq_get(dev, i); > + if (!txq_ctrl) > + continue; > + if (txq_ctrl->type !=3D MLX5_TXQ_TYPE_HAIRPIN) { > + mlx5_txq_release(dev, i); > + continue; > + } > + if (!txq_ctrl->obj) { > + rte_errno =3D ENOMEM; > + DRV_LOG(ERR, "port %u no txq object found: %d", > + dev->data->port_id, i); > + mlx5_txq_release(dev, i); > + return -rte_errno; > + } > + sq =3D txq_ctrl->obj->sq; > + rxq_ctrl =3D mlx5_rxq_get(dev, > + txq_ctrl- > >hairpin_conf.peers[0].queue); > + if (!rxq_ctrl) { > + mlx5_txq_release(dev, i); > + rte_errno =3D EINVAL; > + DRV_LOG(ERR, "port %u no rxq object found: %d", > + dev->data->port_id, > + txq_ctrl->hairpin_conf.peers[0].queue); > + return -rte_errno; > + } > + if (rxq_ctrl->type !=3D MLX5_RXQ_TYPE_HAIRPIN || > + rxq_ctrl->hairpin_conf.peers[0].queue !=3D i) { > + rte_errno =3D ENOMEM; > + DRV_LOG(ERR, "port %u Tx queue %d can't be binded > to " > + "Rx queue %d", dev->data->port_id, > + i, txq_ctrl->hairpin_conf.peers[0].queue); > + goto error; > + } > + rq =3D rxq_ctrl->obj->rq; > + if (!rq) { > + rte_errno =3D ENOMEM; > + DRV_LOG(ERR, "port %u hairpin no matching rxq: > %d", > + dev->data->port_id, > + txq_ctrl->hairpin_conf.peers[0].queue); > + goto error; > + } > + sq_attr.state =3D MLX5_SQC_STATE_RDY; > + sq_attr.sq_state =3D MLX5_SQC_STATE_RST; > + sq_attr.hairpin_peer_rq =3D rq->id; > + sq_attr.hairpin_peer_vhca =3D priv->config.hca_attr.vhca_id; > + ret =3D mlx5_devx_cmd_modify_sq(sq, &sq_attr); > + if (ret) > + goto error; > + rq_attr.state =3D MLX5_SQC_STATE_RDY; > + rq_attr.rq_state =3D MLX5_SQC_STATE_RST; > + rq_attr.hairpin_peer_sq =3D sq->id; > + rq_attr.hairpin_peer_vhca =3D priv->config.hca_attr.vhca_id; > + ret =3D mlx5_devx_cmd_modify_rq(rq, &rq_attr); > + if (ret) > + goto error; > + mlx5_txq_release(dev, i); > + mlx5_rxq_release(dev, txq_ctrl- > >hairpin_conf.peers[0].queue); > + } > + return 0; > +error: > + mlx5_txq_release(dev, i); > + mlx5_rxq_release(dev, txq_ctrl->hairpin_conf.peers[0].queue); > + return -rte_errno; > +} > + > +/** > * DPDK callback to start the device. > * > * Simulate device start by attaching all configured flows. > @@ -192,6 +282,13 @@ > mlx5_txq_stop(dev); > return -rte_errno; > } > + ret =3D mlx5_hairpin_bind(dev); > + if (ret) { > + DRV_LOG(ERR, "port %u hairpin binding failed: %s", > + dev->data->port_id, strerror(rte_errno)); > + mlx5_txq_stop(dev); > + return -rte_errno; > + } > dev->data->dev_started =3D 1; > ret =3D mlx5_rx_intr_vec_enable(dev); > if (ret) { > -- > 1.8.3.1