From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2D3E4A0352; Thu, 16 Jan 2020 12:40:35 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 8739E1C23E; Thu, 16 Jan 2020 12:40:34 +0100 (CET) Received: from EUR02-HE1-obe.outbound.protection.outlook.com (mail-eopbgr10052.outbound.protection.outlook.com [40.107.1.52]) by dpdk.org (Postfix) with ESMTP id D1F191C229 for ; Thu, 16 Jan 2020 12:40:32 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=M5Awb5AYq9JA3HpRBv+5w6RyOYP7VHlveo99lTh2T04w13NW1mfSXA0EGWMYeIChVdHV0VIAg8GwRZTz27+J6HhDlNouJ1l558sE1d3ZlVAzdCtlJMHnq8nzaYUqEg5muVlZ6YTRPLcQSZOj53m8JKCzBfDlOvzg9iKcI3d9RvsW2GZPF2//A9NISyrKxXQnB0zlkBBzfAY4rbFE1D2Og85EtRSqVaz79aKA4Ax3Hx5tdxrV0xFCQ4128W6o+QiU12A/T4ha/fjlwLQQvYRjuAcFIlW7g7YDqaxe+moVTEttQWUtF897RwxxkqHY5jN56Qy+OVmEdLDKlk7S7slbKA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/8aM+RC83XHeSz/SgfzjT9z3BqnaiaFOdANJ/hPIAdo=; b=Ej9OGrcuEK20Pqu5R5YogMHUQbOopuwKLJ0eZl/ULBqDL7WPzh34VbJOvvJgwv63F7rf8X9kqkHqRFjDqEVVpjX/CraQmD+ffjsBuNkAdwT9m2XdmLhDDK1FBg/6VEQJV+GYmx1tpH0KN7T0WVKsPPTYAddyf7U11jyB0MZ7dZkuzJhp6rUnIQn87V580ar0XGDRTlzt64rsyR50UuNNN0jAv7oAI4oT79c+g7A6t6DJ/duUZBJ22okJXMfGWSQ4QPNfHss8DQSyMbYWyS56ToPdseWXQTOJ4GFMKAWGIyVPmxzNg2H+ePOfr5j9BYdPw0XupO2cMkoKqrpRhsvung== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=mellanox.com; dmarc=pass action=none header.from=mellanox.com; dkim=pass header.d=mellanox.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/8aM+RC83XHeSz/SgfzjT9z3BqnaiaFOdANJ/hPIAdo=; b=ZUmCkPYk3FTLtwamIrCLW5hEDdATizEPWbIpk5ceYCFtXOQUgPDGN17nRh6XLemUU5MVQVsc/a6nz4ceyz4hCEIpZPKhfDFtypGCFHZpRivIoaqcWddFk5q36Cu8VY0V6FOzJZ4uv6oeoBU/Fag5hSFPSdWT1qutPW2p2D+e7qA= Received: from AM5PR0501MB2579.eurprd05.prod.outlook.com (10.169.150.151) by AM5PR0501MB2514.eurprd05.prod.outlook.com (10.169.150.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2644.20; Thu, 16 Jan 2020 11:40:31 +0000 Received: from AM5PR0501MB2579.eurprd05.prod.outlook.com ([fe80::189a:c6d4:a60d:9a19]) by AM5PR0501MB2579.eurprd05.prod.outlook.com ([fe80::189a:c6d4:a60d:9a19%8]) with mapi id 15.20.2644.015; Thu, 16 Jan 2020 11:40:31 +0000 From: Ori Kam To: "xiangxia.m.yue@gmail.com" CC: "dev@dpdk.org" Thread-Topic: [PATCH dpdk-dev] net/mlx5: support hairpin between different ports Thread-Index: AQHVqCqoW4PqWU19lEGJ3MDJ+UqaZqftcsBQ Date: Thu, 16 Jan 2020 11:40:31 +0000 Message-ID: References: <1575192909-105146-1-git-send-email-xiangxia.m.yue@gmail.com> In-Reply-To: <1575192909-105146-1-git-send-email-xiangxia.m.yue@gmail.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=orika@mellanox.com; x-originating-ip: [193.47.165.251] x-ms-publictraffictype: Email x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: 69f01338-4a0c-46ff-92cc-08d79a78e446 x-ms-traffictypediagnostic: AM5PR0501MB2514: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:3276; x-forefront-prvs: 02843AA9E0 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(4636009)(366004)(199004)(189003)(66446008)(64756008)(66556008)(66476007)(66946007)(26005)(76116006)(52536014)(186003)(2906002)(4326008)(498600001)(7696005)(6916009)(5660300002)(71200400001)(55016002)(33656002)(6506007)(9686003)(86362001)(53546011)(81166006)(81156014)(8936002)(8676002); DIR:OUT; SFP:1101; SCL:1; SRVR:AM5PR0501MB2514; H:AM5PR0501MB2579.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 8V+A0hzj7xIKkUsTK8cnBhAsQFKbX3Y5qAOOLRy+yjhqAVG62gkEI5MmwKKfE+nNd3T2NA3hfiS3kvHzJQjF8C/cxAhw07QjWLJHy5WDVrlzLVstboVmdePswJYcdEk6UqRmkZhh3/3n7uDh28tV5Jcf4prceyvHQokJ/5ow9CLG1JZt4F9ivY5C0L9sbO0LtiDI0LU/GSTjM2eHC0IfmFpIznHgd0/FLQkrQVQM9a1EhlDZdU1wKN7noPtj20CWdUrDMrQGfBIs1XIE/oH/lPPSmgXAo1utlI3+5h02NW/rVlJALKk3gbNIrnJDgA89gooZ8YBq/pj5D5Yjf+VDTjQ7hDG6lkEPYeIugKGcMQkkU6MLvPse+JKQDHGmoaAsLITdrcgD08Jm0BU+2kNpKCHWKF6/NW7lVoN9BFM8Rr03apLtB6wDUzgUzkpnldJ4 x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 69f01338-4a0c-46ff-92cc-08d79a78e446 X-MS-Exchange-CrossTenant-originalarrivaltime: 16 Jan 2020 11:40:31.1628 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: Tk1ATkxyeI5xJieHGPXz5py929rlOwSobL14S3IATfe9n4YzGaGjSh7wi/9+GfVV+ch1AsP4vT5tzlUgt4ZVSA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM5PR0501MB2514 Subject: Re: [dpdk-dev] [PATCH dpdk-dev] net/mlx5: support hairpin between different ports X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Sorry for the late response, Thank you very much for this patch. We are planning to add such support, but this patch is not the best way. I will keep you updated about the progress, and will be very to hear=20 your inputs. Best, Ori > -----Original Message----- > From: xiangxia.m.yue@gmail.com > Sent: Sunday, December 1, 2019 11:35 AM > To: Ori Kam > Cc: dev@dpdk.org; Tonghao Zhang > Subject: [PATCH dpdk-dev] net/mlx5: support hairpin between different > ports >=20 > From: Tonghao Zhang >=20 > In the dpdk upstream, each hairpin Rxq can be connected > Txq which can belong to a same port. This patch allows > Rxq connected to different port Txq. >=20 > rte_eth_hairpin_conf_check will check the hairpin_conf valid > in high level. >=20 > Signed-off-by: Tonghao Zhang > --- > drivers/net/mlx5/mlx5_rxq.c | 5 ++- > drivers/net/mlx5/mlx5_trigger.c | 41 ++++++++++++++++------- > drivers/net/mlx5/mlx5_txq.c | 4 +-- > lib/librte_ethdev/rte_ethdev.c | 73 ++++++++++++++++++++++++++++---- > --------- > 4 files changed, 83 insertions(+), 40 deletions(-) >=20 > diff --git a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c > index 986ec01..c2daebd 100644 > --- a/drivers/net/mlx5/mlx5_rxq.c > +++ b/drivers/net/mlx5/mlx5_rxq.c > @@ -547,9 +547,8 @@ > res =3D mlx5_rx_queue_pre_setup(dev, idx, desc); > if (res) > return res; > - if (hairpin_conf->peer_count !=3D 1 || > - hairpin_conf->peers[0].port !=3D dev->data->port_id || > - hairpin_conf->peers[0].queue >=3D priv->txqs_n) { > + > + if (hairpin_conf->peer_count !=3D 1) { > DRV_LOG(ERR, "port %u unable to setup hairpin queue index > %u " > " invalid hairpind configuration", dev->data->port_id, > idx); > diff --git a/drivers/net/mlx5/mlx5_trigger.c > b/drivers/net/mlx5/mlx5_trigger.c > index ab6937a..3eb2984 100644 > --- a/drivers/net/mlx5/mlx5_trigger.c > +++ b/drivers/net/mlx5/mlx5_trigger.c > @@ -185,6 +185,10 @@ > struct mlx5_rxq_ctrl *rxq_ctrl; > struct mlx5_devx_obj *sq; > struct mlx5_devx_obj *rq; > + struct rte_eth_dev *rxq_dev; > + struct mlx5_priv *rxq_priv; > + uint16_t peer_queue_id; > + uint16_t peer_port_id; > unsigned int i; > int ret =3D 0; >=20 > @@ -203,37 +207,50 @@ > mlx5_txq_release(dev, i); > return -rte_errno; > } > - sq =3D txq_ctrl->obj->sq; > - rxq_ctrl =3D mlx5_rxq_get(dev, > - txq_ctrl- > >hairpin_conf.peers[0].queue); > + peer_port_id =3D txq_ctrl->hairpin_conf.peers[0].port; > + peer_queue_id =3D txq_ctrl->hairpin_conf.peers[0].queue; > + rxq_dev =3D &rte_eth_devices[peer_port_id]; > + rxq_priv =3D rxq_dev->data->dev_private; > + > + rxq_ctrl =3D mlx5_rxq_get(rxq_dev, peer_queue_id); > if (!rxq_ctrl) { > mlx5_txq_release(dev, i); > rte_errno =3D EINVAL; > DRV_LOG(ERR, "port %u no rxq object found: %d", > - dev->data->port_id, > - txq_ctrl->hairpin_conf.peers[0].queue); > + peer_port_id, peer_queue_id); > return -rte_errno; > } > if (rxq_ctrl->type !=3D MLX5_RXQ_TYPE_HAIRPIN || > rxq_ctrl->hairpin_conf.peers[0].queue !=3D i) { > rte_errno =3D ENOMEM; > DRV_LOG(ERR, "port %u Tx queue %d can't be > binded to " > - "Rx queue %d", dev->data->port_id, > - i, txq_ctrl->hairpin_conf.peers[0].queue); > + "port %u Rx queue %d", > + dev->data->port_id, i, > + peer_port_id, peer_queue_id); > goto error; > } > + if (!rxq_ctrl->obj) { > + DRV_LOG(ERR, "port %u rxq obj not created, " > + "you may start it firstly.", > + peer_port_id); > + goto error; > + } > + > + sq =3D txq_ctrl->obj->sq; > rq =3D rxq_ctrl->obj->rq; > if (!rq) { > rte_errno =3D ENOMEM; > - DRV_LOG(ERR, "port %u hairpin no matching rxq: > %d", > + DRV_LOG(ERR, "port %u hairpin no matching port" > + " %u rxq %d", > dev->data->port_id, > - txq_ctrl->hairpin_conf.peers[0].queue); > + peer_port_id, > + peer_queue_id); > goto error; > } > sq_attr.state =3D MLX5_SQC_STATE_RDY; > sq_attr.sq_state =3D MLX5_SQC_STATE_RST; > sq_attr.hairpin_peer_rq =3D rq->id; > - sq_attr.hairpin_peer_vhca =3D priv->config.hca_attr.vhca_id; > + sq_attr.hairpin_peer_vhca =3D rxq_priv- > >config.hca_attr.vhca_id; > ret =3D mlx5_devx_cmd_modify_sq(sq, &sq_attr); > if (ret) > goto error; > @@ -245,12 +262,12 @@ > if (ret) > goto error; > mlx5_txq_release(dev, i); > - mlx5_rxq_release(dev, txq_ctrl- > >hairpin_conf.peers[0].queue); > + mlx5_rxq_release(rxq_dev, peer_queue_id); > } > return 0; > error: > mlx5_txq_release(dev, i); > - mlx5_rxq_release(dev, txq_ctrl->hairpin_conf.peers[0].queue); > + mlx5_rxq_release(dev, peer_queue_id); > return -rte_errno; > } >=20 > diff --git a/drivers/net/mlx5/mlx5_txq.c b/drivers/net/mlx5/mlx5_txq.c > index bac4f71..98ee35e 100644 > --- a/drivers/net/mlx5/mlx5_txq.c > +++ b/drivers/net/mlx5/mlx5_txq.c > @@ -254,9 +254,7 @@ > res =3D mlx5_tx_queue_pre_setup(dev, idx, desc); > if (res) > return res; > - if (hairpin_conf->peer_count !=3D 1 || > - hairpin_conf->peers[0].port !=3D dev->data->port_id || > - hairpin_conf->peers[0].queue >=3D priv->rxqs_n) { > + if (hairpin_conf->peer_count !=3D 1) { > DRV_LOG(ERR, "port %u unable to setup hairpin queue index > %u " > " invalid hairpind configuration", dev->data->port_id, > idx); > diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethde= v.c > index 8d2ce31..0bd6d87 100644 > --- a/lib/librte_ethdev/rte_ethdev.c > +++ b/lib/librte_ethdev/rte_ethdev.c > @@ -1923,6 +1923,53 @@ struct rte_eth_dev * > return eth_err(port_id, ret); > } >=20 > +static int > +rte_eth_hairpin_conf_check(const struct rte_eth_hairpin_conf *conf, > + const struct rte_eth_hairpin_cap *cap, > + bool is_rx_hairpin) > +{ > + uint16_t max_queues_mapping =3D is_rx_hairpin ? > + cap->max_rx_2_tx : cap->max_tx_2_rx; > + struct rte_eth_dev *dev; > + uint16_t nb_queues; > + uint16_t port_id; > + int i; > + > + if (conf->peer_count =3D=3D 0) { > + RTE_ETHDEV_LOG(ERR, "Invalid value for number of > peers(=3D%hu)," > + " should be: > 0", > + conf->peer_count); > + return -EINVAL; > + } > + if (conf->peer_count > max_queues_mapping) { > + RTE_ETHDEV_LOG(ERR, "Invalid value for number of > peers(=3D%hu)," > + " should be: <=3D %hu", > + conf->peer_count, max_queues_mapping); > + return -EINVAL; > + } > + for (i =3D 0; i < conf->peer_count; i++) { > + port_id =3D conf->peers[i].port; > + > + if (!rte_eth_dev_is_valid_port(port_id)) { > + RTE_ETHDEV_LOG(ERR, "Invalid port_id(=3D%hu) for" > + " hairpin peers", conf->peer_count); > + return -EINVAL; > + } > + > + dev =3D &rte_eth_devices[port_id]; > + nb_queues =3D is_rx_hairpin ? > + dev->data->nb_tx_queues : dev->data- > >nb_rx_queues; > + if (conf->peers[i].queue >=3D nb_queues) { > + RTE_ETHDEV_LOG(ERR, "Invalid queue_id(=3D%hu) > for" > + " hairpin peers, shoud be < %hu", > + conf->peer_count, nb_queues); > + return -EINVAL; > + } > + } > + > + return 0; > +} > + > int > rte_eth_rx_hairpin_queue_setup(uint16_t port_id, uint16_t rx_queue_id, > uint16_t nb_rx_desc, > @@ -1956,18 +2003,9 @@ struct rte_eth_dev * > nb_rx_desc, cap.max_nb_desc); > return -EINVAL; > } > - if (conf->peer_count > cap.max_rx_2_tx) { > - RTE_ETHDEV_LOG(ERR, > - "Invalid value for number of peers for Rx > queue(=3D%hu), should be: <=3D %hu", > - conf->peer_count, cap.max_rx_2_tx); > - return -EINVAL; > - } > - if (conf->peer_count =3D=3D 0) { > - RTE_ETHDEV_LOG(ERR, > - "Invalid value for number of peers for Rx > queue(=3D%hu), should be: > 0", > - conf->peer_count); > + if (rte_eth_hairpin_conf_check(conf, &cap, true)) > return -EINVAL; > - } > + > for (i =3D 0, count =3D 0; i < dev->data->nb_rx_queues && > cap.max_nb_queues !=3D UINT16_MAX; i++) { > if (i =3D=3D rx_queue_id || > rte_eth_dev_is_rx_hairpin_queue(dev, i)) > @@ -2126,18 +2164,9 @@ struct rte_eth_dev * > nb_tx_desc, cap.max_nb_desc); > return -EINVAL; > } > - if (conf->peer_count > cap.max_tx_2_rx) { > - RTE_ETHDEV_LOG(ERR, > - "Invalid value for number of peers for Tx > queue(=3D%hu), should be: <=3D %hu", > - conf->peer_count, cap.max_tx_2_rx); > + if (rte_eth_hairpin_conf_check(conf, &cap, false)) > return -EINVAL; > - } > - if (conf->peer_count =3D=3D 0) { > - RTE_ETHDEV_LOG(ERR, > - "Invalid value for number of peers for Tx > queue(=3D%hu), should be: > 0", > - conf->peer_count); > - return -EINVAL; > - } > + > for (i =3D 0, count =3D 0; i < dev->data->nb_tx_queues && > cap.max_nb_queues !=3D UINT16_MAX; i++) { > if (i =3D=3D tx_queue_id || > rte_eth_dev_is_tx_hairpin_queue(dev, i)) > -- > 1.8.3.1