From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 03ED2A0613 for ; Thu, 26 Sep 2019 11:32:09 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id BE16E1BEF0; Thu, 26 Sep 2019 11:32:09 +0200 (CEST) Received: from EUR02-VE1-obe.outbound.protection.outlook.com (mail-eopbgr20068.outbound.protection.outlook.com [40.107.2.68]) by dpdk.org (Postfix) with ESMTP id DA9A41BE87 for ; Thu, 26 Sep 2019 11:32:07 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=NXv6bAEqDe4OnSoFMQKfkKeBbPEZ02RAeri60K3QERPXnSmlRtfFip4vkeNxzG1x5OTpvmiYI6VR0HpXY135OEo8hTBs4WR8ZhpFNdZkI7SOil+ie6Z+lxdKFJyZRZQtMrVnsqT7MK42Ec6byYvtOYCy8IysplhwPnR6D1tZh/P3sVvE7qihsmnnWprQmDYl5asNIcomLHebTPVrQJr4HC6JT8Dl31lQkFcqThjyS9dw4WMdh2U3nTAVKrgkHftT1Zrx6oxOBJ8JSolx6KXvGB6n1DsszdmayMTBqBqxzmLQth8WPyMVQ4zMyWXX6+z5Fso50cOQg9DLWzxZEzWxcQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=J1J8Sj9Z6/tiOG//I3akZUjHwEw+R1DvwplBAmV/WvQ=; b=hMbQLvkjFT3uiSyhVhHNFzTCYEgqqO/exevpVye/uIrz8hUeuc0TMdhzG7mX2fceXMCvQ7x4BSRmv0MSio50t9wOkk+MmL5PhvbsGeeMH2RReZfnhuot9YEt1Kf9AZPB7MmcjGXbdlWrBCd2Qk2czK89hHOIz4+Yu8aH5JuFaojJxCXmRodCqKvs0LHjsK8B1tWctz0HyKsxaUU3ZfxxszZ2Liw+CTjY2rZEuC3XqB9UfCRnMwCzh+WhvrRXW+D2RmdrxayKkpvS5RkoQEEFCaYrwONdEgKrVgOly9479R7flyV9Zu9+mRrXLGiiEVmZX9/1BC6Ls0PciU2KUy2Brg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=mellanox.com; dmarc=pass action=none header.from=mellanox.com; dkim=pass header.d=mellanox.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=J1J8Sj9Z6/tiOG//I3akZUjHwEw+R1DvwplBAmV/WvQ=; b=WQk4M/1IE9kijSe2ls/MGcrRcsvTQYl3TYLgEDjXbiasKC92Zh2aIPFgfRkPN+hf/RE0dk5Ak/fmHZ3gUzk8IIV8que7oZr8CCKr9xMtPRTW4LComPIlkB97LcbaYtVFzhYUNJ/f1zpS3Vf0QzudrOG0NcTnve9sZoVYY4QR6fE= Received: from AM4PR05MB3265.eurprd05.prod.outlook.com (10.171.188.154) by AM4PR05MB3153.eurprd05.prod.outlook.com (10.171.188.22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2284.25; Thu, 26 Sep 2019 09:32:06 +0000 Received: from AM4PR05MB3265.eurprd05.prod.outlook.com ([fe80::da9:65ba:1323:a39b]) by AM4PR05MB3265.eurprd05.prod.outlook.com ([fe80::da9:65ba:1323:a39b%7]) with mapi id 15.20.2305.017; Thu, 26 Sep 2019 09:32:06 +0000 From: Slava Ovsiienko To: Ori Kam , Matan Azrad , Shahaf Shuler CC: "dev@dpdk.org" , Ori Kam , "jingjing.wu@intel.com" , "stephen@networkplumber.org" Thread-Topic: [PATCH 03/13] net/mlx5: support Rx hairpin queues Thread-Index: AQHVdDPdW8v3zOqAHUW1fvWL9rFXAKc9shLw Date: Thu, 26 Sep 2019 09:32:06 +0000 Message-ID: References: <1569479349-36962-1-git-send-email-orika@mellanox.com> <1569479349-36962-4-git-send-email-orika@mellanox.com> In-Reply-To: <1569479349-36962-4-git-send-email-orika@mellanox.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=viacheslavo@mellanox.com; x-originating-ip: [95.67.35.250] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 4d606dcc-7c8c-4dce-182f-08d742646578 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600167)(711020)(4605104)(1401327)(4618075)(2017052603328)(7193020); SRVR:AM4PR05MB3153; x-ms-traffictypediagnostic: AM4PR05MB3153:|AM4PR05MB3153: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:16; x-forefront-prvs: 0172F0EF77 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(4636009)(346002)(39860400002)(136003)(366004)(376002)(396003)(13464003)(199004)(189003)(74316002)(229853002)(71200400001)(446003)(7736002)(486006)(55016002)(71190400001)(3846002)(102836004)(76116006)(66946007)(7696005)(76176011)(9686003)(5660300002)(86362001)(33656002)(6116002)(52536014)(256004)(6436002)(6246003)(14444005)(316002)(30864003)(476003)(11346002)(305945005)(8676002)(81166006)(6636002)(186003)(26005)(66446008)(64756008)(66556008)(66476007)(81156014)(4326008)(14454004)(110136005)(54906003)(478600001)(8936002)(53546011)(6506007)(66066001)(2906002)(99286004)(25786009); DIR:OUT; SFP:1101; SCL:1; SRVR:AM4PR05MB3153; H:AM4PR05MB3265.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: CZOXqJ2u4KjbknQn+ObEalVZG3HjGFJwJrtUSp8Jm3BXGgAGNdtbk3EQBLZWiTTCxyTc6Q9k8QfM7CCbD46qDsRlZbyxctGOB6u2yx1gKQ/cBnRdj9RmNyHiAXMfNL8cY/ThgxgyxTujr8obSuX8MmGdL15YjGlPSgZcUxAqAPEvD+D9Hwr3zEWS7mCmpcu7KjIUc/SOiuRw2cSGadxG5hsYsv6Wl5gyHLczeaZQOdvvhprW8AmYv7p7uXb+nwXe7hsXLjRtz68q1FLAM+LafvUdnI5vq4j3ejyC20I88KcvvPnylyTc/ZuA3UMWzSNtdzQISQQtk3Z7tKBjTpHHlPDN9erk1dkIqOP0NjnAGdp2XIK0/6rGb9xaeYrYiwM7JLxkoKoXSibP2RE1UkwA1pEqkXLXmWjmPAtWXZf2CN8= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4d606dcc-7c8c-4dce-182f-08d742646578 X-MS-Exchange-CrossTenant-originalarrivaltime: 26 Sep 2019 09:32:06.1147 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: pfqBKbIOySWzDN8peHcoflPlFwuyqIXcj13ke6hVg6Cud4TfrYX3QQC6YhWBwgXYDRwOpQkrT7lABPsZzL6RY1M7PdE5EMtMfWFe2E9NrMU= X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM4PR05MB3153 Subject: Re: [dpdk-dev] [PATCH 03/13] net/mlx5: support Rx hairpin queues X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > -----Original Message----- > From: Ori Kam > Sent: Thursday, September 26, 2019 9:29 > To: Matan Azrad ; Shahaf Shuler > ; Slava Ovsiienko > Cc: dev@dpdk.org; Ori Kam ; jingjing.wu@intel.com; > stephen@networkplumber.org > Subject: [PATCH 03/13] net/mlx5: support Rx hairpin queues >=20 > This commit adds the support for creating Rx hairpin queues. > Hairpin queue is a queue that is created using DevX and only used by the = HW. > This results in that all the data part of the RQ is not being used. >=20 > Signed-off-by: Ori Kam Acked-by: Viacheslav Ovsiienko > --- > drivers/net/mlx5/mlx5.c | 2 + > drivers/net/mlx5/mlx5_rxq.c | 286 > ++++++++++++++++++++++++++++++++++++---- > drivers/net/mlx5/mlx5_rxtx.h | 17 +++ > drivers/net/mlx5/mlx5_trigger.c | 7 + > 4 files changed, 288 insertions(+), 24 deletions(-) >=20 > diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index > be01db9..81894fb 100644 > --- a/drivers/net/mlx5/mlx5.c > +++ b/drivers/net/mlx5/mlx5.c > @@ -974,6 +974,7 @@ struct mlx5_dev_spawn_data { > .dev_supported_ptypes_get =3D mlx5_dev_supported_ptypes_get, > .vlan_filter_set =3D mlx5_vlan_filter_set, > .rx_queue_setup =3D mlx5_rx_queue_setup, > + .rx_hairpin_queue_setup =3D mlx5_rx_hairpin_queue_setup, > .tx_queue_setup =3D mlx5_tx_queue_setup, > .rx_queue_release =3D mlx5_rx_queue_release, > .tx_queue_release =3D mlx5_tx_queue_release, @@ -1040,6 +1041,7 > @@ struct mlx5_dev_spawn_data { > .dev_supported_ptypes_get =3D mlx5_dev_supported_ptypes_get, > .vlan_filter_set =3D mlx5_vlan_filter_set, > .rx_queue_setup =3D mlx5_rx_queue_setup, > + .rx_hairpin_queue_setup =3D mlx5_rx_hairpin_queue_setup, > .tx_queue_setup =3D mlx5_tx_queue_setup, > .rx_queue_release =3D mlx5_rx_queue_release, > .tx_queue_release =3D mlx5_tx_queue_release, diff --git > a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c index > a1fdeef..a673da9 100644 > --- a/drivers/net/mlx5/mlx5_rxq.c > +++ b/drivers/net/mlx5/mlx5_rxq.c > @@ -106,21 +106,25 @@ > struct mlx5_priv *priv =3D dev->data->dev_private; > uint16_t i; > uint16_t n =3D 0; > + uint16_t n_ibv =3D 0; >=20 > if (mlx5_check_mprq_support(dev) < 0) > return 0; > /* All the configured queues should be enabled. */ > for (i =3D 0; i < priv->rxqs_n; ++i) { > struct mlx5_rxq_data *rxq =3D (*priv->rxqs)[i]; > + struct mlx5_rxq_ctrl *rxq_ctrl =3D container_of > + (rxq, struct mlx5_rxq_ctrl, rxq); >=20 > - if (!rxq) > + if (rxq =3D=3D NULL || rxq_ctrl->type !=3D > MLX5_RXQ_TYPE_STANDARD) > continue; > + n_ibv++; > if (mlx5_rxq_mprq_enabled(rxq)) > ++n; > } > /* Multi-Packet RQ can't be partially configured. */ > - assert(n =3D=3D 0 || n =3D=3D priv->rxqs_n); > - return n =3D=3D priv->rxqs_n; > + assert(n =3D=3D 0 || n =3D=3D n_ibv); > + return n =3D=3D n_ibv; > } >=20 > /** > @@ -427,6 +431,7 @@ > } >=20 > /** > + * Rx queue presetup checks. > * > * @param dev > * Pointer to Ethernet device structure. > @@ -434,25 +439,14 @@ > * RX queue index. > * @param desc > * Number of descriptors to configure in queue. > - * @param socket > - * NUMA socket on which memory must be allocated. > - * @param[in] conf > - * Thresholds parameters. > - * @param mp > - * Memory pool for buffer allocations. > * > * @return > * 0 on success, a negative errno value otherwise and rte_errno is set= . > */ > -int > -mlx5_rx_queue_setup(struct rte_eth_dev *dev, uint16_t idx, uint16_t desc= , > - unsigned int socket, const struct rte_eth_rxconf *conf, > - struct rte_mempool *mp) > +static int > +mlx5_rx_queue_pre_setup(struct rte_eth_dev *dev, uint16_t idx, uint16_t > +desc) > { > struct mlx5_priv *priv =3D dev->data->dev_private; > - struct mlx5_rxq_data *rxq =3D (*priv->rxqs)[idx]; > - struct mlx5_rxq_ctrl *rxq_ctrl =3D > - container_of(rxq, struct mlx5_rxq_ctrl, rxq); >=20 > if (!rte_is_power_of_2(desc)) { > desc =3D 1 << log2above(desc); > @@ -476,6 +470,41 @@ > return -rte_errno; > } > mlx5_rxq_release(dev, idx); > + return 0; > +} > + > +/** > + * > + * @param dev > + * Pointer to Ethernet device structure. > + * @param idx > + * RX queue index. > + * @param desc > + * Number of descriptors to configure in queue. > + * @param socket > + * NUMA socket on which memory must be allocated. > + * @param[in] conf > + * Thresholds parameters. > + * @param mp > + * Memory pool for buffer allocations. > + * > + * @return > + * 0 on success, a negative errno value otherwise and rte_errno is set= . > + */ > +int > +mlx5_rx_queue_setup(struct rte_eth_dev *dev, uint16_t idx, uint16_t desc= , > + unsigned int socket, const struct rte_eth_rxconf *conf, > + struct rte_mempool *mp) > +{ > + struct mlx5_priv *priv =3D dev->data->dev_private; > + struct mlx5_rxq_data *rxq =3D (*priv->rxqs)[idx]; > + struct mlx5_rxq_ctrl *rxq_ctrl =3D > + container_of(rxq, struct mlx5_rxq_ctrl, rxq); > + int res; > + > + res =3D mlx5_rx_queue_pre_setup(dev, idx, desc); > + if (res) > + return res; > rxq_ctrl =3D mlx5_rxq_new(dev, idx, desc, socket, conf, mp); > if (!rxq_ctrl) { > DRV_LOG(ERR, "port %u unable to allocate queue index %u", > @@ -490,6 +519,62 @@ } >=20 > /** > + * > + * @param dev > + * Pointer to Ethernet device structure. > + * @param idx > + * RX queue index. > + * @param desc > + * Number of descriptors to configure in queue. > + * @param socket > + * NUMA socket on which memory must be allocated. > + * @param[in] conf > + * Thresholds parameters. > + * @param hairpin_conf > + * Hairpin configuration parameters. > + * > + * @return > + * 0 on success, a negative errno value otherwise and rte_errno is set= . > + */ > +int > +mlx5_rx_hairpin_queue_setup(struct rte_eth_dev *dev, uint16_t idx, > + uint16_t desc, unsigned int socket, > + const struct rte_eth_rxconf *conf, > + const struct rte_eth_hairpin_conf *hairpin_conf) { > + struct mlx5_priv *priv =3D dev->data->dev_private; > + struct mlx5_rxq_data *rxq =3D (*priv->rxqs)[idx]; > + struct mlx5_rxq_ctrl *rxq_ctrl =3D > + container_of(rxq, struct mlx5_rxq_ctrl, rxq); > + int res; > + > + res =3D mlx5_rx_queue_pre_setup(dev, idx, desc); > + if (res) > + return res; > + if (hairpin_conf->peer_n !=3D 1 || > + hairpin_conf->peers[0].port !=3D dev->data->port_id || > + hairpin_conf->peers[0].queue >=3D priv->txqs_n) { > + DRV_LOG(ERR, "port %u unable to setup hairpin queue index > %u " > + " invalid hairpind configuration", dev->data->port_id, > + idx); > + rte_errno =3D EINVAL; > + return -rte_errno; > + } > + rxq_ctrl =3D mlx5_rxq_hairpin_new(dev, idx, desc, socket, conf, > + hairpin_conf); > + if (!rxq_ctrl) { > + DRV_LOG(ERR, "port %u unable to allocate queue index %u", > + dev->data->port_id, idx); > + rte_errno =3D ENOMEM; > + return -rte_errno; > + } > + DRV_LOG(DEBUG, "port %u adding Rx queue %u to list", > + dev->data->port_id, idx); > + (*priv->rxqs)[idx] =3D &rxq_ctrl->rxq; > + return 0; > +} > + > +/** > * DPDK callback to release a RX queue. > * > * @param dpdk_rxq > @@ -561,6 +646,24 @@ > } >=20 > /** > + * Release an Rx hairpin related resources. > + * > + * @param rxq_obj > + * Hairpin Rx queue object. > + */ > +static void > +rxq_obj_hairpin_release(struct mlx5_rxq_obj *rxq_obj) { > + struct mlx5_devx_modify_rq_attr rq_attr =3D { 0 }; > + > + assert(rxq_obj); > + rq_attr.state =3D MLX5_RQC_STATE_RST; > + rq_attr.rq_state =3D MLX5_RQC_STATE_RDY; > + mlx5_devx_cmd_modify_rq(rxq_obj->rq, &rq_attr); > + claim_zero(mlx5_devx_cmd_destroy(rxq_obj->rq)); > +} > + > +/** > * Release an Rx verbs/DevX queue object. > * > * @param rxq_obj > @@ -577,14 +680,22 @@ > assert(rxq_obj->wq); > assert(rxq_obj->cq); > if (rte_atomic32_dec_and_test(&rxq_obj->refcnt)) { > - rxq_free_elts(rxq_obj->rxq_ctrl); > - if (rxq_obj->type =3D=3D MLX5_RXQ_OBJ_TYPE_IBV) { > + switch (rxq_obj->type) { > + case MLX5_RXQ_OBJ_TYPE_IBV: > + rxq_free_elts(rxq_obj->rxq_ctrl); > claim_zero(mlx5_glue->destroy_wq(rxq_obj->wq)); > - } else if (rxq_obj->type =3D=3D MLX5_RXQ_OBJ_TYPE_DEVX_RQ) { > + claim_zero(mlx5_glue->destroy_cq(rxq_obj->cq)); > + break; > + case MLX5_RXQ_OBJ_TYPE_DEVX_RQ: > + rxq_free_elts(rxq_obj->rxq_ctrl); > claim_zero(mlx5_devx_cmd_destroy(rxq_obj->rq)); > rxq_release_rq_resources(rxq_obj->rxq_ctrl); > + claim_zero(mlx5_glue->destroy_cq(rxq_obj->cq)); > + break; > + case MLX5_RXQ_OBJ_TYPE_DEVX_HAIRPIN: > + rxq_obj_hairpin_release(rxq_obj); > + break; > } > - claim_zero(mlx5_glue->destroy_cq(rxq_obj->cq)); > if (rxq_obj->channel) > claim_zero(mlx5_glue->destroy_comp_channel > (rxq_obj->channel)); > @@ -1132,6 +1243,70 @@ > } >=20 > /** > + * Create the Rx hairpin queue object. > + * > + * @param dev > + * Pointer to Ethernet device. > + * @param idx > + * Queue index in DPDK Rx queue array > + * > + * @return > + * The hairpin DevX object initialised, NULL otherwise and rte_errno i= s set. > + */ > +static struct mlx5_rxq_obj * > +mlx5_rxq_obj_hairpin_new(struct rte_eth_dev *dev, uint16_t idx) { > + struct mlx5_priv *priv =3D dev->data->dev_private; > + struct mlx5_rxq_data *rxq_data =3D (*priv->rxqs)[idx]; > + struct mlx5_rxq_ctrl *rxq_ctrl =3D > + container_of(rxq_data, struct mlx5_rxq_ctrl, rxq); > + struct mlx5_devx_create_rq_attr attr =3D { 0 }; > + struct mlx5_rxq_obj *tmpl =3D NULL; > + int ret =3D 0; > + > + assert(rxq_data); > + assert(!rxq_ctrl->obj); > + tmpl =3D rte_calloc_socket(__func__, 1, sizeof(*tmpl), 0, > + rxq_ctrl->socket); > + if (!tmpl) { > + DRV_LOG(ERR, > + "port %u Rx queue %u cannot allocate verbs > resources", > + dev->data->port_id, rxq_data->idx); > + rte_errno =3D ENOMEM; > + goto error; > + } > + tmpl->type =3D MLX5_RXQ_OBJ_TYPE_DEVX_HAIRPIN; > + tmpl->rxq_ctrl =3D rxq_ctrl; > + attr.hairpin =3D 1; > + /* Workaround for hairpin startup */ > + attr.wq_attr.log_hairpin_num_packets =3D log2above(32); > + /* Workaround for packets larger than 1KB */ > + attr.wq_attr.log_hairpin_data_sz =3D > + priv->config.hca_attr.log_max_hairpin_wq_data_sz; > + tmpl->rq =3D mlx5_devx_cmd_create_rq(priv->sh->ctx, &attr, > + rxq_ctrl->socket); > + if (!tmpl->rq) { > + DRV_LOG(ERR, > + "port %u Rx hairpin queue %u can't create rq object", > + dev->data->port_id, idx); > + rte_errno =3D errno; > + goto error; > + } > + DRV_LOG(DEBUG, "port %u rxq %u updated with %p", dev->data- > >port_id, > + idx, (void *)&tmpl); > + rte_atomic32_inc(&tmpl->refcnt); > + LIST_INSERT_HEAD(&priv->rxqsobj, tmpl, next); > + priv->verbs_alloc_ctx.type =3D MLX5_VERBS_ALLOC_TYPE_NONE; > + return tmpl; > +error: > + ret =3D rte_errno; /* Save rte_errno before cleanup. */ > + if (tmpl->rq) > + mlx5_devx_cmd_destroy(tmpl->rq); > + rte_errno =3D ret; /* Restore rte_errno. */ > + return NULL; > +} > + > +/** > * Create the Rx queue Verbs/DevX object. > * > * @param dev > @@ -1163,6 +1338,8 @@ struct mlx5_rxq_obj * >=20 > assert(rxq_data); > assert(!rxq_ctrl->obj); > + if (type =3D=3D MLX5_RXQ_OBJ_TYPE_DEVX_HAIRPIN) > + return mlx5_rxq_obj_hairpin_new(dev, idx); > priv->verbs_alloc_ctx.type =3D MLX5_VERBS_ALLOC_TYPE_RX_QUEUE; > priv->verbs_alloc_ctx.obj =3D rxq_ctrl; > tmpl =3D rte_calloc_socket(__func__, 1, sizeof(*tmpl), 0, @@ -1433,15 > +1610,19 @@ struct mlx5_rxq_obj * > unsigned int strd_num_n =3D 0; > unsigned int strd_sz_n =3D 0; > unsigned int i; > + unsigned int n_ibv =3D 0; >=20 > if (!mlx5_mprq_enabled(dev)) > return 0; > /* Count the total number of descriptors configured. */ > for (i =3D 0; i !=3D priv->rxqs_n; ++i) { > struct mlx5_rxq_data *rxq =3D (*priv->rxqs)[i]; > + struct mlx5_rxq_ctrl *rxq_ctrl =3D container_of > + (rxq, struct mlx5_rxq_ctrl, rxq); >=20 > - if (rxq =3D=3D NULL) > + if (rxq =3D=3D NULL || rxq_ctrl->type !=3D > MLX5_RXQ_TYPE_STANDARD) > continue; > + n_ibv++; > desc +=3D 1 << rxq->elts_n; > /* Get the max number of strides. */ > if (strd_num_n < rxq->strd_num_n) > @@ -1466,7 +1647,7 @@ struct mlx5_rxq_obj * > * this Mempool gets available again. > */ > desc *=3D 4; > - obj_num =3D desc + MLX5_MPRQ_MP_CACHE_SZ * priv->rxqs_n; > + obj_num =3D desc + MLX5_MPRQ_MP_CACHE_SZ * n_ibv; > /* > * rte_mempool_create_empty() has sanity check to refuse large > cache > * size compared to the number of elements. > @@ -1514,8 +1695,10 @@ struct mlx5_rxq_obj * > /* Set mempool for each Rx queue. */ > for (i =3D 0; i !=3D priv->rxqs_n; ++i) { > struct mlx5_rxq_data *rxq =3D (*priv->rxqs)[i]; > + struct mlx5_rxq_ctrl *rxq_ctrl =3D container_of > + (rxq, struct mlx5_rxq_ctrl, rxq); >=20 > - if (rxq =3D=3D NULL) > + if (rxq =3D=3D NULL || rxq_ctrl->type !=3D > MLX5_RXQ_TYPE_STANDARD) > continue; > rxq->mprq_mp =3D mp; > } > @@ -1620,6 +1803,7 @@ struct mlx5_rxq_ctrl * > rte_errno =3D ENOMEM; > return NULL; > } > + tmpl->type =3D MLX5_RXQ_TYPE_STANDARD; > if (mlx5_mr_btree_init(&tmpl->rxq.mr_ctrl.cache_bh, > MLX5_MR_BTREE_CACHE_N, socket)) { > /* rte_errno is already set. */ > @@ -1788,6 +1972,59 @@ struct mlx5_rxq_ctrl * } >=20 > /** > + * Create a DPDK Rx hairpin queue. > + * > + * @param dev > + * Pointer to Ethernet device. > + * @param idx > + * RX queue index. > + * @param desc > + * Number of descriptors to configure in queue. > + * @param socket > + * NUMA socket on which memory must be allocated. > + * @param conf > + * The Rx configuration. > + * @param hairpin_conf > + * The hairpin binding configuration. > + * > + * @return > + * A DPDK queue object on success, NULL otherwise and rte_errno is set= . > + */ > +struct mlx5_rxq_ctrl * > +mlx5_rxq_hairpin_new(struct rte_eth_dev *dev, uint16_t idx, uint16_t des= c, > + unsigned int socket, const struct rte_eth_rxconf *conf, > + const struct rte_eth_hairpin_conf *hairpin_conf) { > + struct mlx5_priv *priv =3D dev->data->dev_private; > + struct mlx5_rxq_ctrl *tmpl; > + uint64_t offloads =3D conf->offloads | > + dev->data->dev_conf.rxmode.offloads; > + > + tmpl =3D rte_calloc_socket("RXQ", 1, sizeof(*tmpl), 0, socket); > + if (!tmpl) { > + rte_errno =3D ENOMEM; > + return NULL; > + } > + tmpl->type =3D MLX5_RXQ_TYPE_HAIRPIN; > + tmpl->socket =3D socket; > + /* Configure VLAN stripping. */ > + tmpl->rxq.vlan_strip =3D !!(offloads & > DEV_RX_OFFLOAD_VLAN_STRIP); > + /* Save port ID. */ > + tmpl->rxq.rss_hash =3D 0; > + tmpl->rxq.port_id =3D dev->data->port_id; > + tmpl->priv =3D priv; > + tmpl->rxq.mp =3D NULL; > + tmpl->rxq.elts_n =3D log2above(desc); > + tmpl->rxq.elts =3D NULL; > + tmpl->rxq.mr_ctrl.cache_bh =3D (struct mlx5_mr_btree) { 0 }; > + tmpl->hairpin_conf =3D *hairpin_conf; > + tmpl->rxq.idx =3D idx; > + rte_atomic32_inc(&tmpl->refcnt); > + LIST_INSERT_HEAD(&priv->rxqsctrl, tmpl, next); > + return tmpl; > +} > + > +/** > * Get a Rx queue. > * > * @param dev > @@ -1841,7 +2078,8 @@ struct mlx5_rxq_ctrl * > if (rxq_ctrl->dbr_umem_id_valid) > claim_zero(mlx5_release_dbr(dev, rxq_ctrl- > >dbr_umem_id, > rxq_ctrl->dbr_offset)); > - mlx5_mr_btree_free(&rxq_ctrl->rxq.mr_ctrl.cache_bh); > + if (rxq_ctrl->type =3D=3D MLX5_RXQ_TYPE_STANDARD) > + mlx5_mr_btree_free(&rxq_ctrl- > >rxq.mr_ctrl.cache_bh); > LIST_REMOVE(rxq_ctrl, next); > rte_free(rxq_ctrl); > (*priv->rxqs)[idx] =3D NULL; > diff --git a/drivers/net/mlx5/mlx5_rxtx.h b/drivers/net/mlx5/mlx5_rxtx.h > index 4bb28a4..dbb616e 100644 > --- a/drivers/net/mlx5/mlx5_rxtx.h > +++ b/drivers/net/mlx5/mlx5_rxtx.h > @@ -159,6 +159,13 @@ struct mlx5_rxq_data { enum mlx5_rxq_obj_type { > MLX5_RXQ_OBJ_TYPE_IBV, /* mlx5_rxq_obj with > ibv_wq. */ > MLX5_RXQ_OBJ_TYPE_DEVX_RQ, /* mlx5_rxq_obj with > mlx5_devx_rq. */ > + MLX5_RXQ_OBJ_TYPE_DEVX_HAIRPIN, > + /* mlx5_rxq_obj with mlx5_devx_rq and hairpin support. */ }; > + > +enum mlx5_rxq_type { > + MLX5_RXQ_TYPE_STANDARD, /* Standard Rx queue. */ > + MLX5_RXQ_TYPE_HAIRPIN, /* Hairpin Rx queue. */ > }; >=20 > /* Verbs/DevX Rx queue elements. */ > @@ -183,6 +190,7 @@ struct mlx5_rxq_ctrl { > rte_atomic32_t refcnt; /* Reference counter. */ > struct mlx5_rxq_obj *obj; /* Verbs/DevX elements. */ > struct mlx5_priv *priv; /* Back pointer to private data. */ > + enum mlx5_rxq_type type; /* Rxq type. */ > unsigned int socket; /* CPU socket ID for allocations. */ > unsigned int irq:1; /* Whether IRQ is enabled. */ > unsigned int dbr_umem_id_valid:1; /* dbr_umem_id holds a valid > value. */ @@ -193,6 +201,7 @@ struct mlx5_rxq_ctrl { > uint32_t dbr_umem_id; /* Storing door-bell information, */ > uint64_t dbr_offset; /* needed when freeing door-bell. */ > struct mlx5dv_devx_umem *wq_umem; /* WQ buffer registration > info. */ > + struct rte_eth_hairpin_conf hairpin_conf; /* Hairpin configuration. > */ > }; >=20 > enum mlx5_ind_tbl_type { > @@ -339,6 +348,10 @@ struct mlx5_txq_ctrl { int > mlx5_rx_queue_setup(struct rte_eth_dev *dev, uint16_t idx, uint16_t desc, > unsigned int socket, const struct rte_eth_rxconf > *conf, > struct rte_mempool *mp); > +int mlx5_rx_hairpin_queue_setup > + (struct rte_eth_dev *dev, uint16_t idx, uint16_t desc, > + unsigned int socket, const struct rte_eth_rxconf *conf, > + const struct rte_eth_hairpin_conf *hairpin_conf); > void mlx5_rx_queue_release(void *dpdk_rxq); int > mlx5_rx_intr_vec_enable(struct rte_eth_dev *dev); void > mlx5_rx_intr_vec_disable(struct rte_eth_dev *dev); @@ -351,6 +364,10 @@ > struct mlx5_rxq_ctrl *mlx5_rxq_new(struct rte_eth_dev *dev, uint16_t idx, > uint16_t desc, unsigned int socket, > const struct rte_eth_rxconf *conf, > struct rte_mempool *mp); > +struct mlx5_rxq_ctrl *mlx5_rxq_hairpin_new > + (struct rte_eth_dev *dev, uint16_t idx, uint16_t desc, > + unsigned int socket, const struct rte_eth_rxconf *conf, > + const struct rte_eth_hairpin_conf *hairpin_conf); > struct mlx5_rxq_ctrl *mlx5_rxq_get(struct rte_eth_dev *dev, uint16_t idx= ); > int mlx5_rxq_release(struct rte_eth_dev *dev, uint16_t idx); int > mlx5_rxq_verify(struct rte_eth_dev *dev); diff --git > a/drivers/net/mlx5/mlx5_trigger.c b/drivers/net/mlx5/mlx5_trigger.c index > 122f31c..cb31ae2 100644 > --- a/drivers/net/mlx5/mlx5_trigger.c > +++ b/drivers/net/mlx5/mlx5_trigger.c > @@ -118,6 +118,13 @@ >=20 > if (!rxq_ctrl) > continue; > + if (rxq_ctrl->type =3D=3D MLX5_RXQ_TYPE_HAIRPIN) { > + rxq_ctrl->obj =3D mlx5_rxq_obj_new > + (dev, i, > MLX5_RXQ_OBJ_TYPE_DEVX_HAIRPIN); > + if (!rxq_ctrl->obj) > + goto error; > + continue; > + } > /* Pre-register Rx mempool. */ > mp =3D mlx5_rxq_mprq_enabled(&rxq_ctrl->rxq) ? > rxq_ctrl->rxq.mprq_mp : rxq_ctrl->rxq.mp; > -- > 1.8.3.1