From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from dpdk.org (dpdk.org [92.243.14.124])
	by inbox.dpdk.org (Postfix) with ESMTP id 5106BA04B7;
	Sun,  4 Oct 2020 11:20:26 +0200 (CEST)
Received: from [92.243.14.124] (localhost [127.0.0.1])
	by dpdk.org (Postfix) with ESMTP id 6FA121BD10;
	Sun,  4 Oct 2020 11:20:23 +0200 (CEST)
Received: from hqnvemgate24.nvidia.com (hqnvemgate24.nvidia.com
 [216.228.121.143]) by dpdk.org (Postfix) with ESMTP id 6FADA1BCAC
 for <dev@dpdk.org>; Sun,  4 Oct 2020 11:20:20 +0200 (CEST)
Received: from hqmail.nvidia.com (Not Verified[216.228.121.13]) by
 hqnvemgate24.nvidia.com (using TLS: TLSv1.2, AES256-SHA)
 id <B5f7993660000>; Sun, 04 Oct 2020 02:18:30 -0700
Received: from HQMAIL101.nvidia.com (172.20.187.10) by HQMAIL111.nvidia.com
 (172.20.187.18) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Sun, 4 Oct
 2020 09:20:12 +0000
Received: from NAM12-BN8-obe.outbound.protection.outlook.com (104.47.55.175)
 by HQMAIL101.nvidia.com (172.20.187.10) with Microsoft SMTP Server (TLS) id
 15.0.1473.3 via Frontend Transport; Sun, 4 Oct 2020 09:20:12 +0000
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=nwQajPQBG2leKqq9HCsHbUprF0beWs9ghXuTVdMkun2b0rOPhJKf2xokMqY+jyc155e+mZPZGzcbtq1wP5i//SrgAM5t7xiSFhgcFv9E4o8BkhQQqCsSAFW4udiBZnRpSAY0MyU7IXCeV1/qlvYlFdZZp74HpMd33yuwyS3QD7gn9cbMn5wtNIDmjl8Gmdl61S4DNsxAVRH1qfy9lluuTJeyw2y12/qSz7NNr9wNeWhA0qYP6uTNf2KelyN0Mhopcl14e7T3ZfgpKUiFyRbvs9GsG6PYHT26EPQeEFJWARKSRLzwoM8nYMdMYBm2KlYrGc+aX8gEGE/9tgANE6xDLA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; 
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=ZhQjXBoWNyajyzrFkVZQHEE5zsg4d3xJhujDBuEpHGk=;
 b=lYHuCSVYE5WEA2yPwoJYe7lH40ddxWyh8rxhyrlfMWscjho0x/y36E7sMA2bCjWTGFm0k1RlBNLUpb03iynoJyTjW9alkhpqTojZG8UVkh0jaLID+vL4ISzXcA0TEBh/ImpLmWBCP4kb1GkRQR/vuNX01fRmdtmXmOWgRokU0zxaD7Onif9P4oiK1SPPyi7blzL3K5nvLkopwlGkMjZkRPsGHexE8UUGdKxHrvLXgUdWvdfH1dIDucDE3FuEW7njhJFG1zS0Nmb1ya9Q6duAK+XW4VXH4HOhQY3HV0rdYFBkhbC+OARlk77++gfDCAcH3Ke0Exi1Bce/zUR5Vo0jCQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com;
 dkim=pass header.d=nvidia.com; arc=none
Received: from MN2PR12MB4286.namprd12.prod.outlook.com (2603:10b6:208:199::22)
 by MN2PR12MB4637.namprd12.prod.outlook.com (2603:10b6:208:3e::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3433.37; Sun, 4 Oct
 2020 09:20:10 +0000
Received: from MN2PR12MB4286.namprd12.prod.outlook.com
 ([fe80::61fd:a36e:cf4f:2d3f]) by MN2PR12MB4286.namprd12.prod.outlook.com
 ([fe80::61fd:a36e:cf4f:2d3f%8]) with mapi id 15.20.3433.042; Sun, 4 Oct 2020
 09:20:10 +0000
From: Ori Kam <orika@nvidia.com>
To: Bing Zhao <bingz@nvidia.com>, NBU-Contact-Thomas Monjalon
 <thomas@monjalon.net>, "ferruh.yigit@intel.com" <ferruh.yigit@intel.com>,
 "arybchenko@solarflare.com" <arybchenko@solarflare.com>, "mdr@ashroe.eu"
 <mdr@ashroe.eu>, "nhorman@tuxdriver.com" <nhorman@tuxdriver.com>,
 "bernard.iremonger@intel.com" <bernard.iremonger@intel.com>,
 "beilei.xing@intel.com" <beilei.xing@intel.com>, "wenzhuo.lu@intel.com"
 <wenzhuo.lu@intel.com>
CC: "dev@dpdk.org" <dev@dpdk.org>
Thread-Topic: [PATCH 1/4] ethdev: add hairpin bind and unbind APIs
Thread-Index: AQHWl4mIK5rf2s6GGEWFCIE+TKZNUKmHI/vA
Date: Sun, 4 Oct 2020 09:20:10 +0000
Message-ID: <MN2PR12MB428657D6959A2F3B0719D58CD60F0@MN2PR12MB4286.namprd12.prod.outlook.com>
References: <1600012140-70151-1-git-send-email-bingz@nvidia.com>
 <1601511962-21532-1-git-send-email-bingz@nvidia.com>
 <1601511962-21532-2-git-send-email-bingz@nvidia.com>
In-Reply-To: <1601511962-21532-2-git-send-email-bingz@nvidia.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: nvidia.com; dkim=none (message not signed)
 header.d=none;nvidia.com; dmarc=none action=none header.from=nvidia.com;
x-originating-ip: [147.236.152.129]
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 551af6d2-87a3-478a-9426-08d86846b12e
x-ms-traffictypediagnostic: MN2PR12MB4637:
x-ld-processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr
x-ms-exchange-transport-forked: True
x-microsoft-antispam-prvs: <MN2PR12MB4637FEA34B253A47F09A7117D60F0@MN2PR12MB4637.namprd12.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:8882;
x-ms-exchange-senderadcheck: 1
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 4nM/lMvPLu59gxlD2cFfIER6uXOE5G6oF1iqRTEsozNRAN3IuVfdEROtVEq5+MVMfj3yC6tTgXYJuaw3BYP7Lptj57a6KynB/GZyv9m21PuxzfKd107JS01Bjh5pHVUUGgMbn5G+Y7KDGXDRD+C0S1Wsjn/rULKzvgwovTAxSD0r3vaHHiKddWkRGZW7LahE+NQ4JLruHE+MRvFyv+91SuPaw8LEPqFv6z9HmlrhYtE7BcBBwLA03vmSS6/Yu3wvSiaAmx/9U8NKAWtHl3VRTcSId/Ya9UdYg3S9EuLbgfmA2oTekQH6uMKX/RPL5KITQEJK6IlXCNeW5SKSpgTvzw==
x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:;
 IPV:NLI; SFV:NSPM; H:MN2PR12MB4286.namprd12.prod.outlook.com; PTR:; CAT:NONE;
 SFS:(4636009)(366004)(396003)(136003)(376002)(39850400004)(346002)(55016002)(8676002)(8936002)(478600001)(33656002)(186003)(26005)(30864003)(2906002)(83380400001)(9686003)(110136005)(5660300002)(52536014)(316002)(7696005)(71200400001)(4326008)(66476007)(66556008)(66446008)(64756008)(86362001)(66946007)(76116006)(6506007);
 DIR:OUT; SFP:1101; 
x-ms-exchange-antispam-messagedata: xeam9XcRejQRFaKjMnPBZ3PZDb1MeFvyesOGDgbFVFeuSWgY3wg4xwIsJGUKaxhH86m0Jts7VATZLlCaQzDLy8mxDfFC+yZ5fOZvG88iATnCDZ6o/Bepl57vT8+6eS7ZCKP/q48B1sR+c1kkdVe4F9ThkSkn34r9MNcs4ErBHQApR8Z+VvGq5+zVg3YfCQ+Im/Tt8EwXRsxAgz0/owltXt7Gt+zHRmgVc84v3+jApvawbtkJaRP9E6cM+aIyfxqyiCzIW99EkVjzhW2rRIoHPkErhw14A3YKrWAiX/kJ5QNlUdo7FCUkqiVmMRyexgO5uB62eSFVWqkvlVvb0r4VjmFy7WCZx30p6T/fu8fnHvGRJ6kB8xwtSqmRnpezc8RynDTZp39s2lNtIr6QhLe5ZSK44tWvKbpFkL74T9gJDsYNdFg3y/6d7NxdjR3WY3eOt2FuCyZVKv0QgJfV3pcukY7WJnnHXyaravCwdKFS7wcNsmfWrFjyz2HH4KOtXY1mR6JciXpeBNn/Cwpnq1IqbLZJcrsmSVMzQ44JLW/7MfMhEnVaBEFCjgU0DapMhUx4iwVBBSfQFoTX2eHeyGdhCWQgyTKa0bW2qiR6sScnTRAA+WFazYx7pnxAHQcQYcybIYEbApUstjcSXzzL23MWZw==
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4286.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 551af6d2-87a3-478a-9426-08d86846b12e
X-MS-Exchange-CrossTenant-originalarrivaltime: 04 Oct 2020 09:20:10.1469 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: u0KajzBfZX3m5sTTtQEG3jbOaV1Gw1oRpCzEEqZH0EehdEaYSrQNpPAbaxU9w4dQmh7iS2BA3E24Y3IwzqZmRw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4637
X-OriginatorOrg: Nvidia.com
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nvidia.com; s=n1;
 t=1601803110; bh=ZhQjXBoWNyajyzrFkVZQHEE5zsg4d3xJhujDBuEpHGk=;
 h=ARC-Seal:ARC-Message-Signature:ARC-Authentication-Results:From:To:
 CC:Subject:Thread-Topic:Thread-Index:Date:Message-ID:References:
 In-Reply-To:Accept-Language:Content-Language:X-MS-Has-Attach:
 X-MS-TNEF-Correlator:authentication-results:x-originating-ip:
 x-ms-publictraffictype:x-ms-office365-filtering-correlation-id:
 x-ms-traffictypediagnostic:x-ld-processed:
 x-ms-exchange-transport-forked:x-microsoft-antispam-prvs:
 x-ms-oob-tlc-oobclassifiers:x-ms-exchange-senderadcheck:
 x-microsoft-antispam:x-microsoft-antispam-message-info:
 x-forefront-antispam-report:x-ms-exchange-antispam-messagedata:
 Content-Type:Content-Transfer-Encoding:MIME-Version:
 X-MS-Exchange-CrossTenant-AuthAs:
 X-MS-Exchange-CrossTenant-AuthSource:
 X-MS-Exchange-CrossTenant-Network-Message-Id:
 X-MS-Exchange-CrossTenant-originalarrivaltime:
 X-MS-Exchange-CrossTenant-fromentityheader:
 X-MS-Exchange-CrossTenant-id:X-MS-Exchange-CrossTenant-mailboxtype:
 X-MS-Exchange-CrossTenant-userprincipalname:
 X-MS-Exchange-Transport-CrossTenantHeadersStamped:X-OriginatorOrg;
 b=YUX/87n7mlpGY4VqSTDilFK30HqPoKltpT1iS7Yu0UYX+6u7HblA+7M7aQPeQbB8j
 uxylI8acCX0JTTv9HxWCCMZNOrLlgt6bsvP97Cd2kMFLLUIDq7akBWMVISsO7ixPoq
 ak53JlD+z34lJwqQz77czy/FpyErjk0mCik36ocBfVq8WQfg1hdJ+Y/2GedSFz5H4h
 YBt56e02awyDsuYDjs1WDQvdw7lbKot/U8myTKSPeNpwRYZiz5/mJPbgmZv0TrOK6S
 7lD4XyFwS1J1bqbAifgeDbyssep9Gdugl9OyusT1RNp8ARattnnQGAQrlIS3Y8Q8KU
 UFOaMXEzBXzFw==
Subject: Re: [dpdk-dev] [PATCH 1/4] ethdev: add hairpin bind and unbind APIs
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Sender: "dev" <dev-bounces@dpdk.org>

Hi Bing,

PSB,

Thanks,
Ori
> -----Original Message-----
> From: Bing Zhao <bingz@nvidia.com>
> Sent: Thursday, October 1, 2020 3:26 AM
> Cc: dev@dpdk.org
> Subject: [PATCH 1/4] ethdev: add hairpin bind and unbind APIs
>=20
> In single port hairpin mode, all the hairpin TX and RX queues belong
> to the same device. After the queues are set up properly, there is
> no other dependency between the TX queue and its RX peer queue. The
> binding process that connected the TX and RX queues together from
> hardware level will be done automatically during the device start
> procedure. Everything required is configured and initialized already
> for the binding process.
>=20
> But in two ports hairpin mode, there will be some cross-dependences
> between two different ports. Usually, the ports will be initialized
> serially by the main thread but not in parallel. The earlier port
> will not be able to enable the bind if the following peer port is
> not yet configured with HW resources. What's more, if one port is
> detached / attached dynamically, it would introduce more trouble
> for the hairpin binding.
>=20
> To overcome these, new APIs for binding and unbinding are added.
> During startup, only the hairpin TX and RX peer queues will be set
> up. Nothing will be done when starting the device if the queues are
> without auto-bind attribute. Only after the required ports pair
> started, the `rte_eth_hairpin_bind()` API can be called to bind the
> all TX queues of the egress port to the RX queues of the peer port.
> Then the connection between the egress and ingress ports pair will
> be established.
>=20
> The `rte_eth_hairpin_unbind()` API could be used to disconnect the
> egress and the peer ingress ports. This should only be called before
> the device is closed if needed. When doing the clean up, all the
> egress and ingress pairs related to a single port should be taken
> into consideration.
>=20
> Signed-off-by: Bing Zhao <bingz@nvidia.com>
> ---
>  lib/librte_ethdev/rte_ethdev.c           | 107
> +++++++++++++++++++++++++++++++
>  lib/librte_ethdev/rte_ethdev.h           |  51 +++++++++++++++
>  lib/librte_ethdev/rte_ethdev_driver.h    |  52 +++++++++++++++
>  lib/librte_ethdev/rte_ethdev_version.map |   2 +
>  4 files changed, 212 insertions(+)
>=20
> diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethde=
v.c
> index dfe5c1b..72f567b 100644
> --- a/lib/librte_ethdev/rte_ethdev.c
> +++ b/lib/librte_ethdev/rte_ethdev.c
> @@ -2175,6 +2175,113 @@ rte_eth_tx_hairpin_queue_setup(uint16_t port_id,
> uint16_t tx_queue_id,
>  	return eth_err(port_id, ret);
>  }
>=20
> +int
> +rte_eth_hairpin_bind(uint16_t tx_port, uint16_t rx_port)
> +{
> +	struct rte_eth_dev *dev;
> +	struct rte_eth_dev *rdev;
> +	uint16_t p;
> +	uint16_t rp;
> +	int ret =3D 0;
> +
> +	RTE_ETH_VALID_PORTID_OR_ERR_RET(tx_port, -EINVAL);
> +	dev =3D &rte_eth_devices[tx_port];
> +	if (!dev->data->dev_started) {
> +		RTE_ETHDEV_LOG(ERR, "TX port %d is not started", tx_port);
> +		return -EBUSY;
> +	}
> +
> +	/*
> +	 * If the all the ports probed belong to two or more separate NICs, it
> +	 * is recommended that each pair is bound independently but not in the
> +	 * loop to bind all ports.
> +	 */

I don't understand your comment.=20

> +	if (rx_port =3D=3D RTE_MAX_ETHPORTS) {

I think maybe this should be done in the tx queue. Since if the bind don't =
need some port why do
we care if it is started?
So either add a new function to get all peer ports from the tx port, or mov=
e this logic to the=20
Target PMD.

> +		RTE_ETH_FOREACH_DEV(p) {
> +			rdev =3D &rte_eth_devices[p];
> +			if (!rdev->data->dev_started) {
> +				RTE_ETHDEV_LOG(ERR,
> +					       "RX port %d is not started", p);
> +				ret =3D -EBUSY;
> +				goto unbind;
> +			}
> +			ret =3D (*dev->dev_ops->hairpin_bind)(dev, p);
> +			if (ret) {
> +				RTE_ETHDEV_LOG(ERR, "Failed to bind hairpin
> TX "
> +					       "%d to RX %d", tx_port, p);
> +				goto unbind;
> +			}
> +		}
> +	} else {
> +		RTE_ETH_VALID_PORTID_OR_ERR_RET(rx_port, -EINVAL);
> +		rdev =3D &rte_eth_devices[rx_port];
> +		if (!rdev->data->dev_started) {
> +			RTE_ETHDEV_LOG(ERR,
> +				       "RX port %d is not started", rx_port);
> +			return -EBUSY;
> +		}
> +		ret =3D (*dev->dev_ops->hairpin_bind)(dev, rx_port);
> +		if (ret)
> +			RTE_ETHDEV_LOG(ERR, "Failed to bind hairpin TX %d "
> +				       "to RX %d", tx_port, rx_port);
> +	}
> +
> +	return ret;
> +
> +unbind:
> +	/* Roll back the previous binding process. */
> +	RTE_ETH_FOREACH_DEV(rp) {
> +		if (rp < p)
> +			(*dev->dev_ops->hairpin_unbind)(dev, rp);
> +		else
> +			break;
> +	}
> +	return ret;
> +}
> +
> +int
> +rte_eth_hairpin_unbind(uint16_t tx_port, uint16_t rx_port)
> +{
> +	struct rte_eth_dev *dev;
> +	struct rte_eth_dev *rdev;
> +	uint16_t p;
> +	int ret =3D 0;
> +
> +	RTE_ETH_VALID_PORTID_OR_ERR_RET(tx_port, -EINVAL);
> +	dev =3D &rte_eth_devices[tx_port];
> +	if (!dev->data->dev_started) {
> +		RTE_ETHDEV_LOG(ERR, "TX port %d is stopped", tx_port);
> +		return -EBUSY;
> +	}
> +
> +	if (rx_port =3D=3D RTE_MAX_ETHPORTS) {
> +		RTE_ETH_FOREACH_DEV(p) {
> +			rdev =3D &rte_eth_devices[p];
> +			if (!rdev->data->dev_started) {

This condition should never be true.
First see my comment above about the list of devices, second port should fa=
il to
stop if it is bounded.

> +				RTE_ETHDEV_LOG(ERR, "RX port %d is
> stopped", p);
> +				ret =3D -EBUSY;
> +				break;
> +			}
> +			ret =3D (*dev->dev_ops->hairpin_unbind)(dev, p);
> +			if (ret) {
> +				RTE_ETHDEV_LOG(ERR, "Failed to unbind
> hairpin "
> +					       "TX %d from RX %d", tx_port, p);
> +				break;
> +			}
> +		}
> +	} else {
> +		RTE_ETH_VALID_PORTID_OR_ERR_RET(rx_port, -EINVAL);
> +		rdev =3D &rte_eth_devices[rx_port];
> +		if (!rdev->data->dev_started) {
> +			RTE_ETHDEV_LOG(ERR, "RX port %d is stopped",
> rx_port);
> +			return -EBUSY;
> +		}
> +		ret =3D (*dev->dev_ops->hairpin_unbind)(dev, rx_port);
> +	}
> +
> +	return ret;
> +}
> +
>  void
>  rte_eth_tx_buffer_drop_callback(struct rte_mbuf **pkts, uint16_t unsent,
>  		void *userdata __rte_unused)
> diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethde=
v.h
> index 645a186..c3fb684 100644
> --- a/lib/librte_ethdev/rte_ethdev.h
> +++ b/lib/librte_ethdev/rte_ethdev.h
> @@ -2133,6 +2133,57 @@ int rte_eth_tx_hairpin_queue_setup
>  	 const struct rte_eth_hairpin_conf *conf);
>=20
>  /**
> + * @warning
> + * @b EXPERIMENTAL: this API may change, or be removed, without prior
> notice
> + *
> + * Bind all hairpin TX queues of one port to the RX queues of the peer p=
ort.
> + * It is only allowed to call this API after all hairpin queues are conf=
igured
> + * properly and the devices of TX and peer RX are in started state.
> + *
> + * @param tx_port
> + *   The TX port identifier of the Ethernet device.
> + * @param rx_port
> + *   The peer RX port identifier of the Ethernet device.
> + *   RTE_MAX_ETHPORTS is allowed for the traversal of all devices.
> + *   RX port ID could have the same value with TX port ID.
> + *
> + * @return
> + *   - (0) if successful.
> + *   - (-EINVAL) if bad parameter.
> + *   - (-EBUSY) if device is not in started state.
> + *   - (-ENOTSUP) if hardware doesn't support.
> + *   - Others detailed errors from PMD drivers.
> + */
> +__rte_experimental
> +int rte_eth_hairpin_bind(uint16_t tx_port, uint16_t rx_port);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change, or be removed, without prior
> notice
> + *
> + * Unbind all hairpin TX queues of one port from the RX queues of the pe=
er
> port.
> + * This should be called before closing the TX or RX devices (optional).=
 After
> + * unbind the hairpin ports pair, it is allowed to bind them again.
> + * Changing queues configuration should be after stopping the device.
> + *
> + * @param tx_port
> + *   The TX port identifier of the Ethernet device.
> + * @param rx_port
> + *   The peer RX port identifier of the Ethernet device.
> + *   RTE_MAX_ETHPORTS is allowed for traversal of all devices.
> + *   RX port ID could have the same value with TX port ID.
> + *
> + * @return
> + *   - (0) if successful.
> + *   - (-EINVAL) if bad parameter.
> + *   - (-EBUSY) if device is in stopped state.
> + *   - (-ENOTSUP) if hardware doesn't support.
> + *   - Others detailed errors from PMD drivers.
> + */
> +__rte_experimental
> +int rte_eth_hairpin_unbind(uint16_t tx_port, uint16_t rx_port);
> +
> +/**
>   * Return the NUMA socket to which an Ethernet device is connected
>   *
>   * @param port_id
> diff --git a/lib/librte_ethdev/rte_ethdev_driver.h
> b/lib/librte_ethdev/rte_ethdev_driver.h
> index 04ac8e9..910433f 100644
> --- a/lib/librte_ethdev/rte_ethdev_driver.h
> +++ b/lib/librte_ethdev/rte_ethdev_driver.h
> @@ -575,6 +575,54 @@ typedef int (*eth_tx_hairpin_queue_setup_t)
>  	 const struct rte_eth_hairpin_conf *hairpin_conf);
>=20
>  /**
> + * @internal
> + * Bind all hairpin TX queues of one port to the RX queues of the peer p=
ort.
> + *
> + * @param dev
> + *   ethdev handle of port.
> + * @param rx_port
> + *   the peer RX port.
> + *
> + * @return
> + *   Negative errno value on error, 0 on success.
> + *
> + * @retval 0
> + *   Success, bind successfully.
> + * @retval -ENOTSUP
> + *   Bind API is not supported.
> + * @retval -EINVAL
> + *   One of the parameters is invalid.
> + * @retval -EBUSY
> + *   Device is not started.
> + */
> +typedef int (*eth_hairpin_bind_t)(struct rte_eth_dev *dev,
> +				uint16_t rx_port);
> +
> +/**
> + * @internal
> + * Unbind all hairpin TX queues of one port from the RX queues of the pe=
er
> port.
> + *
> + * @param dev
> + *   ethdev handle of port.
> + * @param rx_port
> + *   the peer RX port.
> + *
> + * @return
> + *   Negative errno value on error, 0 on success.
> + *
> + * @retval 0
> + *   Success, bind successfully.
> + * @retval -ENOTSUP
> + *   Bind API is not supported.
> + * @retval -EINVAL
> + *   One of the parameters is invalid.
> + * @retval -EBUSY
> + *   Device is already stopped.
> + */
> +typedef int (*eth_hairpin_unbind_t)(struct rte_eth_dev *dev,
> +				  uint16_t rx_port);
> +
> +/**
>   * @internal A structure containing the functions exported by an Etherne=
t
> driver.
>   */
>  struct eth_dev_ops {
> @@ -713,6 +761,10 @@ struct eth_dev_ops {
>  	/**< Set up device RX hairpin queue. */
>  	eth_tx_hairpin_queue_setup_t tx_hairpin_queue_setup;
>  	/**< Set up device TX hairpin queue. */
> +	eth_hairpin_bind_t hairpin_bind;
> +	/**< Bind all hairpin TX queues of device to the peer port RX queues. *=
/
> +	eth_hairpin_unbind_t hairpin_unbind;
> +	/**< Unbind all hairpin TX queues from the peer port RX queues. */
>  };
>=20
>  /**
> diff --git a/lib/librte_ethdev/rte_ethdev_version.map
> b/lib/librte_ethdev/rte_ethdev_version.map
> index c95ef51..18efe4e 100644
> --- a/lib/librte_ethdev/rte_ethdev_version.map
> +++ b/lib/librte_ethdev/rte_ethdev_version.map
> @@ -227,6 +227,8 @@ EXPERIMENTAL {
>  	rte_tm_wred_profile_delete;
>=20
>  	# added in 20.11
> +	rte_eth_hairpin_bind;
> +	rte_eth_hairpin_unbind;
>  	rte_eth_link_speed_to_str;
>  	rte_eth_link_to_str;
>  };
> --
> 2.5.5