From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3232AA04B7; Sun, 4 Oct 2020 11:35:09 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 335651BD41; Sun, 4 Oct 2020 11:35:07 +0200 (CEST) Received: from nat-hk.nvidia.com (nat-hk.nvidia.com [203.18.50.4]) by dpdk.org (Postfix) with ESMTP id A23AA1BD06 for ; Sun, 4 Oct 2020 11:35:04 +0200 (CEST) Received: from HKMAIL101.nvidia.com (Not Verified[10.18.92.77]) by nat-hk.nvidia.com (using TLS: TLSv1.2, AES256-SHA) id ; Sun, 04 Oct 2020 17:35:01 +0800 Received: from HKMAIL102.nvidia.com (10.18.16.11) by HKMAIL101.nvidia.com (10.18.16.10) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Sun, 4 Oct 2020 09:34:57 +0000 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (104.47.55.176) by HKMAIL102.nvidia.com (10.18.16.11) with Microsoft SMTP Server (TLS) id 15.0.1473.3 via Frontend Transport; Sun, 4 Oct 2020 09:34:57 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=O6VqDnU69jT21Ht5jxfMJLv2M4OLloSZfXMj+EsKUtUol+bF9Iwt17XMsKpsNJTlpvnyNXlfltXwfSqlNoWzOdUn357xjI9rdQ5mXVer8zRxLDMIJGwx2QrxPQg60onk0yA6pxwgiXgO4ddSzRB1gDp3CNgPEW65PzS+nnlrG/uR7QT9S/Oyr6Z2sSIT+K7XbiFTYXS+ShMSpQLJa79t3JI098RGH0XTvQVrnjScKhFMtYQY3CDmm6FWaRbEBi+m/RK3ATBpfUhP1IiCWn41wPzPE5LkGfk7bLh192adhkuSy12PmSsah/HDDwi/JoYjjb5Gx+iJ39HDIlxfvS9DyA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=kkyZe3ahtLupmAgCOmEUYBGQKOOzxdU+O/kfR7MQ2/o=; b=V0qG6QuuuEbGHyfr/RBHXhwRYSyFQn9LMWyM773rU5hzLNy5gwHfLHiN1TDfATb9UsN6Nv7qCJi8sxZpIdr6I/0D9+NEdeXKZ+ghLQ2CXNPuos5B6nmSPuQYuAnq3IpP82K2GknaWF7jhNN+ci3qBgCAFLIxSIC3tKZMA/1L5J39xSOcTcnWegEjBPqqAbcTRs6r+5dyT3QG7P4z/wnfVlSdY+eL7dzhkJfQ6c5JiIqbEgQszQtfHcjjHKLAZPofGsjb1JupMRpgmeONSjfjeMt7hoildcgdopFkerrqnnK7NGG8tmjyB/0iQONgursCCRuLi+J6L5n9RXu5NbcAlA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none Received: from MN2PR12MB4286.namprd12.prod.outlook.com (2603:10b6:208:199::22) by MN2PR12MB4471.namprd12.prod.outlook.com (2603:10b6:208:26f::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3433.32; Sun, 4 Oct 2020 09:34:55 +0000 Received: from MN2PR12MB4286.namprd12.prod.outlook.com ([fe80::61fd:a36e:cf4f:2d3f]) by MN2PR12MB4286.namprd12.prod.outlook.com ([fe80::61fd:a36e:cf4f:2d3f%8]) with mapi id 15.20.3433.042; Sun, 4 Oct 2020 09:34:55 +0000 From: Ori Kam To: Bing Zhao , NBU-Contact-Thomas Monjalon , "ferruh.yigit@intel.com" , "arybchenko@solarflare.com" , "mdr@ashroe.eu" , "nhorman@tuxdriver.com" , "bernard.iremonger@intel.com" , "beilei.xing@intel.com" , "wenzhuo.lu@intel.com" CC: "dev@dpdk.org" Thread-Topic: [PATCH 3/4] ethdev: add APIs for hairpin queue operation Thread-Index: AQHWl4m0ZVl95ugDIUGLKV/F4J/z9KmHMWtA Date: Sun, 4 Oct 2020 09:34:55 +0000 Message-ID: References: <1600012140-70151-1-git-send-email-bingz@nvidia.com> <1601511962-21532-1-git-send-email-bingz@nvidia.com> <1601511962-21532-4-git-send-email-bingz@nvidia.com> In-Reply-To: <1601511962-21532-4-git-send-email-bingz@nvidia.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: nvidia.com; dkim=none (message not signed) header.d=none;nvidia.com; dmarc=none action=none header.from=nvidia.com; x-originating-ip: [147.236.152.129] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 64f366da-6090-45e2-38d4-08d86848c0a5 x-ms-traffictypediagnostic: MN2PR12MB4471: x-ld-processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:8882; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 3iGYlbcbS+HgTSHCvFDrGQB1g7v7bWIwyZjuhYazABv5muSSwHauTz9WnXg44kuw+KLog7AtP8mg6KhTuLaCTme9qSgk1+vxH2JJMNswN2aYyTdAxU0e8TXcIM25THYOl4IOJdHvlA2RVD7bmqJaBY4r8eRzi6ccIbgKbaaAj146Jmvf/fhMDmqNoQ+E1kucvBPmOLDolZjsGvDLK8pkY9wvgl7xweTl3K9GOwesaUTCaXdVVZpTYKxqD0Dsw1j6izzsCHzOJNUdKlBiQagzJvCYg+qSBL8/RmHW7UBm995jJOV3AbF/j4OqzqenRe/ZxDP4AFCDzneD+txSKCuOGg== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MN2PR12MB4286.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(346002)(366004)(39850400004)(136003)(376002)(396003)(316002)(9686003)(83380400001)(4326008)(53546011)(110136005)(6506007)(52536014)(76116006)(26005)(66556008)(66446008)(64756008)(66946007)(33656002)(66476007)(5660300002)(7696005)(478600001)(2906002)(71200400001)(186003)(8936002)(55016002)(86362001)(8676002); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: t9+fIdgC1Ml+P7boIvVPH7ckRZjbN7qCewo2VD2bgkSkwKCoQqO3Zky/bDzIo/bTrOTJpKkJiWx5tKXMdwYEtX79dTG7lxMl7upyE9pLWwbYdlimw700g/u4BO+UeWSJakZGyITeQn4KZ/utWOkf8PSsDP1oHMjAsimeHXvM/0MC3t0cqefihX6LZY4XgPVPbJ/BCzq5UesIPkE2DiWMnQFDFNx2Z8a3HA6v2Qt4bmI3t8oJDw4kWT5kT1MVdRYACz3XsrlyL2VVC8zHANKGhutdwt3VWuDESktcrXIVZIDXA9hYAa59dBT/tA3OI9OCX0UL8yc0vwXhaC77wnBnH3hjEthv7SKfm+e4v4gEYHyETHIpaezQL+Lfnl3FwkzQV6MIWCylNw0XcKotQ0rdOUiFCiVoBnnsaSQtpbM4FMVwsWMQNquz34M2otq54RCHBlRf9erVOhtdP1fmtx6GnWTO0j3+wBBgpY92VM6UtI6FtHcvRwSGPofsgJHPdoC2clSlcoafGtRBkEVFPbtGGDi0szunchdsB0E5HgDF4p/yJWEj5YIPevNVWqdsq3DkHQptA3Vq75QIcjIbdrRVAkBzghUQCN0BOhEN51H6dV2MPsM6/YjAy+aew4Ww/PVOYyPdAgOgex0zh3ZkkzMp4A== Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4286.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 64f366da-6090-45e2-38d4-08d86848c0a5 X-MS-Exchange-CrossTenant-originalarrivaltime: 04 Oct 2020 09:34:55.1215 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: qtHMK72BBru9RD4b0lQvDBvMYJd+X5OrGfiN0NtHbV6emJwfGaXu22pb0yGmGhK1to3yukHrtH/MS04iwIZ+vA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4471 X-OriginatorOrg: Nvidia.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nvidia.com; s=n1; t=1601804102; bh=kkyZe3ahtLupmAgCOmEUYBGQKOOzxdU+O/kfR7MQ2/o=; h=ARC-Seal:ARC-Message-Signature:ARC-Authentication-Results:From:To: CC:Subject:Thread-Topic:Thread-Index:Date:Message-ID:References: In-Reply-To:Accept-Language:Content-Language:X-MS-Has-Attach: X-MS-TNEF-Correlator:authentication-results:x-originating-ip: x-ms-publictraffictype:x-ms-office365-filtering-correlation-id: x-ms-traffictypediagnostic:x-ld-processed: x-ms-exchange-transport-forked:x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers:x-ms-exchange-senderadcheck: x-microsoft-antispam:x-microsoft-antispam-message-info: x-forefront-antispam-report:x-ms-exchange-antispam-messagedata: Content-Type:Content-Transfer-Encoding:MIME-Version: X-MS-Exchange-CrossTenant-AuthAs: X-MS-Exchange-CrossTenant-AuthSource: X-MS-Exchange-CrossTenant-Network-Message-Id: X-MS-Exchange-CrossTenant-originalarrivaltime: X-MS-Exchange-CrossTenant-fromentityheader: X-MS-Exchange-CrossTenant-id:X-MS-Exchange-CrossTenant-mailboxtype: X-MS-Exchange-CrossTenant-userprincipalname: X-MS-Exchange-Transport-CrossTenantHeadersStamped:X-OriginatorOrg; b=h9686rYrbxZHnejc+jHOmQm3j1dklGndGbbWyOntYA5uVUyjYe746efUV92Qbxna2 fGS4b1vPKU01iXSxcSY88yHtUi1E1yzSZjIs3RTKcgJqOryqmx2mhUPQ5gr1UKMMtn 1bM49GxzRE8mXNCzrfJNAU4ojl5vhwjpBtZ5MBUM/szZ7x1h5n5en/pIiJu9nOB0WU 3tGM63R6GcVy78DMAd0zVB/jxtOJwiZ5zaYmlnexPodw7ws6XQg4HLljWP5SKlqR/C U9N7nyvCBYqsuK3YPYEuWokKIBLOaTnvWN5Jfr2nEIne1ZhjdsxgHdSQmHc8eVLbfB YKlsR3BaSjhjg== Subject: Re: [dpdk-dev] [PATCH 3/4] ethdev: add APIs for hairpin queue operation X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Bing, PSB Best, Ori > -----Original Message----- > From: Bing Zhao > Sent: Thursday, October 1, 2020 3:26 AM > Subject: [PATCH 3/4] ethdev: add APIs for hairpin queue operation >=20 > Every hairpin queue pair should be configured properly and the > connection between TX and RX queues should be established, before > hairpin function works. In single port hairpin mode, the queues of > each pair belong to the same device. It is easy to get the hardware > and software information of each queue and configure the hairpin > connection with such information. In two ports hairpin mode, it is > not easy or inappropriate to access one queue's information from > another device. >=20 > Since hairpin is configured per queue pair, three new APIs are > introduced and they are internal for the PMD using. >=20 > The peer update API helps to pass one queue's information to the > peer queue and get the peer's information back for the next step. > The peer bind API configures the current queue with the peer's > information. For each hairpin queue pair, this API may need to be > called twice to configure the TX, RX queues separately. > The peer unbind API resets the current queue configuraion and state > to disconnect it from the peer queue. Also, it may need to be called > twice to disconnect TX, RX queues from each other. >=20 > Some parameter of the above APIs might not be mandatory, and it > depends on the PMD implementation. >=20 > The structure of `rte_hairpin_peer_info` is only a declaration and > the actual members will be defined in each PMD when being used. >=20 > Signed-off-by: Bing Zhao > --- > lib/librte_ethdev/rte_ethdev.c | 55 ++++++++++++++++ > lib/librte_ethdev/rte_ethdev_driver.h | 108 > +++++++++++++++++++++++++++++++ > lib/librte_ethdev/rte_ethdev_version.map | 3 + > 3 files changed, 166 insertions(+) >=20 > diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethde= v.c > index 72f567b..4bfc26e 100644 > --- a/lib/librte_ethdev/rte_ethdev.c > +++ b/lib/librte_ethdev/rte_ethdev.c > @@ -5515,6 +5515,61 @@ handle_port_link_status(const char *cmd > __rte_unused, > return 0; > } >=20 > +int > +rte_eth_hairpin_queue_peer_update(uint16_t peer_port, uint16_t > peer_queue, > + struct rte_hairpin_peer_info *cur_info, > + struct rte_hairpin_peer_info *peer_info, > + bool direction) > +{ > + struct rte_eth_dev *dev; > + > + /* Current queue information is not mandatory. */ > + if (peer_info =3D=3D NULL) > + return -EINVAL; > + > + /* No need to check the validity again. */ > + dev =3D &rte_eth_devices[peer_port]; > + RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops- > >hairpin_queue_peer_update, > + -ENOTSUP); > + > + return (*dev->dev_ops->hairpin_queue_peer_update)(dev, > peer_queue, > + cur_info, peer_info, direction); > +} > + > +int > +rte_eth_hairpin_queue_peer_bind(uint16_t cur_port, uint16_t cur_queue, > + struct rte_hairpin_peer_info *peer_info, > + bool direction) > +{ > + struct rte_eth_dev *dev; > + > + if (peer_info =3D=3D NULL) > + return -EINVAL; > + > + /* No need to check the validity again. */ > + dev =3D &rte_eth_devices[cur_port]; > + RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops- > >hairpin_queue_peer_bind, > + -ENOTSUP); > + > + return (*dev->dev_ops->hairpin_queue_peer_bind)(dev, cur_queue, > + peer_info, direction); > +} > + > +int > +rte_eth_hairpin_queue_peer_unbind(uint16_t cur_port, uint16_t cur_queue, > + bool direction) > +{ > + struct rte_eth_dev *dev; > + > + /* No need to check the validity again. */ > + dev =3D &rte_eth_devices[cur_port]; > + RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops- > >hairpin_queue_peer_bind, > + -ENOTSUP); > + > + return (*dev->dev_ops->hairpin_queue_peer_unbind)(dev, cur_queue, > + direction); > +} > + > RTE_LOG_REGISTER(rte_eth_dev_logtype, lib.ethdev, INFO); >=20 > RTE_INIT(ethdev_init_telemetry) > diff --git a/lib/librte_ethdev/rte_ethdev_driver.h > b/lib/librte_ethdev/rte_ethdev_driver.h > index 910433f..d759c58 100644 > --- a/lib/librte_ethdev/rte_ethdev_driver.h > +++ b/lib/librte_ethdev/rte_ethdev_driver.h > @@ -21,6 +21,9 @@ > extern "C" { > #endif >=20 > +/**< @internal Declaration of the hairpin peer queue information structu= re. > */ > +struct rte_hairpin_peer_info; > + > /* > * Definitions of all functions exported by an Ethernet driver through t= he > * generic structure of type *eth_dev_ops* supplied in the *rte_eth_dev* > @@ -622,6 +625,21 @@ typedef int (*eth_hairpin_bind_t)(struct rte_eth_dev > *dev, > typedef int (*eth_hairpin_unbind_t)(struct rte_eth_dev *dev, > uint16_t rx_port); >=20 > +typedef int (*eth_hairpin_queue_peer_update_t) > + (struct rte_eth_dev *dev, uint16_t peer_queue, > + struct rte_hairpin_peer_info *current_info, > + struct rte_hairpin_peer_info *peer_info, bool direction); > +/**< @internal Update and fetch peer queue information. */ > + > +typedef int (*eth_hairpin_queue_peer_bind_t) > + (struct rte_eth_dev *dev, uint16_t cur_queue, > + struct rte_hairpin_peer_info *peer_info, bool direction); > +/**< @internal Bind peer queue to the current queue with fetched > information. */ > + > +typedef int (*eth_hairpin_queue_peer_unbind_t) > + (struct rte_eth_dev *dev, uint16_t cur_queue, bool direction); > +/**< @internal Unbind peer queue from the current queue. */ > + > /** > * @internal A structure containing the functions exported by an Etherne= t > driver. > */ > @@ -765,6 +783,9 @@ struct eth_dev_ops { > /**< Bind all hairpin TX queues of device to the peer port RX queues. *= / > eth_hairpin_unbind_t hairpin_unbind; > /**< Unbind all hairpin TX queues from the peer port RX queues. */ > + eth_hairpin_queue_peer_update_t hairpin_queue_peer_update; > + eth_hairpin_queue_peer_bind_t hairpin_queue_peer_bind; > + eth_hairpin_queue_peer_unbind_t hairpin_queue_peer_unbind; > }; >=20 > /** > @@ -1120,6 +1141,93 @@ __rte_internal > int > rte_eth_dev_destroy(struct rte_eth_dev *ethdev, ethdev_uninit_t > ethdev_uninit); >=20 > +/** > + * @warning > + * @b EXPERIMENTAL: this API may change, or be removed, without prior > notice > + * > + * @internal > + * Pass the current hairpin queue HW and/or HW information to the peer > queue I don't think you need experimental tag when using internal. > + * and fetch back the information of the peer queue. > + * > + * @param peer_port > + * Peer port identifier of the Ethernet device. > + * @param peer_queue > + * Peer queue index of the port. > + * @param cur_info > + * Pointer to the current information structure. > + * @param peer_info > + * Pointer to the peer information, output. > + * @param direction > + * Direction to pass the information. > + * true - pass TX queue information and get peer RX queue information > + * false - pass RX queue information and get peer TX queue information > + * > + * @return > + * Negative errno value on error, 0 on success. > + */ > +__rte_internal > +int > +rte_eth_hairpin_queue_peer_update(uint16_t peer_port, uint16_t > peer_queue, > + struct rte_hairpin_peer_info *cur_info, > + struct rte_hairpin_peer_info *peer_info, > + bool direction); > + > +/** > + * @warning > + * @b EXPERIMENTAL: this API may change, or be removed, without prior > notice > + * > + * @internal > + * Configure current hairpin queue with the peer information fetched to > create Same comment as above. > + * the connection (bind) with peer queue in the specified direction. > + * This function might need to be called twice to fully create the conne= ction. > + * > + * @param cur_port > + * Current port identifier of the Ethernet device. > + * @param cur_queue > + * Current queue index of the port. > + * @param peer_info > + * Pointer to the peer information, input. > + * @param direction > + * Direction to create the connection. > + * true - bind current TX queue to peer RX queue > + * false - bind current RX queue to peer TX queue > + * > + * @return > + * Negative errno value on error, 0 on success. > + */ > +__rte_internal > +int > +rte_eth_hairpin_queue_peer_bind(uint16_t cur_port, uint16_t cur_queue, > + struct rte_hairpin_peer_info *peer_info, > + bool direction); > + > +/** > + * @warning > + * @b EXPERIMENTAL: this API may change, or be removed, without prior > notice > + * > + * @internal > + * Reset the current queue state and configuration to disconnect (unbind= ) it > + * from the peer queue. > + * This function might need to be called twice to disconnect each other. > + * > + * @param cur_port > + * Current port identifier of the Ethernet device. > + * @param cur_queue > + * Current queue index of the port. > + * @param direction > + * Direction to create the connection. > + * true - unbind current TX queue from peer RX queue > + * false - unbind current RX queue from peer TX queue > + * > + * @return > + * Negative errno value on error, 0 on success. > + */ > +__rte_internal > +int > +rte_eth_hairpin_queue_peer_unbind(uint16_t cur_port, uint16_t cur_queue, > + bool direction); > + > + > #ifdef __cplusplus > } > #endif > diff --git a/lib/librte_ethdev/rte_ethdev_version.map > b/lib/librte_ethdev/rte_ethdev_version.map > index 18efe4e..d05cd97 100644 > --- a/lib/librte_ethdev/rte_ethdev_version.map > +++ b/lib/librte_ethdev/rte_ethdev_version.map > @@ -250,6 +250,9 @@ INTERNAL { > rte_eth_devargs_parse; > rte_eth_dma_zone_free; > rte_eth_dma_zone_reserve; > + rte_eth_hairpin_queue_peer_bind; > + rte_eth_hairpin_queue_peer_unbind; > + rte_eth_hairpin_queue_peer_update; > rte_eth_switch_domain_alloc; > rte_eth_switch_domain_free; > rte_flow_expand_rss; > -- > 2.5.5