From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2E703A04B7; Sun, 4 Oct 2020 11:39:59 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 78B781BD06; Sun, 4 Oct 2020 11:39:57 +0200 (CEST) Received: from hqnvemgate25.nvidia.com (hqnvemgate25.nvidia.com [216.228.121.64]) by dpdk.org (Postfix) with ESMTP id 2A9101B87A for ; Sun, 4 Oct 2020 11:39:55 +0200 (CEST) Received: from hqmail.nvidia.com (Not Verified[216.228.121.13]) by hqnvemgate25.nvidia.com (using TLS: TLSv1.2, AES256-SHA) id ; Sun, 04 Oct 2020 02:38:59 -0700 Received: from HQMAIL111.nvidia.com (172.20.187.18) by HQMAIL105.nvidia.com (172.20.187.12) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Sun, 4 Oct 2020 09:39:47 +0000 Received: from NAM02-SN1-obe.outbound.protection.outlook.com (104.47.36.51) by HQMAIL111.nvidia.com (172.20.187.18) with Microsoft SMTP Server (TLS) id 15.0.1473.3 via Frontend Transport; Sun, 4 Oct 2020 09:39:47 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=TkoHQr5R0rPjN8IR0yxqEq/n06p4a+OuEn08DMIQGRyKfb4lQx8sfahV/xMZx/evVIvpbWMXnDABBq8f5PbHNBD5VBwJ71SrH4sobL7GWxjziZBn8WaWKW1LVuKe0nacgQosaixNTt6QFPkum996uzjEgkCtXwDQoghEaqM8tGhNqIVc14wWzHAOiwY0mIuH7C+AtQLRhi9dYILvywXJVXuZH15z+DazJxyRVffHc2i/F3KWmrzBlkhkCqedAryf5IDbo7RHDe1Cpq8UPB5zJg+zQwf/G6K+dhKEoadFaam5Bc4O5sWGrAofwTSMf4xa7gyCMG1KrlvASeOy57HQXw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=SpDUGwrpSSY2sjX8LOpEGvlnnq8Ty/fuS3vy7CwrnCc=; b=GsmVO6SW6WYzIcfIxfPS2n4iaQ6jJbmDgNVZj+UAx+5psw8E0V4WoX1MRgOeE9oeNtiwcCbpDLfOJVnGEABDvtpSVHzhtkcn+4zicKeXpNMEhQ4QNO881GdS5/bbk7kWx2H04FNZccdTuZMST54OIfm+U1c8HY1+qiumnLCqbCtIfoT+Ta/XZIt6QIt7TBxDLTu6LJ/ecjPGvyr3x95200mL+C+2fE3hEvRHjULMhEp2+kZvIPzLOOlrZ2jGMyxOGWXdKMGi5dINnjdiM1w9fPJSzPFUzB3NwQtaH8K3xQHrNlxCQTYULU04Vo+RaMiOT0vjP+AfUq7I4ac2o5B6qg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none Received: from MN2PR12MB4286.namprd12.prod.outlook.com (2603:10b6:208:199::22) by BL0PR12MB4866.namprd12.prod.outlook.com (2603:10b6:208:1cf::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3433.35; Sun, 4 Oct 2020 09:39:44 +0000 Received: from MN2PR12MB4286.namprd12.prod.outlook.com ([fe80::61fd:a36e:cf4f:2d3f]) by MN2PR12MB4286.namprd12.prod.outlook.com ([fe80::61fd:a36e:cf4f:2d3f%8]) with mapi id 15.20.3433.042; Sun, 4 Oct 2020 09:39:44 +0000 From: Ori Kam To: Bing Zhao , NBU-Contact-Thomas Monjalon , "ferruh.yigit@intel.com" , "arybchenko@solarflare.com" , "mdr@ashroe.eu" , "nhorman@tuxdriver.com" , "bernard.iremonger@intel.com" , "beilei.xing@intel.com" , "wenzhuo.lu@intel.com" CC: "dev@dpdk.org" Thread-Topic: [PATCH 4/4] app/testpmd: change hairpin queues setup Thread-Index: AQHWl4mPd//wcAKnI0mGC373RIoyn6mHNIow Date: Sun, 4 Oct 2020 09:39:44 +0000 Message-ID: References: <1600012140-70151-1-git-send-email-bingz@nvidia.com> <1601511962-21532-1-git-send-email-bingz@nvidia.com> <1601511962-21532-5-git-send-email-bingz@nvidia.com> In-Reply-To: <1601511962-21532-5-git-send-email-bingz@nvidia.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: nvidia.com; dkim=none (message not signed) header.d=none;nvidia.com; dmarc=none action=none header.from=nvidia.com; x-originating-ip: [147.236.152.129] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: abf6ff52-d256-463c-4507-08d868496d5c x-ms-traffictypediagnostic: BL0PR12MB4866: x-ld-processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:2276; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: H8vLsXEzdMSuoHVrVgEIfZAHWlfxNNs9n9Eu8OjJjvw0y8BKqxIhFsVpF4xHuwfjl+DsCBmwCTMUlFwhNiEIUHziy+HQI9+dUbMmhpMlkOrk67IAS66EzBqhcsMh10uoJzDwxavueuSH4fcjlwMgmbYlqI5NTuTFzsOcV3HoleX5h2usw8Ts0yIxvbDePU2qVCucwwlqdD5p2S+I37NJAqW2IOusFwAsvzenUhRkV/zXDIR1YUlYnRXeoyOLfAHAnNF/TldDaa0Aig1QJC0z4AWZYcrttSvFDcCkPl9wCuF7qpJgYp9zvGCRKxIH1OR/naZf76q1laXGep/cdvlQZFHh94bYeRKGrvOhQQuFT8yrKzeNum2GEV/wz1hoZFTJ x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MN2PR12MB4286.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(39850400004)(346002)(136003)(366004)(396003)(376002)(52536014)(5660300002)(4326008)(110136005)(2906002)(86362001)(316002)(66476007)(7696005)(66946007)(64756008)(66446008)(76116006)(66556008)(53546011)(6506007)(33656002)(9686003)(186003)(8936002)(26005)(83380400001)(478600001)(71200400001)(8676002)(55016002)(309714004); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: qMSTMeli2UeLBNR05EU8EAJF6R/mdT9UFrTSKKiEXLTA6sgqXGUnQt+aHpkz4rFQ3CQNZRS+6tlNrbom0zZLMddVIA0OQ7c9W0YSI0qhcclHIZsphuSPh0Vm49R5aKXgpPPJkaGMjdLcKFKAsq4RPpHcWH7RJ3SPEKp4iglNUHbbNy6YpXHKUSVbXHmQBSl4YPrvakyfUL5S3TR4sdDfaxWBHhtsVgvPn+O1q6NVK/9MZFFbINpXMmjiLQHuGE6o00BqLweyv0zJvfptxZ6M+rD+N9yMPuey8n2NqbsdU/xMDrgHToRzEJLNu9iF5NbcaPwE5MWTBQ7Usd2eeLaWpB3u5r0dvKP0VHSYscHLYJaA1fTcm/+uYvov5w1eCqmK3F73/t0Fgfh8A/BL6S5EWJ9WmIEharCkGK25SxR9A4Rxwm0IQ5oeZuokOd1qkzjHE26E6awyLyehleWM6XvW1lLvbJYmAYsbGQH4mF/Z/9g7QKr6SNy+nhtluFcDf9SOj27AfEzMSvs5urzbqfU8FuA7j5zq+9sM84l7f8EzOvj9wFMwd4+MxI0xGt1QXK/f+9ZWbFXrhzL0jBjTuoxyc0gs9eWKhCYjG79L3WFATOU58s6sZ2lH0YdoiYT286cyyJ0dxaVseJBwhVl3vO47+w== Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4286.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: abf6ff52-d256-463c-4507-08d868496d5c X-MS-Exchange-CrossTenant-originalarrivaltime: 04 Oct 2020 09:39:44.8873 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: Dn/p+5D2hZBjzQPNzgMDmHtlyRI0EfDfXeofzoFTST4mRO5nV6ly7vVbiRlrxU2PuTM9rUI6L4SiA3fFlT1d2g== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL0PR12MB4866 X-OriginatorOrg: Nvidia.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nvidia.com; s=n1; t=1601804339; bh=SpDUGwrpSSY2sjX8LOpEGvlnnq8Ty/fuS3vy7CwrnCc=; h=ARC-Seal:ARC-Message-Signature:ARC-Authentication-Results:From:To: CC:Subject:Thread-Topic:Thread-Index:Date:Message-ID:References: In-Reply-To:Accept-Language:Content-Language:X-MS-Has-Attach: X-MS-TNEF-Correlator:authentication-results:x-originating-ip: x-ms-publictraffictype:x-ms-office365-filtering-correlation-id: x-ms-traffictypediagnostic:x-ld-processed: x-ms-exchange-transport-forked:x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers:x-ms-exchange-senderadcheck: x-microsoft-antispam:x-microsoft-antispam-message-info: x-forefront-antispam-report:x-ms-exchange-antispam-messagedata: Content-Type:Content-Transfer-Encoding:MIME-Version: X-MS-Exchange-CrossTenant-AuthAs: X-MS-Exchange-CrossTenant-AuthSource: X-MS-Exchange-CrossTenant-Network-Message-Id: X-MS-Exchange-CrossTenant-originalarrivaltime: X-MS-Exchange-CrossTenant-fromentityheader: X-MS-Exchange-CrossTenant-id:X-MS-Exchange-CrossTenant-mailboxtype: X-MS-Exchange-CrossTenant-userprincipalname: X-MS-Exchange-Transport-CrossTenantHeadersStamped:X-OriginatorOrg; b=bnIjlG6CEKO8dL9OeCqMlHe1t2b70SSwATcXgmpFdn/N+e9FjKh+dW9nMYLFG9KXX hBGEmCqCKRTiyqSshZHIemYi3uHHPMLnxOJU6Am830cYhV+V15edLISZ/jAuXSqtek tlpVxLo/f11NaHOyP+iAKqYjsHZngKJgupBlM2OqRXCe1xNhP7ck6SJUPvmBV/2VLv pjUGZyaB99wNrvJxirENTClWHJeBMa+h40Yce597mt2TxbwOnA5zzFahLfAJ1zOKnV p1YIcnrzcwitP2EXxh/LSts9gvO6veLvofS9B7lB5Pv1+B4ye8iP2DTZ5TzqcoRr0a lcFzp++chTRUQ== Subject: Re: [dpdk-dev] [PATCH 4/4] app/testpmd: change hairpin queues setup X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Bing, PSB, Best, Ori > -----Original Message----- > From: Bing Zhao > Sent: Thursday, October 1, 2020 3:26 AM > Subject: [PATCH 4/4] app/testpmd: change hairpin queues setup >=20 > A new parameter `hairpin-mode` is introduced to the testpmd command > line. Bitmask value is used to provide more flexible configuration. >=20 > Bit 0 in the LSB indicates the hairpin will use the loop mode. The > previous port RX queue will be connected to the current port TX > queue. > Bit 1 in the LSB indicates the hairpin will use pair port mode. The > even index port will be paired with the next odd index port. If the > total number of probed port is odd, then the last one will be paired > to itself. > If this byte is zero, then each port will be paired to itself. > Bit 0 takes a higher priority in the checking. >=20 > Bit 4 in the second bytes indicate if the hairpin will use explicit > TX flow mode. >=20 I think some basic example will be great here. > If not set, default value zero will be used and the behavior will > try to get align with the previous single port mode. If the ports > belong to different vendors' NICs, it is suggested to use the `self` > hairpin mode only. >=20 > Since hairpin configures the hardware resources, the port mask of > packets forwarding engine will not be used here. >=20 > Signed-off-by: Bing Zhao > --- > app/test-pmd/parameters.c | 15 +++++++++++ > app/test-pmd/testpmd.c | 68 > ++++++++++++++++++++++++++++++++++++++++++++--- > app/test-pmd/testpmd.h | 2 ++ > 3 files changed, 81 insertions(+), 4 deletions(-) >=20 > diff --git a/app/test-pmd/parameters.c b/app/test-pmd/parameters.c > index 1ead595..991029d 100644 > --- a/app/test-pmd/parameters.c > +++ b/app/test-pmd/parameters.c > @@ -221,6 +221,9 @@ usage(char* progname) > "enabled\n"); > printf(" --record-core-cycles: enable measurement of CPU cycles.\n"); > printf(" --record-burst-stats: enable display of RX and TX bursts.\n")= ; > + printf(" --hairpin-mode=3D0xXX: bitmask set the hairpin port mode.\n " > + " 0x10 - explicit tx rule, 0x02 - hairpin ports paired\n" > + " 0x01 - hairpin ports loop, 0x00 - hairpin port self\n"); > } >=20 > #ifdef RTE_LIBRTE_CMDLINE > @@ -644,6 +647,7 @@ launch_args_parse(int argc, char** argv) > { "rxd", 1, 0, 0 }, > { "txd", 1, 0, 0 }, > { "hairpinq", 1, 0, 0 }, > + { "hairpin-mode", 1, 0, 0 }, > { "burst", 1, 0, 0 }, > { "mbcache", 1, 0, 0 }, > { "txpt", 1, 0, 0 }, > @@ -1111,6 +1115,17 @@ launch_args_parse(int argc, char** argv) > rte_exit(EXIT_FAILURE, "Either rx or tx queues > should " > "be non-zero\n"); > } > + if (!strcmp(lgopts[opt_idx].name, "hairpin-mode")) { > + char *end =3D NULL; > + unsigned int n; > + > + errno =3D 0; > + n =3D strtoul(optarg, &end, 0); > + if (errno !=3D 0 || end =3D=3D optarg) > + rte_exit(EXIT_FAILURE, "hairpin mode > invalid\n"); > + else > + hairpin_mode =3D (uint16_t)n; > + } > if (!strcmp(lgopts[opt_idx].name, "burst")) { > n =3D atoi(optarg); > if (n =3D=3D 0) { > diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c > index fe6450c..c604045 100644 > --- a/app/test-pmd/testpmd.c > +++ b/app/test-pmd/testpmd.c > @@ -367,6 +367,9 @@ bool setup_on_probe_event =3D true; > /* Clear ptypes on port initialization. */ > uint8_t clear_ptypes =3D true; >=20 > +/* Hairpin ports configuration mode. */ > +uint16_t hairpin_mode; > + > /* Pretty printing of ethdev events */ > static const char * const eth_event_desc[] =3D { > [RTE_ETH_EVENT_UNKNOWN] =3D "unknown", > @@ -2345,7 +2348,7 @@ port_is_started(portid_t port_id) >=20 > /* Configure the Rx and Tx hairpin queues for the selected port. */ > static int > -setup_hairpin_queues(portid_t pi) > +setup_hairpin_queues(portid_t pi, portid_t p_pi, uint16_t cnt_pi) > { > queueid_t qi; > struct rte_eth_hairpin_conf hairpin_conf =3D { > @@ -2354,10 +2357,48 @@ setup_hairpin_queues(portid_t pi) > int i; > int diag; > struct rte_port *port =3D &ports[pi]; > + uint16_t peer_rx_port =3D pi; > + uint16_t peer_tx_port =3D pi; > + uint32_t manual =3D 1; > + uint32_t tx_exp =3D hairpin_mode & 0x10; > + > + if (!(hairpin_mode & 0xf)) { > + peer_rx_port =3D pi; > + peer_tx_port =3D pi; > + manual =3D 0; > + } else if (hairpin_mode & 0x1) { > + peer_tx_port =3D rte_eth_find_next_owned_by(pi + 1, > + > RTE_ETH_DEV_NO_OWNER); > + if (peer_tx_port >=3D RTE_MAX_ETHPORTS) > + peer_tx_port =3D rte_eth_find_next_owned_by(0, > + RTE_ETH_DEV_NO_OWNER); > + if (p_pi !=3D RTE_MAX_ETHPORTS) { > + peer_rx_port =3D p_pi; > + } else { > + uint16_t next_pi; > + > + RTE_ETH_FOREACH_DEV(next_pi) > + peer_rx_port =3D next_pi; > + } > + manual =3D 1; > + } else if (hairpin_mode & 0x2) { > + if (cnt_pi & 0x1) { > + peer_rx_port =3D p_pi; > + } else { > + peer_rx_port =3D rte_eth_find_next_owned_by(pi + 1, > + RTE_ETH_DEV_NO_OWNER); > + if (peer_rx_port >=3D RTE_MAX_ETHPORTS) > + peer_rx_port =3D pi; > + } > + peer_tx_port =3D peer_rx_port; > + manual =3D 1; > + } >=20 > for (qi =3D nb_txq, i =3D 0; qi < nb_hairpinq + nb_txq; qi++) { > - hairpin_conf.peers[0].port =3D pi; > + hairpin_conf.peers[0].port =3D peer_rx_port; > hairpin_conf.peers[0].queue =3D i + nb_rxq; > + hairpin_conf.manual_bind =3D !!manual; > + hairpin_conf.tx_explicit =3D !!tx_exp; > diag =3D rte_eth_tx_hairpin_queue_setup > (pi, qi, nb_txd, &hairpin_conf); > i++; > @@ -2377,8 +2418,10 @@ setup_hairpin_queues(portid_t pi) > return -1; > } > for (qi =3D nb_rxq, i =3D 0; qi < nb_hairpinq + nb_rxq; qi++) { > - hairpin_conf.peers[0].port =3D pi; > + hairpin_conf.peers[0].port =3D peer_tx_port; > hairpin_conf.peers[0].queue =3D i + nb_txq; > + hairpin_conf.manual_bind =3D !!manual; > + hairpin_conf.tx_explicit =3D !!tx_exp; > diag =3D rte_eth_rx_hairpin_queue_setup > (pi, qi, nb_rxd, &hairpin_conf); > i++; > @@ -2405,6 +2448,8 @@ start_port(portid_t pid) > { > int diag, need_check_link_status =3D -1; > portid_t pi; > + portid_t p_pi =3D RTE_MAX_ETHPORTS; > + uint16_t cnt_pi =3D 0; > queueid_t qi; > struct rte_port *port; > struct rte_ether_addr mac_addr; > @@ -2544,8 +2589,10 @@ start_port(portid_t pid) > return -1; > } > /* setup hairpin queues */ > - if (setup_hairpin_queues(pi) !=3D 0) > + if (setup_hairpin_queues(pi, p_pi, cnt_pi) !=3D 0) > return -1; > + p_pi =3D pi; > + cnt_pi++; > } > configure_rxtx_dump_callbacks(verbose_level); > if (clear_ptypes) { > @@ -3775,6 +3822,19 @@ main(int argc, char** argv) > rte_exit(EXIT_FAILURE, "Start ports failed\n"); >=20 > /* set all ports to promiscuous mode by default */ > + if (hairpin_mode & 0x3) { > + RTE_ETH_FOREACH_DEV(port_id) { > + ret =3D rte_eth_hairpin_bind(port_id, > RTE_MAX_ETHPORTS); > + if (ret !=3D 0) { > + RTE_LOG(ERR, EAL, "Error during binding " > + "hairpin tx port %u: %s", > + port_id, rte_strerror(-ret)); > + return -1; > + } > + } > + } > + > + /* set all ports to promiscuous mode by default */ > RTE_ETH_FOREACH_DEV(port_id) { > ret =3D rte_eth_promiscuous_enable(port_id); > if (ret !=3D 0) > diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h > index c7e7e41..29ede20 100644 > --- a/app/test-pmd/testpmd.h > +++ b/app/test-pmd/testpmd.h > @@ -398,6 +398,8 @@ extern uint32_t param_total_num_mbufs; >=20 > extern uint16_t stats_period; >=20 > +extern uint16_t hairpin_mode; > + > #ifdef RTE_LIBRTE_LATENCY_STATS > extern uint8_t latencystats_enabled; > extern lcoreid_t latencystats_lcore_id; > -- > 2.5.5