From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id D0846A04BA; Wed, 7 Oct 2020 13:36:22 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id B5CBC1B74B; Wed, 7 Oct 2020 13:36:21 +0200 (CEST) Received: from hqnvemgate25.nvidia.com (hqnvemgate25.nvidia.com [216.228.121.64]) by dpdk.org (Postfix) with ESMTP id 36CE31B3E7 for ; Wed, 7 Oct 2020 13:36:19 +0200 (CEST) Received: from hqmail.nvidia.com (Not Verified[216.228.121.13]) by hqnvemgate25.nvidia.com (using TLS: TLSv1.2, AES256-SHA) id ; Wed, 07 Oct 2020 04:35:21 -0700 Received: from HQMAIL101.nvidia.com (172.20.187.10) by HQMAIL101.nvidia.com (172.20.187.10) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Wed, 7 Oct 2020 11:36:12 +0000 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (104.47.57.173) by HQMAIL101.nvidia.com (172.20.187.10) with Microsoft SMTP Server (TLS) id 15.0.1473.3 via Frontend Transport; Wed, 7 Oct 2020 11:36:12 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=AISXmScjUvkggUmsxvqEtl4zZLOPyNKfDBXG0PJnsRkxzZZfzJuEMnZYJ0lEgzmbq8+9y7KGkam5BogwHzk1qqLlAbT7N4g8YotCwGXKAsw7AX7Oi2VAUtZAAQq+UYmvP2kOWwXSdxoNMVVEzOSP+HiO9uFqJnApRJe2sn5GHC4UhProaRBP229C7KPRWos6xQKXDG/KvKZQA0KaJblwlhdJxAB5F9wvEuM9jSk4aTz5olgUmT0vCilml8sGPKHARkatKUylwB4Nx1+lMgkxFVE5oS6zW4IIkgZ586R/vE4plRflmdcd9i4RzwG6U4snHkxRmQ662Od2ZQ/X+xJ6Xg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=HVVFUyv8oWRF3Abd3+0Qv0HGlxZNSFA+uqUT9xl9ZGg=; b=kZbEYObQFtM5w/JsqOrFgf0R8/BMEtZNcjHUrJSM0+QG2XC5/QNF61fAfLX8cgZcFcQbtef80gZQkTBjcsEt8AGrmdQR1K7CJM/0Zg8sB5Xp/z9y3WiF9QpImTmukVs4BDA2xtZ9tpK9YVChnoL2eqytDwzQHXos3vis7qC+pZK04wdmAwENRRx9W5rjQ31b++fFv/3i6EZweY/dObxLqXNI4BEvMp/RSOkI0j6zCB9HmNVFClQW9PYBaRHsqz/YwAtPccz4c8XMUps3WtTr00uswhL0YkNERrA8gtv9y1qNfUvEuR80aeVHC46y4JMJ/5oeEGSahWgSmkhYzaqoxA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none Received: from CY4PR1201MB0072.namprd12.prod.outlook.com (2603:10b6:910:1b::19) by CY4PR12MB1528.namprd12.prod.outlook.com (2603:10b6:910:6::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3433.34; Wed, 7 Oct 2020 11:36:11 +0000 Received: from CY4PR1201MB0072.namprd12.prod.outlook.com ([fe80::f1b2:c80a:e623:e613]) by CY4PR1201MB0072.namprd12.prod.outlook.com ([fe80::f1b2:c80a:e623:e613%11]) with mapi id 15.20.3433.045; Wed, 7 Oct 2020 11:36:11 +0000 From: Bing Zhao To: Ori Kam , NBU-Contact-Thomas Monjalon , "ferruh.yigit@intel.com" , "arybchenko@solarflare.com" , "mdr@ashroe.eu" , "nhorman@tuxdriver.com" , "bernard.iremonger@intel.com" , "beilei.xing@intel.com" , "wenzhuo.lu@intel.com" CC: "dev@dpdk.org" Thread-Topic: [PATCH 4/4] app/testpmd: change hairpin queues setup Thread-Index: AQHWmjJMAXx1moP2Vk6zjZQtfM7XIKmMB4LQ Date: Wed, 7 Oct 2020 11:36:10 +0000 Message-ID: References: <1600012140-70151-1-git-send-email-bingz@nvidia.com> <1601511962-21532-1-git-send-email-bingz@nvidia.com> <1601511962-21532-5-git-send-email-bingz@nvidia.com> In-Reply-To: Accept-Language: en-US, zh-CN Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: nvidia.com; dkim=none (message not signed) header.d=none;nvidia.com; dmarc=none action=none header.from=nvidia.com; x-originating-ip: [154.18.172.130] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: f91b8d56-144a-459a-89da-08d86ab530b3 x-ms-traffictypediagnostic: CY4PR12MB1528: x-ld-processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:272; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: oRPXcrn9XSENxRl5SwuywXMCpVm2TjllLdOgGveLhEtMg7igBBqKfSURiQcLunxrFF8emMqB6vM5e459O9yuIXFoIeXZ3APPu6JSZxZvu6tDbQ9vMJyQ7iTHxQWV0sYgToxbYuglXDL7VwJwl2VY5PZQF1BumAAB+aMnEMnxFRxEFZ+uORt0qcjKTZWOopvztYnrdODD9dmjNOFk7VVPylrKriLNBxfz9hS9DCWPOVIILMVMn+nGdxZtqp5h7tC7DRsNtN033ybAYwVl2uujqaElu1wtFpWl0pPTcKV0x6Vx4Wxf4VzfisCcyWRLve0Us5MVLnuG7bpYXGm4pX/qcWGnUm1DniD1p2yfcvKDVHsvKlSaqjKC0+2LImf+FWUX x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CY4PR1201MB0072.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(136003)(396003)(376002)(346002)(366004)(39850400004)(66446008)(2906002)(9686003)(478600001)(8676002)(55016002)(8936002)(4326008)(66556008)(6506007)(66946007)(64756008)(71200400001)(110136005)(33656002)(76116006)(83380400001)(86362001)(186003)(66476007)(316002)(26005)(52536014)(5660300002)(7696005)(53546011)(309714004); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: lp/5fpvpxN8A53wDs5Fb5N4Q437xf9Om4NEZAG3btldGnRmnplI+aJtKadHIce0MzLGDInhGP+jFUtJVpzEer6ggKsWYxm2QlewifdCDkw4t/29oMWoFcY5d9jWVRvIFmYkeSvqgmimmJMdbDGuAQRVwvIW9hwWt5aemLkFyv+OSPu5KjVjBDqTGhwQvypfZAONiulmAG/duoX3Z9b8pcTtiHaGLhn25mBLmKJ/9qPFAJhC3M4oGmvZE4XRHUY9x8UXHy/miHYk7Jun+3stP+94ISvXv87TyZSMnYZQ09TZAQ3SNylChnPxatgIF4aVlBD2gniliJeKgIUdv2xlg3w/GR2hYPGWTc7yf9fLU72IOvFBOUBIwuWIHSrH7S784YOAA6d7x6ygtVwZeLKruhDVzdNugDIqnrdD88gBNabZStKu9T7Zzz9A20qQsRzBA6MG+v8icRjwjTljyZc0dvOmvYsoJqx6Zz5R/EY1GXrOzsLZ6a3xbuUcht9SxHdGCKdUOQ/U6yzeJCiZDXnfJPjMtwK/L0XStSKFQETgNqlWP9JFx0TzoR/OpjRCDnZHWYaTvUXmK3FqqvKHxNYfH9Jv0HOTWd3JL80it/t4ojl/3Jpu7plgDGO2WV0LNAF1+50ZQ4QvWJvEsO/V8QkF4bg== Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: CY4PR1201MB0072.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: f91b8d56-144a-459a-89da-08d86ab530b3 X-MS-Exchange-CrossTenant-originalarrivaltime: 07 Oct 2020 11:36:10.9803 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: aBRumy5U0eQ4PYOnOyySbSpZv5b0t0XDs54uPxzY8dzkn1cIrdkpuHMKYAlFD933XLZzXf67DKRjlwSrv0A2wQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR12MB1528 X-OriginatorOrg: Nvidia.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nvidia.com; s=n1; t=1602070522; bh=HVVFUyv8oWRF3Abd3+0Qv0HGlxZNSFA+uqUT9xl9ZGg=; h=ARC-Seal:ARC-Message-Signature:ARC-Authentication-Results:From:To: CC:Subject:Thread-Topic:Thread-Index:Date:Message-ID:References: In-Reply-To:Accept-Language:Content-Language:X-MS-Has-Attach: X-MS-TNEF-Correlator:authentication-results:x-originating-ip: x-ms-publictraffictype:x-ms-office365-filtering-correlation-id: x-ms-traffictypediagnostic:x-ld-processed: x-ms-exchange-transport-forked:x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers:x-ms-exchange-senderadcheck: x-microsoft-antispam:x-microsoft-antispam-message-info: x-forefront-antispam-report:x-ms-exchange-antispam-messagedata: Content-Type:Content-Transfer-Encoding:MIME-Version: X-MS-Exchange-CrossTenant-AuthAs: X-MS-Exchange-CrossTenant-AuthSource: X-MS-Exchange-CrossTenant-Network-Message-Id: X-MS-Exchange-CrossTenant-originalarrivaltime: X-MS-Exchange-CrossTenant-fromentityheader: X-MS-Exchange-CrossTenant-id:X-MS-Exchange-CrossTenant-mailboxtype: X-MS-Exchange-CrossTenant-userprincipalname: X-MS-Exchange-Transport-CrossTenantHeadersStamped:X-OriginatorOrg; b=sRaq5H2WBB9Yraa+zxcbu0RwkivKqyXP90qzyWpcgqBt2KnBKsQ/1pykZNsk//erL s1ZuTjXBG/DPfmRO2gKhqF2sgZsJ70ZSa6rbcaLoJhjHw8m/g6H4QUj2az2v4ilDJc xIXqs+1khB/TLiY5z9L1SJy/XnSGV405rfo44iMDaTov7zf2RUb9We+6AXLZ5QDPeO FqnqoELabUpZAhu5/+rrsVwWzzCJ/8Wn7b6fIvj5p5o2hN9kZcQrc1n3QaNq7AUnIC yc1HoM64yiY15yQtT/o38HPyh7OD6QYckMufndrv7vYv4BL3OIDU6Nl+a+sFhB0E7X AiCY0O0x3VH9A== Subject: Re: [dpdk-dev] [PATCH 4/4] app/testpmd: change hairpin queues setup X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Ori, Thanks for your comments. I will update the commit logs as well as the documents. BR. Bing > -----Original Message----- > From: Ori Kam > Sent: Sunday, October 4, 2020 5:40 PM > To: Bing Zhao ; NBU-Contact-Thomas Monjalon > ; ferruh.yigit@intel.com; > arybchenko@solarflare.com; mdr@ashroe.eu; nhorman@tuxdriver.com; > bernard.iremonger@intel.com; beilei.xing@intel.com; > wenzhuo.lu@intel.com > Cc: dev@dpdk.org > Subject: RE: [PATCH 4/4] app/testpmd: change hairpin queues setup >=20 > Hi Bing, >=20 > PSB, >=20 > Best, > Ori >=20 > > -----Original Message----- > > From: Bing Zhao > > Sent: Thursday, October 1, 2020 3:26 AM > > Subject: [PATCH 4/4] app/testpmd: change hairpin queues setup > > > > A new parameter `hairpin-mode` is introduced to the testpmd > command > > line. Bitmask value is used to provide more flexible configuration. > > > > Bit 0 in the LSB indicates the hairpin will use the loop mode. The > > previous port RX queue will be connected to the current port TX > queue. > > Bit 1 in the LSB indicates the hairpin will use pair port mode. > The > > even index port will be paired with the next odd index port. If > the > > total number of probed port is odd, then the last one will be > paired > > to itself. > > If this byte is zero, then each port will be paired to itself. > > Bit 0 takes a higher priority in the checking. > > > > Bit 4 in the second bytes indicate if the hairpin will use > explicit TX > > flow mode. > > > I think some basic example will be great here. >=20 > > If not set, default value zero will be used and the behavior will > try > > to get align with the previous single port mode. If the ports > belong > > to different vendors' NICs, it is suggested to use the `self` > hairpin > > mode only. > > > > Since hairpin configures the hardware resources, the port mask of > > packets forwarding engine will not be used here. > > > > Signed-off-by: Bing Zhao > > --- > > app/test-pmd/parameters.c | 15 +++++++++++ > > app/test-pmd/testpmd.c | 68 > > ++++++++++++++++++++++++++++++++++++++++++++--- > > app/test-pmd/testpmd.h | 2 ++ > > 3 files changed, 81 insertions(+), 4 deletions(-) > > > > diff --git a/app/test-pmd/parameters.c b/app/test-pmd/parameters.c > > index 1ead595..991029d 100644 > > --- a/app/test-pmd/parameters.c > > +++ b/app/test-pmd/parameters.c > > @@ -221,6 +221,9 @@ usage(char* progname) > > "enabled\n"); > > printf(" --record-core-cycles: enable measurement of CPU > cycles.\n"); > > printf(" --record-burst-stats: enable display of RX and TX > > bursts.\n"); > > + printf(" --hairpin-mode=3D0xXX: bitmask set the hairpin port > mode.\n " > > + " 0x10 - explicit tx rule, 0x02 - hairpin ports > paired\n" > > + " 0x01 - hairpin ports loop, 0x00 - hairpin port > self\n"); > > } > > > > #ifdef RTE_LIBRTE_CMDLINE > > @@ -644,6 +647,7 @@ launch_args_parse(int argc, char** argv) > > { "rxd", 1, 0, 0 }, > > { "txd", 1, 0, 0 }, > > { "hairpinq", 1, 0, 0 }, > > + { "hairpin-mode", 1, 0, 0 }, > > { "burst", 1, 0, 0 }, > > { "mbcache", 1, 0, 0 }, > > { "txpt", 1, 0, 0 }, > > @@ -1111,6 +1115,17 @@ launch_args_parse(int argc, char** argv) > > rte_exit(EXIT_FAILURE, "Either rx or tx > queues should " > > "be non-zero\n"); > > } > > + if (!strcmp(lgopts[opt_idx].name, "hairpin-mode")) > { > > + char *end =3D NULL; > > + unsigned int n; > > + > > + errno =3D 0; > > + n =3D strtoul(optarg, &end, 0); > > + if (errno !=3D 0 || end =3D=3D optarg) > > + rte_exit(EXIT_FAILURE, "hairpin mode > > invalid\n"); > > + else > > + hairpin_mode =3D (uint16_t)n; > > + } > > if (!strcmp(lgopts[opt_idx].name, "burst")) { > > n =3D atoi(optarg); > > if (n =3D=3D 0) { > > diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index > > fe6450c..c604045 100644 > > --- a/app/test-pmd/testpmd.c > > +++ b/app/test-pmd/testpmd.c > > @@ -367,6 +367,9 @@ bool setup_on_probe_event =3D true; > > /* Clear ptypes on port initialization. */ uint8_t clear_ptypes > =3D > > true; > > > > +/* Hairpin ports configuration mode. */ uint16_t hairpin_mode; > > + > > /* Pretty printing of ethdev events */ static const char * const > > eth_event_desc[] =3D { > > [RTE_ETH_EVENT_UNKNOWN] =3D "unknown", @@ -2345,7 +2348,7 @@ > > port_is_started(portid_t port_id) > > > > /* Configure the Rx and Tx hairpin queues for the selected port. > */ > > static int -setup_hairpin_queues(portid_t pi) > > +setup_hairpin_queues(portid_t pi, portid_t p_pi, uint16_t cnt_pi) > > { > > queueid_t qi; > > struct rte_eth_hairpin_conf hairpin_conf =3D { @@ -2354,10 > +2357,48 @@ > > setup_hairpin_queues(portid_t pi) > > int i; > > int diag; > > struct rte_port *port =3D &ports[pi]; > > + uint16_t peer_rx_port =3D pi; > > + uint16_t peer_tx_port =3D pi; > > + uint32_t manual =3D 1; > > + uint32_t tx_exp =3D hairpin_mode & 0x10; > > + > > + if (!(hairpin_mode & 0xf)) { > > + peer_rx_port =3D pi; > > + peer_tx_port =3D pi; > > + manual =3D 0; > > + } else if (hairpin_mode & 0x1) { > > + peer_tx_port =3D rte_eth_find_next_owned_by(pi + 1, > > + > > RTE_ETH_DEV_NO_OWNER); > > + if (peer_tx_port >=3D RTE_MAX_ETHPORTS) > > + peer_tx_port =3D rte_eth_find_next_owned_by(0, > > + RTE_ETH_DEV_NO_OWNER); > > + if (p_pi !=3D RTE_MAX_ETHPORTS) { > > + peer_rx_port =3D p_pi; > > + } else { > > + uint16_t next_pi; > > + > > + RTE_ETH_FOREACH_DEV(next_pi) > > + peer_rx_port =3D next_pi; > > + } > > + manual =3D 1; > > + } else if (hairpin_mode & 0x2) { > > + if (cnt_pi & 0x1) { > > + peer_rx_port =3D p_pi; > > + } else { > > + peer_rx_port =3D rte_eth_find_next_owned_by(pi + 1, > > + RTE_ETH_DEV_NO_OWNER); > > + if (peer_rx_port >=3D RTE_MAX_ETHPORTS) > > + peer_rx_port =3D pi; > > + } > > + peer_tx_port =3D peer_rx_port; > > + manual =3D 1; > > + } > > > > for (qi =3D nb_txq, i =3D 0; qi < nb_hairpinq + nb_txq; qi++) { > > - hairpin_conf.peers[0].port =3D pi; > > + hairpin_conf.peers[0].port =3D peer_rx_port; > > hairpin_conf.peers[0].queue =3D i + nb_rxq; > > + hairpin_conf.manual_bind =3D !!manual; > > + hairpin_conf.tx_explicit =3D !!tx_exp; > > diag =3D rte_eth_tx_hairpin_queue_setup > > (pi, qi, nb_txd, &hairpin_conf); > > i++; > > @@ -2377,8 +2418,10 @@ setup_hairpin_queues(portid_t pi) > > return -1; > > } > > for (qi =3D nb_rxq, i =3D 0; qi < nb_hairpinq + nb_rxq; qi++) { > > - hairpin_conf.peers[0].port =3D pi; > > + hairpin_conf.peers[0].port =3D peer_tx_port; > > hairpin_conf.peers[0].queue =3D i + nb_txq; > > + hairpin_conf.manual_bind =3D !!manual; > > + hairpin_conf.tx_explicit =3D !!tx_exp; > > diag =3D rte_eth_rx_hairpin_queue_setup > > (pi, qi, nb_rxd, &hairpin_conf); > > i++; > > @@ -2405,6 +2448,8 @@ start_port(portid_t pid) { > > int diag, need_check_link_status =3D -1; > > portid_t pi; > > + portid_t p_pi =3D RTE_MAX_ETHPORTS; > > + uint16_t cnt_pi =3D 0; > > queueid_t qi; > > struct rte_port *port; > > struct rte_ether_addr mac_addr; > > @@ -2544,8 +2589,10 @@ start_port(portid_t pid) > > return -1; > > } > > /* setup hairpin queues */ > > - if (setup_hairpin_queues(pi) !=3D 0) > > + if (setup_hairpin_queues(pi, p_pi, cnt_pi) !=3D 0) > > return -1; > > + p_pi =3D pi; > > + cnt_pi++; > > } > > configure_rxtx_dump_callbacks(verbose_level); > > if (clear_ptypes) { > > @@ -3775,6 +3822,19 @@ main(int argc, char** argv) > > rte_exit(EXIT_FAILURE, "Start ports failed\n"); > > > > /* set all ports to promiscuous mode by default */ > > + if (hairpin_mode & 0x3) { > > + RTE_ETH_FOREACH_DEV(port_id) { > > + ret =3D rte_eth_hairpin_bind(port_id, > > RTE_MAX_ETHPORTS); > > + if (ret !=3D 0) { > > + RTE_LOG(ERR, EAL, "Error during binding " > > + "hairpin tx port %u: %s", > > + port_id, rte_strerror(-ret)); > > + return -1; > > + } > > + } > > + } > > + > > + /* set all ports to promiscuous mode by default */ > > RTE_ETH_FOREACH_DEV(port_id) { > > ret =3D rte_eth_promiscuous_enable(port_id); > > if (ret !=3D 0) > > diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h index > > c7e7e41..29ede20 100644 > > --- a/app/test-pmd/testpmd.h > > +++ b/app/test-pmd/testpmd.h > > @@ -398,6 +398,8 @@ extern uint32_t param_total_num_mbufs; > > > > extern uint16_t stats_period; > > > > +extern uint16_t hairpin_mode; > > + > > #ifdef RTE_LIBRTE_LATENCY_STATS > > extern uint8_t latencystats_enabled; > > extern lcoreid_t latencystats_lcore_id; > > -- > > 2.5.5