From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR04-VI1-obe.outbound.protection.outlook.com (mail-eopbgr80087.outbound.protection.outlook.com [40.107.8.87]) by dpdk.org (Postfix) with ESMTP id B172E4C9F for ; Tue, 2 Apr 2019 17:21:32 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=bI7o4BquJ9mj7grA+yteQvG5h9nwMiTau7mvyLqu02U=; b=k3G7R5DvUlrSgS1W5qQiT5jdIIPLqzQHFHhHJ5eWxjkWyNwCeJF3LU1DggtJ6i6PiNQPjMJp6EO1b4zKU9DiCest2kNPHjSPJryMlDt5QuUzshQOY1GvQvnd/LQPUqSbbQfMpVbAqVzE8K1BeseW/nm1NMGH+1fsRZUgL0Hiwak= Received: from DB3PR0502MB3964.eurprd05.prod.outlook.com (52.134.65.161) by DB3PR0502MB4076.eurprd05.prod.outlook.com (52.134.66.11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1750.16; Tue, 2 Apr 2019 15:21:30 +0000 Received: from DB3PR0502MB3964.eurprd05.prod.outlook.com ([fe80::7c3f:eb42:7dca:801f]) by DB3PR0502MB3964.eurprd05.prod.outlook.com ([fe80::7c3f:eb42:7dca:801f%4]) with mapi id 15.20.1750.014; Tue, 2 Apr 2019 15:21:30 +0000 From: Raslan Darawsheh To: Pavan Nikhilesh Bhagavatula , Jerin Jacob Kollanukkaran , Thomas Monjalon , "arybchenko@solarflare.com" , "ferruh.yigit@intel.com" , "bernard.iremonger@intel.com" , Ali Alnubani CC: "dev@dpdk.org" Thread-Topic: [dpdk-dev] [PATCH v6 1/4] app/testpmd: move eth header generation outside the loop Thread-Index: AQHU6Tn2X4pyuQZnRUSeqodyAKxEsKYo/Iow Date: Tue, 2 Apr 2019 15:21:30 +0000 Message-ID: References: <20190228194128.14236-1-pbhagavatula@marvell.com> <20190402095255.848-1-pbhagavatula@marvell.com> In-Reply-To: <20190402095255.848-1-pbhagavatula@marvell.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=rasland@mellanox.com; x-originating-ip: [82.213.2.186] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 0fb94d2b-d8e8-4d7e-c4f3-08d6b77ee223 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(5600139)(711020)(4605104)(4618075)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(2017052603328)(7193020); SRVR:DB3PR0502MB4076; x-ms-traffictypediagnostic: DB3PR0502MB4076: x-ld-processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr x-microsoft-antispam-prvs: x-forefront-prvs: 0995196AA2 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(396003)(136003)(366004)(39860400002)(376002)(346002)(189003)(199004)(13464003)(8936002)(86362001)(6636002)(66066001)(97736004)(52536014)(476003)(11346002)(9686003)(14454004)(2201001)(33656002)(2501003)(7736002)(71190400001)(2906002)(68736007)(74316002)(71200400001)(81166006)(105586002)(55016002)(6116002)(3846002)(256004)(76176011)(186003)(305945005)(25786009)(53936002)(4326008)(6506007)(81156014)(26005)(102836004)(6436002)(53546011)(446003)(486006)(7696005)(316002)(99286004)(106356001)(110136005)(229853002)(5660300002)(478600001)(6246003)(8676002); DIR:OUT; SFP:1101; SCL:1; SRVR:DB3PR0502MB4076; H:DB3PR0502MB3964.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: nTS56nFlzuZC6GIFLXOTQCZmQ7bOFAyOy6ls+JHsgyEnv9+kVnv+R8T+HYc783a5nx+WYxnzk7KaQGNbBvGibxZzEFtswIgtLUCkvJ0fpV5gAVf+icpIF4pJU/RHMgIcNSkafXUP9BMch1ysIg67sSUPtPDNEWk+5CmWHMht2hF40888/NpwjTqnhGM2djDQ6r0d+PIuoZl25J6E3ZxGG+QCMyDCbH5WKiDDAJQ2+8irPKBCH5JnFoO9fPgI5K/hu7STLzFeU0qq/m49Ef99fEyHMczchzHynBzbRpbdCiJPl2NTGKS+yH361qbhsU2T+gwnhmxalpYKXZeq1hKA/zPi3staurLPWkaJNYWkBewDmT282RPPF0eTpZGnTdPMNAShuQtYwMFzC1RPatkm1k01xIHR/fZQRAXbOaYbr6s= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0fb94d2b-d8e8-4d7e-c4f3-08d6b77ee223 X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Apr 2019 15:21:30.6100 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB3PR0502MB4076 Subject: Re: [dpdk-dev] [PATCH v6 1/4] app/testpmd: move eth header generation outside the loop X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Apr 2019 15:21:33 -0000 Hi, The performance issue that we saw in Mellanox is now fixed with latest vers= ion: The issue was with the packet len calculation that was changed in the lates= t versions > 4: diff --git a/app/test-pmd/txonly.c b/app/test-pmd/txonly.c index af0be89..2f40949 100644 --- a/app/test-pmd/txonly.c +++ b/app/test-pmd/txonly.c @@ -176,6 +176,7 @@ pkt_burst_prepare(struct rte_mbuf *pkt, struct rte_memp= ool *mbp, pkt->l3_len =3D sizeof(struct ipv4_hdr); pkt_seg =3D pkt; + pkt_len =3D pkt->data_len; for (i =3D 1; i < nb_segs; i++) { pkt_seg->next =3D pkt_segs[i - 1]; pkt_seg =3D pkt_seg->next; @@ -198,7 +199,7 @@ pkt_burst_prepare(struct rte_mbuf *pkt, struct rte_memp= ool *mbp, * burst of packets to be transmitted. */ pkt->nb_segs =3D nb_segs; - pkt->pkt_len +=3D pkt_len; + pkt->pkt_len =3D pkt_len; return true; and now we see ~20% improvement in performance. Kindest regards, Raslan Darawsheh > -----Original Message----- > From: dev On Behalf Of Pavan Nikhilesh > Bhagavatula > Sent: Tuesday, April 2, 2019 12:53 PM > To: Jerin Jacob Kollanukkaran ; Thomas Monjalon > ; arybchenko@solarflare.com; > ferruh.yigit@intel.com; bernard.iremonger@intel.com; Ali Alnubani > > Cc: dev@dpdk.org; Pavan Nikhilesh Bhagavatula > > Subject: [dpdk-dev] [PATCH v6 1/4] app/testpmd: move eth header > generation outside the loop >=20 > From: Pavan Nikhilesh >=20 > Testpmd txonly copies the src/dst mac address of the port being > processed to ethernet header structure on the stack for every packet. > Move it outside the loop and reuse it. >=20 > Signed-off-by: Pavan Nikhilesh > --- > v6 Changes > - Rebase onto ToT. > - Split the changes further >=20 > v5 Changes > - Remove unnecessary change to struct rte_port *txp (movement). > (Bernard) >=20 > v4 Changes: > - Fix packet len calculation. >=20 > v3 Changes: > - Split the patches for easier review. (Thomas) > - Remove unnecessary assignments to 0. (Bernard) >=20 > v2 Changes: > - Use bulk ops for fetching segments. (Andrew Rybchenko) > - Fallback to rte_mbuf_raw_alloc if bulk get fails. (Andrew Rybchenko) > - Fix mbufs not being freed when there is no more mbufs available for > segments. (Andrew Rybchenko) >=20 > app/test-pmd/txonly.c | 15 ++++++++------- > 1 file changed, 8 insertions(+), 7 deletions(-) >=20 > diff --git a/app/test-pmd/txonly.c b/app/test-pmd/txonly.c > index def52a048..0d411dbf4 100644 > --- a/app/test-pmd/txonly.c > +++ b/app/test-pmd/txonly.c > @@ -190,6 +190,14 @@ pkt_burst_transmit(struct fwd_stream *fs) > ol_flags |=3D PKT_TX_QINQ_PKT; > if (tx_offloads & DEV_TX_OFFLOAD_MACSEC_INSERT) > ol_flags |=3D PKT_TX_MACSEC; > + > + /* > + * Initialize Ethernet header. > + */ > + ether_addr_copy(&peer_eth_addrs[fs->peer_addr], > ð_hdr.d_addr); > + ether_addr_copy(&ports[fs->tx_port].eth_addr, ð_hdr.s_addr); > + eth_hdr.ether_type =3D rte_cpu_to_be_16(ETHER_TYPE_IPv4); > + > for (nb_pkt =3D 0; nb_pkt < nb_pkt_per_burst; nb_pkt++) { > pkt =3D rte_mbuf_raw_alloc(mbp); > if (pkt =3D=3D NULL) { > @@ -226,13 +234,6 @@ pkt_burst_transmit(struct fwd_stream *fs) > } > pkt_seg->next =3D NULL; /* Last segment of packet. */ >=20 > - /* > - * Initialize Ethernet header. > - */ > - ether_addr_copy(&peer_eth_addrs[fs- > >peer_addr],ð_hdr.d_addr); > - ether_addr_copy(&ports[fs->tx_port].eth_addr, > ð_hdr.s_addr); > - eth_hdr.ether_type =3D > rte_cpu_to_be_16(ETHER_TYPE_IPv4); > - > /* > * Copy headers in first packet segment(s). > */ > -- > 2.21.0 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by dpdk.space (Postfix) with ESMTP id 6823EA0679 for ; Tue, 2 Apr 2019 17:21:34 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id F26AB4CA0; Tue, 2 Apr 2019 17:21:33 +0200 (CEST) Received: from EUR04-VI1-obe.outbound.protection.outlook.com (mail-eopbgr80087.outbound.protection.outlook.com [40.107.8.87]) by dpdk.org (Postfix) with ESMTP id B172E4C9F for ; Tue, 2 Apr 2019 17:21:32 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=bI7o4BquJ9mj7grA+yteQvG5h9nwMiTau7mvyLqu02U=; b=k3G7R5DvUlrSgS1W5qQiT5jdIIPLqzQHFHhHJ5eWxjkWyNwCeJF3LU1DggtJ6i6PiNQPjMJp6EO1b4zKU9DiCest2kNPHjSPJryMlDt5QuUzshQOY1GvQvnd/LQPUqSbbQfMpVbAqVzE8K1BeseW/nm1NMGH+1fsRZUgL0Hiwak= Received: from DB3PR0502MB3964.eurprd05.prod.outlook.com (52.134.65.161) by DB3PR0502MB4076.eurprd05.prod.outlook.com (52.134.66.11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1750.16; Tue, 2 Apr 2019 15:21:30 +0000 Received: from DB3PR0502MB3964.eurprd05.prod.outlook.com ([fe80::7c3f:eb42:7dca:801f]) by DB3PR0502MB3964.eurprd05.prod.outlook.com ([fe80::7c3f:eb42:7dca:801f%4]) with mapi id 15.20.1750.014; Tue, 2 Apr 2019 15:21:30 +0000 From: Raslan Darawsheh To: Pavan Nikhilesh Bhagavatula , Jerin Jacob Kollanukkaran , Thomas Monjalon , "arybchenko@solarflare.com" , "ferruh.yigit@intel.com" , "bernard.iremonger@intel.com" , Ali Alnubani CC: "dev@dpdk.org" Thread-Topic: [dpdk-dev] [PATCH v6 1/4] app/testpmd: move eth header generation outside the loop Thread-Index: AQHU6Tn2X4pyuQZnRUSeqodyAKxEsKYo/Iow Date: Tue, 2 Apr 2019 15:21:30 +0000 Message-ID: References: <20190228194128.14236-1-pbhagavatula@marvell.com> <20190402095255.848-1-pbhagavatula@marvell.com> In-Reply-To: <20190402095255.848-1-pbhagavatula@marvell.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=rasland@mellanox.com; x-originating-ip: [82.213.2.186] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 0fb94d2b-d8e8-4d7e-c4f3-08d6b77ee223 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(5600139)(711020)(4605104)(4618075)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(2017052603328)(7193020); SRVR:DB3PR0502MB4076; x-ms-traffictypediagnostic: DB3PR0502MB4076: x-ld-processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr x-microsoft-antispam-prvs: x-forefront-prvs: 0995196AA2 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(396003)(136003)(366004)(39860400002)(376002)(346002)(189003)(199004)(13464003)(8936002)(86362001)(6636002)(66066001)(97736004)(52536014)(476003)(11346002)(9686003)(14454004)(2201001)(33656002)(2501003)(7736002)(71190400001)(2906002)(68736007)(74316002)(71200400001)(81166006)(105586002)(55016002)(6116002)(3846002)(256004)(76176011)(186003)(305945005)(25786009)(53936002)(4326008)(6506007)(81156014)(26005)(102836004)(6436002)(53546011)(446003)(486006)(7696005)(316002)(99286004)(106356001)(110136005)(229853002)(5660300002)(478600001)(6246003)(8676002); DIR:OUT; SFP:1101; SCL:1; SRVR:DB3PR0502MB4076; H:DB3PR0502MB3964.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: nTS56nFlzuZC6GIFLXOTQCZmQ7bOFAyOy6ls+JHsgyEnv9+kVnv+R8T+HYc783a5nx+WYxnzk7KaQGNbBvGibxZzEFtswIgtLUCkvJ0fpV5gAVf+icpIF4pJU/RHMgIcNSkafXUP9BMch1ysIg67sSUPtPDNEWk+5CmWHMht2hF40888/NpwjTqnhGM2djDQ6r0d+PIuoZl25J6E3ZxGG+QCMyDCbH5WKiDDAJQ2+8irPKBCH5JnFoO9fPgI5K/hu7STLzFeU0qq/m49Ef99fEyHMczchzHynBzbRpbdCiJPl2NTGKS+yH361qbhsU2T+gwnhmxalpYKXZeq1hKA/zPi3staurLPWkaJNYWkBewDmT282RPPF0eTpZGnTdPMNAShuQtYwMFzC1RPatkm1k01xIHR/fZQRAXbOaYbr6s= Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0fb94d2b-d8e8-4d7e-c4f3-08d6b77ee223 X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Apr 2019 15:21:30.6100 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB3PR0502MB4076 Subject: Re: [dpdk-dev] [PATCH v6 1/4] app/testpmd: move eth header generation outside the loop X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Message-ID: <20190402152130.-06zH4yQFVaa5jptp1CBI_d_g8a7EEoJxwXtZv8sGv8@z> Hi, The performance issue that we saw in Mellanox is now fixed with latest vers= ion: The issue was with the packet len calculation that was changed in the lates= t versions > 4: diff --git a/app/test-pmd/txonly.c b/app/test-pmd/txonly.c index af0be89..2f40949 100644 --- a/app/test-pmd/txonly.c +++ b/app/test-pmd/txonly.c @@ -176,6 +176,7 @@ pkt_burst_prepare(struct rte_mbuf *pkt, struct rte_memp= ool *mbp, pkt->l3_len =3D sizeof(struct ipv4_hdr); pkt_seg =3D pkt; + pkt_len =3D pkt->data_len; for (i =3D 1; i < nb_segs; i++) { pkt_seg->next =3D pkt_segs[i - 1]; pkt_seg =3D pkt_seg->next; @@ -198,7 +199,7 @@ pkt_burst_prepare(struct rte_mbuf *pkt, struct rte_memp= ool *mbp, * burst of packets to be transmitted. */ pkt->nb_segs =3D nb_segs; - pkt->pkt_len +=3D pkt_len; + pkt->pkt_len =3D pkt_len; return true; and now we see ~20% improvement in performance. Kindest regards, Raslan Darawsheh > -----Original Message----- > From: dev On Behalf Of Pavan Nikhilesh > Bhagavatula > Sent: Tuesday, April 2, 2019 12:53 PM > To: Jerin Jacob Kollanukkaran ; Thomas Monjalon > ; arybchenko@solarflare.com; > ferruh.yigit@intel.com; bernard.iremonger@intel.com; Ali Alnubani > > Cc: dev@dpdk.org; Pavan Nikhilesh Bhagavatula > > Subject: [dpdk-dev] [PATCH v6 1/4] app/testpmd: move eth header > generation outside the loop >=20 > From: Pavan Nikhilesh >=20 > Testpmd txonly copies the src/dst mac address of the port being > processed to ethernet header structure on the stack for every packet. > Move it outside the loop and reuse it. >=20 > Signed-off-by: Pavan Nikhilesh > --- > v6 Changes > - Rebase onto ToT. > - Split the changes further >=20 > v5 Changes > - Remove unnecessary change to struct rte_port *txp (movement). > (Bernard) >=20 > v4 Changes: > - Fix packet len calculation. >=20 > v3 Changes: > - Split the patches for easier review. (Thomas) > - Remove unnecessary assignments to 0. (Bernard) >=20 > v2 Changes: > - Use bulk ops for fetching segments. (Andrew Rybchenko) > - Fallback to rte_mbuf_raw_alloc if bulk get fails. (Andrew Rybchenko) > - Fix mbufs not being freed when there is no more mbufs available for > segments. (Andrew Rybchenko) >=20 > app/test-pmd/txonly.c | 15 ++++++++------- > 1 file changed, 8 insertions(+), 7 deletions(-) >=20 > diff --git a/app/test-pmd/txonly.c b/app/test-pmd/txonly.c > index def52a048..0d411dbf4 100644 > --- a/app/test-pmd/txonly.c > +++ b/app/test-pmd/txonly.c > @@ -190,6 +190,14 @@ pkt_burst_transmit(struct fwd_stream *fs) > ol_flags |=3D PKT_TX_QINQ_PKT; > if (tx_offloads & DEV_TX_OFFLOAD_MACSEC_INSERT) > ol_flags |=3D PKT_TX_MACSEC; > + > + /* > + * Initialize Ethernet header. > + */ > + ether_addr_copy(&peer_eth_addrs[fs->peer_addr], > ð_hdr.d_addr); > + ether_addr_copy(&ports[fs->tx_port].eth_addr, ð_hdr.s_addr); > + eth_hdr.ether_type =3D rte_cpu_to_be_16(ETHER_TYPE_IPv4); > + > for (nb_pkt =3D 0; nb_pkt < nb_pkt_per_burst; nb_pkt++) { > pkt =3D rte_mbuf_raw_alloc(mbp); > if (pkt =3D=3D NULL) { > @@ -226,13 +234,6 @@ pkt_burst_transmit(struct fwd_stream *fs) > } > pkt_seg->next =3D NULL; /* Last segment of packet. */ >=20 > - /* > - * Initialize Ethernet header. > - */ > - ether_addr_copy(&peer_eth_addrs[fs- > >peer_addr],ð_hdr.d_addr); > - ether_addr_copy(&ports[fs->tx_port].eth_addr, > ð_hdr.s_addr); > - eth_hdr.ether_type =3D > rte_cpu_to_be_16(ETHER_TYPE_IPv4); > - > /* > * Copy headers in first packet segment(s). > */ > -- > 2.21.0