From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9E320A04C1; Tue, 26 Nov 2019 09:46:47 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id A61812A5D; Tue, 26 Nov 2019 09:46:46 +0100 (CET) Received: from EUR02-AM5-obe.outbound.protection.outlook.com (mail-eopbgr00082.outbound.protection.outlook.com [40.107.0.82]) by dpdk.org (Postfix) with ESMTP id 330A8235 for ; Tue, 26 Nov 2019 09:46:45 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=qW8DG1upWDGMpu4muvuP555wvqu23ApP2Vx5L/nM/Mw=; b=JQG4f5h3K5VW13EJu07EPiuRrlOEk7C/y65rKsdH1jRaqBl/AHx4nV1aVm5z3HmcdFedVVvPeuab0mVNvcwvSU9FTacHZ2KIiJkqNhH0FlWE9lcPwEnyCtq9aeUWZEsbFwag3aprOfKhxOTpVMs6hHC1rrYBHcn+cEAjd1AqSIM= Received: from VE1PR08CA0026.eurprd08.prod.outlook.com (2603:10a6:803:104::39) by VI1PR0801MB2079.eurprd08.prod.outlook.com (2603:10a6:800:8e::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2474.19; Tue, 26 Nov 2019 08:46:43 +0000 Received: from AM5EUR03FT050.eop-EUR03.prod.protection.outlook.com (2a01:111:f400:7e08::207) by VE1PR08CA0026.outlook.office365.com (2603:10a6:803:104::39) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2474.17 via Frontend Transport; Tue, 26 Nov 2019 08:46:43 +0000 Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dpdk.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dpdk.org; dmarc=bestguesspass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by AM5EUR03FT050.mail.protection.outlook.com (10.152.17.47) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2474.17 via Frontend Transport; Tue, 26 Nov 2019 08:46:43 +0000 Received: ("Tessian outbound 712c40e503a7:v33"); Tue, 26 Nov 2019 08:46:42 +0000 X-CR-MTA-TID: 64aa7808 Received: from 58580b0600a3.2 (ip-172-16-0-2.eu-west-1.compute.internal [104.47.14.52]) by 64aa7808-outbound-1.mta.getcheckrecipient.com id 6B9F2D42-8BF2-4E8A-9172-50524F02484E.1; Tue, 26 Nov 2019 08:46:37 +0000 Received: from EUR04-VI1-obe.outbound.protection.outlook.com (mail-vi1eur04lp2052.outbound.protection.outlook.com [104.47.14.52]) by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 58580b0600a3.2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Tue, 26 Nov 2019 08:46:37 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=MLxX1Wkmvtmmjouox2PkMLQhj93yUeF8it9exk+soZhEQm3n0QD9OuskQMI67mIvNM/Ul+TMB9p0eC2MwFXbVt7D1t38W+xeb1Gi6em9TCJzqxiXZI4TyQiOW2WHVQKUqQaGRpHLdJx142i12IMgwAFdM4mSWI5Rr0E21iyZqide4qOiU+r0rRnwEaFs0j7DkJs5d5w91JffKDvv36Nq735xv7BUmnsGW1NABicHwVjrnYrjyb4h8wLixqxyVEDi4t3qXXL7UKfIAynm9w/KXvhURg4KnBJGFPmSwmbaSBEWG/Rn40hBHei8VutbIjGdwe3g6+OdPvQkePH6vMjv1g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=qW8DG1upWDGMpu4muvuP555wvqu23ApP2Vx5L/nM/Mw=; b=m2eal7pIaky6ymX8RCiHq9ZMxV4HklXUl0/13JvDk8Cyt0wBpIy1Dv6CXEI/50A158qm4F8rQtN2RihJKbwIAJrst55gDH4ujWh8ngLVGqvUQ28oKfxuz+ZfirKBXZMlyzJ1LAB7xD9V3CgBHWTe09OvsW7tM7pklqgubfIWxmmbeEGAy6aALSAud2ycZOz67CCDMfCQq2yzLdgJmj64i1UQd1GY/PtPLOiYVwIc7h7NhoCgvgCTT2Ba8oRNm8cxyPiJfcG+bdhpv5krvrBdRSoxL5i+7zCrrMPr4UDqyhpGmGNvlGhYV99tFTTdAiT8wDvdzJ8jl/NriF5EhO3Agg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=qW8DG1upWDGMpu4muvuP555wvqu23ApP2Vx5L/nM/Mw=; b=JQG4f5h3K5VW13EJu07EPiuRrlOEk7C/y65rKsdH1jRaqBl/AHx4nV1aVm5z3HmcdFedVVvPeuab0mVNvcwvSU9FTacHZ2KIiJkqNhH0FlWE9lcPwEnyCtq9aeUWZEsbFwag3aprOfKhxOTpVMs6hHC1rrYBHcn+cEAjd1AqSIM= Received: from VE1PR08MB4640.eurprd08.prod.outlook.com (10.255.27.75) by VE1PR08MB5101.eurprd08.prod.outlook.com (20.179.29.220) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2474.17; Tue, 26 Nov 2019 08:46:36 +0000 Received: from VE1PR08MB4640.eurprd08.prod.outlook.com ([fe80::3d2a:e5be:952e:541a]) by VE1PR08MB4640.eurprd08.prod.outlook.com ([fe80::3d2a:e5be:952e:541a%6]) with mapi id 15.20.2474.023; Tue, 26 Nov 2019 08:46:36 +0000 From: "Phil Yang (Arm Technology China)" To: Qi Zhang CC: nd , "Gavin Hu (Arm Technology China)" , Honnappa Nagarahalli , "dev@dpdk.org" , nd Thread-Topic: [dpdk-dev] Enable TPH in i40e PMD Thread-Index: AdWkNKK2LjvWoXbiQX6fBoUdzuQxCw== Date: Tue, 26 Nov 2019 08:46:36 +0000 Message-ID: Accept-Language: zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: 07dbbbac-5010-4f83-803c-eda56f3925ce.0 x-checkrecipientchecked: true Authentication-Results-Original: spf=none (sender IP is ) smtp.mailfrom=Phil.Yang@arm.com; x-originating-ip: [113.29.88.7] x-ms-publictraffictype: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: b97e9343-3766-4b93-a072-08d7724d29e1 X-MS-TrafficTypeDiagnostic: VE1PR08MB5101:|VE1PR08MB5101:|VI1PR0801MB2079: X-MS-Exchange-PUrlCount: 1 x-ms-exchange-transport-forked: True X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true x-ms-oob-tlc-oobclassifiers: OLM:4941;OLM:4941; x-forefront-prvs: 0233768B38 X-Forefront-Antispam-Report-Untrusted: SFV:NSPM; SFS:(10009020)(4636009)(39860400002)(136003)(376002)(396003)(366004)(346002)(51874003)(189003)(199004)(186003)(14454004)(478600001)(66476007)(66556008)(64756008)(25786009)(66446008)(7696005)(3846002)(6116002)(790700001)(74316002)(2906002)(4326008)(76116006)(6506007)(7736002)(66946007)(71200400001)(26005)(55236004)(606006)(102836004)(6916009)(99286004)(316002)(55016002)(52536014)(9326002)(54906003)(81166006)(8936002)(5660300002)(71190400001)(86362001)(81156014)(8676002)(14444005)(256004)(66066001)(6436002)(33656002)(54896002)(236005)(6306002)(9686003)(505234006); DIR:OUT; SFP:1101; SCL:1; SRVR:VE1PR08MB5101; H:VE1PR08MB4640.eurprd08.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: 3Yt6bdX461T0DNeeklNaMzved+nQh+JQYbHx6eGCsmYy6kiGUNUUoU547D0v8notZQGO/PyRnhG8LORG29adqYTT5ApumSKRF+LbqGtKEwWJjnZLos3ayH/F6qnn3z4n4Ay0V+C0FyZKjZDaSTdHEN0lU9Cpu99hNQAUoHhNkE9w35FGxUJrCbpUYnAkBT8UsERU68U4YJEXnsCMstmALYQrNZcow990H5bxxOAJSVW30Br8eD9zoHqNz27Kq/mWi6277FpEL+Z72l1aIbPKdkeCJQqwGxLfhtbMrT1WOjzUoXBkRYhc+/DwxCd/MO0TejLZEMoTkEAAA7jHDn0mCw9svtEHRrQO2bjVn7vuWzRIIaVczoRpjkV9w3di2UZ4rBaXwBZczHe7pPGp7UQe+jPulM0Z4n6oSPo6LobXPYFFYDivj1QUkAf3jh3ySnsbqIN6ibANOKW9I1L1NtF3Ch4Vgxw1a2KGeXOYq4EvGrM= MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR08MB5101 Original-Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Phil.Yang@arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM5EUR03FT050.eop-EUR03.prod.protection.outlook.com X-Forefront-Antispam-Report: CIP:63.35.35.123; IPV:CAL; SCL:-1; CTRY:IE; EFV:NLI; SFV:NSPM; SFS:(10009020)(4636009)(376002)(396003)(346002)(39860400002)(136003)(199004)(189003)(51874003)(74316002)(356004)(66066001)(33656002)(3846002)(6116002)(790700001)(7736002)(22756006)(478600001)(81166006)(9326002)(81156014)(8676002)(5660300002)(26005)(14454004)(26826003)(2906002)(106002)(8936002)(606006)(6506007)(86362001)(102836004)(16586007)(186003)(54906003)(99286004)(36906005)(70586007)(70206006)(7696005)(25786009)(6862004)(4326008)(336012)(6306002)(14444005)(76130400001)(54896002)(55016002)(236005)(9686003)(52536014)(71190400001)(316002)(505234006); DIR:OUT; SFP:1101; SCL:1; SRVR:VI1PR0801MB2079; H:64aa7808-outbound-1.mta.getcheckrecipient.com; FPR:; SPF:Pass; LANG:en; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; A:1; MX:1; X-MS-Office365-Filtering-Correlation-Id-Prvs: ff5ccfb6-4046-4fc8-baa0-08d7724d25b9 NoDisclaimer: True X-Forefront-PRVS: 0233768B38 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: r+4u8PK++MAX8lnHGNIo/mTLiv35eknecTai77ZZe07ZTLY3XnmzZk91TqV5nGPsfk5HzlF/m+FApmT3BBnxtEMi3O6ZQQNOYxQ7bfeARA5xPc4Tt5XUCJmCCkPjk1tVRhRJGtVK/57YzXLvdLUgv3CJ4suwULCz32sccouwLtEBg+ZypDt/YfDYyfkyLa7kOYJiXU9iLzpa5lUPdAhLoShClVVPlX69zQPy5nyt6SWYZZdWPIKn/iIdC79dHt4pvKAJvTmWy6TVdTKXg5axnf1NPhQtR187LiVg24a4JqVhwgb5LpjezlsMvdbjRCfWPJu0XjMCdhK1oPRb3jS2BF5QNRkZV/qIynHZ1nkyG0tME58HnYH2lh6thqd+Uj4OUOAWtuJLhiRF5/J1EH3PiQpv/juLhWqXZTrrSvFCisc30Y3AQn6V0m+z6M8UwUmPfgdR/toBZ+dCWXZa+I9+Axe+t+S7F8KnK2USbtqSUWs= X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Nov 2019 08:46:43.6384 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b97e9343-3766-4b93-a072-08d7724d29e1 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0801MB2079 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: [dpdk-dev] Enable TPH in i40e PMD X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Qi, I hope this mail finds you well. I am trying to benchmark the TLP Processing Hints (TPH) feature in XL710 NI= C i40e PMD on our platform. Our test server supports the TPH identification= feature. But It seems that the NIC can't work as expected in TPH enabled mode. I am not sure I did the right operation while opening TPH feature for XL710= . So please correct me if I am wrong. According to the XL710 datasheet (3.1.2.6.2 TLP Processing Hints (TPH)), I did the following instructions to enable= XL710 TPH: 1. Set TPH enable bits in the receive and transmit queue context; 2. Set the CPUID fields in the receive and transmit queue context to off= er steering information; Step 1 & 2 modifications are list below. I fixed the cupid to 3 in this tes= t. ---- | drivers/net/i40e/i40e_rxtx.c | 6 ++++++ | 1 file changed, 6 insertions(+) | | diff --git a/drivers/net/i40e/i40e_rxtx.c b/drivers/net/i40e/i40e_rxtx.c | index 17dc8c7..0b0ffe1 100644 | --- a/drivers/net/i40e/i40e_rxtx.c | +++ b/drivers/net/i40e/i40e_rxtx.c | @@ -2517,6 +2517,11 @@ i40e_tx_queue_init(struct i40e_tx_queue *txq) | if (vsi->type =3D=3D I40E_VSI_FDIR) | tx_ctx.fd_ena =3D TRUE; | | + tx_ctx.cpuid =3D 3; /* The last core on the server */ | + tx_ctx.tphrdesc_ena =3D 1; | + tx_ctx.tphrpacket_ena =3D 1; | + tx_ctx.tphwdesc_ena =3D 1; | + | err =3D i40e_clear_lan_tx_queue_context(hw, pf_q); | if (err !=3D I40E_SUCCESS) { | PMD_DRV_LOG(ERR, "Failure of clean lan tx queue context"); | @@ -2665,6 +2670,7 @@ i40e_rx_queue_init(struct i40e_rx_queue *rxq) | | rx_ctx.base =3D rxq->rx_ring_phys_addr / I40E_QUEUE_BASE_ADDR_UNIT= ; | rx_ctx.qlen =3D rxq->nb_rx_desc; | + rx_ctx.cpuid =3D 3; /* There are 4 cores, this is the last core */ | #ifndef RTE_LIBRTE_I40E_16BYTE_RX_DESC | rx_ctx.dsize =3D 1; | #endif --- 1. Bind XL710 interface with vfio-pci driver and run testpmd. $ sudo ./build/app/dpdk-testpmd -l 2-3 -n 4 --socket-mem=3D2048 -- -i 1. Set PCIe configuration TPH Requester Control register ( Enable TPH R= equester & Set ST Mode to Device Specific Mode) [11.4.5.3 TPH Requester Con= trol Register (0x1A8; R/W)] Before Setting: $ sudo lspci -vvvvxxxx -s 0001:01:00.0 | grep 1a0 Capabilities: [1a0 v1] Transaction Processing Hints 1a0: 17 00 01 1b 05 00 00 00 00 00 00 00 00 00 00 00 $ sudo lspci -vvvvxxxx -s 0001:01:00.1 | grep 1a0 Capabilities: [1a0 v1] Transaction Processing Hints 1a0: 17 00 01 1b 05 00 00 00 00 00 00 00 00 00 00 00 Set TPH: $ sudo setpci -s 0001:01:00.0 1A8.L=3D102 $ sudo setpci -s 0001:01:00.1 1A8.L=3D102 After Setting: $ sudo lspci -vvvvxxxx -s 0001:01:00.0 | grep 1a0 Capabilities: [1a0 v1] Transaction Processing Hints 1a0: 17 00 01 1b 05 00 00 00 02 01 00 00 00 00 00 00 $ sudo lspci -vvvvxxxx -s 0001:01:00.0 | grep 1a0 Capabilities: [1a0 v1] Transaction Processing Hints 1a0: 17 00 01 1b 05 00 00 00 02 01 00 00 00 00 00 00 1. Start testpmd io forwarding with core 3 and start RFC2544 test. Test Result (Use the default XL710 configuration RFC2544 test result as the= baseline ) : 1. Followed the above instructions 1 - 5, the throughput dropped 67%; 2. Skipped steps 1-2 and followed steps 3 - 5, the throughput dropped 67= %; 3. Changed the cupid to 4 (to verify whether the cupid is counting from = 1) and remeasured with RFC2544, the throughput dropped 67%. So after we configured the TPH requester control register, the throughput p= erformance dropped rapidly. Do you know what is the reason for that? Could you show me the right steps = to open the TPH feature for XL710 i40e PMD? Thanks in advance! Best Regards, Phil Yang