From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id CE32BA034F; Mon, 7 Jun 2021 23:36:17 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 56AA54068B; Mon, 7 Jun 2021 23:36:17 +0200 (CEST) Received: from EUR04-DB3-obe.outbound.protection.outlook.com (mail-eopbgr60060.outbound.protection.outlook.com [40.107.6.60]) by mails.dpdk.org (Postfix) with ESMTP id C0F2F4067E for ; Mon, 7 Jun 2021 23:36:15 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=1D7EQ4+KHLYfy+l8i842XaNY85WqawtRQtrAW8+Gesg=; b=F7qV/oWWUN0lSZKdDtwVDuJZPLdtdCYWrZ0ok2ErOncKw4wMLQghi3mDXK06s+gVSQGQpDijnZ21EzhnZt2k0I97dzuzpBrGDreHnGQwaYbTi6mSI5RlMk2KK/3pyQVCHV5nPOW+3S40XFyYhzbWoLZ22CagyxezUQnU+e5G0UY= Received: from DB6P18901CA0009.EURP189.PROD.OUTLOOK.COM (2603:10a6:4:16::19) by AM7PR08MB5432.eurprd08.prod.outlook.com (2603:10a6:20b:10b::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4195.20; Mon, 7 Jun 2021 21:36:14 +0000 Received: from DB5EUR03FT053.eop-EUR03.prod.protection.outlook.com (2603:10a6:4:16:cafe::67) by DB6P18901CA0009.outlook.office365.com (2603:10a6:4:16::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4195.20 via Frontend Transport; Mon, 7 Jun 2021 21:36:13 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dpdk.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dpdk.org; dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by DB5EUR03FT053.mail.protection.outlook.com (10.152.21.119) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4195.21 via Frontend Transport; Mon, 7 Jun 2021 21:36:13 +0000 Received: ("Tessian outbound bf434e582664:v93"); Mon, 07 Jun 2021 21:36:13 +0000 X-CR-MTA-TID: 64aa7808 Received: from 85e6b2e5cbac.3 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 954D295F-6189-4285-A013-A909D5363BB4.1; Mon, 07 Jun 2021 21:36:07 +0000 Received: from EUR05-VI1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 85e6b2e5cbac.3 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Mon, 07 Jun 2021 21:36:07 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Ueopbtbtuj5M4qsuBQeVD5CARw3ogwNJpNMbRqKQA60oAbjenbsu4a1/9SOZyf/hQseEMKDhsZ/W3Nlm9nsrgSYRz3vhlsznoRQqv5n8N5o62b34xdLFvFhBFW748grlpSzn0UFDeT1BzYklkvV/wQ1HANha0b3znOn1Yu5wbW8kFLDfWGNE0p2xjGIokC9LD4+lmQqnOdHtRF0lBcMp0b2jYwCDJ5fMd5vQueynr5FqA3mrZstCoZVNpyCdJTwfa/oDY7UMaV+pI8sNwn3BOIeNdVdtzhR7qMOVpw7Fv73IvmUBaxwZvyOXy1nnbA0QxEZVf9IhnHbH+fW0J5s+Ag== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=1D7EQ4+KHLYfy+l8i842XaNY85WqawtRQtrAW8+Gesg=; b=IjOToaDddoSRocBQnIN410L/mOd3vs57xbWy36MwwdARiz7JNAN9QgPq6Pwte06CFUFij7w026ROo3TvIEGLMQHWmZ33BGNwJIHoqw2hKbqco+w6sNXqf5SeIg2H07xmxLhEy6BXVqe88z8+SncgWxsY5qacv/WRWin3gTDK4qZE04MUlSOOn+FvYF2uZwbKiKjeZ5NAIfooiRodwU3ODZEMqV+P4liGp9FMnV003oZUFM+5jkj92gg+5hrO5xJcQGaiPcw4YoUa9ZA8hWg+14FsRPHSbE4dZHLvJ0elaOCqHTryesMktzTahNTrQQZz00pJz8Z4fMM5gzXhGMvoNg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=1D7EQ4+KHLYfy+l8i842XaNY85WqawtRQtrAW8+Gesg=; b=F7qV/oWWUN0lSZKdDtwVDuJZPLdtdCYWrZ0ok2ErOncKw4wMLQghi3mDXK06s+gVSQGQpDijnZ21EzhnZt2k0I97dzuzpBrGDreHnGQwaYbTi6mSI5RlMk2KK/3pyQVCHV5nPOW+3S40XFyYhzbWoLZ22CagyxezUQnU+e5G0UY= Received: from DBAPR08MB5814.eurprd08.prod.outlook.com (2603:10a6:10:1b1::6) by DB9PR08MB6732.eurprd08.prod.outlook.com (2603:10a6:10:2af::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4195.23; Mon, 7 Jun 2021 21:36:05 +0000 Received: from DBAPR08MB5814.eurprd08.prod.outlook.com ([fe80::f15f:821c:74c5:2482]) by DBAPR08MB5814.eurprd08.prod.outlook.com ([fe80::f15f:821c:74c5:2482%2]) with mapi id 15.20.4195.030; Mon, 7 Jun 2021 21:36:04 +0000 From: Honnappa Nagarahalli To: "Zhang, Qi Z" , Joyce Kong , "Xing, Beilei" , Ruifeng Wang CC: "dev@dpdk.org" , nd , Honnappa Nagarahalli , nd Thread-Topic: [PATCH v1] net/i40e: remove the SMP barrier in HW scanning func Thread-Index: AQHXWt6+SyccfibsvU+3xAtLgQFgXKsHQPmggAFjrICAAEpKsA== Date: Mon, 7 Jun 2021 21:36:04 +0000 Message-ID: References: <20210604073405.14880-1-joyce.kong@arm.com> <561469a10f13450bae9e857f186b0123@intel.com> <2cb94e2e1bf74840acaadc389b4745f5@intel.com> In-Reply-To: <2cb94e2e1bf74840acaadc389b4745f5@intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: AD4EAB4A7DDA0D44B0DC104D3A393E8E.0 x-checkrecipientchecked: true Authentication-Results-Original: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=arm.com; x-originating-ip: [70.113.13.105] x-ms-publictraffictype: Email X-MS-Office365-Filtering-Correlation-Id: c63dca47-46cf-4fc0-84d4-08d929fc4650 x-ms-traffictypediagnostic: DB9PR08MB6732:|AM7PR08MB5432: x-ms-exchange-transport-forked: True X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true nodisclaimer: true x-ms-oob-tlc-oobclassifiers: OLM:4714;OLM:4714; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: eq21VQqj6fVNy8hUdPQKZTsudPd5AQAxX/m9EWHhL3FBmbIse0fmcy6perX0BKntjRYoNAKZLQk95i7RLkGYeyRrv22kNx14xZ1oj62LOYcCoX50SCVXf21Jqj5RfSsLK9Ip/B9NserYAwG8co7mMTcb/HRCK0YwE3ORPSjGwXTPpBGlDTu/zJvu/0HxObGY3FfxS26YyeeU1sGo1DfnCQ/iIwg/xfubjilCV1qzu6vIOSKDhUZ5/d1GWtkGoMggsx6EwTP/ApMv04LWqT9m/tx8WccBYyL4KZvPdCKTvgNKd/VJqefiaQhYLcmSUL+U41PCAbuPstIqlEtF8ckqFAcDqGjK8vMUziwu+qeNuLiNuqHPLKfmEoRndrZiVmPysNr4Nhek8nfVCMPj1HINRFqtR5Is6wUbLYGlBt6ZYTqizIS/vj5DMkoCwVvUIpLYT3kux0nvhNI/N4Cv2lg+DBCcCj1PRg3CYDbtPQjL6SjC7S38vwBXTUgRc1LiL5png6S2BUZYYZB1F2SSJm8dZC2g6DR8EZ8FedD3CraKULoReXDFCqaejhfw5vZpOwXZ8paBav6mRqZyRYrC5RcgKva0Gr/3ahHDjsk5UZrJZMw= X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DBAPR08MB5814.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(346002)(39860400002)(136003)(376002)(366004)(396003)(2906002)(71200400001)(5660300002)(316002)(33656002)(26005)(186003)(9686003)(55016002)(8936002)(86362001)(110136005)(8676002)(54906003)(6506007)(83380400001)(7696005)(6636002)(4326008)(122000001)(66946007)(76116006)(66476007)(66556008)(66446008)(64756008)(38100700002)(478600001)(52536014); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: =?us-ascii?Q?6qQM77fkCa0NaW5HyBxjgkFgvCISmUjFwWmW0d1bvFUiKimd9CdqRUQYKPNQ?= =?us-ascii?Q?h1Plu9itrpdfC02853hO0A1uz+TH+vCWtprm1/8B+3seXYC2KEQUlv4SxcAd?= =?us-ascii?Q?AfR25ITTKDHJFNU6fa9EM/mhDgKXcqdzSWFpzbm0+8S6YpPuhKrqn1ucRw98?= =?us-ascii?Q?zeplEayvxFSjj9/Q6yh+pJLIqLmxk0VcQSb7myY8JK1pSOQjMvuIj1JrkX4N?= =?us-ascii?Q?q9u65NjbmAZfsz8f6h6xTfdt9Bg6n3/+V39PhzEp9bIa66ZJdHGpuv5a/4XD?= =?us-ascii?Q?jmqIAf3k5HiWqmMqP/jEUDep4yuWe+RwBxuyz6taT7yzXhtspepqHXyy0tLa?= =?us-ascii?Q?OhVMk/sRfmwFPNbt3glQ7kUYjgEXg5cdzo1Ov6ia4/e67wAnuQ9CcSrg83Yb?= =?us-ascii?Q?L67x9toQwoNxCX7MKVy2gDK9+B1bbGAPGw4yj2J9MwXS7L2kmxXVTb1fS14s?= =?us-ascii?Q?YfGPMWTw/j196GIs/ICQxV9FGdQtS2/xBuUN9OO/tlSWS6BQJWLHQL/PhbDv?= =?us-ascii?Q?sIcbAIyGwJxHWx8q8C0plz5B6Glv41hcKAsXEJNtcZ0ELsXY5uYV9JU1bOZb?= =?us-ascii?Q?pfLiYCgRwC02sqiyX7WC14R3tgzzel3L6tcqAXkQQCWoyAExHbXwssCjx2kS?= =?us-ascii?Q?ncBSV0H0sJTL4c/CCi1PZgqxaIN/QrW4Fx5NeuNxB3mFC6O1zOB8Lx5QfAWd?= =?us-ascii?Q?2S528BALAsPUXPSt/IcV32SoVoT0EKaLOuopLc7l8E3P6lz/h+7FyxvEMKVS?= =?us-ascii?Q?wjEK6LG9DYp19RehAbmz7B4IdhjUt1VPiD2Jy8MSjnbtB7IQRdWOEljTpsyr?= =?us-ascii?Q?BgwO+ToLx1dFSlLQjJ/aXs6g+6hArU+7+r5rOpmJc6JYdurT7I8af3OLLwpu?= =?us-ascii?Q?vzwoZfWfZd6kUKeUaF7/HrtKQa+ZGGKOFdSCviD1OZIpEmUwnKt92RLBnx1O?= =?us-ascii?Q?ixnhihLkcc60abrfKPkgNGbyEJm+j9N9gCCOCQoWpzgs0rA4hwwQHxhKILPi?= =?us-ascii?Q?hB5enRf1YGBG5Z3upZlWlRzdn7Wpp7+wtJh+u/yGqpcp/lTrEiO0SyqcOoAY?= =?us-ascii?Q?xkm7MA43E99UmUAcgFixKg4Z1Lblzh+VdsH6vnRddJS3oG2Zo30bYm3uE2Qi?= =?us-ascii?Q?AZalArD7cHnIijIxvUTYtuwHof5mTcw0yUiXehf7feUo0NEXQ5MCnjw92AqL?= =?us-ascii?Q?uvb8KcximYh1WRGTAwiyyQXybumFYee9gCuDV8+FoCoUIM+xW+bmvFlOCAp4?= =?us-ascii?Q?UdexB18tvMARShFNvvU0atg0hUr0IK7m2GY/RtHXRTModxlMflWPWexR13UL?= =?us-ascii?Q?MFqI1epMi55/ThlHc8A+1xb+?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR08MB6732 Original-Authentication-Results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: DB5EUR03FT053.eop-EUR03.prod.protection.outlook.com X-MS-Office365-Filtering-Correlation-Id-Prvs: 132400e3-ed1b-4653-5a81-08d929fc40f6 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: sO86o2BBvKpHCPDYPS5lr6MONAvJrdUHI4hbdt3Vcn6l+V+cIXzU9b4MBTFj2E/8o9/0e4Hjk6tZKoVjarF2UGjGlVF7C/oiJa/Dz3L/pjZkKYiwQLlOeQ3lqoIwCocxsy0T8dGxjA84f5O487YH40n3oe/M0A/ZOND9lZabnyUiTWQzSsMXEKifSMNx8eNvbmGJyyIRUB/nwvLnXejuNRRQTzJ4pxa24Og6IjApVnNP3uK/eO4SZH9ggTtfwBWQwL3a/42jRnhRzpLHJskNWawfZId3hW+McLuPoyBs/nk1ClN9pFGuFNIoyMkW9guRZuAA9+H53L7Q48aT83LjGWKGlklPXW9xD+0+RergzK8WXxjlhs9rGCcX0zji7tOXyTkIujrKAF7EuaWy8KCAehL9negjjgL+p2/m2xBIx9QZhDrT3tRNcQhrvndThvvvHTnTjGkQOU/upDVQm2qO863aprvXxVYRtqBJQooFRDxVnXrTB71cWeThYqFS4Ylg/SF5JR8Wqmm97yu82hsTQAIhD7eweH5qHNvmMnn2HRmwBTM14eEPRYh8x6OhDEnK+tpmtztLBIV44nd7K/EeGY7GmBnvObiYskrICYWS7nPo06DcbRQjNYNRu9tvNPPb3p2LGbo2OwSfiiRbTP3AsQ== X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(4636009)(346002)(376002)(39860400002)(396003)(136003)(36840700001)(46966006)(55016002)(70586007)(70206006)(8676002)(81166007)(5660300002)(7696005)(54906003)(83380400001)(52536014)(8936002)(6506007)(86362001)(336012)(9686003)(356005)(26005)(82310400003)(186003)(33656002)(47076005)(4326008)(2906002)(82740400003)(478600001)(6636002)(316002)(110136005)(36860700001); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Jun 2021 21:36:13.7745 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: c63dca47-46cf-4fc0-84d4-08d929fc4650 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: DB5EUR03FT053.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM7PR08MB5432 Subject: Re: [dpdk-dev] [PATCH v1] net/i40e: remove the SMP barrier in HW scanning func X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > > > > > > > > > > Add the logic to determine how many DD bits have been set for > > > > contiguous packets, for removing the SMP barrier while reading desc= s. > > > > > > I didn't understand this. > > > The current logic already guarantee the read out DD bits are from > > > continue packets, as it read Rx descriptor in a reversed order from t= he > ring. > > Qi, the comments in the code mention that there is a race condition if > > the descriptors are not read in the reverse order. But, they do not > > mention what the race condition is and how it can occur. Appreciate if > > you could explain that. >=20 > The Race condition happens between the NIC and CPU, if write and read DD > bit in the same order, there might be a hole (e.g. 1011) with the revers= e read > order, we make sure no more "1" after the first "0" > as the read address are declared as volatile, compiler will not re-ordere= d > them. My understanding is that 1) the NIC will write an entire cache line of descriptors to memory "atomic= ally" (i.e. the entire cache line is visible to the CPU at once) if there a= re enough descriptors ready to fill one cache line. 2) But, if there are not enough descriptors ready (because for ex: there is= not enough traffic), then it might write partial cache lines. Please correct me if I am wrong. For #1, I do not think it matters if we read the descriptors in reverse ord= er or not as the cache line is written atomically. For #1, if we read in reverse order, does it make sense to not check the DD= bits of descriptors that are earlier in the order once we encounter a desc= riptor that has its DD bit set? This is because NIC updates the descriptors= in order. >=20 > > > > On x86, the reads are not re-ordered (though the compiler can > > re-order). On ARM, the reads can get re-ordered and hence the barriers > > are required. In order to avoid the barriers, we are trying to process > > only those descriptors whose DD bits are set such that they are > > contiguous. i.e. if the DD bits are 1011, we process only the first des= criptor. >=20 > Ok, I see. thanks for the explanation. > At this moment, I may prefer not change the behavior of x86, so compile > option for arm can be added, in future when we observe no performance > impact for x86 as well, we can consider to remove it, what do you think? I am ok with this approach. >=20 > > > > > So I didn't see the a new logic be added, would you describe more > > > clear about the purpose of this patch? > > > > > > > > > > > Signed-off-by: Joyce Kong > > > > Reviewed-by: Ruifeng Wang > > > > --- > > > > drivers/net/i40e/i40e_rxtx.c | 13 ++++++++----- > > > > 1 file changed, 8 insertions(+), 5 deletions(-) > > > > > > > > diff --git a/drivers/net/i40e/i40e_rxtx.c > > > > b/drivers/net/i40e/i40e_rxtx.c index > > > > 6c58decec..410a81f30 100644 > > > > --- a/drivers/net/i40e/i40e_rxtx.c > > > > +++ b/drivers/net/i40e/i40e_rxtx.c > > > > @@ -452,7 +452,7 @@ i40e_rx_scan_hw_ring(struct i40e_rx_queue > *rxq) > > > > uint16_t pkt_len; > > > > uint64_t qword1; > > > > uint32_t rx_status; > > > > - int32_t s[I40E_LOOK_AHEAD], nb_dd; > > > > + int32_t s[I40E_LOOK_AHEAD], var, nb_dd; > > > > int32_t i, j, nb_rx =3D 0; > > > > uint64_t pkt_flags; > > > > uint32_t *ptype_tbl =3D rxq->vsi->adapter->ptype_tbl; @@ -482,11 > > > > +482,14 @@ i40e_rx_scan_hw_ring(struct i40e_rx_queue *rxq) > > > > I40E_RXD_QW1_STATUS_SHIFT; > > > > } > > > > > > > > - rte_smp_rmb(); > > > > > > Any performance gain by removing this? and it is not necessary to be > > > combined with below change, right? > > > > > > > - > > > > /* Compute how many status bits were set */ > > > > - for (j =3D 0, nb_dd =3D 0; j < I40E_LOOK_AHEAD; j++) > > > > - nb_dd +=3D s[j] & (1 << > > > I40E_RX_DESC_STATUS_DD_SHIFT); > > > > + for (j =3D 0, nb_dd =3D 0; j < I40E_LOOK_AHEAD; j++) { > > > > + var =3D s[j] & (1 << I40E_RX_DESC_STATUS_DD_SHIFT); > > > > + if (var) > > > > + nb_dd +=3D 1; > > > > + else > > > > + break; > > > > + } > > > > > > > > nb_rx +=3D nb_dd; > > > > > > > > -- > > > > 2.17.1