From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM01-BN3-obe.outbound.protection.outlook.com (mail-bn3nam01on0046.outbound.protection.outlook.com [104.47.33.46]) by dpdk.org (Postfix) with ESMTP id 49F611B202 for ; Fri, 10 Nov 2017 03:46:38 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=5fnkRHAUASPvCRMUKOGq5heH+Gk0LRWsLW6Mo1nJeoQ=; b=X8xUXmNe3HFbVjIeJWbf8qecLVxFWtCzmlTyVEokqSDSjpTuKtq7qazuxu0s+qMQIq7vycJh75kWaG7vjhOiHSiHJbCaX5RDCNFrouJyJoblNAeJPxJH0sNt8r0vhfweaa8OkAOtEyI4pZmWAU9foAN90Bb9lPfrMxurb8x9vtk= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Jerin.JacobKollanukkaran@cavium.com; Received: from jerin (14.140.2.178) by SN2PR07MB2526.namprd07.prod.outlook.com (10.167.14.154) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.218.12; Fri, 10 Nov 2017 02:46:31 +0000 Date: Fri, 10 Nov 2017 08:16:10 +0530 From: Jerin Jacob To: Jia He Cc: dev@dpdk.org, olivier.matz@6wind.com, konstantin.ananyev@intel.com, bruce.richardson@intel.com, jianbo.liu@arm.com, hemant.agrawal@nxp.com, Jia He , jie2.liu@hxt-semitech.com, bing.zhao@hxt-semitech.com Message-ID: <20171110024609.GA5295@jerin> References: <1510118764-29697-1-git-send-email-hejianet@gmail.com> <1510278669-8489-1-git-send-email-hejianet@gmail.com> <1510278669-8489-2-git-send-email-hejianet@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <1510278669-8489-2-git-send-email-hejianet@gmail.com> User-Agent: Mutt/1.9.1 (2017-09-22) X-Originating-IP: [14.140.2.178] X-ClientProxiedBy: BM1PR01CA0118.INDPRD01.PROD.OUTLOOK.COM (10.174.208.34) To SN2PR07MB2526.namprd07.prod.outlook.com (10.167.14.154) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 88cea538-f4dc-45a5-d847-08d527e542a2 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(4534020)(4602075)(4627115)(201703031133081)(201702281549075)(2017052603256); SRVR:SN2PR07MB2526; X-Microsoft-Exchange-Diagnostics: 1; SN2PR07MB2526; 3:D067RhohunOaMS75yhQ0Df9WqWqGnAJ07O03LGf3mK83SmQXSyDxlK/d3wYFG+/C17r0kqfbTlY42T4mWCnyFKESapaQRumWztXJ7y2g+Zj3y4BrW6V+mMu5brTwG0sbuyA12ybJ09y3Yg9ROEjD1J0ANqzm6747J01MDbH2Lv1umALzB67bTRGuGDu23sw9o9yEVhltqaEhz+qXlNNV00fJkZ6LAdIZbAxe37IyATON5wyBbFv0fP929CEzyxna; 25:u3oL0wVMqdQOvO44/G4emgIgIuOFt/2DQW+EMPTkNV2g+GsXz5FCSkMEowCgqXTzef5ErdEmG3yxhuCakH9PYA9w7SltSiR3K0ctPCxqBh7ZgXaBxDqiTOCKg8cgLnJ/5o59YEip8l3TDsUpq4McuKOF6EL+BU8e3kPOE03pfswfX597ZQprXzzpRL9luLkMIzq7q479kQRpnQ0az3W9x2TNpdPYxH/ixcWUdL/ltyrlSnzh7PAmvEcNqm0NLjed5kkVhonoqZL5fplYpSfcw9eLrexMQLJFJvr/1jMVii2Kj2w2oiNIIphwoVzzDOTp8YZXCsvtuoYZm8Uiuqao6Q==; 31:I5nSNePcElI+eB8yiT9Qq12KnsabFUe7SSrtJ/keMQLwktNDFvF1tFC6Va3PmtrZ7UDpiKmtJHNWecgZKQ9y25yvqms0dTluCJvpox7tgMD4FapUcDhhIazPb1N6Z6sB0IJSR1ffRvnBpeV9xabEckpMA/PXjbQ+UdTgBCcE3c3HoDMtrcHuwWf0de5jIHclmpdxEw+7d0ylY9jHwR/7rJIRHkQqenU2GnW2b9o9qFE= X-MS-TrafficTypeDiagnostic: SN2PR07MB2526: X-Microsoft-Exchange-Diagnostics: 1; SN2PR07MB2526; 20:n5rxE9OFQjBo837doageslwOXboUpownrcLzukQGTdJ+ssN5MGlvAUa2Fy38CLWFd0AjpSAHhdMdAIBNNBjJe71f2I1SF0jZ8StPLNLkGyQ7HHRvKKtW50BUxk4oKlBWY+qkLMQ1VKz8z0BrCiHYFpzzpP3pDt44u5lgWwnDnun8RuHoSOdqWhXz4JI809LKk51RU92CsJ3vr7XaVOvT4flFSvzswB9uXhT+bSzEBDS/KdjqVJNmnZ73YwPigVlb9JkP7KdzrclAo2c2WHprDEr0WIrVvq4RTdhHoFM4xsCZWaASiX693CXjMDdnzzKv3I9obkMuvUWosThMOEGIzjYTzA3jESyXOwCmSpFdfIguoby+1zKPA3MFmGxRmerNiDwpdbM/NkE/cgJJVeybeybfTLLOCvCFN1vWqggGTi/FNV/kzgcxOO03lWv244S6KrkiSFGmnXhwR09Bl3w5a8lhln3Nvx0zBkBH28X55oAEGIhepntkhLE99/cCOtZlpyizXIx4JPGqTzLcHXWqh9D8O2BrwtWp/1ld/e69mtTYRxGmd5K5ARNcivmJuZofnSyUOAUHiNZTSNN5liSP2US6l3CIL4Y4cuFdQt+710Y= X-Exchange-Antispam-Report-Test: UriScan:(158342451672863)(180628864354917)(185117386973197)(228905959029699); X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(5005006)(8121501046)(93006095)(3002001)(10201501046)(3231021)(100000703101)(100105400095)(6041248)(20161123560025)(20161123564025)(20161123558100)(20161123555025)(20161123562025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:SN2PR07MB2526; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:SN2PR07MB2526; X-Microsoft-Exchange-Diagnostics: 1; SN2PR07MB2526; 4:ff6mR6FcRDFYNA3ORz+7AOSRivSAQzvrJ41BIWO6TXpv5OhfeOFk9utGF1Z7mlj4Gjbeyg9/x6ydN2ER48zTAY7vCF2CGsdgH9HxLhsy2+i1Ji33N2sfRSnw0IySAVKNwA/ZgoSQSCxBcBghrYb7HE/n8ySHOjrZyE+ew6nlLF1ldhwrVjE42F6BrRe8VXSxDZ1yIWPDdhgVQLzGZhwABBloatJdFAtleK8SGfZBm+SYBKhUy267217EFv1rRtoD+Wn8hzNgYHaZJC6fsOGPWop8XexCJx/i40AgT6MUYDjclrcSWtNcYIyM8R8yfRoa0D8/6H+5KjjX3O0gxne1jOnPhdsAdj8fwTgOJKpKOm/TGiVQ+bhDv9qGMGnd+g1nuXzf8yc1DfX3X4BWYlBcWYmxvONCtncM9x4oNx9Cw70= X-Forefront-PRVS: 0487C0DB7E X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(6009001)(376002)(346002)(189002)(199003)(51234002)(13464003)(42882006)(189998001)(2950100002)(6666003)(55016002)(9686003)(50466002)(97736004)(23676003)(316002)(16526018)(106356001)(58126008)(53936002)(5660300001)(6496005)(6246003)(33656002)(83506002)(81166006)(1411001)(8936002)(39060400002)(8656006)(105586002)(6116002)(1076002)(81156014)(33716001)(6916009)(2870700001)(5009440100003)(4326008)(50986999)(3846002)(66066001)(101416001)(54356999)(76176999)(72206003)(47776003)(305945005)(478600001)(2906002)(229853002)(25786009)(8676002)(7736002)(7416002)(68736007)(110426004)(18370500001); DIR:OUT; SFP:1101; SCL:1; SRVR:SN2PR07MB2526; H:jerin; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; Received-SPF: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtTTjJQUjA3TUIyNTI2OzIzOmFHdUkzRFVpZVNIajB6bEJ6SjhIakMvTjJR?= =?utf-8?B?MDU3SkhxMG02S21FeWUraHVXZW4yMkxoWURxL3dMdFBtTTFpem5YU0xKZFU3?= =?utf-8?B?aXVEaythbWtjcVFpcjN3ZENKOWMzayszc1A2aW4vZ0hkNTd6ajdoV1JLSHla?= =?utf-8?B?REpNeHNiSGZxaUFHVHlBZGhuUFA0VXZlaEREbTcydHpvVXVrOTROU2U4YkFm?= =?utf-8?B?aVYvVFE2SFluZVlHNWEyOVRIVU9qdW42dzF2UGZEb3Jsc01SaGo4MTB4eG1Z?= =?utf-8?B?eWFkMGZkN2R2c2VqMFk0dWx1aktEUVRLWFVHd2Q2N25mYkhrWWtUVGFsMDhW?= =?utf-8?B?THQxSGp2ME43TlJQNExaakZ6K0NlOFh2dDlDNThtU21ZYjhvSUMzdUcrbVVW?= =?utf-8?B?ZjVwSDN5U2dkUm1waXY4a29DaFNzN29jc2F4M3A0ZnN2R1JBVE9aQmV4YU9n?= =?utf-8?B?N1ZaQkYySnpCTDI1VFFZb29GYTMvTVl0dlBvRVZjQ2daYVMzcXEzdkhFQ3Jt?= =?utf-8?B?aUszU2hIVkhJMEE5dU8vWHBaaDEvTGVxM1lBZHBpL1BnOHRUZ3BOSS9ZYzgy?= =?utf-8?B?VzE0UlJWaHVRc3RLdW5zUGVVNWN4UEdBUjRqK2VNWUsxR2I1alU3clQ5ME94?= =?utf-8?B?Y3I3RXFkcmdySU40U2hYTXludVhJN0hVNHVpTlc0bDIyTXVLVVFELzlyYUpE?= =?utf-8?B?SmI1Tm5WV3VQMDh6RkcxMWI3QWU2QXNuMUQyaTNqdEc2c2RQVlRRYTE2OVhE?= =?utf-8?B?STkzYnpsZG0zVWZ5OS9DWFI2VCtWM3A4NU9LRUgyVkhhWXR3SHJ6ZU9kVjJL?= =?utf-8?B?SlVlU3Q2U1ErSE9NeUJuTm01YWpGaURmR1lyUHRiQlpSUUV3OUF0aDNHWmRC?= =?utf-8?B?UUhUY1pFc0VHbGJEZ004L1hselM5ejVkVmdudEhnWTE4b1loaDY1V3Irc3Js?= =?utf-8?B?WVpyakgvVXFua2lpYVdrSlU4RGhqK0NOZUNYcUV0ZzBBUUZta1Riazlwa2dp?= =?utf-8?B?RWhzK2pXclVkWjg5blQxT0tZbm5yemMyek9QeWg2SG9hTHRiNzJ1K2o1UGs2?= =?utf-8?B?UU16UTFkNzRES0h4WDVCRzZzaXFrc1Z1amx3ZmZSYmRyVGhPTHZVOUZ4VnJ4?= =?utf-8?B?V0syKzJOU3RyUCt2TkNMK1F5aTBleTZTcnZoQUEyQ1ZuTk1WZVRpYVA5M09B?= =?utf-8?B?N2lDVmEyb1pHY0tXb0pzZUljbTV4VTlqZXEwOVowNFUyNjBYVzQ3QzQrMy9I?= =?utf-8?B?Y2h6My9vV3Y1WFdDSkdXWDFoOXNhL1FyOHIxZEpBZlc3Nm1sZ0hBanFHN1lM?= =?utf-8?B?VElRWU1EdG1RR2llK0ZqWEh0SmFxb0hLN3JJOGd3OXl5enIwWlF3ZEZTWi85?= =?utf-8?B?clZDVHNIa2RMa05XTUNSdEFTN2RlSWl6Z0x6UE9MN2dpRk4xT2JOQzFGVEp0?= =?utf-8?B?VVl3eXBCeVJhWjg1bXJ2VENrMHo5a3I3cWhzQ2EvbjZVb1VSaktqTDNSSWpH?= =?utf-8?B?ZUNHdHd5ZlEyWFpzVU5sY1ZQT004eHB5NmFTODBMR3pkLzVhaXJpeWl5MFIy?= =?utf-8?B?S2pmTUtTcWNXQlJ5YmlYUlRzSzNkdXNHQzRodkpWVEtlNVY0N0lTVGdpMGpO?= =?utf-8?B?cVVENmJwUmFremVselpCUWEyenVJRGl6Q1lMdHFsWnAxcGRJMmVoTW9zS1BI?= =?utf-8?B?eGNkUGY0ZjFsNlpKMjhFZk5LREsyaW8wUG04RG5QZTdGUFEwS1ZJLzVGb0lo?= =?utf-8?B?M3BaRzRGRXlReTdYSzR4VHJZSnBRSGNmcDF2N2o0dGF5N3VIeHE2dHF3YWtY?= =?utf-8?B?UWdBSThoWGY4azVIclNPZzk3eTFMaUVoYm9NRXlObmh1S1E9PQ==?= X-Microsoft-Exchange-Diagnostics: 1; SN2PR07MB2526; 6:9BDPUrPqV6TM4o6wA6kjQMiH6RuU6f2sQmtBwA7nEzhTMctpvIyYKwDk5g2+6ROSGNoVVNVTE20w8nwl4ttbglxeWeQ/VgmBmTAuJRRdW5pH9/6cLU1Ll4HoWGLDudjWPcMGQreI3k1L25ISmtd8Dm+/Geum7dPH7oblidKCQJfF8mO12rYQjnWI0v4NnW2ADwbYXQP6zG8cz27Q1NZtLHs5rWVlcuhAGKUu0R7crm18KL6kKJeqHddE9FxR7vZUS5Ix/WzbCdjCLMx9YvgQPLb5sIcv7EMK3Ika/5ayrA79xjuwj7O+ki9RWJlWfSBZaxz12aj49PHE8TTtPyg0u0Y+4FEsuCf3bRMlucRSrG8=; 5:I/bi7r5E8PyNX+30QcagKrQp1cfhF0TgobYyYojnqVn4+XL8vSrzjDBr7K3pIybNDRRontjqNMtk2PD647iFnijb27tDsJLPlxUa4X9lkjIW9jHOR1wkoO0kI+7KPvIhOfLakboCxNfeMM0vOANN6O9/5MOc0Q3KVea7zODabno=; 24:M+1NpX1eqDuEXM4UkDNqP41o3GqXbAo4RvAP8BjPqiuPMZ+E8R04+FD56DxFiO9xNW7oRDnHV8WCCnjTy09jZF+zCtdpjLJlOy/jGTOCarU=; 7:5Da2TjrasFDj0Vp8Bkre0FBD2tSNL7EGUFh/AZD6pM0MQFh+7sxdLw4mA8uBV15tmJEZ3JYfyyWhumJH4H36ZWATE5pFwJT2UEbKEIhw/4D9OJwyNvpObGKPw91lLuhJinp6pyJJVuuZh2mfSMqcOxuBGj5nvTJRH6FYRKfRfOOnz732KkbiO4dgsgSJ0k8RdXAivlpadhJhZ8fhitY2+DcQ8+AJPDazorBATC0fmgp1PyyYR3gJDOxvHSHjLIFs SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Nov 2017 02:46:31.5505 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 88cea538-f4dc-45a5-d847-08d527e542a2 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN2PR07MB2526 Subject: Re: [dpdk-dev] [PATCH v5 1/1] ring: guarantee load/load order in enqueue and dequeue X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Nov 2017 02:46:38 -0000 -----Original Message----- > Date: Fri, 10 Nov 2017 01:51:09 +0000 > From: Jia He > To: jerin.jacob@caviumnetworks.com, dev@dpdk.org, olivier.matz@6wind.com > Cc: konstantin.ananyev@intel.com, bruce.richardson@intel.com, > jianbo.liu@arm.com, hemant.agrawal@nxp.com, Jia He , > Jia He , jie2.liu@hxt-semitech.com, > bing.zhao@hxt-semitech.com > Subject: [PATCH v5 1/1] ring: guarantee load/load order in enqueue and > dequeue > X-Mailer: git-send-email 2.7.4 > > We watched a rte panic of mbuf_autotest in our qualcomm arm64 server. I think, the above text can be improved. Something like below. Fix for the rte_panic() in mbuf_autotest on qualcomm arm64 server(...SoC name...) Root cause: > In __rte_ring_move_cons_head() > ... > do { > /* Restore n as it may change every loop */ > n = max; > > *old_head = r->cons.head; //1st load > const uint32_t prod_tail = r->prod.tail; //2nd load > > cpu1(producer) cpu2(consumer) cpu3(consumer) > load r->prod.tail > in enqueue: > load r->cons.tail > load r->prod.head > > store r->prod.tail > > load r->cons.head > load r->prod.tail > ... > store r->cons.{head,tail} > load r->cons.head > > In weak memory order architectures(powerpc,arm), the 2nd load might be > reodered before the 1st load, that makes *entries is bigger than we > wanted. This nasty reording messed enque/deque up. Then, r->cons.head > will be bigger than prod_tail, then make *entries very big and the > consumer will go forward incorrectly. > > After this patch, even with above context switches, the old cons.head > will be recaculated after failure of rte_atomic32_cmpset. So no race > conditions left. > > There is no such issue on X86, because X86 is strong memory order model. > But rte_smp_rmb() doesn't have impact on runtime performance on X86, so > keep the same code without architectures specific concerns. > > Signed-off-by: Jia He > Signed-off-by: jie2.liu@hxt-semitech.com > Signed-off-by: bing.zhao@hxt-semitech.com ➜ [master][dpdk.org] $ ./devtools/checkpatches.sh ### ring: guarantee load/load order in enqueue and dequeue WARNING:BLOCK_COMMENT_STYLE: Block comments use a trailing */ on a separate line #58: FILE: lib/librte_ring/rte_ring.h:414: + * memory model. It is noop on x86 */ WARNING:BLOCK_COMMENT_STYLE: Block comments use a trailing */ on a separate line #70: FILE: lib/librte_ring/rte_ring.h:527: + * memory model. It is noop on x86 */ total: 0 errors, 2 warnings, 22 lines checked 0/1 valid patch ➜ [master][dpdk.org] $ ./devtools/check-git-log.sh Wrong tag: Signed-off-by: jie2.liu@hxt-semitech.com Signed-off-by: bing.zhao@hxt-semitech.com With above fixes: Acked-by: Jerin Jacob