From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM01-BY2-obe.outbound.protection.outlook.com (mail-by2nam01on0056.outbound.protection.outlook.com [104.47.34.56]) by dpdk.org (Postfix) with ESMTP id 2D8FA1B349 for ; Wed, 8 Nov 2017 08:27:54 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=GhSw7QWzwiHWa5G6eY/XqdTfDDDa+MLkfcFudcPShaA=; b=Keqi7O0uhWCgKDfMepqg/ipssu63B6qrJSmA9/lCVGmNARLNGIIhm3jqzD3x6XuJJZG3VwKDdqXDnVgd55jCDTrfHQRqimwAiGtn12cOrNt04pQCcGRhySGDnZHoSy3wJcgKDcSfEnaaB+wUlwXM+046g9/O4ysAFDoxkAgcgUo= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Jerin.JacobKollanukkaran@cavium.com; Received: from jerin (106.200.244.197) by CY1PR07MB2523.namprd07.prod.outlook.com (10.167.16.14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.218.12; Wed, 8 Nov 2017 07:27:48 +0000 Date: Wed, 8 Nov 2017 12:57:32 +0530 From: Jerin Jacob To: Jia He Cc: dev@dpdk.org, olivier.matz@6wind.com, konstantin.ananyev@intel.com, bruce.richardson@intel.com, jianbo.liu@arm.com, hemant.agrawal@nxp.com, jie2.liu@hxt-semitech.com, bing.zhao@hxt-semitech.com, jia.he@hxt-semitech.com Message-ID: <20171108072731.GB21113@jerin> References: <1510121832-16439-1-git-send-email-hejianet@gmail.com> <1510121832-16439-2-git-send-email-hejianet@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1510121832-16439-2-git-send-email-hejianet@gmail.com> User-Agent: Mutt/1.9.1 (2017-09-22) X-Originating-IP: [106.200.244.197] X-ClientProxiedBy: MA1PR01CA0100.INDPRD01.PROD.OUTLOOK.COM (10.174.56.144) To CY1PR07MB2523.namprd07.prod.outlook.com (10.167.16.14) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: e7637f01-680f-42c3-4b23-08d5267a38be X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(4534020)(4602075)(4627115)(201703031133081)(201702281549075)(2017052603199); SRVR:CY1PR07MB2523; X-Microsoft-Exchange-Diagnostics: 1; CY1PR07MB2523; 3:KmDZgNSoish3lj+ReZREgH4xDpG7m/Q8UvlPGSIMHBZUhi6ucfSGQGdMtRNK8vDmoawqGhJjSsB9jF50DqcyAaxZSuQDczLGUyKXLBJ3FNcN804MBVkrXKiKoRzExfl57HvrvqN3+YQ5N7aNlouxV1innwu7Yxf5wsfb7QhHMhs1hIrLD6KmZPgbMdp0ujEGFB3NnpsYEOmit9BcrJPHJcIopQ/ijNWKFZflmli47H9pY50b2YIJeiWIaiAfPn+C; 25:EhJFYiISqopkx4x8HzuJ2E48yHfu+aZxz/Qppdfyhdh9ckwgpMQ12u+xdY2B1v5J29o79x4nOh8oJj2KVUD6d7CBdPamxIogwEcAbFO/jTxNr/e4Hlw96ApmRxXAbdkuss7sW6qukl0yqCyULSQN/LExgZLkV8abKzBD8uV2kYIPi8RyvIRu7/P91U+3nbHC1gk92yBLU6v31VYecIg18b1h/C93a8kUMjuutbQqPRHTsd4hMAllpqshrljSPgHv2yTD3C+N/y6arb7wRqPsQjU0ABwZty+FUryfOkDyk2SZ3mI6l5EEAGZHpDKgAnHCfQLiOOwcSTVzA/jjVXX+ew==; 31:79TR0H7Mjl2/CESQbwEEBGbp3bBvHmz6OHCjKVzSpqwI2FL2bQlC5Ttrz2PoRr5hyxaZ+854EjEG/SyjSjgc29JC1O3/2zPu86zfb37giBf2aQUw+Zwzrzh7IleLl31DSCYjRCn8cfeV0VsDGUkLbnAjlAz7H1xFCjhYfX22ICE0MbUHqQCmwh3CDJFBnUOmYLbUpe+TuxJuPSXucSXH8eOKV0PKvWAl267Zf0CnBlM= X-MS-TrafficTypeDiagnostic: CY1PR07MB2523: X-Microsoft-Exchange-Diagnostics: 1; CY1PR07MB2523; 20:3fMwgTNe30JzCKuE79L6/KGgaKJLQMyqCBR53QAa+l4i4Fnkt7tqNmHyoqw00LACVzljAl0fa8kX9SpPuJkOkevFAZBQqH3fE2uc7k6LboNyvkdlRdgVlY+lO3LQ+G3/Kj2pSL8fsrhB4jgB0Rf1E5b5SHTD2AwH+QbZ9zWLNzbzdLXIcALS7p5DfheF4nSk2uj80MJs9hy/IDjbZn6ffDdSezTq5ZdLCajxynWWdzdl60WpdrELk1pS+s9v4qcYnBi6664fHyqywek88mb37+D98eSCsZt9OQ3PT3TAKF8cwYhkAqhsLD7kbJRN3DYDAWAMQ0kbQXyGppJdOVaVunY1mct18uHx2gGCsXxfR7Ssfw3Pw9lDwD8bfersPXalc5VWmtqrUgrqnVAeZmVt7fCJXWHZOCX7DAWqnjWF9vKPAQPMCvpv38EvZpeMQ5QFGINh9bgYL4FrQ7jKVwI/EriIlLr+KVWFoiLcnEWzCwaD1nG6GB/CXIQYpKGXh5Hz3t/vKotLONCtnrUUveQQ7Pnly7Z2DLayQoeueVN0IGW6r6dLD/AqcQ1EUD82VgzI4Q/EqFQL4ZE8zMubXHVcBGvrNaZe7elu7EohqMCY2p4= X-Exchange-Antispam-Report-Test: UriScan:(180628864354917)(166708455590820)(185117386973197)(228905959029699); X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(8121501046)(5005006)(10201501046)(3002001)(100000703101)(100105400095)(3231021)(93006095)(6041248)(20161123564025)(20161123560025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123562025)(20161123555025)(20161123558100)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:CY1PR07MB2523; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:CY1PR07MB2523; X-Microsoft-Exchange-Diagnostics: 1; CY1PR07MB2523; 4:jtKWbFFBMo8b2TC3LC9N3eS3Zl6rgxCEpi4Z+xJMpILA+sAYluPLcYPisi2/4vBEodkpWuODAZZOzCCxTTE6oIbuWJgL1lRxPI2JDnKUhzNNBB7jGBa7Xj/Pj26EOh8fS692DqdigQpdzVixHPwytp08s1yVICN37CDArlrBB+MGS3dTu0kZ9dTjDQ0ApFYq5FgG+u+Ykt6DooM7s1K8tSb3ajhvxkvwKSu+2VZ0DYTfmOr6i/5sxY+5DsvpiC9lJhJVkqZZJoJTud61ob8onVgVYtIFXIzMl3kyeWL9xuTgszPPrh663E2Ef/t0FDN9oHOS/tMWggLJMuZSk0+7O1Gy8DZR360EoGOw0jCwRS7KbqiEMi5U9VEoRIPAGBi46onZltD26u9wuApCIN1pQB+YsmmUCMw4Zl/OezAkGCk= X-Forefront-PRVS: 0485417665 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(6009001)(346002)(376002)(189002)(13464003)(199003)(7416002)(25786009)(6666003)(1076002)(16526018)(305945005)(229853002)(8656006)(966005)(23726003)(83506002)(72206003)(68736007)(7736002)(1411001)(81166006)(81156014)(8936002)(8676002)(97736004)(3846002)(189998001)(6116002)(2906002)(50466002)(478600001)(5009440100003)(53936002)(6246003)(316002)(6306002)(33656002)(66066001)(4326008)(16586007)(105586002)(106356001)(39060400002)(53376002)(9686003)(33716001)(6496005)(50986999)(6916009)(58126008)(76176999)(55016002)(54356999)(42882006)(47776003)(2950100002)(5660300001)(101416001)(18370500001); DIR:OUT; SFP:1101; SCL:1; SRVR:CY1PR07MB2523; H:jerin; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; Received-SPF: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; CY1PR07MB2523; 23:x+D1LBMEtS5Qjn/xQTbSiqgKQExbhsyrPOKG9waBb?= =?us-ascii?Q?4lJrOcyLLCsda/RUXhUrKCCR1t2artVFeNweUyeCrTtRILaE57+6/vpSnhKO?= =?us-ascii?Q?4VsMiK3qZftRXWyQDZi4fcTtHK6uDbBAUD9/mBc0sB48jYz+CR+ACdmruwbT?= =?us-ascii?Q?ZQ193sluxzo8IVFCY0wnGF/j32vOvctST5mTfKmbuv6/yaPslsOBWyBGDzlw?= =?us-ascii?Q?ckZ+r1nkzufO2NibnhKxFYwxbU2B0lP9/PpbPvQDqNPyd/e1CnkHFbNjSzd7?= =?us-ascii?Q?5+qQNCtHNIG7NBFmoPFiDzcYgYsGLGKe3npnQNofUqihG5nS3oz6t08aNaRm?= =?us-ascii?Q?uupX8P4juOuslWs3RnCcBXqQuGLXXtVkpqYAdUgqtokF11mv+l8NcAL5bn0U?= =?us-ascii?Q?PS5Sv5ae2xakIOLhVTPgzpiZCVAVzrUxN5ErN0yB5QiObHdn4f3cy6zXCknw?= =?us-ascii?Q?PVNVsZEbC4GXNxEjuezyzCszURrPmxFzDvzZjpfHRv0ta7g18VkdGa05KwxG?= =?us-ascii?Q?EUFvYKJyplYG1RBuhHGDBvlDWhtzOyriny+c1y6BzFOc1wgzr942nKQjKdIB?= =?us-ascii?Q?ikKRrmwZwNVukzJIbWVGZQW+tNVkHoqBzYxG/RCDInfDOXc1tUbcIHvex1/C?= =?us-ascii?Q?7wYmN8tTLEh/x/FlFzXLDM8U4iyh9An/Su+hU1JUeXcaxlgN7wcfiufH7xnk?= =?us-ascii?Q?t5xnUAlqni84OT+Ocgu+HPay+slWFSe795Hz2uQAsFDcomtYdF1GmhVHTjSL?= =?us-ascii?Q?X6hzVUzPwypjgyk0n9AxcgzxZV4DJtyRR/MUxXvoXY8hFAV2vVG4CBZ/htJJ?= =?us-ascii?Q?ZrDKvGm1fmYdCsIbhV7U1v8YXtuQrYoon2N/OiCY6EkbBFvQcCw/pQGtPJzO?= =?us-ascii?Q?6iHI6i6SAgR548pToxtQD3zrA1ISxiuQRQOP0JMeLugt7sq+bY1YgOByY9BE?= =?us-ascii?Q?AB2np9JOBhxYD6SYcnrJwybO9P778XeMppyAkDiXhRwaLyMyfOrMC/PJv0nm?= =?us-ascii?Q?Y+kiDLjVEYXHkNNTeMsZD2EBbuKy6KcDSpdC2zEGOeErCemE9WN+B9FG125O?= =?us-ascii?Q?yt4puuEzbqQwSwqIR5+71dFrX0aG8uAoRyeDgv3hmcJ2m++4Gp4DV7dCqbj3?= =?us-ascii?Q?jr4Wh0efCHSDG2KGROSNThrrjVsOn3gCx12k4MZeIjtqGzteBTid9XjxRPK3?= =?us-ascii?Q?e1fKm5FJl6YPuOd1qRV76jB133qdom2CUV8yhWmNsd9P17qFcaMilJXwseJ/?= =?us-ascii?Q?PfcX9z7Vfm5oOC5er8Ls991x2/AF7owb44fEnscaUWn6NbZqTNJCbQgVbell?= =?us-ascii?Q?HWRD7JeQtobIn1UKzyK1rnRCI8J4LaU7I9lUdk/5qsSh2TzCWltWehf0Wh08?= =?us-ascii?Q?EJhF5UoTKYHfv8yaaNoKLsfYro=3D?= X-Microsoft-Exchange-Diagnostics: 1; CY1PR07MB2523; 6:1JaT7OT4ghfK+TVOPH3o8nfLtyfpyA9VOGCkqCZlLSW0OZ6u1Vtuf5qdZC1ivQEHnYzEZ/PS49/3m066Flvf9Tu0sM1dKxydYZThFlQ3jtQSw9UGV+pSqhcJQK7gYVjhY8BTD/Ha19LX7v66XmdtULcMSbozwLQsSobHWyZ23Ht9ayc1eCK745PK+gNvUrtW96mYFrRneCkaysXeCO6A6SyIymK7nUbjDm+0Bd7RIpzWmA/qQ3AdIak+u+7c0YrDoG9vCkjGY0H8F/+4JFMf+M9AYUgweHrxGR2SwyQW6MZjwM59ZHDxPsEJjdvHmh83BNRaFjfqsWXK+iRAswbHYrGCboBPC3MHQalxINfFmZg=; 5:suMgTTp/66DNSTIbK9ZjhgIgNaiqA0HNPpFT3oYgrjz0AY/g1+sYVySABRotC6S7jKc58sTEguR949k7sp4nie1R0iGor6RpWxcZch9nwrK3npllVWjagCfo2g19qwBk4SW5/s5Yy3jAiFiIkt8fJRSRl7FEwFvZ7Ar8ojFmGro=; 24:eiZ3jdFCt8NOLP1WFT3xxZ8bXfp9GV9jjBrRlLBQOkQmbAItaDJlO1QAYj/T7VwKaIniVQLT1LK8VBSx4ukCKIk5s97BSV6t98IwEdtPE7s=; 7:1975oMuQ1fv/7SIbUlZmtKVPi0N+UnU6q/bqZMWdMIEMQDWfc3GWKsGLZI85SnvncogvbgD0I0jZG35vGz80wiPzYRdOGtfRS+2MGPKk47MhQf52S7BIegwA7HGDXRPndpq/C5qKkptHfKn7sMmIAMAi9NJX7hKjAfvea+0UlNtDAumj42aHjO9UZHpvtUlzCINNpKGsYNmDqAvI41w2P+aswF3axmOYb7cHU6V1ZWkjQy6VzXavWvG+yso95vBt SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Nov 2017 07:27:48.6894 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: e7637f01-680f-42c3-4b23-08d5267a38be X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR07MB2523 Subject: Re: [dpdk-dev] [PATCH 2/3] ring: guarantee load ordering of cons/prod when doing enqueue/dequeue X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Nov 2017 07:27:54 -0000 -----Original Message----- > Date: Wed, 8 Nov 2017 06:17:11 +0000 > From: Jia He > To: jerin.jacob@caviumnetworks.com, dev@dpdk.org, olivier.matz@6wind.com > Cc: konstantin.ananyev@intel.com, bruce.richardson@intel.com, > jianbo.liu@arm.com, hemant.agrawal@nxp.com, Jia He , > jie2.liu@hxt-semitech.com, bing.zhao@hxt-semitech.com, > jia.he@hxt-semitech.com > Subject: [PATCH 2/3] ring: guarantee load ordering of cons/prod when doing > enqueue/dequeue > X-Mailer: git-send-email 2.7.4 > It is the V3 patch and subject line does not reflect that. If you send a new version of a patch or series, you must use --in-reply-to with message id of previous version. > We watched a rte panic of mbuf_autotest in our qualcomm arm64 server. > As for the possible race condition, please refer to [1]. Instead of the indirection of [1], add the technical details here to make git comment useful. > > Furthermore, to fix this race, there are 2 options as suggested by Jerin: > 1. use rte_smp_rmb > 2. use load_acquire/store_release(refer to [2]). > CONFIG_RTE_RING_USE_C11_MEM_MODEL is provided, and by > default it is "y" only on arm64. > > The reason why providing 2 options is due to the performance benchmark > difference in different arm machines, please refer to [3]. > > Already fuctionally tested on the machines as follows: > - on X86 > - on arm64 with CONFIG_RTE_RING_USE_C11_MEM_MODEL=y > - on arm64 with CONFIG_RTE_RING_USE_C11_MEM_MODEL=n > > We haven't tested the ppc64 case. If anyone verifies it, he can add > CONFIG_RTE_RING_USE_C11_MEM_MODEL=y to ppc64 config files. The testing part can be moved under "---". ie in the comment section below. > > [1] http://dpdk.org/ml/archives/dev/2017-October/078366.html > [2] https://github.com/freebsd/freebsd/blob/master/sys/sys/buf_ring.h#L170 > [3] http://dpdk.org/ml/archives/dev/2017-October/080861.html > Please split this patches into three: - Your original fix - Introduce rte_ring_generic.h and move the common function to that file - Introduce CONFIG_RTE_RING_USE_C11_MEM_MODEL and move C11 memory implementation to lib/librte_ring/rte_ring_c11_mem.h > --- > +#ifndef _RTE_RING_C11_MEM_H_ > +#define _RTE_RING_C11_MEM_H_ > + > +#define UNUSED_PARAMETER(x) ((void)(x)) > +static __rte_always_inline void > +update_tail(struct rte_ring_headtail *ht, uint32_t old_val, uint32_t new_val, > + uint32_t single, uint32_t enqueue) > +{ > + UNUSED_PARAMETER(enqueue); s/UNUSED_PARAMETER/RTE_SET_USED