From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C9E98A0093; Wed, 20 Apr 2022 17:40:45 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5645C40C35; Wed, 20 Apr 2022 17:40:45 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by mails.dpdk.org (Postfix) with ESMTP id F06F8406A2; Wed, 20 Apr 2022 17:40:42 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1650469243; x=1682005243; h=message-id:date:subject:to:cc:references:from: in-reply-to:content-transfer-encoding:mime-version; bh=DR2XiOhjOStgFdIhQ56BVZcsge/xfkeE6HZeDqku6sA=; b=bu7AGRNncC50wO0wgVYd+OMJ3yb3jpN6L2Kubfb9H26RuVFzA+TXsDLM 7BnMjUQZEZPvt/70S2aJZXXxE5kHCGW+pbYx9JWRxT7GsvvkcNkPFBbXZ faaVMnWomEK5QoTkdGqJDbegk47GT+gwK8LZqWcZ/M0uWd97dASZwE/uG e4Uh+Gf7rSTLwZiHjvkqRvy7aE7oM4ykiitHL3HYub/YA35sJRyDHB83h Eifm22UTdPgWd2ro2oScqKMMt8O670yYJCkou2gfsUf/5RCPx4YEJOc9b nmu9fqJ/GzQ1AlBskEJLt30/K+YL60SLCPsw40jyMOObQYdjsqXjjMam5 g==; X-IronPort-AV: E=McAfee;i="6400,9594,10323"; a="263521093" X-IronPort-AV: E=Sophos;i="5.90,276,1643702400"; d="scan'208";a="263521093" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Apr 2022 08:13:47 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,276,1643702400"; d="scan'208";a="529787569" Received: from fmsmsx601.amr.corp.intel.com ([10.18.126.81]) by orsmga006.jf.intel.com with ESMTP; 20 Apr 2022 08:13:47 -0700 Received: from fmsmsx607.amr.corp.intel.com (10.18.126.87) by fmsmsx601.amr.corp.intel.com (10.18.126.81) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27; Wed, 20 Apr 2022 08:13:46 -0700 Received: from fmsmsx611.amr.corp.intel.com (10.18.126.91) by fmsmsx607.amr.corp.intel.com (10.18.126.87) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27; Wed, 20 Apr 2022 08:13:46 -0700 Received: from fmsedg602.ED.cps.intel.com (10.1.192.136) by fmsmsx611.amr.corp.intel.com (10.18.126.91) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27 via Frontend Transport; Wed, 20 Apr 2022 08:13:46 -0700 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (104.47.58.177) by edgegateway.intel.com (192.55.55.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2308.27; Wed, 20 Apr 2022 08:13:45 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=dYt6IgVR8iyGDsR988xP91f+Qs9qQQAFlLSJumJFtpFqKCwr3H3ZWVnmOf/6pIE19841qvHBgzMVubtO/iiiyltdTXNiv6RvtZhShCVTlG03cnj94nlvzm48HVZtUherJYUpiTpnlDZCZjN2hdAxlfatj11qVSZA1LK6Evb1lROwxAIVJapxnhZJmMX2K4z3ptj8cRPDBS+i7yylqlsQm/cJ7Z2sr9ugzLSc0v7vljTNZcMTfFcHhGV+++QxJVT2HHWY9QP2Msqz/i9hH5fK51xHJSXenk7PsJ5V3rmD3H2V+msjVIs89xv1M8hGZ/K2aiGLzmE3CFl3U8xbXHTugg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=LiHz9joarEIm/Tdj9+bXY71kSUQA6h4wzRtpMIafFAg=; b=NZNjm1ukpwI+TXpvo+t5Hy63C0xQIXDvl0PGvZWdyKXMFXlBjDeUHqhEK8PDp8pjrkC9nqy6vtAEgEVdPNIls3LV//xtB4D6ZWqn5l4R+bBR1/4cHEy/s2WvdU+1Bv2VD8AzOlJFoFdoBUZCXbeaXX3VO2XKLx49BFHQpmaFSIVMQOOYbzh7TOPqOOchwDxHrRnmbEVYOrgWDrn57Ztj98RggLEIz4BtMdAvUOuUnpWWf78Ty7OGMpyG51YmTA7PJZeM/tBFHxgLRathZ25lqcIP6sbfO0C58vfV4AdjsRbadwSDbsPfyjSC+ZPv5WLQcDAFL3+oryyeVh1dSPpdcQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from PH0PR11MB5093.namprd11.prod.outlook.com (2603:10b6:510:3e::23) by MN2PR11MB4094.namprd11.prod.outlook.com (2603:10b6:208:152::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5164.20; Wed, 20 Apr 2022 15:13:43 +0000 Received: from PH0PR11MB5093.namprd11.prod.outlook.com ([fe80::28af:8ec5:1817:1af1]) by PH0PR11MB5093.namprd11.prod.outlook.com ([fe80::28af:8ec5:1817:1af1%3]) with mapi id 15.20.5164.025; Wed, 20 Apr 2022 15:13:43 +0000 Message-ID: <2f2a915c-0059-0ad9-8212-b02a8384915e@intel.com> Date: Wed, 20 Apr 2022 16:13:38 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Firefox/91.0 Thunderbird/91.7.0 Subject: Re: [PATCH] eal: fix data race in multi-process support Content-Language: en-US To: Stephen Hemminger CC: , References: <20211217181649.154972-1-stephen@networkplumber.org> <20211217182922.159503-1-stephen@networkplumber.org> From: "Burakov, Anatoly" In-Reply-To: <20211217182922.159503-1-stephen@networkplumber.org> Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 7bit X-ClientProxiedBy: LO4P123CA0179.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:18a::22) To PH0PR11MB5093.namprd11.prod.outlook.com (2603:10b6:510:3e::23) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: be31f44e-2738-46fb-b0b4-08da22e05b93 X-MS-TrafficTypeDiagnostic: MN2PR11MB4094:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: kQsBr0Coxbzg344YwcuHZIVoje3+MXlZxM6CiCUNMzjr3f01tlazEX7s/qiBiKv6e3uxh4PjlXKjydLpKlIO3EJXxblxKx/uiNhUv3dEOcbb6tabhnR1ILtrBLMZHvT6Ryspy3cht2o/xqaWbjF702NYgW9tSK2zjXlPipJsmDVRVhLa2syWhenP2ftTElgK6H8UdgpRrPCjgmxsKrJ1i3yT6uM4xbFYAqiebr1eiDR0T+h+kk964UQHjuyqDpw7Ghb29HhVlQIqvZZV7/pw7Kluij88fg5wxDs2fDLQnmVGAme1YZmZkD3ljq8p/NVAjE+/SNm9GTyXCFPx2Y6KleMpmHVp93sCzizqyRiNpv/bFWthJP8xegYqXKCbTMWIe5YEAjHotCgyMlhXu02+J6Kvc3B7HQuN7vhh2mj6DzRRsbprKP27a+pVNHqE8AvKsZBtwLDRiFkVL81GsJs6BPuK93GzK5LocGBh/yH1ssp48g0FdHisz6sp0DsFCOayIqvGfNzhQDAZ5ETZSnwTkaLUrEAKDXTQsyo24xOPorIb4BRaUUGXqwEyDvYYHjzeqxapmo5jtq4bVquqiwJ5rWJpOBAyHqQEdMceXn97FV6NZQt5+sJVyV/CQCvFzGPXu4T5zHcftbwaCeLSOsCPZamupLxti+oXF/0HFSPkUSEJEGMINoYjX8+OKQJ2Nox+NTWhKBSvAPio1wb/mk6BsD/QIYZw64/Wppkmcdm65ss= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PH0PR11MB5093.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(366004)(26005)(6506007)(82960400001)(6666004)(31686004)(6512007)(4326008)(5660300002)(2906002)(8676002)(38100700002)(86362001)(31696002)(66946007)(66556008)(66476007)(186003)(508600001)(6916009)(2616005)(316002)(83380400001)(8936002)(53546011)(6486002)(36756003)(45980500001)(43740500002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?dm40OWFpMHJMQU56eXBoWjgydXJLbWxEV3hVWUYvR3lnZkRMbkE4ZnJnTGF1?= =?utf-8?B?MU41cU5CelFzTzBCWkQ5d2IzVnlLbU9nVUFnWFNwSlBPeXJXdUQ4M0gvMFB5?= =?utf-8?B?Y2szY2tKbkdRVHF3UDhTcVk2RXZtRHkyL0dWQTd4YzJDRmUzMXpBTE84R3E4?= =?utf-8?B?VWliVDBSRU9xNXZuemJiNHRDcGI1TnZDQzhRRnF4dWwrZU9qaW96MkVuakVn?= =?utf-8?B?cXdUOENNRFZLbi9JcTZPWWZ5VEVZZzRaVTRlWjcxZ242Z3o2ZEZHL0s0RGUv?= =?utf-8?B?T2dMZTVMR1R1SDFuQjE5bk8vdXB0bmhrWWRidXpJNFY4WjRINEM2V3hxeThR?= =?utf-8?B?T3c2NU9DeG9FSHF4OVpHMDJ1aVROa2JBcy83ZmZVSW9SRmhXNy8rbWJ0U1cv?= =?utf-8?B?WUxkUzZHbVV0QlgrVmozWS9NR1hUUUhGdDRXWENvT25LWnVhajdTOHhvMHE5?= =?utf-8?B?eXp4TjlHS28ycktnV213TU0vTjFHelYvdGQvUWUxWTdmdWVxUVJTaTM1UHBH?= =?utf-8?B?THJNWkRRS2lKR2kyVGIvcFdBaHpBRmxGaXVoWnFkZ25oUUtDeUYzaUxxc0h5?= =?utf-8?B?M0lZMnVCeEl5emJOSDQ3RlRudVd4a0hGdmxVOURuS25KTmpzNWp3QTcvNk93?= =?utf-8?B?eEo2MXFFeGpleVRhczdqZTF4bnhLTEtjZDRjdnhtTnVvSVlZT3VPdGorQ3RO?= =?utf-8?B?T3N6dzBWRlpybmowek5TdTYzekEwaWEzcnpNSHdZdEs0TDJzN2ZkcGFXWGNT?= =?utf-8?B?WlV4bHJuWlgxdm5oaUozVC9UbEEvOXV1TndtTm0yZ01lTEwzeDYvdmZuU3Rr?= =?utf-8?B?WHdlRW9SaXhPc2tzVXNSUFFTb2l0b3p0azBYNTN1Tko5ODZhaHM4QkE1RHNz?= =?utf-8?B?Y0tubTdTb2RnYUVXOXVhUWdlcEZObXFMd1ZKV1JyS3poUC9NcmFFSnhGeWx0?= =?utf-8?B?QkRpc1l0VTdDYmp6dW1nOUJBWHlaNWJaT3J6Q2RxVm1sQVBjS1JwQUsrQW5o?= =?utf-8?B?QlhMR0xLdFVzREV0NlptNUUzWVRoazhsTjByWFhPRzFZMkJmN1NIM1hCUlNx?= =?utf-8?B?U1FyQzJ3aWVNcjFUUS9LMFVMc1pzWnR6eWNIdXZ0VHFyT2dJNEhPYlg2S3dp?= =?utf-8?B?MjdocHdYTEJxbzBGRm1iY2xON3dDaC9CRzVDeWQ3SDEvNWpLS3FuWm01MHN3?= =?utf-8?B?WU1ITGQ2Lyt4UWFEV1o5ZTd4dEJxRnpHMGxMak9JQVRhSUdlYnJkb296RHhi?= =?utf-8?B?Q1JWbVpRczV6bUNXY25ZMUFIYzRoNkxaOHU4R0V5QmVXSndIa3hRNjMyS1Zy?= =?utf-8?B?bmJkWkdVQ2tzT1pTMC9mcHQ1RmJuVzFPSGhiSE5Yb0p5WVplL0hCYjB6NUJ6?= =?utf-8?B?ZEJDRlNDVzFkZVhnUDZuQXpVeEpycjFEcG9XOC9BSzlORTM4d1RtYUwzN0hk?= =?utf-8?B?RFhuY25iZEtGYzloeDNnOGRWMmxrY253RDBqcHphcTBjclhCQ0wxMDB5bU9V?= =?utf-8?B?NW51UGcrOUEzV2Y3ZExKMHQ1MTVPQU05Y2FIRzM3d01iWjlOQ1VVZkUrYmw1?= =?utf-8?B?WmY0Y2hsN21GTTg3NHVIYytrcVQwdWhyeWIzR1FmVkJIQmlxLzV3U2JYV0xa?= =?utf-8?B?V3g0a2FYbFg3Sk52WXNNaE9FZWZGRDFvZEl6cEhmV2RQWEZLM0JPdUlDNXNX?= =?utf-8?B?YWQxSDFiK1c0QUM3SXN6UzRRcEhnYzVVRnMzMXhMRUlhZit5ZVZtVGpScFBB?= =?utf-8?B?ajVhRXBhdnlYNFNxMFMzZCs4OVc0MzJXTkx3NG1HbnhIRFVIdWRpVHVYQnYr?= =?utf-8?B?bVM0b0V3bHpjN1AwK1dYdkUwTGNQVUFDVW1ENTVhVXNSbzhJU3h2NDdTbTlY?= =?utf-8?B?QWJOR3J1UkpBMGlnU1VLVDh0VnVuUVB0Um9pWDY1WVpLQ2FJV2xZSjliSUox?= =?utf-8?B?OXV6NEU4V29xdU11TTBic09YblBOTjRYYytPZjg5bTZYclpXYXBXd08rdkE2?= =?utf-8?B?WDk4YVJBcXFVMWpqL0dwSmFMczRJKzVpQXFmMDRqRkg5eTd6aFdNdWhtTE9w?= =?utf-8?B?ZzJZU1VCeGg0aUNveWNRMFdoYnNveDArSTFhNy9sMTcwRjE5cFBIQlljTkZE?= =?utf-8?B?RWtwNjdTSm80dStlYWhtTE84RmkveDM3U2NqRm9wdG9YKzhuVHVnYU9lUTF1?= =?utf-8?B?L09pbGprMno1aVhGNDIxWHo0ODZoWWh2RWMyM1Y0WHdzTXRxa3dUNk9lTm43?= =?utf-8?B?NEttNVh2LytNbnJMS0FDNVRwSkk1NEM1TENoOExiaFlNaWxhcDAvTWtiYmNZ?= =?utf-8?B?YXFCK2tKdDN5cTZmenIreXdDWWdUYlVZYmd5bUJyS0Mwb0VSL3kwTHFmeWJ2?= =?utf-8?Q?m8QU78I4VB9MlfXU=3D?= X-MS-Exchange-CrossTenant-Network-Message-Id: be31f44e-2738-46fb-b0b4-08da22e05b93 X-MS-Exchange-CrossTenant-AuthSource: PH0PR11MB5093.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Apr 2022 15:13:43.3114 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: QM9pgNH0qmjCUO1ydw8Qv79IYSMsJHosO3SZ5RaBFwfm4nSRM8UVVOGCuqQJO79vUL1JTGKA815nstNPlxTwZj0zzZciG2fHMlGFP9/5Uis= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR11MB4094 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 17-Dec-21 6:29 PM, Stephen Hemminger wrote: > If DPDK is built with thread sanitizer it reports a race > in setting of multiprocess file descriptor. The fix is to > use atomic operations when updating mp_fd. > > Simple example: > $ dpdk-testpmd -l 1-3 --no-huge > ... > EAL: Error - exiting with code: 1 > Cause: Creation of mbuf pool for socket 0 failed: Cannot allocate memory > ================== > WARNING: ThreadSanitizer: data race (pid=83054) > Write of size 4 at 0x55e3b7fce450 by main thread: > #0 rte_mp_channel_cleanup (dpdk-testpmd+0x160d79c) > #1 rte_eal_cleanup (dpdk-testpmd+0x1614fb5) > #2 rte_exit (dpdk-testpmd+0x15ec97a) > #3 mbuf_pool_create.cold (dpdk-testpmd+0x242e1a) > #4 main (dpdk-testpmd+0x5ab05d) > > Previous read of size 4 at 0x55e3b7fce450 by thread T2: > #0 mp_handle (dpdk-testpmd+0x160c979) > #1 ctrl_thread_init (dpdk-testpmd+0x15ff76e) > > As if synchronized via sleep: > #0 nanosleep ../../../../src/libsanitizer/tsan/tsan_interceptors_posix.cpp:362 (libtsan.so.0+0x5cd8e) > #1 get_tsc_freq (dpdk-testpmd+0x1622889) > #2 set_tsc_freq (dpdk-testpmd+0x15ffb9c) > #3 rte_eal_timer_init (dpdk-testpmd+0x1622a34) > #4 rte_eal_init.cold (dpdk-testpmd+0x26b314) > #5 main (dpdk-testpmd+0x5aab45) > > Location is global 'mp_fd' of size 4 at 0x55e3b7fce450 (dpdk-testpmd+0x0000027c7450) > > Thread T2 'rte_mp_handle' (tid=83057, running) created by main thread at: > #0 pthread_create ../../../../src/libsanitizer/tsan/tsan_interceptors_posix.cpp:962 (libtsan.so.0+0x58ba2) > #1 rte_ctrl_thread_create (dpdk-testpmd+0x15ff870) > #2 rte_mp_channel_init.cold (dpdk-testpmd+0x269986) > #3 rte_eal_init (dpdk-testpmd+0x1615b28) > #4 main (dpdk-testpmd+0x5aab45) > > SUMMARY: ThreadSanitizer: data race (/home/shemminger/DPDK/main/build/app/dpdk-testpmd+0x160d79c) in rte_mp_channel_cleanup > ================== > ThreadSanitizer: reported 1 warnings > > Fixes: bacaa2754017 ("eal: add channel for multi-process communication") > Signed-off-by: Stephen Hemminger > > --- > v2 - fix the mp socket bind > Acked-by: Anatoly Burakov -- Thanks, Anatoly