From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM03-DM3-obe.outbound.protection.outlook.com (mail-dm3nam03on0089.outbound.protection.outlook.com [104.47.41.89]) by dpdk.org (Postfix) with ESMTP id D5FAE235 for ; Thu, 16 Aug 2018 09:09:06 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=wq4iHHNdwrNAW4ss/me3W8zF0w3/K8zqkSzG08fc9tw=; b=cMX8ZbrXpWFwZNR84qb1LBWWf6cbwjs0jznvh+BqfGAqo78AOWg+UZQhjJZG75UFbta7/UkQJ6N3ZPUiro43nsPa/tPxSWzo8JhIBS6y2IN8M+r0HcdC8RUr9uLAdfwcL97fws0X/IeVZy+pbXlw4IPG1EQwI64FAHTnETmRPF8= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Jerin.JacobKollanukkaran@cavium.com; Received: from jerin (111.93.218.67) by BYAPR07MB4997.namprd07.prod.outlook.com (2603:10b6:a03:5b::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1059.19; Thu, 16 Aug 2018 06:53:12 +0000 Date: Thu, 16 Aug 2018 12:22:57 +0530 From: Jerin Jacob To: Ferruh Yigit Cc: Kiran Kumar , dev@dpdk.org, Konstantin Ananyev , Bruce Richardson , Santosh Shukla Message-ID: <20180816065256.GA11266@jerin> References: <1533810233-7706-1-git-send-email-kkokkilagadda@caviumnetworks.com> <5debc6df-9bed-597a-03e2-510372b849e0@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5debc6df-9bed-597a-03e2-510372b849e0@intel.com> User-Agent: Mutt/1.10.1 (2018-07-13) X-Originating-IP: [111.93.218.67] X-ClientProxiedBy: PN1PR0101CA0033.INDPRD01.PROD.OUTLOOK.COM (2603:1096:c00:c::19) To BYAPR07MB4997.namprd07.prod.outlook.com (2603:10b6:a03:5b::22) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 202e7357-f3a9-4399-c57a-08d60344f077 X-Microsoft-Antispam: BCL:0; PCL:0; RULEID:(7020095)(4652040)(8989137)(4534165)(4627221)(201703031133081)(201702281549075)(8990107)(5600074)(711020)(2017052603328)(7153060)(7193020); SRVR:BYAPR07MB4997; X-Microsoft-Exchange-Diagnostics: 1; BYAPR07MB4997; 3:17tK6T3tLQTCX3otDY4HUVeHFMe1kNL0+OLRba2/y3xZT6d+80R96PCw8wMRfFmfggl1Q+v6yt64aiolX4fUq2KwFdi80PF+jwXEmlpycFYvoDITdmtaO00Rvh4J6PfoonA1iuCl8R8fazVKeu6IAmxORtzPig2yaUmQp2MSwfWEDT0DVhiO3Jpx4eJVpXOWKYMPd/n1Pw6vK6nTrhlt6eDN0Bd/4PnF8e9P5f8DVYEdN6dpdr2IwLR0liCODEoM; 25:gYYau5Tagcd5R4CkXkOTFzF6LjJ8Yz2FrJ7Zl0nLIuEg6E0queX4dthwUPoEToAXLZbBvtMQBM7YwUe3OD5WczvTiSCbl567vLjESpYMSg6kbQlglILsEUKUvWba+oZVWPOANjF4JXxG+orVzqTXSQoKsdp46zB4sxr6O3Cmqlh8HTczbVtDiKoIvPKVFrJ2H0VgEhxSm4Xk09LPIuibYz/iNKth7f+Cll5QCSCJp7nfU2702DsPH3+U8o4ru8VxkMFr81PJnX3sV2HfSL0E42cdFql8Awl3EkCFabqzosZhaJ0ut5jTgZyge+5C0VULkSrzyPhS9IwwWqT0XgKG6Q==; 31:680nHcYBjRwKipRZLhGd5FiiKm9OzEBgkTpPT3RTsd1Px1QvBJLj+QUfYWcQLS5BFE7tqbD9h49IhmV9ro/FaQinTVVNe6UmB4yWOQCMv3iz0LusVWI2BLdUBP6Lv/K/SPaLIFe/nCqPR/iAu8fiqPOor/vuLuM0o//z+Ns0tMqJiSozhG+u0Ek7Pl97sgo3crhkqQmYsmsdfSc1YW7p1MF+ZVdGSU++K1JXPMhebQs= X-MS-TrafficTypeDiagnostic: BYAPR07MB4997: X-Microsoft-Exchange-Diagnostics: 1; BYAPR07MB4997; 20:NCM3HBkhzJbBpCSAOZR9NcT11J+dhQ9HpEqlQpgNTs51U5m/v1ruZcsD+H20Dp7O/nCQgGtkDn1Npv4PfVqNoD6yC13jjNQG+1s92l7kTAdOmTO9e4WJaIt2cDYlTdwPLLkrqDwZhm8NcVB6C89ZiyUB8x9l5w6QODGX9zQQJz2Ud5VDsyaZWJ3xW695oBQGjuknF7ziMJgK1A7k3no36vg65AjLLpciKnS4G2eqNSYGS+J2h1lMWVRV8L9XKTMX80t33+gp1TgXX+3wVrFVqAeB/VWrVSM63K+ST+5/s/TULSSfYywmVnpqTYZG2Hao7BLxX4S4hQlqKU8OQtm00XfFeq8VA6ARJET/krBjZ5w9pU889Jiq6f05hhdIuRz2qXMx+W7bHxUWqgUhUiUJlx3JiK2rnsv9j+xfpxKwrxTiQK5c+dWKjq0sDH5c1a/x/1ksYZ7cXVvza8RqyJYd2tIpdbnCdE3cpaunEjB928ow+6b+ImDWHDTjHIioKmR1s8O1y80lcoyTSqcJ537I4sP8X3t/Gr8+fn/AFYyukyJ6O9Cx0UVq9/QT32V/P/Nm3UCeeQB8KqyFYK5lKSPIW9BlaDr1Oqmh0CXvaEOl+3o=; 4:c7cdlVG35WXvQ2sFFI4ssbwOL0CWZ6BX9SU4VLF1VkzDzy0eIQ4GgfbFZ6CT93g5xnwyV2YE9HWVQIiq0e31uu1uRmriwQlPGjgKMeFSCw4KSvwV2FubinN1EV6DdLEGzTLbKN0yx2ZAvmC0Ajd0LIoQV+Krdf5jN4js4yCgQYkWBtV0ZPuOHAOvmCnSjK+lGuLAYAy5HGrEW/kK+k1uSog3SruzOn4+8vhKr+tzPFrd+VzhGZPBKfmYnt+63quf95fD4/nKjt9D47x2fFhknvDpMv1dyw6r3VTW8MXH6527cS1G4zJSsOvO4va5tEEd X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(228905959029699); X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(93006095)(3231311)(944501410)(52105095)(10201501046)(3002001)(149027)(150027)(6041310)(20161123560045)(20161123564045)(20161123562045)(20161123558120)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(201708071742011)(7699016); SRVR:BYAPR07MB4997; BCL:0; PCL:0; RULEID:; SRVR:BYAPR07MB4997; X-Forefront-PRVS: 07665BE9D1 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(376002)(39860400002)(136003)(346002)(366004)(396003)(13464003)(43544003)(189003)(199004)(5660300001)(3846002)(476003)(107886003)(6246003)(42882007)(16526019)(186003)(23726003)(956004)(50466002)(53936002)(33896004)(229853002)(486006)(53546011)(55016002)(2906002)(446003)(25786009)(6116002)(1076002)(26005)(4326008)(386003)(11346002)(47776003)(316002)(54906003)(58126008)(44832011)(72206003)(52116002)(66066001)(97736004)(33656002)(105586002)(5009440100003)(106356001)(16586007)(478600001)(6496006)(6916009)(6666003)(9686003)(81166006)(33716001)(7736002)(68736007)(81156014)(8676002)(305945005)(76176011)(8936002)(18370500001); DIR:OUT; SFP:1101; SCL:1; SRVR:BYAPR07MB4997; H:jerin; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; Received-SPF: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BYAPR07MB4997; 23:YYBIv+b6vv3mXU+bFO0G2TTGvVArMsHanIPZBIkK0?= =?us-ascii?Q?W93vND8lPFwSog7kE2TZpnXrvloB2mZvuQfUmlaNyzUjuJaYZiP2fH+30HRQ?= =?us-ascii?Q?4hFWukkRHUXJV6XtfUePzk5kfxhFvNoL4LzpILs3KgREtrMnrQBRYD29ZoR2?= =?us-ascii?Q?B/5c/ZOMpPpYFj/GDiQpL4Eo7zcTRbBmxNb1qMh2je6pJ9+e7QT2vt4W2VC0?= =?us-ascii?Q?JOB08TMBZYFC6gB7x+BkwUUtJc6211XFoF0wI4wpHwKEHtZ0iIJMZW6opthK?= =?us-ascii?Q?VOKMJyRk5GX6zW4dG3ka9MJ48cfmv6JWvTStsAEFe1+X8r6aqX7G2Ox4kyC2?= =?us-ascii?Q?8Tfy2uklXRcAReM+mxj8Uz4eI77YsIrEnf8tmWnv5vt7xwsP5dge9pMDdfgS?= =?us-ascii?Q?ayXSF5+qn59/46OcGWF7ucw21EsPkV07BXLxbH37h0naTaJGRLil0ieGGYeZ?= =?us-ascii?Q?reUE+Pz7RhnaXh7k4nC2veQtqnLxfGV8/Ob3jSpp8+Lxr55Pja1kcl+AfHDG?= =?us-ascii?Q?zNWsbFrlIzzAocXHSgpAhP5IuNmIfOu7u2aQ9WOu1W/ceGdSa/Psv6UvzKhf?= =?us-ascii?Q?a07W+D9SvZoL511AXmjrF3H9vVTciUtcFpAD1WGv3rjWevu9hVALKJn0P6Ft?= =?us-ascii?Q?NOjJjAeDeCXrAofcOwtolWo++JoH73CWcK7oVw46fFfVATrmEMZhgTDP91T+?= =?us-ascii?Q?c1LFLrDhbLXAP1u08kG9hY4D5tDXLwBsLpKkEiEbBsFDzVcgDiw6NCjGo7XG?= =?us-ascii?Q?Wo0msqg8XQuAe+kPcca6tzedJEhtc/fS/t1WmsSkzgfWEcwcVvKqvJTytnzV?= =?us-ascii?Q?7EKKGsuT/oFC+PtI9DQwTOS6FYoKzZW01gnhm7Rz/gXo0qv2n1O7+fl5dYO+?= =?us-ascii?Q?8Db7K//HI9w9T6cj6dIOjg23CRK177JsgZ6YLfsf4ZXQ9+JgE7Qz8Ndd2q4h?= =?us-ascii?Q?RHGxZtMarL/8NGTsW4d6w+0Klntq3eNroNtmfLBox5oavywuiqIie3A5SXan?= =?us-ascii?Q?2MK3dbKyPkzizHBalApxApab3sYMLAFKWk6XBveC2VO4AZt58VjEa9oXHCU6?= =?us-ascii?Q?d1jiBDyM6g+ZVZSVVvEY6S+4LLJbvlHyHN9G6HTZQ3OaMWM/ptLNe4WOwOoO?= =?us-ascii?Q?tU1+VeLd2y5ACDq63oqZPIjjB6Jb9KMCtRODEKNALf7Yq1WDxBaNkZOj9DQ2?= =?us-ascii?Q?z5EC4mZZmy/77rGljKIakzcWMvpM4rXgBPEKneU+LxuLk862pG3L879BAise?= =?us-ascii?Q?J/hK7ox8QxImJGEjBEkFwS6WMVJxW9tA9K1cTlEKdXv862eOV3EoXq5yplK5?= =?us-ascii?Q?yFk4pUOFqtnKdDkLG02oYk6WZMK+0qA28wx3igxFuoaLAnBG2+xgofUeZOys?= =?us-ascii?Q?/NSB3juBWTXC4sWe79ZTP+bWbp8PZ/6oi38ZwiMTtKcefmigO/5OFdFu6Ysd?= =?us-ascii?Q?BUXIrjKZW5/Rh5lC3YId37aOY1eeDU=3D?= X-Microsoft-Antispam-Message-Info: G9MM0hgO6mdJjXh45fxNateNU6MAtb0n1SUYqrZvot1kdf4vjq08azr3D8gZZzyftH1FSZNrNFQrXxBVNmsAz3tKD/vG1LsDWPeBKuJHzLYz0oBSsdMwTTpmXX4kSFe5nm7qqhIKoJ2SROZCNLHKWY+vvGgK2toq5nW355mPxb3UXfV44TMfclxgxxR8WN6WdeygdAqOdM1PG9WJdQ8DnKIM1mwIQvhUFZTm68yrSrd+O7TLGrtNgkgx/lAqFsw1+5fORlcE8rN6Vr0WWbIGzUbm5DhExBdaAlwzQqd4k0sWD+B9PPHtFt71BipgCMfn/XNKEtWVRlISroEoYYw+RucWro4D7fNgN6keEytAans= X-Microsoft-Exchange-Diagnostics: 1; BYAPR07MB4997; 6:yJYBpvE+I/0hHvQYscWCIEH6UYVUCjozZSBpm7UyZDClnsy6qS0v84nERRQEzA3sAfii4C6sM9BmGM5EZRCr+/rD9ikJhzR+qbbeAaGSdWSlJqyvcfT6tU3gN6K0IQiET5ryy/LeZX4vs2VeBX3ks7ogNEUFqkN0YhSUyIueAK5tj48H8S37XVBoY2gF9sEo2viLe80ZjQLzT1W6E2qUUky1e41ozarOU4DRtLn/TIdTH4CICuqSWrD8mU+IpmqKZpQvxZv2D7oR29v00O+qpbCkBA4XTtcvAIRtXqnnrPHOxhUrgUFyvdfv2Sl/CT3za3SJUzgSn2Wme91cRzdt7OUiSYoxgOGvW6w3KIqC5jMSBhY9f0a+2DQjxW8PSD/vCUsopewIM8AWrVfaiqrV5w36QNC0LNsgsbq+EBKi1qJZp6Lb/wn8wR+rhqgM/QK7Nh4wX8195+E3SJ1SZrGG5A==; 5:AGFoJ32spRjXLrsu+5ZLm15uvxQF7WRv7b9IZqvmK5lVZ9k2S4+Npt6DetOU0kC5z17RSepbkGVBCFiHVPjynMY8CqmYhrIZ1f4pK8trFlNkdG5ZChjIYE4Uv6sMFnBpnmoCzIWr2WHquAfRLPtRm1cumdHxKmOjdCpXE8mFi+s=; 7:Vb0NecntVRq344lYU9b3Nj9hgi+AkPFf36dTCj0dooEiUorV0rsntswIPJIShffVOyzRTjMBCuOrzTFcs9pMvYk9sbKmM4+srB4meXUTZQ/Nx5gPhQYUuBW5BI5CDW8mnmtxEkMhf4YoZA1eZrNjyd/iwgPS1yQ92zHhSLD1Is7bpbrXOIdHcFYbaTKV9kiK8FQxfH5J7Go0lgnZWzK6nK+2yJixa1kqzKdebDjgQ4lgLfpCZY8khuuxspgVS1zH SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Aug 2018 06:53:12.3731 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 202e7357-f3a9-4399-c57a-08d60344f077 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR07MB4997 Subject: Re: [dpdk-dev] [PATCH] kni: fix kni rx fifo producer synchronization X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Aug 2018 07:09:07 -0000 -----Original Message----- > Date: Thu, 9 Aug 2018 12:30:57 +0100 > From: Ferruh Yigit > To: Kiran Kumar > CC: dev@dpdk.org, Jerin Jacob , Konstantin > Ananyev , Bruce Richardson > , Santosh Shukla > > Subject: Re: [dpdk-dev] [PATCH] kni: fix kni rx fifo producer > synchronization > User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 > Thunderbird/52.9.1 > > > On 8/9/2018 11:23 AM, Kiran Kumar wrote: > > With existing code in kni_fifo_put, rx_q values are not being updated > > before updating fifo_write. While reading rx_q in kni_net_rx_normal, > > This is causing the sync issue on other core. So adding a write > > barrier to make sure the values being synced before updating fifo_write. > > > > Fixes: 3fc5ca2f6352 ("kni: initial import") > > > > Signed-off-by: Kiran Kumar > > --- > > lib/librte_kni/rte_kni_fifo.h | 1 + > > 1 file changed, 1 insertion(+) > > > > diff --git a/lib/librte_kni/rte_kni_fifo.h b/lib/librte_kni/rte_kni_fifo.h > > index ac26a8c..4d6b33e 100644 > > --- a/lib/librte_kni/rte_kni_fifo.h > > +++ b/lib/librte_kni/rte_kni_fifo.h > > @@ -39,6 +39,7 @@ kni_fifo_put(struct rte_kni_fifo *fifo, void **data, unsigned num) > > fifo->buffer[fifo_write] = data[i]; > > fifo_write = new_write; > > } > > + rte_smp_wmb(); > > fifo->write = fifo_write; > > return i; > > For Intel this is just a compiler barrier so no issue but not sure if a memory > barrier is required here, > > Related code block is: > > |- for (i = 0; i < num; i++) { > || new_write = (new_write + 1) & (fifo->len - 1); > || > || if (new_write == fifo_read) > || break; > || fifo->buffer[fifo_write] = data[i]; > || fifo_write = new_write; > || } > | fifo->write = fifo_write; > > "fifo_write" is updated in the loop, so there is a dependency to it for > "fifo->write". Can memory writes be reordered when there is a dependency? >>From CPU0 compiler instruction generation perspective, It will take care of the dependency. In weakly ordered machine, it does NOT grandees CPU1 sees fifo->write as final write. So, IMO, This patch make sense. We are able to reproduce the problem on arm64 machine with certain traffic. > > Cc'ed a few more people for comment.