From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id CE03343095; Fri, 18 Aug 2023 04:32:22 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5AC7240395; Fri, 18 Aug 2023 04:32:22 +0200 (CEST) Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2052.outbound.protection.outlook.com [40.107.237.52]) by mails.dpdk.org (Postfix) with ESMTP id 3F22940156 for ; Fri, 18 Aug 2023 04:32:21 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Cm6KWfUDzybHfswcASlIbQx3rc91BMq5WtvXqzZNlTKIqfOyvfUg5+N4fp6LPbmyTMrN6NvKNEh8PyMZpeM6d2uu6Q/5VVbQ6MYKWrBWCiDOgRwhHCwWzQ8393rFp4tVDybaczX0BljSxHsh7Gi5ZGjDnCxjsiOVUy7mzq3ujkzY80JkcXsY+dG3z1uJTxJo2uysL4lnZPCCujkAq3kmG6s/rXvkdAHvWu3OoksbVJvi5OjwqBdWZk0bbZmeu34V3JWlV6ag2fxaer6e+7lmGTyxRgZjFnpaAMuMHtNzrIK88iV2cS/GhknbWzlhTscUu/u2WBIjdzyVDfW4mbqtWg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=a1nhp10DSry8wBt5Irp6rL8HqhMWdsuF0TRKaO12f8s=; b=iciG1VlrL82dK7OSvBSpMm8Nb0Y7fUs87QMlcg+kJKVtoqE33slaU4Dcv7LvqQOYTRMDD0G3I/jidBYp22o3vlTgNqclMLVual6mStu7vaIrygnXIyQhtOUredSMjjNkgpcVW8oPmTBnwWeL9OT4FrHbWEF2wqlFcvsPcsXn3UlziEhD1yF92TwdI5bKHJxuAb0cKmmm2HrT5NSRYdUz0JyjMBtunyQaTwds9f1IKnihlNmTFeK0FUq8xTGzA5bpxbOlyX1CNSxpUVQGgMVxnqD+otpY8Ft58gT7+gwtn8GZw9+eYI9pLVZXGhFuxvYr9Nsxh8MP+7lMH7ujGneNcQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=a1nhp10DSry8wBt5Irp6rL8HqhMWdsuF0TRKaO12f8s=; b=fI7aQRwVEU1Gg/VavCkHT3aO3wC46cVP8inqc0Y1l17yI0Ss+QsP9XSuisYpcwvzMBKp7WD9xQCQNid53Jy2HEFgQVCgO83m3MySEBkgKh/PmUt99trkYDrEQyT8dbF+OBmoCdH1LFeVK5DcgLJUKZ1xQqHQSRFYWApgBnV8Z3WizuEWo/+twEDyXM3vryYuOUJ3aqZf3w/PysxcSZcGzyicG6nNp6QUhs2YE+r3aztbLTt8dQWOQjI9XyKhHyAXnw7u2agDitP8twusfJEQk2W5fec8xGsV1o2RKU+CUu4rjR20bIEKA6ryjTYtNq6jRRm91IJlCWUJTDFCGtwG7g== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from CO6PR12MB5459.namprd12.prod.outlook.com (2603:10b6:303:13b::16) by CY5PR12MB6598.namprd12.prod.outlook.com (2603:10b6:930:42::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6699.20; Fri, 18 Aug 2023 02:32:18 +0000 Received: from CO6PR12MB5459.namprd12.prod.outlook.com ([fe80::46b7:f479:cba8:9adf]) by CO6PR12MB5459.namprd12.prod.outlook.com ([fe80::46b7:f479:cba8:9adf%7]) with mapi id 15.20.6699.020; Fri, 18 Aug 2023 02:32:18 +0000 Message-ID: Date: Fri, 18 Aug 2023 10:32:10 +0800 User-Agent: Mozilla Thunderbird Subject: Re: MLX5 PMD access ring library private data Content-Language: en-US To: Stephen Hemminger , Honnappa Nagarahalli Cc: "dev@dpdk.org" , Matan Azrad , "viacheslavo@nvidia.com" , Tyler Retzlaff , Wathsala Wathawana Vithanage , nd References: <20230817070658.45576e6d@hermes.local> From: Jack Min In-Reply-To: <20230817070658.45576e6d@hermes.local> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-ClientProxiedBy: TYCP286CA0101.JPNP286.PROD.OUTLOOK.COM (2603:1096:400:2b4::14) To CO6PR12MB5459.namprd12.prod.outlook.com (2603:10b6:303:13b::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO6PR12MB5459:EE_|CY5PR12MB6598:EE_ X-MS-Office365-Filtering-Correlation-Id: 32aea4ee-4e4b-4724-0d05-08db9f93577e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: W8zX9rMe4H4/TC8zYyE8kfjQKbWUjC991lVf6ExYbSmJK40/8gH9CJ3hFYZT+ZlDamclWOt+HXGHm41tGJOr/tEEF7wp7OzpN/FpnYRr2xHzHpbNs1ZrlNnKh+QyHKpsFzE9RHbb/V2mABvSgJTgfBQ2wriSzbMUxyIAhkZ6rxHubFRUqLZzpnJb3uM0rHEoDZSYJd03bjWKjMBovcH9RcQUwCHlf2i9ts4ob8j27mZhyTr4d8XwoFActkdKPo9vkdfIvrDjKZkA5BCcCLHZVxcfAXuU7MV748+wP10GnFoPjF+jQ/ooYoTGfqvVyitjW5H2X4vUEnW2K35u2SL6IBjWnb6znZ9xZLoPnNv4gBGzObB7HbfctCi4KAHm3w78EVh2YC8mmfF80NYfoApUXqOp9cDVATKeRtq3VRII8d0kCKK1Y3swBZItKjj86sLRJuc0TGwesYGxLg45crQcjYyNgMIsN1/JBg5Mf1C6ke2YWvyFuEtIOjSZj1SVJSfTN3Fw49/EFNSOEeJ/gew4mHz7cKp0r5PHAjQ18wML5L+roLhdhu6LvmNrso30LLg1r4G9/mq5gLNnoYwqubJZCFndnyDSTPACjTVv62qWRAY2UJmyatl2S6mK8omi8vjbuHnUYZNkmy9KX7MLeK7Bzg== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CO6PR12MB5459.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(346002)(136003)(39860400002)(396003)(376002)(366004)(451199024)(1800799009)(186009)(2616005)(26005)(6666004)(6486002)(6512007)(6506007)(53546011)(83380400001)(5660300002)(4326008)(8936002)(8676002)(2906002)(478600001)(41300700001)(110136005)(66946007)(54906003)(66476007)(66556008)(316002)(38100700002)(36756003)(31696002)(86362001)(31686004)(43740500002)(45980500001); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?UXF6NmZ0djJpSTRzZVFzcStPSVYvRjRFalA4RHByaGdJTEx6TlkvalIvWHpk?= =?utf-8?B?QVA3ek52N3czSloycnFqRWU4RUU3STA4WmZRV0cwY1hLTmlVVTgvb09RMTBi?= =?utf-8?B?L0tSTjNhQmtFRmQwdTI3UStqRTJETStGem5PcThxNnlRc05WMXdWOHlXdjBI?= =?utf-8?B?MnRXc2JITzd3S0N1TkNZUGp0YXRoZll4aFJ4MjBSQjgzOE1lZlBMLzlwUk9P?= =?utf-8?B?b0ZNYXYwajBBbTk1SkJPSjgyQXlLeG5rVVh0N0EyYnJCb2twekZWbWtSelA4?= =?utf-8?B?VVdveFh5cjFmc0pWLytUb1FyV1k0aURMQjF1RTFBakNnWlowNW1vTXNvR2w1?= =?utf-8?B?WDlaKytqSjNXTzJKZXhCeEhXbzhpVlFHV0twbXhld2ZiYS9nNHJpZ2dLWWJ3?= =?utf-8?B?TTYzaEdZNDJRTWU3bFBXN2tRQ3lzTGZBUm9yTGptRkhSeHZpZE9xVDN2eGxP?= =?utf-8?B?WU1CRmlHejhIOGlBZHg1S1NaSzF1VDh0YnlnV3VEUldvWFBodXdXeTVDT1ds?= =?utf-8?B?YWpXUXpzRjlZMVR5NFgxQnAwQmtFY2dlbUNmejhrVEs0UTNOOUpCQ0IxdFdN?= =?utf-8?B?b1pTQTVlanZmelRCQkdzY3dnMTQ3Z1F0czFqZDJsSzhmNGdYTlR5N2ZYOWE5?= =?utf-8?B?dXFSL2hkMWtmUU95M3lFR0xCZEgzMm9JWkc1Q0IyMjE2VmJtdHRiQUxWNWtQ?= =?utf-8?B?T3lUSjFra0xIR25qTTNZdFRzV2hNL0tiWTlvZ09rK1dqa2lhTVFvN0dqcjZD?= =?utf-8?B?Q1F3NG1VdWEzWnBDeGhVMk41bXV1ZDF3d3c0bDg0NXg4OHRpTDY5Tkc5V2Zt?= =?utf-8?B?VXc3RXRTWE9uMHJCNG5IaFZlMnBoNU80cUZTNHN2bEJTckFsSmcxbUdrSnIv?= =?utf-8?B?bTZGNnJUQisxMXE2c2FXSTgwWWI3SmVrb0ZLU055K2pRR2RjV3hLd2J4TjJs?= =?utf-8?B?Z1lnaE1JUXpsYjAweUxNVW5wV01CSnFiNDZObmorRWVnWFFrcENkMHQ4R04w?= =?utf-8?B?RnA3RVZEaEh0N1F3cGw1Rk5pb0RKa1VLNGVuaHdoUzVhc2ZzMTRXUGVIcnQz?= =?utf-8?B?b0RLNzkvS2hhWmxMZGRXQ2hzNmNyZHpsSHVOTlNrcWVJejZ0RWNIT01xNWls?= =?utf-8?B?ZURkdjB0ZDgyQWhBUFRvQzVCUkJDaGRwQXcvYmhZR2pJNGJreUFGeWZVemFo?= =?utf-8?B?WGRZZStoSXJDcythZW1hckZweGl0ZHFobFBLRWZodlppN3U3dmpqRk42MWFQ?= =?utf-8?B?UlBzVzF6eXpiSklPMTE2dE9RWml5MjZaSHNYaEptM2Vmd084UUd3Y0FxWEFk?= =?utf-8?B?VkRzU2ZPbTVYaEltZ0xRZ0g0NlRMRnJQYjV0UWxOTWs5OGs0alJ0UmY1cnhZ?= =?utf-8?B?dmZvelNJRVZVcllXRlIyeGNxeW01SWRFSkpXbm5SVXJzVytVcjRnQTFTejQ3?= =?utf-8?B?ZXEzMXZnRS9Rc0hzanVlUVl5QkY2S3M2T3BGSUdsVCt3ZWUzVkNwejg0RXVr?= =?utf-8?B?ODJTQTl1b0F5M2swNHJPVzhWRElSNWkrcVlQZEsrUjRHWjZ2RjV5Y1pCQkRY?= =?utf-8?B?TjJ6WVljaEhFYU5CdUMraVRwNXdrazJ3SHZOK0lVZy9EWGhocXdiUkliWWVJ?= =?utf-8?B?QzRlSkVwakRSWUk2Q201K2xaMlBHT3llaWhzc2grTWJrYmlHU2k1dSs0MjNn?= =?utf-8?B?a2ZoZ2pCZ0NRalc4MklJMk90TmxyVjRZVGpRbXUrcE5zSnB4dXJpendDdGFu?= =?utf-8?B?aW10YnNTSUQ3bThQQVFKMFpJK2p1ZEtaSFExczFpUFIyZ0dtbUVjZnRhNlBk?= =?utf-8?B?bzBqeEtlOHJtY3NqREJ4SXQvRzNETlhrNnBxMGU3eWtGZnlUdmxJa2FwdzQ4?= =?utf-8?B?eHNzUnhiOE9QYzVRSC80Q0d4L0RiS05FN0cvV1lkYjFKSDA4clVVdFF1NEor?= =?utf-8?B?VVJsK0NjNkc3dlgvajFkNEVmRW5vaGxGRVNFRzlLcTNreHR6bFdKQXlaOHh5?= =?utf-8?B?Qzl2dTltaVBKdEN5bFZLYzVrUEFDdjBkUjBmUEh6QXIyZkt3OGhvbkxYd2s0?= =?utf-8?B?Z3lpa2tIYWR2SytPYnhYRlpyUzlyUnZ6Ti82VS85S2RXZkJVQzdFbHR0Q2ts?= =?utf-8?Q?gM/Yl1apbF/8rknP9XsPsGK0M?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 32aea4ee-4e4b-4724-0d05-08db9f93577e X-MS-Exchange-CrossTenant-AuthSource: CO6PR12MB5459.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Aug 2023 02:32:18.1736 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: nDX7W7UAAdLY0hZ2GfBSrY/B2wUPL1vzXsL1ChERjpXh8qRGflRs6lF/RC2qfzQwfY14wGF+xvRqrfmZ2Iml0w== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY5PR12MB6598 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 2023/8/17 22:06, Stephen Hemminger wrote: > On Thu, 17 Aug 2023 05:06:20 +0000 > Honnappa Nagarahalli wrote: > >> Hi Matan, Viacheslav, >> Tyler pointed out that the function __mlx5_hws_cnt_pool_enqueue_revert is accessing the ring private structure members (prod.head and prod.tail) directly. Even though ' struct rte_ring' is a public structure (mainly because the library provides inline functions), the structure members are considered private to the ring library. So, this needs to be corrected. >> >> It looks like the function __mlx5_hws_cnt_pool_enqueue_revert is trying to revert things that were enqueued. It is not clear to me why this functionality is required. Can you provide the use case for this? We can discuss possible solutions. > How can reverting be thread safe? Consumer could have already looked at them? Hey, In our case, this ring is SC/SP, only accessed by one thread (enqueue/dequeue/revert). The scenario we have "revert" is:  We use ring to manager our HW objects (counter in this case) and for each core (thread) has "cache" (a SC/SP ring) for sake of performance. 1. Get objects from "cache" firstly, if cache is empty, we fetch a bulk of free objects from global ring into cache. 2. Put (free) objects also into "cache" firstly, if cache is full, we flush a bulk of objects into global ring in order to make some rooms in cache. However, this HW object cannot be immediately reused after free. It needs time to be reset and then can be used again. So when we flush cache, we want to keep the first enqueued objects still stay there because they have more chance already be reset than the latest enqueued objects. Only flush recently enqueued objects back into global ring, act as "LIFO" behavior. This is why we require "revert" enqueued objects. -Jack