From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 28A5A43F05; Thu, 25 Apr 2024 17:06:13 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2841943882; Thu, 25 Apr 2024 17:06:08 +0200 (CEST) Received: from mail.lysator.liu.se (mail.lysator.liu.se [130.236.254.3]) by mails.dpdk.org (Postfix) with ESMTP id 2EF374387F for ; Thu, 25 Apr 2024 17:06:07 +0200 (CEST) Received: from mail.lysator.liu.se (localhost [127.0.0.1]) by mail.lysator.liu.se (Postfix) with ESMTP id D80B9FA41 for ; Thu, 25 Apr 2024 17:06:06 +0200 (CEST) Received: by mail.lysator.liu.se (Postfix, from userid 1004) id B5E33F9A9; Thu, 25 Apr 2024 17:06:06 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-13) on hermod.lysator.liu.se X-Spam-Level: X-Spam-Status: No, score=-1.3 required=5.0 tests=ALL_TRUSTED,AWL, T_SCC_BODY_TEXT_LINE autolearn=disabled version=4.0.0 X-Spam-Score: -1.3 Received: from [192.168.1.59] (h-62-63-215-114.A163.priv.bahnhof.se [62.63.215.114]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mail.lysator.liu.se (Postfix) with ESMTPSA id 0A7C1FA36; Thu, 25 Apr 2024 17:06:03 +0200 (CEST) Message-ID: <37e85e59-585b-4503-81c2-2f8fc0a6746f@lysator.liu.se> Date: Thu, 25 Apr 2024 17:06:02 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH] net/af_packet: cache align Rx/Tx structs To: Ferruh Yigit , Stephen Hemminger Cc: =?UTF-8?Q?Mattias_R=C3=B6nnblom?= , "John W . Linville" , dev@dpdk.org, Tyler Retzlaff , Honnappa Nagarahalli , =?UTF-8?Q?Morten_Br=C3=B8rup?= References: <20240423090813.94110-1-mattias.ronnblom@ericsson.com> <6f7aabcb-2c12-4cfe-ae9d-73b42bfd4977@amd.com> <63dbb564-61f6-4d9f-9c13-4a21f5e97dc9@lysator.liu.se> <5d2a0887-527a-4948-943c-65f1dfda9328@amd.com> <3b2cf48e-2293-4226-b6cd-5f4dd3969f99@lysator.liu.se> <0ff40e60-926b-44eb-8af5-2e16aff1c336@amd.com> <20240424121330.7547e290@hermes.local> <2371b1a8-bdc5-4184-8491-54e2e3a64211@lysator.liu.se> <20240424165534.2ad8ae0a@hermes.local> <4c0f1625-4387-4870-a60a-dc423885811e@lysator.liu.se> <2d07603e-e024-45a4-bfb1-c350b08ccdfb@amd.com> Content-Language: en-US From: =?UTF-8?Q?Mattias_R=C3=B6nnblom?= In-Reply-To: <2d07603e-e024-45a4-bfb1-c350b08ccdfb@amd.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Virus-Scanned: ClamAV using ClamSMTP X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 2024-04-25 16:04, Ferruh Yigit wrote: > On 4/25/2024 10:26 AM, Mattias Rönnblom wrote: >> On 2024-04-25 01:55, Stephen Hemminger wrote: >>> On Thu, 25 Apr 2024 00:27:36 +0200 >>> Mattias Rönnblom wrote: >>> >>>> On 2024-04-24 21:13, Stephen Hemminger wrote: >>>>> On Wed, 24 Apr 2024 18:50:50 +0100 >>>>> Ferruh Yigit wrote: >>>>> >>>>>>> I don't know how slow af_packet is, but if you care about >>>>>>> performance, >>>>>>> you don't want to use atomic add for statistics. >>>>>>> >>>>>> >>>>>> There are a few soft drivers already using atomics adds for >>>>>> updating stats. >>>>>> If we document expectations from 'rte_eth_stats_reset()', we can >>>>>> update >>>>>> those usages. >>>>> >>>>> Using atomic add is lots of extra overhead. The statistics are not >>>>> guaranteed >>>>> to be perfect.  If nothing else, the bytes and packets can be skewed. >>>>> >>>> >>>> The sad thing here is that in case the counters are reset within the >>>> load-modify-store cycle of the lcore counter update, the reset may end >>>> up being a nop. So, it's not like you missed a packet or two, or suffer >>>> some transient inconsistency, but you completed and permanently ignored >>>> the reset request. >>> >>> That is one of the many reasons the Linux kernel intentionally did not >>> implement a reset statistics operation. >> >> DPDK should deprecate statistics reset, it seems to me. >> >> The current API is broken anyway, if you care about correctness. A reset >> function must return the current state of the counters, at the point >> them being reset. Otherwise, a higher layer may miss counter updates. >> >> The af_packet PMD should return -ENOTSUP on reset (which is allowed by >> the API). >> >> Maintaining an offset-since-last-reset for counters is a control plane >> thing, and shouldn't be in PMDs. (If MT-safe reset for SW-managed >> counters are to be expected from the PMDs, we should have some helper >> API to facilitate its efficient & correct-enough implementation.) >> > > statistics reset works for HW devices, instead of removing statics reset > I am for documenting API that it may be not reliable, I can send a patch > for it. > With API you mean ? If rte_ethdev_stats_reset() sometimes reset the counters, and sometimes doesn't, it should also have a name that reflect those semantics. rte_ethdev_stats_reset_or_perhaps_not() rte_ethdev_stats_usually_reset() Rather than expecting the application to deal with unspecified non-determinism is seems better to specify under which conditions the reset is reliable (i.e., it's not MT safe). A non-MT-safe reset will limit it usefulness though. Also, it will make having an MT safe reset in a PMD pretty useless, except if the app is programmed not toward the API, but toward some particular PMD. > With above change, we can be more relax on stats update specially for > soft drivers, and can convert atomic_add stats updates to "atomic load + > add + atomic store". > > Does this plan make sense? Not really. Short-term the -ENOTSUP seems like the best option. Second best option is to implement a proper MT safe reset. What is unfortunate is that the API is silent on MT safety. I've *assumed* that many users will have assumed it MT safe, but there's nothing in the documentation to support that. Rather the opposite, the generic section of DPDK MT safety only mentions PMD RX and TX functions. This issue is not limited to this PMD or even to ethdev. rte_event_dev_xstats_reset() and some of the event devs have the same problem.