From: "Mattias Rönnblom" <hofors@lysator.liu.se>
To: Bruce Richardson <bruce.richardson@intel.com>,
"Van Haaren, Harry" <harry.van.haaren@intel.com>
Cc: "Honnappa Nagarahalli" <Honnappa.Nagarahalli@arm.com>,
"dev@dpdk.org" <dev@dpdk.org>,
"mattias.ronnblom" <mattias.ronnblom@ericsson.com>,
"Morten Brørup" <mb@smartsharesystems.com>, nd <nd@arm.com>
Subject: Re: [PATCH 2/2] service: fix potential stats race-condition on MT services
Date: Fri, 8 Jul 2022 22:02:24 +0200 [thread overview]
Message-ID: <dfc227f8-e758-ebf3-4cc7-b4811f22c46c@lysator.liu.se> (raw)
In-Reply-To: <YshZcquSc7YNgo9A@bricha3-MOBL.ger.corp.intel.com>
On 2022-07-08 18:21, Bruce Richardson wrote:
> On Fri, Jul 08, 2022 at 03:31:01PM +0000, Van Haaren, Harry wrote:
>>> -----Original Message-----
>>> From: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>
>>> Sent: Friday, July 8, 2022 4:16 PM
>>> To: Van Haaren, Harry <harry.van.haaren@intel.com>; dev@dpdk.org
>>> Cc: mattias.ronnblom <mattias.ronnblom@ericsson.com>; Morten Brørup
>>> <mb@smartsharesystems.com>; nd <nd@arm.com>; nd <nd@arm.com>
>>> Subject: RE: [PATCH 2/2] service: fix potential stats race-condition on MT services
>>
>> <snip previous discussions>
>>
>>>> diff --git a/lib/eal/common/rte_service.c b/lib/eal/common/rte_service.c
>>>> index ef31b1f63c..f045e74ef3 100644
>>>> --- a/lib/eal/common/rte_service.c
>>>> +++ b/lib/eal/common/rte_service.c
>>>> @@ -363,9 +363,15 @@ service_runner_do_callback(struct
>>>> rte_service_spec_impl *s,
>>>> uint64_t start = rte_rdtsc();
>>>> s->spec.callback(userdata);
>>>> uint64_t end = rte_rdtsc();
>>>> - s->cycles_spent += end - start;
>>>> + uint64_t cycles = end - start;
>>>> cs->calls_per_service[service_idx]++;
>>>> - s->calls++;
>>>> + if (service_mt_safe(s)) {
>>>> + __atomic_fetch_add(&s->cycles_spent, cycles,
>>>> __ATOMIC_RELAXED);
>>>> + __atomic_fetch_add(&s->calls, 1,
>>>> __ATOMIC_RELAXED);
>>>> + } else {
>>>> + s->cycles_spent += cycles;
>>>> + s->calls++;
>>> This is still a problem from a reader perspective. It is possible that the writes could be
>>> split while a reader is reading the stats. These need to be atomic adds.
>>
>> Thanks for pointing out; I do "think" in x86 in terms of load/store tearing; and on x86
>> naturally aligned load/stores will not tear. Apologies for missing the ARM angle here.
>>
>> I'm not sure how to best encode the difference between tearing & "locked instructions"
>> to make things multi-writer safe. But they're not the same thing, and I'd prefer not pay
>> the penalty for LOCK instructions (multi-writer) only to satisfy the non-tearing requirements.
>>
>> Is there an rte_atomic-* type that is guaranteed as non-tearing?
>>
>> In that case, changing the type of the calls/cycles_spent variables to such a type to ensure "non-tearing"
>> single-reader, single-writer behaviour is enough, instead of forcing __atomic_fetch_add() everywhere?
>
> Regular read, increment and then atomic store should work without locks
> where alignment is correct on most architectures, and doing the store
> atomically should prevent any tearing.
This is a good pattern, and provides a noticeable performance benefit
compared to using an atomic add (which is an overkill in single-writer
scenarios), in my experience. The DPDK seqcount implementation
increments the count in this manner.
next prev parent reply other threads:[~2022-07-08 20:02 UTC|newest]
Thread overview: 43+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-07-08 12:56 [PATCH 1/2] test/service: add perf measurements for with stats mode Harry van Haaren
2022-07-08 12:56 ` [PATCH 2/2] service: fix potential stats race-condition on MT services Harry van Haaren
2022-07-08 13:23 ` Morten Brørup
2022-07-08 13:44 ` Van Haaren, Harry
2022-07-08 14:14 ` Morten Brørup
2022-07-08 13:48 ` Mattias Rönnblom
2022-07-08 15:16 ` Honnappa Nagarahalli
2022-07-08 15:31 ` Van Haaren, Harry
2022-07-08 16:21 ` Bruce Richardson
2022-07-08 16:33 ` Honnappa Nagarahalli
2022-07-08 20:02 ` Mattias Rönnblom [this message]
2022-07-08 16:29 ` Morten Brørup
2022-07-08 16:45 ` Honnappa Nagarahalli
2022-07-08 17:22 ` Morten Brørup
2022-07-08 17:39 ` Honnappa Nagarahalli
2022-07-08 18:08 ` Morten Brørup
2022-09-06 16:13 ` [PATCH 1/6] service: reduce statistics overhead for parallel services Mattias Rönnblom
2022-09-06 16:13 ` [PATCH 2/6] service: introduce per-lcore cycles counter Mattias Rönnblom
2022-09-06 16:13 ` [PATCH 3/6] service: reduce average case service core overhead Mattias Rönnblom
2022-10-03 13:33 ` Van Haaren, Harry
2022-10-03 14:32 ` Mattias Rönnblom
2022-09-06 16:13 ` [PATCH 4/6] service: tweak cycle statistics semantics Mattias Rönnblom
2022-09-07 8:41 ` Morten Brørup
2022-10-03 13:45 ` Van Haaren, Harry
2022-09-06 16:13 ` [PATCH 5/6] event/sw: report idle when no work is performed Mattias Rönnblom
2022-09-06 16:13 ` [PATCH 6/6] service: provide links to functions in documentation Mattias Rönnblom
2022-10-03 8:06 ` [PATCH 1/6] service: reduce statistics overhead for parallel services David Marchand
2022-10-03 8:40 ` Mattias Rönnblom
2022-10-03 9:53 ` David Marchand
2022-10-03 11:37 ` Mattias Rönnblom
2022-10-03 13:03 ` Van Haaren, Harry
2022-10-03 13:33 ` Van Haaren, Harry
2022-10-03 14:37 ` Mattias Rönnblom
2022-10-05 9:16 ` [PATCH v2 0/6] Service cores performance and statistics improvements Mattias Rönnblom
2022-10-05 9:16 ` [PATCH v2 1/6] service: reduce statistics overhead for parallel services Mattias Rönnblom
2022-10-05 9:16 ` [PATCH v2 2/6] service: introduce per-lcore cycles counter Mattias Rönnblom
2022-10-05 9:16 ` [PATCH v2 3/6] service: reduce average case service core overhead Mattias Rönnblom
2022-10-05 9:16 ` [PATCH v2 4/6] service: tweak cycle statistics semantics Mattias Rönnblom
2022-10-05 9:16 ` [PATCH v2 5/6] event/sw: report idle when no work is performed Mattias Rönnblom
2022-10-05 9:16 ` [PATCH v2 6/6] service: provide links to functions in documentation Mattias Rönnblom
2022-10-05 9:49 ` [PATCH v2 0/6] Service cores performance and statistics improvements Morten Brørup
2022-10-05 10:14 ` Mattias Rönnblom
2022-10-05 13:39 ` David Marchand
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=dfc227f8-e758-ebf3-4cc7-b4811f22c46c@lysator.liu.se \
--to=hofors@lysator.liu.se \
--cc=Honnappa.Nagarahalli@arm.com \
--cc=bruce.richardson@intel.com \
--cc=dev@dpdk.org \
--cc=harry.van.haaren@intel.com \
--cc=mattias.ronnblom@ericsson.com \
--cc=mb@smartsharesystems.com \
--cc=nd@arm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).