From: "Mattias Rönnblom" <hofors@lysator.liu.se>
To: fengchengwen <fengchengwen@huawei.com>,
"Mattias Rönnblom" <mattias.ronnblom@ericsson.com>,
dev@dpdk.org
Cc: Thomas Monjalon <thomas@monjalon.net>,
David Marchand <david.marchand@redhat.com>,
onar.olsen@ericsson.com, Honnappa.Nagarahalli@arm.com,
nd@arm.com, konstantin.ananyev@intel.com,
mb@smartsharesystems.com, stephen@networkplumber.org,
Ola Liljedahl <ola.liljedahl@arm.com>
Subject: Re: [PATCH v5] eal: add seqlock
Date: Sun, 8 May 2022 13:56:00 +0200 [thread overview]
Message-ID: <c13d90ea-f160-6f0b-d672-ff630409b8f8@lysator.liu.se> (raw)
In-Reply-To: <cd675fe0-0fdc-aff7-e2c4-755db32a99b5@huawei.com>
On 2022-05-06 03:26, fengchengwen wrote:
> On 2022/5/1 22:03, Mattias Rönnblom wrote:
>> A sequence lock (seqlock) is synchronization primitive which allows
>> for data-race free, low-overhead, high-frequency reads, especially for
>> data structures shared across many cores and which are updated
>> relatively infrequently.
>>
>
> ...
>
>> +}
>> +
>> +static void
>> +reader_stop(struct reader *reader)
>> +{
>> + __atomic_store_n(&reader->stop, 1, __ATOMIC_RELAXED);
>> +}
>> +
>> +#define NUM_WRITERS (2) /* main lcore + one worker */
>> +#define MIN_NUM_READERS (2)
>> +#define MAX_READERS (RTE_MAX_LCORE - NUM_WRITERS - 1)
>
> Why minus 1 ?
> Suggest define MAX_READERS RTE_MAX_LCORE to avoid underflow with small size VM.
>
OK.
>> +#define MIN_LCORE_COUNT (NUM_WRITERS + MIN_NUM_READERS)
>> +
>> +/* Only a compile-time test */
>> +static rte_seqlock_t __rte_unused static_init_lock = RTE_SEQLOCK_INITIALIZER;
>> +
>> +static int
>> +test_seqlock(void)
>> +{
>> + struct reader readers[MAX_READERS];
>> + unsigned int num_readers;
>> + unsigned int num_lcores;
>> + unsigned int i;
>> + unsigned int lcore_id;
>> + unsigned int reader_lcore_ids[MAX_READERS];
>> + unsigned int worker_writer_lcore_id = 0;
>> + int rc = TEST_SUCCESS;
>> +
>> + num_lcores = rte_lcore_count();
>> +
>> + if (num_lcores < MIN_LCORE_COUNT) {
>> + printf("Too few cores to run test. Skipping.\n");
>> + return TEST_SKIPPED;
>> + }
>> +
>> + num_readers = num_lcores - NUM_WRITERS;
>> +
>> + struct data *data = rte_zmalloc(NULL, sizeof(struct data), 0);
>
> Please check whether the value of data is NULL.
>
OK.
>> +
>> + i = 0;
>> + RTE_LCORE_FOREACH_WORKER(lcore_id) {
>> + if (i == 0) {
>> + rte_eal_remote_launch(writer_run, data, lcore_id);
>> + worker_writer_lcore_id = lcore_id;
>> + } else {
>> + unsigned int reader_idx = i - 1;
>> + struct reader *reader = &readers[reader_idx];
>> +
>> + reader->data = data;
>> + reader->stop = 0;
>> +
>> + rte_eal_remote_launch(reader_run, reader, lcore_id);
>> + reader_lcore_ids[reader_idx] = lcore_id;
>> + }
>> + i++;
>> + }
>> +
>> + if (writer_run(data) != 0 ||
>> + rte_eal_wait_lcore(worker_writer_lcore_id) != 0)
>> + rc = TEST_FAILED;
>> +
>> + for (i = 0; i < num_readers; i++) {
>> + reader_stop(&readers[i]);
>> + if (rte_eal_wait_lcore(reader_lcore_ids[i]) != 0)
>> + rc = TEST_FAILED;
>> + }
>> +
>
> Please free data memory.
>
OK.
>> + return rc;
>> +}
>> +
>> +REGISTER_TEST_COMMAND(seqlock_autotest, test_seqlock);
>> diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md
>> index 4245b9635c..f23e33ae30 100644
>> --- a/doc/api/doxy-api-index.md
>> +++ b/doc/api/doxy-api-index.md
>> @@ -77,6 +77,7 @@ The public API headers are grouped by topics:
>> [rwlock] (@ref rte_rwlock.h),
>> [spinlock] (@ref rte_spinlock.h),
>> [ticketlock] (@ref rte_ticketlock.h),
>> + [seqlock] (@ref rte_seqlock.h),
>> [RCU] (@ref rte_rcu_qsbr.h)
>>
>
> ...
>
>> + */
>> +__rte_experimental
>> +static inline bool
>> +rte_seqlock_read_retry(const rte_seqlock_t *seqlock, uint32_t begin_sn)
>> +{
>> + uint32_t end_sn;
>> +
>> + /* An odd sequence number means the protected data was being
>> + * modified already at the point of the rte_seqlock_read_begin()
>> + * call.
>> + */
>> + if (unlikely(begin_sn & 1))
>> + return true;
>> +
>> + /* make sure the data loads happens before the sn load */
>> + rte_atomic_thread_fence(__ATOMIC_ACQUIRE);
>
> In ARMv8, the rte_atomic_thread_fence(__ATOMIC_ACQUIRE) and rte_smp_rmb() both output 'dma ishld'
> Suggest use rte_smp_rmb(), please see below comment.
>
>> +
>> + end_sn = __atomic_load_n(&seqlock->sn, __ATOMIC_RELAXED);
>> +
>> + /* A writer incremented the sequence number during this read
>> + * critical section.
>> + */
>> + if (unlikely(begin_sn != end_sn))
>> + return true;
>> +
>> + return false;
>> +}
>> +
>> +/**
>> + * @warning
>> + * @b EXPERIMENTAL: this API may change without prior notice.
>> + *
>> + * Begin a write-side critical section.
>> + *
>> + * A call to this function acquires the write lock associated @p
>> + * seqlock, and marks the beginning of a write-side critical section.
>> + *
>> + * After having called this function, the caller may go on to modify
>> + * (both read and write) the protected data, in an atomic or
>> + * non-atomic manner.
>> + *
>> + * After the necessary updates have been performed, the application
>> + * calls rte_seqlock_write_unlock().
>> + *
>> + * This function is not preemption-safe in the sense that preemption
>> + * of the calling thread may block reader progress until the writer
>> + * thread is rescheduled.
>> + *
>> + * Unlike rte_seqlock_read_begin(), each call made to
>> + * rte_seqlock_write_lock() must be matched with an unlock call.
>> + *
>> + * @param seqlock
>> + * A pointer to the seqlock.
>> + *
>> + * @see rte_seqlock_write_unlock()
>> + */
>> +__rte_experimental
>> +static inline void
>> +rte_seqlock_write_lock(rte_seqlock_t *seqlock)
>> +{
>> + uint32_t sn;
>> +
>> + /* to synchronize with other writers */
>> + rte_spinlock_lock(&seqlock->lock);
>> +
>> + sn = seqlock->sn + 1;
>> +
>> + __atomic_store_n(&seqlock->sn, sn, __ATOMIC_RELAXED);
>> +
>> + /* __ATOMIC_RELEASE to prevent stores after (in program order)
>> + * from happening before the sn store.
>> + */
>> + rte_atomic_thread_fence(__ATOMIC_RELEASE);
>
> In ARMv8, rte_atomic_thread_fence(__ATOMIC_RELEASE) will output 'dmb ish', and
> rte_smp_wmb() will output 'dma ishst'.
> Suggest use rte_smp_wmb(). I think here only need to use store mb here.
>
(This has already been discussed further down in the mail thread, and I
have nothing to add.)
>> +}
>> +
>> +/**
>> + * @warning
>> + * @b EXPERIMENTAL: this API may change without prior notice.
>> + *
>> + * End a write-side critical section.
>> + *
>> + * A call to this function marks the end of the write-side critical
>> + * section, for @p seqlock. After this call has been made, the protected
>> + * data may no longer be modified.
>> + *
>> + * @param seqlock
>> + * A pointer to the seqlock.
>> + *
>> + * @see rte_seqlock_write_lock()
>> + */
>> +__rte_experimental
>> +static inline void
>> +rte_seqlock_write_unlock(rte_seqlock_t *seqlock)
>> +{
>> + uint32_t sn;
>> +
>> + sn = seqlock->sn + 1;
>> +
>> + /* synchronizes-with the load acquire in rte_seqlock_read_begin() */
>> + __atomic_store_n(&seqlock->sn, sn, __ATOMIC_RELEASE);
>> +
>> + rte_spinlock_unlock(&seqlock->lock);
>> +}
>> +
>> +#ifdef __cplusplus
>> +}
>> +#endif
>> +
>> +#endif /* _RTE_SEQLOCK_H_ */
>> diff --git a/lib/eal/version.map b/lib/eal/version.map
>> index b53eeb30d7..4a9d0ed899 100644
>> --- a/lib/eal/version.map
>> +++ b/lib/eal/version.map
>> @@ -420,6 +420,9 @@ EXPERIMENTAL {
>> rte_intr_instance_free;
>> rte_intr_type_get;
>> rte_intr_type_set;
>> +
>> + # added in 22.07
>> + rte_seqlock_init;
>> };
>>
>> INTERNAL {
>>
>
> Reviewed-by: Chengwen Feng <fengchengwen@huawei.com>
>
>
Thanks a lot for the review!
next prev parent reply other threads:[~2022-05-08 11:56 UTC|newest]
Thread overview: 104+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-03-22 16:10 DPDK seqlock Mattias Rönnblom
2022-03-22 16:46 ` Ananyev, Konstantin
2022-03-24 4:52 ` Honnappa Nagarahalli
2022-03-24 5:06 ` Stephen Hemminger
2022-03-24 11:34 ` Mattias Rönnblom
2022-03-25 20:24 ` [RFC] eal: add seqlock Mattias Rönnblom
2022-03-25 21:10 ` Stephen Hemminger
2022-03-26 14:57 ` Mattias Rönnblom
2022-03-27 14:49 ` Ananyev, Konstantin
2022-03-27 17:42 ` Mattias Rönnblom
2022-03-28 10:53 ` Ananyev, Konstantin
2022-03-28 14:06 ` Ola Liljedahl
2022-03-29 8:32 ` Mattias Rönnblom
2022-03-29 13:20 ` Ananyev, Konstantin
2022-03-30 10:07 ` [PATCH] " Mattias Rönnblom
2022-03-30 10:50 ` Morten Brørup
2022-03-30 11:24 ` Tyler Retzlaff
2022-03-30 11:25 ` Mattias Rönnblom
2022-03-30 14:26 ` [PATCH v2] " Mattias Rönnblom
2022-03-31 7:46 ` Mattias Rönnblom
2022-03-31 9:04 ` Ola Liljedahl
2022-03-31 9:25 ` Morten Brørup
2022-03-31 9:38 ` Ola Liljedahl
2022-03-31 10:03 ` Morten Brørup
2022-03-31 11:44 ` Ola Liljedahl
2022-03-31 11:50 ` Morten Brørup
2022-03-31 14:02 ` Mattias Rönnblom
2022-04-01 15:07 ` [PATCH v3] " Mattias Rönnblom
2022-04-02 0:21 ` Honnappa Nagarahalli
2022-04-02 11:01 ` Morten Brørup
2022-04-02 19:38 ` Honnappa Nagarahalli
2022-04-10 13:51 ` [RFC 1/3] eal: add macro to warn for unused function return values Mattias Rönnblom
2022-04-10 13:51 ` [RFC 2/3] eal: emit warning for unused trylock return value Mattias Rönnblom
2022-04-10 13:51 ` [RFC 3/3] examples/bond: fix invalid use of trylock Mattias Rönnblom
2022-04-11 1:01 ` Min Hu (Connor)
2022-04-11 14:32 ` Mattias Rönnblom
2022-04-11 11:25 ` David Marchand
2022-04-11 14:33 ` Mattias Rönnblom
2022-04-10 18:02 ` [RFC 1/3] eal: add macro to warn for unused function return values Stephen Hemminger
2022-04-10 18:50 ` Mattias Rönnblom
2022-04-11 7:17 ` Morten Brørup
2022-04-11 14:29 ` Mattias Rönnblom
2022-04-11 9:16 ` Bruce Richardson
2022-04-11 14:27 ` Mattias Rönnblom
2022-04-11 15:15 ` [PATCH " Mattias Rönnblom
2022-04-11 15:15 ` [PATCH 2/3] eal: emit warning for unused trylock return value Mattias Rönnblom
2022-04-11 15:29 ` Morten Brørup
2022-04-11 15:15 ` [PATCH 3/3] examples/bond: fix invalid use of trylock Mattias Rönnblom
2022-04-14 12:06 ` David Marchand
2022-04-11 15:25 ` [PATCH 1/3] eal: add macro to warn for unused function return values Morten Brørup
2022-04-11 18:24 ` [RFC " Tyler Retzlaff
2022-04-03 6:10 ` [PATCH v3] eal: add seqlock Mattias Rönnblom
2022-04-03 17:27 ` Honnappa Nagarahalli
2022-04-03 18:37 ` Ola Liljedahl
2022-04-04 21:56 ` Honnappa Nagarahalli
2022-04-03 6:33 ` Mattias Rönnblom
2022-04-03 17:37 ` Honnappa Nagarahalli
2022-04-08 13:45 ` Mattias Rönnblom
2022-04-02 18:15 ` Ola Liljedahl
2022-04-02 19:31 ` Honnappa Nagarahalli
2022-04-02 20:36 ` Morten Brørup
2022-04-02 22:01 ` Honnappa Nagarahalli
2022-04-03 18:11 ` Ola Liljedahl
2022-04-03 6:51 ` Mattias Rönnblom
2022-03-31 13:51 ` [PATCH v2] " Mattias Rönnblom
2022-04-02 0:54 ` Stephen Hemminger
2022-04-02 10:25 ` Morten Brørup
2022-04-02 17:43 ` Ola Liljedahl
2022-03-31 13:38 ` Mattias Rönnblom
2022-03-31 14:53 ` Ola Liljedahl
2022-04-02 0:52 ` Stephen Hemminger
2022-04-03 6:23 ` Mattias Rönnblom
2022-04-02 0:50 ` Stephen Hemminger
2022-04-02 17:54 ` Ola Liljedahl
2022-04-02 19:37 ` Honnappa Nagarahalli
2022-04-05 20:16 ` Stephen Hemminger
2022-04-08 13:50 ` Mattias Rönnblom
2022-04-08 14:24 ` [PATCH v4] " Mattias Rönnblom
2022-04-08 15:17 ` Stephen Hemminger
2022-04-08 16:24 ` Mattias Rönnblom
2022-04-08 15:19 ` Stephen Hemminger
2022-04-08 16:37 ` Mattias Rönnblom
2022-04-08 16:48 ` Mattias Rönnblom
2022-04-12 17:27 ` Ananyev, Konstantin
2022-04-28 10:28 ` David Marchand
2022-05-01 13:46 ` Mattias Rönnblom
2022-05-01 14:03 ` [PATCH v5] " Mattias Rönnblom
2022-05-01 14:22 ` Mattias Rönnblom
2022-05-02 6:47 ` David Marchand
2022-05-01 20:17 ` Stephen Hemminger
2022-05-02 4:51 ` Mattias Rönnblom
2022-05-06 1:26 ` fengchengwen
2022-05-06 1:33 ` Honnappa Nagarahalli
2022-05-06 4:17 ` fengchengwen
2022-05-06 5:19 ` Honnappa Nagarahalli
2022-05-06 7:03 ` fengchengwen
2022-05-08 11:56 ` Mattias Rönnblom [this message]
2022-05-08 12:12 ` [PATCH v6] " Mattias Rönnblom
2022-05-08 16:10 ` Stephen Hemminger
2022-05-08 19:40 ` Mattias Rönnblom
2022-05-09 3:48 ` Stephen Hemminger
2022-05-09 6:26 ` Morten Brørup
2022-05-13 6:27 ` Mattias Rönnblom
2022-03-23 12:04 ` DPDK seqlock Morten Brørup
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=c13d90ea-f160-6f0b-d672-ff630409b8f8@lysator.liu.se \
--to=hofors@lysator.liu.se \
--cc=Honnappa.Nagarahalli@arm.com \
--cc=david.marchand@redhat.com \
--cc=dev@dpdk.org \
--cc=fengchengwen@huawei.com \
--cc=konstantin.ananyev@intel.com \
--cc=mattias.ronnblom@ericsson.com \
--cc=mb@smartsharesystems.com \
--cc=nd@arm.com \
--cc=ola.liljedahl@arm.com \
--cc=onar.olsen@ericsson.com \
--cc=stephen@networkplumber.org \
--cc=thomas@monjalon.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).