DPDK patches and discussions
 help / color / mirror / Atom feed
From: "Wang, Yipeng1" <yipeng1.wang@intel.com>
To: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>,
	Stephen Hemminger <stephen@networkplumber.org>
Cc: "Gobriel, Sameh" <sameh.gobriel@intel.com>,
	"Richardson, Bruce" <bruce.richardson@intel.com>,
	"dev@dpdk.org" <dev@dpdk.org>,
	"De Lara Guarch, Pablo" <pablo.de.lara.guarch@intel.com>,
	nd <nd@arm.com>, nd <nd@arm.com>
Subject: Re: [dpdk-dev] [PATCH] hash: document breakage with multi-writer thread
Date: Thu, 4 Jun 2020 20:22:06 +0000
Message-ID: <BYAPR11MB34941A418240B00C5AD77D64C3890@BYAPR11MB3494.namprd11.prod.outlook.com> (raw)
In-Reply-To: <AM4PR0802MB2212599ACC592EC4A42965A298890@AM4PR0802MB2212.eurprd08.prod.outlook.com>

> -----Original Message-----
> From: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>
> Sent: Thursday, June 4, 2020 12:34 PM
> To: Wang, Yipeng1 <yipeng1.wang@intel.com>; Stephen Hemminger
> <stephen@networkplumber.org>
> Cc: Gobriel, Sameh <sameh.gobriel@intel.com>; Richardson, Bruce
> <bruce.richardson@intel.com>; dev@dpdk.org; De Lara Guarch, Pablo
> <pablo.de.lara.guarch@intel.com>; nd <nd@arm.com>; Honnappa
> Nagarahalli <Honnappa.Nagarahalli@arm.com>; nd <nd@arm.com>
> Subject: RE: [PATCH] hash: document breakage with multi-writer thread
> 
> <snip>
> 
> > > > >
> > > > > > Subject: [PATCH] hash: document breakage with multi-writer
> > > > > > thread
> > > > > >
> > > > > > The code in rte_cuckoo_hash multi-writer support is broken if
> > > > > > write operations are called from a non-EAL thread.
> > > > > >
> > > > > > rte_lcore_id() wil return LCORE_ID_ANY (UINT32_MAX) for non
> > > > > > EAL thread and that leads to using wrong local cache.
> > > > > >
> > > > > > Add error checks and document the restriction.
> > > > > Having multiple non-EAL writer threads is a valid use case.
> > > > > Should we fix the
> > > > issue instead?
> > > >
> > > > Discovered this the hard way...
> > > >
> > > > Fixing is non-trivial. Basically, the local cache has to be take
> > > > out and that leads to having to do real locking or atomic operations.
> > > Looking at rte_hash_create function:
> > >
> > >         if (params->extra_flag &
> > > RTE_HASH_EXTRA_FLAGS_MULTI_WRITER_ADD) {
> > >                 use_local_cache = 1;
> > >                 writer_takes_lock = 1;
> > >         }
> > >
> > > The writer locks are in place already. The code to handle the case
> > > when local cache is taken out is also there.
> > > What we need is another input flag that says 'multi writer + non-eal
> > threads'
> > > which would set 'use_local_cache = 0' and 'writer_takes_lock = 1'.
> > > Not sure, it would be valuable addition. But looks like this is what
> > > you were expecting when you had enabled
> > > 'RTE_HASH_EXTRA_FLAGS_MULTI_WRITER_ADD'. Many other APIs in
> DPDK
> > do
> > > not provide this kind of MT safety.
> >
> > [Wang, Yipeng]
> > If possible, we can try to not add new flags, because there are
> > already a lot of flag options.
> > How about in the code, we check if the writer is a non-eal or not by
> > checking the rte_lcore_id, and operate on the global queue?
> > Could this work?
> > If(h->use_local_cache) {
> > 	lcore_id = rte_lcore_id();
> > 	if(lcore_id == LCORE_ID_ANY) {   // this is non-eal threads
> > 		<call rte_ring_mp/mc_* to directly operate on global queue>
> > 	}
> > 	Else {
> > 		<original path>
> > 	}
> > }
> The other thing I wanted to do was saving on the memory allocated for the
> local cache when the writers are non-eal threads. Without knowing the kind
> of threads upfront, we might have to create the local cache when a writer
> adds the entry first time.

I got what you mean.  If people only use non-eal threads, we could save the space of local cache completely. 
Creating local cache during the first write is one solution. But the current rte_hash always allocate things during
table creation time. This provides guarantee that the program won't fail in the middle due to memory allocation issue.

Meanwhile I would rather be wasting some space than adding another option flag related to multi-threading.
In my opinion, all those flags are already confusing enough. It would also be harder to maintain in future.

 

  reply	other threads:[~2020-06-04 20:22 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-06-04 17:17 Stephen Hemminger
2020-06-04 17:51 ` Honnappa Nagarahalli
2020-06-04 17:58   ` Stephen Hemminger
2020-06-04 18:43     ` Honnappa Nagarahalli
2020-06-04 19:10       ` Wang, Yipeng1
2020-06-04 19:34         ` Honnappa Nagarahalli
2020-06-04 20:22           ` Wang, Yipeng1 [this message]
2020-06-04 21:06             ` Stephen Hemminger
2020-06-04 21:32 ` Wang, Yipeng1
2020-06-05 18:35 ` Stephen Hemminger
2020-06-16 16:12   ` Thomas Monjalon
2020-11-26 17:56     ` Thomas Monjalon

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=BYAPR11MB34941A418240B00C5AD77D64C3890@BYAPR11MB3494.namprd11.prod.outlook.com \
    --to=yipeng1.wang@intel.com \
    --cc=Honnappa.Nagarahalli@arm.com \
    --cc=bruce.richardson@intel.com \
    --cc=dev@dpdk.org \
    --cc=nd@arm.com \
    --cc=pablo.de.lara.guarch@intel.com \
    --cc=sameh.gobriel@intel.com \
    --cc=stephen@networkplumber.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

DPDK patches and discussions

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://inbox.dpdk.org/dev/0 dev/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 dev dev/ https://inbox.dpdk.org/dev \
		dev@dpdk.org
	public-inbox-index dev

Example config snippet for mirrors.
Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.dev


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git