DPDK patches and discussions
 help / color / mirror / Atom feed
From: "Ananyev, Konstantin" <konstantin.ananyev@intel.com>
To: "Richardson, Bruce" <bruce.richardson@intel.com>
Cc: Ivan Boule <ivan.boule@6wind.com>,
	Thomas Monjalon <thomas.monjalon@6wind.com>,
	"Pattan, Reshma" <reshma.pattan@intel.com>,
	"dev@dpdk.org" <dev@dpdk.org>
Subject: Re: [dpdk-dev] [PATCH v9 1/8] ethdev: use locks to protect Rx/Tx callback lists
Date: Wed, 15 Jun 2016 14:27:15 +0000	[thread overview]
Message-ID: <2601191342CEEE43887BDE71AB97725836B7194A@irsmsx105.ger.corp.intel.com> (raw)
In-Reply-To: <20160615142223.GC11536@bricha3-MOBL3>



> -----Original Message-----
> From: Richardson, Bruce
> Sent: Wednesday, June 15, 2016 3:22 PM
> To: Ananyev, Konstantin
> Cc: Ivan Boule; Thomas Monjalon; Pattan, Reshma; dev@dpdk.org
> Subject: Re: [dpdk-dev] [PATCH v9 1/8] ethdev: use locks to protect Rx/Tx callback lists
> 
> On Wed, Jun 15, 2016 at 03:20:27PM +0100, Ananyev, Konstantin wrote:
> >
> >
> > > -----Original Message-----
> > > From: Ivan Boule [mailto:ivan.boule@6wind.com]
> > > Sent: Wednesday, June 15, 2016 3:07 PM
> > > To: Richardson, Bruce; Ananyev, Konstantin
> > > Cc: Thomas Monjalon; Pattan, Reshma; dev@dpdk.org
> > > Subject: Re: [dpdk-dev] [PATCH v9 1/8] ethdev: use locks to protect Rx/Tx callback lists
> > >
> > > On 06/15/2016 03:29 PM, Bruce Richardson wrote:
> > > > On Wed, Jun 15, 2016 at 12:40:16PM +0000, Ananyev, Konstantin wrote:
> > > >> Hi Ivan,
> > > >>
> > > >>> -----Original Message-----
> > > >>> From: Ivan Boule [mailto:ivan.boule@6wind.com]
> > > >>> Sent: Wednesday, June 15, 2016 1:15 PM
> > > >>> To: Thomas Monjalon; Ananyev, Konstantin
> > > >>> Cc: Pattan, Reshma; dev@dpdk.org
> > > >>> Subject: Re: [dpdk-dev] [PATCH v9 1/8] ethdev: use locks to protect Rx/Tx callback lists
> > > >>>
> > > >>> On 06/15/2016 10:48 AM, Thomas Monjalon wrote:
> > > >>>
> > > >>>>>
> > > >>>>>> I think the read access would need locking but we do not want it
> > > >>>>>> in fast path.
> > > >>>>>
> > > >>>>> I don't think it would be needed.
> > > >>>>> As I said - read/write interaction didn't change from what we have right now.
> > > >>>>> But if you have some particular scenario in mind that you believe would cause
> > > >>>>> a race condition - please speak up.
> > > >>>>
> > > >>>> If we add/remove a callback during a burst? Is it possible that the next
> > > >>>> pointer would have a wrong value leading to a crash?
> > > >>>> Maybe we need a comment to state that we should not alter burst
> > > >>>> callbacks while running burst functions.
> > > >>>>
> > > >>>
> > > >>> Hi Reshma,
> > > >>>
> > > >>> You claim that the "rte_eth_rx_cb_lock" does not need to be hold in the
> > > >>> function "rte_eth_rx_burst()" while parsing the "post_rx_burst_cbs" list
> > > >>> of RX callbacks associated with the polled RX queue to safely remove RX
> > > >>> callback(s) in parallel.
> > > >>> The problem is not [only] with the setting and the loading of "cb->next"
> > > >>> that you assume to be atomic operations, which is certainly true on most
> > > >>> CPUs.
> > > >>> I see the 2 important following issues:
> > > >>>
> > > >>> 1) the "rte_eth_rxtx_callback" data structure associated with a removed
> > > >>> RX callback could still be accessed in the callback parsing loop of the
> > > >>> function "rte_eth_rx_burst()" after having been freed in parallel.
> > > >>>
> > > >>> BUT SUCH A BAD SITUATION WILL NOT CURRENTLY HAPPEN, THANKS TO THE NICE
> > > >>> MEMORY LEAK BUG in the function "rte_eth_remove_rx_callback()"  that
> > > >>> does not free the "rte_eth_rxtx_callback" data structure associated with
> > > >>> the removed callback !
> > > >>
> > > >> Yes, though it is documented behaviour, someone can probably
> > > >> refer it as a feature, not a bug ;)
> > > >>
> > > >
> > > > +1
> > > > This is definitely not a bug, this is absolutely by design. One may argue with
> > > > the design, but it was done for a definite reason, so as to avoid paying the
> > > > penalty of having locks. It pushes more responsibility onto the app, but it
> > > > does allow the app to choose the best solution for managing the freeing of
> > > > memory for its situation. The alternative is to force all apps to pay the cost
> > > > of having locks, even if better options for freeing the memory are available.
> > > >
> > > > /Bruce
> > > >
> > >
> > > -1 (not to say 0xFFFFFFFF)
> > >
> > > This is definitely an API design bug !
> > > I would say that if you don't know how to free a resource that you
> > > allocate, it is very likely that you are wrong allocating it.
> > > And this is exactly what happens here with RX/TX callback data structures.
> > > This problem can easily be addressed by just changing the API as follows:
> > >
> > > Change
> > >      void *
> > >      rte_eth_add_rx_callback(uint8_t port_id, uint16_t queue_id,
> > >                              rte_rx_callback_fn fn, void *user_param)
> > >
> > > to
> > >      int
> > >      rte_eth_add_rx_callback(uint8_t port_id, uint16_t queue_id,
> > >                              struct rte_eth_rxtx_callback *cb)
> > >
> > > In addition of solving the problem, this approach makes the API
> > > consistent and let the application allocate "rte_eth_rxtx_callback" data
> > > structures through any appropriate mean.
> >
> > That might make API a bit cleaner, but I don't see how it fixes the generic problem:
> > there is still no way to know by the upper layer when it is safe to free/re-use
> > removed callback, but to make sure that all IO on that queue is stopped
> > (I.E. some external synchronisation around the queue).
> 
> Actually, it allows other, more creative solutions, like an app having a
> statically allocated set/pool of callback structures that it doesn't need to
> allocate or free at all.

I understand that, and as I said I am not against that change.
But it doesn't solve the problem in general.
Ok, if you have a static object, you wouldn't need to call free() for it,
but you still want to re-use it after remove_cb() has finished, right?
So there is no actual difference - the main question is:
at what point after remove_cb() it is safe to modify it's contents.
Konstantin

> 
> /Bruce

  reply	other threads:[~2016-06-15 14:28 UTC|newest]

Thread overview: 67+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <1465487895-5870-1-git-send-email-reshma.pattan@intel.com>
2016-06-10 16:18 ` [dpdk-dev] [PATCH v8 0/8] add packet capture framework Reshma Pattan
2016-06-10 16:18   ` [dpdk-dev] [PATCH v8 1/8] librte_ether: protect add/remove of rxtx callbacks with spinlocks Reshma Pattan
2016-06-10 16:18   ` [dpdk-dev] [PATCH v8 2/8] librte_ether: add new api rte_eth_add_first_rx_callback Reshma Pattan
2016-06-10 16:18   ` [dpdk-dev] [PATCH v8 3/8] librte_ether: add new fields to rte_eth_dev_info struct Reshma Pattan
2016-06-10 16:18   ` [dpdk-dev] [PATCH v8 4/8] librte_ether: make rte_eth_dev_get_port_by_name rte_eth_dev_get_name_by_port public Reshma Pattan
2016-06-10 16:18   ` [dpdk-dev] [PATCH v8 5/8] lib/librte_pdump: add new library for packet capturing support Reshma Pattan
2016-06-10 18:48     ` Aaron Conole
2016-06-10 22:14       ` Pattan, Reshma
2016-06-13 13:28         ` Aaron Conole
2016-06-10 16:18   ` [dpdk-dev] [PATCH v8 6/8] app/pdump: add pdump tool for packet capturing Reshma Pattan
2016-06-10 16:18   ` [dpdk-dev] [PATCH v8 7/8] app/test-pmd: add pdump initialization uninitialization Reshma Pattan
2016-06-10 16:18   ` [dpdk-dev] [PATCH v8 8/8] doc: update doc for packet capture framework Reshma Pattan
2016-06-10 23:23   ` [dpdk-dev] [PATCH v8 0/8] add " Neil Horman
2016-06-13  8:47     ` Pattan, Reshma
2016-06-14  9:38   ` [dpdk-dev] [PATCH v9 " Reshma Pattan
2016-06-14  9:38     ` [dpdk-dev] [PATCH v9 1/8] ethdev: use locks to protect Rx/Tx callback lists Reshma Pattan
2016-06-14 19:59       ` Thomas Monjalon
2016-06-15  5:30         ` Pattan, Reshma
2016-06-15  8:19           ` Thomas Monjalon
2016-06-15  8:37             ` Ananyev, Konstantin
2016-06-15  8:48               ` Thomas Monjalon
2016-06-15  9:54                 ` Ananyev, Konstantin
2016-06-15 11:17                   ` Thomas Monjalon
2016-06-15 13:49                   ` Thomas Monjalon
2016-06-15 12:15                 ` Ivan Boule
2016-06-15 12:40                   ` Ananyev, Konstantin
2016-06-15 13:29                     ` Bruce Richardson
2016-06-15 14:07                       ` Ivan Boule
2016-06-15 14:19                         ` Bruce Richardson
2016-06-15 14:20                         ` Ananyev, Konstantin
2016-06-15 14:22                           ` Bruce Richardson
2016-06-15 14:27                             ` Ananyev, Konstantin [this message]
2016-06-15 15:33                               ` Ivan Boule
2016-06-14  9:38     ` [dpdk-dev] [PATCH v9 2/8] ethdev: add new api to add Rx callback as head of the list Reshma Pattan
2016-06-14 20:01       ` Thomas Monjalon
2016-06-14 21:43         ` Pattan, Reshma
2016-06-14  9:38     ` [dpdk-dev] [PATCH v9 3/8] ethdev: add new fields to ethdev info struct Reshma Pattan
2016-06-14 20:10       ` Thomas Monjalon
2016-06-14 21:57         ` Pattan, Reshma
2016-06-14  9:38     ` [dpdk-dev] [PATCH v9 4/8] ethdev: make get port by name and get name by port public Reshma Pattan
2016-06-14 20:23       ` Thomas Monjalon
2016-06-14 21:55         ` Pattan, Reshma
2016-06-14  9:38     ` [dpdk-dev] [PATCH v9 5/8] pdump: add new library for packet capturing support Reshma Pattan
2016-06-14 20:28       ` Thomas Monjalon
2016-06-14 21:59         ` Pattan, Reshma
2016-06-15  9:05         ` Mcnamara, John
2016-06-15  9:32           ` Thomas Monjalon
2016-06-15  9:43             ` Bruce Richardson
2016-06-15 15:44             ` Mcnamara, John
2016-06-14  9:38     ` [dpdk-dev] [PATCH v9 6/8] app/pdump: add pdump tool for packet capturing Reshma Pattan
2016-06-14 19:56       ` Thomas Monjalon
2016-06-14  9:38     ` [dpdk-dev] [PATCH v9 7/8] app/testpmd: add pdump initialization uninitialization Reshma Pattan
2016-06-14  9:38     ` [dpdk-dev] [PATCH v9 8/8] doc: update doc for packet capture framework Reshma Pattan
2016-06-14 20:41       ` Thomas Monjalon
2016-06-15  5:44         ` Pattan, Reshma
2016-06-15  8:24           ` Thomas Monjalon
2016-06-15 14:06     ` [dpdk-dev] [PATCH v10 0/7] add " Reshma Pattan
2016-06-15 14:06       ` [dpdk-dev] [PATCH v10 1/7] ethdev: use locks to protect Rx/Tx callback lists Reshma Pattan
2016-06-15 14:06       ` [dpdk-dev] [PATCH v10 2/7] ethdev: add new api to add Rx callback as head of the list Reshma Pattan
2016-06-15 14:06       ` [dpdk-dev] [PATCH v10 3/7] ethdev: add new fields to ethdev info struct Reshma Pattan
2016-06-16 19:14         ` Thomas Monjalon
2016-06-15 14:06       ` [dpdk-dev] [PATCH v10 4/7] ethdev: make get port by name and get name by port public Reshma Pattan
2016-06-16 20:27         ` Thomas Monjalon
2016-06-15 14:06       ` [dpdk-dev] [PATCH v10 5/7] pdump: add new library for packet capturing support Reshma Pattan
2016-06-15 14:06       ` [dpdk-dev] [PATCH v10 6/7] app/pdump: add pdump tool for packet capturing Reshma Pattan
2016-06-15 14:06       ` [dpdk-dev] [PATCH v10 7/7] app/testpmd: add pdump initialization uninitialization Reshma Pattan
2016-06-16 21:55       ` [dpdk-dev] [PATCH v10 0/7] add packet capture framework Thomas Monjalon

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=2601191342CEEE43887BDE71AB97725836B7194A@irsmsx105.ger.corp.intel.com \
    --to=konstantin.ananyev@intel.com \
    --cc=bruce.richardson@intel.com \
    --cc=dev@dpdk.org \
    --cc=ivan.boule@6wind.com \
    --cc=reshma.pattan@intel.com \
    --cc=thomas.monjalon@6wind.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).