DPDK patches and discussions
 help / color / mirror / Atom feed
From: Ferruh Yigit <ferruh.yigit@intel.com>
To: Thomas Monjalon <thomas@monjalon.net>,
	Matan Azrad <matan@mellanox.com>,
	Bernard Iremonger <bernard.iremonger@intel.com>
Cc: dev@dpdk.org, Gaetan Rivet <gaetan.rivet@6wind.com>,
	David Marchand <david.marchand@redhat.com>,
	Jeff Guo <jia.guo@intel.com>, Qi Zhang <qi.z.zhang@intel.com>
Subject: Re: [dpdk-dev] [dpdk-stable] [PATCH 2/2] app/testpmd: fix invalid port detaching
Date: Thu, 13 Feb 2020 14:00:21 +0000	[thread overview]
Message-ID: <7c3daec0-b1da-16f7-088e-0cb8d313a114@intel.com> (raw)
In-Reply-To: <3042500.kGzlxMrEDr@xps>

On 2/13/2020 1:36 PM, Thomas Monjalon wrote:
> More details below about the plan for 20.02.
> 
> 13/02/2020 13:37, Thomas Monjalon:
>> Hi,
>>
>> This discussion becomes confusing so I do a summary below.
>> I think we can do several fixes in 20.02.

Thanks for checking this.

>>
>> 12/02/2020 14:49, Ferruh Yigit:
>>> On 2/3/2020 5:10 PM, Matan Azrad wrote:
>>
>> [stripping long discussion in favor of a summary below]
>>
>>>> Even if the PMD clear the device pointer, the testpmd still may release wrong rte_device.
>>>
>>> Yes it may, although that is less likely to occur, it requires a new device hot
>>> added between close() and detach of the other device.
>>>
>>> Would you be agree to say there are two problems:
>>>
>>> 1) When testpmd close a port, a new attached port can re-use it over writing
>>> some fields, relying the data structures of the closed port is not safe.
>>>
>>> 2) PMD not cleaning ethdev->device pointer in the .remove() may cause issues in
>>> double detach of a port.
>>>
>>>
>>> For (1) I suggest fixing it in the attach path, don't re-use an eth_dev port id
>>> unless it is completely freed, may need to add new state for it. Does it make sense?
>>
>> Yes we could add a CLOSED state which is set on ethdev close.
>> When the rte_device is freed, the PMD could set attached ports as UNUSED.
>> But given some ethdev ports can be open and closed dynamically,
>> I am not sure it is a good solution to keep them in CLOSED state and ask
>> PMD to remember them.
>>
>> An alternative workaround could be to allocate port_id by incrementing
>> a saved biggest id. So the race condition would be very unlikely.
>> The drawbacks are having big port_id numbers and changing the id
>> allocation algorithm (which is not documented anyway).

OK to keep increase port_id instead of re-using closed ones, that simplifies a lot.

>>
>> The proposals above for port_id allocation or states rework cannot be
>> done in 20.02. Let's discuss and work on it in a separated thread.

+1

>>
>>> For (2) PMDs want to get hotplug support needs to fix it.
>>
>> Yes PMDs should clear rte_eth_devices[port_id].device in .remove().
> 
> I am sending a patch adding
> 	memset(eth_dev, 0, sizeof(struct rte_eth_dev));
> in rte_eth_dev_release_port().
> But this patch cannot be merged after 20.02-rc1. It will wait for 20.05.

Not sure about this, close() calls the 'rte_eth_dev_release_port()', memset the
struct in close() will wipe the device pointers and prevents freeing them in hot
remove, silently.

> 
>> We must also protect from user calling detach on a closed port
>> by adding a check in cmd_operate_detach_port_parsed(),
>> before calling detach_port_device().
> 
> I am sending a patch adding RTE_ETH_VALID_PORTID_OR_RET()
> in cmd_operate_detach_port_parsed().
> It should fix the issue observed by Matan with double detach.
> It will be a double protection if keeping the check
> port_id_is_invalid() in detach_port_device().

OK

> 
>> The hotplug rmv_port_callback() must be able to call detach after close.
>> There are three possible fixes:
>> 	- revert the port_id_is_invalid() check in detach_port_device()
>> 	- call rte_dev_remove(rte_device) directly
>> 	- call a new function with rte_device (detach_port_device() can use it)
> 
> I am sending a patch implementing the third alternative
> as it is both keeping the detach behaviour and fixing the race condition
> (i.e. protect from new port re-using the port_id between close and detach).

Should work, only concern if any possible side affect occurs, can be discussed
on patch.

> 
>> About the function detach_port_device() itself, yes this function is
>> strange to say the least. It was a convenience for detaching a rte_device
>> from a port_id.
>> The cleanup of siblings with RTE_ETH_FOREACH_DEV_OF(sibling, dev),
>> should probably be removed. I've added it as a temporary solution
>> before all PMDs are properly fixed:
>> 	rte_eth_devices[sibling].device = NULL;
> 
> I propose sending such patch in 20.05 in order to merge the memset above
> first, and have time to get agreement from all PMD maintainers.

OK

> 
>> For info, there is a function detach_device() used by the command
>> 	"device detach <identifier>"
> 
> 
> 


  reply	other threads:[~2020-02-13 14:00 UTC|newest]

Thread overview: 26+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-11-12  8:47 [dpdk-dev] [PATCH 1/2] bus/pci: fix driver detach clear Matan Azrad
2019-11-12  8:47 ` [dpdk-dev] [PATCH 2/2] app/testpmd: fix invalid port detaching Matan Azrad
2019-11-12 11:20   ` Iremonger, Bernard
2019-11-20 22:52     ` [dpdk-dev] [dpdk-stable] " David Marchand
2020-01-23 13:19   ` [dpdk-dev] " Yigit, Ferruh
2020-01-23 14:05     ` Matan Azrad
2020-01-23 14:48       ` [dpdk-dev] [dpdk-stable] " Ferruh Yigit
2020-01-23 15:29         ` Matan Azrad
2020-01-23 18:14           ` Ferruh Yigit
2020-01-23 19:25             ` Matan Azrad
2020-01-24 16:28               ` Ferruh Yigit
2020-01-25 18:56                 ` Matan Azrad
2020-02-03 15:58                   ` Ferruh Yigit
2020-02-03 17:10                     ` Matan Azrad
2020-02-12 13:49                       ` Ferruh Yigit
2020-02-13 12:37                         ` Thomas Monjalon
2020-02-13 13:36                           ` Thomas Monjalon
2020-02-13 14:00                             ` Ferruh Yigit [this message]
2019-11-19 22:40 ` [dpdk-dev] [dpdk-stable] [PATCH 1/2] bus/pci: fix driver detach clear Thomas Monjalon
2019-11-20  9:02   ` Matan Azrad
2019-11-20  9:47 ` [dpdk-dev] [PATCH v2] " Matan Azrad
2019-11-20 13:03   ` [dpdk-dev] [dpdk-stable] " David Marchand
2019-11-20 13:44     ` Matan Azrad
2019-11-20 13:51     ` Thomas Monjalon
2019-11-20 17:22       ` David Marchand
2019-11-20 22:52   ` David Marchand

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=7c3daec0-b1da-16f7-088e-0cb8d313a114@intel.com \
    --to=ferruh.yigit@intel.com \
    --cc=bernard.iremonger@intel.com \
    --cc=david.marchand@redhat.com \
    --cc=dev@dpdk.org \
    --cc=gaetan.rivet@6wind.com \
    --cc=jia.guo@intel.com \
    --cc=matan@mellanox.com \
    --cc=qi.z.zhang@intel.com \
    --cc=thomas@monjalon.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).