DPDK patches and discussions
 help / color / mirror / Atom feed
From: Ferruh Yigit <ferruh.yigit@intel.com>
To: Thomas Monjalon <thomas@monjalon.net>
Cc: 方统浩50450 <fangtonghao@sangfor.com.cn>,
	arybchenko@solarflare.com, dev@dpdk.org, stable@dpdk.org,
	jia.guo@intel.com, cunming.liang@intel.com, qi.z.zhang@intel.com,
	jungle845943968@outlook.com,
	"Jerin Jacob Kollanukkaran" <jerinj@marvell.com>
Subject: Re: [dpdk-dev] [PATCH v2] Fixes: ethdev: secondary process change shared memory
Date: Thu, 16 Jan 2020 09:00:15 +0000	[thread overview]
Message-ID: <4ed2e2e7-d496-2295-ad5e-429fedb4476b@intel.com> (raw)
In-Reply-To: <3055448.clyjiGRsXx@xps>

On 1/15/2020 8:43 PM, Thomas Monjalon wrote:
> 15/01/2020 19:35, Ferruh Yigit:
>> On 1/15/2020 6:49 AM, 方统浩50450 wrote:
>>> Hi Ferruh, thanks for your message.
>>>
>>>
>>> We developed a ethtool-dpdk which is secondary process based dpdk 17.08 version. Our device
>>> support hotplug detach, but hotplug deatch is failed when we use ethtool-dpdk.We found the
>>> secondary process will change the shared memory when initializing.Secondary process calls
>>> "rte_eth_dev_pci_allocate" function and enters "rte_eth_copy_pci_info" function.
>>> (rte_eth_dev_pci_generic_probe -> rte_eth_dev_pci_allocate -> rte_eth_copy_pci_info)
>>> Then it sets the value of struct "rte_eth_dev_data.dev_flags" to zero.In our platform, this value
>>> is equal to 0x0003.(RTE_ETH_DEV_DETACHABLE | RTE_ETH_DEV_INTR_LSC),but after reset
>>> the "dev_flags", the value changed to 0x0002.(RTE_ETH_DEV_DETACHABLE).So, our device hotplug
>>> detach is failed.I found the similar problem in other dpdk version, include dpdk 19.11.Even though
>>> the deivce hotplug detach is discarded,but i think the shared memory changed is unexpected by primary
>>> process.
>>
>> I agree this is the problem.
>> In the driver code, 'rte_eth_copy_pci_info' is called only by primary process,
>> but the generic code is faulty.
>>
>> And in 19.11 additionally 'eth_dev_pci_specific_init' also seems has same problem.
>>
>>> Our driver is ixgbe, i think this problem has a little relationship with driver, Secondary process
>>> enters "rte_eth_copy_pci_info" by "rte_eth_dev_pci_allocate".And I agree your opinion, the helper
>>> function should simple on what it does.I have two ways to fix this problem, one is add an if-statement
>>>
>>> in "rte_eth_dev_pci_allocate" function to forbid secondary process enters "rte_eth_copy_pci_info" function,
>>> another way is add an if-statement in "rte_eth_copy_pci_info" function to forbid secondary process change
>>> shared memory.And First way need to ensure the "rte_eth_copy_pci_info" function won't be called anywhere else.
>>> I think the second way is simple and lower risk.
>>
>> Yes these are the two options.
>>
>> I agree adding check in the 'rte_eth_copy_pci_info' covers all cases and safer.
>> BUT my concern was adding decision making to simple/leaf function and make it
>> harder to debug/use, instead of giving what primary/secondary process should
>> call decision in higher level.
>>
>> But I just recognized that some PMDs are calling 'rte_eth_copy_pci_info' on
>> secondary process, like mlx4 or szedata2, and most probably this is not their
>> intention.
>> And 'eth_dev->intr_handle' set in 'rte_eth_copy_pci_info', not calling this
>> function may have side affect of 'eth_dev->intr_handle' not set in secondary.
>>
>> With above considerations I am OK to your proposal to cover all cases, Thomas,
>> Andrew, any concern?
> 
> Do you mean drivers need to be fixed?
> 

either it or 'rte_eth_copy_pci_info'.

Right now 'rte_eth_copy_pci_info' updates the shared memory, calling it in
secondary overwrites the memory set by primary.

Options Fang mentioned:
1) Don't call 'rte_eth_copy_pci_info' from secondary process path, this requires
fixing 'rte_eth_dev_pci_allocate', 'eth_dev_pci_specific_init' and possibly some
drivers.

2) Add a check inside the 'rte_eth_copy_pci_info' to prevent updating shared
memory if it is secondary process.

Fang's patch does (2), and I am OK with it as well after latest findings.

  parent reply	other threads:[~2020-01-16  9:00 UTC|newest]

Thread overview: 17+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-01-09 12:27 [dpdk-dev] [PATCH] ethdev: fix secondary process change share memory Fang TongHao
2020-01-10  7:30 ` Jeff Guo
2020-01-10  7:53   ` 方统浩50450
2020-01-13  5:03 ` [dpdk-dev] [PATCH v2] Fixes: ethdev: secondary process change shared memory Fang TongHao
2020-01-14 14:45   ` Ferruh Yigit
2020-01-15  6:49     ` 方统浩50450
2020-01-15 18:35       ` Ferruh Yigit
2020-01-15 20:43         ` Thomas Monjalon
2020-01-16  7:43           ` Andrew Rybchenko
2020-01-16  9:04             ` Ferruh Yigit
2020-01-16 11:35               ` 方统浩50450
2020-01-16 12:18                 ` Ferruh Yigit
2020-01-17  2:11                   ` 方统浩50450
2020-01-16  9:00           ` Ferruh Yigit [this message]
2020-01-17  2:08   ` [dpdk-dev] [PATCH v3] " Fang TongHao
2020-01-17  8:33     ` Andrew Rybchenko
2020-01-17 17:58       ` Ferruh Yigit

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=4ed2e2e7-d496-2295-ad5e-429fedb4476b@intel.com \
    --to=ferruh.yigit@intel.com \
    --cc=arybchenko@solarflare.com \
    --cc=cunming.liang@intel.com \
    --cc=dev@dpdk.org \
    --cc=fangtonghao@sangfor.com.cn \
    --cc=jerinj@marvell.com \
    --cc=jia.guo@intel.com \
    --cc=jungle845943968@outlook.com \
    --cc=qi.z.zhang@intel.com \
    --cc=stable@dpdk.org \
    --cc=thomas@monjalon.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).