From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from dpdk.org (dpdk.org [92.243.14.124])
	by inbox.dpdk.org (Postfix) with ESMTP id 9DA0AA0352;
	Thu, 16 Jan 2020 13:18:25 +0100 (CET)
Received: from [92.243.14.124] (localhost [127.0.0.1])
	by dpdk.org (Postfix) with ESMTP id DE14E1C2ED;
	Thu, 16 Jan 2020 13:18:24 +0100 (CET)
Received: from mga07.intel.com (mga07.intel.com [134.134.136.100])
 by dpdk.org (Postfix) with ESMTP id 462D61C2E8;
 Thu, 16 Jan 2020 13:18:23 +0100 (CET)
X-Amp-Result: SKIPPED(no attachment in message)
X-Amp-File-Uploaded: False
Received: from fmsmga002.fm.intel.com ([10.253.24.26])
 by orsmga105.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384;
 16 Jan 2020 04:18:21 -0800
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="5.70,326,1574150400"; d="scan'208";a="257334280"
Received: from fyigit-mobl.ger.corp.intel.com (HELO [10.237.221.35])
 ([10.237.221.35])
 by fmsmga002.fm.intel.com with ESMTP; 16 Jan 2020 04:18:18 -0800
To: =?UTF-8?B?5pa557uf5rWpNTA0NTA=?= <fangtonghao@sangfor.com.cn>
Cc: Andrew Rybchenko <arybchenko@solarflare.com>,
 Thomas Monjalon <thomas@monjalon.net>, dev@dpdk.org, stable@dpdk.org,
 jia.guo@intel.com, cunming.liang@intel.com, qi.z.zhang@intel.com,
 jungle845943968@outlook.com, Jerin Jacob Kollanukkaran <jerinj@marvell.com>
References: <AM6AEwBQCIJk9JP063jdHKr*.3.1579174548054.Hmail.fangtonghao@sangfor.com.cn>
From: Ferruh Yigit <ferruh.yigit@intel.com>
Autocrypt: addr=ferruh.yigit@intel.com; prefer-encrypt=mutual; keydata=
 mQINBFXZCFABEADCujshBOAaqPZpwShdkzkyGpJ15lmxiSr3jVMqOtQS/sB3FYLT0/d3+bvy
 qbL9YnlbPyRvZfnP3pXiKwkRoR1RJwEo2BOf6hxdzTmLRtGtwWzI9MwrUPj6n/ldiD58VAGQ
 +iR1I/z9UBUN/ZMksElA2D7Jgg7vZ78iKwNnd+vLBD6I61kVrZ45Vjo3r+pPOByUBXOUlxp9
 GWEKKIrJ4eogqkVNSixN16VYK7xR+5OUkBYUO+sE6etSxCr7BahMPKxH+XPlZZjKrxciaWQb
 +dElz3Ab4Opl+ZT/bK2huX+W+NJBEBVzjTkhjSTjcyRdxvS1gwWRuXqAml/sh+KQjPV1PPHF
 YK5LcqLkle+OKTCa82OvUb7cr+ALxATIZXQkgmn+zFT8UzSS3aiBBohg3BtbTIWy51jNlYdy
 ezUZ4UxKSsFuUTPt+JjHQBvF7WKbmNGS3fCid5Iag4tWOfZoqiCNzxApkVugltxoc6rG2TyX
 CmI2rP0mQ0GOsGXA3+3c1MCdQFzdIn/5tLBZyKy4F54UFo35eOX8/g7OaE+xrgY/4bZjpxC1
 1pd66AAtKb3aNXpHvIfkVV6NYloo52H+FUE5ZDPNCGD0/btFGPWmWRmkPybzColTy7fmPaGz
 cBcEEqHK4T0aY4UJmE7Ylvg255Kz7s6wGZe6IR3N0cKNv++O7QARAQABtCVGZXJydWggWWln
 aXQgPGZlcnJ1aC55aWdpdEBpbnRlbC5jb20+iQJUBBMBCgA+AhsDAh4BAheABQsJCAcDBRUK
 CQgLBRYCAwEAFiEE0jZTh0IuwoTjmYHH+TPrQ98TYR8FAl1meboFCQlupOoACgkQ+TPrQ98T
 YR9ACBAAv2tomhyxY0Tp9Up7mNGLfEdBu/7joB/vIdqMRv63ojkwr9orQq5V16V/25+JEAD0
 60cKodBDM6HdUvqLHatS8fooWRueSXHKYwJ3vxyB2tWDyZrLzLI1jxEvunGodoIzUOtum0Ce
 gPynnfQCelXBja0BwLXJMplM6TY1wXX22ap0ZViC0m714U5U4LQpzjabtFtjT8qOUR6L7hfy
 YQ72PBuktGb00UR/N5UrR6GqB0x4W41aZBHXfUQnvWIMmmCrRUJX36hOTYBzh+x86ULgg7H2
 1499tA4o6rvE13FiGccplBNWCAIroAe/G11rdoN5NBgYVXu++38gTa/MBmIt6zRi6ch15oLA
 Ln2vHOdqhrgDuxjhMpG2bpNE36DG/V9WWyWdIRlz3NYPCDM/S3anbHlhjStXHOz1uHOnerXM
 1jEjcsvmj1vSyYoQMyRcRJmBZLrekvgZeh7nJzbPHxtth8M7AoqiZ/o/BpYU+0xZ+J5/szWZ
 aYxxmIRu5ejFf+Wn9s5eXNHmyqxBidpCWvcbKYDBnkw2+Y9E5YTpL0mS0dCCOlrO7gca27ux
 ybtbj84aaW1g0CfIlUnOtHgMCmz6zPXThb+A8H8j3O6qmPoVqT3qnq3Uhy6GOoH8Fdu2Vchh
 TWiF5yo+pvUagQP6LpslffufSnu+RKAagkj7/RSuZV25Ag0EV9ZMvgEQAKc0Db17xNqtSwEv
 mfp4tkddwW9XA0tWWKtY4KUdd/jijYqc3fDD54ESYpV8QWj0xK4YM0dLxnDU2IYxjEshSB1T
 qAatVWz9WtBYvzalsyTqMKP3w34FciuL7orXP4AibPtrHuIXWQOBECcVZTTOdZYGAzaYzxiA
 ONzF9eTiwIqe9/oaOjTwTLnOarHt16QApTYQSnxDUQljeNvKYt1lZE/gAUUxNLWsYyTT+22/
 vU0GDUahsJxs1+f1yEr+OGrFiEAmqrzpF0lCS3f/3HVTU6rS9cK3glVUeaTF4+1SK5ZNO35p
 iVQCwphmxa+dwTG/DvvHYCtgOZorTJ+OHfvCnSVjsM4kcXGjJPy3JZmUtyL9UxEbYlrffGPQ
 I3gLXIGD5AN5XdAXFCjjaID/KR1c9RHd7Oaw0Pdcq9UtMLgM1vdX8RlDuMGPrj5sQrRVbgYH
 fVU/TQCk1C9KhzOwg4Ap2T3tE1umY/DqrXQgsgH71PXFucVjOyHMYXXugLT8YQ0gcBPHy9mZ
 qw5mgOI5lCl6d4uCcUT0l/OEtPG/rA1lxz8ctdFBVOQOxCvwRG2QCgcJ/UTn5vlivul+cThi
 6ERPvjqjblLncQtRg8izj2qgmwQkvfj+h7Ex88bI8iWtu5+I3K3LmNz/UxHBSWEmUnkg4fJl
 Rr7oItHsZ0ia6wWQ8lQnABEBAAGJAjwEGAEKACYCGwwWIQTSNlOHQi7ChOOZgcf5M+tD3xNh
 HwUCXWZ5wAUJB3FgggAKCRD5M+tD3xNhH2O+D/9OEz62YuJQLuIuOfL67eFTIB5/1+0j8Tsu
 o2psca1PUQ61SZJZOMl6VwNxpdvEaolVdrpnSxUF31kPEvR0Igy8HysQ11pj8AcgH0a9FrvU
 /8k2Roccd2ZIdpNLkirGFZR7LtRw41Kt1Jg+lafI0efkiHKMT/6D/P1EUp1RxOBNtWGV2hrd
 0Yg9ds+VMphHHU69fDH02SwgpvXwG8Qm14Zi5WQ66R4CtTkHuYtA63sS17vMl8fDuTCtvfPF
 HzvdJLIhDYN3Mm1oMjKLlq4PUdYh68Fiwm+boJoBUFGuregJFlO3hM7uHBDhSEnXQr5mqpPM
 6R/7Q5BjAxrwVBisH0yQGjsWlnysRWNfExAE2sRePSl0or9q19ddkRYltl6X4FDUXy2DTXa9
 a+Fw4e1EvmcF3PjmTYs9IE3Vc64CRQXkhujcN4ZZh5lvOpU8WgyDxFq7bavFnSS6kx7Tk29/
 wNJBp+cf9qsQxLbqhW5kfORuZGecus0TLcmpZEFKKjTJBK9gELRBB/zoN3j41hlEl7uTUXTI
 JQFLhpsFlEdKLujyvT/aCwP3XWT+B2uZDKrMAElF6ltpTxI53JYi22WO7NH7MR16Fhi4R6vh
 FHNBOkiAhUpoXRZXaCR6+X4qwA8CwHGqHRBfYFSU/Ulq1ZLR+S3hNj2mbnSx0lBs1eEqe2vh cA==
Message-ID: <61a10224-b316-2dd1-0bc1-9db64643bbff@intel.com>
Date: Thu, 16 Jan 2020 12:18:18 +0000
MIME-Version: 1.0
In-Reply-To: <AM6AEwBQCIJk9JP063jdHKr*.3.1579174548054.Hmail.fangtonghao@sangfor.com.cn>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 8bit
Subject: Re: [dpdk-dev] [PATCH v2] Fixes: ethdev: secondary process change
 shared memory
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Sender: "dev" <dev-bounces@dpdk.org>

On 1/16/2020 11:35 AM, 方统浩50450 wrote:
> 
> 
>>@Fang, only can you please make a new version to update the
>>'rte_eth_copy_pci_info' function comment to document shared data is not updated
>>for the secondary process?
> 
>>So this suggest going on with Fang's patch. I only requested an additional note
>>in function comment related to this secondary check.
> 
> @Ferruh Yigit
> Should I update a new version patch of "rte_eth_copy_pci_info" function and explain
> wthether the regular functioning of secondary process is affected or not?
> I cant figure out what you need me to do.

Hi Fang,

Yes can you please send a new version of your patch.
In new version, additionally update the 'rte_eth_copy_pci_info()' function
comment to document that function updates 'eth_dev->data' only for primary process.

Thanks,
ferruh

> 
> 
> 发件人:Ferruh Yigit <ferruh.yigit@intel.com>
> 发送日期:2020-01-16 17:04:09
> 收件人:Andrew Rybchenko <arybchenko@solarflare.com>,Thomas Monjalon <thomas@monjalon.net>
> 抄送人:"方统浩50450" <fangtonghao@sangfor.com.cn>,dev@dpdk.org,stable@dpdk.org,jia.guo@intel.com,cunming.liang@intel.com,qi.z.zhang@intel.com,jungle845943968@outlook.com,Jerin Jacob Kollanukkaran <jerinj@marvell.com>
> 主题:Re: [dpdk-dev] [PATCH v2] Fixes: ethdev: secondary process change shared memory>On 1/16/2020 7:43 AM, Andrew Rybchenko wrote:
>>> On 1/15/20 11:43 PM, Thomas Monjalon wrote:
>>>> 15/01/2020 19:35, Ferruh Yigit:
>>>>> On 1/15/2020 6:49 AM, 方统浩50450 wrote:
>>>>>> Hi Ferruh, thanks for your message.
>>>>>>
>>>>>>
>>>>>> We developed a ethtool-dpdk which is secondary process based dpdk 17.08 version. Our device
>>>>>> support hotplug detach, but hotplug deatch is failed when we use ethtool-dpdk.We found the
>>>>>> secondary process will change the shared memory when initializing.Secondary process calls
>>>>>> "rte_eth_dev_pci_allocate" function and enters "rte_eth_copy_pci_info" function.
>>>>>> (rte_eth_dev_pci_generic_probe -> rte_eth_dev_pci_allocate -> rte_eth_copy_pci_info)
>>>>>> Then it sets the value of struct "rte_eth_dev_data.dev_flags" to zero.In our platform, this value
>>>>>> is equal to 0x0003.(RTE_ETH_DEV_DETACHABLE | RTE_ETH_DEV_INTR_LSC),but after reset
>>>>>> the "dev_flags", the value changed to 0x0002.(RTE_ETH_DEV_DETACHABLE).So, our device hotplug
>>>>>> detach is failed.I found the similar problem in other dpdk version, include dpdk 19.11.Even though
>>>>>> the deivce hotplug detach is discarded,but i think the shared memory changed is unexpected by primary
>>>>>> process.
>>> 
>>> Hold on, just for my understanding. As far as I can see
>>> RTE_ETH_DEV_DETACHABLE was removed in 17.11. Does it
>>> change something in above description?
>>
>>Overall secondary overwrites primary values, I think we should fix it
>>independent from the flags involved.
>>
>>> 
>>>>> I agree this is the problem.
>>>>> In the driver code, 'rte_eth_copy_pci_info' is called only by primary process,
>>>>>
>>>>> but the generic code is faulty.
>>>>>
>>>>> And in 19.11 additionally 'eth_dev_pci_specific_init' also seems has same problem.
>>> 
>>> Yes, as I understand RTE_ETH_DEV_CLOSE_REMOVE,
>>> RTE_ETH_DEV_BONDED_SLAVE, RTE_ETH_DEV_REPRESENTOR and
>>> RTE_ETH_DEV_NOLIVE_MAC_ADDR may be lost because of
>>> reinit (if not restored in other branches). Bad anyway.
>>> 
>>>>>> Our driver is ixgbe, i think this problem has a little relationship with driver, Secondary process
>>>>>> enters "rte_eth_copy_pci_info" by "rte_eth_dev_pci_allocate".And I agree your opinion, the helper
>>>>>> function should simple on what it does.I have two ways to fix this problem, one is add an if-statement
>>>>>>
>>>>>> in "rte_eth_dev_pci_allocate" function to forbid secondary process enters "rte_eth_copy_pci_info" function,
>>>>>> another way is add an if-statement in "rte_eth_copy_pci_info" function to forbid secondary process change
>>>>>> shared memory.And First way need to ensure the "rte_eth_copy_pci_info" function won't be called anywhere else.
>>>>>> I think the second way is simple and lower risk.
>>>>>
>>>>> Yes these are the two options.
>>>>>
>>>>> I agree adding check in the 'rte_eth_copy_pci_info' covers all cases and safer.
>>>>> BUT my concern was adding decision making to simple/leaf function and make it
>>>>> harder to debug/use, instead of giving what primary/secondary process should
>>>>> call decision in higher level.
>>>>>
>>>>> But I just recognized that some PMDs are calling 'rte_eth_copy_pci_info' on
>>>>> secondary process, like mlx4 or szedata2, and most probably this is not their
>>>>> intention.
>>>>> And 'eth_dev->intr_handle' set in 'rte_eth_copy_pci_info', not calling this
>>>>> function may have side affect of 'eth_dev->intr_handle' not set in secondary.
>>>>>
>>>>> With above considerations I am OK to your proposal to cover all cases, Thomas,
>>>>> Andrew, any concern?
>>> 
>>> I would put if condition in rte_eth_copy_pci_info().
>>> It is the function which writes shared space from
>>> secondary process when it should not be done and it
>>> should be fixed there.
>>
>>OK
>>
>>> 
>>>> Do you mean drivers need to be fixed?
>>> 
>>> I'm not sure that I fully understand it. Since copy function
>>> cares about intr_handle copying I'm afraid that it is not
>>> 100% correct to skip it in secondary process completely as
>>> many drivers do right now. Basically it makes eth_dev structure
>>> in secondary process inconsistent. However, it looks like
>>> most of these drivers simply obtain handle from pci_dev
>>> directly and it explains why they are not affected.
>>> There are exceptions which are potentially bugs, e.g.
>>> drivers/net/ice/ice_ethdev.c: ice_interrupt_handler at the end.
>>> 
>>> I think that it would be better if intr_handle is always
>>> correct in eth_dev (both primary and secondary cases) and
>>> drivers use it instead of the same from pci_dev.
>>> 
>>
>>OK
>>
>>So this suggest going on with Fang's patch. I only requested an additional note
>>in function comment related to this secondary check.
> 
>