From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2DED7A0352 for ; Thu, 16 Jan 2020 08:44:25 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 06C471C1B6; Thu, 16 Jan 2020 08:44:25 +0100 (CET) Received: from dispatch1-us1.ppe-hosted.com (dispatch1-us1.ppe-hosted.com [148.163.129.52]) by dpdk.org (Postfix) with ESMTP id E8F9C1BE3D; Thu, 16 Jan 2020 08:44:21 +0100 (CET) X-Virus-Scanned: Proofpoint Essentials engine Received: from webmail.solarflare.com (uk.solarflare.com [193.34.186.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mx1-us1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTPS id EAA92B40060; Thu, 16 Jan 2020 07:44:19 +0000 (UTC) Received: from [192.168.38.17] (10.17.10.39) by ukex01.SolarFlarecom.com (10.17.10.4) with Microsoft SMTP Server (TLS) id 15.0.1395.4; Thu, 16 Jan 2020 07:44:11 +0000 To: Thomas Monjalon , Ferruh Yigit CC: =?UTF-8?B?5pa557uf5rWpNTA0NTA=?= , , , , , , , Jerin Jacob Kollanukkaran References: <32a5b3ff-afe2-75eb-f2cb-b9437a5a8d86@intel.com> <3055448.clyjiGRsXx@xps> From: Andrew Rybchenko Autocrypt: addr=arybchenko@solarflare.com; keydata= mQINBF2681gBEACbdTxu8eLL3UX2oAelsnK9GkeaJeUYSOHPJQpV7RL/iaIskqTwBRnhjXt7 j9UEwGA+omnOmqQMpeQTb/F9Ma2dYE+Hw4/t/1KVjxr3ehFaASvwR4fWJfO4e2l/Rk4rG6Yi 5r6CWU2y8su2654Fr8KFc+cMGOAgKoZTZHZsRy5lHpMlemeF+VZkv8L5sYJWPnsypgqlCG3h v6lbtfZs+QqYbFH6bqoZwBAl5irmxywGR7ZJr1GLUZZ1lfdazSY8r6Vz0/Ip/KVxGu2uxo81 QCsAj0ZsQtwji9Sds/prTiPrIjx8Fc/tfbnAuVuPcnPbczwCJACzQr4q26XATL3kVuZhSBWh 4XfO/EAUuEq5AemUG5DDTM87g7Lp4eT9gMZB6P+rJwWPNWTiV3L7Cn+fO+l9mTPnOqdzBgDe OaulKiNSft1o0DY4bGzOmM2ad2cZt0jfnbMPMTE9zsr6+RFa+M8Ct20o6U1MUE4vP6veErMK of4kZ8PdoMM+Sq1hxMPNtlcVBSP9xMmdSZPlfDYI5VWosOceEcz7XZdjBJKdwKuz70V7eac4 ITSxgNFCTbeJ03zL2MR5s0IvD9ghISAwZ6ieCjU5UATn5+63qpD0nVNLsAdb/UpfvQcKAmvj 0fKlxu/PMVkjBa7/4cfNogYOhWDKUO+1pMaFwvb6/XTo6uMpfQARAQABtCxBbmRyZXcgUnli Y2hlbmtvIDxhcnliY2hlbmtvQHNvbGFyZmxhcmUuY29tPokCVAQTAQoAPhYhBP6NPgcKRj/Y X0yXQahue0sAy4m+BQJduvNYAhsDBQkB4TOABQsJCAcDBRUKCQgLBRYCAwEAAh4BAheAAAoJ EKhue0sAy4m+t3gP/j1MNc63CEozZo1IZ2UpVPAVWTYbLdPjIRdFqhlwvZYIgGIgIBk3ezKL K0/oc4ZeIwL6wQ5+V24ahuXvvcxLlKxfbJ6lo2iQGC7GLGhsDG9Y2k6sW13/sTJB/XuR2yov k5FtIgJ+aHa1PDZnepnGGOt9ka9n/Jzrc9WKYapOIIyLRe9U26ikoVgyqsD37PVeq5tLWHHA NGTUKupe9G6DFWidxx0KzyMoWDTbW2AWYcEmV2eQsgRT094AZwLFN5ErfefYzsGdO8TAUU9X YTiQN2MvP1pBxY/r0/5UfwV4UKBcR0S3ZvzyvrPoYER2Kxdf/qurx0Mn7StiCQ/JlNZb/GWQ TQ7huduuZHNQKWm7ufbqvKSfbPYvfl3akj7Wl8/zXhYdLqb5mmK45HXrgYGEqPN53OnK2Ngx IgYKEWr05KNv09097jLT5ONgYvszflqlLIzC4dV245g7ucuf9fYmsvmM1p/gFnOJBJL18YE5 P1fuGYNfLP+qp4WMiDqXlzaJfB4JcinyU49BXUj3Utd6f6sNBsO8YWcLbKBV9WmA324S3+wj f4NPRp3A5E+6OmTVMLWire2ZvnYp3YvifUj1r8lhoZ2B2vKuWwiTlHOKYBEjnOQJQnqYZEF0 JQQ1xzVDBQKE01BPlA3vy6BGWe6I4psBVqMOB9lAev/H+xa4u6Z3uQINBF269JsBEAC2KB3W 8JES/fh74avN7LOSdK4QA7gFIUQ4egVL81KnxquLzzilABuOhmZf3Rq6rMHSM8xmUAWa7Dkt YtzXStjEBI/uF0mAR3mMz1RcL2Wp+WD/15HjVpA7hPjXSEsWY0K2ymPerK4yrLcfFTHdMonY JfuACCC9NtOZxrWHOJoUS+RT7AWk80q/6D2iwQ47/2dBTznVG+gSeHSes9l91TB09w6f9JX/ sT+Ud0NQfm7HJ7t2pmGI9O6Po/NLZsDogmnIpJp/WwYOZN9JK7u2FyX2UyRzR8jK42aJkRsh DXs16Cc2/eYGakjrdO3x9a+RoxN7EuFtYhGR1PzMXdUiB5i+FyddYXkYUyO43QE/3VPA5l1v TUOagzZq6aONsdNonGJkV3TIG3JmUNtM+D/+r6QKzmgoJ8w576JxEZI09I/ZFN+g7BnUmlMx 6Z3IUOXVX/SWfGFga0YajwajHz03IBhChEbYbbqndVhmshu2GFURxrfUPYWdDXEqkh+08a5U Didia9jm2Opv4oE1e1TXAePyYJl/Zyps4Cv00GObAxibvMBQCUZQ+IBnNldRBOwXXRQV2xpx P+9iO1VYA/QXn0KqRK+SH1JGRXbJYi42YFaW1gE0EU0fiR2Wb9pK+doNEjjOhlzUGuvOEAUS +4m0m3dlfEvpCV9GMr7ERRpZzh9QkQARAQABiQI8BBgBCgAmFiEE/o0+BwpGP9hfTJdBqG57 SwDLib4FAl269JsCGwwFCQlmAYAACgkQqG57SwDLib7x6g//e+eCtNnJz7qFGbjWRJYNLCe5 gQwkhdyEGk4omr3VmjGj3z9kNFy/muh4pmHUngSAnnpwZggx14N4hhKf9y8G4Dwvsqa6b1zB Jq/c4t/SBDtGW4M/E331N04PaQZpcrbTfp1KqHNknk2N7yOk4CcoLVuIZmA5tPguASV8aAfz ZwhWAwn6vUEw9552eXEAnGFGDTCbyryNwzB5jtVQOEEDjTxcCkpcXMB45Tb1QUslRTu/sBAe HhPCQSUcJHR+KOq+P6yKICGAr291PZd6Qc7C3UyE+A3pY/UfdEVWj0STBWx1qvYLaHLrI4O9 KXDgh7luLjZZafcueCaPYmNo4V2lmNb3+7S4TvqhoZS+wN+9ldRQ4gH3wmRZybN6Y/ZCqxol RaZpE3AqdWsGvIgAkD0FpmtZNii9s2pnrhw0K6S4t4tYgXGTossxNSJUltfFQZdXM1xkZhtv dBZuUEectbZWuviGvQXahOMuH2pM64mx2hpdZzPcI2beeJNHkAsGT2KcaMETgvtHUBFRlLVB YxsUYz3UZmi2JSua4tbcGd6iWVN90eb8CxszYtivfpz6o2nPSjNwg0NaVGSHXjAK0tdByZ9t SkwjC3tEPljVycRSDpbauogOiAkvjENfaPd/H26V5hY822kaclaKDAW6ZG9UKiMijcAgb9u5 CJoOyqE8aGS5Ag0EXbr1RwEQAMXZHbafqmZiu6Kudp+Filgdkj2/XJva5Elv3fLfpXvhVt0Y if5Rzds3RpffoLQZk9nPwK8TbZFqNXPu7HSgg9AY7UdCM94WRFTkUCGKzbgiqGdXZ7Vyc8cy teGW+BcdfQycDvjfy50T3fO4kJNVp2LDNdknPaZVe8HJ80Od63+9ksB6Ni+EijMkh6Uk3ulB CSLnT4iFV57KgU2IsxOQVLnm+0bcsWMcCnGfphkY0yKP+aJ6MfmZkEeaDa7kf24N14ktg50m vOGDitcxA/+XXQXOsOIDJx1VeidxYsQ2FfsKu1G8+G6ejuaLf4rV5MI/+B/tfLbbOdikM5PF pxZVgTir9q13qHumMxdme7w5c7hybW412yWAe9TsrlXktFmFjRSFzAAxQhQSQxArS6db4oBk yeYJ59mW52i4occkimPWSm/raSgdSM+0P6zdWUlxxj+r1qiLgCYvruzLNtp5Nts5tR/HRQjE /ohQYaWDSVJEsc/4eGmgwzHzmvHtXeKkasn01381A1Lv3xwtpnfwERMAhxBZ8EGKEkc5gNdk vIPhknnGgPXqKmE1aWu8LcHiY+RHAF8gYPCDMuwyzBYnbiosKcicuIUp0Fj8XIaPao6F+WTi In4UOrqrYhsaCUvhVjsTBbNphGih9xbFJ8E+lkTLL8P3umtTcMPnpsB4xqcDABEBAAGJBHIE GAEKACYWIQT+jT4HCkY/2F9Ml0GobntLAMuJvgUCXbr1RwIbAgUJCWYBgAJACRCobntLAMuJ vsF0IAQZAQoAHRYhBNTYjdjWgdaEN5MrAN+9UR5r/4d3BQJduvVHAAoJEN+9UR5r/4d3EiQP /3lyby6v49HTU94Q2Fn2Xat6uifR7kWE5SO/1pUwYzx6v+z5K2jqPgqUYmuNoejcGl0CTNhg LbsxzUmAuf1OTAdE+ZYvOAjjKQhY4haxHc4enby/ltnHfWJYWJZ9UN5SsIQLvITvYu6rqthO CYjpXJhwkj3ODmC9H1TrvjrBGc6i7CTnR8RCjMEwCs2LI2frHa4R6imViEr9ScMfUnzdABMQ B0T5MOg8NX92/FRjTldU2KovG0ML9mSveSvVHAoEBLy4UIs5nEDdNiO1opJgKb5CXvWQugub 7AR52phNdKVdEB0S4tigJT4NalyTaPiUhFEm+CzZpMQDJ5E+/OowaPRfN4HeJX+c8sB+vUAZ mkAaG75N+IEk5JKFK9Z+bBYgPgaBDFZYdWDB/TMH0ANt+KI5uYg0i12TB4M8pwKG1DEPUmWc F2YpvB3jnbwzsOpSFiJOOlSs6nOB0Sb5GRtPOO3h6XGj+6mzQd6tcL63c9TrrUkjq7LDkxCz SJ2hTYRC8WNX8Uw9skWo5728JNrXdazEYCenUWmYiKLNKLslXCFodUCRDh/sUiyqRwS7PHEA LYC/UIWLMomI0Yvju3KA5v3RQVXhL+Gx2CzSj3GDz9xxGhJB2LfRfjzPbTR/Z27UpjCkd8z0 Ro3Ypmi1FLQwnRgoOKDbetTAIhugEShaLTITzJAP/iRDJCQsrZah5tE8oIl81qKEmBJEGcdt HYikbpQe7ydcXhqTj7+IECa3O7azI5OhCxUH2jNyonJ/phUslHH2G1TTBZK8y4Hrx5RpuRNS esn3P9uKu9DHqBAL7DMsCPwb2p1VNnapD72DBmRhzS/e6zS2R4+r9yNv03Hv7VCxKkmtE63H qpS//qpjfrtsIcHAjnKDaDtL1LYCtHoweI+DOpKKULSAYp/JE6F8LNibPQ0/P3S5ZIJNC4QZ uESjFOalJwFIqGQdkQB7ltRNJENLrHc+2jKGOuyFHm/Sbvp5EMGdaeQ0+u8CY0P+y6oXenwx 7WrJz/GvbNoFhJoJ6RzxCMQrFgxrssVZ7w5HcUj94lbnJ6osdYE/WpSd50B6jet6LKh5revg u9XI9CoqsPQ1V4wKYYdllPuogCye7KNYNKuiiuSNpaF4gHq1ZWGArwZtWHjgc2v3LegOpRQF SwOskMKmWsUyHIRMG1p8RpkBQTqY2rGSeUqPSvaqjT0nq+SUEM6qxEXD/2Wqri/X6bamuPDb S0PkBvFD2+0zr5Bc2YkMGPBYPNGZiTp3UjmZlLfn3TiBKIC92jherY563CULjSsiBEJCOSvv 4VPLn5aAcfbCXJnE3IGCp/hPl50iQqu7BPOYBbWXeb9ptDjGCAThNxSz0WAXkmcjAFE8gdE6 Znk9 Message-ID: <58542797-ae4c-9882-b777-60ee704de9dc@solarflare.com> Date: Thu, 16 Jan 2020 10:43:42 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.3.0 MIME-Version: 1.0 In-Reply-To: <3055448.clyjiGRsXx@xps> Content-Type: text/plain; charset="utf-8" Content-Language: en-US Content-Transfer-Encoding: 8bit X-Originating-IP: [10.17.10.39] X-ClientProxiedBy: ocex03.SolarFlarecom.com (10.20.40.36) To ukex01.SolarFlarecom.com (10.17.10.4) X-TM-AS-Product-Ver: SMEX-12.5.0.1300-8.5.1010-25170.003 X-TM-AS-Result: No-17.161200-8.000000-10 X-TMASE-MatchedRID: gTucSmrmRMN0nsJmoztmWPZvT2zYoYOwptBqZcCRzB2YhWE49s9+q62S wIjG5UkR3wAUzW7g9YgEzNy+qeySxE0v4a437hUkimHWEC28pk0GchEhVwJY31B3AZ+9IiUHy46 qzTSht7dO2BTZgEnrOdRnTglbZHIkzD8138PYfo4mZusHWPhfClxo0H+7nJCrgHhfJtGVOYRT/4 0CMa1mSIYwp4Wk0XROlMaIBm0udyhwoTxwetcEI8qquP3qhQpq5O1s01MUhB1UjspoiX02F5WMX h7PvokjSwnkaPHxLb+Thp4fKG9XwmK3QrkHqcKMVV4ZZmbE3YzcVi8qZmJWczTdFbpwEeew/mPV nk7qcVybc2U22TB+JX1ZHZRqi6C2e8AS3xX9iQW3RxL+7EfzsEupr1fvkyppZ9uMaIUDtOUtVxn mZPjhceu7mxpiZITsd5oahAL71fTpHzzvfxlA9YS/TV9k6ppA8gryAriuOnLLkl8e9W70ixo5TM R6zKbJMxT/btq1Bg/yj0s6jNx5mZeNLvT8ibyELyz9QvAyHjprLj3DxYBINyNGK7UC7ElM6q/JR //r0aLSz/z3K6uFXg3ujPkqhr95AkU1UqnHV9Xp9R8QuoVQf+eGwhAs1OaSL+H1U69PoCB8Sl05 cb4iyq4YUaH/Q8F2mQBA0QJrt3TyzTQFi0yHB7sHVDDM5xAPfS0Ip2eEHnzUHQeTVDUrIke0Gx/ n9RcltVC7UY1n6yI7AFczfjr/7FinaPgxnjvk9akwlEpyxg6s1Y8vfZvUazd+TJ8f9c/ekxtgFE ZKALw= X-TM-AS-User-Approved-Sender: Yes X-TM-AS-User-Blocked-Sender: No X-TMASE-Result: 10--17.161200-8.000000 X-TMASE-Version: SMEX-12.5.0.1300-8.5.1010-25170.003 X-MDID: 1579160661-ctFjy05yMbEu Subject: Re: [dpdk-stable] [dpdk-dev] [PATCH v2] Fixes: ethdev: secondary process change shared memory X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: stable-bounces@dpdk.org Sender: "stable" On 1/15/20 11:43 PM, Thomas Monjalon wrote: > 15/01/2020 19:35, Ferruh Yigit: >> On 1/15/2020 6:49 AM, 方统浩50450 wrote: >>> Hi Ferruh, thanks for your message. >>> >>> >>> We developed a ethtool-dpdk which is secondary process based dpdk 17.08 version. Our device >>> support hotplug detach, but hotplug deatch is failed when we use ethtool-dpdk.We found the >>> secondary process will change the shared memory when initializing.Secondary process calls >>> "rte_eth_dev_pci_allocate" function and enters "rte_eth_copy_pci_info" function. >>> (rte_eth_dev_pci_generic_probe -> rte_eth_dev_pci_allocate -> rte_eth_copy_pci_info) >>> Then it sets the value of struct "rte_eth_dev_data.dev_flags" to zero.In our platform, this value >>> is equal to 0x0003.(RTE_ETH_DEV_DETACHABLE | RTE_ETH_DEV_INTR_LSC),but after reset >>> the "dev_flags", the value changed to 0x0002.(RTE_ETH_DEV_DETACHABLE).So, our device hotplug >>> detach is failed.I found the similar problem in other dpdk version, include dpdk 19.11.Even though >>> the deivce hotplug detach is discarded,but i think the shared memory changed is unexpected by primary >>> process. Hold on, just for my understanding. As far as I can see RTE_ETH_DEV_DETACHABLE was removed in 17.11. Does it change something in above description? >> I agree this is the problem. >> In the driver code, 'rte_eth_copy_pci_info' is called only by primary process, >> >> but the generic code is faulty. >> >> And in 19.11 additionally 'eth_dev_pci_specific_init' also seems has same problem. Yes, as I understand RTE_ETH_DEV_CLOSE_REMOVE, RTE_ETH_DEV_BONDED_SLAVE, RTE_ETH_DEV_REPRESENTOR and RTE_ETH_DEV_NOLIVE_MAC_ADDR may be lost because of reinit (if not restored in other branches). Bad anyway. >>> Our driver is ixgbe, i think this problem has a little relationship with driver, Secondary process >>> enters "rte_eth_copy_pci_info" by "rte_eth_dev_pci_allocate".And I agree your opinion, the helper >>> function should simple on what it does.I have two ways to fix this problem, one is add an if-statement >>> >>> in "rte_eth_dev_pci_allocate" function to forbid secondary process enters "rte_eth_copy_pci_info" function, >>> another way is add an if-statement in "rte_eth_copy_pci_info" function to forbid secondary process change >>> shared memory.And First way need to ensure the "rte_eth_copy_pci_info" function won't be called anywhere else. >>> I think the second way is simple and lower risk. >> >> Yes these are the two options. >> >> I agree adding check in the 'rte_eth_copy_pci_info' covers all cases and safer. >> BUT my concern was adding decision making to simple/leaf function and make it >> harder to debug/use, instead of giving what primary/secondary process should >> call decision in higher level. >> >> But I just recognized that some PMDs are calling 'rte_eth_copy_pci_info' on >> secondary process, like mlx4 or szedata2, and most probably this is not their >> intention. >> And 'eth_dev->intr_handle' set in 'rte_eth_copy_pci_info', not calling this >> function may have side affect of 'eth_dev->intr_handle' not set in secondary. >> >> With above considerations I am OK to your proposal to cover all cases, Thomas, >> Andrew, any concern? I would put if condition in rte_eth_copy_pci_info(). It is the function which writes shared space from secondary process when it should not be done and it should be fixed there. > Do you mean drivers need to be fixed? I'm not sure that I fully understand it. Since copy function cares about intr_handle copying I'm afraid that it is not 100% correct to skip it in secondary process completely as many drivers do right now. Basically it makes eth_dev structure in secondary process inconsistent. However, it looks like most of these drivers simply obtain handle from pci_dev directly and it explains why they are not affected. There are exceptions which are potentially bugs, e.g. drivers/net/ice/ice_ethdev.c: ice_interrupt_handler at the end. I think that it would be better if intr_handle is always correct in eth_dev (both primary and secondary cases) and drivers use it instead of the same from pci_dev.