From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 01280A0542; Thu, 13 Feb 2020 15:00:26 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id C1E811BFE5; Thu, 13 Feb 2020 15:00:26 +0100 (CET) Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by dpdk.org (Postfix) with ESMTP id B5B0A1BFD1 for ; Thu, 13 Feb 2020 15:00:25 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga105.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 13 Feb 2020 06:00:24 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.70,436,1574150400"; d="scan'208";a="238039229" Received: from fyigit-mobl.ger.corp.intel.com (HELO [10.237.221.20]) ([10.237.221.20]) by orsmga006.jf.intel.com with ESMTP; 13 Feb 2020 06:00:21 -0800 To: Thomas Monjalon , Matan Azrad , Bernard Iremonger Cc: dev@dpdk.org, Gaetan Rivet , David Marchand , Jeff Guo , Qi Zhang References: <1573548459-6931-1-git-send-email-matan@mellanox.com> <200f3f01-fedb-b795-a733-e135957e8e99@intel.com> <1645032.4herOUoSWf@xps> <3042500.kGzlxMrEDr@xps> From: Ferruh Yigit Autocrypt: addr=ferruh.yigit@intel.com; prefer-encrypt=mutual; keydata= mQINBFXZCFABEADCujshBOAaqPZpwShdkzkyGpJ15lmxiSr3jVMqOtQS/sB3FYLT0/d3+bvy qbL9YnlbPyRvZfnP3pXiKwkRoR1RJwEo2BOf6hxdzTmLRtGtwWzI9MwrUPj6n/ldiD58VAGQ +iR1I/z9UBUN/ZMksElA2D7Jgg7vZ78iKwNnd+vLBD6I61kVrZ45Vjo3r+pPOByUBXOUlxp9 GWEKKIrJ4eogqkVNSixN16VYK7xR+5OUkBYUO+sE6etSxCr7BahMPKxH+XPlZZjKrxciaWQb +dElz3Ab4Opl+ZT/bK2huX+W+NJBEBVzjTkhjSTjcyRdxvS1gwWRuXqAml/sh+KQjPV1PPHF YK5LcqLkle+OKTCa82OvUb7cr+ALxATIZXQkgmn+zFT8UzSS3aiBBohg3BtbTIWy51jNlYdy ezUZ4UxKSsFuUTPt+JjHQBvF7WKbmNGS3fCid5Iag4tWOfZoqiCNzxApkVugltxoc6rG2TyX CmI2rP0mQ0GOsGXA3+3c1MCdQFzdIn/5tLBZyKy4F54UFo35eOX8/g7OaE+xrgY/4bZjpxC1 1pd66AAtKb3aNXpHvIfkVV6NYloo52H+FUE5ZDPNCGD0/btFGPWmWRmkPybzColTy7fmPaGz cBcEEqHK4T0aY4UJmE7Ylvg255Kz7s6wGZe6IR3N0cKNv++O7QARAQABtCVGZXJydWggWWln aXQgPGZlcnJ1aC55aWdpdEBpbnRlbC5jb20+iQJUBBMBCgA+AhsDAh4BAheABQsJCAcDBRUK CQgLBRYCAwEAFiEE0jZTh0IuwoTjmYHH+TPrQ98TYR8FAl1meboFCQlupOoACgkQ+TPrQ98T YR9ACBAAv2tomhyxY0Tp9Up7mNGLfEdBu/7joB/vIdqMRv63ojkwr9orQq5V16V/25+JEAD0 60cKodBDM6HdUvqLHatS8fooWRueSXHKYwJ3vxyB2tWDyZrLzLI1jxEvunGodoIzUOtum0Ce gPynnfQCelXBja0BwLXJMplM6TY1wXX22ap0ZViC0m714U5U4LQpzjabtFtjT8qOUR6L7hfy YQ72PBuktGb00UR/N5UrR6GqB0x4W41aZBHXfUQnvWIMmmCrRUJX36hOTYBzh+x86ULgg7H2 1499tA4o6rvE13FiGccplBNWCAIroAe/G11rdoN5NBgYVXu++38gTa/MBmIt6zRi6ch15oLA Ln2vHOdqhrgDuxjhMpG2bpNE36DG/V9WWyWdIRlz3NYPCDM/S3anbHlhjStXHOz1uHOnerXM 1jEjcsvmj1vSyYoQMyRcRJmBZLrekvgZeh7nJzbPHxtth8M7AoqiZ/o/BpYU+0xZ+J5/szWZ aYxxmIRu5ejFf+Wn9s5eXNHmyqxBidpCWvcbKYDBnkw2+Y9E5YTpL0mS0dCCOlrO7gca27ux ybtbj84aaW1g0CfIlUnOtHgMCmz6zPXThb+A8H8j3O6qmPoVqT3qnq3Uhy6GOoH8Fdu2Vchh TWiF5yo+pvUagQP6LpslffufSnu+RKAagkj7/RSuZV25Ag0EV9ZMvgEQAKc0Db17xNqtSwEv mfp4tkddwW9XA0tWWKtY4KUdd/jijYqc3fDD54ESYpV8QWj0xK4YM0dLxnDU2IYxjEshSB1T qAatVWz9WtBYvzalsyTqMKP3w34FciuL7orXP4AibPtrHuIXWQOBECcVZTTOdZYGAzaYzxiA ONzF9eTiwIqe9/oaOjTwTLnOarHt16QApTYQSnxDUQljeNvKYt1lZE/gAUUxNLWsYyTT+22/ vU0GDUahsJxs1+f1yEr+OGrFiEAmqrzpF0lCS3f/3HVTU6rS9cK3glVUeaTF4+1SK5ZNO35p iVQCwphmxa+dwTG/DvvHYCtgOZorTJ+OHfvCnSVjsM4kcXGjJPy3JZmUtyL9UxEbYlrffGPQ I3gLXIGD5AN5XdAXFCjjaID/KR1c9RHd7Oaw0Pdcq9UtMLgM1vdX8RlDuMGPrj5sQrRVbgYH fVU/TQCk1C9KhzOwg4Ap2T3tE1umY/DqrXQgsgH71PXFucVjOyHMYXXugLT8YQ0gcBPHy9mZ qw5mgOI5lCl6d4uCcUT0l/OEtPG/rA1lxz8ctdFBVOQOxCvwRG2QCgcJ/UTn5vlivul+cThi 6ERPvjqjblLncQtRg8izj2qgmwQkvfj+h7Ex88bI8iWtu5+I3K3LmNz/UxHBSWEmUnkg4fJl Rr7oItHsZ0ia6wWQ8lQnABEBAAGJAjwEGAEKACYCGwwWIQTSNlOHQi7ChOOZgcf5M+tD3xNh HwUCXWZ5wAUJB3FgggAKCRD5M+tD3xNhH2O+D/9OEz62YuJQLuIuOfL67eFTIB5/1+0j8Tsu o2psca1PUQ61SZJZOMl6VwNxpdvEaolVdrpnSxUF31kPEvR0Igy8HysQ11pj8AcgH0a9FrvU /8k2Roccd2ZIdpNLkirGFZR7LtRw41Kt1Jg+lafI0efkiHKMT/6D/P1EUp1RxOBNtWGV2hrd 0Yg9ds+VMphHHU69fDH02SwgpvXwG8Qm14Zi5WQ66R4CtTkHuYtA63sS17vMl8fDuTCtvfPF HzvdJLIhDYN3Mm1oMjKLlq4PUdYh68Fiwm+boJoBUFGuregJFlO3hM7uHBDhSEnXQr5mqpPM 6R/7Q5BjAxrwVBisH0yQGjsWlnysRWNfExAE2sRePSl0or9q19ddkRYltl6X4FDUXy2DTXa9 a+Fw4e1EvmcF3PjmTYs9IE3Vc64CRQXkhujcN4ZZh5lvOpU8WgyDxFq7bavFnSS6kx7Tk29/ wNJBp+cf9qsQxLbqhW5kfORuZGecus0TLcmpZEFKKjTJBK9gELRBB/zoN3j41hlEl7uTUXTI JQFLhpsFlEdKLujyvT/aCwP3XWT+B2uZDKrMAElF6ltpTxI53JYi22WO7NH7MR16Fhi4R6vh FHNBOkiAhUpoXRZXaCR6+X4qwA8CwHGqHRBfYFSU/Ulq1ZLR+S3hNj2mbnSx0lBs1eEqe2vh cA== Message-ID: <7c3daec0-b1da-16f7-088e-0cb8d313a114@intel.com> Date: Thu, 13 Feb 2020 14:00:21 +0000 MIME-Version: 1.0 In-Reply-To: <3042500.kGzlxMrEDr@xps> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit Subject: Re: [dpdk-dev] [dpdk-stable] [PATCH 2/2] app/testpmd: fix invalid port detaching X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 2/13/2020 1:36 PM, Thomas Monjalon wrote: > More details below about the plan for 20.02. > > 13/02/2020 13:37, Thomas Monjalon: >> Hi, >> >> This discussion becomes confusing so I do a summary below. >> I think we can do several fixes in 20.02. Thanks for checking this. >> >> 12/02/2020 14:49, Ferruh Yigit: >>> On 2/3/2020 5:10 PM, Matan Azrad wrote: >> >> [stripping long discussion in favor of a summary below] >> >>>> Even if the PMD clear the device pointer, the testpmd still may release wrong rte_device. >>> >>> Yes it may, although that is less likely to occur, it requires a new device hot >>> added between close() and detach of the other device. >>> >>> Would you be agree to say there are two problems: >>> >>> 1) When testpmd close a port, a new attached port can re-use it over writing >>> some fields, relying the data structures of the closed port is not safe. >>> >>> 2) PMD not cleaning ethdev->device pointer in the .remove() may cause issues in >>> double detach of a port. >>> >>> >>> For (1) I suggest fixing it in the attach path, don't re-use an eth_dev port id >>> unless it is completely freed, may need to add new state for it. Does it make sense? >> >> Yes we could add a CLOSED state which is set on ethdev close. >> When the rte_device is freed, the PMD could set attached ports as UNUSED. >> But given some ethdev ports can be open and closed dynamically, >> I am not sure it is a good solution to keep them in CLOSED state and ask >> PMD to remember them. >> >> An alternative workaround could be to allocate port_id by incrementing >> a saved biggest id. So the race condition would be very unlikely. >> The drawbacks are having big port_id numbers and changing the id >> allocation algorithm (which is not documented anyway). OK to keep increase port_id instead of re-using closed ones, that simplifies a lot. >> >> The proposals above for port_id allocation or states rework cannot be >> done in 20.02. Let's discuss and work on it in a separated thread. +1 >> >>> For (2) PMDs want to get hotplug support needs to fix it. >> >> Yes PMDs should clear rte_eth_devices[port_id].device in .remove(). > > I am sending a patch adding > memset(eth_dev, 0, sizeof(struct rte_eth_dev)); > in rte_eth_dev_release_port(). > But this patch cannot be merged after 20.02-rc1. It will wait for 20.05. Not sure about this, close() calls the 'rte_eth_dev_release_port()', memset the struct in close() will wipe the device pointers and prevents freeing them in hot remove, silently. > >> We must also protect from user calling detach on a closed port >> by adding a check in cmd_operate_detach_port_parsed(), >> before calling detach_port_device(). > > I am sending a patch adding RTE_ETH_VALID_PORTID_OR_RET() > in cmd_operate_detach_port_parsed(). > It should fix the issue observed by Matan with double detach. > It will be a double protection if keeping the check > port_id_is_invalid() in detach_port_device(). OK > >> The hotplug rmv_port_callback() must be able to call detach after close. >> There are three possible fixes: >> - revert the port_id_is_invalid() check in detach_port_device() >> - call rte_dev_remove(rte_device) directly >> - call a new function with rte_device (detach_port_device() can use it) > > I am sending a patch implementing the third alternative > as it is both keeping the detach behaviour and fixing the race condition > (i.e. protect from new port re-using the port_id between close and detach). Should work, only concern if any possible side affect occurs, can be discussed on patch. > >> About the function detach_port_device() itself, yes this function is >> strange to say the least. It was a convenience for detaching a rte_device >> from a port_id. >> The cleanup of siblings with RTE_ETH_FOREACH_DEV_OF(sibling, dev), >> should probably be removed. I've added it as a temporary solution >> before all PMDs are properly fixed: >> rte_eth_devices[sibling].device = NULL; > > I propose sending such patch in 20.05 in order to merge the memset above > first, and have time to get agreement from all PMD maintainers. OK > >> For info, there is a function detach_device() used by the command >> "device detach " > > >