[dpdk-dev] [PATCH 1/3] ethdev: avoid error on PCI unplug of already closed ethdev

[dpdk-dev] [PATCH 1/3] ethdev: avoid error on PCI unplug of already closed ethdev

[Andrew Rybchenko]

If PCI Ethernet device driver removes it on close
(RTE_ETH_DEV_CLOSE_REMOVE) and later PCI device itself is unplugged,
it should not fail because of Ethernet device is already removed.

Fixes: 23ea57a2a0ce ("ethdev: complete closing of port")
Cc: stable@dpdk.org

Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Ivan Malov <ivan.malov@oktetlabs.ru>
---
Cc: Thomas Monjalon <thomas@monjalon.net>
Cc: Ferruh Yigit <ferruh.yigit@intel.com>

 lib/librte_ethdev/rte_ethdev_pci.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/librte_ethdev/rte_ethdev_pci.h b/lib/librte_ethdev/rte_ethdev_pci.h
index 23257e986..ccdbb46ec 100644
--- a/lib/librte_ethdev/rte_ethdev_pci.h
+++ b/lib/librte_ethdev/rte_ethdev_pci.h
@@ -184,7 +184,7 @@ rte_eth_dev_pci_generic_remove(struct rte_pci_device *pci_dev,
 
 	eth_dev = rte_eth_dev_allocated(pci_dev->device.name);
 	if (!eth_dev)
-		return -ENODEV;
+		return 0;
 
 	if (dev_uninit) {
 		ret = dev_uninit(eth_dev);
-- 
2.17.1

[dpdk-dev] [PATCH 2/3] net/sfc: ensure that device is closed on removal

[Andrew Rybchenko]

PCI device may be unplugged when ethdev is not closed yet.

Fixes: aaa3f5f0f79d ("net/sfc: add configure and close stubs")
Cc: stable@dpdk.org

Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Ivan Malov <ivan.malov@oktetlabs.ru>
---
 drivers/net/sfc/sfc_ethdev.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index be185d5e9..583f31b8c 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -2196,6 +2196,8 @@ sfc_eth_dev_uninit(struct rte_eth_dev *dev)
 		return 0;
 	}
 
+	sfc_dev_close(dev);
+
 	sa = sfc_adapter_by_eth_dev(dev);
 	sfc_log_init(sa, "entry");
 
-- 
2.17.1

[dpdk-dev] [PATCH 3/3] net/sfc: release port upon close

[Andrew Rybchenko]

Set RTE_ETH_DEV_CLOSE_REMOVE upon probe so all the resources
for the port can be freed by rte_eth_dev_close() and the ethdev
port freed.

Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Ivan Malov <ivan.malov@oktetlabs.ru>
---
 drivers/net/sfc/sfc_ethdev.c | 51 ++++++++++++++++++------------------
 1 file changed, 26 insertions(+), 25 deletions(-)

diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c
index 583f31b8c..279b58641 100644
--- a/drivers/net/sfc/sfc_ethdev.c
+++ b/drivers/net/sfc/sfc_ethdev.c
@@ -34,6 +34,10 @@ uint32_t sfc_logtype_driver;
 static struct sfc_dp_list sfc_dp_head =
 	TAILQ_HEAD_INITIALIZER(sfc_dp_head);
 
+
+static void sfc_eth_dev_clear_ops(struct rte_eth_dev *dev);
+
+
 static int
 sfc_fw_version_get(struct rte_eth_dev *dev, char *fw_version, size_t fw_size)
 {
@@ -335,9 +339,29 @@ sfc_dev_close(struct rte_eth_dev *dev)
 		sfc_err(sa, "unexpected adapter state %u on close", sa->state);
 		break;
 	}
+
+	/*
+	 * Cleanup all resources in accordance with RTE_ETH_DEV_CLOSE_REMOVE.
+	 * Rollback primary process sfc_eth_dev_init() below.
+	 */
+
+	sfc_eth_dev_clear_ops(dev);
+
+	sfc_detach(sa);
+	sfc_unprobe(sa);
+
+	sfc_kvargs_cleanup(sa);
+
 	sfc_adapter_unlock(sa);
+	sfc_adapter_lock_fini(sa);
 
 	sfc_log_init(sa, "done");
+
+	/* Required for logging, so cleanup last */
+	sa->eth_dev = NULL;
+
+	dev->process_private = NULL;
+	free(sa);
 }
 
 static void
@@ -2123,6 +2147,8 @@ sfc_eth_dev_init(struct rte_eth_dev *dev)
 
 	sfc_log_init(sa, "entry");
 
+	dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE;
+
 	dev->data->mac_addrs = rte_zmalloc("sfc", RTE_ETHER_ADDR_LEN, 0);
 	if (dev->data->mac_addrs == NULL) {
 		rc = ENOMEM;
@@ -2189,8 +2215,6 @@ sfc_eth_dev_init(struct rte_eth_dev *dev)
 static int
 sfc_eth_dev_uninit(struct rte_eth_dev *dev)
 {
-	struct sfc_adapter *sa;
-
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
 		sfc_eth_dev_secondary_clear_ops(dev);
 		return 0;
@@ -2198,29 +2222,6 @@ sfc_eth_dev_uninit(struct rte_eth_dev *dev)
 
 	sfc_dev_close(dev);
 
-	sa = sfc_adapter_by_eth_dev(dev);
-	sfc_log_init(sa, "entry");
-
-	sfc_adapter_lock(sa);
-
-	sfc_eth_dev_clear_ops(dev);
-
-	sfc_detach(sa);
-	sfc_unprobe(sa);
-
-	sfc_kvargs_cleanup(sa);
-
-	sfc_adapter_unlock(sa);
-	sfc_adapter_lock_fini(sa);
-
-	sfc_log_init(sa, "done");
-
-	/* Required for logging, so cleanup last */
-	sa->eth_dev = NULL;
-
-	dev->process_private = NULL;
-	free(sa);
-
 	return 0;
 }
 
-- 
2.17.1

Re: [dpdk-dev] [dpdk-stable] [PATCH 1/3] ethdev: avoid error on PCI unplug of already closed ethdev

[David Marchand]

On Fri, Jun 21, 2019 at 11:41 AM Andrew Rybchenko <arybchenko@solarflare.com>
wrote:

> If PCI Ethernet device driver removes it on close
> (RTE_ETH_DEV_CLOSE_REMOVE) and later PCI device itself is unplugged,
> it should not fail because of Ethernet device is already removed.
>
> Fixes: 23ea57a2a0ce ("ethdev: complete closing of port")
> Cc: stable@dpdk.org
>
> Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
> Reviewed-by: Ivan Malov <ivan.malov@oktetlabs.ru>
> ---
> Cc: Thomas Monjalon <thomas@monjalon.net>
> Cc: Ferruh Yigit <ferruh.yigit@intel.com>
>
>  lib/librte_ethdev/rte_ethdev_pci.h | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/lib/librte_ethdev/rte_ethdev_pci.h
> b/lib/librte_ethdev/rte_ethdev_pci.h
> index 23257e986..ccdbb46ec 100644
> --- a/lib/librte_ethdev/rte_ethdev_pci.h
> +++ b/lib/librte_ethdev/rte_ethdev_pci.h
> @@ -184,7 +184,7 @@ rte_eth_dev_pci_generic_remove(struct rte_pci_device
> *pci_dev,
>
>         eth_dev = rte_eth_dev_allocated(pci_dev->device.name);
>         if (!eth_dev)
> -               return -ENODEV;
> +               return 0;
>
>         if (dev_uninit) {
>                 ret = dev_uninit(eth_dev);
> --
> 2.17.1
>

We are changing the behavior for all drivers, while I understand this
should apply to the ones that have the RTE_ETH_DEV_CLOSE_REMOVE flag.
Btw, I had reported this earlier [1], care to add a little Reported-by for
me ? :-)

1: http://mails.dpdk.org/archives/dev/2019-June/134150.html

-- 
David Marchand

Re: [dpdk-dev] [dpdk-stable] [PATCH 1/3] ethdev: avoid error on PCI unplug of already closed ethdev

[Andrew Rybchenko]

On 6/21/19 12:52 PM, David Marchand wrote:
> On Fri, Jun 21, 2019 at 11:41 AM Andrew Rybchenko <arybchenko@solarflare.com>
> wrote:
>> If PCI Ethernet device driver removes it on close
>> (RTE_ETH_DEV_CLOSE_REMOVE) and later PCI device itself is unplugged,
>> it should not fail because of Ethernet device is already removed.
>>
>> Fixes: 23ea57a2a0ce ("ethdev: complete closing of port")
>> Cc: stable@dpdk.org
>>
>> Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
>> Reviewed-by: Ivan Malov <ivan.malov@oktetlabs.ru>
>> ---
>> Cc: Thomas Monjalon <thomas@monjalon.net>
>> Cc: Ferruh Yigit <ferruh.yigit@intel.com>
>>
>>   lib/librte_ethdev/rte_ethdev_pci.h | 2 +-
>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/lib/librte_ethdev/rte_ethdev_pci.h
>> b/lib/librte_ethdev/rte_ethdev_pci.h
>> index 23257e986..ccdbb46ec 100644
>> --- a/lib/librte_ethdev/rte_ethdev_pci.h
>> +++ b/lib/librte_ethdev/rte_ethdev_pci.h
>> @@ -184,7 +184,7 @@ rte_eth_dev_pci_generic_remove(struct rte_pci_device
>> *pci_dev,
>>
>>          eth_dev = rte_eth_dev_allocated(pci_dev->device.name);
>>          if (!eth_dev)
>> -               return -ENODEV;
>> +               return 0;
>>
>>          if (dev_uninit) {
>>                  ret = dev_uninit(eth_dev);
>> --
>> 2.17.1
>>
> We are changing the behavior for all drivers, while I understand this
> should apply to the ones that have the RTE_ETH_DEV_CLOSE_REMOVE flag.
> Btw, I had reported this earlier [1], care to add a little Reported-by for
> me ? :-)

Yes, I agree. Unfortunately there is no ethdev here to check
RTE_ETH_DEV_CLOSE_REMOVE. It could be PCI driver flag for the
feature, but I'm not sure if it makes sense to add one more
flag for transition.

> 1: http://mails.dpdk.org/archives/dev/2019-June/134150.html

Re: [dpdk-dev] [dpdk-stable] [PATCH 1/3] ethdev: avoid error on PCI unplug of already closed ethdev

[Ferruh Yigit]

On 6/21/2019 11:15 AM, Andrew Rybchenko wrote:
> On 6/21/19 12:52 PM, David Marchand wrote:
>> On Fri, Jun 21, 2019 at 11:41 AM Andrew Rybchenko <arybchenko@solarflare.com>
>> wrote:
>>> If PCI Ethernet device driver removes it on close
>>> (RTE_ETH_DEV_CLOSE_REMOVE) and later PCI device itself is unplugged,
>>> it should not fail because of Ethernet device is already removed.
>>>
>>> Fixes: 23ea57a2a0ce ("ethdev: complete closing of port")
>>> Cc: stable@dpdk.org
>>>
>>> Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
>>> Reviewed-by: Ivan Malov <ivan.malov@oktetlabs.ru>
>>> ---
>>> Cc: Thomas Monjalon <thomas@monjalon.net>
>>> Cc: Ferruh Yigit <ferruh.yigit@intel.com>
>>>
>>>   lib/librte_ethdev/rte_ethdev_pci.h | 2 +-
>>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>>
>>> diff --git a/lib/librte_ethdev/rte_ethdev_pci.h
>>> b/lib/librte_ethdev/rte_ethdev_pci.h
>>> index 23257e986..ccdbb46ec 100644
>>> --- a/lib/librte_ethdev/rte_ethdev_pci.h
>>> +++ b/lib/librte_ethdev/rte_ethdev_pci.h
>>> @@ -184,7 +184,7 @@ rte_eth_dev_pci_generic_remove(struct rte_pci_device
>>> *pci_dev,
>>>
>>>          eth_dev = rte_eth_dev_allocated(pci_dev->device.name);
>>>          if (!eth_dev)
>>> -               return -ENODEV;
>>> +               return 0;
>>>
>>>          if (dev_uninit) {
>>>                  ret = dev_uninit(eth_dev);
>>> --
>>> 2.17.1
>>>
>> We are changing the behavior for all drivers, while I understand this
>> should apply to the ones that have the RTE_ETH_DEV_CLOSE_REMOVE flag.
>> Btw, I had reported this earlier [1], care to add a little Reported-by for
>> me ? :-)
> 
> Yes, I agree. Unfortunately there is no ethdev here to check
> RTE_ETH_DEV_CLOSE_REMOVE. It could be PCI driver flag for the
> feature, but I'm not sure if it makes sense to add one more
> flag for transition.
> 
>> 1: http://mails.dpdk.org/archives/dev/2019-June/134150.html

Reported-by: David Marchand <david.marchand@redhat.com>

> 

Acked-by: Ferruh Yigit <ferruh.yigit@intel.com>

Re: [dpdk-dev] [dpdk-stable] [PATCH 1/3] ethdev: avoid error on PCI unplug of already closed ethdev

[Ferruh Yigit]

On 7/2/2019 6:38 PM, Ferruh Yigit wrote:
> On 6/21/2019 11:15 AM, Andrew Rybchenko wrote:
>> On 6/21/19 12:52 PM, David Marchand wrote:
>>> On Fri, Jun 21, 2019 at 11:41 AM Andrew Rybchenko <arybchenko@solarflare.com>
>>> wrote:
>>>> If PCI Ethernet device driver removes it on close
>>>> (RTE_ETH_DEV_CLOSE_REMOVE) and later PCI device itself is unplugged,
>>>> it should not fail because of Ethernet device is already removed.
>>>>
>>>> Fixes: 23ea57a2a0ce ("ethdev: complete closing of port")
>>>> Cc: stable@dpdk.org
>>>>
>>>> Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
>>>> Reviewed-by: Ivan Malov <ivan.malov@oktetlabs.ru>
>>>> ---
>>>> Cc: Thomas Monjalon <thomas@monjalon.net>
>>>> Cc: Ferruh Yigit <ferruh.yigit@intel.com>
>>>>
>>>>   lib/librte_ethdev/rte_ethdev_pci.h | 2 +-
>>>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>>>
>>>> diff --git a/lib/librte_ethdev/rte_ethdev_pci.h
>>>> b/lib/librte_ethdev/rte_ethdev_pci.h
>>>> index 23257e986..ccdbb46ec 100644
>>>> --- a/lib/librte_ethdev/rte_ethdev_pci.h
>>>> +++ b/lib/librte_ethdev/rte_ethdev_pci.h
>>>> @@ -184,7 +184,7 @@ rte_eth_dev_pci_generic_remove(struct rte_pci_device
>>>> *pci_dev,
>>>>
>>>>          eth_dev = rte_eth_dev_allocated(pci_dev->device.name);
>>>>          if (!eth_dev)
>>>> -               return -ENODEV;
>>>> +               return 0;
>>>>
>>>>          if (dev_uninit) {
>>>>                  ret = dev_uninit(eth_dev);
>>>> --
>>>> 2.17.1
>>>>
>>> We are changing the behavior for all drivers, while I understand this
>>> should apply to the ones that have the RTE_ETH_DEV_CLOSE_REMOVE flag.
>>> Btw, I had reported this earlier [1], care to add a little Reported-by for
>>> me ? :-)
>>
>> Yes, I agree. Unfortunately there is no ethdev here to check
>> RTE_ETH_DEV_CLOSE_REMOVE. It could be PCI driver flag for the
>> feature, but I'm not sure if it makes sense to add one more
>> flag for transition.
>>
>>> 1: http://mails.dpdk.org/archives/dev/2019-June/134150.html
> 
> Reported-by: David Marchand <david.marchand@redhat.com>
> 
>>
> 
> Acked-by: Ferruh Yigit <ferruh.yigit@intel.com>
> 

Series applied to dpdk-next-net/master, thanks.