Re: [dpdk-dev] [PATCH 1/2] net/failsafe: fix removed sub-device cleanup

[Ophir Munk <ophirmu@mellanox.com>]

Hi,
Please find comments inline.

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Matan Azrad
> Sent: Thursday, May 17, 2018 9:52 PM
> To: Gaetan Rivet <gaetan.rivet@6wind.com>
> Cc: dev@dpdk.org; stable@dpdk.org
> Subject: [dpdk-dev] [PATCH 1/2] net/failsafe: fix removed sub-device cleanup
> 
> The fail-safe PMD registers to RMV event for each removable sub-device
> port in order to cleanup the sub-device resources and switch the Tx sub-
> device directly when it is plugged-out.
> 
> In the removal time, the fail-safe PMD stops and closes the sub-device but it

During removal time...

> doesn't unregister the LSC and RMV callbacks of the sub-device port.
> 
> It can lead the callbacks to be called for a port which is no more associated
> to the fail-safe sub-device, because there is not a guaranty that a sub-device

associated with........ guarantee that

> gets the same port ID for each plug-in process. This port, for example, may
> belong to another sub-device of a different fail-safe device.
> 
> Unregister the LSC and RMV callbacks for sub-devices which are not used.
> 
> Fixes: 598fb8aec6f6 ("net/failsafe: support device removal")
> Cc: stable@dpdk.org
> 
> Signed-off-by: Matan Azrad <matan@mellanox.com>
> ---
>  drivers/net/failsafe/failsafe_ether.c   | 22 ++++++++++++++++++++++
>  drivers/net/failsafe/failsafe_ops.c     |  5 +++++
>  drivers/net/failsafe/failsafe_private.h |  3 +++
>  3 files changed, 30 insertions(+)
> 
> diff --git a/drivers/net/failsafe/failsafe_ether.c
> b/drivers/net/failsafe/failsafe_ether.c
> index 733e95d..2bbee82 100644
> --- a/drivers/net/failsafe/failsafe_ether.c
> +++ b/drivers/net/failsafe/failsafe_ether.c
> @@ -260,6 +260,7 @@
>  		sdev->state = DEV_ACTIVE;
>  		/* fallthrough */
>  	case DEV_ACTIVE:
> +		failsafe_eth_dev_unregister_callbacks(sdev);
>  		rte_eth_dev_close(PORT_ID(sdev));
>  		sdev->state = DEV_PROBED;
>  		/* fallthrough */
> @@ -321,6 +322,27 @@
>  }
> 
>  void
> +failsafe_eth_dev_unregister_callbacks(struct sub_device *sdev) {
> +	if (sdev == NULL)
> +		return;
> +	if (sdev->rmv_callback) {
> +		rte_eth_dev_callback_unregister(PORT_ID(sdev),
> +					RTE_ETH_EVENT_INTR_RMV,
> +					failsafe_eth_rmv_event_callback,
> +					sdev);

Please justify ignoring returned error from rte_eth_dev_callback_unregister call. 
I have noticed that this call returned error is ignored in other places in code (failsafe.c)...but if for example the callback returns with -EAGAIN shouldn't you try again to unregister?
Need to avoid a case where the callback is still registered, while rmv_callback is assigned to 0

> +		sdev->rmv_callback = 0;
> +	}
> +	if (sdev->lsc_callback) {
> +		rte_eth_dev_callback_unregister(PORT_ID(sdev),
> +					RTE_ETH_EVENT_INTR_LSC,
> +					failsafe_eth_lsc_event_callback,
> +					sdev);

Same comment here regarding the returned error from rte_eth_dev_callback_unregister call and lsc_event_callback.

> +		sdev->lsc_callback = 0;
> +	}
> +}
> +
> +void
>  failsafe_dev_remove(struct rte_eth_dev *dev)  {
>  	struct sub_device *sdev;
> diff --git a/drivers/net/failsafe/failsafe_ops.c
> b/drivers/net/failsafe/failsafe_ops.c
> index d04277b..e0570b6 100644
> --- a/drivers/net/failsafe/failsafe_ops.c
> +++ b/drivers/net/failsafe/failsafe_ops.c
> @@ -146,6 +146,8 @@
>  			if (ret)
>  				WARN("Failed to register RMV callback for
> sub_device %d",
>  				     SUB_ID(sdev));
> +			else
> +				sdev->rmv_callback = 1;
>  		}
>  		dev->data->dev_conf.intr_conf.rmv = 0;
>  		if (lsc_interrupt) {
> @@ -156,6 +158,8 @@
>  			if (ret)
>  				WARN("Failed to register LSC callback for
> sub_device %d",
>  				     SUB_ID(sdev));
> +			else
> +				sdev->lsc_callback = 1;
>  		}
>  		dev->data->dev_conf.intr_conf.lsc = lsc_enabled;
>  		sdev->state = DEV_ACTIVE;
> @@ -282,6 +286,7 @@
>  	PRIV(dev)->state = DEV_ACTIVE - 1;
>  	FOREACH_SUBDEV_STATE(sdev, i, dev, DEV_ACTIVE) {
>  		DEBUG("Closing sub_device %d", i);
> +		failsafe_eth_dev_unregister_callbacks(sdev);

DEBUG("Closing...") message should be in its right place just before closing sdev (next line).

>  		rte_eth_dev_close(PORT_ID(sdev));
>  		sdev->state = DEV_ACTIVE - 1;
>  	}
> diff --git a/drivers/net/failsafe/failsafe_private.h
> b/drivers/net/failsafe/failsafe_private.h
> index 7e6a3f8..3222653 100644
> --- a/drivers/net/failsafe/failsafe_private.h
> +++ b/drivers/net/failsafe/failsafe_private.h
> @@ -119,6 +119,8 @@ struct sub_device {
>  	volatile unsigned int remove:1;
>  	/* flow isolation state */
>  	int flow_isolated:1;
> +	unsigned int rmv_callback:1;
> +	unsigned int lsc_callback:1;

Nit-pick: please consider adding a description for rmv_callback and lsc_callback similar to the other fields in this struct.

>  };
> 
>  struct fs_priv {
> @@ -211,6 +213,7 @@ uint16_t failsafe_tx_burst_fast(void *txq,
>  /* ETH_DEV */
> 
>  int failsafe_eth_dev_state_sync(struct rte_eth_dev *dev);
> +void failsafe_eth_dev_unregister_callbacks(struct sub_device *sdev);
>  void failsafe_dev_remove(struct rte_eth_dev *dev);  void
> failsafe_stats_increment(struct rte_eth_stats *to,
>  				struct rte_eth_stats *from);
> --
> 1.9.5