DPDK patches and discussions
 help / color / mirror / Atom feed
From: Ruifeng Wang <Ruifeng.Wang@arm.com>
To: fengchengwen <fengchengwen@huawei.com>,
	Stephen Hemminger <stephen@networkplumber.org>
Cc: Ashok Kaladi <ashok.k.kaladi@intel.com>,
	"jerinj@marvell.com" <jerinj@marvell.com>,
	"thomas@monjalon.net" <thomas@monjalon.net>,
	Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>,
	"dev@dpdk.org" <dev@dpdk.org>,
	"s.v.naga.harish.k@intel.com" <s.v.naga.harish.k@intel.com>,
	"erik.g.carrillo@intel.com" <erik.g.carrillo@intel.com>,
	"abhinandan.gujjar@intel.com" <abhinandan.gujjar@intel.com>,
	"stable@dpdk.org" <stable@dpdk.org>, nd <nd@arm.com>,
	nd <nd@arm.com>
Subject: RE: [PATCH 2/2] ethdev: fix race condition in fast-path ops setup
Date: Wed, 22 Feb 2023 09:41:01 +0000	[thread overview]
Message-ID: <AS8PR08MB70802E50C2627E0BE865F4089EAA9@AS8PR08MB7080.eurprd08.prod.outlook.com> (raw)
In-Reply-To: <c679cda1-d1b0-3efb-cc5e-580b432ff04a@huawei.com>

> -----Original Message-----
> From: fengchengwen <fengchengwen@huawei.com>
> Sent: Wednesday, February 22, 2023 9:07 AM
> To: Stephen Hemminger <stephen@networkplumber.org>; Ruifeng Wang <Ruifeng.Wang@arm.com>
> Cc: Ashok Kaladi <ashok.k.kaladi@intel.com>; jerinj@marvell.com; thomas@monjalon.net;
> Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>; dev@dpdk.org;
> s.v.naga.harish.k@intel.com; erik.g.carrillo@intel.com; abhinandan.gujjar@intel.com;
> stable@dpdk.org; nd <nd@arm.com>
> Subject: Re: [PATCH 2/2] ethdev: fix race condition in fast-path ops setup
> 
> On 2023/2/22 1:00, Stephen Hemminger wrote:
> > On Tue, 21 Feb 2023 07:24:19 +0000
> > Ruifeng Wang <Ruifeng.Wang@arm.com> wrote:
> >
> >>> -----Original Message-----
> >>> From: fengchengwen <fengchengwen@huawei.com>
> >>> Sent: Monday, February 20, 2023 2:58 PM
> >>> To: Ashok Kaladi <ashok.k.kaladi@intel.com>; jerinj@marvell.com;
> >>> thomas@monjalon.net
> >>> Cc: dev@dpdk.org; s.v.naga.harish.k@intel.com;
> >>> erik.g.carrillo@intel.com; abhinandan.gujjar@intel.com;
> >>> stable@dpdk.org; Ruifeng Wang <Ruifeng.Wang@arm.com>
> >>> Subject: Re: [PATCH 2/2] ethdev: fix race condition in fast-path ops
> >>> setup
> >>>
> >>> On 2023/2/20 14:08, Ashok Kaladi wrote:
> >>>> If ethdev enqueue or dequeue function is called during
> >>>> eth_dev_fp_ops_setup(), it may get pre-empted after setting the
> >>>> function pointers, but before setting the pointer to port data.
> >>>> In this case the newly registered enqueue/dequeue function will use
> >>>> dummy port data and end up in seg fault.
> >>>>
> >>>> This patch moves the updation of each data pointers before updating
> >>>> corresponding function pointers.
> >>>>
> >>>> Fixes: c87d435a4d79 ("ethdev: copy fast-path API into separate
> >>>> structure")
> >>>> Cc: stable@dpdk.org
> >
> > Why is something calling enqueue/dequeue when device is not fully started.
> > A correctly written application would not call rx/tx burst until after
> > ethdev start had finished.
> 
> Please refer the eb0d471a894 (ethdev: add proactive error handling mode), when driver
> recover itself, the application may still invoke enqueue/dequeue API.
> 
> >
> > Would something like this work better?
> >
> > Note: there is another bug in current code. The check for link state
> > interrupt and link_ops could return -ENOTSUP and leave device in indeterminate state.
> > The check should be done before calling PMD.
> >
> > diff --git a/lib/ethdev/rte_ethdev.c b/lib/ethdev/rte_ethdev.c index
> > 0266cc82acb6..d6c163ed85e7 100644
> > --- a/lib/ethdev/rte_ethdev.c
> > +++ b/lib/ethdev/rte_ethdev.c
> > @@ -1582,6 +1582,14 @@ rte_eth_dev_start(uint16_t port_id)
> >  		return 0;
> >  	}
> >
> > +	if (dev->data->dev_conf.intr_conf.lsc == 0 &&
> > +	    dev->dev_ops->link_update == NULL) {
> > +		RTE_ETHDEV_LOG(INFO,
> > +			       "Device with port_id=%"PRIu16" link update not supported\n",
> > +			       port_id);
> > +			return -ENOTSUP;
> > +	}
> > +
> >  	ret = rte_eth_dev_info_get(port_id, &dev_info);
> >  	if (ret != 0)
> >  		return ret;
> > @@ -1591,9 +1599,7 @@ rte_eth_dev_start(uint16_t port_id)
> >  		eth_dev_mac_restore(dev, &dev_info);
> >
> >  	diag = (*dev->dev_ops->dev_start)(dev);
> > -	if (diag == 0)
> > -		dev->data->dev_started = 1;
> > -	else
> > +	if (diag != 0)
> >  		return eth_err(port_id, diag);
> >
> >  	ret = eth_dev_config_restore(dev, &dev_info, port_id); @@ -1611,16
> > +1617,18 @@ rte_eth_dev_start(uint16_t port_id)
> >  		return ret;
> >  	}
> >
> > -	if (dev->data->dev_conf.intr_conf.lsc == 0) {
> > -		if (*dev->dev_ops->link_update == NULL)
> > -			return -ENOTSUP;
> > -		(*dev->dev_ops->link_update)(dev, 0);
> > -	}
> > -
> >  	/* expose selection of PMD fast-path functions */
> >  	eth_dev_fp_ops_setup(rte_eth_fp_ops + port_id, dev);
> >
> > +	/* ensure state is set before marking device ready */
> > +	rte_smp_wmb();

Without a read barrier at the reader side (rte_eth_rx_burst), the wmb here may not fulfill the required data sync.

One solution is to change eth_dev_fp_ops_reset. Replacing dummy_eth_rx_burst with rte_eth_pkt_burst_dummy and
not touch rxq.data/txq.data. By doing this, the sync requirement between pkt_burst handle and qdata can be removed.
Because rte_eth_pkt_burst_dummy doesn't work on any data.
The downside is loss of error log and stack dump when dummy handle is called.

> > +
> >  	rte_ethdev_trace_start(port_id);
> > +
> > +	/* Update current link state */
> > +	if (dev->data->dev_conf.intr_conf.lsc == 0)
> > +		(*dev->dev_ops->link_update)(dev, 0);
> > +
> >  	return 0;
> >  }
> >
> >
> > .
> >

  reply	other threads:[~2023-02-22  9:41 UTC|newest]

Thread overview: 29+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-02-20  6:08 [PATCH 1/2] eventdev: fix race condition in fast-path set function Ashok Kaladi
2023-02-20  6:08 ` [PATCH 2/2] ethdev: fix race condition in fast-path ops setup Ashok Kaladi
2023-02-20  6:57   ` fengchengwen
2023-02-21  7:24     ` Ruifeng Wang
2023-02-21 17:00       ` Stephen Hemminger
2023-02-22  1:07         ` fengchengwen
2023-02-22  9:41           ` Ruifeng Wang [this message]
2023-02-22 10:41           ` Konstantin Ananyev
2023-02-22 22:48             ` Honnappa Nagarahalli
2023-02-23  1:15               ` Stephen Hemminger
2023-02-23  4:47                 ` Honnappa Nagarahalli
2023-02-23  4:40             ` Honnappa Nagarahalli
2023-02-23  8:23               ` fengchengwen
2023-02-23 13:31                 ` Konstantin Ananyev
2023-02-25  1:32                   ` fengchengwen
2023-02-26 17:22                     ` Konstantin Ananyev
2023-02-27  2:56                       ` fengchengwen
2023-02-27 19:08                         ` Konstantin Ananyev
2023-03-03 17:19                       ` Ferruh Yigit
2023-03-06  1:57                         ` fengchengwen
2023-03-06  6:13                         ` Ruifeng Wang
2023-03-06 10:32                           ` Konstantin Ananyev
2023-03-06 11:17                             ` Ajit Khaparde
2023-03-06 11:57                             ` Ferruh Yigit
2023-03-06 12:36                               ` Konstantin Ananyev
2023-02-28 23:57                   ` Honnappa Nagarahalli
2023-02-20  7:01   ` fengchengwen
2023-02-20  9:44   ` Konstantin Ananyev
2023-03-03 16:49   ` Ferruh Yigit

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=AS8PR08MB70802E50C2627E0BE865F4089EAA9@AS8PR08MB7080.eurprd08.prod.outlook.com \
    --to=ruifeng.wang@arm.com \
    --cc=Honnappa.Nagarahalli@arm.com \
    --cc=abhinandan.gujjar@intel.com \
    --cc=ashok.k.kaladi@intel.com \
    --cc=dev@dpdk.org \
    --cc=erik.g.carrillo@intel.com \
    --cc=fengchengwen@huawei.com \
    --cc=jerinj@marvell.com \
    --cc=nd@arm.com \
    --cc=s.v.naga.harish.k@intel.com \
    --cc=stable@dpdk.org \
    --cc=stephen@networkplumber.org \
    --cc=thomas@monjalon.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).