DPDK patches and discussions
 help / color / mirror / Atom feed
From: Thomas Monjalon <thomas@monjalon.net>
To: Ferruh Yigit <ferruh.yigit@intel.com>
Cc: "John W. Linville" <linville@tuxdriver.com>,
	Xiaolong Ye <xiaolong.ye@intel.com>,
	Qi Zhang <qi.z.zhang@intel.com>,
	Shepard Siegel <shepard.siegel@atomicrules.com>,
	Ed Czeck <ed.czeck@atomicrules.com>,
	John Miller <john.miller@atomicrules.com>,
	Igor Russkikh <igor.russkikh@aquantia.com>,
	Pavel Belous <pavel.belous@aquantia.com>,
	Allain Legacy <allain.legacy@windriver.com>,
	Matt Peters <matt.peters@windriver.com>,
	Ravi Kumar <ravi1.kumar@amd.com>, Rasesh Mody <rmody@marvell.com>,
	Shahed Shaikh <shshaikh@marvell.com>,
	Ajit Khaparde <ajit.khaparde@broadcom.com>,
	Somnath Kotur <somnath.kotur@broadcom.com>,
	Chas Williams <chas3@att.com>,
	Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>,
	Hemant Agrawal <hemant.agrawal@nxp.com>,
	Shreyansh Jain <shreyansh.jain@nxp.com>,
	Wenzhuo Lu <wenzhuo.lu@intel.com>,
	Marcin Wojtas <mw@semihalf.com>,
	Michal Krawczyk <mk@semihalf.com>,
	Guy Tzalik <gtzalik@amazon.com>,
	Evgeny Schemeilin <evgenys@amazon.com>,
	Gagandeep Singh <g.singh@nxp.com>,
	Pankaj Chauhan <pankaj.chauhan@nxp.com>,
	John Daley <johndale@cisco.com>,
	Hyong Youb Kim <hyonkim@cisco.com>,
	Gaetan Rivet <gaetan.rivet@6wind.com>,
	Xiao Wang <xiao.w.wang@intel.com>,
	Beilei Xing <beilei.xing@intel.com>,
	Jingjing Wu <jingjing.wu@intel.com>,
	Qiming Yang <qiming.yang@intel.com>,
	Konstantin Ananyev <konstantin.ananyev@intel.com>,
	Shijith Thotton <sthotton@marvell.com>,
	Srisivasubramanian Srinivasan <srinivasan@marvell.com>,
	Matan Azrad <matan@mellanox.com>,
	Shahaf Shuler <shahafs@mellanox.com>,
	Yongseok Koh <yskoh@mellanox.com>, Zyta Szpak <zr@semihalf.com>,
	Liron Himi <lironh@marvell.com>,
	Alan Winkowski <walan@marvell.com>,
	Tomasz Duszynski <tdu@semihalf.com>,
	Stephen Hemminger <sthemmin@microsoft.com>,
	"K. Y. Srinivasan" <kys@microsoft.com>,
	Haiyang Zhang <haiyangz@microsoft.com>,
	Rastislav Cernay <cernay@netcope.com>,
	Jan Remes <remes@netcope.com>,
	Alejandro Lucero <alejandro.lucero@netronome.com>,
	Tetsuya Mukawa <mtetsuyah@gmail.com>,
	Jerin Jacob <jerinj@marvell.com>,
	Bruce Richardson <bruce.richardson@intel.com>,
	Andrew Rybchenko <arybchenko@solarflare.com>,
	Jasvinder Singh <jasvinder.singh@intel.com>,
	Cristian Dumitrescu <cristian.dumitrescu@intel.com>,
	Keith Wiles <keith.wiles@intel.com>,
	Maciej Czekaj <mczekaj@marvell.com>,
	Maxime Coquelin <maxime.coquelin@redhat.com>,
	Tiwei Bie <tiwei.bie@intel.com>,
	Zhihong Wang <zhihong.wang@intel.com>,
	Yong Wang <yongwang@vmware.com>,
	Anatoly Burakov <anatoly.burakov@intel.com>,
	dev@dpdk.org
Subject: Re: [dpdk-dev] CALL to eth PMD maintainers: complete closing of port
Date: Mon, 29 Apr 2019 22:30:00 +0200	[thread overview]
Message-ID: <1748144.UFpUr2FPnr@xps> (raw)
In-Reply-To: <aaff1c99-cff7-ea2c-8f40-9c809ca2f5a7@intel.com>

29/04/2019 18:51, Ferruh Yigit:
> On 4/18/2019 11:59 AM, Thomas Monjalon wrote:
> > Hi all,
> > 
> > Since DPDK 18.11, the behaviour of the close operation is changed
> > if RTE_ETH_DEV_CLOSE_REMOVE is enabled in the driver:
> > port is released (i.e. totally freed and data erased) on close.
> > This new behaviour is enabled per driver for a migration period.
> > 
> > Looking at the code, you can see these comments:
> > /* old behaviour: only free queue arrays */
> > RTE_ETHDEV_LOG(DEBUG, "Port closing is using an old behaviour.\n"
> > 	"The driver %s should migrate to the new behaviour.\n",
> > /* new behaviour: send event + reset state + free all data */
> > 
> > You can find an advice in the commit:
> > 	http://git.dpdk.org/dpdk/commit/?id=23ea57a2a
> > "
> > When enabling RTE_ETH_DEV_CLOSE_REMOVE,
> > the PMD must free all its private resources for the port,
> > in its dev_close function.
> > It is advised to call the dev_close function in the remove function
> > in order to support removing a device without closing its ports.
> > "
> > 
> > It would be great to complete this migration for the next LTS
> > version, which will be 19.11.
> > It means the work should be ideally finished during the summer.
> 
> I would like to detail a little more what needs to be done, mainly for the sake
> of the discussion, please comment if something missing/wrong.
> 
> There are two exit paths for driver:
> 1) Hotplug remove the device
>    rte_dev_remove() OR rte_eal_hotplug_remove()
> 
> 2) Application exit:
>    rte_eth_dev_close()
> 
> 
> (1) can be called without ethdev stop and close functions called, so the path
> should cover those functions.
> And in the PMD entry point in this path is .remove() function. In this .remove()
> function PMD should:
> - stop forwarding
> - clean PMD private resources (dev_close() ? )

yes

> - clean ethdev generic resources (rte_eth_dev_release_port() ? )

already done in dev_close thanks to RTE_ETH_DEV_CLOSE_REMOVE

> - remove device resources, which already done by remove APIs

There can be some specific PMD private resources,
not cleaned when closing a port, to clean in "remove".

> (2) ethdev won't be usable after "rte_eth_dev_close()" and it should clear PMD
> private and ethdev generic resources.
> With RTE_ETH_DEV_CLOSE_REMOVE flag 'rte_eth_dev_close()' will free generic
> ethdev resources (rte_eth_dev_release_port()) so PMD specific '.dev_close()' should:
> - stop forwarding
> - clean PMD private resources

yes, but only port-related resources,
not those common with other ports or features of the device.

> If agreed on above, the task is:
> - '.dev_close()'
>   - stop forwarding
>   - clean PMD private resources
> 
> - '.remove()'
>   - call '.dev_close()'

Yes, looks right.

> A question is if application should call 'rte_eth_dev_stop()' explicitly before
> close or should it be part of close? For (2) it makes sense app call the stop
> explicitly, but for device remove it is not clear who to stop the forwarding,
> for this case 'dev_close()' stopping forwarding makes things easier.

"stop" is a prerequisite for "close" anyway.
In most cases, the app should manage "stop" itself to properly free
related resources.
The question is to know whether we return an error on "close"
if the port is not stopped, or we stop it implicitly.
The API says:
	 * Close a stopped Ethernet device. The device cannot be restarted!
We may explicit the behaviour if the port is not stopped.

  parent reply	other threads:[~2019-04-29 20:30 UTC|newest]

Thread overview: 22+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-04-18 10:59 Thomas Monjalon
2019-04-18 10:59 ` Thomas Monjalon
2019-04-28  6:57 ` Matan Azrad
2019-04-28  6:57   ` Matan Azrad
2019-04-28 16:17   ` Stephen Hemminger
2019-04-28 16:17     ` Stephen Hemminger
2019-04-29 16:51 ` Ferruh Yigit
2019-04-29 16:51   ` Ferruh Yigit
2019-04-29 20:30   ` Thomas Monjalon [this message]
2019-04-29 20:30     ` Thomas Monjalon
2019-04-30 12:45     ` Nithin Dabilpuram
2019-04-30 12:45       ` Nithin Dabilpuram
2019-04-30 14:12       ` Thomas Monjalon
2019-04-30 14:12         ` Thomas Monjalon
2020-08-03 18:50 ` Thomas Monjalon
2020-09-12 11:25   ` Thomas Monjalon
2020-09-13 22:16     ` Thomas Monjalon
2020-09-14  9:20       ` Rahul Lakkireddy
2020-09-22  3:04       ` Xu, Rosen
2020-09-22  7:28         ` Thomas Monjalon
2020-09-27  5:42           ` Xu, Rosen
2020-09-25  4:22       ` Rasesh Mody

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1748144.UFpUr2FPnr@xps \
    --to=thomas@monjalon.net \
    --cc=ajit.khaparde@broadcom.com \
    --cc=alejandro.lucero@netronome.com \
    --cc=allain.legacy@windriver.com \
    --cc=anatoly.burakov@intel.com \
    --cc=arybchenko@solarflare.com \
    --cc=beilei.xing@intel.com \
    --cc=bruce.richardson@intel.com \
    --cc=cernay@netcope.com \
    --cc=chas3@att.com \
    --cc=cristian.dumitrescu@intel.com \
    --cc=dev@dpdk.org \
    --cc=ed.czeck@atomicrules.com \
    --cc=evgenys@amazon.com \
    --cc=ferruh.yigit@intel.com \
    --cc=g.singh@nxp.com \
    --cc=gaetan.rivet@6wind.com \
    --cc=gtzalik@amazon.com \
    --cc=haiyangz@microsoft.com \
    --cc=hemant.agrawal@nxp.com \
    --cc=hyonkim@cisco.com \
    --cc=igor.russkikh@aquantia.com \
    --cc=jasvinder.singh@intel.com \
    --cc=jerinj@marvell.com \
    --cc=jingjing.wu@intel.com \
    --cc=john.miller@atomicrules.com \
    --cc=johndale@cisco.com \
    --cc=keith.wiles@intel.com \
    --cc=konstantin.ananyev@intel.com \
    --cc=kys@microsoft.com \
    --cc=linville@tuxdriver.com \
    --cc=lironh@marvell.com \
    --cc=matan@mellanox.com \
    --cc=matt.peters@windriver.com \
    --cc=maxime.coquelin@redhat.com \
    --cc=mczekaj@marvell.com \
    --cc=mk@semihalf.com \
    --cc=mtetsuyah@gmail.com \
    --cc=mw@semihalf.com \
    --cc=pankaj.chauhan@nxp.com \
    --cc=pavel.belous@aquantia.com \
    --cc=qi.z.zhang@intel.com \
    --cc=qiming.yang@intel.com \
    --cc=rahul.lakkireddy@chelsio.com \
    --cc=ravi1.kumar@amd.com \
    --cc=remes@netcope.com \
    --cc=rmody@marvell.com \
    --cc=shahafs@mellanox.com \
    --cc=shepard.siegel@atomicrules.com \
    --cc=shreyansh.jain@nxp.com \
    --cc=shshaikh@marvell.com \
    --cc=somnath.kotur@broadcom.com \
    --cc=srinivasan@marvell.com \
    --cc=sthemmin@microsoft.com \
    --cc=sthotton@marvell.com \
    --cc=tdu@semihalf.com \
    --cc=tiwei.bie@intel.com \
    --cc=walan@marvell.com \
    --cc=wenzhuo.lu@intel.com \
    --cc=xiao.w.wang@intel.com \
    --cc=xiaolong.ye@intel.com \
    --cc=yongwang@vmware.com \
    --cc=yskoh@mellanox.com \
    --cc=zhihong.wang@intel.com \
    --cc=zr@semihalf.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).