From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR03-VE1-obe.outbound.protection.outlook.com (mail-eopbgr50088.outbound.protection.outlook.com [40.107.5.88]) by dpdk.org (Postfix) with ESMTP id C02852C8 for ; Sun, 22 Apr 2018 13:26:55 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=Pv0Pe77AUQmY/IEabXbFtiaHqaqaFtlEOgFNmXUDHBg=; b=m6S6AKvOkX1ZuPKE80dkMrBdhHwiljNJASUsp0h0TWxWra9Vj++vlpQ8/IGxOlG9CknKkO/PdbIOLPXnNEEhuJ+uio/i8rMo4sUl+ZZlm6rf+UJZGyt+ibH33lYcGv0MGXEJD6b/gKqFI7R5FLXOaolsaXlDz4//KyLn1K/Nm8U= Received: from AM4PR0501MB2657.eurprd05.prod.outlook.com (10.172.215.19) by AM4PR0501MB1921.eurprd05.prod.outlook.com (10.167.91.17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.696.12; Sun, 22 Apr 2018 11:26:53 +0000 Received: from AM4PR0501MB2657.eurprd05.prod.outlook.com ([fe80::6885:c169:afcb:37e6]) by AM4PR0501MB2657.eurprd05.prod.outlook.com ([fe80::6885:c169:afcb:37e6%9]) with mapi id 15.20.0696.017; Sun, 22 Apr 2018 11:26:53 +0000 From: Matan Azrad To: "Ananyev, Konstantin" , "Richardson, Bruce" CC: "Burakov, Anatoly" , Thomas Monjalon , "dev@dpdk.org" , "pmatilai@redhat.com" , "david.marchand@6wind.com" , "Guo, Jia" , "stephen@networkplumber.org" , "fbl@redhat.com" Thread-Topic: kernel binding of devices + hotplug Thread-Index: AQHT00ThIlNWXyeugUC+Eesfo2Tt3qP+5c0AgAAQxACAAasfUIACcoSAgAAcrMCAAYQPgIAADrowgAAMXwCAABx6EA== Date: Sun, 22 Apr 2018 11:26:53 +0000 Message-ID: References: <2407757.yEAnF6RcS7@xps> <20180413164046.GD37024@bricha3-MOBL.ger.corp.intel.com> <20180416083153.GA50020@bricha3-MOBL.ger.corp.intel.com> <2601191342CEEE43887BDE71AB977258AE916938@IRSMSX102.ger.corp.intel.com> <2601191342CEEE43887BDE71AB977258AE916AF5@IRSMSX102.ger.corp.intel.com> In-Reply-To: <2601191342CEEE43887BDE71AB977258AE916AF5@IRSMSX102.ger.corp.intel.com> Accept-Language: en-US, he-IL Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=mellanox.com; x-originating-ip: [193.47.165.251] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; AM4PR0501MB1921; 7:1bO3JuqCfjHYY/iIICf+YnaOnRWdEIA4+JoJTMIK8lw+ga+aCbWUImmxv/f8LyoS2H/7SY6nKozmO4EWDBry3c/+UWbmIytePCgoeSOpY+uf+gyovYSSL+lbiGlU4PYSnhqrNke5XEujxlu9lAg495Y0Rt34w4XeElYWEcn0dMtQ8cSxv0953wrNPw0SL2Gswq1nFUtdP8PqVvmQaXsh1d8Pn7hB5GiYqVCziGWRBa4n4CA4eTv5v2y0GTZZTPL/ x-ms-exchange-antispam-srfa-diagnostics: SOS; x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(5600026)(4534165)(4627221)(201703031133081)(201702281549075)(48565401081)(2017052603328)(7153060)(7193020); SRVR:AM4PR0501MB1921; x-ms-traffictypediagnostic: AM4PR0501MB1921: x-ld-processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:; x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(93006095)(93001095)(3231232)(944501410)(52105095)(3002001)(10201501046)(6055026)(6041310)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123562045)(20161123564045)(20161123560045)(20161123558120)(6072148)(201708071742011); SRVR:AM4PR0501MB1921; BCL:0; PCL:0; RULEID:; SRVR:AM4PR0501MB1921; x-forefront-prvs: 0650714AAA x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(39860400002)(346002)(376002)(396003)(39380400002)(366004)(6116002)(25786009)(59450400001)(110136005)(76176011)(54906003)(6506007)(316002)(74316002)(7416002)(81166006)(93886005)(476003)(478600001)(229853002)(7696005)(2900100001)(5660300001)(102836004)(3846002)(33656002)(66066001)(11346002)(9686003)(55016002)(86362001)(4326008)(53936002)(8676002)(6436002)(6246003)(446003)(3280700002)(3660700001)(8936002)(186003)(305945005)(26005)(5890100001)(2906002)(5250100002)(7736002); DIR:OUT; SFP:1101; SCL:1; SRVR:AM4PR0501MB1921; H:AM4PR0501MB2657.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; MLV:sfv; x-microsoft-antispam-message-info: hfhJOD+jxullO6JiGVxLVVrvLwrG0HS6zipW9gGoW62nsWZJHkd1tSlIEGTixFzoq1ykhdZp2SrE+3TnigToyjIlbbi/zg0jq86F5BNP3jwPY4ySoRuFcYZoPOlNKxTy3UdnSvsQyqZcgMqMY/LL1vv3ujeEiDuPuTMVYSZReJ0MDZe24u/lEcdWCNO8t5BS spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Office365-Filtering-Correlation-Id: 6a0da60f-4b7d-43c9-bb11-08d5a843f32a X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 6a0da60f-4b7d-43c9-bb11-08d5a843f32a X-MS-Exchange-CrossTenant-originalarrivaltime: 22 Apr 2018 11:26:53.8031 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM4PR0501MB1921 Subject: Re: [dpdk-dev] kernel binding of devices + hotplug X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Apr 2018 11:26:56 -0000 Hi Konstantin From: Ananyev, Konstantin, Tuesday, April 17, 2018 2:00 PM > Hi Matan, > > Hi Konstantin > > From: Ananyev, Konstantin, Tuesday, April 17, 2018 12:23 PM > > > > Actually you say that the whitelist\blacklist mechanism is not > > > > good enough > > > and the binding workarounds it. > > > > The user need to specify somehow the devices it want to run, I > > > > think that specifying the device you want by -w option (no need to > > > > specify what you don't want in -w case) is really simpler and > > > > more descriptive than > > > binding each device you want by prior process to its correct driver. > > > > > > But what if user changes his mind and decides to give particular > > > device back to the kernel? > > > Should he restart the dpdk application? In some cases it might be > > > not desirable. > > > > And what is the behavior now in this case? >=20 > Now we don't have hotplug support at all, so nothing to worry about, I gu= ess > :) Looks like we are going to support it soon (Guo series have started it) :) We have already fail-safe driver to support hot-plug, and a real use case t= o manage Azure dpdk system. I think we must take it into account when we discuss on binding.=20 > > Looks like if we want to solve it we need to add mechanism to stop thes= e > particular devices DPDK management in any case. >=20 > I am not talking about managing a device itself (start/stop/attach/detach= ). > Let say you start dpdk with '-w dev -w dev2'. > dev1 is active, traffic is going through it, etc., while dev2 is unplugge= d. > Now user is about to plug dev2, though just before that he changed his mi= nd > and decided to give that device to kernel (or different dpdk app), ideall= y > without re-starting the app. > With just command-line support there is no option to do that. > Same for opposite case - you started dpdk app with '-w dev1' and then > decided that you also want that app to manage hot-pluggable dev2 too. And what if the user wants to switch the management from a dpdk application= to other user-space\dpdk application? Here the binding method doesn't work. Looks like you are suggesting a new feature regardless binding.=20 > > > > > It also allows better usability across systems, since the same > > > > > commandline can be used on multiple systems with different > > > > > hardware, with the actual device management rules having been > > > > > already configured at system install/setup time in udev. > > > > > > > > But the user still needs to configure the udev per device for each > > > > system, I > > > think that command line is better. > > > > > > > > > > Regarding the conflict of system rules for a device, it is > > > > > > again the user > > > > > responsibility, whatever we will decide for the binding > > > > > procedure of DPDK application the user needs to take it into > > > > > account and to solve such like conflicts. > > > > > > One option is to remove any binding rules of a DPDK device in > > > > > > the DPDK > > > > > application initialization and adjust the new rules by the PMDs, > > > > > then any conflict should not disturb the user. > > > > > > > > > > If the device management is only managed in one place, i.e. not > > > > > in DPDK, then there is no conflict to manage. > > > > > > > > I can't agree with this statement, The essence of DPDK is to give > > > > a good alternative to managing network devices, DPDK actually > > > > takes a lot of management area to manage by itself to do the user > > > > life better :) > > > > > > From my point - it is not about managing particular device. > > > It is about making decision who (kenel/dpdk) will manage that device. > > > > Doesn't the w\b mechanism comes to solve it? >=20 > It does till some extent, but I don't think it is the best possible way. Why? pros and cons against alternative ways... > > > > > From usability perspective it seems to me that better to keep it in > > > one place for all devices. > > > So udev (or any other sysadmin tool) seems like a right choice here. > > > > Please explain why? > > And if you are talking about 1 place: > > Why not to do the binding in the same place where we define which devic= e > to run? >=20 > Not all devices are always managed by dpdk apps. > Some of them will still be managed by kernel. > Again there could be several independent dpdk apps running on the same > system. > Obviously dpdk app itself can't be a centric place to handle all that var= ieties. You can use whitelist way to specify the devices for each app and that's it= . The binding to UIO driver cannot choose the specific user-space\dpdk applic= ation for the device management. > Again - if there exists a system tool that provides that functionality - = why not > to use it? Use it by dpdk or any other user-space application needs to manage devices.