From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR01-VE1-obe.outbound.protection.outlook.com (mail-ve1eur01on0086.outbound.protection.outlook.com [104.47.1.86]) by dpdk.org (Postfix) with ESMTP id 1E3A7FFA for ; Sat, 20 Jan 2018 19:14:14 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=ZNh+JwVSRd8Mbu+r1RBWBvzwfrjF/x+EQHhC5VnpFcg=; b=K5Q5g6oC4k7fTfbFkNXreMoeBMwUSqYu3euZY2p7Kvx78byLnvK1MywVTA4SOEcQ+7+crLWTSC3tqTeNFE9tL1mXtw8xsSHBeGGm6D+f7KWx1Wbbv9/KmnPbX6WR998b83EZm2RaL5pRgYVKCH7fo0ZZtnC6k3p5nOybNPzbpQM= Received: from AM6PR0502MB3797.eurprd05.prod.outlook.com (52.133.21.26) by AM6PR0502MB4088.eurprd05.prod.outlook.com (52.133.31.11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.428.17; Sat, 20 Jan 2018 18:14:13 +0000 Received: from AM6PR0502MB3797.eurprd05.prod.outlook.com ([fe80::6c28:c6b3:de94:a733]) by AM6PR0502MB3797.eurprd05.prod.outlook.com ([fe80::6c28:c6b3:de94:a733%13]) with mapi id 15.20.0428.019; Sat, 20 Jan 2018 18:14:13 +0000 From: Matan Azrad To: =?iso-8859-1?Q?Ga=EBtan_Rivet?= CC: "Ananyev, Konstantin" , Thomas Monjalon , "Wu, Jingjing" , "dev@dpdk.org" , Neil Horman , "Richardson, Bruce" Thread-Topic: [PATCH v3 7/7] app/testpmd: adjust ethdev port ownership Thread-Index: AQHTkSJSs9wv4iQCB0Cv+2LP7vySOKN7IsfQgAAIKACAAAHXQIAAHWSAgAG5pVA= Date: Sat, 20 Jan 2018 18:14:13 +0000 Message-ID: References: <1515318351-4756-1-git-send-email-matan@mellanox.com> <1516293317-30748-1-git-send-email-matan@mellanox.com> <1516293317-30748-8-git-send-email-matan@mellanox.com> <2601191342CEEE43887BDE71AB97725886280A68@irsmsx105.ger.corp.intel.com> <2601191342CEEE43887BDE71AB97725886280AE8@irsmsx105.ger.corp.intel.com> <20180119150017.mljpcdmldqx32mkq@bidouze.vm.6wind.com> In-Reply-To: <20180119150017.mljpcdmldqx32mkq@bidouze.vm.6wind.com> Accept-Language: en-US, he-IL Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=matan@mellanox.com; x-originating-ip: [85.64.136.190] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; AM6PR0502MB4088; 7:S2/KV6SStldxRFIbRD+lS/JHFf8XnqawUnEU8oF2OKj+qTGTsZtDEdGnXFdkuP4mNUCTIelEzmr3BtVBVV7beqMgttxpuUAh5lcFNQsOXwTvAEJWog+9+nTxQiCNd3ZUsW91IyniWTIv1nBUGiZfOZ/kzkf7wIs6b9Gd/03aB0bnhDOcLSNewUT7f6BzJ1G2tmWv7TTMKl9qAXQh1pLNGtlSsl1yhEcxdUyDoelUFl8WFtF/sgsrTEMiwc3W2ysQ x-ms-exchange-antispam-srfa-diagnostics: SSOS; x-ms-office365-filtering-correlation-id: 2a2b8d18-0df4-4cc7-0514-08d560319c14 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(5600026)(4604075)(3008032)(48565401081)(2017052603307)(7153060)(7193020); SRVR:AM6PR0502MB4088; x-ms-traffictypediagnostic: AM6PR0502MB4088: x-ld-processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(60795455431006)(278428928389397)(228905959029699); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040501)(2401047)(5005006)(8121501046)(3231023)(2400081)(944501161)(3002001)(10201501046)(93006095)(93001095)(6055026)(6041288)(20161123560045)(20161123564045)(20161123558120)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123562045)(6072148)(201708071742011); SRVR:AM6PR0502MB4088; BCL:0; PCL:0; RULEID:(100000803101)(100110400095); SRVR:AM6PR0502MB4088; x-forefront-prvs: 0558D3C5AC x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(346002)(376002)(366004)(39380400002)(39860400002)(396003)(57704003)(13464003)(189003)(199004)(51444003)(53546011)(59450400001)(6506007)(6916009)(3280700002)(102836004)(26005)(305945005)(3660700001)(6246003)(7696005)(5660300001)(4326008)(25786009)(33656002)(105586002)(7736002)(74316002)(9686003)(2950100002)(76176011)(106356001)(6436002)(55016002)(229853002)(53936002)(2900100001)(8936002)(8676002)(81166006)(81156014)(5250100002)(316002)(14454004)(68736007)(478600001)(6116002)(3846002)(97736004)(99286004)(93886005)(86362001)(54906003)(66066001)(2906002); DIR:OUT; SFP:1101; SCL:1; SRVR:AM6PR0502MB4088; H:AM6PR0502MB3797.eurprd05.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: KlyG0Ka2RcrbuArmJJFV4MrLWKRT9oHphJhilhx76A4Y1h0POU5GD1E68+nmdYlz+/sJQnlN2UGo4m742rm84A== spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2a2b8d18-0df4-4cc7-0514-08d560319c14 X-MS-Exchange-CrossTenant-originalarrivaltime: 20 Jan 2018 18:14:13.0546 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR0502MB4088 Subject: Re: [dpdk-dev] [PATCH v3 7/7] app/testpmd: adjust ethdev port ownership X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Jan 2018 18:14:15 -0000 Hi Gaetan From: Ga=EBtan Rivet, Friday, January 19, 2018 5:00 PM > Hi Matan, >=20 > On Fri, Jan 19, 2018 at 01:35:10PM +0000, Matan Azrad wrote: > > Hi Konstantin > > > > From: Ananyev, Konstantin, Friday, January 19, 2018 3:09 PM > > > > -----Original Message----- > > > > From: Matan Azrad [mailto:matan@mellanox.com] > > > > Sent: Friday, January 19, 2018 12:52 PM > > > > To: Ananyev, Konstantin ; Thomas > > > > Monjalon ; Gaetan Rivet > > > ; > > > > Wu, Jingjing > > > > Cc: dev@dpdk.org; Neil Horman ; > Richardson, > > > > Bruce > > > > Subject: RE: [PATCH v3 7/7] app/testpmd: adjust ethdev port > > > > ownership > > > > > > > > Hi Konstantin > > > > > > > > From: Ananyev, Konstantin, Friday, January 19, 2018 2:38 PM > > > > > To: Matan Azrad ; Thomas Monjalon > > > > > ; Gaetan Rivet ; > > > Wu, > > > > > Jingjing > > > > > Cc: dev@dpdk.org; Neil Horman ; > > > > > Richardson, Bruce > > > > > Subject: RE: [PATCH v3 7/7] app/testpmd: adjust ethdev port > > > > > ownership > > > > > > > > > > Hi Matan, > > > > > > > > > > > -----Original Message----- > > > > > > From: Matan Azrad [mailto:matan@mellanox.com] > > > > > > Sent: Thursday, January 18, 2018 4:35 PM > > > > > > To: Thomas Monjalon ; Gaetan Rivet > > > > > > ; Wu, Jingjing > > > > > > Cc: dev@dpdk.org; Neil Horman ; > > > Richardson, > > > > > > Bruce ; Ananyev, Konstantin > > > > > > > > > > > > Subject: [PATCH v3 7/7] app/testpmd: adjust ethdev port > > > > > > ownership > > > > > > > > > > > > Testpmd should not use ethdev ports which are managed by other > > > > > > DPDK entities. > > > > > > > > > > > > Set Testpmd ownership to each port which is not used by other > > > > > > entity and prevent any usage of ethdev ports which are not > > > > > > owned by > > > Testpmd. > > > > > > > > > > > > Signed-off-by: Matan Azrad > > > > > > --- > > > > > > app/test-pmd/cmdline.c | 89 +++++++++++++++++++----------= --- > ---- > > > ---- > > > > > ----- > > > > > > app/test-pmd/cmdline_flow.c | 2 +- > > > > > > app/test-pmd/config.c | 37 ++++++++++--------- > > > > > > app/test-pmd/parameters.c | 4 +- > > > > > > app/test-pmd/testpmd.c | 63 ++++++++++++++++++++---------= - > -- > > > > > > app/test-pmd/testpmd.h | 3 ++ > > > > > > 6 files changed, 103 insertions(+), 95 deletions(-) > > > > > > > > > > > > diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c > > > > > > index > > > > > > 31919ba..6199c64 100644 > > > > > > --- a/app/test-pmd/cmdline.c > > > > > > +++ b/app/test-pmd/cmdline.c > > > > > > @@ -1394,7 +1394,7 @@ struct cmd_config_speed_all { > > > > > > &link_speed) < 0) > > > > > > return; > > > > > > > > > > > > - RTE_ETH_FOREACH_DEV(pid) { > > > > > > + RTE_ETH_FOREACH_DEV_OWNED_BY(pid, my_owner.id) { > > > > > > > > > > Why do we need all these changes? > > > > > As I understand you changed definition of RTE_ETH_FOREACH_DEV(), > > > > > so no testpmd should work ok default (no_owner case). > > > > > Am I missing something here? > > > > > > > > Now, After Gaetan suggestion RTE_ETH_FOREACH_DEV(pid) will iterate > > > over all valid and ownerless ports. > > > > > > Yes. > > > > > > > Here Testpmd wants to iterate over its owned ports. > > > > > > Why? Why it can't just iterate over all valid and ownerless ports? > > > As I understand it would be enough to fix current problems and would > > > allow us to avoid any changes in testmpd (which I think is a good thi= ng). > > > > Yes, I understand that this big change is very daunted, But I think the > current a lot of bugs in testpmd(regarding port ownership) even more > daunted. > > > > Look, > > Testpmd initiates some of its internal databases depends on specific > > port iteration, In some time someone may take ownership of Testpmd > ports and testpmd will continue to touch them. > > >=20 > If I look back on the fail-safe, its sole purpose is to have seamless hot= plug > with existing applications. >=20 Yes. > Port ownership is a genericization of some functions introduced by the fa= il- > safe, that could structure DPDK further. Not only. Port ownership is a new concept saying that not all the ports are only for = the application and defines well the new port usage synchronization rules. It can be a solution for failsafe scenario, but it solves a big generic pro= blem regardless fail-safe. =20 > It should allow applications to have a seamless integration with subsyste= ms using port ownership. Without this, port ownership cannot be used. I do not think it is accurate. We can use different solution to solve the fail-safe case (seamless) by usi= ng the DEFFERED state as you did. Port ownership is not only for failsafe case - it is a generic new concept = which BTW can fix the fail-safe case(full fix). So, application should use port ownership regardless the failsafe using, ju= st to be sure no one touch its ports. > Testpmd should be fixed, but follow the most common design patterns of > DPDK applications. Going with port ownership seems like a paradigm shift. I think this patch is a classic fix for it and for the full generic issue. Do you have simpler fix to the races? > > In addition > > Using the old iterator in some places in testpmd will cause a race for = run- > time new ports(can be created by failsafe or any hotplug code): > > - testpmd finds an ownerless port(just now created) by the old > > iterator and start traffic there, > > - failsafe takes ownership of this new port and start traffic there. > > Problem! >=20 > Testpmd does not handle detection of new port. If it did, testing fail-sa= fe > with it would be wrong. It used the old iterator everywhere. So it see(and uses) all the valid ports all the time. As the new concept - it should be changed to use its owned ports, It is a simple classic solution just to use the new iterator. > At startup, RTE_ETH_FOREACH_DEV already fixed the issue of registering > DEFERRED ports. There are still remaining issues regarding this, but I th= ink > they should be fixed. The architecture does not need to be completely > moved to port ownership. I think port ownership fixes the issues nicely, don't you think so? > If anything, this should serve as a test for your API with common applica= tions. > I think you'd prefer to know and debug with testpmd instead of firing up = VPP > or something like that to determine what went wrong with using the fail- > safe. > > Yes as your examples in docs. > > In addition > > As a good example for well-done application (free from ownership bugs) = I > tried here to adjust Tespmd to the new rules and BTW to fix a lot of bugs= . >=20 > Testpmd has too much cruft, it won't ever be a good example of a well-don= e > application. >=20 > If you want to demonstrate ownership, I think you should start an example > application demonstrating race conditions and their mitigation. >=20 > I think that would be interesting for many DPDK users. I will think about that regardless the testpmd adjustment (need to find tim= e :)) > > > > So actually applications which are not aware to the port ownership stil= l are > exposed to races, but if there use the old iterator(with the new change) = the > amount of races decreases. > > > > Thanks, Matan. > > > Konstantin > > > > > > > > > > > I added to Testpmd ability to take an ownership of ports as the > > > > new ownership and synchronization rules suggested, Since Tespmd is > > > > a DPDK entity which wants that no one will touch its owned ports, > > > > It must allocate > > > an unique ID, set owner for its ports (see in main function) and > > > recognizes them by its owner ID. > > > > > > > > > Konstantin > > > > > >=20 > Regards, > -- > Ga=EBtan Rivet > 6WIND