From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3792CA2EF9 for ; Mon, 16 Sep 2019 04:30:24 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id A676A1C02F; Mon, 16 Sep 2019 04:30:22 +0200 (CEST) Received: from EUR04-DB3-obe.outbound.protection.outlook.com (mail-eopbgr60067.outbound.protection.outlook.com [40.107.6.67]) by dpdk.org (Postfix) with ESMTP id BE5AD1C02C for ; Mon, 16 Sep 2019 04:30:21 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BPBuKLQJJ/UnNV2pxTbhdXRKDuJ8DyImWVq9QfzOb5o=; b=4W4ALjkypoQU+YQOXrHo+c2a8zvKJBNIqtF+B9Hkhq34U+urJkEDM5MeOKCxxkyflwe/QoyFbUPHRADpCShkSNxBDTj9ue5IBONq0E/Tm/KEotqJzztVDiKM8tzeOaUdkjgZAllliQzV55p9WJZvWQEa/8MRSHJUcG1jdVLyr8o= Received: from VI1PR08CA0106.eurprd08.prod.outlook.com (2603:10a6:800:d3::32) by AM6PR08MB4950.eurprd08.prod.outlook.com (2603:10a6:20b:ea::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2263.15; Mon, 16 Sep 2019 02:30:18 +0000 Received: from AM5EUR03FT030.eop-EUR03.prod.protection.outlook.com (2a01:111:f400:7e08::205) by VI1PR08CA0106.outlook.office365.com (2603:10a6:800:d3::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2263.15 via Frontend Transport; Mon, 16 Sep 2019 02:30:17 +0000 Authentication-Results: spf=temperror (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dpdk.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dpdk.org; dmarc=none action=none header.from=arm.com; Received-SPF: TempError (protection.outlook.com: error in processing during lookup of arm.com: DNS Timeout) Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by AM5EUR03FT030.mail.protection.outlook.com (10.152.16.117) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2263.14 via Frontend Transport; Mon, 16 Sep 2019 02:30:16 +0000 Received: ("Tessian outbound 968ab6b62146:v31"); Mon, 16 Sep 2019 02:30:14 +0000 X-CR-MTA-TID: 64aa7808 Received: from 8f6f566d4a03.2 (ip-172-16-0-2.eu-west-1.compute.internal [104.47.1.55]) by 64aa7808-outbound-1.mta.getcheckrecipient.com id 5B4F5B31-8771-46C4-A3A3-8C6D33EF5549.1; Mon, 16 Sep 2019 02:30:09 +0000 Received: from EUR01-VE1-obe.outbound.protection.outlook.com (mail-ve1eur01lp2055.outbound.protection.outlook.com [104.47.1.55]) by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 8f6f566d4a03.2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384); Mon, 16 Sep 2019 02:30:09 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=CU2s/17EjQRiz8Q5I6dAon/N6phOG+70DWTzMBn94zRxekxndBBUep1MhrtD/WjksoI1EogXSGpD9mWZ/W57TsRWkA9aV3BHcKNtGBaWrILSMLBHprye61NLyHboIzgnuSz1SwSdtxlAbedVKwtepmXcVsDBX8Nbt0keqAY/mOO9VFOKMLGMRcrFfCrAWBnM/tt4gEAFrmZvcu1SZBAR6KZxis4SouROv59DljyRZDg5EUuJE5hNMFU7kWqBr3QOM9GBX/+31GH6eeaR4h2YhX//GGy0Ozo+BfNw/mUPoQpVY3EDWDhViYebyO7T4xJAiarCRxlOHaIbqFBAQ2vDnQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BPBuKLQJJ/UnNV2pxTbhdXRKDuJ8DyImWVq9QfzOb5o=; b=CP6Bbh+Rra5JAO4bCXsC4YUpTFvmeoGZbZNqRvfrA1c5NK0iYI0UjhmeCNZSSw1GQEaBcywUdtMtiqbH9tR0XE15rVqjPblTxXgyJUaOjpiOywcmjL7XzWaurXGjtfASHcZGhbsH0lYWtYk1H63Pmn51tMi5FdZXOFighCdBywlrM2D595iSliSwtozvtcHA/iNxinouM7lCD7WDfTOxiwmPTZYcjTwTHoBfrShu5zut0cWkX+fYX/iG8JyG7po3wf+iBvK5fG+4/r+ZZw8eF7ZM0+ZJpLbcB78Ms4RlXsHcVFWkTP6GwCAkslp3ekJncZXQKtKiPZ/u+R+k81w+BQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BPBuKLQJJ/UnNV2pxTbhdXRKDuJ8DyImWVq9QfzOb5o=; b=4W4ALjkypoQU+YQOXrHo+c2a8zvKJBNIqtF+B9Hkhq34U+urJkEDM5MeOKCxxkyflwe/QoyFbUPHRADpCShkSNxBDTj9ue5IBONq0E/Tm/KEotqJzztVDiKM8tzeOaUdkjgZAllliQzV55p9WJZvWQEa/8MRSHJUcG1jdVLyr8o= Received: from AM0PR08MB3986.eurprd08.prod.outlook.com (20.178.118.90) by AM0PR08MB4052.eurprd08.prod.outlook.com (20.178.119.140) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2263.20; Mon, 16 Sep 2019 02:30:07 +0000 Received: from AM0PR08MB3986.eurprd08.prod.outlook.com ([fe80::8194:f229:b0d2:b5df]) by AM0PR08MB3986.eurprd08.prod.outlook.com ([fe80::8194:f229:b0d2:b5df%7]) with mapi id 15.20.2263.023; Mon, 16 Sep 2019 02:30:07 +0000 From: "Ruifeng Wang (Arm Technology China)" To: "Ananyev, Konstantin" , Honnappa Nagarahalli , "Kantecki, Tomasz" CC: "dev@dpdk.org" , "Gavin Hu (Arm Technology China)" , nd , nd , nd , nd Thread-Topic: [dpdk-dev] [PATCH 0/2] add lock-free mode for l3fwd Thread-Index: AQHVZJ2RE3+uO/G+WUmOLRh3JRpdcqcedEgAgAQfuaCAAhDKgIAABz/wgAFQ/wCAAAtF8IAAJiGAgAd0BfA= Date: Mon, 16 Sep 2019 02:30:07 +0000 Message-ID: References: <20190906102615.36942-1-ruifeng.wang@arm.com> <2601191342CEEE43887BDE71AB977258019192657B@irsmsx105.ger.corp.intel.com> <2601191342CEEE43887BDE71AB977258019196218E@irsmsx105.ger.corp.intel.com> <2601191342CEEE43887BDE71AB9772580191962AB2@irsmsx105.ger.corp.intel.com> In-Reply-To: <2601191342CEEE43887BDE71AB9772580191962AB2@irsmsx105.ger.corp.intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: a103a4e5-52a4-40f8-b605-46130c849650.0 x-checkrecipientchecked: true Authentication-Results-Original: spf=none (sender IP is ) smtp.mailfrom=Ruifeng.Wang@arm.com; x-originating-ip: [113.29.88.7] x-ms-publictraffictype: Email X-MS-Office365-Filtering-Correlation-Id: a8b053e4-b286-4af8-5226-08d73a4dcf86 X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam-Untrusted: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(5600167)(711020)(4605104)(1401327)(4618075)(4534185)(7168020)(4627221)(201703031133081)(201702281549075)(8990200)(2017052603328)(7193020); SRVR:AM0PR08MB4052; X-MS-TrafficTypeDiagnostic: AM0PR08MB4052:|AM0PR08MB4052:|AM6PR08MB4950: X-MS-Exchange-PUrlCount: 1 x-ms-exchange-transport-forked: True X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true x-ms-oob-tlc-oobclassifiers: OLM:8273;OLM:8273; x-forefront-prvs: 0162ACCC24 X-Forefront-Antispam-Report-Untrusted: SFV:NSPM; SFS:(10009020)(4636009)(136003)(376002)(346002)(39850400004)(366004)(396003)(13464003)(189003)(199004)(64756008)(6306002)(55236004)(110136005)(76116006)(66946007)(66556008)(86362001)(486006)(7736002)(66446008)(52536014)(66476007)(74316002)(305945005)(478600001)(25786009)(71200400001)(71190400001)(5660300002)(66066001)(6506007)(316002)(53546011)(81166006)(6116002)(8676002)(14444005)(4326008)(256004)(8936002)(6246003)(81156014)(76176011)(966005)(55016002)(3846002)(26005)(476003)(14454004)(7696005)(53936002)(99286004)(9686003)(11346002)(186003)(2906002)(33656002)(229853002)(6436002)(54906003)(446003)(102836004)(32563001); DIR:OUT; SFP:1101; SCL:1; SRVR:AM0PR08MB4052; H:AM0PR08MB3986.eurprd08.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Message-Info-Original: aJbv1IxyZ1J1muHi2JWr2xjgOxX/Zm33Ms4BJNRAHxXvHiXCFzjbFEoyGXvmUlYn2rSvm+69DMjUxMRTOfI0BV4m9H0QtCyeeL7oi8UcWnTWpBSVrTSnDgB6lC4NctWD9ih/5WTfX4tUaV61CjkGWBjDCkN0NguzBeMcrBBrPYFWb3+N7Yf9Ea5fbAdQCvLWYN/U+R2ZEXPO6VVrDnxdh+A2cPZ6Ud+VlCHB/Nos6XpQvsf7h/jTVgSQ0Skt6HsJGmAoHwF8nQa1DHd+AQ+6QHs4wfqu5BOXx8ykDHQy8hhgmJCp3nAOiImPEFmxgKS3MzHoNasaOMAIainNweCtJkwvd3Zj1uV2Uy/OBrAw4pLIIPhyoDRCi6G0aCm5locXGnDv1Y9As//9QXzVCMDkQXb/9g9tIfVymKbizZuPibc= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB4052 Original-Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Ruifeng.Wang@arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM5EUR03FT030.eop-EUR03.prod.protection.outlook.com X-Forefront-Antispam-Report: CIP:63.35.35.123; IPV:CAL; SCL:-1; CTRY:IE; EFV:NLI; SFV:NSPM; SFS:(10009020)(4636009)(136003)(376002)(396003)(39850400004)(346002)(199004)(189003)(13464003)(356004)(33656002)(6116002)(3846002)(50466002)(23726003)(99286004)(52536014)(86362001)(14444005)(316002)(36906005)(9686003)(110136005)(6306002)(6246003)(70586007)(70206006)(55016002)(478600001)(76130400001)(22756006)(54906003)(14454004)(46406003)(229853002)(26826003)(97756001)(25786009)(8676002)(8936002)(8746002)(66066001)(47776003)(2906002)(81156014)(81166006)(76176011)(4326008)(966005)(5660300002)(336012)(446003)(11346002)(486006)(126002)(476003)(7696005)(305945005)(7736002)(6506007)(74316002)(53546011)(26005)(186003)(102836004)(63350400001)(32563001); DIR:OUT; SFP:1101; SCL:1; SRVR:AM6PR08MB4950; H:64aa7808-outbound-1.mta.getcheckrecipient.com; FPR:; SPF:TempError; LANG:en; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; A:1; MX:1; X-MS-Office365-Filtering-Correlation-Id-Prvs: c264dcd3-71b3-44be-76e8-08d73a4dca10 X-Microsoft-Antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(5600167)(710020)(711020)(4605104)(1401327)(4618075)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(2017052603328)(7193020); SRVR:AM6PR08MB4950; NoDisclaimer: True X-Forefront-PRVS: 0162ACCC24 X-Microsoft-Antispam-Message-Info: xil0Gd4nsuTh3giaG2+I/0q8uT93M188xd6j04JyEcmicdoZwPf2Jtzh2c4UQbtkAwJZljRjVdQ6llmFirT8/uyeGE5lgPaMu3mruPZghfFAF5RS4eH8Qu9/KiKQnv24kn1D0chTbPEon0+bQWOGwFCWG8t+tF4gGPha68is02X65NG3ePSn6PO6zue4875uk3SqS0qV5QhPpjTg7u6FN27jIVmdAXhJks2iafSexIlNoFUbdf7BUbZUOXk8J+82bWTLY7Bem5lgx2Ljzc55GNYuEsqL1bg9ITXxgNE5US87o/7mdtjUSmGUkbWowibJEvNxa1PgpgEBrEopJuTvGxLvnkZfDHX4VpQ9lj5ZfelxBip64KX3DaZ1n5TJzXNVU58Cz+8U/HK0COtonNRsZR/IaqgI8IwDFJ25arVrcnw= X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Sep 2019 02:30:16.4243 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: a8b053e4-b286-4af8-5226-08d73a4dcf86 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB4950 Subject: Re: [dpdk-dev] [PATCH 0/2] add lock-free mode for l3fwd X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > -----Original Message----- > From: Ananyev, Konstantin > Sent: Wednesday, September 11, 2019 16:35 > To: Ruifeng Wang (Arm Technology China) ; > Honnappa Nagarahalli ; Kantecki, Tomasz > > Cc: dev@dpdk.org; Gavin Hu (Arm Technology China) ; > nd ; nd ; nd > Subject: RE: [dpdk-dev] [PATCH 0/2] add lock-free mode for l3fwd >=20 >=20 >=20 > > -----Original Message----- > > From: Ruifeng Wang (Arm Technology China) > > [mailto:Ruifeng.Wang@arm.com] > > Sent: Wednesday, September 11, 2019 7:58 AM > > To: Honnappa Nagarahalli ; Ananyev, > > Konstantin ; Kantecki, Tomasz > > > > Cc: dev@dpdk.org; Gavin Hu (Arm Technology China) > ; > > nd ; nd ; nd > > Subject: RE: [dpdk-dev] [PATCH 0/2] add lock-free mode for l3fwd > > > > > -----Original Message----- > > > From: Honnappa Nagarahalli > > > Sent: Wednesday, September 11, 2019 13:38 > > > To: Ruifeng Wang (Arm Technology China) ; > > > Ananyev, Konstantin ; Kantecki, > Tomasz > > > > > > Cc: dev@dpdk.org; Gavin Hu (Arm Technology China) > > > ; Honnappa Nagarahalli > > > ; nd ; nd > > > > Subject: RE: [dpdk-dev] [PATCH 0/2] add lock-free mode for l3fwd > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Lock-free mode is supported by hash library and LPM library= . > > > > > > > > Now we add an option for l3fwd example to enable the > > > > > > > > lock-free > > > > mode. > > > > > > > > Necessary preparation steps are added to use lock-free LPM > mode. > > > > > > > > > > > > > > Can I ask about the purpose of these changes? > > > > > > > Right now in l3fwd both lpm and hash tables are static and > > > > > > > hard- > > > coded. > > > > > > > we initialize them at startup and then just do read from them= . > > > > > > > Do you plan to enhance l3fwd with ability to dynamically > > > > > > > update tables contents? > > > > > > > Though fir that we first have to get rid of hard-coded > > > > > > > values (config file or > > > > > so). > > > > > > > Konstantin > > > > > > > > > > > > > Thanks for your questions. > > > > > > Currently, we have no plan to enhance l3fwd with ability to > > > > > > dynamically > > > > > update table contents. > > > > > > Lock-free method is being integrated into Hash library and LPM > > > > > > library. Lock-free algorithms are not only about control > > > > > > plane (adding or deleting routes), they affect the data path > > > > > > performance as > > > well. > > > > > > Since l3fwd application is showcasing data path performance, > > > > > > we need to show the impact of including the quiescent state > > > > > > reporting on data > > > > path. > > > > > > This change also serves as an example of using the RCU APIs. > > > > > > > > > > > > > > > But what you suggest doesn't provide the complete picture. > > > > > With dynamic updates in place (via control path) the data-path > > > > > impact might be completely different then without. > > > > > Again without dynamic updates how can you test that your > > > > > data-path > > > > > lock- free approach does work as expected? > > > > > Also it can't even be used as a reference implementation for > > > > > users, as half of the functionality they need to implement is sim= ply > missing. > > > > > My opinion - we either need to leave l3fwd as it is (static > > > > > routes), or implement a proper control path with ability to > > > > > dynamically update routes before starting to introduce some > > > > > synchronization schemes (RCU or whatever). > > > > > > > > > > Konstantin > > > > > > > > > > > > > Agree that dynamic control path updates should be included for a > > > > whole picture. > > > > I will add dynamic update to l3fwd and reroll the patch series. > > > > Thanks. > > > I think we should have an agreement on what exactly we mean by > > > 'dynamically update routes'. > > > IMO, we should not disturb the existing static routes as there might > > > be automated tests running in the labs. I suggest that we should > > > add/delete new routes/hash entries which are different from the > > > existing routes/hash entries. This should be sufficient to showcase > > > the functionality as well as measure the impact. > > > > > Yes, existing static routes should be kept intact. > > To perform regular route/hash entries add/delete, a dedicated lcore wil= l be > needed. > > An interactive prompt is not an option since we need automatic add/dele= te. > > We can skip master core for data path main loop. And perform unrelated > route/hash entries add/delete regularly on master core. > > The impact is that command lines used in tests will need update since > master core will no longer do data path work. >=20 > Not sure why it has to be master core? > Why interrupt thread wouldn't do? > I think what we need to: > 1. introduce reading routes from config file instead of having them hard- > coded within the app. > 2. add ability to update routes dynamically. > Probably the easiest (and commonly used way) re-read conf file and > update routes on the signal (SIGUSR1 or so). > Konstantin >=20 >=20 Sorry for delayed response. Just back from vacation.=20 Thanks for your suggestion. I will try the config file based updating approach and get back with new ve= rsion. > > > > > > > > > > > > > > > > > > > > > > > > > > Patch 2/2 has dependency on RCU QSBR integration with LPM > > > library: > > > > > > > > http://patches.dpdk.org/project/dpdk/list/?series=3D6288 > > > > > > > > > > > > > > > > > > > > > > > > Ruifeng Wang (2): > > > > > > > > examples/l3fwd: add lock-free option for l3fwd > > > > > > > > examples/l3fwd: integrate RCU QSBR for LPM mode > > > > > > > > > > > > > > > > doc/guides/sample_app_ug/l3_forward.rst | 3 ++ > > > > > > > > examples/l3fwd/Makefile | 1 + > > > > > > > > examples/l3fwd/l3fwd.h | 4 +- > > > > > > > > examples/l3fwd/l3fwd_em.c | 10 +++- > > > > > > > > examples/l3fwd/l3fwd_lpm.c | 72 > > > > > +++++++++++++++++++++++-- > > > > > > > > examples/l3fwd/main.c | 27 ++++++++-- > > > > > > > > examples/l3fwd/meson.build | 1 + > > > > > > > > 7 files changed, 108 insertions(+), 10 deletions(-) > > > > > > > > > > > > > > > > -- > > > > > > > > 2.17.1 > > > > > > >