From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM03-BY2-obe.outbound.protection.outlook.com (mail-by2nam03on0057.outbound.protection.outlook.com [104.47.42.57]) by dpdk.org (Postfix) with ESMTP id E901B2A5D for ; Mon, 23 Oct 2017 19:52:04 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=gKRmOsxK5WpZGAjT9UlEB95E2yngvnlhqNJXts2f1vI=; b=DOsBCbNJc1Muedsf6Iovu371VVKFm+9FQqrUDOIvz0DIhwdYv8NBeW2ZoPcCOs35UxaZ9QH3Kqmdij+3vNhPWlHUcAgVPIFWoaS2mntIl5K/s6kAHXSjoqibZ2IsO14gYouX66rQXr4Wq5+Yb3i4DDHvdXDCNK/cSRU/2Z/Pf38= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Pavan.Bhagavatula@cavium.com; Received: from PBHAGAVATULA-LT (103.16.71.47) by CY4PR07MB3461.namprd07.prod.outlook.com (10.171.252.142) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.156.4; Mon, 23 Oct 2017 17:52:02 +0000 Date: Mon, 23 Oct 2017 23:21:46 +0530 From: Pavan Nikhilesh Bhagavatula To: "Van Haaren, Harry" Cc: dev@dpdk.org Message-ID: <20171023175145.GA27959@PBHAGAVATULA-LT> References: <1507712990-13064-1-git-send-email-pbhagavatula@caviumnetworks.com> <1507912610-14409-1-git-send-email-pbhagavatula@caviumnetworks.com> <1507912610-14409-5-git-send-email-pbhagavatula@caviumnetworks.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.24 (2015-08-30) X-Originating-IP: [103.16.71.47] X-ClientProxiedBy: DB6PR07CA0204.eurprd07.prod.outlook.com (10.166.153.34) To CY4PR07MB3461.namprd07.prod.outlook.com (10.171.252.142) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: d34c1613-a21e-417a-bc24-08d51a3ec4aa X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(4534020)(4602075)(4627075)(201703031133081)(201702281549075)(2017052603199); SRVR:CY4PR07MB3461; X-Microsoft-Exchange-Diagnostics: 1; CY4PR07MB3461; 3:e6kpzbegxmQ0tFughmKYR90i6HMzjvVDsKGdM6frC/BGsizLscDEpXKJseBrw8V9GKIfG5eS4shQykoGNqvvBpUv5qbZ2aHa6elC75YBca7pGbvMrpISN68t3jg2MC7N+LDgiQS6CsY5ojK8mlFevhPUQw8C56PdjnKYYpmBap5DdCbVxRYnn2VyjNtYxXQEDETYwR/inbayRKJ2RLniukTnuLQ9rHAZmE0ikqHJNDo6kO8X4C0k2Iu+35daNkXc; 25:OaDsuQKUzx+pWdtip2ruAKYqyZ0z5fnZS8yg9Wb85kgtlv9tW/H1VfxcevIvgzpWCa8tNRIEwlSP8QsVD1XD1XbNVBI9IolMD9KgLNcfJLfD3uTP2OZSsLJciX3Z9d+1L2/cctKxTzw+CHaXxU4sMz4iJTCK8iGNHULymEXxnE3q8armE7sHiG4ksEohEu6ovhx7ptKP2a6rDGvyHfZYlInn5w/pfcARqrjYFix5HpNY2dGX1waY2rHTo7Pv4q4+eYIgKWTwAXTZ6yJI0HfExOPkThu7tOKV7z6YXVq1ELKcXfUS/YL7jT7+MVYBcKS5UNbkPBt/fXPyvXEA3uoV/g==; 31:hOH087j2kljAR0odliziqgFzhsWZEMfBeqqGaVhXql0Ympb8vOUt+j48DHtJt+AAOppcYOBgwd4RxN7cicFPB6FNcE/Y1S+VI1gDi4jAaXwuzlipT7oybsMS5VndwU3tKEw8k6fXniLp6WLtIgKBc25RlhebAlvJmtQe9ta3XN6FOf4f1yivwNA2sMYLOml2YgQoYPLyF/Rv5bn6t7x48G/F9UI/ZPWdiookAm7gntU= X-MS-TrafficTypeDiagnostic: CY4PR07MB3461: X-Microsoft-Exchange-Diagnostics: 1; CY4PR07MB3461; 20:EpfW2DmU3CHSS3R90vOHyBRxLb3/i3E6bCZRhVPGHzy6d5eNOKINSWbvsJXiFlZ44pqWvOgZfOCiJxuHyouX4t9eOvsLeCKKI64EA1orGTfxKnkZ3lswVhPt2r0SqT1J9cWQ0N1zZ2EauP+TbHEHzSSVrDfqxiSzHBC0Kd613kcLyqg+PgKMu4c4tTpCmDT+R6vLOipXY/hG94WIikthrT9Tcv/LPDeyVSbR6wYNZL01FZLaKX6dSnKYqrtPKkGXKdoG3WxgW8u6suUd2TYGWYuYTST6eDAtd6r6WWClT3+azaFbI3B/mYYCmAcKLbJFYcMrb5R0m/WRrFH8lpT05VX30o8vPuegenCmeTWNmNeAHfRmOdH5zquz6DPXEr4m/B6pZ2tGTZCsoNYmhBFu3l7ysy5Odi+29ZqkB+NAW+u6OQWB2THDiiAolB66Z+z+CUoG2J0znPg30C1LLtXkkGMQ1H5UrOlkT0GlFYyi/VNZ4NNWk642/lwabqQs5bqtotbMBBMC5NGQ5WukB6in3X4WhJTyRfjaghZmmyNNN0GohO9a5hScjXv0tHa8ZH/6eA/aYG086TDKt4YyLjmsNecrDEJp72P9uWtj6+BykBA= X-Exchange-Antispam-Report-Test: UriScan:(185117386973197)(228905959029699); X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(5005006)(8121501046)(93006095)(100000703101)(100105400095)(3002001)(10201501046)(3231020)(6041248)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123562025)(20161123560025)(20161123564025)(20161123558100)(20161123555025)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:CY4PR07MB3461; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:CY4PR07MB3461; X-Microsoft-Exchange-Diagnostics: 1; CY4PR07MB3461; 4:Ol45/WQNSgI7R99CEMGY174P0Br6zrRG0gCtD9pz3IVLfVQydCxtxM7/lCJAqPKrdxIAQe2fFwrvjfAY1BXUZ3FJTKVmuggDn1BFpf4/ErUQXB/anPtKOCNISgiOG1k0/1js/oPIKWugJhsMtuzJd5821ZToOrD62rZphCafoX8WKPdlkQUWxUFGK7UeDqCD4uZsCjZ37yCMZHWV+jN1BcI1lB4Un85dWZiJW/zBY9zv4WXF7YpcSFKUc+hzqRfe1poEuTmw8kvfNpCPG8mE7x1m+yNPJvLOF5Hr/MGeHbN8I6GWDmaYhBmM7ja/RFnSlSg0GYiT7ytwa5pZdM65UA== X-Forefront-PRVS: 046985391D X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(6009001)(376002)(346002)(189002)(199003)(24454002)(8676002)(53546010)(81156014)(7736002)(305945005)(81166006)(8936002)(189998001)(42882006)(6666003)(2950100002)(6916009)(7110500001)(33656002)(316002)(33716001)(93886005)(97736004)(83506002)(58126008)(68736007)(4326008)(16586007)(66066001)(47776003)(6246003)(55016002)(6496005)(50466002)(2420400007)(966005)(72206003)(53376002)(9686003)(6306002)(15650500001)(25786009)(2906002)(6116002)(106356001)(53936002)(10710500007)(478600001)(3846002)(101416001)(16526018)(76176999)(5660300001)(229853002)(23726003)(54356999)(50986999)(1076002)(105586002)(18370500001)(107986001); DIR:OUT; SFP:1101; SCL:1; SRVR:CY4PR07MB3461; H:PBHAGAVATULA-LT; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; Received-SPF: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; CY4PR07MB3461; 23:LNLxCtdfJUTzgWMzu2d2uPPYMD5Gl6fxy86b7p+Vv?= =?us-ascii?Q?E/V72ORiaWRYjHSvCMTBIPSgQsdGvAUuxLpT/b8ZO/6DdxQjQJIzaXnGZzdN?= =?us-ascii?Q?oDtZ7QXFeuYgabJhl3dtOTL5GUfxrQiPtA7unea7bXsn4jvepUpedIIolgJD?= =?us-ascii?Q?lsIprzdROhu7MITnNrhhXZbPSf3ynW0y/J525OW6Lv9zcDdqLyYEvgai4Vyv?= =?us-ascii?Q?XvZEd+2ZyVQmDcMBHuDaG0Z74/HtwNtSYoPCcBUvpSOWDXZfPXR6Oqj01QVH?= =?us-ascii?Q?NNWJ6Gkn3RuAaR0fOcdNiY346aOWMaxo2633sg1KnQrCNag4iBwdwdeXlVq3?= =?us-ascii?Q?mTEPHMYX7ljmh7Mbi+V2/QaUrdKET2GMRfapXUkzH7MYfAxjGOpI1nuNS0Qd?= =?us-ascii?Q?1I3wfub4A05A8286VuuRs/mbHv0+d3Qt07i6yfmHSVdO8KEDyVdsDlUS/6KP?= =?us-ascii?Q?2mhFm1TpyL2Rb5GFPKjmZ8/67k+yplQWuXOSlBKKqV5oY523deRVtmbgeXqQ?= =?us-ascii?Q?ygZQdzoFMyHH/rGggWS1R2AHDaY9vq/CEXjiqVzRn5KGXYW6RSnPgzCqp8cw?= =?us-ascii?Q?0Qq87Bdm51tgCA+SImQP45guCjN1Ct1fsMO2fDCrXT3NONZ2JMSSjp1iqikL?= =?us-ascii?Q?hfUSkIpc8RCSb1fbxQrzNvoDLhCItW14uL1HNzIgNqcrpFnYgdCp02sgxQ1n?= =?us-ascii?Q?3H0XEwfaeq/BozUDqED4Io62ZGONO3Mb+WKIBOxZrhpr/gKBh56pArlg4KKs?= =?us-ascii?Q?AznL3uO2g5nyJITXPOdfZJSpWMhylITj150ezHwcslx3xEvS/rvHahyMVQGG?= =?us-ascii?Q?pVPm9QHYsfkMPL1EcZmrz/hShvS4c4JN54iuE1zpzXbkN/HiqWSd6K90K8K1?= =?us-ascii?Q?drmYPhLrt93srx9UQYBKXAIxAQf9kagh3c+sjxf6ocCYvfmMdbwZg6kBKpbP?= =?us-ascii?Q?EJntjpMOg4xC0e+BrdU6VLKXSnG5w0A7rKpMnAX1D1gUdCqrBrQkjuAWAYBm?= =?us-ascii?Q?d3LdR0KNwPVEz3toJgTNzACHkUpfjpXwmLXlCCRyF0JAXFjot2Q4Kpb4U2mv?= =?us-ascii?Q?DHWfFOxOyjTPJOiIXEbFS6+OxJ/0DMrs2RiiKJDLV4Ta4DSNmoNp5JzyhZKT?= =?us-ascii?Q?81O0BDW+ydshgeKc0liN0DQudp8A+u4nx3UMIxsL5oaUwLVLnuB/BNx4A4Zg?= =?us-ascii?Q?kLOFLqOnIvB7M4Dw+lpyHDVfmYkf2OqQPkv1nDAr8Ag2Fcp+cyIyVSzNfsnZ?= =?us-ascii?Q?q1KPo79yDBKVgVp/PeMnsk1vQ4rtfuJrYj0IOYAKLe9pVIEw55yiVQEOjIzi?= =?us-ascii?Q?rBpJZSgMiwsBTyZVxvZQgFqqZqVIzwD+fnMIJVJm5KbiBjlTS39BFfzMQW4b?= =?us-ascii?Q?XBvp2iV4ugWMqfvd0HxNxK94mOti0i4D8hRVuRGOUJnDP5QIRya2pWAM5xGw?= =?us-ascii?Q?l+IYjNvXA=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1; CY4PR07MB3461; 6:ExVfoTjdaqHI9ejUxS83m+f65sgjnGsrrnaNhEdytQLKBFV3YOiU6QGPEYIhM9ilQ7WoN0+nKokCLO907/HnJ37nEGD/MFCBfNDAVLesBFoEsb9c/NH4UqmJ7eL+vvn8yt4RLWLffRx935DBLtrdy6rSNAke5Ba0jLUYxTvJXFlGvC73OzMy4nW1AtLWMbwM9+p6+2ohMYLS1QWUfHuvMg2o668VIGNbY2MB93f1giFz7AF3o95IikcGdfaTw1Y9TndbKdr3PYHab2a5bvswh5opbn78pVv/OMRQ7nNzBEIyz1S6Hwdf+yWGp4UgutJqVfCqg9rvmoIHaHO3xjDMkg==; 5:9lL81m1LXRqhueFrZJldXNiULcICqzdbAtgakoi435RzoSIEXH7TcKPFdIh30iJJXl8mSvMnYf+m51zu8VvgElprNA/U1CoZ0SVck8m3M9rGrcpPYYBgbv/FhmBNoPQ1hLU4ShFjQFRwXQrszCQUEQ==; 24:P94p7nmCWCgq/YKkKxNkqqasMWduACN6bwuiZDIKsew/2wB9d1fZyHGlEvhK41PtXaaL4Ofg+9szqx+VCvRK3dKlM47J/XCx8fXazOfOQV4=; 7:RKaSpgU2azNg0TKuFqRr8l2rdyY6SQEXnFGL+3kCmpb8igr74QIxA9UfBabOHHwSKCAVhNNBIZskPb1HBK/ZslEc40Cu4u5zcHJolRuVEtrEtlbYTk/XzHYI4Nef1x7A5FmCNsnR68+ibK1NQ6QPt7nOAxsNSnXMjTx6nW/muTT+yBdjTjYfTyxx1QABHUxcafKJmGcFWNcCwRaA7bVV1+ed/wpQLbDCEBKobMdyvFs= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Oct 2017 17:52:02.0579 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: d34c1613-a21e-417a-bc24-08d51a3ec4aa X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR07MB3461 Subject: Re: [dpdk-dev] [PATCH v2 5/7] examples/eventdev: update sample app to use service X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Oct 2017 17:52:05 -0000 On Mon, Oct 23, 2017 at 05:17:48PM +0000, Van Haaren, Harry wrote: > > From: Pavan Nikhilesh [mailto:pbhagavatula@caviumnetworks.com] > > Sent: Friday, October 13, 2017 5:37 PM > > To: jerin.jacob@caviumnetworks.com; hemant.agrawal@nxp.com; Van Haaren, > > Harry > > Cc: dev@dpdk.org; Pavan Bhagavatula > > Subject: [dpdk-dev] [PATCH v2 5/7] examples/eventdev: update sample app to > > use service > > > > From: Pavan Bhagavatula > > > > Update the sample app eventdev_pipeline_sw_pmd to use service cores for > > event scheduling in case of sw eventdev. > > > > Signed-off-by: Pavan Nikhilesh > > > Comments inline - I think there are some side-effect changes in the application. > > > > --- > > examples/eventdev_pipeline_sw_pmd/main.c | 51 +++++++++++++++++++++-------- > > --- > > 1 file changed, 33 insertions(+), 18 deletions(-) > > > > diff --git a/examples/eventdev_pipeline_sw_pmd/main.c > > b/examples/eventdev_pipeline_sw_pmd/main.c > > index 09b90c3..d5068d2 100644 > > --- a/examples/eventdev_pipeline_sw_pmd/main.c > > +++ b/examples/eventdev_pipeline_sw_pmd/main.c > > @@ -46,6 +46,7 @@ > > #include > > #include > > #include > > +#include > > > > #define MAX_NUM_STAGES 8 > > #define BATCH_SIZE 16 > > @@ -233,7 +234,7 @@ producer(void) > > } > > > > static inline void > > -schedule_devices(uint8_t dev_id, unsigned int lcore_id) > > +schedule_devices(unsigned int lcore_id) > > { > > if (fdata->rx_core[lcore_id] && (fdata->rx_single || > > rte_atomic32_cmpset(&(fdata->rx_lock), 0, 1))) { > > @@ -241,16 +242,6 @@ schedule_devices(uint8_t dev_id, unsigned int lcore_id) > > rte_atomic32_clear((rte_atomic32_t *)&(fdata->rx_lock)); > > } > > > > - if (fdata->sched_core[lcore_id] && (fdata->sched_single || > > - rte_atomic32_cmpset(&(fdata->sched_lock), 0, 1))) { > > - rte_event_schedule(dev_id); > > - if (cdata.dump_dev_signal) { > > - rte_event_dev_dump(0, stdout); > > - cdata.dump_dev_signal = 0; > > - } > > - rte_atomic32_clear((rte_atomic32_t *)&(fdata->sched_lock)); > > - } > > See note below, about keeping the functionality provided by > fdata->sched_core[] intact. > > > > if (fdata->tx_core[lcore_id] && (fdata->tx_single || > > rte_atomic32_cmpset(&(fdata->tx_lock), 0, 1))) { > > consumer(); > > @@ -294,7 +285,7 @@ worker(void *arg) > > while (!fdata->done) { > > uint16_t i; > > > > - schedule_devices(dev_id, lcore_id); > > + schedule_devices(lcore_id); > > > > if (!fdata->worker_core[lcore_id]) { > > rte_pause(); > > @@ -661,6 +652,27 @@ struct port_link { > > }; > > > > static int > > +setup_scheduling_service(unsigned int lcore, uint8_t dev_id) > > +{ > > + int ret; > > + uint32_t service_id; > > + ret = rte_event_dev_service_id_get(dev_id, &service_id); > > + if (ret == -ESRCH) { > > + printf("Event device [%d] doesn't need scheduling service\n", > > + dev_id); > > + return 0; > > + } > > + if (!ret) { > > + rte_service_runstate_set(service_id, 1); > > + rte_service_lcore_add(lcore); > > + rte_service_map_lcore_set(service_id, lcore, 1); > > + rte_service_lcore_start(lcore); > > + } > > + > > + return ret; > > +} > > + > > +static int > > setup_eventdev(struct prod_data *prod_data, > > struct cons_data *cons_data, > > struct worker_data *worker_data) > > @@ -839,6 +851,14 @@ setup_eventdev(struct prod_data *prod_data, > > *cons_data = (struct cons_data){.dev_id = dev_id, > > .port_id = i }; > > > > + for (i = 0; i < MAX_NUM_CORE; i++) { > > + if (fdata->sched_core[i] > > + && setup_scheduling_service(i, dev_id)) { > > + printf("Error setting up schedulig service on %d", i); > > + return -1; > > + } > > + } > > > Previously, the fdata->sched_core[] array contained a "coremask" for scheduling. > A core running the scheduling could *also* perform other work. AKA: a single core > could perform all of RX, Sched, Worker, and TX. > > Due to the service-core requiring to "take" the full core, there is no option to > have a core "split" its work into schedule() and RX,TX,Worker. This is a service core > implementation limitation - however it should be resolved for this sample app too. > > The solution is to enable an ordinary DPDK (non-service-core) thread to run > a service. This MUST be enabled at the service-cores library level, to keep atomics > behavior of services etc), and hence removing rte_event_schedule() is still required. > > The changes should become simpler than proposed here, instead of the wait_schedule() hack, > we can just run an iteration of the SW PMD using the newly-added service core iter function. > > I have (just) sent a patch for service-cores to enable running a service on an ordinary > DPDK lcore, see here: http://dpdk.org/ml/archives/dev/2017-October/080022.html > > Hope you can rework patches 4/7 and 5/7 to use the newly provided functionality! > Let me know if the intended usage of the new function is unclear in any way. > Agreed, current solution for controlled scheduling of event_sw is bit hacky, the added flexibility of service core API helps a lot. Will rebase my patchset on top of service core patches and spin up a v4. Thanks, Pavan > > Regards, -Harry > > > > + > > if (rte_event_dev_start(dev_id) < 0) { > > printf("Error starting eventdev\n"); > > return -1; > > @@ -944,8 +964,7 @@ main(int argc, char **argv) > > > > if (!fdata->rx_core[lcore_id] && > > !fdata->worker_core[lcore_id] && > > - !fdata->tx_core[lcore_id] && > > - !fdata->sched_core[lcore_id]) > > + !fdata->tx_core[lcore_id]) > > continue; > > > > if (fdata->rx_core[lcore_id]) > > @@ -958,10 +977,6 @@ main(int argc, char **argv) > > "[%s()] lcore %d executing NIC Tx, and using eventdev > > port %u\n", > > __func__, lcore_id, cons_data.port_id); > > > > - if (fdata->sched_core[lcore_id]) > > - printf("[%s()] lcore %d executing scheduler\n", > > - __func__, lcore_id); > > - > > if (fdata->worker_core[lcore_id]) > > printf( > > "[%s()] lcore %d executing worker, using eventdev port > > %u\n", > > -- > > 2.7.4 >