From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM03-DM3-obe.outbound.protection.outlook.com (mail-dm3nam03on0062.outbound.protection.outlook.com [104.47.41.62]) by dpdk.org (Postfix) with ESMTP id 9602B1B010 for ; Wed, 10 Jan 2018 21:17:35 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=MJ30svNKPDEOTvPn7/O5ANYPJJun7KdGZ6zhEDd8evE=; b=CGvTXo5JqPICOnKtg144pEmokQmPuo/o9iInNR1QlWxkr9ZJzWwK+xI7DtxmU1TamDJFTlLttHYinCqu101mllgIijMCfMNoWsmGWP2XGpz3LLFPUubiU1DlUmObs//9lvDReOWw/Qnk2PTcnyuGvfr6BL6YI4wkL7/wcEEYDdc= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Pavan.Bhagavatula@cavium.com; Received: from Pavan-LT (103.16.71.47) by DM5PR07MB3467.namprd07.prod.outlook.com (10.164.153.22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.386.5; Wed, 10 Jan 2018 20:17:30 +0000 Date: Thu, 11 Jan 2018 01:47:10 +0530 From: Pavan Nikhilesh To: "Van Haaren, Harry" , "jerin.jacob@caviumnetworks.com" , "santosh.shukla@caviumnetworks.com" , "Eads, Gage" , "hemant.agrawal@nxp.com" , "nipun.gupta@nxp.com" , "Ma, Liang J" Cc: dev@dpdk.org Message-ID: <20180110201710.3uolm2hwzwcowoif@Pavan-LT> References: <20171130072406.15605-1-pbhagavatula@caviumnetworks.com> <20180110145144.28403-1-pbhagavatula@caviumnetworks.com> <20180110145144.28403-9-pbhagavatula@caviumnetworks.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: NeoMutt/20170609 (1.8.3) X-Originating-IP: [103.16.71.47] X-ClientProxiedBy: PS1PR0601CA0106.apcprd06.prod.outlook.com (10.170.176.160) To DM5PR07MB3467.namprd07.prod.outlook.com (10.164.153.22) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: e5dcaed2-dce4-4238-378e-08d558672fb3 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(4534020)(4602075)(4627115)(201703031133081)(201702281549075)(5600026)(4604075)(2017052603307)(7153060)(7193020); SRVR:DM5PR07MB3467; X-Microsoft-Exchange-Diagnostics: 1; DM5PR07MB3467; 3:pGSkbBqWtINDHxZtWs378U+FI7PW3dze1G528LEkDNIsVQF2KbO2RbLCxMPrR5m3cH72HqbrRONiGmAqcs06I2jE9lveC11hkjeZYXKnJvkYvJfBkyhenVFR6gMG+dQxDCvzV6XPymKBiFuHmFEjE6cOQCa/Qx8m5cijCgtEqEcEpuCrM7YU0h/n7jgRNQ4wMxWVDqd9g7ao4IykmPCpXV/2DKeZjSleCSXAotZh3GCNuVo8GZFK3oLvHA5PyLP7; 25:4ufDL4e38E8bhe4nPwD7177XP+ki2NQfrimIunZxzGZVGR62R45BJaQJ4hLDnkxYnESdQidlmBcWOwjXWbKRxq1bgCzWP5GeLNQANVb4mHgeyJVkc/W9inREnz5YLE2O+YvUbDMQlDL7jjzNoQwiliyCwfv/8yROGSEOrF6FnfFZ+e6bLM5mSa3GRGxJxm89ME24y+eyBz/rfAdJPnohnX0Xm23wfmMdTjYsOxeU8dJZDccEAkbdZ+hJGVxwSuTTjUocagRr/ni6RSrgkZPew1/X/AFyHsfEx6aXyfKfNvx6r2vBzZFgwBphGy9krm+a9fKpREIWAnYovdimEMQf0g==; 31:+P1DLLqETl3/gsnvgE8DIWCgiq5yM6NV19GmbVfC1dcrlUeCtOU2zdga/OAX0Lu1CNPGwfI8uItWggMiJvTL7dvjWs+N2f3yu5WY9VBq8QPeqlJdSLNROxIOet390eOImjnwHE0xxjcMGygD0hfsYThutNzjwfq5GN09U6ctHDaQdPhDB2lsfcKDnoAkDdMdpWvROjsqfuhEWXjM5yv3U34NJQ4O3vYQIczGJ/FR7NU= X-MS-TrafficTypeDiagnostic: DM5PR07MB3467: X-Microsoft-Exchange-Diagnostics: 1; DM5PR07MB3467; 20:NYqnEdpzdibWH2bhhgJH8aKj5lhhjmCyxlXNxYrc9mNJgX1dSHCKKB867AFWqAI5acC/BvFnOSEH+WmzYuzLVt5WIGGt2pljAj5PawkSwJKJf+KRFyiWKVvf5MZfxfow1w1QqRI1GN6vh0vWbPM3hXyafweQHDhBzLvobkZbFeEbmrVwMp2esmbJbpQYXmc3WBR+CBgesBRtuikBc6xwNh/TyE6j/S/OVkGS6hxl22tG1nBCVafMi4L5HNBZlqdcOOFUd+ukcO+Mvl9ajleC6eTik9IUtI6AvPAvzeiA8x0ZSzhj39fCjjXaXYDqnasTvE4A0Y6OoqCTOl+2SbJ72eA4uvL5uP+tRCDh5v4Ki4gVFEwtT2opiEaplM5xacr4VEcHampqRnF1+IzmFACrj0qdraotGCzRVoCxjkaDLkRvhtkmTISYVtXC8RMEp8+Ykvv9PSDP+AmUj4Mt47eorbffFXp/8Z1nki7KAeoJuwrWKQKqRAGdulBv6aeiPFBG65GiBPfObaQr6f3Xa65QS6Jd6YdM7sx2MqiXwMM8c/ewqzfIpS+EZc+J5gax4KlpzhpzVpeXVNsnqMdi3mxF+YBridMgWj91d9cZYygS6b4= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(185117386973197)(228905959029699)(17755550239193); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040470)(2401047)(8121501046)(5005006)(3231023)(944501075)(3002001)(93006095)(10201501046)(6041268)(20161123558120)(20161123560045)(20161123562045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123564045)(6072148)(201708071742011); SRVR:DM5PR07MB3467; BCL:0; PCL:0; RULEID:(100000803101)(100110400095); SRVR:DM5PR07MB3467; X-Microsoft-Exchange-Diagnostics: 1; DM5PR07MB3467; 4:p9n5A73E8Q7q64VUI8QTsOHkpJFuaCwI77fYv2kcCuN8esNb4uUFyyX0kecjBMA5HhggAij3FcV/7dh3myJyAvuWzmmT8cLvMX/Akc8fLmg470HS1Ni1lIsQ/ZrchymayFdYmehfwKGDYWQ3WVcVzT7GTkxnSb1x328TN0+Y/N906nFqyXAjgECsvF6mODwwtgd98rFABWtYbWfe48flwABEVkJM095a7fYW7D0CK5qy/OKJYuG+wHbEejT5M4Hyfu7oZGljnRlzGU5RX+RmB2YiRcKbyf61g+cogWnZw36lThWiiQCbWbFUuUVbCAuUCjQawwiqRlOsvafZ86WhdWZsKdk+jzAzIiCvVSKfm43CKYfkDkmzUpjaXv0hrlfA X-Forefront-PRVS: 0548586081 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(396003)(366004)(39380400002)(39860400002)(376002)(346002)(189003)(199004)(13464003)(24454002)(76176011)(3846002)(53936002)(386003)(53546011)(33716001)(25786009)(33896004)(8936002)(6116002)(59450400001)(81166006)(2906002)(81156014)(5660300001)(105586002)(23726003)(478600001)(72206003)(8676002)(1076002)(68736007)(106356001)(6496006)(16586007)(7736002)(305945005)(55016002)(2501003)(50466002)(229853002)(66066001)(58126008)(6666003)(16526018)(110136005)(97736004)(2950100002)(93886005)(4326008)(42882006)(52116002)(2201001)(83506002)(8656006)(47776003)(6246003)(316002)(9686003)(921003)(107986001)(42262002)(1121003); DIR:OUT; SFP:1101; SCL:1; SRVR:DM5PR07MB3467; H:Pavan-LT; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; Received-SPF: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; DM5PR07MB3467; 23:K5P66o9Xc6mo+JptPKcshRp7xpVMMSZ41bRPzdsBq?= =?us-ascii?Q?nUBFwdrjW7TsQ8EqmKAMhj35BPAWs6f6ACHFw8iwZbKsgREPP6cYVnXU8BHX?= =?us-ascii?Q?junqh8XraW0XIKn+a3LZl9pStcOMERNKmvmi9EHP6msCJSEbnlyhQDGcWwXF?= =?us-ascii?Q?IqtSilKLKFt3hpXTIF4pa3WSSAupUytTQyZ9TfmYoW/b1ypAtJqOKbsFOWtx?= =?us-ascii?Q?xUj5knBIcIhXo8mzBQtaPUo3CYc9PIvosq9+3OW3iha2Bx9s+gHWr1r9LcqA?= =?us-ascii?Q?E0ei/PDtYd7Dp2z/owjFCytKykLzCTg+xlEASCwZWmCYT6Hk5Wf1MdIPYwYQ?= =?us-ascii?Q?CklF3SlraKjkczD5+MdQMCdjle4ov6cTLUMcTqBFQ91sKl4GgIfsJFnCS2OJ?= =?us-ascii?Q?vukEiuyZT+yzNCBeauBNJGQ6kkxUCdx55GhGCCrToSObA99wydkpMzb0kNR6?= =?us-ascii?Q?WF2z8HqEkT96OikJTC9MXPbaZZGq6eK26uClANFGar5bH03OwngNbHRiQgkz?= =?us-ascii?Q?zVwvwO2iaz2nYF1XygrxyLoRYN4rIcGb2LL2kh9u5nOgQteGNTTlXhekYb46?= =?us-ascii?Q?ZdmSYaloonwbWrXkO6KgoB2J+RT+ykWIxmUUq2nLJkuFnHsHnirdllObgW1a?= =?us-ascii?Q?pEpGDuMDJTTTRjF4HEXP+sjB9rfvnrVEJ2oowbAYlLDV5n8kItdtzTgsRIgR?= =?us-ascii?Q?kV3+thLLYWM8LlK/dfB99Rfay7JswEr1ijYxjCkxLKs8O/aV1XcGSQJZ8iUU?= =?us-ascii?Q?T/ah+nwrIpnk2pXp0mXamoLrtlB90gIRI1an3Ai7pLJyXwkZchA1Xw0+W0pJ?= =?us-ascii?Q?V/Mho+GFPEq+lN8bTd7osYwpCWUQGZUGoE5crLxPhiA7AhL6C71aqCPVB7+u?= =?us-ascii?Q?/cU/kEF1KrL8lKFcQuXUnAk+PefTLolRNZxnXp/Te1qKJP4rKWocL84c3ap0?= =?us-ascii?Q?XnW51qd8BpwFjj7B3LXQdbOtktKFOI8u1lpoG6dARcsBywm010Lq+ZqoyWKU?= =?us-ascii?Q?D1fn2fD+bnurIPtL9KZ7/0uD+suyo7S55VEYYCD9kcsnNb0WFeEy2RxA97N6?= =?us-ascii?Q?UNHciBvjeXCXcjUHm8L29eaXYphpvif85e+P0OM88Yvh6Sz/TaluFkwYDlcF?= =?us-ascii?Q?wrVrDVS2W+nIKB+L8rkvm7ccJACngeCe2Wul8oyunVp5qDIR7GlQLobUoD9A?= =?us-ascii?Q?LpvGQrfagJ2xVoGlZZ7bkOjxr60kf16L/sQItw0GGJ8nAlUBV1LC8FLr/S6t?= =?us-ascii?Q?d98knE63pLxkxi4qQH6EUu4IlQA5tGrUryCXxmWPvFhGefNhoCsq5g2mHw9G?= =?us-ascii?Q?ybzASlwMGUdJ3rwVXGSlU+eGSXcBuwDXtpYUlXO8n9PT/6M9ADP9GIrL7Bbv?= =?us-ascii?Q?41sUjqVQyxPng7nsGAorB42uETJuAVvEaxEe22M4fwuyOeJslAZ+BpIGPT/y?= =?us-ascii?Q?3UwHj/SkA=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1; DM5PR07MB3467; 6:90YjlpnOJw+7waDPyqX+447QMvPSFN8n3KdIYTbU4tBs5S0WcAAkJ/3iu4hSw1Y02VfcYTGu/APwimZaadVGbxPEAAYJY+GbMoheu84lCGfwlyF7JdZ/P4FVWpYe9kUBQBEE987RjgCxaV+Yj2ZJAVdoJ/keD3xRFpxIT4N9Ix4CHHplE6O9MZ5wwGGCzvp3Dmrqh9sRzhYhe2u30weshRe6LO6r+0Z5oBRPW5UdzlC6hIFGhmECKsZZzWHKB7IsFIXeckZngsRnDzSgfEM4Y2hgXuwiVb8GABt954LVcLvhJ/TnTKPLaX9+3W75LHb51KEkuwxNvbBbpEZSkQzCmuPZeLicMivLIkst0gVvLjQ=; 5:AA9XBv8VnJT7NNHZE/r8Cc4jvd0vZm7Kj693sbbmlQqxMxL+26WYWJ72gwOI1M1xF+yNzUv5oCBfY4sL2D2qxiD7B1NyiMvR3boeBVvJk49OyJaAfBcsEUb7LRTJ1qeWrYupdR8RzuEtq0uL6rqpcjoToSpFRMuu8+8Sz1jRvLI=; 24:4sRCX0IAXYhTDT5kE6ElbWuHjFtjjMzepHtpFgalcHgc8szjNXHgtbNUaTrvXJcga6aLZ3PN89axh/QuFsOPfbE4APZQd6+Jo+Xps/n7PYg=; 7:k264/480jKxA2WhNqwoTCIl9e68ZFo1aWQN7vu5RDoEJpzIGfLQbqUqc3gXKTIbEfBgHuTWSAKhpAEbZduTIEYlQ/6KEoktaDKWP//UBHPzshpK8XyxKtUWOUHEVFUbxsaAqFLQIqfVeuLMVHA0sAAWl0A+pikY7qXPliJSpYgOWpQGQLYdbqBfvXjrdz5DvjQJZ4M40HnL/g7+USTToDCZJNSCgJLqcNkhHLe1qekrVuYITI/MEKYa5B/1mjCWm SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Jan 2018 20:17:30.9818 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: e5dcaed2-dce4-4238-378e-08d558672fb3 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR07MB3467 Subject: Re: [dpdk-dev] [PATCH v3 09/12] app/eventdev: add pipeline queue worker functions X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Jan 2018 20:17:36 -0000 On Wed, Jan 10, 2018 at 04:53:53PM +0000, Van Haaren, Harry wrote: > Replying to self... > > > -----Original Message----- > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Van Haaren, Harry > > Sent: Wednesday, January 10, 2018 4:45 PM > > To: Pavan Nikhilesh ; > > jerin.jacob@caviumnetworks.com; santosh.shukla@caviumnetworks.com; Eads, > > Gage ; hemant.agrawal@nxp.com; nipun.gupta@nxp.com; Ma, > > Liang J > > Cc: dev@dpdk.org > > Subject: Re: [dpdk-dev] [PATCH v3 09/12] app/eventdev: add pipeline queue > > worker functions > > > > > From: Pavan Nikhilesh [mailto:pbhagavatula@caviumnetworks.com] > > > Sent: Wednesday, January 10, 2018 2:52 PM > > > To: jerin.jacob@caviumnetworks.com; santosh.shukla@caviumnetworks.com; Van > > > Haaren, Harry ; Eads, Gage > > > ; hemant.agrawal@nxp.com; nipun.gupta@nxp.com; Ma, > > > Liang J > > > Cc: dev@dpdk.org; Pavan Nikhilesh > > > Subject: [dpdk-dev] [PATCH v3 09/12] app/eventdev: add pipeline queue > > worker > > > functions > > > > > > > > > > > > > > +static __rte_always_inline void > > > +pipeline_tx_pkt_safe(struct rte_mbuf *mbuf) > > > +{ > > > + while (rte_eth_tx_burst(mbuf->port, 0, &mbuf, 1) != 1) > > > + rte_pause(); > > > +} > > > > re safe, see comment below > > > > > + > > > +static __rte_always_inline void > > > +pipeline_tx_pkt_unsafe(struct rte_mbuf *mbuf, struct test_pipeline *t) > > > +{ > > > + rte_spinlock_t *lk = &t->tx_lk[mbuf->port]; > > > + > > > + rte_spinlock_lock(lk); > > > + pipeline_tx_pkt_safe(mbuf); > > > + rte_spinlock_unlock(lk); > > > +} > > > > IIRC usually the "Safe" version of a function has extra locks/protection, > > while the "normal" version has better performance, but less-error-checking. > > > > Here, the "unsafe" function does the extra locking. If looking from the HW > > POV, that makes sense, but I think its inverted from most existing code... > > > > Happy to be proved wrong here .. ? > > > > > > > Thinking a little more about this, also in light of patch 11/12 of this series. > > The code here has a "safe" and "unsafe" version of TX. This involves adding a spinlock inside the code, which is being locked/unlocked before doing the actual TX action. > > I don't understand why this is necessary? DPDK's general stance on locking for data-path is DPDK functions do not provide locks, and that application level must implement thread-synchronization if it is required. > > In this case, the app/eventdev can be considered an App, but I don't like the idea of providing a sample application and code that duplicates core functionality with safe/unsafe versions.. > Some PMD's (net/octeontx) have capability to do multi-thread safe Tx where no thread-synchronization is required. This is exposed via the offload flag 'DEV_TX_OFFLOAD_MT_LOCKFREE'. So, the _safe Tx functions are selected based on the above offload capability and when the capability is absent _unsafe Tx functions are selected i.e. synchronized Tx via spin locks based on the Egress port id. The patch 5/12 has the below check to see if the connected ethernet dev(s) have the capability to do thread safe Tx + for (i = 0; i < rte_eth_dev_count(); i++) { + struct rte_eth_dev_info dev_info; .... + rte_eth_dev_info_get(i, &dev_info); + mt_state = !(dev_info.tx_offload_capa & + DEV_TX_OFFLOAD_MT_LOCKFREE); .... + t->mt_unsafe |= mt_state; + } Based on the value of t->mt_unsafe the appropriate worker function is selected. > Hope I'm making some sense here.. > Hope this clears thing up. Cheers, Pavan. > > > > > > +static int > > > +pipeline_queue_worker_single_stage_safe(void *arg) > > > +{ > > > + struct worker_data *w = arg; > > > + struct test_pipeline *t = w->t; > > > + const uint8_t dev = w->dev_id; > > > + const uint8_t port = w->port_id; > > > + struct rte_event ev; > > > + > > > + while (t->done == false) { > > > + uint16_t event = rte_event_dequeue_burst(dev, port, &ev, 1, 0); > > > + > > > + if (!event) { > > > + rte_pause(); > > > + continue; > > > + } > > > + > > > + if (ev.sched_type == RTE_SCHED_TYPE_ATOMIC) { > > > + pipeline_tx_pkt_safe(ev.mbuf); > > > > I guess that means that the functions where they're used are inverted in > > name too. > > > >