From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 43E4AA0548; Mon, 20 Sep 2021 13:24:48 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C396540DF7; Mon, 20 Sep 2021 13:24:47 +0200 (CEST) Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by mails.dpdk.org (Postfix) with ESMTP id 2E58140DF5 for ; Mon, 20 Sep 2021 13:24:45 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10112"; a="308654148" X-IronPort-AV: E=Sophos;i="5.85,308,1624345200"; d="scan'208";a="308654148" Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Sep 2021 04:24:44 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,308,1624345200"; d="scan'208";a="702443820" Received: from orsmsx604.amr.corp.intel.com ([10.22.229.17]) by fmsmga005.fm.intel.com with ESMTP; 20 Sep 2021 04:24:44 -0700 Received: from orsmsx609.amr.corp.intel.com (10.22.229.22) by ORSMSX604.amr.corp.intel.com (10.22.229.17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2242.12; Mon, 20 Sep 2021 04:24:44 -0700 Received: from orsmsx610.amr.corp.intel.com (10.22.229.23) by ORSMSX609.amr.corp.intel.com (10.22.229.22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2242.12; Mon, 20 Sep 2021 04:24:44 -0700 Received: from ORSEDG602.ED.cps.intel.com (10.7.248.7) by orsmsx610.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2242.12 via Frontend Transport; Mon, 20 Sep 2021 04:24:44 -0700 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (104.47.59.177) by edgegateway.intel.com (134.134.137.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2242.12; Mon, 20 Sep 2021 04:24:43 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=O7F4HZY0apwcXzecQ0FvparK2eqHJULtOgOacqaGQGdby80XRycqLKzS/gK3RyRlJDYuQpC5R0UldXc4dBrsQj5fl5o7p6GNQsCed+D2CeJlLPoD9DWcb3TqhzrsNB4p9Q9nJWslku4mNgX4ZUgVo/IE7+ZOrI54rYTnDLd1ueuZfGOLu/kElanpCcF2Cdgmjn5nXOLU6o7Q989FvVkw/jzFq5p8OMBerpM7qgkrH0E6Svajpzpe4l1iwly2q5o3ovW9oCAYxZyOfSDEGo5umvquQpq35svx9eBu3rTopdanZiAAGr6sbjbnrFe65wR7GCmSKKllIXdX1VsJKC+qkQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=xHu8FNgrIq8Sdw+QPPIdITny9NM/ZNs2doa8br2Sauc=; b=Y861Jg3BGCXjUPGI7Bs5QHq4z7yY+W1nFWnpc7OaN+BHDtcuPsGCmNrP56KCnGFIxLouySGhCMIqBqHH7hHB93dXgTgAkpPHSCTMfY2pGK54BCEO+WnYwblsRJInFDgCH36NBG3tNKX5EsjIp2UBTK2UuZ7xqAguLVFzkPeY27gBDxgPtTNNJ6OyEHP1BLn86r2jnVUFDRNtzB0yVBVZj7QNXDu3KrypjLiSuemwyQ7TGrhKaF6c+NCFjBNA37UrDWbYdNYeVW2dBZ33aSYM498M3XXCrUnq+f+78PRCuqhKUPu4GqDDKxi/2AleAh8buLU3NGHgBC63t/KiWJaWkg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=xHu8FNgrIq8Sdw+QPPIdITny9NM/ZNs2doa8br2Sauc=; b=zOOPte9LRmhyWTODWVvMT6d2OcEd5lAKbhZOJBNuXklmRhFrQoPTCZd0hjytggVb1stOcncEqMSjcOoctPv9Acnfni/IGrRKiCxrTKM3eOwxw5Z2uP4igus3+4cfCDdKb7Zv0XwCzxN4H61Vwn6SqXS8xpU6+RKcoJO7fHA+aQw= Authentication-Results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=intel.com; Received: from PH0PR11MB5207.namprd11.prod.outlook.com (2603:10b6:510:32::15) by PH0PR11MB5094.namprd11.prod.outlook.com (2603:10b6:510:3f::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14; Mon, 20 Sep 2021 11:24:40 +0000 Received: from PH0PR11MB5207.namprd11.prod.outlook.com ([fe80::9578:a4b5:2716:fbaa]) by PH0PR11MB5207.namprd11.prod.outlook.com ([fe80::9578:a4b5:2716:fbaa%6]) with mapi id 15.20.4523.018; Mon, 20 Sep 2021 11:24:40 +0000 To: Kevin Laatz , CC: , , "Konstantin Ananyev" References: <20210910172737.2561156-1-kevin.laatz@intel.com> <20210917164136.3499904-1-kevin.laatz@intel.com> <20210917164136.3499904-2-kevin.laatz@intel.com> From: Conor Walsh Message-ID: <8eefc44f-f7b9-d9aa-7f25-c588c8ddf3a3@intel.com> Date: Mon, 20 Sep 2021 12:24:34 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.14.0 In-Reply-To: <20210917164136.3499904-2-kevin.laatz@intel.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US X-ClientProxiedBy: DB8PR06CA0066.eurprd06.prod.outlook.com (2603:10a6:10:120::40) To PH0PR11MB5207.namprd11.prod.outlook.com (2603:10b6:510:32::15) MIME-Version: 1.0 Received: from [192.168.1.20] (109.76.159.128) by DB8PR06CA0066.eurprd06.prod.outlook.com (2603:10a6:10:120::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14 via Frontend Transport; Mon, 20 Sep 2021 11:24:38 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 7fc47325-b4f7-4679-0d1a-08d97c293c78 X-MS-TrafficTypeDiagnostic: PH0PR11MB5094: X-LD-Processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:9508; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: w2/Cd3FIAiv3XAavO3o3GqldskOHxgQJ3hGxvq6cTbCWEgqEoti055oRWcrfEK4aQ1hGU12sNvmWjGm5joa2o3FqQRtF6ETNru22fJDlztZXOboNNnu0aS6iLzZfXUkuHUeWCwjaVwNWbBeX71t+M3GjhZmy+tEqWllNWsRk1d93me+RQ+SlxVMEFffCmvSS5qKa/KyVLuMuSd1YW6C6/DDJmwi2l1BNPRDEFMAI2UaoSiGvvuOr+83ALWlqgC5/wwTdxgBGfFcl7QZ6/79JNXrPQmlP+K2N/sE0dgnrPWlfMVnIgQJOMHmRWiTrTXy3yYM8DVeRnuvdBdbck4yeckcI/gCFAU1pFINUu5JvccIC7jLas/ph6n/YIuUokTdLrql8XTRHjf8YJtiZztl2x4Ave+D4lVYepdgYN3BB1rG+D3bj+VgDPZTokTU/su3RhYdOT33E63zJC8syN68q6zTIWL6LZcNTpORi9rrQ7XWv3GWeaeQGHxXV7S8GPQfFLei6FHD8DX58qLuEKej3fuvTKEJ0ijVnEZAeVveKM8NjRZNIm0Cvn3YU2gh0ixz39YgYScYkyqx+4Qes5SccwtimA0EBG035AEZJFezjgClYgkxTndz5A/gFBaf/MWt/vg5gujfZWqE03kU9pgZ9A8AjrY5U+LjHoReWc8MpTkdTVg+4dhXLZxOkMYp+u/tuAYOPQ+2sGkQgaTWxKsm9kJv53Cd2Ow9ql+ws91z6AqaZtR7We9Vp5uZJcNPfcJMQ X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PH0PR11MB5207.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(366004)(376002)(39860400002)(136003)(396003)(346002)(4326008)(956004)(38100700002)(44832011)(66476007)(66556008)(6486002)(6666004)(2616005)(2906002)(478600001)(8676002)(26005)(66946007)(316002)(186003)(16576012)(36756003)(5660300002)(8936002)(83380400001)(86362001)(31686004)(31696002)(107886003)(43740500002)(45980500001); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?Z0tBTWxpazZCUlY5dElnOHVWSkhjblM3ZWJBZlRPU3lpRzI0dDFiNjhVclhB?= =?utf-8?B?eVZxbFFqSmlzTFg5YkhJSnliS0puR2hjbUJmVWd6NytOVjZHMnJpRXRWVjl2?= =?utf-8?B?SUhnc09ZWWt0aFEyQkVNbDY5cFpud3FYeDIxVzc4bXpGSjdiWkUzSXNmOW91?= =?utf-8?B?bDR1STVXODFtb1I0L24vYUtkOFNWd3BUc1VjSjNOaG92K2l6RmRuaHNVSmoz?= =?utf-8?B?STdTY21COGVJOTV0dWV5RWZlZlRCVEN2T2R6Y1NqZzNxZkdOaHA3UHQzaU1D?= =?utf-8?B?MUgrelhnRVcxMDZnKzhaNTltNFh5aW5iV2xqTExucVR0Q3M5R2lQRWtGRXRS?= =?utf-8?B?eTRCa1YxU3ZKZml4NUhLa051TVQrNTlpTjBLT3p4QnF0Rk5tODJOTlFHM1p5?= =?utf-8?B?ei85dFEySzBGamo1MVpJa1lmWGhZREhVYWtpaEdvSjZXOHNHTjhuMXpIaFBL?= =?utf-8?B?WnNLcklERTB0L2dyakJ5ZVc1ZzE2NjFES0FJd2ZPbWlaekwxSll5SjhJUXJw?= =?utf-8?B?NngrUDNZQVp2MlhrdUlMSS9CRitrTlRCbmNyRllieHlxR0lRaTQxOHV2cCtJ?= =?utf-8?B?VHZtaFhVQ3ZaR00xUW84T3BOOWZoRW1ocGM5Z1lWeEdPZ01ocHlsZC9nK2I2?= =?utf-8?B?Smg2OTlXbytwVFowcS82VTFuN0xMbGE0eWxobVpnbHB6Ym5uS1U4SGxHTlZD?= =?utf-8?B?MVBJZFhpSFp2NVdMU01VQkFJMHduUFlTNUZNekZUN0IzSStQZXB4OUxOWWRq?= =?utf-8?B?WW5ud1lhZnNQaXFodDZaMTZnRnQ0bFQ4dC9Fd0tJWDZ0cWtxZUw4cElPN1li?= =?utf-8?B?VWRVS21QamZ0a3FXVURyWDNBUnVzdXlCay9wSERkQjFEY3psMjR4V1ZSWHZL?= =?utf-8?B?RHVhY1RINXJIRGtENXpNcDEyTzRnaTRjYWpRRzNSaTdrQVBHWGRPV1JMQUJI?= =?utf-8?B?K01WVk13Q1ZoVUpWUjZpdThiYW1KcWVTTHhpUmZHYW9xT0lxTHBna0Vhb2pU?= =?utf-8?B?UGw2eHJpUDkveFBSMWFrSWtGY05MTm0zQUVyQ2xkY05JQ0N0TXBZOEFxc0Fa?= =?utf-8?B?Tk9yZFROVGwvc3d1QzlTN3E3cE0zSkwxTHlRNnljZUpEZWZ1T0dXYm1jdlAr?= =?utf-8?B?andVaDg2bHpxODFTU09BbzNGelZvcCtYek9CSDQxbVFxU1NCUkZRakVMb0hm?= =?utf-8?B?Y1J0ZngyMFpaNXZMa2RiME5VMHg5VmQwVDJ1dGZjTGt1d1hKQ1BTRXpDWmg1?= =?utf-8?B?MUZFczBmUHk2dEg4UUVKNUV0S1ZEZmk5ZWlUaTVCM3hDTlZtTTAvcm1IZ3lB?= =?utf-8?B?MTFpUERlaDRtNTJGQ1BJZGRiU1haSG0raWUyei9RaVlna0Q1dlZ2VCtNQ1lL?= =?utf-8?B?TWFOcGVsZXBaOUY2OXcxQjBRNkpudkJWV05iWFZpQ2xnYXhHUjFFUldTU3hu?= =?utf-8?B?Y0JWTW8vRktsY2tWOW1LcXJCV293eVdxWXd5akdUVDNZcHhieDd3UG9IYVN0?= =?utf-8?B?OFl1K00zSTViemw4NmpXQUF2SndxY2RQZG5YTEtsdTQ0STNGcWxzaFVKelRm?= =?utf-8?B?U2VjOVMwUHEzUzhiTng3aVEveU5rV1pNVEYyd05YaUhGbkE2N2doL3UyNlRY?= =?utf-8?B?bjMrbWxsK3lVNHJzam1OSzUyRDNCVGt3RGhBTGc3bHE5TXh2UjBwT0RDVFlh?= =?utf-8?B?cWw0VmU0aHNNeXRxUU4xZ1YvNXNKTzltdm1IVE9YREphRTF2ai9WUnNhTFM2?= =?utf-8?Q?307wcOd2O1TeCNTdVsS8D4AlveSqKP0mOYQvEqV?= X-MS-Exchange-CrossTenant-Network-Message-Id: 7fc47325-b4f7-4679-0d1a-08d97c293c78 X-MS-Exchange-CrossTenant-AuthSource: PH0PR11MB5207.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Sep 2021 11:24:40.0874 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: hcz5WMufcsYzYAVuTfV/xAvId9lKzUPfiUuOtPOjqRuOiZLOQj/B5Hr/Fb3M+N6/2M5qp44KYZgLTYNwQ5O/9A== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR11MB5094 X-OriginatorOrg: intel.com Subject: Re: [dpdk-dev] [PATCH v2 1/6] examples/ioat: always use same lcore for both DMA requests enqueue and dequeue X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > From: Konstantin Ananyev > > Few changes in ioat sample behaviour: > - Always do SW copy for packet metadata (mbuf fields) > - Always use same lcore for both DMA requests enqueue and dequeue > > Main reasons for that: > a) it is safer, as idxd PMD doesn't support MT safe enqueue/dequeue (yet). > b) sort of more apples to apples comparison with sw copy. > c) from my testing things are faster that way. > > Signed-off-by: Konstantin Ananyev > --- > examples/ioat/ioatfwd.c | 185 ++++++++++++++++++++++------------------ > 1 file changed, 101 insertions(+), 84 deletions(-) > > diff --git a/examples/ioat/ioatfwd.c b/examples/ioat/ioatfwd.c > index b3977a8be5..1498343492 100644 > --- a/examples/ioat/ioatfwd.c > +++ b/examples/ioat/ioatfwd.c > @@ -331,43 +331,36 @@ update_mac_addrs(struct rte_mbuf *m, uint32_t dest_portid) > > /* Perform packet copy there is a user-defined function. 8< */ > static inline void > -pktmbuf_sw_copy(struct rte_mbuf *src, struct rte_mbuf *dst) > +pktmbuf_metadata_copy(const struct rte_mbuf *src, struct rte_mbuf *dst) > { > - /* Copy packet metadata */ > - rte_memcpy(&dst->rearm_data, > - &src->rearm_data, > - offsetof(struct rte_mbuf, cacheline1) > - - offsetof(struct rte_mbuf, rearm_data)); > + dst->data_off = src->data_off; > + memcpy(&dst->rx_descriptor_fields1, &src->rx_descriptor_fields1, > + offsetof(struct rte_mbuf, buf_len) - > + offsetof(struct rte_mbuf, rx_descriptor_fields1)); > +} > > - /* Copy packet data */ > +/* Copy packet data */ > +static inline void > +pktmbuf_sw_copy(struct rte_mbuf *src, struct rte_mbuf *dst) > +{ > rte_memcpy(rte_pktmbuf_mtod(dst, char *), > rte_pktmbuf_mtod(src, char *), src->data_len); > } > /* >8 End of perform packet copy there is a user-defined function. */ Might need to redo these snippet markers as the function is now split in two. > +static inline uint32_t > +ioat_dequeue(struct rte_mbuf *src[], struct rte_mbuf *dst[], uint32_t num, > + uint16_t dev_id) > +{ > + int32_t rc; rc should be uint32_t, but this is removed in patch 4 of this set during the change from raw to dma so it shouldn't really matter. > + /* Dequeue the mbufs from IOAT device. Since all memory > + * is DPDK pinned memory and therefore all addresses should > + * be valid, we don't check for copy errors > + */ > + rc = rte_ioat_completed_ops(dev_id, num, NULL, NULL, > + (void *)src, (void *)dst); > + if (rc < 0) { > + RTE_LOG(CRIT, IOAT, > + "rte_ioat_completed_ops(%hu) failedi, error: %d\n", > + dev_id, rte_errno); > + rc = 0; > + } > + return rc; > +} Reviewed-by: Conor Walsh