From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D0C78A0C43; Thu, 23 Sep 2021 17:35:00 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9426841260; Thu, 23 Sep 2021 17:35:00 +0200 (CEST) Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by mails.dpdk.org (Postfix) with ESMTP id 9164D41257 for ; Thu, 23 Sep 2021 17:34:58 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10116"; a="210946379" X-IronPort-AV: E=Sophos;i="5.85,316,1624345200"; d="scan'208";a="210946379" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Sep 2021 08:33:40 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,316,1624345200"; d="scan'208";a="475555260" Received: from fmsmsx606.amr.corp.intel.com ([10.18.126.86]) by fmsmga007.fm.intel.com with ESMTP; 23 Sep 2021 08:33:39 -0700 Received: from fmsmsx607.amr.corp.intel.com (10.18.126.87) by fmsmsx606.amr.corp.intel.com (10.18.126.86) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2242.12; Thu, 23 Sep 2021 08:33:39 -0700 Received: from fmsedg602.ED.cps.intel.com (10.1.192.136) by fmsmsx607.amr.corp.intel.com (10.18.126.87) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2242.12 via Frontend Transport; Thu, 23 Sep 2021 08:33:39 -0700 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (104.47.57.169) by edgegateway.intel.com (192.55.55.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2242.12; Thu, 23 Sep 2021 08:33:36 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=aLJUKmTNIsTs0hsV5OSWtn8iT0tIcZFeyEkliHgZSuJkYgoZYw225mPkeQtx30cXwaKjuK5qRB/Ta/ZagdnZp3Pez8HKSY+3zw8yjb9psAKv5XjXvjc/kGUnhRpXC4A2VO5x0irVbwYNMCXOzDtSeakbe1dzcJcNwhjNPmcVQeVpl4CV67N9KV1yACgIJuPbYFVy9p3o7TtNFwWGB/eoWtfjUDGiWheqi4+hj28rAfVm+vpq9yzPSgyDeoFkoygxTg3i6ylH8vD9/1kCml19mRepSEWonovmiyhJclDCHHbIdoVrFGbyheisHvOsK3YjZqAG9sfjwLXsWL073sZRpA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=ErnrdllCdI40Qq31rUrxorb28kl8iSmrMFHs5oyVz/Q=; b=GEPrFp+xtqHFVMogRO+rYL8jFnMjWPSCvI30nh98wznn0EUqAGN2BAhr5TT5PKtnIGVnSNmHZ51eqhg4ZfWVFxIBhZY+HqYs4YjNBFrqlpp5AvAA6wz6zJTD423E+XW7IzFJBlprkGKj9sxyYkIiOFYXIpc0b8SQczNvbBaI72tZtyBf39jQW9i7vwaDnsIVXTgWZLoRptCfY0y+WhecNlOyxO6+2UiqhuM8kyQs/MAzVuHC8OEsMWFVe9Hv0AU0Ld6BVrmF2wlVB0oWY2lSDSeQmR9s/V29e5yoLouseBhgr7shhlyBKv9B2U3qPD5TGPAlOV/6mLFzElh53G1xbg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ErnrdllCdI40Qq31rUrxorb28kl8iSmrMFHs5oyVz/Q=; b=Re+4qQOJmiyBdZWfIT92NLnzuCnJ05wHuUz2/TeEi8Y1AY8XdH6u4HfJ3EXxN7eB/MfgoRYpy8u7SLODzaBkdcn7CqOjZTpzf71av06Vde8w/+dVYoSp78KECalyGWklZEWSO8LBke7CTxhsSCikT1J+kK6a65cQT/Jlhwi6Ad0= Authentication-Results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=intel.com; Received: from SJ0PR11MB5055.namprd11.prod.outlook.com (2603:10b6:a03:2d9::10) by BY5PR11MB3861.namprd11.prod.outlook.com (2603:10b6:a03:18d::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15; Thu, 23 Sep 2021 15:33:27 +0000 Received: from SJ0PR11MB5055.namprd11.prod.outlook.com ([fe80::78d4:6f82:aa87:c271]) by SJ0PR11MB5055.namprd11.prod.outlook.com ([fe80::78d4:6f82:aa87:c271%7]) with mapi id 15.20.4523.018; Thu, 23 Sep 2021 15:33:27 +0000 To: Conor Walsh , CC: , , "Konstantin Ananyev" References: <20210910172737.2561156-1-kevin.laatz@intel.com> <20210917164136.3499904-1-kevin.laatz@intel.com> <20210917164136.3499904-2-kevin.laatz@intel.com> <8eefc44f-f7b9-d9aa-7f25-c588c8ddf3a3@intel.com> From: Kevin Laatz Message-ID: <50e3a3ef-a644-5b02-5dbc-54af0b59dba6@intel.com> Date: Thu, 23 Sep 2021 16:33:21 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.14.0 In-Reply-To: <8eefc44f-f7b9-d9aa-7f25-c588c8ddf3a3@intel.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-GB X-ClientProxiedBy: DU2P251CA0020.EURP251.PROD.OUTLOOK.COM (2603:10a6:10:230::29) To SJ0PR11MB5055.namprd11.prod.outlook.com (2603:10b6:a03:2d9::10) MIME-Version: 1.0 Received: from [192.168.1.15] (93.107.194.240) by DU2P251CA0020.EURP251.PROD.OUTLOOK.COM (2603:10a6:10:230::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15 via Frontend Transport; Thu, 23 Sep 2021 15:33:25 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 7edc0481-5cf4-4db1-b8da-08d97ea77ce4 X-MS-TrafficTypeDiagnostic: BY5PR11MB3861: X-LD-Processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:10000; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: rjTPwCBsLWcvXogf0zL//o8iKgb0JFZCTRWVLGWYmGC+NpsaQ1xS1Ge0ap6oSDxKdr6fNCrzmepbjLeQmPFLnby+C4A4YyaX6NQUGns0g8hWqhpvad0Gi0DyI51eG1IY587X/DmL4n2mVMkMsPZpcdZhAXBo2jyZj/qTpDaimXhkAQn9T/t+hPttyhMCqaULO7FQzv/n9lH8d8oV6SxByYBu/cBEXXieyMBonIJfaKHYGuNwpjx9wSYo1mLCBYLgdmNhnH71g+WIZps/hNx6hmnbmzMdWtPVAI9X+orYRZNtUfgXGarocapVqZy0X+XiJbV4wjjAL6f68zSgKGSJMqE9hLpS53wa6umg/xK7WBxhdfW3tjEcUnZMDqxcWRxQWDepyHLWLdJh15fpwYZGLQQeDD9wZVBBnM0c/NjX1F0OuzP4E7DaeLEtuKW5CnMPBeAxKllMz9ncTNKAvYxwbu1N3Ad7+Bhk0nzXYfmoUUBaIzzH4DwwTpHnlqTeIgDMe+YEy5jyMP15Gm0rmJcFBJj+JMbrUTJ8gSFzchT4RVK2uGBBxdkYfi8uzjV8JUewX6OJr8enhgcQNf6Lolz/txaeDoX2M+f3KkEQL9jKgSFd8/ZEkslVQV14u4YjiFPiWhxc41ckFXTaim1G6XNqvYFYe3uJiIfj97mqMUrsggiweU9Z0P7o3Oc3+OVPMK1Y8Ja63ld4Atb1sWvfQVuygz8cM3CGII6oiToM9W4DbiQ= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SJ0PR11MB5055.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(366004)(2906002)(6486002)(107886003)(53546011)(66556008)(508600001)(66476007)(6666004)(26005)(8936002)(4326008)(8676002)(38100700002)(5660300002)(86362001)(36756003)(956004)(44832011)(66946007)(31696002)(83380400001)(186003)(2616005)(16576012)(316002)(31686004)(45980500001)(43740500002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?WitmK2hOR0NYNUhyYkE5U2dQRDRkNlBSVnRRdFVGYXdrM2hDL1pqSmhTM1pO?= =?utf-8?B?M0x4cDgrUTVkQ0NwczdtV0h3VkhIZFg2YnNQeVJ3aDQyUnlQTVhRREx4ZzBM?= =?utf-8?B?dVVFd0M3RjFCZmUxblJnbENkM01yWjJ5d0FDbFUyQmszNE5xU3ZTbFdGL3do?= =?utf-8?B?aS9mZFZJTnVNTDE0OHZ2S3lUZENWVVh4NVdRMHhPV1h0bVFyOVl2NGlwRys3?= =?utf-8?B?Q0Y5QU9JMVBwd0Z5cCtMMVI3a3A2aCtaMkE0TnJLR2hzZkRSc0VYc21ETEpX?= =?utf-8?B?NFZRUVhVTHFCYzJ5RUJWTFlZajU2NmszZktsd1FPcC9sY2JQek42Ynp1MEJM?= =?utf-8?B?aldyNDdlYXh5VjBwVjhBL0l5TG1hdHZjNHp2ZE1rZXd1NHZ4cmp1NGxXZ01l?= =?utf-8?B?NUt1TzA2Ym9ReDhGNm9Ma3YxTUFTbVdXNXJSLy9YQk9ob05LZm04alR6OUtM?= =?utf-8?B?ZHRCZFg2SHl5K0FQWjdOK0pxOGQvdnZMaEJQdDlYN3FrSEtLOXc3N3U4Rm9F?= =?utf-8?B?SWdDQ2JpM2xOWXlVSS9hKzhQNWUyQXFNMWRKeG9qN2tyTjR0aW5DNlRUNTN6?= =?utf-8?B?aUt3T3JSY1lSYllaU0dPN29aVi85WGdHMzJNdURLdU9reWxCTFI4WlRocjBv?= =?utf-8?B?ZFlmUGJsZWNXQ0M5bFh5QUlPM1NhS0xvQ0JTTWZQelZ6YzR6ekRYL2JVZmdZ?= =?utf-8?B?aFFyUDZJZFQvckRjWEdXT3lZTTQxbzBrU2FUY0QzemVuKzBJUDBYNzdDcmFj?= =?utf-8?B?MGorcktYcURDVjkrQWM2cWhIaEt1Qkl0NHBPVWUyelJWTjkyeG80dGNkOXAz?= =?utf-8?B?bHZ2bGJBREJHS0pTY0t6TDhvaDRwZFZxTlVQd3lIcXN2TXVzcW4yREt2UDN1?= =?utf-8?B?Sml3eVFWMzZ0VHJ2RnNJWWhPeFlkOTJSTFR3bXNPZ1cwNGM2aXFUdFRQblFa?= =?utf-8?B?bFVqeldMR0N1TDBNam1WR01hZ1dwVFlObGRQWm9US2gycVJiY0ZNTVRMMmcr?= =?utf-8?B?eXNtSHRaVFNtVlJSQk1xU0NSQkM5WVNvL2l5QVVXQmYzUExMMHFBUGR1VWYv?= =?utf-8?B?Qk0xdVd4YjU3R2hPM0s2eVVQcjBzTTU0ZGcyVVBCeHIxNW5nWFVsdzJPMllQ?= =?utf-8?B?Q2RUd2xBZ29ub2tteWpvODhxZ3luWk5tWm5mUTZCRGNHaE9NV2dBVFhpbHN6?= =?utf-8?B?WHBRVGFncktZL0Z0WEtPZTFZMnhKcXFQQjVyRHdQdi9SeE91azhvVnhkNng0?= =?utf-8?B?RGdoOVJmR2JXaVZ6aExsYlRKemIwVFJ4RjFQbWcyckdvVFUrTHFIb2NGQXZQ?= =?utf-8?B?OEVLVTNVbkJSaEJrZEZ2a0s5TlVwMktQWTMrQXhiR0x5WTgvaEQxM0RrSEJE?= =?utf-8?B?cHl4c1ozN3Y4UTJXdDhmK1N1a2cwdXhjcHdHYmdjTzd6c2JWR21SUFBQQUVs?= =?utf-8?B?cXpHMHFqaWhQV2NCd29KYThGWUhKdWdGTmpmZTNia2c0Q010VWxIdHM4NEJh?= =?utf-8?B?Njc5T3VjalFKL24wcFpBZlNzV053eEI2RVdhdnVIYytaMTFUN3hXTUNTZEFK?= =?utf-8?B?Y2JhYjdxS0RvZGVPTERDTmx0TGR2ZGZDRGVpTStKaktEeHlqRDFlaE1UaW9r?= =?utf-8?B?RWVUREJSMXZFd1J3eTZBekROM2dFbVRwWEFaT01rTkdkVnN4eE5YdytpNTVM?= =?utf-8?B?cG9GQ1l4dERIMjM2RVlweEtwSDZVQkRoSXlYNXVhbjR5MHBielBINGVDMmJ4?= =?utf-8?Q?6fnsI9oHlTwxBLq7ViGLDqDF9KNC08Sc5rjvzuT?= X-MS-Exchange-CrossTenant-Network-Message-Id: 7edc0481-5cf4-4db1-b8da-08d97ea77ce4 X-MS-Exchange-CrossTenant-AuthSource: SJ0PR11MB5055.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 15:33:27.1191 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: brIZAytWZC6ckV90DO4oQoqs2txXbqGqvflKsu2b0MmpSRGQLqPbUKxMgj5mr32C2KrZPYpqgQkIUGXi2pfoqA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR11MB3861 X-OriginatorOrg: intel.com Subject: Re: [dpdk-dev] [PATCH v2 1/6] examples/ioat: always use same lcore for both DMA requests enqueue and dequeue X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 20/09/2021 12:24, Conor Walsh wrote: > >> From: Konstantin Ananyev >> >> Few changes in ioat sample behaviour: >> - Always do SW copy for packet metadata (mbuf fields) >> - Always use same lcore for both DMA requests enqueue and dequeue >> >> Main reasons for that: >> a) it is safer, as idxd PMD doesn't support MT safe enqueue/dequeue >> (yet). >> b) sort of more apples to apples comparison with sw copy. >> c) from my testing things are faster that way. >> >> Signed-off-by: Konstantin Ananyev >> --- >>   examples/ioat/ioatfwd.c | 185 ++++++++++++++++++++++------------------ >>   1 file changed, 101 insertions(+), 84 deletions(-) >> >> diff --git a/examples/ioat/ioatfwd.c b/examples/ioat/ioatfwd.c >> index b3977a8be5..1498343492 100644 >> --- a/examples/ioat/ioatfwd.c >> +++ b/examples/ioat/ioatfwd.c >> @@ -331,43 +331,36 @@ update_mac_addrs(struct rte_mbuf *m, uint32_t >> dest_portid) >>     /* Perform packet copy there is a user-defined function. 8< */ >>   static inline void >> -pktmbuf_sw_copy(struct rte_mbuf *src, struct rte_mbuf *dst) >> +pktmbuf_metadata_copy(const struct rte_mbuf *src, struct rte_mbuf *dst) >>   { >> -    /* Copy packet metadata */ >> -    rte_memcpy(&dst->rearm_data, >> -        &src->rearm_data, >> -        offsetof(struct rte_mbuf, cacheline1) >> -        - offsetof(struct rte_mbuf, rearm_data)); >> +    dst->data_off = src->data_off; >> +    memcpy(&dst->rx_descriptor_fields1, &src->rx_descriptor_fields1, >> +        offsetof(struct rte_mbuf, buf_len) - >> +        offsetof(struct rte_mbuf, rx_descriptor_fields1)); >> +} >>   -    /* Copy packet data */ >> +/* Copy packet data */ >> +static inline void >> +pktmbuf_sw_copy(struct rte_mbuf *src, struct rte_mbuf *dst) >> +{ >>       rte_memcpy(rte_pktmbuf_mtod(dst, char *), >>           rte_pktmbuf_mtod(src, char *), src->data_len); >>   } >>   /* >8 End of perform packet copy there is a user-defined function. */ > > Might need to redo these snippet markers as the function is now split > in two. Will rework this with the overall documentation update after moving to dmafwd. > > > >> +static inline uint32_t >> +ioat_dequeue(struct rte_mbuf *src[], struct rte_mbuf *dst[], >> uint32_t num, >> +    uint16_t dev_id) >> +{ >> +    int32_t rc; > > rc should be uint32_t, but this is removed in patch 4 of this set > during the change from raw to dma so it shouldn't really matter. I belive int32_t is correct here, since the return value from rte_ioat_completed_ops() is "int". Otherwise we would not be able to error check the return. If rc is negative, we set it to 0 before returning. This ensure that we have a positive value, which can be safely typecast to uint32_t before returning. Thanks for the review, Conor! > >> +    /* Dequeue the mbufs from IOAT device. Since all memory >> +     * is DPDK pinned memory and therefore all addresses should >> +     * be valid, we don't check for copy errors >> +     */ >> +    rc = rte_ioat_completed_ops(dev_id, num, NULL, NULL, >> +        (void *)src, (void *)dst); >> +    if (rc < 0) { >> +        RTE_LOG(CRIT, IOAT, >> +            "rte_ioat_completed_ops(%hu) failedi, error: %d\n", >> +            dev_id, rte_errno); >> +        rc = 0; >> +    } >> +    return rc; >> +} > > Reviewed-by: Conor Walsh >