From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B98E0A034C; Mon, 12 Dec 2022 14:28:36 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id AB06841143; Mon, 12 Dec 2022 14:28:36 +0100 (CET) Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by mails.dpdk.org (Postfix) with ESMTP id 664F540684 for ; Mon, 12 Dec 2022 14:28:33 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1670851714; x=1702387714; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=fBDnYXfxuB4rjlNv3JAvbmToGkhBes2oUYWNQBtPJqc=; b=jySY52IH3fOVhUCgaYFwFPxZlEIz8/mpVv/S1F/4+J+cgikLafagH+c7 0jySzWdRECKVht6jzzr1e7WwnJnlhXr5+1XQGJdyK78hqV8RmYHwH88iF rcdPyaKFQ5r68AF9bw0GRRkWbV239/Ds6xYY1r/vub/EqyDEZERIp2whN 6/1pOgpn5s6j1uVg1cVinJaK3qzd58nh+bj8bGydsz3+jTqmmREBPYkZ1 uWeQ4KBppM9cJT9bPg+st1m7abDf+lzwFOfC8TxkvNMhYhyLtYRWc1ure dHe9de+8Ph8TICVv8UjOSD4zGZPpCNw7hVmm8FqnOFNI3UD+5NNTz/KRU g==; X-IronPort-AV: E=McAfee;i="6500,9779,10558"; a="382159670" X-IronPort-AV: E=Sophos;i="5.96,238,1665471600"; d="scan'208";a="382159670" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Dec 2022 05:28:32 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10558"; a="711681206" X-IronPort-AV: E=Sophos;i="5.96,238,1665471600"; d="scan'208";a="711681206" Received: from fmsmsx601.amr.corp.intel.com ([10.18.126.81]) by fmsmga008.fm.intel.com with ESMTP; 12 Dec 2022 05:28:31 -0800 Received: from fmsmsx610.amr.corp.intel.com (10.18.126.90) by fmsmsx601.amr.corp.intel.com (10.18.126.81) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.16; Mon, 12 Dec 2022 05:28:31 -0800 Received: from fmsedg601.ED.cps.intel.com (10.1.192.135) by fmsmsx610.amr.corp.intel.com (10.18.126.90) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.16 via Frontend Transport; Mon, 12 Dec 2022 05:28:31 -0800 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (104.47.55.175) by edgegateway.intel.com (192.55.55.70) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.16; Mon, 12 Dec 2022 05:28:31 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=mt2mXvyDUMgRs8AlAgFTpwiGxZv0YCyZEIBYwbIg7rAO7eLQ3AzJ6tVQ2cmFIIIv9vn7kQh6G3bnGxDU4rHBFbL9ZE0cxuegjpjf/bGVI36NlKVswKutDhn2RUqTuHbg6Alfb6PftbY0iNOGDL7VXbUzNSWhjxiDqvTTX7IfDK1QGdC8eox/Kbg9LUB7aCzosCOPuoGELopG487W38WTEFvceaBkQ90prcO3NwZJUEZ4zQ2TglWXGpZqQ4pv7oaUQn4EmD00SD46xZhvNZoCX/nXTvlWB1tlL5tYCjwF2ols+lQTRc2QuBdCy4RpFbAdVyCubunmr36GA7rA/2K+Rg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=zb+FjTDTMZESdwSwJNFNjavrpEFcMhoESi9EKSJE+m4=; b=W73Jla1Xu1sDvN1HU6qlqRdkitmjzvC+7oxIivPPPjIFAAHZ5+reDp27Qdl89utpj47iHQ07eDAs0vciXS+l8Av6SkRHKjTCI8KJPay/Kkhf88nt26TB8VZ9rzLDG5ZaQVNd8QZsYBADr53pDH9rBL+rlLpom+q0P89dGwoJmw1WoCklna4cFQmYi2XLqNg3mSI+nu8kQIeJouCve0ijkdkwHk8uZexsY4tIL2jyHL3bYpfwI/pCTfLHT8nGqVHLR76y9CDnIFZmmb49xd1gllML0fyy4wnUkHlJ51F4khiV5CC7cuL79i1720GnWr+swtHuM6K1WTaF49JKw+7/kA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from DM4PR11MB5994.namprd11.prod.outlook.com (2603:10b6:8:5d::20) by SA1PR11MB7037.namprd11.prod.outlook.com (2603:10b6:806:2ba::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5857.22; Mon, 12 Dec 2022 13:28:29 +0000 Received: from DM4PR11MB5994.namprd11.prod.outlook.com ([fe80::f465:abe3:b917:a641]) by DM4PR11MB5994.namprd11.prod.outlook.com ([fe80::f465:abe3:b917:a641%7]) with mapi id 15.20.5880.019; Mon, 12 Dec 2022 13:28:29 +0000 From: "Zhang, Qi Z" To: =?iso-8859-1?Q?Morten_Br=F8rup?= , "Richardson, Bruce" , "Lu, Wenzhuo" CC: "dev@dpdk.org" , "Wu, Wenjun1" , "Matz, Olivier" Subject: RE: [PATCH] net/ice: support IOVA as PA mode Thread-Topic: [PATCH] net/ice: support IOVA as PA mode Thread-Index: AQHZDfwZAIM6DEegbUOU9ZoOYGrN565p5PcAgABaIYA= Date: Mon, 12 Dec 2022 13:28:29 +0000 Message-ID: References: <20221212154501.1303789-1-qi.z.zhang@intel.com> <98CBD80474FA8B44BF855DF32C47DC35D87580@smartserver.smartshare.dk> In-Reply-To: <98CBD80474FA8B44BF855DF32C47DC35D87580@smartserver.smartshare.dk> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: DM4PR11MB5994:EE_|SA1PR11MB7037:EE_ x-ms-office365-filtering-correlation-id: 33c60bfd-aa7c-4bc4-be3b-08dadc44c1a4 x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: f3qfQK8yuoEWpGyDDHLDMwztKkAIMtQMGKT3zDQrPeRVS+AC3t0/lCmcf79zG+rqf6J2huKxRloUH1fIaRhAEVO8tG1+WLwEutW0QFE7+bDUX04zJN10XGCkvhmjfaUAWLeEk97aKPY6nyKiGVxS5/LSX1rzGjopNQNM42ws/KsYs7tx//kQAxbeb7ueq8doL6dX1x0D2CGMGFCpr0tCGNDmBvMXKJq/luKDbCFHsV8t1Am+ujAim4aXOIUX93kX40Ojvu1PHpbtgz4LQJiBVc7Ak7aC0CKmesY/YtQAY/JFFV0PjL0B9Ho5gE+U0uqbadXA8X8JCF3KRkHH/rfrRajY/BbtcF0jY+S/tQF0pjF74SfTDJ6rXqRIavbLICUnwPZydxiAht53tW4ErUdQ6NfZKtnFk4U+mhQ6Mbi1ulc4CqgqtXNkRiRjbVzZ2SnGl0G4II5zQL6HoljvjywGXslTko2Q1bOfW7ZcD9TBSNPQeuqIj5bQw9onA9D0eEdmvR9fCRULhwa/Dd/7HdAoO4062U3Sii+JnTxqoTUQLLggfnH/WIU7F58cSqP98eICOJIKQvk5GTRCDyHtJWNVvyFLwUshxg3ZMbY2+qU1vkaQ7K/jWI1q0AloeUFm/cof+BBnGwec+/XXYVCbAht7QJ3A+uwen+eu1ojmHdfDT3WuOxkLk1K8ziy/s0yVxjlL+9UUZbRm3GYCcQ5UQB+Hcg== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM4PR11MB5994.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230022)(39860400002)(366004)(136003)(376002)(396003)(346002)(451199015)(66574015)(83380400001)(54906003)(110136005)(52536014)(7696005)(82960400001)(9686003)(33656002)(71200400001)(38070700005)(66556008)(6506007)(186003)(26005)(122000001)(38100700002)(86362001)(55016003)(66476007)(64756008)(66446008)(478600001)(41300700001)(53546011)(8676002)(5660300002)(66946007)(8936002)(76116006)(316002)(4326008)(2906002)(6636002); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?Q?5bdGy27dUql1KYtrJlYtZ4IEBc16QCB678rbZ5lHtN0f8r7Y5P2+uGPv/4?= =?iso-8859-1?Q?Dnv6tly4pfAY7nSIxEYfnWZYk/iHkJ2CT3hycIzdgBnXzOFjOz/b6guEq6?= =?iso-8859-1?Q?75Wpbq3zoJt8F+S8M8YZqtipK9GN/xBPt1h3l/QBr9YlaN3TIvWVET4xjj?= =?iso-8859-1?Q?GoyOmHOItmvKBGGmh1Rw/fmg7hzEnWoxqEz0RDbafHXRPdhpPzfDiWtkVB?= =?iso-8859-1?Q?BYKFcc1D8Zr1ts7cY/p3iXZqEbW/so7LKBXa9fd+bR/eXMOxtDom+v1e4P?= =?iso-8859-1?Q?/zxMvZbdsQdwRnx0vXXztgr6WiNeP4EtyB35akBtlh2nZvTT6BLJMGHf5F?= =?iso-8859-1?Q?mmkcB2S6MDqtacfL5oTdHpt7CR1g7pv5Lka+dALukpWW31zFa5sOA7iFjV?= =?iso-8859-1?Q?RMpv7dtrEhnaiBAdvNEAPKaDH1diZfVu0796xZe0WF9MLrR2DTLxGqlJO4?= =?iso-8859-1?Q?Kbn0bWIn8t82VG+ExWu5ly/CPXvGw3HlYPZYLYzUtcaeBpNLdIDRCpyYSX?= =?iso-8859-1?Q?UkSNu4ssEXPhgKAmntcJHkM4OAdN8zKzlmDbZT5JqKsm/aHLKDomsbSqEz?= =?iso-8859-1?Q?ICUJoECsi4xBoTiFziqM2aR1Aah24EUh0LrBadTJkLlvdGLEJt1F2ZwWD8?= =?iso-8859-1?Q?z1wDwF/5CaYyCHkvYlaU7Viv2YLfVrtVJ23IoPJR8m4dLmtHsVTIAtcD+b?= =?iso-8859-1?Q?eARsm5aZ5LK3Ai2dFtU2hYgqgWPqpdPVyf3QZrdW49QUmL33ELkBSwAJmA?= =?iso-8859-1?Q?mD3XPkx+iHkQXu8fKph/Wgw0i85fEtWLXnwwrOrPlMTDrjIS5Yzute/OwI?= =?iso-8859-1?Q?A4PInT32veOV6pYZSZkrrKR9Mr0Jy2yqwDSBGSFO9l/+GVixjAlJAFA4/n?= =?iso-8859-1?Q?hs13ZvGxLrLtVjp9nuSzkjhpZe2FUSx7y70Hb37MRuAJUl5ewx80mhSHXL?= =?iso-8859-1?Q?W0DQWkj/A2cZ8aRffCagHfrxWmzQOSZW+TdrtdPryh5htB3hTM4OK7Zz2V?= =?iso-8859-1?Q?1QpM0WB2LjrW1T1syK5TWQwrc8Dul+8LqvFF7LlZcr8ePvr2utcSzSuWmI?= =?iso-8859-1?Q?5j5WcyAPRhCiyIDsL2qTFv8OOciT9OK2RHnC5RL2bdIDwP6IupjaV3WCI5?= =?iso-8859-1?Q?JsfZfR8r3ZLngtVWM9N4qBpBvXRgrs1Afsbaqm0T/vI1smPC9UpM+qDEfS?= =?iso-8859-1?Q?1KQAv6xCXd3f2BD2QtUnLL4LT1lyjXN4jKOyEsIaYkUocS1gc/6h5SW4/9?= =?iso-8859-1?Q?qVRGuOkVMIU1Gyp1PSUq1+NkQ6OnTfngf4lLbAFxrqXrPXzJNMFgQb6izm?= =?iso-8859-1?Q?JAaK3j4CO1ZkfVSxuh3YhszDC/CvxjIwLdlN1VEd7t8otkTDCF57DdilT5?= =?iso-8859-1?Q?NhhMw8jNz2K/kCUt4hNd0LLBV+RPS9pfEGtzr+pttFQKxRSz2Tk41S72a/?= =?iso-8859-1?Q?+jiJFFF21Yj2t/OHd2+JvP6lA87SsKkZ02EGlIsz8QtjA/HkrluzxQ9JBC?= =?iso-8859-1?Q?BMUEELs8OU6fdtXLtKL1XIqCNpDJQebfuT6m+bpECDQ1Y/sGluDxo+NE+f?= =?iso-8859-1?Q?sNIuHpQYQbuqz22rIZJaw7Xk4d9SBQtPrTBioY3zLEmk1nIVeqW9q+2rdH?= =?iso-8859-1?Q?6H4+6NMO073IEcMjVFLRs4IEOL5M6LKid6?= Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM4PR11MB5994.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 33c60bfd-aa7c-4bc4-be3b-08dadc44c1a4 X-MS-Exchange-CrossTenant-originalarrivaltime: 12 Dec 2022 13:28:29.0407 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 21TqROgCy3eLCRFbkivIni/kWx1GFD526c4Qt6DjNbz6Jq+AFYwBYXEmD1g22Vb/LWGDbZIPP1d4CMmBuOSwsA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR11MB7037 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org > -----Original Message----- > From: Morten Br=F8rup > Sent: Monday, December 12, 2022 4:05 PM > To: Zhang, Qi Z ; Richardson, Bruce > ; Lu, Wenzhuo > Cc: dev@dpdk.org; Wu, Wenjun1 ; Matz, Olivier > > Subject: RE: [PATCH] net/ice: support IOVA as PA mode >=20 > > From: Qi Zhang [mailto:qi.z.zhang@intel.com] > > Sent: Monday, 12 December 2022 16.45 > > Subject: [PATCH] net/ice: support IOVA as PA mode >=20 > Typo: >=20 > "IOVA as PA mode" -> "IOVA as VA mode" >=20 > > > > Claim pmd_supports_disable_iova_as_pa. Remove buf_iova access when > > RTE_IOVA_AS_PA is not defined. > > > > The patch simply replace buf_iova with buf_addr at no IOVA as PA mode. >=20 > Please use "IOVA as VA mode" instead of "no IOVA as PA mode". >=20 > Also in the other PMD patches. >=20 > > Some SIMD instructions in data path may be over used, further > > optimization is expected. > > > > Signed-off-by: Qi Zhang > > --- >=20 > [...] >=20 > > @@ -868,16 +867,12 @@ ice_vtx(volatile struct ice_tx_desc *txdp, > > > > __m256i desc2_3 =3D > > _mm256_set_epi64x > > - (hi_qw3, > > - pkt[3]->buf_iova + pkt[3]->data_off, > > - hi_qw2, > > - pkt[2]->buf_iova + pkt[2]->data_off); > > + (hi_qw3, _PKT_DATA_OFF_AS_U64(pkt[3]), > > + hi_qw2, _PKT_DATA_OFF_AS_U64(pkt[2])); > > __m256i desc0_1 =3D > > _mm256_set_epi64x > > - (hi_qw1, > > - pkt[1]->buf_iova + pkt[1]->data_off, > > - hi_qw0, > > - pkt[0]->buf_iova + pkt[0]->data_off); > > + (hi_qw1, _PKT_DATA_OFF_AS_U64(pkt[1]), > > + hi_qw0, _PKT_DATA_OFF_AS_U64(pkt[0])); > > _mm256_store_si256((void *)(txdp + 2), desc2_3); > > _mm256_store_si256((void *)txdp, desc0_1); > > } > > diff --git a/drivers/net/ice/ice_rxtx_vec_avx512.c > > b/drivers/net/ice/ice_rxtx_vec_avx512.c > > index 5bfd5152df..e76ba1a245 100644 > > --- a/drivers/net/ice/ice_rxtx_vec_avx512.c > > +++ b/drivers/net/ice/ice_rxtx_vec_avx512.c > > @@ -55,9 +55,13 @@ ice_rxq_rearm(struct ice_rx_queue *rxq) > > return; > > } > > } > > - > > +#if RTE_IOVA_AS_PA > > const __m512i iova_offsets =3D _mm512_set1_epi64 > > (offsetof(struct rte_mbuf, buf_iova)); > > +#else > > + const __m512i iova_offsets =3D _mm512_set1_epi64 > > + (offsetof(struct rte_mbuf, buf_addr)); #endif > > const __m512i headroom =3D > _mm512_set1_epi64(RTE_PKTMBUF_HEADROOM); > > > > #ifndef RTE_LIBRTE_ICE_16BYTE_RX_DESC > > @@ -1092,8 +1096,7 @@ ice_vtx1(volatile struct ice_tx_desc *txdp, > > if (do_offload) > > ice_txd_enable_offload(pkt, &high_qw); > > > > - __m128i descriptor =3D _mm_set_epi64x(high_qw, > > - pkt->buf_iova + pkt->data_off); > > + __m128i descriptor =3D _mm_set_epi64x(high_qw, > > _PKT_DATA_OFF_AS_U64(pkt)); > > _mm_store_si128((__m128i *)txdp, descriptor); > > } > > > > @@ -1132,14 +1135,10 @@ ice_vtx(volatile struct ice_tx_desc *txdp, > > struct rte_mbuf **pkt, > > > > __m512i desc0_3 =3D > > _mm512_set_epi64 > > - (hi_qw3, > > - pkt[3]->buf_iova + pkt[3]->data_off, > > - hi_qw2, > > - pkt[2]->buf_iova + pkt[2]->data_off, > > - hi_qw1, > > - pkt[1]->buf_iova + pkt[1]->data_off, > > - hi_qw0, > > - pkt[0]->buf_iova + pkt[0]->data_off); > > + (hi_qw3, _PKT_DATA_OFF_AS_U64(pkt[3]), > > + hi_qw2, _PKT_DATA_OFF_AS_U64(pkt[2]), > > + hi_qw1, _PKT_DATA_OFF_AS_U64(pkt[1]), > > + hi_qw0, _PKT_DATA_OFF_AS_U64(pkt[0])); > > _mm512_storeu_si512((void *)txdp, desc0_3); > > } > > > > diff --git a/drivers/net/ice/ice_rxtx_vec_common.h > > b/drivers/net/ice/ice_rxtx_vec_common.h > > index eec6ea2134..ff530324ba 100644 > > --- a/drivers/net/ice/ice_rxtx_vec_common.h > > +++ b/drivers/net/ice/ice_rxtx_vec_common.h > > @@ -11,6 +11,12 @@ > > #pragma GCC diagnostic ignored "-Wcast-qual" > > #endif > > > > +#if RTE_IOVA_AS_PA > > +#define _PKT_DATA_OFF_AS_U64(pkt) ((pkt)->buf_iova + (pkt)->data_off) > > +#else > > +#define _PKT_DATA_OFF_AS_U64(pkt) ((u64)(pkt)->buf_addr + (pkt)- > > >data_off) > > +#endif > > + >=20 > You should use rte_pktmbuf_iova(m) instead of defining this macro. It doe= s > the same. >=20 > Excellent, thank you. >=20 > Acked-by: Morten Br=F8rup Thanks, v2 has been submitted to address your comments.