From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 250C845E06; Mon, 2 Dec 2024 16:15:50 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C367F40270; Mon, 2 Dec 2024 16:15:49 +0100 (CET) Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.18]) by mails.dpdk.org (Postfix) with ESMTP id 68CE7400D6 for ; Mon, 2 Dec 2024 16:15:47 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1733152547; x=1764688547; h=date:from:to:cc:subject:message-id:references: in-reply-to:mime-version; bh=QrjgzvdJ66UfEcCIC2jNp1llNtaY8P60WFDz0DMpz8o=; b=M5OPeYwGm+JcI2Ftl1i2sf5HL/BtTCAz8iEvSZFwNH29KB0oCuMc/PsR iLVhffyVXasSw+kFXWVraL9Db0rj2UHZRJH86l4DEWETHBOmePZB+wuwO G1pjQtJyI7JylMM5QUDq1csdlYMQZVVAcMzdx8iP7bv0+amjbYRUbhgTU 3e1fIsWWJwxgHzMD5GV5/ChtoJhCYuq+oLlWjE6cN/GzVhhtzBUwKKvnI bzPB25FbmRUJ9tFvas0RPcoXAHOJBBY8BCox22D6bTwm8khkrIRV/6vZo BccT0BKvg+g0w/8gqiwO9/gQgOsIo39yAol1PQiPJFqTKczbt34J1vrp8 w==; X-CSE-ConnectionGUID: fhDZ5YeQSBugbcwIx1E7Lw== X-CSE-MsgGUID: WGsbGg5SR3ujqpxB6xM8Hg== X-IronPort-AV: E=McAfee;i="6700,10204,11274"; a="33474058" X-IronPort-AV: E=Sophos;i="6.12,202,1728975600"; d="scan'208";a="33474058" Received: from orviesa006.jf.intel.com ([10.64.159.146]) by orvoesa110.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Dec 2024 07:15:46 -0800 X-CSE-ConnectionGUID: wrNhpd+KT7q6/brALyN79g== X-CSE-MsgGUID: r2oEvc6tQkSPwNoWml+ABA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,202,1728975600"; d="scan'208";a="93226194" Received: from orsmsx601.amr.corp.intel.com ([10.22.229.14]) by orviesa006.jf.intel.com with ESMTP/TLS/AES256-GCM-SHA384; 02 Dec 2024 07:15:47 -0800 Received: from orsmsx601.amr.corp.intel.com (10.22.229.14) by ORSMSX601.amr.corp.intel.com (10.22.229.14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Mon, 2 Dec 2024 07:15:46 -0800 Received: from ORSEDG601.ED.cps.intel.com (10.7.248.6) by orsmsx601.amr.corp.intel.com (10.22.229.14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39 via Frontend Transport; Mon, 2 Dec 2024 07:15:46 -0800 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (104.47.58.170) by edgegateway.intel.com (134.134.137.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.39; Mon, 2 Dec 2024 07:15:45 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=MD6SYKuGueBJl4zWV+L0BGTRjJ1A4lzwXA679HTJZDsnCe17gGHlOT2lfm00cmZxHkwAo4UT6r9+b8OxDFklIa0NVTzPdNgFWFUgLgOouwvFi0kS4gSFzP8B4MsKxXIUl313DqcN9+Vwxv2+TVDJyGv1LSV8MBH0lMqsImU/NJyG+tIX+xEyU4ZLwfYvhatv8HVCckN+tNREfJq8doD//Nnmp+MZlz1uJN+tM6HGiFjgyt6023kl63zCKyGHJQjUzLP0t/infzi/pjiVb69lR0SeTc0sbLSuGoJlWykzvFvQpitydaMEz+YiZvd/HlWHKJ/EoQE7ovD8HA9CsIO8Dw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=pqs8UhTYffaGDHIrBwNwmog2ab3TWBkXLQUCH4uhfxo=; b=aUQXcG0apYQO6x1uSKw3+eNkaIGD8F4FVIv/D+16+3qdBZy6aj42vyVg3qfsAKK3tgfmlK70rN7O0FRcXJXXql/Q5Jm6NItvoJ8eY74qVbPYF7fQZJ6Kf+RdMIKouOMakaV9Jxb7VHvGzBW1kCBUkLi70UtEsUmHdu/uIBAwxVN4YFzh9f6jTLpm/18rvOb5gyMnUR0kFDwdauD/0LTqfS1isLXUKQz3v+1hrm4D2TWNi5lBLX/Ve5+IuJ/WYZefJeHG4ry3ZFyxENgM4+8OinVB/8NFDLSG1rTkMyoghpp8Hy8cUrTTzt/7WtUNP9eohQd7DXoawoBtkCq+HPh1NA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from DS0PR11MB7309.namprd11.prod.outlook.com (2603:10b6:8:13e::17) by SA0PR11MB4606.namprd11.prod.outlook.com (2603:10b6:806:71::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8207.18; Mon, 2 Dec 2024 15:15:41 +0000 Received: from DS0PR11MB7309.namprd11.prod.outlook.com ([fe80::f120:cc1f:d78d:ae9b]) by DS0PR11MB7309.namprd11.prod.outlook.com ([fe80::f120:cc1f:d78d:ae9b%4]) with mapi id 15.20.8093.018; Mon, 2 Dec 2024 15:15:41 +0000 Date: Mon, 2 Dec 2024 15:15:33 +0000 From: Bruce Richardson To: "Medvedkin, Vladimir" CC: , Anatoly Burakov , "Wathsala Vithanage" , Konstantin Ananyev , Subject: Re: [PATCH v1 09/21] net/ixgbe: use common Tx queue structure Message-ID: References: <20241122125418.2857301-1-bruce.richardson@intel.com> <20241202112444.1517416-1-bruce.richardson@intel.com> <20241202112444.1517416-10-bruce.richardson@intel.com> <69c37ea1-5171-42c1-a5bf-c5572fc2d037@intel.com> Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: X-ClientProxiedBy: DU2PR04CA0194.eurprd04.prod.outlook.com (2603:10a6:10:28d::19) To DS0PR11MB7309.namprd11.prod.outlook.com (2603:10b6:8:13e::17) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR11MB7309:EE_|SA0PR11MB4606:EE_ X-MS-Office365-Filtering-Correlation-Id: 1406ab47-0c89-4f5a-cdba-08dd12e42edc X-LD-Processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|366016|376014; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?in6fTAzihvBEkQBSnf9TD8vuM0by+pgWHVZNnzWdZnhlonRLg8uJ5NIzyave?= =?us-ascii?Q?+XLxQV7daGL2cIEOoJRFYt3ens6uzVZuI5xHHHc7tClDrQnIeaCXxKYqn+LS?= =?us-ascii?Q?MhIj7yQOXr3FAAPQDRNE4AhrZ2rg48ReqOJzaEvqrStXhTsmoW1LlC7L4zc7?= =?us-ascii?Q?ltwhp+OR3/vNGY6v62aKaFh1q3f9Rv4eU5rKJ+zm14oWLByJ5KcCwTSPaEiK?= =?us-ascii?Q?MPaUEtYIudvUZrQ/YVbaYOHuXk7WvorZv7j+F7NURHXQpGBQt6s8NFwzFvr3?= =?us-ascii?Q?5KUSaDpDDQNHS/CILEOevt8l/FSQgoZcWUbDEMdU0r8BbGfjXeVbpidyUB8o?= =?us-ascii?Q?PGV9IY6ZKOC1jyer49IvjNHOoRPYzJGjyZPQQxxX+x53M1zkfokN87lG25ih?= =?us-ascii?Q?hTHjIyUMBNs7k6B0BWqNK45yaFkx3USxMKw82PXn1unKO3Jx1tufJLSZUMK3?= =?us-ascii?Q?LZ0BkQSnpgItK9mfmswJ1ngCWKTRIJD/wRkGlUTF/cNSnr41MCqdarAJmbGd?= =?us-ascii?Q?ngyNLN72bNuOyoREMqJlNW2xkhTv83tGxwrjm9XEUN+RdRm/gmVOvz7W/Kht?= =?us-ascii?Q?oXFUX6FLLnPEGgLurB4E1nj90m64Kc5rM3dU14LSuENZvlZGPIVuavLi6b2a?= =?us-ascii?Q?nzJMi/LLZkqpwqV9LCfelLv/kPqwN/IMTWYNiOs+CBxJPaSckSpFXPwbeFhX?= =?us-ascii?Q?fLEf+yQnJ22Iei7Qi/Q/On40stH4dCCJbM4RjhbtknTnc3nhW8bcwJZaUdpO?= =?us-ascii?Q?uCT5Xz1clvA7O0Rv2z2yQO+TQErGIvcFIlEqijO7SpCKe50pIJUBkhFTTPnL?= =?us-ascii?Q?6XGmx8xOmhBy8Zue3Aj07eKMdjT0tdKvSy42ccZPW1+VLlUkl0VVI78Msnd/?= =?us-ascii?Q?gHKXTenp9fO9CaiV2WwaJ7W9orV8yCp9CgMRndMm4R7NK4SB00BmSco4RlNT?= =?us-ascii?Q?ZAxmOHAUEGjwGH0XclQPTiFABWZ8q6YmiEI0yCPuqP331eARtF45MbWLd2H2?= =?us-ascii?Q?AjN5a/U6woJGT47yRNZ3zs2pEv8vMtuXkpC+yLmjZob+z+1z35r6sQ+aEl8A?= =?us-ascii?Q?j2KCjrAjNayKwa8lnqsJ5tHy8LizyDmJVGTcqUaJrHwOknEk1vFZYyGXtLIq?= =?us-ascii?Q?2zd5J8xC4pEhwgVP5zEkF9Vrrwr63k1sQMdC8CKi4GG5WmYdAJUUiwjd9bod?= =?us-ascii?Q?E/fApS+nuW6IFo23MKiQxG+xWiqGsuBcm1Y0tqdTvsMuJWHKDndzzZQSQW3M?= =?us-ascii?Q?UfQ7aZHpCYRVZRKSaJhVwRNRXemOibiBn2LtMSoJCiTmQNEVu50JhBjMuDXC?= =?us-ascii?Q?vxOXYiOU2dmOo9AfzPHOYRllGNK3JJqXg1jqYIYT1Z7KOg=3D=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DS0PR11MB7309.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(1800799024)(366016)(376014); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?RJT5VLWZChMxOdyKMrN76dlqcWBwnWxKPebzyankKsGJRA4Sv/J5tgbUkwxU?= =?us-ascii?Q?ihWPgM2KRegIb+EopZ1RHGnl7Z1br42LmMeODMejm8yv8QcHqZjPZYCHhnIR?= =?us-ascii?Q?t/lnR4Os5cCrHNMznH7co2eIbNo5g3y377V9HQ2K+xgnqewfOCYZlPi8r59S?= =?us-ascii?Q?lnIDUAi4nkMQMysj4vhrIX6p0ju120gdAe2zajvGeXNXbA5ypKZgEw5fNLwQ?= =?us-ascii?Q?a+qhaXBpFyz+lMIJ0+KLUqpKOG0zyeZUvTwHIhm/gw+coDmIrzL2gY8g/9tl?= =?us-ascii?Q?oUBdln8dB+Z4Ly37IE1pqRwWrpMI2T7SZeEi+fhzI+R4Wbzjf9QCifaabnDy?= =?us-ascii?Q?g+1EHZL2X2uyf+6XWc84X/rpq5poNryeVetFEgU3bHWz20lAtqMi2ZOuAcDz?= =?us-ascii?Q?dZmKoWvN8e1zmez5pFWDY28dyDCqzbsOLEqdwnwt+1JgvDe0uh3WSX7jVKc0?= =?us-ascii?Q?gaE2e+Ryu1LDSHjbbByvmXYq1tOc4Q1ZHgVBpOo7u3LJaCf5vyaQfyl5gGOJ?= =?us-ascii?Q?i8pjzC0Hd/Eik1Ps4WXRU3tS7qSoW/bYz9J4hkz0Jt4kU8+m+ECo4JWUK55p?= =?us-ascii?Q?VjxFETEteFMSNeoBxuDfB6knupvm6gsO8QT5Lg9BXj1QSRaGMzzYKiSP4QGB?= =?us-ascii?Q?WjAGchIPJmhU1qUAsCYRlzUkRAQS3uYjz5T3Y2aPsCtO9gfz9octha+Nwjix?= =?us-ascii?Q?HmS4rF88NGZy+IM94X4vc0uCRYkUkVdE2XxE2ziqpQjhHcygZfTO62U2fnW9?= =?us-ascii?Q?gI1NxOQbqFUZ75Id4iE5DiHgHdDk7Urf2gbWrM3cVigkC7F16Uw8na+Kh+q9?= =?us-ascii?Q?gqT0k9RUxYRBPKKQFWo/jEF6fr3PTu+4lQBKt+Jy0Q8AiqZBxC0ySc+cWCPe?= =?us-ascii?Q?1Ox2u/KQN96P+xPt9FWIWpgvhvV7EHMbmM+M3BFwmyucDsH8SVxKu7CdD78C?= =?us-ascii?Q?swFfcWq0mlzogZW3ZfEEVX63pDR64NJE5wFgSjDQ9/N8ZgYLMI+nJ3yANKpJ?= =?us-ascii?Q?d4qwBE7eAT2RyW5H7R8fm+5ou28RTrGH7qOsUJIHwwsrJ+fyYT338vM9t2xl?= =?us-ascii?Q?RbJ0hPNuevRPG82YZihFnp9shRjAz54NwtnbuT3PNrWWbt4mCowXFDvtp3K5?= =?us-ascii?Q?yAzx1ZAhPemz3vy0ooLnjHGll1a5Zy0TlHIio0QluKj79H2B2k+sJ8U5XcUU?= =?us-ascii?Q?DpEIUeSpiXQHIzDhTtGR3J9mgA1LQvc38oBIGd0orAhUF/FwJpO7uE5TYESj?= =?us-ascii?Q?kTuNewPD3fpcmFlvjXzVRRrpHtE+OG+icOUaHJq2jlCPEKbl2fqKq9rTmfCr?= =?us-ascii?Q?IInm+32EJiteIycAoqyLNpfaywBsp27l8PrJi+pA62VXe/xFx4+fRARtBv7m?= =?us-ascii?Q?6F9VdTpwB/cOLSSE+x+y+sKnui+zw5Oau7reGYA8DE+b0o81vruknexR2XRw?= =?us-ascii?Q?wa9C3PvnU/nojDxWn7La2Ng0UmszU49IAaMB7IiSCcThooM6JW6xxujuoQb+?= =?us-ascii?Q?41WX7S+n220J6gMb9EWWFc2gyRZNkwiK8ecKQVy+J0qjhM13mnyXQW16os+p?= =?us-ascii?Q?TFAPNBJVKLCRbge/sGM7XwJUin98fswKVaNZpMmXsK8kmeF4JG6BFhCNX31k?= =?us-ascii?Q?SA=3D=3D?= X-MS-Exchange-CrossTenant-Network-Message-Id: 1406ab47-0c89-4f5a-cdba-08dd12e42edc X-MS-Exchange-CrossTenant-AuthSource: DS0PR11MB7309.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Dec 2024 15:15:40.8045 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: IjkreLba0PRQEdMQfNYn+DWGGkqi8VVV+b66nT+QI81g882wPhUebuhuT7jwRxI/d0DCVzOIS8esbDjCvgWx7nCPN4lk7u2u3FIIz/8o/nY= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR11MB4606 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Mon, Dec 02, 2024 at 02:09:35PM +0000, Bruce Richardson wrote: > On Mon, Dec 02, 2024 at 01:51:35PM +0000, Medvedkin, Vladimir wrote: > > Hi Bruce, > > > > On 02/12/2024 11:24, Bruce Richardson wrote: > > > > Merge in additional fields used by the ixgbe driver and then convert it > > over to using the common Tx queue structure. > > > > Signed-off-by: Bruce Richardson [1] > > --- > > drivers/net/_common_intel/tx.h | 14 +++- > > drivers/net/ixgbe/ixgbe_ethdev.c | 4 +- > > .../ixgbe/ixgbe_recycle_mbufs_vec_common.c | 2 +- > > drivers/net/ixgbe/ixgbe_rxtx.c | 64 +++++++++---------- > > drivers/net/ixgbe/ixgbe_rxtx.h | 56 ++-------------- > > drivers/net/ixgbe/ixgbe_rxtx_vec_common.h | 26 ++++---- > > drivers/net/ixgbe/ixgbe_rxtx_vec_neon.c | 14 ++-- > > drivers/net/ixgbe/ixgbe_rxtx_vec_sse.c | 14 ++-- > > 8 files changed, 80 insertions(+), 114 deletions(-) > > > > diff --git a/drivers/net/_common_intel/tx.h b/drivers/net/_common_intel/tx.h > > index c4a1a0c816..51ae3b051d 100644 > > --- a/drivers/net/_common_intel/tx.h > > +++ b/drivers/net/_common_intel/tx.h > > @@ -34,9 +34,13 @@ struct ci_tx_queue { > > volatile struct i40e_tx_desc *i40e_tx_ring; > > volatile struct iavf_tx_desc *iavf_tx_ring; > > volatile struct ice_tx_desc *ice_tx_ring; > > + volatile union ixgbe_adv_tx_desc *ixgbe_tx_ring; > > }; > > volatile uint8_t *qtx_tail; /* register address of tail */ > > - struct ci_tx_entry *sw_ring; /* virtual address of SW ring */ > > + union { > > + struct ci_tx_entry *sw_ring; /* virtual address of SW ring */ > > + struct ci_tx_entry_vec *sw_ring_vec; > > + }; > > rte_iova_t tx_ring_dma; /* TX ring DMA address */ > > uint16_t nb_tx_desc; /* number of TX descriptors */ > > uint16_t tx_tail; /* current value of tail register */ > > @@ -87,6 +91,14 @@ struct ci_tx_queue { > > uint8_t tc; > > bool use_ctx; /* with ctx info, each pkt needs two desc > > riptors */ > > }; > > + struct { /* ixgbe specific values */ > > + const struct ixgbe_txq_ops *ops; > > + struct ixgbe_advctx_info *ctx_cache; > > > > 'struct ixgbe_advctx_info ctx_cache[IXGBE_CTX_NUM];' takes only 80 > > bytes of memory, so using a pointer saves 72 bytes. Since the final > > version of the 'struct ci_tx_queue' without driver specific fields > > takes 96 bytes, embedding 'ixgbe_advctx_info ctx_cache[2]' array will > > take one more cache line, which is hot a huge deal in my opinion. > > > > Maybe not, though another way to look at it is that it is that those two > context entries are nearly as big as the rest of the struct! > > > Or consider another (possibly better) approach, where for non IXGBE > > 'struct ci_tx_queue' will remain the same size, but only for IXGBE an > > extra 80 bytes will be alllocated: > > > > struct { /* ixgbe specific values */ > > > > const struct ixgbe_txq_ops *ops; > > > > uint32_t ctx_curr; > > > > uint8_t pthresh; /**< Prefetch threshold > > register. */ > > > > uint8_t hthresh; /**< Host threshold > > register. */ > > > > uint8_t wthresh; /**< Write-back threshold > > reg. */ > > > > uint8_t using_ipsec; /**< indicates that IPsec > > TX feature is in use */ > > struct ixgbe_advctx_info ctx_cache[0]; > > > > }; > > > > + uint32_t ctx_curr; > > +#ifdef RTE_LIB_SECURITY > > + uint8_t using_ipsec; /**< indicates that IPsec TX featu > > re is in use */ > > +#endif > > + }; > > }; > > }; > > > > I prefer solutions where the extra 80 bytes are only allocated for the one > driver that needs them. I'll see if this alternative can work ok for us. > Trying out this solution, I hit the problem described in the commit log of the previous patch to this one - it introduces a dependency on ixgbe structures inside the common driver. By changing the type of the ctx field from an array to a pointer, we remove the need to have the actual type defined at compile time - as long as we never dereference the pointer. This no-reference is why, for example, we have have the union of all the different descriptor types in the structure without having to include the headers that define them. If we include ixgbe_advctx_info as an array rather than a struct - even as a zero-length array - then we need to have the definition of the structure present at that point in the code. This means we either need to: - copy in the definitions of ixgbe_advctx_info and ixgbe_tx_offload into our common header structure, or - put a #include ixgbe_rxtx.h in our common tx.h header file. [And it can't go at the start because that header itself includes tx.h to get the definitions of the queue types, meaning that it can only be included half-way down tx.h] In any case, either approach introduces ixgbe-specific definitions into the common header files. Therefore, I would prefer to keep things as in this patchset, giving us a smaller structure, and a clean separation of driver-specific and common structures. /Bruce