From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4D7FEA0613 for ; Tue, 30 Jul 2019 17:01:10 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 31ABD1C136; Tue, 30 Jul 2019 17:01:09 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by dpdk.org (Postfix) with ESMTP id 9FE601BFF9 for ; Tue, 30 Jul 2019 17:01:07 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id x6UExfMa030265; Tue, 30 Jul 2019 08:01:06 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : subject : date : message-id : references : in-reply-to : content-type : content-transfer-encoding : mime-version; s=pfpt0818; bh=ZNiU7iy5QFsMJHLjtHmZaxmTbVVFnntOmFgClAGLur8=; b=OtvG8TS5Qf5TwrEhE3NXt55ytsZ0EGxpPmHa+ZhTOgOB5KFqYvEMxWx7WeSSASkLFVoM 2/g2gIXO/hAwS9osUDBH3oEUeKwAFGq4hDM8a4lDIa594z7pB7n3F4UbSvR2+yJ27D+9 TYpjA4uUphCEDRsE4j9ZwzWfNzUgF2uPoO0WBwHa5pEAPe7lUo1eWEAilQSMEkX6Fewx HnSkU/oq4bEaV4WuzOHE0sO7WmtW4olZQBuQCadppZQ8piHjV4INRcp+3faxvZ8rxCT+ pIXTuoIbob0WJErgS6cD+Z2wU8zYlXIV/eidULA3euw1iEDwuotJQEXQJKoWegfyda2Y pA== Received: from sc-exch02.marvell.com ([199.233.58.182]) by mx0a-0016f401.pphosted.com with ESMTP id 2u0kyq5ajw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Tue, 30 Jul 2019 08:01:06 -0700 Received: from SC-EXCH01.marvell.com (10.93.176.81) by SC-EXCH02.marvell.com (10.93.176.82) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Tue, 30 Jul 2019 08:01:05 -0700 Received: from NAM05-CO1-obe.outbound.protection.outlook.com (104.47.48.58) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server (TLS) id 15.0.1367.3 via Frontend Transport; Tue, 30 Jul 2019 08:01:05 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=CoYdN4Om5udjxq1dQS1+nBzQf7tE0CdS51QtL/WGN8FfPZYznbJ6FetYBY4wGJSu/O7obLDhp/A0eCenAt0yMXTOEa5X0TU/i6x7b7bJ9+shFFCclXHGuoIb2y0tS3fAEK+QSbUiAPd4hIMqvmOlLV3/w1JdoBwiii9PXSwAzq3iIYAjUMd2EfFn9hE/pHTU1WBjGGVytWFk4ad8QJIeILYQKrWi81aBE0IbPwzabHJSXHrM+bjx5fLwDLw1gFx9jDPeSqptWBNB5H1+eb7jl+QvnckMZIpd3j0nRVLK3xOHnjrW8DqYn88qVKrS3eDEjiYiC10btFMGBofXgp6ZgQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ZNiU7iy5QFsMJHLjtHmZaxmTbVVFnntOmFgClAGLur8=; b=JUXrCpmrbKwwcIhEyBAxe/LEuzPnHpIHXmtS/oEe1BQEVRQdUjAK0auWYGkOK4fSaQDdhT7HKQhvjmWjbvTUrIr4j9n1tOyhkPmp/+7CydKfDoO8oMPcYlCn5TsRti/OEE+gmiPxYZffh/Z3rklmF1fDTr4Q7fm5aJOrnvtfZMApuk9v1ZtN2VNGNXUBJEguGGQbL23ei0Ws86+Pn4YI9njNwdL/JaedingT1S7UDaJC3FAeL4Jdee6BoT3vjqA83gtKSO6dLNG4mNPCv7HUmyu/+mfYz61961nrdaG1Vtpzf0MRYXF0361cC3MMz1rnwih1LVdSy+2usWAQ8moPlg== ARC-Authentication-Results: i=1; mx.microsoft.com 1;spf=pass smtp.mailfrom=marvell.com;dmarc=pass action=none header.from=marvell.com;dkim=pass header.d=marvell.com;arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector2-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ZNiU7iy5QFsMJHLjtHmZaxmTbVVFnntOmFgClAGLur8=; b=Vx5vA5p01ldXl8XEKP6X+MxI4DLAGozKVYYeAAIkZYGqIZzwMohzchFG7uCcJ5+GDjRh6J5RsxymV1NeBsH5r1q3JVutCX/nbMT1H6+FIu395s6aJrofjM6az8o+ui+L3eVKb3hN113gGCBZXbp+4n/0JzCEWyql6yd3pyhosic= Received: from BYAPR18MB2424.namprd18.prod.outlook.com (20.179.91.149) by BYAPR18MB2630.namprd18.prod.outlook.com (20.179.94.155) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2115.14; Tue, 30 Jul 2019 15:01:00 +0000 Received: from BYAPR18MB2424.namprd18.prod.outlook.com ([fe80::2d42:12b6:aa2e:2862]) by BYAPR18MB2424.namprd18.prod.outlook.com ([fe80::2d42:12b6:aa2e:2862%4]) with mapi id 15.20.2115.005; Tue, 30 Jul 2019 15:01:00 +0000 From: Jerin Jacob Kollanukkaran To: Marcin Zapolski , "dev@dpdk.org" Thread-Topic: [dpdk-dev] [RFC 19.11 1/2] ethdev: make DPDK core functions non-inline Thread-Index: AQHVRtpp1fovjVAwbUWiq/bDmyXr0qbjOsIQ Date: Tue, 30 Jul 2019 15:01:00 +0000 Message-ID: References: <20190730124950.1293-1-marcinx.a.zapolski@intel.com> <20190730124950.1293-2-marcinx.a.zapolski@intel.com> In-Reply-To: <20190730124950.1293-2-marcinx.a.zapolski@intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [106.200.247.60] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: fd7aa381-b10d-4fa2-2e06-08d714febc24 x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(5600148)(711020)(4605104)(1401327)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(2017052603328)(7193020); SRVR:BYAPR18MB2630; x-ms-traffictypediagnostic: BYAPR18MB2630: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:8273; x-forefront-prvs: 0114FF88F6 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(4636009)(366004)(396003)(136003)(346002)(39860400002)(376002)(13464003)(189003)(199004)(6436002)(7696005)(76176011)(8936002)(3846002)(229853002)(71190400001)(26005)(14444005)(52536014)(71200400001)(55016002)(25786009)(446003)(68736007)(486006)(186003)(476003)(6246003)(256004)(53936002)(316002)(2501003)(86362001)(11346002)(81156014)(66446008)(478600001)(14454004)(33656002)(102836004)(110136005)(64756008)(66556008)(74316002)(81166006)(6116002)(6506007)(9686003)(2906002)(7736002)(99286004)(76116006)(53546011)(305945005)(66476007)(66066001)(66946007)(5660300002); DIR:OUT; SFP:1101; SCL:1; SRVR:BYAPR18MB2630; H:BYAPR18MB2424.namprd18.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: marvell.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: DmERm4JgTp0ZoYuAjUJ3c/qegRUeJ/q0l38cHSxiQfL7QVLGMW1cqj9OCulkiMHqpw2CI92v3bqg+hnvCmi0XoAZDhQxSI8wK/PPE5ROkatJx4lNksjKp6+p12AKTU9e/TxhKtX47pKnYY8GLgO/NHmqh5FAdBnAsKAbzs1+iHjwQlKrRn5gA1HOoNliy5NnQu1v+2RPRIHT4jsjwt0llZVVXC7YmmJMCN42jQCUxaeKf76L0oEkOXeipcBx/KqqyHwjoi0Yya+ZrlOo8LhQhcPIb+nKaOUttYy+C7CXNVdAOnSm1NZfhK6Q/i0ghxv1VE+/9tVWy6qB4NqMfsoSaGT8zO7GgaRoQDWB25Ckh36AfUWYtbMYONY4OIBD3HejIDWgn/trT08VahdA1eDZo0KwyyM2vsARWdd5afrC+6w= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: fd7aa381-b10d-4fa2-2e06-08d714febc24 X-MS-Exchange-CrossTenant-originalarrivaltime: 30 Jul 2019 15:01:00.4502 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: jerinj@marvell.com X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR18MB2630 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:5.22.84,1.0.8 definitions=2019-07-30_07:2019-07-29,2019-07-30 signatures=0 Subject: Re: [dpdk-dev] [RFC 19.11 1/2] ethdev: make DPDK core functions non-inline X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > -----Original Message----- > From: dev On Behalf Of Marcin Zapolski > Sent: Tuesday, July 30, 2019 6:20 PM > To: dev@dpdk.org > Cc: Marcin Zapolski > Subject: [dpdk-dev] [RFC 19.11 1/2] ethdev: make DPDK core functions non- > inline >=20 > Make rte_eth_rx_burst, rte_eth_tx_burst and other static inline ethdev > functions not inline. They are referencing DPDK internal structures and > inlining forces those structures to be exposed to user applications. >=20 > In internal testing with i40e NICs a performance drop of about 2% was > observed with testpmd. I tested on two class of arm64 machines(Highend and lowend) one has 1.4% dr= op And other one has 3.6% drop. I second to not expose internal data structure to avoid ABI break. IMO, This patch has performance issue due to it is fixing it in simple way. It is not worth two have function call overhead to call the driver function= . Some thoughts below to reduce the performance impact without exposing inter= nal=20 structures. And I think, We need to follow the similar mechanism for cryptodev, Eventde= v, rawdev Etc so bring the common scheme to address this semantics will be use full. >=20 > Signed-off-by: Marcin Zapolski > --- > lib/librte_ethdev/rte_ethdev.c | 168 +++++++++++++++++++++++ > lib/librte_ethdev/rte_ethdev.h | 166 ++-------------------- > lib/librte_ethdev/rte_ethdev_version.map | 12 ++ > 3 files changed, 195 insertions(+), 151 deletions(-) >=20 > diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethde= v.c > index 17d183e1f..31432a956 100644 > --- a/lib/librte_ethdev/rte_ethdev.c > +++ b/lib/librte_ethdev/rte_ethdev.c > @@ -749,6 +749,174 @@ rte_eth_dev_get_sec_ctx(uint16_t port_id) > return rte_eth_devices[port_id].security_ctx; > } >=20 > +uint16_t > +rte_eth_rx_burst(uint16_t port_id, uint16_t queue_id, > + struct rte_mbuf **rx_pkts, const uint16_t nb_pkts) { > + struct rte_eth_dev *dev =3D &rte_eth_devices[port_id]; > + uint16_t nb_rx; I think, we only need to store 3 function pointers per port. IMO, Let have structure for that. i.e split the struct rte_eth_dev content as public and private. I think, We nee only following elements in rte_eth_dev struct rte_eth_dev_fns { eth_rx_burst_t rx_pkt_burst; /**< Pointer to PMD receive function. = */ eth_tx_burst_t tx_pkt_burst; /**< Pointer to PMD transmit function.= */ eth_tx_prep_t tx_pkt_prepare; /**< Pointer to PMD transmit prepare = function. * }; struct rte_eth_dev { struct rte_eth_dev_fns fns; // make it as first item allows type cast to s= truct rte_eth_dev_fns from struct rte_eth_dev =20 private ones } > + > +#ifdef RTE_LIBRTE_ETHDEV_DEBUG > + RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, 0); > + RTE_FUNC_PTR_OR_ERR_RET(*dev->rx_pkt_burst, 0); > + > + if (queue_id >=3D dev->data->nb_rx_queues) { > + RTE_ETHDEV_LOG(ERR, "Invalid RX queue_id=3D%u\n", > queue_id); > + return 0; > + } > +#endif > + nb_rx =3D (*dev->rx_pkt_burst)(dev->data->rx_queues[queue_id], I think, if we make driver funtions as (*dev->rx_pkt_burst)(dev, rx_pkts, n= b_pkts) Then no need to deference data from inline function. Lets expose a helper function from driver layer and let PMD use to access q= ueue memory. No need to expose that helper to user app. > + rx_pkts, nb_pkts); > + > +#ifdef RTE_ETHDEV_RXTX_CALLBACKS # If we have ethdev driver helper function for the same and PMD can call i= t as well no need to call this inline function. # I think, it make sense to as RX_OFFLOAD_FLAGS so that when app needs only It can be included in fastpath. # lastly we are not exposing rte_eth_dev to application then I think we can Remove rte_ from name. > + if (unlikely(dev->post_rx_burst_cbs[queue_id] !=3D NULL)) { > + struct rte_eth_rxtx_callback *cb =3D > + dev->post_rx_burst_cbs[queue_id]; > + > + do { > + nb_rx =3D cb->fn.rx(port_id, queue_id, rx_pkts, nb_rx, > + nb_pkts, cb->param); > + cb =3D cb->next; > + } while (cb !=3D NULL); > + } > +#endif > + > + return nb_rx; > +} > + >