From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5E339A0542; Sun, 9 Oct 2022 11:14:54 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 555F54021E; Sun, 9 Oct 2022 11:14:54 +0200 (CEST) Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by mails.dpdk.org (Postfix) with ESMTP id E985940042 for ; Sun, 9 Oct 2022 11:14:51 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1665306892; x=1696842892; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=AYVzNr2HbJ8rpRdexyao8DxsRtq2wcG8vPR//L8+Bvg=; b=Bh9RSv2FT871SIq4EP7h/tGmnWMjWSpW2ONB4Poi4au9rQ804THbrS7H tboBR0jGp22EvpAjDK79hp3rSASnyTKlDdccw49VE+Zx8vHY5Zt/5nGoA boglnT4LcEHA0UzeULcijnSJ5m0UAsjEx9pqqb57zYbRaa+9wo5X3vNRf 7Pr2aHbbAYsq7QehR7crRCr4kcFMjeLczvtnsKCexB1mfPPmvYr2rno3S BlOPZhHQkMRUrPIFZpZ/+hNfl9YwpjG1fGCBD88qUGUJiaBavu8sYoZcp GgklLqBTF8ulaaAwqj63/+Je/kutg8A/i1aToJscyjZaoWe2BCh128/ls Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10494"; a="283766482" X-IronPort-AV: E=Sophos;i="5.95,171,1661842800"; d="scan'208";a="283766482" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Oct 2022 02:14:50 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10494"; a="625657313" X-IronPort-AV: E=Sophos;i="5.95,171,1661842800"; d="scan'208";a="625657313" Received: from orsmsx602.amr.corp.intel.com ([10.22.229.15]) by orsmga002.jf.intel.com with ESMTP; 09 Oct 2022 02:14:50 -0700 Received: from orsmsx608.amr.corp.intel.com (10.22.229.21) by ORSMSX602.amr.corp.intel.com (10.22.229.15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Sun, 9 Oct 2022 02:14:49 -0700 Received: from orsmsx612.amr.corp.intel.com (10.22.229.25) by ORSMSX608.amr.corp.intel.com (10.22.229.21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Sun, 9 Oct 2022 02:14:49 -0700 Received: from orsedg603.ED.cps.intel.com (10.7.248.4) by orsmsx612.amr.corp.intel.com (10.22.229.25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31 via Frontend Transport; Sun, 9 Oct 2022 02:14:49 -0700 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (104.47.59.177) by edgegateway.intel.com (134.134.137.100) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2375.31; Sun, 9 Oct 2022 02:14:49 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=AYvXFoK/1jvskcXun6cqTLZaWk9+QTKAAvlOFWHzS3UYy+GVFr3bPKrmXT1EpRRiBVbiFUS1Uuv3do/p5ZcrBwUZ7MgIwDhDrHv5uzZ5/Y9T3jrIYV9E2mrlVs2t7NOxotmDd6ae4PwOD1zCAfyouIGKPhjLUZhQgjS7zEOOf/doD52nfkQtzP20fLHXt8UxU5t2kGuOtiFjNdXfepkijVU8uVXtEqvGNeaX8QgFE6nGyFSRkx6MbUoKtJrav0Dilj6/EknGr0s0JB+Ntg48snhpEvUal1tgcQ9cfqIFswoWETte5x4a/H0FvLSVjjTTgEJYFu0/dOuKr/ncMofaMg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=9cRmItOaGBfZejsg8SjMSKyffP3Wf/Je452r78NruWg=; b=nR9a92Fl69EBJoE8f11tTKdnZQdQqC2AgcNS2TrjeJgScmXvvA/ZEKVhuQ1FEWpVD8ath32lL8aQHGlQJ2pa88yoP8AZhEb5Jd+Mk6YcRKwKPnrvS0EoLeiVbLbFuQAe93DCF5vlyGtH8RB/JQZxsqbFYW4qr0I9eDA3c+MYZZGBfG62DFEdU29JnGqysw3f6p3rV9jpyROe2Wh6PPD+3L+hwqFQiIR/4D3I2giQhR84Vd8cCsgUErpwCI43b6gAqJ3xuTvvvmhEHTBiLqplYOXkhb8jiBm3AYojQ8WbGvIJnhFEkUvDFZmm6UCFvBc32844rEYyTC96eCbNyRFTpw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from DM6PR11MB3723.namprd11.prod.outlook.com (2603:10b6:5:13f::25) by MW3PR11MB4652.namprd11.prod.outlook.com (2603:10b6:303:5a::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5676.24; Sun, 9 Oct 2022 09:14:47 +0000 Received: from DM6PR11MB3723.namprd11.prod.outlook.com ([fe80::126d:f905:c1d6:dcb9]) by DM6PR11MB3723.namprd11.prod.outlook.com ([fe80::126d:f905:c1d6:dcb9%5]) with mapi id 15.20.5676.032; Sun, 9 Oct 2022 09:14:47 +0000 From: "Guo, Junfeng" To: Ferruh Yigit , "Zhang, Qi Z" , "Wu, Jingjing" CC: "ferruh.yigit@xilinx.com" , "dev@dpdk.org" , "Li, Xiaoyun" , "awogbemila@google.com" , "Richardson, Bruce" , "Lin, Xueqin" Subject: RE: [PATCH v4 7/9] net/gve: add support for Rx/Tx Thread-Topic: [PATCH v4 7/9] net/gve: add support for Rx/Tx Thread-Index: AQHY0kOGbELMWmhIykCE4qZEKC6EM64BeiuAgAQwjZA= Date: Sun, 9 Oct 2022 09:14:47 +0000 Message-ID: References: <20220923093829.3019525-2-junfeng.guo@intel.com> <20220927073255.1803892-1-junfeng.guo@intel.com> <20220927073255.1803892-8-junfeng.guo@intel.com> <3e2e6a34-168f-b939-2dc3-6af319d3471a@amd.com> In-Reply-To: <3e2e6a34-168f-b939-2dc3-6af319d3471a@amd.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-reaction: no-action dlp-version: 11.6.500.17 authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: DM6PR11MB3723:EE_|MW3PR11MB4652:EE_ x-ms-office365-filtering-correlation-id: fa05aceb-dc17-4d1b-eb5a-08daa9d6b691 x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: RgWoL3SQSa7R7xS7Wg7QyfsGZM0wnR4J0ofmcATXkxyXzaPDaaF9uTFTOuTeICCtEsBSul7ug7sYBrHMFt3t8LMTZi28OE+NfBNokEbLZK9uw9DpGuF+Vumh8Cr9uvCa4rOOQyVNQhEZP9Na8lh3G2Efd2HroUswDDB91AgExD6Dq/jwhFQcVkyIUuEZCO5xw1b3TtQqctZ/gfKRwWnV4N7n1A/Z/xG1ZubyGXbNYcvjoG13+voOQcvtNXn+7yoaMu5rXIXVsdX54iei2tgNCkkvm7myFOub5QraI9Erw/MjD45pqBun8CoGR91gHfB7Zyz/MqdynmSo+QKL6Ru2VGnK/aCO6M0imqzM1025aDkaJFRCbPRAhSPlZP4lCCStaYDIJCNZhlRtHrI5nX7muAtjj4h3Rh72NxfJ/xrmc+TxGjUtoUMG2JcFOTOMeAVoIkOo+7jikD9Y8xrpoA3wmkjBMteZ+imQhcFSVigZX28v+JftC7j/KQfIPAca0KSdcKir3EoIAJfrj+EmEVxLe9MRdZfRejsrI/83xV4wDGptIzne0ZcBOtgAg10nKg/vvqIYcBfAU5dCfvATkmsUXscBFM7XNBlCQdOKz/Txxc1iuzxztP0K8FUAU0p3J/PaQ60xiAsBszA5/WADDYRmk+r+XZP3a/j6cUcFGArYFfrhmXWfccFIhl/IKFzIM5HiGPAcMxHTJEQJaCnu02q3i78tMGLKHUuILgItzWo7vRX6HVEMz1eg4eJhQjBceMrndCEW90oMZJl6xr9MFpVPbw== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM6PR11MB3723.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230022)(396003)(39860400002)(346002)(376002)(366004)(136003)(451199015)(53546011)(6506007)(9686003)(26005)(478600001)(7696005)(107886003)(83380400001)(186003)(5660300002)(2906002)(55016003)(316002)(110136005)(54906003)(6636002)(71200400001)(8676002)(8936002)(41300700001)(52536014)(4326008)(66946007)(66476007)(76116006)(66556008)(64756008)(66446008)(38070700005)(86362001)(33656002)(122000001)(38100700002)(82960400001); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?PkXSsMx6KRUr7usXLPz2NFuC1nYm8H2RbjURGeJnsRxkRJHwnjB4uUfmPGwR?= =?us-ascii?Q?aRsHbEZ7gvqizBGV0Aot+DrIqC7vthLGCKrtXAq5bSShiZus5b3Q1po01cIL?= =?us-ascii?Q?Pb0EX3DRU2Z2DyZ3jM05awX6ulSYRWh2c9xzd9S6CTyp3H+nqBTRv+lPDL1J?= =?us-ascii?Q?eVmSk5mXZOpMAoGMi0ebeQRNHROAdGDbyiOzxTKfBhxElTs3hzSm/Gya6nvQ?= =?us-ascii?Q?Lkk5Ti6v8CgufHdgqBz8KLJ2vNxYCoJzPm/6EZeUkZftW/eRdQqOw47KUegk?= =?us-ascii?Q?ekdQUER2qEJXjnoUmdzkpOe8kTi1UHb7fg5N0PMEA86x1lbXB3XsXWxGPR1v?= =?us-ascii?Q?4eoKWc13mr+nfALhZmgHsqRhoH4lge+La4oOL1R/cdOE6RH0uFtywdr7pLqz?= =?us-ascii?Q?sflghZmzmucU/IwMgytvIdDS+1QZ8OOC8b2JaDhI6kjiKkC6/tUbndVZpuHx?= =?us-ascii?Q?kYc53Y7LGDz/fGMpKiT8CsJZUQ6ijYh6JXs9LF6TAb5P70VjDamkeQu/E4WC?= =?us-ascii?Q?Td6lZcQ+HCcja0zZH+zSSQBYZJcMDsxWnJqPh0uejB+zLQM4y7v05tMjB2zX?= =?us-ascii?Q?P7rfXjSRPDrbeuKUywhmLGfpuBNdU796+QJ7N2UXW+27rYAKvHQ61MizfPXb?= =?us-ascii?Q?/BtEHd4eLEuLA2tPkzI+xPz0W6Jw5av64za5cO5+gVF4HnMgf1fX2/b0FRbX?= =?us-ascii?Q?dTi0U7WKTcBcr+kVAiertA8ui+Ka840SFruSZkbbzDgtaYtzdXiCM/XLs5tO?= =?us-ascii?Q?xJZPa71dJnV/jC36J8hcdeOpLvRJj8QCDN4j11EHkL3FKQbYy7hByupp6R54?= =?us-ascii?Q?RRgBHxLxboR2hEAkMQuDaojtAzqdwzL1j1uGXPXBgJOEcxGG/8JCrt64gMSu?= =?us-ascii?Q?klxt7m1G6Tx4LDS4G/RUxK+a7/9Bw9eb92brT0s2A1fm6X91B9yfWXtbCAoy?= =?us-ascii?Q?gWVw3aQ7t4TUrPLcLEU9Bnds3ZikVJlNqaukhPgUGut0MyaNv63NAWKc/Exb?= =?us-ascii?Q?Uihfbt8S2JBTG4IqwSeSgIpohOMs1XpaiRtbZZUKR2nhQpa067/z4kI1i355?= =?us-ascii?Q?Kxc3y7UXWakc9o7KOqwTn/csHBRGF7JwTUmPOgthI0qTQjKfN2F1Ejbt7oPr?= =?us-ascii?Q?+y4k3k5GVgUBXfcd9L3Db5UqIkt8CRKBDFrhFWPcwD9neoRXUA0GzWQOftDF?= =?us-ascii?Q?1BvFJLtT7G42wFxUkKdUbXom2TN5e6OchDAB4zAdL3ewrZmYIdCcx7VEZP30?= =?us-ascii?Q?8QTtUwXHqkfT1sOAxv5mH6kmPy1VNk2Ke5adYKbWSbRRFmPbnfnge0QVBJ89?= =?us-ascii?Q?+w0Zi5AzXA0foLuMZ9g+/F79li1EjCPXfMuyw0llaU/Ov4FTr8aP/8CWxp7j?= =?us-ascii?Q?qJfjLz4aP5qg0kaK3s1VDwyd5OtYMWwFu83V5cKYq2KljGRmZ4Xd2jd6Sn4m?= =?us-ascii?Q?TK4Hgts1hWhle+LgOnfL1dr1/3nh89iKjyBE09292LAzmMLc9GjpXtm2redv?= =?us-ascii?Q?/PcO4LVvUpY6N3pP+2znLi318dClN8QoFTQCh9Fy07Q9xryQqFBt0Y+c1bFp?= =?us-ascii?Q?nG5V9MjL1P/67IsQoarxoyl2GH2TBrpM0RyNMjOu?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM6PR11MB3723.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: fa05aceb-dc17-4d1b-eb5a-08daa9d6b691 X-MS-Exchange-CrossTenant-originalarrivaltime: 09 Oct 2022 09:14:47.6989 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: XcbCSXr1DXkE65uCW8ugCvEjKrjHhvF9ylBDF5rpcpH39a5Xh2T9gfjy13fPpJC6Yo2Nks1F/lyg3dp7/mewDw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW3PR11MB4652 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org > -----Original Message----- > From: Ferruh Yigit > Sent: Thursday, October 6, 2022 22:25 > To: Guo, Junfeng ; Zhang, Qi Z > ; Wu, Jingjing > Cc: ferruh.yigit@xilinx.com; dev@dpdk.org; Li, Xiaoyun > ; awogbemila@google.com; Richardson, Bruce > ; Lin, Xueqin > Subject: Re: [PATCH v4 7/9] net/gve: add support for Rx/Tx >=20 > On 9/27/2022 8:32 AM, Junfeng Guo wrote: >=20 > > > > Add Rx/Tx of GQI_QPL queue format and GQI_RDA queue format. > > > > Signed-off-by: Xiaoyun Li > > Signed-off-by: Junfeng Guo >=20 > <...> >=20 > > --- a/drivers/net/gve/gve_ethdev.c > > +++ b/drivers/net/gve/gve_ethdev.c > > @@ -583,6 +583,11 @@ gve_dev_init(struct rte_eth_dev *eth_dev) > > if (err) > > return err; > > > > + if (gve_is_gqi(priv)) { > > + eth_dev->rx_pkt_burst =3D gve_rx_burst; > > + eth_dev->tx_pkt_burst =3D gve_tx_burst; > > + } > > + >=20 > What do you think to add a log here for 'else' case, to inform user why > datapath is not working? Agreed, make sense! Currently only one queue mode (i.e., qpl mode) is supported on the GCP env. Will add a log to inform this in the else case. Thanks! >=20 > <...> >=20 > > +uint16_t > > +gve_rx_burst(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t > nb_pkts) > > +{ > > + volatile struct gve_rx_desc *rxr, *rxd; > > + struct gve_rx_queue *rxq =3D rx_queue; > > + uint16_t rx_id =3D rxq->rx_tail; > > + struct rte_mbuf *rxe; > > + uint16_t nb_rx, len; > > + uint64_t addr; > > + > > + rxr =3D rxq->rx_desc_ring; > > + > > + for (nb_rx =3D 0; nb_rx < nb_pkts; nb_rx++) { > > + rxd =3D &rxr[rx_id]; > > + if (GVE_SEQNO(rxd->flags_seq) !=3D rxq->expected_seqno) > > + break; > > + > > + if (rxd->flags_seq & GVE_RXF_ERR) > > + continue; > > + > > + len =3D rte_be_to_cpu_16(rxd->len) - GVE_RX_PAD; > > + rxe =3D rxq->sw_ring[rx_id]; > > + rxe->data_off =3D RTE_PKTMBUF_HEADROOM; > > + if (rxq->is_gqi_qpl) { > > + addr =3D (uint64_t)(rxq->qpl->mz->addr) + rx_id= * PAGE_SIZE > + GVE_RX_PAD; > > + rte_memcpy((void *)((size_t)rxe->buf_addr + rxe= - > >data_off), > > + (void *)(size_t)addr, len); >=20 > Why a 'memcpy' is needed? Can't it DMA to mbuf data buffer? Well, only qpl (queue page list) mode supported on the GCP env now. So the DMA may not be used in current case. >=20 > > + } > > + rxe->nb_segs =3D 1; > > + rxe->next =3D NULL; > > + rxe->pkt_len =3D len; > > + rxe->data_len =3D len; > > + rxe->port =3D rxq->port_id; > > + rxe->packet_type =3D 0; > > + rxe->ol_flags =3D 0; > > + >=20 > As far as I can see 'sw_ring[]' filled using 'rte_pktmbuf_alloc_bulk()' > API, which should reset mbuf fields to default values, so some of the > assignment above can be redundant. Yes, some fields are already assigned at 'rte_pktmbuf_reset()'. Will remove the redundant ones in the coming version. Thanks!=20 >=20 > > + if (rxd->flags_seq & GVE_RXF_TCP) > > + rxe->packet_type |=3D RTE_PTYPE_L4_TCP; > > + if (rxd->flags_seq & GVE_RXF_UDP) > > + rxe->packet_type |=3D RTE_PTYPE_L4_UDP; > > + if (rxd->flags_seq & GVE_RXF_IPV4) > > + rxe->packet_type |=3D RTE_PTYPE_L3_IPV4; > > + if (rxd->flags_seq & GVE_RXF_IPV6) > > + rxe->packet_type |=3D RTE_PTYPE_L3_IPV6; > > + >=20 > If you are setting packet_type, it is better to implement > 'dev_supported_ptypes_get()' dev_ops too, to announce host which > packet > type parsin supporting. (+ dev_ptypes_set() dev_ops) > And later driver can announce "Packet type parsing" feature in .ini file. Well, on current GCP env, the APIs for supported ptypes get/set have not been exposed even in the base code. The only one in the base code is for the dqo mode (gve_adminq_get_ptype_map_dqo). But this also cannot be used on current GCP env. We can only implement this once they are supported and exposed at GCP. Thanks!