From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9DD5442B4B; Fri, 19 May 2023 18:34:17 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 33C1B40E25; Fri, 19 May 2023 18:34:17 +0200 (CEST) Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2080.outbound.protection.outlook.com [40.107.237.80]) by mails.dpdk.org (Postfix) with ESMTP id EA18C40E09; Fri, 19 May 2023 18:34:14 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=UXz4PWbJNU+FwZgxyxw65bnCHEmSXS2DhEI7e7s3rDxo22qSV60D//t5rwsjXGuzQgd8sh1WX1Z1b0YBjkLTWhmSZSe1D252Q/q/fC8fpPDrTt9VnEe6fkA0X1xqq5CtZ3/GyK0eEEqBUP5FgfXQC/a1nTpdMLe64bi7YDoI8ng3HjnV/o0NggHeiRsFN5KekU8z/OyhyxZdUnCF6nbznnfARQMbe4CIuuIjFrG2V3w3sCYC0OoqJzu+a49EOOlEiaujTQw6ou8C1Uwwr3ht3OycNpkOsVxooO6XMgYnR4jlT8R8g3w/k3TJ72MT9oj9FPg/Xhl7NoPd0oIwOn05mw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=xej+VZfbzIyOpBWm+COazMb/i7+AlYMdpQIqrOGgh6w=; b=bLYZUFZQOFdWcEBdJvVjJRUJzh02Ag4+NlCGiSuElE/3tpYCiTxBFRI/EWw8Q609T+9HSc02wiYb4vk8bNIVyFAkS5DhAdVbNex06XGGZ0XoUmtesasfo3dkcp+loHGqKI0690DQ79/TtEGb3D3p6a0iVAQ/YxtzBjTjLnA0to8jvMe0bERimJxvpIYvGuUzEl88/78Zo6knJrIm5WH0PSMShQb+yzJKG+sGoXy+wm/KxayJ4I3E1TaMnGRp0uijWJjxB6sMBnTjJBq0Q/4eg9GU2SRSKfQSM2aiESWg/e22imto/pgOs7OQaoOEVdSCsQ8uAGsiDt3fTI2mTa9EVQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=xej+VZfbzIyOpBWm+COazMb/i7+AlYMdpQIqrOGgh6w=; b=JcxjNxL3xIDN7E7ydVKbYTJxl9xcC84Fhf2Rn/6+k+S+DMMaCqqMGvFWXTxYNi6Ygq+cEP5e831KcaI+zo4yxHHLwS5N2y99EAOZa2F3+optSEysYEaj5Q30woY4XMsRiaDirtQLVrzLh5WO0zW5Kj5vM4IG0YjvhUZycs5KutY= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=amd.com; Received: from CH2PR12MB4294.namprd12.prod.outlook.com (2603:10b6:610:a9::11) by CO6PR12MB5396.namprd12.prod.outlook.com (2603:10b6:303:139::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6411.19; Fri, 19 May 2023 16:34:13 +0000 Received: from CH2PR12MB4294.namprd12.prod.outlook.com ([fe80::7957:641d:6aba:3f9a]) by CH2PR12MB4294.namprd12.prod.outlook.com ([fe80::7957:641d:6aba:3f9a%4]) with mapi id 15.20.6411.019; Fri, 19 May 2023 16:34:13 +0000 Message-ID: Date: Fri, 19 May 2023 17:34:05 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.11.0 Subject: Re: [PATCH v2] net/mana: suppress TX CQE generation whenever possible Content-Language: en-US To: longli@linuxonhyperv.com, Andrew Rybchenko , Luca Boccassi , Kevin Traynor Cc: dev@dpdk.org, Ajay Sharma , Long Li , stable@dpdk.org References: <1682653009-19988-1-git-send-email-longli@linuxonhyperv.com> <1683336767-19526-1-git-send-email-longli@linuxonhyperv.com> From: Ferruh Yigit In-Reply-To: <1683336767-19526-1-git-send-email-longli@linuxonhyperv.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-ClientProxiedBy: LO0P265CA0009.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:355::20) To CH2PR12MB4294.namprd12.prod.outlook.com (2603:10b6:610:a9::11) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR12MB4294:EE_|CO6PR12MB5396:EE_ X-MS-Office365-Filtering-Correlation-Id: 5ef85543-ff34-433a-8000-08db5886e0d1 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: jS0+8Uzn82rQSF+NorrNuDMXWTifmsRakTqXYmsiG0LRLaEkMtvd9psAIm34jRGtNSZV9q++Yk+gkwzYYtDPYyL+6ueKJuDnIZsapisCPowC+pQP8gQ5gRYkut8e2BbJljmBnf5ehi/VV0G0OloIy3QCvviQ8BEtX4zNIEwKkvkFAOLLnqkbaTBtOp8zobcRzFSImwRFopzUXw1lbSilvfn/fb4W3CS4ePDnqlUncwCEkuvn8Np0GqYPr+toDPkLEjU8lqUQa1xig9OETQlDNADCMx3vjpPJi6u9DPehxCfnlmK+gKwt8ysOYHX2DLkRbiCSHgTqpAq/G6QAmjGfESmnnG8i80PLvw+MvNzOTY1u9mhisYGq3lLvIxFdh0aFsMK9AFKCQSM3Dtd1hRndtKEt8bTD15+DCtasjMxWY5rpsp7DTOMlB2+gjZ0gKn0z2whf5ZEjUg08U8YkibcKqeOQUg+2pTbvJkkugjllbNZKSZ3l96AZ8ZiEScSscuzXmb+8/b7se8UHFNtfTZbvT2mwF9jHeUwjR9mZdDpsLUuT42fING1WjbxvilH4YQHer5cdq30YEDDOM5PKmFYaadtxJSiHj9jsrsEtcKr1Jn3E0eXAKaIgkStQBhgpNqpmBSVbPSS+Ke7tbepfd6vRWg== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR12MB4294.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230028)(4636009)(376002)(346002)(136003)(396003)(39860400002)(366004)(451199021)(26005)(6512007)(6506007)(53546011)(36756003)(83380400001)(86362001)(31696002)(2616005)(38100700002)(186003)(6486002)(54906003)(110136005)(44832011)(5660300002)(478600001)(316002)(2906002)(4326008)(31686004)(8676002)(8936002)(41300700001)(66556008)(66476007)(66946007)(6666004)(45080400002)(43740500002)(45980500001); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?YkE0aHNmMmJWWXBSVy9rT0RoUVBnblZ3MGFwUFVJdFJBendBYXk0bFhkczgv?= =?utf-8?B?MWxLWHVCQmhGWFhKcm1SNCtpd1NhQlM4VGpzMnZTVVFtQkRBUkgxRnVwQWsv?= =?utf-8?B?VnhhQlJKeVNoNzE3aDBCSGZ5bW0wMXVuOVN2QzdsbzNpSUZiK0l5Z3BZVUNs?= =?utf-8?B?eU1za2VFbUQwWEdpdFY1aDExMXF6dFVub1JOUlJCZ0tBbEpUVEswOG1oWmQw?= =?utf-8?B?dVlaRFR6WmZnMEkyUTZlQndIanBNQ3RiQWpXWEV4TzFzM0tVWGNHd3JrQTNr?= =?utf-8?B?Rlc3d0dudjVNYWpQZkNUUG96OWRaaTlUeUwwSVBaWnJEd1RJV3QvMzJYYXRq?= =?utf-8?B?akkxTi9oejlYQ1hDNnM4MUFOTXh2UFBaVTBCM0Z6QmY0RGNmTGtDMDNYelNT?= =?utf-8?B?ejBGb1Z0UXpLZmRXY1EzNGF3NjEzamIreHAxTVRZTmQyQm5BM3NXZHNGMHoz?= =?utf-8?B?cDhORHlQM3pYQUxZenZDb2swVmtZZEJ5NkNjbTNibEhEUDdvWko3YytYL2hp?= =?utf-8?B?KzBTK3E1SmJvTUhRN1F5WCthWTZuajhjcWxBV1VhZ21vbXZheTFySmNTeUcx?= =?utf-8?B?ZlJxOFpFU2ZvYzlBUDBVdVo1SUtIaGd6WGk0TTdWclJXUCs0TS8wNFZXTzJG?= =?utf-8?B?ektqTXVVV0xDNHArYk5rajNzdEkyc3pPN2J2UnlFYUwvZFUwd2JncmFuTy9p?= =?utf-8?B?KzVoYjhxQ1dFNlNrL0d0Y3NIMW5DYzZPaWZpdjQ1REhXVHBmNDZqR29Qcm9z?= =?utf-8?B?MVh3aEgxWTl5YjlsR3dKZmxZU3ZyUW56MDc2QWhsSlhxeHpaY1J1UkFnRm1G?= =?utf-8?B?Y1FmNXR2WkROZTlYNUNKdzMwMmlscUMyZGNsS1BrWjZ0VHFZaXFKa0Y0UWh6?= =?utf-8?B?L3FCdUNaTHFkMnFTaGhUT1Z3N0hwaHZPWmVPTDVVOEhzelRWUVBsTWJ5dXBD?= =?utf-8?B?RVRnTmllMm1BQ2o1c1BNQURRajRrNHlWWkVWekVXRnN4U1AzdHZXQk9ubkc0?= =?utf-8?B?U2Yyb2MyQ1lTcWcwR2Y4Zm0xL3k4Y2h3VGEyeDNJV3Q1V0hocnNQTzdQOXNM?= =?utf-8?B?blVWU3ZaUENQY2lZaEhiQitHZlpOU0E4RU80Vnp2YzluRzdQODJVam05TzNP?= =?utf-8?B?Nmc2aVhDVGZSdUU5eUdObWNjUUZvdnI2YXNSN1hZV0J5VDJMNWxJdmxGb29N?= =?utf-8?B?TFBnQmRBWEtzRDMrMFNNbnJPdkFST050YWlIdDdUQWRNWkQ0blppWU5qL0tm?= =?utf-8?B?Qkw1SEdQNDZCdEUyLzRib0ZCZkNWeEJJa09EWlgxbDB3V2s1NFJUSVBBeFJX?= =?utf-8?B?dHcrbWVPbG1qbWppTlpFT0QyYkNqT0YvOFVvVnVhZGJrT2cvYXZ5VlR4T3ZL?= =?utf-8?B?UE9hK2swajFFQ0I4WXFuRkJwWnk5RDVxZXpIanBSMU1CUzY1anhkYXNlb1A5?= =?utf-8?B?TlRabHVjckgrSVkrcUoyQld6MktPbVQzYnpMcWxTOGJyRWlvbFZuU0JIUW9W?= =?utf-8?B?elVocmNaZFo1RzlSOThKVm16bW1ZSU55Y0JjVFZSb2lBdFBhOG50a2JBOFJZ?= =?utf-8?B?WmR6UWJOZHlPazFHQnBNaUs2NCtFc09UMjhtU3BRazhTeld4b084OHU2UlF6?= =?utf-8?B?ZVZRbXp6cW9RVUxsUU0rRWVnaTZXWldnU1pMZmNGaDJLc0VienZ4SkRIT3RV?= =?utf-8?B?UUNheG5tL2s3R05jK3Y3TlhxZjZ6aXpjc05tWUFUaUpOYmZxcjBEMytZUW1p?= =?utf-8?B?eHNWVXhLWkhMTXRCQzZwSjg3K25CVENza0x3WVlIY1dHcHZUbXVyaDA5VndU?= =?utf-8?B?c1VSa1BoL0FYdnh1aGpLWmdyeDg5S3A4Mi84TWxiT0R0enM0eGxjempIaFIv?= =?utf-8?B?aHJPMjcxdktsQ2VRcnowUGNIWk4vWElPTkVaeUIxVkErM0J0V1d3VUlibCt5?= =?utf-8?B?QVRhUTdiWmZiTFNYR0F2ckpNQmpiTFFxVks2RThhK2pNU0hPVVZuYjRNL09W?= =?utf-8?B?RkROUUc0Rk4vOW5EdCtiaC9XRFhudDhJY28wVjlla3RKQVNBVE56bTdXT1Q0?= =?utf-8?B?UEN1d1g2YVBGLy9rbU4zQlEwejRvQnNoSCt5Q25qa0hheUNwUUVZNTdOMXB2?= =?utf-8?Q?S6vjfoJ8rOXbNdrllQ2qP7SqL?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 5ef85543-ff34-433a-8000-08db5886e0d1 X-MS-Exchange-CrossTenant-AuthSource: CH2PR12MB4294.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 May 2023 16:34:12.9440 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: fs4Lq0naIpvLclgRKAbjKHxd79LdMPTz4zd3VkO5Hj9LtLpvxkC0RzFPRFF2trx1 X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO6PR12MB5396 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 5/6/2023 2:32 AM, longli@linuxonhyperv.com wrote: > From: Long Li > > When sending TX packets, we don't need a completion for every packet sent. > If packets are sent in a series, the completion of the last packet can be > used to indicate completion of all prior packets. > > Cc: stable@dpdk.org > Hi Long, Patch looks good to me, but I am not sure on the backport request. This patch is an optimisation update and we tend to get only fixes to stable trees. LTS maintainers cc'ed for comment. > Signed-off-by: Long Li > --- > Change log > v2: rebased to dpdk-next-net > > drivers/net/mana/mana.h | 3 ++- > drivers/net/mana/tx.c | 33 ++++++++++++++++++++++++++++++--- > 2 files changed, 32 insertions(+), 4 deletions(-) > > diff --git a/drivers/net/mana/mana.h b/drivers/net/mana/mana.h > index b653e1dd82..be88537f1a 100644 > --- a/drivers/net/mana/mana.h > +++ b/drivers/net/mana/mana.h > @@ -353,6 +353,7 @@ struct mana_priv { > struct mana_txq_desc { > struct rte_mbuf *pkt; > uint32_t wqe_size_in_bu; > + bool suppress_tx_cqe; > }; > > struct mana_rxq_desc { > @@ -401,7 +402,7 @@ struct mana_txq { > /* desc_ring_head is where we put pending requests to ring, > * completion pull off desc_ring_tail > */ > - uint32_t desc_ring_head, desc_ring_tail; > + uint32_t desc_ring_head, desc_ring_tail, desc_ring_len; > > struct mana_mr_btree mr_btree; > struct mana_stats stats; > diff --git a/drivers/net/mana/tx.c b/drivers/net/mana/tx.c > index 7f570181ad..5947efbe8d 100644 > --- a/drivers/net/mana/tx.c > +++ b/drivers/net/mana/tx.c > @@ -43,9 +43,11 @@ mana_stop_tx_queues(struct rte_eth_dev *dev) > > txq->desc_ring_tail = > (txq->desc_ring_tail + 1) % txq->num_desc; > + txq->desc_ring_len--; > } > txq->desc_ring_head = 0; > txq->desc_ring_tail = 0; > + txq->desc_ring_len = 0; > > memset(&txq->gdma_sq, 0, sizeof(txq->gdma_sq)); > memset(&txq->gdma_cq, 0, sizeof(txq->gdma_cq)); > @@ -173,13 +175,14 @@ mana_tx_burst(void *dpdk_txq, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) > int ret; > void *db_page; > uint16_t pkt_sent = 0; > - uint32_t num_comp; > + uint32_t num_comp, i; > > /* Process send completions from GDMA */ > num_comp = gdma_poll_completion_queue(&txq->gdma_cq, > txq->gdma_comp_buf, txq->num_desc); > > - for (uint32_t i = 0; i < num_comp; i++) { > + i = 0; > + while (i < num_comp) { > struct mana_txq_desc *desc = > &txq->desc_ring[txq->desc_ring_tail]; > struct mana_tx_comp_oob *oob = (struct mana_tx_comp_oob *) > @@ -204,7 +207,16 @@ mana_tx_burst(void *dpdk_txq, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) > > desc->pkt = NULL; > txq->desc_ring_tail = (txq->desc_ring_tail + 1) % txq->num_desc; > + txq->desc_ring_len--; > txq->gdma_sq.tail += desc->wqe_size_in_bu; > + > + /* If TX CQE suppression is used, don't read more CQE but move > + * on to the next packet > + */ > + if (desc->suppress_tx_cqe) > + continue; > + > + i++; > } > > /* Post send requests to GDMA */ > @@ -215,6 +227,9 @@ mana_tx_burst(void *dpdk_txq, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) > struct one_sgl sgl; > uint16_t seg_idx; > > + if (txq->desc_ring_len >= txq->num_desc) > + break; > + > /* Drop the packet if it exceeds max segments */ > if (m_pkt->nb_segs > priv->max_send_sge) { > DP_LOG(ERR, "send packet segments %d exceeding max", > @@ -310,7 +325,6 @@ mana_tx_burst(void *dpdk_txq, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) > tx_oob.short_oob.tx_compute_UDP_checksum = 0; > } > > - tx_oob.short_oob.suppress_tx_CQE_generation = 0; > tx_oob.short_oob.VCQ_number = txq->gdma_cq.id; > > tx_oob.short_oob.VSQ_frame_num = > @@ -362,6 +376,16 @@ mana_tx_burst(void *dpdk_txq, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) > if (seg_idx != m_pkt->nb_segs) > continue; > > + /* If we can at least queue post two WQEs and there are at > + * least two packets to send, use TX CQE suppression for the > + * current WQE > + */ > + if (txq->desc_ring_len + 1 < txq->num_desc && > + pkt_idx + 1 < nb_pkts) > + tx_oob.short_oob.suppress_tx_CQE_generation = 1; > + else > + tx_oob.short_oob.suppress_tx_CQE_generation = 0; > + > struct gdma_work_request work_req; > uint32_t wqe_size_in_bu; > > @@ -384,8 +408,11 @@ mana_tx_burst(void *dpdk_txq, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) > /* Update queue for tracking pending requests */ > desc->pkt = m_pkt; > desc->wqe_size_in_bu = wqe_size_in_bu; > + desc->suppress_tx_cqe = > + tx_oob.short_oob.suppress_tx_CQE_generation; > txq->desc_ring_head = > (txq->desc_ring_head + 1) % txq->num_desc; > + txq->desc_ring_len++; > > pkt_sent++; >