From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 14E03A04F9; Fri, 10 Jan 2020 14:49:57 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id DECF81EA79; Fri, 10 Jan 2020 14:49:56 +0100 (CET) Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by dpdk.org (Postfix) with ESMTP id 048E51EA76 for ; Fri, 10 Jan 2020 14:49:54 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga106.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 10 Jan 2020 05:49:54 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.69,417,1571727600"; d="scan'208";a="218668583" Received: from fmsmsx103.amr.corp.intel.com ([10.18.124.201]) by fmsmga008.fm.intel.com with ESMTP; 10 Jan 2020 05:49:53 -0800 Received: from FMSEDG001.ED.cps.intel.com (10.1.192.133) by FMSMSX103.amr.corp.intel.com (10.18.124.201) with Microsoft SMTP Server (TLS) id 14.3.439.0; Fri, 10 Jan 2020 05:49:53 -0800 Received: from NAM04-CO1-obe.outbound.protection.outlook.com (104.47.45.58) by edgegateway.intel.com (192.55.55.68) with Microsoft SMTP Server (TLS) id 14.3.439.0; Fri, 10 Jan 2020 05:49:53 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=cBXhLHa40t8mgrtiGbnskFiN8oiBfA7cGfkSerO6kNXcOajl0RBf5jwca3XivseVQRjOfdWsOCbYpgN1ULcvuDfEDV0OO7PQMOsbHYzObptvhUKN1Zn3QGBz93QFy9X/b4glYCJwvzRRoLVkGarR2rUIc5z83YD5Do1dfIAdLIV0oc71sY+qULFA7KkOBKboBi+LKWWLmt0VrzJ+F7BJWGliUs4ObcxD+BsgAjupjCL1OkB2vj0TVhnp4aTgQcPC76Li4VnPdskMOntyh4oKh2FsswRn4biGfC7aOjIpRDzBHfVqGv4P+YXt3mYOSGHt2BZ163uFN6b/swR1AtzD/w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=DOzHVJKTtRFgS+H1mLlOj5nB7KvYHgpYLrOb6QmoJwA=; b=kPo1dZ0un+bCWbfp2gmbspR3PpZ8fOOmY+DFUukvfQtZUlufo3vzK1dsRJ9fd5ttee2tAPA1bouxs9JtvhtkpusCrvGhVzEj6CrZTVfai0qVnsjxD2x33P8zedmIv03AjzPFsk0RxxdxzunnzntVRTJjFdLlmJYBblibJxofEl3L+aJarO2675Ml7Rnj0oscyzxSxin1R3ciJdYch9b4GtS57ahnSTqS1AKgxboARTd3cNn27Hw6vVPFsbkYKWqfMfleOIY7bLqxevWQBSZM/orFZQo4gtEqB8qIB7ckdmbyZoEIz8IDmxD7Kjx2pDV3M+CvSVhOkFsxWB3fEmdZzw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=DOzHVJKTtRFgS+H1mLlOj5nB7KvYHgpYLrOb6QmoJwA=; b=RrNPiKnWIarxUzKwVJr2lZR33M1N+VYXj2+gaAJuD1s1Ev+l3N1dmXgGzmnq6aiSWlEKil2mN7oWzdgqjqFmKQNmjpYNILllWlnoSI+Dao0E8J5HbVU6HhPobHtpJ3eSLQpDgLU/a8i5Bzo1m+9kDjnL3bcX/bdXCaK8y1INhK0= Received: from SN6PR11MB2558.namprd11.prod.outlook.com (52.135.94.19) by SN6PR11MB2990.namprd11.prod.outlook.com (52.135.124.213) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2602.12; Fri, 10 Jan 2020 13:49:52 +0000 Received: from SN6PR11MB2558.namprd11.prod.outlook.com ([fe80::4d86:362a:13c3:8386]) by SN6PR11MB2558.namprd11.prod.outlook.com ([fe80::4d86:362a:13c3:8386%7]) with mapi id 15.20.2623.013; Fri, 10 Jan 2020 13:49:52 +0000 From: "Ananyev, Konstantin" To: "Di, ChenxuX" , "dev@dpdk.org" CC: "Yang, Qiming" , "Di, ChenxuX" Thread-Topic: [dpdk-dev] [PATCH v8 3/4] net/ixgbe: cleanup Tx buffers Thread-Index: AQHVx53220JuivPqZEab9qN98qQs0Kfj59qg Date: Fri, 10 Jan 2020 13:49:51 +0000 Message-ID: References: <20191203055134.72874-1-chenxux.di@intel.com> <20200110095900.36148-1-chenxux.di@intel.com> <20200110095900.36148-4-chenxux.di@intel.com> In-Reply-To: <20200110095900.36148-4-chenxux.di@intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiOTE0YTRhYjctY2JiOC00ODcxLTk2NDEtZTAyNzgyYjhmMzMzIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjEwLjE4MDQuNDkiLCJUcnVzdGVkTGFiZWxIYXNoIjoiS1V0b3pCUUhOeitrc2R4bmthTHRRcGJOK3pWWjdWTlRaU2hNQk9iVUsxdnlzREIwVlRyYm5YQVlWc2NqQXp6dyJ9 dlp-product: dlpe-windows dlp-reaction: no-action dlp-version: 11.2.0.6 x-ctpclassification: CTP_NT authentication-results: spf=none (sender IP is ) smtp.mailfrom=konstantin.ananyev@intel.com; x-originating-ip: [192.198.151.162] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 7b636012-7f1e-4365-cf0a-08d795d3f78f x-ms-traffictypediagnostic: SN6PR11MB2990: x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:7691; x-forefront-prvs: 02788FF38E x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(346002)(39860400002)(376002)(136003)(396003)(366004)(189003)(199004)(7696005)(8936002)(33656002)(66476007)(55016002)(26005)(54906003)(186003)(6506007)(86362001)(81166006)(110136005)(64756008)(66946007)(66556008)(66446008)(81156014)(52536014)(9686003)(71200400001)(8676002)(5660300002)(76116006)(2906002)(316002)(107886003)(478600001)(4326008); DIR:OUT; SFP:1102; SCL:1; SRVR:SN6PR11MB2990; H:SN6PR11MB2558.namprd11.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: ksZydMhvMDVywYeWE88GxSI5bK/25rCtJrfxls612bb5IKECMDQFbfNTA9rR2tyBQ3PMtkGWo5aJLN/sjPRfCscXW72vJQctUOheBra9TkzGVgyh16DG8QJ02cFfH7fA7UA+ZaKMI4U9v+RrFW1qoiy9LhNUi/x57Pkq3+ceKRnT+M+HnfY7I/dwmBDeKiwdRmWggHLw8WZ2prutWR/OAYHooaSS9pZr2l0GLa/9ko0oZZ+pllBJuRPG6g+KYrEGBDWzypDq7BJhh9vI9zTN80TeFPn8+snjsbVzRLT1U5TU3VGgLGFgxgKtyRLYjEujUTHyMng0Cq3szMRIYGz6dc3BHXIQB+EXCEbl1ORF/HQC0IBvRtLnIjaMsKbukza7NLFEcDdVCxbvTPllRTTbXhRh4YaZY80fUnXJD0+Ystw5cjz4mIPyqjA/N830TafO Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: 7b636012-7f1e-4365-cf0a-08d795d3f78f X-MS-Exchange-CrossTenant-originalarrivaltime: 10 Jan 2020 13:49:51.6321 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: G0TCURO1rKBirylNrfPmwc+ZHTzi4NeCaKLX26niPTHjbj6KTAwIpmnc2YuV//5OfseROu47CI0t+6Xdeg2gT+DHhOO003lMePuvX6ZLGhg= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR11MB2990 X-OriginatorOrg: intel.com Subject: Re: [dpdk-dev] [PATCH v8 3/4] net/ixgbe: cleanup Tx buffers X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" >=20 > Add support to the ixgbe driver for the API rte_eth_tx_done_cleanup > to force free consumed buffers on Tx ring. >=20 > Signed-off-by: Chenxu Di > --- > drivers/net/ixgbe/ixgbe_ethdev.c | 2 + > drivers/net/ixgbe/ixgbe_rxtx.c | 109 ++++++++++++++++++++++++ > drivers/net/ixgbe/ixgbe_rxtx.h | 8 +- > drivers/net/ixgbe/ixgbe_rxtx_vec_neon.c | 1 + > drivers/net/ixgbe/ixgbe_rxtx_vec_sse.c | 1 + > 5 files changed, 120 insertions(+), 1 deletion(-) >=20 > diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_e= thdev.c > index 2c6fd0f13..75bdd391a 100644 > --- a/drivers/net/ixgbe/ixgbe_ethdev.c > +++ b/drivers/net/ixgbe/ixgbe_ethdev.c > @@ -601,6 +601,7 @@ static const struct eth_dev_ops ixgbe_eth_dev_ops =3D= { > .udp_tunnel_port_add =3D ixgbe_dev_udp_tunnel_port_add, > .udp_tunnel_port_del =3D ixgbe_dev_udp_tunnel_port_del, > .tm_ops_get =3D ixgbe_tm_ops_get, > + .tx_done_cleanup =3D ixgbe_dev_tx_done_cleanup, > }; >=20 > /* > @@ -649,6 +650,7 @@ static const struct eth_dev_ops ixgbevf_eth_dev_ops = =3D { > .reta_query =3D ixgbe_dev_rss_reta_query, > .rss_hash_update =3D ixgbe_dev_rss_hash_update, > .rss_hash_conf_get =3D ixgbe_dev_rss_hash_conf_get, > + .tx_done_cleanup =3D ixgbe_dev_tx_done_cleanup, > }; >=20 > /* store statistics names and its offset in stats structure */ > diff --git a/drivers/net/ixgbe/ixgbe_rxtx.c b/drivers/net/ixgbe/ixgbe_rxt= x.c > index fa572d184..23c897d3a 100644 > --- a/drivers/net/ixgbe/ixgbe_rxtx.c > +++ b/drivers/net/ixgbe/ixgbe_rxtx.c > @@ -2306,6 +2306,114 @@ ixgbe_tx_queue_release_mbufs(struct ixgbe_tx_queu= e *txq) > } > } >=20 > +int > +ixgbe_tx_done_cleanup_full(struct ixgbe_tx_queue *txq, uint32_t free_cnt= ) > +{ > + uint32_t pkt_cnt; > + uint16_t i; > + uint16_t tx_last; > + uint16_t tx_id; > + uint16_t nb_tx_to_clean; > + uint16_t nb_tx_free_last; > + struct ixgbe_tx_entry *swr_ring =3D txq->sw_ring; > + > + /* Start free mbuf from the next of tx_tail */ > + tx_last =3D txq->tx_tail; > + tx_id =3D swr_ring[tx_last].next_id; > + > + if (txq->nb_tx_free =3D=3D 0 && ixgbe_xmit_cleanup(txq)) > + return 0; > + > + nb_tx_to_clean =3D txq->nb_tx_free; > + nb_tx_free_last =3D txq->nb_tx_free; > + if (!free_cnt) > + free_cnt =3D txq->nb_tx_desc; > + > + /* Loop through swr_ring to count the amount of > + * freeable mubfs and packets. > + */ > + for (pkt_cnt =3D 0; pkt_cnt < free_cnt; ) { > + for (i =3D 0; i < nb_tx_to_clean && > + pkt_cnt < free_cnt && > + tx_id !=3D tx_last; i++) { > + if (swr_ring[tx_id].mbuf !=3D NULL) { > + rte_pktmbuf_free_seg(swr_ring[tx_id].mbuf); > + swr_ring[tx_id].mbuf =3D NULL; > + > + /* > + * last segment in the packet, > + * increment packet count > + */ > + pkt_cnt +=3D (swr_ring[tx_id].last_id =3D=3D tx_id); > + } > + > + tx_id =3D swr_ring[tx_id].next_id; > + } > + > + if (txq->tx_rs_thresh > txq->nb_tx_desc - > + txq->nb_tx_free || tx_id =3D=3D tx_last) > + break; > + > + if (pkt_cnt < free_cnt) { > + if (ixgbe_xmit_cleanup(txq)) > + break; > + > + nb_tx_to_clean =3D txq->nb_tx_free - nb_tx_free_last; > + nb_tx_free_last =3D txq->nb_tx_free; > + } > + } > + > + return (int)pkt_cnt; > +} > + > +int > +ixgbe_tx_done_cleanup_vec(struct ixgbe_tx_queue *txq __rte_unused, > + uint32_t free_cnt __rte_unused) > +{ > + return -ENOTSUP; > +} > + > +int > +ixgbe_tx_done_cleanup_simple(struct ixgbe_tx_queue *txq, > + uint32_t free_cnt) > +{ > + int i, n, cnt; > + > + if (free_cnt =3D=3D 0 || free_cnt > txq->nb_tx_desc) > + free_cnt =3D txq->nb_tx_desc; > + > + cnt =3D free_cnt - free_cnt % txq->tx_rs_thresh; > + > + for (i =3D 0; i < cnt; i +=3D n) { > + if (txq->nb_tx_desc - txq->nb_tx_free < txq->tx_rs_thresh) > + break; > + > + n =3D ixgbe_tx_free_bufs(txq); > + > + if (n =3D=3D 0) > + break; > + } > + > + return i; > +} > + > +int > +ixgbe_dev_tx_done_cleanup(void *tx_queue, uint32_t free_cnt) > +{ > + struct ixgbe_tx_queue *txq =3D (struct ixgbe_tx_queue *)tx_queue; > + return txq->ops->txq_done_cleanup(txq, free_cnt); > +} > + > +int > +ixgbe_tx_done_cleanup(struct ixgbe_tx_queue *txq, uint32_t free_cnt) > +{ > + /* Use a simple Tx queue (no offloads, no multi segs) if possible */ > + if (txq->offloads =3D=3D 0) > + return ixgbe_tx_done_cleanup_simple(txq, free_cnt); > + else > + return ixgbe_tx_done_cleanup_full(txq, free_cnt); > +} > + > static void __attribute__((cold)) > ixgbe_tx_free_swring(struct ixgbe_tx_queue *txq) > { > @@ -2375,6 +2483,7 @@ static const struct ixgbe_txq_ops def_txq_ops =3D { > .release_mbufs =3D ixgbe_tx_queue_release_mbufs, > .free_swring =3D ixgbe_tx_free_swring, > .reset =3D ixgbe_reset_tx_queue, > + .txq_done_cleanup =3D ixgbe_tx_done_cleanup, > }; >=20 > /* Takes an ethdev and a queue and sets up the tx function to be used ba= sed on > diff --git a/drivers/net/ixgbe/ixgbe_rxtx.h b/drivers/net/ixgbe/ixgbe_rxt= x.h > index 505d344b9..41a3738ce 100644 > --- a/drivers/net/ixgbe/ixgbe_rxtx.h > +++ b/drivers/net/ixgbe/ixgbe_rxtx.h > @@ -238,6 +238,7 @@ struct ixgbe_txq_ops { > void (*release_mbufs)(struct ixgbe_tx_queue *txq); > void (*free_swring)(struct ixgbe_tx_queue *txq); > void (*reset)(struct ixgbe_tx_queue *txq); > + int (*txq_done_cleanup)(struct ixgbe_tx_queue *txq, uint32_t free_cnt); > }; >=20 > /* > @@ -253,7 +254,6 @@ struct ixgbe_txq_ops { > IXGBE_ADVTXD_DCMD_DEXT |\ > IXGBE_ADVTXD_DCMD_EOP) >=20 > - > /* Takes an ethdev and a queue and sets up the tx function to be used ba= sed on > * the queue parameters. Used in tx_queue_setup by primary process and t= hen > * in dev_init by secondary process when attaching to an existing ethdev= . > @@ -285,6 +285,12 @@ int ixgbe_rx_vec_dev_conf_condition_check(struct rte= _eth_dev *dev); > int ixgbe_rxq_vec_setup(struct ixgbe_rx_queue *rxq); > void ixgbe_rx_queue_release_mbufs_vec(struct ixgbe_rx_queue *rxq); >=20 > +int ixgbe_dev_tx_done_cleanup(void *tx_queue, uint32_t free_cnt); As a nit: I don't think you need to make these 4 functions below extrernal. _cleanup(), cleanup_full and cleanup_simple can be static ones in ixgbe_rxt= x.c _cleanup_vec() can be static in ixgbe_rxtx_vec_common.h BTW, I think _cleanup_vec() will be identical to cleanup_simple(). Apart, from that: Acked-by: Konstantin Ananyev As a side notice, I think we need to add into test-pmd ability to call/test tx_cleanup_done (either as a separate command, or new fwd mode, or...). But that probably subject for separate patch series.=20 > +int ixgbe_tx_done_cleanup(struct ixgbe_tx_queue *txq, uint32_t free_cnt)= ; > +int ixgbe_tx_done_cleanup_full(struct ixgbe_tx_queue *txq, uint32_t free= _cnt); > +int ixgbe_tx_done_cleanup_vec(struct ixgbe_tx_queue *txq, uint32_t free_= cnt); > +int ixgbe_tx_done_cleanup_simple(struct ixgbe_tx_queue *txq, uint32_t fr= ee_cnt); > + > extern const uint32_t ptype_table[IXGBE_PACKET_TYPE_MAX]; > extern const uint32_t ptype_table_tn[IXGBE_PACKET_TYPE_TN_MAX]; >=20 > diff --git a/drivers/net/ixgbe/ixgbe_rxtx_vec_neon.c b/drivers/net/ixgbe/= ixgbe_rxtx_vec_neon.c > index feb86c61e..cd9b7dc01 100644 > --- a/drivers/net/ixgbe/ixgbe_rxtx_vec_neon.c > +++ b/drivers/net/ixgbe/ixgbe_rxtx_vec_neon.c > @@ -559,6 +559,7 @@ static const struct ixgbe_txq_ops vec_txq_ops =3D { > .release_mbufs =3D ixgbe_tx_queue_release_mbufs_vec, > .free_swring =3D ixgbe_tx_free_swring, > .reset =3D ixgbe_reset_tx_queue, > + .txq_done_cleanup =3D ixgbe_tx_done_cleanup_vec, > }; >=20 > int __attribute__((cold)) > diff --git a/drivers/net/ixgbe/ixgbe_rxtx_vec_sse.c b/drivers/net/ixgbe/i= xgbe_rxtx_vec_sse.c > index 599ba30e5..63bfac9fa 100644 > --- a/drivers/net/ixgbe/ixgbe_rxtx_vec_sse.c > +++ b/drivers/net/ixgbe/ixgbe_rxtx_vec_sse.c > @@ -730,6 +730,7 @@ static const struct ixgbe_txq_ops vec_txq_ops =3D { > .release_mbufs =3D ixgbe_tx_queue_release_mbufs_vec, > .free_swring =3D ixgbe_tx_free_swring, > .reset =3D ixgbe_reset_tx_queue, > + .txq_done_cleanup =3D ixgbe_tx_done_cleanup_vec, > }; >=20 > int __attribute__((cold)) > -- > 2.17.1