From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 05188A04DD; Fri, 23 Oct 2020 16:47:01 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 7F3C84F9C; Fri, 23 Oct 2020 16:46:59 +0200 (CEST) Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by dpdk.org (Postfix) with ESMTP id 7D64DA3; Fri, 23 Oct 2020 16:46:56 +0200 (CEST) IronPort-SDR: DdohhFR9+MlWvME/eelit06wmlgzgtERPbRsP8PgsuAnvDVoh73Cj2KyBlDkv66JdEkyyrjvfe ftToYEFSyEgg== X-IronPort-AV: E=McAfee;i="6000,8403,9782"; a="185384994" X-IronPort-AV: E=Sophos;i="5.77,408,1596524400"; d="scan'208";a="185384994" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Oct 2020 07:46:52 -0700 IronPort-SDR: BdvP/BgM3np08OQxVn9WAulqhsH/CUN7p0NCUBbiEorhxYskLjOieZiA7jJPkg1bcXTY4Woq3N L2z3yvQo4f+Q== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.77,408,1596524400"; d="scan'208";a="423442660" Received: from orsmsx601.amr.corp.intel.com ([10.22.229.14]) by fmsmga001.fm.intel.com with ESMTP; 23 Oct 2020 07:46:50 -0700 Received: from orsmsx611.amr.corp.intel.com (10.22.229.24) by ORSMSX601.amr.corp.intel.com (10.22.229.14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Fri, 23 Oct 2020 07:46:50 -0700 Received: from orsmsx601.amr.corp.intel.com (10.22.229.14) by ORSMSX611.amr.corp.intel.com (10.22.229.24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Fri, 23 Oct 2020 07:46:49 -0700 Received: from ORSEDG601.ED.cps.intel.com (10.7.248.6) by orsmsx601.amr.corp.intel.com (10.22.229.14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5 via Frontend Transport; Fri, 23 Oct 2020 07:46:49 -0700 Received: from NAM04-SN1-obe.outbound.protection.outlook.com (104.47.44.55) by edgegateway.intel.com (134.134.137.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.1713.5; Fri, 23 Oct 2020 07:46:45 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=gRZ2l+B46VnunVAupUgx+rJ0bpYZL/3dl8pT5E7FGp6bd865VcOoj1/ifRb2tBWl/INGTMi6w0J4JwFie0ZFo5KBoJpnvAGVBz/XJRfPCC80CxX3xj5FGARtoK/tnvYgN/zA9j+Ad1sxRkGbN+PaTFxkqJdq8naC7aFUdrTr1+eR4f+p+7gh7vKUhF29sRNLr+kjiXESiVp3hPI/WRH5aYWFSznhWmAh/WtHjjT9g7YvLOyrow6K/wIVr/kVYIXxPdARtOSd0lkc7d89o1kl56yQwGdoP8j9ncHsYsTQldy42W3MrWy0Z5SknU6DCskoLtaU2IhJmZD8ZffKfgBdpQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=r2Yv4vVK3VcEgKSJmVsQ/YjZHpWTkPVQoprthNZqiIs=; b=QkJ+h3itafUIU4b7McCQA0FxZUO3SpUbLYS02Nx7EVXeAMmHkOIs4ZGZcmldUORpVloJyamHil6oquUtTczRgaDklbGZzIax87GwWnKJdOCBIDmf89jCHMpuJGHj9n0N96PkgbCz4SQwgKcN+SCstLC/fIQukebY8dHOY+WrBt6B4CQo80REpPMH68EM9ojuap8pZzWvfVPtGuLA9cOyxiFubWYdSv+pPGpebuZlPBdwF3Zz2KtQI+DqLInj4ELk7dsV3WPQPVqOkHnhLwOK5TSu5wwgxHgcw08y/DK6qpeqpQaSnj8q9fVwcbk/g+atzpiZMTQE1cDUl0I9MWRrMw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=r2Yv4vVK3VcEgKSJmVsQ/YjZHpWTkPVQoprthNZqiIs=; b=MZzorOkmwu3dR8sC5ZD6S6pWZKlUucNwuvJ421NyG5rzE36R0+BhyDrbwea9j3avBZIlrpu5LUaExSJyZMONBk+7vWPHr1FaT03ISArfgLDAOteLN6cl44FgQ5UMa8ZeO248g+jJDHumGK1Is+s0sSKHquA0h07nD+0J8v3HVyI= Received: from BYAPR11MB3301.namprd11.prod.outlook.com (2603:10b6:a03:7f::26) by BY5PR11MB3893.namprd11.prod.outlook.com (2603:10b6:a03:183::26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3477.28; Fri, 23 Oct 2020 14:46:42 +0000 Received: from BYAPR11MB3301.namprd11.prod.outlook.com ([fe80::f5a4:3f6b:ade3:296b]) by BYAPR11MB3301.namprd11.prod.outlook.com ([fe80::f5a4:3f6b:ade3:296b%3]) with mapi id 15.20.3499.018; Fri, 23 Oct 2020 14:46:42 +0000 From: "Ananyev, Konstantin" To: yang_y_yi CC: "dev@dpdk.org" , "Hu, Jiayu" , "techboard@dpdk.org" , "thomas@monjalon.net" , "yangyi01@inspur.com" Thread-Topic: Re:RE: [PATCH v2] gso: fix free issue of mbuf gso segments attach to Thread-Index: AQHWqD/tzleaP+GQ60mK7fZdnSpuYqmjlw5wgAGWW4CAABdhYA== Date: Fri, 23 Oct 2020 14:46:42 +0000 Message-ID: References: <20201022065151.10108-1-yang_y_yi@163.com> <3dcc6d13.74bb.175559c0cf4.Coremail.yang_y_yi@163.com> In-Reply-To: <3dcc6d13.74bb.175559c0cf4.Coremail.yang_y_yi@163.com> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-reaction: no-action dlp-version: 11.5.1.3 authentication-results: 163.com; dkim=none (message not signed) header.d=none;163.com; dmarc=none action=none header.from=intel.com; x-originating-ip: [46.7.39.127] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 09c6713a-5dd5-465d-4edd-08d8776274f9 x-ms-traffictypediagnostic: BY5PR11MB3893: x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:9508; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 0Xlg7//nxf6iDAqWUlFzqo43EXm1qeM+kAY5AIEay/GWKvSHRGd/Il8SOKfJ0EUybq/ERiirUIb2C7CJBWXgHM0mUt5dxi5X2XEA1vMJqSvNFl9qXfflOTmB9x9woGeFMcq+/20Abp5yiIVIdNKIbfydNR/EDdqL3MWTaKnaY7sSxeUvIkXVAj4WswDhEpfJ86G2e2LMEUJyoeK0py6ugbi4fbn2JFNlwTBdW6CzANAuuxvmlVJOTtyj6vSQmJ2WlVr7ScjxAVD38Uauty9uMMp9vnGeA7nW2awRuGXEgakC69QSl4i4zbzKPoffvrTu96JdS3ruDeUa6S643dEtMw== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BYAPR11MB3301.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(376002)(396003)(136003)(366004)(39860400002)(346002)(4326008)(478600001)(33656002)(2906002)(8676002)(26005)(6916009)(186003)(53546011)(6506007)(55016002)(83380400001)(86362001)(76116006)(52536014)(9686003)(8936002)(4001150100001)(7696005)(66446008)(66556008)(71200400001)(66946007)(316002)(64756008)(54906003)(5660300002)(66476007); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata: l9W5CgaqLDqGBUuxz5VA7kX9dlG6KJM2lo/6NM4CJUuxybGYqY+XvhpffvoMBEQuW+Kg6pOuh4yKCHP3NorEQmlDDDUa5lgJqY7G+mwm5PGcw78eE5S+Pc7HhzFfHyc6ECP4X92xP58LtX2aC/YV2k6VBSKCqt7kOyWW5RegbujaLux7iFnQA64FY/cgnQgzNKBYyADWDAcOyANSuGlBqMFPOi++RdqYVXQgKPKe/069D3SJXFzGvM36FMKW9RSATtDeiKKyxY5ZgK1zJxDvgyBXMage4NWNrOlP0etCi4E1KibOg4Ie8Oq8CtosmklUHdTKfAyLMQJdgU+wasHm7gsnVtWdgYurFKuY5jyYJ85sCW9qy955gXZA2f/nPliYVfuZmTRGQamgFdnrF/a7O3ZKcuEbGh7X1prHqwaVsKD6L8XzqerxAHYu3FF0B1U6zVCvfpxlMh5f0Ujs8zBEzZucZsFpzJevNi+x/p0PAle3IL74jtS8P+w9g/FVXPE8TFjQeg/2Whes1br1WJLWaK7OwgR+RVa/j33ug8KyFNFYK1/kd2/ZrNtf/ruQV3r20sqfR/rwop2EWeL8aKpCt6BZbgx9OkIFKGi45lwLK0hpEqRrwxVpDykPYKwBWXgyvskHppK46/VsodGNTgETCg== Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: BYAPR11MB3301.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 09c6713a-5dd5-465d-4edd-08d8776274f9 X-MS-Exchange-CrossTenant-originalarrivaltime: 23 Oct 2020 14:46:42.4352 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: zIM2k5rwJJASlHehBTGPRvoSEeP9NoPlenrF+zslkNIloxMTiQdEGAHs5zV+K4NQ8a0g+LCA6upmh5iqF+79H7On3FpdL3PgWCEzDhEojcM= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR11MB3893 X-OriginatorOrg: intel.com Subject: Re: [dpdk-dev] [PATCH v2] gso: fix free issue of mbuf gso segments attach to X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > From: yang_y_yi > Sent: Friday, October 23, 2020 2:18 PM > To: Ananyev, Konstantin > Cc: dev@dpdk.org; Hu, Jiayu ; techboard@dpdk.org; tho= mas@monjalon.net; yangyi01@inspur.com > Subject: Re:RE: [PATCH v2] gso: fix free issue of mbuf gso segments attac= h to >=20 > Konstantin, thank you so much for comments, my replies inline, please che= ck them. > At 2020-10-22 21:16:43, "Ananyev, Konstantin" wrote: > > > >> > >> rte_gso_segment decreased refcnt of pkt by one, but > >> it is wrong if pkt is external mbuf, pkt won't be > >> freed because of incorrect refcnt, the result is > >> application can't allocate mbuf from mempool because > >> mbufs in mempool are run out of. > >> > >> One correct way is application should call > >> rte_pktmbuf_free after calling rte_gso_segment to free > >> pkt explicitly. rte_gso_segment mustn't handle it, this > >> should be responsibility of application. > > > >Probably needs to be stated clearly: > >It is a change in functional behaviour. > >Without deprecation note in advance. >=20 > Ok, I'll add such statement in next version. >=20 > >TB members: please provide your opinion on that patch. > > > >> > >> Fixes: 119583797b6a ("gso: support TCP/IPv4 GSO") > >> Signed-off-by: Yi Yang > >> --- > >> Changelog: > >> > >> v1->v2: > >> - update description of rte_gso_segment(). > >> - change code which calls rte_gso_segment() to > >> fix free issue. > >> > >> --- > >> app/test-pmd/csumonly.c | 3 ++- > >> doc/guides/prog_guide/generic_segmentation_offload_lib.rst | 7 +++++-= - > > > >I think release notes also have to be updated. >=20 > Ok, also will update it to reflect this change. >=20 > > > >> lib/librte_gso/rte_gso.c | 9 +-----= --- > >> lib/librte_gso/rte_gso.h | 7 +++++-= - > >> 4 files changed, 13 insertions(+), 13 deletions(-) > >> > >> diff --git a/app/test-pmd/csumonly.c b/app/test-pmd/csumonly.c > >> index 3d7d244..829e07f 100644 > >> --- a/app/test-pmd/csumonly.c > >> +++ b/app/test-pmd/csumonly.c > >> @@ -1080,11 +1080,12 @@ struct simple_gre_hdr { > >> ret =3D rte_gso_segment(pkts_burst[i], gso_ctx, > >> &gso_segments[nb_segments], > >> GSO_MAX_PKT_BURST - nb_segments); > >> + /* pkts_burst[i] can be freed safely here. */ > >> + rte_pktmbuf_free(pkts_burst[i]); > > > >It doesn't look correct to me. > >I think it should be: > >If (ret > 1) rte_pktmbuf_free(pkts_burst[i]); >=20 > No, in original implementation, if gso failed, application will free it, = otherwise rte_gso_segment will free it (i.e. refcnt update -1 in > rte_gso_segment), this change will change previous behavior. application = will free it for both cases. That's the point - with current implementation: If ret =3D=3D 1, then you shouldn't free input packet. Because in that case: input_pkt =3D=3D output_pkt[0] And if you'll free it, you can't use it after it. In that particular case, you can't TX it. =20 >=20 > > > >> if (ret >=3D 0) > >> nb_segments +=3D ret; > >> else { > >> TESTPMD_LOG(DEBUG, "Unable to segment packet"); > >> - rte_pktmbuf_free(pkts_burst[i]); > >> } > >> } > > > > > >About drivers/net/tap/rte_eth_tap.c: > >I think it has to be modified too, as here: > > > >/* free original mbuf */ > > rte_pktmbuf_free(mbuf_in); > > /* free tso mbufs */ > > if (num_tso_mbufs > 0) > > rte_pktmbuf_free_bulk(mbuf, num_tso_mbufs); > > > >if mbuf[0] =3D=3D mbuf_in > >Will have a double free() for the same mbuf. >=20 > Here I'm not clear, is this code ok before I change? It looks like there = is same issue there before. >=20 > > > >> > >> diff --git a/doc/guides/prog_guide/generic_segmentation_offload_lib.rs= t > b/doc/guides/prog_guide/generic_segmentation_offload_lib.rst > >> index 205cb8a..8577572 100644 > >> --- a/doc/guides/prog_guide/generic_segmentation_offload_lib.rst > >> +++ b/doc/guides/prog_guide/generic_segmentation_offload_lib.rst > >> @@ -25,8 +25,9 @@ Bearing that in mind, the GSO library enables DPDK a= pplications to segment > >> packets in software. Note however, that GSO is implemented as a stand= alone > >> library, and not via a 'fallback' mechanism (i.e. for when TSO is uns= upported > >> in the underlying hardware); that is, applications must explicitly in= voke the > >> -GSO library to segment packets. The size of GSO segments ``(segsz)`` = is > >> -configurable by the application. > >> +GSO library to segment packets, they also must call ``rte_pktmbuf_fre= e()`` to > >> +free mbuf GSO segments attach to after calling ``rte_gso_segment()``. > > > >Probably worth to mention that if return code =3D=3D 1, then > >output mbuf will point to input mbuf and extra care with free() is requi= red. >=20 > Ok. >=20 > > > >> The size > >> +of GSO segments ``(segsz)`` is configurable by the application. > >> > >> Limitations > >> ----------- > >> @@ -233,6 +234,8 @@ To segment an outgoing packet, an application must= : > >> > >> #. Invoke the GSO segmentation API, ``rte_gso_segment()``. > >> > >> +#. Call ``rte_pktmbuf_free()`` to free mbuf ``rte_gso_segment()`` seg= ments. > >> + > >> #. If required, update the L3 and L4 checksums of the newly-created s= egments. > >> For tunneled packets, the outer IPv4 headers' checksums should als= o be > >> updated. Alternatively, the application may offload checksum calcu= lation > >> diff --git a/lib/librte_gso/rte_gso.c b/lib/librte_gso/rte_gso.c > >> index 751b5b6..0d6cae5 100644 > >> --- a/lib/librte_gso/rte_gso.c > >> +++ b/lib/librte_gso/rte_gso.c > >> @@ -30,7 +30,6 @@ > >> uint16_t nb_pkts_out) > >> { > >> struct rte_mempool *direct_pool, *indirect_pool; > >> - struct rte_mbuf *pkt_seg; > >> uint64_t ol_flags; > >> uint16_t gso_size; > >> uint8_t ipid_delta; > >> @@ -80,13 +79,7 @@ > >> return 1; > >> } > >> > >> - if (ret > 1) { > >> - pkt_seg =3D pkt; > >> - while (pkt_seg) { > >> - rte_mbuf_refcnt_update(pkt_seg, -1); > >> - pkt_seg =3D pkt_seg->next; > >> - } > >> - } else if (ret < 0) { > >> + if (ret < 0) { > >> /* Revert the ol_flags in the event of failure. */ > >> pkt->ol_flags =3D ol_flags; > >> } > >> diff --git a/lib/librte_gso/rte_gso.h b/lib/librte_gso/rte_gso.h > >> index 3aab297..f6694f9 100644 > >> --- a/lib/librte_gso/rte_gso.h > >> +++ b/lib/librte_gso/rte_gso.h > >> @@ -89,8 +89,11 @@ struct rte_gso_ctx { > >> * the GSO segments are sent to should support transmission of multi-= segment > >> * packets. > >> * > >> - * If the input packet is GSO'd, its mbuf refcnt reduces by 1. Theref= ore, > >> - * when all GSO segments are freed, the input packet is freed automat= ically. > >> + * If the input packet is GSO'd, all the indirect segments are attach= ed to the > >> + * input packet. > >> + * > >> + * rte_gso_segment() will not free the input packet no matter whether= it is > >> + * GSO'd or not, the application should free it after call rte_gso_se= gment(). > >> * > >> * If the memory space in pkts_out or MBUF pools is insufficient, thi= s > >> * function fails, and it returns (-1) * errno. Otherwise, GSO succee= ds, > >> -- > >> 1.8.3.1 >=20 >=20 >