From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id DC198A04B5; Wed, 28 Oct 2020 01:51:16 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 41D6A4C97; Wed, 28 Oct 2020 01:51:15 +0100 (CET) Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by dpdk.org (Postfix) with ESMTP id D430D2BF9; Wed, 28 Oct 2020 01:51:11 +0100 (CET) IronPort-SDR: JVlIPMgyVTrsxTiqBe3OH1PTz5hFA85JV21IZal9rT0Faif52ir7BF+UEOgnLVTMcZWtIJioBl qYp6TaphJEnQ== X-IronPort-AV: E=McAfee;i="6000,8403,9787"; a="167406266" X-IronPort-AV: E=Sophos;i="5.77,424,1596524400"; d="scan'208";a="167406266" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Oct 2020 17:51:09 -0700 IronPort-SDR: 3w0nJR6gyRxC106fEsIy0AhAktjB/NiSqnh9Xi5EKeONeCc6wyhyVBSITbij1jpXx6rCr7/P+7 tQ9K7KCM7RRg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.77,424,1596524400"; d="scan'208";a="394657736" Received: from fmsmsx605.amr.corp.intel.com ([10.18.126.85]) by orsmga001.jf.intel.com with ESMTP; 27 Oct 2020 17:51:09 -0700 Received: from shsmsx603.ccr.corp.intel.com (10.109.6.143) by fmsmsx605.amr.corp.intel.com (10.18.126.85) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Tue, 27 Oct 2020 17:51:08 -0700 Received: from shsmsx606.ccr.corp.intel.com (10.109.6.216) by SHSMSX603.ccr.corp.intel.com (10.109.6.143) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Wed, 28 Oct 2020 08:51:07 +0800 Received: from shsmsx606.ccr.corp.intel.com ([10.109.6.216]) by SHSMSX606.ccr.corp.intel.com ([10.109.6.216]) with mapi id 15.01.1713.004; Wed, 28 Oct 2020 08:51:07 +0800 From: "Hu, Jiayu" To: "yang_y_yi@163.com" , "dev@dpdk.org" CC: "Ananyev, Konstantin" , "techboard@dpdk.org" , "thomas@monjalon.net" , "yangyi01@inspur.com" Thread-Topic: [PATCH v3] gso: fix free issue of mbuf gso segments attach to Thread-Index: AQHWq2PjC8J4YmvjvUeifSdFB7AT/KmsMGNw Date: Wed, 28 Oct 2020 00:51:06 +0000 Message-ID: <6e66c84f673c41688b58286693d78334@intel.com> References: <20201026064713.33316-1-yang_y_yi@163.com> In-Reply-To: <20201026064713.33316-1-yang_y_yi@163.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-reaction: no-action dlp-version: 11.5.1.3 dlp-product: dlpe-windows x-originating-ip: [10.239.127.36] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH v3] gso: fix free issue of mbuf gso segments attach to X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Acked-by: Jiayu Hu > -----Original Message----- > From: yang_y_yi@163.com > Sent: Monday, October 26, 2020 2:47 PM > To: dev@dpdk.org > Cc: Hu, Jiayu ; Ananyev, Konstantin > ; techboard@dpdk.org; > thomas@monjalon.net; yangyi01@inspur.com; yang_y_yi@163.com > Subject: [PATCH v3] gso: fix free issue of mbuf gso segments attach to >=20 > From: Yi Yang >=20 > rte_gso_segment decreased refcnt of pkt by one, but > it is wrong if pkt is external mbuf, pkt won't be > freed because of incorrect refcnt, the result is > application can't allocate mbuf from mempool because > mbufs in mempool are run out of. >=20 > One correct way is application should call > rte_pktmbuf_free after calling rte_gso_segment to free > pkt explicitly. rte_gso_segment mustn't handle it, this > should be responsibility of application. >=20 > This commit changed rte_gso_segment in functional behavior > and return value, so the application must take appropriate > actions according to return values, "ret < 0" means it > should free and drop 'pkt', "ret =3D=3D 0" means 'pkt' isn't > GSOed but 'pkt' can be transimmitted as a normal packet, > "ret > 0" means 'pkt' has been GSOed into two or multiple > segments, it should use "pkts_out" to transmit these > segments. The application must free 'pkt' after call > rte_gso_segment when return value isn't equal to 0. >=20 > Fixes: 119583797b6a ("gso: support TCP/IPv4 GSO") > Signed-off-by: Yi Yang > --- > Changelog: >=20 > v2->v3: > - add release notes to emphasize behavior and return > value changes of rte_gso_segment(). > - update return value description of rte_gso_segment(). > - modify related code to adapt to the changes. >=20 > v1->v2: > - update description of rte_gso_segment(). > - change code which calls rte_gso_segment() to > fix free issue. >=20 > --- > app/test-pmd/csumonly.c | 12 +++++++++= +-- > .../prog_guide/generic_segmentation_offload_lib.rst | 7 +++++-- > doc/guides/rel_notes/release_20_11.rst | 7 +++++++ > drivers/net/tap/rte_eth_tap.c | 12 +++++++++= +-- > lib/librte_gso/gso_tcp4.c | 6 ++---- > lib/librte_gso/gso_tunnel_tcp4.c | 14 +++++----= ----- > lib/librte_gso/gso_udp4.c | 6 ++---- > lib/librte_gso/rte_gso.c | 15 +++------= ------ > lib/librte_gso/rte_gso.h | 8 ++++++-- > 9 files changed, 50 insertions(+), 37 deletions(-) >=20 > diff --git a/app/test-pmd/csumonly.c b/app/test-pmd/csumonly.c > index 3d7d244..d813d4f 100644 > --- a/app/test-pmd/csumonly.c > +++ b/app/test-pmd/csumonly.c > @@ -1080,9 +1080,17 @@ struct simple_gre_hdr { > ret =3D rte_gso_segment(pkts_burst[i], gso_ctx, > &gso_segments[nb_segments], > GSO_MAX_PKT_BURST - > nb_segments); > - if (ret >=3D 0) > + if (ret >=3D 1) { > + /* pkts_burst[i] can be freed safely here. */ > + rte_pktmbuf_free(pkts_burst[i]); > nb_segments +=3D ret; > - else { > + } else if (ret =3D=3D 0) { > + /* 0 means it can be transmitted directly > + * without gso. > + */ > + gso_segments[nb_segments] =3D pkts_burst[i]; > + nb_segments +=3D 1; > + } else { > TESTPMD_LOG(DEBUG, "Unable to segment > packet"); > rte_pktmbuf_free(pkts_burst[i]); > } > diff --git a/doc/guides/prog_guide/generic_segmentation_offload_lib.rst > b/doc/guides/prog_guide/generic_segmentation_offload_lib.rst > index 205cb8a..8577572 100644 > --- a/doc/guides/prog_guide/generic_segmentation_offload_lib.rst > +++ b/doc/guides/prog_guide/generic_segmentation_offload_lib.rst > @@ -25,8 +25,9 @@ Bearing that in mind, the GSO library enables DPDK > applications to segment > packets in software. Note however, that GSO is implemented as a > standalone > library, and not via a 'fallback' mechanism (i.e. for when TSO is unsupp= orted > in the underlying hardware); that is, applications must explicitly invok= e the > -GSO library to segment packets. The size of GSO segments ``(segsz)`` is > -configurable by the application. > +GSO library to segment packets, they also must call ``rte_pktmbuf_free()= `` to > +free mbuf GSO segments attach to after calling ``rte_gso_segment()``. Th= e > size > +of GSO segments ``(segsz)`` is configurable by the application. >=20 > Limitations > ----------- > @@ -233,6 +234,8 @@ To segment an outgoing packet, an application must: >=20 > #. Invoke the GSO segmentation API, ``rte_gso_segment()``. >=20 > +#. Call ``rte_pktmbuf_free()`` to free mbuf ``rte_gso_segment()`` segmen= ts. > + > #. If required, update the L3 and L4 checksums of the newly-created > segments. > For tunneled packets, the outer IPv4 headers' checksums should also b= e > updated. Alternatively, the application may offload checksum calculat= ion > diff --git a/doc/guides/rel_notes/release_20_11.rst > b/doc/guides/rel_notes/release_20_11.rst > index d8ac359..da77396 100644 > --- a/doc/guides/rel_notes/release_20_11.rst > +++ b/doc/guides/rel_notes/release_20_11.rst > @@ -543,6 +543,13 @@ API Changes > * sched: Removed ``tb_rate``, ``tc_rate``, ``tc_period`` and ``tb_size`` > from ``struct rte_sched_subport_params``. >=20 > +* **Changed ``rte_gso_segment`` in functional behavior and return value.= ** > + > + * Don't save pkt to pkts_out[0] if it isn't GSOed in case of ret =3D= =3D 1. > + * Return 0 instead of 1 for the above case. > + * ``rte_gso_segment`` won't free pkt no matter whether it is GSOed, th= e > + application has responsibility to free it after call ``rte_gso_segme= nt``. > + >=20 > ABI Changes > ----------- > diff --git a/drivers/net/tap/rte_eth_tap.c b/drivers/net/tap/rte_eth_tap.= c > index 81c6884..2f8abb1 100644 > --- a/drivers/net/tap/rte_eth_tap.c > +++ b/drivers/net/tap/rte_eth_tap.c > @@ -751,8 +751,16 @@ struct ipc_queues { > if (num_tso_mbufs < 0) > break; >=20 > - mbuf =3D gso_mbufs; > - num_mbufs =3D num_tso_mbufs; > + if (num_tso_mbufs >=3D 1) { > + mbuf =3D gso_mbufs; > + num_mbufs =3D num_tso_mbufs; > + } else { > + /* 0 means it can be transmitted directly > + * without gso. > + */ > + mbuf =3D &mbuf_in; > + num_mbufs =3D 1; > + } > } else { > /* stats.errs will be incremented */ > if (rte_pktmbuf_pkt_len(mbuf_in) > max_size) > diff --git a/lib/librte_gso/gso_tcp4.c b/lib/librte_gso/gso_tcp4.c > index ade172a..d31feaf 100644 > --- a/lib/librte_gso/gso_tcp4.c > +++ b/lib/librte_gso/gso_tcp4.c > @@ -50,15 +50,13 @@ > pkt->l2_len); > frag_off =3D rte_be_to_cpu_16(ipv4_hdr->fragment_offset); > if (unlikely(IS_FRAGMENTED(frag_off))) { > - pkts_out[0] =3D pkt; > - return 1; > + return 0; > } >=20 > /* Don't process the packet without data */ > hdr_offset =3D pkt->l2_len + pkt->l3_len + pkt->l4_len; > if (unlikely(hdr_offset >=3D pkt->pkt_len)) { > - pkts_out[0] =3D pkt; > - return 1; > + return 0; > } >=20 > pyld_unit_size =3D gso_size - hdr_offset; > diff --git a/lib/librte_gso/gso_tunnel_tcp4.c > b/lib/librte_gso/gso_tunnel_tcp4.c > index e0384c2..166aace 100644 > --- a/lib/librte_gso/gso_tunnel_tcp4.c > +++ b/lib/librte_gso/gso_tunnel_tcp4.c > @@ -62,7 +62,7 @@ > { > struct rte_ipv4_hdr *inner_ipv4_hdr; > uint16_t pyld_unit_size, hdr_offset, frag_off; > - int ret =3D 1; > + int ret; >=20 > hdr_offset =3D pkt->outer_l2_len + pkt->outer_l3_len + pkt->l2_len; > inner_ipv4_hdr =3D (struct rte_ipv4_hdr *)(rte_pktmbuf_mtod(pkt, > char *) + > @@ -73,25 +73,21 @@ > */ > frag_off =3D rte_be_to_cpu_16(inner_ipv4_hdr->fragment_offset); > if (unlikely(IS_FRAGMENTED(frag_off))) { > - pkts_out[0] =3D pkt; > - return 1; > + return 0; > } >=20 > hdr_offset +=3D pkt->l3_len + pkt->l4_len; > /* Don't process the packet without data */ > if (hdr_offset >=3D pkt->pkt_len) { > - pkts_out[0] =3D pkt; > - return 1; > + return 0; > } > pyld_unit_size =3D gso_size - hdr_offset; >=20 > /* Segment the payload */ > ret =3D gso_do_segment(pkt, hdr_offset, pyld_unit_size, direct_pool, > indirect_pool, pkts_out, nb_pkts_out); > - if (ret <=3D 1) > - return ret; > - > - update_tunnel_ipv4_tcp_headers(pkt, ipid_delta, pkts_out, ret); > + if (ret > 1) > + update_tunnel_ipv4_tcp_headers(pkt, ipid_delta, pkts_out, > ret); >=20 > return ret; > } > diff --git a/lib/librte_gso/gso_udp4.c b/lib/librte_gso/gso_udp4.c > index 6fa68f2..5d0186a 100644 > --- a/lib/librte_gso/gso_udp4.c > +++ b/lib/librte_gso/gso_udp4.c > @@ -52,8 +52,7 @@ > pkt->l2_len); > frag_off =3D rte_be_to_cpu_16(ipv4_hdr->fragment_offset); > if (unlikely(IS_FRAGMENTED(frag_off))) { > - pkts_out[0] =3D pkt; > - return 1; > + return 0; > } >=20 > /* > @@ -65,8 +64,7 @@ >=20 > /* Don't process the packet without data. */ > if (unlikely(hdr_offset + pkt->l4_len >=3D pkt->pkt_len)) { > - pkts_out[0] =3D pkt; > - return 1; > + return 0; > } >=20 > /* pyld_unit_size must be a multiple of 8 because frag_off > diff --git a/lib/librte_gso/rte_gso.c b/lib/librte_gso/rte_gso.c > index 751b5b6..896350e 100644 > --- a/lib/librte_gso/rte_gso.c > +++ b/lib/librte_gso/rte_gso.c > @@ -30,7 +30,6 @@ > uint16_t nb_pkts_out) > { > struct rte_mempool *direct_pool, *indirect_pool; > - struct rte_mbuf *pkt_seg; > uint64_t ol_flags; > uint16_t gso_size; > uint8_t ipid_delta; > @@ -44,8 +43,7 @@ >=20 > if (gso_ctx->gso_size >=3D pkt->pkt_len) { > pkt->ol_flags &=3D (~(PKT_TX_TCP_SEG | PKT_TX_UDP_SEG)); > - pkts_out[0] =3D pkt; > - return 1; > + return 0; > } >=20 > direct_pool =3D gso_ctx->direct_pool; > @@ -75,18 +73,11 @@ > indirect_pool, pkts_out, nb_pkts_out); > } else { > /* unsupported packet, skip */ > - pkts_out[0] =3D pkt; > RTE_LOG(DEBUG, GSO, "Unsupported packet type\n"); > - return 1; > + ret =3D 0; > } >=20 > - if (ret > 1) { > - pkt_seg =3D pkt; > - while (pkt_seg) { > - rte_mbuf_refcnt_update(pkt_seg, -1); > - pkt_seg =3D pkt_seg->next; > - } > - } else if (ret < 0) { > + if (ret < 0) { > /* Revert the ol_flags in the event of failure. */ > pkt->ol_flags =3D ol_flags; > } > diff --git a/lib/librte_gso/rte_gso.h b/lib/librte_gso/rte_gso.h > index 3aab297..af480ee 100644 > --- a/lib/librte_gso/rte_gso.h > +++ b/lib/librte_gso/rte_gso.h > @@ -89,8 +89,11 @@ struct rte_gso_ctx { > * the GSO segments are sent to should support transmission of multi- > segment > * packets. > * > - * If the input packet is GSO'd, its mbuf refcnt reduces by 1. Therefore= , > - * when all GSO segments are freed, the input packet is freed automatica= lly. > + * If the input packet is GSO'd, all the indirect segments are attached = to the > + * input packet. > + * > + * rte_gso_segment() will not free the input packet no matter whether it= is > + * GSO'd or not, the application should free it after call rte_gso_segme= nt(). > * > * If the memory space in pkts_out or MBUF pools is insufficient, this > * function fails, and it returns (-1) * errno. Otherwise, GSO succeeds, > @@ -109,6 +112,7 @@ struct rte_gso_ctx { > * > * @return > * - The number of GSO segments filled in pkts_out on success. > + * - Return 0 if it needn't GSOed. > * - Return -ENOMEM if run out of memory in MBUF pools. > * - Return -EINVAL for invalid parameters. > */ > -- > 1.8.3.1