From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id EE8C829AC for ; Fri, 20 Jul 2018 12:38:32 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 20 Jul 2018 03:38:30 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,378,1526367600"; d="scan'208";a="59253736" Received: from irsmsx108.ger.corp.intel.com ([163.33.3.3]) by orsmga006.jf.intel.com with ESMTP; 20 Jul 2018 03:38:21 -0700 Received: from irsmsx111.ger.corp.intel.com (10.108.20.4) by IRSMSX108.ger.corp.intel.com (163.33.3.3) with Microsoft SMTP Server (TLS) id 14.3.319.2; Fri, 20 Jul 2018 11:38:20 +0100 Received: from irsmsx107.ger.corp.intel.com ([169.254.10.193]) by irsmsx111.ger.corp.intel.com ([169.254.2.118]) with mapi id 14.03.0319.002; Fri, 20 Jul 2018 11:38:20 +0100 From: "De Lara Guarch, Pablo" To: "Daly, Lee" CC: "dev@dpdk.org" Thread-Topic: [PATCH v2] compress/isal: add chained mbuf support Thread-Index: AQHUGRRs0OPkQAUUW0WftdR7RTeP8aSX6p0A Date: Fri, 20 Jul 2018 10:38:19 +0000 Message-ID: References: <1530788614-192928-1-git-send-email-lee.daly@intel.com> <1531312811-177746-1-git-send-email-lee.daly@intel.com> In-Reply-To: <1531312811-177746-1-git-send-email-lee.daly@intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiZTYyZTk2ODctZWRmOC00NzZhLTk0MWItNzMwOWNhMjhiYjQwIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjEwLjE4MDQuNDkiLCJUcnVzdGVkTGFiZWxIYXNoIjoiR2c3WW5HWTliXC9sNEhsMjdGeCtuS3JvdFBtenFaWFJNZm9iMTZRWW1Dbzk4UldHNzdUXC9uZDFpRVFZXC9KN1RlNSJ9 x-ctpclassification: CTP_NT dlp-product: dlpe-windows dlp-version: 11.0.200.100 dlp-reaction: no-action x-originating-ip: [163.33.239.182] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH v2] compress/isal: add chained mbuf support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jul 2018 10:38:33 -0000 Hi Lee, > -----Original Message----- > From: Daly, Lee > Sent: Wednesday, July 11, 2018 1:40 PM > To: De Lara Guarch, Pablo > Cc: dev@dpdk.org; Daly, Lee > Subject: [PATCH v2] compress/isal: add chained mbuf support >=20 > This patch adds chained mbuf support for input or output buffers during > compression/decompression operations. >=20 > V2 - Fixed using chained mbufs with offsets. >=20 > Signed-off-by: Lee Daly > --- > doc/guides/compressdevs/features/isal.ini | 1 + > doc/guides/compressdevs/isal.rst | 2 - > drivers/compress/isal/isal_compress_pmd.c | 397 ++++++++++++++++++++= --- > --- > drivers/compress/isal/isal_compress_pmd_ops.c | 5 +- > 4 files changed, 321 insertions(+), 84 deletions(-) >=20 > diff --git a/doc/guides/compressdevs/features/isal.ini > b/doc/guides/compressdevs/features/isal.ini > index 7183d10..06a95fb 100644 > --- a/doc/guides/compressdevs/features/isal.ini > +++ b/doc/guides/compressdevs/features/isal.ini > @@ -8,6 +8,7 @@ CPU SSE =3D Y > CPU AVX =3D Y > CPU AVX2 =3D Y > CPU AVX512 =3D Y > +Chained mbufs =3D Y There are new feature flags replacing this one. > Deflate =3D Y > Fixed =3D Y > Dynamic =3D Y > diff --git a/doc/guides/compressdevs/isal.rst > b/doc/guides/compressdevs/isal.rst > index 276ff06..3bc3022 100644 > --- a/doc/guides/compressdevs/isal.rst > +++ b/doc/guides/compressdevs/isal.rst > @@ -78,8 +78,6 @@ As a result the level mappings from the API to the PMD = are > shown below. > Limitations > ----------- >=20 > -* Chained mbufs will not be supported until a future release, meaning ma= x data > size being passed to PMD through a single mbuf is 64K - 1. If data is lar= ger than > this it will need to be split up and sent as multiple operations. > - > * Compressdev level 0, no compression, is not supported. >=20 > * Checksums will not be supported until future release. > diff --git a/drivers/compress/isal/isal_compress_pmd.c > b/drivers/compress/isal/isal_compress_pmd.c > index 747ded1..70231c4 100644 > --- a/drivers/compress/isal/isal_compress_pmd.c > +++ b/drivers/compress/isal/isal_compress_pmd.c > @@ -188,6 +188,211 @@ isal_comp_set_priv_xform_parameters(struct > isal_priv_xform *priv_xform, > return 0; > } >=20 > +/* Compression using chained mbufs for input/output data */ static int > +chained_mbuf_compression(struct rte_comp_op *op, struct isal_comp_qp > +*qp) { > + int ret, i; "i" should be unsigned int. > + uint8_t segs; > + uint32_t consumed_data; > + uint32_t remaining_data =3D op->src.length; > + struct rte_mbuf *src =3D op->m_src; > + struct rte_mbuf *dst =3D op->m_dst; > + > + /* check for offset passing multiple segments > + * and point compression stream to input/output buffer > + */ > + if (op->src.offset > src->data_len) { >=3D > + segs =3D op->src.offset / src->data_len; This is assuming that each segment has the same data length. Better to do a while loop, decreasing the offset (remaining_offset?), and moving to the next segment, until it is less than the current segment. > + for (i =3D 0; i < segs; i++) > + src =3D src->next; > + > + qp->stream->avail_in =3D src->data_len - > + (op->src.offset % src->data_len); Doing what I suggested above simplifies this, as you can substract "remaini= ng_offset" to data_len. > + qp->stream->next_in =3D rte_pktmbuf_mtod_offset(src, uint8_t *, > + (op->src.offset % src->data_len)); > + } else { > + qp->stream->avail_in =3D src->data_len - op->src.offset; > + qp->stream->next_in =3D rte_pktmbuf_mtod_offset(src, uint8_t *, > + op->src.offset); > + } > + > + if (op->dst.offset > dst->data_len) { > + segs =3D op->dst.offset / dst->data_len; > + for (int i =3D 0; i < segs; i++) > + dst =3D dst->next; Same as above. > + > + qp->stream->avail_out =3D dst->data_len - > + (op->dst.offset % dst->data_len); > + qp->stream->next_out =3D rte_pktmbuf_mtod_offset(dst, uint8_t > *, > + (op->dst.offset % dst->data_len)); ... > + if (remaining_data =3D=3D op->src.length) { > + consumed_data =3D src->data_len - qp->stream->avail_in > - > + (op->src.offset % src->data_len); > + } else > + consumed_data =3D src->data_len - qp->stream->avail_in; > + > + remaining_data -=3D consumed_data; What about "remaining_data =3D op->src.length - qp->stream->total_in"? This way, you don't need the condition (nor "consumed_data"). > + > + if (ret !=3D COMP_OK) { > + ISAL_PMD_LOG(ERR, "Compression operation > failed\n"); > + op->status =3D RTE_COMP_OP_STATUS_ERROR; > + return ret; > + } ... > + } else { > + ISAL_PMD_LOG(ERR, > + "Not enough input buffer segments\n" > + "Output Produced: %d\n", op- > >produced); This will always be 0, as it is only updated at the end of the function. > + op->status =3D > RTE_COMP_OP_STATUS_INVALID_ARGS; > + return -1; > + } > + } > + ... > + ISAL_PMD_LOG(ERR, > + "Not enough output buffer segments\n" > + "Output Produced: %d\n", op- > >produced); Same as above. > + op->status =3D > + > RTE_COMP_OP_STATUS_OUT_OF_SPACE_TERMINATED; > + return -1; > + } > + } > + } > + op->consumed =3D qp->stream->total_in; > + op->produced =3D qp->stream->total_out; > + > + return 0; > +} > + > +/* Decompression using chained mbufs for input/output data */ static > +int chained_mbuf_decompression(struct rte_comp_op *op, struct > +isal_comp_qp *qp) { Most comments from the compression function applies to the decompression on= e. > /* Stateless Compression Function */ > static int > process_isal_deflate(struct rte_comp_op *op, struct isal_comp_qp *qp, @@= - ... > + /* Chained mbufs */ > + if (op->m_src->nb_segs > 1 || op->m_dst->nb_segs > 1) { > + ret =3D chained_mbuf_compression(op, qp); > + if (ret < 0) //check the returns... ensure op status changed. > + return ret; > + } else { > + /* Linear buffer */ > + qp->stream->end_of_stream =3D 1; /* All input consumed in one > go */ > + if ((op->src.length + op->src.offset) > op->m_src->data_len) { This is unnecessary, as the check that you are doing above is effectively t= he same. > + ISAL_PMD_LOG(ERR, "Input mbuf not big enough for > offset.\n"); > + op->status =3D RTE_COMP_OP_STATUS_INVALID_ARGS; ... > + /* Point compression stream to output buffer */ > + qp->stream->avail_out =3D op->m_dst->data_len; Data_len - dst.offset? > + qp->stream->next_out =3D rte_pktmbuf_mtod_offset(op->m_dst, > uint8_t *, > + op->dst.offset); >=20 ... > @@ -291,57 +511,72 @@ process_isal_inflate(struct rte_comp_op *op, struct > isal_comp_qp *qp) ... > + /* Chained mbufs */ > + if (op->m_src->nb_segs > 1 || op->m_dst->nb_segs > 1) { > + ret =3D chained_mbuf_decompression(op, qp); > + if (ret !=3D 0) > + return ret; > + } else { > + /* Linear buffer */ > + if ((op->src.length + op->src.offset) > op->m_src->data_len) { This is unnecessary, as the check that you are doing above is effectively t= he same. > + ISAL_PMD_LOG(ERR, "Input mbuf not big enough for > offset.\n"); > + op->status =3D RTE_COMP_OP_STATUS_INVALID_ARGS; > return -1; > - } ... > + /* Point decompression state to output buffer */ > + qp->state->avail_out =3D op->m_dst->data_len; Data_len - dst.offset? ... > - if (ret !=3D ISAL_DECOMP_OK) { > - op->status =3D RTE_COMP_OP_STATUS_ERROR; > - return ret; > + op->consumed =3D op->src.length - qp->state->avail_in; What about "total_in"? > + op->produced =3D qp->state->total_out; > } >=20 > - op->consumed =3D op->src.length - qp->state->avail_in; > - op->produced =3D qp->state->total_out; > - > -return ret; > + return ret; > }