From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-io1-f68.google.com (mail-io1-f68.google.com [209.85.166.68]) by dpdk.org (Postfix) with ESMTP id E1D25239 for ; Mon, 18 Mar 2019 09:23:24 +0100 (CET) Received: by mail-io1-f68.google.com with SMTP id v4so12613717ioj.5 for ; Mon, 18 Mar 2019 01:23:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=oCcVrpmav/HSIwUajoqnU0uJDGh/e/EyW13LAC4/jLk=; b=LFWYCQ+ffSwxhhij72l2X7CfFrSkVJATMLorlv7460n7v9hXBDTb7FvCj0IAFHemrp dRZeoQPEunH8BbP3GQoLGGp/zu2sxzgb9/kaO0ptTmc59jEVrYmhRGvJ+eH6GN+NZ3Bc dQYbp8jeHX4DWdFRW/PZIQ4Sy2Vqn1K45XPCURj1IJWwZfRAN6i1d96vt7Dc8lEJ9u2N wZGEhAn6geZP59cXNVlQEs0SUqHaBuPCICr8pGyzrwsgjJqsMOwrxRGVszAcWg5NsKVj kDMo6PZTGSKLIX/VPq1IcK1NnajVJEl80RfKuLnn62hz22l7Fk+tbL9T8/uX+VRMGiWi 7K/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=oCcVrpmav/HSIwUajoqnU0uJDGh/e/EyW13LAC4/jLk=; b=Ek7zd8LpIKd4+vQix13Ul2k3GLbCVUZJ8TB2s0TR65+Rq+70o76GTsqLfg0KMeINgc oy2m8bZtHWtsjPEe4zgg5/YgVT63HvuxkHtMOVwR1KDaXi+kPiMgMmVdrcaUECnOl2Cp 9SgdBEaM35mGY4dqZ4yu6uYlcFnkJz+2BLDuCjTLxLMj47IdwT0QAa7rdYexFJ4joqqa 9ayz+YQIP09RSu93o5K+29TVsC5ElC2SfsQjwB+JHitatEuHFPlcDWL5qAShuivKy7U5 obcNwiHiXkleSdfDn09q1bKJi3pceKJyEX5BRbsXSyKYIRU6CEw5FlfFA31WkrXHVRZy UqIg== X-Gm-Message-State: APjAAAWL7J4NI/L9WR+1CWrUQwwLZdcEX4JZpQZMvgB2GrlB2D6N5+NM ipeEHC3B7KedivzfCwwGFw03PwBqVUAUl2MaPKk= X-Google-Smtp-Source: APXvYqw2qdQjKUjTbe5ocxtoMPXjeCFux1My+xRFFPRBIu9DhyyxbOaCXt38k82dVYETgxkWXGl5Y3YIvsE8ELD0QmI= X-Received: by 2002:a6b:b7d8:: with SMTP id h207mr10243092iof.210.1552897404088; Mon, 18 Mar 2019 01:23:24 -0700 (PDT) MIME-Version: 1.0 References: <1551439054-17658-1-git-send-email-tomaszx.jozwiak@intel.com> <1551960136-30358-1-git-send-email-tomaszx.jozwiak@intel.com> <1551960136-30358-2-git-send-email-tomaszx.jozwiak@intel.com> <7fff607f-1a5a-9dec-07e4-dc12e23b4bdf@nxp.com> In-Reply-To: From: arpita das Date: Mon, 18 Mar 2019 13:53:12 +0530 Message-ID: To: "Jozwiak, TomaszX" Cc: Akhil Goyal , "Trahe, Fiona" , "dev@dpdk.org" Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-dev] [PATCH v4 1/1] compress/qat: add dynamic sgl allocation X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Mar 2019 08:23:25 -0000 I know its a wrong forum, but just a 1 liner query, any openings in dodk , please let me know. TIA On Mon, 18 Mar 2019 at 1:43 PM, Jozwiak, TomaszX wrote: > Hi Akhil, > > Please take a look at cover letter (https://patches.dpdk.org/cover/50919/) > > There's dependency on 'malloc: add rte_realloc_socket function patch' > (1551429976-16297-1-git-send-email-tomaszx.jozwiak@intel.com) > which should be applied first. > > Br, Tomek > > > -----Original Message----- > > From: Akhil Goyal [mailto:akhil.goyal@nxp.com] > > Sent: Sunday, March 17, 2019 7:00 PM > > To: Jozwiak, TomaszX ; dev@dpdk.org; Trahe, > > Fiona > > Subject: Re: [dpdk-dev] [PATCH v4 1/1] compress/qat: add dynamic sgl > > allocation > > > > Hi Tomasz, > > > > I can see compilation failure in the patchwork CI tests. > > Could you please check. > > > > Thanks. > > > > On 3/7/2019 5:32 PM, Tomasz Jozwiak wrote: > > > This patch adds dynamic SGL allocation instead of static one. > > > The number of element in SGL can be adjusted in each operation depend > > > of the request. > > > > > > Signed-off-by: Tomasz Jozwiak > > > --- > > > config/common_base | 1 - > > > doc/guides/compressdevs/qat_comp.rst | 1 - > > > doc/guides/cryptodevs/qat.rst | 5 ---- > > > drivers/compress/qat/qat_comp.c | 58 > > ++++++++++++++++++++++++++++++++---- > > > drivers/compress/qat/qat_comp.h | 13 ++++---- > > > drivers/compress/qat/qat_comp_pmd.c | 49 > > +++++++++++++++++++++++++----- > > > 6 files changed, 101 insertions(+), 26 deletions(-) > > > > > > diff --git a/config/common_base b/config/common_base index > > > 0b09a93..91c7b73 100644 > > > --- a/config/common_base > > > +++ b/config/common_base > > > @@ -549,7 +549,6 @@ CONFIG_RTE_LIBRTE_PMD_QAT_SYM=n > > > # Max. number of QuickAssist devices, which can be detected and > > attached > > > # > > > CONFIG_RTE_PMD_QAT_MAX_PCI_DEVICES=48 > > > -CONFIG_RTE_PMD_QAT_COMP_SGL_MAX_SEGMENTS=16 > > > CONFIG_RTE_PMD_QAT_COMP_IM_BUFFER_SIZE=65536 > > > > > > # > > > diff --git a/doc/guides/compressdevs/qat_comp.rst > > > b/doc/guides/compressdevs/qat_comp.rst > > > index 5631cb1..6f583a4 100644 > > > --- a/doc/guides/compressdevs/qat_comp.rst > > > +++ b/doc/guides/compressdevs/qat_comp.rst > > > @@ -35,7 +35,6 @@ Limitations > > > * Compressdev level 0, no compression, is not supported. > > > * Queue pairs are not thread-safe (that is, within a single queue > pair, RX > > and TX from different lcores is not supported). > > > * No BSD support as BSD QAT kernel driver not available. > > > -* Number of segments in mbuf chains in the op must be <= > > RTE_PMD_QAT_COMP_SGL_MAX_SEGMENTS from the config file. > > > * When using Deflate dynamic huffman encoding for compression, the > > input size (op.src.length) > > > must be < CONFIG_RTE_PMD_QAT_COMP_IM_BUFFER_SIZE from the > > config file, > > > see :ref:`building_qat_config` for more details. > > > diff --git a/doc/guides/cryptodevs/qat.rst > > > b/doc/guides/cryptodevs/qat.rst index b7eace1..03bd0c1 100644 > > > --- a/doc/guides/cryptodevs/qat.rst > > > +++ b/doc/guides/cryptodevs/qat.rst > > > @@ -156,7 +156,6 @@ These are the build configuration options affecting > > QAT, and their default value > > > CONFIG_RTE_LIBRTE_PMD_QAT=y > > > CONFIG_RTE_LIBRTE_PMD_QAT_SYM=n > > > CONFIG_RTE_PMD_QAT_MAX_PCI_DEVICES=48 > > > - CONFIG_RTE_PMD_QAT_COMP_SGL_MAX_SEGMENTS=16 > > > CONFIG_RTE_PMD_QAT_COMP_IM_BUFFER_SIZE=65536 > > > > > > CONFIG_RTE_LIBRTE_PMD_QAT must be enabled for any QAT PMD to be > > built. > > > @@ -174,10 +173,6 @@ Note, there are separate config items for max > > cryptodevs CONFIG_RTE_CRYPTO_MAX_D > > > and max compressdevs CONFIG_RTE_COMPRESS_MAX_DEVS, if > > necessary these should be > > > adjusted to handle the total of QAT and other devices which the > process > > will use. > > > > > > -QAT allocates internal structures to handle SGLs. For the compression > > > service -CONFIG_RTE_PMD_QAT_COMP_SGL_MAX_SEGMENTS can be > > changed if more segments are needed. > > > -An extra (max_inflight_ops x 16) bytes per queue_pair will be used for > > every increment. > > > - > > > QAT compression PMD needs intermediate buffers to support Deflate > > compression > > > with Dynamic Huffman encoding. > > CONFIG_RTE_PMD_QAT_COMP_IM_BUFFER_SIZE > > > specifies the size of a single buffer, the PMD will allocate a > > > multiple of these, diff --git a/drivers/compress/qat/qat_comp.c > > > b/drivers/compress/qat/qat_comp.c index 32ca753..c021f4a 100644 > > > --- a/drivers/compress/qat/qat_comp.c > > > +++ b/drivers/compress/qat/qat_comp.c > > > @@ -1,5 +1,5 @@ > > > /* SPDX-License-Identifier: BSD-3-Clause > > > - * Copyright(c) 2018 Intel Corporation > > > + * Copyright(c) 2018-2019 Intel Corporation > > > */ > > > > > > #include > > > @@ -55,22 +55,70 @@ qat_comp_build_request(void *in_op, uint8_t > > *out_msg, > > > ICP_QAT_FW_COMN_PTR_TYPE_SET(comp_req- > > >comn_hdr.comn_req_flags, > > > QAT_COMN_PTR_TYPE_SGL); > > > > > > + if (unlikely(op->m_src->nb_segs > cookie->src_nb_elems)) { > > > + /* we need to allocate more elements in SGL*/ > > > + void *tmp; > > > + > > > + tmp = rte_realloc_socket(cookie->qat_sgl_src_d, > > > + sizeof(struct qat_sgl) + > > > + sizeof(struct qat_flat_buf) * > > > + op->m_src->nb_segs, 64, > > > + rte_socket_id()); > > > + > > > + if (unlikely(tmp == NULL)) { > > > + QAT_DP_LOG(ERR, "QAT PMD can't allocate > > memory" > > > + " for %d elements of SGL", > > > + op->m_src->nb_segs); > > > + op->status = > > RTE_COMP_OP_STATUS_INVALID_ARGS; > > > + return -ENOMEM; > > > + } > > > + /* new SGL is valid now */ > > > + cookie->qat_sgl_src_d = (struct qat_sgl *)tmp; > > > + cookie->src_nb_elems = op->m_src->nb_segs; > > > + cookie->qat_sgl_src_phys_addr = > > > + > rte_malloc_virt2iova(cookie->qat_sgl_src_d); > > > + } > > > + > > > ret = qat_sgl_fill_array(op->m_src, > > > op->src.offset, > > > - &cookie->qat_sgl_src, > > > + cookie->qat_sgl_src_d, > > > op->src.length, > > > - > > RTE_PMD_QAT_COMP_SGL_MAX_SEGMENTS); > > > + (const uint16_t)cookie->src_nb_elems); > > > if (ret) { > > > QAT_DP_LOG(ERR, "QAT PMD Cannot fill source sgl > > array"); > > > op->status = > > RTE_COMP_OP_STATUS_INVALID_ARGS; > > > return ret; > > > } > > > > > > + if (unlikely(op->m_dst->nb_segs > cookie->dst_nb_elems)) { > > > + /* we need to allocate more elements in SGL*/ > > > + struct qat_sgl *tmp; > > > + > > > + tmp = rte_realloc_socket(cookie->qat_sgl_dst_d, > > > + sizeof(struct qat_sgl) + > > > + sizeof(struct qat_flat_buf) * > > > + op->m_dst->nb_segs, 64, > > > + rte_socket_id()); > > > + > > > + if (unlikely(tmp == NULL)) { > > > + QAT_DP_LOG(ERR, "QAT PMD can't allocate > > memory" > > > + " for %d elements of SGL", > > > + op->m_dst->nb_segs); > > > + op->status = > > RTE_COMP_OP_STATUS_INVALID_ARGS; > > > + return -EINVAL; > > > + } > > > + /* new SGL is valid now */ > > > + cookie->qat_sgl_dst_d = (struct qat_sgl *)tmp; > > > + cookie->dst_nb_elems = op->m_dst->nb_segs; > > > + cookie->qat_sgl_dst_phys_addr = > > > + rte_malloc_virt2iova(cookie- > > >qat_sgl_dst_d); > > > + } > > > + > > > ret = qat_sgl_fill_array(op->m_dst, > > > op->dst.offset, > > > - &cookie->qat_sgl_dst, > > > + cookie->qat_sgl_dst_d, > > > comp_req->comp_pars.out_buffer_sz, > > > - > > RTE_PMD_QAT_COMP_SGL_MAX_SEGMENTS); > > > + (const uint16_t)cookie->dst_nb_elems); > > > if (ret) { > > > QAT_DP_LOG(ERR, "QAT PMD Cannot fill dest. sgl > > array"); > > > op->status = > > RTE_COMP_OP_STATUS_INVALID_ARGS; diff --git > > > a/drivers/compress/qat/qat_comp.h b/drivers/compress/qat/qat_comp.h > > > index 19f48df..2465f12 100644 > > > --- a/drivers/compress/qat/qat_comp.h > > > +++ b/drivers/compress/qat/qat_comp.h > > > @@ -1,5 +1,5 @@ > > > /* SPDX-License-Identifier: BSD-3-Clause > > > - * Copyright(c) 2015-2018 Intel Corporation > > > + * Copyright(c) 2015-2019 Intel Corporation > > > */ > > > > > > #ifndef _QAT_COMP_H_ > > > @@ -37,16 +37,15 @@ struct qat_inter_sgl { > > > struct qat_flat_buf buffers[QAT_NUM_BUFS_IN_IM_SGL]; > > > } __rte_packed __rte_cache_aligned; > > > > > > -struct qat_comp_sgl { > > > - qat_sgl_hdr; > > > - struct qat_flat_buf > > buffers[RTE_PMD_QAT_COMP_SGL_MAX_SEGMENTS]; > > > -} __rte_packed __rte_cache_aligned; > > > > > > struct qat_comp_op_cookie { > > > - struct qat_comp_sgl qat_sgl_src; > > > - struct qat_comp_sgl qat_sgl_dst; > > > phys_addr_t qat_sgl_src_phys_addr; > > > phys_addr_t qat_sgl_dst_phys_addr; > > > + /* dynamically created SGLs */ > > > + uint16_t src_nb_elems; > > > + uint16_t dst_nb_elems; > > > + struct qat_sgl *qat_sgl_src_d; > > > + struct qat_sgl *qat_sgl_dst_d; > > > }; > > > > > > struct qat_comp_xform { > > > diff --git a/drivers/compress/qat/qat_comp_pmd.c > > > b/drivers/compress/qat/qat_comp_pmd.c > > > index 27c8856..f034a19 100644 > > > --- a/drivers/compress/qat/qat_comp_pmd.c > > > +++ b/drivers/compress/qat/qat_comp_pmd.c > > > @@ -1,10 +1,14 @@ > > > /* SPDX-License-Identifier: BSD-3-Clause > > > - * Copyright(c) 2015-2018 Intel Corporation > > > + * Copyright(c) 2015-2019 Intel Corporation > > > */ > > > > > > +#include > > > + > > > #include "qat_comp.h" > > > #include "qat_comp_pmd.h" > > > > > > +#define QAT_PMD_COMP_SGL_DEF_SEGMENTS 16 > > > + > > > static const struct rte_compressdev_capabilities > > qat_comp_gen_capabilities[] = { > > > {/* COMPRESSION - deflate */ > > > .algo = RTE_COMP_ALGO_DEFLATE, > > > @@ -60,6 +64,10 @@ static int > > > qat_comp_qp_release(struct rte_compressdev *dev, uint16_t > > queue_pair_id) > > > { > > > struct qat_comp_dev_private *qat_private = dev->data- > > >dev_private; > > > + struct qat_qp **qp_addr = > > > + (struct qat_qp **)&(dev->data- > > >queue_pairs[queue_pair_id]); > > > + struct qat_qp *qp = (struct qat_qp *)*qp_addr; > > > + uint32_t i; > > > > > > QAT_LOG(DEBUG, "Release comp qp %u on device %d", > > > queue_pair_id, dev->data->dev_id); @@ - > > 67,6 +75,14 @@ > > > qat_comp_qp_release(struct rte_compressdev *dev, uint16_t > > queue_pair_id) > > > qat_private->qat_dev- > > >qps_in_use[QAT_SERVICE_COMPRESSION][queue_pair_id] > > > = NULL; > > > > > > + for (i = 0; i < qp->nb_descriptors; i++) { > > > + > > > + struct qat_comp_op_cookie *cookie = qp->op_cookies[i]; > > > + > > > + rte_free(cookie->qat_sgl_src_d); > > > + rte_free(cookie->qat_sgl_dst_d); > > > + } > > > + > > > return qat_qp_release((struct qat_qp **) > > > &(dev->data->queue_pairs[queue_pair_id])); > > > } > > > @@ -122,15 +138,34 @@ qat_comp_qp_setup(struct rte_compressdev > > *dev, uint16_t qp_id, > > > struct qat_comp_op_cookie *cookie = > > > qp->op_cookies[i]; > > > > > > + cookie->qat_sgl_src_d = rte_zmalloc_socket(NULL, > > > + sizeof(struct qat_sgl) + > > > + sizeof(struct qat_flat_buf) * > > > + > > QAT_PMD_COMP_SGL_DEF_SEGMENTS, > > > + 64, socket_id); > > > + > > > + cookie->qat_sgl_dst_d = rte_zmalloc_socket(NULL, > > > + sizeof(struct qat_sgl) + > > > + sizeof(struct qat_flat_buf) * > > > + > > QAT_PMD_COMP_SGL_DEF_SEGMENTS, > > > + 64, socket_id); > > > + > > > + if (cookie->qat_sgl_src_d == NULL || > > > + cookie->qat_sgl_dst_d == NULL) { > > > + QAT_LOG(ERR, "Can't allocate SGL" > > > + " for device %s", > > > + qat_private->qat_dev->name); > > > + return -ENOMEM; > > > + } > > > + > > > cookie->qat_sgl_src_phys_addr = > > > - rte_mempool_virt2iova(cookie) + > > > - offsetof(struct qat_comp_op_cookie, > > > - qat_sgl_src); > > > + > rte_malloc_virt2iova(cookie->qat_sgl_src_d); > > > > > > cookie->qat_sgl_dst_phys_addr = > > > - rte_mempool_virt2iova(cookie) + > > > - offsetof(struct qat_comp_op_cookie, > > > - qat_sgl_dst); > > > + rte_malloc_virt2iova(cookie- > > >qat_sgl_dst_d); > > > + > > > + cookie->dst_nb_elems = cookie->src_nb_elems = > > > + QAT_PMD_COMP_SGL_DEF_SEGMENTS; > > > } > > > > > > return ret; > > From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by dpdk.space (Postfix) with ESMTP id 69F76A0662 for ; Mon, 18 Mar 2019 09:23:27 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 8BEF425A1; Mon, 18 Mar 2019 09:23:26 +0100 (CET) Received: from mail-io1-f68.google.com (mail-io1-f68.google.com [209.85.166.68]) by dpdk.org (Postfix) with ESMTP id E1D25239 for ; Mon, 18 Mar 2019 09:23:24 +0100 (CET) Received: by mail-io1-f68.google.com with SMTP id v4so12613717ioj.5 for ; Mon, 18 Mar 2019 01:23:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=oCcVrpmav/HSIwUajoqnU0uJDGh/e/EyW13LAC4/jLk=; b=LFWYCQ+ffSwxhhij72l2X7CfFrSkVJATMLorlv7460n7v9hXBDTb7FvCj0IAFHemrp dRZeoQPEunH8BbP3GQoLGGp/zu2sxzgb9/kaO0ptTmc59jEVrYmhRGvJ+eH6GN+NZ3Bc dQYbp8jeHX4DWdFRW/PZIQ4Sy2Vqn1K45XPCURj1IJWwZfRAN6i1d96vt7Dc8lEJ9u2N wZGEhAn6geZP59cXNVlQEs0SUqHaBuPCICr8pGyzrwsgjJqsMOwrxRGVszAcWg5NsKVj kDMo6PZTGSKLIX/VPq1IcK1NnajVJEl80RfKuLnn62hz22l7Fk+tbL9T8/uX+VRMGiWi 7K/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=oCcVrpmav/HSIwUajoqnU0uJDGh/e/EyW13LAC4/jLk=; b=Ek7zd8LpIKd4+vQix13Ul2k3GLbCVUZJ8TB2s0TR65+Rq+70o76GTsqLfg0KMeINgc oy2m8bZtHWtsjPEe4zgg5/YgVT63HvuxkHtMOVwR1KDaXi+kPiMgMmVdrcaUECnOl2Cp 9SgdBEaM35mGY4dqZ4yu6uYlcFnkJz+2BLDuCjTLxLMj47IdwT0QAa7rdYexFJ4joqqa 9ayz+YQIP09RSu93o5K+29TVsC5ElC2SfsQjwB+JHitatEuHFPlcDWL5qAShuivKy7U5 obcNwiHiXkleSdfDn09q1bKJi3pceKJyEX5BRbsXSyKYIRU6CEw5FlfFA31WkrXHVRZy UqIg== X-Gm-Message-State: APjAAAWL7J4NI/L9WR+1CWrUQwwLZdcEX4JZpQZMvgB2GrlB2D6N5+NM ipeEHC3B7KedivzfCwwGFw03PwBqVUAUl2MaPKk= X-Google-Smtp-Source: APXvYqw2qdQjKUjTbe5ocxtoMPXjeCFux1My+xRFFPRBIu9DhyyxbOaCXt38k82dVYETgxkWXGl5Y3YIvsE8ELD0QmI= X-Received: by 2002:a6b:b7d8:: with SMTP id h207mr10243092iof.210.1552897404088; Mon, 18 Mar 2019 01:23:24 -0700 (PDT) MIME-Version: 1.0 References: <1551439054-17658-1-git-send-email-tomaszx.jozwiak@intel.com> <1551960136-30358-1-git-send-email-tomaszx.jozwiak@intel.com> <1551960136-30358-2-git-send-email-tomaszx.jozwiak@intel.com> <7fff607f-1a5a-9dec-07e4-dc12e23b4bdf@nxp.com> In-Reply-To: From: arpita das Date: Mon, 18 Mar 2019 13:53:12 +0530 Message-ID: To: "Jozwiak, TomaszX" Cc: Akhil Goyal , "Trahe, Fiona" , "dev@dpdk.org" Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-dev] [PATCH v4 1/1] compress/qat: add dynamic sgl allocation X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Message-ID: <20190318082312.uQXVh32DTwY_FTFDELalzruRjZgdF3Q9-kmB6c-fHvI@z> I know its a wrong forum, but just a 1 liner query, any openings in dodk , please let me know. TIA On Mon, 18 Mar 2019 at 1:43 PM, Jozwiak, TomaszX wrote: > Hi Akhil, > > Please take a look at cover letter (https://patches.dpdk.org/cover/50919/) > > There's dependency on 'malloc: add rte_realloc_socket function patch' > (1551429976-16297-1-git-send-email-tomaszx.jozwiak@intel.com) > which should be applied first. > > Br, Tomek > > > -----Original Message----- > > From: Akhil Goyal [mailto:akhil.goyal@nxp.com] > > Sent: Sunday, March 17, 2019 7:00 PM > > To: Jozwiak, TomaszX ; dev@dpdk.org; Trahe, > > Fiona > > Subject: Re: [dpdk-dev] [PATCH v4 1/1] compress/qat: add dynamic sgl > > allocation > > > > Hi Tomasz, > > > > I can see compilation failure in the patchwork CI tests. > > Could you please check. > > > > Thanks. > > > > On 3/7/2019 5:32 PM, Tomasz Jozwiak wrote: > > > This patch adds dynamic SGL allocation instead of static one. > > > The number of element in SGL can be adjusted in each operation depend > > > of the request. > > > > > > Signed-off-by: Tomasz Jozwiak > > > --- > > > config/common_base | 1 - > > > doc/guides/compressdevs/qat_comp.rst | 1 - > > > doc/guides/cryptodevs/qat.rst | 5 ---- > > > drivers/compress/qat/qat_comp.c | 58 > > ++++++++++++++++++++++++++++++++---- > > > drivers/compress/qat/qat_comp.h | 13 ++++---- > > > drivers/compress/qat/qat_comp_pmd.c | 49 > > +++++++++++++++++++++++++----- > > > 6 files changed, 101 insertions(+), 26 deletions(-) > > > > > > diff --git a/config/common_base b/config/common_base index > > > 0b09a93..91c7b73 100644 > > > --- a/config/common_base > > > +++ b/config/common_base > > > @@ -549,7 +549,6 @@ CONFIG_RTE_LIBRTE_PMD_QAT_SYM=n > > > # Max. number of QuickAssist devices, which can be detected and > > attached > > > # > > > CONFIG_RTE_PMD_QAT_MAX_PCI_DEVICES=48 > > > -CONFIG_RTE_PMD_QAT_COMP_SGL_MAX_SEGMENTS=16 > > > CONFIG_RTE_PMD_QAT_COMP_IM_BUFFER_SIZE=65536 > > > > > > # > > > diff --git a/doc/guides/compressdevs/qat_comp.rst > > > b/doc/guides/compressdevs/qat_comp.rst > > > index 5631cb1..6f583a4 100644 > > > --- a/doc/guides/compressdevs/qat_comp.rst > > > +++ b/doc/guides/compressdevs/qat_comp.rst > > > @@ -35,7 +35,6 @@ Limitations > > > * Compressdev level 0, no compression, is not supported. > > > * Queue pairs are not thread-safe (that is, within a single queue > pair, RX > > and TX from different lcores is not supported). > > > * No BSD support as BSD QAT kernel driver not available. > > > -* Number of segments in mbuf chains in the op must be <= > > RTE_PMD_QAT_COMP_SGL_MAX_SEGMENTS from the config file. > > > * When using Deflate dynamic huffman encoding for compression, the > > input size (op.src.length) > > > must be < CONFIG_RTE_PMD_QAT_COMP_IM_BUFFER_SIZE from the > > config file, > > > see :ref:`building_qat_config` for more details. > > > diff --git a/doc/guides/cryptodevs/qat.rst > > > b/doc/guides/cryptodevs/qat.rst index b7eace1..03bd0c1 100644 > > > --- a/doc/guides/cryptodevs/qat.rst > > > +++ b/doc/guides/cryptodevs/qat.rst > > > @@ -156,7 +156,6 @@ These are the build configuration options affecting > > QAT, and their default value > > > CONFIG_RTE_LIBRTE_PMD_QAT=y > > > CONFIG_RTE_LIBRTE_PMD_QAT_SYM=n > > > CONFIG_RTE_PMD_QAT_MAX_PCI_DEVICES=48 > > > - CONFIG_RTE_PMD_QAT_COMP_SGL_MAX_SEGMENTS=16 > > > CONFIG_RTE_PMD_QAT_COMP_IM_BUFFER_SIZE=65536 > > > > > > CONFIG_RTE_LIBRTE_PMD_QAT must be enabled for any QAT PMD to be > > built. > > > @@ -174,10 +173,6 @@ Note, there are separate config items for max > > cryptodevs CONFIG_RTE_CRYPTO_MAX_D > > > and max compressdevs CONFIG_RTE_COMPRESS_MAX_DEVS, if > > necessary these should be > > > adjusted to handle the total of QAT and other devices which the > process > > will use. > > > > > > -QAT allocates internal structures to handle SGLs. For the compression > > > service -CONFIG_RTE_PMD_QAT_COMP_SGL_MAX_SEGMENTS can be > > changed if more segments are needed. > > > -An extra (max_inflight_ops x 16) bytes per queue_pair will be used for > > every increment. > > > - > > > QAT compression PMD needs intermediate buffers to support Deflate > > compression > > > with Dynamic Huffman encoding. > > CONFIG_RTE_PMD_QAT_COMP_IM_BUFFER_SIZE > > > specifies the size of a single buffer, the PMD will allocate a > > > multiple of these, diff --git a/drivers/compress/qat/qat_comp.c > > > b/drivers/compress/qat/qat_comp.c index 32ca753..c021f4a 100644 > > > --- a/drivers/compress/qat/qat_comp.c > > > +++ b/drivers/compress/qat/qat_comp.c > > > @@ -1,5 +1,5 @@ > > > /* SPDX-License-Identifier: BSD-3-Clause > > > - * Copyright(c) 2018 Intel Corporation > > > + * Copyright(c) 2018-2019 Intel Corporation > > > */ > > > > > > #include > > > @@ -55,22 +55,70 @@ qat_comp_build_request(void *in_op, uint8_t > > *out_msg, > > > ICP_QAT_FW_COMN_PTR_TYPE_SET(comp_req- > > >comn_hdr.comn_req_flags, > > > QAT_COMN_PTR_TYPE_SGL); > > > > > > + if (unlikely(op->m_src->nb_segs > cookie->src_nb_elems)) { > > > + /* we need to allocate more elements in SGL*/ > > > + void *tmp; > > > + > > > + tmp = rte_realloc_socket(cookie->qat_sgl_src_d, > > > + sizeof(struct qat_sgl) + > > > + sizeof(struct qat_flat_buf) * > > > + op->m_src->nb_segs, 64, > > > + rte_socket_id()); > > > + > > > + if (unlikely(tmp == NULL)) { > > > + QAT_DP_LOG(ERR, "QAT PMD can't allocate > > memory" > > > + " for %d elements of SGL", > > > + op->m_src->nb_segs); > > > + op->status = > > RTE_COMP_OP_STATUS_INVALID_ARGS; > > > + return -ENOMEM; > > > + } > > > + /* new SGL is valid now */ > > > + cookie->qat_sgl_src_d = (struct qat_sgl *)tmp; > > > + cookie->src_nb_elems = op->m_src->nb_segs; > > > + cookie->qat_sgl_src_phys_addr = > > > + > rte_malloc_virt2iova(cookie->qat_sgl_src_d); > > > + } > > > + > > > ret = qat_sgl_fill_array(op->m_src, > > > op->src.offset, > > > - &cookie->qat_sgl_src, > > > + cookie->qat_sgl_src_d, > > > op->src.length, > > > - > > RTE_PMD_QAT_COMP_SGL_MAX_SEGMENTS); > > > + (const uint16_t)cookie->src_nb_elems); > > > if (ret) { > > > QAT_DP_LOG(ERR, "QAT PMD Cannot fill source sgl > > array"); > > > op->status = > > RTE_COMP_OP_STATUS_INVALID_ARGS; > > > return ret; > > > } > > > > > > + if (unlikely(op->m_dst->nb_segs > cookie->dst_nb_elems)) { > > > + /* we need to allocate more elements in SGL*/ > > > + struct qat_sgl *tmp; > > > + > > > + tmp = rte_realloc_socket(cookie->qat_sgl_dst_d, > > > + sizeof(struct qat_sgl) + > > > + sizeof(struct qat_flat_buf) * > > > + op->m_dst->nb_segs, 64, > > > + rte_socket_id()); > > > + > > > + if (unlikely(tmp == NULL)) { > > > + QAT_DP_LOG(ERR, "QAT PMD can't allocate > > memory" > > > + " for %d elements of SGL", > > > + op->m_dst->nb_segs); > > > + op->status = > > RTE_COMP_OP_STATUS_INVALID_ARGS; > > > + return -EINVAL; > > > + } > > > + /* new SGL is valid now */ > > > + cookie->qat_sgl_dst_d = (struct qat_sgl *)tmp; > > > + cookie->dst_nb_elems = op->m_dst->nb_segs; > > > + cookie->qat_sgl_dst_phys_addr = > > > + rte_malloc_virt2iova(cookie- > > >qat_sgl_dst_d); > > > + } > > > + > > > ret = qat_sgl_fill_array(op->m_dst, > > > op->dst.offset, > > > - &cookie->qat_sgl_dst, > > > + cookie->qat_sgl_dst_d, > > > comp_req->comp_pars.out_buffer_sz, > > > - > > RTE_PMD_QAT_COMP_SGL_MAX_SEGMENTS); > > > + (const uint16_t)cookie->dst_nb_elems); > > > if (ret) { > > > QAT_DP_LOG(ERR, "QAT PMD Cannot fill dest. sgl > > array"); > > > op->status = > > RTE_COMP_OP_STATUS_INVALID_ARGS; diff --git > > > a/drivers/compress/qat/qat_comp.h b/drivers/compress/qat/qat_comp.h > > > index 19f48df..2465f12 100644 > > > --- a/drivers/compress/qat/qat_comp.h > > > +++ b/drivers/compress/qat/qat_comp.h > > > @@ -1,5 +1,5 @@ > > > /* SPDX-License-Identifier: BSD-3-Clause > > > - * Copyright(c) 2015-2018 Intel Corporation > > > + * Copyright(c) 2015-2019 Intel Corporation > > > */ > > > > > > #ifndef _QAT_COMP_H_ > > > @@ -37,16 +37,15 @@ struct qat_inter_sgl { > > > struct qat_flat_buf buffers[QAT_NUM_BUFS_IN_IM_SGL]; > > > } __rte_packed __rte_cache_aligned; > > > > > > -struct qat_comp_sgl { > > > - qat_sgl_hdr; > > > - struct qat_flat_buf > > buffers[RTE_PMD_QAT_COMP_SGL_MAX_SEGMENTS]; > > > -} __rte_packed __rte_cache_aligned; > > > > > > struct qat_comp_op_cookie { > > > - struct qat_comp_sgl qat_sgl_src; > > > - struct qat_comp_sgl qat_sgl_dst; > > > phys_addr_t qat_sgl_src_phys_addr; > > > phys_addr_t qat_sgl_dst_phys_addr; > > > + /* dynamically created SGLs */ > > > + uint16_t src_nb_elems; > > > + uint16_t dst_nb_elems; > > > + struct qat_sgl *qat_sgl_src_d; > > > + struct qat_sgl *qat_sgl_dst_d; > > > }; > > > > > > struct qat_comp_xform { > > > diff --git a/drivers/compress/qat/qat_comp_pmd.c > > > b/drivers/compress/qat/qat_comp_pmd.c > > > index 27c8856..f034a19 100644 > > > --- a/drivers/compress/qat/qat_comp_pmd.c > > > +++ b/drivers/compress/qat/qat_comp_pmd.c > > > @@ -1,10 +1,14 @@ > > > /* SPDX-License-Identifier: BSD-3-Clause > > > - * Copyright(c) 2015-2018 Intel Corporation > > > + * Copyright(c) 2015-2019 Intel Corporation > > > */ > > > > > > +#include > > > + > > > #include "qat_comp.h" > > > #include "qat_comp_pmd.h" > > > > > > +#define QAT_PMD_COMP_SGL_DEF_SEGMENTS 16 > > > + > > > static const struct rte_compressdev_capabilities > > qat_comp_gen_capabilities[] = { > > > {/* COMPRESSION - deflate */ > > > .algo = RTE_COMP_ALGO_DEFLATE, > > > @@ -60,6 +64,10 @@ static int > > > qat_comp_qp_release(struct rte_compressdev *dev, uint16_t > > queue_pair_id) > > > { > > > struct qat_comp_dev_private *qat_private = dev->data- > > >dev_private; > > > + struct qat_qp **qp_addr = > > > + (struct qat_qp **)&(dev->data- > > >queue_pairs[queue_pair_id]); > > > + struct qat_qp *qp = (struct qat_qp *)*qp_addr; > > > + uint32_t i; > > > > > > QAT_LOG(DEBUG, "Release comp qp %u on device %d", > > > queue_pair_id, dev->data->dev_id); @@ - > > 67,6 +75,14 @@ > > > qat_comp_qp_release(struct rte_compressdev *dev, uint16_t > > queue_pair_id) > > > qat_private->qat_dev- > > >qps_in_use[QAT_SERVICE_COMPRESSION][queue_pair_id] > > > = NULL; > > > > > > + for (i = 0; i < qp->nb_descriptors; i++) { > > > + > > > + struct qat_comp_op_cookie *cookie = qp->op_cookies[i]; > > > + > > > + rte_free(cookie->qat_sgl_src_d); > > > + rte_free(cookie->qat_sgl_dst_d); > > > + } > > > + > > > return qat_qp_release((struct qat_qp **) > > > &(dev->data->queue_pairs[queue_pair_id])); > > > } > > > @@ -122,15 +138,34 @@ qat_comp_qp_setup(struct rte_compressdev > > *dev, uint16_t qp_id, > > > struct qat_comp_op_cookie *cookie = > > > qp->op_cookies[i]; > > > > > > + cookie->qat_sgl_src_d = rte_zmalloc_socket(NULL, > > > + sizeof(struct qat_sgl) + > > > + sizeof(struct qat_flat_buf) * > > > + > > QAT_PMD_COMP_SGL_DEF_SEGMENTS, > > > + 64, socket_id); > > > + > > > + cookie->qat_sgl_dst_d = rte_zmalloc_socket(NULL, > > > + sizeof(struct qat_sgl) + > > > + sizeof(struct qat_flat_buf) * > > > + > > QAT_PMD_COMP_SGL_DEF_SEGMENTS, > > > + 64, socket_id); > > > + > > > + if (cookie->qat_sgl_src_d == NULL || > > > + cookie->qat_sgl_dst_d == NULL) { > > > + QAT_LOG(ERR, "Can't allocate SGL" > > > + " for device %s", > > > + qat_private->qat_dev->name); > > > + return -ENOMEM; > > > + } > > > + > > > cookie->qat_sgl_src_phys_addr = > > > - rte_mempool_virt2iova(cookie) + > > > - offsetof(struct qat_comp_op_cookie, > > > - qat_sgl_src); > > > + > rte_malloc_virt2iova(cookie->qat_sgl_src_d); > > > > > > cookie->qat_sgl_dst_phys_addr = > > > - rte_mempool_virt2iova(cookie) + > > > - offsetof(struct qat_comp_op_cookie, > > > - qat_sgl_dst); > > > + rte_malloc_virt2iova(cookie- > > >qat_sgl_dst_d); > > > + > > > + cookie->dst_nb_elems = cookie->src_nb_elems = > > > + QAT_PMD_COMP_SGL_DEF_SEGMENTS; > > > } > > > > > > return ret; > >