From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id F0D85A04F3; Fri, 3 Jan 2020 12:29:22 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 7B2F01C243; Fri, 3 Jan 2020 12:29:22 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by dpdk.org (Postfix) with ESMTP id 16EF31C23D for ; Fri, 3 Jan 2020 12:29:20 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 003BQxYK020265; Fri, 3 Jan 2020 03:29:17 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-transfer-encoding : mime-version; s=pfpt0818; bh=K/pHhSLkA6r0rztsE0mjrnY2nCyO4AyTG4bqIj2Y+Sw=; b=WvNiGlw9VpcQaGSOPbZA2NCGT7oGipMNnFIF5JCe1QwoXOSyXvNiqTI8qsSjCD3zdwoo TaJGGyJJyrzqqXYhJpp+j5d3j4NRI0ptoz39eAscYUsi8gNSUBoZBopLbQjOGCFPqsva Y0nmr19vYlHyVvt2xY/j2MPuGApDsqYRl0AL0oYSoVv844Yp/dtnQX4RFgOUp0o0X8jA W8pCNhsOtLQ2HBqWSRpqEh4oGQT2lVHw65B9M1sqfUVtzO6duHo5rsW8dquPtx2XQF/z +8OB5kdjwjSLi9Pbaug6VoFvU81K/SwfaqS6SMpYLfP41AfP8R0QjxywMdWKfSeL05fZ Lw== Received: from sc-exch03.marvell.com ([199.233.58.183]) by mx0a-0016f401.pphosted.com with ESMTP id 2xa2g90bsq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Fri, 03 Jan 2020 03:29:16 -0800 Received: from SC-EXCH03.marvell.com (10.93.176.83) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Fri, 3 Jan 2020 03:29:15 -0800 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (104.47.55.177) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server (TLS) id 15.0.1497.2 via Frontend Transport; Fri, 3 Jan 2020 03:29:15 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=cGRw3nQvtP1KWROPj12VMMJoiGjTNoVO6lCIR1yM0N/HUk2atcKHXLwkU7PMU70EFopyP5Hz/tK6UbfvFF5GRT4NLw5YrgkXOsggGx2tPo4Gqv75aX1OjAbLob7ti5AcyJWFnAWOlCkjcbg5IFQ653gmHa6fdTPnRuFHBKHgxaqB4fUhJiT69MvZc8GkrO5d9IPVtRoGu5GKXG0UHOVSwsUwhB2VzXQOaLpvYk23noSAOXZbn+fBXUtQMpjcSnM8MlHlUkujiWaLDueXXtzUPHaiiefqFiSTHnFsbUhbT/aqBsdTIjBS163RgdwgC0xlZYFZnE2bHdiEWZZFX4wBxA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=K/pHhSLkA6r0rztsE0mjrnY2nCyO4AyTG4bqIj2Y+Sw=; b=fbu3G9++LrX0ZuUBt3ipOyHpAA7zQm/Pv+XWyGQNqzGfxmyan1F0fNADDrxD3RV9/dnfmAMbcg1A0QFHNBkwnOsRfuF23VoLQ+E1joE8vUY5C3DZlusvSx4i9W9M3v4xQrTJL24tiZniRZ1Fe+QYXjMM3I80iDIrHHRWrNLjk/3NTi+dj1IJ3Az9kdD5XFJmIFHIMR41YEEjvRxUTZcQqejcJRUA8ZemgbqGqv3RhirA/8miElEmURlcFKoc6VrJeBxYdPBkzL7B0wgrC1YmumDERC2VykV2HpVjfC+qDzAcxgsbfteANMrshF/UGdTAWk75CzXwCbZV+D/xX7R4+A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=K/pHhSLkA6r0rztsE0mjrnY2nCyO4AyTG4bqIj2Y+Sw=; b=ZwxCHOrKgzW+3UpCVr6gvG0sI5cxfLt0MjQXnlHBPXcjjPQJetywtXZfC3cebyh894+9AooEZAa8/B7BWDXHrz+30BK9V8LzbZjEmUMaCW2KLT7D7RzD1yD2VC4k9WHkK56QU6lorB6QAxf6/gskWo1NBJfv1LhowsjYYRI6ELQ= Received: from MWHPR18MB1376.namprd18.prod.outlook.com (10.173.243.18) by MWHPR18MB1166.namprd18.prod.outlook.com (10.173.125.17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2602.12; Fri, 3 Jan 2020 11:29:12 +0000 Received: from MWHPR18MB1376.namprd18.prod.outlook.com ([fe80::3d27:53fc:7a3a:f73a]) by MWHPR18MB1376.namprd18.prod.outlook.com ([fe80::3d27:53fc:7a3a:f73a%5]) with mapi id 15.20.2602.012; Fri, 3 Jan 2020 11:29:12 +0000 From: Mahipal Challa To: Gavin Hu , "dev@dpdk.org" CC: Jerin Jacob Kollanukkaran , "Narayana Prasad Raju Athreya" , Subrahmanyam Nilla , Venkateshwarlu Nalla , nd Thread-Topic: [dpdk-dev] [dpdk-dev PATCH v2 4/6] raw/octeontx2_ep: add enqueue operation Thread-Index: AQHVvLLhz2TE8Arz0EWnh9lkNlDh3afXOWwAgAGZm6A= Date: Fri, 3 Jan 2020 11:29:11 +0000 Message-ID: References: <1577450386-16966-1-git-send-email-mchalla@marvell.com> <1577450386-16966-5-git-send-email-mchalla@marvell.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [115.113.156.2] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 272bc169-d46f-4f69-6135-08d790402817 x-ms-traffictypediagnostic: MWHPR18MB1166: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-forefront-prvs: 0271483E06 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(4636009)(136003)(396003)(39860400002)(366004)(376002)(346002)(13464003)(199004)(189003)(478600001)(55236004)(26005)(186003)(53546011)(6506007)(7696005)(30864003)(9686003)(5660300002)(33656002)(52536014)(2906002)(55016002)(86362001)(66556008)(316002)(71200400001)(4326008)(66946007)(66446008)(110136005)(66476007)(64756008)(76116006)(81156014)(8936002)(81166006)(54906003); DIR:OUT; SFP:1101; SCL:1; SRVR:MWHPR18MB1166; H:MWHPR18MB1376.namprd18.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: marvell.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: YAoLoeRnMjWT9Ify+N/kk8WG2xupL6oOsLomjrUrtLlDFf9oTNBU2TgL/+g54iWbMA9eJNnI8gUmINor6O+4E41ydDl5vc+thupwPiNe9aouT02VL6CYLJc9aRweK+BCc9rnLhoaJtzTHofDIT0kBayOXq+aBVdtdrhETa57ksdPQXKQpF+0fj5yYRpw6/6DTP0GbOalO1M72JUFsQ92Xtn4PrDNZcKAJ18LxSnvrOkn7LMxsA5IXYUzZZGFVP35hlWtcDgY9TZlTs27FxMGgdvMNfjBii8QV4mjHcTlfDMCZjKBeBy0kF1dNiGY6VddLfCvo8sPwF7UH8qMTXWrwjxoPffQjS0IjsjFWq4IC0NpDoHT1qRL3YCivoUeuHOiQxZRm4MtlegAruRl7Fy++2xLpiVoe/3kTqLymSypzJEtDtOrFVW/srslW33QHWYxtKfH+qtwJ2TTOP7vvdqf4qKg4xcrRD07VBMu9VuXwnbLoGq0Y1fYZdX/kZ9fW0DB Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: 272bc169-d46f-4f69-6135-08d790402817 X-MS-Exchange-CrossTenant-originalarrivaltime: 03 Jan 2020 11:29:11.8963 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 7I6xfvHwo3qvNbwGw6p3uh9UrV/6NAZIRhINRG6uZ743k3NgOJRNMU1BAa4WUZfLF8g9wySPp4upeoBaKJ2ryQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR18MB1166 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,18.0.572 definitions=2020-01-03_03:2020-01-02,2020-01-03 signatures=0 Subject: Re: [dpdk-dev] [dpdk-dev PATCH v2 4/6] raw/octeontx2_ep: add enqueue operation X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Gavin, Please see response inline. > -----Original Message----- > From: Gavin Hu > Sent: Thursday, January 2, 2020 4:13 PM > To: Mahipal Challa ; dev@dpdk.org > Cc: Jerin Jacob Kollanukkaran ; Narayana Prasad Raju > Athreya ; Subrahmanyam Nilla > ; Venkateshwarlu Nalla ; nd > > Subject: [EXT] RE: [dpdk-dev] [dpdk-dev PATCH v2 4/6] raw/octeontx2_ep: > add enqueue operation >=20 > External Email >=20 > ---------------------------------------------------------------------- > Hi Mahipal, >=20 > > -----Original Message----- > > From: dev On Behalf Of Mahipal Challa > > Sent: Friday, December 27, 2019 8:40 PM > > To: dev@dpdk.org > > Cc: jerinj@marvell.com; pathreya@marvell.com; snilla@marvell.com; > > venkatn@marvell.com > > Subject: [dpdk-dev] [dpdk-dev PATCH v2 4/6] raw/octeontx2_ep: add > > enqueue operation > > > > Add rawdev enqueue operation for SDP VF devices. > > > > Signed-off-by: Mahipal Challa > > --- > > doc/guides/rawdevs/octeontx2_ep.rst | 6 + > > drivers/raw/octeontx2_ep/otx2_ep_enqdeq.c | 242 > > ++++++++++++++++++++++++++++++ > > drivers/raw/octeontx2_ep/otx2_ep_enqdeq.h | 39 +++++ > > drivers/raw/octeontx2_ep/otx2_ep_rawdev.c | 1 + > > drivers/raw/octeontx2_ep/otx2_ep_rawdev.h | 20 +++ > > drivers/raw/octeontx2_ep/otx2_ep_vf.c | 24 +++ > > 6 files changed, 332 insertions(+) > > > > diff --git a/doc/guides/rawdevs/octeontx2_ep.rst > > b/doc/guides/rawdevs/octeontx2_ep.rst > > index 2507fcf..39a7c29 100644 > > --- a/doc/guides/rawdevs/octeontx2_ep.rst > > +++ b/doc/guides/rawdevs/octeontx2_ep.rst > > @@ -68,3 +68,9 @@ The following code shows how the device is > > configured > > > > rte_rawdev_configure(dev_id, (rte_rawdev_obj_t)&rdev_info); > > > > +Performing Data Transfer > > +------------------------ > > + > > +To perform data transfer using SDP VF EP rawdev devices use standard > > +``rte_rawdev_enqueue_buffers()`` and ``rte_rawdev_dequeue_buffers()`` > > APIs. > > + > > diff --git a/drivers/raw/octeontx2_ep/otx2_ep_enqdeq.c > > b/drivers/raw/octeontx2_ep/otx2_ep_enqdeq.c > > index 584b818..6910f08 100644 > > --- a/drivers/raw/octeontx2_ep/otx2_ep_enqdeq.c > > +++ b/drivers/raw/octeontx2_ep/otx2_ep_enqdeq.c > > @@ -403,3 +403,245 @@ > > return -ENOMEM; > > } > > > > +static inline void > > +sdp_iqreq_delete(struct sdp_device *sdpvf, > > + struct sdp_instr_queue *iq, uint32_t idx) { > > + uint32_t reqtype; > > + void *buf; > > + > > + buf =3D iq->req_list[idx].buf; > > + reqtype =3D iq->req_list[idx].reqtype; > > + > > + switch (reqtype) { > > + case SDP_REQTYPE_NORESP: > > + rte_mempool_put(sdpvf->enqdeq_mpool, buf); > > + otx2_sdp_dbg("IQ buffer freed at idx[%d]", idx); > > + break; > > + > > + case SDP_REQTYPE_NORESP_GATHER: > > + case SDP_REQTYPE_NONE: > > + default: > > + otx2_info("This iqreq mode is not supported:%d", reqtype); > > + > > + } > > + > > + /* Reset the request list at this index */ > > + iq->req_list[idx].buf =3D NULL; > > + iq->req_list[idx].reqtype =3D 0; > > +} > > + > > +static inline void > > +sdp_iqreq_add(struct sdp_instr_queue *iq, void *buf, > > + uint32_t reqtype) > > +{ > > + iq->req_list[iq->host_write_index].buf =3D buf; > > + iq->req_list[iq->host_write_index].reqtype =3D reqtype; > > + > > + otx2_sdp_dbg("IQ buffer added at idx[%d]", iq->host_write_index); > > + > > +} > > + > > +static void > > +sdp_flush_iq(struct sdp_device *sdpvf, > > + struct sdp_instr_queue *iq, > > + uint32_t pending_thresh __rte_unused) { > > + uint32_t instr_processed =3D 0; > > + > > + rte_spinlock_lock(&iq->lock); > > + > > + iq->otx_read_index =3D sdpvf->fn_list.update_iq_read_idx(iq); > > + while (iq->flush_index !=3D iq->otx_read_index) { > > + /* Free the IQ data buffer to the pool */ > > + sdp_iqreq_delete(sdpvf, iq, iq->flush_index); > > + iq->flush_index =3D > > + sdp_incr_index(iq->flush_index, 1, iq->nb_desc); > > + > > + instr_processed++; > > + } > > + > > + iq->stats.instr_processed =3D instr_processed; > > + rte_atomic64_sub(&iq->instr_pending, instr_processed); > > + > > + rte_spinlock_unlock(&iq->lock); > > +} > > + > > +static inline void > > +sdp_ring_doorbell(struct sdp_device *sdpvf __rte_unused, > > + struct sdp_instr_queue *iq) > > +{ > > + otx2_write64(iq->fill_cnt, iq->doorbell_reg); > > + > > + /* Make sure doorbell write goes through */ > > + rte_cio_wmb(); > This rte_cio_wmb() should be moved before the doorbell ring to guarantee > the IQ command submitted in post_iqcmd is observed by the peer HW > before observing the doorbell ring. [Mahipal]: Yes, IQ command is ensured to be in memory well before the doorb= ell ring operation, this is done by "post_iqcmd" routine(please see below). And this "rte_cio_wmb()" is just to ensure the doorbell write operation com= pletion(As per your suggestion in v1, we changed this from rte_wmb() to rte= _cio_wmb()). > > + iq->fill_cnt =3D 0; > > + > > +} > > + > > +static inline int > > +post_iqcmd(struct sdp_instr_queue *iq, uint8_t *iqcmd) { > > + uint8_t *iqptr, cmdsize; > > + > > + /* This ensures that the read index does not wrap around to > > + * the same position if queue gets full before OCTEON TX2 could > > + * fetch any instr. > > + */ > > + if (rte_atomic64_read(&iq->instr_pending) >=3D > > + (int32_t)(iq->nb_desc - 1)) { > > + otx2_err("IQ is full, pending:%ld", > > + (long)rte_atomic64_read(&iq->instr_pending)); > > + > > + return SDP_IQ_SEND_FAILED; > > + } > > + > > + /* Copy cmd into iq */ > > + cmdsize =3D ((iq->iqcmd_64B) ? 64 : 32); > > + iqptr =3D iq->base_addr + (cmdsize * iq->host_write_index); > > + > > + rte_memcpy(iqptr, iqcmd, cmdsize); > This is the DMA memory must be observed by the peer HW before observing > the doorbell ring. >=20 [Mahipal]: Yes, this is ensured, please see the below comment. > > + > > + otx2_sdp_dbg("IQ cmd posted @ index:%d", iq->host_write_index); > > + > > + /* Increment the host write index */ > > + iq->host_write_index =3D > > + sdp_incr_index(iq->host_write_index, 1, iq->nb_desc); > > + > > + iq->fill_cnt++; > > + > > + /* Flush the command into memory. We need to be sure the data > > + * is in memory before indicating that the instruction is > > + * pending. > > + */ > > + rte_io_wmb(); > Who is the observer of instr_pending? If it is the peer HW, then it shoul= d be > rte_cio_wmb. > Otherwise if it is a peer lcore, rte_smp_wmb is the right one, or if both= are > observers, then cio is correct one. [Mahipal]: This is to ensure all the write operation completions(including = the above IQ command memory write operation) before the doorbell ring opera= tion. Just to be very sure, would like to go with rte_io_wmb() instead of rte_cio= _wmb(). > > + rte_atomic64_inc(&iq->instr_pending); > > + > > + /* SDP_IQ_SEND_SUCCESS */ > > + return 0; > > +} > > + > > + > > +static int > > +sdp_send_data(struct sdp_device *sdpvf, > > + struct sdp_instr_queue *iq, void *cmd) { > > + uint32_t ret; > > + > > + /* Lock this IQ command queue before posting instruction */ > > + rte_spinlock_lock(&iq->post_lock); > > + > > + /* Submit IQ command */ > > + ret =3D post_iqcmd(iq, cmd); > > + > > + if (ret =3D=3D SDP_IQ_SEND_SUCCESS) { > > + sdp_ring_doorbell(sdpvf, iq); > > + > > + iq->stats.instr_posted++; > > + otx2_sdp_dbg("Instr submit success posted: %ld\n", > > + (long)iq->stats.instr_posted); > > + > > + } else { > > + iq->stats.instr_dropped++; > > + otx2_err("Instr submit failled, dropped: %ld\n", > > + (long)iq->stats.instr_dropped); > > + > > + } > > + > > + rte_spinlock_unlock(&iq->post_lock); > > + > > + return ret; > > +} > > + > > + > > +/* Enqueue requests/packets to SDP IQ queue. > > + * returns number of requests enqueued successfully */ int > > +sdp_rawdev_enqueue(struct rte_rawdev *rawdev, > > + struct rte_rawdev_buf **buffers __rte_unused, > > + unsigned int count, rte_rawdev_obj_t context) { > > + struct sdp_instr_64B *iqcmd; > > + struct sdp_instr_queue *iq; > > + struct sdp_soft_instr *si; > > + struct sdp_device *sdpvf; > > + > > + struct sdp_instr_ih ihx; > > + > > + sdpvf =3D (struct sdp_device *)rawdev->dev_private; > > + si =3D (struct sdp_soft_instr *)context; > > + > > + iq =3D sdpvf->instr_queue[si->q_no]; > > + > > + if ((count > 1) || (count < 1)) { > > + otx2_err("This mode not supported: req[%d]", count); > > + goto enq_fail; > > + } > > + > > + memset(&ihx, 0, sizeof(struct sdp_instr_ih)); > > + > > + iqcmd =3D &si->command; > > + memset(iqcmd, 0, sizeof(struct sdp_instr_64B)); > > + > > + iqcmd->dptr =3D (uint64_t)si->dptr; > > + > > + /* Populate SDP IH */ > > + ihx.pkind =3D sdpvf->pkind; > > + ihx.fsz =3D si->ih.fsz + 8; /* 8B for NIX IH */ > > + ihx.gather =3D si->ih.gather; > > + > > + /* Direct data instruction */ > > + ihx.tlen =3D si->ih.tlen + ihx.fsz; > > + > > + switch (ihx.gather) { > > + case 0: /* Direct data instr */ > > + ihx.tlen =3D si->ih.tlen + ihx.fsz; > > + break; > > + > > + default: /* Gather */ > > + switch (si->ih.gsz) { > > + case 0: /* Direct gather instr */ > > + otx2_err("Direct Gather instr : not supported"); > > + goto enq_fail; > > + > > + default: /* Indirect gather instr */ > > + otx2_err("Indirect Gather instr : not supported"); > > + goto enq_fail; > > + } > > + } > > + > > + rte_memcpy(&iqcmd->ih, &ihx, sizeof(uint64_t)); > > + iqcmd->rptr =3D (uint64_t)si->rptr; > > + rte_memcpy(&iqcmd->irh, &si->irh, sizeof(uint64_t)); > > + > > + /* Swap FSZ(front data) here, to avoid swapping on OCTEON TX2 side > > */ > > + sdp_swap_8B_data(&iqcmd->rptr, 1); > > + sdp_swap_8B_data(&iqcmd->irh, 1); > > + > > + otx2_sdp_dbg("After swapping"); > > + otx2_sdp_dbg("Word0 [dptr]: 0x%016lx", (unsigned long)iqcmd- > >dptr); > > + otx2_sdp_dbg("Word1 [ihtx]: 0x%016lx", (unsigned long)iqcmd->ih); > > + otx2_sdp_dbg("Word2 [rptr]: 0x%016lx", (unsigned long)iqcmd- > >rptr); > > + otx2_sdp_dbg("Word3 [irh]: 0x%016lx", (unsigned long)iqcmd->irh); > > + otx2_sdp_dbg("Word4 [exhdr[0]]: 0x%016lx", > > + (unsigned long)iqcmd->exhdr[0]); > > + > > + sdp_iqreq_add(iq, si->dptr, si->reqtype); > > + > > + if (sdp_send_data(sdpvf, iq, iqcmd)) { > > + otx2_err("Data send failled :"); > > + sdp_iqreq_delete(sdpvf, iq, iq->host_write_index); > > + goto enq_fail; > > + } > > + > > + if (rte_atomic64_read(&iq->instr_pending) >=3D 1) > > + sdp_flush_iq(sdpvf, iq, 1 /*(iq->nb_desc / 2)*/); > > + > > + /* Return no# of instructions posted successfully. */ > > + return count; > > + > > +enq_fail: > > + return SDP_IQ_SEND_FAILED; > > +} > > + > > diff --git a/drivers/raw/octeontx2_ep/otx2_ep_enqdeq.h > > b/drivers/raw/octeontx2_ep/otx2_ep_enqdeq.h > > index 4c28283..b9b7c0b 100644 > > --- a/drivers/raw/octeontx2_ep/otx2_ep_enqdeq.h > > +++ b/drivers/raw/octeontx2_ep/otx2_ep_enqdeq.h > > @@ -8,4 +8,43 @@ > > #include > > #include "otx2_ep_rawdev.h" > > > > +#define SDP_IQ_SEND_FAILED (-1) > > +#define SDP_IQ_SEND_SUCCESS (0) > > + > > + > > +static inline uint64_t > > +sdp_endian_swap_8B(uint64_t _d) > > +{ > > + return ((((((uint64_t)(_d)) >> 0) & (uint64_t)0xff) << 56) | > > + (((((uint64_t)(_d)) >> 8) & (uint64_t)0xff) << 48) | > > + (((((uint64_t)(_d)) >> 16) & (uint64_t)0xff) << 40) | > > + (((((uint64_t)(_d)) >> 24) & (uint64_t)0xff) << 32) | > > + (((((uint64_t)(_d)) >> 32) & (uint64_t)0xff) << 24) | > > + (((((uint64_t)(_d)) >> 40) & (uint64_t)0xff) << 16) | > > + (((((uint64_t)(_d)) >> 48) & (uint64_t)0xff) << 8) | > > + (((((uint64_t)(_d)) >> 56) & (uint64_t)0xff) << 0)); } > > + > > +static inline void > > +sdp_swap_8B_data(uint64_t *data, uint32_t blocks) { > > + /* Swap 8B blocks */ > > + while (blocks) { > > + *data =3D sdp_endian_swap_8B(*data); > > + blocks--; > > + data++; > > + } > > +} > > + > > +static inline uint32_t > > +sdp_incr_index(uint32_t index, uint32_t count, uint32_t max) { > > + if ((index + count) >=3D max) > > + index =3D index + count - max; > > + else > > + index +=3D count; > > + > > + return index; > > +} > > + > > #endif /* _OTX2_EP_ENQDEQ_H_ */ > > diff --git a/drivers/raw/octeontx2_ep/otx2_ep_rawdev.c > > b/drivers/raw/octeontx2_ep/otx2_ep_rawdev.c > > index 3db5a74..22a6beb 100644 > > --- a/drivers/raw/octeontx2_ep/otx2_ep_rawdev.c > > +++ b/drivers/raw/octeontx2_ep/otx2_ep_rawdev.c > > @@ -251,6 +251,7 @@ > > .dev_start =3D sdp_rawdev_start, > > .dev_stop =3D sdp_rawdev_stop, > > .dev_close =3D sdp_rawdev_close, > > + .enqueue_bufs =3D sdp_rawdev_enqueue, > > }; > > > > static int > > diff --git a/drivers/raw/octeontx2_ep/otx2_ep_rawdev.h > > b/drivers/raw/octeontx2_ep/otx2_ep_rawdev.h > > index a01f48d..8fd06fb 100644 > > --- a/drivers/raw/octeontx2_ep/otx2_ep_rawdev.h > > +++ b/drivers/raw/octeontx2_ep/otx2_ep_rawdev.h > > @@ -8,6 +8,10 @@ > > #include > > #include > > > > +/* IQ instruction req types */ > > +#define SDP_REQTYPE_NONE (0) > > +#define SDP_REQTYPE_NORESP (1) > > +#define SDP_REQTYPE_NORESP_GATHER (2) > > > > /* Input Request Header format */ > > struct sdp_instr_irh { > > @@ -128,6 +132,13 @@ struct sdp_instr_list { }; > > #define SDP_IQREQ_LIST_SIZE (sizeof(struct sdp_instr_list)) > > > > +/* Input Queue statistics. Each input queue has four stats fields. */ > > +struct sdp_iq_stats { > > + uint64_t instr_posted; /* Instructions posted to this queue. */ > > + uint64_t instr_processed; /* Instructions processed in this queue. */ > > + uint64_t instr_dropped; /* Instructions that could not be processed > > +*/ }; > > + > > /* Structure to define the configuration attributes for each Input > > queue. */ struct sdp_iq_config { > > /* Max number of IQs available */ > > @@ -195,6 +206,9 @@ struct sdp_instr_queue { > > /* Number of instructions pending to be posted to OCTEON TX2. */ > > uint32_t fill_cnt; > > > > + /* Statistics for this input queue. */ > > + struct sdp_iq_stats stats; > > + > > /* DMA mapped base address of the input descriptor ring. */ > > uint64_t base_addr_dma; > > > > @@ -380,6 +394,8 @@ struct sdp_fn_list { > > void (*setup_oq_regs)(struct sdp_device *sdpvf, uint32_t q_no); > > > > int (*setup_device_regs)(struct sdp_device *sdpvf); > > + uint32_t (*update_iq_read_idx)(struct sdp_instr_queue *iq); > > + > > void (*enable_io_queues)(struct sdp_device *sdpvf); > > void (*disable_io_queues)(struct sdp_device *sdpvf); > > > > @@ -458,4 +474,8 @@ struct sdp_device { int sdp_setup_oqs(struct > > sdp_device *sdpvf, uint32_t oq_no); int sdp_delete_oqs(struct > > sdp_device *sdpvf, uint32_t oq_no); > > > > +int sdp_rawdev_enqueue(struct rte_rawdev *dev, struct rte_rawdev_buf > > **buffers, > > + unsigned int count, rte_rawdev_obj_t context); > > + > > + > > #endif /* _OTX2_EP_RAWDEV_H_ */ > > diff --git a/drivers/raw/octeontx2_ep/otx2_ep_vf.c > > b/drivers/raw/octeontx2_ep/otx2_ep_vf.c > > index 8e79fe8..c5c0bc3 100644 > > --- a/drivers/raw/octeontx2_ep/otx2_ep_vf.c > > +++ b/drivers/raw/octeontx2_ep/otx2_ep_vf.c > > @@ -409,6 +409,28 @@ > > sdp_vf_disable_oq(sdpvf, q_no); > > } > > > > +static uint32_t > > +sdp_vf_update_read_index(struct sdp_instr_queue *iq) { > > + uint32_t new_idx =3D rte_read32(iq->inst_cnt_reg); > > + > > + /* The new instr cnt reg is a 32-bit counter that can roll over. > > + * We have noted the counter's initial value at init time into > > + * reset_instr_cnt > > + */ > > + if (iq->reset_instr_cnt < new_idx) > > + new_idx -=3D iq->reset_instr_cnt; > > + else > > + new_idx +=3D (0xffffffff - iq->reset_instr_cnt) + 1; > > + > > + /* Modulo of the new index with the IQ size will give us > > + * the new index. > > + */ > > + new_idx %=3D iq->nb_desc; > > + > > + return new_idx; > > +} > > + > > int > > sdp_vf_setup_device(struct sdp_device *sdpvf) { @@ -436,6 +458,8 @@ > > sdpvf->fn_list.setup_oq_regs =3D sdp_vf_setup_oq_regs; > > > > sdpvf->fn_list.setup_device_regs =3D sdp_vf_setup_device_regs; > > + sdpvf->fn_list.update_iq_read_idx =3D sdp_vf_update_read_index; > > + > > sdpvf->fn_list.enable_io_queues =3D sdp_vf_enable_io_queues; > > sdpvf->fn_list.disable_io_queues =3D sdp_vf_disable_io_queues; > > > > -- > > 1.8.3.1 Thanks, Mahipal