From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id A53D7A04F1; Sat, 14 Dec 2019 17:18:28 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id EDEB11BFB8; Sat, 14 Dec 2019 17:18:27 +0100 (CET) Received: from EUR03-DB5-obe.outbound.protection.outlook.com (mail-eopbgr40061.outbound.protection.outlook.com [40.107.4.61]) by dpdk.org (Postfix) with ESMTP id A0C451BFB6 for ; Sat, 14 Dec 2019 17:18:26 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4She6XqMVJbSknC76Ym/pRyC9SJgnJTxSenAoRQw0t0=; b=cay7ymbaO6N02r/12LDcFJgC5MMsiVIIFvEfRHI8R74Na8DgkeZU08KSuuP2BSk/VeG8tg2Ug2Tc5Tg15BCxAGNfHeWKnAQon1u07dCourPip9HH+cd/LK3/AL/m0y4QyBU0JD6El26TJlj5BUbEGbQs3hNsfh/hh2pPgixsAlg= Received: from HE1PR0802CA0020.eurprd08.prod.outlook.com (2603:10a6:3:bd::30) by VI1PR0801MB2032.eurprd08.prod.outlook.com (2603:10a6:800:8c::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2538.17; Sat, 14 Dec 2019 16:18:24 +0000 Received: from VE1EUR03FT040.eop-EUR03.prod.protection.outlook.com (2a01:111:f400:7e09::205) by HE1PR0802CA0020.outlook.office365.com (2603:10a6:3:bd::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2538.16 via Frontend Transport; Sat, 14 Dec 2019 16:18:23 +0000 Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dpdk.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dpdk.org; dmarc=bestguesspass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by VE1EUR03FT040.mail.protection.outlook.com (10.152.18.210) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2495.18 via Frontend Transport; Sat, 14 Dec 2019 16:18:22 +0000 Received: ("Tessian outbound ba41a0333779:v40"); Sat, 14 Dec 2019 16:18:22 +0000 X-CR-MTA-TID: 64aa7808 Received: from b1fcb3d8b980.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 620AFCD1-EC66-4C7C-90C1-9F38CDA0EABD.1; Sat, 14 Dec 2019 16:18:17 +0000 Received: from EUR01-DB5-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id b1fcb3d8b980.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Sat, 14 Dec 2019 16:18:17 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=F4N8yr3aPM2jiKGC0RbVQ3os5bQ0nREBWBktwgugg4mNkGHO/MhZOAgM60GGIUKfaEknWM8BvJCYx+wBxe1cQGuyrd40+/Y979tjr0Hh0DHtjsJZYxplXqjCyiWoQanlWvvC4sJEBn3gJpmYqD/59dEkyVLXvDG/1DoTtizV9ivgegzewl/4X+hPfms9hIl/xKiXpdSquZu/OWRxvuf6f/OgzohjylrSiCFdhx1gwReexiQ4qnDJA7t0D90gOP1Z3Mw74/mYv66ZaUsnt7iMzlekeH+XrXYW0Khe2fEqGo4zxLrhFKfyYz0Gyn2et14KVV8ADz/J6XeVcQ3YGzSiXQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4She6XqMVJbSknC76Ym/pRyC9SJgnJTxSenAoRQw0t0=; b=VGJtDFUZMH8zeZI6CuPWB9wBUqqkjw2QsuSppQGsFuf8YZA0QecVQbFEhbyVoVZ0NHs6biV7h67l1W8G6zMcB5uzM1ALmzQ6o0ndwfnNjuFZHON3UUNuD/C7E8c+Gst8PhiqcYaDC+cI7drUxyKU6fVgYA1SlIbAZk9Odhki3HboeqqFtAtle5MgTEMAH5mcbGGm20Bm9n3KP56Sa0LhHX3MZoYe4sC0bOaj9Eg2x8GigJuoE8u/2426HjkCxwLkteEmvXp6vsTBv4fXe/GKnmpwREz2Mk3y6rnY7jCSDufWl9uDoM0cDgA6awJjyavg535QkV/CMe76B4mJm2wWVQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4She6XqMVJbSknC76Ym/pRyC9SJgnJTxSenAoRQw0t0=; b=cay7ymbaO6N02r/12LDcFJgC5MMsiVIIFvEfRHI8R74Na8DgkeZU08KSuuP2BSk/VeG8tg2Ug2Tc5Tg15BCxAGNfHeWKnAQon1u07dCourPip9HH+cd/LK3/AL/m0y4QyBU0JD6El26TJlj5BUbEGbQs3hNsfh/hh2pPgixsAlg= Received: from VI1PR08MB5376.eurprd08.prod.outlook.com (10.255.196.79) by VI1PR08MB3917.eurprd08.prod.outlook.com (20.178.80.92) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2538.18; Sat, 14 Dec 2019 16:18:15 +0000 Received: from VI1PR08MB5376.eurprd08.prod.outlook.com ([fe80::dde8:216f:6a0b:8cfb]) by VI1PR08MB5376.eurprd08.prod.outlook.com ([fe80::dde8:216f:6a0b:8cfb%7]) with mapi id 15.20.2538.019; Sat, 14 Dec 2019 16:18:15 +0000 From: "Gavin Hu (Arm Technology China)" To: Mahipal Challa , "dev@dpdk.org" CC: "jerinj@marvell.com" , "pathreya@marvell.com" , "snilla@marvell.com" , "venkatn@marvell.com" , nd Thread-Topic: [dpdk-dev] [PATCH v1 5/6] raw/octeontx2_ep: add dequeue operation Thread-Index: AQHVrnLRMyUVtnaV4kay/QEzQCoiQKe503vQ Date: Sat, 14 Dec 2019 16:18:15 +0000 Message-ID: References: <1575614365-8907-1-git-send-email-mchalla@marvell.com> <1575614365-8907-6-git-send-email-mchalla@marvell.com> In-Reply-To: <1575614365-8907-6-git-send-email-mchalla@marvell.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: 29748b64-fe18-425b-a135-5a79130ac373.0 x-checkrecipientchecked: true Authentication-Results-Original: spf=none (sender IP is ) smtp.mailfrom=Gavin.Hu@arm.com; x-originating-ip: [113.29.88.7] x-ms-publictraffictype: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: 38233da9-909c-4760-bb9f-08d780b13d90 X-MS-TrafficTypeDiagnostic: VI1PR08MB3917:|VI1PR0801MB2032: x-ld-processed: f34e5979-57d9-4aaa-ad4d-b122a662184d,ExtAddr X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true x-ms-oob-tlc-oobclassifiers: OLM:8882;OLM:8882; x-forefront-prvs: 025100C802 X-Forefront-Antispam-Report-Untrusted: SFV:NSPM; SFS:(10009020)(4636009)(396003)(366004)(39860400002)(346002)(136003)(376002)(13464003)(189003)(199004)(76116006)(64756008)(66946007)(66556008)(66446008)(5660300002)(66476007)(33656002)(71200400001)(966005)(186003)(478600001)(54906003)(110136005)(8676002)(55016002)(316002)(9686003)(8936002)(7696005)(81156014)(81166006)(2906002)(6506007)(53546011)(55236004)(52536014)(4326008)(26005)(86362001); DIR:OUT; SFP:1101; SCL:1; SRVR:VI1PR08MB3917; H:VI1PR08MB5376.eurprd08.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: lZdXb0Y1xOVTR2MLiq1X1ppYa7soN1gjNvipJ0zj8DBZWFXOi2jiI0sSUJOxZ5E5uvYHbFtAbk3rVySLKlg495jA4JoG1DHMbKCvcBJyLpVEDrcUeCa11qsQq2OVkQYPws4pGadlrhkofoHmaGSHvVm3GgMLrWBrc/z8rNYiGQONjc1pyBBEm0BDzuDjUT8TB5wqb/ME2Q2cUiSRQqU5eKGD0u2ukxvq8mTwp9GRFOVtbGyG/TR9EODNQst38xmR8HRGd+owEoKkjkdWm4dP/5U0e4Mevi1Myw1y45/alsBlS7IsBzmMhX2pshBEhi5GsI+J8jlTqg/Qs4qdlQ5pH/6m2RAWxeDypWo+oTSYGqa6+HezbZXV3hnB64w62Fl1ZFPwBNtJ3tBUIN0K+Tz/vh1SqvXJ/0w8CBKHIO12hpIyou8i7hBbT4JHIRMtpCOpKnxOvqEZ62V0iElRDkvPC8dNG+Bra7csuSnBJs+LVHs4ipEe3EQ3TZ8K0S+7Xg9Hgw8jBEWj7eOqqcd/4clnJW6N5anhtAdgUl8hjpNz0kc= x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB3917 Original-Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Gavin.Hu@arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: VE1EUR03FT040.eop-EUR03.prod.protection.outlook.com X-Forefront-Antispam-Report: CIP:63.35.35.123; IPV:CAL; SCL:-1; CTRY:IE; EFV:NLI; SFV:NSPM; SFS:(10009020)(4636009)(39860400002)(346002)(376002)(136003)(396003)(189003)(199004)(13464003)(8676002)(186003)(9686003)(2906002)(81166006)(81156014)(86362001)(356004)(33656002)(5660300002)(336012)(8936002)(55016002)(26005)(52536014)(53546011)(6506007)(26826003)(4326008)(7696005)(478600001)(70586007)(70206006)(76130400001)(316002)(54906003)(110136005)(966005)(36906005); DIR:OUT; SFP:1101; SCL:1; SRVR:VI1PR0801MB2032; H:64aa7808-outbound-1.mta.getcheckrecipient.com; FPR:; SPF:Pass; LANG:en; PTR:ErrorRetry; MX:1; A:1; X-MS-Office365-Filtering-Correlation-Id-Prvs: 8ddf548a-1e80-4464-68b8-08d780b13921 NoDisclaimer: True X-Forefront-PRVS: 025100C802 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: liAzjZU1cJCuMHjymhfp6XhEXG3jwejEgQq25u0CMIX3c1ql26Us32x/1Lz3cuN1oKb0WKbFi4Y5ZrwFZZxovsrsUG9saH6MIOVvfMxhoug7VvwzsuoNuGSqJSGiqixCbKApxqf9aPecNTXnruj17qnjc3kqePdF9orp86HZ7PaS0hQw7MMUU1HlHpsWZfdcPzKY5hIGiwa2+boCxVcQkCLlylnnBb0oNaP+WVEOs5I2ZIuxrJKSen810G32cHv2SJ7z4gcUdGNW15XrcPJRvb/GpFX464LCZyFTqstdwcQLDvNiicJvy945zcbosm5akb1v90if0Q1K7DgeW22e6ew9BuGkMC1GJ4fra4KLXPFH4826nNUf4Y7LF+plpYAeGaqogGgyJIIZ3xe4fPn06Ux7I/5a1rr8Qj5Yz/kT7ztFrh6pRwX3l77Kq16k/RoIzq8bWy6jAwK6bEoOtalU5db2vSIuLSKgtAb0cqKG084xiPVdvTUlYF/CvhDRvTIPP2eDKL2l5qRnBL/22E45KSQOewrnhxMazuzaL8ivP2A= X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Dec 2019 16:18:22.5315 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 38233da9-909c-4760-bb9f-08d780b13d90 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0801MB2032 Subject: Re: [dpdk-dev] [PATCH v1 5/6] raw/octeontx2_ep: add dequeue operation X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Mahipal, > -----Original Message----- > From: dev On Behalf Of Mahipal Challa > Sent: Friday, December 6, 2019 2:39 PM > To: dev@dpdk.org > Cc: jerinj@marvell.com; pathreya@marvell.com; snilla@marvell.com; > venkatn@marvell.com > Subject: [dpdk-dev] [PATCH v1 5/6] raw/octeontx2_ep: add dequeue > operation >=20 > Add rawdev dequeue operation for SDP VF devices. >=20 > Signed-off-by: Mahipal Challa > --- > drivers/raw/octeontx2_ep/otx2_ep_enqdeq.c | 199 > ++++++++++++++++++++++++++++++ > drivers/raw/octeontx2_ep/otx2_ep_enqdeq.h | 2 + > drivers/raw/octeontx2_ep/otx2_ep_rawdev.c | 1 + > drivers/raw/octeontx2_ep/otx2_ep_rawdev.h | 18 ++- > 4 files changed, 219 insertions(+), 1 deletion(-) >=20 > diff --git a/drivers/raw/octeontx2_ep/otx2_ep_enqdeq.c > b/drivers/raw/octeontx2_ep/otx2_ep_enqdeq.c > index ebbacfb..451fcc0 100644 > --- a/drivers/raw/octeontx2_ep/otx2_ep_enqdeq.c > +++ b/drivers/raw/octeontx2_ep/otx2_ep_enqdeq.c > @@ -260,6 +260,7 @@ > rte_mempool_get(sdpvf->enqdeq_mpool, &buf); > if (buf =3D=3D NULL) { > otx2_err("OQ buffer alloc failed"); > + droq->stats.rx_alloc_failure++; > /* sdp_droq_destroy_ring_buffers(droq);*/ > return -ENOMEM; > } > @@ -645,3 +646,201 @@ > return SDP_IQ_SEND_FAILED; > } >=20 > +static uint32_t > +sdp_droq_refill(struct sdp_device *sdpvf, struct sdp_droq *droq) > +{ > + struct sdp_droq_desc *desc_ring; > + uint32_t desc_refilled =3D 0; > + void *buf =3D NULL; > + > + desc_ring =3D droq->desc_ring; > + > + while (droq->refill_count && (desc_refilled < droq->nb_desc)) { > + /* If a valid buffer exists (happens if there is no dispatch), > + * reuse the buffer, else allocate. > + */ > + if (droq->recv_buf_list[droq->refill_idx].buffer !=3D NULL) > + break; > + > + rte_mempool_get(sdpvf->enqdeq_mpool, &buf); > + /* If a buffer could not be allocated, no point in > + * continuing > + */ > + if (buf =3D=3D NULL) { > + droq->stats.rx_alloc_failure++; > + break; > + } > + > + droq->recv_buf_list[droq->refill_idx].buffer =3D buf; > + desc_ring[droq->refill_idx].buffer_ptr =3D > rte_mem_virt2iova(buf); > + > + /* Reset any previous values in the length field. */ > + droq->info_list[droq->refill_idx].length =3D 0; > + > + droq->refill_idx =3D sdp_incr_index(droq->refill_idx, 1, > + droq->nb_desc); > + > + desc_refilled++; > + droq->refill_count--; > + > + } > + > + return desc_refilled; > +} > + > +static int > +sdp_droq_read_packet(struct sdp_device *sdpvf __rte_unused, > + struct sdp_droq *droq, > + struct sdp_droq_pkt *droq_pkt) > +{ > + struct sdp_droq_info *info; > + uint32_t total_len =3D 0; > + uint32_t pkt_len =3D 0; > + > + info =3D &droq->info_list[droq->read_idx]; > + sdp_swap_8B_data((uint64_t *)&info->length, 1); > + if (!info->length) { > + otx2_err("OQ info_list->length[%ld]", (long)info->length); > + goto oq_read_fail; > + } > + > + /* Deduce the actual data size */ > + info->length -=3D SDP_RH_SIZE; > + total_len +=3D (uint32_t)info->length; > + > + otx2_sdp_dbg("OQ: pkt_len[%ld], buffer_size %d", > + (long)info->length, droq->buffer_size); > + if (info->length > droq->buffer_size) { > + otx2_err("This mode is not supported: pkt_len > buffer_size"); > + goto oq_read_fail; > + } > + > + if (info->length <=3D droq->buffer_size) { > + pkt_len =3D (uint32_t)info->length; > + droq_pkt->data =3D droq->recv_buf_list[droq->read_idx].buffer; > + droq_pkt->len =3D pkt_len; > + > + droq->recv_buf_list[droq->read_idx].buffer =3D NULL; > + droq->read_idx =3D sdp_incr_index(droq->read_idx, 1,/* > count */ > + droq->nb_desc /* max rd idx > */); > + droq->refill_count++; > + > + } > + > + info->length =3D 0; > + > + return SDP_OQ_RECV_SUCCESS; > + > +oq_read_fail: > + return SDP_OQ_RECV_FAILED; > +} > + > +static inline uint32_t > +sdp_check_droq_pkts(struct sdp_droq *droq, uint32_t burst_size) > +{ > + uint32_t min_pkts =3D 0; > + uint32_t new_pkts; > + uint32_t pkt_count; > + > + /* Latest available OQ packets */ > + pkt_count =3D rte_read32(droq->pkts_sent_reg); > + > + /* Newly arrived packets */ > + new_pkts =3D pkt_count - droq->last_pkt_count; > + otx2_sdp_dbg("Recvd [%d] new OQ pkts", new_pkts); > + > + min_pkts =3D (new_pkts > burst_size) ? burst_size : new_pkts; > + if (min_pkts) { > + rte_atomic64_add(&droq->pkts_pending, min_pkts); > + /* Back up the aggregated packet count so far */ > + droq->last_pkt_count +=3D min_pkts; > + } > + > + return min_pkts; > +} > + > +/* Check for response arrival from OCTEON TX2 > + * returns number of requests completed > + */ > +int > +sdp_rawdev_dequeue(struct rte_rawdev *rawdev, > + struct rte_rawdev_buf **buffers, unsigned int count, > + rte_rawdev_obj_t context __rte_unused) > +{ > + struct sdp_droq_pkt *oq_pkt; > + struct sdp_device *sdpvf; > + struct sdp_droq *droq; > + > + uint32_t q_no =3D 0, pkts; > + uint32_t new_pkts; > + uint32_t ret; > + > + sdpvf =3D (struct sdp_device *)rawdev->dev_private; > + > + droq =3D sdpvf->droq[q_no]; > + if (!droq) { > + otx2_err("Invalid droq[%d]", q_no); > + goto deq_fail; > + } > + > + /* Grab the lock */ > + rte_spinlock_lock(&droq->lock); > + > + new_pkts =3D sdp_check_droq_pkts(droq, count); > + if (!new_pkts) { > + otx2_sdp_dbg("Zero new_pkts:%d", new_pkts); > + goto deq_fail; /* No pkts at this moment */ > + } > + > + otx2_sdp_dbg("Received new_pkts =3D %d", new_pkts); > + > + for (pkts =3D 0; pkts < new_pkts; pkts++) { > + > + /* Push the received pkt to application */ > + oq_pkt =3D (struct sdp_droq_pkt *)buffers[pkts]; > + > + ret =3D sdp_droq_read_packet(sdpvf, droq, oq_pkt); > + if (ret) { > + otx2_err("DROQ read pakt failed."); > + goto deq_fail; > + } > + > + /* Stats */ > + droq->stats.pkts_received++; > + droq->stats.bytes_received +=3D oq_pkt->len; > + } > + > + /* Ack the h/w with no# of pkts read by Host */ > + rte_wmb(); The following rte_write32 call has a built-in rte_io_wmb, which is sufficie= nt for IO device. The above barrier can be saved. > + rte_write32(pkts, droq->pkts_sent_reg); > + rte_wmb(); Rte_cio_wmb is sufficient to keep order between coherent memory domain betw= een lcore and I/O device. https://code.dpdk.org/dpdk/latest/source/lib/librte_eal/common/include/gene= ric/rte_atomic.h#L137 > + > + droq->last_pkt_count -=3D pkts; > + > + otx2_sdp_dbg("DROQ pkts[%d] pushed to application", pkts); > + > + /* Refill DROQ buffers */ > + if (droq->refill_count >=3D 2 /* droq->refill_threshold */) { > + int desc_refilled =3D sdp_droq_refill(sdpvf, droq); > + > + /* Flush the droq descriptor data to memory to be sure > + * that when we update the credits the data in memory is > + * accurate. > + */ > + rte_wmb(); Rte_cio_wmb should be the correct one, but the stronger rte_io_wmb within r= te_write32 API suffice. > + rte_write32(desc_refilled, droq->pkts_credit_reg); > + > + /* Ensure mmio write completes */ > + rte_wmb(); If the target to ensure the completeness of the above writes, this is ok, o= therwise it is overkill. > + otx2_sdp_dbg("Refilled count =3D %d", desc_refilled); > + } > + > + /* Release the spin lock */ > + rte_spinlock_unlock(&droq->lock); > + > + return pkts; > + > +deq_fail: > + rte_spinlock_unlock(&droq->lock); > + return SDP_OQ_RECV_FAILED; > +} > diff --git a/drivers/raw/octeontx2_ep/otx2_ep_enqdeq.h > b/drivers/raw/octeontx2_ep/otx2_ep_enqdeq.h > index b9b7c0b..172fdc5 100644 > --- a/drivers/raw/octeontx2_ep/otx2_ep_enqdeq.h > +++ b/drivers/raw/octeontx2_ep/otx2_ep_enqdeq.h > @@ -11,6 +11,8 @@ > #define SDP_IQ_SEND_FAILED (-1) > #define SDP_IQ_SEND_SUCCESS (0) >=20 > +#define SDP_OQ_RECV_FAILED (-1) > +#define SDP_OQ_RECV_SUCCESS (0) >=20 > static inline uint64_t > sdp_endian_swap_8B(uint64_t _d) > diff --git a/drivers/raw/octeontx2_ep/otx2_ep_rawdev.c > b/drivers/raw/octeontx2_ep/otx2_ep_rawdev.c > index 4ba8473..ddb208d 100644 > --- a/drivers/raw/octeontx2_ep/otx2_ep_rawdev.c > +++ b/drivers/raw/octeontx2_ep/otx2_ep_rawdev.c > @@ -252,6 +252,7 @@ > .dev_stop =3D sdp_rawdev_stop, > .dev_close =3D sdp_rawdev_close, > .enqueue_bufs =3D sdp_rawdev_enqueue, > + .dequeue_bufs =3D sdp_rawdev_dequeue, > }; >=20 > static int > diff --git a/drivers/raw/octeontx2_ep/otx2_ep_rawdev.h > b/drivers/raw/octeontx2_ep/otx2_ep_rawdev.h > index 8fd06fb..a77cbab 100644 > --- a/drivers/raw/octeontx2_ep/otx2_ep_rawdev.h > +++ b/drivers/raw/octeontx2_ep/otx2_ep_rawdev.h > @@ -279,6 +279,18 @@ struct sdp_recv_buffer { > }; > #define SDP_DROQ_RECVBUF_SIZE (sizeof(struct sdp_recv_buffer)) >=20 > +/* DROQ statistics. Each output queue has four stats fields. */ > +struct sdp_droq_stats { > + /* Number of packets received in this queue. */ > + uint64_t pkts_received; > + > + /* Bytes received by this queue. */ > + uint64_t bytes_received; > + > + /* Num of failures of rte_pktmbuf_alloc() */ > + uint64_t rx_alloc_failure; > +}; > + > /* Structure to define the configuration attributes for each Output queu= e. */ > struct sdp_oq_config { > /* Max number of OQs available */ > @@ -345,6 +357,9 @@ struct sdp_droq { > */ > void *pkts_sent_reg; >=20 > + /* Statistics for this DROQ. */ > + struct sdp_droq_stats stats; > + > /* DMA mapped address of the DROQ descriptor ring. */ > size_t desc_ring_dma; >=20 > @@ -476,6 +491,7 @@ struct sdp_device { >=20 > int sdp_rawdev_enqueue(struct rte_rawdev *dev, struct rte_rawdev_buf > **buffers, > unsigned int count, rte_rawdev_obj_t context); > - > +int sdp_rawdev_dequeue(struct rte_rawdev *dev, struct rte_rawdev_buf > **buffers, > + unsigned int count, rte_rawdev_obj_t context); >=20 > #endif /* _OTX2_EP_RAWDEV_H_ */ > -- > 1.8.3.1