From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 8F983A04F6; Tue, 7 Jan 2020 06:43:07 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 6A5E21D922; Tue, 7 Jan 2020 06:43:07 +0100 (CET) Received: from EUR02-HE1-obe.outbound.protection.outlook.com (mail-eopbgr10064.outbound.protection.outlook.com [40.107.1.64]) by dpdk.org (Postfix) with ESMTP id 48DEF1D922 for ; Tue, 7 Jan 2020 06:43:06 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=h1dHTsEfhEVrnmccj1ktnFQnPCuwVAOnphIfsJ+FwZg=; b=GPooZl/r93NldzCKfQvFH34RwciLtrKyRzLTgmh09Z48NuEYr3d7gvOutW2zRVOOPknV6CzfL1uNeTm1MEhGjf4U0KQ6nclWEp28TEJNgLbz8NureC64WhNTnwF7HQznS8SMeu00K2m169miPuQUZGu0HukT32c64Fg/rQ5KVZg= Received: from DB6PR0801CA0053.eurprd08.prod.outlook.com (2603:10a6:4:2b::21) by AM0PR08MB4225.eurprd08.prod.outlook.com (2603:10a6:208:137::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2602.14; Tue, 7 Jan 2020 05:43:04 +0000 Received: from VE1EUR03FT017.eop-EUR03.prod.protection.outlook.com (2a01:111:f400:7e09::201) by DB6PR0801CA0053.outlook.office365.com (2603:10a6:4:2b::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2623.8 via Frontend Transport; Tue, 7 Jan 2020 05:43:04 +0000 Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dpdk.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dpdk.org; dmarc=bestguesspass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by VE1EUR03FT017.mail.protection.outlook.com (10.152.18.90) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2602.11 via Frontend Transport; Tue, 7 Jan 2020 05:43:04 +0000 Received: ("Tessian outbound 28955e0c1ca8:v40"); Tue, 07 Jan 2020 05:43:04 +0000 X-CR-MTA-TID: 64aa7808 Received: from a476bbd98d0e.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id C8BDC539-978F-44E9-B34C-23D92B7813C9.1; Tue, 07 Jan 2020 05:42:59 +0000 Received: from EUR04-HE1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id a476bbd98d0e.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Tue, 07 Jan 2020 05:42:59 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=M0s1tb/Aiewk8HCyyyiENuHu3dyeO3F2Bg47ya4h6r3046Gv31Tht79xpsJp2D8dQnmqxVN5ZAXBqpRjQZQwIsMh92UZjiotUbtU0zt7dYujVzxyOWdaY2UKAAbMQPqt7NuGLPT5bGAzZuwlDnztR17wuf6sVAVhWtUhK36qiDIKBbwj5L87X3hjoO4dh1CeX+De+oDj7+M+hTlfUeuhF51FsUo5wkYIp3sK0kP35w3rAY2x2cCSx5XQxQybvs3iO95/N5v5aRzflrI3yDOYN4twBqggEfiyX24+pxuYVzVamziAHGzOuqz2Pp7IBSEMI0A8rI3+gbvkrr1SISeP5Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=h1dHTsEfhEVrnmccj1ktnFQnPCuwVAOnphIfsJ+FwZg=; b=hmH4mgZmcl/OoiZ33SN/sSzJaKNapflyQuwiXzZvBqsDRlcc7BfTT/2vM8J2mAhD5YqZFDXw5EmL7sLwkFYnfccPcSwuBNggzX18ty1YGOTyO2L+cGs/93rhiwxrK1aKb/WDrbgn/zPMpWEjwEiuU4K/s0hu+/LlqleGA7P6lPok9SUZdvZzwyiebi8I796obDW8gBtRL7sZn4bWTd+EFTKljwNwKItvi7Jna3BXjqoIxXJfPoO/KYcpv6SFq8PwHXPVMXMQABU/3WtDFNmHxypauhWI/sF1oWPb54pIH2Ntkfl+rnjBM0RQll3IAV86oRH/wU7VJDSxSTDiqiybLA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=h1dHTsEfhEVrnmccj1ktnFQnPCuwVAOnphIfsJ+FwZg=; b=GPooZl/r93NldzCKfQvFH34RwciLtrKyRzLTgmh09Z48NuEYr3d7gvOutW2zRVOOPknV6CzfL1uNeTm1MEhGjf4U0KQ6nclWEp28TEJNgLbz8NureC64WhNTnwF7HQznS8SMeu00K2m169miPuQUZGu0HukT32c64Fg/rQ5KVZg= Received: from VI1PR08MB5376.eurprd08.prod.outlook.com (10.255.196.79) by VI1PR08MB4061.eurprd08.prod.outlook.com (20.178.127.225) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2602.12; Tue, 7 Jan 2020 05:42:56 +0000 Received: from VI1PR08MB5376.eurprd08.prod.outlook.com ([fe80::e080:9698:5896:6837]) by VI1PR08MB5376.eurprd08.prod.outlook.com ([fe80::e080:9698:5896:6837%5]) with mapi id 15.20.2602.016; Tue, 7 Jan 2020 05:42:56 +0000 From: Gavin Hu To: Mahipal Challa , "dev@dpdk.org" CC: "jerinj@marvell.com" , "pathreya@marvell.com" , "snilla@marvell.com" , "venkatn@marvell.com" , nd Thread-Topic: [dpdk-dev] [PATCH v3 5/6] raw/octeontx2_ep: add dequeue operation Thread-Index: AQHVxInruokN9/2PLUiMt+xgJDYHa6fernoQ Date: Tue, 7 Jan 2020 05:42:55 +0000 Message-ID: References: <1578312447-17277-1-git-send-email-mchalla@marvell.com> <1578312447-17277-6-git-send-email-mchalla@marvell.com> In-Reply-To: <1578312447-17277-6-git-send-email-mchalla@marvell.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: 848f9216-63c2-4d39-bbca-5b25c5249574.0 x-checkrecipientchecked: true Authentication-Results-Original: spf=none (sender IP is ) smtp.mailfrom=Gavin.Hu@arm.com; x-originating-ip: [113.29.88.7] x-ms-publictraffictype: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: e34728da-ae5a-46c8-8dc2-08d793347726 X-MS-TrafficTypeDiagnostic: VI1PR08MB4061:|AM0PR08MB4225: x-ld-processed: f34e5979-57d9-4aaa-ad4d-b122a662184d,ExtAddr X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true x-ms-oob-tlc-oobclassifiers: OLM:7691;OLM:7691; x-forefront-prvs: 027578BB13 X-Forefront-Antispam-Report-Untrusted: SFV:NSPM; SFS:(10009020)(4636009)(136003)(376002)(346002)(366004)(396003)(39860400002)(13464003)(199004)(189003)(316002)(5660300002)(86362001)(2906002)(4326008)(478600001)(76116006)(53546011)(55236004)(81166006)(6506007)(55016002)(54906003)(186003)(26005)(110136005)(8676002)(9686003)(66476007)(81156014)(8936002)(71200400001)(52536014)(33656002)(66446008)(66556008)(7696005)(64756008)(66946007); DIR:OUT; SFP:1101; SCL:1; SRVR:VI1PR08MB4061; H:VI1PR08MB5376.eurprd08.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: KWM3HpEchZhJrxzxXc0PAV0G/q/AQxQK0NvZzaIGYmujnaU9k3EqCGonMRvCQv3fbKZ47Ay2a11Cu4hBHlVEbMJAar05lc6gdyeZ1ynYRkcDGxN2TXH4kbDteZWmcwVJ4NuYHqR6jHdjMFFQwahCFmKWeO/BbsQyVpABxRh3ju3Ys+1RfiWqPBvKSdfyJ9IVitV4lhSFfsqR7tbKGBOCH+w8KraycxobeJAdmCoPRHlBTSaokGohSiuNORL3kTfU2PawMqh5jyYDKp/WYfXdVnW5kZFRyoNiATt0+CqGc3YcTGE69Q0IIsKb7WaJV9K4jKvBQcevrYMIoicoUHA+CWitSzFHgjxuuoa6j6TYzACD1Uar9+Id28knZtmAtWrQN19/eIlRIznvdviX6xyrwqBtvtS7mt/Wkjodrp20vy7uPlNZ6I0hiaRBvpSkiSD/rlOkvjOoGOIQkHiy4yG9pvUUlVrogTqMfYejq5ialU7WrQZKSdQ9D5mtpZZLTVJf x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB4061 Original-Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Gavin.Hu@arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: VE1EUR03FT017.eop-EUR03.prod.protection.outlook.com X-Forefront-Antispam-Report: CIP:63.35.35.123; IPV:CAL; SCL:-1; CTRY:IE; EFV:NLI; SFV:NSPM; SFS:(10009020)(4636009)(376002)(346002)(396003)(136003)(39860400002)(13464003)(189003)(199004)(53546011)(6506007)(478600001)(52536014)(186003)(55016002)(7696005)(8936002)(4326008)(26005)(33656002)(9686003)(26826003)(81156014)(70586007)(8676002)(86362001)(70206006)(336012)(2906002)(81166006)(356004)(36906005)(5660300002)(110136005)(54906003)(316002); DIR:OUT; SFP:1101; SCL:1; SRVR:AM0PR08MB4225; H:64aa7808-outbound-1.mta.getcheckrecipient.com; FPR:; SPF:Pass; LANG:en; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; MX:1; A:1; X-MS-Office365-Filtering-Correlation-Id-Prvs: cc8ac0c9-e98d-4196-f4af-08d793347240 NoDisclaimer: True X-Forefront-PRVS: 027578BB13 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: AnjiWudNYs7C/Y70gZgk6A9FrbnowDtryfiwbh0lOdBwvEmCoZsGuj4kMDRfqI1bqKvmrk5mDC6L6r/8lodXCUx3tqQu9q9niiwCmfKnXPXWQ+phjo3yBR3lqoy6N9ek1gRRYbgmzIlywHLsxyYnJiLmUhiCTTqM+mQqmOiw7TVsoUX+MrdwBIWzKqdpadYHlnsI9F+Ockj4+IUk8tOkNvow0CmkVSNtF5Y1ELa6BEhan7hCJRczLDjmDphb0H5d/RotvYcd/Y3cuVtIj1q+QeLAa51obgsJjGJMYJG9GRtP7jLscs6JUiYuLepudPUpUWCg/0vnDB43rrvTm56OBwkbx8eciD6wCJI2VrHG2nu4+rRvR7+Zy1p8IHn85jqCt+kDaT1NtbAUCcG3z/MiznhXZG4KOr3ZLjXUR6pJ7oLsMuiOjeTYsG2JMv8eEFtvdBsgHgf2PQAIqKwkDM+9R6VdELe6Pvj4IPb1qjEt0VZ9Mr6yXyB+zSdmOukddkoc X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Jan 2020 05:43:04.1351 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: e34728da-ae5a-46c8-8dc2-08d793347726 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB4225 Subject: Re: [dpdk-dev] [PATCH v3 5/6] raw/octeontx2_ep: add dequeue operation X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Mahipal, Jerin, > -----Original Message----- > From: Mahipal Challa > Sent: Monday, January 6, 2020 8:07 PM > To: dev@dpdk.org > Cc: jerinj@marvell.com; pathreya@marvell.com; snilla@marvell.com; > venkatn@marvell.com; Gavin Hu > Subject: [dpdk-dev] [PATCH v3 5/6] raw/octeontx2_ep: add dequeue > operation >=20 > Add rawdev dequeue operation for SDP VF devices. >=20 > Signed-off-by: Mahipal Challa > --- > drivers/raw/octeontx2_ep/otx2_ep_enqdeq.c | 197 > ++++++++++++++++++++++++++++++ > drivers/raw/octeontx2_ep/otx2_ep_enqdeq.h | 2 + > drivers/raw/octeontx2_ep/otx2_ep_rawdev.c | 1 + > drivers/raw/octeontx2_ep/otx2_ep_rawdev.h | 18 ++- > 4 files changed, 217 insertions(+), 1 deletion(-) >=20 > diff --git a/drivers/raw/octeontx2_ep/otx2_ep_enqdeq.c > b/drivers/raw/octeontx2_ep/otx2_ep_enqdeq.c > index 6910f08..dd270b5 100644 > --- a/drivers/raw/octeontx2_ep/otx2_ep_enqdeq.c > +++ b/drivers/raw/octeontx2_ep/otx2_ep_enqdeq.c > @@ -260,6 +260,7 @@ > rte_mempool_get(sdpvf->enqdeq_mpool, &buf); > if (buf =3D=3D NULL) { > otx2_err("OQ buffer alloc failed"); > + droq->stats.rx_alloc_failure++; > /* sdp_droq_destroy_ring_buffers(droq);*/ > return -ENOMEM; > } > @@ -645,3 +646,199 @@ > return SDP_IQ_SEND_FAILED; > } >=20 > +static uint32_t > +sdp_droq_refill(struct sdp_device *sdpvf, struct sdp_droq *droq) > +{ > + struct sdp_droq_desc *desc_ring; > + uint32_t desc_refilled =3D 0; > + void *buf =3D NULL; > + > + desc_ring =3D droq->desc_ring; > + > + while (droq->refill_count && (desc_refilled < droq->nb_desc)) { > + /* If a valid buffer exists (happens if there is no dispatch), > + * reuse the buffer, else allocate. > + */ > + if (droq->recv_buf_list[droq->refill_idx].buffer !=3D NULL) > + break; > + > + rte_mempool_get(sdpvf->enqdeq_mpool, &buf); > + /* If a buffer could not be allocated, no point in > + * continuing > + */ > + if (buf =3D=3D NULL) { > + droq->stats.rx_alloc_failure++; > + break; > + } > + > + droq->recv_buf_list[droq->refill_idx].buffer =3D buf; > + desc_ring[droq->refill_idx].buffer_ptr =3D > rte_mem_virt2iova(buf); > + > + /* Reset any previous values in the length field. */ > + droq->info_list[droq->refill_idx].length =3D 0; > + > + droq->refill_idx =3D sdp_incr_index(droq->refill_idx, 1, > + droq->nb_desc); > + > + desc_refilled++; > + droq->refill_count--; > + > + } > + > + return desc_refilled; > +} > + > +static int > +sdp_droq_read_packet(struct sdp_device *sdpvf __rte_unused, > + struct sdp_droq *droq, > + struct sdp_droq_pkt *droq_pkt) > +{ > + struct sdp_droq_info *info; > + uint32_t total_len =3D 0; > + uint32_t pkt_len =3D 0; > + > + info =3D &droq->info_list[droq->read_idx]; > + sdp_swap_8B_data((uint64_t *)&info->length, 1); > + if (!info->length) { > + otx2_err("OQ info_list->length[%ld]", (long)info->length); > + goto oq_read_fail; > + } > + > + /* Deduce the actual data size */ > + info->length -=3D SDP_RH_SIZE; > + total_len +=3D (uint32_t)info->length; > + > + otx2_sdp_dbg("OQ: pkt_len[%ld], buffer_size %d", > + (long)info->length, droq->buffer_size); > + if (info->length > droq->buffer_size) { > + otx2_err("This mode is not supported: pkt_len > > buffer_size"); > + goto oq_read_fail; > + } > + > + if (info->length <=3D droq->buffer_size) { > + pkt_len =3D (uint32_t)info->length; > + droq_pkt->data =3D droq->recv_buf_list[droq- > >read_idx].buffer; > + droq_pkt->len =3D pkt_len; > + > + droq->recv_buf_list[droq->read_idx].buffer =3D NULL; > + droq->read_idx =3D sdp_incr_index(droq->read_idx, 1,/* > count */ > + droq->nb_desc /* max rd > idx */); > + droq->refill_count++; > + > + } > + > + info->length =3D 0; > + > + return SDP_OQ_RECV_SUCCESS; > + > +oq_read_fail: > + return SDP_OQ_RECV_FAILED; > +} > + > +static inline uint32_t > +sdp_check_droq_pkts(struct sdp_droq *droq, uint32_t burst_size) > +{ > + uint32_t min_pkts =3D 0; > + uint32_t new_pkts; > + uint32_t pkt_count; > + > + /* Latest available OQ packets */ > + pkt_count =3D rte_read32(droq->pkts_sent_reg); > + > + /* Newly arrived packets */ > + new_pkts =3D pkt_count - droq->last_pkt_count; > + otx2_sdp_dbg("Recvd [%d] new OQ pkts", new_pkts); > + > + min_pkts =3D (new_pkts > burst_size) ? burst_size : new_pkts; > + if (min_pkts) { > + rte_atomic64_add(&droq->pkts_pending, min_pkts); > + /* Back up the aggregated packet count so far */ > + droq->last_pkt_count +=3D min_pkts; > + } > + > + return min_pkts; > +} > + > +/* Check for response arrival from OCTEON TX2 > + * returns number of requests completed > + */ > +int > +sdp_rawdev_dequeue(struct rte_rawdev *rawdev, > + struct rte_rawdev_buf **buffers, unsigned int count, > + rte_rawdev_obj_t context __rte_unused) > +{ > + struct sdp_droq_pkt *oq_pkt; > + struct sdp_device *sdpvf; > + struct sdp_droq *droq; > + > + uint32_t q_no =3D 0, pkts; > + uint32_t new_pkts; > + uint32_t ret; > + > + sdpvf =3D (struct sdp_device *)rawdev->dev_private; > + > + droq =3D sdpvf->droq[q_no]; > + if (!droq) { > + otx2_err("Invalid droq[%d]", q_no); > + goto deq_fail; > + } > + > + /* Grab the lock */ > + rte_spinlock_lock(&droq->lock); > + > + new_pkts =3D sdp_check_droq_pkts(droq, count); > + if (!new_pkts) { > + otx2_sdp_dbg("Zero new_pkts:%d", new_pkts); > + goto deq_fail; /* No pkts at this moment */ > + } > + > + otx2_sdp_dbg("Received new_pkts =3D %d", new_pkts); > + > + for (pkts =3D 0; pkts < new_pkts; pkts++) { > + > + /* Push the received pkt to application */ > + oq_pkt =3D (struct sdp_droq_pkt *)buffers[pkts]; > + > + ret =3D sdp_droq_read_packet(sdpvf, droq, oq_pkt); > + if (ret) { > + otx2_err("DROQ read pakt failed."); > + goto deq_fail; > + } > + > + /* Stats */ > + droq->stats.pkts_received++; > + droq->stats.bytes_received +=3D oq_pkt->len; > + } > + > + /* Ack the h/w with no# of pkts read by Host */ > + rte_write32(pkts, droq->pkts_sent_reg); Rte_write32 has a built-in rte_io_wmb, aka "dsb" for aarch64, inside, this = hurts performance, this is on the fast data path?=20 > + rte_cio_wmb(); This barrier should be hoisted up to between the dequeue operation and regi= ster write to keep the ordering of two. I am surprised why rte_cio_mb is not defined within DPDK, it is required he= re to keep the read/write ordering, neither rmb nor wmb can fulfill the ord= ering required. > + > + droq->last_pkt_count -=3D pkts; > + > + otx2_sdp_dbg("DROQ pkts[%d] pushed to application", pkts); > + > + /* Refill DROQ buffers */ > + if (droq->refill_count >=3D 2 /* droq->refill_threshold */) { > + int desc_refilled =3D sdp_droq_refill(sdpvf, droq); > + > + /* Flush the droq descriptor data to memory to be sure > + * that when we update the credits the data in memory is > + * accurate. > + */ > + rte_write32(desc_refilled, droq->pkts_credit_reg); Rte_cio_wmb should be used instead of the built-in rte_io_wmb to improve th= e performance. But if performance is not a concern here, rte_io_wmb rescued the situation.= =20 > + > + /* Ensure mmio write completes */ > + rte_wmb(); > + otx2_sdp_dbg("Refilled count =3D %d", desc_refilled); > + } > + > + /* Release the spin lock */ > + rte_spinlock_unlock(&droq->lock); > + > + return pkts; > + > +deq_fail: > + rte_spinlock_unlock(&droq->lock); > + return SDP_OQ_RECV_FAILED; > +} > diff --git a/drivers/raw/octeontx2_ep/otx2_ep_enqdeq.h > b/drivers/raw/octeontx2_ep/otx2_ep_enqdeq.h > index b9b7c0b..172fdc5 100644 > --- a/drivers/raw/octeontx2_ep/otx2_ep_enqdeq.h > +++ b/drivers/raw/octeontx2_ep/otx2_ep_enqdeq.h > @@ -11,6 +11,8 @@ > #define SDP_IQ_SEND_FAILED (-1) > #define SDP_IQ_SEND_SUCCESS (0) >=20 > +#define SDP_OQ_RECV_FAILED (-1) > +#define SDP_OQ_RECV_SUCCESS (0) >=20 > static inline uint64_t > sdp_endian_swap_8B(uint64_t _d) > diff --git a/drivers/raw/octeontx2_ep/otx2_ep_rawdev.c > b/drivers/raw/octeontx2_ep/otx2_ep_rawdev.c > index 22a6beb..7158b97 100644 > --- a/drivers/raw/octeontx2_ep/otx2_ep_rawdev.c > +++ b/drivers/raw/octeontx2_ep/otx2_ep_rawdev.c > @@ -252,6 +252,7 @@ > .dev_stop =3D sdp_rawdev_stop, > .dev_close =3D sdp_rawdev_close, > .enqueue_bufs =3D sdp_rawdev_enqueue, > + .dequeue_bufs =3D sdp_rawdev_dequeue, > }; >=20 > static int > diff --git a/drivers/raw/octeontx2_ep/otx2_ep_rawdev.h > b/drivers/raw/octeontx2_ep/otx2_ep_rawdev.h > index 8fd06fb..a77cbab 100644 > --- a/drivers/raw/octeontx2_ep/otx2_ep_rawdev.h > +++ b/drivers/raw/octeontx2_ep/otx2_ep_rawdev.h > @@ -279,6 +279,18 @@ struct sdp_recv_buffer { > }; > #define SDP_DROQ_RECVBUF_SIZE (sizeof(struct sdp_recv_buffer)) >=20 > +/* DROQ statistics. Each output queue has four stats fields. */ > +struct sdp_droq_stats { > + /* Number of packets received in this queue. */ > + uint64_t pkts_received; > + > + /* Bytes received by this queue. */ > + uint64_t bytes_received; > + > + /* Num of failures of rte_pktmbuf_alloc() */ > + uint64_t rx_alloc_failure; > +}; > + > /* Structure to define the configuration attributes for each Output queu= e. > */ > struct sdp_oq_config { > /* Max number of OQs available */ > @@ -345,6 +357,9 @@ struct sdp_droq { > */ > void *pkts_sent_reg; >=20 > + /* Statistics for this DROQ. */ > + struct sdp_droq_stats stats; > + > /* DMA mapped address of the DROQ descriptor ring. */ > size_t desc_ring_dma; >=20 > @@ -476,6 +491,7 @@ struct sdp_device { >=20 > int sdp_rawdev_enqueue(struct rte_rawdev *dev, struct rte_rawdev_buf > **buffers, > unsigned int count, rte_rawdev_obj_t context); > - > +int sdp_rawdev_dequeue(struct rte_rawdev *dev, struct rte_rawdev_buf > **buffers, > + unsigned int count, rte_rawdev_obj_t context); >=20 > #endif /* _OTX2_EP_RAWDEV_H_ */ > -- > 1.8.3.1